2022-08-01T19:17:25.546 INFO:root:teuthology version: 1.1.1.dev401+gdd7eb9d9 2022-08-01T19:17:25.585 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_yuriw@teuthology --description fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} --archive /home/teuthworker/archive/yuriw-2022-08-01_19:07:06-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954722 --name yuriw-2022-08-01_19:07:06-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi --verbose -- /home/teuthworker/archive/yuriw-2022-08-01_19:07:06-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954722/orig.config.yaml 2022-08-01T19:17:26.612 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2022-08-01T19:17:27.492 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2022-08-01_19:07:06-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954722 branch: wip-yuri6-testing-2022-08-01-0839-octopus description: fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} email: ceph-qa@ceph.io first_in_suite: false job_id: '6954722' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2022-08-01_19:07:06-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 30 - machine: disk: 200 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-yuri6-testing-2022-08-01-0839-octopus ceph: cephfs_ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 15m rados osd op timeout: 15m mds: debug mds: 20 debug ms: 1 mds bal frag: true mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 15m rados osd op timeout: 15m mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - evicting unresponsive client - POOL_APP_NOT_ENABLED - has not responded to cap revoke by MDS for over - MDS_CLIENT_LATE_RELEASE - responding to mclientcaps sha1: fd96719b0e1200b56e7430604cfdd20ad281ca92 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: flavor: default sha1: fd96719b0e1200b56e7430604cfdd20ad281ca92 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri6-testing-2022-08-01-0839-octopus sha1: fd96719b0e1200b56e7430604cfdd20ad281ca92 owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - - mon.b - mon.c - mgr.x - mds.b - osd.4 - osd.5 - osd.6 - osd.7 - - client.0 - - client.1 sha1: fd96719b0e1200b56e7430604cfdd20ad281ca92 sleep_before_teardown: 0 suite: fs suite_branch: wip-yuri6-testing-2022-08-01-0839-octopus suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: fd96719b0e1200b56e7430604cfdd20ad281ca92 targets: smithi017.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPeRXsoKPmWQ8X7x/I7uJftBgiYaankGpy8Dg2PO6PU288gAAJ4SAkxprQ3QbDL/5C5EXc0KaD6bgfLhFwW+k+ykqS12L1b0csNCfG/s2rM4KKgbOUvdgd7fDxegVfqf5k1J3IXVPSdhrpKKIltEOpLSK5UPxtzzkJ4FIoDkg1zl1FfOZKY2m/K9dEia/62RBVCxINHGkG5GhmOZ83yfczc6NySZ4bglCG+XItbTCpTZM8r8PVgzAfJbhEp6xEUaYQEFyJJcRSD3LZ7nmw+uf7Kk/8w8sh3PusewrE2ccTas6vMxzb7C1Q3Bi1UjJ1aPHpQQfLyS4ipl/NWRflFsLKUsS0J0ZgQJLYkISMuFqnT7yBhvZsWf6aOSa6k/elON9N7+ROzDEHOGpl29hnBPl+nZ2yOXeFrNKaHOyNrV1uYulz8gu0T+0HdG0M4K4naI9VRLG61yGBebKhsp0/4Tx4oteaLwU/eqwnYPwk9irrSTRT4zEV8ivCg+nvAJixwBU= smithi077.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4Txugz93Uts6pxjrZYYwnPLHu33ISUnThQKsd5nl5flpx+Gj08VAkEaHNmHpwpUIGPefJ+GKX1mvXYojuXJmSO2fx7v3SMeQJOtm6dFUUp3LfCWOZGXTyaPY9CgPuDnAzRzQTon++FE3y2R0P2ahC1cZhV/lQiYoSjYpUlLFa1cTYsFvzAbtx7OGh61DYaWynW6xs0VHgfrwGvRutvNf+bOtN+cElouJzI558agEY1zCFbGg3ChC6u1KLUdpMghBuOZj8CdFY4QMosySHmuxe1tySU2bxXLIQwPMiGKpPo+BJKbJO+KMsZFnbIfAcfPAtAHYA8o1HVMsGy1TelDpOIt/XulsLFZDEkp5UfPG6GuECI3i2QJLMZVA+uDzwmuigaEIHSc4cQyVFux3cQiFxXQMuUAqVyNNOyZziuTwlh+GDHHxdDsZ8H8c7gnjPbULaR3fe0wtuZrnLZWq1apfMzZmd9cYkxCu6Wf73hBzyTlfe4ZQK8PWQlms0hyszR68= smithi085.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3+mH3hvP3b0A7FmAp52QaUqw3u6EuprX2mJzBzCH0gKYHLjkqY9lRBoRQ2eb+m8jTk3IX1ILDd4LX2y+CIj7496LOE1V30t6xJxoyfAYqPM+O5o2oNWuYw812+bMAs94G3eqA17YntXan78oo7ryRHxkEvInwhkL3LLpFqDvuP3L9AZ397w7dZ3KE4Xg1rw4TqM3UNnSYvB9UE9cdzIugfcyjteUmEMACvTxKQVmq5MSuHVU3hpH8aWhaw2Hwn1ydFHYzcvo5WrXH2OxuFTHRmA/escL4PY3r9LuJv5hzXAPGn+qJAFn+WBaKAMFET31027QP4eQXbJ15nwOe54frNtZWu0SiIoRPC9f7kRbiNt+YRiHgRL9EtAFAAqbzjzVWdlShN1kWUepWUQ6sgwl9DSQRFWs+CwH9paAqqJs3z0JQ5pRLA/QrnItd3EtWheio8MdyENHicdKo9C2uHMk+2oQiA2NYt6B9TFSDxrqtlBz+hoZw+McJbUPYCVbSYu8= smithi190.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrnkVZHWYRdJqp2GXyKlktsr65gcUBpEGDffjFSYoNjWmM7JFruIZra9PvE5sa6BrDLGCpWwcZBCz+u0+elFMFp6tAfEq+l++9rajWlg31dsRkpg5CDSJDC4npe+P8Y90WJqOLSY7qwMWUEVJJBr1SdnAO94kiP7B7urFPbeBfD/e33YKYTzPxVe+Zp2BbTqYrfccIRY2r0SuEoNEZ/wTohwy2RkzwpBfr/O1i5pUL5dqQFLJShnO1YBOM+l2OYwOH38Bmsu4BQdyMye9x0DN3YWr5jXZFxikVOHVNdd/0XvS6gBwUrh7Q7WefiVSGrHSVROCfCI6aA4vt4xvflm4dqkg8so/xmmL+77Mm3fzFvESQGUmswMhF0luoH9onLY0pfidKlJDu/BsNjCjVI7kRWSPfK13Kgh1HH7OUTWfpAgrsjbpx2i/+CvPUcKBUMQC1OAJf8JK8UWx5aWKmdsZK/hzkOzn6tyyBxwLCu8vqhwWmueNTsRtWn91Aq/vt/DU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell rpm: - python3-cephfs extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_misc teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: dd7eb9d989259767783fb5ee9a761cd0732b84e6 timestamp: 2022-08-01_19:07:06 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.1713 2022-08-01T19:17:27.493 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa; will attempt to use it 2022-08-01T19:17:27.547 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks 2022-08-01T19:17:27.548 INFO:teuthology.run_tasks:Running task internal.check_packages... 2022-08-01T19:17:27.575 INFO:teuthology.task.internal:Checking packages... 2022-08-01T19:17:28.269 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'fd96719b0e1200b56e7430604cfdd20ad281ca92' 2022-08-01T19:17:28.285 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-08-01T19:17:28.286 INFO:teuthology.packaging:ref: None 2022-08-01T19:17:28.287 INFO:teuthology.packaging:tag: None 2022-08-01T19:17:28.287 INFO:teuthology.packaging:branch: wip-yuri6-testing-2022-08-01-0839-octopus 2022-08-01T19:17:28.288 INFO:teuthology.packaging:sha1: fd96719b0e1200b56e7430604cfdd20ad281ca92 2022-08-01T19:17:28.289 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-yuri6-testing-2022-08-01-0839-octopus 2022-08-01T19:17:29.170 INFO:teuthology.task.internal:Found packages for ceph version 15.2.16-343-gfd96719b-1focal 2022-08-01T19:17:29.171 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2022-08-01T19:17:29.269 INFO:teuthology.task.internal:no buildpackages task found 2022-08-01T19:17:29.294 INFO:teuthology.run_tasks:Running task internal.save_config... 2022-08-01T19:17:29.379 INFO:teuthology.task.internal:Saving configuration 2022-08-01T19:17:30.201 INFO:teuthology.run_tasks:Running task internal.check_lock... 2022-08-01T19:17:30.288 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-08-01T19:17:30.515 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi017.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-08-01_19:07:06-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954722', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-08-01 19:08:37.098870', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIB/wVyr1dSw8NWTRAroojehLbqUGBrEmBiHEPXDShpIK'} 2022-08-01T19:17:30.664 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi077.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-08-01_19:07:06-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954722', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-08-01 19:08:37.097692', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAII+aCKNecQoRLXZLciri+Apw0T1S85zcYTRHwFXiZdMk'} 2022-08-01T19:17:30.868 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi085.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-08-01_19:07:06-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954722', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-08-01 19:08:37.096504', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILgnYVozRyl4KyQKQkMYk9dgrDX6CK6/JBnlnTcYLXuQ'} 2022-08-01T19:17:31.028 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-08-01_19:07:06-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954722', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-08-01 19:08:37.094765', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIKyQN3CoJQdFEKrMmU6sRqaHnVe/AJ10FXbDlrascPa'} 2022-08-01T19:17:31.028 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2022-08-01T19:17:31.048 INFO:teuthology.task.internal:roles: ubuntu@smithi017.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2022-08-01T19:17:31.049 INFO:teuthology.task.internal:roles: ubuntu@smithi077.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2022-08-01T19:17:31.049 INFO:teuthology.task.internal:roles: ubuntu@smithi085.front.sepia.ceph.com - ['client.0'] 2022-08-01T19:17:31.128 INFO:teuthology.task.internal:roles: ubuntu@smithi190.front.sepia.ceph.com - ['client.1'] 2022-08-01T19:17:31.152 INFO:teuthology.run_tasks:Running task console_log... 2022-08-01T19:17:33.037 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f274c330bf8>, signals=[15]) 2022-08-01T19:17:33.044 INFO:teuthology.run_tasks:Running task internal.connect... 2022-08-01T19:17:33.093 INFO:teuthology.task.internal:Opening connections... 2022-08-01T19:17:33.093 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi017.front.sepia.ceph.com 2022-08-01T19:17:33.110 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-08-01T19:17:35.260 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi077.front.sepia.ceph.com 2022-08-01T19:17:35.261 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-08-01T19:17:36.132 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi085.front.sepia.ceph.com 2022-08-01T19:17:36.181 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi085.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-08-01T19:17:37.134 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi190.front.sepia.ceph.com 2022-08-01T19:17:37.135 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-08-01T19:17:38.182 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2022-08-01T19:17:38.295 DEBUG:teuthology.orchestra.run.smithi017:> uname -m 2022-08-01T19:17:38.420 INFO:teuthology.orchestra.run.smithi017.stdout:x86_64 2022-08-01T19:17:38.421 DEBUG:teuthology.orchestra.run.smithi017:> cat /etc/os-release 2022-08-01T19:17:38.491 INFO:teuthology.orchestra.run.smithi017.stdout:NAME="Ubuntu" 2022-08-01T19:17:38.492 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-08-01T19:17:38.492 INFO:teuthology.orchestra.run.smithi017.stdout:ID=ubuntu 2022-08-01T19:17:38.493 INFO:teuthology.orchestra.run.smithi017.stdout:ID_LIKE=debian 2022-08-01T19:17:38.493 INFO:teuthology.orchestra.run.smithi017.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-08-01T19:17:38.538 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION_ID="20.04" 2022-08-01T19:17:38.538 INFO:teuthology.orchestra.run.smithi017.stdout:HOME_URL="https://www.ubuntu.com/" 2022-08-01T19:17:38.539 INFO:teuthology.orchestra.run.smithi017.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-08-01T19:17:38.539 INFO:teuthology.orchestra.run.smithi017.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-08-01T19:17:38.540 INFO:teuthology.orchestra.run.smithi017.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-08-01T19:17:38.540 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION_CODENAME=focal 2022-08-01T19:17:38.540 INFO:teuthology.orchestra.run.smithi017.stdout:UBUNTU_CODENAME=focal 2022-08-01T19:17:38.581 INFO:teuthology.lock.ops:Updating smithi017.front.sepia.ceph.com on lock server 2022-08-01T19:17:38.704 DEBUG:teuthology.orchestra.run.smithi077:> uname -m 2022-08-01T19:17:38.781 INFO:teuthology.orchestra.run.smithi077.stdout:x86_64 2022-08-01T19:17:38.819 DEBUG:teuthology.orchestra.run.smithi077:> cat /etc/os-release 2022-08-01T19:17:38.903 INFO:teuthology.orchestra.run.smithi077.stdout:NAME="Ubuntu" 2022-08-01T19:17:38.903 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-08-01T19:17:38.904 INFO:teuthology.orchestra.run.smithi077.stdout:ID=ubuntu 2022-08-01T19:17:38.904 INFO:teuthology.orchestra.run.smithi077.stdout:ID_LIKE=debian 2022-08-01T19:17:38.904 INFO:teuthology.orchestra.run.smithi077.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-08-01T19:17:38.905 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION_ID="20.04" 2022-08-01T19:17:38.962 INFO:teuthology.orchestra.run.smithi077.stdout:HOME_URL="https://www.ubuntu.com/" 2022-08-01T19:17:38.962 INFO:teuthology.orchestra.run.smithi077.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-08-01T19:17:38.963 INFO:teuthology.orchestra.run.smithi077.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-08-01T19:17:38.976 INFO:teuthology.orchestra.run.smithi077.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-08-01T19:17:38.977 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION_CODENAME=focal 2022-08-01T19:17:38.977 INFO:teuthology.orchestra.run.smithi077.stdout:UBUNTU_CODENAME=focal 2022-08-01T19:17:38.998 INFO:teuthology.lock.ops:Updating smithi077.front.sepia.ceph.com on lock server 2022-08-01T19:17:39.107 DEBUG:teuthology.orchestra.run.smithi085:> uname -m 2022-08-01T19:17:39.234 INFO:teuthology.orchestra.run.smithi085.stdout:x86_64 2022-08-01T19:17:39.235 DEBUG:teuthology.orchestra.run.smithi085:> cat /etc/os-release 2022-08-01T19:17:39.330 INFO:teuthology.orchestra.run.smithi085.stdout:NAME="Ubuntu" 2022-08-01T19:17:39.335 INFO:teuthology.orchestra.run.smithi085.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-08-01T19:17:39.336 INFO:teuthology.orchestra.run.smithi085.stdout:ID=ubuntu 2022-08-01T19:17:39.354 INFO:teuthology.orchestra.run.smithi085.stdout:ID_LIKE=debian 2022-08-01T19:17:39.354 INFO:teuthology.orchestra.run.smithi085.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-08-01T19:17:39.354 INFO:teuthology.orchestra.run.smithi085.stdout:VERSION_ID="20.04" 2022-08-01T19:17:39.377 INFO:teuthology.orchestra.run.smithi085.stdout:HOME_URL="https://www.ubuntu.com/" 2022-08-01T19:17:39.377 INFO:teuthology.orchestra.run.smithi085.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-08-01T19:17:39.386 INFO:teuthology.orchestra.run.smithi085.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-08-01T19:17:39.387 INFO:teuthology.orchestra.run.smithi085.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-08-01T19:17:39.387 INFO:teuthology.orchestra.run.smithi085.stdout:VERSION_CODENAME=focal 2022-08-01T19:17:39.387 INFO:teuthology.orchestra.run.smithi085.stdout:UBUNTU_CODENAME=focal 2022-08-01T19:17:39.388 INFO:teuthology.lock.ops:Updating smithi085.front.sepia.ceph.com on lock server 2022-08-01T19:17:39.469 DEBUG:teuthology.orchestra.run.smithi190:> uname -m 2022-08-01T19:17:39.509 INFO:teuthology.orchestra.run.smithi190.stdout:x86_64 2022-08-01T19:17:39.530 DEBUG:teuthology.orchestra.run.smithi190:> cat /etc/os-release 2022-08-01T19:17:39.575 INFO:teuthology.orchestra.run.smithi190.stdout:NAME="Ubuntu" 2022-08-01T19:17:39.576 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-08-01T19:17:39.576 INFO:teuthology.orchestra.run.smithi190.stdout:ID=ubuntu 2022-08-01T19:17:39.576 INFO:teuthology.orchestra.run.smithi190.stdout:ID_LIKE=debian 2022-08-01T19:17:39.577 INFO:teuthology.orchestra.run.smithi190.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-08-01T19:17:39.577 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_ID="20.04" 2022-08-01T19:17:39.577 INFO:teuthology.orchestra.run.smithi190.stdout:HOME_URL="https://www.ubuntu.com/" 2022-08-01T19:17:39.577 INFO:teuthology.orchestra.run.smithi190.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-08-01T19:17:39.577 INFO:teuthology.orchestra.run.smithi190.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-08-01T19:17:39.596 INFO:teuthology.orchestra.run.smithi190.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-08-01T19:17:39.596 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_CODENAME=focal 2022-08-01T19:17:39.596 INFO:teuthology.orchestra.run.smithi190.stdout:UBUNTU_CODENAME=focal 2022-08-01T19:17:39.597 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server 2022-08-01T19:17:39.669 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2022-08-01T19:17:39.722 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2022-08-01T19:17:39.791 INFO:teuthology.task.internal:Checking for old test directory... 2022-08-01T19:17:39.792 DEBUG:teuthology.orchestra.run.smithi017:> test '!' -e /home/ubuntu/cephtest 2022-08-01T19:17:39.820 DEBUG:teuthology.orchestra.run.smithi077:> test '!' -e /home/ubuntu/cephtest 2022-08-01T19:17:39.864 DEBUG:teuthology.orchestra.run.smithi085:> test '!' -e /home/ubuntu/cephtest 2022-08-01T19:17:39.901 DEBUG:teuthology.orchestra.run.smithi190:> test '!' -e /home/ubuntu/cephtest 2022-08-01T19:17:39.937 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2022-08-01T19:17:39.949 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2022-08-01T19:17:40.010 DEBUG:teuthology.orchestra.run.smithi017:> test -z $(ls -A /var/lib/ceph) 2022-08-01T19:17:40.018 DEBUG:teuthology.orchestra.run.smithi077:> test -z $(ls -A /var/lib/ceph) 2022-08-01T19:17:40.077 DEBUG:teuthology.orchestra.run.smithi085:> test -z $(ls -A /var/lib/ceph) 2022-08-01T19:17:40.105 DEBUG:teuthology.orchestra.run.smithi190:> test -z $(ls -A /var/lib/ceph) 2022-08-01T19:17:40.304 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2022-08-01T19:17:40.742 INFO:teuthology.run_tasks:Running task kernel... 2022-08-01T19:17:41.009 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2022-08-01T19:17:41.026 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}, 'client.1': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2022-08-01T19:17:41.028 DEBUG:teuthology.orchestra.run.smithi017:> test -f /run/.containerenv -o -f /.dockerenv 2022-08-01T19:17:41.029 DEBUG:teuthology.orchestra.run.smithi077:> test -f /run/.containerenv -o -f /.dockerenv 2022-08-01T19:17:41.073 DEBUG:teuthology.orchestra.run.smithi085:> test -f /run/.containerenv -o -f /.dockerenv 2022-08-01T19:17:41.074 DEBUG:teuthology.orchestra.run.smithi190:> test -f /run/.containerenv -o -f /.dockerenv 2022-08-01T19:17:41.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:17:41.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:17:41.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:17:41.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:17:41.340 DEBUG:teuthology.orchestra.run.smithi190:> uname -r 2022-08-01T19:17:41.410 DEBUG:teuthology.orchestra.run.smithi017:> uname -r 2022-08-01T19:17:41.412 DEBUG:teuthology.orchestra.run.smithi077:> uname -r 2022-08-01T19:17:41.413 DEBUG:teuthology.orchestra.run.smithi085:> uname -r 2022-08-01T19:17:41.657 INFO:teuthology.orchestra.run.smithi077.stdout:5.4.0-122-generic 2022-08-01T19:17:41.687 INFO:teuthology.orchestra.run.smithi017.stdout:5.4.0-122-generic 2022-08-01T19:17:41.688 INFO:teuthology.task.kernel:Running kernel on smithi077: 5.4.0-122-generic 2022-08-01T19:17:41.688 DEBUG:teuthology.orchestra.run.smithi077:> sudo apt-get clean 2022-08-01T19:17:41.738 INFO:teuthology.task.kernel:Running kernel on smithi017: 5.4.0-122-generic 2022-08-01T19:17:41.738 DEBUG:teuthology.orchestra.run.smithi017:> sudo apt-get clean 2022-08-01T19:17:41.799 INFO:teuthology.orchestra.run.smithi085.stdout:5.4.0-122-generic 2022-08-01T19:17:41.800 INFO:teuthology.orchestra.run.smithi190.stdout:5.4.0-122-generic 2022-08-01T19:17:41.800 INFO:teuthology.task.kernel:Running kernel on smithi085: 5.4.0-122-generic 2022-08-01T19:17:41.801 DEBUG:teuthology.orchestra.run.smithi085:> sudo apt-get clean 2022-08-01T19:17:41.801 INFO:teuthology.task.kernel:Running kernel on smithi190: 5.4.0-122-generic 2022-08-01T19:17:41.817 DEBUG:teuthology.orchestra.run.smithi190:> sudo apt-get clean 2022-08-01T19:17:41.883 DEBUG:teuthology.orchestra.run.smithi077:> sudo apt-get update 2022-08-01T19:17:41.960 DEBUG:teuthology.orchestra.run.smithi017:> sudo apt-get update 2022-08-01T19:17:42.091 DEBUG:teuthology.orchestra.run.smithi190:> sudo apt-get update 2022-08-01T19:17:42.095 DEBUG:teuthology.orchestra.run.smithi085:> sudo apt-get update 2022-08-01T19:17:42.097 INFO:teuthology.orchestra.run.smithi077.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-08-01T19:17:42.130 INFO:teuthology.orchestra.run.smithi077.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2022-08-01T19:17:42.180 INFO:teuthology.orchestra.run.smithi077.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2022-08-01T19:17:42.181 INFO:teuthology.orchestra.run.smithi017.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2022-08-01T19:17:42.209 INFO:teuthology.orchestra.run.smithi077.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2022-08-01T19:17:42.279 INFO:teuthology.orchestra.run.smithi017.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2022-08-01T19:17:42.374 INFO:teuthology.orchestra.run.smithi017.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2022-08-01T19:17:42.395 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-08-01T19:17:42.395 INFO:teuthology.orchestra.run.smithi190.stdout:Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2022-08-01T19:17:42.437 INFO:teuthology.orchestra.run.smithi190.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2022-08-01T19:17:42.462 INFO:teuthology.orchestra.run.smithi077.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [699 kB] 2022-08-01T19:17:42.478 INFO:teuthology.orchestra.run.smithi017.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [1,632 kB] 2022-08-01T19:17:42.481 INFO:teuthology.orchestra.run.smithi085.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-08-01T19:17:42.481 INFO:teuthology.orchestra.run.smithi085.stdout:Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2022-08-01T19:17:42.540 INFO:teuthology.orchestra.run.smithi085.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2022-08-01T19:17:42.647 INFO:teuthology.orchestra.run.smithi077.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1,998 kB] 2022-08-01T19:17:42.719 INFO:teuthology.orchestra.run.smithi017.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2022-08-01T19:17:42.794 INFO:teuthology.orchestra.run.smithi190.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2022-08-01T19:17:42.813 INFO:teuthology.orchestra.run.smithi077.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [472 kB] 2022-08-01T19:17:42.858 INFO:teuthology.orchestra.run.smithi017.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [472 kB] 2022-08-01T19:17:42.876 INFO:teuthology.orchestra.run.smithi085.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2022-08-01T19:17:42.917 INFO:teuthology.orchestra.run.smithi190.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [472 kB] 2022-08-01T19:17:42.939 INFO:teuthology.orchestra.run.smithi017.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1,998 kB] 2022-08-01T19:17:42.940 INFO:teuthology.orchestra.run.smithi017.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [277 kB] 2022-08-01T19:17:42.977 INFO:teuthology.orchestra.run.smithi085.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [1,632 kB] 2022-08-01T19:17:42.991 INFO:teuthology.orchestra.run.smithi017.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [10.8 kB] 2022-08-01T19:17:42.991 INFO:teuthology.orchestra.run.smithi017.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1,092 kB] 2022-08-01T19:17:43.026 INFO:teuthology.orchestra.run.smithi077.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [360 kB] 2022-08-01T19:17:43.062 INFO:teuthology.orchestra.run.smithi190.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [699 kB] 2022-08-01T19:17:43.112 INFO:teuthology.orchestra.run.smithi085.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [699 kB] 2022-08-01T19:17:43.113 INFO:teuthology.orchestra.run.smithi077.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [15.6 kB] 2022-08-01T19:17:43.114 INFO:teuthology.orchestra.run.smithi077.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [1,185 kB] 2022-08-01T19:17:43.194 INFO:teuthology.orchestra.run.smithi017.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [155 kB] 2022-08-01T19:17:43.235 INFO:teuthology.orchestra.run.smithi017.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [711 kB] 2022-08-01T19:17:43.302 INFO:teuthology.orchestra.run.smithi077.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [168 kB] 2022-08-01T19:17:43.343 INFO:teuthology.orchestra.run.smithi077.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [927 kB] 2022-08-01T19:17:43.344 INFO:teuthology.orchestra.run.smithi017.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [557 kB] 2022-08-01T19:17:43.462 INFO:teuthology.orchestra.run.smithi017.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [128 kB] 2022-08-01T19:17:43.463 INFO:teuthology.orchestra.run.smithi017.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [14.7 kB] 2022-08-01T19:17:43.512 INFO:teuthology.orchestra.run.smithi077.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [678 kB] 2022-08-01T19:17:43.518 INFO:teuthology.orchestra.run.smithi077.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [1,632 kB] 2022-08-01T19:17:43.617 INFO:teuthology.orchestra.run.smithi077.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [209 kB] 2022-08-01T19:17:43.645 INFO:teuthology.orchestra.run.smithi190.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [1,632 kB] 2022-08-01T19:17:43.661 INFO:teuthology.orchestra.run.smithi077.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [21.0 kB] 2022-08-01T19:17:44.064 INFO:teuthology.orchestra.run.smithi190.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1,998 kB] 2022-08-01T19:17:44.102 INFO:teuthology.orchestra.run.smithi085.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1,998 kB] 2022-08-01T19:17:45.239 INFO:teuthology.orchestra.run.smithi085.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [472 kB] 2022-08-01T19:17:45.550 INFO:teuthology.orchestra.run.smithi017.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [699 kB] 2022-08-01T19:17:45.721 INFO:teuthology.orchestra.run.smithi077.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [277 kB] 2022-08-01T19:17:45.861 INFO:teuthology.orchestra.run.smithi190.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [277 kB] 2022-08-01T19:17:45.862 INFO:teuthology.orchestra.run.smithi085.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [277 kB] 2022-08-01T19:17:46.015 INFO:teuthology.orchestra.run.smithi077.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [10.8 kB] 2022-08-01T19:17:46.032 INFO:teuthology.orchestra.run.smithi077.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1,092 kB] 2022-08-01T19:17:46.151 INFO:teuthology.orchestra.run.smithi190.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [360 kB] 2022-08-01T19:17:46.204 INFO:teuthology.orchestra.run.smithi085.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [10.8 kB] 2022-08-01T19:17:46.204 INFO:teuthology.orchestra.run.smithi085.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1,092 kB] 2022-08-01T19:17:46.205 INFO:teuthology.orchestra.run.smithi190.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [15.6 kB] 2022-08-01T19:17:46.233 INFO:teuthology.orchestra.run.smithi190.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [1,185 kB] 2022-08-01T19:17:46.233 INFO:teuthology.orchestra.run.smithi190.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [10.8 kB] 2022-08-01T19:17:46.270 INFO:teuthology.orchestra.run.smithi190.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1,092 kB] 2022-08-01T19:17:46.348 INFO:teuthology.orchestra.run.smithi190.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [168 kB] 2022-08-01T19:17:46.356 INFO:teuthology.orchestra.run.smithi190.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [927 kB] 2022-08-01T19:17:46.482 INFO:teuthology.orchestra.run.smithi190.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [678 kB] 2022-08-01T19:17:46.502 INFO:teuthology.orchestra.run.smithi190.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [209 kB] 2022-08-01T19:17:46.510 INFO:teuthology.orchestra.run.smithi190.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [21.0 kB] 2022-08-01T19:17:46.526 INFO:teuthology.orchestra.run.smithi017.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [360 kB] 2022-08-01T19:17:46.695 INFO:teuthology.orchestra.run.smithi085.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [360 kB] 2022-08-01T19:17:47.018 INFO:teuthology.orchestra.run.smithi017.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [15.6 kB] 2022-08-01T19:17:47.030 INFO:teuthology.orchestra.run.smithi017.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [1,185 kB] 2022-08-01T19:17:47.170 INFO:teuthology.orchestra.run.smithi085.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [15.6 kB] 2022-08-01T19:17:47.216 INFO:teuthology.orchestra.run.smithi085.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [1,185 kB] 2022-08-01T19:17:47.501 INFO:teuthology.orchestra.run.smithi077.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [155 kB] 2022-08-01T19:17:47.674 INFO:teuthology.orchestra.run.smithi085.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [155 kB] 2022-08-01T19:17:47.702 INFO:teuthology.orchestra.run.smithi190.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [155 kB] 2022-08-01T19:17:47.703 INFO:teuthology.orchestra.run.smithi077.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [711 kB] 2022-08-01T19:17:47.871 INFO:teuthology.orchestra.run.smithi085.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [557 kB] 2022-08-01T19:17:47.905 INFO:teuthology.orchestra.run.smithi190.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [557 kB] 2022-08-01T19:17:48.576 INFO:teuthology.orchestra.run.smithi085.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [711 kB] 2022-08-01T19:17:48.623 INFO:teuthology.orchestra.run.smithi017.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [168 kB] 2022-08-01T19:17:48.623 INFO:teuthology.orchestra.run.smithi190.stdout:Get:22 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [711 kB] 2022-08-01T19:17:48.678 INFO:teuthology.orchestra.run.smithi077.stdout:Get:22 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [557 kB] 2022-08-01T19:17:48.811 INFO:teuthology.orchestra.run.smithi085.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [168 kB] 2022-08-01T19:17:48.835 INFO:teuthology.orchestra.run.smithi017.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [927 kB] 2022-08-01T19:17:48.959 INFO:teuthology.orchestra.run.smithi085.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [927 kB] 2022-08-01T19:17:49.173 INFO:teuthology.orchestra.run.smithi085.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [678 kB] 2022-08-01T19:17:49.262 INFO:teuthology.orchestra.run.smithi085.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [209 kB] 2022-08-01T19:17:49.298 INFO:teuthology.orchestra.run.smithi085.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [21.0 kB] 2022-08-01T19:17:49.445 INFO:teuthology.orchestra.run.smithi077.stdout:Get:23 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [128 kB] 2022-08-01T19:17:49.503 INFO:teuthology.orchestra.run.smithi077.stdout:Get:24 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [14.7 kB] 2022-08-01T19:17:49.579 INFO:teuthology.orchestra.run.smithi085.stdout:Get:23 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [128 kB] 2022-08-01T19:17:49.590 INFO:teuthology.orchestra.run.smithi190.stdout:Get:23 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [128 kB] 2022-08-01T19:17:49.691 INFO:teuthology.orchestra.run.smithi085.stdout:Get:24 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [14.7 kB] 2022-08-01T19:17:49.781 INFO:teuthology.orchestra.run.smithi190.stdout:Get:24 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [14.7 kB] 2022-08-01T19:17:50.095 INFO:teuthology.orchestra.run.smithi017.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [678 kB] 2022-08-01T19:17:50.993 INFO:teuthology.orchestra.run.smithi017.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [209 kB] 2022-08-01T19:17:51.284 INFO:teuthology.orchestra.run.smithi017.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [21.0 kB] 2022-08-01T19:17:51.401 INFO:teuthology.orchestra.run.smithi077.stdout:Fetched 11.6 MB in 8s (1,547 kB/s) 2022-08-01T19:17:51.521 INFO:teuthology.orchestra.run.smithi085.stdout:Fetched 11.6 MB in 7s (1,573 kB/s) 2022-08-01T19:17:51.686 INFO:teuthology.orchestra.run.smithi190.stdout:Fetched 11.6 MB in 8s (1,525 kB/s) 2022-08-01T19:17:52.691 INFO:teuthology.orchestra.run.smithi077.stdout:Reading package lists... 2022-08-01T19:17:52.729 DEBUG:teuthology.orchestra.run.smithi077:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-08-01T19:17:52.731 INFO:teuthology.orchestra.run.smithi085.stdout:Reading package lists... 2022-08-01T19:17:52.772 DEBUG:teuthology.orchestra.run.smithi085:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-08-01T19:17:52.830 INFO:teuthology.orchestra.run.smithi077.stdout:Reading package lists... 2022-08-01T19:17:52.853 INFO:teuthology.orchestra.run.smithi085.stdout:Reading package lists... 2022-08-01T19:17:52.990 INFO:teuthology.orchestra.run.smithi190.stdout:Reading package lists... 2022-08-01T19:17:53.007 INFO:teuthology.orchestra.run.smithi077.stdout:Building dependency tree... 2022-08-01T19:17:53.008 INFO:teuthology.orchestra.run.smithi077.stdout:Reading state information... 2022-08-01T19:17:53.063 DEBUG:teuthology.orchestra.run.smithi190:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-08-01T19:17:53.065 INFO:teuthology.orchestra.run.smithi085.stdout:Building dependency tree... 2022-08-01T19:17:53.065 INFO:teuthology.orchestra.run.smithi085.stdout:Reading state information... 2022-08-01T19:17:53.146 INFO:teuthology.orchestra.run.smithi190.stdout:Reading package lists... 2022-08-01T19:17:53.231 INFO:teuthology.orchestra.run.smithi077.stdout:linux-image-generic is already the newest version (5.4.0.122.123). 2022-08-01T19:17:53.232 INFO:teuthology.orchestra.run.smithi077.stdout:linux-image-generic set to manually installed. 2022-08-01T19:17:53.232 INFO:teuthology.orchestra.run.smithi077.stdout:0 upgraded, 0 newly installed, 0 to remove and 4 not upgraded. 2022-08-01T19:17:53.233 DEBUG:teuthology.orchestra.run.smithi077:> dpkg -s linux-image-generic 2022-08-01T19:17:53.295 INFO:teuthology.orchestra.run.smithi085.stdout:linux-image-generic is already the newest version (5.4.0.122.123). 2022-08-01T19:17:53.295 INFO:teuthology.orchestra.run.smithi085.stdout:linux-image-generic set to manually installed. 2022-08-01T19:17:53.296 INFO:teuthology.orchestra.run.smithi085.stdout:0 upgraded, 0 newly installed, 0 to remove and 4 not upgraded. 2022-08-01T19:17:53.296 DEBUG:teuthology.orchestra.run.smithi085:> dpkg -s linux-image-generic 2022-08-01T19:17:53.328 INFO:teuthology.orchestra.run.smithi017.stdout:Fetched 11.6 MB in 9s (1,261 kB/s) 2022-08-01T19:17:53.392 INFO:teuthology.orchestra.run.smithi190.stdout:Building dependency tree... 2022-08-01T19:17:53.427 INFO:teuthology.orchestra.run.smithi190.stdout:Reading state information... 2022-08-01T19:17:53.433 INFO:teuthology.orchestra.run.smithi077.stdout:Package: linux-image-generic 2022-08-01T19:17:53.458 INFO:teuthology.orchestra.run.smithi077.stdout:Status: install ok installed 2022-08-01T19:17:53.459 INFO:teuthology.orchestra.run.smithi077.stdout:Priority: optional 2022-08-01T19:17:53.459 INFO:teuthology.orchestra.run.smithi077.stdout:Section: kernel 2022-08-01T19:17:53.459 INFO:teuthology.orchestra.run.smithi077.stdout:Installed-Size: 19 2022-08-01T19:17:53.459 INFO:teuthology.orchestra.run.smithi077.stdout:Maintainer: Ubuntu Kernel Team 2022-08-01T19:17:53.460 INFO:teuthology.orchestra.run.smithi077.stdout:Architecture: amd64 2022-08-01T19:17:53.460 INFO:teuthology.orchestra.run.smithi077.stdout:Source: linux-meta 2022-08-01T19:17:53.460 INFO:teuthology.orchestra.run.smithi077.stdout:Version: 5.4.0.122.123 2022-08-01T19:17:53.460 INFO:teuthology.orchestra.run.smithi077.stdout:Provides: virtualbox-guest-modules (= 6.1.32-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.14) 2022-08-01T19:17:53.461 INFO:teuthology.orchestra.run.smithi077.stdout:Depends: linux-image-5.4.0-122-generic, linux-modules-extra-5.4.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2022-08-01T19:17:53.461 INFO:teuthology.orchestra.run.smithi077.stdout:Recommends: thermald 2022-08-01T19:17:53.461 INFO:teuthology.orchestra.run.smithi077.stdout:Description: Generic Linux kernel image 2022-08-01T19:17:53.461 INFO:teuthology.orchestra.run.smithi077.stdout: This package will always depend on the latest generic kernel image 2022-08-01T19:17:53.462 INFO:teuthology.orchestra.run.smithi077.stdout: available. 2022-08-01T19:17:53.480 INFO:teuthology.orchestra.run.smithi085.stdout:Package: linux-image-generic 2022-08-01T19:17:53.481 INFO:teuthology.orchestra.run.smithi085.stdout:Status: install ok installed 2022-08-01T19:17:53.506 INFO:teuthology.orchestra.run.smithi085.stdout:Priority: optional 2022-08-01T19:17:53.507 INFO:teuthology.orchestra.run.smithi085.stdout:Section: kernel 2022-08-01T19:17:53.507 INFO:teuthology.orchestra.run.smithi085.stdout:Installed-Size: 19 2022-08-01T19:17:53.507 INFO:teuthology.orchestra.run.smithi085.stdout:Maintainer: Ubuntu Kernel Team 2022-08-01T19:17:53.508 INFO:teuthology.orchestra.run.smithi085.stdout:Architecture: amd64 2022-08-01T19:17:53.508 INFO:teuthology.orchestra.run.smithi085.stdout:Source: linux-meta 2022-08-01T19:17:53.556 INFO:teuthology.orchestra.run.smithi085.stdout:Version: 5.4.0.122.123 2022-08-01T19:17:53.556 INFO:teuthology.orchestra.run.smithi085.stdout:Provides: virtualbox-guest-modules (= 6.1.32-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.14) 2022-08-01T19:17:53.556 INFO:teuthology.orchestra.run.smithi085.stdout:Depends: linux-image-5.4.0-122-generic, linux-modules-extra-5.4.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2022-08-01T19:17:53.556 INFO:teuthology.orchestra.run.smithi085.stdout:Recommends: thermald 2022-08-01T19:17:53.557 INFO:teuthology.orchestra.run.smithi085.stdout:Description: Generic Linux kernel image 2022-08-01T19:17:53.557 INFO:teuthology.orchestra.run.smithi085.stdout: This package will always depend on the latest generic kernel image 2022-08-01T19:17:53.557 INFO:teuthology.orchestra.run.smithi085.stdout: available. 2022-08-01T19:17:53.557 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-08-01T19:17:53.558 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2022-08-01T19:17:53.558 DEBUG:teuthology.orchestra.run.smithi077:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-08-01T19:17:53.559 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-08-01T19:17:53.602 INFO:teuthology.task.kernel:Enabling kdb on client.0... 2022-08-01T19:17:53.603 DEBUG:teuthology.orchestra.run.smithi085:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-08-01T19:17:53.639 INFO:teuthology.orchestra.run.smithi190.stdout:linux-image-generic is already the newest version (5.4.0.122.123). 2022-08-01T19:17:53.640 INFO:teuthology.orchestra.run.smithi190.stdout:linux-image-generic set to manually installed. 2022-08-01T19:17:53.640 INFO:teuthology.orchestra.run.smithi190.stdout:0 upgraded, 0 newly installed, 0 to remove and 4 not upgraded. 2022-08-01T19:17:53.690 DEBUG:teuthology.orchestra.run.smithi190:> dpkg -s linux-image-generic 2022-08-01T19:17:53.770 INFO:teuthology.orchestra.run.smithi085.stdout:ttyS1 2022-08-01T19:17:53.771 INFO:teuthology.orchestra.run.smithi077.stdout:ttyS1 2022-08-01T19:17:53.806 DEBUG:teuthology.parallel:result is None 2022-08-01T19:17:53.807 DEBUG:teuthology.parallel:result is None 2022-08-01T19:17:53.832 INFO:teuthology.orchestra.run.smithi190.stdout:Package: linux-image-generic 2022-08-01T19:17:53.860 INFO:teuthology.orchestra.run.smithi190.stdout:Status: install ok installed 2022-08-01T19:17:53.860 INFO:teuthology.orchestra.run.smithi190.stdout:Priority: optional 2022-08-01T19:17:53.861 INFO:teuthology.orchestra.run.smithi190.stdout:Section: kernel 2022-08-01T19:17:53.861 INFO:teuthology.orchestra.run.smithi190.stdout:Installed-Size: 19 2022-08-01T19:17:53.861 INFO:teuthology.orchestra.run.smithi190.stdout:Maintainer: Ubuntu Kernel Team 2022-08-01T19:17:53.880 INFO:teuthology.orchestra.run.smithi190.stdout:Architecture: amd64 2022-08-01T19:17:53.880 INFO:teuthology.orchestra.run.smithi190.stdout:Source: linux-meta 2022-08-01T19:17:53.880 INFO:teuthology.orchestra.run.smithi190.stdout:Version: 5.4.0.122.123 2022-08-01T19:17:53.880 INFO:teuthology.orchestra.run.smithi190.stdout:Provides: virtualbox-guest-modules (= 6.1.32-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.14) 2022-08-01T19:17:53.881 INFO:teuthology.orchestra.run.smithi190.stdout:Depends: linux-image-5.4.0-122-generic, linux-modules-extra-5.4.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2022-08-01T19:17:53.881 INFO:teuthology.orchestra.run.smithi190.stdout:Recommends: thermald 2022-08-01T19:17:53.881 INFO:teuthology.orchestra.run.smithi190.stdout:Description: Generic Linux kernel image 2022-08-01T19:17:53.881 INFO:teuthology.orchestra.run.smithi190.stdout: This package will always depend on the latest generic kernel image 2022-08-01T19:17:53.882 INFO:teuthology.orchestra.run.smithi190.stdout: available. 2022-08-01T19:17:53.882 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-08-01T19:17:53.882 INFO:teuthology.task.kernel:Enabling kdb on client.1... 2022-08-01T19:17:53.901 DEBUG:teuthology.orchestra.run.smithi190:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-08-01T19:17:53.950 INFO:teuthology.orchestra.run.smithi190.stdout:ttyS1 2022-08-01T19:17:53.972 DEBUG:teuthology.parallel:result is None 2022-08-01T19:17:54.632 INFO:teuthology.orchestra.run.smithi017.stdout:Reading package lists... 2022-08-01T19:17:54.659 DEBUG:teuthology.orchestra.run.smithi017:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-08-01T19:17:54.788 INFO:teuthology.orchestra.run.smithi017.stdout:Reading package lists... 2022-08-01T19:17:54.949 INFO:teuthology.orchestra.run.smithi017.stdout:Building dependency tree... 2022-08-01T19:17:54.949 INFO:teuthology.orchestra.run.smithi017.stdout:Reading state information... 2022-08-01T19:17:55.178 INFO:teuthology.orchestra.run.smithi017.stdout:linux-image-generic is already the newest version (5.4.0.122.123). 2022-08-01T19:17:55.191 INFO:teuthology.orchestra.run.smithi017.stdout:linux-image-generic set to manually installed. 2022-08-01T19:17:55.192 INFO:teuthology.orchestra.run.smithi017.stdout:0 upgraded, 0 newly installed, 0 to remove and 4 not upgraded. 2022-08-01T19:17:55.211 DEBUG:teuthology.orchestra.run.smithi017:> dpkg -s linux-image-generic 2022-08-01T19:17:55.298 INFO:teuthology.orchestra.run.smithi017.stdout:Package: linux-image-generic 2022-08-01T19:17:55.302 INFO:teuthology.orchestra.run.smithi017.stdout:Status: install ok installed 2022-08-01T19:17:55.329 INFO:teuthology.orchestra.run.smithi017.stdout:Priority: optional 2022-08-01T19:17:55.350 INFO:teuthology.orchestra.run.smithi017.stdout:Section: kernel 2022-08-01T19:17:55.350 INFO:teuthology.orchestra.run.smithi017.stdout:Installed-Size: 19 2022-08-01T19:17:55.351 INFO:teuthology.orchestra.run.smithi017.stdout:Maintainer: Ubuntu Kernel Team 2022-08-01T19:17:55.351 INFO:teuthology.orchestra.run.smithi017.stdout:Architecture: amd64 2022-08-01T19:17:55.352 INFO:teuthology.orchestra.run.smithi017.stdout:Source: linux-meta 2022-08-01T19:17:55.352 INFO:teuthology.orchestra.run.smithi017.stdout:Version: 5.4.0.122.123 2022-08-01T19:17:55.353 INFO:teuthology.orchestra.run.smithi017.stdout:Provides: virtualbox-guest-modules (= 6.1.32-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.14) 2022-08-01T19:17:55.353 INFO:teuthology.orchestra.run.smithi017.stdout:Depends: linux-image-5.4.0-122-generic, linux-modules-extra-5.4.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2022-08-01T19:17:55.353 INFO:teuthology.orchestra.run.smithi017.stdout:Recommends: thermald 2022-08-01T19:17:55.401 INFO:teuthology.orchestra.run.smithi017.stdout:Description: Generic Linux kernel image 2022-08-01T19:17:55.412 INFO:teuthology.orchestra.run.smithi017.stdout: This package will always depend on the latest generic kernel image 2022-08-01T19:17:55.413 INFO:teuthology.orchestra.run.smithi017.stdout: available. 2022-08-01T19:17:55.421 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-08-01T19:17:55.422 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2022-08-01T19:17:55.435 DEBUG:teuthology.orchestra.run.smithi017:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-08-01T19:17:55.511 INFO:teuthology.orchestra.run.smithi017.stdout:ttyS1 2022-08-01T19:17:55.519 DEBUG:teuthology.parallel:result is None 2022-08-01T19:17:55.528 INFO:teuthology.run_tasks:Running task internal.base... 2022-08-01T19:17:55.634 INFO:teuthology.task.internal:Creating test directory... 2022-08-01T19:17:55.635 DEBUG:teuthology.orchestra.run.smithi017:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-08-01T19:17:55.639 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-08-01T19:17:55.725 DEBUG:teuthology.orchestra.run.smithi085:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-08-01T19:17:55.829 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-08-01T19:17:55.877 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2022-08-01T19:17:55.917 INFO:teuthology.run_tasks:Running task internal.archive... 2022-08-01T19:17:56.068 INFO:teuthology.task.internal:Creating archive directory... 2022-08-01T19:17:56.069 DEBUG:teuthology.orchestra.run.smithi017:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-08-01T19:17:56.115 DEBUG:teuthology.orchestra.run.smithi077:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-08-01T19:17:56.137 DEBUG:teuthology.orchestra.run.smithi085:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-08-01T19:17:56.245 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-08-01T19:17:56.343 INFO:teuthology.run_tasks:Running task internal.coredump... 2022-08-01T19:17:56.409 INFO:teuthology.task.internal:Enabling coredump saving... 2022-08-01T19:17:56.432 DEBUG:teuthology.orchestra.run.smithi017:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-08-01T19:17:56.438 DEBUG:teuthology.orchestra.run.smithi077:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-08-01T19:17:56.447 DEBUG:teuthology.orchestra.run.smithi085:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-08-01T19:17:56.468 INFO:teuthology.orchestra.run.smithi017.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-08-01T19:17:56.501 INFO:teuthology.orchestra.run.smithi017.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-08-01T19:17:56.578 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-08-01T19:17:56.641 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-08-01T19:17:56.647 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-08-01T19:17:56.670 INFO:teuthology.orchestra.run.smithi085.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-08-01T19:17:56.670 INFO:teuthology.orchestra.run.smithi085.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-08-01T19:17:56.720 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-08-01T19:17:56.720 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-08-01T19:17:56.767 INFO:teuthology.run_tasks:Running task internal.sudo... 2022-08-01T19:17:56.863 INFO:teuthology.task.internal:Configuring sudo... 2022-08-01T19:17:56.864 DEBUG:teuthology.orchestra.run.smithi017:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-08-01T19:17:56.926 DEBUG:teuthology.orchestra.run.smithi077:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-08-01T19:17:56.991 DEBUG:teuthology.orchestra.run.smithi085:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-08-01T19:17:57.055 DEBUG:teuthology.orchestra.run.smithi190:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-08-01T19:17:57.157 INFO:teuthology.run_tasks:Running task internal.syslog... 2022-08-01T19:17:57.312 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2022-08-01T19:17:57.313 DEBUG:teuthology.orchestra.run.smithi017:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-08-01T19:17:57.336 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-08-01T19:17:57.385 DEBUG:teuthology.orchestra.run.smithi085:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-08-01T19:17:57.422 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-08-01T19:17:57.511 DEBUG:teuthology.orchestra.run.smithi017:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-08-01T19:17:57.552 DEBUG:teuthology.orchestra.run.smithi017:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-08-01T19:17:57.708 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2022-08-01T19:17:57.709 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-08-01T19:17:57.837 DEBUG:teuthology.orchestra.run.smithi077:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-08-01T19:17:57.943 DEBUG:teuthology.orchestra.run.smithi077:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-08-01T19:17:57.995 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2022-08-01T19:17:57.996 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-08-01T19:17:58.160 DEBUG:teuthology.orchestra.run.smithi085:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-08-01T19:17:58.186 DEBUG:teuthology.orchestra.run.smithi085:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-08-01T19:17:58.232 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-08-01T19:17:58.232 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-08-01T19:17:58.300 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-08-01T19:17:58.399 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-08-01T19:17:58.473 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2022-08-01T19:17:58.473 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-08-01T19:17:58.571 DEBUG:teuthology.orchestra.run.smithi017:> sudo service rsyslog restart 2022-08-01T19:17:58.611 DEBUG:teuthology.orchestra.run.smithi077:> sudo service rsyslog restart 2022-08-01T19:17:58.664 DEBUG:teuthology.orchestra.run.smithi085:> sudo service rsyslog restart 2022-08-01T19:17:58.740 DEBUG:teuthology.orchestra.run.smithi190:> sudo service rsyslog restart 2022-08-01T19:17:58.853 INFO:teuthology.run_tasks:Running task internal.timer... 2022-08-01T19:17:58.939 INFO:teuthology.task.internal:Starting timer... 2022-08-01T19:17:58.940 INFO:teuthology.run_tasks:Running task pcp... 2022-08-01T19:17:58.996 INFO:teuthology.run_tasks:Running task selinux... 2022-08-01T19:17:59.058 DEBUG:teuthology.task.selinux:Excluding smithi017: OS 'ubuntu' does not support SELinux 2022-08-01T19:17:59.059 DEBUG:teuthology.task.selinux:Excluding smithi077: OS 'ubuntu' does not support SELinux 2022-08-01T19:17:59.061 DEBUG:teuthology.task.selinux:Excluding smithi085: OS 'ubuntu' does not support SELinux 2022-08-01T19:17:59.064 DEBUG:teuthology.task.selinux:Excluding smithi190: OS 'ubuntu' does not support SELinux 2022-08-01T19:17:59.065 DEBUG:teuthology.task.selinux:Getting current SELinux state 2022-08-01T19:17:59.065 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2022-08-01T19:17:59.066 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2022-08-01T19:17:59.066 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2022-08-01T19:17:59.151 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2022-08-01T19:17:59.178 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_main to origin/main 2022-08-01T19:17:59.459 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2022-08-01T19:17:59.462 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi017.front.sepia.ceph.com,smithi077.front.sepia.ceph.com,smithi085.front.sepia.ceph.com,smithi190.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_main/cephlab.yml 2022-08-01T19:30:41.466 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi017.front.sepia.ceph.com'), Remote(name='ubuntu@smithi077.front.sepia.ceph.com'), Remote(name='ubuntu@smithi085.front.sepia.ceph.com'), Remote(name='ubuntu@smithi190.front.sepia.ceph.com')] 2022-08-01T19:30:41.470 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2022-08-01T19:30:41.472 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-08-01T19:30:41.578 DEBUG:teuthology.orchestra.run.smithi017:> true 2022-08-01T19:30:41.700 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2022-08-01T19:30:41.701 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-08-01T19:30:41.805 DEBUG:teuthology.orchestra.run.smithi077:> true 2022-08-01T19:30:41.928 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi085.front.sepia.ceph.com' 2022-08-01T19:30:41.929 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi085.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-08-01T19:30:42.025 DEBUG:teuthology.orchestra.run.smithi085:> true 2022-08-01T19:30:42.149 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2022-08-01T19:30:42.150 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-08-01T19:30:42.239 DEBUG:teuthology.orchestra.run.smithi190:> true 2022-08-01T19:30:42.365 INFO:teuthology.run_tasks:Running task clock... 2022-08-01T19:30:42.378 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2022-08-01T19:30:42.379 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-08-01T19:30:42.382 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-08-01T19:30:42.385 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-08-01T19:30:42.385 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-08-01T19:30:42.388 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-08-01T19:30:42.388 DEBUG:teuthology.orchestra.run.smithi085:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-08-01T19:30:42.390 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-08-01T19:30:42.391 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-08-01T19:30:42.420 INFO:teuthology.orchestra.run.smithi017.stdout: 1 Aug 19:30:42 ntpd[17934]: ntpd 4.2.8p12@1.3728-o (1): Starting 2022-08-01T19:30:42.421 INFO:teuthology.orchestra.run.smithi017.stdout: 1 Aug 19:30:42 ntpd[17934]: Command line: ntpd -gq 2022-08-01T19:30:42.422 INFO:teuthology.orchestra.run.smithi017.stdout: 1 Aug 19:30:42 ntpd[17934]: proto: precision = 0.117 usec (-23) 2022-08-01T19:30:42.423 INFO:teuthology.orchestra.run.smithi077.stdout: 1 Aug 19:30:42 ntpd[18271]: ntpd 4.2.8p12@1.3728-o (1): Starting 2022-08-01T19:30:42.423 INFO:teuthology.orchestra.run.smithi077.stdout: 1 Aug 19:30:42 ntpd[18271]: Command line: ntpd -gq 2022-08-01T19:30:42.424 INFO:teuthology.orchestra.run.smithi017.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-08-01T19:30:42.424 INFO:teuthology.orchestra.run.smithi017.stdout: 1 Aug 19:30:42 ntpd[17934]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-08-01T19:30:42.425 INFO:teuthology.orchestra.run.smithi017.stderr:restrict ::: KOD does nothing without LIMITED. 2022-08-01T19:30:42.426 INFO:teuthology.orchestra.run.smithi017.stdout: 1 Aug 19:30:42 ntpd[17934]: restrict ::: KOD does nothing without LIMITED. 2022-08-01T19:30:42.426 INFO:teuthology.orchestra.run.smithi017.stdout: 1 Aug 19:30:42 ntpd[17934]: Listen and drop on 0 v6wildcard [::]:123 2022-08-01T19:30:42.427 INFO:teuthology.orchestra.run.smithi017.stdout: 1 Aug 19:30:42 ntpd[17934]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2022-08-01T19:30:42.427 INFO:teuthology.orchestra.run.smithi017.stdout: 1 Aug 19:30:42 ntpd[17934]: Listen normally on 2 lo 127.0.0.1:123 2022-08-01T19:30:42.427 INFO:teuthology.orchestra.run.smithi017.stdout: 1 Aug 19:30:42 ntpd[17934]: Listen normally on 3 ens1f0 172.21.15.17:123 2022-08-01T19:30:42.427 INFO:teuthology.orchestra.run.smithi017.stdout: 1 Aug 19:30:42 ntpd[17934]: Listen normally on 4 lo [::1]:123 2022-08-01T19:30:42.427 INFO:teuthology.orchestra.run.smithi017.stdout: 1 Aug 19:30:42 ntpd[17934]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:fe59:d992%3]:123 2022-08-01T19:30:42.428 INFO:teuthology.orchestra.run.smithi017.stdout: 1 Aug 19:30:42 ntpd[17934]: Listening on routing socket on fd #22 for interface updates 2022-08-01T19:30:42.429 INFO:teuthology.orchestra.run.smithi077.stdout: 1 Aug 19:30:42 ntpd[18271]: proto: precision = 0.123 usec (-23) 2022-08-01T19:30:42.429 INFO:teuthology.orchestra.run.smithi085.stdout: 1 Aug 19:30:42 ntpd[18231]: ntpd 4.2.8p12@1.3728-o (1): Starting 2022-08-01T19:30:42.430 INFO:teuthology.orchestra.run.smithi085.stdout: 1 Aug 19:30:42 ntpd[18231]: Command line: ntpd -gq 2022-08-01T19:30:42.430 INFO:teuthology.orchestra.run.smithi085.stdout: 1 Aug 19:30:42 ntpd[18231]: proto: precision = 0.067 usec (-24) 2022-08-01T19:30:42.430 INFO:teuthology.orchestra.run.smithi190.stdout: 1 Aug 19:30:42 ntpd[18381]: ntpd 4.2.8p12@1.3728-o (1): Starting 2022-08-01T19:30:42.431 INFO:teuthology.orchestra.run.smithi190.stdout: 1 Aug 19:30:42 ntpd[18381]: Command line: ntpd -gq 2022-08-01T19:30:42.431 INFO:teuthology.orchestra.run.smithi190.stdout: 1 Aug 19:30:42 ntpd[18381]: proto: precision = 0.204 usec (-22) 2022-08-01T19:30:42.431 INFO:teuthology.orchestra.run.smithi077.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-08-01T19:30:42.432 INFO:teuthology.orchestra.run.smithi085.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-08-01T19:30:42.432 INFO:teuthology.orchestra.run.smithi190.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-08-01T19:30:42.433 INFO:teuthology.orchestra.run.smithi077.stdout: 1 Aug 19:30:42 ntpd[18271]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-08-01T19:30:42.433 INFO:teuthology.orchestra.run.smithi085.stdout: 1 Aug 19:30:42 ntpd[18231]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-08-01T19:30:42.434 INFO:teuthology.orchestra.run.smithi190.stdout: 1 Aug 19:30:42 ntpd[18381]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-08-01T19:30:42.434 INFO:teuthology.orchestra.run.smithi077.stderr:restrict ::: KOD does nothing without LIMITED. 2022-08-01T19:30:42.434 INFO:teuthology.orchestra.run.smithi085.stderr:restrict ::: KOD does nothing without LIMITED. 2022-08-01T19:30:42.435 INFO:teuthology.orchestra.run.smithi190.stderr:restrict ::: KOD does nothing without LIMITED. 2022-08-01T19:30:42.438 INFO:teuthology.orchestra.run.smithi077.stdout: 1 Aug 19:30:42 ntpd[18271]: restrict ::: KOD does nothing without LIMITED. 2022-08-01T19:30:42.438 INFO:teuthology.orchestra.run.smithi077.stdout: 1 Aug 19:30:42 ntpd[18271]: Listen and drop on 0 v6wildcard [::]:123 2022-08-01T19:30:42.438 INFO:teuthology.orchestra.run.smithi077.stdout: 1 Aug 19:30:42 ntpd[18271]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2022-08-01T19:30:42.438 INFO:teuthology.orchestra.run.smithi077.stdout: 1 Aug 19:30:42 ntpd[18271]: Listen normally on 2 lo 127.0.0.1:123 2022-08-01T19:30:42.439 INFO:teuthology.orchestra.run.smithi077.stdout: 1 Aug 19:30:42 ntpd[18271]: Listen normally on 3 enp3s0f1 172.21.15.77:123 2022-08-01T19:30:42.439 INFO:teuthology.orchestra.run.smithi077.stdout: 1 Aug 19:30:42 ntpd[18271]: Listen normally on 4 lo [::1]:123 2022-08-01T19:30:42.439 INFO:teuthology.orchestra.run.smithi077.stdout: 1 Aug 19:30:42 ntpd[18271]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d3d3%5]:123 2022-08-01T19:30:42.439 INFO:teuthology.orchestra.run.smithi077.stdout: 1 Aug 19:30:42 ntpd[18271]: Listening on routing socket on fd #22 for interface updates 2022-08-01T19:30:42.440 INFO:teuthology.orchestra.run.smithi085.stdout: 1 Aug 19:30:42 ntpd[18231]: restrict ::: KOD does nothing without LIMITED. 2022-08-01T19:30:42.440 INFO:teuthology.orchestra.run.smithi085.stdout: 1 Aug 19:30:42 ntpd[18231]: Listen and drop on 0 v6wildcard [::]:123 2022-08-01T19:30:42.440 INFO:teuthology.orchestra.run.smithi085.stdout: 1 Aug 19:30:42 ntpd[18231]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2022-08-01T19:30:42.440 INFO:teuthology.orchestra.run.smithi085.stdout: 1 Aug 19:30:42 ntpd[18231]: Listen normally on 2 lo 127.0.0.1:123 2022-08-01T19:30:42.440 INFO:teuthology.orchestra.run.smithi085.stdout: 1 Aug 19:30:42 ntpd[18231]: Listen normally on 3 enp3s0f1 172.21.15.85:123 2022-08-01T19:30:42.441 INFO:teuthology.orchestra.run.smithi085.stdout: 1 Aug 19:30:42 ntpd[18231]: Listen normally on 4 lo [::1]:123 2022-08-01T19:30:42.441 INFO:teuthology.orchestra.run.smithi085.stdout: 1 Aug 19:30:42 ntpd[18231]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d7bb%5]:123 2022-08-01T19:30:42.441 INFO:teuthology.orchestra.run.smithi085.stdout: 1 Aug 19:30:42 ntpd[18231]: Listening on routing socket on fd #22 for interface updates 2022-08-01T19:30:42.441 INFO:teuthology.orchestra.run.smithi190.stdout: 1 Aug 19:30:42 ntpd[18381]: restrict ::: KOD does nothing without LIMITED. 2022-08-01T19:30:42.442 INFO:teuthology.orchestra.run.smithi190.stdout: 1 Aug 19:30:42 ntpd[18381]: Listen and drop on 0 v6wildcard [::]:123 2022-08-01T19:30:42.442 INFO:teuthology.orchestra.run.smithi190.stdout: 1 Aug 19:30:42 ntpd[18381]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2022-08-01T19:30:42.442 INFO:teuthology.orchestra.run.smithi190.stdout: 1 Aug 19:30:42 ntpd[18381]: Listen normally on 2 lo 127.0.0.1:123 2022-08-01T19:30:42.442 INFO:teuthology.orchestra.run.smithi190.stdout: 1 Aug 19:30:42 ntpd[18381]: Listen normally on 3 enp3s0f1 172.21.15.190:123 2022-08-01T19:30:42.442 INFO:teuthology.orchestra.run.smithi190.stdout: 1 Aug 19:30:42 ntpd[18381]: Listen normally on 4 lo [::1]:123 2022-08-01T19:30:42.443 INFO:teuthology.orchestra.run.smithi190.stdout: 1 Aug 19:30:42 ntpd[18381]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7763%5]:123 2022-08-01T19:30:42.443 INFO:teuthology.orchestra.run.smithi190.stdout: 1 Aug 19:30:42 ntpd[18381]: Listening on routing socket on fd #22 for interface updates 2022-08-01T19:30:43.422 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:43 ntpd[17934]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:43.422 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:43 ntpd[17934]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:43.423 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:43 ntpd[17934]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:43.423 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:43 ntpd[17934]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:43.423 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:43 ntpd[17934]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:43.423 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:43 ntpd[17934]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:43.424 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:43 ntpd[17934]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T19:30:43.424 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:43 ntpd[17934]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T19:30:43.425 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:43 ntpd[18231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:43.426 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:43 ntpd[18231]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:43.427 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:43 ntpd[18381]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:43.428 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:43 ntpd[18381]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:43.428 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:43 ntpd[18381]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:43.428 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:43 ntpd[18381]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:43.428 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:43 ntpd[18381]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:43.429 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:43 ntpd[18381]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:43.429 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:43 ntpd[18381]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T19:30:43.429 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:43 ntpd[18381]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T19:30:43.429 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:43 ntpd[18271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:43.430 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:43 ntpd[18271]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:43.430 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:43 ntpd[18271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:43.430 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:43 ntpd[18271]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:43.430 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:43 ntpd[18271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:43.431 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:43 ntpd[18271]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:43.431 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:43 ntpd[18271]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T19:30:43.431 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:43 ntpd[18271]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T19:30:44.422 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:44 ntpd[17934]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:44.422 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:44 ntpd[17934]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:44.422 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:44 ntpd[17934]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:44.423 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:44 ntpd[17934]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:44.423 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:44 ntpd[17934]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T19:30:44.423 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:44 ntpd[17934]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T19:30:44.424 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:44 ntpd[18381]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:44.424 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:44 ntpd[18381]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:44.425 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:44 ntpd[18381]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:44.425 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:44 ntpd[18381]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:44.425 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:44 ntpd[18381]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T19:30:44.425 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:44 ntpd[18381]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T19:30:45.422 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:45 ntpd[17934]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:45.422 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:45 ntpd[17934]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:45.423 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:45 ntpd[17934]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:45.423 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:45 ntpd[17934]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:45.423 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:45 ntpd[17934]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T19:30:45.424 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:45 ntpd[17934]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T19:30:45.425 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:45 ntpd[18381]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:45.426 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:45 ntpd[18381]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:45.426 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:45 ntpd[18381]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:45.426 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:45 ntpd[18381]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:45.427 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:45 ntpd[18381]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T19:30:45.427 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:45 ntpd[18381]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T19:30:45.427 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:45 ntpd[18271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:45.427 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:45 ntpd[18271]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:45.428 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:45 ntpd[18271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:45.428 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:45 ntpd[18271]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:45.428 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:45 ntpd[18271]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T19:30:45.428 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:45 ntpd[18271]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T19:30:46.422 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:46 ntpd[17934]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:46.422 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:46 ntpd[17934]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:46.422 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:46 ntpd[17934]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:46.422 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:46 ntpd[17934]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:46.423 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:46 ntpd[17934]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T19:30:46.423 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:46 ntpd[17934]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T19:30:46.425 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:46 ntpd[18271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:46.425 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:46 ntpd[18271]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:46.426 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:46 ntpd[18271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:46.426 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:46 ntpd[18271]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:46.426 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:46 ntpd[18271]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T19:30:46.426 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:46 ntpd[18271]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T19:30:46.427 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:46 ntpd[18381]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:46.427 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:46 ntpd[18381]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:46.428 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:46 ntpd[18381]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:46.428 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:46 ntpd[18381]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:46.428 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:46 ntpd[18381]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:46.428 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:46 ntpd[18381]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:46.429 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:46 ntpd[18381]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T19:30:46.429 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:46 ntpd[18381]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T19:30:46.429 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:46 ntpd[18381]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:46.429 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:46 ntpd[18381]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:46.430 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:46 ntpd[18381]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T19:30:46.430 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:46 ntpd[18381]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T19:30:46.432 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:46 ntpd[18231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:46.432 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:46 ntpd[18231]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:46.432 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:46 ntpd[18231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:46.432 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:46 ntpd[18231]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:46.433 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:46 ntpd[18231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:46.433 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:46 ntpd[18231]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:46.433 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:46 ntpd[18231]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T19:30:46.433 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:46 ntpd[18231]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T19:30:46.433 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:46 ntpd[18231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:46.434 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:46 ntpd[18231]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:46.434 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:46 ntpd[18231]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T19:30:46.434 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:46 ntpd[18231]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T19:30:47.423 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:47 ntpd[17934]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:47.423 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:47 ntpd[17934]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:47.423 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:47 ntpd[17934]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:47.424 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:47 ntpd[17934]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:47.424 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:47 ntpd[17934]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:47.424 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:47 ntpd[17934]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:47.424 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:47 ntpd[17934]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T19:30:47.425 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:47 ntpd[17934]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T19:30:47.425 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:47 ntpd[17934]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:47.425 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:47 ntpd[17934]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:47.425 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:47 ntpd[17934]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T19:30:47.426 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:47 ntpd[17934]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T19:30:47.428 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:47 ntpd[18231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:47.429 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:47 ntpd[18231]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:47.429 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:47 ntpd[18231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:47.429 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:47 ntpd[18231]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:47.429 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:47 ntpd[18231]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T19:30:47.429 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:47 ntpd[18231]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T19:30:47.430 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:47 ntpd[18381]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:47.430 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:47 ntpd[18381]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:47.430 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:47 ntpd[18381]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:47.430 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:47 ntpd[18381]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:47.431 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:47 ntpd[18381]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T19:30:47.431 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:47 ntpd[18381]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T19:30:47.432 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:47 ntpd[18271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:47.432 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:47 ntpd[18271]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:47.433 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:47 ntpd[18271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:47.433 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:47 ntpd[18271]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:47.433 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:47 ntpd[18271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:47.433 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:47 ntpd[18271]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:47.434 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:47 ntpd[18271]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T19:30:47.434 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:47 ntpd[18271]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T19:30:47.434 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:47 ntpd[18271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:47.434 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:47 ntpd[18271]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:47.435 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:47 ntpd[18271]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T19:30:47.435 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:47 ntpd[18271]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T19:30:48.422 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:48 ntpd[17934]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:48.422 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:48 ntpd[17934]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:48.422 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:48 ntpd[17934]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:48.422 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:48 ntpd[17934]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:48.423 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:48 ntpd[17934]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T19:30:48.423 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:48 ntpd[17934]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T19:30:48.425 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:48 ntpd[18271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:48.425 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:48 ntpd[18271]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:48.426 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:48 ntpd[18271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:48.426 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:48 ntpd[18271]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:48.426 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:48 ntpd[18271]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T19:30:48.426 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:48 ntpd[18271]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T19:30:48.427 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:48 ntpd[18381]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:48.427 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:48 ntpd[18381]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:48.428 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:48 ntpd[18381]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:48.428 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:48 ntpd[18381]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:48.428 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:48 ntpd[18381]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:48.428 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:48 ntpd[18381]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:48.429 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:48 ntpd[18381]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T19:30:48.429 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:48 ntpd[18381]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T19:30:48.429 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:48 ntpd[18381]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:48.429 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:48 ntpd[18381]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:48.429 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:48 ntpd[18381]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T19:30:48.430 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:48 ntpd[18381]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T19:30:48.431 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:48 ntpd[18231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:48.432 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:48 ntpd[18231]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:48.432 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:48 ntpd[18231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:48.432 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:48 ntpd[18231]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:48.432 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:48 ntpd[18231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:48.433 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:48 ntpd[18231]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:48.433 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:48 ntpd[18231]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T19:30:48.433 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:48 ntpd[18231]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T19:30:48.433 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:48 ntpd[18231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:48.433 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:48 ntpd[18231]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:48.434 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:48 ntpd[18231]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T19:30:48.434 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:48 ntpd[18231]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T19:30:49.422 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:49 ntpd[17934]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:49.422 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:49 ntpd[17934]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:49.422 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:49 ntpd[17934]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:49.423 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:49 ntpd[17934]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:49.423 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:49 ntpd[17934]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:49.423 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:49 ntpd[17934]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:49.423 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:49 ntpd[17934]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T19:30:49.424 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:49 ntpd[17934]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T19:30:49.424 INFO:teuthology.orchestra.run.smithi017.stdout: 1 Aug 19:30:49 ntpd[17934]: ntpd: time slew -0.001050 s 2022-08-01T19:30:49.424 INFO:teuthology.orchestra.run.smithi017.stdout:ntpd: time slew -0.001050s 2022-08-01T19:30:49.425 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:49 ntpd[17934]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2022-08-01T19:30:49.425 INFO:teuthology.orchestra.run.smithi017.stderr: 1 Aug 19:30:49 ntpd[17934]: can't open /var/log/ntpstats/loopstats.20220801: Permission denied 2022-08-01T19:30:49.428 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:49 ntpd[18231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:49.428 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:49 ntpd[18231]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:49.428 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:49 ntpd[18231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:49.429 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:49 ntpd[18231]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:49.429 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:49 ntpd[18231]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T19:30:49.429 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:49 ntpd[18231]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T19:30:49.429 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:49 ntpd[18381]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:49.430 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:49 ntpd[18381]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:49.430 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:49 ntpd[18381]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:49.430 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:49 ntpd[18381]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:49.430 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:49 ntpd[18381]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T19:30:49.431 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:49 ntpd[18381]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T19:30:49.431 INFO:teuthology.orchestra.run.smithi190.stdout: 1 Aug 19:30:49 ntpd[18381]: ntpd: time slew +0.000257 s 2022-08-01T19:30:49.431 INFO:teuthology.orchestra.run.smithi190.stdout:ntpd: time slew +0.000257s 2022-08-01T19:30:49.433 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:49 ntpd[18381]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2022-08-01T19:30:49.433 INFO:teuthology.orchestra.run.smithi190.stderr: 1 Aug 19:30:49 ntpd[18381]: can't open /var/log/ntpstats/loopstats.20220801: Permission denied 2022-08-01T19:30:49.433 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:49 ntpd[18271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:49.434 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:49 ntpd[18271]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:49.434 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:49 ntpd[18271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:49.434 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:49 ntpd[18271]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:49.434 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:49 ntpd[18271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:49.435 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:49 ntpd[18271]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:49.435 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:49 ntpd[18271]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T19:30:49.435 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:49 ntpd[18271]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T19:30:49.435 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:49 ntpd[18271]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:49.435 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:49 ntpd[18271]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:49.436 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:49 ntpd[18271]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T19:30:49.436 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:49 ntpd[18271]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T19:30:49.436 INFO:teuthology.orchestra.run.smithi077.stdout: 1 Aug 19:30:49 ntpd[18271]: ntpd: time slew +0.002114 s 2022-08-01T19:30:49.437 INFO:teuthology.orchestra.run.smithi077.stdout:ntpd: time slew +0.002114s 2022-08-01T19:30:49.437 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:49 ntpd[18271]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2022-08-01T19:30:49.437 INFO:teuthology.orchestra.run.smithi077.stderr: 1 Aug 19:30:49 ntpd[18271]: can't open /var/log/ntpstats/loopstats.20220801: Permission denied 2022-08-01T19:30:49.469 INFO:teuthology.orchestra.run.smithi190.stdout: remote refid st t when poll reach delay offset jitter 2022-08-01T19:30:49.470 INFO:teuthology.orchestra.run.smithi190.stdout:============================================================================== 2022-08-01T19:30:49.470 INFO:teuthology.orchestra.run.smithi190.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-08-01T19:30:49.470 INFO:teuthology.orchestra.run.smithi190.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-08-01T19:30:49.470 INFO:teuthology.orchestra.run.smithi190.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-08-01T19:30:49.471 INFO:teuthology.orchestra.run.smithi190.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-08-01T19:30:49.474 INFO:teuthology.orchestra.run.smithi077.stdout: remote refid st t when poll reach delay offset jitter 2022-08-01T19:30:49.474 INFO:teuthology.orchestra.run.smithi077.stdout:============================================================================== 2022-08-01T19:30:49.475 INFO:teuthology.orchestra.run.smithi077.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-08-01T19:30:49.475 INFO:teuthology.orchestra.run.smithi077.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-08-01T19:30:49.475 INFO:teuthology.orchestra.run.smithi077.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-08-01T19:30:49.475 INFO:teuthology.orchestra.run.smithi077.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-08-01T19:30:49.477 INFO:teuthology.orchestra.run.smithi017.stdout: remote refid st t when poll reach delay offset jitter 2022-08-01T19:30:49.477 INFO:teuthology.orchestra.run.smithi017.stdout:============================================================================== 2022-08-01T19:30:49.478 INFO:teuthology.orchestra.run.smithi017.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-08-01T19:30:49.478 INFO:teuthology.orchestra.run.smithi017.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-08-01T19:30:49.478 INFO:teuthology.orchestra.run.smithi017.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-08-01T19:30:49.478 INFO:teuthology.orchestra.run.smithi017.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-08-01T19:30:50.425 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:50 ntpd[18231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:50.426 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:50 ntpd[18231]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:50.426 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:50 ntpd[18231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:50.426 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:50 ntpd[18231]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:50.427 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:50 ntpd[18231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:50.427 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:50 ntpd[18231]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:50.427 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:50 ntpd[18231]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T19:30:50.428 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:50 ntpd[18231]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T19:30:50.428 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:50 ntpd[18231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:50.428 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:50 ntpd[18231]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:50.428 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:50 ntpd[18231]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T19:30:50.428 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:50 ntpd[18231]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T19:30:51.425 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:51 ntpd[18231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:51.425 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:51 ntpd[18231]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:51.425 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:51 ntpd[18231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:51.426 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:51 ntpd[18231]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:51.426 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:51 ntpd[18231]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T19:30:51.426 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:51 ntpd[18231]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T19:30:52.425 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:52 ntpd[18231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:52.425 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:52 ntpd[18231]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:52.426 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:52 ntpd[18231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:52.426 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:52 ntpd[18231]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:52.426 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:52 ntpd[18231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-08-01T19:30:52.426 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:52 ntpd[18231]: can't open /var/log/ntpstats/rawstats.20220801: Permission denied 2022-08-01T19:30:52.427 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:52 ntpd[18231]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-08-01T19:30:52.427 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:52 ntpd[18231]: can't open /var/log/ntpstats/peerstats.20220801: Permission denied 2022-08-01T19:30:52.427 INFO:teuthology.orchestra.run.smithi085.stdout: 1 Aug 19:30:52 ntpd[18231]: ntpd: time slew -0.000959 s 2022-08-01T19:30:52.428 INFO:teuthology.orchestra.run.smithi085.stdout:ntpd: time slew -0.000959s 2022-08-01T19:30:52.428 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:52 ntpd[18231]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2022-08-01T19:30:52.429 INFO:teuthology.orchestra.run.smithi085.stderr: 1 Aug 19:30:52 ntpd[18231]: can't open /var/log/ntpstats/loopstats.20220801: Permission denied 2022-08-01T19:30:52.485 INFO:teuthology.orchestra.run.smithi085.stdout: remote refid st t when poll reach delay offset jitter 2022-08-01T19:30:52.485 INFO:teuthology.orchestra.run.smithi085.stdout:============================================================================== 2022-08-01T19:30:52.485 INFO:teuthology.orchestra.run.smithi085.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-08-01T19:30:52.485 INFO:teuthology.orchestra.run.smithi085.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-08-01T19:30:52.486 INFO:teuthology.orchestra.run.smithi085.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-08-01T19:30:52.486 INFO:teuthology.orchestra.run.smithi085.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-08-01T19:30:52.486 INFO:teuthology.run_tasks:Running task install... 2022-08-01T19:30:52.496 DEBUG:teuthology.task.install:project ceph 2022-08-01T19:30:52.496 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'fd96719b0e1200b56e7430604cfdd20ad281ca92'}} 2022-08-01T19:30:52.497 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': 'fd96719b0e1200b56e7430604cfdd20ad281ca92'} 2022-08-01T19:30:52.497 INFO:teuthology.task.install:Using flavor: default 2022-08-01T19:30:52.504 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2022-08-01T19:30:52.504 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']} 2022-08-01T19:30:52.505 DEBUG:teuthology.orchestra.run.smithi017:> sudo lsb_release -is 2022-08-01T19:30:52.557 INFO:teuthology.orchestra.run.smithi017.stdout:Ubuntu 2022-08-01T19:30:52.563 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-08-01T19:30:52.564 DEBUG:teuthology.orchestra.run.smithi077:> sudo lsb_release -is 2022-08-01T19:30:52.564 DEBUG:teuthology.orchestra.run.smithi017:> sudo apt-key list | grep Ceph 2022-08-01T19:30:52.613 INFO:teuthology.orchestra.run.smithi077.stdout:Ubuntu 2022-08-01T19:30:52.618 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-08-01T19:30:52.619 DEBUG:teuthology.orchestra.run.smithi085:> sudo lsb_release -is 2022-08-01T19:30:52.620 DEBUG:teuthology.orchestra.run.smithi077:> sudo apt-key list | grep Ceph 2022-08-01T19:30:52.663 INFO:teuthology.orchestra.run.smithi017.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2022-08-01T19:30:52.669 INFO:teuthology.orchestra.run.smithi085.stdout:Ubuntu 2022-08-01T19:30:52.675 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-08-01T19:30:52.675 DEBUG:teuthology.orchestra.run.smithi190:> sudo lsb_release -is 2022-08-01T19:30:52.676 DEBUG:teuthology.orchestra.run.smithi085:> sudo apt-key list | grep Ceph 2022-08-01T19:30:52.714 INFO:teuthology.orchestra.run.smithi077.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2022-08-01T19:30:52.726 INFO:teuthology.orchestra.run.smithi017.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2022-08-01T19:30:52.727 INFO:teuthology.orchestra.run.smithi017.stdout:uid [ unknown] Ceph.com (release key) 2022-08-01T19:30:52.727 INFO:teuthology.orchestra.run.smithi190.stdout:Ubuntu 2022-08-01T19:30:52.728 DEBUG:teuthology.orchestra.run.smithi017:> sudo lsb_release -is 2022-08-01T19:30:52.732 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-08-01T19:30:52.733 DEBUG:teuthology.orchestra.run.smithi190:> sudo apt-key list | grep Ceph 2022-08-01T19:30:52.771 INFO:teuthology.orchestra.run.smithi085.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2022-08-01T19:30:52.775 INFO:teuthology.orchestra.run.smithi077.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2022-08-01T19:30:52.775 INFO:teuthology.orchestra.run.smithi077.stdout:uid [ unknown] Ceph.com (release key) 2022-08-01T19:30:52.776 INFO:teuthology.orchestra.run.smithi017.stdout:Ubuntu 2022-08-01T19:30:52.777 DEBUG:teuthology.orchestra.run.smithi077:> sudo lsb_release -is 2022-08-01T19:30:52.781 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-08-01T19:30:52.781 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2022-08-01T19:30:52.782 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2022-08-01T19:30:52.782 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=fd96719b0e1200b56e7430604cfdd20ad281ca92 2022-08-01T19:30:52.827 INFO:teuthology.orchestra.run.smithi085.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2022-08-01T19:30:52.828 INFO:teuthology.orchestra.run.smithi085.stdout:uid [ unknown] Ceph.com (release key) 2022-08-01T19:30:52.829 INFO:teuthology.orchestra.run.smithi077.stdout:Ubuntu 2022-08-01T19:30:52.830 DEBUG:teuthology.orchestra.run.smithi085:> sudo lsb_release -is 2022-08-01T19:30:52.833 INFO:teuthology.orchestra.run.smithi190.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2022-08-01T19:30:52.834 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-08-01T19:30:52.835 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2022-08-01T19:30:52.835 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2022-08-01T19:30:52.835 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=fd96719b0e1200b56e7430604cfdd20ad281ca92 2022-08-01T19:30:52.877 INFO:teuthology.orchestra.run.smithi085.stdout:Ubuntu 2022-08-01T19:30:52.882 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-08-01T19:30:52.883 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2022-08-01T19:30:52.883 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2022-08-01T19:30:52.884 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=fd96719b0e1200b56e7430604cfdd20ad281ca92 2022-08-01T19:30:52.892 INFO:teuthology.orchestra.run.smithi190.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2022-08-01T19:30:52.892 INFO:teuthology.orchestra.run.smithi190.stdout:uid [ unknown] Ceph.com (release key) 2022-08-01T19:30:52.893 DEBUG:teuthology.orchestra.run.smithi190:> sudo lsb_release -is 2022-08-01T19:30:52.942 INFO:teuthology.orchestra.run.smithi190.stdout:Ubuntu 2022-08-01T19:30:52.948 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default/ 2022-08-01T19:30:52.948 INFO:teuthology.task.install.deb:Package version is 15.2.16-343-gfd96719b-1focal 2022-08-01T19:30:52.952 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-08-01T19:30:52.953 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2022-08-01T19:30:52.953 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2022-08-01T19:30:52.954 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=fd96719b0e1200b56e7430604cfdd20ad281ca92 2022-08-01T19:30:52.998 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default/ 2022-08-01T19:30:52.999 INFO:teuthology.task.install.deb:Package version is 15.2.16-343-gfd96719b-1focal 2022-08-01T19:30:53.063 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default/ 2022-08-01T19:30:53.064 INFO:teuthology.task.install.deb:Package version is 15.2.16-343-gfd96719b-1focal 2022-08-01T19:30:53.075 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2022-08-01T19:30:53.076 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2022-08-01T19:30:53.103 DEBUG:teuthology.orchestra.run.smithi017:> sudo apt-get update 2022-08-01T19:30:53.111 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default/ 2022-08-01T19:30:53.112 INFO:teuthology.task.install.deb:Package version is 15.2.16-343-gfd96719b-1focal 2022-08-01T19:30:53.117 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2022-08-01T19:30:53.117 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2022-08-01T19:30:53.135 DEBUG:teuthology.orchestra.run.smithi077:> sudo apt-get update 2022-08-01T19:30:53.182 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-08-01T19:30:53.183 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2022-08-01T19:30:53.200 DEBUG:teuthology.orchestra.run.smithi085:> sudo apt-get update 2022-08-01T19:30:53.227 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2022-08-01T19:30:53.227 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2022-08-01T19:30:53.244 DEBUG:teuthology.orchestra.run.smithi190:> sudo apt-get update 2022-08-01T19:30:53.312 INFO:teuthology.orchestra.run.smithi017.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-08-01T19:30:53.340 INFO:teuthology.orchestra.run.smithi085.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-08-01T19:30:53.340 INFO:teuthology.orchestra.run.smithi085.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-08-01T19:30:53.345 INFO:teuthology.orchestra.run.smithi085.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-08-01T19:30:53.354 INFO:teuthology.orchestra.run.smithi017.stdout:Ign:2 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal InRelease 2022-08-01T19:30:53.363 INFO:teuthology.orchestra.run.smithi085.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-08-01T19:30:53.377 INFO:teuthology.orchestra.run.smithi017.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal Release [7,680 B] 2022-08-01T19:30:53.384 INFO:teuthology.orchestra.run.smithi077.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal InRelease 2022-08-01T19:30:53.391 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-08-01T19:30:53.399 INFO:teuthology.orchestra.run.smithi085.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal InRelease 2022-08-01T19:30:53.402 INFO:teuthology.orchestra.run.smithi017.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal Release.gpg 2022-08-01T19:30:53.405 INFO:teuthology.orchestra.run.smithi017.stdout:Hit:5 http://archive.ubuntu.com/ubuntu focal InRelease 2022-08-01T19:30:53.407 INFO:teuthology.orchestra.run.smithi077.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal Release [7,680 B] 2022-08-01T19:30:53.422 INFO:teuthology.orchestra.run.smithi085.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal Release [7,680 B] 2022-08-01T19:30:53.425 INFO:teuthology.orchestra.run.smithi017.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main i386 Packages [3,063 B] 2022-08-01T19:30:53.432 INFO:teuthology.orchestra.run.smithi077.stdout:Ign:3 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal Release.gpg 2022-08-01T19:30:53.447 INFO:teuthology.orchestra.run.smithi077.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal InRelease 2022-08-01T19:30:53.447 INFO:teuthology.orchestra.run.smithi077.stdout:Hit:5 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-08-01T19:30:53.448 INFO:teuthology.orchestra.run.smithi190.stdout:Ign:2 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal InRelease 2022-08-01T19:30:53.448 INFO:teuthology.orchestra.run.smithi085.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal Release.gpg 2022-08-01T19:30:53.455 INFO:teuthology.orchestra.run.smithi077.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main i386 Packages [3,063 B] 2022-08-01T19:30:53.460 INFO:teuthology.orchestra.run.smithi017.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 Packages [15.8 kB] 2022-08-01T19:30:53.470 INFO:teuthology.orchestra.run.smithi190.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal Release [7,680 B] 2022-08-01T19:30:53.471 INFO:teuthology.orchestra.run.smithi085.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 Packages [15.8 kB] 2022-08-01T19:30:53.485 INFO:teuthology.orchestra.run.smithi017.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-08-01T19:30:53.489 INFO:teuthology.orchestra.run.smithi077.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 Packages [15.8 kB] 2022-08-01T19:30:53.494 INFO:teuthology.orchestra.run.smithi190.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal Release.gpg 2022-08-01T19:30:53.498 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:5 http://archive.ubuntu.com/ubuntu focal InRelease 2022-08-01T19:30:53.511 INFO:teuthology.orchestra.run.smithi085.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main i386 Packages [3,063 B] 2022-08-01T19:30:53.515 INFO:teuthology.orchestra.run.smithi077.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-08-01T19:30:53.517 INFO:teuthology.orchestra.run.smithi190.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 Packages [15.8 kB] 2022-08-01T19:30:53.555 INFO:teuthology.orchestra.run.smithi190.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main i386 Packages [3,063 B] 2022-08-01T19:30:53.566 INFO:teuthology.orchestra.run.smithi017.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-08-01T19:30:53.578 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-08-01T19:30:53.597 INFO:teuthology.orchestra.run.smithi077.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-08-01T19:30:53.658 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-08-01T19:30:53.987 INFO:teuthology.orchestra.run.smithi017.stdout:Fetched 26.6 kB in 1s (39.6 kB/s) 2022-08-01T19:30:54.008 INFO:teuthology.orchestra.run.smithi085.stdout:Fetched 26.6 kB in 1s (41.2 kB/s) 2022-08-01T19:30:54.035 INFO:teuthology.orchestra.run.smithi190.stdout:Fetched 26.6 kB in 1s (42.4 kB/s) 2022-08-01T19:30:54.095 INFO:teuthology.orchestra.run.smithi077.stdout:Fetched 26.6 kB in 1s (35.6 kB/s) 2022-08-01T19:30:54.890 INFO:teuthology.orchestra.run.smithi085.stdout:Reading package lists... 2022-08-01T19:30:54.906 INFO:teuthology.orchestra.run.smithi017.stdout:Reading package lists... 2022-08-01T19:30:54.913 DEBUG:teuthology.orchestra.run.smithi085:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.16-343-gfd96719b-1focal cephadm=15.2.16-343-gfd96719b-1focal ceph-mds=15.2.16-343-gfd96719b-1focal ceph-mgr=15.2.16-343-gfd96719b-1focal ceph-common=15.2.16-343-gfd96719b-1focal ceph-fuse=15.2.16-343-gfd96719b-1focal ceph-test=15.2.16-343-gfd96719b-1focal radosgw=15.2.16-343-gfd96719b-1focal python3-rados=15.2.16-343-gfd96719b-1focal python3-rgw=15.2.16-343-gfd96719b-1focal python3-cephfs=15.2.16-343-gfd96719b-1focal python3-rbd=15.2.16-343-gfd96719b-1focal libcephfs2=15.2.16-343-gfd96719b-1focal libcephfs-dev=15.2.16-343-gfd96719b-1focal librados2=15.2.16-343-gfd96719b-1focal librbd1=15.2.16-343-gfd96719b-1focal rbd-fuse=15.2.16-343-gfd96719b-1focal python3-cephfs=15.2.16-343-gfd96719b-1focal cephfs-shell=15.2.16-343-gfd96719b-1focal 2022-08-01T19:30:54.919 INFO:teuthology.orchestra.run.smithi190.stdout:Reading package lists... 2022-08-01T19:30:54.930 DEBUG:teuthology.orchestra.run.smithi017:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.16-343-gfd96719b-1focal cephadm=15.2.16-343-gfd96719b-1focal ceph-mds=15.2.16-343-gfd96719b-1focal ceph-mgr=15.2.16-343-gfd96719b-1focal ceph-common=15.2.16-343-gfd96719b-1focal ceph-fuse=15.2.16-343-gfd96719b-1focal ceph-test=15.2.16-343-gfd96719b-1focal radosgw=15.2.16-343-gfd96719b-1focal python3-rados=15.2.16-343-gfd96719b-1focal python3-rgw=15.2.16-343-gfd96719b-1focal python3-cephfs=15.2.16-343-gfd96719b-1focal python3-rbd=15.2.16-343-gfd96719b-1focal libcephfs2=15.2.16-343-gfd96719b-1focal libcephfs-dev=15.2.16-343-gfd96719b-1focal librados2=15.2.16-343-gfd96719b-1focal librbd1=15.2.16-343-gfd96719b-1focal rbd-fuse=15.2.16-343-gfd96719b-1focal python3-cephfs=15.2.16-343-gfd96719b-1focal cephfs-shell=15.2.16-343-gfd96719b-1focal 2022-08-01T19:30:54.942 DEBUG:teuthology.orchestra.run.smithi190:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.16-343-gfd96719b-1focal cephadm=15.2.16-343-gfd96719b-1focal ceph-mds=15.2.16-343-gfd96719b-1focal ceph-mgr=15.2.16-343-gfd96719b-1focal ceph-common=15.2.16-343-gfd96719b-1focal ceph-fuse=15.2.16-343-gfd96719b-1focal ceph-test=15.2.16-343-gfd96719b-1focal radosgw=15.2.16-343-gfd96719b-1focal python3-rados=15.2.16-343-gfd96719b-1focal python3-rgw=15.2.16-343-gfd96719b-1focal python3-cephfs=15.2.16-343-gfd96719b-1focal python3-rbd=15.2.16-343-gfd96719b-1focal libcephfs2=15.2.16-343-gfd96719b-1focal libcephfs-dev=15.2.16-343-gfd96719b-1focal librados2=15.2.16-343-gfd96719b-1focal librbd1=15.2.16-343-gfd96719b-1focal rbd-fuse=15.2.16-343-gfd96719b-1focal python3-cephfs=15.2.16-343-gfd96719b-1focal cephfs-shell=15.2.16-343-gfd96719b-1focal 2022-08-01T19:30:54.978 INFO:teuthology.orchestra.run.smithi085.stdout:Reading package lists... 2022-08-01T19:30:54.981 INFO:teuthology.orchestra.run.smithi077.stdout:Reading package lists... 2022-08-01T19:30:54.999 INFO:teuthology.orchestra.run.smithi017.stdout:Reading package lists... 2022-08-01T19:30:55.004 DEBUG:teuthology.orchestra.run.smithi077:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.16-343-gfd96719b-1focal cephadm=15.2.16-343-gfd96719b-1focal ceph-mds=15.2.16-343-gfd96719b-1focal ceph-mgr=15.2.16-343-gfd96719b-1focal ceph-common=15.2.16-343-gfd96719b-1focal ceph-fuse=15.2.16-343-gfd96719b-1focal ceph-test=15.2.16-343-gfd96719b-1focal radosgw=15.2.16-343-gfd96719b-1focal python3-rados=15.2.16-343-gfd96719b-1focal python3-rgw=15.2.16-343-gfd96719b-1focal python3-cephfs=15.2.16-343-gfd96719b-1focal python3-rbd=15.2.16-343-gfd96719b-1focal libcephfs2=15.2.16-343-gfd96719b-1focal libcephfs-dev=15.2.16-343-gfd96719b-1focal librados2=15.2.16-343-gfd96719b-1focal librbd1=15.2.16-343-gfd96719b-1focal rbd-fuse=15.2.16-343-gfd96719b-1focal python3-cephfs=15.2.16-343-gfd96719b-1focal cephfs-shell=15.2.16-343-gfd96719b-1focal 2022-08-01T19:30:55.007 INFO:teuthology.orchestra.run.smithi190.stdout:Reading package lists... 2022-08-01T19:30:55.075 INFO:teuthology.orchestra.run.smithi077.stdout:Reading package lists... 2022-08-01T19:30:55.155 INFO:teuthology.orchestra.run.smithi085.stdout:Building dependency tree... 2022-08-01T19:30:55.155 INFO:teuthology.orchestra.run.smithi085.stdout:Reading state information... 2022-08-01T19:30:55.180 INFO:teuthology.orchestra.run.smithi017.stdout:Building dependency tree... 2022-08-01T19:30:55.181 INFO:teuthology.orchestra.run.smithi017.stdout:Reading state information... 2022-08-01T19:30:55.185 INFO:teuthology.orchestra.run.smithi190.stdout:Building dependency tree... 2022-08-01T19:30:55.186 INFO:teuthology.orchestra.run.smithi190.stdout:Reading state information... 2022-08-01T19:30:55.245 INFO:teuthology.orchestra.run.smithi077.stdout:Building dependency tree... 2022-08-01T19:30:55.246 INFO:teuthology.orchestra.run.smithi077.stdout:Reading state information... 2022-08-01T19:30:55.310 INFO:teuthology.orchestra.run.smithi085.stdout:The following packages were automatically installed and are no longer required: 2022-08-01T19:30:55.312 INFO:teuthology.orchestra.run.smithi085.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-08-01T19:30:55.312 INFO:teuthology.orchestra.run.smithi085.stdout:Use 'sudo apt autoremove' to remove them. 2022-08-01T19:30:55.312 INFO:teuthology.orchestra.run.smithi085.stdout:The following additional packages will be installed: 2022-08-01T19:30:55.312 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-08-01T19:30:55.313 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-08-01T19:30:55.313 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mon ceph-osd jq libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2022-08-01T19:30:55.313 INFO:teuthology.orchestra.run.smithi085.stdout: libleveldb1d liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 2022-08-01T19:30:55.314 INFO:teuthology.orchestra.run.smithi085.stdout: librdkafka1 librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl 2022-08-01T19:30:55.314 INFO:teuthology.orchestra.run.smithi085.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2022-08-01T19:30:55.314 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2022-08-01T19:30:55.315 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2022-08-01T19:30:55.315 INFO:teuthology.orchestra.run.smithi085.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2022-08-01T19:30:55.315 INFO:teuthology.orchestra.run.smithi085.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2022-08-01T19:30:55.315 INFO:teuthology.orchestra.run.smithi085.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2022-08-01T19:30:55.315 INFO:teuthology.orchestra.run.smithi085.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2022-08-01T19:30:55.316 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2022-08-01T19:30:55.316 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2022-08-01T19:30:55.316 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2022-08-01T19:30:55.317 INFO:teuthology.orchestra.run.smithi085.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2022-08-01T19:30:55.317 INFO:teuthology.orchestra.run.smithi085.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2022-08-01T19:30:55.317 INFO:teuthology.orchestra.run.smithi085.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2022-08-01T19:30:55.317 INFO:teuthology.orchestra.run.smithi085.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug socat 2022-08-01T19:30:55.317 INFO:teuthology.orchestra.run.smithi085.stdout: xmlstarlet 2022-08-01T19:30:55.318 INFO:teuthology.orchestra.run.smithi085.stdout:Suggested packages: 2022-08-01T19:30:55.318 INFO:teuthology.orchestra.run.smithi085.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2022-08-01T19:30:55.319 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2022-08-01T19:30:55.319 INFO:teuthology.orchestra.run.smithi085.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2022-08-01T19:30:55.319 INFO:teuthology.orchestra.run.smithi085.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2022-08-01T19:30:55.319 INFO:teuthology.orchestra.run.smithi085.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2022-08-01T19:30:55.320 INFO:teuthology.orchestra.run.smithi085.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2022-08-01T19:30:55.320 INFO:teuthology.orchestra.run.smithi085.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2022-08-01T19:30:55.320 INFO:teuthology.orchestra.run.smithi085.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2022-08-01T19:30:55.320 INFO:teuthology.orchestra.run.smithi085.stdout:Recommended packages: 2022-08-01T19:30:55.321 INFO:teuthology.orchestra.run.smithi085.stdout: btrfs-tools 2022-08-01T19:30:55.334 INFO:teuthology.orchestra.run.smithi017.stdout:The following packages were automatically installed and are no longer required: 2022-08-01T19:30:55.337 INFO:teuthology.orchestra.run.smithi017.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-08-01T19:30:55.338 INFO:teuthology.orchestra.run.smithi017.stdout:Use 'sudo apt autoremove' to remove them. 2022-08-01T19:30:55.339 INFO:teuthology.orchestra.run.smithi017.stdout:The following additional packages will be installed: 2022-08-01T19:30:55.339 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-08-01T19:30:55.339 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-08-01T19:30:55.339 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mon ceph-osd jq libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2022-08-01T19:30:55.340 INFO:teuthology.orchestra.run.smithi017.stdout: libleveldb1d liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 2022-08-01T19:30:55.340 INFO:teuthology.orchestra.run.smithi017.stdout: librdkafka1 librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl 2022-08-01T19:30:55.341 INFO:teuthology.orchestra.run.smithi017.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2022-08-01T19:30:55.341 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2022-08-01T19:30:55.342 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2022-08-01T19:30:55.342 INFO:teuthology.orchestra.run.smithi017.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2022-08-01T19:30:55.342 INFO:teuthology.orchestra.run.smithi017.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2022-08-01T19:30:55.343 INFO:teuthology.orchestra.run.smithi017.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2022-08-01T19:30:55.343 INFO:teuthology.orchestra.run.smithi017.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2022-08-01T19:30:55.343 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2022-08-01T19:30:55.344 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2022-08-01T19:30:55.344 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2022-08-01T19:30:55.345 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2022-08-01T19:30:55.345 INFO:teuthology.orchestra.run.smithi017.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2022-08-01T19:30:55.345 INFO:teuthology.orchestra.run.smithi017.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2022-08-01T19:30:55.345 INFO:teuthology.orchestra.run.smithi017.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug socat 2022-08-01T19:30:55.346 INFO:teuthology.orchestra.run.smithi017.stdout: xmlstarlet 2022-08-01T19:30:55.346 INFO:teuthology.orchestra.run.smithi017.stdout:Suggested packages: 2022-08-01T19:30:55.346 INFO:teuthology.orchestra.run.smithi017.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2022-08-01T19:30:55.346 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2022-08-01T19:30:55.347 INFO:teuthology.orchestra.run.smithi017.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2022-08-01T19:30:55.347 INFO:teuthology.orchestra.run.smithi017.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2022-08-01T19:30:55.347 INFO:teuthology.orchestra.run.smithi017.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2022-08-01T19:30:55.347 INFO:teuthology.orchestra.run.smithi017.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2022-08-01T19:30:55.348 INFO:teuthology.orchestra.run.smithi017.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2022-08-01T19:30:55.348 INFO:teuthology.orchestra.run.smithi017.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2022-08-01T19:30:55.348 INFO:teuthology.orchestra.run.smithi017.stdout:Recommended packages: 2022-08-01T19:30:55.348 INFO:teuthology.orchestra.run.smithi017.stdout: btrfs-tools 2022-08-01T19:30:55.350 INFO:teuthology.orchestra.run.smithi190.stdout:The following packages were automatically installed and are no longer required: 2022-08-01T19:30:55.350 INFO:teuthology.orchestra.run.smithi190.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-08-01T19:30:55.350 INFO:teuthology.orchestra.run.smithi190.stdout:Use 'sudo apt autoremove' to remove them. 2022-08-01T19:30:55.351 INFO:teuthology.orchestra.run.smithi190.stdout:The following additional packages will be installed: 2022-08-01T19:30:55.351 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-08-01T19:30:55.351 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-08-01T19:30:55.351 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon ceph-osd jq libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2022-08-01T19:30:55.352 INFO:teuthology.orchestra.run.smithi190.stdout: libleveldb1d liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 2022-08-01T19:30:55.352 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka1 librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl 2022-08-01T19:30:55.352 INFO:teuthology.orchestra.run.smithi190.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2022-08-01T19:30:55.352 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2022-08-01T19:30:55.353 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2022-08-01T19:30:55.353 INFO:teuthology.orchestra.run.smithi190.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2022-08-01T19:30:55.354 INFO:teuthology.orchestra.run.smithi190.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2022-08-01T19:30:55.354 INFO:teuthology.orchestra.run.smithi190.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2022-08-01T19:30:55.355 INFO:teuthology.orchestra.run.smithi190.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2022-08-01T19:30:55.355 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2022-08-01T19:30:55.356 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2022-08-01T19:30:55.356 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2022-08-01T19:30:55.357 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2022-08-01T19:30:55.357 INFO:teuthology.orchestra.run.smithi190.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2022-08-01T19:30:55.358 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2022-08-01T19:30:55.358 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug socat 2022-08-01T19:30:55.359 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet 2022-08-01T19:30:55.359 INFO:teuthology.orchestra.run.smithi190.stdout:Suggested packages: 2022-08-01T19:30:55.360 INFO:teuthology.orchestra.run.smithi190.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2022-08-01T19:30:55.361 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2022-08-01T19:30:55.361 INFO:teuthology.orchestra.run.smithi190.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2022-08-01T19:30:55.362 INFO:teuthology.orchestra.run.smithi190.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2022-08-01T19:30:55.362 INFO:teuthology.orchestra.run.smithi190.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2022-08-01T19:30:55.363 INFO:teuthology.orchestra.run.smithi190.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2022-08-01T19:30:55.363 INFO:teuthology.orchestra.run.smithi190.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2022-08-01T19:30:55.364 INFO:teuthology.orchestra.run.smithi190.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2022-08-01T19:30:55.364 INFO:teuthology.orchestra.run.smithi190.stdout:Recommended packages: 2022-08-01T19:30:55.365 INFO:teuthology.orchestra.run.smithi190.stdout: btrfs-tools 2022-08-01T19:30:55.384 INFO:teuthology.orchestra.run.smithi077.stdout:The following packages were automatically installed and are no longer required: 2022-08-01T19:30:55.387 INFO:teuthology.orchestra.run.smithi077.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-08-01T19:30:55.388 INFO:teuthology.orchestra.run.smithi077.stdout:Use 'sudo apt autoremove' to remove them. 2022-08-01T19:30:55.388 INFO:teuthology.orchestra.run.smithi077.stdout:The following additional packages will be installed: 2022-08-01T19:30:55.388 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-08-01T19:30:55.388 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-08-01T19:30:55.389 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mon ceph-osd jq libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2022-08-01T19:30:55.389 INFO:teuthology.orchestra.run.smithi077.stdout: libleveldb1d liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 2022-08-01T19:30:55.390 INFO:teuthology.orchestra.run.smithi077.stdout: librdkafka1 librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl 2022-08-01T19:30:55.390 INFO:teuthology.orchestra.run.smithi077.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2022-08-01T19:30:55.391 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2022-08-01T19:30:55.391 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2022-08-01T19:30:55.392 INFO:teuthology.orchestra.run.smithi077.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2022-08-01T19:30:55.392 INFO:teuthology.orchestra.run.smithi077.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2022-08-01T19:30:55.393 INFO:teuthology.orchestra.run.smithi077.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2022-08-01T19:30:55.393 INFO:teuthology.orchestra.run.smithi077.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2022-08-01T19:30:55.394 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2022-08-01T19:30:55.395 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2022-08-01T19:30:55.395 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2022-08-01T19:30:55.396 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2022-08-01T19:30:55.396 INFO:teuthology.orchestra.run.smithi077.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2022-08-01T19:30:55.397 INFO:teuthology.orchestra.run.smithi077.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2022-08-01T19:30:55.397 INFO:teuthology.orchestra.run.smithi077.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug socat 2022-08-01T19:30:55.398 INFO:teuthology.orchestra.run.smithi077.stdout: xmlstarlet 2022-08-01T19:30:55.398 INFO:teuthology.orchestra.run.smithi077.stdout:Suggested packages: 2022-08-01T19:30:55.399 INFO:teuthology.orchestra.run.smithi077.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2022-08-01T19:30:55.399 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2022-08-01T19:30:55.400 INFO:teuthology.orchestra.run.smithi077.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2022-08-01T19:30:55.400 INFO:teuthology.orchestra.run.smithi077.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2022-08-01T19:30:55.401 INFO:teuthology.orchestra.run.smithi077.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2022-08-01T19:30:55.401 INFO:teuthology.orchestra.run.smithi077.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2022-08-01T19:30:55.402 INFO:teuthology.orchestra.run.smithi077.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2022-08-01T19:30:55.402 INFO:teuthology.orchestra.run.smithi077.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2022-08-01T19:30:55.403 INFO:teuthology.orchestra.run.smithi077.stdout:Recommended packages: 2022-08-01T19:30:55.403 INFO:teuthology.orchestra.run.smithi077.stdout: btrfs-tools 2022-08-01T19:30:55.417 INFO:teuthology.orchestra.run.smithi085.stdout:The following NEW packages will be installed: 2022-08-01T19:30:55.417 INFO:teuthology.orchestra.run.smithi085.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2022-08-01T19:30:55.417 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-08-01T19:30:55.418 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-08-01T19:30:55.418 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq libcephfs-dev libcephfs2 2022-08-01T19:30:55.418 INFO:teuthology.orchestra.run.smithi085.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2022-08-01T19:30:55.418 INFO:teuthology.orchestra.run.smithi085.stdout: liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 librdkafka1 2022-08-01T19:30:55.420 INFO:teuthology.orchestra.run.smithi085.stdout: librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl python3-atomicwrites 2022-08-01T19:30:55.420 INFO:teuthology.orchestra.run.smithi085.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2022-08-01T19:30:55.421 INFO:teuthology.orchestra.run.smithi085.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2022-08-01T19:30:55.421 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2022-08-01T19:30:55.421 INFO:teuthology.orchestra.run.smithi085.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2022-08-01T19:30:55.421 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2022-08-01T19:30:55.422 INFO:teuthology.orchestra.run.smithi085.stdout: python3-markupsafe python3-oauthlib python3-openssl python3-packaging 2022-08-01T19:30:55.422 INFO:teuthology.orchestra.run.smithi085.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2022-08-01T19:30:55.423 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2022-08-01T19:30:55.423 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2022-08-01T19:30:55.424 INFO:teuthology.orchestra.run.smithi085.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2022-08-01T19:30:55.424 INFO:teuthology.orchestra.run.smithi085.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2022-08-01T19:30:55.425 INFO:teuthology.orchestra.run.smithi085.stdout: python3-simplejson python3-singledispatch python3-sklearn 2022-08-01T19:30:55.425 INFO:teuthology.orchestra.run.smithi085.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2022-08-01T19:30:55.425 INFO:teuthology.orchestra.run.smithi085.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2022-08-01T19:30:55.425 INFO:teuthology.orchestra.run.smithi085.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2022-08-01T19:30:55.426 INFO:teuthology.orchestra.run.smithi085.stdout:The following packages will be upgraded: 2022-08-01T19:30:55.426 INFO:teuthology.orchestra.run.smithi085.stdout: librados2 librbd1 2022-08-01T19:30:55.447 INFO:teuthology.orchestra.run.smithi017.stdout:The following NEW packages will be installed: 2022-08-01T19:30:55.448 INFO:teuthology.orchestra.run.smithi017.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2022-08-01T19:30:55.448 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-08-01T19:30:55.449 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-08-01T19:30:55.450 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq libcephfs-dev libcephfs2 2022-08-01T19:30:55.451 INFO:teuthology.orchestra.run.smithi017.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2022-08-01T19:30:55.451 INFO:teuthology.orchestra.run.smithi017.stdout: liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 librdkafka1 2022-08-01T19:30:55.452 INFO:teuthology.orchestra.run.smithi017.stdout: librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl python3-atomicwrites 2022-08-01T19:30:55.452 INFO:teuthology.orchestra.run.smithi017.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2022-08-01T19:30:55.452 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2022-08-01T19:30:55.453 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2022-08-01T19:30:55.453 INFO:teuthology.orchestra.run.smithi017.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2022-08-01T19:30:55.454 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2022-08-01T19:30:55.454 INFO:teuthology.orchestra.run.smithi017.stdout: python3-markupsafe python3-oauthlib python3-openssl python3-packaging 2022-08-01T19:30:55.455 INFO:teuthology.orchestra.run.smithi017.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2022-08-01T19:30:55.455 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2022-08-01T19:30:55.456 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2022-08-01T19:30:55.456 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2022-08-01T19:30:55.457 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2022-08-01T19:30:55.457 INFO:teuthology.orchestra.run.smithi017.stdout: python3-simplejson python3-singledispatch python3-sklearn 2022-08-01T19:30:55.458 INFO:teuthology.orchestra.run.smithi017.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2022-08-01T19:30:55.458 INFO:teuthology.orchestra.run.smithi017.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2022-08-01T19:30:55.459 INFO:teuthology.orchestra.run.smithi017.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2022-08-01T19:30:55.459 INFO:teuthology.orchestra.run.smithi017.stdout:The following packages will be upgraded: 2022-08-01T19:30:55.460 INFO:teuthology.orchestra.run.smithi017.stdout: librados2 librbd1 2022-08-01T19:30:55.463 INFO:teuthology.orchestra.run.smithi190.stdout:The following NEW packages will be installed: 2022-08-01T19:30:55.463 INFO:teuthology.orchestra.run.smithi190.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2022-08-01T19:30:55.464 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-08-01T19:30:55.464 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-08-01T19:30:55.465 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq libcephfs-dev libcephfs2 2022-08-01T19:30:55.465 INFO:teuthology.orchestra.run.smithi190.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2022-08-01T19:30:55.466 INFO:teuthology.orchestra.run.smithi190.stdout: liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 librdkafka1 2022-08-01T19:30:55.466 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl python3-atomicwrites 2022-08-01T19:30:55.467 INFO:teuthology.orchestra.run.smithi190.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2022-08-01T19:30:55.467 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2022-08-01T19:30:55.467 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2022-08-01T19:30:55.468 INFO:teuthology.orchestra.run.smithi190.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2022-08-01T19:30:55.468 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2022-08-01T19:30:55.469 INFO:teuthology.orchestra.run.smithi190.stdout: python3-markupsafe python3-oauthlib python3-openssl python3-packaging 2022-08-01T19:30:55.469 INFO:teuthology.orchestra.run.smithi190.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2022-08-01T19:30:55.470 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2022-08-01T19:30:55.470 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2022-08-01T19:30:55.471 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2022-08-01T19:30:55.471 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2022-08-01T19:30:55.472 INFO:teuthology.orchestra.run.smithi190.stdout: python3-simplejson python3-singledispatch python3-sklearn 2022-08-01T19:30:55.472 INFO:teuthology.orchestra.run.smithi190.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2022-08-01T19:30:55.473 INFO:teuthology.orchestra.run.smithi190.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2022-08-01T19:30:55.473 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2022-08-01T19:30:55.474 INFO:teuthology.orchestra.run.smithi190.stdout:The following packages will be upgraded: 2022-08-01T19:30:55.474 INFO:teuthology.orchestra.run.smithi190.stdout: librados2 librbd1 2022-08-01T19:30:55.481 INFO:teuthology.orchestra.run.smithi085.stdout:2 upgraded, 101 newly installed, 0 to remove and 4 not upgraded. 2022-08-01T19:30:55.482 INFO:teuthology.orchestra.run.smithi085.stdout:Need to get 153 MB of archives. 2022-08-01T19:30:55.482 INFO:teuthology.orchestra.run.smithi085.stdout:After this operation, 699 MB of additional disk space will be used. 2022-08-01T19:30:55.483 INFO:teuthology.orchestra.run.smithi085.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2022-08-01T19:30:55.492 INFO:teuthology.orchestra.run.smithi077.stdout:The following NEW packages will be installed: 2022-08-01T19:30:55.492 INFO:teuthology.orchestra.run.smithi077.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2022-08-01T19:30:55.492 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-08-01T19:30:55.493 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-08-01T19:30:55.495 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq libcephfs-dev libcephfs2 2022-08-01T19:30:55.496 INFO:teuthology.orchestra.run.smithi077.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2022-08-01T19:30:55.496 INFO:teuthology.orchestra.run.smithi077.stdout: liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 librdkafka1 2022-08-01T19:30:55.496 INFO:teuthology.orchestra.run.smithi077.stdout: librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl python3-atomicwrites 2022-08-01T19:30:55.496 INFO:teuthology.orchestra.run.smithi077.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2022-08-01T19:30:55.497 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2022-08-01T19:30:55.497 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2022-08-01T19:30:55.497 INFO:teuthology.orchestra.run.smithi077.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2022-08-01T19:30:55.497 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2022-08-01T19:30:55.497 INFO:teuthology.orchestra.run.smithi077.stdout: python3-markupsafe python3-oauthlib python3-openssl python3-packaging 2022-08-01T19:30:55.498 INFO:teuthology.orchestra.run.smithi077.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2022-08-01T19:30:55.498 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2022-08-01T19:30:55.498 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2022-08-01T19:30:55.498 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2022-08-01T19:30:55.499 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2022-08-01T19:30:55.499 INFO:teuthology.orchestra.run.smithi077.stdout: python3-simplejson python3-singledispatch python3-sklearn 2022-08-01T19:30:55.499 INFO:teuthology.orchestra.run.smithi077.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2022-08-01T19:30:55.499 INFO:teuthology.orchestra.run.smithi077.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2022-08-01T19:30:55.500 INFO:teuthology.orchestra.run.smithi077.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2022-08-01T19:30:55.500 INFO:teuthology.orchestra.run.smithi077.stdout:The following packages will be upgraded: 2022-08-01T19:30:55.500 INFO:teuthology.orchestra.run.smithi077.stdout: librados2 librbd1 2022-08-01T19:30:55.501 INFO:teuthology.orchestra.run.smithi085.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2022-08-01T19:30:55.508 INFO:teuthology.orchestra.run.smithi085.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2022-08-01T19:30:55.512 INFO:teuthology.orchestra.run.smithi017.stdout:2 upgraded, 101 newly installed, 0 to remove and 4 not upgraded. 2022-08-01T19:30:55.512 INFO:teuthology.orchestra.run.smithi017.stdout:Need to get 153 MB of archives. 2022-08-01T19:30:55.512 INFO:teuthology.orchestra.run.smithi017.stdout:After this operation, 699 MB of additional disk space will be used. 2022-08-01T19:30:55.512 INFO:teuthology.orchestra.run.smithi017.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2022-08-01T19:30:55.530 INFO:teuthology.orchestra.run.smithi017.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2022-08-01T19:30:55.537 INFO:teuthology.orchestra.run.smithi085.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2022-08-01T19:30:55.538 INFO:teuthology.orchestra.run.smithi017.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2022-08-01T19:30:55.566 INFO:teuthology.orchestra.run.smithi017.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2022-08-01T19:30:55.574 INFO:teuthology.orchestra.run.smithi085.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 15.2.16-343-gfd96719b-1focal [1,698 kB] 2022-08-01T19:30:55.589 INFO:teuthology.orchestra.run.smithi085.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2022-08-01T19:30:55.591 INFO:teuthology.orchestra.run.smithi085.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2022-08-01T19:30:55.594 INFO:teuthology.orchestra.run.smithi085.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2022-08-01T19:30:55.604 INFO:teuthology.orchestra.run.smithi017.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 15.2.16-343-gfd96719b-1focal [1,698 kB] 2022-08-01T19:30:55.607 INFO:teuthology.orchestra.run.smithi190.stdout:2 upgraded, 101 newly installed, 0 to remove and 4 not upgraded. 2022-08-01T19:30:55.607 INFO:teuthology.orchestra.run.smithi190.stdout:Need to get 153 MB of archives. 2022-08-01T19:30:55.607 INFO:teuthology.orchestra.run.smithi190.stdout:After this operation, 699 MB of additional disk space will be used. 2022-08-01T19:30:55.607 INFO:teuthology.orchestra.run.smithi190.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 15.2.16-343-gfd96719b-1focal [1,698 kB] 2022-08-01T19:30:55.616 INFO:teuthology.orchestra.run.smithi017.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2022-08-01T19:30:55.617 INFO:teuthology.orchestra.run.smithi017.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2022-08-01T19:30:55.619 INFO:teuthology.orchestra.run.smithi017.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2022-08-01T19:30:55.642 INFO:teuthology.orchestra.run.smithi190.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2022-08-01T19:30:55.653 INFO:teuthology.orchestra.run.smithi077.stdout:2 upgraded, 101 newly installed, 0 to remove and 4 not upgraded. 2022-08-01T19:30:55.653 INFO:teuthology.orchestra.run.smithi077.stdout:Need to get 153 MB of archives. 2022-08-01T19:30:55.654 INFO:teuthology.orchestra.run.smithi077.stdout:After this operation, 699 MB of additional disk space will be used. 2022-08-01T19:30:55.654 INFO:teuthology.orchestra.run.smithi077.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 15.2.16-343-gfd96719b-1focal [1,698 kB] 2022-08-01T19:30:55.678 INFO:teuthology.orchestra.run.smithi085.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2022-08-01T19:30:55.683 INFO:teuthology.orchestra.run.smithi077.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2022-08-01T19:30:55.700 INFO:teuthology.orchestra.run.smithi085.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2022-08-01T19:30:55.703 INFO:teuthology.orchestra.run.smithi017.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2022-08-01T19:30:55.716 INFO:teuthology.orchestra.run.smithi085.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2022-08-01T19:30:55.724 INFO:teuthology.orchestra.run.smithi190.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2022-08-01T19:30:55.725 INFO:teuthology.orchestra.run.smithi017.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2022-08-01T19:30:55.741 INFO:teuthology.orchestra.run.smithi017.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2022-08-01T19:30:55.744 INFO:teuthology.orchestra.run.smithi085.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2022-08-01T19:30:55.757 INFO:teuthology.orchestra.run.smithi190.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2022-08-01T19:30:55.759 INFO:teuthology.orchestra.run.smithi085.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2022-08-01T19:30:55.762 INFO:teuthology.orchestra.run.smithi085.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2022-08-01T19:30:55.765 INFO:teuthology.orchestra.run.smithi077.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2022-08-01T19:30:55.782 INFO:teuthology.orchestra.run.smithi017.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2022-08-01T19:30:55.783 INFO:teuthology.orchestra.run.smithi017.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2022-08-01T19:30:55.784 INFO:teuthology.orchestra.run.smithi085.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2022-08-01T19:30:55.786 INFO:teuthology.orchestra.run.smithi017.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2022-08-01T19:30:55.786 INFO:teuthology.orchestra.run.smithi085.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2022-08-01T19:30:55.802 INFO:teuthology.orchestra.run.smithi077.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2022-08-01T19:30:55.803 INFO:teuthology.orchestra.run.smithi085.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2022-08-01T19:30:55.803 INFO:teuthology.orchestra.run.smithi085.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2 [59.1 kB] 2022-08-01T19:30:55.805 INFO:teuthology.orchestra.run.smithi085.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2022-08-01T19:30:55.806 INFO:teuthology.orchestra.run.smithi085.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2022-08-01T19:30:55.807 INFO:teuthology.orchestra.run.smithi085.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2022-08-01T19:30:55.809 INFO:teuthology.orchestra.run.smithi017.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2022-08-01T19:30:55.810 INFO:teuthology.orchestra.run.smithi017.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2022-08-01T19:30:55.810 INFO:teuthology.orchestra.run.smithi017.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2022-08-01T19:30:55.815 INFO:teuthology.orchestra.run.smithi085.stdout:Get:22 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 15.2.16-343-gfd96719b-1focal [3,259 kB] 2022-08-01T19:30:55.826 INFO:teuthology.orchestra.run.smithi085.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2022-08-01T19:30:55.826 INFO:teuthology.orchestra.run.smithi017.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2 [59.1 kB] 2022-08-01T19:30:55.827 INFO:teuthology.orchestra.run.smithi085.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2022-08-01T19:30:55.828 INFO:teuthology.orchestra.run.smithi017.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2022-08-01T19:30:55.829 INFO:teuthology.orchestra.run.smithi017.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2022-08-01T19:30:55.830 INFO:teuthology.orchestra.run.smithi017.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2022-08-01T19:30:55.847 INFO:teuthology.orchestra.run.smithi085.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2022-08-01T19:30:55.848 INFO:teuthology.orchestra.run.smithi017.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2022-08-01T19:30:55.850 INFO:teuthology.orchestra.run.smithi017.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2022-08-01T19:30:55.850 INFO:teuthology.orchestra.run.smithi017.stdout:Get:24 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 15.2.16-343-gfd96719b-1focal [3,259 kB] 2022-08-01T19:30:55.851 INFO:teuthology.orchestra.run.smithi085.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2022-08-01T19:30:55.864 INFO:teuthology.orchestra.run.smithi190.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 15.2.16-343-gfd96719b-1focal [3,259 kB] 2022-08-01T19:30:55.867 INFO:teuthology.orchestra.run.smithi085.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2022-08-01T19:30:55.868 INFO:teuthology.orchestra.run.smithi017.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2022-08-01T19:30:55.872 INFO:teuthology.orchestra.run.smithi017.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2022-08-01T19:30:55.873 INFO:teuthology.orchestra.run.smithi017.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2022-08-01T19:30:55.880 INFO:teuthology.orchestra.run.smithi190.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2022-08-01T19:30:55.935 INFO:teuthology.orchestra.run.smithi077.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2022-08-01T19:30:55.952 INFO:teuthology.orchestra.run.smithi085.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2022-08-01T19:30:55.952 INFO:teuthology.orchestra.run.smithi085.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2022-08-01T19:30:55.953 INFO:teuthology.orchestra.run.smithi017.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2022-08-01T19:30:55.954 INFO:teuthology.orchestra.run.smithi085.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2022-08-01T19:30:55.955 INFO:teuthology.orchestra.run.smithi017.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2022-08-01T19:30:55.956 INFO:teuthology.orchestra.run.smithi017.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2022-08-01T19:30:55.956 INFO:teuthology.orchestra.run.smithi017.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2022-08-01T19:30:55.957 INFO:teuthology.orchestra.run.smithi085.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2022-08-01T19:30:55.974 INFO:teuthology.orchestra.run.smithi017.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2022-08-01T19:30:55.975 INFO:teuthology.orchestra.run.smithi085.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2022-08-01T19:30:55.996 INFO:teuthology.orchestra.run.smithi077.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 15.2.16-343-gfd96719b-1focal [3,259 kB] 2022-08-01T19:30:55.997 INFO:teuthology.orchestra.run.smithi017.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2022-08-01T19:30:55.998 INFO:teuthology.orchestra.run.smithi085.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2022-08-01T19:30:56.015 INFO:teuthology.orchestra.run.smithi017.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2022-08-01T19:30:56.016 INFO:teuthology.orchestra.run.smithi017.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2022-08-01T19:30:56.018 INFO:teuthology.orchestra.run.smithi085.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2022-08-01T19:30:56.019 INFO:teuthology.orchestra.run.smithi085.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2022-08-01T19:30:56.020 INFO:teuthology.orchestra.run.smithi017.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2022-08-01T19:30:56.038 INFO:teuthology.orchestra.run.smithi017.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2022-08-01T19:30:56.041 INFO:teuthology.orchestra.run.smithi085.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2022-08-01T19:30:56.056 INFO:teuthology.orchestra.run.smithi017.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2022-08-01T19:30:56.057 INFO:teuthology.orchestra.run.smithi017.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2022-08-01T19:30:56.058 INFO:teuthology.orchestra.run.smithi017.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2022-08-01T19:30:56.059 INFO:teuthology.orchestra.run.smithi085.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2022-08-01T19:30:56.064 INFO:teuthology.orchestra.run.smithi085.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2022-08-01T19:30:56.064 INFO:teuthology.orchestra.run.smithi085.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2022-08-01T19:30:56.081 INFO:teuthology.orchestra.run.smithi085.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2022-08-01T19:30:56.131 INFO:teuthology.orchestra.run.smithi190.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2022-08-01T19:30:56.141 INFO:teuthology.orchestra.run.smithi190.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2022-08-01T19:30:56.155 INFO:teuthology.orchestra.run.smithi190.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2022-08-01T19:30:56.173 INFO:teuthology.orchestra.run.smithi085.stdout:Get:41 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 15.2.16-343-gfd96719b-1focal [463 kB] 2022-08-01T19:30:56.183 INFO:teuthology.orchestra.run.smithi077.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2022-08-01T19:30:56.193 INFO:teuthology.orchestra.run.smithi077.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2022-08-01T19:30:56.232 INFO:teuthology.orchestra.run.smithi085.stdout:Get:42 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 15.2.16-343-gfd96719b-1focal [365 kB] 2022-08-01T19:30:56.243 INFO:teuthology.orchestra.run.smithi017.stdout:Get:41 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 15.2.16-343-gfd96719b-1focal [463 kB] 2022-08-01T19:30:56.259 INFO:teuthology.orchestra.run.smithi190.stdout:Get:10 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 15.2.16-343-gfd96719b-1focal [463 kB] 2022-08-01T19:30:56.275 INFO:teuthology.orchestra.run.smithi077.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2022-08-01T19:30:56.286 INFO:teuthology.orchestra.run.smithi085.stdout:Get:43 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 15.2.16-343-gfd96719b-1focal [20.9 kB] 2022-08-01T19:30:56.287 INFO:teuthology.orchestra.run.smithi085.stdout:Get:44 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 15.2.16-343-gfd96719b-1focal [156 kB] 2022-08-01T19:30:56.292 INFO:teuthology.orchestra.run.smithi085.stdout:Get:45 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 15.2.16-343-gfd96719b-1focal [29.4 kB] 2022-08-01T19:30:56.293 INFO:teuthology.orchestra.run.smithi085.stdout:Get:46 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 15.2.16-343-gfd96719b-1focal [362 kB] 2022-08-01T19:30:56.302 INFO:teuthology.orchestra.run.smithi017.stdout:Get:42 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 15.2.16-343-gfd96719b-1focal [365 kB] 2022-08-01T19:30:56.314 INFO:teuthology.orchestra.run.smithi017.stdout:Get:43 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 15.2.16-343-gfd96719b-1focal [20.9 kB] 2022-08-01T19:30:56.315 INFO:teuthology.orchestra.run.smithi017.stdout:Get:44 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 15.2.16-343-gfd96719b-1focal [156 kB] 2022-08-01T19:30:56.320 INFO:teuthology.orchestra.run.smithi190.stdout:Get:11 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 15.2.16-343-gfd96719b-1focal [365 kB] 2022-08-01T19:30:56.343 INFO:teuthology.orchestra.run.smithi190.stdout:Get:12 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 15.2.16-343-gfd96719b-1focal [20.9 kB] 2022-08-01T19:30:56.345 INFO:teuthology.orchestra.run.smithi190.stdout:Get:13 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 15.2.16-343-gfd96719b-1focal [156 kB] 2022-08-01T19:30:56.351 INFO:teuthology.orchestra.run.smithi085.stdout:Get:47 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 15.2.16-343-gfd96719b-1focal [3,148 kB] 2022-08-01T19:30:56.367 INFO:teuthology.orchestra.run.smithi017.stdout:Get:45 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 15.2.16-343-gfd96719b-1focal [29.4 kB] 2022-08-01T19:30:56.369 INFO:teuthology.orchestra.run.smithi017.stdout:Get:46 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 15.2.16-343-gfd96719b-1focal [362 kB] 2022-08-01T19:30:56.386 INFO:teuthology.orchestra.run.smithi017.stdout:Get:47 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 15.2.16-343-gfd96719b-1focal [3,148 kB] 2022-08-01T19:30:56.387 INFO:teuthology.orchestra.run.smithi190.stdout:Get:14 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 15.2.16-343-gfd96719b-1focal [29.4 kB] 2022-08-01T19:30:56.387 INFO:teuthology.orchestra.run.smithi190.stdout:Get:15 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 15.2.16-343-gfd96719b-1focal [362 kB] 2022-08-01T19:30:56.410 INFO:teuthology.orchestra.run.smithi190.stdout:Get:16 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 15.2.16-343-gfd96719b-1focal [3,148 kB] 2022-08-01T19:30:56.548 INFO:teuthology.orchestra.run.smithi077.stdout:Get:10 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 15.2.16-343-gfd96719b-1focal [463 kB] 2022-08-01T19:30:56.625 INFO:teuthology.orchestra.run.smithi077.stdout:Get:11 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 15.2.16-343-gfd96719b-1focal [365 kB] 2022-08-01T19:30:56.703 INFO:teuthology.orchestra.run.smithi077.stdout:Get:12 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 15.2.16-343-gfd96719b-1focal [20.9 kB] 2022-08-01T19:30:56.704 INFO:teuthology.orchestra.run.smithi077.stdout:Get:13 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 15.2.16-343-gfd96719b-1focal [156 kB] 2022-08-01T19:30:56.718 INFO:teuthology.orchestra.run.smithi190.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2022-08-01T19:30:56.723 INFO:teuthology.orchestra.run.smithi085.stdout:Get:48 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 15.2.16-343-gfd96719b-1focal [116 kB] 2022-08-01T19:30:56.726 INFO:teuthology.orchestra.run.smithi085.stdout:Get:49 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 15.2.16-343-gfd96719b-1focal [361 kB] 2022-08-01T19:30:56.773 INFO:teuthology.orchestra.run.smithi077.stdout:Get:14 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 15.2.16-343-gfd96719b-1focal [29.4 kB] 2022-08-01T19:30:56.776 INFO:teuthology.orchestra.run.smithi077.stdout:Get:15 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 15.2.16-343-gfd96719b-1focal [362 kB] 2022-08-01T19:30:56.778 INFO:teuthology.orchestra.run.smithi077.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2022-08-01T19:30:56.840 INFO:teuthology.orchestra.run.smithi017.stdout:Get:48 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 15.2.16-343-gfd96719b-1focal [116 kB] 2022-08-01T19:30:56.855 INFO:teuthology.orchestra.run.smithi017.stdout:Get:49 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 15.2.16-343-gfd96719b-1focal [361 kB] 2022-08-01T19:30:56.867 INFO:teuthology.orchestra.run.smithi077.stdout:Get:17 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 15.2.16-343-gfd96719b-1focal [3,148 kB] 2022-08-01T19:30:56.871 INFO:teuthology.orchestra.run.smithi190.stdout:Get:18 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 15.2.16-343-gfd96719b-1focal [116 kB] 2022-08-01T19:30:56.873 INFO:teuthology.orchestra.run.smithi085.stdout:Get:50 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 15.2.16-343-gfd96719b-1focal [20.1 MB] 2022-08-01T19:30:56.881 INFO:teuthology.orchestra.run.smithi190.stdout:Get:19 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 15.2.16-343-gfd96719b-1focal [361 kB] 2022-08-01T19:30:56.886 INFO:teuthology.orchestra.run.smithi190.stdout:Get:20 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 15.2.16-343-gfd96719b-1focal [20.1 MB] 2022-08-01T19:30:56.896 INFO:teuthology.orchestra.run.smithi017.stdout:Get:50 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 15.2.16-343-gfd96719b-1focal [20.1 MB] 2022-08-01T19:30:56.906 INFO:teuthology.orchestra.run.smithi190.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2022-08-01T19:30:56.924 INFO:teuthology.orchestra.run.smithi190.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2022-08-01T19:30:56.968 INFO:teuthology.orchestra.run.smithi077.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2022-08-01T19:30:57.055 INFO:teuthology.orchestra.run.smithi077.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2022-08-01T19:30:57.207 INFO:teuthology.orchestra.run.smithi190.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2022-08-01T19:30:57.275 INFO:teuthology.orchestra.run.smithi077.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2022-08-01T19:30:57.291 INFO:teuthology.orchestra.run.smithi190.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2022-08-01T19:30:57.303 INFO:teuthology.orchestra.run.smithi190.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2022-08-01T19:30:57.360 INFO:teuthology.orchestra.run.smithi077.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2022-08-01T19:30:57.363 INFO:teuthology.orchestra.run.smithi077.stdout:Get:22 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 15.2.16-343-gfd96719b-1focal [116 kB] 2022-08-01T19:30:57.366 INFO:teuthology.orchestra.run.smithi077.stdout:Get:23 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 15.2.16-343-gfd96719b-1focal [361 kB] 2022-08-01T19:30:57.372 INFO:teuthology.orchestra.run.smithi077.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2022-08-01T19:30:57.442 INFO:teuthology.orchestra.run.smithi077.stdout:Get:25 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 15.2.16-343-gfd96719b-1focal [20.1 MB] 2022-08-01T19:30:57.501 INFO:teuthology.orchestra.run.smithi190.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2022-08-01T19:30:57.573 INFO:teuthology.orchestra.run.smithi190.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2022-08-01T19:30:57.581 INFO:teuthology.orchestra.run.smithi190.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2022-08-01T19:30:57.587 INFO:teuthology.orchestra.run.smithi190.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2 [59.1 kB] 2022-08-01T19:30:57.640 INFO:teuthology.orchestra.run.smithi077.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2022-08-01T19:30:57.647 INFO:teuthology.orchestra.run.smithi077.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2022-08-01T19:30:57.654 INFO:teuthology.orchestra.run.smithi077.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2022-08-01T19:30:57.660 INFO:teuthology.orchestra.run.smithi077.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2 [59.1 kB] 2022-08-01T19:30:57.676 INFO:teuthology.orchestra.run.smithi190.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2022-08-01T19:30:57.681 INFO:teuthology.orchestra.run.smithi190.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2022-08-01T19:30:57.684 INFO:teuthology.orchestra.run.smithi190.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2022-08-01T19:30:57.752 INFO:teuthology.orchestra.run.smithi077.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2022-08-01T19:30:57.756 INFO:teuthology.orchestra.run.smithi077.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2022-08-01T19:30:57.759 INFO:teuthology.orchestra.run.smithi077.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2022-08-01T19:30:57.773 INFO:teuthology.orchestra.run.smithi017.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2022-08-01T19:30:57.784 INFO:teuthology.orchestra.run.smithi190.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2022-08-01T19:30:57.861 INFO:teuthology.orchestra.run.smithi077.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2022-08-01T19:30:57.862 INFO:teuthology.orchestra.run.smithi190.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2022-08-01T19:30:57.940 INFO:teuthology.orchestra.run.smithi077.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2022-08-01T19:30:57.962 INFO:teuthology.orchestra.run.smithi190.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2022-08-01T19:30:58.039 INFO:teuthology.orchestra.run.smithi085.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2022-08-01T19:30:58.041 INFO:teuthology.orchestra.run.smithi077.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2022-08-01T19:30:58.061 INFO:teuthology.orchestra.run.smithi190.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2022-08-01T19:30:58.068 INFO:teuthology.orchestra.run.smithi190.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2022-08-01T19:30:58.102 INFO:teuthology.orchestra.run.smithi017.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2022-08-01T19:30:58.139 INFO:teuthology.orchestra.run.smithi017.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2022-08-01T19:30:58.142 INFO:teuthology.orchestra.run.smithi077.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2022-08-01T19:30:58.148 INFO:teuthology.orchestra.run.smithi077.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2022-08-01T19:30:58.404 INFO:teuthology.orchestra.run.smithi017.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2022-08-01T19:30:58.405 INFO:teuthology.orchestra.run.smithi017.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2022-08-01T19:30:58.405 INFO:teuthology.orchestra.run.smithi017.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2022-08-01T19:30:58.405 INFO:teuthology.orchestra.run.smithi017.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2022-08-01T19:30:58.417 INFO:teuthology.orchestra.run.smithi017.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2022-08-01T19:30:58.418 INFO:teuthology.orchestra.run.smithi017.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2022-08-01T19:30:58.419 INFO:teuthology.orchestra.run.smithi017.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2022-08-01T19:30:58.437 INFO:teuthology.orchestra.run.smithi017.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2022-08-01T19:30:58.438 INFO:teuthology.orchestra.run.smithi017.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2022-08-01T19:30:58.448 INFO:teuthology.orchestra.run.smithi085.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2022-08-01T19:30:58.454 INFO:teuthology.orchestra.run.smithi017.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2022-08-01T19:30:58.485 INFO:teuthology.orchestra.run.smithi085.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2022-08-01T19:30:58.494 INFO:teuthology.orchestra.run.smithi017.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2022-08-01T19:30:58.500 INFO:teuthology.orchestra.run.smithi017.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2022-08-01T19:30:58.519 INFO:teuthology.orchestra.run.smithi017.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2022-08-01T19:30:58.535 INFO:teuthology.orchestra.run.smithi017.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2022-08-01T19:30:58.577 INFO:teuthology.orchestra.run.smithi017.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 libxslt1.1 amd64 1.1.34-4 [152 kB] 2022-08-01T19:30:58.598 INFO:teuthology.orchestra.run.smithi017.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2022-08-01T19:30:58.649 INFO:teuthology.orchestra.run.smithi190.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2022-08-01T19:30:58.651 INFO:teuthology.orchestra.run.smithi190.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2022-08-01T19:30:58.652 INFO:teuthology.orchestra.run.smithi017.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2022-08-01T19:30:58.653 INFO:teuthology.orchestra.run.smithi017.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2022-08-01T19:30:58.694 INFO:teuthology.orchestra.run.smithi017.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2022-08-01T19:30:58.709 INFO:teuthology.orchestra.run.smithi017.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2022-08-01T19:30:58.710 INFO:teuthology.orchestra.run.smithi017.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2022-08-01T19:30:58.711 INFO:teuthology.orchestra.run.smithi017.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2022-08-01T19:30:58.714 INFO:teuthology.orchestra.run.smithi017.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2022-08-01T19:30:58.727 INFO:teuthology.orchestra.run.smithi190.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2022-08-01T19:30:58.738 INFO:teuthology.orchestra.run.smithi077.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2022-08-01T19:30:58.738 INFO:teuthology.orchestra.run.smithi190.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2022-08-01T19:30:58.739 INFO:teuthology.orchestra.run.smithi077.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2022-08-01T19:30:58.780 INFO:teuthology.orchestra.run.smithi085.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2022-08-01T19:30:58.781 INFO:teuthology.orchestra.run.smithi085.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2022-08-01T19:30:58.782 INFO:teuthology.orchestra.run.smithi085.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2022-08-01T19:30:58.798 INFO:teuthology.orchestra.run.smithi085.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2022-08-01T19:30:58.799 INFO:teuthology.orchestra.run.smithi085.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2022-08-01T19:30:58.801 INFO:teuthology.orchestra.run.smithi085.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2022-08-01T19:30:58.802 INFO:teuthology.orchestra.run.smithi085.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2022-08-01T19:30:58.817 INFO:teuthology.orchestra.run.smithi077.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2022-08-01T19:30:58.821 INFO:teuthology.orchestra.run.smithi085.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2022-08-01T19:30:58.822 INFO:teuthology.orchestra.run.smithi085.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2022-08-01T19:30:58.825 INFO:teuthology.orchestra.run.smithi085.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2022-08-01T19:30:58.829 INFO:teuthology.orchestra.run.smithi077.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2022-08-01T19:30:58.841 INFO:teuthology.orchestra.run.smithi190.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2022-08-01T19:30:58.867 INFO:teuthology.orchestra.run.smithi085.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2022-08-01T19:30:58.888 INFO:teuthology.orchestra.run.smithi085.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2022-08-01T19:30:58.908 INFO:teuthology.orchestra.run.smithi085.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2022-08-01T19:30:58.911 INFO:teuthology.orchestra.run.smithi085.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2022-08-01T19:30:58.927 INFO:teuthology.orchestra.run.smithi017.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2022-08-01T19:30:58.928 INFO:teuthology.orchestra.run.smithi017.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2022-08-01T19:30:58.930 INFO:teuthology.orchestra.run.smithi017.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2022-08-01T19:30:58.931 INFO:teuthology.orchestra.run.smithi017.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2022-08-01T19:30:58.931 INFO:teuthology.orchestra.run.smithi077.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2022-08-01T19:30:58.954 INFO:teuthology.orchestra.run.smithi017.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2022-08-01T19:30:58.970 INFO:teuthology.orchestra.run.smithi085.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 libxslt1.1 amd64 1.1.34-4 [152 kB] 2022-08-01T19:30:58.971 INFO:teuthology.orchestra.run.smithi017.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2022-08-01T19:30:58.972 INFO:teuthology.orchestra.run.smithi017.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2022-08-01T19:30:58.992 INFO:teuthology.orchestra.run.smithi085.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2022-08-01T19:30:58.993 INFO:teuthology.orchestra.run.smithi017.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2022-08-01T19:30:59.033 INFO:teuthology.orchestra.run.smithi085.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2022-08-01T19:30:59.035 INFO:teuthology.orchestra.run.smithi085.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2022-08-01T19:30:59.077 INFO:teuthology.orchestra.run.smithi085.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2022-08-01T19:30:59.078 INFO:teuthology.orchestra.run.smithi085.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2022-08-01T19:30:59.079 INFO:teuthology.orchestra.run.smithi085.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2022-08-01T19:30:59.094 INFO:teuthology.orchestra.run.smithi085.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2022-08-01T19:30:59.097 INFO:teuthology.orchestra.run.smithi085.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2022-08-01T19:30:59.112 INFO:teuthology.orchestra.run.smithi190.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2022-08-01T19:30:59.207 INFO:teuthology.orchestra.run.smithi077.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2022-08-01T19:30:59.217 INFO:teuthology.orchestra.run.smithi190.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2022-08-01T19:30:59.217 INFO:teuthology.orchestra.run.smithi085.stdout:Get:77 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 15.2.16-343-gfd96719b-1focal [5,346 kB] 2022-08-01T19:30:59.222 INFO:teuthology.orchestra.run.smithi190.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2022-08-01T19:30:59.310 INFO:teuthology.orchestra.run.smithi077.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2022-08-01T19:30:59.315 INFO:teuthology.orchestra.run.smithi077.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2022-08-01T19:30:59.341 INFO:teuthology.orchestra.run.smithi085.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2022-08-01T19:30:59.342 INFO:teuthology.orchestra.run.smithi085.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2022-08-01T19:30:59.344 INFO:teuthology.orchestra.run.smithi085.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2022-08-01T19:30:59.345 INFO:teuthology.orchestra.run.smithi085.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2022-08-01T19:30:59.357 INFO:teuthology.orchestra.run.smithi077.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2022-08-01T19:30:59.366 INFO:teuthology.orchestra.run.smithi085.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2022-08-01T19:30:59.383 INFO:teuthology.orchestra.run.smithi017.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 15.2.16-343-gfd96719b-1focal [5,346 kB] 2022-08-01T19:30:59.384 INFO:teuthology.orchestra.run.smithi085.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2022-08-01T19:30:59.385 INFO:teuthology.orchestra.run.smithi085.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2022-08-01T19:30:59.389 INFO:teuthology.orchestra.run.smithi190.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2022-08-01T19:30:59.390 INFO:teuthology.orchestra.run.smithi190.stdout:Get:47 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 15.2.16-343-gfd96719b-1focal [5,346 kB] 2022-08-01T19:30:59.398 INFO:teuthology.orchestra.run.smithi077.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2022-08-01T19:30:59.408 INFO:teuthology.orchestra.run.smithi085.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2022-08-01T19:30:59.431 INFO:teuthology.orchestra.run.smithi077.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2022-08-01T19:30:59.435 INFO:teuthology.orchestra.run.smithi077.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2022-08-01T19:30:59.445 INFO:teuthology.orchestra.run.smithi077.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2022-08-01T19:30:59.500 INFO:teuthology.orchestra.run.smithi190.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2022-08-01T19:30:59.601 INFO:teuthology.orchestra.run.smithi190.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2022-08-01T19:30:59.606 INFO:teuthology.orchestra.run.smithi190.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2022-08-01T19:30:59.618 INFO:teuthology.orchestra.run.smithi190.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2022-08-01T19:30:59.769 INFO:teuthology.orchestra.run.smithi085.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 15.2.16-343-gfd96719b-1focal [163 kB] 2022-08-01T19:30:59.820 INFO:teuthology.orchestra.run.smithi085.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 15.2.16-343-gfd96719b-1focal [1,403 kB] 2022-08-01T19:30:59.934 INFO:teuthology.orchestra.run.smithi017.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 15.2.16-343-gfd96719b-1focal [163 kB] 2022-08-01T19:30:59.938 INFO:teuthology.orchestra.run.smithi017.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 15.2.16-343-gfd96719b-1focal [1,403 kB] 2022-08-01T19:30:59.988 INFO:teuthology.orchestra.run.smithi190.stdout:Get:52 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 15.2.16-343-gfd96719b-1focal [163 kB] 2022-08-01T19:30:59.989 INFO:teuthology.orchestra.run.smithi085.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 15.2.16-343-gfd96719b-1focal [6,162 kB] 2022-08-01T19:31:00.020 INFO:teuthology.orchestra.run.smithi190.stdout:Get:53 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 15.2.16-343-gfd96719b-1focal [1,403 kB] 2022-08-01T19:31:00.085 INFO:teuthology.orchestra.run.smithi017.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 15.2.16-343-gfd96719b-1focal [6,162 kB] 2022-08-01T19:31:00.163 INFO:teuthology.orchestra.run.smithi190.stdout:Get:54 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 15.2.16-343-gfd96719b-1focal [6,162 kB] 2022-08-01T19:31:00.674 INFO:teuthology.orchestra.run.smithi077.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2022-08-01T19:31:00.743 INFO:teuthology.orchestra.run.smithi017.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 15.2.16-343-gfd96719b-1focal [24.0 MB] 2022-08-01T19:31:00.819 INFO:teuthology.orchestra.run.smithi190.stdout:Get:55 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 15.2.16-343-gfd96719b-1focal [24.0 MB] 2022-08-01T19:31:00.862 INFO:teuthology.orchestra.run.smithi085.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 15.2.16-343-gfd96719b-1focal [24.0 MB] 2022-08-01T19:31:00.919 INFO:teuthology.orchestra.run.smithi077.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2022-08-01T19:31:00.924 INFO:teuthology.orchestra.run.smithi077.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2022-08-01T19:31:01.116 INFO:teuthology.orchestra.run.smithi077.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2022-08-01T19:31:01.117 INFO:teuthology.orchestra.run.smithi077.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2022-08-01T19:31:01.118 INFO:teuthology.orchestra.run.smithi077.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2022-08-01T19:31:01.119 INFO:teuthology.orchestra.run.smithi077.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2022-08-01T19:31:01.120 INFO:teuthology.orchestra.run.smithi077.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2022-08-01T19:31:01.121 INFO:teuthology.orchestra.run.smithi077.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2022-08-01T19:31:01.122 INFO:teuthology.orchestra.run.smithi077.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2022-08-01T19:31:01.124 INFO:teuthology.orchestra.run.smithi077.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2022-08-01T19:31:01.125 INFO:teuthology.orchestra.run.smithi077.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2022-08-01T19:31:01.197 INFO:teuthology.orchestra.run.smithi077.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2022-08-01T19:31:01.229 INFO:teuthology.orchestra.run.smithi077.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2022-08-01T19:31:01.230 INFO:teuthology.orchestra.run.smithi077.stdout:Get:65 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 15.2.16-343-gfd96719b-1focal [5,346 kB] 2022-08-01T19:31:01.232 INFO:teuthology.orchestra.run.smithi077.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2022-08-01T19:31:01.236 INFO:teuthology.orchestra.run.smithi077.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2022-08-01T19:31:01.237 INFO:teuthology.orchestra.run.smithi077.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2022-08-01T19:31:01.244 INFO:teuthology.orchestra.run.smithi077.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 libxslt1.1 amd64 1.1.34-4 [152 kB] 2022-08-01T19:31:01.246 INFO:teuthology.orchestra.run.smithi077.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2022-08-01T19:31:01.317 INFO:teuthology.orchestra.run.smithi077.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2022-08-01T19:31:01.317 INFO:teuthology.orchestra.run.smithi077.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2022-08-01T19:31:01.327 INFO:teuthology.orchestra.run.smithi077.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2022-08-01T19:31:01.389 INFO:teuthology.orchestra.run.smithi077.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2022-08-01T19:31:01.390 INFO:teuthology.orchestra.run.smithi077.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2022-08-01T19:31:01.391 INFO:teuthology.orchestra.run.smithi077.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2022-08-01T19:31:01.393 INFO:teuthology.orchestra.run.smithi077.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2022-08-01T19:31:01.498 INFO:teuthology.orchestra.run.smithi077.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2022-08-01T19:31:01.499 INFO:teuthology.orchestra.run.smithi077.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2022-08-01T19:31:01.499 INFO:teuthology.orchestra.run.smithi077.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2022-08-01T19:31:01.499 INFO:teuthology.orchestra.run.smithi077.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2022-08-01T19:31:01.503 INFO:teuthology.orchestra.run.smithi077.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2022-08-01T19:31:01.562 INFO:teuthology.orchestra.run.smithi077.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2022-08-01T19:31:01.562 INFO:teuthology.orchestra.run.smithi077.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2022-08-01T19:31:01.568 INFO:teuthology.orchestra.run.smithi077.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2022-08-01T19:31:02.278 INFO:teuthology.orchestra.run.smithi077.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 15.2.16-343-gfd96719b-1focal [163 kB] 2022-08-01T19:31:02.327 INFO:teuthology.orchestra.run.smithi077.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 15.2.16-343-gfd96719b-1focal [1,403 kB] 2022-08-01T19:31:02.593 INFO:teuthology.orchestra.run.smithi077.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 15.2.16-343-gfd96719b-1focal [6,162 kB] 2022-08-01T19:31:03.336 INFO:teuthology.orchestra.run.smithi017.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 15.2.16-343-gfd96719b-1focal [4,028 B] 2022-08-01T19:31:03.337 INFO:teuthology.orchestra.run.smithi017.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 15.2.16-343-gfd96719b-1focal [537 kB] 2022-08-01T19:31:03.347 INFO:teuthology.orchestra.run.smithi017.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 15.2.16-343-gfd96719b-1focal [2,007 kB] 2022-08-01T19:31:03.441 INFO:teuthology.orchestra.run.smithi190.stdout:Get:56 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 15.2.16-343-gfd96719b-1focal [4,028 B] 2022-08-01T19:31:03.441 INFO:teuthology.orchestra.run.smithi190.stdout:Get:57 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 15.2.16-343-gfd96719b-1focal [537 kB] 2022-08-01T19:31:03.453 INFO:teuthology.orchestra.run.smithi190.stdout:Get:58 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 15.2.16-343-gfd96719b-1focal [2,007 kB] 2022-08-01T19:31:03.491 INFO:teuthology.orchestra.run.smithi017.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 15.2.16-343-gfd96719b-1focal [53.2 kB] 2022-08-01T19:31:03.526 INFO:teuthology.orchestra.run.smithi017.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 15.2.16-343-gfd96719b-1focal [59.1 kB] 2022-08-01T19:31:03.528 INFO:teuthology.orchestra.run.smithi017.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 15.2.16-343-gfd96719b-1focal [3,218 kB] 2022-08-01T19:31:03.625 INFO:teuthology.orchestra.run.smithi190.stdout:Get:59 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 15.2.16-343-gfd96719b-1focal [53.2 kB] 2022-08-01T19:31:03.647 INFO:teuthology.orchestra.run.smithi190.stdout:Get:60 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 15.2.16-343-gfd96719b-1focal [59.1 kB] 2022-08-01T19:31:03.649 INFO:teuthology.orchestra.run.smithi190.stdout:Get:61 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 15.2.16-343-gfd96719b-1focal [3,218 kB] 2022-08-01T19:31:03.853 INFO:teuthology.orchestra.run.smithi017.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-cloud all 15.2.16-343-gfd96719b-1focal [34.4 kB] 2022-08-01T19:31:03.853 INFO:teuthology.orchestra.run.smithi017.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 15.2.16-343-gfd96719b-1focal [7,876 kB] 2022-08-01T19:31:03.961 INFO:teuthology.orchestra.run.smithi190.stdout:Get:62 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-cloud all 15.2.16-343-gfd96719b-1focal [34.4 kB] 2022-08-01T19:31:03.977 INFO:teuthology.orchestra.run.smithi190.stdout:Get:63 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 15.2.16-343-gfd96719b-1focal [7,876 kB] 2022-08-01T19:31:04.029 INFO:teuthology.orchestra.run.smithi077.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 15.2.16-343-gfd96719b-1focal [24.0 MB] 2022-08-01T19:31:04.596 INFO:teuthology.orchestra.run.smithi085.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 15.2.16-343-gfd96719b-1focal [4,028 B] 2022-08-01T19:31:04.597 INFO:teuthology.orchestra.run.smithi085.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 15.2.16-343-gfd96719b-1focal [537 kB] 2022-08-01T19:31:04.633 INFO:teuthology.orchestra.run.smithi085.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 15.2.16-343-gfd96719b-1focal [2,007 kB] 2022-08-01T19:31:04.642 INFO:teuthology.orchestra.run.smithi017.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 15.2.16-343-gfd96719b-1focal [4,148 B] 2022-08-01T19:31:04.643 INFO:teuthology.orchestra.run.smithi017.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 15.2.16-343-gfd96719b-1focal [40.2 MB] 2022-08-01T19:31:04.760 INFO:teuthology.orchestra.run.smithi190.stdout:Get:64 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 15.2.16-343-gfd96719b-1focal [4,148 B] 2022-08-01T19:31:04.761 INFO:teuthology.orchestra.run.smithi190.stdout:Get:65 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 15.2.16-343-gfd96719b-1focal [40.2 MB] 2022-08-01T19:31:04.926 INFO:teuthology.orchestra.run.smithi085.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 15.2.16-343-gfd96719b-1focal [53.2 kB] 2022-08-01T19:31:04.952 INFO:teuthology.orchestra.run.smithi085.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 15.2.16-343-gfd96719b-1focal [59.1 kB] 2022-08-01T19:31:04.955 INFO:teuthology.orchestra.run.smithi085.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 15.2.16-343-gfd96719b-1focal [3,218 kB] 2022-08-01T19:31:05.466 INFO:teuthology.orchestra.run.smithi085.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-cloud all 15.2.16-343-gfd96719b-1focal [34.4 kB] 2022-08-01T19:31:05.467 INFO:teuthology.orchestra.run.smithi085.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 15.2.16-343-gfd96719b-1focal [7,876 kB] 2022-08-01T19:31:06.679 INFO:teuthology.orchestra.run.smithi085.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 15.2.16-343-gfd96719b-1focal [4,148 B] 2022-08-01T19:31:06.680 INFO:teuthology.orchestra.run.smithi085.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 15.2.16-343-gfd96719b-1focal [40.2 MB] 2022-08-01T19:31:09.411 INFO:teuthology.orchestra.run.smithi017.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 15.2.16-343-gfd96719b-1focal [15.9 kB] 2022-08-01T19:31:09.413 INFO:teuthology.orchestra.run.smithi017.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 15.2.16-343-gfd96719b-1focal [18.1 kB] 2022-08-01T19:31:09.415 INFO:teuthology.orchestra.run.smithi017.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 15.2.16-343-gfd96719b-1focal [9,672 kB] 2022-08-01T19:31:09.688 INFO:teuthology.orchestra.run.smithi190.stdout:Get:66 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 15.2.16-343-gfd96719b-1focal [15.9 kB] 2022-08-01T19:31:09.689 INFO:teuthology.orchestra.run.smithi190.stdout:Get:67 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 15.2.16-343-gfd96719b-1focal [18.1 kB] 2022-08-01T19:31:09.689 INFO:teuthology.orchestra.run.smithi190.stdout:Get:68 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 15.2.16-343-gfd96719b-1focal [9,672 kB] 2022-08-01T19:31:10.154 INFO:teuthology.orchestra.run.smithi017.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 15.2.16-343-gfd96719b-1focal [61.3 kB] 2022-08-01T19:31:10.192 INFO:teuthology.orchestra.run.smithi077.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 15.2.16-343-gfd96719b-1focal [4,028 B] 2022-08-01T19:31:10.192 INFO:teuthology.orchestra.run.smithi077.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 15.2.16-343-gfd96719b-1focal [537 kB] 2022-08-01T19:31:10.275 INFO:teuthology.orchestra.run.smithi077.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 15.2.16-343-gfd96719b-1focal [2,007 kB] 2022-08-01T19:31:10.768 INFO:teuthology.orchestra.run.smithi017.stdout:Fetched 153 MB in 15s (10.4 MB/s) 2022-08-01T19:31:10.836 INFO:teuthology.orchestra.run.smithi077.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 15.2.16-343-gfd96719b-1focal [53.2 kB] 2022-08-01T19:31:10.838 INFO:teuthology.orchestra.run.smithi077.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 15.2.16-343-gfd96719b-1focal [59.1 kB] 2022-08-01T19:31:10.882 INFO:teuthology.orchestra.run.smithi077.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 15.2.16-343-gfd96719b-1focal [3,218 kB] 2022-08-01T19:31:10.907 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pyperclip. 2022-08-01T19:31:10.927 INFO:teuthology.orchestra.run.smithi190.stdout:Get:69 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 15.2.16-343-gfd96719b-1focal [61.3 kB] 2022-08-01T19:31:10.949 INFO:teuthology.orchestra.run.smithi017.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102338 files and directories currently installed.) 2022-08-01T19:31:10.952 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2022-08-01T19:31:10.989 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2022-08-01T19:31:11.308 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-wcwidth. 2022-08-01T19:31:11.321 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2022-08-01T19:31:11.358 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2022-08-01T19:31:11.395 INFO:teuthology.orchestra.run.smithi085.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 15.2.16-343-gfd96719b-1focal [15.9 kB] 2022-08-01T19:31:11.396 INFO:teuthology.orchestra.run.smithi085.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 15.2.16-343-gfd96719b-1focal [18.1 kB] 2022-08-01T19:31:11.396 INFO:teuthology.orchestra.run.smithi085.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 15.2.16-343-gfd96719b-1focal [9,672 kB] 2022-08-01T19:31:11.591 INFO:teuthology.orchestra.run.smithi077.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-cloud all 15.2.16-343-gfd96719b-1focal [34.4 kB] 2022-08-01T19:31:11.592 INFO:teuthology.orchestra.run.smithi077.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 15.2.16-343-gfd96719b-1focal [7,876 kB] 2022-08-01T19:31:11.636 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-cmd2. 2022-08-01T19:31:11.648 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2022-08-01T19:31:11.678 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2022-08-01T19:31:11.879 INFO:teuthology.orchestra.run.smithi085.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 15.2.16-343-gfd96719b-1focal [61.3 kB] 2022-08-01T19:31:12.005 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2022-08-01T19:31:12.019 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2022-08-01T19:31:12.130 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2022-08-01T19:31:12.471 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../004-librbd1_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:12.472 INFO:teuthology.orchestra.run.smithi085.stdout:Fetched 153 MB in 16s (9,318 kB/s) 2022-08-01T19:31:12.514 INFO:teuthology.orchestra.run.smithi077.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 15.2.16-343-gfd96719b-1focal [4,148 B] 2022-08-01T19:31:12.514 INFO:teuthology.orchestra.run.smithi077.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 15.2.16-343-gfd96719b-1focal [40.2 MB] 2022-08-01T19:31:12.613 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-pyperclip. 2022-08-01T19:31:12.626 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking librbd1 (15.2.16-343-gfd96719b-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-08-01T19:31:12.656 INFO:teuthology.orchestra.run.smithi085.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102338 files and directories currently installed.) 2022-08-01T19:31:12.659 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2022-08-01T19:31:12.727 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2022-08-01T19:31:13.014 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-wcwidth. 2022-08-01T19:31:13.028 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2022-08-01T19:31:13.056 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2022-08-01T19:31:13.159 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../005-librados2_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:13.255 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking librados2 (15.2.16-343-gfd96719b-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-08-01T19:31:13.333 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-cmd2. 2022-08-01T19:31:13.347 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2022-08-01T19:31:13.383 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2022-08-01T19:31:13.719 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2022-08-01T19:31:13.732 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2022-08-01T19:31:13.811 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2022-08-01T19:31:13.925 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libcephfs2. 2022-08-01T19:31:13.938 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../006-libcephfs2_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:13.975 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libcephfs2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:14.179 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../004-librbd1_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:14.307 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking librbd1 (15.2.16-343-gfd96719b-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-08-01T19:31:14.310 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-rados. 2022-08-01T19:31:14.325 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../007-python3-rados_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:14.369 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-rados (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:14.408 INFO:teuthology.orchestra.run.smithi190.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2022-08-01T19:31:14.680 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-ceph-argparse. 2022-08-01T19:31:14.696 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../008-python3-ceph-argparse_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:31:14.755 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-ceph-argparse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:14.856 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../005-librados2_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:14.977 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking librados2 (15.2.16-343-gfd96719b-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-08-01T19:31:14.999 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-cephfs. 2022-08-01T19:31:15.015 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../009-python3-cephfs_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:15.057 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-cephfs (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:15.410 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-ceph-common. 2022-08-01T19:31:15.425 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../010-python3-ceph-common_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:31:15.468 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-ceph-common (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:15.581 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package libcephfs2. 2022-08-01T19:31:15.593 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../006-libcephfs2_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:15.639 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking libcephfs2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:15.804 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-prettytable. 2022-08-01T19:31:15.818 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2022-08-01T19:31:15.863 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2022-08-01T19:31:15.891 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-rados. 2022-08-01T19:31:15.905 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../007-python3-rados_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:15.941 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-rados (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:16.140 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-rbd. 2022-08-01T19:31:16.154 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../012-python3-rbd_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:16.171 INFO:teuthology.orchestra.run.smithi077.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 15.2.16-343-gfd96719b-1focal [15.9 kB] 2022-08-01T19:31:16.171 INFO:teuthology.orchestra.run.smithi077.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 15.2.16-343-gfd96719b-1focal [18.1 kB] 2022-08-01T19:31:16.172 INFO:teuthology.orchestra.run.smithi077.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 15.2.16-343-gfd96719b-1focal [9,672 kB] 2022-08-01T19:31:16.198 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-rbd (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:16.319 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-ceph-argparse. 2022-08-01T19:31:16.333 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../008-python3-ceph-argparse_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:31:16.377 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-ceph-argparse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:16.484 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package librabbitmq4:amd64. 2022-08-01T19:31:16.498 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../013-librabbitmq4_0.10.0-1_amd64.deb ... 2022-08-01T19:31:16.568 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2022-08-01T19:31:16.688 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-cephfs. 2022-08-01T19:31:16.700 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../009-python3-cephfs_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:16.754 INFO:teuthology.orchestra.run.smithi077.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2022-08-01-0839-octopus/fd96719b0e1200b56e7430604cfdd20ad281ca92/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 15.2.16-343-gfd96719b-1focal [61.3 kB] 2022-08-01T19:31:16.762 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-cephfs (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:16.895 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package librdkafka1:amd64. 2022-08-01T19:31:16.908 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../014-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2022-08-01T19:31:16.945 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-08-01T19:31:17.040 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-ceph-common. 2022-08-01T19:31:17.056 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../010-python3-ceph-common_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:31:17.090 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-ceph-common (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:17.298 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package librgw2. 2022-08-01T19:31:17.311 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../015-librgw2_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:17.347 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking librgw2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:17.401 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-prettytable. 2022-08-01T19:31:17.407 INFO:teuthology.orchestra.run.smithi077.stdout:Fetched 153 MB in 21s (7,195 kB/s) 2022-08-01T19:31:17.408 INFO:teuthology.orchestra.run.smithi190.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2022-08-01T19:31:17.413 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2022-08-01T19:31:17.442 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2022-08-01T19:31:17.556 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-pyperclip. 2022-08-01T19:31:17.600 INFO:teuthology.orchestra.run.smithi077.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102338 files and directories currently installed.) 2022-08-01T19:31:17.603 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2022-08-01T19:31:17.646 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2022-08-01T19:31:17.662 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-rbd. 2022-08-01T19:31:17.674 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../012-python3-rbd_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:17.682 INFO:teuthology.orchestra.run.smithi190.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2022-08-01T19:31:17.711 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-rbd (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:17.911 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-rgw. 2022-08-01T19:31:17.923 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-wcwidth. 2022-08-01T19:31:17.924 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../016-python3-rgw_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:17.937 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2022-08-01T19:31:17.960 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-rgw (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:17.973 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2022-08-01T19:31:18.139 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package librabbitmq4:amd64. 2022-08-01T19:31:18.152 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../013-librabbitmq4_0.10.0-1_amd64.deb ... 2022-08-01T19:31:18.189 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2022-08-01T19:31:18.280 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libleveldb1d:amd64. 2022-08-01T19:31:18.294 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../017-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2022-08-01T19:31:18.301 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-cmd2. 2022-08-01T19:31:18.313 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2022-08-01T19:31:18.351 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2022-08-01T19:31:18.379 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-08-01T19:31:18.561 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package librdkafka1:amd64. 2022-08-01T19:31:18.571 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../014-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2022-08-01T19:31:18.608 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-08-01T19:31:18.721 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2022-08-01T19:31:18.724 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package liboath0. 2022-08-01T19:31:18.735 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2022-08-01T19:31:18.738 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../018-liboath0_2.6.1-1.3_amd64.deb ... 2022-08-01T19:31:18.807 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2022-08-01T19:31:18.846 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2022-08-01T19:31:18.902 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package librgw2. 2022-08-01T19:31:18.917 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../015-librgw2_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:18.952 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking librgw2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:19.051 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libradosstriper1. 2022-08-01T19:31:19.067 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../019-libradosstriper1_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:19.101 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libradosstriper1 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:19.213 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../004-librbd1_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:19.324 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking librbd1 (15.2.16-343-gfd96719b-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-08-01T19:31:19.488 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-rgw. 2022-08-01T19:31:19.500 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../016-python3-rgw_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:19.538 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-rgw (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:19.563 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-common. 2022-08-01T19:31:19.578 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../020-ceph-common_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:19.621 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-common (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:19.868 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../005-librados2_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:19.891 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package libleveldb1d:amd64. 2022-08-01T19:31:19.904 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../017-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2022-08-01T19:31:19.941 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-08-01T19:31:19.987 INFO:teuthology.orchestra.run.smithi190.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2022-08-01T19:31:19.991 INFO:teuthology.orchestra.run.smithi190.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2022-08-01T19:31:19.995 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking librados2 (15.2.16-343-gfd96719b-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-08-01T19:31:20.076 INFO:teuthology.orchestra.run.smithi190.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2022-08-01T19:31:20.089 INFO:teuthology.orchestra.run.smithi190.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2022-08-01T19:31:20.100 INFO:teuthology.orchestra.run.smithi190.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2022-08-01T19:31:20.180 INFO:teuthology.orchestra.run.smithi190.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2022-08-01T19:31:20.185 INFO:teuthology.orchestra.run.smithi190.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2022-08-01T19:31:20.285 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package liboath0. 2022-08-01T19:31:20.286 INFO:teuthology.orchestra.run.smithi190.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2022-08-01T19:31:20.292 INFO:teuthology.orchestra.run.smithi190.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2022-08-01T19:31:20.297 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../018-liboath0_2.6.1-1.3_amd64.deb ... 2022-08-01T19:31:20.335 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2022-08-01T19:31:20.372 INFO:teuthology.orchestra.run.smithi190.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2022-08-01T19:31:20.579 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package libradosstriper1. 2022-08-01T19:31:20.594 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../019-libradosstriper1_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:20.616 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package libcephfs2. 2022-08-01T19:31:20.629 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking libradosstriper1 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:20.631 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../006-libcephfs2_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:20.665 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking libcephfs2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:20.746 INFO:teuthology.orchestra.run.smithi190.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2022-08-01T19:31:20.976 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-rados. 2022-08-01T19:31:21.031 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../007-python3-rados_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:21.059 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-rados (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:21.099 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package ceph-common. 2022-08-01T19:31:21.111 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../020-ceph-common_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:21.149 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking ceph-common (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:21.178 INFO:teuthology.orchestra.run.smithi190.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2022-08-01T19:31:21.360 INFO:teuthology.orchestra.run.smithi190.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2022-08-01T19:31:21.429 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-ceph-argparse. 2022-08-01T19:31:21.443 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../008-python3-ceph-argparse_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:31:21.445 INFO:teuthology.orchestra.run.smithi190.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2022-08-01T19:31:21.487 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-ceph-argparse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:21.743 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-base. 2022-08-01T19:31:21.758 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../021-ceph-base_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:21.798 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-cephfs. 2022-08-01T19:31:21.803 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-base (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:21.814 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../009-python3-cephfs_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:21.841 INFO:teuthology.orchestra.run.smithi190.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 libxslt1.1 amd64 1.1.34-4 [152 kB] 2022-08-01T19:31:21.848 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-cephfs (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:22.034 INFO:teuthology.orchestra.run.smithi190.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2022-08-01T19:31:22.175 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-ceph-common. 2022-08-01T19:31:22.189 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../010-python3-ceph-common_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:31:22.233 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-ceph-common (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:22.407 INFO:teuthology.orchestra.run.smithi190.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2022-08-01T19:31:22.416 INFO:teuthology.orchestra.run.smithi190.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2022-08-01T19:31:22.555 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-prettytable. 2022-08-01T19:31:22.571 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2022-08-01T19:31:22.638 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2022-08-01T19:31:22.672 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-cryptography. 2022-08-01T19:31:22.687 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../022-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2022-08-01T19:31:22.721 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2022-08-01T19:31:22.794 INFO:teuthology.orchestra.run.smithi190.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2022-08-01T19:31:22.798 INFO:teuthology.orchestra.run.smithi190.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2022-08-01T19:31:22.811 INFO:teuthology.orchestra.run.smithi190.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2022-08-01T19:31:22.816 INFO:teuthology.orchestra.run.smithi190.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2022-08-01T19:31:22.824 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-rbd. 2022-08-01T19:31:22.837 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../012-python3-rbd_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:22.866 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-rbd (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:22.983 INFO:teuthology.orchestra.run.smithi190.stdout:Get:95 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2022-08-01T19:31:23.100 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-openssl. 2022-08-01T19:31:23.114 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../023-python3-openssl_19.0.0-1build1_all.deb ... 2022-08-01T19:31:23.158 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2022-08-01T19:31:23.212 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package ceph-base. 2022-08-01T19:31:23.226 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../021-ceph-base_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:23.272 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking ceph-base (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:23.294 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package librabbitmq4:amd64. 2022-08-01T19:31:23.306 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../013-librabbitmq4_0.10.0-1_amd64.deb ... 2022-08-01T19:31:23.344 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2022-08-01T19:31:23.436 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2022-08-01T19:31:23.448 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../024-ceph-mgr-modules-core_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:31:23.477 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-mgr-modules-core (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:23.722 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package librdkafka1:amd64. 2022-08-01T19:31:23.736 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../014-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2022-08-01T19:31:23.780 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-08-01T19:31:23.807 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-bcrypt. 2022-08-01T19:31:23.821 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../025-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2022-08-01T19:31:23.856 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2022-08-01T19:31:24.108 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package librgw2. 2022-08-01T19:31:24.121 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../015-librgw2_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:24.124 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-cryptography. 2022-08-01T19:31:24.136 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../022-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2022-08-01T19:31:24.149 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking librgw2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:24.165 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2022-08-01T19:31:24.192 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-cherrypy3. 2022-08-01T19:31:24.208 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../026-python3-cherrypy3_8.9.1-6_all.deb ... 2022-08-01T19:31:24.250 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2022-08-01T19:31:24.527 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-openssl. 2022-08-01T19:31:24.542 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../023-python3-openssl_19.0.0-1build1_all.deb ... 2022-08-01T19:31:24.562 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-jwt. 2022-08-01T19:31:24.577 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../027-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2022-08-01T19:31:24.578 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2022-08-01T19:31:24.637 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2022-08-01T19:31:24.719 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-rgw. 2022-08-01T19:31:24.732 INFO:teuthology.orchestra.run.smithi190.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2022-08-01T19:31:24.733 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../016-python3-rgw_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:24.777 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-rgw (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:24.808 INFO:teuthology.orchestra.run.smithi190.stdout:Get:97 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2022-08-01T19:31:24.813 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2022-08-01T19:31:24.827 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../024-ceph-mgr-modules-core_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:31:24.831 INFO:teuthology.orchestra.run.smithi190.stdout:Get:98 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2022-08-01T19:31:24.904 INFO:teuthology.orchestra.run.smithi190.stdout:Get:99 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2022-08-01T19:31:24.905 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking ceph-mgr-modules-core (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:25.028 INFO:teuthology.orchestra.run.smithi190.stdout:Get:100 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2022-08-01T19:31:25.131 INFO:teuthology.orchestra.run.smithi190.stdout:Get:101 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2022-08-01T19:31:25.183 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package libleveldb1d:amd64. 2022-08-01T19:31:25.199 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../017-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2022-08-01T19:31:25.203 INFO:teuthology.orchestra.run.smithi190.stdout:Get:102 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2022-08-01T19:31:25.247 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-08-01T19:31:25.450 INFO:teuthology.orchestra.run.smithi190.stdout:Get:103 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2022-08-01T19:31:25.557 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-logutils. 2022-08-01T19:31:25.566 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package liboath0. 2022-08-01T19:31:25.570 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../028-python3-logutils_0.3.3-7_all.deb ... 2022-08-01T19:31:25.580 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../018-liboath0_2.6.1-1.3_amd64.deb ... 2022-08-01T19:31:25.616 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2022-08-01T19:31:25.632 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-logutils (0.3.3-7) ... 2022-08-01T19:31:25.892 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-bcrypt. 2022-08-01T19:31:25.907 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../025-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2022-08-01T19:31:25.927 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package libradosstriper1. 2022-08-01T19:31:25.934 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-markupsafe. 2022-08-01T19:31:25.941 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../019-libradosstriper1_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:25.947 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../029-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2022-08-01T19:31:25.950 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2022-08-01T19:31:25.984 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2022-08-01T19:31:25.985 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking libradosstriper1 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:26.123 INFO:teuthology.orchestra.run.smithi190.stdout:Fetched 153 MB in 30s (5,099 kB/s) 2022-08-01T19:31:26.261 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-cherrypy3. 2022-08-01T19:31:26.262 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-mako. 2022-08-01T19:31:26.263 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pyperclip. 2022-08-01T19:31:26.275 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../030-python3-mako_1.1.0+ds1-1ubuntu2_all.deb ... 2022-08-01T19:31:26.277 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../026-python3-cherrypy3_8.9.1-6_all.deb ... 2022-08-01T19:31:26.304 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-08-01T19:31:26.304 INFO:teuthology.orchestra.run.smithi190.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102338 files and directories currently installed.) 2022-08-01T19:31:26.307 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2022-08-01T19:31:26.319 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2022-08-01T19:31:26.369 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2022-08-01T19:31:26.498 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package ceph-common. 2022-08-01T19:31:26.514 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../020-ceph-common_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:26.556 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking ceph-common (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:26.615 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-simplegeneric. 2022-08-01T19:31:26.630 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../031-python3-simplegeneric_0.8.1-3_all.deb ... 2022-08-01T19:31:26.673 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2022-08-01T19:31:26.680 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-wcwidth. 2022-08-01T19:31:26.696 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2022-08-01T19:31:26.698 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-jwt. 2022-08-01T19:31:26.711 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../027-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2022-08-01T19:31:26.730 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2022-08-01T19:31:26.748 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2022-08-01T19:31:27.009 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-singledispatch. 2022-08-01T19:31:27.016 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-cmd2. 2022-08-01T19:31:27.022 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../032-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2022-08-01T19:31:27.029 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2022-08-01T19:31:27.057 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2022-08-01T19:31:27.059 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-logutils. 2022-08-01T19:31:27.059 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2022-08-01T19:31:27.075 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../028-python3-logutils_0.3.3-7_all.deb ... 2022-08-01T19:31:27.117 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-logutils (0.3.3-7) ... 2022-08-01T19:31:27.354 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-webob. 2022-08-01T19:31:27.367 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../033-python3-webob_1%3a1.8.5-2_all.deb ... 2022-08-01T19:31:27.403 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-markupsafe. 2022-08-01T19:31:27.417 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../029-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2022-08-01T19:31:27.435 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2022-08-01T19:31:27.437 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2022-08-01T19:31:27.449 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2022-08-01T19:31:27.453 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2022-08-01T19:31:27.527 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2022-08-01T19:31:27.706 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-mako. 2022-08-01T19:31:27.718 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../030-python3-mako_1.1.0+ds1-1ubuntu2_all.deb ... 2022-08-01T19:31:27.731 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-soupsieve. 2022-08-01T19:31:27.746 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../034-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2022-08-01T19:31:27.755 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-08-01T19:31:27.781 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2022-08-01T19:31:27.893 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../004-librbd1_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:28.048 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking librbd1 (15.2.16-343-gfd96719b-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-08-01T19:31:28.067 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-bs4. 2022-08-01T19:31:28.080 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../035-python3-bs4_4.8.2-1_all.deb ... 2022-08-01T19:31:28.108 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-simplegeneric. 2022-08-01T19:31:28.117 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2022-08-01T19:31:28.122 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../031-python3-simplegeneric_0.8.1-3_all.deb ... 2022-08-01T19:31:28.166 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2022-08-01T19:31:28.420 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-waitress. 2022-08-01T19:31:28.434 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../036-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2022-08-01T19:31:28.469 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-singledispatch. 2022-08-01T19:31:28.473 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2022-08-01T19:31:28.483 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../032-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2022-08-01T19:31:28.527 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2022-08-01T19:31:28.582 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../005-librados2_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:28.653 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package ceph-base. 2022-08-01T19:31:28.665 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../021-ceph-base_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:28.702 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking librados2 (15.2.16-343-gfd96719b-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-08-01T19:31:28.713 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking ceph-base (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:28.763 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-webob. 2022-08-01T19:31:28.765 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-tempita. 2022-08-01T19:31:28.778 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../033-python3-webob_1%3a1.8.5-2_all.deb ... 2022-08-01T19:31:28.779 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../037-python3-tempita_0.5.2-6_all.deb ... 2022-08-01T19:31:28.823 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-tempita (0.5.2-6) ... 2022-08-01T19:31:28.830 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2022-08-01T19:31:29.109 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-paste. 2022-08-01T19:31:29.125 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../038-python3-paste_3.4.0+dfsg1-1_all.deb ... 2022-08-01T19:31:29.191 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-soupsieve. 2022-08-01T19:31:29.200 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2022-08-01T19:31:29.206 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../034-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2022-08-01T19:31:29.249 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2022-08-01T19:31:29.389 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libcephfs2. 2022-08-01T19:31:29.404 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../006-libcephfs2_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:29.439 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libcephfs2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:29.520 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2022-08-01T19:31:29.534 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../039-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2022-08-01T19:31:29.577 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-bs4. 2022-08-01T19:31:29.578 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2022-08-01T19:31:29.591 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../035-python3-bs4_4.8.2-1_all.deb ... 2022-08-01T19:31:29.635 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2022-08-01T19:31:29.649 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-cryptography. 2022-08-01T19:31:29.664 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../022-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2022-08-01T19:31:29.720 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2022-08-01T19:31:29.741 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-rados. 2022-08-01T19:31:29.753 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../007-python3-rados_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:29.782 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-rados (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:29.906 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pastedeploy. 2022-08-01T19:31:29.919 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../040-python3-pastedeploy_2.1.0-1_all.deb ... 2022-08-01T19:31:29.956 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2022-08-01T19:31:29.971 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-waitress. 2022-08-01T19:31:29.984 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../036-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2022-08-01T19:31:30.024 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2022-08-01T19:31:30.194 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-ceph-argparse. 2022-08-01T19:31:30.209 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../008-python3-ceph-argparse_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:31:30.252 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-ceph-argparse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:30.284 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-webtest. 2022-08-01T19:31:30.298 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../041-python3-webtest_2.0.32-2_all.deb ... 2022-08-01T19:31:30.342 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-webtest (2.0.32-2) ... 2022-08-01T19:31:30.357 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-tempita. 2022-08-01T19:31:30.371 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../037-python3-tempita_0.5.2-6_all.deb ... 2022-08-01T19:31:30.407 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-tempita (0.5.2-6) ... 2022-08-01T19:31:30.588 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-cephfs. 2022-08-01T19:31:30.603 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../009-python3-cephfs_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:30.646 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-cephfs (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:30.735 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-paste. 2022-08-01T19:31:30.751 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../038-python3-paste_3.4.0+dfsg1-1_all.deb ... 2022-08-01T19:31:30.793 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2022-08-01T19:31:30.795 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pecan. 2022-08-01T19:31:30.810 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../042-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2022-08-01T19:31:30.844 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2022-08-01T19:31:30.928 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-openssl. 2022-08-01T19:31:30.941 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../023-python3-openssl_19.0.0-1build1_all.deb ... 2022-08-01T19:31:30.969 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2022-08-01T19:31:30.990 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-ceph-common. 2022-08-01T19:31:31.005 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../010-python3-ceph-common_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:31:31.048 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-ceph-common (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:31.216 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-werkzeug. 2022-08-01T19:31:31.232 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../043-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2022-08-01T19:31:31.265 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2022-08-01T19:31:31.272 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2022-08-01T19:31:31.285 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../024-ceph-mgr-modules-core_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:31:31.304 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2022-08-01T19:31:31.320 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../039-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2022-08-01T19:31:31.322 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking ceph-mgr-modules-core (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:31.359 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-prettytable. 2022-08-01T19:31:31.362 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2022-08-01T19:31:31.374 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2022-08-01T19:31:31.409 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2022-08-01T19:31:31.649 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-pastedeploy. 2022-08-01T19:31:31.660 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-bcrypt. 2022-08-01T19:31:31.661 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-mgr. 2022-08-01T19:31:31.662 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-rbd. 2022-08-01T19:31:31.664 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../040-python3-pastedeploy_2.1.0-1_all.deb ... 2022-08-01T19:31:31.674 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../044-ceph-mgr_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:31.675 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../025-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2022-08-01T19:31:31.677 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../012-python3-rbd_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:31.707 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2022-08-01T19:31:31.710 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2022-08-01T19:31:31.718 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-mgr (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:31.720 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-rbd (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:31.996 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-cherrypy3. 2022-08-01T19:31:32.009 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../026-python3-cherrypy3_8.9.1-6_all.deb ... 2022-08-01T19:31:32.035 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-webtest. 2022-08-01T19:31:32.046 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2022-08-01T19:31:32.047 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../041-python3-webtest_2.0.32-2_all.deb ... 2022-08-01T19:31:32.084 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-webtest (2.0.32-2) ... 2022-08-01T19:31:32.156 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package librabbitmq4:amd64. 2022-08-01T19:31:32.172 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../013-librabbitmq4_0.10.0-1_amd64.deb ... 2022-08-01T19:31:32.214 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2022-08-01T19:31:32.255 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-mon. 2022-08-01T19:31:32.271 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../045-ceph-mon_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:32.338 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-mon (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:32.350 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-jwt. 2022-08-01T19:31:32.366 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../027-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2022-08-01T19:31:32.400 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2022-08-01T19:31:32.420 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-pecan. 2022-08-01T19:31:32.436 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../042-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2022-08-01T19:31:32.479 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2022-08-01T19:31:32.584 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package librdkafka1:amd64. 2022-08-01T19:31:32.599 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../014-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2022-08-01T19:31:32.642 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-08-01T19:31:32.720 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-logutils. 2022-08-01T19:31:32.735 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../028-python3-logutils_0.3.3-7_all.deb ... 2022-08-01T19:31:32.778 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-logutils (0.3.3-7) ... 2022-08-01T19:31:32.808 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-werkzeug. 2022-08-01T19:31:32.822 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../043-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2022-08-01T19:31:32.867 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2022-08-01T19:31:33.037 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package librgw2. 2022-08-01T19:31:33.049 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../015-librgw2_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:33.106 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-markupsafe. 2022-08-01T19:31:33.111 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking librgw2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:33.118 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../029-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2022-08-01T19:31:33.181 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2022-08-01T19:31:33.253 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package ceph-mgr. 2022-08-01T19:31:33.267 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../044-ceph-mgr_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:33.311 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking ceph-mgr (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:33.463 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-mako. 2022-08-01T19:31:33.476 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../030-python3-mako_1.1.0+ds1-1ubuntu2_all.deb ... 2022-08-01T19:31:33.483 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-osd. 2022-08-01T19:31:33.498 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../046-ceph-osd_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:33.525 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-08-01T19:31:33.541 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-osd (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:33.697 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-rgw. 2022-08-01T19:31:33.709 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../016-python3-rgw_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:33.772 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-rgw (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:33.789 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package ceph-mon. 2022-08-01T19:31:33.802 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../045-ceph-mon_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:33.830 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking ceph-mon (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:33.878 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-simplegeneric. 2022-08-01T19:31:33.893 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../031-python3-simplegeneric_0.8.1-3_all.deb ... 2022-08-01T19:31:33.937 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2022-08-01T19:31:34.116 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libleveldb1d:amd64. 2022-08-01T19:31:34.132 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../017-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2022-08-01T19:31:34.174 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-08-01T19:31:34.281 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-singledispatch. 2022-08-01T19:31:34.294 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../032-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2022-08-01T19:31:34.331 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2022-08-01T19:31:34.486 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package liboath0. 2022-08-01T19:31:34.500 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../018-liboath0_2.6.1-1.3_amd64.deb ... 2022-08-01T19:31:34.536 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2022-08-01T19:31:34.634 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-webob. 2022-08-01T19:31:34.650 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../033-python3-webob_1%3a1.8.5-2_all.deb ... 2022-08-01T19:31:34.684 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2022-08-01T19:31:34.876 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package ceph-osd. 2022-08-01T19:31:34.880 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libradosstriper1. 2022-08-01T19:31:34.890 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../046-ceph-osd_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:34.895 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../019-libradosstriper1_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:34.934 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking ceph-osd (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:34.938 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libradosstriper1 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:35.012 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-soupsieve. 2022-08-01T19:31:35.025 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../034-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2022-08-01T19:31:35.095 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2022-08-01T19:31:35.423 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-bs4. 2022-08-01T19:31:35.433 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-common. 2022-08-01T19:31:35.436 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../035-python3-bs4_4.8.2-1_all.deb ... 2022-08-01T19:31:35.445 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../020-ceph-common_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:35.473 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2022-08-01T19:31:35.474 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-common (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:35.851 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-waitress. 2022-08-01T19:31:35.865 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../036-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2022-08-01T19:31:35.880 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph. 2022-08-01T19:31:35.896 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../047-ceph_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:35.912 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2022-08-01T19:31:35.930 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:36.263 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-tempita. 2022-08-01T19:31:36.265 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-fuse. 2022-08-01T19:31:36.277 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../037-python3-tempita_0.5.2-6_all.deb ... 2022-08-01T19:31:36.281 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../048-ceph-fuse_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:36.324 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-fuse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:36.345 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-tempita (0.5.2-6) ... 2022-08-01T19:31:36.683 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-paste. 2022-08-01T19:31:36.705 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../038-python3-paste_3.4.0+dfsg1-1_all.deb ... 2022-08-01T19:31:36.741 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2022-08-01T19:31:36.744 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-mds. 2022-08-01T19:31:36.758 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../049-ceph-mds_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:36.802 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-mds (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:37.044 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2022-08-01T19:31:37.059 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../039-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2022-08-01T19:31:37.102 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2022-08-01T19:31:37.246 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package cephadm. 2022-08-01T19:31:37.259 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../050-cephadm_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:37.288 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking cephadm (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:37.322 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package ceph. 2022-08-01T19:31:37.337 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../047-ceph_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:37.372 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking ceph (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:37.422 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-pastedeploy. 2022-08-01T19:31:37.438 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../040-python3-pastedeploy_2.1.0-1_all.deb ... 2022-08-01T19:31:37.480 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-base. 2022-08-01T19:31:37.488 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2022-08-01T19:31:37.493 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../021-ceph-base_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:37.539 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-base (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:37.608 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package ceph-fuse. 2022-08-01T19:31:37.623 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../048-ceph-fuse_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:37.624 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-jinja2. 2022-08-01T19:31:37.638 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../051-python3-jinja2_2.10.1-2_all.deb ... 2022-08-01T19:31:37.682 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2022-08-01T19:31:37.691 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking ceph-fuse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:37.825 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-webtest. 2022-08-01T19:31:37.840 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../041-python3-webtest_2.0.32-2_all.deb ... 2022-08-01T19:31:37.883 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-webtest (2.0.32-2) ... 2022-08-01T19:31:38.018 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2022-08-01T19:31:38.031 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../052-ceph-mgr-cephadm_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:31:38.068 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-mgr-cephadm (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:38.077 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package ceph-mds. 2022-08-01T19:31:38.090 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../049-ceph-mds_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:38.127 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking ceph-mds (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:38.195 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-pecan. 2022-08-01T19:31:38.207 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../042-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2022-08-01T19:31:38.236 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2022-08-01T19:31:38.405 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-repoze.lru. 2022-08-01T19:31:38.420 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../053-python3-repoze.lru_0.7-2_all.deb ... 2022-08-01T19:31:38.459 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-cryptography. 2022-08-01T19:31:38.463 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2022-08-01T19:31:38.472 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../022-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2022-08-01T19:31:38.508 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2022-08-01T19:31:38.600 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-werkzeug. 2022-08-01T19:31:38.615 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../043-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2022-08-01T19:31:38.638 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package cephadm. 2022-08-01T19:31:38.649 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2022-08-01T19:31:38.653 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../050-cephadm_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:38.697 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking cephadm (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:38.741 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libjs-underscore. 2022-08-01T19:31:38.753 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../054-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2022-08-01T19:31:38.790 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-08-01T19:31:38.879 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-openssl. 2022-08-01T19:31:38.892 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../023-python3-openssl_19.0.0-1build1_all.deb ... 2022-08-01T19:31:38.962 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2022-08-01T19:31:38.999 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-jinja2. 2022-08-01T19:31:39.014 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../051-python3-jinja2_2.10.1-2_all.deb ... 2022-08-01T19:31:39.019 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package ceph-mgr. 2022-08-01T19:31:39.035 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../044-ceph-mgr_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:39.042 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libjs-sphinxdoc. 2022-08-01T19:31:39.049 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2022-08-01T19:31:39.059 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../055-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2022-08-01T19:31:39.069 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking ceph-mgr (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:39.101 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-08-01T19:31:39.253 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2022-08-01T19:31:39.269 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../024-ceph-mgr-modules-core_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:31:39.314 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mgr-modules-core (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:39.335 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2022-08-01T19:31:39.351 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../052-ceph-mgr-cephadm_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:31:39.418 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking ceph-mgr-cephadm (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:39.429 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-routes. 2022-08-01T19:31:39.442 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../056-python3-routes_2.4.1-2_all.deb ... 2022-08-01T19:31:39.487 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-routes (2.4.1-2) ... 2022-08-01T19:31:39.515 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package ceph-mon. 2022-08-01T19:31:39.529 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../045-ceph-mon_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:39.581 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking ceph-mon (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:39.680 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-repoze.lru. 2022-08-01T19:31:39.692 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../053-python3-repoze.lru_0.7-2_all.deb ... 2022-08-01T19:31:39.710 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-bcrypt. 2022-08-01T19:31:39.721 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2022-08-01T19:31:39.724 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../025-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2022-08-01T19:31:39.760 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2022-08-01T19:31:39.840 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2022-08-01T19:31:39.856 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:31:39.898 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-mgr-dashboard (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:40.010 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package libjs-underscore. 2022-08-01T19:31:40.023 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../054-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2022-08-01T19:31:40.068 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-08-01T19:31:40.088 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-cherrypy3. 2022-08-01T19:31:40.102 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../026-python3-cherrypy3_8.9.1-6_all.deb ... 2022-08-01T19:31:40.146 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2022-08-01T19:31:40.329 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package libjs-sphinxdoc. 2022-08-01T19:31:40.344 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../055-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2022-08-01T19:31:40.379 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-08-01T19:31:40.566 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-jwt. 2022-08-01T19:31:40.582 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../027-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2022-08-01T19:31:40.624 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2022-08-01T19:31:40.635 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package ceph-osd. 2022-08-01T19:31:40.649 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../046-ceph-osd_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:40.682 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-routes. 2022-08-01T19:31:40.693 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking ceph-osd (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:40.695 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../056-python3-routes_2.4.1-2_all.deb ... 2022-08-01T19:31:40.723 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-routes (2.4.1-2) ... 2022-08-01T19:31:40.897 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2022-08-01T19:31:40.911 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../058-ceph-mgr-diskprediction-cloud_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:31:40.988 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:41.027 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2022-08-01T19:31:41.043 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:31:41.090 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking ceph-mgr-dashboard (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:41.094 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-logutils. 2022-08-01T19:31:41.108 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../028-python3-logutils_0.3.3-7_all.deb ... 2022-08-01T19:31:41.174 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-logutils (0.3.3-7) ... 2022-08-01T19:31:41.367 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-decorator. 2022-08-01T19:31:41.382 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../059-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2022-08-01T19:31:41.422 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-markupsafe. 2022-08-01T19:31:41.437 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../029-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2022-08-01T19:31:41.441 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2022-08-01T19:31:41.494 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2022-08-01T19:31:41.744 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2022-08-01T19:31:41.758 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../060-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2022-08-01T19:31:41.802 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-08-01T19:31:41.833 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-mako. 2022-08-01T19:31:41.848 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../030-python3-mako_1.1.0+ds1-1ubuntu2_all.deb ... 2022-08-01T19:31:41.891 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-08-01T19:31:42.036 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2022-08-01T19:31:42.052 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../058-ceph-mgr-diskprediction-cloud_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:31:42.094 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:42.113 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-scipy. 2022-08-01T19:31:42.128 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../061-python3-scipy_1.3.3-3build1_amd64.deb ... 2022-08-01T19:31:42.163 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2022-08-01T19:31:42.202 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-simplegeneric. 2022-08-01T19:31:42.215 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../031-python3-simplegeneric_0.8.1-3_all.deb ... 2022-08-01T19:31:42.244 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2022-08-01T19:31:42.397 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-decorator. 2022-08-01T19:31:42.412 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../059-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2022-08-01T19:31:42.447 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2022-08-01T19:31:42.522 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-singledispatch. 2022-08-01T19:31:42.537 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../032-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2022-08-01T19:31:42.571 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2022-08-01T19:31:42.725 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2022-08-01T19:31:42.739 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../060-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2022-08-01T19:31:42.783 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-08-01T19:31:42.891 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-webob. 2022-08-01T19:31:42.904 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../033-python3-webob_1%3a1.8.5-2_all.deb ... 2022-08-01T19:31:42.940 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2022-08-01T19:31:43.132 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package ceph. 2022-08-01T19:31:43.145 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-scipy. 2022-08-01T19:31:43.147 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../047-ceph_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:43.157 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../061-python3-scipy_1.3.3-3build1_amd64.deb ... 2022-08-01T19:31:43.190 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking ceph (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:43.194 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2022-08-01T19:31:43.260 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-soupsieve. 2022-08-01T19:31:43.276 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../034-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2022-08-01T19:31:43.327 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2022-08-01T19:31:43.477 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package ceph-fuse. 2022-08-01T19:31:43.480 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-sklearn-lib. 2022-08-01T19:31:43.491 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../048-ceph-fuse_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:43.494 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../062-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2022-08-01T19:31:43.526 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking ceph-fuse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:43.538 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-08-01T19:31:43.646 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-bs4. 2022-08-01T19:31:43.661 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../035-python3-bs4_4.8.2-1_all.deb ... 2022-08-01T19:31:43.704 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2022-08-01T19:31:43.955 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package ceph-mds. 2022-08-01T19:31:43.969 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../049-ceph-mds_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:44.013 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking ceph-mds (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:44.050 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-joblib. 2022-08-01T19:31:44.064 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../063-python3-joblib_0.14.0-3_all.deb ... 2022-08-01T19:31:44.099 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-waitress. 2022-08-01T19:31:44.112 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../036-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2022-08-01T19:31:44.116 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-joblib (0.14.0-3) ... 2022-08-01T19:31:44.152 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2022-08-01T19:31:44.462 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-sklearn-lib. 2022-08-01T19:31:44.478 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../062-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2022-08-01T19:31:44.487 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-sklearn. 2022-08-01T19:31:44.500 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../064-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2022-08-01T19:31:44.510 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-tempita. 2022-08-01T19:31:44.520 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-08-01T19:31:44.524 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../037-python3-tempita_0.5.2-6_all.deb ... 2022-08-01T19:31:44.562 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-08-01T19:31:44.575 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package cephadm. 2022-08-01T19:31:44.577 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-tempita (0.5.2-6) ... 2022-08-01T19:31:44.589 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../050-cephadm_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:44.633 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking cephadm (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:44.905 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-paste. 2022-08-01T19:31:44.911 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-jinja2. 2022-08-01T19:31:44.917 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../038-python3-paste_3.4.0+dfsg1-1_all.deb ... 2022-08-01T19:31:44.923 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../051-python3-jinja2_2.10.1-2_all.deb ... 2022-08-01T19:31:44.952 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2022-08-01T19:31:44.954 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2022-08-01T19:31:45.033 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-joblib. 2022-08-01T19:31:45.048 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../063-python3-joblib_0.14.0-3_all.deb ... 2022-08-01T19:31:45.099 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-joblib (0.14.0-3) ... 2022-08-01T19:31:45.281 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2022-08-01T19:31:45.286 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2022-08-01T19:31:45.291 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2022-08-01T19:31:45.295 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../052-ceph-mgr-cephadm_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:31:45.302 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../065-ceph-mgr-diskprediction-local_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:31:45.306 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../039-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2022-08-01T19:31:45.336 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:45.339 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking ceph-mgr-cephadm (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:45.374 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2022-08-01T19:31:45.495 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-sklearn. 2022-08-01T19:31:45.508 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../064-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2022-08-01T19:31:45.545 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-08-01T19:31:45.610 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pastedeploy. 2022-08-01T19:31:45.625 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../040-python3-pastedeploy_2.1.0-1_all.deb ... 2022-08-01T19:31:45.668 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-repoze.lru. 2022-08-01T19:31:45.683 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../053-python3-repoze.lru_0.7-2_all.deb ... 2022-08-01T19:31:45.702 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2022-08-01T19:31:45.725 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2022-08-01T19:31:46.012 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package libjs-underscore. 2022-08-01T19:31:46.023 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-webtest. 2022-08-01T19:31:46.024 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../054-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2022-08-01T19:31:46.038 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../041-python3-webtest_2.0.32-2_all.deb ... 2022-08-01T19:31:46.061 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-08-01T19:31:46.081 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-webtest (2.0.32-2) ... 2022-08-01T19:31:46.229 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2022-08-01T19:31:46.245 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../065-ceph-mgr-diskprediction-local_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:31:46.320 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:46.417 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pecan. 2022-08-01T19:31:46.433 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../042-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2022-08-01T19:31:46.456 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package libjs-sphinxdoc. 2022-08-01T19:31:46.469 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../055-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2022-08-01T19:31:46.472 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-cachetools. 2022-08-01T19:31:46.475 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2022-08-01T19:31:46.489 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../066-python3-cachetools_4.0.0-1_all.deb ... 2022-08-01T19:31:46.530 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2022-08-01T19:31:46.539 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-08-01T19:31:46.839 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-werkzeug. 2022-08-01T19:31:46.841 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pyasn1. 2022-08-01T19:31:46.851 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../043-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2022-08-01T19:31:46.857 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../067-python3-pyasn1_0.4.2-3build1_all.deb ... 2022-08-01T19:31:46.888 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2022-08-01T19:31:46.892 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-routes. 2022-08-01T19:31:46.899 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2022-08-01T19:31:46.905 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../056-python3-routes_2.4.1-2_all.deb ... 2022-08-01T19:31:46.942 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-routes (2.4.1-2) ... 2022-08-01T19:31:47.278 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pyasn1-modules. 2022-08-01T19:31:47.279 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2022-08-01T19:31:47.292 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:31:47.294 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../068-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2022-08-01T19:31:47.333 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mgr. 2022-08-01T19:31:47.336 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-08-01T19:31:47.337 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking ceph-mgr-dashboard (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:47.346 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../044-ceph-mgr_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:47.383 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mgr (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:47.457 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-cachetools. 2022-08-01T19:31:47.473 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../066-python3-cachetools_4.0.0-1_all.deb ... 2022-08-01T19:31:47.506 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2022-08-01T19:31:47.671 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-rsa. 2022-08-01T19:31:47.688 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../069-python3-rsa_4.0-3_all.deb ... 2022-08-01T19:31:47.739 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-rsa (4.0-3) ... 2022-08-01T19:31:47.776 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-pyasn1. 2022-08-01T19:31:47.791 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../067-python3-pyasn1_0.4.2-3build1_all.deb ... 2022-08-01T19:31:47.835 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2022-08-01T19:31:47.911 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mon. 2022-08-01T19:31:47.926 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../045-ceph-mon_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:47.969 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mon (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:48.009 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-google-auth. 2022-08-01T19:31:48.025 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../070-python3-google-auth_1.5.1-2_all.deb ... 2022-08-01T19:31:48.058 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2022-08-01T19:31:48.188 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-pyasn1-modules. 2022-08-01T19:31:48.201 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../068-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2022-08-01T19:31:48.238 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-08-01T19:31:48.379 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2022-08-01T19:31:48.395 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../058-ceph-mgr-diskprediction-cloud_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:31:48.420 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-blinker. 2022-08-01T19:31:48.436 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../071-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2022-08-01T19:31:48.437 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:48.478 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-08-01T19:31:48.524 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-rsa. 2022-08-01T19:31:48.537 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../069-python3-rsa_4.0-3_all.deb ... 2022-08-01T19:31:48.574 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-rsa (4.0-3) ... 2022-08-01T19:31:48.790 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-decorator. 2022-08-01T19:31:48.806 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../059-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2022-08-01T19:31:48.814 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-oauthlib. 2022-08-01T19:31:48.827 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../072-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2022-08-01T19:31:48.864 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2022-08-01T19:31:48.890 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2022-08-01T19:31:48.911 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-google-auth. 2022-08-01T19:31:48.924 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../070-python3-google-auth_1.5.1-2_all.deb ... 2022-08-01T19:31:48.961 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2022-08-01T19:31:49.023 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-osd. 2022-08-01T19:31:49.036 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../046-ceph-osd_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:49.073 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-osd (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:49.201 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-requests-oauthlib. 2022-08-01T19:31:49.217 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../073-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2022-08-01T19:31:49.226 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2022-08-01T19:31:49.240 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../060-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2022-08-01T19:31:49.258 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-08-01T19:31:49.285 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-08-01T19:31:49.306 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-blinker. 2022-08-01T19:31:49.321 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../071-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2022-08-01T19:31:49.364 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-08-01T19:31:49.588 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-scipy. 2022-08-01T19:31:49.603 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../061-python3-scipy_1.3.3-3build1_amd64.deb ... 2022-08-01T19:31:49.611 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-websocket. 2022-08-01T19:31:49.624 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../074-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2022-08-01T19:31:49.637 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2022-08-01T19:31:49.642 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-oauthlib. 2022-08-01T19:31:49.658 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../072-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2022-08-01T19:31:49.661 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2022-08-01T19:31:49.692 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2022-08-01T19:31:49.998 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-kubernetes. 2022-08-01T19:31:50.011 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../075-python3-kubernetes_7.0.0~a1-2_all.deb ... 2022-08-01T19:31:50.021 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-requests-oauthlib. 2022-08-01T19:31:50.035 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../073-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2022-08-01T19:31:50.056 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2022-08-01T19:31:50.071 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-08-01T19:31:50.374 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-websocket. 2022-08-01T19:31:50.389 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../074-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2022-08-01T19:31:50.432 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2022-08-01T19:31:50.636 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2022-08-01T19:31:50.650 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:31:50.694 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-mgr-k8sevents (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:50.752 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-kubernetes. 2022-08-01T19:31:50.767 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../075-python3-kubernetes_7.0.0~a1-2_all.deb ... 2022-08-01T19:31:50.810 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2022-08-01T19:31:50.980 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libonig5:amd64. 2022-08-01T19:31:50.996 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../077-libonig5_6.9.4-1_amd64.deb ... 2022-08-01T19:31:51.013 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-sklearn-lib. 2022-08-01T19:31:51.030 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2022-08-01T19:31:51.030 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../062-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2022-08-01T19:31:51.063 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-08-01T19:31:51.349 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2022-08-01T19:31:51.363 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:31:51.391 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libjq1:amd64. 2022-08-01T19:31:51.407 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking ceph-mgr-k8sevents (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:51.408 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../078-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-08-01T19:31:51.449 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-08-01T19:31:51.545 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph. 2022-08-01T19:31:51.559 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../047-ceph_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:51.602 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:51.693 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-joblib. 2022-08-01T19:31:51.716 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../063-python3-joblib_0.14.0-3_all.deb ... 2022-08-01T19:31:51.742 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-joblib (0.14.0-3) ... 2022-08-01T19:31:51.751 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package jq. 2022-08-01T19:31:51.766 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../079-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-08-01T19:31:51.768 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package libonig5:amd64. 2022-08-01T19:31:51.783 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../077-libonig5_6.9.4-1_amd64.deb ... 2022-08-01T19:31:51.810 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2022-08-01T19:31:51.826 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2022-08-01T19:31:51.905 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-fuse. 2022-08-01T19:31:51.917 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../048-ceph-fuse_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:51.955 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-fuse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:52.062 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package socat. 2022-08-01T19:31:52.078 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../080-socat_1.7.3.3-2_amd64.deb ... 2022-08-01T19:31:52.120 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking socat (1.7.3.3-2) ... 2022-08-01T19:31:52.171 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-sklearn. 2022-08-01T19:31:52.185 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../064-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2022-08-01T19:31:52.222 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package libjq1:amd64. 2022-08-01T19:31:52.229 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-08-01T19:31:52.235 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../078-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-08-01T19:31:52.280 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-08-01T19:31:52.283 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mds. 2022-08-01T19:31:52.298 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../049-ceph-mds_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:52.349 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mds (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:52.524 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package jq. 2022-08-01T19:31:52.540 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../079-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-08-01T19:31:52.574 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2022-08-01T19:31:52.626 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libxslt1.1:amd64. 2022-08-01T19:31:52.641 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../081-libxslt1.1_1.1.34-4_amd64.deb ... 2022-08-01T19:31:52.684 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libxslt1.1:amd64 (1.1.34-4) ... 2022-08-01T19:31:52.860 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package socat. 2022-08-01T19:31:52.874 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../080-socat_1.7.3.3-2_amd64.deb ... 2022-08-01T19:31:52.877 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package cephadm. 2022-08-01T19:31:52.891 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../050-cephadm_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:52.919 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking socat (1.7.3.3-2) ... 2022-08-01T19:31:52.954 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package xmlstarlet. 2022-08-01T19:31:52.960 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking cephadm (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:52.969 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../082-xmlstarlet_1.6.1-2build1_amd64.deb ... 2022-08-01T19:31:53.012 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2022-08-01T19:31:53.030 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2022-08-01T19:31:53.042 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../065-ceph-mgr-diskprediction-local_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:31:53.079 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:53.289 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-jinja2. 2022-08-01T19:31:53.304 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../051-python3-jinja2_2.10.1-2_all.deb ... 2022-08-01T19:31:53.338 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2022-08-01T19:31:53.366 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-test. 2022-08-01T19:31:53.381 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../083-ceph-test_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:53.400 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package libxslt1.1:amd64. 2022-08-01T19:31:53.416 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../081-libxslt1.1_1.1.34-4_amd64.deb ... 2022-08-01T19:31:53.424 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-test (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:53.458 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking libxslt1.1:amd64 (1.1.34-4) ... 2022-08-01T19:31:53.649 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2022-08-01T19:31:53.662 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../052-ceph-mgr-cephadm_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:31:53.699 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mgr-cephadm (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:53.728 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package xmlstarlet. 2022-08-01T19:31:53.744 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../082-xmlstarlet_1.6.1-2build1_amd64.deb ... 2022-08-01T19:31:53.820 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2022-08-01T19:31:54.019 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-repoze.lru. 2022-08-01T19:31:54.032 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../053-python3-repoze.lru_0.7-2_all.deb ... 2022-08-01T19:31:54.069 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2022-08-01T19:31:54.158 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package ceph-test. 2022-08-01T19:31:54.173 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../083-ceph-test_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:54.215 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking ceph-test (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:54.250 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-cachetools. 2022-08-01T19:31:54.262 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../066-python3-cachetools_4.0.0-1_all.deb ... 2022-08-01T19:31:54.299 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2022-08-01T19:31:54.355 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libjs-underscore. 2022-08-01T19:31:54.368 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../054-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2022-08-01T19:31:54.405 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-08-01T19:31:54.619 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-pyasn1. 2022-08-01T19:31:54.633 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../067-python3-pyasn1_0.4.2-3build1_all.deb ... 2022-08-01T19:31:54.669 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2022-08-01T19:31:54.699 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libjs-sphinxdoc. 2022-08-01T19:31:54.714 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../055-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2022-08-01T19:31:54.757 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-08-01T19:31:55.006 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-pyasn1-modules. 2022-08-01T19:31:55.019 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../068-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2022-08-01T19:31:55.055 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-08-01T19:31:55.093 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-routes. 2022-08-01T19:31:55.106 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../056-python3-routes_2.4.1-2_all.deb ... 2022-08-01T19:31:55.143 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-routes (2.4.1-2) ... 2022-08-01T19:31:55.400 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-rsa. 2022-08-01T19:31:55.413 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../069-python3-rsa_4.0-3_all.deb ... 2022-08-01T19:31:55.450 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-rsa (4.0-3) ... 2022-08-01T19:31:55.554 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2022-08-01T19:31:55.568 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:31:55.604 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mgr-dashboard (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:55.787 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-google-auth. 2022-08-01T19:31:55.803 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../070-python3-google-auth_1.5.1-2_all.deb ... 2022-08-01T19:31:55.870 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2022-08-01T19:31:56.240 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-blinker. 2022-08-01T19:31:56.253 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../071-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2022-08-01T19:31:56.298 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-08-01T19:31:56.585 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-oauthlib. 2022-08-01T19:31:56.598 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../072-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2022-08-01T19:31:56.634 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2022-08-01T19:31:56.678 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2022-08-01T19:31:56.694 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../058-ceph-mgr-diskprediction-cloud_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:31:56.728 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:56.922 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-requests-oauthlib. 2022-08-01T19:31:56.937 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../073-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2022-08-01T19:31:56.971 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-08-01T19:31:57.048 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-decorator. 2022-08-01T19:31:57.064 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../059-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2022-08-01T19:31:57.106 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2022-08-01T19:31:57.120 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-colorama. 2022-08-01T19:31:57.134 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../084-python3-colorama_0.4.3-1build1_all.deb ... 2022-08-01T19:31:57.178 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2022-08-01T19:31:57.291 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-websocket. 2022-08-01T19:31:57.305 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../074-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2022-08-01T19:31:57.349 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2022-08-01T19:31:57.425 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2022-08-01T19:31:57.439 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../060-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2022-08-01T19:31:57.481 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package cephfs-shell. 2022-08-01T19:31:57.495 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../085-cephfs-shell_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:31:57.517 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-08-01T19:31:57.539 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking cephfs-shell (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:57.628 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-colorama. 2022-08-01T19:31:57.641 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../084-python3-colorama_0.4.3-1build1_all.deb ... 2022-08-01T19:31:57.678 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2022-08-01T19:31:57.703 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-kubernetes. 2022-08-01T19:31:57.717 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../075-python3-kubernetes_7.0.0~a1-2_all.deb ... 2022-08-01T19:31:57.786 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2022-08-01T19:31:57.791 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libcephfs-dev. 2022-08-01T19:31:57.808 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../086-libcephfs-dev_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:57.841 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libcephfs-dev (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:57.878 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-scipy. 2022-08-01T19:31:57.891 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../061-python3-scipy_1.3.3-3build1_amd64.deb ... 2022-08-01T19:31:57.936 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2022-08-01T19:31:57.956 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package cephfs-shell. 2022-08-01T19:31:57.972 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../085-cephfs-shell_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:31:58.014 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking cephfs-shell (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:58.226 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package libcephfs-dev. 2022-08-01T19:31:58.236 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package nvme-cli. 2022-08-01T19:31:58.241 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../086-libcephfs-dev_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:31:58.252 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../087-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2022-08-01T19:31:58.275 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking libcephfs-dev (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:58.312 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2022-08-01T19:31:58.333 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2022-08-01T19:31:58.347 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:31:58.424 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking ceph-mgr-k8sevents (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:31:58.695 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package nvme-cli. 2022-08-01T19:31:58.709 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../087-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2022-08-01T19:31:58.741 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-atomicwrites. 2022-08-01T19:31:58.755 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../088-python3-atomicwrites_1.1.5-2build1_all.deb ... 2022-08-01T19:31:58.769 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package libonig5:amd64. 2022-08-01T19:31:58.769 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2022-08-01T19:31:58.782 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../077-libonig5_6.9.4-1_amd64.deb ... 2022-08-01T19:31:58.800 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2022-08-01T19:31:58.827 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2022-08-01T19:31:59.111 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-attr. 2022-08-01T19:31:59.124 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../089-python3-attr_19.3.0-2_all.deb ... 2022-08-01T19:31:59.147 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package libjq1:amd64. 2022-08-01T19:31:59.160 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-attr (19.3.0-2) ... 2022-08-01T19:31:59.161 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../078-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-08-01T19:31:59.168 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-atomicwrites. 2022-08-01T19:31:59.185 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../088-python3-atomicwrites_1.1.5-2build1_all.deb ... 2022-08-01T19:31:59.203 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-sklearn-lib. 2022-08-01T19:31:59.205 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-08-01T19:31:59.219 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../062-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2022-08-01T19:31:59.226 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2022-08-01T19:31:59.253 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-08-01T19:31:59.479 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-attr. 2022-08-01T19:31:59.492 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../089-python3-attr_19.3.0-2_all.deb ... 2022-08-01T19:31:59.508 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package jq. 2022-08-01T19:31:59.513 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-webencodings. 2022-08-01T19:31:59.521 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../079-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-08-01T19:31:59.527 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../090-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2022-08-01T19:31:59.529 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-attr (19.3.0-2) ... 2022-08-01T19:31:59.566 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2022-08-01T19:31:59.571 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2022-08-01T19:31:59.782 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-joblib. 2022-08-01T19:31:59.797 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../063-python3-joblib_0.14.0-3_all.deb ... 2022-08-01T19:31:59.831 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-joblib (0.14.0-3) ... 2022-08-01T19:31:59.832 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-webencodings. 2022-08-01T19:31:59.847 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../090-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2022-08-01T19:31:59.852 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package socat. 2022-08-01T19:31:59.866 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../080-socat_1.7.3.3-2_amd64.deb ... 2022-08-01T19:31:59.890 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2022-08-01T19:31:59.895 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-html5lib. 2022-08-01T19:31:59.911 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../091-python3-html5lib_1.0.1-2_all.deb ... 2022-08-01T19:31:59.911 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking socat (1.7.3.3-2) ... 2022-08-01T19:31:59.957 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2022-08-01T19:32:00.227 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-html5lib. 2022-08-01T19:32:00.240 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../091-python3-html5lib_1.0.1-2_all.deb ... 2022-08-01T19:32:00.243 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-sklearn. 2022-08-01T19:32:00.256 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../064-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2022-08-01T19:32:00.285 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2022-08-01T19:32:00.293 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-08-01T19:32:00.302 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-lxml:amd64. 2022-08-01T19:32:00.316 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../092-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2022-08-01T19:32:00.393 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-08-01T19:32:00.442 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package libxslt1.1:amd64. 2022-08-01T19:32:00.455 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../081-libxslt1.1_1.1.34-4_amd64.deb ... 2022-08-01T19:32:00.491 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking libxslt1.1:amd64 (1.1.34-4) ... 2022-08-01T19:32:00.630 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-lxml:amd64. 2022-08-01T19:32:00.645 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../092-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2022-08-01T19:32:00.688 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-08-01T19:32:00.803 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package xmlstarlet. 2022-08-01T19:32:00.817 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../082-xmlstarlet_1.6.1-2build1_amd64.deb ... 2022-08-01T19:32:00.861 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2022-08-01T19:32:00.889 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-packaging. 2022-08-01T19:32:00.903 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../093-python3-packaging_20.3-1_all.deb ... 2022-08-01T19:32:00.947 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-packaging (20.3-1) ... 2022-08-01T19:32:00.984 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2022-08-01T19:32:01.001 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../065-ceph-mgr-diskprediction-local_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:32:01.042 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:01.142 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-packaging. 2022-08-01T19:32:01.155 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../093-python3-packaging_20.3-1_all.deb ... 2022-08-01T19:32:01.200 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-packaging (20.3-1) ... 2022-08-01T19:32:01.233 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package ceph-test. 2022-08-01T19:32:01.246 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../083-ceph-test_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:32:01.283 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pastescript. 2022-08-01T19:32:01.296 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../094-python3-pastescript_2.0.2-4_all.deb ... 2022-08-01T19:32:01.324 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking ceph-test (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:01.332 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2022-08-01T19:32:01.512 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-pastescript. 2022-08-01T19:32:01.525 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../094-python3-pastescript_2.0.2-4_all.deb ... 2022-08-01T19:32:01.561 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2022-08-01T19:32:01.919 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pluggy. 2022-08-01T19:32:01.933 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../095-python3-pluggy_0.13.0-2_all.deb ... 2022-08-01T19:32:01.969 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2022-08-01T19:32:02.065 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-pluggy. 2022-08-01T19:32:02.081 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../095-python3-pluggy_0.13.0-2_all.deb ... 2022-08-01T19:32:02.204 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-cachetools. 2022-08-01T19:32:02.219 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../066-python3-cachetools_4.0.0-1_all.deb ... 2022-08-01T19:32:02.223 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2022-08-01T19:32:02.262 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2022-08-01T19:32:02.308 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-psutil. 2022-08-01T19:32:02.321 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../096-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2022-08-01T19:32:02.357 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2022-08-01T19:32:02.577 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-psutil. 2022-08-01T19:32:02.591 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../096-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2022-08-01T19:32:02.598 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pyasn1. 2022-08-01T19:32:02.611 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../067-python3-pyasn1_0.4.2-3build1_all.deb ... 2022-08-01T19:32:02.635 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2022-08-01T19:32:02.656 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2022-08-01T19:32:02.685 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-py. 2022-08-01T19:32:02.701 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../097-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2022-08-01T19:32:02.743 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2022-08-01T19:32:02.971 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-py. 2022-08-01T19:32:02.984 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pyasn1-modules. 2022-08-01T19:32:02.986 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../097-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2022-08-01T19:32:02.997 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../068-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2022-08-01T19:32:03.034 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-08-01T19:32:03.055 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pyinotify. 2022-08-01T19:32:03.062 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2022-08-01T19:32:03.070 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../098-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2022-08-01T19:32:03.139 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-08-01T19:32:03.354 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-rsa. 2022-08-01T19:32:03.368 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../069-python3-rsa_4.0-3_all.deb ... 2022-08-01T19:32:03.383 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-pyinotify. 2022-08-01T19:32:03.398 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../098-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2022-08-01T19:32:03.411 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-rsa (4.0-3) ... 2022-08-01T19:32:03.441 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-08-01T19:32:03.450 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pytest. 2022-08-01T19:32:03.463 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../099-python3-pytest_4.6.9-1_all.deb ... 2022-08-01T19:32:03.508 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pytest (4.6.9-1) ... 2022-08-01T19:32:03.715 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-google-auth. 2022-08-01T19:32:03.730 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../070-python3-google-auth_1.5.1-2_all.deb ... 2022-08-01T19:32:03.753 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-pytest. 2022-08-01T19:32:03.769 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../099-python3-pytest_4.6.9-1_all.deb ... 2022-08-01T19:32:03.806 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2022-08-01T19:32:03.810 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-pytest (4.6.9-1) ... 2022-08-01T19:32:03.878 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-simplejson. 2022-08-01T19:32:03.894 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../100-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2022-08-01T19:32:03.936 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2022-08-01T19:32:04.143 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-blinker. 2022-08-01T19:32:04.156 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package python3-simplejson. 2022-08-01T19:32:04.158 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../071-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2022-08-01T19:32:04.171 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../100-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2022-08-01T19:32:04.201 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-08-01T19:32:04.231 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2022-08-01T19:32:04.407 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package radosgw. 2022-08-01T19:32:04.421 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../101-radosgw_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:32:04.487 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking radosgw (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:04.529 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-oauthlib. 2022-08-01T19:32:04.542 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../072-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2022-08-01T19:32:04.578 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2022-08-01T19:32:04.618 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package radosgw. 2022-08-01T19:32:04.632 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../101-radosgw_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:32:04.676 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking radosgw (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:04.780 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-colorama. 2022-08-01T19:32:04.794 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../084-python3-colorama_0.4.3-1build1_all.deb ... 2022-08-01T19:32:04.863 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2022-08-01T19:32:04.949 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-requests-oauthlib. 2022-08-01T19:32:04.963 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../073-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2022-08-01T19:32:05.007 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-08-01T19:32:05.116 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package cephfs-shell. 2022-08-01T19:32:05.130 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../085-cephfs-shell_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:32:05.199 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking cephfs-shell (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:05.385 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-websocket. 2022-08-01T19:32:05.401 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../074-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2022-08-01T19:32:05.402 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package libcephfs-dev. 2022-08-01T19:32:05.417 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../086-libcephfs-dev_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:32:05.443 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2022-08-01T19:32:05.452 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking libcephfs-dev (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:05.610 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package rbd-fuse. 2022-08-01T19:32:05.624 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../102-rbd-fuse_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:32:05.668 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking rbd-fuse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:05.730 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package rbd-fuse. 2022-08-01T19:32:05.743 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../102-rbd-fuse_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:32:05.762 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-kubernetes. 2022-08-01T19:32:05.776 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../075-python3-kubernetes_7.0.0~a1-2_all.deb ... 2022-08-01T19:32:05.788 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking rbd-fuse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:05.813 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2022-08-01T19:32:05.914 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package nvme-cli. 2022-08-01T19:32:05.929 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../087-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2022-08-01T19:32:05.989 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2022-08-01T19:32:05.992 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-attr (19.3.0-2) ... 2022-08-01T19:32:06.103 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-attr (19.3.0-2) ... 2022-08-01T19:32:06.315 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-08-01T19:32:06.370 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-atomicwrites. 2022-08-01T19:32:06.384 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2022-08-01T19:32:06.385 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../088-python3-atomicwrites_1.1.5-2build1_all.deb ... 2022-08-01T19:32:06.394 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-08-01T19:32:06.397 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:32:06.420 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2022-08-01T19:32:06.432 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2022-08-01T19:32:06.433 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mgr-k8sevents (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:06.533 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2022-08-01T19:32:06.554 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2022-08-01T19:32:06.655 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2022-08-01T19:32:06.715 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-attr. 2022-08-01T19:32:06.727 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up cephadm (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:06.729 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../089-python3-attr_19.3.0-2_all.deb ... 2022-08-01T19:32:06.761 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libonig5:amd64. 2022-08-01T19:32:06.775 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../077-libonig5_6.9.4-1_amd64.deb ... 2022-08-01T19:32:06.814 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-attr (19.3.0-2) ... 2022-08-01T19:32:06.815 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up cephadm (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:06.819 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2022-08-01T19:32:07.159 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-webencodings. 2022-08-01T19:32:07.173 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../090-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2022-08-01T19:32:07.214 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libjq1:amd64. 2022-08-01T19:32:07.217 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2022-08-01T19:32:07.230 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../078-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-08-01T19:32:07.272 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-08-01T19:32:07.371 INFO:teuthology.orchestra.run.smithi017.stdout:Adding system user cephadm....done 2022-08-01T19:32:07.418 INFO:teuthology.orchestra.run.smithi085.stdout:Adding system user cephadm....done 2022-08-01T19:32:07.541 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2022-08-01T19:32:07.554 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-html5lib. 2022-08-01T19:32:07.563 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2022-08-01T19:32:07.569 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../091-python3-html5lib_1.0.1-2_all.deb ... 2022-08-01T19:32:07.575 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package jq. 2022-08-01T19:32:07.588 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../079-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-08-01T19:32:07.612 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2022-08-01T19:32:07.666 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2022-08-01T19:32:07.903 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package socat. 2022-08-01T19:32:07.906 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-repoze.lru (0.7-2) ... 2022-08-01T19:32:07.909 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-repoze.lru (0.7-2) ... 2022-08-01T19:32:07.916 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../080-socat_1.7.3.3-2_amd64.deb ... 2022-08-01T19:32:07.974 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-lxml:amd64. 2022-08-01T19:32:07.985 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking socat (1.7.3.3-2) ... 2022-08-01T19:32:07.987 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../092-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2022-08-01T19:32:08.024 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-08-01T19:32:08.166 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up liboath0 (2.6.1-1.3) ... 2022-08-01T19:32:08.185 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up liboath0 (2.6.1-1.3) ... 2022-08-01T19:32:08.294 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2022-08-01T19:32:08.325 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2022-08-01T19:32:08.450 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libxslt1.1:amd64. 2022-08-01T19:32:08.464 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../081-libxslt1.1_1.1.34-4_amd64.deb ... 2022-08-01T19:32:08.494 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-packaging. 2022-08-01T19:32:08.508 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libxslt1.1:amd64 (1.1.34-4) ... 2022-08-01T19:32:08.510 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../093-python3-packaging_20.3-1_all.deb ... 2022-08-01T19:32:08.552 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-packaging (20.3-1) ... 2022-08-01T19:32:08.621 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-joblib (0.14.0-3) ... 2022-08-01T19:32:08.652 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-joblib (0.14.0-3) ... 2022-08-01T19:32:08.811 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package xmlstarlet. 2022-08-01T19:32:08.826 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../082-xmlstarlet_1.6.1-2build1_amd64.deb ... 2022-08-01T19:32:08.830 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-pastescript. 2022-08-01T19:32:08.843 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../094-python3-pastescript_2.0.2-4_all.deb ... 2022-08-01T19:32:08.869 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2022-08-01T19:32:08.871 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2022-08-01T19:32:09.023 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2022-08-01T19:32:09.070 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2022-08-01T19:32:09.208 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-pluggy. 2022-08-01T19:32:09.222 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../095-python3-pluggy_0.13.0-2_all.deb ... 2022-08-01T19:32:09.267 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2022-08-01T19:32:09.290 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-test. 2022-08-01T19:32:09.304 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../083-ceph-test_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:32:09.316 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-cachetools (4.0.0-1) ... 2022-08-01T19:32:09.348 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-test (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:09.405 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-cachetools (4.0.0-1) ... 2022-08-01T19:32:09.603 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-psutil. 2022-08-01T19:32:09.616 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../096-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2022-08-01T19:32:09.617 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2022-08-01T19:32:09.686 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2022-08-01T19:32:09.689 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2022-08-01T19:32:09.893 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2022-08-01T19:32:09.982 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2022-08-01T19:32:10.031 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-py. 2022-08-01T19:32:10.044 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../097-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2022-08-01T19:32:10.081 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2022-08-01T19:32:10.144 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-08-01T19:32:10.250 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-08-01T19:32:10.437 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-ceph-argparse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:10.451 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-pyinotify. 2022-08-01T19:32:10.465 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../098-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2022-08-01T19:32:10.509 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-08-01T19:32:10.552 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-ceph-argparse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:10.755 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-08-01T19:32:10.821 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-pytest. 2022-08-01T19:32:10.836 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../099-python3-pytest_4.6.9-1_all.deb ... 2022-08-01T19:32:10.853 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-08-01T19:32:10.870 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-pytest (4.6.9-1) ... 2022-08-01T19:32:10.880 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2022-08-01T19:32:10.979 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2022-08-01T19:32:11.199 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2022-08-01T19:32:11.263 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2022-08-01T19:32:11.282 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package python3-simplejson. 2022-08-01T19:32:11.295 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../100-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2022-08-01T19:32:11.333 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2022-08-01T19:32:11.475 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2022-08-01T19:32:11.548 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2022-08-01T19:32:11.734 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2022-08-01T19:32:11.770 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package radosgw. 2022-08-01T19:32:11.782 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../101-radosgw_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:32:11.811 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking radosgw (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:11.833 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2022-08-01T19:32:12.177 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2022-08-01T19:32:12.192 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2022-08-01T19:32:12.445 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2022-08-01T19:32:12.585 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2022-08-01T19:32:12.830 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2022-08-01T19:32:12.852 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-colorama. 2022-08-01T19:32:12.864 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../084-python3-colorama_0.4.3-1build1_all.deb ... 2022-08-01T19:32:12.898 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package rbd-fuse. 2022-08-01T19:32:12.914 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../102-rbd-fuse_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:32:12.920 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2022-08-01T19:32:12.935 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2022-08-01T19:32:12.956 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking rbd-fuse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:13.122 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2022-08-01T19:32:13.162 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package cephfs-shell. 2022-08-01T19:32:13.175 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../085-cephfs-shell_15.2.16-343-gfd96719b-1focal_all.deb ... 2022-08-01T19:32:13.204 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2022-08-01T19:32:13.237 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking cephfs-shell (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:13.288 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-attr (19.3.0-2) ... 2022-08-01T19:32:13.440 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libcephfs-dev. 2022-08-01T19:32:13.454 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../086-libcephfs-dev_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:32:13.465 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-tempita (0.5.2-6) ... 2022-08-01T19:32:13.498 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libcephfs-dev (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:13.547 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-tempita (0.5.2-6) ... 2022-08-01T19:32:13.603 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-08-01T19:32:13.738 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2022-08-01T19:32:13.758 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-packaging (20.3-1) ... 2022-08-01T19:32:13.824 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-packaging (20.3-1) ... 2022-08-01T19:32:13.897 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2022-08-01T19:32:13.951 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package nvme-cli. 2022-08-01T19:32:13.966 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../087-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2022-08-01T19:32:14.027 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2022-08-01T19:32:14.059 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2022-08-01T19:32:14.099 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up cephadm (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:14.125 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2022-08-01T19:32:14.160 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2022-08-01T19:32:14.251 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2022-08-01T19:32:14.436 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2022-08-01T19:32:14.440 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-atomicwrites. 2022-08-01T19:32:14.454 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../088-python3-atomicwrites_1.1.5-2build1_all.deb ... 2022-08-01T19:32:14.490 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2022-08-01T19:32:14.594 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2022-08-01T19:32:14.768 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-attr. 2022-08-01T19:32:14.780 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../089-python3-attr_19.3.0-2_all.deb ... 2022-08-01T19:32:14.787 INFO:teuthology.orchestra.run.smithi077.stdout:Adding system user cephadm....done 2022-08-01T19:32:14.809 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-attr (19.3.0-2) ... 2022-08-01T19:32:14.846 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2022-08-01T19:32:14.922 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2022-08-01T19:32:14.987 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2022-08-01T19:32:15.147 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2022-08-01T19:32:15.195 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-webencodings. 2022-08-01T19:32:15.210 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../090-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2022-08-01T19:32:15.253 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2022-08-01T19:32:15.274 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-repoze.lru (0.7-2) ... 2022-08-01T19:32:15.288 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2022-08-01T19:32:15.557 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2022-08-01T19:32:15.558 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up liboath0 (2.6.1-1.3) ... 2022-08-01T19:32:15.590 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-html5lib. 2022-08-01T19:32:15.603 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../091-python3-html5lib_1.0.1-2_all.deb ... 2022-08-01T19:32:15.648 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2022-08-01T19:32:15.659 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2022-08-01T19:32:15.665 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2022-08-01T19:32:15.951 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up socat (1.7.3.3-2) ... 2022-08-01T19:32:15.975 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-lxml:amd64. 2022-08-01T19:32:15.988 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../092-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2022-08-01T19:32:15.994 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-joblib (0.14.0-3) ... 2022-08-01T19:32:16.025 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-08-01T19:32:16.050 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up socat (1.7.3.3-2) ... 2022-08-01T19:32:16.052 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-ceph-common (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:16.176 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-ceph-common (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:16.378 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-html5lib (1.0.1-2) ... 2022-08-01T19:32:16.455 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2022-08-01T19:32:16.470 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-html5lib (1.0.1-2) ... 2022-08-01T19:32:16.529 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-packaging. 2022-08-01T19:32:16.543 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../093-python3-packaging_20.3-1_all.deb ... 2022-08-01T19:32:16.587 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-packaging (20.3-1) ... 2022-08-01T19:32:16.730 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-08-01T19:32:16.748 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-cachetools (4.0.0-1) ... 2022-08-01T19:32:16.804 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-08-01T19:32:16.856 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pluggy (0.13.0-2) ... 2022-08-01T19:32:16.948 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pastescript. 2022-08-01T19:32:16.963 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../094-python3-pastescript_2.0.2-4_all.deb ... 2022-08-01T19:32:16.964 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-pluggy (0.13.0-2) ... 2022-08-01T19:32:17.006 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2022-08-01T19:32:17.058 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2022-08-01T19:32:17.149 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libxslt1.1:amd64 (1.1.34-4) ... 2022-08-01T19:32:17.257 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up libxslt1.1:amd64 (1.1.34-4) ... 2022-08-01T19:32:17.292 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2022-08-01T19:32:17.367 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2022-08-01T19:32:17.368 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pluggy. 2022-08-01T19:32:17.383 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2022-08-01T19:32:17.384 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../095-python3-pluggy_0.13.0-2_all.deb ... 2022-08-01T19:32:17.434 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2022-08-01T19:32:17.601 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2022-08-01T19:32:17.657 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-08-01T19:32:17.728 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2022-08-01T19:32:17.762 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-psutil. 2022-08-01T19:32:17.778 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../096-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2022-08-01T19:32:17.820 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2022-08-01T19:32:17.911 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-logutils (0.3.3-7) ... 2022-08-01T19:32:17.928 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-ceph-argparse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:18.029 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-logutils (0.3.3-7) ... 2022-08-01T19:32:18.173 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-py. 2022-08-01T19:32:18.178 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2022-08-01T19:32:18.187 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../097-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2022-08-01T19:32:18.231 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2022-08-01T19:32:18.254 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-08-01T19:32:18.331 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2022-08-01T19:32:18.332 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2022-08-01T19:32:18.422 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2022-08-01T19:32:18.457 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2022-08-01T19:32:18.584 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pyinotify. 2022-08-01T19:32:18.597 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-prettytable (0.7.2-5) ... 2022-08-01T19:32:18.599 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../098-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2022-08-01T19:32:18.642 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-08-01T19:32:18.723 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2022-08-01T19:32:18.741 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-prettytable (0.7.2-5) ... 2022-08-01T19:32:18.882 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2022-08-01T19:32:19.006 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pytest. 2022-08-01T19:32:19.016 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2022-08-01T19:32:19.022 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../099-python3-pytest_4.6.9-1_all.deb ... 2022-08-01T19:32:19.026 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2022-08-01T19:32:19.070 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pytest (4.6.9-1) ... 2022-08-01T19:32:19.166 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2022-08-01T19:32:19.300 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2022-08-01T19:32:19.327 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2022-08-01T19:32:19.449 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-simplejson. 2022-08-01T19:32:19.465 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../100-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2022-08-01T19:32:19.507 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2022-08-01T19:32:19.510 INFO:teuthology.orchestra.run.smithi017.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2022-08-01T19:32:19.614 INFO:teuthology.orchestra.run.smithi085.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2022-08-01T19:32:19.652 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2022-08-01T19:32:19.760 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2022-08-01T19:32:19.855 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2022-08-01T19:32:19.861 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-08-01T19:32:19.928 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2022-08-01T19:32:19.981 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-08-01T19:32:19.985 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package radosgw. 2022-08-01T19:32:20.001 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../101-radosgw_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:32:20.043 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking radosgw (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:20.180 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2022-08-01T19:32:20.313 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2022-08-01T19:32:20.341 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2022-08-01T19:32:20.522 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-08-01T19:32:20.597 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2022-08-01T19:32:20.682 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-08-01T19:32:20.692 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-08-01T19:32:20.808 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-webob (1:1.8.5-2) ... 2022-08-01T19:32:20.818 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-08-01T19:32:20.940 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-tempita (0.5.2-6) ... 2022-08-01T19:32:20.953 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-webob (1:1.8.5-2) ... 2022-08-01T19:32:21.112 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package rbd-fuse. 2022-08-01T19:32:21.127 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../102-rbd-fuse_15.2.16-343-gfd96719b-1focal_amd64.deb ... 2022-08-01T19:32:21.151 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-08-01T19:32:21.170 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking rbd-fuse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:21.217 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-packaging (20.3-1) ... 2022-08-01T19:32:21.254 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-08-01T19:32:21.419 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up librados2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:21.505 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-attr (19.3.0-2) ... 2022-08-01T19:32:21.526 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2022-08-01T19:32:21.531 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up librados2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:21.553 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2022-08-01T19:32:21.627 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2022-08-01T19:32:21.657 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2022-08-01T19:32:21.825 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-08-01T19:32:21.904 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2022-08-01T19:32:21.984 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2022-08-01T19:32:22.110 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2022-08-01T19:32:22.314 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2022-08-01T19:32:22.338 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up cephadm (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:22.623 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2022-08-01T19:32:22.956 INFO:teuthology.orchestra.run.smithi190.stdout:Adding system user cephadm....done 2022-08-01T19:32:23.000 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2022-08-01T19:32:23.059 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up librgw2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:23.096 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up librgw2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:23.118 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2022-08-01T19:32:23.160 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-08-01T19:32:23.256 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-08-01T19:32:23.294 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pytest (4.6.9-1) ... 2022-08-01T19:32:23.382 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-pytest (4.6.9-1) ... 2022-08-01T19:32:23.443 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up socat (1.7.3.3-2) ... 2022-08-01T19:32:23.510 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-repoze.lru (0.7-2) ... 2022-08-01T19:32:23.553 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-ceph-common (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:23.654 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2022-08-01T19:32:23.792 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2022-08-01T19:32:23.838 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-html5lib (1.0.1-2) ... 2022-08-01T19:32:23.845 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up liboath0 (2.6.1-1.3) ... 2022-08-01T19:32:23.971 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2022-08-01T19:32:23.980 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2022-08-01T19:32:24.101 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2022-08-01T19:32:24.189 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-08-01T19:32:24.273 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-bs4 (4.8.2-1) ... 2022-08-01T19:32:24.299 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-joblib (0.14.0-3) ... 2022-08-01T19:32:24.315 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-pluggy (0.13.0-2) ... 2022-08-01T19:32:24.386 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-bs4 (4.8.2-1) ... 2022-08-01T19:32:24.641 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libcephfs2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:24.650 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up libxslt1.1:amd64 (1.1.34-4) ... 2022-08-01T19:32:24.713 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up libcephfs2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:24.718 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2022-08-01T19:32:24.767 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2022-08-01T19:32:24.801 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2022-08-01T19:32:24.839 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2022-08-01T19:32:25.053 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-cachetools (4.0.0-1) ... 2022-08-01T19:32:25.086 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libradosstriper1 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:25.111 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2022-08-01T19:32:25.187 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-08-01T19:32:25.190 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up libradosstriper1 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:25.317 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-08-01T19:32:25.370 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-logutils (0.3.3-7) ... 2022-08-01T19:32:25.379 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2022-08-01T19:32:25.530 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-08-01T19:32:25.660 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-08-01T19:32:25.688 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2022-08-01T19:32:25.713 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2022-08-01T19:32:25.814 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2022-08-01T19:32:26.006 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-08-01T19:32:26.099 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-prettytable (0.7.2-5) ... 2022-08-01T19:32:26.316 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-ceph-argparse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:26.408 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2022-08-01T19:32:26.602 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up librbd1 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:26.633 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-08-01T19:32:26.691 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up librbd1 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:26.703 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-08-01T19:32:26.717 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2022-08-01T19:32:26.793 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2022-08-01T19:32:26.804 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2022-08-01T19:32:26.817 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-08-01T19:32:26.943 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2022-08-01T19:32:26.963 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-08-01T19:32:27.028 INFO:teuthology.orchestra.run.smithi077.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2022-08-01T19:32:27.061 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-08-01T19:32:27.086 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2022-08-01T19:32:27.323 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2022-08-01T19:32:27.328 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2022-08-01T19:32:27.354 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2022-08-01T19:32:27.395 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2022-08-01T19:32:27.447 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-08-01T19:32:27.466 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-rsa (4.0-3) ... 2022-08-01T19:32:27.480 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-rsa (4.0-3) ... 2022-08-01T19:32:27.713 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2022-08-01T19:32:27.773 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2022-08-01T19:32:27.773 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-webtest (2.0.32-2) ... 2022-08-01T19:32:27.783 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-webtest (2.0.32-2) ... 2022-08-01T19:32:28.089 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2022-08-01T19:32:28.099 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up ceph-mgr-modules-core (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:28.101 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-mgr-modules-core (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:28.141 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-08-01T19:32:28.225 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up ceph-fuse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:28.236 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-fuse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:28.292 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-08-01T19:32:28.341 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2022-08-01T19:32:28.418 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-webob (1:1.8.5-2) ... 2022-08-01T19:32:28.443 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-08-01T19:32:28.444 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-08-01T19:32:28.445 INFO:teuthology.orchestra.run.smithi085.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-08-01T19:32:28.445 INFO:teuthology.orchestra.run.smithi085.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-08-01T19:32:28.734 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2022-08-01T19:32:28.744 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-08-01T19:32:28.877 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up libcephfs-dev (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:28.921 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libcephfs-dev (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:29.018 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2022-08-01T19:32:29.020 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up librados2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:29.044 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-08-01T19:32:29.047 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-08-01T19:32:29.147 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2022-08-01T19:32:29.329 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-pastescript (2.0.2-4) ... 2022-08-01T19:32:29.353 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-tempita (0.5.2-6) ... 2022-08-01T19:32:29.373 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pastescript (2.0.2-4) ... 2022-08-01T19:32:29.646 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-packaging (20.3-1) ... 2022-08-01T19:32:29.656 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2022-08-01T19:32:29.700 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2022-08-01T19:32:29.955 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2022-08-01T19:32:30.008 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-rados (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:30.056 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2022-08-01T19:32:30.060 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-rados (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:30.151 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-rbd (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:30.161 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-rbd (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:30.262 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up rbd-fuse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:30.277 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up rbd-fuse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:30.333 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2022-08-01T19:32:30.388 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-routes (2.4.1-2) ... 2022-08-01T19:32:30.412 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-routes (2.4.1-2) ... 2022-08-01T19:32:30.645 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up librgw2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:30.726 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2022-08-01T19:32:30.790 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-google-auth (1.5.1-2) ... 2022-08-01T19:32:30.799 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-google-auth (1.5.1-2) ... 2022-08-01T19:32:30.812 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-08-01T19:32:30.938 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-pytest (4.6.9-1) ... 2022-08-01T19:32:31.035 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2022-08-01T19:32:31.091 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-rgw (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:31.140 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-rgw (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:31.192 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-cephfs (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:31.244 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-cephfs (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:31.324 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2022-08-01T19:32:31.412 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2022-08-01T19:32:31.519 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up cephfs-shell (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:31.545 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up cephfs-shell (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:31.645 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2022-08-01T19:32:31.658 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2022-08-01T19:32:31.662 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2022-08-01T19:32:31.827 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up socat (1.7.3.3-2) ... 2022-08-01T19:32:31.940 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-ceph-common (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:31.993 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-bs4 (4.8.2-1) ... 2022-08-01T19:32:32.225 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-html5lib (1.0.1-2) ... 2022-08-01T19:32:32.377 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up libcephfs2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:32.503 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2022-08-01T19:32:32.573 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up ceph-common (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:32.585 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-08-01T19:32:32.710 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pluggy (0.13.0-2) ... 2022-08-01T19:32:32.774 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-common (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:32.847 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up libradosstriper1 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:32.973 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-08-01T19:32:33.029 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libxslt1.1:amd64 (1.1.34-4) ... 2022-08-01T19:32:33.188 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2022-08-01T19:32:33.309 INFO:teuthology.orchestra.run.smithi017.stdout:Adding group ceph....done 2022-08-01T19:32:33.326 INFO:teuthology.orchestra.run.smithi085.stdout:Adding group ceph....done 2022-08-01T19:32:33.358 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-08-01T19:32:33.506 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2022-08-01T19:32:33.712 INFO:teuthology.orchestra.run.smithi017.stdout:Adding system user ceph....done 2022-08-01T19:32:33.719 INFO:teuthology.orchestra.run.smithi085.stdout:Adding system user ceph....done 2022-08-01T19:32:33.790 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-logutils (0.3.3-7) ... 2022-08-01T19:32:33.872 INFO:teuthology.orchestra.run.smithi017.stdout:Setting system user ceph properties....done 2022-08-01T19:32:33.888 INFO:teuthology.orchestra.run.smithi085.stdout:Setting system user ceph properties....done 2022-08-01T19:32:33.889 INFO:teuthology.orchestra.run.smithi017.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2022-08-01T19:32:33.903 INFO:teuthology.orchestra.run.smithi085.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2022-08-01T19:32:34.046 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2022-08-01T19:32:34.049 INFO:teuthology.orchestra.run.smithi085.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2022-08-01T19:32:34.084 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2022-08-01T19:32:34.209 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2022-08-01T19:32:34.365 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2022-08-01T19:32:34.369 INFO:teuthology.orchestra.run.smithi085.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2022-08-01T19:32:34.414 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up librbd1 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:34.486 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-prettytable (0.7.2-5) ... 2022-08-01T19:32:34.540 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-08-01T19:32:34.699 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2022-08-01T19:32:34.745 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2022-08-01T19:32:34.818 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-test (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:34.825 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-08-01T19:32:34.826 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up ceph-test (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:34.952 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up radosgw (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:34.965 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up radosgw (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:35.063 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2022-08-01T19:32:35.168 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2022-08-01T19:32:35.222 INFO:teuthology.orchestra.run.smithi085.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-08-01T19:32:35.222 INFO:teuthology.orchestra.run.smithi085.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-08-01T19:32:35.243 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-08-01T19:32:35.244 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-08-01T19:32:35.295 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-rsa (4.0-3) ... 2022-08-01T19:32:35.417 INFO:teuthology.orchestra.run.smithi190.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2022-08-01T19:32:35.604 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-webtest (2.0.32-2) ... 2022-08-01T19:32:35.732 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2022-08-01T19:32:35.867 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-08-01T19:32:35.887 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up ceph-base (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:35.904 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-base (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:35.930 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up ceph-mgr-modules-core (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:36.056 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up ceph-fuse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:36.193 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2022-08-01T19:32:36.261 INFO:teuthology.orchestra.run.smithi085.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2022-08-01T19:32:36.280 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-08-01T19:32:36.280 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-08-01T19:32:36.295 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2022-08-01T19:32:36.544 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-08-01T19:32:36.679 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-08-01T19:32:36.689 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up ceph-mds (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:36.741 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up libcephfs-dev (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:36.788 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-webob (1:1.8.5-2) ... 2022-08-01T19:32:36.789 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-mds (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:36.876 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-08-01T19:32:36.903 INFO:teuthology.orchestra.run.smithi085.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-08-01T19:32:36.903 INFO:teuthology.orchestra.run.smithi085.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-08-01T19:32:37.032 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-08-01T19:32:37.032 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-08-01T19:32:37.123 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-08-01T19:32:37.187 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-pastescript (2.0.2-4) ... 2022-08-01T19:32:37.349 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up ceph-mgr (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:37.408 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up librados2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:37.466 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-mgr (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:37.534 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2022-08-01T19:32:37.566 INFO:teuthology.orchestra.run.smithi085.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-08-01T19:32:37.566 INFO:teuthology.orchestra.run.smithi085.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-08-01T19:32:37.637 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2022-08-01T19:32:37.698 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-08-01T19:32:37.698 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-08-01T19:32:38.009 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:38.014 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-rados (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:38.127 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up ceph-osd (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:38.140 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-rbd (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:38.151 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:38.294 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-osd (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:38.300 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up rbd-fuse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:38.426 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-routes (2.4.1-2) ... 2022-08-01T19:32:38.659 INFO:teuthology.orchestra.run.smithi085.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-08-01T19:32:38.659 INFO:teuthology.orchestra.run.smithi085.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-08-01T19:32:38.818 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-08-01T19:32:38.818 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-08-01T19:32:38.829 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-google-auth (1.5.1-2) ... 2022-08-01T19:32:39.014 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up librgw2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:39.096 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up ceph-mgr-k8sevents (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:39.140 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-08-01T19:32:39.164 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-rgw (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:39.222 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up ceph-mgr-diskprediction-local (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:39.262 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-mgr-k8sevents (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:39.300 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pytest (4.6.9-1) ... 2022-08-01T19:32:39.323 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-cephfs (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:39.388 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-mgr-diskprediction-local (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:39.398 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up ceph-mon (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:39.556 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-mon (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:39.610 INFO:teuthology.orchestra.run.smithi085.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-08-01T19:32:39.610 INFO:teuthology.orchestra.run.smithi085.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-08-01T19:32:39.616 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up cephfs-shell (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:39.676 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2022-08-01T19:32:39.742 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2022-08-01T19:32:39.786 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-08-01T19:32:39.787 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-08-01T19:32:39.961 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2022-08-01T19:32:40.066 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up ceph-mgr-cephadm (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:40.184 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up ceph (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:40.241 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-mgr-cephadm (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:40.245 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-bs4 (4.8.2-1) ... 2022-08-01T19:32:40.285 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up ceph-mgr-dashboard (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:40.384 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:40.429 INFO:teuthology.orchestra.run.smithi085.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2022-08-01T19:32:40.501 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-mgr-dashboard (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:40.580 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libcephfs2 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:40.637 INFO:teuthology.orchestra.run.smithi017.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2022-08-01T19:32:40.679 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up ceph-common (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:40.706 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2022-08-01T19:32:40.754 INFO:teuthology.orchestra.run.smithi085.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-08-01T19:32:40.929 INFO:teuthology.orchestra.run.smithi017.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-08-01T19:32:41.025 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libradosstriper1 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:41.176 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-08-01T19:32:41.257 INFO:teuthology.orchestra.run.smithi077.stdout:Adding group ceph....done 2022-08-01T19:32:41.519 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-08-01T19:32:41.634 INFO:teuthology.orchestra.run.smithi077.stdout:Adding system user ceph....done 2022-08-01T19:32:41.844 INFO:teuthology.orchestra.run.smithi077.stdout:Setting system user ceph properties....done 2022-08-01T19:32:41.859 INFO:teuthology.orchestra.run.smithi077.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2022-08-01T19:32:42.009 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2022-08-01T19:32:42.311 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2022-08-01T19:32:42.575 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up librbd1 (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:42.790 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up ceph-test (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:42.916 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up radosgw (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:43.026 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-08-01T19:32:43.162 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-08-01T19:32:43.163 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-08-01T19:32:43.202 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2022-08-01T19:32:43.311 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-08-01T19:32:43.591 INFO:teuthology.orchestra.run.smithi085.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-08-01T19:32:43.612 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2022-08-01T19:32:43.763 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-rsa (4.0-3) ... 2022-08-01T19:32:43.801 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up ceph-base (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:43.849 INFO:teuthology.orchestra.run.smithi017.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-08-01T19:32:44.018 INFO:teuthology.orchestra.run.smithi085.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-08-01T19:32:44.022 DEBUG:teuthology.orchestra.run.smithi085:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2022-08-01T19:32:44.090 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-webtest (2.0.32-2) ... 2022-08-01T19:32:44.090 INFO:teuthology.orchestra.run.smithi085.stdout:Reading package lists... 2022-08-01T19:32:44.186 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2022-08-01T19:32:44.268 INFO:teuthology.orchestra.run.smithi085.stdout:Building dependency tree... 2022-08-01T19:32:44.268 INFO:teuthology.orchestra.run.smithi085.stdout:Reading state information... 2022-08-01T19:32:44.269 INFO:teuthology.orchestra.run.smithi017.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-08-01T19:32:44.271 DEBUG:teuthology.orchestra.run.smithi017:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2022-08-01T19:32:44.346 INFO:teuthology.orchestra.run.smithi017.stdout:Reading package lists... 2022-08-01T19:32:44.416 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mgr-modules-core (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:44.421 INFO:teuthology.orchestra.run.smithi085.stdout:libtool-bin is already the newest version (2.4.6-14). 2022-08-01T19:32:44.421 INFO:teuthology.orchestra.run.smithi085.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2022-08-01T19:32:44.421 INFO:teuthology.orchestra.run.smithi085.stdout:uuid-dev is already the newest version (2.34-0.1ubuntu9.3). 2022-08-01T19:32:44.422 INFO:teuthology.orchestra.run.smithi085.stdout:The following packages were automatically installed and are no longer required: 2022-08-01T19:32:44.422 INFO:teuthology.orchestra.run.smithi085.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-08-01T19:32:44.422 INFO:teuthology.orchestra.run.smithi085.stdout:Use 'sudo apt autoremove' to remove them. 2022-08-01T19:32:44.423 INFO:teuthology.orchestra.run.smithi085.stdout:Suggested packages: 2022-08-01T19:32:44.423 INFO:teuthology.orchestra.run.smithi085.stdout: bison-doc flex-doc indent-doc libssl-doc 2022-08-01T19:32:44.452 INFO:teuthology.orchestra.run.smithi085.stdout:The following NEW packages will be installed: 2022-08-01T19:32:44.453 INFO:teuthology.orchestra.run.smithi085.stdout: bison dump flex indent libaio-dev libelf-dev libfl-dev libfl2 liblzo2-2 2022-08-01T19:32:44.454 INFO:teuthology.orchestra.run.smithi085.stdout: libssl-dev 2022-08-01T19:32:44.530 INFO:teuthology.orchestra.run.smithi085.stdout:0 upgraded, 10 newly installed, 0 to remove and 4 not upgraded. 2022-08-01T19:32:44.530 INFO:teuthology.orchestra.run.smithi085.stdout:Need to get 2,914 kB of archives. 2022-08-01T19:32:44.530 INFO:teuthology.orchestra.run.smithi085.stdout:After this operation, 12.4 MB of additional disk space will be used. 2022-08-01T19:32:44.531 INFO:teuthology.orchestra.run.smithi085.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 flex amd64 2.6.4-6.2 [317 kB] 2022-08-01T19:32:44.531 INFO:teuthology.orchestra.run.smithi017.stdout:Building dependency tree... 2022-08-01T19:32:44.532 INFO:teuthology.orchestra.run.smithi017.stdout:Reading state information... 2022-08-01T19:32:44.542 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-fuse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:44.659 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up ceph-mds (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:44.669 INFO:teuthology.orchestra.run.smithi085.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 bison amd64 2:3.5.1+dfsg-1 [657 kB] 2022-08-01T19:32:44.694 INFO:teuthology.orchestra.run.smithi017.stdout:libtool-bin is already the newest version (2.4.6-14). 2022-08-01T19:32:44.695 INFO:teuthology.orchestra.run.smithi017.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2022-08-01T19:32:44.695 INFO:teuthology.orchestra.run.smithi017.stdout:uuid-dev is already the newest version (2.34-0.1ubuntu9.3). 2022-08-01T19:32:44.695 INFO:teuthology.orchestra.run.smithi017.stdout:The following packages were automatically installed and are no longer required: 2022-08-01T19:32:44.696 INFO:teuthology.orchestra.run.smithi017.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-08-01T19:32:44.696 INFO:teuthology.orchestra.run.smithi017.stdout:Use 'sudo apt autoremove' to remove them. 2022-08-01T19:32:44.696 INFO:teuthology.orchestra.run.smithi017.stdout:Suggested packages: 2022-08-01T19:32:44.697 INFO:teuthology.orchestra.run.smithi017.stdout: bison-doc flex-doc indent-doc libssl-doc 2022-08-01T19:32:44.729 INFO:teuthology.orchestra.run.smithi017.stdout:The following NEW packages will be installed: 2022-08-01T19:32:44.730 INFO:teuthology.orchestra.run.smithi017.stdout: bison dump flex indent libaio-dev libelf-dev libfl-dev libfl2 liblzo2-2 2022-08-01T19:32:44.730 INFO:teuthology.orchestra.run.smithi017.stdout: libssl-dev 2022-08-01T19:32:44.762 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-08-01T19:32:44.762 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-08-01T19:32:44.785 INFO:teuthology.orchestra.run.smithi017.stdout:0 upgraded, 10 newly installed, 0 to remove and 4 not upgraded. 2022-08-01T19:32:44.785 INFO:teuthology.orchestra.run.smithi017.stdout:Need to get 2,914 kB of archives. 2022-08-01T19:32:44.786 INFO:teuthology.orchestra.run.smithi017.stdout:After this operation, 12.4 MB of additional disk space will be used. 2022-08-01T19:32:44.786 INFO:teuthology.orchestra.run.smithi017.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 flex amd64 2.6.4-6.2 [317 kB] 2022-08-01T19:32:44.800 INFO:teuthology.orchestra.run.smithi085.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2022-08-01T19:32:44.808 INFO:teuthology.orchestra.run.smithi085.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2022-08-01T19:32:44.843 INFO:teuthology.orchestra.run.smithi085.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2022-08-01T19:32:44.849 INFO:teuthology.orchestra.run.smithi085.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2022-08-01T19:32:44.850 INFO:teuthology.orchestra.run.smithi085.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2022-08-01T19:32:44.866 INFO:teuthology.orchestra.run.smithi085.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl2 amd64 2.6.4-6.2 [11.5 kB] 2022-08-01T19:32:44.867 INFO:teuthology.orchestra.run.smithi085.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl-dev amd64 2.6.4-6.2 [6,316 B] 2022-08-01T19:32:44.868 INFO:teuthology.orchestra.run.smithi085.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.16 [1,584 kB] 2022-08-01T19:32:44.885 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-08-01T19:32:44.885 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-08-01T19:32:44.917 INFO:teuthology.orchestra.run.smithi017.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 bison amd64 2:3.5.1+dfsg-1 [657 kB] 2022-08-01T19:32:45.045 INFO:teuthology.orchestra.run.smithi017.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2022-08-01T19:32:45.049 INFO:teuthology.orchestra.run.smithi017.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2022-08-01T19:32:45.086 INFO:teuthology.orchestra.run.smithi017.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2022-08-01T19:32:45.090 INFO:teuthology.orchestra.run.smithi017.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2022-08-01T19:32:45.105 INFO:teuthology.orchestra.run.smithi017.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2022-08-01T19:32:45.108 INFO:teuthology.orchestra.run.smithi017.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl2 amd64 2.6.4-6.2 [11.5 kB] 2022-08-01T19:32:45.109 INFO:teuthology.orchestra.run.smithi017.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl-dev amd64 2.6.4-6.2 [6,316 B] 2022-08-01T19:32:45.110 INFO:teuthology.orchestra.run.smithi017.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.16 [1,584 kB] 2022-08-01T19:32:45.236 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libcephfs-dev (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:45.330 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up ceph-mgr (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:45.370 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-08-01T19:32:45.510 INFO:teuthology.orchestra.run.smithi085.stdout:Fetched 2,914 kB in 1s (4,203 kB/s) 2022-08-01T19:32:45.554 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-08-01T19:32:45.554 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-08-01T19:32:45.631 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package flex. 2022-08-01T19:32:45.671 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pastescript (2.0.2-4) ... 2022-08-01T19:32:45.674 INFO:teuthology.orchestra.run.smithi085.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 109762 files and directories currently installed.) 2022-08-01T19:32:45.677 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../0-flex_2.6.4-6.2_amd64.deb ... 2022-08-01T19:32:45.720 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking flex (2.6.4-6.2) ... 2022-08-01T19:32:45.782 INFO:teuthology.orchestra.run.smithi017.stdout:Fetched 2,914 kB in 1s (4,296 kB/s) 2022-08-01T19:32:45.897 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package flex. 2022-08-01T19:32:45.944 INFO:teuthology.orchestra.run.smithi017.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 109762 files and directories currently installed.) 2022-08-01T19:32:45.948 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../0-flex_2.6.4-6.2_amd64.deb ... 2022-08-01T19:32:45.995 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking flex (2.6.4-6.2) ... 2022-08-01T19:32:46.006 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:46.014 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2022-08-01T19:32:46.124 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up ceph-osd (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:46.192 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package bison. 2022-08-01T19:32:46.206 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../1-bison_2%3a3.5.1+dfsg-1_amd64.deb ... 2022-08-01T19:32:46.279 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking bison (2:3.5.1+dfsg-1) ... 2022-08-01T19:32:46.366 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-rados (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:46.491 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package bison. 2022-08-01T19:32:46.492 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-rbd (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:46.505 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../1-bison_2%3a3.5.1+dfsg-1_amd64.deb ... 2022-08-01T19:32:46.554 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking bison (2:3.5.1+dfsg-1) ... 2022-08-01T19:32:46.655 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-08-01T19:32:46.655 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-08-01T19:32:46.660 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up rbd-fuse (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:46.688 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package liblzo2-2:amd64. 2022-08-01T19:32:46.701 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../2-liblzo2-2_2.10-2_amd64.deb ... 2022-08-01T19:32:46.771 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2022-08-01T19:32:46.777 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-routes (2.4.1-2) ... 2022-08-01T19:32:47.004 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package liblzo2-2:amd64. 2022-08-01T19:32:47.020 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../2-liblzo2-2_2.10-2_amd64.deb ... 2022-08-01T19:32:47.084 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up ceph-mgr-k8sevents (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:47.108 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package dump. 2022-08-01T19:32:47.122 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../3-dump_0.4b46-6_amd64.deb ... 2022-08-01T19:32:47.145 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2022-08-01T19:32:47.166 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking dump (0.4b46-6) ... 2022-08-01T19:32:47.172 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-google-auth (1.5.1-2) ... 2022-08-01T19:32:47.185 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up ceph-mgr-diskprediction-local (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:47.337 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up ceph-mon (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:47.482 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-rgw (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:47.515 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package dump. 2022-08-01T19:32:47.520 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package indent. 2022-08-01T19:32:47.532 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../3-dump_0.4b46-6_amd64.deb ... 2022-08-01T19:32:47.536 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../4-indent_2.2.12-1_amd64.deb ... 2022-08-01T19:32:47.544 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-08-01T19:32:47.544 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-08-01T19:32:47.578 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking indent (2.2.12-1) ... 2022-08-01T19:32:47.582 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking dump (0.4b46-6) ... 2022-08-01T19:32:47.650 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-cephfs (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:47.839 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package libaio-dev:amd64. 2022-08-01T19:32:47.852 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../5-libaio-dev_0.3.112-5_amd64.deb ... 2022-08-01T19:32:47.889 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2022-08-01T19:32:47.959 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up cephfs-shell (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:47.988 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up ceph-mgr-cephadm (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:48.003 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package indent. 2022-08-01T19:32:48.018 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../4-indent_2.2.12-1_amd64.deb ... 2022-08-01T19:32:48.061 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking indent (2.2.12-1) ... 2022-08-01T19:32:48.085 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2022-08-01T19:32:48.131 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up ceph (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:48.159 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package libelf-dev:amd64. 2022-08-01T19:32:48.173 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../6-libelf-dev_0.176-1.1build1_amd64.deb ... 2022-08-01T19:32:48.217 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2022-08-01T19:32:48.291 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up ceph-mgr-dashboard (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:48.297 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libaio-dev:amd64. 2022-08-01T19:32:48.311 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../5-libaio-dev_0.3.112-5_amd64.deb ... 2022-08-01T19:32:48.347 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2022-08-01T19:32:48.435 INFO:teuthology.orchestra.run.smithi077.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2022-08-01T19:32:48.587 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package libfl2:amd64. 2022-08-01T19:32:48.600 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../7-libfl2_2.6.4-6.2_amd64.deb ... 2022-08-01T19:32:48.637 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking libfl2:amd64 (2.6.4-6.2) ... 2022-08-01T19:32:48.659 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libelf-dev:amd64. 2022-08-01T19:32:48.672 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../6-libelf-dev_0.176-1.1build1_amd64.deb ... 2022-08-01T19:32:48.709 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2022-08-01T19:32:48.727 INFO:teuthology.orchestra.run.smithi077.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-08-01T19:32:48.941 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package libfl-dev:amd64. 2022-08-01T19:32:48.954 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../8-libfl-dev_2.6.4-6.2_amd64.deb ... 2022-08-01T19:32:48.997 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-common (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:48.999 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking libfl-dev:amd64 (2.6.4-6.2) ... 2022-08-01T19:32:49.078 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libfl2:amd64. 2022-08-01T19:32:49.093 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../7-libfl2_2.6.4-6.2_amd64.deb ... 2022-08-01T19:32:49.137 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libfl2:amd64 (2.6.4-6.2) ... 2022-08-01T19:32:49.252 INFO:teuthology.orchestra.run.smithi085.stdout:Selecting previously unselected package libssl-dev:amd64. 2022-08-01T19:32:49.266 INFO:teuthology.orchestra.run.smithi085.stdout:Preparing to unpack .../9-libssl-dev_1.1.1f-1ubuntu2.16_amd64.deb ... 2022-08-01T19:32:49.310 INFO:teuthology.orchestra.run.smithi085.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.16) ... 2022-08-01T19:32:49.422 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libfl-dev:amd64. 2022-08-01T19:32:49.438 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../8-libfl-dev_2.6.4-6.2_amd64.deb ... 2022-08-01T19:32:49.481 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libfl-dev:amd64 (2.6.4-6.2) ... 2022-08-01T19:32:49.633 INFO:teuthology.orchestra.run.smithi190.stdout:Adding group ceph....done 2022-08-01T19:32:49.700 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libssl-dev:amd64. 2022-08-01T19:32:49.716 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../9-libssl-dev_1.1.1f-1ubuntu2.16_amd64.deb ... 2022-08-01T19:32:49.780 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.16) ... 2022-08-01T19:32:49.808 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up flex (2.6.4-6.2) ... 2022-08-01T19:32:49.915 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2022-08-01T19:32:50.010 INFO:teuthology.orchestra.run.smithi190.stdout:Adding system user ceph....done 2022-08-01T19:32:50.033 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2022-08-01T19:32:50.159 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.16) ... 2022-08-01T19:32:50.179 INFO:teuthology.orchestra.run.smithi190.stdout:Setting system user ceph properties....done 2022-08-01T19:32:50.194 INFO:teuthology.orchestra.run.smithi190.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2022-08-01T19:32:50.232 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up flex (2.6.4-6.2) ... 2022-08-01T19:32:50.310 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2022-08-01T19:32:50.337 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2022-08-01T19:32:50.363 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2022-08-01T19:32:50.436 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up libfl2:amd64 (2.6.4-6.2) ... 2022-08-01T19:32:50.489 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2022-08-01T19:32:50.571 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up dump (0.4b46-6) ... 2022-08-01T19:32:50.615 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.16) ... 2022-08-01T19:32:50.653 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2022-08-01T19:32:50.666 INFO:teuthology.orchestra.run.smithi085.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2022-08-01T19:32:50.739 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up bison (2:3.5.1+dfsg-1) ... 2022-08-01T19:32:50.749 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2022-08-01T19:32:50.839 INFO:teuthology.orchestra.run.smithi085.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2022-08-01T19:32:50.851 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libfl2:amd64 (2.6.4-6.2) ... 2022-08-01T19:32:50.899 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up indent (2.2.12-1) ... 2022-08-01T19:32:50.951 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up dump (0.4b46-6) ... 2022-08-01T19:32:51.025 INFO:teuthology.orchestra.run.smithi085.stdout:Setting up libfl-dev:amd64 (2.6.4-6.2) ... 2022-08-01T19:32:51.039 INFO:teuthology.orchestra.run.smithi017.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2022-08-01T19:32:51.125 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up bison (2:3.5.1+dfsg-1) ... 2022-08-01T19:32:51.149 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-test (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:51.152 INFO:teuthology.orchestra.run.smithi085.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-08-01T19:32:51.218 INFO:teuthology.orchestra.run.smithi017.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2022-08-01T19:32:51.275 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up radosgw (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:51.280 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up indent (2.2.12-1) ... 2022-08-01T19:32:51.304 INFO:teuthology.orchestra.run.smithi085.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-08-01T19:32:51.406 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libfl-dev:amd64 (2.6.4-6.2) ... 2022-08-01T19:32:51.455 INFO:teuthology.orchestra.run.smithi077.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-08-01T19:32:51.534 INFO:teuthology.orchestra.run.smithi017.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-08-01T19:32:51.543 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-08-01T19:32:51.543 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-08-01T19:32:51.711 INFO:teuthology.orchestra.run.smithi017.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-08-01T19:32:51.874 INFO:teuthology.orchestra.run.smithi077.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-08-01T19:32:51.877 DEBUG:teuthology.orchestra.run.smithi077:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2022-08-01T19:32:51.948 INFO:teuthology.orchestra.run.smithi077.stdout:Reading package lists... 2022-08-01T19:32:52.126 INFO:teuthology.orchestra.run.smithi077.stdout:Building dependency tree... 2022-08-01T19:32:52.127 INFO:teuthology.orchestra.run.smithi077.stdout:Reading state information... 2022-08-01T19:32:52.202 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-base (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:52.282 INFO:teuthology.orchestra.run.smithi077.stdout:libtool-bin is already the newest version (2.4.6-14). 2022-08-01T19:32:52.282 INFO:teuthology.orchestra.run.smithi077.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2022-08-01T19:32:52.282 INFO:teuthology.orchestra.run.smithi077.stdout:uuid-dev is already the newest version (2.34-0.1ubuntu9.3). 2022-08-01T19:32:52.283 INFO:teuthology.orchestra.run.smithi077.stdout:The following packages were automatically installed and are no longer required: 2022-08-01T19:32:52.283 INFO:teuthology.orchestra.run.smithi077.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-08-01T19:32:52.283 INFO:teuthology.orchestra.run.smithi077.stdout:Use 'sudo apt autoremove' to remove them. 2022-08-01T19:32:52.284 INFO:teuthology.orchestra.run.smithi077.stdout:Suggested packages: 2022-08-01T19:32:52.284 INFO:teuthology.orchestra.run.smithi077.stdout: bison-doc flex-doc indent-doc libssl-doc 2022-08-01T19:32:52.316 INFO:teuthology.orchestra.run.smithi077.stdout:The following NEW packages will be installed: 2022-08-01T19:32:52.316 INFO:teuthology.orchestra.run.smithi077.stdout: bison dump flex indent libaio-dev libelf-dev libfl-dev libfl2 liblzo2-2 2022-08-01T19:32:52.317 INFO:teuthology.orchestra.run.smithi077.stdout: libssl-dev 2022-08-01T19:32:52.371 INFO:teuthology.orchestra.run.smithi077.stdout:0 upgraded, 10 newly installed, 0 to remove and 4 not upgraded. 2022-08-01T19:32:52.372 INFO:teuthology.orchestra.run.smithi077.stdout:Need to get 2,914 kB of archives. 2022-08-01T19:32:52.372 INFO:teuthology.orchestra.run.smithi077.stdout:After this operation, 12.4 MB of additional disk space will be used. 2022-08-01T19:32:52.373 INFO:teuthology.orchestra.run.smithi077.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 flex amd64 2.6.4-6.2 [317 kB] 2022-08-01T19:32:52.533 INFO:teuthology.orchestra.run.smithi077.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 bison amd64 2:3.5.1+dfsg-1 [657 kB] 2022-08-01T19:32:52.562 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2022-08-01T19:32:52.686 INFO:teuthology.orchestra.run.smithi077.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2022-08-01T19:32:52.707 INFO:teuthology.orchestra.run.smithi077.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2022-08-01T19:32:52.740 INFO:teuthology.orchestra.run.smithi077.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2022-08-01T19:32:52.759 INFO:teuthology.orchestra.run.smithi077.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2022-08-01T19:32:52.759 INFO:teuthology.orchestra.run.smithi077.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2022-08-01T19:32:52.777 INFO:teuthology.orchestra.run.smithi077.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl2 amd64 2.6.4-6.2 [11.5 kB] 2022-08-01T19:32:52.778 INFO:teuthology.orchestra.run.smithi077.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl-dev amd64 2.6.4-6.2 [6,316 B] 2022-08-01T19:32:52.778 INFO:teuthology.orchestra.run.smithi077.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.16 [1,584 kB] 2022-08-01T19:32:53.079 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mds (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:53.323 INFO:teuthology.orchestra.run.smithi085.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2022-08-01T19:32:53.383 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-08-01T19:32:53.384 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-08-01T19:32:53.413 INFO:teuthology.orchestra.run.smithi077.stdout:Fetched 2,914 kB in 1s (3,935 kB/s) 2022-08-01T19:32:53.520 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package flex. 2022-08-01T19:32:53.565 INFO:teuthology.orchestra.run.smithi077.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 109762 files and directories currently installed.) 2022-08-01T19:32:53.569 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../0-flex_2.6.4-6.2_amd64.deb ... 2022-08-01T19:32:53.614 INFO:teuthology.orchestra.run.smithi017.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2022-08-01T19:32:53.643 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking flex (2.6.4-6.2) ... 2022-08-01T19:32:53.831 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mgr (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:54.022 INFO:teuthology.orchestra.run.smithi085.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-08-01T19:32:54.026 DEBUG:teuthology.parallel:result is None 2022-08-01T19:32:54.037 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-08-01T19:32:54.038 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-08-01T19:32:54.183 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package bison. 2022-08-01T19:32:54.197 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../1-bison_2%3a3.5.1+dfsg-1_amd64.deb ... 2022-08-01T19:32:54.268 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking bison (2:3.5.1+dfsg-1) ... 2022-08-01T19:32:54.364 INFO:teuthology.orchestra.run.smithi017.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-08-01T19:32:54.367 DEBUG:teuthology.parallel:result is None 2022-08-01T19:32:54.482 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:54.653 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-osd (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:54.720 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package liblzo2-2:amd64. 2022-08-01T19:32:54.733 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../2-liblzo2-2_2.10-2_amd64.deb ... 2022-08-01T19:32:54.820 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2022-08-01T19:32:55.158 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-08-01T19:32:55.158 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-08-01T19:32:55.224 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package dump. 2022-08-01T19:32:55.239 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../3-dump_0.4b46-6_amd64.deb ... 2022-08-01T19:32:55.290 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking dump (0.4b46-6) ... 2022-08-01T19:32:55.601 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package indent. 2022-08-01T19:32:55.616 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../4-indent_2.2.12-1_amd64.deb ... 2022-08-01T19:32:55.627 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mgr-k8sevents (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:55.651 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking indent (2.2.12-1) ... 2022-08-01T19:32:55.753 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mgr-diskprediction-local (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:55.895 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mon (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:55.904 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package libaio-dev:amd64. 2022-08-01T19:32:55.919 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../5-libaio-dev_0.3.112-5_amd64.deb ... 2022-08-01T19:32:55.962 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2022-08-01T19:32:56.171 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-08-01T19:32:56.171 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-08-01T19:32:56.249 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package libelf-dev:amd64. 2022-08-01T19:32:56.266 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../6-libelf-dev_0.176-1.1build1_amd64.deb ... 2022-08-01T19:32:56.307 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2022-08-01T19:32:56.638 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mgr-cephadm (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:56.659 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package libfl2:amd64. 2022-08-01T19:32:56.673 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../7-libfl2_2.6.4-6.2_amd64.deb ... 2022-08-01T19:32:56.718 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking libfl2:amd64 (2.6.4-6.2) ... 2022-08-01T19:32:56.756 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:56.982 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mgr-dashboard (15.2.16-343-gfd96719b-1focal) ... 2022-08-01T19:32:56.987 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package libfl-dev:amd64. 2022-08-01T19:32:57.003 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../8-libfl-dev_2.6.4-6.2_amd64.deb ... 2022-08-01T19:32:57.045 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking libfl-dev:amd64 (2.6.4-6.2) ... 2022-08-01T19:32:57.126 INFO:teuthology.orchestra.run.smithi190.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2022-08-01T19:32:57.323 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package libssl-dev:amd64. 2022-08-01T19:32:57.336 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../9-libssl-dev_1.1.1f-1ubuntu2.16_amd64.deb ... 2022-08-01T19:32:57.373 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.16) ... 2022-08-01T19:32:57.418 INFO:teuthology.orchestra.run.smithi190.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-08-01T19:32:57.871 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up flex (2.6.4-6.2) ... 2022-08-01T19:32:57.986 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2022-08-01T19:32:58.195 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2022-08-01T19:32:58.321 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.16) ... 2022-08-01T19:32:58.472 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2022-08-01T19:32:58.598 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up libfl2:amd64 (2.6.4-6.2) ... 2022-08-01T19:32:58.708 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up dump (0.4b46-6) ... 2022-08-01T19:32:58.836 INFO:teuthology.orchestra.run.smithi077.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2022-08-01T19:32:58.884 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up bison (2:3.5.1+dfsg-1) ... 2022-08-01T19:32:58.957 INFO:teuthology.orchestra.run.smithi077.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2022-08-01T19:32:59.027 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up indent (2.2.12-1) ... 2022-08-01T19:32:59.187 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up libfl-dev:amd64 (2.6.4-6.2) ... 2022-08-01T19:32:59.313 INFO:teuthology.orchestra.run.smithi077.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-08-01T19:32:59.498 INFO:teuthology.orchestra.run.smithi077.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-08-01T19:33:00.246 INFO:teuthology.orchestra.run.smithi190.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-08-01T19:33:00.720 INFO:teuthology.orchestra.run.smithi190.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-08-01T19:33:00.723 DEBUG:teuthology.orchestra.run.smithi190:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2022-08-01T19:33:00.793 INFO:teuthology.orchestra.run.smithi190.stdout:Reading package lists... 2022-08-01T19:33:00.970 INFO:teuthology.orchestra.run.smithi190.stdout:Building dependency tree... 2022-08-01T19:33:00.971 INFO:teuthology.orchestra.run.smithi190.stdout:Reading state information... 2022-08-01T19:33:01.126 INFO:teuthology.orchestra.run.smithi190.stdout:libtool-bin is already the newest version (2.4.6-14). 2022-08-01T19:33:01.126 INFO:teuthology.orchestra.run.smithi190.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2022-08-01T19:33:01.126 INFO:teuthology.orchestra.run.smithi190.stdout:uuid-dev is already the newest version (2.34-0.1ubuntu9.3). 2022-08-01T19:33:01.127 INFO:teuthology.orchestra.run.smithi190.stdout:The following packages were automatically installed and are no longer required: 2022-08-01T19:33:01.127 INFO:teuthology.orchestra.run.smithi190.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-08-01T19:33:01.128 INFO:teuthology.orchestra.run.smithi190.stdout:Use 'sudo apt autoremove' to remove them. 2022-08-01T19:33:01.128 INFO:teuthology.orchestra.run.smithi190.stdout:Suggested packages: 2022-08-01T19:33:01.128 INFO:teuthology.orchestra.run.smithi190.stdout: bison-doc flex-doc indent-doc libssl-doc 2022-08-01T19:33:01.159 INFO:teuthology.orchestra.run.smithi190.stdout:The following NEW packages will be installed: 2022-08-01T19:33:01.160 INFO:teuthology.orchestra.run.smithi190.stdout: bison dump flex indent libaio-dev libelf-dev libfl-dev libfl2 liblzo2-2 2022-08-01T19:33:01.160 INFO:teuthology.orchestra.run.smithi190.stdout: libssl-dev 2022-08-01T19:33:01.342 INFO:teuthology.orchestra.run.smithi190.stdout:0 upgraded, 10 newly installed, 0 to remove and 4 not upgraded. 2022-08-01T19:33:01.342 INFO:teuthology.orchestra.run.smithi190.stdout:Need to get 2,914 kB of archives. 2022-08-01T19:33:01.343 INFO:teuthology.orchestra.run.smithi190.stdout:After this operation, 12.4 MB of additional disk space will be used. 2022-08-01T19:33:01.343 INFO:teuthology.orchestra.run.smithi190.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 flex amd64 2.6.4-6.2 [317 kB] 2022-08-01T19:33:01.417 INFO:teuthology.orchestra.run.smithi077.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2022-08-01T19:33:01.978 INFO:teuthology.orchestra.run.smithi190.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 bison amd64 2:3.5.1+dfsg-1 [657 kB] 2022-08-01T19:33:02.134 INFO:teuthology.orchestra.run.smithi077.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-08-01T19:33:02.137 DEBUG:teuthology.parallel:result is None 2022-08-01T19:33:02.780 INFO:teuthology.orchestra.run.smithi190.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2022-08-01T19:33:02.867 INFO:teuthology.orchestra.run.smithi190.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2022-08-01T19:33:03.053 INFO:teuthology.orchestra.run.smithi190.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2022-08-01T19:33:03.155 INFO:teuthology.orchestra.run.smithi190.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2022-08-01T19:33:03.161 INFO:teuthology.orchestra.run.smithi190.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2022-08-01T19:33:03.251 INFO:teuthology.orchestra.run.smithi190.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl2 amd64 2.6.4-6.2 [11.5 kB] 2022-08-01T19:33:03.255 INFO:teuthology.orchestra.run.smithi190.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl-dev amd64 2.6.4-6.2 [6,316 B] 2022-08-01T19:33:03.258 INFO:teuthology.orchestra.run.smithi190.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.16 [1,584 kB] 2022-08-01T19:33:05.714 INFO:teuthology.orchestra.run.smithi190.stdout:Fetched 2,914 kB in 4s (693 kB/s) 2022-08-01T19:33:05.845 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package flex. 2022-08-01T19:33:05.890 INFO:teuthology.orchestra.run.smithi190.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 109762 files and directories currently installed.) 2022-08-01T19:33:05.894 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../0-flex_2.6.4-6.2_amd64.deb ... 2022-08-01T19:33:05.960 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking flex (2.6.4-6.2) ... 2022-08-01T19:33:06.464 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package bison. 2022-08-01T19:33:06.478 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../1-bison_2%3a3.5.1+dfsg-1_amd64.deb ... 2022-08-01T19:33:06.535 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking bison (2:3.5.1+dfsg-1) ... 2022-08-01T19:33:06.993 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package liblzo2-2:amd64. 2022-08-01T19:33:07.011 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../2-liblzo2-2_2.10-2_amd64.deb ... 2022-08-01T19:33:07.094 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2022-08-01T19:33:07.428 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package dump. 2022-08-01T19:33:07.437 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../3-dump_0.4b46-6_amd64.deb ... 2022-08-01T19:33:07.471 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking dump (0.4b46-6) ... 2022-08-01T19:33:07.800 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package indent. 2022-08-01T19:33:07.816 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../4-indent_2.2.12-1_amd64.deb ... 2022-08-01T19:33:07.900 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking indent (2.2.12-1) ... 2022-08-01T19:33:08.211 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libaio-dev:amd64. 2022-08-01T19:33:08.228 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../5-libaio-dev_0.3.112-5_amd64.deb ... 2022-08-01T19:33:08.269 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2022-08-01T19:33:08.548 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libelf-dev:amd64. 2022-08-01T19:33:08.561 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../6-libelf-dev_0.176-1.1build1_amd64.deb ... 2022-08-01T19:33:08.606 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2022-08-01T19:33:09.119 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libfl2:amd64. 2022-08-01T19:33:09.133 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../7-libfl2_2.6.4-6.2_amd64.deb ... 2022-08-01T19:33:09.177 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libfl2:amd64 (2.6.4-6.2) ... 2022-08-01T19:33:09.463 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libfl-dev:amd64. 2022-08-01T19:33:09.477 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../8-libfl-dev_2.6.4-6.2_amd64.deb ... 2022-08-01T19:33:09.521 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libfl-dev:amd64 (2.6.4-6.2) ... 2022-08-01T19:33:09.799 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libssl-dev:amd64. 2022-08-01T19:33:09.812 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../9-libssl-dev_1.1.1f-1ubuntu2.16_amd64.deb ... 2022-08-01T19:33:09.841 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.16) ... 2022-08-01T19:33:10.350 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up flex (2.6.4-6.2) ... 2022-08-01T19:33:10.478 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2022-08-01T19:33:10.629 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2022-08-01T19:33:10.730 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.16) ... 2022-08-01T19:33:10.882 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2022-08-01T19:33:10.991 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libfl2:amd64 (2.6.4-6.2) ... 2022-08-01T19:33:11.117 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up dump (0.4b46-6) ... 2022-08-01T19:33:11.245 INFO:teuthology.orchestra.run.smithi190.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2022-08-01T19:33:11.310 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up bison (2:3.5.1+dfsg-1) ... 2022-08-01T19:33:11.400 INFO:teuthology.orchestra.run.smithi190.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2022-08-01T19:33:11.495 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up indent (2.2.12-1) ... 2022-08-01T19:33:11.621 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libfl-dev:amd64 (2.6.4-6.2) ... 2022-08-01T19:33:11.723 INFO:teuthology.orchestra.run.smithi190.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-08-01T19:33:11.883 INFO:teuthology.orchestra.run.smithi190.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-08-01T19:33:13.785 INFO:teuthology.orchestra.run.smithi190.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2022-08-01T19:33:14.880 INFO:teuthology.orchestra.run.smithi190.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-08-01T19:33:14.883 DEBUG:teuthology.parallel:result is None 2022-08-01T19:33:14.884 DEBUG:teuthology.orchestra.run.smithi017:> sudo lsb_release -is 2022-08-01T19:33:14.938 INFO:teuthology.orchestra.run.smithi017.stdout:Ubuntu 2022-08-01T19:33:14.943 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-08-01T19:33:14.945 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=fd96719b0e1200b56e7430604cfdd20ad281ca92 2022-08-01T19:33:15.099 DEBUG:teuthology.orchestra.run.smithi017:> dpkg-query -W -f '${Version}' ceph 2022-08-01T19:33:15.121 INFO:teuthology.orchestra.run.smithi017.stdout:15.2.16-343-gfd96719b-1focal 2022-08-01T19:33:15.123 INFO:teuthology.packaging:The installed version of ceph is 15.2.16-343-gfd96719b-1focal 2022-08-01T19:33:15.123 INFO:teuthology.task.install:The correct ceph version 15.2.16-343-gfd96719b-1focal is installed. 2022-08-01T19:33:15.124 DEBUG:teuthology.orchestra.run.smithi077:> sudo lsb_release -is 2022-08-01T19:33:15.177 INFO:teuthology.orchestra.run.smithi077.stdout:Ubuntu 2022-08-01T19:33:15.182 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-08-01T19:33:15.182 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=fd96719b0e1200b56e7430604cfdd20ad281ca92 2022-08-01T19:33:15.331 DEBUG:teuthology.orchestra.run.smithi077:> dpkg-query -W -f '${Version}' ceph 2022-08-01T19:33:15.353 INFO:teuthology.orchestra.run.smithi077.stdout:15.2.16-343-gfd96719b-1focal 2022-08-01T19:33:15.353 INFO:teuthology.packaging:The installed version of ceph is 15.2.16-343-gfd96719b-1focal 2022-08-01T19:33:15.354 INFO:teuthology.task.install:The correct ceph version 15.2.16-343-gfd96719b-1focal is installed. 2022-08-01T19:33:15.354 DEBUG:teuthology.orchestra.run.smithi085:> sudo lsb_release -is 2022-08-01T19:33:15.407 INFO:teuthology.orchestra.run.smithi085.stdout:Ubuntu 2022-08-01T19:33:15.413 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-08-01T19:33:15.413 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=fd96719b0e1200b56e7430604cfdd20ad281ca92 2022-08-01T19:33:15.564 DEBUG:teuthology.orchestra.run.smithi085:> dpkg-query -W -f '${Version}' ceph 2022-08-01T19:33:15.588 INFO:teuthology.orchestra.run.smithi085.stdout:15.2.16-343-gfd96719b-1focal 2022-08-01T19:33:15.589 INFO:teuthology.packaging:The installed version of ceph is 15.2.16-343-gfd96719b-1focal 2022-08-01T19:33:15.589 INFO:teuthology.task.install:The correct ceph version 15.2.16-343-gfd96719b-1focal is installed. 2022-08-01T19:33:15.590 DEBUG:teuthology.orchestra.run.smithi190:> sudo lsb_release -is 2022-08-01T19:33:15.642 INFO:teuthology.orchestra.run.smithi190.stdout:Ubuntu 2022-08-01T19:33:15.647 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-08-01T19:33:15.648 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=fd96719b0e1200b56e7430604cfdd20ad281ca92 2022-08-01T19:33:15.802 DEBUG:teuthology.orchestra.run.smithi190:> dpkg-query -W -f '${Version}' ceph 2022-08-01T19:33:15.828 INFO:teuthology.orchestra.run.smithi190.stdout:15.2.16-343-gfd96719b-1focal 2022-08-01T19:33:15.829 INFO:teuthology.packaging:The installed version of ceph is 15.2.16-343-gfd96719b-1focal 2022-08-01T19:33:15.829 INFO:teuthology.task.install:The correct ceph version 15.2.16-343-gfd96719b-1focal is installed. 2022-08-01T19:33:15.830 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2022-08-01T19:33:15.830 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2022-08-01T19:33:15.831 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-08-01T19:33:15.848 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2022-08-01T19:33:15.848 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-08-01T19:33:15.865 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-08-01T19:33:15.866 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-08-01T19:33:15.882 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2022-08-01T19:33:15.883 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-08-01T19:33:15.900 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2022-08-01T19:33:15.900 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2022-08-01T19:33:15.901 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/usr/bin/daemon-helper 2022-08-01T19:33:15.915 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-08-01T19:33:15.929 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2022-08-01T19:33:15.930 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/daemon-helper 2022-08-01T19:33:15.947 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-08-01T19:33:15.958 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-08-01T19:33:15.959 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd of=/usr/bin/daemon-helper 2022-08-01T19:33:15.975 DEBUG:teuthology.orchestra.run.smithi085:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-08-01T19:33:16.029 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2022-08-01T19:33:16.030 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/daemon-helper 2022-08-01T19:33:16.047 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-08-01T19:33:16.059 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2022-08-01T19:33:16.060 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2022-08-01T19:33:16.060 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/usr/bin/adjust-ulimits 2022-08-01T19:33:16.076 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-08-01T19:33:16.087 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2022-08-01T19:33:16.088 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/adjust-ulimits 2022-08-01T19:33:16.104 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-08-01T19:33:16.113 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-08-01T19:33:16.114 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd of=/usr/bin/adjust-ulimits 2022-08-01T19:33:16.130 DEBUG:teuthology.orchestra.run.smithi085:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-08-01T19:33:16.142 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2022-08-01T19:33:16.143 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/adjust-ulimits 2022-08-01T19:33:16.159 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-08-01T19:33:16.216 INFO:teuthology.run_tasks:Running task ceph... 2022-08-01T19:33:16.280 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2022-08-01T19:33:16.281 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 777 /var/log/ceph 2022-08-01T19:33:16.284 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 777 /var/log/ceph 2022-08-01T19:33:16.286 DEBUG:teuthology.orchestra.run.smithi085:> sudo chmod 777 /var/log/ceph 2022-08-01T19:33:16.288 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 777 /var/log/ceph 2022-08-01T19:33:16.304 INFO:tasks.ceph:Disabling ceph logrotate... 2022-08-01T19:33:16.305 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f -- /etc/logrotate.d/ceph 2022-08-01T19:33:16.307 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f -- /etc/logrotate.d/ceph 2022-08-01T19:33:16.342 DEBUG:teuthology.orchestra.run.smithi085:> sudo rm -f -- /etc/logrotate.d/ceph 2022-08-01T19:33:16.350 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/logrotate.d/ceph 2022-08-01T19:33:16.365 INFO:tasks.ceph:Creating extra log directories... 2022-08-01T19:33:16.366 DEBUG:teuthology.orchestra.run.smithi017:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-08-01T19:33:16.370 DEBUG:teuthology.orchestra.run.smithi077:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-08-01T19:33:16.373 DEBUG:teuthology.orchestra.run.smithi085:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-08-01T19:33:16.377 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-08-01T19:33:16.430 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2022-08-01T19:33:16.431 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2022-08-01T19:33:16.431 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2022-08-01T19:33:16.431 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2022-08-01T19:33:16.432 DEBUG:teuthology.orchestra.run.smithi017:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-08-01T19:33:16.439 DEBUG:teuthology.orchestra.run.smithi017:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-08-01T19:33:16.521 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2022-08-01T19:33:16.521 DEBUG:teuthology.orchestra.run.smithi077:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-08-01T19:33:16.528 DEBUG:teuthology.orchestra.run.smithi077:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-08-01T19:33:16.569 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-08-01T19:33:16.569 DEBUG:teuthology.orchestra.run.smithi085:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-08-01T19:33:16.576 DEBUG:teuthology.orchestra.run.smithi085:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-08-01T19:33:16.660 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2022-08-01T19:33:16.660 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-08-01T19:33:16.668 DEBUG:teuthology.orchestra.run.smithi190:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-08-01T19:33:16.713 INFO:tasks.ceph:Creating ceph cluster ceph... 2022-08-01T19:33:16.714 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'POOL_APP_NOT_ENABLED', 'has not responded to cap revoke by MDS for over', 'MDS_CLIENT_LATE_RELEASE', 'responding to mclientcaps'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2022-08-01T19:33:16.714 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2022-08-01_19:07:06-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954722', 'branch': 'wip-yuri6-testing-2022-08-01-0839-octopus', 'description': 'fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests}', 'email': 'ceph-qa@ceph.io', 'first_in_suite': False, 'job_id': '6954722', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2022-08-01_19:07:06-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 30}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-yuri6-testing-2022-08-01-0839-octopus'}, 'ceph': {'cephfs_ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'POOL_APP_NOT_ENABLED', 'has not responded to cap revoke by MDS for over', 'MDS_CLIENT_LATE_RELEASE', 'responding to mclientcaps'], 'sha1': 'fd96719b0e1200b56e7430604cfdd20ad281ca92'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'fd96719b0e1200b56e7430604cfdd20ad281ca92'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri6-testing-2022-08-01-0839-octopus', 'sha1': 'fd96719b0e1200b56e7430604cfdd20ad281ca92'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 71, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'], ['client.0'], ['client.1']], 'sha1': 'fd96719b0e1200b56e7430604cfdd20ad281ca92', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-yuri6-testing-2022-08-01-0839-octopus', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'fd96719b0e1200b56e7430604cfdd20ad281ca92', 'targets': {'smithi017.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPeRXsoKPmWQ8X7x/I7uJftBgiYaankGpy8Dg2PO6PU288gAAJ4SAkxprQ3QbDL/5C5EXc0KaD6bgfLhFwW+k+ykqS12L1b0csNCfG/s2rM4KKgbOUvdgd7fDxegVfqf5k1J3IXVPSdhrpKKIltEOpLSK5UPxtzzkJ4FIoDkg1zl1FfOZKY2m/K9dEia/62RBVCxINHGkG5GhmOZ83yfczc6NySZ4bglCG+XItbTCpTZM8r8PVgzAfJbhEp6xEUaYQEFyJJcRSD3LZ7nmw+uf7Kk/8w8sh3PusewrE2ccTas6vMxzb7C1Q3Bi1UjJ1aPHpQQfLyS4ipl/NWRflFsLKUsS0J0ZgQJLYkISMuFqnT7yBhvZsWf6aOSa6k/elON9N7+ROzDEHOGpl29hnBPl+nZ2yOXeFrNKaHOyNrV1uYulz8gu0T+0HdG0M4K4naI9VRLG61yGBebKhsp0/4Tx4oteaLwU/eqwnYPwk9irrSTRT4zEV8ivCg+nvAJixwBU=', 'smithi077.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4Txugz93Uts6pxjrZYYwnPLHu33ISUnThQKsd5nl5flpx+Gj08VAkEaHNmHpwpUIGPefJ+GKX1mvXYojuXJmSO2fx7v3SMeQJOtm6dFUUp3LfCWOZGXTyaPY9CgPuDnAzRzQTon++FE3y2R0P2ahC1cZhV/lQiYoSjYpUlLFa1cTYsFvzAbtx7OGh61DYaWynW6xs0VHgfrwGvRutvNf+bOtN+cElouJzI558agEY1zCFbGg3ChC6u1KLUdpMghBuOZj8CdFY4QMosySHmuxe1tySU2bxXLIQwPMiGKpPo+BJKbJO+KMsZFnbIfAcfPAtAHYA8o1HVMsGy1TelDpOIt/XulsLFZDEkp5UfPG6GuECI3i2QJLMZVA+uDzwmuigaEIHSc4cQyVFux3cQiFxXQMuUAqVyNNOyZziuTwlh+GDHHxdDsZ8H8c7gnjPbULaR3fe0wtuZrnLZWq1apfMzZmd9cYkxCu6Wf73hBzyTlfe4ZQK8PWQlms0hyszR68=', 'smithi085.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3+mH3hvP3b0A7FmAp52QaUqw3u6EuprX2mJzBzCH0gKYHLjkqY9lRBoRQ2eb+m8jTk3IX1ILDd4LX2y+CIj7496LOE1V30t6xJxoyfAYqPM+O5o2oNWuYw812+bMAs94G3eqA17YntXan78oo7ryRHxkEvInwhkL3LLpFqDvuP3L9AZ397w7dZ3KE4Xg1rw4TqM3UNnSYvB9UE9cdzIugfcyjteUmEMACvTxKQVmq5MSuHVU3hpH8aWhaw2Hwn1ydFHYzcvo5WrXH2OxuFTHRmA/escL4PY3r9LuJv5hzXAPGn+qJAFn+WBaKAMFET31027QP4eQXbJ15nwOe54frNtZWu0SiIoRPC9f7kRbiNt+YRiHgRL9EtAFAAqbzjzVWdlShN1kWUepWUQ6sgwl9DSQRFWs+CwH9paAqqJs3z0JQ5pRLA/QrnItd3EtWheio8MdyENHicdKo9C2uHMk+2oQiA2NYt6B9TFSDxrqtlBz+hoZw+McJbUPYCVbSYu8=', 'smithi190.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrnkVZHWYRdJqp2GXyKlktsr65gcUBpEGDffjFSYoNjWmM7JFruIZra9PvE5sa6BrDLGCpWwcZBCz+u0+elFMFp6tAfEq+l++9rajWlg31dsRkpg5CDSJDC4npe+P8Y90WJqOLSY7qwMWUEVJJBr1SdnAO94kiP7B7urFPbeBfD/e33YKYTzPxVe+Zp2BbTqYrfccIRY2r0SuEoNEZ/wTohwy2RkzwpBfr/O1i5pUL5dqQFLJShnO1YBOM+l2OYwOH38Bmsu4BQdyMye9x0DN3YWr5jXZFxikVOHVNdd/0XvS6gBwUrh7Q7WefiVSGrHSVROCfCI6aA4vt4xvflm4dqkg8so/xmmL+77Mm3fzFvESQGUmswMhF0luoH9onLY0pfidKlJDu/BsNjCjVI7kRWSPfK13Kgh1HH7OUTWfpAgrsjbpx2i/+CvPUcKBUMQC1OAJf8JK8UWx5aWKmdsZK/hzkOzn6tyyBxwLCu8vqhwWmueNTsRtWn91Aq/vt/DU='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': 'fd96719b0e1200b56e7430604cfdd20ad281ca92'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_misc']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'dd7eb9d989259767783fb5ee9a761cd0732b84e6', 'timestamp': '2022-08-01_19:07:06', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.1713'} 2022-08-01T19:33:16.715 DEBUG:teuthology.orchestra.run.smithi017:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-08-01T19:33:16.718 DEBUG:teuthology.orchestra.run.smithi077:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-08-01T19:33:16.720 DEBUG:teuthology.orchestra.run.smithi085:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-08-01T19:33:16.724 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-08-01T19:33:16.756 DEBUG:teuthology.orchestra.run.smithi017:> sudo install -d -m0777 -- /var/run/ceph 2022-08-01T19:33:16.766 DEBUG:teuthology.orchestra.run.smithi077:> sudo install -d -m0777 -- /var/run/ceph 2022-08-01T19:33:16.769 DEBUG:teuthology.orchestra.run.smithi085:> sudo install -d -m0777 -- /var/run/ceph 2022-08-01T19:33:16.774 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -d -m0777 -- /var/run/ceph 2022-08-01T19:33:16.790 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2022-08-01T19:33:16.791 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/scratch_devs of=/dev/stdout 2022-08-01T19:33:16.799 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2022-08-01T19:33:16.800 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2022-08-01T19:33:16.847 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2022-08-01T19:33:16.847 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-08-01T19:33:16.848 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 6h/6d Inode: 358 Links: 1 2022-08-01T19:33:16.848 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-08-01T19:33:16.848 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2022-08-01 19:30:03.478299396 +0000 2022-08-01T19:33:16.848 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2022-08-01 19:30:03.150302401 +0000 2022-08-01T19:33:16.849 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2022-08-01 19:30:03.150302401 +0000 2022-08-01T19:33:16.849 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: - 2022-08-01T19:33:16.858 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2022-08-01T19:33:16.858 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2022-08-01T19:33:16.859 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.000302217 s, 1.7 MB/s 2022-08-01T19:33:16.863 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2022-08-01T19:33:16.872 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2022-08-01T19:33:16.872 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-08-01T19:33:16.873 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 6h/6d Inode: 364 Links: 1 2022-08-01T19:33:16.873 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-08-01T19:33:16.874 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2022-08-01 19:30:03.986294743 +0000 2022-08-01T19:33:16.874 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2022-08-01 19:30:03.678297564 +0000 2022-08-01T19:33:16.875 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2022-08-01 19:30:03.678297564 +0000 2022-08-01T19:33:16.875 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: - 2022-08-01T19:33:16.882 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2022-08-01T19:33:16.882 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2022-08-01T19:33:16.883 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.000292964 s, 1.7 MB/s 2022-08-01T19:33:16.886 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2022-08-01T19:33:16.894 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2022-08-01T19:33:16.894 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-08-01T19:33:16.894 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 6h/6d Inode: 373 Links: 1 2022-08-01T19:33:16.895 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-08-01T19:33:16.895 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2022-08-01 19:30:04.530289761 +0000 2022-08-01T19:33:16.895 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2022-08-01 19:30:04.198292802 +0000 2022-08-01T19:33:16.895 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2022-08-01 19:30:04.198292802 +0000 2022-08-01T19:33:16.896 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: - 2022-08-01T19:33:16.901 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2022-08-01T19:33:16.901 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2022-08-01T19:33:16.901 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.00030802 s, 1.7 MB/s 2022-08-01T19:33:16.905 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2022-08-01T19:33:16.912 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2022-08-01T19:33:16.913 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-08-01T19:33:16.913 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 6h/6d Inode: 379 Links: 1 2022-08-01T19:33:16.913 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-08-01T19:33:16.913 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2022-08-01 19:30:05.058284925 +0000 2022-08-01T19:33:16.913 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2022-08-01 19:30:04.742287819 +0000 2022-08-01T19:33:16.914 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2022-08-01 19:30:04.742287819 +0000 2022-08-01T19:33:16.914 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: - 2022-08-01T19:33:16.921 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2022-08-01T19:33:16.921 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2022-08-01T19:33:16.921 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.000271624 s, 1.9 MB/s 2022-08-01T19:33:16.926 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-08-01T19:33:16.926 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2022-08-01T19:33:16.927 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/scratch_devs of=/dev/stdout 2022-08-01T19:33:16.933 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2022-08-01T19:33:16.934 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2022-08-01T19:33:16.941 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2022-08-01T19:33:16.941 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-08-01T19:33:16.942 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 6h/6d Inode: 359 Links: 1 2022-08-01T19:33:16.942 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-08-01T19:33:16.942 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2022-08-01 19:30:04.254654551 +0000 2022-08-01T19:33:16.942 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2022-08-01 19:30:03.946655617 +0000 2022-08-01T19:33:16.943 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2022-08-01 19:30:03.946655617 +0000 2022-08-01T19:33:16.943 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: - 2022-08-01T19:33:16.951 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2022-08-01T19:33:16.951 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2022-08-01T19:33:16.951 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000396613 s, 1.3 MB/s 2022-08-01T19:33:16.955 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2022-08-01T19:33:16.962 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2022-08-01T19:33:16.962 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-08-01T19:33:16.963 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 6h/6d Inode: 365 Links: 1 2022-08-01T19:33:16.963 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-08-01T19:33:16.963 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2022-08-01 19:30:04.762652792 +0000 2022-08-01T19:33:16.963 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2022-08-01 19:30:04.454653858 +0000 2022-08-01T19:33:16.964 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2022-08-01 19:30:04.454653858 +0000 2022-08-01T19:33:16.964 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: - 2022-08-01T19:33:16.972 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2022-08-01T19:33:16.973 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2022-08-01T19:33:16.973 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000351484 s, 1.5 MB/s 2022-08-01T19:33:16.977 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2022-08-01T19:33:17.030 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2022-08-01T19:33:17.031 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-08-01T19:33:17.031 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 6h/6d Inode: 372 Links: 1 2022-08-01T19:33:17.031 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-08-01T19:33:17.032 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2022-08-01 19:30:05.342650787 +0000 2022-08-01T19:33:17.032 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2022-08-01 19:30:05.046651810 +0000 2022-08-01T19:33:17.032 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2022-08-01 19:30:05.046651810 +0000 2022-08-01T19:33:17.032 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: - 2022-08-01T19:33:17.039 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2022-08-01T19:33:17.040 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2022-08-01T19:33:17.040 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000395197 s, 1.3 MB/s 2022-08-01T19:33:17.045 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2022-08-01T19:33:17.052 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2022-08-01T19:33:17.053 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-08-01T19:33:17.053 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 6h/6d Inode: 380 Links: 1 2022-08-01T19:33:17.053 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-08-01T19:33:17.054 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2022-08-01 19:30:05.886648907 +0000 2022-08-01T19:33:17.054 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2022-08-01 19:30:05.582649958 +0000 2022-08-01T19:33:17.054 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2022-08-01 19:30:05.582649958 +0000 2022-08-01T19:33:17.054 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: - 2022-08-01T19:33:17.061 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2022-08-01T19:33:17.061 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2022-08-01T19:33:17.061 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000372042 s, 1.4 MB/s 2022-08-01T19:33:17.066 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-08-01T19:33:17.066 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi017.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi077.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2022-08-01T19:33:17.067 INFO:tasks.ceph:Generating config... 2022-08-01T19:33:17.069 INFO:tasks.ceph:[client] client mount timeout = 600 2022-08-01T19:33:17.070 INFO:tasks.ceph:[client] debug client = 20 2022-08-01T19:33:17.070 INFO:tasks.ceph:[client] debug ms = 1 2022-08-01T19:33:17.070 INFO:tasks.ceph:[client] rados mon op timeout = 15m 2022-08-01T19:33:17.071 INFO:tasks.ceph:[client] rados osd op timeout = 15m 2022-08-01T19:33:17.072 INFO:tasks.ceph:[mds] debug mds = 20 2022-08-01T19:33:17.072 INFO:tasks.ceph:[mds] debug ms = 1 2022-08-01T19:33:17.073 INFO:tasks.ceph:[mds] mds bal frag = True 2022-08-01T19:33:17.073 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2022-08-01T19:33:17.074 INFO:tasks.ceph:[mds] mds bal merge size = 5 2022-08-01T19:33:17.075 INFO:tasks.ceph:[mds] mds bal split bits = 3 2022-08-01T19:33:17.075 INFO:tasks.ceph:[mds] mds bal split size = 100 2022-08-01T19:33:17.076 INFO:tasks.ceph:[mds] mds debug frag = True 2022-08-01T19:33:17.076 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2022-08-01T19:33:17.076 INFO:tasks.ceph:[mds] mds op complaint time = 180 2022-08-01T19:33:17.077 INFO:tasks.ceph:[mds] mds verify scatter = True 2022-08-01T19:33:17.077 INFO:tasks.ceph:[mds] osd op complaint time = 180 2022-08-01T19:33:17.077 INFO:tasks.ceph:[mds] rados mon op timeout = 15m 2022-08-01T19:33:17.078 INFO:tasks.ceph:[mds] rados osd op timeout = 15m 2022-08-01T19:33:17.079 INFO:tasks.ceph:[mgr] debug mgr = 20 2022-08-01T19:33:17.079 INFO:tasks.ceph:[mgr] debug ms = 1 2022-08-01T19:33:17.080 INFO:tasks.ceph:[mon] debug mon = 20 2022-08-01T19:33:17.080 INFO:tasks.ceph:[mon] debug ms = 1 2022-08-01T19:33:17.081 INFO:tasks.ceph:[mon] debug paxos = 20 2022-08-01T19:33:17.082 INFO:tasks.ceph:[mon] mon op complaint time = 120 2022-08-01T19:33:17.082 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2022-08-01T19:33:17.083 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2022-08-01T19:33:17.084 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2022-08-01T19:33:17.084 INFO:tasks.ceph:[osd] debug bluefs = 20 2022-08-01T19:33:17.085 INFO:tasks.ceph:[osd] debug bluestore = 20 2022-08-01T19:33:17.085 INFO:tasks.ceph:[osd] debug ms = 1 2022-08-01T19:33:17.086 INFO:tasks.ceph:[osd] debug osd = 20 2022-08-01T19:33:17.087 INFO:tasks.ceph:[osd] debug rocksdb = 10 2022-08-01T19:33:17.087 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2022-08-01T19:33:17.088 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2022-08-01T19:33:17.089 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2022-08-01T19:33:17.089 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2022-08-01T19:33:17.090 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2022-08-01T19:33:17.090 INFO:tasks.ceph:[osd] osd op complaint time = 180 2022-08-01T19:33:17.091 INFO:tasks.ceph:Setting up mon.a... 2022-08-01T19:33:17.092 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2022-08-01T19:33:17.136 INFO:teuthology.orchestra.run.smithi017.stdout:creating /etc/ceph/ceph.keyring 2022-08-01T19:33:17.139 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2022-08-01T19:33:17.217 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-08-01T19:33:17.232 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.17'), ('mon.b', '172.21.15.77'), ('mon.c', '[v2:172.21.15.77:3301,v1:172.21.15.77:6790]')] 2022-08-01T19:33:17.233 DEBUG:teuthology.orchestra.run.smithi017:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool --create --clobber --enable-all-features --add a 172.21.15.17 --add b 172.21.15.77 --addv c '[v2:172.21.15.77:3301,v1:172.21.15.77:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2022-08-01T19:33:17.311 INFO:teuthology.orchestra.run.smithi017.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2022-08-01T19:33:17.311 INFO:teuthology.orchestra.run.smithi017.stdout:monmaptool: generated fsid be659963-3cf3-4374-b3c4-cbe717b6bac2 2022-08-01T19:33:17.312 INFO:teuthology.orchestra.run.smithi017.stdout:epoch 0 2022-08-01T19:33:17.312 INFO:teuthology.orchestra.run.smithi017.stdout:fsid be659963-3cf3-4374-b3c4-cbe717b6bac2 2022-08-01T19:33:17.312 INFO:teuthology.orchestra.run.smithi017.stdout:last_changed 2022-08-01T19:33:17.313086+0000 2022-08-01T19:33:17.312 INFO:teuthology.orchestra.run.smithi017.stdout:created 2022-08-01T19:33:17.313086+0000 2022-08-01T19:33:17.313 INFO:teuthology.orchestra.run.smithi017.stdout:min_mon_release 0 (unknown) 2022-08-01T19:33:17.313 INFO:teuthology.orchestra.run.smithi017.stdout:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2022-08-01T19:33:17.313 INFO:teuthology.orchestra.run.smithi017.stdout:1: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.b 2022-08-01T19:33:17.313 INFO:teuthology.orchestra.run.smithi017.stdout:2: [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] mon.c 2022-08-01T19:33:17.314 INFO:teuthology.orchestra.run.smithi017.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2022-08-01T19:33:17.315 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID be659963-3cf3-4374-b3c4-cbe717b6bac2... 2022-08-01T19:33:17.316 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-08-01T19:33:17.320 DEBUG:teuthology.orchestra.run.smithi077:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-08-01T19:33:17.323 DEBUG:teuthology.orchestra.run.smithi085:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-08-01T19:33:17.325 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-08-01T19:33:17.349 INFO:teuthology.orchestra.run.smithi017.stdout:[global] 2022-08-01T19:33:17.349 INFO:teuthology.orchestra.run.smithi017.stdout: chdir = "" 2022-08-01T19:33:17.349 INFO:teuthology.orchestra.run.smithi017.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-08-01T19:33:17.349 INFO:teuthology.orchestra.run.smithi017.stdout: auth supported = cephx 2022-08-01T19:33:17.350 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:33:17.350 INFO:teuthology.orchestra.run.smithi017.stdout: filestore xattr use omap = true 2022-08-01T19:33:17.350 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:33:17.350 INFO:teuthology.orchestra.run.smithi017.stdout: mon clock drift allowed = 1.000 2022-08-01T19:33:17.351 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:33:17.351 INFO:teuthology.orchestra.run.smithi017.stdout: osd crush chooseleaf type = 0 2022-08-01T19:33:17.351 INFO:teuthology.orchestra.run.smithi017.stdout: auth debug = true 2022-08-01T19:33:17.351 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:33:17.352 INFO:teuthology.orchestra.run.smithi017.stdout: ms die on old message = true 2022-08-01T19:33:17.352 INFO:teuthology.orchestra.run.smithi017.stdout: ms die on bug = true 2022-08-01T19:33:17.352 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:33:17.352 INFO:teuthology.orchestra.run.smithi017.stdout: mon max pg per osd = 10000 # >= luminous 2022-08-01T19:33:17.353 INFO:teuthology.orchestra.run.smithi017.stdout: mon pg warn max object skew = 0 2022-08-01T19:33:17.353 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:33:17.353 INFO:teuthology.orchestra.run.smithi017.stdout: # disable pg_autoscaler by default for new pools 2022-08-01T19:33:17.353 INFO:teuthology.orchestra.run.smithi017.stdout: osd_pool_default_pg_autoscale_mode = off 2022-08-01T19:33:17.354 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:33:17.354 INFO:teuthology.orchestra.run.smithi017.stdout: osd pool default size = 2 2022-08-01T19:33:17.354 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:33:17.354 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd allow primary affinity = true 2022-08-01T19:33:17.354 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd allow pg remap = true 2022-08-01T19:33:17.355 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on legacy crush tunables = false 2022-08-01T19:33:17.355 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on crush straw calc version zero = false 2022-08-01T19:33:17.355 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on no sortbitwise = false 2022-08-01T19:33:17.355 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on osd down out interval zero = false 2022-08-01T19:33:17.356 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on too few osds = false 2022-08-01T19:33:17.356 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-08-01T19:33:17.356 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_pool_no_redundancy = false 2022-08-01T19:33:17.356 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:33:17.357 INFO:teuthology.orchestra.run.smithi017.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-08-01T19:33:17.357 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:33:17.357 INFO:teuthology.orchestra.run.smithi017.stdout: osd default data pool replay window = 5 2022-08-01T19:33:17.357 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:33:17.358 INFO:teuthology.orchestra.run.smithi017.stdout: mon allow pool delete = true 2022-08-01T19:33:17.358 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:33:17.358 INFO:teuthology.orchestra.run.smithi017.stdout: mon cluster log file level = debug 2022-08-01T19:33:17.358 INFO:teuthology.orchestra.run.smithi017.stdout: debug asserts on shutdown = true 2022-08-01T19:33:17.359 INFO:teuthology.orchestra.run.smithi017.stdout: mon health detail to clog = false 2022-08-01T19:33:17.359 INFO:teuthology.orchestra.run.smithi017.stdout: mon host = "172.21.15.17,172.21.15.77,[v2:172.21.15.77:3301,v1:172.21.15.77:6790]" 2022-08-01T19:33:17.359 INFO:teuthology.orchestra.run.smithi017.stdout: fsid = be659963-3cf3-4374-b3c4-cbe717b6bac2 2022-08-01T19:33:17.359 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:33:17.360 INFO:teuthology.orchestra.run.smithi017.stdout:[osd] 2022-08-01T19:33:17.360 INFO:teuthology.orchestra.run.smithi017.stdout: osd journal size = 100 2022-08-01T19:33:17.360 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:33:17.360 INFO:teuthology.orchestra.run.smithi017.stdout: osd scrub load threshold = 5.0 2022-08-01T19:33:17.360 INFO:teuthology.orchestra.run.smithi017.stdout: osd scrub max interval = 600 2022-08-01T19:33:17.361 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:33:17.361 INFO:teuthology.orchestra.run.smithi017.stdout: osd recover clone overlap = true 2022-08-01T19:33:17.361 INFO:teuthology.orchestra.run.smithi017.stdout: osd recovery max chunk = 1048576 2022-08-01T19:33:17.361 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:33:17.362 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug shutdown = true 2022-08-01T19:33:17.362 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug op order = true 2022-08-01T19:33:17.362 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug verify stray on activate = true 2022-08-01T19:33:17.362 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:33:17.363 INFO:teuthology.orchestra.run.smithi017.stdout: osd open classes on start = true 2022-08-01T19:33:17.363 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug pg log writeout = true 2022-08-01T19:33:17.363 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:33:17.363 INFO:teuthology.orchestra.run.smithi017.stdout: osd deep scrub update digest min age = 30 2022-08-01T19:33:17.363 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:33:17.364 INFO:teuthology.orchestra.run.smithi017.stdout: osd map max advance = 10 2022-08-01T19:33:17.364 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:33:17.364 INFO:teuthology.orchestra.run.smithi017.stdout: journal zero on create = true 2022-08-01T19:33:17.364 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:33:17.365 INFO:teuthology.orchestra.run.smithi017.stdout: filestore ondisk finisher threads = 3 2022-08-01T19:33:17.365 INFO:teuthology.orchestra.run.smithi017.stdout: filestore apply finisher threads = 3 2022-08-01T19:33:17.365 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:33:17.365 INFO:teuthology.orchestra.run.smithi017.stdout: bdev debug aio = true 2022-08-01T19:33:17.366 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug misdirected ops = true 2022-08-01T19:33:17.366 INFO:teuthology.orchestra.run.smithi017.stdout: bluestore block size = 96636764160 2022-08-01T19:33:17.366 INFO:teuthology.orchestra.run.smithi017.stdout: bluestore compression mode = aggressive 2022-08-01T19:33:17.366 INFO:teuthology.orchestra.run.smithi017.stdout: bluestore fsck on mount = True 2022-08-01T19:33:17.367 INFO:teuthology.orchestra.run.smithi017.stdout: debug bluefs = 20 2022-08-01T19:33:17.367 INFO:teuthology.orchestra.run.smithi017.stdout: debug bluestore = 20 2022-08-01T19:33:17.367 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2022-08-01T19:33:17.367 INFO:teuthology.orchestra.run.smithi017.stdout: debug osd = 20 2022-08-01T19:33:17.367 INFO:teuthology.orchestra.run.smithi017.stdout: debug rocksdb = 10 2022-08-01T19:33:17.368 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd backfillfull_ratio = 0.85 2022-08-01T19:33:17.368 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd full ratio = 0.9 2022-08-01T19:33:17.368 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd nearfull ratio = 0.8 2022-08-01T19:33:17.368 INFO:teuthology.orchestra.run.smithi017.stdout: osd failsafe full ratio = 0.95 2022-08-01T19:33:17.369 INFO:teuthology.orchestra.run.smithi017.stdout: osd objectstore = bluestore 2022-08-01T19:33:17.369 INFO:teuthology.orchestra.run.smithi017.stdout: osd op complaint time = 180 2022-08-01T19:33:17.369 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:33:17.369 INFO:teuthology.orchestra.run.smithi017.stdout:[mgr] 2022-08-01T19:33:17.370 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2022-08-01T19:33:17.370 INFO:teuthology.orchestra.run.smithi017.stdout: debug mgr = 20 2022-08-01T19:33:17.370 INFO:teuthology.orchestra.run.smithi017.stdout: debug mon = 20 2022-08-01T19:33:17.370 INFO:teuthology.orchestra.run.smithi017.stdout: debug auth = 20 2022-08-01T19:33:17.371 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min pgs per osd = 4 2022-08-01T19:33:17.371 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min bytes per osd = 10 2022-08-01T19:33:17.371 INFO:teuthology.orchestra.run.smithi017.stdout: mgr/telemetry/nag = false 2022-08-01T19:33:17.371 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:33:17.371 INFO:teuthology.orchestra.run.smithi017.stdout:[mon] 2022-08-01T19:33:17.372 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2022-08-01T19:33:17.372 INFO:teuthology.orchestra.run.smithi017.stdout: debug mon = 20 2022-08-01T19:33:17.372 INFO:teuthology.orchestra.run.smithi017.stdout: debug paxos = 20 2022-08-01T19:33:17.372 INFO:teuthology.orchestra.run.smithi017.stdout: debug auth = 20 2022-08-01T19:33:17.373 INFO:teuthology.orchestra.run.smithi017.stdout: mon data avail warn = 5 2022-08-01T19:33:17.373 INFO:teuthology.orchestra.run.smithi017.stdout: mon mgr mkfs grace = 240 2022-08-01T19:33:17.373 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min pgs per osd = 4 2022-08-01T19:33:17.373 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd reporter subtree level = osd 2022-08-01T19:33:17.373 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd prime pg temp = true 2022-08-01T19:33:17.374 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min bytes per osd = 10 2022-08-01T19:33:17.374 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:33:17.374 INFO:teuthology.orchestra.run.smithi017.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-08-01T19:33:17.374 INFO:teuthology.orchestra.run.smithi017.stdout: auth mon ticket ttl = 660 # 11m 2022-08-01T19:33:17.375 INFO:teuthology.orchestra.run.smithi017.stdout: auth service ticket ttl = 240 # 4m 2022-08-01T19:33:17.375 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:33:17.375 INFO:teuthology.orchestra.run.smithi017.stdout: # don't complain about insecure global_id in the test suite 2022-08-01T19:33:17.375 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-08-01T19:33:17.375 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-08-01T19:33:17.376 INFO:teuthology.orchestra.run.smithi017.stdout: mon op complaint time = 120 2022-08-01T19:33:17.376 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:33:17.376 INFO:teuthology.orchestra.run.smithi017.stdout:[client] 2022-08-01T19:33:17.376 INFO:teuthology.orchestra.run.smithi017.stdout: rgw cache enabled = true 2022-08-01T19:33:17.377 INFO:teuthology.orchestra.run.smithi017.stdout: rgw enable ops log = true 2022-08-01T19:33:17.377 INFO:teuthology.orchestra.run.smithi017.stdout: rgw enable usage log = true 2022-08-01T19:33:17.377 INFO:teuthology.orchestra.run.smithi017.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-08-01T19:33:17.377 INFO:teuthology.orchestra.run.smithi017.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-08-01T19:33:17.377 INFO:teuthology.orchestra.run.smithi017.stdout: client mount timeout = 600 2022-08-01T19:33:17.378 INFO:teuthology.orchestra.run.smithi017.stdout: debug client = 20 2022-08-01T19:33:17.378 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2022-08-01T19:33:17.378 INFO:teuthology.orchestra.run.smithi017.stdout: rados mon op timeout = 15m 2022-08-01T19:33:17.378 INFO:teuthology.orchestra.run.smithi017.stdout: rados osd op timeout = 15m 2022-08-01T19:33:17.379 INFO:teuthology.orchestra.run.smithi017.stdout:[mon.a] 2022-08-01T19:33:17.379 INFO:teuthology.orchestra.run.smithi017.stdout:[mon.b] 2022-08-01T19:33:17.379 INFO:teuthology.orchestra.run.smithi017.stdout:[mon.c] 2022-08-01T19:33:17.379 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.a] 2022-08-01T19:33:17.379 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.c] 2022-08-01T19:33:17.380 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.b] 2022-08-01T19:33:17.380 INFO:teuthology.orchestra.run.smithi017.stdout:[mds] 2022-08-01T19:33:17.380 INFO:teuthology.orchestra.run.smithi017.stdout: debug mds = 20 2022-08-01T19:33:17.380 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2022-08-01T19:33:17.381 INFO:teuthology.orchestra.run.smithi017.stdout: mds bal frag = True 2022-08-01T19:33:17.381 INFO:teuthology.orchestra.run.smithi017.stdout: mds bal fragment size max = 10000 2022-08-01T19:33:17.381 INFO:teuthology.orchestra.run.smithi017.stdout: mds bal merge size = 5 2022-08-01T19:33:17.381 INFO:teuthology.orchestra.run.smithi017.stdout: mds bal split bits = 3 2022-08-01T19:33:17.382 INFO:teuthology.orchestra.run.smithi017.stdout: mds bal split size = 100 2022-08-01T19:33:17.382 INFO:teuthology.orchestra.run.smithi017.stdout: mds debug frag = True 2022-08-01T19:33:17.382 INFO:teuthology.orchestra.run.smithi017.stdout: mds debug scatterstat = True 2022-08-01T19:33:17.382 INFO:teuthology.orchestra.run.smithi017.stdout: mds op complaint time = 180 2022-08-01T19:33:17.382 INFO:teuthology.orchestra.run.smithi017.stdout: mds verify scatter = True 2022-08-01T19:33:17.383 INFO:teuthology.orchestra.run.smithi017.stdout: osd op complaint time = 180 2022-08-01T19:33:17.383 INFO:teuthology.orchestra.run.smithi017.stdout: rados mon op timeout = 15m 2022-08-01T19:33:17.383 INFO:teuthology.orchestra.run.smithi017.stdout: rados osd op timeout = 15m 2022-08-01T19:33:17.386 INFO:teuthology.orchestra.run.smithi190.stdout:[global] 2022-08-01T19:33:17.387 INFO:teuthology.orchestra.run.smithi190.stdout: chdir = "" 2022-08-01T19:33:17.387 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-08-01T19:33:17.387 INFO:teuthology.orchestra.run.smithi190.stdout: auth supported = cephx 2022-08-01T19:33:17.387 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:33:17.387 INFO:teuthology.orchestra.run.smithi190.stdout: filestore xattr use omap = true 2022-08-01T19:33:17.388 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:33:17.388 INFO:teuthology.orchestra.run.smithi190.stdout: mon clock drift allowed = 1.000 2022-08-01T19:33:17.388 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:33:17.388 INFO:teuthology.orchestra.run.smithi190.stdout: osd crush chooseleaf type = 0 2022-08-01T19:33:17.389 INFO:teuthology.orchestra.run.smithi190.stdout: auth debug = true 2022-08-01T19:33:17.389 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:33:17.389 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on old message = true 2022-08-01T19:33:17.389 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on bug = true 2022-08-01T19:33:17.389 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:33:17.390 INFO:teuthology.orchestra.run.smithi190.stdout: mon max pg per osd = 10000 # >= luminous 2022-08-01T19:33:17.390 INFO:teuthology.orchestra.run.smithi190.stdout: mon pg warn max object skew = 0 2022-08-01T19:33:17.390 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:33:17.391 INFO:teuthology.orchestra.run.smithi190.stdout: # disable pg_autoscaler by default for new pools 2022-08-01T19:33:17.391 INFO:teuthology.orchestra.run.smithi190.stdout: osd_pool_default_pg_autoscale_mode = off 2022-08-01T19:33:17.391 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:33:17.391 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default size = 2 2022-08-01T19:33:17.391 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:33:17.392 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow primary affinity = true 2022-08-01T19:33:17.392 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow pg remap = true 2022-08-01T19:33:17.392 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on legacy crush tunables = false 2022-08-01T19:33:17.392 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on crush straw calc version zero = false 2022-08-01T19:33:17.392 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on no sortbitwise = false 2022-08-01T19:33:17.393 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on osd down out interval zero = false 2022-08-01T19:33:17.393 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on too few osds = false 2022-08-01T19:33:17.393 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-08-01T19:33:17.393 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_no_redundancy = false 2022-08-01T19:33:17.393 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:33:17.394 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-08-01T19:33:17.394 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:33:17.394 INFO:teuthology.orchestra.run.smithi190.stdout: osd default data pool replay window = 5 2022-08-01T19:33:17.394 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:33:17.395 INFO:teuthology.orchestra.run.smithi190.stdout: mon allow pool delete = true 2022-08-01T19:33:17.395 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:33:17.395 INFO:teuthology.orchestra.run.smithi190.stdout: mon cluster log file level = debug 2022-08-01T19:33:17.395 INFO:teuthology.orchestra.run.smithi190.stdout: debug asserts on shutdown = true 2022-08-01T19:33:17.395 INFO:teuthology.orchestra.run.smithi190.stdout: mon health detail to clog = false 2022-08-01T19:33:17.396 INFO:teuthology.orchestra.run.smithi190.stdout: mon host = "172.21.15.17,172.21.15.77,[v2:172.21.15.77:3301,v1:172.21.15.77:6790]" 2022-08-01T19:33:17.396 INFO:teuthology.orchestra.run.smithi190.stdout: fsid = be659963-3cf3-4374-b3c4-cbe717b6bac2 2022-08-01T19:33:17.396 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:33:17.396 INFO:teuthology.orchestra.run.smithi190.stdout:[osd] 2022-08-01T19:33:17.397 INFO:teuthology.orchestra.run.smithi190.stdout: osd journal size = 100 2022-08-01T19:33:17.397 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:33:17.397 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub load threshold = 5.0 2022-08-01T19:33:17.397 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub max interval = 600 2022-08-01T19:33:17.397 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:33:17.398 INFO:teuthology.orchestra.run.smithi190.stdout: osd recover clone overlap = true 2022-08-01T19:33:17.398 INFO:teuthology.orchestra.run.smithi190.stdout: osd recovery max chunk = 1048576 2022-08-01T19:33:17.398 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:33:17.398 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug shutdown = true 2022-08-01T19:33:17.399 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug op order = true 2022-08-01T19:33:17.399 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug verify stray on activate = true 2022-08-01T19:33:17.399 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:33:17.399 INFO:teuthology.orchestra.run.smithi190.stdout: osd open classes on start = true 2022-08-01T19:33:17.400 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug pg log writeout = true 2022-08-01T19:33:17.400 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:33:17.400 INFO:teuthology.orchestra.run.smithi190.stdout: osd deep scrub update digest min age = 30 2022-08-01T19:33:17.400 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:33:17.400 INFO:teuthology.orchestra.run.smithi190.stdout: osd map max advance = 10 2022-08-01T19:33:17.401 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:33:17.401 INFO:teuthology.orchestra.run.smithi190.stdout: journal zero on create = true 2022-08-01T19:33:17.401 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:33:17.401 INFO:teuthology.orchestra.run.smithi190.stdout: filestore ondisk finisher threads = 3 2022-08-01T19:33:17.402 INFO:teuthology.orchestra.run.smithi190.stdout: filestore apply finisher threads = 3 2022-08-01T19:33:17.402 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:33:17.402 INFO:teuthology.orchestra.run.smithi190.stdout: bdev debug aio = true 2022-08-01T19:33:17.402 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug misdirected ops = true 2022-08-01T19:33:17.403 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore block size = 96636764160 2022-08-01T19:33:17.403 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore compression mode = aggressive 2022-08-01T19:33:17.403 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore fsck on mount = True 2022-08-01T19:33:17.403 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluefs = 20 2022-08-01T19:33:17.403 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluestore = 20 2022-08-01T19:33:17.404 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2022-08-01T19:33:17.404 INFO:teuthology.orchestra.run.smithi190.stdout: debug osd = 20 2022-08-01T19:33:17.404 INFO:teuthology.orchestra.run.smithi190.stdout: debug rocksdb = 10 2022-08-01T19:33:17.404 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd backfillfull_ratio = 0.85 2022-08-01T19:33:17.405 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd full ratio = 0.9 2022-08-01T19:33:17.405 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd nearfull ratio = 0.8 2022-08-01T19:33:17.405 INFO:teuthology.orchestra.run.smithi190.stdout: osd failsafe full ratio = 0.95 2022-08-01T19:33:17.405 INFO:teuthology.orchestra.run.smithi190.stdout: osd objectstore = bluestore 2022-08-01T19:33:17.405 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2022-08-01T19:33:17.406 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:33:17.406 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr] 2022-08-01T19:33:17.406 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2022-08-01T19:33:17.406 INFO:teuthology.orchestra.run.smithi190.stdout: debug mgr = 20 2022-08-01T19:33:17.407 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2022-08-01T19:33:17.407 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2022-08-01T19:33:17.407 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2022-08-01T19:33:17.407 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2022-08-01T19:33:17.407 INFO:teuthology.orchestra.run.smithi190.stdout: mgr/telemetry/nag = false 2022-08-01T19:33:17.408 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:33:17.408 INFO:teuthology.orchestra.run.smithi190.stdout:[mon] 2022-08-01T19:33:17.408 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2022-08-01T19:33:17.408 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2022-08-01T19:33:17.408 INFO:teuthology.orchestra.run.smithi190.stdout: debug paxos = 20 2022-08-01T19:33:17.409 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2022-08-01T19:33:17.409 INFO:teuthology.orchestra.run.smithi190.stdout: mon data avail warn = 5 2022-08-01T19:33:17.409 INFO:teuthology.orchestra.run.smithi190.stdout: mon mgr mkfs grace = 240 2022-08-01T19:33:17.409 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2022-08-01T19:33:17.410 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd reporter subtree level = osd 2022-08-01T19:33:17.410 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd prime pg temp = true 2022-08-01T19:33:17.410 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2022-08-01T19:33:17.410 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:33:17.410 INFO:teuthology.orchestra.run.smithi190.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-08-01T19:33:17.411 INFO:teuthology.orchestra.run.smithi190.stdout: auth mon ticket ttl = 660 # 11m 2022-08-01T19:33:17.411 INFO:teuthology.orchestra.run.smithi190.stdout: auth service ticket ttl = 240 # 4m 2022-08-01T19:33:17.411 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:33:17.411 INFO:teuthology.orchestra.run.smithi190.stdout: # don't complain about insecure global_id in the test suite 2022-08-01T19:33:17.412 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-08-01T19:33:17.412 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-08-01T19:33:17.412 INFO:teuthology.orchestra.run.smithi190.stdout: mon op complaint time = 120 2022-08-01T19:33:17.412 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:33:17.413 INFO:teuthology.orchestra.run.smithi190.stdout:[client] 2022-08-01T19:33:17.413 INFO:teuthology.orchestra.run.smithi190.stdout: rgw cache enabled = true 2022-08-01T19:33:17.413 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable ops log = true 2022-08-01T19:33:17.413 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable usage log = true 2022-08-01T19:33:17.413 INFO:teuthology.orchestra.run.smithi190.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-08-01T19:33:17.414 INFO:teuthology.orchestra.run.smithi190.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-08-01T19:33:17.414 INFO:teuthology.orchestra.run.smithi190.stdout: client mount timeout = 600 2022-08-01T19:33:17.414 INFO:teuthology.orchestra.run.smithi190.stdout: debug client = 20 2022-08-01T19:33:17.414 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2022-08-01T19:33:17.415 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 15m 2022-08-01T19:33:17.415 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 15m 2022-08-01T19:33:17.415 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.a] 2022-08-01T19:33:17.415 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.b] 2022-08-01T19:33:17.416 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.c] 2022-08-01T19:33:17.416 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.a] 2022-08-01T19:33:17.416 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.c] 2022-08-01T19:33:17.416 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.b] 2022-08-01T19:33:17.417 INFO:teuthology.orchestra.run.smithi190.stdout:[mds] 2022-08-01T19:33:17.417 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds = 20 2022-08-01T19:33:17.417 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2022-08-01T19:33:17.417 INFO:teuthology.orchestra.run.smithi190.stdout: mds bal frag = True 2022-08-01T19:33:17.418 INFO:teuthology.orchestra.run.smithi190.stdout: mds bal fragment size max = 10000 2022-08-01T19:33:17.418 INFO:teuthology.orchestra.run.smithi190.stdout: mds bal merge size = 5 2022-08-01T19:33:17.418 INFO:teuthology.orchestra.run.smithi190.stdout: mds bal split bits = 3 2022-08-01T19:33:17.418 INFO:teuthology.orchestra.run.smithi190.stdout: mds bal split size = 100 2022-08-01T19:33:17.419 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug frag = True 2022-08-01T19:33:17.419 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug scatterstat = True 2022-08-01T19:33:17.419 INFO:teuthology.orchestra.run.smithi190.stdout: mds op complaint time = 180 2022-08-01T19:33:17.419 INFO:teuthology.orchestra.run.smithi190.stdout: mds verify scatter = True 2022-08-01T19:33:17.419 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2022-08-01T19:33:17.420 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 15m 2022-08-01T19:33:17.420 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 15m 2022-08-01T19:33:17.420 INFO:teuthology.orchestra.run.smithi085.stdout:[global] 2022-08-01T19:33:17.420 INFO:teuthology.orchestra.run.smithi085.stdout: chdir = "" 2022-08-01T19:33:17.421 INFO:teuthology.orchestra.run.smithi085.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-08-01T19:33:17.421 INFO:teuthology.orchestra.run.smithi085.stdout: auth supported = cephx 2022-08-01T19:33:17.421 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:33:17.421 INFO:teuthology.orchestra.run.smithi085.stdout: filestore xattr use omap = true 2022-08-01T19:33:17.421 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:33:17.422 INFO:teuthology.orchestra.run.smithi085.stdout: mon clock drift allowed = 1.000 2022-08-01T19:33:17.422 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:33:17.422 INFO:teuthology.orchestra.run.smithi085.stdout: osd crush chooseleaf type = 0 2022-08-01T19:33:17.422 INFO:teuthology.orchestra.run.smithi085.stdout: auth debug = true 2022-08-01T19:33:17.423 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:33:17.423 INFO:teuthology.orchestra.run.smithi085.stdout: ms die on old message = true 2022-08-01T19:33:17.423 INFO:teuthology.orchestra.run.smithi085.stdout: ms die on bug = true 2022-08-01T19:33:17.423 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:33:17.424 INFO:teuthology.orchestra.run.smithi085.stdout: mon max pg per osd = 10000 # >= luminous 2022-08-01T19:33:17.424 INFO:teuthology.orchestra.run.smithi085.stdout: mon pg warn max object skew = 0 2022-08-01T19:33:17.424 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:33:17.424 INFO:teuthology.orchestra.run.smithi085.stdout: # disable pg_autoscaler by default for new pools 2022-08-01T19:33:17.425 INFO:teuthology.orchestra.run.smithi085.stdout: osd_pool_default_pg_autoscale_mode = off 2022-08-01T19:33:17.425 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:33:17.425 INFO:teuthology.orchestra.run.smithi085.stdout: osd pool default size = 2 2022-08-01T19:33:17.425 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:33:17.426 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd allow primary affinity = true 2022-08-01T19:33:17.426 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd allow pg remap = true 2022-08-01T19:33:17.426 INFO:teuthology.orchestra.run.smithi085.stdout: mon warn on legacy crush tunables = false 2022-08-01T19:33:17.426 INFO:teuthology.orchestra.run.smithi085.stdout: mon warn on crush straw calc version zero = false 2022-08-01T19:33:17.426 INFO:teuthology.orchestra.run.smithi085.stdout: mon warn on no sortbitwise = false 2022-08-01T19:33:17.427 INFO:teuthology.orchestra.run.smithi085.stdout: mon warn on osd down out interval zero = false 2022-08-01T19:33:17.427 INFO:teuthology.orchestra.run.smithi085.stdout: mon warn on too few osds = false 2022-08-01T19:33:17.427 INFO:teuthology.orchestra.run.smithi085.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-08-01T19:33:17.427 INFO:teuthology.orchestra.run.smithi085.stdout: mon_warn_on_pool_no_redundancy = false 2022-08-01T19:33:17.427 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:33:17.428 INFO:teuthology.orchestra.run.smithi085.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-08-01T19:33:17.428 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:33:17.428 INFO:teuthology.orchestra.run.smithi085.stdout: osd default data pool replay window = 5 2022-08-01T19:33:17.428 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:33:17.428 INFO:teuthology.orchestra.run.smithi085.stdout: mon allow pool delete = true 2022-08-01T19:33:17.429 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:33:17.429 INFO:teuthology.orchestra.run.smithi085.stdout: mon cluster log file level = debug 2022-08-01T19:33:17.429 INFO:teuthology.orchestra.run.smithi085.stdout: debug asserts on shutdown = true 2022-08-01T19:33:17.429 INFO:teuthology.orchestra.run.smithi085.stdout: mon health detail to clog = false 2022-08-01T19:33:17.430 INFO:teuthology.orchestra.run.smithi085.stdout: mon host = "172.21.15.17,172.21.15.77,[v2:172.21.15.77:3301,v1:172.21.15.77:6790]" 2022-08-01T19:33:17.430 INFO:teuthology.orchestra.run.smithi085.stdout: fsid = be659963-3cf3-4374-b3c4-cbe717b6bac2 2022-08-01T19:33:17.430 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:33:17.430 INFO:teuthology.orchestra.run.smithi085.stdout:[osd] 2022-08-01T19:33:17.431 INFO:teuthology.orchestra.run.smithi085.stdout: osd journal size = 100 2022-08-01T19:33:17.431 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:33:17.431 INFO:teuthology.orchestra.run.smithi085.stdout: osd scrub load threshold = 5.0 2022-08-01T19:33:17.431 INFO:teuthology.orchestra.run.smithi085.stdout: osd scrub max interval = 600 2022-08-01T19:33:17.431 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:33:17.432 INFO:teuthology.orchestra.run.smithi085.stdout: osd recover clone overlap = true 2022-08-01T19:33:17.432 INFO:teuthology.orchestra.run.smithi085.stdout: osd recovery max chunk = 1048576 2022-08-01T19:33:17.432 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:33:17.432 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug shutdown = true 2022-08-01T19:33:17.432 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug op order = true 2022-08-01T19:33:17.433 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug verify stray on activate = true 2022-08-01T19:33:17.433 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:33:17.433 INFO:teuthology.orchestra.run.smithi085.stdout: osd open classes on start = true 2022-08-01T19:33:17.433 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug pg log writeout = true 2022-08-01T19:33:17.434 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:33:17.434 INFO:teuthology.orchestra.run.smithi085.stdout: osd deep scrub update digest min age = 30 2022-08-01T19:33:17.434 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:33:17.434 INFO:teuthology.orchestra.run.smithi085.stdout: osd map max advance = 10 2022-08-01T19:33:17.434 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:33:17.435 INFO:teuthology.orchestra.run.smithi085.stdout: journal zero on create = true 2022-08-01T19:33:17.435 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:33:17.435 INFO:teuthology.orchestra.run.smithi085.stdout: filestore ondisk finisher threads = 3 2022-08-01T19:33:17.435 INFO:teuthology.orchestra.run.smithi085.stdout: filestore apply finisher threads = 3 2022-08-01T19:33:17.436 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:33:17.436 INFO:teuthology.orchestra.run.smithi085.stdout: bdev debug aio = true 2022-08-01T19:33:17.436 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug misdirected ops = true 2022-08-01T19:33:17.436 INFO:teuthology.orchestra.run.smithi085.stdout: bluestore block size = 96636764160 2022-08-01T19:33:17.436 INFO:teuthology.orchestra.run.smithi085.stdout: bluestore compression mode = aggressive 2022-08-01T19:33:17.437 INFO:teuthology.orchestra.run.smithi085.stdout: bluestore fsck on mount = True 2022-08-01T19:33:17.437 INFO:teuthology.orchestra.run.smithi085.stdout: debug bluefs = 20 2022-08-01T19:33:17.437 INFO:teuthology.orchestra.run.smithi085.stdout: debug bluestore = 20 2022-08-01T19:33:17.437 INFO:teuthology.orchestra.run.smithi085.stdout: debug ms = 1 2022-08-01T19:33:17.437 INFO:teuthology.orchestra.run.smithi085.stdout: debug osd = 20 2022-08-01T19:33:17.438 INFO:teuthology.orchestra.run.smithi085.stdout: debug rocksdb = 10 2022-08-01T19:33:17.438 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd backfillfull_ratio = 0.85 2022-08-01T19:33:17.438 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd full ratio = 0.9 2022-08-01T19:33:17.438 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd nearfull ratio = 0.8 2022-08-01T19:33:17.439 INFO:teuthology.orchestra.run.smithi085.stdout: osd failsafe full ratio = 0.95 2022-08-01T19:33:17.439 INFO:teuthology.orchestra.run.smithi085.stdout: osd objectstore = bluestore 2022-08-01T19:33:17.439 INFO:teuthology.orchestra.run.smithi085.stdout: osd op complaint time = 180 2022-08-01T19:33:17.439 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:33:17.439 INFO:teuthology.orchestra.run.smithi085.stdout:[mgr] 2022-08-01T19:33:17.440 INFO:teuthology.orchestra.run.smithi085.stdout: debug ms = 1 2022-08-01T19:33:17.440 INFO:teuthology.orchestra.run.smithi085.stdout: debug mgr = 20 2022-08-01T19:33:17.440 INFO:teuthology.orchestra.run.smithi085.stdout: debug mon = 20 2022-08-01T19:33:17.440 INFO:teuthology.orchestra.run.smithi085.stdout: debug auth = 20 2022-08-01T19:33:17.441 INFO:teuthology.orchestra.run.smithi085.stdout: mon reweight min pgs per osd = 4 2022-08-01T19:33:17.441 INFO:teuthology.orchestra.run.smithi085.stdout: mon reweight min bytes per osd = 10 2022-08-01T19:33:17.441 INFO:teuthology.orchestra.run.smithi085.stdout: mgr/telemetry/nag = false 2022-08-01T19:33:17.441 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:33:17.442 INFO:teuthology.orchestra.run.smithi085.stdout:[mon] 2022-08-01T19:33:17.442 INFO:teuthology.orchestra.run.smithi085.stdout: debug ms = 1 2022-08-01T19:33:17.442 INFO:teuthology.orchestra.run.smithi085.stdout: debug mon = 20 2022-08-01T19:33:17.442 INFO:teuthology.orchestra.run.smithi085.stdout: debug paxos = 20 2022-08-01T19:33:17.442 INFO:teuthology.orchestra.run.smithi085.stdout: debug auth = 20 2022-08-01T19:33:17.443 INFO:teuthology.orchestra.run.smithi085.stdout: mon data avail warn = 5 2022-08-01T19:33:17.443 INFO:teuthology.orchestra.run.smithi085.stdout: mon mgr mkfs grace = 240 2022-08-01T19:33:17.443 INFO:teuthology.orchestra.run.smithi085.stdout: mon reweight min pgs per osd = 4 2022-08-01T19:33:17.443 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd reporter subtree level = osd 2022-08-01T19:33:17.444 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd prime pg temp = true 2022-08-01T19:33:17.444 INFO:teuthology.orchestra.run.smithi085.stdout: mon reweight min bytes per osd = 10 2022-08-01T19:33:17.444 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:33:17.444 INFO:teuthology.orchestra.run.smithi085.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-08-01T19:33:17.444 INFO:teuthology.orchestra.run.smithi085.stdout: auth mon ticket ttl = 660 # 11m 2022-08-01T19:33:17.445 INFO:teuthology.orchestra.run.smithi085.stdout: auth service ticket ttl = 240 # 4m 2022-08-01T19:33:17.445 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:33:17.445 INFO:teuthology.orchestra.run.smithi085.stdout: # don't complain about insecure global_id in the test suite 2022-08-01T19:33:17.445 INFO:teuthology.orchestra.run.smithi085.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-08-01T19:33:17.445 INFO:teuthology.orchestra.run.smithi085.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-08-01T19:33:17.446 INFO:teuthology.orchestra.run.smithi085.stdout: mon op complaint time = 120 2022-08-01T19:33:17.446 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:33:17.446 INFO:teuthology.orchestra.run.smithi085.stdout:[client] 2022-08-01T19:33:17.446 INFO:teuthology.orchestra.run.smithi085.stdout: rgw cache enabled = true 2022-08-01T19:33:17.447 INFO:teuthology.orchestra.run.smithi085.stdout: rgw enable ops log = true 2022-08-01T19:33:17.447 INFO:teuthology.orchestra.run.smithi085.stdout: rgw enable usage log = true 2022-08-01T19:33:17.447 INFO:teuthology.orchestra.run.smithi085.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-08-01T19:33:17.447 INFO:teuthology.orchestra.run.smithi085.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-08-01T19:33:17.447 INFO:teuthology.orchestra.run.smithi085.stdout: client mount timeout = 600 2022-08-01T19:33:17.448 INFO:teuthology.orchestra.run.smithi085.stdout: debug client = 20 2022-08-01T19:33:17.448 INFO:teuthology.orchestra.run.smithi085.stdout: debug ms = 1 2022-08-01T19:33:17.448 INFO:teuthology.orchestra.run.smithi085.stdout: rados mon op timeout = 15m 2022-08-01T19:33:17.448 INFO:teuthology.orchestra.run.smithi085.stdout: rados osd op timeout = 15m 2022-08-01T19:33:17.448 INFO:teuthology.orchestra.run.smithi085.stdout:[mon.a] 2022-08-01T19:33:17.449 INFO:teuthology.orchestra.run.smithi085.stdout:[mon.b] 2022-08-01T19:33:17.449 INFO:teuthology.orchestra.run.smithi085.stdout:[mon.c] 2022-08-01T19:33:17.449 INFO:teuthology.orchestra.run.smithi085.stdout:[mds.a] 2022-08-01T19:33:17.449 INFO:teuthology.orchestra.run.smithi085.stdout:[mds.c] 2022-08-01T19:33:17.450 INFO:teuthology.orchestra.run.smithi085.stdout:[mds.b] 2022-08-01T19:33:17.450 INFO:teuthology.orchestra.run.smithi085.stdout:[mds] 2022-08-01T19:33:17.450 INFO:teuthology.orchestra.run.smithi085.stdout: debug mds = 20 2022-08-01T19:33:17.450 INFO:teuthology.orchestra.run.smithi085.stdout: debug ms = 1 2022-08-01T19:33:17.450 INFO:teuthology.orchestra.run.smithi085.stdout: mds bal frag = True 2022-08-01T19:33:17.451 INFO:teuthology.orchestra.run.smithi085.stdout: mds bal fragment size max = 10000 2022-08-01T19:33:17.451 INFO:teuthology.orchestra.run.smithi085.stdout: mds bal merge size = 5 2022-08-01T19:33:17.451 INFO:teuthology.orchestra.run.smithi085.stdout: mds bal split bits = 3 2022-08-01T19:33:17.451 INFO:teuthology.orchestra.run.smithi085.stdout: mds bal split size = 100 2022-08-01T19:33:17.452 INFO:teuthology.orchestra.run.smithi085.stdout: mds debug frag = True 2022-08-01T19:33:17.452 INFO:teuthology.orchestra.run.smithi085.stdout: mds debug scatterstat = True 2022-08-01T19:33:17.452 INFO:teuthology.orchestra.run.smithi085.stdout: mds op complaint time = 180 2022-08-01T19:33:17.452 INFO:teuthology.orchestra.run.smithi085.stdout: mds verify scatter = True 2022-08-01T19:33:17.452 INFO:teuthology.orchestra.run.smithi085.stdout: osd op complaint time = 180 2022-08-01T19:33:17.453 INFO:teuthology.orchestra.run.smithi085.stdout: rados mon op timeout = 15m 2022-08-01T19:33:17.453 INFO:teuthology.orchestra.run.smithi085.stdout: rados osd op timeout = 15m 2022-08-01T19:33:17.453 INFO:teuthology.orchestra.run.smithi077.stdout:[global] 2022-08-01T19:33:17.453 INFO:teuthology.orchestra.run.smithi077.stdout: chdir = "" 2022-08-01T19:33:17.454 INFO:teuthology.orchestra.run.smithi077.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-08-01T19:33:17.454 INFO:teuthology.orchestra.run.smithi077.stdout: auth supported = cephx 2022-08-01T19:33:17.454 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:33:17.454 INFO:teuthology.orchestra.run.smithi077.stdout: filestore xattr use omap = true 2022-08-01T19:33:17.454 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:33:17.455 INFO:teuthology.orchestra.run.smithi077.stdout: mon clock drift allowed = 1.000 2022-08-01T19:33:17.455 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:33:17.455 INFO:teuthology.orchestra.run.smithi077.stdout: osd crush chooseleaf type = 0 2022-08-01T19:33:17.455 INFO:teuthology.orchestra.run.smithi077.stdout: auth debug = true 2022-08-01T19:33:17.456 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:33:17.456 INFO:teuthology.orchestra.run.smithi077.stdout: ms die on old message = true 2022-08-01T19:33:17.456 INFO:teuthology.orchestra.run.smithi077.stdout: ms die on bug = true 2022-08-01T19:33:17.456 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:33:17.457 INFO:teuthology.orchestra.run.smithi077.stdout: mon max pg per osd = 10000 # >= luminous 2022-08-01T19:33:17.457 INFO:teuthology.orchestra.run.smithi077.stdout: mon pg warn max object skew = 0 2022-08-01T19:33:17.457 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:33:17.457 INFO:teuthology.orchestra.run.smithi077.stdout: # disable pg_autoscaler by default for new pools 2022-08-01T19:33:17.457 INFO:teuthology.orchestra.run.smithi077.stdout: osd_pool_default_pg_autoscale_mode = off 2022-08-01T19:33:17.458 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:33:17.458 INFO:teuthology.orchestra.run.smithi077.stdout: osd pool default size = 2 2022-08-01T19:33:17.458 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:33:17.458 INFO:teuthology.orchestra.run.smithi077.stdout: mon osd allow primary affinity = true 2022-08-01T19:33:17.458 INFO:teuthology.orchestra.run.smithi077.stdout: mon osd allow pg remap = true 2022-08-01T19:33:17.459 INFO:teuthology.orchestra.run.smithi077.stdout: mon warn on legacy crush tunables = false 2022-08-01T19:33:17.459 INFO:teuthology.orchestra.run.smithi077.stdout: mon warn on crush straw calc version zero = false 2022-08-01T19:33:17.459 INFO:teuthology.orchestra.run.smithi077.stdout: mon warn on no sortbitwise = false 2022-08-01T19:33:17.459 INFO:teuthology.orchestra.run.smithi077.stdout: mon warn on osd down out interval zero = false 2022-08-01T19:33:17.460 INFO:teuthology.orchestra.run.smithi077.stdout: mon warn on too few osds = false 2022-08-01T19:33:17.460 INFO:teuthology.orchestra.run.smithi077.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-08-01T19:33:17.460 INFO:teuthology.orchestra.run.smithi077.stdout: mon_warn_on_pool_no_redundancy = false 2022-08-01T19:33:17.460 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:33:17.460 INFO:teuthology.orchestra.run.smithi077.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-08-01T19:33:17.461 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:33:17.461 INFO:teuthology.orchestra.run.smithi077.stdout: osd default data pool replay window = 5 2022-08-01T19:33:17.461 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:33:17.461 INFO:teuthology.orchestra.run.smithi077.stdout: mon allow pool delete = true 2022-08-01T19:33:17.461 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:33:17.462 INFO:teuthology.orchestra.run.smithi077.stdout: mon cluster log file level = debug 2022-08-01T19:33:17.462 INFO:teuthology.orchestra.run.smithi077.stdout: debug asserts on shutdown = true 2022-08-01T19:33:17.462 INFO:teuthology.orchestra.run.smithi077.stdout: mon health detail to clog = false 2022-08-01T19:33:17.462 INFO:teuthology.orchestra.run.smithi077.stdout: mon host = "172.21.15.17,172.21.15.77,[v2:172.21.15.77:3301,v1:172.21.15.77:6790]" 2022-08-01T19:33:17.463 INFO:teuthology.orchestra.run.smithi077.stdout: fsid = be659963-3cf3-4374-b3c4-cbe717b6bac2 2022-08-01T19:33:17.463 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:33:17.463 INFO:teuthology.orchestra.run.smithi077.stdout:[osd] 2022-08-01T19:33:17.463 INFO:teuthology.orchestra.run.smithi077.stdout: osd journal size = 100 2022-08-01T19:33:17.463 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:33:17.464 INFO:teuthology.orchestra.run.smithi077.stdout: osd scrub load threshold = 5.0 2022-08-01T19:33:17.464 INFO:teuthology.orchestra.run.smithi077.stdout: osd scrub max interval = 600 2022-08-01T19:33:17.464 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:33:17.464 INFO:teuthology.orchestra.run.smithi077.stdout: osd recover clone overlap = true 2022-08-01T19:33:17.464 INFO:teuthology.orchestra.run.smithi077.stdout: osd recovery max chunk = 1048576 2022-08-01T19:33:17.465 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:33:17.465 INFO:teuthology.orchestra.run.smithi077.stdout: osd debug shutdown = true 2022-08-01T19:33:17.465 INFO:teuthology.orchestra.run.smithi077.stdout: osd debug op order = true 2022-08-01T19:33:17.465 INFO:teuthology.orchestra.run.smithi077.stdout: osd debug verify stray on activate = true 2022-08-01T19:33:17.465 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:33:17.466 INFO:teuthology.orchestra.run.smithi077.stdout: osd open classes on start = true 2022-08-01T19:33:17.466 INFO:teuthology.orchestra.run.smithi077.stdout: osd debug pg log writeout = true 2022-08-01T19:33:17.466 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:33:17.466 INFO:teuthology.orchestra.run.smithi077.stdout: osd deep scrub update digest min age = 30 2022-08-01T19:33:17.467 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:33:17.467 INFO:teuthology.orchestra.run.smithi077.stdout: osd map max advance = 10 2022-08-01T19:33:17.467 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:33:17.467 INFO:teuthology.orchestra.run.smithi077.stdout: journal zero on create = true 2022-08-01T19:33:17.468 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:33:17.468 INFO:teuthology.orchestra.run.smithi077.stdout: filestore ondisk finisher threads = 3 2022-08-01T19:33:17.468 INFO:teuthology.orchestra.run.smithi077.stdout: filestore apply finisher threads = 3 2022-08-01T19:33:17.468 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:33:17.468 INFO:teuthology.orchestra.run.smithi077.stdout: bdev debug aio = true 2022-08-01T19:33:17.469 INFO:teuthology.orchestra.run.smithi077.stdout: osd debug misdirected ops = true 2022-08-01T19:33:17.469 INFO:teuthology.orchestra.run.smithi077.stdout: bluestore block size = 96636764160 2022-08-01T19:33:17.469 INFO:teuthology.orchestra.run.smithi077.stdout: bluestore compression mode = aggressive 2022-08-01T19:33:17.469 INFO:teuthology.orchestra.run.smithi077.stdout: bluestore fsck on mount = True 2022-08-01T19:33:17.470 INFO:teuthology.orchestra.run.smithi077.stdout: debug bluefs = 20 2022-08-01T19:33:17.470 INFO:teuthology.orchestra.run.smithi077.stdout: debug bluestore = 20 2022-08-01T19:33:17.470 INFO:teuthology.orchestra.run.smithi077.stdout: debug ms = 1 2022-08-01T19:33:17.470 INFO:teuthology.orchestra.run.smithi077.stdout: debug osd = 20 2022-08-01T19:33:17.470 INFO:teuthology.orchestra.run.smithi077.stdout: debug rocksdb = 10 2022-08-01T19:33:17.471 INFO:teuthology.orchestra.run.smithi077.stdout: mon osd backfillfull_ratio = 0.85 2022-08-01T19:33:17.471 INFO:teuthology.orchestra.run.smithi077.stdout: mon osd full ratio = 0.9 2022-08-01T19:33:17.471 INFO:teuthology.orchestra.run.smithi077.stdout: mon osd nearfull ratio = 0.8 2022-08-01T19:33:17.471 INFO:teuthology.orchestra.run.smithi077.stdout: osd failsafe full ratio = 0.95 2022-08-01T19:33:17.471 INFO:teuthology.orchestra.run.smithi077.stdout: osd objectstore = bluestore 2022-08-01T19:33:17.472 INFO:teuthology.orchestra.run.smithi077.stdout: osd op complaint time = 180 2022-08-01T19:33:17.472 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:33:17.472 INFO:teuthology.orchestra.run.smithi077.stdout:[mgr] 2022-08-01T19:33:17.472 INFO:teuthology.orchestra.run.smithi077.stdout: debug ms = 1 2022-08-01T19:33:17.473 INFO:teuthology.orchestra.run.smithi077.stdout: debug mgr = 20 2022-08-01T19:33:17.473 INFO:teuthology.orchestra.run.smithi077.stdout: debug mon = 20 2022-08-01T19:33:17.473 INFO:teuthology.orchestra.run.smithi077.stdout: debug auth = 20 2022-08-01T19:33:17.473 INFO:teuthology.orchestra.run.smithi077.stdout: mon reweight min pgs per osd = 4 2022-08-01T19:33:17.473 INFO:teuthology.orchestra.run.smithi077.stdout: mon reweight min bytes per osd = 10 2022-08-01T19:33:17.474 INFO:teuthology.orchestra.run.smithi077.stdout: mgr/telemetry/nag = false 2022-08-01T19:33:17.474 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:33:17.474 INFO:teuthology.orchestra.run.smithi077.stdout:[mon] 2022-08-01T19:33:17.474 INFO:teuthology.orchestra.run.smithi077.stdout: debug ms = 1 2022-08-01T19:33:17.475 INFO:teuthology.orchestra.run.smithi077.stdout: debug mon = 20 2022-08-01T19:33:17.475 INFO:teuthology.orchestra.run.smithi077.stdout: debug paxos = 20 2022-08-01T19:33:17.475 INFO:teuthology.orchestra.run.smithi077.stdout: debug auth = 20 2022-08-01T19:33:17.475 INFO:teuthology.orchestra.run.smithi077.stdout: mon data avail warn = 5 2022-08-01T19:33:17.475 INFO:teuthology.orchestra.run.smithi077.stdout: mon mgr mkfs grace = 240 2022-08-01T19:33:17.476 INFO:teuthology.orchestra.run.smithi077.stdout: mon reweight min pgs per osd = 4 2022-08-01T19:33:17.476 INFO:teuthology.orchestra.run.smithi077.stdout: mon osd reporter subtree level = osd 2022-08-01T19:33:17.476 INFO:teuthology.orchestra.run.smithi077.stdout: mon osd prime pg temp = true 2022-08-01T19:33:17.476 INFO:teuthology.orchestra.run.smithi077.stdout: mon reweight min bytes per osd = 10 2022-08-01T19:33:17.476 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:33:17.477 INFO:teuthology.orchestra.run.smithi077.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-08-01T19:33:17.477 INFO:teuthology.orchestra.run.smithi077.stdout: auth mon ticket ttl = 660 # 11m 2022-08-01T19:33:17.477 INFO:teuthology.orchestra.run.smithi077.stdout: auth service ticket ttl = 240 # 4m 2022-08-01T19:33:17.477 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:33:17.478 INFO:teuthology.orchestra.run.smithi077.stdout: # don't complain about insecure global_id in the test suite 2022-08-01T19:33:17.478 INFO:teuthology.orchestra.run.smithi077.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-08-01T19:33:17.478 INFO:teuthology.orchestra.run.smithi077.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-08-01T19:33:17.478 INFO:teuthology.orchestra.run.smithi077.stdout: mon op complaint time = 120 2022-08-01T19:33:17.478 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:33:17.479 INFO:teuthology.orchestra.run.smithi077.stdout:[client] 2022-08-01T19:33:17.479 INFO:teuthology.orchestra.run.smithi077.stdout: rgw cache enabled = true 2022-08-01T19:33:17.479 INFO:teuthology.orchestra.run.smithi077.stdout: rgw enable ops log = true 2022-08-01T19:33:17.479 INFO:teuthology.orchestra.run.smithi077.stdout: rgw enable usage log = true 2022-08-01T19:33:17.480 INFO:teuthology.orchestra.run.smithi077.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-08-01T19:33:17.480 INFO:teuthology.orchestra.run.smithi077.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-08-01T19:33:17.480 INFO:teuthology.orchestra.run.smithi077.stdout: client mount timeout = 600 2022-08-01T19:33:17.480 INFO:teuthology.orchestra.run.smithi077.stdout: debug client = 20 2022-08-01T19:33:17.480 INFO:teuthology.orchestra.run.smithi077.stdout: debug ms = 1 2022-08-01T19:33:17.481 INFO:teuthology.orchestra.run.smithi077.stdout: rados mon op timeout = 15m 2022-08-01T19:33:17.481 INFO:teuthology.orchestra.run.smithi077.stdout: rados osd op timeout = 15m 2022-08-01T19:33:17.481 INFO:teuthology.orchestra.run.smithi077.stdout:[mon.a] 2022-08-01T19:33:17.481 INFO:teuthology.orchestra.run.smithi077.stdout:[mon.b] 2022-08-01T19:33:17.482 INFO:teuthology.orchestra.run.smithi077.stdout:[mon.c] 2022-08-01T19:33:17.482 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.a] 2022-08-01T19:33:17.482 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.c] 2022-08-01T19:33:17.482 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.b] 2022-08-01T19:33:17.482 INFO:teuthology.orchestra.run.smithi077.stdout:[mds] 2022-08-01T19:33:17.483 INFO:teuthology.orchestra.run.smithi077.stdout: debug mds = 20 2022-08-01T19:33:17.483 INFO:teuthology.orchestra.run.smithi077.stdout: debug ms = 1 2022-08-01T19:33:17.483 INFO:teuthology.orchestra.run.smithi077.stdout: mds bal frag = True 2022-08-01T19:33:17.483 INFO:teuthology.orchestra.run.smithi077.stdout: mds bal fragment size max = 10000 2022-08-01T19:33:17.484 INFO:teuthology.orchestra.run.smithi077.stdout: mds bal merge size = 5 2022-08-01T19:33:17.484 INFO:teuthology.orchestra.run.smithi077.stdout: mds bal split bits = 3 2022-08-01T19:33:17.484 INFO:teuthology.orchestra.run.smithi077.stdout: mds bal split size = 100 2022-08-01T19:33:17.484 INFO:teuthology.orchestra.run.smithi077.stdout: mds debug frag = True 2022-08-01T19:33:17.484 INFO:teuthology.orchestra.run.smithi077.stdout: mds debug scatterstat = True 2022-08-01T19:33:17.485 INFO:teuthology.orchestra.run.smithi077.stdout: mds op complaint time = 180 2022-08-01T19:33:17.485 INFO:teuthology.orchestra.run.smithi077.stdout: mds verify scatter = True 2022-08-01T19:33:17.485 INFO:teuthology.orchestra.run.smithi077.stdout: osd op complaint time = 180 2022-08-01T19:33:17.485 INFO:teuthology.orchestra.run.smithi077.stdout: rados mon op timeout = 15m 2022-08-01T19:33:17.485 INFO:teuthology.orchestra.run.smithi077.stdout: rados osd op timeout = 15m 2022-08-01T19:33:17.486 INFO:tasks.ceph:Creating admin key on mon.a... 2022-08-01T19:33:17.486 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2022-08-01T19:33:17.534 INFO:tasks.ceph:Copying monmap to all nodes... 2022-08-01T19:33:17.547 DEBUG:teuthology.orchestra.remote:smithi017:/etc/ceph/ceph.keyring is 206B 2022-08-01T19:33:17.568 DEBUG:teuthology.orchestra.remote:smithi017:/home/ubuntu/cephtest/ceph.monmap is 379B 2022-08-01T19:33:17.582 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi017.front.sepia.ceph.com 2022-08-01T19:33:17.583 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2022-08-01T19:33:17.583 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/ceph/ceph.keyring 2022-08-01T19:33:17.584 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-08-01T19:33:17.609 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2022-08-01T19:33:17.609 DEBUG:teuthology.orchestra.run.smithi017:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-08-01T19:33:17.659 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi077.front.sepia.ceph.com 2022-08-01T19:33:17.660 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2022-08-01T19:33:17.660 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/ceph/ceph.keyring 2022-08-01T19:33:17.661 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-08-01T19:33:17.688 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2022-08-01T19:33:17.689 DEBUG:teuthology.orchestra.run.smithi077:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-08-01T19:33:17.695 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi085.front.sepia.ceph.com 2022-08-01T19:33:17.695 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-08-01T19:33:17.696 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd of=/etc/ceph/ceph.keyring 2022-08-01T19:33:17.696 DEBUG:teuthology.orchestra.run.smithi085:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-08-01T19:33:17.723 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-08-01T19:33:17.724 DEBUG:teuthology.orchestra.run.smithi085:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-08-01T19:33:17.729 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi190.front.sepia.ceph.com 2022-08-01T19:33:17.730 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2022-08-01T19:33:17.731 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.keyring 2022-08-01T19:33:17.731 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-08-01T19:33:17.758 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2022-08-01T19:33:17.759 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-08-01T19:33:17.807 INFO:tasks.ceph:Setting up mon nodes... 2022-08-01T19:33:17.807 INFO:tasks.ceph:Setting up mgr nodes... 2022-08-01T19:33:17.808 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2022-08-01T19:33:17.852 INFO:teuthology.orchestra.run.smithi017.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2022-08-01T19:33:17.855 DEBUG:teuthology.orchestra.run.smithi077:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2022-08-01T19:33:17.915 INFO:teuthology.orchestra.run.smithi077.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2022-08-01T19:33:17.917 INFO:tasks.ceph:Setting up mds nodes... 2022-08-01T19:33:17.918 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2022-08-01T19:33:17.962 INFO:teuthology.orchestra.run.smithi017.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2022-08-01T19:33:17.965 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2022-08-01T19:33:18.025 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2022-08-01T19:33:18.067 INFO:teuthology.orchestra.run.smithi017.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2022-08-01T19:33:18.069 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2022-08-01T19:33:18.080 DEBUG:teuthology.orchestra.run.smithi077:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2022-08-01T19:33:18.125 INFO:teuthology.orchestra.run.smithi077.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2022-08-01T19:33:18.127 DEBUG:teuthology.orchestra.run.smithi077:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2022-08-01T19:33:18.140 INFO:tasks.ceph_client:Setting up client nodes... 2022-08-01T19:33:18.141 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2022-08-01T19:33:18.184 INFO:teuthology.orchestra.run.smithi085.stdout:creating /etc/ceph/ceph.client.0.keyring 2022-08-01T19:33:18.197 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2022-08-01T19:33:18.237 INFO:teuthology.orchestra.run.smithi190.stdout:creating /etc/ceph/ceph.client.1.keyring 2022-08-01T19:33:18.251 INFO:tasks.ceph:Running mkfs on osd nodes... 2022-08-01T19:33:18.252 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi017.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi077.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2022-08-01T19:33:18.252 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2022-08-01T19:33:18.267 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-08-01T19:33:18.267 INFO:tasks.ceph:role: osd.0 2022-08-01T19:33:18.268 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi017.front.sepia.ceph.com 2022-08-01T19:33:18.268 DEBUG:teuthology.orchestra.run.smithi017:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2022-08-01T19:33:18.352 INFO:teuthology.orchestra.run.smithi017.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2022-08-01T19:33:18.352 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 attr=2, projid32bit=1 2022-08-01T19:33:18.353 INFO:teuthology.orchestra.run.smithi017.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-08-01T19:33:18.353 INFO:teuthology.orchestra.run.smithi017.stdout: = reflink=1 2022-08-01T19:33:18.354 INFO:teuthology.orchestra.run.smithi017.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-08-01T19:33:18.354 INFO:teuthology.orchestra.run.smithi017.stdout: = sunit=0 swidth=0 blks 2022-08-01T19:33:18.355 INFO:teuthology.orchestra.run.smithi017.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-08-01T19:33:18.355 INFO:teuthology.orchestra.run.smithi017.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-08-01T19:33:18.355 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-08-01T19:33:18.356 INFO:teuthology.orchestra.run.smithi017.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-08-01T19:33:18.357 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi017.front.sepia.ceph.com -o noatime 2022-08-01T19:33:18.358 DEBUG:teuthology.orchestra.run.smithi017:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2022-08-01T19:33:18.379 DEBUG:teuthology.orchestra.run.smithi017:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2022-08-01T19:33:18.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:33:18.390 INFO:teuthology.orchestra.run.smithi017.stderr:sudo: /sbin/restorecon: command not found 2022-08-01T19:33:18.390 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2022-08-01T19:33:18.447 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-08-01T19:33:18.447 INFO:tasks.ceph:role: osd.1 2022-08-01T19:33:18.447 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi017.front.sepia.ceph.com 2022-08-01T19:33:18.448 DEBUG:teuthology.orchestra.run.smithi017:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2022-08-01T19:33:18.491 INFO:teuthology.orchestra.run.smithi017.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2022-08-01T19:33:18.491 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 attr=2, projid32bit=1 2022-08-01T19:33:18.492 INFO:teuthology.orchestra.run.smithi017.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-08-01T19:33:18.492 INFO:teuthology.orchestra.run.smithi017.stdout: = reflink=1 2022-08-01T19:33:18.493 INFO:teuthology.orchestra.run.smithi017.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-08-01T19:33:18.493 INFO:teuthology.orchestra.run.smithi017.stdout: = sunit=0 swidth=0 blks 2022-08-01T19:33:18.493 INFO:teuthology.orchestra.run.smithi017.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-08-01T19:33:18.494 INFO:teuthology.orchestra.run.smithi017.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-08-01T19:33:18.494 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-08-01T19:33:18.494 INFO:teuthology.orchestra.run.smithi017.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-08-01T19:33:18.495 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi017.front.sepia.ceph.com -o noatime 2022-08-01T19:33:18.496 DEBUG:teuthology.orchestra.run.smithi017:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2022-08-01T19:33:18.516 DEBUG:teuthology.orchestra.run.smithi017:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2022-08-01T19:33:18.568 INFO:teuthology.orchestra.run.smithi017.stderr:sudo: /sbin/restorecon: command not found 2022-08-01T19:33:18.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:33:18.570 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2022-08-01T19:33:18.581 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-08-01T19:33:18.581 INFO:tasks.ceph:role: osd.2 2022-08-01T19:33:18.581 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi017.front.sepia.ceph.com 2022-08-01T19:33:18.582 DEBUG:teuthology.orchestra.run.smithi017:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2022-08-01T19:33:18.625 INFO:teuthology.orchestra.run.smithi017.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2022-08-01T19:33:18.626 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 attr=2, projid32bit=1 2022-08-01T19:33:18.627 INFO:teuthology.orchestra.run.smithi017.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-08-01T19:33:18.628 INFO:teuthology.orchestra.run.smithi017.stdout: = reflink=1 2022-08-01T19:33:18.628 INFO:teuthology.orchestra.run.smithi017.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-08-01T19:33:18.628 INFO:teuthology.orchestra.run.smithi017.stdout: = sunit=0 swidth=0 blks 2022-08-01T19:33:18.629 INFO:teuthology.orchestra.run.smithi017.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-08-01T19:33:18.629 INFO:teuthology.orchestra.run.smithi017.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-08-01T19:33:18.629 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-08-01T19:33:18.629 INFO:teuthology.orchestra.run.smithi017.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-08-01T19:33:18.631 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi017.front.sepia.ceph.com -o noatime 2022-08-01T19:33:18.631 DEBUG:teuthology.orchestra.run.smithi017:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2022-08-01T19:33:18.651 DEBUG:teuthology.orchestra.run.smithi017:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2022-08-01T19:33:18.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:33:18.704 INFO:teuthology.orchestra.run.smithi017.stderr:sudo: /sbin/restorecon: command not found 2022-08-01T19:33:18.704 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2022-08-01T19:33:18.716 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-08-01T19:33:18.717 INFO:tasks.ceph:role: osd.3 2022-08-01T19:33:18.717 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi017.front.sepia.ceph.com 2022-08-01T19:33:18.718 DEBUG:teuthology.orchestra.run.smithi017:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2022-08-01T19:33:18.766 INFO:teuthology.orchestra.run.smithi017.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2022-08-01T19:33:18.767 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 attr=2, projid32bit=1 2022-08-01T19:33:18.767 INFO:teuthology.orchestra.run.smithi017.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-08-01T19:33:18.767 INFO:teuthology.orchestra.run.smithi017.stdout: = reflink=1 2022-08-01T19:33:18.767 INFO:teuthology.orchestra.run.smithi017.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-08-01T19:33:18.768 INFO:teuthology.orchestra.run.smithi017.stdout: = sunit=0 swidth=0 blks 2022-08-01T19:33:18.768 INFO:teuthology.orchestra.run.smithi017.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-08-01T19:33:18.768 INFO:teuthology.orchestra.run.smithi017.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-08-01T19:33:18.768 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-08-01T19:33:18.769 INFO:teuthology.orchestra.run.smithi017.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-08-01T19:33:18.770 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi017.front.sepia.ceph.com -o noatime 2022-08-01T19:33:18.771 DEBUG:teuthology.orchestra.run.smithi017:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2022-08-01T19:33:18.791 DEBUG:teuthology.orchestra.run.smithi017:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2022-08-01T19:33:18.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:33:18.803 INFO:teuthology.orchestra.run.smithi017.stderr:sudo: /sbin/restorecon: command not found 2022-08-01T19:33:18.804 DEBUG:teuthology.orchestra.run.smithi017:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-08-01T19:33:18.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:18.900+0000 7fae96932d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2022-08-01T19:33:18.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:18.900+0000 7fae96932d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2022-08-01T19:33:18.907 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:18.900+0000 7fae96932d80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-08-01T19:33:18.907 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:18.900+0000 7fae96932d80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-08-01T19:33:18.908 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:18.900+0000 7fae96932d80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-08-01T19:33:18.908 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:18.904+0000 7fae96932d80 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2022-08-01T19:33:18.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:18.920+0000 7fae96932d80 -1 freelist read_size_meta_from_db missing size meta in DB 2022-08-01T19:33:21.368 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2022-08-01T19:33:21.383 DEBUG:teuthology.orchestra.run.smithi017:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-08-01T19:33:21.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:21.416+0000 7fa483dded80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2022-08-01T19:33:21.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:21.416+0000 7fa483dded80 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2022-08-01T19:33:21.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:21.416+0000 7fa483dded80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-08-01T19:33:21.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:21.416+0000 7fa483dded80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-08-01T19:33:21.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:21.416+0000 7fa483dded80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-08-01T19:33:21.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:21.416+0000 7fa483dded80 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2022-08-01T19:33:21.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:21.436+0000 7fa483dded80 -1 freelist read_size_meta_from_db missing size meta in DB 2022-08-01T19:33:23.864 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2022-08-01T19:33:23.919 DEBUG:teuthology.orchestra.run.smithi017:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-08-01T19:33:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:23.952+0000 7f7cc6275d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2022-08-01T19:33:23.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:23.952+0000 7f7cc6275d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2022-08-01T19:33:23.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:23.952+0000 7f7cc6275d80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-08-01T19:33:23.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:23.952+0000 7f7cc6275d80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-08-01T19:33:23.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:23.952+0000 7f7cc6275d80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-08-01T19:33:23.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:23.952+0000 7f7cc6275d80 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2022-08-01T19:33:23.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:23.968+0000 7f7cc6275d80 -1 freelist read_size_meta_from_db missing size meta in DB 2022-08-01T19:33:26.415 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2022-08-01T19:33:26.432 DEBUG:teuthology.orchestra.run.smithi017:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-08-01T19:33:26.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:26.464+0000 7fd228534d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2022-08-01T19:33:26.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:26.464+0000 7fd228534d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2022-08-01T19:33:26.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:26.464+0000 7fd228534d80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-08-01T19:33:26.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:26.464+0000 7fd228534d80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-08-01T19:33:26.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:26.464+0000 7fd228534d80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-08-01T19:33:26.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:26.464+0000 7fd228534d80 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2022-08-01T19:33:26.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:26.484+0000 7fd228534d80 -1 freelist read_size_meta_from_db missing size meta in DB 2022-08-01T19:33:28.935 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2022-08-01T19:33:28.993 DEBUG:teuthology.orchestra.run.smithi077:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2022-08-01T19:33:29.009 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-08-01T19:33:29.010 INFO:tasks.ceph:role: osd.4 2022-08-01T19:33:29.010 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi077.front.sepia.ceph.com 2022-08-01T19:33:29.011 DEBUG:teuthology.orchestra.run.smithi077:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2022-08-01T19:33:29.533 INFO:teuthology.orchestra.run.smithi077.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2022-08-01T19:33:29.533 INFO:teuthology.orchestra.run.smithi077.stdout: = sectsz=512 attr=2, projid32bit=1 2022-08-01T19:33:29.533 INFO:teuthology.orchestra.run.smithi077.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-08-01T19:33:29.534 INFO:teuthology.orchestra.run.smithi077.stdout: = reflink=1 2022-08-01T19:33:29.534 INFO:teuthology.orchestra.run.smithi077.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-08-01T19:33:29.534 INFO:teuthology.orchestra.run.smithi077.stdout: = sunit=0 swidth=0 blks 2022-08-01T19:33:29.534 INFO:teuthology.orchestra.run.smithi077.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-08-01T19:33:29.535 INFO:teuthology.orchestra.run.smithi077.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-08-01T19:33:29.535 INFO:teuthology.orchestra.run.smithi077.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-08-01T19:33:29.535 INFO:teuthology.orchestra.run.smithi077.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-08-01T19:33:29.536 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi077.front.sepia.ceph.com -o noatime 2022-08-01T19:33:29.537 DEBUG:teuthology.orchestra.run.smithi077:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2022-08-01T19:33:29.624 DEBUG:teuthology.orchestra.run.smithi077:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2022-08-01T19:33:29.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:33:29.637 INFO:teuthology.orchestra.run.smithi077.stderr:sudo: /sbin/restorecon: command not found 2022-08-01T19:33:29.637 DEBUG:teuthology.orchestra.run.smithi077:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2022-08-01T19:33:29.652 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-08-01T19:33:29.652 INFO:tasks.ceph:role: osd.5 2022-08-01T19:33:29.653 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi077.front.sepia.ceph.com 2022-08-01T19:33:29.654 DEBUG:teuthology.orchestra.run.smithi077:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2022-08-01T19:33:30.192 INFO:teuthology.orchestra.run.smithi077.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2022-08-01T19:33:30.193 INFO:teuthology.orchestra.run.smithi077.stdout: = sectsz=512 attr=2, projid32bit=1 2022-08-01T19:33:30.193 INFO:teuthology.orchestra.run.smithi077.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-08-01T19:33:30.193 INFO:teuthology.orchestra.run.smithi077.stdout: = reflink=1 2022-08-01T19:33:30.193 INFO:teuthology.orchestra.run.smithi077.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-08-01T19:33:30.194 INFO:teuthology.orchestra.run.smithi077.stdout: = sunit=0 swidth=0 blks 2022-08-01T19:33:30.194 INFO:teuthology.orchestra.run.smithi077.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-08-01T19:33:30.194 INFO:teuthology.orchestra.run.smithi077.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-08-01T19:33:30.196 INFO:teuthology.orchestra.run.smithi077.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-08-01T19:33:30.196 INFO:teuthology.orchestra.run.smithi077.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-08-01T19:33:30.198 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi077.front.sepia.ceph.com -o noatime 2022-08-01T19:33:30.199 DEBUG:teuthology.orchestra.run.smithi077:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2022-08-01T19:33:30.243 DEBUG:teuthology.orchestra.run.smithi077:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2022-08-01T19:33:30.251 INFO:teuthology.orchestra.run.smithi077.stderr:sudo: /sbin/restorecon: command not found 2022-08-01T19:33:30.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:33:30.253 DEBUG:teuthology.orchestra.run.smithi077:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2022-08-01T19:33:30.264 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-08-01T19:33:30.265 INFO:tasks.ceph:role: osd.6 2022-08-01T19:33:30.265 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi077.front.sepia.ceph.com 2022-08-01T19:33:30.265 DEBUG:teuthology.orchestra.run.smithi077:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2022-08-01T19:33:30.770 INFO:teuthology.orchestra.run.smithi077.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2022-08-01T19:33:30.771 INFO:teuthology.orchestra.run.smithi077.stdout: = sectsz=512 attr=2, projid32bit=1 2022-08-01T19:33:30.771 INFO:teuthology.orchestra.run.smithi077.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-08-01T19:33:30.771 INFO:teuthology.orchestra.run.smithi077.stdout: = reflink=1 2022-08-01T19:33:30.772 INFO:teuthology.orchestra.run.smithi077.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-08-01T19:33:30.772 INFO:teuthology.orchestra.run.smithi077.stdout: = sunit=0 swidth=0 blks 2022-08-01T19:33:30.772 INFO:teuthology.orchestra.run.smithi077.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-08-01T19:33:30.772 INFO:teuthology.orchestra.run.smithi077.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-08-01T19:33:30.773 INFO:teuthology.orchestra.run.smithi077.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-08-01T19:33:30.773 INFO:teuthology.orchestra.run.smithi077.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-08-01T19:33:30.774 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi077.front.sepia.ceph.com -o noatime 2022-08-01T19:33:30.774 DEBUG:teuthology.orchestra.run.smithi077:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2022-08-01T19:33:30.828 DEBUG:teuthology.orchestra.run.smithi077:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2022-08-01T19:33:30.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:33:30.842 INFO:teuthology.orchestra.run.smithi077.stderr:sudo: /sbin/restorecon: command not found 2022-08-01T19:33:30.842 DEBUG:teuthology.orchestra.run.smithi077:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2022-08-01T19:33:30.899 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-08-01T19:33:30.900 INFO:tasks.ceph:role: osd.7 2022-08-01T19:33:30.900 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi077.front.sepia.ceph.com 2022-08-01T19:33:30.901 DEBUG:teuthology.orchestra.run.smithi077:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2022-08-01T19:33:31.400 INFO:teuthology.orchestra.run.smithi077.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2022-08-01T19:33:31.400 INFO:teuthology.orchestra.run.smithi077.stdout: = sectsz=512 attr=2, projid32bit=1 2022-08-01T19:33:31.401 INFO:teuthology.orchestra.run.smithi077.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-08-01T19:33:31.401 INFO:teuthology.orchestra.run.smithi077.stdout: = reflink=1 2022-08-01T19:33:31.401 INFO:teuthology.orchestra.run.smithi077.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-08-01T19:33:31.401 INFO:teuthology.orchestra.run.smithi077.stdout: = sunit=0 swidth=0 blks 2022-08-01T19:33:31.402 INFO:teuthology.orchestra.run.smithi077.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-08-01T19:33:31.402 INFO:teuthology.orchestra.run.smithi077.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-08-01T19:33:31.402 INFO:teuthology.orchestra.run.smithi077.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-08-01T19:33:31.402 INFO:teuthology.orchestra.run.smithi077.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-08-01T19:33:31.403 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi077.front.sepia.ceph.com -o noatime 2022-08-01T19:33:31.404 DEBUG:teuthology.orchestra.run.smithi077:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2022-08-01T19:33:31.460 DEBUG:teuthology.orchestra.run.smithi077:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2022-08-01T19:33:31.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:33:31.472 INFO:teuthology.orchestra.run.smithi077.stderr:sudo: /sbin/restorecon: command not found 2022-08-01T19:33:31.472 DEBUG:teuthology.orchestra.run.smithi077:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-08-01T19:33:31.530 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:31.528+0000 7f61762ffd80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2022-08-01T19:33:31.531 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:31.528+0000 7f61762ffd80 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2022-08-01T19:33:31.531 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:31.528+0000 7f61762ffd80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-08-01T19:33:31.532 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:31.528+0000 7f61762ffd80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-08-01T19:33:31.532 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:31.528+0000 7f61762ffd80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-08-01T19:33:31.533 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:31.532+0000 7f61762ffd80 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2022-08-01T19:33:31.556 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:31.556+0000 7f61762ffd80 -1 freelist read_size_meta_from_db missing size meta in DB 2022-08-01T19:33:33.975 DEBUG:teuthology.orchestra.run.smithi077:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2022-08-01T19:33:33.992 DEBUG:teuthology.orchestra.run.smithi077:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-08-01T19:33:34.077 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:34.076+0000 7fc83ff58d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2022-08-01T19:33:34.078 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:34.076+0000 7fc83ff58d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2022-08-01T19:33:34.078 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:34.076+0000 7fc83ff58d80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-08-01T19:33:34.078 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:34.076+0000 7fc83ff58d80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-08-01T19:33:34.079 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:34.076+0000 7fc83ff58d80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-08-01T19:33:34.079 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:34.076+0000 7fc83ff58d80 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2022-08-01T19:33:34.098 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:34.096+0000 7fc83ff58d80 -1 freelist read_size_meta_from_db missing size meta in DB 2022-08-01T19:33:36.547 DEBUG:teuthology.orchestra.run.smithi077:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2022-08-01T19:33:36.564 DEBUG:teuthology.orchestra.run.smithi077:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-08-01T19:33:36.600 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:36.596+0000 7fa010a64d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2022-08-01T19:33:36.600 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:36.596+0000 7fa010a64d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2022-08-01T19:33:36.600 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:36.596+0000 7fa010a64d80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-08-01T19:33:36.601 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:36.596+0000 7fa010a64d80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-08-01T19:33:36.601 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:36.596+0000 7fa010a64d80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-08-01T19:33:36.602 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:36.600+0000 7fa010a64d80 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2022-08-01T19:33:36.620 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:36.620+0000 7fa010a64d80 -1 freelist read_size_meta_from_db missing size meta in DB 2022-08-01T19:33:39.070 DEBUG:teuthology.orchestra.run.smithi077:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2022-08-01T19:33:39.127 DEBUG:teuthology.orchestra.run.smithi077:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-08-01T19:33:39.164 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:39.164+0000 7f8fc04fbd80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2022-08-01T19:33:39.165 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:39.164+0000 7f8fc04fbd80 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2022-08-01T19:33:39.165 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:39.164+0000 7f8fc04fbd80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-08-01T19:33:39.165 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:39.164+0000 7f8fc04fbd80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-08-01T19:33:39.166 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:39.164+0000 7f8fc04fbd80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-08-01T19:33:39.166 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:39.164+0000 7f8fc04fbd80 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2022-08-01T19:33:39.183 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:39.180+0000 7f8fc04fbd80 -1 freelist read_size_meta_from_db missing size meta in DB 2022-08-01T19:33:41.650 DEBUG:teuthology.orchestra.run.smithi077:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2022-08-01T19:33:41.674 INFO:tasks.ceph:Reading keys from all nodes... 2022-08-01T19:33:41.675 DEBUG:teuthology.orchestra.run.smithi017:> mktemp 2022-08-01T19:33:41.682 INFO:teuthology.orchestra.run.smithi017.stdout:/tmp/tmp.YeWKrRf7vs 2022-08-01T19:33:41.683 DEBUG:teuthology.orchestra.run.smithi017:> sudo cp /var/lib/ceph/mgr/ceph-y/keyring /tmp/tmp.YeWKrRf7vs 2022-08-01T19:33:41.740 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0666 /tmp/tmp.YeWKrRf7vs 2022-08-01T19:33:41.760 DEBUG:teuthology.orchestra.remote:smithi017:/tmp/tmp.YeWKrRf7vs is 56B 2022-08-01T19:33:41.775 DEBUG:teuthology.orchestra.run.smithi017:> rm -fr /tmp/tmp.YeWKrRf7vs 2022-08-01T19:33:41.782 DEBUG:teuthology.orchestra.run.smithi017:> mktemp 2022-08-01T19:33:41.830 INFO:teuthology.orchestra.run.smithi017.stdout:/tmp/tmp.6qkaM55QkF 2022-08-01T19:33:41.831 DEBUG:teuthology.orchestra.run.smithi017:> sudo cp /var/lib/ceph/mds/ceph-a/keyring /tmp/tmp.6qkaM55QkF 2022-08-01T19:33:41.845 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0666 /tmp/tmp.6qkaM55QkF 2022-08-01T19:33:41.919 DEBUG:teuthology.orchestra.remote:smithi017:/tmp/tmp.6qkaM55QkF is 56B 2022-08-01T19:33:41.934 DEBUG:teuthology.orchestra.run.smithi017:> rm -fr /tmp/tmp.6qkaM55QkF 2022-08-01T19:33:41.940 DEBUG:teuthology.orchestra.run.smithi017:> mktemp 2022-08-01T19:33:41.986 INFO:teuthology.orchestra.run.smithi017.stdout:/tmp/tmp.GpMMe5Ao6Y 2022-08-01T19:33:41.987 DEBUG:teuthology.orchestra.run.smithi017:> sudo cp /var/lib/ceph/mds/ceph-c/keyring /tmp/tmp.GpMMe5Ao6Y 2022-08-01T19:33:42.002 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0666 /tmp/tmp.GpMMe5Ao6Y 2022-08-01T19:33:42.024 DEBUG:teuthology.orchestra.remote:smithi017:/tmp/tmp.GpMMe5Ao6Y is 56B 2022-08-01T19:33:42.036 DEBUG:teuthology.orchestra.run.smithi017:> rm -fr /tmp/tmp.GpMMe5Ao6Y 2022-08-01T19:33:42.042 DEBUG:teuthology.orchestra.run.smithi017:> mktemp 2022-08-01T19:33:42.090 INFO:teuthology.orchestra.run.smithi017.stdout:/tmp/tmp.wmHB7nT6Js 2022-08-01T19:33:42.091 DEBUG:teuthology.orchestra.run.smithi017:> sudo cp /var/lib/ceph/osd/ceph-0/keyring /tmp/tmp.wmHB7nT6Js 2022-08-01T19:33:42.106 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0666 /tmp/tmp.wmHB7nT6Js 2022-08-01T19:33:42.176 DEBUG:teuthology.orchestra.remote:smithi017:/tmp/tmp.wmHB7nT6Js is 56B 2022-08-01T19:33:42.190 DEBUG:teuthology.orchestra.run.smithi017:> rm -fr /tmp/tmp.wmHB7nT6Js 2022-08-01T19:33:42.196 DEBUG:teuthology.orchestra.run.smithi017:> mktemp 2022-08-01T19:33:42.242 INFO:teuthology.orchestra.run.smithi017.stdout:/tmp/tmp.5SKKxlR9yt 2022-08-01T19:33:42.243 DEBUG:teuthology.orchestra.run.smithi017:> sudo cp /var/lib/ceph/osd/ceph-1/keyring /tmp/tmp.5SKKxlR9yt 2022-08-01T19:33:42.256 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0666 /tmp/tmp.5SKKxlR9yt 2022-08-01T19:33:42.324 DEBUG:teuthology.orchestra.remote:smithi017:/tmp/tmp.5SKKxlR9yt is 56B 2022-08-01T19:33:42.338 DEBUG:teuthology.orchestra.run.smithi017:> rm -fr /tmp/tmp.5SKKxlR9yt 2022-08-01T19:33:42.345 DEBUG:teuthology.orchestra.run.smithi017:> mktemp 2022-08-01T19:33:42.394 INFO:teuthology.orchestra.run.smithi017.stdout:/tmp/tmp.Hu4YAD3C1k 2022-08-01T19:33:42.395 DEBUG:teuthology.orchestra.run.smithi017:> sudo cp /var/lib/ceph/osd/ceph-2/keyring /tmp/tmp.Hu4YAD3C1k 2022-08-01T19:33:42.412 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0666 /tmp/tmp.Hu4YAD3C1k 2022-08-01T19:33:42.475 DEBUG:teuthology.orchestra.remote:smithi017:/tmp/tmp.Hu4YAD3C1k is 56B 2022-08-01T19:33:42.490 DEBUG:teuthology.orchestra.run.smithi017:> rm -fr /tmp/tmp.Hu4YAD3C1k 2022-08-01T19:33:42.496 DEBUG:teuthology.orchestra.run.smithi017:> mktemp 2022-08-01T19:33:42.546 INFO:teuthology.orchestra.run.smithi017.stdout:/tmp/tmp.5KGRdfQ1Aw 2022-08-01T19:33:42.547 DEBUG:teuthology.orchestra.run.smithi017:> sudo cp /var/lib/ceph/osd/ceph-3/keyring /tmp/tmp.5KGRdfQ1Aw 2022-08-01T19:33:42.564 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0666 /tmp/tmp.5KGRdfQ1Aw 2022-08-01T19:33:42.627 DEBUG:teuthology.orchestra.remote:smithi017:/tmp/tmp.5KGRdfQ1Aw is 56B 2022-08-01T19:33:42.642 DEBUG:teuthology.orchestra.run.smithi017:> rm -fr /tmp/tmp.5KGRdfQ1Aw 2022-08-01T19:33:42.648 DEBUG:teuthology.orchestra.run.smithi077:> mktemp 2022-08-01T19:33:42.655 INFO:teuthology.orchestra.run.smithi077.stdout:/tmp/tmp.2UgkJI32Ux 2022-08-01T19:33:42.655 DEBUG:teuthology.orchestra.run.smithi077:> sudo cp /var/lib/ceph/mgr/ceph-x/keyring /tmp/tmp.2UgkJI32Ux 2022-08-01T19:33:42.712 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 0666 /tmp/tmp.2UgkJI32Ux 2022-08-01T19:33:42.733 DEBUG:teuthology.orchestra.remote:smithi077:/tmp/tmp.2UgkJI32Ux is 56B 2022-08-01T19:33:42.746 DEBUG:teuthology.orchestra.run.smithi077:> rm -fr /tmp/tmp.2UgkJI32Ux 2022-08-01T19:33:42.753 DEBUG:teuthology.orchestra.run.smithi077:> mktemp 2022-08-01T19:33:42.806 INFO:teuthology.orchestra.run.smithi077.stdout:/tmp/tmp.pTesV79055 2022-08-01T19:33:42.806 DEBUG:teuthology.orchestra.run.smithi077:> sudo cp /var/lib/ceph/mds/ceph-b/keyring /tmp/tmp.pTesV79055 2022-08-01T19:33:42.822 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 0666 /tmp/tmp.pTesV79055 2022-08-01T19:33:42.842 DEBUG:teuthology.orchestra.remote:smithi077:/tmp/tmp.pTesV79055 is 56B 2022-08-01T19:33:42.893 DEBUG:teuthology.orchestra.run.smithi077:> rm -fr /tmp/tmp.pTesV79055 2022-08-01T19:33:42.900 DEBUG:teuthology.orchestra.run.smithi077:> mktemp 2022-08-01T19:33:42.954 INFO:teuthology.orchestra.run.smithi077.stdout:/tmp/tmp.XtZN4S83O8 2022-08-01T19:33:42.954 DEBUG:teuthology.orchestra.run.smithi077:> sudo cp /var/lib/ceph/osd/ceph-4/keyring /tmp/tmp.XtZN4S83O8 2022-08-01T19:33:42.971 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 0666 /tmp/tmp.XtZN4S83O8 2022-08-01T19:33:43.035 DEBUG:teuthology.orchestra.remote:smithi077:/tmp/tmp.XtZN4S83O8 is 56B 2022-08-01T19:33:43.049 DEBUG:teuthology.orchestra.run.smithi077:> rm -fr /tmp/tmp.XtZN4S83O8 2022-08-01T19:33:43.055 DEBUG:teuthology.orchestra.run.smithi077:> mktemp 2022-08-01T19:33:43.102 INFO:teuthology.orchestra.run.smithi077.stdout:/tmp/tmp.YU4x5cMSXz 2022-08-01T19:33:43.102 DEBUG:teuthology.orchestra.run.smithi077:> sudo cp /var/lib/ceph/osd/ceph-5/keyring /tmp/tmp.YU4x5cMSXz 2022-08-01T19:33:43.117 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 0666 /tmp/tmp.YU4x5cMSXz 2022-08-01T19:33:43.139 DEBUG:teuthology.orchestra.remote:smithi077:/tmp/tmp.YU4x5cMSXz is 56B 2022-08-01T19:33:43.152 DEBUG:teuthology.orchestra.run.smithi077:> rm -fr /tmp/tmp.YU4x5cMSXz 2022-08-01T19:33:43.158 DEBUG:teuthology.orchestra.run.smithi077:> mktemp 2022-08-01T19:33:43.206 INFO:teuthology.orchestra.run.smithi077.stdout:/tmp/tmp.Lxn6WoiGN0 2022-08-01T19:33:43.207 DEBUG:teuthology.orchestra.run.smithi077:> sudo cp /var/lib/ceph/osd/ceph-6/keyring /tmp/tmp.Lxn6WoiGN0 2022-08-01T19:33:43.224 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 0666 /tmp/tmp.Lxn6WoiGN0 2022-08-01T19:33:43.292 DEBUG:teuthology.orchestra.remote:smithi077:/tmp/tmp.Lxn6WoiGN0 is 56B 2022-08-01T19:33:43.306 DEBUG:teuthology.orchestra.run.smithi077:> rm -fr /tmp/tmp.Lxn6WoiGN0 2022-08-01T19:33:43.313 DEBUG:teuthology.orchestra.run.smithi077:> mktemp 2022-08-01T19:33:43.361 INFO:teuthology.orchestra.run.smithi077.stdout:/tmp/tmp.YvC8sdrgY1 2022-08-01T19:33:43.362 DEBUG:teuthology.orchestra.run.smithi077:> sudo cp /var/lib/ceph/osd/ceph-7/keyring /tmp/tmp.YvC8sdrgY1 2022-08-01T19:33:43.377 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 0666 /tmp/tmp.YvC8sdrgY1 2022-08-01T19:33:43.396 DEBUG:teuthology.orchestra.remote:smithi077:/tmp/tmp.YvC8sdrgY1 is 56B 2022-08-01T19:33:43.450 DEBUG:teuthology.orchestra.run.smithi077:> rm -fr /tmp/tmp.YvC8sdrgY1 2022-08-01T19:33:43.470 DEBUG:teuthology.orchestra.remote:smithi085:/etc/ceph/ceph.client.0.keyring is 59B 2022-08-01T19:33:43.538 DEBUG:teuthology.orchestra.remote:smithi190:/etc/ceph/ceph.client.1.keyring is 59B 2022-08-01T19:33:43.554 INFO:tasks.ceph:Adding keys to all mons... 2022-08-01T19:33:43.554 DEBUG:teuthology.orchestra.run.smithi017:> sudo tee -a /etc/ceph/ceph.keyring 2022-08-01T19:33:43.557 DEBUG:teuthology.orchestra.run.smithi077:> sudo tee -a /etc/ceph/ceph.keyring 2022-08-01T19:33:43.570 INFO:teuthology.orchestra.run.smithi017.stdout:[mgr.y] 2022-08-01T19:33:43.570 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQB9KuhiDJXpMhAA8aek67ydbRgNO0h+miws+w== 2022-08-01T19:33:43.570 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.a] 2022-08-01T19:33:43.570 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQB9KuhijotxORAAVc04rsrlFj0NShsyc6Qbhg== 2022-08-01T19:33:43.571 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.c] 2022-08-01T19:33:43.571 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQB+KuhiMzMbBBAAfdT5mtpYwboNL+7f2B09KQ== 2022-08-01T19:33:43.571 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.0] 2022-08-01T19:33:43.571 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQB+Kuhia27/NRAAZXBp+nYelJVJ6Mge9r+3Vg== 2022-08-01T19:33:43.572 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.1] 2022-08-01T19:33:43.572 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQCBKuhiOggRGRAAvgA0MxwBJwmpXV6S9DTdjA== 2022-08-01T19:33:43.572 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.2] 2022-08-01T19:33:43.572 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQCDKuhi5PgOORAAhcORK4bW2T3dXPeYjcCm+g== 2022-08-01T19:33:43.573 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.3] 2022-08-01T19:33:43.573 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQCGKuhiutn6GxAAk2ucLuGbT5YV/R4xXQGh0w== 2022-08-01T19:33:43.573 INFO:teuthology.orchestra.run.smithi017.stdout:[mgr.x] 2022-08-01T19:33:43.573 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQB9Kuhi2jvCNhAAyUxiGByZ+OZr8Y7dh3Qt5w== 2022-08-01T19:33:43.574 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.b] 2022-08-01T19:33:43.574 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQB+Kuhiud2xBxAAcn6/O5ysdIp/oIbF5Y1dAw== 2022-08-01T19:33:43.574 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.4] 2022-08-01T19:33:43.574 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQCLKuhiGHTRHxAArr3TwXA0gsCAAF/Qs38wUQ== 2022-08-01T19:33:43.575 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.5] 2022-08-01T19:33:43.575 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQCOKuhiMMDbBBAAvTzBvOcv7pFWm/P/lNqB0w== 2022-08-01T19:33:43.575 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.6] 2022-08-01T19:33:43.575 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQCQKuhiG2H3IxAAufUcWqo9ULquxIrUD5vSfA== 2022-08-01T19:33:43.576 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.7] 2022-08-01T19:33:43.576 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQCTKuhiU50LChAAXuhrYBbHkk3eKzAPiEJFTg== 2022-08-01T19:33:43.576 INFO:teuthology.orchestra.run.smithi017.stdout:[client.0] 2022-08-01T19:33:43.576 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQB+Kuhig3EnCxAANM20jSIQf/AKwuRX1qwT/A== 2022-08-01T19:33:43.577 INFO:teuthology.orchestra.run.smithi017.stdout:[client.1] 2022-08-01T19:33:43.577 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQB+KuhicJR0DhAAivqEAZUhXnSwjVRwWm2bJg== 2022-08-01T19:33:43.579 INFO:teuthology.orchestra.run.smithi077.stdout:[mgr.y] 2022-08-01T19:33:43.579 INFO:teuthology.orchestra.run.smithi077.stdout: key = AQB9KuhiDJXpMhAA8aek67ydbRgNO0h+miws+w== 2022-08-01T19:33:43.579 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.a] 2022-08-01T19:33:43.579 INFO:teuthology.orchestra.run.smithi077.stdout: key = AQB9KuhijotxORAAVc04rsrlFj0NShsyc6Qbhg== 2022-08-01T19:33:43.580 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.c] 2022-08-01T19:33:43.580 INFO:teuthology.orchestra.run.smithi077.stdout: key = AQB+KuhiMzMbBBAAfdT5mtpYwboNL+7f2B09KQ== 2022-08-01T19:33:43.580 INFO:teuthology.orchestra.run.smithi077.stdout:[osd.0] 2022-08-01T19:33:43.580 INFO:teuthology.orchestra.run.smithi077.stdout: key = AQB+Kuhia27/NRAAZXBp+nYelJVJ6Mge9r+3Vg== 2022-08-01T19:33:43.581 INFO:teuthology.orchestra.run.smithi077.stdout:[osd.1] 2022-08-01T19:33:43.581 INFO:teuthology.orchestra.run.smithi077.stdout: key = AQCBKuhiOggRGRAAvgA0MxwBJwmpXV6S9DTdjA== 2022-08-01T19:33:43.581 INFO:teuthology.orchestra.run.smithi077.stdout:[osd.2] 2022-08-01T19:33:43.581 INFO:teuthology.orchestra.run.smithi077.stdout: key = AQCDKuhi5PgOORAAhcORK4bW2T3dXPeYjcCm+g== 2022-08-01T19:33:43.582 INFO:teuthology.orchestra.run.smithi077.stdout:[osd.3] 2022-08-01T19:33:43.582 INFO:teuthology.orchestra.run.smithi077.stdout: key = AQCGKuhiutn6GxAAk2ucLuGbT5YV/R4xXQGh0w== 2022-08-01T19:33:43.582 INFO:teuthology.orchestra.run.smithi077.stdout:[mgr.x] 2022-08-01T19:33:43.582 INFO:teuthology.orchestra.run.smithi077.stdout: key = AQB9Kuhi2jvCNhAAyUxiGByZ+OZr8Y7dh3Qt5w== 2022-08-01T19:33:43.583 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.b] 2022-08-01T19:33:43.583 INFO:teuthology.orchestra.run.smithi077.stdout: key = AQB+Kuhiud2xBxAAcn6/O5ysdIp/oIbF5Y1dAw== 2022-08-01T19:33:43.583 INFO:teuthology.orchestra.run.smithi077.stdout:[osd.4] 2022-08-01T19:33:43.583 INFO:teuthology.orchestra.run.smithi077.stdout: key = AQCLKuhiGHTRHxAArr3TwXA0gsCAAF/Qs38wUQ== 2022-08-01T19:33:43.584 INFO:teuthology.orchestra.run.smithi077.stdout:[osd.5] 2022-08-01T19:33:43.584 INFO:teuthology.orchestra.run.smithi077.stdout: key = AQCOKuhiMMDbBBAAvTzBvOcv7pFWm/P/lNqB0w== 2022-08-01T19:33:43.584 INFO:teuthology.orchestra.run.smithi077.stdout:[osd.6] 2022-08-01T19:33:43.584 INFO:teuthology.orchestra.run.smithi077.stdout: key = AQCQKuhiG2H3IxAAufUcWqo9ULquxIrUD5vSfA== 2022-08-01T19:33:43.585 INFO:teuthology.orchestra.run.smithi077.stdout:[osd.7] 2022-08-01T19:33:43.585 INFO:teuthology.orchestra.run.smithi077.stdout: key = AQCTKuhiU50LChAAXuhrYBbHkk3eKzAPiEJFTg== 2022-08-01T19:33:43.585 INFO:teuthology.orchestra.run.smithi077.stdout:[client.0] 2022-08-01T19:33:43.585 INFO:teuthology.orchestra.run.smithi077.stdout: key = AQB+Kuhig3EnCxAANM20jSIQf/AKwuRX1qwT/A== 2022-08-01T19:33:43.585 INFO:teuthology.orchestra.run.smithi077.stdout:[client.1] 2022-08-01T19:33:43.586 INFO:teuthology.orchestra.run.smithi077.stdout: key = AQB+KuhicJR0DhAAivqEAZUhXnSwjVRwWm2bJg== 2022-08-01T19:33:43.586 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-08-01T19:33:43.589 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-08-01T19:33:43.629 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-08-01T19:33:43.632 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-08-01T19:33:43.713 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-08-01T19:33:43.716 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-08-01T19:33:43.761 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-08-01T19:33:43.764 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-08-01T19:33:43.805 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-08-01T19:33:43.846 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-08-01T19:33:43.891 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-08-01T19:33:43.893 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-08-01T19:33:43.935 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-08-01T19:33:43.938 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-08-01T19:33:43.977 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-08-01T19:33:44.018 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-08-01T19:33:44.061 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-08-01T19:33:44.063 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-08-01T19:33:44.107 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-08-01T19:33:44.109 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-08-01T19:33:44.149 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-08-01T19:33:44.194 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-08-01T19:33:44.238 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-08-01T19:33:44.240 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-08-01T19:33:44.281 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-08-01T19:33:44.284 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-08-01T19:33:44.361 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-08-01T19:33:44.370 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-08-01T19:33:44.412 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-08-01T19:33:44.415 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-08-01T19:33:44.502 INFO:tasks.ceph:Running mkfs on mon nodes... 2022-08-01T19:33:44.502 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2022-08-01T19:33:44.519 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-08-01T19:33:44.621 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2022-08-01T19:33:44.636 DEBUG:teuthology.orchestra.run.smithi077:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2022-08-01T19:33:44.652 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-08-01T19:33:44.727 DEBUG:teuthology.orchestra.run.smithi077:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2022-08-01T19:33:44.743 DEBUG:teuthology.orchestra.run.smithi077:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2022-08-01T19:33:44.799 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-08-01T19:33:44.855 DEBUG:teuthology.orchestra.run.smithi077:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2022-08-01T19:33:44.867 DEBUG:teuthology.orchestra.run.smithi017:> rm -- /home/ubuntu/cephtest/ceph.monmap 2022-08-01T19:33:44.870 DEBUG:teuthology.orchestra.run.smithi077:> rm -- /home/ubuntu/cephtest/ceph.monmap 2022-08-01T19:33:44.919 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2022-08-01T19:33:44.919 INFO:tasks.ceph.mon.a:Restarting daemon 2022-08-01T19:33:44.920 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2022-08-01T19:33:44.923 INFO:tasks.ceph.mon.a:Started 2022-08-01T19:33:44.924 INFO:tasks.ceph.mon.b:Restarting daemon 2022-08-01T19:33:44.924 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2022-08-01T19:33:44.927 INFO:tasks.ceph.mon.b:Started 2022-08-01T19:33:44.927 INFO:tasks.ceph.mon.c:Restarting daemon 2022-08-01T19:33:44.927 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2022-08-01T19:33:44.930 INFO:tasks.ceph.mon.c:Started 2022-08-01T19:33:44.930 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2022-08-01T19:33:44.930 INFO:tasks.ceph.mgr.y:Restarting daemon 2022-08-01T19:33:44.931 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2022-08-01T19:33:44.933 INFO:tasks.ceph.mgr.y:Started 2022-08-01T19:33:44.934 INFO:tasks.ceph.mgr.x:Restarting daemon 2022-08-01T19:33:44.934 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2022-08-01T19:33:44.937 INFO:tasks.ceph.mgr.x:Started 2022-08-01T19:33:44.937 INFO:tasks.ceph:Setting crush tunables to default 2022-08-01T19:33:44.938 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph osd crush tunables default 2022-08-01T19:33:45.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.040+0000 7f70d6d04700 1 Processor -- start 2022-08-01T19:33:45.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.040+0000 7f70d6d04700 1 -- start start 2022-08-01T19:33:45.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.040+0000 7f70d6d04700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f70d00b52d0 0x7f70d00b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:45.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.040+0000 7f70d6d04700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f70d013d540 0x7f70d013d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:45.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.040+0000 7f70d6d04700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70d00b56b0 con 0x7f70d00b15d0 2022-08-01T19:33:45.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.040+0000 7f70d6d04700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d00ae190 con 0x7f70d00b52d0 2022-08-01T19:33:45.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.040+0000 7f70d6d04700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d00b1dd0 con 0x7f70d013d540 2022-08-01T19:33:45.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.044+0000 7f70cffff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f70d00b15d0 0x7f70d00af5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56110/0 (socket says 172.21.15.17:56110) 2022-08-01T19:33:45.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.044+0000 7f70cffff700 1 -- 172.21.15.17:0/3533222795 learned_addr learned my addr 172.21.15.17:0/3533222795 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:33:45.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.044+0000 7f70d4aa0700 1 --2- 172.21.15.17:0/3533222795 >> v2:172.21.15.17:3300/0 conn(0x7f70d00b52d0 0x7f70d00b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:45.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.044+0000 7f70d52a1700 1 --2- 172.21.15.17:0/3533222795 >> v2:172.21.15.77:3300/0 conn(0x7f70d013d540 0x7f70d013d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:45.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.044+0000 7f70d4aa0700 1 -- 172.21.15.17:0/3533222795 >> v2:172.21.15.17:3300/0 conn(0x7f70d00b52d0 msgr2=0x7f70d00b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2022-08-01T19:33:45.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.044+0000 7f70d4aa0700 1 -- 172.21.15.17:0/3533222795 >> v2:172.21.15.17:3300/0 conn(0x7f70d00b52d0 msgr2=0x7f70d00b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-08-01T19:33:45.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.044+0000 7f70d4aa0700 1 --2- 172.21.15.17:0/3533222795 >> v2:172.21.15.17:3300/0 conn(0x7f70d00b52d0 0x7f70d00b0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-08-01T19:33:45.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.044+0000 7f70d52a1700 1 -- 172.21.15.17:0/3533222795 >> v2:172.21.15.77:3300/0 conn(0x7f70d013d540 msgr2=0x7f70d013d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-08-01T19:33:45.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.044+0000 7f70d52a1700 1 -- 172.21.15.17:0/3533222795 >> v2:172.21.15.77:3300/0 conn(0x7f70d013d540 msgr2=0x7f70d013d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-08-01T19:33:45.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.044+0000 7f70d52a1700 1 --2- 172.21.15.17:0/3533222795 >> v2:172.21.15.77:3300/0 conn(0x7f70d013d540 0x7f70d013d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-08-01T19:33:45.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.044+0000 7f70d52a1700 1 --2- 172.21.15.17:0/3533222795 >> v2:172.21.15.77:3300/0 conn(0x7f70d013d540 0x7f70d013d940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-08-01T19:33:45.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.044+0000 7f70d4aa0700 1 --2- 172.21.15.17:0/3533222795 >> v2:172.21.15.17:3300/0 conn(0x7f70d00b52d0 0x7f70d00b0ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-08-01T19:33:45.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.248+0000 7f70d4aa0700 1 --2- 172.21.15.17:0/3533222795 >> v2:172.21.15.17:3300/0 conn(0x7f70d00b52d0 0x7f70d00b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:45.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.248+0000 7f70d52a1700 1 --2- 172.21.15.17:0/3533222795 >> v2:172.21.15.77:3300/0 conn(0x7f70d013d540 0x7f70d013d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:45.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.248+0000 7f70d4aa0700 1 -- 172.21.15.17:0/3533222795 >> v2:172.21.15.17:3300/0 conn(0x7f70d00b52d0 msgr2=0x7f70d00b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2022-08-01T19:33:45.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.248+0000 7f70d4aa0700 1 -- 172.21.15.17:0/3533222795 >> v2:172.21.15.17:3300/0 conn(0x7f70d00b52d0 msgr2=0x7f70d00b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-08-01T19:33:45.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.248+0000 7f70d4aa0700 1 --2- 172.21.15.17:0/3533222795 >> v2:172.21.15.17:3300/0 conn(0x7f70d00b52d0 0x7f70d00b0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-08-01T19:33:45.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.248+0000 7f70d4aa0700 1 --2- 172.21.15.17:0/3533222795 >> v2:172.21.15.17:3300/0 conn(0x7f70d00b52d0 0x7f70d00b0ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2022-08-01T19:33:45.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.248+0000 7f70d52a1700 1 -- 172.21.15.17:0/3533222795 >> v2:172.21.15.77:3300/0 conn(0x7f70d013d540 msgr2=0x7f70d013d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-08-01T19:33:45.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.248+0000 7f70d52a1700 1 -- 172.21.15.17:0/3533222795 >> v2:172.21.15.77:3300/0 conn(0x7f70d013d540 msgr2=0x7f70d013d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-08-01T19:33:45.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.248+0000 7f70d52a1700 1 --2- 172.21.15.17:0/3533222795 >> v2:172.21.15.77:3300/0 conn(0x7f70d013d540 0x7f70d013d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-08-01T19:33:45.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.248+0000 7f70d52a1700 1 --2- 172.21.15.17:0/3533222795 >> v2:172.21.15.77:3300/0 conn(0x7f70d013d540 0x7f70d013d940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2022-08-01T19:33:45.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.652+0000 7f70d4aa0700 1 --2- 172.21.15.17:0/3533222795 >> v2:172.21.15.17:3300/0 conn(0x7f70d00b52d0 0x7f70d00b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:45.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.652+0000 7f70d4aa0700 1 -- 172.21.15.17:0/3533222795 >> v2:172.21.15.17:3300/0 conn(0x7f70d00b52d0 msgr2=0x7f70d00b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-08-01T19:33:45.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.652+0000 7f70d52a1700 1 --2- 172.21.15.17:0/3533222795 >> v2:172.21.15.77:3300/0 conn(0x7f70d013d540 0x7f70d013d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:45.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.652+0000 7f70d4aa0700 1 -- 172.21.15.17:0/3533222795 >> v2:172.21.15.17:3300/0 conn(0x7f70d00b52d0 msgr2=0x7f70d00b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-08-01T19:33:45.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.652+0000 7f70d4aa0700 1 --2- 172.21.15.17:0/3533222795 >> v2:172.21.15.17:3300/0 conn(0x7f70d00b52d0 0x7f70d00b0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-08-01T19:33:45.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.652+0000 7f70d4aa0700 1 --2- 172.21.15.17:0/3533222795 >> v2:172.21.15.17:3300/0 conn(0x7f70d00b52d0 0x7f70d00b0ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2022-08-01T19:33:45.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.652+0000 7f70d52a1700 1 -- 172.21.15.17:0/3533222795 >> v2:172.21.15.77:3300/0 conn(0x7f70d013d540 msgr2=0x7f70d013d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2022-08-01T19:33:45.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.652+0000 7f70d52a1700 1 -- 172.21.15.17:0/3533222795 >> v2:172.21.15.77:3300/0 conn(0x7f70d013d540 msgr2=0x7f70d013d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-08-01T19:33:45.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.652+0000 7f70d52a1700 1 --2- 172.21.15.17:0/3533222795 >> v2:172.21.15.77:3300/0 conn(0x7f70d013d540 0x7f70d013d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-08-01T19:33:45.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:45.652+0000 7f70d52a1700 1 --2- 172.21.15.17:0/3533222795 >> v2:172.21.15.77:3300/0 conn(0x7f70d013d540 0x7f70d013d940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2022-08-01T19:33:46.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:46.456+0000 7f70d4aa0700 1 --2- 172.21.15.17:0/3533222795 >> v2:172.21.15.17:3300/0 conn(0x7f70d00b52d0 0x7f70d00b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:46.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:46.456+0000 7f70d4aa0700 1 -- 172.21.15.17:0/3533222795 >> v2:172.21.15.17:3300/0 conn(0x7f70d00b52d0 msgr2=0x7f70d00b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2022-08-01T19:33:46.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:46.456+0000 7f70d52a1700 1 --2- 172.21.15.17:0/3533222795 >> v2:172.21.15.77:3300/0 conn(0x7f70d013d540 0x7f70d013d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:46.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:46.456+0000 7f70d4aa0700 1 -- 172.21.15.17:0/3533222795 >> v2:172.21.15.17:3300/0 conn(0x7f70d00b52d0 msgr2=0x7f70d00b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-08-01T19:33:46.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:46.456+0000 7f70d4aa0700 1 --2- 172.21.15.17:0/3533222795 >> v2:172.21.15.17:3300/0 conn(0x7f70d00b52d0 0x7f70d00b0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-08-01T19:33:46.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:46.456+0000 7f70d4aa0700 1 --2- 172.21.15.17:0/3533222795 >> v2:172.21.15.17:3300/0 conn(0x7f70d00b52d0 0x7f70d00b0ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2022-08-01T19:33:46.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:46.456+0000 7f70d52a1700 1 -- 172.21.15.17:0/3533222795 >> v2:172.21.15.77:3300/0 conn(0x7f70d013d540 msgr2=0x7f70d013d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-08-01T19:33:46.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:46.456+0000 7f70d52a1700 1 -- 172.21.15.17:0/3533222795 >> v2:172.21.15.77:3300/0 conn(0x7f70d013d540 msgr2=0x7f70d013d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-08-01T19:33:46.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:46.456+0000 7f70d52a1700 1 --2- 172.21.15.17:0/3533222795 >> v2:172.21.15.77:3300/0 conn(0x7f70d013d540 0x7f70d013d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-08-01T19:33:46.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:46.456+0000 7f70d52a1700 1 --2- 172.21.15.17:0/3533222795 >> v2:172.21.15.77:3300/0 conn(0x7f70d013d540 0x7f70d013d940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2022-08-01T19:33:46.717 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:33:46.720 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:33:46.724 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:33:46.727 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:33:48.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:48.044+0000 7f70ce7fc700 1 -- 172.21.15.17:0/3533222795 >> v2:172.21.15.77:3300/0 conn(0x7f70d013d540 msgr2=0x7f70d013d940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-08-01T19:33:48.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:48.044+0000 7f70ce7fc700 1 --2- 172.21.15.17:0/3533222795 >> v2:172.21.15.77:3300/0 conn(0x7f70d013d540 0x7f70d013d940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:48.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:48.044+0000 7f70ce7fc700 1 -- 172.21.15.17:0/3533222795 >> v2:172.21.15.17:3300/0 conn(0x7f70d00b52d0 msgr2=0x7f70d00b0ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-08-01T19:33:48.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:48.044+0000 7f70ce7fc700 1 --2- 172.21.15.17:0/3533222795 >> v2:172.21.15.17:3300/0 conn(0x7f70d00b52d0 0x7f70d00b0ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:48.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:48.044+0000 7f70ce7fc700 1 -- 172.21.15.17:0/3533222795 >> v1:172.21.15.17:6789/0 conn(0x7f70d00b15d0 legacy=0x7f70d00af5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:48.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:48.044+0000 7f70ce7fc700 1 --2- 172.21.15.17:0/3533222795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70b8003380 0x7f70b80057d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:48.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:48.044+0000 7f70ce7fc700 1 -- 172.21.15.17:0/3533222795 --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70d00b2030 con 0x7f70b8005e10 2022-08-01T19:33:48.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:48.044+0000 7f70ce7fc700 1 -- 172.21.15.17:0/3533222795 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d00ae190 con 0x7f70b8003380 2022-08-01T19:33:48.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:48.044+0000 7f70d4aa0700 1 --2- 172.21.15.17:0/3533222795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70b8003380 0x7f70b80057d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:48.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:48.044+0000 7f70d4aa0700 1 -- 172.21.15.17:0/3533222795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70b8003380 msgr2=0x7f70b80057d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-08-01T19:33:48.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:48.044+0000 7f70d4aa0700 1 -- 172.21.15.17:0/3533222795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70b8003380 msgr2=0x7f70b80057d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-08-01T19:33:48.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:48.044+0000 7f70d4aa0700 1 --2- 172.21.15.17:0/3533222795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70b8003380 0x7f70b80057d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-08-01T19:33:48.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:48.044+0000 7f70d4aa0700 1 --2- 172.21.15.17:0/3533222795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70b8003380 0x7f70b80057d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-08-01T19:33:48.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:48.248+0000 7f70d4aa0700 1 --2- 172.21.15.17:0/3533222795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70b8003380 0x7f70b80057d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:48.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:48.248+0000 7f70d4aa0700 1 -- 172.21.15.17:0/3533222795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70b8003380 msgr2=0x7f70b80057d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-08-01T19:33:48.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:48.248+0000 7f70d4aa0700 1 -- 172.21.15.17:0/3533222795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70b8003380 msgr2=0x7f70b80057d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-08-01T19:33:48.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:48.248+0000 7f70d4aa0700 1 --2- 172.21.15.17:0/3533222795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70b8003380 0x7f70b80057d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-08-01T19:33:48.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:48.248+0000 7f70d4aa0700 1 --2- 172.21.15.17:0/3533222795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70b8003380 0x7f70b80057d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2022-08-01T19:33:48.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:48.652+0000 7f70d4aa0700 1 --2- 172.21.15.17:0/3533222795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70b8003380 0x7f70b80057d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:48.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:48.652+0000 7f70d4aa0700 1 -- 172.21.15.17:0/3533222795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70b8003380 msgr2=0x7f70b80057d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-08-01T19:33:48.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:48.652+0000 7f70d4aa0700 1 -- 172.21.15.17:0/3533222795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70b8003380 msgr2=0x7f70b80057d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-08-01T19:33:48.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:48.652+0000 7f70d4aa0700 1 --2- 172.21.15.17:0/3533222795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70b8003380 0x7f70b80057d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-08-01T19:33:48.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:48.652+0000 7f70d4aa0700 1 --2- 172.21.15.17:0/3533222795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70b8003380 0x7f70b80057d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2022-08-01T19:33:49.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:49.456+0000 7f70d4aa0700 1 --2- 172.21.15.17:0/3533222795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70b8003380 0x7f70b80057d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:49.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:49.456+0000 7f70d4aa0700 1 -- 172.21.15.17:0/3533222795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70b8003380 msgr2=0x7f70b80057d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-08-01T19:33:49.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:49.456+0000 7f70d4aa0700 1 -- 172.21.15.17:0/3533222795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70b8003380 msgr2=0x7f70b80057d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-08-01T19:33:49.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:49.456+0000 7f70d4aa0700 1 --2- 172.21.15.17:0/3533222795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70b8003380 0x7f70b80057d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-08-01T19:33:49.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:49.456+0000 7f70d4aa0700 1 --2- 172.21.15.17:0/3533222795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70b8003380 0x7f70b80057d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2022-08-01T19:33:50.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.128+0000 7f70cf7fe700 1 -- 172.21.15.17:0/3533222795 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 145267503 0 0) 0x7f70d00b2030 con 0x7f70b8005e10 2022-08-01T19:33:50.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.128+0000 7f70cf7fe700 1 -- 172.21.15.17:0/3533222795 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70b4003260 con 0x7f70b8005e10 2022-08-01T19:33:50.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.128+0000 7f70cf7fe700 1 -- 172.21.15.17:0/3533222795 <== mon.1 v1:172.21.15.77:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 581745009 0 0) 0x7f70b4003260 con 0x7f70b8005e10 2022-08-01T19:33:50.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.128+0000 7f70cf7fe700 1 -- 172.21.15.17:0/3533222795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70b8003380 msgr2=0x7f70b80057d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-08-01T19:33:50.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.128+0000 7f70cf7fe700 1 --2- 172.21.15.17:0/3533222795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70b8003380 0x7f70b80057d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:50.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.128+0000 7f70cf7fe700 1 -- 172.21.15.17:0/3533222795 --> v1:172.21.15.77:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b800aca0 con 0x7f70b8005e10 2022-08-01T19:33:50.134 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.128+0000 7f70cf7fe700 1 -- 172.21.15.17:0/3533222795 <== mon.1 v1:172.21.15.77:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7f70bc0016e0 con 0x7f70b8005e10 2022-08-01T19:33:50.134 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.128+0000 7f70cf7fe700 1 -- 172.21.15.17:0/3533222795 >> v1:172.21.15.77:6789/0 conn(0x7f70b8005e10 legacy=0x7f70b80082b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:50.134 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.128+0000 7f70cf7fe700 1 --2- 172.21.15.17:0/3533222795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b4003750 0x7f70b4003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:50.134 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.128+0000 7f70cf7fe700 1 --2- 172.21.15.17:0/3533222795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70b4004500 0x7f70b4004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:50.135 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.128+0000 7f70d52a1700 1 -- 172.21.15.17:0/3533222795 reap_dead start 2022-08-01T19:33:50.135 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.128+0000 7f70cf7fe700 1 --2- 172.21.15.17:0/3533222795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70b4004e70 0x7f70b400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:50.135 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.128+0000 7f70cf7fe700 1 -- 172.21.15.17:0/3533222795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d00ae190 con 0x7f70b4003750 2022-08-01T19:33:50.135 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.128+0000 7f70cf7fe700 1 -- 172.21.15.17:0/3533222795 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b400bdd0 con 0x7f70b4004500 2022-08-01T19:33:50.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.128+0000 7f70cf7fe700 1 -- 172.21.15.17:0/3533222795 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b400bf90 con 0x7f70b4004e70 2022-08-01T19:33:50.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.128+0000 7f70cffff700 1 --2- 172.21.15.17:0/3533222795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70b4004500 0x7f70b4004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:50.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.128+0000 7f70d4aa0700 1 --2- 172.21.15.17:0/3533222795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b4003750 0x7f70b4003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:50.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.132+0000 7f70d52a1700 1 --2- 172.21.15.17:0/3533222795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70b4004e70 0x7f70b400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:50.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.132+0000 7f70cffff700 1 -- 172.21.15.17:0/3533222795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70b4004e70 msgr2=0x7f70b400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:33:50.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.132+0000 7f70cffff700 1 --2- 172.21.15.17:0/3533222795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70b4004e70 0x7f70b400b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:50.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.132+0000 7f70cffff700 1 -- 172.21.15.17:0/3533222795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b4003750 msgr2=0x7f70b4003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:33:50.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.132+0000 7f70cffff700 1 --2- 172.21.15.17:0/3533222795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b4003750 0x7f70b4003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:50.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.132+0000 7f70cffff700 1 -- 172.21.15.17:0/3533222795 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b4003260 con 0x7f70b4004500 2022-08-01T19:33:50.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.132+0000 7f70d4aa0700 1 --2- 172.21.15.17:0/3533222795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b4003750 0x7f70b4003b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:33:50.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.132+0000 7f70cffff700 1 --2- 172.21.15.17:0/3533222795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70b4004500 0x7f70b4004980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f70bc0013f0 tx=0x7f70bc001e30).ready entity=mon.1 client_cookie=7a64bfb4ef3acf1c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:33:50.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.132+0000 7f70cf7fe700 1 -- 172.21.15.17:0/3533222795 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f70bc0070d0 con 0x7f70b4004500 2022-08-01T19:33:50.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.132+0000 7f70cf7fe700 1 -- 172.21.15.17:0/3533222795 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70bc007230 con 0x7f70b4004500 2022-08-01T19:33:50.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.132+0000 7f70cf7fe700 1 -- 172.21.15.17:0/3533222795 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f70bc00c8f0 con 0x7f70b4004500 2022-08-01T19:33:50.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.132+0000 7f70d6d04700 1 -- 172.21.15.17:0/3533222795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70b4004500 msgr2=0x7f70b4004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:50.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.132+0000 7f70d6d04700 1 --2- 172.21.15.17:0/3533222795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70b4004500 0x7f70b4004980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f70bc0013f0 tx=0x7f70bc001e30).stop 2022-08-01T19:33:50.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.132+0000 7f70d6d04700 1 -- 172.21.15.17:0/3533222795 shutdown_connections 2022-08-01T19:33:50.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.132+0000 7f70d6d04700 1 --2- 172.21.15.17:0/3533222795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70b4004e70 0x7f70b400b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:50.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.132+0000 7f70d6d04700 1 --2- 172.21.15.17:0/3533222795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b4003750 0x7f70b4003b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:50.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.132+0000 7f70d6d04700 1 --2- 172.21.15.17:0/3533222795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70b4004500 0x7f70b4004980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:50.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.132+0000 7f70d6d04700 1 -- 172.21.15.17:0/3533222795 >> 172.21.15.17:0/3533222795 conn(0x7f70d006a2a0 msgr2=0x7f70d005ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:33:50.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.132+0000 7f70d6d04700 1 -- 172.21.15.17:0/3533222795 shutdown_connections 2022-08-01T19:33:50.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.132+0000 7f70d6d04700 1 -- 172.21.15.17:0/3533222795 wait complete. 2022-08-01T19:33:50.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.136+0000 7f70d6d04700 1 Processor -- start 2022-08-01T19:33:50.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.136+0000 7f70d6d04700 1 -- start start 2022-08-01T19:33:50.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.140+0000 7f70d6d04700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70b4003750 0x7f70d0146290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:50.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.140+0000 7f70d6d04700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b4004500 0x7f70d013f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:50.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.140+0000 7f70d6d04700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70b4004e70 0x7f70d01381d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:50.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.140+0000 7f70d6d04700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d013dc40 con 0x7f70b4004500 2022-08-01T19:33:50.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.140+0000 7f70d6d04700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d013dee0 con 0x7f70b4003750 2022-08-01T19:33:50.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.140+0000 7f70d6d04700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d013f190 con 0x7f70b4004e70 2022-08-01T19:33:50.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.140+0000 7f70cffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b4004500 0x7f70d013f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:50.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.140+0000 7f70d52a1700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70b4004e70 0x7f70d01381d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:50.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.140+0000 7f70d52a1700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70b4004e70 0x7f70d01381d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:60086/0 (socket says 172.21.15.17:60086) 2022-08-01T19:33:50.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.140+0000 7f70d52a1700 1 -- 172.21.15.17:0/672432290 learned_addr learned my addr 172.21.15.17:0/672432290 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:33:50.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.140+0000 7f70d4aa0700 1 --2- 172.21.15.17:0/672432290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70b4003750 0x7f70d0146290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:50.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.140+0000 7f70d52a1700 1 -- 172.21.15.17:0/672432290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70b4003750 msgr2=0x7f70d0146290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:50.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.140+0000 7f70d52a1700 1 --2- 172.21.15.17:0/672432290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70b4003750 0x7f70d0146290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:50.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.140+0000 7f70d52a1700 1 -- 172.21.15.17:0/672432290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b4004500 msgr2=0x7f70d013f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:50.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.140+0000 7f70d52a1700 1 --2- 172.21.15.17:0/672432290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b4004500 0x7f70d013f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:50.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.140+0000 7f70d52a1700 1 -- 172.21.15.17:0/672432290 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d013e180 con 0x7f70b4004e70 2022-08-01T19:33:50.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.140+0000 7f70cffff700 1 --2- 172.21.15.17:0/672432290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b4004500 0x7f70d013f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:33:50.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.140+0000 7f70d52a1700 1 --2- 172.21.15.17:0/672432290 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70b4004e70 0x7f70d01381d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f70c800a000 tx=0x7f70c8008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:33:50.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.144+0000 7f70cdffb700 1 -- 172.21.15.17:0/672432290 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f70c8018850 con 0x7f70b4004e70 2022-08-01T19:33:50.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.144+0000 7f70cdffb700 1 -- 172.21.15.17:0/672432290 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c800c040 con 0x7f70b4004e70 2022-08-01T19:33:50.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.144+0000 7f70d6d04700 1 -- 172.21.15.17:0/672432290 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d0065e70 con 0x7f70b4004e70 2022-08-01T19:33:50.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.144+0000 7f70cdffb700 1 -- 172.21.15.17:0/672432290 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f70c8018ba0 con 0x7f70b4004e70 2022-08-01T19:33:50.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.144+0000 7f70d6d04700 1 -- 172.21.15.17:0/672432290 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d013c380 con 0x7f70b4004e70 2022-08-01T19:33:50.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.144+0000 7f70cdffb700 1 -- 172.21.15.17:0/672432290 <== mon.2 v2:172.21.15.77:3301/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f70c8012040 con 0x7f70b4004e70 2022-08-01T19:33:50.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.144+0000 7f70cdffb700 1 -- 172.21.15.17:0/672432290 <== mon.2 v2:172.21.15.77:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f70c8006390 con 0x7f70b4004e70 2022-08-01T19:33:50.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.144+0000 7f70d6d04700 1 -- 172.21.15.17:0/672432290 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70d00b22d0 con 0x7f70b4004e70 2022-08-01T19:33:50.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.152+0000 7f70cdffb700 1 -- 172.21.15.17:0/672432290 <== mon.2 v2:172.21.15.77:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f70c800f040 con 0x7f70b4004e70 2022-08-01T19:33:50.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.216+0000 7f70d6d04700 1 -- 172.21.15.17:0/672432290 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f70d00574e0 con 0x7f70b4004e70 2022-08-01T19:33:50.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.224+0000 7f70cdffb700 1 -- 172.21.15.17:0/672432290 <== mon.2 v2:172.21.15.77:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f70c8005d40 con 0x7f70b4004e70 2022-08-01T19:33:50.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.228+0000 7f70d6d04700 1 -- 172.21.15.17:0/672432290 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70b4004e70 msgr2=0x7f70d01381d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:50.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.228+0000 7f70d6d04700 1 --2- 172.21.15.17:0/672432290 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70b4004e70 0x7f70d01381d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f70c800a000 tx=0x7f70c8008040).stop 2022-08-01T19:33:50.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.228+0000 7f70d6d04700 1 -- 172.21.15.17:0/672432290 shutdown_connections 2022-08-01T19:33:50.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.228+0000 7f70d6d04700 1 --2- 172.21.15.17:0/672432290 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70b4004e70 0x7f70d01381d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:50.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.228+0000 7f70d6d04700 1 --2- 172.21.15.17:0/672432290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b4004500 0x7f70d013f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:50.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.228+0000 7f70d6d04700 1 --2- 172.21.15.17:0/672432290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70b4003750 0x7f70d0146290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:50.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.228+0000 7f70d6d04700 1 -- 172.21.15.17:0/672432290 >> 172.21.15.17:0/672432290 conn(0x7f70d006a2a0 msgr2=0x7f70d00b4780 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:33:50.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.228+0000 7f70d6d04700 1 -- 172.21.15.17:0/672432290 shutdown_connections 2022-08-01T19:33:50.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:33:50.228+0000 7f70d6d04700 1 -- 172.21.15.17:0/672432290 wait complete. 2022-08-01T19:33:50.233 INFO:teuthology.orchestra.run.smithi017.stderr:adjusted tunables profile to default 2022-08-01T19:33:50.242 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2022-08-01T19:33:50.243 DEBUG:teuthology.orchestra.run.smithi017:> mktemp 2022-08-01T19:33:50.250 INFO:teuthology.orchestra.run.smithi017.stdout:/tmp/tmp.0YYWJ74Wad 2022-08-01T19:33:50.251 DEBUG:teuthology.orchestra.run.smithi017:> sudo cp /var/lib/ceph/osd/ceph-0/fsid /tmp/tmp.0YYWJ74Wad 2022-08-01T19:33:50.267 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0666 /tmp/tmp.0YYWJ74Wad 2022-08-01T19:33:50.331 DEBUG:teuthology.orchestra.remote:smithi017:/tmp/tmp.0YYWJ74Wad is 37B 2022-08-01T19:33:50.381 DEBUG:teuthology.orchestra.run.smithi017:> rm -fr /tmp/tmp.0YYWJ74Wad 2022-08-01T19:33:50.388 DEBUG:teuthology.orchestra.run.smithi017:> mktemp 2022-08-01T19:33:50.394 INFO:teuthology.orchestra.run.smithi017.stdout:/tmp/tmp.ivGLHvM0N6 2022-08-01T19:33:50.395 DEBUG:teuthology.orchestra.run.smithi017:> sudo cp /var/lib/ceph/osd/ceph-1/fsid /tmp/tmp.ivGLHvM0N6 2022-08-01T19:33:50.450 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0666 /tmp/tmp.ivGLHvM0N6 2022-08-01T19:33:50.468 DEBUG:teuthology.orchestra.remote:smithi017:/tmp/tmp.ivGLHvM0N6 is 37B 2022-08-01T19:33:50.482 DEBUG:teuthology.orchestra.run.smithi017:> rm -fr /tmp/tmp.ivGLHvM0N6 2022-08-01T19:33:50.488 DEBUG:teuthology.orchestra.run.smithi017:> mktemp 2022-08-01T19:33:50.533 INFO:teuthology.orchestra.run.smithi017.stdout:/tmp/tmp.Cwnikc4w1G 2022-08-01T19:33:50.534 DEBUG:teuthology.orchestra.run.smithi017:> sudo cp /var/lib/ceph/osd/ceph-2/fsid /tmp/tmp.Cwnikc4w1G 2022-08-01T19:33:50.550 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0666 /tmp/tmp.Cwnikc4w1G 2022-08-01T19:33:50.575 DEBUG:teuthology.orchestra.remote:smithi017:/tmp/tmp.Cwnikc4w1G is 37B 2022-08-01T19:33:50.625 DEBUG:teuthology.orchestra.run.smithi017:> rm -fr /tmp/tmp.Cwnikc4w1G 2022-08-01T19:33:50.632 DEBUG:teuthology.orchestra.run.smithi017:> mktemp 2022-08-01T19:33:50.639 INFO:teuthology.orchestra.run.smithi017.stdout:/tmp/tmp.AEu6NIvHJY 2022-08-01T19:33:50.640 DEBUG:teuthology.orchestra.run.smithi017:> sudo cp /var/lib/ceph/osd/ceph-3/fsid /tmp/tmp.AEu6NIvHJY 2022-08-01T19:33:50.697 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0666 /tmp/tmp.AEu6NIvHJY 2022-08-01T19:33:50.722 DEBUG:teuthology.orchestra.remote:smithi017:/tmp/tmp.AEu6NIvHJY is 37B 2022-08-01T19:33:50.773 DEBUG:teuthology.orchestra.run.smithi017:> rm -fr /tmp/tmp.AEu6NIvHJY 2022-08-01T19:33:50.781 DEBUG:teuthology.orchestra.run.smithi077:> mktemp 2022-08-01T19:33:50.789 INFO:teuthology.orchestra.run.smithi077.stdout:/tmp/tmp.xxPtWwED3Z 2022-08-01T19:33:50.790 DEBUG:teuthology.orchestra.run.smithi077:> sudo cp /var/lib/ceph/osd/ceph-4/fsid /tmp/tmp.xxPtWwED3Z 2022-08-01T19:33:50.806 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 0666 /tmp/tmp.xxPtWwED3Z 2022-08-01T19:33:50.871 DEBUG:teuthology.orchestra.remote:smithi077:/tmp/tmp.xxPtWwED3Z is 37B 2022-08-01T19:33:50.885 DEBUG:teuthology.orchestra.run.smithi077:> rm -fr /tmp/tmp.xxPtWwED3Z 2022-08-01T19:33:50.891 DEBUG:teuthology.orchestra.run.smithi077:> mktemp 2022-08-01T19:33:50.938 INFO:teuthology.orchestra.run.smithi077.stdout:/tmp/tmp.5LCOD3kESi 2022-08-01T19:33:50.939 DEBUG:teuthology.orchestra.run.smithi077:> sudo cp /var/lib/ceph/osd/ceph-5/fsid /tmp/tmp.5LCOD3kESi 2022-08-01T19:33:50.956 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 0666 /tmp/tmp.5LCOD3kESi 2022-08-01T19:33:51.018 DEBUG:teuthology.orchestra.remote:smithi077:/tmp/tmp.5LCOD3kESi is 37B 2022-08-01T19:33:51.032 DEBUG:teuthology.orchestra.run.smithi077:> rm -fr /tmp/tmp.5LCOD3kESi 2022-08-01T19:33:51.038 DEBUG:teuthology.orchestra.run.smithi077:> mktemp 2022-08-01T19:33:51.086 INFO:teuthology.orchestra.run.smithi077.stdout:/tmp/tmp.ESKfCyg7yR 2022-08-01T19:33:51.086 DEBUG:teuthology.orchestra.run.smithi077:> sudo cp /var/lib/ceph/osd/ceph-6/fsid /tmp/tmp.ESKfCyg7yR 2022-08-01T19:33:51.101 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 0666 /tmp/tmp.ESKfCyg7yR 2022-08-01T19:33:51.171 DEBUG:teuthology.orchestra.remote:smithi077:/tmp/tmp.ESKfCyg7yR is 37B 2022-08-01T19:33:51.185 DEBUG:teuthology.orchestra.run.smithi077:> rm -fr /tmp/tmp.ESKfCyg7yR 2022-08-01T19:33:51.192 DEBUG:teuthology.orchestra.run.smithi077:> mktemp 2022-08-01T19:33:51.238 INFO:teuthology.orchestra.run.smithi077.stdout:/tmp/tmp.2AUBdvTHpJ 2022-08-01T19:33:51.239 DEBUG:teuthology.orchestra.run.smithi077:> sudo cp /var/lib/ceph/osd/ceph-7/fsid /tmp/tmp.2AUBdvTHpJ 2022-08-01T19:33:51.252 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 0666 /tmp/tmp.2AUBdvTHpJ 2022-08-01T19:33:51.318 DEBUG:teuthology.orchestra.remote:smithi077:/tmp/tmp.2AUBdvTHpJ is 37B 2022-08-01T19:33:51.333 DEBUG:teuthology.orchestra.run.smithi077:> rm -fr /tmp/tmp.2AUBdvTHpJ 2022-08-01T19:33:51.339 DEBUG:teuthology.orchestra.run.smithi077:> sudo ceph --cluster ceph osd new a962f14f-cf16-4fa8-84ff-c85325db9132 0 2022-08-01T19:33:51.482 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.480+0000 7f35f240d700 1 Processor -- start 2022-08-01T19:33:51.482 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.480+0000 7f35f240d700 1 -- start start 2022-08-01T19:33:51.485 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.480+0000 7f35f240d700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f35ec0b41f0 0x7f35ec061d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:51.486 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.480+0000 7f35f240d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f35ec142ff0 0x7f35ec143480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:51.486 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.480+0000 7f35f240d700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35ec0b60e0 con 0x7f35ec0b6420 2022-08-01T19:33:51.486 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.480+0000 7f35f240d700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35ec0b2100 con 0x7f35ec142ff0 2022-08-01T19:33:51.486 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.480+0000 7f35f240d700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35ec0b23a0 con 0x7f35ec0b41f0 2022-08-01T19:33:51.487 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.480+0000 7f35f0c0a700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f35ec0b41f0 0x7f35ec061d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:51.487 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.480+0000 7f35f140b700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7f35ec0b6420 0x7f35ec061600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.77:52116/0 (socket says 172.21.15.77:52116) 2022-08-01T19:33:51.487 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.480+0000 7f35f140b700 1 -- 172.21.15.77:0/4075574498 learned_addr learned my addr 172.21.15.77:0/4075574498 (peer_addr_for_me v1:172.21.15.77:0/0) 2022-08-01T19:33:51.487 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.480+0000 7f35f1c0c700 1 --2- 172.21.15.77:0/4075574498 >> v2:172.21.15.17:3300/0 conn(0x7f35ec142ff0 0x7f35ec143480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:51.488 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.480+0000 7f35f0c0a700 1 -- 172.21.15.77:0/4075574498 >> v2:172.21.15.17:3300/0 conn(0x7f35ec142ff0 msgr2=0x7f35ec143480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:33:51.488 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.480+0000 7f35f0c0a700 1 --2- 172.21.15.77:0/4075574498 >> v2:172.21.15.17:3300/0 conn(0x7f35ec142ff0 0x7f35ec143480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:51.488 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.480+0000 7f35f0c0a700 1 -- 172.21.15.77:0/4075574498 >> v1:172.21.15.77:6789/0 conn(0x7f35ec0b6420 legacy=0x7f35ec061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:51.488 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.480+0000 7f35f0c0a700 1 -- 172.21.15.77:0/4075574498 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35ec0b2640 con 0x7f35ec0b41f0 2022-08-01T19:33:51.488 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.480+0000 7f35f0c0a700 1 --2- 172.21.15.77:0/4075574498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35ec0b41f0 0x7f35ec061d10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f35dc002f70 tx=0x7f35dc006df0).ready entity=mon.1 client_cookie=4d65197d9b45ecc3 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:33:51.489 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.484+0000 7f35dbfff700 1 -- 172.21.15.77:0/4075574498 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35dc00d5c0 con 0x7f35ec0b41f0 2022-08-01T19:33:51.489 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.484+0000 7f35dbfff700 1 -- 172.21.15.77:0/4075574498 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35dc000e90 con 0x7f35ec0b41f0 2022-08-01T19:33:51.489 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.484+0000 7f35dbfff700 1 -- 172.21.15.77:0/4075574498 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35dc00d910 con 0x7f35ec0b41f0 2022-08-01T19:33:51.489 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.484+0000 7f35f240d700 1 -- 172.21.15.77:0/4075574498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35ec0b41f0 msgr2=0x7f35ec061d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:51.490 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.484+0000 7f35f240d700 1 --2- 172.21.15.77:0/4075574498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35ec0b41f0 0x7f35ec061d10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f35dc002f70 tx=0x7f35dc006df0).stop 2022-08-01T19:33:51.490 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.484+0000 7f35f240d700 1 -- 172.21.15.77:0/4075574498 shutdown_connections 2022-08-01T19:33:51.490 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.484+0000 7f35f240d700 1 --2- 172.21.15.77:0/4075574498 >> v2:172.21.15.17:3300/0 conn(0x7f35ec142ff0 0x7f35ec143480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:51.490 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.484+0000 7f35f240d700 1 --2- 172.21.15.77:0/4075574498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35ec0b41f0 0x7f35ec061d10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:51.491 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.484+0000 7f35f240d700 1 -- 172.21.15.77:0/4075574498 >> 172.21.15.77:0/4075574498 conn(0x7f35ec05c620 msgr2=0x7f35ec05ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:33:51.491 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.484+0000 7f35f240d700 1 -- 172.21.15.77:0/4075574498 shutdown_connections 2022-08-01T19:33:51.491 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.484+0000 7f35f240d700 1 -- 172.21.15.77:0/4075574498 wait complete. 2022-08-01T19:33:51.494 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.484+0000 7f35f240d700 1 Processor -- start 2022-08-01T19:33:51.494 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.484+0000 7f35f240d700 1 -- start start 2022-08-01T19:33:51.494 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.484+0000 7f35f240d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35ec0b41f0 0x7f35ec0b7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:51.495 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.484+0000 7f35f240d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35ec0b6420 0x7f35ec0af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:51.495 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.484+0000 7f35f240d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35ec142ff0 0x7f35ec0afe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:51.495 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.484+0000 7f35f240d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec13e830 con 0x7f35ec0b6420 2022-08-01T19:33:51.495 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.484+0000 7f35f240d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec13d800 con 0x7f35ec142ff0 2022-08-01T19:33:51.495 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.484+0000 7f35f240d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec13e590 con 0x7f35ec0b41f0 2022-08-01T19:33:51.496 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.484+0000 7f35f1c0c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35ec142ff0 0x7f35ec0afe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:51.496 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.484+0000 7f35f1c0c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35ec142ff0 0x7f35ec0afe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49962/0 (socket says 172.21.15.77:49962) 2022-08-01T19:33:51.496 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.484+0000 7f35f140b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35ec0b41f0 0x7f35ec0b7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:51.497 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.484+0000 7f35f1c0c700 1 -- 172.21.15.77:0/3439121297 learned_addr learned my addr 172.21.15.77:0/3439121297 (peer_addr_for_me v2:172.21.15.77:0/0) 2022-08-01T19:33:51.497 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.484+0000 7f35f0c0a700 1 --2- 172.21.15.77:0/3439121297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35ec0b6420 0x7f35ec0af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:51.497 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.484+0000 7f35f1c0c700 1 -- 172.21.15.77:0/3439121297 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35ec0b41f0 msgr2=0x7f35ec0b7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:51.497 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.484+0000 7f35f1c0c700 1 --2- 172.21.15.77:0/3439121297 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35ec0b41f0 0x7f35ec0b7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:51.497 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.484+0000 7f35f1c0c700 1 -- 172.21.15.77:0/3439121297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35ec0b6420 msgr2=0x7f35ec0af950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:51.498 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.484+0000 7f35f1c0c700 1 --2- 172.21.15.77:0/3439121297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35ec0b6420 0x7f35ec0af950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:51.498 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.484+0000 7f35f1c0c700 1 -- 172.21.15.77:0/3439121297 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35dc00b180 con 0x7f35ec142ff0 2022-08-01T19:33:51.498 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.484+0000 7f35f1c0c700 1 --2- 172.21.15.77:0/3439121297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35ec142ff0 0x7f35ec0afe40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f35e400a000 tx=0x7f35e4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:33:51.498 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.484+0000 7f35da7fc700 1 -- 172.21.15.77:0/3439121297 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35e40127e0 con 0x7f35ec142ff0 2022-08-01T19:33:51.499 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.484+0000 7f35da7fc700 1 -- 172.21.15.77:0/3439121297 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e4012940 con 0x7f35ec142ff0 2022-08-01T19:33:51.499 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.484+0000 7f35f240d700 1 -- 172.21.15.77:0/3439121297 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35ec13da10 con 0x7f35ec142ff0 2022-08-01T19:33:51.499 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.488+0000 7f35da7fc700 1 -- 172.21.15.77:0/3439121297 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35e4012ca0 con 0x7f35ec142ff0 2022-08-01T19:33:51.499 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.488+0000 7f35f240d700 1 -- 172.21.15.77:0/3439121297 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35ec13e150 con 0x7f35ec142ff0 2022-08-01T19:33:51.499 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.488+0000 7f35da7fc700 1 -- 172.21.15.77:0/3439121297 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f35e4005a70 con 0x7f35ec142ff0 2022-08-01T19:33:51.500 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.488+0000 7f35da7fc700 1 -- 172.21.15.77:0/3439121297 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 754+0+0 (secure 0 0 0) 0x7f35e4006150 con 0x7f35ec142ff0 2022-08-01T19:33:51.500 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.488+0000 7f35f240d700 1 -- 172.21.15.77:0/3439121297 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35ec13deb0 con 0x7f35ec142ff0 2022-08-01T19:33:51.500 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.488+0000 7f35da7fc700 1 -- 172.21.15.77:0/3439121297 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f35ec13deb0 con 0x7f35ec142ff0 2022-08-01T19:33:51.557 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.556+0000 7f35f240d700 1 -- 172.21.15.77:0/3439121297 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a962f14f-cf16-4fa8-84ff-c85325db9132", "id": 0} v 0) v1 -- 0x7f35ec0b28e0 con 0x7f35ec142ff0 2022-08-01T19:33:51.566 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.564+0000 7f35da7fc700 1 -- 172.21.15.77:0/3439121297 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a962f14f-cf16-4fa8-84ff-c85325db9132", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f35e402b0b0 con 0x7f35ec142ff0 2022-08-01T19:33:51.566 INFO:teuthology.orchestra.run.smithi077.stdout:0 2022-08-01T19:33:51.567 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.564+0000 7f35f240d700 1 -- 172.21.15.77:0/3439121297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35ec142ff0 msgr2=0x7f35ec0afe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:51.567 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.564+0000 7f35f240d700 1 --2- 172.21.15.77:0/3439121297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35ec142ff0 0x7f35ec0afe40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f35e400a000 tx=0x7f35e4008040).stop 2022-08-01T19:33:51.568 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.564+0000 7f35f240d700 1 -- 172.21.15.77:0/3439121297 shutdown_connections 2022-08-01T19:33:51.568 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.564+0000 7f35f240d700 1 --2- 172.21.15.77:0/3439121297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35ec142ff0 0x7f35ec0afe40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:51.569 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.564+0000 7f35f240d700 1 --2- 172.21.15.77:0/3439121297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35ec0b6420 0x7f35ec0af950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:51.569 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.564+0000 7f35f240d700 1 --2- 172.21.15.77:0/3439121297 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35ec0b41f0 0x7f35ec0b7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:51.569 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.564+0000 7f35f240d700 1 -- 172.21.15.77:0/3439121297 >> 172.21.15.77:0/3439121297 conn(0x7f35ec05c620 msgr2=0x7f35ec1424a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:33:51.569 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.568+0000 7f35f240d700 1 -- 172.21.15.77:0/3439121297 shutdown_connections 2022-08-01T19:33:51.570 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.568+0000 7f35f240d700 1 -- 172.21.15.77:0/3439121297 wait complete. 2022-08-01T19:33:51.578 DEBUG:teuthology.orchestra.run.smithi077:> sudo ceph --cluster ceph osd new 805f7051-a527-4e98-8844-b6e406a63248 1 2022-08-01T19:33:51.680 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.680+0000 7f97ab59e700 1 Processor -- start 2022-08-01T19:33:51.681 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.680+0000 7f97ab59e700 1 -- start start 2022-08-01T19:33:51.692 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.680+0000 7f97ab59e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f97ac0a9320 0x7f97ac1466e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:51.692 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.680+0000 7f97ab59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f97ac0b0560 0x7f97ac140350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:51.692 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.680+0000 7f97ab59e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97ac0b12d0 con 0x7f97ac0b0d10 2022-08-01T19:33:51.693 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.680+0000 7f97ab59e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97ac0a9700 con 0x7f97ac0a9320 2022-08-01T19:33:51.693 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.680+0000 7f97ab59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac0b1500 con 0x7f97ac0b0560 2022-08-01T19:33:51.693 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.680+0000 7f97aad9d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f97ac0b0560 0x7f97ac140350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:51.693 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.680+0000 7f97a9d9b700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7f97ac0b0d10 0x7f97ac146bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.77:52130/0 (socket says 172.21.15.77:52130) 2022-08-01T19:33:51.694 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.680+0000 7f97a9d9b700 1 -- 172.21.15.77:0/2658066302 learned_addr learned my addr 172.21.15.77:0/2658066302 (peer_addr_for_me v1:172.21.15.77:0/0) 2022-08-01T19:33:51.694 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.680+0000 7f97aa59c700 1 --2- 172.21.15.77:0/2658066302 >> v2:172.21.15.17:3300/0 conn(0x7f97ac0a9320 0x7f97ac1466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:51.694 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.680+0000 7f97a959a700 1 -- 172.21.15.77:0/2658066302 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1266812670 0 0) 0x7f97ac0b12d0 con 0x7f97ac0b0d10 2022-08-01T19:33:51.694 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.680+0000 7f97aad9d700 1 -- 172.21.15.77:0/2658066302 >> v2:172.21.15.17:3300/0 conn(0x7f97ac0a9320 msgr2=0x7f97ac1466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:33:51.695 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.680+0000 7f97aad9d700 1 --2- 172.21.15.77:0/2658066302 >> v2:172.21.15.17:3300/0 conn(0x7f97ac0a9320 0x7f97ac1466e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:51.695 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.680+0000 7f97aad9d700 1 -- 172.21.15.77:0/2658066302 >> v1:172.21.15.77:6789/0 conn(0x7f97ac0b0d10 legacy=0x7f97ac146bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:51.695 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.680+0000 7f97aad9d700 1 -- 172.21.15.77:0/2658066302 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ac0b17a0 con 0x7f97ac0b0560 2022-08-01T19:33:51.695 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.680+0000 7f97aad9d700 1 --2- 172.21.15.77:0/2658066302 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f97ac0b0560 0x7f97ac140350 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f97a40032f0 tx=0x7f97a40091d0).ready entity=mon.2 client_cookie=f95bb6fa0328e5e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:33:51.695 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.680+0000 7f97a959a700 1 -- 172.21.15.77:0/2658066302 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97a4009ea0 con 0x7f97ac0b0560 2022-08-01T19:33:51.696 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.680+0000 7f97a959a700 1 -- 172.21.15.77:0/2658066302 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a400b8a0 con 0x7f97ac0b0560 2022-08-01T19:33:51.696 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.680+0000 7f97a959a700 1 -- 172.21.15.77:0/2658066302 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97a400ba10 con 0x7f97ac0b0560 2022-08-01T19:33:51.696 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.680+0000 7f97ab59e700 1 -- 172.21.15.77:0/2658066302 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f97ac0b0560 msgr2=0x7f97ac140350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:51.696 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.680+0000 7f97ab59e700 1 --2- 172.21.15.77:0/2658066302 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f97ac0b0560 0x7f97ac140350 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f97a40032f0 tx=0x7f97a40091d0).stop 2022-08-01T19:33:51.697 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.680+0000 7f97ab59e700 1 -- 172.21.15.77:0/2658066302 shutdown_connections 2022-08-01T19:33:51.697 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.680+0000 7f97ab59e700 1 --2- 172.21.15.77:0/2658066302 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f97ac0b0560 0x7f97ac140350 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:51.697 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.680+0000 7f97ab59e700 1 --2- 172.21.15.77:0/2658066302 >> v2:172.21.15.17:3300/0 conn(0x7f97ac0a9320 0x7f97ac1466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:51.697 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.680+0000 7f97ab59e700 1 -- 172.21.15.77:0/2658066302 >> 172.21.15.77:0/2658066302 conn(0x7f97ac0612d0 msgr2=0x7f97ac062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:33:51.698 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.680+0000 7f97ab59e700 1 -- 172.21.15.77:0/2658066302 shutdown_connections 2022-08-01T19:33:51.698 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.680+0000 7f97ab59e700 1 -- 172.21.15.77:0/2658066302 wait complete. 2022-08-01T19:33:51.698 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.684+0000 7f97ab59e700 1 Processor -- start 2022-08-01T19:33:51.698 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.684+0000 7f97ab59e700 1 -- start start 2022-08-01T19:33:51.698 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.684+0000 7f97ab59e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97ac0a9320 0x7f97ac155980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:51.699 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.684+0000 7f97ab59e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f97ac0b0560 0x7f97ac155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:51.699 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.684+0000 7f97ab59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f97ac0b0d10 0x7f97ac156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:51.699 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.684+0000 7f97ab59e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac156850 con 0x7f97ac0b0560 2022-08-01T19:33:51.699 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.684+0000 7f97ab59e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac145c70 con 0x7f97ac0a9320 2022-08-01T19:33:51.699 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.684+0000 7f97ab59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac152770 con 0x7f97ac0b0d10 2022-08-01T19:33:51.700 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.684+0000 7f97aa59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97ac0a9320 0x7f97ac155980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:51.700 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.684+0000 7f97aad9d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f97ac0b0d10 0x7f97ac156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:51.700 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.684+0000 7f97aa59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97ac0a9320 0x7f97ac155980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49970/0 (socket says 172.21.15.77:49970) 2022-08-01T19:33:51.700 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.684+0000 7f97aa59c700 1 -- 172.21.15.77:0/4209059331 learned_addr learned my addr 172.21.15.77:0/4209059331 (peer_addr_for_me v2:172.21.15.77:0/0) 2022-08-01T19:33:51.701 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.684+0000 7f97a9d9b700 1 --2- 172.21.15.77:0/4209059331 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f97ac0b0560 0x7f97ac155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:51.701 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.684+0000 7f97aa59c700 1 -- 172.21.15.77:0/4209059331 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f97ac0b0d10 msgr2=0x7f97ac156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:51.701 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.684+0000 7f97aa59c700 1 --2- 172.21.15.77:0/4209059331 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f97ac0b0d10 0x7f97ac156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:51.701 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.684+0000 7f97aa59c700 1 -- 172.21.15.77:0/4209059331 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f97ac0b0560 msgr2=0x7f97ac155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:51.702 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.684+0000 7f97aa59c700 1 --2- 172.21.15.77:0/4209059331 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f97ac0b0560 0x7f97ac155e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:51.702 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.684+0000 7f97aa59c700 1 -- 172.21.15.77:0/4209059331 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a4009470 con 0x7f97ac0a9320 2022-08-01T19:33:51.702 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.684+0000 7f97aad9d700 1 --2- 172.21.15.77:0/4209059331 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f97ac0b0d10 0x7f97ac156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:33:51.702 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.684+0000 7f97aa59c700 1 --2- 172.21.15.77:0/4209059331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97ac0a9320 0x7f97ac155980 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f979c0030b0 tx=0x7f979c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:33:51.703 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.684+0000 7f979b7fe700 1 -- 172.21.15.77:0/4209059331 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f979c011780 con 0x7f97ac0a9320 2022-08-01T19:33:51.703 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.684+0000 7f979b7fe700 1 -- 172.21.15.77:0/4209059331 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979c0118e0 con 0x7f97ac0a9320 2022-08-01T19:33:51.703 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.684+0000 7f97ab59e700 1 -- 172.21.15.77:0/4209059331 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ac152a10 con 0x7f97ac0a9320 2022-08-01T19:33:51.703 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.684+0000 7f979b7fe700 1 -- 172.21.15.77:0/4209059331 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f979c011c40 con 0x7f97ac0a9320 2022-08-01T19:33:51.704 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.684+0000 7f97ab59e700 1 -- 172.21.15.77:0/4209059331 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ac152f50 con 0x7f97ac0a9320 2022-08-01T19:33:51.704 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.684+0000 7f979b7fe700 1 -- 172.21.15.77:0/4209059331 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f979c006020 con 0x7f97ac0a9320 2022-08-01T19:33:51.704 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.688+0000 7f979b7fe700 1 -- 172.21.15.77:0/4209059331 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 869+0+0 (secure 0 0 0) 0x7f979c00de40 con 0x7f97ac0a9320 2022-08-01T19:33:51.704 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.688+0000 7f97ab59e700 1 -- 172.21.15.77:0/4209059331 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97ac153110 con 0x7f97ac0a9320 2022-08-01T19:33:51.704 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.688+0000 7f979b7fe700 1 -- 172.21.15.77:0/4209059331 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f979c005b20 con 0x7f97ac0a9320 2022-08-01T19:33:51.759 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.756+0000 7f97ab59e700 1 -- 172.21.15.77:0/4209059331 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd new", "uuid": "805f7051-a527-4e98-8844-b6e406a63248", "id": 1} v 0) v1 -- 0x7f97ac0574e0 con 0x7f97ac0a9320 2022-08-01T19:33:51.765 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.764+0000 7f979b7fe700 1 -- 172.21.15.77:0/4209059331 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "805f7051-a527-4e98-8844-b6e406a63248", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f979c0290b0 con 0x7f97ac0a9320 2022-08-01T19:33:51.766 INFO:teuthology.orchestra.run.smithi077.stdout:1 2022-08-01T19:33:51.767 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.764+0000 7f97ab59e700 1 -- 172.21.15.77:0/4209059331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97ac0a9320 msgr2=0x7f97ac155980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:51.767 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.764+0000 7f97ab59e700 1 --2- 172.21.15.77:0/4209059331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97ac0a9320 0x7f97ac155980 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f979c0030b0 tx=0x7f979c008040).stop 2022-08-01T19:33:51.768 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.764+0000 7f97ab59e700 1 -- 172.21.15.77:0/4209059331 shutdown_connections 2022-08-01T19:33:51.768 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.764+0000 7f97ab59e700 1 --2- 172.21.15.77:0/4209059331 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f97ac0b0d10 0x7f97ac156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:51.768 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.764+0000 7f97ab59e700 1 --2- 172.21.15.77:0/4209059331 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f97ac0b0560 0x7f97ac155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:51.768 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.764+0000 7f97ab59e700 1 --2- 172.21.15.77:0/4209059331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97ac0a9320 0x7f97ac155980 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:51.768 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.764+0000 7f97ab59e700 1 -- 172.21.15.77:0/4209059331 >> 172.21.15.77:0/4209059331 conn(0x7f97ac0612d0 msgr2=0x7f97ac13f640 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:33:51.769 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.764+0000 7f97ab59e700 1 -- 172.21.15.77:0/4209059331 shutdown_connections 2022-08-01T19:33:51.769 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.764+0000 7f97ab59e700 1 -- 172.21.15.77:0/4209059331 wait complete. 2022-08-01T19:33:51.778 DEBUG:teuthology.orchestra.run.smithi077:> sudo ceph --cluster ceph osd new 89f7f3df-7c10-48e1-8b69-9e4243923a61 2 2022-08-01T19:33:51.875 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.872+0000 7f8fb9e36700 1 Processor -- start 2022-08-01T19:33:51.875 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.872+0000 7f8fb9e36700 1 -- start start 2022-08-01T19:33:51.878 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.876+0000 7f8fb9e36700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f8fb40a9320 0x7f8fb41466e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:51.878 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.876+0000 7f8fb9e36700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8fb40b0d10 0x7f8fb4146bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:51.878 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.876+0000 7f8fb9e36700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fb40b12d0 con 0x7f8fb40b0560 2022-08-01T19:33:51.879 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.876+0000 7f8fb9e36700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fb40a9700 con 0x7f8fb40a9320 2022-08-01T19:33:51.879 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.876+0000 7f8fb9e36700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb40b1500 con 0x7f8fb40b0d10 2022-08-01T19:33:51.879 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.876+0000 7f8fb3fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8fb40b0d10 0x7f8fb4146bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:51.879 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.876+0000 7f8fb8e34700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f8fb40a9320 0x7f8fb41466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:51.880 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.876+0000 7f8fb3fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8fb40b0d10 0x7f8fb4146bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.77:38668/0 (socket says 172.21.15.77:38668) 2022-08-01T19:33:51.880 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.876+0000 7f8fb3fff700 1 -- 172.21.15.77:0/2090480399 learned_addr learned my addr 172.21.15.77:0/2090480399 (peer_addr_for_me v2:172.21.15.77:0/0) 2022-08-01T19:33:51.880 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.876+0000 7f8fb37fe700 1 -- 172.21.15.77:0/2090480399 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1449772442 0 0) 0x7f8fb40b12d0 con 0x7f8fb40b0560 2022-08-01T19:33:51.880 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.876+0000 7f8fb37fe700 1 -- 172.21.15.77:0/2090480399 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f98003260 con 0x7f8fb40b0560 2022-08-01T19:33:51.881 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.876+0000 7f8fb8e34700 1 -- 172.21.15.77:0/2090480399 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8fb40b0d10 msgr2=0x7f8fb4146bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:33:51.881 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.876+0000 7f8fb8e34700 1 --2- 172.21.15.77:0/2090480399 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8fb40b0d10 0x7f8fb4146bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:51.881 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.876+0000 7f8fb8e34700 1 -- 172.21.15.77:0/2090480399 >> v1:172.21.15.77:6789/0 conn(0x7f8fb40b0560 legacy=0x7f8fb4140350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:51.881 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.876+0000 7f8fb8e34700 1 -- 172.21.15.77:0/2090480399 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb40b17a0 con 0x7f8fb40a9320 2022-08-01T19:33:51.881 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.876+0000 7f8fb8e34700 1 --2- 172.21.15.77:0/2090480399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fb40a9320 0x7f8fb41466e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f8fa8002f70 tx=0x7f8fa8006df0).ready entity=mon.1 client_cookie=373a522dceea073f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:33:51.882 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.876+0000 7f8fb37fe700 1 -- 172.21.15.77:0/2090480399 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fa800d4a0 con 0x7f8fb40a9320 2022-08-01T19:33:51.882 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.876+0000 7f8fb37fe700 1 -- 172.21.15.77:0/2090480399 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa8000e90 con 0x7f8fb40a9320 2022-08-01T19:33:51.882 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.876+0000 7f8fb37fe700 1 -- 172.21.15.77:0/2090480399 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fa800d7f0 con 0x7f8fb40a9320 2022-08-01T19:33:51.886 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.876+0000 7f8fb9e36700 1 -- 172.21.15.77:0/2090480399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fb40a9320 msgr2=0x7f8fb41466e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:51.886 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.876+0000 7f8fb9e36700 1 --2- 172.21.15.77:0/2090480399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fb40a9320 0x7f8fb41466e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f8fa8002f70 tx=0x7f8fa8006df0).stop 2022-08-01T19:33:51.887 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.876+0000 7f8fb9e36700 1 -- 172.21.15.77:0/2090480399 shutdown_connections 2022-08-01T19:33:51.887 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.876+0000 7f8fb9e36700 1 --2- 172.21.15.77:0/2090480399 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8fb40b0d10 0x7f8fb4146bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:51.887 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.876+0000 7f8fb9e36700 1 --2- 172.21.15.77:0/2090480399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fb40a9320 0x7f8fb41466e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:51.887 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.876+0000 7f8fb9e36700 1 -- 172.21.15.77:0/2090480399 >> 172.21.15.77:0/2090480399 conn(0x7f8fb40612d0 msgr2=0x7f8fb4062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:33:51.887 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.876+0000 7f8fb9e36700 1 -- 172.21.15.77:0/2090480399 shutdown_connections 2022-08-01T19:33:51.888 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.876+0000 7f8fb9e36700 1 -- 172.21.15.77:0/2090480399 wait complete. 2022-08-01T19:33:51.888 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.880+0000 7f8fb9e36700 1 Processor -- start 2022-08-01T19:33:51.888 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.880+0000 7f8fb9e36700 1 -- start start 2022-08-01T19:33:51.888 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.880+0000 7f8fb9e36700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fb40b0560 0x7f8fb40b6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:51.889 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.880+0000 7f8fb9e36700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8fb40b0d10 0x7f8fb4142590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:51.889 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.880+0000 7f8fb9e36700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fb4141c80 0x7f8fb40b5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:51.889 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.880+0000 7f8fb9e36700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb40b6670 con 0x7f8fb4141c80 2022-08-01T19:33:51.889 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.880+0000 7f8fb9e36700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb40b4380 con 0x7f8fb40b0560 2022-08-01T19:33:51.890 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.880+0000 7f8fb9e36700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb41438c0 con 0x7f8fb40b0d10 2022-08-01T19:33:51.890 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.880+0000 7f8fb8e34700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fb40b0560 0x7f8fb40b6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:51.890 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.880+0000 7f8fb3fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8fb40b0d10 0x7f8fb4142590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:51.890 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.880+0000 7f8fb8e34700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fb40b0560 0x7f8fb40b6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49982/0 (socket says 172.21.15.77:49982) 2022-08-01T19:33:51.891 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.880+0000 7f8fb8e34700 1 -- 172.21.15.77:0/4021685831 learned_addr learned my addr 172.21.15.77:0/4021685831 (peer_addr_for_me v2:172.21.15.77:0/0) 2022-08-01T19:33:51.891 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.880+0000 7f8fb9635700 1 --2- 172.21.15.77:0/4021685831 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fb4141c80 0x7f8fb40b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:51.891 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.880+0000 7f8fb8e34700 1 -- 172.21.15.77:0/4021685831 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8fb40b0d10 msgr2=0x7f8fb4142590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:51.891 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.880+0000 7f8fb8e34700 1 --2- 172.21.15.77:0/4021685831 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8fb40b0d10 0x7f8fb4142590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:51.892 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.880+0000 7f8fb8e34700 1 -- 172.21.15.77:0/4021685831 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fb4141c80 msgr2=0x7f8fb40b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:51.892 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.880+0000 7f8fb8e34700 1 --2- 172.21.15.77:0/4021685831 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fb4141c80 0x7f8fb40b5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:51.892 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.880+0000 7f8fb8e34700 1 -- 172.21.15.77:0/4021685831 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa800b180 con 0x7f8fb40b0560 2022-08-01T19:33:51.892 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.880+0000 7f8fb9635700 1 --2- 172.21.15.77:0/4021685831 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fb4141c80 0x7f8fb40b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:33:51.893 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.880+0000 7f8fb8e34700 1 --2- 172.21.15.77:0/4021685831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fb40b0560 0x7f8fb40b6040 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f8fb40b3b30 tx=0x7f8fa8014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:33:51.893 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.880+0000 7f8fb1ffb700 1 -- 172.21.15.77:0/4021685831 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fa8013aa0 con 0x7f8fb40b0560 2022-08-01T19:33:51.893 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.880+0000 7f8fb1ffb700 1 -- 172.21.15.77:0/4021685831 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa8013c00 con 0x7f8fb40b0560 2022-08-01T19:33:51.893 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.880+0000 7f8fb9e36700 1 -- 172.21.15.77:0/4021685831 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fb40b3860 con 0x7f8fb40b0560 2022-08-01T19:33:51.893 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.880+0000 7f8fb1ffb700 1 -- 172.21.15.77:0/4021685831 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fa801e400 con 0x7f8fb40b0560 2022-08-01T19:33:51.894 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.880+0000 7f8fb9e36700 1 -- 172.21.15.77:0/4021685831 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fb4142a80 con 0x7f8fb40b0560 2022-08-01T19:33:51.894 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.880+0000 7f8fb1ffb700 1 -- 172.21.15.77:0/4021685831 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f8fa8025030 con 0x7f8fb40b0560 2022-08-01T19:33:51.894 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.880+0000 7f8fb1ffb700 1 -- 172.21.15.77:0/4021685831 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 984+0+0 (secure 0 0 0) 0x7f8fa8011810 con 0x7f8fb40b0560 2022-08-01T19:33:51.894 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.884+0000 7f8fb9e36700 1 -- 172.21.15.77:0/4021685831 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fb40574e0 con 0x7f8fb40b0560 2022-08-01T19:33:51.895 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.884+0000 7f8fb1ffb700 1 -- 172.21.15.77:0/4021685831 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f8fa8023070 con 0x7f8fb40b0560 2022-08-01T19:33:51.955 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.952+0000 7f8fb9e36700 1 -- 172.21.15.77:0/4021685831 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd new", "uuid": "89f7f3df-7c10-48e1-8b69-9e4243923a61", "id": 2} v 0) v1 -- 0x7f8fb41475c0 con 0x7f8fb40b0560 2022-08-01T19:33:51.962 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.960+0000 7f8fb1ffb700 1 -- 172.21.15.77:0/4021685831 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "89f7f3df-7c10-48e1-8b69-9e4243923a61", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8fa8019030 con 0x7f8fb40b0560 2022-08-01T19:33:51.963 INFO:teuthology.orchestra.run.smithi077.stdout:2 2022-08-01T19:33:51.964 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.960+0000 7f8fb9e36700 1 -- 172.21.15.77:0/4021685831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fb40b0560 msgr2=0x7f8fb40b6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:51.964 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.960+0000 7f8fb9e36700 1 --2- 172.21.15.77:0/4021685831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fb40b0560 0x7f8fb40b6040 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f8fb40b3b30 tx=0x7f8fa8014000).stop 2022-08-01T19:33:51.965 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.960+0000 7f8fb9e36700 1 -- 172.21.15.77:0/4021685831 shutdown_connections 2022-08-01T19:33:51.965 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.960+0000 7f8fb9e36700 1 --2- 172.21.15.77:0/4021685831 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8fb4141c80 0x7f8fb40b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:51.965 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.960+0000 7f8fb9e36700 1 --2- 172.21.15.77:0/4021685831 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8fb40b0d10 0x7f8fb4142590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:51.965 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.960+0000 7f8fb9e36700 1 --2- 172.21.15.77:0/4021685831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fb40b0560 0x7f8fb40b6040 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:51.966 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.960+0000 7f8fb9e36700 1 -- 172.21.15.77:0/4021685831 >> 172.21.15.77:0/4021685831 conn(0x7f8fb40612d0 msgr2=0x7f8fb413c060 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:33:51.966 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.964+0000 7f8fb9e36700 1 -- 172.21.15.77:0/4021685831 shutdown_connections 2022-08-01T19:33:51.966 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:51.964+0000 7f8fb9e36700 1 -- 172.21.15.77:0/4021685831 wait complete. 2022-08-01T19:33:51.974 DEBUG:teuthology.orchestra.run.smithi077:> sudo ceph --cluster ceph osd new 96123468-aabc-4a64-909e-a192c369090d 3 2022-08-01T19:33:52.080 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.080+0000 7ffa070f0700 1 Processor -- start 2022-08-01T19:33:52.081 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.080+0000 7ffa070f0700 1 -- start start 2022-08-01T19:33:52.089 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.080+0000 7ffa070f0700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ffa000a9da0 0x7ffa00060950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:52.089 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.080+0000 7ffa070f0700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa00067030 con 0x7ffa000b31d0 2022-08-01T19:33:52.089 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.080+0000 7ffa070f0700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa000b35b0 con 0x7ffa001402d0 2022-08-01T19:33:52.089 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.080+0000 7ffa070f0700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa00064a70 con 0x7ffa000a9da0 2022-08-01T19:33:52.090 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.080+0000 7ffa058ed700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ffa000a9da0 0x7ffa00060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:52.090 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.080+0000 7ffa068ef700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7ffa001402d0 0x7ffa00140720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.77:52156/0 (socket says 172.21.15.77:52156) 2022-08-01T19:33:52.091 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.080+0000 7ffa068ef700 1 -- 172.21.15.77:0/1005175628 learned_addr learned my addr 172.21.15.77:0/1005175628 (peer_addr_for_me v1:172.21.15.77:0/0) 2022-08-01T19:33:52.091 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.080+0000 7ffa058ed700 1 -- 172.21.15.77:0/1005175628 >> v1:172.21.15.77:6789/0 conn(0x7ffa001402d0 legacy=0x7ffa00140720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:52.092 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.080+0000 7ffa058ed700 1 -- 172.21.15.77:0/1005175628 >> v1:172.21.15.17:6789/0 conn(0x7ffa000b31d0 legacy=0x7ffa00066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:52.092 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.080+0000 7ffa058ed700 1 -- 172.21.15.77:0/1005175628 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa00066af0 con 0x7ffa000a9da0 2022-08-01T19:33:52.093 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.080+0000 7ffa058ed700 1 --2- 172.21.15.77:0/1005175628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa000a9da0 0x7ffa00060950 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7ff9f0002f70 tx=0x7ff9f0006df0).ready entity=mon.1 client_cookie=fc17e03216a47ad2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:33:52.093 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.080+0000 7ffa050ec700 1 -- 172.21.15.77:0/1005175628 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9f000d610 con 0x7ffa000a9da0 2022-08-01T19:33:52.094 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.080+0000 7ffa050ec700 1 -- 172.21.15.77:0/1005175628 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f0000e90 con 0x7ffa000a9da0 2022-08-01T19:33:52.094 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.080+0000 7ffa050ec700 1 -- 172.21.15.77:0/1005175628 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9f000d960 con 0x7ffa000a9da0 2022-08-01T19:33:52.095 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.080+0000 7ffa070f0700 1 -- 172.21.15.77:0/1005175628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa000a9da0 msgr2=0x7ffa00060950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:52.095 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.080+0000 7ffa070f0700 1 --2- 172.21.15.77:0/1005175628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa000a9da0 0x7ffa00060950 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7ff9f0002f70 tx=0x7ff9f0006df0).stop 2022-08-01T19:33:52.096 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.080+0000 7ffa070f0700 1 -- 172.21.15.77:0/1005175628 shutdown_connections 2022-08-01T19:33:52.096 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.080+0000 7ffa070f0700 1 --2- 172.21.15.77:0/1005175628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa000a9da0 0x7ffa00060950 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:52.097 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.080+0000 7ffa070f0700 1 -- 172.21.15.77:0/1005175628 >> 172.21.15.77:0/1005175628 conn(0x7ffa00063870 msgr2=0x7ffa00063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:33:52.097 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.080+0000 7ffa070f0700 1 -- 172.21.15.77:0/1005175628 shutdown_connections 2022-08-01T19:33:52.098 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.080+0000 7ffa070f0700 1 -- 172.21.15.77:0/1005175628 wait complete. 2022-08-01T19:33:52.098 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.084+0000 7ffa070f0700 1 Processor -- start 2022-08-01T19:33:52.099 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.084+0000 7ffa070f0700 1 -- start start 2022-08-01T19:33:52.099 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.084+0000 7ffa070f0700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffa000a9da0 0x7ffa000b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:52.100 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.084+0000 7ffa070f0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa000b31d0 0x7ffa000aca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:52.100 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.084+0000 7ffa070f0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa001402d0 0x7ffa000acf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:52.101 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.084+0000 7ffa070f0700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0005f130 con 0x7ffa001402d0 2022-08-01T19:33:52.101 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.084+0000 7ffa070f0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa000ad420 con 0x7ffa000b31d0 2022-08-01T19:33:52.102 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.084+0000 7ffa070f0700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa000aeee0 con 0x7ffa000a9da0 2022-08-01T19:33:52.103 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.084+0000 7ffa058ed700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa000b31d0 0x7ffa000aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:52.103 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.084+0000 7ffa058ed700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa000b31d0 0x7ffa000aca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49996/0 (socket says 172.21.15.77:49996) 2022-08-01T19:33:52.104 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.084+0000 7ffa058ed700 1 -- 172.21.15.77:0/1554569712 learned_addr learned my addr 172.21.15.77:0/1554569712 (peer_addr_for_me v2:172.21.15.77:0/0) 2022-08-01T19:33:52.104 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.084+0000 7ffa060ee700 1 --2- 172.21.15.77:0/1554569712 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffa000a9da0 0x7ffa000b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:52.105 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.084+0000 7ffa068ef700 1 --2- 172.21.15.77:0/1554569712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa001402d0 0x7ffa000acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:52.105 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.084+0000 7ffa058ed700 1 -- 172.21.15.77:0/1554569712 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffa000a9da0 msgr2=0x7ffa000b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:52.106 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.084+0000 7ffa058ed700 1 --2- 172.21.15.77:0/1554569712 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffa000a9da0 0x7ffa000b1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:52.106 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.084+0000 7ffa058ed700 1 -- 172.21.15.77:0/1554569712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa001402d0 msgr2=0x7ffa000acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:52.107 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.084+0000 7ffa058ed700 1 --2- 172.21.15.77:0/1554569712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa001402d0 0x7ffa000acf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:52.107 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.084+0000 7ffa058ed700 1 -- 172.21.15.77:0/1554569712 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f000b180 con 0x7ffa000b31d0 2022-08-01T19:33:52.108 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.084+0000 7ffa068ef700 1 --2- 172.21.15.77:0/1554569712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa001402d0 0x7ffa000acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:33:52.108 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.084+0000 7ffa058ed700 1 --2- 172.21.15.77:0/1554569712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa000b31d0 0x7ffa000aca40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7ffa000abc30 tx=0x7ff9f0015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:33:52.109 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.084+0000 7ff9ef7fe700 1 -- 172.21.15.77:0/1554569712 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9f000d990 con 0x7ffa000b31d0 2022-08-01T19:33:52.110 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.084+0000 7ff9ef7fe700 1 -- 172.21.15.77:0/1554569712 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f0011c20 con 0x7ffa000b31d0 2022-08-01T19:33:52.110 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.084+0000 7ffa070f0700 1 -- 172.21.15.77:0/1554569712 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa000b01f0 con 0x7ffa000b31d0 2022-08-01T19:33:52.111 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.084+0000 7ff9ef7fe700 1 -- 172.21.15.77:0/1554569712 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9f001e460 con 0x7ffa000b31d0 2022-08-01T19:33:52.111 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.084+0000 7ffa070f0700 1 -- 172.21.15.77:0/1554569712 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa00140030 con 0x7ffa000b31d0 2022-08-01T19:33:52.112 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.088+0000 7ff9ef7fe700 1 -- 172.21.15.77:0/1554569712 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7ff9f0025030 con 0x7ffa000b31d0 2022-08-01T19:33:52.113 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.088+0000 7ff9ef7fe700 1 -- 172.21.15.77:0/1554569712 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1099+0+0 (secure 0 0 0) 0x7ff9f001d880 con 0x7ffa000b31d0 2022-08-01T19:33:52.114 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.092+0000 7ffa070f0700 1 -- 172.21.15.77:0/1554569712 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa00066d90 con 0x7ffa000b31d0 2022-08-01T19:33:52.115 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.092+0000 7ff9ef7fe700 1 -- 172.21.15.77:0/1554569712 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7ff9f0023070 con 0x7ffa000b31d0 2022-08-01T19:33:52.166 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.164+0000 7ffa070f0700 1 -- 172.21.15.77:0/1554569712 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd new", "uuid": "96123468-aabc-4a64-909e-a192c369090d", "id": 3} v 0) v1 -- 0x7ffa000574e0 con 0x7ffa000b31d0 2022-08-01T19:33:52.175 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.172+0000 7ff9ef7fe700 1 -- 172.21.15.77:0/1554569712 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "96123468-aabc-4a64-909e-a192c369090d", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff9f001d3c0 con 0x7ffa000b31d0 2022-08-01T19:33:52.175 INFO:teuthology.orchestra.run.smithi077.stdout:3 2022-08-01T19:33:52.176 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.176+0000 7ffa070f0700 1 -- 172.21.15.77:0/1554569712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa000b31d0 msgr2=0x7ffa000aca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:52.177 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.176+0000 7ffa070f0700 1 --2- 172.21.15.77:0/1554569712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa000b31d0 0x7ffa000aca40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7ffa000abc30 tx=0x7ff9f0015000).stop 2022-08-01T19:33:52.177 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.176+0000 7ffa070f0700 1 -- 172.21.15.77:0/1554569712 shutdown_connections 2022-08-01T19:33:52.177 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.176+0000 7ffa070f0700 1 --2- 172.21.15.77:0/1554569712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa001402d0 0x7ffa000acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:52.177 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.176+0000 7ffa070f0700 1 --2- 172.21.15.77:0/1554569712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa000b31d0 0x7ffa000aca40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:52.178 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.176+0000 7ffa070f0700 1 --2- 172.21.15.77:0/1554569712 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffa000a9da0 0x7ffa000b1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:52.178 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.176+0000 7ffa070f0700 1 -- 172.21.15.77:0/1554569712 >> 172.21.15.77:0/1554569712 conn(0x7ffa00063870 msgr2=0x7ffa00060950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:33:52.178 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.176+0000 7ffa070f0700 1 -- 172.21.15.77:0/1554569712 shutdown_connections 2022-08-01T19:33:52.178 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.176+0000 7ffa070f0700 1 -- 172.21.15.77:0/1554569712 wait complete. 2022-08-01T19:33:52.188 DEBUG:teuthology.orchestra.run.smithi077:> sudo ceph --cluster ceph osd new a6e26665-e04b-46ec-ab66-e912acbf044c 4 2022-08-01T19:33:52.293 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.292+0000 7f503e35a700 1 Processor -- start 2022-08-01T19:33:52.293 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.292+0000 7f503e35a700 1 -- start start 2022-08-01T19:33:52.295 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.292+0000 7f503e35a700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f50380b3720 0x7f5038141590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:52.295 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.292+0000 7f503e35a700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5038145220 0x7f5038141a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:52.295 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.292+0000 7f503e35a700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50380ada80 con 0x7f50380b3340 2022-08-01T19:33:52.295 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.292+0000 7f503e35a700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50380b0090 con 0x7f50380b3720 2022-08-01T19:33:52.296 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.292+0000 7f503e35a700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50381456f0 con 0x7f5038145220 2022-08-01T19:33:52.296 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.292+0000 7f503c8f7700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5038145220 0x7f5038141a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:52.296 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.292+0000 7f5037fff700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7f50380b3340 0x7f503813e6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.77:52164/0 (socket says 172.21.15.77:52164) 2022-08-01T19:33:52.296 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.292+0000 7f5037fff700 1 -- 172.21.15.77:0/1517642209 learned_addr learned my addr 172.21.15.77:0/1517642209 (peer_addr_for_me v1:172.21.15.77:0/0) 2022-08-01T19:33:52.297 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.292+0000 7f50377fe700 1 --2- 172.21.15.77:0/1517642209 >> v2:172.21.15.17:3300/0 conn(0x7f50380b3720 0x7f5038141590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:52.297 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.292+0000 7f503c8f7700 1 -- 172.21.15.77:0/1517642209 >> v2:172.21.15.17:3300/0 conn(0x7f50380b3720 msgr2=0x7f5038141590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:33:52.297 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.292+0000 7f503c8f7700 1 --2- 172.21.15.77:0/1517642209 >> v2:172.21.15.17:3300/0 conn(0x7f50380b3720 0x7f5038141590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:52.297 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.292+0000 7f503c8f7700 1 -- 172.21.15.77:0/1517642209 >> v1:172.21.15.77:6789/0 conn(0x7f50380b3340 legacy=0x7f503813e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:52.298 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.292+0000 7f5036ffd700 1 -- 172.21.15.77:0/1517642209 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 228588443 0 0) 0x7f50380ada80 con 0x7f50380b3340 2022-08-01T19:33:52.298 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.292+0000 7f503c8f7700 1 -- 172.21.15.77:0/1517642209 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50380b02f0 con 0x7f5038145220 2022-08-01T19:33:52.298 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.292+0000 7f503c8f7700 1 --2- 172.21.15.77:0/1517642209 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5038145220 0x7f5038141a80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f5038141320 tx=0x7f5030006df0).ready entity=mon.2 client_cookie=b6ea51ff47b43b2c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:33:52.298 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.292+0000 7f5036ffd700 1 -- 172.21.15.77:0/1517642209 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f503000d570 con 0x7f5038145220 2022-08-01T19:33:52.298 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.292+0000 7f5036ffd700 1 -- 172.21.15.77:0/1517642209 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5030000e90 con 0x7f5038145220 2022-08-01T19:33:52.299 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.292+0000 7f5036ffd700 1 -- 172.21.15.77:0/1517642209 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f503000d8c0 con 0x7f5038145220 2022-08-01T19:33:52.302 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.292+0000 7f503e35a700 1 -- 172.21.15.77:0/1517642209 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5038145220 msgr2=0x7f5038141a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:52.302 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.292+0000 7f503e35a700 1 --2- 172.21.15.77:0/1517642209 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5038145220 0x7f5038141a80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f5038141320 tx=0x7f5030006df0).stop 2022-08-01T19:33:52.303 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.292+0000 7f503e35a700 1 -- 172.21.15.77:0/1517642209 shutdown_connections 2022-08-01T19:33:52.303 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.292+0000 7f503e35a700 1 --2- 172.21.15.77:0/1517642209 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5038145220 0x7f5038141a80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:52.303 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.292+0000 7f503e35a700 1 --2- 172.21.15.77:0/1517642209 >> v2:172.21.15.17:3300/0 conn(0x7f50380b3720 0x7f5038141590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:52.303 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.292+0000 7f503e35a700 1 -- 172.21.15.77:0/1517642209 >> 172.21.15.77:0/1517642209 conn(0x7f5038060950 msgr2=0x7f5038060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:33:52.304 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.296+0000 7f503e35a700 1 -- 172.21.15.77:0/1517642209 shutdown_connections 2022-08-01T19:33:52.304 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.296+0000 7f503e35a700 1 -- 172.21.15.77:0/1517642209 wait complete. 2022-08-01T19:33:52.304 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.296+0000 7f503e35a700 1 Processor -- start 2022-08-01T19:33:52.304 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.296+0000 7f503e35a700 1 -- start start 2022-08-01T19:33:52.305 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.296+0000 7f503e35a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50380b3340 0x7f5038148600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:52.305 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.296+0000 7f503e35a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50380b3720 0x7f5038146170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:52.305 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.296+0000 7f503e35a700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5038145220 0x7f5038146660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:52.305 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.296+0000 7f503e35a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5038147810 con 0x7f50380b3720 2022-08-01T19:33:52.305 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.296+0000 7f503e35a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5038145cf0 con 0x7f50380b3340 2022-08-01T19:33:52.306 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.296+0000 7f503e35a700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50380a9210 con 0x7f5038145220 2022-08-01T19:33:52.306 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.296+0000 7f5037fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50380b3340 0x7f5038148600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:52.306 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.296+0000 7f5037fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50380b3340 0x7f5038148600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50006/0 (socket says 172.21.15.77:50006) 2022-08-01T19:33:52.306 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.296+0000 7f5037fff700 1 -- 172.21.15.77:0/1310589128 learned_addr learned my addr 172.21.15.77:0/1310589128 (peer_addr_for_me v2:172.21.15.77:0/0) 2022-08-01T19:33:52.307 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.296+0000 7f503c8f7700 1 --2- 172.21.15.77:0/1310589128 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5038145220 0x7f5038146660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:52.307 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.296+0000 7f50377fe700 1 --2- 172.21.15.77:0/1310589128 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50380b3720 0x7f5038146170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:52.307 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.296+0000 7f5037fff700 1 -- 172.21.15.77:0/1310589128 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5038145220 msgr2=0x7f5038146660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:52.307 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.296+0000 7f5037fff700 1 --2- 172.21.15.77:0/1310589128 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5038145220 0x7f5038146660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:52.308 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.296+0000 7f5037fff700 1 -- 172.21.15.77:0/1310589128 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50380b3720 msgr2=0x7f5038146170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:52.308 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.296+0000 7f5037fff700 1 --2- 172.21.15.77:0/1310589128 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50380b3720 0x7f5038146170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:52.308 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.296+0000 7f5037fff700 1 -- 172.21.15.77:0/1310589128 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f503000b180 con 0x7f50380b3340 2022-08-01T19:33:52.308 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.296+0000 7f5037fff700 1 --2- 172.21.15.77:0/1310589128 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50380b3340 0x7f5038148600 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f50380a95d0 tx=0x7f5028007e40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:33:52.308 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.296+0000 7f50357fa700 1 -- 172.21.15.77:0/1310589128 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5028005ba0 con 0x7f50380b3340 2022-08-01T19:33:52.309 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.296+0000 7f50357fa700 1 -- 172.21.15.77:0/1310589128 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5028005d00 con 0x7f50380b3340 2022-08-01T19:33:52.309 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.296+0000 7f503e35a700 1 -- 172.21.15.77:0/1310589128 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5038144430 con 0x7f50380b3340 2022-08-01T19:33:52.309 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.296+0000 7f50357fa700 1 -- 172.21.15.77:0/1310589128 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50280072c0 con 0x7f50380b3340 2022-08-01T19:33:52.309 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.296+0000 7f503e35a700 1 -- 172.21.15.77:0/1310589128 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5038144bd0 con 0x7f50380b3340 2022-08-01T19:33:52.310 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.296+0000 7f50357fa700 1 -- 172.21.15.77:0/1310589128 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f502801a060 con 0x7f50380b3340 2022-08-01T19:33:52.310 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.296+0000 7f50357fa700 1 -- 172.21.15.77:0/1310589128 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1214+0+0 (secure 0 0 0) 0x7f50280075f0 con 0x7f50380b3340 2022-08-01T19:33:52.310 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.300+0000 7f503e35a700 1 -- 172.21.15.77:0/1310589128 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50380af410 con 0x7f50380b3340 2022-08-01T19:33:52.311 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.300+0000 7f50357fa700 1 -- 172.21.15.77:0/1310589128 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f5028018070 con 0x7f50380b3340 2022-08-01T19:33:52.370 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.368+0000 7f503e35a700 1 -- 172.21.15.77:0/1310589128 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a6e26665-e04b-46ec-ab66-e912acbf044c", "id": 4} v 0) v1 -- 0x7f50380b0590 con 0x7f50380b3340 2022-08-01T19:33:52.377 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.376+0000 7f50357fa700 1 -- 172.21.15.77:0/1310589128 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a6e26665-e04b-46ec-ab66-e912acbf044c", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5028013450 con 0x7f50380b3340 2022-08-01T19:33:52.377 INFO:teuthology.orchestra.run.smithi077.stdout:4 2022-08-01T19:33:52.380 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.376+0000 7f503e35a700 1 -- 172.21.15.77:0/1310589128 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50380b3340 msgr2=0x7f5038148600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:52.380 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.376+0000 7f503e35a700 1 --2- 172.21.15.77:0/1310589128 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50380b3340 0x7f5038148600 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f50380a95d0 tx=0x7f5028007e40).stop 2022-08-01T19:33:52.380 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.376+0000 7f503e35a700 1 -- 172.21.15.77:0/1310589128 shutdown_connections 2022-08-01T19:33:52.380 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.376+0000 7f503e35a700 1 --2- 172.21.15.77:0/1310589128 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5038145220 0x7f5038146660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:52.381 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.376+0000 7f503e35a700 1 --2- 172.21.15.77:0/1310589128 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50380b3720 0x7f5038146170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:52.381 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.376+0000 7f503e35a700 1 --2- 172.21.15.77:0/1310589128 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50380b3340 0x7f5038148600 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:52.381 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.376+0000 7f503e35a700 1 -- 172.21.15.77:0/1310589128 >> 172.21.15.77:0/1310589128 conn(0x7f5038060950 msgr2=0x7f50380b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:33:52.381 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.376+0000 7f503e35a700 1 -- 172.21.15.77:0/1310589128 shutdown_connections 2022-08-01T19:33:52.381 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.376+0000 7f503e35a700 1 -- 172.21.15.77:0/1310589128 wait complete. 2022-08-01T19:33:52.392 DEBUG:teuthology.orchestra.run.smithi077:> sudo ceph --cluster ceph osd new 502c0034-18ee-44ac-87ff-19b9dddb66a0 5 2022-08-01T19:33:52.494 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.492+0000 7f4b2739a700 1 Processor -- start 2022-08-01T19:33:52.494 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.492+0000 7f4b2739a700 1 -- start start 2022-08-01T19:33:52.496 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.492+0000 7f4b2739a700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f4b20060290 0x7f4b20064ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:52.496 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.492+0000 7f4b2739a700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b20060670 con 0x7f4b201445d0 2022-08-01T19:33:52.496 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.492+0000 7f4b2739a700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b200b2540 con 0x7f4b200b3a60 2022-08-01T19:33:52.497 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.492+0000 7f4b2739a700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b20065e10 con 0x7f4b20060290 2022-08-01T19:33:52.497 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.492+0000 7f4b25136700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f4b20060290 0x7f4b20064ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:52.497 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.492+0000 7f4b25937700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7f4b200b3a60 0x7f4b2014b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.77:52180/0 (socket says 172.21.15.77:52180) 2022-08-01T19:33:52.497 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.492+0000 7f4b25937700 1 -- 172.21.15.77:0/3351649136 learned_addr learned my addr 172.21.15.77:0/3351649136 (peer_addr_for_me v1:172.21.15.77:0/0) 2022-08-01T19:33:52.498 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.492+0000 7f4b25136700 1 -- 172.21.15.77:0/3351649136 >> v1:172.21.15.77:6789/0 conn(0x7f4b200b3a60 legacy=0x7f4b2014b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:52.498 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.492+0000 7f4b25136700 1 -- 172.21.15.77:0/3351649136 >> v1:172.21.15.17:6789/0 conn(0x7f4b201445d0 legacy=0x7f4b20144b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:52.498 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.492+0000 7f4b25136700 1 -- 172.21.15.77:0/3351649136 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b200b27e0 con 0x7f4b20060290 2022-08-01T19:33:52.498 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.496+0000 7f4b25136700 1 --2- 172.21.15.77:0/3351649136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b20060290 0x7f4b20064ea0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f4b1c002f70 tx=0x7f4b1c006df0).ready entity=mon.1 client_cookie=3dd47b381a7a4a8e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:33:52.499 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.496+0000 7f4b0ffff700 1 -- 172.21.15.77:0/3351649136 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b1c00d610 con 0x7f4b20060290 2022-08-01T19:33:52.499 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.496+0000 7f4b0ffff700 1 -- 172.21.15.77:0/3351649136 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b1c000e90 con 0x7f4b20060290 2022-08-01T19:33:52.499 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.496+0000 7f4b0ffff700 1 -- 172.21.15.77:0/3351649136 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b1c00d960 con 0x7f4b20060290 2022-08-01T19:33:52.503 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.496+0000 7f4b2739a700 1 -- 172.21.15.77:0/3351649136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b20060290 msgr2=0x7f4b20064ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:52.503 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.496+0000 7f4b2739a700 1 --2- 172.21.15.77:0/3351649136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b20060290 0x7f4b20064ea0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f4b1c002f70 tx=0x7f4b1c006df0).stop 2022-08-01T19:33:52.503 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.496+0000 7f4b2739a700 1 -- 172.21.15.77:0/3351649136 shutdown_connections 2022-08-01T19:33:52.503 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.496+0000 7f4b2739a700 1 --2- 172.21.15.77:0/3351649136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b20060290 0x7f4b20064ea0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:52.504 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.496+0000 7f4b2739a700 1 -- 172.21.15.77:0/3351649136 >> 172.21.15.77:0/3351649136 conn(0x7f4b2005d0f0 msgr2=0x7f4b2005f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:33:52.504 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.496+0000 7f4b2739a700 1 -- 172.21.15.77:0/3351649136 shutdown_connections 2022-08-01T19:33:52.504 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.496+0000 7f4b2739a700 1 -- 172.21.15.77:0/3351649136 wait complete. 2022-08-01T19:33:52.504 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.496+0000 7f4b2739a700 1 Processor -- start 2022-08-01T19:33:52.504 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.496+0000 7f4b2739a700 1 -- start start 2022-08-01T19:33:52.505 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.500+0000 7f4b2739a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b20060290 0x7f4b2013f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:52.505 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.500+0000 7f4b2739a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b200b3a60 0x7f4b20153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:52.505 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.500+0000 7f4b2739a700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4b2013c2d0 0x7f4b201541f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:52.505 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.500+0000 7f4b2739a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2013d6c0 con 0x7f4b200b3a60 2022-08-01T19:33:52.506 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.500+0000 7f4b2739a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2013d920 con 0x7f4b20060290 2022-08-01T19:33:52.506 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.500+0000 7f4b2739a700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b200b1010 con 0x7f4b2013c2d0 2022-08-01T19:33:52.506 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.500+0000 7f4b25136700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b20060290 0x7f4b2013f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:52.506 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.500+0000 7f4b25937700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4b2013c2d0 0x7f4b201541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:52.507 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.500+0000 7f4b25136700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b20060290 0x7f4b2013f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50018/0 (socket says 172.21.15.77:50018) 2022-08-01T19:33:52.507 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.500+0000 7f4b25136700 1 -- 172.21.15.77:0/2581918702 learned_addr learned my addr 172.21.15.77:0/2581918702 (peer_addr_for_me v2:172.21.15.77:0/0) 2022-08-01T19:33:52.507 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.500+0000 7f4b24935700 1 --2- 172.21.15.77:0/2581918702 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b200b3a60 0x7f4b20153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:52.507 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.500+0000 7f4b25136700 1 -- 172.21.15.77:0/2581918702 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4b2013c2d0 msgr2=0x7f4b201541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:52.508 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.500+0000 7f4b25136700 1 --2- 172.21.15.77:0/2581918702 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4b2013c2d0 0x7f4b201541f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:52.508 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.500+0000 7f4b25136700 1 -- 172.21.15.77:0/2581918702 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b200b3a60 msgr2=0x7f4b20153d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:52.508 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.500+0000 7f4b25136700 1 --2- 172.21.15.77:0/2581918702 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b200b3a60 0x7f4b20153d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:52.508 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.500+0000 7f4b25136700 1 -- 172.21.15.77:0/2581918702 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b1c00b180 con 0x7f4b20060290 2022-08-01T19:33:52.509 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.500+0000 7f4b25136700 1 --2- 172.21.15.77:0/2581918702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b20060290 0x7f4b2013f1e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f4b1c002f70 tx=0x7f4b1c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:33:52.509 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.500+0000 7f4b0e7fc700 1 -- 172.21.15.77:0/2581918702 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b1c00de90 con 0x7f4b20060290 2022-08-01T19:33:52.509 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.500+0000 7f4b0e7fc700 1 -- 172.21.15.77:0/2581918702 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b1c006a60 con 0x7f4b20060290 2022-08-01T19:33:52.509 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.500+0000 7f4b2739a700 1 -- 172.21.15.77:0/2581918702 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b200b12b0 con 0x7f4b20060290 2022-08-01T19:33:52.509 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.500+0000 7f4b0e7fc700 1 -- 172.21.15.77:0/2581918702 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b1c011e20 con 0x7f4b20060290 2022-08-01T19:33:52.510 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.500+0000 7f4b2739a700 1 -- 172.21.15.77:0/2581918702 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b200b0890 con 0x7f4b20060290 2022-08-01T19:33:52.510 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.500+0000 7f4b0e7fc700 1 -- 172.21.15.77:0/2581918702 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f4b1c025030 con 0x7f4b20060290 2022-08-01T19:33:52.510 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.500+0000 7f4b0e7fc700 1 -- 172.21.15.77:0/2581918702 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1329+0+0 (secure 0 0 0) 0x7f4b1c01d970 con 0x7f4b20060290 2022-08-01T19:33:52.510 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.500+0000 7f4b2739a700 1 -- 172.21.15.77:0/2581918702 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b20064a00 con 0x7f4b20060290 2022-08-01T19:33:52.511 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.504+0000 7f4b0e7fc700 1 -- 172.21.15.77:0/2581918702 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f4b1c023070 con 0x7f4b20060290 2022-08-01T19:33:52.572 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.572+0000 7f4b2739a700 1 -- 172.21.15.77:0/2581918702 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd new", "uuid": "502c0034-18ee-44ac-87ff-19b9dddb66a0", "id": 5} v 0) v1 -- 0x7f4b200574e0 con 0x7f4b20060290 2022-08-01T19:33:52.580 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.580+0000 7f4b0e7fc700 1 -- 172.21.15.77:0/2581918702 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "502c0034-18ee-44ac-87ff-19b9dddb66a0", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4b1c01db80 con 0x7f4b20060290 2022-08-01T19:33:52.581 INFO:teuthology.orchestra.run.smithi077.stdout:5 2022-08-01T19:33:52.582 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.580+0000 7f4b2739a700 1 -- 172.21.15.77:0/2581918702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b20060290 msgr2=0x7f4b2013f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:52.582 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.580+0000 7f4b2739a700 1 --2- 172.21.15.77:0/2581918702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b20060290 0x7f4b2013f1e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f4b1c002f70 tx=0x7f4b1c015000).stop 2022-08-01T19:33:52.583 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.580+0000 7f4b2739a700 1 -- 172.21.15.77:0/2581918702 shutdown_connections 2022-08-01T19:33:52.583 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.580+0000 7f4b2739a700 1 --2- 172.21.15.77:0/2581918702 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4b2013c2d0 0x7f4b201541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:52.583 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.580+0000 7f4b2739a700 1 --2- 172.21.15.77:0/2581918702 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b200b3a60 0x7f4b20153d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:52.583 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.580+0000 7f4b2739a700 1 --2- 172.21.15.77:0/2581918702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b20060290 0x7f4b2013f1e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:52.584 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.580+0000 7f4b2739a700 1 -- 172.21.15.77:0/2581918702 >> 172.21.15.77:0/2581918702 conn(0x7f4b2005d0f0 msgr2=0x7f4b20140830 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:33:52.584 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.580+0000 7f4b2739a700 1 -- 172.21.15.77:0/2581918702 shutdown_connections 2022-08-01T19:33:52.584 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.580+0000 7f4b2739a700 1 -- 172.21.15.77:0/2581918702 wait complete. 2022-08-01T19:33:52.592 DEBUG:teuthology.orchestra.run.smithi077:> sudo ceph --cluster ceph osd new ac4e755a-7fe7-46cb-8407-852d5af9ffc5 6 2022-08-01T19:33:52.685 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.684+0000 7fd304d32700 1 Processor -- start 2022-08-01T19:33:52.686 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.684+0000 7fd304d32700 1 -- start start 2022-08-01T19:33:52.688 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.684+0000 7fd304d32700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd3000b41f0 0x7fd300061d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:52.689 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.684+0000 7fd304d32700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3000b60e0 con 0x7fd3000b6420 2022-08-01T19:33:52.689 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.684+0000 7fd304d32700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3000b2100 con 0x7fd300142ff0 2022-08-01T19:33:52.689 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.684+0000 7fd304d32700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3000b23a0 con 0x7fd3000b41f0 2022-08-01T19:33:52.689 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.684+0000 7fd2feffd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd3000b41f0 0x7fd300061d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:52.690 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.684+0000 7fd2fffff700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7fd300142ff0 0x7fd3001434a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.77:52192/0 (socket says 172.21.15.77:52192) 2022-08-01T19:33:52.690 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.684+0000 7fd2fffff700 1 -- 172.21.15.77:0/2396479601 learned_addr learned my addr 172.21.15.77:0/2396479601 (peer_addr_for_me v1:172.21.15.77:0/0) 2022-08-01T19:33:52.690 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.684+0000 7fd2feffd700 1 -- 172.21.15.77:0/2396479601 >> v1:172.21.15.77:6789/0 conn(0x7fd300142ff0 legacy=0x7fd3001434a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:52.690 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.684+0000 7fd2feffd700 1 -- 172.21.15.77:0/2396479601 >> v1:172.21.15.17:6789/0 conn(0x7fd3000b6420 legacy=0x7fd300061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:52.691 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.684+0000 7fd2fe7fc700 1 -- 172.21.15.77:0/2396479601 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1700673365 0 0) 0x7fd3000b2100 con 0x7fd300142ff0 2022-08-01T19:33:52.691 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.684+0000 7fd2feffd700 1 -- 172.21.15.77:0/2396479601 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3000b2640 con 0x7fd3000b41f0 2022-08-01T19:33:52.691 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.684+0000 7fd2feffd700 1 --2- 172.21.15.77:0/2396479601 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd3000b41f0 0x7fd300061d10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fd2f0002f70 tx=0x7fd2f0006df0).ready entity=mon.2 client_cookie=c8c1ac252316101c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:33:52.691 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.684+0000 7fd2fe7fc700 1 -- 172.21.15.77:0/2396479601 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2f000d5c0 con 0x7fd3000b41f0 2022-08-01T19:33:52.692 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.684+0000 7fd2fe7fc700 1 -- 172.21.15.77:0/2396479601 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f0000e90 con 0x7fd3000b41f0 2022-08-01T19:33:52.692 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.684+0000 7fd2fe7fc700 1 -- 172.21.15.77:0/2396479601 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2f000d910 con 0x7fd3000b41f0 2022-08-01T19:33:52.692 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.688+0000 7fd304d32700 1 -- 172.21.15.77:0/2396479601 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd3000b41f0 msgr2=0x7fd300061d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:52.692 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.688+0000 7fd304d32700 1 --2- 172.21.15.77:0/2396479601 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd3000b41f0 0x7fd300061d10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fd2f0002f70 tx=0x7fd2f0006df0).stop 2022-08-01T19:33:52.692 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.688+0000 7fd304d32700 1 -- 172.21.15.77:0/2396479601 shutdown_connections 2022-08-01T19:33:52.693 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.688+0000 7fd304d32700 1 --2- 172.21.15.77:0/2396479601 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd3000b41f0 0x7fd300061d10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:52.693 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.688+0000 7fd304d32700 1 -- 172.21.15.77:0/2396479601 >> 172.21.15.77:0/2396479601 conn(0x7fd30005c620 msgr2=0x7fd30005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:33:52.693 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.688+0000 7fd304d32700 1 -- 172.21.15.77:0/2396479601 shutdown_connections 2022-08-01T19:33:52.693 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.688+0000 7fd304d32700 1 -- 172.21.15.77:0/2396479601 wait complete. 2022-08-01T19:33:52.696 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.688+0000 7fd304d32700 1 Processor -- start 2022-08-01T19:33:52.697 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.688+0000 7fd304d32700 1 -- start start 2022-08-01T19:33:52.697 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.688+0000 7fd304d32700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3000b41f0 0x7fd30013f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:52.697 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.688+0000 7fd304d32700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd3000b6420 0x7fd3000adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:52.697 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.688+0000 7fd304d32700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd300142ff0 0x7fd300155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:52.698 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.688+0000 7fd304d32700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3001400f0 con 0x7fd3000b41f0 2022-08-01T19:33:52.698 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.688+0000 7fd304d32700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30013fa20 con 0x7fd300142ff0 2022-08-01T19:33:52.698 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.688+0000 7fd304d32700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3000ae580 con 0x7fd3000b6420 2022-08-01T19:33:52.698 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.688+0000 7fd2fffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd300142ff0 0x7fd300155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:52.699 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.688+0000 7fd2feffd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd3000b6420 0x7fd3000adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:52.699 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.688+0000 7fd2fffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd300142ff0 0x7fd300155c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50034/0 (socket says 172.21.15.77:50034) 2022-08-01T19:33:52.699 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.688+0000 7fd2fffff700 1 -- 172.21.15.77:0/3727540124 learned_addr learned my addr 172.21.15.77:0/3727540124 (peer_addr_for_me v2:172.21.15.77:0/0) 2022-08-01T19:33:52.699 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.688+0000 7fd2ff7fe700 1 --2- 172.21.15.77:0/3727540124 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3000b41f0 0x7fd30013f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:52.700 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.688+0000 7fd2fffff700 1 -- 172.21.15.77:0/3727540124 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd3000b6420 msgr2=0x7fd3000adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:52.700 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.688+0000 7fd2fffff700 1 --2- 172.21.15.77:0/3727540124 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd3000b6420 0x7fd3000adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:52.700 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.688+0000 7fd2fffff700 1 -- 172.21.15.77:0/3727540124 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3000b41f0 msgr2=0x7fd30013f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:52.700 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.688+0000 7fd2fffff700 1 --2- 172.21.15.77:0/3727540124 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3000b41f0 0x7fd30013f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:52.700 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.688+0000 7fd2fffff700 1 -- 172.21.15.77:0/3727540124 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f000b180 con 0x7fd300142ff0 2022-08-01T19:33:52.701 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.688+0000 7fd2feffd700 1 --2- 172.21.15.77:0/3727540124 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd3000b6420 0x7fd3000adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:33:52.701 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.688+0000 7fd2fffff700 1 --2- 172.21.15.77:0/3727540124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd300142ff0 0x7fd300155c90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fd300140230 tx=0x7fd2f8001360).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:33:52.701 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.688+0000 7fd2fcff9700 1 -- 172.21.15.77:0/3727540124 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2f8005ba0 con 0x7fd300142ff0 2022-08-01T19:33:52.701 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.688+0000 7fd2fcff9700 1 -- 172.21.15.77:0/3727540124 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f8005d00 con 0x7fd300142ff0 2022-08-01T19:33:52.702 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.688+0000 7fd2fcff9700 1 -- 172.21.15.77:0/3727540124 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2f8004a90 con 0x7fd300142ff0 2022-08-01T19:33:52.702 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.688+0000 7fd304d32700 1 -- 172.21.15.77:0/3727540124 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd30013fc50 con 0x7fd300142ff0 2022-08-01T19:33:52.702 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.688+0000 7fd304d32700 1 -- 172.21.15.77:0/3727540124 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3000b0420 con 0x7fd300142ff0 2022-08-01T19:33:52.702 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.692+0000 7fd2fcff9700 1 -- 172.21.15.77:0/3727540124 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7fd2f8007110 con 0x7fd300142ff0 2022-08-01T19:33:52.703 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.692+0000 7fd2fcff9700 1 -- 172.21.15.77:0/3727540124 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1444+0+0 (secure 0 0 0) 0x7fd2f8004dc0 con 0x7fd300142ff0 2022-08-01T19:33:52.703 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.692+0000 7fd304d32700 1 -- 172.21.15.77:0/3727540124 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3000ae3e0 con 0x7fd300142ff0 2022-08-01T19:33:52.703 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.696+0000 7fd2fcff9700 1 -- 172.21.15.77:0/3727540124 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7fd3000ae3e0 con 0x7fd300142ff0 2022-08-01T19:33:52.765 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.764+0000 7fd304d32700 1 -- 172.21.15.77:0/3727540124 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ac4e755a-7fe7-46cb-8407-852d5af9ffc5", "id": 6} v 0) v1 -- 0x7fd30013fde0 con 0x7fd300142ff0 2022-08-01T19:33:52.772 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.772+0000 7fd2fcff9700 1 -- 172.21.15.77:0/3727540124 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ac4e755a-7fe7-46cb-8407-852d5af9ffc5", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd2f8005030 con 0x7fd300142ff0 2022-08-01T19:33:52.772 INFO:teuthology.orchestra.run.smithi077.stdout:6 2022-08-01T19:33:52.773 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.772+0000 7fd304d32700 1 -- 172.21.15.77:0/3727540124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd300142ff0 msgr2=0x7fd300155c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:52.774 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.772+0000 7fd304d32700 1 --2- 172.21.15.77:0/3727540124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd300142ff0 0x7fd300155c90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fd300140230 tx=0x7fd2f8001360).stop 2022-08-01T19:33:52.774 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.772+0000 7fd304d32700 1 -- 172.21.15.77:0/3727540124 shutdown_connections 2022-08-01T19:33:52.774 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.772+0000 7fd304d32700 1 --2- 172.21.15.77:0/3727540124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd300142ff0 0x7fd300155c90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:52.775 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.772+0000 7fd304d32700 1 --2- 172.21.15.77:0/3727540124 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd3000b6420 0x7fd3000adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:52.775 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.772+0000 7fd304d32700 1 --2- 172.21.15.77:0/3727540124 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3000b41f0 0x7fd30013f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:52.775 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.772+0000 7fd304d32700 1 -- 172.21.15.77:0/3727540124 >> 172.21.15.77:0/3727540124 conn(0x7fd30005c620 msgr2=0x7fd3000b9720 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:33:52.775 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.772+0000 7fd304d32700 1 -- 172.21.15.77:0/3727540124 shutdown_connections 2022-08-01T19:33:52.775 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.772+0000 7fd304d32700 1 -- 172.21.15.77:0/3727540124 wait complete. 2022-08-01T19:33:52.784 DEBUG:teuthology.orchestra.run.smithi077:> sudo ceph --cluster ceph osd new 874761e2-6410-40fc-b60c-e58abe5ce489 7 2022-08-01T19:33:52.882 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.880+0000 7f5f2ce69700 1 Processor -- start 2022-08-01T19:33:52.883 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.880+0000 7f5f2ce69700 1 -- start start 2022-08-01T19:33:52.894 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.880+0000 7f5f2ce69700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f5f280a9320 0x7f5f281466e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:52.894 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.880+0000 7f5f2ce69700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5f280b0d10 0x7f5f28146bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:52.894 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.880+0000 7f5f2ce69700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f280b12d0 con 0x7f5f280b0560 2022-08-01T19:33:52.895 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.880+0000 7f5f2ce69700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f280a9700 con 0x7f5f280b0d10 2022-08-01T19:33:52.895 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.880+0000 7f5f2ce69700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f280b1500 con 0x7f5f280a9320 2022-08-01T19:33:52.895 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.880+0000 7f5f277fe700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f5f280a9320 0x7f5f281466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:52.895 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.880+0000 7f5f27fff700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7f5f280b0560 0x7f5f28140350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.77:52202/0 (socket says 172.21.15.77:52202) 2022-08-01T19:33:52.896 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.880+0000 7f5f27fff700 1 -- 172.21.15.77:0/1946455021 learned_addr learned my addr 172.21.15.77:0/1946455021 (peer_addr_for_me v1:172.21.15.77:0/0) 2022-08-01T19:33:52.896 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.880+0000 7f5f26ffd700 1 --2- 172.21.15.77:0/1946455021 >> v2:172.21.15.17:3300/0 conn(0x7f5f280b0d10 0x7f5f28146bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:52.896 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.880+0000 7f5f277fe700 1 -- 172.21.15.77:0/1946455021 >> v2:172.21.15.17:3300/0 conn(0x7f5f280b0d10 msgr2=0x7f5f28146bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:33:52.896 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.880+0000 7f5f277fe700 1 --2- 172.21.15.77:0/1946455021 >> v2:172.21.15.17:3300/0 conn(0x7f5f280b0d10 0x7f5f28146bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:52.897 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.880+0000 7f5f277fe700 1 -- 172.21.15.77:0/1946455021 >> v1:172.21.15.77:6789/0 conn(0x7f5f280b0560 legacy=0x7f5f28140350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:52.897 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.880+0000 7f5f267fc700 1 -- 172.21.15.77:0/1946455021 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3542563768 0 0) 0x7f5f280b12d0 con 0x7f5f280b0560 2022-08-01T19:33:52.898 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.880+0000 7f5f277fe700 1 -- 172.21.15.77:0/1946455021 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f280b17a0 con 0x7f5f280a9320 2022-08-01T19:33:52.898 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.880+0000 7f5f26ffd700 1 --2- 172.21.15.77:0/1946455021 >> v2:172.21.15.17:3300/0 conn(0x7f5f280b0d10 0x7f5f28146bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:33:52.899 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.880+0000 7f5f277fe700 1 --2- 172.21.15.77:0/1946455021 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f280a9320 0x7f5f281466e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f5f18002f70 tx=0x7f5f18006df0).ready entity=mon.1 client_cookie=6775a5509b9bad23 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:33:52.899 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.880+0000 7f5f267fc700 1 -- 172.21.15.77:0/1946455021 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f1800d5c0 con 0x7f5f280a9320 2022-08-01T19:33:52.900 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.880+0000 7f5f267fc700 1 -- 172.21.15.77:0/1946455021 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f18000e90 con 0x7f5f280a9320 2022-08-01T19:33:52.900 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.880+0000 7f5f267fc700 1 -- 172.21.15.77:0/1946455021 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f1800d910 con 0x7f5f280a9320 2022-08-01T19:33:52.901 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.884+0000 7f5f2ce69700 1 -- 172.21.15.77:0/1946455021 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f280a9320 msgr2=0x7f5f281466e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:52.901 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.884+0000 7f5f2ce69700 1 --2- 172.21.15.77:0/1946455021 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f280a9320 0x7f5f281466e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f5f18002f70 tx=0x7f5f18006df0).stop 2022-08-01T19:33:52.902 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.884+0000 7f5f2ce69700 1 -- 172.21.15.77:0/1946455021 shutdown_connections 2022-08-01T19:33:52.902 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.884+0000 7f5f2ce69700 1 --2- 172.21.15.77:0/1946455021 >> v2:172.21.15.17:3300/0 conn(0x7f5f280b0d10 0x7f5f28146bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:52.902 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.884+0000 7f5f2ce69700 1 --2- 172.21.15.77:0/1946455021 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f280a9320 0x7f5f281466e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:52.903 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.884+0000 7f5f2ce69700 1 -- 172.21.15.77:0/1946455021 >> 172.21.15.77:0/1946455021 conn(0x7f5f280612d0 msgr2=0x7f5f28062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:33:52.903 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.884+0000 7f5f2ce69700 1 -- 172.21.15.77:0/1946455021 shutdown_connections 2022-08-01T19:33:52.904 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.884+0000 7f5f2ce69700 1 -- 172.21.15.77:0/1946455021 wait complete. 2022-08-01T19:33:52.904 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.884+0000 7f5f2ce69700 1 Processor -- start 2022-08-01T19:33:52.905 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.884+0000 7f5f2ce69700 1 -- start start 2022-08-01T19:33:52.905 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.884+0000 7f5f2ce69700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f280a9320 0x7f5f2813fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:52.906 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.884+0000 7f5f2ce69700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f280b0560 0x7f5f28152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:52.906 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.884+0000 7f5f2ce69700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5f280b0d10 0x7f5f28152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:52.907 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.884+0000 7f5f2ce69700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f28147d10 con 0x7f5f280a9320 2022-08-01T19:33:52.907 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.884+0000 7f5f2ce69700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2813b380 con 0x7f5f280b0560 2022-08-01T19:33:52.908 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.884+0000 7f5f2ce69700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2813b620 con 0x7f5f280b0d10 2022-08-01T19:33:52.908 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.884+0000 7f5f27fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5f280b0d10 0x7f5f28152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:52.909 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.884+0000 7f5f26ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f280b0560 0x7f5f28152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:52.909 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.884+0000 7f5f27fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5f280b0d10 0x7f5f28152810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.77:38738/0 (socket says 172.21.15.77:38738) 2022-08-01T19:33:52.910 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.884+0000 7f5f27fff700 1 -- 172.21.15.77:0/4180646722 learned_addr learned my addr 172.21.15.77:0/4180646722 (peer_addr_for_me v2:172.21.15.77:0/0) 2022-08-01T19:33:52.910 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.884+0000 7f5f277fe700 1 --2- 172.21.15.77:0/4180646722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f280a9320 0x7f5f2813fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:52.911 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.884+0000 7f5f27fff700 1 -- 172.21.15.77:0/4180646722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f280b0560 msgr2=0x7f5f28152320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:52.911 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.884+0000 7f5f27fff700 1 --2- 172.21.15.77:0/4180646722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f280b0560 0x7f5f28152320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:52.912 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.884+0000 7f5f27fff700 1 -- 172.21.15.77:0/4180646722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f280a9320 msgr2=0x7f5f2813fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:52.912 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.884+0000 7f5f27fff700 1 --2- 172.21.15.77:0/4180646722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f280a9320 0x7f5f2813fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:52.913 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.884+0000 7f5f27fff700 1 -- 172.21.15.77:0/4180646722 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1800b180 con 0x7f5f280b0d10 2022-08-01T19:33:52.913 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.884+0000 7f5f27fff700 1 --2- 172.21.15.77:0/4180646722 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5f280b0d10 0x7f5f28152810 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f5f20001250 tx=0x7f5f20007dc0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:33:52.914 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.884+0000 7f5f24ff9700 1 -- 172.21.15.77:0/4180646722 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f20005c70 con 0x7f5f280b0d10 2022-08-01T19:33:52.914 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.884+0000 7f5f24ff9700 1 -- 172.21.15.77:0/4180646722 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f20005dd0 con 0x7f5f280b0d10 2022-08-01T19:33:52.915 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.884+0000 7f5f2ce69700 1 -- 172.21.15.77:0/4180646722 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f2813b8c0 con 0x7f5f280b0d10 2022-08-01T19:33:52.915 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.884+0000 7f5f24ff9700 1 -- 172.21.15.77:0/4180646722 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f20004a90 con 0x7f5f280b0d10 2022-08-01T19:33:52.916 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.884+0000 7f5f2ce69700 1 -- 172.21.15.77:0/4180646722 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f2813ad60 con 0x7f5f280b0d10 2022-08-01T19:33:52.916 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.884+0000 7f5f24ff9700 1 -- 172.21.15.77:0/4180646722 <== mon.2 v2:172.21.15.77:3301/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f5f2001b070 con 0x7f5f280b0d10 2022-08-01T19:33:52.917 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.884+0000 7f5f24ff9700 1 -- 172.21.15.77:0/4180646722 <== mon.2 v2:172.21.15.77:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1559+0+0 (secure 0 0 0) 0x7f5f20004dc0 con 0x7f5f280b0d10 2022-08-01T19:33:52.917 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.888+0000 7f5f2ce69700 1 -- 172.21.15.77:0/4180646722 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f28147e80 con 0x7f5f280b0d10 2022-08-01T19:33:52.918 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.888+0000 7f5f24ff9700 1 -- 172.21.15.77:0/4180646722 <== mon.2 v2:172.21.15.77:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f5f20019060 con 0x7f5f280b0d10 2022-08-01T19:33:52.943 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.940+0000 7f5f24ff9700 1 -- 172.21.15.77:0/4180646722 <== mon.2 v2:172.21.15.77:3301/0 7 ==== mgrmap(e 2) v1 ==== 32894+0+0 (secure 0 0 0) 0x7f5f2001b070 con 0x7f5f280b0d10 2022-08-01T19:33:52.965 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:52.964+0000 7f5f2ce69700 1 -- 172.21.15.77:0/4180646722 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "osd new", "uuid": "874761e2-6410-40fc-b60c-e58abe5ce489", "id": 7} v 0) v1 -- 0x7f5f280574e0 con 0x7f5f280b0d10 2022-08-01T19:33:53.009 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:53.008+0000 7f5f24ff9700 1 -- 172.21.15.77:0/4180646722 <== mon.2 v2:172.21.15.77:3301/0 8 ==== mon_command_ack([{"prefix": "osd new", "uuid": "874761e2-6410-40fc-b60c-e58abe5ce489", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5f2001b580 con 0x7f5f280b0d10 2022-08-01T19:33:53.010 INFO:teuthology.orchestra.run.smithi077.stdout:7 2022-08-01T19:33:53.014 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:53.012+0000 7f5f2ce69700 1 -- 172.21.15.77:0/4180646722 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5f280b0d10 msgr2=0x7f5f28152810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:53.014 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:53.012+0000 7f5f2ce69700 1 --2- 172.21.15.77:0/4180646722 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5f280b0d10 0x7f5f28152810 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f5f20001250 tx=0x7f5f20007dc0).stop 2022-08-01T19:33:53.014 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:53.012+0000 7f5f2ce69700 1 -- 172.21.15.77:0/4180646722 shutdown_connections 2022-08-01T19:33:53.014 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:53.012+0000 7f5f2ce69700 1 --2- 172.21.15.77:0/4180646722 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5f280b0d10 0x7f5f28152810 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:53.015 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:53.012+0000 7f5f2ce69700 1 --2- 172.21.15.77:0/4180646722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f280b0560 0x7f5f28152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:53.015 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:53.012+0000 7f5f2ce69700 1 --2- 172.21.15.77:0/4180646722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f280a9320 0x7f5f2813fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:53.015 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:53.012+0000 7f5f2ce69700 1 -- 172.21.15.77:0/4180646722 >> 172.21.15.77:0/4180646722 conn(0x7f5f280612d0 msgr2=0x7f5f2813cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:33:53.015 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:53.012+0000 7f5f2ce69700 1 -- 172.21.15.77:0/4180646722 shutdown_connections 2022-08-01T19:33:53.016 INFO:teuthology.orchestra.run.smithi077.stderr:2022-08-01T19:33:53.012+0000 7f5f2ce69700 1 -- 172.21.15.77:0/4180646722 wait complete. 2022-08-01T19:33:53.029 INFO:tasks.ceph.osd.0:Restarting daemon 2022-08-01T19:33:53.029 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2022-08-01T19:33:53.033 INFO:tasks.ceph.osd.0:Started 2022-08-01T19:33:53.033 INFO:tasks.ceph.osd.1:Restarting daemon 2022-08-01T19:33:53.034 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2022-08-01T19:33:53.037 INFO:tasks.ceph.osd.1:Started 2022-08-01T19:33:53.038 INFO:tasks.ceph.osd.2:Restarting daemon 2022-08-01T19:33:53.039 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2022-08-01T19:33:53.042 INFO:tasks.ceph.osd.2:Started 2022-08-01T19:33:53.043 INFO:tasks.ceph.osd.3:Restarting daemon 2022-08-01T19:33:53.043 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2022-08-01T19:33:53.046 INFO:tasks.ceph.osd.3:Started 2022-08-01T19:33:53.047 INFO:tasks.ceph.osd.4:Restarting daemon 2022-08-01T19:33:53.047 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2022-08-01T19:33:53.049 INFO:tasks.ceph.osd.4:Started 2022-08-01T19:33:53.050 INFO:tasks.ceph.osd.5:Restarting daemon 2022-08-01T19:33:53.050 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2022-08-01T19:33:53.053 INFO:tasks.ceph.osd.5:Started 2022-08-01T19:33:53.053 INFO:tasks.ceph.osd.6:Restarting daemon 2022-08-01T19:33:53.053 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2022-08-01T19:33:53.056 INFO:tasks.ceph.osd.6:Started 2022-08-01T19:33:53.056 INFO:tasks.ceph.osd.7:Restarting daemon 2022-08-01T19:33:53.057 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2022-08-01T19:33:53.059 INFO:tasks.ceph.osd.7:Started 2022-08-01T19:33:53.059 INFO:tasks.ceph:Waiting for OSDs to come up 2022-08-01T19:33:53.403 INFO:tasks.ceph.osd.0.smithi017.stderr:2022-08-01T19:33:53.400+0000 7f7d37c1bd80 -1 Falling back to public interface 2022-08-01T19:33:53.642 INFO:tasks.ceph.osd.1.smithi017.stderr:2022-08-01T19:33:53.640+0000 7f1cd10c7d80 -1 Falling back to public interface 2022-08-01T19:33:53.646 INFO:tasks.ceph.osd.5.smithi077.stderr:2022-08-01T19:33:53.644+0000 7fb4857bad80 -1 Falling back to public interface 2022-08-01T19:33:53.650 INFO:tasks.ceph.osd.4.smithi077.stderr:2022-08-01T19:33:53.648+0000 7f7515be4d80 -1 Falling back to public interface 2022-08-01T19:33:53.654 INFO:tasks.ceph.osd.2.smithi017.stderr:2022-08-01T19:33:53.652+0000 7fc9509f8d80 -1 Falling back to public interface 2022-08-01T19:33:53.655 INFO:tasks.ceph.osd.3.smithi017.stderr:2022-08-01T19:33:53.652+0000 7f0349f47d80 -1 Falling back to public interface 2022-08-01T19:33:53.673 INFO:tasks.ceph.osd.7.smithi077.stderr:2022-08-01T19:33:53.672+0000 7f2518bbad80 -1 Falling back to public interface 2022-08-01T19:33:53.674 INFO:tasks.ceph.osd.6.smithi077.stderr:2022-08-01T19:33:53.672+0000 7ff5bf24ed80 -1 Falling back to public interface 2022-08-01T19:33:53.865 DEBUG:teuthology.orchestra.run.smithi017:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-08-01T19:33:53.962 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.960+0000 7f42f7fff700 1 Processor -- start 2022-08-01T19:33:53.963 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.960+0000 7f42f7fff700 1 -- start start 2022-08-01T19:33:53.965 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.960+0000 7f42f7fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f42f80603a0 0x7f42f80b1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:53.965 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.960+0000 7f42f7fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f42f8060bd0 0x7f42f8141a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:53.965 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.960+0000 7f42f7fff700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42f80b0c20 con 0x7f42f805fac0 2022-08-01T19:33:53.965 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.960+0000 7f42f7fff700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42f80b08a0 con 0x7f42f80603a0 2022-08-01T19:33:53.966 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.960+0000 7f42f7fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f80aa560 con 0x7f42f8060bd0 2022-08-01T19:33:53.966 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.960+0000 7f42effff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f42f80603a0 0x7f42f80b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:53.966 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.960+0000 7f42effff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f42f80603a0 0x7f42f80b1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35794/0 (socket says 172.21.15.17:35794) 2022-08-01T19:33:53.967 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.960+0000 7f42effff700 1 -- 172.21.15.17:0/3134349537 learned_addr learned my addr 172.21.15.17:0/3134349537 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:33:53.967 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.960+0000 7f42effff700 1 -- 172.21.15.17:0/3134349537 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f42f8060bd0 msgr2=0x7f42f8141a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:33:53.967 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.960+0000 7f42effff700 1 --2- 172.21.15.17:0/3134349537 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f42f8060bd0 0x7f42f8141a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2022-08-01T19:33:53.967 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.960+0000 7f42effff700 1 -- 172.21.15.17:0/3134349537 >> v1:172.21.15.77:6789/0 conn(0x7f42f805fac0 legacy=0x7f42f80b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:53.967 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.960+0000 7f42effff700 1 -- 172.21.15.17:0/3134349537 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f80a9c70 con 0x7f42f80603a0 2022-08-01T19:33:53.968 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.960+0000 7f42effff700 1 --2- 172.21.15.17:0/3134349537 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42f80603a0 0x7f42f80b1700 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f42f80b1c10 tx=0x7f42e0006eb0).ready entity=mon.0 client_cookie=13ed3487a472dbc2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:33:53.968 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.960+0000 7f42f67fc700 1 -- 172.21.15.17:0/3134349537 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f42e000fa60 con 0x7f42f80603a0 2022-08-01T19:33:53.968 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.960+0000 7f42f67fc700 1 -- 172.21.15.17:0/3134349537 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e0000e90 con 0x7f42f80603a0 2022-08-01T19:33:53.968 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.960+0000 7f42f67fc700 1 -- 172.21.15.17:0/3134349537 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f42e000fdb0 con 0x7f42f80603a0 2022-08-01T19:33:53.972 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.964+0000 7f42f7fff700 1 -- 172.21.15.17:0/3134349537 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42f80603a0 msgr2=0x7f42f80b1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:53.972 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.964+0000 7f42f7fff700 1 --2- 172.21.15.17:0/3134349537 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42f80603a0 0x7f42f80b1700 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f42f80b1c10 tx=0x7f42e0006eb0).stop 2022-08-01T19:33:53.973 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.964+0000 7f42f7fff700 1 -- 172.21.15.17:0/3134349537 shutdown_connections 2022-08-01T19:33:53.973 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.964+0000 7f42f7fff700 1 --2- 172.21.15.17:0/3134349537 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f42f8060bd0 0x7f42f8141a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2022-08-01T19:33:53.973 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.964+0000 7f42f7fff700 1 --2- 172.21.15.17:0/3134349537 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42f80603a0 0x7f42f80b1700 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:53.973 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.964+0000 7f42f7fff700 1 -- 172.21.15.17:0/3134349537 >> 172.21.15.17:0/3134349537 conn(0x7f42f8061670 msgr2=0x7f42f8062ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:33:53.974 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.964+0000 7f42f7fff700 1 -- 172.21.15.17:0/3134349537 shutdown_connections 2022-08-01T19:33:53.974 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.964+0000 7f42f7fff700 1 -- 172.21.15.17:0/3134349537 wait complete. 2022-08-01T19:33:53.974 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.964+0000 7f42f7fff700 1 Processor -- start 2022-08-01T19:33:53.974 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.964+0000 7f42f7fff700 1 -- start start 2022-08-01T19:33:53.975 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.964+0000 7f42f7fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42f805fac0 0x7f42f814e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:53.975 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.964+0000 7f42f7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42f80603a0 0x7f42f8152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:53.975 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.964+0000 7f42f7fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f42f8060bd0 0x7f42f80ab250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:53.975 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.964+0000 7f42f7fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f8140d50 con 0x7f42f80603a0 2022-08-01T19:33:53.975 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.964+0000 7f42f7fff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f814e8d0 con 0x7f42f805fac0 2022-08-01T19:33:53.976 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.964+0000 7f42f7fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f8151b30 con 0x7f42f8060bd0 2022-08-01T19:33:53.976 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.964+0000 7f42effff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42f80603a0 0x7f42f8152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:53.976 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.964+0000 7f42effff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42f80603a0 0x7f42f8152230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35800/0 (socket says 172.21.15.17:35800) 2022-08-01T19:33:53.976 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.964+0000 7f42effff700 1 -- 172.21.15.17:0/3622918875 learned_addr learned my addr 172.21.15.17:0/3622918875 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:33:53.977 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.964+0000 7f42f77fe700 1 --2- 172.21.15.17:0/3622918875 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f42f8060bd0 0x7f42f80ab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:53.977 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.964+0000 7f42f6ffd700 1 --2- 172.21.15.17:0/3622918875 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42f805fac0 0x7f42f814e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:53.977 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.968+0000 7f42effff700 1 -- 172.21.15.17:0/3622918875 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f42f8060bd0 msgr2=0x7f42f80ab250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:53.977 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.968+0000 7f42effff700 1 --2- 172.21.15.17:0/3622918875 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f42f8060bd0 0x7f42f80ab250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:53.977 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.968+0000 7f42effff700 1 -- 172.21.15.17:0/3622918875 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42f805fac0 msgr2=0x7f42f814e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:53.978 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.968+0000 7f42effff700 1 --2- 172.21.15.17:0/3622918875 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42f805fac0 0x7f42f814e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:53.978 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.968+0000 7f42effff700 1 -- 172.21.15.17:0/3622918875 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e000b1e0 con 0x7f42f80603a0 2022-08-01T19:33:53.978 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.968+0000 7f42f77fe700 1 --2- 172.21.15.17:0/3622918875 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f42f8060bd0 0x7f42f80ab250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:33:53.978 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.968+0000 7f42effff700 1 --2- 172.21.15.17:0/3622918875 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42f80603a0 0x7f42f8152230 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f42f80b1c10 tx=0x7f42e0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:33:53.979 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.968+0000 7f42f4ff9700 1 -- 172.21.15.17:0/3622918875 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f42e001e070 con 0x7f42f80603a0 2022-08-01T19:33:53.979 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.968+0000 7f42f4ff9700 1 -- 172.21.15.17:0/3622918875 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e0019270 con 0x7f42f80603a0 2022-08-01T19:33:53.979 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.968+0000 7f42f7fff700 1 -- 172.21.15.17:0/3622918875 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f8140ab0 con 0x7f42f80603a0 2022-08-01T19:33:53.979 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.968+0000 7f42f4ff9700 1 -- 172.21.15.17:0/3622918875 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f42e0019d90 con 0x7f42f80603a0 2022-08-01T19:33:53.980 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.968+0000 7f42f7fff700 1 -- 172.21.15.17:0/3622918875 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f8152720 con 0x7f42f80603a0 2022-08-01T19:33:53.980 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.968+0000 7f42f4ff9700 1 -- 172.21.15.17:0/3622918875 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 3) v1 ==== 65884+0+0 (secure 0 0 0) 0x7f42e00223b0 con 0x7f42f80603a0 2022-08-01T19:33:53.980 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.968+0000 7f42f7fff700 1 -- 172.21.15.17:0/3622918875 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42f80a9f10 con 0x7f42f80603a0 2022-08-01T19:33:53.980 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.968+0000 7f42f4ff9700 1 --2- 172.21.15.17:0/3622918875 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f42dc053a30 0x7f42dc055ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:33:53.981 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.968+0000 7f42f4ff9700 1 -- 172.21.15.17:0/3622918875 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 2050+0+0 (secure 0 0 0) 0x7f42e006b5e0 con 0x7f42f80603a0 2022-08-01T19:33:53.981 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.968+0000 7f42f6ffd700 1 --2- 172.21.15.17:0/3622918875 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f42dc053a30 0x7f42dc055ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:33:53.981 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.968+0000 7f42f6ffd700 1 --2- 172.21.15.17:0/3622918875 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f42dc053a30 0x7f42dc055ef0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f42e800d000 tx=0x7f42e8007ba0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:33:53.982 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:53.976+0000 7f42f4ff9700 1 -- 172.21.15.17:0/3622918875 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f42e0043530 con 0x7f42f80603a0 2022-08-01T19:33:54.112 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:54.108+0000 7f42f7fff700 1 -- 172.21.15.17:0/3622918875 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42f805f320 con 0x7f42f80603a0 2022-08-01T19:33:54.172 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:54.168+0000 7f42f4ff9700 1 -- 172.21.15.17:0/3622918875 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) v1 ==== 74+0+8275 (secure 0 0 0) 0x7f42e0046b50 con 0x7f42f80603a0 2022-08-01T19:33:54.172 INFO:teuthology.misc.health.smithi017.stdout: 2022-08-01T19:33:54.173 INFO:teuthology.misc.health.smithi017.stdout:{"epoch":11,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:33:54.004613+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:33:54.174 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:54.172+0000 7f42f7fff700 1 -- 172.21.15.17:0/3622918875 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f42dc053a30 msgr2=0x7f42dc055ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:54.175 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:54.172+0000 7f42f7fff700 1 --2- 172.21.15.17:0/3622918875 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f42dc053a30 0x7f42dc055ef0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f42e800d000 tx=0x7f42e8007ba0).stop 2022-08-01T19:33:54.175 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:54.172+0000 7f42f7fff700 1 -- 172.21.15.17:0/3622918875 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42f80603a0 msgr2=0x7f42f8152230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:33:54.175 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:54.172+0000 7f42f7fff700 1 --2- 172.21.15.17:0/3622918875 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42f80603a0 0x7f42f8152230 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f42f80b1c10 tx=0x7f42e0017000).stop 2022-08-01T19:33:54.175 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:54.172+0000 7f42f7fff700 1 -- 172.21.15.17:0/3622918875 shutdown_connections 2022-08-01T19:33:54.176 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:54.172+0000 7f42f7fff700 1 --2- 172.21.15.17:0/3622918875 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f42dc053a30 0x7f42dc055ef0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:54.176 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:54.172+0000 7f42f7fff700 1 --2- 172.21.15.17:0/3622918875 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f42f8060bd0 0x7f42f80ab250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:54.176 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:54.172+0000 7f42f7fff700 1 --2- 172.21.15.17:0/3622918875 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42f80603a0 0x7f42f8152230 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:54.176 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:54.172+0000 7f42f7fff700 1 --2- 172.21.15.17:0/3622918875 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42f805fac0 0x7f42f814e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:33:54.176 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:54.172+0000 7f42f7fff700 1 -- 172.21.15.17:0/3622918875 >> 172.21.15.17:0/3622918875 conn(0x7f42f8061670 msgr2=0x7f42f80ac450 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:33:54.177 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:54.172+0000 7f42f7fff700 1 -- 172.21.15.17:0/3622918875 shutdown_connections 2022-08-01T19:33:54.177 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:33:54.172+0000 7f42f7fff700 1 -- 172.21.15.17:0/3622918875 wait complete. 2022-08-01T19:33:54.185 DEBUG:teuthology.misc:0 of 8 OSDs are up 2022-08-01T19:33:54.609 INFO:tasks.ceph.osd.0.smithi017.stderr:2022-08-01T19:33:54.608+0000 7f7d37c1bd80 -1 osd.0 0 log_to_monitors {default=true} 2022-08-01T19:33:54.819 INFO:tasks.ceph.osd.1.smithi017.stderr:2022-08-01T19:33:54.816+0000 7f1cd10c7d80 -1 osd.1 0 log_to_monitors {default=true} 2022-08-01T19:33:54.832 INFO:tasks.ceph.osd.2.smithi017.stderr:2022-08-01T19:33:54.828+0000 7fc9509f8d80 -1 osd.2 0 log_to_monitors {default=true} 2022-08-01T19:33:54.837 INFO:tasks.ceph.osd.3.smithi017.stderr:2022-08-01T19:33:54.836+0000 7f0349f47d80 -1 osd.3 0 log_to_monitors {default=true} 2022-08-01T19:33:54.866 INFO:tasks.ceph.osd.6.smithi077.stderr:2022-08-01T19:33:54.864+0000 7ff5bf24ed80 -1 osd.6 0 log_to_monitors {default=true} 2022-08-01T19:33:54.870 INFO:tasks.ceph.osd.7.smithi077.stderr:2022-08-01T19:33:54.868+0000 7f2518bbad80 -1 osd.7 0 log_to_monitors {default=true} 2022-08-01T19:33:55.058 INFO:tasks.ceph.osd.5.smithi077.stderr:2022-08-01T19:33:55.056+0000 7fb4857bad80 -1 osd.5 0 log_to_monitors {default=true} 2022-08-01T19:33:55.091 INFO:tasks.ceph.osd.4.smithi077.stderr:2022-08-01T19:33:55.088+0000 7f7515be4d80 -1 osd.4 0 log_to_monitors {default=true} 2022-08-01T19:33:56.029 INFO:tasks.ceph.osd.0.smithi017.stderr:2022-08-01T19:33:56.028+0000 7f7d29cc5700 -1 osd.0 0 waiting for initial osdmap 2022-08-01T19:33:56.030 INFO:tasks.ceph.osd.1.smithi017.stderr:2022-08-01T19:33:56.028+0000 7f1cc3171700 -1 osd.1 0 waiting for initial osdmap 2022-08-01T19:33:56.030 INFO:tasks.ceph.osd.2.smithi017.stderr:2022-08-01T19:33:56.028+0000 7fc942aa2700 -1 osd.2 0 waiting for initial osdmap 2022-08-01T19:33:56.031 INFO:tasks.ceph.osd.6.smithi077.stderr:2022-08-01T19:33:56.028+0000 7ff5b12f8700 -1 osd.6 0 waiting for initial osdmap 2022-08-01T19:33:56.031 INFO:tasks.ceph.osd.7.smithi077.stderr:2022-08-01T19:33:56.028+0000 7f250ac64700 -1 osd.7 0 waiting for initial osdmap 2022-08-01T19:33:56.032 INFO:tasks.ceph.osd.3.smithi017.stderr:2022-08-01T19:33:56.028+0000 7f033bff1700 -1 osd.3 0 waiting for initial osdmap 2022-08-01T19:33:56.038 INFO:tasks.ceph.osd.2.smithi017.stderr:2022-08-01T19:33:56.036+0000 7fc949f1c700 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-08-01T19:33:56.039 INFO:tasks.ceph.osd.1.smithi017.stderr:2022-08-01T19:33:56.036+0000 7f1cca5eb700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-08-01T19:33:56.040 INFO:tasks.ceph.osd.0.smithi017.stderr:2022-08-01T19:33:56.036+0000 7f7d3113f700 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-08-01T19:33:56.040 INFO:tasks.ceph.osd.3.smithi017.stderr:2022-08-01T19:33:56.036+0000 7f034346b700 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-08-01T19:33:56.041 INFO:tasks.ceph.osd.7.smithi077.stderr:2022-08-01T19:33:56.036+0000 7f25120de700 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-08-01T19:33:56.041 INFO:tasks.ceph.osd.6.smithi077.stderr:2022-08-01T19:33:56.036+0000 7ff5b8772700 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-08-01T19:33:57.041 INFO:tasks.ceph.osd.5.smithi077.stderr:2022-08-01T19:33:57.040+0000 7fb477864700 -1 osd.5 0 waiting for initial osdmap 2022-08-01T19:33:57.042 INFO:tasks.ceph.osd.4.smithi077.stderr:2022-08-01T19:33:57.040+0000 7f7507c8e700 -1 osd.4 0 waiting for initial osdmap 2022-08-01T19:33:57.047 INFO:tasks.ceph.osd.5.smithi077.stderr:2022-08-01T19:33:57.044+0000 7fb47ecde700 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-08-01T19:33:57.049 INFO:tasks.ceph.osd.4.smithi077.stderr:2022-08-01T19:33:57.048+0000 7f750f108700 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-08-01T19:34:00.993 DEBUG:teuthology.orchestra.run.smithi017:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-08-01T19:34:01.094 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.092+0000 7f3bfcf4f700 1 Processor -- start 2022-08-01T19:34:01.094 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.092+0000 7f3bfcf4f700 1 -- start start 2022-08-01T19:34:01.096 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.092+0000 7f3bfcf4f700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3bf80b1d10 0x7f3bf80b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:01.097 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.092+0000 7f3bfcf4f700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3bf80ad310 0x7f3bf80ad7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:01.097 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.092+0000 7f3bfcf4f700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf8140290 con 0x7f3bf80b2920 2022-08-01T19:34:01.097 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.092+0000 7f3bfcf4f700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bf8140530 con 0x7f3bf80b1d10 2022-08-01T19:34:01.097 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.092+0000 7f3bfcf4f700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf81407d0 con 0x7f3bf80ad310 2022-08-01T19:34:01.098 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.092+0000 7f3bf659c700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f3bf80b2920 0x7f3bf8139ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56446/0 (socket says 172.21.15.17:56446) 2022-08-01T19:34:01.098 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.092+0000 7f3bf659c700 1 -- 172.21.15.17:0/3084356478 learned_addr learned my addr 172.21.15.17:0/3084356478 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:01.098 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.092+0000 7f3bedd9b700 1 --2- 172.21.15.17:0/3084356478 >> v2:172.21.15.17:3300/0 conn(0x7f3bf80b1d10 0x7f3bf80b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:01.098 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.096+0000 7f3bf6d9d700 1 --2- 172.21.15.17:0/3084356478 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3bf80ad310 0x7f3bf80ad7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:01.099 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.096+0000 7f3bf5d9b700 1 -- 172.21.15.17:0/3084356478 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 900514608 0 0) 0x7f3bf8140290 con 0x7f3bf80b2920 2022-08-01T19:34:01.099 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.096+0000 7f3bf5d9b700 1 -- 172.21.15.17:0/3084356478 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bd8003260 con 0x7f3bf80b2920 2022-08-01T19:34:01.103 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.096+0000 7f3bf5d9b700 1 -- 172.21.15.17:0/3084356478 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1397164461 0 0) 0x7f3bd8003260 con 0x7f3bf80b2920 2022-08-01T19:34:01.103 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.096+0000 7f3bedd9b700 1 -- 172.21.15.17:0/3084356478 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3bf80ad310 msgr2=0x7f3bf80ad7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:01.104 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.096+0000 7f3bedd9b700 1 --2- 172.21.15.17:0/3084356478 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3bf80ad310 0x7f3bf80ad7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:01.104 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.096+0000 7f3bedd9b700 1 -- 172.21.15.17:0/3084356478 >> v1:172.21.15.17:6789/0 conn(0x7f3bf80b2920 legacy=0x7f3bf8139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:01.104 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.096+0000 7f3bedd9b700 1 -- 172.21.15.17:0/3084356478 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf8140a70 con 0x7f3bf80b1d10 2022-08-01T19:34:01.104 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.096+0000 7f3bedd9b700 1 --2- 172.21.15.17:0/3084356478 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bf80b1d10 0x7f3bf80b0190 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f3be8001800 tx=0x7f3be0006df0).ready entity=mon.0 client_cookie=98738a6e313ef75d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:01.105 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.096+0000 7f3bf5d9b700 1 -- 172.21.15.17:0/3084356478 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3be8001660 con 0x7f3bf80b1d10 2022-08-01T19:34:01.105 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.096+0000 7f3bf5d9b700 1 -- 172.21.15.17:0/3084356478 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be0000e90 con 0x7f3bf80b1d10 2022-08-01T19:34:01.105 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.096+0000 7f3bf5d9b700 1 -- 172.21.15.17:0/3084356478 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3be0012460 con 0x7f3bf80b1d10 2022-08-01T19:34:01.105 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.096+0000 7f3bfcf4f700 1 -- 172.21.15.17:0/3084356478 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bf80b1d10 msgr2=0x7f3bf80b0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:01.105 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.096+0000 7f3bfcf4f700 1 --2- 172.21.15.17:0/3084356478 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bf80b1d10 0x7f3bf80b0190 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f3be8001800 tx=0x7f3be0006df0).stop 2022-08-01T19:34:01.106 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.096+0000 7f3bfcf4f700 1 -- 172.21.15.17:0/3084356478 shutdown_connections 2022-08-01T19:34:01.106 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.096+0000 7f3bfcf4f700 1 --2- 172.21.15.17:0/3084356478 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3bf80ad310 0x7f3bf80ad7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:01.106 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.096+0000 7f3bfcf4f700 1 --2- 172.21.15.17:0/3084356478 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bf80b1d10 0x7f3bf80b0190 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:01.106 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.096+0000 7f3bfcf4f700 1 -- 172.21.15.17:0/3084356478 >> 172.21.15.17:0/3084356478 conn(0x7f3bf8063290 msgr2=0x7f3bf8063670 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:01.107 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.096+0000 7f3bfcf4f700 1 -- 172.21.15.17:0/3084356478 shutdown_connections 2022-08-01T19:34:01.107 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.096+0000 7f3bfcf4f700 1 -- 172.21.15.17:0/3084356478 wait complete. 2022-08-01T19:34:01.107 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.096+0000 7f3bfcf4f700 1 Processor -- start 2022-08-01T19:34:01.107 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.096+0000 7f3bfcf4f700 1 -- start start 2022-08-01T19:34:01.108 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.100+0000 7f3bfcf4f700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3bf80ad310 0x7f3bf813bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:01.108 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.100+0000 7f3bfcf4f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bf80b1d10 0x7f3bf8150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:01.108 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.100+0000 7f3bfcf4f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bf80b2920 0x7f3bf813c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:01.108 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.100+0000 7f3bfcf4f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf8138d50 con 0x7f3bf80b1d10 2022-08-01T19:34:01.108 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.100+0000 7f3bfcf4f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf813b400 con 0x7f3bf80b2920 2022-08-01T19:34:01.109 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.100+0000 7f3bfcf4f700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf813d3a0 con 0x7f3bf80ad310 2022-08-01T19:34:01.109 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.100+0000 7f3bedd9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bf80b1d10 0x7f3bf8150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:01.109 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.100+0000 7f3bedd9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bf80b1d10 0x7f3bf8150480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36028/0 (socket says 172.21.15.17:36028) 2022-08-01T19:34:01.109 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.100+0000 7f3bedd9b700 1 -- 172.21.15.17:0/1648139356 learned_addr learned my addr 172.21.15.17:0/1648139356 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:01.110 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.100+0000 7f3bf659c700 1 --2- 172.21.15.17:0/1648139356 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3bf80ad310 0x7f3bf813bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:01.110 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.100+0000 7f3bf6d9d700 1 --2- 172.21.15.17:0/1648139356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bf80b2920 0x7f3bf813c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:01.110 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.100+0000 7f3bedd9b700 1 -- 172.21.15.17:0/1648139356 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3bf80ad310 msgr2=0x7f3bf813bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:01.111 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.100+0000 7f3bedd9b700 1 --2- 172.21.15.17:0/1648139356 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3bf80ad310 0x7f3bf813bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:01.111 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.100+0000 7f3bedd9b700 1 -- 172.21.15.17:0/1648139356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bf80b2920 msgr2=0x7f3bf813c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:01.112 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.100+0000 7f3bedd9b700 1 --2- 172.21.15.17:0/1648139356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bf80b2920 0x7f3bf813c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:01.112 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.100+0000 7f3bedd9b700 1 -- 172.21.15.17:0/1648139356 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be000b180 con 0x7f3bf80b1d10 2022-08-01T19:34:01.113 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.100+0000 7f3bf659c700 1 --2- 172.21.15.17:0/1648139356 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3bf80ad310 0x7f3bf813bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:34:01.113 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.100+0000 7f3bf6d9d700 1 --2- 172.21.15.17:0/1648139356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bf80b2920 0x7f3bf813c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:34:01.113 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.100+0000 7f3bedd9b700 1 --2- 172.21.15.17:0/1648139356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bf80b1d10 0x7f3bf8150480 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f3be8001800 tx=0x7f3be0014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:01.114 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.100+0000 7f3beffff700 1 -- 172.21.15.17:0/1648139356 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3be000da20 con 0x7f3bf80b1d10 2022-08-01T19:34:01.114 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.100+0000 7f3beffff700 1 -- 172.21.15.17:0/1648139356 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be000db80 con 0x7f3bf80b1d10 2022-08-01T19:34:01.115 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.100+0000 7f3bfcf4f700 1 -- 172.21.15.17:0/1648139356 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bf813a9b0 con 0x7f3bf80b1d10 2022-08-01T19:34:01.115 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.100+0000 7f3beffff700 1 -- 172.21.15.17:0/1648139356 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3be001e400 con 0x7f3bf80b1d10 2022-08-01T19:34:01.116 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.100+0000 7f3bfcf4f700 1 -- 172.21.15.17:0/1648139356 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf813afb0 con 0x7f3bf80b1d10 2022-08-01T19:34:01.117 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.100+0000 7f3bfcf4f700 1 -- 172.21.15.17:0/1648139356 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bf813b910 con 0x7f3bf80b1d10 2022-08-01T19:34:01.118 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.104+0000 7f3beffff700 1 -- 172.21.15.17:0/1648139356 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3be0025030 con 0x7f3bf80b1d10 2022-08-01T19:34:01.118 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.104+0000 7f3beffff700 1 --2- 172.21.15.17:0/1648139356 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3bdc053950 0x7f3bdc055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:01.118 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.104+0000 7f3beffff700 1 -- 172.21.15.17:0/1648139356 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4900+0+0 (secure 0 0 0) 0x7f3be006c7c0 con 0x7f3bf80b1d10 2022-08-01T19:34:01.118 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.104+0000 7f3bf659c700 1 --2- 172.21.15.17:0/1648139356 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3bdc053950 0x7f3bdc055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:01.119 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.104+0000 7f3bf659c700 1 --2- 172.21.15.17:0/1648139356 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3bdc053950 0x7f3bdc055e10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f3be800b000 tx=0x7f3be8001cc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:01.119 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.108+0000 7f3beffff700 1 -- 172.21.15.17:0/1648139356 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3bf813b910 con 0x7f3bf80b1d10 2022-08-01T19:34:01.236 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.232+0000 7f3bfcf4f700 1 -- 172.21.15.17:0/1648139356 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3bf813b910 con 0x7f3bf80b1d10 2022-08-01T19:34:01.238 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.236+0000 7f3beffff700 1 -- 172.21.15.17:0/1648139356 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+11805 (secure 0 0 0) 0x7f3bf813b910 con 0x7f3bf80b1d10 2022-08-01T19:34:01.239 INFO:teuthology.misc.health.smithi017.stdout: 2022-08-01T19:34:01.239 INFO:teuthology.misc.health.smithi017.stdout:{"epoch":16,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:33:59.051027+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:34:01.241 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.236+0000 7f3bfcf4f700 1 -- 172.21.15.17:0/1648139356 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3bdc053950 msgr2=0x7f3bdc055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:01.241 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.236+0000 7f3bfcf4f700 1 --2- 172.21.15.17:0/1648139356 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3bdc053950 0x7f3bdc055e10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f3be800b000 tx=0x7f3be8001cc0).stop 2022-08-01T19:34:01.242 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.236+0000 7f3bfcf4f700 1 -- 172.21.15.17:0/1648139356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bf80b1d10 msgr2=0x7f3bf8150480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:01.242 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.236+0000 7f3bfcf4f700 1 --2- 172.21.15.17:0/1648139356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bf80b1d10 0x7f3bf8150480 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f3be8001800 tx=0x7f3be0014000).stop 2022-08-01T19:34:01.242 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.236+0000 7f3bfcf4f700 1 -- 172.21.15.17:0/1648139356 shutdown_connections 2022-08-01T19:34:01.243 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.236+0000 7f3bfcf4f700 1 --2- 172.21.15.17:0/1648139356 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3bdc053950 0x7f3bdc055e10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:01.243 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.236+0000 7f3bfcf4f700 1 --2- 172.21.15.17:0/1648139356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bf80b2920 0x7f3bf813c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:01.243 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.236+0000 7f3bfcf4f700 1 --2- 172.21.15.17:0/1648139356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bf80b1d10 0x7f3bf8150480 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:01.243 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.236+0000 7f3bfcf4f700 1 --2- 172.21.15.17:0/1648139356 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3bf80ad310 0x7f3bf813bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:01.244 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.236+0000 7f3bfcf4f700 1 -- 172.21.15.17:0/1648139356 >> 172.21.15.17:0/1648139356 conn(0x7f3bf8063290 msgr2=0x7f3bf80b5220 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:01.244 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.236+0000 7f3bfcf4f700 1 -- 172.21.15.17:0/1648139356 shutdown_connections 2022-08-01T19:34:01.244 INFO:teuthology.misc.health.smithi017.stderr:2022-08-01T19:34:01.240+0000 7f3bfcf4f700 1 -- 172.21.15.17:0/1648139356 wait complete. 2022-08-01T19:34:01.250 DEBUG:teuthology.misc:8 of 8 OSDs are up 2022-08-01T19:34:01.251 INFO:tasks.ceph:Creating RBD pool 2022-08-01T19:34:01.252 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph osd pool create rbd 8 2022-08-01T19:34:01.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.352+0000 7f04b359e700 1 Processor -- start 2022-08-01T19:34:01.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.352+0000 7f04b359e700 1 -- start start 2022-08-01T19:34:01.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.356+0000 7f04b359e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f04b40b31f0 0x7f04b40b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:01.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.356+0000 7f04b359e700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04b40b2510 con 0x7f04b40aadb0 2022-08-01T19:34:01.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.356+0000 7f04b359e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04b41443e0 con 0x7f04b40b7500 2022-08-01T19:34:01.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.356+0000 7f04b359e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04b40b3700 con 0x7f04b40b31f0 2022-08-01T19:34:01.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.356+0000 7f04b2d9d700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f04b40aadb0 0x7f04b4064d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56464/0 (socket says 172.21.15.17:56464) 2022-08-01T19:34:01.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.356+0000 7f04a9d9b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f04b40b31f0 0x7f04b40b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:01.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.356+0000 7f04b2d9d700 1 -- 172.21.15.17:0/2824537551 learned_addr learned my addr 172.21.15.17:0/2824537551 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:01.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.356+0000 7f04b1d9b700 1 -- 172.21.15.17:0/2824537551 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3731287400 0 0) 0x7f04b40b2510 con 0x7f04b40aadb0 2022-08-01T19:34:01.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.356+0000 7f04b1d9b700 1 -- 172.21.15.17:0/2824537551 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0498003260 con 0x7f04b40aadb0 2022-08-01T19:34:01.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.356+0000 7f04b1d9b700 1 -- 172.21.15.17:0/2824537551 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1801860743 0 0) 0x7f04b41443e0 con 0x7f04b40b7500 2022-08-01T19:34:01.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.356+0000 7f04b1d9b700 1 -- 172.21.15.17:0/2824537551 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04b40b2510 con 0x7f04b40b7500 2022-08-01T19:34:01.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.356+0000 7f04a9d9b700 1 -- 172.21.15.17:0/2824537551 >> v1:172.21.15.77:6789/0 conn(0x7f04b40b7500 legacy=0x7f04b413e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:01.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.356+0000 7f04a9d9b700 1 -- 172.21.15.17:0/2824537551 >> v1:172.21.15.17:6789/0 conn(0x7f04b40aadb0 legacy=0x7f04b4064d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:01.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.356+0000 7f04a9d9b700 1 -- 172.21.15.17:0/2824537551 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b40b2740 con 0x7f04b40b31f0 2022-08-01T19:34:01.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.356+0000 7f04a9d9b700 1 --2- 172.21.15.17:0/2824537551 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04b40b31f0 0x7f04b40b78e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f049c002f70 tx=0x7f049c006df0).ready entity=mon.0 client_cookie=1cede6eb19efa1e7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:01.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.356+0000 7f04b1d9b700 1 -- 172.21.15.17:0/2824537551 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f049c00d540 con 0x7f04b40b31f0 2022-08-01T19:34:01.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.356+0000 7f04b1d9b700 1 -- 172.21.15.17:0/2824537551 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f049c000e90 con 0x7f04b40b31f0 2022-08-01T19:34:01.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.356+0000 7f04b1d9b700 1 -- 172.21.15.17:0/2824537551 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f049c00d890 con 0x7f04b40b31f0 2022-08-01T19:34:01.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.356+0000 7f04b359e700 1 -- 172.21.15.17:0/2824537551 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04b40b31f0 msgr2=0x7f04b40b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:01.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.356+0000 7f04b359e700 1 --2- 172.21.15.17:0/2824537551 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04b40b31f0 0x7f04b40b78e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f049c002f70 tx=0x7f049c006df0).stop 2022-08-01T19:34:01.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.356+0000 7f04b359e700 1 -- 172.21.15.17:0/2824537551 shutdown_connections 2022-08-01T19:34:01.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.356+0000 7f04b359e700 1 --2- 172.21.15.17:0/2824537551 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04b40b31f0 0x7f04b40b78e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:01.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.356+0000 7f04b359e700 1 -- 172.21.15.17:0/2824537551 >> 172.21.15.17:0/2824537551 conn(0x7f04b405f840 msgr2=0x7f04b405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:01.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.356+0000 7f04b359e700 1 -- 172.21.15.17:0/2824537551 shutdown_connections 2022-08-01T19:34:01.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.356+0000 7f04b359e700 1 -- 172.21.15.17:0/2824537551 wait complete. 2022-08-01T19:34:01.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.360+0000 7f04b359e700 1 Processor -- start 2022-08-01T19:34:01.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.360+0000 7f04b359e700 1 -- start start 2022-08-01T19:34:01.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.360+0000 7f04b359e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04b40aadb0 0x7f04b4141aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:01.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.360+0000 7f04b359e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f04b40b31f0 0x7f04b40abe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:01.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.360+0000 7f04b359e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04b40b7500 0x7f04b4153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:01.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.360+0000 7f04b359e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b4141f90 con 0x7f04b40aadb0 2022-08-01T19:34:01.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.360+0000 7f04b359e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b41545b0 con 0x7f04b40b7500 2022-08-01T19:34:01.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.360+0000 7f04b359e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04b40ad6b0 con 0x7f04b40b31f0 2022-08-01T19:34:01.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.360+0000 7f04b259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04b40aadb0 0x7f04b4141aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:01.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.360+0000 7f04b259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04b40aadb0 0x7f04b4141aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36040/0 (socket says 172.21.15.17:36040) 2022-08-01T19:34:01.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.360+0000 7f04b259c700 1 -- 172.21.15.17:0/2117607519 learned_addr learned my addr 172.21.15.17:0/2117607519 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:01.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.360+0000 7f04b259c700 1 -- 172.21.15.17:0/2117607519 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f04b40b31f0 msgr2=0x7f04b40abe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:01.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.360+0000 7f04b259c700 1 --2- 172.21.15.17:0/2117607519 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f04b40b31f0 0x7f04b40abe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-08-01T19:34:01.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.360+0000 7f04b259c700 1 -- 172.21.15.17:0/2117607519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04b40b7500 msgr2=0x7f04b4153be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:01.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.360+0000 7f04b2d9d700 1 --2- 172.21.15.17:0/2117607519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04b40b7500 0x7f04b4153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:01.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.360+0000 7f04b259c700 1 --2- 172.21.15.17:0/2117607519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04b40b7500 0x7f04b4153be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:01.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.360+0000 7f04b259c700 1 -- 172.21.15.17:0/2117607519 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f049c00b180 con 0x7f04b40aadb0 2022-08-01T19:34:01.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.360+0000 7f04b2d9d700 1 --2- 172.21.15.17:0/2117607519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04b40b7500 0x7f04b4153be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:34:01.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.360+0000 7f04b259c700 1 --2- 172.21.15.17:0/2117607519 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04b40aadb0 0x7f04b4141aa0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f04a4001160 tx=0x7f04a4007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:01.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.360+0000 7f04abfff700 1 -- 172.21.15.17:0/2117607519 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04a400ce10 con 0x7f04b40aadb0 2022-08-01T19:34:01.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.360+0000 7f04abfff700 1 -- 172.21.15.17:0/2117607519 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a400f470 con 0x7f04b40aadb0 2022-08-01T19:34:01.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.360+0000 7f04b359e700 1 -- 172.21.15.17:0/2117607519 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04b40ad950 con 0x7f04b40aadb0 2022-08-01T19:34:01.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.360+0000 7f04abfff700 1 -- 172.21.15.17:0/2117607519 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04a400f7d0 con 0x7f04b40aadb0 2022-08-01T19:34:01.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.360+0000 7f04b359e700 1 -- 172.21.15.17:0/2117607519 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04b4143530 con 0x7f04b40aadb0 2022-08-01T19:34:01.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.364+0000 7f04abfff700 1 -- 172.21.15.17:0/2117607519 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f04a4019030 con 0x7f04b40aadb0 2022-08-01T19:34:01.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.364+0000 7f04b359e700 1 -- 172.21.15.17:0/2117607519 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04b40adae0 con 0x7f04b40aadb0 2022-08-01T19:34:01.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.364+0000 7f04abfff700 1 --2- 172.21.15.17:0/2117607519 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f048c0539e0 0x7f048c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:01.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.364+0000 7f04abfff700 1 -- 172.21.15.17:0/2117607519 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4900+0+0 (secure 0 0 0) 0x7f04a406afa0 con 0x7f04b40aadb0 2022-08-01T19:34:01.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.364+0000 7f04a9d9b700 1 --2- 172.21.15.17:0/2117607519 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f048c0539e0 0x7f048c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:01.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.364+0000 7f04a9d9b700 1 --2- 172.21.15.17:0/2117607519 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f048c0539e0 0x7f048c055ea0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f049c002f70 tx=0x7f049c014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:01.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.372+0000 7f04abfff700 1 -- 172.21.15.17:0/2117607519 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f04a40422b0 con 0x7f04b40aadb0 2022-08-01T19:34:01.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:01.496+0000 7f04b359e700 1 -- 172.21.15.17:0/2117607519 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f04b40574e0 con 0x7f04b40aadb0 2022-08-01T19:34:02.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.224+0000 7f04abfff700 1 -- 172.21.15.17:0/2117607519 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v17) v1 ==== 109+0+0 (secure 0 0 0) 0x7f04a40458d0 con 0x7f04b40aadb0 2022-08-01T19:34:02.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.228+0000 7f04b359e700 1 -- 172.21.15.17:0/2117607519 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f048c0539e0 msgr2=0x7f048c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:02.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.228+0000 7f04b359e700 1 --2- 172.21.15.17:0/2117607519 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f048c0539e0 0x7f048c055ea0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f049c002f70 tx=0x7f049c014000).stop 2022-08-01T19:34:02.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.228+0000 7f04b359e700 1 -- 172.21.15.17:0/2117607519 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04b40aadb0 msgr2=0x7f04b4141aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:02.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.228+0000 7f04b359e700 1 --2- 172.21.15.17:0/2117607519 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04b40aadb0 0x7f04b4141aa0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f04a4001160 tx=0x7f04a4007ad0).stop 2022-08-01T19:34:02.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.228+0000 7f04b359e700 1 -- 172.21.15.17:0/2117607519 shutdown_connections 2022-08-01T19:34:02.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.228+0000 7f04b359e700 1 --2- 172.21.15.17:0/2117607519 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f048c0539e0 0x7f048c055ea0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:02.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.228+0000 7f04b359e700 1 --2- 172.21.15.17:0/2117607519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04b40b7500 0x7f04b4153be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:02.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.228+0000 7f04b359e700 1 --2- 172.21.15.17:0/2117607519 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f04b40b31f0 0x7f04b40abe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-08-01T19:34:02.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.228+0000 7f04b359e700 1 --2- 172.21.15.17:0/2117607519 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04b40aadb0 0x7f04b4141aa0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:02.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.228+0000 7f04b359e700 1 -- 172.21.15.17:0/2117607519 >> 172.21.15.17:0/2117607519 conn(0x7f04b405f840 msgr2=0x7f04b4147b30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:02.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.228+0000 7f04b359e700 1 -- 172.21.15.17:0/2117607519 shutdown_connections 2022-08-01T19:34:02.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.228+0000 7f04b359e700 1 -- 172.21.15.17:0/2117607519 wait complete. 2022-08-01T19:34:02.233 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'rbd' created 2022-08-01T19:34:02.244 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2022-08-01T19:34:02.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.348+0000 7f3b44bd9700 1 Processor -- start 2022-08-01T19:34:02.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.348+0000 7f3b44bd9700 1 -- start start 2022-08-01T19:34:02.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.348+0000 7f3b44bd9700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f3b400b2340 0x7f3b4005f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:02.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.348+0000 7f3b44bd9700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3b40143e20 0x7f3b4012c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:02.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.348+0000 7f3b44bd9700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b400b3dd0 con 0x7f3b40060eb0 2022-08-01T19:34:02.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.348+0000 7f3b44bd9700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b40063f80 con 0x7f3b400b2340 2022-08-01T19:34:02.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.348+0000 7f3b44bd9700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b400b2720 con 0x7f3b40143e20 2022-08-01T19:34:02.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.348+0000 7f3b3effd700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f3b40060eb0 0x7f3b400b36c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56474/0 (socket says 172.21.15.17:56474) 2022-08-01T19:34:02.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.348+0000 7f3b3effd700 1 -- 172.21.15.17:0/4158439662 learned_addr learned my addr 172.21.15.17:0/4158439662 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:02.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.348+0000 7f3b3f7fe700 1 --2- 172.21.15.17:0/4158439662 >> v2:172.21.15.77:3300/0 conn(0x7f3b400b2340 0x7f3b4005f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:02.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.348+0000 7f3b3ffff700 1 --2- 172.21.15.17:0/4158439662 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3b40143e20 0x7f3b4012c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:02.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.348+0000 7f3b3e7fc700 1 -- 172.21.15.17:0/4158439662 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2400626144 0 0) 0x7f3b400b3dd0 con 0x7f3b40060eb0 2022-08-01T19:34:02.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.348+0000 7f3b3e7fc700 1 -- 172.21.15.17:0/4158439662 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b20003260 con 0x7f3b40060eb0 2022-08-01T19:34:02.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.348+0000 7f3b3f7fe700 1 -- 172.21.15.17:0/4158439662 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3b40143e20 msgr2=0x7f3b4012c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:02.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.348+0000 7f3b3f7fe700 1 --2- 172.21.15.17:0/4158439662 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3b40143e20 0x7f3b4012c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:02.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.348+0000 7f3b3f7fe700 1 -- 172.21.15.17:0/4158439662 >> v1:172.21.15.17:6789/0 conn(0x7f3b40060eb0 legacy=0x7f3b400b36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:02.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.348+0000 7f3b3f7fe700 1 -- 172.21.15.17:0/4158439662 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b400b2fc0 con 0x7f3b400b2340 2022-08-01T19:34:02.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.352+0000 7f3b3f7fe700 1 --2- 172.21.15.17:0/4158439662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b400b2340 0x7f3b4005f840 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f3b300032f0 tx=0x7f3b300091d0).ready entity=mon.1 client_cookie=827e24ce454e5ef1 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:02.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.352+0000 7f3b3e7fc700 1 -- 172.21.15.17:0/4158439662 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b3000b600 con 0x7f3b400b2340 2022-08-01T19:34:02.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.352+0000 7f3b3e7fc700 1 -- 172.21.15.17:0/4158439662 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b3000b760 con 0x7f3b400b2340 2022-08-01T19:34:02.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.352+0000 7f3b3e7fc700 1 -- 172.21.15.17:0/4158439662 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b3000b8d0 con 0x7f3b400b2340 2022-08-01T19:34:02.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.352+0000 7f3b44bd9700 1 -- 172.21.15.17:0/4158439662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b400b2340 msgr2=0x7f3b4005f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:02.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.352+0000 7f3b44bd9700 1 --2- 172.21.15.17:0/4158439662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b400b2340 0x7f3b4005f840 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f3b300032f0 tx=0x7f3b300091d0).stop 2022-08-01T19:34:02.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.352+0000 7f3b44bd9700 1 -- 172.21.15.17:0/4158439662 shutdown_connections 2022-08-01T19:34:02.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.352+0000 7f3b44bd9700 1 --2- 172.21.15.17:0/4158439662 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3b40143e20 0x7f3b4012c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:02.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.352+0000 7f3b44bd9700 1 --2- 172.21.15.17:0/4158439662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b400b2340 0x7f3b4005f840 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:02.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.352+0000 7f3b44bd9700 1 -- 172.21.15.17:0/4158439662 >> 172.21.15.17:0/4158439662 conn(0x7f3b4005c620 msgr2=0x7f3b4005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:02.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.352+0000 7f3b44bd9700 1 -- 172.21.15.17:0/4158439662 shutdown_connections 2022-08-01T19:34:02.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.352+0000 7f3b44bd9700 1 -- 172.21.15.17:0/4158439662 wait complete. 2022-08-01T19:34:02.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.352+0000 7f3b44bd9700 1 Processor -- start 2022-08-01T19:34:02.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.352+0000 7f3b44bd9700 1 -- start start 2022-08-01T19:34:02.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.356+0000 7f3b44bd9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b40060eb0 0x7f3b400b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:02.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.356+0000 7f3b44bd9700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3b400b2340 0x7f3b400ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:02.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.356+0000 7f3b44bd9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b40143e20 0x7f3b400ae660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:02.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.356+0000 7f3b44bd9700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b40066d30 con 0x7f3b40060eb0 2022-08-01T19:34:02.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.356+0000 7f3b44bd9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b40144850 con 0x7f3b40143e20 2022-08-01T19:34:02.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.356+0000 7f3b44bd9700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b40146630 con 0x7f3b400b2340 2022-08-01T19:34:02.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.356+0000 7f3b3f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b40060eb0 0x7f3b400b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:02.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.356+0000 7f3b3f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b40060eb0 0x7f3b400b5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36054/0 (socket says 172.21.15.17:36054) 2022-08-01T19:34:02.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.356+0000 7f3b3f7fe700 1 -- 172.21.15.17:0/644320770 learned_addr learned my addr 172.21.15.17:0/644320770 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:02.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.356+0000 7f3b3effd700 1 --2- 172.21.15.17:0/644320770 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3b400b2340 0x7f3b400ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:02.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.356+0000 7f3b3ffff700 1 --2- 172.21.15.17:0/644320770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b40143e20 0x7f3b400ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:02.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.356+0000 7f3b3f7fe700 1 -- 172.21.15.17:0/644320770 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3b400b2340 msgr2=0x7f3b400ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:02.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.356+0000 7f3b3f7fe700 1 --2- 172.21.15.17:0/644320770 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3b400b2340 0x7f3b400ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:02.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.356+0000 7f3b3f7fe700 1 -- 172.21.15.17:0/644320770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b40143e20 msgr2=0x7f3b400ae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:02.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.356+0000 7f3b3f7fe700 1 --2- 172.21.15.17:0/644320770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b40143e20 0x7f3b400ae660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:02.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.356+0000 7f3b3f7fe700 1 -- 172.21.15.17:0/644320770 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b30009470 con 0x7f3b40060eb0 2022-08-01T19:34:02.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.356+0000 7f3b3f7fe700 1 --2- 172.21.15.17:0/644320770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b40060eb0 0x7f3b400b5c20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f3b300032f0 tx=0x7f3b30016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:02.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.356+0000 7f3b3cff9700 1 -- 172.21.15.17:0/644320770 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b3000b8d0 con 0x7f3b40060eb0 2022-08-01T19:34:02.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.356+0000 7f3b3cff9700 1 -- 172.21.15.17:0/644320770 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b30021670 con 0x7f3b40060eb0 2022-08-01T19:34:02.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.356+0000 7f3b44bd9700 1 -- 172.21.15.17:0/644320770 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b401468d0 con 0x7f3b40060eb0 2022-08-01T19:34:02.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.356+0000 7f3b3cff9700 1 -- 172.21.15.17:0/644320770 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b3000b600 con 0x7f3b40060eb0 2022-08-01T19:34:02.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.356+0000 7f3b44bd9700 1 -- 172.21.15.17:0/644320770 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b400aeb50 con 0x7f3b40060eb0 2022-08-01T19:34:02.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.356+0000 7f3b44bd9700 1 -- 172.21.15.17:0/644320770 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b40144a50 con 0x7f3b40060eb0 2022-08-01T19:34:02.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.360+0000 7f3b3cff9700 1 -- 172.21.15.17:0/644320770 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3b3000f070 con 0x7f3b40060eb0 2022-08-01T19:34:02.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.360+0000 7f3b3cff9700 1 --2- 172.21.15.17:0/644320770 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3b24053a10 0x7f3b24055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:02.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.360+0000 7f3b3cff9700 1 -- 172.21.15.17:0/644320770 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5242+0+0 (secure 0 0 0) 0x7f3b3006d0b0 con 0x7f3b40060eb0 2022-08-01T19:34:02.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.360+0000 7f3b3effd700 1 --2- 172.21.15.17:0/644320770 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3b24053a10 0x7f3b24055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:02.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.360+0000 7f3b3effd700 1 --2- 172.21.15.17:0/644320770 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3b24053a10 0x7f3b24055ed0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f3b2800d000 tx=0x7f3b28009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:02.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.368+0000 7f3b3cff9700 1 -- 172.21.15.17:0/644320770 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3b30044180 con 0x7f3b40060eb0 2022-08-01T19:34:02.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:02.496+0000 7f3b44bd9700 1 -- 172.21.15.17:0/644320770 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3b40061d60 con 0x7f3b40060eb0 2022-08-01T19:34:03.234 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.232+0000 7f3b3cff9700 1 -- 172.21.15.17:0/644320770 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v18) v1 ==== 173+0+0 (secure 0 0 0) 0x7f3b30015020 con 0x7f3b40060eb0 2022-08-01T19:34:03.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.236+0000 7f3b44bd9700 1 -- 172.21.15.17:0/644320770 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3b24053a10 msgr2=0x7f3b24055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:03.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.236+0000 7f3b44bd9700 1 --2- 172.21.15.17:0/644320770 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3b24053a10 0x7f3b24055ed0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f3b2800d000 tx=0x7f3b28009040).stop 2022-08-01T19:34:03.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.236+0000 7f3b44bd9700 1 -- 172.21.15.17:0/644320770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b40060eb0 msgr2=0x7f3b400b5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:03.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.236+0000 7f3b44bd9700 1 --2- 172.21.15.17:0/644320770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b40060eb0 0x7f3b400b5c20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f3b300032f0 tx=0x7f3b30016000).stop 2022-08-01T19:34:03.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.236+0000 7f3b44bd9700 1 -- 172.21.15.17:0/644320770 shutdown_connections 2022-08-01T19:34:03.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.236+0000 7f3b44bd9700 1 --2- 172.21.15.17:0/644320770 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3b24053a10 0x7f3b24055ed0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:03.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.236+0000 7f3b44bd9700 1 --2- 172.21.15.17:0/644320770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b40143e20 0x7f3b400ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:03.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.236+0000 7f3b44bd9700 1 --2- 172.21.15.17:0/644320770 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3b400b2340 0x7f3b400ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:03.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.236+0000 7f3b44bd9700 1 --2- 172.21.15.17:0/644320770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b40060eb0 0x7f3b400b5c20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:03.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.236+0000 7f3b44bd9700 1 -- 172.21.15.17:0/644320770 >> 172.21.15.17:0/644320770 conn(0x7f3b4005c620 msgr2=0x7f3b40147b60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:03.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.236+0000 7f3b44bd9700 1 -- 172.21.15.17:0/644320770 shutdown_connections 2022-08-01T19:34:03.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.236+0000 7f3b44bd9700 1 -- 172.21.15.17:0/644320770 wait complete. 2022-08-01T19:34:03.243 INFO:teuthology.orchestra.run.smithi017.stderr:enabled application 'rbd' on pool 'rbd' 2022-08-01T19:34:03.254 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2022-08-01T19:34:03.255 INFO:tasks.ceph.mds.a:Restarting daemon 2022-08-01T19:34:03.256 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-08-01T19:34:03.258 INFO:tasks.ceph.mds.a:Started 2022-08-01T19:34:03.258 INFO:tasks.ceph.mds.c:Restarting daemon 2022-08-01T19:34:03.259 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-08-01T19:34:03.261 INFO:tasks.ceph.mds.c:Started 2022-08-01T19:34:03.261 INFO:tasks.ceph.mds.b:Restarting daemon 2022-08-01T19:34:03.262 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-08-01T19:34:03.264 INFO:tasks.ceph.mds.b:Started 2022-08-01T19:34:03.265 INFO:tasks.ceph:Setting up CephFS filesystem... 2022-08-01T19:34:03.265 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:34:03.323 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2022-08-01T19:34:03.328 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2022-08-01T19:34:03.343 INFO:tasks.ceph.mds.b.smithi077.stdout:starting mds.b at 2022-08-01T19:34:03.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.360+0000 7f5d73b0a700 1 Processor -- start 2022-08-01T19:34:03.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.360+0000 7f5d73b0a700 1 -- start start 2022-08-01T19:34:03.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.360+0000 7f5d73b0a700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f5d6c139e50 0x7f5d6c0acd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:03.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.360+0000 7f5d73b0a700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5d6c13a230 0x7f5d6c0b6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:03.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.360+0000 7f5d73b0a700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d6c0ad200 con 0x7f5d6c0b2350 2022-08-01T19:34:03.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.360+0000 7f5d73b0a700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d6c0b28c0 con 0x7f5d6c13a230 2022-08-01T19:34:03.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.360+0000 7f5d73b0a700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d6c060ed0 con 0x7f5d6c139e50 2022-08-01T19:34:03.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.360+0000 7f5d710a5700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5d6c0b2350 0x7f5d6c143680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56516/0 (socket says 172.21.15.17:56516) 2022-08-01T19:34:03.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.360+0000 7f5d710a5700 1 -- 172.21.15.17:0/3249719075 learned_addr learned my addr 172.21.15.17:0/3249719075 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:03.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.360+0000 7f5d720a7700 1 --2- 172.21.15.17:0/3249719075 >> v2:172.21.15.17:3300/0 conn(0x7f5d6c13a230 0x7f5d6c0b6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:03.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.360+0000 7f5d718a6700 1 --2- 172.21.15.17:0/3249719075 >> v2:172.21.15.77:3300/0 conn(0x7f5d6c139e50 0x7f5d6c0acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:03.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.360+0000 7f5d708a4700 1 -- 172.21.15.17:0/3249719075 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3565356641 0 0) 0x7f5d5c001110 con 0x7f5d6c0b2350 2022-08-01T19:34:03.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.360+0000 7f5d708a4700 1 -- 172.21.15.17:0/3249719075 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d54003260 con 0x7f5d6c0b2350 2022-08-01T19:34:03.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.360+0000 7f5d720a7700 1 -- 172.21.15.17:0/3249719075 >> v2:172.21.15.77:3300/0 conn(0x7f5d6c139e50 msgr2=0x7f5d6c0acd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:03.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.360+0000 7f5d720a7700 1 --2- 172.21.15.17:0/3249719075 >> v2:172.21.15.77:3300/0 conn(0x7f5d6c139e50 0x7f5d6c0acd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:03.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.360+0000 7f5d720a7700 1 -- 172.21.15.17:0/3249719075 >> v1:172.21.15.17:6789/0 conn(0x7f5d6c0b2350 legacy=0x7f5d6c143680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:03.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.360+0000 7f5d720a7700 1 -- 172.21.15.17:0/3249719075 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d6c061130 con 0x7f5d6c13a230 2022-08-01T19:34:03.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.360+0000 7f5d718a6700 1 --2- 172.21.15.17:0/3249719075 >> v2:172.21.15.77:3300/0 conn(0x7f5d6c139e50 0x7f5d6c0acd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:34:03.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.360+0000 7f5d720a7700 1 --2- 172.21.15.17:0/3249719075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d6c13a230 0x7f5d6c0b6750 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f5d640032f0 tx=0x7f5d640091d0).ready entity=mon.0 client_cookie=1e70a9a7f3626370 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:03.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.364+0000 7f5d708a4700 1 -- 172.21.15.17:0/3249719075 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d6400f8a0 con 0x7f5d6c13a230 2022-08-01T19:34:03.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.364+0000 7f5d708a4700 1 -- 172.21.15.17:0/3249719075 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d6400fa00 con 0x7f5d6c13a230 2022-08-01T19:34:03.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.364+0000 7f5d708a4700 1 -- 172.21.15.17:0/3249719075 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d6400fd60 con 0x7f5d6c13a230 2022-08-01T19:34:03.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.364+0000 7f5d73b0a700 1 -- 172.21.15.17:0/3249719075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d6c13a230 msgr2=0x7f5d6c0b6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:03.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.364+0000 7f5d73b0a700 1 --2- 172.21.15.17:0/3249719075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d6c13a230 0x7f5d6c0b6750 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f5d640032f0 tx=0x7f5d640091d0).stop 2022-08-01T19:34:03.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.364+0000 7f5d73b0a700 1 -- 172.21.15.17:0/3249719075 shutdown_connections 2022-08-01T19:34:03.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.364+0000 7f5d73b0a700 1 --2- 172.21.15.17:0/3249719075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d6c13a230 0x7f5d6c0b6750 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:03.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.364+0000 7f5d73b0a700 1 --2- 172.21.15.17:0/3249719075 >> v2:172.21.15.77:3300/0 conn(0x7f5d6c139e50 0x7f5d6c0acd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:03.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.364+0000 7f5d73b0a700 1 -- 172.21.15.17:0/3249719075 >> 172.21.15.17:0/3249719075 conn(0x7f5d6c0b3610 msgr2=0x7f5d6c0b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:03.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.364+0000 7f5d73b0a700 1 -- 172.21.15.17:0/3249719075 shutdown_connections 2022-08-01T19:34:03.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.364+0000 7f5d73b0a700 1 -- 172.21.15.17:0/3249719075 wait complete. 2022-08-01T19:34:03.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.364+0000 7f5d73b0a700 1 Processor -- start 2022-08-01T19:34:03.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.364+0000 7f5d73b0a700 1 -- start start 2022-08-01T19:34:03.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.368+0000 7f5d73b0a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d6c0b2350 0x7f5d6c1565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:03.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.368+0000 7f5d73b0a700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d6c139e50 0x7f5d6c1436b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:03.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.368+0000 7f5d73b0a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d6c13a230 0x7f5d6c145a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:03.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.368+0000 7f5d73b0a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c0ad990 con 0x7f5d6c0b2350 2022-08-01T19:34:03.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.368+0000 7f5d73b0a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c1463f0 con 0x7f5d6c13a230 2022-08-01T19:34:03.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.368+0000 7f5d73b0a700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c140b60 con 0x7f5d6c139e50 2022-08-01T19:34:03.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.368+0000 7f5d718a6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d6c0b2350 0x7f5d6c1565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:03.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.368+0000 7f5d718a6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d6c0b2350 0x7f5d6c1565c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36094/0 (socket says 172.21.15.17:36094) 2022-08-01T19:34:03.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.368+0000 7f5d718a6700 1 -- 172.21.15.17:0/900833085 learned_addr learned my addr 172.21.15.17:0/900833085 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:03.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.368+0000 7f5d710a5700 1 --2- 172.21.15.17:0/900833085 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d6c139e50 0x7f5d6c1436b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:03.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.368+0000 7f5d720a7700 1 --2- 172.21.15.17:0/900833085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d6c13a230 0x7f5d6c145a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:03.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.368+0000 7f5d718a6700 1 -- 172.21.15.17:0/900833085 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d6c139e50 msgr2=0x7f5d6c1436b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:03.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.368+0000 7f5d718a6700 1 --2- 172.21.15.17:0/900833085 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d6c139e50 0x7f5d6c1436b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:03.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.368+0000 7f5d718a6700 1 -- 172.21.15.17:0/900833085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d6c13a230 msgr2=0x7f5d6c145a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:03.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.368+0000 7f5d718a6700 1 --2- 172.21.15.17:0/900833085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d6c13a230 0x7f5d6c145a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:03.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.368+0000 7f5d718a6700 1 -- 172.21.15.17:0/900833085 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d6800b180 con 0x7f5d6c0b2350 2022-08-01T19:34:03.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.368+0000 7f5d720a7700 1 --2- 172.21.15.17:0/900833085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d6c13a230 0x7f5d6c145a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:34:03.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.368+0000 7f5d718a6700 1 --2- 172.21.15.17:0/900833085 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d6c0b2350 0x7f5d6c1565c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f5d6800c000 tx=0x7f5d68006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:03.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.368+0000 7f5d5affd700 1 -- 172.21.15.17:0/900833085 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d6800c890 con 0x7f5d6c0b2350 2022-08-01T19:34:03.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.368+0000 7f5d5affd700 1 -- 172.21.15.17:0/900833085 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d6800c9f0 con 0x7f5d6c0b2350 2022-08-01T19:34:03.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.368+0000 7f5d73b0a700 1 -- 172.21.15.17:0/900833085 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d64009470 con 0x7f5d6c0b2350 2022-08-01T19:34:03.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.368+0000 7f5d5affd700 1 -- 172.21.15.17:0/900833085 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d6800cd50 con 0x7f5d6c0b2350 2022-08-01T19:34:03.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.368+0000 7f5d73b0a700 1 -- 172.21.15.17:0/900833085 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d6c0ad4f0 con 0x7f5d6c0b2350 2022-08-01T19:34:03.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.372+0000 7f5d73b0a700 1 -- 172.21.15.17:0/900833085 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d6c0ad6f0 con 0x7f5d6c0b2350 2022-08-01T19:34:03.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.372+0000 7f5d5affd700 1 -- 172.21.15.17:0/900833085 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5d6801c460 con 0x7f5d6c0b2350 2022-08-01T19:34:03.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.372+0000 7f5d5affd700 1 --2- 172.21.15.17:0/900833085 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5d4c053a70 0x7f5d4c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:03.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.372+0000 7f5d5affd700 1 -- 172.21.15.17:0/900833085 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f5d6806c230 con 0x7f5d6c0b2350 2022-08-01T19:34:03.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.372+0000 7f5d710a5700 1 --2- 172.21.15.17:0/900833085 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5d4c053a70 0x7f5d4c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:03.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.372+0000 7f5d710a5700 1 --2- 172.21.15.17:0/900833085 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5d4c053a70 0x7f5d4c055f30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f5d5c00d000 tx=0x7f5d5c0078a0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:03.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.376+0000 7f5d5affd700 1 -- 172.21.15.17:0/900833085 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5d6c0ad6f0 con 0x7f5d6c0b2350 2022-08-01T19:34:03.502 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.500+0000 7f5d73b0a700 1 -- 172.21.15.17:0/900833085 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d6c0ad6f0 con 0x7f5d6c0b2350 2022-08-01T19:34:03.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.500+0000 7f5d5affd700 1 -- 172.21.15.17:0/900833085 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+13191 (secure 0 0 0) 0x7f5d6c0ad6f0 con 0x7f5d6c0b2350 2022-08-01T19:34:03.504 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:03.505 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":18,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:34:03.222263+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:34:03.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.504+0000 7f5d73b0a700 1 -- 172.21.15.17:0/900833085 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5d4c053a70 msgr2=0x7f5d4c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:03.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.504+0000 7f5d73b0a700 1 --2- 172.21.15.17:0/900833085 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5d4c053a70 0x7f5d4c055f30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f5d5c00d000 tx=0x7f5d5c0078a0).stop 2022-08-01T19:34:03.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.504+0000 7f5d73b0a700 1 -- 172.21.15.17:0/900833085 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d6c0b2350 msgr2=0x7f5d6c1565c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:03.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.504+0000 7f5d73b0a700 1 --2- 172.21.15.17:0/900833085 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d6c0b2350 0x7f5d6c1565c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f5d6800c000 tx=0x7f5d68006df0).stop 2022-08-01T19:34:03.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.504+0000 7f5d73b0a700 1 -- 172.21.15.17:0/900833085 shutdown_connections 2022-08-01T19:34:03.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.504+0000 7f5d73b0a700 1 --2- 172.21.15.17:0/900833085 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5d4c053a70 0x7f5d4c055f30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:03.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.504+0000 7f5d73b0a700 1 --2- 172.21.15.17:0/900833085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d6c13a230 0x7f5d6c145a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:03.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.504+0000 7f5d73b0a700 1 --2- 172.21.15.17:0/900833085 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d6c139e50 0x7f5d6c1436b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:03.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.504+0000 7f5d73b0a700 1 --2- 172.21.15.17:0/900833085 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d6c0b2350 0x7f5d6c1565c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:03.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.504+0000 7f5d73b0a700 1 -- 172.21.15.17:0/900833085 >> 172.21.15.17:0/900833085 conn(0x7f5d6c0b3610 msgr2=0x7f5d6c1493b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:03.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.504+0000 7f5d73b0a700 1 -- 172.21.15.17:0/900833085 shutdown_connections 2022-08-01T19:34:03.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.504+0000 7f5d73b0a700 1 -- 172.21.15.17:0/900833085 wait complete. 2022-08-01T19:34:03.523 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-08-01T19:34:03.523 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:34:03.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.632+0000 7fc97c6b7700 1 Processor -- start 2022-08-01T19:34:03.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.632+0000 7fc97c6b7700 1 -- start start 2022-08-01T19:34:03.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.632+0000 7fc97c6b7700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9740b4310 0x7fc9740b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:03.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.632+0000 7fc97c6b7700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9740b5fd0 con 0x7fc9740b3980 2022-08-01T19:34:03.648 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.632+0000 7fc97c6b7700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc974148630 con 0x7fc974144830 2022-08-01T19:34:03.648 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.632+0000 7fc97c6b7700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9740b3d60 con 0x7fc9740b4310 2022-08-01T19:34:03.649 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.632+0000 7fc97a453700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc9740b3980 0x7fc974147f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56530/0 (socket says 172.21.15.17:56530) 2022-08-01T19:34:03.649 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.632+0000 7fc97a453700 1 -- 172.21.15.17:0/2217289627 learned_addr learned my addr 172.21.15.17:0/2217289627 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:03.650 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.632+0000 7fc97ac54700 1 --2- 172.21.15.17:0/2217289627 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9740b4310 0x7fc9740b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:03.650 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.632+0000 7fc979451700 1 -- 172.21.15.17:0/2217289627 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3332511969 0 0) 0x7fc9740b5fd0 con 0x7fc9740b3980 2022-08-01T19:34:03.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.632+0000 7fc979451700 1 -- 172.21.15.17:0/2217289627 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc95c003260 con 0x7fc9740b3980 2022-08-01T19:34:03.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.632+0000 7fc97ac54700 1 -- 172.21.15.17:0/2217289627 >> v1:172.21.15.77:6789/0 conn(0x7fc974144830 legacy=0x7fc974144c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:03.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.632+0000 7fc979451700 1 -- 172.21.15.17:0/2217289627 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1982210108 0 0) 0x7fc964001110 con 0x7fc974144830 2022-08-01T19:34:03.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.632+0000 7fc97ac54700 1 -- 172.21.15.17:0/2217289627 >> v1:172.21.15.17:6789/0 conn(0x7fc9740b3980 legacy=0x7fc974147f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:03.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.632+0000 7fc97ac54700 1 -- 172.21.15.17:0/2217289627 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9740b6200 con 0x7fc9740b4310 2022-08-01T19:34:03.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.632+0000 7fc97ac54700 1 --2- 172.21.15.17:0/2217289627 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9740b4310 0x7fc9740b47a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7fc96c0032f0 tx=0x7fc96c0091d0).ready entity=mon.2 client_cookie=785eb210c07fefa1 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:03.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.636+0000 7fc979451700 1 -- 172.21.15.17:0/2217289627 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc96c00b6c0 con 0x7fc9740b4310 2022-08-01T19:34:03.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.636+0000 7fc979451700 1 -- 172.21.15.17:0/2217289627 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc96c00b820 con 0x7fc9740b4310 2022-08-01T19:34:03.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.636+0000 7fc979451700 1 -- 172.21.15.17:0/2217289627 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc96c00b990 con 0x7fc9740b4310 2022-08-01T19:34:03.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.636+0000 7fc97c6b7700 1 -- 172.21.15.17:0/2217289627 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9740b4310 msgr2=0x7fc9740b47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:03.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.636+0000 7fc97c6b7700 1 --2- 172.21.15.17:0/2217289627 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9740b4310 0x7fc9740b47a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7fc96c0032f0 tx=0x7fc96c0091d0).stop 2022-08-01T19:34:03.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.636+0000 7fc97c6b7700 1 -- 172.21.15.17:0/2217289627 shutdown_connections 2022-08-01T19:34:03.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.636+0000 7fc97c6b7700 1 --2- 172.21.15.17:0/2217289627 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9740b4310 0x7fc9740b47a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:03.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.636+0000 7fc97c6b7700 1 -- 172.21.15.17:0/2217289627 >> 172.21.15.17:0/2217289627 conn(0x7fc97405c620 msgr2=0x7fc97405ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:03.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.636+0000 7fc97c6b7700 1 -- 172.21.15.17:0/2217289627 shutdown_connections 2022-08-01T19:34:03.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.636+0000 7fc97c6b7700 1 -- 172.21.15.17:0/2217289627 wait complete. 2022-08-01T19:34:03.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.636+0000 7fc97c6b7700 1 Processor -- start 2022-08-01T19:34:03.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.636+0000 7fc97c6b7700 1 -- start start 2022-08-01T19:34:03.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.636+0000 7fc97c6b7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9740b3980 0x7fc97414b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:03.660 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.636+0000 7fc97c6b7700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9740b4310 0x7fc974065d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:03.660 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.636+0000 7fc97c6b7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc974144830 0x7fc974066210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:03.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.636+0000 7fc97c6b7700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9741574b0 con 0x7fc9740b3980 2022-08-01T19:34:03.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.636+0000 7fc97c6b7700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc974157720 con 0x7fc974144830 2022-08-01T19:34:03.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.636+0000 7fc97c6b7700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9741579c0 con 0x7fc9740b4310 2022-08-01T19:34:03.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.636+0000 7fc97a453700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9740b3980 0x7fc97414b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:03.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.636+0000 7fc97a453700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9740b3980 0x7fc97414b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36108/0 (socket says 172.21.15.17:36108) 2022-08-01T19:34:03.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.636+0000 7fc97a453700 1 -- 172.21.15.17:0/1545764730 learned_addr learned my addr 172.21.15.17:0/1545764730 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:03.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.636+0000 7fc979c52700 1 --2- 172.21.15.17:0/1545764730 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9740b4310 0x7fc974065d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:03.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.636+0000 7fc97ac54700 1 --2- 172.21.15.17:0/1545764730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc974144830 0x7fc974066210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:03.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.636+0000 7fc97a453700 1 -- 172.21.15.17:0/1545764730 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9740b4310 msgr2=0x7fc974065d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:03.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.636+0000 7fc97a453700 1 --2- 172.21.15.17:0/1545764730 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9740b4310 0x7fc974065d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:03.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.636+0000 7fc97a453700 1 -- 172.21.15.17:0/1545764730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc974144830 msgr2=0x7fc974066210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:03.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.636+0000 7fc97a453700 1 --2- 172.21.15.17:0/1545764730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc974144830 0x7fc974066210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:03.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.640+0000 7fc97a453700 1 -- 172.21.15.17:0/1545764730 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc96c009470 con 0x7fc9740b3980 2022-08-01T19:34:03.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.640+0000 7fc97a453700 1 --2- 172.21.15.17:0/1545764730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9740b3980 0x7fc97414b500 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fc970001110 tx=0x7fc970007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:03.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.640+0000 7fc9637fe700 1 -- 172.21.15.17:0/1545764730 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9700070d0 con 0x7fc9740b3980 2022-08-01T19:34:03.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.640+0000 7fc9637fe700 1 -- 172.21.15.17:0/1545764730 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc970007230 con 0x7fc9740b3980 2022-08-01T19:34:03.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.640+0000 7fc9637fe700 1 -- 172.21.15.17:0/1545764730 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc970005610 con 0x7fc9740b3980 2022-08-01T19:34:03.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.640+0000 7fc97c6b7700 1 -- 172.21.15.17:0/1545764730 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9740af830 con 0x7fc9740b3980 2022-08-01T19:34:03.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.640+0000 7fc97c6b7700 1 -- 172.21.15.17:0/1545764730 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9740b1950 con 0x7fc9740b3980 2022-08-01T19:34:03.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.640+0000 7fc9637fe700 1 -- 172.21.15.17:0/1545764730 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc970010070 con 0x7fc9740b3980 2022-08-01T19:34:03.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.640+0000 7fc97c6b7700 1 -- 172.21.15.17:0/1545764730 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9740afcd0 con 0x7fc9740b3980 2022-08-01T19:34:03.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.640+0000 7fc9637fe700 1 --2- 172.21.15.17:0/1545764730 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9540539e0 0x7fc954055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:03.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.640+0000 7fc9637fe700 1 -- 172.21.15.17:0/1545764730 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fc97006b330 con 0x7fc9740b3980 2022-08-01T19:34:03.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.640+0000 7fc979c52700 1 --2- 172.21.15.17:0/1545764730 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9540539e0 0x7fc954055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:03.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.644+0000 7fc979c52700 1 --2- 172.21.15.17:0/1545764730 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9540539e0 0x7fc954055ea0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fc96400d000 tx=0x7fc964009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:03.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.648+0000 7fc9637fe700 1 -- 172.21.15.17:0/1545764730 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc9740afcd0 con 0x7fc9740b3980 2022-08-01T19:34:03.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.776+0000 7fc97c6b7700 1 -- 172.21.15.17:0/1545764730 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc9740afcd0 con 0x7fc9740b3980 2022-08-01T19:34:03.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.776+0000 7fc9637fe700 1 -- 172.21.15.17:0/1545764730 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v18) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc9740afcd0 con 0x7fc9740b3980 2022-08-01T19:34:03.779 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:34:03.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.780+0000 7fc97c6b7700 1 -- 172.21.15.17:0/1545764730 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9540539e0 msgr2=0x7fc954055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:03.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.780+0000 7fc97c6b7700 1 --2- 172.21.15.17:0/1545764730 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9540539e0 0x7fc954055ea0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fc96400d000 tx=0x7fc964009040).stop 2022-08-01T19:34:03.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.780+0000 7fc97c6b7700 1 -- 172.21.15.17:0/1545764730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9740b3980 msgr2=0x7fc97414b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:03.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.780+0000 7fc97c6b7700 1 --2- 172.21.15.17:0/1545764730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9740b3980 0x7fc97414b500 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fc970001110 tx=0x7fc970007ad0).stop 2022-08-01T19:34:03.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.780+0000 7fc97c6b7700 1 -- 172.21.15.17:0/1545764730 shutdown_connections 2022-08-01T19:34:03.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.780+0000 7fc97c6b7700 1 --2- 172.21.15.17:0/1545764730 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9540539e0 0x7fc954055ea0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:03.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.780+0000 7fc97c6b7700 1 --2- 172.21.15.17:0/1545764730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc974144830 0x7fc974066210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:03.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.780+0000 7fc97c6b7700 1 --2- 172.21.15.17:0/1545764730 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9740b4310 0x7fc974065d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:03.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.780+0000 7fc97c6b7700 1 --2- 172.21.15.17:0/1545764730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9740b3980 0x7fc97414b500 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:03.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.780+0000 7fc97c6b7700 1 -- 172.21.15.17:0/1545764730 >> 172.21.15.17:0/1545764730 conn(0x7fc97405c620 msgr2=0x7fc97414c500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:03.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.780+0000 7fc97c6b7700 1 -- 172.21.15.17:0/1545764730 shutdown_connections 2022-08-01T19:34:03.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.780+0000 7fc97c6b7700 1 -- 172.21.15.17:0/1545764730 wait complete. 2022-08-01T19:34:03.792 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:34:03.903 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.900+0000 7f798759e700 1 Processor -- start 2022-08-01T19:34:03.904 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.900+0000 7f798759e700 1 -- start start 2022-08-01T19:34:03.905 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.904+0000 7f798759e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79880a9320 0x7f79881466e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:03.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.904+0000 7f798759e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f79880b0d10 0x7f7988146bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:03.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.904+0000 7f798759e700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79880b12d0 con 0x7f79880b0560 2022-08-01T19:34:03.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.904+0000 7f798759e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79880a9700 con 0x7f79880b0d10 2022-08-01T19:34:03.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.904+0000 7f798759e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79880b1500 con 0x7f79880a9320 2022-08-01T19:34:03.907 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.904+0000 7f7985d9b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f79880b0d10 0x7f7988146bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:03.907 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.904+0000 7f7986d9d700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f79880b0560 0x7f7988140350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56546/0 (socket says 172.21.15.17:56546) 2022-08-01T19:34:03.907 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.904+0000 7f7986d9d700 1 -- 172.21.15.17:0/1734701521 learned_addr learned my addr 172.21.15.17:0/1734701521 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:03.907 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.904+0000 7f798659c700 1 --2- 172.21.15.17:0/1734701521 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79880a9320 0x7f79881466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:03.909 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.904+0000 7f798559a700 1 -- 172.21.15.17:0/1734701521 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3715680294 0 0) 0x7f79880b12d0 con 0x7f79880b0560 2022-08-01T19:34:03.910 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.904+0000 7f798559a700 1 -- 172.21.15.17:0/1734701521 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f796c003260 con 0x7f79880b0560 2022-08-01T19:34:03.910 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.904+0000 7f7985d9b700 1 -- 172.21.15.17:0/1734701521 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79880a9320 msgr2=0x7f79881466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:03.910 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.904+0000 7f7985d9b700 1 --2- 172.21.15.17:0/1734701521 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79880a9320 0x7f79881466e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:03.910 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.904+0000 7f7985d9b700 1 -- 172.21.15.17:0/1734701521 >> v1:172.21.15.17:6789/0 conn(0x7f79880b0560 legacy=0x7f7988140350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:03.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.904+0000 7f7985d9b700 1 -- 172.21.15.17:0/1734701521 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79880b17a0 con 0x7f79880b0d10 2022-08-01T19:34:03.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.904+0000 7f798659c700 1 --2- 172.21.15.17:0/1734701521 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79880a9320 0x7f79881466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:34:03.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.904+0000 7f7985d9b700 1 --2- 172.21.15.17:0/1734701521 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79880b0d10 0x7f7988146bd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f797c002f70 tx=0x7f797c006df0).ready entity=mon.0 client_cookie=7dcacd0fa3903fe7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:03.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.904+0000 7f798559a700 1 -- 172.21.15.17:0/1734701521 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f797c00d4a0 con 0x7f79880b0d10 2022-08-01T19:34:03.912 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.904+0000 7f798559a700 1 -- 172.21.15.17:0/1734701521 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797c000e90 con 0x7f79880b0d10 2022-08-01T19:34:03.912 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.904+0000 7f798559a700 1 -- 172.21.15.17:0/1734701521 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f797c00d7f0 con 0x7f79880b0d10 2022-08-01T19:34:03.912 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.904+0000 7f798759e700 1 -- 172.21.15.17:0/1734701521 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79880b0d10 msgr2=0x7f7988146bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:03.912 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.904+0000 7f798759e700 1 --2- 172.21.15.17:0/1734701521 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79880b0d10 0x7f7988146bd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f797c002f70 tx=0x7f797c006df0).stop 2022-08-01T19:34:03.913 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.904+0000 7f798759e700 1 -- 172.21.15.17:0/1734701521 shutdown_connections 2022-08-01T19:34:03.913 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.904+0000 7f798759e700 1 --2- 172.21.15.17:0/1734701521 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79880b0d10 0x7f7988146bd0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:03.913 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.904+0000 7f798759e700 1 --2- 172.21.15.17:0/1734701521 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79880a9320 0x7f79881466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:03.913 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.904+0000 7f798759e700 1 -- 172.21.15.17:0/1734701521 >> 172.21.15.17:0/1734701521 conn(0x7f79880612d0 msgr2=0x7f7988062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:03.913 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.904+0000 7f798759e700 1 -- 172.21.15.17:0/1734701521 shutdown_connections 2022-08-01T19:34:03.914 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.904+0000 7f798759e700 1 -- 172.21.15.17:0/1734701521 wait complete. 2022-08-01T19:34:03.914 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.908+0000 7f798759e700 1 Processor -- start 2022-08-01T19:34:03.914 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.908+0000 7f798759e700 1 -- start start 2022-08-01T19:34:03.917 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.908+0000 7f798759e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79880a9320 0x7f79880b4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:03.917 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.908+0000 7f798759e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79880b0560 0x7f7988142a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:03.917 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.908+0000 7f798759e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79880b0d10 0x7f7988142ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:03.918 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.908+0000 7f798759e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79880ae410 con 0x7f79880a9320 2022-08-01T19:34:03.918 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.908+0000 7f798759e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79880ae640 con 0x7f79880b0560 2022-08-01T19:34:03.918 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.908+0000 7f798759e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79880ae8e0 con 0x7f79880b0d10 2022-08-01T19:34:03.918 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.908+0000 7f798659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79880a9320 0x7f79880b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:03.919 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.908+0000 7f798659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79880a9320 0x7f79880b4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36122/0 (socket says 172.21.15.17:36122) 2022-08-01T19:34:03.919 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.908+0000 7f798659c700 1 -- 172.21.15.17:0/2954423508 learned_addr learned my addr 172.21.15.17:0/2954423508 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:03.919 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.908+0000 7f798659c700 1 -- 172.21.15.17:0/2954423508 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79880b0d10 msgr2=0x7f7988142ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:03.919 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.908+0000 7f798659c700 1 --2- 172.21.15.17:0/2954423508 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79880b0d10 0x7f7988142ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-08-01T19:34:03.919 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.908+0000 7f798659c700 1 -- 172.21.15.17:0/2954423508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79880b0560 msgr2=0x7f7988142a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:03.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.908+0000 7f798659c700 1 --2- 172.21.15.17:0/2954423508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79880b0560 0x7f7988142a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-08-01T19:34:03.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.908+0000 7f798659c700 1 -- 172.21.15.17:0/2954423508 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f797800b180 con 0x7f79880a9320 2022-08-01T19:34:03.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.908+0000 7f798659c700 1 --2- 172.21.15.17:0/2954423508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79880a9320 0x7f79880b4860 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f797800c000 tx=0x7f7978006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:03.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.908+0000 7f79777fe700 1 -- 172.21.15.17:0/2954423508 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f797800ca40 con 0x7f79880a9320 2022-08-01T19:34:03.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.908+0000 7f79777fe700 1 -- 172.21.15.17:0/2954423508 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797800cba0 con 0x7f79880a9320 2022-08-01T19:34:03.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.908+0000 7f798759e700 1 -- 172.21.15.17:0/2954423508 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f797c00b180 con 0x7f79880a9320 2022-08-01T19:34:03.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.908+0000 7f79777fe700 1 -- 172.21.15.17:0/2954423508 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79780123f0 con 0x7f79880a9320 2022-08-01T19:34:03.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.908+0000 7f798759e700 1 -- 172.21.15.17:0/2954423508 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79880b3960 con 0x7f79880a9320 2022-08-01T19:34:03.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.912+0000 7f798759e700 1 -- 172.21.15.17:0/2954423508 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79880574e0 con 0x7f79880a9320 2022-08-01T19:34:03.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.912+0000 7f79777fe700 1 -- 172.21.15.17:0/2954423508 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7978012550 con 0x7f79880a9320 2022-08-01T19:34:03.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.912+0000 7f79777fe700 1 --2- 172.21.15.17:0/2954423508 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f79640539e0 0x7f7964055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:03.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.912+0000 7f79777fe700 1 -- 172.21.15.17:0/2954423508 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f797806c3e0 con 0x7f79880a9320 2022-08-01T19:34:03.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.912+0000 7f7985d9b700 1 --2- 172.21.15.17:0/2954423508 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f79640539e0 0x7f7964055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:03.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.912+0000 7f7985d9b700 1 --2- 172.21.15.17:0/2954423508 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f79640539e0 0x7f7964055ea0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f797c002f70 tx=0x7f797c014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:03.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:03.916+0000 7f79777fe700 1 -- 172.21.15.17:0/2954423508 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f79780434c0 con 0x7f79880a9320 2022-08-01T19:34:04.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.044+0000 7f798759e700 1 -- 172.21.15.17:0/2954423508 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f79880b3260 con 0x7f79880a9320 2022-08-01T19:34:04.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.044+0000 7f79777fe700 1 -- 172.21.15.17:0/2954423508 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v18) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7978046ae0 con 0x7f79880a9320 2022-08-01T19:34:04.047 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:34:04.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.048+0000 7f798759e700 1 -- 172.21.15.17:0/2954423508 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f79640539e0 msgr2=0x7f7964055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:04.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.048+0000 7f798759e700 1 --2- 172.21.15.17:0/2954423508 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f79640539e0 0x7f7964055ea0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f797c002f70 tx=0x7f797c014000).stop 2022-08-01T19:34:04.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.048+0000 7f798759e700 1 -- 172.21.15.17:0/2954423508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79880a9320 msgr2=0x7f79880b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:04.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.048+0000 7f798759e700 1 --2- 172.21.15.17:0/2954423508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79880a9320 0x7f79880b4860 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f797800c000 tx=0x7f7978006df0).stop 2022-08-01T19:34:04.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.048+0000 7f798759e700 1 -- 172.21.15.17:0/2954423508 shutdown_connections 2022-08-01T19:34:04.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.048+0000 7f798759e700 1 --2- 172.21.15.17:0/2954423508 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f79640539e0 0x7f7964055ea0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:04.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.048+0000 7f798759e700 1 --2- 172.21.15.17:0/2954423508 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79880b0d10 0x7f7988142ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-08-01T19:34:04.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.048+0000 7f798759e700 1 --2- 172.21.15.17:0/2954423508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79880b0560 0x7f7988142a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-08-01T19:34:04.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.048+0000 7f798759e700 1 --2- 172.21.15.17:0/2954423508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79880a9320 0x7f79880b4860 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:04.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.048+0000 7f798759e700 1 -- 172.21.15.17:0/2954423508 >> 172.21.15.17:0/2954423508 conn(0x7f79880612d0 msgr2=0x7f798813dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:04.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.048+0000 7f798759e700 1 -- 172.21.15.17:0/2954423508 shutdown_connections 2022-08-01T19:34:04.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.048+0000 7f798759e700 1 -- 172.21.15.17:0/2954423508 wait complete. 2022-08-01T19:34:04.066 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-08-01T19:34:04.170 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.168+0000 7fb539900700 1 Processor -- start 2022-08-01T19:34:04.170 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.168+0000 7fb539900700 1 -- start start 2022-08-01T19:34:04.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.168+0000 7fb539900700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb5340b5630 0x7fb5340af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:04.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.168+0000 7fb539900700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fb5340b07a0 0x7fb534147730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:04.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.168+0000 7fb539900700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5340b5a10 con 0x7fb534067730 2022-08-01T19:34:04.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.168+0000 7fb539900700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb534145470 con 0x7fb5340b5630 2022-08-01T19:34:04.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.168+0000 7fb539900700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5340ae180 con 0x7fb5340b07a0 2022-08-01T19:34:04.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.168+0000 7fb5388fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb5340b5630 0x7fb5340af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:04.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.168+0000 7fb5388fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb5340b5630 0x7fb5340af6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36132/0 (socket says 172.21.15.17:36132) 2022-08-01T19:34:04.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.168+0000 7fb5388fe700 1 -- 172.21.15.17:0/1179005774 learned_addr learned my addr 172.21.15.17:0/1179005774 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:04.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.168+0000 7fb533fff700 1 --2- 172.21.15.17:0/1179005774 >> v2:172.21.15.77:3300/0 conn(0x7fb5340b07a0 0x7fb534147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:04.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.168+0000 7fb533fff700 1 -- 172.21.15.17:0/1179005774 >> v2:172.21.15.17:3300/0 conn(0x7fb5340b5630 msgr2=0x7fb5340af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:04.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.168+0000 7fb533fff700 1 --2- 172.21.15.17:0/1179005774 >> v2:172.21.15.17:3300/0 conn(0x7fb5340b5630 0x7fb5340af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:04.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.168+0000 7fb533fff700 1 -- 172.21.15.17:0/1179005774 >> v1:172.21.15.77:6789/0 conn(0x7fb534067730 legacy=0x7fb5340acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:04.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.168+0000 7fb533fff700 1 -- 172.21.15.17:0/1179005774 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5340ac990 con 0x7fb5340b07a0 2022-08-01T19:34:04.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.168+0000 7fb5337fe700 1 -- 172.21.15.17:0/1179005774 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3410699435 0 0) 0x7fb5340b5a10 con 0x7fb534067730 2022-08-01T19:34:04.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.168+0000 7fb533fff700 1 --2- 172.21.15.17:0/1179005774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5340b07a0 0x7fb534147730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fb5240032f0 tx=0x7fb5240091d0).ready entity=mon.1 client_cookie=91db8c1d022bb88a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:04.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.168+0000 7fb5337fe700 1 -- 172.21.15.17:0/1179005774 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb52400b680 con 0x7fb5340b07a0 2022-08-01T19:34:04.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.172+0000 7fb5337fe700 1 -- 172.21.15.17:0/1179005774 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52400b7e0 con 0x7fb5340b07a0 2022-08-01T19:34:04.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.172+0000 7fb5337fe700 1 -- 172.21.15.17:0/1179005774 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb52400b950 con 0x7fb5340b07a0 2022-08-01T19:34:04.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.172+0000 7fb539900700 1 -- 172.21.15.17:0/1179005774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5340b07a0 msgr2=0x7fb534147730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:04.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.172+0000 7fb539900700 1 --2- 172.21.15.17:0/1179005774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5340b07a0 0x7fb534147730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fb5240032f0 tx=0x7fb5240091d0).stop 2022-08-01T19:34:04.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.172+0000 7fb539900700 1 -- 172.21.15.17:0/1179005774 shutdown_connections 2022-08-01T19:34:04.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.172+0000 7fb539900700 1 --2- 172.21.15.17:0/1179005774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5340b07a0 0x7fb534147730 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:04.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.172+0000 7fb539900700 1 --2- 172.21.15.17:0/1179005774 >> v2:172.21.15.17:3300/0 conn(0x7fb5340b5630 0x7fb5340af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:04.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.172+0000 7fb539900700 1 -- 172.21.15.17:0/1179005774 >> 172.21.15.17:0/1179005774 conn(0x7fb5340626f0 msgr2=0x7fb534062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:04.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.172+0000 7fb539900700 1 -- 172.21.15.17:0/1179005774 shutdown_connections 2022-08-01T19:34:04.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.172+0000 7fb539900700 1 -- 172.21.15.17:0/1179005774 wait complete. 2022-08-01T19:34:04.182 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.172+0000 7fb539900700 1 Processor -- start 2022-08-01T19:34:04.182 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.172+0000 7fb539900700 1 -- start start 2022-08-01T19:34:04.182 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.172+0000 7fb539900700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb534067730 0x7fb5340664c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:04.182 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.172+0000 7fb539900700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5340b07a0 0x7fb534066d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:04.182 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.172+0000 7fb539900700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb5340b5630 0x7fb5341468e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:04.183 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.172+0000 7fb539900700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5340b3ec0 con 0x7fb534067730 2022-08-01T19:34:04.183 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.172+0000 7fb539900700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5340b4160 con 0x7fb5340b07a0 2022-08-01T19:34:04.183 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.172+0000 7fb539900700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb534065d80 con 0x7fb5340b5630 2022-08-01T19:34:04.183 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.172+0000 7fb5388fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb534067730 0x7fb5340664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:04.184 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.172+0000 7fb5388fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb534067730 0x7fb5340664c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36136/0 (socket says 172.21.15.17:36136) 2022-08-01T19:34:04.184 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.172+0000 7fb5388fe700 1 -- 172.21.15.17:0/2099099152 learned_addr learned my addr 172.21.15.17:0/2099099152 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:04.184 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.172+0000 7fb533fff700 1 --2- 172.21.15.17:0/2099099152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5340b07a0 0x7fb534066d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:04.184 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.176+0000 7fb5390ff700 1 --2- 172.21.15.17:0/2099099152 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb5340b5630 0x7fb5341468e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:04.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.176+0000 7fb5388fe700 1 -- 172.21.15.17:0/2099099152 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb5340b5630 msgr2=0x7fb5341468e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:04.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.176+0000 7fb5388fe700 1 --2- 172.21.15.17:0/2099099152 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb5340b5630 0x7fb5341468e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:04.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.176+0000 7fb5388fe700 1 -- 172.21.15.17:0/2099099152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5340b07a0 msgr2=0x7fb534066d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:04.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.176+0000 7fb5388fe700 1 --2- 172.21.15.17:0/2099099152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5340b07a0 0x7fb534066d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:04.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.176+0000 7fb5388fe700 1 -- 172.21.15.17:0/2099099152 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb524009470 con 0x7fb534067730 2022-08-01T19:34:04.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.176+0000 7fb5388fe700 1 --2- 172.21.15.17:0/2099099152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb534067730 0x7fb5340664c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fb52800c000 tx=0x7fb528008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:04.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.176+0000 7fb531ffb700 1 -- 172.21.15.17:0/2099099152 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb52800fe30 con 0x7fb534067730 2022-08-01T19:34:04.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.176+0000 7fb531ffb700 1 -- 172.21.15.17:0/2099099152 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb528013610 con 0x7fb534067730 2022-08-01T19:34:04.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.176+0000 7fb539900700 1 -- 172.21.15.17:0/2099099152 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb534066020 con 0x7fb534067730 2022-08-01T19:34:04.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.176+0000 7fb531ffb700 1 -- 172.21.15.17:0/2099099152 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb528013970 con 0x7fb534067730 2022-08-01T19:34:04.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.176+0000 7fb539900700 1 -- 172.21.15.17:0/2099099152 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5340b39f0 con 0x7fb534067730 2022-08-01T19:34:04.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.176+0000 7fb539900700 1 -- 172.21.15.17:0/2099099152 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5340b1680 con 0x7fb534067730 2022-08-01T19:34:04.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.176+0000 7fb531ffb700 1 -- 172.21.15.17:0/2099099152 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb528013ad0 con 0x7fb534067730 2022-08-01T19:34:04.188 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.180+0000 7fb531ffb700 1 --2- 172.21.15.17:0/2099099152 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb514053b00 0x7fb514055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:04.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.180+0000 7fb531ffb700 1 -- 172.21.15.17:0/2099099152 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fb52806b960 con 0x7fb534067730 2022-08-01T19:34:04.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.180+0000 7fb533fff700 1 --2- 172.21.15.17:0/2099099152 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb514053b00 0x7fb514055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:04.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.180+0000 7fb533fff700 1 --2- 172.21.15.17:0/2099099152 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb514053b00 0x7fb514055fc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fb52401b000 tx=0x7fb524016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:04.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.184+0000 7fb531ffb700 1 -- 172.21.15.17:0/2099099152 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb528042b10 con 0x7fb534067730 2022-08-01T19:34:04.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.312+0000 7fb539900700 1 -- 172.21.15.17:0/2099099152 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb534061020 con 0x7fb534067730 2022-08-01T19:34:04.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.316+0000 7fb531ffb700 1 -- 172.21.15.17:0/2099099152 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+143 (secure 0 0 0) 0x7fb528046130 con 0x7fb534067730 2022-08-01T19:34:04.317 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:04.317 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2022-08-01T19:34:04.318 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:04.318 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 1, 2022-08-01T19:34:04.318 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": "device_health_metrics" 2022-08-01T19:34:04.318 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:04.319 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:04.319 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 2, 2022-08-01T19:34:04.319 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": "rbd" 2022-08-01T19:34:04.319 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:04.320 INFO:teuthology.orchestra.run.smithi017.stdout:] 2022-08-01T19:34:04.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.316+0000 7fb539900700 1 -- 172.21.15.17:0/2099099152 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb514053b00 msgr2=0x7fb514055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:04.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.316+0000 7fb539900700 1 --2- 172.21.15.17:0/2099099152 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb514053b00 0x7fb514055fc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fb52401b000 tx=0x7fb524016000).stop 2022-08-01T19:34:04.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.316+0000 7fb539900700 1 -- 172.21.15.17:0/2099099152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb534067730 msgr2=0x7fb5340664c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:04.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.316+0000 7fb539900700 1 --2- 172.21.15.17:0/2099099152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb534067730 0x7fb5340664c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fb52800c000 tx=0x7fb528008040).stop 2022-08-01T19:34:04.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.316+0000 7fb539900700 1 -- 172.21.15.17:0/2099099152 shutdown_connections 2022-08-01T19:34:04.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.316+0000 7fb539900700 1 --2- 172.21.15.17:0/2099099152 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb514053b00 0x7fb514055fc0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:04.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.316+0000 7fb539900700 1 --2- 172.21.15.17:0/2099099152 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb5340b5630 0x7fb5341468e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:04.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.316+0000 7fb539900700 1 --2- 172.21.15.17:0/2099099152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5340b07a0 0x7fb534066d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:04.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.316+0000 7fb539900700 1 --2- 172.21.15.17:0/2099099152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb534067730 0x7fb5340664c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:04.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.316+0000 7fb539900700 1 -- 172.21.15.17:0/2099099152 >> 172.21.15.17:0/2099099152 conn(0x7fb5340626f0 msgr2=0x7fb5340acd10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:04.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.316+0000 7fb539900700 1 -- 172.21.15.17:0/2099099152 shutdown_connections 2022-08-01T19:34:04.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.316+0000 7fb539900700 1 -- 172.21.15.17:0/2099099152 wait complete. 2022-08-01T19:34:04.334 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-08-01T19:34:04.335 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-08-01T19:34:04.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.436+0000 7fe9c6b54700 1 Processor -- start 2022-08-01T19:34:04.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.436+0000 7fe9c6b54700 1 -- start start 2022-08-01T19:34:04.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.436+0000 7fe9c6b54700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fe9c00b4310 0x7fe9c0140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:04.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.436+0000 7fe9c6b54700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe9c00b4820 0x7fe9c00b5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:04.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.436+0000 7fe9c6b54700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c0143cf0 con 0x7fe9c0146ae0 2022-08-01T19:34:04.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.436+0000 7fe9c6b54700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9c01476a0 con 0x7fe9c00b4310 2022-08-01T19:34:04.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.436+0000 7fe9c6b54700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c0143f90 con 0x7fe9c00b4820 2022-08-01T19:34:04.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.436+0000 7fe9c48f0700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe9c0146ae0 0x7fe9c01403c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56570/0 (socket says 172.21.15.17:56570) 2022-08-01T19:34:04.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.436+0000 7fe9c48f0700 1 -- 172.21.15.17:0/714915757 learned_addr learned my addr 172.21.15.17:0/714915757 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:04.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.440+0000 7fe9bffff700 1 --2- 172.21.15.17:0/714915757 >> v2:172.21.15.77:3300/0 conn(0x7fe9c00b4310 0x7fe9c0140ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:04.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.440+0000 7fe9c50f1700 1 --2- 172.21.15.17:0/714915757 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe9c00b4820 0x7fe9c00b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:04.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.440+0000 7fe9bf7fe700 1 -- 172.21.15.17:0/714915757 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3977264798 0 0) 0x7fe9c0143cf0 con 0x7fe9c0146ae0 2022-08-01T19:34:04.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.440+0000 7fe9bf7fe700 1 -- 172.21.15.17:0/714915757 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9a0003260 con 0x7fe9c0146ae0 2022-08-01T19:34:04.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.440+0000 7fe9c50f1700 1 -- 172.21.15.17:0/714915757 >> v2:172.21.15.77:3300/0 conn(0x7fe9c00b4310 msgr2=0x7fe9c0140ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:04.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.440+0000 7fe9c50f1700 1 --2- 172.21.15.17:0/714915757 >> v2:172.21.15.77:3300/0 conn(0x7fe9c00b4310 0x7fe9c0140ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:04.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.440+0000 7fe9c50f1700 1 -- 172.21.15.17:0/714915757 >> v1:172.21.15.17:6789/0 conn(0x7fe9c0146ae0 legacy=0x7fe9c01403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:04.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.440+0000 7fe9c50f1700 1 -- 172.21.15.17:0/714915757 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c00b0e20 con 0x7fe9c00b4820 2022-08-01T19:34:04.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.440+0000 7fe9c50f1700 1 --2- 172.21.15.17:0/714915757 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe9c00b4820 0x7fe9c00b5060 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7fe9b80032f0 tx=0x7fe9b80091d0).ready entity=mon.2 client_cookie=6338e654cc517ff2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:04.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.440+0000 7fe9bf7fe700 1 -- 172.21.15.17:0/714915757 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9b800b600 con 0x7fe9c00b4820 2022-08-01T19:34:04.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.440+0000 7fe9bf7fe700 1 -- 172.21.15.17:0/714915757 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b800b760 con 0x7fe9c00b4820 2022-08-01T19:34:04.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.440+0000 7fe9bf7fe700 1 -- 172.21.15.17:0/714915757 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9b800b8d0 con 0x7fe9c00b4820 2022-08-01T19:34:04.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.440+0000 7fe9c6b54700 1 -- 172.21.15.17:0/714915757 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe9c00b4820 msgr2=0x7fe9c00b5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:04.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.440+0000 7fe9c6b54700 1 --2- 172.21.15.17:0/714915757 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe9c00b4820 0x7fe9c00b5060 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7fe9b80032f0 tx=0x7fe9b80091d0).stop 2022-08-01T19:34:04.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.440+0000 7fe9c6b54700 1 -- 172.21.15.17:0/714915757 shutdown_connections 2022-08-01T19:34:04.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.440+0000 7fe9c6b54700 1 --2- 172.21.15.17:0/714915757 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe9c00b4820 0x7fe9c00b5060 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:04.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.440+0000 7fe9c6b54700 1 --2- 172.21.15.17:0/714915757 >> v2:172.21.15.77:3300/0 conn(0x7fe9c00b4310 0x7fe9c0140ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:04.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.440+0000 7fe9c6b54700 1 -- 172.21.15.17:0/714915757 >> 172.21.15.17:0/714915757 conn(0x7fe9c005c620 msgr2=0x7fe9c005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:04.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.440+0000 7fe9c6b54700 1 -- 172.21.15.17:0/714915757 shutdown_connections 2022-08-01T19:34:04.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.440+0000 7fe9c6b54700 1 -- 172.21.15.17:0/714915757 wait complete. 2022-08-01T19:34:04.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.444+0000 7fe9c6b54700 1 Processor -- start 2022-08-01T19:34:04.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.444+0000 7fe9c6b54700 1 -- start start 2022-08-01T19:34:04.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.444+0000 7fe9c6b54700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9c00b4310 0x7fe9c00ae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:04.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.444+0000 7fe9c6b54700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe9c00b4820 0x7fe9c00b8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:04.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.444+0000 7fe9c6b54700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9c0146ae0 0x7fe9c0155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:04.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.444+0000 7fe9c6b54700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c0152f50 con 0x7fe9c00b4310 2022-08-01T19:34:04.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.444+0000 7fe9c6b54700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c01531f0 con 0x7fe9c0146ae0 2022-08-01T19:34:04.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.444+0000 7fe9c6b54700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c00add60 con 0x7fe9c00b4820 2022-08-01T19:34:04.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.444+0000 7fe9c48f0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9c00b4310 0x7fe9c00ae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:04.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.444+0000 7fe9c48f0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9c00b4310 0x7fe9c00ae4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36150/0 (socket says 172.21.15.17:36150) 2022-08-01T19:34:04.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.444+0000 7fe9c48f0700 1 -- 172.21.15.17:0/2429847959 learned_addr learned my addr 172.21.15.17:0/2429847959 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:04.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.444+0000 7fe9bffff700 1 --2- 172.21.15.17:0/2429847959 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe9c00b4820 0x7fe9c00b8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:04.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.444+0000 7fe9c50f1700 1 --2- 172.21.15.17:0/2429847959 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9c0146ae0 0x7fe9c0155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:04.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.444+0000 7fe9c48f0700 1 -- 172.21.15.17:0/2429847959 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe9c00b4820 msgr2=0x7fe9c00b8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:04.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.444+0000 7fe9c48f0700 1 --2- 172.21.15.17:0/2429847959 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe9c00b4820 0x7fe9c00b8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:04.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.444+0000 7fe9c48f0700 1 -- 172.21.15.17:0/2429847959 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9c0146ae0 msgr2=0x7fe9c0155920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:04.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.444+0000 7fe9c48f0700 1 --2- 172.21.15.17:0/2429847959 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9c0146ae0 0x7fe9c0155920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:04.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.444+0000 7fe9c48f0700 1 -- 172.21.15.17:0/2429847959 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b8009470 con 0x7fe9c00b4310 2022-08-01T19:34:04.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.444+0000 7fe9c48f0700 1 --2- 172.21.15.17:0/2429847959 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9c00b4310 0x7fe9c00ae4a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fe9b0001110 tx=0x7fe9b0007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:04.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.444+0000 7fe9bdffb700 1 -- 172.21.15.17:0/2429847959 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9b000ce80 con 0x7fe9c00b4310 2022-08-01T19:34:04.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.444+0000 7fe9bdffb700 1 -- 172.21.15.17:0/2429847959 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b00070d0 con 0x7fe9c00b4310 2022-08-01T19:34:04.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.444+0000 7fe9c6b54700 1 -- 172.21.15.17:0/2429847959 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c00ae000 con 0x7fe9c00b4310 2022-08-01T19:34:04.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.444+0000 7fe9bdffb700 1 -- 172.21.15.17:0/2429847959 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9b0007430 con 0x7fe9c00b4310 2022-08-01T19:34:04.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.444+0000 7fe9c6b54700 1 -- 172.21.15.17:0/2429847959 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c00b8a20 con 0x7fe9c00b4310 2022-08-01T19:34:04.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.448+0000 7fe9c6b54700 1 -- 172.21.15.17:0/2429847959 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9c00ae190 con 0x7fe9c00b4310 2022-08-01T19:34:04.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.448+0000 7fe9bdffb700 1 -- 172.21.15.17:0/2429847959 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe9b0010070 con 0x7fe9c00b4310 2022-08-01T19:34:04.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.448+0000 7fe9bdffb700 1 --2- 172.21.15.17:0/2429847959 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe9a4053a70 0x7fe9a4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:04.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.448+0000 7fe9bdffb700 1 -- 172.21.15.17:0/2429847959 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fe9b006b270 con 0x7fe9c00b4310 2022-08-01T19:34:04.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.448+0000 7fe9bffff700 1 --2- 172.21.15.17:0/2429847959 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe9a4053a70 0x7fe9a4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:04.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.452+0000 7fe9bffff700 1 --2- 172.21.15.17:0/2429847959 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe9a4053a70 0x7fe9a4055f30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fe9a800b000 tx=0x7fe9a8006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:04.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.456+0000 7fe9bdffb700 1 -- 172.21.15.17:0/2429847959 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe9b00423b0 con 0x7fe9c00b4310 2022-08-01T19:34:04.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:04.580+0000 7fe9c6b54700 1 -- 172.21.15.17:0/2429847959 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fe9c00574e0 con 0x7fe9c00b4310 2022-08-01T19:34:05.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.348+0000 7fe9bdffb700 1 -- 172.21.15.17:0/2429847959 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v20) v1 ==== 133+0+0 (secure 0 0 0) 0x7fe9b0018070 con 0x7fe9c00b4310 2022-08-01T19:34:05.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.352+0000 7fe9c6b54700 1 -- 172.21.15.17:0/2429847959 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe9a4053a70 msgr2=0x7fe9a4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:05.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.352+0000 7fe9c6b54700 1 --2- 172.21.15.17:0/2429847959 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe9a4053a70 0x7fe9a4055f30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fe9a800b000 tx=0x7fe9a8006df0).stop 2022-08-01T19:34:05.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.352+0000 7fe9c6b54700 1 -- 172.21.15.17:0/2429847959 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9c00b4310 msgr2=0x7fe9c00ae4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:05.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.352+0000 7fe9c6b54700 1 --2- 172.21.15.17:0/2429847959 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9c00b4310 0x7fe9c00ae4a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fe9b0001110 tx=0x7fe9b0007aa0).stop 2022-08-01T19:34:05.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.356+0000 7fe9c6b54700 1 -- 172.21.15.17:0/2429847959 shutdown_connections 2022-08-01T19:34:05.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.356+0000 7fe9c6b54700 1 --2- 172.21.15.17:0/2429847959 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe9a4053a70 0x7fe9a4055f30 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:05.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.356+0000 7fe9c6b54700 1 --2- 172.21.15.17:0/2429847959 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9c0146ae0 0x7fe9c0155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:05.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.356+0000 7fe9c6b54700 1 --2- 172.21.15.17:0/2429847959 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe9c00b4820 0x7fe9c00b8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:05.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.356+0000 7fe9c6b54700 1 --2- 172.21.15.17:0/2429847959 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9c00b4310 0x7fe9c00ae4a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:05.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.356+0000 7fe9c6b54700 1 -- 172.21.15.17:0/2429847959 >> 172.21.15.17:0/2429847959 conn(0x7fe9c005c620 msgr2=0x7fe9c00b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:05.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.356+0000 7fe9c6b54700 1 -- 172.21.15.17:0/2429847959 shutdown_connections 2022-08-01T19:34:05.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.356+0000 7fe9c6b54700 1 -- 172.21.15.17:0/2429847959 wait complete. 2022-08-01T19:34:05.362 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' created 2022-08-01T19:34:05.373 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2022-08-01T19:34:05.373 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data m=2 k=2 crush-failure-domain=osd 2022-08-01T19:34:05.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.484+0000 7f7a67fff700 1 Processor -- start 2022-08-01T19:34:05.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.484+0000 7f7a67fff700 1 -- start start 2022-08-01T19:34:05.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.484+0000 7f7a67fff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f7a680b3190 0x7f7a680b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:05.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.484+0000 7f7a67fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7a680a9da0 0x7f7a68147e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:05.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.484+0000 7f7a67fff700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a680b3570 con 0x7f7a680b1a80 2022-08-01T19:34:05.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.484+0000 7f7a67fff700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a680b2980 con 0x7f7a680a9da0 2022-08-01T19:34:05.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.484+0000 7f7a67fff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a680b0e00 con 0x7f7a680b3190 2022-08-01T19:34:05.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.484+0000 7f7a677fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7a680a9da0 0x7f7a68147e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:05.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.484+0000 7f7a677fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7a680a9da0 0x7f7a68147e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36160/0 (socket says 172.21.15.17:36160) 2022-08-01T19:34:05.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.484+0000 7f7a677fe700 1 -- 172.21.15.17:0/215645236 learned_addr learned my addr 172.21.15.17:0/215645236 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:05.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.484+0000 7f7a66ffd700 1 --2- 172.21.15.17:0/215645236 >> v2:172.21.15.77:3300/0 conn(0x7f7a680b3190 0x7f7a680b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:05.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.484+0000 7f7a66ffd700 1 -- 172.21.15.17:0/215645236 >> v2:172.21.15.17:3300/0 conn(0x7f7a680a9da0 msgr2=0x7f7a68147e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:05.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.484+0000 7f7a66ffd700 1 --2- 172.21.15.17:0/215645236 >> v2:172.21.15.17:3300/0 conn(0x7f7a680a9da0 0x7f7a68147e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:05.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.484+0000 7f7a65ffb700 1 -- 172.21.15.17:0/215645236 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2473968974 0 0) 0x7f7a680b3570 con 0x7f7a680b1a80 2022-08-01T19:34:05.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.484+0000 7f7a66ffd700 1 -- 172.21.15.17:0/215645236 >> v1:172.21.15.77:6789/0 conn(0x7f7a680b1a80 legacy=0x7f7a680656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:05.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.484+0000 7f7a66ffd700 1 -- 172.21.15.17:0/215645236 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a680b10a0 con 0x7f7a680b3190 2022-08-01T19:34:05.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.484+0000 7f7a66ffd700 1 --2- 172.21.15.17:0/215645236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a680b3190 0x7f7a680b2490 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f7a5c002f70 tx=0x7f7a5c006df0).ready entity=mon.1 client_cookie=6943d136a1ab1179 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:05.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.484+0000 7f7a65ffb700 1 -- 172.21.15.17:0/215645236 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a5c00d570 con 0x7f7a680b3190 2022-08-01T19:34:05.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.484+0000 7f7a65ffb700 1 -- 172.21.15.17:0/215645236 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5c000e90 con 0x7f7a680b3190 2022-08-01T19:34:05.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.484+0000 7f7a65ffb700 1 -- 172.21.15.17:0/215645236 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a5c00d8c0 con 0x7f7a680b3190 2022-08-01T19:34:05.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.484+0000 7f7a67fff700 1 -- 172.21.15.17:0/215645236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a680b3190 msgr2=0x7f7a680b2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:05.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.484+0000 7f7a67fff700 1 --2- 172.21.15.17:0/215645236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a680b3190 0x7f7a680b2490 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f7a5c002f70 tx=0x7f7a5c006df0).stop 2022-08-01T19:34:05.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.484+0000 7f7a67fff700 1 -- 172.21.15.17:0/215645236 shutdown_connections 2022-08-01T19:34:05.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.484+0000 7f7a67fff700 1 --2- 172.21.15.17:0/215645236 >> v2:172.21.15.17:3300/0 conn(0x7f7a680a9da0 0x7f7a68147e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:05.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.484+0000 7f7a67fff700 1 --2- 172.21.15.17:0/215645236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a680b3190 0x7f7a680b2490 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:05.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.484+0000 7f7a67fff700 1 -- 172.21.15.17:0/215645236 >> 172.21.15.17:0/215645236 conn(0x7f7a68063870 msgr2=0x7f7a68063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:05.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.488+0000 7f7a67fff700 1 -- 172.21.15.17:0/215645236 shutdown_connections 2022-08-01T19:34:05.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.488+0000 7f7a67fff700 1 -- 172.21.15.17:0/215645236 wait complete. 2022-08-01T19:34:05.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.488+0000 7f7a67fff700 1 Processor -- start 2022-08-01T19:34:05.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.488+0000 7f7a67fff700 1 -- start start 2022-08-01T19:34:05.499 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.488+0000 7f7a67fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a680a9da0 0x7f7a680ad830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:05.499 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.488+0000 7f7a67fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7a680b1a80 0x7f7a680acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:05.499 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.488+0000 7f7a67fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a680b3190 0x7f7a681565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:05.499 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.488+0000 7f7a67fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a680ab980 con 0x7f7a680a9da0 2022-08-01T19:34:05.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.488+0000 7f7a67fff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a680aadc0 con 0x7f7a680b3190 2022-08-01T19:34:05.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.488+0000 7f7a67fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a680ab490 con 0x7f7a680b1a80 2022-08-01T19:34:05.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.488+0000 7f7a66ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a680a9da0 0x7f7a680ad830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:05.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.488+0000 7f7a66ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a680a9da0 0x7f7a680ad830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36164/0 (socket says 172.21.15.17:36164) 2022-08-01T19:34:05.501 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.488+0000 7f7a66ffd700 1 -- 172.21.15.17:0/2731596321 learned_addr learned my addr 172.21.15.17:0/2731596321 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:05.501 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.488+0000 7f7a677fe700 1 --2- 172.21.15.17:0/2731596321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a680b3190 0x7f7a681565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:05.501 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.488+0000 7f7a667fc700 1 --2- 172.21.15.17:0/2731596321 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7a680b1a80 0x7f7a680acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:05.501 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.488+0000 7f7a66ffd700 1 -- 172.21.15.17:0/2731596321 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7a680b1a80 msgr2=0x7f7a680acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:05.501 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.488+0000 7f7a66ffd700 1 --2- 172.21.15.17:0/2731596321 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7a680b1a80 0x7f7a680acbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:05.502 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.488+0000 7f7a66ffd700 1 -- 172.21.15.17:0/2731596321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a680b3190 msgr2=0x7f7a681565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:05.502 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.488+0000 7f7a66ffd700 1 --2- 172.21.15.17:0/2731596321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a680b3190 0x7f7a681565d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:05.502 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.488+0000 7f7a66ffd700 1 -- 172.21.15.17:0/2731596321 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5c00b180 con 0x7f7a680a9da0 2022-08-01T19:34:05.502 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.488+0000 7f7a66ffd700 1 --2- 172.21.15.17:0/2731596321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a680a9da0 0x7f7a680ad830 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f7a5c002f70 tx=0x7f7a5c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:05.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.492+0000 7f7a57fff700 1 -- 172.21.15.17:0/2731596321 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a5c016070 con 0x7f7a680a9da0 2022-08-01T19:34:05.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.492+0000 7f7a57fff700 1 -- 172.21.15.17:0/2731596321 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5c0138d0 con 0x7f7a680a9da0 2022-08-01T19:34:05.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.492+0000 7f7a57fff700 1 -- 172.21.15.17:0/2731596321 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a5c013c30 con 0x7f7a680a9da0 2022-08-01T19:34:05.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.492+0000 7f7a67fff700 1 -- 172.21.15.17:0/2731596321 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a680abbb0 con 0x7f7a680a9da0 2022-08-01T19:34:05.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.492+0000 7f7a67fff700 1 -- 172.21.15.17:0/2731596321 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a680aaff0 con 0x7f7a680a9da0 2022-08-01T19:34:05.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.492+0000 7f7a67fff700 1 -- 172.21.15.17:0/2731596321 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a680574e0 con 0x7f7a680a9da0 2022-08-01T19:34:05.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.492+0000 7f7a57fff700 1 -- 172.21.15.17:0/2731596321 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7a5c00f030 con 0x7f7a680a9da0 2022-08-01T19:34:05.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.492+0000 7f7a57fff700 1 --2- 172.21.15.17:0/2731596321 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7a44053950 0x7f7a44055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:05.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.492+0000 7f7a57fff700 1 -- 172.21.15.17:0/2731596321 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f7a5c06c840 con 0x7f7a680a9da0 2022-08-01T19:34:05.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.492+0000 7f7a667fc700 1 --2- 172.21.15.17:0/2731596321 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7a44053950 0x7f7a44055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:05.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.496+0000 7f7a667fc700 1 --2- 172.21.15.17:0/2731596321 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7a44053950 0x7f7a44055e10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f7a5800d000 tx=0x7f7a58007a00).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:05.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.500+0000 7f7a57fff700 1 -- 172.21.15.17:0/2731596321 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7a5c043940 con 0x7f7a680a9da0 2022-08-01T19:34:05.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:05.632+0000 7f7a67fff700 1 -- 172.21.15.17:0/2731596321 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f7a680ac630 con 0x7f7a680a9da0 2022-08-01T19:34:06.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.356+0000 7f7a57fff700 1 -- 172.21.15.17:0/2731596321 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) v1 ==== 154+0+0 (secure 0 0 0) 0x7f7a5c023020 con 0x7f7a680a9da0 2022-08-01T19:34:06.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.360+0000 7f7a67fff700 1 -- 172.21.15.17:0/2731596321 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7a44053950 msgr2=0x7f7a44055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:06.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.360+0000 7f7a67fff700 1 --2- 172.21.15.17:0/2731596321 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7a44053950 0x7f7a44055e10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f7a5800d000 tx=0x7f7a58007a00).stop 2022-08-01T19:34:06.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.360+0000 7f7a67fff700 1 -- 172.21.15.17:0/2731596321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a680a9da0 msgr2=0x7f7a680ad830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:06.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.360+0000 7f7a67fff700 1 --2- 172.21.15.17:0/2731596321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a680a9da0 0x7f7a680ad830 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f7a5c002f70 tx=0x7f7a5c014000).stop 2022-08-01T19:34:06.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.360+0000 7f7a67fff700 1 -- 172.21.15.17:0/2731596321 shutdown_connections 2022-08-01T19:34:06.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.360+0000 7f7a67fff700 1 --2- 172.21.15.17:0/2731596321 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7a44053950 0x7f7a44055e10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:06.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.360+0000 7f7a67fff700 1 --2- 172.21.15.17:0/2731596321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a680b3190 0x7f7a681565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:06.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.360+0000 7f7a67fff700 1 --2- 172.21.15.17:0/2731596321 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7a680b1a80 0x7f7a680acbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:06.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.360+0000 7f7a67fff700 1 --2- 172.21.15.17:0/2731596321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a680a9da0 0x7f7a680ad830 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:06.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.360+0000 7f7a67fff700 1 -- 172.21.15.17:0/2731596321 >> 172.21.15.17:0/2731596321 conn(0x7f7a68063870 msgr2=0x7f7a6805fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:06.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.360+0000 7f7a67fff700 1 -- 172.21.15.17:0/2731596321 shutdown_connections 2022-08-01T19:34:06.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.360+0000 7f7a67fff700 1 -- 172.21.15.17:0/2731596321 wait complete. 2022-08-01T19:34:06.378 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 erasure cephfs_data 2022-08-01T19:34:06.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.488+0000 7fe2e6c30700 1 Processor -- start 2022-08-01T19:34:06.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.488+0000 7fe2e6c30700 1 -- start start 2022-08-01T19:34:06.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.492+0000 7fe2e6c30700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe2e00af2c0 0x7fe2e00679f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:06.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.492+0000 7fe2e6c30700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe2e00afef0 0x7fe2e0141010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:06.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.492+0000 7fe2e6c30700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e01445e0 con 0x7fe2e00b29e0 2022-08-01T19:34:06.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.492+0000 7fe2e6c30700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2e00b2dc0 con 0x7fe2e00afef0 2022-08-01T19:34:06.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.492+0000 7fe2e6c30700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e0145490 con 0x7fe2e00af2c0 2022-08-01T19:34:06.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.492+0000 7fe2e5c2e700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe2e00b29e0 0x7fe2e00672e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56598/0 (socket says 172.21.15.17:56598) 2022-08-01T19:34:06.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.492+0000 7fe2e5c2e700 1 -- 172.21.15.17:0/4186068193 learned_addr learned my addr 172.21.15.17:0/4186068193 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:06.499 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.492+0000 7fe2e642f700 1 --2- 172.21.15.17:0/4186068193 >> v2:172.21.15.17:3300/0 conn(0x7fe2e00afef0 0x7fe2e0141010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:06.499 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.492+0000 7fe2e542d700 1 --2- 172.21.15.17:0/4186068193 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe2e00af2c0 0x7fe2e00679f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:06.499 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.492+0000 7fe2e4c2c700 1 -- 172.21.15.17:0/4186068193 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2015399925 0 0) 0x7fe2e01445e0 con 0x7fe2e00b29e0 2022-08-01T19:34:06.499 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.492+0000 7fe2e4c2c700 1 -- 172.21.15.17:0/4186068193 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2c8003260 con 0x7fe2e00b29e0 2022-08-01T19:34:06.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.492+0000 7fe2e642f700 1 -- 172.21.15.17:0/4186068193 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe2e00af2c0 msgr2=0x7fe2e00679f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:06.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.492+0000 7fe2e642f700 1 --2- 172.21.15.17:0/4186068193 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe2e00af2c0 0x7fe2e00679f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:06.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.492+0000 7fe2e642f700 1 -- 172.21.15.17:0/4186068193 >> v1:172.21.15.17:6789/0 conn(0x7fe2e00b29e0 legacy=0x7fe2e00672e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:06.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.492+0000 7fe2e642f700 1 -- 172.21.15.17:0/4186068193 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e0144810 con 0x7fe2e00afef0 2022-08-01T19:34:06.501 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.492+0000 7fe2e542d700 1 --2- 172.21.15.17:0/4186068193 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe2e00af2c0 0x7fe2e00679f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:34:06.501 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.492+0000 7fe2e642f700 1 --2- 172.21.15.17:0/4186068193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2e00afef0 0x7fe2e0141010 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fe2d80032f0 tx=0x7fe2d80091d0).ready entity=mon.0 client_cookie=9ef19a8a93ee3bc3 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:06.501 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.492+0000 7fe2e4c2c700 1 -- 172.21.15.17:0/4186068193 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2d800b6c0 con 0x7fe2e00afef0 2022-08-01T19:34:06.501 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.492+0000 7fe2e4c2c700 1 -- 172.21.15.17:0/4186068193 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d800b820 con 0x7fe2e00afef0 2022-08-01T19:34:06.502 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.492+0000 7fe2e4c2c700 1 -- 172.21.15.17:0/4186068193 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2d800b990 con 0x7fe2e00afef0 2022-08-01T19:34:06.502 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.492+0000 7fe2e6c30700 1 -- 172.21.15.17:0/4186068193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2e00afef0 msgr2=0x7fe2e0141010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:06.502 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.492+0000 7fe2e6c30700 1 --2- 172.21.15.17:0/4186068193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2e00afef0 0x7fe2e0141010 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fe2d80032f0 tx=0x7fe2d80091d0).stop 2022-08-01T19:34:06.502 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.496+0000 7fe2e6c30700 1 -- 172.21.15.17:0/4186068193 shutdown_connections 2022-08-01T19:34:06.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.496+0000 7fe2e6c30700 1 --2- 172.21.15.17:0/4186068193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2e00afef0 0x7fe2e0141010 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:06.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.496+0000 7fe2e6c30700 1 --2- 172.21.15.17:0/4186068193 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe2e00af2c0 0x7fe2e00679f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:06.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.496+0000 7fe2e6c30700 1 -- 172.21.15.17:0/4186068193 >> 172.21.15.17:0/4186068193 conn(0x7fe2e00613c0 msgr2=0x7fe2e0062f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:06.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.496+0000 7fe2e6c30700 1 -- 172.21.15.17:0/4186068193 shutdown_connections 2022-08-01T19:34:06.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.496+0000 7fe2e6c30700 1 -- 172.21.15.17:0/4186068193 wait complete. 2022-08-01T19:34:06.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.496+0000 7fe2e6c30700 1 Processor -- start 2022-08-01T19:34:06.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.496+0000 7fe2e6c30700 1 -- start start 2022-08-01T19:34:06.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.496+0000 7fe2e6c30700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe2e00af2c0 0x7fe2e00b0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:06.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.496+0000 7fe2e6c30700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2e00afef0 0x7fe2e014f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:06.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.496+0000 7fe2e6c30700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2e00b29e0 0x7fe2e014fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:06.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.496+0000 7fe2e6c30700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e01501a0 con 0x7fe2e00afef0 2022-08-01T19:34:06.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.496+0000 7fe2e6c30700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e0155c40 con 0x7fe2e00b29e0 2022-08-01T19:34:06.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.496+0000 7fe2e6c30700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e0155ee0 con 0x7fe2e00af2c0 2022-08-01T19:34:06.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.496+0000 7fe2e542d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2e00afef0 0x7fe2e014f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:06.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.496+0000 7fe2e542d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2e00afef0 0x7fe2e014f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36178/0 (socket says 172.21.15.17:36178) 2022-08-01T19:34:06.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.496+0000 7fe2e542d700 1 -- 172.21.15.17:0/934863134 learned_addr learned my addr 172.21.15.17:0/934863134 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:06.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.496+0000 7fe2e5c2e700 1 --2- 172.21.15.17:0/934863134 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe2e00af2c0 0x7fe2e00b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:06.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.496+0000 7fe2e642f700 1 --2- 172.21.15.17:0/934863134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2e00b29e0 0x7fe2e014fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:06.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.496+0000 7fe2e542d700 1 -- 172.21.15.17:0/934863134 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe2e00af2c0 msgr2=0x7fe2e00b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:06.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.496+0000 7fe2e542d700 1 --2- 172.21.15.17:0/934863134 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe2e00af2c0 0x7fe2e00b0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:06.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.496+0000 7fe2e542d700 1 -- 172.21.15.17:0/934863134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2e00b29e0 msgr2=0x7fe2e014fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:06.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.496+0000 7fe2e542d700 1 --2- 172.21.15.17:0/934863134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2e00b29e0 0x7fe2e014fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:06.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.496+0000 7fe2e542d700 1 -- 172.21.15.17:0/934863134 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d8009470 con 0x7fe2e00afef0 2022-08-01T19:34:06.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.496+0000 7fe2e542d700 1 --2- 172.21.15.17:0/934863134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2e00afef0 0x7fe2e014f7c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fe2d000a000 tx=0x7fe2d0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:06.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.496+0000 7fe2ceffd700 1 -- 172.21.15.17:0/934863134 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2d0011850 con 0x7fe2e00afef0 2022-08-01T19:34:06.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.496+0000 7fe2ceffd700 1 -- 172.21.15.17:0/934863134 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d00119b0 con 0x7fe2e00afef0 2022-08-01T19:34:06.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.496+0000 7fe2e6c30700 1 -- 172.21.15.17:0/934863134 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e0156180 con 0x7fe2e00afef0 2022-08-01T19:34:06.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.496+0000 7fe2ceffd700 1 -- 172.21.15.17:0/934863134 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2d0011d10 con 0x7fe2e00afef0 2022-08-01T19:34:06.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.496+0000 7fe2e6c30700 1 -- 172.21.15.17:0/934863134 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e00b0d50 con 0x7fe2e00afef0 2022-08-01T19:34:06.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.500+0000 7fe2ceffd700 1 -- 172.21.15.17:0/934863134 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe2d0012420 con 0x7fe2e00afef0 2022-08-01T19:34:06.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.500+0000 7fe2ceffd700 1 --2- 172.21.15.17:0/934863134 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe2c0053a10 0x7fe2c0055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:06.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.500+0000 7fe2ceffd700 1 -- 172.21.15.17:0/934863134 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5863+0+0 (secure 0 0 0) 0x7fe2d006bd60 con 0x7fe2e00afef0 2022-08-01T19:34:06.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.500+0000 7fe2e5c2e700 1 --2- 172.21.15.17:0/934863134 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe2c0053a10 0x7fe2c0055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:06.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.500+0000 7fe2e6c30700 1 -- 172.21.15.17:0/934863134 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e00574e0 con 0x7fe2e00afef0 2022-08-01T19:34:06.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.500+0000 7fe2e5c2e700 1 --2- 172.21.15.17:0/934863134 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe2c0053a10 0x7fe2c0055ed0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fe2dc00b000 tx=0x7fe2dc009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:06.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.504+0000 7fe2ceffd700 1 -- 172.21.15.17:0/934863134 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe2d0042cf0 con 0x7fe2e00afef0 2022-08-01T19:34:06.637 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:06.636+0000 7fe2e6c30700 1 -- 172.21.15.17:0/934863134 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8, "pool_type": "erasure", "erasure_code_profile": "cephfs_data"} v 0) v1 -- 0x7fe2e005f3d0 con 0x7fe2e00afef0 2022-08-01T19:34:08.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.367+0000 7fe2ceffd700 1 -- 172.21.15.17:0/934863134 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8, "pool_type": "erasure", "erasure_code_profile": "cephfs_data"}]=0 pool 'cephfs_data' created v23) v1 ==== 188+0+0 (secure 0 0 0) 0x7fe2d0046310 con 0x7fe2e00afef0 2022-08-01T19:34:08.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.371+0000 7fe2e6c30700 1 -- 172.21.15.17:0/934863134 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe2c0053a10 msgr2=0x7fe2c0055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:08.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.371+0000 7fe2e6c30700 1 --2- 172.21.15.17:0/934863134 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe2c0053a10 0x7fe2c0055ed0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fe2dc00b000 tx=0x7fe2dc009040).stop 2022-08-01T19:34:08.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.371+0000 7fe2e6c30700 1 -- 172.21.15.17:0/934863134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2e00afef0 msgr2=0x7fe2e014f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:08.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.371+0000 7fe2e6c30700 1 --2- 172.21.15.17:0/934863134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2e00afef0 0x7fe2e014f7c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fe2d000a000 tx=0x7fe2d0008040).stop 2022-08-01T19:34:08.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.371+0000 7fe2e6c30700 1 -- 172.21.15.17:0/934863134 shutdown_connections 2022-08-01T19:34:08.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.371+0000 7fe2e6c30700 1 --2- 172.21.15.17:0/934863134 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe2c0053a10 0x7fe2c0055ed0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:08.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.371+0000 7fe2e6c30700 1 --2- 172.21.15.17:0/934863134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2e00b29e0 0x7fe2e014fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:08.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.371+0000 7fe2e6c30700 1 --2- 172.21.15.17:0/934863134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2e00afef0 0x7fe2e014f7c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:08.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.371+0000 7fe2e6c30700 1 --2- 172.21.15.17:0/934863134 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe2e00af2c0 0x7fe2e00b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:08.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.375+0000 7fe2e6c30700 1 -- 172.21.15.17:0/934863134 >> 172.21.15.17:0/934863134 conn(0x7fe2e00613c0 msgr2=0x7fe2e013d310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:08.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.375+0000 7fe2e6c30700 1 -- 172.21.15.17:0/934863134 shutdown_connections 2022-08-01T19:34:08.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.375+0000 7fe2e6c30700 1 -- 172.21.15.17:0/934863134 wait complete. 2022-08-01T19:34:08.380 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' created 2022-08-01T19:34:08.395 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data allow_ec_overwrites true 2022-08-01T19:34:08.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.499+0000 7f549cf8b700 1 Processor -- start 2022-08-01T19:34:08.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.499+0000 7f549cf8b700 1 -- start start 2022-08-01T19:34:08.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.503+0000 7f549cf8b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f54980614b0 0x7f5498061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:08.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.503+0000 7f549cf8b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f54980ad750 0x7f54980b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:08.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.503+0000 7f549cf8b700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54980aee80 con 0x7f54981405d0 2022-08-01T19:34:08.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.503+0000 7f549cf8b700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54980b70b0 con 0x7f54980614b0 2022-08-01T19:34:08.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.503+0000 7f549cf8b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54980af0b0 con 0x7f54980ad750 2022-08-01T19:34:08.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.503+0000 7f54977fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f54980614b0 0x7f5498061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:08.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.503+0000 7f5497fff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f54981405d0 0x7f54981466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56618/0 (socket says 172.21.15.17:56618) 2022-08-01T19:34:08.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.503+0000 7f5497fff700 1 -- 172.21.15.17:0/500151149 learned_addr learned my addr 172.21.15.17:0/500151149 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:08.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.503+0000 7f5496ffd700 1 --2- 172.21.15.17:0/500151149 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f54980ad750 0x7f54980b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:08.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.503+0000 7f54977fe700 1 -- 172.21.15.17:0/500151149 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f54980ad750 msgr2=0x7f54980b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:08.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.503+0000 7f54977fe700 1 --2- 172.21.15.17:0/500151149 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f54980ad750 0x7f54980b9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:08.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.503+0000 7f54977fe700 1 -- 172.21.15.17:0/500151149 >> v1:172.21.15.17:6789/0 conn(0x7f54981405d0 legacy=0x7f54981466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:08.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.503+0000 7f54977fe700 1 -- 172.21.15.17:0/500151149 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54980af350 con 0x7f54980614b0 2022-08-01T19:34:08.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.503+0000 7f54977fe700 1 --2- 172.21.15.17:0/500151149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54980614b0 0x7f5498061890 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f54900032f0 tx=0x7f54900091d0).ready entity=mon.0 client_cookie=8165e70f0cb3c79c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:08.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.503+0000 7f54967fc700 1 -- 172.21.15.17:0/500151149 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5490009ea0 con 0x7f54980614b0 2022-08-01T19:34:08.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.503+0000 7f54967fc700 1 -- 172.21.15.17:0/500151149 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549000b8a0 con 0x7f54980614b0 2022-08-01T19:34:08.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.503+0000 7f54967fc700 1 -- 172.21.15.17:0/500151149 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f549000ba10 con 0x7f54980614b0 2022-08-01T19:34:08.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.503+0000 7f549cf8b700 1 -- 172.21.15.17:0/500151149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54980614b0 msgr2=0x7f5498061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:08.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.503+0000 7f549cf8b700 1 --2- 172.21.15.17:0/500151149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54980614b0 0x7f5498061890 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f54900032f0 tx=0x7f54900091d0).stop 2022-08-01T19:34:08.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.503+0000 7f549cf8b700 1 -- 172.21.15.17:0/500151149 shutdown_connections 2022-08-01T19:34:08.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.503+0000 7f549cf8b700 1 --2- 172.21.15.17:0/500151149 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f54980ad750 0x7f54980b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:08.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.503+0000 7f549cf8b700 1 --2- 172.21.15.17:0/500151149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54980614b0 0x7f5498061890 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:08.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.503+0000 7f549cf8b700 1 -- 172.21.15.17:0/500151149 >> 172.21.15.17:0/500151149 conn(0x7f5498066f70 msgr2=0x7f5498067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:08.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.503+0000 7f549cf8b700 1 -- 172.21.15.17:0/500151149 shutdown_connections 2022-08-01T19:34:08.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.503+0000 7f549cf8b700 1 -- 172.21.15.17:0/500151149 wait complete. 2022-08-01T19:34:08.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.507+0000 7f549cf8b700 1 Processor -- start 2022-08-01T19:34:08.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.507+0000 7f549cf8b700 1 -- start start 2022-08-01T19:34:08.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.507+0000 7f549cf8b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f54980614b0 0x7f54980b8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:08.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.507+0000 7f549cf8b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54980ad750 0x7f5498155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:08.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.507+0000 7f549cf8b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54981405d0 0x7f5498156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:08.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.507+0000 7f549cf8b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549809f640 con 0x7f54981405d0 2022-08-01T19:34:08.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.507+0000 7f549cf8b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54980b37f0 con 0x7f54980ad750 2022-08-01T19:34:08.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.507+0000 7f549cf8b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f549809f390 con 0x7f54980614b0 2022-08-01T19:34:08.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.507+0000 7f5497fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54981405d0 0x7f5498156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:08.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.507+0000 7f5497fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54981405d0 0x7f5498156110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36198/0 (socket says 172.21.15.17:36198) 2022-08-01T19:34:08.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.507+0000 7f5497fff700 1 -- 172.21.15.17:0/32809549 learned_addr learned my addr 172.21.15.17:0/32809549 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:08.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.507+0000 7f54977fe700 1 --2- 172.21.15.17:0/32809549 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f54980614b0 0x7f54980b8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:08.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.507+0000 7f5496ffd700 1 --2- 172.21.15.17:0/32809549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54980ad750 0x7f5498155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:08.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.507+0000 7f5497fff700 1 -- 172.21.15.17:0/32809549 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f54980614b0 msgr2=0x7f54980b8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:08.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.507+0000 7f5497fff700 1 --2- 172.21.15.17:0/32809549 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f54980614b0 0x7f54980b8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:08.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.507+0000 7f5497fff700 1 -- 172.21.15.17:0/32809549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54980ad750 msgr2=0x7f5498155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:08.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.507+0000 7f5497fff700 1 --2- 172.21.15.17:0/32809549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54980ad750 0x7f5498155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:08.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.507+0000 7f5497fff700 1 -- 172.21.15.17:0/32809549 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5490009470 con 0x7f54981405d0 2022-08-01T19:34:08.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.507+0000 7f5497fff700 1 --2- 172.21.15.17:0/32809549 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54981405d0 0x7f5498156110 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f5498143190 tx=0x7f5488007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:08.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.507+0000 7f5494ff9700 1 -- 172.21.15.17:0/32809549 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5488001d70 con 0x7f54981405d0 2022-08-01T19:34:08.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.507+0000 7f5494ff9700 1 -- 172.21.15.17:0/32809549 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54880070d0 con 0x7f54981405d0 2022-08-01T19:34:08.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.507+0000 7f549cf8b700 1 -- 172.21.15.17:0/32809549 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f549809f0f0 con 0x7f54981405d0 2022-08-01T19:34:08.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.507+0000 7f5494ff9700 1 -- 172.21.15.17:0/32809549 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5488007430 con 0x7f54981405d0 2022-08-01T19:34:08.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.507+0000 7f549cf8b700 1 -- 172.21.15.17:0/32809549 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54980b2b20 con 0x7f54981405d0 2022-08-01T19:34:08.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.511+0000 7f549cf8b700 1 -- 172.21.15.17:0/32809549 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54980b3c50 con 0x7f54981405d0 2022-08-01T19:34:08.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.511+0000 7f5494ff9700 1 -- 172.21.15.17:0/32809549 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5488010070 con 0x7f54981405d0 2022-08-01T19:34:08.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.511+0000 7f5494ff9700 1 --2- 172.21.15.17:0/32809549 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5478053950 0x7f5478055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:08.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.511+0000 7f5494ff9700 1 -- 172.21.15.17:0/32809549 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6315+0+0 (secure 0 0 0) 0x7f548806b5b0 con 0x7f54981405d0 2022-08-01T19:34:08.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.511+0000 7f54977fe700 1 --2- 172.21.15.17:0/32809549 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5478053950 0x7f5478055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:08.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.511+0000 7f54977fe700 1 --2- 172.21.15.17:0/32809549 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5478053950 0x7f5478055e10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f549001b000 tx=0x7f5490016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:08.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.519+0000 7f5494ff9700 1 -- 172.21.15.17:0/32809549 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f54980b3c50 con 0x7f54981405d0 2022-08-01T19:34:08.650 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:08.647+0000 7f549cf8b700 1 -- 172.21.15.17:0/32809549 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f54980b3c50 con 0x7f54981405d0 2022-08-01T19:34:09.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.371+0000 7f5494ff9700 1 -- 172.21.15.17:0/32809549 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) v1 ==== 166+0+0 (secure 0 0 0) 0x7f54980b3c50 con 0x7f54981405d0 2022-08-01T19:34:09.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.379+0000 7f549cf8b700 1 -- 172.21.15.17:0/32809549 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5478053950 msgr2=0x7f5478055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:09.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.379+0000 7f549cf8b700 1 --2- 172.21.15.17:0/32809549 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5478053950 0x7f5478055e10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f549001b000 tx=0x7f5490016000).stop 2022-08-01T19:34:09.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.379+0000 7f549cf8b700 1 -- 172.21.15.17:0/32809549 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54981405d0 msgr2=0x7f5498156110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:09.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.379+0000 7f549cf8b700 1 --2- 172.21.15.17:0/32809549 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54981405d0 0x7f5498156110 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f5498143190 tx=0x7f5488007c30).stop 2022-08-01T19:34:09.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.383+0000 7f549cf8b700 1 -- 172.21.15.17:0/32809549 shutdown_connections 2022-08-01T19:34:09.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.383+0000 7f549cf8b700 1 --2- 172.21.15.17:0/32809549 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5478053950 0x7f5478055e10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:09.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.383+0000 7f549cf8b700 1 --2- 172.21.15.17:0/32809549 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54981405d0 0x7f5498156110 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:09.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.383+0000 7f549cf8b700 1 --2- 172.21.15.17:0/32809549 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54980ad750 0x7f5498155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:09.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.383+0000 7f549cf8b700 1 --2- 172.21.15.17:0/32809549 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f54980614b0 0x7f54980b8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:09.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.383+0000 7f549cf8b700 1 -- 172.21.15.17:0/32809549 >> 172.21.15.17:0/32809549 conn(0x7f5498066f70 msgr2=0x7f5498140ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:09.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.383+0000 7f549cf8b700 1 -- 172.21.15.17:0/32809549 shutdown_connections 2022-08-01T19:34:09.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.383+0000 7f549cf8b700 1 -- 172.21.15.17:0/32809549 wait complete. 2022-08-01T19:34:09.389 INFO:teuthology.orchestra.run.smithi017.stderr:set pool 4 allow_ec_overwrites to true 2022-08-01T19:34:09.399 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-08-01T19:34:09.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.507+0000 7f3e11695700 1 Processor -- start 2022-08-01T19:34:09.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.507+0000 7f3e11695700 1 -- start start 2022-08-01T19:34:09.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.507+0000 7f3e11695700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3e0c0b2930 0x7f3e0c147600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:09.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.507+0000 7f3e11695700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3e0c05c620 0x7f3e0c0a8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:09.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.507+0000 7f3e11695700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e0c14ba70 con 0x7f3e0c0b1ea0 2022-08-01T19:34:09.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.507+0000 7f3e11695700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e0c145340 con 0x7f3e0c05c620 2022-08-01T19:34:09.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.507+0000 7f3e11695700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0c0b2d10 con 0x7f3e0c0b2930 2022-08-01T19:34:09.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.507+0000 7f3e10e94700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3e0c05c620 0x7f3e0c0a8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:09.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.507+0000 7f3e10e94700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3e0c05c620 0x7f3e0c0a8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36206/0 (socket says 172.21.15.17:36206) 2022-08-01T19:34:09.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.507+0000 7f3e10e94700 1 -- 172.21.15.17:0/3364051749 learned_addr learned my addr 172.21.15.17:0/3364051749 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:09.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.507+0000 7f3e0bfff700 1 --2- 172.21.15.17:0/3364051749 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3e0c0b2930 0x7f3e0c147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:09.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.507+0000 7f3e10e94700 1 -- 172.21.15.17:0/3364051749 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3e0c0b2930 msgr2=0x7f3e0c147600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:09.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.507+0000 7f3e10e94700 1 --2- 172.21.15.17:0/3364051749 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3e0c0b2930 0x7f3e0c147600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:09.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.507+0000 7f3e0affd700 1 -- 172.21.15.17:0/3364051749 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 689930971 0 0) 0x7f3e0c14ba70 con 0x7f3e0c0b1ea0 2022-08-01T19:34:09.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.507+0000 7f3e10e94700 1 -- 172.21.15.17:0/3364051749 >> v1:172.21.15.77:6789/0 conn(0x7f3e0c0b1ea0 legacy=0x7f3e0c14b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:09.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.507+0000 7f3e10e94700 1 -- 172.21.15.17:0/3364051749 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e0c14ac60 con 0x7f3e0c05c620 2022-08-01T19:34:09.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.507+0000 7f3e10e94700 1 --2- 172.21.15.17:0/3364051749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e0c05c620 0x7f3e0c0a8d10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f3e040032f0 tx=0x7f3e04009290).ready entity=mon.0 client_cookie=2a75295eba09ffaa server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:09.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.507+0000 7f3e0affd700 1 -- 172.21.15.17:0/3364051749 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e0400b8a0 con 0x7f3e0c05c620 2022-08-01T19:34:09.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.507+0000 7f3e0affd700 1 -- 172.21.15.17:0/3364051749 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0400ba00 con 0x7f3e0c05c620 2022-08-01T19:34:09.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.507+0000 7f3e0affd700 1 -- 172.21.15.17:0/3364051749 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e0400bb70 con 0x7f3e0c05c620 2022-08-01T19:34:09.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.511+0000 7f3e11695700 1 -- 172.21.15.17:0/3364051749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e0c05c620 msgr2=0x7f3e0c0a8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:09.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.511+0000 7f3e11695700 1 --2- 172.21.15.17:0/3364051749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e0c05c620 0x7f3e0c0a8d10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f3e040032f0 tx=0x7f3e04009290).stop 2022-08-01T19:34:09.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.511+0000 7f3e11695700 1 -- 172.21.15.17:0/3364051749 shutdown_connections 2022-08-01T19:34:09.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.511+0000 7f3e11695700 1 --2- 172.21.15.17:0/3364051749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e0c05c620 0x7f3e0c0a8d10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:09.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.511+0000 7f3e11695700 1 --2- 172.21.15.17:0/3364051749 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3e0c0b2930 0x7f3e0c147600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:09.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.511+0000 7f3e11695700 1 -- 172.21.15.17:0/3364051749 >> 172.21.15.17:0/3364051749 conn(0x7f3e0c0612d0 msgr2=0x7f3e0c062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:09.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.511+0000 7f3e11695700 1 -- 172.21.15.17:0/3364051749 shutdown_connections 2022-08-01T19:34:09.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.511+0000 7f3e11695700 1 -- 172.21.15.17:0/3364051749 wait complete. 2022-08-01T19:34:09.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.511+0000 7f3e11695700 1 Processor -- start 2022-08-01T19:34:09.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.511+0000 7f3e11695700 1 -- start start 2022-08-01T19:34:09.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.511+0000 7f3e11695700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e0c05c620 0x7f3e0c1454a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:09.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.511+0000 7f3e11695700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3e0c0b1ea0 0x7f3e0c159f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:09.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.511+0000 7f3e11695700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e0c0b2930 0x7f3e0c0ac160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:09.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.511+0000 7f3e11695700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0c0aa8a0 con 0x7f3e0c05c620 2022-08-01T19:34:09.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.511+0000 7f3e11695700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0c0aaad0 con 0x7f3e0c0b2930 2022-08-01T19:34:09.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.511+0000 7f3e11695700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0c0a9ef0 con 0x7f3e0c0b1ea0 2022-08-01T19:34:09.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.511+0000 7f3e0bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e0c05c620 0x7f3e0c1454a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:09.525 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.511+0000 7f3e0bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e0c05c620 0x7f3e0c1454a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36208/0 (socket says 172.21.15.17:36208) 2022-08-01T19:34:09.525 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.511+0000 7f3e0bfff700 1 -- 172.21.15.17:0/2518352797 learned_addr learned my addr 172.21.15.17:0/2518352797 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:09.525 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.511+0000 7f3e0b7fe700 1 --2- 172.21.15.17:0/2518352797 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3e0c0b1ea0 0x7f3e0c159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:09.525 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.511+0000 7f3e10e94700 1 --2- 172.21.15.17:0/2518352797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e0c0b2930 0x7f3e0c0ac160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:09.526 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.511+0000 7f3e0bfff700 1 -- 172.21.15.17:0/2518352797 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3e0c0b1ea0 msgr2=0x7f3e0c159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:09.526 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.511+0000 7f3e0bfff700 1 --2- 172.21.15.17:0/2518352797 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3e0c0b1ea0 0x7f3e0c159f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:09.526 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.511+0000 7f3e0bfff700 1 -- 172.21.15.17:0/2518352797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e0c0b2930 msgr2=0x7f3e0c0ac160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:09.526 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.511+0000 7f3e0bfff700 1 --2- 172.21.15.17:0/2518352797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e0c0b2930 0x7f3e0c0ac160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:09.526 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.511+0000 7f3e0bfff700 1 -- 172.21.15.17:0/2518352797 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e04009530 con 0x7f3e0c05c620 2022-08-01T19:34:09.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.511+0000 7f3e0bfff700 1 --2- 172.21.15.17:0/2518352797 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e0c05c620 0x7f3e0c1454a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f3e0000c000 tx=0x7f3e00006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:09.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.515+0000 7f3e097fa700 1 -- 172.21.15.17:0/2518352797 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e00010780 con 0x7f3e0c05c620 2022-08-01T19:34:09.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.515+0000 7f3e097fa700 1 -- 172.21.15.17:0/2518352797 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e000108e0 con 0x7f3e0c05c620 2022-08-01T19:34:09.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.515+0000 7f3e11695700 1 -- 172.21.15.17:0/2518352797 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e0c15a790 con 0x7f3e0c05c620 2022-08-01T19:34:09.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.515+0000 7f3e097fa700 1 -- 172.21.15.17:0/2518352797 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e00010c40 con 0x7f3e0c05c620 2022-08-01T19:34:09.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.515+0000 7f3e11695700 1 -- 172.21.15.17:0/2518352797 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e0c15a440 con 0x7f3e0c05c620 2022-08-01T19:34:09.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.515+0000 7f3e097fa700 1 -- 172.21.15.17:0/2518352797 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3e0001c460 con 0x7f3e0c05c620 2022-08-01T19:34:09.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.515+0000 7f3e11695700 1 -- 172.21.15.17:0/2518352797 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e0c0574e0 con 0x7f3e0c05c620 2022-08-01T19:34:09.529 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.515+0000 7f3e097fa700 1 --2- 172.21.15.17:0/2518352797 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3dec053980 0x7f3dec055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:09.529 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.515+0000 7f3e097fa700 1 -- 172.21.15.17:0/2518352797 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6315+0+0 (secure 0 0 0) 0x7f3e0006c3d0 con 0x7f3e0c05c620 2022-08-01T19:34:09.529 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.515+0000 7f3e0b7fe700 1 --2- 172.21.15.17:0/2518352797 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3dec053980 0x7f3dec055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:09.530 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.515+0000 7f3e0b7fe700 1 --2- 172.21.15.17:0/2518352797 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3dec053980 0x7f3dec055e40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f3dfc00d000 tx=0x7f3dfc007b90).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:09.530 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.523+0000 7f3e097fa700 1 -- 172.21.15.17:0/2518352797 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3e00043100 con 0x7f3e0c05c620 2022-08-01T19:34:09.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.687+0000 7f3e11695700 1 -- 172.21.15.17:0/2518352797 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f3e0c0aacd0 con 0x7f3e0c05c620 2022-08-01T19:34:09.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.695+0000 7f3e097fa700 1 -- 172.21.15.17:0/2518352797 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 3 and data pool 4 v5) v1 ==== 187+0+0 (secure 0 0 0) 0x7f3e00046720 con 0x7f3e0c05c620 2022-08-01T19:34:09.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.699+0000 7f3e11695700 1 -- 172.21.15.17:0/2518352797 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3dec053980 msgr2=0x7f3dec055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:09.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.699+0000 7f3e11695700 1 --2- 172.21.15.17:0/2518352797 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3dec053980 0x7f3dec055e40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f3dfc00d000 tx=0x7f3dfc007b90).stop 2022-08-01T19:34:09.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.699+0000 7f3e11695700 1 -- 172.21.15.17:0/2518352797 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e0c05c620 msgr2=0x7f3e0c1454a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:09.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.699+0000 7f3e11695700 1 --2- 172.21.15.17:0/2518352797 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e0c05c620 0x7f3e0c1454a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f3e0000c000 tx=0x7f3e00006df0).stop 2022-08-01T19:34:09.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.699+0000 7f3e11695700 1 -- 172.21.15.17:0/2518352797 shutdown_connections 2022-08-01T19:34:09.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.699+0000 7f3e11695700 1 --2- 172.21.15.17:0/2518352797 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3dec053980 0x7f3dec055e40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:09.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.699+0000 7f3e11695700 1 --2- 172.21.15.17:0/2518352797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e0c0b2930 0x7f3e0c0ac160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:09.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.699+0000 7f3e11695700 1 --2- 172.21.15.17:0/2518352797 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3e0c0b1ea0 0x7f3e0c159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:09.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.699+0000 7f3e11695700 1 --2- 172.21.15.17:0/2518352797 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e0c05c620 0x7f3e0c1454a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:09.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.699+0000 7f3e11695700 1 -- 172.21.15.17:0/2518352797 >> 172.21.15.17:0/2518352797 conn(0x7f3e0c0612d0 msgr2=0x7f3e0c0b5030 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:09.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.699+0000 7f3e11695700 1 -- 172.21.15.17:0/2518352797 shutdown_connections 2022-08-01T19:34:09.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.699+0000 7f3e11695700 1 -- 172.21.15.17:0/2518352797 wait complete. 2022-08-01T19:34:09.708 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 3 and data pool 4 2022-08-01T19:34:09.721 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:34:09.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.807+0000 7fece88b4700 1 Processor -- start 2022-08-01T19:34:09.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.807+0000 7fece88b4700 1 -- start start 2022-08-01T19:34:09.816 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.807+0000 7fece88b4700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fece40b08a0 0x7fece4142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:09.817 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.807+0000 7fece88b4700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fece4143ab0 con 0x7fece40b3260 2022-08-01T19:34:09.817 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.807+0000 7fece88b4700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fece40b0c80 con 0x7fece40ad000 2022-08-01T19:34:09.817 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.807+0000 7fece88b4700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fece40ae810 con 0x7fece40b08a0 2022-08-01T19:34:09.818 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.807+0000 7fecdbfff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fece40b3260 0x7fece41433a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56642/0 (socket says 172.21.15.17:56642) 2022-08-01T19:34:09.818 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.811+0000 7fecdbfff700 1 -- 172.21.15.17:0/2325357538 learned_addr learned my addr 172.21.15.17:0/2325357538 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:09.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.811+0000 7fece37fe700 1 --2- 172.21.15.17:0/2325357538 >> v2:172.21.15.77:3300/0 conn(0x7fece40b08a0 0x7fece4142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:09.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.811+0000 7fece2ffd700 1 -- 172.21.15.17:0/2325357538 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2064123428 0 0) 0x7fece4143ab0 con 0x7fece40b3260 2022-08-01T19:34:09.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.811+0000 7fece2ffd700 1 -- 172.21.15.17:0/2325357538 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecc4003260 con 0x7fece40b3260 2022-08-01T19:34:09.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.811+0000 7fece37fe700 1 -- 172.21.15.17:0/2325357538 >> v1:172.21.15.77:6789/0 conn(0x7fece40ad000 legacy=0x7fece413fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:09.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.811+0000 7fece37fe700 1 -- 172.21.15.17:0/2325357538 >> v1:172.21.15.17:6789/0 conn(0x7fece40b3260 legacy=0x7fece41433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:09.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.811+0000 7fece37fe700 1 -- 172.21.15.17:0/2325357538 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece40aeab0 con 0x7fece40b08a0 2022-08-01T19:34:09.822 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.811+0000 7fece37fe700 1 --2- 172.21.15.17:0/2325357538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fece40b08a0 0x7fece4142eb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fecd4002f70 tx=0x7fecd4006df0).ready entity=mon.1 client_cookie=a58f18f01524c9a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:09.822 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.811+0000 7fece2ffd700 1 -- 172.21.15.17:0/2325357538 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fecd400d5c0 con 0x7fece40b08a0 2022-08-01T19:34:09.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.811+0000 7fece2ffd700 1 -- 172.21.15.17:0/2325357538 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd4000e90 con 0x7fece40b08a0 2022-08-01T19:34:09.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.811+0000 7fece2ffd700 1 -- 172.21.15.17:0/2325357538 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fecd400d910 con 0x7fece40b08a0 2022-08-01T19:34:09.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.811+0000 7fece88b4700 1 -- 172.21.15.17:0/2325357538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fece40b08a0 msgr2=0x7fece4142eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:09.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.811+0000 7fece88b4700 1 --2- 172.21.15.17:0/2325357538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fece40b08a0 0x7fece4142eb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fecd4002f70 tx=0x7fecd4006df0).stop 2022-08-01T19:34:09.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.811+0000 7fece88b4700 1 -- 172.21.15.17:0/2325357538 shutdown_connections 2022-08-01T19:34:09.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.811+0000 7fece88b4700 1 --2- 172.21.15.17:0/2325357538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fece40b08a0 0x7fece4142eb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:09.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.811+0000 7fece88b4700 1 -- 172.21.15.17:0/2325357538 >> 172.21.15.17:0/2325357538 conn(0x7fece40612d0 msgr2=0x7fece4062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:09.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.811+0000 7fece88b4700 1 -- 172.21.15.17:0/2325357538 shutdown_connections 2022-08-01T19:34:09.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.811+0000 7fece88b4700 1 -- 172.21.15.17:0/2325357538 wait complete. 2022-08-01T19:34:09.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.815+0000 7fece88b4700 1 Processor -- start 2022-08-01T19:34:09.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.815+0000 7fece88b4700 1 -- start start 2022-08-01T19:34:09.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.815+0000 7fece88b4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fece40ad000 0x7fece40ac2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:09.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.815+0000 7fece88b4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fece40b08a0 0x7fece413ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:09.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.815+0000 7fece88b4700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fece40b3260 0x7fece413b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:09.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.815+0000 7fece88b4700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece4067bd0 con 0x7fece40b08a0 2022-08-01T19:34:09.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.815+0000 7fece88b4700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece4066490 con 0x7fece40ad000 2022-08-01T19:34:09.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.815+0000 7fece88b4700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fece413b630 con 0x7fece40b3260 2022-08-01T19:34:09.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.815+0000 7fecdbfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fece40b08a0 0x7fece413ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:09.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.815+0000 7fece37fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fece40ad000 0x7fece40ac2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:09.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.815+0000 7fecdbfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fece40b08a0 0x7fece413ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36224/0 (socket says 172.21.15.17:36224) 2022-08-01T19:34:09.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.815+0000 7fecdbfff700 1 -- 172.21.15.17:0/446523229 learned_addr learned my addr 172.21.15.17:0/446523229 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:09.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.815+0000 7fece3fff700 1 --2- 172.21.15.17:0/446523229 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fece40b3260 0x7fece413b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:09.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.819+0000 7fecdbfff700 1 -- 172.21.15.17:0/446523229 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fece40b3260 msgr2=0x7fece413b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:09.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.819+0000 7fecdbfff700 1 --2- 172.21.15.17:0/446523229 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fece40b3260 0x7fece413b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:09.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.819+0000 7fecdbfff700 1 -- 172.21.15.17:0/446523229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fece40ad000 msgr2=0x7fece40ac2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:09.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.819+0000 7fecdbfff700 1 --2- 172.21.15.17:0/446523229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fece40ad000 0x7fece40ac2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:09.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.819+0000 7fecdbfff700 1 -- 172.21.15.17:0/446523229 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd400b180 con 0x7fece40b08a0 2022-08-01T19:34:09.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.819+0000 7fece37fe700 1 --2- 172.21.15.17:0/446523229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fece40ad000 0x7fece40ac2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:34:09.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.819+0000 7fece3fff700 1 --2- 172.21.15.17:0/446523229 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fece40b3260 0x7fece413b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:34:09.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.819+0000 7fecdbfff700 1 --2- 172.21.15.17:0/446523229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fece40b08a0 0x7fece413ac50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7feccc001110 tx=0x7feccc007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:09.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.819+0000 7fece17fa700 1 -- 172.21.15.17:0/446523229 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feccc0070d0 con 0x7fece40b08a0 2022-08-01T19:34:09.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.819+0000 7fece17fa700 1 -- 172.21.15.17:0/446523229 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feccc007230 con 0x7fece40b08a0 2022-08-01T19:34:09.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.819+0000 7fece88b4700 1 -- 172.21.15.17:0/446523229 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fece4067930 con 0x7fece40b08a0 2022-08-01T19:34:09.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.819+0000 7fece17fa700 1 -- 172.21.15.17:0/446523229 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feccc005610 con 0x7fece40b08a0 2022-08-01T19:34:09.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.819+0000 7fece88b4700 1 -- 172.21.15.17:0/446523229 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece4066920 con 0x7fece40b08a0 2022-08-01T19:34:09.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.819+0000 7fece88b4700 1 -- 172.21.15.17:0/446523229 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fece4066ae0 con 0x7fece40b08a0 2022-08-01T19:34:09.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.819+0000 7fece17fa700 1 -- 172.21.15.17:0/446523229 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7feccc010070 con 0x7fece40b08a0 2022-08-01T19:34:09.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.823+0000 7fece17fa700 1 --2- 172.21.15.17:0/446523229 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fecc8053b00 0x7fecc8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:09.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.823+0000 7fece17fa700 1 -- 172.21.15.17:0/446523229 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7feccc06b6b0 con 0x7fece40b08a0 2022-08-01T19:34:09.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.823+0000 7fece37fe700 1 --2- 172.21.15.17:0/446523229 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fecc8053b00 0x7fecc8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:09.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.823+0000 7fece37fe700 1 --2- 172.21.15.17:0/446523229 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fecc8053b00 0x7fecc8055fc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fecd401a000 tx=0x7fecd4014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:09.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.827+0000 7fece17fa700 1 -- 172.21.15.17:0/446523229 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7feccc042360 con 0x7fece40b08a0 2022-08-01T19:34:09.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.951+0000 7fece88b4700 1 -- 172.21.15.17:0/446523229 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fece40574e0 con 0x7fece40b08a0 2022-08-01T19:34:09.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.955+0000 7fece17fa700 1 -- 172.21.15.17:0/446523229 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16299 (secure 0 0 0) 0x7feccc018030 con 0x7fece40b08a0 2022-08-01T19:34:09.959 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:09.959 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:34:09.692404+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:34:04.635473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-08-01T19:34:07.409434+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:34:09.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.955+0000 7fece88b4700 1 -- 172.21.15.17:0/446523229 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fecc8053b00 msgr2=0x7fecc8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:09.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.955+0000 7fece88b4700 1 --2- 172.21.15.17:0/446523229 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fecc8053b00 0x7fecc8055fc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fecd401a000 tx=0x7fecd4014000).stop 2022-08-01T19:34:09.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.955+0000 7fece88b4700 1 -- 172.21.15.17:0/446523229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fece40b08a0 msgr2=0x7fece413ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:09.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.955+0000 7fece88b4700 1 --2- 172.21.15.17:0/446523229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fece40b08a0 0x7fece413ac50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7feccc001110 tx=0x7feccc007ad0).stop 2022-08-01T19:34:09.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.955+0000 7fece88b4700 1 -- 172.21.15.17:0/446523229 shutdown_connections 2022-08-01T19:34:09.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.955+0000 7fece88b4700 1 --2- 172.21.15.17:0/446523229 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fecc8053b00 0x7fecc8055fc0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:09.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.955+0000 7fece88b4700 1 --2- 172.21.15.17:0/446523229 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fece40b3260 0x7fece413b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:09.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.955+0000 7fece88b4700 1 --2- 172.21.15.17:0/446523229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fece40b08a0 0x7fece413ac50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:09.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.955+0000 7fece88b4700 1 --2- 172.21.15.17:0/446523229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fece40ad000 0x7fece40ac2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:09.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.955+0000 7fece88b4700 1 -- 172.21.15.17:0/446523229 >> 172.21.15.17:0/446523229 conn(0x7fece40612d0 msgr2=0x7fece41437d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:09.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.955+0000 7fece88b4700 1 -- 172.21.15.17:0/446523229 shutdown_connections 2022-08-01T19:34:09.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:09.955+0000 7fece88b4700 1 -- 172.21.15.17:0/446523229 wait complete. 2022-08-01T19:34:09.973 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:34:10.066 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.063+0000 7f566af2b700 1 Processor -- start 2022-08-01T19:34:10.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.063+0000 7f566af2b700 1 -- start start 2022-08-01T19:34:10.069 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.063+0000 7f566af2b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f56640614b0 0x7f5664061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:10.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.063+0000 7f566af2b700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f56641405d0 0x7f5664140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:10.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.063+0000 7f566af2b700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56640aee80 con 0x7f56640ad750 2022-08-01T19:34:10.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.063+0000 7f566af2b700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56640b70b0 con 0x7f56640614b0 2022-08-01T19:34:10.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.063+0000 7f566af2b700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56640af0b0 con 0x7f56641405d0 2022-08-01T19:34:10.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.063+0000 7f5669f29700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f56640614b0 0x7f5664061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:10.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.063+0000 7f5669728700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f56640ad750 0x7f56640b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56656/0 (socket says 172.21.15.17:56656) 2022-08-01T19:34:10.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.063+0000 7f5669728700 1 -- 172.21.15.17:0/1065628709 learned_addr learned my addr 172.21.15.17:0/1065628709 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:10.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.063+0000 7f566a72a700 1 --2- 172.21.15.17:0/1065628709 >> v2:172.21.15.77:3300/0 conn(0x7f56641405d0 0x7f5664140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:10.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.063+0000 7f5668f27700 1 -- 172.21.15.17:0/1065628709 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2196114141 0 0) 0x7f56640aee80 con 0x7f56640ad750 2022-08-01T19:34:10.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.063+0000 7f5668f27700 1 -- 172.21.15.17:0/1065628709 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f564c003260 con 0x7f56640ad750 2022-08-01T19:34:10.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.063+0000 7f5669f29700 1 -- 172.21.15.17:0/1065628709 >> v2:172.21.15.77:3300/0 conn(0x7f56641405d0 msgr2=0x7f5664140a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:10.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.063+0000 7f5669f29700 1 --2- 172.21.15.17:0/1065628709 >> v2:172.21.15.77:3300/0 conn(0x7f56641405d0 0x7f5664140a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:10.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.063+0000 7f5669f29700 1 -- 172.21.15.17:0/1065628709 >> v1:172.21.15.17:6789/0 conn(0x7f56640ad750 legacy=0x7f56640b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:10.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.063+0000 7f5669f29700 1 -- 172.21.15.17:0/1065628709 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56640af350 con 0x7f56640614b0 2022-08-01T19:34:10.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.063+0000 7f5669f29700 1 --2- 172.21.15.17:0/1065628709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56640614b0 0x7f5664061890 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f56640aed30 tx=0x7f5660006df0).ready entity=mon.0 client_cookie=83957f7189703dd3 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:10.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.063+0000 7f5668f27700 1 -- 172.21.15.17:0/1065628709 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f566000d4f0 con 0x7f56640614b0 2022-08-01T19:34:10.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.063+0000 7f5668f27700 1 -- 172.21.15.17:0/1065628709 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5660000e90 con 0x7f56640614b0 2022-08-01T19:34:10.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.063+0000 7f5668f27700 1 -- 172.21.15.17:0/1065628709 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f566000d840 con 0x7f56640614b0 2022-08-01T19:34:10.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.063+0000 7f566af2b700 1 -- 172.21.15.17:0/1065628709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56640614b0 msgr2=0x7f5664061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:10.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.063+0000 7f566af2b700 1 --2- 172.21.15.17:0/1065628709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56640614b0 0x7f5664061890 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f56640aed30 tx=0x7f5660006df0).stop 2022-08-01T19:34:10.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.063+0000 7f566af2b700 1 -- 172.21.15.17:0/1065628709 shutdown_connections 2022-08-01T19:34:10.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.063+0000 7f566af2b700 1 --2- 172.21.15.17:0/1065628709 >> v2:172.21.15.77:3300/0 conn(0x7f56641405d0 0x7f5664140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:10.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.063+0000 7f566af2b700 1 --2- 172.21.15.17:0/1065628709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56640614b0 0x7f5664061890 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:10.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.063+0000 7f566af2b700 1 -- 172.21.15.17:0/1065628709 >> 172.21.15.17:0/1065628709 conn(0x7f5664066f70 msgr2=0x7f5664067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:10.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.063+0000 7f566af2b700 1 -- 172.21.15.17:0/1065628709 shutdown_connections 2022-08-01T19:34:10.076 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.063+0000 7f566af2b700 1 -- 172.21.15.17:0/1065628709 wait complete. 2022-08-01T19:34:10.079 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.067+0000 7f566af2b700 1 Processor -- start 2022-08-01T19:34:10.079 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.067+0000 7f566af2b700 1 -- start start 2022-08-01T19:34:10.079 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.067+0000 7f566af2b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56640614b0 0x7f5664144ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:10.079 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.067+0000 7f566af2b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56640ad750 0x7f5664142700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:10.080 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.067+0000 7f566af2b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f56641405d0 0x7f5664142bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:10.080 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.067+0000 7f566af2b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56641454f0 con 0x7f56640ad750 2022-08-01T19:34:10.080 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.067+0000 7f566af2b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56640b3ff0 con 0x7f56640614b0 2022-08-01T19:34:10.080 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.067+0000 7f566af2b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5664153280 con 0x7f56641405d0 2022-08-01T19:34:10.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.067+0000 7f5669728700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56640ad750 0x7f5664142700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:10.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.067+0000 7f5669728700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56640ad750 0x7f5664142700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36238/0 (socket says 172.21.15.17:36238) 2022-08-01T19:34:10.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.067+0000 7f5669728700 1 -- 172.21.15.17:0/2586478624 learned_addr learned my addr 172.21.15.17:0/2586478624 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:10.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.067+0000 7f5669f29700 1 --2- 172.21.15.17:0/2586478624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56640614b0 0x7f5664144ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:10.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.067+0000 7f566a72a700 1 --2- 172.21.15.17:0/2586478624 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f56641405d0 0x7f5664142bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:10.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.067+0000 7f5669728700 1 -- 172.21.15.17:0/2586478624 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f56641405d0 msgr2=0x7f5664142bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:10.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.067+0000 7f5669728700 1 --2- 172.21.15.17:0/2586478624 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f56641405d0 0x7f5664142bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:10.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.067+0000 7f5669728700 1 -- 172.21.15.17:0/2586478624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56640614b0 msgr2=0x7f5664144ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:10.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.067+0000 7f5669728700 1 --2- 172.21.15.17:0/2586478624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56640614b0 0x7f5664144ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:10.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.067+0000 7f5669728700 1 -- 172.21.15.17:0/2586478624 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f566000b180 con 0x7f56640ad750 2022-08-01T19:34:10.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.067+0000 7f5669728700 1 --2- 172.21.15.17:0/2586478624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56640ad750 0x7f5664142700 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f5654001110 tx=0x7f5654007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:10.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.067+0000 7f5652ffd700 1 -- 172.21.15.17:0/2586478624 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f565400ce80 con 0x7f56640ad750 2022-08-01T19:34:10.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.067+0000 7f5652ffd700 1 -- 172.21.15.17:0/2586478624 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56540070d0 con 0x7f56640ad750 2022-08-01T19:34:10.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.067+0000 7f5652ffd700 1 -- 172.21.15.17:0/2586478624 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5654007430 con 0x7f56640ad750 2022-08-01T19:34:10.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.067+0000 7f566af2b700 1 -- 172.21.15.17:0/2586478624 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5664140fb0 con 0x7f56640ad750 2022-08-01T19:34:10.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.067+0000 7f566af2b700 1 -- 172.21.15.17:0/2586478624 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5664152f90 con 0x7f56640ad750 2022-08-01T19:34:10.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.071+0000 7f566af2b700 1 -- 172.21.15.17:0/2586478624 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56640af5f0 con 0x7f56640ad750 2022-08-01T19:34:10.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.071+0000 7f5652ffd700 1 -- 172.21.15.17:0/2586478624 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5654010070 con 0x7f56640ad750 2022-08-01T19:34:10.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.071+0000 7f5652ffd700 1 --2- 172.21.15.17:0/2586478624 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f56440539e0 0x7f5644055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:10.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.071+0000 7f5652ffd700 1 -- 172.21.15.17:0/2586478624 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f565406b5b0 con 0x7f56640ad750 2022-08-01T19:34:10.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.071+0000 7f5669f29700 1 --2- 172.21.15.17:0/2586478624 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f56440539e0 0x7f5644055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:10.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.075+0000 7f5669f29700 1 --2- 172.21.15.17:0/2586478624 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f56440539e0 0x7f5644055ea0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f566001a000 tx=0x7f5660014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:10.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.079+0000 7f5652ffd700 1 -- 172.21.15.17:0/2586478624 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5654042260 con 0x7f56640ad750 2022-08-01T19:34:10.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.203+0000 7f566af2b700 1 -- 172.21.15.17:0/2586478624 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5664153440 con 0x7f56640ad750 2022-08-01T19:34:10.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.207+0000 7f5652ffd700 1 -- 172.21.15.17:0/2586478624 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f5654018070 con 0x7f56640ad750 2022-08-01T19:34:10.211 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:10.211 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:34:09.692404+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:34:04.635473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-08-01T19:34:07.409434+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:34:10.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.207+0000 7f566af2b700 1 -- 172.21.15.17:0/2586478624 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f56440539e0 msgr2=0x7f5644055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:10.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.207+0000 7f566af2b700 1 --2- 172.21.15.17:0/2586478624 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f56440539e0 0x7f5644055ea0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f566001a000 tx=0x7f5660014000).stop 2022-08-01T19:34:10.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.207+0000 7f566af2b700 1 -- 172.21.15.17:0/2586478624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56640ad750 msgr2=0x7f5664142700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:10.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.207+0000 7f566af2b700 1 --2- 172.21.15.17:0/2586478624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56640ad750 0x7f5664142700 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f5654001110 tx=0x7f5654007a40).stop 2022-08-01T19:34:10.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.207+0000 7f566af2b700 1 -- 172.21.15.17:0/2586478624 shutdown_connections 2022-08-01T19:34:10.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.207+0000 7f566af2b700 1 --2- 172.21.15.17:0/2586478624 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f56440539e0 0x7f5644055ea0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:10.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.207+0000 7f566af2b700 1 --2- 172.21.15.17:0/2586478624 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f56641405d0 0x7f5664142bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:10.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.207+0000 7f566af2b700 1 --2- 172.21.15.17:0/2586478624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56640ad750 0x7f5664142700 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:10.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.207+0000 7f566af2b700 1 --2- 172.21.15.17:0/2586478624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56640614b0 0x7f5664144ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:10.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.207+0000 7f566af2b700 1 -- 172.21.15.17:0/2586478624 >> 172.21.15.17:0/2586478624 conn(0x7f5664066f70 msgr2=0x7f56640b85d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:10.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.207+0000 7f566af2b700 1 -- 172.21.15.17:0/2586478624 shutdown_connections 2022-08-01T19:34:10.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.207+0000 7f566af2b700 1 -- 172.21.15.17:0/2586478624 wait complete. 2022-08-01T19:34:10.226 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-08-01T19:34:10.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.331+0000 7f74c15f3700 1 Processor -- start 2022-08-01T19:34:10.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.331+0000 7f74c15f3700 1 -- start start 2022-08-01T19:34:10.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.335+0000 7f74c15f3700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f74bc0b2050 0x7f74bc0b1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:10.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.335+0000 7f74c15f3700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f74bc0b0140 0x7f74bc0b2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:10.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.335+0000 7f74c15f3700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74bc0b05b0 con 0x7f74bc0a7250 2022-08-01T19:34:10.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.335+0000 7f74c15f3700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74bc0b2920 con 0x7f74bc0b0140 2022-08-01T19:34:10.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.335+0000 7f74c15f3700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74bc0b07e0 con 0x7f74bc0b2050 2022-08-01T19:34:10.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.335+0000 7f74ba7fc700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f74bc0b0140 0x7f74bc0b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:10.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.335+0000 7f74ba7fc700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f74bc0b0140 0x7f74bc0b2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36244/0 (socket says 172.21.15.17:36244) 2022-08-01T19:34:10.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.335+0000 7f74bb7fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f74bc0a7250 0x7f74bc1413e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56674/0 (socket says 172.21.15.17:56674) 2022-08-01T19:34:10.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.335+0000 7f74ba7fc700 1 -- 172.21.15.17:0/2111577817 learned_addr learned my addr 172.21.15.17:0/2111577817 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:10.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.335+0000 7f74baffd700 1 --2- 172.21.15.17:0/2111577817 >> v2:172.21.15.77:3300/0 conn(0x7f74bc0b2050 0x7f74bc0b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:10.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.335+0000 7f74ba7fc700 1 -- 172.21.15.17:0/2111577817 >> v2:172.21.15.77:3300/0 conn(0x7f74bc0b2050 msgr2=0x7f74bc0b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:10.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.335+0000 7f74ba7fc700 1 --2- 172.21.15.17:0/2111577817 >> v2:172.21.15.77:3300/0 conn(0x7f74bc0b2050 0x7f74bc0b1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:10.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.335+0000 7f74ba7fc700 1 -- 172.21.15.17:0/2111577817 >> v1:172.21.15.17:6789/0 conn(0x7f74bc0a7250 legacy=0x7f74bc1413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:10.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.335+0000 7f74b9ffb700 1 -- 172.21.15.17:0/2111577817 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2320303856 0 0) 0x7f74bc0b05b0 con 0x7f74bc0a7250 2022-08-01T19:34:10.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.335+0000 7f74ba7fc700 1 -- 172.21.15.17:0/2111577817 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74bc0b0a80 con 0x7f74bc0b0140 2022-08-01T19:34:10.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.335+0000 7f74baffd700 1 --2- 172.21.15.17:0/2111577817 >> v2:172.21.15.77:3300/0 conn(0x7f74bc0b2050 0x7f74bc0b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:34:10.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.335+0000 7f74ba7fc700 1 --2- 172.21.15.17:0/2111577817 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74bc0b0140 0x7f74bc0b2430 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f74ac0032f0 tx=0x7f74ac009290).ready entity=mon.0 client_cookie=5a77ddad7d1781c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:10.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.335+0000 7f74b9ffb700 1 -- 172.21.15.17:0/2111577817 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74ac00b870 con 0x7f74bc0b0140 2022-08-01T19:34:10.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.335+0000 7f74b9ffb700 1 -- 172.21.15.17:0/2111577817 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74ac00b9d0 con 0x7f74bc0b0140 2022-08-01T19:34:10.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.335+0000 7f74b9ffb700 1 -- 172.21.15.17:0/2111577817 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74ac00bb40 con 0x7f74bc0b0140 2022-08-01T19:34:10.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.335+0000 7f74c15f3700 1 -- 172.21.15.17:0/2111577817 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74bc0b0140 msgr2=0x7f74bc0b2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:10.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.335+0000 7f74c15f3700 1 --2- 172.21.15.17:0/2111577817 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74bc0b0140 0x7f74bc0b2430 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f74ac0032f0 tx=0x7f74ac009290).stop 2022-08-01T19:34:10.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.335+0000 7f74c15f3700 1 -- 172.21.15.17:0/2111577817 shutdown_connections 2022-08-01T19:34:10.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.335+0000 7f74c15f3700 1 --2- 172.21.15.17:0/2111577817 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74bc0b0140 0x7f74bc0b2430 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:10.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.335+0000 7f74c15f3700 1 --2- 172.21.15.17:0/2111577817 >> v2:172.21.15.77:3300/0 conn(0x7f74bc0b2050 0x7f74bc0b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:10.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.335+0000 7f74c15f3700 1 -- 172.21.15.17:0/2111577817 >> 172.21.15.17:0/2111577817 conn(0x7f74bc05f840 msgr2=0x7f74bc05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:10.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.335+0000 7f74c15f3700 1 -- 172.21.15.17:0/2111577817 shutdown_connections 2022-08-01T19:34:10.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.335+0000 7f74c15f3700 1 -- 172.21.15.17:0/2111577817 wait complete. 2022-08-01T19:34:10.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.339+0000 7f74c15f3700 1 Processor -- start 2022-08-01T19:34:10.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.339+0000 7f74c15f3700 1 -- start start 2022-08-01T19:34:10.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.339+0000 7f74c15f3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74bc0a7250 0x7f74bc156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:10.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.339+0000 7f74c15f3700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f74bc0b0140 0x7f74bc1570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:10.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.339+0000 7f74c15f3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74bc0b2050 0x7f74bc0b4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:10.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.339+0000 7f74c15f3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc0b51b0 con 0x7f74bc0b2050 2022-08-01T19:34:10.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.339+0000 7f74c15f3700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc1576f0 con 0x7f74bc0a7250 2022-08-01T19:34:10.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.339+0000 7f74c15f3700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc157990 con 0x7f74bc0b0140 2022-08-01T19:34:10.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.339+0000 7f74bb7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74bc0b2050 0x7f74bc0b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:10.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.339+0000 7f74bb7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74bc0b2050 0x7f74bc0b4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36252/0 (socket says 172.21.15.17:36252) 2022-08-01T19:34:10.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.339+0000 7f74bb7fe700 1 -- 172.21.15.17:0/377303012 learned_addr learned my addr 172.21.15.17:0/377303012 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:10.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.339+0000 7f74ba7fc700 1 --2- 172.21.15.17:0/377303012 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f74bc0b0140 0x7f74bc1570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:10.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.339+0000 7f74bb7fe700 1 -- 172.21.15.17:0/377303012 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f74bc0b0140 msgr2=0x7f74bc1570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:10.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.339+0000 7f74bb7fe700 1 --2- 172.21.15.17:0/377303012 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f74bc0b0140 0x7f74bc1570c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:10.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.339+0000 7f74bb7fe700 1 -- 172.21.15.17:0/377303012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74bc0a7250 msgr2=0x7f74bc156bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:10.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.339+0000 7f74baffd700 1 --2- 172.21.15.17:0/377303012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74bc0a7250 0x7f74bc156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:10.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.339+0000 7f74bb7fe700 1 --2- 172.21.15.17:0/377303012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74bc0a7250 0x7f74bc156bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:10.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.339+0000 7f74bb7fe700 1 -- 172.21.15.17:0/377303012 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ac009530 con 0x7f74bc0b2050 2022-08-01T19:34:10.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.339+0000 7f74baffd700 1 --2- 172.21.15.17:0/377303012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74bc0a7250 0x7f74bc156bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:34:10.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.339+0000 7f74bb7fe700 1 --2- 172.21.15.17:0/377303012 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74bc0b2050 0x7f74bc0b4650 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f74b4000bc0 tx=0x7f74b40079e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:10.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.339+0000 7f74abfff700 1 -- 172.21.15.17:0/377303012 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74b40070d0 con 0x7f74bc0b2050 2022-08-01T19:34:10.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.339+0000 7f74abfff700 1 -- 172.21.15.17:0/377303012 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b4007230 con 0x7f74bc0b2050 2022-08-01T19:34:10.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.339+0000 7f74c15f3700 1 -- 172.21.15.17:0/377303012 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74bc0b53e0 con 0x7f74bc0b2050 2022-08-01T19:34:10.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.339+0000 7f74abfff700 1 -- 172.21.15.17:0/377303012 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74b4005740 con 0x7f74bc0b2050 2022-08-01T19:34:10.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.339+0000 7f74c15f3700 1 -- 172.21.15.17:0/377303012 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74bc1496c0 con 0x7f74bc0b2050 2022-08-01T19:34:10.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.339+0000 7f74c15f3700 1 -- 172.21.15.17:0/377303012 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74bc0b5570 con 0x7f74bc0b2050 2022-08-01T19:34:10.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.343+0000 7f74abfff700 1 -- 172.21.15.17:0/377303012 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f74b401a060 con 0x7f74bc0b2050 2022-08-01T19:34:10.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.343+0000 7f74abfff700 1 --2- 172.21.15.17:0/377303012 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7498053b00 0x7f7498055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:10.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.343+0000 7f74abfff700 1 -- 172.21.15.17:0/377303012 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f74b406b820 con 0x7f74bc0b2050 2022-08-01T19:34:10.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.343+0000 7f74baffd700 1 --2- 172.21.15.17:0/377303012 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7498053b00 0x7f7498055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:10.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.343+0000 7f74baffd700 1 --2- 172.21.15.17:0/377303012 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7498053b00 0x7f7498055fc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f74b0008000 tx=0x7f74b0008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:10.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.347+0000 7f74abfff700 1 -- 172.21.15.17:0/377303012 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f74b40425e0 con 0x7f74bc0b2050 2022-08-01T19:34:10.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:10.511+0000 7f74c15f3700 1 -- 172.21.15.17:0/377303012 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f74bc0b0d20 con 0x7f74bc0b2050 2022-08-01T19:34:11.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.430+0000 7f74abfff700 1 -- 172.21.15.17:0/377303012 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f74b4018070 con 0x7f74bc0b2050 2022-08-01T19:34:11.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.434+0000 7f74c15f3700 1 -- 172.21.15.17:0/377303012 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7498053b00 msgr2=0x7f7498055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:11.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.434+0000 7f74c15f3700 1 --2- 172.21.15.17:0/377303012 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7498053b00 0x7f7498055fc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f74b0008000 tx=0x7f74b0008040).stop 2022-08-01T19:34:11.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.434+0000 7f74c15f3700 1 -- 172.21.15.17:0/377303012 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74bc0b2050 msgr2=0x7f74bc0b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:11.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.434+0000 7f74c15f3700 1 --2- 172.21.15.17:0/377303012 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74bc0b2050 0x7f74bc0b4650 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f74b4000bc0 tx=0x7f74b40079e0).stop 2022-08-01T19:34:11.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.434+0000 7f74c15f3700 1 -- 172.21.15.17:0/377303012 shutdown_connections 2022-08-01T19:34:11.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.434+0000 7f74c15f3700 1 --2- 172.21.15.17:0/377303012 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7498053b00 0x7f7498055fc0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:11.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.434+0000 7f74c15f3700 1 --2- 172.21.15.17:0/377303012 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74bc0b2050 0x7f74bc0b4650 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:11.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.434+0000 7f74c15f3700 1 --2- 172.21.15.17:0/377303012 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f74bc0b0140 0x7f74bc1570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:11.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.434+0000 7f74c15f3700 1 --2- 172.21.15.17:0/377303012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74bc0a7250 0x7f74bc156bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:11.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.434+0000 7f74c15f3700 1 -- 172.21.15.17:0/377303012 >> 172.21.15.17:0/377303012 conn(0x7f74bc05f840 msgr2=0x7f74bc13ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:11.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.434+0000 7f74c15f3700 1 -- 172.21.15.17:0/377303012 shutdown_connections 2022-08-01T19:34:11.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.434+0000 7f74c15f3700 1 -- 172.21.15.17:0/377303012 wait complete. 2022-08-01T19:34:11.458 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:34:11.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a5aafe700 1 Processor -- start 2022-08-01T19:34:11.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a5aafe700 1 -- start start 2022-08-01T19:34:11.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a5aafe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2a54064b10 0x7f2a54144e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:11.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a5aafe700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f2a540b36e0 0x7f2a54141370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:11.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a5aafe700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2a54141860 0x7f2a54066ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:11.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a5aafe700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a5413ebf0 con 0x7f2a54064b10 2022-08-01T19:34:11.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a5aafe700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a54064ef0 con 0x7f2a540b36e0 2022-08-01T19:34:11.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a5aafe700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a54066690 con 0x7f2a54141860 2022-08-01T19:34:11.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a59afc700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2a54064b10 0x7f2a54144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:11.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a59afc700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2a54064b10 0x7f2a54144e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36258/0 (socket says 172.21.15.17:36258) 2022-08-01T19:34:11.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a59afc700 1 -- 172.21.15.17:0/43170899 learned_addr learned my addr 172.21.15.17:0/43170899 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:11.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a592fb700 1 --2- 172.21.15.17:0/43170899 >> v2:172.21.15.77:3300/0 conn(0x7f2a540b36e0 0x7f2a54141370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:11.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a5a2fd700 1 --2- 172.21.15.17:0/43170899 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2a54141860 0x7f2a54066ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:11.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a59afc700 1 -- 172.21.15.17:0/43170899 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2a54141860 msgr2=0x7f2a54066ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:11.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a59afc700 1 --2- 172.21.15.17:0/43170899 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2a54141860 0x7f2a54066ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:11.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a59afc700 1 -- 172.21.15.17:0/43170899 >> v2:172.21.15.77:3300/0 conn(0x7f2a540b36e0 msgr2=0x7f2a54141370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:11.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a59afc700 1 --2- 172.21.15.17:0/43170899 >> v2:172.21.15.77:3300/0 conn(0x7f2a540b36e0 0x7f2a54141370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:11.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a59afc700 1 -- 172.21.15.17:0/43170899 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a540b4850 con 0x7f2a54064b10 2022-08-01T19:34:11.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a59afc700 1 --2- 172.21.15.17:0/43170899 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a54064b10 0x7f2a54144e60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f2a4c0032f0 tx=0x7f2a4c009290).ready entity=mon.0 client_cookie=6c2d25f1df798529 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:11.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a58afa700 1 -- 172.21.15.17:0/43170899 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a4c00b870 con 0x7f2a54064b10 2022-08-01T19:34:11.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a58afa700 1 -- 172.21.15.17:0/43170899 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a4c00b9d0 con 0x7f2a54064b10 2022-08-01T19:34:11.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a58afa700 1 -- 172.21.15.17:0/43170899 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a4c00bb40 con 0x7f2a54064b10 2022-08-01T19:34:11.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a5aafe700 1 -- 172.21.15.17:0/43170899 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a54064b10 msgr2=0x7f2a54144e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:11.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a5aafe700 1 --2- 172.21.15.17:0/43170899 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a54064b10 0x7f2a54144e60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f2a4c0032f0 tx=0x7f2a4c009290).stop 2022-08-01T19:34:11.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a5aafe700 1 -- 172.21.15.17:0/43170899 shutdown_connections 2022-08-01T19:34:11.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a5aafe700 1 --2- 172.21.15.17:0/43170899 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2a54141860 0x7f2a54066ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:11.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a5aafe700 1 --2- 172.21.15.17:0/43170899 >> v2:172.21.15.77:3300/0 conn(0x7f2a540b36e0 0x7f2a54141370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:11.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a5aafe700 1 --2- 172.21.15.17:0/43170899 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a54064b10 0x7f2a54144e60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:11.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a5aafe700 1 -- 172.21.15.17:0/43170899 >> 172.21.15.17:0/43170899 conn(0x7f2a5405f840 msgr2=0x7f2a5405fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:11.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a5aafe700 1 -- 172.21.15.17:0/43170899 shutdown_connections 2022-08-01T19:34:11.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a5aafe700 1 -- 172.21.15.17:0/43170899 wait complete. 2022-08-01T19:34:11.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a5aafe700 1 Processor -- start 2022-08-01T19:34:11.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a5aafe700 1 -- start start 2022-08-01T19:34:11.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a5aafe700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2a54064b10 0x7f2a5405c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:11.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a5aafe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a540b36e0 0x7f2a5413a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:11.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a5aafe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a54141860 0x7f2a5413ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:11.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a5aafe700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a540af890 con 0x7f2a540b36e0 2022-08-01T19:34:11.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a5aafe700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a540afb30 con 0x7f2a54141860 2022-08-01T19:34:11.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a5aafe700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a540afdd0 con 0x7f2a54064b10 2022-08-01T19:34:11.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a592fb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a540b36e0 0x7f2a5413a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:11.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a592fb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a540b36e0 0x7f2a5413a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36266/0 (socket says 172.21.15.17:36266) 2022-08-01T19:34:11.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a592fb700 1 -- 172.21.15.17:0/532353339 learned_addr learned my addr 172.21.15.17:0/532353339 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:11.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a5a2fd700 1 --2- 172.21.15.17:0/532353339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a54141860 0x7f2a5413ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:11.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a59afc700 1 --2- 172.21.15.17:0/532353339 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2a54064b10 0x7f2a5405c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:11.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a592fb700 1 -- 172.21.15.17:0/532353339 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2a54064b10 msgr2=0x7f2a5405c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:11.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a592fb700 1 --2- 172.21.15.17:0/532353339 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2a54064b10 0x7f2a5405c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:11.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a592fb700 1 -- 172.21.15.17:0/532353339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a54141860 msgr2=0x7f2a5413ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:11.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a592fb700 1 --2- 172.21.15.17:0/532353339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a54141860 0x7f2a5413ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:11.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a592fb700 1 -- 172.21.15.17:0/532353339 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a4c009530 con 0x7f2a540b36e0 2022-08-01T19:34:11.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a592fb700 1 --2- 172.21.15.17:0/532353339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a540b36e0 0x7f2a5413a780 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f2a480030b0 tx=0x7f2a48008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:11.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a52ffd700 1 -- 172.21.15.17:0/532353339 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a48011780 con 0x7f2a540b36e0 2022-08-01T19:34:11.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a52ffd700 1 -- 172.21.15.17:0/532353339 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a480118e0 con 0x7f2a540b36e0 2022-08-01T19:34:11.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a5aafe700 1 -- 172.21.15.17:0/532353339 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a540b0070 con 0x7f2a540b36e0 2022-08-01T19:34:11.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a52ffd700 1 -- 172.21.15.17:0/532353339 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a48011c40 con 0x7f2a540b36e0 2022-08-01T19:34:11.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.566+0000 7f2a5aafe700 1 -- 172.21.15.17:0/532353339 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a5413b160 con 0x7f2a540b36e0 2022-08-01T19:34:11.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.570+0000 7f2a52ffd700 1 -- 172.21.15.17:0/532353339 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2a48012420 con 0x7f2a540b36e0 2022-08-01T19:34:11.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.570+0000 7f2a5aafe700 1 -- 172.21.15.17:0/532353339 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a540b0200 con 0x7f2a540b36e0 2022-08-01T19:34:11.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.570+0000 7f2a52ffd700 1 --2- 172.21.15.17:0/532353339 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2a34053950 0x7f2a34055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:11.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.570+0000 7f2a52ffd700 1 -- 172.21.15.17:0/532353339 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f2a4806bec0 con 0x7f2a540b36e0 2022-08-01T19:34:11.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.570+0000 7f2a59afc700 1 --2- 172.21.15.17:0/532353339 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2a34053950 0x7f2a34055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:11.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.570+0000 7f2a59afc700 1 --2- 172.21.15.17:0/532353339 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2a34053950 0x7f2a34055e10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f2a4c01b000 tx=0x7f2a4c016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:11.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.578+0000 7f2a52ffd700 1 -- 172.21.15.17:0/532353339 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2a48042b50 con 0x7f2a540b36e0 2022-08-01T19:34:11.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.738+0000 7f2a5aafe700 1 -- 172.21.15.17:0/532353339 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a540b4af0 con 0x7f2a540b36e0 2022-08-01T19:34:11.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.742+0000 7f2a52ffd700 1 -- 172.21.15.17:0/532353339 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+2486 (secure 0 0 0) 0x7f2a48046170 con 0x7f2a540b36e0 2022-08-01T19:34:11.747 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:11.748 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2116883486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2116883486},{"type":"v1","addr":"172.21.15.17:6833","nonce":2116883486}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.17:6835/2974833789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2974833789},{"type":"v1","addr":"172.21.15.17:6835","nonce":2974833789}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":4}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:34:09.692662+0000","modified":"2022-08-01T19:34:11.425752+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":5,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6835/1842780453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1842780453},{"type":"v1","addr":"172.21.15.77:6835","nonce":1842780453}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2022-08-01T19:34:11.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.742+0000 7f2a5aafe700 1 -- 172.21.15.17:0/532353339 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2a34053950 msgr2=0x7f2a34055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:11.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.742+0000 7f2a5aafe700 1 --2- 172.21.15.17:0/532353339 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2a34053950 0x7f2a34055e10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f2a4c01b000 tx=0x7f2a4c016000).stop 2022-08-01T19:34:11.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.742+0000 7f2a5aafe700 1 -- 172.21.15.17:0/532353339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a540b36e0 msgr2=0x7f2a5413a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:11.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.742+0000 7f2a5aafe700 1 --2- 172.21.15.17:0/532353339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a540b36e0 0x7f2a5413a780 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f2a480030b0 tx=0x7f2a48008040).stop 2022-08-01T19:34:11.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.742+0000 7f2a5aafe700 1 -- 172.21.15.17:0/532353339 shutdown_connections 2022-08-01T19:34:11.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.742+0000 7f2a5aafe700 1 --2- 172.21.15.17:0/532353339 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2a34053950 0x7f2a34055e10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:11.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.742+0000 7f2a5aafe700 1 --2- 172.21.15.17:0/532353339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a54141860 0x7f2a5413ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:11.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.742+0000 7f2a5aafe700 1 --2- 172.21.15.17:0/532353339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a540b36e0 0x7f2a5413a780 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:11.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.742+0000 7f2a5aafe700 1 --2- 172.21.15.17:0/532353339 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2a54064b10 0x7f2a5405c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:11.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.746+0000 7f2a5aafe700 1 -- 172.21.15.17:0/532353339 >> 172.21.15.17:0/532353339 conn(0x7f2a5405f840 msgr2=0x7f2a540b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:11.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.746+0000 7f2a5aafe700 1 -- 172.21.15.17:0/532353339 shutdown_connections 2022-08-01T19:34:11.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.746+0000 7f2a5aafe700 1 -- 172.21.15.17:0/532353339 wait complete. 2022-08-01T19:34:11.754 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 7 2022-08-01T19:34:11.767 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:34:11.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.866+0000 7f65d4ee4700 1 Processor -- start 2022-08-01T19:34:11.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.866+0000 7f65d4ee4700 1 -- start start 2022-08-01T19:34:11.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.870+0000 7f65d4ee4700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f65d00b01a0 0x7f65d00a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:11.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.870+0000 7f65d4ee4700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65d00a7250 con 0x7f65d00afd60 2022-08-01T19:34:11.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.870+0000 7f65d4ee4700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65d00a7480 con 0x7f65d00b2710 2022-08-01T19:34:11.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.870+0000 7f65d4ee4700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d00a7720 con 0x7f65d00b01a0 2022-08-01T19:34:11.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.870+0000 7f65cffff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f65d00afd60 0x7f65d0151d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56702/0 (socket says 172.21.15.17:56702) 2022-08-01T19:34:11.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.870+0000 7f65cffff700 1 -- 172.21.15.17:0/2075599826 learned_addr learned my addr 172.21.15.17:0/2075599826 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:11.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.870+0000 7f65ceffd700 1 --2- 172.21.15.17:0/2075599826 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f65d00b01a0 0x7f65d00a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:11.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.870+0000 7f65ce7fc700 1 -- 172.21.15.17:0/2075599826 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2090310801 0 0) 0x7f65d00a7250 con 0x7f65d00afd60 2022-08-01T19:34:11.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.870+0000 7f65ce7fc700 1 -- 172.21.15.17:0/2075599826 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65bc003260 con 0x7f65d00afd60 2022-08-01T19:34:11.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.870+0000 7f65ceffd700 1 -- 172.21.15.17:0/2075599826 >> v1:172.21.15.77:6789/0 conn(0x7f65d00b2710 legacy=0x7f65d014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:11.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.870+0000 7f65ce7fc700 1 -- 172.21.15.17:0/2075599826 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3597438216 0 0) 0x7f65d00a7480 con 0x7f65d00b2710 2022-08-01T19:34:11.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.870+0000 7f65ceffd700 1 -- 172.21.15.17:0/2075599826 >> v1:172.21.15.17:6789/0 conn(0x7f65d00afd60 legacy=0x7f65d0151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:11.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.870+0000 7f65ceffd700 1 -- 172.21.15.17:0/2075599826 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d00a79c0 con 0x7f65d00b01a0 2022-08-01T19:34:11.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.870+0000 7f65ceffd700 1 --2- 172.21.15.17:0/2075599826 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f65d00b01a0 0x7f65d00a8540 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f65c00032f0 tx=0x7f65c00091d0).ready entity=mon.2 client_cookie=c6dd6b81ca07b468 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:11.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.870+0000 7f65ce7fc700 1 -- 172.21.15.17:0/2075599826 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65c000b6c0 con 0x7f65d00b01a0 2022-08-01T19:34:11.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.870+0000 7f65ce7fc700 1 -- 172.21.15.17:0/2075599826 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c000b820 con 0x7f65d00b01a0 2022-08-01T19:34:11.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.870+0000 7f65ce7fc700 1 -- 172.21.15.17:0/2075599826 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65c000b990 con 0x7f65d00b01a0 2022-08-01T19:34:11.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.870+0000 7f65d4ee4700 1 -- 172.21.15.17:0/2075599826 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f65d00b01a0 msgr2=0x7f65d00a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:11.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.870+0000 7f65d4ee4700 1 --2- 172.21.15.17:0/2075599826 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f65d00b01a0 0x7f65d00a8540 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f65c00032f0 tx=0x7f65c00091d0).stop 2022-08-01T19:34:11.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.870+0000 7f65d4ee4700 1 -- 172.21.15.17:0/2075599826 shutdown_connections 2022-08-01T19:34:11.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.870+0000 7f65d4ee4700 1 --2- 172.21.15.17:0/2075599826 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f65d00b01a0 0x7f65d00a8540 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:11.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.870+0000 7f65d4ee4700 1 -- 172.21.15.17:0/2075599826 >> 172.21.15.17:0/2075599826 conn(0x7f65d0060310 msgr2=0x7f65d00b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:11.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.870+0000 7f65d4ee4700 1 -- 172.21.15.17:0/2075599826 shutdown_connections 2022-08-01T19:34:11.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.870+0000 7f65d4ee4700 1 -- 172.21.15.17:0/2075599826 wait complete. 2022-08-01T19:34:11.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.874+0000 7f65d4ee4700 1 Processor -- start 2022-08-01T19:34:11.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.874+0000 7f65d4ee4700 1 -- start start 2022-08-01T19:34:11.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.874+0000 7f65d4ee4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65d00afd60 0x7f65d0143c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:11.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.874+0000 7f65d4ee4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65d00b01a0 0x7f65d0156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:11.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.874+0000 7f65d4ee4700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f65d00b2710 0x7f65d014bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:11.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.874+0000 7f65d4ee4700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d00b35f0 con 0x7f65d00b01a0 2022-08-01T19:34:11.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.874+0000 7f65d4ee4700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d014baa0 con 0x7f65d00afd60 2022-08-01T19:34:11.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.874+0000 7f65d4ee4700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d00b3850 con 0x7f65d00b2710 2022-08-01T19:34:11.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.874+0000 7f65ceffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65d00b01a0 0x7f65d0156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:11.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.874+0000 7f65ceffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65d00b01a0 0x7f65d0156ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36278/0 (socket says 172.21.15.17:36278) 2022-08-01T19:34:11.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.874+0000 7f65ceffd700 1 -- 172.21.15.17:0/2748114565 learned_addr learned my addr 172.21.15.17:0/2748114565 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:11.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.874+0000 7f65cf7fe700 1 --2- 172.21.15.17:0/2748114565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65d00afd60 0x7f65d0143c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:11.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.874+0000 7f65cffff700 1 --2- 172.21.15.17:0/2748114565 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f65d00b2710 0x7f65d014bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:11.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.874+0000 7f65cf7fe700 1 -- 172.21.15.17:0/2748114565 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f65d00b2710 msgr2=0x7f65d014bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:11.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.874+0000 7f65cf7fe700 1 --2- 172.21.15.17:0/2748114565 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f65d00b2710 0x7f65d014bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:11.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.874+0000 7f65cf7fe700 1 -- 172.21.15.17:0/2748114565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65d00b01a0 msgr2=0x7f65d0156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:11.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.874+0000 7f65cf7fe700 1 --2- 172.21.15.17:0/2748114565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65d00b01a0 0x7f65d0156ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:11.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.874+0000 7f65cf7fe700 1 -- 172.21.15.17:0/2748114565 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c0009470 con 0x7f65d00afd60 2022-08-01T19:34:11.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.874+0000 7f65ceffd700 1 --2- 172.21.15.17:0/2748114565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65d00b01a0 0x7f65d0156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:34:11.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.874+0000 7f65cf7fe700 1 --2- 172.21.15.17:0/2748114565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65d00afd60 0x7f65d0143c30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f65c4000ed0 tx=0x7f65c4007b90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:11.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.874+0000 7f65ccff9700 1 -- 172.21.15.17:0/2748114565 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65c4007e20 con 0x7f65d00afd60 2022-08-01T19:34:11.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.874+0000 7f65ccff9700 1 -- 172.21.15.17:0/2748114565 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c4012400 con 0x7f65d00afd60 2022-08-01T19:34:11.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.874+0000 7f65d4ee4700 1 -- 172.21.15.17:0/2748114565 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d00b3af0 con 0x7f65d00afd60 2022-08-01T19:34:11.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.874+0000 7f65ccff9700 1 -- 172.21.15.17:0/2748114565 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65c4012760 con 0x7f65d00afd60 2022-08-01T19:34:11.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.874+0000 7f65d4ee4700 1 -- 172.21.15.17:0/2748114565 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d00b2f60 con 0x7f65d00afd60 2022-08-01T19:34:11.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.878+0000 7f65d4ee4700 1 -- 172.21.15.17:0/2748114565 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d00574e0 con 0x7f65d00afd60 2022-08-01T19:34:11.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.878+0000 7f65ccff9700 1 -- 172.21.15.17:0/2748114565 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f65c4019030 con 0x7f65d00afd60 2022-08-01T19:34:11.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.878+0000 7f65ccff9700 1 --2- 172.21.15.17:0/2748114565 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f65b0053aa0 0x7f65b0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:11.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.878+0000 7f65ccff9700 1 -- 172.21.15.17:0/2748114565 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f65c406b7d0 con 0x7f65d00afd60 2022-08-01T19:34:11.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.878+0000 7f65ceffd700 1 --2- 172.21.15.17:0/2748114565 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f65b0053aa0 0x7f65b0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:11.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.878+0000 7f65ceffd700 1 --2- 172.21.15.17:0/2748114565 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f65b0053aa0 0x7f65b0055f60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f65c001b000 tx=0x7f65c0016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:11.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:11.890+0000 7f65ccff9700 1 -- 172.21.15.17:0/2748114565 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f65c4042480 con 0x7f65d00afd60 2022-08-01T19:34:12.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.018+0000 7f65d4ee4700 1 -- 172.21.15.17:0/2748114565 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65d014bbe0 con 0x7f65d00afd60 2022-08-01T19:34:12.027 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.026+0000 7f65ccff9700 1 -- 172.21.15.17:0/2748114565 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f65c4045aa0 con 0x7f65d00afd60 2022-08-01T19:34:12.028 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:12.028 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:34:09.692404+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:34:04.635473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-08-01T19:34:07.409434+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:34:12.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.026+0000 7f65d4ee4700 1 -- 172.21.15.17:0/2748114565 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f65b0053aa0 msgr2=0x7f65b0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:12.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.026+0000 7f65d4ee4700 1 --2- 172.21.15.17:0/2748114565 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f65b0053aa0 0x7f65b0055f60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f65c001b000 tx=0x7f65c0016000).stop 2022-08-01T19:34:12.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.026+0000 7f65d4ee4700 1 -- 172.21.15.17:0/2748114565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65d00afd60 msgr2=0x7f65d0143c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:12.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.026+0000 7f65d4ee4700 1 --2- 172.21.15.17:0/2748114565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65d00afd60 0x7f65d0143c30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f65c4000ed0 tx=0x7f65c4007b90).stop 2022-08-01T19:34:12.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.026+0000 7f65d4ee4700 1 -- 172.21.15.17:0/2748114565 shutdown_connections 2022-08-01T19:34:12.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.026+0000 7f65d4ee4700 1 --2- 172.21.15.17:0/2748114565 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f65b0053aa0 0x7f65b0055f60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:12.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.026+0000 7f65d4ee4700 1 --2- 172.21.15.17:0/2748114565 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f65d00b2710 0x7f65d014bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:12.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.026+0000 7f65d4ee4700 1 --2- 172.21.15.17:0/2748114565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65d00b01a0 0x7f65d0156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:12.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.026+0000 7f65d4ee4700 1 --2- 172.21.15.17:0/2748114565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65d00afd60 0x7f65d0143c30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:12.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.026+0000 7f65d4ee4700 1 -- 172.21.15.17:0/2748114565 >> 172.21.15.17:0/2748114565 conn(0x7f65d0060310 msgr2=0x7f65d0145ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:12.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.026+0000 7f65d4ee4700 1 -- 172.21.15.17:0/2748114565 shutdown_connections 2022-08-01T19:34:12.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.026+0000 7f65d4ee4700 1 -- 172.21.15.17:0/2748114565 wait complete. 2022-08-01T19:34:12.042 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:34:12.042 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2022-08-01T19:34:12.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f8022581700 1 Processor -- start 2022-08-01T19:34:12.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f8022581700 1 -- start start 2022-08-01T19:34:12.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f8022581700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f801c060aa0 0x7f801c0acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:12.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f8022581700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f801c0b6e70 con 0x7f801c0b6a90 2022-08-01T19:34:12.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f8022581700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f801c061370 con 0x7f801c0aa760 2022-08-01T19:34:12.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f8022581700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f801c061610 con 0x7f801c060aa0 2022-08-01T19:34:12.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f801b7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f801c060aa0 0x7f801c0acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:12.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f801b7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f801c060aa0 0x7f801c0acee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36288/0 (socket says 172.21.15.17:36288) 2022-08-01T19:34:12.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f801bfff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f801c0b6a90 0x7f801c0aa050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56712/0 (socket says 172.21.15.17:56712) 2022-08-01T19:34:12.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f801b7fe700 1 -- 172.21.15.17:0/3288614159 learned_addr learned my addr 172.21.15.17:0/3288614159 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:12.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f801affd700 1 -- 172.21.15.17:0/3288614159 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1790124880 0 0) 0x7f801c0b6e70 con 0x7f801c0b6a90 2022-08-01T19:34:12.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f801affd700 1 -- 172.21.15.17:0/3288614159 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ffc003260 con 0x7f801c0b6a90 2022-08-01T19:34:12.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f801affd700 1 -- 172.21.15.17:0/3288614159 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 688538141 0 0) 0x7f7ffc003260 con 0x7f801c0b6a90 2022-08-01T19:34:12.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f801b7fe700 1 -- 172.21.15.17:0/3288614159 >> v1:172.21.15.77:6789/0 conn(0x7f801c0aa760 legacy=0x7f801c13c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:12.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f801b7fe700 1 -- 172.21.15.17:0/3288614159 >> v1:172.21.15.17:6789/0 conn(0x7f801c0b6a90 legacy=0x7f801c0aa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:12.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f801b7fe700 1 -- 172.21.15.17:0/3288614159 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f801c0618b0 con 0x7f801c060aa0 2022-08-01T19:34:12.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f801b7fe700 1 --2- 172.21.15.17:0/3288614159 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f801c060aa0 0x7f801c0acee0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f8004001820 tx=0x7f8014009290).ready entity=mon.0 client_cookie=f40cbe41910e4fe7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:12.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f801affd700 1 -- 172.21.15.17:0/3288614159 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8004001680 con 0x7f801c060aa0 2022-08-01T19:34:12.154 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f801affd700 1 -- 172.21.15.17:0/3288614159 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8014009eb0 con 0x7f801c060aa0 2022-08-01T19:34:12.154 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f801affd700 1 -- 172.21.15.17:0/3288614159 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f801400b760 con 0x7f801c060aa0 2022-08-01T19:34:12.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f8022581700 1 -- 172.21.15.17:0/3288614159 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f801c060aa0 msgr2=0x7f801c0acee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:12.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f8022581700 1 --2- 172.21.15.17:0/3288614159 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f801c060aa0 0x7f801c0acee0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f8004001820 tx=0x7f8014009290).stop 2022-08-01T19:34:12.158 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f8022581700 1 -- 172.21.15.17:0/3288614159 shutdown_connections 2022-08-01T19:34:12.158 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f8022581700 1 --2- 172.21.15.17:0/3288614159 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f801c060aa0 0x7f801c0acee0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:12.158 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f8022581700 1 -- 172.21.15.17:0/3288614159 >> 172.21.15.17:0/3288614159 conn(0x7f801c0b3610 msgr2=0x7f801c0b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:12.158 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f8022581700 1 -- 172.21.15.17:0/3288614159 shutdown_connections 2022-08-01T19:34:12.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f8022581700 1 -- 172.21.15.17:0/3288614159 wait complete. 2022-08-01T19:34:12.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f8022581700 1 Processor -- start 2022-08-01T19:34:12.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f8022581700 1 -- start start 2022-08-01T19:34:12.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f8022581700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f801c060aa0 0x7f801c0af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:12.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f8022581700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f801c0aa760 0x7f801c0658a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:12.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f8022581700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f801c0b6a90 0x7f801c065d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:12.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f8022581700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801c13b700 con 0x7f801c0b6a90 2022-08-01T19:34:12.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f8022581700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801c150330 con 0x7f801c0aa760 2022-08-01T19:34:12.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f8022581700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f801c150520 con 0x7f801c060aa0 2022-08-01T19:34:12.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f8020b1e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f801c0b6a90 0x7f801c065d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:12.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f801b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f801c0aa760 0x7f801c0658a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:12.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f8020b1e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f801c0b6a90 0x7f801c065d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36296/0 (socket says 172.21.15.17:36296) 2022-08-01T19:34:12.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f8020b1e700 1 -- 172.21.15.17:0/1817944147 learned_addr learned my addr 172.21.15.17:0/1817944147 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:12.162 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f801bfff700 1 --2- 172.21.15.17:0/1817944147 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f801c060aa0 0x7f801c0af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:12.162 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f8020b1e700 1 -- 172.21.15.17:0/1817944147 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f801c060aa0 msgr2=0x7f801c0af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:12.162 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f8020b1e700 1 --2- 172.21.15.17:0/1817944147 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f801c060aa0 0x7f801c0af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:12.162 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f8020b1e700 1 -- 172.21.15.17:0/1817944147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f801c0aa760 msgr2=0x7f801c0658a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:12.163 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f8020b1e700 1 --2- 172.21.15.17:0/1817944147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f801c0aa760 0x7f801c0658a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:12.163 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f8020b1e700 1 -- 172.21.15.17:0/1817944147 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8014009530 con 0x7f801c0b6a90 2022-08-01T19:34:12.163 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.146+0000 7f8020b1e700 1 --2- 172.21.15.17:0/1817944147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f801c0b6a90 0x7f801c065d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f801c0afbb0 tx=0x7f800c007b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:12.163 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.150+0000 7f80197fa700 1 -- 172.21.15.17:0/1817944147 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f800c012400 con 0x7f801c0b6a90 2022-08-01T19:34:12.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.150+0000 7f80197fa700 1 -- 172.21.15.17:0/1817944147 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800c012560 con 0x7f801c0b6a90 2022-08-01T19:34:12.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.150+0000 7f80197fa700 1 -- 172.21.15.17:0/1817944147 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f800c0128c0 con 0x7f801c0b6a90 2022-08-01T19:34:12.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.150+0000 7f8022581700 1 -- 172.21.15.17:0/1817944147 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f801c065400 con 0x7f801c0b6a90 2022-08-01T19:34:12.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.150+0000 7f8022581700 1 -- 172.21.15.17:0/1817944147 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f801c0631c0 con 0x7f801c0b6a90 2022-08-01T19:34:12.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.150+0000 7f8022581700 1 -- 172.21.15.17:0/1817944147 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f801c065590 con 0x7f801c0b6a90 2022-08-01T19:34:12.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.150+0000 7f80197fa700 1 -- 172.21.15.17:0/1817944147 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f800c019030 con 0x7f801c0b6a90 2022-08-01T19:34:12.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.150+0000 7f80197fa700 1 --2- 172.21.15.17:0/1817944147 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8000053a10 0x7f8000055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:12.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.150+0000 7f80197fa700 1 -- 172.21.15.17:0/1817944147 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f800c06b2d0 con 0x7f801c0b6a90 2022-08-01T19:34:12.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.150+0000 7f801bfff700 1 --2- 172.21.15.17:0/1817944147 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8000053a10 0x7f8000055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:12.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.154+0000 7f801bfff700 1 --2- 172.21.15.17:0/1817944147 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8000053a10 0x7f8000055ed0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f800400d000 tx=0x7f8004000bc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:12.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.158+0000 7f80197fa700 1 -- 172.21.15.17:0/1817944147 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f800c042140 con 0x7f801c0b6a90 2022-08-01T19:34:12.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.278+0000 7f8022581700 1 -- 172.21.15.17:0/1817944147 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f801c061b50 con 0x7f801c0b6a90 2022-08-01T19:34:12.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.278+0000 7f80197fa700 1 -- 172.21.15.17:0/1817944147 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f800c024fc0 con 0x7f801c0b6a90 2022-08-01T19:34:12.286 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:12.286 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:34:09.692404+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:34:04.635473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-08-01T19:34:07.409434+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:34:12.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.282+0000 7f8022581700 1 -- 172.21.15.17:0/1817944147 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8000053a10 msgr2=0x7f8000055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:12.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.282+0000 7f8022581700 1 --2- 172.21.15.17:0/1817944147 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8000053a10 0x7f8000055ed0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f800400d000 tx=0x7f8004000bc0).stop 2022-08-01T19:34:12.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.282+0000 7f8022581700 1 -- 172.21.15.17:0/1817944147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f801c0b6a90 msgr2=0x7f801c065d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:12.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.282+0000 7f8022581700 1 --2- 172.21.15.17:0/1817944147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f801c0b6a90 0x7f801c065d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f801c0afbb0 tx=0x7f800c007b90).stop 2022-08-01T19:34:12.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.282+0000 7f8022581700 1 -- 172.21.15.17:0/1817944147 shutdown_connections 2022-08-01T19:34:12.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.282+0000 7f8022581700 1 --2- 172.21.15.17:0/1817944147 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8000053a10 0x7f8000055ed0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:12.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.282+0000 7f8022581700 1 --2- 172.21.15.17:0/1817944147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f801c0b6a90 0x7f801c065d90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:12.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.282+0000 7f8022581700 1 --2- 172.21.15.17:0/1817944147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f801c0aa760 0x7f801c0658a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:12.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.282+0000 7f8022581700 1 --2- 172.21.15.17:0/1817944147 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f801c060aa0 0x7f801c0af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:12.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.282+0000 7f8022581700 1 -- 172.21.15.17:0/1817944147 >> 172.21.15.17:0/1817944147 conn(0x7f801c0b3610 msgr2=0x7f801c0b4870 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:12.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.282+0000 7f8022581700 1 -- 172.21.15.17:0/1817944147 shutdown_connections 2022-08-01T19:34:12.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.282+0000 7f8022581700 1 -- 172.21.15.17:0/1817944147 wait complete. 2022-08-01T19:34:12.303 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T19:34:04.635473+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T19:34:07.409434+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T19:34:12.303 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:34:12.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.410+0000 7f4fd24bd700 1 Processor -- start 2022-08-01T19:34:12.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.410+0000 7f4fd24bd700 1 -- start start 2022-08-01T19:34:12.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.410+0000 7f4fd24bd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4fcc0b29b0 0x7f4fcc0afd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:12.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.410+0000 7f4fd24bd700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fcc0b1960 con 0x7f4fcc0ae0b0 2022-08-01T19:34:12.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.410+0000 7f4fd24bd700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fcc0b3480 con 0x7f4fcc0b1580 2022-08-01T19:34:12.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.410+0000 7f4fd24bd700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fcc14bf10 con 0x7f4fcc0b29b0 2022-08-01T19:34:12.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.410+0000 7f4fd0a5a700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4fcc0b29b0 0x7f4fcc0afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:12.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.410+0000 7f4fcb7fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f4fcc0ae0b0 0x7f4fcc0af610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56728/0 (socket says 172.21.15.17:56728) 2022-08-01T19:34:12.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.410+0000 7f4fcb7fe700 1 -- 172.21.15.17:0/3386848677 learned_addr learned my addr 172.21.15.17:0/3386848677 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:12.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.410+0000 7f4fd0a5a700 1 -- 172.21.15.17:0/3386848677 >> v1:172.21.15.77:6789/0 conn(0x7f4fcc0b1580 legacy=0x7f4fcc14b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:12.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.410+0000 7f4fd0a5a700 1 -- 172.21.15.17:0/3386848677 >> v1:172.21.15.17:6789/0 conn(0x7f4fcc0ae0b0 legacy=0x7f4fcc0af610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:12.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.410+0000 7f4fd0a5a700 1 -- 172.21.15.17:0/3386848677 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fcc0b3720 con 0x7f4fcc0b29b0 2022-08-01T19:34:12.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.410+0000 7f4fcaffd700 1 -- 172.21.15.17:0/3386848677 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2624805891 0 0) 0x7f4fcc0b1960 con 0x7f4fcc0ae0b0 2022-08-01T19:34:12.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.410+0000 7f4fd0a5a700 1 --2- 172.21.15.17:0/3386848677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fcc0b29b0 0x7f4fcc0afd20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f4fc40032f0 tx=0x7f4fc40091d0).ready entity=mon.0 client_cookie=b4b8980333288214 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:12.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.410+0000 7f4fcaffd700 1 -- 172.21.15.17:0/3386848677 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fc4009ea0 con 0x7f4fcc0b29b0 2022-08-01T19:34:12.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.410+0000 7f4fcaffd700 1 -- 172.21.15.17:0/3386848677 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc400b8a0 con 0x7f4fcc0b29b0 2022-08-01T19:34:12.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.410+0000 7f4fcaffd700 1 -- 172.21.15.17:0/3386848677 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fc400ba10 con 0x7f4fcc0b29b0 2022-08-01T19:34:12.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.410+0000 7f4fd24bd700 1 -- 172.21.15.17:0/3386848677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fcc0b29b0 msgr2=0x7f4fcc0afd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:12.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.410+0000 7f4fd24bd700 1 --2- 172.21.15.17:0/3386848677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fcc0b29b0 0x7f4fcc0afd20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f4fc40032f0 tx=0x7f4fc40091d0).stop 2022-08-01T19:34:12.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.410+0000 7f4fd24bd700 1 -- 172.21.15.17:0/3386848677 shutdown_connections 2022-08-01T19:34:12.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.410+0000 7f4fd24bd700 1 --2- 172.21.15.17:0/3386848677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fcc0b29b0 0x7f4fcc0afd20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:12.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.410+0000 7f4fd24bd700 1 -- 172.21.15.17:0/3386848677 >> 172.21.15.17:0/3386848677 conn(0x7f4fcc060950 msgr2=0x7f4fcc060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:12.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.410+0000 7f4fd24bd700 1 -- 172.21.15.17:0/3386848677 shutdown_connections 2022-08-01T19:34:12.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.410+0000 7f4fd24bd700 1 -- 172.21.15.17:0/3386848677 wait complete. 2022-08-01T19:34:12.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.414+0000 7f4fd24bd700 1 Processor -- start 2022-08-01T19:34:12.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.414+0000 7f4fd24bd700 1 -- start start 2022-08-01T19:34:12.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.414+0000 7f4fd24bd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fcc0ae0b0 0x7f4fcc0acd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:12.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.414+0000 7f4fd24bd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fcc0b1580 0x7f4fcc066e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:12.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.414+0000 7f4fd24bd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4fcc0b29b0 0x7f4fcc067360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:12.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.414+0000 7f4fd24bd700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc065470 con 0x7f4fcc0b1580 2022-08-01T19:34:12.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.414+0000 7f4fd24bd700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc156ae0 con 0x7f4fcc0ae0b0 2022-08-01T19:34:12.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.414+0000 7f4fd24bd700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc0656a0 con 0x7f4fcc0b29b0 2022-08-01T19:34:12.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.414+0000 7f4fcb7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fcc0b1580 0x7f4fcc066e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:12.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.414+0000 7f4fd0a5a700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4fcc0b29b0 0x7f4fcc067360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:12.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.414+0000 7f4fcbfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fcc0ae0b0 0x7f4fcc0acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:12.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.414+0000 7f4fcb7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fcc0b1580 0x7f4fcc066e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36310/0 (socket says 172.21.15.17:36310) 2022-08-01T19:34:12.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.414+0000 7f4fcbfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fcc0ae0b0 0x7f4fcc0acd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:34228/0 (socket says 172.21.15.17:34228) 2022-08-01T19:34:12.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.414+0000 7f4fd0a5a700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4fcc0b29b0 0x7f4fcc067360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:60640/0 (socket says 172.21.15.17:60640) 2022-08-01T19:34:12.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.414+0000 7f4fcb7fe700 1 -- 172.21.15.17:0/3266687201 learned_addr learned my addr 172.21.15.17:0/3266687201 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:12.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.414+0000 7f4fcbfff700 1 -- 172.21.15.17:0/3266687201 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4fcc0b29b0 msgr2=0x7f4fcc067360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:12.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.414+0000 7f4fcbfff700 1 --2- 172.21.15.17:0/3266687201 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4fcc0b29b0 0x7f4fcc067360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:12.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.414+0000 7f4fcbfff700 1 -- 172.21.15.17:0/3266687201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fcc0b1580 msgr2=0x7f4fcc066e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:12.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.414+0000 7f4fcbfff700 1 --2- 172.21.15.17:0/3266687201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fcc0b1580 0x7f4fcc066e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:12.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.414+0000 7f4fcbfff700 1 -- 172.21.15.17:0/3266687201 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc4009470 con 0x7f4fcc0ae0b0 2022-08-01T19:34:12.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.414+0000 7f4fcb7fe700 1 --2- 172.21.15.17:0/3266687201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fcc0b1580 0x7f4fcc066e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:34:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.418+0000 7f4fcbfff700 1 --2- 172.21.15.17:0/3266687201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fcc0ae0b0 0x7f4fcc0acd60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f4fcc05d020 tx=0x7f4fbc007b90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.418+0000 7f4fc97fa700 1 -- 172.21.15.17:0/3266687201 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fbc012400 con 0x7f4fcc0ae0b0 2022-08-01T19:34:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.418+0000 7f4fc97fa700 1 -- 172.21.15.17:0/3266687201 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fbc012560 con 0x7f4fcc0ae0b0 2022-08-01T19:34:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.418+0000 7f4fd24bd700 1 -- 172.21.15.17:0/3266687201 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fcc065940 con 0x7f4fcc0ae0b0 2022-08-01T19:34:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.418+0000 7f4fc97fa700 1 -- 172.21.15.17:0/3266687201 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fbc0128c0 con 0x7f4fcc0ae0b0 2022-08-01T19:34:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.418+0000 7f4fd24bd700 1 -- 172.21.15.17:0/3266687201 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fcc0a0b80 con 0x7f4fcc0ae0b0 2022-08-01T19:34:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.418+0000 7f4fd24bd700 1 -- 172.21.15.17:0/3266687201 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fcc0b39c0 con 0x7f4fcc0ae0b0 2022-08-01T19:34:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.418+0000 7f4fc97fa700 1 -- 172.21.15.17:0/3266687201 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4fbc019030 con 0x7f4fcc0ae0b0 2022-08-01T19:34:12.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.418+0000 7f4fc97fa700 1 --2- 172.21.15.17:0/3266687201 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4fac0539e0 0x7f4fac055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:12.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.418+0000 7f4fc97fa700 1 -- 172.21.15.17:0/3266687201 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f4fbc06b370 con 0x7f4fcc0ae0b0 2022-08-01T19:34:12.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.418+0000 7f4fcb7fe700 1 --2- 172.21.15.17:0/3266687201 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4fac0539e0 0x7f4fac055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:12.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.422+0000 7f4fcb7fe700 1 --2- 172.21.15.17:0/3266687201 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4fac0539e0 0x7f4fac055ea0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f4fc000d000 tx=0x7f4fc0009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:12.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.430+0000 7f4fc97fa700 1 -- 172.21.15.17:0/3266687201 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4fbc042090 con 0x7f4fcc0ae0b0 2022-08-01T19:34:12.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.550+0000 7f4fd24bd700 1 -- 172.21.15.17:0/3266687201 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f4fcc0574e0 con 0x7f4fcc0ae0b0 2022-08-01T19:34:12.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.558+0000 7f4fc97fa700 1 -- 172.21.15.17:0/3266687201 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v25) v1 ==== 110+0+10 (secure 0 0 0) 0x7f4fbc0456b0 con 0x7f4fcc0ae0b0 2022-08-01T19:34:12.559 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:34:12.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.558+0000 7f4fd24bd700 1 -- 172.21.15.17:0/3266687201 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4fac0539e0 msgr2=0x7f4fac055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:12.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.558+0000 7f4fd24bd700 1 --2- 172.21.15.17:0/3266687201 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4fac0539e0 0x7f4fac055ea0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f4fc000d000 tx=0x7f4fc0009040).stop 2022-08-01T19:34:12.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.558+0000 7f4fd24bd700 1 -- 172.21.15.17:0/3266687201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fcc0ae0b0 msgr2=0x7f4fcc0acd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:12.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.558+0000 7f4fd24bd700 1 --2- 172.21.15.17:0/3266687201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fcc0ae0b0 0x7f4fcc0acd60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f4fcc05d020 tx=0x7f4fbc007b90).stop 2022-08-01T19:34:12.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.558+0000 7f4fd24bd700 1 -- 172.21.15.17:0/3266687201 shutdown_connections 2022-08-01T19:34:12.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.558+0000 7f4fd24bd700 1 --2- 172.21.15.17:0/3266687201 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4fac0539e0 0x7f4fac055ea0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:12.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.558+0000 7f4fd24bd700 1 --2- 172.21.15.17:0/3266687201 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4fcc0b29b0 0x7f4fcc067360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:12.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.558+0000 7f4fd24bd700 1 --2- 172.21.15.17:0/3266687201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fcc0b1580 0x7f4fcc066e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:12.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.558+0000 7f4fd24bd700 1 --2- 172.21.15.17:0/3266687201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fcc0ae0b0 0x7f4fcc0acd60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:12.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.558+0000 7f4fd24bd700 1 -- 172.21.15.17:0/3266687201 >> 172.21.15.17:0/3266687201 conn(0x7f4fcc060950 msgr2=0x7f4fcc0b5880 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:12.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.558+0000 7f4fd24bd700 1 -- 172.21.15.17:0/3266687201 shutdown_connections 2022-08-01T19:34:12.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.558+0000 7f4fd24bd700 1 -- 172.21.15.17:0/3266687201 wait complete. 2022-08-01T19:34:12.572 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:34:12.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c299a700 1 Processor -- start 2022-08-01T19:34:12.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c299a700 1 -- start start 2022-08-01T19:34:12.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c299a700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98bc0614b0 0x7f98bc061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:12.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c299a700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f98bc1405d0 0x7f98bc140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:12.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c299a700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98bc0aee80 con 0x7f98bc0ad750 2022-08-01T19:34:12.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c299a700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98bc0b70b0 con 0x7f98bc1405d0 2022-08-01T19:34:12.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c299a700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc0af0b0 con 0x7f98bc0614b0 2022-08-01T19:34:12.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c2199700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f98bc1405d0 0x7f98bc140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:12.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c2199700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f98bc1405d0 0x7f98bc140a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36316/0 (socket says 172.21.15.17:36316) 2022-08-01T19:34:12.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c2199700 1 -- 172.21.15.17:0/3076747918 learned_addr learned my addr 172.21.15.17:0/3076747918 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:12.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c1998700 1 --2- 172.21.15.17:0/3076747918 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98bc0614b0 0x7f98bc061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:12.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c2199700 1 -- 172.21.15.17:0/3076747918 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98bc0614b0 msgr2=0x7f98bc061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:12.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c2199700 1 --2- 172.21.15.17:0/3076747918 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98bc0614b0 0x7f98bc061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:12.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c2199700 1 -- 172.21.15.17:0/3076747918 >> v1:172.21.15.77:6789/0 conn(0x7f98bc0ad750 legacy=0x7f98bc0b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:12.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c2199700 1 -- 172.21.15.17:0/3076747918 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98bc0af350 con 0x7f98bc1405d0 2022-08-01T19:34:12.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c2199700 1 --2- 172.21.15.17:0/3076747918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98bc1405d0 0x7f98bc140a60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f98b4002f70 tx=0x7f98b4006eb0).ready entity=mon.0 client_cookie=28aba3a6165bed00 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:12.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c0996700 1 -- 172.21.15.17:0/3076747918 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98b400d570 con 0x7f98bc1405d0 2022-08-01T19:34:12.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c0996700 1 -- 172.21.15.17:0/3076747918 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b4000e90 con 0x7f98bc1405d0 2022-08-01T19:34:12.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c0996700 1 -- 172.21.15.17:0/3076747918 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98b400d900 con 0x7f98bc1405d0 2022-08-01T19:34:12.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c299a700 1 -- 172.21.15.17:0/3076747918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98bc1405d0 msgr2=0x7f98bc140a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:12.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c299a700 1 --2- 172.21.15.17:0/3076747918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98bc1405d0 0x7f98bc140a60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f98b4002f70 tx=0x7f98b4006eb0).stop 2022-08-01T19:34:12.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c299a700 1 -- 172.21.15.17:0/3076747918 shutdown_connections 2022-08-01T19:34:12.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c299a700 1 --2- 172.21.15.17:0/3076747918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98bc1405d0 0x7f98bc140a60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:12.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c299a700 1 --2- 172.21.15.17:0/3076747918 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98bc0614b0 0x7f98bc061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:12.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c299a700 1 -- 172.21.15.17:0/3076747918 >> 172.21.15.17:0/3076747918 conn(0x7f98bc066f70 msgr2=0x7f98bc067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:12.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c299a700 1 -- 172.21.15.17:0/3076747918 shutdown_connections 2022-08-01T19:34:12.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c299a700 1 -- 172.21.15.17:0/3076747918 wait complete. 2022-08-01T19:34:12.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c299a700 1 Processor -- start 2022-08-01T19:34:12.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c299a700 1 -- start start 2022-08-01T19:34:12.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c299a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98bc0614b0 0x7f98bc0ab880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:12.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c299a700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98bc0ad750 0x7f98bc05cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:12.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c299a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98bc1405d0 0x7f98bc0abe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:12.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c299a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc0ab220 con 0x7f98bc0614b0 2022-08-01T19:34:12.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c299a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc0aa520 con 0x7f98bc1405d0 2022-08-01T19:34:12.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c299a700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc0aa7c0 con 0x7f98bc0ad750 2022-08-01T19:34:12.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.682+0000 7f98c1998700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98bc0614b0 0x7f98bc0ab880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:12.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.686+0000 7f98c1998700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98bc0614b0 0x7f98bc0ab880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36320/0 (socket says 172.21.15.17:36320) 2022-08-01T19:34:12.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.686+0000 7f98c1998700 1 -- 172.21.15.17:0/3492821400 learned_addr learned my addr 172.21.15.17:0/3492821400 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:12.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.686+0000 7f98c1197700 1 --2- 172.21.15.17:0/3492821400 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98bc0ad750 0x7f98bc05cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:12.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.686+0000 7f98c2199700 1 --2- 172.21.15.17:0/3492821400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98bc1405d0 0x7f98bc0abe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:12.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.686+0000 7f98c1998700 1 -- 172.21.15.17:0/3492821400 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98bc0ad750 msgr2=0x7f98bc05cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:12.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.686+0000 7f98c1998700 1 --2- 172.21.15.17:0/3492821400 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98bc0ad750 0x7f98bc05cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:12.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.686+0000 7f98c1998700 1 -- 172.21.15.17:0/3492821400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98bc1405d0 msgr2=0x7f98bc0abe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:12.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.686+0000 7f98c1998700 1 --2- 172.21.15.17:0/3492821400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98bc1405d0 0x7f98bc0abe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:12.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.686+0000 7f98c1998700 1 -- 172.21.15.17:0/3492821400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b400b1e0 con 0x7f98bc0614b0 2022-08-01T19:34:12.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.686+0000 7f98c1998700 1 --2- 172.21.15.17:0/3492821400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98bc0614b0 0x7f98bc0ab880 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f98b000c000 tx=0x7f98b0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:12.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.686+0000 7f98aeffd700 1 -- 172.21.15.17:0/3492821400 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98b0010780 con 0x7f98bc0614b0 2022-08-01T19:34:12.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.686+0000 7f98aeffd700 1 -- 172.21.15.17:0/3492821400 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b00108e0 con 0x7f98bc0614b0 2022-08-01T19:34:12.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.686+0000 7f98aeffd700 1 -- 172.21.15.17:0/3492821400 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98b0010c40 con 0x7f98bc0614b0 2022-08-01T19:34:12.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.686+0000 7f98c299a700 1 -- 172.21.15.17:0/3492821400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98bc0b1470 con 0x7f98bc0614b0 2022-08-01T19:34:12.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.686+0000 7f98c299a700 1 -- 172.21.15.17:0/3492821400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98bc0ad260 con 0x7f98bc0614b0 2022-08-01T19:34:12.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.686+0000 7f98aeffd700 1 -- 172.21.15.17:0/3492821400 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f98b001c460 con 0x7f98bc0614b0 2022-08-01T19:34:12.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.686+0000 7f98c299a700 1 -- 172.21.15.17:0/3492821400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98bc0af5f0 con 0x7f98bc0614b0 2022-08-01T19:34:12.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.686+0000 7f98aeffd700 1 --2- 172.21.15.17:0/3492821400 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f989c053950 0x7f989c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:12.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.686+0000 7f98aeffd700 1 -- 172.21.15.17:0/3492821400 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f98b006c4d0 con 0x7f98bc0614b0 2022-08-01T19:34:12.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.686+0000 7f98c1197700 1 --2- 172.21.15.17:0/3492821400 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f989c053950 0x7f989c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:12.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.690+0000 7f98c1197700 1 --2- 172.21.15.17:0/3492821400 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f989c053950 0x7f989c055e10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f98b800d000 tx=0x7f98b8007f10).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:12.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.694+0000 7f98aeffd700 1 -- 172.21.15.17:0/3492821400 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f98b0043190 con 0x7f98bc0614b0 2022-08-01T19:34:12.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.818+0000 7f98c299a700 1 -- 172.21.15.17:0/3492821400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f98bc0574e0 con 0x7f98bc0614b0 2022-08-01T19:34:12.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.818+0000 7f98aeffd700 1 -- 172.21.15.17:0/3492821400 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f98b00467b0 con 0x7f98bc0614b0 2022-08-01T19:34:12.826 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:34:12.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.822+0000 7f98c299a700 1 -- 172.21.15.17:0/3492821400 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f989c053950 msgr2=0x7f989c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:12.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.822+0000 7f98c299a700 1 --2- 172.21.15.17:0/3492821400 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f989c053950 0x7f989c055e10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f98b800d000 tx=0x7f98b8007f10).stop 2022-08-01T19:34:12.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.822+0000 7f98c299a700 1 -- 172.21.15.17:0/3492821400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98bc0614b0 msgr2=0x7f98bc0ab880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:12.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.822+0000 7f98c299a700 1 --2- 172.21.15.17:0/3492821400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98bc0614b0 0x7f98bc0ab880 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f98b000c000 tx=0x7f98b0006df0).stop 2022-08-01T19:34:12.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.822+0000 7f98c299a700 1 -- 172.21.15.17:0/3492821400 shutdown_connections 2022-08-01T19:34:12.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.822+0000 7f98c299a700 1 --2- 172.21.15.17:0/3492821400 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f989c053950 0x7f989c055e10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:12.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.822+0000 7f98c299a700 1 --2- 172.21.15.17:0/3492821400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98bc1405d0 0x7f98bc0abe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:12.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.822+0000 7f98c299a700 1 --2- 172.21.15.17:0/3492821400 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98bc0ad750 0x7f98bc05cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:12.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.822+0000 7f98c299a700 1 --2- 172.21.15.17:0/3492821400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98bc0614b0 0x7f98bc0ab880 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:12.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.822+0000 7f98c299a700 1 -- 172.21.15.17:0/3492821400 >> 172.21.15.17:0/3492821400 conn(0x7f98bc066f70 msgr2=0x7f98bc0b8990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:12.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.822+0000 7f98c299a700 1 -- 172.21.15.17:0/3492821400 shutdown_connections 2022-08-01T19:34:12.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.822+0000 7f98c299a700 1 -- 172.21.15.17:0/3492821400 wait complete. 2022-08-01T19:34:12.844 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T19:34:12.951 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.946+0000 7f6383fff700 1 Processor -- start 2022-08-01T19:34:12.951 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.946+0000 7f6383fff700 1 -- start start 2022-08-01T19:34:12.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.946+0000 7f6383fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f63840ad750 0x7f6384061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:12.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.946+0000 7f6383fff700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63840aee80 con 0x7f63841405d0 2022-08-01T19:34:12.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.946+0000 7f6383fff700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63840b70b0 con 0x7f63840614b0 2022-08-01T19:34:12.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.946+0000 7f6383fff700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63840af0b0 con 0x7f63840ad750 2022-08-01T19:34:12.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.946+0000 7f63827fc700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f63840ad750 0x7f6384061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:12.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.946+0000 7f63837fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f63841405d0 0x7f63841466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56758/0 (socket says 172.21.15.17:56758) 2022-08-01T19:34:12.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.946+0000 7f63837fe700 1 -- 172.21.15.17:0/2787817648 learned_addr learned my addr 172.21.15.17:0/2787817648 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:12.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.946+0000 7f63827fc700 1 -- 172.21.15.17:0/2787817648 >> v1:172.21.15.77:6789/0 conn(0x7f63840614b0 legacy=0x7f63840b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:12.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.946+0000 7f63827fc700 1 -- 172.21.15.17:0/2787817648 >> v1:172.21.15.17:6789/0 conn(0x7f63841405d0 legacy=0x7f63841466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:12.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.946+0000 7f63827fc700 1 -- 172.21.15.17:0/2787817648 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63840af350 con 0x7f63840ad750 2022-08-01T19:34:12.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.946+0000 7f63827fc700 1 --2- 172.21.15.17:0/2787817648 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63840ad750 0x7f6384061890 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f637c0032f0 tx=0x7f637c0091d0).ready entity=mon.0 client_cookie=463d01db85972a98 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:12.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.946+0000 7f6381ffb700 1 -- 172.21.15.17:0/2787817648 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f637c009ea0 con 0x7f63840ad750 2022-08-01T19:34:12.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.946+0000 7f6381ffb700 1 -- 172.21.15.17:0/2787817648 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f637c00b8a0 con 0x7f63840ad750 2022-08-01T19:34:12.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.946+0000 7f6381ffb700 1 -- 172.21.15.17:0/2787817648 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f637c00ba10 con 0x7f63840ad750 2022-08-01T19:34:12.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.950+0000 7f6383fff700 1 -- 172.21.15.17:0/2787817648 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63840ad750 msgr2=0x7f6384061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:12.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.950+0000 7f6383fff700 1 --2- 172.21.15.17:0/2787817648 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63840ad750 0x7f6384061890 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f637c0032f0 tx=0x7f637c0091d0).stop 2022-08-01T19:34:12.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.950+0000 7f6383fff700 1 -- 172.21.15.17:0/2787817648 shutdown_connections 2022-08-01T19:34:12.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.950+0000 7f6383fff700 1 --2- 172.21.15.17:0/2787817648 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63840ad750 0x7f6384061890 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:12.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.950+0000 7f6383fff700 1 -- 172.21.15.17:0/2787817648 >> 172.21.15.17:0/2787817648 conn(0x7f6384066f70 msgr2=0x7f6384067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:12.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.950+0000 7f6383fff700 1 -- 172.21.15.17:0/2787817648 shutdown_connections 2022-08-01T19:34:12.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.950+0000 7f6383fff700 1 -- 172.21.15.17:0/2787817648 wait complete. 2022-08-01T19:34:12.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.950+0000 7f6383fff700 1 Processor -- start 2022-08-01T19:34:12.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.950+0000 7f6383fff700 1 -- start start 2022-08-01T19:34:12.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.950+0000 7f6383fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63840614b0 0x7f6384144850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:12.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.950+0000 7f6383fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f63840ad750 0x7f6384155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:12.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.950+0000 7f6383fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63841405d0 0x7f6384156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:12.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.950+0000 7f6383fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63840b40c0 con 0x7f63840614b0 2022-08-01T19:34:12.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.950+0000 7f6383fff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63840afac0 con 0x7f63841405d0 2022-08-01T19:34:12.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.950+0000 7f6383fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6384155b40 con 0x7f63840ad750 2022-08-01T19:34:12.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.950+0000 7f6382ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63840614b0 0x7f6384144850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:12.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.950+0000 7f6382ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63840614b0 0x7f6384144850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36334/0 (socket says 172.21.15.17:36334) 2022-08-01T19:34:12.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.950+0000 7f6382ffd700 1 -- 172.21.15.17:0/1822789441 learned_addr learned my addr 172.21.15.17:0/1822789441 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:12.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.950+0000 7f63837fe700 1 --2- 172.21.15.17:0/1822789441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63841405d0 0x7f6384156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:12.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.950+0000 7f63827fc700 1 --2- 172.21.15.17:0/1822789441 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f63840ad750 0x7f6384155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:12.966 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.954+0000 7f6382ffd700 1 -- 172.21.15.17:0/1822789441 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f63840ad750 msgr2=0x7f6384155fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:12.966 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.954+0000 7f6382ffd700 1 --2- 172.21.15.17:0/1822789441 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f63840ad750 0x7f6384155fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:12.966 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.954+0000 7f6382ffd700 1 -- 172.21.15.17:0/1822789441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63841405d0 msgr2=0x7f6384156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:12.967 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.954+0000 7f6382ffd700 1 --2- 172.21.15.17:0/1822789441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63841405d0 0x7f6384156490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:12.967 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.954+0000 7f6382ffd700 1 -- 172.21.15.17:0/1822789441 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637c009470 con 0x7f63840614b0 2022-08-01T19:34:12.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.954+0000 7f63837fe700 1 --2- 172.21.15.17:0/1822789441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63841405d0 0x7f6384156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:34:12.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.954+0000 7f6382ffd700 1 --2- 172.21.15.17:0/1822789441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63840614b0 0x7f6384144850 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f63840b4200 tx=0x7f6374007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:12.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.954+0000 7f637bfff700 1 -- 172.21.15.17:0/1822789441 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f637400ce20 con 0x7f63840614b0 2022-08-01T19:34:12.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.954+0000 7f637bfff700 1 -- 172.21.15.17:0/1822789441 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f637400f470 con 0x7f63840614b0 2022-08-01T19:34:12.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.954+0000 7f6383fff700 1 -- 172.21.15.17:0/1822789441 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6384156980 con 0x7f63840614b0 2022-08-01T19:34:12.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.954+0000 7f637bfff700 1 -- 172.21.15.17:0/1822789441 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f637400f7d0 con 0x7f63840614b0 2022-08-01T19:34:12.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.954+0000 7f6383fff700 1 -- 172.21.15.17:0/1822789441 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63840afcf0 con 0x7f63840614b0 2022-08-01T19:34:12.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.954+0000 7f6383fff700 1 -- 172.21.15.17:0/1822789441 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63840af5f0 con 0x7f63840614b0 2022-08-01T19:34:12.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.954+0000 7f637bfff700 1 -- 172.21.15.17:0/1822789441 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6374019030 con 0x7f63840614b0 2022-08-01T19:34:12.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.954+0000 7f637bfff700 1 --2- 172.21.15.17:0/1822789441 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f635c053950 0x7f635c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:12.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.954+0000 7f637bfff700 1 -- 172.21.15.17:0/1822789441 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f637406b4b0 con 0x7f63840614b0 2022-08-01T19:34:12.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.954+0000 7f63827fc700 1 --2- 172.21.15.17:0/1822789441 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f635c053950 0x7f635c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:12.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.958+0000 7f63827fc700 1 --2- 172.21.15.17:0/1822789441 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f635c053950 0x7f635c055e10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f637c019000 tx=0x7f637c016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:12.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:12.962+0000 7f637bfff700 1 -- 172.21.15.17:0/1822789441 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6374042270 con 0x7f63840614b0 2022-08-01T19:34:13.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.090+0000 7f6383fff700 1 -- 172.21.15.17:0/1822789441 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f63840574e0 con 0x7f63840614b0 2022-08-01T19:34:13.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.090+0000 7f637bfff700 1 -- 172.21.15.17:0/1822789441 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+10 (secure 0 0 0) 0x7f6374045890 con 0x7f63840614b0 2022-08-01T19:34:13.096 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:34:13.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.090+0000 7f6383fff700 1 -- 172.21.15.17:0/1822789441 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f635c053950 msgr2=0x7f635c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:13.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.090+0000 7f6383fff700 1 --2- 172.21.15.17:0/1822789441 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f635c053950 0x7f635c055e10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f637c019000 tx=0x7f637c016000).stop 2022-08-01T19:34:13.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.090+0000 7f6383fff700 1 -- 172.21.15.17:0/1822789441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63840614b0 msgr2=0x7f6384144850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:13.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.090+0000 7f6383fff700 1 --2- 172.21.15.17:0/1822789441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63840614b0 0x7f6384144850 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f63840b4200 tx=0x7f6374007c50).stop 2022-08-01T19:34:13.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.090+0000 7f6383fff700 1 -- 172.21.15.17:0/1822789441 shutdown_connections 2022-08-01T19:34:13.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.090+0000 7f6383fff700 1 --2- 172.21.15.17:0/1822789441 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f635c053950 0x7f635c055e10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:13.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.090+0000 7f6383fff700 1 --2- 172.21.15.17:0/1822789441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63841405d0 0x7f6384156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:13.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.090+0000 7f6383fff700 1 --2- 172.21.15.17:0/1822789441 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f63840ad750 0x7f6384155fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:13.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.090+0000 7f6383fff700 1 --2- 172.21.15.17:0/1822789441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63840614b0 0x7f6384144850 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:13.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.090+0000 7f6383fff700 1 -- 172.21.15.17:0/1822789441 >> 172.21.15.17:0/1822789441 conn(0x7f6384066f70 msgr2=0x7f63840b8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:13.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.090+0000 7f6383fff700 1 -- 172.21.15.17:0/1822789441 shutdown_connections 2022-08-01T19:34:13.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.090+0000 7f6383fff700 1 -- 172.21.15.17:0/1822789441 wait complete. 2022-08-01T19:34:13.109 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T19:34:13.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.214+0000 7fd10eca5700 1 Processor -- start 2022-08-01T19:34:13.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.214+0000 7fd10eca5700 1 -- start start 2022-08-01T19:34:13.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.214+0000 7fd10eca5700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd1080b0e60 0x7fd1080610d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:13.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.214+0000 7fd10eca5700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd108066af0 con 0x7fd1080b6510 2022-08-01T19:34:13.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.214+0000 7fd10eca5700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd108066d90 con 0x7fd10813d9b0 2022-08-01T19:34:13.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.214+0000 7fd10eca5700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd108067030 con 0x7fd1080b0e60 2022-08-01T19:34:13.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.218+0000 7fd10e4a4700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd1080b0e60 0x7fd1080610d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:13.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.218+0000 7fd10dca3700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd1080b6510 0x7fd10813d2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56768/0 (socket says 172.21.15.17:56768) 2022-08-01T19:34:13.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.218+0000 7fd10dca3700 1 -- 172.21.15.17:0/1184831767 learned_addr learned my addr 172.21.15.17:0/1184831767 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:13.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.218+0000 7fd10e4a4700 1 -- 172.21.15.17:0/1184831767 >> v1:172.21.15.77:6789/0 conn(0x7fd10813d9b0 legacy=0x7fd1080609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:13.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.218+0000 7fd10cca1700 1 -- 172.21.15.17:0/1184831767 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 14569946 0 0) 0x7fd108066af0 con 0x7fd1080b6510 2022-08-01T19:34:13.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.218+0000 7fd10e4a4700 1 -- 172.21.15.17:0/1184831767 >> v1:172.21.15.17:6789/0 conn(0x7fd1080b6510 legacy=0x7fd10813d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:13.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.218+0000 7fd10e4a4700 1 -- 172.21.15.17:0/1184831767 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1080672d0 con 0x7fd1080b0e60 2022-08-01T19:34:13.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.218+0000 7fd10e4a4700 1 --2- 172.21.15.17:0/1184831767 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1080b0e60 0x7fd1080610d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fd100002f70 tx=0x7fd100006df0).ready entity=mon.0 client_cookie=9c8caffc3ebb809d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:13.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.218+0000 7fd10cca1700 1 -- 172.21.15.17:0/1184831767 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd10000d610 con 0x7fd1080b0e60 2022-08-01T19:34:13.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.218+0000 7fd10cca1700 1 -- 172.21.15.17:0/1184831767 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd100000e90 con 0x7fd1080b0e60 2022-08-01T19:34:13.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.218+0000 7fd10cca1700 1 -- 172.21.15.17:0/1184831767 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd10000d960 con 0x7fd1080b0e60 2022-08-01T19:34:13.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.218+0000 7fd10eca5700 1 -- 172.21.15.17:0/1184831767 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1080b0e60 msgr2=0x7fd1080610d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:13.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.218+0000 7fd10eca5700 1 --2- 172.21.15.17:0/1184831767 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1080b0e60 0x7fd1080610d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fd100002f70 tx=0x7fd100006df0).stop 2022-08-01T19:34:13.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.218+0000 7fd10eca5700 1 -- 172.21.15.17:0/1184831767 shutdown_connections 2022-08-01T19:34:13.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.218+0000 7fd10eca5700 1 --2- 172.21.15.17:0/1184831767 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1080b0e60 0x7fd1080610d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:13.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.218+0000 7fd10eca5700 1 -- 172.21.15.17:0/1184831767 >> 172.21.15.17:0/1184831767 conn(0x7fd108063800 msgr2=0x7fd108063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:13.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.218+0000 7fd10eca5700 1 -- 172.21.15.17:0/1184831767 shutdown_connections 2022-08-01T19:34:13.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.218+0000 7fd10eca5700 1 -- 172.21.15.17:0/1184831767 wait complete. 2022-08-01T19:34:13.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.218+0000 7fd10eca5700 1 Processor -- start 2022-08-01T19:34:13.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.218+0000 7fd10eca5700 1 -- start start 2022-08-01T19:34:13.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.222+0000 7fd10eca5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1080b0e60 0x7fd108143a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:13.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.222+0000 7fd10eca5700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd1080b6510 0x7fd108142c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:13.234 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.222+0000 7fd10eca5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd10813d9b0 0x7fd108154c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:13.234 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.222+0000 7fd10eca5700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1081414d0 con 0x7fd10813d9b0 2022-08-01T19:34:13.234 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.222+0000 7fd10eca5700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10813e730 con 0x7fd1080b0e60 2022-08-01T19:34:13.234 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.222+0000 7fd10eca5700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd108143330 con 0x7fd1080b6510 2022-08-01T19:34:13.234 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.222+0000 7fd10e4a4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd10813d9b0 0x7fd108154c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:13.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.222+0000 7fd10e4a4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd10813d9b0 0x7fd108154c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36352/0 (socket says 172.21.15.17:36352) 2022-08-01T19:34:13.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.222+0000 7fd10e4a4700 1 -- 172.21.15.17:0/3735090379 learned_addr learned my addr 172.21.15.17:0/3735090379 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:13.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.222+0000 7fd10dca3700 1 --2- 172.21.15.17:0/3735090379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1080b0e60 0x7fd108143a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:13.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.222+0000 7fd10d4a2700 1 --2- 172.21.15.17:0/3735090379 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd1080b6510 0x7fd108142c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:13.236 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.222+0000 7fd10e4a4700 1 -- 172.21.15.17:0/3735090379 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd1080b6510 msgr2=0x7fd108142c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:13.236 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.222+0000 7fd10e4a4700 1 --2- 172.21.15.17:0/3735090379 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd1080b6510 0x7fd108142c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:13.236 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.222+0000 7fd10e4a4700 1 -- 172.21.15.17:0/3735090379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1080b0e60 msgr2=0x7fd108143a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:13.236 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.222+0000 7fd10e4a4700 1 --2- 172.21.15.17:0/3735090379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1080b0e60 0x7fd108143a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:13.236 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.222+0000 7fd10e4a4700 1 -- 172.21.15.17:0/3735090379 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd10000b180 con 0x7fd10813d9b0 2022-08-01T19:34:13.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.222+0000 7fd10d4a2700 1 --2- 172.21.15.17:0/3735090379 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd1080b6510 0x7fd108142c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:34:13.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.222+0000 7fd10e4a4700 1 --2- 172.21.15.17:0/3735090379 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd10813d9b0 0x7fd108154c40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fd10813e8c0 tx=0x7fd100015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:13.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.222+0000 7fd106ffd700 1 -- 172.21.15.17:0/3735090379 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd10000d990 con 0x7fd10813d9b0 2022-08-01T19:34:13.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.222+0000 7fd106ffd700 1 -- 172.21.15.17:0/3735090379 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd100011c90 con 0x7fd10813d9b0 2022-08-01T19:34:13.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.222+0000 7fd10eca5700 1 -- 172.21.15.17:0/3735090379 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd108141700 con 0x7fd10813d9b0 2022-08-01T19:34:13.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.222+0000 7fd106ffd700 1 -- 172.21.15.17:0/3735090379 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd10001e460 con 0x7fd10813d9b0 2022-08-01T19:34:13.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.222+0000 7fd10eca5700 1 -- 172.21.15.17:0/3735090379 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd108142090 con 0x7fd10813d9b0 2022-08-01T19:34:13.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.222+0000 7fd10eca5700 1 -- 172.21.15.17:0/3735090379 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd108141ba0 con 0x7fd10813d9b0 2022-08-01T19:34:13.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.222+0000 7fd106ffd700 1 -- 172.21.15.17:0/3735090379 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd100025030 con 0x7fd10813d9b0 2022-08-01T19:34:13.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.226+0000 7fd106ffd700 1 --2- 172.21.15.17:0/3735090379 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd0e8053b00 0x7fd0e8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:13.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.226+0000 7fd106ffd700 1 -- 172.21.15.17:0/3735090379 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fd10006d520 con 0x7fd10813d9b0 2022-08-01T19:34:13.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.226+0000 7fd10dca3700 1 --2- 172.21.15.17:0/3735090379 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd0e8053b00 0x7fd0e8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:13.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.226+0000 7fd10dca3700 1 --2- 172.21.15.17:0/3735090379 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd0e8053b00 0x7fd0e8055fc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fd0f800b000 tx=0x7fd0f8009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:13.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.230+0000 7fd106ffd700 1 -- 172.21.15.17:0/3735090379 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd1000442e0 con 0x7fd10813d9b0 2022-08-01T19:34:13.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.354+0000 7fd10eca5700 1 -- 172.21.15.17:0/3735090379 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd1080574e0 con 0x7fd10813d9b0 2022-08-01T19:34:13.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.358+0000 7fd106ffd700 1 -- 172.21.15.17:0/3735090379 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+10 (secure 0 0 0) 0x7fd100023020 con 0x7fd10813d9b0 2022-08-01T19:34:13.363 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:34:13.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.358+0000 7fd10eca5700 1 -- 172.21.15.17:0/3735090379 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd0e8053b00 msgr2=0x7fd0e8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:13.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.358+0000 7fd10eca5700 1 --2- 172.21.15.17:0/3735090379 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd0e8053b00 0x7fd0e8055fc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fd0f800b000 tx=0x7fd0f8009040).stop 2022-08-01T19:34:13.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.358+0000 7fd10eca5700 1 -- 172.21.15.17:0/3735090379 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd10813d9b0 msgr2=0x7fd108154c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:13.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.358+0000 7fd10eca5700 1 --2- 172.21.15.17:0/3735090379 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd10813d9b0 0x7fd108154c40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fd10813e8c0 tx=0x7fd100015000).stop 2022-08-01T19:34:13.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.358+0000 7fd10eca5700 1 -- 172.21.15.17:0/3735090379 shutdown_connections 2022-08-01T19:34:13.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.358+0000 7fd10eca5700 1 --2- 172.21.15.17:0/3735090379 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd0e8053b00 0x7fd0e8055fc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:13.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.358+0000 7fd10eca5700 1 --2- 172.21.15.17:0/3735090379 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd10813d9b0 0x7fd108154c40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:13.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.358+0000 7fd10eca5700 1 --2- 172.21.15.17:0/3735090379 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd1080b6510 0x7fd108142c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:13.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.358+0000 7fd10eca5700 1 --2- 172.21.15.17:0/3735090379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1080b0e60 0x7fd108143a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:13.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.358+0000 7fd10eca5700 1 -- 172.21.15.17:0/3735090379 >> 172.21.15.17:0/3735090379 conn(0x7fd108063800 msgr2=0x7fd108063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:13.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.358+0000 7fd10eca5700 1 -- 172.21.15.17:0/3735090379 shutdown_connections 2022-08-01T19:34:13.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.358+0000 7fd10eca5700 1 -- 172.21.15.17:0/3735090379 wait complete. 2022-08-01T19:34:13.376 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2022-08-01T19:34:13.376 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2022-08-01T19:34:13.377 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2022-08-01T19:34:13.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.470+0000 7f0b73fff700 1 Processor -- start 2022-08-01T19:34:13.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.470+0000 7f0b73fff700 1 -- start start 2022-08-01T19:34:13.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.474+0000 7f0b73fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0b740ad750 0x7f0b74061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:13.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.474+0000 7f0b73fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0b741405d0 0x7f0b74140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:13.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.474+0000 7f0b73fff700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b740aee80 con 0x7f0b740614b0 2022-08-01T19:34:13.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.474+0000 7f0b73fff700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b740b70b0 con 0x7f0b741405d0 2022-08-01T19:34:13.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.474+0000 7f0b73fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b740af0b0 con 0x7f0b740ad750 2022-08-01T19:34:13.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.474+0000 7f0b72ffd700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f0b740614b0 0x7f0b740b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56782/0 (socket says 172.21.15.17:56782) 2022-08-01T19:34:13.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.474+0000 7f0b72ffd700 1 -- 172.21.15.17:0/903322993 learned_addr learned my addr 172.21.15.17:0/903322993 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:13.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.474+0000 7f0b737fe700 1 --2- 172.21.15.17:0/903322993 >> v2:172.21.15.17:3300/0 conn(0x7f0b741405d0 0x7f0b74140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:13.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.474+0000 7f0b727fc700 1 --2- 172.21.15.17:0/903322993 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0b740ad750 0x7f0b74061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:13.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.474+0000 7f0b71ffb700 1 -- 172.21.15.17:0/903322993 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3613483957 0 0) 0x7f0b740aee80 con 0x7f0b740614b0 2022-08-01T19:34:13.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.474+0000 7f0b71ffb700 1 -- 172.21.15.17:0/903322993 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b58003260 con 0x7f0b740614b0 2022-08-01T19:34:13.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.474+0000 7f0b71ffb700 1 -- 172.21.15.17:0/903322993 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 744056903 0 0) 0x7f0b58003260 con 0x7f0b740614b0 2022-08-01T19:34:13.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.474+0000 7f0b737fe700 1 -- 172.21.15.17:0/903322993 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0b740ad750 msgr2=0x7f0b74061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:13.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.474+0000 7f0b737fe700 1 --2- 172.21.15.17:0/903322993 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0b740ad750 0x7f0b74061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:13.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.474+0000 7f0b737fe700 1 -- 172.21.15.17:0/903322993 >> v1:172.21.15.17:6789/0 conn(0x7f0b740614b0 legacy=0x7f0b740b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:13.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.474+0000 7f0b737fe700 1 -- 172.21.15.17:0/903322993 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b740af350 con 0x7f0b741405d0 2022-08-01T19:34:13.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.474+0000 7f0b737fe700 1 --2- 172.21.15.17:0/903322993 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b741405d0 0x7f0b74140a80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f0b64001800 tx=0x7f0b6c0091d0).ready entity=mon.0 client_cookie=dd499e88a36b6214 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:13.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.474+0000 7f0b71ffb700 1 -- 172.21.15.17:0/903322993 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b64001660 con 0x7f0b741405d0 2022-08-01T19:34:13.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.474+0000 7f0b71ffb700 1 -- 172.21.15.17:0/903322993 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b6c00f510 con 0x7f0b741405d0 2022-08-01T19:34:13.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.474+0000 7f0b71ffb700 1 -- 172.21.15.17:0/903322993 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b6c00f680 con 0x7f0b741405d0 2022-08-01T19:34:13.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.474+0000 7f0b73fff700 1 -- 172.21.15.17:0/903322993 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b741405d0 msgr2=0x7f0b74140a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:13.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.474+0000 7f0b73fff700 1 --2- 172.21.15.17:0/903322993 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b741405d0 0x7f0b74140a80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f0b64001800 tx=0x7f0b6c0091d0).stop 2022-08-01T19:34:13.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.474+0000 7f0b73fff700 1 -- 172.21.15.17:0/903322993 shutdown_connections 2022-08-01T19:34:13.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.474+0000 7f0b73fff700 1 --2- 172.21.15.17:0/903322993 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b741405d0 0x7f0b74140a80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:13.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.474+0000 7f0b73fff700 1 --2- 172.21.15.17:0/903322993 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0b740ad750 0x7f0b74061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:13.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.474+0000 7f0b73fff700 1 -- 172.21.15.17:0/903322993 >> 172.21.15.17:0/903322993 conn(0x7f0b74066f70 msgr2=0x7f0b74067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:13.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.474+0000 7f0b73fff700 1 -- 172.21.15.17:0/903322993 shutdown_connections 2022-08-01T19:34:13.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.474+0000 7f0b73fff700 1 -- 172.21.15.17:0/903322993 wait complete. 2022-08-01T19:34:13.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.478+0000 7f0b73fff700 1 Processor -- start 2022-08-01T19:34:13.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.478+0000 7f0b73fff700 1 -- start start 2022-08-01T19:34:13.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.478+0000 7f0b73fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b740614b0 0x7f0b740b39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:13.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.478+0000 7f0b73fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b740ad750 0x7f0b740b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:13.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.478+0000 7f0b73fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0b741405d0 0x7f0b74152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:13.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.478+0000 7f0b73fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b740b2ba0 con 0x7f0b740614b0 2022-08-01T19:34:13.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.478+0000 7f0b73fff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b74145990 con 0x7f0b740ad750 2022-08-01T19:34:13.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.478+0000 7f0b73fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b74145bf0 con 0x7f0b741405d0 2022-08-01T19:34:13.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.478+0000 7f0b72ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b740614b0 0x7f0b740b39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:13.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.478+0000 7f0b72ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b740614b0 0x7f0b740b39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36362/0 (socket says 172.21.15.17:36362) 2022-08-01T19:34:13.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.478+0000 7f0b72ffd700 1 -- 172.21.15.17:0/3318996523 learned_addr learned my addr 172.21.15.17:0/3318996523 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:13.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.478+0000 7f0b727fc700 1 --2- 172.21.15.17:0/3318996523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b740ad750 0x7f0b740b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:13.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.478+0000 7f0b737fe700 1 --2- 172.21.15.17:0/3318996523 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0b741405d0 0x7f0b74152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:13.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.478+0000 7f0b72ffd700 1 -- 172.21.15.17:0/3318996523 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0b741405d0 msgr2=0x7f0b74152440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:13.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.478+0000 7f0b72ffd700 1 --2- 172.21.15.17:0/3318996523 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0b741405d0 0x7f0b74152440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:13.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.478+0000 7f0b72ffd700 1 -- 172.21.15.17:0/3318996523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b740ad750 msgr2=0x7f0b740b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:13.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.478+0000 7f0b72ffd700 1 --2- 172.21.15.17:0/3318996523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b740ad750 0x7f0b740b1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:13.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.478+0000 7f0b72ffd700 1 -- 172.21.15.17:0/3318996523 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6c009470 con 0x7f0b740614b0 2022-08-01T19:34:13.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.478+0000 7f0b72ffd700 1 --2- 172.21.15.17:0/3318996523 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b740614b0 0x7f0b740b39a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f0b640018c0 tx=0x7f0b64001b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:13.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.478+0000 7f0b63fff700 1 -- 172.21.15.17:0/3318996523 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b64007170 con 0x7f0b740614b0 2022-08-01T19:34:13.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.478+0000 7f0b63fff700 1 -- 172.21.15.17:0/3318996523 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b640072d0 con 0x7f0b740614b0 2022-08-01T19:34:13.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.478+0000 7f0b73fff700 1 -- 172.21.15.17:0/3318996523 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b74155ab0 con 0x7f0b740614b0 2022-08-01T19:34:13.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.478+0000 7f0b63fff700 1 -- 172.21.15.17:0/3318996523 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b64004920 con 0x7f0b740614b0 2022-08-01T19:34:13.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.478+0000 7f0b73fff700 1 -- 172.21.15.17:0/3318996523 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b7414a470 con 0x7f0b740614b0 2022-08-01T19:34:13.499 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.482+0000 7f0b73fff700 1 -- 172.21.15.17:0/3318996523 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b740af5f0 con 0x7f0b740614b0 2022-08-01T19:34:13.501 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.482+0000 7f0b63fff700 1 -- 172.21.15.17:0/3318996523 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0b64004a80 con 0x7f0b740614b0 2022-08-01T19:34:13.501 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.482+0000 7f0b63fff700 1 --2- 172.21.15.17:0/3318996523 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0b50053a70 0x7f0b50055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:13.502 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.482+0000 7f0b63fff700 1 -- 172.21.15.17:0/3318996523 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f0b6406bc50 con 0x7f0b740614b0 2022-08-01T19:34:13.502 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.482+0000 7f0b727fc700 1 --2- 172.21.15.17:0/3318996523 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0b50053a70 0x7f0b50055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:13.502 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.486+0000 7f0b727fc700 1 --2- 172.21.15.17:0/3318996523 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0b50053a70 0x7f0b50055f30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f0b6800b000 tx=0x7f0b68006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:13.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.490+0000 7f0b63fff700 1 -- 172.21.15.17:0/3318996523 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0b64042a10 con 0x7f0b740614b0 2022-08-01T19:34:13.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.634+0000 7f0b73fff700 1 -- 172.21.15.17:0/3318996523 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f0b740574e0 con 0x7f0b740614b0 2022-08-01T19:34:13.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.638+0000 7f0b63fff700 1 -- 172.21.15.17:0/3318996523 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v4) v1 ==== 74+0+131775 (secure 0 0 0) 0x7f0b64001c50 con 0x7f0b740614b0 2022-08-01T19:34:13.647 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:13.649 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.642+0000 7f0b73fff700 1 -- 172.21.15.17:0/3318996523 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0b50053a70 msgr2=0x7f0b50055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:13.649 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.642+0000 7f0b73fff700 1 --2- 172.21.15.17:0/3318996523 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0b50053a70 0x7f0b50055f30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f0b6800b000 tx=0x7f0b68006df0).stop 2022-08-01T19:34:13.649 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.642+0000 7f0b73fff700 1 -- 172.21.15.17:0/3318996523 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b740614b0 msgr2=0x7f0b740b39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:13.650 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.642+0000 7f0b73fff700 1 --2- 172.21.15.17:0/3318996523 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b740614b0 0x7f0b740b39a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f0b640018c0 tx=0x7f0b64001b30).stop 2022-08-01T19:34:13.650 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.642+0000 7f0b73fff700 1 -- 172.21.15.17:0/3318996523 shutdown_connections 2022-08-01T19:34:13.650 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.642+0000 7f0b73fff700 1 --2- 172.21.15.17:0/3318996523 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0b50053a70 0x7f0b50055f30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:13.650 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.642+0000 7f0b73fff700 1 --2- 172.21.15.17:0/3318996523 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0b741405d0 0x7f0b74152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:13.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.642+0000 7f0b73fff700 1 --2- 172.21.15.17:0/3318996523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b740ad750 0x7f0b740b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:13.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.642+0000 7f0b73fff700 1 --2- 172.21.15.17:0/3318996523 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b740614b0 0x7f0b740b39a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:13.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.642+0000 7f0b73fff700 1 -- 172.21.15.17:0/3318996523 >> 172.21.15.17:0/3318996523 conn(0x7f0b74066f70 msgr2=0x7f0b74067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:13.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.642+0000 7f0b73fff700 1 -- 172.21.15.17:0/3318996523 shutdown_connections 2022-08-01T19:34:13.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.642+0000 7f0b73fff700 1 -- 172.21.15.17:0/3318996523 wait complete. 2022-08-01T19:34:13.664 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":4,"active_gid":4101,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":23151},{"type":"v1","addr":"172.21.15.77:6801","nonce":23151}]},"active_addr":"172.21.15.77:6801/23151","active_change":"2022-08-01T19:33:52.939849+0000","active_mgr_features":4540138292840890367,"available":true,"standbys":[{"gid":4108,"name":"y","mgr_features":4540138292840890367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.77:0","nonce":2594219895}]},{"addrvec":[{"type":"v2","addr":"172.21.15.77:0","nonce":3560608292}]},{"addrvec":[{"type":"v2","addr":"172.21.15.77:0","nonce":557872593}]}]}} 2022-08-01T19:34:13.668 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2022-08-01T19:34:13.668 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2022-08-01T19:34:13.668 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:34:13.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.770+0000 7fac5d71c700 1 Processor -- start 2022-08-01T19:34:13.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.770+0000 7fac5d71c700 1 -- start start 2022-08-01T19:34:13.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.770+0000 7fac5d71c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac580b0570 0x7fac580af120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:13.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.770+0000 7fac5d71c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fac5805c620 0x7fac5813fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:13.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.770+0000 7fac5d71c700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac580af610 con 0x7fac580ad670 2022-08-01T19:34:13.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.770+0000 7fac5d71c700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac580b0c40 con 0x7fac5805c620 2022-08-01T19:34:13.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.770+0000 7fac5d71c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac580b0ee0 con 0x7fac580b0570 2022-08-01T19:34:13.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.770+0000 7fac57fff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fac580ad670 0x7fac580b12a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56796/0 (socket says 172.21.15.17:56796) 2022-08-01T19:34:13.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.770+0000 7fac57fff700 1 -- 172.21.15.17:0/789807359 learned_addr learned my addr 172.21.15.17:0/789807359 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:13.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.770+0000 7fac5cf1b700 1 --2- 172.21.15.17:0/789807359 >> v2:172.21.15.17:3300/0 conn(0x7fac5805c620 0x7fac5813fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:13.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.770+0000 7fac577fe700 1 --2- 172.21.15.17:0/789807359 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac580b0570 0x7fac580af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:13.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.770+0000 7fac5cf1b700 1 -- 172.21.15.17:0/789807359 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac580b0570 msgr2=0x7fac580af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:13.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.770+0000 7fac5cf1b700 1 --2- 172.21.15.17:0/789807359 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac580b0570 0x7fac580af120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:13.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.770+0000 7fac5cf1b700 1 -- 172.21.15.17:0/789807359 >> v1:172.21.15.17:6789/0 conn(0x7fac580ad670 legacy=0x7fac580b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:13.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.770+0000 7fac56ffd700 1 -- 172.21.15.17:0/789807359 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2786792219 0 0) 0x7fac40001110 con 0x7fac580ad670 2022-08-01T19:34:13.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.770+0000 7fac5cf1b700 1 -- 172.21.15.17:0/789807359 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac580ae9e0 con 0x7fac5805c620 2022-08-01T19:34:13.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.770+0000 7fac5cf1b700 1 --2- 172.21.15.17:0/789807359 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac5805c620 0x7fac5813fa40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fac500032f0 tx=0x7fac500091d0).ready entity=mon.0 client_cookie=df0fbe4508ee05e3 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:13.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.770+0000 7fac56ffd700 1 -- 172.21.15.17:0/789807359 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac5000b6c0 con 0x7fac5805c620 2022-08-01T19:34:13.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.770+0000 7fac56ffd700 1 -- 172.21.15.17:0/789807359 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac5000b820 con 0x7fac5805c620 2022-08-01T19:34:13.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.770+0000 7fac56ffd700 1 -- 172.21.15.17:0/789807359 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac5000b990 con 0x7fac5805c620 2022-08-01T19:34:13.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.770+0000 7fac5d71c700 1 -- 172.21.15.17:0/789807359 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac5805c620 msgr2=0x7fac5813fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:13.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.770+0000 7fac5d71c700 1 --2- 172.21.15.17:0/789807359 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac5805c620 0x7fac5813fa40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fac500032f0 tx=0x7fac500091d0).stop 2022-08-01T19:34:13.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.770+0000 7fac5d71c700 1 -- 172.21.15.17:0/789807359 shutdown_connections 2022-08-01T19:34:13.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.770+0000 7fac5d71c700 1 --2- 172.21.15.17:0/789807359 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac5805c620 0x7fac5813fa40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:13.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.770+0000 7fac5d71c700 1 --2- 172.21.15.17:0/789807359 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac580b0570 0x7fac580af120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:13.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.770+0000 7fac5d71c700 1 -- 172.21.15.17:0/789807359 >> 172.21.15.17:0/789807359 conn(0x7fac58061360 msgr2=0x7fac58062f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:13.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.770+0000 7fac5d71c700 1 -- 172.21.15.17:0/789807359 shutdown_connections 2022-08-01T19:34:13.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.770+0000 7fac5d71c700 1 -- 172.21.15.17:0/789807359 wait complete. 2022-08-01T19:34:13.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.774+0000 7fac5d71c700 1 Processor -- start 2022-08-01T19:34:13.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.774+0000 7fac5d71c700 1 -- start start 2022-08-01T19:34:13.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.782+0000 7fac5d71c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac5805c620 0x7fac581425b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:13.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.782+0000 7fac5d71c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac580ad670 0x7fac580ac1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:13.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.782+0000 7fac5d71c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac580b0570 0x7fac580ac6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:13.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.782+0000 7fac5d71c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac580a97f0 con 0x7fac5805c620 2022-08-01T19:34:13.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.782+0000 7fac5d71c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac58134d00 con 0x7fac580ad670 2022-08-01T19:34:13.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.782+0000 7fac5d71c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac58134fa0 con 0x7fac580b0570 2022-08-01T19:34:13.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.782+0000 7fac57fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac5805c620 0x7fac581425b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:13.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.782+0000 7fac57fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac5805c620 0x7fac581425b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36376/0 (socket says 172.21.15.17:36376) 2022-08-01T19:34:13.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.782+0000 7fac57fff700 1 -- 172.21.15.17:0/1612164963 learned_addr learned my addr 172.21.15.17:0/1612164963 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:13.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.782+0000 7fac577fe700 1 --2- 172.21.15.17:0/1612164963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac580ad670 0x7fac580ac1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:13.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.782+0000 7fac5cf1b700 1 --2- 172.21.15.17:0/1612164963 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac580b0570 0x7fac580ac6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:13.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.782+0000 7fac57fff700 1 -- 172.21.15.17:0/1612164963 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac580b0570 msgr2=0x7fac580ac6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:13.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.782+0000 7fac57fff700 1 --2- 172.21.15.17:0/1612164963 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac580b0570 0x7fac580ac6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:13.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.782+0000 7fac57fff700 1 -- 172.21.15.17:0/1612164963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac580ad670 msgr2=0x7fac580ac1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:13.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.782+0000 7fac57fff700 1 --2- 172.21.15.17:0/1612164963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac580ad670 0x7fac580ac1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:13.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.782+0000 7fac57fff700 1 -- 172.21.15.17:0/1612164963 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac50009470 con 0x7fac5805c620 2022-08-01T19:34:13.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.782+0000 7fac57fff700 1 --2- 172.21.15.17:0/1612164963 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac5805c620 0x7fac581425b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fac40000ed0 tx=0x7fac40007920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:13.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.782+0000 7fac557fa700 1 -- 172.21.15.17:0/1612164963 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac4000ce40 con 0x7fac5805c620 2022-08-01T19:34:13.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.782+0000 7fac557fa700 1 -- 172.21.15.17:0/1612164963 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac400072e0 con 0x7fac5805c620 2022-08-01T19:34:13.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.782+0000 7fac557fa700 1 -- 172.21.15.17:0/1612164963 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac40005800 con 0x7fac5805c620 2022-08-01T19:34:13.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.782+0000 7fac5d71c700 1 -- 172.21.15.17:0/1612164963 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac580b5280 con 0x7fac5805c620 2022-08-01T19:34:13.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.782+0000 7fac5d71c700 1 -- 172.21.15.17:0/1612164963 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac581366b0 con 0x7fac5805c620 2022-08-01T19:34:13.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.782+0000 7fac5d71c700 1 -- 172.21.15.17:0/1612164963 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac58142aa0 con 0x7fac5805c620 2022-08-01T19:34:13.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.782+0000 7fac557fa700 1 -- 172.21.15.17:0/1612164963 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fac40010070 con 0x7fac5805c620 2022-08-01T19:34:13.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.782+0000 7fac557fa700 1 --2- 172.21.15.17:0/1612164963 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fac3c053b00 0x7fac3c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:13.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.782+0000 7fac557fa700 1 -- 172.21.15.17:0/1612164963 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fac4006b540 con 0x7fac5805c620 2022-08-01T19:34:13.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.782+0000 7fac577fe700 1 --2- 172.21.15.17:0/1612164963 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fac3c053b00 0x7fac3c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:13.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.782+0000 7fac577fe700 1 --2- 172.21.15.17:0/1612164963 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fac3c053b00 0x7fac3c055fc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fac4800b000 tx=0x7fac48006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:13.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.786+0000 7fac557fa700 1 -- 172.21.15.17:0/1612164963 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fac40042300 con 0x7fac5805c620 2022-08-01T19:34:13.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.914+0000 7fac5d71c700 1 -- 172.21.15.17:0/1612164963 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac580af610 con 0x7fac5805c620 2022-08-01T19:34:13.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.918+0000 7fac557fa700 1 -- 172.21.15.17:0/1612164963 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16299 (secure 0 0 0) 0x7fac580af610 con 0x7fac5805c620 2022-08-01T19:34:13.923 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:13.924 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:34:09.692404+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:34:04.635473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-08-01T19:34:07.409434+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:34:13.927 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.918+0000 7fac5d71c700 1 -- 172.21.15.17:0/1612164963 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fac3c053b00 msgr2=0x7fac3c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:13.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.918+0000 7fac5d71c700 1 --2- 172.21.15.17:0/1612164963 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fac3c053b00 0x7fac3c055fc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fac4800b000 tx=0x7fac48006df0).stop 2022-08-01T19:34:13.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.918+0000 7fac5d71c700 1 -- 172.21.15.17:0/1612164963 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac5805c620 msgr2=0x7fac581425b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:13.929 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.918+0000 7fac5d71c700 1 --2- 172.21.15.17:0/1612164963 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac5805c620 0x7fac581425b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fac40000ed0 tx=0x7fac40007920).stop 2022-08-01T19:34:13.929 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.918+0000 7fac5d71c700 1 -- 172.21.15.17:0/1612164963 shutdown_connections 2022-08-01T19:34:13.930 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.918+0000 7fac5d71c700 1 --2- 172.21.15.17:0/1612164963 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fac3c053b00 0x7fac3c055fc0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:13.930 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.918+0000 7fac5d71c700 1 --2- 172.21.15.17:0/1612164963 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac580b0570 0x7fac580ac6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:13.931 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.918+0000 7fac5d71c700 1 --2- 172.21.15.17:0/1612164963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac580ad670 0x7fac580ac1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:13.931 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.918+0000 7fac5d71c700 1 --2- 172.21.15.17:0/1612164963 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac5805c620 0x7fac581425b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:13.932 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.918+0000 7fac5d71c700 1 -- 172.21.15.17:0/1612164963 >> 172.21.15.17:0/1612164963 conn(0x7fac58061360 msgr2=0x7fac581436d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:13.932 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.918+0000 7fac5d71c700 1 -- 172.21.15.17:0/1612164963 shutdown_connections 2022-08-01T19:34:13.933 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:13.918+0000 7fac5d71c700 1 -- 172.21.15.17:0/1612164963 wait complete. 2022-08-01T19:34:13.941 INFO:tasks.ceph.ceph_manager.ceph:all up! 2022-08-01T19:34:13.942 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:34:14.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.042+0000 7f2351dcd700 1 Processor -- start 2022-08-01T19:34:14.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.042+0000 7f2351dcd700 1 -- start start 2022-08-01T19:34:14.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.042+0000 7f2351dcd700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f234c0a8c50 0x7f234c152d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.042+0000 7f2351dcd700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f234c0a7250 con 0x7f234c0b2710 2022-08-01T19:34:14.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.042+0000 7f2351dcd700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f234c0a7480 con 0x7f234c0b01a0 2022-08-01T19:34:14.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.042+0000 7f2351dcd700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f234c0a7720 con 0x7f234c0a8c50 2022-08-01T19:34:14.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.042+0000 7f2350dcb700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f234c0b2710 0x7f234c14afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56812/0 (socket says 172.21.15.17:56812) 2022-08-01T19:34:14.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.042+0000 7f2350dcb700 1 -- 172.21.15.17:0/2613353229 learned_addr learned my addr 172.21.15.17:0/2613353229 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:14.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.042+0000 7f23515cc700 1 --2- 172.21.15.17:0/2613353229 >> v2:172.21.15.77:3300/0 conn(0x7f234c0a8c50 0x7f234c152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.042+0000 7f234b7fe700 1 -- 172.21.15.17:0/2613353229 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3430211091 0 0) 0x7f234c0a7480 con 0x7f234c0b01a0 2022-08-01T19:34:14.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.042+0000 7f234b7fe700 1 -- 172.21.15.17:0/2613353229 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2338003260 con 0x7f234c0b01a0 2022-08-01T19:34:14.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.042+0000 7f234b7fe700 1 -- 172.21.15.17:0/2613353229 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3831116604 0 0) 0x7f234c0a7250 con 0x7f234c0b2710 2022-08-01T19:34:14.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.042+0000 7f234b7fe700 1 -- 172.21.15.17:0/2613353229 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f234c0a7480 con 0x7f234c0b2710 2022-08-01T19:34:14.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.042+0000 7f23515cc700 1 -- 172.21.15.17:0/2613353229 >> v1:172.21.15.77:6789/0 conn(0x7f234c0b01a0 legacy=0x7f234c0a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.042+0000 7f23515cc700 1 -- 172.21.15.17:0/2613353229 >> v1:172.21.15.17:6789/0 conn(0x7f234c0b2710 legacy=0x7f234c14afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.042+0000 7f23515cc700 1 -- 172.21.15.17:0/2613353229 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f234c0a79c0 con 0x7f234c0a8c50 2022-08-01T19:34:14.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.042+0000 7f23515cc700 1 --2- 172.21.15.17:0/2613353229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f234c0a8c50 0x7f234c152d70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f23440032f0 tx=0x7f23440091d0).ready entity=mon.1 client_cookie=cde12e243db72613 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:14.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.042+0000 7f234b7fe700 1 -- 172.21.15.17:0/2613353229 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f234400b600 con 0x7f234c0a8c50 2022-08-01T19:34:14.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.042+0000 7f234b7fe700 1 -- 172.21.15.17:0/2613353229 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f234400b760 con 0x7f234c0a8c50 2022-08-01T19:34:14.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.042+0000 7f234b7fe700 1 -- 172.21.15.17:0/2613353229 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f234400b8d0 con 0x7f234c0a8c50 2022-08-01T19:34:14.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.042+0000 7f2351dcd700 1 -- 172.21.15.17:0/2613353229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f234c0a8c50 msgr2=0x7f234c152d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.042+0000 7f2351dcd700 1 --2- 172.21.15.17:0/2613353229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f234c0a8c50 0x7f234c152d70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f23440032f0 tx=0x7f23440091d0).stop 2022-08-01T19:34:14.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.042+0000 7f2351dcd700 1 -- 172.21.15.17:0/2613353229 shutdown_connections 2022-08-01T19:34:14.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.042+0000 7f2351dcd700 1 --2- 172.21.15.17:0/2613353229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f234c0a8c50 0x7f234c152d70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:14.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.042+0000 7f2351dcd700 1 -- 172.21.15.17:0/2613353229 >> 172.21.15.17:0/2613353229 conn(0x7f234c060310 msgr2=0x7f234c0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:14.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.042+0000 7f2351dcd700 1 -- 172.21.15.17:0/2613353229 shutdown_connections 2022-08-01T19:34:14.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.042+0000 7f2351dcd700 1 -- 172.21.15.17:0/2613353229 wait complete. 2022-08-01T19:34:14.063 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.046+0000 7f2351dcd700 1 Processor -- start 2022-08-01T19:34:14.064 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.046+0000 7f2351dcd700 1 -- start start 2022-08-01T19:34:14.065 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.046+0000 7f2351dcd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f234c0a8c50 0x7f234c14ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.065 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.046+0000 7f2351dcd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f234c0b01a0 0x7f234c159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.065 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.046+0000 7f2351dcd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f234c0b2710 0x7f234c15a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.066 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.046+0000 7f2351dcd700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234c14a7a0 con 0x7f234c0a8c50 2022-08-01T19:34:14.066 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.046+0000 7f2351dcd700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234c0b5a90 con 0x7f234c0b2710 2022-08-01T19:34:14.066 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.046+0000 7f2351dcd700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f234c14a9d0 con 0x7f234c0b01a0 2022-08-01T19:34:14.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.046+0000 7f2350dcb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f234c0a8c50 0x7f234c14ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.046+0000 7f2350dcb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f234c0a8c50 0x7f234c14ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36390/0 (socket says 172.21.15.17:36390) 2022-08-01T19:34:14.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.046+0000 7f2350dcb700 1 -- 172.21.15.17:0/684485070 learned_addr learned my addr 172.21.15.17:0/684485070 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:14.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.046+0000 7f23515cc700 1 --2- 172.21.15.17:0/684485070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f234c0b2710 0x7f234c15a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.046+0000 7f234bfff700 1 --2- 172.21.15.17:0/684485070 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f234c0b01a0 0x7f234c159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.046+0000 7f2350dcb700 1 -- 172.21.15.17:0/684485070 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f234c0b01a0 msgr2=0x7f234c159f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.046+0000 7f2350dcb700 1 --2- 172.21.15.17:0/684485070 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f234c0b01a0 0x7f234c159f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:14.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.046+0000 7f2350dcb700 1 -- 172.21.15.17:0/684485070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f234c0b2710 msgr2=0x7f234c15a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.069 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.046+0000 7f2350dcb700 1 --2- 172.21.15.17:0/684485070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f234c0b2710 0x7f234c15a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:14.069 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.046+0000 7f2350dcb700 1 -- 172.21.15.17:0/684485070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2344009470 con 0x7f234c0a8c50 2022-08-01T19:34:14.069 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.046+0000 7f234bfff700 1 --2- 172.21.15.17:0/684485070 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f234c0b01a0 0x7f234c159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:34:14.069 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.046+0000 7f2350dcb700 1 --2- 172.21.15.17:0/684485070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f234c0a8c50 0x7f234c14ca40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f2340001110 tx=0x7f2340007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:14.069 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.046+0000 7f2349ffb700 1 -- 172.21.15.17:0/684485070 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23400070d0 con 0x7f234c0a8c50 2022-08-01T19:34:14.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.046+0000 7f2349ffb700 1 -- 172.21.15.17:0/684485070 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2340007230 con 0x7f234c0a8c50 2022-08-01T19:34:14.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.046+0000 7f2351dcd700 1 -- 172.21.15.17:0/684485070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f234c14ac70 con 0x7f234c0a8c50 2022-08-01T19:34:14.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.046+0000 7f2349ffb700 1 -- 172.21.15.17:0/684485070 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2340005610 con 0x7f234c0a8c50 2022-08-01T19:34:14.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.046+0000 7f2351dcd700 1 -- 172.21.15.17:0/684485070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f234c0aa480 con 0x7f234c0a8c50 2022-08-01T19:34:14.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.050+0000 7f2351dcd700 1 -- 172.21.15.17:0/684485070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f234c14b370 con 0x7f234c0a8c50 2022-08-01T19:34:14.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.050+0000 7f2349ffb700 1 -- 172.21.15.17:0/684485070 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2340010070 con 0x7f234c0a8c50 2022-08-01T19:34:14.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.050+0000 7f2349ffb700 1 --2- 172.21.15.17:0/684485070 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f232c053950 0x7f232c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.050+0000 7f2349ffb700 1 -- 172.21.15.17:0/684485070 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f234006b690 con 0x7f234c0a8c50 2022-08-01T19:34:14.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.050+0000 7f234bfff700 1 --2- 172.21.15.17:0/684485070 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f232c053950 0x7f232c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.050+0000 7f234bfff700 1 --2- 172.21.15.17:0/684485070 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f232c053950 0x7f232c055e10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f233c00d000 tx=0x7f233c007a00).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:14.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.054+0000 7f2349ffb700 1 -- 172.21.15.17:0/684485070 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f234c14b370 con 0x7f234c0a8c50 2022-08-01T19:34:14.188 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.182+0000 7f2351dcd700 1 -- 172.21.15.17:0/684485070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f234c14b370 con 0x7f234c0a8c50 2022-08-01T19:34:14.190 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.186+0000 7f2349ffb700 1 -- 172.21.15.17:0/684485070 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f234c14b370 con 0x7f234c0a8c50 2022-08-01T19:34:14.191 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:14.191 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:34:09.692404+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:34:04.635473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-08-01T19:34:07.409434+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:34:14.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.186+0000 7f2351dcd700 1 -- 172.21.15.17:0/684485070 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f232c053950 msgr2=0x7f232c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.186+0000 7f2351dcd700 1 --2- 172.21.15.17:0/684485070 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f232c053950 0x7f232c055e10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f233c00d000 tx=0x7f233c007a00).stop 2022-08-01T19:34:14.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.186+0000 7f2351dcd700 1 -- 172.21.15.17:0/684485070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f234c0a8c50 msgr2=0x7f234c14ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.186+0000 7f2351dcd700 1 --2- 172.21.15.17:0/684485070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f234c0a8c50 0x7f234c14ca40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f2340001110 tx=0x7f2340007ad0).stop 2022-08-01T19:34:14.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.186+0000 7f2351dcd700 1 -- 172.21.15.17:0/684485070 shutdown_connections 2022-08-01T19:34:14.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.186+0000 7f2351dcd700 1 --2- 172.21.15.17:0/684485070 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f232c053950 0x7f232c055e10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:14.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.186+0000 7f2351dcd700 1 --2- 172.21.15.17:0/684485070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f234c0b2710 0x7f234c15a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:14.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.186+0000 7f2351dcd700 1 --2- 172.21.15.17:0/684485070 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f234c0b01a0 0x7f234c159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:14.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.186+0000 7f2351dcd700 1 --2- 172.21.15.17:0/684485070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f234c0a8c50 0x7f234c14ca40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:14.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.186+0000 7f2351dcd700 1 -- 172.21.15.17:0/684485070 >> 172.21.15.17:0/684485070 conn(0x7f234c060310 msgr2=0x7f234c146130 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:14.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.186+0000 7f2351dcd700 1 -- 172.21.15.17:0/684485070 shutdown_connections 2022-08-01T19:34:14.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.186+0000 7f2351dcd700 1 -- 172.21.15.17:0/684485070 wait complete. 2022-08-01T19:34:14.204 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2022-08-01T19:34:14.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.306+0000 7f3dd9948700 1 Processor -- start 2022-08-01T19:34:14.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.306+0000 7f3dd9948700 1 -- start start 2022-08-01T19:34:14.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.306+0000 7f3dd9948700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3dd40b01a0 0x7f3dd40a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.306+0000 7f3dd9948700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dd40a7250 con 0x7f3dd40afd60 2022-08-01T19:34:14.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.306+0000 7f3dd9948700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dd40a7480 con 0x7f3dd40b2710 2022-08-01T19:34:14.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.306+0000 7f3dd9948700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dd40a7720 con 0x7f3dd40b01a0 2022-08-01T19:34:14.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.306+0000 7f3dd3fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3dd40b01a0 0x7f3dd40a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.306+0000 7f3dd3fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3dd40b01a0 0x7f3dd40a8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36398/0 (socket says 172.21.15.17:36398) 2022-08-01T19:34:14.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.306+0000 7f3dd3fff700 1 -- 172.21.15.17:0/3540278492 learned_addr learned my addr 172.21.15.17:0/3540278492 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:14.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.306+0000 7f3dd3fff700 1 -- 172.21.15.17:0/3540278492 >> v1:172.21.15.77:6789/0 conn(0x7f3dd40b2710 legacy=0x7f3dd414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.306+0000 7f3dd3fff700 1 -- 172.21.15.17:0/3540278492 >> v1:172.21.15.17:6789/0 conn(0x7f3dd40afd60 legacy=0x7f3dd4151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.306+0000 7f3dd3fff700 1 -- 172.21.15.17:0/3540278492 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd40a79c0 con 0x7f3dd40b01a0 2022-08-01T19:34:14.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.306+0000 7f3dd3fff700 1 --2- 172.21.15.17:0/3540278492 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dd40b01a0 0x7f3dd40a8540 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f3dc4002f70 tx=0x7f3dc4006eb0).ready entity=mon.0 client_cookie=ab9dba1b51d6dd6 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:14.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.306+0000 7f3dd37fe700 1 -- 172.21.15.17:0/3540278492 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dc400d660 con 0x7f3dd40b01a0 2022-08-01T19:34:14.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.306+0000 7f3dd37fe700 1 -- 172.21.15.17:0/3540278492 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc4000e90 con 0x7f3dd40b01a0 2022-08-01T19:34:14.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.306+0000 7f3dd37fe700 1 -- 172.21.15.17:0/3540278492 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dc400d9f0 con 0x7f3dd40b01a0 2022-08-01T19:34:14.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.310+0000 7f3dd9948700 1 -- 172.21.15.17:0/3540278492 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dd40b01a0 msgr2=0x7f3dd40a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.310+0000 7f3dd9948700 1 --2- 172.21.15.17:0/3540278492 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dd40b01a0 0x7f3dd40a8540 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f3dc4002f70 tx=0x7f3dc4006eb0).stop 2022-08-01T19:34:14.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.310+0000 7f3dd9948700 1 -- 172.21.15.17:0/3540278492 shutdown_connections 2022-08-01T19:34:14.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.310+0000 7f3dd9948700 1 --2- 172.21.15.17:0/3540278492 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dd40b01a0 0x7f3dd40a8540 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:14.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.310+0000 7f3dd9948700 1 -- 172.21.15.17:0/3540278492 >> 172.21.15.17:0/3540278492 conn(0x7f3dd4060310 msgr2=0x7f3dd40b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:14.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.310+0000 7f3dd9948700 1 -- 172.21.15.17:0/3540278492 shutdown_connections 2022-08-01T19:34:14.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.310+0000 7f3dd9948700 1 -- 172.21.15.17:0/3540278492 wait complete. 2022-08-01T19:34:14.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.310+0000 7f3dd9948700 1 Processor -- start 2022-08-01T19:34:14.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.310+0000 7f3dd9948700 1 -- start start 2022-08-01T19:34:14.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.310+0000 7f3dd9948700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dd40afd60 0x7f3dd4064ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.310+0000 7f3dd9948700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3dd40b01a0 0x7f3dd4063f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.310+0000 7f3dd9948700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3dd40b2710 0x7f3dd40657f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.310+0000 7f3dd9948700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd4064fa0 con 0x7f3dd40afd60 2022-08-01T19:34:14.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.310+0000 7f3dd9948700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd4156790 con 0x7f3dd40b01a0 2022-08-01T19:34:14.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.310+0000 7f3dd9948700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd4156a30 con 0x7f3dd40b2710 2022-08-01T19:34:14.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.310+0000 7f3dd8946700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dd40afd60 0x7f3dd4064ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.310+0000 7f3dd8946700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dd40afd60 0x7f3dd4064ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36404/0 (socket says 172.21.15.17:36404) 2022-08-01T19:34:14.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.310+0000 7f3dd8946700 1 -- 172.21.15.17:0/2783456252 learned_addr learned my addr 172.21.15.17:0/2783456252 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:14.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.310+0000 7f3dd9147700 1 --2- 172.21.15.17:0/2783456252 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3dd40b2710 0x7f3dd40657f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.310+0000 7f3dd3fff700 1 --2- 172.21.15.17:0/2783456252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3dd40b01a0 0x7f3dd4063f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.310+0000 7f3dd8946700 1 -- 172.21.15.17:0/2783456252 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3dd40b2710 msgr2=0x7f3dd40657f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.310+0000 7f3dd8946700 1 --2- 172.21.15.17:0/2783456252 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3dd40b2710 0x7f3dd40657f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:14.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.310+0000 7f3dd8946700 1 -- 172.21.15.17:0/2783456252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3dd40b01a0 msgr2=0x7f3dd4063f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.310+0000 7f3dd8946700 1 --2- 172.21.15.17:0/2783456252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3dd40b01a0 0x7f3dd4063f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:14.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.310+0000 7f3dd8946700 1 -- 172.21.15.17:0/2783456252 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc400b1e0 con 0x7f3dd40afd60 2022-08-01T19:34:14.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.310+0000 7f3dd3fff700 1 --2- 172.21.15.17:0/2783456252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3dd40b01a0 0x7f3dd4063f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:34:14.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.310+0000 7f3dd8946700 1 --2- 172.21.15.17:0/2783456252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dd40afd60 0x7f3dd4064ab0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f3dd4061c10 tx=0x7f3dc8007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:14.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.310+0000 7f3dd1ffb700 1 -- 172.21.15.17:0/2783456252 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dc8012400 con 0x7f3dd40afd60 2022-08-01T19:34:14.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.310+0000 7f3dd1ffb700 1 -- 172.21.15.17:0/2783456252 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc8012560 con 0x7f3dd40afd60 2022-08-01T19:34:14.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.310+0000 7f3dd9948700 1 -- 172.21.15.17:0/2783456252 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dd4156cd0 con 0x7f3dd40afd60 2022-08-01T19:34:14.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.314+0000 7f3dd1ffb700 1 -- 172.21.15.17:0/2783456252 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dc80128c0 con 0x7f3dd40afd60 2022-08-01T19:34:14.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.314+0000 7f3dd9948700 1 -- 172.21.15.17:0/2783456252 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dd40633d0 con 0x7f3dd40afd60 2022-08-01T19:34:14.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.314+0000 7f3dd9948700 1 -- 172.21.15.17:0/2783456252 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3dd4157170 con 0x7f3dd40afd60 2022-08-01T19:34:14.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.314+0000 7f3dd1ffb700 1 -- 172.21.15.17:0/2783456252 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3dc8019030 con 0x7f3dd40afd60 2022-08-01T19:34:14.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.314+0000 7f3dd1ffb700 1 --2- 172.21.15.17:0/2783456252 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3db4053b00 0x7f3db4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.314+0000 7f3dd1ffb700 1 -- 172.21.15.17:0/2783456252 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f3dc806b3b0 con 0x7f3dd40afd60 2022-08-01T19:34:14.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.314+0000 7f3dd3fff700 1 --2- 172.21.15.17:0/2783456252 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3db4053b00 0x7f3db4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.314+0000 7f3dd1ffb700 1 --2- 172.21.15.17:0/2783456252 >> [v2:172.21.15.17:6800/23008,v1:172.21.15.17:6801/23008] conn(0x7f3db405b480 0x7f3db405d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.314+0000 7f3dd1ffb700 1 -- 172.21.15.17:0/2783456252 --> [v2:172.21.15.17:6800/23008,v1:172.21.15.17:6801/23008] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3db405df00 con 0x7f3db405b480 2022-08-01T19:34:14.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.314+0000 7f3dd1ffb700 1 -- 172.21.15.17:0/2783456252 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3dc806da40 con 0x7f3dd40afd60 2022-08-01T19:34:14.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.314+0000 7f3dd9147700 1 --2- 172.21.15.17:0/2783456252 >> [v2:172.21.15.17:6800/23008,v1:172.21.15.17:6801/23008] conn(0x7f3db405b480 0x7f3db405d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.314+0000 7f3dd3fff700 1 --2- 172.21.15.17:0/2783456252 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3db4053b00 0x7f3db4055fc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f3dc401a000 tx=0x7f3dc4015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:14.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.314+0000 7f3dd9147700 1 --2- 172.21.15.17:0/2783456252 >> [v2:172.21.15.17:6800/23008,v1:172.21.15.17:6801/23008] conn(0x7f3db405b480 0x7f3db405d920 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:14.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.318+0000 7f3dd1ffb700 1 -- 172.21.15.17:0/2783456252 <== osd.0 v2:172.21.15.17:6800/23008 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f3db405df00 con 0x7f3db405b480 2022-08-01T19:34:14.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.338+0000 7f3dd9948700 1 -- 172.21.15.17:0/2783456252 --> [v2:172.21.15.17:6800/23008,v1:172.21.15.17:6801/23008] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3dd4157730 con 0x7f3db405b480 2022-08-01T19:34:14.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.338+0000 7f3dd1ffb700 1 -- 172.21.15.17:0/2783456252 <== osd.0 v2:172.21.15.17:6800/23008 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3dd4157730 con 0x7f3db405b480 2022-08-01T19:34:14.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.338+0000 7f3dd9948700 1 -- 172.21.15.17:0/2783456252 >> [v2:172.21.15.17:6800/23008,v1:172.21.15.17:6801/23008] conn(0x7f3db405b480 msgr2=0x7f3db405d920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.338+0000 7f3dd9948700 1 --2- 172.21.15.17:0/2783456252 >> [v2:172.21.15.17:6800/23008,v1:172.21.15.17:6801/23008] conn(0x7f3db405b480 0x7f3db405d920 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:14.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.338+0000 7f3dd9948700 1 -- 172.21.15.17:0/2783456252 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3db4053b00 msgr2=0x7f3db4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.338+0000 7f3dd9948700 1 --2- 172.21.15.17:0/2783456252 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3db4053b00 0x7f3db4055fc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f3dc401a000 tx=0x7f3dc4015000).stop 2022-08-01T19:34:14.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.338+0000 7f3dd9948700 1 -- 172.21.15.17:0/2783456252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dd40afd60 msgr2=0x7f3dd4064ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.338+0000 7f3dd9948700 1 --2- 172.21.15.17:0/2783456252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dd40afd60 0x7f3dd4064ab0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f3dd4061c10 tx=0x7f3dc8007c50).stop 2022-08-01T19:34:14.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.338+0000 7f3dd9147700 1 -- 172.21.15.17:0/2783456252 reap_dead start 2022-08-01T19:34:14.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.338+0000 7f3dd9948700 1 -- 172.21.15.17:0/2783456252 shutdown_connections 2022-08-01T19:34:14.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.338+0000 7f3dd9948700 1 -- 172.21.15.17:0/2783456252 >> 172.21.15.17:0/2783456252 conn(0x7f3dd4060310 msgr2=0x7f3dd40676e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:14.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.338+0000 7f3dd9948700 1 -- 172.21.15.17:0/2783456252 shutdown_connections 2022-08-01T19:34:14.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.338+0000 7f3dd9948700 1 -- 172.21.15.17:0/2783456252 wait complete. 2022-08-01T19:34:14.354 INFO:teuthology.orchestra.run.smithi017.stdout:60129542149 2022-08-01T19:34:14.354 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2022-08-01T19:34:14.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.450+0000 7f015c10d700 1 Processor -- start 2022-08-01T19:34:14.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.450+0000 7f015c10d700 1 -- start start 2022-08-01T19:34:14.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.450+0000 7f015c10d700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f01540b6710 0x7f01540b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.450+0000 7f015c10d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f01540b6af0 0x7f015412bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.450+0000 7f015c10d700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0154144240 con 0x7f01540b4550 2022-08-01T19:34:14.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.450+0000 7f015c10d700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0154143d00 con 0x7f01540b6af0 2022-08-01T19:34:14.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.450+0000 7f015c10d700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0154143fa0 con 0x7f01540b6710 2022-08-01T19:34:14.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.450+0000 7f01596a8700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f01540b4550 0x7f0154061ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56840/0 (socket says 172.21.15.17:56840) 2022-08-01T19:34:14.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.450+0000 7f01596a8700 1 -- 172.21.15.17:0/2108788975 learned_addr learned my addr 172.21.15.17:0/2108788975 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:14.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.450+0000 7f015a6aa700 1 --2- 172.21.15.17:0/2108788975 >> v2:172.21.15.17:3300/0 conn(0x7f01540b6af0 0x7f015412bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.450+0000 7f0159ea9700 1 --2- 172.21.15.17:0/2108788975 >> v2:172.21.15.77:3300/0 conn(0x7f01540b6710 0x7f01540b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.450+0000 7f0158ea7700 1 -- 172.21.15.17:0/2108788975 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2735552108 0 0) 0x7f0154144240 con 0x7f01540b4550 2022-08-01T19:34:14.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.450+0000 7f0158ea7700 1 -- 172.21.15.17:0/2108788975 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f013c003260 con 0x7f01540b4550 2022-08-01T19:34:14.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.450+0000 7f015a6aa700 1 -- 172.21.15.17:0/2108788975 >> v2:172.21.15.77:3300/0 conn(0x7f01540b6710 msgr2=0x7f01540b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:14.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.450+0000 7f015a6aa700 1 --2- 172.21.15.17:0/2108788975 >> v2:172.21.15.77:3300/0 conn(0x7f01540b6710 0x7f01540b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:14.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.450+0000 7f015a6aa700 1 -- 172.21.15.17:0/2108788975 >> v1:172.21.15.17:6789/0 conn(0x7f01540b4550 legacy=0x7f0154061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.450+0000 7f0158ea7700 1 -- 172.21.15.17:0/2108788975 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1647575667 0 0) 0x7f013c003260 con 0x7f01540b4550 2022-08-01T19:34:14.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.450+0000 7f015a6aa700 1 -- 172.21.15.17:0/2108788975 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0154061780 con 0x7f01540b6af0 2022-08-01T19:34:14.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.454+0000 7f015a6aa700 1 --2- 172.21.15.17:0/2108788975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01540b6af0 0x7f015412bb90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f01541432b0 tx=0x7f014c006df0).ready entity=mon.0 client_cookie=b7420e9bc694d65 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:14.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.454+0000 7f0158ea7700 1 -- 172.21.15.17:0/2108788975 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01500019e0 con 0x7f01540b6af0 2022-08-01T19:34:14.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.454+0000 7f0158ea7700 1 -- 172.21.15.17:0/2108788975 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f014c000e90 con 0x7f01540b6af0 2022-08-01T19:34:14.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.454+0000 7f0158ea7700 1 -- 172.21.15.17:0/2108788975 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f014c012460 con 0x7f01540b6af0 2022-08-01T19:34:14.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.454+0000 7f015c10d700 1 -- 172.21.15.17:0/2108788975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01540b6af0 msgr2=0x7f015412bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.454+0000 7f015c10d700 1 --2- 172.21.15.17:0/2108788975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01540b6af0 0x7f015412bb90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f01541432b0 tx=0x7f014c006df0).stop 2022-08-01T19:34:14.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.454+0000 7f015c10d700 1 -- 172.21.15.17:0/2108788975 shutdown_connections 2022-08-01T19:34:14.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.454+0000 7f015c10d700 1 --2- 172.21.15.17:0/2108788975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01540b6af0 0x7f015412bb90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:14.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.454+0000 7f015c10d700 1 --2- 172.21.15.17:0/2108788975 >> v2:172.21.15.77:3300/0 conn(0x7f01540b6710 0x7f01540b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:14.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.454+0000 7f015c10d700 1 -- 172.21.15.17:0/2108788975 >> 172.21.15.17:0/2108788975 conn(0x7f015405c620 msgr2=0x7f015405ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:14.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.454+0000 7f015c10d700 1 -- 172.21.15.17:0/2108788975 shutdown_connections 2022-08-01T19:34:14.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.454+0000 7f015c10d700 1 -- 172.21.15.17:0/2108788975 wait complete. 2022-08-01T19:34:14.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.454+0000 7f015c10d700 1 Processor -- start 2022-08-01T19:34:14.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.454+0000 7f015c10d700 1 -- start start 2022-08-01T19:34:14.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.458+0000 7f015c10d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01540b4550 0x7f0154127350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.458+0000 7f015c10d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01540b6710 0x7f0154155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.458+0000 7f015c10d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f01540b6af0 0x7f0154156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.458+0000 7f015c10d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0154156650 con 0x7f01540b6710 2022-08-01T19:34:14.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.458+0000 7f015c10d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01540adec0 con 0x7f01540b4550 2022-08-01T19:34:14.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.458+0000 7f015c10d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01540ae160 con 0x7f01540b6af0 2022-08-01T19:34:14.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.458+0000 7f01596a8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01540b6710 0x7f0154155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.458+0000 7f01596a8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01540b6710 0x7f0154155c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36422/0 (socket says 172.21.15.17:36422) 2022-08-01T19:34:14.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.458+0000 7f01596a8700 1 -- 172.21.15.17:0/3402795926 learned_addr learned my addr 172.21.15.17:0/3402795926 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:14.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.458+0000 7f0159ea9700 1 --2- 172.21.15.17:0/3402795926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01540b4550 0x7f0154127350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.458+0000 7f015a6aa700 1 --2- 172.21.15.17:0/3402795926 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f01540b6af0 0x7f0154156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.458+0000 7f01596a8700 1 -- 172.21.15.17:0/3402795926 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f01540b6af0 msgr2=0x7f0154156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.458+0000 7f01596a8700 1 --2- 172.21.15.17:0/3402795926 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f01540b6af0 0x7f0154156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:14.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.458+0000 7f01596a8700 1 -- 172.21.15.17:0/3402795926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01540b4550 msgr2=0x7f0154127350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.458+0000 7f01596a8700 1 --2- 172.21.15.17:0/3402795926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01540b4550 0x7f0154127350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:14.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.458+0000 7f01596a8700 1 -- 172.21.15.17:0/3402795926 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f014c00b180 con 0x7f01540b6710 2022-08-01T19:34:14.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.458+0000 7f015a6aa700 1 --2- 172.21.15.17:0/3402795926 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f01540b6af0 0x7f0154156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:34:14.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.458+0000 7f01596a8700 1 --2- 172.21.15.17:0/3402795926 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01540b6710 0x7f0154155c70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f0150001c40 tx=0x7f0150002180).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:14.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.458+0000 7f0142ffd700 1 -- 172.21.15.17:0/3402795926 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0150005ba0 con 0x7f01540b6710 2022-08-01T19:34:14.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.458+0000 7f0142ffd700 1 -- 172.21.15.17:0/3402795926 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0150005d00 con 0x7f01540b6710 2022-08-01T19:34:14.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.458+0000 7f015c10d700 1 -- 172.21.15.17:0/3402795926 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01541557d0 con 0x7f01540b6710 2022-08-01T19:34:14.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.458+0000 7f0142ffd700 1 -- 172.21.15.17:0/3402795926 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0150007300 con 0x7f01540b6710 2022-08-01T19:34:14.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.458+0000 7f015c10d700 1 -- 172.21.15.17:0/3402795926 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01540ae400 con 0x7f01540b6710 2022-08-01T19:34:14.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.458+0000 7f015c10d700 1 -- 172.21.15.17:0/3402795926 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0154155960 con 0x7f01540b6710 2022-08-01T19:34:14.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.458+0000 7f0142ffd700 1 -- 172.21.15.17:0/3402795926 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0150010030 con 0x7f01540b6710 2022-08-01T19:34:14.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.462+0000 7f0142ffd700 1 --2- 172.21.15.17:0/3402795926 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f01340539e0 0x7f0134055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.462+0000 7f0142ffd700 1 -- 172.21.15.17:0/3402795926 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f015006c100 con 0x7f01540b6710 2022-08-01T19:34:14.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.462+0000 7f0142ffd700 1 --2- 172.21.15.17:0/3402795926 >> [v2:172.21.15.17:6808/23007,v1:172.21.15.17:6809/23007] conn(0x7f013405b360 0x7f013405d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.462+0000 7f0142ffd700 1 -- 172.21.15.17:0/3402795926 --> [v2:172.21.15.17:6808/23007,v1:172.21.15.17:6809/23007] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f013405de80 con 0x7f013405b360 2022-08-01T19:34:14.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.462+0000 7f0159ea9700 1 --2- 172.21.15.17:0/3402795926 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f01340539e0 0x7f0134055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.462+0000 7f0142ffd700 1 -- 172.21.15.17:0/3402795926 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f015006e790 con 0x7f01540b6710 2022-08-01T19:34:14.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.462+0000 7f015a6aa700 1 --2- 172.21.15.17:0/3402795926 >> [v2:172.21.15.17:6808/23007,v1:172.21.15.17:6809/23007] conn(0x7f013405b360 0x7f013405d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.462+0000 7f0159ea9700 1 --2- 172.21.15.17:0/3402795926 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f01340539e0 0x7f0134055ea0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f014400b000 tx=0x7f0144006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:14.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.462+0000 7f015a6aa700 1 --2- 172.21.15.17:0/3402795926 >> [v2:172.21.15.17:6808/23007,v1:172.21.15.17:6809/23007] conn(0x7f013405b360 0x7f013405d800 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:14.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.462+0000 7f0142ffd700 1 -- 172.21.15.17:0/3402795926 <== osd.1 v2:172.21.15.17:6808/23007 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f013405de80 con 0x7f013405b360 2022-08-01T19:34:14.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.486+0000 7f015c10d700 1 -- 172.21.15.17:0/3402795926 --> [v2:172.21.15.17:6808/23007,v1:172.21.15.17:6809/23007] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f015412b230 con 0x7f013405b360 2022-08-01T19:34:14.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.486+0000 7f0142ffd700 1 -- 172.21.15.17:0/3402795926 <== osd.1 v2:172.21.15.17:6808/23007 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f015412b230 con 0x7f013405b360 2022-08-01T19:34:14.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.486+0000 7f015c10d700 1 -- 172.21.15.17:0/3402795926 >> [v2:172.21.15.17:6808/23007,v1:172.21.15.17:6809/23007] conn(0x7f013405b360 msgr2=0x7f013405d800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.486+0000 7f015c10d700 1 --2- 172.21.15.17:0/3402795926 >> [v2:172.21.15.17:6808/23007,v1:172.21.15.17:6809/23007] conn(0x7f013405b360 0x7f013405d800 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:14.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.486+0000 7f015c10d700 1 -- 172.21.15.17:0/3402795926 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f01340539e0 msgr2=0x7f0134055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.486+0000 7f015c10d700 1 --2- 172.21.15.17:0/3402795926 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f01340539e0 0x7f0134055ea0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f014400b000 tx=0x7f0144006df0).stop 2022-08-01T19:34:14.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.486+0000 7f015c10d700 1 -- 172.21.15.17:0/3402795926 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01540b6710 msgr2=0x7f0154155c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.486+0000 7f015c10d700 1 --2- 172.21.15.17:0/3402795926 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01540b6710 0x7f0154155c70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f0150001c40 tx=0x7f0150002180).stop 2022-08-01T19:34:14.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.486+0000 7f015a6aa700 1 -- 172.21.15.17:0/3402795926 reap_dead start 2022-08-01T19:34:14.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.486+0000 7f015c10d700 1 -- 172.21.15.17:0/3402795926 shutdown_connections 2022-08-01T19:34:14.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.486+0000 7f015c10d700 1 -- 172.21.15.17:0/3402795926 >> 172.21.15.17:0/3402795926 conn(0x7f015405c620 msgr2=0x7f01541432f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:14.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.486+0000 7f015c10d700 1 -- 172.21.15.17:0/3402795926 shutdown_connections 2022-08-01T19:34:14.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.486+0000 7f015c10d700 1 -- 172.21.15.17:0/3402795926 wait complete. 2022-08-01T19:34:14.497 INFO:teuthology.orchestra.run.smithi017.stdout:60129542150 2022-08-01T19:34:14.498 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2022-08-01T19:34:14.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.598+0000 7f98530a6700 1 Processor -- start 2022-08-01T19:34:14.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.602+0000 7f98530a6700 1 -- start start 2022-08-01T19:34:14.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.602+0000 7f98530a6700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f984c060aa0 0x7f984c0acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.602+0000 7f98530a6700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f984c0aa760 0x7f984c13c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.602+0000 7f98530a6700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f984c0b6e70 con 0x7f984c0b6a90 2022-08-01T19:34:14.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.602+0000 7f98530a6700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f984c061370 con 0x7f984c060aa0 2022-08-01T19:34:14.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.602+0000 7f98530a6700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f984c061610 con 0x7f984c0aa760 2022-08-01T19:34:14.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.602+0000 7f984bfff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f984c060aa0 0x7f984c0acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.602+0000 7f9850e42700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7f984c0b6a90 0x7f984c0aa050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:45424/0 (socket says 172.21.15.17:45424) 2022-08-01T19:34:14.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.602+0000 7f9851643700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f984c0aa760 0x7f984c13c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.602+0000 7f9850e42700 1 -- 172.21.15.17:0/2103534317 learned_addr learned my addr 172.21.15.17:0/2103534317 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:14.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.602+0000 7f984bfff700 1 -- 172.21.15.17:0/2103534317 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f984c0aa760 msgr2=0x7f984c13c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:14.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.602+0000 7f984bfff700 1 --2- 172.21.15.17:0/2103534317 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f984c0aa760 0x7f984c13c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:14.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.602+0000 7f984bfff700 1 -- 172.21.15.17:0/2103534317 >> v1:172.21.15.77:6789/0 conn(0x7f984c0b6a90 legacy=0x7f984c0aa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.602+0000 7f984b7fe700 1 -- 172.21.15.17:0/2103534317 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1344492369 0 0) 0x7f984c0b6e70 con 0x7f984c0b6a90 2022-08-01T19:34:14.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.602+0000 7f984bfff700 1 -- 172.21.15.17:0/2103534317 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f984c0618b0 con 0x7f984c060aa0 2022-08-01T19:34:14.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.602+0000 7f984bfff700 1 --2- 172.21.15.17:0/2103534317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f984c060aa0 0x7f984c0acee0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f9838002f70 tx=0x7f9838006d80).ready entity=mon.1 client_cookie=818f4601c43f515f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:14.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.602+0000 7f984b7fe700 1 -- 172.21.15.17:0/2103534317 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f983800bac0 con 0x7f984c060aa0 2022-08-01T19:34:14.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.606+0000 7f984b7fe700 1 -- 172.21.15.17:0/2103534317 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f983800bc20 con 0x7f984c060aa0 2022-08-01T19:34:14.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.606+0000 7f984b7fe700 1 -- 172.21.15.17:0/2103534317 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98380153f0 con 0x7f984c060aa0 2022-08-01T19:34:14.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.606+0000 7f98530a6700 1 -- 172.21.15.17:0/2103534317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f984c060aa0 msgr2=0x7f984c0acee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.606+0000 7f98530a6700 1 --2- 172.21.15.17:0/2103534317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f984c060aa0 0x7f984c0acee0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f9838002f70 tx=0x7f9838006d80).stop 2022-08-01T19:34:14.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.606+0000 7f98530a6700 1 -- 172.21.15.17:0/2103534317 shutdown_connections 2022-08-01T19:34:14.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.606+0000 7f98530a6700 1 --2- 172.21.15.17:0/2103534317 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f984c0aa760 0x7f984c13c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:14.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.606+0000 7f98530a6700 1 --2- 172.21.15.17:0/2103534317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f984c060aa0 0x7f984c0acee0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:14.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.606+0000 7f98530a6700 1 -- 172.21.15.17:0/2103534317 >> 172.21.15.17:0/2103534317 conn(0x7f984c0b3610 msgr2=0x7f984c0b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:14.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.606+0000 7f98530a6700 1 -- 172.21.15.17:0/2103534317 shutdown_connections 2022-08-01T19:34:14.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.606+0000 7f98530a6700 1 -- 172.21.15.17:0/2103534317 wait complete. 2022-08-01T19:34:14.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.606+0000 7f98530a6700 1 Processor -- start 2022-08-01T19:34:14.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.606+0000 7f98530a6700 1 -- start start 2022-08-01T19:34:14.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.610+0000 7f98530a6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f984c060aa0 0x7f984c136590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.610+0000 7f98530a6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f984c0aa760 0x7f984c13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.610+0000 7f98530a6700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f984c0b6a90 0x7f984c151910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.610+0000 7f98530a6700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984c13e270 con 0x7f984c060aa0 2022-08-01T19:34:14.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.610+0000 7f98530a6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984c0afcd0 con 0x7f984c0aa760 2022-08-01T19:34:14.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.610+0000 7f98530a6700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f984c13f730 con 0x7f984c0b6a90 2022-08-01T19:34:14.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.610+0000 7f9850e42700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f984c060aa0 0x7f984c136590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.610+0000 7f9850e42700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f984c060aa0 0x7f984c136590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36436/0 (socket says 172.21.15.17:36436) 2022-08-01T19:34:14.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.610+0000 7f9850e42700 1 -- 172.21.15.17:0/3030191580 learned_addr learned my addr 172.21.15.17:0/3030191580 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:14.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.610+0000 7f984bfff700 1 --2- 172.21.15.17:0/3030191580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f984c0aa760 0x7f984c13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.610+0000 7f9851643700 1 --2- 172.21.15.17:0/3030191580 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f984c0b6a90 0x7f984c151910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.610+0000 7f9850e42700 1 -- 172.21.15.17:0/3030191580 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f984c0b6a90 msgr2=0x7f984c151910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.610+0000 7f9850e42700 1 --2- 172.21.15.17:0/3030191580 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f984c0b6a90 0x7f984c151910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:14.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.610+0000 7f9850e42700 1 -- 172.21.15.17:0/3030191580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f984c0aa760 msgr2=0x7f984c13fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.610+0000 7f9850e42700 1 --2- 172.21.15.17:0/3030191580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f984c0aa760 0x7f984c13fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:14.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.610+0000 7f9850e42700 1 -- 172.21.15.17:0/3030191580 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f983800d410 con 0x7f984c060aa0 2022-08-01T19:34:14.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.610+0000 7f984bfff700 1 --2- 172.21.15.17:0/3030191580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f984c0aa760 0x7f984c13fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:34:14.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.610+0000 7f9850e42700 1 --2- 172.21.15.17:0/3030191580 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f984c060aa0 0x7f984c136590 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f9840000bc0 tx=0x7f9840007c60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:14.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.610+0000 7f9849ffb700 1 -- 172.21.15.17:0/3030191580 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98400070d0 con 0x7f984c060aa0 2022-08-01T19:34:14.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.610+0000 7f9849ffb700 1 -- 172.21.15.17:0/3030191580 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9840007230 con 0x7f984c060aa0 2022-08-01T19:34:14.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.610+0000 7f98530a6700 1 -- 172.21.15.17:0/3030191580 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f984c13f9d0 con 0x7f984c060aa0 2022-08-01T19:34:14.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.610+0000 7f9849ffb700 1 -- 172.21.15.17:0/3030191580 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98400055f0 con 0x7f984c060aa0 2022-08-01T19:34:14.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.610+0000 7f98530a6700 1 -- 172.21.15.17:0/3030191580 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f984c0ae6e0 con 0x7f984c060aa0 2022-08-01T19:34:14.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.610+0000 7f98530a6700 1 -- 172.21.15.17:0/3030191580 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f984c13f5b0 con 0x7f984c060aa0 2022-08-01T19:34:14.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.614+0000 7f9849ffb700 1 -- 172.21.15.17:0/3030191580 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f984001a060 con 0x7f984c060aa0 2022-08-01T19:34:14.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.614+0000 7f9849ffb700 1 --2- 172.21.15.17:0/3030191580 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f982c053950 0x7f982c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.614+0000 7f9849ffb700 1 -- 172.21.15.17:0/3030191580 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f984006b5c0 con 0x7f984c060aa0 2022-08-01T19:34:14.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.614+0000 7f9849ffb700 1 --2- 172.21.15.17:0/3030191580 >> [v2:172.21.15.17:6817/23010,v1:172.21.15.17:6819/23010] conn(0x7f982c05b370 0x7f982c05d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.614+0000 7f9849ffb700 1 -- 172.21.15.17:0/3030191580 --> [v2:172.21.15.17:6817/23010,v1:172.21.15.17:6819/23010] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f982c05de90 con 0x7f982c05b370 2022-08-01T19:34:14.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.614+0000 7f984bfff700 1 --2- 172.21.15.17:0/3030191580 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f982c053950 0x7f982c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.614+0000 7f9849ffb700 1 -- 172.21.15.17:0/3030191580 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f984006dc50 con 0x7f984c060aa0 2022-08-01T19:34:14.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.614+0000 7f9851643700 1 --2- 172.21.15.17:0/3030191580 >> [v2:172.21.15.17:6817/23010,v1:172.21.15.17:6819/23010] conn(0x7f982c05b370 0x7f982c05d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.614+0000 7f984bfff700 1 --2- 172.21.15.17:0/3030191580 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f982c053950 0x7f982c055e10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f9838016000 tx=0x7f983801a000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:14.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.614+0000 7f9851643700 1 --2- 172.21.15.17:0/3030191580 >> [v2:172.21.15.17:6817/23010,v1:172.21.15.17:6819/23010] conn(0x7f982c05b370 0x7f982c05d810 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:14.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.614+0000 7f9849ffb700 1 -- 172.21.15.17:0/3030191580 <== osd.2 v2:172.21.15.17:6817/23010 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f982c05de90 con 0x7f982c05b370 2022-08-01T19:34:14.639 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.638+0000 7f98530a6700 1 -- 172.21.15.17:0/3030191580 --> [v2:172.21.15.17:6817/23010,v1:172.21.15.17:6819/23010] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f984c0b0370 con 0x7f982c05b370 2022-08-01T19:34:14.639 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.638+0000 7f9849ffb700 1 -- 172.21.15.17:0/3030191580 <== osd.2 v2:172.21.15.17:6817/23010 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9844009da0 con 0x7f982c05b370 2022-08-01T19:34:14.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.638+0000 7f98530a6700 1 -- 172.21.15.17:0/3030191580 >> [v2:172.21.15.17:6817/23010,v1:172.21.15.17:6819/23010] conn(0x7f982c05b370 msgr2=0x7f982c05d810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.638+0000 7f98530a6700 1 --2- 172.21.15.17:0/3030191580 >> [v2:172.21.15.17:6817/23010,v1:172.21.15.17:6819/23010] conn(0x7f982c05b370 0x7f982c05d810 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:14.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.638+0000 7f98530a6700 1 -- 172.21.15.17:0/3030191580 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f982c053950 msgr2=0x7f982c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.638+0000 7f98530a6700 1 --2- 172.21.15.17:0/3030191580 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f982c053950 0x7f982c055e10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f9838016000 tx=0x7f983801a000).stop 2022-08-01T19:34:14.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.638+0000 7f98530a6700 1 -- 172.21.15.17:0/3030191580 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f984c060aa0 msgr2=0x7f984c136590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.638+0000 7f98530a6700 1 --2- 172.21.15.17:0/3030191580 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f984c060aa0 0x7f984c136590 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f9840000bc0 tx=0x7f9840007c60).stop 2022-08-01T19:34:14.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.638+0000 7f9851643700 1 -- 172.21.15.17:0/3030191580 reap_dead start 2022-08-01T19:34:14.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.638+0000 7f98530a6700 1 -- 172.21.15.17:0/3030191580 shutdown_connections 2022-08-01T19:34:14.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.638+0000 7f98530a6700 1 -- 172.21.15.17:0/3030191580 >> 172.21.15.17:0/3030191580 conn(0x7f984c0b3610 msgr2=0x7f984c0b6020 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:14.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.638+0000 7f98530a6700 1 -- 172.21.15.17:0/3030191580 shutdown_connections 2022-08-01T19:34:14.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.638+0000 7f98530a6700 1 -- 172.21.15.17:0/3030191580 wait complete. 2022-08-01T19:34:14.652 INFO:teuthology.orchestra.run.smithi017.stdout:60129542151 2022-08-01T19:34:14.652 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2022-08-01T19:34:14.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.758+0000 7ff532099700 1 Processor -- start 2022-08-01T19:34:14.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.758+0000 7ff532099700 1 -- start start 2022-08-01T19:34:14.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.758+0000 7ff532099700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ff52c0ad750 0x7ff52c061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.758+0000 7ff532099700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff52c1405d0 0x7ff52c140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.758+0000 7ff532099700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff52c0aee80 con 0x7ff52c0614b0 2022-08-01T19:34:14.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.758+0000 7ff532099700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff52c0b70b0 con 0x7ff52c1405d0 2022-08-01T19:34:14.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.758+0000 7ff532099700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff52c0af0b0 con 0x7ff52c0ad750 2022-08-01T19:34:14.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.758+0000 7ff531097700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff52c0614b0 0x7ff52c0b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56872/0 (socket says 172.21.15.17:56872) 2022-08-01T19:34:14.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.758+0000 7ff531898700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff52c1405d0 0x7ff52c140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.758+0000 7ff531097700 1 -- 172.21.15.17:0/772160582 learned_addr learned my addr 172.21.15.17:0/772160582 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:14.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.758+0000 7ff530896700 1 --2- 172.21.15.17:0/772160582 >> v2:172.21.15.77:3300/0 conn(0x7ff52c0ad750 0x7ff52c061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.758+0000 7ff531898700 1 -- 172.21.15.17:0/772160582 >> v2:172.21.15.77:3300/0 conn(0x7ff52c0ad750 msgr2=0x7ff52c061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:14.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.758+0000 7ff531898700 1 --2- 172.21.15.17:0/772160582 >> v2:172.21.15.77:3300/0 conn(0x7ff52c0ad750 0x7ff52c061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:14.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.758+0000 7ff531898700 1 -- 172.21.15.17:0/772160582 >> v1:172.21.15.17:6789/0 conn(0x7ff52c0614b0 legacy=0x7ff52c0b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.758+0000 7ff531898700 1 -- 172.21.15.17:0/772160582 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff52c0af350 con 0x7ff52c1405d0 2022-08-01T19:34:14.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.758+0000 7ff530896700 1 --2- 172.21.15.17:0/772160582 >> v2:172.21.15.77:3300/0 conn(0x7ff52c0ad750 0x7ff52c061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:34:14.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.758+0000 7ff531898700 1 --2- 172.21.15.17:0/772160582 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff52c1405d0 0x7ff52c140a80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7ff5240032f0 tx=0x7ff5240091d0).ready entity=mon.0 client_cookie=c4e79b136f50e70 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:14.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.762+0000 7ff51bfff700 1 -- 172.21.15.17:0/772160582 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff52400b680 con 0x7ff52c1405d0 2022-08-01T19:34:14.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.762+0000 7ff51bfff700 1 -- 172.21.15.17:0/772160582 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52400b7e0 con 0x7ff52c1405d0 2022-08-01T19:34:14.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.762+0000 7ff51bfff700 1 -- 172.21.15.17:0/772160582 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff52400b950 con 0x7ff52c1405d0 2022-08-01T19:34:14.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.762+0000 7ff532099700 1 -- 172.21.15.17:0/772160582 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff52c1405d0 msgr2=0x7ff52c140a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.762+0000 7ff532099700 1 --2- 172.21.15.17:0/772160582 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff52c1405d0 0x7ff52c140a80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7ff5240032f0 tx=0x7ff5240091d0).stop 2022-08-01T19:34:14.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.762+0000 7ff532099700 1 -- 172.21.15.17:0/772160582 shutdown_connections 2022-08-01T19:34:14.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.762+0000 7ff532099700 1 --2- 172.21.15.17:0/772160582 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff52c1405d0 0x7ff52c140a80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:14.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.762+0000 7ff532099700 1 --2- 172.21.15.17:0/772160582 >> v2:172.21.15.77:3300/0 conn(0x7ff52c0ad750 0x7ff52c061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:14.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.762+0000 7ff532099700 1 -- 172.21.15.17:0/772160582 >> 172.21.15.17:0/772160582 conn(0x7ff52c066f70 msgr2=0x7ff52c067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:14.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.762+0000 7ff532099700 1 -- 172.21.15.17:0/772160582 shutdown_connections 2022-08-01T19:34:14.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.762+0000 7ff532099700 1 -- 172.21.15.17:0/772160582 wait complete. 2022-08-01T19:34:14.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.762+0000 7ff532099700 1 Processor -- start 2022-08-01T19:34:14.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.762+0000 7ff532099700 1 -- start start 2022-08-01T19:34:14.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.762+0000 7ff532099700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff52c0614b0 0x7ff52c1525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.762+0000 7ff532099700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff52c0ad750 0x7ff52c1563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.762+0000 7ff532099700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff52c1405d0 0x7ff52c14a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.762+0000 7ff532099700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c0ad430 con 0x7ff52c0614b0 2022-08-01T19:34:14.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.762+0000 7ff532099700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c143bb0 con 0x7ff52c0ad750 2022-08-01T19:34:14.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.762+0000 7ff532099700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c155c90 con 0x7ff52c1405d0 2022-08-01T19:34:14.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.762+0000 7ff531097700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff52c0614b0 0x7ff52c1525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.762+0000 7ff531097700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff52c0614b0 0x7ff52c1525c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36452/0 (socket says 172.21.15.17:36452) 2022-08-01T19:34:14.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.762+0000 7ff531097700 1 -- 172.21.15.17:0/1241128559 learned_addr learned my addr 172.21.15.17:0/1241128559 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:14.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.762+0000 7ff530896700 1 --2- 172.21.15.17:0/1241128559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff52c0ad750 0x7ff52c1563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.762+0000 7ff531898700 1 --2- 172.21.15.17:0/1241128559 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff52c1405d0 0x7ff52c14a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.762+0000 7ff531097700 1 -- 172.21.15.17:0/1241128559 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff52c1405d0 msgr2=0x7ff52c14a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.762+0000 7ff531097700 1 --2- 172.21.15.17:0/1241128559 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff52c1405d0 0x7ff52c14a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:14.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.762+0000 7ff531097700 1 -- 172.21.15.17:0/1241128559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff52c0ad750 msgr2=0x7ff52c1563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.762+0000 7ff531097700 1 --2- 172.21.15.17:0/1241128559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff52c0ad750 0x7ff52c1563d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:14.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.762+0000 7ff531097700 1 -- 172.21.15.17:0/1241128559 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51c00b180 con 0x7ff52c0614b0 2022-08-01T19:34:14.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.762+0000 7ff531097700 1 --2- 172.21.15.17:0/1241128559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff52c0614b0 0x7ff52c1525c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7ff52c0ae2b0 tx=0x7ff528007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:14.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.766+0000 7ff51a7fc700 1 -- 172.21.15.17:0/1241128559 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5280070d0 con 0x7ff52c0614b0 2022-08-01T19:34:14.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.766+0000 7ff51a7fc700 1 -- 172.21.15.17:0/1241128559 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff528007230 con 0x7ff52c0614b0 2022-08-01T19:34:14.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.766+0000 7ff51a7fc700 1 -- 172.21.15.17:0/1241128559 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff528005610 con 0x7ff52c0614b0 2022-08-01T19:34:14.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.766+0000 7ff532099700 1 -- 172.21.15.17:0/1241128559 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff524009470 con 0x7ff52c0614b0 2022-08-01T19:34:14.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.766+0000 7ff532099700 1 -- 172.21.15.17:0/1241128559 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff52c152ab0 con 0x7ff52c0614b0 2022-08-01T19:34:14.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.766+0000 7ff532099700 1 -- 172.21.15.17:0/1241128559 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff52c156130 con 0x7ff52c0614b0 2022-08-01T19:34:14.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.766+0000 7ff51a7fc700 1 -- 172.21.15.17:0/1241128559 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff528010070 con 0x7ff52c0614b0 2022-08-01T19:34:14.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.766+0000 7ff51a7fc700 1 --2- 172.21.15.17:0/1241128559 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff50c053980 0x7ff50c055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.766+0000 7ff51a7fc700 1 -- 172.21.15.17:0/1241128559 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7ff52806b470 con 0x7ff52c0614b0 2022-08-01T19:34:14.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.766+0000 7ff51a7fc700 1 --2- 172.21.15.17:0/1241128559 >> [v2:172.21.15.17:6816/23009,v1:172.21.15.17:6818/23009] conn(0x7ff50c05b320 0x7ff50c05d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.766+0000 7ff51a7fc700 1 -- 172.21.15.17:0/1241128559 --> [v2:172.21.15.17:6816/23009,v1:172.21.15.17:6818/23009] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff50c05de40 con 0x7ff50c05b320 2022-08-01T19:34:14.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.766+0000 7ff51a7fc700 1 -- 172.21.15.17:0/1241128559 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff528042230 con 0x7ff52c0614b0 2022-08-01T19:34:14.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.766+0000 7ff530896700 1 --2- 172.21.15.17:0/1241128559 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff50c053980 0x7ff50c055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.766+0000 7ff531898700 1 --2- 172.21.15.17:0/1241128559 >> [v2:172.21.15.17:6816/23009,v1:172.21.15.17:6818/23009] conn(0x7ff50c05b320 0x7ff50c05d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.766+0000 7ff530896700 1 --2- 172.21.15.17:0/1241128559 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff50c053980 0x7ff50c055e40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7ff51c00b000 tx=0x7ff51c006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:14.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.766+0000 7ff531898700 1 --2- 172.21.15.17:0/1241128559 >> [v2:172.21.15.17:6816/23009,v1:172.21.15.17:6818/23009] conn(0x7ff50c05b320 0x7ff50c05d7c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:14.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.770+0000 7ff51a7fc700 1 -- 172.21.15.17:0/1241128559 <== osd.3 v2:172.21.15.17:6816/23009 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7ff50c05de40 con 0x7ff50c05b320 2022-08-01T19:34:14.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.790+0000 7ff532099700 1 -- 172.21.15.17:0/1241128559 --> [v2:172.21.15.17:6816/23009,v1:172.21.15.17:6818/23009] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff52c0af5f0 con 0x7ff50c05b320 2022-08-01T19:34:14.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.790+0000 7ff51a7fc700 1 -- 172.21.15.17:0/1241128559 <== osd.3 v2:172.21.15.17:6816/23009 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff52c0af5f0 con 0x7ff50c05b320 2022-08-01T19:34:14.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.790+0000 7ff532099700 1 -- 172.21.15.17:0/1241128559 >> [v2:172.21.15.17:6816/23009,v1:172.21.15.17:6818/23009] conn(0x7ff50c05b320 msgr2=0x7ff50c05d7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.790+0000 7ff532099700 1 --2- 172.21.15.17:0/1241128559 >> [v2:172.21.15.17:6816/23009,v1:172.21.15.17:6818/23009] conn(0x7ff50c05b320 0x7ff50c05d7c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:14.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.790+0000 7ff532099700 1 -- 172.21.15.17:0/1241128559 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff50c053980 msgr2=0x7ff50c055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.790+0000 7ff532099700 1 --2- 172.21.15.17:0/1241128559 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff50c053980 0x7ff50c055e40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7ff51c00b000 tx=0x7ff51c006df0).stop 2022-08-01T19:34:14.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.790+0000 7ff532099700 1 -- 172.21.15.17:0/1241128559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff52c0614b0 msgr2=0x7ff52c1525c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.790+0000 7ff532099700 1 --2- 172.21.15.17:0/1241128559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff52c0614b0 0x7ff52c1525c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7ff52c0ae2b0 tx=0x7ff528007c00).stop 2022-08-01T19:34:14.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.790+0000 7ff531898700 1 -- 172.21.15.17:0/1241128559 reap_dead start 2022-08-01T19:34:14.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.790+0000 7ff532099700 1 -- 172.21.15.17:0/1241128559 shutdown_connections 2022-08-01T19:34:14.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.790+0000 7ff532099700 1 -- 172.21.15.17:0/1241128559 >> 172.21.15.17:0/1241128559 conn(0x7ff52c066f70 msgr2=0x7ff52c05cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:14.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.790+0000 7ff532099700 1 -- 172.21.15.17:0/1241128559 shutdown_connections 2022-08-01T19:34:14.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.790+0000 7ff532099700 1 -- 172.21.15.17:0/1241128559 wait complete. 2022-08-01T19:34:14.806 INFO:teuthology.orchestra.run.smithi017.stdout:60129542149 2022-08-01T19:34:14.806 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2022-08-01T19:34:14.917 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.910+0000 7ff014e15700 1 Processor -- start 2022-08-01T19:34:14.918 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.910+0000 7ff014e15700 1 -- start start 2022-08-01T19:34:14.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.914+0000 7ff014e15700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff01005d240 0x7ff0100b1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.914+0000 7ff014e15700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff01013f520 0x7ff0100aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.914+0000 7ff014e15700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff01005cd40 con 0x7ff01005c960 2022-08-01T19:34:14.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.914+0000 7ff014e15700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0100ad6d0 con 0x7ff01005d240 2022-08-01T19:34:14.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.914+0000 7ff014e15700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff010067f00 con 0x7ff01013f520 2022-08-01T19:34:14.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.914+0000 7ff00effd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff01005d240 0x7ff0100b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.914+0000 7ff00f7fe700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7ff01005c960 0x7ff0100acfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:45456/0 (socket says 172.21.15.17:45456) 2022-08-01T19:34:14.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.914+0000 7ff00effd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff01005d240 0x7ff0100b1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36464/0 (socket says 172.21.15.17:36464) 2022-08-01T19:34:14.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.914+0000 7ff00f7fe700 1 -- 172.21.15.17:0/665722789 learned_addr learned my addr 172.21.15.17:0/665722789 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:14.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.914+0000 7ff00ffff700 1 --2- 172.21.15.17:0/665722789 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff01013f520 0x7ff0100aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.914+0000 7ff00e7fc700 1 -- 172.21.15.17:0/665722789 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3963040530 0 0) 0x7ff01005cd40 con 0x7ff01005c960 2022-08-01T19:34:14.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.914+0000 7ff00effd700 1 -- 172.21.15.17:0/665722789 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff01013f520 msgr2=0x7ff0100aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:14.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.914+0000 7ff00effd700 1 --2- 172.21.15.17:0/665722789 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff01013f520 0x7ff0100aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:14.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.914+0000 7ff00effd700 1 -- 172.21.15.17:0/665722789 >> v1:172.21.15.77:6789/0 conn(0x7ff01005c960 legacy=0x7ff0100acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.914+0000 7ff00effd700 1 -- 172.21.15.17:0/665722789 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0100b1920 con 0x7ff01005d240 2022-08-01T19:34:14.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.914+0000 7ff00effd700 1 --2- 172.21.15.17:0/665722789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff01005d240 0x7ff0100b1280 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7ff004002f70 tx=0x7ff004006df0).ready entity=mon.0 client_cookie=79afadda92c0d5b9 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:14.927 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.914+0000 7ff00e7fc700 1 -- 172.21.15.17:0/665722789 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff00400d570 con 0x7ff01005d240 2022-08-01T19:34:14.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.914+0000 7ff00e7fc700 1 -- 172.21.15.17:0/665722789 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff004000e90 con 0x7ff01005d240 2022-08-01T19:34:14.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.914+0000 7ff00e7fc700 1 -- 172.21.15.17:0/665722789 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff00400d8c0 con 0x7ff01005d240 2022-08-01T19:34:14.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.914+0000 7ff014e15700 1 -- 172.21.15.17:0/665722789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff01005d240 msgr2=0x7ff0100b1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.914+0000 7ff014e15700 1 --2- 172.21.15.17:0/665722789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff01005d240 0x7ff0100b1280 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7ff004002f70 tx=0x7ff004006df0).stop 2022-08-01T19:34:14.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.914+0000 7ff014e15700 1 -- 172.21.15.17:0/665722789 shutdown_connections 2022-08-01T19:34:14.929 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.914+0000 7ff014e15700 1 --2- 172.21.15.17:0/665722789 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff01013f520 0x7ff0100aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:14.929 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.914+0000 7ff014e15700 1 --2- 172.21.15.17:0/665722789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff01005d240 0x7ff0100b1280 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:14.929 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.914+0000 7ff014e15700 1 -- 172.21.15.17:0/665722789 >> 172.21.15.17:0/665722789 conn(0x7ff010060310 msgr2=0x7ff0100b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:14.929 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.914+0000 7ff014e15700 1 -- 172.21.15.17:0/665722789 shutdown_connections 2022-08-01T19:34:14.930 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.914+0000 7ff014e15700 1 -- 172.21.15.17:0/665722789 wait complete. 2022-08-01T19:34:14.930 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.918+0000 7ff014e15700 1 Processor -- start 2022-08-01T19:34:14.930 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.918+0000 7ff014e15700 1 -- start start 2022-08-01T19:34:14.930 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.918+0000 7ff014e15700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff01005c960 0x7ff0100b0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.930 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.918+0000 7ff014e15700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff01005d240 0x7ff0100ab080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.931 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.918+0000 7ff014e15700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff01013f520 0x7ff0100ab570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.931 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.918+0000 7ff014e15700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0101310e0 con 0x7ff01013f520 2022-08-01T19:34:14.931 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.918+0000 7ff014e15700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0100aac20 con 0x7ff01005d240 2022-08-01T19:34:14.931 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.918+0000 7ff014e15700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0100aba60 con 0x7ff01005c960 2022-08-01T19:34:14.932 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.918+0000 7ff00ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff01013f520 0x7ff0100ab570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.932 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.918+0000 7ff00ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff01013f520 0x7ff0100ab570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36472/0 (socket says 172.21.15.17:36472) 2022-08-01T19:34:14.932 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.918+0000 7ff00ffff700 1 -- 172.21.15.17:0/4217036149 learned_addr learned my addr 172.21.15.17:0/4217036149 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:14.932 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.918+0000 7ff00effd700 1 --2- 172.21.15.17:0/4217036149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff01005d240 0x7ff0100ab080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.932 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.918+0000 7ff00f7fe700 1 --2- 172.21.15.17:0/4217036149 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff01005c960 0x7ff0100b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.933 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.918+0000 7ff00f7fe700 1 -- 172.21.15.17:0/4217036149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff01005d240 msgr2=0x7ff0100ab080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.933 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.918+0000 7ff00f7fe700 1 --2- 172.21.15.17:0/4217036149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff01005d240 0x7ff0100ab080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:14.933 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.918+0000 7ff00f7fe700 1 -- 172.21.15.17:0/4217036149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff01013f520 msgr2=0x7ff0100ab570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.933 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.918+0000 7ff00f7fe700 1 --2- 172.21.15.17:0/4217036149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff01013f520 0x7ff0100ab570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:14.934 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.918+0000 7ff00f7fe700 1 -- 172.21.15.17:0/4217036149 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff00400b180 con 0x7ff01005c960 2022-08-01T19:34:14.934 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.918+0000 7ff00f7fe700 1 --2- 172.21.15.17:0/4217036149 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff01005c960 0x7ff0100b0140 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7ff000000ed0 tx=0x7ff000007a20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:14.934 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.918+0000 7ff00cff9700 1 -- 172.21.15.17:0/4217036149 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff00000ce80 con 0x7ff01005c960 2022-08-01T19:34:14.934 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.918+0000 7ff00cff9700 1 -- 172.21.15.17:0/4217036149 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0000070d0 con 0x7ff01005c960 2022-08-01T19:34:14.934 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.918+0000 7ff014e15700 1 -- 172.21.15.17:0/4217036149 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff01013cf80 con 0x7ff01005c960 2022-08-01T19:34:14.935 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.918+0000 7ff00cff9700 1 -- 172.21.15.17:0/4217036149 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff000007430 con 0x7ff01005c960 2022-08-01T19:34:14.935 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.918+0000 7ff014e15700 1 -- 172.21.15.17:0/4217036149 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0100ac0d0 con 0x7ff01005c960 2022-08-01T19:34:14.935 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.918+0000 7ff014e15700 1 -- 172.21.15.17:0/4217036149 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff0100b1bc0 con 0x7ff01005c960 2022-08-01T19:34:14.935 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.922+0000 7ff00cff9700 1 -- 172.21.15.17:0/4217036149 <== mon.2 v2:172.21.15.77:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff000010070 con 0x7ff01005c960 2022-08-01T19:34:14.936 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.922+0000 7ff00cff9700 1 --2- 172.21.15.17:0/4217036149 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7feff0053a10 0x7feff0055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.936 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.922+0000 7ff00cff9700 1 -- 172.21.15.17:0/4217036149 <== mon.2 v2:172.21.15.77:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7ff00006bed0 con 0x7ff01005c960 2022-08-01T19:34:14.936 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.922+0000 7ff00cff9700 1 --2- 172.21.15.17:0/4217036149 >> [v2:172.21.15.77:6810/23584,v1:172.21.15.77:6811/23584] conn(0x7feff005b310 0x7feff005d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:14.936 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.922+0000 7ff00cff9700 1 -- 172.21.15.17:0/4217036149 --> [v2:172.21.15.77:6810/23584,v1:172.21.15.77:6811/23584] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feff005de30 con 0x7feff005b310 2022-08-01T19:34:14.936 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.922+0000 7ff00cff9700 1 -- 172.21.15.17:0/4217036149 <== mon.2 v2:172.21.15.77:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff00006c290 con 0x7ff01005c960 2022-08-01T19:34:14.937 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.922+0000 7ff00effd700 1 --2- 172.21.15.17:0/4217036149 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7feff0053a10 0x7feff0055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.937 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.922+0000 7ff00ffff700 1 --2- 172.21.15.17:0/4217036149 >> [v2:172.21.15.77:6810/23584,v1:172.21.15.77:6811/23584] conn(0x7feff005b310 0x7feff005d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:14.938 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.922+0000 7ff00effd700 1 --2- 172.21.15.17:0/4217036149 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7feff0053a10 0x7feff0055ed0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7ff00401a000 tx=0x7ff004014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:14.938 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.922+0000 7ff00ffff700 1 --2- 172.21.15.17:0/4217036149 >> [v2:172.21.15.77:6810/23584,v1:172.21.15.77:6811/23584] conn(0x7feff005b310 0x7feff005d7b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:14.938 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.926+0000 7ff00cff9700 1 -- 172.21.15.17:0/4217036149 <== osd.4 v2:172.21.15.77:6810/23584 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7feff005de30 con 0x7feff005b310 2022-08-01T19:34:14.949 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.946+0000 7ff014e15700 1 -- 172.21.15.17:0/4217036149 --> [v2:172.21.15.77:6810/23584,v1:172.21.15.77:6811/23584] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff0100574e0 con 0x7feff005b310 2022-08-01T19:34:14.950 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.950+0000 7ff00cff9700 1 -- 172.21.15.17:0/4217036149 <== osd.4 v2:172.21.15.77:6810/23584 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff0100574e0 con 0x7feff005b310 2022-08-01T19:34:14.952 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.950+0000 7ff014e15700 1 -- 172.21.15.17:0/4217036149 >> [v2:172.21.15.77:6810/23584,v1:172.21.15.77:6811/23584] conn(0x7feff005b310 msgr2=0x7feff005d7b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.952 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.950+0000 7ff014e15700 1 --2- 172.21.15.17:0/4217036149 >> [v2:172.21.15.77:6810/23584,v1:172.21.15.77:6811/23584] conn(0x7feff005b310 0x7feff005d7b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:14.952 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.950+0000 7ff014e15700 1 -- 172.21.15.17:0/4217036149 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7feff0053a10 msgr2=0x7feff0055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.952 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.950+0000 7ff014e15700 1 --2- 172.21.15.17:0/4217036149 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7feff0053a10 0x7feff0055ed0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7ff00401a000 tx=0x7ff004014000).stop 2022-08-01T19:34:14.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.950+0000 7ff014e15700 1 -- 172.21.15.17:0/4217036149 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff01005c960 msgr2=0x7ff0100b0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:14.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.950+0000 7ff014e15700 1 --2- 172.21.15.17:0/4217036149 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff01005c960 0x7ff0100b0140 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7ff000000ed0 tx=0x7ff000007a20).stop 2022-08-01T19:34:14.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.950+0000 7ff00ffff700 1 -- 172.21.15.17:0/4217036149 reap_dead start 2022-08-01T19:34:14.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.950+0000 7ff014e15700 1 -- 172.21.15.17:0/4217036149 shutdown_connections 2022-08-01T19:34:14.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.950+0000 7ff014e15700 1 -- 172.21.15.17:0/4217036149 >> 172.21.15.17:0/4217036149 conn(0x7ff010060310 msgr2=0x7ff01013f970 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:14.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.950+0000 7ff014e15700 1 -- 172.21.15.17:0/4217036149 shutdown_connections 2022-08-01T19:34:14.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:14.950+0000 7ff014e15700 1 -- 172.21.15.17:0/4217036149 wait complete. 2022-08-01T19:34:14.968 INFO:teuthology.orchestra.run.smithi017.stdout:64424509445 2022-08-01T19:34:14.969 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2022-08-01T19:34:15.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.078+0000 7f5d9fa1a700 1 Processor -- start 2022-08-01T19:34:15.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.078+0000 7f5d9fa1a700 1 -- start start 2022-08-01T19:34:15.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.078+0000 7f5d9fa1a700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5d980b0a10 0x7f5d9814adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:15.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.078+0000 7f5d9fa1a700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d980b2c80 con 0x7f5d980b0630 2022-08-01T19:34:15.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.078+0000 7f5d9fa1a700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d980b2900 con 0x7f5d980aa2b0 2022-08-01T19:34:15.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.078+0000 7f5d9fa1a700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d980b2eb0 con 0x7f5d980b0a10 2022-08-01T19:34:15.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.078+0000 7f5d9dfb7700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5d980b0a10 0x7f5d9814adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:15.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.078+0000 7f5d9dfb7700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5d980b0a10 0x7f5d9814adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36482/0 (socket says 172.21.15.17:36482) 2022-08-01T19:34:15.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.078+0000 7f5d9dfb7700 1 -- 172.21.15.17:0/949955372 learned_addr learned my addr 172.21.15.17:0/949955372 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:15.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.082+0000 7f5d9dfb7700 1 -- 172.21.15.17:0/949955372 >> v1:172.21.15.77:6789/0 conn(0x7f5d980aa2b0 legacy=0x7f5d980ae5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.082+0000 7f5d9dfb7700 1 -- 172.21.15.17:0/949955372 >> v1:172.21.15.17:6789/0 conn(0x7f5d980b0630 legacy=0x7f5d980adec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.082+0000 7f5d9dfb7700 1 -- 172.21.15.17:0/949955372 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d980a9d00 con 0x7f5d980b0a10 2022-08-01T19:34:15.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.082+0000 7f5d9dfb7700 1 --2- 172.21.15.17:0/949955372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d980b0a10 0x7f5d9814adf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f5d980ad560 tx=0x7f5d90009290).ready entity=mon.0 client_cookie=216e51031d589975 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:15.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.082+0000 7f5d97fff700 1 -- 172.21.15.17:0/949955372 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d9000b860 con 0x7f5d980b0a10 2022-08-01T19:34:15.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.082+0000 7f5d97fff700 1 -- 172.21.15.17:0/949955372 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d9000b9c0 con 0x7f5d980b0a10 2022-08-01T19:34:15.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.082+0000 7f5d97fff700 1 -- 172.21.15.17:0/949955372 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d9000bb30 con 0x7f5d980b0a10 2022-08-01T19:34:15.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.082+0000 7f5d9fa1a700 1 -- 172.21.15.17:0/949955372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d980b0a10 msgr2=0x7f5d9814adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.082+0000 7f5d9fa1a700 1 --2- 172.21.15.17:0/949955372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d980b0a10 0x7f5d9814adf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f5d980ad560 tx=0x7f5d90009290).stop 2022-08-01T19:34:15.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.082+0000 7f5d9fa1a700 1 -- 172.21.15.17:0/949955372 shutdown_connections 2022-08-01T19:34:15.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.082+0000 7f5d9fa1a700 1 --2- 172.21.15.17:0/949955372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d980b0a10 0x7f5d9814adf0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:15.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.082+0000 7f5d9fa1a700 1 -- 172.21.15.17:0/949955372 >> 172.21.15.17:0/949955372 conn(0x7f5d98060950 msgr2=0x7f5d98060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:15.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.082+0000 7f5d9fa1a700 1 -- 172.21.15.17:0/949955372 shutdown_connections 2022-08-01T19:34:15.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.082+0000 7f5d9fa1a700 1 -- 172.21.15.17:0/949955372 wait complete. 2022-08-01T19:34:15.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.082+0000 7f5d9fa1a700 1 Processor -- start 2022-08-01T19:34:15.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.082+0000 7f5d9fa1a700 1 -- start start 2022-08-01T19:34:15.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.086+0000 7f5d9fa1a700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d980aa2b0 0x7f5d98063fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:15.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.086+0000 7f5d9fa1a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d980b0630 0x7f5d9813c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:15.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.086+0000 7f5d9fa1a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d980b0a10 0x7f5d9813c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:15.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.086+0000 7f5d9fa1a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d981441b0 con 0x7f5d980b0630 2022-08-01T19:34:15.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.086+0000 7f5d9fa1a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d98144920 con 0x7f5d980b0a10 2022-08-01T19:34:15.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.086+0000 7f5d9fa1a700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d981443e0 con 0x7f5d980aa2b0 2022-08-01T19:34:15.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.086+0000 7f5d9cfb5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d980b0630 0x7f5d9813c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:15.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.086+0000 7f5d9cfb5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d980b0630 0x7f5d9813c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36484/0 (socket says 172.21.15.17:36484) 2022-08-01T19:34:15.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.086+0000 7f5d9cfb5700 1 -- 172.21.15.17:0/1329945440 learned_addr learned my addr 172.21.15.17:0/1329945440 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:15.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.086+0000 7f5d9dfb7700 1 --2- 172.21.15.17:0/1329945440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d980b0a10 0x7f5d9813c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:15.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.086+0000 7f5d9d7b6700 1 --2- 172.21.15.17:0/1329945440 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d980aa2b0 0x7f5d98063fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:15.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.086+0000 7f5d9cfb5700 1 -- 172.21.15.17:0/1329945440 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d980aa2b0 msgr2=0x7f5d98063fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.086+0000 7f5d9cfb5700 1 --2- 172.21.15.17:0/1329945440 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d980aa2b0 0x7f5d98063fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:15.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.086+0000 7f5d9cfb5700 1 -- 172.21.15.17:0/1329945440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d980b0a10 msgr2=0x7f5d9813c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.086+0000 7f5d9cfb5700 1 --2- 172.21.15.17:0/1329945440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d980b0a10 0x7f5d9813c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:15.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.086+0000 7f5d9cfb5700 1 -- 172.21.15.17:0/1329945440 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d90009530 con 0x7f5d980b0630 2022-08-01T19:34:15.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.086+0000 7f5d9dfb7700 1 --2- 172.21.15.17:0/1329945440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d980b0a10 0x7f5d9813c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:34:15.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.086+0000 7f5d9d7b6700 1 --2- 172.21.15.17:0/1329945440 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d980aa2b0 0x7f5d98063fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:34:15.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.086+0000 7f5d9cfb5700 1 --2- 172.21.15.17:0/1329945440 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d980b0630 0x7f5d9813c500 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f5d980b7900 tx=0x7f5d8c007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:15.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.086+0000 7f5d967fc700 1 -- 172.21.15.17:0/1329945440 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d8c00ce30 con 0x7f5d980b0630 2022-08-01T19:34:15.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.086+0000 7f5d967fc700 1 -- 172.21.15.17:0/1329945440 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8c007240 con 0x7f5d980b0630 2022-08-01T19:34:15.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.086+0000 7f5d9fa1a700 1 -- 172.21.15.17:0/1329945440 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d98144680 con 0x7f5d980b0630 2022-08-01T19:34:15.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.086+0000 7f5d967fc700 1 -- 172.21.15.17:0/1329945440 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d8c006700 con 0x7f5d980b0630 2022-08-01T19:34:15.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.086+0000 7f5d9fa1a700 1 -- 172.21.15.17:0/1329945440 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d9813ba40 con 0x7f5d980b0630 2022-08-01T19:34:15.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.086+0000 7f5d9fa1a700 1 -- 172.21.15.17:0/1329945440 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5d980574e0 con 0x7f5d980b0630 2022-08-01T19:34:15.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.086+0000 7f5d967fc700 1 -- 172.21.15.17:0/1329945440 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5d8c010030 con 0x7f5d980b0630 2022-08-01T19:34:15.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.086+0000 7f5d967fc700 1 --2- 172.21.15.17:0/1329945440 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5d78053b30 0x7f5d78055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:15.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.090+0000 7f5d967fc700 1 -- 172.21.15.17:0/1329945440 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f5d8c06bcc0 con 0x7f5d980b0630 2022-08-01T19:34:15.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.090+0000 7f5d967fc700 1 --2- 172.21.15.17:0/1329945440 >> [v2:172.21.15.77:6802/23585,v1:172.21.15.77:6803/23585] conn(0x7f5d7805b4b0 0x7f5d7805d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:15.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.090+0000 7f5d967fc700 1 -- 172.21.15.17:0/1329945440 --> [v2:172.21.15.77:6802/23585,v1:172.21.15.77:6803/23585] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5d7805df30 con 0x7f5d7805b4b0 2022-08-01T19:34:15.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.090+0000 7f5d967fc700 1 -- 172.21.15.17:0/1329945440 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d8c06f020 con 0x7f5d980b0630 2022-08-01T19:34:15.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.090+0000 7f5d9d7b6700 1 --2- 172.21.15.17:0/1329945440 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5d78053b30 0x7f5d78055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:15.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.090+0000 7f5d9dfb7700 1 --2- 172.21.15.17:0/1329945440 >> [v2:172.21.15.77:6802/23585,v1:172.21.15.77:6803/23585] conn(0x7f5d7805b4b0 0x7f5d7805d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:15.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.090+0000 7f5d9dfb7700 1 --2- 172.21.15.17:0/1329945440 >> [v2:172.21.15.77:6802/23585,v1:172.21.15.77:6803/23585] conn(0x7f5d7805b4b0 0x7f5d7805d950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:15.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.090+0000 7f5d9d7b6700 1 --2- 172.21.15.17:0/1329945440 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5d78053b30 0x7f5d78055ff0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f5d88008000 tx=0x7f5d88009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:15.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.090+0000 7f5d967fc700 1 -- 172.21.15.17:0/1329945440 <== osd.5 v2:172.21.15.77:6802/23585 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f5d7805df30 con 0x7f5d7805b4b0 2022-08-01T19:34:15.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.114+0000 7f5d9fa1a700 1 -- 172.21.15.17:0/1329945440 --> [v2:172.21.15.77:6802/23585,v1:172.21.15.77:6803/23585] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5d980447e0 con 0x7f5d7805b4b0 2022-08-01T19:34:15.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.114+0000 7f5d967fc700 1 -- 172.21.15.17:0/1329945440 <== osd.5 v2:172.21.15.77:6802/23585 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5d980447e0 con 0x7f5d7805b4b0 2022-08-01T19:34:15.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.114+0000 7f5d9fa1a700 1 -- 172.21.15.17:0/1329945440 >> [v2:172.21.15.77:6802/23585,v1:172.21.15.77:6803/23585] conn(0x7f5d7805b4b0 msgr2=0x7f5d7805d950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.114+0000 7f5d9fa1a700 1 --2- 172.21.15.17:0/1329945440 >> [v2:172.21.15.77:6802/23585,v1:172.21.15.77:6803/23585] conn(0x7f5d7805b4b0 0x7f5d7805d950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:15.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.114+0000 7f5d9fa1a700 1 -- 172.21.15.17:0/1329945440 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5d78053b30 msgr2=0x7f5d78055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.114+0000 7f5d9fa1a700 1 --2- 172.21.15.17:0/1329945440 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5d78053b30 0x7f5d78055ff0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f5d88008000 tx=0x7f5d88009040).stop 2022-08-01T19:34:15.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.114+0000 7f5d9fa1a700 1 -- 172.21.15.17:0/1329945440 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d980b0630 msgr2=0x7f5d9813c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.114+0000 7f5d9fa1a700 1 --2- 172.21.15.17:0/1329945440 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d980b0630 0x7f5d9813c500 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f5d980b7900 tx=0x7f5d8c007be0).stop 2022-08-01T19:34:15.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.114+0000 7f5d9dfb7700 1 -- 172.21.15.17:0/1329945440 reap_dead start 2022-08-01T19:34:15.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.114+0000 7f5d9fa1a700 1 -- 172.21.15.17:0/1329945440 shutdown_connections 2022-08-01T19:34:15.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.114+0000 7f5d9fa1a700 1 -- 172.21.15.17:0/1329945440 >> 172.21.15.17:0/1329945440 conn(0x7f5d98060950 msgr2=0x7f5d980b4960 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:15.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.114+0000 7f5d9fa1a700 1 -- 172.21.15.17:0/1329945440 shutdown_connections 2022-08-01T19:34:15.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.114+0000 7f5d9fa1a700 1 -- 172.21.15.17:0/1329945440 wait complete. 2022-08-01T19:34:15.131 INFO:teuthology.orchestra.run.smithi017.stdout:64424509446 2022-08-01T19:34:15.132 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2022-08-01T19:34:15.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.238+0000 7fd566b11700 1 Processor -- start 2022-08-01T19:34:15.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.238+0000 7fd566b11700 1 -- start start 2022-08-01T19:34:15.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.238+0000 7fd566b11700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd5600b1160 0x7fd5601448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:15.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.238+0000 7fd566b11700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5600b04e0 con 0x7fd5600b1f40 2022-08-01T19:34:15.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.238+0000 7fd566b11700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5600b0780 con 0x7fd560148530 2022-08-01T19:34:15.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.238+0000 7fd566b11700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5600a9250 con 0x7fd5600b1160 2022-08-01T19:34:15.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.238+0000 7fd5650ae700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd5600b1160 0x7fd5601448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:15.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.238+0000 7fd5648ad700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd5600b1f40 0x7fd5600b2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56922/0 (socket says 172.21.15.17:56922) 2022-08-01T19:34:15.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.238+0000 7fd5648ad700 1 -- 172.21.15.17:0/3773917991 learned_addr learned my addr 172.21.15.17:0/3773917991 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:15.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.238+0000 7fd5650ae700 1 -- 172.21.15.17:0/3773917991 >> v1:172.21.15.77:6789/0 conn(0x7fd560148530 legacy=0x7fd5600a7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.238+0000 7fd5650ae700 1 -- 172.21.15.17:0/3773917991 >> v1:172.21.15.17:6789/0 conn(0x7fd5600b1f40 legacy=0x7fd5600b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.238+0000 7fd55f7fe700 1 -- 172.21.15.17:0/3773917991 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4071575038 0 0) 0x7fd5600b04e0 con 0x7fd5600b1f40 2022-08-01T19:34:15.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.238+0000 7fd5650ae700 1 -- 172.21.15.17:0/3773917991 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5600b0a20 con 0x7fd5600b1160 2022-08-01T19:34:15.246 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.238+0000 7fd5650ae700 1 --2- 172.21.15.17:0/3773917991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5600b1160 0x7fd5601448b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fd5601478c0 tx=0x7fd5580091d0).ready entity=mon.0 client_cookie=304c80ae9bf732c8 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:15.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.238+0000 7fd55f7fe700 1 -- 172.21.15.17:0/3773917991 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd55800b860 con 0x7fd5600b1160 2022-08-01T19:34:15.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.238+0000 7fd55f7fe700 1 -- 172.21.15.17:0/3773917991 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd55800b9c0 con 0x7fd5600b1160 2022-08-01T19:34:15.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.238+0000 7fd55f7fe700 1 -- 172.21.15.17:0/3773917991 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd55800bb30 con 0x7fd5600b1160 2022-08-01T19:34:15.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.238+0000 7fd566b11700 1 -- 172.21.15.17:0/3773917991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5600b1160 msgr2=0x7fd5601448b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.238+0000 7fd566b11700 1 --2- 172.21.15.17:0/3773917991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5600b1160 0x7fd5601448b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fd5601478c0 tx=0x7fd5580091d0).stop 2022-08-01T19:34:15.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.238+0000 7fd566b11700 1 -- 172.21.15.17:0/3773917991 shutdown_connections 2022-08-01T19:34:15.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.238+0000 7fd566b11700 1 --2- 172.21.15.17:0/3773917991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5600b1160 0x7fd5601448b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:15.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.238+0000 7fd566b11700 1 -- 172.21.15.17:0/3773917991 >> 172.21.15.17:0/3773917991 conn(0x7fd56005f840 msgr2=0x7fd56005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:15.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.238+0000 7fd566b11700 1 -- 172.21.15.17:0/3773917991 shutdown_connections 2022-08-01T19:34:15.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.238+0000 7fd566b11700 1 -- 172.21.15.17:0/3773917991 wait complete. 2022-08-01T19:34:15.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.238+0000 7fd566b11700 1 Processor -- start 2022-08-01T19:34:15.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.242+0000 7fd566b11700 1 -- start start 2022-08-01T19:34:15.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.242+0000 7fd566b11700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5600b1160 0x7fd560142a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:15.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.242+0000 7fd566b11700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd5600b1f40 0x7fd5600add90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:15.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.242+0000 7fd566b11700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd560148530 0x7fd56013faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:15.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.242+0000 7fd566b11700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5600ad330 con 0x7fd560148530 2022-08-01T19:34:15.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.242+0000 7fd566b11700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56013f6c0 con 0x7fd5600b1160 2022-08-01T19:34:15.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.242+0000 7fd566b11700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd56013dc10 con 0x7fd5600b1f40 2022-08-01T19:34:15.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.242+0000 7fd5650ae700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd560148530 0x7fd56013faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:15.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.242+0000 7fd5648ad700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5600b1160 0x7fd560142a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:15.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.242+0000 7fd5650ae700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd560148530 0x7fd56013faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36504/0 (socket says 172.21.15.17:36504) 2022-08-01T19:34:15.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.242+0000 7fd5648ad700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5600b1160 0x7fd560142a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:34422/0 (socket says 172.21.15.17:34422) 2022-08-01T19:34:15.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.242+0000 7fd5650ae700 1 -- 172.21.15.17:0/462844933 learned_addr learned my addr 172.21.15.17:0/462844933 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:15.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.242+0000 7fd55ffff700 1 --2- 172.21.15.17:0/462844933 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd5600b1f40 0x7fd5600add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:15.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.242+0000 7fd5650ae700 1 -- 172.21.15.17:0/462844933 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd5600b1f40 msgr2=0x7fd5600add90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.242+0000 7fd5650ae700 1 --2- 172.21.15.17:0/462844933 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd5600b1f40 0x7fd5600add90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:15.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.242+0000 7fd5650ae700 1 -- 172.21.15.17:0/462844933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5600b1160 msgr2=0x7fd560142a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.242+0000 7fd5650ae700 1 --2- 172.21.15.17:0/462844933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5600b1160 0x7fd560142a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:15.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.242+0000 7fd5650ae700 1 -- 172.21.15.17:0/462844933 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd558009470 con 0x7fd560148530 2022-08-01T19:34:15.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.242+0000 7fd5648ad700 1 --2- 172.21.15.17:0/462844933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5600b1160 0x7fd560142a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:34:15.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.242+0000 7fd5650ae700 1 --2- 172.21.15.17:0/462844933 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd560148530 0x7fd56013faf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fd5600ad470 tx=0x7fd558016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:15.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.242+0000 7fd55dffb700 1 -- 172.21.15.17:0/462844933 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd558011e60 con 0x7fd560148530 2022-08-01T19:34:15.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.242+0000 7fd55dffb700 1 -- 172.21.15.17:0/462844933 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5580032f0 con 0x7fd560148530 2022-08-01T19:34:15.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.242+0000 7fd566b11700 1 -- 172.21.15.17:0/462844933 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd56013deb0 con 0x7fd560148530 2022-08-01T19:34:15.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.242+0000 7fd55dffb700 1 -- 172.21.15.17:0/462844933 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd55801f5e0 con 0x7fd560148530 2022-08-01T19:34:15.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.242+0000 7fd566b11700 1 -- 172.21.15.17:0/462844933 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd56013e3f0 con 0x7fd560148530 2022-08-01T19:34:15.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.242+0000 7fd566b11700 1 -- 172.21.15.17:0/462844933 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd56013f850 con 0x7fd560148530 2022-08-01T19:34:15.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.246+0000 7fd55dffb700 1 -- 172.21.15.17:0/462844933 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd558026030 con 0x7fd560148530 2022-08-01T19:34:15.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.246+0000 7fd55dffb700 1 --2- 172.21.15.17:0/462844933 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd540053b00 0x7fd540055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:15.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.246+0000 7fd55dffb700 1 -- 172.21.15.17:0/462844933 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fd55806cd40 con 0x7fd560148530 2022-08-01T19:34:15.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.246+0000 7fd55dffb700 1 --2- 172.21.15.17:0/462844933 >> [v2:172.21.15.77:6820/23587,v1:172.21.15.77:6821/23587] conn(0x7fd54005b480 0x7fd54005d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:15.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.246+0000 7fd55dffb700 1 -- 172.21.15.17:0/462844933 --> [v2:172.21.15.77:6820/23587,v1:172.21.15.77:6821/23587] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd54005df00 con 0x7fd54005b480 2022-08-01T19:34:15.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.246+0000 7fd5648ad700 1 --2- 172.21.15.17:0/462844933 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd540053b00 0x7fd540055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:15.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.246+0000 7fd55dffb700 1 -- 172.21.15.17:0/462844933 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd55806f2c0 con 0x7fd560148530 2022-08-01T19:34:15.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.246+0000 7fd55ffff700 1 --2- 172.21.15.17:0/462844933 >> [v2:172.21.15.77:6820/23587,v1:172.21.15.77:6821/23587] conn(0x7fd54005b480 0x7fd54005d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:15.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.246+0000 7fd5648ad700 1 --2- 172.21.15.17:0/462844933 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd540053b00 0x7fd540055fc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fd55400d000 tx=0x7fd554009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:15.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.246+0000 7fd55ffff700 1 --2- 172.21.15.17:0/462844933 >> [v2:172.21.15.77:6820/23587,v1:172.21.15.77:6821/23587] conn(0x7fd54005b480 0x7fd54005d920 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:15.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.250+0000 7fd55dffb700 1 -- 172.21.15.17:0/462844933 <== osd.6 v2:172.21.15.77:6820/23587 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7fd54005df00 con 0x7fd54005b480 2022-08-01T19:34:15.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.266+0000 7fd566b11700 1 -- 172.21.15.17:0/462844933 --> [v2:172.21.15.77:6820/23587,v1:172.21.15.77:6821/23587] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd5600b0cc0 con 0x7fd54005b480 2022-08-01T19:34:15.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.270+0000 7fd55dffb700 1 -- 172.21.15.17:0/462844933 <== osd.6 v2:172.21.15.77:6820/23587 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd5600b0cc0 con 0x7fd54005b480 2022-08-01T19:34:15.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.270+0000 7fd566b11700 1 -- 172.21.15.17:0/462844933 >> [v2:172.21.15.77:6820/23587,v1:172.21.15.77:6821/23587] conn(0x7fd54005b480 msgr2=0x7fd54005d920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.270+0000 7fd566b11700 1 --2- 172.21.15.17:0/462844933 >> [v2:172.21.15.77:6820/23587,v1:172.21.15.77:6821/23587] conn(0x7fd54005b480 0x7fd54005d920 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:15.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.270+0000 7fd566b11700 1 -- 172.21.15.17:0/462844933 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd540053b00 msgr2=0x7fd540055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.270+0000 7fd566b11700 1 --2- 172.21.15.17:0/462844933 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd540053b00 0x7fd540055fc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fd55400d000 tx=0x7fd554009040).stop 2022-08-01T19:34:15.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.270+0000 7fd566b11700 1 -- 172.21.15.17:0/462844933 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd560148530 msgr2=0x7fd56013faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.270+0000 7fd566b11700 1 --2- 172.21.15.17:0/462844933 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd560148530 0x7fd56013faf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fd5600ad470 tx=0x7fd558016000).stop 2022-08-01T19:34:15.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.270+0000 7fd5650ae700 1 -- 172.21.15.17:0/462844933 reap_dead start 2022-08-01T19:34:15.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.270+0000 7fd566b11700 1 -- 172.21.15.17:0/462844933 shutdown_connections 2022-08-01T19:34:15.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.270+0000 7fd566b11700 1 -- 172.21.15.17:0/462844933 >> 172.21.15.17:0/462844933 conn(0x7fd56005f840 msgr2=0x7fd5600a7250 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:15.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.270+0000 7fd566b11700 1 -- 172.21.15.17:0/462844933 shutdown_connections 2022-08-01T19:34:15.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.270+0000 7fd566b11700 1 -- 172.21.15.17:0/462844933 wait complete. 2022-08-01T19:34:15.287 INFO:teuthology.orchestra.run.smithi017.stdout:60129542149 2022-08-01T19:34:15.287 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2022-08-01T19:34:15.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.398+0000 7fc532328700 1 Processor -- start 2022-08-01T19:34:15.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.398+0000 7fc532328700 1 -- start start 2022-08-01T19:34:15.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.398+0000 7fc532328700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fc52c13d9b0 0x7fc52c0609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:15.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.398+0000 7fc532328700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc52c066af0 con 0x7fc52c0b6510 2022-08-01T19:34:15.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.398+0000 7fc532328700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc52c066d90 con 0x7fc52c0b0e60 2022-08-01T19:34:15.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.398+0000 7fc532328700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc52c067030 con 0x7fc52c13d9b0 2022-08-01T19:34:15.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.398+0000 7fc531326700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc52c0b6510 0x7fc52c13d2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56936/0 (socket says 172.21.15.17:56936) 2022-08-01T19:34:15.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.398+0000 7fc531326700 1 -- 172.21.15.17:0/1118067398 learned_addr learned my addr 172.21.15.17:0/1118067398 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:15.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.398+0000 7fc530b25700 1 --2- 172.21.15.17:0/1118067398 >> v2:172.21.15.77:3300/0 conn(0x7fc52c13d9b0 0x7fc52c0609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:15.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.398+0000 7fc52bfff700 1 -- 172.21.15.17:0/1118067398 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3741829244 0 0) 0x7fc52c066af0 con 0x7fc52c0b6510 2022-08-01T19:34:15.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.398+0000 7fc52bfff700 1 -- 172.21.15.17:0/1118067398 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc518003260 con 0x7fc52c0b6510 2022-08-01T19:34:15.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.398+0000 7fc530b25700 1 -- 172.21.15.17:0/1118067398 >> v1:172.21.15.77:6789/0 conn(0x7fc52c0b0e60 legacy=0x7fc52c060eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.398+0000 7fc52bfff700 1 -- 172.21.15.17:0/1118067398 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3493014482 0 0) 0x7fc52c066d90 con 0x7fc52c0b0e60 2022-08-01T19:34:15.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.398+0000 7fc530b25700 1 -- 172.21.15.17:0/1118067398 >> v1:172.21.15.17:6789/0 conn(0x7fc52c0b6510 legacy=0x7fc52c13d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.398+0000 7fc530b25700 1 -- 172.21.15.17:0/1118067398 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52c0672d0 con 0x7fc52c13d9b0 2022-08-01T19:34:15.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.398+0000 7fc530b25700 1 --2- 172.21.15.17:0/1118067398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc52c13d9b0 0x7fc52c0609c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fc520002f70 tx=0x7fc520006df0).ready entity=mon.1 client_cookie=931b9ab72628c056 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:15.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.398+0000 7fc52bfff700 1 -- 172.21.15.17:0/1118067398 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc52000d610 con 0x7fc52c13d9b0 2022-08-01T19:34:15.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.398+0000 7fc52bfff700 1 -- 172.21.15.17:0/1118067398 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc520000e90 con 0x7fc52c13d9b0 2022-08-01T19:34:15.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.398+0000 7fc52bfff700 1 -- 172.21.15.17:0/1118067398 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc52000d960 con 0x7fc52c13d9b0 2022-08-01T19:34:15.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.398+0000 7fc532328700 1 -- 172.21.15.17:0/1118067398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc52c13d9b0 msgr2=0x7fc52c0609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.398+0000 7fc532328700 1 --2- 172.21.15.17:0/1118067398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc52c13d9b0 0x7fc52c0609c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fc520002f70 tx=0x7fc520006df0).stop 2022-08-01T19:34:15.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.398+0000 7fc532328700 1 -- 172.21.15.17:0/1118067398 shutdown_connections 2022-08-01T19:34:15.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.398+0000 7fc532328700 1 --2- 172.21.15.17:0/1118067398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc52c13d9b0 0x7fc52c0609c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:15.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.398+0000 7fc532328700 1 -- 172.21.15.17:0/1118067398 >> 172.21.15.17:0/1118067398 conn(0x7fc52c063800 msgr2=0x7fc52c063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:15.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.398+0000 7fc532328700 1 -- 172.21.15.17:0/1118067398 shutdown_connections 2022-08-01T19:34:15.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.398+0000 7fc532328700 1 -- 172.21.15.17:0/1118067398 wait complete. 2022-08-01T19:34:15.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.398+0000 7fc532328700 1 Processor -- start 2022-08-01T19:34:15.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.398+0000 7fc532328700 1 -- start start 2022-08-01T19:34:15.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.398+0000 7fc532328700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc52c0b0e60 0x7fc52c1440d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:15.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.398+0000 7fc532328700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc52c0b6510 0x7fc52c143750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:15.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.398+0000 7fc532328700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc52c13d9b0 0x7fc52c0aa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:15.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.398+0000 7fc532328700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c142d40 con 0x7fc52c0b6510 2022-08-01T19:34:15.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.398+0000 7fc532328700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c1500a0 con 0x7fc52c0b0e60 2022-08-01T19:34:15.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.398+0000 7fc532328700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c14f180 con 0x7fc52c13d9b0 2022-08-01T19:34:15.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.402+0000 7fc530b25700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc52c0b6510 0x7fc52c143750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:15.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.402+0000 7fc530b25700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc52c0b6510 0x7fc52c143750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36518/0 (socket says 172.21.15.17:36518) 2022-08-01T19:34:15.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.402+0000 7fc530b25700 1 -- 172.21.15.17:0/4129778428 learned_addr learned my addr 172.21.15.17:0/4129778428 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:15.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.402+0000 7fc531326700 1 --2- 172.21.15.17:0/4129778428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc52c0b0e60 0x7fc52c1440d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:15.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.402+0000 7fc531b27700 1 --2- 172.21.15.17:0/4129778428 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc52c13d9b0 0x7fc52c0aa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:15.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.402+0000 7fc530b25700 1 -- 172.21.15.17:0/4129778428 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc52c13d9b0 msgr2=0x7fc52c0aa3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.402+0000 7fc530b25700 1 --2- 172.21.15.17:0/4129778428 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc52c13d9b0 0x7fc52c0aa3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:15.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.402+0000 7fc530b25700 1 -- 172.21.15.17:0/4129778428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc52c0b0e60 msgr2=0x7fc52c1440d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.402+0000 7fc530b25700 1 --2- 172.21.15.17:0/4129778428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc52c0b0e60 0x7fc52c1440d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:15.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.402+0000 7fc530b25700 1 -- 172.21.15.17:0/4129778428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52000b180 con 0x7fc52c0b6510 2022-08-01T19:34:15.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.402+0000 7fc530b25700 1 --2- 172.21.15.17:0/4129778428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc52c0b6510 0x7fc52c143750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fc520002f70 tx=0x7fc520014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:15.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.402+0000 7fc52a7fc700 1 -- 172.21.15.17:0/4129778428 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc520013de0 con 0x7fc52c0b6510 2022-08-01T19:34:15.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.402+0000 7fc52a7fc700 1 -- 172.21.15.17:0/4129778428 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52000d290 con 0x7fc52c0b6510 2022-08-01T19:34:15.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.402+0000 7fc52a7fc700 1 -- 172.21.15.17:0/4129778428 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc52001e5f0 con 0x7fc52c0b6510 2022-08-01T19:34:15.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.402+0000 7fc532328700 1 -- 172.21.15.17:0/4129778428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc52c14f420 con 0x7fc52c0b6510 2022-08-01T19:34:15.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.402+0000 7fc532328700 1 -- 172.21.15.17:0/4129778428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc52c14fc00 con 0x7fc52c0b6510 2022-08-01T19:34:15.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.402+0000 7fc52a7fc700 1 -- 172.21.15.17:0/4129778428 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc52001c070 con 0x7fc52c0b6510 2022-08-01T19:34:15.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.402+0000 7fc532328700 1 -- 172.21.15.17:0/4129778428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc52c14fe00 con 0x7fc52c0b6510 2022-08-01T19:34:15.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.402+0000 7fc52a7fc700 1 --2- 172.21.15.17:0/4129778428 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc50c053a70 0x7fc50c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:15.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.402+0000 7fc52a7fc700 1 -- 172.21.15.17:0/4129778428 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fc52006ca70 con 0x7fc52c0b6510 2022-08-01T19:34:15.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.402+0000 7fc52a7fc700 1 --2- 172.21.15.17:0/4129778428 >> [v2:172.21.15.77:6818/23586,v1:172.21.15.77:6819/23586] conn(0x7fc50c05b3f0 0x7fc50c05d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:15.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.402+0000 7fc52a7fc700 1 -- 172.21.15.17:0/4129778428 --> [v2:172.21.15.77:6818/23586,v1:172.21.15.77:6819/23586] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc50c05de70 con 0x7fc50c05b3f0 2022-08-01T19:34:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.402+0000 7fc52a7fc700 1 -- 172.21.15.17:0/4129778428 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc520071240 con 0x7fc52c0b6510 2022-08-01T19:34:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.402+0000 7fc531326700 1 --2- 172.21.15.17:0/4129778428 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc50c053a70 0x7fc50c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.406+0000 7fc531b27700 1 --2- 172.21.15.17:0/4129778428 >> [v2:172.21.15.77:6818/23586,v1:172.21.15.77:6819/23586] conn(0x7fc50c05b3f0 0x7fc50c05d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.406+0000 7fc531326700 1 --2- 172.21.15.17:0/4129778428 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc50c053a70 0x7fc50c055f30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fc51c00d000 tx=0x7fc51c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.406+0000 7fc531b27700 1 --2- 172.21.15.17:0/4129778428 >> [v2:172.21.15.77:6818/23586,v1:172.21.15.77:6819/23586] conn(0x7fc50c05b3f0 0x7fc50c05d890 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.406+0000 7fc52a7fc700 1 -- 172.21.15.17:0/4129778428 <== osd.7 v2:172.21.15.77:6818/23586 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7fc50c05de70 con 0x7fc50c05b3f0 2022-08-01T19:34:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.426+0000 7fc532328700 1 -- 172.21.15.17:0/4129778428 --> [v2:172.21.15.77:6818/23586,v1:172.21.15.77:6819/23586] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc52c0574e0 con 0x7fc50c05b3f0 2022-08-01T19:34:15.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.426+0000 7fc52a7fc700 1 -- 172.21.15.17:0/4129778428 <== osd.7 v2:172.21.15.77:6818/23586 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc52c0574e0 con 0x7fc50c05b3f0 2022-08-01T19:34:15.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.426+0000 7fc532328700 1 -- 172.21.15.17:0/4129778428 >> [v2:172.21.15.77:6818/23586,v1:172.21.15.77:6819/23586] conn(0x7fc50c05b3f0 msgr2=0x7fc50c05d890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.426+0000 7fc532328700 1 --2- 172.21.15.17:0/4129778428 >> [v2:172.21.15.77:6818/23586,v1:172.21.15.77:6819/23586] conn(0x7fc50c05b3f0 0x7fc50c05d890 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:15.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.426+0000 7fc532328700 1 -- 172.21.15.17:0/4129778428 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc50c053a70 msgr2=0x7fc50c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.426+0000 7fc532328700 1 --2- 172.21.15.17:0/4129778428 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc50c053a70 0x7fc50c055f30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fc51c00d000 tx=0x7fc51c009040).stop 2022-08-01T19:34:15.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.426+0000 7fc532328700 1 -- 172.21.15.17:0/4129778428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc52c0b6510 msgr2=0x7fc52c143750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.426+0000 7fc532328700 1 --2- 172.21.15.17:0/4129778428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc52c0b6510 0x7fc52c143750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fc520002f70 tx=0x7fc520014000).stop 2022-08-01T19:34:15.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.426+0000 7fc531b27700 1 -- 172.21.15.17:0/4129778428 reap_dead start 2022-08-01T19:34:15.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.426+0000 7fc532328700 1 -- 172.21.15.17:0/4129778428 shutdown_connections 2022-08-01T19:34:15.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.426+0000 7fc532328700 1 -- 172.21.15.17:0/4129778428 >> 172.21.15.17:0/4129778428 conn(0x7fc52c063800 msgr2=0x7fc52c063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:15.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.426+0000 7fc532328700 1 -- 172.21.15.17:0/4129778428 shutdown_connections 2022-08-01T19:34:15.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.426+0000 7fc532328700 1 -- 172.21.15.17:0/4129778428 wait complete. 2022-08-01T19:34:15.444 INFO:teuthology.orchestra.run.smithi017.stdout:60129542150 2022-08-01T19:34:15.445 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2022-08-01T19:34:15.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9aa10d2700 1 Processor -- start 2022-08-01T19:34:15.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9aa10d2700 1 -- start start 2022-08-01T19:34:15.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9aa10d2700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9a9c0b3260 0x7f9a9c1435c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:15.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9aa10d2700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f9a9c0ad000 0x7f9a9c13fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:15.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9aa10d2700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a9c143ab0 con 0x7f9a9c0b08a0 2022-08-01T19:34:15.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9aa10d2700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a9c0b0c80 con 0x7f9a9c0b3260 2022-08-01T19:34:15.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9aa10d2700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a9c0ae810 con 0x7f9a9c0ad000 2022-08-01T19:34:15.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9a9bfff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9a9c0b08a0 0x7f9a9c142eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56952/0 (socket says 172.21.15.17:56952) 2022-08-01T19:34:15.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9a9bfff700 1 -- 172.21.15.17:0/3657112691 learned_addr learned my addr 172.21.15.17:0/3657112691 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:15.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9a9b7fe700 1 --2- 172.21.15.17:0/3657112691 >> v2:172.21.15.17:3300/0 conn(0x7f9a9c0b3260 0x7f9a9c1435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:15.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9aa08d1700 1 --2- 172.21.15.17:0/3657112691 >> v2:172.21.15.77:3300/0 conn(0x7f9a9c0ad000 0x7f9a9c13fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:15.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9a9b7fe700 1 -- 172.21.15.17:0/3657112691 >> v2:172.21.15.77:3300/0 conn(0x7f9a9c0ad000 msgr2=0x7f9a9c13fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:15.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9a9b7fe700 1 --2- 172.21.15.17:0/3657112691 >> v2:172.21.15.77:3300/0 conn(0x7f9a9c0ad000 0x7f9a9c13fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:15.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9a9b7fe700 1 -- 172.21.15.17:0/3657112691 >> v1:172.21.15.17:6789/0 conn(0x7f9a9c0b08a0 legacy=0x7f9a9c142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9a9b7fe700 1 -- 172.21.15.17:0/3657112691 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a9c0aeab0 con 0x7f9a9c0b3260 2022-08-01T19:34:15.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9a9b7fe700 1 --2- 172.21.15.17:0/3657112691 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9c0b3260 0x7f9a9c1435c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f9a90002f70 tx=0x7f9a90006df0).ready entity=mon.0 client_cookie=a75f78a13c55fa69 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:15.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9a9affd700 1 -- 172.21.15.17:0/3657112691 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a9000fa60 con 0x7f9a9c0b3260 2022-08-01T19:34:15.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9a9affd700 1 -- 172.21.15.17:0/3657112691 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a90000e90 con 0x7f9a9c0b3260 2022-08-01T19:34:15.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9a9affd700 1 -- 172.21.15.17:0/3657112691 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a9000fdb0 con 0x7f9a9c0b3260 2022-08-01T19:34:15.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9aa10d2700 1 -- 172.21.15.17:0/3657112691 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9c0b3260 msgr2=0x7f9a9c1435c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9aa10d2700 1 --2- 172.21.15.17:0/3657112691 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9c0b3260 0x7f9a9c1435c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f9a90002f70 tx=0x7f9a90006df0).stop 2022-08-01T19:34:15.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9aa10d2700 1 -- 172.21.15.17:0/3657112691 shutdown_connections 2022-08-01T19:34:15.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9aa10d2700 1 --2- 172.21.15.17:0/3657112691 >> v2:172.21.15.77:3300/0 conn(0x7f9a9c0ad000 0x7f9a9c13fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:15.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9aa10d2700 1 --2- 172.21.15.17:0/3657112691 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9c0b3260 0x7f9a9c1435c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:15.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9aa10d2700 1 -- 172.21.15.17:0/3657112691 >> 172.21.15.17:0/3657112691 conn(0x7f9a9c0612d0 msgr2=0x7f9a9c062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:15.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9aa10d2700 1 -- 172.21.15.17:0/3657112691 shutdown_connections 2022-08-01T19:34:15.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9aa10d2700 1 -- 172.21.15.17:0/3657112691 wait complete. 2022-08-01T19:34:15.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9aa10d2700 1 Processor -- start 2022-08-01T19:34:15.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9aa10d2700 1 -- start start 2022-08-01T19:34:15.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9aa10d2700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9a9c0ad000 0x7f9a9c0aa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:15.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9aa10d2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9c0b08a0 0x7f9a9c0ab480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:15.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9aa10d2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a9c0b3260 0x7f9a9c063f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:15.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9aa10d2700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c064550 con 0x7f9a9c0b08a0 2022-08-01T19:34:15.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9aa10d2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c0a99d0 con 0x7f9a9c0b3260 2022-08-01T19:34:15.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9aa10d2700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c13e750 con 0x7f9a9c0ad000 2022-08-01T19:34:15.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9a9b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9c0b08a0 0x7f9a9c0ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:15.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9a9b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9c0b08a0 0x7f9a9c0ab480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36536/0 (socket says 172.21.15.17:36536) 2022-08-01T19:34:15.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9a9b7fe700 1 -- 172.21.15.17:0/4122704812 learned_addr learned my addr 172.21.15.17:0/4122704812 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:15.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9a9bfff700 1 --2- 172.21.15.17:0/4122704812 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9a9c0ad000 0x7f9a9c0aa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:15.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9aa08d1700 1 --2- 172.21.15.17:0/4122704812 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a9c0b3260 0x7f9a9c063f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:15.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9a9b7fe700 1 -- 172.21.15.17:0/4122704812 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9a9c0ad000 msgr2=0x7f9a9c0aa5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9a9b7fe700 1 --2- 172.21.15.17:0/4122704812 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9a9c0ad000 0x7f9a9c0aa5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:15.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9a9b7fe700 1 -- 172.21.15.17:0/4122704812 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a9c0b3260 msgr2=0x7f9a9c063f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9a9b7fe700 1 --2- 172.21.15.17:0/4122704812 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a9c0b3260 0x7f9a9c063f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:15.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9a9b7fe700 1 -- 172.21.15.17:0/4122704812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a9000b180 con 0x7f9a9c0b08a0 2022-08-01T19:34:15.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9aa08d1700 1 --2- 172.21.15.17:0/4122704812 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a9c0b3260 0x7f9a9c063f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:34:15.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.554+0000 7f9a9b7fe700 1 --2- 172.21.15.17:0/4122704812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9c0b08a0 0x7f9a9c0ab480 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f9a90002f70 tx=0x7f9a90016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:15.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.558+0000 7f9a997fa700 1 -- 172.21.15.17:0/4122704812 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a900175f0 con 0x7f9a9c0b08a0 2022-08-01T19:34:15.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.558+0000 7f9a997fa700 1 -- 172.21.15.17:0/4122704812 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a90017750 con 0x7f9a9c0b08a0 2022-08-01T19:34:15.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.558+0000 7f9aa10d2700 1 -- 172.21.15.17:0/4122704812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a9c0a9c00 con 0x7f9a9c0b08a0 2022-08-01T19:34:15.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.558+0000 7f9a997fa700 1 -- 172.21.15.17:0/4122704812 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a90017ab0 con 0x7f9a9c0b08a0 2022-08-01T19:34:15.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.558+0000 7f9aa10d2700 1 -- 172.21.15.17:0/4122704812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a9c0aac50 con 0x7f9a9c0b08a0 2022-08-01T19:34:15.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.558+0000 7f9aa10d2700 1 -- 172.21.15.17:0/4122704812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a9c0574e0 con 0x7f9a9c0b08a0 2022-08-01T19:34:15.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.558+0000 7f9a997fa700 1 -- 172.21.15.17:0/4122704812 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9a9002a070 con 0x7f9a9c0b08a0 2022-08-01T19:34:15.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.558+0000 7f9a997fa700 1 --2- 172.21.15.17:0/4122704812 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9a7c053a70 0x7f9a7c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:15.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.558+0000 7f9a997fa700 1 -- 172.21.15.17:0/4122704812 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f9a9006d030 con 0x7f9a9c0b08a0 2022-08-01T19:34:15.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.562+0000 7f9a9bfff700 1 --2- 172.21.15.17:0/4122704812 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9a7c053a70 0x7f9a7c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:15.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.562+0000 7f9a9bfff700 1 --2- 172.21.15.17:0/4122704812 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9a7c053a70 0x7f9a7c055f30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f9a8c00d000 tx=0x7f9a8c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:15.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.566+0000 7f9a997fa700 1 -- 172.21.15.17:0/4122704812 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9a90043d50 con 0x7f9a9c0b08a0 2022-08-01T19:34:15.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.694+0000 7f9aa10d2700 1 -- 172.21.15.17:0/4122704812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9a9c0b2e00 con 0x7f9a9c0b08a0 2022-08-01T19:34:15.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.694+0000 7f9a997fa700 1 -- 172.21.15.17:0/4122704812 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9a90047370 con 0x7f9a9c0b08a0 2022-08-01T19:34:15.700 INFO:teuthology.orchestra.run.smithi017.stdout:60129542149 2022-08-01T19:34:15.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.694+0000 7f9aa10d2700 1 -- 172.21.15.17:0/4122704812 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9a7c053a70 msgr2=0x7f9a7c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.694+0000 7f9aa10d2700 1 --2- 172.21.15.17:0/4122704812 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9a7c053a70 0x7f9a7c055f30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f9a8c00d000 tx=0x7f9a8c009040).stop 2022-08-01T19:34:15.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.694+0000 7f9aa10d2700 1 -- 172.21.15.17:0/4122704812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9c0b08a0 msgr2=0x7f9a9c0ab480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.694+0000 7f9aa10d2700 1 --2- 172.21.15.17:0/4122704812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9c0b08a0 0x7f9a9c0ab480 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f9a90002f70 tx=0x7f9a90016000).stop 2022-08-01T19:34:15.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.694+0000 7f9aa10d2700 1 -- 172.21.15.17:0/4122704812 shutdown_connections 2022-08-01T19:34:15.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.694+0000 7f9aa10d2700 1 --2- 172.21.15.17:0/4122704812 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9a7c053a70 0x7f9a7c055f30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:15.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.694+0000 7f9aa10d2700 1 --2- 172.21.15.17:0/4122704812 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a9c0b3260 0x7f9a9c063f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:15.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.694+0000 7f9aa10d2700 1 --2- 172.21.15.17:0/4122704812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9c0b08a0 0x7f9a9c0ab480 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:15.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.694+0000 7f9aa10d2700 1 --2- 172.21.15.17:0/4122704812 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9a9c0ad000 0x7f9a9c0aa5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:15.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.694+0000 7f9aa10d2700 1 -- 172.21.15.17:0/4122704812 >> 172.21.15.17:0/4122704812 conn(0x7f9a9c0612d0 msgr2=0x7f9a9c142dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:15.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.694+0000 7f9aa10d2700 1 -- 172.21.15.17:0/4122704812 shutdown_connections 2022-08-01T19:34:15.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.694+0000 7f9aa10d2700 1 -- 172.21.15.17:0/4122704812 wait complete. 2022-08-01T19:34:15.713 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.0 2022-08-01T19:34:15.713 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2022-08-01T19:34:15.816 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.806+0000 7fc966051700 1 Processor -- start 2022-08-01T19:34:15.816 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.806+0000 7fc966051700 1 -- start start 2022-08-01T19:34:15.816 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.806+0000 7fc966051700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fc9600ac6a0 0x7fc96014cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.806+0000 7fc966051700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9601453e0 con 0x7fc9600b2390 2022-08-01T19:34:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.806+0000 7fc966051700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc960145680 con 0x7fc9600b0160 2022-08-01T19:34:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.806+0000 7fc966051700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc960141390 con 0x7fc9600ac6a0 2022-08-01T19:34:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.810+0000 7fc95f7fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc9600b2390 0x7fc9600abf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56966/0 (socket says 172.21.15.17:56966) 2022-08-01T19:34:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.810+0000 7fc95f7fe700 1 -- 172.21.15.17:0/1269046251 learned_addr learned my addr 172.21.15.17:0/1269046251 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.810+0000 7fc95ffff700 1 --2- 172.21.15.17:0/1269046251 >> v2:172.21.15.77:3300/0 conn(0x7fc9600ac6a0 0x7fc96014cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.810+0000 7fc95e7fc700 1 -- 172.21.15.17:0/1269046251 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3155065828 0 0) 0x7fc9601453e0 con 0x7fc9600b2390 2022-08-01T19:34:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.810+0000 7fc95e7fc700 1 -- 172.21.15.17:0/1269046251 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc94c003260 con 0x7fc9600b2390 2022-08-01T19:34:15.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.810+0000 7fc95e7fc700 1 -- 172.21.15.17:0/1269046251 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2746621610 0 0) 0x7fc960145680 con 0x7fc9600b0160 2022-08-01T19:34:15.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.810+0000 7fc95e7fc700 1 -- 172.21.15.17:0/1269046251 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9601453e0 con 0x7fc9600b0160 2022-08-01T19:34:15.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.810+0000 7fc95ffff700 1 -- 172.21.15.17:0/1269046251 >> v1:172.21.15.77:6789/0 conn(0x7fc9600b0160 legacy=0x7fc960141830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.810+0000 7fc95ffff700 1 -- 172.21.15.17:0/1269046251 >> v1:172.21.15.17:6789/0 conn(0x7fc9600b2390 legacy=0x7fc9600abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.810+0000 7fc95ffff700 1 -- 172.21.15.17:0/1269046251 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9600b0670 con 0x7fc9600ac6a0 2022-08-01T19:34:15.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.810+0000 7fc95ffff700 1 --2- 172.21.15.17:0/1269046251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9600ac6a0 0x7fc96014cc00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fc958002f70 tx=0x7fc958006df0).ready entity=mon.1 client_cookie=16bb7de783a03463 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:15.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.810+0000 7fc95e7fc700 1 -- 172.21.15.17:0/1269046251 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc95800d540 con 0x7fc9600ac6a0 2022-08-01T19:34:15.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.810+0000 7fc95e7fc700 1 -- 172.21.15.17:0/1269046251 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc958000e90 con 0x7fc9600ac6a0 2022-08-01T19:34:15.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.810+0000 7fc95e7fc700 1 -- 172.21.15.17:0/1269046251 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc95800d890 con 0x7fc9600ac6a0 2022-08-01T19:34:15.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.810+0000 7fc966051700 1 -- 172.21.15.17:0/1269046251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9600ac6a0 msgr2=0x7fc96014cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.810+0000 7fc966051700 1 --2- 172.21.15.17:0/1269046251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9600ac6a0 0x7fc96014cc00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fc958002f70 tx=0x7fc958006df0).stop 2022-08-01T19:34:15.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.810+0000 7fc966051700 1 -- 172.21.15.17:0/1269046251 shutdown_connections 2022-08-01T19:34:15.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.810+0000 7fc966051700 1 --2- 172.21.15.17:0/1269046251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9600ac6a0 0x7fc96014cc00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:15.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.810+0000 7fc966051700 1 -- 172.21.15.17:0/1269046251 >> 172.21.15.17:0/1269046251 conn(0x7fc96005f840 msgr2=0x7fc96005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:15.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.810+0000 7fc966051700 1 -- 172.21.15.17:0/1269046251 shutdown_connections 2022-08-01T19:34:15.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.810+0000 7fc966051700 1 -- 172.21.15.17:0/1269046251 wait complete. 2022-08-01T19:34:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.814+0000 7fc966051700 1 Processor -- start 2022-08-01T19:34:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.814+0000 7fc966051700 1 -- start start 2022-08-01T19:34:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.814+0000 7fc966051700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9600ac6a0 0x7fc9600670a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.814+0000 7fc966051700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9600b0160 0x7fc960156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:15.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.814+0000 7fc966051700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9600b2390 0x7fc960156670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:15.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.814+0000 7fc966051700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9601504a0 con 0x7fc9600ac6a0 2022-08-01T19:34:15.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.814+0000 7fc966051700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc960155d20 con 0x7fc9600b2390 2022-08-01T19:34:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.814+0000 7fc966051700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc960156b60 con 0x7fc9600b0160 2022-08-01T19:34:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.814+0000 7fc95f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9600ac6a0 0x7fc9600670a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.814+0000 7fc95f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9600ac6a0 0x7fc9600670a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36546/0 (socket says 172.21.15.17:36546) 2022-08-01T19:34:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.814+0000 7fc95f7fe700 1 -- 172.21.15.17:0/215689053 learned_addr learned my addr 172.21.15.17:0/215689053 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:15.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.814+0000 7fc95effd700 1 --2- 172.21.15.17:0/215689053 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9600b0160 0x7fc960156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:15.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.814+0000 7fc95ffff700 1 --2- 172.21.15.17:0/215689053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9600b2390 0x7fc960156670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:15.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.814+0000 7fc95f7fe700 1 -- 172.21.15.17:0/215689053 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9600b0160 msgr2=0x7fc960156180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.814+0000 7fc95f7fe700 1 --2- 172.21.15.17:0/215689053 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9600b0160 0x7fc960156180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:15.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.814+0000 7fc95f7fe700 1 -- 172.21.15.17:0/215689053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9600b2390 msgr2=0x7fc960156670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.814+0000 7fc95f7fe700 1 --2- 172.21.15.17:0/215689053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9600b2390 0x7fc960156670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.814+0000 7fc95f7fe700 1 -- 172.21.15.17:0/215689053 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95800b180 con 0x7fc9600ac6a0 2022-08-01T19:34:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.814+0000 7fc95effd700 1 --2- 172.21.15.17:0/215689053 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9600b0160 0x7fc960156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:34:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.814+0000 7fc95f7fe700 1 --2- 172.21.15.17:0/215689053 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9600ac6a0 0x7fc9600670a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fc9540011d0 tx=0x7fc954007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.814+0000 7fc95cff9700 1 -- 172.21.15.17:0/215689053 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9540070d0 con 0x7fc9600ac6a0 2022-08-01T19:34:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.814+0000 7fc95cff9700 1 -- 172.21.15.17:0/215689053 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc954007230 con 0x7fc9600ac6a0 2022-08-01T19:34:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.814+0000 7fc966051700 1 -- 172.21.15.17:0/215689053 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc960143be0 con 0x7fc9600ac6a0 2022-08-01T19:34:15.834 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.814+0000 7fc95cff9700 1 -- 172.21.15.17:0/215689053 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc954005610 con 0x7fc9600ac6a0 2022-08-01T19:34:15.834 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.814+0000 7fc966051700 1 -- 172.21.15.17:0/215689053 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc960144600 con 0x7fc9600ac6a0 2022-08-01T19:34:15.834 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.814+0000 7fc966051700 1 -- 172.21.15.17:0/215689053 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9600574e0 con 0x7fc9600ac6a0 2022-08-01T19:34:15.834 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.818+0000 7fc95cff9700 1 -- 172.21.15.17:0/215689053 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc95401a060 con 0x7fc9600ac6a0 2022-08-01T19:34:15.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.818+0000 7fc95cff9700 1 --2- 172.21.15.17:0/215689053 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9400539e0 0x7fc940055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:15.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.818+0000 7fc95cff9700 1 -- 172.21.15.17:0/215689053 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fc95406b860 con 0x7fc9600ac6a0 2022-08-01T19:34:15.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.818+0000 7fc95effd700 1 --2- 172.21.15.17:0/215689053 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9400539e0 0x7fc940055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:15.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.818+0000 7fc95effd700 1 --2- 172.21.15.17:0/215689053 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9400539e0 0x7fc940055ea0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fc95000b000 tx=0x7fc950009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:15.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.822+0000 7fc95cff9700 1 -- 172.21.15.17:0/215689053 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc954042620 con 0x7fc9600ac6a0 2022-08-01T19:34:15.951 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.946+0000 7fc966051700 1 -- 172.21.15.17:0/215689053 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc9600447e0 con 0x7fc9600ac6a0 2022-08-01T19:34:15.952 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.946+0000 7fc95cff9700 1 -- 172.21.15.17:0/215689053 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc954018020 con 0x7fc9600ac6a0 2022-08-01T19:34:15.952 INFO:teuthology.orchestra.run.smithi017.stdout:60129542150 2022-08-01T19:34:15.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.946+0000 7fc966051700 1 -- 172.21.15.17:0/215689053 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9400539e0 msgr2=0x7fc940055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.946+0000 7fc966051700 1 --2- 172.21.15.17:0/215689053 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9400539e0 0x7fc940055ea0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fc95000b000 tx=0x7fc950009040).stop 2022-08-01T19:34:15.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.946+0000 7fc966051700 1 -- 172.21.15.17:0/215689053 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9600ac6a0 msgr2=0x7fc9600670a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:15.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.946+0000 7fc966051700 1 --2- 172.21.15.17:0/215689053 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9600ac6a0 0x7fc9600670a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fc9540011d0 tx=0x7fc954007b50).stop 2022-08-01T19:34:15.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.946+0000 7fc966051700 1 -- 172.21.15.17:0/215689053 shutdown_connections 2022-08-01T19:34:15.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.946+0000 7fc966051700 1 --2- 172.21.15.17:0/215689053 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9400539e0 0x7fc940055ea0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:15.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.946+0000 7fc966051700 1 --2- 172.21.15.17:0/215689053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9600b2390 0x7fc960156670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:15.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.946+0000 7fc966051700 1 --2- 172.21.15.17:0/215689053 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9600b0160 0x7fc960156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:15.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.946+0000 7fc966051700 1 --2- 172.21.15.17:0/215689053 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9600ac6a0 0x7fc9600670a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:15.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.946+0000 7fc966051700 1 -- 172.21.15.17:0/215689053 >> 172.21.15.17:0/215689053 conn(0x7fc96005f840 msgr2=0x7fc960141750 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:15.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.946+0000 7fc966051700 1 -- 172.21.15.17:0/215689053 shutdown_connections 2022-08-01T19:34:15.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:15.946+0000 7fc966051700 1 -- 172.21.15.17:0/215689053 wait complete. 2022-08-01T19:34:15.972 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.1 2022-08-01T19:34:15.973 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2022-08-01T19:34:16.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.074+0000 7f0c8f605700 1 Processor -- start 2022-08-01T19:34:16.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.074+0000 7f0c8f605700 1 -- start start 2022-08-01T19:34:16.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.078+0000 7f0c8f605700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0c880b68f0 0x7f0c8813ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:16.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.078+0000 7f0c8f605700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f0c8805ff40 0x7f0c8813d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:16.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.078+0000 7f0c8f605700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c8813d860 con 0x7f0c880aa210 2022-08-01T19:34:16.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.078+0000 7f0c8f605700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c880ac6e0 con 0x7f0c880b68f0 2022-08-01T19:34:16.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.078+0000 7f0c8f605700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c880ac980 con 0x7f0c8805ff40 2022-08-01T19:34:16.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.078+0000 7f0c8d3a1700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0c880b68f0 0x7f0c8813ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:16.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.078+0000 7f0c8d3a1700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0c880b68f0 0x7f0c8813ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36554/0 (socket says 172.21.15.17:36554) 2022-08-01T19:34:16.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.078+0000 7f0c8d3a1700 1 -- 172.21.15.17:0/1765481677 learned_addr learned my addr 172.21.15.17:0/1765481677 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:16.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.078+0000 7f0c8dba2700 1 --2- 172.21.15.17:0/1765481677 >> v2:172.21.15.77:3300/0 conn(0x7f0c8805ff40 0x7f0c8813d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:16.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.078+0000 7f0c8d3a1700 1 -- 172.21.15.17:0/1765481677 >> v2:172.21.15.77:3300/0 conn(0x7f0c8805ff40 msgr2=0x7f0c8813d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:16.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.078+0000 7f0c8d3a1700 1 --2- 172.21.15.17:0/1765481677 >> v2:172.21.15.77:3300/0 conn(0x7f0c8805ff40 0x7f0c8813d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:16.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.078+0000 7f0c8d3a1700 1 -- 172.21.15.17:0/1765481677 >> v1:172.21.15.77:6789/0 conn(0x7f0c880aa210 legacy=0x7f0c88146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:16.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.078+0000 7f0c8d3a1700 1 -- 172.21.15.17:0/1765481677 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c880acc20 con 0x7f0c880b68f0 2022-08-01T19:34:16.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.078+0000 7f0c8d3a1700 1 --2- 172.21.15.17:0/1765481677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c880b68f0 0x7f0c8813ce80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f0c78002f70 tx=0x7f0c78006eb0).ready entity=mon.0 client_cookie=1d1e8ae38be7316e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:16.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.078+0000 7f0c7ffff700 1 -- 172.21.15.17:0/1765481677 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c7800d5c0 con 0x7f0c880b68f0 2022-08-01T19:34:16.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.078+0000 7f0c7ffff700 1 -- 172.21.15.17:0/1765481677 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c78000e90 con 0x7f0c880b68f0 2022-08-01T19:34:16.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.078+0000 7f0c7ffff700 1 -- 172.21.15.17:0/1765481677 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c7800d950 con 0x7f0c880b68f0 2022-08-01T19:34:16.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.078+0000 7f0c8f605700 1 -- 172.21.15.17:0/1765481677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c880b68f0 msgr2=0x7f0c8813ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:16.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.078+0000 7f0c8f605700 1 --2- 172.21.15.17:0/1765481677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c880b68f0 0x7f0c8813ce80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f0c78002f70 tx=0x7f0c78006eb0).stop 2022-08-01T19:34:16.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.078+0000 7f0c8f605700 1 -- 172.21.15.17:0/1765481677 shutdown_connections 2022-08-01T19:34:16.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.078+0000 7f0c8f605700 1 --2- 172.21.15.17:0/1765481677 >> v2:172.21.15.77:3300/0 conn(0x7f0c8805ff40 0x7f0c8813d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:16.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.078+0000 7f0c8f605700 1 --2- 172.21.15.17:0/1765481677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c880b68f0 0x7f0c8813ce80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:16.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.078+0000 7f0c8f605700 1 -- 172.21.15.17:0/1765481677 >> 172.21.15.17:0/1765481677 conn(0x7f0c880b3610 msgr2=0x7f0c880b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:16.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.078+0000 7f0c8f605700 1 -- 172.21.15.17:0/1765481677 shutdown_connections 2022-08-01T19:34:16.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.078+0000 7f0c8f605700 1 -- 172.21.15.17:0/1765481677 wait complete. 2022-08-01T19:34:16.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.082+0000 7f0c8f605700 1 Processor -- start 2022-08-01T19:34:16.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.082+0000 7f0c8f605700 1 -- start start 2022-08-01T19:34:16.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.082+0000 7f0c8f605700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c8805ff40 0x7f0c880b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:16.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.082+0000 7f0c8f605700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c880aa210 0x7f0c88152f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:16.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.082+0000 7f0c8f605700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0c880b68f0 0x7f0c88063890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:16.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.082+0000 7f0c8f605700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c88152860 con 0x7f0c880aa210 2022-08-01T19:34:16.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.082+0000 7f0c8f605700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c880650d0 con 0x7f0c8805ff40 2022-08-01T19:34:16.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.082+0000 7f0c8f605700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c88152a90 con 0x7f0c880b68f0 2022-08-01T19:34:16.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.082+0000 7f0c8cba0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c880aa210 0x7f0c88152f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:16.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.082+0000 7f0c8cba0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c880aa210 0x7f0c88152f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36562/0 (socket says 172.21.15.17:36562) 2022-08-01T19:34:16.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.082+0000 7f0c8cba0700 1 -- 172.21.15.17:0/2388424061 learned_addr learned my addr 172.21.15.17:0/2388424061 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:16.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.082+0000 7f0c8d3a1700 1 --2- 172.21.15.17:0/2388424061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c8805ff40 0x7f0c880b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:16.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.082+0000 7f0c8dba2700 1 --2- 172.21.15.17:0/2388424061 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0c880b68f0 0x7f0c88063890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:16.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.082+0000 7f0c8cba0700 1 -- 172.21.15.17:0/2388424061 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0c880b68f0 msgr2=0x7f0c88063890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:16.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.082+0000 7f0c8cba0700 1 --2- 172.21.15.17:0/2388424061 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0c880b68f0 0x7f0c88063890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:16.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.082+0000 7f0c8cba0700 1 -- 172.21.15.17:0/2388424061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c8805ff40 msgr2=0x7f0c880b42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:16.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.082+0000 7f0c8cba0700 1 --2- 172.21.15.17:0/2388424061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c8805ff40 0x7f0c880b42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:16.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.082+0000 7f0c8cba0700 1 -- 172.21.15.17:0/2388424061 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7800b1e0 con 0x7f0c880aa210 2022-08-01T19:34:16.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.082+0000 7f0c8cba0700 1 --2- 172.21.15.17:0/2388424061 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c880aa210 0x7f0c88152f30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f0c880b5f50 tx=0x7f0c84007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:16.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.082+0000 7f0c7e7fc700 1 -- 172.21.15.17:0/2388424061 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c84012400 con 0x7f0c880aa210 2022-08-01T19:34:16.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.082+0000 7f0c7e7fc700 1 -- 172.21.15.17:0/2388424061 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c84012560 con 0x7f0c880aa210 2022-08-01T19:34:16.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.082+0000 7f0c7e7fc700 1 -- 172.21.15.17:0/2388424061 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c840128c0 con 0x7f0c880aa210 2022-08-01T19:34:16.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.086+0000 7f0c8f605700 1 -- 172.21.15.17:0/2388424061 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c88065b80 con 0x7f0c880aa210 2022-08-01T19:34:16.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.086+0000 7f0c8f605700 1 -- 172.21.15.17:0/2388424061 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c88065370 con 0x7f0c880aa210 2022-08-01T19:34:16.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.086+0000 7f0c7e7fc700 1 -- 172.21.15.17:0/2388424061 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0c84019030 con 0x7f0c880aa210 2022-08-01T19:34:16.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.086+0000 7f0c8f605700 1 -- 172.21.15.17:0/2388424061 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c88152c50 con 0x7f0c880aa210 2022-08-01T19:34:16.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.086+0000 7f0c7e7fc700 1 --2- 172.21.15.17:0/2388424061 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0c700539e0 0x7f0c70055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:16.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.086+0000 7f0c7e7fc700 1 -- 172.21.15.17:0/2388424061 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f0c8406b490 con 0x7f0c880aa210 2022-08-01T19:34:16.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.086+0000 7f0c8d3a1700 1 --2- 172.21.15.17:0/2388424061 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0c700539e0 0x7f0c70055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:16.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.086+0000 7f0c8d3a1700 1 --2- 172.21.15.17:0/2388424061 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0c700539e0 0x7f0c70055ea0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f0c7801c000 tx=0x7f0c78015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:16.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.094+0000 7f0c7e7fc700 1 -- 172.21.15.17:0/2388424061 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0c84042140 con 0x7f0c880aa210 2022-08-01T19:34:16.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.214+0000 7f0c8f605700 1 -- 172.21.15.17:0/2388424061 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0c880acec0 con 0x7f0c880aa210 2022-08-01T19:34:16.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.214+0000 7f0c7e7fc700 1 -- 172.21.15.17:0/2388424061 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0c84045760 con 0x7f0c880aa210 2022-08-01T19:34:16.223 INFO:teuthology.orchestra.run.smithi017.stdout:60129542151 2022-08-01T19:34:16.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.218+0000 7f0c8f605700 1 -- 172.21.15.17:0/2388424061 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0c700539e0 msgr2=0x7f0c70055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:16.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.218+0000 7f0c8f605700 1 --2- 172.21.15.17:0/2388424061 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0c700539e0 0x7f0c70055ea0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f0c7801c000 tx=0x7f0c78015000).stop 2022-08-01T19:34:16.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.218+0000 7f0c8f605700 1 -- 172.21.15.17:0/2388424061 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c880aa210 msgr2=0x7f0c88152f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:16.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.218+0000 7f0c8f605700 1 --2- 172.21.15.17:0/2388424061 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c880aa210 0x7f0c88152f30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f0c880b5f50 tx=0x7f0c84007c50).stop 2022-08-01T19:34:16.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.218+0000 7f0c8f605700 1 -- 172.21.15.17:0/2388424061 shutdown_connections 2022-08-01T19:34:16.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.218+0000 7f0c8f605700 1 --2- 172.21.15.17:0/2388424061 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0c700539e0 0x7f0c70055ea0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:16.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.218+0000 7f0c8f605700 1 --2- 172.21.15.17:0/2388424061 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0c880b68f0 0x7f0c88063890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:16.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.218+0000 7f0c8f605700 1 --2- 172.21.15.17:0/2388424061 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c880aa210 0x7f0c88152f30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:16.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.218+0000 7f0c8f605700 1 --2- 172.21.15.17:0/2388424061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0c8805ff40 0x7f0c880b42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:16.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.218+0000 7f0c8f605700 1 -- 172.21.15.17:0/2388424061 >> 172.21.15.17:0/2388424061 conn(0x7f0c880b3610 msgr2=0x7f0c88143b60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:16.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.218+0000 7f0c8f605700 1 -- 172.21.15.17:0/2388424061 shutdown_connections 2022-08-01T19:34:16.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.218+0000 7f0c8f605700 1 -- 172.21.15.17:0/2388424061 wait complete. 2022-08-01T19:34:16.235 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.2 2022-08-01T19:34:16.235 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2022-08-01T19:34:16.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.334+0000 7fd5ce89f700 1 Processor -- start 2022-08-01T19:34:16.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.334+0000 7fd5ce89f700 1 -- start start 2022-08-01T19:34:16.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.334+0000 7fd5ce89f700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd5c80614b0 0x7fd5c8061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:16.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.334+0000 7fd5ce89f700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fd5c81405d0 0x7fd5c8140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:16.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.334+0000 7fd5ce89f700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5c80aee80 con 0x7fd5c80ad750 2022-08-01T19:34:16.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.334+0000 7fd5ce89f700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5c80b70b0 con 0x7fd5c80614b0 2022-08-01T19:34:16.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.334+0000 7fd5ce89f700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5c80af0b0 con 0x7fd5c81405d0 2022-08-01T19:34:16.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.334+0000 7fd5cd89d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd5c80614b0 0x7fd5c8061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:16.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.334+0000 7fd5cd89d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd5c80614b0 0x7fd5c8061890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36568/0 (socket says 172.21.15.17:36568) 2022-08-01T19:34:16.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.334+0000 7fd5cd89d700 1 -- 172.21.15.17:0/1455222820 learned_addr learned my addr 172.21.15.17:0/1455222820 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:16.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.334+0000 7fd5ce09e700 1 --2- 172.21.15.17:0/1455222820 >> v2:172.21.15.77:3300/0 conn(0x7fd5c81405d0 0x7fd5c8140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:16.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.334+0000 7fd5cd89d700 1 -- 172.21.15.17:0/1455222820 >> v2:172.21.15.77:3300/0 conn(0x7fd5c81405d0 msgr2=0x7fd5c8140a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:16.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.334+0000 7fd5cd89d700 1 --2- 172.21.15.17:0/1455222820 >> v2:172.21.15.77:3300/0 conn(0x7fd5c81405d0 0x7fd5c8140a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:16.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.334+0000 7fd5cd89d700 1 -- 172.21.15.17:0/1455222820 >> v1:172.21.15.77:6789/0 conn(0x7fd5c80ad750 legacy=0x7fd5c80b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:16.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.334+0000 7fd5cd89d700 1 -- 172.21.15.17:0/1455222820 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c80af350 con 0x7fd5c80614b0 2022-08-01T19:34:16.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.334+0000 7fd5cd89d700 1 --2- 172.21.15.17:0/1455222820 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5c80614b0 0x7fd5c8061890 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fd5c80aed30 tx=0x7fd5b0009290).ready entity=mon.0 client_cookie=bfa68bee9fc291dd server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:16.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.334+0000 7fd5cc89b700 1 -- 172.21.15.17:0/1455222820 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5b000b860 con 0x7fd5c80614b0 2022-08-01T19:34:16.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.334+0000 7fd5cc89b700 1 -- 172.21.15.17:0/1455222820 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b000b9c0 con 0x7fd5c80614b0 2022-08-01T19:34:16.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.334+0000 7fd5cc89b700 1 -- 172.21.15.17:0/1455222820 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5b000bb30 con 0x7fd5c80614b0 2022-08-01T19:34:16.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.334+0000 7fd5ce89f700 1 -- 172.21.15.17:0/1455222820 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5c80614b0 msgr2=0x7fd5c8061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:16.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.334+0000 7fd5ce89f700 1 --2- 172.21.15.17:0/1455222820 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5c80614b0 0x7fd5c8061890 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fd5c80aed30 tx=0x7fd5b0009290).stop 2022-08-01T19:34:16.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.334+0000 7fd5ce89f700 1 -- 172.21.15.17:0/1455222820 shutdown_connections 2022-08-01T19:34:16.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.334+0000 7fd5ce89f700 1 --2- 172.21.15.17:0/1455222820 >> v2:172.21.15.77:3300/0 conn(0x7fd5c81405d0 0x7fd5c8140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:16.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.334+0000 7fd5ce89f700 1 --2- 172.21.15.17:0/1455222820 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5c80614b0 0x7fd5c8061890 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:16.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.334+0000 7fd5ce89f700 1 -- 172.21.15.17:0/1455222820 >> 172.21.15.17:0/1455222820 conn(0x7fd5c8066f70 msgr2=0x7fd5c8067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:16.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.334+0000 7fd5ce89f700 1 -- 172.21.15.17:0/1455222820 shutdown_connections 2022-08-01T19:34:16.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.334+0000 7fd5ce89f700 1 -- 172.21.15.17:0/1455222820 wait complete. 2022-08-01T19:34:16.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.338+0000 7fd5ce89f700 1 Processor -- start 2022-08-01T19:34:16.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.338+0000 7fd5ce89f700 1 -- start start 2022-08-01T19:34:16.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.338+0000 7fd5ce89f700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd5c80614b0 0x7fd5c809ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:16.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.338+0000 7fd5ce89f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5c80ad750 0x7fd5c809f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:16.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.338+0000 7fd5ce89f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5c81405d0 0x7fd5c80acb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:16.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.338+0000 7fd5ce89f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c8153110 con 0x7fd5c81405d0 2022-08-01T19:34:16.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.338+0000 7fd5ce89f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c809f940 con 0x7fd5c80ad750 2022-08-01T19:34:16.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.338+0000 7fd5ce89f700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c80b0360 con 0x7fd5c80614b0 2022-08-01T19:34:16.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.338+0000 7fd5ce09e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5c81405d0 0x7fd5c80acb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:16.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.338+0000 7fd5ce09e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5c81405d0 0x7fd5c80acb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36578/0 (socket says 172.21.15.17:36578) 2022-08-01T19:34:16.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.338+0000 7fd5ce09e700 1 -- 172.21.15.17:0/414873080 learned_addr learned my addr 172.21.15.17:0/414873080 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:16.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.338+0000 7fd5cd89d700 1 --2- 172.21.15.17:0/414873080 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd5c80614b0 0x7fd5c809ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:16.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.338+0000 7fd5cd09c700 1 --2- 172.21.15.17:0/414873080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5c80ad750 0x7fd5c809f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:16.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.338+0000 7fd5ce09e700 1 -- 172.21.15.17:0/414873080 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd5c80614b0 msgr2=0x7fd5c809ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:16.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.338+0000 7fd5ce09e700 1 --2- 172.21.15.17:0/414873080 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd5c80614b0 0x7fd5c809ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:16.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.338+0000 7fd5ce09e700 1 -- 172.21.15.17:0/414873080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5c80ad750 msgr2=0x7fd5c809f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:16.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.338+0000 7fd5ce09e700 1 --2- 172.21.15.17:0/414873080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5c80ad750 0x7fd5c809f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:16.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.338+0000 7fd5ce09e700 1 -- 172.21.15.17:0/414873080 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b0009530 con 0x7fd5c81405d0 2022-08-01T19:34:16.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.338+0000 7fd5ce09e700 1 --2- 172.21.15.17:0/414873080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5c81405d0 0x7fd5c80acb90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fd5c000c000 tx=0x7fd5c0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:16.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.338+0000 7fd5c6ffd700 1 -- 172.21.15.17:0/414873080 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5c0010780 con 0x7fd5c81405d0 2022-08-01T19:34:16.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.338+0000 7fd5c6ffd700 1 -- 172.21.15.17:0/414873080 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5c00108e0 con 0x7fd5c81405d0 2022-08-01T19:34:16.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.338+0000 7fd5c6ffd700 1 -- 172.21.15.17:0/414873080 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5c0010c40 con 0x7fd5c81405d0 2022-08-01T19:34:16.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.338+0000 7fd5ce89f700 1 -- 172.21.15.17:0/414873080 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5c8153300 con 0x7fd5c81405d0 2022-08-01T19:34:16.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.338+0000 7fd5ce89f700 1 -- 172.21.15.17:0/414873080 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c80b0760 con 0x7fd5c81405d0 2022-08-01T19:34:16.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.342+0000 7fd5ce89f700 1 -- 172.21.15.17:0/414873080 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5c80af5f0 con 0x7fd5c81405d0 2022-08-01T19:34:16.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.342+0000 7fd5c6ffd700 1 -- 172.21.15.17:0/414873080 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd5c001c460 con 0x7fd5c81405d0 2022-08-01T19:34:16.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.342+0000 7fd5c6ffd700 1 --2- 172.21.15.17:0/414873080 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd5ac053aa0 0x7fd5ac055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:16.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.342+0000 7fd5c6ffd700 1 -- 172.21.15.17:0/414873080 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fd5c006d060 con 0x7fd5c81405d0 2022-08-01T19:34:16.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.342+0000 7fd5cd89d700 1 --2- 172.21.15.17:0/414873080 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd5ac053aa0 0x7fd5ac055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:16.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.342+0000 7fd5cd89d700 1 --2- 172.21.15.17:0/414873080 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd5ac053aa0 0x7fd5ac055f60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fd5b0019000 tx=0x7fd5b0016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:16.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.346+0000 7fd5c6ffd700 1 -- 172.21.15.17:0/414873080 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd5c0043cb0 con 0x7fd5c81405d0 2022-08-01T19:34:16.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.470+0000 7fd5ce89f700 1 -- 172.21.15.17:0/414873080 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd5c80574e0 con 0x7fd5c81405d0 2022-08-01T19:34:16.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.474+0000 7fd5c6ffd700 1 -- 172.21.15.17:0/414873080 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd5c00472d0 con 0x7fd5c81405d0 2022-08-01T19:34:16.479 INFO:teuthology.orchestra.run.smithi017.stdout:60129542149 2022-08-01T19:34:16.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.474+0000 7fd5ce89f700 1 -- 172.21.15.17:0/414873080 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd5ac053aa0 msgr2=0x7fd5ac055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:16.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.474+0000 7fd5ce89f700 1 --2- 172.21.15.17:0/414873080 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd5ac053aa0 0x7fd5ac055f60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fd5b0019000 tx=0x7fd5b0016000).stop 2022-08-01T19:34:16.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.474+0000 7fd5ce89f700 1 -- 172.21.15.17:0/414873080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5c81405d0 msgr2=0x7fd5c80acb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:16.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.474+0000 7fd5ce89f700 1 --2- 172.21.15.17:0/414873080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5c81405d0 0x7fd5c80acb90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fd5c000c000 tx=0x7fd5c0006df0).stop 2022-08-01T19:34:16.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.474+0000 7fd5ce89f700 1 -- 172.21.15.17:0/414873080 shutdown_connections 2022-08-01T19:34:16.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.474+0000 7fd5ce89f700 1 --2- 172.21.15.17:0/414873080 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd5ac053aa0 0x7fd5ac055f60 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:16.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.474+0000 7fd5ce89f700 1 --2- 172.21.15.17:0/414873080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5c81405d0 0x7fd5c80acb90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:16.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.474+0000 7fd5ce89f700 1 --2- 172.21.15.17:0/414873080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5c80ad750 0x7fd5c809f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:16.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.474+0000 7fd5ce89f700 1 --2- 172.21.15.17:0/414873080 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd5c80614b0 0x7fd5c809ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:16.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.474+0000 7fd5ce89f700 1 -- 172.21.15.17:0/414873080 >> 172.21.15.17:0/414873080 conn(0x7fd5c8066f70 msgr2=0x7fd5c8067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:16.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.474+0000 7fd5ce89f700 1 -- 172.21.15.17:0/414873080 shutdown_connections 2022-08-01T19:34:16.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.474+0000 7fd5ce89f700 1 -- 172.21.15.17:0/414873080 wait complete. 2022-08-01T19:34:16.491 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.3 2022-08-01T19:34:16.492 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2022-08-01T19:34:16.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.590+0000 7f2bcafa6700 1 Processor -- start 2022-08-01T19:34:16.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.590+0000 7f2bcafa6700 1 -- start start 2022-08-01T19:34:16.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bcafa6700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2bc40a9da0 0x7f2bc4060950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:16.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bcafa6700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc4067030 con 0x7f2bc41402d0 2022-08-01T19:34:16.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bcafa6700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc40b35b0 con 0x7f2bc40b31d0 2022-08-01T19:34:16.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bcafa6700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc4064a70 con 0x7f2bc40a9da0 2022-08-01T19:34:16.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bca7a5700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2bc41402d0 0x7f2bc4140760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57012/0 (socket says 172.21.15.17:57012) 2022-08-01T19:34:16.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bca7a5700 1 -- 172.21.15.17:0/2426912060 learned_addr learned my addr 172.21.15.17:0/2426912060 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:16.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bc97a3700 1 --2- 172.21.15.17:0/2426912060 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2bc40a9da0 0x7f2bc4060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:16.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bc8fa2700 1 -- 172.21.15.17:0/2426912060 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2447092269 0 0) 0x7f2bc4067030 con 0x7f2bc41402d0 2022-08-01T19:34:16.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bc8fa2700 1 -- 172.21.15.17:0/2426912060 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bac003260 con 0x7f2bc41402d0 2022-08-01T19:34:16.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bc97a3700 1 -- 172.21.15.17:0/2426912060 >> v1:172.21.15.77:6789/0 conn(0x7f2bc40b31d0 legacy=0x7f2bc4066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:16.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bc97a3700 1 -- 172.21.15.17:0/2426912060 >> v1:172.21.15.17:6789/0 conn(0x7f2bc41402d0 legacy=0x7f2bc4140760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:16.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bc97a3700 1 -- 172.21.15.17:0/2426912060 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc4066af0 con 0x7f2bc40a9da0 2022-08-01T19:34:16.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bc97a3700 1 --2- 172.21.15.17:0/2426912060 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2bc40a9da0 0x7f2bc4060950 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f2bb4002f70 tx=0x7f2bb4006df0).ready entity=mon.2 client_cookie=e99739e7e85c36ce server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:16.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bc8fa2700 1 -- 172.21.15.17:0/2426912060 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2bb4014070 con 0x7f2bc40a9da0 2022-08-01T19:34:16.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bc8fa2700 1 -- 172.21.15.17:0/2426912060 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb4000e90 con 0x7f2bc40a9da0 2022-08-01T19:34:16.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bc8fa2700 1 -- 172.21.15.17:0/2426912060 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2bb400fb60 con 0x7f2bc40a9da0 2022-08-01T19:34:16.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bcafa6700 1 -- 172.21.15.17:0/2426912060 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2bc40a9da0 msgr2=0x7f2bc4060950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:16.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bcafa6700 1 --2- 172.21.15.17:0/2426912060 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2bc40a9da0 0x7f2bc4060950 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f2bb4002f70 tx=0x7f2bb4006df0).stop 2022-08-01T19:34:16.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bcafa6700 1 -- 172.21.15.17:0/2426912060 shutdown_connections 2022-08-01T19:34:16.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bcafa6700 1 --2- 172.21.15.17:0/2426912060 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2bc40a9da0 0x7f2bc4060950 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:16.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bcafa6700 1 -- 172.21.15.17:0/2426912060 >> 172.21.15.17:0/2426912060 conn(0x7f2bc4063870 msgr2=0x7f2bc4063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:16.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bcafa6700 1 -- 172.21.15.17:0/2426912060 shutdown_connections 2022-08-01T19:34:16.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bcafa6700 1 -- 172.21.15.17:0/2426912060 wait complete. 2022-08-01T19:34:16.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bcafa6700 1 Processor -- start 2022-08-01T19:34:16.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bcafa6700 1 -- start start 2022-08-01T19:34:16.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bcafa6700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2bc40a9da0 0x7f2bc40608e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:16.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bcafa6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2bc40b31d0 0x7f2bc4151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:16.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bcafa6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bc41402d0 0x7f2bc40ad200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:16.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bcafa6700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc4152300 con 0x7f2bc40b31d0 2022-08-01T19:34:16.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bcafa6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc40aca40 con 0x7f2bc41402d0 2022-08-01T19:34:16.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bcafa6700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc4151430 con 0x7f2bc40a9da0 2022-08-01T19:34:16.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bc97a3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2bc40b31d0 0x7f2bc4151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:16.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bc97a3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2bc40b31d0 0x7f2bc4151e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36590/0 (socket says 172.21.15.17:36590) 2022-08-01T19:34:16.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bc97a3700 1 -- 172.21.15.17:0/3176683640 learned_addr learned my addr 172.21.15.17:0/3176683640 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:16.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bca7a5700 1 --2- 172.21.15.17:0/3176683640 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bc41402d0 0x7f2bc40ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:16.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.598+0000 7f2bc9fa4700 1 --2- 172.21.15.17:0/3176683640 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2bc40a9da0 0x7f2bc40608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:16.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.602+0000 7f2bc97a3700 1 -- 172.21.15.17:0/3176683640 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2bc40a9da0 msgr2=0x7f2bc40608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:16.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.602+0000 7f2bc97a3700 1 --2- 172.21.15.17:0/3176683640 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2bc40a9da0 0x7f2bc40608e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:16.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.602+0000 7f2bc97a3700 1 -- 172.21.15.17:0/3176683640 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bc41402d0 msgr2=0x7f2bc40ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:16.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.602+0000 7f2bc97a3700 1 --2- 172.21.15.17:0/3176683640 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bc41402d0 0x7f2bc40ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:16.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.602+0000 7f2bc97a3700 1 -- 172.21.15.17:0/3176683640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb400b180 con 0x7f2bc40b31d0 2022-08-01T19:34:16.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.602+0000 7f2bca7a5700 1 --2- 172.21.15.17:0/3176683640 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bc41402d0 0x7f2bc40ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:34:16.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.602+0000 7f2bc97a3700 1 --2- 172.21.15.17:0/3176683640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2bc40b31d0 0x7f2bc4151e10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f2bb4002f70 tx=0x7f2bb4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:16.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.602+0000 7f2bb2ffd700 1 -- 172.21.15.17:0/3176683640 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2bb401d450 con 0x7f2bc40b31d0 2022-08-01T19:34:16.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.602+0000 7f2bb2ffd700 1 -- 172.21.15.17:0/3176683640 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb401d5b0 con 0x7f2bc40b31d0 2022-08-01T19:34:16.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.602+0000 7f2bcafa6700 1 -- 172.21.15.17:0/3176683640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc41516d0 con 0x7f2bc40b31d0 2022-08-01T19:34:16.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.602+0000 7f2bb2ffd700 1 -- 172.21.15.17:0/3176683640 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2bb401d910 con 0x7f2bc40b31d0 2022-08-01T19:34:16.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.602+0000 7f2bcafa6700 1 -- 172.21.15.17:0/3176683640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc40ad6f0 con 0x7f2bc40b31d0 2022-08-01T19:34:16.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.602+0000 7f2bcafa6700 1 -- 172.21.15.17:0/3176683640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc4066d90 con 0x7f2bc40b31d0 2022-08-01T19:34:16.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.602+0000 7f2bb2ffd700 1 -- 172.21.15.17:0/3176683640 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2bb401da70 con 0x7f2bc40b31d0 2022-08-01T19:34:16.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.606+0000 7f2bb2ffd700 1 --2- 172.21.15.17:0/3176683640 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2ba4053a10 0x7f2ba4055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:16.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.606+0000 7f2bb2ffd700 1 -- 172.21.15.17:0/3176683640 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f2bb4070240 con 0x7f2bc40b31d0 2022-08-01T19:34:16.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.606+0000 7f2bc9fa4700 1 --2- 172.21.15.17:0/3176683640 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2ba4053a10 0x7f2ba4055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:16.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.606+0000 7f2bc9fa4700 1 --2- 172.21.15.17:0/3176683640 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2ba4053a10 0x7f2ba4055ed0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f2bc000b000 tx=0x7f2bc0009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:16.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.614+0000 7f2bb2ffd700 1 -- 172.21.15.17:0/3176683640 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2bb4047000 con 0x7f2bc40b31d0 2022-08-01T19:34:16.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.734+0000 7f2bcafa6700 1 -- 172.21.15.17:0/3176683640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f2bc40574e0 con 0x7f2bc40b31d0 2022-08-01T19:34:16.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.738+0000 7f2bb2ffd700 1 -- 172.21.15.17:0/3176683640 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2bb4006980 con 0x7f2bc40b31d0 2022-08-01T19:34:16.743 INFO:teuthology.orchestra.run.smithi017.stdout:64424509445 2022-08-01T19:34:16.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.738+0000 7f2bcafa6700 1 -- 172.21.15.17:0/3176683640 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2ba4053a10 msgr2=0x7f2ba4055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:16.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.738+0000 7f2bcafa6700 1 --2- 172.21.15.17:0/3176683640 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2ba4053a10 0x7f2ba4055ed0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f2bc000b000 tx=0x7f2bc0009040).stop 2022-08-01T19:34:16.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.738+0000 7f2bcafa6700 1 -- 172.21.15.17:0/3176683640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2bc40b31d0 msgr2=0x7f2bc4151e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:16.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.738+0000 7f2bcafa6700 1 --2- 172.21.15.17:0/3176683640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2bc40b31d0 0x7f2bc4151e10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f2bb4002f70 tx=0x7f2bb4016000).stop 2022-08-01T19:34:16.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.738+0000 7f2bcafa6700 1 -- 172.21.15.17:0/3176683640 shutdown_connections 2022-08-01T19:34:16.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.738+0000 7f2bcafa6700 1 --2- 172.21.15.17:0/3176683640 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2ba4053a10 0x7f2ba4055ed0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:16.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.738+0000 7f2bcafa6700 1 --2- 172.21.15.17:0/3176683640 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bc41402d0 0x7f2bc40ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:16.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.738+0000 7f2bcafa6700 1 --2- 172.21.15.17:0/3176683640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2bc40b31d0 0x7f2bc4151e10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:16.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.738+0000 7f2bcafa6700 1 --2- 172.21.15.17:0/3176683640 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2bc40a9da0 0x7f2bc40608e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:16.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.738+0000 7f2bcafa6700 1 -- 172.21.15.17:0/3176683640 >> 172.21.15.17:0/3176683640 conn(0x7f2bc4063870 msgr2=0x7f2bc4140950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:16.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.738+0000 7f2bcafa6700 1 -- 172.21.15.17:0/3176683640 shutdown_connections 2022-08-01T19:34:16.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.738+0000 7f2bcafa6700 1 -- 172.21.15.17:0/3176683640 wait complete. 2022-08-01T19:34:16.755 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.4 2022-08-01T19:34:16.756 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2022-08-01T19:34:16.805 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:34:16.808 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:34:16.811 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:34:16.814 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:34:16.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.858+0000 7f1062812700 1 Processor -- start 2022-08-01T19:34:16.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.858+0000 7f1062812700 1 -- start start 2022-08-01T19:34:16.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.858+0000 7f1062812700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f105c066a50 0x7f105c0b2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:16.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.858+0000 7f1062812700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f105c05fe20 0x7f105c0b2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:16.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.858+0000 7f1062812700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f105c0b1700 con 0x7f105c0b0e60 2022-08-01T19:34:16.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.858+0000 7f1062812700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f105c066e30 con 0x7f105c066a50 2022-08-01T19:34:16.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.858+0000 7f1062812700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f105c0b1500 con 0x7f105c05fe20 2022-08-01T19:34:16.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.858+0000 7f106100f700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f105c0b0e60 0x7f105c0b6370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57024/0 (socket says 172.21.15.17:57024) 2022-08-01T19:34:16.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.858+0000 7f106100f700 1 -- 172.21.15.17:0/1632495080 learned_addr learned my addr 172.21.15.17:0/1632495080 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:16.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.858+0000 7f1062011700 1 --2- 172.21.15.17:0/1632495080 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f105c05fe20 0x7f105c0b2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:16.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.858+0000 7f1061810700 1 --2- 172.21.15.17:0/1632495080 >> v2:172.21.15.77:3300/0 conn(0x7f105c066a50 0x7f105c0b2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:16.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.858+0000 7f106080e700 1 -- 172.21.15.17:0/1632495080 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 528167973 0 0) 0x7f105c0b1700 con 0x7f105c0b0e60 2022-08-01T19:34:16.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.858+0000 7f106080e700 1 -- 172.21.15.17:0/1632495080 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1044003260 con 0x7f105c0b0e60 2022-08-01T19:34:16.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.858+0000 7f1062011700 1 -- 172.21.15.17:0/1632495080 >> v2:172.21.15.77:3300/0 conn(0x7f105c066a50 msgr2=0x7f105c0b2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:16.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.858+0000 7f1062011700 1 --2- 172.21.15.17:0/1632495080 >> v2:172.21.15.77:3300/0 conn(0x7f105c066a50 0x7f105c0b2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:16.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.858+0000 7f1062011700 1 -- 172.21.15.17:0/1632495080 >> v1:172.21.15.17:6789/0 conn(0x7f105c0b0e60 legacy=0x7f105c0b6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:16.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.858+0000 7f1062011700 1 -- 172.21.15.17:0/1632495080 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105c05f9c0 con 0x7f105c05fe20 2022-08-01T19:34:16.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.858+0000 7f1062011700 1 --2- 172.21.15.17:0/1632495080 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f105c05fe20 0x7f105c0b2820 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f1058002f70 tx=0x7f1058006df0).ready entity=mon.2 client_cookie=493dcae7cab5c4a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:16.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.858+0000 7f106080e700 1 -- 172.21.15.17:0/1632495080 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f105800d4a0 con 0x7f105c05fe20 2022-08-01T19:34:16.870 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.858+0000 7f106080e700 1 -- 172.21.15.17:0/1632495080 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1058000e90 con 0x7f105c05fe20 2022-08-01T19:34:16.870 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.858+0000 7f106080e700 1 -- 172.21.15.17:0/1632495080 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f105800d7f0 con 0x7f105c05fe20 2022-08-01T19:34:16.871 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.858+0000 7f1062812700 1 -- 172.21.15.17:0/1632495080 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f105c05fe20 msgr2=0x7f105c0b2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:16.871 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.858+0000 7f1062812700 1 --2- 172.21.15.17:0/1632495080 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f105c05fe20 0x7f105c0b2820 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f1058002f70 tx=0x7f1058006df0).stop 2022-08-01T19:34:16.871 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.858+0000 7f1062812700 1 -- 172.21.15.17:0/1632495080 shutdown_connections 2022-08-01T19:34:16.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.858+0000 7f1062812700 1 --2- 172.21.15.17:0/1632495080 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f105c05fe20 0x7f105c0b2820 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:16.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.858+0000 7f1062812700 1 --2- 172.21.15.17:0/1632495080 >> v2:172.21.15.77:3300/0 conn(0x7f105c066a50 0x7f105c0b2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:16.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.858+0000 7f1062812700 1 -- 172.21.15.17:0/1632495080 >> 172.21.15.17:0/1632495080 conn(0x7f105c063800 msgr2=0x7f105c063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:16.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.858+0000 7f1062812700 1 -- 172.21.15.17:0/1632495080 shutdown_connections 2022-08-01T19:34:16.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.858+0000 7f1062812700 1 -- 172.21.15.17:0/1632495080 wait complete. 2022-08-01T19:34:16.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.862+0000 7f1062812700 1 Processor -- start 2022-08-01T19:34:16.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.862+0000 7f1062812700 1 -- start start 2022-08-01T19:34:16.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.862+0000 7f1062812700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f105c05fe20 0x7f105c147d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:16.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.862+0000 7f1062812700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f105c066a50 0x7f105c0b4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:16.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.862+0000 7f1062812700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f105c0b0e60 0x7f105c0b4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:16.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.862+0000 7f1062812700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c0abd90 con 0x7f105c0b0e60 2022-08-01T19:34:16.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.862+0000 7f1062812700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c0abaf0 con 0x7f105c066a50 2022-08-01T19:34:16.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.862+0000 7f1062812700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f105c065c20 con 0x7f105c05fe20 2022-08-01T19:34:16.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.862+0000 7f1062011700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f105c0b0e60 0x7f105c0b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:16.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.862+0000 7f1062011700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f105c0b0e60 0x7f105c0b4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36606/0 (socket says 172.21.15.17:36606) 2022-08-01T19:34:16.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.862+0000 7f1062011700 1 -- 172.21.15.17:0/1728654726 learned_addr learned my addr 172.21.15.17:0/1728654726 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:16.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.862+0000 7f1061810700 1 --2- 172.21.15.17:0/1728654726 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f105c05fe20 0x7f105c147d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:16.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.862+0000 7f106100f700 1 --2- 172.21.15.17:0/1728654726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f105c066a50 0x7f105c0b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:16.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.862+0000 7f1062011700 1 -- 172.21.15.17:0/1728654726 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f105c05fe20 msgr2=0x7f105c147d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:16.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.862+0000 7f1062011700 1 --2- 172.21.15.17:0/1728654726 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f105c05fe20 0x7f105c147d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:16.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.862+0000 7f1062011700 1 -- 172.21.15.17:0/1728654726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f105c066a50 msgr2=0x7f105c0b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:16.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.862+0000 7f1062011700 1 --2- 172.21.15.17:0/1728654726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f105c066a50 0x7f105c0b4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:16.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.862+0000 7f1062011700 1 -- 172.21.15.17:0/1728654726 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105800b180 con 0x7f105c0b0e60 2022-08-01T19:34:16.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.862+0000 7f1062011700 1 --2- 172.21.15.17:0/1728654726 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f105c0b0e60 0x7f105c0b4550 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f1058002f70 tx=0x7f1058014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:16.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.862+0000 7f104affd700 1 -- 172.21.15.17:0/1728654726 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1058016070 con 0x7f105c0b0e60 2022-08-01T19:34:16.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.862+0000 7f104affd700 1 -- 172.21.15.17:0/1728654726 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10580137d0 con 0x7f105c0b0e60 2022-08-01T19:34:16.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.862+0000 7f104affd700 1 -- 172.21.15.17:0/1728654726 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1058013b30 con 0x7f105c0b0e60 2022-08-01T19:34:16.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.862+0000 7f1062812700 1 -- 172.21.15.17:0/1728654726 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f105c065ec0 con 0x7f105c0b0e60 2022-08-01T19:34:16.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.862+0000 7f1062812700 1 -- 172.21.15.17:0/1728654726 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f105c065710 con 0x7f105c0b0e60 2022-08-01T19:34:16.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.866+0000 7f1062812700 1 -- 172.21.15.17:0/1728654726 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f105c066050 con 0x7f105c0b0e60 2022-08-01T19:34:16.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.866+0000 7f104affd700 1 -- 172.21.15.17:0/1728654726 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1058025030 con 0x7f105c0b0e60 2022-08-01T19:34:16.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.866+0000 7f104affd700 1 --2- 172.21.15.17:0/1728654726 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f103c053b30 0x7f103c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:16.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.866+0000 7f104affd700 1 -- 172.21.15.17:0/1728654726 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f105806cff0 con 0x7f105c0b0e60 2022-08-01T19:34:16.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.866+0000 7f1061810700 1 --2- 172.21.15.17:0/1728654726 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f103c053b30 0x7f103c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:16.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.866+0000 7f1061810700 1 --2- 172.21.15.17:0/1728654726 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f103c053b30 0x7f103c055ff0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f105400b000 tx=0x7f10540091d0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:16.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.870+0000 7f104affd700 1 -- 172.21.15.17:0/1728654726 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1058043d00 con 0x7f105c0b0e60 2022-08-01T19:34:17.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.994+0000 7f1062812700 1 -- 172.21.15.17:0/1728654726 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f105c0574e0 con 0x7f105c0b0e60 2022-08-01T19:34:17.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.994+0000 7f104affd700 1 -- 172.21.15.17:0/1728654726 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f10580472e0 con 0x7f105c0b0e60 2022-08-01T19:34:17.002 INFO:teuthology.orchestra.run.smithi017.stdout:64424509445 2022-08-01T19:34:17.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.998+0000 7f1062812700 1 -- 172.21.15.17:0/1728654726 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f103c053b30 msgr2=0x7f103c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:17.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.998+0000 7f1062812700 1 --2- 172.21.15.17:0/1728654726 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f103c053b30 0x7f103c055ff0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f105400b000 tx=0x7f10540091d0).stop 2022-08-01T19:34:17.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.998+0000 7f1062812700 1 -- 172.21.15.17:0/1728654726 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f105c0b0e60 msgr2=0x7f105c0b4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:17.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.998+0000 7f1062812700 1 --2- 172.21.15.17:0/1728654726 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f105c0b0e60 0x7f105c0b4550 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f1058002f70 tx=0x7f1058014000).stop 2022-08-01T19:34:17.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.998+0000 7f1062812700 1 -- 172.21.15.17:0/1728654726 shutdown_connections 2022-08-01T19:34:17.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.998+0000 7f1062812700 1 --2- 172.21.15.17:0/1728654726 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f103c053b30 0x7f103c055ff0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:17.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.998+0000 7f1062812700 1 --2- 172.21.15.17:0/1728654726 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f105c0b0e60 0x7f105c0b4550 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:17.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.998+0000 7f1062812700 1 --2- 172.21.15.17:0/1728654726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f105c066a50 0x7f105c0b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:17.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.998+0000 7f1062812700 1 --2- 172.21.15.17:0/1728654726 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f105c05fe20 0x7f105c147d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:17.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.998+0000 7f1062812700 1 -- 172.21.15.17:0/1728654726 >> 172.21.15.17:0/1728654726 conn(0x7f105c063800 msgr2=0x7f105c0b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:17.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.998+0000 7f1062812700 1 -- 172.21.15.17:0/1728654726 shutdown_connections 2022-08-01T19:34:17.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:16.998+0000 7f1062812700 1 -- 172.21.15.17:0/1728654726 wait complete. 2022-08-01T19:34:17.014 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.5 2022-08-01T19:34:18.016 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2022-08-01T19:34:18.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.126+0000 7fb6e1301700 1 Processor -- start 2022-08-01T19:34:18.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.126+0000 7fb6e1301700 1 -- start start 2022-08-01T19:34:18.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.126+0000 7fb6e1301700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fb6dc0b2710 0x7fb6dc14afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:18.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.126+0000 7fb6e1301700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb6dc0b01a0 0x7fb6dc14b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:18.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.126+0000 7fb6e1301700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6dc0a7250 con 0x7fb6dc0afd60 2022-08-01T19:34:18.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.126+0000 7fb6e1301700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6dc0a7480 con 0x7fb6dc0b01a0 2022-08-01T19:34:18.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.126+0000 7fb6e1301700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6dc0a7720 con 0x7fb6dc0b2710 2022-08-01T19:34:18.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.126+0000 7fb6db7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb6dc0b01a0 0x7fb6dc14b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:18.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.126+0000 7fb6db7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb6dc0b01a0 0x7fb6dc14b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36610/0 (socket says 172.21.15.17:36610) 2022-08-01T19:34:18.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.126+0000 7fb6db7fe700 1 -- 172.21.15.17:0/1661267635 learned_addr learned my addr 172.21.15.17:0/1661267635 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:18.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.126+0000 7fb6dbfff700 1 --2- 172.21.15.17:0/1661267635 >> v2:172.21.15.77:3300/0 conn(0x7fb6dc0b2710 0x7fb6dc14afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:18.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.126+0000 7fb6db7fe700 1 -- 172.21.15.17:0/1661267635 >> v2:172.21.15.77:3300/0 conn(0x7fb6dc0b2710 msgr2=0x7fb6dc14afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:18.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.126+0000 7fb6db7fe700 1 --2- 172.21.15.17:0/1661267635 >> v2:172.21.15.77:3300/0 conn(0x7fb6dc0b2710 0x7fb6dc14afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:18.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.126+0000 7fb6db7fe700 1 -- 172.21.15.17:0/1661267635 >> v1:172.21.15.77:6789/0 conn(0x7fb6dc0afd60 legacy=0x7fb6dc0a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:18.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.126+0000 7fb6db7fe700 1 -- 172.21.15.17:0/1661267635 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6dc0a79c0 con 0x7fb6dc0b01a0 2022-08-01T19:34:18.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.126+0000 7fb6db7fe700 1 --2- 172.21.15.17:0/1661267635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6dc0b01a0 0x7fb6dc14b4c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fb6dc06b950 tx=0x7fb6cc009290).ready entity=mon.0 client_cookie=6d5fc4401f1d32ef server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:18.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.126+0000 7fb6daffd700 1 -- 172.21.15.17:0/1661267635 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6cc00b830 con 0x7fb6dc0b01a0 2022-08-01T19:34:18.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.126+0000 7fb6daffd700 1 -- 172.21.15.17:0/1661267635 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6cc00b990 con 0x7fb6dc0b01a0 2022-08-01T19:34:18.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.126+0000 7fb6daffd700 1 -- 172.21.15.17:0/1661267635 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6cc00bb40 con 0x7fb6dc0b01a0 2022-08-01T19:34:18.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.130+0000 7fb6e1301700 1 -- 172.21.15.17:0/1661267635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6dc0b01a0 msgr2=0x7fb6dc14b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:18.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.130+0000 7fb6e1301700 1 --2- 172.21.15.17:0/1661267635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6dc0b01a0 0x7fb6dc14b4c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fb6dc06b950 tx=0x7fb6cc009290).stop 2022-08-01T19:34:18.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.130+0000 7fb6e1301700 1 -- 172.21.15.17:0/1661267635 shutdown_connections 2022-08-01T19:34:18.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.130+0000 7fb6e1301700 1 --2- 172.21.15.17:0/1661267635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6dc0b01a0 0x7fb6dc14b4c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:18.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.130+0000 7fb6e1301700 1 --2- 172.21.15.17:0/1661267635 >> v2:172.21.15.77:3300/0 conn(0x7fb6dc0b2710 0x7fb6dc14afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:18.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.130+0000 7fb6e1301700 1 -- 172.21.15.17:0/1661267635 >> 172.21.15.17:0/1661267635 conn(0x7fb6dc060310 msgr2=0x7fb6dc0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:18.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.130+0000 7fb6e1301700 1 -- 172.21.15.17:0/1661267635 shutdown_connections 2022-08-01T19:34:18.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.130+0000 7fb6e1301700 1 -- 172.21.15.17:0/1661267635 wait complete. 2022-08-01T19:34:18.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.130+0000 7fb6e1301700 1 Processor -- start 2022-08-01T19:34:18.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.130+0000 7fb6e1301700 1 -- start start 2022-08-01T19:34:18.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.130+0000 7fb6e1301700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb6dc0afd60 0x7fb6dc149e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:18.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.130+0000 7fb6e1301700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6dc0b01a0 0x7fb6dc0abef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:18.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.130+0000 7fb6e1301700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6dc0b2710 0x7fb6dc0ac3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:18.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.130+0000 7fb6e1301700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6dc0ac8d0 con 0x7fb6dc0b2710 2022-08-01T19:34:18.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.130+0000 7fb6e1301700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6dc0aa8a0 con 0x7fb6dc0b01a0 2022-08-01T19:34:18.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.130+0000 7fb6e1301700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6dc0aab40 con 0x7fb6dc0afd60 2022-08-01T19:34:18.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.130+0000 7fb6e0b00700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6dc0b2710 0x7fb6dc0ac3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:18.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.130+0000 7fb6e0b00700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6dc0b2710 0x7fb6dc0ac3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36620/0 (socket says 172.21.15.17:36620) 2022-08-01T19:34:18.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.130+0000 7fb6e0b00700 1 -- 172.21.15.17:0/1805498036 learned_addr learned my addr 172.21.15.17:0/1805498036 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:18.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.130+0000 7fb6dbfff700 1 --2- 172.21.15.17:0/1805498036 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb6dc0afd60 0x7fb6dc149e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:18.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.130+0000 7fb6db7fe700 1 --2- 172.21.15.17:0/1805498036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6dc0b01a0 0x7fb6dc0abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:18.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.130+0000 7fb6e0b00700 1 -- 172.21.15.17:0/1805498036 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb6dc0afd60 msgr2=0x7fb6dc149e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:18.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.130+0000 7fb6e0b00700 1 --2- 172.21.15.17:0/1805498036 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb6dc0afd60 0x7fb6dc149e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:18.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.130+0000 7fb6e0b00700 1 -- 172.21.15.17:0/1805498036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6dc0b01a0 msgr2=0x7fb6dc0abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:18.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.130+0000 7fb6e0b00700 1 --2- 172.21.15.17:0/1805498036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6dc0b01a0 0x7fb6dc0abef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:18.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.130+0000 7fb6e0b00700 1 -- 172.21.15.17:0/1805498036 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6cc009530 con 0x7fb6dc0b2710 2022-08-01T19:34:18.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.134+0000 7fb6e0b00700 1 --2- 172.21.15.17:0/1805498036 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6dc0b2710 0x7fb6dc0ac3e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fb6d4000ed0 tx=0x7fb6d4007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:18.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.134+0000 7fb6d97fa700 1 -- 172.21.15.17:0/1805498036 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6d4007030 con 0x7fb6dc0b2710 2022-08-01T19:34:18.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.134+0000 7fb6d97fa700 1 -- 172.21.15.17:0/1805498036 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d4007190 con 0x7fb6dc0b2710 2022-08-01T19:34:18.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.134+0000 7fb6e1301700 1 -- 172.21.15.17:0/1805498036 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6dc065460 con 0x7fb6dc0b2710 2022-08-01T19:34:18.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.134+0000 7fb6d97fa700 1 -- 172.21.15.17:0/1805498036 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6d4006510 con 0x7fb6dc0b2710 2022-08-01T19:34:18.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.134+0000 7fb6e1301700 1 -- 172.21.15.17:0/1805498036 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6dc063330 con 0x7fb6dc0b2710 2022-08-01T19:34:18.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.134+0000 7fb6d97fa700 1 -- 172.21.15.17:0/1805498036 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb6d4010030 con 0x7fb6dc0b2710 2022-08-01T19:34:18.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.134+0000 7fb6e1301700 1 -- 172.21.15.17:0/1805498036 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6dc0ab580 con 0x7fb6dc0b2710 2022-08-01T19:34:18.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.134+0000 7fb6d97fa700 1 --2- 172.21.15.17:0/1805498036 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb6c0053b00 0x7fb6c0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:18.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.134+0000 7fb6d97fa700 1 -- 172.21.15.17:0/1805498036 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fb6d406bcb0 con 0x7fb6dc0b2710 2022-08-01T19:34:18.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.134+0000 7fb6dbfff700 1 --2- 172.21.15.17:0/1805498036 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb6c0053b00 0x7fb6c0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:18.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.138+0000 7fb6dbfff700 1 --2- 172.21.15.17:0/1805498036 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb6c0053b00 0x7fb6c0055fc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fb6c400c000 tx=0x7fb6c4008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:18.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.138+0000 7fb6d97fa700 1 -- 172.21.15.17:0/1805498036 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb6dc0ab580 con 0x7fb6dc0b2710 2022-08-01T19:34:18.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.266+0000 7fb6e1301700 1 -- 172.21.15.17:0/1805498036 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb6dc0ab580 con 0x7fb6dc0b2710 2022-08-01T19:34:18.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.266+0000 7fb6d97fa700 1 -- 172.21.15.17:0/1805498036 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb6dc0ab580 con 0x7fb6dc0b2710 2022-08-01T19:34:18.275 INFO:teuthology.orchestra.run.smithi017.stdout:64424509446 2022-08-01T19:34:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.270+0000 7fb6e1301700 1 -- 172.21.15.17:0/1805498036 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb6c0053b00 msgr2=0x7fb6c0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.270+0000 7fb6e1301700 1 --2- 172.21.15.17:0/1805498036 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb6c0053b00 0x7fb6c0055fc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fb6c400c000 tx=0x7fb6c4008040).stop 2022-08-01T19:34:18.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.270+0000 7fb6e1301700 1 -- 172.21.15.17:0/1805498036 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6dc0b2710 msgr2=0x7fb6dc0ac3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:18.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.270+0000 7fb6e1301700 1 --2- 172.21.15.17:0/1805498036 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6dc0b2710 0x7fb6dc0ac3e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fb6d4000ed0 tx=0x7fb6d4007bd0).stop 2022-08-01T19:34:18.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.270+0000 7fb6e1301700 1 -- 172.21.15.17:0/1805498036 shutdown_connections 2022-08-01T19:34:18.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.270+0000 7fb6e1301700 1 --2- 172.21.15.17:0/1805498036 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb6c0053b00 0x7fb6c0055fc0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:18.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.270+0000 7fb6e1301700 1 --2- 172.21.15.17:0/1805498036 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6dc0b2710 0x7fb6dc0ac3e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:18.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.270+0000 7fb6e1301700 1 --2- 172.21.15.17:0/1805498036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6dc0b01a0 0x7fb6dc0abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:18.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.270+0000 7fb6e1301700 1 --2- 172.21.15.17:0/1805498036 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb6dc0afd60 0x7fb6dc149e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:18.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.270+0000 7fb6e1301700 1 -- 172.21.15.17:0/1805498036 >> 172.21.15.17:0/1805498036 conn(0x7fb6dc060310 msgr2=0x7fb6dc05be30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:18.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.270+0000 7fb6e1301700 1 -- 172.21.15.17:0/1805498036 shutdown_connections 2022-08-01T19:34:18.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.270+0000 7fb6e1301700 1 -- 172.21.15.17:0/1805498036 wait complete. 2022-08-01T19:34:18.287 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.5 2022-08-01T19:34:18.288 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2022-08-01T19:34:18.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.390+0000 7fd742c59700 1 Processor -- start 2022-08-01T19:34:18.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.390+0000 7fd742c59700 1 -- start start 2022-08-01T19:34:18.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.390+0000 7fd742c59700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd73c0b6710 0x7fd73c0b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:18.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.390+0000 7fd742c59700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd73c144240 con 0x7fd73c0b6af0 2022-08-01T19:34:18.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.390+0000 7fd742c59700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd73c143d00 con 0x7fd73c0b4550 2022-08-01T19:34:18.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.390+0000 7fd742c59700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c143fa0 con 0x7fd73c0b6710 2022-08-01T19:34:18.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.390+0000 7fd7411f6700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd73c0b6af0 0x7fd73c12bb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57054/0 (socket says 172.21.15.17:57054) 2022-08-01T19:34:18.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.390+0000 7fd7411f6700 1 -- 172.21.15.17:0/1551641370 learned_addr learned my addr 172.21.15.17:0/1551641370 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:18.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.390+0000 7fd7409f5700 1 --2- 172.21.15.17:0/1551641370 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd73c0b6710 0x7fd73c0b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:18.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.394+0000 7fd73b7fe700 1 -- 172.21.15.17:0/1551641370 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3273698049 0 0) 0x7fd73c144240 con 0x7fd73c0b6af0 2022-08-01T19:34:18.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.394+0000 7fd7409f5700 1 -- 172.21.15.17:0/1551641370 >> v1:172.21.15.77:6789/0 conn(0x7fd73c0b4550 legacy=0x7fd73c061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:18.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.394+0000 7fd7409f5700 1 -- 172.21.15.17:0/1551641370 >> v1:172.21.15.17:6789/0 conn(0x7fd73c0b6af0 legacy=0x7fd73c12bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:18.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.394+0000 7fd7409f5700 1 -- 172.21.15.17:0/1551641370 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd73c061780 con 0x7fd73c0b6710 2022-08-01T19:34:18.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.394+0000 7fd7409f5700 1 --2- 172.21.15.17:0/1551641370 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd73c0b6710 0x7fd73c0b5d80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fd724001110 tx=0x7fd72c006df0).ready entity=mon.2 client_cookie=e118b3c43524fbaf server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:18.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.394+0000 7fd73b7fe700 1 -- 172.21.15.17:0/1551641370 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd72c00d570 con 0x7fd73c0b6710 2022-08-01T19:34:18.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.394+0000 7fd73b7fe700 1 -- 172.21.15.17:0/1551641370 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd72c000e90 con 0x7fd73c0b6710 2022-08-01T19:34:18.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.394+0000 7fd73b7fe700 1 -- 172.21.15.17:0/1551641370 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd72c00d8c0 con 0x7fd73c0b6710 2022-08-01T19:34:18.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.394+0000 7fd742c59700 1 -- 172.21.15.17:0/1551641370 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd73c0b6710 msgr2=0x7fd73c0b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:18.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.394+0000 7fd742c59700 1 --2- 172.21.15.17:0/1551641370 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd73c0b6710 0x7fd73c0b5d80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fd724001110 tx=0x7fd72c006df0).stop 2022-08-01T19:34:18.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.394+0000 7fd742c59700 1 -- 172.21.15.17:0/1551641370 shutdown_connections 2022-08-01T19:34:18.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.394+0000 7fd742c59700 1 --2- 172.21.15.17:0/1551641370 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd73c0b6710 0x7fd73c0b5d80 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:18.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.394+0000 7fd742c59700 1 -- 172.21.15.17:0/1551641370 >> 172.21.15.17:0/1551641370 conn(0x7fd73c05c620 msgr2=0x7fd73c05ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:18.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.394+0000 7fd742c59700 1 -- 172.21.15.17:0/1551641370 shutdown_connections 2022-08-01T19:34:18.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.394+0000 7fd742c59700 1 -- 172.21.15.17:0/1551641370 wait complete. 2022-08-01T19:34:18.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.394+0000 7fd742c59700 1 Processor -- start 2022-08-01T19:34:18.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.394+0000 7fd742c59700 1 -- start start 2022-08-01T19:34:18.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.398+0000 7fd742c59700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd73c0b4550 0x7fd73c0ba3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:18.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.398+0000 7fd742c59700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd73c0b6710 0x7fd73c0b93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:18.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.398+0000 7fd742c59700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd73c0b6af0 0x7fd73c0b9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:18.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.398+0000 7fd742c59700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c0af760 con 0x7fd73c0b6710 2022-08-01T19:34:18.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.398+0000 7fd742c59700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c0b9d80 con 0x7fd73c0b6af0 2022-08-01T19:34:18.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.398+0000 7fd742c59700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c0af140 con 0x7fd73c0b4550 2022-08-01T19:34:18.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.398+0000 7fd73bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd73c0b6710 0x7fd73c0b93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:18.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.398+0000 7fd73bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd73c0b6710 0x7fd73c0b93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36634/0 (socket says 172.21.15.17:36634) 2022-08-01T19:34:18.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.398+0000 7fd7409f5700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd73c0b4550 0x7fd73c0ba3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:18.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.398+0000 7fd73bfff700 1 -- 172.21.15.17:0/3814705842 learned_addr learned my addr 172.21.15.17:0/3814705842 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:18.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.398+0000 7fd7411f6700 1 --2- 172.21.15.17:0/3814705842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd73c0b6af0 0x7fd73c0b9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:18.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.398+0000 7fd7409f5700 1 -- 172.21.15.17:0/3814705842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd73c0b6af0 msgr2=0x7fd73c0b9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:18.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.398+0000 7fd7409f5700 1 --2- 172.21.15.17:0/3814705842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd73c0b6af0 0x7fd73c0b9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:18.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.398+0000 7fd7409f5700 1 -- 172.21.15.17:0/3814705842 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd73c0b6710 msgr2=0x7fd73c0b93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:18.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.398+0000 7fd7409f5700 1 --2- 172.21.15.17:0/3814705842 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd73c0b6710 0x7fd73c0b93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:18.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.398+0000 7fd7409f5700 1 -- 172.21.15.17:0/3814705842 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd72c00b180 con 0x7fd73c0b4550 2022-08-01T19:34:18.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.398+0000 7fd73bfff700 1 --2- 172.21.15.17:0/3814705842 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd73c0b6710 0x7fd73c0b93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:34:18.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.398+0000 7fd7409f5700 1 --2- 172.21.15.17:0/3814705842 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd73c0b4550 0x7fd73c0ba3e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fd724001110 tx=0x7fd72c014000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:18.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.398+0000 7fd739ffb700 1 -- 172.21.15.17:0/3814705842 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd72c016070 con 0x7fd73c0b4550 2022-08-01T19:34:18.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.398+0000 7fd739ffb700 1 -- 172.21.15.17:0/3814705842 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd72c013900 con 0x7fd73c0b4550 2022-08-01T19:34:18.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.398+0000 7fd742c59700 1 -- 172.21.15.17:0/3814705842 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd73c12bea0 con 0x7fd73c0b4550 2022-08-01T19:34:18.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.398+0000 7fd739ffb700 1 -- 172.21.15.17:0/3814705842 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd72c013c60 con 0x7fd73c0b4550 2022-08-01T19:34:18.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.398+0000 7fd742c59700 1 -- 172.21.15.17:0/3814705842 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd73c0af4c0 con 0x7fd73c0b4550 2022-08-01T19:34:18.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.402+0000 7fd742c59700 1 -- 172.21.15.17:0/3814705842 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd73c061a20 con 0x7fd73c0b4550 2022-08-01T19:34:18.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.402+0000 7fd739ffb700 1 -- 172.21.15.17:0/3814705842 <== mon.2 v2:172.21.15.77:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd72c00f030 con 0x7fd73c0b4550 2022-08-01T19:34:18.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.402+0000 7fd739ffb700 1 --2- 172.21.15.17:0/3814705842 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd7200539e0 0x7fd720055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:18.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.402+0000 7fd739ffb700 1 -- 172.21.15.17:0/3814705842 <== mon.2 v2:172.21.15.77:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fd72c06cce0 con 0x7fd73c0b4550 2022-08-01T19:34:18.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.402+0000 7fd73bfff700 1 --2- 172.21.15.17:0/3814705842 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd7200539e0 0x7fd720055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:18.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.402+0000 7fd73bfff700 1 --2- 172.21.15.17:0/3814705842 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd7200539e0 0x7fd720055ea0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fd72400d000 tx=0x7fd724009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:18.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.414+0000 7fd739ffb700 1 -- 172.21.15.17:0/3814705842 <== mon.2 v2:172.21.15.77:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd72c043a00 con 0x7fd73c0b4550 2022-08-01T19:34:18.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.538+0000 7fd742c59700 1 -- 172.21.15.17:0/3814705842 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fd73c0574e0 con 0x7fd73c0b4550 2022-08-01T19:34:18.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.542+0000 7fd739ffb700 1 -- 172.21.15.17:0/3814705842 <== mon.2 v2:172.21.15.77:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd72c023070 con 0x7fd73c0b4550 2022-08-01T19:34:18.546 INFO:teuthology.orchestra.run.smithi017.stdout:60129542149 2022-08-01T19:34:18.550 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.542+0000 7fd742c59700 1 -- 172.21.15.17:0/3814705842 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd7200539e0 msgr2=0x7fd720055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:18.550 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.542+0000 7fd742c59700 1 --2- 172.21.15.17:0/3814705842 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd7200539e0 0x7fd720055ea0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fd72400d000 tx=0x7fd724009040).stop 2022-08-01T19:34:18.550 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.542+0000 7fd742c59700 1 -- 172.21.15.17:0/3814705842 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd73c0b4550 msgr2=0x7fd73c0ba3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:18.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.542+0000 7fd742c59700 1 --2- 172.21.15.17:0/3814705842 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd73c0b4550 0x7fd73c0ba3e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fd724001110 tx=0x7fd72c014000).stop 2022-08-01T19:34:18.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.542+0000 7fd742c59700 1 -- 172.21.15.17:0/3814705842 shutdown_connections 2022-08-01T19:34:18.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.542+0000 7fd742c59700 1 --2- 172.21.15.17:0/3814705842 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd7200539e0 0x7fd720055ea0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:18.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.542+0000 7fd742c59700 1 --2- 172.21.15.17:0/3814705842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd73c0b6af0 0x7fd73c0b9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:18.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.542+0000 7fd742c59700 1 --2- 172.21.15.17:0/3814705842 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd73c0b6710 0x7fd73c0b93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:18.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.542+0000 7fd742c59700 1 --2- 172.21.15.17:0/3814705842 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd73c0b4550 0x7fd73c0ba3e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:18.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.542+0000 7fd742c59700 1 -- 172.21.15.17:0/3814705842 >> 172.21.15.17:0/3814705842 conn(0x7fd73c05c620 msgr2=0x7fd73c061de0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:18.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.542+0000 7fd742c59700 1 -- 172.21.15.17:0/3814705842 shutdown_connections 2022-08-01T19:34:18.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.542+0000 7fd742c59700 1 -- 172.21.15.17:0/3814705842 wait complete. 2022-08-01T19:34:18.564 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.6 2022-08-01T19:34:18.564 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2022-08-01T19:34:18.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.662+0000 7fb973699700 1 Processor -- start 2022-08-01T19:34:18.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.662+0000 7fb973699700 1 -- start start 2022-08-01T19:34:18.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.662+0000 7fb973699700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb96c139c70 0x7fb96c142f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:18.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.662+0000 7fb973699700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fb96c143430 0x7fb96c0ab6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:18.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.662+0000 7fb973699700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb96c0b1de0 0x7fb96c0abbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:18.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.662+0000 7fb973699700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb96c0b31a0 con 0x7fb96c0b1de0 2022-08-01T19:34:18.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.662+0000 7fb973699700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb96c0af4b0 con 0x7fb96c143430 2022-08-01T19:34:18.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.662+0000 7fb973699700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c0ac600 con 0x7fb96c139c70 2022-08-01T19:34:18.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.662+0000 7fb971c36700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb96c0b1de0 0x7fb96c0abbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:18.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.662+0000 7fb971c36700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb96c0b1de0 0x7fb96c0abbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36638/0 (socket says 172.21.15.17:36638) 2022-08-01T19:34:18.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.662+0000 7fb971c36700 1 -- 172.21.15.17:0/3600306893 learned_addr learned my addr 172.21.15.17:0/3600306893 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:18.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.662+0000 7fb970c34700 1 --2- 172.21.15.17:0/3600306893 >> v2:172.21.15.77:3300/0 conn(0x7fb96c143430 0x7fb96c0ab6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:18.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.662+0000 7fb971435700 1 --2- 172.21.15.17:0/3600306893 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb96c139c70 0x7fb96c142f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:18.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.666+0000 7fb971c36700 1 -- 172.21.15.17:0/3600306893 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb96c139c70 msgr2=0x7fb96c142f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:18.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.666+0000 7fb971c36700 1 --2- 172.21.15.17:0/3600306893 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb96c139c70 0x7fb96c142f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:18.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.666+0000 7fb971c36700 1 -- 172.21.15.17:0/3600306893 >> v2:172.21.15.77:3300/0 conn(0x7fb96c143430 msgr2=0x7fb96c0ab6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:18.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.666+0000 7fb971c36700 1 --2- 172.21.15.17:0/3600306893 >> v2:172.21.15.77:3300/0 conn(0x7fb96c143430 0x7fb96c0ab6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:18.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.666+0000 7fb971c36700 1 -- 172.21.15.17:0/3600306893 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb96c1438c0 con 0x7fb96c0b1de0 2022-08-01T19:34:18.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.666+0000 7fb971c36700 1 --2- 172.21.15.17:0/3600306893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb96c0b1de0 0x7fb96c0abbb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7fb96c060480 tx=0x7fb964009290).ready entity=mon.0 client_cookie=4953adff843b5c2b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:18.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.666+0000 7fb957fff700 1 -- 172.21.15.17:0/3600306893 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb96400b680 con 0x7fb96c0b1de0 2022-08-01T19:34:18.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.666+0000 7fb957fff700 1 -- 172.21.15.17:0/3600306893 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb96400b7e0 con 0x7fb96c0b1de0 2022-08-01T19:34:18.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.666+0000 7fb957fff700 1 -- 172.21.15.17:0/3600306893 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb96400b990 con 0x7fb96c0b1de0 2022-08-01T19:34:18.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.666+0000 7fb973699700 1 -- 172.21.15.17:0/3600306893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb96c0b1de0 msgr2=0x7fb96c0abbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:18.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.666+0000 7fb973699700 1 --2- 172.21.15.17:0/3600306893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb96c0b1de0 0x7fb96c0abbb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7fb96c060480 tx=0x7fb964009290).stop 2022-08-01T19:34:18.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.666+0000 7fb973699700 1 -- 172.21.15.17:0/3600306893 shutdown_connections 2022-08-01T19:34:18.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.666+0000 7fb973699700 1 --2- 172.21.15.17:0/3600306893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb96c0b1de0 0x7fb96c0abbb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:18.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.666+0000 7fb973699700 1 --2- 172.21.15.17:0/3600306893 >> v2:172.21.15.77:3300/0 conn(0x7fb96c143430 0x7fb96c0ab6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:18.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.666+0000 7fb973699700 1 --2- 172.21.15.17:0/3600306893 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb96c139c70 0x7fb96c142f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:18.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.666+0000 7fb973699700 1 -- 172.21.15.17:0/3600306893 >> 172.21.15.17:0/3600306893 conn(0x7fb96c05f840 msgr2=0x7fb96c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:18.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.666+0000 7fb973699700 1 -- 172.21.15.17:0/3600306893 shutdown_connections 2022-08-01T19:34:18.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.666+0000 7fb973699700 1 -- 172.21.15.17:0/3600306893 wait complete. 2022-08-01T19:34:18.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.666+0000 7fb973699700 1 Processor -- start 2022-08-01T19:34:18.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.666+0000 7fb973699700 1 -- start start 2022-08-01T19:34:18.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.670+0000 7fb973699700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb96c0b1de0 0x7fb96c14f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:18.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.670+0000 7fb973699700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb96c13d810 0x7fb96c0ab540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:18.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.670+0000 7fb973699700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb96c1466d0 0x7fb96c13fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:18.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.670+0000 7fb973699700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c136c20 con 0x7fb96c1466d0 2022-08-01T19:34:18.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.670+0000 7fb973699700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c1504a0 con 0x7fb96c0b1de0 2022-08-01T19:34:18.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.670+0000 7fb973699700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c153600 con 0x7fb96c13d810 2022-08-01T19:34:18.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.670+0000 7fb971c36700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb96c1466d0 0x7fb96c13fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:18.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.670+0000 7fb971c36700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb96c1466d0 0x7fb96c13fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36646/0 (socket says 172.21.15.17:36646) 2022-08-01T19:34:18.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.670+0000 7fb971435700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb96c0b1de0 0x7fb96c14f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:18.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.670+0000 7fb970c34700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb96c13d810 0x7fb96c0ab540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:18.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.670+0000 7fb971c36700 1 -- 172.21.15.17:0/1185806518 learned_addr learned my addr 172.21.15.17:0/1185806518 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:18.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.670+0000 7fb971c36700 1 -- 172.21.15.17:0/1185806518 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb96c13d810 msgr2=0x7fb96c0ab540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:18.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.670+0000 7fb971c36700 1 --2- 172.21.15.17:0/1185806518 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb96c13d810 0x7fb96c0ab540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:18.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.670+0000 7fb971c36700 1 -- 172.21.15.17:0/1185806518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb96c0b1de0 msgr2=0x7fb96c14f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:18.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.670+0000 7fb971c36700 1 --2- 172.21.15.17:0/1185806518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb96c0b1de0 0x7fb96c14f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:18.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.670+0000 7fb971c36700 1 -- 172.21.15.17:0/1185806518 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb964009530 con 0x7fb96c1466d0 2022-08-01T19:34:18.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.670+0000 7fb970c34700 1 --2- 172.21.15.17:0/1185806518 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb96c13d810 0x7fb96c0ab540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:34:18.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.670+0000 7fb971c36700 1 --2- 172.21.15.17:0/1185806518 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb96c1466d0 0x7fb96c13fea0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7fb96c060480 tx=0x7fb964016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:18.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.670+0000 7fb95effd700 1 -- 172.21.15.17:0/1185806518 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb96400b680 con 0x7fb96c1466d0 2022-08-01T19:34:18.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.670+0000 7fb95effd700 1 -- 172.21.15.17:0/1185806518 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb964021670 con 0x7fb96c1466d0 2022-08-01T19:34:18.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.670+0000 7fb973699700 1 -- 172.21.15.17:0/1185806518 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb96c1538a0 con 0x7fb96c1466d0 2022-08-01T19:34:18.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.670+0000 7fb95effd700 1 -- 172.21.15.17:0/1185806518 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9640219d0 con 0x7fb96c1466d0 2022-08-01T19:34:18.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.670+0000 7fb973699700 1 -- 172.21.15.17:0/1185806518 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb96c134c70 con 0x7fb96c1466d0 2022-08-01T19:34:18.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.674+0000 7fb973699700 1 -- 172.21.15.17:0/1185806518 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb96c153430 con 0x7fb96c1466d0 2022-08-01T19:34:18.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.674+0000 7fb95effd700 1 -- 172.21.15.17:0/1185806518 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb96400f070 con 0x7fb96c1466d0 2022-08-01T19:34:18.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.674+0000 7fb95effd700 1 --2- 172.21.15.17:0/1185806518 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb9480539e0 0x7fb948055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:18.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.674+0000 7fb95effd700 1 -- 172.21.15.17:0/1185806518 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fb96406d320 con 0x7fb96c1466d0 2022-08-01T19:34:18.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.674+0000 7fb971435700 1 --2- 172.21.15.17:0/1185806518 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb9480539e0 0x7fb948055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:18.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.674+0000 7fb971435700 1 --2- 172.21.15.17:0/1185806518 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb9480539e0 0x7fb948055ea0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fb96800b000 tx=0x7fb968006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:18.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.682+0000 7fb95effd700 1 -- 172.21.15.17:0/1185806518 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb964043fd0 con 0x7fb96c1466d0 2022-08-01T19:34:18.815 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.810+0000 7fb973699700 1 -- 172.21.15.17:0/1185806518 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fb96c143b60 con 0x7fb96c1466d0 2022-08-01T19:34:18.816 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.810+0000 7fb95effd700 1 -- 172.21.15.17:0/1185806518 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb964015020 con 0x7fb96c1466d0 2022-08-01T19:34:18.817 INFO:teuthology.orchestra.run.smithi017.stdout:60129542150 2022-08-01T19:34:18.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.814+0000 7fb973699700 1 -- 172.21.15.17:0/1185806518 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb9480539e0 msgr2=0x7fb948055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:18.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.814+0000 7fb973699700 1 --2- 172.21.15.17:0/1185806518 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb9480539e0 0x7fb948055ea0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fb96800b000 tx=0x7fb968006df0).stop 2022-08-01T19:34:18.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.814+0000 7fb973699700 1 -- 172.21.15.17:0/1185806518 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb96c1466d0 msgr2=0x7fb96c13fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:18.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.814+0000 7fb973699700 1 --2- 172.21.15.17:0/1185806518 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb96c1466d0 0x7fb96c13fea0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7fb96c060480 tx=0x7fb964016000).stop 2022-08-01T19:34:18.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.814+0000 7fb973699700 1 -- 172.21.15.17:0/1185806518 shutdown_connections 2022-08-01T19:34:18.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.814+0000 7fb973699700 1 --2- 172.21.15.17:0/1185806518 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb9480539e0 0x7fb948055ea0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:18.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.814+0000 7fb973699700 1 --2- 172.21.15.17:0/1185806518 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb96c1466d0 0x7fb96c13fea0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:18.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.814+0000 7fb973699700 1 --2- 172.21.15.17:0/1185806518 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb96c13d810 0x7fb96c0ab540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:18.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.814+0000 7fb973699700 1 --2- 172.21.15.17:0/1185806518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb96c0b1de0 0x7fb96c14f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:18.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.814+0000 7fb973699700 1 -- 172.21.15.17:0/1185806518 >> 172.21.15.17:0/1185806518 conn(0x7fb96c05f840 msgr2=0x7fb96c0b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:18.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.814+0000 7fb973699700 1 -- 172.21.15.17:0/1185806518 shutdown_connections 2022-08-01T19:34:18.822 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.814+0000 7fb973699700 1 -- 172.21.15.17:0/1185806518 wait complete. 2022-08-01T19:34:18.834 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.7 2022-08-01T19:34:18.834 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2022-08-01T19:34:18.835 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2022-08-01T19:34:18.937 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.930+0000 7fe46b59e700 1 Processor -- start 2022-08-01T19:34:18.937 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.930+0000 7fe46b59e700 1 -- start start 2022-08-01T19:34:18.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.930+0000 7fe46b59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe46c0b36e0 0x7fe46c141a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:18.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.930+0000 7fe46b59e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fe46c144e60 0x7fe46c066ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:18.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.930+0000 7fe46b59e700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe46c064ef0 con 0x7fe46c064b10 2022-08-01T19:34:18.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.930+0000 7fe46b59e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe46c13ebf0 con 0x7fe46c144e60 2022-08-01T19:34:18.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.930+0000 7fe46b59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c1452d0 con 0x7fe46c0b36e0 2022-08-01T19:34:18.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.930+0000 7fe46a59c700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe46c064b10 0x7fe46c141370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57076/0 (socket says 172.21.15.17:57076) 2022-08-01T19:34:18.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.930+0000 7fe46a59c700 1 -- 172.21.15.17:0/4066732233 learned_addr learned my addr 172.21.15.17:0/4066732233 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:18.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.934+0000 7fe469d9b700 1 --2- 172.21.15.17:0/4066732233 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe46c0b36e0 0x7fe46c141a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:18.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.934+0000 7fe46ad9d700 1 --2- 172.21.15.17:0/4066732233 >> v2:172.21.15.77:3300/0 conn(0x7fe46c144e60 0x7fe46c066ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:18.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.934+0000 7fe46959a700 1 -- 172.21.15.17:0/4066732233 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3688645179 0 0) 0x7fe46c064ef0 con 0x7fe46c064b10 2022-08-01T19:34:18.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.934+0000 7fe46959a700 1 -- 172.21.15.17:0/4066732233 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe44c003260 con 0x7fe46c064b10 2022-08-01T19:34:18.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.934+0000 7fe46959a700 1 -- 172.21.15.17:0/4066732233 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4005378341 0 0) 0x7fe44c003260 con 0x7fe46c064b10 2022-08-01T19:34:18.944 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.934+0000 7fe469d9b700 1 -- 172.21.15.17:0/4066732233 >> v2:172.21.15.77:3300/0 conn(0x7fe46c144e60 msgr2=0x7fe46c066ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:18.944 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.934+0000 7fe469d9b700 1 --2- 172.21.15.17:0/4066732233 >> v2:172.21.15.77:3300/0 conn(0x7fe46c144e60 0x7fe46c066ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:18.944 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.934+0000 7fe469d9b700 1 -- 172.21.15.17:0/4066732233 >> v1:172.21.15.17:6789/0 conn(0x7fe46c064b10 legacy=0x7fe46c141370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:18.945 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.934+0000 7fe469d9b700 1 -- 172.21.15.17:0/4066732233 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46c066690 con 0x7fe46c0b36e0 2022-08-01T19:34:18.945 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.934+0000 7fe469d9b700 1 --2- 172.21.15.17:0/4066732233 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe46c0b36e0 0x7fe46c141a80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fe454001820 tx=0x7fe45c0091d0).ready entity=mon.2 client_cookie=a6ce9367ed278c5b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:18.945 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.934+0000 7fe46959a700 1 -- 172.21.15.17:0/4066732233 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe454001680 con 0x7fe46c0b36e0 2022-08-01T19:34:18.945 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.934+0000 7fe46959a700 1 -- 172.21.15.17:0/4066732233 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe45c00f510 con 0x7fe46c0b36e0 2022-08-01T19:34:18.946 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.934+0000 7fe46959a700 1 -- 172.21.15.17:0/4066732233 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe45c00f680 con 0x7fe46c0b36e0 2022-08-01T19:34:18.946 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.934+0000 7fe46b59e700 1 -- 172.21.15.17:0/4066732233 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe46c0b36e0 msgr2=0x7fe46c141a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:18.946 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.934+0000 7fe46b59e700 1 --2- 172.21.15.17:0/4066732233 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe46c0b36e0 0x7fe46c141a80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fe454001820 tx=0x7fe45c0091d0).stop 2022-08-01T19:34:18.946 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.934+0000 7fe46b59e700 1 -- 172.21.15.17:0/4066732233 shutdown_connections 2022-08-01T19:34:18.946 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.934+0000 7fe46b59e700 1 --2- 172.21.15.17:0/4066732233 >> v2:172.21.15.77:3300/0 conn(0x7fe46c144e60 0x7fe46c066ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:18.947 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.934+0000 7fe46b59e700 1 --2- 172.21.15.17:0/4066732233 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe46c0b36e0 0x7fe46c141a80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:18.947 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.934+0000 7fe46b59e700 1 -- 172.21.15.17:0/4066732233 >> 172.21.15.17:0/4066732233 conn(0x7fe46c05f840 msgr2=0x7fe46c05fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:18.947 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.934+0000 7fe46b59e700 1 -- 172.21.15.17:0/4066732233 shutdown_connections 2022-08-01T19:34:18.947 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.934+0000 7fe46b59e700 1 -- 172.21.15.17:0/4066732233 wait complete. 2022-08-01T19:34:18.951 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.938+0000 7fe46b59e700 1 Processor -- start 2022-08-01T19:34:18.951 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.938+0000 7fe46b59e700 1 -- start start 2022-08-01T19:34:18.952 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.938+0000 7fe46b59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe46c064b10 0x7fe46c0b2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:18.952 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.938+0000 7fe46b59e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe46c0b36e0 0x7fe46c13b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:18.952 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.938+0000 7fe46b59e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe46c144e60 0x7fe46c0ae860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:18.952 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.938+0000 7fe46b59e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c140090 con 0x7fe46c0b36e0 2022-08-01T19:34:18.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.938+0000 7fe46b59e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c0b1280 con 0x7fe46c144e60 2022-08-01T19:34:18.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.938+0000 7fe46b59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c0b0340 con 0x7fe46c064b10 2022-08-01T19:34:18.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.938+0000 7fe469d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe46c0b36e0 0x7fe46c13b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:18.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.938+0000 7fe469d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe46c0b36e0 0x7fe46c13b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36658/0 (socket says 172.21.15.17:36658) 2022-08-01T19:34:18.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.938+0000 7fe469d9b700 1 -- 172.21.15.17:0/169486966 learned_addr learned my addr 172.21.15.17:0/169486966 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:18.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.938+0000 7fe46ad9d700 1 --2- 172.21.15.17:0/169486966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe46c144e60 0x7fe46c0ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:18.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.938+0000 7fe46a59c700 1 --2- 172.21.15.17:0/169486966 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe46c064b10 0x7fe46c0b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:18.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.938+0000 7fe469d9b700 1 -- 172.21.15.17:0/169486966 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe46c064b10 msgr2=0x7fe46c0b2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:18.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.938+0000 7fe469d9b700 1 --2- 172.21.15.17:0/169486966 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe46c064b10 0x7fe46c0b2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:18.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.938+0000 7fe469d9b700 1 -- 172.21.15.17:0/169486966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe46c144e60 msgr2=0x7fe46c0ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:18.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.938+0000 7fe469d9b700 1 --2- 172.21.15.17:0/169486966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe46c144e60 0x7fe46c0ae860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:18.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.938+0000 7fe469d9b700 1 -- 172.21.15.17:0/169486966 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe45c009470 con 0x7fe46c0b36e0 2022-08-01T19:34:18.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.938+0000 7fe469d9b700 1 --2- 172.21.15.17:0/169486966 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe46c0b36e0 0x7fe46c13b600 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fe454001820 tx=0x7fe45c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:18.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.938+0000 7fe4637fe700 1 -- 172.21.15.17:0/169486966 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe45c00f680 con 0x7fe46c0b36e0 2022-08-01T19:34:18.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.938+0000 7fe4637fe700 1 -- 172.21.15.17:0/169486966 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe45c0032f0 con 0x7fe46c0b36e0 2022-08-01T19:34:18.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.938+0000 7fe46b59e700 1 -- 172.21.15.17:0/169486966 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe46c0b0fe0 con 0x7fe46c0b36e0 2022-08-01T19:34:18.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.938+0000 7fe4637fe700 1 -- 172.21.15.17:0/169486966 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe45c021670 con 0x7fe46c0b36e0 2022-08-01T19:34:18.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.938+0000 7fe46b59e700 1 -- 172.21.15.17:0/169486966 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe46c0b1400 con 0x7fe46c0b36e0 2022-08-01T19:34:18.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.942+0000 7fe46b59e700 1 -- 172.21.15.17:0/169486966 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe46c0b4850 con 0x7fe46c0b36e0 2022-08-01T19:34:18.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.942+0000 7fe4637fe700 1 -- 172.21.15.17:0/169486966 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe45c00f070 con 0x7fe46c0b36e0 2022-08-01T19:34:18.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.942+0000 7fe4637fe700 1 --2- 172.21.15.17:0/169486966 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe450053aa0 0x7fe450055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:18.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.942+0000 7fe4637fe700 1 -- 172.21.15.17:0/169486966 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fe45c06d360 con 0x7fe46c0b36e0 2022-08-01T19:34:18.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.942+0000 7fe46a59c700 1 --2- 172.21.15.17:0/169486966 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe450053aa0 0x7fe450055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:18.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.946+0000 7fe46a59c700 1 --2- 172.21.15.17:0/169486966 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe450053aa0 0x7fe450055f60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fe45400d000 tx=0x7fe454001970).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:18.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:18.950+0000 7fe4637fe700 1 -- 172.21.15.17:0/169486966 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe45c044010 con 0x7fe46c0b36e0 2022-08-01T19:34:19.077 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.070+0000 7fe46b59e700 1 -- 172.21.15.17:0/169486966 --> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe46c0b07a0 con 0x7fe450053aa0 2022-08-01T19:34:19.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.082+0000 7fe4637fe700 1 -- 172.21.15.17:0/169486966 <== mgr.4101 v2:172.21.15.77:6800/23151 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82283 (secure 0 0 0) 0x7fe45400f620 con 0x7fe450053aa0 2022-08-01T19:34:19.085 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:19.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.086+0000 7fe46b59e700 1 -- 172.21.15.17:0/169486966 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe450053aa0 msgr2=0x7fe450055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:19.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.086+0000 7fe46b59e700 1 --2- 172.21.15.17:0/169486966 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe450053aa0 0x7fe450055f60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fe45400d000 tx=0x7fe454001970).stop 2022-08-01T19:34:19.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.086+0000 7fe46b59e700 1 -- 172.21.15.17:0/169486966 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe46c0b36e0 msgr2=0x7fe46c13b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:19.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.086+0000 7fe46b59e700 1 --2- 172.21.15.17:0/169486966 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe46c0b36e0 0x7fe46c13b600 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fe454001820 tx=0x7fe45c016000).stop 2022-08-01T19:34:19.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.086+0000 7fe46b59e700 1 -- 172.21.15.17:0/169486966 shutdown_connections 2022-08-01T19:34:19.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.086+0000 7fe46b59e700 1 --2- 172.21.15.17:0/169486966 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe450053aa0 0x7fe450055f60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:19.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.086+0000 7fe46b59e700 1 --2- 172.21.15.17:0/169486966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe46c144e60 0x7fe46c0ae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:19.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.086+0000 7fe46b59e700 1 --2- 172.21.15.17:0/169486966 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe46c0b36e0 0x7fe46c13b600 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:19.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.086+0000 7fe46b59e700 1 --2- 172.21.15.17:0/169486966 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe46c064b10 0x7fe46c0b2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:19.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.086+0000 7fe46b59e700 1 -- 172.21.15.17:0/169486966 >> 172.21.15.17:0/169486966 conn(0x7fe46c05f840 msgr2=0x7fe46c0b74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:19.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.086+0000 7fe46b59e700 1 -- 172.21.15.17:0/169486966 shutdown_connections 2022-08-01T19:34:19.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.086+0000 7fe46b59e700 1 -- 172.21.15.17:0/169486966 wait complete. 2022-08-01T19:34:19.090 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2022-08-01T19:34:19.099 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2022-08-01T19:34:18.956977+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8390776,"kb_used_data":2104,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746583944,"statfs":{"total":773094113280,"available":764501958656,"internally_reserved":8589934592,"allocated":2154496,"data_stored":832740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.580691"},"pg_stats":[{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.461567+0000","last_change":"2022-08-01T19:34:09.456612+0000","last_active":"2022-08-01T19:34:09.461567+0000","last_peered":"2022-08-01T19:34:09.461567+0000","last_clean":"2022-08-01T19:34:09.461567+0000","last_became_active":"2022-08-01T19:34:09.454913+0000","last_became_peered":"2022-08-01T19:34:09.454913+0000","last_unstale":"2022-08-01T19:34:09.461567+0000","last_undegraded":"2022-08-01T19:34:09.461567+0000","last_fullsized":"2022-08-01T19:34:09.461567+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:08.365038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:08.365038+0000","last_clean_scrub_stamp":"2022-08-01T19:34:08.365038+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'2","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.452568+0000","last_change":"2022-08-01T19:33:58.589491+0000","last_active":"2022-08-01T19:34:09.452568+0000","last_peered":"2022-08-01T19:34:09.452568+0000","last_clean":"2022-08-01T19:34:09.452568+0000","last_became_active":"2022-08-01T19:33:58.588671+0000","last_became_peered":"2022-08-01T19:33:58.588671+0000","last_unstale":"2022-08-01T19:34:09.452568+0000","last_undegraded":"2022-08-01T19:34:09.452568+0000","last_fullsized":"2022-08-01T19:34:09.452568+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:33:53.002518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:33:53.002518+0000","last_clean_scrub_stamp":"2022-08-01T19:33:53.002518+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.379946+0000","last_change":"2022-08-01T19:34:03.246302+0000","last_active":"2022-08-01T19:34:09.379946+0000","last_peered":"2022-08-01T19:34:09.379946+0000","last_clean":"2022-08-01T19:34:09.379946+0000","last_became_active":"2022-08-01T19:34:03.245544+0000","last_became_peered":"2022-08-01T19:34:03.245544+0000","last_unstale":"2022-08-01T19:34:09.379946+0000","last_undegraded":"2022-08-01T19:34:09.379946+0000","last_fullsized":"2022-08-01T19:34:09.379946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_clean_scrub_stamp":"2022-08-01T19:34:02.212190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"25'1","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.712291+0000","last_change":"2022-08-01T19:34:06.369285+0000","last_active":"2022-08-01T19:34:09.712291+0000","last_peered":"2022-08-01T19:34:09.712291+0000","last_clean":"2022-08-01T19:34:09.712291+0000","last_became_active":"2022-08-01T19:34:06.369025+0000","last_became_peered":"2022-08-01T19:34:06.369025+0000","last_unstale":"2022-08-01T19:34:09.712291+0000","last_undegraded":"2022-08-01T19:34:09.712291+0000","last_fullsized":"2022-08-01T19:34:09.712291+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:05.341790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:05.341790+0000","last_clean_scrub_stamp":"2022-08-01T19:34:05.341790+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.458610+0000","last_change":"2022-08-01T19:34:09.457544+0000","last_active":"2022-08-01T19:34:09.458610+0000","last_peered":"2022-08-01T19:34:09.458610+0000","last_clean":"2022-08-01T19:34:09.458610+0000","last_became_active":"2022-08-01T19:34:09.456653+0000","last_became_peered":"2022-08-01T19:34:09.456653+0000","last_unstale":"2022-08-01T19:34:09.458610+0000","last_undegraded":"2022-08-01T19:34:09.458610+0000","last_fullsized":"2022-08-01T19:34:09.458610+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:08.365038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:08.365038+0000","last_clean_scrub_stamp":"2022-08-01T19:34:08.365038+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.452551+0000","last_change":"2022-08-01T19:34:03.246244+0000","last_active":"2022-08-01T19:34:09.452551+0000","last_peered":"2022-08-01T19:34:09.452551+0000","last_clean":"2022-08-01T19:34:09.452551+0000","last_became_active":"2022-08-01T19:34:03.245832+0000","last_became_peered":"2022-08-01T19:34:03.245832+0000","last_unstale":"2022-08-01T19:34:09.452551+0000","last_undegraded":"2022-08-01T19:34:09.452551+0000","last_fullsized":"2022-08-01T19:34:09.452551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_clean_scrub_stamp":"2022-08-01T19:34:02.212190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.381673+0000","last_change":"2022-08-01T19:34:06.367369+0000","last_active":"2022-08-01T19:34:09.381673+0000","last_peered":"2022-08-01T19:34:09.381673+0000","last_clean":"2022-08-01T19:34:09.381673+0000","last_became_active":"2022-08-01T19:34:06.366788+0000","last_became_peered":"2022-08-01T19:34:06.366788+0000","last_unstale":"2022-08-01T19:34:09.381673+0000","last_undegraded":"2022-08-01T19:34:09.381673+0000","last_fullsized":"2022-08-01T19:34:09.381673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:05.341790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:05.341790+0000","last_clean_scrub_stamp":"2022-08-01T19:34:05.341790+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.390104+0000","last_change":"2022-08-01T19:34:09.388701+0000","last_active":"2022-08-01T19:34:09.390104+0000","last_peered":"2022-08-01T19:34:09.390104+0000","last_clean":"2022-08-01T19:34:09.390104+0000","last_became_active":"2022-08-01T19:34:09.388125+0000","last_became_peered":"2022-08-01T19:34:09.388125+0000","last_unstale":"2022-08-01T19:34:09.390104+0000","last_undegraded":"2022-08-01T19:34:09.390104+0000","last_fullsized":"2022-08-01T19:34:09.390104+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:08.365038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:08.365038+0000","last_clean_scrub_stamp":"2022-08-01T19:34:08.365038+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.448306+0000","last_change":"2022-08-01T19:34:03.244647+0000","last_active":"2022-08-01T19:34:09.448306+0000","last_peered":"2022-08-01T19:34:09.448306+0000","last_clean":"2022-08-01T19:34:09.448306+0000","last_became_active":"2022-08-01T19:34:03.244231+0000","last_became_peered":"2022-08-01T19:34:03.244231+0000","last_unstale":"2022-08-01T19:34:09.448306+0000","last_undegraded":"2022-08-01T19:34:09.448306+0000","last_fullsized":"2022-08-01T19:34:09.448306+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_clean_scrub_stamp":"2022-08-01T19:34:02.212190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"25'8","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.723891+0000","last_change":"2022-08-01T19:34:06.532484+0000","last_active":"2022-08-01T19:34:09.723891+0000","last_peered":"2022-08-01T19:34:09.723891+0000","last_clean":"2022-08-01T19:34:09.723891+0000","last_became_active":"2022-08-01T19:34:06.531654+0000","last_became_peered":"2022-08-01T19:34:06.531654+0000","last_unstale":"2022-08-01T19:34:09.723891+0000","last_undegraded":"2022-08-01T19:34:09.723891+0000","last_fullsized":"2022-08-01T19:34:09.723891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:05.341790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:05.341790+0000","last_clean_scrub_stamp":"2022-08-01T19:34:05.341790+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.453630+0000","last_change":"2022-08-01T19:34:09.452647+0000","last_active":"2022-08-01T19:34:09.453630+0000","last_peered":"2022-08-01T19:34:09.453630+0000","last_clean":"2022-08-01T19:34:09.453630+0000","last_became_active":"2022-08-01T19:34:09.451111+0000","last_became_peered":"2022-08-01T19:34:09.451111+0000","last_unstale":"2022-08-01T19:34:09.453630+0000","last_undegraded":"2022-08-01T19:34:09.453630+0000","last_fullsized":"2022-08-01T19:34:09.453630+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:08.365038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:08.365038+0000","last_clean_scrub_stamp":"2022-08-01T19:34:08.365038+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.380051+0000","last_change":"2022-08-01T19:34:03.246395+0000","last_active":"2022-08-01T19:34:09.380051+0000","last_peered":"2022-08-01T19:34:09.380051+0000","last_clean":"2022-08-01T19:34:09.380051+0000","last_became_active":"2022-08-01T19:34:03.245763+0000","last_became_peered":"2022-08-01T19:34:03.245763+0000","last_unstale":"2022-08-01T19:34:09.380051+0000","last_undegraded":"2022-08-01T19:34:09.380051+0000","last_fullsized":"2022-08-01T19:34:09.380051+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_clean_scrub_stamp":"2022-08-01T19:34:02.212190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"25'5","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.717679+0000","last_change":"2022-08-01T19:34:06.369332+0000","last_active":"2022-08-01T19:34:09.717679+0000","last_peered":"2022-08-01T19:34:09.717679+0000","last_clean":"2022-08-01T19:34:09.717679+0000","last_became_active":"2022-08-01T19:34:06.368812+0000","last_became_peered":"2022-08-01T19:34:06.368812+0000","last_unstale":"2022-08-01T19:34:09.717679+0000","last_undegraded":"2022-08-01T19:34:09.717679+0000","last_fullsized":"2022-08-01T19:34:09.717679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:05.341790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:05.341790+0000","last_clean_scrub_stamp":"2022-08-01T19:34:05.341790+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.390115+0000","last_change":"2022-08-01T19:34:09.388798+0000","last_active":"2022-08-01T19:34:09.390115+0000","last_peered":"2022-08-01T19:34:09.390115+0000","last_clean":"2022-08-01T19:34:09.390115+0000","last_became_active":"2022-08-01T19:34:09.388201+0000","last_became_peered":"2022-08-01T19:34:09.388201+0000","last_unstale":"2022-08-01T19:34:09.390115+0000","last_undegraded":"2022-08-01T19:34:09.390115+0000","last_fullsized":"2022-08-01T19:34:09.390115+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:08.365038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:08.365038+0000","last_clean_scrub_stamp":"2022-08-01T19:34:08.365038+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"25'6","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.720114+0000","last_change":"2022-08-01T19:34:06.368730+0000","last_active":"2022-08-01T19:34:09.720114+0000","last_peered":"2022-08-01T19:34:09.720114+0000","last_clean":"2022-08-01T19:34:09.720114+0000","last_became_active":"2022-08-01T19:34:06.368245+0000","last_became_peered":"2022-08-01T19:34:06.368245+0000","last_unstale":"2022-08-01T19:34:09.720114+0000","last_undegraded":"2022-08-01T19:34:09.720114+0000","last_fullsized":"2022-08-01T19:34:09.720114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:05.341790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:05.341790+0000","last_clean_scrub_stamp":"2022-08-01T19:34:05.341790+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.382047+0000","last_change":"2022-08-01T19:34:03.661980+0000","last_active":"2022-08-01T19:34:09.382047+0000","last_peered":"2022-08-01T19:34:09.382047+0000","last_clean":"2022-08-01T19:34:09.382047+0000","last_became_active":"2022-08-01T19:34:03.661358+0000","last_became_peered":"2022-08-01T19:34:03.661358+0000","last_unstale":"2022-08-01T19:34:09.382047+0000","last_undegraded":"2022-08-01T19:34:09.382047+0000","last_fullsized":"2022-08-01T19:34:09.382047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_clean_scrub_stamp":"2022-08-01T19:34:02.212190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.458128+0000","last_change":"2022-08-01T19:34:09.456445+0000","last_active":"2022-08-01T19:34:09.458128+0000","last_peered":"2022-08-01T19:34:09.458128+0000","last_clean":"2022-08-01T19:34:09.458128+0000","last_became_active":"2022-08-01T19:34:09.454810+0000","last_became_peered":"2022-08-01T19:34:09.454810+0000","last_unstale":"2022-08-01T19:34:09.458128+0000","last_undegraded":"2022-08-01T19:34:09.458128+0000","last_fullsized":"2022-08-01T19:34:09.458128+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:08.365038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:08.365038+0000","last_clean_scrub_stamp":"2022-08-01T19:34:08.365038+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"25'8","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.723782+0000","last_change":"2022-08-01T19:34:06.533170+0000","last_active":"2022-08-01T19:34:09.723782+0000","last_peered":"2022-08-01T19:34:09.723782+0000","last_clean":"2022-08-01T19:34:09.723782+0000","last_became_active":"2022-08-01T19:34:06.532373+0000","last_became_peered":"2022-08-01T19:34:06.532373+0000","last_unstale":"2022-08-01T19:34:09.723782+0000","last_undegraded":"2022-08-01T19:34:09.723782+0000","last_fullsized":"2022-08-01T19:34:09.723782+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:05.341790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:05.341790+0000","last_clean_scrub_stamp":"2022-08-01T19:34:05.341790+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1535,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.452199+0000","last_change":"2022-08-01T19:34:03.665604+0000","last_active":"2022-08-01T19:34:09.452199+0000","last_peered":"2022-08-01T19:34:09.452199+0000","last_clean":"2022-08-01T19:34:09.452199+0000","last_became_active":"2022-08-01T19:34:03.664627+0000","last_became_peered":"2022-08-01T19:34:03.664627+0000","last_unstale":"2022-08-01T19:34:09.452199+0000","last_undegraded":"2022-08-01T19:34:09.452199+0000","last_fullsized":"2022-08-01T19:34:09.452199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_clean_scrub_stamp":"2022-08-01T19:34:02.212190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.457710+0000","last_change":"2022-08-01T19:34:09.456064+0000","last_active":"2022-08-01T19:34:09.457710+0000","last_peered":"2022-08-01T19:34:09.457710+0000","last_clean":"2022-08-01T19:34:09.457710+0000","last_became_active":"2022-08-01T19:34:09.455098+0000","last_became_peered":"2022-08-01T19:34:09.455098+0000","last_unstale":"2022-08-01T19:34:09.457710+0000","last_undegraded":"2022-08-01T19:34:09.457710+0000","last_fullsized":"2022-08-01T19:34:09.457710+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:08.365038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:08.365038+0000","last_clean_scrub_stamp":"2022-08-01T19:34:08.365038+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"25'7","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.720933+0000","last_change":"2022-08-01T19:34:06.533868+0000","last_active":"2022-08-01T19:34:09.720933+0000","last_peered":"2022-08-01T19:34:09.720933+0000","last_clean":"2022-08-01T19:34:09.720933+0000","last_became_active":"2022-08-01T19:34:06.532945+0000","last_became_peered":"2022-08-01T19:34:06.532945+0000","last_unstale":"2022-08-01T19:34:09.720933+0000","last_undegraded":"2022-08-01T19:34:09.720933+0000","last_fullsized":"2022-08-01T19:34:09.720933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:05.341790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:05.341790+0000","last_clean_scrub_stamp":"2022-08-01T19:34:05.341790+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.382030+0000","last_change":"2022-08-01T19:34:03.243940+0000","last_active":"2022-08-01T19:34:09.382030+0000","last_peered":"2022-08-01T19:34:09.382030+0000","last_clean":"2022-08-01T19:34:09.382030+0000","last_became_active":"2022-08-01T19:34:03.243496+0000","last_became_peered":"2022-08-01T19:34:03.243496+0000","last_unstale":"2022-08-01T19:34:09.382030+0000","last_undegraded":"2022-08-01T19:34:09.382030+0000","last_fullsized":"2022-08-01T19:34:09.382030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_clean_scrub_stamp":"2022-08-01T19:34:02.212190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.466895+0000","last_change":"2022-08-01T19:34:09.458607+0000","last_active":"2022-08-01T19:34:09.466895+0000","last_peered":"2022-08-01T19:34:09.466895+0000","last_clean":"2022-08-01T19:34:09.466895+0000","last_became_active":"2022-08-01T19:34:09.457527+0000","last_became_peered":"2022-08-01T19:34:09.457527+0000","last_unstale":"2022-08-01T19:34:09.466895+0000","last_undegraded":"2022-08-01T19:34:09.466895+0000","last_fullsized":"2022-08-01T19:34:09.466895+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:08.365038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:08.365038+0000","last_clean_scrub_stamp":"2022-08-01T19:34:08.365038+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"25'4","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.720983+0000","last_change":"2022-08-01T19:34:06.366911+0000","last_active":"2022-08-01T19:34:09.720983+0000","last_peered":"2022-08-01T19:34:09.720983+0000","last_clean":"2022-08-01T19:34:09.720983+0000","last_became_active":"2022-08-01T19:34:06.366232+0000","last_became_peered":"2022-08-01T19:34:06.366232+0000","last_unstale":"2022-08-01T19:34:09.720983+0000","last_undegraded":"2022-08-01T19:34:09.720983+0000","last_fullsized":"2022-08-01T19:34:09.720983+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:05.341790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:05.341790+0000","last_clean_scrub_stamp":"2022-08-01T19:34:05.341790+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.386041+0000","last_change":"2022-08-01T19:34:03.245622+0000","last_active":"2022-08-01T19:34:09.386041+0000","last_peered":"2022-08-01T19:34:09.386041+0000","last_clean":"2022-08-01T19:34:09.386041+0000","last_became_active":"2022-08-01T19:34:03.245060+0000","last_became_peered":"2022-08-01T19:34:03.245060+0000","last_unstale":"2022-08-01T19:34:09.386041+0000","last_undegraded":"2022-08-01T19:34:09.386041+0000","last_fullsized":"2022-08-01T19:34:09.386041+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_clean_scrub_stamp":"2022-08-01T19:34:02.212190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":16,"acting":16,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048848,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322992,"statfs":{"total":96636764160,"available":95562743808,"internally_reserved":1073741824,"allocated":270336,"data_stored":103821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048844,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322996,"statfs":{"total":96636764160,"available":95562747904,"internally_reserved":1073741824,"allocated":266240,"data_stored":103695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.155}]}]},{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048848,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322992,"statfs":{"total":96636764160,"available":95562743808,"internally_reserved":1073741824,"allocated":270336,"data_stored":104236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]}]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048840,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323000,"statfs":{"total":96636764160,"available":95562752000,"internally_reserved":1073741824,"allocated":262144,"data_stored":103661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0349999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]}]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048852,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322988,"statfs":{"total":96636764160,"available":95562739712,"internally_reserved":1073741824,"allocated":274432,"data_stored":103818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5820000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]}]},{"osd":2,"up_from":14,"seq":60129542152,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048848,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322992,"statfs":{"total":96636764160,"available":95562743808,"internally_reserved":1073741824,"allocated":270336,"data_stored":104618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]}]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048852,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322988,"statfs":{"total":96636764160,"available":95562739712,"internally_reserved":1073741824,"allocated":274432,"data_stored":105196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]}]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048844,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322996,"statfs":{"total":96636764160,"available":95562747904,"internally_reserved":1073741824,"allocated":266240,"data_stored":103695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.01}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.107}]}]}]}} 2022-08-01T19:34:19.102 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2022-08-01T19:34:19.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.194+0000 7fca02f59700 1 Processor -- start 2022-08-01T19:34:19.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.194+0000 7fca02f59700 1 -- start start 2022-08-01T19:34:19.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.198+0000 7fca02f59700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9fc0b6710 0x7fc9fc0b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:19.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.198+0000 7fca02f59700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fc9fc0b4550 0x7fc9fc0b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:19.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.198+0000 7fca02f59700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9fc144240 con 0x7fc9fc061ec0 2022-08-01T19:34:19.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.198+0000 7fca02f59700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9fc0b6fe0 con 0x7fc9fc0b4550 2022-08-01T19:34:19.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.198+0000 7fca02f59700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc143d00 con 0x7fc9fc0b6710 2022-08-01T19:34:19.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.198+0000 7fc9fbfff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fc9fc0b4550 0x7fc9fc0b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:19.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.198+0000 7fca00cf5700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9fc0b6710 0x7fc9fc0b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:19.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.198+0000 7fc9fbfff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fc9fc0b4550 0x7fc9fc0b6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:34586/0 (socket says 172.21.15.17:34586) 2022-08-01T19:34:19.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.198+0000 7fc9fbfff700 1 -- 172.21.15.17:0/791307454 learned_addr learned my addr 172.21.15.17:0/791307454 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:19.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.198+0000 7fca00cf5700 1 -- 172.21.15.17:0/791307454 >> v2:172.21.15.77:3300/0 conn(0x7fc9fc0b4550 msgr2=0x7fc9fc0b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:19.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.198+0000 7fca00cf5700 1 --2- 172.21.15.17:0/791307454 >> v2:172.21.15.77:3300/0 conn(0x7fc9fc0b4550 0x7fc9fc0b6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:19.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.198+0000 7fca00cf5700 1 -- 172.21.15.17:0/791307454 >> v1:172.21.15.77:6789/0 conn(0x7fc9fc061ec0 legacy=0x7fc9fc146850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:19.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.198+0000 7fc9fb7fe700 1 -- 172.21.15.17:0/791307454 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3427652663 0 0) 0x7fc9fc144240 con 0x7fc9fc061ec0 2022-08-01T19:34:19.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.198+0000 7fca00cf5700 1 -- 172.21.15.17:0/791307454 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9fc143fa0 con 0x7fc9fc0b6710 2022-08-01T19:34:19.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.198+0000 7fc9fbfff700 1 --2- 172.21.15.17:0/791307454 >> v2:172.21.15.77:3300/0 conn(0x7fc9fc0b4550 0x7fc9fc0b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:34:19.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.198+0000 7fca00cf5700 1 --2- 172.21.15.17:0/791307454 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9fc0b6710 0x7fc9fc0b5d80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fc9ec002f70 tx=0x7fc9ec006df0).ready entity=mon.2 client_cookie=83048d6cb3b1b8cc server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:19.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.198+0000 7fc9fb7fe700 1 -- 172.21.15.17:0/791307454 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9ec00d570 con 0x7fc9fc0b6710 2022-08-01T19:34:19.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.198+0000 7fc9fb7fe700 1 -- 172.21.15.17:0/791307454 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9ec000e90 con 0x7fc9fc0b6710 2022-08-01T19:34:19.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.198+0000 7fc9fb7fe700 1 -- 172.21.15.17:0/791307454 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9ec00d8c0 con 0x7fc9fc0b6710 2022-08-01T19:34:19.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.198+0000 7fca02f59700 1 -- 172.21.15.17:0/791307454 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9fc0b6710 msgr2=0x7fc9fc0b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:19.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.198+0000 7fca02f59700 1 --2- 172.21.15.17:0/791307454 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9fc0b6710 0x7fc9fc0b5d80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fc9ec002f70 tx=0x7fc9ec006df0).stop 2022-08-01T19:34:19.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.198+0000 7fca02f59700 1 -- 172.21.15.17:0/791307454 shutdown_connections 2022-08-01T19:34:19.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.198+0000 7fca02f59700 1 --2- 172.21.15.17:0/791307454 >> v2:172.21.15.77:3300/0 conn(0x7fc9fc0b4550 0x7fc9fc0b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:19.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.198+0000 7fca02f59700 1 --2- 172.21.15.17:0/791307454 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9fc0b6710 0x7fc9fc0b5d80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:19.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.198+0000 7fca02f59700 1 -- 172.21.15.17:0/791307454 >> 172.21.15.17:0/791307454 conn(0x7fc9fc05c620 msgr2=0x7fc9fc05ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:19.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.198+0000 7fca02f59700 1 -- 172.21.15.17:0/791307454 shutdown_connections 2022-08-01T19:34:19.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.202+0000 7fca02f59700 1 -- 172.21.15.17:0/791307454 wait complete. 2022-08-01T19:34:19.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.202+0000 7fca02f59700 1 Processor -- start 2022-08-01T19:34:19.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.202+0000 7fca02f59700 1 -- start start 2022-08-01T19:34:19.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.210+0000 7fca02f59700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9fc061ec0 0x7fc9fc1403d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:19.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.210+0000 7fca02f59700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9fc0b4550 0x7fc9fc0b3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:19.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.210+0000 7fca02f59700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9fc0b6710 0x7fc9fc0b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:19.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.210+0000 7fca02f59700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc125af0 con 0x7fc9fc0b6710 2022-08-01T19:34:19.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.210+0000 7fca02f59700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc155b00 con 0x7fc9fc0b4550 2022-08-01T19:34:19.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.210+0000 7fca02f59700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc155da0 con 0x7fc9fc061ec0 2022-08-01T19:34:19.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.210+0000 7fca014f6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9fc0b6710 0x7fc9fc0b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:19.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.210+0000 7fca00cf5700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9fc061ec0 0x7fc9fc1403d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:19.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.210+0000 7fc9fbfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9fc0b4550 0x7fc9fc0b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:19.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.210+0000 7fca00cf5700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9fc061ec0 0x7fc9fc1403d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:32770/0 (socket says 172.21.15.17:32770) 2022-08-01T19:34:19.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.210+0000 7fca00cf5700 1 -- 172.21.15.17:0/2628070711 learned_addr learned my addr 172.21.15.17:0/2628070711 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:19.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.210+0000 7fca014f6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9fc0b6710 0x7fc9fc0b3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36674/0 (socket says 172.21.15.17:36674) 2022-08-01T19:34:19.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.210+0000 7fca014f6700 1 -- 172.21.15.17:0/2628070711 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9fc061ec0 msgr2=0x7fc9fc1403d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:19.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.210+0000 7fca014f6700 1 --2- 172.21.15.17:0/2628070711 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9fc061ec0 0x7fc9fc1403d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:19.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.210+0000 7fca014f6700 1 -- 172.21.15.17:0/2628070711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9fc0b4550 msgr2=0x7fc9fc0b3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:19.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.210+0000 7fca014f6700 1 --2- 172.21.15.17:0/2628070711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9fc0b4550 0x7fc9fc0b3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:19.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.210+0000 7fca014f6700 1 -- 172.21.15.17:0/2628070711 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e4009530 con 0x7fc9fc0b6710 2022-08-01T19:34:19.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.210+0000 7fca00cf5700 1 --2- 172.21.15.17:0/2628070711 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9fc061ec0 0x7fc9fc1403d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:34:19.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.210+0000 7fca014f6700 1 --2- 172.21.15.17:0/2628070711 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9fc0b6710 0x7fc9fc0b3850 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fc9f4000ed0 tx=0x7fc9f4007b70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:19.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.210+0000 7fc9f9ffb700 1 -- 172.21.15.17:0/2628070711 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9f400ccf0 con 0x7fc9fc0b6710 2022-08-01T19:34:19.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.210+0000 7fc9f9ffb700 1 -- 172.21.15.17:0/2628070711 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f400ce50 con 0x7fc9fc0b6710 2022-08-01T19:34:19.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.210+0000 7fca02f59700 1 -- 172.21.15.17:0/2628070711 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9ec00b180 con 0x7fc9fc0b6710 2022-08-01T19:34:19.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.210+0000 7fc9f9ffb700 1 -- 172.21.15.17:0/2628070711 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9f4007220 con 0x7fc9fc0b6710 2022-08-01T19:34:19.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.210+0000 7fca02f59700 1 -- 172.21.15.17:0/2628070711 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9fc126ad0 con 0x7fc9fc0b6710 2022-08-01T19:34:19.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.210+0000 7fca02f59700 1 -- 172.21.15.17:0/2628070711 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9fc155870 con 0x7fc9fc0b6710 2022-08-01T19:34:19.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.210+0000 7fc9f9ffb700 1 -- 172.21.15.17:0/2628070711 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc9f4010030 con 0x7fc9fc0b6710 2022-08-01T19:34:19.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.210+0000 7fc9f9ffb700 1 --2- 172.21.15.17:0/2628070711 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9e0053b30 0x7fc9e0055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:19.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.210+0000 7fc9f9ffb700 1 -- 172.21.15.17:0/2628070711 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fc9f406ba00 con 0x7fc9fc0b6710 2022-08-01T19:34:19.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.210+0000 7fca00cf5700 1 --2- 172.21.15.17:0/2628070711 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9e0053b30 0x7fc9e0055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:19.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.210+0000 7fca00cf5700 1 --2- 172.21.15.17:0/2628070711 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9e0053b30 0x7fc9e0055ff0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fc9ec01a000 tx=0x7fc9ec014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:19.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.218+0000 7fc9f9ffb700 1 -- 172.21.15.17:0/2628070711 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc9f40427c0 con 0x7fc9fc0b6710 2022-08-01T19:34:19.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.338+0000 7fca02f59700 1 -- 172.21.15.17:0/2628070711 --> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc9fc061780 con 0x7fc9e0053b30 2022-08-01T19:34:19.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.350+0000 7fc9f9ffb700 1 -- 172.21.15.17:0/2628070711 <== mgr.4101 v2:172.21.15.77:6800/23151 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82283 (secure 0 0 0) 0x7fc9fc061780 con 0x7fc9e0053b30 2022-08-01T19:34:19.353 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:19.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.350+0000 7fca02f59700 1 -- 172.21.15.17:0/2628070711 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9e0053b30 msgr2=0x7fc9e0055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:19.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.350+0000 7fca02f59700 1 --2- 172.21.15.17:0/2628070711 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9e0053b30 0x7fc9e0055ff0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fc9ec01a000 tx=0x7fc9ec014000).stop 2022-08-01T19:34:19.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.350+0000 7fca02f59700 1 -- 172.21.15.17:0/2628070711 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9fc0b6710 msgr2=0x7fc9fc0b3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:19.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.350+0000 7fca02f59700 1 --2- 172.21.15.17:0/2628070711 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9fc0b6710 0x7fc9fc0b3850 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fc9f4000ed0 tx=0x7fc9f4007b70).stop 2022-08-01T19:34:19.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.350+0000 7fca02f59700 1 -- 172.21.15.17:0/2628070711 shutdown_connections 2022-08-01T19:34:19.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.350+0000 7fca02f59700 1 --2- 172.21.15.17:0/2628070711 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9e0053b30 0x7fc9e0055ff0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:19.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.350+0000 7fca02f59700 1 --2- 172.21.15.17:0/2628070711 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9fc0b6710 0x7fc9fc0b3850 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:19.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.350+0000 7fca02f59700 1 --2- 172.21.15.17:0/2628070711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9fc0b4550 0x7fc9fc0b3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:19.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.350+0000 7fca02f59700 1 --2- 172.21.15.17:0/2628070711 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9fc061ec0 0x7fc9fc1403d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:19.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.350+0000 7fca02f59700 1 -- 172.21.15.17:0/2628070711 >> 172.21.15.17:0/2628070711 conn(0x7fc9fc05c620 msgr2=0x7fc9fc126e80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:19.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.350+0000 7fca02f59700 1 -- 172.21.15.17:0/2628070711 shutdown_connections 2022-08-01T19:34:19.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.350+0000 7fca02f59700 1 -- 172.21.15.17:0/2628070711 wait complete. 2022-08-01T19:34:19.358 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2022-08-01T19:34:19.367 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2022-08-01T19:34:18.956977+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8390776,"kb_used_data":2104,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746583944,"statfs":{"total":773094113280,"available":764501958656,"internally_reserved":8589934592,"allocated":2154496,"data_stored":832740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.580691"},"pg_stats":[{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.461567+0000","last_change":"2022-08-01T19:34:09.456612+0000","last_active":"2022-08-01T19:34:09.461567+0000","last_peered":"2022-08-01T19:34:09.461567+0000","last_clean":"2022-08-01T19:34:09.461567+0000","last_became_active":"2022-08-01T19:34:09.454913+0000","last_became_peered":"2022-08-01T19:34:09.454913+0000","last_unstale":"2022-08-01T19:34:09.461567+0000","last_undegraded":"2022-08-01T19:34:09.461567+0000","last_fullsized":"2022-08-01T19:34:09.461567+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:08.365038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:08.365038+0000","last_clean_scrub_stamp":"2022-08-01T19:34:08.365038+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'2","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.452568+0000","last_change":"2022-08-01T19:33:58.589491+0000","last_active":"2022-08-01T19:34:09.452568+0000","last_peered":"2022-08-01T19:34:09.452568+0000","last_clean":"2022-08-01T19:34:09.452568+0000","last_became_active":"2022-08-01T19:33:58.588671+0000","last_became_peered":"2022-08-01T19:33:58.588671+0000","last_unstale":"2022-08-01T19:34:09.452568+0000","last_undegraded":"2022-08-01T19:34:09.452568+0000","last_fullsized":"2022-08-01T19:34:09.452568+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:33:53.002518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:33:53.002518+0000","last_clean_scrub_stamp":"2022-08-01T19:33:53.002518+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.379946+0000","last_change":"2022-08-01T19:34:03.246302+0000","last_active":"2022-08-01T19:34:09.379946+0000","last_peered":"2022-08-01T19:34:09.379946+0000","last_clean":"2022-08-01T19:34:09.379946+0000","last_became_active":"2022-08-01T19:34:03.245544+0000","last_became_peered":"2022-08-01T19:34:03.245544+0000","last_unstale":"2022-08-01T19:34:09.379946+0000","last_undegraded":"2022-08-01T19:34:09.379946+0000","last_fullsized":"2022-08-01T19:34:09.379946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_clean_scrub_stamp":"2022-08-01T19:34:02.212190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"25'1","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.712291+0000","last_change":"2022-08-01T19:34:06.369285+0000","last_active":"2022-08-01T19:34:09.712291+0000","last_peered":"2022-08-01T19:34:09.712291+0000","last_clean":"2022-08-01T19:34:09.712291+0000","last_became_active":"2022-08-01T19:34:06.369025+0000","last_became_peered":"2022-08-01T19:34:06.369025+0000","last_unstale":"2022-08-01T19:34:09.712291+0000","last_undegraded":"2022-08-01T19:34:09.712291+0000","last_fullsized":"2022-08-01T19:34:09.712291+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:05.341790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:05.341790+0000","last_clean_scrub_stamp":"2022-08-01T19:34:05.341790+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.458610+0000","last_change":"2022-08-01T19:34:09.457544+0000","last_active":"2022-08-01T19:34:09.458610+0000","last_peered":"2022-08-01T19:34:09.458610+0000","last_clean":"2022-08-01T19:34:09.458610+0000","last_became_active":"2022-08-01T19:34:09.456653+0000","last_became_peered":"2022-08-01T19:34:09.456653+0000","last_unstale":"2022-08-01T19:34:09.458610+0000","last_undegraded":"2022-08-01T19:34:09.458610+0000","last_fullsized":"2022-08-01T19:34:09.458610+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:08.365038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:08.365038+0000","last_clean_scrub_stamp":"2022-08-01T19:34:08.365038+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.452551+0000","last_change":"2022-08-01T19:34:03.246244+0000","last_active":"2022-08-01T19:34:09.452551+0000","last_peered":"2022-08-01T19:34:09.452551+0000","last_clean":"2022-08-01T19:34:09.452551+0000","last_became_active":"2022-08-01T19:34:03.245832+0000","last_became_peered":"2022-08-01T19:34:03.245832+0000","last_unstale":"2022-08-01T19:34:09.452551+0000","last_undegraded":"2022-08-01T19:34:09.452551+0000","last_fullsized":"2022-08-01T19:34:09.452551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_clean_scrub_stamp":"2022-08-01T19:34:02.212190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.381673+0000","last_change":"2022-08-01T19:34:06.367369+0000","last_active":"2022-08-01T19:34:09.381673+0000","last_peered":"2022-08-01T19:34:09.381673+0000","last_clean":"2022-08-01T19:34:09.381673+0000","last_became_active":"2022-08-01T19:34:06.366788+0000","last_became_peered":"2022-08-01T19:34:06.366788+0000","last_unstale":"2022-08-01T19:34:09.381673+0000","last_undegraded":"2022-08-01T19:34:09.381673+0000","last_fullsized":"2022-08-01T19:34:09.381673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:05.341790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:05.341790+0000","last_clean_scrub_stamp":"2022-08-01T19:34:05.341790+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.390104+0000","last_change":"2022-08-01T19:34:09.388701+0000","last_active":"2022-08-01T19:34:09.390104+0000","last_peered":"2022-08-01T19:34:09.390104+0000","last_clean":"2022-08-01T19:34:09.390104+0000","last_became_active":"2022-08-01T19:34:09.388125+0000","last_became_peered":"2022-08-01T19:34:09.388125+0000","last_unstale":"2022-08-01T19:34:09.390104+0000","last_undegraded":"2022-08-01T19:34:09.390104+0000","last_fullsized":"2022-08-01T19:34:09.390104+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:08.365038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:08.365038+0000","last_clean_scrub_stamp":"2022-08-01T19:34:08.365038+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.448306+0000","last_change":"2022-08-01T19:34:03.244647+0000","last_active":"2022-08-01T19:34:09.448306+0000","last_peered":"2022-08-01T19:34:09.448306+0000","last_clean":"2022-08-01T19:34:09.448306+0000","last_became_active":"2022-08-01T19:34:03.244231+0000","last_became_peered":"2022-08-01T19:34:03.244231+0000","last_unstale":"2022-08-01T19:34:09.448306+0000","last_undegraded":"2022-08-01T19:34:09.448306+0000","last_fullsized":"2022-08-01T19:34:09.448306+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_clean_scrub_stamp":"2022-08-01T19:34:02.212190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"25'8","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.723891+0000","last_change":"2022-08-01T19:34:06.532484+0000","last_active":"2022-08-01T19:34:09.723891+0000","last_peered":"2022-08-01T19:34:09.723891+0000","last_clean":"2022-08-01T19:34:09.723891+0000","last_became_active":"2022-08-01T19:34:06.531654+0000","last_became_peered":"2022-08-01T19:34:06.531654+0000","last_unstale":"2022-08-01T19:34:09.723891+0000","last_undegraded":"2022-08-01T19:34:09.723891+0000","last_fullsized":"2022-08-01T19:34:09.723891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:05.341790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:05.341790+0000","last_clean_scrub_stamp":"2022-08-01T19:34:05.341790+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.453630+0000","last_change":"2022-08-01T19:34:09.452647+0000","last_active":"2022-08-01T19:34:09.453630+0000","last_peered":"2022-08-01T19:34:09.453630+0000","last_clean":"2022-08-01T19:34:09.453630+0000","last_became_active":"2022-08-01T19:34:09.451111+0000","last_became_peered":"2022-08-01T19:34:09.451111+0000","last_unstale":"2022-08-01T19:34:09.453630+0000","last_undegraded":"2022-08-01T19:34:09.453630+0000","last_fullsized":"2022-08-01T19:34:09.453630+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:08.365038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:08.365038+0000","last_clean_scrub_stamp":"2022-08-01T19:34:08.365038+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.380051+0000","last_change":"2022-08-01T19:34:03.246395+0000","last_active":"2022-08-01T19:34:09.380051+0000","last_peered":"2022-08-01T19:34:09.380051+0000","last_clean":"2022-08-01T19:34:09.380051+0000","last_became_active":"2022-08-01T19:34:03.245763+0000","last_became_peered":"2022-08-01T19:34:03.245763+0000","last_unstale":"2022-08-01T19:34:09.380051+0000","last_undegraded":"2022-08-01T19:34:09.380051+0000","last_fullsized":"2022-08-01T19:34:09.380051+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_clean_scrub_stamp":"2022-08-01T19:34:02.212190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"25'5","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.717679+0000","last_change":"2022-08-01T19:34:06.369332+0000","last_active":"2022-08-01T19:34:09.717679+0000","last_peered":"2022-08-01T19:34:09.717679+0000","last_clean":"2022-08-01T19:34:09.717679+0000","last_became_active":"2022-08-01T19:34:06.368812+0000","last_became_peered":"2022-08-01T19:34:06.368812+0000","last_unstale":"2022-08-01T19:34:09.717679+0000","last_undegraded":"2022-08-01T19:34:09.717679+0000","last_fullsized":"2022-08-01T19:34:09.717679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:05.341790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:05.341790+0000","last_clean_scrub_stamp":"2022-08-01T19:34:05.341790+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.390115+0000","last_change":"2022-08-01T19:34:09.388798+0000","last_active":"2022-08-01T19:34:09.390115+0000","last_peered":"2022-08-01T19:34:09.390115+0000","last_clean":"2022-08-01T19:34:09.390115+0000","last_became_active":"2022-08-01T19:34:09.388201+0000","last_became_peered":"2022-08-01T19:34:09.388201+0000","last_unstale":"2022-08-01T19:34:09.390115+0000","last_undegraded":"2022-08-01T19:34:09.390115+0000","last_fullsized":"2022-08-01T19:34:09.390115+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:08.365038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:08.365038+0000","last_clean_scrub_stamp":"2022-08-01T19:34:08.365038+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"25'6","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.720114+0000","last_change":"2022-08-01T19:34:06.368730+0000","last_active":"2022-08-01T19:34:09.720114+0000","last_peered":"2022-08-01T19:34:09.720114+0000","last_clean":"2022-08-01T19:34:09.720114+0000","last_became_active":"2022-08-01T19:34:06.368245+0000","last_became_peered":"2022-08-01T19:34:06.368245+0000","last_unstale":"2022-08-01T19:34:09.720114+0000","last_undegraded":"2022-08-01T19:34:09.720114+0000","last_fullsized":"2022-08-01T19:34:09.720114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:05.341790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:05.341790+0000","last_clean_scrub_stamp":"2022-08-01T19:34:05.341790+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.382047+0000","last_change":"2022-08-01T19:34:03.661980+0000","last_active":"2022-08-01T19:34:09.382047+0000","last_peered":"2022-08-01T19:34:09.382047+0000","last_clean":"2022-08-01T19:34:09.382047+0000","last_became_active":"2022-08-01T19:34:03.661358+0000","last_became_peered":"2022-08-01T19:34:03.661358+0000","last_unstale":"2022-08-01T19:34:09.382047+0000","last_undegraded":"2022-08-01T19:34:09.382047+0000","last_fullsized":"2022-08-01T19:34:09.382047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_clean_scrub_stamp":"2022-08-01T19:34:02.212190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.458128+0000","last_change":"2022-08-01T19:34:09.456445+0000","last_active":"2022-08-01T19:34:09.458128+0000","last_peered":"2022-08-01T19:34:09.458128+0000","last_clean":"2022-08-01T19:34:09.458128+0000","last_became_active":"2022-08-01T19:34:09.454810+0000","last_became_peered":"2022-08-01T19:34:09.454810+0000","last_unstale":"2022-08-01T19:34:09.458128+0000","last_undegraded":"2022-08-01T19:34:09.458128+0000","last_fullsized":"2022-08-01T19:34:09.458128+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:08.365038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:08.365038+0000","last_clean_scrub_stamp":"2022-08-01T19:34:08.365038+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"25'8","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.723782+0000","last_change":"2022-08-01T19:34:06.533170+0000","last_active":"2022-08-01T19:34:09.723782+0000","last_peered":"2022-08-01T19:34:09.723782+0000","last_clean":"2022-08-01T19:34:09.723782+0000","last_became_active":"2022-08-01T19:34:06.532373+0000","last_became_peered":"2022-08-01T19:34:06.532373+0000","last_unstale":"2022-08-01T19:34:09.723782+0000","last_undegraded":"2022-08-01T19:34:09.723782+0000","last_fullsized":"2022-08-01T19:34:09.723782+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:05.341790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:05.341790+0000","last_clean_scrub_stamp":"2022-08-01T19:34:05.341790+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1535,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.452199+0000","last_change":"2022-08-01T19:34:03.665604+0000","last_active":"2022-08-01T19:34:09.452199+0000","last_peered":"2022-08-01T19:34:09.452199+0000","last_clean":"2022-08-01T19:34:09.452199+0000","last_became_active":"2022-08-01T19:34:03.664627+0000","last_became_peered":"2022-08-01T19:34:03.664627+0000","last_unstale":"2022-08-01T19:34:09.452199+0000","last_undegraded":"2022-08-01T19:34:09.452199+0000","last_fullsized":"2022-08-01T19:34:09.452199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_clean_scrub_stamp":"2022-08-01T19:34:02.212190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.457710+0000","last_change":"2022-08-01T19:34:09.456064+0000","last_active":"2022-08-01T19:34:09.457710+0000","last_peered":"2022-08-01T19:34:09.457710+0000","last_clean":"2022-08-01T19:34:09.457710+0000","last_became_active":"2022-08-01T19:34:09.455098+0000","last_became_peered":"2022-08-01T19:34:09.455098+0000","last_unstale":"2022-08-01T19:34:09.457710+0000","last_undegraded":"2022-08-01T19:34:09.457710+0000","last_fullsized":"2022-08-01T19:34:09.457710+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:08.365038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:08.365038+0000","last_clean_scrub_stamp":"2022-08-01T19:34:08.365038+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"25'7","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.720933+0000","last_change":"2022-08-01T19:34:06.533868+0000","last_active":"2022-08-01T19:34:09.720933+0000","last_peered":"2022-08-01T19:34:09.720933+0000","last_clean":"2022-08-01T19:34:09.720933+0000","last_became_active":"2022-08-01T19:34:06.532945+0000","last_became_peered":"2022-08-01T19:34:06.532945+0000","last_unstale":"2022-08-01T19:34:09.720933+0000","last_undegraded":"2022-08-01T19:34:09.720933+0000","last_fullsized":"2022-08-01T19:34:09.720933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:05.341790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:05.341790+0000","last_clean_scrub_stamp":"2022-08-01T19:34:05.341790+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.382030+0000","last_change":"2022-08-01T19:34:03.243940+0000","last_active":"2022-08-01T19:34:09.382030+0000","last_peered":"2022-08-01T19:34:09.382030+0000","last_clean":"2022-08-01T19:34:09.382030+0000","last_became_active":"2022-08-01T19:34:03.243496+0000","last_became_peered":"2022-08-01T19:34:03.243496+0000","last_unstale":"2022-08-01T19:34:09.382030+0000","last_undegraded":"2022-08-01T19:34:09.382030+0000","last_fullsized":"2022-08-01T19:34:09.382030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_clean_scrub_stamp":"2022-08-01T19:34:02.212190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.466895+0000","last_change":"2022-08-01T19:34:09.458607+0000","last_active":"2022-08-01T19:34:09.466895+0000","last_peered":"2022-08-01T19:34:09.466895+0000","last_clean":"2022-08-01T19:34:09.466895+0000","last_became_active":"2022-08-01T19:34:09.457527+0000","last_became_peered":"2022-08-01T19:34:09.457527+0000","last_unstale":"2022-08-01T19:34:09.466895+0000","last_undegraded":"2022-08-01T19:34:09.466895+0000","last_fullsized":"2022-08-01T19:34:09.466895+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:08.365038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:08.365038+0000","last_clean_scrub_stamp":"2022-08-01T19:34:08.365038+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"25'4","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.720983+0000","last_change":"2022-08-01T19:34:06.366911+0000","last_active":"2022-08-01T19:34:09.720983+0000","last_peered":"2022-08-01T19:34:09.720983+0000","last_clean":"2022-08-01T19:34:09.720983+0000","last_became_active":"2022-08-01T19:34:06.366232+0000","last_became_peered":"2022-08-01T19:34:06.366232+0000","last_unstale":"2022-08-01T19:34:09.720983+0000","last_undegraded":"2022-08-01T19:34:09.720983+0000","last_fullsized":"2022-08-01T19:34:09.720983+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:05.341790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:05.341790+0000","last_clean_scrub_stamp":"2022-08-01T19:34:05.341790+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2022-08-01T19:34:09.386041+0000","last_change":"2022-08-01T19:34:03.245622+0000","last_active":"2022-08-01T19:34:09.386041+0000","last_peered":"2022-08-01T19:34:09.386041+0000","last_clean":"2022-08-01T19:34:09.386041+0000","last_became_active":"2022-08-01T19:34:03.245060+0000","last_became_peered":"2022-08-01T19:34:03.245060+0000","last_unstale":"2022-08-01T19:34:09.386041+0000","last_undegraded":"2022-08-01T19:34:09.386041+0000","last_fullsized":"2022-08-01T19:34:09.386041+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_clean_scrub_stamp":"2022-08-01T19:34:02.212190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":16,"acting":16,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048848,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322992,"statfs":{"total":96636764160,"available":95562743808,"internally_reserved":1073741824,"allocated":270336,"data_stored":103821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048844,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322996,"statfs":{"total":96636764160,"available":95562747904,"internally_reserved":1073741824,"allocated":266240,"data_stored":103695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.155}]}]},{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048848,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322992,"statfs":{"total":96636764160,"available":95562743808,"internally_reserved":1073741824,"allocated":270336,"data_stored":104236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]}]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048840,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323000,"statfs":{"total":96636764160,"available":95562752000,"internally_reserved":1073741824,"allocated":262144,"data_stored":103661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0349999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]}]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048852,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322988,"statfs":{"total":96636764160,"available":95562739712,"internally_reserved":1073741824,"allocated":274432,"data_stored":103818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5820000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]}]},{"osd":2,"up_from":14,"seq":60129542152,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048848,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322992,"statfs":{"total":96636764160,"available":95562743808,"internally_reserved":1073741824,"allocated":270336,"data_stored":104618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]}]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048852,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322988,"statfs":{"total":96636764160,"available":95562739712,"internally_reserved":1073741824,"allocated":274432,"data_stored":105196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]}]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048844,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322996,"statfs":{"total":96636764160,"available":95562747904,"internally_reserved":1073741824,"allocated":266240,"data_stored":103695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.01}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.107}]}]}]}} 2022-08-01T19:34:19.369 INFO:tasks.ceph.ceph_manager.ceph:clean! 2022-08-01T19:34:19.369 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2022-08-01T19:34:19.370 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2022-08-01T19:34:19.371 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2022-08-01T19:34:19.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.466+0000 7f6003fff700 1 Processor -- start 2022-08-01T19:34:19.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.466+0000 7f6003fff700 1 -- start start 2022-08-01T19:34:19.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.466+0000 7f6003fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f60040af550 0x7f60040b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:19.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.466+0000 7f6003fff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f6004060330 0x7f60041445e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:19.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.466+0000 7f6003fff700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60040aa630 con 0x7f6004144ad0 2022-08-01T19:34:19.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.470+0000 7f6003fff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60040b15e0 con 0x7f6004060330 2022-08-01T19:34:19.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.470+0000 7f6003fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60040af930 con 0x7f60040af550 2022-08-01T19:34:19.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.470+0000 7f60037fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f6004144ad0 0x7f60040a8d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57108/0 (socket says 172.21.15.17:57108) 2022-08-01T19:34:19.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.470+0000 7f60037fe700 1 -- 172.21.15.17:0/2896662062 learned_addr learned my addr 172.21.15.17:0/2896662062 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:19.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.470+0000 7f6002ffd700 1 --2- 172.21.15.17:0/2896662062 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f60040af550 0x7f60040b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:19.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.470+0000 7f5ffa7fc700 1 --2- 172.21.15.17:0/2896662062 >> v2:172.21.15.77:3300/0 conn(0x7f6004060330 0x7f60041445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:19.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.470+0000 7f60027fc700 1 -- 172.21.15.17:0/2896662062 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 415487061 0 0) 0x7f60040aa630 con 0x7f6004144ad0 2022-08-01T19:34:19.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.470+0000 7f60027fc700 1 -- 172.21.15.17:0/2896662062 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fe4003260 con 0x7f6004144ad0 2022-08-01T19:34:19.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.470+0000 7f6002ffd700 1 -- 172.21.15.17:0/2896662062 >> v2:172.21.15.77:3300/0 conn(0x7f6004060330 msgr2=0x7f60041445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:19.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.470+0000 7f6002ffd700 1 --2- 172.21.15.17:0/2896662062 >> v2:172.21.15.77:3300/0 conn(0x7f6004060330 0x7f60041445e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:19.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.470+0000 7f6002ffd700 1 -- 172.21.15.17:0/2896662062 >> v1:172.21.15.17:6789/0 conn(0x7f6004144ad0 legacy=0x7f60040a8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:19.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.470+0000 7f6002ffd700 1 -- 172.21.15.17:0/2896662062 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600405fd10 con 0x7f60040af550 2022-08-01T19:34:19.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.470+0000 7f6002ffd700 1 --2- 172.21.15.17:0/2896662062 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f60040af550 0x7f60040b0e40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f5ff4002f70 tx=0x7f5ff4006df0).ready entity=mon.2 client_cookie=6b70f4155b8f452b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:19.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.470+0000 7f60027fc700 1 -- 172.21.15.17:0/2896662062 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ff400d4a0 con 0x7f60040af550 2022-08-01T19:34:19.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.470+0000 7f60027fc700 1 -- 172.21.15.17:0/2896662062 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff4000e90 con 0x7f60040af550 2022-08-01T19:34:19.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.470+0000 7f60027fc700 1 -- 172.21.15.17:0/2896662062 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ff400d7f0 con 0x7f60040af550 2022-08-01T19:34:19.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.470+0000 7f6003fff700 1 -- 172.21.15.17:0/2896662062 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f60040af550 msgr2=0x7f60040b0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:19.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.470+0000 7f6003fff700 1 --2- 172.21.15.17:0/2896662062 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f60040af550 0x7f60040b0e40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f5ff4002f70 tx=0x7f5ff4006df0).stop 2022-08-01T19:34:19.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.470+0000 7f6003fff700 1 -- 172.21.15.17:0/2896662062 shutdown_connections 2022-08-01T19:34:19.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.470+0000 7f6003fff700 1 --2- 172.21.15.17:0/2896662062 >> v2:172.21.15.77:3300/0 conn(0x7f6004060330 0x7f60041445e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:19.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.470+0000 7f6003fff700 1 --2- 172.21.15.17:0/2896662062 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f60040af550 0x7f60040b0e40 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:19.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.470+0000 7f6003fff700 1 -- 172.21.15.17:0/2896662062 >> 172.21.15.17:0/2896662062 conn(0x7f60040612d0 msgr2=0x7f6004062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:19.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.470+0000 7f6003fff700 1 -- 172.21.15.17:0/2896662062 shutdown_connections 2022-08-01T19:34:19.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.470+0000 7f6003fff700 1 -- 172.21.15.17:0/2896662062 wait complete. 2022-08-01T19:34:19.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.474+0000 7f6003fff700 1 Processor -- start 2022-08-01T19:34:19.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.474+0000 7f6003fff700 1 -- start start 2022-08-01T19:34:19.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.474+0000 7f6003fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6004060330 0x7f600413fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:19.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.474+0000 7f6003fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60040af550 0x7f60040ae580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:19.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.474+0000 7f6003fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6004144ad0 0x7f600413ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:19.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.474+0000 7f6003fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6004067a00 con 0x7f60040af550 2022-08-01T19:34:19.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.474+0000 7f6003fff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60041444f0 con 0x7f6004144ad0 2022-08-01T19:34:19.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.474+0000 7f6003fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6004067760 con 0x7f6004060330 2022-08-01T19:34:19.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.474+0000 7f5ffa7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60040af550 0x7f60040ae580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:19.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.474+0000 7f6002ffd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6004060330 0x7f600413fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:19.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.474+0000 7f5ffa7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60040af550 0x7f60040ae580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36686/0 (socket says 172.21.15.17:36686) 2022-08-01T19:34:19.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.474+0000 7f5ffa7fc700 1 -- 172.21.15.17:0/2917146214 learned_addr learned my addr 172.21.15.17:0/2917146214 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:19.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.474+0000 7f60037fe700 1 --2- 172.21.15.17:0/2917146214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6004144ad0 0x7f600413ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:19.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.474+0000 7f5ffa7fc700 1 -- 172.21.15.17:0/2917146214 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6004060330 msgr2=0x7f600413fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:19.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.474+0000 7f5ffa7fc700 1 --2- 172.21.15.17:0/2917146214 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6004060330 0x7f600413fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:19.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.474+0000 7f5ffa7fc700 1 -- 172.21.15.17:0/2917146214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6004144ad0 msgr2=0x7f600413ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:19.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.474+0000 7f5ffa7fc700 1 --2- 172.21.15.17:0/2917146214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6004144ad0 0x7f600413ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:19.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.474+0000 7f5ffa7fc700 1 -- 172.21.15.17:0/2917146214 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff400b180 con 0x7f60040af550 2022-08-01T19:34:19.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.474+0000 7f5ffa7fc700 1 --2- 172.21.15.17:0/2917146214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60040af550 0x7f60040ae580 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f6004067b40 tx=0x7f5fec006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:19.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.474+0000 7f6000ff9700 1 -- 172.21.15.17:0/2917146214 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fec010780 con 0x7f60040af550 2022-08-01T19:34:19.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.478+0000 7f6000ff9700 1 -- 172.21.15.17:0/2917146214 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fec0108e0 con 0x7f60040af550 2022-08-01T19:34:19.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.478+0000 7f6003fff700 1 -- 172.21.15.17:0/2917146214 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f600405f840 con 0x7f60040af550 2022-08-01T19:34:19.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.478+0000 7f6000ff9700 1 -- 172.21.15.17:0/2917146214 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fec010c40 con 0x7f60040af550 2022-08-01T19:34:19.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.478+0000 7f6003fff700 1 -- 172.21.15.17:0/2917146214 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6004141ce0 con 0x7f60040af550 2022-08-01T19:34:19.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.478+0000 7f6003fff700 1 -- 172.21.15.17:0/2917146214 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f600405fa70 con 0x7f60040af550 2022-08-01T19:34:19.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.478+0000 7f6000ff9700 1 -- 172.21.15.17:0/2917146214 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5fec01c460 con 0x7f60040af550 2022-08-01T19:34:19.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.478+0000 7f6000ff9700 1 --2- 172.21.15.17:0/2917146214 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5fe8053950 0x7f5fe8055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:19.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.478+0000 7f6000ff9700 1 -- 172.21.15.17:0/2917146214 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f5fec06c5e0 con 0x7f60040af550 2022-08-01T19:34:19.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.478+0000 7f6002ffd700 1 --2- 172.21.15.17:0/2917146214 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5fe8053950 0x7f5fe8055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:19.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.482+0000 7f6002ffd700 1 --2- 172.21.15.17:0/2917146214 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5fe8053950 0x7f5fe8055e10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f5ff401a000 tx=0x7f5ff4014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:19.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.490+0000 7f6000ff9700 1 -- 172.21.15.17:0/2917146214 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5fec0431b0 con 0x7f60040af550 2022-08-01T19:34:19.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.662+0000 7f6003fff700 1 -- 172.21.15.17:0/2917146214 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f60040574e0 con 0x7f60040af550 2022-08-01T19:34:19.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.662+0000 7f6000ff9700 1 -- 172.21.15.17:0/2917146214 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7f5fec0467d0 con 0x7f60040af550 2022-08-01T19:34:19.668 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:19.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.662+0000 7f6003fff700 1 -- 172.21.15.17:0/2917146214 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5fe8053950 msgr2=0x7f5fe8055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:19.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.662+0000 7f6003fff700 1 --2- 172.21.15.17:0/2917146214 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5fe8053950 0x7f5fe8055e10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f5ff401a000 tx=0x7f5ff4014000).stop 2022-08-01T19:34:19.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.662+0000 7f6003fff700 1 -- 172.21.15.17:0/2917146214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60040af550 msgr2=0x7f60040ae580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:19.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.662+0000 7f6003fff700 1 --2- 172.21.15.17:0/2917146214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60040af550 0x7f60040ae580 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f6004067b40 tx=0x7f5fec006df0).stop 2022-08-01T19:34:19.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.662+0000 7f6003fff700 1 -- 172.21.15.17:0/2917146214 shutdown_connections 2022-08-01T19:34:19.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.662+0000 7f6003fff700 1 --2- 172.21.15.17:0/2917146214 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5fe8053950 0x7f5fe8055e10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:19.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.662+0000 7f6003fff700 1 --2- 172.21.15.17:0/2917146214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6004144ad0 0x7f600413ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:19.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.662+0000 7f6003fff700 1 --2- 172.21.15.17:0/2917146214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60040af550 0x7f60040ae580 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:19.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.662+0000 7f6003fff700 1 --2- 172.21.15.17:0/2917146214 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6004060330 0x7f600413fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:19.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.662+0000 7f6003fff700 1 -- 172.21.15.17:0/2917146214 >> 172.21.15.17:0/2917146214 conn(0x7f60040612d0 msgr2=0x7f60040b6390 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:19.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.662+0000 7f6003fff700 1 -- 172.21.15.17:0/2917146214 shutdown_connections 2022-08-01T19:34:19.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.662+0000 7f6003fff700 1 -- 172.21.15.17:0/2917146214 wait complete. 2022-08-01T19:34:19.680 INFO:teuthology.orchestra.run.smithi017.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2022-08-01T19:34:19.681 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2022-08-01T19:34:19.681 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:34:19.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.782+0000 7f77a67b2700 1 Processor -- start 2022-08-01T19:34:19.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.782+0000 7f77a67b2700 1 -- start start 2022-08-01T19:34:19.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.782+0000 7f77a67b2700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f77a00b65c0 0x7f77a0061600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:19.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.782+0000 7f77a67b2700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f77a0147d80 0x7f77a0144d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:19.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.782+0000 7f77a67b2700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77a00b2100 con 0x7f77a00b41f0 2022-08-01T19:34:19.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.782+0000 7f77a67b2700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77a00b1630 con 0x7f77a0147d80 2022-08-01T19:34:19.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.782+0000 7f77a67b2700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77a00b69a0 con 0x7f77a00b65c0 2022-08-01T19:34:19.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.782+0000 7f77a5fb1700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f77a0147d80 0x7f77a0144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:19.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.782+0000 7f77a57b0700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f77a00b65c0 0x7f77a0061600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:19.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.782+0000 7f77a5fb1700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f77a0147d80 0x7f77a0144d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36696/0 (socket says 172.21.15.17:36696) 2022-08-01T19:34:19.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.782+0000 7f77a5fb1700 1 -- 172.21.15.17:0/1695161358 learned_addr learned my addr 172.21.15.17:0/1695161358 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:19.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.782+0000 7f77a5fb1700 1 -- 172.21.15.17:0/1695161358 >> v2:172.21.15.77:3300/0 conn(0x7f77a00b65c0 msgr2=0x7f77a0061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:19.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.782+0000 7f77a5fb1700 1 --2- 172.21.15.17:0/1695161358 >> v2:172.21.15.77:3300/0 conn(0x7f77a00b65c0 0x7f77a0061600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:19.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.782+0000 7f77a5fb1700 1 -- 172.21.15.17:0/1695161358 >> v1:172.21.15.77:6789/0 conn(0x7f77a00b41f0 legacy=0x7f77a0144630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:19.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.782+0000 7f7793fff700 1 -- 172.21.15.17:0/1695161358 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3653920871 0 0) 0x7f77a00b2100 con 0x7f77a00b41f0 2022-08-01T19:34:19.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.782+0000 7f77a5fb1700 1 -- 172.21.15.17:0/1695161358 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a01482a0 con 0x7f77a0147d80 2022-08-01T19:34:19.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.782+0000 7f77a5fb1700 1 --2- 172.21.15.17:0/1695161358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77a0147d80 0x7f77a0144d40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f7798002f70 tx=0x7f7798006eb0).ready entity=mon.0 client_cookie=681bf7f2e7f8b847 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:19.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.782+0000 7f7793fff700 1 -- 172.21.15.17:0/1695161358 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f779800d5c0 con 0x7f77a0147d80 2022-08-01T19:34:19.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.782+0000 7f7793fff700 1 -- 172.21.15.17:0/1695161358 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7798000e90 con 0x7f77a0147d80 2022-08-01T19:34:19.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.782+0000 7f7793fff700 1 -- 172.21.15.17:0/1695161358 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f779800d910 con 0x7f77a0147d80 2022-08-01T19:34:19.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.786+0000 7f77a67b2700 1 -- 172.21.15.17:0/1695161358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77a0147d80 msgr2=0x7f77a0144d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:19.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.786+0000 7f77a67b2700 1 --2- 172.21.15.17:0/1695161358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77a0147d80 0x7f77a0144d40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f7798002f70 tx=0x7f7798006eb0).stop 2022-08-01T19:34:19.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.786+0000 7f77a67b2700 1 -- 172.21.15.17:0/1695161358 shutdown_connections 2022-08-01T19:34:19.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.786+0000 7f77a67b2700 1 --2- 172.21.15.17:0/1695161358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77a0147d80 0x7f77a0144d40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:19.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.786+0000 7f77a67b2700 1 --2- 172.21.15.17:0/1695161358 >> v2:172.21.15.77:3300/0 conn(0x7f77a00b65c0 0x7f77a0061600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:19.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.786+0000 7f77a67b2700 1 -- 172.21.15.17:0/1695161358 >> 172.21.15.17:0/1695161358 conn(0x7f77a005c620 msgr2=0x7f77a005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:19.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.786+0000 7f77a67b2700 1 -- 172.21.15.17:0/1695161358 shutdown_connections 2022-08-01T19:34:19.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.786+0000 7f77a67b2700 1 -- 172.21.15.17:0/1695161358 wait complete. 2022-08-01T19:34:19.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.786+0000 7f77a67b2700 1 Processor -- start 2022-08-01T19:34:19.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.786+0000 7f77a67b2700 1 -- start start 2022-08-01T19:34:19.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.786+0000 7f77a67b2700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f77a00b41f0 0x7f77a00af590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:19.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.786+0000 7f77a67b2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77a00b65c0 0x7f77a015a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:19.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.786+0000 7f77a67b2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77a0147d80 0x7f77a01496b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:19.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.786+0000 7f77a67b2700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a014de70 con 0x7f77a00b65c0 2022-08-01T19:34:19.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.786+0000 7f77a67b2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a0062e00 con 0x7f77a0147d80 2022-08-01T19:34:19.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.786+0000 7f77a67b2700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a014e6f0 con 0x7f77a00b41f0 2022-08-01T19:34:19.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.786+0000 7f77a4faf700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77a00b65c0 0x7f77a015a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:19.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.786+0000 7f77a57b0700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f77a00b41f0 0x7f77a00af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:19.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.786+0000 7f77a4faf700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77a00b65c0 0x7f77a015a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36700/0 (socket says 172.21.15.17:36700) 2022-08-01T19:34:19.802 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.786+0000 7f77a4faf700 1 -- 172.21.15.17:0/3233435110 learned_addr learned my addr 172.21.15.17:0/3233435110 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:19.802 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.790+0000 7f77a5fb1700 1 --2- 172.21.15.17:0/3233435110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77a0147d80 0x7f77a01496b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:19.802 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.790+0000 7f77a4faf700 1 -- 172.21.15.17:0/3233435110 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f77a00b41f0 msgr2=0x7f77a00af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:19.802 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.790+0000 7f77a4faf700 1 --2- 172.21.15.17:0/3233435110 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f77a00b41f0 0x7f77a00af590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:19.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.790+0000 7f77a4faf700 1 -- 172.21.15.17:0/3233435110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77a0147d80 msgr2=0x7f77a01496b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:19.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.790+0000 7f77a4faf700 1 --2- 172.21.15.17:0/3233435110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77a0147d80 0x7f77a01496b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:19.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.790+0000 7f77a4faf700 1 -- 172.21.15.17:0/3233435110 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f779800b1e0 con 0x7f77a00b65c0 2022-08-01T19:34:19.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.790+0000 7f77a4faf700 1 --2- 172.21.15.17:0/3233435110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77a00b65c0 0x7f77a015a780 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f7794001250 tx=0x7f7794007e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:19.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.790+0000 7f7792ffd700 1 -- 172.21.15.17:0/3233435110 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7794010510 con 0x7f77a00b65c0 2022-08-01T19:34:19.804 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.790+0000 7f7792ffd700 1 -- 172.21.15.17:0/3233435110 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7794010670 con 0x7f77a00b65c0 2022-08-01T19:34:19.804 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.790+0000 7f77a67b2700 1 -- 172.21.15.17:0/3233435110 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a014e920 con 0x7f77a00b65c0 2022-08-01T19:34:19.804 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.790+0000 7f7792ffd700 1 -- 172.21.15.17:0/3233435110 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7794010820 con 0x7f77a00b65c0 2022-08-01T19:34:19.804 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.790+0000 7f77a67b2700 1 -- 172.21.15.17:0/3233435110 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a0149ba0 con 0x7f77a00b65c0 2022-08-01T19:34:19.804 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.790+0000 7f77a67b2700 1 -- 172.21.15.17:0/3233435110 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77a00b0110 con 0x7f77a00b65c0 2022-08-01T19:34:19.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.790+0000 7f7792ffd700 1 -- 172.21.15.17:0/3233435110 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7794010070 con 0x7f77a00b65c0 2022-08-01T19:34:19.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.790+0000 7f7792ffd700 1 --2- 172.21.15.17:0/3233435110 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f777c053950 0x7f777c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:19.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.790+0000 7f7792ffd700 1 -- 172.21.15.17:0/3233435110 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f779406be60 con 0x7f77a00b65c0 2022-08-01T19:34:19.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.790+0000 7f77a57b0700 1 --2- 172.21.15.17:0/3233435110 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f777c053950 0x7f777c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:19.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.794+0000 7f77a57b0700 1 --2- 172.21.15.17:0/3233435110 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f777c053950 0x7f777c055e10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f779c00b000 tx=0x7f779c006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:19.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.798+0000 7f7792ffd700 1 -- 172.21.15.17:0/3233435110 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f77a00b0110 con 0x7f77a00b65c0 2022-08-01T19:34:19.927 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.922+0000 7f77a67b2700 1 -- 172.21.15.17:0/3233435110 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77a00b0110 con 0x7f77a00b65c0 2022-08-01T19:34:19.929 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.922+0000 7f7792ffd700 1 -- 172.21.15.17:0/3233435110 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f77a00b0110 con 0x7f77a00b65c0 2022-08-01T19:34:19.930 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:19.930 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:34:09.692404+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:34:04.635473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-08-01T19:34:07.409434+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:34:19.932 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.926+0000 7f77a67b2700 1 -- 172.21.15.17:0/3233435110 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f777c053950 msgr2=0x7f777c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:19.932 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.926+0000 7f77a67b2700 1 --2- 172.21.15.17:0/3233435110 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f777c053950 0x7f777c055e10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f779c00b000 tx=0x7f779c006df0).stop 2022-08-01T19:34:19.933 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.926+0000 7f77a67b2700 1 -- 172.21.15.17:0/3233435110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77a00b65c0 msgr2=0x7f77a015a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:19.933 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.926+0000 7f77a67b2700 1 --2- 172.21.15.17:0/3233435110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77a00b65c0 0x7f77a015a780 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f7794001250 tx=0x7f7794007e40).stop 2022-08-01T19:34:19.933 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.926+0000 7f77a67b2700 1 -- 172.21.15.17:0/3233435110 shutdown_connections 2022-08-01T19:34:19.933 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.926+0000 7f77a67b2700 1 --2- 172.21.15.17:0/3233435110 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f777c053950 0x7f777c055e10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:19.934 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.926+0000 7f77a67b2700 1 --2- 172.21.15.17:0/3233435110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77a0147d80 0x7f77a01496b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:19.934 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.926+0000 7f77a67b2700 1 --2- 172.21.15.17:0/3233435110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77a00b65c0 0x7f77a015a780 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:19.934 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.926+0000 7f77a67b2700 1 --2- 172.21.15.17:0/3233435110 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f77a00b41f0 0x7f77a00af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:19.934 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.926+0000 7f77a67b2700 1 -- 172.21.15.17:0/3233435110 >> 172.21.15.17:0/3233435110 conn(0x7f77a005c620 msgr2=0x7f77a0061c70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:19.935 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.926+0000 7f77a67b2700 1 -- 172.21.15.17:0/3233435110 shutdown_connections 2022-08-01T19:34:19.935 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:19.926+0000 7f77a67b2700 1 -- 172.21.15.17:0/3233435110 wait complete. 2022-08-01T19:34:19.943 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T19:34:04.635473+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T19:34:07.409434+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T19:34:19.944 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:34:20.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.050+0000 7fb51136c700 1 Processor -- start 2022-08-01T19:34:20.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.050+0000 7fb51136c700 1 -- start start 2022-08-01T19:34:20.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.050+0000 7fb51136c700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fb50c0b3340 0x7fb50c13e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:20.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.050+0000 7fb51136c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb50c0b3720 0x7fb50c13ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:20.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.050+0000 7fb51136c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb50c145220 0x7fb50c141590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:20.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.050+0000 7fb51136c700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb50c0ada80 con 0x7fb50c145220 2022-08-01T19:34:20.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.050+0000 7fb51136c700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb50c0b0090 con 0x7fb50c0b3340 2022-08-01T19:34:20.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.050+0000 7fb51136c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c1456f0 con 0x7fb50c0b3720 2022-08-01T19:34:20.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.054+0000 7fb50b7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb50c145220 0x7fb50c141590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:20.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.054+0000 7fb50b7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb50c145220 0x7fb50c141590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36710/0 (socket says 172.21.15.17:36710) 2022-08-01T19:34:20.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.054+0000 7fb50b7fe700 1 -- 172.21.15.17:0/1178604869 learned_addr learned my addr 172.21.15.17:0/1178604869 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:20.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.054+0000 7fb50affd700 1 --2- 172.21.15.17:0/1178604869 >> v2:172.21.15.77:3300/0 conn(0x7fb50c0b3340 0x7fb50c13e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:20.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.054+0000 7fb5027fc700 1 --2- 172.21.15.17:0/1178604869 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb50c0b3720 0x7fb50c13ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:20.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.054+0000 7fb50b7fe700 1 -- 172.21.15.17:0/1178604869 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb50c0b3720 msgr2=0x7fb50c13ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:20.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.054+0000 7fb50b7fe700 1 --2- 172.21.15.17:0/1178604869 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb50c0b3720 0x7fb50c13ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.054+0000 7fb50b7fe700 1 -- 172.21.15.17:0/1178604869 >> v2:172.21.15.77:3300/0 conn(0x7fb50c0b3340 msgr2=0x7fb50c13e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:20.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.054+0000 7fb50b7fe700 1 --2- 172.21.15.17:0/1178604869 >> v2:172.21.15.77:3300/0 conn(0x7fb50c0b3340 0x7fb50c13e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.063 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.054+0000 7fb50b7fe700 1 -- 172.21.15.17:0/1178604869 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50c0b02f0 con 0x7fb50c145220 2022-08-01T19:34:20.063 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.054+0000 7fb50b7fe700 1 --2- 172.21.15.17:0/1178604869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb50c145220 0x7fb50c141590 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fb5040032f0 tx=0x7fb504009290).ready entity=mon.0 client_cookie=8fe6148a55072eaf server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:20.063 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.054+0000 7fb50a7fc700 1 -- 172.21.15.17:0/1178604869 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb50400b680 con 0x7fb50c145220 2022-08-01T19:34:20.063 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.054+0000 7fb50a7fc700 1 -- 172.21.15.17:0/1178604869 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50400b7e0 con 0x7fb50c145220 2022-08-01T19:34:20.063 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.054+0000 7fb50a7fc700 1 -- 172.21.15.17:0/1178604869 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb50400b950 con 0x7fb50c145220 2022-08-01T19:34:20.066 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.054+0000 7fb51136c700 1 -- 172.21.15.17:0/1178604869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb50c145220 msgr2=0x7fb50c141590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:20.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.054+0000 7fb51136c700 1 --2- 172.21.15.17:0/1178604869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb50c145220 0x7fb50c141590 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fb5040032f0 tx=0x7fb504009290).stop 2022-08-01T19:34:20.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.054+0000 7fb51136c700 1 -- 172.21.15.17:0/1178604869 shutdown_connections 2022-08-01T19:34:20.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.054+0000 7fb51136c700 1 --2- 172.21.15.17:0/1178604869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb50c145220 0x7fb50c141590 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.054+0000 7fb51136c700 1 --2- 172.21.15.17:0/1178604869 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb50c0b3720 0x7fb50c13ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.054+0000 7fb51136c700 1 --2- 172.21.15.17:0/1178604869 >> v2:172.21.15.77:3300/0 conn(0x7fb50c0b3340 0x7fb50c13e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.054+0000 7fb51136c700 1 -- 172.21.15.17:0/1178604869 >> 172.21.15.17:0/1178604869 conn(0x7fb50c060950 msgr2=0x7fb50c060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:20.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.054+0000 7fb51136c700 1 -- 172.21.15.17:0/1178604869 shutdown_connections 2022-08-01T19:34:20.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.054+0000 7fb51136c700 1 -- 172.21.15.17:0/1178604869 wait complete. 2022-08-01T19:34:20.069 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.058+0000 7fb51136c700 1 Processor -- start 2022-08-01T19:34:20.069 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.058+0000 7fb51136c700 1 -- start start 2022-08-01T19:34:20.069 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.058+0000 7fb51136c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb50c0b3340 0x7fb50c0a96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:20.069 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.058+0000 7fb51136c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb50c0b3720 0x7fb50c063fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:20.069 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.058+0000 7fb51136c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb50c145220 0x7fb50c064d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:20.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.058+0000 7fb51136c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c0ac260 con 0x7fb50c0b3720 2022-08-01T19:34:20.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.058+0000 7fb51136c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c0ac9d0 con 0x7fb50c0b3340 2022-08-01T19:34:20.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.058+0000 7fb51136c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c0ac490 con 0x7fb50c145220 2022-08-01T19:34:20.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.058+0000 7fb5027fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb50c0b3720 0x7fb50c063fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:20.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.058+0000 7fb50affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb50c0b3340 0x7fb50c0a96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:20.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.058+0000 7fb5027fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb50c0b3720 0x7fb50c063fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36714/0 (socket says 172.21.15.17:36714) 2022-08-01T19:34:20.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.058+0000 7fb5027fc700 1 -- 172.21.15.17:0/2631369069 learned_addr learned my addr 172.21.15.17:0/2631369069 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:20.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.058+0000 7fb50b7fe700 1 --2- 172.21.15.17:0/2631369069 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb50c145220 0x7fb50c064d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:20.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.058+0000 7fb5027fc700 1 -- 172.21.15.17:0/2631369069 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb50c145220 msgr2=0x7fb50c064d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:20.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.058+0000 7fb5027fc700 1 --2- 172.21.15.17:0/2631369069 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb50c145220 0x7fb50c064d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.058+0000 7fb5027fc700 1 -- 172.21.15.17:0/2631369069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb50c0b3340 msgr2=0x7fb50c0a96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:20.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.058+0000 7fb5027fc700 1 --2- 172.21.15.17:0/2631369069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb50c0b3340 0x7fb50c0a96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.058+0000 7fb5027fc700 1 -- 172.21.15.17:0/2631369069 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb504009530 con 0x7fb50c0b3720 2022-08-01T19:34:20.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.058+0000 7fb5027fc700 1 --2- 172.21.15.17:0/2631369069 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb50c0b3720 0x7fb50c063fa0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fb4f400c000 tx=0x7fb4f4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:20.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.062+0000 7fb508ff9700 1 -- 172.21.15.17:0/2631369069 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4f4010780 con 0x7fb50c0b3720 2022-08-01T19:34:20.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.062+0000 7fb508ff9700 1 -- 172.21.15.17:0/2631369069 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f400b3b0 con 0x7fb50c0b3720 2022-08-01T19:34:20.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.062+0000 7fb51136c700 1 -- 172.21.15.17:0/2631369069 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb50c0ac730 con 0x7fb50c0b3720 2022-08-01T19:34:20.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.062+0000 7fb508ff9700 1 -- 172.21.15.17:0/2631369069 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4f4010ad0 con 0x7fb50c0b3720 2022-08-01T19:34:20.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.062+0000 7fb51136c700 1 -- 172.21.15.17:0/2631369069 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb50c064490 con 0x7fb50c0b3720 2022-08-01T19:34:20.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.062+0000 7fb51136c700 1 -- 172.21.15.17:0/2631369069 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb50c0b0590 con 0x7fb50c0b3720 2022-08-01T19:34:20.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.062+0000 7fb508ff9700 1 -- 172.21.15.17:0/2631369069 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb4f4010c30 con 0x7fb50c0b3720 2022-08-01T19:34:20.076 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.062+0000 7fb508ff9700 1 --2- 172.21.15.17:0/2631369069 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb4f00539e0 0x7fb4f0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:20.076 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.062+0000 7fb508ff9700 1 -- 172.21.15.17:0/2631369069 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fb4f406c410 con 0x7fb50c0b3720 2022-08-01T19:34:20.076 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.062+0000 7fb50affd700 1 --2- 172.21.15.17:0/2631369069 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb4f00539e0 0x7fb4f0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:20.076 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.066+0000 7fb50affd700 1 --2- 172.21.15.17:0/2631369069 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb4f00539e0 0x7fb4f0055ea0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fb4fc00b000 tx=0x7fb4fc006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:20.077 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.070+0000 7fb508ff9700 1 -- 172.21.15.17:0/2631369069 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb4f4043130 con 0x7fb50c0b3720 2022-08-01T19:34:20.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.194+0000 7fb51136c700 1 -- 172.21.15.17:0/2631369069 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fb50c0574e0 con 0x7fb50c0b3720 2022-08-01T19:34:20.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.198+0000 7fb508ff9700 1 -- 172.21.15.17:0/2631369069 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v25) v1 ==== 110+0+10 (secure 0 0 0) 0x7fb4f4046750 con 0x7fb50c0b3720 2022-08-01T19:34:20.203 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:34:20.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.198+0000 7fb51136c700 1 -- 172.21.15.17:0/2631369069 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb4f00539e0 msgr2=0x7fb4f0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:20.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.198+0000 7fb51136c700 1 --2- 172.21.15.17:0/2631369069 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb4f00539e0 0x7fb4f0055ea0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fb4fc00b000 tx=0x7fb4fc006df0).stop 2022-08-01T19:34:20.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.198+0000 7fb51136c700 1 -- 172.21.15.17:0/2631369069 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb50c0b3720 msgr2=0x7fb50c063fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:20.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.198+0000 7fb51136c700 1 --2- 172.21.15.17:0/2631369069 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb50c0b3720 0x7fb50c063fa0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fb4f400c000 tx=0x7fb4f4006df0).stop 2022-08-01T19:34:20.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.198+0000 7fb51136c700 1 -- 172.21.15.17:0/2631369069 shutdown_connections 2022-08-01T19:34:20.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.198+0000 7fb51136c700 1 --2- 172.21.15.17:0/2631369069 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb4f00539e0 0x7fb4f0055ea0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.198+0000 7fb51136c700 1 --2- 172.21.15.17:0/2631369069 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb50c145220 0x7fb50c064d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.198+0000 7fb51136c700 1 --2- 172.21.15.17:0/2631369069 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb50c0b3720 0x7fb50c063fa0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.198+0000 7fb51136c700 1 --2- 172.21.15.17:0/2631369069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb50c0b3340 0x7fb50c0a96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.198+0000 7fb51136c700 1 -- 172.21.15.17:0/2631369069 >> 172.21.15.17:0/2631369069 conn(0x7fb50c060950 msgr2=0x7fb50c0b53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:20.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.198+0000 7fb51136c700 1 -- 172.21.15.17:0/2631369069 shutdown_connections 2022-08-01T19:34:20.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.198+0000 7fb51136c700 1 -- 172.21.15.17:0/2631369069 wait complete. 2022-08-01T19:34:20.215 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:34:20.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8983fa6700 1 Processor -- start 2022-08-01T19:34:20.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8983fa6700 1 -- start start 2022-08-01T19:34:20.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8983fa6700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f897c060960 0x7f897c140230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:20.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8983fa6700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f897c060010 con 0x7f897c05fc30 2022-08-01T19:34:20.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8983fa6700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f897c0acdf0 con 0x7f897c0ac690 2022-08-01T19:34:20.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8983fa6700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f897c1433d0 con 0x7f897c060960 2022-08-01T19:34:20.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8981d42700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f897c05fc30 0x7f897c143a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57146/0 (socket says 172.21.15.17:57146) 2022-08-01T19:34:20.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8981d42700 1 -- 172.21.15.17:0/158093992 learned_addr learned my addr 172.21.15.17:0/158093992 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:20.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8981541700 1 --2- 172.21.15.17:0/158093992 >> v2:172.21.15.77:3300/0 conn(0x7f897c060960 0x7f897c140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:20.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8980d40700 1 -- 172.21.15.17:0/158093992 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 719438410 0 0) 0x7f897c060010 con 0x7f897c05fc30 2022-08-01T19:34:20.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8980d40700 1 -- 172.21.15.17:0/158093992 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8964003260 con 0x7f897c05fc30 2022-08-01T19:34:20.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8980d40700 1 -- 172.21.15.17:0/158093992 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2802086508 0 0) 0x7f897c0acdf0 con 0x7f897c0ac690 2022-08-01T19:34:20.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8980d40700 1 -- 172.21.15.17:0/158093992 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f897c060010 con 0x7f897c0ac690 2022-08-01T19:34:20.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8980d40700 1 -- 172.21.15.17:0/158093992 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1353796643 0 0) 0x7f8964003260 con 0x7f897c05fc30 2022-08-01T19:34:20.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8980d40700 1 -- 172.21.15.17:0/158093992 >> v2:172.21.15.77:3300/0 conn(0x7f897c060960 msgr2=0x7f897c140230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:20.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8980d40700 1 --2- 172.21.15.17:0/158093992 >> v2:172.21.15.77:3300/0 conn(0x7f897c060960 0x7f897c140230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8980d40700 1 -- 172.21.15.17:0/158093992 >> v1:172.21.15.77:6789/0 conn(0x7f897c0ac690 legacy=0x7f897c140720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:20.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8980d40700 1 -- 172.21.15.17:0/158093992 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897c0ad090 con 0x7f897c05fc30 2022-08-01T19:34:20.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8980d40700 1 -- 172.21.15.17:0/158093992 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7f8978001660 con 0x7f897c05fc30 2022-08-01T19:34:20.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8980d40700 1 -- 172.21.15.17:0/158093992 >> v1:172.21.15.17:6789/0 conn(0x7f897c05fc30 legacy=0x7f897c143a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:20.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8980d40700 1 --2- 172.21.15.17:0/158093992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89640038e0 0x7f8964003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:20.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8980d40700 1 --2- 172.21.15.17:0/158093992 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8964004500 0x7f8964004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:20.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8980d40700 1 --2- 172.21.15.17:0/158093992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8964004e70 0x7f896400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:20.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8980d40700 1 -- 172.21.15.17:0/158093992 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896400bcf0 con 0x7f89640038e0 2022-08-01T19:34:20.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8980d40700 1 -- 172.21.15.17:0/158093992 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896400be30 con 0x7f8964004e70 2022-08-01T19:34:20.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8980d40700 1 -- 172.21.15.17:0/158093992 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f896400bff0 con 0x7f8964004500 2022-08-01T19:34:20.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8981d42700 1 --2- 172.21.15.17:0/158093992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89640038e0 0x7f8964003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:20.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8981541700 1 --2- 172.21.15.17:0/158093992 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8964004500 0x7f8964004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:20.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8982543700 1 --2- 172.21.15.17:0/158093992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8964004e70 0x7f896400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:20.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8981d42700 1 -- 172.21.15.17:0/158093992 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8964004500 msgr2=0x7f8964004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:20.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8981d42700 1 --2- 172.21.15.17:0/158093992 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8964004500 0x7f8964004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8981d42700 1 -- 172.21.15.17:0/158093992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8964004e70 msgr2=0x7f896400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:20.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8981d42700 1 --2- 172.21.15.17:0/158093992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8964004e70 0x7f896400b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8982543700 1 -- 172.21.15.17:0/158093992 reap_dead start 2022-08-01T19:34:20.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8981d42700 1 -- 172.21.15.17:0/158093992 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8964003260 con 0x7f89640038e0 2022-08-01T19:34:20.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8981d42700 1 --2- 172.21.15.17:0/158093992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89640038e0 0x7f8964003ce0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f8978001430 tx=0x7f897c0ad220).ready entity=mon.0 client_cookie=ee69ad257e2a0407 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:20.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8980d40700 1 -- 172.21.15.17:0/158093992 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89780070d0 con 0x7f89640038e0 2022-08-01T19:34:20.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8980d40700 1 -- 172.21.15.17:0/158093992 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8978007230 con 0x7f89640038e0 2022-08-01T19:34:20.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.322+0000 7f8980d40700 1 -- 172.21.15.17:0/158093992 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f897800c8f0 con 0x7f89640038e0 2022-08-01T19:34:20.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.326+0000 7f8983fa6700 1 -- 172.21.15.17:0/158093992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89640038e0 msgr2=0x7f8964003ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:20.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.326+0000 7f8983fa6700 1 --2- 172.21.15.17:0/158093992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89640038e0 0x7f8964003ce0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f8978001430 tx=0x7f897c0ad220).stop 2022-08-01T19:34:20.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.326+0000 7f8983fa6700 1 -- 172.21.15.17:0/158093992 shutdown_connections 2022-08-01T19:34:20.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.326+0000 7f8983fa6700 1 --2- 172.21.15.17:0/158093992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89640038e0 0x7f8964003ce0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.326+0000 7f8983fa6700 1 -- 172.21.15.17:0/158093992 >> 172.21.15.17:0/158093992 conn(0x7f897c0b2b00 msgr2=0x7f897c0b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:20.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.326+0000 7f8983fa6700 1 -- 172.21.15.17:0/158093992 shutdown_connections 2022-08-01T19:34:20.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.326+0000 7f8983fa6700 1 -- 172.21.15.17:0/158093992 wait complete. 2022-08-01T19:34:20.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.326+0000 7f8983fa6700 1 Processor -- start 2022-08-01T19:34:20.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.326+0000 7f8983fa6700 1 -- start start 2022-08-01T19:34:20.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.326+0000 7f8983fa6700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f89640038e0 0x7f897c155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:20.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.326+0000 7f8983fa6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f897c13b560 0x7f897c152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:20.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.326+0000 7f8983fa6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f897c152b70 0x7f897c1446d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:20.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.326+0000 7f8983fa6700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897c152ff0 con 0x7f897c152b70 2022-08-01T19:34:20.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.326+0000 7f8983fa6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897c155fe0 con 0x7f897c13b560 2022-08-01T19:34:20.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.326+0000 7f8983fa6700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f897c13f770 con 0x7f89640038e0 2022-08-01T19:34:20.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.326+0000 7f8982543700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f897c152b70 0x7f897c1446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:20.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.326+0000 7f8982543700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f897c152b70 0x7f897c1446d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36734/0 (socket says 172.21.15.17:36734) 2022-08-01T19:34:20.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.326+0000 7f8982543700 1 -- 172.21.15.17:0/539122263 learned_addr learned my addr 172.21.15.17:0/539122263 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:20.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.326+0000 7f8981d42700 1 --2- 172.21.15.17:0/539122263 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f89640038e0 0x7f897c155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:20.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.326+0000 7f8981541700 1 --2- 172.21.15.17:0/539122263 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f897c13b560 0x7f897c152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:20.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.330+0000 7f8982543700 1 -- 172.21.15.17:0/539122263 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f89640038e0 msgr2=0x7f897c155af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:20.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.330+0000 7f8982543700 1 --2- 172.21.15.17:0/539122263 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f89640038e0 0x7f897c155af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.330+0000 7f8982543700 1 -- 172.21.15.17:0/539122263 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f897c13b560 msgr2=0x7f897c152680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:20.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.330+0000 7f8982543700 1 --2- 172.21.15.17:0/539122263 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f897c13b560 0x7f897c152680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.330+0000 7f8982543700 1 -- 172.21.15.17:0/539122263 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897c153250 con 0x7f897c152b70 2022-08-01T19:34:20.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.330+0000 7f8982543700 1 --2- 172.21.15.17:0/539122263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f897c152b70 0x7f897c1446d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f8974001550 tx=0x7f8974009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:20.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.330+0000 7f896effd700 1 -- 172.21.15.17:0/539122263 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8974003220 con 0x7f897c152b70 2022-08-01T19:34:20.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.330+0000 7f896effd700 1 -- 172.21.15.17:0/539122263 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8974003380 con 0x7f897c152b70 2022-08-01T19:34:20.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.330+0000 7f896effd700 1 -- 172.21.15.17:0/539122263 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89740016e0 con 0x7f897c152b70 2022-08-01T19:34:20.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.330+0000 7f8983fa6700 1 -- 172.21.15.17:0/539122263 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f897c13b0c0 con 0x7f897c152b70 2022-08-01T19:34:20.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.330+0000 7f8983fa6700 1 -- 172.21.15.17:0/539122263 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f897c13b9e0 con 0x7f897c152b70 2022-08-01T19:34:20.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.330+0000 7f8983fa6700 1 -- 172.21.15.17:0/539122263 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f897c0574e0 con 0x7f897c152b70 2022-08-01T19:34:20.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.330+0000 7f896effd700 1 -- 172.21.15.17:0/539122263 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f897401e070 con 0x7f897c152b70 2022-08-01T19:34:20.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.330+0000 7f896effd700 1 --2- 172.21.15.17:0/539122263 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f895c053b30 0x7f895c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:20.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.330+0000 7f896effd700 1 -- 172.21.15.17:0/539122263 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f897406b5e0 con 0x7f897c152b70 2022-08-01T19:34:20.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.330+0000 7f8981d42700 1 --2- 172.21.15.17:0/539122263 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f895c053b30 0x7f895c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:20.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.334+0000 7f8981d42700 1 --2- 172.21.15.17:0/539122263 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f895c053b30 0x7f895c055ff0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f8978006000 tx=0x7f8978015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:20.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.338+0000 7f896effd700 1 -- 172.21.15.17:0/539122263 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f89740423a0 con 0x7f897c152b70 2022-08-01T19:34:20.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.462+0000 7f8983fa6700 1 -- 172.21.15.17:0/539122263 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f897c0447e0 con 0x7f897c152b70 2022-08-01T19:34:20.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.462+0000 7f896effd700 1 -- 172.21.15.17:0/539122263 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f89740459c0 con 0x7f897c152b70 2022-08-01T19:34:20.469 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:34:20.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.466+0000 7f8983fa6700 1 -- 172.21.15.17:0/539122263 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f895c053b30 msgr2=0x7f895c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:20.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.466+0000 7f8983fa6700 1 --2- 172.21.15.17:0/539122263 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f895c053b30 0x7f895c055ff0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f8978006000 tx=0x7f8978015000).stop 2022-08-01T19:34:20.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.466+0000 7f8983fa6700 1 -- 172.21.15.17:0/539122263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f897c152b70 msgr2=0x7f897c1446d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:20.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.466+0000 7f8983fa6700 1 --2- 172.21.15.17:0/539122263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f897c152b70 0x7f897c1446d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f8974001550 tx=0x7f8974009040).stop 2022-08-01T19:34:20.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.466+0000 7f8983fa6700 1 -- 172.21.15.17:0/539122263 shutdown_connections 2022-08-01T19:34:20.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.466+0000 7f8983fa6700 1 --2- 172.21.15.17:0/539122263 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f895c053b30 0x7f895c055ff0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.466+0000 7f8983fa6700 1 --2- 172.21.15.17:0/539122263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f897c152b70 0x7f897c1446d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.466+0000 7f8983fa6700 1 --2- 172.21.15.17:0/539122263 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f897c13b560 0x7f897c152680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.466+0000 7f8983fa6700 1 --2- 172.21.15.17:0/539122263 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f89640038e0 0x7f897c155af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.466+0000 7f8983fa6700 1 -- 172.21.15.17:0/539122263 >> 172.21.15.17:0/539122263 conn(0x7f897c0b2b00 msgr2=0x7f897c143980 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:20.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.466+0000 7f8983fa6700 1 -- 172.21.15.17:0/539122263 shutdown_connections 2022-08-01T19:34:20.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.466+0000 7f8983fa6700 1 -- 172.21.15.17:0/539122263 wait complete. 2022-08-01T19:34:20.483 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T19:34:20.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.586+0000 7fca0f2b6700 1 Processor -- start 2022-08-01T19:34:20.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.586+0000 7fca0f2b6700 1 -- start start 2022-08-01T19:34:20.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.586+0000 7fca0f2b6700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fca08066780 0x7fca08066b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:20.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.586+0000 7fca0f2b6700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fca080b1bc0 0x7fca0805ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:20.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.586+0000 7fca0f2b6700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fca08140e40 0x7fca0813e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:20.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.586+0000 7fca0f2b6700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca0805f8e0 con 0x7fca080b1bc0 2022-08-01T19:34:20.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.586+0000 7fca0f2b6700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca080b1630 con 0x7fca08140e40 2022-08-01T19:34:20.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.586+0000 7fca0f2b6700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca0805fb10 con 0x7fca08066780 2022-08-01T19:34:20.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.586+0000 7fca0dab3700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fca080b1bc0 0x7fca0805ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:20.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.586+0000 7fca0dab3700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fca080b1bc0 0x7fca0805ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36740/0 (socket says 172.21.15.17:36740) 2022-08-01T19:34:20.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.586+0000 7fca0dab3700 1 -- 172.21.15.17:0/387434336 learned_addr learned my addr 172.21.15.17:0/387434336 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:20.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.586+0000 7fca0e2b4700 1 --2- 172.21.15.17:0/387434336 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fca08066780 0x7fca08066b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:20.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.586+0000 7fca0eab5700 1 --2- 172.21.15.17:0/387434336 >> v2:172.21.15.77:3300/0 conn(0x7fca08140e40 0x7fca0813e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:20.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.586+0000 7fca0dab3700 1 -- 172.21.15.17:0/387434336 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fca08066780 msgr2=0x7fca08066b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:20.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.586+0000 7fca0dab3700 1 --2- 172.21.15.17:0/387434336 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fca08066780 0x7fca08066b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.586+0000 7fca0dab3700 1 -- 172.21.15.17:0/387434336 >> v2:172.21.15.77:3300/0 conn(0x7fca08140e40 msgr2=0x7fca0813e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:20.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.586+0000 7fca0dab3700 1 --2- 172.21.15.17:0/387434336 >> v2:172.21.15.77:3300/0 conn(0x7fca08140e40 0x7fca0813e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.586+0000 7fca0dab3700 1 -- 172.21.15.17:0/387434336 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca080604a0 con 0x7fca080b1bc0 2022-08-01T19:34:20.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.586+0000 7fca0eab5700 1 --2- 172.21.15.17:0/387434336 >> v2:172.21.15.77:3300/0 conn(0x7fca08140e40 0x7fca0813e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:34:20.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.586+0000 7fca0dab3700 1 --2- 172.21.15.17:0/387434336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca080b1bc0 0x7fca0805ffb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fc9fc002f70 tx=0x7fc9fc006eb0).ready entity=mon.0 client_cookie=75e2ef10b3f08f15 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:20.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.586+0000 7fca0d2b2700 1 -- 172.21.15.17:0/387434336 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9fc00d570 con 0x7fca080b1bc0 2022-08-01T19:34:20.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.586+0000 7fca0d2b2700 1 -- 172.21.15.17:0/387434336 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9fc000e90 con 0x7fca080b1bc0 2022-08-01T19:34:20.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.586+0000 7fca0d2b2700 1 -- 172.21.15.17:0/387434336 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9fc00d900 con 0x7fca080b1bc0 2022-08-01T19:34:20.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.586+0000 7fca0f2b6700 1 -- 172.21.15.17:0/387434336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca080b1bc0 msgr2=0x7fca0805ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:20.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.586+0000 7fca0f2b6700 1 --2- 172.21.15.17:0/387434336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca080b1bc0 0x7fca0805ffb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fc9fc002f70 tx=0x7fc9fc006eb0).stop 2022-08-01T19:34:20.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.586+0000 7fca0f2b6700 1 -- 172.21.15.17:0/387434336 shutdown_connections 2022-08-01T19:34:20.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.586+0000 7fca0f2b6700 1 --2- 172.21.15.17:0/387434336 >> v2:172.21.15.77:3300/0 conn(0x7fca08140e40 0x7fca0813e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.586+0000 7fca0f2b6700 1 --2- 172.21.15.17:0/387434336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca080b1bc0 0x7fca0805ffb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.586+0000 7fca0f2b6700 1 --2- 172.21.15.17:0/387434336 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fca08066780 0x7fca08066b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.586+0000 7fca0f2b6700 1 -- 172.21.15.17:0/387434336 >> 172.21.15.17:0/387434336 conn(0x7fca08063870 msgr2=0x7fca08063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:20.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.586+0000 7fca0f2b6700 1 -- 172.21.15.17:0/387434336 shutdown_connections 2022-08-01T19:34:20.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.586+0000 7fca0f2b6700 1 -- 172.21.15.17:0/387434336 wait complete. 2022-08-01T19:34:20.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.586+0000 7fca0f2b6700 1 Processor -- start 2022-08-01T19:34:20.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.586+0000 7fca0f2b6700 1 -- start start 2022-08-01T19:34:20.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.590+0000 7fca0f2b6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca08066780 0x7fca080b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:20.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.590+0000 7fca0f2b6700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fca080b1bc0 0x7fca08150540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:20.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.590+0000 7fca0f2b6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca08140e40 0x7fca08150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:20.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.590+0000 7fca0f2b6700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca081543d0 con 0x7fca08140e40 2022-08-01T19:34:20.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.590+0000 7fca0f2b6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca08142da0 con 0x7fca08066780 2022-08-01T19:34:20.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.590+0000 7fca0f2b6700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca08153eb0 con 0x7fca080b1bc0 2022-08-01T19:34:20.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.590+0000 7fca0eab5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca08140e40 0x7fca08150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:20.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.590+0000 7fca0eab5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca08140e40 0x7fca08150a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36746/0 (socket says 172.21.15.17:36746) 2022-08-01T19:34:20.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.590+0000 7fca0eab5700 1 -- 172.21.15.17:0/1560459427 learned_addr learned my addr 172.21.15.17:0/1560459427 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:20.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.590+0000 7fca0e2b4700 1 --2- 172.21.15.17:0/1560459427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca08066780 0x7fca080b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:20.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.590+0000 7fca0dab3700 1 --2- 172.21.15.17:0/1560459427 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fca080b1bc0 0x7fca08150540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:20.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.590+0000 7fca0eab5700 1 -- 172.21.15.17:0/1560459427 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fca080b1bc0 msgr2=0x7fca08150540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:20.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.590+0000 7fca0eab5700 1 --2- 172.21.15.17:0/1560459427 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fca080b1bc0 0x7fca08150540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.590+0000 7fca0eab5700 1 -- 172.21.15.17:0/1560459427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca08066780 msgr2=0x7fca080b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:20.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.590+0000 7fca0eab5700 1 --2- 172.21.15.17:0/1560459427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca08066780 0x7fca080b5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.590+0000 7fca0eab5700 1 -- 172.21.15.17:0/1560459427 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9fc00b1e0 con 0x7fca08140e40 2022-08-01T19:34:20.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.590+0000 7fca0e2b4700 1 --2- 172.21.15.17:0/1560459427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca08066780 0x7fca080b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:34:20.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.590+0000 7fca0dab3700 1 --2- 172.21.15.17:0/1560459427 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fca080b1bc0 0x7fca08150540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:34:20.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.590+0000 7fca0eab5700 1 --2- 172.21.15.17:0/1560459427 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca08140e40 0x7fca08150a30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fc9f8007000 tx=0x7fc9f8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:20.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.590+0000 7fca077fe700 1 -- 172.21.15.17:0/1560459427 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9f800f4e0 con 0x7fca08140e40 2022-08-01T19:34:20.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.590+0000 7fca077fe700 1 -- 172.21.15.17:0/1560459427 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f800f640 con 0x7fca08140e40 2022-08-01T19:34:20.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.590+0000 7fca0f2b6700 1 -- 172.21.15.17:0/1560459427 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca08154130 con 0x7fca08140e40 2022-08-01T19:34:20.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.590+0000 7fca077fe700 1 -- 172.21.15.17:0/1560459427 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9f800f9a0 con 0x7fca08140e40 2022-08-01T19:34:20.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.590+0000 7fca0f2b6700 1 -- 172.21.15.17:0/1560459427 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca081427e0 con 0x7fca08140e40 2022-08-01T19:34:20.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.590+0000 7fca0f2b6700 1 -- 172.21.15.17:0/1560459427 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca08143430 con 0x7fca08140e40 2022-08-01T19:34:20.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.594+0000 7fca077fe700 1 -- 172.21.15.17:0/1560459427 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc9f8019030 con 0x7fca08140e40 2022-08-01T19:34:20.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.594+0000 7fca077fe700 1 --2- 172.21.15.17:0/1560459427 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9e8053a70 0x7fc9e8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:20.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.594+0000 7fca077fe700 1 -- 172.21.15.17:0/1560459427 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fc9f806bb20 con 0x7fca08140e40 2022-08-01T19:34:20.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.594+0000 7fca0e2b4700 1 --2- 172.21.15.17:0/1560459427 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9e8053a70 0x7fc9e8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:20.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.594+0000 7fca0e2b4700 1 --2- 172.21.15.17:0/1560459427 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9e8053a70 0x7fc9e8055f30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fca0000d000 tx=0x7fca00008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:20.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.606+0000 7fca077fe700 1 -- 172.21.15.17:0/1560459427 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc9f80427d0 con 0x7fca08140e40 2022-08-01T19:34:20.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.726+0000 7fca0f2b6700 1 -- 172.21.15.17:0/1560459427 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fca080b18d0 con 0x7fca08140e40 2022-08-01T19:34:20.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.730+0000 7fca077fe700 1 -- 172.21.15.17:0/1560459427 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+10 (secure 0 0 0) 0x7fc9f8045df0 con 0x7fca08140e40 2022-08-01T19:34:20.735 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:34:20.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.730+0000 7fca0f2b6700 1 -- 172.21.15.17:0/1560459427 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9e8053a70 msgr2=0x7fc9e8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:20.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.730+0000 7fca0f2b6700 1 --2- 172.21.15.17:0/1560459427 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9e8053a70 0x7fc9e8055f30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fca0000d000 tx=0x7fca00008040).stop 2022-08-01T19:34:20.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.730+0000 7fca0f2b6700 1 -- 172.21.15.17:0/1560459427 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca08140e40 msgr2=0x7fca08150a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:20.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.730+0000 7fca0f2b6700 1 --2- 172.21.15.17:0/1560459427 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca08140e40 0x7fca08150a30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fc9f8007000 tx=0x7fc9f8008040).stop 2022-08-01T19:34:20.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.730+0000 7fca0f2b6700 1 -- 172.21.15.17:0/1560459427 shutdown_connections 2022-08-01T19:34:20.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.730+0000 7fca0f2b6700 1 --2- 172.21.15.17:0/1560459427 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9e8053a70 0x7fc9e8055f30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.730+0000 7fca0f2b6700 1 --2- 172.21.15.17:0/1560459427 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca08140e40 0x7fca08150a30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.730+0000 7fca0f2b6700 1 --2- 172.21.15.17:0/1560459427 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fca080b1bc0 0x7fca08150540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.730+0000 7fca0f2b6700 1 --2- 172.21.15.17:0/1560459427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca08066780 0x7fca080b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.730+0000 7fca0f2b6700 1 -- 172.21.15.17:0/1560459427 >> 172.21.15.17:0/1560459427 conn(0x7fca08063870 msgr2=0x7fca0813c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:20.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.730+0000 7fca0f2b6700 1 -- 172.21.15.17:0/1560459427 shutdown_connections 2022-08-01T19:34:20.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.730+0000 7fca0f2b6700 1 -- 172.21.15.17:0/1560459427 wait complete. 2022-08-01T19:34:20.753 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T19:34:20.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.854+0000 7f968ca0c700 1 Processor -- start 2022-08-01T19:34:20.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.854+0000 7f968ca0c700 1 -- start start 2022-08-01T19:34:20.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.854+0000 7f968ca0c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f96880b2710 0x7f968814afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:20.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.854+0000 7f968ca0c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96880b01a0 0x7f968814b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:20.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.854+0000 7f968ca0c700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96880a7250 con 0x7f96880afd60 2022-08-01T19:34:20.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.854+0000 7f968ca0c700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96880a7480 con 0x7f96880b2710 2022-08-01T19:34:20.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.854+0000 7f968ca0c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96880a7720 con 0x7f96880b01a0 2022-08-01T19:34:20.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.854+0000 7f96877fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f96880b2710 0x7f968814afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:20.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.854+0000 7f96877fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f96880b2710 0x7f968814afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36752/0 (socket says 172.21.15.17:36752) 2022-08-01T19:34:20.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.854+0000 7f96877fe700 1 -- 172.21.15.17:0/3166853258 learned_addr learned my addr 172.21.15.17:0/3166853258 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:20.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.854+0000 7f9686ffd700 1 --2- 172.21.15.17:0/3166853258 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96880b01a0 0x7f968814b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:20.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.854+0000 7f96877fe700 1 -- 172.21.15.17:0/3166853258 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96880b01a0 msgr2=0x7f968814b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:20.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.854+0000 7f96877fe700 1 --2- 172.21.15.17:0/3166853258 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96880b01a0 0x7f968814b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.858+0000 7f96877fe700 1 -- 172.21.15.17:0/3166853258 >> v1:172.21.15.77:6789/0 conn(0x7f96880afd60 legacy=0x7f96880a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:20.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.858+0000 7f96877fe700 1 -- 172.21.15.17:0/3166853258 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96880a79c0 con 0x7f96880b2710 2022-08-01T19:34:20.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.858+0000 7f96877fe700 1 --2- 172.21.15.17:0/3166853258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96880b2710 0x7f968814afd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f96800032f0 tx=0x7f9680009290).ready entity=mon.0 client_cookie=33aacac6d5232ad2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:20.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.858+0000 7f96867fc700 1 -- 172.21.15.17:0/3166853258 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f968000b8a0 con 0x7f96880b2710 2022-08-01T19:34:20.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.858+0000 7f96867fc700 1 -- 172.21.15.17:0/3166853258 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f968000ba00 con 0x7f96880b2710 2022-08-01T19:34:20.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.858+0000 7f96867fc700 1 -- 172.21.15.17:0/3166853258 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f968000bb70 con 0x7f96880b2710 2022-08-01T19:34:20.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.858+0000 7f968ca0c700 1 -- 172.21.15.17:0/3166853258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96880b2710 msgr2=0x7f968814afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:20.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.858+0000 7f968ca0c700 1 --2- 172.21.15.17:0/3166853258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96880b2710 0x7f968814afd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f96800032f0 tx=0x7f9680009290).stop 2022-08-01T19:34:20.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.858+0000 7f968ca0c700 1 -- 172.21.15.17:0/3166853258 shutdown_connections 2022-08-01T19:34:20.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.858+0000 7f968ca0c700 1 --2- 172.21.15.17:0/3166853258 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96880b01a0 0x7f968814b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.858+0000 7f968ca0c700 1 --2- 172.21.15.17:0/3166853258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96880b2710 0x7f968814afd0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.870 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.858+0000 7f968ca0c700 1 -- 172.21.15.17:0/3166853258 >> 172.21.15.17:0/3166853258 conn(0x7f9688060310 msgr2=0x7f96880b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:20.870 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.858+0000 7f968ca0c700 1 -- 172.21.15.17:0/3166853258 shutdown_connections 2022-08-01T19:34:20.870 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.858+0000 7f968ca0c700 1 -- 172.21.15.17:0/3166853258 wait complete. 2022-08-01T19:34:20.870 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.858+0000 7f968ca0c700 1 Processor -- start 2022-08-01T19:34:20.870 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.858+0000 7f968ca0c700 1 -- start start 2022-08-01T19:34:20.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.858+0000 7f968ca0c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96880afd60 0x7f968805c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:20.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.858+0000 7f968ca0c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96880b01a0 0x7f9688156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:20.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.858+0000 7f968ca0c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96880b2710 0x7f9688157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:20.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.858+0000 7f968ca0c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9688063190 con 0x7f96880b01a0 2022-08-01T19:34:20.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.858+0000 7f968ca0c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96880ab300 con 0x7f96880b2710 2022-08-01T19:34:20.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.858+0000 7f968ca0c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96880aae00 con 0x7f96880afd60 2022-08-01T19:34:20.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.862+0000 7f9686ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96880b01a0 0x7f9688156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:20.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.862+0000 7f9686ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96880b01a0 0x7f9688156c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36760/0 (socket says 172.21.15.17:36760) 2022-08-01T19:34:20.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.862+0000 7f9686ffd700 1 -- 172.21.15.17:0/2337889687 learned_addr learned my addr 172.21.15.17:0/2337889687 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:20.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.862+0000 7f96877fe700 1 --2- 172.21.15.17:0/2337889687 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96880afd60 0x7f968805c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:20.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.862+0000 7f9687fff700 1 --2- 172.21.15.17:0/2337889687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96880b2710 0x7f9688157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:20.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.862+0000 7f9686ffd700 1 -- 172.21.15.17:0/2337889687 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96880afd60 msgr2=0x7f968805c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:20.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.862+0000 7f9686ffd700 1 --2- 172.21.15.17:0/2337889687 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96880afd60 0x7f968805c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.862+0000 7f9686ffd700 1 -- 172.21.15.17:0/2337889687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96880b2710 msgr2=0x7f9688157100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:20.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.862+0000 7f9686ffd700 1 --2- 172.21.15.17:0/2337889687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96880b2710 0x7f9688157100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:20.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.862+0000 7f9686ffd700 1 -- 172.21.15.17:0/2337889687 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9680009530 con 0x7f96880b01a0 2022-08-01T19:34:20.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.862+0000 7f9686ffd700 1 --2- 172.21.15.17:0/2337889687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96880b01a0 0x7f9688156c10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f967c00c000 tx=0x7f967c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:20.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.862+0000 7f9684ff9700 1 -- 172.21.15.17:0/2337889687 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f967c00c830 con 0x7f96880b01a0 2022-08-01T19:34:20.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.862+0000 7f9684ff9700 1 -- 172.21.15.17:0/2337889687 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967c00c990 con 0x7f96880b01a0 2022-08-01T19:34:20.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.862+0000 7f968ca0c700 1 -- 172.21.15.17:0/2337889687 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96880633c0 con 0x7f96880b01a0 2022-08-01T19:34:20.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.862+0000 7f9684ff9700 1 -- 172.21.15.17:0/2337889687 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f967c00ccf0 con 0x7f96880b01a0 2022-08-01T19:34:20.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.862+0000 7f968ca0c700 1 -- 172.21.15.17:0/2337889687 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96880674c0 con 0x7f96880b01a0 2022-08-01T19:34:20.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.862+0000 7f968ca0c700 1 -- 172.21.15.17:0/2337889687 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96880574e0 con 0x7f96880b01a0 2022-08-01T19:34:20.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.862+0000 7f9684ff9700 1 -- 172.21.15.17:0/2337889687 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f967c01c460 con 0x7f96880b01a0 2022-08-01T19:34:20.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.862+0000 7f9684ff9700 1 --2- 172.21.15.17:0/2337889687 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9668053a70 0x7f9668055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:20.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.866+0000 7f9684ff9700 1 -- 172.21.15.17:0/2337889687 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f967c06c480 con 0x7f96880b01a0 2022-08-01T19:34:20.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.866+0000 7f96877fe700 1 --2- 172.21.15.17:0/2337889687 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9668053a70 0x7f9668055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:20.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.866+0000 7f96877fe700 1 --2- 172.21.15.17:0/2337889687 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9668053a70 0x7f9668055f30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f9680019000 tx=0x7f9680016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:20.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.870+0000 7f9684ff9700 1 -- 172.21.15.17:0/2337889687 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f967c0431e0 con 0x7f96880b01a0 2022-08-01T19:34:20.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.994+0000 7f968ca0c700 1 -- 172.21.15.17:0/2337889687 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f96880b2180 con 0x7f96880b01a0 2022-08-01T19:34:21.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.994+0000 7f9684ff9700 1 -- 172.21.15.17:0/2337889687 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+10 (secure 0 0 0) 0x7f967c046800 con 0x7f96880b01a0 2022-08-01T19:34:21.000 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:34:21.002 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.994+0000 7f968ca0c700 1 -- 172.21.15.17:0/2337889687 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9668053a70 msgr2=0x7f9668055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:21.002 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.994+0000 7f968ca0c700 1 --2- 172.21.15.17:0/2337889687 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9668053a70 0x7f9668055f30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f9680019000 tx=0x7f9680016000).stop 2022-08-01T19:34:21.002 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.994+0000 7f968ca0c700 1 -- 172.21.15.17:0/2337889687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96880b01a0 msgr2=0x7f9688156c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:21.003 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.994+0000 7f968ca0c700 1 --2- 172.21.15.17:0/2337889687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96880b01a0 0x7f9688156c10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f967c00c000 tx=0x7f967c006df0).stop 2022-08-01T19:34:21.003 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.994+0000 7f968ca0c700 1 -- 172.21.15.17:0/2337889687 shutdown_connections 2022-08-01T19:34:21.003 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.994+0000 7f968ca0c700 1 --2- 172.21.15.17:0/2337889687 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9668053a70 0x7f9668055f30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:21.003 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.994+0000 7f968ca0c700 1 --2- 172.21.15.17:0/2337889687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96880b2710 0x7f9688157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:21.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.994+0000 7f968ca0c700 1 --2- 172.21.15.17:0/2337889687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96880b01a0 0x7f9688156c10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:21.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.994+0000 7f968ca0c700 1 --2- 172.21.15.17:0/2337889687 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96880afd60 0x7f968805c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:21.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.994+0000 7f968ca0c700 1 -- 172.21.15.17:0/2337889687 >> 172.21.15.17:0/2337889687 conn(0x7f9688060310 msgr2=0x7f96881489d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:21.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.994+0000 7f968ca0c700 1 -- 172.21.15.17:0/2337889687 shutdown_connections 2022-08-01T19:34:21.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:20.994+0000 7f968ca0c700 1 -- 172.21.15.17:0/2337889687 wait complete. 2022-08-01T19:34:21.018 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:34:21.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.118+0000 7f2fdaa3e700 1 Processor -- start 2022-08-01T19:34:21.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.118+0000 7f2fdaa3e700 1 -- start start 2022-08-01T19:34:21.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.118+0000 7f2fdaa3e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2fd40b01a0 0x7f2fd40a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:21.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.118+0000 7f2fdaa3e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2fd40afd60 0x7f2fd40a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:21.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.118+0000 7f2fdaa3e700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fd40a7250 con 0x7f2fd40b2710 2022-08-01T19:34:21.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.118+0000 7f2fdaa3e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd40a7480 con 0x7f2fd40afd60 2022-08-01T19:34:21.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.118+0000 7f2fdaa3e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd40a7720 con 0x7f2fd40b01a0 2022-08-01T19:34:21.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.118+0000 7f2fda23d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2fd40afd60 0x7f2fd40a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:21.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.118+0000 7f2fd9a3c700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2fd40b2710 0x7f2fd414afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57192/0 (socket says 172.21.15.17:57192) 2022-08-01T19:34:21.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.118+0000 7f2fd9a3c700 1 -- 172.21.15.17:0/2315398736 learned_addr learned my addr 172.21.15.17:0/2315398736 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:21.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.118+0000 7f2fda23d700 1 --2- 172.21.15.17:0/2315398736 >> v2:172.21.15.17:3300/0 conn(0x7f2fd40afd60 0x7f2fd40a8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36768/0 (socket says 172.21.15.17:36768) 2022-08-01T19:34:21.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.118+0000 7f2fd923b700 1 --2- 172.21.15.17:0/2315398736 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2fd40b01a0 0x7f2fd40a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:21.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.118+0000 7f2fda23d700 1 -- 172.21.15.17:0/2315398736 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2fd40b01a0 msgr2=0x7f2fd40a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:21.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.118+0000 7f2fda23d700 1 --2- 172.21.15.17:0/2315398736 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2fd40b01a0 0x7f2fd40a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:21.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.118+0000 7f2fda23d700 1 -- 172.21.15.17:0/2315398736 >> v1:172.21.15.17:6789/0 conn(0x7f2fd40b2710 legacy=0x7f2fd414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:21.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.118+0000 7f2fda23d700 1 -- 172.21.15.17:0/2315398736 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd40a79c0 con 0x7f2fd40afd60 2022-08-01T19:34:21.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.122+0000 7f2fda23d700 1 --2- 172.21.15.17:0/2315398736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fd40afd60 0x7f2fd40a8a30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f2fd0002f70 tx=0x7f2fd0006df0).ready entity=mon.0 client_cookie=6cf43d3079edca1 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:21.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.122+0000 7f2fd8a3a700 1 -- 172.21.15.17:0/2315398736 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fd000d570 con 0x7f2fd40afd60 2022-08-01T19:34:21.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.122+0000 7f2fd8a3a700 1 -- 172.21.15.17:0/2315398736 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd0000e90 con 0x7f2fd40afd60 2022-08-01T19:34:21.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.122+0000 7f2fd8a3a700 1 -- 172.21.15.17:0/2315398736 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fd000d8c0 con 0x7f2fd40afd60 2022-08-01T19:34:21.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.122+0000 7f2fdaa3e700 1 -- 172.21.15.17:0/2315398736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fd40afd60 msgr2=0x7f2fd40a8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:21.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.122+0000 7f2fdaa3e700 1 --2- 172.21.15.17:0/2315398736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fd40afd60 0x7f2fd40a8a30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f2fd0002f70 tx=0x7f2fd0006df0).stop 2022-08-01T19:34:21.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.122+0000 7f2fdaa3e700 1 -- 172.21.15.17:0/2315398736 shutdown_connections 2022-08-01T19:34:21.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.122+0000 7f2fdaa3e700 1 --2- 172.21.15.17:0/2315398736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fd40afd60 0x7f2fd40a8a30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:21.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.122+0000 7f2fdaa3e700 1 --2- 172.21.15.17:0/2315398736 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2fd40b01a0 0x7f2fd40a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:21.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.122+0000 7f2fdaa3e700 1 -- 172.21.15.17:0/2315398736 >> 172.21.15.17:0/2315398736 conn(0x7f2fd4060310 msgr2=0x7f2fd40b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:21.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.122+0000 7f2fdaa3e700 1 -- 172.21.15.17:0/2315398736 shutdown_connections 2022-08-01T19:34:21.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.122+0000 7f2fdaa3e700 1 -- 172.21.15.17:0/2315398736 wait complete. 2022-08-01T19:34:21.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.122+0000 7f2fdaa3e700 1 Processor -- start 2022-08-01T19:34:21.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.122+0000 7f2fdaa3e700 1 -- start start 2022-08-01T19:34:21.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.122+0000 7f2fdaa3e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fd40afd60 0x7f2fd40ab650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:21.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.122+0000 7f2fdaa3e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2fd40b01a0 0x7f2fd4063d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:21.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.122+0000 7f2fdaa3e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2fd40b2710 0x7f2fd41465b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:21.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.122+0000 7f2fdaa3e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd40a9fe0 con 0x7f2fd40afd60 2022-08-01T19:34:21.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.122+0000 7f2fdaa3e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd40aa210 con 0x7f2fd40b2710 2022-08-01T19:34:21.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.122+0000 7f2fdaa3e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd40a97c0 con 0x7f2fd40b01a0 2022-08-01T19:34:21.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.122+0000 7f2fd9a3c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fd40afd60 0x7f2fd40ab650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:21.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.122+0000 7f2fd9a3c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fd40afd60 0x7f2fd40ab650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36772/0 (socket says 172.21.15.17:36772) 2022-08-01T19:34:21.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.122+0000 7f2fd9a3c700 1 -- 172.21.15.17:0/792849799 learned_addr learned my addr 172.21.15.17:0/792849799 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:21.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.122+0000 7f2fd923b700 1 --2- 172.21.15.17:0/792849799 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2fd40b01a0 0x7f2fd4063d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:21.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.122+0000 7f2fda23d700 1 --2- 172.21.15.17:0/792849799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2fd40b2710 0x7f2fd41465b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:21.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.122+0000 7f2fd9a3c700 1 -- 172.21.15.17:0/792849799 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2fd40b01a0 msgr2=0x7f2fd4063d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:21.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.122+0000 7f2fd9a3c700 1 --2- 172.21.15.17:0/792849799 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2fd40b01a0 0x7f2fd4063d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:21.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.122+0000 7f2fd9a3c700 1 -- 172.21.15.17:0/792849799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2fd40b2710 msgr2=0x7f2fd41465b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:21.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.122+0000 7f2fd9a3c700 1 --2- 172.21.15.17:0/792849799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2fd40b2710 0x7f2fd41465b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:21.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.122+0000 7f2fd9a3c700 1 -- 172.21.15.17:0/792849799 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd000b180 con 0x7f2fd40afd60 2022-08-01T19:34:21.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.122+0000 7f2fda23d700 1 --2- 172.21.15.17:0/792849799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2fd40b2710 0x7f2fd41465b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:34:21.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.122+0000 7f2fd9a3c700 1 --2- 172.21.15.17:0/792849799 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fd40afd60 0x7f2fd40ab650 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f2fd4061c10 tx=0x7f2fcc0013a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:21.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.126+0000 7f2fc2ffd700 1 -- 172.21.15.17:0/792849799 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fcc005ba0 con 0x7f2fd40afd60 2022-08-01T19:34:21.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.126+0000 7f2fc2ffd700 1 -- 172.21.15.17:0/792849799 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fcc005d00 con 0x7f2fd40afd60 2022-08-01T19:34:21.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.126+0000 7f2fdaa3e700 1 -- 172.21.15.17:0/792849799 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd40a9a80 con 0x7f2fd40afd60 2022-08-01T19:34:21.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.126+0000 7f2fc2ffd700 1 -- 172.21.15.17:0/792849799 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fcc004a90 con 0x7f2fd40afd60 2022-08-01T19:34:21.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.126+0000 7f2fdaa3e700 1 -- 172.21.15.17:0/792849799 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd4064210 con 0x7f2fd40afd60 2022-08-01T19:34:21.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.126+0000 7f2fdaa3e700 1 -- 172.21.15.17:0/792849799 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd40aa3d0 con 0x7f2fd40afd60 2022-08-01T19:34:21.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.126+0000 7f2fc2ffd700 1 -- 172.21.15.17:0/792849799 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2fcc01b070 con 0x7f2fd40afd60 2022-08-01T19:34:21.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.126+0000 7f2fc2ffd700 1 --2- 172.21.15.17:0/792849799 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2fb4053b00 0x7f2fb4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:21.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.126+0000 7f2fc2ffd700 1 -- 172.21.15.17:0/792849799 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f2fcc06b940 con 0x7f2fd40afd60 2022-08-01T19:34:21.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.126+0000 7f2fd923b700 1 --2- 172.21.15.17:0/792849799 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2fb4053b00 0x7f2fb4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:21.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.126+0000 7f2fd923b700 1 --2- 172.21.15.17:0/792849799 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2fb4053b00 0x7f2fb4055fc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f2fc400c000 tx=0x7f2fc4008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:21.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.134+0000 7f2fc2ffd700 1 -- 172.21.15.17:0/792849799 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2fcc042660 con 0x7f2fd40afd60 2022-08-01T19:34:21.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.302+0000 7f2fdaa3e700 1 -- 172.21.15.17:0/792849799 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2fd40574e0 con 0x7f2fd40afd60 2022-08-01T19:34:21.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.306+0000 7f2fc2ffd700 1 -- 172.21.15.17:0/792849799 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+2486 (secure 0 0 0) 0x7f2fcc045c80 con 0x7f2fd40afd60 2022-08-01T19:34:21.310 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:21.311 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2116883486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2116883486},{"type":"v1","addr":"172.21.15.17:6833","nonce":2116883486}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.17:6835/2974833789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2974833789},{"type":"v1","addr":"172.21.15.17:6835","nonce":2974833789}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":4}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:34:09.692662+0000","modified":"2022-08-01T19:34:11.425752+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":5,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6835/1842780453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1842780453},{"type":"v1","addr":"172.21.15.77:6835","nonce":1842780453}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2022-08-01T19:34:21.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.306+0000 7f2fdaa3e700 1 -- 172.21.15.17:0/792849799 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2fb4053b00 msgr2=0x7f2fb4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:21.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.306+0000 7f2fdaa3e700 1 --2- 172.21.15.17:0/792849799 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2fb4053b00 0x7f2fb4055fc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f2fc400c000 tx=0x7f2fc4008040).stop 2022-08-01T19:34:21.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.306+0000 7f2fdaa3e700 1 -- 172.21.15.17:0/792849799 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fd40afd60 msgr2=0x7f2fd40ab650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:21.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.306+0000 7f2fdaa3e700 1 --2- 172.21.15.17:0/792849799 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fd40afd60 0x7f2fd40ab650 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f2fd4061c10 tx=0x7f2fcc0013a0).stop 2022-08-01T19:34:21.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.306+0000 7f2fdaa3e700 1 -- 172.21.15.17:0/792849799 shutdown_connections 2022-08-01T19:34:21.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.306+0000 7f2fdaa3e700 1 --2- 172.21.15.17:0/792849799 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2fb4053b00 0x7f2fb4055fc0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:21.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.306+0000 7f2fdaa3e700 1 --2- 172.21.15.17:0/792849799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2fd40b2710 0x7f2fd41465b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:21.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.306+0000 7f2fdaa3e700 1 --2- 172.21.15.17:0/792849799 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2fd40b01a0 0x7f2fd4063d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:21.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.306+0000 7f2fdaa3e700 1 --2- 172.21.15.17:0/792849799 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fd40afd60 0x7f2fd40ab650 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:21.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.306+0000 7f2fdaa3e700 1 -- 172.21.15.17:0/792849799 >> 172.21.15.17:0/792849799 conn(0x7f2fd4060310 msgr2=0x7f2fd405ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:21.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.306+0000 7f2fdaa3e700 1 -- 172.21.15.17:0/792849799 shutdown_connections 2022-08-01T19:34:21.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.306+0000 7f2fdaa3e700 1 -- 172.21.15.17:0/792849799 wait complete. 2022-08-01T19:34:21.317 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 7 2022-08-01T19:34:21.630 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:34:09.692662+0000', 'modified': '2022-08-01T19:34:11.425752+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4285}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4285': {'gid': 4285, 'name': 'b', 'rank': 0, 'incarnation': 5, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.77:6835/1842780453', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6834', 'nonce': 1842780453}, {'type': 'v1', 'addr': '172.21.15.77:6835', 'nonce': 1842780453}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:34:21.630 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:34:21.631 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-08-01T19:34:21.746 INFO:teuthology.orchestra.run.smithi077.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":4285,"want_state":"up:active","state":"up:active","rank_uptime":12.039654933,"mdsmap_epoch":7,"osdmap_epoch":25,"osdmap_epoch_barrier":25,"uptime":18.389963422000001} 2022-08-01T19:34:21.746 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 4285, "mdsmap_epoch": 7, "osdmap_epoch": 25, "osdmap_epoch_barrier": 25, "rank_uptime": 12.039654933, "state": "up:active", "uptime": 18.389963422, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:34:21.747 INFO:teuthology.run_tasks:Running task ceph-fuse... 2022-08-01T19:34:21.766 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2022-08-01T19:34:21.766 INFO:tasks.ceph_fuse:config is None 2022-08-01T19:34:21.767 INFO:tasks.ceph_fuse:new config is {'client.0': None, 'client.1': None} 2022-08-01T19:34:21.767 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2022-08-01T19:34:21.768 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:34:21.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.874+0000 7f41b7fff700 1 Processor -- start 2022-08-01T19:34:21.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.874+0000 7f41b7fff700 1 -- start start 2022-08-01T19:34:21.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.874+0000 7f41b7fff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f41b80b6420 0x7f41b8061600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:21.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.874+0000 7f41b7fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f41b8142ff0 0x7f41b8143480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:21.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.874+0000 7f41b7fff700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41b80b60e0 con 0x7f41b80b41f0 2022-08-01T19:34:21.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.874+0000 7f41b7fff700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41b80b2100 con 0x7f41b8142ff0 2022-08-01T19:34:21.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.874+0000 7f41b7fff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41b80b23a0 con 0x7f41b80b6420 2022-08-01T19:34:21.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.874+0000 7f41b77fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f41b8142ff0 0x7f41b8143480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:21.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.874+0000 7f41b77fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f41b8142ff0 0x7f41b8143480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36784/0 (socket says 172.21.15.17:36784) 2022-08-01T19:34:21.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.874+0000 7f41b77fe700 1 -- 172.21.15.17:0/868791005 learned_addr learned my addr 172.21.15.17:0/868791005 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:21.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.874+0000 7f41b6ffd700 1 --2- 172.21.15.17:0/868791005 >> v2:172.21.15.77:3300/0 conn(0x7f41b80b6420 0x7f41b8061600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:21.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.874+0000 7f41b77fe700 1 -- 172.21.15.17:0/868791005 >> v2:172.21.15.77:3300/0 conn(0x7f41b80b6420 msgr2=0x7f41b8061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:21.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.874+0000 7f41b77fe700 1 --2- 172.21.15.17:0/868791005 >> v2:172.21.15.77:3300/0 conn(0x7f41b80b6420 0x7f41b8061600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:21.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.874+0000 7f41b77fe700 1 -- 172.21.15.17:0/868791005 >> v1:172.21.15.77:6789/0 conn(0x7f41b80b41f0 legacy=0x7f41b8061af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:21.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.874+0000 7f41b77fe700 1 -- 172.21.15.17:0/868791005 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b80b2640 con 0x7f41b8142ff0 2022-08-01T19:34:21.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.874+0000 7f41b77fe700 1 --2- 172.21.15.17:0/868791005 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41b8142ff0 0x7f41b8143480 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f41b0002f70 tx=0x7f41b0006eb0).ready entity=mon.0 client_cookie=eed243c121771fe5 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:21.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.878+0000 7f41b5ffb700 1 -- 172.21.15.17:0/868791005 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41b000d5c0 con 0x7f41b8142ff0 2022-08-01T19:34:21.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.878+0000 7f41b5ffb700 1 -- 172.21.15.17:0/868791005 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41b0000e90 con 0x7f41b8142ff0 2022-08-01T19:34:21.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.878+0000 7f41b5ffb700 1 -- 172.21.15.17:0/868791005 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41b000d950 con 0x7f41b8142ff0 2022-08-01T19:34:21.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.878+0000 7f41b7fff700 1 -- 172.21.15.17:0/868791005 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41b8142ff0 msgr2=0x7f41b8143480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:21.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.878+0000 7f41b7fff700 1 --2- 172.21.15.17:0/868791005 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41b8142ff0 0x7f41b8143480 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f41b0002f70 tx=0x7f41b0006eb0).stop 2022-08-01T19:34:21.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.878+0000 7f41b7fff700 1 -- 172.21.15.17:0/868791005 shutdown_connections 2022-08-01T19:34:21.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.878+0000 7f41b7fff700 1 --2- 172.21.15.17:0/868791005 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41b8142ff0 0x7f41b8143480 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:21.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.878+0000 7f41b7fff700 1 --2- 172.21.15.17:0/868791005 >> v2:172.21.15.77:3300/0 conn(0x7f41b80b6420 0x7f41b8061600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:21.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.878+0000 7f41b7fff700 1 -- 172.21.15.17:0/868791005 >> 172.21.15.17:0/868791005 conn(0x7f41b805c620 msgr2=0x7f41b805ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:21.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.878+0000 7f41b7fff700 1 -- 172.21.15.17:0/868791005 shutdown_connections 2022-08-01T19:34:21.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.878+0000 7f41b7fff700 1 -- 172.21.15.17:0/868791005 wait complete. 2022-08-01T19:34:21.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.882+0000 7f41b7fff700 1 Processor -- start 2022-08-01T19:34:21.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.882+0000 7f41b7fff700 1 -- start start 2022-08-01T19:34:21.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.882+0000 7f41b7fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f41b80b41f0 0x7f41b8146c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:21.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.882+0000 7f41b7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41b80b6420 0x7f41b8155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:21.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.882+0000 7f41b7fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41b8142ff0 0x7f41b81560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:21.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.882+0000 7f41b7fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b813f870 con 0x7f41b80b6420 2022-08-01T19:34:21.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.882+0000 7f41b7fff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b813d0c0 con 0x7f41b8142ff0 2022-08-01T19:34:21.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.882+0000 7f41b7fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41b813faa0 con 0x7f41b80b41f0 2022-08-01T19:34:21.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.882+0000 7f41b67fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41b80b6420 0x7f41b8155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:21.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.882+0000 7f41b67fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41b80b6420 0x7f41b8155bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36788/0 (socket says 172.21.15.17:36788) 2022-08-01T19:34:21.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.882+0000 7f41b67fc700 1 -- 172.21.15.17:0/3644534233 learned_addr learned my addr 172.21.15.17:0/3644534233 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:21.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.882+0000 7f41b6ffd700 1 --2- 172.21.15.17:0/3644534233 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f41b80b41f0 0x7f41b8146c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:21.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.882+0000 7f41b77fe700 1 --2- 172.21.15.17:0/3644534233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41b8142ff0 0x7f41b81560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:21.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.882+0000 7f41b67fc700 1 -- 172.21.15.17:0/3644534233 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f41b80b41f0 msgr2=0x7f41b8146c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:21.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.882+0000 7f41b67fc700 1 --2- 172.21.15.17:0/3644534233 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f41b80b41f0 0x7f41b8146c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:21.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.882+0000 7f41b67fc700 1 -- 172.21.15.17:0/3644534233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41b8142ff0 msgr2=0x7f41b81560a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:21.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.882+0000 7f41b67fc700 1 --2- 172.21.15.17:0/3644534233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41b8142ff0 0x7f41b81560a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:21.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.882+0000 7f41b67fc700 1 -- 172.21.15.17:0/3644534233 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b000b1e0 con 0x7f41b80b6420 2022-08-01T19:34:21.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.882+0000 7f41b67fc700 1 --2- 172.21.15.17:0/3644534233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41b80b6420 0x7f41b8155bb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f41b80b2c60 tx=0x7f41a0007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:21.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.882+0000 7f41affff700 1 -- 172.21.15.17:0/3644534233 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41a0007030 con 0x7f41b80b6420 2022-08-01T19:34:21.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.882+0000 7f41affff700 1 -- 172.21.15.17:0/3644534233 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a0007190 con 0x7f41b80b6420 2022-08-01T19:34:21.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.882+0000 7f41b7fff700 1 -- 172.21.15.17:0/3644534233 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41b813fdf0 con 0x7f41b80b6420 2022-08-01T19:34:21.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.882+0000 7f41affff700 1 -- 172.21.15.17:0/3644534233 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41a0006510 con 0x7f41b80b6420 2022-08-01T19:34:21.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.882+0000 7f41b7fff700 1 -- 172.21.15.17:0/3644534233 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41b813e920 con 0x7f41b80b6420 2022-08-01T19:34:21.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.886+0000 7f41b7fff700 1 -- 172.21.15.17:0/3644534233 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41b813ff80 con 0x7f41b80b6420 2022-08-01T19:34:21.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.886+0000 7f41affff700 1 -- 172.21.15.17:0/3644534233 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f41a0006670 con 0x7f41b80b6420 2022-08-01T19:34:21.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.886+0000 7f41affff700 1 --2- 172.21.15.17:0/3644534233 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f419c053aa0 0x7f419c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:21.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.886+0000 7f41affff700 1 -- 172.21.15.17:0/3644534233 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f41a006b740 con 0x7f41b80b6420 2022-08-01T19:34:21.899 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.886+0000 7f41b6ffd700 1 --2- 172.21.15.17:0/3644534233 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f419c053aa0 0x7f419c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:21.899 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.886+0000 7f41b6ffd700 1 --2- 172.21.15.17:0/3644534233 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f419c053aa0 0x7f419c055f60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f41a800b000 tx=0x7f41a80091d0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:21.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:21.894+0000 7f41affff700 1 -- 172.21.15.17:0/3644534233 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f41a00424a0 con 0x7f41b80b6420 2022-08-01T19:34:22.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.026+0000 7f41b7fff700 1 -- 172.21.15.17:0/3644534233 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41b80b60e0 con 0x7f41b80b6420 2022-08-01T19:34:22.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.026+0000 7f41affff700 1 -- 172.21.15.17:0/3644534233 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f41b80b60e0 con 0x7f41b80b6420 2022-08-01T19:34:22.034 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:22.034 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:34:09.692404+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:34:04.635473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-08-01T19:34:07.409434+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:34:22.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.030+0000 7f41b7fff700 1 -- 172.21.15.17:0/3644534233 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f419c053aa0 msgr2=0x7f419c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:22.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.030+0000 7f41b7fff700 1 --2- 172.21.15.17:0/3644534233 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f419c053aa0 0x7f419c055f60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f41a800b000 tx=0x7f41a80091d0).stop 2022-08-01T19:34:22.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.030+0000 7f41b7fff700 1 -- 172.21.15.17:0/3644534233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41b80b6420 msgr2=0x7f41b8155bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:22.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.030+0000 7f41b7fff700 1 --2- 172.21.15.17:0/3644534233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41b80b6420 0x7f41b8155bb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f41b80b2c60 tx=0x7f41a0007bb0).stop 2022-08-01T19:34:22.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.030+0000 7f41b7fff700 1 -- 172.21.15.17:0/3644534233 shutdown_connections 2022-08-01T19:34:22.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.030+0000 7f41b7fff700 1 --2- 172.21.15.17:0/3644534233 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f419c053aa0 0x7f419c055f60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:22.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.030+0000 7f41b7fff700 1 --2- 172.21.15.17:0/3644534233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41b8142ff0 0x7f41b81560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:22.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.030+0000 7f41b7fff700 1 --2- 172.21.15.17:0/3644534233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41b80b6420 0x7f41b8155bb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:22.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.030+0000 7f41b7fff700 1 --2- 172.21.15.17:0/3644534233 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f41b80b41f0 0x7f41b8146c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:22.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.030+0000 7f41b7fff700 1 -- 172.21.15.17:0/3644534233 >> 172.21.15.17:0/3644534233 conn(0x7f41b805c620 msgr2=0x7f41b80b4700 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:22.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.030+0000 7f41b7fff700 1 -- 172.21.15.17:0/3644534233 shutdown_connections 2022-08-01T19:34:22.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.030+0000 7f41b7fff700 1 -- 172.21.15.17:0/3644534233 wait complete. 2022-08-01T19:34:22.047 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T19:34:04.635473+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T19:34:07.409434+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T19:34:22.049 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:34:22.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.154+0000 7fe715e11700 1 Processor -- start 2022-08-01T19:34:22.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.154+0000 7fe715e11700 1 -- start start 2022-08-01T19:34:22.162 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.154+0000 7fe715e11700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe7100ad750 0x7fe710061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:22.162 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.154+0000 7fe715e11700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fe7101405d0 0x7fe710140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:22.162 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.154+0000 7fe715e11700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7100aee80 con 0x7fe7100614b0 2022-08-01T19:34:22.163 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.154+0000 7fe715e11700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7100b70b0 con 0x7fe7100ad750 2022-08-01T19:34:22.163 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.154+0000 7fe715e11700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7100af0b0 con 0x7fe7101405d0 2022-08-01T19:34:22.163 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.154+0000 7fe714e0f700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe7100614b0 0x7fe7100b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57220/0 (socket says 172.21.15.17:57220) 2022-08-01T19:34:22.163 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.154+0000 7fe714e0f700 1 -- 172.21.15.17:0/2039533806 learned_addr learned my addr 172.21.15.17:0/2039533806 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:22.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.154+0000 7fe70ffff700 1 --2- 172.21.15.17:0/2039533806 >> v2:172.21.15.17:3300/0 conn(0x7fe7100ad750 0x7fe710061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:22.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.154+0000 7fe715610700 1 --2- 172.21.15.17:0/2039533806 >> v2:172.21.15.77:3300/0 conn(0x7fe7101405d0 0x7fe710140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:22.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.154+0000 7fe70ffff700 1 -- 172.21.15.17:0/2039533806 >> v2:172.21.15.77:3300/0 conn(0x7fe7101405d0 msgr2=0x7fe710140a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:22.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.154+0000 7fe70ffff700 1 --2- 172.21.15.17:0/2039533806 >> v2:172.21.15.77:3300/0 conn(0x7fe7101405d0 0x7fe710140a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:22.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.154+0000 7fe70ffff700 1 -- 172.21.15.17:0/2039533806 >> v1:172.21.15.17:6789/0 conn(0x7fe7100614b0 legacy=0x7fe7100b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:22.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.154+0000 7fe70ffff700 1 -- 172.21.15.17:0/2039533806 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7100af350 con 0x7fe7100ad750 2022-08-01T19:34:22.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.154+0000 7fe70ffff700 1 --2- 172.21.15.17:0/2039533806 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7100ad750 0x7fe710061890 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7fe6f8002f70 tx=0x7fe6f8006df0).ready entity=mon.0 client_cookie=c8873fe6acceca4f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:22.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.154+0000 7fe70f7fe700 1 -- 172.21.15.17:0/2039533806 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6f800d570 con 0x7fe7100ad750 2022-08-01T19:34:22.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.154+0000 7fe70f7fe700 1 -- 172.21.15.17:0/2039533806 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f8000e90 con 0x7fe7100ad750 2022-08-01T19:34:22.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.154+0000 7fe70f7fe700 1 -- 172.21.15.17:0/2039533806 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6f800d8c0 con 0x7fe7100ad750 2022-08-01T19:34:22.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.154+0000 7fe715e11700 1 -- 172.21.15.17:0/2039533806 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7100ad750 msgr2=0x7fe710061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:22.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.154+0000 7fe715e11700 1 --2- 172.21.15.17:0/2039533806 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7100ad750 0x7fe710061890 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7fe6f8002f70 tx=0x7fe6f8006df0).stop 2022-08-01T19:34:22.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.154+0000 7fe715e11700 1 -- 172.21.15.17:0/2039533806 shutdown_connections 2022-08-01T19:34:22.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.154+0000 7fe715e11700 1 --2- 172.21.15.17:0/2039533806 >> v2:172.21.15.77:3300/0 conn(0x7fe7101405d0 0x7fe710140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:22.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.154+0000 7fe715e11700 1 --2- 172.21.15.17:0/2039533806 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7100ad750 0x7fe710061890 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:22.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.154+0000 7fe715e11700 1 -- 172.21.15.17:0/2039533806 >> 172.21.15.17:0/2039533806 conn(0x7fe710066f70 msgr2=0x7fe710067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:22.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.158+0000 7fe715e11700 1 -- 172.21.15.17:0/2039533806 shutdown_connections 2022-08-01T19:34:22.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.158+0000 7fe715e11700 1 -- 172.21.15.17:0/2039533806 wait complete. 2022-08-01T19:34:22.171 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.158+0000 7fe715e11700 1 Processor -- start 2022-08-01T19:34:22.171 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.158+0000 7fe715e11700 1 -- start start 2022-08-01T19:34:22.171 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.158+0000 7fe715e11700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7100614b0 0x7fe710152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:22.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.158+0000 7fe715e11700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe7100ad750 0x7fe710143450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:22.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.158+0000 7fe715e11700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7101405d0 0x7fe7100aaec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:22.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.158+0000 7fe715e11700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe710144e40 con 0x7fe7100614b0 2022-08-01T19:34:22.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.158+0000 7fe715e11700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe710143fd0 con 0x7fe7101405d0 2022-08-01T19:34:22.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.158+0000 7fe715e11700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7100aa240 con 0x7fe7100ad750 2022-08-01T19:34:22.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.162+0000 7fe714e0f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7100614b0 0x7fe710152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:22.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.162+0000 7fe714e0f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7100614b0 0x7fe710152720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36800/0 (socket says 172.21.15.17:36800) 2022-08-01T19:34:22.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.162+0000 7fe714e0f700 1 -- 172.21.15.17:0/212776447 learned_addr learned my addr 172.21.15.17:0/212776447 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:22.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.162+0000 7fe70ffff700 1 --2- 172.21.15.17:0/212776447 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe7100ad750 0x7fe710143450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:22.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.162+0000 7fe715610700 1 --2- 172.21.15.17:0/212776447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7101405d0 0x7fe7100aaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:22.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.162+0000 7fe714e0f700 1 -- 172.21.15.17:0/212776447 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe7100ad750 msgr2=0x7fe710143450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:22.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.162+0000 7fe714e0f700 1 --2- 172.21.15.17:0/212776447 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe7100ad750 0x7fe710143450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:22.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.162+0000 7fe714e0f700 1 -- 172.21.15.17:0/212776447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7101405d0 msgr2=0x7fe7100aaec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:22.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.162+0000 7fe714e0f700 1 --2- 172.21.15.17:0/212776447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7101405d0 0x7fe7100aaec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:22.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.162+0000 7fe714e0f700 1 -- 172.21.15.17:0/212776447 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f800b180 con 0x7fe7100614b0 2022-08-01T19:34:22.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.162+0000 7fe70ffff700 1 --2- 172.21.15.17:0/212776447 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe7100ad750 0x7fe710143450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:34:22.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.162+0000 7fe715610700 1 --2- 172.21.15.17:0/212776447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7101405d0 0x7fe7100aaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:34:22.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.162+0000 7fe714e0f700 1 --2- 172.21.15.17:0/212776447 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7100614b0 0x7fe710152720 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fe7100ae2b0 tx=0x7fe700007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:22.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.162+0000 7fe70dffb700 1 -- 172.21.15.17:0/212776447 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe70000ce30 con 0x7fe7100614b0 2022-08-01T19:34:22.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.162+0000 7fe70dffb700 1 -- 172.21.15.17:0/212776447 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7000072e0 con 0x7fe7100614b0 2022-08-01T19:34:22.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.162+0000 7fe715e11700 1 -- 172.21.15.17:0/212776447 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7100aa4e0 con 0x7fe7100614b0 2022-08-01T19:34:22.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.162+0000 7fe70dffb700 1 -- 172.21.15.17:0/212776447 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe700005800 con 0x7fe7100614b0 2022-08-01T19:34:22.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.162+0000 7fe715e11700 1 -- 172.21.15.17:0/212776447 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7100aaa20 con 0x7fe7100614b0 2022-08-01T19:34:22.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.162+0000 7fe70dffb700 1 -- 172.21.15.17:0/212776447 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe700010070 con 0x7fe7100614b0 2022-08-01T19:34:22.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.162+0000 7fe715e11700 1 -- 172.21.15.17:0/212776447 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7100af5f0 con 0x7fe7100614b0 2022-08-01T19:34:22.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.162+0000 7fe70dffb700 1 --2- 172.21.15.17:0/212776447 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe6f4053a10 0x7fe6f4055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:22.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.162+0000 7fe70dffb700 1 -- 172.21.15.17:0/212776447 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fe70006b7a0 con 0x7fe7100614b0 2022-08-01T19:34:22.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.162+0000 7fe70ffff700 1 --2- 172.21.15.17:0/212776447 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe6f4053a10 0x7fe6f4055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:22.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.166+0000 7fe70ffff700 1 --2- 172.21.15.17:0/212776447 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe6f4053a10 0x7fe6f4055ed0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fe6f8015000 tx=0x7fe6f8014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:22.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.170+0000 7fe70dffb700 1 -- 172.21.15.17:0/212776447 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe700042450 con 0x7fe7100614b0 2022-08-01T19:34:22.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.294+0000 7fe715e11700 1 -- 172.21.15.17:0/212776447 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe710062290 con 0x7fe7100614b0 2022-08-01T19:34:22.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.294+0000 7fe70dffb700 1 -- 172.21.15.17:0/212776447 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v25) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe700045a70 con 0x7fe7100614b0 2022-08-01T19:34:22.303 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:34:22.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.294+0000 7fe715e11700 1 -- 172.21.15.17:0/212776447 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe6f4053a10 msgr2=0x7fe6f4055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:22.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.298+0000 7fe715e11700 1 --2- 172.21.15.17:0/212776447 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe6f4053a10 0x7fe6f4055ed0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fe6f8015000 tx=0x7fe6f8014000).stop 2022-08-01T19:34:22.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.298+0000 7fe715e11700 1 -- 172.21.15.17:0/212776447 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7100614b0 msgr2=0x7fe710152720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:22.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.298+0000 7fe715e11700 1 --2- 172.21.15.17:0/212776447 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7100614b0 0x7fe710152720 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fe7100ae2b0 tx=0x7fe700007c40).stop 2022-08-01T19:34:22.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.298+0000 7fe715e11700 1 -- 172.21.15.17:0/212776447 shutdown_connections 2022-08-01T19:34:22.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.298+0000 7fe715e11700 1 --2- 172.21.15.17:0/212776447 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe6f4053a10 0x7fe6f4055ed0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:22.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.298+0000 7fe715e11700 1 --2- 172.21.15.17:0/212776447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7101405d0 0x7fe7100aaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:22.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.298+0000 7fe715e11700 1 --2- 172.21.15.17:0/212776447 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe7100ad750 0x7fe710143450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:22.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.298+0000 7fe715e11700 1 --2- 172.21.15.17:0/212776447 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7100614b0 0x7fe710152720 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:22.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.298+0000 7fe715e11700 1 -- 172.21.15.17:0/212776447 >> 172.21.15.17:0/212776447 conn(0x7fe710066f70 msgr2=0x7fe7101456c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:22.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.298+0000 7fe715e11700 1 -- 172.21.15.17:0/212776447 shutdown_connections 2022-08-01T19:34:22.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.298+0000 7fe715e11700 1 -- 172.21.15.17:0/212776447 wait complete. 2022-08-01T19:34:22.313 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:34:22.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.414+0000 7fa308425700 1 Processor -- start 2022-08-01T19:34:22.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.414+0000 7fa308425700 1 -- start start 2022-08-01T19:34:22.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.418+0000 7fa308425700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa3000b4310 0x7fa300140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:22.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.418+0000 7fa308425700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa3000b4820 0x7fa3000b5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:22.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.418+0000 7fa308425700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa300143cf0 con 0x7fa300146ae0 2022-08-01T19:34:22.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.418+0000 7fa308425700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3001476a0 con 0x7fa3000b4310 2022-08-01T19:34:22.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.418+0000 7fa308425700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa300143f90 con 0x7fa3000b4820 2022-08-01T19:34:22.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.418+0000 7fa3059c0700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa3000b4310 0x7fa300140ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:22.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.418+0000 7fa3061c1700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa300146ae0 0x7fa3001403c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57234/0 (socket says 172.21.15.17:57234) 2022-08-01T19:34:22.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.418+0000 7fa3061c1700 1 -- 172.21.15.17:0/3808161122 learned_addr learned my addr 172.21.15.17:0/3808161122 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:22.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.418+0000 7fa3069c2700 1 --2- 172.21.15.17:0/3808161122 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa3000b4820 0x7fa3000b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:22.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.418+0000 7fa3059c0700 1 -- 172.21.15.17:0/3808161122 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa3000b4820 msgr2=0x7fa3000b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:22.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.418+0000 7fa3059c0700 1 --2- 172.21.15.17:0/3808161122 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa3000b4820 0x7fa3000b5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:22.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.418+0000 7fa3059c0700 1 -- 172.21.15.17:0/3808161122 >> v1:172.21.15.17:6789/0 conn(0x7fa300146ae0 legacy=0x7fa3001403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:22.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.418+0000 7fa3059c0700 1 -- 172.21.15.17:0/3808161122 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3000b0e20 con 0x7fa3000b4310 2022-08-01T19:34:22.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.418+0000 7fa3059c0700 1 --2- 172.21.15.17:0/3808161122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3000b4310 0x7fa300140ad0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fa2fc002f70 tx=0x7fa2fc006df0).ready entity=mon.0 client_cookie=5d24b8a3371a6fe7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:22.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.418+0000 7fa3051bf700 1 -- 172.21.15.17:0/3808161122 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2fc00d570 con 0x7fa3000b4310 2022-08-01T19:34:22.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.418+0000 7fa3051bf700 1 -- 172.21.15.17:0/3808161122 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2fc000e90 con 0x7fa3000b4310 2022-08-01T19:34:22.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.418+0000 7fa3051bf700 1 -- 172.21.15.17:0/3808161122 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2fc00d8c0 con 0x7fa3000b4310 2022-08-01T19:34:22.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.418+0000 7fa308425700 1 -- 172.21.15.17:0/3808161122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3000b4310 msgr2=0x7fa300140ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:22.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.418+0000 7fa308425700 1 --2- 172.21.15.17:0/3808161122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3000b4310 0x7fa300140ad0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fa2fc002f70 tx=0x7fa2fc006df0).stop 2022-08-01T19:34:22.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.418+0000 7fa308425700 1 -- 172.21.15.17:0/3808161122 shutdown_connections 2022-08-01T19:34:22.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.418+0000 7fa308425700 1 --2- 172.21.15.17:0/3808161122 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa3000b4820 0x7fa3000b5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:22.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.418+0000 7fa308425700 1 --2- 172.21.15.17:0/3808161122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3000b4310 0x7fa300140ad0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:22.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.418+0000 7fa308425700 1 -- 172.21.15.17:0/3808161122 >> 172.21.15.17:0/3808161122 conn(0x7fa30005c620 msgr2=0x7fa30005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:22.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.418+0000 7fa308425700 1 -- 172.21.15.17:0/3808161122 shutdown_connections 2022-08-01T19:34:22.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.418+0000 7fa308425700 1 -- 172.21.15.17:0/3808161122 wait complete. 2022-08-01T19:34:22.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.422+0000 7fa308425700 1 Processor -- start 2022-08-01T19:34:22.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.422+0000 7fa308425700 1 -- start start 2022-08-01T19:34:22.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.422+0000 7fa308425700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa3000b4310 0x7fa3000b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:22.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.422+0000 7fa308425700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3000b4820 0x7fa3000b78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:22.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.422+0000 7fa308425700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa300146ae0 0x7fa3000ae850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:22.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.422+0000 7fa308425700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3000ba6e0 con 0x7fa3000b4820 2022-08-01T19:34:22.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.422+0000 7fa308425700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa300152eb0 con 0x7fa300146ae0 2022-08-01T19:34:22.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.422+0000 7fa308425700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3000b8ff0 con 0x7fa3000b4310 2022-08-01T19:34:22.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.422+0000 7fa3059c0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3000b4820 0x7fa3000b78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:22.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.422+0000 7fa3059c0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3000b4820 0x7fa3000b78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36816/0 (socket says 172.21.15.17:36816) 2022-08-01T19:34:22.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.422+0000 7fa3059c0700 1 -- 172.21.15.17:0/1153172840 learned_addr learned my addr 172.21.15.17:0/1153172840 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:22.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.422+0000 7fa3061c1700 1 --2- 172.21.15.17:0/1153172840 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa3000b4310 0x7fa3000b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:22.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.422+0000 7fa3069c2700 1 --2- 172.21.15.17:0/1153172840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa300146ae0 0x7fa3000ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:22.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.422+0000 7fa3059c0700 1 -- 172.21.15.17:0/1153172840 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa3000b4310 msgr2=0x7fa3000b8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:22.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.422+0000 7fa3059c0700 1 --2- 172.21.15.17:0/1153172840 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa3000b4310 0x7fa3000b8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:22.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.422+0000 7fa3059c0700 1 -- 172.21.15.17:0/1153172840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa300146ae0 msgr2=0x7fa3000ae850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:22.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.422+0000 7fa3059c0700 1 --2- 172.21.15.17:0/1153172840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa300146ae0 0x7fa3000ae850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:22.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.422+0000 7fa3059c0700 1 -- 172.21.15.17:0/1153172840 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2fc00b180 con 0x7fa3000b4820 2022-08-01T19:34:22.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.422+0000 7fa3061c1700 1 --2- 172.21.15.17:0/1153172840 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa3000b4310 0x7fa3000b8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:34:22.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.422+0000 7fa3069c2700 1 --2- 172.21.15.17:0/1153172840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa300146ae0 0x7fa3000ae850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:34:22.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.422+0000 7fa3059c0700 1 --2- 172.21.15.17:0/1153172840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3000b4820 0x7fa3000b78c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fa3000ba820 tx=0x7fa2fc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:22.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.422+0000 7fa2f37fe700 1 -- 172.21.15.17:0/1153172840 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2fc00bed0 con 0x7fa3000b4820 2022-08-01T19:34:22.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.422+0000 7fa2f37fe700 1 -- 172.21.15.17:0/1153172840 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2fc00de50 con 0x7fa3000b4820 2022-08-01T19:34:22.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.422+0000 7fa308425700 1 -- 172.21.15.17:0/1153172840 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3000b80f0 con 0x7fa3000b4820 2022-08-01T19:34:22.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.422+0000 7fa2f37fe700 1 -- 172.21.15.17:0/1153172840 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2fc011e90 con 0x7fa3000b4820 2022-08-01T19:34:22.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.422+0000 7fa308425700 1 -- 172.21.15.17:0/1153172840 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3000b7db0 con 0x7fa3000b4820 2022-08-01T19:34:22.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.422+0000 7fa308425700 1 -- 172.21.15.17:0/1153172840 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3000574e0 con 0x7fa3000b4820 2022-08-01T19:34:22.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.426+0000 7fa2f37fe700 1 -- 172.21.15.17:0/1153172840 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa2fc01a040 con 0x7fa3000b4820 2022-08-01T19:34:22.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.426+0000 7fa2f37fe700 1 --2- 172.21.15.17:0/1153172840 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa2e00539e0 0x7fa2e0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:22.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.426+0000 7fa2f37fe700 1 -- 172.21.15.17:0/1153172840 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fa2fc06d0f0 con 0x7fa3000b4820 2022-08-01T19:34:22.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.426+0000 7fa3061c1700 1 --2- 172.21.15.17:0/1153172840 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa2e00539e0 0x7fa2e0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:22.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.426+0000 7fa3061c1700 1 --2- 172.21.15.17:0/1153172840 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa2e00539e0 0x7fa2e0055ea0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fa2f4008000 tx=0x7fa2f4009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:22.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.430+0000 7fa2f37fe700 1 -- 172.21.15.17:0/1153172840 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa2fc043e10 con 0x7fa3000b4820 2022-08-01T19:34:22.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.558+0000 7fa308425700 1 -- 172.21.15.17:0/1153172840 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa300069a70 con 0x7fa3000b4820 2022-08-01T19:34:22.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.558+0000 7fa2f37fe700 1 -- 172.21.15.17:0/1153172840 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa2fc0230a0 con 0x7fa3000b4820 2022-08-01T19:34:22.563 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:34:22.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.558+0000 7fa308425700 1 -- 172.21.15.17:0/1153172840 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa2e00539e0 msgr2=0x7fa2e0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:22.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.558+0000 7fa308425700 1 --2- 172.21.15.17:0/1153172840 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa2e00539e0 0x7fa2e0055ea0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fa2f4008000 tx=0x7fa2f4009040).stop 2022-08-01T19:34:22.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.558+0000 7fa308425700 1 -- 172.21.15.17:0/1153172840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3000b4820 msgr2=0x7fa3000b78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:22.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.558+0000 7fa308425700 1 --2- 172.21.15.17:0/1153172840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3000b4820 0x7fa3000b78c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fa3000ba820 tx=0x7fa2fc015000).stop 2022-08-01T19:34:22.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.558+0000 7fa308425700 1 -- 172.21.15.17:0/1153172840 shutdown_connections 2022-08-01T19:34:22.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.558+0000 7fa308425700 1 --2- 172.21.15.17:0/1153172840 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa2e00539e0 0x7fa2e0055ea0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:22.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.558+0000 7fa308425700 1 --2- 172.21.15.17:0/1153172840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa300146ae0 0x7fa3000ae850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:22.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.558+0000 7fa308425700 1 --2- 172.21.15.17:0/1153172840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3000b4820 0x7fa3000b78c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:22.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.558+0000 7fa308425700 1 --2- 172.21.15.17:0/1153172840 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa3000b4310 0x7fa3000b8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:22.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.558+0000 7fa308425700 1 -- 172.21.15.17:0/1153172840 >> 172.21.15.17:0/1153172840 conn(0x7fa30005c620 msgr2=0x7fa3000b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:22.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.558+0000 7fa308425700 1 -- 172.21.15.17:0/1153172840 shutdown_connections 2022-08-01T19:34:22.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.558+0000 7fa308425700 1 -- 172.21.15.17:0/1153172840 wait complete. 2022-08-01T19:34:22.576 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T19:34:22.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.678+0000 7fca60b3e700 1 Processor -- start 2022-08-01T19:34:22.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.678+0000 7fca60b3e700 1 -- start start 2022-08-01T19:34:22.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.682+0000 7fca60b3e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fca5c0614b0 0x7fca5c061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:22.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.682+0000 7fca60b3e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fca5c1405d0 0x7fca5c140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:22.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.682+0000 7fca60b3e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca5c0aee80 con 0x7fca5c0ad750 2022-08-01T19:34:22.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.682+0000 7fca60b3e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca5c0b70b0 con 0x7fca5c1405d0 2022-08-01T19:34:22.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.682+0000 7fca60b3e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c0af0b0 con 0x7fca5c0614b0 2022-08-01T19:34:22.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.682+0000 7fca5bfff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fca5c1405d0 0x7fca5c140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:22.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.682+0000 7fca5b7fe700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fca5c0614b0 0x7fca5c061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:22.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.682+0000 7fca5bfff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fca5c1405d0 0x7fca5c140a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:34748/0 (socket says 172.21.15.17:34748) 2022-08-01T19:34:22.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.682+0000 7fca5bfff700 1 -- 172.21.15.17:0/3678491925 learned_addr learned my addr 172.21.15.17:0/3678491925 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:22.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.682+0000 7fca5bfff700 1 -- 172.21.15.17:0/3678491925 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fca5c0614b0 msgr2=0x7fca5c061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:22.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.682+0000 7fca5bfff700 1 --2- 172.21.15.17:0/3678491925 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fca5c0614b0 0x7fca5c061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:22.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.682+0000 7fca5bfff700 1 -- 172.21.15.17:0/3678491925 >> v1:172.21.15.77:6789/0 conn(0x7fca5c0ad750 legacy=0x7fca5c0b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:22.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.682+0000 7fca5bfff700 1 -- 172.21.15.17:0/3678491925 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca5c0af350 con 0x7fca5c1405d0 2022-08-01T19:34:22.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.682+0000 7fca5bfff700 1 --2- 172.21.15.17:0/3678491925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca5c1405d0 0x7fca5c140a60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fca540032f0 tx=0x7fca54009290).ready entity=mon.1 client_cookie=c54fe922ed69a800 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:22.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.682+0000 7fca5affd700 1 -- 172.21.15.17:0/3678491925 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca5400b8a0 con 0x7fca5c1405d0 2022-08-01T19:34:22.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.682+0000 7fca5affd700 1 -- 172.21.15.17:0/3678491925 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca5400ba00 con 0x7fca5c1405d0 2022-08-01T19:34:22.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.682+0000 7fca5affd700 1 -- 172.21.15.17:0/3678491925 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca5400bb70 con 0x7fca5c1405d0 2022-08-01T19:34:22.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.682+0000 7fca60b3e700 1 -- 172.21.15.17:0/3678491925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca5c1405d0 msgr2=0x7fca5c140a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:22.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.682+0000 7fca60b3e700 1 --2- 172.21.15.17:0/3678491925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca5c1405d0 0x7fca5c140a60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fca540032f0 tx=0x7fca54009290).stop 2022-08-01T19:34:22.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.686+0000 7fca60b3e700 1 -- 172.21.15.17:0/3678491925 shutdown_connections 2022-08-01T19:34:22.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.686+0000 7fca60b3e700 1 --2- 172.21.15.17:0/3678491925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca5c1405d0 0x7fca5c140a60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:22.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.686+0000 7fca60b3e700 1 --2- 172.21.15.17:0/3678491925 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fca5c0614b0 0x7fca5c061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:22.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.686+0000 7fca60b3e700 1 -- 172.21.15.17:0/3678491925 >> 172.21.15.17:0/3678491925 conn(0x7fca5c066f70 msgr2=0x7fca5c067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:22.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.686+0000 7fca60b3e700 1 -- 172.21.15.17:0/3678491925 shutdown_connections 2022-08-01T19:34:22.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.686+0000 7fca60b3e700 1 -- 172.21.15.17:0/3678491925 wait complete. 2022-08-01T19:34:22.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.686+0000 7fca60b3e700 1 Processor -- start 2022-08-01T19:34:22.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.686+0000 7fca60b3e700 1 -- start start 2022-08-01T19:34:22.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.686+0000 7fca60b3e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca5c0614b0 0x7fca5c0b7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:22.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.686+0000 7fca60b3e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca5c0ad750 0x7fca5c13fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:22.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.686+0000 7fca60b3e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fca5c1405d0 0x7fca5c140040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:22.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.686+0000 7fca60b3e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c09f530 con 0x7fca5c0ad750 2022-08-01T19:34:22.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.686+0000 7fca60b3e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c0b0d60 con 0x7fca5c0614b0 2022-08-01T19:34:22.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.686+0000 7fca60b3e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c0b1000 con 0x7fca5c1405d0 2022-08-01T19:34:22.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.690+0000 7fca52ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca5c0ad750 0x7fca5c13fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:22.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.690+0000 7fca52ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca5c0ad750 0x7fca5c13fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36830/0 (socket says 172.21.15.17:36830) 2022-08-01T19:34:22.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.690+0000 7fca5bfff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fca5c1405d0 0x7fca5c140040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:22.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.690+0000 7fca52ffd700 1 -- 172.21.15.17:0/1819720028 learned_addr learned my addr 172.21.15.17:0/1819720028 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:22.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.690+0000 7fca5b7fe700 1 --2- 172.21.15.17:0/1819720028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca5c0614b0 0x7fca5c0b7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:22.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.690+0000 7fca52ffd700 1 -- 172.21.15.17:0/1819720028 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fca5c1405d0 msgr2=0x7fca5c140040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:22.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.690+0000 7fca52ffd700 1 --2- 172.21.15.17:0/1819720028 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fca5c1405d0 0x7fca5c140040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:22.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.690+0000 7fca52ffd700 1 -- 172.21.15.17:0/1819720028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca5c0614b0 msgr2=0x7fca5c0b7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:22.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.690+0000 7fca52ffd700 1 --2- 172.21.15.17:0/1819720028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca5c0614b0 0x7fca5c0b7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:22.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.690+0000 7fca52ffd700 1 -- 172.21.15.17:0/1819720028 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca54009530 con 0x7fca5c0ad750 2022-08-01T19:34:22.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.690+0000 7fca52ffd700 1 --2- 172.21.15.17:0/1819720028 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca5c0ad750 0x7fca5c13fb50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fca5c0b6c70 tx=0x7fca44007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:22.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.690+0000 7fca597fa700 1 -- 172.21.15.17:0/1819720028 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca44007030 con 0x7fca5c0ad750 2022-08-01T19:34:22.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.690+0000 7fca597fa700 1 -- 172.21.15.17:0/1819720028 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca44007190 con 0x7fca5c0ad750 2022-08-01T19:34:22.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.690+0000 7fca60b3e700 1 -- 172.21.15.17:0/1819720028 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca5c0b12a0 con 0x7fca5c0ad750 2022-08-01T19:34:22.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.690+0000 7fca597fa700 1 -- 172.21.15.17:0/1819720028 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca44006510 con 0x7fca5c0ad750 2022-08-01T19:34:22.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.690+0000 7fca60b3e700 1 -- 172.21.15.17:0/1819720028 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca5c0b0790 con 0x7fca5c0ad750 2022-08-01T19:34:22.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.690+0000 7fca60b3e700 1 -- 172.21.15.17:0/1819720028 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca5c0af5f0 con 0x7fca5c0ad750 2022-08-01T19:34:22.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.690+0000 7fca597fa700 1 -- 172.21.15.17:0/1819720028 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fca44010030 con 0x7fca5c0ad750 2022-08-01T19:34:22.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.694+0000 7fca597fa700 1 --2- 172.21.15.17:0/1819720028 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fca400539e0 0x7fca40055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:22.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.694+0000 7fca597fa700 1 -- 172.21.15.17:0/1819720028 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fca4406bc70 con 0x7fca5c0ad750 2022-08-01T19:34:22.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.694+0000 7fca5b7fe700 1 --2- 172.21.15.17:0/1819720028 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fca400539e0 0x7fca40055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:22.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.694+0000 7fca5b7fe700 1 --2- 172.21.15.17:0/1819720028 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fca400539e0 0x7fca40055ea0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fca4c00b000 tx=0x7fca4c006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:22.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.702+0000 7fca597fa700 1 -- 172.21.15.17:0/1819720028 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fca44042990 con 0x7fca5c0ad750 2022-08-01T19:34:22.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.826+0000 7fca60b3e700 1 -- 172.21.15.17:0/1819720028 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fca5c0574e0 con 0x7fca5c0ad750 2022-08-01T19:34:22.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.826+0000 7fca597fa700 1 -- 172.21.15.17:0/1819720028 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+10 (secure 0 0 0) 0x7fca44045fb0 con 0x7fca5c0ad750 2022-08-01T19:34:22.833 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:34:22.834 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.826+0000 7fca60b3e700 1 -- 172.21.15.17:0/1819720028 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fca400539e0 msgr2=0x7fca40055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:22.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.826+0000 7fca60b3e700 1 --2- 172.21.15.17:0/1819720028 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fca400539e0 0x7fca40055ea0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fca4c00b000 tx=0x7fca4c006df0).stop 2022-08-01T19:34:22.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.826+0000 7fca60b3e700 1 -- 172.21.15.17:0/1819720028 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca5c0ad750 msgr2=0x7fca5c13fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:22.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.826+0000 7fca60b3e700 1 --2- 172.21.15.17:0/1819720028 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca5c0ad750 0x7fca5c13fb50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fca5c0b6c70 tx=0x7fca44007c10).stop 2022-08-01T19:34:22.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.826+0000 7fca60b3e700 1 -- 172.21.15.17:0/1819720028 shutdown_connections 2022-08-01T19:34:22.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.826+0000 7fca60b3e700 1 --2- 172.21.15.17:0/1819720028 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fca400539e0 0x7fca40055ea0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:22.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.826+0000 7fca60b3e700 1 --2- 172.21.15.17:0/1819720028 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fca5c1405d0 0x7fca5c140040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:22.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.826+0000 7fca60b3e700 1 --2- 172.21.15.17:0/1819720028 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca5c0ad750 0x7fca5c13fb50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:22.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.830+0000 7fca60b3e700 1 --2- 172.21.15.17:0/1819720028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca5c0614b0 0x7fca5c0b7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:22.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.830+0000 7fca60b3e700 1 -- 172.21.15.17:0/1819720028 >> 172.21.15.17:0/1819720028 conn(0x7fca5c066f70 msgr2=0x7fca5c05cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:22.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.830+0000 7fca60b3e700 1 -- 172.21.15.17:0/1819720028 shutdown_connections 2022-08-01T19:34:22.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.830+0000 7fca60b3e700 1 -- 172.21.15.17:0/1819720028 wait complete. 2022-08-01T19:34:22.846 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T19:34:22.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.954+0000 7f24f6889700 1 Processor -- start 2022-08-01T19:34:22.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.954+0000 7f24f6889700 1 -- start start 2022-08-01T19:34:22.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.954+0000 7f24f6889700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f24f00b4310 0x7f24f0140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:22.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.954+0000 7f24f6889700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24f0143cf0 con 0x7f24f0146ae0 2022-08-01T19:34:22.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.954+0000 7f24f6889700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24f0143f90 con 0x7f24f00b4820 2022-08-01T19:34:22.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.954+0000 7f24f6889700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24f01476a0 con 0x7f24f00b4310 2022-08-01T19:34:22.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.954+0000 7f24effff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f24f0146ae0 0x7f24f01403c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57262/0 (socket says 172.21.15.17:57262) 2022-08-01T19:34:22.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.954+0000 7f24effff700 1 -- 172.21.15.17:0/183104853 learned_addr learned my addr 172.21.15.17:0/183104853 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:22.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.954+0000 7f24ef7fe700 1 --2- 172.21.15.17:0/183104853 >> v2:172.21.15.77:3300/0 conn(0x7f24f00b4310 0x7f24f0140ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:22.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.954+0000 7f24eeffd700 1 -- 172.21.15.17:0/183104853 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1062455820 0 0) 0x7f24f0143cf0 con 0x7f24f0146ae0 2022-08-01T19:34:22.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.954+0000 7f24eeffd700 1 -- 172.21.15.17:0/183104853 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24d4003260 con 0x7f24f0146ae0 2022-08-01T19:34:22.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.954+0000 7f24ef7fe700 1 -- 172.21.15.17:0/183104853 >> v1:172.21.15.77:6789/0 conn(0x7f24f00b4820 legacy=0x7f24f00b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:22.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.954+0000 7f24eeffd700 1 -- 172.21.15.17:0/183104853 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2277705822 0 0) 0x7f24f0143f90 con 0x7f24f00b4820 2022-08-01T19:34:22.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.954+0000 7f24ef7fe700 1 -- 172.21.15.17:0/183104853 >> v1:172.21.15.17:6789/0 conn(0x7f24f0146ae0 legacy=0x7f24f01403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:22.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.954+0000 7f24ef7fe700 1 -- 172.21.15.17:0/183104853 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f00b0e20 con 0x7f24f00b4310 2022-08-01T19:34:22.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.954+0000 7f24ef7fe700 1 --2- 172.21.15.17:0/183104853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24f00b4310 0x7f24f0140ad0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f24e8001b80 tx=0x7f24d8006df0).ready entity=mon.1 client_cookie=d397ae82c828e466 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:22.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.954+0000 7f24eeffd700 1 -- 172.21.15.17:0/183104853 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24e80019e0 con 0x7f24f00b4310 2022-08-01T19:34:22.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.954+0000 7f24eeffd700 1 -- 172.21.15.17:0/183104853 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d8000e90 con 0x7f24f00b4310 2022-08-01T19:34:22.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.954+0000 7f24eeffd700 1 -- 172.21.15.17:0/183104853 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24d8011400 con 0x7f24f00b4310 2022-08-01T19:34:22.967 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.954+0000 7f24f6889700 1 -- 172.21.15.17:0/183104853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24f00b4310 msgr2=0x7f24f0140ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:22.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.954+0000 7f24f6889700 1 --2- 172.21.15.17:0/183104853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24f00b4310 0x7f24f0140ad0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f24e8001b80 tx=0x7f24d8006df0).stop 2022-08-01T19:34:22.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.954+0000 7f24f6889700 1 -- 172.21.15.17:0/183104853 shutdown_connections 2022-08-01T19:34:22.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.954+0000 7f24f6889700 1 --2- 172.21.15.17:0/183104853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24f00b4310 0x7f24f0140ad0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:22.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.954+0000 7f24f6889700 1 -- 172.21.15.17:0/183104853 >> 172.21.15.17:0/183104853 conn(0x7f24f005c620 msgr2=0x7f24f005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:22.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.954+0000 7f24f6889700 1 -- 172.21.15.17:0/183104853 shutdown_connections 2022-08-01T19:34:22.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.954+0000 7f24f6889700 1 -- 172.21.15.17:0/183104853 wait complete. 2022-08-01T19:34:22.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.958+0000 7f24f6889700 1 Processor -- start 2022-08-01T19:34:22.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.958+0000 7f24f6889700 1 -- start start 2022-08-01T19:34:22.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.958+0000 7f24f6889700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24f00b4310 0x7f24f00add30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:22.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.958+0000 7f24f6889700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24f00b4820 0x7f24f00af130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:22.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.958+0000 7f24f6889700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24f0146ae0 0x7f24f00af620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:22.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.958+0000 7f24f6889700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f0152680 con 0x7f24f00b4820 2022-08-01T19:34:22.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.958+0000 7f24f6889700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f0152920 con 0x7f24f00b4310 2022-08-01T19:34:22.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.958+0000 7f24f6889700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f0152bc0 con 0x7f24f0146ae0 2022-08-01T19:34:22.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.958+0000 7f24ef7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24f00b4820 0x7f24f00af130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:22.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.958+0000 7f24ef7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24f00b4820 0x7f24f00af130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36842/0 (socket says 172.21.15.17:36842) 2022-08-01T19:34:22.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.958+0000 7f24ef7fe700 1 -- 172.21.15.17:0/777172138 learned_addr learned my addr 172.21.15.17:0/777172138 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:22.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.958+0000 7f24f4e26700 1 --2- 172.21.15.17:0/777172138 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24f0146ae0 0x7f24f00af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:22.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.958+0000 7f24effff700 1 --2- 172.21.15.17:0/777172138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24f00b4310 0x7f24f00add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:22.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.958+0000 7f24ef7fe700 1 -- 172.21.15.17:0/777172138 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24f0146ae0 msgr2=0x7f24f00af620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:22.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.958+0000 7f24ef7fe700 1 --2- 172.21.15.17:0/777172138 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24f0146ae0 0x7f24f00af620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:22.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.958+0000 7f24ef7fe700 1 -- 172.21.15.17:0/777172138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24f00b4310 msgr2=0x7f24f00add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:22.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.958+0000 7f24ef7fe700 1 --2- 172.21.15.17:0/777172138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24f00b4310 0x7f24f00add30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:22.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.958+0000 7f24ef7fe700 1 -- 172.21.15.17:0/777172138 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d800b180 con 0x7f24f00b4820 2022-08-01T19:34:22.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.958+0000 7f24ef7fe700 1 --2- 172.21.15.17:0/777172138 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24f00b4820 0x7f24f00af130 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f24e8001b80 tx=0x7f24f0147200).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:22.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.958+0000 7f24ed7fa700 1 -- 172.21.15.17:0/777172138 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24d8015070 con 0x7f24f00b4820 2022-08-01T19:34:22.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.958+0000 7f24ed7fa700 1 -- 172.21.15.17:0/777172138 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d8012e00 con 0x7f24f00b4820 2022-08-01T19:34:22.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.958+0000 7f24f6889700 1 -- 172.21.15.17:0/777172138 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f0152e60 con 0x7f24f00b4820 2022-08-01T19:34:22.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.958+0000 7f24ed7fa700 1 -- 172.21.15.17:0/777172138 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24d800de60 con 0x7f24f00b4820 2022-08-01T19:34:22.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.958+0000 7f24f6889700 1 -- 172.21.15.17:0/777172138 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f00aec10 con 0x7f24f00b4820 2022-08-01T19:34:22.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.962+0000 7f24f6889700 1 -- 172.21.15.17:0/777172138 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24f0152ff0 con 0x7f24f00b4820 2022-08-01T19:34:22.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.962+0000 7f24ed7fa700 1 -- 172.21.15.17:0/777172138 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f24d8024070 con 0x7f24f00b4820 2022-08-01T19:34:22.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.962+0000 7f24ed7fa700 1 --2- 172.21.15.17:0/777172138 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f24c8053b00 0x7f24c8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:22.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.962+0000 7f24ed7fa700 1 -- 172.21.15.17:0/777172138 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f24d806c6b0 con 0x7f24f00b4820 2022-08-01T19:34:22.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.962+0000 7f24effff700 1 --2- 172.21.15.17:0/777172138 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f24c8053b00 0x7f24c8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:22.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.962+0000 7f24effff700 1 --2- 172.21.15.17:0/777172138 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f24c8053b00 0x7f24c8055fc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f24e800b000 tx=0x7f24e8000f40).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:22.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:22.970+0000 7f24ed7fa700 1 -- 172.21.15.17:0/777172138 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f24d8043360 con 0x7f24f00b4820 2022-08-01T19:34:23.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.098+0000 7f24f6889700 1 -- 172.21.15.17:0/777172138 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f24f00aec10 con 0x7f24f00b4820 2022-08-01T19:34:23.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.098+0000 7f24ed7fa700 1 -- 172.21.15.17:0/777172138 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+10 (secure 0 0 0) 0x7f24f00aec10 con 0x7f24f00b4820 2022-08-01T19:34:23.104 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:34:23.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.098+0000 7f24f6889700 1 -- 172.21.15.17:0/777172138 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f24c8053b00 msgr2=0x7f24c8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:23.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.098+0000 7f24f6889700 1 --2- 172.21.15.17:0/777172138 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f24c8053b00 0x7f24c8055fc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f24e800b000 tx=0x7f24e8000f40).stop 2022-08-01T19:34:23.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.098+0000 7f24f6889700 1 -- 172.21.15.17:0/777172138 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24f00b4820 msgr2=0x7f24f00af130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:23.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.098+0000 7f24f6889700 1 --2- 172.21.15.17:0/777172138 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24f00b4820 0x7f24f00af130 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f24e8001b80 tx=0x7f24f0147200).stop 2022-08-01T19:34:23.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.098+0000 7f24f6889700 1 -- 172.21.15.17:0/777172138 shutdown_connections 2022-08-01T19:34:23.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.098+0000 7f24f6889700 1 --2- 172.21.15.17:0/777172138 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f24c8053b00 0x7f24c8055fc0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:23.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.098+0000 7f24f6889700 1 --2- 172.21.15.17:0/777172138 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24f0146ae0 0x7f24f00af620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:23.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.098+0000 7f24f6889700 1 --2- 172.21.15.17:0/777172138 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24f00b4820 0x7f24f00af130 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:23.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.098+0000 7f24f6889700 1 --2- 172.21.15.17:0/777172138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24f00b4310 0x7f24f00add30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:23.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.098+0000 7f24f6889700 1 -- 172.21.15.17:0/777172138 >> 172.21.15.17:0/777172138 conn(0x7f24f005c620 msgr2=0x7f24f013b350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:23.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.098+0000 7f24f6889700 1 -- 172.21.15.17:0/777172138 shutdown_connections 2022-08-01T19:34:23.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.098+0000 7f24f6889700 1 -- 172.21.15.17:0/777172138 wait complete. 2022-08-01T19:34:23.117 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T19:34:23.117 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:34:23.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.218+0000 7f9c970ce700 1 Processor -- start 2022-08-01T19:34:23.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.218+0000 7f9c970ce700 1 -- start start 2022-08-01T19:34:23.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.218+0000 7f9c970ce700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9c900b6a90 0x7f9c900acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:23.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.218+0000 7f9c970ce700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c900b6e70 con 0x7f9c900aa760 2022-08-01T19:34:23.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.218+0000 7f9c970ce700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c90061370 con 0x7f9c90060aa0 2022-08-01T19:34:23.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.218+0000 7f9c970ce700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c90061610 con 0x7f9c900b6a90 2022-08-01T19:34:23.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.218+0000 7f9c94e6a700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9c900b6a90 0x7f9c900acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:23.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.218+0000 7f9c9566b700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9c900aa760 0x7f9c9013c600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57278/0 (socket says 172.21.15.17:57278) 2022-08-01T19:34:23.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.218+0000 7f9c9566b700 1 -- 172.21.15.17:0/1024097464 learned_addr learned my addr 172.21.15.17:0/1024097464 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:23.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.218+0000 7f9c94e6a700 1 -- 172.21.15.17:0/1024097464 >> v1:172.21.15.77:6789/0 conn(0x7f9c90060aa0 legacy=0x7f9c900aa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:23.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.218+0000 7f9c94e6a700 1 -- 172.21.15.17:0/1024097464 >> v1:172.21.15.17:6789/0 conn(0x7f9c900aa760 legacy=0x7f9c9013c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:23.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.218+0000 7f9c94e6a700 1 -- 172.21.15.17:0/1024097464 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c900618b0 con 0x7f9c900b6a90 2022-08-01T19:34:23.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.218+0000 7f9c94e6a700 1 --2- 172.21.15.17:0/1024097464 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c900b6a90 0x7f9c900acee0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f9c84002f70 tx=0x7f9c84006df0).ready entity=mon.0 client_cookie=1df655f0b72f8b0d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:23.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.218+0000 7f9c8f7fe700 1 -- 172.21.15.17:0/1024097464 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c8400fa10 con 0x7f9c900b6a90 2022-08-01T19:34:23.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.218+0000 7f9c8f7fe700 1 -- 172.21.15.17:0/1024097464 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c84000e90 con 0x7f9c900b6a90 2022-08-01T19:34:23.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.218+0000 7f9c8f7fe700 1 -- 172.21.15.17:0/1024097464 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c8400fd60 con 0x7f9c900b6a90 2022-08-01T19:34:23.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.218+0000 7f9c970ce700 1 -- 172.21.15.17:0/1024097464 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c900b6a90 msgr2=0x7f9c900acee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:23.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.218+0000 7f9c970ce700 1 --2- 172.21.15.17:0/1024097464 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c900b6a90 0x7f9c900acee0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f9c84002f70 tx=0x7f9c84006df0).stop 2022-08-01T19:34:23.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.218+0000 7f9c970ce700 1 -- 172.21.15.17:0/1024097464 shutdown_connections 2022-08-01T19:34:23.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.218+0000 7f9c970ce700 1 --2- 172.21.15.17:0/1024097464 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c900b6a90 0x7f9c900acee0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:23.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.218+0000 7f9c970ce700 1 -- 172.21.15.17:0/1024097464 >> 172.21.15.17:0/1024097464 conn(0x7f9c900b3610 msgr2=0x7f9c900b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:23.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.218+0000 7f9c970ce700 1 -- 172.21.15.17:0/1024097464 shutdown_connections 2022-08-01T19:34:23.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.218+0000 7f9c970ce700 1 -- 172.21.15.17:0/1024097464 wait complete. 2022-08-01T19:34:23.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.222+0000 7f9c970ce700 1 Processor -- start 2022-08-01T19:34:23.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.222+0000 7f9c970ce700 1 -- start start 2022-08-01T19:34:23.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.222+0000 7f9c970ce700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c90060aa0 0x7f9c900b4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:23.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.222+0000 7f9c970ce700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9c900aa760 0x7f9c9014ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:23.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.222+0000 7f9c970ce700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c900b6a90 0x7f9c9014ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:23.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.222+0000 7f9c970ce700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c90141760 con 0x7f9c900b6a90 2022-08-01T19:34:23.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.222+0000 7f9c970ce700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c90152670 con 0x7f9c90060aa0 2022-08-01T19:34:23.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.222+0000 7f9c970ce700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c90152910 con 0x7f9c900aa760 2022-08-01T19:34:23.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.222+0000 7f9c9566b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c900b6a90 0x7f9c9014ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:23.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.222+0000 7f9c9566b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c900b6a90 0x7f9c9014ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36858/0 (socket says 172.21.15.17:36858) 2022-08-01T19:34:23.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.222+0000 7f9c9566b700 1 -- 172.21.15.17:0/2743082018 learned_addr learned my addr 172.21.15.17:0/2743082018 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:23.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.222+0000 7f9c8ffff700 1 --2- 172.21.15.17:0/2743082018 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9c900aa760 0x7f9c9014ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:23.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.222+0000 7f9c94e6a700 1 --2- 172.21.15.17:0/2743082018 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c90060aa0 0x7f9c900b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:23.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.222+0000 7f9c9566b700 1 -- 172.21.15.17:0/2743082018 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9c900aa760 msgr2=0x7f9c9014ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:23.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.222+0000 7f9c9566b700 1 --2- 172.21.15.17:0/2743082018 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9c900aa760 0x7f9c9014ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:23.234 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.222+0000 7f9c9566b700 1 -- 172.21.15.17:0/2743082018 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c90060aa0 msgr2=0x7f9c900b4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:23.234 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.222+0000 7f9c9566b700 1 --2- 172.21.15.17:0/2743082018 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c90060aa0 0x7f9c900b4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:23.234 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.222+0000 7f9c9566b700 1 -- 172.21.15.17:0/2743082018 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c8400b180 con 0x7f9c900b6a90 2022-08-01T19:34:23.234 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.222+0000 7f9c9566b700 1 --2- 172.21.15.17:0/2743082018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c900b6a90 0x7f9c9014ef40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f9c90060490 tx=0x7f9c88007f30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:23.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.222+0000 7f9c8dffb700 1 -- 172.21.15.17:0/2743082018 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c88005b80 con 0x7f9c900b6a90 2022-08-01T19:34:23.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.222+0000 7f9c8dffb700 1 -- 172.21.15.17:0/2743082018 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c88005ce0 con 0x7f9c900b6a90 2022-08-01T19:34:23.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.222+0000 7f9c970ce700 1 -- 172.21.15.17:0/2743082018 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c90142070 con 0x7f9c900b6a90 2022-08-01T19:34:23.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.222+0000 7f9c8dffb700 1 -- 172.21.15.17:0/2743082018 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c880072c0 con 0x7f9c900b6a90 2022-08-01T19:34:23.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.222+0000 7f9c970ce700 1 -- 172.21.15.17:0/2743082018 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c9014f430 con 0x7f9c900b6a90 2022-08-01T19:34:23.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.226+0000 7f9c970ce700 1 -- 172.21.15.17:0/2743082018 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c900b28f0 con 0x7f9c900b6a90 2022-08-01T19:34:23.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.226+0000 7f9c8dffb700 1 -- 172.21.15.17:0/2743082018 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9c8801a060 con 0x7f9c900b6a90 2022-08-01T19:34:23.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.226+0000 7f9c8dffb700 1 --2- 172.21.15.17:0/2743082018 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9c70053b00 0x7f9c70055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:23.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.226+0000 7f9c8dffb700 1 -- 172.21.15.17:0/2743082018 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f9c8806bab0 con 0x7f9c900b6a90 2022-08-01T19:34:23.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.226+0000 7f9c94e6a700 1 --2- 172.21.15.17:0/2743082018 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9c70053b00 0x7f9c70055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:23.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.226+0000 7f9c94e6a700 1 --2- 172.21.15.17:0/2743082018 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9c70053b00 0x7f9c70055fc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f9c8401e000 tx=0x7f9c84017000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:23.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.230+0000 7f9c8dffb700 1 -- 172.21.15.17:0/2743082018 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9c88042760 con 0x7f9c900b6a90 2022-08-01T19:34:23.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.394+0000 7f9c970ce700 1 -- 172.21.15.17:0/2743082018 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c9014e770 con 0x7f9c900b6a90 2022-08-01T19:34:23.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.394+0000 7f9c8dffb700 1 -- 172.21.15.17:0/2743082018 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+2486 (secure 0 0 0) 0x7f9c88018030 con 0x7f9c900b6a90 2022-08-01T19:34:23.402 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:23.403 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2116883486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2116883486},{"type":"v1","addr":"172.21.15.17:6833","nonce":2116883486}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.17:6835/2974833789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2974833789},{"type":"v1","addr":"172.21.15.17:6835","nonce":2974833789}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":4}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:34:09.692662+0000","modified":"2022-08-01T19:34:11.425752+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":5,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6835/1842780453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1842780453},{"type":"v1","addr":"172.21.15.77:6835","nonce":1842780453}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2022-08-01T19:34:23.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.398+0000 7f9c970ce700 1 -- 172.21.15.17:0/2743082018 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9c70053b00 msgr2=0x7f9c70055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:23.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.398+0000 7f9c970ce700 1 --2- 172.21.15.17:0/2743082018 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9c70053b00 0x7f9c70055fc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f9c8401e000 tx=0x7f9c84017000).stop 2022-08-01T19:34:23.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.398+0000 7f9c970ce700 1 -- 172.21.15.17:0/2743082018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c900b6a90 msgr2=0x7f9c9014ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:23.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.398+0000 7f9c970ce700 1 --2- 172.21.15.17:0/2743082018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c900b6a90 0x7f9c9014ef40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f9c90060490 tx=0x7f9c88007f30).stop 2022-08-01T19:34:23.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.398+0000 7f9c970ce700 1 -- 172.21.15.17:0/2743082018 shutdown_connections 2022-08-01T19:34:23.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.398+0000 7f9c970ce700 1 --2- 172.21.15.17:0/2743082018 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9c70053b00 0x7f9c70055fc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:23.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.398+0000 7f9c970ce700 1 --2- 172.21.15.17:0/2743082018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c900b6a90 0x7f9c9014ef40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:23.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.398+0000 7f9c970ce700 1 --2- 172.21.15.17:0/2743082018 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9c900aa760 0x7f9c9014ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:23.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.398+0000 7f9c970ce700 1 --2- 172.21.15.17:0/2743082018 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c90060aa0 0x7f9c900b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:23.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.398+0000 7f9c970ce700 1 -- 172.21.15.17:0/2743082018 >> 172.21.15.17:0/2743082018 conn(0x7f9c900b3610 msgr2=0x7f9c90137650 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:23.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.398+0000 7f9c970ce700 1 -- 172.21.15.17:0/2743082018 shutdown_connections 2022-08-01T19:34:23.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:23.398+0000 7f9c970ce700 1 -- 172.21.15.17:0/2743082018 wait complete. 2022-08-01T19:34:23.409 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 7 2022-08-01T19:34:23.723 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:34:09.692662+0000', 'modified': '2022-08-01T19:34:11.425752+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4285}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4285': {'gid': 4285, 'name': 'b', 'rank': 0, 'incarnation': 5, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.77:6835/1842780453', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6834', 'nonce': 1842780453}, {'type': 'v1', 'addr': '172.21.15.77:6835', 'nonce': 1842780453}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:34:23.723 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:34:23.724 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-08-01T19:34:23.839 INFO:teuthology.orchestra.run.smithi077.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":4285,"want_state":"up:active","state":"up:active","rank_uptime":14.133930250000001,"mdsmap_epoch":7,"osdmap_epoch":25,"osdmap_epoch_barrier":25,"uptime":20.484239601999999} 2022-08-01T19:34:23.839 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 4285, "mdsmap_epoch": 7, "osdmap_epoch": 25, "osdmap_epoch_barrier": 25, "rank_uptime": 14.13393025, "state": "up:active", "uptime": 20.484239602, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:34:23.840 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T19:34:23.840 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-08-01T19:34:23.841 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi085.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-08-01T19:34:23.841 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:34:23.841 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:34:23.850 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:34:23.866 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:34:23.866 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:34:23.926 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:34:23.927 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:34:23.928 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:34:23.928 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:34:23.935 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T19:34:23.936 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:34:23.938 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:34:23.954 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:34:23.955 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:34:23.964 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:34:23.964 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:34:23.965 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:34:23.966 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:34:24.027 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:2022-08-01T19:34:24.027+0000 7f88a9740080 -1 init, newargv = 0x55a4f3420a10 newargc=9 2022-08-01T19:34:24.041 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:ceph-fuse[22399]: starting ceph client 2022-08-01T19:34:24.042 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:ceph-fuse[22399]: starting fuse 2022-08-01T19:34:25.114 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:34:25.132 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:34:25.133 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:34:25.147 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:34:25.148 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:34:25.149 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:34:25.150 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:34:25.155 INFO:teuthology.orchestra.run.smithi085.stdout:52 2022-08-01T19:34:25.156 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T19:34:25.156 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:34:25.157 DEBUG:teuthology.orchestra.run.smithi085:> sudo python3 -c ' 2022-08-01T19:34:25.157 DEBUG:teuthology.orchestra.run.smithi085:> import glob 2022-08-01T19:34:25.157 DEBUG:teuthology.orchestra.run.smithi085:> import re 2022-08-01T19:34:25.158 DEBUG:teuthology.orchestra.run.smithi085:> import os 2022-08-01T19:34:25.158 DEBUG:teuthology.orchestra.run.smithi085:> import subprocess 2022-08-01T19:34:25.158 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:34:25.159 DEBUG:teuthology.orchestra.run.smithi085:> def find_socket(client_name): 2022-08-01T19:34:25.159 DEBUG:teuthology.orchestra.run.smithi085:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-08-01T19:34:25.159 DEBUG:teuthology.orchestra.run.smithi085:> files = glob.glob(asok_path) 2022-08-01T19:34:25.160 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:34:25.160 DEBUG:teuthology.orchestra.run.smithi085:> # Given a non-glob path, it better be there 2022-08-01T19:34:25.160 DEBUG:teuthology.orchestra.run.smithi085:> if "*" not in asok_path: 2022-08-01T19:34:25.161 DEBUG:teuthology.orchestra.run.smithi085:> assert(len(files) == 1) 2022-08-01T19:34:25.161 DEBUG:teuthology.orchestra.run.smithi085:> return files[0] 2022-08-01T19:34:25.161 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:34:25.162 DEBUG:teuthology.orchestra.run.smithi085:> for f in files: 2022-08-01T19:34:25.162 DEBUG:teuthology.orchestra.run.smithi085:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T19:34:25.163 DEBUG:teuthology.orchestra.run.smithi085:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T19:34:25.163 DEBUG:teuthology.orchestra.run.smithi085:> return f 2022-08-01T19:34:25.163 DEBUG:teuthology.orchestra.run.smithi085:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T19:34:25.164 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:34:25.164 DEBUG:teuthology.orchestra.run.smithi085:> print(find_socket("client.0")) 2022-08-01T19:34:25.164 DEBUG:teuthology.orchestra.run.smithi085:> ' 2022-08-01T19:34:25.243 INFO:teuthology.orchestra.run.smithi085.stdout:/var/run/ceph/ceph-client.0.22399.asok 2022-08-01T19:34:25.248 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22399.asok 2022-08-01T19:34:25.249 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:34:25.249 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22399.asok status 2022-08-01T19:34:25.337 INFO:teuthology.orchestra.run.smithi085.stdout:{ 2022-08-01T19:34:25.353 INFO:teuthology.orchestra.run.smithi085.stdout: "metadata": { 2022-08-01T19:34:25.354 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T19:34:25.354 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T19:34:25.354 INFO:teuthology.orchestra.run.smithi085.stdout: "entity_id": "0", 2022-08-01T19:34:25.355 INFO:teuthology.orchestra.run.smithi085.stdout: "hostname": "smithi085", 2022-08-01T19:34:25.355 INFO:teuthology.orchestra.run.smithi085.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-08-01T19:34:25.355 INFO:teuthology.orchestra.run.smithi085.stdout: "pid": "22399", 2022-08-01T19:34:25.355 INFO:teuthology.orchestra.run.smithi085.stdout: "root": "/" 2022-08-01T19:34:25.356 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:34:25.356 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_count": 0, 2022-08-01T19:34:25.356 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_pinned_count": 0, 2022-08-01T19:34:25.356 INFO:teuthology.orchestra.run.smithi085.stdout: "id": 4650, 2022-08-01T19:34:25.357 INFO:teuthology.orchestra.run.smithi085.stdout: "inst": { 2022-08-01T19:34:25.357 INFO:teuthology.orchestra.run.smithi085.stdout: "name": { 2022-08-01T19:34:25.357 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "client", 2022-08-01T19:34:25.358 INFO:teuthology.orchestra.run.smithi085.stdout: "num": 4650 2022-08-01T19:34:25.358 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:34:25.358 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2022-08-01T19:34:25.358 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2022-08-01T19:34:25.359 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2022-08-01T19:34:25.359 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 1174544650 2022-08-01T19:34:25.359 INFO:teuthology.orchestra.run.smithi085.stdout: } 2022-08-01T19:34:25.359 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:34:25.360 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2022-08-01T19:34:25.360 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2022-08-01T19:34:25.360 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2022-08-01T19:34:25.360 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 1174544650 2022-08-01T19:34:25.361 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:34:25.361 INFO:teuthology.orchestra.run.smithi085.stdout: "inst_str": "client.4650 172.21.15.85:0/1174544650", 2022-08-01T19:34:25.361 INFO:teuthology.orchestra.run.smithi085.stdout: "addr_str": "172.21.15.85:0/1174544650", 2022-08-01T19:34:25.361 INFO:teuthology.orchestra.run.smithi085.stdout: "inode_count": 1, 2022-08-01T19:34:25.361 INFO:teuthology.orchestra.run.smithi085.stdout: "mds_epoch": 7, 2022-08-01T19:34:25.362 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch": 25, 2022-08-01T19:34:25.362 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch_barrier": 0, 2022-08-01T19:34:25.362 INFO:teuthology.orchestra.run.smithi085.stdout: "blacklisted": false, 2022-08-01T19:34:25.362 INFO:teuthology.orchestra.run.smithi085.stdout: "fs_name": "cephfs" 2022-08-01T19:34:25.363 INFO:teuthology.orchestra.run.smithi085.stdout:} 2022-08-01T19:34:25.364 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:34:25.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.470+0000 7fea5359e700 1 Processor -- start 2022-08-01T19:34:25.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.470+0000 7fea5359e700 1 -- start start 2022-08-01T19:34:25.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.474+0000 7fea5359e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fea540b08a0 0x7fea54142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:25.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.474+0000 7fea5359e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fea540b3260 0x7fea541433a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:25.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.474+0000 7fea5359e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea540b0c80 con 0x7fea540ad000 2022-08-01T19:34:25.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.474+0000 7fea5359e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea540ae810 con 0x7fea540b08a0 2022-08-01T19:34:25.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.474+0000 7fea5359e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea540aeab0 con 0x7fea540b3260 2022-08-01T19:34:25.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.474+0000 7fea5259c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fea540b08a0 0x7fea54142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:25.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.474+0000 7fea5259c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fea540b08a0 0x7fea54142eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36862/0 (socket says 172.21.15.17:36862) 2022-08-01T19:34:25.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.474+0000 7fea5259c700 1 -- 172.21.15.17:0/1916120447 learned_addr learned my addr 172.21.15.17:0/1916120447 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:25.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.474+0000 7fea51d9b700 1 --2- 172.21.15.17:0/1916120447 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fea540b3260 0x7fea541433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:25.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.474+0000 7fea5259c700 1 -- 172.21.15.17:0/1916120447 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fea540b3260 msgr2=0x7fea541433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:25.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.474+0000 7fea5259c700 1 --2- 172.21.15.17:0/1916120447 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fea540b3260 0x7fea541433a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:25.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.474+0000 7fea5259c700 1 -- 172.21.15.17:0/1916120447 >> v1:172.21.15.77:6789/0 conn(0x7fea540ad000 legacy=0x7fea5405f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:25.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.474+0000 7fea5259c700 1 -- 172.21.15.17:0/1916120447 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea540aed50 con 0x7fea540b08a0 2022-08-01T19:34:25.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.474+0000 7fea5259c700 1 --2- 172.21.15.17:0/1916120447 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea540b08a0 0x7fea54142eb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7fea44002f70 tx=0x7fea44006f40).ready entity=mon.0 client_cookie=38daca1ae4aafa6c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:25.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.474+0000 7fea5159a700 1 -- 172.21.15.17:0/1916120447 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea4400c880 con 0x7fea540b08a0 2022-08-01T19:34:25.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.474+0000 7fea5159a700 1 -- 172.21.15.17:0/1916120447 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea44000e90 con 0x7fea540b08a0 2022-08-01T19:34:25.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.474+0000 7fea5159a700 1 -- 172.21.15.17:0/1916120447 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea4400cc10 con 0x7fea540b08a0 2022-08-01T19:34:25.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.474+0000 7fea5359e700 1 -- 172.21.15.17:0/1916120447 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea540b08a0 msgr2=0x7fea54142eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:25.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.474+0000 7fea5359e700 1 --2- 172.21.15.17:0/1916120447 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea540b08a0 0x7fea54142eb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7fea44002f70 tx=0x7fea44006f40).stop 2022-08-01T19:34:25.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.474+0000 7fea5359e700 1 -- 172.21.15.17:0/1916120447 shutdown_connections 2022-08-01T19:34:25.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.474+0000 7fea5359e700 1 --2- 172.21.15.17:0/1916120447 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fea540b3260 0x7fea541433a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:25.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.474+0000 7fea5359e700 1 --2- 172.21.15.17:0/1916120447 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea540b08a0 0x7fea54142eb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:25.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.474+0000 7fea5359e700 1 -- 172.21.15.17:0/1916120447 >> 172.21.15.17:0/1916120447 conn(0x7fea540612d0 msgr2=0x7fea54062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:25.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.474+0000 7fea5359e700 1 -- 172.21.15.17:0/1916120447 shutdown_connections 2022-08-01T19:34:25.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.474+0000 7fea5359e700 1 -- 172.21.15.17:0/1916120447 wait complete. 2022-08-01T19:34:25.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.478+0000 7fea5359e700 1 Processor -- start 2022-08-01T19:34:25.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.478+0000 7fea5359e700 1 -- start start 2022-08-01T19:34:25.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.478+0000 7fea5359e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fea540ad000 0x7fea5413f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:25.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.478+0000 7fea5359e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea540b08a0 0x7fea54066620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:25.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.478+0000 7fea5359e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea540b3260 0x7fea54066b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:25.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.478+0000 7fea5359e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea540ac120 con 0x7fea540b3260 2022-08-01T19:34:25.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.478+0000 7fea5359e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea54065d00 con 0x7fea540b08a0 2022-08-01T19:34:25.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.478+0000 7fea5359e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea54064000 con 0x7fea540ad000 2022-08-01T19:34:25.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.478+0000 7fea52d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea540b3260 0x7fea54066b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:25.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.478+0000 7fea52d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea540b3260 0x7fea54066b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36870/0 (socket says 172.21.15.17:36870) 2022-08-01T19:34:25.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.478+0000 7fea52d9d700 1 -- 172.21.15.17:0/303764955 learned_addr learned my addr 172.21.15.17:0/303764955 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:25.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.478+0000 7fea5259c700 1 --2- 172.21.15.17:0/303764955 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fea540ad000 0x7fea5413f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:25.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.478+0000 7fea51d9b700 1 --2- 172.21.15.17:0/303764955 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea540b08a0 0x7fea54066620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:25.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.478+0000 7fea52d9d700 1 -- 172.21.15.17:0/303764955 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fea540ad000 msgr2=0x7fea5413f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:25.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.478+0000 7fea52d9d700 1 --2- 172.21.15.17:0/303764955 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fea540ad000 0x7fea5413f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:25.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.478+0000 7fea52d9d700 1 -- 172.21.15.17:0/303764955 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea540b08a0 msgr2=0x7fea54066620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:25.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.478+0000 7fea52d9d700 1 --2- 172.21.15.17:0/303764955 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea540b08a0 0x7fea54066620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:25.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.478+0000 7fea52d9d700 1 -- 172.21.15.17:0/303764955 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea4400a360 con 0x7fea540b3260 2022-08-01T19:34:25.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.478+0000 7fea52d9d700 1 --2- 172.21.15.17:0/303764955 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea540b3260 0x7fea54066b10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fea540b00e0 tx=0x7fea4c007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:25.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.478+0000 7fea437fe700 1 -- 172.21.15.17:0/303764955 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea4c00f470 con 0x7fea540b3260 2022-08-01T19:34:25.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.478+0000 7fea437fe700 1 -- 172.21.15.17:0/303764955 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea4c00f5d0 con 0x7fea540b3260 2022-08-01T19:34:25.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.478+0000 7fea5359e700 1 -- 172.21.15.17:0/303764955 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea54067000 con 0x7fea540b3260 2022-08-01T19:34:25.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.478+0000 7fea437fe700 1 -- 172.21.15.17:0/303764955 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea4c00f930 con 0x7fea540b3260 2022-08-01T19:34:25.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.478+0000 7fea5359e700 1 -- 172.21.15.17:0/303764955 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea540ac5b0 con 0x7fea540b3260 2022-08-01T19:34:25.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.482+0000 7fea5359e700 1 -- 172.21.15.17:0/303764955 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea540ac420 con 0x7fea540b3260 2022-08-01T19:34:25.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.482+0000 7fea437fe700 1 -- 172.21.15.17:0/303764955 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fea4c019030 con 0x7fea540b3260 2022-08-01T19:34:25.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.482+0000 7fea437fe700 1 --2- 172.21.15.17:0/303764955 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fea30053a70 0x7fea30055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:25.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.482+0000 7fea437fe700 1 -- 172.21.15.17:0/303764955 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fea4c06b900 con 0x7fea540b3260 2022-08-01T19:34:25.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.482+0000 7fea5259c700 1 --2- 172.21.15.17:0/303764955 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fea30053a70 0x7fea30055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:25.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.482+0000 7fea5259c700 1 --2- 172.21.15.17:0/303764955 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fea30053a70 0x7fea30055f30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fea4401b000 tx=0x7fea540a95b0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:25.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.486+0000 7fea437fe700 1 -- 172.21.15.17:0/303764955 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fea4c0425b0 con 0x7fea540b3260 2022-08-01T19:34:25.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.614+0000 7fea5359e700 1 -- 172.21.15.17:0/303764955 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea540ac770 con 0x7fea540b3260 2022-08-01T19:34:25.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.614+0000 7fea437fe700 1 -- 172.21.15.17:0/303764955 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16299 (secure 0 0 0) 0x7fea4c045bd0 con 0x7fea540b3260 2022-08-01T19:34:25.622 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:25.623 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:34:09.692404+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:34:04.635473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-08-01T19:34:07.409434+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:34:25.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.618+0000 7fea5359e700 1 -- 172.21.15.17:0/303764955 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fea30053a70 msgr2=0x7fea30055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:25.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.618+0000 7fea5359e700 1 --2- 172.21.15.17:0/303764955 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fea30053a70 0x7fea30055f30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fea4401b000 tx=0x7fea540a95b0).stop 2022-08-01T19:34:25.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.618+0000 7fea5359e700 1 -- 172.21.15.17:0/303764955 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea540b3260 msgr2=0x7fea54066b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:25.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.618+0000 7fea5359e700 1 --2- 172.21.15.17:0/303764955 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea540b3260 0x7fea54066b10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fea540b00e0 tx=0x7fea4c007f10).stop 2022-08-01T19:34:25.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.618+0000 7fea5359e700 1 -- 172.21.15.17:0/303764955 shutdown_connections 2022-08-01T19:34:25.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.618+0000 7fea5359e700 1 --2- 172.21.15.17:0/303764955 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fea30053a70 0x7fea30055f30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:25.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.618+0000 7fea5359e700 1 --2- 172.21.15.17:0/303764955 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea540b3260 0x7fea54066b10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:25.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.618+0000 7fea5359e700 1 --2- 172.21.15.17:0/303764955 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fea540b08a0 0x7fea54066620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:25.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.618+0000 7fea5359e700 1 --2- 172.21.15.17:0/303764955 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fea540ad000 0x7fea5413f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:25.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.618+0000 7fea5359e700 1 -- 172.21.15.17:0/303764955 >> 172.21.15.17:0/303764955 conn(0x7fea540612d0 msgr2=0x7fea54062ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:25.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.618+0000 7fea5359e700 1 -- 172.21.15.17:0/303764955 shutdown_connections 2022-08-01T19:34:25.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.618+0000 7fea5359e700 1 -- 172.21.15.17:0/303764955 wait complete. 2022-08-01T19:34:25.636 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T19:34:04.635473+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T19:34:07.409434+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T19:34:25.636 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:34:25.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.746+0000 7ff7a60a7700 1 Processor -- start 2022-08-01T19:34:25.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.746+0000 7ff7a60a7700 1 -- start start 2022-08-01T19:34:25.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.746+0000 7ff7a60a7700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff7a00b2710 0x7ff7a014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:25.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.746+0000 7ff7a60a7700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ff7a00a8c50 0x7ff7a014b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:25.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.746+0000 7ff7a60a7700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7a00a7250 con 0x7ff7a00b01a0 2022-08-01T19:34:25.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.746+0000 7ff7a60a7700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7a00a7480 con 0x7ff7a00a8c50 2022-08-01T19:34:25.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.746+0000 7ff7a60a7700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a00a7720 con 0x7ff7a00b2710 2022-08-01T19:34:25.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.746+0000 7ff7a50a5700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff7a00b2710 0x7ff7a014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:25.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.746+0000 7ff7a58a6700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ff7a00a8c50 0x7ff7a014b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:25.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.746+0000 7ff7a50a5700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff7a00b2710 0x7ff7a014afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:32976/0 (socket says 172.21.15.17:32976) 2022-08-01T19:34:25.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.746+0000 7ff7a58a6700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ff7a00a8c50 0x7ff7a014b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:34800/0 (socket says 172.21.15.17:34800) 2022-08-01T19:34:25.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.746+0000 7ff7a50a5700 1 -- 172.21.15.17:0/3223252250 learned_addr learned my addr 172.21.15.17:0/3223252250 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:25.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.746+0000 7ff7a48a4700 1 --1- 172.21.15.17:0/3223252250 >> v1:172.21.15.77:6789/0 conn(0x7ff7a00b01a0 0x7ff7a00a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:45872/0 (socket says 172.21.15.17:45872) 2022-08-01T19:34:25.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.746+0000 7ff7a58a6700 1 -- 172.21.15.17:0/3223252250 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff7a00b2710 msgr2=0x7ff7a014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:25.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.746+0000 7ff7a58a6700 1 --2- 172.21.15.17:0/3223252250 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff7a00b2710 0x7ff7a014afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:25.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.746+0000 7ff7a58a6700 1 -- 172.21.15.17:0/3223252250 >> v1:172.21.15.77:6789/0 conn(0x7ff7a00b01a0 legacy=0x7ff7a00a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:25.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.746+0000 7ff797fff700 1 -- 172.21.15.17:0/3223252250 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2949944123 0 0) 0x7ff7a00a7250 con 0x7ff7a00b01a0 2022-08-01T19:34:25.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.746+0000 7ff7a58a6700 1 -- 172.21.15.17:0/3223252250 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a00a79c0 con 0x7ff7a00a8c50 2022-08-01T19:34:25.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.746+0000 7ff7a58a6700 1 --2- 172.21.15.17:0/3223252250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7a00a8c50 0x7ff7a014b4c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7ff798002f70 tx=0x7ff798006df0).ready entity=mon.1 client_cookie=6a452e411eac570a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:25.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.746+0000 7ff797fff700 1 -- 172.21.15.17:0/3223252250 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff79800d570 con 0x7ff7a00a8c50 2022-08-01T19:34:25.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.746+0000 7ff797fff700 1 -- 172.21.15.17:0/3223252250 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff798000e90 con 0x7ff7a00a8c50 2022-08-01T19:34:25.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.746+0000 7ff797fff700 1 -- 172.21.15.17:0/3223252250 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff79800d8c0 con 0x7ff7a00a8c50 2022-08-01T19:34:25.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.746+0000 7ff7a60a7700 1 -- 172.21.15.17:0/3223252250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7a00a8c50 msgr2=0x7ff7a014b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:25.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.746+0000 7ff7a60a7700 1 --2- 172.21.15.17:0/3223252250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7a00a8c50 0x7ff7a014b4c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7ff798002f70 tx=0x7ff798006df0).stop 2022-08-01T19:34:25.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.746+0000 7ff7a60a7700 1 -- 172.21.15.17:0/3223252250 shutdown_connections 2022-08-01T19:34:25.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.746+0000 7ff7a60a7700 1 --2- 172.21.15.17:0/3223252250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7a00a8c50 0x7ff7a014b4c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:25.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.746+0000 7ff7a60a7700 1 --2- 172.21.15.17:0/3223252250 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff7a00b2710 0x7ff7a014afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:25.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.746+0000 7ff7a60a7700 1 -- 172.21.15.17:0/3223252250 >> 172.21.15.17:0/3223252250 conn(0x7ff7a0060310 msgr2=0x7ff7a00b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:25.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.750+0000 7ff7a60a7700 1 -- 172.21.15.17:0/3223252250 shutdown_connections 2022-08-01T19:34:25.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.750+0000 7ff7a60a7700 1 -- 172.21.15.17:0/3223252250 wait complete. 2022-08-01T19:34:25.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.750+0000 7ff7a60a7700 1 Processor -- start 2022-08-01T19:34:25.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.750+0000 7ff7a60a7700 1 -- start start 2022-08-01T19:34:25.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.750+0000 7ff7a60a7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7a00a8c50 0x7ff7a00ac480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:25.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.750+0000 7ff7a60a7700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff7a00b01a0 0x7ff7a0064cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:25.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.750+0000 7ff7a60a7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7a00b2710 0x7ff7a00651b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:25.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.750+0000 7ff7a60a7700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a00ab060 con 0x7ff7a00a8c50 2022-08-01T19:34:25.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.750+0000 7ff7a60a7700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a0063310 con 0x7ff7a00b2710 2022-08-01T19:34:25.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.750+0000 7ff7a60a7700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a00635b0 con 0x7ff7a00b01a0 2022-08-01T19:34:25.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.750+0000 7ff7a50a5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7a00a8c50 0x7ff7a00ac480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:25.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.750+0000 7ff7a48a4700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff7a00b01a0 0x7ff7a0064cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:25.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.750+0000 7ff7a50a5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7a00a8c50 0x7ff7a00ac480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36884/0 (socket says 172.21.15.17:36884) 2022-08-01T19:34:25.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.750+0000 7ff7a50a5700 1 -- 172.21.15.17:0/2933258967 learned_addr learned my addr 172.21.15.17:0/2933258967 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:25.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.750+0000 7ff7a58a6700 1 --2- 172.21.15.17:0/2933258967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7a00b2710 0x7ff7a00651b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:25.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.750+0000 7ff7a50a5700 1 -- 172.21.15.17:0/2933258967 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff7a00b01a0 msgr2=0x7ff7a0064cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:25.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.750+0000 7ff7a50a5700 1 --2- 172.21.15.17:0/2933258967 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff7a00b01a0 0x7ff7a0064cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:25.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.750+0000 7ff7a50a5700 1 -- 172.21.15.17:0/2933258967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7a00b2710 msgr2=0x7ff7a00651b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:25.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.750+0000 7ff7a50a5700 1 --2- 172.21.15.17:0/2933258967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7a00b2710 0x7ff7a00651b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:25.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.750+0000 7ff7a50a5700 1 -- 172.21.15.17:0/2933258967 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff79800b180 con 0x7ff7a00a8c50 2022-08-01T19:34:25.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.750+0000 7ff7a50a5700 1 --2- 172.21.15.17:0/2933258967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7a00a8c50 0x7ff7a00ac480 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7ff79000c000 tx=0x7ff790008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:25.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.754+0000 7ff7967fc700 1 -- 172.21.15.17:0/2933258967 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff79000fe30 con 0x7ff7a00a8c50 2022-08-01T19:34:25.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.754+0000 7ff7967fc700 1 -- 172.21.15.17:0/2933258967 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff790013610 con 0x7ff7a00a8c50 2022-08-01T19:34:25.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.754+0000 7ff7a60a7700 1 -- 172.21.15.17:0/2933258967 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a0063850 con 0x7ff7a00a8c50 2022-08-01T19:34:25.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.754+0000 7ff7967fc700 1 -- 172.21.15.17:0/2933258967 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff790013970 con 0x7ff7a00a8c50 2022-08-01T19:34:25.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.754+0000 7ff7a60a7700 1 -- 172.21.15.17:0/2933258967 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a0159c00 con 0x7ff7a00a8c50 2022-08-01T19:34:25.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.754+0000 7ff7967fc700 1 -- 172.21.15.17:0/2933258967 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff790023070 con 0x7ff7a00a8c50 2022-08-01T19:34:25.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.754+0000 7ff7a60a7700 1 -- 172.21.15.17:0/2933258967 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7a0064ac0 con 0x7ff7a00a8c50 2022-08-01T19:34:25.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.754+0000 7ff7967fc700 1 --2- 172.21.15.17:0/2933258967 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff788053b00 0x7ff788055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:25.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.754+0000 7ff7967fc700 1 -- 172.21.15.17:0/2933258967 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7ff79006c1f0 con 0x7ff7a00a8c50 2022-08-01T19:34:25.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.758+0000 7ff7a48a4700 1 --2- 172.21.15.17:0/2933258967 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff788053b00 0x7ff788055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:25.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.758+0000 7ff7a48a4700 1 --2- 172.21.15.17:0/2933258967 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff788053b00 0x7ff788055fc0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7ff79c00d000 tx=0x7ff79c007d80).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:25.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.762+0000 7ff7967fc700 1 -- 172.21.15.17:0/2933258967 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff790042fb0 con 0x7ff7a00a8c50 2022-08-01T19:34:25.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.886+0000 7ff7a60a7700 1 -- 172.21.15.17:0/2933258967 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ff7a00574e0 con 0x7ff7a00a8c50 2022-08-01T19:34:25.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.886+0000 7ff7967fc700 1 -- 172.21.15.17:0/2933258967 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v25) v1 ==== 110+0+10 (secure 0 0 0) 0x7ff7900465d0 con 0x7ff7a00a8c50 2022-08-01T19:34:25.894 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:34:25.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.890+0000 7ff7a60a7700 1 -- 172.21.15.17:0/2933258967 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff788053b00 msgr2=0x7ff788055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:25.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.890+0000 7ff7a60a7700 1 --2- 172.21.15.17:0/2933258967 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff788053b00 0x7ff788055fc0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7ff79c00d000 tx=0x7ff79c007d80).stop 2022-08-01T19:34:25.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.890+0000 7ff7a60a7700 1 -- 172.21.15.17:0/2933258967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7a00a8c50 msgr2=0x7ff7a00ac480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:25.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.890+0000 7ff7a60a7700 1 --2- 172.21.15.17:0/2933258967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7a00a8c50 0x7ff7a00ac480 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7ff79000c000 tx=0x7ff790008040).stop 2022-08-01T19:34:25.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.890+0000 7ff7a60a7700 1 -- 172.21.15.17:0/2933258967 shutdown_connections 2022-08-01T19:34:25.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.890+0000 7ff7a60a7700 1 --2- 172.21.15.17:0/2933258967 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff788053b00 0x7ff788055fc0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.890+0000 7ff7a60a7700 1 --2- 172.21.15.17:0/2933258967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7a00b2710 0x7ff7a00651b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.890+0000 7ff7a60a7700 1 --2- 172.21.15.17:0/2933258967 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff7a00b01a0 0x7ff7a0064cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.890+0000 7ff7a60a7700 1 --2- 172.21.15.17:0/2933258967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7a00a8c50 0x7ff7a00ac480 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.890+0000 7ff7a60a7700 1 -- 172.21.15.17:0/2933258967 >> 172.21.15.17:0/2933258967 conn(0x7ff7a0060310 msgr2=0x7ff7a0148850 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:25.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.890+0000 7ff7a60a7700 1 -- 172.21.15.17:0/2933258967 shutdown_connections 2022-08-01T19:34:25.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:25.890+0000 7ff7a60a7700 1 -- 172.21.15.17:0/2933258967 wait complete. 2022-08-01T19:34:25.906 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:34:26.018 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.014+0000 7f409d202700 1 Processor -- start 2022-08-01T19:34:26.018 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.014+0000 7f409d202700 1 -- start start 2022-08-01T19:34:26.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.014+0000 7f409d202700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f40980b1280 0x7f409813f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:26.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.014+0000 7f409d202700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f409813fa10 con 0x7f409805d240 2022-08-01T19:34:26.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.014+0000 7f409d202700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f409805cd40 con 0x7f409805c960 2022-08-01T19:34:26.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.014+0000 7f409d202700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40980ad6d0 con 0x7f40980b1280 2022-08-01T19:34:26.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.014+0000 7f40977fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f409805d240 0x7f40980aee70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57320/0 (socket says 172.21.15.17:57320) 2022-08-01T19:34:26.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.014+0000 7f40977fe700 1 -- 172.21.15.17:0/1505663005 learned_addr learned my addr 172.21.15.17:0/1505663005 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:26.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.014+0000 7f409ca01700 1 --2- 172.21.15.17:0/1505663005 >> v2:172.21.15.77:3300/0 conn(0x7f40980b1280 0x7f409813f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:26.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.014+0000 7f4096ffd700 1 -- 172.21.15.17:0/1505663005 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3566562483 0 0) 0x7f4080001110 con 0x7f409805d240 2022-08-01T19:34:26.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.014+0000 7f4096ffd700 1 -- 172.21.15.17:0/1505663005 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4078003260 con 0x7f409805d240 2022-08-01T19:34:26.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.014+0000 7f409ca01700 1 -- 172.21.15.17:0/1505663005 >> v1:172.21.15.77:6789/0 conn(0x7f409805c960 legacy=0x7f40980acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:26.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.014+0000 7f4096ffd700 1 -- 172.21.15.17:0/1505663005 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 937628026 0 0) 0x7f409805cd40 con 0x7f409805c960 2022-08-01T19:34:26.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.014+0000 7f409ca01700 1 -- 172.21.15.17:0/1505663005 >> v1:172.21.15.17:6789/0 conn(0x7f409805d240 legacy=0x7f40980aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:26.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.014+0000 7f409ca01700 1 -- 172.21.15.17:0/1505663005 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40980b16b0 con 0x7f40980b1280 2022-08-01T19:34:26.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.014+0000 7f409ca01700 1 --2- 172.21.15.17:0/1505663005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40980b1280 0x7f409813f520 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f40900032f0 tx=0x7f409000a370).ready entity=mon.1 client_cookie=4a0068339971cbe4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:26.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.014+0000 7f4096ffd700 1 -- 172.21.15.17:0/1505663005 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f409000cba0 con 0x7f40980b1280 2022-08-01T19:34:26.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.014+0000 7f4096ffd700 1 -- 172.21.15.17:0/1505663005 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409000cd00 con 0x7f40980b1280 2022-08-01T19:34:26.026 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.014+0000 7f4096ffd700 1 -- 172.21.15.17:0/1505663005 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f409000f660 con 0x7f40980b1280 2022-08-01T19:34:26.026 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.014+0000 7f409d202700 1 -- 172.21.15.17:0/1505663005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40980b1280 msgr2=0x7f409813f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:26.026 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.014+0000 7f409d202700 1 --2- 172.21.15.17:0/1505663005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40980b1280 0x7f409813f520 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f40900032f0 tx=0x7f409000a370).stop 2022-08-01T19:34:26.027 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.014+0000 7f409d202700 1 -- 172.21.15.17:0/1505663005 shutdown_connections 2022-08-01T19:34:26.027 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.014+0000 7f409d202700 1 --2- 172.21.15.17:0/1505663005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40980b1280 0x7f409813f520 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:26.027 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.014+0000 7f409d202700 1 -- 172.21.15.17:0/1505663005 >> 172.21.15.17:0/1505663005 conn(0x7f4098060310 msgr2=0x7f40980b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:26.027 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.014+0000 7f409d202700 1 -- 172.21.15.17:0/1505663005 shutdown_connections 2022-08-01T19:34:26.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.014+0000 7f409d202700 1 -- 172.21.15.17:0/1505663005 wait complete. 2022-08-01T19:34:26.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.018+0000 7f409d202700 1 Processor -- start 2022-08-01T19:34:26.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.018+0000 7f409d202700 1 -- start start 2022-08-01T19:34:26.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.018+0000 7f409d202700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f409805c960 0x7f4098142d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:26.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.018+0000 7f409d202700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f409805d240 0x7f4098140be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:26.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.018+0000 7f409d202700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40980b1280 0x7f40980ab5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:26.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.018+0000 7f409d202700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4098142690 con 0x7f40980b1280 2022-08-01T19:34:26.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.018+0000 7f409d202700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40980aa6a0 con 0x7f409805d240 2022-08-01T19:34:26.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.018+0000 7f409d202700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40981428c0 con 0x7f409805c960 2022-08-01T19:34:26.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.018+0000 7f409ca01700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40980b1280 0x7f40980ab5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:26.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.018+0000 7f409ca01700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40980b1280 0x7f40980ab5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36900/0 (socket says 172.21.15.17:36900) 2022-08-01T19:34:26.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.018+0000 7f409ca01700 1 -- 172.21.15.17:0/3074536058 learned_addr learned my addr 172.21.15.17:0/3074536058 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:26.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.018+0000 7f40977fe700 1 --2- 172.21.15.17:0/3074536058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f409805d240 0x7f4098140be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:26.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.018+0000 7f4097fff700 1 --2- 172.21.15.17:0/3074536058 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f409805c960 0x7f4098142d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:26.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.018+0000 7f409ca01700 1 -- 172.21.15.17:0/3074536058 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f409805c960 msgr2=0x7f4098142d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:26.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.018+0000 7f409ca01700 1 --2- 172.21.15.17:0/3074536058 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f409805c960 0x7f4098142d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:26.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.018+0000 7f409ca01700 1 -- 172.21.15.17:0/3074536058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f409805d240 msgr2=0x7f4098140be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:26.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.018+0000 7f409ca01700 1 --2- 172.21.15.17:0/3074536058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f409805d240 0x7f4098140be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:26.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.018+0000 7f409ca01700 1 -- 172.21.15.17:0/3074536058 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409000a610 con 0x7f40980b1280 2022-08-01T19:34:26.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.018+0000 7f409ca01700 1 --2- 172.21.15.17:0/3074536058 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40980b1280 0x7f40980ab5f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f40900032f0 tx=0x7f4090018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:26.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.018+0000 7f40957fa700 1 -- 172.21.15.17:0/3074536058 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f409000ce70 con 0x7f40980b1280 2022-08-01T19:34:26.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.018+0000 7f409d202700 1 -- 172.21.15.17:0/3074536058 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40980aa940 con 0x7f40980b1280 2022-08-01T19:34:26.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.018+0000 7f40957fa700 1 -- 172.21.15.17:0/3074536058 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4090005a70 con 0x7f40980b1280 2022-08-01T19:34:26.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.018+0000 7f40957fa700 1 -- 172.21.15.17:0/3074536058 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4090005dd0 con 0x7f40980b1280 2022-08-01T19:34:26.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.018+0000 7f409d202700 1 -- 172.21.15.17:0/3074536058 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40981432f0 con 0x7f40980b1280 2022-08-01T19:34:26.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.022+0000 7f409d202700 1 -- 172.21.15.17:0/3074536058 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4098142a80 con 0x7f40980b1280 2022-08-01T19:34:26.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.022+0000 7f40957fa700 1 -- 172.21.15.17:0/3074536058 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4090029070 con 0x7f40980b1280 2022-08-01T19:34:26.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.022+0000 7f40957fa700 1 --2- 172.21.15.17:0/3074536058 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f407c0539e0 0x7f407c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:26.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.022+0000 7f40957fa700 1 -- 172.21.15.17:0/3074536058 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f409006d170 con 0x7f40980b1280 2022-08-01T19:34:26.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.022+0000 7f4097fff700 1 --2- 172.21.15.17:0/3074536058 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f407c0539e0 0x7f407c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:26.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.026+0000 7f4097fff700 1 --2- 172.21.15.17:0/3074536058 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f407c0539e0 0x7f407c055ea0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f408800d000 tx=0x7f4088007a00).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:26.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.030+0000 7f40957fa700 1 -- 172.21.15.17:0/3074536058 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4090043da0 con 0x7f40980b1280 2022-08-01T19:34:26.163 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.158+0000 7f409d202700 1 -- 172.21.15.17:0/3074536058 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f40980b1920 con 0x7f40980b1280 2022-08-01T19:34:26.163 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.158+0000 7f40957fa700 1 -- 172.21.15.17:0/3074536058 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f40900473c0 con 0x7f40980b1280 2022-08-01T19:34:26.164 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:34:26.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.158+0000 7f409d202700 1 -- 172.21.15.17:0/3074536058 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f407c0539e0 msgr2=0x7f407c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:26.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.158+0000 7f409d202700 1 --2- 172.21.15.17:0/3074536058 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f407c0539e0 0x7f407c055ea0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f408800d000 tx=0x7f4088007a00).stop 2022-08-01T19:34:26.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.158+0000 7f409d202700 1 -- 172.21.15.17:0/3074536058 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40980b1280 msgr2=0x7f40980ab5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:26.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.158+0000 7f409d202700 1 --2- 172.21.15.17:0/3074536058 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40980b1280 0x7f40980ab5f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f40900032f0 tx=0x7f4090018000).stop 2022-08-01T19:34:26.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.158+0000 7f409d202700 1 -- 172.21.15.17:0/3074536058 shutdown_connections 2022-08-01T19:34:26.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.158+0000 7f409d202700 1 --2- 172.21.15.17:0/3074536058 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f407c0539e0 0x7f407c055ea0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:26.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.158+0000 7f409d202700 1 --2- 172.21.15.17:0/3074536058 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40980b1280 0x7f40980ab5f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:26.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.158+0000 7f409d202700 1 --2- 172.21.15.17:0/3074536058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f409805d240 0x7f4098140be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:26.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.158+0000 7f409d202700 1 --2- 172.21.15.17:0/3074536058 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f409805c960 0x7f4098142d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:26.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.158+0000 7f409d202700 1 -- 172.21.15.17:0/3074536058 >> 172.21.15.17:0/3074536058 conn(0x7f4098060310 msgr2=0x7f409813b960 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:26.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.158+0000 7f409d202700 1 -- 172.21.15.17:0/3074536058 shutdown_connections 2022-08-01T19:34:26.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.158+0000 7f409d202700 1 -- 172.21.15.17:0/3074536058 wait complete. 2022-08-01T19:34:26.177 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T19:34:26.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.278+0000 7fdd79b16700 1 Processor -- start 2022-08-01T19:34:26.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.278+0000 7fdd79b16700 1 -- start start 2022-08-01T19:34:26.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.278+0000 7fdd79b16700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdd740ad750 0x7fdd74061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:26.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.278+0000 7fdd79b16700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd740aee80 con 0x7fdd740614b0 2022-08-01T19:34:26.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.278+0000 7fdd79b16700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd740b70b0 con 0x7fdd741405d0 2022-08-01T19:34:26.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.278+0000 7fdd79b16700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd740af0b0 con 0x7fdd740ad750 2022-08-01T19:34:26.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.278+0000 7fdd78b14700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fdd740614b0 0x7fdd740b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57330/0 (socket says 172.21.15.17:57330) 2022-08-01T19:34:26.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.278+0000 7fdd78b14700 1 -- 172.21.15.17:0/1518563248 learned_addr learned my addr 172.21.15.17:0/1518563248 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:26.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.278+0000 7fdd73fff700 1 --2- 172.21.15.17:0/1518563248 >> v2:172.21.15.17:3300/0 conn(0x7fdd740ad750 0x7fdd74061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:26.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.278+0000 7fdd737fe700 1 -- 172.21.15.17:0/1518563248 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 57408450 0 0) 0x7fdd740aee80 con 0x7fdd740614b0 2022-08-01T19:34:26.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.278+0000 7fdd737fe700 1 -- 172.21.15.17:0/1518563248 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd54003260 con 0x7fdd740614b0 2022-08-01T19:34:26.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.278+0000 7fdd73fff700 1 -- 172.21.15.17:0/1518563248 >> v1:172.21.15.77:6789/0 conn(0x7fdd741405d0 legacy=0x7fdd741466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:26.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.278+0000 7fdd73fff700 1 -- 172.21.15.17:0/1518563248 >> v1:172.21.15.17:6789/0 conn(0x7fdd740614b0 legacy=0x7fdd740b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:26.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.278+0000 7fdd73fff700 1 -- 172.21.15.17:0/1518563248 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd740af350 con 0x7fdd740ad750 2022-08-01T19:34:26.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.278+0000 7fdd73fff700 1 --2- 172.21.15.17:0/1518563248 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd740ad750 0x7fdd74061890 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fdd5c002f70 tx=0x7fdd5c006df0).ready entity=mon.0 client_cookie=625a3b731a82b55f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:26.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.278+0000 7fdd737fe700 1 -- 172.21.15.17:0/1518563248 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd5c00d610 con 0x7fdd740ad750 2022-08-01T19:34:26.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.278+0000 7fdd737fe700 1 -- 172.21.15.17:0/1518563248 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5c000e90 con 0x7fdd740ad750 2022-08-01T19:34:26.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.278+0000 7fdd737fe700 1 -- 172.21.15.17:0/1518563248 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd5c00d960 con 0x7fdd740ad750 2022-08-01T19:34:26.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.282+0000 7fdd79b16700 1 -- 172.21.15.17:0/1518563248 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd740ad750 msgr2=0x7fdd74061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:26.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.282+0000 7fdd79b16700 1 --2- 172.21.15.17:0/1518563248 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd740ad750 0x7fdd74061890 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fdd5c002f70 tx=0x7fdd5c006df0).stop 2022-08-01T19:34:26.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.282+0000 7fdd79b16700 1 -- 172.21.15.17:0/1518563248 shutdown_connections 2022-08-01T19:34:26.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.282+0000 7fdd79b16700 1 --2- 172.21.15.17:0/1518563248 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd740ad750 0x7fdd74061890 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:26.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.282+0000 7fdd79b16700 1 -- 172.21.15.17:0/1518563248 >> 172.21.15.17:0/1518563248 conn(0x7fdd74066f70 msgr2=0x7fdd74067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:26.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.282+0000 7fdd79b16700 1 -- 172.21.15.17:0/1518563248 shutdown_connections 2022-08-01T19:34:26.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.282+0000 7fdd79b16700 1 -- 172.21.15.17:0/1518563248 wait complete. 2022-08-01T19:34:26.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.282+0000 7fdd79b16700 1 Processor -- start 2022-08-01T19:34:26.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.282+0000 7fdd79b16700 1 -- start start 2022-08-01T19:34:26.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.282+0000 7fdd79b16700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdd740614b0 0x7fdd74155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:26.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.282+0000 7fdd79b16700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd740ad750 0x7fdd740b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:26.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.282+0000 7fdd79b16700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd741405d0 0x7fdd740b3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:26.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.282+0000 7fdd79b16700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7414ab00 con 0x7fdd741405d0 2022-08-01T19:34:26.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.282+0000 7fdd79b16700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd740b0440 con 0x7fdd740ad750 2022-08-01T19:34:26.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.282+0000 7fdd79b16700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd741418a0 con 0x7fdd740614b0 2022-08-01T19:34:26.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.282+0000 7fdd79315700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd741405d0 0x7fdd740b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:26.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.282+0000 7fdd79315700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd741405d0 0x7fdd740b3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36914/0 (socket says 172.21.15.17:36914) 2022-08-01T19:34:26.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.282+0000 7fdd79315700 1 -- 172.21.15.17:0/2076439755 learned_addr learned my addr 172.21.15.17:0/2076439755 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:26.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.282+0000 7fdd73fff700 1 --2- 172.21.15.17:0/2076439755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd740ad750 0x7fdd740b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:26.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.282+0000 7fdd78b14700 1 --2- 172.21.15.17:0/2076439755 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdd740614b0 0x7fdd74155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:26.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.282+0000 7fdd79315700 1 -- 172.21.15.17:0/2076439755 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdd740614b0 msgr2=0x7fdd74155d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:26.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.282+0000 7fdd79315700 1 --2- 172.21.15.17:0/2076439755 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdd740614b0 0x7fdd74155d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:26.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.282+0000 7fdd79315700 1 -- 172.21.15.17:0/2076439755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd740ad750 msgr2=0x7fdd740b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:26.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.282+0000 7fdd79315700 1 --2- 172.21.15.17:0/2076439755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd740ad750 0x7fdd740b2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:26.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.282+0000 7fdd79315700 1 -- 172.21.15.17:0/2076439755 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5c00b180 con 0x7fdd741405d0 2022-08-01T19:34:26.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.286+0000 7fdd79315700 1 --2- 172.21.15.17:0/2076439755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd741405d0 0x7fdd740b3350 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fdd740af970 tx=0x7fdd64007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:26.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.286+0000 7fdd71ffb700 1 -- 172.21.15.17:0/2076439755 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd64007030 con 0x7fdd741405d0 2022-08-01T19:34:26.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.286+0000 7fdd71ffb700 1 -- 172.21.15.17:0/2076439755 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd64007190 con 0x7fdd741405d0 2022-08-01T19:34:26.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.286+0000 7fdd79b16700 1 -- 172.21.15.17:0/2076439755 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd74141ad0 con 0x7fdd741405d0 2022-08-01T19:34:26.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.286+0000 7fdd71ffb700 1 -- 172.21.15.17:0/2076439755 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd64006510 con 0x7fdd741405d0 2022-08-01T19:34:26.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.286+0000 7fdd79b16700 1 -- 172.21.15.17:0/2076439755 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd74156270 con 0x7fdd741405d0 2022-08-01T19:34:26.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.286+0000 7fdd79b16700 1 -- 172.21.15.17:0/2076439755 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd74141e50 con 0x7fdd741405d0 2022-08-01T19:34:26.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.286+0000 7fdd71ffb700 1 -- 172.21.15.17:0/2076439755 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdd64010030 con 0x7fdd741405d0 2022-08-01T19:34:26.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.286+0000 7fdd71ffb700 1 --2- 172.21.15.17:0/2076439755 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdd58053950 0x7fdd58055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:26.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.290+0000 7fdd71ffb700 1 -- 172.21.15.17:0/2076439755 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fdd6406bb60 con 0x7fdd741405d0 2022-08-01T19:34:26.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.290+0000 7fdd78b14700 1 --2- 172.21.15.17:0/2076439755 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdd58053950 0x7fdd58055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:26.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.290+0000 7fdd78b14700 1 --2- 172.21.15.17:0/2076439755 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdd58053950 0x7fdd58055e10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fdd6c00d000 tx=0x7fdd6c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:26.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.294+0000 7fdd71ffb700 1 -- 172.21.15.17:0/2076439755 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdd64042920 con 0x7fdd741405d0 2022-08-01T19:34:26.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.418+0000 7fdd79b16700 1 -- 172.21.15.17:0/2076439755 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdd740574e0 con 0x7fdd741405d0 2022-08-01T19:34:26.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.422+0000 7fdd71ffb700 1 -- 172.21.15.17:0/2076439755 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+10 (secure 0 0 0) 0x7fdd64045f40 con 0x7fdd741405d0 2022-08-01T19:34:26.427 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:34:26.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.422+0000 7fdd79b16700 1 -- 172.21.15.17:0/2076439755 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdd58053950 msgr2=0x7fdd58055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:26.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.422+0000 7fdd79b16700 1 --2- 172.21.15.17:0/2076439755 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdd58053950 0x7fdd58055e10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fdd6c00d000 tx=0x7fdd6c009040).stop 2022-08-01T19:34:26.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.422+0000 7fdd79b16700 1 -- 172.21.15.17:0/2076439755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd741405d0 msgr2=0x7fdd740b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:26.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.422+0000 7fdd79b16700 1 --2- 172.21.15.17:0/2076439755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd741405d0 0x7fdd740b3350 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fdd740af970 tx=0x7fdd64007c20).stop 2022-08-01T19:34:26.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.422+0000 7fdd79b16700 1 -- 172.21.15.17:0/2076439755 shutdown_connections 2022-08-01T19:34:26.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.422+0000 7fdd79b16700 1 --2- 172.21.15.17:0/2076439755 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdd58053950 0x7fdd58055e10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:26.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.422+0000 7fdd79b16700 1 --2- 172.21.15.17:0/2076439755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd741405d0 0x7fdd740b3350 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:26.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.422+0000 7fdd79b16700 1 --2- 172.21.15.17:0/2076439755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd740ad750 0x7fdd740b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:26.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.422+0000 7fdd79b16700 1 --2- 172.21.15.17:0/2076439755 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdd740614b0 0x7fdd74155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:26.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.422+0000 7fdd79b16700 1 -- 172.21.15.17:0/2076439755 >> 172.21.15.17:0/2076439755 conn(0x7fdd74066f70 msgr2=0x7fdd74146380 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:26.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.422+0000 7fdd79b16700 1 -- 172.21.15.17:0/2076439755 shutdown_connections 2022-08-01T19:34:26.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.422+0000 7fdd79b16700 1 -- 172.21.15.17:0/2076439755 wait complete. 2022-08-01T19:34:26.439 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T19:34:26.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.538+0000 7f803c95d700 1 Processor -- start 2022-08-01T19:34:26.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.538+0000 7f803c95d700 1 -- start start 2022-08-01T19:34:26.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.538+0000 7f803c95d700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f80380614b0 0x7f8038061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:26.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.538+0000 7f803c95d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f80380ad750 0x7f80380b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:26.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.538+0000 7f803c95d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f80381405d0 0x7f8038140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:26.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.538+0000 7f803c95d700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80380aee80 con 0x7f80381405d0 2022-08-01T19:34:26.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.538+0000 7f803c95d700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80380b70b0 con 0x7f80380614b0 2022-08-01T19:34:26.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.538+0000 7f803c95d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80380af0b0 con 0x7f80380ad750 2022-08-01T19:34:26.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.538+0000 7f8037fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f80381405d0 0x7f8038140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:26.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.538+0000 7f8037fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f80381405d0 0x7f8038140a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36922/0 (socket says 172.21.15.17:36922) 2022-08-01T19:34:26.547 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.538+0000 7f8037fff700 1 -- 172.21.15.17:0/1178347764 learned_addr learned my addr 172.21.15.17:0/1178347764 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:26.547 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.538+0000 7f8036ffd700 1 --2- 172.21.15.17:0/1178347764 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f80380ad750 0x7f80380b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:26.547 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.538+0000 7f80377fe700 1 --2- 172.21.15.17:0/1178347764 >> v2:172.21.15.77:3300/0 conn(0x7f80380614b0 0x7f8038061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:26.550 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.538+0000 7f8037fff700 1 -- 172.21.15.17:0/1178347764 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f80380ad750 msgr2=0x7f80380b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:26.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.538+0000 7f8037fff700 1 --2- 172.21.15.17:0/1178347764 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f80380ad750 0x7f80380b9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:26.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.538+0000 7f8037fff700 1 -- 172.21.15.17:0/1178347764 >> v2:172.21.15.77:3300/0 conn(0x7f80380614b0 msgr2=0x7f8038061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:26.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.538+0000 7f8037fff700 1 --2- 172.21.15.17:0/1178347764 >> v2:172.21.15.77:3300/0 conn(0x7f80380614b0 0x7f8038061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:26.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.538+0000 7f8037fff700 1 -- 172.21.15.17:0/1178347764 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80380af350 con 0x7f80381405d0 2022-08-01T19:34:26.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.538+0000 7f8037fff700 1 --2- 172.21.15.17:0/1178347764 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80381405d0 0x7f8038140a60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f80300032f0 tx=0x7f8030009290).ready entity=mon.0 client_cookie=167225b2664995c6 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:26.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.542+0000 7f80367fc700 1 -- 172.21.15.17:0/1178347764 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f803000b680 con 0x7f80381405d0 2022-08-01T19:34:26.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.542+0000 7f80367fc700 1 -- 172.21.15.17:0/1178347764 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803000b7e0 con 0x7f80381405d0 2022-08-01T19:34:26.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.542+0000 7f80367fc700 1 -- 172.21.15.17:0/1178347764 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f803000b950 con 0x7f80381405d0 2022-08-01T19:34:26.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.542+0000 7f803c95d700 1 -- 172.21.15.17:0/1178347764 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80381405d0 msgr2=0x7f8038140a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:26.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.542+0000 7f803c95d700 1 --2- 172.21.15.17:0/1178347764 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80381405d0 0x7f8038140a60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f80300032f0 tx=0x7f8030009290).stop 2022-08-01T19:34:26.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.542+0000 7f803c95d700 1 -- 172.21.15.17:0/1178347764 shutdown_connections 2022-08-01T19:34:26.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.542+0000 7f803c95d700 1 --2- 172.21.15.17:0/1178347764 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80381405d0 0x7f8038140a60 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:26.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.542+0000 7f803c95d700 1 --2- 172.21.15.17:0/1178347764 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f80380ad750 0x7f80380b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:26.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.542+0000 7f803c95d700 1 --2- 172.21.15.17:0/1178347764 >> v2:172.21.15.77:3300/0 conn(0x7f80380614b0 0x7f8038061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:26.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.542+0000 7f803c95d700 1 -- 172.21.15.17:0/1178347764 >> 172.21.15.17:0/1178347764 conn(0x7f8038066f70 msgr2=0x7f8038067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:26.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.542+0000 7f803c95d700 1 -- 172.21.15.17:0/1178347764 shutdown_connections 2022-08-01T19:34:26.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.542+0000 7f803c95d700 1 -- 172.21.15.17:0/1178347764 wait complete. 2022-08-01T19:34:26.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.542+0000 7f803c95d700 1 Processor -- start 2022-08-01T19:34:26.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.542+0000 7f803c95d700 1 -- start start 2022-08-01T19:34:26.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.542+0000 7f803c95d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80380614b0 0x7f80380b0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:26.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.542+0000 7f803c95d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f80380ad750 0x7f80380ac500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:26.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.542+0000 7f803c95d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80381405d0 0x7f80380ac9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:26.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.542+0000 7f803c95d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803809eae0 con 0x7f80381405d0 2022-08-01T19:34:26.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.542+0000 7f803c95d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80380acee0 con 0x7f80380614b0 2022-08-01T19:34:26.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.542+0000 7f803c95d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8038152db0 con 0x7f80380ad750 2022-08-01T19:34:26.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.542+0000 7f8037fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80381405d0 0x7f80380ac9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:26.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.542+0000 7f8037fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80381405d0 0x7f80380ac9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36924/0 (socket says 172.21.15.17:36924) 2022-08-01T19:34:26.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.542+0000 7f8037fff700 1 -- 172.21.15.17:0/2403918235 learned_addr learned my addr 172.21.15.17:0/2403918235 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:26.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.542+0000 7f8036ffd700 1 --2- 172.21.15.17:0/2403918235 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f80380ad750 0x7f80380ac500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:26.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.542+0000 7f80377fe700 1 --2- 172.21.15.17:0/2403918235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80380614b0 0x7f80380b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:26.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.542+0000 7f8037fff700 1 -- 172.21.15.17:0/2403918235 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f80380ad750 msgr2=0x7f80380ac500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:26.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.542+0000 7f8037fff700 1 --2- 172.21.15.17:0/2403918235 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f80380ad750 0x7f80380ac500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:26.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.542+0000 7f8037fff700 1 -- 172.21.15.17:0/2403918235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80380614b0 msgr2=0x7f80380b0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:26.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.542+0000 7f8037fff700 1 --2- 172.21.15.17:0/2403918235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80380614b0 0x7f80380b0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:26.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.542+0000 7f8037fff700 1 -- 172.21.15.17:0/2403918235 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8030009530 con 0x7f80381405d0 2022-08-01T19:34:26.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.542+0000 7f8036ffd700 1 --2- 172.21.15.17:0/2403918235 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f80380ad750 0x7f80380ac500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:34:26.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.546+0000 7f8037fff700 1 --2- 172.21.15.17:0/2403918235 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80381405d0 0x7f80380ac9f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f8038143000 tx=0x7f8030017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:26.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.546+0000 7f8034ff9700 1 -- 172.21.15.17:0/2403918235 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f803001f070 con 0x7f80381405d0 2022-08-01T19:34:26.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.546+0000 7f8034ff9700 1 -- 172.21.15.17:0/2403918235 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8030011e70 con 0x7f80381405d0 2022-08-01T19:34:26.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.546+0000 7f8034ff9700 1 -- 172.21.15.17:0/2403918235 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f803000b680 con 0x7f80381405d0 2022-08-01T19:34:26.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.546+0000 7f803c95d700 1 -- 172.21.15.17:0/2403918235 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8038153020 con 0x7f80381405d0 2022-08-01T19:34:26.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.546+0000 7f803c95d700 1 -- 172.21.15.17:0/2403918235 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80380b11d0 con 0x7f80381405d0 2022-08-01T19:34:26.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.546+0000 7f803c95d700 1 -- 172.21.15.17:0/2403918235 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80380af5f0 con 0x7f80381405d0 2022-08-01T19:34:26.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.546+0000 7f8034ff9700 1 -- 172.21.15.17:0/2403918235 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f803000f070 con 0x7f80381405d0 2022-08-01T19:34:26.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.546+0000 7f8034ff9700 1 --2- 172.21.15.17:0/2403918235 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8018053a10 0x7f8018055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:26.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.546+0000 7f8034ff9700 1 -- 172.21.15.17:0/2403918235 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f803006d540 con 0x7f80381405d0 2022-08-01T19:34:26.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.546+0000 7f80377fe700 1 --2- 172.21.15.17:0/2403918235 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8018053a10 0x7f8018055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:26.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.550+0000 7f80377fe700 1 --2- 172.21.15.17:0/2403918235 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8018053a10 0x7f8018055ed0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f802c00b000 tx=0x7f802c008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:26.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.554+0000 7f8034ff9700 1 -- 172.21.15.17:0/2403918235 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f80300443b0 con 0x7f80381405d0 2022-08-01T19:34:26.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.678+0000 7f803c95d700 1 -- 172.21.15.17:0/2403918235 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f80380ad080 con 0x7f80381405d0 2022-08-01T19:34:26.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.678+0000 7f8034ff9700 1 -- 172.21.15.17:0/2403918235 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+10 (secure 0 0 0) 0x7f80300479d0 con 0x7f80381405d0 2022-08-01T19:34:26.685 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:34:26.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.682+0000 7f803c95d700 1 -- 172.21.15.17:0/2403918235 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8018053a10 msgr2=0x7f8018055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:26.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.682+0000 7f803c95d700 1 --2- 172.21.15.17:0/2403918235 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8018053a10 0x7f8018055ed0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f802c00b000 tx=0x7f802c008040).stop 2022-08-01T19:34:26.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.682+0000 7f803c95d700 1 -- 172.21.15.17:0/2403918235 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80381405d0 msgr2=0x7f80380ac9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:26.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.682+0000 7f803c95d700 1 --2- 172.21.15.17:0/2403918235 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80381405d0 0x7f80380ac9f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f8038143000 tx=0x7f8030017000).stop 2022-08-01T19:34:26.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.682+0000 7f803c95d700 1 -- 172.21.15.17:0/2403918235 shutdown_connections 2022-08-01T19:34:26.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.682+0000 7f803c95d700 1 --2- 172.21.15.17:0/2403918235 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8018053a10 0x7f8018055ed0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:26.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.682+0000 7f803c95d700 1 --2- 172.21.15.17:0/2403918235 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80381405d0 0x7f80380ac9f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:26.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.682+0000 7f803c95d700 1 --2- 172.21.15.17:0/2403918235 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f80380ad750 0x7f80380ac500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:26.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.682+0000 7f803c95d700 1 --2- 172.21.15.17:0/2403918235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80380614b0 0x7f80380b0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:26.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.682+0000 7f803c95d700 1 -- 172.21.15.17:0/2403918235 >> 172.21.15.17:0/2403918235 conn(0x7f8038066f70 msgr2=0x7f8038143920 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:26.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.682+0000 7f803c95d700 1 -- 172.21.15.17:0/2403918235 shutdown_connections 2022-08-01T19:34:26.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.682+0000 7f803c95d700 1 -- 172.21.15.17:0/2403918235 wait complete. 2022-08-01T19:34:26.698 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T19:34:26.699 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:34:26.802 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.794+0000 7f9ea58bc700 1 Processor -- start 2022-08-01T19:34:26.802 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.794+0000 7f9ea58bc700 1 -- start start 2022-08-01T19:34:26.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.798+0000 7f9ea58bc700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9ea00ad670 0x7f9ea00b0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:26.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.798+0000 7f9ea58bc700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ea00af120 0x7f9ea00b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:26.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.798+0000 7f9ea58bc700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ea00b0c40 con 0x7f9ea005f840 2022-08-01T19:34:26.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.798+0000 7f9ea58bc700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea00b0ee0 con 0x7f9ea00ad670 2022-08-01T19:34:26.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.798+0000 7f9ea58bc700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea00ae9e0 con 0x7f9ea00af120 2022-08-01T19:34:26.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.798+0000 7f9ea48ba700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9ea00ad670 0x7f9ea00b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:26.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.798+0000 7f9ea48ba700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9ea00ad670 0x7f9ea00b0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36932/0 (socket says 172.21.15.17:36932) 2022-08-01T19:34:26.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.798+0000 7f9ea48ba700 1 -- 172.21.15.17:0/2464753179 learned_addr learned my addr 172.21.15.17:0/2464753179 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:26.807 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.798+0000 7f9ea48ba700 1 -- 172.21.15.17:0/2464753179 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ea00af120 msgr2=0x7f9ea00b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:26.807 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.798+0000 7f9e9ffff700 1 --2- 172.21.15.17:0/2464753179 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ea00af120 0x7f9ea00b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:26.807 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.798+0000 7f9ea48ba700 1 --2- 172.21.15.17:0/2464753179 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ea00af120 0x7f9ea00b12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:26.807 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.798+0000 7f9ea48ba700 1 -- 172.21.15.17:0/2464753179 >> v1:172.21.15.17:6789/0 conn(0x7f9ea005f840 legacy=0x7f9ea005fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:26.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.798+0000 7f9ea48ba700 1 -- 172.21.15.17:0/2464753179 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea00aec80 con 0x7f9ea00ad670 2022-08-01T19:34:26.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.798+0000 7f9e9ffff700 1 --2- 172.21.15.17:0/2464753179 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ea00af120 0x7f9ea00b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:34:26.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.798+0000 7f9ea48ba700 1 --2- 172.21.15.17:0/2464753179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea00ad670 0x7f9ea00b0570 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f9e90002f70 tx=0x7f9e90006eb0).ready entity=mon.0 client_cookie=955f97df985ba9de server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:26.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.798+0000 7f9e9f7fe700 1 -- 172.21.15.17:0/2464753179 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e9000d5c0 con 0x7f9ea00ad670 2022-08-01T19:34:26.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.798+0000 7f9e9f7fe700 1 -- 172.21.15.17:0/2464753179 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e90000e90 con 0x7f9ea00ad670 2022-08-01T19:34:26.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.798+0000 7f9e9f7fe700 1 -- 172.21.15.17:0/2464753179 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e9000d950 con 0x7f9ea00ad670 2022-08-01T19:34:26.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.798+0000 7f9ea58bc700 1 -- 172.21.15.17:0/2464753179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea00ad670 msgr2=0x7f9ea00b0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:26.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.798+0000 7f9ea58bc700 1 --2- 172.21.15.17:0/2464753179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea00ad670 0x7f9ea00b0570 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f9e90002f70 tx=0x7f9e90006eb0).stop 2022-08-01T19:34:26.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.798+0000 7f9ea58bc700 1 -- 172.21.15.17:0/2464753179 shutdown_connections 2022-08-01T19:34:26.810 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.798+0000 7f9ea58bc700 1 --2- 172.21.15.17:0/2464753179 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ea00af120 0x7f9ea00b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:26.810 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.798+0000 7f9ea58bc700 1 --2- 172.21.15.17:0/2464753179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea00ad670 0x7f9ea00b0570 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:26.810 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.798+0000 7f9ea58bc700 1 -- 172.21.15.17:0/2464753179 >> 172.21.15.17:0/2464753179 conn(0x7f9ea0061360 msgr2=0x7f9ea0062f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:26.810 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.798+0000 7f9ea58bc700 1 -- 172.21.15.17:0/2464753179 shutdown_connections 2022-08-01T19:34:26.811 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.798+0000 7f9ea58bc700 1 -- 172.21.15.17:0/2464753179 wait complete. 2022-08-01T19:34:26.814 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.802+0000 7f9ea58bc700 1 Processor -- start 2022-08-01T19:34:26.814 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.802+0000 7f9ea58bc700 1 -- start start 2022-08-01T19:34:26.814 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.802+0000 7f9ea58bc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea005f840 0x7f9ea0063f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:26.814 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.802+0000 7f9ea58bc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ea00ad670 0x7f9ea0134ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:26.815 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.802+0000 7f9ea58bc700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ea00af120 0x7f9ea01353b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:26.815 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.802+0000 7f9ea58bc700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea01358a0 con 0x7f9ea005f840 2022-08-01T19:34:26.815 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.802+0000 7f9ea58bc700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea0065920 con 0x7f9ea00ad670 2022-08-01T19:34:26.815 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.802+0000 7f9ea58bc700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea0065bc0 con 0x7f9ea00af120 2022-08-01T19:34:26.816 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.802+0000 7f9ea48ba700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea005f840 0x7f9ea0063f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:26.816 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.802+0000 7f9ea48ba700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea005f840 0x7f9ea0063f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36938/0 (socket says 172.21.15.17:36938) 2022-08-01T19:34:26.816 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.802+0000 7f9ea48ba700 1 -- 172.21.15.17:0/3613595299 learned_addr learned my addr 172.21.15.17:0/3613595299 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:26.816 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.802+0000 7f9e9ffff700 1 --2- 172.21.15.17:0/3613595299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ea00ad670 0x7f9ea0134ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:26.817 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.802+0000 7f9ea50bb700 1 --2- 172.21.15.17:0/3613595299 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ea00af120 0x7f9ea01353b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:26.817 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.802+0000 7f9ea48ba700 1 -- 172.21.15.17:0/3613595299 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ea00af120 msgr2=0x7f9ea01353b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:26.817 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.802+0000 7f9ea48ba700 1 --2- 172.21.15.17:0/3613595299 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ea00af120 0x7f9ea01353b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:26.817 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.802+0000 7f9ea48ba700 1 -- 172.21.15.17:0/3613595299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ea00ad670 msgr2=0x7f9ea0134ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:26.817 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.802+0000 7f9ea48ba700 1 --2- 172.21.15.17:0/3613595299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ea00ad670 0x7f9ea0134ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:26.818 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.802+0000 7f9ea48ba700 1 -- 172.21.15.17:0/3613595299 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9000b1e0 con 0x7f9ea005f840 2022-08-01T19:34:26.818 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.802+0000 7f9e9ffff700 1 --2- 172.21.15.17:0/3613595299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ea00ad670 0x7f9ea0134ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:34:26.818 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.802+0000 7f9ea48ba700 1 --2- 172.21.15.17:0/3613595299 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea005f840 0x7f9ea0063f80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f9e90002f70 tx=0x7f9e90015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:26.818 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.802+0000 7f9e9dffb700 1 -- 172.21.15.17:0/3613595299 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e9000de80 con 0x7f9ea005f840 2022-08-01T19:34:26.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.802+0000 7f9ea58bc700 1 -- 172.21.15.17:0/3613595299 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea0065e60 con 0x7f9ea005f840 2022-08-01T19:34:26.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.802+0000 7f9e9dffb700 1 -- 172.21.15.17:0/3613595299 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e90011b80 con 0x7f9ea005f840 2022-08-01T19:34:26.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.802+0000 7f9e9dffb700 1 -- 172.21.15.17:0/3613595299 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e9001e460 con 0x7f9ea005f840 2022-08-01T19:34:26.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.802+0000 7f9ea58bc700 1 -- 172.21.15.17:0/3613595299 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea00aac30 con 0x7f9ea005f840 2022-08-01T19:34:26.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.806+0000 7f9ea58bc700 1 -- 172.21.15.17:0/3613595299 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea0067e90 con 0x7f9ea005f840 2022-08-01T19:34:26.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.806+0000 7f9e9dffb700 1 -- 172.21.15.17:0/3613595299 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9e90025030 con 0x7f9ea005f840 2022-08-01T19:34:26.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.806+0000 7f9e9dffb700 1 --2- 172.21.15.17:0/3613595299 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9e800539e0 0x7f9e80055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:26.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.806+0000 7f9e9dffb700 1 -- 172.21.15.17:0/3613595299 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f9e9006d510 con 0x7f9ea005f840 2022-08-01T19:34:26.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.806+0000 7f9e9ffff700 1 --2- 172.21.15.17:0/3613595299 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9e800539e0 0x7f9e80055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:26.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.806+0000 7f9e9ffff700 1 --2- 172.21.15.17:0/3613595299 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9e800539e0 0x7f9e80055ea0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f9e9400d000 tx=0x7f9e94008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:26.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.814+0000 7f9e9dffb700 1 -- 172.21.15.17:0/3613595299 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9e90044140 con 0x7f9ea005f840 2022-08-01T19:34:26.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.978+0000 7f9ea58bc700 1 -- 172.21.15.17:0/3613595299 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ea00b34e0 con 0x7f9ea005f840 2022-08-01T19:34:26.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.982+0000 7f9e9dffb700 1 -- 172.21.15.17:0/3613595299 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+2486 (secure 0 0 0) 0x7f9e90023090 con 0x7f9ea005f840 2022-08-01T19:34:26.987 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:26.988 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2116883486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2116883486},{"type":"v1","addr":"172.21.15.17:6833","nonce":2116883486}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.17:6835/2974833789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2974833789},{"type":"v1","addr":"172.21.15.17:6835","nonce":2974833789}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":4}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:34:09.692662+0000","modified":"2022-08-01T19:34:11.425752+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":5,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6835/1842780453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1842780453},{"type":"v1","addr":"172.21.15.77:6835","nonce":1842780453}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2022-08-01T19:34:26.991 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.982+0000 7f9ea58bc700 1 -- 172.21.15.17:0/3613595299 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9e800539e0 msgr2=0x7f9e80055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:26.991 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.982+0000 7f9ea58bc700 1 --2- 172.21.15.17:0/3613595299 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9e800539e0 0x7f9e80055ea0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f9e9400d000 tx=0x7f9e94008040).stop 2022-08-01T19:34:26.992 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.982+0000 7f9ea58bc700 1 -- 172.21.15.17:0/3613595299 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea005f840 msgr2=0x7f9ea0063f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:26.992 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.982+0000 7f9ea58bc700 1 --2- 172.21.15.17:0/3613595299 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea005f840 0x7f9ea0063f80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f9e90002f70 tx=0x7f9e90015000).stop 2022-08-01T19:34:26.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.982+0000 7f9ea58bc700 1 -- 172.21.15.17:0/3613595299 shutdown_connections 2022-08-01T19:34:26.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.982+0000 7f9ea58bc700 1 --2- 172.21.15.17:0/3613595299 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9e800539e0 0x7f9e80055ea0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:26.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.982+0000 7f9ea58bc700 1 --2- 172.21.15.17:0/3613595299 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ea00af120 0x7f9ea01353b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:26.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.982+0000 7f9ea58bc700 1 --2- 172.21.15.17:0/3613595299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ea00ad670 0x7f9ea0134ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:26.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.982+0000 7f9ea58bc700 1 --2- 172.21.15.17:0/3613595299 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea005f840 0x7f9ea0063f80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:26.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.982+0000 7f9ea58bc700 1 -- 172.21.15.17:0/3613595299 >> 172.21.15.17:0/3613595299 conn(0x7f9ea0061360 msgr2=0x7f9ea00b4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:26.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.982+0000 7f9ea58bc700 1 -- 172.21.15.17:0/3613595299 shutdown_connections 2022-08-01T19:34:26.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:26.982+0000 7f9ea58bc700 1 -- 172.21.15.17:0/3613595299 wait complete. 2022-08-01T19:34:26.996 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 7 2022-08-01T19:34:27.301 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:34:09.692662+0000', 'modified': '2022-08-01T19:34:11.425752+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4285}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4285': {'gid': 4285, 'name': 'b', 'rank': 0, 'incarnation': 5, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.77:6835/1842780453', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6834', 'nonce': 1842780453}, {'type': 'v1', 'addr': '172.21.15.77:6835', 'nonce': 1842780453}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:34:27.302 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:34:27.302 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-08-01T19:34:27.418 INFO:teuthology.orchestra.run.smithi077.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":4285,"want_state":"up:active","state":"up:active","rank_uptime":17.712380126999999,"mdsmap_epoch":7,"osdmap_epoch":25,"osdmap_epoch_barrier":25,"uptime":24.062690798999999} 2022-08-01T19:34:27.418 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 4285, "mdsmap_epoch": 7, "osdmap_epoch": 25, "osdmap_epoch_barrier": 25, "rank_uptime": 17.712380127, "state": "up:active", "uptime": 24.062690799, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:34:27.419 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T19:34:27.419 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-08-01T19:34:27.420 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi190.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-08-01T19:34:27.420 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:34:27.421 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:34:27.430 DEBUG:teuthology.orchestra.run.smithi190:> sudo modprobe fuse 2022-08-01T19:34:27.489 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:34:27.489 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:34:27.505 INFO:teuthology.orchestra.run.smithi190.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:34:27.507 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:34:27.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:34:27.508 DEBUG:teuthology.orchestra.run.smithi190:> ls /sys/fs/fuse/connections 2022-08-01T19:34:27.515 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T19:34:27.516 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:34:27.558 DEBUG:teuthology.orchestra.run.smithi190:> sudo modprobe fuse 2022-08-01T19:34:27.576 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:34:27.577 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:34:27.587 INFO:teuthology.orchestra.run.smithi190.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:34:27.588 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:34:27.589 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:34:27.589 DEBUG:teuthology.orchestra.run.smithi190:> ls /sys/fs/fuse/connections 2022-08-01T19:34:27.646 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:2022-08-01T19:34:27.644+0000 7f79425af080 -1 init, newargv = 0x55f317544830 newargc=9ceph-fuse[22493]: starting ceph client 2022-08-01T19:34:27.646 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr: 2022-08-01T19:34:27.657 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:ceph-fuse[22493]: starting fuse 2022-08-01T19:34:28.696 DEBUG:teuthology.orchestra.run.smithi190:> sudo modprobe fuse 2022-08-01T19:34:28.712 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:34:28.713 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:34:28.726 INFO:teuthology.orchestra.run.smithi190.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:34:28.727 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:34:28.729 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:34:28.729 DEBUG:teuthology.orchestra.run.smithi190:> ls /sys/fs/fuse/connections 2022-08-01T19:34:28.779 INFO:teuthology.orchestra.run.smithi190.stdout:52 2022-08-01T19:34:28.779 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T19:34:28.780 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:34:28.780 DEBUG:teuthology.orchestra.run.smithi190:> sudo python3 -c ' 2022-08-01T19:34:28.780 DEBUG:teuthology.orchestra.run.smithi190:> import glob 2022-08-01T19:34:28.781 DEBUG:teuthology.orchestra.run.smithi190:> import re 2022-08-01T19:34:28.781 DEBUG:teuthology.orchestra.run.smithi190:> import os 2022-08-01T19:34:28.782 DEBUG:teuthology.orchestra.run.smithi190:> import subprocess 2022-08-01T19:34:28.782 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:34:28.782 DEBUG:teuthology.orchestra.run.smithi190:> def find_socket(client_name): 2022-08-01T19:34:28.783 DEBUG:teuthology.orchestra.run.smithi190:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-08-01T19:34:28.783 DEBUG:teuthology.orchestra.run.smithi190:> files = glob.glob(asok_path) 2022-08-01T19:34:28.783 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:34:28.783 DEBUG:teuthology.orchestra.run.smithi190:> # Given a non-glob path, it better be there 2022-08-01T19:34:28.784 DEBUG:teuthology.orchestra.run.smithi190:> if "*" not in asok_path: 2022-08-01T19:34:28.784 DEBUG:teuthology.orchestra.run.smithi190:> assert(len(files) == 1) 2022-08-01T19:34:28.784 DEBUG:teuthology.orchestra.run.smithi190:> return files[0] 2022-08-01T19:34:28.785 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:34:28.785 DEBUG:teuthology.orchestra.run.smithi190:> for f in files: 2022-08-01T19:34:28.785 DEBUG:teuthology.orchestra.run.smithi190:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T19:34:28.786 DEBUG:teuthology.orchestra.run.smithi190:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T19:34:28.786 DEBUG:teuthology.orchestra.run.smithi190:> return f 2022-08-01T19:34:28.786 DEBUG:teuthology.orchestra.run.smithi190:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T19:34:28.787 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:34:28.787 DEBUG:teuthology.orchestra.run.smithi190:> print(find_socket("client.1")) 2022-08-01T19:34:28.788 DEBUG:teuthology.orchestra.run.smithi190:> ' 2022-08-01T19:34:28.831 INFO:teuthology.orchestra.run.smithi190.stdout:/var/run/ceph/ceph-client.1.22493.asok 2022-08-01T19:34:28.836 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.22493.asok 2022-08-01T19:34:28.837 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:34:28.837 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.22493.asok status 2022-08-01T19:34:28.927 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2022-08-01T19:34:28.927 INFO:teuthology.orchestra.run.smithi190.stdout: "metadata": { 2022-08-01T19:34:28.928 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T19:34:28.928 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T19:34:28.928 INFO:teuthology.orchestra.run.smithi190.stdout: "entity_id": "1", 2022-08-01T19:34:28.928 INFO:teuthology.orchestra.run.smithi190.stdout: "hostname": "smithi190", 2022-08-01T19:34:28.929 INFO:teuthology.orchestra.run.smithi190.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-08-01T19:34:28.929 INFO:teuthology.orchestra.run.smithi190.stdout: "pid": "22493", 2022-08-01T19:34:28.929 INFO:teuthology.orchestra.run.smithi190.stdout: "root": "/" 2022-08-01T19:34:28.929 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:34:28.929 INFO:teuthology.orchestra.run.smithi190.stdout: "dentry_count": 0, 2022-08-01T19:34:28.930 INFO:teuthology.orchestra.run.smithi190.stdout: "dentry_pinned_count": 0, 2022-08-01T19:34:28.930 INFO:teuthology.orchestra.run.smithi190.stdout: "id": 4663, 2022-08-01T19:34:28.930 INFO:teuthology.orchestra.run.smithi190.stdout: "inst": { 2022-08-01T19:34:28.930 INFO:teuthology.orchestra.run.smithi190.stdout: "name": { 2022-08-01T19:34:28.931 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "client", 2022-08-01T19:34:28.931 INFO:teuthology.orchestra.run.smithi190.stdout: "num": 4663 2022-08-01T19:34:28.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:34:28.931 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": { 2022-08-01T19:34:28.932 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "v1", 2022-08-01T19:34:28.932 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": "172.21.15.190:0", 2022-08-01T19:34:28.932 INFO:teuthology.orchestra.run.smithi190.stdout: "nonce": 873706768 2022-08-01T19:34:28.932 INFO:teuthology.orchestra.run.smithi190.stdout: } 2022-08-01T19:34:28.932 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:34:28.933 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": { 2022-08-01T19:34:28.933 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "v1", 2022-08-01T19:34:28.933 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": "172.21.15.190:0", 2022-08-01T19:34:28.933 INFO:teuthology.orchestra.run.smithi190.stdout: "nonce": 873706768 2022-08-01T19:34:28.934 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:34:28.934 INFO:teuthology.orchestra.run.smithi190.stdout: "inst_str": "client.4663 172.21.15.190:0/873706768", 2022-08-01T19:34:28.934 INFO:teuthology.orchestra.run.smithi190.stdout: "addr_str": "172.21.15.190:0/873706768", 2022-08-01T19:34:28.934 INFO:teuthology.orchestra.run.smithi190.stdout: "inode_count": 1, 2022-08-01T19:34:28.934 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_epoch": 7, 2022-08-01T19:34:28.935 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_epoch": 25, 2022-08-01T19:34:28.935 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_epoch_barrier": 0, 2022-08-01T19:34:28.935 INFO:teuthology.orchestra.run.smithi190.stdout: "blacklisted": false, 2022-08-01T19:34:28.935 INFO:teuthology.orchestra.run.smithi190.stdout: "fs_name": "cephfs" 2022-08-01T19:34:28.936 INFO:teuthology.orchestra.run.smithi190.stdout:} 2022-08-01T19:34:28.937 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:34:28.937 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:34:28.938 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:34:28.946 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:34:28.947 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:34:28.948 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:34:28.948 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:34:28.968 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:34:28.968 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:34:28.968 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:34:28.977 INFO:teuthology.orchestra.run.smithi190.stdout:fuseblk 2022-08-01T19:34:28.978 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T19:34:28.979 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:34:28.979 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:34:29.003 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2022-08-01T19:34:29.028 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:34:29.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.130+0000 7ff7d0643700 1 Processor -- start 2022-08-01T19:34:29.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.130+0000 7ff7d0643700 1 -- start start 2022-08-01T19:34:29.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.134+0000 7ff7d0643700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ff7c80aa2b0 0x7ff7c80ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:29.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.134+0000 7ff7d0643700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7c80b2c80 con 0x7ff7c80b0630 2022-08-01T19:34:29.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.134+0000 7ff7d0643700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7c80b2900 con 0x7ff7c80b0a10 2022-08-01T19:34:29.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.134+0000 7ff7d0643700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7c80b2eb0 con 0x7ff7c80aa2b0 2022-08-01T19:34:29.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.134+0000 7ff7ce3df700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff7c80b0630 0x7ff7c80adec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57372/0 (socket says 172.21.15.17:57372) 2022-08-01T19:34:29.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.134+0000 7ff7ce3df700 1 -- 172.21.15.17:0/3190610677 learned_addr learned my addr 172.21.15.17:0/3190610677 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:29.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.134+0000 7ff7cdbde700 1 --2- 172.21.15.17:0/3190610677 >> v2:172.21.15.77:3300/0 conn(0x7ff7c80aa2b0 0x7ff7c80ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:29.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.134+0000 7ff7cd3dd700 1 -- 172.21.15.17:0/3190610677 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3286240156 0 0) 0x7ff7c80b2c80 con 0x7ff7c80b0630 2022-08-01T19:34:29.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.134+0000 7ff7cd3dd700 1 -- 172.21.15.17:0/3190610677 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7b0003260 con 0x7ff7c80b0630 2022-08-01T19:34:29.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.134+0000 7ff7cd3dd700 1 -- 172.21.15.17:0/3190610677 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2554566299 0 0) 0x7ff7c80b2900 con 0x7ff7c80b0a10 2022-08-01T19:34:29.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.134+0000 7ff7cd3dd700 1 -- 172.21.15.17:0/3190610677 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7c80b2c80 con 0x7ff7c80b0a10 2022-08-01T19:34:29.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.134+0000 7ff7cd3dd700 1 -- 172.21.15.17:0/3190610677 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2621351533 0 0) 0x7ff7b0003260 con 0x7ff7c80b0630 2022-08-01T19:34:29.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.134+0000 7ff7cdbde700 1 -- 172.21.15.17:0/3190610677 >> v1:172.21.15.77:6789/0 conn(0x7ff7c80b0a10 legacy=0x7ff7c8140730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:29.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.134+0000 7ff7cdbde700 1 -- 172.21.15.17:0/3190610677 >> v1:172.21.15.17:6789/0 conn(0x7ff7c80b0630 legacy=0x7ff7c80adec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:29.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.134+0000 7ff7cdbde700 1 -- 172.21.15.17:0/3190610677 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c80a9d00 con 0x7ff7c80aa2b0 2022-08-01T19:34:29.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.134+0000 7ff7cdbde700 1 --2- 172.21.15.17:0/3190610677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7c80aa2b0 0x7ff7c80ae5d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7ff7c4001110 tx=0x7ff7b8006df0).ready entity=mon.1 client_cookie=9868c92a2d3d4daa server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:29.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.134+0000 7ff7cd3dd700 1 -- 172.21.15.17:0/3190610677 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7c4001660 con 0x7ff7c80aa2b0 2022-08-01T19:34:29.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.134+0000 7ff7cd3dd700 1 -- 172.21.15.17:0/3190610677 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b8000e90 con 0x7ff7c80aa2b0 2022-08-01T19:34:29.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.134+0000 7ff7cd3dd700 1 -- 172.21.15.17:0/3190610677 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7b8012460 con 0x7ff7c80aa2b0 2022-08-01T19:34:29.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.134+0000 7ff7d0643700 1 -- 172.21.15.17:0/3190610677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7c80aa2b0 msgr2=0x7ff7c80ae5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:29.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.134+0000 7ff7d0643700 1 --2- 172.21.15.17:0/3190610677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7c80aa2b0 0x7ff7c80ae5d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7ff7c4001110 tx=0x7ff7b8006df0).stop 2022-08-01T19:34:29.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.134+0000 7ff7d0643700 1 -- 172.21.15.17:0/3190610677 shutdown_connections 2022-08-01T19:34:29.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.134+0000 7ff7d0643700 1 --2- 172.21.15.17:0/3190610677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7c80aa2b0 0x7ff7c80ae5d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:29.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.134+0000 7ff7d0643700 1 -- 172.21.15.17:0/3190610677 >> 172.21.15.17:0/3190610677 conn(0x7ff7c8060950 msgr2=0x7ff7c8060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:29.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.134+0000 7ff7d0643700 1 -- 172.21.15.17:0/3190610677 shutdown_connections 2022-08-01T19:34:29.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.134+0000 7ff7d0643700 1 -- 172.21.15.17:0/3190610677 wait complete. 2022-08-01T19:34:29.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.138+0000 7ff7d0643700 1 Processor -- start 2022-08-01T19:34:29.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.138+0000 7ff7d0643700 1 -- start start 2022-08-01T19:34:29.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.138+0000 7ff7d0643700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7c80aa2b0 0x7ff7c80add00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:29.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.138+0000 7ff7d0643700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7c80b0630 0x7ff7c813f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:29.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.138+0000 7ff7d0643700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff7c80b0a10 0x7ff7c813fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:29.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.138+0000 7ff7d0643700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c8064bc0 con 0x7ff7c80aa2b0 2022-08-01T19:34:29.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.138+0000 7ff7d0643700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c8066870 con 0x7ff7c80b0630 2022-08-01T19:34:29.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.138+0000 7ff7d0643700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c8064df0 con 0x7ff7c80b0a10 2022-08-01T19:34:29.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.138+0000 7ff7ce3df700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7c80aa2b0 0x7ff7c80add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:29.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.138+0000 7ff7cdbde700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7c80b0630 0x7ff7c813f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:29.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.138+0000 7ff7ce3df700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7c80aa2b0 0x7ff7c80add00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36952/0 (socket says 172.21.15.17:36952) 2022-08-01T19:34:29.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.138+0000 7ff7ce3df700 1 -- 172.21.15.17:0/2704580121 learned_addr learned my addr 172.21.15.17:0/2704580121 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:29.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.138+0000 7ff7cebe0700 1 --2- 172.21.15.17:0/2704580121 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff7c80b0a10 0x7ff7c813fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:29.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.138+0000 7ff7ce3df700 1 -- 172.21.15.17:0/2704580121 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff7c80b0a10 msgr2=0x7ff7c813fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:29.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.138+0000 7ff7ce3df700 1 --2- 172.21.15.17:0/2704580121 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff7c80b0a10 0x7ff7c813fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:29.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.138+0000 7ff7ce3df700 1 -- 172.21.15.17:0/2704580121 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7c80b0630 msgr2=0x7ff7c813f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:29.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.138+0000 7ff7ce3df700 1 --2- 172.21.15.17:0/2704580121 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7c80b0630 0x7ff7c813f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:29.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.138+0000 7ff7ce3df700 1 -- 172.21.15.17:0/2704580121 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b800b180 con 0x7ff7c80aa2b0 2022-08-01T19:34:29.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.142+0000 7ff7ce3df700 1 --2- 172.21.15.17:0/2704580121 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7c80aa2b0 0x7ff7c80add00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7ff7c4001840 tx=0x7ff7c4001720).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:29.154 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.142+0000 7ff7b77fe700 1 -- 172.21.15.17:0/2704580121 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7c4007170 con 0x7ff7c80aa2b0 2022-08-01T19:34:29.154 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.142+0000 7ff7b77fe700 1 -- 172.21.15.17:0/2704580121 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c40072d0 con 0x7ff7c80aa2b0 2022-08-01T19:34:29.154 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.142+0000 7ff7b77fe700 1 -- 172.21.15.17:0/2704580121 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7c4004710 con 0x7ff7c80aa2b0 2022-08-01T19:34:29.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.142+0000 7ff7d0643700 1 -- 172.21.15.17:0/2704580121 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7c8065090 con 0x7ff7c80aa2b0 2022-08-01T19:34:29.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.142+0000 7ff7d0643700 1 -- 172.21.15.17:0/2704580121 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c8066510 con 0x7ff7c80aa2b0 2022-08-01T19:34:29.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.142+0000 7ff7d0643700 1 -- 172.21.15.17:0/2704580121 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7c80574e0 con 0x7ff7c80aa2b0 2022-08-01T19:34:29.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.142+0000 7ff7b77fe700 1 -- 172.21.15.17:0/2704580121 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff7c401b070 con 0x7ff7c80aa2b0 2022-08-01T19:34:29.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.142+0000 7ff7b77fe700 1 --2- 172.21.15.17:0/2704580121 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff7a8053b00 0x7ff7a8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:29.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.146+0000 7ff7b77fe700 1 -- 172.21.15.17:0/2704580121 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7ff7c406b6b0 con 0x7ff7c80aa2b0 2022-08-01T19:34:29.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.146+0000 7ff7cdbde700 1 --2- 172.21.15.17:0/2704580121 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff7a8053b00 0x7ff7a8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:29.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.146+0000 7ff7cdbde700 1 --2- 172.21.15.17:0/2704580121 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff7a8053b00 0x7ff7a8055fc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7ff7b801a000 tx=0x7ff7b8014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:29.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.158+0000 7ff7b77fe700 1 -- 172.21.15.17:0/2704580121 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff7c40423d0 con 0x7ff7c80aa2b0 2022-08-01T19:34:29.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.274+0000 7ff7d0643700 1 -- 172.21.15.17:0/2704580121 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7c8065220 con 0x7ff7c80aa2b0 2022-08-01T19:34:29.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.278+0000 7ff7b77fe700 1 -- 172.21.15.17:0/2704580121 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16299 (secure 0 0 0) 0x7ff7c40459f0 con 0x7ff7c80aa2b0 2022-08-01T19:34:29.282 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:29.283 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:34:09.692404+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:34:04.635473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-08-01T19:34:07.409434+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:34:29.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.278+0000 7ff7d0643700 1 -- 172.21.15.17:0/2704580121 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff7a8053b00 msgr2=0x7ff7a8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:29.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.278+0000 7ff7d0643700 1 --2- 172.21.15.17:0/2704580121 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff7a8053b00 0x7ff7a8055fc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7ff7b801a000 tx=0x7ff7b8014000).stop 2022-08-01T19:34:29.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.278+0000 7ff7d0643700 1 -- 172.21.15.17:0/2704580121 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7c80aa2b0 msgr2=0x7ff7c80add00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:29.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.278+0000 7ff7d0643700 1 --2- 172.21.15.17:0/2704580121 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7c80aa2b0 0x7ff7c80add00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7ff7c4001840 tx=0x7ff7c4001720).stop 2022-08-01T19:34:29.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.278+0000 7ff7d0643700 1 -- 172.21.15.17:0/2704580121 shutdown_connections 2022-08-01T19:34:29.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.278+0000 7ff7d0643700 1 --2- 172.21.15.17:0/2704580121 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff7a8053b00 0x7ff7a8055fc0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:29.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.278+0000 7ff7d0643700 1 --2- 172.21.15.17:0/2704580121 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff7c80b0a10 0x7ff7c813fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:29.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.278+0000 7ff7d0643700 1 --2- 172.21.15.17:0/2704580121 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7c80b0630 0x7ff7c813f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:29.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.278+0000 7ff7d0643700 1 --2- 172.21.15.17:0/2704580121 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7c80aa2b0 0x7ff7c80add00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:29.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.278+0000 7ff7d0643700 1 -- 172.21.15.17:0/2704580121 >> 172.21.15.17:0/2704580121 conn(0x7ff7c8060950 msgr2=0x7ff7c81408d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:29.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.278+0000 7ff7d0643700 1 -- 172.21.15.17:0/2704580121 shutdown_connections 2022-08-01T19:34:29.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.278+0000 7ff7d0643700 1 -- 172.21.15.17:0/2704580121 wait complete. 2022-08-01T19:34:29.295 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T19:34:04.635473+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T19:34:07.409434+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T19:34:29.296 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:34:29.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.402+0000 7f67d359e700 1 Processor -- start 2022-08-01T19:34:29.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.402+0000 7f67d359e700 1 -- start start 2022-08-01T19:34:29.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.402+0000 7f67d359e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f67d4064b10 0x7f67d4144e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:29.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.402+0000 7f67d359e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f67d4141a80 0x7f67d4066ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:29.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.402+0000 7f67d359e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67d4064ef0 con 0x7f67d40b36e0 2022-08-01T19:34:29.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.402+0000 7f67d359e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67d413ebf0 con 0x7f67d4064b10 2022-08-01T19:34:29.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.402+0000 7f67d359e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67d4066690 con 0x7f67d4141a80 2022-08-01T19:34:29.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.402+0000 7f67d259c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f67d4064b10 0x7f67d4144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:29.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.402+0000 7f67d259c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f67d4064b10 0x7f67d4144e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36964/0 (socket says 172.21.15.17:36964) 2022-08-01T19:34:29.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.402+0000 7f67d2d9d700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f67d4141a80 0x7f67d4066ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:29.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.402+0000 7f67d1d9b700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7f67d40b36e0 0x7f67d4141370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:45954/0 (socket says 172.21.15.17:45954) 2022-08-01T19:34:29.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.402+0000 7f67d259c700 1 -- 172.21.15.17:0/2058661112 learned_addr learned my addr 172.21.15.17:0/2058661112 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:29.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.402+0000 7f67d2d9d700 1 -- 172.21.15.17:0/2058661112 >> v2:172.21.15.17:3300/0 conn(0x7f67d4064b10 msgr2=0x7f67d4144e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:29.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.402+0000 7f67d2d9d700 1 --2- 172.21.15.17:0/2058661112 >> v2:172.21.15.17:3300/0 conn(0x7f67d4064b10 0x7f67d4144e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:29.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.402+0000 7f67d2d9d700 1 -- 172.21.15.17:0/2058661112 >> v1:172.21.15.77:6789/0 conn(0x7f67d40b36e0 legacy=0x7f67d4141370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:29.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.402+0000 7f67d2d9d700 1 -- 172.21.15.17:0/2058661112 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d40b4850 con 0x7f67d4141a80 2022-08-01T19:34:29.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.406+0000 7f67d2d9d700 1 --2- 172.21.15.17:0/2058661112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67d4141a80 0x7f67d4066ac0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f67cc0032f0 tx=0x7f67cc0091d0).ready entity=mon.1 client_cookie=f773b526bd29223c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:29.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.406+0000 7f67d159a700 1 -- 172.21.15.17:0/2058661112 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f67cc00b680 con 0x7f67d4141a80 2022-08-01T19:34:29.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.406+0000 7f67d159a700 1 -- 172.21.15.17:0/2058661112 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67cc00b7e0 con 0x7f67d4141a80 2022-08-01T19:34:29.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.406+0000 7f67d159a700 1 -- 172.21.15.17:0/2058661112 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f67cc00b950 con 0x7f67d4141a80 2022-08-01T19:34:29.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.406+0000 7f67d359e700 1 -- 172.21.15.17:0/2058661112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67d4141a80 msgr2=0x7f67d4066ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:29.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.406+0000 7f67d359e700 1 --2- 172.21.15.17:0/2058661112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67d4141a80 0x7f67d4066ac0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f67cc0032f0 tx=0x7f67cc0091d0).stop 2022-08-01T19:34:29.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.406+0000 7f67d359e700 1 -- 172.21.15.17:0/2058661112 shutdown_connections 2022-08-01T19:34:29.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.406+0000 7f67d359e700 1 --2- 172.21.15.17:0/2058661112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67d4141a80 0x7f67d4066ac0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:29.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.406+0000 7f67d359e700 1 --2- 172.21.15.17:0/2058661112 >> v2:172.21.15.17:3300/0 conn(0x7f67d4064b10 0x7f67d4144e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:29.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.406+0000 7f67d359e700 1 -- 172.21.15.17:0/2058661112 >> 172.21.15.17:0/2058661112 conn(0x7f67d405f840 msgr2=0x7f67d405fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:29.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.406+0000 7f67d359e700 1 -- 172.21.15.17:0/2058661112 shutdown_connections 2022-08-01T19:34:29.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.406+0000 7f67d359e700 1 -- 172.21.15.17:0/2058661112 wait complete. 2022-08-01T19:34:29.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.406+0000 7f67d359e700 1 Processor -- start 2022-08-01T19:34:29.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.406+0000 7f67d359e700 1 -- start start 2022-08-01T19:34:29.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.410+0000 7f67d359e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67d4064b10 0x7f67d40aed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:29.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.410+0000 7f67d359e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67d40b36e0 0x7f67d41432e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:29.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.410+0000 7f67d359e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67d4141a80 0x7f67d41437d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:29.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.410+0000 7f67d359e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d4144210 con 0x7f67d40b36e0 2022-08-01T19:34:29.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.410+0000 7f67d359e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d41508f0 con 0x7f67d4141a80 2022-08-01T19:34:29.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.410+0000 7f67d359e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d4150b90 con 0x7f67d4064b10 2022-08-01T19:34:29.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.410+0000 7f67d1d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67d40b36e0 0x7f67d41432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:29.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.410+0000 7f67d1d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67d40b36e0 0x7f67d41432e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36968/0 (socket says 172.21.15.17:36968) 2022-08-01T19:34:29.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.410+0000 7f67d1d9b700 1 -- 172.21.15.17:0/1194708333 learned_addr learned my addr 172.21.15.17:0/1194708333 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:29.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.410+0000 7f67d2d9d700 1 --2- 172.21.15.17:0/1194708333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67d4141a80 0x7f67d41437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:29.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.410+0000 7f67d259c700 1 --2- 172.21.15.17:0/1194708333 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67d4064b10 0x7f67d40aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:29.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.410+0000 7f67d1d9b700 1 -- 172.21.15.17:0/1194708333 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67d4064b10 msgr2=0x7f67d40aed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:29.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.410+0000 7f67d1d9b700 1 --2- 172.21.15.17:0/1194708333 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67d4064b10 0x7f67d40aed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:29.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.410+0000 7f67d1d9b700 1 -- 172.21.15.17:0/1194708333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67d4141a80 msgr2=0x7f67d41437d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:29.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.410+0000 7f67d1d9b700 1 --2- 172.21.15.17:0/1194708333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67d4141a80 0x7f67d41437d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:29.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.410+0000 7f67d1d9b700 1 -- 172.21.15.17:0/1194708333 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67cc009470 con 0x7f67d40b36e0 2022-08-01T19:34:29.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.410+0000 7f67d1d9b700 1 --2- 172.21.15.17:0/1194708333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67d40b36e0 0x7f67d41432e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f67d40b74a0 tx=0x7f67c4007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:29.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.410+0000 7f67c37fe700 1 -- 172.21.15.17:0/1194708333 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f67c40070d0 con 0x7f67d40b36e0 2022-08-01T19:34:29.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.410+0000 7f67c37fe700 1 -- 172.21.15.17:0/1194708333 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c4007230 con 0x7f67d40b36e0 2022-08-01T19:34:29.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.410+0000 7f67d359e700 1 -- 172.21.15.17:0/1194708333 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d4150e30 con 0x7f67d40b36e0 2022-08-01T19:34:29.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.410+0000 7f67c37fe700 1 -- 172.21.15.17:0/1194708333 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f67c4005550 con 0x7f67d40b36e0 2022-08-01T19:34:29.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.410+0000 7f67d359e700 1 -- 172.21.15.17:0/1194708333 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d4143cc0 con 0x7f67d40b36e0 2022-08-01T19:34:29.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.410+0000 7f67c37fe700 1 -- 172.21.15.17:0/1194708333 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f67c4010070 con 0x7f67d40b36e0 2022-08-01T19:34:29.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.410+0000 7f67d359e700 1 -- 172.21.15.17:0/1194708333 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67d40b4af0 con 0x7f67d40b36e0 2022-08-01T19:34:29.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.414+0000 7f67c37fe700 1 --2- 172.21.15.17:0/1194708333 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f67b00539e0 0x7f67b0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:29.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.414+0000 7f67c37fe700 1 -- 172.21.15.17:0/1194708333 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f67c406b6e0 con 0x7f67d40b36e0 2022-08-01T19:34:29.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.414+0000 7f67d259c700 1 --2- 172.21.15.17:0/1194708333 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f67b00539e0 0x7f67b0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:29.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.414+0000 7f67d259c700 1 --2- 172.21.15.17:0/1194708333 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f67b00539e0 0x7f67b0055ea0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f67c8008000 tx=0x7f67c8008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:29.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.418+0000 7f67c37fe700 1 -- 172.21.15.17:0/1194708333 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f67c40424a0 con 0x7f67d40b36e0 2022-08-01T19:34:29.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.546+0000 7f67d359e700 1 -- 172.21.15.17:0/1194708333 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f67d40574e0 con 0x7f67d40b36e0 2022-08-01T19:34:29.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.546+0000 7f67c37fe700 1 -- 172.21.15.17:0/1194708333 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v25) v1 ==== 110+0+10 (secure 0 0 0) 0x7f67c4045ac0 con 0x7f67d40b36e0 2022-08-01T19:34:29.552 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:34:29.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.546+0000 7f67d359e700 1 -- 172.21.15.17:0/1194708333 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f67b00539e0 msgr2=0x7f67b0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:29.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.546+0000 7f67d359e700 1 --2- 172.21.15.17:0/1194708333 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f67b00539e0 0x7f67b0055ea0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f67c8008000 tx=0x7f67c8008040).stop 2022-08-01T19:34:29.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.546+0000 7f67d359e700 1 -- 172.21.15.17:0/1194708333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67d40b36e0 msgr2=0x7f67d41432e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:29.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.546+0000 7f67d359e700 1 --2- 172.21.15.17:0/1194708333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67d40b36e0 0x7f67d41432e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f67d40b74a0 tx=0x7f67c4007c40).stop 2022-08-01T19:34:29.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.546+0000 7f67d359e700 1 -- 172.21.15.17:0/1194708333 shutdown_connections 2022-08-01T19:34:29.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.546+0000 7f67d359e700 1 --2- 172.21.15.17:0/1194708333 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f67b00539e0 0x7f67b0055ea0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:29.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.546+0000 7f67d359e700 1 --2- 172.21.15.17:0/1194708333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67d4141a80 0x7f67d41437d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:29.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.546+0000 7f67d359e700 1 --2- 172.21.15.17:0/1194708333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67d40b36e0 0x7f67d41432e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:29.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.546+0000 7f67d359e700 1 --2- 172.21.15.17:0/1194708333 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67d4064b10 0x7f67d40aed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:29.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.546+0000 7f67d359e700 1 -- 172.21.15.17:0/1194708333 >> 172.21.15.17:0/1194708333 conn(0x7f67d405f840 msgr2=0x7f67d40b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:29.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.550+0000 7f67d359e700 1 -- 172.21.15.17:0/1194708333 shutdown_connections 2022-08-01T19:34:29.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.550+0000 7f67d359e700 1 -- 172.21.15.17:0/1194708333 wait complete. 2022-08-01T19:34:29.565 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:34:29.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.666+0000 7f550b0d8700 1 Processor -- start 2022-08-01T19:34:29.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.666+0000 7f550b0d8700 1 -- start start 2022-08-01T19:34:29.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.666+0000 7f550b0d8700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f55040a8c50 0x7f5504152d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:29.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.666+0000 7f550b0d8700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55040a7250 con 0x7f55040b01a0 2022-08-01T19:34:29.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.666+0000 7f550b0d8700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55040a7480 con 0x7f55040b2710 2022-08-01T19:34:29.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.666+0000 7f550b0d8700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55040a7720 con 0x7f55040a8c50 2022-08-01T19:34:29.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.666+0000 7f55098d5700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f55040b01a0 0x7f55040a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57400/0 (socket says 172.21.15.17:57400) 2022-08-01T19:34:29.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.666+0000 7f55098d5700 1 -- 172.21.15.17:0/601730680 learned_addr learned my addr 172.21.15.17:0/601730680 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:29.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.666+0000 7f550a8d7700 1 --2- 172.21.15.17:0/601730680 >> v2:172.21.15.77:3300/0 conn(0x7f55040a8c50 0x7f5504152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:29.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.666+0000 7f55090d4700 1 -- 172.21.15.17:0/601730680 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2650425930 0 0) 0x7f55040a7250 con 0x7f55040b01a0 2022-08-01T19:34:29.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.666+0000 7f55090d4700 1 -- 172.21.15.17:0/601730680 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54ec003260 con 0x7f55040b01a0 2022-08-01T19:34:29.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.666+0000 7f550a8d7700 1 -- 172.21.15.17:0/601730680 >> v1:172.21.15.77:6789/0 conn(0x7f55040b2710 legacy=0x7f550414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:29.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.666+0000 7f550a8d7700 1 -- 172.21.15.17:0/601730680 >> v1:172.21.15.17:6789/0 conn(0x7f55040b01a0 legacy=0x7f55040a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:29.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.666+0000 7f55090d4700 1 -- 172.21.15.17:0/601730680 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 993252500 0 0) 0x7f54ec003260 con 0x7f55040b01a0 2022-08-01T19:34:29.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.670+0000 7f550a8d7700 1 -- 172.21.15.17:0/601730680 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55040a79c0 con 0x7f55040a8c50 2022-08-01T19:34:29.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.670+0000 7f550a8d7700 1 --2- 172.21.15.17:0/601730680 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55040a8c50 0x7f5504152d70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f5500001b80 tx=0x7f54fc006df0).ready entity=mon.1 client_cookie=21eec603fe59ef43 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:29.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.670+0000 7f55090d4700 1 -- 172.21.15.17:0/601730680 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55000019e0 con 0x7f55040a8c50 2022-08-01T19:34:29.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.670+0000 7f55090d4700 1 -- 172.21.15.17:0/601730680 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54fc000e90 con 0x7f55040a8c50 2022-08-01T19:34:29.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.670+0000 7f55090d4700 1 -- 172.21.15.17:0/601730680 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54fc00d4f0 con 0x7f55040a8c50 2022-08-01T19:34:29.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.670+0000 7f550b0d8700 1 -- 172.21.15.17:0/601730680 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55040a8c50 msgr2=0x7f5504152d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:29.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.670+0000 7f550b0d8700 1 --2- 172.21.15.17:0/601730680 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55040a8c50 0x7f5504152d70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f5500001b80 tx=0x7f54fc006df0).stop 2022-08-01T19:34:29.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.670+0000 7f550b0d8700 1 -- 172.21.15.17:0/601730680 shutdown_connections 2022-08-01T19:34:29.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.670+0000 7f550b0d8700 1 --2- 172.21.15.17:0/601730680 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55040a8c50 0x7f5504152d70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:29.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.670+0000 7f550b0d8700 1 -- 172.21.15.17:0/601730680 >> 172.21.15.17:0/601730680 conn(0x7f5504060310 msgr2=0x7f55040b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:29.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.670+0000 7f550b0d8700 1 -- 172.21.15.17:0/601730680 shutdown_connections 2022-08-01T19:34:29.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.670+0000 7f550b0d8700 1 -- 172.21.15.17:0/601730680 wait complete. 2022-08-01T19:34:29.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.670+0000 7f550b0d8700 1 Processor -- start 2022-08-01T19:34:29.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.670+0000 7f550b0d8700 1 -- start start 2022-08-01T19:34:29.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.674+0000 7f550b0d8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55040a8c50 0x7f55041497c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:29.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.674+0000 7f550b0d8700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f55040b01a0 0x7f5504063eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:29.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.674+0000 7f550b0d8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f55040b2710 0x7f55040643a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:29.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.674+0000 7f550b0d8700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5504149cb0 con 0x7f55040b2710 2022-08-01T19:34:29.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.674+0000 7f550b0d8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5504067720 con 0x7f55040a8c50 2022-08-01T19:34:29.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.674+0000 7f550b0d8700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55041571b0 con 0x7f55040b01a0 2022-08-01T19:34:29.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.674+0000 7f550a8d7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f55040b2710 0x7f55040643a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:29.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.674+0000 7f550a8d7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f55040b2710 0x7f55040643a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36984/0 (socket says 172.21.15.17:36984) 2022-08-01T19:34:29.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.674+0000 7f550a8d7700 1 -- 172.21.15.17:0/3774335501 learned_addr learned my addr 172.21.15.17:0/3774335501 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:29.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.674+0000 7f55098d5700 1 --2- 172.21.15.17:0/3774335501 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f55040b01a0 0x7f5504063eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:29.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.674+0000 7f550a0d6700 1 --2- 172.21.15.17:0/3774335501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55040a8c50 0x7f55041497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:29.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.674+0000 7f550a8d7700 1 -- 172.21.15.17:0/3774335501 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f55040b01a0 msgr2=0x7f5504063eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:29.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.674+0000 7f550a8d7700 1 --2- 172.21.15.17:0/3774335501 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f55040b01a0 0x7f5504063eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:29.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.674+0000 7f550a8d7700 1 -- 172.21.15.17:0/3774335501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55040a8c50 msgr2=0x7f55041497c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:29.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.674+0000 7f550a8d7700 1 --2- 172.21.15.17:0/3774335501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55040a8c50 0x7f55041497c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:29.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.674+0000 7f550a8d7700 1 -- 172.21.15.17:0/3774335501 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54fc00b180 con 0x7f55040b2710 2022-08-01T19:34:29.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.674+0000 7f550a0d6700 1 --2- 172.21.15.17:0/3774335501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55040a8c50 0x7f55041497c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:34:29.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.674+0000 7f550a8d7700 1 --2- 172.21.15.17:0/3774335501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f55040b2710 0x7f55040643a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f5500001b80 tx=0x7f54fc014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:29.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.674+0000 7f54f37fe700 1 -- 172.21.15.17:0/3774335501 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54fc016030 con 0x7f55040b2710 2022-08-01T19:34:29.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.674+0000 7f54f37fe700 1 -- 172.21.15.17:0/3774335501 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54fc013c20 con 0x7f55040b2710 2022-08-01T19:34:29.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.674+0000 7f550b0d8700 1 -- 172.21.15.17:0/3774335501 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5504067480 con 0x7f55040b2710 2022-08-01T19:34:29.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.674+0000 7f54f37fe700 1 -- 172.21.15.17:0/3774335501 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54fc00d4f0 con 0x7f55040b2710 2022-08-01T19:34:29.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.674+0000 7f550b0d8700 1 -- 172.21.15.17:0/3774335501 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5504064890 con 0x7f55040b2710 2022-08-01T19:34:29.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.674+0000 7f550b0d8700 1 -- 172.21.15.17:0/3774335501 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5504157780 con 0x7f55040b2710 2022-08-01T19:34:29.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.674+0000 7f54f37fe700 1 -- 172.21.15.17:0/3774335501 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f54fc024020 con 0x7f55040b2710 2022-08-01T19:34:29.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.678+0000 7f54f37fe700 1 --2- 172.21.15.17:0/3774335501 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f54e4053a10 0x7f54e4055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:29.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.678+0000 7f54f37fe700 1 -- 172.21.15.17:0/3774335501 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f54fc017070 con 0x7f55040b2710 2022-08-01T19:34:29.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.678+0000 7f550a0d6700 1 --2- 172.21.15.17:0/3774335501 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f54e4053a10 0x7f54e4055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:29.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.678+0000 7f550a0d6700 1 --2- 172.21.15.17:0/3774335501 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f54e4053a10 0x7f54e4055ed0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f54f400b000 tx=0x7f54f4009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:29.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.682+0000 7f54f37fe700 1 -- 172.21.15.17:0/3774335501 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f54fc071070 con 0x7f55040b2710 2022-08-01T19:34:29.815 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.810+0000 7f550b0d8700 1 -- 172.21.15.17:0/3774335501 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5504149600 con 0x7f55040b2710 2022-08-01T19:34:29.816 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.810+0000 7f54f37fe700 1 -- 172.21.15.17:0/3774335501 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f54fc043c20 con 0x7f55040b2710 2022-08-01T19:34:29.817 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:34:29.818 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.814+0000 7f550b0d8700 1 -- 172.21.15.17:0/3774335501 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f54e4053a10 msgr2=0x7f54e4055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:29.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.814+0000 7f550b0d8700 1 --2- 172.21.15.17:0/3774335501 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f54e4053a10 0x7f54e4055ed0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f54f400b000 tx=0x7f54f4009040).stop 2022-08-01T19:34:29.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.814+0000 7f550b0d8700 1 -- 172.21.15.17:0/3774335501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f55040b2710 msgr2=0x7f55040643a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:29.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.814+0000 7f550b0d8700 1 --2- 172.21.15.17:0/3774335501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f55040b2710 0x7f55040643a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f5500001b80 tx=0x7f54fc014000).stop 2022-08-01T19:34:29.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.814+0000 7f550b0d8700 1 -- 172.21.15.17:0/3774335501 shutdown_connections 2022-08-01T19:34:29.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.814+0000 7f550b0d8700 1 --2- 172.21.15.17:0/3774335501 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f54e4053a10 0x7f54e4055ed0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:29.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.814+0000 7f550b0d8700 1 --2- 172.21.15.17:0/3774335501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f55040b2710 0x7f55040643a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:29.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.814+0000 7f550b0d8700 1 --2- 172.21.15.17:0/3774335501 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f55040b01a0 0x7f5504063eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:29.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.814+0000 7f550b0d8700 1 --2- 172.21.15.17:0/3774335501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55040a8c50 0x7f55041497c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:29.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.814+0000 7f550b0d8700 1 -- 172.21.15.17:0/3774335501 >> 172.21.15.17:0/3774335501 conn(0x7f5504060310 msgr2=0x7f550405be30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:29.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.814+0000 7f550b0d8700 1 -- 172.21.15.17:0/3774335501 shutdown_connections 2022-08-01T19:34:29.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.814+0000 7f550b0d8700 1 -- 172.21.15.17:0/3774335501 wait complete. 2022-08-01T19:34:29.829 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T19:34:29.939 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.934+0000 7fbfc8881700 1 Processor -- start 2022-08-01T19:34:29.939 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.934+0000 7fbfc8881700 1 -- start start 2022-08-01T19:34:29.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.934+0000 7fbfc8881700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbfc4146ae0 0x7fbfc41403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:29.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.934+0000 7fbfc8881700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbfc40b4310 0x7fbfc41408b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:29.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.934+0000 7fbfc8881700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc4143cf0 con 0x7fbfc40b4820 2022-08-01T19:34:29.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.934+0000 7fbfc8881700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc41476a0 con 0x7fbfc40b4310 2022-08-01T19:34:29.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.934+0000 7fbfc8881700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc4143f90 con 0x7fbfc4146ae0 2022-08-01T19:34:29.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.934+0000 7fbfc259c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbfc4146ae0 0x7fbfc41403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:29.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.934+0000 7fbfc1d9b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbfc40b4310 0x7fbfc41408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:29.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.934+0000 7fbfc259c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbfc4146ae0 0x7fbfc41403c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:33088/0 (socket says 172.21.15.17:33088) 2022-08-01T19:34:29.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.934+0000 7fbfc259c700 1 -- 172.21.15.17:0/751083811 learned_addr learned my addr 172.21.15.17:0/751083811 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:29.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.934+0000 7fbfc2d9d700 1 --1- 172.21.15.17:0/751083811 >> v1:172.21.15.17:6789/0 conn(0x7fbfc40b4820 0x7fbfc40b5060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57416/0 (socket says 172.21.15.17:57416) 2022-08-01T19:34:29.947 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.934+0000 7fbfc259c700 1 -- 172.21.15.17:0/751083811 >> v2:172.21.15.17:3300/0 conn(0x7fbfc40b4310 msgr2=0x7fbfc41408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:29.947 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.934+0000 7fbfc259c700 1 --2- 172.21.15.17:0/751083811 >> v2:172.21.15.17:3300/0 conn(0x7fbfc40b4310 0x7fbfc41408b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:29.948 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.934+0000 7fbfc259c700 1 -- 172.21.15.17:0/751083811 >> v1:172.21.15.17:6789/0 conn(0x7fbfc40b4820 legacy=0x7fbfc40b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:29.948 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.934+0000 7fbfc259c700 1 -- 172.21.15.17:0/751083811 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc40b0e20 con 0x7fbfc4146ae0 2022-08-01T19:34:29.948 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.934+0000 7fbfc1d9b700 1 --2- 172.21.15.17:0/751083811 >> v2:172.21.15.17:3300/0 conn(0x7fbfc40b4310 0x7fbfc41408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:34:29.948 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.938+0000 7fbfc259c700 1 --2- 172.21.15.17:0/751083811 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbfc4146ae0 0x7fbfc41403c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fbfb40032f0 tx=0x7fbfb4009290).ready entity=mon.2 client_cookie=12478604e241a0d9 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:29.949 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.938+0000 7fbfc159a700 1 -- 172.21.15.17:0/751083811 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfb400b8a0 con 0x7fbfc4146ae0 2022-08-01T19:34:29.949 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.938+0000 7fbfc159a700 1 -- 172.21.15.17:0/751083811 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb400ba00 con 0x7fbfc4146ae0 2022-08-01T19:34:29.949 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.938+0000 7fbfc159a700 1 -- 172.21.15.17:0/751083811 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfb400bb70 con 0x7fbfc4146ae0 2022-08-01T19:34:29.949 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.938+0000 7fbfc8881700 1 -- 172.21.15.17:0/751083811 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbfc4146ae0 msgr2=0x7fbfc41403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:29.950 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.938+0000 7fbfc8881700 1 --2- 172.21.15.17:0/751083811 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbfc4146ae0 0x7fbfc41403c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fbfb40032f0 tx=0x7fbfb4009290).stop 2022-08-01T19:34:29.950 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.938+0000 7fbfc8881700 1 -- 172.21.15.17:0/751083811 shutdown_connections 2022-08-01T19:34:29.950 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.938+0000 7fbfc8881700 1 --2- 172.21.15.17:0/751083811 >> v2:172.21.15.17:3300/0 conn(0x7fbfc40b4310 0x7fbfc41408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:29.950 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.938+0000 7fbfc8881700 1 --2- 172.21.15.17:0/751083811 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbfc4146ae0 0x7fbfc41403c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:29.950 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.938+0000 7fbfc8881700 1 -- 172.21.15.17:0/751083811 >> 172.21.15.17:0/751083811 conn(0x7fbfc405c620 msgr2=0x7fbfc405ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:29.951 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.938+0000 7fbfc8881700 1 -- 172.21.15.17:0/751083811 shutdown_connections 2022-08-01T19:34:29.951 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.938+0000 7fbfc8881700 1 -- 172.21.15.17:0/751083811 wait complete. 2022-08-01T19:34:29.951 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.938+0000 7fbfc8881700 1 Processor -- start 2022-08-01T19:34:29.951 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.938+0000 7fbfc8881700 1 -- start start 2022-08-01T19:34:29.952 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.938+0000 7fbfc8881700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbfc40b4310 0x7fbfc4112d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:29.952 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.938+0000 7fbfc8881700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbfc40b4820 0x7fbfc41131f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:29.952 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.938+0000 7fbfc8881700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfc4146ae0 0x7fbfc414f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:29.952 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.938+0000 7fbfc8881700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc41532c0 con 0x7fbfc4146ae0 2022-08-01T19:34:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.938+0000 7fbfc8881700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc4143950 con 0x7fbfc40b4310 2022-08-01T19:34:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.938+0000 7fbfc8881700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc4143390 con 0x7fbfc40b4820 2022-08-01T19:34:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.938+0000 7fbfc2d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfc4146ae0 0x7fbfc414f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.938+0000 7fbfc2d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfc4146ae0 0x7fbfc414f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36998/0 (socket says 172.21.15.17:36998) 2022-08-01T19:34:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.938+0000 7fbfc2d9d700 1 -- 172.21.15.17:0/2589179249 learned_addr learned my addr 172.21.15.17:0/2589179249 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:29.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.938+0000 7fbfc1d9b700 1 --2- 172.21.15.17:0/2589179249 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbfc40b4820 0x7fbfc41131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:29.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.938+0000 7fbfc259c700 1 --2- 172.21.15.17:0/2589179249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbfc40b4310 0x7fbfc4112d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:29.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.938+0000 7fbfc2d9d700 1 -- 172.21.15.17:0/2589179249 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbfc40b4820 msgr2=0x7fbfc41131f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:29.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.938+0000 7fbfc2d9d700 1 --2- 172.21.15.17:0/2589179249 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbfc40b4820 0x7fbfc41131f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:29.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.938+0000 7fbfc2d9d700 1 -- 172.21.15.17:0/2589179249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbfc40b4310 msgr2=0x7fbfc4112d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:29.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.938+0000 7fbfc2d9d700 1 --2- 172.21.15.17:0/2589179249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbfc40b4310 0x7fbfc4112d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:29.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.938+0000 7fbfc2d9d700 1 -- 172.21.15.17:0/2589179249 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb800b180 con 0x7fbfc4146ae0 2022-08-01T19:34:29.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.938+0000 7fbfc2d9d700 1 --2- 172.21.15.17:0/2589179249 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfc4146ae0 0x7fbfc414f8e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7fbfc4060de0 tx=0x7fbfbc007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:29.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.942+0000 7fbfb37fe700 1 -- 172.21.15.17:0/2589179249 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfbc0070d0 con 0x7fbfc4146ae0 2022-08-01T19:34:29.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.942+0000 7fbfb37fe700 1 -- 172.21.15.17:0/2589179249 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfbc007230 con 0x7fbfc4146ae0 2022-08-01T19:34:29.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.942+0000 7fbfc8881700 1 -- 172.21.15.17:0/2589179249 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb4009530 con 0x7fbfc4146ae0 2022-08-01T19:34:29.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.942+0000 7fbfb37fe700 1 -- 172.21.15.17:0/2589179249 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfbc005740 con 0x7fbfc4146ae0 2022-08-01T19:34:29.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.942+0000 7fbfc8881700 1 -- 172.21.15.17:0/2589179249 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc4142c10 con 0x7fbfc4146ae0 2022-08-01T19:34:29.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.942+0000 7fbfc8881700 1 -- 172.21.15.17:0/2589179249 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfc41530e0 con 0x7fbfc4146ae0 2022-08-01T19:34:29.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.942+0000 7fbfb37fe700 1 -- 172.21.15.17:0/2589179249 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbfbc01a070 con 0x7fbfc4146ae0 2022-08-01T19:34:29.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.942+0000 7fbfb37fe700 1 --2- 172.21.15.17:0/2589179249 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbfa0053a70 0x7fbfa0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:29.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.942+0000 7fbfb37fe700 1 -- 172.21.15.17:0/2589179249 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fbfbc06b350 con 0x7fbfc4146ae0 2022-08-01T19:34:29.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.942+0000 7fbfc259c700 1 --2- 172.21.15.17:0/2589179249 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbfa0053a70 0x7fbfa0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:29.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.946+0000 7fbfc259c700 1 --2- 172.21.15.17:0/2589179249 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbfa0053a70 0x7fbfa0055f30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fbfb4019000 tx=0x7fbfb4016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:29.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:29.946+0000 7fbfb37fe700 1 -- 172.21.15.17:0/2589179249 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbfbc042000 con 0x7fbfc4146ae0 2022-08-01T19:34:30.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.074+0000 7fbfc8881700 1 -- 172.21.15.17:0/2589179249 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbfc40574e0 con 0x7fbfc4146ae0 2022-08-01T19:34:30.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.074+0000 7fbfb37fe700 1 -- 172.21.15.17:0/2589179249 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+10 (secure 0 0 0) 0x7fbfbc045620 con 0x7fbfc4146ae0 2022-08-01T19:34:30.085 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:34:30.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.078+0000 7fbfc8881700 1 -- 172.21.15.17:0/2589179249 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbfa0053a70 msgr2=0x7fbfa0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:30.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.078+0000 7fbfc8881700 1 --2- 172.21.15.17:0/2589179249 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbfa0053a70 0x7fbfa0055f30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fbfb4019000 tx=0x7fbfb4016000).stop 2022-08-01T19:34:30.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.078+0000 7fbfc8881700 1 -- 172.21.15.17:0/2589179249 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfc4146ae0 msgr2=0x7fbfc414f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:30.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.078+0000 7fbfc8881700 1 --2- 172.21.15.17:0/2589179249 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfc4146ae0 0x7fbfc414f8e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7fbfc4060de0 tx=0x7fbfbc007bc0).stop 2022-08-01T19:34:30.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.078+0000 7fbfc8881700 1 -- 172.21.15.17:0/2589179249 shutdown_connections 2022-08-01T19:34:30.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.078+0000 7fbfc8881700 1 --2- 172.21.15.17:0/2589179249 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbfa0053a70 0x7fbfa0055f30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:30.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.078+0000 7fbfc8881700 1 --2- 172.21.15.17:0/2589179249 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfc4146ae0 0x7fbfc414f8e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:30.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.078+0000 7fbfc8881700 1 --2- 172.21.15.17:0/2589179249 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbfc40b4820 0x7fbfc41131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:30.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.078+0000 7fbfc8881700 1 --2- 172.21.15.17:0/2589179249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbfc40b4310 0x7fbfc4112d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:30.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.078+0000 7fbfc8881700 1 -- 172.21.15.17:0/2589179249 >> 172.21.15.17:0/2589179249 conn(0x7fbfc405c620 msgr2=0x7fbfc40b5060 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:30.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.078+0000 7fbfc8881700 1 -- 172.21.15.17:0/2589179249 shutdown_connections 2022-08-01T19:34:30.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.078+0000 7fbfc8881700 1 -- 172.21.15.17:0/2589179249 wait complete. 2022-08-01T19:34:30.099 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T19:34:30.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.206+0000 7f81d5fe4700 1 Processor -- start 2022-08-01T19:34:30.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.206+0000 7f81d5fe4700 1 -- start start 2022-08-01T19:34:30.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.206+0000 7f81d5fe4700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f81d00ad750 0x7f81d0061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:30.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.206+0000 7f81d5fe4700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f81d01405d0 0x7f81d0140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:30.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.206+0000 7f81d5fe4700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81d00aee80 con 0x7f81d00614b0 2022-08-01T19:34:30.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.206+0000 7f81d5fe4700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81d00b70b0 con 0x7f81d01405d0 2022-08-01T19:34:30.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.206+0000 7f81d5fe4700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d00af0b0 con 0x7f81d00ad750 2022-08-01T19:34:30.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.206+0000 7f81d4fe2700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7f81d00614b0 0x7f81d00b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:45996/0 (socket says 172.21.15.17:45996) 2022-08-01T19:34:30.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.206+0000 7f81c7fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f81d00ad750 0x7f81d0061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:30.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.206+0000 7f81d4fe2700 1 -- 172.21.15.17:0/1541361360 learned_addr learned my addr 172.21.15.17:0/1541361360 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:30.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.206+0000 7f81d57e3700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f81d01405d0 0x7f81d0140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:30.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.206+0000 7f81c7fff700 1 -- 172.21.15.17:0/1541361360 >> v2:172.21.15.77:3300/0 conn(0x7f81d01405d0 msgr2=0x7f81d0140a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:30.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.206+0000 7f81c7fff700 1 --2- 172.21.15.17:0/1541361360 >> v2:172.21.15.77:3300/0 conn(0x7f81d01405d0 0x7f81d0140a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:30.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.206+0000 7f81c7fff700 1 -- 172.21.15.17:0/1541361360 >> v1:172.21.15.77:6789/0 conn(0x7f81d00614b0 legacy=0x7f81d00b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:30.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.206+0000 7f81c77fe700 1 -- 172.21.15.17:0/1541361360 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4268732435 0 0) 0x7f81d00aee80 con 0x7f81d00614b0 2022-08-01T19:34:30.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.206+0000 7f81c7fff700 1 -- 172.21.15.17:0/1541361360 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d00af350 con 0x7f81d00ad750 2022-08-01T19:34:30.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.206+0000 7f81d57e3700 1 --2- 172.21.15.17:0/1541361360 >> v2:172.21.15.77:3300/0 conn(0x7f81d01405d0 0x7f81d0140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:34:30.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.206+0000 7f81c7fff700 1 --2- 172.21.15.17:0/1541361360 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f81d00ad750 0x7f81d0061890 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f81c0002f70 tx=0x7f81c0006df0).ready entity=mon.2 client_cookie=a0634ae82c0b10c5 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:30.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.206+0000 7f81c77fe700 1 -- 172.21.15.17:0/1541361360 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81c000d570 con 0x7f81d00ad750 2022-08-01T19:34:30.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.206+0000 7f81c77fe700 1 -- 172.21.15.17:0/1541361360 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81c0000e90 con 0x7f81d00ad750 2022-08-01T19:34:30.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.206+0000 7f81c77fe700 1 -- 172.21.15.17:0/1541361360 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81c000d8c0 con 0x7f81d00ad750 2022-08-01T19:34:30.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.206+0000 7f81d5fe4700 1 -- 172.21.15.17:0/1541361360 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f81d00ad750 msgr2=0x7f81d0061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:30.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.206+0000 7f81d5fe4700 1 --2- 172.21.15.17:0/1541361360 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f81d00ad750 0x7f81d0061890 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f81c0002f70 tx=0x7f81c0006df0).stop 2022-08-01T19:34:30.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.206+0000 7f81d5fe4700 1 -- 172.21.15.17:0/1541361360 shutdown_connections 2022-08-01T19:34:30.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.206+0000 7f81d5fe4700 1 --2- 172.21.15.17:0/1541361360 >> v2:172.21.15.77:3300/0 conn(0x7f81d01405d0 0x7f81d0140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:30.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.206+0000 7f81d5fe4700 1 --2- 172.21.15.17:0/1541361360 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f81d00ad750 0x7f81d0061890 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:30.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.206+0000 7f81d5fe4700 1 -- 172.21.15.17:0/1541361360 >> 172.21.15.17:0/1541361360 conn(0x7f81d0066f70 msgr2=0x7f81d0067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:30.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.206+0000 7f81d5fe4700 1 -- 172.21.15.17:0/1541361360 shutdown_connections 2022-08-01T19:34:30.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.206+0000 7f81d5fe4700 1 -- 172.21.15.17:0/1541361360 wait complete. 2022-08-01T19:34:30.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.206+0000 7f81d5fe4700 1 Processor -- start 2022-08-01T19:34:30.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.206+0000 7f81d5fe4700 1 -- start start 2022-08-01T19:34:30.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.210+0000 7f81d5fe4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81d00614b0 0x7f81d00b1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:30.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.210+0000 7f81d5fe4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81d00ad750 0x7f81d00b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:30.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.210+0000 7f81d5fe4700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f81d01405d0 0x7f81d00ac420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:30.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.210+0000 7f81d5fe4700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d00b40c0 con 0x7f81d00614b0 2022-08-01T19:34:30.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.210+0000 7f81d5fe4700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d0152fc0 con 0x7f81d00ad750 2022-08-01T19:34:30.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.210+0000 7f81d5fe4700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d0153260 con 0x7f81d01405d0 2022-08-01T19:34:30.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.210+0000 7f81d4fe2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81d00614b0 0x7f81d00b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:30.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.210+0000 7f81c7fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81d00ad750 0x7f81d00b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:30.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.210+0000 7f81d57e3700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f81d01405d0 0x7f81d00ac420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:30.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.210+0000 7f81d4fe2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81d00614b0 0x7f81d00b1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37006/0 (socket says 172.21.15.17:37006) 2022-08-01T19:34:30.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.210+0000 7f81d4fe2700 1 -- 172.21.15.17:0/3142413028 learned_addr learned my addr 172.21.15.17:0/3142413028 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:30.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.210+0000 7f81d4fe2700 1 -- 172.21.15.17:0/3142413028 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f81d01405d0 msgr2=0x7f81d00ac420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:30.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.210+0000 7f81d4fe2700 1 --2- 172.21.15.17:0/3142413028 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f81d01405d0 0x7f81d00ac420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:30.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.210+0000 7f81d4fe2700 1 -- 172.21.15.17:0/3142413028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81d00ad750 msgr2=0x7f81d00b0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:30.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.210+0000 7f81d4fe2700 1 --2- 172.21.15.17:0/3142413028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81d00ad750 0x7f81d00b0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:30.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.210+0000 7f81d4fe2700 1 -- 172.21.15.17:0/3142413028 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c8009470 con 0x7f81d00614b0 2022-08-01T19:34:30.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.210+0000 7f81c7fff700 1 --2- 172.21.15.17:0/3142413028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81d00ad750 0x7f81d00b0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:34:30.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.210+0000 7f81d4fe2700 1 --2- 172.21.15.17:0/3142413028 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81d00614b0 0x7f81d00b1170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f81cc000ed0 tx=0x7f81cc007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:30.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.210+0000 7f81c5ffb700 1 -- 172.21.15.17:0/3142413028 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81cc0070d0 con 0x7f81d00614b0 2022-08-01T19:34:30.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.210+0000 7f81c5ffb700 1 -- 172.21.15.17:0/3142413028 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81cc007230 con 0x7f81d00614b0 2022-08-01T19:34:30.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.210+0000 7f81d5fe4700 1 -- 172.21.15.17:0/3142413028 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81c000b180 con 0x7f81d00614b0 2022-08-01T19:34:30.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.210+0000 7f81c5ffb700 1 -- 172.21.15.17:0/3142413028 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81cc005610 con 0x7f81d00614b0 2022-08-01T19:34:30.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.210+0000 7f81d5fe4700 1 -- 172.21.15.17:0/3142413028 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81d0145f60 con 0x7f81d00614b0 2022-08-01T19:34:30.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.214+0000 7f81d5fe4700 1 -- 172.21.15.17:0/3142413028 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81d00b3e20 con 0x7f81d00614b0 2022-08-01T19:34:30.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.214+0000 7f81c5ffb700 1 -- 172.21.15.17:0/3142413028 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f81cc01a060 con 0x7f81d00614b0 2022-08-01T19:34:30.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.214+0000 7f81c5ffb700 1 --2- 172.21.15.17:0/3142413028 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f81b0053950 0x7f81b0055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:30.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.214+0000 7f81c5ffb700 1 -- 172.21.15.17:0/3142413028 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f81cc06b950 con 0x7f81d00614b0 2022-08-01T19:34:30.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.214+0000 7f81c7fff700 1 --2- 172.21.15.17:0/3142413028 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f81b0053950 0x7f81b0055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:30.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.214+0000 7f81c7fff700 1 --2- 172.21.15.17:0/3142413028 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f81b0053950 0x7f81b0055e10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f81c001a000 tx=0x7f81c0014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:30.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.218+0000 7f81c5ffb700 1 -- 172.21.15.17:0/3142413028 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f81d00b3e20 con 0x7f81d00614b0 2022-08-01T19:34:30.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.346+0000 7f81d5fe4700 1 -- 172.21.15.17:0/3142413028 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f81d00b3e20 con 0x7f81d00614b0 2022-08-01T19:34:30.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.346+0000 7f81c5ffb700 1 -- 172.21.15.17:0/3142413028 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+10 (secure 0 0 0) 0x7f81d00b3e20 con 0x7f81d00614b0 2022-08-01T19:34:30.353 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:34:30.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.350+0000 7f81d5fe4700 1 -- 172.21.15.17:0/3142413028 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f81b0053950 msgr2=0x7f81b0055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:30.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.350+0000 7f81d5fe4700 1 --2- 172.21.15.17:0/3142413028 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f81b0053950 0x7f81b0055e10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f81c001a000 tx=0x7f81c0014000).stop 2022-08-01T19:34:30.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.350+0000 7f81d5fe4700 1 -- 172.21.15.17:0/3142413028 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81d00614b0 msgr2=0x7f81d00b1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:30.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.350+0000 7f81d5fe4700 1 --2- 172.21.15.17:0/3142413028 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81d00614b0 0x7f81d00b1170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f81cc000ed0 tx=0x7f81cc007ac0).stop 2022-08-01T19:34:30.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.350+0000 7f81d5fe4700 1 -- 172.21.15.17:0/3142413028 shutdown_connections 2022-08-01T19:34:30.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.350+0000 7f81d5fe4700 1 --2- 172.21.15.17:0/3142413028 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f81b0053950 0x7f81b0055e10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:30.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.350+0000 7f81d5fe4700 1 --2- 172.21.15.17:0/3142413028 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f81d01405d0 0x7f81d00ac420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:30.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.350+0000 7f81d5fe4700 1 --2- 172.21.15.17:0/3142413028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81d00ad750 0x7f81d00b0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:30.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.350+0000 7f81d5fe4700 1 --2- 172.21.15.17:0/3142413028 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81d00614b0 0x7f81d00b1170 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:30.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.350+0000 7f81d5fe4700 1 -- 172.21.15.17:0/3142413028 >> 172.21.15.17:0/3142413028 conn(0x7f81d0066f70 msgr2=0x7f81d00b8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:30.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.350+0000 7f81d5fe4700 1 -- 172.21.15.17:0/3142413028 shutdown_connections 2022-08-01T19:34:30.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.350+0000 7f81d5fe4700 1 -- 172.21.15.17:0/3142413028 wait complete. 2022-08-01T19:34:30.367 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:34:30.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.458+0000 7f9ea24d0700 1 Processor -- start 2022-08-01T19:34:30.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.458+0000 7f9ea24d0700 1 -- start start 2022-08-01T19:34:30.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.462+0000 7f9ea24d0700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9e9c0647d0 0x7f9e9c064bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:30.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.462+0000 7f9ea24d0700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9e9c0b38a0 0x7f9e9c1470d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:30.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.462+0000 7f9ea24d0700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f9e9c1436f0 0x7f9e9c143b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:30.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.462+0000 7f9ea24d0700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e9c0b4f60 con 0x7f9e9c0647d0 2022-08-01T19:34:30.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.462+0000 7f9ea24d0700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e9c1475c0 con 0x7f9e9c1436f0 2022-08-01T19:34:30.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.462+0000 7f9ea24d0700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c0b4a60 con 0x7f9e9c0b38a0 2022-08-01T19:34:30.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.462+0000 7f9ea14ce700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9e9c0647d0 0x7f9e9c064bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:30.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.462+0000 7f9ea14ce700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9e9c0647d0 0x7f9e9c064bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37014/0 (socket says 172.21.15.17:37014) 2022-08-01T19:34:30.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.462+0000 7f9ea14ce700 1 -- 172.21.15.17:0/607079311 learned_addr learned my addr 172.21.15.17:0/607079311 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:30.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.462+0000 7f9ea1ccf700 1 --2- 172.21.15.17:0/607079311 >> v2:172.21.15.77:3300/0 conn(0x7f9e9c1436f0 0x7f9e9c143b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:30.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.462+0000 7f9ea0ccd700 1 --2- 172.21.15.17:0/607079311 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9e9c0b38a0 0x7f9e9c1470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:30.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.462+0000 7f9ea14ce700 1 -- 172.21.15.17:0/607079311 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9e9c0b38a0 msgr2=0x7f9e9c1470d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:30.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.462+0000 7f9ea14ce700 1 --2- 172.21.15.17:0/607079311 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9e9c0b38a0 0x7f9e9c1470d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:30.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.462+0000 7f9ea14ce700 1 -- 172.21.15.17:0/607079311 >> v2:172.21.15.77:3300/0 conn(0x7f9e9c1436f0 msgr2=0x7f9e9c143b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:30.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.462+0000 7f9ea14ce700 1 --2- 172.21.15.17:0/607079311 >> v2:172.21.15.77:3300/0 conn(0x7f9e9c1436f0 0x7f9e9c143b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:30.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.462+0000 7f9ea14ce700 1 -- 172.21.15.17:0/607079311 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9c0b4cc0 con 0x7f9e9c0647d0 2022-08-01T19:34:30.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.462+0000 7f9ea1ccf700 1 --2- 172.21.15.17:0/607079311 >> v2:172.21.15.77:3300/0 conn(0x7f9e9c1436f0 0x7f9e9c143b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:34:30.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.462+0000 7f9ea14ce700 1 --2- 172.21.15.17:0/607079311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e9c0647d0 0x7f9e9c064bb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f9e90002f70 tx=0x7f9e90006eb0).ready entity=mon.0 client_cookie=a501e4a32a8e3690 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:30.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.462+0000 7f9e8ffff700 1 -- 172.21.15.17:0/607079311 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e9000d570 con 0x7f9e9c0647d0 2022-08-01T19:34:30.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.462+0000 7f9e8ffff700 1 -- 172.21.15.17:0/607079311 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e90000e90 con 0x7f9e9c0647d0 2022-08-01T19:34:30.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.462+0000 7f9e8ffff700 1 -- 172.21.15.17:0/607079311 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e9000d900 con 0x7f9e9c0647d0 2022-08-01T19:34:30.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.462+0000 7f9ea24d0700 1 -- 172.21.15.17:0/607079311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e9c0647d0 msgr2=0x7f9e9c064bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:30.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.462+0000 7f9ea24d0700 1 --2- 172.21.15.17:0/607079311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e9c0647d0 0x7f9e9c064bb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f9e90002f70 tx=0x7f9e90006eb0).stop 2022-08-01T19:34:30.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.462+0000 7f9ea24d0700 1 -- 172.21.15.17:0/607079311 shutdown_connections 2022-08-01T19:34:30.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.462+0000 7f9ea24d0700 1 --2- 172.21.15.17:0/607079311 >> v2:172.21.15.77:3300/0 conn(0x7f9e9c1436f0 0x7f9e9c143b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:30.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.462+0000 7f9ea24d0700 1 --2- 172.21.15.17:0/607079311 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9e9c0b38a0 0x7f9e9c1470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:30.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.462+0000 7f9ea24d0700 1 --2- 172.21.15.17:0/607079311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e9c0647d0 0x7f9e9c064bb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:30.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.462+0000 7f9ea24d0700 1 -- 172.21.15.17:0/607079311 >> 172.21.15.17:0/607079311 conn(0x7f9e9c05f840 msgr2=0x7f9e9c05fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:30.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.462+0000 7f9ea24d0700 1 -- 172.21.15.17:0/607079311 shutdown_connections 2022-08-01T19:34:30.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.462+0000 7f9ea24d0700 1 -- 172.21.15.17:0/607079311 wait complete. 2022-08-01T19:34:30.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.466+0000 7f9ea24d0700 1 Processor -- start 2022-08-01T19:34:30.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.466+0000 7f9ea24d0700 1 -- start start 2022-08-01T19:34:30.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.466+0000 7f9ea24d0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e9c0647d0 0x7f9e9c145920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:30.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.466+0000 7f9ea24d0700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9e9c0b38a0 0x7f9e9c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:30.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.466+0000 7f9ea24d0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e9c1436f0 0x7f9e9c0a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:30.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.466+0000 7f9ea24d0700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c0b82f0 con 0x7f9e9c0647d0 2022-08-01T19:34:30.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.466+0000 7f9ea24d0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c0a6670 con 0x7f9e9c1436f0 2022-08-01T19:34:30.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.466+0000 7f9ea24d0700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c0b74c0 con 0x7f9e9c0b38a0 2022-08-01T19:34:30.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.466+0000 7f9ea14ce700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e9c0647d0 0x7f9e9c145920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:30.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.466+0000 7f9ea14ce700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e9c0647d0 0x7f9e9c145920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37018/0 (socket says 172.21.15.17:37018) 2022-08-01T19:34:30.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.466+0000 7f9ea14ce700 1 -- 172.21.15.17:0/2609084599 learned_addr learned my addr 172.21.15.17:0/2609084599 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:30.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.466+0000 7f9ea0ccd700 1 --2- 172.21.15.17:0/2609084599 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9e9c0b38a0 0x7f9e9c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:30.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.466+0000 7f9ea1ccf700 1 --2- 172.21.15.17:0/2609084599 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e9c1436f0 0x7f9e9c0a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:30.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.466+0000 7f9ea14ce700 1 -- 172.21.15.17:0/2609084599 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9e9c0b38a0 msgr2=0x7f9e9c0a3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:30.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.466+0000 7f9ea14ce700 1 --2- 172.21.15.17:0/2609084599 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9e9c0b38a0 0x7f9e9c0a3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:30.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.466+0000 7f9ea14ce700 1 -- 172.21.15.17:0/2609084599 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e9c1436f0 msgr2=0x7f9e9c0a3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:30.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.466+0000 7f9ea14ce700 1 --2- 172.21.15.17:0/2609084599 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e9c1436f0 0x7f9e9c0a3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:30.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.466+0000 7f9ea14ce700 1 -- 172.21.15.17:0/2609084599 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9000b1e0 con 0x7f9e9c0647d0 2022-08-01T19:34:30.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.466+0000 7f9ea14ce700 1 --2- 172.21.15.17:0/2609084599 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e9c0647d0 0x7f9e9c145920 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f9e90002f70 tx=0x7f9e90015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:30.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.466+0000 7f9e8e7fc700 1 -- 172.21.15.17:0/2609084599 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e900118a0 con 0x7f9e9c0647d0 2022-08-01T19:34:30.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.466+0000 7f9e8e7fc700 1 -- 172.21.15.17:0/2609084599 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e90011a00 con 0x7f9e9c0647d0 2022-08-01T19:34:30.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.466+0000 7f9ea24d0700 1 -- 172.21.15.17:0/2609084599 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e9c0b7760 con 0x7f9e9c0647d0 2022-08-01T19:34:30.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.466+0000 7f9e8e7fc700 1 -- 172.21.15.17:0/2609084599 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e90011d60 con 0x7f9e9c0647d0 2022-08-01T19:34:30.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.466+0000 7f9ea24d0700 1 -- 172.21.15.17:0/2609084599 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e9c0a63a0 con 0x7f9e9c0647d0 2022-08-01T19:34:30.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.466+0000 7f9ea24d0700 1 -- 172.21.15.17:0/2609084599 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e9c0a3850 con 0x7f9e9c0647d0 2022-08-01T19:34:30.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.470+0000 7f9e8e7fc700 1 -- 172.21.15.17:0/2609084599 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9e90025030 con 0x7f9e9c0647d0 2022-08-01T19:34:30.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.470+0000 7f9e8e7fc700 1 --2- 172.21.15.17:0/2609084599 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9e7c053a10 0x7f9e7c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:30.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.470+0000 7f9e8e7fc700 1 -- 172.21.15.17:0/2609084599 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f9e9006d2c0 con 0x7f9e9c0647d0 2022-08-01T19:34:30.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.470+0000 7f9ea0ccd700 1 --2- 172.21.15.17:0/2609084599 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9e7c053a10 0x7f9e7c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:30.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.470+0000 7f9ea0ccd700 1 --2- 172.21.15.17:0/2609084599 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9e7c053a10 0x7f9e7c055ed0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f9e9800c000 tx=0x7f9e98008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:30.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.474+0000 7f9e8e7fc700 1 -- 172.21.15.17:0/2609084599 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9e90044000 con 0x7f9e9c0647d0 2022-08-01T19:34:30.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.598+0000 7f9ea24d0700 1 -- 172.21.15.17:0/2609084599 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e9c143250 con 0x7f9e9c0647d0 2022-08-01T19:34:30.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.598+0000 7f9e8e7fc700 1 -- 172.21.15.17:0/2609084599 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f9e90023090 con 0x7f9e9c0647d0 2022-08-01T19:34:30.607 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:30.607 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:34:09.692404+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:34:04.635473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-08-01T19:34:07.409434+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:34:30.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.602+0000 7f9ea24d0700 1 -- 172.21.15.17:0/2609084599 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9e7c053a10 msgr2=0x7f9e7c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:30.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.602+0000 7f9ea24d0700 1 --2- 172.21.15.17:0/2609084599 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9e7c053a10 0x7f9e7c055ed0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f9e9800c000 tx=0x7f9e98008040).stop 2022-08-01T19:34:30.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.602+0000 7f9ea24d0700 1 -- 172.21.15.17:0/2609084599 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e9c0647d0 msgr2=0x7f9e9c145920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:30.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.602+0000 7f9ea24d0700 1 --2- 172.21.15.17:0/2609084599 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e9c0647d0 0x7f9e9c145920 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f9e90002f70 tx=0x7f9e90015000).stop 2022-08-01T19:34:30.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.602+0000 7f9ea24d0700 1 -- 172.21.15.17:0/2609084599 shutdown_connections 2022-08-01T19:34:30.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.602+0000 7f9ea24d0700 1 --2- 172.21.15.17:0/2609084599 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9e7c053a10 0x7f9e7c055ed0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:30.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.602+0000 7f9ea24d0700 1 --2- 172.21.15.17:0/2609084599 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e9c1436f0 0x7f9e9c0a3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:30.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.602+0000 7f9ea24d0700 1 --2- 172.21.15.17:0/2609084599 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9e9c0b38a0 0x7f9e9c0a3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:30.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.602+0000 7f9ea24d0700 1 --2- 172.21.15.17:0/2609084599 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e9c0647d0 0x7f9e9c145920 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:30.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.602+0000 7f9ea24d0700 1 -- 172.21.15.17:0/2609084599 >> 172.21.15.17:0/2609084599 conn(0x7f9e9c05f840 msgr2=0x7f9e9c061ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:30.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.602+0000 7f9ea24d0700 1 -- 172.21.15.17:0/2609084599 shutdown_connections 2022-08-01T19:34:30.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.602+0000 7f9ea24d0700 1 -- 172.21.15.17:0/2609084599 wait complete. 2022-08-01T19:34:30.618 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T19:34:04.635473+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T19:34:07.409434+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T19:34:30.619 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:34:30.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.726+0000 7f71c88e4700 1 Processor -- start 2022-08-01T19:34:30.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.726+0000 7f71c88e4700 1 -- start start 2022-08-01T19:34:30.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.726+0000 7f71c88e4700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f71c40b2710 0x7f71c414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:30.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.726+0000 7f71c88e4700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71c40b01a0 0x7f71c414b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:30.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.726+0000 7f71c88e4700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f71c40afd60 0x7f71c40a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:30.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.726+0000 7f71c88e4700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71c40a7250 con 0x7f71c40b2710 2022-08-01T19:34:30.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.726+0000 7f71c88e4700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71c40a7480 con 0x7f71c40afd60 2022-08-01T19:34:30.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.726+0000 7f71c88e4700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c40a7720 con 0x7f71c40b01a0 2022-08-01T19:34:30.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.726+0000 7f71c37fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f71c40b2710 0x7f71c414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:30.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.726+0000 7f71c37fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f71c40b2710 0x7f71c414afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37026/0 (socket says 172.21.15.17:37026) 2022-08-01T19:34:30.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.726+0000 7f71c37fe700 1 -- 172.21.15.17:0/4204519613 learned_addr learned my addr 172.21.15.17:0/4204519613 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.726+0000 7f71c2ffd700 1 --2- 172.21.15.17:0/4204519613 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71c40b01a0 0x7f71c414b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.726+0000 7f71c3fff700 1 --2- 172.21.15.17:0/4204519613 >> v2:172.21.15.77:3300/0 conn(0x7f71c40afd60 0x7f71c40a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.726+0000 7f71c37fe700 1 -- 172.21.15.17:0/4204519613 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71c40b01a0 msgr2=0x7f71c414b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.726+0000 7f71c37fe700 1 --2- 172.21.15.17:0/4204519613 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71c40b01a0 0x7f71c414b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.726+0000 7f71c37fe700 1 -- 172.21.15.17:0/4204519613 >> v2:172.21.15.77:3300/0 conn(0x7f71c40afd60 msgr2=0x7f71c40a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:30.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.726+0000 7f71c37fe700 1 --2- 172.21.15.17:0/4204519613 >> v2:172.21.15.77:3300/0 conn(0x7f71c40afd60 0x7f71c40a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:30.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.726+0000 7f71c37fe700 1 -- 172.21.15.17:0/4204519613 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c40a79c0 con 0x7f71c40b2710 2022-08-01T19:34:30.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.726+0000 7f71c37fe700 1 --2- 172.21.15.17:0/4204519613 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71c40b2710 0x7f71c414afd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f71b4002f70 tx=0x7f71b4006eb0).ready entity=mon.0 client_cookie=40849b8001e8bbc7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:30.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.726+0000 7f71c27fc700 1 -- 172.21.15.17:0/4204519613 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71b400d570 con 0x7f71c40b2710 2022-08-01T19:34:30.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.726+0000 7f71c27fc700 1 -- 172.21.15.17:0/4204519613 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b4000e90 con 0x7f71c40b2710 2022-08-01T19:34:30.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.726+0000 7f71c27fc700 1 -- 172.21.15.17:0/4204519613 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71b400d900 con 0x7f71c40b2710 2022-08-01T19:34:30.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.726+0000 7f71c88e4700 1 -- 172.21.15.17:0/4204519613 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71c40b2710 msgr2=0x7f71c414afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:30.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.726+0000 7f71c88e4700 1 --2- 172.21.15.17:0/4204519613 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71c40b2710 0x7f71c414afd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f71b4002f70 tx=0x7f71b4006eb0).stop 2022-08-01T19:34:30.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.726+0000 7f71c88e4700 1 -- 172.21.15.17:0/4204519613 shutdown_connections 2022-08-01T19:34:30.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.726+0000 7f71c88e4700 1 --2- 172.21.15.17:0/4204519613 >> v2:172.21.15.77:3300/0 conn(0x7f71c40afd60 0x7f71c40a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:30.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.726+0000 7f71c88e4700 1 --2- 172.21.15.17:0/4204519613 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71c40b01a0 0x7f71c414b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:30.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.726+0000 7f71c88e4700 1 --2- 172.21.15.17:0/4204519613 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71c40b2710 0x7f71c414afd0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:30.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.726+0000 7f71c88e4700 1 -- 172.21.15.17:0/4204519613 >> 172.21.15.17:0/4204519613 conn(0x7f71c4060310 msgr2=0x7f71c40b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:30.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.726+0000 7f71c88e4700 1 -- 172.21.15.17:0/4204519613 shutdown_connections 2022-08-01T19:34:30.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.726+0000 7f71c88e4700 1 -- 172.21.15.17:0/4204519613 wait complete. 2022-08-01T19:34:30.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.730+0000 7f71c88e4700 1 Processor -- start 2022-08-01T19:34:30.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.730+0000 7f71c88e4700 1 -- start start 2022-08-01T19:34:30.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.730+0000 7f71c88e4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71c40afd60 0x7f71c40af140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:30.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.730+0000 7f71c88e4700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71c40b01a0 0x7f71c40aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:30.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.730+0000 7f71c88e4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71c40b2710 0x7f71c40aaf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:30.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.730+0000 7f71c88e4700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c41428c0 con 0x7f71c40afd60 2022-08-01T19:34:30.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.730+0000 7f71c88e4700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c40b39c0 con 0x7f71c40b2710 2022-08-01T19:34:30.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.730+0000 7f71c88e4700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c40b3c60 con 0x7f71c40b01a0 2022-08-01T19:34:30.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.730+0000 7f71c37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71c40afd60 0x7f71c40af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:30.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.730+0000 7f71c37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71c40afd60 0x7f71c40af140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37032/0 (socket says 172.21.15.17:37032) 2022-08-01T19:34:30.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.730+0000 7f71c37fe700 1 -- 172.21.15.17:0/3365029911 learned_addr learned my addr 172.21.15.17:0/3365029911 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:30.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.730+0000 7f71c3fff700 1 --2- 172.21.15.17:0/3365029911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71c40b2710 0x7f71c40aaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:30.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.730+0000 7f71c2ffd700 1 --2- 172.21.15.17:0/3365029911 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71c40b01a0 0x7f71c40aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:30.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.730+0000 7f71c37fe700 1 -- 172.21.15.17:0/3365029911 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71c40b01a0 msgr2=0x7f71c40aaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:30.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.730+0000 7f71c37fe700 1 --2- 172.21.15.17:0/3365029911 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71c40b01a0 0x7f71c40aaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:30.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.730+0000 7f71c37fe700 1 -- 172.21.15.17:0/3365029911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71c40b2710 msgr2=0x7f71c40aaf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:30.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.730+0000 7f71c37fe700 1 --2- 172.21.15.17:0/3365029911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71c40b2710 0x7f71c40aaf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:30.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.730+0000 7f71c37fe700 1 -- 172.21.15.17:0/3365029911 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b400b1e0 con 0x7f71c40afd60 2022-08-01T19:34:30.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.730+0000 7f71c3fff700 1 --2- 172.21.15.17:0/3365029911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71c40b2710 0x7f71c40aaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:34:30.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.730+0000 7f71c37fe700 1 --2- 172.21.15.17:0/3365029911 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71c40afd60 0x7f71c40af140 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f71b4002f70 tx=0x7f71b4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:30.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.730+0000 7f71c0ff9700 1 -- 172.21.15.17:0/3365029911 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71b4011990 con 0x7f71c40afd60 2022-08-01T19:34:30.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.730+0000 7f71c0ff9700 1 -- 172.21.15.17:0/3365029911 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b4011af0 con 0x7f71c40afd60 2022-08-01T19:34:30.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.730+0000 7f71c88e4700 1 -- 172.21.15.17:0/3365029911 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c40b3f00 con 0x7f71c40afd60 2022-08-01T19:34:30.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.730+0000 7f71c0ff9700 1 -- 172.21.15.17:0/3365029911 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71b4011e50 con 0x7f71c40afd60 2022-08-01T19:34:30.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.730+0000 7f71c88e4700 1 -- 172.21.15.17:0/3365029911 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c40b2f60 con 0x7f71c40afd60 2022-08-01T19:34:30.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.730+0000 7f71c88e4700 1 -- 172.21.15.17:0/3365029911 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71c40b3650 con 0x7f71c40afd60 2022-08-01T19:34:30.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.734+0000 7f71c0ff9700 1 -- 172.21.15.17:0/3365029911 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f71b4025030 con 0x7f71c40afd60 2022-08-01T19:34:30.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.734+0000 7f71c0ff9700 1 --2- 172.21.15.17:0/3365029911 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f71a4053a70 0x7f71a4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:30.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.734+0000 7f71c0ff9700 1 -- 172.21.15.17:0/3365029911 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f71b406d470 con 0x7f71c40afd60 2022-08-01T19:34:30.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.734+0000 7f71c2ffd700 1 --2- 172.21.15.17:0/3365029911 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f71a4053a70 0x7f71a4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:30.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.734+0000 7f71c2ffd700 1 --2- 172.21.15.17:0/3365029911 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f71a4053a70 0x7f71a4055f30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f71b800c000 tx=0x7f71b8008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:30.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.738+0000 7f71c0ff9700 1 -- 172.21.15.17:0/3365029911 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f71b40441b0 con 0x7f71c40afd60 2022-08-01T19:34:30.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.862+0000 7f71c88e4700 1 -- 172.21.15.17:0/3365029911 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f71c40b4090 con 0x7f71c40afd60 2022-08-01T19:34:30.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.862+0000 7f71c0ff9700 1 -- 172.21.15.17:0/3365029911 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v25) v1 ==== 110+0+10 (secure 0 0 0) 0x7f71b40230b0 con 0x7f71c40afd60 2022-08-01T19:34:30.869 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:34:30.871 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.866+0000 7f71c88e4700 1 -- 172.21.15.17:0/3365029911 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f71a4053a70 msgr2=0x7f71a4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:30.871 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.866+0000 7f71c88e4700 1 --2- 172.21.15.17:0/3365029911 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f71a4053a70 0x7f71a4055f30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f71b800c000 tx=0x7f71b8008040).stop 2022-08-01T19:34:30.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.866+0000 7f71c88e4700 1 -- 172.21.15.17:0/3365029911 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71c40afd60 msgr2=0x7f71c40af140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:30.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.866+0000 7f71c88e4700 1 --2- 172.21.15.17:0/3365029911 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71c40afd60 0x7f71c40af140 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f71b4002f70 tx=0x7f71b4015000).stop 2022-08-01T19:34:30.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.866+0000 7f71c88e4700 1 -- 172.21.15.17:0/3365029911 shutdown_connections 2022-08-01T19:34:30.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.866+0000 7f71c88e4700 1 --2- 172.21.15.17:0/3365029911 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f71a4053a70 0x7f71a4055f30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:30.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.866+0000 7f71c88e4700 1 --2- 172.21.15.17:0/3365029911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71c40b2710 0x7f71c40aaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:30.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.866+0000 7f71c88e4700 1 --2- 172.21.15.17:0/3365029911 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71c40b01a0 0x7f71c40aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:30.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.866+0000 7f71c88e4700 1 --2- 172.21.15.17:0/3365029911 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71c40afd60 0x7f71c40af140 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:30.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.866+0000 7f71c88e4700 1 -- 172.21.15.17:0/3365029911 >> 172.21.15.17:0/3365029911 conn(0x7f71c4060310 msgr2=0x7f71c41498c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:30.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.866+0000 7f71c88e4700 1 -- 172.21.15.17:0/3365029911 shutdown_connections 2022-08-01T19:34:30.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.866+0000 7f71c88e4700 1 -- 172.21.15.17:0/3365029911 wait complete. 2022-08-01T19:34:30.882 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:34:30.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.990+0000 7f09b29e3700 1 Processor -- start 2022-08-01T19:34:30.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.990+0000 7f09b29e3700 1 -- start start 2022-08-01T19:34:30.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.990+0000 7f09b29e3700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f09ac0b2710 0x7f09ac14afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:30.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.990+0000 7f09b29e3700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09ac0a7250 con 0x7f09ac0a8c50 2022-08-01T19:34:30.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.990+0000 7f09b29e3700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09ac0a7480 con 0x7f09ac0b01a0 2022-08-01T19:34:30.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.990+0000 7f09b29e3700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09ac0a7720 con 0x7f09ac0b2710 2022-08-01T19:34:30.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.990+0000 7f09b19e1700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f09ac0b2710 0x7f09ac14afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:30.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.990+0000 7f09b21e2700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f09ac0a8c50 0x7f09ac151d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57470/0 (socket says 172.21.15.17:57470) 2022-08-01T19:34:30.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.990+0000 7f09b21e2700 1 -- 172.21.15.17:0/553357076 learned_addr learned my addr 172.21.15.17:0/553357076 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:30.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.990+0000 7f09b19e1700 1 -- 172.21.15.17:0/553357076 >> v1:172.21.15.77:6789/0 conn(0x7f09ac0b01a0 legacy=0x7f09ac0a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:31.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.990+0000 7f09b19e1700 1 -- 172.21.15.17:0/553357076 >> v1:172.21.15.17:6789/0 conn(0x7f09ac0a8c50 legacy=0x7f09ac151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:31.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.990+0000 7f09b19e1700 1 -- 172.21.15.17:0/553357076 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ac0a79c0 con 0x7f09ac0b2710 2022-08-01T19:34:31.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.990+0000 7f09b19e1700 1 --2- 172.21.15.17:0/553357076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09ac0b2710 0x7f09ac14afd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f09a80032f0 tx=0x7f09a80091d0).ready entity=mon.0 client_cookie=77593cada9da56f7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:31.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.990+0000 7f09b09df700 1 -- 172.21.15.17:0/553357076 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09a8009ea0 con 0x7f09ac0b2710 2022-08-01T19:34:31.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.990+0000 7f09b09df700 1 -- 172.21.15.17:0/553357076 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a800b8a0 con 0x7f09ac0b2710 2022-08-01T19:34:31.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.990+0000 7f09b09df700 1 -- 172.21.15.17:0/553357076 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09a800ba10 con 0x7f09ac0b2710 2022-08-01T19:34:31.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.994+0000 7f09b29e3700 1 -- 172.21.15.17:0/553357076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09ac0b2710 msgr2=0x7f09ac14afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:31.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.994+0000 7f09b29e3700 1 --2- 172.21.15.17:0/553357076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09ac0b2710 0x7f09ac14afd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f09a80032f0 tx=0x7f09a80091d0).stop 2022-08-01T19:34:31.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.994+0000 7f09b29e3700 1 -- 172.21.15.17:0/553357076 shutdown_connections 2022-08-01T19:34:31.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.994+0000 7f09b29e3700 1 --2- 172.21.15.17:0/553357076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09ac0b2710 0x7f09ac14afd0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:31.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.994+0000 7f09b29e3700 1 -- 172.21.15.17:0/553357076 >> 172.21.15.17:0/553357076 conn(0x7f09ac060310 msgr2=0x7f09ac0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:31.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.994+0000 7f09b29e3700 1 -- 172.21.15.17:0/553357076 shutdown_connections 2022-08-01T19:34:31.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.994+0000 7f09b29e3700 1 -- 172.21.15.17:0/553357076 wait complete. 2022-08-01T19:34:31.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.994+0000 7f09b29e3700 1 Processor -- start 2022-08-01T19:34:31.007 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.994+0000 7f09b29e3700 1 -- start start 2022-08-01T19:34:31.007 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.998+0000 7f09b29e3700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09ac0a8c50 0x7f09ac0b3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:31.007 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.998+0000 7f09b29e3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09ac0b01a0 0x7f09ac15a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:31.007 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.998+0000 7f09b29e3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09ac0b2710 0x7f09ac15a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:31.008 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.998+0000 7f09b29e3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac063c80 con 0x7f09ac0b01a0 2022-08-01T19:34:31.008 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.998+0000 7f09b29e3700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac0a9650 con 0x7f09ac0b2710 2022-08-01T19:34:31.008 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.998+0000 7f09b29e3700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac0a98f0 con 0x7f09ac0a8c50 2022-08-01T19:34:31.008 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.998+0000 7f09b11e0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09ac0b01a0 0x7f09ac15a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:31.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.998+0000 7f09b11e0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09ac0b01a0 0x7f09ac15a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37048/0 (socket says 172.21.15.17:37048) 2022-08-01T19:34:31.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.998+0000 7f09b11e0700 1 -- 172.21.15.17:0/4079002582 learned_addr learned my addr 172.21.15.17:0/4079002582 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:31.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.998+0000 7f09b21e2700 1 --2- 172.21.15.17:0/4079002582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09ac0b2710 0x7f09ac15a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:31.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.998+0000 7f09b19e1700 1 --2- 172.21.15.17:0/4079002582 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09ac0a8c50 0x7f09ac0b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:31.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.998+0000 7f09b11e0700 1 -- 172.21.15.17:0/4079002582 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09ac0a8c50 msgr2=0x7f09ac0b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:31.010 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.998+0000 7f09b11e0700 1 --2- 172.21.15.17:0/4079002582 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09ac0a8c50 0x7f09ac0b3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:31.010 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.998+0000 7f09b11e0700 1 -- 172.21.15.17:0/4079002582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09ac0b2710 msgr2=0x7f09ac15a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:31.010 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.998+0000 7f09b11e0700 1 --2- 172.21.15.17:0/4079002582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09ac0b2710 0x7f09ac15a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:31.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.998+0000 7f09b11e0700 1 -- 172.21.15.17:0/4079002582 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a8009470 con 0x7f09ac0b01a0 2022-08-01T19:34:31.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.998+0000 7f09b11e0700 1 --2- 172.21.15.17:0/4079002582 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09ac0b01a0 0x7f09ac15a2d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f09ac061c10 tx=0x7f09a0007bf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:31.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.998+0000 7f099effd700 1 -- 172.21.15.17:0/4079002582 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09a0012400 con 0x7f09ac0b01a0 2022-08-01T19:34:31.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.998+0000 7f099effd700 1 -- 172.21.15.17:0/4079002582 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a0012560 con 0x7f09ac0b01a0 2022-08-01T19:34:31.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.998+0000 7f09b29e3700 1 -- 172.21.15.17:0/4079002582 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09ac159e30 con 0x7f09ac0b01a0 2022-08-01T19:34:31.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.998+0000 7f099effd700 1 -- 172.21.15.17:0/4079002582 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09a00128c0 con 0x7f09ac0b01a0 2022-08-01T19:34:31.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.998+0000 7f09b29e3700 1 -- 172.21.15.17:0/4079002582 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09ac0b3f40 con 0x7f09ac0b01a0 2022-08-01T19:34:31.013 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:30.998+0000 7f09b29e3700 1 -- 172.21.15.17:0/4079002582 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09ac0a9ab0 con 0x7f09ac0b01a0 2022-08-01T19:34:31.013 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.002+0000 7f099effd700 1 -- 172.21.15.17:0/4079002582 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f09a0019030 con 0x7f09ac0b01a0 2022-08-01T19:34:31.014 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.002+0000 7f099effd700 1 --2- 172.21.15.17:0/4079002582 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f098c053b00 0x7f098c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:31.014 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.002+0000 7f099effd700 1 -- 172.21.15.17:0/4079002582 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f09a006b300 con 0x7f09ac0b01a0 2022-08-01T19:34:31.014 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.002+0000 7f09b19e1700 1 --2- 172.21.15.17:0/4079002582 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f098c053b00 0x7f098c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:31.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.002+0000 7f09b19e1700 1 --2- 172.21.15.17:0/4079002582 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f098c053b00 0x7f098c055fc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f09a801b000 tx=0x7f09a8016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:31.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.006+0000 7f099effd700 1 -- 172.21.15.17:0/4079002582 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f09a0042060 con 0x7f09ac0b01a0 2022-08-01T19:34:31.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.138+0000 7f09b29e3700 1 -- 172.21.15.17:0/4079002582 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f09ac159c40 con 0x7f09ac0b01a0 2022-08-01T19:34:31.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.138+0000 7f099effd700 1 -- 172.21.15.17:0/4079002582 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f09a0045680 con 0x7f09ac0b01a0 2022-08-01T19:34:31.143 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:34:31.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.138+0000 7f09b29e3700 1 -- 172.21.15.17:0/4079002582 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f098c053b00 msgr2=0x7f098c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:31.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.138+0000 7f09b29e3700 1 --2- 172.21.15.17:0/4079002582 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f098c053b00 0x7f098c055fc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f09a801b000 tx=0x7f09a8016000).stop 2022-08-01T19:34:31.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.138+0000 7f09b29e3700 1 -- 172.21.15.17:0/4079002582 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09ac0b01a0 msgr2=0x7f09ac15a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:31.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.138+0000 7f09b29e3700 1 --2- 172.21.15.17:0/4079002582 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09ac0b01a0 0x7f09ac15a2d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f09ac061c10 tx=0x7f09a0007bf0).stop 2022-08-01T19:34:31.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.138+0000 7f09b29e3700 1 -- 172.21.15.17:0/4079002582 shutdown_connections 2022-08-01T19:34:31.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.138+0000 7f09b29e3700 1 --2- 172.21.15.17:0/4079002582 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f098c053b00 0x7f098c055fc0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:31.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.138+0000 7f09b29e3700 1 --2- 172.21.15.17:0/4079002582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09ac0b2710 0x7f09ac15a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:31.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.138+0000 7f09b29e3700 1 --2- 172.21.15.17:0/4079002582 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09ac0b01a0 0x7f09ac15a2d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:31.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.138+0000 7f09b29e3700 1 --2- 172.21.15.17:0/4079002582 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09ac0a8c50 0x7f09ac0b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:31.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.138+0000 7f09b29e3700 1 -- 172.21.15.17:0/4079002582 >> 172.21.15.17:0/4079002582 conn(0x7f09ac060310 msgr2=0x7f09ac1467b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:31.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.138+0000 7f09b29e3700 1 -- 172.21.15.17:0/4079002582 shutdown_connections 2022-08-01T19:34:31.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.138+0000 7f09b29e3700 1 -- 172.21.15.17:0/4079002582 wait complete. 2022-08-01T19:34:31.156 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T19:34:31.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.258+0000 7f857d80d700 1 Processor -- start 2022-08-01T19:34:31.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.258+0000 7f857d80d700 1 -- start start 2022-08-01T19:34:31.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.258+0000 7f857d80d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f85780b7500 0x7f85780b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:31.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.258+0000 7f857d80d700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f85780aadb0 0x7f8578064d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:31.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.258+0000 7f857d80d700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85780b2510 con 0x7f85780b31f0 2022-08-01T19:34:31.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.258+0000 7f857d80d700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85781443e0 con 0x7f85780aadb0 2022-08-01T19:34:31.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.258+0000 7f857d80d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85780b3700 con 0x7f85780b7500 2022-08-01T19:34:31.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.258+0000 7f8577fff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f85780b31f0 0x7f857813e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57484/0 (socket says 172.21.15.17:57484) 2022-08-01T19:34:31.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.258+0000 7f8577fff700 1 -- 172.21.15.17:0/3974155710 learned_addr learned my addr 172.21.15.17:0/3974155710 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:31.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.258+0000 7f857c80b700 1 --2- 172.21.15.17:0/3974155710 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f85780b7500 0x7f85780b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:31.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.258+0000 7f857d00c700 1 --2- 172.21.15.17:0/3974155710 >> v2:172.21.15.77:3300/0 conn(0x7f85780aadb0 0x7f8578064d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:31.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.258+0000 7f85777fe700 1 -- 172.21.15.17:0/3974155710 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3210163879 0 0) 0x7f85780b2510 con 0x7f85780b31f0 2022-08-01T19:34:31.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.258+0000 7f85777fe700 1 -- 172.21.15.17:0/3974155710 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8558003260 con 0x7f85780b31f0 2022-08-01T19:34:31.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.258+0000 7f857c80b700 1 -- 172.21.15.17:0/3974155710 >> v2:172.21.15.77:3300/0 conn(0x7f85780aadb0 msgr2=0x7f8578064d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:31.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.258+0000 7f857c80b700 1 --2- 172.21.15.17:0/3974155710 >> v2:172.21.15.77:3300/0 conn(0x7f85780aadb0 0x7f8578064d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:31.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.258+0000 7f857c80b700 1 -- 172.21.15.17:0/3974155710 >> v1:172.21.15.17:6789/0 conn(0x7f85780b31f0 legacy=0x7f857813e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:31.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.258+0000 7f857c80b700 1 -- 172.21.15.17:0/3974155710 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85780b2740 con 0x7f85780b7500 2022-08-01T19:34:31.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.258+0000 7f857c80b700 1 --2- 172.21.15.17:0/3974155710 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f85780b7500 0x7f85780b78e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f8560002f70 tx=0x7f8560006d90).ready entity=mon.2 client_cookie=ea355f478332e38b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:31.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.258+0000 7f85777fe700 1 -- 172.21.15.17:0/3974155710 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f856000e590 con 0x7f85780b7500 2022-08-01T19:34:31.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.258+0000 7f85777fe700 1 -- 172.21.15.17:0/3974155710 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8560000e90 con 0x7f85780b7500 2022-08-01T19:34:31.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.258+0000 7f85777fe700 1 -- 172.21.15.17:0/3974155710 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f856000e8e0 con 0x7f85780b7500 2022-08-01T19:34:31.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.258+0000 7f857d80d700 1 -- 172.21.15.17:0/3974155710 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f85780b7500 msgr2=0x7f85780b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:31.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.258+0000 7f857d80d700 1 --2- 172.21.15.17:0/3974155710 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f85780b7500 0x7f85780b78e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f8560002f70 tx=0x7f8560006d90).stop 2022-08-01T19:34:31.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.258+0000 7f857d80d700 1 -- 172.21.15.17:0/3974155710 shutdown_connections 2022-08-01T19:34:31.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.258+0000 7f857d80d700 1 --2- 172.21.15.17:0/3974155710 >> v2:172.21.15.77:3300/0 conn(0x7f85780aadb0 0x7f8578064d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:31.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.258+0000 7f857d80d700 1 --2- 172.21.15.17:0/3974155710 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f85780b7500 0x7f85780b78e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:31.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.258+0000 7f857d80d700 1 -- 172.21.15.17:0/3974155710 >> 172.21.15.17:0/3974155710 conn(0x7f857805f840 msgr2=0x7f857805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:31.271 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.258+0000 7f857d80d700 1 -- 172.21.15.17:0/3974155710 shutdown_connections 2022-08-01T19:34:31.271 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.258+0000 7f857d80d700 1 -- 172.21.15.17:0/3974155710 wait complete. 2022-08-01T19:34:31.271 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.262+0000 7f857d80d700 1 Processor -- start 2022-08-01T19:34:31.271 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.262+0000 7f857d80d700 1 -- start start 2022-08-01T19:34:31.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.262+0000 7f857d80d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85780aadb0 0x7f857813bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:31.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.262+0000 7f857d80d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f85780b31f0 0x7f85780b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:31.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.262+0000 7f857d80d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85780b7500 0x7f85780b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:31.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.262+0000 7f857d80d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857813c890 con 0x7f85780b7500 2022-08-01T19:34:31.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.262+0000 7f857d80d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857813cba0 con 0x7f85780aadb0 2022-08-01T19:34:31.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.262+0000 7f857d80d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85780b63c0 con 0x7f85780b31f0 2022-08-01T19:34:31.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.262+0000 7f857d00c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85780b7500 0x7f85780b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:31.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.262+0000 7f857d00c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85780b7500 0x7f85780b02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37064/0 (socket says 172.21.15.17:37064) 2022-08-01T19:34:31.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.262+0000 7f857d00c700 1 -- 172.21.15.17:0/946965272 learned_addr learned my addr 172.21.15.17:0/946965272 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:31.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.262+0000 7f8577fff700 1 --2- 172.21.15.17:0/946965272 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f85780b31f0 0x7f85780b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:31.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.262+0000 7f857c80b700 1 --2- 172.21.15.17:0/946965272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85780aadb0 0x7f857813bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:31.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.262+0000 7f857d00c700 1 -- 172.21.15.17:0/946965272 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f85780b31f0 msgr2=0x7f85780b5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:31.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.262+0000 7f857d00c700 1 --2- 172.21.15.17:0/946965272 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f85780b31f0 0x7f85780b5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:31.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.262+0000 7f857d00c700 1 -- 172.21.15.17:0/946965272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85780aadb0 msgr2=0x7f857813bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:31.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.262+0000 7f857d00c700 1 --2- 172.21.15.17:0/946965272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85780aadb0 0x7f857813bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:31.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.262+0000 7f857d00c700 1 -- 172.21.15.17:0/946965272 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f856000b120 con 0x7f85780b7500 2022-08-01T19:34:31.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.262+0000 7f8577fff700 1 --2- 172.21.15.17:0/946965272 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f85780b31f0 0x7f85780b5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:34:31.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.262+0000 7f857d00c700 1 --2- 172.21.15.17:0/946965272 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85780b7500 0x7f85780b02e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f857813c9d0 tx=0x7f85700091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:31.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.262+0000 7f8575ffb700 1 -- 172.21.15.17:0/946965272 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f857000c640 con 0x7f85780b7500 2022-08-01T19:34:31.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.262+0000 7f8575ffb700 1 -- 172.21.15.17:0/946965272 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f857000c7a0 con 0x7f85780b7500 2022-08-01T19:34:31.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.262+0000 7f8575ffb700 1 -- 172.21.15.17:0/946965272 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f857000cb00 con 0x7f85780b7500 2022-08-01T19:34:31.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.262+0000 7f857d80d700 1 -- 172.21.15.17:0/946965272 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f857813ba60 con 0x7f85780b7500 2022-08-01T19:34:31.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.266+0000 7f857d80d700 1 -- 172.21.15.17:0/946965272 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85780b6730 con 0x7f85780b7500 2022-08-01T19:34:31.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.266+0000 7f857d80d700 1 -- 172.21.15.17:0/946965272 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f857813c630 con 0x7f85780b7500 2022-08-01T19:34:31.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.266+0000 7f8575ffb700 1 -- 172.21.15.17:0/946965272 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8570022070 con 0x7f85780b7500 2022-08-01T19:34:31.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.266+0000 7f8575ffb700 1 --2- 172.21.15.17:0/946965272 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f855c053950 0x7f855c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:31.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.266+0000 7f8575ffb700 1 -- 172.21.15.17:0/946965272 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f857006cc10 con 0x7f85780b7500 2022-08-01T19:34:31.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.266+0000 7f857c80b700 1 --2- 172.21.15.17:0/946965272 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f855c053950 0x7f855c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:31.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.270+0000 7f857c80b700 1 --2- 172.21.15.17:0/946965272 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f855c053950 0x7f855c055e10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f856001a000 tx=0x7f8560014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:31.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.274+0000 7f8575ffb700 1 -- 172.21.15.17:0/946965272 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f85700439d0 con 0x7f85780b7500 2022-08-01T19:34:31.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.398+0000 7f857d80d700 1 -- 172.21.15.17:0/946965272 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f85780574e0 con 0x7f85780b7500 2022-08-01T19:34:31.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.402+0000 7f8575ffb700 1 -- 172.21.15.17:0/946965272 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+10 (secure 0 0 0) 0x7f85700290a0 con 0x7f85780b7500 2022-08-01T19:34:31.407 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:34:31.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.402+0000 7f857d80d700 1 -- 172.21.15.17:0/946965272 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f855c053950 msgr2=0x7f855c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:31.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.402+0000 7f857d80d700 1 --2- 172.21.15.17:0/946965272 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f855c053950 0x7f855c055e10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f856001a000 tx=0x7f8560014000).stop 2022-08-01T19:34:31.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.402+0000 7f857d80d700 1 -- 172.21.15.17:0/946965272 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85780b7500 msgr2=0x7f85780b02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:31.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.402+0000 7f857d80d700 1 --2- 172.21.15.17:0/946965272 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85780b7500 0x7f85780b02e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f857813c9d0 tx=0x7f85700091d0).stop 2022-08-01T19:34:31.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.402+0000 7f857d80d700 1 -- 172.21.15.17:0/946965272 shutdown_connections 2022-08-01T19:34:31.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.402+0000 7f857d80d700 1 --2- 172.21.15.17:0/946965272 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f855c053950 0x7f855c055e10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:31.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.402+0000 7f857d80d700 1 --2- 172.21.15.17:0/946965272 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85780b7500 0x7f85780b02e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:31.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.402+0000 7f857d80d700 1 --2- 172.21.15.17:0/946965272 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f85780b31f0 0x7f85780b5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:31.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.402+0000 7f857d80d700 1 --2- 172.21.15.17:0/946965272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85780aadb0 0x7f857813bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:31.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.402+0000 7f857d80d700 1 -- 172.21.15.17:0/946965272 >> 172.21.15.17:0/946965272 conn(0x7f857805f840 msgr2=0x7f857813f740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:31.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.402+0000 7f857d80d700 1 -- 172.21.15.17:0/946965272 shutdown_connections 2022-08-01T19:34:31.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.402+0000 7f857d80d700 1 -- 172.21.15.17:0/946965272 wait complete. 2022-08-01T19:34:31.420 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T19:34:31.530 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.522+0000 7f294c82e700 1 Processor -- start 2022-08-01T19:34:31.530 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.522+0000 7f294c82e700 1 -- start start 2022-08-01T19:34:31.532 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.526+0000 7f294c82e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f29480b01a0 0x7f29480a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:31.532 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.526+0000 7f294c82e700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29480a7250 con 0x7f29480afd60 2022-08-01T19:34:31.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.526+0000 7f294c82e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29480a7480 con 0x7f29480b2710 2022-08-01T19:34:31.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.526+0000 7f294c82e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29480a7720 con 0x7f29480b01a0 2022-08-01T19:34:31.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.526+0000 7f294759e700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f29480afd60 0x7f2948151d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57498/0 (socket says 172.21.15.17:57498) 2022-08-01T19:34:31.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.526+0000 7f294759e700 1 -- 172.21.15.17:0/525617031 learned_addr learned my addr 172.21.15.17:0/525617031 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:31.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.526+0000 7f294659c700 1 --2- 172.21.15.17:0/525617031 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f29480b01a0 0x7f29480a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:31.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.526+0000 7f2945d9b700 1 -- 172.21.15.17:0/525617031 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1058901336 0 0) 0x7f29480a7250 con 0x7f29480afd60 2022-08-01T19:34:31.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.526+0000 7f2945d9b700 1 -- 172.21.15.17:0/525617031 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f292c003260 con 0x7f29480afd60 2022-08-01T19:34:31.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.526+0000 7f294659c700 1 -- 172.21.15.17:0/525617031 >> v1:172.21.15.77:6789/0 conn(0x7f29480b2710 legacy=0x7f294814afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:31.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.526+0000 7f2945d9b700 1 -- 172.21.15.17:0/525617031 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4130638179 0 0) 0x7f29480a7480 con 0x7f29480b2710 2022-08-01T19:34:31.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.526+0000 7f294659c700 1 -- 172.21.15.17:0/525617031 >> v1:172.21.15.17:6789/0 conn(0x7f29480afd60 legacy=0x7f2948151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:31.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.526+0000 7f294659c700 1 -- 172.21.15.17:0/525617031 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29480a79c0 con 0x7f29480b01a0 2022-08-01T19:34:31.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.526+0000 7f294659c700 1 --2- 172.21.15.17:0/525617031 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f29480b01a0 0x7f29480a8540 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f29380032f0 tx=0x7f29380091d0).ready entity=mon.2 client_cookie=c157ab6840111adf server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:31.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.526+0000 7f2945d9b700 1 -- 172.21.15.17:0/525617031 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f293800b6c0 con 0x7f29480b01a0 2022-08-01T19:34:31.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.526+0000 7f2945d9b700 1 -- 172.21.15.17:0/525617031 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293800b820 con 0x7f29480b01a0 2022-08-01T19:34:31.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.526+0000 7f2945d9b700 1 -- 172.21.15.17:0/525617031 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f293800b990 con 0x7f29480b01a0 2022-08-01T19:34:31.540 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.526+0000 7f294c82e700 1 -- 172.21.15.17:0/525617031 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f29480b01a0 msgr2=0x7f29480a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:31.540 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.526+0000 7f294c82e700 1 --2- 172.21.15.17:0/525617031 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f29480b01a0 0x7f29480a8540 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f29380032f0 tx=0x7f29380091d0).stop 2022-08-01T19:34:31.540 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.526+0000 7f294c82e700 1 -- 172.21.15.17:0/525617031 shutdown_connections 2022-08-01T19:34:31.540 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.526+0000 7f294c82e700 1 --2- 172.21.15.17:0/525617031 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f29480b01a0 0x7f29480a8540 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:31.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.526+0000 7f294c82e700 1 -- 172.21.15.17:0/525617031 >> 172.21.15.17:0/525617031 conn(0x7f2948060310 msgr2=0x7f29480b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:31.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.526+0000 7f294c82e700 1 -- 172.21.15.17:0/525617031 shutdown_connections 2022-08-01T19:34:31.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.526+0000 7f294c82e700 1 -- 172.21.15.17:0/525617031 wait complete. 2022-08-01T19:34:31.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.530+0000 7f294c82e700 1 Processor -- start 2022-08-01T19:34:31.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.530+0000 7f294c82e700 1 -- start start 2022-08-01T19:34:31.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.530+0000 7f294c82e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f29480afd60 0x7f29480b4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:31.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.530+0000 7f294c82e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29480b01a0 0x7f29480aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:31.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.530+0000 7f294c82e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f29480b2710 0x7f29481415c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:31.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.530+0000 7f294c82e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29480b5dc0 con 0x7f29480afd60 2022-08-01T19:34:31.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.530+0000 7f294c82e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29480b5fd0 con 0x7f29480b01a0 2022-08-01T19:34:31.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.530+0000 7f294c82e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29480b51e0 con 0x7f29480b2710 2022-08-01T19:34:31.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.530+0000 7f2946d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f29480afd60 0x7f29480b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:31.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.530+0000 7f2946d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f29480afd60 0x7f29480b4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37076/0 (socket says 172.21.15.17:37076) 2022-08-01T19:34:31.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.530+0000 7f2946d9d700 1 -- 172.21.15.17:0/869952990 learned_addr learned my addr 172.21.15.17:0/869952990 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:31.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.530+0000 7f294759e700 1 --2- 172.21.15.17:0/869952990 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f29480b2710 0x7f29481415c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:31.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.530+0000 7f294659c700 1 --2- 172.21.15.17:0/869952990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29480b01a0 0x7f29480aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:31.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.530+0000 7f2946d9d700 1 -- 172.21.15.17:0/869952990 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f29480b2710 msgr2=0x7f29481415c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:31.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.530+0000 7f2946d9d700 1 --2- 172.21.15.17:0/869952990 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f29480b2710 0x7f29481415c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:31.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.530+0000 7f2946d9d700 1 -- 172.21.15.17:0/869952990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29480b01a0 msgr2=0x7f29480aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:31.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.530+0000 7f2946d9d700 1 --2- 172.21.15.17:0/869952990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29480b01a0 0x7f29480aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:31.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.530+0000 7f2946d9d700 1 -- 172.21.15.17:0/869952990 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2938009470 con 0x7f29480afd60 2022-08-01T19:34:31.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.530+0000 7f2946d9d700 1 --2- 172.21.15.17:0/869952990 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f29480afd60 0x7f29480b4900 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f293c000bc0 tx=0x7f293c007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:31.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.530+0000 7f2937fff700 1 -- 172.21.15.17:0/869952990 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f293c001d70 con 0x7f29480afd60 2022-08-01T19:34:31.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.530+0000 7f2937fff700 1 -- 172.21.15.17:0/869952990 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293c007030 con 0x7f29480afd60 2022-08-01T19:34:31.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.530+0000 7f294c82e700 1 -- 172.21.15.17:0/869952990 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29480b5480 con 0x7f29480afd60 2022-08-01T19:34:31.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.530+0000 7f2937fff700 1 -- 172.21.15.17:0/869952990 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f293c007390 con 0x7f29480afd60 2022-08-01T19:34:31.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.530+0000 7f294c82e700 1 -- 172.21.15.17:0/869952990 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29480b4500 con 0x7f29480afd60 2022-08-01T19:34:31.547 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.534+0000 7f294c82e700 1 -- 172.21.15.17:0/869952990 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29480b6190 con 0x7f29480afd60 2022-08-01T19:34:31.547 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.534+0000 7f2937fff700 1 -- 172.21.15.17:0/869952990 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f293c010030 con 0x7f29480afd60 2022-08-01T19:34:31.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.534+0000 7f2937fff700 1 --2- 172.21.15.17:0/869952990 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2924053a70 0x7f2924055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:31.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.534+0000 7f2937fff700 1 -- 172.21.15.17:0/869952990 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f293c06bca0 con 0x7f29480afd60 2022-08-01T19:34:31.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.534+0000 7f294659c700 1 --2- 172.21.15.17:0/869952990 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2924053a70 0x7f2924055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:31.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.534+0000 7f294659c700 1 --2- 172.21.15.17:0/869952990 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2924053a70 0x7f2924055f30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f2938019000 tx=0x7f2938016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:31.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.538+0000 7f2937fff700 1 -- 172.21.15.17:0/869952990 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f293c042950 con 0x7f29480afd60 2022-08-01T19:34:31.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.666+0000 7f294c82e700 1 -- 172.21.15.17:0/869952990 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f29480b5020 con 0x7f29480afd60 2022-08-01T19:34:31.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.666+0000 7f2937fff700 1 -- 172.21.15.17:0/869952990 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+10 (secure 0 0 0) 0x7f293c045f70 con 0x7f29480afd60 2022-08-01T19:34:31.672 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:34:31.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.670+0000 7f294c82e700 1 -- 172.21.15.17:0/869952990 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2924053a70 msgr2=0x7f2924055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:31.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.670+0000 7f294c82e700 1 --2- 172.21.15.17:0/869952990 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2924053a70 0x7f2924055f30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f2938019000 tx=0x7f2938016000).stop 2022-08-01T19:34:31.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.670+0000 7f294c82e700 1 -- 172.21.15.17:0/869952990 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f29480afd60 msgr2=0x7f29480b4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:31.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.670+0000 7f294c82e700 1 --2- 172.21.15.17:0/869952990 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f29480afd60 0x7f29480b4900 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f293c000bc0 tx=0x7f293c007c30).stop 2022-08-01T19:34:31.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.670+0000 7f294c82e700 1 -- 172.21.15.17:0/869952990 shutdown_connections 2022-08-01T19:34:31.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.670+0000 7f294c82e700 1 --2- 172.21.15.17:0/869952990 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2924053a70 0x7f2924055f30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:31.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.670+0000 7f294c82e700 1 --2- 172.21.15.17:0/869952990 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f29480b2710 0x7f29481415c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:31.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.670+0000 7f294c82e700 1 --2- 172.21.15.17:0/869952990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29480b01a0 0x7f29480aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:31.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.670+0000 7f294c82e700 1 --2- 172.21.15.17:0/869952990 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f29480afd60 0x7f29480b4900 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:31.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.670+0000 7f294c82e700 1 -- 172.21.15.17:0/869952990 >> 172.21.15.17:0/869952990 conn(0x7f2948060310 msgr2=0x7f29481497b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:31.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.670+0000 7f294c82e700 1 -- 172.21.15.17:0/869952990 shutdown_connections 2022-08-01T19:34:31.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.670+0000 7f294c82e700 1 -- 172.21.15.17:0/869952990 wait complete. 2022-08-01T19:34:31.691 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:34:31.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.790+0000 7fad3fdd6700 1 Processor -- start 2022-08-01T19:34:31.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.790+0000 7fad3fdd6700 1 -- start start 2022-08-01T19:34:31.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.790+0000 7fad3fdd6700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fad380b1f40 0x7fad380b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:31.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.790+0000 7fad3fdd6700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fad38148530 0x7fad380b2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:31.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.790+0000 7fad3fdd6700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fad380a7250 0x7fad381448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:31.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.790+0000 7fad3fdd6700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad380a9250 con 0x7fad38148530 2022-08-01T19:34:31.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.790+0000 7fad3fdd6700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad380b04e0 con 0x7fad380b1f40 2022-08-01T19:34:31.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.790+0000 7fad3fdd6700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad380b0780 con 0x7fad380a7250 2022-08-01T19:34:31.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.790+0000 7fad3d371700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fad38148530 0x7fad380b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:31.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.790+0000 7fad3d371700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fad38148530 0x7fad380b2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37082/0 (socket says 172.21.15.17:37082) 2022-08-01T19:34:31.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.790+0000 7fad3d371700 1 -- 172.21.15.17:0/3419998181 learned_addr learned my addr 172.21.15.17:0/3419998181 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:31.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.790+0000 7fad3e373700 1 --2- 172.21.15.17:0/3419998181 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fad380a7250 0x7fad381448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:31.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.790+0000 7fad3db72700 1 --2- 172.21.15.17:0/3419998181 >> v2:172.21.15.77:3300/0 conn(0x7fad380b1f40 0x7fad380b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:31.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.790+0000 7fad3d371700 1 -- 172.21.15.17:0/3419998181 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fad380a7250 msgr2=0x7fad381448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:31.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.790+0000 7fad3d371700 1 --2- 172.21.15.17:0/3419998181 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fad380a7250 0x7fad381448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:31.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.790+0000 7fad3d371700 1 -- 172.21.15.17:0/3419998181 >> v2:172.21.15.77:3300/0 conn(0x7fad380b1f40 msgr2=0x7fad380b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:31.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.790+0000 7fad3d371700 1 --2- 172.21.15.17:0/3419998181 >> v2:172.21.15.77:3300/0 conn(0x7fad380b1f40 0x7fad380b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:31.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.790+0000 7fad3d371700 1 -- 172.21.15.17:0/3419998181 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad380b0a20 con 0x7fad38148530 2022-08-01T19:34:31.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.790+0000 7fad3e373700 1 --2- 172.21.15.17:0/3419998181 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fad380a7250 0x7fad381448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:34:31.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.790+0000 7fad3d371700 1 --2- 172.21.15.17:0/3419998181 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad38148530 0x7fad380b2320 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7fad381478c0 tx=0x7fad34006eb0).ready entity=mon.0 client_cookie=c25313d5af2d5408 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:31.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.790+0000 7fad3cb70700 1 -- 172.21.15.17:0/3419998181 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad3400d570 con 0x7fad38148530 2022-08-01T19:34:31.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.790+0000 7fad3cb70700 1 -- 172.21.15.17:0/3419998181 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad34000e90 con 0x7fad38148530 2022-08-01T19:34:31.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.790+0000 7fad3cb70700 1 -- 172.21.15.17:0/3419998181 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad3400d8c0 con 0x7fad38148530 2022-08-01T19:34:31.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.790+0000 7fad3fdd6700 1 -- 172.21.15.17:0/3419998181 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad38148530 msgr2=0x7fad380b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:31.802 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.790+0000 7fad3fdd6700 1 --2- 172.21.15.17:0/3419998181 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad38148530 0x7fad380b2320 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7fad381478c0 tx=0x7fad34006eb0).stop 2022-08-01T19:34:31.802 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.790+0000 7fad3fdd6700 1 -- 172.21.15.17:0/3419998181 shutdown_connections 2022-08-01T19:34:31.802 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.790+0000 7fad3fdd6700 1 --2- 172.21.15.17:0/3419998181 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fad380a7250 0x7fad381448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:31.802 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.790+0000 7fad3fdd6700 1 --2- 172.21.15.17:0/3419998181 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad38148530 0x7fad380b2320 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:31.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.790+0000 7fad3fdd6700 1 --2- 172.21.15.17:0/3419998181 >> v2:172.21.15.77:3300/0 conn(0x7fad380b1f40 0x7fad380b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:31.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.790+0000 7fad3fdd6700 1 -- 172.21.15.17:0/3419998181 >> 172.21.15.17:0/3419998181 conn(0x7fad3805f840 msgr2=0x7fad3805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:31.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.790+0000 7fad3fdd6700 1 -- 172.21.15.17:0/3419998181 shutdown_connections 2022-08-01T19:34:31.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.790+0000 7fad3fdd6700 1 -- 172.21.15.17:0/3419998181 wait complete. 2022-08-01T19:34:31.807 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.794+0000 7fad3fdd6700 1 Processor -- start 2022-08-01T19:34:31.807 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.794+0000 7fad3fdd6700 1 -- start start 2022-08-01T19:34:31.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.794+0000 7fad3fdd6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad380a7250 0x7fad3814a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:31.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.794+0000 7fad3fdd6700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fad380b1f40 0x7fad380ac4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:31.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.794+0000 7fad3fdd6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad38148530 0x7fad380ab820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:31.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.794+0000 7fad3fdd6700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad380aa0c0 con 0x7fad380a7250 2022-08-01T19:34:31.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.794+0000 7fad3fdd6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad380aeb10 con 0x7fad38148530 2022-08-01T19:34:31.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.794+0000 7fad3fdd6700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad380aedb0 con 0x7fad380b1f40 2022-08-01T19:34:31.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.794+0000 7fad3db72700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad380a7250 0x7fad3814a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:31.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.794+0000 7fad3db72700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad380a7250 0x7fad3814a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37088/0 (socket says 172.21.15.17:37088) 2022-08-01T19:34:31.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.794+0000 7fad3db72700 1 -- 172.21.15.17:0/845397101 learned_addr learned my addr 172.21.15.17:0/845397101 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:31.810 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.794+0000 7fad3d371700 1 --2- 172.21.15.17:0/845397101 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fad380b1f40 0x7fad380ac4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:31.810 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.794+0000 7fad3e373700 1 --2- 172.21.15.17:0/845397101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad38148530 0x7fad380ab820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:31.810 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.794+0000 7fad3db72700 1 -- 172.21.15.17:0/845397101 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fad380b1f40 msgr2=0x7fad380ac4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:31.810 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.794+0000 7fad3db72700 1 --2- 172.21.15.17:0/845397101 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fad380b1f40 0x7fad380ac4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:31.811 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.794+0000 7fad3db72700 1 -- 172.21.15.17:0/845397101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad38148530 msgr2=0x7fad380ab820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:31.811 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.794+0000 7fad3db72700 1 --2- 172.21.15.17:0/845397101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad38148530 0x7fad380ab820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:31.811 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.794+0000 7fad3db72700 1 -- 172.21.15.17:0/845397101 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad3400b1e0 con 0x7fad380a7250 2022-08-01T19:34:31.811 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.794+0000 7fad3db72700 1 --2- 172.21.15.17:0/845397101 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad380a7250 0x7fad3814a380 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7fad280030b0 tx=0x7fad28008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:31.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.794+0000 7fad26ffd700 1 -- 172.21.15.17:0/845397101 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad28011780 con 0x7fad380a7250 2022-08-01T19:34:31.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.794+0000 7fad26ffd700 1 -- 172.21.15.17:0/845397101 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad280118e0 con 0x7fad380a7250 2022-08-01T19:34:31.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.794+0000 7fad3fdd6700 1 -- 172.21.15.17:0/845397101 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad380af050 con 0x7fad380a7250 2022-08-01T19:34:31.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.794+0000 7fad26ffd700 1 -- 172.21.15.17:0/845397101 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad28011c40 con 0x7fad380a7250 2022-08-01T19:34:31.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.794+0000 7fad3fdd6700 1 -- 172.21.15.17:0/845397101 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad380ac9d0 con 0x7fad380a7250 2022-08-01T19:34:31.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.798+0000 7fad3fdd6700 1 -- 172.21.15.17:0/845397101 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad380b0cc0 con 0x7fad380a7250 2022-08-01T19:34:31.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.798+0000 7fad26ffd700 1 -- 172.21.15.17:0/845397101 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fad2801c460 con 0x7fad380a7250 2022-08-01T19:34:31.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.798+0000 7fad26ffd700 1 --2- 172.21.15.17:0/845397101 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fad18053950 0x7fad18055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:31.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.798+0000 7fad26ffd700 1 -- 172.21.15.17:0/845397101 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fad2806c270 con 0x7fad380a7250 2022-08-01T19:34:31.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.798+0000 7fad3d371700 1 --2- 172.21.15.17:0/845397101 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fad18053950 0x7fad18055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:31.814 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.798+0000 7fad3d371700 1 --2- 172.21.15.17:0/845397101 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fad18053950 0x7fad18055e10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fad3401c000 tx=0x7fad34015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:31.814 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.802+0000 7fad26ffd700 1 -- 172.21.15.17:0/845397101 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fad28042ec0 con 0x7fad380a7250 2022-08-01T19:34:31.936 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.930+0000 7fad3fdd6700 1 -- 172.21.15.17:0/845397101 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad380574e0 con 0x7fad380a7250 2022-08-01T19:34:31.938 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.934+0000 7fad26ffd700 1 -- 172.21.15.17:0/845397101 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16299 (secure 0 0 0) 0x7fad280464e0 con 0x7fad380a7250 2022-08-01T19:34:31.939 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:31.939 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:34:09.692404+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:34:04.635473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-08-01T19:34:07.409434+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:34:31.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.934+0000 7fad3fdd6700 1 -- 172.21.15.17:0/845397101 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fad18053950 msgr2=0x7fad18055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:31.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.934+0000 7fad3fdd6700 1 --2- 172.21.15.17:0/845397101 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fad18053950 0x7fad18055e10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fad3401c000 tx=0x7fad34015000).stop 2022-08-01T19:34:31.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.934+0000 7fad3fdd6700 1 -- 172.21.15.17:0/845397101 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad380a7250 msgr2=0x7fad3814a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:31.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.934+0000 7fad3fdd6700 1 --2- 172.21.15.17:0/845397101 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad380a7250 0x7fad3814a380 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7fad280030b0 tx=0x7fad28008040).stop 2022-08-01T19:34:31.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.934+0000 7fad3fdd6700 1 -- 172.21.15.17:0/845397101 shutdown_connections 2022-08-01T19:34:31.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.934+0000 7fad3fdd6700 1 --2- 172.21.15.17:0/845397101 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fad18053950 0x7fad18055e10 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:31.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.934+0000 7fad3fdd6700 1 --2- 172.21.15.17:0/845397101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad38148530 0x7fad380ab820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:31.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.934+0000 7fad3fdd6700 1 --2- 172.21.15.17:0/845397101 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fad380b1f40 0x7fad380ac4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:31.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.934+0000 7fad3fdd6700 1 --2- 172.21.15.17:0/845397101 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad380a7250 0x7fad3814a380 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:31.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.934+0000 7fad3fdd6700 1 -- 172.21.15.17:0/845397101 >> 172.21.15.17:0/845397101 conn(0x7fad3805f840 msgr2=0x7fad3814e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:31.944 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.934+0000 7fad3fdd6700 1 -- 172.21.15.17:0/845397101 shutdown_connections 2022-08-01T19:34:31.944 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:31.934+0000 7fad3fdd6700 1 -- 172.21.15.17:0/845397101 wait complete. 2022-08-01T19:34:31.953 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T19:34:04.635473+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T19:34:07.409434+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T19:34:31.954 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:34:32.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.058+0000 7f4053287700 1 Processor -- start 2022-08-01T19:34:32.063 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.058+0000 7f4053287700 1 -- start start 2022-08-01T19:34:32.065 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.058+0000 7f4053287700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f404c060960 0x7f404c140230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:32.065 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.058+0000 7f4053287700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f404c060010 con 0x7f404c05fc30 2022-08-01T19:34:32.065 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.058+0000 7f4053287700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f404c0acdf0 con 0x7f404c0ac690 2022-08-01T19:34:32.065 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.058+0000 7f4053287700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f404c1433d0 con 0x7f404c060960 2022-08-01T19:34:32.066 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.058+0000 7f4051023700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f404c05fc30 0x7f404c143a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57522/0 (socket says 172.21.15.17:57522) 2022-08-01T19:34:32.066 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.058+0000 7f4051023700 1 -- 172.21.15.17:0/3149211345 learned_addr learned my addr 172.21.15.17:0/3149211345 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:32.066 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.058+0000 7f4050822700 1 --2- 172.21.15.17:0/3149211345 >> v2:172.21.15.77:3300/0 conn(0x7f404c060960 0x7f404c140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:32.066 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.058+0000 7f403ffff700 1 -- 172.21.15.17:0/3149211345 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3518267593 0 0) 0x7f404c060010 con 0x7f404c05fc30 2022-08-01T19:34:32.066 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.058+0000 7f403ffff700 1 -- 172.21.15.17:0/3149211345 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4030003260 con 0x7f404c05fc30 2022-08-01T19:34:32.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.058+0000 7f4050822700 1 -- 172.21.15.17:0/3149211345 >> v1:172.21.15.77:6789/0 conn(0x7f404c0ac690 legacy=0x7f404c140720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:32.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.058+0000 7f4050822700 1 -- 172.21.15.17:0/3149211345 >> v1:172.21.15.17:6789/0 conn(0x7f404c05fc30 legacy=0x7f404c143a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:32.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.058+0000 7f4050822700 1 -- 172.21.15.17:0/3149211345 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f404c0ad090 con 0x7f404c060960 2022-08-01T19:34:32.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.058+0000 7f4050822700 1 --2- 172.21.15.17:0/3149211345 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f404c060960 0x7f404c140230 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f4040002f70 tx=0x7f4040006df0).ready entity=mon.1 client_cookie=b9960496e9b96434 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:32.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.058+0000 7f403ffff700 1 -- 172.21.15.17:0/3149211345 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f404000d610 con 0x7f404c060960 2022-08-01T19:34:32.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.058+0000 7f403ffff700 1 -- 172.21.15.17:0/3149211345 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4040000e90 con 0x7f404c060960 2022-08-01T19:34:32.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.058+0000 7f403ffff700 1 -- 172.21.15.17:0/3149211345 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f404000d960 con 0x7f404c060960 2022-08-01T19:34:32.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.058+0000 7f4053287700 1 -- 172.21.15.17:0/3149211345 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f404c060960 msgr2=0x7f404c140230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:32.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.058+0000 7f4053287700 1 --2- 172.21.15.17:0/3149211345 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f404c060960 0x7f404c140230 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f4040002f70 tx=0x7f4040006df0).stop 2022-08-01T19:34:32.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.058+0000 7f4053287700 1 -- 172.21.15.17:0/3149211345 shutdown_connections 2022-08-01T19:34:32.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.058+0000 7f4053287700 1 --2- 172.21.15.17:0/3149211345 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f404c060960 0x7f404c140230 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:32.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.058+0000 7f4053287700 1 -- 172.21.15.17:0/3149211345 >> 172.21.15.17:0/3149211345 conn(0x7f404c0b2b00 msgr2=0x7f404c0b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:32.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.058+0000 7f4053287700 1 -- 172.21.15.17:0/3149211345 shutdown_connections 2022-08-01T19:34:32.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.062+0000 7f4053287700 1 -- 172.21.15.17:0/3149211345 wait complete. 2022-08-01T19:34:32.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.062+0000 7f4053287700 1 Processor -- start 2022-08-01T19:34:32.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.062+0000 7f4053287700 1 -- start start 2022-08-01T19:34:32.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.062+0000 7f4053287700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f404c05fc30 0x7f404c145540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:32.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.062+0000 7f4053287700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f404c060960 0x7f404c13d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:32.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.062+0000 7f4053287700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f404c0ac690 0x7f404c13d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:32.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.062+0000 7f4053287700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404c0b11b0 con 0x7f404c0ac690 2022-08-01T19:34:32.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.062+0000 7f4053287700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404c0648c0 con 0x7f404c05fc30 2022-08-01T19:34:32.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.062+0000 7f4053287700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f404c146f20 con 0x7f404c060960 2022-08-01T19:34:32.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.062+0000 7f4051824700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f404c0ac690 0x7f404c13d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:32.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.062+0000 7f4050822700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f404c060960 0x7f404c13d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:32.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.062+0000 7f4051023700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f404c05fc30 0x7f404c145540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:32.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.062+0000 7f4051824700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f404c0ac690 0x7f404c13d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37106/0 (socket says 172.21.15.17:37106) 2022-08-01T19:34:32.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.062+0000 7f4051824700 1 -- 172.21.15.17:0/3660192701 learned_addr learned my addr 172.21.15.17:0/3660192701 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:32.076 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.062+0000 7f4050822700 1 -- 172.21.15.17:0/3660192701 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f404c05fc30 msgr2=0x7f404c145540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:32.076 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.062+0000 7f4050822700 1 --2- 172.21.15.17:0/3660192701 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f404c05fc30 0x7f404c145540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:32.077 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.062+0000 7f4050822700 1 -- 172.21.15.17:0/3660192701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f404c0ac690 msgr2=0x7f404c13d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:32.077 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.062+0000 7f4050822700 1 --2- 172.21.15.17:0/3660192701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f404c0ac690 0x7f404c13d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:32.078 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.062+0000 7f4050822700 1 -- 172.21.15.17:0/3660192701 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f404000b180 con 0x7f404c060960 2022-08-01T19:34:32.078 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.062+0000 7f4050822700 1 --2- 172.21.15.17:0/3660192701 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f404c060960 0x7f404c13d390 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f4040002f70 tx=0x7f4040014000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:32.078 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.066+0000 7f403e7fc700 1 -- 172.21.15.17:0/3660192701 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4040013de0 con 0x7f404c060960 2022-08-01T19:34:32.079 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.066+0000 7f403e7fc700 1 -- 172.21.15.17:0/3660192701 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f404000d290 con 0x7f404c060960 2022-08-01T19:34:32.079 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.066+0000 7f4053287700 1 -- 172.21.15.17:0/3660192701 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f404c0b1410 con 0x7f404c060960 2022-08-01T19:34:32.080 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.066+0000 7f403e7fc700 1 -- 172.21.15.17:0/3660192701 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f404001e5f0 con 0x7f404c060960 2022-08-01T19:34:32.080 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.066+0000 7f4053287700 1 -- 172.21.15.17:0/3660192701 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f404c0b18b0 con 0x7f404c060960 2022-08-01T19:34:32.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.066+0000 7f4053287700 1 -- 172.21.15.17:0/3660192701 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f404c0b15a0 con 0x7f404c060960 2022-08-01T19:34:32.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.066+0000 7f403e7fc700 1 -- 172.21.15.17:0/3660192701 <== mon.2 v2:172.21.15.77:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4040025030 con 0x7f404c060960 2022-08-01T19:34:32.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.066+0000 7f403e7fc700 1 --2- 172.21.15.17:0/3660192701 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4034053b30 0x7f4034055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:32.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.066+0000 7f403e7fc700 1 -- 172.21.15.17:0/3660192701 <== mon.2 v2:172.21.15.77:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f404006d0f0 con 0x7f404c060960 2022-08-01T19:34:32.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.066+0000 7f4051023700 1 --2- 172.21.15.17:0/3660192701 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4034053b30 0x7f4034055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:32.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.070+0000 7f4051023700 1 --2- 172.21.15.17:0/3660192701 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4034053b30 0x7f4034055ff0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f404800d000 tx=0x7f4048009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:32.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.078+0000 7f403e7fc700 1 -- 172.21.15.17:0/3660192701 <== mon.2 v2:172.21.15.77:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4040043da0 con 0x7f404c060960 2022-08-01T19:34:32.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.206+0000 7f4053287700 1 -- 172.21.15.17:0/3660192701 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f404c0574e0 con 0x7f404c060960 2022-08-01T19:34:32.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.214+0000 7f403e7fc700 1 -- 172.21.15.17:0/3660192701 <== mon.2 v2:172.21.15.77:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v25) v1 ==== 110+0+10 (secure 0 0 0) 0x7f40400230b0 con 0x7f404c060960 2022-08-01T19:34:32.214 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:34:32.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.214+0000 7f4053287700 1 -- 172.21.15.17:0/3660192701 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4034053b30 msgr2=0x7f4034055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:32.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.214+0000 7f4053287700 1 --2- 172.21.15.17:0/3660192701 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4034053b30 0x7f4034055ff0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f404800d000 tx=0x7f4048009040).stop 2022-08-01T19:34:32.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.214+0000 7f4053287700 1 -- 172.21.15.17:0/3660192701 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f404c060960 msgr2=0x7f404c13d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:32.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.214+0000 7f4053287700 1 --2- 172.21.15.17:0/3660192701 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f404c060960 0x7f404c13d390 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f4040002f70 tx=0x7f4040014000).stop 2022-08-01T19:34:32.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.214+0000 7f4053287700 1 -- 172.21.15.17:0/3660192701 shutdown_connections 2022-08-01T19:34:32.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.214+0000 7f4053287700 1 --2- 172.21.15.17:0/3660192701 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4034053b30 0x7f4034055ff0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:32.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.214+0000 7f4053287700 1 --2- 172.21.15.17:0/3660192701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f404c0ac690 0x7f404c13d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:32.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.214+0000 7f4053287700 1 --2- 172.21.15.17:0/3660192701 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f404c060960 0x7f404c13d390 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:32.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.214+0000 7f4053287700 1 --2- 172.21.15.17:0/3660192701 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f404c05fc30 0x7f404c145540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:32.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.214+0000 7f4053287700 1 -- 172.21.15.17:0/3660192701 >> 172.21.15.17:0/3660192701 conn(0x7f404c0b2b00 msgr2=0x7f404c13e700 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:32.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.214+0000 7f4053287700 1 -- 172.21.15.17:0/3660192701 shutdown_connections 2022-08-01T19:34:32.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.214+0000 7f4053287700 1 -- 172.21.15.17:0/3660192701 wait complete. 2022-08-01T19:34:32.227 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:34:32.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.334+0000 7f65af59e700 1 Processor -- start 2022-08-01T19:34:32.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.334+0000 7f65af59e700 1 -- start start 2022-08-01T19:34:32.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.334+0000 7f65af59e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f65b00b1280 0x7f65b013f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:32.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.334+0000 7f65af59e700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65b013fa10 con 0x7f65b005c960 2022-08-01T19:34:32.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.334+0000 7f65af59e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65b005cd40 con 0x7f65b005d240 2022-08-01T19:34:32.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.334+0000 7f65af59e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65b00ad6d0 con 0x7f65b00b1280 2022-08-01T19:34:32.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.334+0000 7f65ae59c700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f65b005c960 0x7f65b00acfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57536/0 (socket says 172.21.15.17:57536) 2022-08-01T19:34:32.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.334+0000 7f65ae59c700 1 -- 172.21.15.17:0/1332923881 learned_addr learned my addr 172.21.15.17:0/1332923881 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:32.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.334+0000 7f65aed9d700 1 --2- 172.21.15.17:0/1332923881 >> v2:172.21.15.77:3300/0 conn(0x7f65b00b1280 0x7f65b013f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:32.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.334+0000 7f65ad59a700 1 -- 172.21.15.17:0/1332923881 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3774128839 0 0) 0x7f65a0001110 con 0x7f65b005c960 2022-08-01T19:34:32.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.334+0000 7f65ad59a700 1 -- 172.21.15.17:0/1332923881 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6594003260 con 0x7f65b005c960 2022-08-01T19:34:32.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.334+0000 7f65ad59a700 1 -- 172.21.15.17:0/1332923881 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2410950175 0 0) 0x7f65b005cd40 con 0x7f65b005d240 2022-08-01T19:34:32.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.334+0000 7f65ad59a700 1 -- 172.21.15.17:0/1332923881 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65a0001110 con 0x7f65b005d240 2022-08-01T19:34:32.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.334+0000 7f65aed9d700 1 -- 172.21.15.17:0/1332923881 >> v1:172.21.15.77:6789/0 conn(0x7f65b005d240 legacy=0x7f65b00aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:32.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.334+0000 7f65aed9d700 1 -- 172.21.15.17:0/1332923881 >> v1:172.21.15.17:6789/0 conn(0x7f65b005c960 legacy=0x7f65b00acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:32.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.334+0000 7f65aed9d700 1 -- 172.21.15.17:0/1332923881 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b00b16b0 con 0x7f65b00b1280 2022-08-01T19:34:32.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.334+0000 7f65aed9d700 1 --2- 172.21.15.17:0/1332923881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65b00b1280 0x7f65b013f520 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f65a80032f0 tx=0x7f65a800a370).ready entity=mon.1 client_cookie=17e659f9c8a41ca server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:32.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.334+0000 7f65ad59a700 1 -- 172.21.15.17:0/1332923881 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65a800cba0 con 0x7f65b00b1280 2022-08-01T19:34:32.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.334+0000 7f65ad59a700 1 -- 172.21.15.17:0/1332923881 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a800cd00 con 0x7f65b00b1280 2022-08-01T19:34:32.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.334+0000 7f65ad59a700 1 -- 172.21.15.17:0/1332923881 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65a8010600 con 0x7f65b00b1280 2022-08-01T19:34:32.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.334+0000 7f65af59e700 1 -- 172.21.15.17:0/1332923881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65b00b1280 msgr2=0x7f65b013f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:32.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.334+0000 7f65af59e700 1 --2- 172.21.15.17:0/1332923881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65b00b1280 0x7f65b013f520 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f65a80032f0 tx=0x7f65a800a370).stop 2022-08-01T19:34:32.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.334+0000 7f65af59e700 1 -- 172.21.15.17:0/1332923881 shutdown_connections 2022-08-01T19:34:32.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.334+0000 7f65af59e700 1 --2- 172.21.15.17:0/1332923881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65b00b1280 0x7f65b013f520 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:32.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.334+0000 7f65af59e700 1 -- 172.21.15.17:0/1332923881 >> 172.21.15.17:0/1332923881 conn(0x7f65b0060310 msgr2=0x7f65b00b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:32.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.334+0000 7f65af59e700 1 -- 172.21.15.17:0/1332923881 shutdown_connections 2022-08-01T19:34:32.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.338+0000 7f65af59e700 1 -- 172.21.15.17:0/1332923881 wait complete. 2022-08-01T19:34:32.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.338+0000 7f65af59e700 1 Processor -- start 2022-08-01T19:34:32.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.338+0000 7f65af59e700 1 -- start start 2022-08-01T19:34:32.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.342+0000 7f65af59e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65b005c960 0x7f65b00ab720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:32.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.342+0000 7f65af59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f65b005d240 0x7f65b0066ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:32.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.342+0000 7f65af59e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65b00b1280 0x7f65b0152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:32.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.342+0000 7f65af59e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b014a5f0 con 0x7f65b005c960 2022-08-01T19:34:32.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.342+0000 7f65af59e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b00664f0 con 0x7f65b00b1280 2022-08-01T19:34:32.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.342+0000 7f65af59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65b0066790 con 0x7f65b005d240 2022-08-01T19:34:32.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.342+0000 7f65ae59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65b005c960 0x7f65b00ab720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:32.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.342+0000 7f65ae59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65b005c960 0x7f65b00ab720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37116/0 (socket says 172.21.15.17:37116) 2022-08-01T19:34:32.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.342+0000 7f65ae59c700 1 -- 172.21.15.17:0/1592552282 learned_addr learned my addr 172.21.15.17:0/1592552282 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:32.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.342+0000 7f65add9b700 1 --2- 172.21.15.17:0/1592552282 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f65b005d240 0x7f65b0066ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:32.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.342+0000 7f65aed9d700 1 --2- 172.21.15.17:0/1592552282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65b00b1280 0x7f65b0152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:32.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.342+0000 7f65ae59c700 1 -- 172.21.15.17:0/1592552282 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f65b005d240 msgr2=0x7f65b0066ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:32.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.342+0000 7f65ae59c700 1 --2- 172.21.15.17:0/1592552282 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f65b005d240 0x7f65b0066ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:32.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.342+0000 7f65ae59c700 1 -- 172.21.15.17:0/1592552282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65b00b1280 msgr2=0x7f65b0152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:32.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.342+0000 7f65ae59c700 1 --2- 172.21.15.17:0/1592552282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65b00b1280 0x7f65b0152610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:32.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.342+0000 7f65ae59c700 1 -- 172.21.15.17:0/1592552282 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a800a610 con 0x7f65b005c960 2022-08-01T19:34:32.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.342+0000 7f65ae59c700 1 --2- 172.21.15.17:0/1592552282 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65b005c960 0x7f65b00ab720 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f65a0001360 tx=0x7f65a0007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:32.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.342+0000 7f659f7fe700 1 -- 172.21.15.17:0/1592552282 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65a00072e0 con 0x7f65b005c960 2022-08-01T19:34:32.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.342+0000 7f659f7fe700 1 -- 172.21.15.17:0/1592552282 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a0007440 con 0x7f65b005c960 2022-08-01T19:34:32.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.342+0000 7f65af59e700 1 -- 172.21.15.17:0/1592552282 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65b0066a30 con 0x7f65b005c960 2022-08-01T19:34:32.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.342+0000 7f659f7fe700 1 -- 172.21.15.17:0/1592552282 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65a0005800 con 0x7f65b005c960 2022-08-01T19:34:32.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.342+0000 7f65af59e700 1 -- 172.21.15.17:0/1592552282 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65b0152b00 con 0x7f65b005c960 2022-08-01T19:34:32.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.342+0000 7f659f7fe700 1 -- 172.21.15.17:0/1592552282 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f65a001a060 con 0x7f65b005c960 2022-08-01T19:34:32.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.342+0000 7f65af59e700 1 -- 172.21.15.17:0/1592552282 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65b0152db0 con 0x7f65b005c960 2022-08-01T19:34:32.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.346+0000 7f659f7fe700 1 --2- 172.21.15.17:0/1592552282 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f658c053950 0x7f658c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:32.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.346+0000 7f659f7fe700 1 -- 172.21.15.17:0/1592552282 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f65a006b750 con 0x7f65b005c960 2022-08-01T19:34:32.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.346+0000 7f65add9b700 1 --2- 172.21.15.17:0/1592552282 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f658c053950 0x7f658c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:32.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.346+0000 7f65add9b700 1 --2- 172.21.15.17:0/1592552282 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f658c053950 0x7f658c055e10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f65a400d000 tx=0x7f65a4007a20).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:32.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.354+0000 7f659f7fe700 1 -- 172.21.15.17:0/1592552282 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f65a0042510 con 0x7f65b005c960 2022-08-01T19:34:32.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.482+0000 7f65af59e700 1 -- 172.21.15.17:0/1592552282 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f65b013fa10 con 0x7f65b005c960 2022-08-01T19:34:32.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.482+0000 7f659f7fe700 1 -- 172.21.15.17:0/1592552282 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f65b013fa10 con 0x7f65b005c960 2022-08-01T19:34:32.488 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:34:32.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.486+0000 7f65af59e700 1 -- 172.21.15.17:0/1592552282 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f658c053950 msgr2=0x7f658c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:32.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.486+0000 7f65af59e700 1 --2- 172.21.15.17:0/1592552282 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f658c053950 0x7f658c055e10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f65a400d000 tx=0x7f65a4007a20).stop 2022-08-01T19:34:32.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.486+0000 7f65af59e700 1 -- 172.21.15.17:0/1592552282 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65b005c960 msgr2=0x7f65b00ab720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:32.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.486+0000 7f65af59e700 1 --2- 172.21.15.17:0/1592552282 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65b005c960 0x7f65b00ab720 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f65a0001360 tx=0x7f65a0007b50).stop 2022-08-01T19:34:32.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.486+0000 7f65af59e700 1 -- 172.21.15.17:0/1592552282 shutdown_connections 2022-08-01T19:34:32.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.486+0000 7f65af59e700 1 --2- 172.21.15.17:0/1592552282 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f658c053950 0x7f658c055e10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:32.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.486+0000 7f65af59e700 1 --2- 172.21.15.17:0/1592552282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65b00b1280 0x7f65b0152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:32.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.486+0000 7f65af59e700 1 --2- 172.21.15.17:0/1592552282 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f65b005d240 0x7f65b0066ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:32.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.486+0000 7f65af59e700 1 --2- 172.21.15.17:0/1592552282 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65b005c960 0x7f65b00ab720 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:32.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.486+0000 7f65af59e700 1 -- 172.21.15.17:0/1592552282 >> 172.21.15.17:0/1592552282 conn(0x7f65b0060310 msgr2=0x7f65b013e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:32.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.486+0000 7f65af59e700 1 -- 172.21.15.17:0/1592552282 shutdown_connections 2022-08-01T19:34:32.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.486+0000 7f65af59e700 1 -- 172.21.15.17:0/1592552282 wait complete. 2022-08-01T19:34:32.502 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T19:34:32.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.614+0000 7f8b8122d700 1 Processor -- start 2022-08-01T19:34:32.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.614+0000 7f8b8122d700 1 -- start start 2022-08-01T19:34:32.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b8122d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8b7c0a9320 0x7f8b7c1466e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:32.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b8122d700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f8b7c0b0d10 0x7f8b7c146bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:32.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b8122d700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b7c0b12d0 con 0x7f8b7c0b0560 2022-08-01T19:34:32.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b8122d700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b7c0a9700 con 0x7f8b7c0b0d10 2022-08-01T19:34:32.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b8122d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c0b1500 con 0x7f8b7c0a9320 2022-08-01T19:34:32.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b80a2c700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f8b7c0b0560 0x7f8b7c140350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57554/0 (socket says 172.21.15.17:57554) 2022-08-01T19:34:32.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b80a2c700 1 -- 172.21.15.17:0/2271612918 learned_addr learned my addr 172.21.15.17:0/2271612918 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:32.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b7bfff700 1 --2- 172.21.15.17:0/2271612918 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8b7c0a9320 0x7f8b7c1466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:32.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b737fe700 1 --2- 172.21.15.17:0/2271612918 >> v2:172.21.15.77:3300/0 conn(0x7f8b7c0b0d10 0x7f8b7c146bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:32.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b7b7fe700 1 -- 172.21.15.17:0/2271612918 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4254587776 0 0) 0x7f8b7c0b12d0 con 0x7f8b7c0b0560 2022-08-01T19:34:32.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b7b7fe700 1 -- 172.21.15.17:0/2271612918 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b5c003260 con 0x7f8b7c0b0560 2022-08-01T19:34:32.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b7b7fe700 1 -- 172.21.15.17:0/2271612918 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2044216234 0 0) 0x7f8b5c003260 con 0x7f8b7c0b0560 2022-08-01T19:34:32.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b7b7fe700 1 -- 172.21.15.17:0/2271612918 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8b7c0a9320 msgr2=0x7f8b7c1466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:32.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b7b7fe700 1 --2- 172.21.15.17:0/2271612918 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8b7c0a9320 0x7f8b7c1466e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:32.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b7b7fe700 1 -- 172.21.15.17:0/2271612918 >> v2:172.21.15.77:3300/0 conn(0x7f8b7c0b0d10 msgr2=0x7f8b7c146bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:32.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b7b7fe700 1 --2- 172.21.15.17:0/2271612918 >> v2:172.21.15.77:3300/0 conn(0x7f8b7c0b0d10 0x7f8b7c146bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:32.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b7b7fe700 1 -- 172.21.15.17:0/2271612918 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7c0b17a0 con 0x7f8b7c0b0560 2022-08-01T19:34:32.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b7b7fe700 1 -- 172.21.15.17:0/2271612918 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7f8b740019e0 con 0x7f8b7c0b0560 2022-08-01T19:34:32.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b7b7fe700 1 -- 172.21.15.17:0/2271612918 >> v1:172.21.15.17:6789/0 conn(0x7f8b7c0b0560 legacy=0x7f8b7c140350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:32.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b7b7fe700 1 --2- 172.21.15.17:0/2271612918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8b5c003750 0x7f8b5c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:32.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b7b7fe700 1 --2- 172.21.15.17:0/2271612918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b5c004500 0x7f8b5c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:32.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b7b7fe700 1 --2- 172.21.15.17:0/2271612918 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8b7c0a9320 0x7f8b5c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:32.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b7b7fe700 1 -- 172.21.15.17:0/2271612918 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c0a9700 con 0x7f8b5c003750 2022-08-01T19:34:32.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b7b7fe700 1 -- 172.21.15.17:0/2271612918 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c0b1500 con 0x7f8b5c004500 2022-08-01T19:34:32.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b7b7fe700 1 -- 172.21.15.17:0/2271612918 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c00b750 con 0x7f8b7c0a9320 2022-08-01T19:34:32.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b7bfff700 1 --2- 172.21.15.17:0/2271612918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8b5c003750 0x7f8b5c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:32.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b737fe700 1 --2- 172.21.15.17:0/2271612918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b5c004500 0x7f8b5c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:32.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b80a2c700 1 --2- 172.21.15.17:0/2271612918 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8b7c0a9320 0x7f8b5c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:32.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b7bfff700 1 -- 172.21.15.17:0/2271612918 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8b7c0a9320 msgr2=0x7f8b5c004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:32.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b7bfff700 1 --2- 172.21.15.17:0/2271612918 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8b7c0a9320 0x7f8b5c004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:32.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b7bfff700 1 -- 172.21.15.17:0/2271612918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b5c004500 msgr2=0x7f8b5c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:32.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b7bfff700 1 --2- 172.21.15.17:0/2271612918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b5c004500 0x7f8b5c004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:32.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b7bfff700 1 -- 172.21.15.17:0/2271612918 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b5c003260 con 0x7f8b5c003750 2022-08-01T19:34:32.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b7bfff700 1 --2- 172.21.15.17:0/2271612918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8b5c003750 0x7f8b5c003b70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f8b6c00c000 tx=0x7f8b6c006df0).ready entity=mon.0 client_cookie=aa1ea184d9847b23 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:32.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b7b7fe700 1 -- 172.21.15.17:0/2271612918 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b6c00e830 con 0x7f8b5c003750 2022-08-01T19:34:32.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b7b7fe700 1 -- 172.21.15.17:0/2271612918 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6c00e990 con 0x7f8b5c003750 2022-08-01T19:34:32.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b7b7fe700 1 -- 172.21.15.17:0/2271612918 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b6c00ecf0 con 0x7f8b5c003750 2022-08-01T19:34:32.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b8122d700 1 -- 172.21.15.17:0/2271612918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8b5c003750 msgr2=0x7f8b5c003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:32.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b8122d700 1 --2- 172.21.15.17:0/2271612918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8b5c003750 0x7f8b5c003b70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f8b6c00c000 tx=0x7f8b6c006df0).stop 2022-08-01T19:34:32.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b80a2c700 1 -- 172.21.15.17:0/2271612918 reap_dead start 2022-08-01T19:34:32.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b8122d700 1 -- 172.21.15.17:0/2271612918 shutdown_connections 2022-08-01T19:34:32.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b8122d700 1 -- 172.21.15.17:0/2271612918 >> 172.21.15.17:0/2271612918 conn(0x7f8b7c0612d0 msgr2=0x7f8b7c062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:32.636 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b8122d700 1 -- 172.21.15.17:0/2271612918 shutdown_connections 2022-08-01T19:34:32.636 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b8122d700 1 -- 172.21.15.17:0/2271612918 wait complete. 2022-08-01T19:34:32.636 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b8122d700 1 Processor -- start 2022-08-01T19:34:32.636 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b8122d700 1 -- start start 2022-08-01T19:34:32.637 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b8122d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8b5c003750 0x7f8b7c142480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:32.637 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b8122d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8b7c0b52e0 0x7f8b7c152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:32.637 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b8122d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b7c155bc0 0x7f8b7c156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:32.637 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b8122d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c0b5860 con 0x7f8b7c0b52e0 2022-08-01T19:34:32.638 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b8122d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c1525f0 con 0x7f8b7c155bc0 2022-08-01T19:34:32.638 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b8122d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c152890 con 0x7f8b5c003750 2022-08-01T19:34:32.638 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b737fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8b7c0b52e0 0x7f8b7c152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:32.638 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b737fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8b7c0b52e0 0x7f8b7c152fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37138/0 (socket says 172.21.15.17:37138) 2022-08-01T19:34:32.639 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b737fe700 1 -- 172.21.15.17:0/492450393 learned_addr learned my addr 172.21.15.17:0/492450393 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:32.639 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b80a2c700 1 --2- 172.21.15.17:0/492450393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b7c155bc0 0x7f8b7c156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:32.639 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b7bfff700 1 --2- 172.21.15.17:0/492450393 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8b5c003750 0x7f8b7c142480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:32.640 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b80a2c700 1 -- 172.21.15.17:0/492450393 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8b5c003750 msgr2=0x7f8b7c142480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:32.640 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b80a2c700 1 --2- 172.21.15.17:0/492450393 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8b5c003750 0x7f8b7c142480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:32.640 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b80a2c700 1 -- 172.21.15.17:0/492450393 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8b7c0b52e0 msgr2=0x7f8b7c152fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:32.640 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b80a2c700 1 --2- 172.21.15.17:0/492450393 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8b7c0b52e0 0x7f8b7c152fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:32.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b80a2c700 1 -- 172.21.15.17:0/492450393 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7c152b30 con 0x7f8b7c155bc0 2022-08-01T19:34:32.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.618+0000 7f8b737fe700 1 --2- 172.21.15.17:0/492450393 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8b7c0b52e0 0x7f8b7c152fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:34:32.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.622+0000 7f8b80a2c700 1 --2- 172.21.15.17:0/492450393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b7c155bc0 0x7f8b7c156040 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f8b74000b60 tx=0x7f8b7c0adee0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:32.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.622+0000 7f8b79ffb700 1 -- 172.21.15.17:0/492450393 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b74011780 con 0x7f8b7c155bc0 2022-08-01T19:34:32.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.622+0000 7f8b79ffb700 1 -- 172.21.15.17:0/492450393 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b740118e0 con 0x7f8b7c155bc0 2022-08-01T19:34:32.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.622+0000 7f8b8122d700 1 -- 172.21.15.17:0/492450393 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b7c142970 con 0x7f8b7c155bc0 2022-08-01T19:34:32.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.622+0000 7f8b79ffb700 1 -- 172.21.15.17:0/492450393 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b74011c40 con 0x7f8b7c155bc0 2022-08-01T19:34:32.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.622+0000 7f8b8122d700 1 -- 172.21.15.17:0/492450393 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b7c0b67a0 con 0x7f8b7c155bc0 2022-08-01T19:34:32.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.622+0000 7f8b8122d700 1 -- 172.21.15.17:0/492450393 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b7c0b59d0 con 0x7f8b7c155bc0 2022-08-01T19:34:32.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.622+0000 7f8b79ffb700 1 -- 172.21.15.17:0/492450393 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8b7401a310 con 0x7f8b7c155bc0 2022-08-01T19:34:32.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.622+0000 7f8b79ffb700 1 --2- 172.21.15.17:0/492450393 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8b60053b30 0x7f8b60055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:32.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.622+0000 7f8b79ffb700 1 -- 172.21.15.17:0/492450393 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f8b7406b890 con 0x7f8b7c155bc0 2022-08-01T19:34:32.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.622+0000 7f8b7bfff700 1 --2- 172.21.15.17:0/492450393 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8b60053b30 0x7f8b60055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:32.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.622+0000 7f8b7bfff700 1 --2- 172.21.15.17:0/492450393 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8b60053b30 0x7f8b60055ff0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f8b6c01e000 tx=0x7f8b6c019000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:32.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.634+0000 7f8b79ffb700 1 -- 172.21.15.17:0/492450393 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8b7c0b59d0 con 0x7f8b7c155bc0 2022-08-01T19:34:32.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.758+0000 7f8b8122d700 1 -- 172.21.15.17:0/492450393 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8b7c0b59d0 con 0x7f8b7c155bc0 2022-08-01T19:34:32.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.766+0000 7f8b79ffb700 1 -- 172.21.15.17:0/492450393 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+10 (secure 0 0 0) 0x7f8b7c0b59d0 con 0x7f8b7c155bc0 2022-08-01T19:34:32.767 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:34:32.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.766+0000 7f8b8122d700 1 -- 172.21.15.17:0/492450393 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8b60053b30 msgr2=0x7f8b60055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:32.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.766+0000 7f8b8122d700 1 --2- 172.21.15.17:0/492450393 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8b60053b30 0x7f8b60055ff0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f8b6c01e000 tx=0x7f8b6c019000).stop 2022-08-01T19:34:32.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.766+0000 7f8b8122d700 1 -- 172.21.15.17:0/492450393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b7c155bc0 msgr2=0x7f8b7c156040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:32.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.766+0000 7f8b8122d700 1 --2- 172.21.15.17:0/492450393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b7c155bc0 0x7f8b7c156040 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f8b74000b60 tx=0x7f8b7c0adee0).stop 2022-08-01T19:34:32.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.766+0000 7f8b8122d700 1 -- 172.21.15.17:0/492450393 shutdown_connections 2022-08-01T19:34:32.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.766+0000 7f8b8122d700 1 --2- 172.21.15.17:0/492450393 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8b60053b30 0x7f8b60055ff0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:32.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.766+0000 7f8b8122d700 1 --2- 172.21.15.17:0/492450393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b7c155bc0 0x7f8b7c156040 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:32.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.766+0000 7f8b8122d700 1 --2- 172.21.15.17:0/492450393 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8b7c0b52e0 0x7f8b7c152fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:32.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.766+0000 7f8b8122d700 1 --2- 172.21.15.17:0/492450393 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8b5c003750 0x7f8b7c142480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:32.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.766+0000 7f8b8122d700 1 -- 172.21.15.17:0/492450393 >> 172.21.15.17:0/492450393 conn(0x7f8b7c0612d0 msgr2=0x7f8b7c13cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:32.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.766+0000 7f8b8122d700 1 -- 172.21.15.17:0/492450393 shutdown_connections 2022-08-01T19:34:32.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.766+0000 7f8b8122d700 1 -- 172.21.15.17:0/492450393 wait complete. 2022-08-01T19:34:32.780 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T19:34:32.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.886+0000 7f07e169d700 1 Processor -- start 2022-08-01T19:34:32.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.886+0000 7f07e169d700 1 -- start start 2022-08-01T19:34:32.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.886+0000 7f07e169d700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f07dc0b31d0 0x7f07dc066020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:32.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.886+0000 7f07e169d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f07dc0a9da0 0x7f07dc060950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:32.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.886+0000 7f07e169d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f07dc1434b0 0x7f07dc143940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:32.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.886+0000 7f07e169d700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07dc064a70 con 0x7f07dc0a9da0 2022-08-01T19:34:32.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.886+0000 7f07e169d700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07dc061f60 con 0x7f07dc0b31d0 2022-08-01T19:34:32.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.886+0000 7f07e169d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc067030 con 0x7f07dc1434b0 2022-08-01T19:34:32.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.886+0000 7f07db7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f07dc0a9da0 0x7f07dc060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:32.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.886+0000 7f07dbfff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f07dc0b31d0 0x7f07dc066020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:32.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.886+0000 7f07db7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f07dc0a9da0 0x7f07dc060950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37146/0 (socket says 172.21.15.17:37146) 2022-08-01T19:34:32.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.886+0000 7f07db7fe700 1 -- 172.21.15.17:0/4007396921 learned_addr learned my addr 172.21.15.17:0/4007396921 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:32.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.886+0000 7f07e0e9c700 1 --2- 172.21.15.17:0/4007396921 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f07dc1434b0 0x7f07dc143940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:32.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.886+0000 7f07db7fe700 1 -- 172.21.15.17:0/4007396921 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f07dc1434b0 msgr2=0x7f07dc143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:32.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.886+0000 7f07db7fe700 1 --2- 172.21.15.17:0/4007396921 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f07dc1434b0 0x7f07dc143940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:32.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.886+0000 7f07db7fe700 1 -- 172.21.15.17:0/4007396921 >> v2:172.21.15.77:3300/0 conn(0x7f07dc0b31d0 msgr2=0x7f07dc066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:32.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.886+0000 7f07db7fe700 1 --2- 172.21.15.17:0/4007396921 >> v2:172.21.15.77:3300/0 conn(0x7f07dc0b31d0 0x7f07dc066020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:32.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.886+0000 7f07db7fe700 1 -- 172.21.15.17:0/4007396921 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07dc0b35b0 con 0x7f07dc0a9da0 2022-08-01T19:34:32.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.886+0000 7f07dbfff700 1 --2- 172.21.15.17:0/4007396921 >> v2:172.21.15.77:3300/0 conn(0x7f07dc0b31d0 0x7f07dc066020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:34:32.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.886+0000 7f07db7fe700 1 --2- 172.21.15.17:0/4007396921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07dc0a9da0 0x7f07dc060950 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f07d0002f70 tx=0x7f07d0006eb0).ready entity=mon.0 client_cookie=422bdfcf943a807 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:32.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.886+0000 7f07daffd700 1 -- 172.21.15.17:0/4007396921 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07d000d570 con 0x7f07dc0a9da0 2022-08-01T19:34:32.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.886+0000 7f07daffd700 1 -- 172.21.15.17:0/4007396921 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d0000e90 con 0x7f07dc0a9da0 2022-08-01T19:34:32.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.886+0000 7f07daffd700 1 -- 172.21.15.17:0/4007396921 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07d000d900 con 0x7f07dc0a9da0 2022-08-01T19:34:32.902 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.890+0000 7f07e169d700 1 -- 172.21.15.17:0/4007396921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07dc0a9da0 msgr2=0x7f07dc060950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:32.902 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.890+0000 7f07e169d700 1 --2- 172.21.15.17:0/4007396921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07dc0a9da0 0x7f07dc060950 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f07d0002f70 tx=0x7f07d0006eb0).stop 2022-08-01T19:34:32.902 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.890+0000 7f07e169d700 1 -- 172.21.15.17:0/4007396921 shutdown_connections 2022-08-01T19:34:32.903 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.890+0000 7f07e169d700 1 --2- 172.21.15.17:0/4007396921 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f07dc1434b0 0x7f07dc143940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:32.903 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.890+0000 7f07e169d700 1 --2- 172.21.15.17:0/4007396921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07dc0a9da0 0x7f07dc060950 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:32.903 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.890+0000 7f07e169d700 1 --2- 172.21.15.17:0/4007396921 >> v2:172.21.15.77:3300/0 conn(0x7f07dc0b31d0 0x7f07dc066020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:32.903 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.890+0000 7f07e169d700 1 -- 172.21.15.17:0/4007396921 >> 172.21.15.17:0/4007396921 conn(0x7f07dc063870 msgr2=0x7f07dc063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:32.903 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.890+0000 7f07e169d700 1 -- 172.21.15.17:0/4007396921 shutdown_connections 2022-08-01T19:34:32.904 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.890+0000 7f07e169d700 1 -- 172.21.15.17:0/4007396921 wait complete. 2022-08-01T19:34:32.904 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.890+0000 7f07e169d700 1 Processor -- start 2022-08-01T19:34:32.904 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.890+0000 7f07e169d700 1 -- start start 2022-08-01T19:34:32.904 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.894+0000 7f07e169d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07dc0a9da0 0x7f07dc05fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:32.905 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.894+0000 7f07e169d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07dc0b31d0 0x7f07dc0b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:32.905 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.894+0000 7f07e169d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f07dc1434b0 0x7f07dc13a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:32.905 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.894+0000 7f07e169d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc145560 con 0x7f07dc0a9da0 2022-08-01T19:34:32.905 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.894+0000 7f07e169d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc145800 con 0x7f07dc0b31d0 2022-08-01T19:34:32.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.894+0000 7f07e169d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc145aa0 con 0x7f07dc1434b0 2022-08-01T19:34:32.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.894+0000 7f07dbfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07dc0a9da0 0x7f07dc05fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:32.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.894+0000 7f07dbfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07dc0a9da0 0x7f07dc05fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37150/0 (socket says 172.21.15.17:37150) 2022-08-01T19:34:32.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.894+0000 7f07dbfff700 1 -- 172.21.15.17:0/1263418868 learned_addr learned my addr 172.21.15.17:0/1263418868 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:32.907 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.894+0000 7f07e0e9c700 1 --2- 172.21.15.17:0/1263418868 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f07dc1434b0 0x7f07dc13a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:32.907 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.894+0000 7f07db7fe700 1 --2- 172.21.15.17:0/1263418868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07dc0b31d0 0x7f07dc0b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:32.907 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.894+0000 7f07dbfff700 1 -- 172.21.15.17:0/1263418868 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f07dc1434b0 msgr2=0x7f07dc13a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:32.907 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.894+0000 7f07dbfff700 1 --2- 172.21.15.17:0/1263418868 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f07dc1434b0 0x7f07dc13a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:32.908 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.894+0000 7f07dbfff700 1 -- 172.21.15.17:0/1263418868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07dc0b31d0 msgr2=0x7f07dc0b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:32.908 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.894+0000 7f07dbfff700 1 --2- 172.21.15.17:0/1263418868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07dc0b31d0 0x7f07dc0b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:32.908 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.894+0000 7f07dbfff700 1 -- 172.21.15.17:0/1263418868 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07cc00b180 con 0x7f07dc0a9da0 2022-08-01T19:34:32.908 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.894+0000 7f07dbfff700 1 --2- 172.21.15.17:0/1263418868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07dc0a9da0 0x7f07dc05fe10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f07cc00c000 tx=0x7f07cc006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:32.908 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.894+0000 7f07d97fa700 1 -- 172.21.15.17:0/1263418868 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07cc00c7e0 con 0x7f07dc0a9da0 2022-08-01T19:34:32.909 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.894+0000 7f07d97fa700 1 -- 172.21.15.17:0/1263418868 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07cc00c940 con 0x7f07dc0a9da0 2022-08-01T19:34:32.909 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.894+0000 7f07d97fa700 1 -- 172.21.15.17:0/1263418868 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07cc00cca0 con 0x7f07dc0a9da0 2022-08-01T19:34:32.909 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.894+0000 7f07e169d700 1 -- 172.21.15.17:0/1263418868 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07d000b1e0 con 0x7f07dc0a9da0 2022-08-01T19:34:32.909 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.894+0000 7f07e169d700 1 -- 172.21.15.17:0/1263418868 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07dc060400 con 0x7f07dc0a9da0 2022-08-01T19:34:32.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.898+0000 7f07e169d700 1 -- 172.21.15.17:0/1263418868 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07dc066af0 con 0x7f07dc0a9da0 2022-08-01T19:34:32.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.898+0000 7f07d97fa700 1 -- 172.21.15.17:0/1263418868 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f07cc01c460 con 0x7f07dc0a9da0 2022-08-01T19:34:32.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.898+0000 7f07d97fa700 1 --2- 172.21.15.17:0/1263418868 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f07bc0539e0 0x7f07bc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:32.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.898+0000 7f07d97fa700 1 -- 172.21.15.17:0/1263418868 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f07cc06c720 con 0x7f07dc0a9da0 2022-08-01T19:34:32.912 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.898+0000 7f07db7fe700 1 --2- 172.21.15.17:0/1263418868 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f07bc0539e0 0x7f07bc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:32.912 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.898+0000 7f07db7fe700 1 --2- 172.21.15.17:0/1263418868 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f07bc0539e0 0x7f07bc055ea0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f07d001c000 tx=0x7f07d0015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:32.913 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:32.906+0000 7f07d97fa700 1 -- 172.21.15.17:0/1263418868 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f07cc043370 con 0x7f07dc0a9da0 2022-08-01T19:34:33.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.026+0000 7f07e169d700 1 -- 172.21.15.17:0/1263418868 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f07dc0574e0 con 0x7f07dc0a9da0 2022-08-01T19:34:33.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.030+0000 7f07d97fa700 1 -- 172.21.15.17:0/1263418868 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+10 (secure 0 0 0) 0x7f07cc00ce00 con 0x7f07dc0a9da0 2022-08-01T19:34:33.035 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:34:33.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.030+0000 7f07e169d700 1 -- 172.21.15.17:0/1263418868 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f07bc0539e0 msgr2=0x7f07bc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:33.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.030+0000 7f07e169d700 1 --2- 172.21.15.17:0/1263418868 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f07bc0539e0 0x7f07bc055ea0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f07d001c000 tx=0x7f07d0015000).stop 2022-08-01T19:34:33.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.030+0000 7f07e169d700 1 -- 172.21.15.17:0/1263418868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07dc0a9da0 msgr2=0x7f07dc05fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:33.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.030+0000 7f07e169d700 1 --2- 172.21.15.17:0/1263418868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07dc0a9da0 0x7f07dc05fe10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f07cc00c000 tx=0x7f07cc006df0).stop 2022-08-01T19:34:33.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.030+0000 7f07e169d700 1 -- 172.21.15.17:0/1263418868 shutdown_connections 2022-08-01T19:34:33.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.030+0000 7f07e169d700 1 --2- 172.21.15.17:0/1263418868 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f07bc0539e0 0x7f07bc055ea0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:33.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.030+0000 7f07e169d700 1 --2- 172.21.15.17:0/1263418868 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f07dc1434b0 0x7f07dc13a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:33.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.030+0000 7f07e169d700 1 --2- 172.21.15.17:0/1263418868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07dc0b31d0 0x7f07dc0b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:33.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.030+0000 7f07e169d700 1 --2- 172.21.15.17:0/1263418868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07dc0a9da0 0x7f07dc05fe10 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:33.040 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.030+0000 7f07e169d700 1 -- 172.21.15.17:0/1263418868 >> 172.21.15.17:0/1263418868 conn(0x7f07dc063870 msgr2=0x7f07dc063c50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:33.040 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.030+0000 7f07e169d700 1 -- 172.21.15.17:0/1263418868 shutdown_connections 2022-08-01T19:34:33.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.030+0000 7f07e169d700 1 -- 172.21.15.17:0/1263418868 wait complete. 2022-08-01T19:34:33.049 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:34:33.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.142+0000 7f48cda90700 1 Processor -- start 2022-08-01T19:34:33.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.142+0000 7f48cda90700 1 -- start start 2022-08-01T19:34:33.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.146+0000 7f48cda90700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f48c80617e0 0x7f48c8066760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:33.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.146+0000 7f48cda90700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f48c8066c50 0x7f48c814db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:33.154 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.146+0000 7f48cda90700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48c8060670 con 0x7f48c8060290 2022-08-01T19:34:33.154 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.146+0000 7f48cda90700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48c8060e00 con 0x7f48c80617e0 2022-08-01T19:34:33.154 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.146+0000 7f48cda90700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48c80610a0 con 0x7f48c8066c50 2022-08-01T19:34:33.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.146+0000 7f48c6ffd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f48c80617e0 0x7f48c8066760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:33.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.146+0000 7f48c6ffd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f48c80617e0 0x7f48c8066760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37160/0 (socket says 172.21.15.17:37160) 2022-08-01T19:34:33.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.146+0000 7f48c6ffd700 1 -- 172.21.15.17:0/2232087882 learned_addr learned my addr 172.21.15.17:0/2232087882 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:33.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.146+0000 7f48c7fff700 1 --2- 172.21.15.17:0/2232087882 >> v2:172.21.15.77:3300/0 conn(0x7f48c8066c50 0x7f48c814db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:33.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.146+0000 7f48c6ffd700 1 -- 172.21.15.17:0/2232087882 >> v2:172.21.15.77:3300/0 conn(0x7f48c8066c50 msgr2=0x7f48c814db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:33.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.146+0000 7f48c6ffd700 1 --2- 172.21.15.17:0/2232087882 >> v2:172.21.15.77:3300/0 conn(0x7f48c8066c50 0x7f48c814db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:33.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.146+0000 7f48c6ffd700 1 -- 172.21.15.17:0/2232087882 >> v1:172.21.15.77:6789/0 conn(0x7f48c8060290 legacy=0x7f48c8065b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:33.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.146+0000 7f48c6ffd700 1 -- 172.21.15.17:0/2232087882 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c8061340 con 0x7f48c80617e0 2022-08-01T19:34:33.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.146+0000 7f48c6ffd700 1 --2- 172.21.15.17:0/2232087882 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48c80617e0 0x7f48c8066760 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f48c80aeac0 tx=0x7f48b8006eb0).ready entity=mon.0 client_cookie=ec7ee6234c7633d6 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:33.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.146+0000 7f48c67fc700 1 -- 172.21.15.17:0/2232087882 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48b800d5c0 con 0x7f48c80617e0 2022-08-01T19:34:33.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.146+0000 7f48c67fc700 1 -- 172.21.15.17:0/2232087882 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48b8000e90 con 0x7f48c80617e0 2022-08-01T19:34:33.158 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.146+0000 7f48c67fc700 1 -- 172.21.15.17:0/2232087882 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48b800d910 con 0x7f48c80617e0 2022-08-01T19:34:33.158 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.146+0000 7f48cda90700 1 -- 172.21.15.17:0/2232087882 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48c80617e0 msgr2=0x7f48c8066760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:33.158 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.146+0000 7f48cda90700 1 --2- 172.21.15.17:0/2232087882 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48c80617e0 0x7f48c8066760 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f48c80aeac0 tx=0x7f48b8006eb0).stop 2022-08-01T19:34:33.158 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.146+0000 7f48cda90700 1 -- 172.21.15.17:0/2232087882 shutdown_connections 2022-08-01T19:34:33.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.146+0000 7f48cda90700 1 --2- 172.21.15.17:0/2232087882 >> v2:172.21.15.77:3300/0 conn(0x7f48c8066c50 0x7f48c814db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:33.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.146+0000 7f48cda90700 1 --2- 172.21.15.17:0/2232087882 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48c80617e0 0x7f48c8066760 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:33.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.146+0000 7f48cda90700 1 -- 172.21.15.17:0/2232087882 >> 172.21.15.17:0/2232087882 conn(0x7f48c805d0c0 msgr2=0x7f48c805f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:33.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.146+0000 7f48cda90700 1 -- 172.21.15.17:0/2232087882 shutdown_connections 2022-08-01T19:34:33.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.146+0000 7f48cda90700 1 -- 172.21.15.17:0/2232087882 wait complete. 2022-08-01T19:34:33.163 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.150+0000 7f48cda90700 1 Processor -- start 2022-08-01T19:34:33.163 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.150+0000 7f48cda90700 1 -- start start 2022-08-01T19:34:33.163 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.150+0000 7f48cda90700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48c8060290 0x7f48c80b4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:33.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.150+0000 7f48cda90700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48c80617e0 0x7f48c8152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:33.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.150+0000 7f48cda90700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f48c8066c50 0x7f48c81531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:33.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.150+0000 7f48cda90700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c8146d20 con 0x7f48c8060290 2022-08-01T19:34:33.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.150+0000 7f48cda90700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c81527a0 con 0x7f48c80617e0 2022-08-01T19:34:33.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.150+0000 7f48cda90700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48c80b03b0 con 0x7f48c8066c50 2022-08-01T19:34:33.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.150+0000 7f48c77fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48c8060290 0x7f48c80b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:33.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.150+0000 7f48c77fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48c8060290 0x7f48c80b4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37164/0 (socket says 172.21.15.17:37164) 2022-08-01T19:34:33.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.150+0000 7f48c77fe700 1 -- 172.21.15.17:0/1434013994 learned_addr learned my addr 172.21.15.17:0/1434013994 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:33.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.150+0000 7f48c7fff700 1 --2- 172.21.15.17:0/1434013994 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f48c8066c50 0x7f48c81531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:33.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.150+0000 7f48c6ffd700 1 --2- 172.21.15.17:0/1434013994 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48c80617e0 0x7f48c8152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:33.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.150+0000 7f48c77fe700 1 -- 172.21.15.17:0/1434013994 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f48c8066c50 msgr2=0x7f48c81531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:33.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.150+0000 7f48c77fe700 1 --2- 172.21.15.17:0/1434013994 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f48c8066c50 0x7f48c81531d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:33.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.150+0000 7f48c77fe700 1 -- 172.21.15.17:0/1434013994 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48c80617e0 msgr2=0x7f48c8152ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:33.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.150+0000 7f48c77fe700 1 --2- 172.21.15.17:0/1434013994 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48c80617e0 0x7f48c8152ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:33.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.150+0000 7f48c77fe700 1 -- 172.21.15.17:0/1434013994 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b800b1e0 con 0x7f48c8060290 2022-08-01T19:34:33.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.150+0000 7f48c77fe700 1 --2- 172.21.15.17:0/1434013994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48c8060290 0x7f48c80b4360 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f48c80a42a0 tx=0x7f48c0007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:33.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.150+0000 7f48c4ff9700 1 -- 172.21.15.17:0/1434013994 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48c000ce70 con 0x7f48c8060290 2022-08-01T19:34:33.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.150+0000 7f48c4ff9700 1 -- 172.21.15.17:0/1434013994 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c0007240 con 0x7f48c8060290 2022-08-01T19:34:33.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.150+0000 7f48cda90700 1 -- 172.21.15.17:0/1434013994 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c80b0650 con 0x7f48c8060290 2022-08-01T19:34:33.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.150+0000 7f48c4ff9700 1 -- 172.21.15.17:0/1434013994 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48c0006510 con 0x7f48c8060290 2022-08-01T19:34:33.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.150+0000 7f48cda90700 1 -- 172.21.15.17:0/1434013994 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c80b0b90 con 0x7f48c8060290 2022-08-01T19:34:33.169 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.154+0000 7f48cda90700 1 -- 172.21.15.17:0/1434013994 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48c80b0d20 con 0x7f48c8060290 2022-08-01T19:34:33.169 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.154+0000 7f48c4ff9700 1 -- 172.21.15.17:0/1434013994 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f48c0010030 con 0x7f48c8060290 2022-08-01T19:34:33.169 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.154+0000 7f48c4ff9700 1 --2- 172.21.15.17:0/1434013994 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f48ac0539e0 0x7f48ac055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:33.169 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.154+0000 7f48c4ff9700 1 -- 172.21.15.17:0/1434013994 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f48c006bea0 con 0x7f48c8060290 2022-08-01T19:34:33.170 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.154+0000 7f48c6ffd700 1 --2- 172.21.15.17:0/1434013994 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f48ac0539e0 0x7f48ac055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:33.170 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.154+0000 7f48c6ffd700 1 --2- 172.21.15.17:0/1434013994 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f48ac0539e0 0x7f48ac055ea0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f48b801a000 tx=0x7f48b8015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:33.171 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.158+0000 7f48c4ff9700 1 -- 172.21.15.17:0/1434013994 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f48c0042bc0 con 0x7f48c8060290 2022-08-01T19:34:33.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.286+0000 7f48cda90700 1 -- 172.21.15.17:0/1434013994 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48c80574e0 con 0x7f48c8060290 2022-08-01T19:34:33.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.286+0000 7f48c4ff9700 1 -- 172.21.15.17:0/1434013994 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f48c00461e0 con 0x7f48c8060290 2022-08-01T19:34:33.297 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:33.297 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:34:09.692404+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:34:04.635473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-08-01T19:34:07.409434+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:34:33.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.290+0000 7f48cda90700 1 -- 172.21.15.17:0/1434013994 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f48ac0539e0 msgr2=0x7f48ac055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:33.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.290+0000 7f48cda90700 1 --2- 172.21.15.17:0/1434013994 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f48ac0539e0 0x7f48ac055ea0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f48b801a000 tx=0x7f48b8015000).stop 2022-08-01T19:34:33.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.290+0000 7f48cda90700 1 -- 172.21.15.17:0/1434013994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48c8060290 msgr2=0x7f48c80b4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:33.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.290+0000 7f48cda90700 1 --2- 172.21.15.17:0/1434013994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48c8060290 0x7f48c80b4360 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f48c80a42a0 tx=0x7f48c0007c30).stop 2022-08-01T19:34:33.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.290+0000 7f48cda90700 1 -- 172.21.15.17:0/1434013994 shutdown_connections 2022-08-01T19:34:33.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.290+0000 7f48cda90700 1 --2- 172.21.15.17:0/1434013994 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f48ac0539e0 0x7f48ac055ea0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:33.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.290+0000 7f48cda90700 1 --2- 172.21.15.17:0/1434013994 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f48c8066c50 0x7f48c81531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:33.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.290+0000 7f48cda90700 1 --2- 172.21.15.17:0/1434013994 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48c80617e0 0x7f48c8152ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:33.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.290+0000 7f48cda90700 1 --2- 172.21.15.17:0/1434013994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48c8060290 0x7f48c80b4360 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:33.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.290+0000 7f48cda90700 1 -- 172.21.15.17:0/1434013994 >> 172.21.15.17:0/1434013994 conn(0x7f48c805d0c0 msgr2=0x7f48c80b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:33.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.290+0000 7f48cda90700 1 -- 172.21.15.17:0/1434013994 shutdown_connections 2022-08-01T19:34:33.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.290+0000 7f48cda90700 1 -- 172.21.15.17:0/1434013994 wait complete. 2022-08-01T19:34:33.306 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T19:34:04.635473+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T19:34:07.409434+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T19:34:33.307 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:34:33.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.418+0000 7fe6a359e700 1 Processor -- start 2022-08-01T19:34:33.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.418+0000 7fe6a359e700 1 -- start start 2022-08-01T19:34:33.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.418+0000 7fe6a359e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe6a40b3260 0x7fe6a41435c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:33.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.418+0000 7fe6a359e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe6a40ad000 0x7fe6a413fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:33.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.418+0000 7fe6a359e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6a4143ab0 con 0x7fe6a40b08a0 2022-08-01T19:34:33.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.418+0000 7fe6a359e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6a40b0c80 con 0x7fe6a40ad000 2022-08-01T19:34:33.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.418+0000 7fe6a359e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a40ae810 con 0x7fe6a40b3260 2022-08-01T19:34:33.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.418+0000 7fe6a2d9d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe6a40ad000 0x7fe6a413fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:33.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.418+0000 7fe6a2d9d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe6a40ad000 0x7fe6a413fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37176/0 (socket says 172.21.15.17:37176) 2022-08-01T19:34:33.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.418+0000 7fe6a2d9d700 1 -- 172.21.15.17:0/1158758440 learned_addr learned my addr 172.21.15.17:0/1158758440 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:33.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.418+0000 7fe69bfff700 1 --2- 172.21.15.17:0/1158758440 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe6a40b3260 0x7fe6a41435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:33.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.418+0000 7fe6a2d9d700 1 -- 172.21.15.17:0/1158758440 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe6a40b3260 msgr2=0x7fe6a41435c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:33.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.418+0000 7fe6a2d9d700 1 --2- 172.21.15.17:0/1158758440 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe6a40b3260 0x7fe6a41435c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:33.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.418+0000 7fe6a2d9d700 1 -- 172.21.15.17:0/1158758440 >> v1:172.21.15.77:6789/0 conn(0x7fe6a40b08a0 legacy=0x7fe6a4142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:33.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.418+0000 7fe6a2d9d700 1 -- 172.21.15.17:0/1158758440 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a40aeab0 con 0x7fe6a40ad000 2022-08-01T19:34:33.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.418+0000 7fe6a1d9b700 1 -- 172.21.15.17:0/1158758440 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3622848687 0 0) 0x7fe6a4143ab0 con 0x7fe6a40b08a0 2022-08-01T19:34:33.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.418+0000 7fe6a2d9d700 1 --2- 172.21.15.17:0/1158758440 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6a40ad000 0x7fe6a413fa30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7fe69c0032f0 tx=0x7fe69c009290).ready entity=mon.0 client_cookie=cc0df49720355a72 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:33.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.418+0000 7fe6a1d9b700 1 -- 172.21.15.17:0/1158758440 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe69c00b8a0 con 0x7fe6a40ad000 2022-08-01T19:34:33.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.418+0000 7fe6a1d9b700 1 -- 172.21.15.17:0/1158758440 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69c00ba00 con 0x7fe6a40ad000 2022-08-01T19:34:33.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.418+0000 7fe6a1d9b700 1 -- 172.21.15.17:0/1158758440 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe69c00bb70 con 0x7fe6a40ad000 2022-08-01T19:34:33.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.418+0000 7fe6a359e700 1 -- 172.21.15.17:0/1158758440 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6a40ad000 msgr2=0x7fe6a413fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:33.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.418+0000 7fe6a359e700 1 --2- 172.21.15.17:0/1158758440 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6a40ad000 0x7fe6a413fa30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7fe69c0032f0 tx=0x7fe69c009290).stop 2022-08-01T19:34:33.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.418+0000 7fe6a359e700 1 -- 172.21.15.17:0/1158758440 shutdown_connections 2022-08-01T19:34:33.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.418+0000 7fe6a359e700 1 --2- 172.21.15.17:0/1158758440 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6a40ad000 0x7fe6a413fa30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:33.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.418+0000 7fe6a359e700 1 --2- 172.21.15.17:0/1158758440 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe6a40b3260 0x7fe6a41435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:33.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.418+0000 7fe6a359e700 1 -- 172.21.15.17:0/1158758440 >> 172.21.15.17:0/1158758440 conn(0x7fe6a40612d0 msgr2=0x7fe6a4062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:33.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.418+0000 7fe6a359e700 1 -- 172.21.15.17:0/1158758440 shutdown_connections 2022-08-01T19:34:33.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.418+0000 7fe6a359e700 1 -- 172.21.15.17:0/1158758440 wait complete. 2022-08-01T19:34:33.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.422+0000 7fe6a359e700 1 Processor -- start 2022-08-01T19:34:33.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.422+0000 7fe6a359e700 1 -- start start 2022-08-01T19:34:33.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.422+0000 7fe6a359e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6a40ad000 0x7fe6a4067a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:33.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.422+0000 7fe6a359e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe6a40b08a0 0x7fe6a40b64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:33.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.422+0000 7fe6a359e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6a40b3260 0x7fe6a40ac4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:33.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.422+0000 7fe6a359e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a40a9f60 con 0x7fe6a40b3260 2022-08-01T19:34:33.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.422+0000 7fe6a359e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a4065810 con 0x7fe6a40ad000 2022-08-01T19:34:33.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.422+0000 7fe6a359e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a4065ab0 con 0x7fe6a40b08a0 2022-08-01T19:34:33.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.422+0000 7fe6a2d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6a40b3260 0x7fe6a40ac4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:33.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.422+0000 7fe6a2d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6a40b3260 0x7fe6a40ac4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37182/0 (socket says 172.21.15.17:37182) 2022-08-01T19:34:33.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.422+0000 7fe6a2d9d700 1 -- 172.21.15.17:0/3965029748 learned_addr learned my addr 172.21.15.17:0/3965029748 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:33.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.422+0000 7fe69bfff700 1 --2- 172.21.15.17:0/3965029748 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe6a40b08a0 0x7fe6a40b64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:33.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.422+0000 7fe6a259c700 1 --2- 172.21.15.17:0/3965029748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6a40ad000 0x7fe6a4067a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:33.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.426+0000 7fe6a2d9d700 1 -- 172.21.15.17:0/3965029748 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe6a40b08a0 msgr2=0x7fe6a40b64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:33.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.426+0000 7fe6a2d9d700 1 --2- 172.21.15.17:0/3965029748 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe6a40b08a0 0x7fe6a40b64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:33.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.426+0000 7fe6a2d9d700 1 -- 172.21.15.17:0/3965029748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6a40ad000 msgr2=0x7fe6a4067a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:33.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.426+0000 7fe6a2d9d700 1 --2- 172.21.15.17:0/3965029748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6a40ad000 0x7fe6a4067a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:33.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.426+0000 7fe6a2d9d700 1 -- 172.21.15.17:0/3965029748 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe69c009530 con 0x7fe6a40b3260 2022-08-01T19:34:33.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.426+0000 7fe6a2d9d700 1 --2- 172.21.15.17:0/3965029748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6a40b3260 0x7fe6a40ac4d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7fe69c0032f0 tx=0x7fe69c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:33.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.426+0000 7fe69b7fe700 1 -- 172.21.15.17:0/3965029748 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe69c00bb70 con 0x7fe6a40b3260 2022-08-01T19:34:33.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.426+0000 7fe69b7fe700 1 -- 172.21.15.17:0/3965029748 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69c021630 con 0x7fe6a40b3260 2022-08-01T19:34:33.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.426+0000 7fe6a359e700 1 -- 172.21.15.17:0/3965029748 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6a4065d50 con 0x7fe6a40b3260 2022-08-01T19:34:33.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.426+0000 7fe69b7fe700 1 -- 172.21.15.17:0/3965029748 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe69c00b8a0 con 0x7fe6a40b3260 2022-08-01T19:34:33.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.426+0000 7fe6a359e700 1 -- 172.21.15.17:0/3965029748 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a413adb0 con 0x7fe6a40b3260 2022-08-01T19:34:33.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.426+0000 7fe6a359e700 1 -- 172.21.15.17:0/3965029748 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6a40574e0 con 0x7fe6a40b3260 2022-08-01T19:34:33.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.426+0000 7fe69b7fe700 1 -- 172.21.15.17:0/3965029748 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe69c00f070 con 0x7fe6a40b3260 2022-08-01T19:34:33.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.426+0000 7fe69b7fe700 1 --2- 172.21.15.17:0/3965029748 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe688053aa0 0x7fe688055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:33.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.426+0000 7fe69b7fe700 1 -- 172.21.15.17:0/3965029748 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fe69c06cf50 con 0x7fe6a40b3260 2022-08-01T19:34:33.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.426+0000 7fe6a259c700 1 --2- 172.21.15.17:0/3965029748 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe688053aa0 0x7fe688055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:33.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.430+0000 7fe6a259c700 1 --2- 172.21.15.17:0/3965029748 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe688053aa0 0x7fe688055f60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fe69400d000 tx=0x7fe6940079b0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:33.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.434+0000 7fe69b7fe700 1 -- 172.21.15.17:0/3965029748 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe69c043d10 con 0x7fe6a40b3260 2022-08-01T19:34:33.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.570+0000 7fe6a359e700 1 -- 172.21.15.17:0/3965029748 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe6a40abfc0 con 0x7fe6a40b3260 2022-08-01T19:34:33.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.570+0000 7fe69b7fe700 1 -- 172.21.15.17:0/3965029748 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v25) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe69c047330 con 0x7fe6a40b3260 2022-08-01T19:34:33.575 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:34:33.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.570+0000 7fe6a359e700 1 -- 172.21.15.17:0/3965029748 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe688053aa0 msgr2=0x7fe688055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:33.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.570+0000 7fe6a359e700 1 --2- 172.21.15.17:0/3965029748 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe688053aa0 0x7fe688055f60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fe69400d000 tx=0x7fe6940079b0).stop 2022-08-01T19:34:33.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.570+0000 7fe6a359e700 1 -- 172.21.15.17:0/3965029748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6a40b3260 msgr2=0x7fe6a40ac4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:33.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.570+0000 7fe6a359e700 1 --2- 172.21.15.17:0/3965029748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6a40b3260 0x7fe6a40ac4d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7fe69c0032f0 tx=0x7fe69c016000).stop 2022-08-01T19:34:33.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.570+0000 7fe6a359e700 1 -- 172.21.15.17:0/3965029748 shutdown_connections 2022-08-01T19:34:33.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.570+0000 7fe6a359e700 1 --2- 172.21.15.17:0/3965029748 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe688053aa0 0x7fe688055f60 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:33.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.570+0000 7fe6a359e700 1 --2- 172.21.15.17:0/3965029748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6a40b3260 0x7fe6a40ac4d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:33.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.570+0000 7fe6a359e700 1 --2- 172.21.15.17:0/3965029748 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe6a40b08a0 0x7fe6a40b64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:33.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.570+0000 7fe6a359e700 1 --2- 172.21.15.17:0/3965029748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6a40ad000 0x7fe6a4067a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:33.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.570+0000 7fe6a359e700 1 -- 172.21.15.17:0/3965029748 >> 172.21.15.17:0/3965029748 conn(0x7fe6a40612d0 msgr2=0x7fe6a4141330 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:33.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.570+0000 7fe6a359e700 1 -- 172.21.15.17:0/3965029748 shutdown_connections 2022-08-01T19:34:33.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.570+0000 7fe6a359e700 1 -- 172.21.15.17:0/3965029748 wait complete. 2022-08-01T19:34:33.588 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:34:33.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.694+0000 7f90d17d8700 1 Processor -- start 2022-08-01T19:34:33.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.694+0000 7f90d17d8700 1 -- start start 2022-08-01T19:34:33.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.698+0000 7f90d17d8700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f90cc0b28b0 0x7f90cc13e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:33.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.698+0000 7f90d17d8700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f90cc141920 0x7f90cc13e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:33.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.698+0000 7f90d17d8700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90cc0aac20 con 0x7f90cc0b1180 2022-08-01T19:34:33.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.698+0000 7f90d17d8700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90cc141db0 con 0x7f90cc141920 2022-08-01T19:34:33.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.698+0000 7f90d17d8700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90cc140e20 con 0x7f90cc0b28b0 2022-08-01T19:34:33.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.698+0000 7f90d0fd7700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f90cc141920 0x7f90cc13e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:33.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.698+0000 7f90d0fd7700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f90cc141920 0x7f90cc13e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37190/0 (socket says 172.21.15.17:37190) 2022-08-01T19:34:33.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.698+0000 7f90cb7fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f90cc0b1180 0x7f90cc141210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57612/0 (socket says 172.21.15.17:57612) 2022-08-01T19:34:33.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.698+0000 7f90d0fd7700 1 -- 172.21.15.17:0/4220937280 learned_addr learned my addr 172.21.15.17:0/4220937280 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:33.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.698+0000 7f90cbfff700 1 --2- 172.21.15.17:0/4220937280 >> v2:172.21.15.77:3300/0 conn(0x7f90cc0b28b0 0x7f90cc13e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:33.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.698+0000 7f90d0fd7700 1 -- 172.21.15.17:0/4220937280 >> v2:172.21.15.77:3300/0 conn(0x7f90cc0b28b0 msgr2=0x7f90cc13e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:33.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.698+0000 7f90d0fd7700 1 --2- 172.21.15.17:0/4220937280 >> v2:172.21.15.77:3300/0 conn(0x7f90cc0b28b0 0x7f90cc13e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:33.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.698+0000 7f90d0fd7700 1 -- 172.21.15.17:0/4220937280 >> v1:172.21.15.17:6789/0 conn(0x7f90cc0b1180 legacy=0x7f90cc141210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:33.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.698+0000 7f90d0fd7700 1 -- 172.21.15.17:0/4220937280 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90cc0b2c90 con 0x7f90cc141920 2022-08-01T19:34:33.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.698+0000 7f90caffd700 1 -- 172.21.15.17:0/4220937280 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 833042016 0 0) 0x7f90bc001110 con 0x7f90cc0b1180 2022-08-01T19:34:33.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.698+0000 7f90d0fd7700 1 --2- 172.21.15.17:0/4220937280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90cc141920 0x7f90cc13e6b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f90c40032f0 tx=0x7f90c4009290).ready entity=mon.0 client_cookie=cef921c1d04d6cdf server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:33.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.698+0000 7f90caffd700 1 -- 172.21.15.17:0/4220937280 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90c400b870 con 0x7f90cc141920 2022-08-01T19:34:33.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.698+0000 7f90caffd700 1 -- 172.21.15.17:0/4220937280 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c400b9d0 con 0x7f90cc141920 2022-08-01T19:34:33.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.698+0000 7f90caffd700 1 -- 172.21.15.17:0/4220937280 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90c400bb40 con 0x7f90cc141920 2022-08-01T19:34:33.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.698+0000 7f90d17d8700 1 -- 172.21.15.17:0/4220937280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90cc141920 msgr2=0x7f90cc13e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:33.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.698+0000 7f90d17d8700 1 --2- 172.21.15.17:0/4220937280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90cc141920 0x7f90cc13e6b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f90c40032f0 tx=0x7f90c4009290).stop 2022-08-01T19:34:33.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.698+0000 7f90d17d8700 1 -- 172.21.15.17:0/4220937280 shutdown_connections 2022-08-01T19:34:33.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.698+0000 7f90d17d8700 1 --2- 172.21.15.17:0/4220937280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90cc141920 0x7f90cc13e6b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:33.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.698+0000 7f90d17d8700 1 --2- 172.21.15.17:0/4220937280 >> v2:172.21.15.77:3300/0 conn(0x7f90cc0b28b0 0x7f90cc13e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:33.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.698+0000 7f90d17d8700 1 -- 172.21.15.17:0/4220937280 >> 172.21.15.17:0/4220937280 conn(0x7f90cc060310 msgr2=0x7f90cc0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:33.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.698+0000 7f90d17d8700 1 -- 172.21.15.17:0/4220937280 shutdown_connections 2022-08-01T19:34:33.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.698+0000 7f90d17d8700 1 -- 172.21.15.17:0/4220937280 wait complete. 2022-08-01T19:34:33.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.702+0000 7f90d17d8700 1 Processor -- start 2022-08-01T19:34:33.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.702+0000 7f90d17d8700 1 -- start start 2022-08-01T19:34:33.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.702+0000 7f90d17d8700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f90cc0b1180 0x7f90cc155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:33.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.702+0000 7f90d17d8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90cc0b28b0 0x7f90cc156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:33.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.702+0000 7f90d17d8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90cc141920 0x7f90cc0b3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:33.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.702+0000 7f90d17d8700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc156720 con 0x7f90cc0b28b0 2022-08-01T19:34:33.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.702+0000 7f90d17d8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc0adca0 con 0x7f90cc141920 2022-08-01T19:34:33.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.702+0000 7f90d17d8700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc0ae660 con 0x7f90cc0b1180 2022-08-01T19:34:33.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.702+0000 7f90cb7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90cc0b28b0 0x7f90cc156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:33.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.702+0000 7f90cb7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90cc0b28b0 0x7f90cc156230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37194/0 (socket says 172.21.15.17:37194) 2022-08-01T19:34:33.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.702+0000 7f90cb7fe700 1 -- 172.21.15.17:0/2378915032 learned_addr learned my addr 172.21.15.17:0/2378915032 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:33.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.702+0000 7f90cbfff700 1 --2- 172.21.15.17:0/2378915032 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f90cc0b1180 0x7f90cc155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:33.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.702+0000 7f90d0fd7700 1 --2- 172.21.15.17:0/2378915032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90cc141920 0x7f90cc0b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:33.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.702+0000 7f90cb7fe700 1 -- 172.21.15.17:0/2378915032 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f90cc0b1180 msgr2=0x7f90cc155bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:33.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.702+0000 7f90cb7fe700 1 --2- 172.21.15.17:0/2378915032 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f90cc0b1180 0x7f90cc155bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:33.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.702+0000 7f90cb7fe700 1 -- 172.21.15.17:0/2378915032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90cc141920 msgr2=0x7f90cc0b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:33.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.702+0000 7f90cb7fe700 1 --2- 172.21.15.17:0/2378915032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90cc141920 0x7f90cc0b3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:33.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.702+0000 7f90cb7fe700 1 -- 172.21.15.17:0/2378915032 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c4009530 con 0x7f90cc0b28b0 2022-08-01T19:34:33.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.702+0000 7f90cbfff700 1 --2- 172.21.15.17:0/2378915032 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f90cc0b1180 0x7f90cc155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:34:33.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.702+0000 7f90d0fd7700 1 --2- 172.21.15.17:0/2378915032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90cc141920 0x7f90cc0b3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:34:33.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.702+0000 7f90cb7fe700 1 --2- 172.21.15.17:0/2378915032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90cc0b28b0 0x7f90cc156230 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f90bc000ed0 tx=0x7f90bc007920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:33.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.702+0000 7f90c97fa700 1 -- 172.21.15.17:0/2378915032 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90bc0072e0 con 0x7f90cc0b28b0 2022-08-01T19:34:33.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.702+0000 7f90c97fa700 1 -- 172.21.15.17:0/2378915032 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90bc005550 con 0x7f90cc0b28b0 2022-08-01T19:34:33.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.702+0000 7f90d17d8700 1 -- 172.21.15.17:0/2378915032 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90cc0acc80 con 0x7f90cc0b28b0 2022-08-01T19:34:33.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.702+0000 7f90c97fa700 1 -- 172.21.15.17:0/2378915032 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90bc0058b0 con 0x7f90cc0b28b0 2022-08-01T19:34:33.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.702+0000 7f90d17d8700 1 -- 172.21.15.17:0/2378915032 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90cc0adf00 con 0x7f90cc0b28b0 2022-08-01T19:34:33.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.706+0000 7f90d17d8700 1 -- 172.21.15.17:0/2378915032 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90cc0ae0c0 con 0x7f90cc0b28b0 2022-08-01T19:34:33.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.706+0000 7f90c97fa700 1 -- 172.21.15.17:0/2378915032 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f90bc010070 con 0x7f90cc0b28b0 2022-08-01T19:34:33.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.706+0000 7f90c97fa700 1 --2- 172.21.15.17:0/2378915032 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f90ac053a70 0x7f90ac055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:33.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.706+0000 7f90c97fa700 1 -- 172.21.15.17:0/2378915032 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f90bc06b7d0 con 0x7f90cc0b28b0 2022-08-01T19:34:33.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.706+0000 7f90cbfff700 1 --2- 172.21.15.17:0/2378915032 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f90ac053a70 0x7f90ac055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:33.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.706+0000 7f90cbfff700 1 --2- 172.21.15.17:0/2378915032 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f90ac053a70 0x7f90ac055f30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f90c0008000 tx=0x7f90c0008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:33.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.710+0000 7f90c97fa700 1 -- 172.21.15.17:0/2378915032 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f90bc042480 con 0x7f90cc0b28b0 2022-08-01T19:34:33.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.838+0000 7f90d17d8700 1 -- 172.21.15.17:0/2378915032 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f90cc0574e0 con 0x7f90cc0b28b0 2022-08-01T19:34:33.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.838+0000 7f90c97fa700 1 -- 172.21.15.17:0/2378915032 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f90bc045aa0 con 0x7f90cc0b28b0 2022-08-01T19:34:33.844 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:34:33.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.838+0000 7f90d17d8700 1 -- 172.21.15.17:0/2378915032 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f90ac053a70 msgr2=0x7f90ac055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:33.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.838+0000 7f90d17d8700 1 --2- 172.21.15.17:0/2378915032 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f90ac053a70 0x7f90ac055f30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f90c0008000 tx=0x7f90c0008040).stop 2022-08-01T19:34:33.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.838+0000 7f90d17d8700 1 -- 172.21.15.17:0/2378915032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90cc0b28b0 msgr2=0x7f90cc156230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:33.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.838+0000 7f90d17d8700 1 --2- 172.21.15.17:0/2378915032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90cc0b28b0 0x7f90cc156230 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f90bc000ed0 tx=0x7f90bc007920).stop 2022-08-01T19:34:33.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.838+0000 7f90d17d8700 1 -- 172.21.15.17:0/2378915032 shutdown_connections 2022-08-01T19:34:33.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.838+0000 7f90d17d8700 1 --2- 172.21.15.17:0/2378915032 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f90ac053a70 0x7f90ac055f30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:33.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.838+0000 7f90d17d8700 1 --2- 172.21.15.17:0/2378915032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90cc141920 0x7f90cc0b3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:33.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.838+0000 7f90d17d8700 1 --2- 172.21.15.17:0/2378915032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90cc0b28b0 0x7f90cc156230 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:33.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.838+0000 7f90d17d8700 1 --2- 172.21.15.17:0/2378915032 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f90cc0b1180 0x7f90cc155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:33.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.838+0000 7f90d17d8700 1 -- 172.21.15.17:0/2378915032 >> 172.21.15.17:0/2378915032 conn(0x7f90cc060310 msgr2=0x7f90cc13ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:33.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.838+0000 7f90d17d8700 1 -- 172.21.15.17:0/2378915032 shutdown_connections 2022-08-01T19:34:33.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.838+0000 7f90d17d8700 1 -- 172.21.15.17:0/2378915032 wait complete. 2022-08-01T19:34:33.857 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T19:34:33.966 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.962+0000 7f5517f8a700 1 Processor -- start 2022-08-01T19:34:33.966 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.962+0000 7f5517f8a700 1 -- start start 2022-08-01T19:34:33.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.962+0000 7f5517f8a700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f55100b4310 0x7f5510140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:33.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.962+0000 7f5517f8a700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5510143cf0 con 0x7f55100b4820 2022-08-01T19:34:33.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.962+0000 7f5517f8a700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5510143f90 con 0x7f5510146ae0 2022-08-01T19:34:33.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.962+0000 7f5517f8a700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55101476a0 con 0x7f55100b4310 2022-08-01T19:34:33.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.962+0000 7f5516527700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f55100b4820 0x7f55100b5060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57628/0 (socket says 172.21.15.17:57628) 2022-08-01T19:34:33.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.962+0000 7f5516527700 1 -- 172.21.15.17:0/2676830298 learned_addr learned my addr 172.21.15.17:0/2676830298 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:33.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.962+0000 7f5515525700 1 --2- 172.21.15.17:0/2676830298 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f55100b4310 0x7f5510140ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:33.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.962+0000 7f5514d24700 1 -- 172.21.15.17:0/2676830298 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 702761611 0 0) 0x7f5510143cf0 con 0x7f55100b4820 2022-08-01T19:34:33.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.962+0000 7f5514d24700 1 -- 172.21.15.17:0/2676830298 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54f8003260 con 0x7f55100b4820 2022-08-01T19:34:33.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.962+0000 7f5514d24700 1 -- 172.21.15.17:0/2676830298 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2646297559 0 0) 0x7f5510143f90 con 0x7f5510146ae0 2022-08-01T19:34:33.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.962+0000 7f5514d24700 1 -- 172.21.15.17:0/2676830298 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5510143cf0 con 0x7f5510146ae0 2022-08-01T19:34:33.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.962+0000 7f5515525700 1 -- 172.21.15.17:0/2676830298 >> v1:172.21.15.77:6789/0 conn(0x7f5510146ae0 legacy=0x7f55101403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:33.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.962+0000 7f5515525700 1 -- 172.21.15.17:0/2676830298 >> v1:172.21.15.17:6789/0 conn(0x7f55100b4820 legacy=0x7f55100b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:33.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.962+0000 7f5515525700 1 -- 172.21.15.17:0/2676830298 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55100b0e20 con 0x7f55100b4310 2022-08-01T19:34:33.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.962+0000 7f5515525700 1 --2- 172.21.15.17:0/2676830298 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f55100b4310 0x7f5510140ad0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f550c002f70 tx=0x7f550c006df0).ready entity=mon.2 client_cookie=af085c105f89cf00 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:33.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.962+0000 7f5514d24700 1 -- 172.21.15.17:0/2676830298 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f550c00d5c0 con 0x7f55100b4310 2022-08-01T19:34:33.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.962+0000 7f5514d24700 1 -- 172.21.15.17:0/2676830298 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f550c000e90 con 0x7f55100b4310 2022-08-01T19:34:33.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.962+0000 7f5514d24700 1 -- 172.21.15.17:0/2676830298 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f550c00d910 con 0x7f55100b4310 2022-08-01T19:34:33.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.962+0000 7f5517f8a700 1 -- 172.21.15.17:0/2676830298 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f55100b4310 msgr2=0x7f5510140ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:33.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.962+0000 7f5517f8a700 1 --2- 172.21.15.17:0/2676830298 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f55100b4310 0x7f5510140ad0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f550c002f70 tx=0x7f550c006df0).stop 2022-08-01T19:34:33.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.962+0000 7f5517f8a700 1 -- 172.21.15.17:0/2676830298 shutdown_connections 2022-08-01T19:34:33.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.962+0000 7f5517f8a700 1 --2- 172.21.15.17:0/2676830298 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f55100b4310 0x7f5510140ad0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:33.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.962+0000 7f5517f8a700 1 -- 172.21.15.17:0/2676830298 >> 172.21.15.17:0/2676830298 conn(0x7f551005c620 msgr2=0x7f551005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:33.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.962+0000 7f5517f8a700 1 -- 172.21.15.17:0/2676830298 shutdown_connections 2022-08-01T19:34:33.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.962+0000 7f5517f8a700 1 -- 172.21.15.17:0/2676830298 wait complete. 2022-08-01T19:34:33.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.966+0000 7f5517f8a700 1 Processor -- start 2022-08-01T19:34:33.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.966+0000 7f5517f8a700 1 -- start start 2022-08-01T19:34:33.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.966+0000 7f5517f8a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55100b4310 0x7f551014a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:33.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.966+0000 7f5517f8a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f55100b4820 0x7f5510152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:33.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.966+0000 7f5517f8a700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5510146ae0 0x7f5510142210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:33.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.966+0000 7f5517f8a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551005f130 con 0x7f55100b4820 2022-08-01T19:34:33.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.966+0000 7f5517f8a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5510142dd0 con 0x7f55100b4310 2022-08-01T19:34:33.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.966+0000 7f5517f8a700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5510143070 con 0x7f5510146ae0 2022-08-01T19:34:33.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.966+0000 7f5515525700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f55100b4820 0x7f5510152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:33.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.966+0000 7f5515525700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f55100b4820 0x7f5510152450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37208/0 (socket says 172.21.15.17:37208) 2022-08-01T19:34:33.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.966+0000 7f5515525700 1 -- 172.21.15.17:0/961293086 learned_addr learned my addr 172.21.15.17:0/961293086 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:33.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.966+0000 7f5515d26700 1 --2- 172.21.15.17:0/961293086 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55100b4310 0x7f551014a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:33.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.966+0000 7f5516527700 1 --2- 172.21.15.17:0/961293086 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5510146ae0 0x7f5510142210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:33.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.966+0000 7f5515525700 1 -- 172.21.15.17:0/961293086 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5510146ae0 msgr2=0x7f5510142210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:33.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.966+0000 7f5515525700 1 --2- 172.21.15.17:0/961293086 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5510146ae0 0x7f5510142210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:33.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.966+0000 7f5515525700 1 -- 172.21.15.17:0/961293086 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55100b4310 msgr2=0x7f551014a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:33.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.966+0000 7f5515525700 1 --2- 172.21.15.17:0/961293086 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55100b4310 0x7f551014a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:33.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.966+0000 7f5515525700 1 -- 172.21.15.17:0/961293086 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f550c00b180 con 0x7f55100b4820 2022-08-01T19:34:33.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.966+0000 7f5516527700 1 --2- 172.21.15.17:0/961293086 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5510146ae0 0x7f5510142210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:34:33.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.966+0000 7f5515525700 1 --2- 172.21.15.17:0/961293086 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f55100b4820 0x7f5510152450 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f550c002f70 tx=0x7f550c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:33.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.966+0000 7f5502ffd700 1 -- 172.21.15.17:0/961293086 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f550c01e400 con 0x7f55100b4820 2022-08-01T19:34:33.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.966+0000 7f5517f8a700 1 -- 172.21.15.17:0/961293086 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5510143310 con 0x7f55100b4820 2022-08-01T19:34:33.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.966+0000 7f5502ffd700 1 -- 172.21.15.17:0/961293086 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f550c01e560 con 0x7f55100b4820 2022-08-01T19:34:33.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.966+0000 7f5502ffd700 1 -- 172.21.15.17:0/961293086 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f550c01e8c0 con 0x7f55100b4820 2022-08-01T19:34:33.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.966+0000 7f5517f8a700 1 -- 172.21.15.17:0/961293086 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5510142700 con 0x7f55100b4820 2022-08-01T19:34:33.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.970+0000 7f5517f8a700 1 -- 172.21.15.17:0/961293086 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55101428f0 con 0x7f55100b4820 2022-08-01T19:34:33.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.970+0000 7f5502ffd700 1 -- 172.21.15.17:0/961293086 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f550c01c070 con 0x7f55100b4820 2022-08-01T19:34:33.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.970+0000 7f5502ffd700 1 --2- 172.21.15.17:0/961293086 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f54f0053950 0x7f54f0055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:33.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.970+0000 7f5502ffd700 1 -- 172.21.15.17:0/961293086 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f550c06cbd0 con 0x7f55100b4820 2022-08-01T19:34:33.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.970+0000 7f5515d26700 1 --2- 172.21.15.17:0/961293086 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f54f0053950 0x7f54f0055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:33.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.970+0000 7f5515d26700 1 --2- 172.21.15.17:0/961293086 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f54f0053950 0x7f54f0055e10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f550400b000 tx=0x7f5504009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:33.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:33.974+0000 7f5502ffd700 1 -- 172.21.15.17:0/961293086 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f550c043800 con 0x7f55100b4820 2022-08-01T19:34:34.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.098+0000 7f5517f8a700 1 -- 172.21.15.17:0/961293086 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f55100adcc0 con 0x7f55100b4820 2022-08-01T19:34:34.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.098+0000 7f5502ffd700 1 -- 172.21.15.17:0/961293086 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+10 (secure 0 0 0) 0x7f55100adcc0 con 0x7f55100b4820 2022-08-01T19:34:34.105 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:34:34.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.102+0000 7f5517f8a700 1 -- 172.21.15.17:0/961293086 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f54f0053950 msgr2=0x7f54f0055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:34.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.102+0000 7f5517f8a700 1 --2- 172.21.15.17:0/961293086 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f54f0053950 0x7f54f0055e10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f550400b000 tx=0x7f5504009040).stop 2022-08-01T19:34:34.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.102+0000 7f5517f8a700 1 -- 172.21.15.17:0/961293086 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f55100b4820 msgr2=0x7f5510152450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:34.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.102+0000 7f5517f8a700 1 --2- 172.21.15.17:0/961293086 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f55100b4820 0x7f5510152450 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f550c002f70 tx=0x7f550c014000).stop 2022-08-01T19:34:34.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.102+0000 7f5517f8a700 1 -- 172.21.15.17:0/961293086 shutdown_connections 2022-08-01T19:34:34.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.102+0000 7f5517f8a700 1 --2- 172.21.15.17:0/961293086 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f54f0053950 0x7f54f0055e10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:34.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.102+0000 7f5517f8a700 1 --2- 172.21.15.17:0/961293086 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5510146ae0 0x7f5510142210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:34.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.102+0000 7f5517f8a700 1 --2- 172.21.15.17:0/961293086 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f55100b4820 0x7f5510152450 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:34.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.102+0000 7f5517f8a700 1 --2- 172.21.15.17:0/961293086 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f55100b4310 0x7f551014a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:34.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.102+0000 7f5517f8a700 1 -- 172.21.15.17:0/961293086 >> 172.21.15.17:0/961293086 conn(0x7f551005c620 msgr2=0x7f55100b5060 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:34.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.102+0000 7f5517f8a700 1 -- 172.21.15.17:0/961293086 shutdown_connections 2022-08-01T19:34:34.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.102+0000 7f5517f8a700 1 -- 172.21.15.17:0/961293086 wait complete. 2022-08-01T19:34:34.123 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T19:34:34.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.230+0000 7fb414a17700 1 Processor -- start 2022-08-01T19:34:34.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.230+0000 7fb414a17700 1 -- start start 2022-08-01T19:34:34.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.230+0000 7fb414a17700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb4100aadb0 0x7fb4100b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:34.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.230+0000 7fb414a17700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4100b2510 con 0x7fb4100b31f0 2022-08-01T19:34:34.236 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.230+0000 7fb414a17700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4101443e0 con 0x7fb4100b7500 2022-08-01T19:34:34.236 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.230+0000 7fb414a17700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4100b3700 con 0x7fb4100aadb0 2022-08-01T19:34:34.236 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.230+0000 7fb40effd700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb4100b31f0 0x7fb410064d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57642/0 (socket says 172.21.15.17:57642) 2022-08-01T19:34:34.236 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.230+0000 7fb40effd700 1 -- 172.21.15.17:0/2322873971 learned_addr learned my addr 172.21.15.17:0/2322873971 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:34.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.230+0000 7fb40ffff700 1 --2- 172.21.15.17:0/2322873971 >> v2:172.21.15.17:3300/0 conn(0x7fb4100aadb0 0x7fb4100b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:34.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.230+0000 7fb40e7fc700 1 -- 172.21.15.17:0/2322873971 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3036613466 0 0) 0x7fb4100b2510 con 0x7fb4100b31f0 2022-08-01T19:34:34.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.230+0000 7fb40e7fc700 1 -- 172.21.15.17:0/2322873971 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3f0003260 con 0x7fb4100b31f0 2022-08-01T19:34:34.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.230+0000 7fb40e7fc700 1 -- 172.21.15.17:0/2322873971 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2456081848 0 0) 0x7fb4101443e0 con 0x7fb4100b7500 2022-08-01T19:34:34.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.230+0000 7fb40e7fc700 1 -- 172.21.15.17:0/2322873971 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4100b2510 con 0x7fb4100b7500 2022-08-01T19:34:34.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.230+0000 7fb40ffff700 1 -- 172.21.15.17:0/2322873971 >> v1:172.21.15.77:6789/0 conn(0x7fb4100b7500 legacy=0x7fb41013e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:34.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.230+0000 7fb40ffff700 1 -- 172.21.15.17:0/2322873971 >> v1:172.21.15.17:6789/0 conn(0x7fb4100b31f0 legacy=0x7fb410064d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:34.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.230+0000 7fb40ffff700 1 -- 172.21.15.17:0/2322873971 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4100b2740 con 0x7fb4100aadb0 2022-08-01T19:34:34.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.230+0000 7fb40ffff700 1 --2- 172.21.15.17:0/2322873971 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4100aadb0 0x7fb4100b78e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7fb4080032f0 tx=0x7fb4080091d0).ready entity=mon.0 client_cookie=a25047fd000df9d1 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:34.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.230+0000 7fb40e7fc700 1 -- 172.21.15.17:0/2322873971 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb40800b650 con 0x7fb4100aadb0 2022-08-01T19:34:34.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.230+0000 7fb40e7fc700 1 -- 172.21.15.17:0/2322873971 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40800b7b0 con 0x7fb4100aadb0 2022-08-01T19:34:34.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.230+0000 7fb40e7fc700 1 -- 172.21.15.17:0/2322873971 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb40800b920 con 0x7fb4100aadb0 2022-08-01T19:34:34.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.230+0000 7fb414a17700 1 -- 172.21.15.17:0/2322873971 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4100aadb0 msgr2=0x7fb4100b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:34.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.230+0000 7fb414a17700 1 --2- 172.21.15.17:0/2322873971 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4100aadb0 0x7fb4100b78e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7fb4080032f0 tx=0x7fb4080091d0).stop 2022-08-01T19:34:34.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.230+0000 7fb414a17700 1 -- 172.21.15.17:0/2322873971 shutdown_connections 2022-08-01T19:34:34.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.230+0000 7fb414a17700 1 --2- 172.21.15.17:0/2322873971 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4100aadb0 0x7fb4100b78e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:34.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.230+0000 7fb414a17700 1 -- 172.21.15.17:0/2322873971 >> 172.21.15.17:0/2322873971 conn(0x7fb41005f840 msgr2=0x7fb41005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:34.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.230+0000 7fb414a17700 1 -- 172.21.15.17:0/2322873971 shutdown_connections 2022-08-01T19:34:34.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.230+0000 7fb414a17700 1 -- 172.21.15.17:0/2322873971 wait complete. 2022-08-01T19:34:34.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.234+0000 7fb414a17700 1 Processor -- start 2022-08-01T19:34:34.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.234+0000 7fb414a17700 1 -- start start 2022-08-01T19:34:34.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.234+0000 7fb414a17700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4100aadb0 0x7fb41013eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:34.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.234+0000 7fb414a17700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb4100b31f0 0x7fb410153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:34.246 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.234+0000 7fb414a17700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4100b7500 0x7fb410154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:34.246 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.234+0000 7fb414a17700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb410140ca0 con 0x7fb4100aadb0 2022-08-01T19:34:34.246 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.234+0000 7fb414a17700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4100ac100 con 0x7fb4100b7500 2022-08-01T19:34:34.246 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.234+0000 7fb414a17700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb410154750 con 0x7fb4100b31f0 2022-08-01T19:34:34.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.234+0000 7fb40f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4100aadb0 0x7fb41013eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:34.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.234+0000 7fb40f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4100aadb0 0x7fb41013eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37220/0 (socket says 172.21.15.17:37220) 2022-08-01T19:34:34.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.234+0000 7fb40f7fe700 1 -- 172.21.15.17:0/2118866312 learned_addr learned my addr 172.21.15.17:0/2118866312 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:34.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.234+0000 7fb40ffff700 1 --2- 172.21.15.17:0/2118866312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4100b7500 0x7fb410154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:34.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.234+0000 7fb40effd700 1 --2- 172.21.15.17:0/2118866312 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb4100b31f0 0x7fb410153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:34.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.234+0000 7fb40f7fe700 1 -- 172.21.15.17:0/2118866312 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb4100b31f0 msgr2=0x7fb410153d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:34.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.234+0000 7fb40f7fe700 1 --2- 172.21.15.17:0/2118866312 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb4100b31f0 0x7fb410153d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:34.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.234+0000 7fb40f7fe700 1 -- 172.21.15.17:0/2118866312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4100b7500 msgr2=0x7fb410154260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:34.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.234+0000 7fb40f7fe700 1 --2- 172.21.15.17:0/2118866312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4100b7500 0x7fb410154260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:34.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.234+0000 7fb40f7fe700 1 -- 172.21.15.17:0/2118866312 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb408009470 con 0x7fb4100aadb0 2022-08-01T19:34:34.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.234+0000 7fb40f7fe700 1 --2- 172.21.15.17:0/2118866312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4100aadb0 0x7fb41013eeb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7fb400001160 tx=0x7fb400007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:34.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.234+0000 7fb40cff9700 1 -- 172.21.15.17:0/2118866312 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb400012400 con 0x7fb4100aadb0 2022-08-01T19:34:34.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.234+0000 7fb40cff9700 1 -- 172.21.15.17:0/2118866312 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb400012560 con 0x7fb4100aadb0 2022-08-01T19:34:34.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.234+0000 7fb414a17700 1 -- 172.21.15.17:0/2118866312 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4100abb60 con 0x7fb4100aadb0 2022-08-01T19:34:34.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.234+0000 7fb40cff9700 1 -- 172.21.15.17:0/2118866312 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4000128c0 con 0x7fb4100aadb0 2022-08-01T19:34:34.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.234+0000 7fb414a17700 1 -- 172.21.15.17:0/2118866312 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb41013f3a0 con 0x7fb4100aadb0 2022-08-01T19:34:34.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.238+0000 7fb414a17700 1 -- 172.21.15.17:0/2118866312 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4100574e0 con 0x7fb4100aadb0 2022-08-01T19:34:34.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.238+0000 7fb40cff9700 1 -- 172.21.15.17:0/2118866312 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb400019030 con 0x7fb4100aadb0 2022-08-01T19:34:34.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.238+0000 7fb40cff9700 1 --2- 172.21.15.17:0/2118866312 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb3f40539e0 0x7fb3f4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:34.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.238+0000 7fb40cff9700 1 -- 172.21.15.17:0/2118866312 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fb40006b3e0 con 0x7fb4100aadb0 2022-08-01T19:34:34.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.238+0000 7fb40effd700 1 --2- 172.21.15.17:0/2118866312 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb3f40539e0 0x7fb3f4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:34.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.238+0000 7fb40effd700 1 --2- 172.21.15.17:0/2118866312 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb3f40539e0 0x7fb3f4055ea0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fb3f800d000 tx=0x7fb3f8009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:34.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.246+0000 7fb40cff9700 1 -- 172.21.15.17:0/2118866312 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb400042100 con 0x7fb4100aadb0 2022-08-01T19:34:34.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.374+0000 7fb414a17700 1 -- 172.21.15.17:0/2118866312 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb410141ff0 con 0x7fb4100aadb0 2022-08-01T19:34:34.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.374+0000 7fb40cff9700 1 -- 172.21.15.17:0/2118866312 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+10 (secure 0 0 0) 0x7fb400045720 con 0x7fb4100aadb0 2022-08-01T19:34:34.379 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:34:34.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.374+0000 7fb414a17700 1 -- 172.21.15.17:0/2118866312 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb3f40539e0 msgr2=0x7fb3f4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:34.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.374+0000 7fb414a17700 1 --2- 172.21.15.17:0/2118866312 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb3f40539e0 0x7fb3f4055ea0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fb3f800d000 tx=0x7fb3f8009040).stop 2022-08-01T19:34:34.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.374+0000 7fb414a17700 1 -- 172.21.15.17:0/2118866312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4100aadb0 msgr2=0x7fb41013eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:34.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.374+0000 7fb414a17700 1 --2- 172.21.15.17:0/2118866312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4100aadb0 0x7fb41013eeb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7fb400001160 tx=0x7fb400007ad0).stop 2022-08-01T19:34:34.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.374+0000 7fb414a17700 1 -- 172.21.15.17:0/2118866312 shutdown_connections 2022-08-01T19:34:34.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.374+0000 7fb414a17700 1 --2- 172.21.15.17:0/2118866312 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb3f40539e0 0x7fb3f4055ea0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:34.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.374+0000 7fb414a17700 1 --2- 172.21.15.17:0/2118866312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4100b7500 0x7fb410154260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:34.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.374+0000 7fb414a17700 1 --2- 172.21.15.17:0/2118866312 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb4100b31f0 0x7fb410153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:34.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.374+0000 7fb414a17700 1 --2- 172.21.15.17:0/2118866312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4100aadb0 0x7fb41013eeb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:34.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.374+0000 7fb414a17700 1 -- 172.21.15.17:0/2118866312 >> 172.21.15.17:0/2118866312 conn(0x7fb41005f840 msgr2=0x7fb4100b5fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:34.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.374+0000 7fb414a17700 1 -- 172.21.15.17:0/2118866312 shutdown_connections 2022-08-01T19:34:34.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.374+0000 7fb414a17700 1 -- 172.21.15.17:0/2118866312 wait complete. 2022-08-01T19:34:34.397 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command (tasks.cephfs.test_misc.TestCacheDrop) 2022-08-01T19:34:34.399 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command' 2022-08-01T19:34:34.502 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.498+0000 7f98e133c700 1 Processor -- start 2022-08-01T19:34:34.502 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.498+0000 7f98e133c700 1 -- start start 2022-08-01T19:34:34.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.498+0000 7f98e133c700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f98dc0b2710 0x7f98dc0abca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:34.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.498+0000 7f98e133c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98dc0b13f0 0x7f98dc0ac190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:34.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.498+0000 7f98e133c700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98dc0adf10 con 0x7f98dc0b1920 2022-08-01T19:34:34.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.498+0000 7f98e133c700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98dc0ae1b0 con 0x7f98dc0b2710 2022-08-01T19:34:34.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.498+0000 7f98e133c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc0a9520 con 0x7f98dc0b13f0 2022-08-01T19:34:34.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.498+0000 7f98db7fe700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98dc0b13f0 0x7f98dc0ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:34.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.498+0000 7f98e0b3b700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7f98dc0b1920 0x7f98dc0afd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:46224/0 (socket says 172.21.15.17:46224) 2022-08-01T19:34:34.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.498+0000 7f98dbfff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f98dc0b2710 0x7f98dc0abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:34.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.498+0000 7f98e0b3b700 1 -- 172.21.15.17:0/2827001253 learned_addr learned my addr 172.21.15.17:0/2827001253 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:34.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.498+0000 7f98db7fe700 1 -- 172.21.15.17:0/2827001253 >> v2:172.21.15.77:3300/0 conn(0x7f98dc0b2710 msgr2=0x7f98dc0abca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:34.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.498+0000 7f98db7fe700 1 --2- 172.21.15.17:0/2827001253 >> v2:172.21.15.77:3300/0 conn(0x7f98dc0b2710 0x7f98dc0abca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:34.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.498+0000 7f98db7fe700 1 -- 172.21.15.17:0/2827001253 >> v1:172.21.15.77:6789/0 conn(0x7f98dc0b1920 legacy=0x7f98dc0afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:34.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.498+0000 7f98db7fe700 1 -- 172.21.15.17:0/2827001253 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98dc0a99c0 con 0x7f98dc0b13f0 2022-08-01T19:34:34.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.498+0000 7f98db7fe700 1 --2- 172.21.15.17:0/2827001253 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98dc0b13f0 0x7f98dc0ac190 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f98dc067bc0 tx=0x7f98d0006df0).ready entity=mon.2 client_cookie=cbe4250aaeea762a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:34.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.498+0000 7f98daffd700 1 -- 172.21.15.17:0/2827001253 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98d000d570 con 0x7f98dc0b13f0 2022-08-01T19:34:34.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.498+0000 7f98daffd700 1 -- 172.21.15.17:0/2827001253 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d0000e90 con 0x7f98dc0b13f0 2022-08-01T19:34:34.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.498+0000 7f98daffd700 1 -- 172.21.15.17:0/2827001253 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98d000d8c0 con 0x7f98dc0b13f0 2022-08-01T19:34:34.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.498+0000 7f98e133c700 1 -- 172.21.15.17:0/2827001253 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98dc0b13f0 msgr2=0x7f98dc0ac190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:34.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.498+0000 7f98e133c700 1 --2- 172.21.15.17:0/2827001253 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98dc0b13f0 0x7f98dc0ac190 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f98dc067bc0 tx=0x7f98d0006df0).stop 2022-08-01T19:34:34.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.498+0000 7f98e133c700 1 -- 172.21.15.17:0/2827001253 shutdown_connections 2022-08-01T19:34:34.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.498+0000 7f98e133c700 1 --2- 172.21.15.17:0/2827001253 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98dc0b13f0 0x7f98dc0ac190 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:34.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.498+0000 7f98e133c700 1 --2- 172.21.15.17:0/2827001253 >> v2:172.21.15.77:3300/0 conn(0x7f98dc0b2710 0x7f98dc0abca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:34.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.498+0000 7f98e133c700 1 -- 172.21.15.17:0/2827001253 >> 172.21.15.17:0/2827001253 conn(0x7f98dc060310 msgr2=0x7f98dc0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:34.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.498+0000 7f98e133c700 1 -- 172.21.15.17:0/2827001253 shutdown_connections 2022-08-01T19:34:34.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.498+0000 7f98e133c700 1 -- 172.21.15.17:0/2827001253 wait complete. 2022-08-01T19:34:34.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.502+0000 7f98e133c700 1 Processor -- start 2022-08-01T19:34:34.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.502+0000 7f98e133c700 1 -- start start 2022-08-01T19:34:34.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.502+0000 7f98e133c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98dc0b13f0 0x7f98dc066350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:34.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.502+0000 7f98e133c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98dc0b1920 0x7f98dc1384a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:34.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.502+0000 7f98e133c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98dc0b2710 0x7f98dc138990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:34.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.502+0000 7f98e133c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc139800 con 0x7f98dc0b13f0 2022-08-01T19:34:34.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.502+0000 7f98e133c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc13a470 con 0x7f98dc0b1920 2022-08-01T19:34:34.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.502+0000 7f98e133c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc13a1d0 con 0x7f98dc0b2710 2022-08-01T19:34:34.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.502+0000 7f98dbfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98dc0b13f0 0x7f98dc066350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:34.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.502+0000 7f98dbfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98dc0b13f0 0x7f98dc066350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37234/0 (socket says 172.21.15.17:37234) 2022-08-01T19:34:34.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.502+0000 7f98dbfff700 1 -- 172.21.15.17:0/707789970 learned_addr learned my addr 172.21.15.17:0/707789970 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:34.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.502+0000 7f98db7fe700 1 --2- 172.21.15.17:0/707789970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98dc0b1920 0x7f98dc1384a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:34.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.502+0000 7f98e0b3b700 1 --2- 172.21.15.17:0/707789970 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98dc0b2710 0x7f98dc138990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:34.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.502+0000 7f98dbfff700 1 -- 172.21.15.17:0/707789970 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98dc0b2710 msgr2=0x7f98dc138990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:34.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.502+0000 7f98dbfff700 1 --2- 172.21.15.17:0/707789970 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98dc0b2710 0x7f98dc138990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:34.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.502+0000 7f98dbfff700 1 -- 172.21.15.17:0/707789970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98dc0b1920 msgr2=0x7f98dc1384a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:34.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.502+0000 7f98dbfff700 1 --2- 172.21.15.17:0/707789970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98dc0b1920 0x7f98dc1384a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:34.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.502+0000 7f98dbfff700 1 -- 172.21.15.17:0/707789970 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d000b180 con 0x7f98dc0b13f0 2022-08-01T19:34:34.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.502+0000 7f98dbfff700 1 --2- 172.21.15.17:0/707789970 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98dc0b13f0 0x7f98dc066350 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f98cc00c000 tx=0x7f98cc006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:34.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.506+0000 7f98d97fa700 1 -- 172.21.15.17:0/707789970 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98cc011750 con 0x7f98dc0b13f0 2022-08-01T19:34:34.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.506+0000 7f98d97fa700 1 -- 172.21.15.17:0/707789970 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98cc0118b0 con 0x7f98dc0b13f0 2022-08-01T19:34:34.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.506+0000 7f98e133c700 1 -- 172.21.15.17:0/707789970 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98dc139a60 con 0x7f98dc0b13f0 2022-08-01T19:34:34.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.506+0000 7f98d97fa700 1 -- 172.21.15.17:0/707789970 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98cc011c10 con 0x7f98dc0b13f0 2022-08-01T19:34:34.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.506+0000 7f98e133c700 1 -- 172.21.15.17:0/707789970 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98dc13e030 con 0x7f98dc0b13f0 2022-08-01T19:34:34.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.506+0000 7f98d97fa700 1 -- 172.21.15.17:0/707789970 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f98cc011d70 con 0x7f98dc0b13f0 2022-08-01T19:34:34.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.506+0000 7f98e133c700 1 -- 172.21.15.17:0/707789970 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98dc139bf0 con 0x7f98dc0b13f0 2022-08-01T19:34:34.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.506+0000 7f98d97fa700 1 --2- 172.21.15.17:0/707789970 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f98bc053a10 0x7f98bc055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:34.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.506+0000 7f98d97fa700 1 -- 172.21.15.17:0/707789970 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f98cc06c440 con 0x7f98dc0b13f0 2022-08-01T19:34:34.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.506+0000 7f98db7fe700 1 --2- 172.21.15.17:0/707789970 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f98bc053a10 0x7f98bc055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:34.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.506+0000 7f98db7fe700 1 --2- 172.21.15.17:0/707789970 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f98bc053a10 0x7f98bc055ed0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f98d001a000 tx=0x7f98d0014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:34.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.514+0000 7f98d97fa700 1 -- 172.21.15.17:0/707789970 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f98cc043160 con 0x7f98dc0b13f0 2022-08-01T19:34:34.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:34.678+0000 7f98e133c700 1 -- 172.21.15.17:0/707789970 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]} v 0) v1 -- 0x7f98dc0574e0 con 0x7f98dc0b13f0 2022-08-01T19:34:35.162 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.158+0000 7f98d97fa700 1 -- 172.21.15.17:0/707789970 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]}]=0 v43) v1 ==== 142+0+0 (secure 0 0 0) 0x7f98cc046780 con 0x7f98dc0b13f0 2022-08-01T19:34:35.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.162+0000 7f98e133c700 1 -- 172.21.15.17:0/707789970 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f98bc053a10 msgr2=0x7f98bc055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:35.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.162+0000 7f98e133c700 1 --2- 172.21.15.17:0/707789970 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f98bc053a10 0x7f98bc055ed0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f98d001a000 tx=0x7f98d0014000).stop 2022-08-01T19:34:35.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.162+0000 7f98e133c700 1 -- 172.21.15.17:0/707789970 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98dc0b13f0 msgr2=0x7f98dc066350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:35.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.162+0000 7f98e133c700 1 --2- 172.21.15.17:0/707789970 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98dc0b13f0 0x7f98dc066350 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f98cc00c000 tx=0x7f98cc006df0).stop 2022-08-01T19:34:35.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.162+0000 7f98e133c700 1 -- 172.21.15.17:0/707789970 shutdown_connections 2022-08-01T19:34:35.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.162+0000 7f98e133c700 1 --2- 172.21.15.17:0/707789970 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f98bc053a10 0x7f98bc055ed0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:35.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.162+0000 7f98e133c700 1 --2- 172.21.15.17:0/707789970 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98dc0b2710 0x7f98dc138990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:35.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.162+0000 7f98e133c700 1 --2- 172.21.15.17:0/707789970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98dc0b1920 0x7f98dc1384a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:35.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.162+0000 7f98e133c700 1 --2- 172.21.15.17:0/707789970 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98dc0b13f0 0x7f98dc066350 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:35.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.162+0000 7f98e133c700 1 -- 172.21.15.17:0/707789970 >> 172.21.15.17:0/707789970 conn(0x7f98dc060310 msgr2=0x7f98dc0b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:35.169 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.162+0000 7f98e133c700 1 -- 172.21.15.17:0/707789970 shutdown_connections 2022-08-01T19:34:35.169 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.162+0000 7f98e133c700 1 -- 172.21.15.17:0/707789970 wait complete. 2022-08-01T19:34:35.182 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-08-01T19:34:35.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.290+0000 7fd51359e700 1 Processor -- start 2022-08-01T19:34:35.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.290+0000 7fd51359e700 1 -- start start 2022-08-01T19:34:35.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.294+0000 7fd51359e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd5140b31d0 0x7fd514066020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:35.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.294+0000 7fd51359e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fd5140a9da0 0x7fd514060950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:35.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.294+0000 7fd51359e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd514067030 con 0x7fd5141434b0 2022-08-01T19:34:35.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.294+0000 7fd51359e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd514064a70 con 0x7fd5140a9da0 2022-08-01T19:34:35.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.294+0000 7fd51359e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd514061f60 con 0x7fd5140b31d0 2022-08-01T19:34:35.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.294+0000 7fd509d9b700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fd5140a9da0 0x7fd514060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:35.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.294+0000 7fd509d9b700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fd5140a9da0 0x7fd514060950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:35166/0 (socket says 172.21.15.17:35166) 2022-08-01T19:34:35.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.294+0000 7fd51259c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd5140b31d0 0x7fd514066020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:35.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.294+0000 7fd512d9d700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7fd5141434b0 0x7fd514143940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:46240/0 (socket says 172.21.15.17:46240) 2022-08-01T19:34:35.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.294+0000 7fd509d9b700 1 -- 172.21.15.17:0/2500988935 learned_addr learned my addr 172.21.15.17:0/2500988935 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:35.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.294+0000 7fd509d9b700 1 -- 172.21.15.17:0/2500988935 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd5140b31d0 msgr2=0x7fd514066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:35.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.294+0000 7fd509d9b700 1 --2- 172.21.15.17:0/2500988935 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd5140b31d0 0x7fd514066020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:35.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.294+0000 7fd509d9b700 1 -- 172.21.15.17:0/2500988935 >> v1:172.21.15.77:6789/0 conn(0x7fd5141434b0 legacy=0x7fd514143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:35.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.294+0000 7fd509d9b700 1 -- 172.21.15.17:0/2500988935 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5140b35b0 con 0x7fd5140a9da0 2022-08-01T19:34:35.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.294+0000 7fd509d9b700 1 --2- 172.21.15.17:0/2500988935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5140a9da0 0x7fd514060950 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fd4fc002f70 tx=0x7fd4fc006eb0).ready entity=mon.1 client_cookie=a6f474275a1ed14 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:35.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.294+0000 7fd511d9b700 1 -- 172.21.15.17:0/2500988935 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4fc00d570 con 0x7fd5140a9da0 2022-08-01T19:34:35.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.294+0000 7fd511d9b700 1 -- 172.21.15.17:0/2500988935 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4fc000e90 con 0x7fd5140a9da0 2022-08-01T19:34:35.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.294+0000 7fd511d9b700 1 -- 172.21.15.17:0/2500988935 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4fc00d900 con 0x7fd5140a9da0 2022-08-01T19:34:35.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.294+0000 7fd51359e700 1 -- 172.21.15.17:0/2500988935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5140a9da0 msgr2=0x7fd514060950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:35.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.294+0000 7fd51359e700 1 --2- 172.21.15.17:0/2500988935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5140a9da0 0x7fd514060950 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fd4fc002f70 tx=0x7fd4fc006eb0).stop 2022-08-01T19:34:35.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.294+0000 7fd51359e700 1 -- 172.21.15.17:0/2500988935 shutdown_connections 2022-08-01T19:34:35.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.294+0000 7fd51359e700 1 --2- 172.21.15.17:0/2500988935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5140a9da0 0x7fd514060950 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:35.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.294+0000 7fd51359e700 1 --2- 172.21.15.17:0/2500988935 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd5140b31d0 0x7fd514066020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:35.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.298+0000 7fd51359e700 1 -- 172.21.15.17:0/2500988935 >> 172.21.15.17:0/2500988935 conn(0x7fd514063870 msgr2=0x7fd514063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:35.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.298+0000 7fd51359e700 1 -- 172.21.15.17:0/2500988935 shutdown_connections 2022-08-01T19:34:35.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.298+0000 7fd51359e700 1 -- 172.21.15.17:0/2500988935 wait complete. 2022-08-01T19:34:35.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.298+0000 7fd51359e700 1 Processor -- start 2022-08-01T19:34:35.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.298+0000 7fd51359e700 1 -- start start 2022-08-01T19:34:35.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.298+0000 7fd51359e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5140a9da0 0x7fd5140adf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:35.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.298+0000 7fd51359e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5140b31d0 0x7fd5140af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:35.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.298+0000 7fd51359e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd5141434b0 0x7fd5141408c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:35.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.298+0000 7fd51359e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd514152ef0 con 0x7fd5140a9da0 2022-08-01T19:34:35.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.298+0000 7fd51359e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd514153190 con 0x7fd5140b31d0 2022-08-01T19:34:35.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.298+0000 7fd51359e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5141558b0 con 0x7fd5141434b0 2022-08-01T19:34:35.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.298+0000 7fd51259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5140a9da0 0x7fd5140adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:35.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.298+0000 7fd51259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5140a9da0 0x7fd5140adf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37248/0 (socket says 172.21.15.17:37248) 2022-08-01T19:34:35.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.298+0000 7fd51259c700 1 -- 172.21.15.17:0/373828504 learned_addr learned my addr 172.21.15.17:0/373828504 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:35.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.298+0000 7fd509d9b700 1 --2- 172.21.15.17:0/373828504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5140b31d0 0x7fd5140af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:35.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.298+0000 7fd512d9d700 1 --2- 172.21.15.17:0/373828504 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd5141434b0 0x7fd5141408c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:35.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.298+0000 7fd51259c700 1 -- 172.21.15.17:0/373828504 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd5141434b0 msgr2=0x7fd5141408c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:35.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.298+0000 7fd51259c700 1 --2- 172.21.15.17:0/373828504 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd5141434b0 0x7fd5141408c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:35.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.298+0000 7fd51259c700 1 -- 172.21.15.17:0/373828504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5140b31d0 msgr2=0x7fd5140af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:35.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.298+0000 7fd51259c700 1 --2- 172.21.15.17:0/373828504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5140b31d0 0x7fd5140af360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:35.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.298+0000 7fd51259c700 1 -- 172.21.15.17:0/373828504 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4fc00b1e0 con 0x7fd5140a9da0 2022-08-01T19:34:35.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.298+0000 7fd512d9d700 1 --2- 172.21.15.17:0/373828504 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd5141434b0 0x7fd5141408c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:34:35.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.302+0000 7fd51259c700 1 --2- 172.21.15.17:0/373828504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5140a9da0 0x7fd5140adf20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7fd50400c000 tx=0x7fd504006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:35.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.302+0000 7fd50bfff700 1 -- 172.21.15.17:0/373828504 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd504010780 con 0x7fd5140a9da0 2022-08-01T19:34:35.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.302+0000 7fd50bfff700 1 -- 172.21.15.17:0/373828504 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5040108e0 con 0x7fd5140a9da0 2022-08-01T19:34:35.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.302+0000 7fd51359e700 1 -- 172.21.15.17:0/373828504 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd514155b50 con 0x7fd5140a9da0 2022-08-01T19:34:35.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.302+0000 7fd50bfff700 1 -- 172.21.15.17:0/373828504 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd504010c40 con 0x7fd5140a9da0 2022-08-01T19:34:35.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.302+0000 7fd51359e700 1 -- 172.21.15.17:0/373828504 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd514140db0 con 0x7fd5140a9da0 2022-08-01T19:34:35.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.302+0000 7fd51359e700 1 -- 172.21.15.17:0/373828504 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5140b61f0 con 0x7fd5140a9da0 2022-08-01T19:34:35.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.302+0000 7fd50bfff700 1 -- 172.21.15.17:0/373828504 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd50401c460 con 0x7fd5140a9da0 2022-08-01T19:34:35.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.302+0000 7fd50bfff700 1 --2- 172.21.15.17:0/373828504 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd4f8053980 0x7fd4f8055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:35.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.302+0000 7fd50bfff700 1 -- 172.21.15.17:0/373828504 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fd50406c5b0 con 0x7fd5140a9da0 2022-08-01T19:34:35.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.302+0000 7fd509d9b700 1 --2- 172.21.15.17:0/373828504 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd4f8053980 0x7fd4f8055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:35.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.306+0000 7fd509d9b700 1 --2- 172.21.15.17:0/373828504 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd4f8053980 0x7fd4f8055e40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fd4fc01c000 tx=0x7fd4fc015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:35.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.306+0000 7fd50bfff700 1 -- 172.21.15.17:0/373828504 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd5140b61f0 con 0x7fd5140a9da0 2022-08-01T19:34:35.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.434+0000 7fd51359e700 1 -- 172.21.15.17:0/373828504 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd5140b61f0 con 0x7fd5140a9da0 2022-08-01T19:34:35.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.438+0000 7fd50bfff700 1 -- 172.21.15.17:0/373828504 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v2)=0 v2) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd5140b61f0 con 0x7fd5140a9da0 2022-08-01T19:34:35.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.442+0000 7fd51359e700 1 -- 172.21.15.17:0/373828504 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd4f8053980 msgr2=0x7fd4f8055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:35.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.442+0000 7fd51359e700 1 --2- 172.21.15.17:0/373828504 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd4f8053980 0x7fd4f8055e40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fd4fc01c000 tx=0x7fd4fc015000).stop 2022-08-01T19:34:35.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.442+0000 7fd51359e700 1 -- 172.21.15.17:0/373828504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5140a9da0 msgr2=0x7fd5140adf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:35.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.442+0000 7fd51359e700 1 --2- 172.21.15.17:0/373828504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5140a9da0 0x7fd5140adf20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7fd50400c000 tx=0x7fd504006df0).stop 2022-08-01T19:34:35.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.442+0000 7fd51359e700 1 -- 172.21.15.17:0/373828504 shutdown_connections 2022-08-01T19:34:35.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.442+0000 7fd51359e700 1 --2- 172.21.15.17:0/373828504 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd4f8053980 0x7fd4f8055e40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:35.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.442+0000 7fd51359e700 1 --2- 172.21.15.17:0/373828504 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd5141434b0 0x7fd5141408c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:35.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.442+0000 7fd51359e700 1 --2- 172.21.15.17:0/373828504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5140b31d0 0x7fd5140af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:35.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.442+0000 7fd51359e700 1 --2- 172.21.15.17:0/373828504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5140a9da0 0x7fd5140adf20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:35.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.442+0000 7fd51359e700 1 -- 172.21.15.17:0/373828504 >> 172.21.15.17:0/373828504 conn(0x7fd514063870 msgr2=0x7fd514141c90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:35.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.442+0000 7fd51359e700 1 -- 172.21.15.17:0/373828504 shutdown_connections 2022-08-01T19:34:35.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:35.442+0000 7fd51359e700 1 -- 172.21.15.17:0/373828504 wait complete. 2022-08-01T19:34:35.458 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T19:34:35.459 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:34:35.480 DEBUG:teuthology.orchestra.run.smithi077:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:34:35.501 DEBUG:teuthology.orchestra.run.smithi085:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:34:35.521 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:34:35.543 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T19:34:35.544 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:34:35.544 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:34:35.544 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:34:35.570 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:34:35.572 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:34:35.572 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:34:35.573 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:34:35.574 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:34:35.583 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:34:35.585 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:34:35.586 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:34:35.586 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:34:35.587 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:34:35.632 INFO:teuthology.orchestra.run.smithi085.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2022-08-01T19:34:35.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:34:35.634 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2022-08-01T19:34:35.635 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi085.front.sepia.ceph.com... 2022-08-01T19:34:35.635 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T19:34:35.636 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:34:35.666 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:34:35.666 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:34:35.667 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:34:35.716 INFO:teuthology.orchestra.run.smithi085.stdout:ext2/ext3 2022-08-01T19:34:35.718 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T19:34:35.718 INFO:teuthology.orchestra.run:waiting for 900 2022-08-01T19:34:35.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T19:34:35.719 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:34:35.738 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:34:35.739 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:34:35.739 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:34:35.747 INFO:teuthology.orchestra.run.smithi190.stdout:fuseblk 2022-08-01T19:34:35.749 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T19:34:35.749 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:34:35.750 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:34:35.751 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:34:35.760 INFO:teuthology.orchestra.run.smithi190.stdout:fuseblk 2022-08-01T19:34:35.761 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T19:34:35.762 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:34:35.763 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:34:35.764 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:34:35.771 INFO:teuthology.orchestra.run.smithi190.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': Transport endpoint is not connected 2022-08-01T19:34:35.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:34:35.774 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.1 2022-08-01T19:34:35.775 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi190.front.sepia.ceph.com... 2022-08-01T19:34:35.776 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T19:34:35.777 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:34:35.856 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:34:35.857 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:34:35.858 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:34:35.866 INFO:teuthology.orchestra.run.smithi190.stdout:ext2/ext3 2022-08-01T19:34:35.867 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T19:34:35.868 INFO:teuthology.orchestra.run:waiting for 900 2022-08-01T19:34:35.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T19:34:35.869 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:34:35.945 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T19:34:36.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.050+0000 7f716cf2e700 1 Processor -- start 2022-08-01T19:34:36.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.050+0000 7f716cf2e700 1 -- start start 2022-08-01T19:34:36.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.050+0000 7f716cf2e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f71680b6710 0x7f71680b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:36.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.050+0000 7f716cf2e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f71680b4550 0x7f71680b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:36.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.050+0000 7f716cf2e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7168144240 con 0x7f7168061ec0 2022-08-01T19:34:36.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.050+0000 7f716cf2e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71680b6fe0 con 0x7f71680b6710 2022-08-01T19:34:36.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.050+0000 7f716cf2e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7168143d00 con 0x7f71680b4550 2022-08-01T19:34:36.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.050+0000 7f716659c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f71680b6710 0x7f71680b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:36.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.050+0000 7f716659c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f71680b6710 0x7f71680b5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37258/0 (socket says 172.21.15.17:37258) 2022-08-01T19:34:36.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.050+0000 7f716659c700 1 -- 172.21.15.17:0/4262116579 learned_addr learned my addr 172.21.15.17:0/4262116579 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:36.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.050+0000 7f7165d9b700 1 --2- 172.21.15.17:0/4262116579 >> v2:172.21.15.77:3300/0 conn(0x7f71680b4550 0x7f71680b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:36.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.050+0000 7f716659c700 1 -- 172.21.15.17:0/4262116579 >> v2:172.21.15.77:3300/0 conn(0x7f71680b4550 msgr2=0x7f71680b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:36.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.050+0000 7f716659c700 1 --2- 172.21.15.17:0/4262116579 >> v2:172.21.15.77:3300/0 conn(0x7f71680b4550 0x7f71680b6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:36.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.050+0000 7f716659c700 1 -- 172.21.15.17:0/4262116579 >> v1:172.21.15.77:6789/0 conn(0x7f7168061ec0 legacy=0x7f7168146850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:36.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.050+0000 7f716659c700 1 -- 172.21.15.17:0/4262116579 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7168143fa0 con 0x7f71680b6710 2022-08-01T19:34:36.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.050+0000 7f7165d9b700 1 --2- 172.21.15.17:0/4262116579 >> v2:172.21.15.77:3300/0 conn(0x7f71680b4550 0x7f71680b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:34:36.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.050+0000 7f716659c700 1 --2- 172.21.15.17:0/4262116579 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71680b6710 0x7f71680b5d80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f71681432b0 tx=0x7f715c006eb0).ready entity=mon.0 client_cookie=a654cf7c7ab3674b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:36.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.050+0000 7f716559a700 1 -- 172.21.15.17:0/4262116579 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f715c00d610 con 0x7f71680b6710 2022-08-01T19:34:36.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.050+0000 7f716559a700 1 -- 172.21.15.17:0/4262116579 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f715c000e90 con 0x7f71680b6710 2022-08-01T19:34:36.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.050+0000 7f716559a700 1 -- 172.21.15.17:0/4262116579 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f715c00d9a0 con 0x7f71680b6710 2022-08-01T19:34:36.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.050+0000 7f716cf2e700 1 -- 172.21.15.17:0/4262116579 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71680b6710 msgr2=0x7f71680b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:36.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.050+0000 7f716cf2e700 1 --2- 172.21.15.17:0/4262116579 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71680b6710 0x7f71680b5d80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f71681432b0 tx=0x7f715c006eb0).stop 2022-08-01T19:34:36.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.054+0000 7f716cf2e700 1 -- 172.21.15.17:0/4262116579 shutdown_connections 2022-08-01T19:34:36.069 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.054+0000 7f716cf2e700 1 --2- 172.21.15.17:0/4262116579 >> v2:172.21.15.77:3300/0 conn(0x7f71680b4550 0x7f71680b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:36.069 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.054+0000 7f716cf2e700 1 --2- 172.21.15.17:0/4262116579 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71680b6710 0x7f71680b5d80 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:36.069 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.054+0000 7f716cf2e700 1 -- 172.21.15.17:0/4262116579 >> 172.21.15.17:0/4262116579 conn(0x7f716805c620 msgr2=0x7f716805ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:36.069 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.054+0000 7f716cf2e700 1 -- 172.21.15.17:0/4262116579 shutdown_connections 2022-08-01T19:34:36.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.054+0000 7f716cf2e700 1 -- 172.21.15.17:0/4262116579 wait complete. 2022-08-01T19:34:36.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.054+0000 7f716cf2e700 1 Processor -- start 2022-08-01T19:34:36.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.054+0000 7f716cf2e700 1 -- start start 2022-08-01T19:34:36.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.054+0000 7f716cf2e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7168061ec0 0x7f7168128c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:36.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.054+0000 7f716cf2e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71680b4550 0x7f71680b2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:36.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.054+0000 7f716cf2e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71680b6710 0x7f7168146850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:36.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.054+0000 7f716cf2e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71680b2b40 con 0x7f71680b4550 2022-08-01T19:34:36.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.054+0000 7f716cf2e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71680b3010 con 0x7f7168061ec0 2022-08-01T19:34:36.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.054+0000 7f716cf2e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71680b32b0 con 0x7f71680b6710 2022-08-01T19:34:36.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.054+0000 7f7165d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71680b4550 0x7f71680b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:36.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.054+0000 7f7165d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71680b4550 0x7f71680b2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37264/0 (socket says 172.21.15.17:37264) 2022-08-01T19:34:36.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.054+0000 7f7165d9b700 1 -- 172.21.15.17:0/3953139274 learned_addr learned my addr 172.21.15.17:0/3953139274 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:36.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.054+0000 7f716659c700 1 --2- 172.21.15.17:0/3953139274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7168061ec0 0x7f7168128c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:36.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.058+0000 7f7166d9d700 1 --2- 172.21.15.17:0/3953139274 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71680b6710 0x7f7168146850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:36.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.058+0000 7f7165d9b700 1 -- 172.21.15.17:0/3953139274 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71680b6710 msgr2=0x7f7168146850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:36.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.058+0000 7f7165d9b700 1 --2- 172.21.15.17:0/3953139274 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71680b6710 0x7f7168146850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:36.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.058+0000 7f7165d9b700 1 -- 172.21.15.17:0/3953139274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7168061ec0 msgr2=0x7f7168128c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:36.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.058+0000 7f7165d9b700 1 --2- 172.21.15.17:0/3953139274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7168061ec0 0x7f7168128c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:36.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.058+0000 7f7165d9b700 1 -- 172.21.15.17:0/3953139274 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715800b180 con 0x7f71680b4550 2022-08-01T19:34:36.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.058+0000 7f7166d9d700 1 --2- 172.21.15.17:0/3953139274 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71680b6710 0x7f7168146850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:34:36.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.058+0000 7f7165d9b700 1 --2- 172.21.15.17:0/3953139274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71680b4550 0x7f71680b2650 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f715800c000 tx=0x7f7158006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:36.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.058+0000 7f71577fe700 1 -- 172.21.15.17:0/3953139274 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7158010780 con 0x7f71680b4550 2022-08-01T19:34:36.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.058+0000 7f71577fe700 1 -- 172.21.15.17:0/3953139274 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71580108e0 con 0x7f71680b4550 2022-08-01T19:34:36.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.058+0000 7f716cf2e700 1 -- 172.21.15.17:0/3953139274 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f715c00b1e0 con 0x7f71680b4550 2022-08-01T19:34:36.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.058+0000 7f71577fe700 1 -- 172.21.15.17:0/3953139274 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7158010c40 con 0x7f71680b4550 2022-08-01T19:34:36.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.058+0000 7f716cf2e700 1 -- 172.21.15.17:0/3953139274 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71680b37f0 con 0x7f71680b4550 2022-08-01T19:34:36.076 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.058+0000 7f716cf2e700 1 -- 172.21.15.17:0/3953139274 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71680b2190 con 0x7f71680b4550 2022-08-01T19:34:36.076 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.058+0000 7f71577fe700 1 -- 172.21.15.17:0/3953139274 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f715801c460 con 0x7f71680b4550 2022-08-01T19:34:36.076 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.058+0000 7f71577fe700 1 --2- 172.21.15.17:0/3953139274 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7144053a70 0x7f7144055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:36.076 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.062+0000 7f71577fe700 1 -- 172.21.15.17:0/3953139274 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f715806c620 con 0x7f71680b4550 2022-08-01T19:34:36.077 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.062+0000 7f716659c700 1 --2- 172.21.15.17:0/3953139274 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7144053a70 0x7f7144055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:36.077 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.062+0000 7f716659c700 1 --2- 172.21.15.17:0/3953139274 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7144053a70 0x7f7144055f30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f715c01c000 tx=0x7f715c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:36.077 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.066+0000 7f71577fe700 1 -- 172.21.15.17:0/3953139274 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f71680b2190 con 0x7f71680b4550 2022-08-01T19:34:36.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.194+0000 7f716cf2e700 1 -- 172.21.15.17:0/3953139274 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f71680b2190 con 0x7f71680b4550 2022-08-01T19:34:36.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.198+0000 7f71577fe700 1 -- 172.21.15.17:0/3953139274 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v25) v1 ==== 81+0+34065 (secure 0 0 0) 0x7f71680b2190 con 0x7f71680b4550 2022-08-01T19:34:36.204 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:36.204 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:34:36.204 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 25, 2022-08-01T19:34:36.205 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", 2022-08-01T19:34:36.205 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2022-08-01T19:33:50.105743+0000", 2022-08-01T19:34:36.205 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2022-08-01T19:34:09.692404+0000", 2022-08-01T19:34:36.205 INFO:teuthology.orchestra.run.smithi017.stdout: "last_up_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:34:36.205 INFO:teuthology.orchestra.run.smithi017.stdout: "last_in_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:34:36.206 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T19:34:36.206 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_num": 5799936, 2022-08-01T19:34:36.206 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_set": [ 2022-08-01T19:34:36.206 INFO:teuthology.orchestra.run.smithi017.stdout: "pglog_hardlimit", 2022-08-01T19:34:36.207 INFO:teuthology.orchestra.run.smithi017.stdout: "purged_snapdirs", 2022-08-01T19:34:36.207 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_deletes", 2022-08-01T19:34:36.207 INFO:teuthology.orchestra.run.smithi017.stdout: "sortbitwise" 2022-08-01T19:34:36.207 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:34:36.208 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_version": 6, 2022-08-01T19:34:36.208 INFO:teuthology.orchestra.run.smithi017.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T19:34:36.208 INFO:teuthology.orchestra.run.smithi017.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T19:34:36.208 INFO:teuthology.orchestra.run.smithi017.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T19:34:36.208 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_snapshot": "", 2022-08-01T19:34:36.209 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_max": 4, 2022-08-01T19:34:36.209 INFO:teuthology.orchestra.run.smithi017.stdout: "max_osd": 8, 2022-08-01T19:34:36.209 INFO:teuthology.orchestra.run.smithi017.stdout: "require_min_compat_client": "jewel", 2022-08-01T19:34:36.209 INFO:teuthology.orchestra.run.smithi017.stdout: "min_compat_client": "jewel", 2022-08-01T19:34:36.210 INFO:teuthology.orchestra.run.smithi017.stdout: "require_osd_release": "octopus", 2022-08-01T19:34:36.210 INFO:teuthology.orchestra.run.smithi017.stdout: "pools": [ 2022-08-01T19:34:36.210 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.210 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 1, 2022-08-01T19:34:36.211 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "device_health_metrics", 2022-08-01T19:34:36.211 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:33:53.001841+0000", 2022-08-01T19:34:36.211 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:34:36.211 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:34:36.211 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:34:36.212 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:34:36.212 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:34:36.212 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:34:36.212 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:34:36.213 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:34:36.213 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 1, 2022-08-01T19:34:36.213 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 1, 2022-08-01T19:34:36.213 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 1, 2022-08-01T19:34:36.214 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 1, 2022-08-01T19:34:36.214 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 1, 2022-08-01T19:34:36.214 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:34:36.214 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:34:36.215 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:34:36.215 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:34:36.215 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:34:36.215 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:34:36.216 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:34:36.216 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.216 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "16", 2022-08-01T19:34:36.216 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:34:36.216 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:34:36.217 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:34:36.217 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:34:36.217 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:34:36.217 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:34:36.218 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:34:36.218 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:34:36.218 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:34:36.218 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:34:36.219 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:34:36.219 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:34:36.219 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:34:36.219 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:34:36.220 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:34:36.220 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:34:36.220 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:34:36.221 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:34:36.221 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:34:36.221 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:34:36.221 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:34:36.221 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:34:36.222 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:34:36.222 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:34:36.222 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:34:36.222 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:34:36.223 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.223 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:34:36.223 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:34:36.223 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:34:36.223 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:34:36.224 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:34:36.224 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:34:36.224 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:34:36.224 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:34:36.225 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:34:36.225 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:34:36.225 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:34:36.225 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:34:36.226 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 1 2022-08-01T19:34:36.226 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.226 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:34:36.226 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_devicehealth": {} 2022-08-01T19:34:36.226 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.227 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.227 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.227 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 2, 2022-08-01T19:34:36.227 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "rbd", 2022-08-01T19:34:36.228 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:34:01.553098+0000", 2022-08-01T19:34:36.228 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:34:36.228 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:34:36.228 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:34:36.229 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:34:36.229 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:34:36.229 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:34:36.229 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:34:36.229 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:34:36.230 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:34:36.230 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:34:36.230 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:34:36.230 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:34:36.231 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:34:36.231 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:34:36.231 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:34:36.231 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:34:36.231 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:34:36.232 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:34:36.232 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:34:36.232 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:34:36.232 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.233 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "19", 2022-08-01T19:34:36.233 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:34:36.233 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:34:36.233 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:34:36.233 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:34:36.234 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:34:36.234 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:34:36.234 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:34:36.234 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:34:36.235 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:34:36.235 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:34:36.235 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:34:36.235 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:34:36.235 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:34:36.236 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:34:36.236 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:34:36.236 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:34:36.236 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:34:36.237 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:34:36.237 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:34:36.237 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:34:36.237 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:34:36.237 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:34:36.238 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:34:36.238 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:34:36.238 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:34:36.238 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:34:36.239 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.239 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:34:36.239 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:34:36.239 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:34:36.240 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:34:36.240 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:34:36.240 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:34:36.240 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:34:36.240 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:34:36.241 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:34:36.241 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:34:36.241 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:34:36.241 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:34:36.241 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:34:36.242 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd": {} 2022-08-01T19:34:36.242 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.242 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.242 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.243 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 3, 2022-08-01T19:34:36.243 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "cephfs_metadata", 2022-08-01T19:34:36.243 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:34:04.635473+0000", 2022-08-01T19:34:36.243 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:34:36.244 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:34:36.244 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:34:36.244 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:34:36.244 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:34:36.244 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:34:36.245 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:34:36.245 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:34:36.245 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:34:36.245 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:34:36.246 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:34:36.246 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:34:36.246 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:34:36.246 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:34:36.247 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:34:36.247 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:34:36.247 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:34:36.247 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:34:36.247 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:34:36.248 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:34:36.248 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.248 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "25", 2022-08-01T19:34:36.248 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:34:36.249 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:34:36.249 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:34:36.249 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:34:36.249 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:34:36.250 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:34:36.250 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:34:36.250 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:34:36.250 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:34:36.250 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:34:36.251 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:34:36.251 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:34:36.251 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:34:36.251 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:34:36.252 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:34:36.252 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:34:36.252 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:34:36.252 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:34:36.252 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:34:36.253 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:34:36.253 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:34:36.253 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:34:36.253 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:34:36.254 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:34:36.254 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:34:36.254 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:34:36.254 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.254 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:34:36.255 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:34:36.255 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:34:36.255 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:34:36.255 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:34:36.256 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:34:36.256 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:34:36.256 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:34:36.256 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:34:36.256 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:34:36.257 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:34:36.257 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:34:36.257 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_bias": 4, 2022-08-01T19:34:36.257 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 16, 2022-08-01T19:34:36.258 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_priority": 5 2022-08-01T19:34:36.258 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.258 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:34:36.258 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs": { 2022-08-01T19:34:36.258 INFO:teuthology.orchestra.run.smithi017.stdout: "metadata": "cephfs" 2022-08-01T19:34:36.259 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.259 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.259 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.259 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.260 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 4, 2022-08-01T19:34:36.260 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "cephfs_data", 2022-08-01T19:34:36.260 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:34:07.409434+0000", 2022-08-01T19:34:36.260 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 5, 2022-08-01T19:34:36.261 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool,ec_overwrites", 2022-08-01T19:34:36.261 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 3, 2022-08-01T19:34:36.261 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 4, 2022-08-01T19:34:36.261 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 3, 2022-08-01T19:34:36.261 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 1, 2022-08-01T19:34:36.262 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:34:36.262 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:34:36.262 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:34:36.262 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:34:36.263 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:34:36.263 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:34:36.263 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:34:36.263 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:34:36.263 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:34:36.264 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:34:36.264 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:34:36.264 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:34:36.264 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:34:36.265 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:34:36.265 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.265 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "25", 2022-08-01T19:34:36.265 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:34:36.265 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:34:36.266 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:34:36.266 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:34:36.266 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:34:36.266 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:34:36.267 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:34:36.267 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:34:36.267 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:34:36.267 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:34:36.267 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:34:36.268 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:34:36.268 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:34:36.268 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:34:36.268 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:34:36.269 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:34:36.269 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:34:36.269 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:34:36.269 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:34:36.270 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:34:36.270 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:34:36.270 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:34:36.270 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:34:36.271 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "cephfs_data", 2022-08-01T19:34:36.271 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:34:36.271 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:34:36.271 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.272 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:34:36.272 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:34:36.272 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:34:36.272 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:34:36.272 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:34:36.273 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:34:36.273 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:34:36.273 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:34:36.273 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 8192, 2022-08-01T19:34:36.274 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:34:36.274 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:34:36.274 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:34:36.274 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:34:36.274 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs": { 2022-08-01T19:34:36.275 INFO:teuthology.orchestra.run.smithi017.stdout: "data": "cephfs" 2022-08-01T19:34:36.275 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.275 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.275 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.276 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:34:36.276 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2022-08-01T19:34:36.276 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.276 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:34:36.276 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a962f14f-cf16-4fa8-84ff-c85325db9132", 2022-08-01T19:34:36.277 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:34:36.277 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:34:36.277 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:34:36.277 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:34:36.278 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:34:36.278 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:34:36.278 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:34:36.278 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 23, 2022-08-01T19:34:36.278 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:34:36.279 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:34:36.279 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:34:36.279 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:34:36.279 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.280 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:34:36.280 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6800", 2022-08-01T19:34:36.280 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:34:36.280 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.280 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.281 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:34:36.281 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6801", 2022-08-01T19:34:36.281 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:34:36.281 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.282 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.282 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.282 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:34:36.282 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:34:36.282 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.283 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:34:36.283 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6802", 2022-08-01T19:34:36.283 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:34:36.283 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.284 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.284 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:34:36.284 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6803", 2022-08-01T19:34:36.284 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:34:36.284 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.285 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.285 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.285 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:34:36.285 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:34:36.285 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.286 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:34:36.286 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6806", 2022-08-01T19:34:36.286 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:34:36.286 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.287 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.287 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:34:36.287 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6807", 2022-08-01T19:34:36.287 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:34:36.288 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.288 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.288 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.288 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:34:36.289 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:34:36.289 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.289 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:34:36.289 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6804", 2022-08-01T19:34:36.289 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:34:36.290 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.290 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.290 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:34:36.291 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6805", 2022-08-01T19:34:36.291 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:34:36.291 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.291 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.291 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.292 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6801/23008", 2022-08-01T19:34:36.292 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6803/23008", 2022-08-01T19:34:36.292 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6807/23008", 2022-08-01T19:34:36.292 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6805/23008", 2022-08-01T19:34:36.293 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:34:36.293 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:34:36.293 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:34:36.293 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.294 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.294 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.294 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:34:36.294 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "805f7051-a527-4e98-8844-b6e406a63248", 2022-08-01T19:34:36.294 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:34:36.295 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:34:36.295 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:34:36.295 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:34:36.295 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:34:36.296 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:34:36.296 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:34:36.296 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 23, 2022-08-01T19:34:36.296 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:34:36.296 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:34:36.297 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:34:36.297 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:34:36.297 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.297 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:34:36.298 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6808", 2022-08-01T19:34:36.298 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:34:36.298 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.298 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.299 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:34:36.299 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6809", 2022-08-01T19:34:36.299 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:34:36.299 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.299 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.300 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.300 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:34:36.300 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:34:36.300 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.301 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:34:36.301 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6810", 2022-08-01T19:34:36.301 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:34:36.301 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.302 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.302 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:34:36.302 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6811", 2022-08-01T19:34:36.302 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:34:36.303 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.303 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.303 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.303 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:34:36.303 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:34:36.304 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.304 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:34:36.304 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6814", 2022-08-01T19:34:36.304 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:34:36.305 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.305 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.305 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:34:36.305 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6815", 2022-08-01T19:34:36.305 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:34:36.306 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.306 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.306 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.306 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:34:36.307 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:34:36.307 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.307 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:34:36.307 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6812", 2022-08-01T19:34:36.307 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:34:36.308 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.308 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.308 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:34:36.308 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6813", 2022-08-01T19:34:36.309 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:34:36.309 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.309 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.309 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.310 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6809/23007", 2022-08-01T19:34:36.310 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6811/23007", 2022-08-01T19:34:36.310 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6815/23007", 2022-08-01T19:34:36.310 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6813/23007", 2022-08-01T19:34:36.310 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:34:36.311 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:34:36.311 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:34:36.311 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.312 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.312 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.312 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:34:36.312 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "89f7f3df-7c10-48e1-8b69-9e4243923a61", 2022-08-01T19:34:36.313 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:34:36.313 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:34:36.313 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:34:36.313 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:34:36.314 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:34:36.314 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:34:36.314 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:34:36.315 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 17, 2022-08-01T19:34:36.315 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:34:36.315 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:34:36.315 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:34:36.316 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:34:36.316 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.316 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:34:36.316 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6817", 2022-08-01T19:34:36.316 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:34:36.317 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.317 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.317 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:34:36.318 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6819", 2022-08-01T19:34:36.318 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:34:36.318 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.319 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.319 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.319 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:34:36.319 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:34:36.319 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.320 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:34:36.320 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6822", 2022-08-01T19:34:36.320 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:34:36.320 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.321 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.321 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:34:36.321 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6824", 2022-08-01T19:34:36.321 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:34:36.322 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.322 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.322 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.322 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:34:36.323 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:34:36.323 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.323 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:34:36.323 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6830", 2022-08-01T19:34:36.324 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:34:36.324 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.324 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.324 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:34:36.325 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6831", 2022-08-01T19:34:36.325 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:34:36.325 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.326 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.326 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.326 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:34:36.326 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:34:36.327 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.327 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:34:36.327 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6826", 2022-08-01T19:34:36.328 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:34:36.328 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.328 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.328 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:34:36.329 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6829", 2022-08-01T19:34:36.329 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:34:36.329 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.330 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.330 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.330 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6819/23010", 2022-08-01T19:34:36.330 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6824/23010", 2022-08-01T19:34:36.331 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6831/23010", 2022-08-01T19:34:36.331 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6829/23010", 2022-08-01T19:34:36.331 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:34:36.331 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:34:36.332 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:34:36.332 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.332 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.332 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.333 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:34:36.333 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "96123468-aabc-4a64-909e-a192c369090d", 2022-08-01T19:34:36.333 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:34:36.333 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:34:36.334 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:34:36.334 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:34:36.334 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:34:36.334 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:34:36.334 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:34:36.335 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 23, 2022-08-01T19:34:36.335 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:34:36.335 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:34:36.335 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:34:36.336 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:34:36.336 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.336 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:34:36.336 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6816", 2022-08-01T19:34:36.336 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:34:36.337 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.337 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.337 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:34:36.337 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6818", 2022-08-01T19:34:36.338 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:34:36.338 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.338 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.338 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.338 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:34:36.339 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:34:36.339 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.339 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:34:36.339 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6820", 2022-08-01T19:34:36.340 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:34:36.340 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.340 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.340 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:34:36.340 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6821", 2022-08-01T19:34:36.341 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:34:36.341 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.341 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.341 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.342 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:34:36.342 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:34:36.342 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.342 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:34:36.342 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6827", 2022-08-01T19:34:36.343 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:34:36.343 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.343 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.344 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:34:36.344 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6828", 2022-08-01T19:34:36.344 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:34:36.345 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.345 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.345 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.345 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:34:36.346 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:34:36.346 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.346 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:34:36.347 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6823", 2022-08-01T19:34:36.347 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:34:36.347 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.347 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.348 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:34:36.348 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6825", 2022-08-01T19:34:36.348 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:34:36.349 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.349 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.349 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.349 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6818/23009", 2022-08-01T19:34:36.350 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6821/23009", 2022-08-01T19:34:36.350 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6828/23009", 2022-08-01T19:34:36.350 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6825/23009", 2022-08-01T19:34:36.350 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:34:36.351 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:34:36.351 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:34:36.351 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.352 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.352 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.352 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:34:36.353 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a6e26665-e04b-46ec-ab66-e912acbf044c", 2022-08-01T19:34:36.353 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:34:36.353 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:34:36.353 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:34:36.354 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:34:36.354 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:34:36.354 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:34:36.354 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:34:36.354 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 23, 2022-08-01T19:34:36.355 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:34:36.355 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:34:36.355 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:34:36.355 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:34:36.356 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.356 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:34:36.356 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6810", 2022-08-01T19:34:36.356 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:34:36.356 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.357 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.357 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:34:36.357 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6811", 2022-08-01T19:34:36.357 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:34:36.358 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.358 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.358 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.358 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:34:36.358 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:34:36.359 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.359 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:34:36.359 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6812", 2022-08-01T19:34:36.359 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:34:36.360 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.360 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.360 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:34:36.360 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6813", 2022-08-01T19:34:36.361 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:34:36.361 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.361 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.361 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.361 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:34:36.362 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:34:36.362 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.362 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:34:36.362 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6816", 2022-08-01T19:34:36.363 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:34:36.363 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.363 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.363 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:34:36.363 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6817", 2022-08-01T19:34:36.364 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:34:36.364 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.364 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.364 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.365 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:34:36.365 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:34:36.365 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.365 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:34:36.366 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6814", 2022-08-01T19:34:36.366 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:34:36.366 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.366 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.366 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:34:36.367 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6815", 2022-08-01T19:34:36.367 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:34:36.367 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.367 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.368 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.368 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6811/23584", 2022-08-01T19:34:36.368 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6813/23584", 2022-08-01T19:34:36.368 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6817/23584", 2022-08-01T19:34:36.369 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6815/23584", 2022-08-01T19:34:36.369 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:34:36.369 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:34:36.369 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:34:36.370 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.370 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.370 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.370 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:34:36.371 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "502c0034-18ee-44ac-87ff-19b9dddb66a0", 2022-08-01T19:34:36.371 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:34:36.372 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:34:36.372 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:34:36.372 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:34:36.372 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:34:36.373 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:34:36.373 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:34:36.373 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 20, 2022-08-01T19:34:36.373 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:34:36.374 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:34:36.374 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:34:36.374 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:34:36.374 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.375 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:34:36.375 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6802", 2022-08-01T19:34:36.375 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:34:36.375 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.376 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.376 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:34:36.376 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6803", 2022-08-01T19:34:36.376 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:34:36.376 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.377 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.377 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.377 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:34:36.377 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:34:36.378 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.378 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:34:36.378 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6804", 2022-08-01T19:34:36.378 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:34:36.379 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.379 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.379 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:34:36.379 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6805", 2022-08-01T19:34:36.380 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:34:36.380 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.380 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.380 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.380 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:34:36.381 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:34:36.381 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.381 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:34:36.381 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6808", 2022-08-01T19:34:36.382 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:34:36.382 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.382 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:34:36.383 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6809", 2022-08-01T19:34:36.383 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:34:36.383 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.383 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.384 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:34:36.384 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:34:36.384 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.384 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:34:36.385 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6806", 2022-08-01T19:34:36.385 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:34:36.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.385 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.385 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:34:36.386 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6807", 2022-08-01T19:34:36.386 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:34:36.386 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.386 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.387 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.387 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6803/23585", 2022-08-01T19:34:36.387 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6805/23585", 2022-08-01T19:34:36.387 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6809/23585", 2022-08-01T19:34:36.387 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6807/23585", 2022-08-01T19:34:36.388 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:34:36.388 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:34:36.388 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:34:36.388 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.389 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.389 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.389 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:34:36.389 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "ac4e755a-7fe7-46cb-8407-852d5af9ffc5", 2022-08-01T19:34:36.389 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:34:36.390 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:34:36.390 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:34:36.390 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:34:36.390 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:34:36.391 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:34:36.391 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:34:36.391 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 23, 2022-08-01T19:34:36.391 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:34:36.392 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:34:36.392 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:34:36.392 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:34:36.392 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.393 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:34:36.393 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6820", 2022-08-01T19:34:36.393 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:34:36.394 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.394 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.394 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:34:36.395 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6821", 2022-08-01T19:34:36.395 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:34:36.395 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.396 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.396 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.396 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:34:36.396 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:34:36.397 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.397 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:34:36.397 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6823", 2022-08-01T19:34:36.397 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:34:36.398 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.398 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.398 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:34:36.398 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6825", 2022-08-01T19:34:36.398 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:34:36.399 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.399 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.399 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.399 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:34:36.400 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:34:36.400 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.400 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:34:36.400 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6831", 2022-08-01T19:34:36.400 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:34:36.401 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.401 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.401 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:34:36.401 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6833", 2022-08-01T19:34:36.402 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:34:36.402 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.402 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.402 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.402 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:34:36.403 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:34:36.403 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.403 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:34:36.403 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6827", 2022-08-01T19:34:36.404 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:34:36.404 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.404 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.404 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:34:36.404 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6829", 2022-08-01T19:34:36.405 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:34:36.405 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.405 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.405 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.406 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6821/23587", 2022-08-01T19:34:36.406 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6825/23587", 2022-08-01T19:34:36.406 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6833/23587", 2022-08-01T19:34:36.406 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6829/23587", 2022-08-01T19:34:36.406 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:34:36.407 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:34:36.407 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:34:36.407 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.407 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.408 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.408 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:34:36.408 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "874761e2-6410-40fc-b60c-e58abe5ce489", 2022-08-01T19:34:36.408 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:34:36.409 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:34:36.409 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:34:36.409 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:34:36.409 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:34:36.409 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:34:36.410 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:34:36.410 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 17, 2022-08-01T19:34:36.410 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:34:36.410 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:34:36.411 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:34:36.411 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:34:36.411 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.411 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:34:36.411 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6818", 2022-08-01T19:34:36.412 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:34:36.412 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.412 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.412 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:34:36.413 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6819", 2022-08-01T19:34:36.413 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:34:36.413 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.413 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.413 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.414 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:34:36.414 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:34:36.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.415 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:34:36.415 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6822", 2022-08-01T19:34:36.415 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:34:36.415 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.416 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.416 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:34:36.416 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6824", 2022-08-01T19:34:36.417 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:34:36.417 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.417 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.418 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.418 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:34:36.418 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:34:36.418 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.419 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:34:36.419 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6830", 2022-08-01T19:34:36.419 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:34:36.420 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.420 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.420 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:34:36.420 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6832", 2022-08-01T19:34:36.420 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:34:36.421 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.421 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.421 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.421 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:34:36.422 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:34:36.422 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.422 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:34:36.422 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6826", 2022-08-01T19:34:36.422 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:34:36.423 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.423 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.423 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:34:36.423 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6828", 2022-08-01T19:34:36.424 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:34:36.424 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.424 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.424 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.424 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6819/23586", 2022-08-01T19:34:36.425 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6824/23586", 2022-08-01T19:34:36.425 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6832/23586", 2022-08-01T19:34:36.425 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6828/23586", 2022-08-01T19:34:36.425 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:34:36.426 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:34:36.426 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:34:36.426 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:36.426 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.426 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:34:36.427 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_xinfo": [ 2022-08-01T19:34:36.427 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.427 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:34:36.427 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:34:36.428 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:34:36.428 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:34:36.428 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:34:36.428 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:34:36.428 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.588817+0000", 2022-08-01T19:34:36.429 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:34:36.429 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.429 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.429 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:34:36.430 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:34:36.430 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:34:36.430 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:34:36.430 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:34:36.430 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:34:36.431 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.782753+0000", 2022-08-01T19:34:36.431 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:34:36.431 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.431 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.432 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:34:36.432 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:34:36.432 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:34:36.432 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:34:36.432 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:34:36.433 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:34:36.433 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.837924+0000", 2022-08-01T19:34:36.433 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:34:36.433 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.434 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.434 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:34:36.434 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:34:36.434 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:34:36.435 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:34:36.435 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:34:36.435 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:34:36.435 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.839948+0000", 2022-08-01T19:34:36.435 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:34:36.436 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.436 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.436 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:34:36.436 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:34:36.437 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:34:36.437 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:34:36.437 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:34:36.437 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:34:36.437 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.112801+0000", 2022-08-01T19:34:36.438 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:34:36.438 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.438 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.438 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:34:36.439 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:34:36.439 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:34:36.439 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:34:36.439 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:34:36.439 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:34:36.440 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.106704+0000", 2022-08-01T19:34:36.440 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:34:36.440 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.440 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.441 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:34:36.441 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:34:36.441 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:34:36.441 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:34:36.441 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:34:36.442 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:34:36.442 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.893170+0000", 2022-08-01T19:34:36.442 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:34:36.442 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.443 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:36.443 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:34:36.443 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:34:36.443 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:34:36.443 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:34:36.444 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:34:36.444 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:34:36.444 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.886947+0000", 2022-08-01T19:34:36.444 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:34:36.445 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.445 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:34:36.445 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap": [], 2022-08-01T19:34:36.445 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap_items": [], 2022-08-01T19:34:36.445 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_temp": [], 2022-08-01T19:34:36.446 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_temp": [], 2022-08-01T19:34:36.446 INFO:teuthology.orchestra.run.smithi017.stdout: "blacklist": {}, 2022-08-01T19:34:36.446 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profiles": { 2022-08-01T19:34:36.446 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_data": { 2022-08-01T19:34:36.447 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-device-class": "", 2022-08-01T19:34:36.447 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:34:36.447 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-root": "default", 2022-08-01T19:34:36.447 INFO:teuthology.orchestra.run.smithi017.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T19:34:36.448 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:34:36.448 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "2", 2022-08-01T19:34:36.448 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:34:36.448 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:34:36.449 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van", 2022-08-01T19:34:36.449 INFO:teuthology.orchestra.run.smithi017.stdout: "w": "8" 2022-08-01T19:34:36.449 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.449 INFO:teuthology.orchestra.run.smithi017.stdout: "default": { 2022-08-01T19:34:36.449 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:34:36.450 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:34:36.450 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "1", 2022-08-01T19:34:36.450 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:34:36.450 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:34:36.451 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van" 2022-08-01T19:34:36.451 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:36.451 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:36.451 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps_queue": [], 2022-08-01T19:34:36.451 INFO:teuthology.orchestra.run.smithi017.stdout: "new_removed_snaps": [], 2022-08-01T19:34:36.452 INFO:teuthology.orchestra.run.smithi017.stdout: "new_purged_snaps": [], 2022-08-01T19:34:36.452 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_node_flags": {}, 2022-08-01T19:34:36.452 INFO:teuthology.orchestra.run.smithi017.stdout: "device_class_flags": {} 2022-08-01T19:34:36.452 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:34:36.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.198+0000 7f716cf2e700 1 -- 172.21.15.17:0/3953139274 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7144053a70 msgr2=0x7f7144055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:36.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.198+0000 7f716cf2e700 1 --2- 172.21.15.17:0/3953139274 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7144053a70 0x7f7144055f30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f715c01c000 tx=0x7f715c015000).stop 2022-08-01T19:34:36.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.198+0000 7f716cf2e700 1 -- 172.21.15.17:0/3953139274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71680b4550 msgr2=0x7f71680b2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:36.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.198+0000 7f716cf2e700 1 --2- 172.21.15.17:0/3953139274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71680b4550 0x7f71680b2650 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f715800c000 tx=0x7f7158006df0).stop 2022-08-01T19:34:36.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.198+0000 7f716cf2e700 1 -- 172.21.15.17:0/3953139274 shutdown_connections 2022-08-01T19:34:36.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.198+0000 7f716cf2e700 1 --2- 172.21.15.17:0/3953139274 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7144053a70 0x7f7144055f30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:36.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.198+0000 7f716cf2e700 1 --2- 172.21.15.17:0/3953139274 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71680b6710 0x7f7168146850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:36.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.198+0000 7f716cf2e700 1 --2- 172.21.15.17:0/3953139274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71680b4550 0x7f71680b2650 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:36.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.198+0000 7f716cf2e700 1 --2- 172.21.15.17:0/3953139274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7168061ec0 0x7f7168128c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:36.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.198+0000 7f716cf2e700 1 -- 172.21.15.17:0/3953139274 >> 172.21.15.17:0/3953139274 conn(0x7f716805c620 msgr2=0x7f7168127150 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:36.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.198+0000 7f716cf2e700 1 -- 172.21.15.17:0/3953139274 shutdown_connections 2022-08-01T19:34:36.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.198+0000 7f716cf2e700 1 -- 172.21.15.17:0/3953139274 wait complete. 2022-08-01T19:34:36.458 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:34:36.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.570+0000 7f858359e700 1 Processor -- start 2022-08-01T19:34:36.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.570+0000 7f858359e700 1 -- start start 2022-08-01T19:34:36.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.570+0000 7f858359e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f85840614b0 0x7f8584061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:36.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.570+0000 7f858359e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f85840ad750 0x7f85840b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:36.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.570+0000 7f858359e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f85841405d0 0x7f8584140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:36.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.570+0000 7f858359e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85840aee80 con 0x7f85841405d0 2022-08-01T19:34:36.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.570+0000 7f858359e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85840b70b0 con 0x7f85840ad750 2022-08-01T19:34:36.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.570+0000 7f858359e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85840af0b0 con 0x7f85840614b0 2022-08-01T19:34:36.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.570+0000 7f8582d9d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f85841405d0 0x7f8584140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:36.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.570+0000 7f8582d9d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f85841405d0 0x7f8584140a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37274/0 (socket says 172.21.15.17:37274) 2022-08-01T19:34:36.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.570+0000 7f8582d9d700 1 -- 172.21.15.17:0/2232616461 learned_addr learned my addr 172.21.15.17:0/2232616461 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:36.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.570+0000 7f8581d9b700 1 --2- 172.21.15.17:0/2232616461 >> v2:172.21.15.77:3300/0 conn(0x7f85840ad750 0x7f85840b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:36.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.570+0000 7f858259c700 1 --2- 172.21.15.17:0/2232616461 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f85840614b0 0x7f8584061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:36.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.570+0000 7f8582d9d700 1 -- 172.21.15.17:0/2232616461 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f85840614b0 msgr2=0x7f8584061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:36.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.570+0000 7f8582d9d700 1 --2- 172.21.15.17:0/2232616461 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f85840614b0 0x7f8584061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:36.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.570+0000 7f8582d9d700 1 -- 172.21.15.17:0/2232616461 >> v2:172.21.15.77:3300/0 conn(0x7f85840ad750 msgr2=0x7f85840b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:36.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.570+0000 7f8582d9d700 1 --2- 172.21.15.17:0/2232616461 >> v2:172.21.15.77:3300/0 conn(0x7f85840ad750 0x7f85840b9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:36.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.570+0000 7f8582d9d700 1 -- 172.21.15.17:0/2232616461 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85840af350 con 0x7f85841405d0 2022-08-01T19:34:36.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.570+0000 7f8582d9d700 1 --2- 172.21.15.17:0/2232616461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85841405d0 0x7f8584140a80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f857c0032f0 tx=0x7f857c009290).ready entity=mon.0 client_cookie=30d97ebc6ab9cd65 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:36.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.570+0000 7f858159a700 1 -- 172.21.15.17:0/2232616461 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f857c00b680 con 0x7f85841405d0 2022-08-01T19:34:36.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.570+0000 7f858159a700 1 -- 172.21.15.17:0/2232616461 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f857c00b7e0 con 0x7f85841405d0 2022-08-01T19:34:36.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.570+0000 7f858159a700 1 -- 172.21.15.17:0/2232616461 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f857c00b950 con 0x7f85841405d0 2022-08-01T19:34:36.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.570+0000 7f858359e700 1 -- 172.21.15.17:0/2232616461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85841405d0 msgr2=0x7f8584140a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:36.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.570+0000 7f858359e700 1 --2- 172.21.15.17:0/2232616461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85841405d0 0x7f8584140a80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f857c0032f0 tx=0x7f857c009290).stop 2022-08-01T19:34:36.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.570+0000 7f858359e700 1 -- 172.21.15.17:0/2232616461 shutdown_connections 2022-08-01T19:34:36.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.570+0000 7f858359e700 1 --2- 172.21.15.17:0/2232616461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85841405d0 0x7f8584140a80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:36.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.570+0000 7f858359e700 1 --2- 172.21.15.17:0/2232616461 >> v2:172.21.15.77:3300/0 conn(0x7f85840ad750 0x7f85840b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:36.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.570+0000 7f858359e700 1 --2- 172.21.15.17:0/2232616461 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f85840614b0 0x7f8584061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:36.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.570+0000 7f858359e700 1 -- 172.21.15.17:0/2232616461 >> 172.21.15.17:0/2232616461 conn(0x7f8584066f70 msgr2=0x7f8584067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:36.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.570+0000 7f858359e700 1 -- 172.21.15.17:0/2232616461 shutdown_connections 2022-08-01T19:34:36.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.570+0000 7f858359e700 1 -- 172.21.15.17:0/2232616461 wait complete. 2022-08-01T19:34:36.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.570+0000 7f858359e700 1 Processor -- start 2022-08-01T19:34:36.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.570+0000 7f858359e700 1 -- start start 2022-08-01T19:34:36.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.574+0000 7f858359e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85840614b0 0x7f8584145080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:36.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.574+0000 7f858359e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f85840ad750 0x7f85840ab3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:36.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.574+0000 7f858359e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85841405d0 0x7f85840ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:36.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.574+0000 7f858359e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858413ca70 con 0x7f85841405d0 2022-08-01T19:34:36.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.574+0000 7f858359e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85840b3e90 con 0x7f85840614b0 2022-08-01T19:34:36.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.574+0000 7f858359e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8584145570 con 0x7f85840ad750 2022-08-01T19:34:36.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.574+0000 7f8582d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85841405d0 0x7f85840ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:36.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.574+0000 7f8582d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85841405d0 0x7f85840ab8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37280/0 (socket says 172.21.15.17:37280) 2022-08-01T19:34:36.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.574+0000 7f8582d9d700 1 -- 172.21.15.17:0/1999952856 learned_addr learned my addr 172.21.15.17:0/1999952856 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:36.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.574+0000 7f858259c700 1 --2- 172.21.15.17:0/1999952856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85840614b0 0x7f8584145080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:36.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.574+0000 7f8581d9b700 1 --2- 172.21.15.17:0/1999952856 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f85840ad750 0x7f85840ab3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:36.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.574+0000 7f8582d9d700 1 -- 172.21.15.17:0/1999952856 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f85840ad750 msgr2=0x7f85840ab3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:36.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.574+0000 7f8582d9d700 1 --2- 172.21.15.17:0/1999952856 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f85840ad750 0x7f85840ab3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:36.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.574+0000 7f8582d9d700 1 -- 172.21.15.17:0/1999952856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85840614b0 msgr2=0x7f8584145080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:36.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.574+0000 7f8582d9d700 1 --2- 172.21.15.17:0/1999952856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85840614b0 0x7f8584145080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:36.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.574+0000 7f8582d9d700 1 -- 172.21.15.17:0/1999952856 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f857c009530 con 0x7f85841405d0 2022-08-01T19:34:36.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.574+0000 7f8582d9d700 1 --2- 172.21.15.17:0/1999952856 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85841405d0 0x7f85840ab8b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f857c0032f0 tx=0x7f857c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:36.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.574+0000 7f85737fe700 1 -- 172.21.15.17:0/1999952856 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f857c01f070 con 0x7f85841405d0 2022-08-01T19:34:36.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.574+0000 7f85737fe700 1 -- 172.21.15.17:0/1999952856 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f857c011e80 con 0x7f85841405d0 2022-08-01T19:34:36.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.574+0000 7f858359e700 1 -- 172.21.15.17:0/1999952856 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85840ad040 con 0x7f85841405d0 2022-08-01T19:34:36.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.574+0000 7f85737fe700 1 -- 172.21.15.17:0/1999952856 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f857c00b680 con 0x7f85841405d0 2022-08-01T19:34:36.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.574+0000 7f858359e700 1 -- 172.21.15.17:0/1999952856 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8584142e60 con 0x7f85841405d0 2022-08-01T19:34:36.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.574+0000 7f858359e700 1 -- 172.21.15.17:0/1999952856 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f858413cc30 con 0x7f85841405d0 2022-08-01T19:34:36.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.578+0000 7f85737fe700 1 -- 172.21.15.17:0/1999952856 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f857c00f070 con 0x7f85841405d0 2022-08-01T19:34:36.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.578+0000 7f85737fe700 1 --2- 172.21.15.17:0/1999952856 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8560053950 0x7f8560055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:36.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.578+0000 7f85737fe700 1 -- 172.21.15.17:0/1999952856 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f857c06d520 con 0x7f85841405d0 2022-08-01T19:34:36.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.578+0000 7f858259c700 1 --2- 172.21.15.17:0/1999952856 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8560053950 0x7f8560055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:36.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.578+0000 7f858259c700 1 --2- 172.21.15.17:0/1999952856 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8560053950 0x7f8560055e10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f857800b000 tx=0x7f8578006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:36.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.582+0000 7f85737fe700 1 -- 172.21.15.17:0/1999952856 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f857c0442e0 con 0x7f85841405d0 2022-08-01T19:34:36.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.750+0000 7f858359e700 1 -- 172.21.15.17:0/1999952856 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85840af5f0 con 0x7f85841405d0 2022-08-01T19:34:36.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.750+0000 7f85737fe700 1 -- 172.21.15.17:0/1999952856 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+2486 (secure 0 0 0) 0x7f857c047900 con 0x7f85841405d0 2022-08-01T19:34:36.757 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:36.757 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2116883486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2116883486},{"type":"v1","addr":"172.21.15.17:6833","nonce":2116883486}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.17:6835/2974833789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2974833789},{"type":"v1","addr":"172.21.15.17:6835","nonce":2974833789}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":4}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:34:09.692662+0000","modified":"2022-08-01T19:34:11.425752+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":5,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6835/1842780453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1842780453},{"type":"v1","addr":"172.21.15.77:6835","nonce":1842780453}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2022-08-01T19:34:36.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.754+0000 7f858359e700 1 -- 172.21.15.17:0/1999952856 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8560053950 msgr2=0x7f8560055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:36.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.754+0000 7f858359e700 1 --2- 172.21.15.17:0/1999952856 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8560053950 0x7f8560055e10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f857800b000 tx=0x7f8578006df0).stop 2022-08-01T19:34:36.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.754+0000 7f858359e700 1 -- 172.21.15.17:0/1999952856 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85841405d0 msgr2=0x7f85840ab8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:36.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.754+0000 7f858359e700 1 --2- 172.21.15.17:0/1999952856 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85841405d0 0x7f85840ab8b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f857c0032f0 tx=0x7f857c017000).stop 2022-08-01T19:34:36.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.754+0000 7f858359e700 1 -- 172.21.15.17:0/1999952856 shutdown_connections 2022-08-01T19:34:36.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.754+0000 7f858359e700 1 --2- 172.21.15.17:0/1999952856 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8560053950 0x7f8560055e10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:36.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.754+0000 7f858359e700 1 --2- 172.21.15.17:0/1999952856 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85841405d0 0x7f85840ab8b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:36.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.754+0000 7f858359e700 1 --2- 172.21.15.17:0/1999952856 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f85840ad750 0x7f85840ab3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:36.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.754+0000 7f858359e700 1 --2- 172.21.15.17:0/1999952856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85840614b0 0x7f8584145080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:36.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.754+0000 7f858359e700 1 -- 172.21.15.17:0/1999952856 >> 172.21.15.17:0/1999952856 conn(0x7f8584066f70 msgr2=0x7f8584067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:36.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.754+0000 7f858359e700 1 -- 172.21.15.17:0/1999952856 shutdown_connections 2022-08-01T19:34:36.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.754+0000 7f858359e700 1 -- 172.21.15.17:0/1999952856 wait complete. 2022-08-01T19:34:36.762 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 7 2022-08-01T19:34:36.770 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-08-01T19:34:36.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.870+0000 7f911b59e700 1 Processor -- start 2022-08-01T19:34:36.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.870+0000 7f911b59e700 1 -- start start 2022-08-01T19:34:36.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.870+0000 7f911b59e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f911c0b07a0 0x7f911c0af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:36.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.870+0000 7f911b59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f911c147730 0x7f911c0acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:36.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.870+0000 7f911b59e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f911c0b5a10 con 0x7f911c0b5630 2022-08-01T19:34:36.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.870+0000 7f911b59e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f911c145470 con 0x7f911c0b07a0 2022-08-01T19:34:36.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.870+0000 7f911b59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f911c0ae180 con 0x7f911c147730 2022-08-01T19:34:36.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.870+0000 7f9111d9b700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f911c0b07a0 0x7f911c0af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:36.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.870+0000 7f911ad9d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f911c147730 0x7f911c0acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:36.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.870+0000 7f9111d9b700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f911c0b07a0 0x7f911c0af6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:35208/0 (socket says 172.21.15.17:35208) 2022-08-01T19:34:36.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.870+0000 7f911a59c700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7f911c0b5630 0x7f911c067730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:46278/0 (socket says 172.21.15.17:46278) 2022-08-01T19:34:36.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.870+0000 7f9111d9b700 1 -- 172.21.15.17:0/1126306992 learned_addr learned my addr 172.21.15.17:0/1126306992 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:36.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.870+0000 7f911ad9d700 1 -- 172.21.15.17:0/1126306992 >> v2:172.21.15.77:3300/0 conn(0x7f911c0b07a0 msgr2=0x7f911c0af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:36.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.870+0000 7f911ad9d700 1 --2- 172.21.15.17:0/1126306992 >> v2:172.21.15.77:3300/0 conn(0x7f911c0b07a0 0x7f911c0af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:36.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.870+0000 7f911ad9d700 1 -- 172.21.15.17:0/1126306992 >> v1:172.21.15.77:6789/0 conn(0x7f911c0b5630 legacy=0x7f911c067730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:36.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.870+0000 7f911ad9d700 1 -- 172.21.15.17:0/1126306992 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911c0ac990 con 0x7f911c147730 2022-08-01T19:34:36.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.870+0000 7f9111d9b700 1 --2- 172.21.15.17:0/1126306992 >> v2:172.21.15.77:3300/0 conn(0x7f911c0b07a0 0x7f911c0af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:34:36.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.870+0000 7f9119d9b700 1 -- 172.21.15.17:0/1126306992 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1287678479 0 0) 0x7f911c0b5a10 con 0x7f911c0b5630 2022-08-01T19:34:36.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.870+0000 7f911ad9d700 1 --2- 172.21.15.17:0/1126306992 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f911c147730 0x7f911c0acdf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f91140032f0 tx=0x7f91140091d0).ready entity=mon.2 client_cookie=c3db403e4f43a54f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:36.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.870+0000 7f9119d9b700 1 -- 172.21.15.17:0/1126306992 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f911400b680 con 0x7f911c147730 2022-08-01T19:34:36.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.870+0000 7f9119d9b700 1 -- 172.21.15.17:0/1126306992 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911400b7e0 con 0x7f911c147730 2022-08-01T19:34:36.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.870+0000 7f9119d9b700 1 -- 172.21.15.17:0/1126306992 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f911400b950 con 0x7f911c147730 2022-08-01T19:34:36.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.870+0000 7f911b59e700 1 -- 172.21.15.17:0/1126306992 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f911c147730 msgr2=0x7f911c0acdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:36.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.870+0000 7f911b59e700 1 --2- 172.21.15.17:0/1126306992 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f911c147730 0x7f911c0acdf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f91140032f0 tx=0x7f91140091d0).stop 2022-08-01T19:34:36.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.870+0000 7f911b59e700 1 -- 172.21.15.17:0/1126306992 shutdown_connections 2022-08-01T19:34:36.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.870+0000 7f911b59e700 1 --2- 172.21.15.17:0/1126306992 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f911c147730 0x7f911c0acdf0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:36.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.870+0000 7f911b59e700 1 --2- 172.21.15.17:0/1126306992 >> v2:172.21.15.77:3300/0 conn(0x7f911c0b07a0 0x7f911c0af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:36.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.870+0000 7f911b59e700 1 -- 172.21.15.17:0/1126306992 >> 172.21.15.17:0/1126306992 conn(0x7f911c0626f0 msgr2=0x7f911c062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:36.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.870+0000 7f911b59e700 1 -- 172.21.15.17:0/1126306992 shutdown_connections 2022-08-01T19:34:36.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.870+0000 7f911b59e700 1 -- 172.21.15.17:0/1126306992 wait complete. 2022-08-01T19:34:36.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.874+0000 7f911b59e700 1 Processor -- start 2022-08-01T19:34:36.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.874+0000 7f911b59e700 1 -- start start 2022-08-01T19:34:36.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.874+0000 7f911b59e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f911c0b07a0 0x7f911c140c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:36.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.874+0000 7f911b59e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f911c0b5630 0x7f911c1402d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:36.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.874+0000 7f911b59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f911c147730 0x7f911c145690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:36.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.874+0000 7f911b59e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911c0b3160 con 0x7f911c0b5630 2022-08-01T19:34:36.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.874+0000 7f911b59e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911c0b3400 con 0x7f911c0b07a0 2022-08-01T19:34:36.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.874+0000 7f911b59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f911c0b36a0 con 0x7f911c147730 2022-08-01T19:34:36.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.874+0000 7f9111d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f911c0b5630 0x7f911c1402d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:36.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.874+0000 7f9111d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f911c0b5630 0x7f911c1402d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37292/0 (socket says 172.21.15.17:37292) 2022-08-01T19:34:36.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.874+0000 7f9111d9b700 1 -- 172.21.15.17:0/4733487 learned_addr learned my addr 172.21.15.17:0/4733487 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:36.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.874+0000 7f911ad9d700 1 --2- 172.21.15.17:0/4733487 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f911c147730 0x7f911c145690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:36.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.874+0000 7f911a59c700 1 --2- 172.21.15.17:0/4733487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f911c0b07a0 0x7f911c140c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:36.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.874+0000 7f9111d9b700 1 -- 172.21.15.17:0/4733487 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f911c147730 msgr2=0x7f911c145690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:36.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.874+0000 7f9111d9b700 1 --2- 172.21.15.17:0/4733487 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f911c147730 0x7f911c145690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:36.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.874+0000 7f9111d9b700 1 -- 172.21.15.17:0/4733487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f911c0b07a0 msgr2=0x7f911c140c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:36.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.874+0000 7f9111d9b700 1 --2- 172.21.15.17:0/4733487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f911c0b07a0 0x7f911c140c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:36.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.874+0000 7f9111d9b700 1 -- 172.21.15.17:0/4733487 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f910400b1e0 con 0x7f911c0b5630 2022-08-01T19:34:36.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.874+0000 7f9111d9b700 1 --2- 172.21.15.17:0/4733487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f911c0b5630 0x7f911c1402d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f910400c000 tx=0x7f9104006eb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:36.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.874+0000 7f9113fff700 1 -- 172.21.15.17:0/4733487 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9104012600 con 0x7f911c0b5630 2022-08-01T19:34:36.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.874+0000 7f9113fff700 1 -- 172.21.15.17:0/4733487 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9104012760 con 0x7f911c0b5630 2022-08-01T19:34:36.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.874+0000 7f911b59e700 1 -- 172.21.15.17:0/4733487 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9114009470 con 0x7f911c0b5630 2022-08-01T19:34:36.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.874+0000 7f9113fff700 1 -- 172.21.15.17:0/4733487 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9104012ac0 con 0x7f911c0b5630 2022-08-01T19:34:36.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.874+0000 7f911b59e700 1 -- 172.21.15.17:0/4733487 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f911c141550 con 0x7f911c0b5630 2022-08-01T19:34:36.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.878+0000 7f911b59e700 1 -- 172.21.15.17:0/4733487 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f911c0574e0 con 0x7f911c0b5630 2022-08-01T19:34:36.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.878+0000 7f9113fff700 1 -- 172.21.15.17:0/4733487 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9104022070 con 0x7f911c0b5630 2022-08-01T19:34:36.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.878+0000 7f9113fff700 1 --2- 172.21.15.17:0/4733487 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9100053aa0 0x7f9100055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:36.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.878+0000 7f9113fff700 1 -- 172.21.15.17:0/4733487 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f910406c460 con 0x7f911c0b5630 2022-08-01T19:34:36.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.878+0000 7f911a59c700 1 --2- 172.21.15.17:0/4733487 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9100053aa0 0x7f9100055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:36.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.878+0000 7f911a59c700 1 --2- 172.21.15.17:0/4733487 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9100053aa0 0x7f9100055f60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f910c00d000 tx=0x7f910c007a00).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:36.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:36.886+0000 7f9113fff700 1 -- 172.21.15.17:0/4733487 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9104043220 con 0x7f911c0b5630 2022-08-01T19:34:37.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.054+0000 7f911b59e700 1 -- 172.21.15.17:0/4733487 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f911c05f3d0 con 0x7f911c0b5630 2022-08-01T19:34:37.162 INFO:tasks.ceph.mon.a.smithi017.stderr:2022-08-01T19:34:37.158+0000 7f6a53773700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-08-01T19:34:37.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.166+0000 7f9113fff700 1 -- 172.21.15.17:0/4733487 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9104046840 con 0x7f911c0b5630 2022-08-01T19:34:37.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.170+0000 7f911b59e700 1 -- 172.21.15.17:0/4733487 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9100053aa0 msgr2=0x7f9100055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:37.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.170+0000 7f911b59e700 1 --2- 172.21.15.17:0/4733487 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9100053aa0 0x7f9100055f60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f910c00d000 tx=0x7f910c007a00).stop 2022-08-01T19:34:37.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.170+0000 7f911b59e700 1 -- 172.21.15.17:0/4733487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f911c0b5630 msgr2=0x7f911c1402d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:37.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.170+0000 7f911b59e700 1 --2- 172.21.15.17:0/4733487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f911c0b5630 0x7f911c1402d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f910400c000 tx=0x7f9104006eb0).stop 2022-08-01T19:34:37.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.170+0000 7f911b59e700 1 -- 172.21.15.17:0/4733487 shutdown_connections 2022-08-01T19:34:37.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.170+0000 7f911b59e700 1 --2- 172.21.15.17:0/4733487 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9100053aa0 0x7f9100055f60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:37.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.170+0000 7f911b59e700 1 --2- 172.21.15.17:0/4733487 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f911c147730 0x7f911c145690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:37.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.170+0000 7f911b59e700 1 --2- 172.21.15.17:0/4733487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f911c0b5630 0x7f911c1402d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:37.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.170+0000 7f911b59e700 1 --2- 172.21.15.17:0/4733487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f911c0b07a0 0x7f911c140c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:37.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.170+0000 7f911b59e700 1 -- 172.21.15.17:0/4733487 >> 172.21.15.17:0/4733487 conn(0x7f911c0626f0 msgr2=0x7f911c0acd10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:37.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.170+0000 7f911b59e700 1 -- 172.21.15.17:0/4733487 shutdown_connections 2022-08-01T19:34:37.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.170+0000 7f911b59e700 1 -- 172.21.15.17:0/4733487 wait complete. 2022-08-01T19:34:37.178 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-08-01T19:34:37.189 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:34:37.201 INFO:tasks.ceph.mds.b.smithi077.stdout:starting mds.b at 2022-08-01T19:34:37.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.302+0000 7fbca7fff700 1 Processor -- start 2022-08-01T19:34:37.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.302+0000 7fbca7fff700 1 -- start start 2022-08-01T19:34:37.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.302+0000 7fbca7fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbca80b2710 0x7fbca814afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:37.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.302+0000 7fbca7fff700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbca80a7250 con 0x7fbca80a8c50 2022-08-01T19:34:37.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.302+0000 7fbca7fff700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbca80a7480 con 0x7fbca80b01a0 2022-08-01T19:34:37.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.302+0000 7fbca7fff700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbca80a7720 con 0x7fbca80b2710 2022-08-01T19:34:37.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.306+0000 7fbca6ffd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbca80b2710 0x7fbca814afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:37.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.306+0000 7fbca77fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fbca80a8c50 0x7fbca8151d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57728/0 (socket says 172.21.15.17:57728) 2022-08-01T19:34:37.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.306+0000 7fbca67fc700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7fbca80b01a0 0x7fbca80a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:46292/0 (socket says 172.21.15.17:46292) 2022-08-01T19:34:37.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.306+0000 7fbca77fe700 1 -- 172.21.15.17:0/1325309549 learned_addr learned my addr 172.21.15.17:0/1325309549 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:37.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.306+0000 7fbca5ffb700 1 -- 172.21.15.17:0/1325309549 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 765665689 0 0) 0x7fbca80a7250 con 0x7fbca80a8c50 2022-08-01T19:34:37.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.306+0000 7fbca5ffb700 1 -- 172.21.15.17:0/1325309549 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc88003260 con 0x7fbca80a8c50 2022-08-01T19:34:37.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.306+0000 7fbca5ffb700 1 -- 172.21.15.17:0/1325309549 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 689301422 0 0) 0x7fbca80a7480 con 0x7fbca80b01a0 2022-08-01T19:34:37.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.306+0000 7fbca5ffb700 1 -- 172.21.15.17:0/1325309549 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbca80a7250 con 0x7fbca80b01a0 2022-08-01T19:34:37.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.306+0000 7fbca6ffd700 1 -- 172.21.15.17:0/1325309549 >> v1:172.21.15.77:6789/0 conn(0x7fbca80b01a0 legacy=0x7fbca80a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:37.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.306+0000 7fbca6ffd700 1 -- 172.21.15.17:0/1325309549 >> v1:172.21.15.17:6789/0 conn(0x7fbca80a8c50 legacy=0x7fbca8151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:37.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.306+0000 7fbca6ffd700 1 -- 172.21.15.17:0/1325309549 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca80a79c0 con 0x7fbca80b2710 2022-08-01T19:34:37.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.306+0000 7fbca6ffd700 1 --2- 172.21.15.17:0/1325309549 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbca80b2710 0x7fbca814afd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fbc90002f70 tx=0x7fbc90006df0).ready entity=mon.0 client_cookie=8d8c33bf29040ef2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:37.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.306+0000 7fbca5ffb700 1 -- 172.21.15.17:0/1325309549 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc9000d540 con 0x7fbca80b2710 2022-08-01T19:34:37.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.306+0000 7fbca5ffb700 1 -- 172.21.15.17:0/1325309549 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc90000e90 con 0x7fbca80b2710 2022-08-01T19:34:37.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.306+0000 7fbca5ffb700 1 -- 172.21.15.17:0/1325309549 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc9000d890 con 0x7fbca80b2710 2022-08-01T19:34:37.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.306+0000 7fbca7fff700 1 -- 172.21.15.17:0/1325309549 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbca80b2710 msgr2=0x7fbca814afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:37.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.306+0000 7fbca7fff700 1 --2- 172.21.15.17:0/1325309549 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbca80b2710 0x7fbca814afd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fbc90002f70 tx=0x7fbc90006df0).stop 2022-08-01T19:34:37.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.306+0000 7fbca7fff700 1 -- 172.21.15.17:0/1325309549 shutdown_connections 2022-08-01T19:34:37.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.306+0000 7fbca7fff700 1 --2- 172.21.15.17:0/1325309549 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbca80b2710 0x7fbca814afd0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:37.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.306+0000 7fbca7fff700 1 -- 172.21.15.17:0/1325309549 >> 172.21.15.17:0/1325309549 conn(0x7fbca8060310 msgr2=0x7fbca80b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:37.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.306+0000 7fbca7fff700 1 -- 172.21.15.17:0/1325309549 shutdown_connections 2022-08-01T19:34:37.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.306+0000 7fbca7fff700 1 -- 172.21.15.17:0/1325309549 wait complete. 2022-08-01T19:34:37.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.310+0000 7fbca7fff700 1 Processor -- start 2022-08-01T19:34:37.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.310+0000 7fbca7fff700 1 -- start start 2022-08-01T19:34:37.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.310+0000 7fbca7fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbca80a8c50 0x7fbca80af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:37.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.310+0000 7fbca7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbca80b01a0 0x7fbca80b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:37.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.310+0000 7fbca7fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbca80b2710 0x7fbca81572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:37.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.310+0000 7fbca7fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca80b3450 con 0x7fbca80b01a0 2022-08-01T19:34:37.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.310+0000 7fbca7fff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca80b3670 con 0x7fbca80a8c50 2022-08-01T19:34:37.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.310+0000 7fbca7fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbca8156910 con 0x7fbca80b2710 2022-08-01T19:34:37.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.310+0000 7fbca67fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbca80b01a0 0x7fbca80b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:37.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.310+0000 7fbca6ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbca80a8c50 0x7fbca80af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:37.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.310+0000 7fbca67fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbca80b01a0 0x7fbca80b2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37306/0 (socket says 172.21.15.17:37306) 2022-08-01T19:34:37.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.310+0000 7fbca67fc700 1 -- 172.21.15.17:0/481564526 learned_addr learned my addr 172.21.15.17:0/481564526 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:37.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.310+0000 7fbca77fe700 1 --2- 172.21.15.17:0/481564526 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbca80b2710 0x7fbca81572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:37.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.314+0000 7fbca67fc700 1 -- 172.21.15.17:0/481564526 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbca80b2710 msgr2=0x7fbca81572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:37.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.314+0000 7fbca67fc700 1 --2- 172.21.15.17:0/481564526 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbca80b2710 0x7fbca81572f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:37.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.314+0000 7fbca67fc700 1 -- 172.21.15.17:0/481564526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbca80a8c50 msgr2=0x7fbca80af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:37.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.314+0000 7fbca67fc700 1 --2- 172.21.15.17:0/481564526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbca80a8c50 0x7fbca80af2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:37.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.314+0000 7fbca67fc700 1 -- 172.21.15.17:0/481564526 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc9000b180 con 0x7fbca80b01a0 2022-08-01T19:34:37.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.314+0000 7fbca6ffd700 1 --2- 172.21.15.17:0/481564526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbca80a8c50 0x7fbca80af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:34:37.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.314+0000 7fbca67fc700 1 --2- 172.21.15.17:0/481564526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbca80b01a0 0x7fbca80b2f60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7fbca0001110 tx=0x7fbca0007a30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:37.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.314+0000 7fbc9ffff700 1 -- 172.21.15.17:0/481564526 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbca00070d0 con 0x7fbca80b01a0 2022-08-01T19:34:37.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.314+0000 7fbc9ffff700 1 -- 172.21.15.17:0/481564526 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca0007230 con 0x7fbca80b01a0 2022-08-01T19:34:37.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.314+0000 7fbca7fff700 1 -- 172.21.15.17:0/481564526 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca8156bb0 con 0x7fbca80b01a0 2022-08-01T19:34:37.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.314+0000 7fbc9ffff700 1 -- 172.21.15.17:0/481564526 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbca0005740 con 0x7fbca80b01a0 2022-08-01T19:34:37.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.314+0000 7fbca7fff700 1 -- 172.21.15.17:0/481564526 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca81577e0 con 0x7fbca80b01a0 2022-08-01T19:34:37.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.314+0000 7fbca7fff700 1 -- 172.21.15.17:0/481564526 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbca80574e0 con 0x7fbca80b01a0 2022-08-01T19:34:37.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.314+0000 7fbc9ffff700 1 -- 172.21.15.17:0/481564526 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbca0010070 con 0x7fbca80b01a0 2022-08-01T19:34:37.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.314+0000 7fbc9ffff700 1 --2- 172.21.15.17:0/481564526 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbc8c0539e0 0x7fbc8c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:37.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.314+0000 7fbc9ffff700 1 -- 172.21.15.17:0/481564526 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6517+0+0 (secure 0 0 0) 0x7fbca006b5e0 con 0x7fbca80b01a0 2022-08-01T19:34:37.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.318+0000 7fbca6ffd700 1 --2- 172.21.15.17:0/481564526 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbc8c0539e0 0x7fbc8c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:37.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.318+0000 7fbca6ffd700 1 --2- 172.21.15.17:0/481564526 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbc8c0539e0 0x7fbc8c055ea0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fbc90018000 tx=0x7fbc90014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:37.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.322+0000 7fbc9ffff700 1 -- 172.21.15.17:0/481564526 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbca00422a0 con 0x7fbca80b01a0 2022-08-01T19:34:37.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.490+0000 7fbca7fff700 1 -- 172.21.15.17:0/481564526 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbca8157050 con 0x7fbca80b01a0 2022-08-01T19:34:37.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.490+0000 7fbc9ffff700 1 -- 172.21.15.17:0/481564526 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+2126 (secure 0 0 0) 0x7fbca0018070 con 0x7fbca80b01a0 2022-08-01T19:34:37.495 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:37.495 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2116883486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2116883486},{"type":"v1","addr":"172.21.15.17:6833","nonce":2116883486}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.17:6835/2974833789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2974833789},{"type":"v1","addr":"172.21.15.17:6835","nonce":2974833789}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":4}],"filesystems":[{"mdsmap":{"epoch":8,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:34:09.692662+0000","modified":"2022-08-01T19:34:37.162525+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2022-08-01T19:34:37.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.490+0000 7fbca7fff700 1 -- 172.21.15.17:0/481564526 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbc8c0539e0 msgr2=0x7fbc8c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:37.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.490+0000 7fbca7fff700 1 --2- 172.21.15.17:0/481564526 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbc8c0539e0 0x7fbc8c055ea0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fbc90018000 tx=0x7fbc90014000).stop 2022-08-01T19:34:37.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.490+0000 7fbca7fff700 1 -- 172.21.15.17:0/481564526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbca80b01a0 msgr2=0x7fbca80b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:37.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.490+0000 7fbca7fff700 1 --2- 172.21.15.17:0/481564526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbca80b01a0 0x7fbca80b2f60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7fbca0001110 tx=0x7fbca0007a30).stop 2022-08-01T19:34:37.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.490+0000 7fbca7fff700 1 -- 172.21.15.17:0/481564526 shutdown_connections 2022-08-01T19:34:37.499 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.490+0000 7fbca7fff700 1 --2- 172.21.15.17:0/481564526 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbc8c0539e0 0x7fbc8c055ea0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:37.499 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.490+0000 7fbca7fff700 1 --2- 172.21.15.17:0/481564526 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbca80b2710 0x7fbca81572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:37.499 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.490+0000 7fbca7fff700 1 --2- 172.21.15.17:0/481564526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbca80b01a0 0x7fbca80b2f60 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:37.499 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.490+0000 7fbca7fff700 1 --2- 172.21.15.17:0/481564526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbca80a8c50 0x7fbca80af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:37.499 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.490+0000 7fbca7fff700 1 -- 172.21.15.17:0/481564526 >> 172.21.15.17:0/481564526 conn(0x7fbca8060310 msgr2=0x7fbca8144b30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:37.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.490+0000 7fbca7fff700 1 -- 172.21.15.17:0/481564526 shutdown_connections 2022-08-01T19:34:37.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.490+0000 7fbca7fff700 1 -- 172.21.15.17:0/481564526 wait complete. 2022-08-01T19:34:37.501 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 8 2022-08-01T19:34:37.509 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-08-01T19:34:37.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.610+0000 7f4b5c25f700 1 Processor -- start 2022-08-01T19:34:37.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.610+0000 7f4b5c25f700 1 -- start start 2022-08-01T19:34:37.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.614+0000 7f4b5c25f700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4b54146ae0 0x7f4b541403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:37.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.614+0000 7f4b5c25f700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f4b540b4820 0x7f4b540b5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:37.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.614+0000 7f4b5c25f700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b54143cf0 con 0x7f4b540b4310 2022-08-01T19:34:37.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.614+0000 7f4b5c25f700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b541476a0 con 0x7f4b540b4820 2022-08-01T19:34:37.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.614+0000 7f4b5c25f700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b54143f90 con 0x7f4b54146ae0 2022-08-01T19:34:37.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.614+0000 7f4b597fa700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f4b540b4310 0x7f4b541408b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57740/0 (socket says 172.21.15.17:57740) 2022-08-01T19:34:37.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.614+0000 7f4b597fa700 1 -- 172.21.15.17:0/3764357320 learned_addr learned my addr 172.21.15.17:0/3764357320 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:37.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.614+0000 7f4b59ffb700 1 --2- 172.21.15.17:0/3764357320 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4b54146ae0 0x7f4b541403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:37.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.614+0000 7f4b5a7fc700 1 --2- 172.21.15.17:0/3764357320 >> v2:172.21.15.77:3300/0 conn(0x7f4b540b4820 0x7f4b540b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:37.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.614+0000 7f4b58ff9700 1 -- 172.21.15.17:0/3764357320 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1613444192 0 0) 0x7f4b54143cf0 con 0x7f4b540b4310 2022-08-01T19:34:37.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.614+0000 7f4b58ff9700 1 -- 172.21.15.17:0/3764357320 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b3c003260 con 0x7f4b540b4310 2022-08-01T19:34:37.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.614+0000 7f4b59ffb700 1 -- 172.21.15.17:0/3764357320 >> v2:172.21.15.77:3300/0 conn(0x7f4b540b4820 msgr2=0x7f4b540b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:37.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.614+0000 7f4b59ffb700 1 --2- 172.21.15.17:0/3764357320 >> v2:172.21.15.77:3300/0 conn(0x7f4b540b4820 0x7f4b540b5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:37.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.614+0000 7f4b59ffb700 1 -- 172.21.15.17:0/3764357320 >> v1:172.21.15.17:6789/0 conn(0x7f4b540b4310 legacy=0x7f4b541408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:37.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.614+0000 7f4b59ffb700 1 -- 172.21.15.17:0/3764357320 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b540b0e20 con 0x7f4b54146ae0 2022-08-01T19:34:37.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.614+0000 7f4b5a7fc700 1 --2- 172.21.15.17:0/3764357320 >> v2:172.21.15.77:3300/0 conn(0x7f4b540b4820 0x7f4b540b5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:34:37.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.614+0000 7f4b59ffb700 1 --2- 172.21.15.17:0/3764357320 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4b54146ae0 0x7f4b541403c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f4b50002f70 tx=0x7f4b50006df0).ready entity=mon.2 client_cookie=339e4bf76ca2544a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:37.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.614+0000 7f4b58ff9700 1 -- 172.21.15.17:0/3764357320 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b5000d4a0 con 0x7f4b54146ae0 2022-08-01T19:34:37.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.614+0000 7f4b58ff9700 1 -- 172.21.15.17:0/3764357320 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b50000e90 con 0x7f4b54146ae0 2022-08-01T19:34:37.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.614+0000 7f4b58ff9700 1 -- 172.21.15.17:0/3764357320 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b5000d7f0 con 0x7f4b54146ae0 2022-08-01T19:34:37.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.614+0000 7f4b5c25f700 1 -- 172.21.15.17:0/3764357320 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4b54146ae0 msgr2=0x7f4b541403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:37.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.614+0000 7f4b5c25f700 1 --2- 172.21.15.17:0/3764357320 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4b54146ae0 0x7f4b541403c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f4b50002f70 tx=0x7f4b50006df0).stop 2022-08-01T19:34:37.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.614+0000 7f4b5c25f700 1 -- 172.21.15.17:0/3764357320 shutdown_connections 2022-08-01T19:34:37.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.614+0000 7f4b5c25f700 1 --2- 172.21.15.17:0/3764357320 >> v2:172.21.15.77:3300/0 conn(0x7f4b540b4820 0x7f4b540b5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:37.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.614+0000 7f4b5c25f700 1 --2- 172.21.15.17:0/3764357320 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4b54146ae0 0x7f4b541403c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:37.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.614+0000 7f4b5c25f700 1 -- 172.21.15.17:0/3764357320 >> 172.21.15.17:0/3764357320 conn(0x7f4b5405c620 msgr2=0x7f4b5405ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:37.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.614+0000 7f4b5c25f700 1 -- 172.21.15.17:0/3764357320 shutdown_connections 2022-08-01T19:34:37.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.614+0000 7f4b5c25f700 1 -- 172.21.15.17:0/3764357320 wait complete. 2022-08-01T19:34:37.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.618+0000 7f4b5c25f700 1 Processor -- start 2022-08-01T19:34:37.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.618+0000 7f4b5c25f700 1 -- start start 2022-08-01T19:34:37.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.618+0000 7f4b5c25f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b540b4310 0x7f4b540adbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:37.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.618+0000 7f4b5c25f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b540b4820 0x7f4b54145e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:37.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.618+0000 7f4b5c25f700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4b54146ae0 0x7f4b541563f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:37.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.618+0000 7f4b5c25f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b540ae370 con 0x7f4b540b4820 2022-08-01T19:34:37.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.618+0000 7f4b5c25f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b54142000 con 0x7f4b540b4310 2022-08-01T19:34:37.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.618+0000 7f4b5c25f700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b541422a0 con 0x7f4b54146ae0 2022-08-01T19:34:37.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.618+0000 7f4b597fa700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b540b4820 0x7f4b54145e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:37.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.618+0000 7f4b597fa700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b540b4820 0x7f4b54145e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37320/0 (socket says 172.21.15.17:37320) 2022-08-01T19:34:37.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.618+0000 7f4b597fa700 1 -- 172.21.15.17:0/2833609627 learned_addr learned my addr 172.21.15.17:0/2833609627 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:37.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.618+0000 7f4b5a7fc700 1 --2- 172.21.15.17:0/2833609627 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4b54146ae0 0x7f4b541563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:37.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.618+0000 7f4b59ffb700 1 --2- 172.21.15.17:0/2833609627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b540b4310 0x7f4b540adbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:37.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.618+0000 7f4b597fa700 1 -- 172.21.15.17:0/2833609627 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4b54146ae0 msgr2=0x7f4b541563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:37.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.618+0000 7f4b597fa700 1 --2- 172.21.15.17:0/2833609627 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4b54146ae0 0x7f4b541563f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:37.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.618+0000 7f4b597fa700 1 -- 172.21.15.17:0/2833609627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b540b4310 msgr2=0x7f4b540adbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:37.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.618+0000 7f4b597fa700 1 --2- 172.21.15.17:0/2833609627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b540b4310 0x7f4b540adbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:37.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.618+0000 7f4b597fa700 1 -- 172.21.15.17:0/2833609627 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b4c009470 con 0x7f4b540b4820 2022-08-01T19:34:37.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.618+0000 7f4b597fa700 1 --2- 172.21.15.17:0/2833609627 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b540b4820 0x7f4b54145e40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f4b44001110 tx=0x7f4b44007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:37.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.622+0000 7f4b42ffd700 1 -- 172.21.15.17:0/2833609627 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b4400ce00 con 0x7f4b540b4820 2022-08-01T19:34:37.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.622+0000 7f4b42ffd700 1 -- 172.21.15.17:0/2833609627 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b440072e0 con 0x7f4b540b4820 2022-08-01T19:34:37.636 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.622+0000 7f4b42ffd700 1 -- 172.21.15.17:0/2833609627 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b44005800 con 0x7f4b540b4820 2022-08-01T19:34:37.636 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.622+0000 7f4b5c25f700 1 -- 172.21.15.17:0/2833609627 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b5000b180 con 0x7f4b540b4820 2022-08-01T19:34:37.636 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.622+0000 7f4b5c25f700 1 -- 172.21.15.17:0/2833609627 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b541427e0 con 0x7f4b540b4820 2022-08-01T19:34:37.636 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.622+0000 7f4b5c25f700 1 -- 172.21.15.17:0/2833609627 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b541429a0 con 0x7f4b540b4820 2022-08-01T19:34:37.637 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.622+0000 7f4b42ffd700 1 -- 172.21.15.17:0/2833609627 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4b44010070 con 0x7f4b540b4820 2022-08-01T19:34:37.637 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.622+0000 7f4b42ffd700 1 --2- 172.21.15.17:0/2833609627 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4b34053a10 0x7f4b34055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:37.637 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.622+0000 7f4b42ffd700 1 -- 172.21.15.17:0/2833609627 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6517+0+0 (secure 0 0 0) 0x7f4b4406b840 con 0x7f4b540b4820 2022-08-01T19:34:37.637 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.622+0000 7f4b59ffb700 1 --2- 172.21.15.17:0/2833609627 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4b34053a10 0x7f4b34055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:37.637 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.626+0000 7f4b59ffb700 1 --2- 172.21.15.17:0/2833609627 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4b34053a10 0x7f4b34055ed0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f4b50018000 tx=0x7f4b50014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:37.638 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.630+0000 7f4b42ffd700 1 -- 172.21.15.17:0/2833609627 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4b440425a0 con 0x7f4b540b4820 2022-08-01T19:34:37.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:37.794+0000 7f4b5c25f700 1 -- 172.21.15.17:0/2833609627 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4b540574e0 con 0x7f4b540b4820 2022-08-01T19:34:38.183 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.178+0000 7f4b42ffd700 1 -- 172.21.15.17:0/2833609627 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4b44045bc0 con 0x7f4b540b4820 2022-08-01T19:34:38.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.186+0000 7f4b5c25f700 1 -- 172.21.15.17:0/2833609627 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4b34053a10 msgr2=0x7f4b34055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:38.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.186+0000 7f4b5c25f700 1 --2- 172.21.15.17:0/2833609627 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4b34053a10 0x7f4b34055ed0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f4b50018000 tx=0x7f4b50014000).stop 2022-08-01T19:34:38.188 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.186+0000 7f4b5c25f700 1 -- 172.21.15.17:0/2833609627 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b540b4820 msgr2=0x7f4b54145e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:38.188 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.186+0000 7f4b5c25f700 1 --2- 172.21.15.17:0/2833609627 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b540b4820 0x7f4b54145e40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f4b44001110 tx=0x7f4b44007aa0).stop 2022-08-01T19:34:38.188 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.186+0000 7f4b5c25f700 1 -- 172.21.15.17:0/2833609627 shutdown_connections 2022-08-01T19:34:38.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.186+0000 7f4b5c25f700 1 --2- 172.21.15.17:0/2833609627 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4b34053a10 0x7f4b34055ed0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:38.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.186+0000 7f4b5c25f700 1 --2- 172.21.15.17:0/2833609627 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4b54146ae0 0x7f4b541563f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:38.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.186+0000 7f4b5c25f700 1 --2- 172.21.15.17:0/2833609627 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b540b4820 0x7f4b54145e40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:38.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.186+0000 7f4b5c25f700 1 --2- 172.21.15.17:0/2833609627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b540b4310 0x7f4b540adbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:38.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.186+0000 7f4b5c25f700 1 -- 172.21.15.17:0/2833609627 >> 172.21.15.17:0/2833609627 conn(0x7f4b5405c620 msgr2=0x7f4b5413f580 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:38.190 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.186+0000 7f4b5c25f700 1 -- 172.21.15.17:0/2833609627 shutdown_connections 2022-08-01T19:34:38.190 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.186+0000 7f4b5c25f700 1 -- 172.21.15.17:0/2833609627 wait complete. 2022-08-01T19:34:38.203 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-08-01T19:34:38.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.314+0000 7ff6138cd700 1 Processor -- start 2022-08-01T19:34:38.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.314+0000 7ff6138cd700 1 -- start start 2022-08-01T19:34:38.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.314+0000 7ff6138cd700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ff60c0b3630 0x7ff60c143f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:38.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.314+0000 7ff6138cd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff60c0aa0d0 0x7ff60c0af200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:38.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.314+0000 7ff6138cd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff60c0af6f0 0x7ff60c05f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:38.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.314+0000 7ff6138cd700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff60c1443f0 con 0x7ff60c0aa0d0 2022-08-01T19:34:38.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.314+0000 7ff6138cd700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff60c0b3a10 con 0x7ff60c0b3630 2022-08-01T19:34:38.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.314+0000 7ff6138cd700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c0afb80 con 0x7ff60c0af6f0 2022-08-01T19:34:38.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.318+0000 7ff610e68700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff60c0aa0d0 0x7ff60c0af200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:38.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.318+0000 7ff610e68700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff60c0aa0d0 0x7ff60c0af200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37328/0 (socket says 172.21.15.17:37328) 2022-08-01T19:34:38.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.318+0000 7ff610e68700 1 -- 172.21.15.17:0/2100006458 learned_addr learned my addr 172.21.15.17:0/2100006458 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:38.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.318+0000 7ff611e6a700 1 --2- 172.21.15.17:0/2100006458 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff60c0af6f0 0x7ff60c05f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:38.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.318+0000 7ff611669700 1 --2- 172.21.15.17:0/2100006458 >> v2:172.21.15.77:3300/0 conn(0x7ff60c0b3630 0x7ff60c143f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:38.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.318+0000 7ff610e68700 1 -- 172.21.15.17:0/2100006458 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff60c0af6f0 msgr2=0x7ff60c05f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:38.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.318+0000 7ff610e68700 1 --2- 172.21.15.17:0/2100006458 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff60c0af6f0 0x7ff60c05f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:38.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.318+0000 7ff610e68700 1 -- 172.21.15.17:0/2100006458 >> v2:172.21.15.77:3300/0 conn(0x7ff60c0b3630 msgr2=0x7ff60c143f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:38.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.318+0000 7ff610e68700 1 --2- 172.21.15.17:0/2100006458 >> v2:172.21.15.77:3300/0 conn(0x7ff60c0b3630 0x7ff60c143f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:38.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.318+0000 7ff610e68700 1 -- 172.21.15.17:0/2100006458 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60c146b40 con 0x7ff60c0aa0d0 2022-08-01T19:34:38.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.318+0000 7ff611669700 1 --2- 172.21.15.17:0/2100006458 >> v2:172.21.15.77:3300/0 conn(0x7ff60c0b3630 0x7ff60c143f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:34:38.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.318+0000 7ff610e68700 1 --2- 172.21.15.17:0/2100006458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff60c0aa0d0 0x7ff60c0af200 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7ff608002f70 tx=0x7ff608006eb0).ready entity=mon.0 client_cookie=c59feef0f783bfde server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:38.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.318+0000 7ff603fff700 1 -- 172.21.15.17:0/2100006458 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff60800d570 con 0x7ff60c0aa0d0 2022-08-01T19:34:38.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.318+0000 7ff603fff700 1 -- 172.21.15.17:0/2100006458 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff608000e90 con 0x7ff60c0aa0d0 2022-08-01T19:34:38.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.318+0000 7ff603fff700 1 -- 172.21.15.17:0/2100006458 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff60800d900 con 0x7ff60c0aa0d0 2022-08-01T19:34:38.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.318+0000 7ff6138cd700 1 -- 172.21.15.17:0/2100006458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff60c0aa0d0 msgr2=0x7ff60c0af200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:38.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.318+0000 7ff6138cd700 1 --2- 172.21.15.17:0/2100006458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff60c0aa0d0 0x7ff60c0af200 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7ff608002f70 tx=0x7ff608006eb0).stop 2022-08-01T19:34:38.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.318+0000 7ff6138cd700 1 -- 172.21.15.17:0/2100006458 shutdown_connections 2022-08-01T19:34:38.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.318+0000 7ff6138cd700 1 --2- 172.21.15.17:0/2100006458 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff60c0af6f0 0x7ff60c05f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:38.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.318+0000 7ff6138cd700 1 --2- 172.21.15.17:0/2100006458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff60c0aa0d0 0x7ff60c0af200 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:38.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.318+0000 7ff6138cd700 1 --2- 172.21.15.17:0/2100006458 >> v2:172.21.15.77:3300/0 conn(0x7ff60c0b3630 0x7ff60c143f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:38.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.318+0000 7ff6138cd700 1 -- 172.21.15.17:0/2100006458 >> 172.21.15.17:0/2100006458 conn(0x7ff60c060950 msgr2=0x7ff60c060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:38.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.318+0000 7ff6138cd700 1 -- 172.21.15.17:0/2100006458 shutdown_connections 2022-08-01T19:34:38.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.318+0000 7ff6138cd700 1 -- 172.21.15.17:0/2100006458 wait complete. 2022-08-01T19:34:38.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.318+0000 7ff6138cd700 1 Processor -- start 2022-08-01T19:34:38.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.318+0000 7ff6138cd700 1 -- start start 2022-08-01T19:34:38.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.322+0000 7ff6138cd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff60c0aa0d0 0x7ff60c13f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:38.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.322+0000 7ff6138cd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff60c0af6f0 0x7ff60c149710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:38.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.322+0000 7ff6138cd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff60c0b3630 0x7ff60c13bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:38.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.322+0000 7ff6138cd700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c148bd0 con 0x7ff60c0aa0d0 2022-08-01T19:34:38.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.322+0000 7ff6138cd700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c146870 con 0x7ff60c0af6f0 2022-08-01T19:34:38.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.322+0000 7ff6138cd700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c0b5820 con 0x7ff60c0b3630 2022-08-01T19:34:38.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.322+0000 7ff611669700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff60c0aa0d0 0x7ff60c13f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:38.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.322+0000 7ff611669700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff60c0aa0d0 0x7ff60c13f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37332/0 (socket says 172.21.15.17:37332) 2022-08-01T19:34:38.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.322+0000 7ff611669700 1 -- 172.21.15.17:0/4240551530 learned_addr learned my addr 172.21.15.17:0/4240551530 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:38.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.322+0000 7ff611e6a700 1 --2- 172.21.15.17:0/4240551530 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff60c0b3630 0x7ff60c13bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:38.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.322+0000 7ff610e68700 1 --2- 172.21.15.17:0/4240551530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff60c0af6f0 0x7ff60c149710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:38.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.322+0000 7ff611669700 1 -- 172.21.15.17:0/4240551530 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff60c0b3630 msgr2=0x7ff60c13bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:38.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.322+0000 7ff611669700 1 --2- 172.21.15.17:0/4240551530 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff60c0b3630 0x7ff60c13bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:38.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.322+0000 7ff611669700 1 -- 172.21.15.17:0/4240551530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff60c0af6f0 msgr2=0x7ff60c149710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:38.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.322+0000 7ff611669700 1 --2- 172.21.15.17:0/4240551530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff60c0af6f0 0x7ff60c149710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:38.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.322+0000 7ff611669700 1 -- 172.21.15.17:0/4240551530 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60800b1e0 con 0x7ff60c0aa0d0 2022-08-01T19:34:38.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.322+0000 7ff611669700 1 --2- 172.21.15.17:0/4240551530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff60c0aa0d0 0x7ff60c13f3f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7ff60c1464f0 tx=0x7ff5fc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:38.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.322+0000 7ff6027fc700 1 -- 172.21.15.17:0/4240551530 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5fc011780 con 0x7ff60c0aa0d0 2022-08-01T19:34:38.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.322+0000 7ff6027fc700 1 -- 172.21.15.17:0/4240551530 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5fc0118e0 con 0x7ff60c0aa0d0 2022-08-01T19:34:38.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.322+0000 7ff6138cd700 1 -- 172.21.15.17:0/4240551530 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff60c1465d0 con 0x7ff60c0aa0d0 2022-08-01T19:34:38.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.322+0000 7ff6027fc700 1 -- 172.21.15.17:0/4240551530 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5fc011c40 con 0x7ff60c0aa0d0 2022-08-01T19:34:38.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.322+0000 7ff6138cd700 1 -- 172.21.15.17:0/4240551530 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff60c147e20 con 0x7ff60c0aa0d0 2022-08-01T19:34:38.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.322+0000 7ff6138cd700 1 -- 172.21.15.17:0/4240551530 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff60c146de0 con 0x7ff60c0aa0d0 2022-08-01T19:34:38.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.326+0000 7ff6027fc700 1 -- 172.21.15.17:0/4240551530 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff5fc01c460 con 0x7ff60c0aa0d0 2022-08-01T19:34:38.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.326+0000 7ff6027fc700 1 --2- 172.21.15.17:0/4240551530 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff5f4053a70 0x7ff5f4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:38.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.326+0000 7ff6027fc700 1 -- 172.21.15.17:0/4240551530 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6517+0+0 (secure 0 0 0) 0x7ff5fc06c1f0 con 0x7ff60c0aa0d0 2022-08-01T19:34:38.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.326+0000 7ff610e68700 1 --2- 172.21.15.17:0/4240551530 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff5f4053a70 0x7ff5f4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:38.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.326+0000 7ff610e68700 1 --2- 172.21.15.17:0/4240551530 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff5f4053a70 0x7ff5f4055f30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7ff60801c000 tx=0x7ff608015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:38.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.330+0000 7ff6027fc700 1 -- 172.21.15.17:0/4240551530 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff5fc042ef0 con 0x7ff60c0aa0d0 2022-08-01T19:34:38.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:38.466+0000 7ff6138cd700 1 -- 172.21.15.17:0/4240551530 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff60c0574e0 con 0x7ff60c0aa0d0 2022-08-01T19:34:39.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.186+0000 7ff6027fc700 1 -- 172.21.15.17:0/4240551530 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v27) v1 ==== 185+0+0 (secure 0 0 0) 0x7ff5fc046510 con 0x7ff60c0aa0d0 2022-08-01T19:34:39.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.194+0000 7ff6138cd700 1 -- 172.21.15.17:0/4240551530 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff5f4053a70 msgr2=0x7ff5f4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:39.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.194+0000 7ff6138cd700 1 --2- 172.21.15.17:0/4240551530 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff5f4053a70 0x7ff5f4055f30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7ff60801c000 tx=0x7ff608015000).stop 2022-08-01T19:34:39.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.194+0000 7ff6138cd700 1 -- 172.21.15.17:0/4240551530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff60c0aa0d0 msgr2=0x7ff60c13f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:39.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.194+0000 7ff6138cd700 1 --2- 172.21.15.17:0/4240551530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff60c0aa0d0 0x7ff60c13f3f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7ff60c1464f0 tx=0x7ff5fc008040).stop 2022-08-01T19:34:39.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.194+0000 7ff6138cd700 1 -- 172.21.15.17:0/4240551530 shutdown_connections 2022-08-01T19:34:39.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.194+0000 7ff6138cd700 1 --2- 172.21.15.17:0/4240551530 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff5f4053a70 0x7ff5f4055f30 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:39.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.194+0000 7ff6138cd700 1 --2- 172.21.15.17:0/4240551530 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff60c0b3630 0x7ff60c13bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:39.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.194+0000 7ff6138cd700 1 --2- 172.21.15.17:0/4240551530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff60c0af6f0 0x7ff60c149710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:39.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.194+0000 7ff6138cd700 1 --2- 172.21.15.17:0/4240551530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff60c0aa0d0 0x7ff60c13f3f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:39.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.194+0000 7ff6138cd700 1 -- 172.21.15.17:0/4240551530 >> 172.21.15.17:0/4240551530 conn(0x7ff60c060950 msgr2=0x7ff60c13e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:39.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.194+0000 7ff6138cd700 1 -- 172.21.15.17:0/4240551530 shutdown_connections 2022-08-01T19:34:39.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.194+0000 7ff6138cd700 1 -- 172.21.15.17:0/4240551530 wait complete. 2022-08-01T19:34:39.202 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' removed 2022-08-01T19:34:39.212 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-08-01T19:34:39.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.322+0000 7fa033fff700 1 Processor -- start 2022-08-01T19:34:39.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.322+0000 7fa033fff700 1 -- start start 2022-08-01T19:34:39.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa033fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa0340b6420 0x7fa034061600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:39.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa033fff700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0340b60e0 con 0x7fa0340b41f0 2022-08-01T19:34:39.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa033fff700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0340b2100 con 0x7fa034142ff0 2022-08-01T19:34:39.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa033fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0340b23a0 con 0x7fa0340b6420 2022-08-01T19:34:39.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa0327fc700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa0340b41f0 0x7fa034061af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57766/0 (socket says 172.21.15.17:57766) 2022-08-01T19:34:39.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa0327fc700 1 -- 172.21.15.17:0/1677931896 learned_addr learned my addr 172.21.15.17:0/1677931896 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:39.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa032ffd700 1 --2- 172.21.15.17:0/1677931896 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa0340b6420 0x7fa034061600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:39.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa031ffb700 1 -- 172.21.15.17:0/1677931896 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3440009576 0 0) 0x7fa024001110 con 0x7fa0340b41f0 2022-08-01T19:34:39.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa031ffb700 1 -- 172.21.15.17:0/1677931896 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa014003260 con 0x7fa0340b41f0 2022-08-01T19:34:39.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa031ffb700 1 -- 172.21.15.17:0/1677931896 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1665175389 0 0) 0x7fa014003260 con 0x7fa0340b41f0 2022-08-01T19:34:39.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa031ffb700 1 -- 172.21.15.17:0/1677931896 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa0340b6420 msgr2=0x7fa034061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:39.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa031ffb700 1 --2- 172.21.15.17:0/1677931896 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa0340b6420 0x7fa034061600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:39.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa031ffb700 1 -- 172.21.15.17:0/1677931896 >> v1:172.21.15.77:6789/0 conn(0x7fa034142ff0 legacy=0x7fa0341434a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:39.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa031ffb700 1 -- 172.21.15.17:0/1677931896 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0340b2640 con 0x7fa0340b41f0 2022-08-01T19:34:39.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa031ffb700 1 -- 172.21.15.17:0/1677931896 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7fa0240017f0 con 0x7fa0340b41f0 2022-08-01T19:34:39.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa031ffb700 1 -- 172.21.15.17:0/1677931896 >> v1:172.21.15.17:6789/0 conn(0x7fa0340b41f0 legacy=0x7fa034061af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:39.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa031ffb700 1 --2- 172.21.15.17:0/1677931896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa014003780 0x7fa014003b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:39.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa031ffb700 1 --2- 172.21.15.17:0/1677931896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa014004500 0x7fa014004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:39.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa031ffb700 1 --2- 172.21.15.17:0/1677931896 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa0340b6420 0x7fa014004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:39.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa031ffb700 1 -- 172.21.15.17:0/1677931896 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0340b23a0 con 0x7fa014004500 2022-08-01T19:34:39.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa031ffb700 1 -- 172.21.15.17:0/1677931896 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01400b750 con 0x7fa014003780 2022-08-01T19:34:39.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa031ffb700 1 -- 172.21.15.17:0/1677931896 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa01400b8c0 con 0x7fa0340b6420 2022-08-01T19:34:39.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa0327fc700 1 --2- 172.21.15.17:0/1677931896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa014004500 0x7fa014004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:39.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa032ffd700 1 --2- 172.21.15.17:0/1677931896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa014003780 0x7fa014003b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:39.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa0337fe700 1 --2- 172.21.15.17:0/1677931896 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa0340b6420 0x7fa014004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:39.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa0337fe700 1 -- 172.21.15.17:0/1677931896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa014003780 msgr2=0x7fa014003b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:39.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa0337fe700 1 --2- 172.21.15.17:0/1677931896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa014003780 0x7fa014003b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:39.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa0337fe700 1 -- 172.21.15.17:0/1677931896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa014004500 msgr2=0x7fa014004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:39.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa0337fe700 1 --2- 172.21.15.17:0/1677931896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa014004500 0x7fa014004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:39.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa0337fe700 1 -- 172.21.15.17:0/1677931896 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa014003260 con 0x7fa0340b6420 2022-08-01T19:34:39.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa0327fc700 1 --2- 172.21.15.17:0/1677931896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa014004500 0x7fa014004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:34:39.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa0337fe700 1 --2- 172.21.15.17:0/1677931896 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa0340b6420 0x7fa014004e70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fa02c0013a0 tx=0x7fa02c007d60).ready entity=mon.2 client_cookie=99c2d56d258b236b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:39.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa031ffb700 1 -- 172.21.15.17:0/1677931896 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa02c0016c0 con 0x7fa0340b6420 2022-08-01T19:34:39.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa031ffb700 1 -- 172.21.15.17:0/1677931896 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa02c006890 con 0x7fa0340b6420 2022-08-01T19:34:39.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa031ffb700 1 -- 172.21.15.17:0/1677931896 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa02c006bf0 con 0x7fa0340b6420 2022-08-01T19:34:39.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa033fff700 1 -- 172.21.15.17:0/1677931896 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa0340b6420 msgr2=0x7fa014004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:39.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa033fff700 1 --2- 172.21.15.17:0/1677931896 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa0340b6420 0x7fa014004e70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fa02c0013a0 tx=0x7fa02c007d60).stop 2022-08-01T19:34:39.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa0337fe700 1 -- 172.21.15.17:0/1677931896 reap_dead start 2022-08-01T19:34:39.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa033fff700 1 -- 172.21.15.17:0/1677931896 shutdown_connections 2022-08-01T19:34:39.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa033fff700 1 -- 172.21.15.17:0/1677931896 >> 172.21.15.17:0/1677931896 conn(0x7fa03405c620 msgr2=0x7fa03405ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:39.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa033fff700 1 -- 172.21.15.17:0/1677931896 shutdown_connections 2022-08-01T19:34:39.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa033fff700 1 -- 172.21.15.17:0/1677931896 wait complete. 2022-08-01T19:34:39.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa033fff700 1 Processor -- start 2022-08-01T19:34:39.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.330+0000 7fa033fff700 1 -- start start 2022-08-01T19:34:39.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.334+0000 7fa033fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa0340b6420 0x7fa0340b7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:39.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.334+0000 7fa033fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0340ae8b0 0x7fa034152710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:39.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.334+0000 7fa033fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0340ad740 0x7fa034152c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:39.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.334+0000 7fa033fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0340adc50 con 0x7fa0340ad740 2022-08-01T19:34:39.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.334+0000 7fa033fff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa034156800 con 0x7fa0340ae8b0 2022-08-01T19:34:39.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.334+0000 7fa033fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa034156aa0 con 0x7fa0340b6420 2022-08-01T19:34:39.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.334+0000 7fa0337fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0340ad740 0x7fa034152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:39.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.334+0000 7fa0337fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0340ad740 0x7fa034152c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37354/0 (socket says 172.21.15.17:37354) 2022-08-01T19:34:39.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.334+0000 7fa0337fe700 1 -- 172.21.15.17:0/3359358397 learned_addr learned my addr 172.21.15.17:0/3359358397 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:39.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.334+0000 7fa0327fc700 1 --2- 172.21.15.17:0/3359358397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0340ae8b0 0x7fa034152710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:39.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.334+0000 7fa032ffd700 1 --2- 172.21.15.17:0/3359358397 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa0340b6420 0x7fa0340b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:39.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.334+0000 7fa0337fe700 1 -- 172.21.15.17:0/3359358397 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa0340b6420 msgr2=0x7fa0340b7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:39.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.334+0000 7fa0337fe700 1 --2- 172.21.15.17:0/3359358397 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa0340b6420 0x7fa0340b7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:39.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.334+0000 7fa0337fe700 1 -- 172.21.15.17:0/3359358397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0340ae8b0 msgr2=0x7fa034152710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:39.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.334+0000 7fa0337fe700 1 --2- 172.21.15.17:0/3359358397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0340ae8b0 0x7fa034152710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:39.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.334+0000 7fa0337fe700 1 -- 172.21.15.17:0/3359358397 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0340adeb0 con 0x7fa0340ad740 2022-08-01T19:34:39.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.334+0000 7fa0337fe700 1 --2- 172.21.15.17:0/3359358397 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0340ad740 0x7fa034152c00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7fa0340b1fe0 tx=0x7fa02c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:39.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.334+0000 7fa02bfff700 1 -- 172.21.15.17:0/3359358397 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa02c0016c0 con 0x7fa0340ad740 2022-08-01T19:34:39.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.334+0000 7fa02bfff700 1 -- 172.21.15.17:0/3359358397 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa02c00c930 con 0x7fa0340ad740 2022-08-01T19:34:39.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.334+0000 7fa033fff700 1 -- 172.21.15.17:0/3359358397 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0340ae350 con 0x7fa0340ad740 2022-08-01T19:34:39.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.334+0000 7fa02bfff700 1 -- 172.21.15.17:0/3359358397 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa02c00cc90 con 0x7fa0340ad740 2022-08-01T19:34:39.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.334+0000 7fa033fff700 1 -- 172.21.15.17:0/3359358397 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa034140870 con 0x7fa0340ad740 2022-08-01T19:34:39.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.338+0000 7fa033fff700 1 -- 172.21.15.17:0/3359358397 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0340ae040 con 0x7fa0340ad740 2022-08-01T19:34:39.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.338+0000 7fa02bfff700 1 -- 172.21.15.17:0/3359358397 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa02c01d070 con 0x7fa0340ad740 2022-08-01T19:34:39.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.338+0000 7fa02bfff700 1 --2- 172.21.15.17:0/3359358397 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa018053980 0x7fa018055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:39.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.338+0000 7fa02bfff700 1 -- 172.21.15.17:0/3359358397 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6079+0+0 (secure 0 0 0) 0x7fa02c06cb70 con 0x7fa0340ad740 2022-08-01T19:34:39.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.338+0000 7fa032ffd700 1 --2- 172.21.15.17:0/3359358397 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa018053980 0x7fa018055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:39.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.338+0000 7fa032ffd700 1 --2- 172.21.15.17:0/3359358397 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa018053980 0x7fa018055e40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fa01c00b000 tx=0x7fa01c006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:39.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.350+0000 7fa02bfff700 1 -- 172.21.15.17:0/3359358397 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa02c043a30 con 0x7fa0340ad740 2022-08-01T19:34:39.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:39.474+0000 7fa033fff700 1 -- 172.21.15.17:0/3359358397 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa0340b28e0 con 0x7fa0340ad740 2022-08-01T19:34:40.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.198+0000 7fa02bfff700 1 -- 172.21.15.17:0/3359358397 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v28) v1 ==== 173+0+0 (secure 0 0 0) 0x7fa02c01a030 con 0x7fa0340ad740 2022-08-01T19:34:40.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.202+0000 7fa033fff700 1 -- 172.21.15.17:0/3359358397 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa018053980 msgr2=0x7fa018055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:40.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.202+0000 7fa033fff700 1 --2- 172.21.15.17:0/3359358397 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa018053980 0x7fa018055e40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fa01c00b000 tx=0x7fa01c006df0).stop 2022-08-01T19:34:40.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.202+0000 7fa033fff700 1 -- 172.21.15.17:0/3359358397 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0340ad740 msgr2=0x7fa034152c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:40.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.202+0000 7fa033fff700 1 --2- 172.21.15.17:0/3359358397 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0340ad740 0x7fa034152c00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7fa0340b1fe0 tx=0x7fa02c015000).stop 2022-08-01T19:34:40.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.202+0000 7fa033fff700 1 -- 172.21.15.17:0/3359358397 shutdown_connections 2022-08-01T19:34:40.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.202+0000 7fa033fff700 1 --2- 172.21.15.17:0/3359358397 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa018053980 0x7fa018055e40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:40.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.202+0000 7fa033fff700 1 --2- 172.21.15.17:0/3359358397 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0340ad740 0x7fa034152c00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:40.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.202+0000 7fa033fff700 1 --2- 172.21.15.17:0/3359358397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0340ae8b0 0x7fa034152710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:40.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.202+0000 7fa033fff700 1 --2- 172.21.15.17:0/3359358397 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa0340b6420 0x7fa0340b7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:40.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.202+0000 7fa033fff700 1 -- 172.21.15.17:0/3359358397 >> 172.21.15.17:0/3359358397 conn(0x7fa03405c620 msgr2=0x7fa0340b84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:40.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.202+0000 7fa033fff700 1 -- 172.21.15.17:0/3359358397 shutdown_connections 2022-08-01T19:34:40.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.202+0000 7fa033fff700 1 -- 172.21.15.17:0/3359358397 wait complete. 2022-08-01T19:34:40.212 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' removed 2022-08-01T19:34:40.223 INFO:tasks.ceph.mds.a:Restarting daemon 2022-08-01T19:34:40.223 INFO:tasks.ceph.mds.a:Stopping old one... 2022-08-01T19:34:40.224 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-08-01T19:34:40.224 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:34:40.224 INFO:tasks.ceph.mds.c:Restarting daemon 2022-08-01T19:34:40.225 INFO:tasks.ceph.mds.c:Stopping old one... 2022-08-01T19:34:40.225 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-08-01T19:34:40.225 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:34:40.226 INFO:tasks.ceph.mds.b:Restarting daemon 2022-08-01T19:34:40.226 INFO:tasks.ceph.mds.b:Stopping old one... 2022-08-01T19:34:40.226 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-08-01T19:34:40.226 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:34:40.237 INFO:tasks.ceph.mds.c:Stopped 2022-08-01T19:34:40.237 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-08-01T19:34:40.238 INFO:tasks.ceph.mds.a:Stopped 2022-08-01T19:34:40.239 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-08-01T19:34:40.245 INFO:tasks.ceph.mds.b:Stopped 2022-08-01T19:34:40.246 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-08-01T19:34:40.247 INFO:tasks.ceph.mds.c:Started 2022-08-01T19:34:40.247 INFO:tasks.ceph.mds.a:Started 2022-08-01T19:34:40.248 DEBUG:teuthology.parallel:result is None 2022-08-01T19:34:40.248 DEBUG:teuthology.parallel:result is None 2022-08-01T19:34:40.251 INFO:tasks.ceph.mds.b:Started 2022-08-01T19:34:40.251 DEBUG:teuthology.parallel:result is None 2022-08-01T19:34:40.252 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-08-01T19:34:40.316 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2022-08-01T19:34:40.322 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2022-08-01T19:34:40.328 INFO:tasks.ceph.mds.b.smithi077.stdout:starting mds.b at 2022-08-01T19:34:40.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.350+0000 7fb5d49ba700 1 Processor -- start 2022-08-01T19:34:40.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.350+0000 7fb5d49ba700 1 -- start start 2022-08-01T19:34:40.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.350+0000 7fb5d49ba700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb5d00b56b0 0x7fb5d00ac880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:40.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.350+0000 7fb5d49ba700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fb5d00b0540 0x7fb5d0140260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:40.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.350+0000 7fb5d49ba700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d00b5a90 con 0x7fb5d00b1560 2022-08-01T19:34:40.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.350+0000 7fb5d49ba700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5d00aed20 con 0x7fb5d00b0540 2022-08-01T19:34:40.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.350+0000 7fb5d49ba700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d00aff00 con 0x7fb5d00b56b0 2022-08-01T19:34:40.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.350+0000 7fb5cffff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fb5d00b0540 0x7fb5d0140260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:40.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.350+0000 7fb5cf7fe700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb5d00b56b0 0x7fb5d00ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:40.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.350+0000 7fb5cffff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fb5d00b0540 0x7fb5d0140260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:35310/0 (socket says 172.21.15.17:35310) 2022-08-01T19:34:40.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.350+0000 7fb5cffff700 1 -- 172.21.15.17:0/2248548160 learned_addr learned my addr 172.21.15.17:0/2248548160 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:40.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.350+0000 7fb5cffff700 1 -- 172.21.15.17:0/2248548160 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb5d00b56b0 msgr2=0x7fb5d00ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:40.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.350+0000 7fb5cffff700 1 --2- 172.21.15.17:0/2248548160 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb5d00b56b0 0x7fb5d00ac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:40.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.350+0000 7fb5cffff700 1 -- 172.21.15.17:0/2248548160 >> v1:172.21.15.77:6789/0 conn(0x7fb5d00b1560 legacy=0x7fb5d00aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:40.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.350+0000 7fb5cffff700 1 -- 172.21.15.17:0/2248548160 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d00acf50 con 0x7fb5d00b0540 2022-08-01T19:34:40.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.350+0000 7fb5ce7fc700 1 -- 172.21.15.17:0/2248548160 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 316864712 0 0) 0x7fb5d00b5a90 con 0x7fb5d00b1560 2022-08-01T19:34:40.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.350+0000 7fb5cffff700 1 --2- 172.21.15.17:0/2248548160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5d00b0540 0x7fb5d0140260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fb5c8002f70 tx=0x7fb5c8006eb0).ready entity=mon.1 client_cookie=ee41b1f97d13e526 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:40.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.350+0000 7fb5ce7fc700 1 -- 172.21.15.17:0/2248548160 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5c800d570 con 0x7fb5d00b0540 2022-08-01T19:34:40.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.350+0000 7fb5ce7fc700 1 -- 172.21.15.17:0/2248548160 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c8000e90 con 0x7fb5d00b0540 2022-08-01T19:34:40.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.350+0000 7fb5ce7fc700 1 -- 172.21.15.17:0/2248548160 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5c800d8c0 con 0x7fb5d00b0540 2022-08-01T19:34:40.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.354+0000 7fb5d49ba700 1 -- 172.21.15.17:0/2248548160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5d00b0540 msgr2=0x7fb5d0140260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:40.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.354+0000 7fb5d49ba700 1 --2- 172.21.15.17:0/2248548160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5d00b0540 0x7fb5d0140260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fb5c8002f70 tx=0x7fb5c8006eb0).stop 2022-08-01T19:34:40.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.354+0000 7fb5d49ba700 1 -- 172.21.15.17:0/2248548160 shutdown_connections 2022-08-01T19:34:40.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.354+0000 7fb5d49ba700 1 --2- 172.21.15.17:0/2248548160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5d00b0540 0x7fb5d0140260 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:40.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.354+0000 7fb5d49ba700 1 --2- 172.21.15.17:0/2248548160 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb5d00b56b0 0x7fb5d00ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:40.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.354+0000 7fb5d49ba700 1 -- 172.21.15.17:0/2248548160 >> 172.21.15.17:0/2248548160 conn(0x7fb5d005c620 msgr2=0x7fb5d00626f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:40.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.354+0000 7fb5d49ba700 1 -- 172.21.15.17:0/2248548160 shutdown_connections 2022-08-01T19:34:40.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.354+0000 7fb5d49ba700 1 -- 172.21.15.17:0/2248548160 wait complete. 2022-08-01T19:34:40.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.354+0000 7fb5d49ba700 1 Processor -- start 2022-08-01T19:34:40.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.354+0000 7fb5d49ba700 1 -- start start 2022-08-01T19:34:40.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.354+0000 7fb5d49ba700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5d00b0540 0x7fb5d014a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:40.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.354+0000 7fb5d49ba700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb5d00b1560 0x7fb5d0139950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:40.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.354+0000 7fb5d49ba700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5d00b56b0 0x7fb5d0137d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:40.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.354+0000 7fb5d49ba700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d0138ba0 con 0x7fb5d00b56b0 2022-08-01T19:34:40.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.354+0000 7fb5d49ba700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d0138e40 con 0x7fb5d00b0540 2022-08-01T19:34:40.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.354+0000 7fb5d49ba700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d01394b0 con 0x7fb5d00b1560 2022-08-01T19:34:40.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.358+0000 7fb5cffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5d00b56b0 0x7fb5d0137d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:40.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.358+0000 7fb5ceffd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb5d00b1560 0x7fb5d0139950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:40.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.358+0000 7fb5cf7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5d00b0540 0x7fb5d014a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:40.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.358+0000 7fb5cffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5d00b56b0 0x7fb5d0137d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37394/0 (socket says 172.21.15.17:37394) 2022-08-01T19:34:40.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.358+0000 7fb5cffff700 1 -- 172.21.15.17:0/2847309473 learned_addr learned my addr 172.21.15.17:0/2847309473 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:40.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.358+0000 7fb5cffff700 1 -- 172.21.15.17:0/2847309473 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb5d00b1560 msgr2=0x7fb5d0139950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:40.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.358+0000 7fb5cffff700 1 --2- 172.21.15.17:0/2847309473 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb5d00b1560 0x7fb5d0139950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:40.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.358+0000 7fb5cffff700 1 -- 172.21.15.17:0/2847309473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5d00b0540 msgr2=0x7fb5d014a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:40.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.358+0000 7fb5cffff700 1 --2- 172.21.15.17:0/2847309473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5d00b0540 0x7fb5d014a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:40.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.358+0000 7fb5cffff700 1 -- 172.21.15.17:0/2847309473 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c800b1e0 con 0x7fb5d00b56b0 2022-08-01T19:34:40.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.358+0000 7fb5cf7fe700 1 --2- 172.21.15.17:0/2847309473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5d00b0540 0x7fb5d014a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:34:40.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.358+0000 7fb5ceffd700 1 --2- 172.21.15.17:0/2847309473 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb5d00b1560 0x7fb5d0139950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:34:40.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.358+0000 7fb5cffff700 1 --2- 172.21.15.17:0/2847309473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5d00b56b0 0x7fb5d0137d20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7fb5c8002f70 tx=0x7fb5c8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:40.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.358+0000 7fb5ccff9700 1 -- 172.21.15.17:0/2847309473 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5c8011760 con 0x7fb5d00b56b0 2022-08-01T19:34:40.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.358+0000 7fb5ccff9700 1 -- 172.21.15.17:0/2847309473 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c80118c0 con 0x7fb5d00b56b0 2022-08-01T19:34:40.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.358+0000 7fb5d49ba700 1 -- 172.21.15.17:0/2847309473 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d0139e40 con 0x7fb5d00b56b0 2022-08-01T19:34:40.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.358+0000 7fb5ccff9700 1 -- 172.21.15.17:0/2847309473 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5c8011c20 con 0x7fb5d00b56b0 2022-08-01T19:34:40.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.358+0000 7fb5d49ba700 1 -- 172.21.15.17:0/2847309473 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d0138210 con 0x7fb5d00b56b0 2022-08-01T19:34:40.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.358+0000 7fb5ccff9700 1 -- 172.21.15.17:0/2847309473 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb5c8025030 con 0x7fb5d00b56b0 2022-08-01T19:34:40.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.358+0000 7fb5d49ba700 1 -- 172.21.15.17:0/2847309473 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5d0139040 con 0x7fb5d00b56b0 2022-08-01T19:34:40.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.358+0000 7fb5ccff9700 1 --2- 172.21.15.17:0/2847309473 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb5b4053b00 0x7fb5b4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:40.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.358+0000 7fb5ccff9700 1 -- 172.21.15.17:0/2847309473 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fb5c806cfc0 con 0x7fb5d00b56b0 2022-08-01T19:34:40.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.358+0000 7fb5cf7fe700 1 --2- 172.21.15.17:0/2847309473 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb5b4053b00 0x7fb5b4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:40.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.362+0000 7fb5cf7fe700 1 --2- 172.21.15.17:0/2847309473 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb5b4053b00 0x7fb5b4055fc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7fb5c000b000 tx=0x7fb5c00091d0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:40.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.366+0000 7fb5ccff9700 1 -- 172.21.15.17:0/2847309473 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb5c8043fc0 con 0x7fb5d00b56b0 2022-08-01T19:34:40.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:40.502+0000 7fb5d49ba700 1 -- 172.21.15.17:0/2847309473 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fb5d0139670 con 0x7fb5d00b56b0 2022-08-01T19:34:41.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.338+0000 7fb5ccff9700 1 -- 172.21.15.17:0/2847309473 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v30) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb5c8023020 con 0x7fb5d00b56b0 2022-08-01T19:34:41.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.342+0000 7fb5d49ba700 1 -- 172.21.15.17:0/2847309473 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb5b4053b00 msgr2=0x7fb5b4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:41.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.342+0000 7fb5d49ba700 1 --2- 172.21.15.17:0/2847309473 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb5b4053b00 0x7fb5b4055fc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7fb5c000b000 tx=0x7fb5c00091d0).stop 2022-08-01T19:34:41.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.342+0000 7fb5d49ba700 1 -- 172.21.15.17:0/2847309473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5d00b56b0 msgr2=0x7fb5d0137d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:41.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.342+0000 7fb5d49ba700 1 --2- 172.21.15.17:0/2847309473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5d00b56b0 0x7fb5d0137d20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7fb5c8002f70 tx=0x7fb5c8015000).stop 2022-08-01T19:34:41.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.342+0000 7fb5d49ba700 1 -- 172.21.15.17:0/2847309473 shutdown_connections 2022-08-01T19:34:41.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.342+0000 7fb5d49ba700 1 --2- 172.21.15.17:0/2847309473 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb5b4053b00 0x7fb5b4055fc0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:41.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.342+0000 7fb5d49ba700 1 --2- 172.21.15.17:0/2847309473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5d00b56b0 0x7fb5d0137d20 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:41.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.342+0000 7fb5d49ba700 1 --2- 172.21.15.17:0/2847309473 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb5d00b1560 0x7fb5d0139950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:41.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.342+0000 7fb5d49ba700 1 --2- 172.21.15.17:0/2847309473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5d00b0540 0x7fb5d014a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:41.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.342+0000 7fb5d49ba700 1 -- 172.21.15.17:0/2847309473 >> 172.21.15.17:0/2847309473 conn(0x7fb5d005c620 msgr2=0x7fb5d01429d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:41.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.342+0000 7fb5d49ba700 1 -- 172.21.15.17:0/2847309473 shutdown_connections 2022-08-01T19:34:41.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.342+0000 7fb5d49ba700 1 -- 172.21.15.17:0/2847309473 wait complete. 2022-08-01T19:34:41.350 INFO:teuthology.orchestra.run.smithi017.stderr: removed all blacklist entries 2022-08-01T19:34:41.362 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-08-01T19:34:41.362 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-08-01T19:34:41.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.466+0000 7fec6b5ef700 1 Processor -- start 2022-08-01T19:34:41.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.466+0000 7fec6b5ef700 1 -- start start 2022-08-01T19:34:41.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.466+0000 7fec6b5ef700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fec640b6a90 0x7fec640acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:41.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.466+0000 7fec6b5ef700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fec640aa760 0x7fec6413c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:41.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.466+0000 7fec6b5ef700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec640b6e70 con 0x7fec64060aa0 2022-08-01T19:34:41.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.466+0000 7fec6b5ef700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec64061370 con 0x7fec640b6a90 2022-08-01T19:34:41.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.466+0000 7fec6b5ef700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec64061610 con 0x7fec640aa760 2022-08-01T19:34:41.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.466+0000 7fec6938b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fec640b6a90 0x7fec640acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:41.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.466+0000 7fec6938b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fec640b6a90 0x7fec640acee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37402/0 (socket says 172.21.15.17:37402) 2022-08-01T19:34:41.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.466+0000 7fec6938b700 1 -- 172.21.15.17:0/3101638275 learned_addr learned my addr 172.21.15.17:0/3101638275 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:41.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.466+0000 7fec69b8c700 1 --2- 172.21.15.17:0/3101638275 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fec640aa760 0x7fec6413c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:41.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.466+0000 7fec53fff700 1 -- 172.21.15.17:0/3101638275 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 717855387 0 0) 0x7fec640b6e70 con 0x7fec64060aa0 2022-08-01T19:34:41.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.466+0000 7fec53fff700 1 -- 172.21.15.17:0/3101638275 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec4c003260 con 0x7fec64060aa0 2022-08-01T19:34:41.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.466+0000 7fec6938b700 1 -- 172.21.15.17:0/3101638275 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fec640aa760 msgr2=0x7fec6413c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:41.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.466+0000 7fec6938b700 1 --2- 172.21.15.17:0/3101638275 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fec640aa760 0x7fec6413c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:41.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.466+0000 7fec6938b700 1 -- 172.21.15.17:0/3101638275 >> v1:172.21.15.77:6789/0 conn(0x7fec64060aa0 legacy=0x7fec640aa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:41.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.466+0000 7fec6938b700 1 -- 172.21.15.17:0/3101638275 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec640618b0 con 0x7fec640b6a90 2022-08-01T19:34:41.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.466+0000 7fec69b8c700 1 --2- 172.21.15.17:0/3101638275 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fec640aa760 0x7fec6413c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:34:41.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.466+0000 7fec6938b700 1 --2- 172.21.15.17:0/3101638275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec640b6a90 0x7fec640acee0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7fec60002f70 tx=0x7fec60006eb0).ready entity=mon.0 client_cookie=18bf4c2e0b47ea55 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:41.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.466+0000 7fec53fff700 1 -- 172.21.15.17:0/3101638275 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec6000d5c0 con 0x7fec640b6a90 2022-08-01T19:34:41.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.466+0000 7fec53fff700 1 -- 172.21.15.17:0/3101638275 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec60000e90 con 0x7fec640b6a90 2022-08-01T19:34:41.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.466+0000 7fec53fff700 1 -- 172.21.15.17:0/3101638275 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec6000d950 con 0x7fec640b6a90 2022-08-01T19:34:41.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.466+0000 7fec6b5ef700 1 -- 172.21.15.17:0/3101638275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec640b6a90 msgr2=0x7fec640acee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:41.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.466+0000 7fec6b5ef700 1 --2- 172.21.15.17:0/3101638275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec640b6a90 0x7fec640acee0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7fec60002f70 tx=0x7fec60006eb0).stop 2022-08-01T19:34:41.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.466+0000 7fec6b5ef700 1 -- 172.21.15.17:0/3101638275 shutdown_connections 2022-08-01T19:34:41.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.466+0000 7fec6b5ef700 1 --2- 172.21.15.17:0/3101638275 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fec640aa760 0x7fec6413c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:41.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.466+0000 7fec6b5ef700 1 --2- 172.21.15.17:0/3101638275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec640b6a90 0x7fec640acee0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:41.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.466+0000 7fec6b5ef700 1 -- 172.21.15.17:0/3101638275 >> 172.21.15.17:0/3101638275 conn(0x7fec640b3610 msgr2=0x7fec640b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:41.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.466+0000 7fec6b5ef700 1 -- 172.21.15.17:0/3101638275 shutdown_connections 2022-08-01T19:34:41.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.466+0000 7fec6b5ef700 1 -- 172.21.15.17:0/3101638275 wait complete. 2022-08-01T19:34:41.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.470+0000 7fec6b5ef700 1 Processor -- start 2022-08-01T19:34:41.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.470+0000 7fec6b5ef700 1 -- start start 2022-08-01T19:34:41.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.470+0000 7fec6b5ef700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fec64060aa0 0x7fec64143710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:41.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.470+0000 7fec6b5ef700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec640aa760 0x7fec641411e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:41.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.470+0000 7fec6b5ef700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec640b6a90 0x7fec641416d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:41.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.470+0000 7fec6b5ef700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6413d710 con 0x7fec640aa760 2022-08-01T19:34:41.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.470+0000 7fec6b5ef700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec640b02a0 con 0x7fec640b6a90 2022-08-01T19:34:41.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.470+0000 7fec6b5ef700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec64140c50 con 0x7fec64060aa0 2022-08-01T19:34:41.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.470+0000 7fec68b8a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec640aa760 0x7fec641411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:41.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.470+0000 7fec68b8a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec640aa760 0x7fec641411e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37408/0 (socket says 172.21.15.17:37408) 2022-08-01T19:34:41.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.470+0000 7fec68b8a700 1 -- 172.21.15.17:0/1770782413 learned_addr learned my addr 172.21.15.17:0/1770782413 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:41.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.470+0000 7fec69b8c700 1 --2- 172.21.15.17:0/1770782413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec640b6a90 0x7fec641416d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:41.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.470+0000 7fec68b8a700 1 -- 172.21.15.17:0/1770782413 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fec64060aa0 msgr2=0x7fec64143710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:41.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.470+0000 7fec68b8a700 1 --2- 172.21.15.17:0/1770782413 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fec64060aa0 0x7fec64143710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-08-01T19:34:41.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.470+0000 7fec68b8a700 1 -- 172.21.15.17:0/1770782413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec640b6a90 msgr2=0x7fec641416d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:41.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.470+0000 7fec68b8a700 1 --2- 172.21.15.17:0/1770782413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec640b6a90 0x7fec641416d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:41.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.470+0000 7fec68b8a700 1 -- 172.21.15.17:0/1770782413 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec5c009470 con 0x7fec640aa760 2022-08-01T19:34:41.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.470+0000 7fec68b8a700 1 --2- 172.21.15.17:0/1770782413 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec640aa760 0x7fec641411e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7fec54001110 tx=0x7fec54007ab0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:41.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.470+0000 7fec527fc700 1 -- 172.21.15.17:0/1770782413 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec5400ce00 con 0x7fec640aa760 2022-08-01T19:34:41.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.470+0000 7fec527fc700 1 -- 172.21.15.17:0/1770782413 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5400f470 con 0x7fec640aa760 2022-08-01T19:34:41.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.470+0000 7fec6b5ef700 1 -- 172.21.15.17:0/1770782413 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec6000b1e0 con 0x7fec640aa760 2022-08-01T19:34:41.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.470+0000 7fec527fc700 1 -- 172.21.15.17:0/1770782413 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec5400f7d0 con 0x7fec640aa760 2022-08-01T19:34:41.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.470+0000 7fec6b5ef700 1 -- 172.21.15.17:0/1770782413 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec640b1f20 con 0x7fec640aa760 2022-08-01T19:34:41.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.474+0000 7fec6b5ef700 1 -- 172.21.15.17:0/1770782413 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec640b20e0 con 0x7fec640aa760 2022-08-01T19:34:41.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.474+0000 7fec527fc700 1 -- 172.21.15.17:0/1770782413 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fec54019030 con 0x7fec640aa760 2022-08-01T19:34:41.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.474+0000 7fec527fc700 1 --2- 172.21.15.17:0/1770782413 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fec44053b30 0x7fec44055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:41.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.474+0000 7fec527fc700 1 -- 172.21.15.17:0/1770782413 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fec5406b350 con 0x7fec640aa760 2022-08-01T19:34:41.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.474+0000 7fec6938b700 1 --2- 172.21.15.17:0/1770782413 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fec44053b30 0x7fec44055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:41.499 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.474+0000 7fec6938b700 1 --2- 172.21.15.17:0/1770782413 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fec44053b30 0x7fec44055ff0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7fec60002f70 tx=0x7fec60015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:41.499 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.486+0000 7fec527fc700 1 -- 172.21.15.17:0/1770782413 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fec54042340 con 0x7fec640aa760 2022-08-01T19:34:41.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.650+0000 7fec6b5ef700 1 -- 172.21.15.17:0/1770782413 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fec64061b50 con 0x7fec640aa760 2022-08-01T19:34:41.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.650+0000 7fec527fc700 1 -- 172.21.15.17:0/1770782413 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v8) v1 ==== 80+0+5621 (secure 0 0 0) 0x7fec54045960 con 0x7fec640aa760 2022-08-01T19:34:41.656 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:41.656 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:34:41.657 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_dump": [ 2022-08-01T19:34:41.657 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:41.657 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.a", 2022-08-01T19:34:41.657 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhijotxORAAVc04rsrlFj0NShsyc6Qbhg==", 2022-08-01T19:34:41.658 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:34:41.658 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:34:41.658 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:34:41.658 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:34:41.659 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:34:41.659 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:41.659 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:41.659 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:41.660 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.b", 2022-08-01T19:34:41.660 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhiud2xBxAAcn6/O5ysdIp/oIbF5Y1dAw==", 2022-08-01T19:34:41.660 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:34:41.660 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:34:41.660 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:34:41.661 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:34:41.661 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:34:41.661 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:41.661 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:41.662 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:41.662 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.c", 2022-08-01T19:34:41.662 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+KuhiMzMbBBAAfdT5mtpYwboNL+7f2B09KQ==", 2022-08-01T19:34:41.663 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:34:41.663 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:34:41.663 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:34:41.663 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:34:41.664 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:34:41.664 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:41.664 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:41.664 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:41.664 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.0", 2022-08-01T19:34:41.665 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhia27/NRAAZXBp+nYelJVJ6Mge9r+3Vg==", 2022-08-01T19:34:41.665 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:34:41.665 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:34:41.665 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:34:41.666 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:34:41.666 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:41.666 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:41.666 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:41.667 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.1", 2022-08-01T19:34:41.667 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCBKuhiOggRGRAAvgA0MxwBJwmpXV6S9DTdjA==", 2022-08-01T19:34:41.667 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:34:41.667 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:34:41.668 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:34:41.668 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:34:41.669 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:41.669 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:41.670 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:41.670 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.2", 2022-08-01T19:34:41.670 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCDKuhi5PgOORAAhcORK4bW2T3dXPeYjcCm+g==", 2022-08-01T19:34:41.671 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:34:41.671 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:34:41.672 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:34:41.672 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:34:41.673 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:41.673 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:41.674 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:41.674 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.3", 2022-08-01T19:34:41.675 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCGKuhiutn6GxAAk2ucLuGbT5YV/R4xXQGh0w==", 2022-08-01T19:34:41.675 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:34:41.675 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:34:41.676 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:34:41.676 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:34:41.677 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:41.677 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:41.678 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:41.678 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.4", 2022-08-01T19:34:41.678 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCLKuhiGHTRHxAArr3TwXA0gsCAAF/Qs38wUQ==", 2022-08-01T19:34:41.679 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:34:41.679 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:34:41.679 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:34:41.679 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:34:41.679 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:41.680 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:41.680 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:41.680 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.5", 2022-08-01T19:34:41.680 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCOKuhiMMDbBBAAvTzBvOcv7pFWm/P/lNqB0w==", 2022-08-01T19:34:41.681 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:34:41.681 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:34:41.681 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:34:41.681 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:34:41.682 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:41.682 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:41.682 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:41.682 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.6", 2022-08-01T19:34:41.683 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCQKuhiG2H3IxAAufUcWqo9ULquxIrUD5vSfA==", 2022-08-01T19:34:41.683 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:34:41.683 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:34:41.683 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:34:41.684 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:34:41.684 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:41.684 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:41.684 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:41.685 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.7", 2022-08-01T19:34:41.685 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCTKuhiU50LChAAXuhrYBbHkk3eKzAPiEJFTg==", 2022-08-01T19:34:41.685 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:34:41.686 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:34:41.686 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:34:41.686 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:34:41.686 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:41.687 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:41.687 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:41.687 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.0", 2022-08-01T19:34:41.687 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhig3EnCxAANM20jSIQf/AKwuRX1qwT/A==", 2022-08-01T19:34:41.687 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:34:41.688 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:34:41.688 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow r", 2022-08-01T19:34:41.688 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow rw", 2022-08-01T19:34:41.688 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rwx" 2022-08-01T19:34:41.689 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:41.689 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:41.689 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:41.689 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.1", 2022-08-01T19:34:41.689 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+KuhicJR0DhAAivqEAZUhXnSwjVRwWm2bJg==", 2022-08-01T19:34:41.690 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:34:41.690 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:34:41.690 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow r", 2022-08-01T19:34:41.690 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow rw", 2022-08-01T19:34:41.691 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rwx" 2022-08-01T19:34:41.691 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:41.691 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:41.691 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:41.691 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.admin", 2022-08-01T19:34:41.692 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhiPTTGHxAAgfthCR6RX6j8F1YseFGNeA==", 2022-08-01T19:34:41.692 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:34:41.692 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:34:41.692 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:34:41.693 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:34:41.693 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:34:41.693 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:41.693 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:41.694 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:41.694 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.bootstrap-mds", 2022-08-01T19:34:41.694 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCeKuhitB9kBhAAsUnfbSq2/LlDZMDY4Ynjtg==", 2022-08-01T19:34:41.694 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:34:41.694 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile bootstrap-mds" 2022-08-01T19:34:41.695 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:41.695 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:41.695 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:41.695 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.bootstrap-mgr", 2022-08-01T19:34:41.696 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCeKuhiAjtkBhAA681VoCHnfYCZAs0tJw/upg==", 2022-08-01T19:34:41.696 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:34:41.696 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile bootstrap-mgr" 2022-08-01T19:34:41.696 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:41.696 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:41.697 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:41.697 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.bootstrap-osd", 2022-08-01T19:34:41.697 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCeKuhiH1JkBhAAqipSij+e9YhcN2r2Dp2IWw==", 2022-08-01T19:34:41.697 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:34:41.698 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile bootstrap-osd" 2022-08-01T19:34:41.698 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:41.698 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:41.698 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:41.699 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.bootstrap-rbd", 2022-08-01T19:34:41.699 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCeKuhimWlkBhAAGgP1L0s3g7Eb7HA0LDyJMw==", 2022-08-01T19:34:41.699 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:34:41.699 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile bootstrap-rbd" 2022-08-01T19:34:41.699 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:41.700 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:41.700 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:41.700 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.bootstrap-rbd-mirror", 2022-08-01T19:34:41.700 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCeKuhix4FkBhAANjR/FRBicIVjVV4Mjd63bg==", 2022-08-01T19:34:41.701 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:34:41.701 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2022-08-01T19:34:41.701 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:41.701 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:41.701 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:41.702 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.bootstrap-rgw", 2022-08-01T19:34:41.702 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCeKuhicZxkBhAAGNKPZmAVs810hNnflhsFCw==", 2022-08-01T19:34:41.702 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:34:41.702 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile bootstrap-rgw" 2022-08-01T19:34:41.703 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:41.703 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:41.703 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:41.703 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.x", 2022-08-01T19:34:41.704 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9Kuhi2jvCNhAAyUxiGByZ+OZr8Y7dh3Qt5w==", 2022-08-01T19:34:41.704 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:34:41.704 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:34:41.704 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2022-08-01T19:34:41.704 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:34:41.705 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:41.705 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:41.705 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:41.705 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.y", 2022-08-01T19:34:41.706 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhiDJXpMhAA8aek67ydbRgNO0h+miws+w==", 2022-08-01T19:34:41.706 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:34:41.706 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:34:41.706 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2022-08-01T19:34:41.706 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:34:41.707 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:41.707 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:41.707 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:34:41.707 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:34:41.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.650+0000 7fec6b5ef700 1 -- 172.21.15.17:0/1770782413 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fec44053b30 msgr2=0x7fec44055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:41.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.650+0000 7fec6b5ef700 1 --2- 172.21.15.17:0/1770782413 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fec44053b30 0x7fec44055ff0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7fec60002f70 tx=0x7fec60015000).stop 2022-08-01T19:34:41.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.650+0000 7fec6b5ef700 1 -- 172.21.15.17:0/1770782413 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec640aa760 msgr2=0x7fec641411e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:41.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.650+0000 7fec6b5ef700 1 --2- 172.21.15.17:0/1770782413 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec640aa760 0x7fec641411e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7fec54001110 tx=0x7fec54007ab0).stop 2022-08-01T19:34:41.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.650+0000 7fec6b5ef700 1 -- 172.21.15.17:0/1770782413 shutdown_connections 2022-08-01T19:34:41.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.650+0000 7fec6b5ef700 1 --2- 172.21.15.17:0/1770782413 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fec44053b30 0x7fec44055ff0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:41.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.650+0000 7fec6b5ef700 1 --2- 172.21.15.17:0/1770782413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec640b6a90 0x7fec641416d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:41.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.650+0000 7fec6b5ef700 1 --2- 172.21.15.17:0/1770782413 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec640aa760 0x7fec641411e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:41.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.650+0000 7fec6b5ef700 1 --2- 172.21.15.17:0/1770782413 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fec64060aa0 0x7fec64143710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-08-01T19:34:41.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.654+0000 7fec6b5ef700 1 -- 172.21.15.17:0/1770782413 >> 172.21.15.17:0/1770782413 conn(0x7fec640b3610 msgr2=0x7fec640a9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:41.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.654+0000 7fec6b5ef700 1 -- 172.21.15.17:0/1770782413 shutdown_connections 2022-08-01T19:34:41.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.654+0000 7fec6b5ef700 1 -- 172.21.15.17:0/1770782413 wait complete. 2022-08-01T19:34:41.713 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2022-08-01T19:34:41.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.818+0000 7fe58a0c7700 1 Processor -- start 2022-08-01T19:34:41.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.818+0000 7fe58a0c7700 1 -- start start 2022-08-01T19:34:41.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.818+0000 7fe58a0c7700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe5840b65c0 0x7fe584061600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:41.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.818+0000 7fe58a0c7700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe584147d80 0x7fe584144d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:41.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.818+0000 7fe58a0c7700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5840b2100 con 0x7fe5840b41f0 2022-08-01T19:34:41.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.818+0000 7fe58a0c7700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5840b1630 con 0x7fe584147d80 2022-08-01T19:34:41.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.818+0000 7fe58a0c7700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5840b69a0 con 0x7fe5840b65c0 2022-08-01T19:34:41.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.818+0000 7fe5898c6700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe584147d80 0x7fe584144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:41.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.818+0000 7fe5888c4700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe5840b41f0 0x7fe584144630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57844/0 (socket says 172.21.15.17:57844) 2022-08-01T19:34:41.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.818+0000 7fe5888c4700 1 -- 172.21.15.17:0/3867219657 learned_addr learned my addr 172.21.15.17:0/3867219657 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:41.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.818+0000 7fe5890c5700 1 --2- 172.21.15.17:0/3867219657 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe5840b65c0 0x7fe584061600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:41.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.818+0000 7fe57bfff700 1 -- 172.21.15.17:0/3867219657 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1789652960 0 0) 0x7fe5840b2100 con 0x7fe5840b41f0 2022-08-01T19:34:41.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.818+0000 7fe57bfff700 1 -- 172.21.15.17:0/3867219657 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe568003260 con 0x7fe5840b41f0 2022-08-01T19:34:41.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.818+0000 7fe5898c6700 1 -- 172.21.15.17:0/3867219657 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe5840b65c0 msgr2=0x7fe584061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:41.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.818+0000 7fe5898c6700 1 --2- 172.21.15.17:0/3867219657 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe5840b65c0 0x7fe584061600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:41.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.818+0000 7fe5898c6700 1 -- 172.21.15.17:0/3867219657 >> v1:172.21.15.17:6789/0 conn(0x7fe5840b41f0 legacy=0x7fe584144630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:41.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.818+0000 7fe5898c6700 1 -- 172.21.15.17:0/3867219657 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5841482a0 con 0x7fe584147d80 2022-08-01T19:34:41.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.818+0000 7fe5898c6700 1 --2- 172.21.15.17:0/3867219657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe584147d80 0x7fe584144d40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7fe57c0032f0 tx=0x7fe57c0091d0).ready entity=mon.0 client_cookie=eb4698b1228e79a2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:41.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.818+0000 7fe57bfff700 1 -- 172.21.15.17:0/3867219657 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe57c00b600 con 0x7fe584147d80 2022-08-01T19:34:41.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.818+0000 7fe57bfff700 1 -- 172.21.15.17:0/3867219657 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57c00b760 con 0x7fe584147d80 2022-08-01T19:34:41.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.818+0000 7fe57bfff700 1 -- 172.21.15.17:0/3867219657 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe57c00b8d0 con 0x7fe584147d80 2022-08-01T19:34:41.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.822+0000 7fe58a0c7700 1 -- 172.21.15.17:0/3867219657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe584147d80 msgr2=0x7fe584144d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:41.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.822+0000 7fe58a0c7700 1 --2- 172.21.15.17:0/3867219657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe584147d80 0x7fe584144d40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7fe57c0032f0 tx=0x7fe57c0091d0).stop 2022-08-01T19:34:41.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.822+0000 7fe58a0c7700 1 -- 172.21.15.17:0/3867219657 shutdown_connections 2022-08-01T19:34:41.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.822+0000 7fe58a0c7700 1 --2- 172.21.15.17:0/3867219657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe584147d80 0x7fe584144d40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:41.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.822+0000 7fe58a0c7700 1 --2- 172.21.15.17:0/3867219657 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe5840b65c0 0x7fe584061600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:41.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.822+0000 7fe58a0c7700 1 -- 172.21.15.17:0/3867219657 >> 172.21.15.17:0/3867219657 conn(0x7fe58405c620 msgr2=0x7fe58405ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:41.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.822+0000 7fe58a0c7700 1 -- 172.21.15.17:0/3867219657 shutdown_connections 2022-08-01T19:34:41.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.822+0000 7fe58a0c7700 1 -- 172.21.15.17:0/3867219657 wait complete. 2022-08-01T19:34:41.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.822+0000 7fe58a0c7700 1 Processor -- start 2022-08-01T19:34:41.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.822+0000 7fe58a0c7700 1 -- start start 2022-08-01T19:34:41.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.822+0000 7fe58a0c7700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe5840b41f0 0x7fe584149d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:41.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.822+0000 7fe58a0c7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5840b65c0 0x7fe584156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:41.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.822+0000 7fe58a0c7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe584147d80 0x7fe5840ada00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:41.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.822+0000 7fe58a0c7700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58414a650 con 0x7fe584147d80 2022-08-01T19:34:41.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.822+0000 7fe58a0c7700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58414a810 con 0x7fe5840b65c0 2022-08-01T19:34:41.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.822+0000 7fe58a0c7700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe58414a270 con 0x7fe5840b41f0 2022-08-01T19:34:41.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.826+0000 7fe5898c6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe584147d80 0x7fe5840ada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:41.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.826+0000 7fe5898c6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe584147d80 0x7fe5840ada00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37424/0 (socket says 172.21.15.17:37424) 2022-08-01T19:34:41.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.826+0000 7fe5898c6700 1 -- 172.21.15.17:0/1368405858 learned_addr learned my addr 172.21.15.17:0/1368405858 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:41.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.826+0000 7fe5890c5700 1 --2- 172.21.15.17:0/1368405858 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe5840b41f0 0x7fe584149d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:41.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.826+0000 7fe5888c4700 1 --2- 172.21.15.17:0/1368405858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5840b65c0 0x7fe584156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:41.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.826+0000 7fe5898c6700 1 -- 172.21.15.17:0/1368405858 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe5840b41f0 msgr2=0x7fe584149d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:41.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.826+0000 7fe5898c6700 1 --2- 172.21.15.17:0/1368405858 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe5840b41f0 0x7fe584149d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:41.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.826+0000 7fe5898c6700 1 -- 172.21.15.17:0/1368405858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5840b65c0 msgr2=0x7fe584156730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:41.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.826+0000 7fe5898c6700 1 --2- 172.21.15.17:0/1368405858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5840b65c0 0x7fe584156730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:41.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.826+0000 7fe5898c6700 1 -- 172.21.15.17:0/1368405858 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe57c009470 con 0x7fe584147d80 2022-08-01T19:34:41.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.826+0000 7fe5898c6700 1 --2- 172.21.15.17:0/1368405858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe584147d80 0x7fe5840ada00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7fe584146b20 tx=0x7fe57c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:41.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.826+0000 7fe57a7fc700 1 -- 172.21.15.17:0/1368405858 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe57c00b600 con 0x7fe584147d80 2022-08-01T19:34:41.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.826+0000 7fe57a7fc700 1 -- 172.21.15.17:0/1368405858 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57c011e90 con 0x7fe584147d80 2022-08-01T19:34:41.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.826+0000 7fe58a0c7700 1 -- 172.21.15.17:0/1368405858 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe584141a70 con 0x7fe584147d80 2022-08-01T19:34:41.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.826+0000 7fe57a7fc700 1 -- 172.21.15.17:0/1368405858 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe57c0205f0 con 0x7fe584147d80 2022-08-01T19:34:41.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.826+0000 7fe58a0c7700 1 -- 172.21.15.17:0/1368405858 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe584156c20 con 0x7fe584147d80 2022-08-01T19:34:41.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.826+0000 7fe58a0c7700 1 -- 172.21.15.17:0/1368405858 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe58414ab30 con 0x7fe584147d80 2022-08-01T19:34:41.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.826+0000 7fe57a7fc700 1 -- 172.21.15.17:0/1368405858 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe57c027030 con 0x7fe584147d80 2022-08-01T19:34:41.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.826+0000 7fe57a7fc700 1 --2- 172.21.15.17:0/1368405858 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe56c053b00 0x7fe56c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:41.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.826+0000 7fe57a7fc700 1 -- 172.21.15.17:0/1368405858 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fe57c06cc20 con 0x7fe584147d80 2022-08-01T19:34:41.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.826+0000 7fe5890c5700 1 --2- 172.21.15.17:0/1368405858 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe56c053b00 0x7fe56c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:41.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.830+0000 7fe5890c5700 1 --2- 172.21.15.17:0/1368405858 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe56c053b00 0x7fe56c055fc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fe58000b000 tx=0x7fe580008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:41.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:41.834+0000 7fe57a7fc700 1 -- 172.21.15.17:0/1368405858 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe57c043d20 con 0x7fe584147d80 2022-08-01T19:34:42.008 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.002+0000 7fe58a0c7700 1 -- 172.21.15.17:0/1368405858 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fe5840b2330 con 0x7fe584147d80 2022-08-01T19:34:42.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.010+0000 7fe57a7fc700 1 -- 172.21.15.17:0/1368405858 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 updated v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe57c0250a0 con 0x7fe584147d80 2022-08-01T19:34:42.018 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.014+0000 7fe58a0c7700 1 -- 172.21.15.17:0/1368405858 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe56c053b00 msgr2=0x7fe56c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:42.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.014+0000 7fe58a0c7700 1 --2- 172.21.15.17:0/1368405858 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe56c053b00 0x7fe56c055fc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fe58000b000 tx=0x7fe580008040).stop 2022-08-01T19:34:42.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.014+0000 7fe58a0c7700 1 -- 172.21.15.17:0/1368405858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe584147d80 msgr2=0x7fe5840ada00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:42.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.014+0000 7fe58a0c7700 1 --2- 172.21.15.17:0/1368405858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe584147d80 0x7fe5840ada00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7fe584146b20 tx=0x7fe57c016000).stop 2022-08-01T19:34:42.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.014+0000 7fe58a0c7700 1 -- 172.21.15.17:0/1368405858 shutdown_connections 2022-08-01T19:34:42.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.014+0000 7fe58a0c7700 1 --2- 172.21.15.17:0/1368405858 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe56c053b00 0x7fe56c055fc0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:42.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.014+0000 7fe58a0c7700 1 --2- 172.21.15.17:0/1368405858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe584147d80 0x7fe5840ada00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:42.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.014+0000 7fe58a0c7700 1 --2- 172.21.15.17:0/1368405858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5840b65c0 0x7fe584156730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:42.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.014+0000 7fe58a0c7700 1 --2- 172.21.15.17:0/1368405858 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe5840b41f0 0x7fe584149d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:42.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.014+0000 7fe58a0c7700 1 -- 172.21.15.17:0/1368405858 >> 172.21.15.17:0/1368405858 conn(0x7fe58405c620 msgr2=0x7fe5840b5540 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:42.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.014+0000 7fe58a0c7700 1 -- 172.21.15.17:0/1368405858 shutdown_connections 2022-08-01T19:34:42.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.014+0000 7fe58a0c7700 1 -- 172.21.15.17:0/1368405858 wait complete. 2022-08-01T19:34:42.022 INFO:teuthology.orchestra.run.smithi017.stderr:updated 2022-08-01T19:34:42.030 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2022-08-01T19:34:42.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.134+0000 7fefd4a34700 1 Processor -- start 2022-08-01T19:34:42.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.134+0000 7fefd4a34700 1 -- start start 2022-08-01T19:34:42.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.138+0000 7fefd4a34700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fefd00b41f0 0x7fefd0061d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:42.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.138+0000 7fefd4a34700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fefd0142ff0 0x7fefd0143480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:42.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.138+0000 7fefd4a34700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefd00b60e0 con 0x7fefd00b6420 2022-08-01T19:34:42.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.138+0000 7fefd4a34700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefd00b2100 con 0x7fefd0142ff0 2022-08-01T19:34:42.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.138+0000 7fefd4a34700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefd00b23a0 con 0x7fefd00b41f0 2022-08-01T19:34:42.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.138+0000 7fefcf59e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fefd0142ff0 0x7fefd0143480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:42.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.138+0000 7fefcf59e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fefd0142ff0 0x7fefd0143480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37432/0 (socket says 172.21.15.17:37432) 2022-08-01T19:34:42.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.138+0000 7fefcf59e700 1 -- 172.21.15.17:0/961647020 learned_addr learned my addr 172.21.15.17:0/961647020 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:42.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.138+0000 7fefce59c700 1 --2- 172.21.15.17:0/961647020 >> v2:172.21.15.77:3300/0 conn(0x7fefd00b41f0 0x7fefd0061d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:42.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.138+0000 7fefcf59e700 1 -- 172.21.15.17:0/961647020 >> v2:172.21.15.77:3300/0 conn(0x7fefd00b41f0 msgr2=0x7fefd0061d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:42.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.138+0000 7fefcf59e700 1 --2- 172.21.15.17:0/961647020 >> v2:172.21.15.77:3300/0 conn(0x7fefd00b41f0 0x7fefd0061d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:42.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.138+0000 7fefcf59e700 1 -- 172.21.15.17:0/961647020 >> v1:172.21.15.77:6789/0 conn(0x7fefd00b6420 legacy=0x7fefd0061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:42.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.138+0000 7fefcf59e700 1 -- 172.21.15.17:0/961647020 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd00b2640 con 0x7fefd0142ff0 2022-08-01T19:34:42.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.138+0000 7fefcf59e700 1 --2- 172.21.15.17:0/961647020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefd0142ff0 0x7fefd0143480 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fefc80032f0 tx=0x7fefc8009290).ready entity=mon.0 client_cookie=adc96771cc29fc68 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:42.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.138+0000 7fefcdd9b700 1 -- 172.21.15.17:0/961647020 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefc800b870 con 0x7fefd0142ff0 2022-08-01T19:34:42.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.138+0000 7fefcdd9b700 1 -- 172.21.15.17:0/961647020 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc800b9d0 con 0x7fefd0142ff0 2022-08-01T19:34:42.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.138+0000 7fefcdd9b700 1 -- 172.21.15.17:0/961647020 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefc800bb40 con 0x7fefd0142ff0 2022-08-01T19:34:42.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.138+0000 7fefd4a34700 1 -- 172.21.15.17:0/961647020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefd0142ff0 msgr2=0x7fefd0143480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:42.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.138+0000 7fefd4a34700 1 --2- 172.21.15.17:0/961647020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefd0142ff0 0x7fefd0143480 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fefc80032f0 tx=0x7fefc8009290).stop 2022-08-01T19:34:42.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.138+0000 7fefd4a34700 1 -- 172.21.15.17:0/961647020 shutdown_connections 2022-08-01T19:34:42.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.138+0000 7fefd4a34700 1 --2- 172.21.15.17:0/961647020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefd0142ff0 0x7fefd0143480 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:42.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.138+0000 7fefd4a34700 1 --2- 172.21.15.17:0/961647020 >> v2:172.21.15.77:3300/0 conn(0x7fefd00b41f0 0x7fefd0061d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:42.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.138+0000 7fefd4a34700 1 -- 172.21.15.17:0/961647020 >> 172.21.15.17:0/961647020 conn(0x7fefd005c620 msgr2=0x7fefd005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:42.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.138+0000 7fefd4a34700 1 -- 172.21.15.17:0/961647020 shutdown_connections 2022-08-01T19:34:42.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.138+0000 7fefd4a34700 1 -- 172.21.15.17:0/961647020 wait complete. 2022-08-01T19:34:42.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.142+0000 7fefd4a34700 1 Processor -- start 2022-08-01T19:34:42.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.142+0000 7fefd4a34700 1 -- start start 2022-08-01T19:34:42.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.142+0000 7fefd4a34700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefd00b41f0 0x7fefd013d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:42.154 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.142+0000 7fefd4a34700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fefd00b6420 0x7fefd0140460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:42.154 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.142+0000 7fefd4a34700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefd0142ff0 0x7fefd00b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:42.154 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.142+0000 7fefd4a34700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd013d020 con 0x7fefd0142ff0 2022-08-01T19:34:42.154 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.142+0000 7fefd4a34700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd0140c90 con 0x7fefd00b41f0 2022-08-01T19:34:42.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.142+0000 7fefd4a34700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefd01531e0 con 0x7fefd00b6420 2022-08-01T19:34:42.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.142+0000 7fefcf59e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefd0142ff0 0x7fefd00b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:42.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.142+0000 7fefcf59e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefd0142ff0 0x7fefd00b0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37438/0 (socket says 172.21.15.17:37438) 2022-08-01T19:34:42.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.142+0000 7fefcf59e700 1 -- 172.21.15.17:0/4023918188 learned_addr learned my addr 172.21.15.17:0/4023918188 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:42.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.142+0000 7fefced9d700 1 --2- 172.21.15.17:0/4023918188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefd00b41f0 0x7fefd013d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:42.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.142+0000 7fefce59c700 1 --2- 172.21.15.17:0/4023918188 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fefd00b6420 0x7fefd0140460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:42.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.142+0000 7fefcf59e700 1 -- 172.21.15.17:0/4023918188 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fefd00b6420 msgr2=0x7fefd0140460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:42.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.142+0000 7fefcf59e700 1 --2- 172.21.15.17:0/4023918188 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fefd00b6420 0x7fefd0140460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:42.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.142+0000 7fefcf59e700 1 -- 172.21.15.17:0/4023918188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefd00b41f0 msgr2=0x7fefd013d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:42.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.142+0000 7fefcf59e700 1 --2- 172.21.15.17:0/4023918188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefd00b41f0 0x7fefd013d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:42.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.142+0000 7fefcf59e700 1 -- 172.21.15.17:0/4023918188 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc8009530 con 0x7fefd0142ff0 2022-08-01T19:34:42.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.142+0000 7fefcf59e700 1 --2- 172.21.15.17:0/4023918188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefd0142ff0 0x7fefd00b0040 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7fefc80032f0 tx=0x7fefc8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:42.158 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.142+0000 7fefc7fff700 1 -- 172.21.15.17:0/4023918188 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefc800b870 con 0x7fefd0142ff0 2022-08-01T19:34:42.158 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.142+0000 7fefc7fff700 1 -- 172.21.15.17:0/4023918188 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc8021440 con 0x7fefd0142ff0 2022-08-01T19:34:42.158 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.142+0000 7fefc7fff700 1 -- 172.21.15.17:0/4023918188 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefc80217a0 con 0x7fefd0142ff0 2022-08-01T19:34:42.158 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.142+0000 7fefd4a34700 1 -- 172.21.15.17:0/4023918188 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefd013ffc0 con 0x7fefd0142ff0 2022-08-01T19:34:42.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.146+0000 7fefd4a34700 1 -- 172.21.15.17:0/4023918188 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefd0153480 con 0x7fefd0142ff0 2022-08-01T19:34:42.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.146+0000 7fefd4a34700 1 -- 172.21.15.17:0/4023918188 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefd00b28e0 con 0x7fefd0142ff0 2022-08-01T19:34:42.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.146+0000 7fefc7fff700 1 -- 172.21.15.17:0/4023918188 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fefc8027020 con 0x7fefd0142ff0 2022-08-01T19:34:42.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.146+0000 7fefc7fff700 1 --2- 172.21.15.17:0/4023918188 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fefb4053a10 0x7fefb4055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:42.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.146+0000 7fefc7fff700 1 -- 172.21.15.17:0/4023918188 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fefc806d170 con 0x7fefd0142ff0 2022-08-01T19:34:42.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.146+0000 7fefced9d700 1 --2- 172.21.15.17:0/4023918188 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fefb4053a10 0x7fefb4055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:42.162 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.150+0000 7fefced9d700 1 --2- 172.21.15.17:0/4023918188 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fefb4053a10 0x7fefb4055ed0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fefc000d000 tx=0x7fefc0000bc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:42.162 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.154+0000 7fefc7fff700 1 -- 172.21.15.17:0/4023918188 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fefc80442b0 con 0x7fefd0142ff0 2022-08-01T19:34:42.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.322+0000 7fefd4a34700 1 -- 172.21.15.17:0/4023918188 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fefd00574e0 con 0x7fefd0142ff0 2022-08-01T19:34:42.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.330+0000 7fefc7fff700 1 -- 172.21.15.17:0/4023918188 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 updated v10) v1 ==== 97+0+0 (secure 0 0 0) 0x7fefc8015020 con 0x7fefd0142ff0 2022-08-01T19:34:42.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.334+0000 7fefd4a34700 1 -- 172.21.15.17:0/4023918188 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fefb4053a10 msgr2=0x7fefb4055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:42.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.334+0000 7fefd4a34700 1 --2- 172.21.15.17:0/4023918188 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fefb4053a10 0x7fefb4055ed0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fefc000d000 tx=0x7fefc0000bc0).stop 2022-08-01T19:34:42.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.334+0000 7fefd4a34700 1 -- 172.21.15.17:0/4023918188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefd0142ff0 msgr2=0x7fefd00b0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:42.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.334+0000 7fefd4a34700 1 --2- 172.21.15.17:0/4023918188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefd0142ff0 0x7fefd00b0040 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7fefc80032f0 tx=0x7fefc8016000).stop 2022-08-01T19:34:42.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.334+0000 7fefd4a34700 1 -- 172.21.15.17:0/4023918188 shutdown_connections 2022-08-01T19:34:42.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.334+0000 7fefd4a34700 1 --2- 172.21.15.17:0/4023918188 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fefb4053a10 0x7fefb4055ed0 secure :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0x7fefc000d000 tx=0x7fefc0000bc0).stop 2022-08-01T19:34:42.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.334+0000 7fefd4a34700 1 --2- 172.21.15.17:0/4023918188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefd0142ff0 0x7fefd00b0040 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:42.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.334+0000 7fefd4a34700 1 --2- 172.21.15.17:0/4023918188 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fefd00b6420 0x7fefd0140460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:42.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.334+0000 7fefd4a34700 1 --2- 172.21.15.17:0/4023918188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefd00b41f0 0x7fefd013d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:42.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.334+0000 7fefd4a34700 1 -- 172.21.15.17:0/4023918188 >> 172.21.15.17:0/4023918188 conn(0x7fefd005c620 msgr2=0x7fefd01424a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:42.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.334+0000 7fefd4a34700 1 -- 172.21.15.17:0/4023918188 shutdown_connections 2022-08-01T19:34:42.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.334+0000 7fefd4a34700 1 -- 172.21.15.17:0/4023918188 wait complete. 2022-08-01T19:34:42.343 INFO:teuthology.orchestra.run.smithi017.stderr:updated 2022-08-01T19:34:42.355 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2022-08-01T19:34:42.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feaed7f5700 1 Processor -- start 2022-08-01T19:34:42.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feaed7f5700 1 -- start start 2022-08-01T19:34:42.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feaed7f5700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7feae80b36e0 0x7feae8141a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:42.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feaed7f5700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7feae8144e60 0x7feae8066ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:42.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feaed7f5700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feae8064ef0 con 0x7feae8064b10 2022-08-01T19:34:42.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feaed7f5700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feae813ebf0 con 0x7feae8144e60 2022-08-01T19:34:42.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feaed7f5700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae81452d0 con 0x7feae80b36e0 2022-08-01T19:34:42.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feae7fff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7feae8064b10 0x7feae8141370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57868/0 (socket says 172.21.15.17:57868) 2022-08-01T19:34:42.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feae7fff700 1 -- 172.21.15.17:0/188509381 learned_addr learned my addr 172.21.15.17:0/188509381 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:42.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feadf7fe700 1 --2- 172.21.15.17:0/188509381 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7feae80b36e0 0x7feae8141a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:42.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feaecff4700 1 --2- 172.21.15.17:0/188509381 >> v2:172.21.15.77:3300/0 conn(0x7feae8144e60 0x7feae8066ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:42.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feae77fe700 1 -- 172.21.15.17:0/188509381 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 583762090 0 0) 0x7feae8064ef0 con 0x7feae8064b10 2022-08-01T19:34:42.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feae77fe700 1 -- 172.21.15.17:0/188509381 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feac8003260 con 0x7feae8064b10 2022-08-01T19:34:42.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feadf7fe700 1 -- 172.21.15.17:0/188509381 >> v2:172.21.15.77:3300/0 conn(0x7feae8144e60 msgr2=0x7feae8066ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:42.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feadf7fe700 1 --2- 172.21.15.17:0/188509381 >> v2:172.21.15.77:3300/0 conn(0x7feae8144e60 0x7feae8066ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:42.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feadf7fe700 1 -- 172.21.15.17:0/188509381 >> v1:172.21.15.17:6789/0 conn(0x7feae8064b10 legacy=0x7feae8141370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:42.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feadf7fe700 1 -- 172.21.15.17:0/188509381 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae8066690 con 0x7feae80b36e0 2022-08-01T19:34:42.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feadf7fe700 1 --2- 172.21.15.17:0/188509381 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7feae80b36e0 0x7feae8141a80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fead0002f70 tx=0x7fead0006df0).ready entity=mon.2 client_cookie=cde22683bd17d02f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:42.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feae77fe700 1 -- 172.21.15.17:0/188509381 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fead000d4a0 con 0x7feae80b36e0 2022-08-01T19:34:42.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feae77fe700 1 -- 172.21.15.17:0/188509381 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead0000e90 con 0x7feae80b36e0 2022-08-01T19:34:42.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feae77fe700 1 -- 172.21.15.17:0/188509381 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fead000d7f0 con 0x7feae80b36e0 2022-08-01T19:34:42.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feaed7f5700 1 -- 172.21.15.17:0/188509381 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7feae80b36e0 msgr2=0x7feae8141a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:42.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feaed7f5700 1 --2- 172.21.15.17:0/188509381 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7feae80b36e0 0x7feae8141a80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fead0002f70 tx=0x7fead0006df0).stop 2022-08-01T19:34:42.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feaed7f5700 1 -- 172.21.15.17:0/188509381 shutdown_connections 2022-08-01T19:34:42.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feaed7f5700 1 --2- 172.21.15.17:0/188509381 >> v2:172.21.15.77:3300/0 conn(0x7feae8144e60 0x7feae8066ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:42.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feaed7f5700 1 --2- 172.21.15.17:0/188509381 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7feae80b36e0 0x7feae8141a80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:42.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feaed7f5700 1 -- 172.21.15.17:0/188509381 >> 172.21.15.17:0/188509381 conn(0x7feae805f840 msgr2=0x7feae805fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:42.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feaed7f5700 1 -- 172.21.15.17:0/188509381 shutdown_connections 2022-08-01T19:34:42.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feaed7f5700 1 -- 172.21.15.17:0/188509381 wait complete. 2022-08-01T19:34:42.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feaed7f5700 1 Processor -- start 2022-08-01T19:34:42.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feaed7f5700 1 -- start start 2022-08-01T19:34:42.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feaed7f5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feae8064b10 0x7feae813b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:42.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feaed7f5700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7feae80b36e0 0x7feae80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:42.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feaed7f5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feae8144e60 0x7feae80b1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:42.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feaed7f5700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae8154640 con 0x7feae8064b10 2022-08-01T19:34:42.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feaed7f5700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae80addf0 con 0x7feae8144e60 2022-08-01T19:34:42.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feaed7f5700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae80ae090 con 0x7feae80b36e0 2022-08-01T19:34:42.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feae7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feae8064b10 0x7feae813b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:42.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feae7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feae8064b10 0x7feae813b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37448/0 (socket says 172.21.15.17:37448) 2022-08-01T19:34:42.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feae7fff700 1 -- 172.21.15.17:0/72282900 learned_addr learned my addr 172.21.15.17:0/72282900 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:42.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feaecff4700 1 --2- 172.21.15.17:0/72282900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feae8144e60 0x7feae80b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:42.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feadf7fe700 1 --2- 172.21.15.17:0/72282900 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7feae80b36e0 0x7feae80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:42.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feae7fff700 1 -- 172.21.15.17:0/72282900 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7feae80b36e0 msgr2=0x7feae80b7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:42.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feae7fff700 1 --2- 172.21.15.17:0/72282900 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7feae80b36e0 0x7feae80b7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:42.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feae7fff700 1 -- 172.21.15.17:0/72282900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feae8144e60 msgr2=0x7feae80b1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:42.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feae7fff700 1 --2- 172.21.15.17:0/72282900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feae8144e60 0x7feae80b1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:42.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feae7fff700 1 -- 172.21.15.17:0/72282900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead000b180 con 0x7feae8064b10 2022-08-01T19:34:42.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feaecff4700 1 --2- 172.21.15.17:0/72282900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feae8144e60 0x7feae80b1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:34:42.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.458+0000 7feae7fff700 1 --2- 172.21.15.17:0/72282900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feae8064b10 0x7feae813b6f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7fead8001110 tx=0x7fead8007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:42.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.462+0000 7feae5ffb700 1 -- 172.21.15.17:0/72282900 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fead8007e00 con 0x7feae8064b10 2022-08-01T19:34:42.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.462+0000 7feae5ffb700 1 -- 172.21.15.17:0/72282900 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead80072e0 con 0x7feae8064b10 2022-08-01T19:34:42.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.462+0000 7feaed7f5700 1 -- 172.21.15.17:0/72282900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae80ae330 con 0x7feae8064b10 2022-08-01T19:34:42.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.462+0000 7feae5ffb700 1 -- 172.21.15.17:0/72282900 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fead8005800 con 0x7feae8064b10 2022-08-01T19:34:42.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.462+0000 7feaed7f5700 1 -- 172.21.15.17:0/72282900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae813bcd0 con 0x7feae8064b10 2022-08-01T19:34:42.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.462+0000 7feaed7f5700 1 -- 172.21.15.17:0/72282900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feae80b1310 con 0x7feae8064b10 2022-08-01T19:34:42.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.462+0000 7feae5ffb700 1 -- 172.21.15.17:0/72282900 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fead8010070 con 0x7feae8064b10 2022-08-01T19:34:42.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.462+0000 7feae5ffb700 1 --2- 172.21.15.17:0/72282900 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7feacc0539e0 0x7feacc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:42.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.462+0000 7feae5ffb700 1 -- 172.21.15.17:0/72282900 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fead806b3c0 con 0x7feae8064b10 2022-08-01T19:34:42.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.462+0000 7feadf7fe700 1 --2- 172.21.15.17:0/72282900 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7feacc0539e0 0x7feacc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:42.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.462+0000 7feadf7fe700 1 --2- 172.21.15.17:0/72282900 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7feacc0539e0 0x7feacc055ea0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fead0018000 tx=0x7fead0014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:42.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.470+0000 7feae5ffb700 1 -- 172.21.15.17:0/72282900 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fead80424c0 con 0x7feae8064b10 2022-08-01T19:34:42.640 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.634+0000 7feaed7f5700 1 -- 172.21.15.17:0/72282900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7feae80574e0 con 0x7feae8064b10 2022-08-01T19:34:42.648 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.642+0000 7feae5ffb700 1 -- 172.21.15.17:0/72282900 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 updated v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7fead8045ae0 con 0x7feae8064b10 2022-08-01T19:34:42.650 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.642+0000 7feaed7f5700 1 -- 172.21.15.17:0/72282900 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7feacc0539e0 msgr2=0x7feacc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:42.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.642+0000 7feaed7f5700 1 --2- 172.21.15.17:0/72282900 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7feacc0539e0 0x7feacc055ea0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fead0018000 tx=0x7fead0014000).stop 2022-08-01T19:34:42.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.642+0000 7feaed7f5700 1 -- 172.21.15.17:0/72282900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feae8064b10 msgr2=0x7feae813b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:42.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.642+0000 7feaed7f5700 1 --2- 172.21.15.17:0/72282900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feae8064b10 0x7feae813b6f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7fead8001110 tx=0x7fead8007aa0).stop 2022-08-01T19:34:42.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.646+0000 7feaed7f5700 1 -- 172.21.15.17:0/72282900 shutdown_connections 2022-08-01T19:34:42.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.646+0000 7feaed7f5700 1 --2- 172.21.15.17:0/72282900 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7feacc0539e0 0x7feacc055ea0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:42.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.646+0000 7feaed7f5700 1 --2- 172.21.15.17:0/72282900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feae8144e60 0x7feae80b1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:42.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.646+0000 7feaed7f5700 1 --2- 172.21.15.17:0/72282900 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7feae80b36e0 0x7feae80b7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:42.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.646+0000 7feaed7f5700 1 --2- 172.21.15.17:0/72282900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feae8064b10 0x7feae813b6f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:42.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.646+0000 7feaed7f5700 1 -- 172.21.15.17:0/72282900 >> 172.21.15.17:0/72282900 conn(0x7feae805f840 msgr2=0x7feae805ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:42.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.646+0000 7feaed7f5700 1 -- 172.21.15.17:0/72282900 shutdown_connections 2022-08-01T19:34:42.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.646+0000 7feaed7f5700 1 -- 172.21.15.17:0/72282900 wait complete. 2022-08-01T19:34:42.654 INFO:teuthology.orchestra.run.smithi017.stderr:updated 2022-08-01T19:34:42.663 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2022-08-01T19:34:42.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.762+0000 7fc48d0cb700 1 Processor -- start 2022-08-01T19:34:42.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.762+0000 7fc48d0cb700 1 -- start start 2022-08-01T19:34:42.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.762+0000 7fc48d0cb700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4880614b0 0x7fc488061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:42.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.762+0000 7fc48d0cb700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc4880ad750 0x7fc4880b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:42.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.762+0000 7fc48d0cb700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fc4881405d0 0x7fc488140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:42.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.762+0000 7fc48d0cb700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4880aee80 con 0x7fc4880ad750 2022-08-01T19:34:42.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.762+0000 7fc48d0cb700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4880b70b0 con 0x7fc4881405d0 2022-08-01T19:34:42.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.762+0000 7fc48d0cb700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4880af0b0 con 0x7fc4880614b0 2022-08-01T19:34:42.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.762+0000 7fc4877fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc4880ad750 0x7fc4880b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:42.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.762+0000 7fc48c8ca700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fc4881405d0 0x7fc488140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:42.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.762+0000 7fc4877fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc4880ad750 0x7fc4880b9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37458/0 (socket says 172.21.15.17:37458) 2022-08-01T19:34:42.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.762+0000 7fc487fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4880614b0 0x7fc488061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:42.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.762+0000 7fc4877fe700 1 -- 172.21.15.17:0/563265547 learned_addr learned my addr 172.21.15.17:0/563265547 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:42.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.762+0000 7fc48c8ca700 1 -- 172.21.15.17:0/563265547 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4880614b0 msgr2=0x7fc488061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:42.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.762+0000 7fc48c8ca700 1 --2- 172.21.15.17:0/563265547 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4880614b0 0x7fc488061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:42.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.762+0000 7fc48c8ca700 1 -- 172.21.15.17:0/563265547 >> v2:172.21.15.17:3300/0 conn(0x7fc4880ad750 msgr2=0x7fc4880b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:42.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.762+0000 7fc48c8ca700 1 --2- 172.21.15.17:0/563265547 >> v2:172.21.15.17:3300/0 conn(0x7fc4880ad750 0x7fc4880b9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:42.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.762+0000 7fc48c8ca700 1 -- 172.21.15.17:0/563265547 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4880af350 con 0x7fc4881405d0 2022-08-01T19:34:42.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.762+0000 7fc4877fe700 1 --2- 172.21.15.17:0/563265547 >> v2:172.21.15.17:3300/0 conn(0x7fc4880ad750 0x7fc4880b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:34:42.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.762+0000 7fc48c8ca700 1 --2- 172.21.15.17:0/563265547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4881405d0 0x7fc488140a80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fc478002f70 tx=0x7fc478006df0).ready entity=mon.1 client_cookie=3eeef35d779bf561 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:42.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.762+0000 7fc486ffd700 1 -- 172.21.15.17:0/563265547 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc47800d4a0 con 0x7fc4881405d0 2022-08-01T19:34:42.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.762+0000 7fc486ffd700 1 -- 172.21.15.17:0/563265547 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc478000e90 con 0x7fc4881405d0 2022-08-01T19:34:42.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.762+0000 7fc486ffd700 1 -- 172.21.15.17:0/563265547 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc47800d7f0 con 0x7fc4881405d0 2022-08-01T19:34:42.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.762+0000 7fc48d0cb700 1 -- 172.21.15.17:0/563265547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4881405d0 msgr2=0x7fc488140a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:42.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.762+0000 7fc48d0cb700 1 --2- 172.21.15.17:0/563265547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4881405d0 0x7fc488140a80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fc478002f70 tx=0x7fc478006df0).stop 2022-08-01T19:34:42.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.762+0000 7fc48d0cb700 1 -- 172.21.15.17:0/563265547 shutdown_connections 2022-08-01T19:34:42.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.762+0000 7fc48d0cb700 1 --2- 172.21.15.17:0/563265547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4881405d0 0x7fc488140a80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:42.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.762+0000 7fc48d0cb700 1 --2- 172.21.15.17:0/563265547 >> v2:172.21.15.17:3300/0 conn(0x7fc4880ad750 0x7fc4880b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:42.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.762+0000 7fc48d0cb700 1 --2- 172.21.15.17:0/563265547 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4880614b0 0x7fc488061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:42.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.762+0000 7fc48d0cb700 1 -- 172.21.15.17:0/563265547 >> 172.21.15.17:0/563265547 conn(0x7fc488066f70 msgr2=0x7fc488067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:42.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.762+0000 7fc48d0cb700 1 -- 172.21.15.17:0/563265547 shutdown_connections 2022-08-01T19:34:42.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.762+0000 7fc48d0cb700 1 -- 172.21.15.17:0/563265547 wait complete. 2022-08-01T19:34:42.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.766+0000 7fc48d0cb700 1 Processor -- start 2022-08-01T19:34:42.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.766+0000 7fc48d0cb700 1 -- start start 2022-08-01T19:34:42.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.766+0000 7fc48d0cb700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4880614b0 0x7fc48805d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:42.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.766+0000 7fc48d0cb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4880ad750 0x7fc48813f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:42.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.766+0000 7fc48d0cb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4881405d0 0x7fc48813fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:42.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.766+0000 7fc48d0cb700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4880aa2c0 con 0x7fc4880ad750 2022-08-01T19:34:42.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.766+0000 7fc48d0cb700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc488146420 con 0x7fc4881405d0 2022-08-01T19:34:42.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.766+0000 7fc48d0cb700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4881566b0 con 0x7fc4880614b0 2022-08-01T19:34:42.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.766+0000 7fc4877fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4880ad750 0x7fc48813f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:42.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.766+0000 7fc487fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4880614b0 0x7fc48805d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:42.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.766+0000 7fc4877fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4880ad750 0x7fc48813f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37464/0 (socket says 172.21.15.17:37464) 2022-08-01T19:34:42.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.766+0000 7fc4877fe700 1 -- 172.21.15.17:0/787463592 learned_addr learned my addr 172.21.15.17:0/787463592 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:42.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.766+0000 7fc48c8ca700 1 --2- 172.21.15.17:0/787463592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4881405d0 0x7fc48813fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:42.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.766+0000 7fc4877fe700 1 -- 172.21.15.17:0/787463592 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4880614b0 msgr2=0x7fc48805d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:42.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.766+0000 7fc4877fe700 1 --2- 172.21.15.17:0/787463592 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4880614b0 0x7fc48805d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:42.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.766+0000 7fc4877fe700 1 -- 172.21.15.17:0/787463592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4881405d0 msgr2=0x7fc48813fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:42.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.766+0000 7fc4877fe700 1 --2- 172.21.15.17:0/787463592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4881405d0 0x7fc48813fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:42.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.766+0000 7fc4877fe700 1 -- 172.21.15.17:0/787463592 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc47c00b1e0 con 0x7fc4880ad750 2022-08-01T19:34:42.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.766+0000 7fc4877fe700 1 --2- 172.21.15.17:0/787463592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4880ad750 0x7fc48813f920 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7fc47c00c000 tx=0x7fc47c006eb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:42.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.766+0000 7fc4857fa700 1 -- 172.21.15.17:0/787463592 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc47c012600 con 0x7fc4880ad750 2022-08-01T19:34:42.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.766+0000 7fc4857fa700 1 -- 172.21.15.17:0/787463592 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc47c012760 con 0x7fc4880ad750 2022-08-01T19:34:42.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.766+0000 7fc48d0cb700 1 -- 172.21.15.17:0/787463592 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc47800b180 con 0x7fc4880ad750 2022-08-01T19:34:42.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.766+0000 7fc4857fa700 1 -- 172.21.15.17:0/787463592 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc47c012ac0 con 0x7fc4880ad750 2022-08-01T19:34:42.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.766+0000 7fc48d0cb700 1 -- 172.21.15.17:0/787463592 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4880aa790 con 0x7fc4880ad750 2022-08-01T19:34:42.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.770+0000 7fc48d0cb700 1 -- 172.21.15.17:0/787463592 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4880aa950 con 0x7fc4880ad750 2022-08-01T19:34:42.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.770+0000 7fc4857fa700 1 -- 172.21.15.17:0/787463592 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc47c022070 con 0x7fc4880ad750 2022-08-01T19:34:42.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.770+0000 7fc4857fa700 1 --2- 172.21.15.17:0/787463592 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc468053980 0x7fc468055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:42.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.770+0000 7fc4857fa700 1 -- 172.21.15.17:0/787463592 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fc47c06c880 con 0x7fc4880ad750 2022-08-01T19:34:42.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.770+0000 7fc487fff700 1 --2- 172.21.15.17:0/787463592 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc468053980 0x7fc468055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:42.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.774+0000 7fc487fff700 1 --2- 172.21.15.17:0/787463592 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc468053980 0x7fc468055e40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fc48000b000 tx=0x7fc4800091d0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:42.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.774+0000 7fc4857fa700 1 -- 172.21.15.17:0/787463592 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc47c043870 con 0x7fc4880ad750 2022-08-01T19:34:42.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.946+0000 7fc48d0cb700 1 -- 172.21.15.17:0/787463592 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fc4880574e0 con 0x7fc4880ad750 2022-08-01T19:34:42.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.954+0000 7fc4857fa700 1 -- 172.21.15.17:0/787463592 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 updated v12) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc47c046e90 con 0x7fc4880ad750 2022-08-01T19:34:42.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.954+0000 7fc48d0cb700 1 -- 172.21.15.17:0/787463592 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc468053980 msgr2=0x7fc468055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:42.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.954+0000 7fc48d0cb700 1 --2- 172.21.15.17:0/787463592 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc468053980 0x7fc468055e40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fc48000b000 tx=0x7fc4800091d0).stop 2022-08-01T19:34:42.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.954+0000 7fc48d0cb700 1 -- 172.21.15.17:0/787463592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4880ad750 msgr2=0x7fc48813f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:42.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.954+0000 7fc48d0cb700 1 --2- 172.21.15.17:0/787463592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4880ad750 0x7fc48813f920 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7fc47c00c000 tx=0x7fc47c006eb0).stop 2022-08-01T19:34:42.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.954+0000 7fc48d0cb700 1 -- 172.21.15.17:0/787463592 shutdown_connections 2022-08-01T19:34:42.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.954+0000 7fc48d0cb700 1 --2- 172.21.15.17:0/787463592 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc468053980 0x7fc468055e40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:42.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.954+0000 7fc48d0cb700 1 --2- 172.21.15.17:0/787463592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4881405d0 0x7fc48813fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:42.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.954+0000 7fc48d0cb700 1 --2- 172.21.15.17:0/787463592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4880ad750 0x7fc48813f920 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:42.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.954+0000 7fc48d0cb700 1 --2- 172.21.15.17:0/787463592 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4880614b0 0x7fc48805d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:42.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.954+0000 7fc48d0cb700 1 -- 172.21.15.17:0/787463592 >> 172.21.15.17:0/787463592 conn(0x7fc488066f70 msgr2=0x7fc4881467e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:42.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.958+0000 7fc48d0cb700 1 -- 172.21.15.17:0/787463592 shutdown_connections 2022-08-01T19:34:42.966 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:42.958+0000 7fc48d0cb700 1 -- 172.21.15.17:0/787463592 wait complete. 2022-08-01T19:34:42.966 INFO:teuthology.orchestra.run.smithi017.stderr:updated 2022-08-01T19:34:42.973 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2022-08-01T19:34:43.079 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2192efd700 1 Processor -- start 2022-08-01T19:34:43.080 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2192efd700 1 -- start start 2022-08-01T19:34:43.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2192efd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f218c05c960 0x7f218c0b1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:43.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2192efd700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f218c05cd40 con 0x7f218c05d240 2022-08-01T19:34:43.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2192efd700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f218c0b1920 con 0x7f218c13f520 2022-08-01T19:34:43.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2192efd700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f218c0ad6d0 con 0x7f218c05c960 2022-08-01T19:34:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f21916fa700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f218c05d240 0x7f218c0acfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57896/0 (socket says 172.21.15.17:57896) 2022-08-01T19:34:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f21916fa700 1 -- 172.21.15.17:0/3925308979 learned_addr learned my addr 172.21.15.17:0/3925308979 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2191efb700 1 --2- 172.21.15.17:0/3925308979 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f218c05c960 0x7f218c0b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2190ef9700 1 -- 172.21.15.17:0/3925308979 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1150920972 0 0) 0x7f218c05cd40 con 0x7f218c05d240 2022-08-01T19:34:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2190ef9700 1 -- 172.21.15.17:0/3925308979 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2174003260 con 0x7f218c05d240 2022-08-01T19:34:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2190ef9700 1 -- 172.21.15.17:0/3925308979 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 585883869 0 0) 0x7f218c0b1920 con 0x7f218c13f520 2022-08-01T19:34:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2190ef9700 1 -- 172.21.15.17:0/3925308979 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f218c05cd40 con 0x7f218c13f520 2022-08-01T19:34:43.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2190ef9700 1 -- 172.21.15.17:0/3925308979 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1817028635 0 0) 0x7f2174003260 con 0x7f218c05d240 2022-08-01T19:34:43.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2190ef9700 1 -- 172.21.15.17:0/3925308979 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f218c05c960 msgr2=0x7f218c0b1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:43.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2190ef9700 1 --2- 172.21.15.17:0/3925308979 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f218c05c960 0x7f218c0b1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:43.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2190ef9700 1 -- 172.21.15.17:0/3925308979 >> v1:172.21.15.77:6789/0 conn(0x7f218c13f520 legacy=0x7f218c0aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:43.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2190ef9700 1 -- 172.21.15.17:0/3925308979 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f218c0b1bc0 con 0x7f218c05d240 2022-08-01T19:34:43.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2190ef9700 1 -- 172.21.15.17:0/3925308979 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7f217c001680 con 0x7f218c05d240 2022-08-01T19:34:43.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2190ef9700 1 -- 172.21.15.17:0/3925308979 >> v1:172.21.15.17:6789/0 conn(0x7f218c05d240 legacy=0x7f218c0acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:43.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2190ef9700 1 --2- 172.21.15.17:0/3925308979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2174004970 0x7f2174004d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:43.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2190ef9700 1 --2- 172.21.15.17:0/3925308979 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f218c05c960 0x7f21740034a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:43.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2190ef9700 1 --2- 172.21.15.17:0/3925308979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2174003990 0x7f2174003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:43.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2190ef9700 1 -- 172.21.15.17:0/3925308979 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218c0ad6d0 con 0x7f2174003990 2022-08-01T19:34:43.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2190ef9700 1 -- 172.21.15.17:0/3925308979 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217400b780 con 0x7f2174004970 2022-08-01T19:34:43.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2190ef9700 1 -- 172.21.15.17:0/3925308979 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217400b940 con 0x7f218c05c960 2022-08-01T19:34:43.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f21926fc700 1 --2- 172.21.15.17:0/3925308979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2174003990 0x7f2174003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:43.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2191efb700 1 --2- 172.21.15.17:0/3925308979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2174004970 0x7f2174004d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:43.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f21916fa700 1 --2- 172.21.15.17:0/3925308979 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f218c05c960 0x7f21740034a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:43.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f21926fc700 1 -- 172.21.15.17:0/3925308979 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f218c05c960 msgr2=0x7f21740034a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:43.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f21926fc700 1 --2- 172.21.15.17:0/3925308979 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f218c05c960 0x7f21740034a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:43.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f21926fc700 1 -- 172.21.15.17:0/3925308979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2174004970 msgr2=0x7f2174004d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:43.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f21926fc700 1 --2- 172.21.15.17:0/3925308979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2174004970 0x7f2174004d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:43.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f21926fc700 1 -- 172.21.15.17:0/3925308979 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2174003260 con 0x7f2174003990 2022-08-01T19:34:43.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2191efb700 1 --2- 172.21.15.17:0/3925308979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2174004970 0x7f2174004d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:34:43.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f21926fc700 1 --2- 172.21.15.17:0/3925308979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2174003990 0x7f2174003e10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f2184001490 tx=0x7f2184007d80).ready entity=mon.0 client_cookie=306f9bd61d0242d7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:43.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2190ef9700 1 -- 172.21.15.17:0/3925308979 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2184001780 con 0x7f2174003990 2022-08-01T19:34:43.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2190ef9700 1 -- 172.21.15.17:0/3925308979 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2184006890 con 0x7f2174003990 2022-08-01T19:34:43.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2190ef9700 1 -- 172.21.15.17:0/3925308979 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2184006bf0 con 0x7f2174003990 2022-08-01T19:34:43.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2192efd700 1 -- 172.21.15.17:0/3925308979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2174003990 msgr2=0x7f2174003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:43.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2192efd700 1 --2- 172.21.15.17:0/3925308979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2174003990 0x7f2174003e10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f2184001490 tx=0x7f2184007d80).stop 2022-08-01T19:34:43.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f21926fc700 1 -- 172.21.15.17:0/3925308979 reap_dead start 2022-08-01T19:34:43.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2192efd700 1 -- 172.21.15.17:0/3925308979 shutdown_connections 2022-08-01T19:34:43.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2192efd700 1 -- 172.21.15.17:0/3925308979 >> 172.21.15.17:0/3925308979 conn(0x7f218c060310 msgr2=0x7f218c0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:43.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2192efd700 1 -- 172.21.15.17:0/3925308979 shutdown_connections 2022-08-01T19:34:43.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2192efd700 1 -- 172.21.15.17:0/3925308979 wait complete. 2022-08-01T19:34:43.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.078+0000 7f2192efd700 1 Processor -- start 2022-08-01T19:34:43.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.082+0000 7f2192efd700 1 -- start start 2022-08-01T19:34:43.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.082+0000 7f2192efd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2174003990 0x7f218c13ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:43.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.082+0000 7f2192efd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f218c0ab510 0x7f218c0ab990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:43.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.082+0000 7f2192efd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f218c1537d0 0x7f218c153c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:43.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.082+0000 7f2192efd700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218c0abfb0 con 0x7f218c1537d0 2022-08-01T19:34:43.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.082+0000 7f2192efd700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218c066810 con 0x7f2174003990 2022-08-01T19:34:43.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.082+0000 7f2192efd700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f218c066b30 con 0x7f218c0ab510 2022-08-01T19:34:43.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.082+0000 7f21926fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f218c1537d0 0x7f218c153c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:43.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.082+0000 7f2191efb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2174003990 0x7f218c13ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:43.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.082+0000 7f2191efb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2174003990 0x7f218c13ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:35404/0 (socket says 172.21.15.17:35404) 2022-08-01T19:34:43.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.082+0000 7f2191efb700 1 -- 172.21.15.17:0/3295118553 learned_addr learned my addr 172.21.15.17:0/3295118553 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:43.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.082+0000 7f21916fa700 1 --2- 172.21.15.17:0/3295118553 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f218c0ab510 0x7f218c0ab990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:43.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.082+0000 7f21926fc700 1 -- 172.21.15.17:0/3295118553 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f218c0ab510 msgr2=0x7f218c0ab990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:43.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.082+0000 7f21926fc700 1 --2- 172.21.15.17:0/3295118553 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f218c0ab510 0x7f218c0ab990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:43.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.082+0000 7f21926fc700 1 -- 172.21.15.17:0/3295118553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2174003990 msgr2=0x7f218c13ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:43.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.082+0000 7f21926fc700 1 --2- 172.21.15.17:0/3295118553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2174003990 0x7f218c13ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:43.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.082+0000 7f21926fc700 1 -- 172.21.15.17:0/3295118553 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f218c066dd0 con 0x7f218c1537d0 2022-08-01T19:34:43.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.082+0000 7f21926fc700 1 --2- 172.21.15.17:0/3295118553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f218c1537d0 0x7f218c153c50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f218c061eb0 tx=0x7f2184016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:43.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.082+0000 7f217affd700 1 -- 172.21.15.17:0/3295118553 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2184001780 con 0x7f218c1537d0 2022-08-01T19:34:43.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.082+0000 7f217affd700 1 -- 172.21.15.17:0/3295118553 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2184007860 con 0x7f218c1537d0 2022-08-01T19:34:43.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.082+0000 7f2192efd700 1 -- 172.21.15.17:0/3295118553 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f218c067070 con 0x7f218c1537d0 2022-08-01T19:34:43.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.082+0000 7f217affd700 1 -- 172.21.15.17:0/3295118553 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f218400cc00 con 0x7f218c1537d0 2022-08-01T19:34:43.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.082+0000 7f2192efd700 1 -- 172.21.15.17:0/3295118553 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f218c067510 con 0x7f218c1537d0 2022-08-01T19:34:43.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.086+0000 7f217affd700 1 -- 172.21.15.17:0/3295118553 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2184025030 con 0x7f218c1537d0 2022-08-01T19:34:43.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.086+0000 7f2192efd700 1 -- 172.21.15.17:0/3295118553 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f218c0574e0 con 0x7f218c1537d0 2022-08-01T19:34:43.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.086+0000 7f217affd700 1 --2- 172.21.15.17:0/3295118553 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f216c053b30 0x7f216c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:43.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.086+0000 7f217affd700 1 -- 172.21.15.17:0/3295118553 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f218406ccb0 con 0x7f218c1537d0 2022-08-01T19:34:43.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.086+0000 7f2191efb700 1 --2- 172.21.15.17:0/3295118553 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f216c053b30 0x7f216c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:43.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.086+0000 7f2191efb700 1 --2- 172.21.15.17:0/3295118553 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f216c053b30 0x7f216c055ff0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f218800b000 tx=0x7f2188006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:43.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.090+0000 7f217affd700 1 -- 172.21.15.17:0/3295118553 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2184043db0 con 0x7f218c1537d0 2022-08-01T19:34:43.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.258+0000 7f2192efd700 1 -- 172.21.15.17:0/3295118553 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f218c0a8ae0 con 0x7f218c1537d0 2022-08-01T19:34:43.271 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.266+0000 7f217affd700 1 -- 172.21.15.17:0/3295118553 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 updated v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2184023020 con 0x7f218c1537d0 2022-08-01T19:34:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.266+0000 7f2192efd700 1 -- 172.21.15.17:0/3295118553 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f216c053b30 msgr2=0x7f216c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.266+0000 7f2192efd700 1 --2- 172.21.15.17:0/3295118553 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f216c053b30 0x7f216c055ff0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f218800b000 tx=0x7f2188006df0).stop 2022-08-01T19:34:43.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.266+0000 7f2192efd700 1 -- 172.21.15.17:0/3295118553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f218c1537d0 msgr2=0x7f218c153c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:43.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.266+0000 7f2192efd700 1 --2- 172.21.15.17:0/3295118553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f218c1537d0 0x7f218c153c50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f218c061eb0 tx=0x7f2184016000).stop 2022-08-01T19:34:43.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.270+0000 7f2192efd700 1 -- 172.21.15.17:0/3295118553 shutdown_connections 2022-08-01T19:34:43.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.270+0000 7f2192efd700 1 --2- 172.21.15.17:0/3295118553 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f216c053b30 0x7f216c055ff0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:43.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.270+0000 7f2192efd700 1 --2- 172.21.15.17:0/3295118553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f218c1537d0 0x7f218c153c50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:43.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.270+0000 7f2192efd700 1 --2- 172.21.15.17:0/3295118553 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f218c0ab510 0x7f218c0ab990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:43.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.270+0000 7f2192efd700 1 --2- 172.21.15.17:0/3295118553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2174003990 0x7f218c13ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:43.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.270+0000 7f2192efd700 1 -- 172.21.15.17:0/3295118553 >> 172.21.15.17:0/3295118553 conn(0x7f218c060310 msgr2=0x7f218c13f930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:43.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.270+0000 7f2192efd700 1 -- 172.21.15.17:0/3295118553 shutdown_connections 2022-08-01T19:34:43.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.270+0000 7f2192efd700 1 -- 172.21.15.17:0/3295118553 wait complete. 2022-08-01T19:34:43.279 INFO:teuthology.orchestra.run.smithi017.stderr:updated 2022-08-01T19:34:43.286 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2022-08-01T19:34:43.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.382+0000 7f12ba658700 1 Processor -- start 2022-08-01T19:34:43.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.382+0000 7f12ba658700 1 -- start start 2022-08-01T19:34:43.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.382+0000 7f12ba658700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f12b40b0630 0x7f12b40adec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:43.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.382+0000 7f12ba658700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f12b40aa2b0 0x7f12b40ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:43.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.382+0000 7f12ba658700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12b40b2c80 con 0x7f12b40b0a10 2022-08-01T19:34:43.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.382+0000 7f12ba658700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12b40b2900 con 0x7f12b40aa2b0 2022-08-01T19:34:43.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.382+0000 7f12ba658700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b40b2eb0 con 0x7f12b40b0630 2022-08-01T19:34:43.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.382+0000 7f12b37fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f12b40aa2b0 0x7f12b40ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:43.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.382+0000 7f12b37fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f12b40aa2b0 0x7f12b40ae3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37492/0 (socket says 172.21.15.17:37492) 2022-08-01T19:34:43.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.382+0000 7f12b37fe700 1 -- 172.21.15.17:0/3565432622 learned_addr learned my addr 172.21.15.17:0/3565432622 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:43.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.382+0000 7f12b3fff700 1 --2- 172.21.15.17:0/3565432622 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f12b40b0630 0x7f12b40adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:43.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.386+0000 7f12b37fe700 1 -- 172.21.15.17:0/3565432622 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f12b40b0630 msgr2=0x7f12b40adec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:43.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.386+0000 7f12b37fe700 1 --2- 172.21.15.17:0/3565432622 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f12b40b0630 0x7f12b40adec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:43.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.386+0000 7f12b37fe700 1 -- 172.21.15.17:0/3565432622 >> v1:172.21.15.77:6789/0 conn(0x7f12b40b0a10 legacy=0x7f12b41438a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:43.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.386+0000 7f12b37fe700 1 -- 172.21.15.17:0/3565432622 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b40a9d00 con 0x7f12b40aa2b0 2022-08-01T19:34:43.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.386+0000 7f12b37fe700 1 --2- 172.21.15.17:0/3565432622 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12b40aa2b0 0x7f12b40ae3b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f129c002f70 tx=0x7f129c006eb0).ready entity=mon.0 client_cookie=a5c3d467204ed1ff server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:43.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.386+0000 7f12b2ffd700 1 -- 172.21.15.17:0/3565432622 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f129c00d570 con 0x7f12b40aa2b0 2022-08-01T19:34:43.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.386+0000 7f12b2ffd700 1 -- 172.21.15.17:0/3565432622 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129c000e90 con 0x7f12b40aa2b0 2022-08-01T19:34:43.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.386+0000 7f12b2ffd700 1 -- 172.21.15.17:0/3565432622 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f129c00d900 con 0x7f12b40aa2b0 2022-08-01T19:34:43.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.386+0000 7f12ba658700 1 -- 172.21.15.17:0/3565432622 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12b40aa2b0 msgr2=0x7f12b40ae3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:43.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.386+0000 7f12ba658700 1 --2- 172.21.15.17:0/3565432622 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12b40aa2b0 0x7f12b40ae3b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f129c002f70 tx=0x7f129c006eb0).stop 2022-08-01T19:34:43.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.386+0000 7f12ba658700 1 -- 172.21.15.17:0/3565432622 shutdown_connections 2022-08-01T19:34:43.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.386+0000 7f12ba658700 1 --2- 172.21.15.17:0/3565432622 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12b40aa2b0 0x7f12b40ae3b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:43.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.386+0000 7f12ba658700 1 --2- 172.21.15.17:0/3565432622 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f12b40b0630 0x7f12b40adec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:43.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.386+0000 7f12ba658700 1 -- 172.21.15.17:0/3565432622 >> 172.21.15.17:0/3565432622 conn(0x7f12b4060950 msgr2=0x7f12b4060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:43.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.386+0000 7f12ba658700 1 -- 172.21.15.17:0/3565432622 shutdown_connections 2022-08-01T19:34:43.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.386+0000 7f12ba658700 1 -- 172.21.15.17:0/3565432622 wait complete. 2022-08-01T19:34:43.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.386+0000 7f12ba658700 1 Processor -- start 2022-08-01T19:34:43.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.386+0000 7f12ba658700 1 -- start start 2022-08-01T19:34:43.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.390+0000 7f12ba658700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12b40aa2b0 0x7f12b4145cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:43.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.390+0000 7f12ba658700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12b40b0630 0x7f12b413cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:43.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.390+0000 7f12ba658700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f12b40b0a10 0x7f12b4153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:43.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.390+0000 7f12ba658700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b413d4e0 con 0x7f12b40aa2b0 2022-08-01T19:34:43.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.390+0000 7f12ba658700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b413d770 con 0x7f12b40b0630 2022-08-01T19:34:43.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.390+0000 7f12ba658700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b41433b0 con 0x7f12b40b0a10 2022-08-01T19:34:43.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.390+0000 7f12b3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12b40aa2b0 0x7f12b4145cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:43.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.390+0000 7f12b3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12b40aa2b0 0x7f12b4145cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37496/0 (socket says 172.21.15.17:37496) 2022-08-01T19:34:43.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.390+0000 7f12b3fff700 1 -- 172.21.15.17:0/1812854483 learned_addr learned my addr 172.21.15.17:0/1812854483 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:43.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.390+0000 7f12b37fe700 1 --2- 172.21.15.17:0/1812854483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12b40b0630 0x7f12b413cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:43.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.390+0000 7f12b8bf5700 1 --2- 172.21.15.17:0/1812854483 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f12b40b0a10 0x7f12b4153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:43.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.390+0000 7f12b3fff700 1 -- 172.21.15.17:0/1812854483 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f12b40b0a10 msgr2=0x7f12b4153c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:43.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.390+0000 7f12b3fff700 1 --2- 172.21.15.17:0/1812854483 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f12b40b0a10 0x7f12b4153c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:43.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.390+0000 7f12b3fff700 1 -- 172.21.15.17:0/1812854483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12b40b0630 msgr2=0x7f12b413cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:43.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.390+0000 7f12b3fff700 1 --2- 172.21.15.17:0/1812854483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12b40b0630 0x7f12b413cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:43.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.390+0000 7f12b3fff700 1 -- 172.21.15.17:0/1812854483 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f129c00b1e0 con 0x7f12b40aa2b0 2022-08-01T19:34:43.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.390+0000 7f12b37fe700 1 --2- 172.21.15.17:0/1812854483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12b40b0630 0x7f12b413cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:34:43.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.390+0000 7f12b3fff700 1 --2- 172.21.15.17:0/1812854483 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12b40aa2b0 0x7f12b4145cd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f12a400c000 tx=0x7f12a4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:43.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.390+0000 7f12b17fa700 1 -- 172.21.15.17:0/1812854483 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12a4010780 con 0x7f12b40aa2b0 2022-08-01T19:34:43.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.390+0000 7f12b17fa700 1 -- 172.21.15.17:0/1812854483 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a40108e0 con 0x7f12b40aa2b0 2022-08-01T19:34:43.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.390+0000 7f12ba658700 1 -- 172.21.15.17:0/1812854483 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12b4143650 con 0x7f12b40aa2b0 2022-08-01T19:34:43.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.390+0000 7f12b17fa700 1 -- 172.21.15.17:0/1812854483 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12a4010c40 con 0x7f12b40aa2b0 2022-08-01T19:34:43.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.390+0000 7f12ba658700 1 -- 172.21.15.17:0/1812854483 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b4143d90 con 0x7f12b40aa2b0 2022-08-01T19:34:43.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.390+0000 7f12ba658700 1 -- 172.21.15.17:0/1812854483 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12b40574e0 con 0x7f12b40aa2b0 2022-08-01T19:34:43.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.390+0000 7f12b17fa700 1 -- 172.21.15.17:0/1812854483 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f12a401c460 con 0x7f12b40aa2b0 2022-08-01T19:34:43.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.394+0000 7f12b17fa700 1 --2- 172.21.15.17:0/1812854483 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1298053aa0 0x7f1298055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:43.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.394+0000 7f12b17fa700 1 -- 172.21.15.17:0/1812854483 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f12a406c240 con 0x7f12b40aa2b0 2022-08-01T19:34:43.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.394+0000 7f12b37fe700 1 --2- 172.21.15.17:0/1812854483 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1298053aa0 0x7f1298055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:43.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.394+0000 7f12b37fe700 1 --2- 172.21.15.17:0/1812854483 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1298053aa0 0x7f1298055f60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f129c01a000 tx=0x7f129c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:43.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.398+0000 7f12b17fa700 1 -- 172.21.15.17:0/1812854483 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f12a40432e0 con 0x7f12b40aa2b0 2022-08-01T19:34:43.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.566+0000 7f12ba658700 1 -- 172.21.15.17:0/1812854483 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f12b40447e0 con 0x7f12b40aa2b0 2022-08-01T19:34:43.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.574+0000 7f12b17fa700 1 -- 172.21.15.17:0/1812854483 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 updated v14) v1 ==== 97+0+0 (secure 0 0 0) 0x7f12a4046900 con 0x7f12b40aa2b0 2022-08-01T19:34:43.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.574+0000 7f12ba658700 1 -- 172.21.15.17:0/1812854483 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1298053aa0 msgr2=0x7f1298055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:43.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.574+0000 7f12ba658700 1 --2- 172.21.15.17:0/1812854483 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1298053aa0 0x7f1298055f60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f129c01a000 tx=0x7f129c015000).stop 2022-08-01T19:34:43.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.574+0000 7f12ba658700 1 -- 172.21.15.17:0/1812854483 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12b40aa2b0 msgr2=0x7f12b4145cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:43.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.574+0000 7f12ba658700 1 --2- 172.21.15.17:0/1812854483 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12b40aa2b0 0x7f12b4145cd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f12a400c000 tx=0x7f12a4006df0).stop 2022-08-01T19:34:43.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.574+0000 7f12ba658700 1 -- 172.21.15.17:0/1812854483 shutdown_connections 2022-08-01T19:34:43.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.574+0000 7f12ba658700 1 --2- 172.21.15.17:0/1812854483 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1298053aa0 0x7f1298055f60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:43.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.574+0000 7f12ba658700 1 --2- 172.21.15.17:0/1812854483 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f12b40b0a10 0x7f12b4153c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:43.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.574+0000 7f12ba658700 1 --2- 172.21.15.17:0/1812854483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12b40b0630 0x7f12b413cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:43.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.574+0000 7f12ba658700 1 --2- 172.21.15.17:0/1812854483 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12b40aa2b0 0x7f12b4145cd0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:43.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.574+0000 7f12ba658700 1 -- 172.21.15.17:0/1812854483 >> 172.21.15.17:0/1812854483 conn(0x7f12b4060950 msgr2=0x7f12b40b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:43.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.574+0000 7f12ba658700 1 -- 172.21.15.17:0/1812854483 shutdown_connections 2022-08-01T19:34:43.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.574+0000 7f12ba658700 1 -- 172.21.15.17:0/1812854483 wait complete. 2022-08-01T19:34:43.584 INFO:teuthology.orchestra.run.smithi017.stderr:updated 2022-08-01T19:34:43.593 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:34:43.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.690+0000 7f85009e2700 1 Processor -- start 2022-08-01T19:34:43.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.690+0000 7f85009e2700 1 -- start start 2022-08-01T19:34:43.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.690+0000 7f85009e2700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f84fc064b10 0x7f84fc144e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:43.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.690+0000 7f85009e2700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84fc0b36e0 0x7f84fc141370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:43.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.690+0000 7f85009e2700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f84fc141860 0x7f84fc066ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:43.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.690+0000 7f85009e2700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84fc13ebf0 con 0x7f84fc141860 2022-08-01T19:34:43.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.690+0000 7f85009e2700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84fc064ef0 con 0x7f84fc064b10 2022-08-01T19:34:43.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.690+0000 7f85009e2700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc066690 con 0x7f84fc0b36e0 2022-08-01T19:34:43.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.694+0000 7f84fbfff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f84fc141860 0x7f84fc066ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:43.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.694+0000 7f84fbfff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f84fc141860 0x7f84fc066ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37506/0 (socket says 172.21.15.17:37506) 2022-08-01T19:34:43.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.694+0000 7f84fbfff700 1 -- 172.21.15.17:0/1792398489 learned_addr learned my addr 172.21.15.17:0/1792398489 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:43.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.694+0000 7f84fb7fe700 1 --2- 172.21.15.17:0/1792398489 >> v2:172.21.15.77:3300/0 conn(0x7f84fc064b10 0x7f84fc144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:43.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.694+0000 7f84faffd700 1 --2- 172.21.15.17:0/1792398489 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84fc0b36e0 0x7f84fc141370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:43.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.694+0000 7f84fbfff700 1 -- 172.21.15.17:0/1792398489 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84fc0b36e0 msgr2=0x7f84fc141370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:43.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.694+0000 7f84fbfff700 1 --2- 172.21.15.17:0/1792398489 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84fc0b36e0 0x7f84fc141370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:43.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.694+0000 7f84fbfff700 1 -- 172.21.15.17:0/1792398489 >> v2:172.21.15.77:3300/0 conn(0x7f84fc064b10 msgr2=0x7f84fc144e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:43.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.694+0000 7f84fbfff700 1 --2- 172.21.15.17:0/1792398489 >> v2:172.21.15.77:3300/0 conn(0x7f84fc064b10 0x7f84fc144e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:43.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.694+0000 7f84fbfff700 1 -- 172.21.15.17:0/1792398489 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84fc0b4850 con 0x7f84fc141860 2022-08-01T19:34:43.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.694+0000 7f84fb7fe700 1 --2- 172.21.15.17:0/1792398489 >> v2:172.21.15.77:3300/0 conn(0x7f84fc064b10 0x7f84fc144e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:34:43.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.694+0000 7f84fbfff700 1 --2- 172.21.15.17:0/1792398489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84fc141860 0x7f84fc066ac0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f84f40032f0 tx=0x7f84f4009290).ready entity=mon.0 client_cookie=434687a089cba180 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:43.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.694+0000 7f84fa7fc700 1 -- 172.21.15.17:0/1792398489 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84f400b870 con 0x7f84fc141860 2022-08-01T19:34:43.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.694+0000 7f84fa7fc700 1 -- 172.21.15.17:0/1792398489 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f400b9d0 con 0x7f84fc141860 2022-08-01T19:34:43.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.694+0000 7f84fa7fc700 1 -- 172.21.15.17:0/1792398489 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84f400bb40 con 0x7f84fc141860 2022-08-01T19:34:43.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.694+0000 7f85009e2700 1 -- 172.21.15.17:0/1792398489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84fc141860 msgr2=0x7f84fc066ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:43.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.694+0000 7f85009e2700 1 --2- 172.21.15.17:0/1792398489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84fc141860 0x7f84fc066ac0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f84f40032f0 tx=0x7f84f4009290).stop 2022-08-01T19:34:43.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.694+0000 7f85009e2700 1 -- 172.21.15.17:0/1792398489 shutdown_connections 2022-08-01T19:34:43.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.694+0000 7f85009e2700 1 --2- 172.21.15.17:0/1792398489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84fc141860 0x7f84fc066ac0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:43.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.694+0000 7f85009e2700 1 --2- 172.21.15.17:0/1792398489 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84fc0b36e0 0x7f84fc141370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:43.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.694+0000 7f85009e2700 1 --2- 172.21.15.17:0/1792398489 >> v2:172.21.15.77:3300/0 conn(0x7f84fc064b10 0x7f84fc144e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:43.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.694+0000 7f85009e2700 1 -- 172.21.15.17:0/1792398489 >> 172.21.15.17:0/1792398489 conn(0x7f84fc05f840 msgr2=0x7f84fc05fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:43.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.694+0000 7f85009e2700 1 -- 172.21.15.17:0/1792398489 shutdown_connections 2022-08-01T19:34:43.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.694+0000 7f85009e2700 1 -- 172.21.15.17:0/1792398489 wait complete. 2022-08-01T19:34:43.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.698+0000 7f85009e2700 1 Processor -- start 2022-08-01T19:34:43.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.698+0000 7f85009e2700 1 -- start start 2022-08-01T19:34:43.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.698+0000 7f85009e2700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84fc064b10 0x7f84fc13c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:43.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.698+0000 7f85009e2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84fc0b36e0 0x7f84fc13c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:43.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.698+0000 7f85009e2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84fc141860 0x7f84fc0b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:43.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.698+0000 7f85009e2700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc13ce70 con 0x7f84fc0b36e0 2022-08-01T19:34:43.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.698+0000 7f85009e2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc1511a0 con 0x7f84fc141860 2022-08-01T19:34:43.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.698+0000 7f85009e2700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc142450 con 0x7f84fc064b10 2022-08-01T19:34:43.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.698+0000 7f84faffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84fc0b36e0 0x7f84fc13c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:43.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.698+0000 7f84faffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84fc0b36e0 0x7f84fc13c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37512/0 (socket says 172.21.15.17:37512) 2022-08-01T19:34:43.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.698+0000 7f84faffd700 1 -- 172.21.15.17:0/1059779888 learned_addr learned my addr 172.21.15.17:0/1059779888 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:43.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.698+0000 7f84fb7fe700 1 --2- 172.21.15.17:0/1059779888 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84fc064b10 0x7f84fc13c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:43.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.698+0000 7f84fbfff700 1 --2- 172.21.15.17:0/1059779888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84fc141860 0x7f84fc0b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:43.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.698+0000 7f84faffd700 1 -- 172.21.15.17:0/1059779888 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84fc064b10 msgr2=0x7f84fc13c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:43.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.698+0000 7f84faffd700 1 --2- 172.21.15.17:0/1059779888 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84fc064b10 0x7f84fc13c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:43.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.698+0000 7f84faffd700 1 -- 172.21.15.17:0/1059779888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84fc141860 msgr2=0x7f84fc0b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:43.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.698+0000 7f84faffd700 1 --2- 172.21.15.17:0/1059779888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84fc141860 0x7f84fc0b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:43.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.698+0000 7f84faffd700 1 -- 172.21.15.17:0/1059779888 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f4009530 con 0x7f84fc0b36e0 2022-08-01T19:34:43.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.698+0000 7f84fbfff700 1 --2- 172.21.15.17:0/1059779888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84fc141860 0x7f84fc0b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:34:43.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.698+0000 7f84faffd700 1 --2- 172.21.15.17:0/1059779888 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84fc0b36e0 0x7f84fc13c980 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f84f000c000 tx=0x7f84f0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:43.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.698+0000 7f84f8ff9700 1 -- 172.21.15.17:0/1059779888 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84f0010780 con 0x7f84fc0b36e0 2022-08-01T19:34:43.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.698+0000 7f84f8ff9700 1 -- 172.21.15.17:0/1059779888 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f00108e0 con 0x7f84fc0b36e0 2022-08-01T19:34:43.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.698+0000 7f85009e2700 1 -- 172.21.15.17:0/1059779888 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84fc150e90 con 0x7f84fc0b36e0 2022-08-01T19:34:43.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.698+0000 7f84f8ff9700 1 -- 172.21.15.17:0/1059779888 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84f0010c40 con 0x7f84fc0b36e0 2022-08-01T19:34:43.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.698+0000 7f85009e2700 1 -- 172.21.15.17:0/1059779888 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84fc0af040 con 0x7f84fc0b36e0 2022-08-01T19:34:43.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.702+0000 7f85009e2700 1 -- 172.21.15.17:0/1059779888 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84fc151360 con 0x7f84fc0b36e0 2022-08-01T19:34:43.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.702+0000 7f84f8ff9700 1 -- 172.21.15.17:0/1059779888 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f84f001c460 con 0x7f84fc0b36e0 2022-08-01T19:34:43.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.702+0000 7f84f8ff9700 1 --2- 172.21.15.17:0/1059779888 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f84dc053aa0 0x7f84dc055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:43.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.702+0000 7f84f8ff9700 1 -- 172.21.15.17:0/1059779888 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f84f006c130 con 0x7f84fc0b36e0 2022-08-01T19:34:43.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.702+0000 7f84fb7fe700 1 --2- 172.21.15.17:0/1059779888 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f84dc053aa0 0x7f84dc055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:43.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.702+0000 7f84fb7fe700 1 --2- 172.21.15.17:0/1059779888 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f84dc053aa0 0x7f84dc055f60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f84ec00c000 tx=0x7f84ec008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:43.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.710+0000 7f84f8ff9700 1 -- 172.21.15.17:0/1059779888 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f84f0043130 con 0x7f84fc0b36e0 2022-08-01T19:34:43.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.838+0000 7f85009e2700 1 -- 172.21.15.17:0/1059779888 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84fc0b4af0 con 0x7f84fc0b36e0 2022-08-01T19:34:43.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.838+0000 7f84f8ff9700 1 -- 172.21.15.17:0/1059779888 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+13414 (secure 0 0 0) 0x7f84f0046750 con 0x7f84fc0b36e0 2022-08-01T19:34:43.845 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:43.845 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":30,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:34:41.332169+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:34:43.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.842+0000 7f85009e2700 1 -- 172.21.15.17:0/1059779888 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f84dc053aa0 msgr2=0x7f84dc055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:43.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.842+0000 7f85009e2700 1 --2- 172.21.15.17:0/1059779888 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f84dc053aa0 0x7f84dc055f60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f84ec00c000 tx=0x7f84ec008040).stop 2022-08-01T19:34:43.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.842+0000 7f85009e2700 1 -- 172.21.15.17:0/1059779888 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84fc0b36e0 msgr2=0x7f84fc13c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:43.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.842+0000 7f85009e2700 1 --2- 172.21.15.17:0/1059779888 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84fc0b36e0 0x7f84fc13c980 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f84f000c000 tx=0x7f84f0006df0).stop 2022-08-01T19:34:43.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.842+0000 7f85009e2700 1 -- 172.21.15.17:0/1059779888 shutdown_connections 2022-08-01T19:34:43.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.842+0000 7f85009e2700 1 --2- 172.21.15.17:0/1059779888 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f84dc053aa0 0x7f84dc055f60 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:43.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.842+0000 7f85009e2700 1 --2- 172.21.15.17:0/1059779888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84fc141860 0x7f84fc0b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:43.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.842+0000 7f85009e2700 1 --2- 172.21.15.17:0/1059779888 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84fc0b36e0 0x7f84fc13c980 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:43.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.842+0000 7f85009e2700 1 --2- 172.21.15.17:0/1059779888 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84fc064b10 0x7f84fc13c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:43.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.842+0000 7f85009e2700 1 -- 172.21.15.17:0/1059779888 >> 172.21.15.17:0/1059779888 conn(0x7f84fc05f840 msgr2=0x7f84fc0b5190 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:43.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.842+0000 7f85009e2700 1 -- 172.21.15.17:0/1059779888 shutdown_connections 2022-08-01T19:34:43.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.842+0000 7f85009e2700 1 -- 172.21.15.17:0/1059779888 wait complete. 2022-08-01T19:34:43.858 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-08-01T19:34:43.859 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:34:43.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.958+0000 7f79b9554700 1 Processor -- start 2022-08-01T19:34:43.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.958+0000 7f79b9554700 1 -- start start 2022-08-01T19:34:43.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.958+0000 7f79b9554700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79b40a8c50 0x7f79b4152d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:43.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b9554700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b40a7250 con 0x7f79b40b2710 2022-08-01T19:34:43.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b9554700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b40a7480 con 0x7f79b40b01a0 2022-08-01T19:34:43.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b9554700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b40a7720 con 0x7f79b40a8c50 2022-08-01T19:34:43.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b3fff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f79b40b2710 0x7f79b414afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57944/0 (socket says 172.21.15.17:57944) 2022-08-01T19:34:43.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b3fff700 1 -- 172.21.15.17:0/3529569145 learned_addr learned my addr 172.21.15.17:0/3529569145 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:43.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b8d53700 1 --2- 172.21.15.17:0/3529569145 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79b40a8c50 0x7f79b4152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:43.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b2ffd700 1 -- 172.21.15.17:0/3529569145 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3598803843 0 0) 0x7f79b40a7250 con 0x7f79b40b2710 2022-08-01T19:34:43.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b2ffd700 1 -- 172.21.15.17:0/3529569145 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7998003260 con 0x7f79b40b2710 2022-08-01T19:34:43.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b2ffd700 1 -- 172.21.15.17:0/3529569145 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1009534154 0 0) 0x7f79b40a7480 con 0x7f79b40b01a0 2022-08-01T19:34:43.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b2ffd700 1 -- 172.21.15.17:0/3529569145 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79b40a7250 con 0x7f79b40b01a0 2022-08-01T19:34:43.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b2ffd700 1 -- 172.21.15.17:0/3529569145 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3877361343 0 0) 0x7f7998003260 con 0x7f79b40b2710 2022-08-01T19:34:43.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b2ffd700 1 -- 172.21.15.17:0/3529569145 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79b40a8c50 msgr2=0x7f79b4152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:43.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b2ffd700 1 --2- 172.21.15.17:0/3529569145 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79b40a8c50 0x7f79b4152d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:43.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b2ffd700 1 -- 172.21.15.17:0/3529569145 >> v1:172.21.15.77:6789/0 conn(0x7f79b40b01a0 legacy=0x7f79b40a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:43.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b2ffd700 1 -- 172.21.15.17:0/3529569145 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b40a79c0 con 0x7f79b40b2710 2022-08-01T19:34:43.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b2ffd700 1 -- 172.21.15.17:0/3529569145 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7f79ac0019e0 con 0x7f79b40b2710 2022-08-01T19:34:43.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b2ffd700 1 -- 172.21.15.17:0/3529569145 >> v1:172.21.15.17:6789/0 conn(0x7f79b40b2710 legacy=0x7f79b414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:43.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b2ffd700 1 --2- 172.21.15.17:0/3529569145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7998004970 0x7f7998004d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:43.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b2ffd700 1 --2- 172.21.15.17:0/3529569145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79980034a0 0x7f7998003920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:43.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b2ffd700 1 --2- 172.21.15.17:0/3529569145 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79b40a8c50 0x7f7998003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:43.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b2ffd700 1 -- 172.21.15.17:0/3529569145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b40a7720 con 0x7f79980034a0 2022-08-01T19:34:43.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b2ffd700 1 -- 172.21.15.17:0/3529569145 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799800b780 con 0x7f7998004970 2022-08-01T19:34:43.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b2ffd700 1 -- 172.21.15.17:0/3529569145 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f799800b940 con 0x7f79b40a8c50 2022-08-01T19:34:43.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b37fe700 1 --2- 172.21.15.17:0/3529569145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79980034a0 0x7f7998003920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:43.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b3fff700 1 --2- 172.21.15.17:0/3529569145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7998004970 0x7f7998004d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:43.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b8d53700 1 --2- 172.21.15.17:0/3529569145 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79b40a8c50 0x7f7998003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:43.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b37fe700 1 -- 172.21.15.17:0/3529569145 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79b40a8c50 msgr2=0x7f7998003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:43.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b37fe700 1 --2- 172.21.15.17:0/3529569145 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79b40a8c50 0x7f7998003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:43.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b37fe700 1 -- 172.21.15.17:0/3529569145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7998004970 msgr2=0x7f7998004d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:43.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b37fe700 1 --2- 172.21.15.17:0/3529569145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7998004970 0x7f7998004d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:43.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b37fe700 1 -- 172.21.15.17:0/3529569145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7998003260 con 0x7f79980034a0 2022-08-01T19:34:43.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b37fe700 1 --2- 172.21.15.17:0/3529569145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79980034a0 0x7f7998003920 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f799c001110 tx=0x7f799c007a00).ready entity=mon.0 client_cookie=3019b3468e2b9a84 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:43.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b2ffd700 1 -- 172.21.15.17:0/3529569145 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f799c007d10 con 0x7f79980034a0 2022-08-01T19:34:43.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b2ffd700 1 -- 172.21.15.17:0/3529569145 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f799c007030 con 0x7f79980034a0 2022-08-01T19:34:43.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b2ffd700 1 -- 172.21.15.17:0/3529569145 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f799c007390 con 0x7f79980034a0 2022-08-01T19:34:43.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b9554700 1 -- 172.21.15.17:0/3529569145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79980034a0 msgr2=0x7f7998003920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:43.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b9554700 1 --2- 172.21.15.17:0/3529569145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79980034a0 0x7f7998003920 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f799c001110 tx=0x7f799c007a00).stop 2022-08-01T19:34:43.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b8d53700 1 -- 172.21.15.17:0/3529569145 reap_dead start 2022-08-01T19:34:43.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b9554700 1 -- 172.21.15.17:0/3529569145 shutdown_connections 2022-08-01T19:34:43.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b9554700 1 -- 172.21.15.17:0/3529569145 >> 172.21.15.17:0/3529569145 conn(0x7f79b4060310 msgr2=0x7f79b40b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:43.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b9554700 1 -- 172.21.15.17:0/3529569145 shutdown_connections 2022-08-01T19:34:43.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.962+0000 7f79b9554700 1 -- 172.21.15.17:0/3529569145 wait complete. 2022-08-01T19:34:43.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.966+0000 7f79b9554700 1 Processor -- start 2022-08-01T19:34:43.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.966+0000 7f79b9554700 1 -- start start 2022-08-01T19:34:43.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.966+0000 7f79b9554700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79b4066a30 0x7f79b40642e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:43.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.966+0000 7f79b9554700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79b40633d0 0x7f79b4066e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:43.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.966+0000 7f79b9554700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79b4067300 0x7f79b4064a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:43.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.966+0000 7f79b9554700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b4157380 con 0x7f79b40633d0 2022-08-01T19:34:43.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.966+0000 7f79b9554700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b40ac190 con 0x7f79b4066a30 2022-08-01T19:34:43.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.966+0000 7f79b9554700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b40ac430 con 0x7f79b4067300 2022-08-01T19:34:43.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.970+0000 7f79b37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79b40633d0 0x7f79b4066e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:43.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.970+0000 7f79b3fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79b4066a30 0x7f79b40642e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:43.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.970+0000 7f79b37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79b40633d0 0x7f79b4066e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37532/0 (socket says 172.21.15.17:37532) 2022-08-01T19:34:43.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.970+0000 7f79b8d53700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79b4067300 0x7f79b4064a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:43.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.970+0000 7f79b37fe700 1 -- 172.21.15.17:0/3533412839 learned_addr learned my addr 172.21.15.17:0/3533412839 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:43.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.970+0000 7f79b8d53700 1 -- 172.21.15.17:0/3533412839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79b4066a30 msgr2=0x7f79b40642e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:43.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.970+0000 7f79b8d53700 1 --2- 172.21.15.17:0/3533412839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79b4066a30 0x7f79b40642e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:43.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.970+0000 7f79b8d53700 1 -- 172.21.15.17:0/3533412839 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79b40633d0 msgr2=0x7f79b4066e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:43.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.970+0000 7f79b8d53700 1 --2- 172.21.15.17:0/3533412839 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79b40633d0 0x7f79b4066e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:43.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.970+0000 7f79b8d53700 1 -- 172.21.15.17:0/3533412839 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b40ab610 con 0x7f79b4067300 2022-08-01T19:34:43.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.970+0000 7f79b8d53700 1 --2- 172.21.15.17:0/3533412839 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79b4067300 0x7f79b4064a50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f79a400b000 tx=0x7f79a4008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:43.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.970+0000 7f79b17fa700 1 -- 172.21.15.17:0/3533412839 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79a4014430 con 0x7f79b4067300 2022-08-01T19:34:43.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.970+0000 7f79b17fa700 1 -- 172.21.15.17:0/3533412839 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a4014590 con 0x7f79b4067300 2022-08-01T19:34:43.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.970+0000 7f79b9554700 1 -- 172.21.15.17:0/3533412839 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b4157620 con 0x7f79b4067300 2022-08-01T19:34:43.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.970+0000 7f79b17fa700 1 -- 172.21.15.17:0/3533412839 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79a4014930 con 0x7f79b4067300 2022-08-01T19:34:43.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.970+0000 7f79b9554700 1 -- 172.21.15.17:0/3533412839 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b4063850 con 0x7f79b4067300 2022-08-01T19:34:43.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.970+0000 7f79b9554700 1 -- 172.21.15.17:0/3533412839 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b40ac5f0 con 0x7f79b4067300 2022-08-01T19:34:43.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.970+0000 7f79b17fa700 1 -- 172.21.15.17:0/3533412839 <== mon.2 v2:172.21.15.77:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f79a4014a90 con 0x7f79b4067300 2022-08-01T19:34:43.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.974+0000 7f79b17fa700 1 --2- 172.21.15.17:0/3533412839 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f798c053aa0 0x7f798c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:43.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.974+0000 7f79b17fa700 1 -- 172.21.15.17:0/3533412839 <== mon.2 v2:172.21.15.77:3301/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f79a406b570 con 0x7f79b4067300 2022-08-01T19:34:43.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.974+0000 7f79b3fff700 1 --2- 172.21.15.17:0/3533412839 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f798c053aa0 0x7f798c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:43.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.974+0000 7f79b3fff700 1 --2- 172.21.15.17:0/3533412839 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f798c053aa0 0x7f798c055f60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f79ac005000 tx=0x7f79b40b0640).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:43.991 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:43.982+0000 7f79b17fa700 1 -- 172.21.15.17:0/3533412839 <== mon.2 v2:172.21.15.77:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f79a40425d0 con 0x7f79b4067300 2022-08-01T19:34:44.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.110+0000 7f79b9554700 1 -- 172.21.15.17:0/3533412839 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f79b40574e0 con 0x7f79b4067300 2022-08-01T19:34:44.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.110+0000 7f79b17fa700 1 -- 172.21.15.17:0/3533412839 <== mon.2 v2:172.21.15.77:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v30) v1 ==== 110+0+10 (secure 0 0 0) 0x7f79a4045bf0 con 0x7f79b4067300 2022-08-01T19:34:44.115 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:34:44.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.114+0000 7f79b9554700 1 -- 172.21.15.17:0/3533412839 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f798c053aa0 msgr2=0x7f798c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:44.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.114+0000 7f79b9554700 1 --2- 172.21.15.17:0/3533412839 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f798c053aa0 0x7f798c055f60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f79ac005000 tx=0x7f79b40b0640).stop 2022-08-01T19:34:44.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.114+0000 7f79b9554700 1 -- 172.21.15.17:0/3533412839 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79b4067300 msgr2=0x7f79b4064a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:44.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.114+0000 7f79b9554700 1 --2- 172.21.15.17:0/3533412839 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79b4067300 0x7f79b4064a50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f79a400b000 tx=0x7f79a4008040).stop 2022-08-01T19:34:44.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.114+0000 7f79b9554700 1 -- 172.21.15.17:0/3533412839 shutdown_connections 2022-08-01T19:34:44.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.114+0000 7f79b9554700 1 --2- 172.21.15.17:0/3533412839 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f798c053aa0 0x7f798c055f60 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:44.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.114+0000 7f79b9554700 1 --2- 172.21.15.17:0/3533412839 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79b4067300 0x7f79b4064a50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:44.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.114+0000 7f79b9554700 1 --2- 172.21.15.17:0/3533412839 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79b40633d0 0x7f79b4066e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:44.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.114+0000 7f79b9554700 1 --2- 172.21.15.17:0/3533412839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79b4066a30 0x7f79b40642e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:44.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.114+0000 7f79b9554700 1 -- 172.21.15.17:0/3533412839 >> 172.21.15.17:0/3533412839 conn(0x7f79b4060310 msgr2=0x7f79b405be30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:44.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.114+0000 7f79b9554700 1 -- 172.21.15.17:0/3533412839 shutdown_connections 2022-08-01T19:34:44.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.114+0000 7f79b9554700 1 -- 172.21.15.17:0/3533412839 wait complete. 2022-08-01T19:34:44.135 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:34:44.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.238+0000 7f2e46f93700 1 Processor -- start 2022-08-01T19:34:44.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.238+0000 7f2e46f93700 1 -- start start 2022-08-01T19:34:44.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.238+0000 7f2e46f93700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2e400b0270 0x7f2e40143000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:44.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.238+0000 7f2e46f93700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f2e400ac880 0x7f2e401434f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:44.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.238+0000 7f2e46f93700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e40067a10 con 0x7f2e400b2710 2022-08-01T19:34:44.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.238+0000 7f2e46f93700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e400ad740 con 0x7f2e400b0270 2022-08-01T19:34:44.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.238+0000 7f2e46f93700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e40067cb0 con 0x7f2e400ac880 2022-08-01T19:34:44.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.238+0000 7f2e45790700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2e400b0270 0x7f2e40143000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:44.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.238+0000 7f2e45790700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2e400b0270 0x7f2e40143000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37540/0 (socket says 172.21.15.17:37540) 2022-08-01T19:34:44.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.238+0000 7f2e45790700 1 -- 172.21.15.17:0/1842177830 learned_addr learned my addr 172.21.15.17:0/1842177830 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:44.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.238+0000 7f2e46792700 1 --2- 172.21.15.17:0/1842177830 >> v2:172.21.15.77:3300/0 conn(0x7f2e400ac880 0x7f2e401434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:44.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.238+0000 7f2e46792700 1 -- 172.21.15.17:0/1842177830 >> v2:172.21.15.17:3300/0 conn(0x7f2e400b0270 msgr2=0x7f2e40143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:44.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.238+0000 7f2e46792700 1 --2- 172.21.15.17:0/1842177830 >> v2:172.21.15.17:3300/0 conn(0x7f2e400b0270 0x7f2e40143000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:44.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.238+0000 7f2e46792700 1 -- 172.21.15.17:0/1842177830 >> v1:172.21.15.77:6789/0 conn(0x7f2e400b2710 legacy=0x7f2e401401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:44.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.238+0000 7f2e46792700 1 -- 172.21.15.17:0/1842177830 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e40147160 con 0x7f2e400ac880 2022-08-01T19:34:44.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.242+0000 7f2e46792700 1 --2- 172.21.15.17:0/1842177830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e400ac880 0x7f2e401434f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f2e380032f0 tx=0x7f2e380091d0).ready entity=mon.1 client_cookie=46768fdf09e60256 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:44.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.242+0000 7f2e44f8f700 1 -- 172.21.15.17:0/1842177830 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e3800b680 con 0x7f2e400ac880 2022-08-01T19:34:44.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.242+0000 7f2e44f8f700 1 -- 172.21.15.17:0/1842177830 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3800b7e0 con 0x7f2e400ac880 2022-08-01T19:34:44.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.242+0000 7f2e44f8f700 1 -- 172.21.15.17:0/1842177830 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e3800b950 con 0x7f2e400ac880 2022-08-01T19:34:44.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.242+0000 7f2e46f93700 1 -- 172.21.15.17:0/1842177830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e400ac880 msgr2=0x7f2e401434f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:44.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.242+0000 7f2e46f93700 1 --2- 172.21.15.17:0/1842177830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e400ac880 0x7f2e401434f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f2e380032f0 tx=0x7f2e380091d0).stop 2022-08-01T19:34:44.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.242+0000 7f2e46f93700 1 -- 172.21.15.17:0/1842177830 shutdown_connections 2022-08-01T19:34:44.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.242+0000 7f2e46f93700 1 --2- 172.21.15.17:0/1842177830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e400ac880 0x7f2e401434f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:44.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.242+0000 7f2e46f93700 1 --2- 172.21.15.17:0/1842177830 >> v2:172.21.15.17:3300/0 conn(0x7f2e400b0270 0x7f2e40143000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:44.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.242+0000 7f2e46f93700 1 -- 172.21.15.17:0/1842177830 >> 172.21.15.17:0/1842177830 conn(0x7f2e40060310 msgr2=0x7f2e400b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:44.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.242+0000 7f2e46f93700 1 -- 172.21.15.17:0/1842177830 shutdown_connections 2022-08-01T19:34:44.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.242+0000 7f2e46f93700 1 -- 172.21.15.17:0/1842177830 wait complete. 2022-08-01T19:34:44.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.242+0000 7f2e46f93700 1 Processor -- start 2022-08-01T19:34:44.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.242+0000 7f2e46f93700 1 -- start start 2022-08-01T19:34:44.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.242+0000 7f2e46f93700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e400ac880 0x7f2e40145530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:44.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.242+0000 7f2e46f93700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2e400b0270 0x7f2e40155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:44.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.242+0000 7f2e46f93700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e400b2710 0x7f2e401564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:44.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.242+0000 7f2e46f93700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e40155b50 con 0x7f2e400b2710 2022-08-01T19:34:44.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.242+0000 7f2e46f93700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4009eae0 con 0x7f2e400ac880 2022-08-01T19:34:44.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.242+0000 7f2e46f93700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e400ac220 con 0x7f2e400b0270 2022-08-01T19:34:44.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.242+0000 7f2e46792700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e400b2710 0x7f2e401564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:44.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.242+0000 7f2e46792700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e400b2710 0x7f2e401564d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37548/0 (socket says 172.21.15.17:37548) 2022-08-01T19:34:44.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.242+0000 7f2e45f91700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e400ac880 0x7f2e40145530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:44.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.242+0000 7f2e46792700 1 -- 172.21.15.17:0/3148293940 learned_addr learned my addr 172.21.15.17:0/3148293940 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:44.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.242+0000 7f2e45790700 1 --2- 172.21.15.17:0/3148293940 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2e400b0270 0x7f2e40155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:44.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.246+0000 7f2e46792700 1 -- 172.21.15.17:0/3148293940 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2e400b0270 msgr2=0x7f2e40155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:44.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.246+0000 7f2e46792700 1 --2- 172.21.15.17:0/3148293940 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2e400b0270 0x7f2e40155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:44.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.246+0000 7f2e46792700 1 -- 172.21.15.17:0/3148293940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e400ac880 msgr2=0x7f2e40145530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:44.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.246+0000 7f2e46792700 1 --2- 172.21.15.17:0/3148293940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e400ac880 0x7f2e40145530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:44.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.246+0000 7f2e46792700 1 -- 172.21.15.17:0/3148293940 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e38009470 con 0x7f2e400b2710 2022-08-01T19:34:44.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.246+0000 7f2e46792700 1 --2- 172.21.15.17:0/3148293940 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e400b2710 0x7f2e401564d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f2e380032f0 tx=0x7f2e38016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:44.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.246+0000 7f2e2effd700 1 -- 172.21.15.17:0/3148293940 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e38011e20 con 0x7f2e400b2710 2022-08-01T19:34:44.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.246+0000 7f2e2effd700 1 -- 172.21.15.17:0/3148293940 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e38021440 con 0x7f2e400b2710 2022-08-01T19:34:44.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.246+0000 7f2e46f93700 1 -- 172.21.15.17:0/3148293940 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e4009ed10 con 0x7f2e400b2710 2022-08-01T19:34:44.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.246+0000 7f2e2effd700 1 -- 172.21.15.17:0/3148293940 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e380217a0 con 0x7f2e400b2710 2022-08-01T19:34:44.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.246+0000 7f2e46f93700 1 -- 172.21.15.17:0/3148293940 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e401569c0 con 0x7f2e400b2710 2022-08-01T19:34:44.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.246+0000 7f2e46f93700 1 -- 172.21.15.17:0/3148293940 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e400574e0 con 0x7f2e400b2710 2022-08-01T19:34:44.266 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.246+0000 7f2e2effd700 1 -- 172.21.15.17:0/3148293940 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2e38027020 con 0x7f2e400b2710 2022-08-01T19:34:44.266 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.246+0000 7f2e2effd700 1 --2- 172.21.15.17:0/3148293940 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2e20053aa0 0x7f2e20055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:44.266 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.250+0000 7f2e2effd700 1 -- 172.21.15.17:0/3148293940 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f2e3806d3a0 con 0x7f2e400b2710 2022-08-01T19:34:44.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.250+0000 7f2e45f91700 1 --2- 172.21.15.17:0/3148293940 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2e20053aa0 0x7f2e20055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:44.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.250+0000 7f2e45f91700 1 --2- 172.21.15.17:0/3148293940 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2e20053aa0 0x7f2e20055f60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f2e3000d000 tx=0x7f2e30007b70).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:44.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.254+0000 7f2e2effd700 1 -- 172.21.15.17:0/3148293940 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2e38044310 con 0x7f2e400b2710 2022-08-01T19:34:44.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.382+0000 7f2e46f93700 1 -- 172.21.15.17:0/3148293940 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2e400b1bb0 con 0x7f2e400b2710 2022-08-01T19:34:44.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.382+0000 7f2e2effd700 1 -- 172.21.15.17:0/3148293940 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v31) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2e38015020 con 0x7f2e400b2710 2022-08-01T19:34:44.388 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:34:44.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.382+0000 7f2e46f93700 1 -- 172.21.15.17:0/3148293940 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2e20053aa0 msgr2=0x7f2e20055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:44.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.382+0000 7f2e46f93700 1 --2- 172.21.15.17:0/3148293940 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2e20053aa0 0x7f2e20055f60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f2e3000d000 tx=0x7f2e30007b70).stop 2022-08-01T19:34:44.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.382+0000 7f2e46f93700 1 -- 172.21.15.17:0/3148293940 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e400b2710 msgr2=0x7f2e401564d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:44.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.382+0000 7f2e46f93700 1 --2- 172.21.15.17:0/3148293940 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e400b2710 0x7f2e401564d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f2e380032f0 tx=0x7f2e38016000).stop 2022-08-01T19:34:44.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.382+0000 7f2e46f93700 1 -- 172.21.15.17:0/3148293940 shutdown_connections 2022-08-01T19:34:44.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.382+0000 7f2e46f93700 1 --2- 172.21.15.17:0/3148293940 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2e20053aa0 0x7f2e20055f60 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:44.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.382+0000 7f2e46f93700 1 --2- 172.21.15.17:0/3148293940 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e400b2710 0x7f2e401564d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:44.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.382+0000 7f2e46f93700 1 --2- 172.21.15.17:0/3148293940 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2e400b0270 0x7f2e40155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:44.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.382+0000 7f2e46f93700 1 --2- 172.21.15.17:0/3148293940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e400ac880 0x7f2e40145530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:44.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.382+0000 7f2e46f93700 1 -- 172.21.15.17:0/3148293940 >> 172.21.15.17:0/3148293940 conn(0x7f2e40060310 msgr2=0x7f2e4005cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:44.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.386+0000 7f2e46f93700 1 -- 172.21.15.17:0/3148293940 shutdown_connections 2022-08-01T19:34:44.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.386+0000 7f2e46f93700 1 -- 172.21.15.17:0/3148293940 wait complete. 2022-08-01T19:34:44.401 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-08-01T19:34:44.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.498+0000 7ff224b48700 1 Processor -- start 2022-08-01T19:34:44.501 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.498+0000 7ff224b48700 1 -- start start 2022-08-01T19:34:44.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.498+0000 7ff224b48700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff2200619b0 0x7ff2200b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:44.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.498+0000 7ff224b48700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff2200b6ee0 0x7ff220143890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:44.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.498+0000 7ff224b48700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff220061d90 con 0x7ff2200b5ed0 2022-08-01T19:34:44.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.498+0000 7ff224b48700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff220142f60 con 0x7ff2200619b0 2022-08-01T19:34:44.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.498+0000 7ff224b48700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2200ba5a0 con 0x7ff2200b6ee0 2022-08-01T19:34:44.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.498+0000 7ff21ffff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff2200b5ed0 0x7ff2200b5190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57980/0 (socket says 172.21.15.17:57980) 2022-08-01T19:34:44.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.498+0000 7ff21f7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff2200619b0 0x7ff2200b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:44.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.498+0000 7ff21ffff700 1 -- 172.21.15.17:0/1594299772 learned_addr learned my addr 172.21.15.17:0/1594299772 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:44.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.498+0000 7ff21effd700 1 --2- 172.21.15.17:0/1594299772 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff2200b6ee0 0x7ff220143890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:44.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.498+0000 7ff21e7fc700 1 -- 172.21.15.17:0/1594299772 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3131253577 0 0) 0x7ff220061d90 con 0x7ff2200b5ed0 2022-08-01T19:34:44.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.498+0000 7ff21e7fc700 1 -- 172.21.15.17:0/1594299772 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff200003260 con 0x7ff2200b5ed0 2022-08-01T19:34:44.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.498+0000 7ff21f7fe700 1 -- 172.21.15.17:0/1594299772 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff2200b6ee0 msgr2=0x7ff220143890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:44.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.498+0000 7ff21f7fe700 1 --2- 172.21.15.17:0/1594299772 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff2200b6ee0 0x7ff220143890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:44.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.498+0000 7ff21f7fe700 1 -- 172.21.15.17:0/1594299772 >> v1:172.21.15.17:6789/0 conn(0x7ff2200b5ed0 legacy=0x7ff2200b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:44.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.498+0000 7ff21f7fe700 1 -- 172.21.15.17:0/1594299772 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff220143150 con 0x7ff2200619b0 2022-08-01T19:34:44.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.498+0000 7ff21f7fe700 1 --2- 172.21.15.17:0/1594299772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2200619b0 0x7ff2200b48a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7ff210002f70 tx=0x7ff210006df0).ready entity=mon.0 client_cookie=1025358ab527abbf server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:44.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.498+0000 7ff21e7fc700 1 -- 172.21.15.17:0/1594299772 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff21000d4a0 con 0x7ff2200619b0 2022-08-01T19:34:44.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.498+0000 7ff21e7fc700 1 -- 172.21.15.17:0/1594299772 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff210000e90 con 0x7ff2200619b0 2022-08-01T19:34:44.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.498+0000 7ff21e7fc700 1 -- 172.21.15.17:0/1594299772 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff21000d7f0 con 0x7ff2200619b0 2022-08-01T19:34:44.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.498+0000 7ff224b48700 1 -- 172.21.15.17:0/1594299772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2200619b0 msgr2=0x7ff2200b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:44.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.498+0000 7ff224b48700 1 --2- 172.21.15.17:0/1594299772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2200619b0 0x7ff2200b48a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7ff210002f70 tx=0x7ff210006df0).stop 2022-08-01T19:34:44.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.498+0000 7ff224b48700 1 -- 172.21.15.17:0/1594299772 shutdown_connections 2022-08-01T19:34:44.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.498+0000 7ff224b48700 1 --2- 172.21.15.17:0/1594299772 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff2200b6ee0 0x7ff220143890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:44.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.498+0000 7ff224b48700 1 --2- 172.21.15.17:0/1594299772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2200619b0 0x7ff2200b48a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:44.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.498+0000 7ff224b48700 1 -- 172.21.15.17:0/1594299772 >> 172.21.15.17:0/1594299772 conn(0x7ff22006af80 msgr2=0x7ff22005fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:44.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.498+0000 7ff224b48700 1 -- 172.21.15.17:0/1594299772 shutdown_connections 2022-08-01T19:34:44.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.498+0000 7ff224b48700 1 -- 172.21.15.17:0/1594299772 wait complete. 2022-08-01T19:34:44.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.498+0000 7ff224b48700 1 Processor -- start 2022-08-01T19:34:44.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.498+0000 7ff224b48700 1 -- start start 2022-08-01T19:34:44.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.502+0000 7ff224b48700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2200619b0 0x7ff2201402f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:44.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.502+0000 7ff224b48700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff2200b5ed0 0x7ff2201407e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:44.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.502+0000 7ff224b48700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2200b6ee0 0x7ff2201132a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:44.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.502+0000 7ff224b48700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff220156720 con 0x7ff2200b6ee0 2022-08-01T19:34:44.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.502+0000 7ff224b48700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2200ae590 con 0x7ff2200619b0 2022-08-01T19:34:44.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.502+0000 7ff224b48700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2200ae830 con 0x7ff2200b5ed0 2022-08-01T19:34:44.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.502+0000 7ff21ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2200b6ee0 0x7ff2201132a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:44.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.502+0000 7ff21ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2200b6ee0 0x7ff2201132a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37562/0 (socket says 172.21.15.17:37562) 2022-08-01T19:34:44.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.502+0000 7ff21ffff700 1 -- 172.21.15.17:0/2977093883 learned_addr learned my addr 172.21.15.17:0/2977093883 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:44.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.502+0000 7ff21f7fe700 1 --2- 172.21.15.17:0/2977093883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2200619b0 0x7ff2201402f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:44.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.502+0000 7ff21effd700 1 --2- 172.21.15.17:0/2977093883 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff2200b5ed0 0x7ff2201407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:44.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.502+0000 7ff21ffff700 1 -- 172.21.15.17:0/2977093883 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff2200b5ed0 msgr2=0x7ff2201407e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:44.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.502+0000 7ff21ffff700 1 --2- 172.21.15.17:0/2977093883 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff2200b5ed0 0x7ff2201407e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:44.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.502+0000 7ff21ffff700 1 -- 172.21.15.17:0/2977093883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2200619b0 msgr2=0x7ff2201402f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:44.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.502+0000 7ff21ffff700 1 --2- 172.21.15.17:0/2977093883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2200619b0 0x7ff2201402f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:44.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.502+0000 7ff21ffff700 1 -- 172.21.15.17:0/2977093883 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff21000b180 con 0x7ff2200b6ee0 2022-08-01T19:34:44.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.502+0000 7ff21ffff700 1 --2- 172.21.15.17:0/2977093883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2200b6ee0 0x7ff2201132a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7ff218001490 tx=0x7ff218007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:44.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.502+0000 7ff21cff9700 1 -- 172.21.15.17:0/2977093883 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff218005ba0 con 0x7ff2200b6ee0 2022-08-01T19:34:44.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.502+0000 7ff21cff9700 1 -- 172.21.15.17:0/2977093883 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff218005d00 con 0x7ff2200b6ee0 2022-08-01T19:34:44.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.502+0000 7ff224b48700 1 -- 172.21.15.17:0/2977093883 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2201563a0 con 0x7ff2200b6ee0 2022-08-01T19:34:44.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.502+0000 7ff21cff9700 1 -- 172.21.15.17:0/2977093883 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2180072c0 con 0x7ff2200b6ee0 2022-08-01T19:34:44.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.502+0000 7ff224b48700 1 -- 172.21.15.17:0/2977093883 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2200ade40 con 0x7ff2200b6ee0 2022-08-01T19:34:44.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.502+0000 7ff224b48700 1 -- 172.21.15.17:0/2977093883 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2200ae9f0 con 0x7ff2200b6ee0 2022-08-01T19:34:44.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.506+0000 7ff21cff9700 1 -- 172.21.15.17:0/2977093883 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff218010070 con 0x7ff2200b6ee0 2022-08-01T19:34:44.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.506+0000 7ff21cff9700 1 --2- 172.21.15.17:0/2977093883 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff204053b00 0x7ff204055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:44.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.506+0000 7ff21cff9700 1 -- 172.21.15.17:0/2977093883 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5600+0+0 (secure 0 0 0) 0x7ff21806b800 con 0x7ff2200b6ee0 2022-08-01T19:34:44.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.506+0000 7ff21f7fe700 1 --2- 172.21.15.17:0/2977093883 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff204053b00 0x7ff204055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:44.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.506+0000 7ff21f7fe700 1 --2- 172.21.15.17:0/2977093883 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff204053b00 0x7ff204055fc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7ff210018000 tx=0x7ff210014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:44.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.510+0000 7ff21cff9700 1 -- 172.21.15.17:0/2977093883 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff218042900 con 0x7ff2200b6ee0 2022-08-01T19:34:44.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.638+0000 7ff224b48700 1 -- 172.21.15.17:0/2977093883 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff2201568d0 con 0x7ff2200b6ee0 2022-08-01T19:34:44.645 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.638+0000 7ff21cff9700 1 -- 172.21.15.17:0/2977093883 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v31) v1 ==== 84+0+143 (secure 0 0 0) 0x7ff218045f20 con 0x7ff2200b6ee0 2022-08-01T19:34:44.645 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:44.645 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2022-08-01T19:34:44.646 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:44.646 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 1, 2022-08-01T19:34:44.646 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": "device_health_metrics" 2022-08-01T19:34:44.647 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:34:44.647 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:34:44.647 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 2, 2022-08-01T19:34:44.647 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": "rbd" 2022-08-01T19:34:44.647 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:34:44.648 INFO:teuthology.orchestra.run.smithi017.stdout:] 2022-08-01T19:34:44.649 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.642+0000 7ff224b48700 1 -- 172.21.15.17:0/2977093883 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff204053b00 msgr2=0x7ff204055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:44.649 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.642+0000 7ff224b48700 1 --2- 172.21.15.17:0/2977093883 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff204053b00 0x7ff204055fc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7ff210018000 tx=0x7ff210014000).stop 2022-08-01T19:34:44.650 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.642+0000 7ff224b48700 1 -- 172.21.15.17:0/2977093883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2200b6ee0 msgr2=0x7ff2201132a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:44.650 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.642+0000 7ff224b48700 1 --2- 172.21.15.17:0/2977093883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2200b6ee0 0x7ff2201132a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7ff218001490 tx=0x7ff218007dc0).stop 2022-08-01T19:34:44.650 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.642+0000 7ff224b48700 1 -- 172.21.15.17:0/2977093883 shutdown_connections 2022-08-01T19:34:44.650 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.642+0000 7ff224b48700 1 --2- 172.21.15.17:0/2977093883 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff204053b00 0x7ff204055fc0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:44.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.642+0000 7ff224b48700 1 --2- 172.21.15.17:0/2977093883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2200b6ee0 0x7ff2201132a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:44.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.642+0000 7ff224b48700 1 --2- 172.21.15.17:0/2977093883 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff2200b5ed0 0x7ff2201407e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:44.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.642+0000 7ff224b48700 1 --2- 172.21.15.17:0/2977093883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff2200619b0 0x7ff2201402f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:44.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.642+0000 7ff224b48700 1 -- 172.21.15.17:0/2977093883 >> 172.21.15.17:0/2977093883 conn(0x7ff22006af80 msgr2=0x7ff22005fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:44.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.642+0000 7ff224b48700 1 -- 172.21.15.17:0/2977093883 shutdown_connections 2022-08-01T19:34:44.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.642+0000 7ff224b48700 1 -- 172.21.15.17:0/2977093883 wait complete. 2022-08-01T19:34:44.658 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-08-01T19:34:44.658 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-08-01T19:34:44.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa347367700 1 Processor -- start 2022-08-01T19:34:44.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa347367700 1 -- start start 2022-08-01T19:34:44.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa347367700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa3400a8c50 0x7fa340152d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:44.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa347367700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3400a7250 con 0x7fa3400b2710 2022-08-01T19:34:44.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa347367700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3400a7480 con 0x7fa3400b01a0 2022-08-01T19:34:44.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa347367700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3400a7720 con 0x7fa3400a8c50 2022-08-01T19:34:44.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa346365700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa3400b2710 0x7fa34014afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57992/0 (socket says 172.21.15.17:57992) 2022-08-01T19:34:44.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa346365700 1 -- 172.21.15.17:0/2166177595 learned_addr learned my addr 172.21.15.17:0/2166177595 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:44.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa346b66700 1 --2- 172.21.15.17:0/2166177595 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa3400a8c50 0x7fa340152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:44.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa345363700 1 -- 172.21.15.17:0/2166177595 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3403094215 0 0) 0x7fa3400a7250 con 0x7fa3400b2710 2022-08-01T19:34:44.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa345363700 1 -- 172.21.15.17:0/2166177595 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa328003260 con 0x7fa3400b2710 2022-08-01T19:34:44.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa345363700 1 -- 172.21.15.17:0/2166177595 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2902345634 0 0) 0x7fa3400a7480 con 0x7fa3400b01a0 2022-08-01T19:34:44.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa345363700 1 -- 172.21.15.17:0/2166177595 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3400a7250 con 0x7fa3400b01a0 2022-08-01T19:34:44.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa346b66700 1 -- 172.21.15.17:0/2166177595 >> v1:172.21.15.77:6789/0 conn(0x7fa3400b01a0 legacy=0x7fa3400a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:44.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa346b66700 1 -- 172.21.15.17:0/2166177595 >> v1:172.21.15.17:6789/0 conn(0x7fa3400b2710 legacy=0x7fa34014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:44.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa346b66700 1 -- 172.21.15.17:0/2166177595 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3400a79c0 con 0x7fa3400a8c50 2022-08-01T19:34:44.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa346b66700 1 --2- 172.21.15.17:0/2166177595 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa3400a8c50 0x7fa340152d70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fa3380032f0 tx=0x7fa3380091d0).ready entity=mon.2 client_cookie=7f59cc80c39ceb2a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:44.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa345363700 1 -- 172.21.15.17:0/2166177595 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa33800b640 con 0x7fa3400a8c50 2022-08-01T19:34:44.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa345363700 1 -- 172.21.15.17:0/2166177595 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa33800b7a0 con 0x7fa3400a8c50 2022-08-01T19:34:44.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa345363700 1 -- 172.21.15.17:0/2166177595 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa33800b910 con 0x7fa3400a8c50 2022-08-01T19:34:44.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa347367700 1 -- 172.21.15.17:0/2166177595 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa3400a8c50 msgr2=0x7fa340152d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:44.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa347367700 1 --2- 172.21.15.17:0/2166177595 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa3400a8c50 0x7fa340152d70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fa3380032f0 tx=0x7fa3380091d0).stop 2022-08-01T19:34:44.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa347367700 1 -- 172.21.15.17:0/2166177595 shutdown_connections 2022-08-01T19:34:44.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa347367700 1 --2- 172.21.15.17:0/2166177595 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa3400a8c50 0x7fa340152d70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:44.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa347367700 1 -- 172.21.15.17:0/2166177595 >> 172.21.15.17:0/2166177595 conn(0x7fa340060310 msgr2=0x7fa3400b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:44.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa347367700 1 -- 172.21.15.17:0/2166177595 shutdown_connections 2022-08-01T19:34:44.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa347367700 1 -- 172.21.15.17:0/2166177595 wait complete. 2022-08-01T19:34:44.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa347367700 1 Processor -- start 2022-08-01T19:34:44.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa347367700 1 -- start start 2022-08-01T19:34:44.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa347367700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa3400a8c50 0x7fa340146670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:44.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa347367700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3400b01a0 0x7fa3400adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:44.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa347367700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3400b2710 0x7fa3400ae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:44.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa347367700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa340156b90 con 0x7fa3400b2710 2022-08-01T19:34:44.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa347367700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3400b4ed0 con 0x7fa3400b01a0 2022-08-01T19:34:44.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa347367700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3400b3510 con 0x7fa3400a8c50 2022-08-01T19:34:44.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa346b66700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3400b2710 0x7fa3400ae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:44.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa346b66700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3400b2710 0x7fa3400ae1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37576/0 (socket says 172.21.15.17:37576) 2022-08-01T19:34:44.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa346b66700 1 -- 172.21.15.17:0/369038916 learned_addr learned my addr 172.21.15.17:0/369038916 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:44.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa346365700 1 --2- 172.21.15.17:0/369038916 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa3400a8c50 0x7fa340146670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:44.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa345b64700 1 --2- 172.21.15.17:0/369038916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3400b01a0 0x7fa3400adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:44.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa346b66700 1 -- 172.21.15.17:0/369038916 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa3400a8c50 msgr2=0x7fa340146670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa346b66700 1 --2- 172.21.15.17:0/369038916 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa3400a8c50 0x7fa340146670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa346b66700 1 -- 172.21.15.17:0/369038916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3400b01a0 msgr2=0x7fa3400adcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa346b66700 1 --2- 172.21.15.17:0/369038916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3400b01a0 0x7fa3400adcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa346b66700 1 -- 172.21.15.17:0/369038916 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa338009470 con 0x7fa3400b2710 2022-08-01T19:34:44.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa346b66700 1 --2- 172.21.15.17:0/369038916 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3400b2710 0x7fa3400ae1e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7fa3380032f0 tx=0x7fa338016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:44.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa32f7fe700 1 -- 172.21.15.17:0/369038916 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa33800b910 con 0x7fa3400b2710 2022-08-01T19:34:44.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa347367700 1 -- 172.21.15.17:0/369038916 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3401547d0 con 0x7fa3400b2710 2022-08-01T19:34:44.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa32f7fe700 1 -- 172.21.15.17:0/369038916 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa33801f3f0 con 0x7fa3400b2710 2022-08-01T19:34:44.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa32f7fe700 1 -- 172.21.15.17:0/369038916 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa33800b640 con 0x7fa3400b2710 2022-08-01T19:34:44.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.766+0000 7fa347367700 1 -- 172.21.15.17:0/369038916 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa340146ea0 con 0x7fa3400b2710 2022-08-01T19:34:44.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.770+0000 7fa347367700 1 -- 172.21.15.17:0/369038916 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3400b36d0 con 0x7fa3400b2710 2022-08-01T19:34:44.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.770+0000 7fa32f7fe700 1 -- 172.21.15.17:0/369038916 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa33800f070 con 0x7fa3400b2710 2022-08-01T19:34:44.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.770+0000 7fa32f7fe700 1 --2- 172.21.15.17:0/369038916 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa320053aa0 0x7fa320055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:44.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.770+0000 7fa32f7fe700 1 -- 172.21.15.17:0/369038916 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fa33806c780 con 0x7fa3400b2710 2022-08-01T19:34:44.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.770+0000 7fa346365700 1 --2- 172.21.15.17:0/369038916 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa320053aa0 0x7fa320055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:44.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.770+0000 7fa346365700 1 --2- 172.21.15.17:0/369038916 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa320053aa0 0x7fa320055f60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fa33000d000 tx=0x7fa330009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:44.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.782+0000 7fa32f7fe700 1 -- 172.21.15.17:0/369038916 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa3400b36d0 con 0x7fa3400b2710 2022-08-01T19:34:44.908 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:44.902+0000 7fa347367700 1 -- 172.21.15.17:0/369038916 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fa3400b36d0 con 0x7fa3400b2710 2022-08-01T19:34:45.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.354+0000 7fa32f7fe700 1 -- 172.21.15.17:0/369038916 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v32) v1 ==== 133+0+0 (secure 0 0 0) 0x7fa3400b36d0 con 0x7fa3400b2710 2022-08-01T19:34:45.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.362+0000 7fa347367700 1 -- 172.21.15.17:0/369038916 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa320053aa0 msgr2=0x7fa320055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:45.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.362+0000 7fa347367700 1 --2- 172.21.15.17:0/369038916 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa320053aa0 0x7fa320055f60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fa33000d000 tx=0x7fa330009040).stop 2022-08-01T19:34:45.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.362+0000 7fa347367700 1 -- 172.21.15.17:0/369038916 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3400b2710 msgr2=0x7fa3400ae1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:45.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.362+0000 7fa347367700 1 --2- 172.21.15.17:0/369038916 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3400b2710 0x7fa3400ae1e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7fa3380032f0 tx=0x7fa338016000).stop 2022-08-01T19:34:45.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.362+0000 7fa347367700 1 -- 172.21.15.17:0/369038916 shutdown_connections 2022-08-01T19:34:45.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.362+0000 7fa347367700 1 --2- 172.21.15.17:0/369038916 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa320053aa0 0x7fa320055f60 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:45.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.362+0000 7fa347367700 1 --2- 172.21.15.17:0/369038916 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3400b2710 0x7fa3400ae1e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:45.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.362+0000 7fa347367700 1 --2- 172.21.15.17:0/369038916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3400b01a0 0x7fa3400adcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:45.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.362+0000 7fa347367700 1 --2- 172.21.15.17:0/369038916 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa3400a8c50 0x7fa340146670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:45.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.362+0000 7fa347367700 1 -- 172.21.15.17:0/369038916 >> 172.21.15.17:0/369038916 conn(0x7fa340060310 msgr2=0x7fa34014ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:45.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.362+0000 7fa347367700 1 -- 172.21.15.17:0/369038916 shutdown_connections 2022-08-01T19:34:45.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.362+0000 7fa347367700 1 -- 172.21.15.17:0/369038916 wait complete. 2022-08-01T19:34:45.371 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' created 2022-08-01T19:34:45.381 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-08-01T19:34:45.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.490+0000 7fdc90b7c700 1 Processor -- start 2022-08-01T19:34:45.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.490+0000 7fdc90b7c700 1 -- start start 2022-08-01T19:34:45.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.490+0000 7fdc90b7c700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fdc8c0b2710 0x7fdc8c14afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:45.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.490+0000 7fdc90b7c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdc8c0b01a0 0x7fdc8c14b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:45.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.490+0000 7fdc90b7c700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc8c0a7250 con 0x7fdc8c0afd60 2022-08-01T19:34:45.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.490+0000 7fdc90b7c700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc8c0a7480 con 0x7fdc8c0b2710 2022-08-01T19:34:45.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.490+0000 7fdc90b7c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c0a7720 con 0x7fdc8c0b01a0 2022-08-01T19:34:45.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.490+0000 7fdc8b7fe700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fdc8c0b2710 0x7fdc8c14afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:45.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.490+0000 7fdc8affd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdc8c0b01a0 0x7fdc8c14b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:45.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.490+0000 7fdc8affd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdc8c0b01a0 0x7fdc8c14b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:33682/0 (socket says 172.21.15.17:33682) 2022-08-01T19:34:45.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.490+0000 7fdc8bfff700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7fdc8c0afd60 0x7fdc8c0a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:46580/0 (socket says 172.21.15.17:46580) 2022-08-01T19:34:45.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.490+0000 7fdc8affd700 1 -- 172.21.15.17:0/1481176660 learned_addr learned my addr 172.21.15.17:0/1481176660 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:45.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.490+0000 7fdc8affd700 1 -- 172.21.15.17:0/1481176660 >> v2:172.21.15.77:3300/0 conn(0x7fdc8c0b2710 msgr2=0x7fdc8c14afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:45.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.490+0000 7fdc8affd700 1 --2- 172.21.15.17:0/1481176660 >> v2:172.21.15.77:3300/0 conn(0x7fdc8c0b2710 0x7fdc8c14afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:45.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.490+0000 7fdc8affd700 1 -- 172.21.15.17:0/1481176660 >> v1:172.21.15.77:6789/0 conn(0x7fdc8c0afd60 legacy=0x7fdc8c0a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:45.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.490+0000 7fdc8affd700 1 -- 172.21.15.17:0/1481176660 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc8c0a79c0 con 0x7fdc8c0b01a0 2022-08-01T19:34:45.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.490+0000 7fdc8a7fc700 1 -- 172.21.15.17:0/1481176660 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3195702506 0 0) 0x7fdc8c0a7250 con 0x7fdc8c0afd60 2022-08-01T19:34:45.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.490+0000 7fdc8affd700 1 --2- 172.21.15.17:0/1481176660 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdc8c0b01a0 0x7fdc8c14b4c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fdc7c0032f0 tx=0x7fdc7c009290).ready entity=mon.2 client_cookie=9e6c4ede37ad15dd server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:45.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.490+0000 7fdc8a7fc700 1 -- 172.21.15.17:0/1481176660 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc7c00b8a0 con 0x7fdc8c0b01a0 2022-08-01T19:34:45.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.490+0000 7fdc8a7fc700 1 -- 172.21.15.17:0/1481176660 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc7c00ba00 con 0x7fdc8c0b01a0 2022-08-01T19:34:45.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.490+0000 7fdc8a7fc700 1 -- 172.21.15.17:0/1481176660 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc7c00bb70 con 0x7fdc8c0b01a0 2022-08-01T19:34:45.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.490+0000 7fdc90b7c700 1 -- 172.21.15.17:0/1481176660 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdc8c0b01a0 msgr2=0x7fdc8c14b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:45.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.490+0000 7fdc90b7c700 1 --2- 172.21.15.17:0/1481176660 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdc8c0b01a0 0x7fdc8c14b4c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fdc7c0032f0 tx=0x7fdc7c009290).stop 2022-08-01T19:34:45.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.490+0000 7fdc90b7c700 1 -- 172.21.15.17:0/1481176660 shutdown_connections 2022-08-01T19:34:45.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.490+0000 7fdc90b7c700 1 --2- 172.21.15.17:0/1481176660 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdc8c0b01a0 0x7fdc8c14b4c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:45.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.490+0000 7fdc90b7c700 1 --2- 172.21.15.17:0/1481176660 >> v2:172.21.15.77:3300/0 conn(0x7fdc8c0b2710 0x7fdc8c14afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:45.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.490+0000 7fdc90b7c700 1 -- 172.21.15.17:0/1481176660 >> 172.21.15.17:0/1481176660 conn(0x7fdc8c060310 msgr2=0x7fdc8c0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:45.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.490+0000 7fdc90b7c700 1 -- 172.21.15.17:0/1481176660 shutdown_connections 2022-08-01T19:34:45.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.490+0000 7fdc90b7c700 1 -- 172.21.15.17:0/1481176660 wait complete. 2022-08-01T19:34:45.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.494+0000 7fdc90b7c700 1 Processor -- start 2022-08-01T19:34:45.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.494+0000 7fdc90b7c700 1 -- start start 2022-08-01T19:34:45.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.494+0000 7fdc90b7c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdc8c0afd60 0x7fdc8c0b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:45.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.494+0000 7fdc90b7c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc8c0b01a0 0x7fdc8c1565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:45.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.494+0000 7fdc90b7c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc8c0b2710 0x7fdc8c148ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:45.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.494+0000 7fdc90b7c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c0b3040 con 0x7fdc8c0b2710 2022-08-01T19:34:45.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.494+0000 7fdc90b7c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c15a610 con 0x7fdc8c0b01a0 2022-08-01T19:34:45.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.494+0000 7fdc90b7c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c15a8b0 con 0x7fdc8c0afd60 2022-08-01T19:34:45.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.494+0000 7fdc8bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc8c0b2710 0x7fdc8c148ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:45.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.494+0000 7fdc8bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc8c0b2710 0x7fdc8c148ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37590/0 (socket says 172.21.15.17:37590) 2022-08-01T19:34:45.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.494+0000 7fdc8bfff700 1 -- 172.21.15.17:0/2304214211 learned_addr learned my addr 172.21.15.17:0/2304214211 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:45.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.494+0000 7fdc8affd700 1 --2- 172.21.15.17:0/2304214211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc8c0b01a0 0x7fdc8c1565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:45.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.494+0000 7fdc8b7fe700 1 --2- 172.21.15.17:0/2304214211 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdc8c0afd60 0x7fdc8c0b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:45.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.494+0000 7fdc8bfff700 1 -- 172.21.15.17:0/2304214211 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdc8c0afd60 msgr2=0x7fdc8c0b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:45.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.494+0000 7fdc8bfff700 1 --2- 172.21.15.17:0/2304214211 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdc8c0afd60 0x7fdc8c0b36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:45.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.494+0000 7fdc8bfff700 1 -- 172.21.15.17:0/2304214211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc8c0b01a0 msgr2=0x7fdc8c1565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:45.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.494+0000 7fdc8bfff700 1 --2- 172.21.15.17:0/2304214211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc8c0b01a0 0x7fdc8c1565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:45.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.494+0000 7fdc8bfff700 1 -- 172.21.15.17:0/2304214211 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc7c009530 con 0x7fdc8c0b2710 2022-08-01T19:34:45.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.494+0000 7fdc8b7fe700 1 --2- 172.21.15.17:0/2304214211 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdc8c0afd60 0x7fdc8c0b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:34:45.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.494+0000 7fdc8bfff700 1 --2- 172.21.15.17:0/2304214211 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc8c0b2710 0x7fdc8c148ab0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7fdc84000ed0 tx=0x7fdc84007a30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:45.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.494+0000 7fdc88ff9700 1 -- 172.21.15.17:0/2304214211 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc840070d0 con 0x7fdc8c0b2710 2022-08-01T19:34:45.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.494+0000 7fdc88ff9700 1 -- 172.21.15.17:0/2304214211 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc84007230 con 0x7fdc8c0b2710 2022-08-01T19:34:45.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.494+0000 7fdc90b7c700 1 -- 172.21.15.17:0/2304214211 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc8c0b3230 con 0x7fdc8c0b2710 2022-08-01T19:34:45.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.494+0000 7fdc88ff9700 1 -- 172.21.15.17:0/2304214211 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc84005740 con 0x7fdc8c0b2710 2022-08-01T19:34:45.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.494+0000 7fdc90b7c700 1 -- 172.21.15.17:0/2304214211 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc8c156ae0 con 0x7fdc8c0b2710 2022-08-01T19:34:45.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.494+0000 7fdc90b7c700 1 -- 172.21.15.17:0/2304214211 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc8c15aa70 con 0x7fdc8c0b2710 2022-08-01T19:34:45.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.498+0000 7fdc88ff9700 1 -- 172.21.15.17:0/2304214211 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdc8401a060 con 0x7fdc8c0b2710 2022-08-01T19:34:45.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.498+0000 7fdc88ff9700 1 --2- 172.21.15.17:0/2304214211 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdc6c053a70 0x7fdc6c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:45.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.498+0000 7fdc88ff9700 1 -- 172.21.15.17:0/2304214211 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5954+0+0 (secure 0 0 0) 0x7fdc8406b5f0 con 0x7fdc8c0b2710 2022-08-01T19:34:45.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.498+0000 7fdc8b7fe700 1 --2- 172.21.15.17:0/2304214211 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdc6c053a70 0x7fdc6c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:45.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.498+0000 7fdc8b7fe700 1 --2- 172.21.15.17:0/2304214211 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdc6c053a70 0x7fdc6c055f30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fdc8000b000 tx=0x7fdc80008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:45.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.502+0000 7fdc88ff9700 1 -- 172.21.15.17:0/2304214211 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdc84042580 con 0x7fdc8c0b2710 2022-08-01T19:34:45.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:45.630+0000 7fdc90b7c700 1 -- 172.21.15.17:0/2304214211 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fdc8c0b33c0 con 0x7fdc8c0b2710 2022-08-01T19:34:46.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.358+0000 7fdc88ff9700 1 -- 172.21.15.17:0/2304214211 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v33) v1 ==== 125+0+0 (secure 0 0 0) 0x7fdc84018020 con 0x7fdc8c0b2710 2022-08-01T19:34:46.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.366+0000 7fdc90b7c700 1 -- 172.21.15.17:0/2304214211 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdc6c053a70 msgr2=0x7fdc6c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:46.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.366+0000 7fdc90b7c700 1 --2- 172.21.15.17:0/2304214211 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdc6c053a70 0x7fdc6c055f30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fdc8000b000 tx=0x7fdc80008040).stop 2022-08-01T19:34:46.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.366+0000 7fdc90b7c700 1 -- 172.21.15.17:0/2304214211 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc8c0b2710 msgr2=0x7fdc8c148ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:46.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.366+0000 7fdc90b7c700 1 --2- 172.21.15.17:0/2304214211 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc8c0b2710 0x7fdc8c148ab0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7fdc84000ed0 tx=0x7fdc84007a30).stop 2022-08-01T19:34:46.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.370+0000 7fdc90b7c700 1 -- 172.21.15.17:0/2304214211 shutdown_connections 2022-08-01T19:34:46.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.370+0000 7fdc90b7c700 1 --2- 172.21.15.17:0/2304214211 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdc6c053a70 0x7fdc6c055f30 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:46.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.370+0000 7fdc90b7c700 1 --2- 172.21.15.17:0/2304214211 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc8c0b2710 0x7fdc8c148ab0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:46.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.370+0000 7fdc90b7c700 1 --2- 172.21.15.17:0/2304214211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc8c0b01a0 0x7fdc8c1565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:46.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.370+0000 7fdc90b7c700 1 --2- 172.21.15.17:0/2304214211 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdc8c0afd60 0x7fdc8c0b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:46.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.370+0000 7fdc90b7c700 1 -- 172.21.15.17:0/2304214211 >> 172.21.15.17:0/2304214211 conn(0x7fdc8c060310 msgr2=0x7fdc8c14a170 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:46.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.370+0000 7fdc90b7c700 1 -- 172.21.15.17:0/2304214211 shutdown_connections 2022-08-01T19:34:46.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.370+0000 7fdc90b7c700 1 -- 172.21.15.17:0/2304214211 wait complete. 2022-08-01T19:34:46.377 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' created 2022-08-01T19:34:46.388 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-08-01T19:34:46.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.498+0000 7f9b1b59e700 1 Processor -- start 2022-08-01T19:34:46.501 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.498+0000 7f9b1b59e700 1 -- start start 2022-08-01T19:34:46.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.498+0000 7f9b1b59e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9b1c0614b0 0x7f9b1c061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:46.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.498+0000 7f9b1b59e700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b1c0aee80 con 0x7f9b1c1405d0 2022-08-01T19:34:46.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.498+0000 7f9b1b59e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b1c0b70b0 con 0x7f9b1c0ad750 2022-08-01T19:34:46.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.498+0000 7f9b1b59e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b1c0af0b0 con 0x7f9b1c0614b0 2022-08-01T19:34:46.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.498+0000 7f9b1a59c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9b1c0614b0 0x7f9b1c061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:46.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.498+0000 7f9b1ad9d700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9b1c1405d0 0x7f9b1c1466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58024/0 (socket says 172.21.15.17:58024) 2022-08-01T19:34:46.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.498+0000 7f9b1ad9d700 1 -- 172.21.15.17:0/3400772736 learned_addr learned my addr 172.21.15.17:0/3400772736 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:46.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.498+0000 7f9b1959a700 1 -- 172.21.15.17:0/3400772736 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 962185586 0 0) 0x7f9b1c0aee80 con 0x7f9b1c1405d0 2022-08-01T19:34:46.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.498+0000 7f9b1959a700 1 -- 172.21.15.17:0/3400772736 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b00003260 con 0x7f9b1c1405d0 2022-08-01T19:34:46.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.498+0000 7f9b1959a700 1 -- 172.21.15.17:0/3400772736 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2402534302 0 0) 0x7f9b00003260 con 0x7f9b1c1405d0 2022-08-01T19:34:46.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.498+0000 7f9b1a59c700 1 -- 172.21.15.17:0/3400772736 >> v1:172.21.15.77:6789/0 conn(0x7f9b1c0ad750 legacy=0x7f9b1c0b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:46.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.498+0000 7f9b1a59c700 1 -- 172.21.15.17:0/3400772736 >> v1:172.21.15.17:6789/0 conn(0x7f9b1c1405d0 legacy=0x7f9b1c1466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:46.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.498+0000 7f9b1a59c700 1 -- 172.21.15.17:0/3400772736 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b1c0af350 con 0x7f9b1c0614b0 2022-08-01T19:34:46.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.498+0000 7f9b1a59c700 1 --2- 172.21.15.17:0/3400772736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b1c0614b0 0x7f9b1c061890 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f9b140017c0 tx=0x7f9b10006df0).ready entity=mon.0 client_cookie=9852fc88ff398264 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:46.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.498+0000 7f9b1959a700 1 -- 172.21.15.17:0/3400772736 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b14001660 con 0x7f9b1c0614b0 2022-08-01T19:34:46.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.498+0000 7f9b1959a700 1 -- 172.21.15.17:0/3400772736 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b10000e90 con 0x7f9b1c0614b0 2022-08-01T19:34:46.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.498+0000 7f9b1959a700 1 -- 172.21.15.17:0/3400772736 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b1000d4a0 con 0x7f9b1c0614b0 2022-08-01T19:34:46.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.498+0000 7f9b1b59e700 1 -- 172.21.15.17:0/3400772736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b1c0614b0 msgr2=0x7f9b1c061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:46.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.498+0000 7f9b1b59e700 1 --2- 172.21.15.17:0/3400772736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b1c0614b0 0x7f9b1c061890 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f9b140017c0 tx=0x7f9b10006df0).stop 2022-08-01T19:34:46.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.498+0000 7f9b1b59e700 1 -- 172.21.15.17:0/3400772736 shutdown_connections 2022-08-01T19:34:46.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.498+0000 7f9b1b59e700 1 --2- 172.21.15.17:0/3400772736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b1c0614b0 0x7f9b1c061890 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:46.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.498+0000 7f9b1b59e700 1 -- 172.21.15.17:0/3400772736 >> 172.21.15.17:0/3400772736 conn(0x7f9b1c066f70 msgr2=0x7f9b1c067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:46.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.498+0000 7f9b1b59e700 1 -- 172.21.15.17:0/3400772736 shutdown_connections 2022-08-01T19:34:46.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.498+0000 7f9b1b59e700 1 -- 172.21.15.17:0/3400772736 wait complete. 2022-08-01T19:34:46.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.498+0000 7f9b1b59e700 1 Processor -- start 2022-08-01T19:34:46.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.498+0000 7f9b1b59e700 1 -- start start 2022-08-01T19:34:46.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.502+0000 7f9b1b59e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b1c0614b0 0x7f9b1c152f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:46.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.502+0000 7f9b1b59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9b1c0ad750 0x7f9b1c0acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:46.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.502+0000 7f9b1b59e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b1c1405d0 0x7f9b1c0b3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:46.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.502+0000 7f9b1b59e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c1417b0 con 0x7f9b1c1405d0 2022-08-01T19:34:46.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.502+0000 7f9b1b59e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c0b0c70 con 0x7f9b1c0614b0 2022-08-01T19:34:46.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.502+0000 7f9b1b59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c0b0f10 con 0x7f9b1c0ad750 2022-08-01T19:34:46.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.502+0000 7f9b1ad9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b1c1405d0 0x7f9b1c0b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:46.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.502+0000 7f9b1ad9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b1c1405d0 0x7f9b1c0b3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37606/0 (socket says 172.21.15.17:37606) 2022-08-01T19:34:46.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.502+0000 7f9b1ad9d700 1 -- 172.21.15.17:0/4026410261 learned_addr learned my addr 172.21.15.17:0/4026410261 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:46.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.502+0000 7f9b1a59c700 1 --2- 172.21.15.17:0/4026410261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b1c0614b0 0x7f9b1c152f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:46.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.502+0000 7f9b19d9b700 1 --2- 172.21.15.17:0/4026410261 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9b1c0ad750 0x7f9b1c0acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:46.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.502+0000 7f9b1ad9d700 1 -- 172.21.15.17:0/4026410261 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9b1c0ad750 msgr2=0x7f9b1c0acdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:46.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.502+0000 7f9b1ad9d700 1 --2- 172.21.15.17:0/4026410261 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9b1c0ad750 0x7f9b1c0acdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:46.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.502+0000 7f9b1ad9d700 1 -- 172.21.15.17:0/4026410261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b1c0614b0 msgr2=0x7f9b1c152f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:46.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.502+0000 7f9b1ad9d700 1 --2- 172.21.15.17:0/4026410261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b1c0614b0 0x7f9b1c152f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:46.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.502+0000 7f9b1ad9d700 1 -- 172.21.15.17:0/4026410261 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b1000b180 con 0x7f9b1c1405d0 2022-08-01T19:34:46.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.502+0000 7f9b1ad9d700 1 --2- 172.21.15.17:0/4026410261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b1c1405d0 0x7f9b1c0b3680 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f9b14000ed0 tx=0x7f9b14001800).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:46.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.502+0000 7f9b0b7fe700 1 -- 172.21.15.17:0/4026410261 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b14007170 con 0x7f9b1c1405d0 2022-08-01T19:34:46.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.502+0000 7f9b0b7fe700 1 -- 172.21.15.17:0/4026410261 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b140072d0 con 0x7f9b1c1405d0 2022-08-01T19:34:46.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.502+0000 7f9b1b59e700 1 -- 172.21.15.17:0/4026410261 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b1c141150 con 0x7f9b1c1405d0 2022-08-01T19:34:46.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.502+0000 7f9b0b7fe700 1 -- 172.21.15.17:0/4026410261 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b14004710 con 0x7f9b1c1405d0 2022-08-01T19:34:46.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.502+0000 7f9b1b59e700 1 -- 172.21.15.17:0/4026410261 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b1c1441d0 con 0x7f9b1c1405d0 2022-08-01T19:34:46.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.502+0000 7f9b1b59e700 1 -- 172.21.15.17:0/4026410261 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b1c0b0ad0 con 0x7f9b1c1405d0 2022-08-01T19:34:46.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.506+0000 7f9b0b7fe700 1 -- 172.21.15.17:0/4026410261 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9b14004870 con 0x7f9b1c1405d0 2022-08-01T19:34:46.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.506+0000 7f9b0b7fe700 1 --2- 172.21.15.17:0/4026410261 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9af80539e0 0x7f9af8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:46.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.506+0000 7f9b0b7fe700 1 -- 172.21.15.17:0/4026410261 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f9b1406ba50 con 0x7f9b1c1405d0 2022-08-01T19:34:46.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.506+0000 7f9b1a59c700 1 --2- 172.21.15.17:0/4026410261 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9af80539e0 0x7f9af8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:46.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.506+0000 7f9b1a59c700 1 --2- 172.21.15.17:0/4026410261 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9af80539e0 0x7f9af8055ea0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f9b10019000 tx=0x7f9b1c0af4e0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:46.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.514+0000 7f9b0b7fe700 1 -- 172.21.15.17:0/4026410261 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9b1c0b0ad0 con 0x7f9b1c1405d0 2022-08-01T19:34:46.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.682+0000 7f9b1b59e700 1 -- 172.21.15.17:0/4026410261 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f9b1c0b0ad0 con 0x7f9b1c1405d0 2022-08-01T19:34:46.692 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.686+0000 7f9b0b7fe700 1 -- 172.21.15.17:0/4026410261 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 5 and data pool 6 v12) v1 ==== 187+0+0 (secure 0 0 0) 0x7f9b1c0b0ad0 con 0x7f9b1c1405d0 2022-08-01T19:34:46.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.690+0000 7f9b1b59e700 1 -- 172.21.15.17:0/4026410261 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9af80539e0 msgr2=0x7f9af8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:46.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.690+0000 7f9b1b59e700 1 --2- 172.21.15.17:0/4026410261 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9af80539e0 0x7f9af8055ea0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f9b10019000 tx=0x7f9b1c0af4e0).stop 2022-08-01T19:34:46.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.690+0000 7f9b1b59e700 1 -- 172.21.15.17:0/4026410261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b1c1405d0 msgr2=0x7f9b1c0b3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:46.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.690+0000 7f9b1b59e700 1 --2- 172.21.15.17:0/4026410261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b1c1405d0 0x7f9b1c0b3680 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f9b14000ed0 tx=0x7f9b14001800).stop 2022-08-01T19:34:46.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.690+0000 7f9b1b59e700 1 -- 172.21.15.17:0/4026410261 shutdown_connections 2022-08-01T19:34:46.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.690+0000 7f9b1b59e700 1 --2- 172.21.15.17:0/4026410261 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9af80539e0 0x7f9af8055ea0 secure :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0x7f9b10019000 tx=0x7f9b1c0af4e0).stop 2022-08-01T19:34:46.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.690+0000 7f9b1b59e700 1 --2- 172.21.15.17:0/4026410261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b1c1405d0 0x7f9b1c0b3680 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:46.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.690+0000 7f9b1b59e700 1 --2- 172.21.15.17:0/4026410261 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9b1c0ad750 0x7f9b1c0acdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:46.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.690+0000 7f9b1b59e700 1 --2- 172.21.15.17:0/4026410261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b1c0614b0 0x7f9b1c152f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:46.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.690+0000 7f9b1b59e700 1 -- 172.21.15.17:0/4026410261 >> 172.21.15.17:0/4026410261 conn(0x7f9b1c066f70 msgr2=0x7f9b1c067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:46.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.690+0000 7f9b1b59e700 1 -- 172.21.15.17:0/4026410261 shutdown_connections 2022-08-01T19:34:46.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.690+0000 7f9b1b59e700 1 -- 172.21.15.17:0/4026410261 wait complete. 2022-08-01T19:34:46.699 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 5 and data pool 6 2022-08-01T19:34:46.714 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:34:46.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.802+0000 7f3c57fff700 1 Processor -- start 2022-08-01T19:34:46.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.802+0000 7f3c57fff700 1 -- start start 2022-08-01T19:34:46.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.802+0000 7f3c57fff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f3c580b7840 0x7f3c58060be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:46.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.802+0000 7f3c57fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3c58143400 0x7f3c58143890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:46.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.802+0000 7f3c57fff700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c580671e0 con 0x7f3c580b2930 2022-08-01T19:34:46.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.802+0000 7f3c57fff700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c58065540 con 0x7f3c58143400 2022-08-01T19:34:46.810 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.802+0000 7f3c57fff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c580b7c20 con 0x7f3c580b7840 2022-08-01T19:34:46.810 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.802+0000 7f3c577fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3c58143400 0x7f3c58143890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:46.810 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.802+0000 7f3c577fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3c58143400 0x7f3c58143890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37624/0 (socket says 172.21.15.17:37624) 2022-08-01T19:34:46.810 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.802+0000 7f3c577fe700 1 -- 172.21.15.17:0/78685749 learned_addr learned my addr 172.21.15.17:0/78685749 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:46.811 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.802+0000 7f3c56ffd700 1 --2- 172.21.15.17:0/78685749 >> v2:172.21.15.77:3300/0 conn(0x7f3c580b7840 0x7f3c58060be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:46.811 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.802+0000 7f3c577fe700 1 -- 172.21.15.17:0/78685749 >> v2:172.21.15.77:3300/0 conn(0x7f3c580b7840 msgr2=0x7f3c58060be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:46.811 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.802+0000 7f3c577fe700 1 --2- 172.21.15.17:0/78685749 >> v2:172.21.15.77:3300/0 conn(0x7f3c580b7840 0x7f3c58060be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:46.811 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.802+0000 7f3c577fe700 1 -- 172.21.15.17:0/78685749 >> v1:172.21.15.17:6789/0 conn(0x7f3c580b2930 legacy=0x7f3c580610d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:46.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.802+0000 7f3c577fe700 1 -- 172.21.15.17:0/78685749 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c580617e0 con 0x7f3c58143400 2022-08-01T19:34:46.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.802+0000 7f3c577fe700 1 --2- 172.21.15.17:0/78685749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c58143400 0x7f3c58143890 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f3c50002f70 tx=0x7f3c50006eb0).ready entity=mon.0 client_cookie=1c0fd1b234e15389 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:46.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.802+0000 7f3c55ffb700 1 -- 172.21.15.17:0/78685749 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c5000d610 con 0x7f3c58143400 2022-08-01T19:34:46.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.802+0000 7f3c55ffb700 1 -- 172.21.15.17:0/78685749 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c50000e90 con 0x7f3c58143400 2022-08-01T19:34:46.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.802+0000 7f3c55ffb700 1 -- 172.21.15.17:0/78685749 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c5000d9a0 con 0x7f3c58143400 2022-08-01T19:34:46.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.802+0000 7f3c57fff700 1 -- 172.21.15.17:0/78685749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c58143400 msgr2=0x7f3c58143890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:46.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.802+0000 7f3c57fff700 1 --2- 172.21.15.17:0/78685749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c58143400 0x7f3c58143890 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f3c50002f70 tx=0x7f3c50006eb0).stop 2022-08-01T19:34:46.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.802+0000 7f3c57fff700 1 -- 172.21.15.17:0/78685749 shutdown_connections 2022-08-01T19:34:46.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.802+0000 7f3c57fff700 1 --2- 172.21.15.17:0/78685749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c58143400 0x7f3c58143890 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:46.814 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.802+0000 7f3c57fff700 1 --2- 172.21.15.17:0/78685749 >> v2:172.21.15.77:3300/0 conn(0x7f3c580b7840 0x7f3c58060be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:46.814 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.802+0000 7f3c57fff700 1 -- 172.21.15.17:0/78685749 >> 172.21.15.17:0/78685749 conn(0x7f3c5805f840 msgr2=0x7f3c5805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:46.814 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.802+0000 7f3c57fff700 1 -- 172.21.15.17:0/78685749 shutdown_connections 2022-08-01T19:34:46.814 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.802+0000 7f3c57fff700 1 -- 172.21.15.17:0/78685749 wait complete. 2022-08-01T19:34:46.817 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.806+0000 7f3c57fff700 1 Processor -- start 2022-08-01T19:34:46.817 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.806+0000 7f3c57fff700 1 -- start start 2022-08-01T19:34:46.818 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.806+0000 7f3c57fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3c580b2930 0x7f3c580b6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:46.818 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.806+0000 7f3c57fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c580b7840 0x7f3c58152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:46.818 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.806+0000 7f3c57fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c58143400 0x7f3c580b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:46.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.806+0000 7f3c57fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c580ac820 con 0x7f3c58143400 2022-08-01T19:34:46.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.806+0000 7f3c57fff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c581529f0 con 0x7f3c580b7840 2022-08-01T19:34:46.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.806+0000 7f3c57fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c580aca50 con 0x7f3c580b2930 2022-08-01T19:34:46.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.806+0000 7f3c577fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c58143400 0x7f3c580b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:46.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.806+0000 7f3c577fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c58143400 0x7f3c580b12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37632/0 (socket says 172.21.15.17:37632) 2022-08-01T19:34:46.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.806+0000 7f3c577fe700 1 -- 172.21.15.17:0/4133593814 learned_addr learned my addr 172.21.15.17:0/4133593814 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:46.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.806+0000 7f3c577fe700 1 -- 172.21.15.17:0/4133593814 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3c580b2930 msgr2=0x7f3c580b6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:46.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.806+0000 7f3c567fc700 1 --2- 172.21.15.17:0/4133593814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c580b7840 0x7f3c58152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:46.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.806+0000 7f3c56ffd700 1 --2- 172.21.15.17:0/4133593814 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3c580b2930 0x7f3c580b6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:46.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.806+0000 7f3c577fe700 1 --2- 172.21.15.17:0/4133593814 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3c580b2930 0x7f3c580b6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:46.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.806+0000 7f3c577fe700 1 -- 172.21.15.17:0/4133593814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c580b7840 msgr2=0x7f3c58152500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:46.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.806+0000 7f3c577fe700 1 --2- 172.21.15.17:0/4133593814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c580b7840 0x7f3c58152500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:46.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.806+0000 7f3c577fe700 1 -- 172.21.15.17:0/4133593814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c5000b1e0 con 0x7f3c58143400 2022-08-01T19:34:46.822 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.806+0000 7f3c567fc700 1 --2- 172.21.15.17:0/4133593814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c580b7840 0x7f3c58152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:34:46.822 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.806+0000 7f3c56ffd700 1 --2- 172.21.15.17:0/4133593814 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3c580b2930 0x7f3c580b6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:34:46.822 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.806+0000 7f3c577fe700 1 --2- 172.21.15.17:0/4133593814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c58143400 0x7f3c580b12d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f3c50002f70 tx=0x7f3c50015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:46.822 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.806+0000 7f3c4ffff700 1 -- 172.21.15.17:0/4133593814 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c500118a0 con 0x7f3c58143400 2022-08-01T19:34:46.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.806+0000 7f3c4ffff700 1 -- 172.21.15.17:0/4133593814 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c50011a00 con 0x7f3c58143400 2022-08-01T19:34:46.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.806+0000 7f3c57fff700 1 -- 172.21.15.17:0/4133593814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c580accf0 con 0x7f3c58143400 2022-08-01T19:34:46.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.806+0000 7f3c4ffff700 1 -- 172.21.15.17:0/4133593814 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c50011d60 con 0x7f3c58143400 2022-08-01T19:34:46.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.806+0000 7f3c57fff700 1 -- 172.21.15.17:0/4133593814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c58150140 con 0x7f3c58143400 2022-08-01T19:34:46.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.806+0000 7f3c57fff700 1 -- 172.21.15.17:0/4133593814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c580574e0 con 0x7f3c58143400 2022-08-01T19:34:46.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.810+0000 7f3c4ffff700 1 -- 172.21.15.17:0/4133593814 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3c50025030 con 0x7f3c58143400 2022-08-01T19:34:46.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.810+0000 7f3c4ffff700 1 --2- 172.21.15.17:0/4133593814 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3c3c053b30 0x7f3c3c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:46.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.810+0000 7f3c4ffff700 1 -- 172.21.15.17:0/4133593814 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f3c5006d200 con 0x7f3c58143400 2022-08-01T19:34:46.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.810+0000 7f3c56ffd700 1 --2- 172.21.15.17:0/4133593814 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3c3c053b30 0x7f3c3c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:46.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.810+0000 7f3c56ffd700 1 --2- 172.21.15.17:0/4133593814 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3c3c053b30 0x7f3c3c055ff0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f3c48008000 tx=0x7f3c48008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:46.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.814+0000 7f3c4ffff700 1 -- 172.21.15.17:0/4133593814 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3c50043fc0 con 0x7f3c58143400 2022-08-01T19:34:46.886 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:34:46.890 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:34:46.893 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:34:46.895 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:34:46.949 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.942+0000 7f3c57fff700 1 -- 172.21.15.17:0/4133593814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c58060330 con 0x7f3c58143400 2022-08-01T19:34:46.951 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.946+0000 7f3c4ffff700 1 -- 172.21.15.17:0/4133593814 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16284 (secure 0 0 0) 0x7f3c50023020 con 0x7f3c58143400 2022-08-01T19:34:46.951 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:46.952 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":34,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:34:46.687755+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:34:44.961205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2022-08-01T19:34:45.687600+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:34:46.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.946+0000 7f3c57fff700 1 -- 172.21.15.17:0/4133593814 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3c3c053b30 msgr2=0x7f3c3c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:46.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.946+0000 7f3c57fff700 1 --2- 172.21.15.17:0/4133593814 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3c3c053b30 0x7f3c3c055ff0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f3c48008000 tx=0x7f3c48008040).stop 2022-08-01T19:34:46.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.946+0000 7f3c57fff700 1 -- 172.21.15.17:0/4133593814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c58143400 msgr2=0x7f3c580b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:46.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.946+0000 7f3c57fff700 1 --2- 172.21.15.17:0/4133593814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c58143400 0x7f3c580b12d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f3c50002f70 tx=0x7f3c50015000).stop 2022-08-01T19:34:46.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.946+0000 7f3c57fff700 1 -- 172.21.15.17:0/4133593814 shutdown_connections 2022-08-01T19:34:46.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.946+0000 7f3c57fff700 1 --2- 172.21.15.17:0/4133593814 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3c3c053b30 0x7f3c3c055ff0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:46.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.946+0000 7f3c57fff700 1 --2- 172.21.15.17:0/4133593814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c58143400 0x7f3c580b12d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:46.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.946+0000 7f3c57fff700 1 --2- 172.21.15.17:0/4133593814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c580b7840 0x7f3c58152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:46.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.946+0000 7f3c57fff700 1 --2- 172.21.15.17:0/4133593814 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3c580b2930 0x7f3c580b6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:46.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.946+0000 7f3c57fff700 1 -- 172.21.15.17:0/4133593814 >> 172.21.15.17:0/4133593814 conn(0x7f3c5805f840 msgr2=0x7f3c580b6060 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:46.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.946+0000 7f3c57fff700 1 -- 172.21.15.17:0/4133593814 shutdown_connections 2022-08-01T19:34:46.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:46.946+0000 7f3c57fff700 1 -- 172.21.15.17:0/4133593814 wait complete. 2022-08-01T19:34:46.965 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:34:47.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb2a60dc700 1 Processor -- start 2022-08-01T19:34:47.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb2a60dc700 1 -- start start 2022-08-01T19:34:47.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb2a60dc700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fb2a00b2710 0x7fb2a014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:47.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb2a60dc700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb2a00a8c50 0x7fb2a014b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:47.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb2a60dc700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2a00a7250 con 0x7fb2a00b01a0 2022-08-01T19:34:47.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb2a60dc700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2a00a7480 con 0x7fb2a00b2710 2022-08-01T19:34:47.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb2a60dc700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a00a7720 con 0x7fb2a00a8c50 2022-08-01T19:34:47.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb2a48d9700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb2a00b01a0 0x7fb2a00a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58062/0 (socket says 172.21.15.17:58062) 2022-08-01T19:34:47.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb2a48d9700 1 -- 172.21.15.17:0/826467561 learned_addr learned my addr 172.21.15.17:0/826467561 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:47.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb2a58db700 1 --2- 172.21.15.17:0/826467561 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb2a00a8c50 0x7fb2a014b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:47.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb2a50da700 1 --2- 172.21.15.17:0/826467561 >> v2:172.21.15.77:3300/0 conn(0x7fb2a00b2710 0x7fb2a014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:47.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb297fff700 1 -- 172.21.15.17:0/826467561 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4073903868 0 0) 0x7fb2a00a7250 con 0x7fb2a00b01a0 2022-08-01T19:34:47.076 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb297fff700 1 -- 172.21.15.17:0/826467561 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb284003260 con 0x7fb2a00b01a0 2022-08-01T19:34:47.076 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb297fff700 1 -- 172.21.15.17:0/826467561 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2545308106 0 0) 0x7fb284003260 con 0x7fb2a00b01a0 2022-08-01T19:34:47.077 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb297fff700 1 -- 172.21.15.17:0/826467561 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb2a00a8c50 msgr2=0x7fb2a014b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:47.077 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb297fff700 1 --2- 172.21.15.17:0/826467561 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb2a00a8c50 0x7fb2a014b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:47.078 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb297fff700 1 -- 172.21.15.17:0/826467561 >> v2:172.21.15.77:3300/0 conn(0x7fb2a00b2710 msgr2=0x7fb2a014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:47.078 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb297fff700 1 --2- 172.21.15.17:0/826467561 >> v2:172.21.15.77:3300/0 conn(0x7fb2a00b2710 0x7fb2a014afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb297fff700 1 -- 172.21.15.17:0/826467561 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a00a79c0 con 0x7fb2a00b01a0 2022-08-01T19:34:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb297fff700 1 -- 172.21.15.17:0/826467561 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7fb290001660 con 0x7fb2a00b01a0 2022-08-01T19:34:47.080 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb297fff700 1 -- 172.21.15.17:0/826467561 >> v1:172.21.15.17:6789/0 conn(0x7fb2a00b01a0 legacy=0x7fb2a00a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:47.080 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb297fff700 1 --2- 172.21.15.17:0/826467561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb284003750 0x7fb284003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:47.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb297fff700 1 --2- 172.21.15.17:0/826467561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb284004500 0x7fb284004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:47.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb297fff700 1 --2- 172.21.15.17:0/826467561 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb2a00a8c50 0x7fb284004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:47.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb297fff700 1 -- 172.21.15.17:0/826467561 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a00a7480 con 0x7fb284003750 2022-08-01T19:34:47.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb297fff700 1 -- 172.21.15.17:0/826467561 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a00a7720 con 0x7fb284004500 2022-08-01T19:34:47.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb297fff700 1 -- 172.21.15.17:0/826467561 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb28400b750 con 0x7fb2a00a8c50 2022-08-01T19:34:47.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb2a50da700 1 --2- 172.21.15.17:0/826467561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb284003750 0x7fb284003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:47.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb2a48d9700 1 --2- 172.21.15.17:0/826467561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb284004500 0x7fb284004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:47.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb2a58db700 1 --2- 172.21.15.17:0/826467561 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb2a00a8c50 0x7fb284004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:47.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb2a50da700 1 -- 172.21.15.17:0/826467561 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb2a00a8c50 msgr2=0x7fb284004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:47.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb2a50da700 1 --2- 172.21.15.17:0/826467561 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb2a00a8c50 0x7fb284004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:47.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb2a50da700 1 -- 172.21.15.17:0/826467561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb284004500 msgr2=0x7fb284004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:47.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb2a50da700 1 --2- 172.21.15.17:0/826467561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb284004500 0x7fb284004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:47.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb2a50da700 1 -- 172.21.15.17:0/826467561 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb284003260 con 0x7fb284003750 2022-08-01T19:34:47.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb2a50da700 1 --2- 172.21.15.17:0/826467561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb284003750 0x7fb284003b70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7fb29c00c000 tx=0x7fb29c006df0).ready entity=mon.0 client_cookie=a918117fb1ec07fe server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:47.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb297fff700 1 -- 172.21.15.17:0/826467561 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb29c00e830 con 0x7fb284003750 2022-08-01T19:34:47.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb297fff700 1 -- 172.21.15.17:0/826467561 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb29c00e990 con 0x7fb284003750 2022-08-01T19:34:47.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.062+0000 7fb297fff700 1 -- 172.21.15.17:0/826467561 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb29c00ecf0 con 0x7fb284003750 2022-08-01T19:34:47.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.066+0000 7fb2a60dc700 1 -- 172.21.15.17:0/826467561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb284003750 msgr2=0x7fb284003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:47.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.066+0000 7fb2a60dc700 1 --2- 172.21.15.17:0/826467561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb284003750 0x7fb284003b70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7fb29c00c000 tx=0x7fb29c006df0).stop 2022-08-01T19:34:47.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.066+0000 7fb2a58db700 1 -- 172.21.15.17:0/826467561 reap_dead start 2022-08-01T19:34:47.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.066+0000 7fb2a60dc700 1 -- 172.21.15.17:0/826467561 shutdown_connections 2022-08-01T19:34:47.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.066+0000 7fb2a60dc700 1 -- 172.21.15.17:0/826467561 >> 172.21.15.17:0/826467561 conn(0x7fb2a0060310 msgr2=0x7fb2a00b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:47.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.066+0000 7fb2a60dc700 1 -- 172.21.15.17:0/826467561 shutdown_connections 2022-08-01T19:34:47.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.066+0000 7fb2a60dc700 1 -- 172.21.15.17:0/826467561 wait complete. 2022-08-01T19:34:47.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.066+0000 7fb2a60dc700 1 Processor -- start 2022-08-01T19:34:47.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.066+0000 7fb2a60dc700 1 -- start start 2022-08-01T19:34:47.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.066+0000 7fb2a60dc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb284003750 0x7fb2a0156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:47.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.066+0000 7fb2a60dc700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb2a0066d40 0x7fb2a005c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:47.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.066+0000 7fb2a60dc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2a005ca70 0x7fb2a015a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:47.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.066+0000 7fb2a60dc700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a00ac0f0 con 0x7fb2a005ca70 2022-08-01T19:34:47.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.066+0000 7fb2a60dc700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a0159de0 con 0x7fb284003750 2022-08-01T19:34:47.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.066+0000 7fb2a60dc700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a015a080 con 0x7fb2a0066d40 2022-08-01T19:34:47.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.066+0000 7fb2a58db700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2a005ca70 0x7fb2a015a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:47.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.066+0000 7fb2a50da700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb284003750 0x7fb2a0156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:47.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.066+0000 7fb2a48d9700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb2a0066d40 0x7fb2a005c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:47.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.066+0000 7fb2a58db700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2a005ca70 0x7fb2a015a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37652/0 (socket says 172.21.15.17:37652) 2022-08-01T19:34:47.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.066+0000 7fb2a48d9700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb2a0066d40 0x7fb2a005c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:33750/0 (socket says 172.21.15.17:33750) 2022-08-01T19:34:47.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.066+0000 7fb2a50da700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb284003750 0x7fb2a0156690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:35570/0 (socket says 172.21.15.17:35570) 2022-08-01T19:34:47.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.066+0000 7fb2a58db700 1 -- 172.21.15.17:0/2532992064 learned_addr learned my addr 172.21.15.17:0/2532992064 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:47.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.066+0000 7fb2a58db700 1 -- 172.21.15.17:0/2532992064 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb2a0066d40 msgr2=0x7fb2a005c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:47.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.066+0000 7fb2a58db700 1 --2- 172.21.15.17:0/2532992064 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb2a0066d40 0x7fb2a005c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:47.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.066+0000 7fb2a58db700 1 -- 172.21.15.17:0/2532992064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb284003750 msgr2=0x7fb2a0156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:47.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.066+0000 7fb2a58db700 1 --2- 172.21.15.17:0/2532992064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb284003750 0x7fb2a0156690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.066+0000 7fb2a58db700 1 -- 172.21.15.17:0/2532992064 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a0066b00 con 0x7fb2a005ca70 2022-08-01T19:34:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.070+0000 7fb2a58db700 1 --2- 172.21.15.17:0/2532992064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2a005ca70 0x7fb2a015a520 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7fb29800b000 tx=0x7fb2980091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:47.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.070+0000 7fb2967fc700 1 -- 172.21.15.17:0/2532992064 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb298019070 con 0x7fb2a005ca70 2022-08-01T19:34:47.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.070+0000 7fb2967fc700 1 -- 172.21.15.17:0/2532992064 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb298013750 con 0x7fb2a005ca70 2022-08-01T19:34:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.070+0000 7fb2a60dc700 1 -- 172.21.15.17:0/2532992064 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2a0156b80 con 0x7fb2a005ca70 2022-08-01T19:34:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.070+0000 7fb2967fc700 1 -- 172.21.15.17:0/2532992064 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb298013ab0 con 0x7fb2a005ca70 2022-08-01T19:34:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.070+0000 7fb2a60dc700 1 -- 172.21.15.17:0/2532992064 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a015aa10 con 0x7fb2a005ca70 2022-08-01T19:34:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.070+0000 7fb2a60dc700 1 -- 172.21.15.17:0/2532992064 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2a00574e0 con 0x7fb2a005ca70 2022-08-01T19:34:47.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.070+0000 7fb2967fc700 1 -- 172.21.15.17:0/2532992064 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb298013c10 con 0x7fb2a005ca70 2022-08-01T19:34:47.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.070+0000 7fb2967fc700 1 --2- 172.21.15.17:0/2532992064 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb288053aa0 0x7fb288055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:47.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.070+0000 7fb2967fc700 1 -- 172.21.15.17:0/2532992064 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb29806ca50 con 0x7fb2a005ca70 2022-08-01T19:34:47.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.074+0000 7fb2a50da700 1 --2- 172.21.15.17:0/2532992064 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb288053aa0 0x7fb288055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.074+0000 7fb2a50da700 1 --2- 172.21.15.17:0/2532992064 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb288053aa0 0x7fb288055f60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7fb29c01c000 tx=0x7fb29c019000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.078+0000 7fb2967fc700 1 -- 172.21.15.17:0/2532992064 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb2980436a0 con 0x7fb2a005ca70 2022-08-01T19:34:47.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.206+0000 7fb2a60dc700 1 -- 172.21.15.17:0/2532992064 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2a0069a70 con 0x7fb2a005ca70 2022-08-01T19:34:47.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.206+0000 7fb2967fc700 1 -- 172.21.15.17:0/2532992064 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16284 (secure 0 0 0) 0x7fb298046cc0 con 0x7fb2a005ca70 2022-08-01T19:34:47.213 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:47.213 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":34,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:34:46.687755+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:34:44.961205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2022-08-01T19:34:45.687600+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:34:47.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.210+0000 7fb2a60dc700 1 -- 172.21.15.17:0/2532992064 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb288053aa0 msgr2=0x7fb288055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:47.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.210+0000 7fb2a60dc700 1 --2- 172.21.15.17:0/2532992064 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb288053aa0 0x7fb288055f60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7fb29c01c000 tx=0x7fb29c019000).stop 2022-08-01T19:34:47.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.210+0000 7fb2a60dc700 1 -- 172.21.15.17:0/2532992064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2a005ca70 msgr2=0x7fb2a015a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:47.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.210+0000 7fb2a60dc700 1 --2- 172.21.15.17:0/2532992064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2a005ca70 0x7fb2a015a520 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7fb29800b000 tx=0x7fb2980091d0).stop 2022-08-01T19:34:47.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.210+0000 7fb2a60dc700 1 -- 172.21.15.17:0/2532992064 shutdown_connections 2022-08-01T19:34:47.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.210+0000 7fb2a60dc700 1 --2- 172.21.15.17:0/2532992064 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb288053aa0 0x7fb288055f60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:47.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.210+0000 7fb2a60dc700 1 --2- 172.21.15.17:0/2532992064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2a005ca70 0x7fb2a015a520 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:47.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.210+0000 7fb2a60dc700 1 --2- 172.21.15.17:0/2532992064 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb2a0066d40 0x7fb2a005c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:47.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.210+0000 7fb2a60dc700 1 --2- 172.21.15.17:0/2532992064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb284003750 0x7fb2a0156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:47.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.210+0000 7fb2a60dc700 1 -- 172.21.15.17:0/2532992064 >> 172.21.15.17:0/2532992064 conn(0x7fb2a0060310 msgr2=0x7fb2a005cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:47.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.210+0000 7fb2a60dc700 1 -- 172.21.15.17:0/2532992064 shutdown_connections 2022-08-01T19:34:47.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.210+0000 7fb2a60dc700 1 -- 172.21.15.17:0/2532992064 wait complete. 2022-08-01T19:34:47.227 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-08-01T19:34:47.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.334+0000 7f5245357700 1 Processor -- start 2022-08-01T19:34:47.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.334+0000 7f5245357700 1 -- start start 2022-08-01T19:34:47.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.334+0000 7f5245357700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f52400b4310 0x7f5240140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:47.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.334+0000 7f5245357700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5240143cf0 con 0x7f52400b4820 2022-08-01T19:34:47.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.334+0000 7f5245357700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5240143f90 con 0x7f5240146ae0 2022-08-01T19:34:47.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.334+0000 7f5245357700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52401476a0 con 0x7f52400b4310 2022-08-01T19:34:47.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.334+0000 7f523f7fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f52400b4820 0x7f52400b5060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58086/0 (socket says 172.21.15.17:58086) 2022-08-01T19:34:47.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.334+0000 7f523f7fe700 1 -- 172.21.15.17:0/980258589 learned_addr learned my addr 172.21.15.17:0/980258589 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:47.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.334+0000 7f523e7fc700 1 --2- 172.21.15.17:0/980258589 >> v2:172.21.15.77:3300/0 conn(0x7f52400b4310 0x7f5240140ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:47.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.334+0000 7f523dffb700 1 -- 172.21.15.17:0/980258589 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3474048851 0 0) 0x7f5240143cf0 con 0x7f52400b4820 2022-08-01T19:34:47.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.334+0000 7f523dffb700 1 -- 172.21.15.17:0/980258589 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5220003260 con 0x7f52400b4820 2022-08-01T19:34:47.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.334+0000 7f523e7fc700 1 -- 172.21.15.17:0/980258589 >> v1:172.21.15.77:6789/0 conn(0x7f5240146ae0 legacy=0x7f52401403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:47.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.334+0000 7f523e7fc700 1 -- 172.21.15.17:0/980258589 >> v1:172.21.15.17:6789/0 conn(0x7f52400b4820 legacy=0x7f52400b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:47.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.334+0000 7f523e7fc700 1 -- 172.21.15.17:0/980258589 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52400b0e20 con 0x7f52400b4310 2022-08-01T19:34:47.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.334+0000 7f523e7fc700 1 --2- 172.21.15.17:0/980258589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52400b4310 0x7f5240140ad0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f5228002f70 tx=0x7f5228006df0).ready entity=mon.1 client_cookie=858c137fc87a35e2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:47.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.334+0000 7f523dffb700 1 -- 172.21.15.17:0/980258589 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f522800d610 con 0x7f52400b4310 2022-08-01T19:34:47.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.334+0000 7f523dffb700 1 -- 172.21.15.17:0/980258589 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5228000e90 con 0x7f52400b4310 2022-08-01T19:34:47.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.334+0000 7f523dffb700 1 -- 172.21.15.17:0/980258589 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f522800d960 con 0x7f52400b4310 2022-08-01T19:34:47.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.334+0000 7f5245357700 1 -- 172.21.15.17:0/980258589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52400b4310 msgr2=0x7f5240140ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:47.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.334+0000 7f5245357700 1 --2- 172.21.15.17:0/980258589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52400b4310 0x7f5240140ad0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f5228002f70 tx=0x7f5228006df0).stop 2022-08-01T19:34:47.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.334+0000 7f5245357700 1 -- 172.21.15.17:0/980258589 shutdown_connections 2022-08-01T19:34:47.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.334+0000 7f5245357700 1 --2- 172.21.15.17:0/980258589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52400b4310 0x7f5240140ad0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:47.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.334+0000 7f5245357700 1 -- 172.21.15.17:0/980258589 >> 172.21.15.17:0/980258589 conn(0x7f524005c620 msgr2=0x7f524005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:47.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.334+0000 7f5245357700 1 -- 172.21.15.17:0/980258589 shutdown_connections 2022-08-01T19:34:47.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.334+0000 7f5245357700 1 -- 172.21.15.17:0/980258589 wait complete. 2022-08-01T19:34:47.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.338+0000 7f5245357700 1 Processor -- start 2022-08-01T19:34:47.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.338+0000 7f5245357700 1 -- start start 2022-08-01T19:34:47.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.338+0000 7f5245357700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52400b4310 0x7f5240145ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:47.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.338+0000 7f5245357700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f52400b4820 0x7f52400b8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:47.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.338+0000 7f5245357700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5240146ae0 0x7f52401458b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:47.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.338+0000 7f5245357700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52401485a0 con 0x7f52400b4310 2022-08-01T19:34:47.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.338+0000 7f5245357700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5240147dc0 con 0x7f5240146ae0 2022-08-01T19:34:47.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.338+0000 7f5245357700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5240148060 con 0x7f52400b4820 2022-08-01T19:34:47.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.338+0000 7f523effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52400b4310 0x7f5240145ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:47.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.338+0000 7f523effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52400b4310 0x7f5240145ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37662/0 (socket says 172.21.15.17:37662) 2022-08-01T19:34:47.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.338+0000 7f523effd700 1 -- 172.21.15.17:0/3329977064 learned_addr learned my addr 172.21.15.17:0/3329977064 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:47.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.338+0000 7f523f7fe700 1 --2- 172.21.15.17:0/3329977064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5240146ae0 0x7f52401458b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:47.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.338+0000 7f523e7fc700 1 --2- 172.21.15.17:0/3329977064 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f52400b4820 0x7f52400b8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:47.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.338+0000 7f523effd700 1 -- 172.21.15.17:0/3329977064 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f52400b4820 msgr2=0x7f52400b8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:47.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.338+0000 7f523effd700 1 --2- 172.21.15.17:0/3329977064 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f52400b4820 0x7f52400b8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:47.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.338+0000 7f523effd700 1 -- 172.21.15.17:0/3329977064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5240146ae0 msgr2=0x7f52401458b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:47.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.338+0000 7f523effd700 1 --2- 172.21.15.17:0/3329977064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5240146ae0 0x7f52401458b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:47.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.338+0000 7f523effd700 1 -- 172.21.15.17:0/3329977064 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f522800b180 con 0x7f52400b4310 2022-08-01T19:34:47.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.338+0000 7f523effd700 1 --2- 172.21.15.17:0/3329977064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52400b4310 0x7f5240145ea0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f5240060e20 tx=0x7f5230007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:47.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.338+0000 7f5237fff700 1 -- 172.21.15.17:0/3329977064 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5230012400 con 0x7f52400b4310 2022-08-01T19:34:47.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.338+0000 7f5237fff700 1 -- 172.21.15.17:0/3329977064 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5230012560 con 0x7f52400b4310 2022-08-01T19:34:47.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.338+0000 7f5245357700 1 -- 172.21.15.17:0/3329977064 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5240148300 con 0x7f52400b4310 2022-08-01T19:34:47.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.338+0000 7f5237fff700 1 -- 172.21.15.17:0/3329977064 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52300128c0 con 0x7f52400b4310 2022-08-01T19:34:47.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.338+0000 7f5245357700 1 -- 172.21.15.17:0/3329977064 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52401446e0 con 0x7f52400b4310 2022-08-01T19:34:47.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.342+0000 7f5245357700 1 -- 172.21.15.17:0/3329977064 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52400574e0 con 0x7f52400b4310 2022-08-01T19:34:47.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.342+0000 7f5237fff700 1 -- 172.21.15.17:0/3329977064 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5230019030 con 0x7f52400b4310 2022-08-01T19:34:47.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.342+0000 7f5237fff700 1 --2- 172.21.15.17:0/3329977064 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f52240539e0 0x7f5224055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:47.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.342+0000 7f5237fff700 1 -- 172.21.15.17:0/3329977064 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f523006b400 con 0x7f52400b4310 2022-08-01T19:34:47.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.342+0000 7f523e7fc700 1 --2- 172.21.15.17:0/3329977064 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f52240539e0 0x7f5224055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:47.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.342+0000 7f523e7fc700 1 --2- 172.21.15.17:0/3329977064 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f52240539e0 0x7f5224055ea0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f522801a000 tx=0x7f5228014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:47.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.350+0000 7f5237fff700 1 -- 172.21.15.17:0/3329977064 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f52300420b0 con 0x7f52400b4310 2022-08-01T19:34:47.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.510+0000 7f5245357700 1 -- 172.21.15.17:0/3329977064 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f52400b9cb0 con 0x7f52400b4310 2022-08-01T19:34:47.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.690+0000 7f5237fff700 1 -- 172.21.15.17:0/3329977064 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v13) v1 ==== 118+0+0 (secure 0 0 0) 0x7f52300456d0 con 0x7f52400b4310 2022-08-01T19:34:47.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.694+0000 7f5245357700 1 -- 172.21.15.17:0/3329977064 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f52240539e0 msgr2=0x7f5224055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:47.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.694+0000 7f5245357700 1 --2- 172.21.15.17:0/3329977064 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f52240539e0 0x7f5224055ea0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f522801a000 tx=0x7f5228014000).stop 2022-08-01T19:34:47.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.694+0000 7f5245357700 1 -- 172.21.15.17:0/3329977064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52400b4310 msgr2=0x7f5240145ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:47.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.694+0000 7f5245357700 1 --2- 172.21.15.17:0/3329977064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52400b4310 0x7f5240145ea0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f5240060e20 tx=0x7f5230007c50).stop 2022-08-01T19:34:47.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.694+0000 7f5245357700 1 -- 172.21.15.17:0/3329977064 shutdown_connections 2022-08-01T19:34:47.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.694+0000 7f5245357700 1 --2- 172.21.15.17:0/3329977064 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f52240539e0 0x7f5224055ea0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:47.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.694+0000 7f5245357700 1 --2- 172.21.15.17:0/3329977064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5240146ae0 0x7f52401458b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:47.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.694+0000 7f5245357700 1 --2- 172.21.15.17:0/3329977064 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f52400b4820 0x7f52400b8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:47.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.694+0000 7f5245357700 1 --2- 172.21.15.17:0/3329977064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52400b4310 0x7f5240145ea0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:47.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.694+0000 7f5245357700 1 -- 172.21.15.17:0/3329977064 >> 172.21.15.17:0/3329977064 conn(0x7f524005c620 msgr2=0x7f52400616e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:47.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.694+0000 7f5245357700 1 -- 172.21.15.17:0/3329977064 shutdown_connections 2022-08-01T19:34:47.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.694+0000 7f5245357700 1 -- 172.21.15.17:0/3329977064 wait complete. 2022-08-01T19:34:47.714 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:34:47.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.818+0000 7f00e03f2700 1 Processor -- start 2022-08-01T19:34:47.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.818+0000 7f00e03f2700 1 -- start start 2022-08-01T19:34:47.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00e03f2700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f00d80b1f40 0x7f00d80b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:47.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00e03f2700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f00d8148530 0x7f00d80b2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:47.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00e03f2700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d80b04e0 con 0x7f00d80a7250 2022-08-01T19:34:47.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00e03f2700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00d80a9250 con 0x7f00d80b1f40 2022-08-01T19:34:47.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00e03f2700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00d80b0780 con 0x7f00d8148530 2022-08-01T19:34:47.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00de18e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f00d80b1f40 0x7f00d80b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:47.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00de98f700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f00d80a7250 0x7f00d81448b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58100/0 (socket says 172.21.15.17:58100) 2022-08-01T19:34:47.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00de98f700 1 -- 172.21.15.17:0/1861841202 learned_addr learned my addr 172.21.15.17:0/1861841202 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:47.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00dd98d700 1 --2- 172.21.15.17:0/1861841202 >> v2:172.21.15.77:3300/0 conn(0x7f00d8148530 0x7f00d80b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:47.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00de18e700 1 -- 172.21.15.17:0/1861841202 >> v2:172.21.15.77:3300/0 conn(0x7f00d8148530 msgr2=0x7f00d80b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:47.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00de18e700 1 --2- 172.21.15.17:0/1861841202 >> v2:172.21.15.77:3300/0 conn(0x7f00d8148530 0x7f00d80b2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:47.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00de18e700 1 -- 172.21.15.17:0/1861841202 >> v1:172.21.15.17:6789/0 conn(0x7f00d80a7250 legacy=0x7f00d81448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:47.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00dd18c700 1 -- 172.21.15.17:0/1861841202 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3400558889 0 0) 0x7f00d80b04e0 con 0x7f00d80a7250 2022-08-01T19:34:47.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00de18e700 1 -- 172.21.15.17:0/1861841202 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d80b0a20 con 0x7f00d80b1f40 2022-08-01T19:34:47.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00dd98d700 1 --2- 172.21.15.17:0/1861841202 >> v2:172.21.15.77:3300/0 conn(0x7f00d8148530 0x7f00d80b2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:34:47.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00de18e700 1 --2- 172.21.15.17:0/1861841202 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00d80b1f40 0x7f00d80b1160 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f00d81478c0 tx=0x7f00c8006df0).ready entity=mon.0 client_cookie=ca4a9a632694a99b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:47.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00dd18c700 1 -- 172.21.15.17:0/1861841202 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00c800d5c0 con 0x7f00d80b1f40 2022-08-01T19:34:47.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00dd18c700 1 -- 172.21.15.17:0/1861841202 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c8000e90 con 0x7f00d80b1f40 2022-08-01T19:34:47.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00dd18c700 1 -- 172.21.15.17:0/1861841202 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00c800d910 con 0x7f00d80b1f40 2022-08-01T19:34:47.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00e03f2700 1 -- 172.21.15.17:0/1861841202 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00d80b1f40 msgr2=0x7f00d80b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:47.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00e03f2700 1 --2- 172.21.15.17:0/1861841202 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00d80b1f40 0x7f00d80b1160 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f00d81478c0 tx=0x7f00c8006df0).stop 2022-08-01T19:34:47.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00e03f2700 1 -- 172.21.15.17:0/1861841202 shutdown_connections 2022-08-01T19:34:47.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00e03f2700 1 --2- 172.21.15.17:0/1861841202 >> v2:172.21.15.77:3300/0 conn(0x7f00d8148530 0x7f00d80b2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:47.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00e03f2700 1 --2- 172.21.15.17:0/1861841202 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00d80b1f40 0x7f00d80b1160 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:47.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00e03f2700 1 -- 172.21.15.17:0/1861841202 >> 172.21.15.17:0/1861841202 conn(0x7f00d805f840 msgr2=0x7f00d805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:47.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00e03f2700 1 -- 172.21.15.17:0/1861841202 shutdown_connections 2022-08-01T19:34:47.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00e03f2700 1 -- 172.21.15.17:0/1861841202 wait complete. 2022-08-01T19:34:47.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00e03f2700 1 Processor -- start 2022-08-01T19:34:47.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00e03f2700 1 -- start start 2022-08-01T19:34:47.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00e03f2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00d80a7250 0x7f00d80a9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:47.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00e03f2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00d80b1f40 0x7f00d80abe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:47.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00e03f2700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f00d8148530 0x7f00d80ac300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:47.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00e03f2700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d814a2a0 con 0x7f00d80b1f40 2022-08-01T19:34:47.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00e03f2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d814a9e0 con 0x7f00d80a7250 2022-08-01T19:34:47.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00e03f2700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d80abb60 con 0x7f00d8148530 2022-08-01T19:34:47.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00dd98d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00d80b1f40 0x7f00d80abe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:47.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00de18e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00d80a7250 0x7f00d80a9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:47.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00dd98d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00d80b1f40 0x7f00d80abe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37678/0 (socket says 172.21.15.17:37678) 2022-08-01T19:34:47.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00dd98d700 1 -- 172.21.15.17:0/3821588907 learned_addr learned my addr 172.21.15.17:0/3821588907 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:47.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.822+0000 7f00de98f700 1 --2- 172.21.15.17:0/3821588907 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f00d8148530 0x7f00d80ac300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:47.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.826+0000 7f00dd98d700 1 -- 172.21.15.17:0/3821588907 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f00d8148530 msgr2=0x7f00d80ac300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:47.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.826+0000 7f00dd98d700 1 --2- 172.21.15.17:0/3821588907 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f00d8148530 0x7f00d80ac300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:47.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.826+0000 7f00dd98d700 1 -- 172.21.15.17:0/3821588907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00d80a7250 msgr2=0x7f00d80a9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:47.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.826+0000 7f00dd98d700 1 --2- 172.21.15.17:0/3821588907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00d80a7250 0x7f00d80a9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:47.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.826+0000 7f00dd98d700 1 -- 172.21.15.17:0/3821588907 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d400b180 con 0x7f00d80b1f40 2022-08-01T19:34:47.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.826+0000 7f00dd98d700 1 --2- 172.21.15.17:0/3821588907 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00d80b1f40 0x7f00d80abe10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f00d400c000 tx=0x7f00d4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:47.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.826+0000 7f00c77fe700 1 -- 172.21.15.17:0/3821588907 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00d4010780 con 0x7f00d80b1f40 2022-08-01T19:34:47.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.826+0000 7f00c77fe700 1 -- 172.21.15.17:0/3821588907 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d40108e0 con 0x7f00d80b1f40 2022-08-01T19:34:47.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.826+0000 7f00e03f2700 1 -- 172.21.15.17:0/3821588907 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00c800b180 con 0x7f00d80b1f40 2022-08-01T19:34:47.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.826+0000 7f00c77fe700 1 -- 172.21.15.17:0/3821588907 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00d4010c40 con 0x7f00d80b1f40 2022-08-01T19:34:47.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.826+0000 7f00e03f2700 1 -- 172.21.15.17:0/3821588907 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d814b930 con 0x7f00d80b1f40 2022-08-01T19:34:47.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.826+0000 7f00c77fe700 1 -- 172.21.15.17:0/3821588907 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f00d401c460 con 0x7f00d80b1f40 2022-08-01T19:34:47.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.826+0000 7f00e03f2700 1 -- 172.21.15.17:0/3821588907 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00d80b0cc0 con 0x7f00d80b1f40 2022-08-01T19:34:47.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.826+0000 7f00c77fe700 1 --2- 172.21.15.17:0/3821588907 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f00b8053aa0 0x7f00b8055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:47.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.826+0000 7f00c77fe700 1 -- 172.21.15.17:0/3821588907 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f00d406c630 con 0x7f00d80b1f40 2022-08-01T19:34:47.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.826+0000 7f00de18e700 1 --2- 172.21.15.17:0/3821588907 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f00b8053aa0 0x7f00b8055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:47.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.830+0000 7f00de18e700 1 --2- 172.21.15.17:0/3821588907 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f00b8053aa0 0x7f00b8055f60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f00c801a000 tx=0x7f00c8014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:47.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.834+0000 7f00c77fe700 1 -- 172.21.15.17:0/3821588907 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f00d4043200 con 0x7f00d80b1f40 2022-08-01T19:34:47.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.994+0000 7f00e03f2700 1 -- 172.21.15.17:0/3821588907 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00d814a470 con 0x7f00d80b1f40 2022-08-01T19:34:48.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.994+0000 7f00c77fe700 1 -- 172.21.15.17:0/3821588907 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+2491 (secure 0 0 0) 0x7f00d4046820 con 0x7f00d80b1f40 2022-08-01T19:34:48.000 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:48.001 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2487504686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2487504686},{"type":"v1","addr":"172.21.15.17:6833","nonce":2487504686}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10},{"gid":4885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/4255478434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4255478434},{"type":"v1","addr":"172.21.15.77:6835","nonce":4255478434}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":11}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:34:46.688039+0000","modified":"2022-08-01T19:34:47.690492+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4890},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4890":{"gid":4890,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/3876538769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":3876538769},{"type":"v1","addr":"172.21.15.17:6835","nonce":3876538769}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-08-01T19:34:48.002 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.998+0000 7f00e03f2700 1 -- 172.21.15.17:0/3821588907 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f00b8053aa0 msgr2=0x7f00b8055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:48.003 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.998+0000 7f00e03f2700 1 --2- 172.21.15.17:0/3821588907 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f00b8053aa0 0x7f00b8055f60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f00c801a000 tx=0x7f00c8014000).stop 2022-08-01T19:34:48.003 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.998+0000 7f00e03f2700 1 -- 172.21.15.17:0/3821588907 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00d80b1f40 msgr2=0x7f00d80abe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:48.003 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.998+0000 7f00e03f2700 1 --2- 172.21.15.17:0/3821588907 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00d80b1f40 0x7f00d80abe10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f00d400c000 tx=0x7f00d4006df0).stop 2022-08-01T19:34:48.003 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.998+0000 7f00e03f2700 1 -- 172.21.15.17:0/3821588907 shutdown_connections 2022-08-01T19:34:48.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.998+0000 7f00e03f2700 1 --2- 172.21.15.17:0/3821588907 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f00b8053aa0 0x7f00b8055f60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:48.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.998+0000 7f00e03f2700 1 --2- 172.21.15.17:0/3821588907 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f00d8148530 0x7f00d80ac300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:48.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.998+0000 7f00e03f2700 1 --2- 172.21.15.17:0/3821588907 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00d80b1f40 0x7f00d80abe10 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:48.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.998+0000 7f00e03f2700 1 --2- 172.21.15.17:0/3821588907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00d80a7250 0x7f00d80a9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:48.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.998+0000 7f00e03f2700 1 -- 172.21.15.17:0/3821588907 >> 172.21.15.17:0/3821588907 conn(0x7f00d805f840 msgr2=0x7f00d80b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:48.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.998+0000 7f00e03f2700 1 -- 172.21.15.17:0/3821588907 shutdown_connections 2022-08-01T19:34:48.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:47.998+0000 7f00e03f2700 1 -- 172.21.15.17:0/3821588907 wait complete. 2022-08-01T19:34:48.005 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 13 2022-08-01T19:34:48.013 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:34:48.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.110+0000 7f218184d700 1 Processor -- start 2022-08-01T19:34:48.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.110+0000 7f218184d700 1 -- start start 2022-08-01T19:34:48.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.110+0000 7f218184d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f217c0614b0 0x7f217c061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:48.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.110+0000 7f218184d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f217c0ad750 0x7f217c0b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:48.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.110+0000 7f218184d700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f217c0aee80 con 0x7f217c1405d0 2022-08-01T19:34:48.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.110+0000 7f218184d700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f217c0b70b0 con 0x7f217c0614b0 2022-08-01T19:34:48.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.110+0000 7f218184d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217c0af0b0 con 0x7f217c0ad750 2022-08-01T19:34:48.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.110+0000 7f218104c700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f217c1405d0 0x7f217c1466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58112/0 (socket says 172.21.15.17:58112) 2022-08-01T19:34:48.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.110+0000 7f218084b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f217c0614b0 0x7f217c061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:48.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.110+0000 7f218104c700 1 -- 172.21.15.17:0/2273184013 learned_addr learned my addr 172.21.15.17:0/2273184013 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:48.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.110+0000 7f217bfff700 1 --2- 172.21.15.17:0/2273184013 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f217c0ad750 0x7f217c0b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:48.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.110+0000 7f218084b700 1 -- 172.21.15.17:0/2273184013 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f217c0ad750 msgr2=0x7f217c0b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:48.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.110+0000 7f218084b700 1 --2- 172.21.15.17:0/2273184013 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f217c0ad750 0x7f217c0b9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:48.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.110+0000 7f218084b700 1 -- 172.21.15.17:0/2273184013 >> v1:172.21.15.17:6789/0 conn(0x7f217c1405d0 legacy=0x7f217c1466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:48.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.110+0000 7f218084b700 1 -- 172.21.15.17:0/2273184013 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217c0af350 con 0x7f217c0614b0 2022-08-01T19:34:48.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.110+0000 7f218084b700 1 --2- 172.21.15.17:0/2273184013 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f217c0614b0 0x7f217c061890 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f2164002f70 tx=0x7f2164006df0).ready entity=mon.0 client_cookie=87cc6b955c26aee8 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:48.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.110+0000 7f217b7fe700 1 -- 172.21.15.17:0/2273184013 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f216400d570 con 0x7f217c0614b0 2022-08-01T19:34:48.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.110+0000 7f217b7fe700 1 -- 172.21.15.17:0/2273184013 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2164000e90 con 0x7f217c0614b0 2022-08-01T19:34:48.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.110+0000 7f217b7fe700 1 -- 172.21.15.17:0/2273184013 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f216400d8c0 con 0x7f217c0614b0 2022-08-01T19:34:48.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.110+0000 7f218184d700 1 -- 172.21.15.17:0/2273184013 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f217c0614b0 msgr2=0x7f217c061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:48.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.110+0000 7f218184d700 1 --2- 172.21.15.17:0/2273184013 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f217c0614b0 0x7f217c061890 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f2164002f70 tx=0x7f2164006df0).stop 2022-08-01T19:34:48.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.110+0000 7f218184d700 1 -- 172.21.15.17:0/2273184013 shutdown_connections 2022-08-01T19:34:48.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.110+0000 7f218184d700 1 --2- 172.21.15.17:0/2273184013 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f217c0ad750 0x7f217c0b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:48.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.110+0000 7f218184d700 1 --2- 172.21.15.17:0/2273184013 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f217c0614b0 0x7f217c061890 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:48.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.110+0000 7f218184d700 1 -- 172.21.15.17:0/2273184013 >> 172.21.15.17:0/2273184013 conn(0x7f217c066f70 msgr2=0x7f217c067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:48.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.110+0000 7f218184d700 1 -- 172.21.15.17:0/2273184013 shutdown_connections 2022-08-01T19:34:48.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.110+0000 7f218184d700 1 -- 172.21.15.17:0/2273184013 wait complete. 2022-08-01T19:34:48.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.110+0000 7f218184d700 1 Processor -- start 2022-08-01T19:34:48.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.110+0000 7f218184d700 1 -- start start 2022-08-01T19:34:48.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.114+0000 7f218184d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f217c0614b0 0x7f217c141820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:48.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.114+0000 7f218184d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f217c0ad750 0x7f217c152e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:48.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.114+0000 7f218184d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f217c1405d0 0x7f217c1532f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:48.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.114+0000 7f218184d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c1537e0 con 0x7f217c1405d0 2022-08-01T19:34:48.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.114+0000 7f218184d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c0b2610 con 0x7f217c0614b0 2022-08-01T19:34:48.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.114+0000 7f218184d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217c0b28b0 con 0x7f217c0ad750 2022-08-01T19:34:48.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.114+0000 7f218104c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f217c1405d0 0x7f217c1532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:48.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.114+0000 7f218104c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f217c1405d0 0x7f217c1532f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37694/0 (socket says 172.21.15.17:37694) 2022-08-01T19:34:48.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.114+0000 7f218104c700 1 -- 172.21.15.17:0/2933897447 learned_addr learned my addr 172.21.15.17:0/2933897447 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:48.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.114+0000 7f218084b700 1 --2- 172.21.15.17:0/2933897447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f217c0614b0 0x7f217c141820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:48.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.114+0000 7f217bfff700 1 --2- 172.21.15.17:0/2933897447 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f217c0ad750 0x7f217c152e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:48.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.114+0000 7f218104c700 1 -- 172.21.15.17:0/2933897447 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f217c0ad750 msgr2=0x7f217c152e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:48.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.114+0000 7f218104c700 1 --2- 172.21.15.17:0/2933897447 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f217c0ad750 0x7f217c152e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:48.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.114+0000 7f218104c700 1 -- 172.21.15.17:0/2933897447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f217c0614b0 msgr2=0x7f217c141820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:48.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.114+0000 7f218104c700 1 --2- 172.21.15.17:0/2933897447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f217c0614b0 0x7f217c141820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:48.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.114+0000 7f218104c700 1 -- 172.21.15.17:0/2933897447 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f216400b180 con 0x7f217c1405d0 2022-08-01T19:34:48.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.114+0000 7f218104c700 1 --2- 172.21.15.17:0/2933897447 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f217c1405d0 0x7f217c1532f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f217c143190 tx=0x7f2174007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:48.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.114+0000 7f2179ffb700 1 -- 172.21.15.17:0/2933897447 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2174001d70 con 0x7f217c1405d0 2022-08-01T19:34:48.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.114+0000 7f2179ffb700 1 -- 172.21.15.17:0/2933897447 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21740070d0 con 0x7f217c1405d0 2022-08-01T19:34:48.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.114+0000 7f218184d700 1 -- 172.21.15.17:0/2933897447 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f217c0b2b50 con 0x7f217c1405d0 2022-08-01T19:34:48.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.114+0000 7f2179ffb700 1 -- 172.21.15.17:0/2933897447 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2174007430 con 0x7f217c1405d0 2022-08-01T19:34:48.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.114+0000 7f218184d700 1 -- 172.21.15.17:0/2933897447 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f217c150400 con 0x7f217c1405d0 2022-08-01T19:34:48.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.114+0000 7f218184d700 1 -- 172.21.15.17:0/2933897447 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f217c0a1b70 con 0x7f217c1405d0 2022-08-01T19:34:48.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.114+0000 7f2179ffb700 1 -- 172.21.15.17:0/2933897447 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2174010070 con 0x7f217c1405d0 2022-08-01T19:34:48.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.118+0000 7f2179ffb700 1 --2- 172.21.15.17:0/2933897447 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2160053950 0x7f2160055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:48.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.118+0000 7f2179ffb700 1 -- 172.21.15.17:0/2933897447 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f217406b4b0 con 0x7f217c1405d0 2022-08-01T19:34:48.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.118+0000 7f218084b700 1 --2- 172.21.15.17:0/2933897447 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2160053950 0x7f2160055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:48.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.118+0000 7f218084b700 1 --2- 172.21.15.17:0/2933897447 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2160053950 0x7f2160055e10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f216401a000 tx=0x7f2164014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:48.134 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.122+0000 7f2179ffb700 1 -- 172.21.15.17:0/2933897447 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2174042270 con 0x7f217c1405d0 2022-08-01T19:34:48.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.250+0000 7f218184d700 1 -- 172.21.15.17:0/2933897447 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f217c0af5f0 con 0x7f217c1405d0 2022-08-01T19:34:48.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.250+0000 7f2179ffb700 1 -- 172.21.15.17:0/2933897447 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16271 (secure 0 0 0) 0x7f2174045890 con 0x7f217c1405d0 2022-08-01T19:34:48.256 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:48.257 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":35,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:34:47.691276+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:34:44.961205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2022-08-01T19:34:45.687600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:34:48.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.254+0000 7f218184d700 1 -- 172.21.15.17:0/2933897447 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2160053950 msgr2=0x7f2160055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:48.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.254+0000 7f218184d700 1 --2- 172.21.15.17:0/2933897447 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2160053950 0x7f2160055e10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f216401a000 tx=0x7f2164014000).stop 2022-08-01T19:34:48.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.254+0000 7f218184d700 1 -- 172.21.15.17:0/2933897447 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f217c1405d0 msgr2=0x7f217c1532f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:48.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.254+0000 7f218184d700 1 --2- 172.21.15.17:0/2933897447 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f217c1405d0 0x7f217c1532f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f217c143190 tx=0x7f2174007c30).stop 2022-08-01T19:34:48.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.254+0000 7f218184d700 1 -- 172.21.15.17:0/2933897447 shutdown_connections 2022-08-01T19:34:48.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.254+0000 7f218184d700 1 --2- 172.21.15.17:0/2933897447 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2160053950 0x7f2160055e10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:48.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.254+0000 7f218184d700 1 --2- 172.21.15.17:0/2933897447 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f217c1405d0 0x7f217c1532f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:48.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.254+0000 7f218184d700 1 --2- 172.21.15.17:0/2933897447 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f217c0ad750 0x7f217c152e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:48.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.254+0000 7f218184d700 1 --2- 172.21.15.17:0/2933897447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f217c0614b0 0x7f217c141820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:48.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.254+0000 7f218184d700 1 -- 172.21.15.17:0/2933897447 >> 172.21.15.17:0/2933897447 conn(0x7f217c066f70 msgr2=0x7f217c13fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:48.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.254+0000 7f218184d700 1 -- 172.21.15.17:0/2933897447 shutdown_connections 2022-08-01T19:34:48.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.254+0000 7f218184d700 1 -- 172.21.15.17:0/2933897447 wait complete. 2022-08-01T19:34:48.269 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T19:34:48.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.378+0000 7f1836ceb700 1 Processor -- start 2022-08-01T19:34:48.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.378+0000 7f1836ceb700 1 -- start start 2022-08-01T19:34:48.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.378+0000 7f1836ceb700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f18300b1bc0 0x7f1830066b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:48.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.378+0000 7f1836ceb700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1830140e40 0x7f183013e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:48.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.378+0000 7f1836ceb700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f183005f8e0 con 0x7f1830066780 2022-08-01T19:34:48.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.378+0000 7f1836ceb700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18300b1630 con 0x7f1830140e40 2022-08-01T19:34:48.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.378+0000 7f1836ceb700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f183005fb10 con 0x7f18300b1bc0 2022-08-01T19:34:48.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.378+0000 7f18364ea700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1830140e40 0x7f183013e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:48.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.378+0000 7f1835ce9700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1830066780 0x7f183005ffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58124/0 (socket says 172.21.15.17:58124) 2022-08-01T19:34:48.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.378+0000 7f1835ce9700 1 -- 172.21.15.17:0/1170906738 learned_addr learned my addr 172.21.15.17:0/1170906738 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:48.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.378+0000 7f18354e8700 1 --2- 172.21.15.17:0/1170906738 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f18300b1bc0 0x7f1830066b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:48.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.378+0000 7f18364ea700 1 -- 172.21.15.17:0/1170906738 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f18300b1bc0 msgr2=0x7f1830066b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:48.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.378+0000 7f18364ea700 1 --2- 172.21.15.17:0/1170906738 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f18300b1bc0 0x7f1830066b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:48.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.378+0000 7f1834ce7700 1 -- 172.21.15.17:0/1170906738 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1242310024 0 0) 0x7f183005f8e0 con 0x7f1830066780 2022-08-01T19:34:48.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.378+0000 7f18364ea700 1 -- 172.21.15.17:0/1170906738 >> v1:172.21.15.17:6789/0 conn(0x7f1830066780 legacy=0x7f183005ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:48.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.378+0000 7f18364ea700 1 -- 172.21.15.17:0/1170906738 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18300b18d0 con 0x7f1830140e40 2022-08-01T19:34:48.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.378+0000 7f18364ea700 1 --2- 172.21.15.17:0/1170906738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1830140e40 0x7f183013e2b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f18280032f0 tx=0x7f18280091d0).ready entity=mon.0 client_cookie=dd8ac5860d36af0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:48.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.378+0000 7f1834ce7700 1 -- 172.21.15.17:0/1170906738 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f182800b6c0 con 0x7f1830140e40 2022-08-01T19:34:48.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.378+0000 7f1834ce7700 1 -- 172.21.15.17:0/1170906738 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182800b820 con 0x7f1830140e40 2022-08-01T19:34:48.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.378+0000 7f1834ce7700 1 -- 172.21.15.17:0/1170906738 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f182800b990 con 0x7f1830140e40 2022-08-01T19:34:48.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.382+0000 7f1836ceb700 1 -- 172.21.15.17:0/1170906738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1830140e40 msgr2=0x7f183013e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:48.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.382+0000 7f1836ceb700 1 --2- 172.21.15.17:0/1170906738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1830140e40 0x7f183013e2b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f18280032f0 tx=0x7f18280091d0).stop 2022-08-01T19:34:48.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.382+0000 7f1836ceb700 1 -- 172.21.15.17:0/1170906738 shutdown_connections 2022-08-01T19:34:48.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.382+0000 7f1836ceb700 1 --2- 172.21.15.17:0/1170906738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1830140e40 0x7f183013e2b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:48.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.382+0000 7f1836ceb700 1 --2- 172.21.15.17:0/1170906738 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f18300b1bc0 0x7f1830066b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:48.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.382+0000 7f1836ceb700 1 -- 172.21.15.17:0/1170906738 >> 172.21.15.17:0/1170906738 conn(0x7f1830063870 msgr2=0x7f1830063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:48.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.382+0000 7f1836ceb700 1 -- 172.21.15.17:0/1170906738 shutdown_connections 2022-08-01T19:34:48.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.382+0000 7f1836ceb700 1 -- 172.21.15.17:0/1170906738 wait complete. 2022-08-01T19:34:48.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.382+0000 7f1836ceb700 1 Processor -- start 2022-08-01T19:34:48.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.382+0000 7f1836ceb700 1 -- start start 2022-08-01T19:34:48.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.386+0000 7f1836ceb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1830066780 0x7f18300b2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:48.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.386+0000 7f1836ceb700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f18300b1bc0 0x7f1830145f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:48.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.386+0000 7f1836ceb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1830140e40 0x7f183014ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:48.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.386+0000 7f1836ceb700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183013bd40 con 0x7f1830066780 2022-08-01T19:34:48.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.386+0000 7f1836ceb700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183013c4b0 con 0x7f1830140e40 2022-08-01T19:34:48.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.386+0000 7f1836ceb700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f183013bf70 con 0x7f18300b1bc0 2022-08-01T19:34:48.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.386+0000 7f1835ce9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1830066780 0x7f18300b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:48.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.386+0000 7f1835ce9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1830066780 0x7f18300b2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37704/0 (socket says 172.21.15.17:37704) 2022-08-01T19:34:48.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.386+0000 7f1835ce9700 1 -- 172.21.15.17:0/3158459116 learned_addr learned my addr 172.21.15.17:0/3158459116 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:48.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.386+0000 7f18354e8700 1 --2- 172.21.15.17:0/3158459116 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f18300b1bc0 0x7f1830145f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:48.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.386+0000 7f18364ea700 1 --2- 172.21.15.17:0/3158459116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1830140e40 0x7f183014ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:48.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.386+0000 7f1835ce9700 1 -- 172.21.15.17:0/3158459116 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f18300b1bc0 msgr2=0x7f1830145f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:48.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.386+0000 7f1835ce9700 1 --2- 172.21.15.17:0/3158459116 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f18300b1bc0 0x7f1830145f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:48.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.386+0000 7f1835ce9700 1 -- 172.21.15.17:0/3158459116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1830140e40 msgr2=0x7f183014ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:48.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.386+0000 7f1835ce9700 1 --2- 172.21.15.17:0/3158459116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1830140e40 0x7f183014ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:48.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.386+0000 7f1835ce9700 1 -- 172.21.15.17:0/3158459116 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1828009470 con 0x7f1830066780 2022-08-01T19:34:48.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.386+0000 7f18364ea700 1 --2- 172.21.15.17:0/3158459116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1830140e40 0x7f183014ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:34:48.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.386+0000 7f1835ce9700 1 --2- 172.21.15.17:0/3158459116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1830066780 0x7f18300b2320 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f182c000ed0 tx=0x7f182c007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:48.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.386+0000 7f181effd700 1 -- 172.21.15.17:0/3158459116 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f182c00ce50 con 0x7f1830066780 2022-08-01T19:34:48.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.386+0000 7f181effd700 1 -- 172.21.15.17:0/3158459116 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182c0072e0 con 0x7f1830066780 2022-08-01T19:34:48.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.386+0000 7f1836ceb700 1 -- 172.21.15.17:0/3158459116 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f183013c210 con 0x7f1830066780 2022-08-01T19:34:48.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.386+0000 7f181effd700 1 -- 172.21.15.17:0/3158459116 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f182c005610 con 0x7f1830066780 2022-08-01T19:34:48.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.386+0000 7f1836ceb700 1 -- 172.21.15.17:0/3158459116 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18301463f0 con 0x7f1830066780 2022-08-01T19:34:48.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.386+0000 7f1836ceb700 1 -- 172.21.15.17:0/3158459116 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18300574e0 con 0x7f1830066780 2022-08-01T19:34:48.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.386+0000 7f181effd700 1 -- 172.21.15.17:0/3158459116 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f182c01a060 con 0x7f1830066780 2022-08-01T19:34:48.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.390+0000 7f181effd700 1 --2- 172.21.15.17:0/3158459116 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1810053a70 0x7f1810055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:48.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.390+0000 7f181effd700 1 -- 172.21.15.17:0/3158459116 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f182c06b500 con 0x7f1830066780 2022-08-01T19:34:48.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.390+0000 7f18354e8700 1 --2- 172.21.15.17:0/3158459116 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1810053a70 0x7f1810055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:48.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.390+0000 7f18354e8700 1 --2- 172.21.15.17:0/3158459116 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1810053a70 0x7f1810055f30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f182000b000 tx=0x7f1820006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:48.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.394+0000 7f181effd700 1 -- 172.21.15.17:0/3158459116 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f182c0422c0 con 0x7f1830066780 2022-08-01T19:34:48.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.558+0000 7f1836ceb700 1 -- 172.21.15.17:0/3158459116 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f1830145c60 con 0x7f1830066780 2022-08-01T19:34:48.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.570+0000 7f181effd700 1 -- 172.21.15.17:0/3158459116 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v15) v1 ==== 184+0+0 (secure 0 0 0) 0x7f182c018020 con 0x7f1830066780 2022-08-01T19:34:48.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.570+0000 7f1836ceb700 1 -- 172.21.15.17:0/3158459116 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1810053a70 msgr2=0x7f1810055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:48.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.570+0000 7f1836ceb700 1 --2- 172.21.15.17:0/3158459116 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1810053a70 0x7f1810055f30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f182000b000 tx=0x7f1820006df0).stop 2022-08-01T19:34:48.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.570+0000 7f1836ceb700 1 -- 172.21.15.17:0/3158459116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1830066780 msgr2=0x7f18300b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:48.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.570+0000 7f1836ceb700 1 --2- 172.21.15.17:0/3158459116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1830066780 0x7f18300b2320 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f182c000ed0 tx=0x7f182c007ac0).stop 2022-08-01T19:34:48.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.570+0000 7f1836ceb700 1 -- 172.21.15.17:0/3158459116 shutdown_connections 2022-08-01T19:34:48.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.570+0000 7f1836ceb700 1 --2- 172.21.15.17:0/3158459116 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1810053a70 0x7f1810055f30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:48.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.570+0000 7f1836ceb700 1 --2- 172.21.15.17:0/3158459116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1830140e40 0x7f183014ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:48.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.570+0000 7f1836ceb700 1 --2- 172.21.15.17:0/3158459116 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f18300b1bc0 0x7f1830145f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:48.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.570+0000 7f1836ceb700 1 --2- 172.21.15.17:0/3158459116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1830066780 0x7f18300b2320 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:48.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.570+0000 7f1836ceb700 1 -- 172.21.15.17:0/3158459116 >> 172.21.15.17:0/3158459116 conn(0x7f1830063870 msgr2=0x7f18300b56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:48.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.570+0000 7f1836ceb700 1 -- 172.21.15.17:0/3158459116 shutdown_connections 2022-08-01T19:34:48.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.570+0000 7f1836ceb700 1 -- 172.21.15.17:0/3158459116 wait complete. 2022-08-01T19:34:48.578 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.0 2022-08-01T19:34:48.588 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T19:34:48.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.698+0000 7fdf56841700 1 Processor -- start 2022-08-01T19:34:48.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.698+0000 7fdf56841700 1 -- start start 2022-08-01T19:34:48.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.698+0000 7fdf56841700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdf500b5d80 0x7fdf500b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:48.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.698+0000 7fdf56841700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf50144240 con 0x7fdf500b4550 2022-08-01T19:34:48.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.698+0000 7fdf56841700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf500b6fe0 con 0x7fdf500b6710 2022-08-01T19:34:48.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.698+0000 7fdf56841700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf50143d00 con 0x7fdf500b5d80 2022-08-01T19:34:48.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.698+0000 7fdf54dde700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdf500b5d80 0x7fdf500b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:48.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.698+0000 7fdf4f7fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fdf500b4550 0x7fdf5012bb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58140/0 (socket says 172.21.15.17:58140) 2022-08-01T19:34:48.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.698+0000 7fdf4f7fe700 1 -- 172.21.15.17:0/2661344693 learned_addr learned my addr 172.21.15.17:0/2661344693 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:48.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.698+0000 7fdf54dde700 1 -- 172.21.15.17:0/2661344693 >> v1:172.21.15.77:6789/0 conn(0x7fdf500b6710 legacy=0x7fdf50061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:48.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.698+0000 7fdf54dde700 1 -- 172.21.15.17:0/2661344693 >> v1:172.21.15.17:6789/0 conn(0x7fdf500b4550 legacy=0x7fdf5012bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:48.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.698+0000 7fdf54dde700 1 -- 172.21.15.17:0/2661344693 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf50143fa0 con 0x7fdf500b5d80 2022-08-01T19:34:48.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.698+0000 7fdf54dde700 1 --2- 172.21.15.17:0/2661344693 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf500b5d80 0x7fdf500b6af0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7fdf480032f0 tx=0x7fdf480091d0).ready entity=mon.0 client_cookie=858aac309034ec0b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:48.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.698+0000 7fdf4effd700 1 -- 172.21.15.17:0/2661344693 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf48009ea0 con 0x7fdf500b5d80 2022-08-01T19:34:48.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.698+0000 7fdf4effd700 1 -- 172.21.15.17:0/2661344693 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4800b8a0 con 0x7fdf500b5d80 2022-08-01T19:34:48.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.698+0000 7fdf4effd700 1 -- 172.21.15.17:0/2661344693 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf4800ba10 con 0x7fdf500b5d80 2022-08-01T19:34:48.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.702+0000 7fdf56841700 1 -- 172.21.15.17:0/2661344693 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf500b5d80 msgr2=0x7fdf500b6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:48.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.702+0000 7fdf56841700 1 --2- 172.21.15.17:0/2661344693 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf500b5d80 0x7fdf500b6af0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7fdf480032f0 tx=0x7fdf480091d0).stop 2022-08-01T19:34:48.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.702+0000 7fdf56841700 1 -- 172.21.15.17:0/2661344693 shutdown_connections 2022-08-01T19:34:48.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.702+0000 7fdf56841700 1 --2- 172.21.15.17:0/2661344693 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf500b5d80 0x7fdf500b6af0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:48.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.702+0000 7fdf56841700 1 -- 172.21.15.17:0/2661344693 >> 172.21.15.17:0/2661344693 conn(0x7fdf5005c620 msgr2=0x7fdf5005ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:48.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.702+0000 7fdf56841700 1 -- 172.21.15.17:0/2661344693 shutdown_connections 2022-08-01T19:34:48.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.702+0000 7fdf56841700 1 -- 172.21.15.17:0/2661344693 wait complete. 2022-08-01T19:34:48.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.702+0000 7fdf56841700 1 Processor -- start 2022-08-01T19:34:48.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.702+0000 7fdf56841700 1 -- start start 2022-08-01T19:34:48.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.702+0000 7fdf56841700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf500b4550 0x7fdf5012aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:48.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.702+0000 7fdf56841700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf500b5d80 0x7fdf500b3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:48.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.702+0000 7fdf56841700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdf500b6710 0x7fdf500b3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:48.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.702+0000 7fdf56841700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf50141330 con 0x7fdf500b5d80 2022-08-01T19:34:48.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.702+0000 7fdf56841700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf501415d0 con 0x7fdf500b4550 2022-08-01T19:34:48.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.702+0000 7fdf56841700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf50141870 con 0x7fdf500b6710 2022-08-01T19:34:48.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.702+0000 7fdf4f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf500b5d80 0x7fdf500b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:48.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.702+0000 7fdf4f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf500b5d80 0x7fdf500b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37720/0 (socket says 172.21.15.17:37720) 2022-08-01T19:34:48.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.702+0000 7fdf4f7fe700 1 -- 172.21.15.17:0/1632256505 learned_addr learned my addr 172.21.15.17:0/1632256505 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:48.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.702+0000 7fdf54dde700 1 --2- 172.21.15.17:0/1632256505 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdf500b6710 0x7fdf500b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:48.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.702+0000 7fdf4ffff700 1 --2- 172.21.15.17:0/1632256505 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf500b4550 0x7fdf5012aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:48.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.706+0000 7fdf4f7fe700 1 -- 172.21.15.17:0/1632256505 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdf500b6710 msgr2=0x7fdf500b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:48.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.706+0000 7fdf4f7fe700 1 --2- 172.21.15.17:0/1632256505 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdf500b6710 0x7fdf500b3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:48.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.706+0000 7fdf4f7fe700 1 -- 172.21.15.17:0/1632256505 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf500b4550 msgr2=0x7fdf5012aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:48.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.706+0000 7fdf4f7fe700 1 --2- 172.21.15.17:0/1632256505 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf500b4550 0x7fdf5012aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:48.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.706+0000 7fdf4f7fe700 1 -- 172.21.15.17:0/1632256505 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf48009470 con 0x7fdf500b5d80 2022-08-01T19:34:48.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.706+0000 7fdf4f7fe700 1 --2- 172.21.15.17:0/1632256505 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf500b5d80 0x7fdf500b3190 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7fdf500b1e10 tx=0x7fdf44007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:48.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.706+0000 7fdf4d7fa700 1 -- 172.21.15.17:0/1632256505 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf440070d0 con 0x7fdf500b5d80 2022-08-01T19:34:48.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.706+0000 7fdf4d7fa700 1 -- 172.21.15.17:0/1632256505 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf44007230 con 0x7fdf500b5d80 2022-08-01T19:34:48.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.706+0000 7fdf56841700 1 -- 172.21.15.17:0/1632256505 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf50141b10 con 0x7fdf500b5d80 2022-08-01T19:34:48.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.706+0000 7fdf4d7fa700 1 -- 172.21.15.17:0/1632256505 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf44005610 con 0x7fdf500b5d80 2022-08-01T19:34:48.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.706+0000 7fdf56841700 1 -- 172.21.15.17:0/1632256505 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf50142df0 con 0x7fdf500b5d80 2022-08-01T19:34:48.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.706+0000 7fdf56841700 1 -- 172.21.15.17:0/1632256505 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf50061780 con 0x7fdf500b5d80 2022-08-01T19:34:48.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.706+0000 7fdf4d7fa700 1 -- 172.21.15.17:0/1632256505 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdf44010070 con 0x7fdf500b5d80 2022-08-01T19:34:48.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.710+0000 7fdf4d7fa700 1 --2- 172.21.15.17:0/1632256505 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdf30053b00 0x7fdf30055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:48.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.710+0000 7fdf4d7fa700 1 -- 172.21.15.17:0/1632256505 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fdf4406b500 con 0x7fdf500b5d80 2022-08-01T19:34:48.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.710+0000 7fdf4ffff700 1 --2- 172.21.15.17:0/1632256505 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdf30053b00 0x7fdf30055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:48.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.710+0000 7fdf4ffff700 1 --2- 172.21.15.17:0/1632256505 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdf30053b00 0x7fdf30055fc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7fdf4000d000 tx=0x7fdf40007b70).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:48.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.714+0000 7fdf4d7fa700 1 -- 172.21.15.17:0/1632256505 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdf440422c0 con 0x7fdf500b5d80 2022-08-01T19:34:48.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.882+0000 7fdf56841700 1 -- 172.21.15.17:0/1632256505 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fdf500574e0 con 0x7fdf500b5d80 2022-08-01T19:34:48.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.890+0000 7fdf4d7fa700 1 -- 172.21.15.17:0/1632256505 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v16) v1 ==== 184+0+0 (secure 0 0 0) 0x7fdf440458e0 con 0x7fdf500b5d80 2022-08-01T19:34:48.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.890+0000 7fdf56841700 1 -- 172.21.15.17:0/1632256505 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdf30053b00 msgr2=0x7fdf30055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:48.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.890+0000 7fdf56841700 1 --2- 172.21.15.17:0/1632256505 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdf30053b00 0x7fdf30055fc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7fdf4000d000 tx=0x7fdf40007b70).stop 2022-08-01T19:34:48.899 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.890+0000 7fdf56841700 1 -- 172.21.15.17:0/1632256505 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf500b5d80 msgr2=0x7fdf500b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:48.899 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.890+0000 7fdf56841700 1 --2- 172.21.15.17:0/1632256505 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf500b5d80 0x7fdf500b3190 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7fdf500b1e10 tx=0x7fdf44007c10).stop 2022-08-01T19:34:48.899 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.890+0000 7fdf56841700 1 -- 172.21.15.17:0/1632256505 shutdown_connections 2022-08-01T19:34:48.899 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.890+0000 7fdf56841700 1 --2- 172.21.15.17:0/1632256505 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdf30053b00 0x7fdf30055fc0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:48.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.890+0000 7fdf56841700 1 --2- 172.21.15.17:0/1632256505 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdf500b6710 0x7fdf500b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:48.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.890+0000 7fdf56841700 1 --2- 172.21.15.17:0/1632256505 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf500b5d80 0x7fdf500b3190 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:48.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.890+0000 7fdf56841700 1 --2- 172.21.15.17:0/1632256505 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf500b4550 0x7fdf5012aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:48.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.890+0000 7fdf56841700 1 -- 172.21.15.17:0/1632256505 >> 172.21.15.17:0/1632256505 conn(0x7fdf5005c620 msgr2=0x7fdf50143340 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:48.901 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.894+0000 7fdf56841700 1 -- 172.21.15.17:0/1632256505 shutdown_connections 2022-08-01T19:34:48.901 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:48.894+0000 7fdf56841700 1 -- 172.21.15.17:0/1632256505 wait complete. 2022-08-01T19:34:48.902 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.1 2022-08-01T19:34:48.910 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:34:49.013 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.010+0000 7f06d359e700 1 Processor -- start 2022-08-01T19:34:49.014 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.010+0000 7f06d359e700 1 -- start start 2022-08-01T19:34:49.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.010+0000 7f06d359e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f06d40b01a0 0x7f06d40a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:49.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.010+0000 7f06d359e700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06d40a7250 con 0x7f06d40afd60 2022-08-01T19:34:49.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.010+0000 7f06d359e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06d40a7480 con 0x7f06d40b2710 2022-08-01T19:34:49.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.010+0000 7f06d359e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d40a7720 con 0x7f06d40b01a0 2022-08-01T19:34:49.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.010+0000 7f06d2d9d700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f06d40afd60 0x7f06d4151d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58156/0 (socket says 172.21.15.17:58156) 2022-08-01T19:34:49.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.010+0000 7f06d2d9d700 1 -- 172.21.15.17:0/756511860 learned_addr learned my addr 172.21.15.17:0/756511860 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:49.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.010+0000 7f06d1d9b700 1 --2- 172.21.15.17:0/756511860 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f06d40b01a0 0x7f06d40a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:49.018 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.010+0000 7f06d159a700 1 -- 172.21.15.17:0/756511860 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3856532084 0 0) 0x7f06d40a7250 con 0x7f06d40afd60 2022-08-01T19:34:49.018 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.010+0000 7f06d159a700 1 -- 172.21.15.17:0/756511860 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06b8003260 con 0x7f06d40afd60 2022-08-01T19:34:49.018 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.010+0000 7f06d159a700 1 -- 172.21.15.17:0/756511860 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4020385987 0 0) 0x7f06d40a7480 con 0x7f06d40b2710 2022-08-01T19:34:49.018 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.010+0000 7f06d159a700 1 -- 172.21.15.17:0/756511860 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06d40a7250 con 0x7f06d40b2710 2022-08-01T19:34:49.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.010+0000 7f06d1d9b700 1 -- 172.21.15.17:0/756511860 >> v1:172.21.15.77:6789/0 conn(0x7f06d40b2710 legacy=0x7f06d414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:49.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.010+0000 7f06d1d9b700 1 -- 172.21.15.17:0/756511860 >> v1:172.21.15.17:6789/0 conn(0x7f06d40afd60 legacy=0x7f06d4151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:49.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.010+0000 7f06d1d9b700 1 -- 172.21.15.17:0/756511860 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d40a79c0 con 0x7f06d40b01a0 2022-08-01T19:34:49.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.010+0000 7f06d1d9b700 1 --2- 172.21.15.17:0/756511860 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f06d40b01a0 0x7f06d40a8540 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f06c4002f70 tx=0x7f06c4006df0).ready entity=mon.2 client_cookie=366710b2f75727d8 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:49.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.010+0000 7f06d159a700 1 -- 172.21.15.17:0/756511860 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06c400d5c0 con 0x7f06d40b01a0 2022-08-01T19:34:49.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.010+0000 7f06d159a700 1 -- 172.21.15.17:0/756511860 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c4000e90 con 0x7f06d40b01a0 2022-08-01T19:34:49.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.010+0000 7f06d159a700 1 -- 172.21.15.17:0/756511860 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06c400d910 con 0x7f06d40b01a0 2022-08-01T19:34:49.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.010+0000 7f06d359e700 1 -- 172.21.15.17:0/756511860 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f06d40b01a0 msgr2=0x7f06d40a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:49.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.010+0000 7f06d359e700 1 --2- 172.21.15.17:0/756511860 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f06d40b01a0 0x7f06d40a8540 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f06c4002f70 tx=0x7f06c4006df0).stop 2022-08-01T19:34:49.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.010+0000 7f06d359e700 1 -- 172.21.15.17:0/756511860 shutdown_connections 2022-08-01T19:34:49.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.010+0000 7f06d359e700 1 --2- 172.21.15.17:0/756511860 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f06d40b01a0 0x7f06d40a8540 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:49.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.010+0000 7f06d359e700 1 -- 172.21.15.17:0/756511860 >> 172.21.15.17:0/756511860 conn(0x7f06d4060310 msgr2=0x7f06d40b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:49.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.010+0000 7f06d359e700 1 -- 172.21.15.17:0/756511860 shutdown_connections 2022-08-01T19:34:49.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.010+0000 7f06d359e700 1 -- 172.21.15.17:0/756511860 wait complete. 2022-08-01T19:34:49.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.014+0000 7f06d359e700 1 Processor -- start 2022-08-01T19:34:49.026 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.014+0000 7f06d359e700 1 -- start start 2022-08-01T19:34:49.026 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.014+0000 7f06d359e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06d40afd60 0x7f06d40b5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:49.027 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.014+0000 7f06d359e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06d40b01a0 0x7f06d41423a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:49.027 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.014+0000 7f06d359e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f06d40b2710 0x7f06d41497e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:49.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.014+0000 7f06d359e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d40b5e00 con 0x7f06d40b01a0 2022-08-01T19:34:49.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.014+0000 7f06d359e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d4142890 con 0x7f06d40afd60 2022-08-01T19:34:49.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.014+0000 7f06d359e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d40b2f60 con 0x7f06d40b2710 2022-08-01T19:34:49.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.014+0000 7f06d1d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06d40b01a0 0x7f06d41423a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:49.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.014+0000 7f06d1d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06d40b01a0 0x7f06d41423a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37734/0 (socket says 172.21.15.17:37734) 2022-08-01T19:34:49.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.014+0000 7f06d1d9b700 1 -- 172.21.15.17:0/1974001277 learned_addr learned my addr 172.21.15.17:0/1974001277 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:49.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.014+0000 7f06d2d9d700 1 --2- 172.21.15.17:0/1974001277 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f06d40b2710 0x7f06d41497e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:49.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.014+0000 7f06d259c700 1 --2- 172.21.15.17:0/1974001277 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06d40afd60 0x7f06d40b5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:49.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.014+0000 7f06d1d9b700 1 -- 172.21.15.17:0/1974001277 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f06d40b2710 msgr2=0x7f06d41497e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:49.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.014+0000 7f06d1d9b700 1 --2- 172.21.15.17:0/1974001277 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f06d40b2710 0x7f06d41497e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:49.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.014+0000 7f06d1d9b700 1 -- 172.21.15.17:0/1974001277 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06d40afd60 msgr2=0x7f06d40b5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:49.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.014+0000 7f06d1d9b700 1 --2- 172.21.15.17:0/1974001277 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06d40afd60 0x7f06d40b5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:49.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.014+0000 7f06d1d9b700 1 -- 172.21.15.17:0/1974001277 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c400b180 con 0x7f06d40b01a0 2022-08-01T19:34:49.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.014+0000 7f06d1d9b700 1 --2- 172.21.15.17:0/1974001277 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06d40b01a0 0x7f06d41423a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f06d4156610 tx=0x7f06c4014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:49.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.014+0000 7f06c37fe700 1 -- 172.21.15.17:0/1974001277 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06c4016070 con 0x7f06d40b01a0 2022-08-01T19:34:49.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.014+0000 7f06c37fe700 1 -- 172.21.15.17:0/1974001277 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c40137d0 con 0x7f06d40b01a0 2022-08-01T19:34:49.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.014+0000 7f06d359e700 1 -- 172.21.15.17:0/1974001277 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06d40b3720 con 0x7f06d40b01a0 2022-08-01T19:34:49.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.014+0000 7f06c37fe700 1 -- 172.21.15.17:0/1974001277 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06c4013b30 con 0x7f06d40b01a0 2022-08-01T19:34:49.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.014+0000 7f06d359e700 1 -- 172.21.15.17:0/1974001277 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06d40b3980 con 0x7f06d40b01a0 2022-08-01T19:34:49.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.018+0000 7f06d359e700 1 -- 172.21.15.17:0/1974001277 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06d40b3cb0 con 0x7f06d40b01a0 2022-08-01T19:34:49.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.018+0000 7f06c37fe700 1 -- 172.21.15.17:0/1974001277 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f06c4025030 con 0x7f06d40b01a0 2022-08-01T19:34:49.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.018+0000 7f06c37fe700 1 --2- 172.21.15.17:0/1974001277 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f06b0053950 0x7f06b0055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:49.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.018+0000 7f06c37fe700 1 -- 172.21.15.17:0/1974001277 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f06c406cf00 con 0x7f06d40b01a0 2022-08-01T19:34:49.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.018+0000 7f06d259c700 1 --2- 172.21.15.17:0/1974001277 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f06b0053950 0x7f06b0055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:49.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.022+0000 7f06d259c700 1 --2- 172.21.15.17:0/1974001277 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f06b0053950 0x7f06b0055e10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f06c800d000 tx=0x7f06c8007a00).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:49.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.026+0000 7f06c37fe700 1 -- 172.21.15.17:0/1974001277 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f06c4043cc0 con 0x7f06d40b01a0 2022-08-01T19:34:49.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.190+0000 7f06d359e700 1 -- 172.21.15.17:0/1974001277 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06d40574e0 con 0x7f06d40b01a0 2022-08-01T19:34:49.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.190+0000 7f06c37fe700 1 -- 172.21.15.17:0/1974001277 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+2491 (secure 0 0 0) 0x7f06c4023090 con 0x7f06d40b01a0 2022-08-01T19:34:49.197 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:49.197 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2487504686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2487504686},{"type":"v1","addr":"172.21.15.17:6833","nonce":2487504686}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10},{"gid":4885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/4255478434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4255478434},{"type":"v1","addr":"172.21.15.77:6835","nonce":4255478434}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":11}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:34:46.688039+0000","modified":"2022-08-01T19:34:47.690492+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4890},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4890":{"gid":4890,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/3876538769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":3876538769},{"type":"v1","addr":"172.21.15.17:6835","nonce":3876538769}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-08-01T19:34:49.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.194+0000 7f06d359e700 1 -- 172.21.15.17:0/1974001277 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f06b0053950 msgr2=0x7f06b0055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:49.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.194+0000 7f06d359e700 1 --2- 172.21.15.17:0/1974001277 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f06b0053950 0x7f06b0055e10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f06c800d000 tx=0x7f06c8007a00).stop 2022-08-01T19:34:49.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.194+0000 7f06d359e700 1 -- 172.21.15.17:0/1974001277 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06d40b01a0 msgr2=0x7f06d41423a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:49.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.194+0000 7f06d359e700 1 --2- 172.21.15.17:0/1974001277 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06d40b01a0 0x7f06d41423a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f06d4156610 tx=0x7f06c4014000).stop 2022-08-01T19:34:49.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.194+0000 7f06d359e700 1 -- 172.21.15.17:0/1974001277 shutdown_connections 2022-08-01T19:34:49.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.194+0000 7f06d359e700 1 --2- 172.21.15.17:0/1974001277 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f06b0053950 0x7f06b0055e10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:49.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.194+0000 7f06d359e700 1 --2- 172.21.15.17:0/1974001277 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f06d40b2710 0x7f06d41497e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:49.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.194+0000 7f06d359e700 1 --2- 172.21.15.17:0/1974001277 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06d40b01a0 0x7f06d41423a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:49.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.194+0000 7f06d359e700 1 --2- 172.21.15.17:0/1974001277 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06d40afd60 0x7f06d40b5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:49.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.194+0000 7f06d359e700 1 -- 172.21.15.17:0/1974001277 >> 172.21.15.17:0/1974001277 conn(0x7f06d4060310 msgr2=0x7f06d4145310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:49.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.194+0000 7f06d359e700 1 -- 172.21.15.17:0/1974001277 shutdown_connections 2022-08-01T19:34:49.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.194+0000 7f06d359e700 1 -- 172.21.15.17:0/1974001277 wait complete. 2022-08-01T19:34:49.204 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 13 2022-08-01T19:34:49.512 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 13, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:34:46.688039+0000', 'modified': '2022-08-01T19:34:47.690492+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4890}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4890': {'gid': 4890, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/3876538769', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 3876538769}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 3876538769}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:34:49.512 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:34:49.513 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-08-01T19:34:49.633 INFO:teuthology.orchestra.run.smithi017.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":4890,"want_state":"up:active","state":"up:active","rank_uptime":2.9336794419999999,"mdsmap_epoch":13,"osdmap_epoch":35,"osdmap_epoch_barrier":34,"uptime":9.2999748259999997} 2022-08-01T19:34:49.634 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 4890, "mdsmap_epoch": 13, "osdmap_epoch": 35, "osdmap_epoch_barrier": 34, "rank_uptime": 2.933679442, "state": "up:active", "uptime": 9.299974826, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:34:49.635 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:34:49.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.730+0000 7f9f21ffa700 1 Processor -- start 2022-08-01T19:34:49.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.730+0000 7f9f21ffa700 1 -- start start 2022-08-01T19:34:49.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.730+0000 7f9f21ffa700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9f1c0b2710 0x7f9f1c14afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:49.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.730+0000 7f9f21ffa700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f9f1c0b01a0 0x7f9f1c14b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:49.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.730+0000 7f9f21ffa700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9f1c0afd60 0x7f9f1c0a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:49.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.730+0000 7f9f21ffa700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f1c0a7250 con 0x7f9f1c0b2710 2022-08-01T19:34:49.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.730+0000 7f9f21ffa700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f1c0a7480 con 0x7f9f1c0b01a0 2022-08-01T19:34:49.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.730+0000 7f9f21ffa700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c0a7720 con 0x7f9f1c0afd60 2022-08-01T19:34:49.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.730+0000 7f9f20ff8700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9f1c0b2710 0x7f9f1c14afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:49.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.730+0000 7f9f20ff8700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9f1c0b2710 0x7f9f1c14afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37740/0 (socket says 172.21.15.17:37740) 2022-08-01T19:34:49.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.730+0000 7f9f20ff8700 1 -- 172.21.15.17:0/1458221572 learned_addr learned my addr 172.21.15.17:0/1458221572 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:49.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.730+0000 7f9f1bfff700 1 --2- 172.21.15.17:0/1458221572 >> v2:172.21.15.77:3300/0 conn(0x7f9f1c0b01a0 0x7f9f1c14b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:49.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.730+0000 7f9f217f9700 1 --2- 172.21.15.17:0/1458221572 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9f1c0afd60 0x7f9f1c0a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:49.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.730+0000 7f9f20ff8700 1 -- 172.21.15.17:0/1458221572 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9f1c0afd60 msgr2=0x7f9f1c0a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:49.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.730+0000 7f9f20ff8700 1 --2- 172.21.15.17:0/1458221572 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9f1c0afd60 0x7f9f1c0a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:49.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.730+0000 7f9f20ff8700 1 -- 172.21.15.17:0/1458221572 >> v2:172.21.15.77:3300/0 conn(0x7f9f1c0b01a0 msgr2=0x7f9f1c14b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:49.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.730+0000 7f9f20ff8700 1 --2- 172.21.15.17:0/1458221572 >> v2:172.21.15.77:3300/0 conn(0x7f9f1c0b01a0 0x7f9f1c14b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:49.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.730+0000 7f9f20ff8700 1 -- 172.21.15.17:0/1458221572 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f1c0a79c0 con 0x7f9f1c0b2710 2022-08-01T19:34:49.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.730+0000 7f9f217f9700 1 --2- 172.21.15.17:0/1458221572 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9f1c0afd60 0x7f9f1c0a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:34:49.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.734+0000 7f9f20ff8700 1 --2- 172.21.15.17:0/1458221572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f1c0b2710 0x7f9f1c14afd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f9f10002f70 tx=0x7f9f10006eb0).ready entity=mon.0 client_cookie=caab3bfb353427ee server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:49.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.734+0000 7f9f1b7fe700 1 -- 172.21.15.17:0/1458221572 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f1000d570 con 0x7f9f1c0b2710 2022-08-01T19:34:49.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.734+0000 7f9f1b7fe700 1 -- 172.21.15.17:0/1458221572 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f10000e90 con 0x7f9f1c0b2710 2022-08-01T19:34:49.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.734+0000 7f9f1b7fe700 1 -- 172.21.15.17:0/1458221572 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f1000d900 con 0x7f9f1c0b2710 2022-08-01T19:34:49.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.734+0000 7f9f21ffa700 1 -- 172.21.15.17:0/1458221572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f1c0b2710 msgr2=0x7f9f1c14afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:49.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.734+0000 7f9f21ffa700 1 --2- 172.21.15.17:0/1458221572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f1c0b2710 0x7f9f1c14afd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f9f10002f70 tx=0x7f9f10006eb0).stop 2022-08-01T19:34:49.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.734+0000 7f9f21ffa700 1 -- 172.21.15.17:0/1458221572 shutdown_connections 2022-08-01T19:34:49.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.734+0000 7f9f21ffa700 1 --2- 172.21.15.17:0/1458221572 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9f1c0afd60 0x7f9f1c0a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:49.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.734+0000 7f9f21ffa700 1 --2- 172.21.15.17:0/1458221572 >> v2:172.21.15.77:3300/0 conn(0x7f9f1c0b01a0 0x7f9f1c14b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:49.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.734+0000 7f9f21ffa700 1 --2- 172.21.15.17:0/1458221572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f1c0b2710 0x7f9f1c14afd0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:49.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.734+0000 7f9f21ffa700 1 -- 172.21.15.17:0/1458221572 >> 172.21.15.17:0/1458221572 conn(0x7f9f1c060310 msgr2=0x7f9f1c0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:49.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.734+0000 7f9f21ffa700 1 -- 172.21.15.17:0/1458221572 shutdown_connections 2022-08-01T19:34:49.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.734+0000 7f9f21ffa700 1 -- 172.21.15.17:0/1458221572 wait complete. 2022-08-01T19:34:49.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.734+0000 7f9f21ffa700 1 Processor -- start 2022-08-01T19:34:49.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.734+0000 7f9f21ffa700 1 -- start start 2022-08-01T19:34:49.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.734+0000 7f9f21ffa700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f1c0afd60 0x7f9f1c1455f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:49.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.734+0000 7f9f21ffa700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9f1c0b01a0 0x7f9f1c0a9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:49.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.734+0000 7f9f21ffa700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f1c0b2710 0x7f9f1c0aa080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:49.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.734+0000 7f9f21ffa700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c0b2f60 con 0x7f9f1c0afd60 2022-08-01T19:34:49.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.734+0000 7f9f21ffa700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c0b5c10 con 0x7f9f1c0b2710 2022-08-01T19:34:49.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.734+0000 7f9f21ffa700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c0b3a10 con 0x7f9f1c0b01a0 2022-08-01T19:34:49.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.734+0000 7f9f20ff8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f1c0afd60 0x7f9f1c1455f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:49.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.738+0000 7f9f20ff8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f1c0afd60 0x7f9f1c1455f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37746/0 (socket says 172.21.15.17:37746) 2022-08-01T19:34:49.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.738+0000 7f9f20ff8700 1 -- 172.21.15.17:0/401052500 learned_addr learned my addr 172.21.15.17:0/401052500 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:49.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.738+0000 7f9f217f9700 1 --2- 172.21.15.17:0/401052500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f1c0b2710 0x7f9f1c0aa080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:49.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.738+0000 7f9f1bfff700 1 --2- 172.21.15.17:0/401052500 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9f1c0b01a0 0x7f9f1c0a9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:49.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.738+0000 7f9f20ff8700 1 -- 172.21.15.17:0/401052500 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9f1c0b01a0 msgr2=0x7f9f1c0a9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:49.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.738+0000 7f9f20ff8700 1 --2- 172.21.15.17:0/401052500 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9f1c0b01a0 0x7f9f1c0a9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:49.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.738+0000 7f9f20ff8700 1 -- 172.21.15.17:0/401052500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f1c0b2710 msgr2=0x7f9f1c0aa080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:49.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.738+0000 7f9f20ff8700 1 --2- 172.21.15.17:0/401052500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f1c0b2710 0x7f9f1c0aa080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:49.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.738+0000 7f9f20ff8700 1 -- 172.21.15.17:0/401052500 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f1000b1e0 con 0x7f9f1c0afd60 2022-08-01T19:34:49.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.738+0000 7f9f20ff8700 1 --2- 172.21.15.17:0/401052500 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f1c0afd60 0x7f9f1c1455f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f9f1c0af320 tx=0x7f9f10015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:49.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.738+0000 7f9f19ffb700 1 -- 172.21.15.17:0/401052500 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f10006be0 con 0x7f9f1c0afd60 2022-08-01T19:34:49.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.738+0000 7f9f19ffb700 1 -- 172.21.15.17:0/401052500 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f100068b0 con 0x7f9f1c0afd60 2022-08-01T19:34:49.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.738+0000 7f9f21ffa700 1 -- 172.21.15.17:0/401052500 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f1c0af480 con 0x7f9f1c0afd60 2022-08-01T19:34:49.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.738+0000 7f9f19ffb700 1 -- 172.21.15.17:0/401052500 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f10011e90 con 0x7f9f1c0afd60 2022-08-01T19:34:49.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.738+0000 7f9f21ffa700 1 -- 172.21.15.17:0/401052500 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f1c0b3ee0 con 0x7f9f1c0afd60 2022-08-01T19:34:49.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.738+0000 7f9f19ffb700 1 -- 172.21.15.17:0/401052500 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9f10025030 con 0x7f9f1c0afd60 2022-08-01T19:34:49.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.738+0000 7f9f21ffa700 1 -- 172.21.15.17:0/401052500 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f1c0574e0 con 0x7f9f1c0afd60 2022-08-01T19:34:49.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.738+0000 7f9f19ffb700 1 --2- 172.21.15.17:0/401052500 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9efc0539e0 0x7f9efc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:49.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.738+0000 7f9f19ffb700 1 -- 172.21.15.17:0/401052500 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9f1006d2f0 con 0x7f9f1c0afd60 2022-08-01T19:34:49.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.738+0000 7f9f1bfff700 1 --2- 172.21.15.17:0/401052500 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9efc0539e0 0x7f9efc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:49.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.742+0000 7f9f1bfff700 1 --2- 172.21.15.17:0/401052500 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9efc0539e0 0x7f9efc055ea0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f9f0c00b000 tx=0x7f9f0c008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:49.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.742+0000 7f9f19ffb700 1 -- 172.21.15.17:0/401052500 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9f10044010 con 0x7f9f1c0afd60 2022-08-01T19:34:49.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.870+0000 7f9f21ffa700 1 -- 172.21.15.17:0/401052500 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f1c06a5e0 con 0x7f9f1c0afd60 2022-08-01T19:34:49.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.870+0000 7f9f19ffb700 1 -- 172.21.15.17:0/401052500 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16271 (secure 0 0 0) 0x7f9f10047630 con 0x7f9f1c0afd60 2022-08-01T19:34:49.878 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:49.878 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":35,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:34:47.691276+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:34:44.961205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2022-08-01T19:34:45.687600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:34:49.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.874+0000 7f9f21ffa700 1 -- 172.21.15.17:0/401052500 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9efc0539e0 msgr2=0x7f9efc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:49.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.874+0000 7f9f21ffa700 1 --2- 172.21.15.17:0/401052500 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9efc0539e0 0x7f9efc055ea0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f9f0c00b000 tx=0x7f9f0c008040).stop 2022-08-01T19:34:49.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.874+0000 7f9f21ffa700 1 -- 172.21.15.17:0/401052500 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f1c0afd60 msgr2=0x7f9f1c1455f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:49.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.874+0000 7f9f21ffa700 1 --2- 172.21.15.17:0/401052500 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f1c0afd60 0x7f9f1c1455f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f9f1c0af320 tx=0x7f9f10015000).stop 2022-08-01T19:34:49.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.874+0000 7f9f21ffa700 1 -- 172.21.15.17:0/401052500 shutdown_connections 2022-08-01T19:34:49.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.874+0000 7f9f21ffa700 1 --2- 172.21.15.17:0/401052500 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9efc0539e0 0x7f9efc055ea0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:49.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.874+0000 7f9f21ffa700 1 --2- 172.21.15.17:0/401052500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f1c0b2710 0x7f9f1c0aa080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:49.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.874+0000 7f9f21ffa700 1 --2- 172.21.15.17:0/401052500 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9f1c0b01a0 0x7f9f1c0a9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:49.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.874+0000 7f9f21ffa700 1 --2- 172.21.15.17:0/401052500 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f1c0afd60 0x7f9f1c1455f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:49.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.874+0000 7f9f21ffa700 1 -- 172.21.15.17:0/401052500 >> 172.21.15.17:0/401052500 conn(0x7f9f1c060310 msgr2=0x7f9f1c146200 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:49.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.878+0000 7f9f21ffa700 1 -- 172.21.15.17:0/401052500 shutdown_connections 2022-08-01T19:34:49.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:49.878+0000 7f9f21ffa700 1 -- 172.21.15.17:0/401052500 wait complete. 2022-08-01T19:34:49.896 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T19:34:44.961205+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T19:34:45.687600+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T19:34:49.897 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:34:50.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.002+0000 7f434ebfd700 1 Processor -- start 2022-08-01T19:34:50.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.002+0000 7f434ebfd700 1 -- start start 2022-08-01T19:34:50.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.002+0000 7f434ebfd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f43480a8c50 0x7f4348152d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:50.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.002+0000 7f434ebfd700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43480a7250 con 0x7f43480b01a0 2022-08-01T19:34:50.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.002+0000 7f434ebfd700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43480a7480 con 0x7f43480b2710 2022-08-01T19:34:50.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.002+0000 7f434ebfd700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43480a7720 con 0x7f43480a8c50 2022-08-01T19:34:50.010 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.002+0000 7f434e3fc700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f43480a8c50 0x7f4348152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:50.010 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.002+0000 7f434d3fa700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f43480b01a0 0x7f43480a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58182/0 (socket says 172.21.15.17:58182) 2022-08-01T19:34:50.010 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.002+0000 7f434d3fa700 1 -- 172.21.15.17:0/665644956 learned_addr learned my addr 172.21.15.17:0/665644956 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:34:50.010 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.002+0000 7f434e3fc700 1 -- 172.21.15.17:0/665644956 >> v1:172.21.15.77:6789/0 conn(0x7f43480b2710 legacy=0x7f434814afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:50.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.002+0000 7f434e3fc700 1 -- 172.21.15.17:0/665644956 >> v1:172.21.15.17:6789/0 conn(0x7f43480b01a0 legacy=0x7f43480a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:50.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.002+0000 7f434e3fc700 1 -- 172.21.15.17:0/665644956 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43480a79c0 con 0x7f43480a8c50 2022-08-01T19:34:50.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.002+0000 7f434e3fc700 1 --2- 172.21.15.17:0/665644956 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43480a8c50 0x7f4348152d70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f4340002f70 tx=0x7f4340006df0).ready entity=mon.0 client_cookie=865ad37f5080474 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:50.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.002+0000 7f434cbf9700 1 -- 172.21.15.17:0/665644956 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f434000d610 con 0x7f43480a8c50 2022-08-01T19:34:50.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.002+0000 7f434cbf9700 1 -- 172.21.15.17:0/665644956 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4340000e90 con 0x7f43480a8c50 2022-08-01T19:34:50.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.002+0000 7f434cbf9700 1 -- 172.21.15.17:0/665644956 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f434000d960 con 0x7f43480a8c50 2022-08-01T19:34:50.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.002+0000 7f434ebfd700 1 -- 172.21.15.17:0/665644956 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43480a8c50 msgr2=0x7f4348152d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:50.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.002+0000 7f434ebfd700 1 --2- 172.21.15.17:0/665644956 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43480a8c50 0x7f4348152d70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f4340002f70 tx=0x7f4340006df0).stop 2022-08-01T19:34:50.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.002+0000 7f434ebfd700 1 -- 172.21.15.17:0/665644956 shutdown_connections 2022-08-01T19:34:50.013 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.002+0000 7f434ebfd700 1 --2- 172.21.15.17:0/665644956 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43480a8c50 0x7f4348152d70 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:50.013 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.002+0000 7f434ebfd700 1 -- 172.21.15.17:0/665644956 >> 172.21.15.17:0/665644956 conn(0x7f4348060310 msgr2=0x7f43480b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:50.013 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.002+0000 7f434ebfd700 1 -- 172.21.15.17:0/665644956 shutdown_connections 2022-08-01T19:34:50.013 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.002+0000 7f434ebfd700 1 -- 172.21.15.17:0/665644956 wait complete. 2022-08-01T19:34:50.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.006+0000 7f434ebfd700 1 Processor -- start 2022-08-01T19:34:50.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.006+0000 7f434ebfd700 1 -- start start 2022-08-01T19:34:50.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.006+0000 7f434ebfd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43480a8c50 0x7f43480abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:50.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.006+0000 7f434ebfd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43480b01a0 0x7f43480662a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:50.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.006+0000 7f434ebfd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f43480b2710 0x7f43480b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:50.018 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.006+0000 7f434ebfd700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43480b59e0 con 0x7f43480b01a0 2022-08-01T19:34:50.018 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.006+0000 7f434ebfd700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4348065e40 con 0x7f43480a8c50 2022-08-01T19:34:50.018 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.006+0000 7f434ebfd700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43480b5740 con 0x7f43480b2710 2022-08-01T19:34:50.018 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.006+0000 7f434d3fa700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43480b01a0 0x7f43480662a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:50.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.006+0000 7f434d3fa700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43480b01a0 0x7f43480662a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37762/0 (socket says 172.21.15.17:37762) 2022-08-01T19:34:50.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.006+0000 7f434d3fa700 1 -- 172.21.15.17:0/3533535777 learned_addr learned my addr 172.21.15.17:0/3533535777 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:50.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.006+0000 7f434dbfb700 1 --2- 172.21.15.17:0/3533535777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43480a8c50 0x7f43480abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:50.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.006+0000 7f434e3fc700 1 --2- 172.21.15.17:0/3533535777 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f43480b2710 0x7f43480b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:50.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.006+0000 7f434d3fa700 1 -- 172.21.15.17:0/3533535777 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f43480b2710 msgr2=0x7f43480b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:50.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.006+0000 7f434d3fa700 1 --2- 172.21.15.17:0/3533535777 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f43480b2710 0x7f43480b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:50.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.006+0000 7f434d3fa700 1 -- 172.21.15.17:0/3533535777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43480a8c50 msgr2=0x7f43480abbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:50.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.006+0000 7f434d3fa700 1 --2- 172.21.15.17:0/3533535777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43480a8c50 0x7f43480abbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:50.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.006+0000 7f434d3fa700 1 -- 172.21.15.17:0/3533535777 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f434000b180 con 0x7f43480b01a0 2022-08-01T19:34:50.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.006+0000 7f434e3fc700 1 --2- 172.21.15.17:0/3533535777 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f43480b2710 0x7f43480b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:34:50.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.006+0000 7f434d3fa700 1 --2- 172.21.15.17:0/3533535777 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43480b01a0 0x7f43480662a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f434806b950 tx=0x7f433c007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:50.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.006+0000 7f4346ffd700 1 -- 172.21.15.17:0/3533535777 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f433c0070d0 con 0x7f43480b01a0 2022-08-01T19:34:50.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.006+0000 7f4346ffd700 1 -- 172.21.15.17:0/3533535777 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f433c007230 con 0x7f43480b01a0 2022-08-01T19:34:50.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.006+0000 7f434ebfd700 1 -- 172.21.15.17:0/3533535777 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f434815a7e0 con 0x7f43480b01a0 2022-08-01T19:34:50.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.006+0000 7f4346ffd700 1 -- 172.21.15.17:0/3533535777 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f433c005610 con 0x7f43480b01a0 2022-08-01T19:34:50.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.006+0000 7f434ebfd700 1 -- 172.21.15.17:0/3533535777 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43480afec0 con 0x7f43480b01a0 2022-08-01T19:34:50.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.006+0000 7f4346ffd700 1 -- 172.21.15.17:0/3533535777 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f433c010070 con 0x7f43480b01a0 2022-08-01T19:34:50.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.006+0000 7f4346ffd700 1 --2- 172.21.15.17:0/3533535777 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4328053b00 0x7f4328055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:50.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.006+0000 7f434ebfd700 1 -- 172.21.15.17:0/3533535777 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f434815a640 con 0x7f43480b01a0 2022-08-01T19:34:50.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.010+0000 7f4346ffd700 1 -- 172.21.15.17:0/3533535777 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f433c06b500 con 0x7f43480b01a0 2022-08-01T19:34:50.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.010+0000 7f434dbfb700 1 --2- 172.21.15.17:0/3533535777 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4328053b00 0x7f4328055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:50.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.010+0000 7f434dbfb700 1 --2- 172.21.15.17:0/3533535777 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4328053b00 0x7f4328055fc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f433800d000 tx=0x7f4338007f10).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:50.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.014+0000 7f4346ffd700 1 -- 172.21.15.17:0/3533535777 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f434815a640 con 0x7f43480b01a0 2022-08-01T19:34:50.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.134+0000 7f434ebfd700 1 -- 172.21.15.17:0/3533535777 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f434815a640 con 0x7f43480b01a0 2022-08-01T19:34:50.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.134+0000 7f4346ffd700 1 -- 172.21.15.17:0/3533535777 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v35) v1 ==== 110+0+10 (secure 0 0 0) 0x7f434815a640 con 0x7f43480b01a0 2022-08-01T19:34:50.142 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:34:50.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.138+0000 7f434ebfd700 1 -- 172.21.15.17:0/3533535777 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4328053b00 msgr2=0x7f4328055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:50.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.138+0000 7f434ebfd700 1 --2- 172.21.15.17:0/3533535777 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4328053b00 0x7f4328055fc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f433800d000 tx=0x7f4338007f10).stop 2022-08-01T19:34:50.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.138+0000 7f434ebfd700 1 -- 172.21.15.17:0/3533535777 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43480b01a0 msgr2=0x7f43480662a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:50.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.138+0000 7f434ebfd700 1 --2- 172.21.15.17:0/3533535777 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43480b01a0 0x7f43480662a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f434806b950 tx=0x7f433c007be0).stop 2022-08-01T19:34:50.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.138+0000 7f434ebfd700 1 -- 172.21.15.17:0/3533535777 shutdown_connections 2022-08-01T19:34:50.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.138+0000 7f434ebfd700 1 --2- 172.21.15.17:0/3533535777 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4328053b00 0x7f4328055fc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:50.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.138+0000 7f434ebfd700 1 --2- 172.21.15.17:0/3533535777 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f43480b2710 0x7f43480b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:50.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.138+0000 7f434ebfd700 1 --2- 172.21.15.17:0/3533535777 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43480b01a0 0x7f43480662a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:50.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.138+0000 7f434ebfd700 1 --2- 172.21.15.17:0/3533535777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43480a8c50 0x7f43480abbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:50.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.138+0000 7f434ebfd700 1 -- 172.21.15.17:0/3533535777 >> 172.21.15.17:0/3533535777 conn(0x7f4348060310 msgr2=0x7f434805be30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:50.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.138+0000 7f434ebfd700 1 -- 172.21.15.17:0/3533535777 shutdown_connections 2022-08-01T19:34:50.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.138+0000 7f434ebfd700 1 -- 172.21.15.17:0/3533535777 wait complete. 2022-08-01T19:34:50.158 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:34:50.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.266+0000 7f22418d5700 1 Processor -- start 2022-08-01T19:34:50.271 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.266+0000 7f22418d5700 1 -- start start 2022-08-01T19:34:50.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.266+0000 7f22418d5700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f223c060330 0x7f223c0b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:50.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.266+0000 7f22418d5700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f223c05c6f0 0x7f223c0a8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:50.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.266+0000 7f22418d5700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f223c0aa630 con 0x7f223c0af550 2022-08-01T19:34:50.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.266+0000 7f22418d5700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f223c0b15e0 con 0x7f223c060330 2022-08-01T19:34:50.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.266+0000 7f22418d5700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f223c0af930 con 0x7f223c05c6f0 2022-08-01T19:34:50.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.266+0000 7f223bfff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f223c060330 0x7f223c0b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:50.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.266+0000 7f22410d4700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f223c05c6f0 0x7f223c0a8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:50.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.266+0000 7f223bfff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f223c060330 0x7f223c0b0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37770/0 (socket says 172.21.15.17:37770) 2022-08-01T19:34:50.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.266+0000 7f223bfff700 1 -- 172.21.15.17:0/3079292877 learned_addr learned my addr 172.21.15.17:0/3079292877 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:50.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.266+0000 7f223bfff700 1 -- 172.21.15.17:0/3079292877 >> v2:172.21.15.77:3300/0 conn(0x7f223c05c6f0 msgr2=0x7f223c0a8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:50.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.266+0000 7f223bfff700 1 --2- 172.21.15.17:0/3079292877 >> v2:172.21.15.77:3300/0 conn(0x7f223c05c6f0 0x7f223c0a8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:50.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.266+0000 7f223bfff700 1 -- 172.21.15.17:0/3079292877 >> v1:172.21.15.77:6789/0 conn(0x7f223c0af550 legacy=0x7f223c1445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:50.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.266+0000 7f223bfff700 1 -- 172.21.15.17:0/3079292877 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f223c05fd10 con 0x7f223c060330 2022-08-01T19:34:50.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.266+0000 7f223bfff700 1 --2- 172.21.15.17:0/3079292877 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f223c060330 0x7f223c0b0e40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f2228002f70 tx=0x7f2228006eb0).ready entity=mon.0 client_cookie=e8f12ad2bea3d96c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:50.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.266+0000 7f223b7fe700 1 -- 172.21.15.17:0/3079292877 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f222800d5c0 con 0x7f223c060330 2022-08-01T19:34:50.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.266+0000 7f223b7fe700 1 -- 172.21.15.17:0/3079292877 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2228000e90 con 0x7f223c060330 2022-08-01T19:34:50.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.270+0000 7f223b7fe700 1 -- 172.21.15.17:0/3079292877 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f222800d950 con 0x7f223c060330 2022-08-01T19:34:50.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.270+0000 7f22418d5700 1 -- 172.21.15.17:0/3079292877 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f223c060330 msgr2=0x7f223c0b0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:50.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.270+0000 7f22418d5700 1 --2- 172.21.15.17:0/3079292877 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f223c060330 0x7f223c0b0e40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f2228002f70 tx=0x7f2228006eb0).stop 2022-08-01T19:34:50.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.270+0000 7f22418d5700 1 -- 172.21.15.17:0/3079292877 shutdown_connections 2022-08-01T19:34:50.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.270+0000 7f22418d5700 1 --2- 172.21.15.17:0/3079292877 >> v2:172.21.15.77:3300/0 conn(0x7f223c05c6f0 0x7f223c0a8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:50.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.270+0000 7f22418d5700 1 --2- 172.21.15.17:0/3079292877 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f223c060330 0x7f223c0b0e40 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:50.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.270+0000 7f22418d5700 1 -- 172.21.15.17:0/3079292877 >> 172.21.15.17:0/3079292877 conn(0x7f223c0612d0 msgr2=0x7f223c062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:50.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.270+0000 7f22418d5700 1 -- 172.21.15.17:0/3079292877 shutdown_connections 2022-08-01T19:34:50.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.270+0000 7f22418d5700 1 -- 172.21.15.17:0/3079292877 wait complete. 2022-08-01T19:34:50.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.270+0000 7f22418d5700 1 Processor -- start 2022-08-01T19:34:50.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.270+0000 7f22418d5700 1 -- start start 2022-08-01T19:34:50.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.274+0000 7f22418d5700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f223c05c6f0 0x7f223c142ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:50.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.274+0000 7f22418d5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f223c060330 0x7f223c1501c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:50.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.274+0000 7f22418d5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f223c0af550 0x7f223c063ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:50.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.274+0000 7f22418d5700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223c143090 con 0x7f223c0af550 2022-08-01T19:34:50.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.274+0000 7f22418d5700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223c064780 con 0x7f223c060330 2022-08-01T19:34:50.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.274+0000 7f22418d5700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f223c143700 con 0x7f223c05c6f0 2022-08-01T19:34:50.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.274+0000 7f22410d4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f223c0af550 0x7f223c063ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:50.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.274+0000 7f22410d4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f223c0af550 0x7f223c063ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37778/0 (socket says 172.21.15.17:37778) 2022-08-01T19:34:50.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.274+0000 7f22410d4700 1 -- 172.21.15.17:0/3732293937 learned_addr learned my addr 172.21.15.17:0/3732293937 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:50.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.274+0000 7f22408d3700 1 --2- 172.21.15.17:0/3732293937 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f223c05c6f0 0x7f223c142ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:50.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.274+0000 7f223bfff700 1 --2- 172.21.15.17:0/3732293937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f223c060330 0x7f223c1501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:50.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.274+0000 7f22410d4700 1 -- 172.21.15.17:0/3732293937 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f223c05c6f0 msgr2=0x7f223c142ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:50.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.274+0000 7f22410d4700 1 --2- 172.21.15.17:0/3732293937 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f223c05c6f0 0x7f223c142ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:50.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.274+0000 7f22410d4700 1 -- 172.21.15.17:0/3732293937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f223c060330 msgr2=0x7f223c1501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:50.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.274+0000 7f22410d4700 1 --2- 172.21.15.17:0/3732293937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f223c060330 0x7f223c1501c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:50.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.274+0000 7f22410d4700 1 -- 172.21.15.17:0/3732293937 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f222800b1e0 con 0x7f223c0af550 2022-08-01T19:34:50.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.274+0000 7f22410d4700 1 --2- 172.21.15.17:0/3732293937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f223c0af550 0x7f223c063ad0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f2234006e00 tx=0x7f22340091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:50.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.274+0000 7f2239ffb700 1 -- 172.21.15.17:0/3732293937 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f223400c640 con 0x7f223c0af550 2022-08-01T19:34:50.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.274+0000 7f2239ffb700 1 -- 172.21.15.17:0/3732293937 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f223400c7a0 con 0x7f223c0af550 2022-08-01T19:34:50.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.274+0000 7f2239ffb700 1 -- 172.21.15.17:0/3732293937 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f223400cb00 con 0x7f223c0af550 2022-08-01T19:34:50.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.274+0000 7f22418d5700 1 -- 172.21.15.17:0/3732293937 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f223c05f840 con 0x7f223c0af550 2022-08-01T19:34:50.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.274+0000 7f22418d5700 1 -- 172.21.15.17:0/3732293937 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f223c143e70 con 0x7f223c0af550 2022-08-01T19:34:50.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.274+0000 7f22418d5700 1 -- 172.21.15.17:0/3732293937 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f223c1438c0 con 0x7f223c0af550 2022-08-01T19:34:50.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.274+0000 7f2239ffb700 1 -- 172.21.15.17:0/3732293937 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f223400cc60 con 0x7f223c0af550 2022-08-01T19:34:50.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.278+0000 7f2239ffb700 1 --2- 172.21.15.17:0/3732293937 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f221c053a70 0x7f221c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:50.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.278+0000 7f2239ffb700 1 -- 172.21.15.17:0/3732293937 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f223406c2f0 con 0x7f223c0af550 2022-08-01T19:34:50.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.278+0000 7f22408d3700 1 --2- 172.21.15.17:0/3732293937 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f221c053a70 0x7f221c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:50.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.278+0000 7f22408d3700 1 --2- 172.21.15.17:0/3732293937 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f221c053a70 0x7f221c055f30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f223000d000 tx=0x7f2230009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:50.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.290+0000 7f2239ffb700 1 -- 172.21.15.17:0/3732293937 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f223c1438c0 con 0x7f223c0af550 2022-08-01T19:34:50.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.410+0000 7f22418d5700 1 -- 172.21.15.17:0/3732293937 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f223c1438c0 con 0x7f223c0af550 2022-08-01T19:34:50.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.414+0000 7f2239ffb700 1 -- 172.21.15.17:0/3732293937 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v35) v1 ==== 92+0+10 (secure 0 0 0) 0x7f223c1438c0 con 0x7f223c0af550 2022-08-01T19:34:50.418 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:34:50.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.414+0000 7f22418d5700 1 -- 172.21.15.17:0/3732293937 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f221c053a70 msgr2=0x7f221c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:50.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.414+0000 7f22418d5700 1 --2- 172.21.15.17:0/3732293937 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f221c053a70 0x7f221c055f30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f223000d000 tx=0x7f2230009040).stop 2022-08-01T19:34:50.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.414+0000 7f22418d5700 1 -- 172.21.15.17:0/3732293937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f223c0af550 msgr2=0x7f223c063ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:50.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.414+0000 7f22418d5700 1 --2- 172.21.15.17:0/3732293937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f223c0af550 0x7f223c063ad0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f2234006e00 tx=0x7f22340091d0).stop 2022-08-01T19:34:50.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.414+0000 7f22418d5700 1 -- 172.21.15.17:0/3732293937 shutdown_connections 2022-08-01T19:34:50.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.414+0000 7f22418d5700 1 --2- 172.21.15.17:0/3732293937 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f221c053a70 0x7f221c055f30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:50.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.414+0000 7f22418d5700 1 --2- 172.21.15.17:0/3732293937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f223c0af550 0x7f223c063ad0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:50.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.414+0000 7f22418d5700 1 --2- 172.21.15.17:0/3732293937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f223c060330 0x7f223c1501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:50.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.414+0000 7f22418d5700 1 --2- 172.21.15.17:0/3732293937 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f223c05c6f0 0x7f223c142ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:50.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.414+0000 7f22418d5700 1 -- 172.21.15.17:0/3732293937 >> 172.21.15.17:0/3732293937 conn(0x7f223c0612d0 msgr2=0x7f223c141a10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:50.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.414+0000 7f22418d5700 1 -- 172.21.15.17:0/3732293937 shutdown_connections 2022-08-01T19:34:50.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.414+0000 7f22418d5700 1 -- 172.21.15.17:0/3732293937 wait complete. 2022-08-01T19:34:50.436 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T19:34:50.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.546+0000 7f6e35522700 1 Processor -- start 2022-08-01T19:34:50.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.546+0000 7f6e35522700 1 -- start start 2022-08-01T19:34:50.550 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.546+0000 7f6e35522700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f6e300b2720 0x7f6e300b0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:50.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.546+0000 7f6e35522700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6e3005c6d0 0x7f6e30143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:50.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.546+0000 7f6e35522700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e30147470 con 0x7f6e300b1ed0 2022-08-01T19:34:50.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.546+0000 7f6e35522700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e30143110 con 0x7f6e3005c6d0 2022-08-01T19:34:50.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.546+0000 7f6e35522700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e30140e40 con 0x7f6e300b2720 2022-08-01T19:34:50.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.546+0000 7f6e2f7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6e3005c6d0 0x7f6e30143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:50.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.546+0000 7f6e2ffff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f6e300b2720 0x7f6e300b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:50.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.546+0000 7f6e2ffff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f6e300b2720 0x7f6e300b0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:35704/0 (socket says 172.21.15.17:35704) 2022-08-01T19:34:50.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.546+0000 7f6e2f7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6e3005c6d0 0x7f6e30143a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37784/0 (socket says 172.21.15.17:37784) 2022-08-01T19:34:50.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.546+0000 7f6e2ffff700 1 -- 172.21.15.17:0/781638681 learned_addr learned my addr 172.21.15.17:0/781638681 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:50.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.546+0000 7f6e2f7fe700 1 -- 172.21.15.17:0/781638681 >> v2:172.21.15.77:3300/0 conn(0x7f6e300b2720 msgr2=0x7f6e300b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:50.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.546+0000 7f6e2f7fe700 1 --2- 172.21.15.17:0/781638681 >> v2:172.21.15.77:3300/0 conn(0x7f6e300b2720 0x7f6e300b0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:50.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.546+0000 7f6e2f7fe700 1 -- 172.21.15.17:0/781638681 >> v1:172.21.15.77:6789/0 conn(0x7f6e300b1ed0 legacy=0x7f6e300b0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:50.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.546+0000 7f6e2f7fe700 1 -- 172.21.15.17:0/781638681 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e30143300 con 0x7f6e3005c6d0 2022-08-01T19:34:50.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.546+0000 7f6e2f7fe700 1 --2- 172.21.15.17:0/781638681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e3005c6d0 0x7f6e30143a40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f6e18002f70 tx=0x7f6e18006df0).ready entity=mon.0 client_cookie=41d6b95968eb68db server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:50.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.546+0000 7f6e2effd700 1 -- 172.21.15.17:0/781638681 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e1800d570 con 0x7f6e3005c6d0 2022-08-01T19:34:50.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.546+0000 7f6e2effd700 1 -- 172.21.15.17:0/781638681 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e18000e90 con 0x7f6e3005c6d0 2022-08-01T19:34:50.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.546+0000 7f6e2effd700 1 -- 172.21.15.17:0/781638681 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e1800d8c0 con 0x7f6e3005c6d0 2022-08-01T19:34:50.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.546+0000 7f6e35522700 1 -- 172.21.15.17:0/781638681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e3005c6d0 msgr2=0x7f6e30143a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:50.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.546+0000 7f6e35522700 1 --2- 172.21.15.17:0/781638681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e3005c6d0 0x7f6e30143a40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f6e18002f70 tx=0x7f6e18006df0).stop 2022-08-01T19:34:50.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.546+0000 7f6e35522700 1 -- 172.21.15.17:0/781638681 shutdown_connections 2022-08-01T19:34:50.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.546+0000 7f6e35522700 1 --2- 172.21.15.17:0/781638681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e3005c6d0 0x7f6e30143a40 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:50.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.546+0000 7f6e35522700 1 --2- 172.21.15.17:0/781638681 >> v2:172.21.15.77:3300/0 conn(0x7f6e300b2720 0x7f6e300b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:50.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.546+0000 7f6e35522700 1 -- 172.21.15.17:0/781638681 >> 172.21.15.17:0/781638681 conn(0x7f6e30061360 msgr2=0x7f6e30062f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:50.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.546+0000 7f6e35522700 1 -- 172.21.15.17:0/781638681 shutdown_connections 2022-08-01T19:34:50.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.546+0000 7f6e35522700 1 -- 172.21.15.17:0/781638681 wait complete. 2022-08-01T19:34:50.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.550+0000 7f6e35522700 1 Processor -- start 2022-08-01T19:34:50.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.550+0000 7f6e35522700 1 -- start start 2022-08-01T19:34:50.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.550+0000 7f6e35522700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6e300b1ed0 0x7f6e30144790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:50.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.550+0000 7f6e35522700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e300b2720 0x7f6e300abbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:50.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.550+0000 7f6e35522700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e300aab60 0x7f6e300ac1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:50.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.550+0000 7f6e35522700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e300656a0 con 0x7f6e300aab60 2022-08-01T19:34:50.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.550+0000 7f6e35522700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e30144360 con 0x7f6e300b2720 2022-08-01T19:34:50.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.550+0000 7f6e35522700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e300aa0e0 con 0x7f6e300b1ed0 2022-08-01T19:34:50.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.550+0000 7f6e34d21700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e300aab60 0x7f6e300ac1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:50.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.550+0000 7f6e2ffff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6e300b1ed0 0x7f6e30144790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:50.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.550+0000 7f6e34d21700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e300aab60 0x7f6e300ac1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37792/0 (socket says 172.21.15.17:37792) 2022-08-01T19:34:50.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.550+0000 7f6e34d21700 1 -- 172.21.15.17:0/3953630300 learned_addr learned my addr 172.21.15.17:0/3953630300 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:50.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.550+0000 7f6e2f7fe700 1 --2- 172.21.15.17:0/3953630300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e300b2720 0x7f6e300abbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:50.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.550+0000 7f6e34d21700 1 -- 172.21.15.17:0/3953630300 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6e300b1ed0 msgr2=0x7f6e30144790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:50.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.550+0000 7f6e34d21700 1 --2- 172.21.15.17:0/3953630300 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6e300b1ed0 0x7f6e30144790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:50.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.550+0000 7f6e34d21700 1 -- 172.21.15.17:0/3953630300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e300b2720 msgr2=0x7f6e300abbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:50.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.550+0000 7f6e34d21700 1 --2- 172.21.15.17:0/3953630300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e300b2720 0x7f6e300abbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:50.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.550+0000 7f6e34d21700 1 -- 172.21.15.17:0/3953630300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e1800b180 con 0x7f6e300aab60 2022-08-01T19:34:50.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.550+0000 7f6e34d21700 1 --2- 172.21.15.17:0/3953630300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e300aab60 0x7f6e300ac1a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f6e30064510 tx=0x7f6e28007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:50.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.550+0000 7f6e2d7fa700 1 -- 172.21.15.17:0/3953630300 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e2800e4a0 con 0x7f6e300aab60 2022-08-01T19:34:50.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.550+0000 7f6e35522700 1 -- 172.21.15.17:0/3953630300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e300aa380 con 0x7f6e300aab60 2022-08-01T19:34:50.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.550+0000 7f6e2d7fa700 1 -- 172.21.15.17:0/3953630300 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e2800e600 con 0x7f6e300aab60 2022-08-01T19:34:50.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.550+0000 7f6e2d7fa700 1 -- 172.21.15.17:0/3953630300 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e2800e960 con 0x7f6e300aab60 2022-08-01T19:34:50.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.550+0000 7f6e35522700 1 -- 172.21.15.17:0/3953630300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e30145140 con 0x7f6e300aab60 2022-08-01T19:34:50.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.554+0000 7f6e35522700 1 -- 172.21.15.17:0/3953630300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e301444f0 con 0x7f6e300aab60 2022-08-01T19:34:50.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.554+0000 7f6e2d7fa700 1 -- 172.21.15.17:0/3953630300 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6e28019030 con 0x7f6e300aab60 2022-08-01T19:34:50.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.554+0000 7f6e2d7fa700 1 --2- 172.21.15.17:0/3953630300 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6e14053aa0 0x7f6e14055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:50.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.554+0000 7f6e2d7fa700 1 -- 172.21.15.17:0/3953630300 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6e2806b7f0 con 0x7f6e300aab60 2022-08-01T19:34:50.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.554+0000 7f6e2ffff700 1 --2- 172.21.15.17:0/3953630300 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6e14053aa0 0x7f6e14055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:50.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.554+0000 7f6e2ffff700 1 --2- 172.21.15.17:0/3953630300 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6e14053aa0 0x7f6e14055f60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f6e20008000 tx=0x7f6e20008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:50.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.562+0000 7f6e2d7fa700 1 -- 172.21.15.17:0/3953630300 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6e301444f0 con 0x7f6e300aab60 2022-08-01T19:34:50.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.686+0000 7f6e35522700 1 -- 172.21.15.17:0/3953630300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6e301444f0 con 0x7f6e300aab60 2022-08-01T19:34:50.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.690+0000 7f6e2d7fa700 1 -- 172.21.15.17:0/3953630300 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) v1 ==== 104+0+10 (secure 0 0 0) 0x7f6e301444f0 con 0x7f6e300aab60 2022-08-01T19:34:50.695 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:34:50.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.690+0000 7f6e35522700 1 -- 172.21.15.17:0/3953630300 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6e14053aa0 msgr2=0x7f6e14055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:50.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.690+0000 7f6e35522700 1 --2- 172.21.15.17:0/3953630300 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6e14053aa0 0x7f6e14055f60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f6e20008000 tx=0x7f6e20008040).stop 2022-08-01T19:34:50.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.690+0000 7f6e35522700 1 -- 172.21.15.17:0/3953630300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e300aab60 msgr2=0x7f6e300ac1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:50.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.690+0000 7f6e35522700 1 --2- 172.21.15.17:0/3953630300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e300aab60 0x7f6e300ac1a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f6e30064510 tx=0x7f6e28007f40).stop 2022-08-01T19:34:50.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.690+0000 7f6e35522700 1 -- 172.21.15.17:0/3953630300 shutdown_connections 2022-08-01T19:34:50.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.690+0000 7f6e35522700 1 --2- 172.21.15.17:0/3953630300 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6e14053aa0 0x7f6e14055f60 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:50.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.690+0000 7f6e35522700 1 --2- 172.21.15.17:0/3953630300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e300aab60 0x7f6e300ac1a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:50.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.690+0000 7f6e35522700 1 --2- 172.21.15.17:0/3953630300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e300b2720 0x7f6e300abbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:50.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.690+0000 7f6e35522700 1 --2- 172.21.15.17:0/3953630300 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6e300b1ed0 0x7f6e30144790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:50.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.690+0000 7f6e35522700 1 -- 172.21.15.17:0/3953630300 >> 172.21.15.17:0/3953630300 conn(0x7f6e30061360 msgr2=0x7f6e300b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:50.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.690+0000 7f6e35522700 1 -- 172.21.15.17:0/3953630300 shutdown_connections 2022-08-01T19:34:50.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.690+0000 7f6e35522700 1 -- 172.21.15.17:0/3953630300 wait complete. 2022-08-01T19:34:50.708 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T19:34:50.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.814+0000 7fc402869700 1 Processor -- start 2022-08-01T19:34:50.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.814+0000 7fc402869700 1 -- start start 2022-08-01T19:34:50.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.814+0000 7fc402869700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc3fc0b2710 0x7fc3fc14afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:50.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.814+0000 7fc402869700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc3fc0a8c50 0x7fc3fc14b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:50.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.814+0000 7fc402869700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3fc0a7250 con 0x7fc3fc0b01a0 2022-08-01T19:34:50.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.814+0000 7fc402869700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3fc0a7480 con 0x7fc3fc0a8c50 2022-08-01T19:34:50.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.814+0000 7fc402869700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc0a7720 con 0x7fc3fc0b2710 2022-08-01T19:34:50.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.814+0000 7fc402068700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc3fc0a8c50 0x7fc3fc14b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:50.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.814+0000 7fc402068700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc3fc0a8c50 0x7fc3fc14b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37798/0 (socket says 172.21.15.17:37798) 2022-08-01T19:34:50.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.814+0000 7fc402068700 1 -- 172.21.15.17:0/3719107483 learned_addr learned my addr 172.21.15.17:0/3719107483 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:50.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.814+0000 7fc401867700 1 --2- 172.21.15.17:0/3719107483 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc3fc0b2710 0x7fc3fc14afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:50.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.814+0000 7fc402068700 1 -- 172.21.15.17:0/3719107483 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc3fc0b2710 msgr2=0x7fc3fc14afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:50.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.814+0000 7fc402068700 1 --2- 172.21.15.17:0/3719107483 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc3fc0b2710 0x7fc3fc14afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:50.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.814+0000 7fc402068700 1 -- 172.21.15.17:0/3719107483 >> v1:172.21.15.77:6789/0 conn(0x7fc3fc0b01a0 legacy=0x7fc3fc0a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:50.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.814+0000 7fc402068700 1 -- 172.21.15.17:0/3719107483 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3fc0a79c0 con 0x7fc3fc0a8c50 2022-08-01T19:34:50.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.814+0000 7fc401867700 1 --2- 172.21.15.17:0/3719107483 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc3fc0b2710 0x7fc3fc14afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:34:50.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.814+0000 7fc402068700 1 --2- 172.21.15.17:0/3719107483 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3fc0a8c50 0x7fc3fc14b4c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7fc3f4002f70 tx=0x7fc3f4006eb0).ready entity=mon.0 client_cookie=20a83b4786fb36c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:50.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.814+0000 7fc400865700 1 -- 172.21.15.17:0/3719107483 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3f400d5c0 con 0x7fc3fc0a8c50 2022-08-01T19:34:50.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.814+0000 7fc400865700 1 -- 172.21.15.17:0/3719107483 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3f4000e90 con 0x7fc3fc0a8c50 2022-08-01T19:34:50.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.814+0000 7fc400865700 1 -- 172.21.15.17:0/3719107483 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3f400d950 con 0x7fc3fc0a8c50 2022-08-01T19:34:50.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.818+0000 7fc402869700 1 -- 172.21.15.17:0/3719107483 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3fc0a8c50 msgr2=0x7fc3fc14b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:50.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.818+0000 7fc402869700 1 --2- 172.21.15.17:0/3719107483 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3fc0a8c50 0x7fc3fc14b4c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7fc3f4002f70 tx=0x7fc3f4006eb0).stop 2022-08-01T19:34:50.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.818+0000 7fc402869700 1 -- 172.21.15.17:0/3719107483 shutdown_connections 2022-08-01T19:34:50.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.818+0000 7fc402869700 1 --2- 172.21.15.17:0/3719107483 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3fc0a8c50 0x7fc3fc14b4c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:50.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.818+0000 7fc402869700 1 --2- 172.21.15.17:0/3719107483 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc3fc0b2710 0x7fc3fc14afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:50.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.818+0000 7fc402869700 1 -- 172.21.15.17:0/3719107483 >> 172.21.15.17:0/3719107483 conn(0x7fc3fc060310 msgr2=0x7fc3fc0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:50.834 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.818+0000 7fc402869700 1 -- 172.21.15.17:0/3719107483 shutdown_connections 2022-08-01T19:34:50.834 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.818+0000 7fc402869700 1 -- 172.21.15.17:0/3719107483 wait complete. 2022-08-01T19:34:50.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.818+0000 7fc402869700 1 Processor -- start 2022-08-01T19:34:50.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.818+0000 7fc402869700 1 -- start start 2022-08-01T19:34:50.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.818+0000 7fc402869700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc3fc0a8c50 0x7fc3fc0b3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:50.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.818+0000 7fc402869700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3fc0b01a0 0x7fc3fc0aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:50.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.818+0000 7fc402869700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3fc0b2710 0x7fc3fc1433e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:50.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.818+0000 7fc402869700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc157820 con 0x7fc3fc0b2710 2022-08-01T19:34:50.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.818+0000 7fc402869700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc142ce0 con 0x7fc3fc0b01a0 2022-08-01T19:34:50.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.818+0000 7fc402869700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc156bd0 con 0x7fc3fc0a8c50 2022-08-01T19:34:50.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.818+0000 7fc402068700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3fc0b2710 0x7fc3fc1433e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:50.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.818+0000 7fc402068700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3fc0b2710 0x7fc3fc1433e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37806/0 (socket says 172.21.15.17:37806) 2022-08-01T19:34:50.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.818+0000 7fc402068700 1 -- 172.21.15.17:0/3513485610 learned_addr learned my addr 172.21.15.17:0/3513485610 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:50.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.818+0000 7fc401867700 1 --2- 172.21.15.17:0/3513485610 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc3fc0a8c50 0x7fc3fc0b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:50.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.818+0000 7fc401066700 1 --2- 172.21.15.17:0/3513485610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3fc0b01a0 0x7fc3fc0aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:50.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.818+0000 7fc402068700 1 -- 172.21.15.17:0/3513485610 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc3fc0a8c50 msgr2=0x7fc3fc0b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:50.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.818+0000 7fc402068700 1 --2- 172.21.15.17:0/3513485610 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc3fc0a8c50 0x7fc3fc0b3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:50.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.818+0000 7fc402068700 1 -- 172.21.15.17:0/3513485610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3fc0b01a0 msgr2=0x7fc3fc0aef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:50.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.818+0000 7fc402068700 1 --2- 172.21.15.17:0/3513485610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3fc0b01a0 0x7fc3fc0aef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:50.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.818+0000 7fc402068700 1 -- 172.21.15.17:0/3513485610 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f400b1e0 con 0x7fc3fc0b2710 2022-08-01T19:34:50.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.818+0000 7fc402068700 1 --2- 172.21.15.17:0/3513485610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3fc0b2710 0x7fc3fc1433e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7fc3f4002f70 tx=0x7fc3f4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:50.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.822+0000 7fc3f2ffd700 1 -- 172.21.15.17:0/3513485610 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3f40116e0 con 0x7fc3fc0b2710 2022-08-01T19:34:50.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.822+0000 7fc3f2ffd700 1 -- 172.21.15.17:0/3513485610 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3f4011840 con 0x7fc3fc0b2710 2022-08-01T19:34:50.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.822+0000 7fc402869700 1 -- 172.21.15.17:0/3513485610 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3fc156930 con 0x7fc3fc0b2710 2022-08-01T19:34:50.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.822+0000 7fc3f2ffd700 1 -- 172.21.15.17:0/3513485610 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3f4011ba0 con 0x7fc3fc0b2710 2022-08-01T19:34:50.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.822+0000 7fc402869700 1 -- 172.21.15.17:0/3513485610 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3fc0af570 con 0x7fc3fc0b2710 2022-08-01T19:34:50.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.822+0000 7fc402869700 1 -- 172.21.15.17:0/3513485610 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3fc0574e0 con 0x7fc3fc0b2710 2022-08-01T19:34:50.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.822+0000 7fc3f2ffd700 1 -- 172.21.15.17:0/3513485610 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc3f4025030 con 0x7fc3fc0b2710 2022-08-01T19:34:50.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.822+0000 7fc3f2ffd700 1 --2- 172.21.15.17:0/3513485610 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc3e4053aa0 0x7fc3e4055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:50.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.822+0000 7fc3f2ffd700 1 -- 172.21.15.17:0/3513485610 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc3f406d190 con 0x7fc3fc0b2710 2022-08-01T19:34:50.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.822+0000 7fc401867700 1 --2- 172.21.15.17:0/3513485610 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc3e4053aa0 0x7fc3e4055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:50.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.822+0000 7fc401867700 1 --2- 172.21.15.17:0/3513485610 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc3e4053aa0 0x7fc3e4055f60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fc3f800c000 tx=0x7fc3f8008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:50.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.826+0000 7fc3f2ffd700 1 -- 172.21.15.17:0/3513485610 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc3f4043f50 con 0x7fc3fc0b2710 2022-08-01T19:34:50.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.954+0000 7fc402869700 1 -- 172.21.15.17:0/3513485610 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc3fc05d410 con 0x7fc3fc0b2710 2022-08-01T19:34:50.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.954+0000 7fc3f2ffd700 1 -- 172.21.15.17:0/3513485610 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) v1 ==== 100+0+10 (secure 0 0 0) 0x7fc3f40230a0 con 0x7fc3fc0b2710 2022-08-01T19:34:50.962 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:34:50.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.958+0000 7fc402869700 1 -- 172.21.15.17:0/3513485610 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc3e4053aa0 msgr2=0x7fc3e4055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:50.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.958+0000 7fc402869700 1 --2- 172.21.15.17:0/3513485610 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc3e4053aa0 0x7fc3e4055f60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fc3f800c000 tx=0x7fc3f8008040).stop 2022-08-01T19:34:50.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.958+0000 7fc402869700 1 -- 172.21.15.17:0/3513485610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3fc0b2710 msgr2=0x7fc3fc1433e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:50.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.958+0000 7fc402869700 1 --2- 172.21.15.17:0/3513485610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3fc0b2710 0x7fc3fc1433e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7fc3f4002f70 tx=0x7fc3f4015000).stop 2022-08-01T19:34:50.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.958+0000 7fc402869700 1 -- 172.21.15.17:0/3513485610 shutdown_connections 2022-08-01T19:34:50.966 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.958+0000 7fc402869700 1 --2- 172.21.15.17:0/3513485610 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc3e4053aa0 0x7fc3e4055f60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:50.966 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.958+0000 7fc402869700 1 --2- 172.21.15.17:0/3513485610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3fc0b2710 0x7fc3fc1433e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:50.966 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.958+0000 7fc402869700 1 --2- 172.21.15.17:0/3513485610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3fc0b01a0 0x7fc3fc0aef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:50.966 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.958+0000 7fc402869700 1 --2- 172.21.15.17:0/3513485610 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc3fc0a8c50 0x7fc3fc0b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:50.967 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.958+0000 7fc402869700 1 -- 172.21.15.17:0/3513485610 >> 172.21.15.17:0/3513485610 conn(0x7fc3fc060310 msgr2=0x7fc3fc1461a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:50.967 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.958+0000 7fc402869700 1 -- 172.21.15.17:0/3513485610 shutdown_connections 2022-08-01T19:34:50.967 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:50.958+0000 7fc402869700 1 -- 172.21.15.17:0/3513485610 wait complete. 2022-08-01T19:34:50.974 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T19:34:50.975 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:34:51.078 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.070+0000 7f144ab20700 1 Processor -- start 2022-08-01T19:34:51.078 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.070+0000 7f144ab20700 1 -- start start 2022-08-01T19:34:51.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.074+0000 7f144ab20700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f14440b01a0 0x7f14440a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:51.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.074+0000 7f144ab20700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f14440afd60 0x7f14440a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:51.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.074+0000 7f144ab20700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14440a7250 con 0x7f14440b2710 2022-08-01T19:34:51.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.074+0000 7f144ab20700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14440a7480 con 0x7f14440afd60 2022-08-01T19:34:51.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.074+0000 7f144ab20700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14440a7720 con 0x7f14440b01a0 2022-08-01T19:34:51.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.074+0000 7f144a31f700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f14440afd60 0x7f14440a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:51.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.074+0000 7f144931d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f14440b01a0 0x7f14440a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:51.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.074+0000 7f144a31f700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f14440afd60 0x7f14440a8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:35736/0 (socket says 172.21.15.17:35736) 2022-08-01T19:34:51.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.074+0000 7f144a31f700 1 -- 172.21.15.17:0/273587315 learned_addr learned my addr 172.21.15.17:0/273587315 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:51.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.074+0000 7f144a31f700 1 -- 172.21.15.17:0/273587315 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f14440b01a0 msgr2=0x7f14440a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:34:51.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.074+0000 7f144a31f700 1 --2- 172.21.15.17:0/273587315 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f14440b01a0 0x7f14440a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:51.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.074+0000 7f144a31f700 1 -- 172.21.15.17:0/273587315 >> v1:172.21.15.77:6789/0 conn(0x7f14440b2710 legacy=0x7f144414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:51.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.074+0000 7f144a31f700 1 -- 172.21.15.17:0/273587315 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14440a79c0 con 0x7f14440afd60 2022-08-01T19:34:51.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.074+0000 7f144a31f700 1 --2- 172.21.15.17:0/273587315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14440afd60 0x7f14440a8a30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f143c0032f0 tx=0x7f143c009290).ready entity=mon.1 client_cookie=eaccbbd523500d8d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:51.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.074+0000 7f1448b1c700 1 -- 172.21.15.17:0/273587315 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f143c00b8a0 con 0x7f14440afd60 2022-08-01T19:34:51.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.074+0000 7f1448b1c700 1 -- 172.21.15.17:0/273587315 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143c00ba00 con 0x7f14440afd60 2022-08-01T19:34:51.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.074+0000 7f1448b1c700 1 -- 172.21.15.17:0/273587315 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f143c00bb70 con 0x7f14440afd60 2022-08-01T19:34:51.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.074+0000 7f144ab20700 1 -- 172.21.15.17:0/273587315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14440afd60 msgr2=0x7f14440a8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:51.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.074+0000 7f144ab20700 1 --2- 172.21.15.17:0/273587315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14440afd60 0x7f14440a8a30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f143c0032f0 tx=0x7f143c009290).stop 2022-08-01T19:34:51.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.074+0000 7f144ab20700 1 -- 172.21.15.17:0/273587315 shutdown_connections 2022-08-01T19:34:51.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.074+0000 7f144ab20700 1 --2- 172.21.15.17:0/273587315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14440afd60 0x7f14440a8a30 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:51.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.074+0000 7f144ab20700 1 --2- 172.21.15.17:0/273587315 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f14440b01a0 0x7f14440a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:51.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.074+0000 7f144ab20700 1 -- 172.21.15.17:0/273587315 >> 172.21.15.17:0/273587315 conn(0x7f1444060310 msgr2=0x7f14440b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:51.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.074+0000 7f144ab20700 1 -- 172.21.15.17:0/273587315 shutdown_connections 2022-08-01T19:34:51.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.074+0000 7f144ab20700 1 -- 172.21.15.17:0/273587315 wait complete. 2022-08-01T19:34:51.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.078+0000 7f144ab20700 1 Processor -- start 2022-08-01T19:34:51.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.078+0000 7f144ab20700 1 -- start start 2022-08-01T19:34:51.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.078+0000 7f144ab20700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14440afd60 0x7f1444148cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:51.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.078+0000 7f144ab20700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14440b01a0 0x7f14440b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:51.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.078+0000 7f144ab20700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f14440b2710 0x7f14440b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:51.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.078+0000 7f144ab20700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1444065380 con 0x7f14440afd60 2022-08-01T19:34:51.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.078+0000 7f144ab20700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14440b3cd0 con 0x7f14440b01a0 2022-08-01T19:34:51.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.078+0000 7f144ab20700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14440b4400 con 0x7f14440b2710 2022-08-01T19:34:51.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.078+0000 7f1449b1e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14440afd60 0x7f1444148cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:51.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.078+0000 7f144931d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14440b01a0 0x7f14440b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:51.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.078+0000 7f144931d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14440b01a0 0x7f14440b57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:35740/0 (socket says 172.21.15.17:35740) 2022-08-01T19:34:51.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.078+0000 7f1449b1e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14440afd60 0x7f1444148cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37816/0 (socket says 172.21.15.17:37816) 2022-08-01T19:34:51.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.078+0000 7f144931d700 1 -- 172.21.15.17:0/3218221464 learned_addr learned my addr 172.21.15.17:0/3218221464 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:34:51.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.078+0000 7f144a31f700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f14440b2710 0x7f14440b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:51.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.078+0000 7f144931d700 1 -- 172.21.15.17:0/3218221464 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f14440b2710 msgr2=0x7f14440b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:51.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.078+0000 7f144931d700 1 --2- 172.21.15.17:0/3218221464 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f14440b2710 0x7f14440b5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:51.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.078+0000 7f144931d700 1 -- 172.21.15.17:0/3218221464 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14440afd60 msgr2=0x7f1444148cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:51.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.078+0000 7f144931d700 1 --2- 172.21.15.17:0/3218221464 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14440afd60 0x7f1444148cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:51.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.078+0000 7f144931d700 1 -- 172.21.15.17:0/3218221464 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f143c009530 con 0x7f14440b01a0 2022-08-01T19:34:51.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.078+0000 7f144a31f700 1 --2- 172.21.15.17:0/3218221464 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f14440b2710 0x7f14440b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:34:51.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.078+0000 7f1449b1e700 1 --2- 172.21.15.17:0/3218221464 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14440afd60 0x7f1444148cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:34:51.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.078+0000 7f144931d700 1 --2- 172.21.15.17:0/3218221464 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14440b01a0 0x7f14440b57c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f143400c000 tx=0x7f1434006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:51.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.078+0000 7f143affd700 1 -- 172.21.15.17:0/3218221464 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1434010780 con 0x7f14440b01a0 2022-08-01T19:34:51.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.078+0000 7f143affd700 1 -- 172.21.15.17:0/3218221464 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14340108e0 con 0x7f14440b01a0 2022-08-01T19:34:51.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.078+0000 7f144ab20700 1 -- 172.21.15.17:0/3218221464 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14440b3ec0 con 0x7f14440b01a0 2022-08-01T19:34:51.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.078+0000 7f143affd700 1 -- 172.21.15.17:0/3218221464 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1434010c40 con 0x7f14440b01a0 2022-08-01T19:34:51.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.078+0000 7f144ab20700 1 -- 172.21.15.17:0/3218221464 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14440b3110 con 0x7f14440b01a0 2022-08-01T19:34:51.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.082+0000 7f144ab20700 1 -- 172.21.15.17:0/3218221464 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14440574e0 con 0x7f14440b01a0 2022-08-01T19:34:51.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.082+0000 7f143affd700 1 -- 172.21.15.17:0/3218221464 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f143401c460 con 0x7f14440b01a0 2022-08-01T19:34:51.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.082+0000 7f143affd700 1 --2- 172.21.15.17:0/3218221464 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f142c0539e0 0x7f142c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:34:51.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.082+0000 7f143affd700 1 -- 172.21.15.17:0/3218221464 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f143406c450 con 0x7f14440b01a0 2022-08-01T19:34:51.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.082+0000 7f1449b1e700 1 --2- 172.21.15.17:0/3218221464 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f142c0539e0 0x7f142c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:34:51.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.082+0000 7f1449b1e700 1 --2- 172.21.15.17:0/3218221464 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f142c0539e0 0x7f142c055ea0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f144000d000 tx=0x7f1440009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:34:51.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.090+0000 7f143affd700 1 -- 172.21.15.17:0/3218221464 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f14340431b0 con 0x7f14440b01a0 2022-08-01T19:34:51.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.254+0000 7f144ab20700 1 -- 172.21.15.17:0/3218221464 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1444069a70 con 0x7f14440b01a0 2022-08-01T19:34:51.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.254+0000 7f143affd700 1 -- 172.21.15.17:0/3218221464 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+2491 (secure 0 0 0) 0x7f14340467d0 con 0x7f14440b01a0 2022-08-01T19:34:51.260 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:34:51.260 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2487504686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2487504686},{"type":"v1","addr":"172.21.15.17:6833","nonce":2487504686}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10},{"gid":4885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/4255478434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4255478434},{"type":"v1","addr":"172.21.15.77:6835","nonce":4255478434}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":11}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:34:46.688039+0000","modified":"2022-08-01T19:34:47.690492+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4890},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4890":{"gid":4890,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/3876538769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":3876538769},{"type":"v1","addr":"172.21.15.17:6835","nonce":3876538769}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-08-01T19:34:51.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.254+0000 7f144ab20700 1 -- 172.21.15.17:0/3218221464 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f142c0539e0 msgr2=0x7f142c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:51.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.254+0000 7f144ab20700 1 --2- 172.21.15.17:0/3218221464 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f142c0539e0 0x7f142c055ea0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f144000d000 tx=0x7f1440009040).stop 2022-08-01T19:34:51.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.254+0000 7f144ab20700 1 -- 172.21.15.17:0/3218221464 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14440b01a0 msgr2=0x7f14440b57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:34:51.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.254+0000 7f144ab20700 1 --2- 172.21.15.17:0/3218221464 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14440b01a0 0x7f14440b57c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f143400c000 tx=0x7f1434006df0).stop 2022-08-01T19:34:51.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.258+0000 7f144ab20700 1 -- 172.21.15.17:0/3218221464 shutdown_connections 2022-08-01T19:34:51.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.258+0000 7f144ab20700 1 --2- 172.21.15.17:0/3218221464 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f142c0539e0 0x7f142c055ea0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:51.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.258+0000 7f144ab20700 1 --2- 172.21.15.17:0/3218221464 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f14440b2710 0x7f14440b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:51.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.258+0000 7f144ab20700 1 --2- 172.21.15.17:0/3218221464 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14440b01a0 0x7f14440b57c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:51.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.258+0000 7f144ab20700 1 --2- 172.21.15.17:0/3218221464 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14440afd60 0x7f1444148cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:34:51.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.258+0000 7f144ab20700 1 -- 172.21.15.17:0/3218221464 >> 172.21.15.17:0/3218221464 conn(0x7f1444060310 msgr2=0x7f14440af620 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:34:51.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.258+0000 7f144ab20700 1 -- 172.21.15.17:0/3218221464 shutdown_connections 2022-08-01T19:34:51.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:34:51.258+0000 7f144ab20700 1 -- 172.21.15.17:0/3218221464 wait complete. 2022-08-01T19:34:51.265 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 13 2022-08-01T19:34:51.575 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 13, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:34:46.688039+0000', 'modified': '2022-08-01T19:34:47.690492+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4890}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4890': {'gid': 4890, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/3876538769', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 3876538769}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 3876538769}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:34:51.576 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:34:51.577 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-08-01T19:34:51.697 INFO:teuthology.orchestra.run.smithi017.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":4890,"want_state":"up:active","state":"up:active","rank_uptime":4.9981167299999996,"mdsmap_epoch":13,"osdmap_epoch":35,"osdmap_epoch_barrier":34,"uptime":11.364413897} 2022-08-01T19:34:51.698 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 4890, "mdsmap_epoch": 13, "osdmap_epoch": 35, "osdmap_epoch_barrier": 34, "rank_uptime": 4.99811673, "state": "up:active", "uptime": 11.364413897, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:34:51.698 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T19:34:51.699 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-08-01T19:34:51.699 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi085.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-08-01T19:34:51.700 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:34:51.700 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:34:51.708 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:34:51.769 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:34:51.769 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:34:51.780 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:34:51.781 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:34:51.783 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:34:51.783 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:34:51.832 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T19:34:51.833 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:34:51.836 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:34:51.854 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:34:51.854 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:34:51.909 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:2022-08-01T19:34:51.907+0000 7f1078694080 -1 init, newargv = 0x5580b95667b0 newargc=9ceph-fuse[22481]: starting ceph client 2022-08-01T19:34:51.909 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr: 2022-08-01T19:34:51.916 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:34:51.917 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:34:51.918 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:34:51.919 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:34:51.920 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:ceph-fuse[22481]: starting fuse 2022-08-01T19:34:51.925 INFO:teuthology.orchestra.run.smithi085.stdout:52 2022-08-01T19:34:51.926 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T19:34:51.926 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:34:51.927 DEBUG:teuthology.orchestra.run.smithi085:> sudo python3 -c ' 2022-08-01T19:34:51.927 DEBUG:teuthology.orchestra.run.smithi085:> import glob 2022-08-01T19:34:51.928 DEBUG:teuthology.orchestra.run.smithi085:> import re 2022-08-01T19:34:51.928 DEBUG:teuthology.orchestra.run.smithi085:> import os 2022-08-01T19:34:51.929 DEBUG:teuthology.orchestra.run.smithi085:> import subprocess 2022-08-01T19:34:51.929 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:34:51.930 DEBUG:teuthology.orchestra.run.smithi085:> def find_socket(client_name): 2022-08-01T19:34:51.930 DEBUG:teuthology.orchestra.run.smithi085:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-08-01T19:34:51.931 DEBUG:teuthology.orchestra.run.smithi085:> files = glob.glob(asok_path) 2022-08-01T19:34:51.931 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:34:51.931 DEBUG:teuthology.orchestra.run.smithi085:> # Given a non-glob path, it better be there 2022-08-01T19:34:51.932 DEBUG:teuthology.orchestra.run.smithi085:> if "*" not in asok_path: 2022-08-01T19:34:51.932 DEBUG:teuthology.orchestra.run.smithi085:> assert(len(files) == 1) 2022-08-01T19:34:51.933 DEBUG:teuthology.orchestra.run.smithi085:> return files[0] 2022-08-01T19:34:51.933 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:34:51.934 DEBUG:teuthology.orchestra.run.smithi085:> for f in files: 2022-08-01T19:34:51.934 DEBUG:teuthology.orchestra.run.smithi085:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T19:34:51.934 DEBUG:teuthology.orchestra.run.smithi085:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T19:34:51.935 DEBUG:teuthology.orchestra.run.smithi085:> return f 2022-08-01T19:34:51.935 DEBUG:teuthology.orchestra.run.smithi085:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T19:34:51.936 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:34:51.936 DEBUG:teuthology.orchestra.run.smithi085:> print(find_socket("client.0")) 2022-08-01T19:34:51.937 DEBUG:teuthology.orchestra.run.smithi085:> ' 2022-08-01T19:34:52.014 INFO:teuthology.orchestra.run.smithi085.stdout:/var/run/ceph/ceph-client.0.22481.asok 2022-08-01T19:34:52.018 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22481.asok 2022-08-01T19:34:52.019 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:34:52.019 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22481.asok status 2022-08-01T19:34:52.105 INFO:teuthology.orchestra.run.smithi085.stdout:{ 2022-08-01T19:34:52.105 INFO:teuthology.orchestra.run.smithi085.stdout: "metadata": { 2022-08-01T19:34:52.106 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T19:34:52.106 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T19:34:52.106 INFO:teuthology.orchestra.run.smithi085.stdout: "entity_id": "0", 2022-08-01T19:34:52.106 INFO:teuthology.orchestra.run.smithi085.stdout: "hostname": "smithi085", 2022-08-01T19:34:52.107 INFO:teuthology.orchestra.run.smithi085.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-08-01T19:34:52.107 INFO:teuthology.orchestra.run.smithi085.stdout: "pid": "22481", 2022-08-01T19:34:52.107 INFO:teuthology.orchestra.run.smithi085.stdout: "root": "/" 2022-08-01T19:34:52.107 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:34:52.108 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_count": 0, 2022-08-01T19:34:52.108 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_pinned_count": 0, 2022-08-01T19:34:52.108 INFO:teuthology.orchestra.run.smithi085.stdout: "id": 5071, 2022-08-01T19:34:52.108 INFO:teuthology.orchestra.run.smithi085.stdout: "inst": { 2022-08-01T19:34:52.109 INFO:teuthology.orchestra.run.smithi085.stdout: "name": { 2022-08-01T19:34:52.109 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "client", 2022-08-01T19:34:52.109 INFO:teuthology.orchestra.run.smithi085.stdout: "num": 5071 2022-08-01T19:34:52.109 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:34:52.110 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2022-08-01T19:34:52.110 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2022-08-01T19:34:52.110 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2022-08-01T19:34:52.110 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 2505696545 2022-08-01T19:34:52.110 INFO:teuthology.orchestra.run.smithi085.stdout: } 2022-08-01T19:34:52.111 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:34:52.111 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2022-08-01T19:34:52.111 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2022-08-01T19:34:52.111 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2022-08-01T19:34:52.112 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 2505696545 2022-08-01T19:34:52.112 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:34:52.112 INFO:teuthology.orchestra.run.smithi085.stdout: "inst_str": "client.5071 172.21.15.85:0/2505696545", 2022-08-01T19:34:52.112 INFO:teuthology.orchestra.run.smithi085.stdout: "addr_str": "172.21.15.85:0/2505696545", 2022-08-01T19:34:52.113 INFO:teuthology.orchestra.run.smithi085.stdout: "inode_count": 1, 2022-08-01T19:34:52.113 INFO:teuthology.orchestra.run.smithi085.stdout: "mds_epoch": 13, 2022-08-01T19:34:52.113 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch": 35, 2022-08-01T19:34:52.113 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch_barrier": 0, 2022-08-01T19:34:52.113 INFO:teuthology.orchestra.run.smithi085.stdout: "blacklisted": false, 2022-08-01T19:34:52.114 INFO:teuthology.orchestra.run.smithi085.stdout: "fs_name": "cephfs" 2022-08-01T19:34:52.114 INFO:teuthology.orchestra.run.smithi085.stdout:} 2022-08-01T19:34:52.115 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:34:52.116 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:34:52.116 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:34:52.123 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:34:52.124 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:34:52.125 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:34:52.125 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:34:52.183 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_min_caps_per_client 2022-08-01T19:34:52.291 INFO:teuthology.orchestra.run.smithi017.stdout:{"mds_min_caps_per_client":"100"} 2022-08-01T19:34:52.291 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_min_caps_per_client": "100" } 2022-08-01T19:34:52.292 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:34:52.292 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:34:52.293 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:34:52.301 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:34:52.302 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:34:52.303 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-08-01T19:34:52.304 DEBUG:teuthology.orchestra.run.smithi085:> import sys 2022-08-01T19:34:52.304 DEBUG:teuthology.orchestra.run.smithi085:> import time 2022-08-01T19:34:52.305 DEBUG:teuthology.orchestra.run.smithi085:> import os 2022-08-01T19:34:52.305 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:34:52.305 DEBUG:teuthology.orchestra.run.smithi085:> n = 1000 2022-08-01T19:34:52.306 DEBUG:teuthology.orchestra.run.smithi085:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2022-08-01T19:34:52.306 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:34:52.307 DEBUG:teuthology.orchestra.run.smithi085:> if not os.path.exists(os.path.dirname(abs_path)): 2022-08-01T19:34:52.307 DEBUG:teuthology.orchestra.run.smithi085:> os.makedirs(os.path.dirname(abs_path)) 2022-08-01T19:34:52.308 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:34:52.308 DEBUG:teuthology.orchestra.run.smithi085:> for i in range(0, n): 2022-08-01T19:34:52.308 DEBUG:teuthology.orchestra.run.smithi085:> fname = "{0}_{1}".format(abs_path, i) 2022-08-01T19:34:52.309 DEBUG:teuthology.orchestra.run.smithi085:> with open(fname, '"'"'w'"'"') as f: 2022-08-01T19:34:52.309 DEBUG:teuthology.orchestra.run.smithi085:> f.write('"'"'content'"'"') 2022-08-01T19:34:52.310 DEBUG:teuthology.orchestra.run.smithi085:> if True: 2022-08-01T19:34:52.310 DEBUG:teuthology.orchestra.run.smithi085:> f.flush() 2022-08-01T19:34:52.310 DEBUG:teuthology.orchestra.run.smithi085:> os.fsync(f.fileno()) 2022-08-01T19:34:52.311 DEBUG:teuthology.orchestra.run.smithi085:> ' 2022-08-01T19:35:03.396 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:35:03.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.506+0000 7f8dfca57700 1 Processor -- start 2022-08-01T19:35:03.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.506+0000 7f8dfca57700 1 -- start start 2022-08-01T19:35:03.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.506+0000 7f8dfca57700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f8df80b1f40 0x7f8df80b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:03.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.506+0000 7f8dfca57700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8df80b04e0 con 0x7f8df80a7250 2022-08-01T19:35:03.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.506+0000 7f8dfca57700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8df80b0780 con 0x7f8df8148530 2022-08-01T19:35:03.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.506+0000 7f8dfca57700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8df80a9250 con 0x7f8df80b1f40 2022-08-01T19:35:03.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.506+0000 7f8df6d9d700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f8df80a7250 0x7f8df81448b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58256/0 (socket says 172.21.15.17:58256) 2022-08-01T19:35:03.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.506+0000 7f8df6d9d700 1 -- 172.21.15.17:0/257247096 learned_addr learned my addr 172.21.15.17:0/257247096 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:35:03.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.506+0000 7f8df659c700 1 --2- 172.21.15.17:0/257247096 >> v2:172.21.15.77:3300/0 conn(0x7f8df80b1f40 0x7f8df80b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:03.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.506+0000 7f8df559a700 1 -- 172.21.15.17:0/257247096 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3543217916 0 0) 0x7f8df80b04e0 con 0x7f8df80a7250 2022-08-01T19:35:03.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.506+0000 7f8df559a700 1 -- 172.21.15.17:0/257247096 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ddc003260 con 0x7f8df80a7250 2022-08-01T19:35:03.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.506+0000 7f8df659c700 1 -- 172.21.15.17:0/257247096 >> v1:172.21.15.77:6789/0 conn(0x7f8df8148530 legacy=0x7f8df80b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:03.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.506+0000 7f8df659c700 1 -- 172.21.15.17:0/257247096 >> v1:172.21.15.17:6789/0 conn(0x7f8df80a7250 legacy=0x7f8df81448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:03.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.506+0000 7f8df659c700 1 -- 172.21.15.17:0/257247096 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df80b0a20 con 0x7f8df80b1f40 2022-08-01T19:35:03.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.506+0000 7f8df659c700 1 --2- 172.21.15.17:0/257247096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8df80b1f40 0x7f8df80b1160 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f8df81478c0 tx=0x7f8dec006df0).ready entity=mon.1 client_cookie=72e9e4b3e7a5f626 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:03.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.506+0000 7f8df559a700 1 -- 172.21.15.17:0/257247096 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8dec00d610 con 0x7f8df80b1f40 2022-08-01T19:35:03.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.506+0000 7f8df559a700 1 -- 172.21.15.17:0/257247096 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dec000e90 con 0x7f8df80b1f40 2022-08-01T19:35:03.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.506+0000 7f8df559a700 1 -- 172.21.15.17:0/257247096 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8dec00d960 con 0x7f8df80b1f40 2022-08-01T19:35:03.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.506+0000 7f8dfca57700 1 -- 172.21.15.17:0/257247096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8df80b1f40 msgr2=0x7f8df80b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:03.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.506+0000 7f8dfca57700 1 --2- 172.21.15.17:0/257247096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8df80b1f40 0x7f8df80b1160 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f8df81478c0 tx=0x7f8dec006df0).stop 2022-08-01T19:35:03.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.506+0000 7f8dfca57700 1 -- 172.21.15.17:0/257247096 shutdown_connections 2022-08-01T19:35:03.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.506+0000 7f8dfca57700 1 --2- 172.21.15.17:0/257247096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8df80b1f40 0x7f8df80b1160 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:03.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.506+0000 7f8dfca57700 1 -- 172.21.15.17:0/257247096 >> 172.21.15.17:0/257247096 conn(0x7f8df805f840 msgr2=0x7f8df805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:03.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.506+0000 7f8dfca57700 1 -- 172.21.15.17:0/257247096 shutdown_connections 2022-08-01T19:35:03.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.506+0000 7f8dfca57700 1 -- 172.21.15.17:0/257247096 wait complete. 2022-08-01T19:35:03.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.510+0000 7f8dfca57700 1 Processor -- start 2022-08-01T19:35:03.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.510+0000 7f8dfca57700 1 -- start start 2022-08-01T19:35:03.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.510+0000 7f8dfca57700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8df80a7250 0x7f8df8063bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:03.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.510+0000 7f8dfca57700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8df80b1f40 0x7f8df8153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:03.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.510+0000 7f8dfca57700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8df8148530 0x7f8df8154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:03.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.510+0000 7f8dfca57700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df80abe70 con 0x7f8df80a7250 2022-08-01T19:35:03.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.510+0000 7f8dfca57700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df80b35d0 con 0x7f8df80b1f40 2022-08-01T19:35:03.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.510+0000 7f8dfca57700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df80b3800 con 0x7f8df8148530 2022-08-01T19:35:03.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.510+0000 7f8df659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8df80a7250 0x7f8df8063bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:03.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.510+0000 7f8df659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8df80a7250 0x7f8df8063bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37832/0 (socket says 172.21.15.17:37832) 2022-08-01T19:35:03.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.510+0000 7f8df659c700 1 -- 172.21.15.17:0/742183244 learned_addr learned my addr 172.21.15.17:0/742183244 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:03.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.510+0000 7f8df6d9d700 1 --2- 172.21.15.17:0/742183244 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8df8148530 0x7f8df8154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:03.525 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.510+0000 7f8df5d9b700 1 --2- 172.21.15.17:0/742183244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8df80b1f40 0x7f8df8153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:03.525 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.514+0000 7f8df659c700 1 -- 172.21.15.17:0/742183244 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8df8148530 msgr2=0x7f8df8154490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:03.525 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.514+0000 7f8df659c700 1 --2- 172.21.15.17:0/742183244 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8df8148530 0x7f8df8154490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:03.525 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.514+0000 7f8df659c700 1 -- 172.21.15.17:0/742183244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8df80b1f40 msgr2=0x7f8df8153fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:03.526 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.514+0000 7f8df659c700 1 --2- 172.21.15.17:0/742183244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8df80b1f40 0x7f8df8153fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:03.526 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.514+0000 7f8df659c700 1 -- 172.21.15.17:0/742183244 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dec00b180 con 0x7f8df80a7250 2022-08-01T19:35:03.526 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.514+0000 7f8df5d9b700 1 --2- 172.21.15.17:0/742183244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8df80b1f40 0x7f8df8153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:35:03.526 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.514+0000 7f8df659c700 1 --2- 172.21.15.17:0/742183244 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8df80a7250 0x7f8df8063bb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f8df80a9f90 tx=0x7f8dec014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:03.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.514+0000 7f8de77fe700 1 -- 172.21.15.17:0/742183244 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8dec016070 con 0x7f8df80a7250 2022-08-01T19:35:03.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.514+0000 7f8de77fe700 1 -- 172.21.15.17:0/742183244 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dec00de90 con 0x7f8df80a7250 2022-08-01T19:35:03.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.514+0000 7f8dfca57700 1 -- 172.21.15.17:0/742183244 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df80aa070 con 0x7f8df80a7250 2022-08-01T19:35:03.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.514+0000 7f8de77fe700 1 -- 172.21.15.17:0/742183244 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8dec002f70 con 0x7f8df80a7250 2022-08-01T19:35:03.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.514+0000 7f8dfca57700 1 -- 172.21.15.17:0/742183244 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df80aa9f0 con 0x7f8df80a7250 2022-08-01T19:35:03.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.514+0000 7f8de77fe700 1 -- 172.21.15.17:0/742183244 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8dec025030 con 0x7f8df80a7250 2022-08-01T19:35:03.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.514+0000 7f8dfca57700 1 -- 172.21.15.17:0/742183244 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df80aa200 con 0x7f8df80a7250 2022-08-01T19:35:03.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.514+0000 7f8de77fe700 1 --2- 172.21.15.17:0/742183244 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8dd4053b00 0x7f8dd4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:03.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.514+0000 7f8de77fe700 1 -- 172.21.15.17:0/742183244 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8dec06d090 con 0x7f8df80a7250 2022-08-01T19:35:03.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.514+0000 7f8df5d9b700 1 --2- 172.21.15.17:0/742183244 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8dd4053b00 0x7f8dd4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:03.529 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.514+0000 7f8df5d9b700 1 --2- 172.21.15.17:0/742183244 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8dd4053b00 0x7f8dd4055fc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f8de800b000 tx=0x7f8de8009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:03.530 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.526+0000 7f8de77fe700 1 -- 172.21.15.17:0/742183244 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8dec0441a0 con 0x7f8df80a7250 2022-08-01T19:35:03.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.690+0000 7f8dfca57700 1 -- 172.21.15.17:0/742183244 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8df80b0cc0 con 0x7f8df80a7250 2022-08-01T19:35:03.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.690+0000 7f8de77fe700 1 -- 172.21.15.17:0/742183244 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+2491 (secure 0 0 0) 0x7f8dec023020 con 0x7f8df80a7250 2022-08-01T19:35:03.695 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:35:03.696 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2487504686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2487504686},{"type":"v1","addr":"172.21.15.17:6833","nonce":2487504686}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10},{"gid":4885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/4255478434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4255478434},{"type":"v1","addr":"172.21.15.77:6835","nonce":4255478434}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":11}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:34:46.688039+0000","modified":"2022-08-01T19:34:47.690492+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4890},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4890":{"gid":4890,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/3876538769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":3876538769},{"type":"v1","addr":"172.21.15.17:6835","nonce":3876538769}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-08-01T19:35:03.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.694+0000 7f8dfca57700 1 -- 172.21.15.17:0/742183244 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8dd4053b00 msgr2=0x7f8dd4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:03.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.694+0000 7f8dfca57700 1 --2- 172.21.15.17:0/742183244 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8dd4053b00 0x7f8dd4055fc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f8de800b000 tx=0x7f8de8009040).stop 2022-08-01T19:35:03.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.694+0000 7f8dfca57700 1 -- 172.21.15.17:0/742183244 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8df80a7250 msgr2=0x7f8df8063bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:03.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.694+0000 7f8dfca57700 1 --2- 172.21.15.17:0/742183244 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8df80a7250 0x7f8df8063bb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f8df80a9f90 tx=0x7f8dec014000).stop 2022-08-01T19:35:03.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.694+0000 7f8dfca57700 1 -- 172.21.15.17:0/742183244 shutdown_connections 2022-08-01T19:35:03.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.694+0000 7f8dfca57700 1 --2- 172.21.15.17:0/742183244 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8dd4053b00 0x7f8dd4055fc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:03.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.694+0000 7f8dfca57700 1 --2- 172.21.15.17:0/742183244 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8df8148530 0x7f8df8154490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:03.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.694+0000 7f8dfca57700 1 --2- 172.21.15.17:0/742183244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8df80b1f40 0x7f8df8153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:03.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.694+0000 7f8dfca57700 1 --2- 172.21.15.17:0/742183244 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8df80a7250 0x7f8df8063bb0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:03.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.694+0000 7f8dfca57700 1 -- 172.21.15.17:0/742183244 >> 172.21.15.17:0/742183244 conn(0x7f8df805f840 msgr2=0x7f8df8147900 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:03.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.694+0000 7f8dfca57700 1 -- 172.21.15.17:0/742183244 shutdown_connections 2022-08-01T19:35:03.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.694+0000 7f8dfca57700 1 -- 172.21.15.17:0/742183244 wait complete. 2022-08-01T19:35:03.701 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 13 2022-08-01T19:35:03.713 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_caps 20 2022-08-01T19:35:03.818 INFO:teuthology.orchestra.run.smithi017.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2022-08-01T19:35:03.819 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2022-08-01T19:35:03.820 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:35:03.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.914+0000 7fb261174700 1 Processor -- start 2022-08-01T19:35:03.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.914+0000 7fb261174700 1 -- start start 2022-08-01T19:35:03.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.914+0000 7fb261174700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fb25c0b2710 0x7fb25c1401e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:03.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.914+0000 7fb261174700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb25c0ac880 0x7fb25c1406d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:03.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.914+0000 7fb261174700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb25c067a10 con 0x7fb25c0b0270 2022-08-01T19:35:03.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.914+0000 7fb261174700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb25c0ad740 con 0x7fb25c0ac880 2022-08-01T19:35:03.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.914+0000 7fb261174700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb25c067cb0 con 0x7fb25c0b2710 2022-08-01T19:35:03.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.914+0000 7fb260973700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb25c0ac880 0x7fb25c1406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:03.924 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.914+0000 7fb260973700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb25c0ac880 0x7fb25c1406d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37844/0 (socket says 172.21.15.17:37844) 2022-08-01T19:35:03.924 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.918+0000 7fb260973700 1 -- 172.21.15.17:0/3327296514 learned_addr learned my addr 172.21.15.17:0/3327296514 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:03.924 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.918+0000 7fb25bfff700 1 --2- 172.21.15.17:0/3327296514 >> v2:172.21.15.77:3300/0 conn(0x7fb25c0b2710 0x7fb25c1401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:03.924 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.918+0000 7fb260973700 1 -- 172.21.15.17:0/3327296514 >> v2:172.21.15.77:3300/0 conn(0x7fb25c0b2710 msgr2=0x7fb25c1401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:03.925 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.918+0000 7fb260973700 1 --2- 172.21.15.17:0/3327296514 >> v2:172.21.15.77:3300/0 conn(0x7fb25c0b2710 0x7fb25c1401e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:03.925 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.918+0000 7fb260973700 1 -- 172.21.15.17:0/3327296514 >> v1:172.21.15.77:6789/0 conn(0x7fb25c0b0270 legacy=0x7fb25c143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:03.925 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.918+0000 7fb260973700 1 -- 172.21.15.17:0/3327296514 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb25c147160 con 0x7fb25c0ac880 2022-08-01T19:35:03.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.918+0000 7fb260973700 1 --2- 172.21.15.17:0/3327296514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb25c0ac880 0x7fb25c1406d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7fb2540032f0 tx=0x7fb254009290).ready entity=mon.0 client_cookie=c50c5321c41d86cb server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:03.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.918+0000 7fb25affd700 1 -- 172.21.15.17:0/3327296514 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb25400b870 con 0x7fb25c0ac880 2022-08-01T19:35:03.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.918+0000 7fb25affd700 1 -- 172.21.15.17:0/3327296514 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb25400b9d0 con 0x7fb25c0ac880 2022-08-01T19:35:03.929 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.918+0000 7fb25affd700 1 -- 172.21.15.17:0/3327296514 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb25400bb40 con 0x7fb25c0ac880 2022-08-01T19:35:03.929 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.918+0000 7fb261174700 1 -- 172.21.15.17:0/3327296514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb25c0ac880 msgr2=0x7fb25c1406d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:03.929 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.918+0000 7fb261174700 1 --2- 172.21.15.17:0/3327296514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb25c0ac880 0x7fb25c1406d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7fb2540032f0 tx=0x7fb254009290).stop 2022-08-01T19:35:03.929 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.918+0000 7fb261174700 1 -- 172.21.15.17:0/3327296514 shutdown_connections 2022-08-01T19:35:03.930 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.918+0000 7fb261174700 1 --2- 172.21.15.17:0/3327296514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb25c0ac880 0x7fb25c1406d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:03.930 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.918+0000 7fb261174700 1 --2- 172.21.15.17:0/3327296514 >> v2:172.21.15.77:3300/0 conn(0x7fb25c0b2710 0x7fb25c1401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:03.930 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.918+0000 7fb261174700 1 -- 172.21.15.17:0/3327296514 >> 172.21.15.17:0/3327296514 conn(0x7fb25c060310 msgr2=0x7fb25c0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:03.930 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.918+0000 7fb261174700 1 -- 172.21.15.17:0/3327296514 shutdown_connections 2022-08-01T19:35:03.931 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.918+0000 7fb261174700 1 -- 172.21.15.17:0/3327296514 wait complete. 2022-08-01T19:35:03.931 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.918+0000 7fb261174700 1 Processor -- start 2022-08-01T19:35:03.931 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.918+0000 7fb261174700 1 -- start start 2022-08-01T19:35:03.931 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.922+0000 7fb261174700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb25c0ac880 0x7fb25c146560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:03.931 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.922+0000 7fb261174700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb25c0b0270 0x7fb25c0ab130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:03.932 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.922+0000 7fb261174700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb25c0b2710 0x7fb25c0ab620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:03.932 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.922+0000 7fb261174700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c152530 con 0x7fb25c0ac880 2022-08-01T19:35:03.932 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.922+0000 7fb261174700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c0a0ec0 con 0x7fb25c0b2710 2022-08-01T19:35:03.932 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.922+0000 7fb261174700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c0a1160 con 0x7fb25c0b0270 2022-08-01T19:35:03.933 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.922+0000 7fb25bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb25c0ac880 0x7fb25c146560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:03.933 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.922+0000 7fb25bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb25c0ac880 0x7fb25c146560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37846/0 (socket says 172.21.15.17:37846) 2022-08-01T19:35:03.933 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.922+0000 7fb25bfff700 1 -- 172.21.15.17:0/2025780731 learned_addr learned my addr 172.21.15.17:0/2025780731 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:03.933 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.922+0000 7fb25b7fe700 1 --2- 172.21.15.17:0/2025780731 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb25c0b0270 0x7fb25c0ab130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:03.933 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.922+0000 7fb260973700 1 --2- 172.21.15.17:0/2025780731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb25c0b2710 0x7fb25c0ab620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:03.934 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.922+0000 7fb25bfff700 1 -- 172.21.15.17:0/2025780731 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb25c0b0270 msgr2=0x7fb25c0ab130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:03.934 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.922+0000 7fb25bfff700 1 --2- 172.21.15.17:0/2025780731 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb25c0b0270 0x7fb25c0ab130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:03.934 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.922+0000 7fb25bfff700 1 -- 172.21.15.17:0/2025780731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb25c0b2710 msgr2=0x7fb25c0ab620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:03.934 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.922+0000 7fb25bfff700 1 --2- 172.21.15.17:0/2025780731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb25c0b2710 0x7fb25c0ab620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:03.935 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.922+0000 7fb25bfff700 1 -- 172.21.15.17:0/2025780731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb254009530 con 0x7fb25c0ac880 2022-08-01T19:35:03.935 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.922+0000 7fb25bfff700 1 --2- 172.21.15.17:0/2025780731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb25c0ac880 0x7fb25c146560 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7fb2500030b0 tx=0x7fb250008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:03.937 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.926+0000 7fb2597fa700 1 -- 172.21.15.17:0/2025780731 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb250011780 con 0x7fb25c0ac880 2022-08-01T19:35:03.937 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.926+0000 7fb2597fa700 1 -- 172.21.15.17:0/2025780731 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2500118e0 con 0x7fb25c0ac880 2022-08-01T19:35:03.937 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.926+0000 7fb261174700 1 -- 172.21.15.17:0/2025780731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb25c0a1400 con 0x7fb25c0ac880 2022-08-01T19:35:03.938 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.926+0000 7fb2597fa700 1 -- 172.21.15.17:0/2025780731 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb250011c40 con 0x7fb25c0ac880 2022-08-01T19:35:03.938 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.926+0000 7fb261174700 1 -- 172.21.15.17:0/2025780731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb25c1500e0 con 0x7fb25c0ac880 2022-08-01T19:35:03.938 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.926+0000 7fb261174700 1 -- 172.21.15.17:0/2025780731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb25c0ac570 con 0x7fb25c0ac880 2022-08-01T19:35:03.938 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.926+0000 7fb2597fa700 1 -- 172.21.15.17:0/2025780731 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb25001c460 con 0x7fb25c0ac880 2022-08-01T19:35:03.939 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.926+0000 7fb2597fa700 1 --2- 172.21.15.17:0/2025780731 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb23c053ac0 0x7fb23c055f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:03.939 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.926+0000 7fb2597fa700 1 -- 172.21.15.17:0/2025780731 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb25006c260 con 0x7fb25c0ac880 2022-08-01T19:35:03.939 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.926+0000 7fb25b7fe700 1 --2- 172.21.15.17:0/2025780731 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb23c053ac0 0x7fb23c055f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:03.939 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.926+0000 7fb25b7fe700 1 --2- 172.21.15.17:0/2025780731 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb23c053ac0 0x7fb23c055f80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fb24c00b000 tx=0x7fb24c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:03.940 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:03.930+0000 7fb2597fa700 1 -- 172.21.15.17:0/2025780731 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb250042fc0 con 0x7fb25c0ac880 2022-08-01T19:35:04.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.090+0000 7fb261174700 1 -- 172.21.15.17:0/2025780731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb25c0574e0 con 0x7fb25c0ac880 2022-08-01T19:35:04.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.090+0000 7fb2597fa700 1 -- 172.21.15.17:0/2025780731 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+2491 (secure 0 0 0) 0x7fb2500465e0 con 0x7fb25c0ac880 2022-08-01T19:35:04.096 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:35:04.096 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2487504686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2487504686},{"type":"v1","addr":"172.21.15.17:6833","nonce":2487504686}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10},{"gid":4885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/4255478434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4255478434},{"type":"v1","addr":"172.21.15.77:6835","nonce":4255478434}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":11}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:34:46.688039+0000","modified":"2022-08-01T19:34:47.690492+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4890},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4890":{"gid":4890,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/3876538769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":3876538769},{"type":"v1","addr":"172.21.15.17:6835","nonce":3876538769}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-08-01T19:35:04.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.094+0000 7fb261174700 1 -- 172.21.15.17:0/2025780731 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb23c053ac0 msgr2=0x7fb23c055f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:04.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.094+0000 7fb261174700 1 --2- 172.21.15.17:0/2025780731 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb23c053ac0 0x7fb23c055f80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fb24c00b000 tx=0x7fb24c009040).stop 2022-08-01T19:35:04.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.094+0000 7fb261174700 1 -- 172.21.15.17:0/2025780731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb25c0ac880 msgr2=0x7fb25c146560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:04.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.094+0000 7fb261174700 1 --2- 172.21.15.17:0/2025780731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb25c0ac880 0x7fb25c146560 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7fb2500030b0 tx=0x7fb250008040).stop 2022-08-01T19:35:04.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.094+0000 7fb261174700 1 -- 172.21.15.17:0/2025780731 shutdown_connections 2022-08-01T19:35:04.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.094+0000 7fb261174700 1 --2- 172.21.15.17:0/2025780731 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb23c053ac0 0x7fb23c055f80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.094+0000 7fb261174700 1 --2- 172.21.15.17:0/2025780731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb25c0b2710 0x7fb25c0ab620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.094+0000 7fb261174700 1 --2- 172.21.15.17:0/2025780731 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb25c0b0270 0x7fb25c0ab130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.094+0000 7fb261174700 1 --2- 172.21.15.17:0/2025780731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb25c0ac880 0x7fb25c146560 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.094+0000 7fb261174700 1 -- 172.21.15.17:0/2025780731 >> 172.21.15.17:0/2025780731 conn(0x7fb25c060310 msgr2=0x7fb25c140460 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:04.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.094+0000 7fb261174700 1 -- 172.21.15.17:0/2025780731 shutdown_connections 2022-08-01T19:35:04.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.094+0000 7fb261174700 1 -- 172.21.15.17:0/2025780731 wait complete. 2022-08-01T19:35:04.105 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 13 2022-08-01T19:35:04.109 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_decay_threshold 400 2022-08-01T19:35:04.216 INFO:teuthology.orchestra.run.smithi017.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2022-08-01T19:35:04.216 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2022-08-01T19:35:04.217 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:35:04.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.314+0000 7fd78e44a700 1 Processor -- start 2022-08-01T19:35:04.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.314+0000 7fd78e44a700 1 -- start start 2022-08-01T19:35:04.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.314+0000 7fd78e44a700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd7880a8c50 0x7fd788152d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:04.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.314+0000 7fd78e44a700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7880a7250 con 0x7fd7880b2710 2022-08-01T19:35:04.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.314+0000 7fd78e44a700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7880a7480 con 0x7fd7880b01a0 2022-08-01T19:35:04.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.314+0000 7fd78e44a700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7880a7720 con 0x7fd7880a8c50 2022-08-01T19:35:04.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.314+0000 7fd78d448700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd7880b2710 0x7fd78814afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58280/0 (socket says 172.21.15.17:58280) 2022-08-01T19:35:04.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.314+0000 7fd78d448700 1 -- 172.21.15.17:0/4205040937 learned_addr learned my addr 172.21.15.17:0/4205040937 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:35:04.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.314+0000 7fd78dc49700 1 --2- 172.21.15.17:0/4205040937 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd7880a8c50 0x7fd788152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:04.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.314+0000 7fd77ffff700 1 -- 172.21.15.17:0/4205040937 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1891361395 0 0) 0x7fd7880a7250 con 0x7fd7880b2710 2022-08-01T19:35:04.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.314+0000 7fd77ffff700 1 -- 172.21.15.17:0/4205040937 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd774003260 con 0x7fd7880b2710 2022-08-01T19:35:04.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.314+0000 7fd77ffff700 1 -- 172.21.15.17:0/4205040937 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3044929636 0 0) 0x7fd7880a7480 con 0x7fd7880b01a0 2022-08-01T19:35:04.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.314+0000 7fd77ffff700 1 -- 172.21.15.17:0/4205040937 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7880a7250 con 0x7fd7880b01a0 2022-08-01T19:35:04.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.314+0000 7fd77ffff700 1 -- 172.21.15.17:0/4205040937 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3124560288 0 0) 0x7fd774003260 con 0x7fd7880b2710 2022-08-01T19:35:04.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.314+0000 7fd77ffff700 1 -- 172.21.15.17:0/4205040937 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd7880a8c50 msgr2=0x7fd788152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:04.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.314+0000 7fd77ffff700 1 --2- 172.21.15.17:0/4205040937 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd7880a8c50 0x7fd788152d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.314+0000 7fd77ffff700 1 -- 172.21.15.17:0/4205040937 >> v1:172.21.15.77:6789/0 conn(0x7fd7880b01a0 legacy=0x7fd7880a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:04.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.314+0000 7fd77ffff700 1 -- 172.21.15.17:0/4205040937 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7880a79c0 con 0x7fd7880b2710 2022-08-01T19:35:04.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.314+0000 7fd77ffff700 1 -- 172.21.15.17:0/4205040937 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7fd784001660 con 0x7fd7880b2710 2022-08-01T19:35:04.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.314+0000 7fd77ffff700 1 -- 172.21.15.17:0/4205040937 >> v1:172.21.15.17:6789/0 conn(0x7fd7880b2710 legacy=0x7fd78814afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:04.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.314+0000 7fd77ffff700 1 --2- 172.21.15.17:0/4205040937 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd7880a8c50 0x7fd774003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:04.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.314+0000 7fd77ffff700 1 --2- 172.21.15.17:0/4205040937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd774003e20 0x7fd774004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:04.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.314+0000 7fd77ffff700 1 --2- 172.21.15.17:0/4205040937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7740049f0 0x7fd774004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:04.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.314+0000 7fd77ffff700 1 -- 172.21.15.17:0/4205040937 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7880a7720 con 0x7fd7740049f0 2022-08-01T19:35:04.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.314+0000 7fd77ffff700 1 -- 172.21.15.17:0/4205040937 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77400b800 con 0x7fd774003e20 2022-08-01T19:35:04.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.314+0000 7fd77ffff700 1 -- 172.21.15.17:0/4205040937 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd77400b9c0 con 0x7fd7880a8c50 2022-08-01T19:35:04.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.314+0000 7fd78d448700 1 --2- 172.21.15.17:0/4205040937 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd7880a8c50 0x7fd774003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:04.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.314+0000 7fd78cc47700 1 --2- 172.21.15.17:0/4205040937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd774003e20 0x7fd774004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:04.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.314+0000 7fd78dc49700 1 --2- 172.21.15.17:0/4205040937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7740049f0 0x7fd774004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:04.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.314+0000 7fd78cc47700 1 -- 172.21.15.17:0/4205040937 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd7880a8c50 msgr2=0x7fd774003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:04.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.314+0000 7fd78cc47700 1 --2- 172.21.15.17:0/4205040937 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd7880a8c50 0x7fd774003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.314+0000 7fd78cc47700 1 -- 172.21.15.17:0/4205040937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7740049f0 msgr2=0x7fd774004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:04.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.314+0000 7fd78cc47700 1 --2- 172.21.15.17:0/4205040937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7740049f0 0x7fd774004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.314+0000 7fd78cc47700 1 -- 172.21.15.17:0/4205040937 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd774003260 con 0x7fd774003e20 2022-08-01T19:35:04.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.314+0000 7fd78cc47700 1 --2- 172.21.15.17:0/4205040937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd774003e20 0x7fd774004500 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fd780001490 tx=0x7fd780007d80).ready entity=mon.1 client_cookie=99c12379c6fe1886 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:04.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.318+0000 7fd77ffff700 1 -- 172.21.15.17:0/4205040937 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd780001780 con 0x7fd774003e20 2022-08-01T19:35:04.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.318+0000 7fd77ffff700 1 -- 172.21.15.17:0/4205040937 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd780006890 con 0x7fd774003e20 2022-08-01T19:35:04.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.318+0000 7fd77ffff700 1 -- 172.21.15.17:0/4205040937 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd780006bf0 con 0x7fd774003e20 2022-08-01T19:35:04.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.318+0000 7fd78e44a700 1 -- 172.21.15.17:0/4205040937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd774003e20 msgr2=0x7fd774004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:04.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.318+0000 7fd78e44a700 1 --2- 172.21.15.17:0/4205040937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd774003e20 0x7fd774004500 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fd780001490 tx=0x7fd780007d80).stop 2022-08-01T19:35:04.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.318+0000 7fd78dc49700 1 -- 172.21.15.17:0/4205040937 reap_dead start 2022-08-01T19:35:04.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.318+0000 7fd78e44a700 1 -- 172.21.15.17:0/4205040937 shutdown_connections 2022-08-01T19:35:04.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.318+0000 7fd78e44a700 1 -- 172.21.15.17:0/4205040937 >> 172.21.15.17:0/4205040937 conn(0x7fd788060310 msgr2=0x7fd7880b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:04.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.318+0000 7fd78e44a700 1 -- 172.21.15.17:0/4205040937 shutdown_connections 2022-08-01T19:35:04.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.318+0000 7fd78e44a700 1 -- 172.21.15.17:0/4205040937 wait complete. 2022-08-01T19:35:04.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.318+0000 7fd78e44a700 1 Processor -- start 2022-08-01T19:35:04.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.318+0000 7fd78e44a700 1 -- start start 2022-08-01T19:35:04.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.318+0000 7fd78e44a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7880b5440 0x7fd788156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:04.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.318+0000 7fd78e44a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7880b5cc0 0x7fd788157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:04.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.318+0000 7fd78e44a700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd788159ba0 0x7fd78815a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:04.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.318+0000 7fd78e44a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd788157560 con 0x7fd7880b5440 2022-08-01T19:35:04.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.318+0000 7fd78e44a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd788067310 con 0x7fd7880b5cc0 2022-08-01T19:35:04.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.318+0000 7fd78e44a700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd788066b80 con 0x7fd788159ba0 2022-08-01T19:35:04.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.318+0000 7fd78d448700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7880b5440 0x7fd788156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:04.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.318+0000 7fd78d448700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7880b5440 0x7fd788156b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37868/0 (socket says 172.21.15.17:37868) 2022-08-01T19:35:04.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.318+0000 7fd78d448700 1 -- 172.21.15.17:0/170726168 learned_addr learned my addr 172.21.15.17:0/170726168 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:04.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.318+0000 7fd78cc47700 1 --2- 172.21.15.17:0/170726168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7880b5cc0 0x7fd788157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:04.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.322+0000 7fd78dc49700 1 --2- 172.21.15.17:0/170726168 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd788159ba0 0x7fd78815a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:04.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.322+0000 7fd78cc47700 1 -- 172.21.15.17:0/170726168 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd788159ba0 msgr2=0x7fd78815a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:04.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.322+0000 7fd78cc47700 1 --2- 172.21.15.17:0/170726168 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd788159ba0 0x7fd78815a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.322+0000 7fd78cc47700 1 -- 172.21.15.17:0/170726168 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7880b5440 msgr2=0x7fd788156b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:04.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.322+0000 7fd78cc47700 1 --2- 172.21.15.17:0/170726168 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7880b5440 0x7fd788156b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.322+0000 7fd78cc47700 1 -- 172.21.15.17:0/170726168 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7880b5820 con 0x7fd7880b5cc0 2022-08-01T19:35:04.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.322+0000 7fd78d448700 1 --2- 172.21.15.17:0/170726168 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7880b5440 0x7fd788156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:35:04.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.322+0000 7fd78cc47700 1 --2- 172.21.15.17:0/170726168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7880b5cc0 0x7fd788157070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fd788156600 tx=0x7fd780016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:04.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.322+0000 7fd77e7fc700 1 -- 172.21.15.17:0/170726168 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd780001780 con 0x7fd7880b5cc0 2022-08-01T19:35:04.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.322+0000 7fd78e44a700 1 -- 172.21.15.17:0/170726168 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd788066dd0 con 0x7fd7880b5cc0 2022-08-01T19:35:04.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.322+0000 7fd77e7fc700 1 -- 172.21.15.17:0/170726168 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd780013d40 con 0x7fd7880b5cc0 2022-08-01T19:35:04.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.322+0000 7fd77e7fc700 1 -- 172.21.15.17:0/170726168 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd78001e5e0 con 0x7fd7880b5cc0 2022-08-01T19:35:04.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.322+0000 7fd78e44a700 1 -- 172.21.15.17:0/170726168 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7881566e0 con 0x7fd7880b5cc0 2022-08-01T19:35:04.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.322+0000 7fd78e44a700 1 -- 172.21.15.17:0/170726168 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7880574e0 con 0x7fd7880b5cc0 2022-08-01T19:35:04.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.322+0000 7fd77e7fc700 1 -- 172.21.15.17:0/170726168 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd78001b070 con 0x7fd7880b5cc0 2022-08-01T19:35:04.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.326+0000 7fd77e7fc700 1 --2- 172.21.15.17:0/170726168 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd768053a70 0x7fd768055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:04.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.326+0000 7fd77e7fc700 1 -- 172.21.15.17:0/170726168 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd78006d170 con 0x7fd7880b5cc0 2022-08-01T19:35:04.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.326+0000 7fd78d448700 1 --2- 172.21.15.17:0/170726168 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd768053a70 0x7fd768055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:04.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.326+0000 7fd78d448700 1 --2- 172.21.15.17:0/170726168 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd768053a70 0x7fd768055f30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7fd784005000 tx=0x7fd784009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:04.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.334+0000 7fd77e7fc700 1 -- 172.21.15.17:0/170726168 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd780043de0 con 0x7fd7880b5cc0 2022-08-01T19:35:04.501 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.494+0000 7fd78e44a700 1 -- 172.21.15.17:0/170726168 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd788069a70 con 0x7fd7880b5cc0 2022-08-01T19:35:04.502 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.498+0000 7fd77e7fc700 1 -- 172.21.15.17:0/170726168 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+2491 (secure 0 0 0) 0x7fd7800239d0 con 0x7fd7880b5cc0 2022-08-01T19:35:04.502 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:35:04.503 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2487504686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2487504686},{"type":"v1","addr":"172.21.15.17:6833","nonce":2487504686}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10},{"gid":4885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/4255478434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4255478434},{"type":"v1","addr":"172.21.15.77:6835","nonce":4255478434}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":11}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:34:46.688039+0000","modified":"2022-08-01T19:34:47.690492+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4890},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4890":{"gid":4890,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/3876538769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":3876538769},{"type":"v1","addr":"172.21.15.17:6835","nonce":3876538769}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-08-01T19:35:04.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.498+0000 7fd78e44a700 1 -- 172.21.15.17:0/170726168 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd768053a70 msgr2=0x7fd768055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:04.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.498+0000 7fd78e44a700 1 --2- 172.21.15.17:0/170726168 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd768053a70 0x7fd768055f30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7fd784005000 tx=0x7fd784009040).stop 2022-08-01T19:35:04.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.498+0000 7fd78e44a700 1 -- 172.21.15.17:0/170726168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7880b5cc0 msgr2=0x7fd788157070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:04.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.498+0000 7fd78e44a700 1 --2- 172.21.15.17:0/170726168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7880b5cc0 0x7fd788157070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fd788156600 tx=0x7fd780016000).stop 2022-08-01T19:35:04.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.498+0000 7fd78e44a700 1 -- 172.21.15.17:0/170726168 shutdown_connections 2022-08-01T19:35:04.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.498+0000 7fd78e44a700 1 --2- 172.21.15.17:0/170726168 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd768053a70 0x7fd768055f30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.498+0000 7fd78e44a700 1 --2- 172.21.15.17:0/170726168 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd788159ba0 0x7fd78815a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.498+0000 7fd78e44a700 1 --2- 172.21.15.17:0/170726168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7880b5cc0 0x7fd788157070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.498+0000 7fd78e44a700 1 --2- 172.21.15.17:0/170726168 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7880b5440 0x7fd788156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.498+0000 7fd78e44a700 1 -- 172.21.15.17:0/170726168 >> 172.21.15.17:0/170726168 conn(0x7fd788060310 msgr2=0x7fd788067540 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:04.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.498+0000 7fd78e44a700 1 -- 172.21.15.17:0/170726168 shutdown_connections 2022-08-01T19:35:04.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.498+0000 7fd78e44a700 1 -- 172.21.15.17:0/170726168 wait complete. 2022-08-01T19:35:04.508 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 13 2022-08-01T19:35:04.516 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a cache drop 2022-08-01T19:35:04.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.618+0000 7fe487444700 1 Processor -- start 2022-08-01T19:35:04.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.618+0000 7fe487444700 1 -- start start 2022-08-01T19:35:04.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.618+0000 7fe487444700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe4800614b0 0x7fe480061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:04.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.618+0000 7fe487444700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe4800ad750 0x7fe4800b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:04.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.618+0000 7fe487444700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4800aee80 con 0x7fe4801405d0 2022-08-01T19:35:04.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.618+0000 7fe487444700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4800b70b0 con 0x7fe4800ad750 2022-08-01T19:35:04.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.618+0000 7fe487444700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4800af0b0 con 0x7fe4800614b0 2022-08-01T19:35:04.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.618+0000 7fe485c41700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe4800ad750 0x7fe4800b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:04.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.618+0000 7fe486442700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe4800614b0 0x7fe480061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:04.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.618+0000 7fe485c41700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe4800ad750 0x7fe4800b9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37876/0 (socket says 172.21.15.17:37876) 2022-08-01T19:35:04.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.618+0000 7fe486c43700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7fe4801405d0 0x7fe4801466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:46872/0 (socket says 172.21.15.17:46872) 2022-08-01T19:35:04.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.618+0000 7fe485c41700 1 -- 172.21.15.17:0/3226962535 learned_addr learned my addr 172.21.15.17:0/3226962535 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:04.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.618+0000 7fe485440700 1 -- 172.21.15.17:0/3226962535 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3731032513 0 0) 0x7fe4800aee80 con 0x7fe4801405d0 2022-08-01T19:35:04.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.618+0000 7fe485440700 1 -- 172.21.15.17:0/3226962535 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe468003260 con 0x7fe4801405d0 2022-08-01T19:35:04.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.618+0000 7fe486442700 1 -- 172.21.15.17:0/3226962535 >> v2:172.21.15.17:3300/0 conn(0x7fe4800ad750 msgr2=0x7fe4800b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:04.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.618+0000 7fe486442700 1 --2- 172.21.15.17:0/3226962535 >> v2:172.21.15.17:3300/0 conn(0x7fe4800ad750 0x7fe4800b9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.618+0000 7fe486442700 1 -- 172.21.15.17:0/3226962535 >> v1:172.21.15.77:6789/0 conn(0x7fe4801405d0 legacy=0x7fe4801466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:04.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.618+0000 7fe486442700 1 -- 172.21.15.17:0/3226962535 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4800af350 con 0x7fe4800614b0 2022-08-01T19:35:04.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.618+0000 7fe485c41700 1 --2- 172.21.15.17:0/3226962535 >> v2:172.21.15.17:3300/0 conn(0x7fe4800ad750 0x7fe4800b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:35:04.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.618+0000 7fe486442700 1 --2- 172.21.15.17:0/3226962535 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe4800614b0 0x7fe480061890 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fe47c002f70 tx=0x7fe47c006df0).ready entity=mon.2 client_cookie=9c33b3fb9c0e669d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:04.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.618+0000 7fe485440700 1 -- 172.21.15.17:0/3226962535 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe47c00d4a0 con 0x7fe4800614b0 2022-08-01T19:35:04.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.618+0000 7fe485440700 1 -- 172.21.15.17:0/3226962535 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe47c000e90 con 0x7fe4800614b0 2022-08-01T19:35:04.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.618+0000 7fe485440700 1 -- 172.21.15.17:0/3226962535 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe47c00d7f0 con 0x7fe4800614b0 2022-08-01T19:35:04.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.622+0000 7fe487444700 1 -- 172.21.15.17:0/3226962535 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe4800614b0 msgr2=0x7fe480061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:04.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.622+0000 7fe487444700 1 --2- 172.21.15.17:0/3226962535 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe4800614b0 0x7fe480061890 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fe47c002f70 tx=0x7fe47c006df0).stop 2022-08-01T19:35:04.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.622+0000 7fe487444700 1 -- 172.21.15.17:0/3226962535 shutdown_connections 2022-08-01T19:35:04.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.622+0000 7fe487444700 1 --2- 172.21.15.17:0/3226962535 >> v2:172.21.15.17:3300/0 conn(0x7fe4800ad750 0x7fe4800b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.622+0000 7fe487444700 1 --2- 172.21.15.17:0/3226962535 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe4800614b0 0x7fe480061890 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.622+0000 7fe487444700 1 -- 172.21.15.17:0/3226962535 >> 172.21.15.17:0/3226962535 conn(0x7fe480066f70 msgr2=0x7fe480067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:04.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.622+0000 7fe487444700 1 -- 172.21.15.17:0/3226962535 shutdown_connections 2022-08-01T19:35:04.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.622+0000 7fe487444700 1 -- 172.21.15.17:0/3226962535 wait complete. 2022-08-01T19:35:04.636 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.622+0000 7fe487444700 1 Processor -- start 2022-08-01T19:35:04.636 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.622+0000 7fe487444700 1 -- start start 2022-08-01T19:35:04.645 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.622+0000 7fe487444700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4800614b0 0x7fe4801483b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:04.646 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.622+0000 7fe487444700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe4800ad750 0x7fe480149500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:04.646 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.622+0000 7fe487444700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4801405d0 0x7fe4800aca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:04.646 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.622+0000 7fe487444700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4800b4110 con 0x7fe4800614b0 2022-08-01T19:35:04.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.622+0000 7fe487444700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4800acf20 con 0x7fe4801405d0 2022-08-01T19:35:04.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.622+0000 7fe487444700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4800b1570 con 0x7fe4800ad750 2022-08-01T19:35:04.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.622+0000 7fe486442700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4800614b0 0x7fe4801483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:04.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.622+0000 7fe486442700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4800614b0 0x7fe4801483b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37880/0 (socket says 172.21.15.17:37880) 2022-08-01T19:35:04.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.622+0000 7fe486442700 1 -- 172.21.15.17:0/888892808 learned_addr learned my addr 172.21.15.17:0/888892808 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:04.648 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.626+0000 7fe486c43700 1 --2- 172.21.15.17:0/888892808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4801405d0 0x7fe4800aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:04.648 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.626+0000 7fe485c41700 1 --2- 172.21.15.17:0/888892808 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe4800ad750 0x7fe480149500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:04.648 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.626+0000 7fe486442700 1 -- 172.21.15.17:0/888892808 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe4800ad750 msgr2=0x7fe480149500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:04.648 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.626+0000 7fe486442700 1 --2- 172.21.15.17:0/888892808 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe4800ad750 0x7fe480149500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.649 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.626+0000 7fe486442700 1 -- 172.21.15.17:0/888892808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4801405d0 msgr2=0x7fe4800aca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:04.649 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.626+0000 7fe486442700 1 --2- 172.21.15.17:0/888892808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4801405d0 0x7fe4800aca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.649 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.626+0000 7fe486442700 1 -- 172.21.15.17:0/888892808 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe47000b1e0 con 0x7fe4800614b0 2022-08-01T19:35:04.649 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.626+0000 7fe486442700 1 --2- 172.21.15.17:0/888892808 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4800614b0 0x7fe4801483b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fe47c002f70 tx=0x7fe47c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:04.650 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.626+0000 7fe46f7fe700 1 -- 172.21.15.17:0/888892808 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe47c0139c0 con 0x7fe4800614b0 2022-08-01T19:35:04.650 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.626+0000 7fe46f7fe700 1 -- 172.21.15.17:0/888892808 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe47c013b20 con 0x7fe4800614b0 2022-08-01T19:35:04.650 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.626+0000 7fe487444700 1 -- 172.21.15.17:0/888892808 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe47c00b180 con 0x7fe4800614b0 2022-08-01T19:35:04.650 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.626+0000 7fe46f7fe700 1 -- 172.21.15.17:0/888892808 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe47c013e80 con 0x7fe4800614b0 2022-08-01T19:35:04.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.626+0000 7fe487444700 1 -- 172.21.15.17:0/888892808 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe480148dc0 con 0x7fe4800614b0 2022-08-01T19:35:04.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.626+0000 7fe46f7fe700 1 -- 172.21.15.17:0/888892808 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe47c01c070 con 0x7fe4800614b0 2022-08-01T19:35:04.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.626+0000 7fe46f7fe700 1 --2- 172.21.15.17:0/888892808 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe460053a70 0x7fe460055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:04.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.626+0000 7fe46f7fe700 1 -- 172.21.15.17:0/888892808 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe47c06cae0 con 0x7fe4800614b0 2022-08-01T19:35:04.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.626+0000 7fe485c41700 1 --2- 172.21.15.17:0/888892808 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe460053a70 0x7fe460055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:04.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.630+0000 7fe485c41700 1 --2- 172.21.15.17:0/888892808 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe460053a70 0x7fe460055f30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fe47000b000 tx=0x7fe470006eb0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:04.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.630+0000 7fe4928fd740 1 Processor -- start 2022-08-01T19:35:04.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.630+0000 7fe4928fd740 1 -- start start 2022-08-01T19:35:04.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.630+0000 7fe4928fd740 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1e32e00 0x1e31e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:04.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.630+0000 7fe4928fd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1e331e0 0x1e32380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:04.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.630+0000 7fe4928fd740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1e65920 0x1e65dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:04.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.630+0000 7fe4928fd740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d90320 con 0x1e65920 2022-08-01T19:35:04.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.630+0000 7fe4928fd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x1da3c50 con 0x1e331e0 2022-08-01T19:35:04.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.630+0000 7fe4928fd740 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x1d841f0 con 0x1e32e00 2022-08-01T19:35:04.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.630+0000 7fe486c43700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1e65920 0x1e65dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:04.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.630+0000 7fe486c43700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1e65920 0x1e65dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37892/0 (socket says 172.21.15.17:37892) 2022-08-01T19:35:04.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.630+0000 7fe486c43700 1 -- 172.21.15.17:0/1358519619 learned_addr learned my addr 172.21.15.17:0/1358519619 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:04.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.630+0000 7fe486442700 1 --2- 172.21.15.17:0/1358519619 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1e32e00 0x1e31e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:04.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.630+0000 7fe485c41700 1 --2- 172.21.15.17:0/1358519619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1e331e0 0x1e32380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:04.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.630+0000 7fe486c43700 1 -- 172.21.15.17:0/1358519619 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1e32e00 msgr2=0x1e31e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:04.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.630+0000 7fe486c43700 1 --2- 172.21.15.17:0/1358519619 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1e32e00 0x1e31e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.630+0000 7fe486c43700 1 -- 172.21.15.17:0/1358519619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1e331e0 msgr2=0x1e32380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:04.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.630+0000 7fe486c43700 1 --2- 172.21.15.17:0/1358519619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1e331e0 0x1e32380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.630+0000 7fe486c43700 1 -- 172.21.15.17:0/1358519619 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1ddbec0 con 0x1e65920 2022-08-01T19:35:04.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.630+0000 7fe485c41700 1 --2- 172.21.15.17:0/1358519619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1e331e0 0x1e32380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:35:04.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.630+0000 7fe486c43700 1 --2- 172.21.15.17:0/1358519619 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1e65920 0x1e65dd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x1e36770 tx=0x7fe478007d20).ready entity=mon.0 client_cookie=e07701ecc8664c3e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:04.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.630+0000 7fe487444700 1 -- 172.21.15.17:0/1358519619 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe478001990 con 0x1e65920 2022-08-01T19:35:04.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.630+0000 7fe487444700 1 -- 172.21.15.17:0/1358519619 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe478001af0 con 0x1e65920 2022-08-01T19:35:04.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.630+0000 7fe487444700 1 -- 172.21.15.17:0/1358519619 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe478001e50 con 0x1e65920 2022-08-01T19:35:04.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe4928fd740 1 -- 172.21.15.17:0/1358519619 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1e65920 msgr2=0x1e65dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:04.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe4928fd740 1 --2- 172.21.15.17:0/1358519619 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1e65920 0x1e65dd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x1e36770 tx=0x7fe478007d20).stop 2022-08-01T19:35:04.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe4928fd740 1 -- 172.21.15.17:0/1358519619 shutdown_connections 2022-08-01T19:35:04.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe4928fd740 1 --2- 172.21.15.17:0/1358519619 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1e65920 0x1e65dd0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe4928fd740 1 --2- 172.21.15.17:0/1358519619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1e331e0 0x1e32380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe4928fd740 1 --2- 172.21.15.17:0/1358519619 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1e32e00 0x1e31e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe4928fd740 1 -- 172.21.15.17:0/1358519619 >> 172.21.15.17:0/1358519619 conn(0x1e30180 msgr2=0x1e30e20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:04.660 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe4928fd740 1 -- 172.21.15.17:0/1358519619 shutdown_connections 2022-08-01T19:35:04.660 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe4928fd740 1 -- 172.21.15.17:0/1358519619 wait complete. 2022-08-01T19:35:04.660 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe4928fd740 1 -- start start 2022-08-01T19:35:04.660 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe4928fd740 1 Processor -- start 2022-08-01T19:35:04.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe4928fd740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1e331e0 0x1e787b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:04.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe4928fd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1e65920 0x1e6a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:04.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe4928fd740 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1e7b960 0x1e7be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:04.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe4928fd740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e7b5a0 con 0x1e331e0 2022-08-01T19:35:04.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe4928fd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e77dd0 con 0x1e65920 2022-08-01T19:35:04.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe4928fd740 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e78070 con 0x1e7b960 2022-08-01T19:35:04.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe4928fd740 10 client.0 ll_register_callbacks cb 0x1d89310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-08-01T19:35:04.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe486442700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1e331e0 0x1e787b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:04.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe486442700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1e331e0 0x1e787b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37894/0 (socket says 172.21.15.17:37894) 2022-08-01T19:35:04.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe486442700 1 -- 172.21.15.17:0/2270298890 learned_addr learned my addr 172.21.15.17:0/2270298890 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:04.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe486c43700 1 --2- 172.21.15.17:0/2270298890 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1e7b960 0x1e7be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:04.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe485c41700 1 --2- 172.21.15.17:0/2270298890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1e65920 0x1e6a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:04.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe486442700 1 -- 172.21.15.17:0/2270298890 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1e7b960 msgr2=0x1e7be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:04.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe486442700 1 --2- 172.21.15.17:0/2270298890 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1e7b960 0x1e7be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe486442700 1 -- 172.21.15.17:0/2270298890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1e65920 msgr2=0x1e6a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:04.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe486442700 1 --2- 172.21.15.17:0/2270298890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1e65920 0x1e6a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe486442700 1 -- 172.21.15.17:0/2270298890 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe468003260 con 0x1e331e0 2022-08-01T19:35:04.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe486442700 1 --2- 172.21.15.17:0/2270298890 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1e331e0 0x1e787b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7fe47c020000 tx=0x7fe47c010000).ready entity=mon.0 client_cookie=5b3eea59921cee90 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:04.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe45d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2022-08-01T19:35:04.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe45d7fa700 1 -- 172.21.15.17:0/2270298890 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe47c045d30 con 0x1e331e0 2022-08-01T19:35:04.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe45d7fa700 1 -- 172.21.15.17:0/2270298890 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe47c045e90 con 0x1e331e0 2022-08-01T19:35:04.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe4928fd740 1 -- 172.21.15.17:0/2270298890 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1d99c70 con 0x1e331e0 2022-08-01T19:35:04.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe45d7fa700 1 -- 172.21.15.17:0/2270298890 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe47c046050 con 0x1e331e0 2022-08-01T19:35:04.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe4928fd740 10 client.5124 fetch_fsmap learned FSMap version 13 2022-08-01T19:35:04.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.634+0000 7fe4928fd740 1 -- 172.21.15.17:0/2270298890 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x1d76950 con 0x1e331e0 2022-08-01T19:35:04.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.638+0000 7fe45d7fa700 1 -- 172.21.15.17:0/2270298890 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1d99c70 con 0x1e331e0 2022-08-01T19:35:04.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.638+0000 7fe45d7fa700 1 -- 172.21.15.17:0/2270298890 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 13) v1 ==== 1434+0+0 (secure 0 0 0) 0x7fe47c06c7d0 con 0x1e331e0 2022-08-01T19:35:04.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.638+0000 7fe4928fd740 10 client.5124 fetch_fsmap finished waiting for FSMap version 13 2022-08-01T19:35:04.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.638+0000 7fe4928fd740 10 client.5124 resolve_mds: resolved ID 'a' to GID 4890 2022-08-01T19:35:04.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.638+0000 7fe4928fd740 20 client.5124 populate_metadata read hostname 'smithi017' 2022-08-01T19:35:04.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.638+0000 7fe4928fd740 1 --2- 172.21.15.17:0/2270298890 >> [v2:172.21.15.17:6834/3876538769,v1:172.21.15.17:6835/3876538769] conn(0x1e6d210 0x1e6d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:04.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.638+0000 7fe4928fd740 4 client.5124 mds_command: new command op to 4890 tid=0[{"prefix": "get_command_descriptions"}] 2022-08-01T19:35:04.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.638+0000 7fe4928fd740 1 -- 172.21.15.17:0/2270298890 --> [v2:172.21.15.17:6834/3876538769,v1:172.21.15.17:6835/3876538769] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1d91fe0 con 0x1e6d210 2022-08-01T19:35:04.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.638+0000 7fe485c41700 1 --2- 172.21.15.17:0/2270298890 >> [v2:172.21.15.17:6834/3876538769,v1:172.21.15.17:6835/3876538769] conn(0x1e6d210 0x1e6d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:04.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.638+0000 7fe45d7fa700 10 client.5124 ms_handle_connect on v2:172.21.15.17:6834/3876538769 2022-08-01T19:35:04.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.638+0000 7fe485c41700 1 --2- 172.21.15.17:0/2270298890 >> [v2:172.21.15.17:6834/3876538769,v1:172.21.15.17:6835/3876538769] conn(0x1e6d210 0x1e6d5f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=f3afe78e745aa6ca server_cookie=e010f8aef42ce5e9 in_seq=0 out_seq=0 2022-08-01T19:35:04.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.638+0000 7fe45d7fa700 1 -- 172.21.15.17:0/2270298890 <== mds.0 v2:172.21.15.17:6834/3876538769 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1d91fe0 con 0x1e6d210 2022-08-01T19:35:04.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.638+0000 7fe45d7fa700 10 client.5124 handle_command_reply: tid=0 2022-08-01T19:35:04.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.638+0000 7fe4928fd740 1 client.5124 shutdown 2022-08-01T19:35:04.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.638+0000 7fe4928fd740 1 -- 172.21.15.17:0/2270298890 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1e331e0 msgr2=0x1e787b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:04.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.638+0000 7fe4928fd740 1 --2- 172.21.15.17:0/2270298890 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1e331e0 0x1e787b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7fe47c020000 tx=0x7fe47c010000).stop 2022-08-01T19:35:04.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.638+0000 7fe4928fd740 1 -- 172.21.15.17:0/2270298890 shutdown_connections 2022-08-01T19:35:04.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.638+0000 7fe4928fd740 1 --2- 172.21.15.17:0/2270298890 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1e7b960 0x1e7be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.638+0000 7fe4928fd740 1 --2- 172.21.15.17:0/2270298890 >> [v2:172.21.15.17:6834/3876538769,v1:172.21.15.17:6835/3876538769] conn(0x1e6d210 0x1e6d5f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.638+0000 7fe4928fd740 1 --2- 172.21.15.17:0/2270298890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1e65920 0x1e6a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.638+0000 7fe4928fd740 1 --2- 172.21.15.17:0/2270298890 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1e331e0 0x1e787b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.638+0000 7fe45d7fa700 0 client.5124 ms_handle_reset on v2:172.21.15.17:6834/3876538769 2022-08-01T19:35:04.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.638+0000 7fe4928fd740 1 -- 172.21.15.17:0/2270298890 >> 172.21.15.17:0/2270298890 conn(0x1e32e00 msgr2=0x1e2dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:04.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.638+0000 7fe4928fd740 1 -- 172.21.15.17:0/2270298890 shutdown_connections 2022-08-01T19:35:04.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.638+0000 7fe4928fd740 1 -- 172.21.15.17:0/2270298890 wait complete. 2022-08-01T19:35:04.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.638+0000 7fe4928fd740 20 client.5124 trim_cache size 0 max 16384 2022-08-01T19:35:04.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.654+0000 7fe4928fd740 1 Processor -- start 2022-08-01T19:35:04.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.654+0000 7fe4928fd740 1 -- start start 2022-08-01T19:35:04.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.654+0000 7fe4928fd740 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1e331e0 0x1e63ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:04.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.654+0000 7fe4928fd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1e65920 0x1e61290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:04.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.654+0000 7fe4928fd740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1e7b960 0x1e61780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:04.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.654+0000 7fe4928fd740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e31490 con 0x1e7b960 2022-08-01T19:35:04.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.654+0000 7fe4928fd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e78200 con 0x1e65920 2022-08-01T19:35:04.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.654+0000 7fe4928fd740 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x1df7480 con 0x1e331e0 2022-08-01T19:35:04.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.658+0000 7fe486c43700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1e7b960 0x1e61780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:04.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.658+0000 7fe486c43700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1e7b960 0x1e61780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37906/0 (socket says 172.21.15.17:37906) 2022-08-01T19:35:04.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.658+0000 7fe486c43700 1 -- 172.21.15.17:0/1554443942 learned_addr learned my addr 172.21.15.17:0/1554443942 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:04.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.658+0000 7fe486442700 1 --2- 172.21.15.17:0/1554443942 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1e331e0 0x1e63ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:04.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.658+0000 7fe485c41700 1 --2- 172.21.15.17:0/1554443942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1e65920 0x1e61290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:04.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.658+0000 7fe486c43700 1 -- 172.21.15.17:0/1554443942 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1e331e0 msgr2=0x1e63ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:04.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.658+0000 7fe486c43700 1 --2- 172.21.15.17:0/1554443942 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1e331e0 0x1e63ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.658+0000 7fe486c43700 1 -- 172.21.15.17:0/1554443942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1e65920 msgr2=0x1e61290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:04.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.658+0000 7fe486c43700 1 --2- 172.21.15.17:0/1554443942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1e65920 0x1e61290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.658+0000 7fe486c43700 1 -- 172.21.15.17:0/1554443942 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1d93550 con 0x1e7b960 2022-08-01T19:35:04.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.658+0000 7fe486442700 1 --2- 172.21.15.17:0/1554443942 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1e331e0 0x1e63ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:35:04.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.658+0000 7fe486c43700 1 --2- 172.21.15.17:0/1554443942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1e7b960 0x1e61780 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x1e65880 tx=0x7fe478013000).ready entity=mon.0 client_cookie=e973a6b3a9bfcbfc server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:04.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.658+0000 7fe45f7fe700 1 -- 172.21.15.17:0/1554443942 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe478007a50 con 0x1e7b960 2022-08-01T19:35:04.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.658+0000 7fe45f7fe700 1 -- 172.21.15.17:0/1554443942 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4780017a0 con 0x1e7b960 2022-08-01T19:35:04.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.658+0000 7fe45f7fe700 1 -- 172.21.15.17:0/1554443942 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe478001910 con 0x1e7b960 2022-08-01T19:35:04.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.658+0000 7fe4928fd740 1 -- 172.21.15.17:0/1554443942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1e7b960 msgr2=0x1e61780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:04.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.658+0000 7fe4928fd740 1 --2- 172.21.15.17:0/1554443942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1e7b960 0x1e61780 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x1e65880 tx=0x7fe478013000).stop 2022-08-01T19:35:04.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.658+0000 7fe4928fd740 1 -- 172.21.15.17:0/1554443942 shutdown_connections 2022-08-01T19:35:04.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.658+0000 7fe4928fd740 1 --2- 172.21.15.17:0/1554443942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1e7b960 0x1e61780 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.658+0000 7fe4928fd740 1 --2- 172.21.15.17:0/1554443942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1e65920 0x1e61290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.658+0000 7fe4928fd740 1 --2- 172.21.15.17:0/1554443942 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1e331e0 0x1e63ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.658+0000 7fe4928fd740 1 -- 172.21.15.17:0/1554443942 >> 172.21.15.17:0/1554443942 conn(0x1e32e00 msgr2=0x1e33630 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:04.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.658+0000 7fe4928fd740 1 -- 172.21.15.17:0/1554443942 shutdown_connections 2022-08-01T19:35:04.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.658+0000 7fe4928fd740 1 -- 172.21.15.17:0/1554443942 wait complete. 2022-08-01T19:35:04.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.658+0000 7fe4928fd740 1 -- start start 2022-08-01T19:35:04.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.658+0000 7fe4928fd740 1 Processor -- start 2022-08-01T19:35:04.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe4928fd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1e331e0 0x1e63200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:04.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe4928fd740 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1e65920 0x1e636f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:04.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe4928fd740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1e7b960 0x1ec7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:04.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe4928fd740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ecf900 con 0x1e7b960 2022-08-01T19:35:04.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe4928fd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e753a0 con 0x1e331e0 2022-08-01T19:35:04.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe4928fd740 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x1ecf460 con 0x1e65920 2022-08-01T19:35:04.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe4928fd740 10 client.0 ll_register_callbacks cb 0x7fe47c032030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-08-01T19:35:04.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe486c43700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1e7b960 0x1ec7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:04.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe486c43700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1e7b960 0x1ec7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37912/0 (socket says 172.21.15.17:37912) 2022-08-01T19:35:04.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe486c43700 1 -- 172.21.15.17:0/3301088425 learned_addr learned my addr 172.21.15.17:0/3301088425 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:04.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe486442700 1 --2- 172.21.15.17:0/3301088425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1e331e0 0x1e63200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:04.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe485c41700 1 --2- 172.21.15.17:0/3301088425 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1e65920 0x1e636f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:04.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe486c43700 1 -- 172.21.15.17:0/3301088425 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1e65920 msgr2=0x1e636f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:04.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe486c43700 1 --2- 172.21.15.17:0/3301088425 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1e65920 0x1e636f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe486c43700 1 -- 172.21.15.17:0/3301088425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1e331e0 msgr2=0x1e63200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:04.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe486c43700 1 --2- 172.21.15.17:0/3301088425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1e331e0 0x1e63200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:04.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe486c43700 1 -- 172.21.15.17:0/3301088425 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1ddbec0 con 0x1e7b960 2022-08-01T19:35:04.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe486c43700 1 --2- 172.21.15.17:0/3301088425 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1e7b960 0x1ec7fb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x1e65880 tx=0x7fe478020000).ready entity=mon.0 client_cookie=35c79770ae315421 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:04.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe45e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2022-08-01T19:35:04.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe45e7fc700 1 -- 172.21.15.17:0/3301088425 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe478001910 con 0x1e7b960 2022-08-01T19:35:04.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe45e7fc700 1 -- 172.21.15.17:0/3301088425 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe47802ba00 con 0x1e7b960 2022-08-01T19:35:04.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe4928fd740 1 -- 172.21.15.17:0/3301088425 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe478001af0 con 0x1e7b960 2022-08-01T19:35:04.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe45e7fc700 1 -- 172.21.15.17:0/3301088425 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe47802bbc0 con 0x1e7b960 2022-08-01T19:35:04.692 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe4928fd740 10 client.5130 fetch_fsmap learned FSMap version 13 2022-08-01T19:35:04.692 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe4928fd740 1 -- 172.21.15.17:0/3301088425 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x1d9be50 con 0x1e7b960 2022-08-01T19:35:04.692 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe45e7fc700 1 -- 172.21.15.17:0/3301088425 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe478001af0 con 0x1e7b960 2022-08-01T19:35:04.692 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe45e7fc700 1 -- 172.21.15.17:0/3301088425 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 13) v1 ==== 1434+0+0 (secure 0 0 0) 0x7fe478029c60 con 0x1e7b960 2022-08-01T19:35:04.692 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe4928fd740 10 client.5130 fetch_fsmap finished waiting for FSMap version 13 2022-08-01T19:35:04.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe4928fd740 10 client.5130 resolve_mds: resolved ID 'a' to GID 4890 2022-08-01T19:35:04.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe4928fd740 20 client.5130 populate_metadata read hostname 'smithi017' 2022-08-01T19:35:04.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe4928fd740 1 --2- 172.21.15.17:0/3301088425 >> [v2:172.21.15.17:6834/3876538769,v1:172.21.15.17:6835/3876538769] conn(0x1ecef70 0x1ec84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:04.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe4928fd740 4 client.5130 mds_command: new command op to 4890 tid=0[{"prefix": "cache drop"}] 2022-08-01T19:35:04.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe4928fd740 1 -- 172.21.15.17:0/3301088425 --> [v2:172.21.15.17:6834/3876538769,v1:172.21.15.17:6835/3876538769] -- command(tid 0: {"prefix": "cache drop"}) v1 -- 0x1c87590 con 0x1ecef70 2022-08-01T19:35:04.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe486442700 1 --2- 172.21.15.17:0/3301088425 >> [v2:172.21.15.17:6834/3876538769,v1:172.21.15.17:6835/3876538769] conn(0x1ecef70 0x1ec84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:04.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe45e7fc700 10 client.5130 ms_handle_connect on v2:172.21.15.17:6834/3876538769 2022-08-01T19:35:04.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:04.662+0000 7fe486442700 1 --2- 172.21.15.17:0/3301088425 >> [v2:172.21.15.17:6834/3876538769,v1:172.21.15.17:6835/3876538769] conn(0x1ecef70 0x1ec84a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=7a24c07b1d209bbc server_cookie=77601f20b1baf42 in_seq=0 out_seq=0 2022-08-01T19:35:16.949 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:35:16.952 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:35:16.955 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:35:16.958 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:35:47.021 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:35:47.025 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:35:47.028 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:35:47.031 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:35:49.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.018+0000 7fe45e7fc700 1 -- 172.21.15.17:0/3301088425 <== mds.0 v2:172.21.15.17:6834/3876538769 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+386 (crc 0 0 0) 0x1c87590 con 0x1ecef70 2022-08-01T19:35:49.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.018+0000 7fe45e7fc700 10 client.5130 handle_command_reply: tid=0 2022-08-01T19:35:49.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.018+0000 7fe4928fd740 1 client.5130 shutdown 2022-08-01T19:35:49.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.018+0000 7fe4928fd740 1 -- 172.21.15.17:0/3301088425 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1e7b960 msgr2=0x1ec7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:49.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.018+0000 7fe4928fd740 1 --2- 172.21.15.17:0/3301088425 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1e7b960 0x1ec7fb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x1e65880 tx=0x7fe478020000).stop 2022-08-01T19:35:49.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.018+0000 7fe4928fd740 1 -- 172.21.15.17:0/3301088425 shutdown_connections 2022-08-01T19:35:49.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.018+0000 7fe4928fd740 1 --2- 172.21.15.17:0/3301088425 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1e7b960 0x1ec7fb0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:49.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.018+0000 7fe4928fd740 1 --2- 172.21.15.17:0/3301088425 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1e65920 0x1e636f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:49.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.018+0000 7fe4928fd740 1 --2- 172.21.15.17:0/3301088425 >> [v2:172.21.15.17:6834/3876538769,v1:172.21.15.17:6835/3876538769] conn(0x1ecef70 0x1ec84a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:49.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.018+0000 7fe4928fd740 1 --2- 172.21.15.17:0/3301088425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1e331e0 0x1e63200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:49.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.018+0000 7fe45e7fc700 0 client.5130 ms_handle_reset on v2:172.21.15.17:6834/3876538769 2022-08-01T19:35:49.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.018+0000 7fe4928fd740 1 -- 172.21.15.17:0/3301088425 >> 172.21.15.17:0/3301088425 conn(0x1e32e00 msgr2=0x1e6b140 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:49.026 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.018+0000 7fe4928fd740 1 -- 172.21.15.17:0/3301088425 shutdown_connections 2022-08-01T19:35:49.026 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.018+0000 7fe4928fd740 1 -- 172.21.15.17:0/3301088425 wait complete. 2022-08-01T19:35:49.026 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.018+0000 7fe4928fd740 20 client.5130 trim_cache size 0 max 16384 2022-08-01T19:35:49.027 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:35:49.027 INFO:teuthology.orchestra.run.smithi017.stdout: "client_recall": { 2022-08-01T19:35:49.028 INFO:teuthology.orchestra.run.smithi017.stdout: "return_code": 0, 2022-08-01T19:35:49.028 INFO:teuthology.orchestra.run.smithi017.stdout: "message": "(0) Success", 2022-08-01T19:35:49.028 INFO:teuthology.orchestra.run.smithi017.stdout: "recalled": 920 2022-08-01T19:35:49.028 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.029 INFO:teuthology.orchestra.run.smithi017.stdout: "flush_journal": { 2022-08-01T19:35:49.029 INFO:teuthology.orchestra.run.smithi017.stdout: "return_code": 0, 2022-08-01T19:35:49.029 INFO:teuthology.orchestra.run.smithi017.stdout: "message": "" 2022-08-01T19:35:49.029 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.030 INFO:teuthology.orchestra.run.smithi017.stdout: "trim_cache": { 2022-08-01T19:35:49.030 INFO:teuthology.orchestra.run.smithi017.stdout: "trimmed": 900 2022-08-01T19:35:49.030 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.030 INFO:teuthology.orchestra.run.smithi017.stdout: "cache": { 2022-08-01T19:35:49.031 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": { 2022-08-01T19:35:49.031 INFO:teuthology.orchestra.run.smithi017.stdout: "items": 2525, 2022-08-01T19:35:49.031 INFO:teuthology.orchestra.run.smithi017.stdout: "bytes": 431920 2022-08-01T19:35:49.031 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.032 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.032 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 44.352274540000003 2022-08-01T19:35:49.032 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:35:49.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.018+0000 7fe487444700 1 -- 172.21.15.17:0/888892808 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe460053a70 msgr2=0x7fe460055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:49.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.018+0000 7fe487444700 1 --2- 172.21.15.17:0/888892808 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe460053a70 0x7fe460055f30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fe47000b000 tx=0x7fe470006eb0).stop 2022-08-01T19:35:49.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.018+0000 7fe487444700 1 -- 172.21.15.17:0/888892808 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4800614b0 msgr2=0x7fe4801483b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:49.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.018+0000 7fe487444700 1 --2- 172.21.15.17:0/888892808 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4800614b0 0x7fe4801483b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fe47c002f70 tx=0x7fe47c014000).stop 2022-08-01T19:35:49.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.018+0000 7fe487444700 1 -- 172.21.15.17:0/888892808 shutdown_connections 2022-08-01T19:35:49.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.018+0000 7fe487444700 1 --2- 172.21.15.17:0/888892808 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe460053a70 0x7fe460055f30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:49.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.018+0000 7fe487444700 1 --2- 172.21.15.17:0/888892808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4801405d0 0x7fe4800aca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:49.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.018+0000 7fe487444700 1 --2- 172.21.15.17:0/888892808 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe4800ad750 0x7fe480149500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:49.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.018+0000 7fe487444700 1 --2- 172.21.15.17:0/888892808 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4800614b0 0x7fe4801483b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:49.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.018+0000 7fe487444700 1 -- 172.21.15.17:0/888892808 >> 172.21.15.17:0/888892808 conn(0x7fe480066f70 msgr2=0x7fe4800b0210 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:49.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.018+0000 7fe487444700 1 -- 172.21.15.17:0/888892808 shutdown_connections 2022-08-01T19:35:49.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.018+0000 7fe487444700 1 -- 172.21.15.17:0/888892808 wait complete. 2022-08-01T19:35:49.038 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T19:35:49.038 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:35:49.056 DEBUG:teuthology.orchestra.run.smithi077:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:35:49.077 DEBUG:teuthology.orchestra.run.smithi085:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:35:49.097 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:35:49.119 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T19:35:49.120 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:35:49.121 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:35:49.122 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:35:49.149 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:35:49.151 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:35:49.151 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi085.front.sepia.ceph.com... 2022-08-01T19:35:49.152 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T19:35:49.153 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:35:49.170 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:ceph-fuse[22481]: fuse finished with error 0 and tester_r 0 2022-08-01T19:35:49.172 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:35:49.173 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:35:49.174 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:35:49.219 INFO:teuthology.orchestra.run.smithi085.stdout:ext2/ext3 2022-08-01T19:35:49.220 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T19:35:49.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T19:35:49.339 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:35:49.346 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:35:49.346 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:35:49.347 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:35:49.354 INFO:teuthology.orchestra.run.smithi190.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T19:35:49.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:35:49.356 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-08-01T19:35:49.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T19:35:49.357 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:35:49.403 INFO:teuthology.orchestra.run.smithi190.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T19:35:49.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:35:49.405 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T19:35:49.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.518+0000 7f322b59e700 1 Processor -- start 2022-08-01T19:35:49.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.518+0000 7f322b59e700 1 -- start start 2022-08-01T19:35:49.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.518+0000 7f322b59e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f322c0b2710 0x7f322c1401e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:49.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.518+0000 7f322b59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f322c0ac880 0x7f322c1406d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:49.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.518+0000 7f322b59e700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f322c067a10 con 0x7f322c0b0270 2022-08-01T19:35:49.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.518+0000 7f322b59e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f322c0ad740 con 0x7f322c0b2710 2022-08-01T19:35:49.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.518+0000 7f322b59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322c067cb0 con 0x7f322c0ac880 2022-08-01T19:35:49.525 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.518+0000 7f3223fff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f322c0b0270 0x7f322c143000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58344/0 (socket says 172.21.15.17:58344) 2022-08-01T19:35:49.525 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.518+0000 7f3223fff700 1 -- 172.21.15.17:0/3840837009 learned_addr learned my addr 172.21.15.17:0/3840837009 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:35:49.525 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.518+0000 7f322a59c700 1 --2- 172.21.15.17:0/3840837009 >> v2:172.21.15.77:3300/0 conn(0x7f322c0b2710 0x7f322c1401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:49.526 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.518+0000 7f322ad9d700 1 --2- 172.21.15.17:0/3840837009 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f322c0ac880 0x7f322c1406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:49.526 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.518+0000 7f3229d9b700 1 -- 172.21.15.17:0/3840837009 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2932031238 0 0) 0x7f322c067a10 con 0x7f322c0b0270 2022-08-01T19:35:49.526 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.518+0000 7f3229d9b700 1 -- 172.21.15.17:0/3840837009 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f320c003260 con 0x7f322c0b0270 2022-08-01T19:35:49.526 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.518+0000 7f322ad9d700 1 -- 172.21.15.17:0/3840837009 >> v2:172.21.15.77:3300/0 conn(0x7f322c0b2710 msgr2=0x7f322c1401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:49.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.518+0000 7f322ad9d700 1 --2- 172.21.15.17:0/3840837009 >> v2:172.21.15.77:3300/0 conn(0x7f322c0b2710 0x7f322c1401e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:49.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.518+0000 7f322ad9d700 1 -- 172.21.15.17:0/3840837009 >> v1:172.21.15.17:6789/0 conn(0x7f322c0b0270 legacy=0x7f322c143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:49.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.518+0000 7f322ad9d700 1 -- 172.21.15.17:0/3840837009 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322c147160 con 0x7f322c0ac880 2022-08-01T19:35:49.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.518+0000 7f322ad9d700 1 --2- 172.21.15.17:0/3840837009 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f322c0ac880 0x7f322c1406d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f32240032f0 tx=0x7f32240091d0).ready entity=mon.2 client_cookie=5c879b2d239c892a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:49.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.518+0000 7f3229d9b700 1 -- 172.21.15.17:0/3840837009 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f322400b600 con 0x7f322c0ac880 2022-08-01T19:35:49.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.518+0000 7f3229d9b700 1 -- 172.21.15.17:0/3840837009 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f322400b760 con 0x7f322c0ac880 2022-08-01T19:35:49.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.518+0000 7f3229d9b700 1 -- 172.21.15.17:0/3840837009 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f322400b8d0 con 0x7f322c0ac880 2022-08-01T19:35:49.532 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.518+0000 7f322b59e700 1 -- 172.21.15.17:0/3840837009 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f322c0ac880 msgr2=0x7f322c1406d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:49.532 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.518+0000 7f322b59e700 1 --2- 172.21.15.17:0/3840837009 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f322c0ac880 0x7f322c1406d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f32240032f0 tx=0x7f32240091d0).stop 2022-08-01T19:35:49.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.518+0000 7f322b59e700 1 -- 172.21.15.17:0/3840837009 shutdown_connections 2022-08-01T19:35:49.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.518+0000 7f322b59e700 1 --2- 172.21.15.17:0/3840837009 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f322c0ac880 0x7f322c1406d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:49.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.518+0000 7f322b59e700 1 --2- 172.21.15.17:0/3840837009 >> v2:172.21.15.77:3300/0 conn(0x7f322c0b2710 0x7f322c1401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:49.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.518+0000 7f322b59e700 1 -- 172.21.15.17:0/3840837009 >> 172.21.15.17:0/3840837009 conn(0x7f322c060310 msgr2=0x7f322c0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:49.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.518+0000 7f322b59e700 1 -- 172.21.15.17:0/3840837009 shutdown_connections 2022-08-01T19:35:49.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.518+0000 7f322b59e700 1 -- 172.21.15.17:0/3840837009 wait complete. 2022-08-01T19:35:49.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.522+0000 7f322b59e700 1 Processor -- start 2022-08-01T19:35:49.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.522+0000 7f322b59e700 1 -- start start 2022-08-01T19:35:49.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.522+0000 7f322b59e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f322c0ac880 0x7f322c1459b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:49.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.522+0000 7f322b59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f322c0b0270 0x7f322c156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:49.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.522+0000 7f322b59e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f322c0b2710 0x7f322c13d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:49.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.522+0000 7f322b59e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c155a50 con 0x7f322c0ac880 2022-08-01T19:35:49.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.522+0000 7f322b59e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c0a9b60 con 0x7f322c0b2710 2022-08-01T19:35:49.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.522+0000 7f322b59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322c0a98c0 con 0x7f322c0b0270 2022-08-01T19:35:49.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.522+0000 7f322a59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f322c0ac880 0x7f322c1459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:49.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.522+0000 7f322a59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f322c0ac880 0x7f322c1459b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37924/0 (socket says 172.21.15.17:37924) 2022-08-01T19:35:49.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.522+0000 7f322a59c700 1 -- 172.21.15.17:0/801464591 learned_addr learned my addr 172.21.15.17:0/801464591 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:49.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.522+0000 7f322ad9d700 1 --2- 172.21.15.17:0/801464591 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f322c0b2710 0x7f322c13d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:49.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.522+0000 7f3223fff700 1 --2- 172.21.15.17:0/801464591 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f322c0b0270 0x7f322c156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:49.538 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.522+0000 7f3223fff700 1 -- 172.21.15.17:0/801464591 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f322c0b2710 msgr2=0x7f322c13d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:49.538 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.522+0000 7f3223fff700 1 --2- 172.21.15.17:0/801464591 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f322c0b2710 0x7f322c13d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:49.538 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.522+0000 7f3223fff700 1 -- 172.21.15.17:0/801464591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f322c0ac880 msgr2=0x7f322c1459b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:49.539 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.522+0000 7f3223fff700 1 --2- 172.21.15.17:0/801464591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f322c0ac880 0x7f322c1459b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:49.539 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.522+0000 7f3223fff700 1 -- 172.21.15.17:0/801464591 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3224009470 con 0x7f322c0b0270 2022-08-01T19:35:49.539 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.522+0000 7f3223fff700 1 --2- 172.21.15.17:0/801464591 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f322c0b0270 0x7f322c156150 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f3214001110 tx=0x7f32140079e0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:49.539 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.522+0000 7f32237fe700 1 -- 172.21.15.17:0/801464591 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f321400cdb0 con 0x7f322c0b0270 2022-08-01T19:35:49.539 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.522+0000 7f32237fe700 1 -- 172.21.15.17:0/801464591 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32140070d0 con 0x7f322c0b0270 2022-08-01T19:35:49.540 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.522+0000 7f322b59e700 1 -- 172.21.15.17:0/801464591 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f322c155cb0 con 0x7f322c0b0270 2022-08-01T19:35:49.540 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.522+0000 7f32237fe700 1 -- 172.21.15.17:0/801464591 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3214007430 con 0x7f322c0b0270 2022-08-01T19:35:49.540 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.522+0000 7f322b59e700 1 -- 172.21.15.17:0/801464591 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f322c141e20 con 0x7f322c0b0270 2022-08-01T19:35:49.540 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.526+0000 7f322b59e700 1 -- 172.21.15.17:0/801464591 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f322c155e40 con 0x7f322c0b0270 2022-08-01T19:35:49.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.526+0000 7f32237fe700 1 -- 172.21.15.17:0/801464591 <== mon.2 v2:172.21.15.77:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3214010070 con 0x7f322c0b0270 2022-08-01T19:35:49.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.526+0000 7f32237fe700 1 --2- 172.21.15.17:0/801464591 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3210053950 0x7f3210055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:49.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.526+0000 7f32237fe700 1 -- 172.21.15.17:0/801464591 <== mon.2 v2:172.21.15.77:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f321406b550 con 0x7f322c0b0270 2022-08-01T19:35:49.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.526+0000 7f322a59c700 1 --2- 172.21.15.17:0/801464591 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3210053950 0x7f3210055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:49.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.526+0000 7f322a59c700 1 --2- 172.21.15.17:0/801464591 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3210053950 0x7f3210055e10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f321c00b000 tx=0x7f321c006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:49.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.538+0000 7f32237fe700 1 -- 172.21.15.17:0/801464591 <== mon.2 v2:172.21.15.77:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3214042270 con 0x7f322c0b0270 2022-08-01T19:35:49.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.658+0000 7f322b59e700 1 -- 172.21.15.17:0/801464591 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f322c0574e0 con 0x7f322c0b0270 2022-08-01T19:35:49.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.666+0000 7f32237fe700 1 -- 172.21.15.17:0/801464591 <== mon.2 v2:172.21.15.77:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v35) v1 ==== 81+0+34037 (secure 0 0 0) 0x7f3214018070 con 0x7f322c0b0270 2022-08-01T19:35:49.667 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:35:49.668 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:35:49.668 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 35, 2022-08-01T19:35:49.668 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", 2022-08-01T19:35:49.669 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2022-08-01T19:33:50.105743+0000", 2022-08-01T19:35:49.669 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2022-08-01T19:34:47.691276+0000", 2022-08-01T19:35:49.669 INFO:teuthology.orchestra.run.smithi017.stdout: "last_up_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:35:49.670 INFO:teuthology.orchestra.run.smithi017.stdout: "last_in_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:35:49.670 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T19:35:49.670 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_num": 5799936, 2022-08-01T19:35:49.671 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_set": [ 2022-08-01T19:35:49.671 INFO:teuthology.orchestra.run.smithi017.stdout: "pglog_hardlimit", 2022-08-01T19:35:49.671 INFO:teuthology.orchestra.run.smithi017.stdout: "purged_snapdirs", 2022-08-01T19:35:49.671 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_deletes", 2022-08-01T19:35:49.671 INFO:teuthology.orchestra.run.smithi017.stdout: "sortbitwise" 2022-08-01T19:35:49.672 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:35:49.672 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_version": 6, 2022-08-01T19:35:49.672 INFO:teuthology.orchestra.run.smithi017.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T19:35:49.672 INFO:teuthology.orchestra.run.smithi017.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T19:35:49.673 INFO:teuthology.orchestra.run.smithi017.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T19:35:49.673 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_snapshot": "", 2022-08-01T19:35:49.673 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_max": 6, 2022-08-01T19:35:49.673 INFO:teuthology.orchestra.run.smithi017.stdout: "max_osd": 8, 2022-08-01T19:35:49.674 INFO:teuthology.orchestra.run.smithi017.stdout: "require_min_compat_client": "jewel", 2022-08-01T19:35:49.674 INFO:teuthology.orchestra.run.smithi017.stdout: "min_compat_client": "jewel", 2022-08-01T19:35:49.674 INFO:teuthology.orchestra.run.smithi017.stdout: "require_osd_release": "octopus", 2022-08-01T19:35:49.674 INFO:teuthology.orchestra.run.smithi017.stdout: "pools": [ 2022-08-01T19:35:49.675 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.675 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 1, 2022-08-01T19:35:49.675 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "device_health_metrics", 2022-08-01T19:35:49.675 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:33:53.001841+0000", 2022-08-01T19:35:49.676 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:35:49.676 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:35:49.676 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:35:49.676 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:35:49.677 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:35:49.677 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:35:49.677 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:35:49.677 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:35:49.678 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 1, 2022-08-01T19:35:49.678 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 1, 2022-08-01T19:35:49.678 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 1, 2022-08-01T19:35:49.678 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 1, 2022-08-01T19:35:49.679 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 1, 2022-08-01T19:35:49.679 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:35:49.679 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:35:49.679 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:35:49.680 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:35:49.680 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:35:49.681 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:35:49.681 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:35:49.682 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.682 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "16", 2022-08-01T19:35:49.682 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:35:49.682 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:35:49.683 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:35:49.683 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:35:49.683 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:35:49.683 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:35:49.684 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:35:49.684 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:35:49.684 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:35:49.685 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:35:49.685 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:35:49.685 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:35:49.686 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:35:49.686 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:35:49.686 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:35:49.686 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:35:49.687 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:35:49.687 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:35:49.687 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:35:49.687 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:35:49.688 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:35:49.688 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:35:49.688 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:35:49.688 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:35:49.689 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:35:49.689 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:35:49.689 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.689 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:35:49.690 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:35:49.690 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:35:49.690 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:35:49.691 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:35:49.691 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:35:49.691 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:35:49.691 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:35:49.692 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:35:49.692 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:35:49.692 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:35:49.693 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:35:49.693 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 1 2022-08-01T19:35:49.693 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.694 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:35:49.694 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_devicehealth": {} 2022-08-01T19:35:49.694 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.695 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.695 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.695 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 2, 2022-08-01T19:35:49.695 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "rbd", 2022-08-01T19:35:49.696 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:34:01.553098+0000", 2022-08-01T19:35:49.696 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:35:49.696 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:35:49.697 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:35:49.697 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:35:49.697 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:35:49.698 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:35:49.698 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:35:49.698 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:35:49.699 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:35:49.699 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:35:49.699 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:35:49.699 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:35:49.700 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:35:49.700 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:35:49.700 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:35:49.700 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:35:49.701 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:35:49.701 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:35:49.701 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:35:49.702 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:35:49.702 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.702 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "19", 2022-08-01T19:35:49.702 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:35:49.703 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:35:49.703 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:35:49.703 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:35:49.703 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:35:49.704 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:35:49.704 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:35:49.704 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:35:49.704 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:35:49.705 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:35:49.705 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:35:49.705 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:35:49.705 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:35:49.705 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:35:49.706 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:35:49.706 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:35:49.706 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:35:49.707 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:35:49.707 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:35:49.707 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:35:49.707 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:35:49.707 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:35:49.708 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:35:49.708 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:35:49.708 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:35:49.708 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:35:49.709 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.709 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:35:49.709 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:35:49.709 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:35:49.710 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:35:49.710 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:35:49.710 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:35:49.710 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:35:49.711 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:35:49.711 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:35:49.711 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:35:49.711 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:35:49.712 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:35:49.712 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:35:49.712 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd": {} 2022-08-01T19:35:49.713 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.713 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.713 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.713 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 5, 2022-08-01T19:35:49.714 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "cephfs_metadata", 2022-08-01T19:35:49.714 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:34:44.961205+0000", 2022-08-01T19:35:49.714 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:35:49.715 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:35:49.715 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:35:49.715 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:35:49.715 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:35:49.716 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:35:49.716 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:35:49.716 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:35:49.716 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:35:49.717 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:35:49.717 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:35:49.717 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:35:49.717 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:35:49.718 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:35:49.718 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:35:49.718 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:35:49.718 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:35:49.719 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:35:49.719 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:35:49.719 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:35:49.719 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.720 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "34", 2022-08-01T19:35:49.720 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:35:49.720 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:35:49.720 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:35:49.721 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:35:49.721 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:35:49.721 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:35:49.721 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:35:49.722 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:35:49.722 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:35:49.722 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:35:49.723 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:35:49.723 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:35:49.723 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:35:49.724 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:35:49.724 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:35:49.724 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:35:49.724 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:35:49.725 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:35:49.725 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:35:49.725 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:35:49.725 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:35:49.725 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:35:49.726 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:35:49.726 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:35:49.726 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:35:49.727 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:35:49.727 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.727 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:35:49.727 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:35:49.727 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:35:49.728 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:35:49.728 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:35:49.728 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:35:49.728 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:35:49.729 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:35:49.729 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:35:49.729 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:35:49.729 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:35:49.730 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:35:49.730 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_bias": 4, 2022-08-01T19:35:49.730 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 16, 2022-08-01T19:35:49.730 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_priority": 5 2022-08-01T19:35:49.731 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.731 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:35:49.731 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs": { 2022-08-01T19:35:49.732 INFO:teuthology.orchestra.run.smithi017.stdout: "metadata": "cephfs" 2022-08-01T19:35:49.732 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.732 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.732 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.733 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.733 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 6, 2022-08-01T19:35:49.733 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "cephfs_data", 2022-08-01T19:35:49.733 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:34:45.687600+0000", 2022-08-01T19:35:49.734 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:35:49.734 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:35:49.734 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:35:49.734 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:35:49.735 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:35:49.735 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:35:49.735 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:35:49.735 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:35:49.736 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:35:49.736 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:35:49.736 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:35:49.736 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:35:49.737 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:35:49.737 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:35:49.737 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:35:49.737 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:35:49.737 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:35:49.738 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:35:49.738 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:35:49.738 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:35:49.738 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.739 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "35", 2022-08-01T19:35:49.739 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:35:49.739 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:35:49.739 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:35:49.740 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:35:49.740 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:35:49.740 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:35:49.740 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:35:49.741 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:35:49.741 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:35:49.741 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:35:49.741 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:35:49.742 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:35:49.742 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:35:49.742 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:35:49.742 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:35:49.743 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:35:49.743 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:35:49.743 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:35:49.743 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:35:49.743 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:35:49.744 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:35:49.744 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:35:49.744 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:35:49.744 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:35:49.745 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:35:49.745 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:35:49.745 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.745 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:35:49.746 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:35:49.746 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:35:49.746 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:35:49.746 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:35:49.747 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:35:49.747 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:35:49.747 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:35:49.747 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:35:49.748 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:35:49.748 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:35:49.748 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:35:49.748 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:35:49.749 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs": { 2022-08-01T19:35:49.749 INFO:teuthology.orchestra.run.smithi017.stdout: "data": "cephfs" 2022-08-01T19:35:49.749 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.749 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.749 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.750 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:35:49.750 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2022-08-01T19:35:49.750 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.750 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:35:49.751 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a962f14f-cf16-4fa8-84ff-c85325db9132", 2022-08-01T19:35:49.751 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:35:49.751 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:35:49.751 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:35:49.752 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:35:49.752 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:35:49.752 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:35:49.752 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:35:49.753 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 33, 2022-08-01T19:35:49.753 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:35:49.753 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:35:49.753 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:35:49.754 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:49.754 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.754 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:49.754 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6800", 2022-08-01T19:35:49.755 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:35:49.755 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.755 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.755 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:49.755 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6801", 2022-08-01T19:35:49.756 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:35:49.756 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.756 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.756 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.757 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:35:49.757 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:49.757 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.757 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:49.758 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6802", 2022-08-01T19:35:49.758 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:35:49.758 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.758 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.758 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:49.759 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6803", 2022-08-01T19:35:49.759 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:35:49.759 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.759 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.760 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.760 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:35:49.760 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:49.760 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.761 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:49.761 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6806", 2022-08-01T19:35:49.761 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:35:49.761 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.761 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.762 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:49.762 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6807", 2022-08-01T19:35:49.762 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:35:49.762 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.763 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.763 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.763 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:35:49.763 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:49.764 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.764 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:49.764 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6804", 2022-08-01T19:35:49.764 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:35:49.765 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.765 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.765 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:49.765 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6805", 2022-08-01T19:35:49.765 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:35:49.766 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.766 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.766 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.766 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6801/23008", 2022-08-01T19:35:49.767 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6803/23008", 2022-08-01T19:35:49.767 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6807/23008", 2022-08-01T19:35:49.767 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6805/23008", 2022-08-01T19:35:49.767 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:35:49.768 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:35:49.768 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:35:49.768 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.768 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.769 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.769 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:35:49.769 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "805f7051-a527-4e98-8844-b6e406a63248", 2022-08-01T19:35:49.769 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:35:49.769 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:35:49.770 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:35:49.770 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:35:49.770 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:35:49.770 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:35:49.771 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:35:49.771 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 33, 2022-08-01T19:35:49.771 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:35:49.771 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:35:49.772 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:35:49.772 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:49.772 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.772 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:49.773 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6808", 2022-08-01T19:35:49.773 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:35:49.773 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.773 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.773 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:49.774 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6809", 2022-08-01T19:35:49.774 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:35:49.774 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.774 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.775 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.775 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:35:49.775 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:49.775 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.776 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:49.776 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6810", 2022-08-01T19:35:49.776 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:35:49.776 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.776 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.777 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:49.777 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6811", 2022-08-01T19:35:49.777 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:35:49.777 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.778 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.778 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.778 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:35:49.778 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:49.779 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.779 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:49.779 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6814", 2022-08-01T19:35:49.779 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:35:49.780 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.780 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.780 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:49.780 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6815", 2022-08-01T19:35:49.780 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:35:49.781 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.781 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.781 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.781 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:35:49.782 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:49.782 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.782 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:49.782 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6812", 2022-08-01T19:35:49.783 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:35:49.783 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.783 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.783 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:49.784 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6813", 2022-08-01T19:35:49.784 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:35:49.784 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.784 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.784 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.785 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6809/23007", 2022-08-01T19:35:49.785 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6811/23007", 2022-08-01T19:35:49.785 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6815/23007", 2022-08-01T19:35:49.785 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6813/23007", 2022-08-01T19:35:49.786 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:35:49.786 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:35:49.786 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:35:49.786 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.787 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.787 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.787 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:35:49.787 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "89f7f3df-7c10-48e1-8b69-9e4243923a61", 2022-08-01T19:35:49.787 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:35:49.788 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:35:49.788 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:35:49.788 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:35:49.788 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:35:49.789 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:35:49.789 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:35:49.789 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 32, 2022-08-01T19:35:49.789 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:35:49.790 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:35:49.790 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:35:49.790 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:49.790 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.791 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:49.791 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6817", 2022-08-01T19:35:49.791 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:35:49.791 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.791 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.792 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:49.792 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6819", 2022-08-01T19:35:49.792 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:35:49.792 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.793 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.793 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.793 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:35:49.793 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:49.794 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.794 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:49.794 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6822", 2022-08-01T19:35:49.794 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:35:49.795 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.795 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.795 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:49.795 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6824", 2022-08-01T19:35:49.795 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:35:49.796 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.796 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.796 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.796 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:35:49.797 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:49.797 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.797 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:49.797 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6830", 2022-08-01T19:35:49.798 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:35:49.798 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.798 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.798 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:49.799 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6831", 2022-08-01T19:35:49.799 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:35:49.799 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.799 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.799 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.800 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:35:49.800 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:49.800 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.800 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:49.801 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6826", 2022-08-01T19:35:49.801 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:35:49.801 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.801 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.802 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:49.802 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6829", 2022-08-01T19:35:49.802 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:35:49.802 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.803 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.803 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.803 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6819/23010", 2022-08-01T19:35:49.803 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6824/23010", 2022-08-01T19:35:49.803 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6831/23010", 2022-08-01T19:35:49.804 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6829/23010", 2022-08-01T19:35:49.804 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:35:49.804 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:35:49.804 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:35:49.805 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.805 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.805 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.805 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:35:49.806 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "96123468-aabc-4a64-909e-a192c369090d", 2022-08-01T19:35:49.806 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:35:49.806 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:35:49.806 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:35:49.807 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:35:49.807 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:35:49.807 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:35:49.807 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:35:49.807 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 33, 2022-08-01T19:35:49.808 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:35:49.808 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:35:49.808 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:35:49.808 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:49.809 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.809 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:49.809 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6816", 2022-08-01T19:35:49.809 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:35:49.810 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.810 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.810 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:49.810 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6818", 2022-08-01T19:35:49.810 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:35:49.811 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.811 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.811 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.811 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:35:49.812 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:49.812 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.812 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:49.812 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6820", 2022-08-01T19:35:49.813 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:35:49.813 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.813 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.813 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:49.814 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6821", 2022-08-01T19:35:49.814 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:35:49.814 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.814 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.814 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.815 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:35:49.815 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:49.815 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.815 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:49.816 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6827", 2022-08-01T19:35:49.816 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:35:49.816 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.816 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.817 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:49.817 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6828", 2022-08-01T19:35:49.817 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:35:49.817 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.818 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.818 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.818 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:35:49.818 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:49.818 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.819 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:49.819 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6823", 2022-08-01T19:35:49.819 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:35:49.819 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.820 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.820 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:49.820 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6825", 2022-08-01T19:35:49.820 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:35:49.821 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.821 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.821 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.821 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6818/23009", 2022-08-01T19:35:49.822 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6821/23009", 2022-08-01T19:35:49.822 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6828/23009", 2022-08-01T19:35:49.822 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6825/23009", 2022-08-01T19:35:49.822 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:35:49.822 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:35:49.823 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:35:49.823 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.823 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.823 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.824 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:35:49.824 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a6e26665-e04b-46ec-ab66-e912acbf044c", 2022-08-01T19:35:49.824 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:35:49.824 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:35:49.825 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:35:49.825 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:35:49.825 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:35:49.825 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:35:49.825 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:35:49.826 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 33, 2022-08-01T19:35:49.826 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:35:49.826 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:35:49.826 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:35:49.827 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:49.827 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.827 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:49.827 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6810", 2022-08-01T19:35:49.828 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:35:49.828 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.828 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.828 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:49.829 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6811", 2022-08-01T19:35:49.829 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:35:49.829 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.829 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.829 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.830 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:35:49.830 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:49.830 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.830 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:49.831 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6812", 2022-08-01T19:35:49.831 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:35:49.831 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.831 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.832 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:49.832 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6813", 2022-08-01T19:35:49.832 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:35:49.832 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.833 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.833 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.833 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:35:49.833 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:49.833 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.834 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:49.834 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6816", 2022-08-01T19:35:49.834 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:35:49.834 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.835 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.835 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:49.835 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6817", 2022-08-01T19:35:49.835 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:35:49.836 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.836 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.836 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.836 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:35:49.837 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:49.837 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.837 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:49.837 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6814", 2022-08-01T19:35:49.837 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:35:49.838 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.838 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.838 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:49.838 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6815", 2022-08-01T19:35:49.839 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:35:49.839 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.839 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.839 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.840 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6811/23584", 2022-08-01T19:35:49.840 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6813/23584", 2022-08-01T19:35:49.840 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6817/23584", 2022-08-01T19:35:49.840 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6815/23584", 2022-08-01T19:35:49.841 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:35:49.841 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:35:49.841 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:35:49.841 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.842 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.842 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.842 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:35:49.842 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "502c0034-18ee-44ac-87ff-19b9dddb66a0", 2022-08-01T19:35:49.842 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:35:49.843 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:35:49.843 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:35:49.843 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:35:49.843 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:35:49.844 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:35:49.844 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:35:49.844 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 33, 2022-08-01T19:35:49.844 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:35:49.845 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:35:49.845 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:35:49.845 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:49.845 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.846 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:49.846 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6802", 2022-08-01T19:35:49.846 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:35:49.846 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.847 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.847 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:49.847 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6803", 2022-08-01T19:35:49.847 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:35:49.847 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.848 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.848 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.848 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:35:49.848 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:49.849 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.849 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:49.849 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6804", 2022-08-01T19:35:49.849 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:35:49.850 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.850 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.850 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:49.850 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6805", 2022-08-01T19:35:49.851 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:35:49.851 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.851 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.851 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.851 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:35:49.852 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:49.852 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.852 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:49.852 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6808", 2022-08-01T19:35:49.853 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:35:49.853 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.853 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.853 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:49.854 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6809", 2022-08-01T19:35:49.854 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:35:49.854 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.854 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.855 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.855 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:35:49.855 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:49.855 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.856 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:49.856 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6806", 2022-08-01T19:35:49.856 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:35:49.856 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.856 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.857 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:49.857 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6807", 2022-08-01T19:35:49.857 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:35:49.857 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.858 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.858 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.858 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6803/23585", 2022-08-01T19:35:49.858 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6805/23585", 2022-08-01T19:35:49.859 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6809/23585", 2022-08-01T19:35:49.859 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6807/23585", 2022-08-01T19:35:49.859 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:35:49.859 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:35:49.860 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:35:49.860 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.860 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.860 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.860 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:35:49.861 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "ac4e755a-7fe7-46cb-8407-852d5af9ffc5", 2022-08-01T19:35:49.861 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:35:49.861 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:35:49.861 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:35:49.862 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:35:49.862 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:35:49.862 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:35:49.862 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:35:49.863 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 32, 2022-08-01T19:35:49.863 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:35:49.863 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:35:49.863 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:35:49.864 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:49.864 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.864 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:49.864 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6820", 2022-08-01T19:35:49.864 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:35:49.865 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.865 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.865 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:49.865 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6821", 2022-08-01T19:35:49.866 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:35:49.866 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.866 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.866 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.867 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:35:49.867 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:49.867 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.867 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:49.868 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6823", 2022-08-01T19:35:49.868 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:35:49.868 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.868 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.868 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:49.869 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6825", 2022-08-01T19:35:49.869 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:35:49.869 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.869 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.870 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.870 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:35:49.870 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:49.870 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.871 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:49.871 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6831", 2022-08-01T19:35:49.871 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:35:49.871 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.872 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.872 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:49.872 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6833", 2022-08-01T19:35:49.872 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:35:49.872 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.873 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.873 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.873 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:35:49.873 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:49.874 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.874 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:49.874 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6827", 2022-08-01T19:35:49.874 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:35:49.875 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.875 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.875 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:49.875 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6829", 2022-08-01T19:35:49.876 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:35:49.876 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.876 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.876 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.876 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6821/23587", 2022-08-01T19:35:49.877 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6825/23587", 2022-08-01T19:35:49.877 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6833/23587", 2022-08-01T19:35:49.877 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6829/23587", 2022-08-01T19:35:49.877 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:35:49.878 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:35:49.878 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:35:49.878 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.878 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.879 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.879 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:35:49.879 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "874761e2-6410-40fc-b60c-e58abe5ce489", 2022-08-01T19:35:49.879 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:35:49.880 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:35:49.880 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:35:49.880 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:35:49.880 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:35:49.880 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:35:49.881 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:35:49.881 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 33, 2022-08-01T19:35:49.881 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:35:49.881 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:35:49.882 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:35:49.882 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:49.882 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.882 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:49.883 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6818", 2022-08-01T19:35:49.883 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:35:49.883 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.883 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.883 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:49.884 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6819", 2022-08-01T19:35:49.884 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:35:49.884 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.884 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.885 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.885 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:35:49.885 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:49.885 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.886 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:49.886 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6822", 2022-08-01T19:35:49.886 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:35:49.886 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.887 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.887 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:49.887 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6824", 2022-08-01T19:35:49.887 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:35:49.887 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.888 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.888 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.888 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:35:49.888 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:49.889 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.889 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:49.889 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6830", 2022-08-01T19:35:49.889 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:35:49.890 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.890 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.890 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:49.890 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6832", 2022-08-01T19:35:49.891 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:35:49.891 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.891 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.891 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.892 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:35:49.892 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:49.892 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.892 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:49.893 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6826", 2022-08-01T19:35:49.893 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:35:49.893 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.893 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.894 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:49.894 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6828", 2022-08-01T19:35:49.894 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:35:49.894 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.895 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.895 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.895 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6819/23586", 2022-08-01T19:35:49.895 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6824/23586", 2022-08-01T19:35:49.895 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6832/23586", 2022-08-01T19:35:49.896 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6828/23586", 2022-08-01T19:35:49.896 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:35:49.896 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:35:49.896 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:35:49.897 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:49.897 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.897 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:35:49.897 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_xinfo": [ 2022-08-01T19:35:49.898 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.898 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:35:49.898 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:35:49.898 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:35:49.898 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:35:49.899 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:35:49.899 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:35:49.899 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.588817+0000", 2022-08-01T19:35:49.899 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:35:49.900 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.900 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.900 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:35:49.900 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:35:49.901 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:35:49.901 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:35:49.901 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:35:49.901 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:35:49.901 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.782753+0000", 2022-08-01T19:35:49.902 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:35:49.902 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.902 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.902 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:35:49.903 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:35:49.903 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:35:49.903 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:35:49.903 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:35:49.904 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:35:49.904 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.837924+0000", 2022-08-01T19:35:49.904 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:35:49.904 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.905 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.905 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:35:49.905 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:35:49.905 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:35:49.905 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:35:49.906 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:35:49.906 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:35:49.906 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.839948+0000", 2022-08-01T19:35:49.906 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:35:49.907 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.907 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.907 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:35:49.907 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:35:49.908 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:35:49.908 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:35:49.908 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:35:49.908 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:35:49.908 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.112801+0000", 2022-08-01T19:35:49.909 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:35:49.909 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.909 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.909 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:35:49.910 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:35:49.910 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:35:49.910 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:35:49.910 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:35:49.911 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:35:49.911 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.106704+0000", 2022-08-01T19:35:49.911 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:35:49.911 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.912 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.912 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:35:49.912 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:35:49.912 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:35:49.912 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:35:49.913 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:35:49.913 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:35:49.913 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.893170+0000", 2022-08-01T19:35:49.913 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:35:49.914 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.914 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:49.914 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:35:49.914 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:35:49.915 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:35:49.915 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:35:49.915 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:35:49.915 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:35:49.916 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.886947+0000", 2022-08-01T19:35:49.916 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:35:49.916 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.916 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:35:49.916 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap": [], 2022-08-01T19:35:49.917 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap_items": [], 2022-08-01T19:35:49.917 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_temp": [], 2022-08-01T19:35:49.917 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_temp": [], 2022-08-01T19:35:49.917 INFO:teuthology.orchestra.run.smithi017.stdout: "blacklist": {}, 2022-08-01T19:35:49.918 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profiles": { 2022-08-01T19:35:49.918 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_data": { 2022-08-01T19:35:49.918 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-device-class": "", 2022-08-01T19:35:49.918 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:35:49.919 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-root": "default", 2022-08-01T19:35:49.919 INFO:teuthology.orchestra.run.smithi017.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T19:35:49.919 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:35:49.919 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "2", 2022-08-01T19:35:49.919 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:35:49.920 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:35:49.920 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van", 2022-08-01T19:35:49.920 INFO:teuthology.orchestra.run.smithi017.stdout: "w": "8" 2022-08-01T19:35:49.920 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.921 INFO:teuthology.orchestra.run.smithi017.stdout: "default": { 2022-08-01T19:35:49.921 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:35:49.921 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:35:49.921 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "1", 2022-08-01T19:35:49.922 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:35:49.922 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:35:49.922 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van" 2022-08-01T19:35:49.922 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:49.923 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:49.923 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps_queue": [], 2022-08-01T19:35:49.923 INFO:teuthology.orchestra.run.smithi017.stdout: "new_removed_snaps": [], 2022-08-01T19:35:49.923 INFO:teuthology.orchestra.run.smithi017.stdout: "new_purged_snaps": [], 2022-08-01T19:35:49.923 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_node_flags": {}, 2022-08-01T19:35:49.924 INFO:teuthology.orchestra.run.smithi017.stdout: "device_class_flags": {} 2022-08-01T19:35:49.924 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:35:49.926 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.666+0000 7f322b59e700 1 -- 172.21.15.17:0/801464591 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3210053950 msgr2=0x7f3210055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:49.926 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.666+0000 7f322b59e700 1 --2- 172.21.15.17:0/801464591 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3210053950 0x7f3210055e10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f321c00b000 tx=0x7f321c006df0).stop 2022-08-01T19:35:49.927 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.666+0000 7f322b59e700 1 -- 172.21.15.17:0/801464591 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f322c0b0270 msgr2=0x7f322c156150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:49.927 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.666+0000 7f322b59e700 1 --2- 172.21.15.17:0/801464591 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f322c0b0270 0x7f322c156150 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f3214001110 tx=0x7f32140079e0).stop 2022-08-01T19:35:49.927 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.666+0000 7f322b59e700 1 -- 172.21.15.17:0/801464591 shutdown_connections 2022-08-01T19:35:49.927 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.666+0000 7f322b59e700 1 --2- 172.21.15.17:0/801464591 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3210053950 0x7f3210055e10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:49.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.666+0000 7f322b59e700 1 --2- 172.21.15.17:0/801464591 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f322c0b2710 0x7f322c13d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:49.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.666+0000 7f322b59e700 1 --2- 172.21.15.17:0/801464591 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f322c0b0270 0x7f322c156150 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:49.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.666+0000 7f322b59e700 1 --2- 172.21.15.17:0/801464591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f322c0ac880 0x7f322c1459b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:49.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.666+0000 7f322b59e700 1 -- 172.21.15.17:0/801464591 >> 172.21.15.17:0/801464591 conn(0x7f322c060310 msgr2=0x7f322c0b3230 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:49.929 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.666+0000 7f322b59e700 1 -- 172.21.15.17:0/801464591 shutdown_connections 2022-08-01T19:35:49.929 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:49.666+0000 7f322b59e700 1 -- 172.21.15.17:0/801464591 wait complete. 2022-08-01T19:35:49.930 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:35:50.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.038+0000 7f107aa26700 1 Processor -- start 2022-08-01T19:35:50.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.038+0000 7f107aa26700 1 -- start start 2022-08-01T19:35:50.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.038+0000 7f107aa26700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f1074144190 0x7f107414c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:50.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.038+0000 7f107aa26700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10740aad40 con 0x7f1074067490 2022-08-01T19:35:50.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.038+0000 7f107aa26700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1074060670 con 0x7f1074060290 2022-08-01T19:35:50.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.038+0000 7f107aa26700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1074067030 con 0x7f1074144190 2022-08-01T19:35:50.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.038+0000 7f10737fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1074067490 0x7f1074143a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58358/0 (socket says 172.21.15.17:58358) 2022-08-01T19:35:50.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.038+0000 7f10737fe700 1 -- 172.21.15.17:0/1598794575 learned_addr learned my addr 172.21.15.17:0/1598794575 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:35:50.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.038+0000 7f1078fc3700 1 --2- 172.21.15.17:0/1598794575 >> v2:172.21.15.77:3300/0 conn(0x7f1074144190 0x7f107414c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:50.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.038+0000 7f1072ffd700 1 -- 172.21.15.17:0/1598794575 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2079891264 0 0) 0x7f10740aad40 con 0x7f1074067490 2022-08-01T19:35:50.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.038+0000 7f1072ffd700 1 -- 172.21.15.17:0/1598794575 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1060003260 con 0x7f1074067490 2022-08-01T19:35:50.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.038+0000 7f1078fc3700 1 -- 172.21.15.17:0/1598794575 >> v1:172.21.15.77:6789/0 conn(0x7f1074060290 legacy=0x7f10740b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:50.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.038+0000 7f1078fc3700 1 -- 172.21.15.17:0/1598794575 >> v1:172.21.15.17:6789/0 conn(0x7f1074067490 legacy=0x7f1074143a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:50.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.038+0000 7f1078fc3700 1 -- 172.21.15.17:0/1598794575 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10740b3540 con 0x7f1074144190 2022-08-01T19:35:50.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.038+0000 7f1078fc3700 1 --2- 172.21.15.17:0/1598794575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1074144190 0x7f107414c9f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f106c0032f0 tx=0x7f106c0091d0).ready entity=mon.1 client_cookie=cb237ca1610c0c0b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:50.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.038+0000 7f1072ffd700 1 -- 172.21.15.17:0/1598794575 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f106c00b6c0 con 0x7f1074144190 2022-08-01T19:35:50.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.038+0000 7f1072ffd700 1 -- 172.21.15.17:0/1598794575 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106c00b820 con 0x7f1074144190 2022-08-01T19:35:50.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.038+0000 7f1072ffd700 1 -- 172.21.15.17:0/1598794575 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f106c00b990 con 0x7f1074144190 2022-08-01T19:35:50.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.038+0000 7f107aa26700 1 -- 172.21.15.17:0/1598794575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1074144190 msgr2=0x7f107414c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:50.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.038+0000 7f107aa26700 1 --2- 172.21.15.17:0/1598794575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1074144190 0x7f107414c9f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f106c0032f0 tx=0x7f106c0091d0).stop 2022-08-01T19:35:50.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.038+0000 7f107aa26700 1 -- 172.21.15.17:0/1598794575 shutdown_connections 2022-08-01T19:35:50.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.038+0000 7f107aa26700 1 --2- 172.21.15.17:0/1598794575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1074144190 0x7f107414c9f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:50.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.038+0000 7f107aa26700 1 -- 172.21.15.17:0/1598794575 >> 172.21.15.17:0/1598794575 conn(0x7f107405d0c0 msgr2=0x7f107405f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:50.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.038+0000 7f107aa26700 1 -- 172.21.15.17:0/1598794575 shutdown_connections 2022-08-01T19:35:50.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.042+0000 7f107aa26700 1 -- 172.21.15.17:0/1598794575 wait complete. 2022-08-01T19:35:50.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.042+0000 7f107aa26700 1 Processor -- start 2022-08-01T19:35:50.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.042+0000 7f107aa26700 1 -- start start 2022-08-01T19:35:50.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.042+0000 7f107aa26700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1074060290 0x7f107409ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:50.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.042+0000 7f107aa26700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1074067490 0x7f10740a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:50.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.042+0000 7f107aa26700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1074144190 0x7f10740a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:50.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.042+0000 7f107aa26700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107405f0c0 con 0x7f1074060290 2022-08-01T19:35:50.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.042+0000 7f107aa26700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10740ae360 con 0x7f1074144190 2022-08-01T19:35:50.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.042+0000 7f107aa26700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10740ad610 con 0x7f1074067490 2022-08-01T19:35:50.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.042+0000 7f1073fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1074060290 0x7f107409ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:50.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.042+0000 7f1073fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1074060290 0x7f107409ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37938/0 (socket says 172.21.15.17:37938) 2022-08-01T19:35:50.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.042+0000 7f1073fff700 1 -- 172.21.15.17:0/2980861290 learned_addr learned my addr 172.21.15.17:0/2980861290 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:50.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.042+0000 7f10737fe700 1 --2- 172.21.15.17:0/2980861290 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1074067490 0x7f10740a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:50.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.042+0000 7f1078fc3700 1 --2- 172.21.15.17:0/2980861290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1074144190 0x7f10740a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:50.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.046+0000 7f1073fff700 1 -- 172.21.15.17:0/2980861290 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1074067490 msgr2=0x7f10740a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:50.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.046+0000 7f1073fff700 1 --2- 172.21.15.17:0/2980861290 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1074067490 0x7f10740a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:50.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.046+0000 7f1073fff700 1 -- 172.21.15.17:0/2980861290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1074144190 msgr2=0x7f10740a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:50.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.046+0000 7f1073fff700 1 --2- 172.21.15.17:0/2980861290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1074144190 0x7f10740a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:50.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.046+0000 7f1073fff700 1 -- 172.21.15.17:0/2980861290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106c009470 con 0x7f1074060290 2022-08-01T19:35:50.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.046+0000 7f1073fff700 1 --2- 172.21.15.17:0/2980861290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1074060290 0x7f107409ed80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f107405f3b0 tx=0x7f1064007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:50.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.046+0000 7f10717fa700 1 -- 172.21.15.17:0/2980861290 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1064007030 con 0x7f1074060290 2022-08-01T19:35:50.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.046+0000 7f10717fa700 1 -- 172.21.15.17:0/2980861290 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1064007190 con 0x7f1074060290 2022-08-01T19:35:50.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.046+0000 7f107aa26700 1 -- 172.21.15.17:0/2980861290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10740ad8b0 con 0x7f1074060290 2022-08-01T19:35:50.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.046+0000 7f10717fa700 1 -- 172.21.15.17:0/2980861290 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1064006510 con 0x7f1074060290 2022-08-01T19:35:50.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.046+0000 7f107aa26700 1 -- 172.21.15.17:0/2980861290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10740ab5a0 con 0x7f1074060290 2022-08-01T19:35:50.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.046+0000 7f10717fa700 1 -- 172.21.15.17:0/2980861290 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1064010030 con 0x7f1074060290 2022-08-01T19:35:50.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.046+0000 7f107aa26700 1 -- 172.21.15.17:0/2980861290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10740574e0 con 0x7f1074060290 2022-08-01T19:35:50.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.046+0000 7f10717fa700 1 --2- 172.21.15.17:0/2980861290 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1054053a70 0x7f1054055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:50.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.046+0000 7f10717fa700 1 -- 172.21.15.17:0/2980861290 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f106406bba0 con 0x7f1074060290 2022-08-01T19:35:50.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.046+0000 7f10737fe700 1 --2- 172.21.15.17:0/2980861290 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1054053a70 0x7f1054055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:50.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.046+0000 7f10737fe700 1 --2- 172.21.15.17:0/2980861290 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1054053a70 0x7f1054055f30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f106800d000 tx=0x7f1068009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:50.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.054+0000 7f10717fa700 1 -- 172.21.15.17:0/2980861290 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1064042960 con 0x7f1074060290 2022-08-01T19:35:50.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.214+0000 7f107aa26700 1 -- 172.21.15.17:0/2980861290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10740a0480 con 0x7f1074060290 2022-08-01T19:35:50.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.214+0000 7f10717fa700 1 -- 172.21.15.17:0/2980861290 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+2491 (secure 0 0 0) 0x7f1064045f80 con 0x7f1074060290 2022-08-01T19:35:50.219 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:35:50.219 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2487504686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2487504686},{"type":"v1","addr":"172.21.15.17:6833","nonce":2487504686}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10},{"gid":4885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/4255478434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4255478434},{"type":"v1","addr":"172.21.15.77:6835","nonce":4255478434}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":11}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:34:46.688039+0000","modified":"2022-08-01T19:34:47.690492+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4890},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4890":{"gid":4890,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/3876538769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":3876538769},{"type":"v1","addr":"172.21.15.17:6835","nonce":3876538769}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-08-01T19:35:50.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.214+0000 7f107aa26700 1 -- 172.21.15.17:0/2980861290 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1054053a70 msgr2=0x7f1054055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:50.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.214+0000 7f107aa26700 1 --2- 172.21.15.17:0/2980861290 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1054053a70 0x7f1054055f30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f106800d000 tx=0x7f1068009040).stop 2022-08-01T19:35:50.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.214+0000 7f107aa26700 1 -- 172.21.15.17:0/2980861290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1074060290 msgr2=0x7f107409ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:50.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.214+0000 7f107aa26700 1 --2- 172.21.15.17:0/2980861290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1074060290 0x7f107409ed80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f107405f3b0 tx=0x7f1064007c10).stop 2022-08-01T19:35:50.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.218+0000 7f107aa26700 1 -- 172.21.15.17:0/2980861290 shutdown_connections 2022-08-01T19:35:50.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.218+0000 7f107aa26700 1 --2- 172.21.15.17:0/2980861290 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1054053a70 0x7f1054055f30 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:50.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.218+0000 7f107aa26700 1 --2- 172.21.15.17:0/2980861290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1074144190 0x7f10740a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:50.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.218+0000 7f107aa26700 1 --2- 172.21.15.17:0/2980861290 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1074067490 0x7f10740a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:50.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.218+0000 7f107aa26700 1 --2- 172.21.15.17:0/2980861290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1074060290 0x7f107409ed80 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:50.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.218+0000 7f107aa26700 1 -- 172.21.15.17:0/2980861290 >> 172.21.15.17:0/2980861290 conn(0x7f107405d0c0 msgr2=0x7f1074141e80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:50.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.218+0000 7f107aa26700 1 -- 172.21.15.17:0/2980861290 shutdown_connections 2022-08-01T19:35:50.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.218+0000 7f107aa26700 1 -- 172.21.15.17:0/2980861290 wait complete. 2022-08-01T19:35:50.225 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 13 2022-08-01T19:35:50.233 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-08-01T19:35:50.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f19a0887700 1 Processor -- start 2022-08-01T19:35:50.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f19a0887700 1 -- start start 2022-08-01T19:35:50.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f19a0887700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f199c0b6ee0 0x7f199c0b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:50.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f19a0887700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f199c0b5ed0 0x7f199c0b5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:50.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f19a0887700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f199c061d90 con 0x7f199c0619b0 2022-08-01T19:35:50.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f19a0887700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f199c143fa0 con 0x7f199c0b5ed0 2022-08-01T19:35:50.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f19a0887700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f199c142f60 con 0x7f199c0b6ee0 2022-08-01T19:35:50.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f199b7fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f199c0619b0 0x7f199c143890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58372/0 (socket says 172.21.15.17:58372) 2022-08-01T19:35:50.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f199b7fe700 1 -- 172.21.15.17:0/2776862242 learned_addr learned my addr 172.21.15.17:0/2776862242 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:35:50.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f199affd700 1 --2- 172.21.15.17:0/2776862242 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f199c0b6ee0 0x7f199c0b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:50.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f199bfff700 1 --2- 172.21.15.17:0/2776862242 >> v2:172.21.15.77:3300/0 conn(0x7f199c0b5ed0 0x7f199c0b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:50.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f199a7fc700 1 -- 172.21.15.17:0/2776862242 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3398688418 0 0) 0x7f199c061d90 con 0x7f199c0619b0 2022-08-01T19:35:50.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f199a7fc700 1 -- 172.21.15.17:0/2776862242 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1988003260 con 0x7f199c0619b0 2022-08-01T19:35:50.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f199a7fc700 1 -- 172.21.15.17:0/2776862242 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 661523064 0 0) 0x7f1988003260 con 0x7f199c0619b0 2022-08-01T19:35:50.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f199a7fc700 1 -- 172.21.15.17:0/2776862242 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f199c0b6ee0 msgr2=0x7f199c0b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:50.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f199a7fc700 1 --2- 172.21.15.17:0/2776862242 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f199c0b6ee0 0x7f199c0b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:50.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f199a7fc700 1 -- 172.21.15.17:0/2776862242 >> v2:172.21.15.77:3300/0 conn(0x7f199c0b5ed0 msgr2=0x7f199c0b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:50.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f199a7fc700 1 --2- 172.21.15.17:0/2776862242 >> v2:172.21.15.77:3300/0 conn(0x7f199c0b5ed0 0x7f199c0b5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:50.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f199a7fc700 1 -- 172.21.15.17:0/2776862242 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f199c0ba5a0 con 0x7f199c0619b0 2022-08-01T19:35:50.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f199a7fc700 1 -- 172.21.15.17:0/2776862242 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7f1990001680 con 0x7f199c0619b0 2022-08-01T19:35:50.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f199a7fc700 1 -- 172.21.15.17:0/2776862242 >> v1:172.21.15.17:6789/0 conn(0x7f199c0619b0 legacy=0x7f199c143890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:50.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f199a7fc700 1 --2- 172.21.15.17:0/2776862242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1988003750 0x7f1988003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:50.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f199a7fc700 1 --2- 172.21.15.17:0/2776862242 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f199c0b6ee0 0x7f1988004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:50.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f199a7fc700 1 --2- 172.21.15.17:0/2776862242 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19880049f0 0x7f1988004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:50.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f199a7fc700 1 -- 172.21.15.17:0/2776862242 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199c143fa0 con 0x7f1988003750 2022-08-01T19:35:50.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f199a7fc700 1 -- 172.21.15.17:0/2776862242 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1988004360 con 0x7f19880049f0 2022-08-01T19:35:50.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f199a7fc700 1 -- 172.21.15.17:0/2776862242 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198800b7d0 con 0x7f199c0b6ee0 2022-08-01T19:35:50.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f199b7fe700 1 --2- 172.21.15.17:0/2776862242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1988003750 0x7f1988003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:50.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f199affd700 1 --2- 172.21.15.17:0/2776862242 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f199c0b6ee0 0x7f1988004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:50.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f199bfff700 1 --2- 172.21.15.17:0/2776862242 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19880049f0 0x7f1988004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:50.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f199b7fe700 1 -- 172.21.15.17:0/2776862242 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f199c0b6ee0 msgr2=0x7f1988004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:50.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f199b7fe700 1 --2- 172.21.15.17:0/2776862242 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f199c0b6ee0 0x7f1988004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:50.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f199b7fe700 1 -- 172.21.15.17:0/2776862242 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19880049f0 msgr2=0x7f1988004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:50.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f199b7fe700 1 --2- 172.21.15.17:0/2776862242 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19880049f0 0x7f1988004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:50.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f199b7fe700 1 -- 172.21.15.17:0/2776862242 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1988003260 con 0x7f1988003750 2022-08-01T19:35:50.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f199bfff700 1 --2- 172.21.15.17:0/2776862242 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19880049f0 0x7f1988004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:35:50.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f199b7fe700 1 --2- 172.21.15.17:0/2776862242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1988003750 0x7f1988003b70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f1990001390 tx=0x7f199c067890).ready entity=mon.0 client_cookie=7e5d3ae4bc17ff0f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:50.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f199a7fc700 1 -- 172.21.15.17:0/2776862242 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1990007170 con 0x7f1988003750 2022-08-01T19:35:50.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f199a7fc700 1 -- 172.21.15.17:0/2776862242 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19900072d0 con 0x7f1988003750 2022-08-01T19:35:50.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.334+0000 7f199a7fc700 1 -- 172.21.15.17:0/2776862242 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f199000c9c0 con 0x7f1988003750 2022-08-01T19:35:50.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.338+0000 7f19a0887700 1 -- 172.21.15.17:0/2776862242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1988003750 msgr2=0x7f1988003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:50.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.338+0000 7f19a0887700 1 --2- 172.21.15.17:0/2776862242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1988003750 0x7f1988003b70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f1990001390 tx=0x7f199c067890).stop 2022-08-01T19:35:50.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.338+0000 7f199bfff700 1 -- 172.21.15.17:0/2776862242 reap_dead start 2022-08-01T19:35:50.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.338+0000 7f19a0887700 1 -- 172.21.15.17:0/2776862242 shutdown_connections 2022-08-01T19:35:50.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.338+0000 7f19a0887700 1 -- 172.21.15.17:0/2776862242 >> 172.21.15.17:0/2776862242 conn(0x7f199c06af80 msgr2=0x7f199c05fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:50.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.338+0000 7f19a0887700 1 -- 172.21.15.17:0/2776862242 shutdown_connections 2022-08-01T19:35:50.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.338+0000 7f19a0887700 1 -- 172.21.15.17:0/2776862242 wait complete. 2022-08-01T19:35:50.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.338+0000 7f19a0887700 1 Processor -- start 2022-08-01T19:35:50.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.338+0000 7f19a0887700 1 -- start start 2022-08-01T19:35:50.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.338+0000 7f19a0887700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1988003750 0x7f199c0adb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:50.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.338+0000 7f19a0887700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f199c152f50 0x7f199c145a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:50.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.338+0000 7f19a0887700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f199c145f80 0x7f199c155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:50.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.338+0000 7f19a0887700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199c1468e0 con 0x7f1988003750 2022-08-01T19:35:50.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.338+0000 7f19a0887700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199c145210 con 0x7f199c145f80 2022-08-01T19:35:50.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.338+0000 7f19a0887700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f199c0ae020 con 0x7f199c152f50 2022-08-01T19:35:50.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.338+0000 7f199b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1988003750 0x7f199c0adb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:50.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.338+0000 7f199b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1988003750 0x7f199c0adb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37958/0 (socket says 172.21.15.17:37958) 2022-08-01T19:35:50.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.338+0000 7f199b7fe700 1 -- 172.21.15.17:0/1698265211 learned_addr learned my addr 172.21.15.17:0/1698265211 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:50.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.338+0000 7f199bfff700 1 --2- 172.21.15.17:0/1698265211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f199c145f80 0x7f199c155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:50.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.338+0000 7f199affd700 1 --2- 172.21.15.17:0/1698265211 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f199c152f50 0x7f199c145a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:50.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.338+0000 7f199b7fe700 1 -- 172.21.15.17:0/1698265211 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f199c152f50 msgr2=0x7f199c145a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:50.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.338+0000 7f199b7fe700 1 --2- 172.21.15.17:0/1698265211 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f199c152f50 0x7f199c145a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:50.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.338+0000 7f199b7fe700 1 -- 172.21.15.17:0/1698265211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f199c145f80 msgr2=0x7f199c155a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:50.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.338+0000 7f199b7fe700 1 --2- 172.21.15.17:0/1698265211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f199c145f80 0x7f199c155a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:50.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.338+0000 7f199b7fe700 1 -- 172.21.15.17:0/1698265211 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f199c144470 con 0x7f1988003750 2022-08-01T19:35:50.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.342+0000 7f199b7fe700 1 --2- 172.21.15.17:0/1698265211 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1988003750 0x7f199c0adb30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f199c152550 tx=0x7f1990013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:50.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.342+0000 7f1998ff9700 1 -- 172.21.15.17:0/1698265211 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f199000c6d0 con 0x7f1988003750 2022-08-01T19:35:50.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.342+0000 7f1998ff9700 1 -- 172.21.15.17:0/1698265211 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1990007440 con 0x7f1988003750 2022-08-01T19:35:50.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.342+0000 7f19a0887700 1 -- 172.21.15.17:0/1698265211 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f199c146640 con 0x7f1988003750 2022-08-01T19:35:50.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.342+0000 7f1998ff9700 1 -- 172.21.15.17:0/1698265211 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1990004920 con 0x7f1988003750 2022-08-01T19:35:50.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.342+0000 7f19a0887700 1 -- 172.21.15.17:0/1698265211 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f199c0af8a0 con 0x7f1988003750 2022-08-01T19:35:50.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.342+0000 7f19a0887700 1 -- 172.21.15.17:0/1698265211 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f199c144600 con 0x7f1988003750 2022-08-01T19:35:50.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.342+0000 7f1998ff9700 1 -- 172.21.15.17:0/1698265211 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f199001d030 con 0x7f1988003750 2022-08-01T19:35:50.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.342+0000 7f1998ff9700 1 --2- 172.21.15.17:0/1698265211 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f197c0539e0 0x7f197c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:50.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.342+0000 7f1998ff9700 1 -- 172.21.15.17:0/1698265211 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f199006cba0 con 0x7f1988003750 2022-08-01T19:35:50.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.342+0000 7f199affd700 1 --2- 172.21.15.17:0/1698265211 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f197c0539e0 0x7f197c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:50.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.346+0000 7f199affd700 1 --2- 172.21.15.17:0/1698265211 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f197c0539e0 0x7f197c055ea0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f198c00b000 tx=0x7f198c008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:50.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.350+0000 7f1998ff9700 1 -- 172.21.15.17:0/1698265211 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1990043850 con 0x7f1988003750 2022-08-01T19:35:50.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:50.506+0000 7f19a0887700 1 -- 172.21.15.17:0/1698265211 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f199c0574e0 con 0x7f1988003750 2022-08-01T19:35:51.122 INFO:tasks.ceph.mon.a.smithi017.stderr:2022-08-01T19:35:51.118+0000 7f6a53773700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-08-01T19:35:51.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.130+0000 7f1998ff9700 1 -- 172.21.15.17:0/1698265211 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v14) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1990046e70 con 0x7f1988003750 2022-08-01T19:35:51.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.130+0000 7f19a0887700 1 -- 172.21.15.17:0/1698265211 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f197c0539e0 msgr2=0x7f197c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:51.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.130+0000 7f19a0887700 1 --2- 172.21.15.17:0/1698265211 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f197c0539e0 0x7f197c055ea0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f198c00b000 tx=0x7f198c008040).stop 2022-08-01T19:35:51.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.130+0000 7f19a0887700 1 -- 172.21.15.17:0/1698265211 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1988003750 msgr2=0x7f199c0adb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:51.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.130+0000 7f19a0887700 1 --2- 172.21.15.17:0/1698265211 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1988003750 0x7f199c0adb30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f199c152550 tx=0x7f1990013000).stop 2022-08-01T19:35:51.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.130+0000 7f19a0887700 1 -- 172.21.15.17:0/1698265211 shutdown_connections 2022-08-01T19:35:51.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.130+0000 7f19a0887700 1 --2- 172.21.15.17:0/1698265211 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f197c0539e0 0x7f197c055ea0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:51.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.130+0000 7f19a0887700 1 --2- 172.21.15.17:0/1698265211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f199c145f80 0x7f199c155a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:51.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.130+0000 7f19a0887700 1 --2- 172.21.15.17:0/1698265211 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f199c152f50 0x7f199c145a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:51.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.130+0000 7f19a0887700 1 --2- 172.21.15.17:0/1698265211 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1988003750 0x7f199c0adb30 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:51.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.130+0000 7f19a0887700 1 -- 172.21.15.17:0/1698265211 >> 172.21.15.17:0/1698265211 conn(0x7f199c06af80 msgr2=0x7f199c0b9820 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:51.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.134+0000 7f19a0887700 1 -- 172.21.15.17:0/1698265211 shutdown_connections 2022-08-01T19:35:51.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.134+0000 7f19a0887700 1 -- 172.21.15.17:0/1698265211 wait complete. 2022-08-01T19:35:51.144 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-08-01T19:35:51.152 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:35:51.172 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2022-08-01T19:35:51.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.254+0000 7f94d5e58700 1 Processor -- start 2022-08-01T19:35:51.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.254+0000 7f94d5e58700 1 -- start start 2022-08-01T19:35:51.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.254+0000 7f94d5e58700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f94d00b08a0 0x7f94d0142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:51.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.254+0000 7f94d5e58700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f94d00b3260 0x7f94d01433a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:51.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.254+0000 7f94d5e58700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f94d00ad000 0x7f94d005f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:51.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.254+0000 7f94d5e58700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94d00b0c80 con 0x7f94d00b3260 2022-08-01T19:35:51.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.254+0000 7f94d5e58700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94d00ae810 con 0x7f94d00b08a0 2022-08-01T19:35:51.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.254+0000 7f94d5e58700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d00aeab0 con 0x7f94d00ad000 2022-08-01T19:35:51.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.254+0000 7f94cffff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f94d00b3260 0x7f94d01433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:51.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.254+0000 7f94cffff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f94d00b3260 0x7f94d01433a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37980/0 (socket says 172.21.15.17:37980) 2022-08-01T19:35:51.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.254+0000 7f94cffff700 1 -- 172.21.15.17:0/347712889 learned_addr learned my addr 172.21.15.17:0/347712889 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:51.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.254+0000 7f94d4e56700 1 --2- 172.21.15.17:0/347712889 >> v2:172.21.15.77:3300/0 conn(0x7f94d00b08a0 0x7f94d0142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:51.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.254+0000 7f94d5657700 1 --2- 172.21.15.17:0/347712889 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f94d00ad000 0x7f94d005f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:51.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.258+0000 7f94cffff700 1 -- 172.21.15.17:0/347712889 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f94d00ad000 msgr2=0x7f94d005f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:51.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.258+0000 7f94cffff700 1 --2- 172.21.15.17:0/347712889 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f94d00ad000 0x7f94d005f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:51.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.258+0000 7f94cffff700 1 -- 172.21.15.17:0/347712889 >> v2:172.21.15.77:3300/0 conn(0x7f94d00b08a0 msgr2=0x7f94d0142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:51.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.258+0000 7f94cffff700 1 --2- 172.21.15.17:0/347712889 >> v2:172.21.15.77:3300/0 conn(0x7f94d00b08a0 0x7f94d0142eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:51.266 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.258+0000 7f94cffff700 1 -- 172.21.15.17:0/347712889 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d00aed50 con 0x7f94d00b3260 2022-08-01T19:35:51.266 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.258+0000 7f94cffff700 1 --2- 172.21.15.17:0/347712889 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94d00b3260 0x7f94d01433a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f94c4002f70 tx=0x7f94c4006eb0).ready entity=mon.0 client_cookie=267520b10d96b0e9 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:51.266 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.258+0000 7f94cf7fe700 1 -- 172.21.15.17:0/347712889 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94c400d570 con 0x7f94d00b3260 2022-08-01T19:35:51.266 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.258+0000 7f94cf7fe700 1 -- 172.21.15.17:0/347712889 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c4000e90 con 0x7f94d00b3260 2022-08-01T19:35:51.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.258+0000 7f94cf7fe700 1 -- 172.21.15.17:0/347712889 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94c400d900 con 0x7f94d00b3260 2022-08-01T19:35:51.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.258+0000 7f94d5e58700 1 -- 172.21.15.17:0/347712889 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94d00b3260 msgr2=0x7f94d01433a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:51.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.258+0000 7f94d5e58700 1 --2- 172.21.15.17:0/347712889 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94d00b3260 0x7f94d01433a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f94c4002f70 tx=0x7f94c4006eb0).stop 2022-08-01T19:35:51.271 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.258+0000 7f94d5e58700 1 -- 172.21.15.17:0/347712889 shutdown_connections 2022-08-01T19:35:51.271 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.258+0000 7f94d5e58700 1 --2- 172.21.15.17:0/347712889 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f94d00ad000 0x7f94d005f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:51.271 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.258+0000 7f94d5e58700 1 --2- 172.21.15.17:0/347712889 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94d00b3260 0x7f94d01433a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:51.271 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.258+0000 7f94d5e58700 1 --2- 172.21.15.17:0/347712889 >> v2:172.21.15.77:3300/0 conn(0x7f94d00b08a0 0x7f94d0142eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:51.272 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.258+0000 7f94d5e58700 1 -- 172.21.15.17:0/347712889 >> 172.21.15.17:0/347712889 conn(0x7f94d00612d0 msgr2=0x7f94d0062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:51.272 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.258+0000 7f94d5e58700 1 -- 172.21.15.17:0/347712889 shutdown_connections 2022-08-01T19:35:51.272 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.258+0000 7f94d5e58700 1 -- 172.21.15.17:0/347712889 wait complete. 2022-08-01T19:35:51.272 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.262+0000 7f94d5e58700 1 Processor -- start 2022-08-01T19:35:51.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.262+0000 7f94d5e58700 1 -- start start 2022-08-01T19:35:51.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.262+0000 7f94d5e58700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94d00ad000 0x7f94d00abf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:51.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.262+0000 7f94d5e58700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f94d00b08a0 0x7f94d0136d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:51.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.262+0000 7f94d5e58700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94d00b3260 0x7f94d01371f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:51.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.262+0000 7f94d5e58700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d01549d0 con 0x7f94d00b3260 2022-08-01T19:35:51.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.262+0000 7f94d5e58700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d0065100 con 0x7f94d00ad000 2022-08-01T19:35:51.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.262+0000 7f94d5e58700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d00653a0 con 0x7f94d00b08a0 2022-08-01T19:35:51.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.262+0000 7f94d5657700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94d00b3260 0x7f94d01371f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:51.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.262+0000 7f94d5657700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94d00b3260 0x7f94d01371f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37988/0 (socket says 172.21.15.17:37988) 2022-08-01T19:35:51.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.262+0000 7f94d4e56700 1 --2- 172.21.15.17:0/3653603346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94d00ad000 0x7f94d00abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:51.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.262+0000 7f94d5657700 1 -- 172.21.15.17:0/3653603346 learned_addr learned my addr 172.21.15.17:0/3653603346 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:51.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.262+0000 7f94cffff700 1 --2- 172.21.15.17:0/3653603346 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f94d00b08a0 0x7f94d0136d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:51.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.262+0000 7f94d5657700 1 -- 172.21.15.17:0/3653603346 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f94d00b08a0 msgr2=0x7f94d0136d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:51.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.262+0000 7f94d5657700 1 --2- 172.21.15.17:0/3653603346 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f94d00b08a0 0x7f94d0136d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:51.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.262+0000 7f94d5657700 1 -- 172.21.15.17:0/3653603346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94d00ad000 msgr2=0x7f94d00abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:51.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.262+0000 7f94d5657700 1 --2- 172.21.15.17:0/3653603346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94d00ad000 0x7f94d00abf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:51.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.262+0000 7f94d5657700 1 -- 172.21.15.17:0/3653603346 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c400b1e0 con 0x7f94d00b3260 2022-08-01T19:35:51.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.262+0000 7f94d4e56700 1 --2- 172.21.15.17:0/3653603346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94d00ad000 0x7f94d00abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:35:51.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.262+0000 7f94d5657700 1 --2- 172.21.15.17:0/3653603346 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94d00b3260 0x7f94d01371f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f94c8006e00 tx=0x7f94c80091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:51.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.262+0000 7f94cdffb700 1 -- 172.21.15.17:0/3653603346 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94c800eeb0 con 0x7f94d00b3260 2022-08-01T19:35:51.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.262+0000 7f94cdffb700 1 -- 172.21.15.17:0/3653603346 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c800c850 con 0x7f94d00b3260 2022-08-01T19:35:51.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.262+0000 7f94cdffb700 1 -- 172.21.15.17:0/3653603346 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94c800cbb0 con 0x7f94d00b3260 2022-08-01T19:35:51.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.262+0000 7f94d5e58700 1 -- 172.21.15.17:0/3653603346 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d0065640 con 0x7f94d00b3260 2022-08-01T19:35:51.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.262+0000 7f94d5e58700 1 -- 172.21.15.17:0/3653603346 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d0064750 con 0x7f94d00b3260 2022-08-01T19:35:51.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.266+0000 7f94d5e58700 1 -- 172.21.15.17:0/3653603346 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d00657d0 con 0x7f94d00b3260 2022-08-01T19:35:51.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.266+0000 7f94cdffb700 1 -- 172.21.15.17:0/3653603346 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f94c800cd10 con 0x7f94d00b3260 2022-08-01T19:35:51.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.266+0000 7f94cdffb700 1 --2- 172.21.15.17:0/3653603346 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f94b00539e0 0x7f94b0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:51.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.266+0000 7f94cdffb700 1 -- 172.21.15.17:0/3653603346 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f94c806c550 con 0x7f94d00b3260 2022-08-01T19:35:51.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.266+0000 7f94d4e56700 1 --2- 172.21.15.17:0/3653603346 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f94b00539e0 0x7f94b0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:51.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.266+0000 7f94d4e56700 1 --2- 172.21.15.17:0/3653603346 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f94b00539e0 0x7f94b0055ea0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f94c000b000 tx=0x7f94c0006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:51.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.278+0000 7f94cdffb700 1 -- 172.21.15.17:0/3653603346 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f94c8043130 con 0x7f94d00b3260 2022-08-01T19:35:51.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.434+0000 7f94d5e58700 1 -- 172.21.15.17:0/3653603346 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94d00574e0 con 0x7f94d00b3260 2022-08-01T19:35:51.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.434+0000 7f94cdffb700 1 -- 172.21.15.17:0/3653603346 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+2130 (secure 0 0 0) 0x7f94c8046750 con 0x7f94d00b3260 2022-08-01T19:35:51.441 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:35:51.441 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2487504686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2487504686},{"type":"v1","addr":"172.21.15.17:6833","nonce":2487504686}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10},{"gid":4885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/4255478434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4255478434},{"type":"v1","addr":"172.21.15.77:6835","nonce":4255478434}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:34:46.688039+0000","modified":"2022-08-01T19:35:51.123851+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-08-01T19:35:51.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.438+0000 7f94d5e58700 1 -- 172.21.15.17:0/3653603346 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f94b00539e0 msgr2=0x7f94b0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:51.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.438+0000 7f94d5e58700 1 --2- 172.21.15.17:0/3653603346 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f94b00539e0 0x7f94b0055ea0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f94c000b000 tx=0x7f94c0006df0).stop 2022-08-01T19:35:51.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.438+0000 7f94d5e58700 1 -- 172.21.15.17:0/3653603346 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94d00b3260 msgr2=0x7f94d01371f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:51.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.438+0000 7f94d5e58700 1 --2- 172.21.15.17:0/3653603346 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94d00b3260 0x7f94d01371f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f94c8006e00 tx=0x7f94c80091d0).stop 2022-08-01T19:35:51.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.438+0000 7f94d5e58700 1 -- 172.21.15.17:0/3653603346 shutdown_connections 2022-08-01T19:35:51.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.438+0000 7f94d5e58700 1 --2- 172.21.15.17:0/3653603346 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f94b00539e0 0x7f94b0055ea0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:51.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.438+0000 7f94d5e58700 1 --2- 172.21.15.17:0/3653603346 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94d00b3260 0x7f94d01371f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:51.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.438+0000 7f94d5e58700 1 --2- 172.21.15.17:0/3653603346 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f94d00b08a0 0x7f94d0136d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:51.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.438+0000 7f94d5e58700 1 --2- 172.21.15.17:0/3653603346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94d00ad000 0x7f94d00abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:51.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.438+0000 7f94d5e58700 1 -- 172.21.15.17:0/3653603346 >> 172.21.15.17:0/3653603346 conn(0x7f94d00612d0 msgr2=0x7f94d013b710 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:51.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.438+0000 7f94d5e58700 1 -- 172.21.15.17:0/3653603346 shutdown_connections 2022-08-01T19:35:51.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.438+0000 7f94d5e58700 1 -- 172.21.15.17:0/3653603346 wait complete. 2022-08-01T19:35:51.446 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 14 2022-08-01T19:35:51.454 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-08-01T19:35:51.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.602+0000 7fb8fe728700 1 Processor -- start 2022-08-01T19:35:51.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.602+0000 7fb8fe728700 1 -- start start 2022-08-01T19:35:51.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.602+0000 7fb8fe728700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb8f80b2710 0x7fb8f814afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:51.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.602+0000 7fb8fe728700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb8f80b01a0 0x7fb8f814b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:51.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.602+0000 7fb8fe728700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8f80a7250 con 0x7fb8f80afd60 2022-08-01T19:35:51.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.602+0000 7fb8fe728700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8f80a7480 con 0x7fb8f80b2710 2022-08-01T19:35:51.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.602+0000 7fb8fe728700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f80a7720 con 0x7fb8f80b01a0 2022-08-01T19:35:51.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.602+0000 7fb8fd726700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb8f80b2710 0x7fb8f814afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:51.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.602+0000 7fb8fd726700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb8f80b2710 0x7fb8f814afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37992/0 (socket says 172.21.15.17:37992) 2022-08-01T19:35:51.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.602+0000 7fb8fdf27700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb8f80afd60 0x7fb8f80a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58420/0 (socket says 172.21.15.17:58420) 2022-08-01T19:35:51.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.602+0000 7fb8fd726700 1 -- 172.21.15.17:0/2285459060 learned_addr learned my addr 172.21.15.17:0/2285459060 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:51.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.602+0000 7fb8fcf25700 1 --2- 172.21.15.17:0/2285459060 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb8f80b01a0 0x7fb8f814b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:51.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.602+0000 7fb8fd726700 1 -- 172.21.15.17:0/2285459060 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb8f80b01a0 msgr2=0x7fb8f814b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:51.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.602+0000 7fb8fd726700 1 --2- 172.21.15.17:0/2285459060 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb8f80b01a0 0x7fb8f814b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:51.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.602+0000 7fb8fd726700 1 -- 172.21.15.17:0/2285459060 >> v1:172.21.15.17:6789/0 conn(0x7fb8f80afd60 legacy=0x7fb8f80a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:51.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.602+0000 7fb8fd726700 1 -- 172.21.15.17:0/2285459060 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f80a79c0 con 0x7fb8f80b2710 2022-08-01T19:35:51.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.602+0000 7fb8fcf25700 1 --2- 172.21.15.17:0/2285459060 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb8f80b01a0 0x7fb8f814b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:35:51.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.606+0000 7fb8fd726700 1 --2- 172.21.15.17:0/2285459060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8f80b2710 0x7fb8f814afd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7fb8f00032f0 tx=0x7fb8f0009290).ready entity=mon.0 client_cookie=ab1f81d8168fcd28 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:51.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.606+0000 7fb8f7fff700 1 -- 172.21.15.17:0/2285459060 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8f000b8a0 con 0x7fb8f80b2710 2022-08-01T19:35:51.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.606+0000 7fb8f7fff700 1 -- 172.21.15.17:0/2285459060 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f000ba00 con 0x7fb8f80b2710 2022-08-01T19:35:51.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.606+0000 7fb8f7fff700 1 -- 172.21.15.17:0/2285459060 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8f000bb70 con 0x7fb8f80b2710 2022-08-01T19:35:51.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.606+0000 7fb8fe728700 1 -- 172.21.15.17:0/2285459060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8f80b2710 msgr2=0x7fb8f814afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:51.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.606+0000 7fb8fe728700 1 --2- 172.21.15.17:0/2285459060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8f80b2710 0x7fb8f814afd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7fb8f00032f0 tx=0x7fb8f0009290).stop 2022-08-01T19:35:51.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.606+0000 7fb8fe728700 1 -- 172.21.15.17:0/2285459060 shutdown_connections 2022-08-01T19:35:51.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.606+0000 7fb8fe728700 1 --2- 172.21.15.17:0/2285459060 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb8f80b01a0 0x7fb8f814b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:51.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.606+0000 7fb8fe728700 1 --2- 172.21.15.17:0/2285459060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8f80b2710 0x7fb8f814afd0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:51.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.606+0000 7fb8fe728700 1 -- 172.21.15.17:0/2285459060 >> 172.21.15.17:0/2285459060 conn(0x7fb8f8060310 msgr2=0x7fb8f80b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:51.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.606+0000 7fb8fe728700 1 -- 172.21.15.17:0/2285459060 shutdown_connections 2022-08-01T19:35:51.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.606+0000 7fb8fe728700 1 -- 172.21.15.17:0/2285459060 wait complete. 2022-08-01T19:35:51.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.606+0000 7fb8fe728700 1 Processor -- start 2022-08-01T19:35:51.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.606+0000 7fb8fe728700 1 -- start start 2022-08-01T19:35:51.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.606+0000 7fb8fe728700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb8f80afd60 0x7fb8f80af540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:51.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.606+0000 7fb8fe728700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb8f80b01a0 0x7fb8f8156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:51.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.606+0000 7fb8fe728700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8f80b2710 0x7fb8f81571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:51.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.606+0000 7fb8fe728700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f80aa3e0 con 0x7fb8f80b2710 2022-08-01T19:35:51.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.610+0000 7fb8fe728700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f8156850 con 0x7fb8f80afd60 2022-08-01T19:35:51.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.610+0000 7fb8fe728700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f80b32a0 con 0x7fb8f80b01a0 2022-08-01T19:35:51.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.610+0000 7fb8fdf27700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8f80b2710 0x7fb8f81571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:51.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.610+0000 7fb8fdf27700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8f80b2710 0x7fb8f81571a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38002/0 (socket says 172.21.15.17:38002) 2022-08-01T19:35:51.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.610+0000 7fb8fdf27700 1 -- 172.21.15.17:0/4048956810 learned_addr learned my addr 172.21.15.17:0/4048956810 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:51.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.610+0000 7fb8fd726700 1 --2- 172.21.15.17:0/4048956810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb8f80afd60 0x7fb8f80af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:51.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.610+0000 7fb8fcf25700 1 --2- 172.21.15.17:0/4048956810 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb8f80b01a0 0x7fb8f8156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:51.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.610+0000 7fb8fdf27700 1 -- 172.21.15.17:0/4048956810 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb8f80b01a0 msgr2=0x7fb8f8156cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:51.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.610+0000 7fb8fdf27700 1 --2- 172.21.15.17:0/4048956810 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb8f80b01a0 0x7fb8f8156cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:51.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.610+0000 7fb8fdf27700 1 -- 172.21.15.17:0/4048956810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb8f80afd60 msgr2=0x7fb8f80af540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:51.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.610+0000 7fb8fdf27700 1 --2- 172.21.15.17:0/4048956810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb8f80afd60 0x7fb8f80af540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:51.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.610+0000 7fb8fdf27700 1 -- 172.21.15.17:0/4048956810 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f0009530 con 0x7fb8f80b2710 2022-08-01T19:35:51.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.610+0000 7fb8fcf25700 1 --2- 172.21.15.17:0/4048956810 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb8f80b01a0 0x7fb8f8156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:35:51.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.610+0000 7fb8fdf27700 1 --2- 172.21.15.17:0/4048956810 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8f80b2710 0x7fb8f81571a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7fb8f8061c10 tx=0x7fb8e8007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:51.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.610+0000 7fb8f67fc700 1 -- 172.21.15.17:0/4048956810 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8e80070d0 con 0x7fb8f80b2710 2022-08-01T19:35:51.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.610+0000 7fb8f67fc700 1 -- 172.21.15.17:0/4048956810 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e8007230 con 0x7fb8f80b2710 2022-08-01T19:35:51.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.610+0000 7fb8fe728700 1 -- 172.21.15.17:0/4048956810 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f814d990 con 0x7fb8f80b2710 2022-08-01T19:35:51.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.610+0000 7fb8f67fc700 1 -- 172.21.15.17:0/4048956810 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8e8005740 con 0x7fb8f80b2710 2022-08-01T19:35:51.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.610+0000 7fb8fe728700 1 -- 172.21.15.17:0/4048956810 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f814d6c0 con 0x7fb8f80b2710 2022-08-01T19:35:51.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.610+0000 7fb8f67fc700 1 -- 172.21.15.17:0/4048956810 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb8e8010070 con 0x7fb8f80b2710 2022-08-01T19:35:51.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.610+0000 7fb8fe728700 1 -- 172.21.15.17:0/4048956810 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8f81569c0 con 0x7fb8f80b2710 2022-08-01T19:35:51.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.610+0000 7fb8f67fc700 1 --2- 172.21.15.17:0/4048956810 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb8d8053b00 0x7fb8d8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:51.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.610+0000 7fb8f67fc700 1 -- 172.21.15.17:0/4048956810 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fb8e806b5e0 con 0x7fb8f80b2710 2022-08-01T19:35:51.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.610+0000 7fb8fd726700 1 --2- 172.21.15.17:0/4048956810 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb8d8053b00 0x7fb8d8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:51.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.614+0000 7fb8fd726700 1 --2- 172.21.15.17:0/4048956810 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb8d8053b00 0x7fb8d8055fc0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fb8f001a000 tx=0x7fb8f0016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:51.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.614+0000 7fb8f67fc700 1 -- 172.21.15.17:0/4048956810 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb8e8042350 con 0x7fb8f80b2710 2022-08-01T19:35:51.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:51.778+0000 7fb8fe728700 1 -- 172.21.15.17:0/4048956810 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb8f8159e20 con 0x7fb8f80b2710 2022-08-01T19:35:52.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.138+0000 7fb8f67fc700 1 -- 172.21.15.17:0/4048956810 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb8e8045970 con 0x7fb8f80b2710 2022-08-01T19:35:52.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.138+0000 7fb8fe728700 1 -- 172.21.15.17:0/4048956810 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb8d8053b00 msgr2=0x7fb8d8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:52.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.138+0000 7fb8fe728700 1 --2- 172.21.15.17:0/4048956810 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb8d8053b00 0x7fb8d8055fc0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fb8f001a000 tx=0x7fb8f0016000).stop 2022-08-01T19:35:52.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.138+0000 7fb8fe728700 1 -- 172.21.15.17:0/4048956810 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8f80b2710 msgr2=0x7fb8f81571a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:52.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.138+0000 7fb8fe728700 1 --2- 172.21.15.17:0/4048956810 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8f80b2710 0x7fb8f81571a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7fb8f8061c10 tx=0x7fb8e8007bc0).stop 2022-08-01T19:35:52.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.142+0000 7fb8fe728700 1 -- 172.21.15.17:0/4048956810 shutdown_connections 2022-08-01T19:35:52.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.142+0000 7fb8fe728700 1 --2- 172.21.15.17:0/4048956810 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb8d8053b00 0x7fb8d8055fc0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:52.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.142+0000 7fb8fe728700 1 --2- 172.21.15.17:0/4048956810 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8f80b2710 0x7fb8f81571a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:52.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.142+0000 7fb8fe728700 1 --2- 172.21.15.17:0/4048956810 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb8f80b01a0 0x7fb8f8156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:52.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.142+0000 7fb8fe728700 1 --2- 172.21.15.17:0/4048956810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb8f80afd60 0x7fb8f80af540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:52.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.142+0000 7fb8fe728700 1 -- 172.21.15.17:0/4048956810 >> 172.21.15.17:0/4048956810 conn(0x7fb8f8060310 msgr2=0x7fb8f814a090 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:52.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.142+0000 7fb8fe728700 1 -- 172.21.15.17:0/4048956810 shutdown_connections 2022-08-01T19:35:52.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.142+0000 7fb8fe728700 1 -- 172.21.15.17:0/4048956810 wait complete. 2022-08-01T19:35:52.162 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-08-01T19:35:52.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.278+0000 7f98dd56c700 1 Processor -- start 2022-08-01T19:35:52.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.278+0000 7f98dd56c700 1 -- start start 2022-08-01T19:35:52.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.278+0000 7f98dd56c700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f98d80b41f0 0x7f98d8061d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:52.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.278+0000 7f98dd56c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98d8142ff0 0x7f98d81434a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:52.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.278+0000 7f98dd56c700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98d80b60e0 con 0x7f98d80b6420 2022-08-01T19:35:52.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.278+0000 7f98dd56c700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98d80b2100 con 0x7f98d80b41f0 2022-08-01T19:35:52.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.278+0000 7f98dd56c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d80b23a0 con 0x7f98d8142ff0 2022-08-01T19:35:52.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.278+0000 7f98dcd6b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98d8142ff0 0x7f98d81434a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:52.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.278+0000 7f98d77fe700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f98d80b41f0 0x7f98d8061d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:52.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.278+0000 7f98dcd6b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98d8142ff0 0x7f98d81434a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:34110/0 (socket says 172.21.15.17:34110) 2022-08-01T19:35:52.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.278+0000 7f98d77fe700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f98d80b41f0 0x7f98d8061d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:35932/0 (socket says 172.21.15.17:35932) 2022-08-01T19:35:52.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.278+0000 7f98dcd6b700 1 -- 172.21.15.17:0/739428168 learned_addr learned my addr 172.21.15.17:0/739428168 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:52.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.278+0000 7f98d7fff700 1 --1- 172.21.15.17:0/739428168 >> v1:172.21.15.77:6789/0 conn(0x7f98d80b6420 0x7f98d8061600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:47002/0 (socket says 172.21.15.17:47002) 2022-08-01T19:35:52.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.278+0000 7f98dcd6b700 1 -- 172.21.15.17:0/739428168 >> v2:172.21.15.77:3300/0 conn(0x7f98d80b41f0 msgr2=0x7f98d8061d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:52.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.278+0000 7f98dcd6b700 1 --2- 172.21.15.17:0/739428168 >> v2:172.21.15.77:3300/0 conn(0x7f98d80b41f0 0x7f98d8061d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:52.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.278+0000 7f98dcd6b700 1 -- 172.21.15.17:0/739428168 >> v1:172.21.15.77:6789/0 conn(0x7f98d80b6420 legacy=0x7f98d8061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:52.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.278+0000 7f98dcd6b700 1 -- 172.21.15.17:0/739428168 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d80b2640 con 0x7f98d8142ff0 2022-08-01T19:35:52.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.278+0000 7f98dcd6b700 1 --2- 172.21.15.17:0/739428168 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98d8142ff0 0x7f98d81434a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f98d0002f70 tx=0x7f98d0006eb0).ready entity=mon.2 client_cookie=125dd0b2d0ea413c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:52.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.278+0000 7f98d6ffd700 1 -- 172.21.15.17:0/739428168 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98d000d570 con 0x7f98d8142ff0 2022-08-01T19:35:52.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.278+0000 7f98d6ffd700 1 -- 172.21.15.17:0/739428168 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d0000e90 con 0x7f98d8142ff0 2022-08-01T19:35:52.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.278+0000 7f98d6ffd700 1 -- 172.21.15.17:0/739428168 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98d000d900 con 0x7f98d8142ff0 2022-08-01T19:35:52.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.278+0000 7f98dd56c700 1 -- 172.21.15.17:0/739428168 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98d8142ff0 msgr2=0x7f98d81434a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:52.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.278+0000 7f98dd56c700 1 --2- 172.21.15.17:0/739428168 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98d8142ff0 0x7f98d81434a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f98d0002f70 tx=0x7f98d0006eb0).stop 2022-08-01T19:35:52.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.278+0000 7f98dd56c700 1 -- 172.21.15.17:0/739428168 shutdown_connections 2022-08-01T19:35:52.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.278+0000 7f98dd56c700 1 --2- 172.21.15.17:0/739428168 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98d8142ff0 0x7f98d81434a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:52.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.278+0000 7f98dd56c700 1 --2- 172.21.15.17:0/739428168 >> v2:172.21.15.77:3300/0 conn(0x7f98d80b41f0 0x7f98d8061d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:52.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.278+0000 7f98dd56c700 1 -- 172.21.15.17:0/739428168 >> 172.21.15.17:0/739428168 conn(0x7f98d805c620 msgr2=0x7f98d805ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:52.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.278+0000 7f98dd56c700 1 -- 172.21.15.17:0/739428168 shutdown_connections 2022-08-01T19:35:52.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.278+0000 7f98dd56c700 1 -- 172.21.15.17:0/739428168 wait complete. 2022-08-01T19:35:52.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.278+0000 7f98dd56c700 1 Processor -- start 2022-08-01T19:35:52.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.278+0000 7f98dd56c700 1 -- start start 2022-08-01T19:35:52.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.282+0000 7f98dd56c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98d80b41f0 0x7f98d8152820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:52.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.282+0000 7f98dd56c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98d80b6420 0x7f98d80b7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:52.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.282+0000 7f98dd56c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98d8142ff0 0x7f98d80b0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:52.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.282+0000 7f98dd56c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d80b0650 con 0x7f98d80b41f0 2022-08-01T19:35:52.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.282+0000 7f98dd56c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d80b08e0 con 0x7f98d80b6420 2022-08-01T19:35:52.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.282+0000 7f98dd56c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d80b0b80 con 0x7f98d8142ff0 2022-08-01T19:35:52.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.282+0000 7f98d7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98d80b41f0 0x7f98d8152820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:52.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.282+0000 7f98d77fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98d80b6420 0x7f98d80b7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:52.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.282+0000 7f98dcd6b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98d8142ff0 0x7f98d80b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:52.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.282+0000 7f98d77fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98d80b6420 0x7f98d80b7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:35936/0 (socket says 172.21.15.17:35936) 2022-08-01T19:35:52.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.282+0000 7f98d77fe700 1 -- 172.21.15.17:0/1214766939 learned_addr learned my addr 172.21.15.17:0/1214766939 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:52.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.282+0000 7f98d7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98d80b41f0 0x7f98d8152820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38016/0 (socket says 172.21.15.17:38016) 2022-08-01T19:35:52.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.282+0000 7f98d77fe700 1 -- 172.21.15.17:0/1214766939 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98d8142ff0 msgr2=0x7f98d80b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:52.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.282+0000 7f98d77fe700 1 --2- 172.21.15.17:0/1214766939 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98d8142ff0 0x7f98d80b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:52.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.282+0000 7f98d77fe700 1 -- 172.21.15.17:0/1214766939 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98d80b41f0 msgr2=0x7f98d8152820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:52.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.282+0000 7f98d77fe700 1 --2- 172.21.15.17:0/1214766939 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98d80b41f0 0x7f98d8152820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:52.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.282+0000 7f98d77fe700 1 -- 172.21.15.17:0/1214766939 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d000b1e0 con 0x7f98d80b6420 2022-08-01T19:35:52.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.282+0000 7f98dcd6b700 1 --2- 172.21.15.17:0/1214766939 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98d8142ff0 0x7f98d80b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:35:52.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.282+0000 7f98d7fff700 1 --2- 172.21.15.17:0/1214766939 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98d80b41f0 0x7f98d8152820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:35:52.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.282+0000 7f98d77fe700 1 --2- 172.21.15.17:0/1214766939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98d80b6420 0x7f98d80b7f00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f98c800c000 tx=0x7f98c80091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:52.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.282+0000 7f98d57fa700 1 -- 172.21.15.17:0/1214766939 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98c800c670 con 0x7f98d80b6420 2022-08-01T19:35:52.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.282+0000 7f98d57fa700 1 -- 172.21.15.17:0/1214766939 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c800c7d0 con 0x7f98d80b6420 2022-08-01T19:35:52.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.282+0000 7f98dd56c700 1 -- 172.21.15.17:0/1214766939 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98d8155df0 con 0x7f98d80b6420 2022-08-01T19:35:52.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.282+0000 7f98d57fa700 1 -- 172.21.15.17:0/1214766939 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98c800cb30 con 0x7f98d80b6420 2022-08-01T19:35:52.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.282+0000 7f98dd56c700 1 -- 172.21.15.17:0/1214766939 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98d8140410 con 0x7f98d80b6420 2022-08-01T19:35:52.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.282+0000 7f98dd56c700 1 -- 172.21.15.17:0/1214766939 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98d80b28e0 con 0x7f98d80b6420 2022-08-01T19:35:52.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.286+0000 7f98d57fa700 1 -- 172.21.15.17:0/1214766939 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f98c8018020 con 0x7f98d80b6420 2022-08-01T19:35:52.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.286+0000 7f98d57fa700 1 --2- 172.21.15.17:0/1214766939 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f98b8053a70 0x7f98b8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:52.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.286+0000 7f98d57fa700 1 -- 172.21.15.17:0/1214766939 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f98c806c5f0 con 0x7f98d80b6420 2022-08-01T19:35:52.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.286+0000 7f98d7fff700 1 --2- 172.21.15.17:0/1214766939 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f98b8053a70 0x7f98b8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:52.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.286+0000 7f98d7fff700 1 --2- 172.21.15.17:0/1214766939 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f98b8053a70 0x7f98b8055f30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f98cc00d000 tx=0x7f98cc007b80).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:52.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.298+0000 7f98d57fa700 1 -- 172.21.15.17:0/1214766939 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f98c8043300 con 0x7f98d80b6420 2022-08-01T19:35:52.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:52.422+0000 7f98dd56c700 1 -- 172.21.15.17:0/1214766939 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f98d80574e0 con 0x7f98d80b6420 2022-08-01T19:35:53.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.146+0000 7f98d57fa700 1 -- 172.21.15.17:0/1214766939 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v37) v1 ==== 185+0+0 (secure 0 0 0) 0x7f98c8046920 con 0x7f98d80b6420 2022-08-01T19:35:53.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.150+0000 7f98dd56c700 1 -- 172.21.15.17:0/1214766939 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f98b8053a70 msgr2=0x7f98b8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:53.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.150+0000 7f98dd56c700 1 --2- 172.21.15.17:0/1214766939 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f98b8053a70 0x7f98b8055f30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f98cc00d000 tx=0x7f98cc007b80).stop 2022-08-01T19:35:53.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.150+0000 7f98dd56c700 1 -- 172.21.15.17:0/1214766939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98d80b6420 msgr2=0x7f98d80b7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:53.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.150+0000 7f98dd56c700 1 --2- 172.21.15.17:0/1214766939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98d80b6420 0x7f98d80b7f00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f98c800c000 tx=0x7f98c80091d0).stop 2022-08-01T19:35:53.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.150+0000 7f98dd56c700 1 -- 172.21.15.17:0/1214766939 shutdown_connections 2022-08-01T19:35:53.158 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.150+0000 7f98dd56c700 1 --2- 172.21.15.17:0/1214766939 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f98b8053a70 0x7f98b8055f30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:53.158 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.150+0000 7f98dd56c700 1 --2- 172.21.15.17:0/1214766939 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98d8142ff0 0x7f98d80b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:53.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.150+0000 7f98dd56c700 1 --2- 172.21.15.17:0/1214766939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98d80b6420 0x7f98d80b7f00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:53.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.150+0000 7f98dd56c700 1 --2- 172.21.15.17:0/1214766939 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98d80b41f0 0x7f98d8152820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:53.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.150+0000 7f98dd56c700 1 -- 172.21.15.17:0/1214766939 >> 172.21.15.17:0/1214766939 conn(0x7f98d805c620 msgr2=0x7f98d80b8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:53.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.150+0000 7f98dd56c700 1 -- 172.21.15.17:0/1214766939 shutdown_connections 2022-08-01T19:35:53.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.150+0000 7f98dd56c700 1 -- 172.21.15.17:0/1214766939 wait complete. 2022-08-01T19:35:53.162 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' removed 2022-08-01T19:35:53.171 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-08-01T19:35:53.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.282+0000 7eff5af64700 1 Processor -- start 2022-08-01T19:35:53.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.282+0000 7eff5af64700 1 -- start start 2022-08-01T19:35:53.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.282+0000 7eff5af64700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7eff54060290 0x7eff540b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:53.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.282+0000 7eff5af64700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7eff54067490 0x7eff54143a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:53.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.286+0000 7eff5af64700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff540aad40 con 0x7eff540b4dd0 2022-08-01T19:35:53.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.286+0000 7eff5af64700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff54060670 con 0x7eff54067490 2022-08-01T19:35:53.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.286+0000 7eff5af64700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff54067030 con 0x7eff54060290 2022-08-01T19:35:53.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.286+0000 7eff58d00700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7eff54060290 0x7eff540b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:53.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.286+0000 7eff53fff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7eff54067490 0x7eff54143a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:53.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.286+0000 7eff59501700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7eff540b4dd0 0x7eff541446a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:47018/0 (socket says 172.21.15.17:47018) 2022-08-01T19:35:53.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.286+0000 7eff58d00700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7eff54060290 0x7eff540b48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:34122/0 (socket says 172.21.15.17:34122) 2022-08-01T19:35:53.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.286+0000 7eff53fff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7eff54067490 0x7eff54143a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:35946/0 (socket says 172.21.15.17:35946) 2022-08-01T19:35:53.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.286+0000 7eff59501700 1 -- 172.21.15.17:0/67251985 learned_addr learned my addr 172.21.15.17:0/67251985 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:35:53.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.286+0000 7eff53fff700 1 -- 172.21.15.17:0/67251985 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7eff54060290 msgr2=0x7eff540b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:53.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.286+0000 7eff53fff700 1 --2- 172.21.15.17:0/67251985 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7eff54060290 0x7eff540b48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:53.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.286+0000 7eff53fff700 1 -- 172.21.15.17:0/67251985 >> v1:172.21.15.77:6789/0 conn(0x7eff540b4dd0 legacy=0x7eff541446a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:53.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.286+0000 7eff53fff700 1 -- 172.21.15.17:0/67251985 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff540b3540 con 0x7eff54067490 2022-08-01T19:35:53.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.286+0000 7eff58d00700 1 --2- 172.21.15.17:0/67251985 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7eff54060290 0x7eff540b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:35:53.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.286+0000 7eff537fe700 1 -- 172.21.15.17:0/67251985 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2762266970 0 0) 0x7eff540aad40 con 0x7eff540b4dd0 2022-08-01T19:35:53.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.286+0000 7eff53fff700 1 --2- 172.21.15.17:0/67251985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff54067490 0x7eff54143a80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7eff48002f70 tx=0x7eff48006df0).ready entity=mon.1 client_cookie=c2b5b517443475dc server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:53.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.286+0000 7eff537fe700 1 -- 172.21.15.17:0/67251985 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff4800d570 con 0x7eff54067490 2022-08-01T19:35:53.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.286+0000 7eff537fe700 1 -- 172.21.15.17:0/67251985 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff48000e90 con 0x7eff54067490 2022-08-01T19:35:53.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.286+0000 7eff537fe700 1 -- 172.21.15.17:0/67251985 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff4800d8c0 con 0x7eff54067490 2022-08-01T19:35:53.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.286+0000 7eff5af64700 1 -- 172.21.15.17:0/67251985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff54067490 msgr2=0x7eff54143a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:53.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.286+0000 7eff5af64700 1 --2- 172.21.15.17:0/67251985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff54067490 0x7eff54143a80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7eff48002f70 tx=0x7eff48006df0).stop 2022-08-01T19:35:53.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.286+0000 7eff5af64700 1 -- 172.21.15.17:0/67251985 shutdown_connections 2022-08-01T19:35:53.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.286+0000 7eff5af64700 1 --2- 172.21.15.17:0/67251985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff54067490 0x7eff54143a80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:53.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.286+0000 7eff5af64700 1 --2- 172.21.15.17:0/67251985 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7eff54060290 0x7eff540b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:53.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.286+0000 7eff5af64700 1 -- 172.21.15.17:0/67251985 >> 172.21.15.17:0/67251985 conn(0x7eff5405d0c0 msgr2=0x7eff5405f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:53.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.286+0000 7eff5af64700 1 -- 172.21.15.17:0/67251985 shutdown_connections 2022-08-01T19:35:53.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.286+0000 7eff5af64700 1 -- 172.21.15.17:0/67251985 wait complete. 2022-08-01T19:35:53.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.290+0000 7eff5af64700 1 Processor -- start 2022-08-01T19:35:53.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.290+0000 7eff5af64700 1 -- start start 2022-08-01T19:35:53.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.290+0000 7eff5af64700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff54060290 0x7eff540b0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:53.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.290+0000 7eff5af64700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff54067490 0x7eff540ae130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:53.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.290+0000 7eff5af64700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7eff540b4dd0 0x7eff540ae620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:53.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.290+0000 7eff5af64700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5409f7d0 con 0x7eff54067490 2022-08-01T19:35:53.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.290+0000 7eff5af64700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff540acd70 con 0x7eff54060290 2022-08-01T19:35:53.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.290+0000 7eff5af64700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff540ad490 con 0x7eff540b4dd0 2022-08-01T19:35:53.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.290+0000 7eff53fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff54067490 0x7eff540ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:53.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.290+0000 7eff53fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff54067490 0x7eff540ae130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38030/0 (socket says 172.21.15.17:38030) 2022-08-01T19:35:53.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.290+0000 7eff58d00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff54060290 0x7eff540b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:53.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.290+0000 7eff53fff700 1 -- 172.21.15.17:0/2862511000 learned_addr learned my addr 172.21.15.17:0/2862511000 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:53.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.290+0000 7eff59501700 1 --2- 172.21.15.17:0/2862511000 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7eff540b4dd0 0x7eff540ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:53.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.290+0000 7eff53fff700 1 -- 172.21.15.17:0/2862511000 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7eff540b4dd0 msgr2=0x7eff540ae620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:53.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.290+0000 7eff53fff700 1 --2- 172.21.15.17:0/2862511000 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7eff540b4dd0 0x7eff540ae620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:53.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.290+0000 7eff53fff700 1 -- 172.21.15.17:0/2862511000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff54060290 msgr2=0x7eff540b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:53.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.290+0000 7eff53fff700 1 --2- 172.21.15.17:0/2862511000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff54060290 0x7eff540b0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:53.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.290+0000 7eff53fff700 1 -- 172.21.15.17:0/2862511000 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff44009470 con 0x7eff54067490 2022-08-01T19:35:53.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.290+0000 7eff58d00700 1 --2- 172.21.15.17:0/2862511000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff54060290 0x7eff540b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:35:53.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.290+0000 7eff53fff700 1 --2- 172.21.15.17:0/2862511000 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff54067490 0x7eff540ae130 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7eff48002f70 tx=0x7eff48014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:53.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.290+0000 7eff51ffb700 1 -- 172.21.15.17:0/2862511000 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff4800dea0 con 0x7eff54067490 2022-08-01T19:35:53.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.294+0000 7eff51ffb700 1 -- 172.21.15.17:0/2862511000 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff48013de0 con 0x7eff54067490 2022-08-01T19:35:53.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.294+0000 7eff5af64700 1 -- 172.21.15.17:0/2862511000 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff4800b180 con 0x7eff54067490 2022-08-01T19:35:53.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.294+0000 7eff51ffb700 1 -- 172.21.15.17:0/2862511000 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff4801e5f0 con 0x7eff54067490 2022-08-01T19:35:53.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.294+0000 7eff5af64700 1 -- 172.21.15.17:0/2862511000 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff540a09b0 con 0x7eff54067490 2022-08-01T19:35:53.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.294+0000 7eff5af64700 1 -- 172.21.15.17:0/2862511000 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff540574e0 con 0x7eff54067490 2022-08-01T19:35:53.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.294+0000 7eff51ffb700 1 -- 172.21.15.17:0/2862511000 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7eff48025030 con 0x7eff54067490 2022-08-01T19:35:53.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.294+0000 7eff51ffb700 1 --2- 172.21.15.17:0/2862511000 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7eff34053a70 0x7eff34055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:53.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.294+0000 7eff51ffb700 1 -- 172.21.15.17:0/2862511000 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6068+0+0 (secure 0 0 0) 0x7eff4806cbf0 con 0x7eff54067490 2022-08-01T19:35:53.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.294+0000 7eff58d00700 1 --2- 172.21.15.17:0/2862511000 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7eff34053a70 0x7eff34055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:53.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.298+0000 7eff58d00700 1 --2- 172.21.15.17:0/2862511000 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7eff34053a70 0x7eff34055f30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7eff4400b000 tx=0x7eff440091d0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:53.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.302+0000 7eff51ffb700 1 -- 172.21.15.17:0/2862511000 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7eff48043a00 con 0x7eff54067490 2022-08-01T19:35:53.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:53.430+0000 7eff5af64700 1 -- 172.21.15.17:0/2862511000 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7eff54069a70 con 0x7eff54067490 2022-08-01T19:35:54.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.146+0000 7eff51ffb700 1 -- 172.21.15.17:0/2862511000 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v38) v1 ==== 173+0+0 (secure 0 0 0) 0x7eff48023070 con 0x7eff54067490 2022-08-01T19:35:54.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.150+0000 7eff5af64700 1 -- 172.21.15.17:0/2862511000 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7eff34053a70 msgr2=0x7eff34055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:54.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.150+0000 7eff5af64700 1 --2- 172.21.15.17:0/2862511000 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7eff34053a70 0x7eff34055f30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7eff4400b000 tx=0x7eff440091d0).stop 2022-08-01T19:35:54.158 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.150+0000 7eff5af64700 1 -- 172.21.15.17:0/2862511000 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff54067490 msgr2=0x7eff540ae130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:54.158 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.150+0000 7eff5af64700 1 --2- 172.21.15.17:0/2862511000 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff54067490 0x7eff540ae130 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7eff48002f70 tx=0x7eff48014000).stop 2022-08-01T19:35:54.158 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.150+0000 7eff5af64700 1 -- 172.21.15.17:0/2862511000 shutdown_connections 2022-08-01T19:35:54.158 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.150+0000 7eff5af64700 1 --2- 172.21.15.17:0/2862511000 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7eff34053a70 0x7eff34055f30 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:54.158 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.150+0000 7eff5af64700 1 --2- 172.21.15.17:0/2862511000 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7eff540b4dd0 0x7eff540ae620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:54.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.150+0000 7eff5af64700 1 --2- 172.21.15.17:0/2862511000 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff54067490 0x7eff540ae130 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:54.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.150+0000 7eff5af64700 1 --2- 172.21.15.17:0/2862511000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff54060290 0x7eff540b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:54.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.150+0000 7eff5af64700 1 -- 172.21.15.17:0/2862511000 >> 172.21.15.17:0/2862511000 conn(0x7eff5405d0c0 msgr2=0x7eff540b4660 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:54.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.150+0000 7eff5af64700 1 -- 172.21.15.17:0/2862511000 shutdown_connections 2022-08-01T19:35:54.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.150+0000 7eff5af64700 1 -- 172.21.15.17:0/2862511000 wait complete. 2022-08-01T19:35:54.160 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' removed 2022-08-01T19:35:54.173 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-08-01T19:35:54.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.286+0000 7faa45e1b700 1 Processor -- start 2022-08-01T19:35:54.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.286+0000 7faa45e1b700 1 -- start start 2022-08-01T19:35:54.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.286+0000 7faa45e1b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faa400614b0 0x7faa40061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:54.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.286+0000 7faa45e1b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7faa400ad750 0x7faa400b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:54.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.286+0000 7faa45e1b700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa400aee80 con 0x7faa401405d0 2022-08-01T19:35:54.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.286+0000 7faa45e1b700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa400b70b0 con 0x7faa400614b0 2022-08-01T19:35:54.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.286+0000 7faa45e1b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa400af0b0 con 0x7faa400ad750 2022-08-01T19:35:54.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.286+0000 7faa44e19700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faa400614b0 0x7faa40061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:54.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.286+0000 7faa44e19700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faa400614b0 0x7faa40061890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38038/0 (socket says 172.21.15.17:38038) 2022-08-01T19:35:54.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.286+0000 7faa44e19700 1 -- 172.21.15.17:0/1864656235 learned_addr learned my addr 172.21.15.17:0/1864656235 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:54.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.286+0000 7faa4561a700 1 --1- 172.21.15.17:0/1864656235 >> v1:172.21.15.17:6789/0 conn(0x7faa401405d0 0x7faa401466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58466/0 (socket says 172.21.15.17:58466) 2022-08-01T19:35:54.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.286+0000 7faa3ffff700 1 --2- 172.21.15.17:0/1864656235 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7faa400ad750 0x7faa400b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:54.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.286+0000 7faa44e19700 1 -- 172.21.15.17:0/1864656235 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7faa400ad750 msgr2=0x7faa400b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:54.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.286+0000 7faa44e19700 1 --2- 172.21.15.17:0/1864656235 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7faa400ad750 0x7faa400b9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:54.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.286+0000 7faa44e19700 1 -- 172.21.15.17:0/1864656235 >> v1:172.21.15.17:6789/0 conn(0x7faa401405d0 legacy=0x7faa401466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:54.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.286+0000 7faa3f7fe700 1 -- 172.21.15.17:0/1864656235 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2683185238 0 0) 0x7faa400aee80 con 0x7faa401405d0 2022-08-01T19:35:54.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.286+0000 7faa44e19700 1 -- 172.21.15.17:0/1864656235 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa400af350 con 0x7faa400614b0 2022-08-01T19:35:54.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.286+0000 7faa44e19700 1 --2- 172.21.15.17:0/1864656235 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa400614b0 0x7faa40061890 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7faa28002f70 tx=0x7faa28006eb0).ready entity=mon.0 client_cookie=44d4a3dc35523ecb server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:54.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.286+0000 7faa3f7fe700 1 -- 172.21.15.17:0/1864656235 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa2800d570 con 0x7faa400614b0 2022-08-01T19:35:54.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.286+0000 7faa3f7fe700 1 -- 172.21.15.17:0/1864656235 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa28000e90 con 0x7faa400614b0 2022-08-01T19:35:54.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.286+0000 7faa3f7fe700 1 -- 172.21.15.17:0/1864656235 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa2800d8c0 con 0x7faa400614b0 2022-08-01T19:35:54.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.286+0000 7faa45e1b700 1 -- 172.21.15.17:0/1864656235 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa400614b0 msgr2=0x7faa40061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:54.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.286+0000 7faa45e1b700 1 --2- 172.21.15.17:0/1864656235 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa400614b0 0x7faa40061890 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7faa28002f70 tx=0x7faa28006eb0).stop 2022-08-01T19:35:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.286+0000 7faa45e1b700 1 -- 172.21.15.17:0/1864656235 shutdown_connections 2022-08-01T19:35:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.286+0000 7faa45e1b700 1 --2- 172.21.15.17:0/1864656235 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7faa400ad750 0x7faa400b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.286+0000 7faa45e1b700 1 --2- 172.21.15.17:0/1864656235 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa400614b0 0x7faa40061890 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.286+0000 7faa45e1b700 1 -- 172.21.15.17:0/1864656235 >> 172.21.15.17:0/1864656235 conn(0x7faa40066f70 msgr2=0x7faa40067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:54.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.286+0000 7faa45e1b700 1 -- 172.21.15.17:0/1864656235 shutdown_connections 2022-08-01T19:35:54.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.286+0000 7faa45e1b700 1 -- 172.21.15.17:0/1864656235 wait complete. 2022-08-01T19:35:54.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.290+0000 7faa45e1b700 1 Processor -- start 2022-08-01T19:35:54.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.290+0000 7faa45e1b700 1 -- start start 2022-08-01T19:35:54.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.290+0000 7faa45e1b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7faa400614b0 0x7faa4009ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:54.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.290+0000 7faa45e1b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa400ad750 0x7faa400ac760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:54.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.290+0000 7faa45e1b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa401405d0 0x7faa400acc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:54.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.290+0000 7faa45e1b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa401523a0 con 0x7faa400ad750 2022-08-01T19:35:54.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.290+0000 7faa45e1b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa40140d00 con 0x7faa401405d0 2022-08-01T19:35:54.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.290+0000 7faa45e1b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa4014a960 con 0x7faa400614b0 2022-08-01T19:35:54.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.290+0000 7faa3ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa400ad750 0x7faa400ac760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:54.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.290+0000 7faa3ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa400ad750 0x7faa400ac760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38044/0 (socket says 172.21.15.17:38044) 2022-08-01T19:35:54.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.290+0000 7faa3ffff700 1 -- 172.21.15.17:0/3546951687 learned_addr learned my addr 172.21.15.17:0/3546951687 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:54.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.290+0000 7faa44e19700 1 --2- 172.21.15.17:0/3546951687 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7faa400614b0 0x7faa4009ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:54.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.290+0000 7faa4561a700 1 --2- 172.21.15.17:0/3546951687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa401405d0 0x7faa400acc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:54.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.290+0000 7faa4561a700 1 -- 172.21.15.17:0/3546951687 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7faa400614b0 msgr2=0x7faa4009ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:54.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.290+0000 7faa4561a700 1 --2- 172.21.15.17:0/3546951687 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7faa400614b0 0x7faa4009ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:54.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.290+0000 7faa4561a700 1 -- 172.21.15.17:0/3546951687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa400ad750 msgr2=0x7faa400ac760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:54.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.290+0000 7faa4561a700 1 --2- 172.21.15.17:0/3546951687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa400ad750 0x7faa400ac760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:54.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.290+0000 7faa4561a700 1 -- 172.21.15.17:0/3546951687 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2800b1e0 con 0x7faa401405d0 2022-08-01T19:35:54.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.290+0000 7faa3ffff700 1 --2- 172.21.15.17:0/3546951687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa400ad750 0x7faa400ac760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:35:54.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.294+0000 7faa4561a700 1 --2- 172.21.15.17:0/3546951687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa401405d0 0x7faa400acc50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7faa38001250 tx=0x7faa38007f20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:54.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.294+0000 7faa3dffb700 1 -- 172.21.15.17:0/3546951687 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa3800ce80 con 0x7faa401405d0 2022-08-01T19:35:54.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.294+0000 7faa3dffb700 1 -- 172.21.15.17:0/3546951687 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa38005ae0 con 0x7faa401405d0 2022-08-01T19:35:54.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.294+0000 7faa45e1b700 1 -- 172.21.15.17:0/3546951687 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa40140a60 con 0x7faa401405d0 2022-08-01T19:35:54.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.294+0000 7faa3dffb700 1 -- 172.21.15.17:0/3546951687 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa38005e40 con 0x7faa401405d0 2022-08-01T19:35:54.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.294+0000 7faa45e1b700 1 -- 172.21.15.17:0/3546951687 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa4014ff50 con 0x7faa401405d0 2022-08-01T19:35:54.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.294+0000 7faa45e1b700 1 -- 172.21.15.17:0/3546951687 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa400af5f0 con 0x7faa401405d0 2022-08-01T19:35:54.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.294+0000 7faa3dffb700 1 -- 172.21.15.17:0/3546951687 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7faa3801a060 con 0x7faa401405d0 2022-08-01T19:35:54.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.294+0000 7faa3dffb700 1 --2- 172.21.15.17:0/3546951687 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7faa24053950 0x7faa24055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:54.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.294+0000 7faa3dffb700 1 -- 172.21.15.17:0/3546951687 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5686+0+0 (secure 0 0 0) 0x7faa3806b6e0 con 0x7faa401405d0 2022-08-01T19:35:54.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.294+0000 7faa44e19700 1 --2- 172.21.15.17:0/3546951687 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7faa24053950 0x7faa24055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:54.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.298+0000 7faa44e19700 1 --2- 172.21.15.17:0/3546951687 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7faa24053950 0x7faa24055e10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7faa2801c000 tx=0x7faa28015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:54.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.306+0000 7faa3dffb700 1 -- 172.21.15.17:0/3546951687 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7faa38042670 con 0x7faa401405d0 2022-08-01T19:35:54.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.438+0000 7faa45e1b700 1 -- 172.21.15.17:0/3546951687 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7faa400574e0 con 0x7faa401405d0 2022-08-01T19:35:54.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.446+0000 7faa3dffb700 1 -- 172.21.15.17:0/3546951687 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v3) v1 ==== 104+0+0 (secure 0 0 0) 0x7faa38018020 con 0x7faa401405d0 2022-08-01T19:35:54.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.450+0000 7faa45e1b700 1 -- 172.21.15.17:0/3546951687 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7faa24053950 msgr2=0x7faa24055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:54.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.450+0000 7faa45e1b700 1 --2- 172.21.15.17:0/3546951687 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7faa24053950 0x7faa24055e10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7faa2801c000 tx=0x7faa28015000).stop 2022-08-01T19:35:54.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.450+0000 7faa45e1b700 1 -- 172.21.15.17:0/3546951687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa401405d0 msgr2=0x7faa400acc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:54.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.450+0000 7faa45e1b700 1 --2- 172.21.15.17:0/3546951687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa401405d0 0x7faa400acc50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7faa38001250 tx=0x7faa38007f20).stop 2022-08-01T19:35:54.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.450+0000 7faa45e1b700 1 -- 172.21.15.17:0/3546951687 shutdown_connections 2022-08-01T19:35:54.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.450+0000 7faa45e1b700 1 --2- 172.21.15.17:0/3546951687 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7faa24053950 0x7faa24055e10 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:54.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.450+0000 7faa45e1b700 1 --2- 172.21.15.17:0/3546951687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa401405d0 0x7faa400acc50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:54.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.450+0000 7faa45e1b700 1 --2- 172.21.15.17:0/3546951687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa400ad750 0x7faa400ac760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:54.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.450+0000 7faa45e1b700 1 --2- 172.21.15.17:0/3546951687 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7faa400614b0 0x7faa4009ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:54.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.450+0000 7faa45e1b700 1 -- 172.21.15.17:0/3546951687 >> 172.21.15.17:0/3546951687 conn(0x7faa40066f70 msgr2=0x7faa4005be30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:54.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.450+0000 7faa45e1b700 1 -- 172.21.15.17:0/3546951687 shutdown_connections 2022-08-01T19:35:54.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.450+0000 7faa45e1b700 1 -- 172.21.15.17:0/3546951687 wait complete. 2022-08-01T19:35:54.467 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command' 2022-08-01T19:35:54.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.566+0000 7f39edb5f700 1 Processor -- start 2022-08-01T19:35:54.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.566+0000 7f39edb5f700 1 -- start start 2022-08-01T19:35:54.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.566+0000 7f39edb5f700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f39e80b2710 0x7f39e814afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:54.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.566+0000 7f39edb5f700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f39e80b01a0 0x7f39e814b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:54.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.566+0000 7f39edb5f700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f39e80afd60 0x7f39e80a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:54.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.566+0000 7f39edb5f700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39e80a7250 con 0x7f39e80b2710 2022-08-01T19:35:54.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.566+0000 7f39edb5f700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39e80a7480 con 0x7f39e80afd60 2022-08-01T19:35:54.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.566+0000 7f39edb5f700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e80a7720 con 0x7f39e80b01a0 2022-08-01T19:35:54.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.566+0000 7f39ecb5d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f39e80b2710 0x7f39e814afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:54.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.566+0000 7f39ecb5d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f39e80b2710 0x7f39e814afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38052/0 (socket says 172.21.15.17:38052) 2022-08-01T19:35:54.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.566+0000 7f39ecb5d700 1 -- 172.21.15.17:0/1110012812 learned_addr learned my addr 172.21.15.17:0/1110012812 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:54.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.566+0000 7f39ed35e700 1 --2- 172.21.15.17:0/1110012812 >> v2:172.21.15.77:3300/0 conn(0x7f39e80afd60 0x7f39e80a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:54.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.566+0000 7f39e7fff700 1 --2- 172.21.15.17:0/1110012812 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f39e80b01a0 0x7f39e814b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:54.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.566+0000 7f39ecb5d700 1 -- 172.21.15.17:0/1110012812 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f39e80b01a0 msgr2=0x7f39e814b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:54.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.566+0000 7f39ecb5d700 1 --2- 172.21.15.17:0/1110012812 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f39e80b01a0 0x7f39e814b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:54.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.566+0000 7f39ecb5d700 1 -- 172.21.15.17:0/1110012812 >> v2:172.21.15.77:3300/0 conn(0x7f39e80afd60 msgr2=0x7f39e80a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:54.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.566+0000 7f39ecb5d700 1 --2- 172.21.15.17:0/1110012812 >> v2:172.21.15.77:3300/0 conn(0x7f39e80afd60 0x7f39e80a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:54.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.566+0000 7f39ecb5d700 1 -- 172.21.15.17:0/1110012812 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e80a79c0 con 0x7f39e80b2710 2022-08-01T19:35:54.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.566+0000 7f39ed35e700 1 --2- 172.21.15.17:0/1110012812 >> v2:172.21.15.77:3300/0 conn(0x7f39e80afd60 0x7f39e80a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:35:54.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.566+0000 7f39ecb5d700 1 --2- 172.21.15.17:0/1110012812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39e80b2710 0x7f39e814afd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f39d0002f70 tx=0x7f39d0006eb0).ready entity=mon.0 client_cookie=9215f69df78e8510 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:54.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.566+0000 7f39e77fe700 1 -- 172.21.15.17:0/1110012812 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39d000d570 con 0x7f39e80b2710 2022-08-01T19:35:54.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.566+0000 7f39e77fe700 1 -- 172.21.15.17:0/1110012812 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d0000e90 con 0x7f39e80b2710 2022-08-01T19:35:54.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.566+0000 7f39e77fe700 1 -- 172.21.15.17:0/1110012812 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39d000d900 con 0x7f39e80b2710 2022-08-01T19:35:54.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.566+0000 7f39edb5f700 1 -- 172.21.15.17:0/1110012812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39e80b2710 msgr2=0x7f39e814afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:54.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.566+0000 7f39edb5f700 1 --2- 172.21.15.17:0/1110012812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39e80b2710 0x7f39e814afd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f39d0002f70 tx=0x7f39d0006eb0).stop 2022-08-01T19:35:54.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.566+0000 7f39edb5f700 1 -- 172.21.15.17:0/1110012812 shutdown_connections 2022-08-01T19:35:54.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.566+0000 7f39edb5f700 1 --2- 172.21.15.17:0/1110012812 >> v2:172.21.15.77:3300/0 conn(0x7f39e80afd60 0x7f39e80a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:54.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.566+0000 7f39edb5f700 1 --2- 172.21.15.17:0/1110012812 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f39e80b01a0 0x7f39e814b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:54.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.566+0000 7f39edb5f700 1 --2- 172.21.15.17:0/1110012812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39e80b2710 0x7f39e814afd0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:54.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.566+0000 7f39edb5f700 1 -- 172.21.15.17:0/1110012812 >> 172.21.15.17:0/1110012812 conn(0x7f39e8060310 msgr2=0x7f39e80b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:54.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.566+0000 7f39edb5f700 1 -- 172.21.15.17:0/1110012812 shutdown_connections 2022-08-01T19:35:54.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.566+0000 7f39edb5f700 1 -- 172.21.15.17:0/1110012812 wait complete. 2022-08-01T19:35:54.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.570+0000 7f39edb5f700 1 Processor -- start 2022-08-01T19:35:54.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.570+0000 7f39edb5f700 1 -- start start 2022-08-01T19:35:54.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.570+0000 7f39edb5f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39e80afd60 0x7f39e80aef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:54.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.570+0000 7f39edb5f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39e80b01a0 0x7f39e8142bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:54.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.570+0000 7f39edb5f700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f39e80b2710 0x7f39e8143f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:54.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.570+0000 7f39edb5f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e8143c10 con 0x7f39e80afd60 2022-08-01T19:35:54.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.570+0000 7f39edb5f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e80677a0 con 0x7f39e80b01a0 2022-08-01T19:35:54.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.570+0000 7f39edb5f700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e8157080 con 0x7f39e80b2710 2022-08-01T19:35:54.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.570+0000 7f39ecb5d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39e80afd60 0x7f39e80aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:54.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.570+0000 7f39ecb5d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39e80afd60 0x7f39e80aef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38056/0 (socket says 172.21.15.17:38056) 2022-08-01T19:35:54.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.570+0000 7f39ecb5d700 1 -- 172.21.15.17:0/3642711064 learned_addr learned my addr 172.21.15.17:0/3642711064 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:54.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.570+0000 7f39e7fff700 1 --2- 172.21.15.17:0/3642711064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39e80b01a0 0x7f39e8142bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:54.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.570+0000 7f39ed35e700 1 --2- 172.21.15.17:0/3642711064 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f39e80b2710 0x7f39e8143f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:54.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.570+0000 7f39ecb5d700 1 -- 172.21.15.17:0/3642711064 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f39e80b2710 msgr2=0x7f39e8143f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:54.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.570+0000 7f39ecb5d700 1 --2- 172.21.15.17:0/3642711064 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f39e80b2710 0x7f39e8143f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:54.602 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.570+0000 7f39ecb5d700 1 -- 172.21.15.17:0/3642711064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39e80b01a0 msgr2=0x7f39e8142bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:54.602 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.570+0000 7f39ecb5d700 1 --2- 172.21.15.17:0/3642711064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39e80b01a0 0x7f39e8142bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:54.602 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.570+0000 7f39ecb5d700 1 -- 172.21.15.17:0/3642711064 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d000b1e0 con 0x7f39e80afd60 2022-08-01T19:35:54.602 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.570+0000 7f39ecb5d700 1 --2- 172.21.15.17:0/3642711064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39e80afd60 0x7f39e80aef50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f39e8143d50 tx=0x7f39d0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:54.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.570+0000 7f39e5ffb700 1 -- 172.21.15.17:0/3642711064 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39d0006be0 con 0x7f39e80afd60 2022-08-01T19:35:54.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.570+0000 7f39e5ffb700 1 -- 172.21.15.17:0/3642711064 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d0011ca0 con 0x7f39e80afd60 2022-08-01T19:35:54.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.570+0000 7f39edb5f700 1 -- 172.21.15.17:0/3642711064 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39e80a9410 con 0x7f39e80afd60 2022-08-01T19:35:54.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.570+0000 7f39e5ffb700 1 -- 172.21.15.17:0/3642711064 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39d001e460 con 0x7f39e80afd60 2022-08-01T19:35:54.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.570+0000 7f39edb5f700 1 -- 172.21.15.17:0/3642711064 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e8156db0 con 0x7f39e80afd60 2022-08-01T19:35:54.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.574+0000 7f39edb5f700 1 -- 172.21.15.17:0/3642711064 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39e8157280 con 0x7f39e80afd60 2022-08-01T19:35:54.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.574+0000 7f39e5ffb700 1 -- 172.21.15.17:0/3642711064 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f39d0025030 con 0x7f39e80afd60 2022-08-01T19:35:54.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.574+0000 7f39e5ffb700 1 --2- 172.21.15.17:0/3642711064 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f39c00539e0 0x7f39c0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:54.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.574+0000 7f39e5ffb700 1 -- 172.21.15.17:0/3642711064 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f39d006cfe0 con 0x7f39e80afd60 2022-08-01T19:35:54.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.574+0000 7f39e7fff700 1 --2- 172.21.15.17:0/3642711064 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f39c00539e0 0x7f39c0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:54.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.574+0000 7f39e7fff700 1 --2- 172.21.15.17:0/3642711064 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f39c00539e0 0x7f39c0055ea0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f39e000b000 tx=0x7f39e0008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:54.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.582+0000 7f39e5ffb700 1 -- 172.21.15.17:0/3642711064 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f39d0044080 con 0x7f39e80afd60 2022-08-01T19:35:54.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:54.746+0000 7f39edb5f700 1 -- 172.21.15.17:0/3642711064 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]} v 0) v1 -- 0x7f39e80574e0 con 0x7f39e80afd60 2022-08-01T19:35:55.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.162+0000 7f39e5ffb700 1 -- 172.21.15.17:0/3642711064 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]}]=0 v95) v1 ==== 139+0+0 (secure 0 0 0) 0x7f39d0023020 con 0x7f39e80afd60 2022-08-01T19:35:55.170 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.162+0000 7f39edb5f700 1 -- 172.21.15.17:0/3642711064 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f39c00539e0 msgr2=0x7f39c0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:55.171 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.162+0000 7f39edb5f700 1 --2- 172.21.15.17:0/3642711064 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f39c00539e0 0x7f39c0055ea0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f39e000b000 tx=0x7f39e0008040).stop 2022-08-01T19:35:55.171 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.162+0000 7f39edb5f700 1 -- 172.21.15.17:0/3642711064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39e80afd60 msgr2=0x7f39e80aef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:55.171 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.162+0000 7f39edb5f700 1 --2- 172.21.15.17:0/3642711064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39e80afd60 0x7f39e80aef50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f39e8143d50 tx=0x7f39d0015000).stop 2022-08-01T19:35:55.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.166+0000 7f39edb5f700 1 -- 172.21.15.17:0/3642711064 shutdown_connections 2022-08-01T19:35:55.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.166+0000 7f39edb5f700 1 --2- 172.21.15.17:0/3642711064 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f39c00539e0 0x7f39c0055ea0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:55.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.166+0000 7f39edb5f700 1 --2- 172.21.15.17:0/3642711064 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f39e80b2710 0x7f39e8143f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:55.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.166+0000 7f39edb5f700 1 --2- 172.21.15.17:0/3642711064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39e80b01a0 0x7f39e8142bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:55.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.166+0000 7f39edb5f700 1 --2- 172.21.15.17:0/3642711064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39e80afd60 0x7f39e80aef50 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:55.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.166+0000 7f39edb5f700 1 -- 172.21.15.17:0/3642711064 >> 172.21.15.17:0/3642711064 conn(0x7f39e8060310 msgr2=0x7f39e81499d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:55.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.166+0000 7f39edb5f700 1 -- 172.21.15.17:0/3642711064 shutdown_connections 2022-08-01T19:35:55.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.166+0000 7f39edb5f700 1 -- 172.21.15.17:0/3642711064 wait complete. 2022-08-01T19:35:55.183 INFO:tasks.cephfs_test_runner:test_drop_cache_command (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2022-08-01T19:35:55.184 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command_dead (tasks.cephfs.test_misc.TestCacheDrop) 2022-08-01T19:35:55.184 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead' 2022-08-01T19:35:55.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.294+0000 7f4fe61d3700 1 Processor -- start 2022-08-01T19:35:55.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.294+0000 7f4fe61d3700 1 -- start start 2022-08-01T19:35:55.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.294+0000 7f4fe61d3700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f4fe00b0870 0x7f4fe0144170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:55.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.294+0000 7f4fe61d3700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fe0067560 con 0x7f4fe0067180 2022-08-01T19:35:55.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.294+0000 7f4fe61d3700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fe00b0160 con 0x7f4fe00ad1c0 2022-08-01T19:35:55.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.294+0000 7f4fe61d3700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fe00b0400 con 0x7f4fe00b0870 2022-08-01T19:35:55.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.294+0000 7f4fdf7fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f4fe0067180 0x7f4fe0143a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58490/0 (socket says 172.21.15.17:58490) 2022-08-01T19:35:55.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.294+0000 7f4fdf7fe700 1 -- 172.21.15.17:0/4106293535 learned_addr learned my addr 172.21.15.17:0/4106293535 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:35:55.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.294+0000 7f4fdffff700 1 --2- 172.21.15.17:0/4106293535 >> v2:172.21.15.77:3300/0 conn(0x7f4fe00b0870 0x7f4fe0144170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:55.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.294+0000 7f4fde7fc700 1 -- 172.21.15.17:0/4106293535 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 805876441 0 0) 0x7f4fe0067560 con 0x7f4fe0067180 2022-08-01T19:35:55.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.294+0000 7f4fde7fc700 1 -- 172.21.15.17:0/4106293535 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fc0003260 con 0x7f4fe0067180 2022-08-01T19:35:55.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.294+0000 7f4fde7fc700 1 -- 172.21.15.17:0/4106293535 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1622877764 0 0) 0x7f4fe00b0160 con 0x7f4fe00ad1c0 2022-08-01T19:35:55.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.294+0000 7f4fde7fc700 1 -- 172.21.15.17:0/4106293535 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fe0067560 con 0x7f4fe00ad1c0 2022-08-01T19:35:55.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.294+0000 7f4fde7fc700 1 -- 172.21.15.17:0/4106293535 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1781254123 0 0) 0x7f4fc0003260 con 0x7f4fe0067180 2022-08-01T19:35:55.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.294+0000 7f4fde7fc700 1 -- 172.21.15.17:0/4106293535 >> v2:172.21.15.77:3300/0 conn(0x7f4fe00b0870 msgr2=0x7f4fe0144170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:55.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.294+0000 7f4fde7fc700 1 --2- 172.21.15.17:0/4106293535 >> v2:172.21.15.77:3300/0 conn(0x7f4fe00b0870 0x7f4fe0144170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:55.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.294+0000 7f4fde7fc700 1 -- 172.21.15.17:0/4106293535 >> v1:172.21.15.77:6789/0 conn(0x7f4fe00ad1c0 legacy=0x7f4fe00b1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:55.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.294+0000 7f4fde7fc700 1 -- 172.21.15.17:0/4106293535 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe00af530 con 0x7f4fe0067180 2022-08-01T19:35:55.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.294+0000 7f4fde7fc700 1 -- 172.21.15.17:0/4106293535 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7f4fd0001a00 con 0x7f4fe0067180 2022-08-01T19:35:55.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.294+0000 7f4fde7fc700 1 -- 172.21.15.17:0/4106293535 >> v1:172.21.15.17:6789/0 conn(0x7f4fe0067180 legacy=0x7f4fe0143a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:55.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.294+0000 7f4fde7fc700 1 --2- 172.21.15.17:0/4106293535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fc0003930 0x7f4fc0003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:55.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.294+0000 7f4fde7fc700 1 --2- 172.21.15.17:0/4106293535 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4fc0004500 0x7f4fc0004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:55.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.294+0000 7f4fde7fc700 1 --2- 172.21.15.17:0/4106293535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc0004e70 0x7f4fc000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:55.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.294+0000 7f4fde7fc700 1 -- 172.21.15.17:0/4106293535 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe00b0400 con 0x7f4fc0003930 2022-08-01T19:35:55.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.294+0000 7f4fdf7fe700 1 --2- 172.21.15.17:0/4106293535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fc0003930 0x7f4fc0003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:55.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.294+0000 7f4fde7fc700 1 -- 172.21.15.17:0/4106293535 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc000bd20 con 0x7f4fc0004e70 2022-08-01T19:35:55.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.294+0000 7f4fde7fc700 1 -- 172.21.15.17:0/4106293535 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc000bee0 con 0x7f4fc0004500 2022-08-01T19:35:55.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.294+0000 7f4fdffff700 1 --2- 172.21.15.17:0/4106293535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc0004e70 0x7f4fc000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:55.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.294+0000 7f4fdeffd700 1 --2- 172.21.15.17:0/4106293535 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4fc0004500 0x7f4fc0004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:55.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.298+0000 7f4fdf7fe700 1 -- 172.21.15.17:0/4106293535 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4fc0004500 msgr2=0x7f4fc0004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:55.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.298+0000 7f4fdf7fe700 1 --2- 172.21.15.17:0/4106293535 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4fc0004500 0x7f4fc0004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:55.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.298+0000 7f4fdf7fe700 1 -- 172.21.15.17:0/4106293535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc0004e70 msgr2=0x7f4fc000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:55.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.298+0000 7f4fdf7fe700 1 --2- 172.21.15.17:0/4106293535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc0004e70 0x7f4fc000b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:55.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.298+0000 7f4fdf7fe700 1 -- 172.21.15.17:0/4106293535 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc0003260 con 0x7f4fc0003930 2022-08-01T19:35:55.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.298+0000 7f4fdffff700 1 -- 172.21.15.17:0/4106293535 reap_dead start 2022-08-01T19:35:55.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.298+0000 7f4fdf7fe700 1 --2- 172.21.15.17:0/4106293535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fc0003930 0x7f4fc0003d30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f4fd00017d0 tx=0x7f4fe00b5750).ready entity=mon.0 client_cookie=e08ed5c380b08d7d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:55.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.298+0000 7f4fde7fc700 1 -- 172.21.15.17:0/4106293535 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fd000cb80 con 0x7f4fc0003930 2022-08-01T19:35:55.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.298+0000 7f4fde7fc700 1 -- 172.21.15.17:0/4106293535 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd000cce0 con 0x7f4fc0003930 2022-08-01T19:35:55.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.298+0000 7f4fde7fc700 1 -- 172.21.15.17:0/4106293535 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fd0004b50 con 0x7f4fc0003930 2022-08-01T19:35:55.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.298+0000 7f4fe61d3700 1 -- 172.21.15.17:0/4106293535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fc0003930 msgr2=0x7f4fc0003d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:55.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.298+0000 7f4fe61d3700 1 --2- 172.21.15.17:0/4106293535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fc0003930 0x7f4fc0003d30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f4fd00017d0 tx=0x7f4fe00b5750).stop 2022-08-01T19:35:55.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.298+0000 7f4fe61d3700 1 -- 172.21.15.17:0/4106293535 shutdown_connections 2022-08-01T19:35:55.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.298+0000 7f4fe61d3700 1 --2- 172.21.15.17:0/4106293535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fc0003930 0x7f4fc0003d30 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:55.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.298+0000 7f4fe61d3700 1 -- 172.21.15.17:0/4106293535 >> 172.21.15.17:0/4106293535 conn(0x7f4fe005f840 msgr2=0x7f4fe005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:55.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.298+0000 7f4fe61d3700 1 -- 172.21.15.17:0/4106293535 shutdown_connections 2022-08-01T19:35:55.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.298+0000 7f4fe61d3700 1 -- 172.21.15.17:0/4106293535 wait complete. 2022-08-01T19:35:55.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.302+0000 7f4fe61d3700 1 Processor -- start 2022-08-01T19:35:55.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.302+0000 7f4fe61d3700 1 -- start start 2022-08-01T19:35:55.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.302+0000 7f4fe61d3700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4fc0003930 0x7f4fe00a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:55.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.302+0000 7f4fe61d3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fe0156850 0x7f4fe00a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:55.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.302+0000 7f4fe61d3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fe00b5b50 0x7f4fe00a25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:55.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.302+0000 7f4fe61d3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe00a1710 con 0x7f4fe0156850 2022-08-01T19:35:55.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.302+0000 7f4fe61d3700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe00a2430 con 0x7f4fe00b5b50 2022-08-01T19:35:55.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.302+0000 7f4fdeffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fe0156850 0x7f4fe00a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:55.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.302+0000 7f4fe61d3700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe00a11d0 con 0x7f4fc0003930 2022-08-01T19:35:55.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.302+0000 7f4fdeffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fe0156850 0x7f4fe00a0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38078/0 (socket says 172.21.15.17:38078) 2022-08-01T19:35:55.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.302+0000 7f4fdeffd700 1 -- 172.21.15.17:0/2640112978 learned_addr learned my addr 172.21.15.17:0/2640112978 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:55.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.302+0000 7f4fdf7fe700 1 --2- 172.21.15.17:0/2640112978 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4fc0003930 0x7f4fe00a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:55.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.302+0000 7f4fdffff700 1 --2- 172.21.15.17:0/2640112978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fe00b5b50 0x7f4fe00a25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:55.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.302+0000 7f4fdeffd700 1 -- 172.21.15.17:0/2640112978 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4fc0003930 msgr2=0x7f4fe00a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:55.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.302+0000 7f4fdeffd700 1 --2- 172.21.15.17:0/2640112978 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4fc0003930 0x7f4fe00a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:55.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.302+0000 7f4fdeffd700 1 -- 172.21.15.17:0/2640112978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fe00b5b50 msgr2=0x7f4fe00a25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:55.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.302+0000 7f4fdeffd700 1 --2- 172.21.15.17:0/2640112978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fe00b5b50 0x7f4fe00a25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:55.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.302+0000 7f4fdeffd700 1 -- 172.21.15.17:0/2640112978 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe00a1470 con 0x7f4fe0156850 2022-08-01T19:35:55.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.302+0000 7f4fdeffd700 1 --2- 172.21.15.17:0/2640112978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fe0156850 0x7f4fe00a0870 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f4fc800b000 tx=0x7f4fc8009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:55.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.302+0000 7f4fdcff9700 1 -- 172.21.15.17:0/2640112978 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fc8014620 con 0x7f4fe0156850 2022-08-01T19:35:55.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.302+0000 7f4fdcff9700 1 -- 172.21.15.17:0/2640112978 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc8014780 con 0x7f4fe0156850 2022-08-01T19:35:55.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.302+0000 7f4fe61d3700 1 -- 172.21.15.17:0/2640112978 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe00a2ad0 con 0x7f4fe0156850 2022-08-01T19:35:55.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.302+0000 7f4fdcff9700 1 -- 172.21.15.17:0/2640112978 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fc8014ae0 con 0x7f4fe0156850 2022-08-01T19:35:55.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.306+0000 7f4fe61d3700 1 -- 172.21.15.17:0/2640112978 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe0142880 con 0x7f4fe0156850 2022-08-01T19:35:55.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.306+0000 7f4fe61d3700 1 -- 172.21.15.17:0/2640112978 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe00574e0 con 0x7f4fe0156850 2022-08-01T19:35:55.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.306+0000 7f4fdcff9700 1 -- 172.21.15.17:0/2640112978 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4fc8014c40 con 0x7f4fe0156850 2022-08-01T19:35:55.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.306+0000 7f4fdcff9700 1 --2- 172.21.15.17:0/2640112978 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4fc4053a70 0x7f4fc4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:55.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.306+0000 7f4fdcff9700 1 -- 172.21.15.17:0/2640112978 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f4fc806bd10 con 0x7f4fe0156850 2022-08-01T19:35:55.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.306+0000 7f4fdf7fe700 1 --2- 172.21.15.17:0/2640112978 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4fc4053a70 0x7f4fc4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:55.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.306+0000 7f4fdf7fe700 1 --2- 172.21.15.17:0/2640112978 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4fc4053a70 0x7f4fc4055f30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f4fd001a000 tx=0x7f4fd0013000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:55.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.314+0000 7f4fdcff9700 1 -- 172.21.15.17:0/2640112978 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4fc8042db0 con 0x7f4fe0156850 2022-08-01T19:35:55.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:55.482+0000 7f4fe61d3700 1 -- 172.21.15.17:0/2640112978 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]} v 0) v1 -- 0x7f4fe00679a0 con 0x7f4fe0156850 2022-08-01T19:35:56.170 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.162+0000 7f4fdcff9700 1 -- 172.21.15.17:0/2640112978 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]}]=0 v96) v1 ==== 147+0+0 (secure 0 0 0) 0x7f4fc80463d0 con 0x7f4fe0156850 2022-08-01T19:35:56.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.166+0000 7f4fe61d3700 1 -- 172.21.15.17:0/2640112978 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4fc4053a70 msgr2=0x7f4fc4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:56.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.166+0000 7f4fe61d3700 1 --2- 172.21.15.17:0/2640112978 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4fc4053a70 0x7f4fc4055f30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f4fd001a000 tx=0x7f4fd0013000).stop 2022-08-01T19:35:56.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.166+0000 7f4fe61d3700 1 -- 172.21.15.17:0/2640112978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fe0156850 msgr2=0x7f4fe00a0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:56.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.166+0000 7f4fe61d3700 1 --2- 172.21.15.17:0/2640112978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fe0156850 0x7f4fe00a0870 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f4fc800b000 tx=0x7f4fc8009040).stop 2022-08-01T19:35:56.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.166+0000 7f4fe61d3700 1 -- 172.21.15.17:0/2640112978 shutdown_connections 2022-08-01T19:35:56.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.166+0000 7f4fe61d3700 1 --2- 172.21.15.17:0/2640112978 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4fc4053a70 0x7f4fc4055f30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:56.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.166+0000 7f4fe61d3700 1 --2- 172.21.15.17:0/2640112978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fe00b5b50 0x7f4fe00a25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:56.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.166+0000 7f4fe61d3700 1 --2- 172.21.15.17:0/2640112978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fe0156850 0x7f4fe00a0870 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:56.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.166+0000 7f4fe61d3700 1 --2- 172.21.15.17:0/2640112978 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4fc0003930 0x7f4fe00a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:56.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.166+0000 7f4fe61d3700 1 -- 172.21.15.17:0/2640112978 >> 172.21.15.17:0/2640112978 conn(0x7f4fe005f840 msgr2=0x7f4fe0149780 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:56.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.166+0000 7f4fe61d3700 1 -- 172.21.15.17:0/2640112978 shutdown_connections 2022-08-01T19:35:56.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.166+0000 7f4fe61d3700 1 -- 172.21.15.17:0/2640112978 wait complete. 2022-08-01T19:35:56.189 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-08-01T19:35:56.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.302+0000 7fc9ea562700 1 Processor -- start 2022-08-01T19:35:56.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.302+0000 7fc9ea562700 1 -- start start 2022-08-01T19:35:56.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.302+0000 7fc9ea562700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc9e40b31d0 0x7fc9e4066020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:56.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.302+0000 7fc9ea562700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9e40a9da0 0x7fc9e4060950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:56.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.302+0000 7fc9ea562700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9e4067030 con 0x7fc9e41434b0 2022-08-01T19:35:56.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.302+0000 7fc9ea562700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9e4064a70 con 0x7fc9e40b31d0 2022-08-01T19:35:56.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.302+0000 7fc9ea562700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e4061f60 con 0x7fc9e40a9da0 2022-08-01T19:35:56.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.302+0000 7fc9e9560700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc9e40b31d0 0x7fc9e4066020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:56.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.302+0000 7fc9e9560700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc9e40b31d0 0x7fc9e4066020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38086/0 (socket says 172.21.15.17:38086) 2022-08-01T19:35:56.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.302+0000 7fc9e8d5f700 1 --2- 172.21.15.17:0/1391639078 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9e40a9da0 0x7fc9e4060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:56.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.302+0000 7fc9e9d61700 1 --1- 172.21.15.17:0/1391639078 >> v1:172.21.15.77:6789/0 conn(0x7fc9e41434b0 0x7fc9e4143940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:47080/0 (socket says 172.21.15.17:47080) 2022-08-01T19:35:56.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.302+0000 7fc9e9560700 1 -- 172.21.15.17:0/1391639078 learned_addr learned my addr 172.21.15.17:0/1391639078 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:56.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.302+0000 7fc9cffff700 1 -- 172.21.15.17:0/1391639078 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3579788082 0 0) 0x7fc9e4067030 con 0x7fc9e41434b0 2022-08-01T19:35:56.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.302+0000 7fc9e8d5f700 1 -- 172.21.15.17:0/1391639078 >> v2:172.21.15.17:3300/0 conn(0x7fc9e40b31d0 msgr2=0x7fc9e4066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:56.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.302+0000 7fc9e8d5f700 1 --2- 172.21.15.17:0/1391639078 >> v2:172.21.15.17:3300/0 conn(0x7fc9e40b31d0 0x7fc9e4066020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:56.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.302+0000 7fc9e8d5f700 1 -- 172.21.15.17:0/1391639078 >> v1:172.21.15.77:6789/0 conn(0x7fc9e41434b0 legacy=0x7fc9e4143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:56.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.302+0000 7fc9e8d5f700 1 -- 172.21.15.17:0/1391639078 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e40b35b0 con 0x7fc9e40a9da0 2022-08-01T19:35:56.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.302+0000 7fc9e8d5f700 1 --2- 172.21.15.17:0/1391639078 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9e40a9da0 0x7fc9e4060950 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fc9d8002f70 tx=0x7fc9d8006df0).ready entity=mon.2 client_cookie=fdc0f6c6981300f3 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:56.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.302+0000 7fc9cffff700 1 -- 172.21.15.17:0/1391639078 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9d800d570 con 0x7fc9e40a9da0 2022-08-01T19:35:56.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.302+0000 7fc9cffff700 1 -- 172.21.15.17:0/1391639078 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d8000e90 con 0x7fc9e40a9da0 2022-08-01T19:35:56.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.302+0000 7fc9cffff700 1 -- 172.21.15.17:0/1391639078 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9d800d8c0 con 0x7fc9e40a9da0 2022-08-01T19:35:56.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.302+0000 7fc9ea562700 1 -- 172.21.15.17:0/1391639078 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9e40a9da0 msgr2=0x7fc9e4060950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:56.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.302+0000 7fc9ea562700 1 --2- 172.21.15.17:0/1391639078 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9e40a9da0 0x7fc9e4060950 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fc9d8002f70 tx=0x7fc9d8006df0).stop 2022-08-01T19:35:56.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.302+0000 7fc9ea562700 1 -- 172.21.15.17:0/1391639078 shutdown_connections 2022-08-01T19:35:56.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.302+0000 7fc9ea562700 1 --2- 172.21.15.17:0/1391639078 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9e40a9da0 0x7fc9e4060950 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:56.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.302+0000 7fc9ea562700 1 --2- 172.21.15.17:0/1391639078 >> v2:172.21.15.17:3300/0 conn(0x7fc9e40b31d0 0x7fc9e4066020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:56.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.302+0000 7fc9ea562700 1 -- 172.21.15.17:0/1391639078 >> 172.21.15.17:0/1391639078 conn(0x7fc9e4063870 msgr2=0x7fc9e4063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:56.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.306+0000 7fc9ea562700 1 -- 172.21.15.17:0/1391639078 shutdown_connections 2022-08-01T19:35:56.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.306+0000 7fc9ea562700 1 -- 172.21.15.17:0/1391639078 wait complete. 2022-08-01T19:35:56.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.306+0000 7fc9ea562700 1 Processor -- start 2022-08-01T19:35:56.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.306+0000 7fc9ea562700 1 -- start start 2022-08-01T19:35:56.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.310+0000 7fc9ea562700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9e40a9da0 0x7fc9e4154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:56.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.310+0000 7fc9ea562700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9e40b31d0 0x7fc9e413c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:56.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.310+0000 7fc9ea562700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9e41434b0 0x7fc9e413e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:56.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.310+0000 7fc9ea562700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e4153680 con 0x7fc9e41434b0 2022-08-01T19:35:56.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.310+0000 7fc9ea562700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e4153920 con 0x7fc9e40a9da0 2022-08-01T19:35:56.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.310+0000 7fc9ea562700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e4153bc0 con 0x7fc9e40b31d0 2022-08-01T19:35:56.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.310+0000 7fc9e9d61700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9e41434b0 0x7fc9e413e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:56.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.310+0000 7fc9e9d61700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9e41434b0 0x7fc9e413e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38094/0 (socket says 172.21.15.17:38094) 2022-08-01T19:35:56.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.310+0000 7fc9e9d61700 1 -- 172.21.15.17:0/571172978 learned_addr learned my addr 172.21.15.17:0/571172978 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:56.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.310+0000 7fc9e9560700 1 --2- 172.21.15.17:0/571172978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9e40a9da0 0x7fc9e4154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:56.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.310+0000 7fc9e8d5f700 1 --2- 172.21.15.17:0/571172978 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9e40b31d0 0x7fc9e413c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:56.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.310+0000 7fc9e9d61700 1 -- 172.21.15.17:0/571172978 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9e40b31d0 msgr2=0x7fc9e413c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:56.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.310+0000 7fc9e9d61700 1 --2- 172.21.15.17:0/571172978 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9e40b31d0 0x7fc9e413c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:56.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.310+0000 7fc9e9d61700 1 -- 172.21.15.17:0/571172978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9e40a9da0 msgr2=0x7fc9e4154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:56.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.310+0000 7fc9e9d61700 1 --2- 172.21.15.17:0/571172978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9e40a9da0 0x7fc9e4154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:56.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.310+0000 7fc9e9d61700 1 -- 172.21.15.17:0/571172978 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d800b180 con 0x7fc9e41434b0 2022-08-01T19:35:56.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.310+0000 7fc9e9d61700 1 --2- 172.21.15.17:0/571172978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9e41434b0 0x7fc9e413e2a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7fc9dc001250 tx=0x7fc9dc001360).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:56.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.310+0000 7fc9d6ffd700 1 -- 172.21.15.17:0/571172978 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9dc005bb0 con 0x7fc9e41434b0 2022-08-01T19:35:56.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.310+0000 7fc9d6ffd700 1 -- 172.21.15.17:0/571172978 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9dc005d10 con 0x7fc9e41434b0 2022-08-01T19:35:56.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.310+0000 7fc9ea562700 1 -- 172.21.15.17:0/571172978 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e413e790 con 0x7fc9e41434b0 2022-08-01T19:35:56.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.310+0000 7fc9d6ffd700 1 -- 172.21.15.17:0/571172978 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9dc0072c0 con 0x7fc9e41434b0 2022-08-01T19:35:56.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.310+0000 7fc9ea562700 1 -- 172.21.15.17:0/571172978 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e414ffb0 con 0x7fc9e41434b0 2022-08-01T19:35:56.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.310+0000 7fc9ea562700 1 -- 172.21.15.17:0/571172978 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e4153d80 con 0x7fc9e41434b0 2022-08-01T19:35:56.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.314+0000 7fc9d6ffd700 1 -- 172.21.15.17:0/571172978 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc9dc01a060 con 0x7fc9e41434b0 2022-08-01T19:35:56.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.314+0000 7fc9d6ffd700 1 --2- 172.21.15.17:0/571172978 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9c0053aa0 0x7fc9c0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:56.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.314+0000 7fc9d6ffd700 1 -- 172.21.15.17:0/571172978 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fc9dc06b830 con 0x7fc9e41434b0 2022-08-01T19:35:56.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.314+0000 7fc9e9560700 1 --2- 172.21.15.17:0/571172978 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9c0053aa0 0x7fc9c0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:56.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.314+0000 7fc9e9560700 1 --2- 172.21.15.17:0/571172978 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9c0053aa0 0x7fc9c0055f60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fc9e0008000 tx=0x7fc9e0008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:56.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.318+0000 7fc9d6ffd700 1 -- 172.21.15.17:0/571172978 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc9dc0428d0 con 0x7fc9e41434b0 2022-08-01T19:35:56.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.446+0000 7fc9ea562700 1 -- 172.21.15.17:0/571172978 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc9e4066af0 con 0x7fc9e41434b0 2022-08-01T19:35:56.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.454+0000 7fc9d6ffd700 1 -- 172.21.15.17:0/571172978 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v4)=0 v4) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc9dc018030 con 0x7fc9e41434b0 2022-08-01T19:35:56.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.458+0000 7fc9ea562700 1 -- 172.21.15.17:0/571172978 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9c0053aa0 msgr2=0x7fc9c0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:56.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.458+0000 7fc9ea562700 1 --2- 172.21.15.17:0/571172978 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9c0053aa0 0x7fc9c0055f60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fc9e0008000 tx=0x7fc9e0008040).stop 2022-08-01T19:35:56.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.458+0000 7fc9ea562700 1 -- 172.21.15.17:0/571172978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9e41434b0 msgr2=0x7fc9e413e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:56.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.458+0000 7fc9ea562700 1 --2- 172.21.15.17:0/571172978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9e41434b0 0x7fc9e413e2a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7fc9dc001250 tx=0x7fc9dc001360).stop 2022-08-01T19:35:56.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.458+0000 7fc9ea562700 1 -- 172.21.15.17:0/571172978 shutdown_connections 2022-08-01T19:35:56.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.458+0000 7fc9ea562700 1 --2- 172.21.15.17:0/571172978 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9c0053aa0 0x7fc9c0055f60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:56.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.458+0000 7fc9ea562700 1 --2- 172.21.15.17:0/571172978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9e41434b0 0x7fc9e413e2a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:56.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.458+0000 7fc9ea562700 1 --2- 172.21.15.17:0/571172978 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9e40b31d0 0x7fc9e413c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:56.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.458+0000 7fc9ea562700 1 --2- 172.21.15.17:0/571172978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9e40a9da0 0x7fc9e4154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:56.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.458+0000 7fc9ea562700 1 -- 172.21.15.17:0/571172978 >> 172.21.15.17:0/571172978 conn(0x7fc9e4063870 msgr2=0x7fc9e4065f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:56.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.458+0000 7fc9ea562700 1 -- 172.21.15.17:0/571172978 shutdown_connections 2022-08-01T19:35:56.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.458+0000 7fc9ea562700 1 -- 172.21.15.17:0/571172978 wait complete. 2022-08-01T19:35:56.476 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T19:35:56.477 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:35:56.495 DEBUG:teuthology.orchestra.run.smithi077:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:35:56.516 DEBUG:teuthology.orchestra.run.smithi085:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:35:56.538 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:35:56.560 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T19:35:56.561 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:35:56.562 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:35:56.562 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:35:56.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:35:56.590 INFO:teuthology.orchestra.run.smithi085.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-08-01T19:35:56.591 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-08-01T19:35:56.592 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:35:56.593 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:35:56.594 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:35:56.608 INFO:teuthology.orchestra.run.smithi190.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T19:35:56.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:35:56.610 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-08-01T19:35:56.611 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T19:35:56.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.718+0000 7f581c62c700 1 Processor -- start 2022-08-01T19:35:56.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.718+0000 7f581c62c700 1 -- start start 2022-08-01T19:35:56.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.718+0000 7f581c62c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f58140b0630 0x7f58140adec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:56.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.718+0000 7f581c62c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f58140aa2b0 0x7f58140ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:56.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.718+0000 7f581c62c700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f58140b0a10 0x7f58141438a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:56.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.718+0000 7f581c62c700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58140b2c80 con 0x7f58140aa2b0 2022-08-01T19:35:56.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.718+0000 7f581c62c700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58140b2900 con 0x7f58140b0a10 2022-08-01T19:35:56.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.718+0000 7f581c62c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58140b2eb0 con 0x7f58140b0630 2022-08-01T19:35:56.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.718+0000 7f5819bc7700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f58140aa2b0 0x7f58140ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:56.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.718+0000 7f5819bc7700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f58140aa2b0 0x7f58140ae3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38098/0 (socket says 172.21.15.17:38098) 2022-08-01T19:35:56.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.718+0000 7f5819bc7700 1 -- 172.21.15.17:0/3712136768 learned_addr learned my addr 172.21.15.17:0/3712136768 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:56.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.718+0000 7f581abc9700 1 --2- 172.21.15.17:0/3712136768 >> v2:172.21.15.77:3300/0 conn(0x7f58140b0a10 0x7f58141438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:56.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.718+0000 7f581a3c8700 1 --2- 172.21.15.17:0/3712136768 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f58140b0630 0x7f58140adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:56.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.718+0000 7f5819bc7700 1 -- 172.21.15.17:0/3712136768 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f58140b0630 msgr2=0x7f58140adec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:56.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.718+0000 7f5819bc7700 1 --2- 172.21.15.17:0/3712136768 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f58140b0630 0x7f58140adec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:56.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.718+0000 7f5819bc7700 1 -- 172.21.15.17:0/3712136768 >> v2:172.21.15.77:3300/0 conn(0x7f58140b0a10 msgr2=0x7f58141438a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:56.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.718+0000 7f5819bc7700 1 --2- 172.21.15.17:0/3712136768 >> v2:172.21.15.77:3300/0 conn(0x7f58140b0a10 0x7f58141438a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:56.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.718+0000 7f5819bc7700 1 -- 172.21.15.17:0/3712136768 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58140a9d00 con 0x7f58140aa2b0 2022-08-01T19:35:56.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.718+0000 7f581abc9700 1 --2- 172.21.15.17:0/3712136768 >> v2:172.21.15.77:3300/0 conn(0x7f58140b0a10 0x7f58141438a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:35:56.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.718+0000 7f5819bc7700 1 --2- 172.21.15.17:0/3712136768 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58140aa2b0 0x7f58140ae3b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f58140b2bd0 tx=0x7f5810006eb0).ready entity=mon.0 client_cookie=bb3bcf5b3a689032 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:56.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.718+0000 7f58193c6700 1 -- 172.21.15.17:0/3712136768 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f581000d570 con 0x7f58140aa2b0 2022-08-01T19:35:56.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.718+0000 7f58193c6700 1 -- 172.21.15.17:0/3712136768 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5810000e90 con 0x7f58140aa2b0 2022-08-01T19:35:56.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.718+0000 7f58193c6700 1 -- 172.21.15.17:0/3712136768 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f581000d8c0 con 0x7f58140aa2b0 2022-08-01T19:35:56.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.718+0000 7f581c62c700 1 -- 172.21.15.17:0/3712136768 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58140aa2b0 msgr2=0x7f58140ae3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:56.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.718+0000 7f581c62c700 1 --2- 172.21.15.17:0/3712136768 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58140aa2b0 0x7f58140ae3b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f58140b2bd0 tx=0x7f5810006eb0).stop 2022-08-01T19:35:56.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.718+0000 7f581c62c700 1 -- 172.21.15.17:0/3712136768 shutdown_connections 2022-08-01T19:35:56.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.718+0000 7f581c62c700 1 --2- 172.21.15.17:0/3712136768 >> v2:172.21.15.77:3300/0 conn(0x7f58140b0a10 0x7f58141438a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:56.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.718+0000 7f581c62c700 1 --2- 172.21.15.17:0/3712136768 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58140aa2b0 0x7f58140ae3b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:56.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.718+0000 7f581c62c700 1 --2- 172.21.15.17:0/3712136768 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f58140b0630 0x7f58140adec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:56.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.718+0000 7f581c62c700 1 -- 172.21.15.17:0/3712136768 >> 172.21.15.17:0/3712136768 conn(0x7f5814060950 msgr2=0x7f5814060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:56.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.718+0000 7f581c62c700 1 -- 172.21.15.17:0/3712136768 shutdown_connections 2022-08-01T19:35:56.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.718+0000 7f581c62c700 1 -- 172.21.15.17:0/3712136768 wait complete. 2022-08-01T19:35:56.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.722+0000 7f581c62c700 1 Processor -- start 2022-08-01T19:35:56.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.722+0000 7f581c62c700 1 -- start start 2022-08-01T19:35:56.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.722+0000 7f581c62c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58140aa2b0 0x7f5814140b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:56.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.722+0000 7f581c62c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58140b0630 0x7f5814130990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:56.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.722+0000 7f581c62c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f58140b0a10 0x7f5814130e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:56.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.722+0000 7f581c62c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58141334f0 con 0x7f58140aa2b0 2022-08-01T19:35:56.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.722+0000 7f581c62c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5814133720 con 0x7f58140b0630 2022-08-01T19:35:56.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.722+0000 7f581c62c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58141339c0 con 0x7f58140b0a10 2022-08-01T19:35:56.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.722+0000 7f581a3c8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58140aa2b0 0x7f5814140b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:56.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.722+0000 7f581a3c8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58140aa2b0 0x7f5814140b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38102/0 (socket says 172.21.15.17:38102) 2022-08-01T19:35:56.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.722+0000 7f581a3c8700 1 -- 172.21.15.17:0/703154988 learned_addr learned my addr 172.21.15.17:0/703154988 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:56.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.722+0000 7f581abc9700 1 --2- 172.21.15.17:0/703154988 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f58140b0a10 0x7f5814130e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:56.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.722+0000 7f5819bc7700 1 --2- 172.21.15.17:0/703154988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58140b0630 0x7f5814130990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:56.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.722+0000 7f581a3c8700 1 -- 172.21.15.17:0/703154988 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f58140b0a10 msgr2=0x7f5814130e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:56.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.726+0000 7f581a3c8700 1 --2- 172.21.15.17:0/703154988 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f58140b0a10 0x7f5814130e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:56.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.726+0000 7f581a3c8700 1 -- 172.21.15.17:0/703154988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58140b0630 msgr2=0x7f5814130990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:56.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.726+0000 7f581a3c8700 1 --2- 172.21.15.17:0/703154988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58140b0630 0x7f5814130990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:56.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.726+0000 7f581a3c8700 1 -- 172.21.15.17:0/703154988 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581000b1e0 con 0x7f58140aa2b0 2022-08-01T19:35:56.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.726+0000 7f581a3c8700 1 --2- 172.21.15.17:0/703154988 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58140aa2b0 0x7f5814140b40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f58080030b0 tx=0x7f5808008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:56.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.726+0000 7f58077fe700 1 -- 172.21.15.17:0/703154988 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5808011780 con 0x7f58140aa2b0 2022-08-01T19:35:56.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.726+0000 7f58077fe700 1 -- 172.21.15.17:0/703154988 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58080118e0 con 0x7f58140aa2b0 2022-08-01T19:35:56.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.726+0000 7f581c62c700 1 -- 172.21.15.17:0/703154988 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5814133c60 con 0x7f58140aa2b0 2022-08-01T19:35:56.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.726+0000 7f58077fe700 1 -- 172.21.15.17:0/703154988 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5808011c40 con 0x7f58140aa2b0 2022-08-01T19:35:56.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.726+0000 7f581c62c700 1 -- 172.21.15.17:0/703154988 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f581413a4c0 con 0x7f58140aa2b0 2022-08-01T19:35:56.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.726+0000 7f581c62c700 1 -- 172.21.15.17:0/703154988 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5814155b30 con 0x7f58140aa2b0 2022-08-01T19:35:56.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.726+0000 7f58077fe700 1 -- 172.21.15.17:0/703154988 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f580801c460 con 0x7f58140aa2b0 2022-08-01T19:35:56.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.730+0000 7f58077fe700 1 --2- 172.21.15.17:0/703154988 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f57f40539e0 0x7f57f4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:56.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.730+0000 7f58077fe700 1 -- 172.21.15.17:0/703154988 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f580806c1c0 con 0x7f58140aa2b0 2022-08-01T19:35:56.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.730+0000 7f5819bc7700 1 --2- 172.21.15.17:0/703154988 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f57f40539e0 0x7f57f4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:56.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.730+0000 7f5819bc7700 1 --2- 172.21.15.17:0/703154988 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f57f40539e0 0x7f57f4055ea0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f581001b000 tx=0x7f5810015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:56.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.734+0000 7f58077fe700 1 -- 172.21.15.17:0/703154988 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f58080430f0 con 0x7f58140aa2b0 2022-08-01T19:35:56.870 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.866+0000 7f581c62c700 1 -- 172.21.15.17:0/703154988 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f58140574e0 con 0x7f58140aa2b0 2022-08-01T19:35:56.871 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.866+0000 7f58077fe700 1 -- 172.21.15.17:0/703154988 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v38) v1 ==== 81+0+29219 (secure 0 0 0) 0x7f5808046710 con 0x7f58140aa2b0 2022-08-01T19:35:56.872 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:35:56.872 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:35:56.873 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 38, 2022-08-01T19:35:56.873 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", 2022-08-01T19:35:56.873 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2022-08-01T19:33:50.105743+0000", 2022-08-01T19:35:56.874 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2022-08-01T19:35:54.146310+0000", 2022-08-01T19:35:56.874 INFO:teuthology.orchestra.run.smithi017.stdout: "last_up_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:35:56.874 INFO:teuthology.orchestra.run.smithi017.stdout: "last_in_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:35:56.875 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T19:35:56.875 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_num": 5799936, 2022-08-01T19:35:56.875 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_set": [ 2022-08-01T19:35:56.875 INFO:teuthology.orchestra.run.smithi017.stdout: "pglog_hardlimit", 2022-08-01T19:35:56.875 INFO:teuthology.orchestra.run.smithi017.stdout: "purged_snapdirs", 2022-08-01T19:35:56.876 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_deletes", 2022-08-01T19:35:56.876 INFO:teuthology.orchestra.run.smithi017.stdout: "sortbitwise" 2022-08-01T19:35:56.876 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:35:56.876 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_version": 6, 2022-08-01T19:35:56.877 INFO:teuthology.orchestra.run.smithi017.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T19:35:56.877 INFO:teuthology.orchestra.run.smithi017.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T19:35:56.877 INFO:teuthology.orchestra.run.smithi017.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T19:35:56.877 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_snapshot": "", 2022-08-01T19:35:56.878 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_max": 6, 2022-08-01T19:35:56.878 INFO:teuthology.orchestra.run.smithi017.stdout: "max_osd": 8, 2022-08-01T19:35:56.878 INFO:teuthology.orchestra.run.smithi017.stdout: "require_min_compat_client": "jewel", 2022-08-01T19:35:56.878 INFO:teuthology.orchestra.run.smithi017.stdout: "min_compat_client": "jewel", 2022-08-01T19:35:56.879 INFO:teuthology.orchestra.run.smithi017.stdout: "require_osd_release": "octopus", 2022-08-01T19:35:56.879 INFO:teuthology.orchestra.run.smithi017.stdout: "pools": [ 2022-08-01T19:35:56.879 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.879 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 1, 2022-08-01T19:35:56.879 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "device_health_metrics", 2022-08-01T19:35:56.880 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:33:53.001841+0000", 2022-08-01T19:35:56.880 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:35:56.880 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:35:56.880 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:35:56.881 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:35:56.881 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:35:56.881 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:35:56.881 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:35:56.881 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:35:56.882 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 1, 2022-08-01T19:35:56.882 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 1, 2022-08-01T19:35:56.882 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 1, 2022-08-01T19:35:56.882 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 1, 2022-08-01T19:35:56.883 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 1, 2022-08-01T19:35:56.883 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:35:56.883 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:35:56.883 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:35:56.883 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:35:56.884 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:35:56.884 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:35:56.884 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:35:56.884 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.885 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "16", 2022-08-01T19:35:56.885 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:35:56.885 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:35:56.885 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:35:56.886 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:35:56.886 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:35:56.886 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:35:56.886 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:35:56.887 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:35:56.887 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:35:56.887 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:35:56.887 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:35:56.887 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:35:56.888 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:35:56.888 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:35:56.888 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:35:56.888 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:35:56.888 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:35:56.889 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:35:56.889 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:35:56.889 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:35:56.889 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:35:56.890 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:35:56.890 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:35:56.890 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:35:56.890 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:35:56.891 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:35:56.891 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.891 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:35:56.891 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:35:56.891 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:35:56.892 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:35:56.892 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:35:56.892 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:35:56.892 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:35:56.893 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:35:56.893 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:35:56.893 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:35:56.893 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:35:56.893 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:35:56.894 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 1 2022-08-01T19:35:56.894 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.894 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:35:56.894 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_devicehealth": {} 2022-08-01T19:35:56.895 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:56.895 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.895 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.895 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 2, 2022-08-01T19:35:56.896 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "rbd", 2022-08-01T19:35:56.896 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:34:01.553098+0000", 2022-08-01T19:35:56.896 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:35:56.896 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:35:56.896 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:35:56.897 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:35:56.897 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:35:56.897 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:35:56.897 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:35:56.898 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:35:56.898 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:35:56.898 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:35:56.898 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:35:56.899 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:35:56.899 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:35:56.899 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:35:56.899 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:35:56.899 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:35:56.900 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:35:56.900 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:35:56.900 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:35:56.900 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:35:56.901 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.901 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "19", 2022-08-01T19:35:56.901 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:35:56.901 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:35:56.901 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:35:56.902 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:35:56.902 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:35:56.902 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:35:56.902 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:35:56.903 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:35:56.903 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:35:56.903 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:35:56.903 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:35:56.903 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:35:56.904 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:35:56.904 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:35:56.904 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:35:56.904 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:35:56.905 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:35:56.905 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:35:56.905 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:35:56.905 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:35:56.905 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:35:56.906 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:35:56.906 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:35:56.906 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:35:56.906 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:35:56.907 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:35:56.907 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.907 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:35:56.907 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:35:56.907 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:35:56.908 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:35:56.908 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:35:56.908 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:35:56.908 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:35:56.909 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:35:56.909 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:35:56.909 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:35:56.909 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:35:56.909 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:35:56.910 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:35:56.910 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd": {} 2022-08-01T19:35:56.910 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:56.910 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:56.911 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:35:56.911 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2022-08-01T19:35:56.911 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.911 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:35:56.912 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a962f14f-cf16-4fa8-84ff-c85325db9132", 2022-08-01T19:35:56.912 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:35:56.912 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:35:56.912 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:35:56.913 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:35:56.913 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:35:56.913 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:35:56.913 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:35:56.913 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 33, 2022-08-01T19:35:56.914 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:35:56.914 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:35:56.914 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:35:56.914 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:56.915 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.915 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:56.915 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6800", 2022-08-01T19:35:56.915 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:35:56.915 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.916 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.916 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:56.916 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6801", 2022-08-01T19:35:56.916 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:35:56.917 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:56.917 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:56.917 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.917 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:35:56.917 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:56.918 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.918 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:56.918 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6802", 2022-08-01T19:35:56.918 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:35:56.919 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.919 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.919 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:56.919 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6803", 2022-08-01T19:35:56.919 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:35:56.920 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:56.920 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:56.920 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.920 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:35:56.921 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:56.921 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.921 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:56.921 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6806", 2022-08-01T19:35:56.922 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:35:56.922 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.922 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.922 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:56.922 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6807", 2022-08-01T19:35:56.923 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:35:56.923 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:56.923 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:56.923 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.924 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:35:56.924 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:56.924 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.924 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:56.924 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6804", 2022-08-01T19:35:56.925 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:35:56.925 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.925 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.925 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:56.926 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6805", 2022-08-01T19:35:56.926 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:35:56.926 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:56.926 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:56.926 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.927 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6801/23008", 2022-08-01T19:35:56.928 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6803/23008", 2022-08-01T19:35:56.929 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6807/23008", 2022-08-01T19:35:56.929 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6805/23008", 2022-08-01T19:35:56.929 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:35:56.929 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:35:56.930 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:35:56.930 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:56.930 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.931 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.931 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:35:56.931 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "805f7051-a527-4e98-8844-b6e406a63248", 2022-08-01T19:35:56.931 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:35:56.932 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:35:56.932 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:35:56.932 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:35:56.932 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:35:56.933 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:35:56.933 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:35:56.933 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 33, 2022-08-01T19:35:56.933 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:35:56.933 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:35:56.934 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:35:56.934 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:56.934 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.934 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:56.935 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6808", 2022-08-01T19:35:56.935 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:35:56.935 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.935 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.935 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:56.936 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6809", 2022-08-01T19:35:56.936 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:35:56.936 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:56.936 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:56.937 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.937 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:35:56.937 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:56.937 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.938 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:56.938 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6810", 2022-08-01T19:35:56.939 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:35:56.939 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.939 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.940 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:56.940 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6811", 2022-08-01T19:35:56.941 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:35:56.941 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:56.942 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:56.942 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.942 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:35:56.943 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:56.943 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.944 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:56.944 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6814", 2022-08-01T19:35:56.944 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:35:56.944 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.945 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.945 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:56.945 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6815", 2022-08-01T19:35:56.945 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:35:56.945 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:56.946 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:56.946 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.946 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:35:56.946 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:56.947 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.947 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:56.947 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6812", 2022-08-01T19:35:56.947 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:35:56.947 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.948 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.948 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:56.948 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6813", 2022-08-01T19:35:56.948 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:35:56.949 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:56.949 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:56.949 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.949 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6809/23007", 2022-08-01T19:35:56.950 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6811/23007", 2022-08-01T19:35:56.950 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6815/23007", 2022-08-01T19:35:56.950 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6813/23007", 2022-08-01T19:35:56.950 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:35:56.950 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:35:56.951 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:35:56.951 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:56.951 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.951 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.952 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:35:56.952 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "89f7f3df-7c10-48e1-8b69-9e4243923a61", 2022-08-01T19:35:56.952 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:35:56.952 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:35:56.953 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:35:56.953 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:35:56.953 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:35:56.953 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:35:56.953 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:35:56.954 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 32, 2022-08-01T19:35:56.954 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:35:56.955 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:35:56.955 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:35:56.955 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:56.955 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.955 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:56.956 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6817", 2022-08-01T19:35:56.956 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:35:56.956 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.956 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.957 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:56.957 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6819", 2022-08-01T19:35:56.957 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:35:56.957 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:56.958 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:56.958 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.958 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:35:56.958 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:56.959 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.959 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:56.959 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6822", 2022-08-01T19:35:56.959 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:35:56.959 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.960 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.960 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:56.960 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6824", 2022-08-01T19:35:56.960 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:35:56.961 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:56.961 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:56.961 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.961 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:35:56.961 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:56.962 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.962 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:56.962 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6830", 2022-08-01T19:35:56.962 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:35:56.963 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.963 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.963 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:56.963 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6831", 2022-08-01T19:35:56.963 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:35:56.964 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:56.964 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:56.964 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.965 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:35:56.965 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:56.965 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.965 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:56.966 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6826", 2022-08-01T19:35:56.966 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:35:56.966 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.966 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.967 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:56.967 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6829", 2022-08-01T19:35:56.967 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:35:56.967 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:56.967 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:56.968 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.968 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6819/23010", 2022-08-01T19:35:56.968 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6824/23010", 2022-08-01T19:35:56.968 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6831/23010", 2022-08-01T19:35:56.968 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6829/23010", 2022-08-01T19:35:56.969 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:35:56.969 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:35:56.969 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:35:56.969 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:56.970 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.970 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.970 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:35:56.970 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "96123468-aabc-4a64-909e-a192c369090d", 2022-08-01T19:35:56.971 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:35:56.971 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:35:56.971 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:35:56.971 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:35:56.971 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:35:56.972 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:35:56.972 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:35:56.972 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 33, 2022-08-01T19:35:56.972 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:35:56.973 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:35:56.973 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:35:56.973 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:56.973 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.973 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:56.974 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6816", 2022-08-01T19:35:56.974 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:35:56.974 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.974 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.975 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:56.975 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6818", 2022-08-01T19:35:56.975 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:35:56.976 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:56.976 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:56.976 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.976 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:35:56.977 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:56.977 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.977 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:56.977 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6820", 2022-08-01T19:35:56.978 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:35:56.978 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.978 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.978 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:56.979 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6821", 2022-08-01T19:35:56.979 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:35:56.979 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:56.979 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:56.980 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.980 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:35:56.980 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:56.980 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.980 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:56.981 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6827", 2022-08-01T19:35:56.981 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:35:56.981 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.981 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.982 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:56.982 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6828", 2022-08-01T19:35:56.982 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:35:56.982 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:56.983 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:56.983 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.983 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:35:56.983 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:56.983 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.984 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:56.984 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6823", 2022-08-01T19:35:56.984 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:35:56.984 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.985 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.985 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:56.985 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6825", 2022-08-01T19:35:56.985 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:35:56.985 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:56.986 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:56.986 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.986 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6818/23009", 2022-08-01T19:35:56.987 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6821/23009", 2022-08-01T19:35:56.987 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6828/23009", 2022-08-01T19:35:56.987 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6825/23009", 2022-08-01T19:35:56.987 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:35:56.988 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:35:56.988 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:35:56.988 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:56.988 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.988 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.989 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:35:56.989 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a6e26665-e04b-46ec-ab66-e912acbf044c", 2022-08-01T19:35:56.989 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:35:56.989 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:35:56.990 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:35:56.990 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:35:56.990 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:35:56.991 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:35:56.991 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:35:56.991 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 33, 2022-08-01T19:35:56.991 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:35:56.992 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:35:56.992 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:35:56.992 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:56.992 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.992 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:56.993 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6810", 2022-08-01T19:35:56.993 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:35:56.993 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.993 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.994 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:56.994 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6811", 2022-08-01T19:35:56.994 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:35:56.994 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:56.995 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:56.995 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.995 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:35:56.995 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:56.995 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.996 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:56.996 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6812", 2022-08-01T19:35:56.996 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:35:56.996 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.997 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.997 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:56.997 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6813", 2022-08-01T19:35:56.997 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:35:56.997 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:56.998 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:56.998 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:56.998 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:35:56.998 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:56.999 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:56.999 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:56.999 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6816", 2022-08-01T19:35:56.999 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:35:56.999 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.000 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.000 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:57.000 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6817", 2022-08-01T19:35:57.000 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:35:57.001 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:57.001 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:57.001 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.001 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:35:57.001 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:57.002 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.002 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:57.002 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6814", 2022-08-01T19:35:57.002 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:35:57.003 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.003 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.003 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:57.003 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6815", 2022-08-01T19:35:57.003 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:35:57.004 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:57.004 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:57.004 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.004 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6811/23584", 2022-08-01T19:35:57.005 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6813/23584", 2022-08-01T19:35:57.005 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6817/23584", 2022-08-01T19:35:57.005 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6815/23584", 2022-08-01T19:35:57.005 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:35:57.005 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:35:57.006 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:35:57.006 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:57.006 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.006 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.007 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:35:57.007 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "502c0034-18ee-44ac-87ff-19b9dddb66a0", 2022-08-01T19:35:57.007 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:35:57.007 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:35:57.007 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:35:57.008 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:35:57.008 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:35:57.008 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:35:57.008 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:35:57.009 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 33, 2022-08-01T19:35:57.009 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:35:57.009 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:35:57.009 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:35:57.010 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:57.010 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.010 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:57.010 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6802", 2022-08-01T19:35:57.011 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:35:57.011 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.011 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.011 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:57.011 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6803", 2022-08-01T19:35:57.012 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:35:57.012 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:57.012 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:57.012 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.013 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:35:57.013 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:57.013 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.013 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:57.013 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6804", 2022-08-01T19:35:57.014 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:35:57.014 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.014 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.014 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:57.015 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6805", 2022-08-01T19:35:57.015 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:35:57.015 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:57.015 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:57.016 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.016 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:35:57.016 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:57.016 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.017 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:57.017 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6808", 2022-08-01T19:35:57.017 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:35:57.017 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.018 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.018 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:57.018 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6809", 2022-08-01T19:35:57.018 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:35:57.019 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:57.019 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:57.019 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.019 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:35:57.019 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:57.020 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.020 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:57.020 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6806", 2022-08-01T19:35:57.020 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:35:57.021 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.021 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.021 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:57.021 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6807", 2022-08-01T19:35:57.021 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:35:57.022 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:57.022 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:57.022 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.022 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6803/23585", 2022-08-01T19:35:57.023 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6805/23585", 2022-08-01T19:35:57.023 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6809/23585", 2022-08-01T19:35:57.023 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6807/23585", 2022-08-01T19:35:57.023 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:35:57.023 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:35:57.024 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:35:57.024 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:57.024 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.024 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.024 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:35:57.025 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "ac4e755a-7fe7-46cb-8407-852d5af9ffc5", 2022-08-01T19:35:57.025 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:35:57.025 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:35:57.025 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:35:57.026 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:35:57.026 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:35:57.026 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:35:57.026 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:35:57.026 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 32, 2022-08-01T19:35:57.027 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:35:57.027 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:35:57.027 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:35:57.027 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:57.028 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.028 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:57.028 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6820", 2022-08-01T19:35:57.028 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:35:57.028 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.029 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.029 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:57.029 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6821", 2022-08-01T19:35:57.029 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:35:57.029 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:57.030 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:57.030 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.030 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:35:57.030 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:57.031 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.031 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:57.031 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6823", 2022-08-01T19:35:57.031 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:35:57.031 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.032 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.032 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:57.032 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6825", 2022-08-01T19:35:57.032 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:35:57.033 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:57.033 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:57.033 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.033 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:35:57.033 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:57.034 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.034 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:57.034 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6831", 2022-08-01T19:35:57.034 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:35:57.034 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.035 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.035 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:57.035 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6833", 2022-08-01T19:35:57.035 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:35:57.036 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:57.036 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:57.036 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.036 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:35:57.036 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:57.037 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.037 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:57.037 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6827", 2022-08-01T19:35:57.037 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:35:57.038 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.038 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.038 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:57.038 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6829", 2022-08-01T19:35:57.038 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:35:57.039 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:57.039 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:57.039 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.039 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6821/23587", 2022-08-01T19:35:57.040 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6825/23587", 2022-08-01T19:35:57.040 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6833/23587", 2022-08-01T19:35:57.040 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6829/23587", 2022-08-01T19:35:57.040 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:35:57.040 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:35:57.041 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:35:57.041 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:57.041 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.041 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.041 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:35:57.042 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "874761e2-6410-40fc-b60c-e58abe5ce489", 2022-08-01T19:35:57.042 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:35:57.042 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:35:57.042 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:35:57.043 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:35:57.043 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:35:57.043 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:35:57.043 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:35:57.043 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 33, 2022-08-01T19:35:57.044 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:35:57.044 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:35:57.044 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:35:57.044 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:57.044 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.045 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:57.045 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6818", 2022-08-01T19:35:57.045 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:35:57.045 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.046 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.046 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:57.046 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6819", 2022-08-01T19:35:57.046 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:35:57.046 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:57.047 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:57.047 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.047 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:35:57.047 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:57.047 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.048 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:57.048 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6822", 2022-08-01T19:35:57.048 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:35:57.048 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.049 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.049 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:57.049 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6824", 2022-08-01T19:35:57.049 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:35:57.049 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:57.050 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:57.050 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.050 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:35:57.050 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:57.051 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.051 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:57.051 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6830", 2022-08-01T19:35:57.051 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:35:57.052 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.052 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.052 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:57.052 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6832", 2022-08-01T19:35:57.052 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:35:57.053 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:57.053 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:57.053 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.053 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:35:57.054 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:35:57.054 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.054 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:35:57.054 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6826", 2022-08-01T19:35:57.054 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:35:57.055 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.055 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.055 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:35:57.055 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6828", 2022-08-01T19:35:57.056 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:35:57.056 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:57.056 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:57.056 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.056 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6819/23586", 2022-08-01T19:35:57.057 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6824/23586", 2022-08-01T19:35:57.057 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6832/23586", 2022-08-01T19:35:57.057 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6828/23586", 2022-08-01T19:35:57.057 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:35:57.057 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:35:57.058 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:35:57.058 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:57.058 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:57.058 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:35:57.059 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_xinfo": [ 2022-08-01T19:35:57.059 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.059 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:35:57.059 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:35:57.060 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:35:57.060 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:35:57.060 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:35:57.060 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:35:57.060 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.588817+0000", 2022-08-01T19:35:57.061 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:35:57.061 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.061 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.061 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:35:57.062 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:35:57.062 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:35:57.062 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:35:57.062 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:35:57.062 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:35:57.063 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.782753+0000", 2022-08-01T19:35:57.063 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:35:57.063 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.063 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.064 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:35:57.064 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:35:57.064 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:35:57.064 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:35:57.064 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:35:57.065 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:35:57.065 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.837924+0000", 2022-08-01T19:35:57.065 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:35:57.065 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.065 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.066 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:35:57.066 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:35:57.066 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:35:57.066 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:35:57.067 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:35:57.067 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:35:57.067 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.839948+0000", 2022-08-01T19:35:57.067 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:35:57.068 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.068 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.068 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:35:57.068 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:35:57.068 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:35:57.069 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:35:57.069 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:35:57.069 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:35:57.069 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.112801+0000", 2022-08-01T19:35:57.070 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:35:57.070 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.070 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.070 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:35:57.070 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:35:57.071 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:35:57.071 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:35:57.071 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:35:57.071 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:35:57.072 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.106704+0000", 2022-08-01T19:35:57.072 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:35:57.072 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.072 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.072 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:35:57.073 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:35:57.073 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:35:57.073 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:35:57.073 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:35:57.074 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:35:57.074 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.893170+0000", 2022-08-01T19:35:57.074 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:35:57.074 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.074 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:57.075 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:35:57.075 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:35:57.075 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:35:57.075 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:35:57.075 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:35:57.076 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:35:57.076 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.886947+0000", 2022-08-01T19:35:57.076 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:35:57.076 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:57.077 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:35:57.077 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap": [], 2022-08-01T19:35:57.077 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap_items": [], 2022-08-01T19:35:57.077 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_temp": [], 2022-08-01T19:35:57.077 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_temp": [], 2022-08-01T19:35:57.078 INFO:teuthology.orchestra.run.smithi017.stdout: "blacklist": { 2022-08-01T19:35:57.078 INFO:teuthology.orchestra.run.smithi017.stdout: "172.21.15.17:6835/3876538769": "2022-08-02T19:35:50.513326+0000", 2022-08-01T19:35:57.078 INFO:teuthology.orchestra.run.smithi017.stdout: "172.21.15.17:6834/3876538769": "2022-08-02T19:35:50.513326+0000" 2022-08-01T19:35:57.078 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.079 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profiles": { 2022-08-01T19:35:57.079 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_data": { 2022-08-01T19:35:57.079 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-device-class": "", 2022-08-01T19:35:57.079 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:35:57.079 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-root": "default", 2022-08-01T19:35:57.080 INFO:teuthology.orchestra.run.smithi017.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T19:35:57.080 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:35:57.080 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "2", 2022-08-01T19:35:57.080 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:35:57.080 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:35:57.081 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van", 2022-08-01T19:35:57.081 INFO:teuthology.orchestra.run.smithi017.stdout: "w": "8" 2022-08-01T19:35:57.081 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.081 INFO:teuthology.orchestra.run.smithi017.stdout: "default": { 2022-08-01T19:35:57.082 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:35:57.082 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:35:57.082 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "1", 2022-08-01T19:35:57.082 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:35:57.082 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:35:57.083 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van" 2022-08-01T19:35:57.083 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:57.083 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:57.083 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps_queue": [], 2022-08-01T19:35:57.084 INFO:teuthology.orchestra.run.smithi017.stdout: "new_removed_snaps": [], 2022-08-01T19:35:57.084 INFO:teuthology.orchestra.run.smithi017.stdout: "new_purged_snaps": [], 2022-08-01T19:35:57.084 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_node_flags": {}, 2022-08-01T19:35:57.084 INFO:teuthology.orchestra.run.smithi017.stdout: "device_class_flags": {} 2022-08-01T19:35:57.084 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:35:57.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.870+0000 7f581c62c700 1 -- 172.21.15.17:0/703154988 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f57f40539e0 msgr2=0x7f57f4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:57.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.870+0000 7f581c62c700 1 --2- 172.21.15.17:0/703154988 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f57f40539e0 0x7f57f4055ea0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f581001b000 tx=0x7f5810015000).stop 2022-08-01T19:35:57.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.870+0000 7f581c62c700 1 -- 172.21.15.17:0/703154988 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58140aa2b0 msgr2=0x7f5814140b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:57.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.870+0000 7f581c62c700 1 --2- 172.21.15.17:0/703154988 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58140aa2b0 0x7f5814140b40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f58080030b0 tx=0x7f5808008040).stop 2022-08-01T19:35:57.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.870+0000 7f581c62c700 1 -- 172.21.15.17:0/703154988 shutdown_connections 2022-08-01T19:35:57.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.870+0000 7f581c62c700 1 --2- 172.21.15.17:0/703154988 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f57f40539e0 0x7f57f4055ea0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:57.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.870+0000 7f581c62c700 1 --2- 172.21.15.17:0/703154988 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f58140b0a10 0x7f5814130e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:57.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.870+0000 7f581c62c700 1 --2- 172.21.15.17:0/703154988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58140b0630 0x7f5814130990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:57.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.870+0000 7f581c62c700 1 --2- 172.21.15.17:0/703154988 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58140aa2b0 0x7f5814140b40 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:57.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.870+0000 7f581c62c700 1 -- 172.21.15.17:0/703154988 >> 172.21.15.17:0/703154988 conn(0x7f5814060950 msgr2=0x7f58140b5da0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:57.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.870+0000 7f581c62c700 1 -- 172.21.15.17:0/703154988 shutdown_connections 2022-08-01T19:35:57.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:56.870+0000 7f581c62c700 1 -- 172.21.15.17:0/703154988 wait complete. 2022-08-01T19:35:57.090 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:35:57.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.198+0000 7fba5f59e700 1 Processor -- start 2022-08-01T19:35:57.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.198+0000 7fba5f59e700 1 -- start start 2022-08-01T19:35:57.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.202+0000 7fba5f59e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fba600b2710 0x7fba6014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:57.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.202+0000 7fba5f59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fba600a8c50 0x7fba6014b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:57.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.202+0000 7fba5f59e700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba600a7250 con 0x7fba600b01a0 2022-08-01T19:35:57.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.202+0000 7fba5f59e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba600a7480 con 0x7fba600b2710 2022-08-01T19:35:57.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.202+0000 7fba5f59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba600a7720 con 0x7fba600a8c50 2022-08-01T19:35:57.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.202+0000 7fba5e59c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fba600b2710 0x7fba6014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:57.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.202+0000 7fba5dd9b700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fba600b01a0 0x7fba600a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58540/0 (socket says 172.21.15.17:58540) 2022-08-01T19:35:57.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.202+0000 7fba5dd9b700 1 -- 172.21.15.17:0/3432300788 learned_addr learned my addr 172.21.15.17:0/3432300788 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:35:57.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.202+0000 7fba5ed9d700 1 --2- 172.21.15.17:0/3432300788 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fba600a8c50 0x7fba6014b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:57.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.202+0000 7fba5e59c700 1 -- 172.21.15.17:0/3432300788 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fba600a8c50 msgr2=0x7fba6014b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:57.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.202+0000 7fba5e59c700 1 --2- 172.21.15.17:0/3432300788 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fba600a8c50 0x7fba6014b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:57.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.202+0000 7fba5e59c700 1 -- 172.21.15.17:0/3432300788 >> v1:172.21.15.17:6789/0 conn(0x7fba600b01a0 legacy=0x7fba600a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:57.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.202+0000 7fba5e59c700 1 -- 172.21.15.17:0/3432300788 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba600a79c0 con 0x7fba600b2710 2022-08-01T19:35:57.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.202+0000 7fba5e59c700 1 --2- 172.21.15.17:0/3432300788 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba600b2710 0x7fba6014afd0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7fba50002f70 tx=0x7fba50006df0).ready entity=mon.0 client_cookie=cdcd564b9086b65 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:57.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.202+0000 7fba5d59a700 1 -- 172.21.15.17:0/3432300788 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba5000d570 con 0x7fba600b2710 2022-08-01T19:35:57.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.202+0000 7fba5d59a700 1 -- 172.21.15.17:0/3432300788 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba50000e90 con 0x7fba600b2710 2022-08-01T19:35:57.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.202+0000 7fba5d59a700 1 -- 172.21.15.17:0/3432300788 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba5000d8c0 con 0x7fba600b2710 2022-08-01T19:35:57.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.202+0000 7fba5f59e700 1 -- 172.21.15.17:0/3432300788 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba600b2710 msgr2=0x7fba6014afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:57.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.202+0000 7fba5f59e700 1 --2- 172.21.15.17:0/3432300788 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba600b2710 0x7fba6014afd0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7fba50002f70 tx=0x7fba50006df0).stop 2022-08-01T19:35:57.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.202+0000 7fba5f59e700 1 -- 172.21.15.17:0/3432300788 shutdown_connections 2022-08-01T19:35:57.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.202+0000 7fba5f59e700 1 --2- 172.21.15.17:0/3432300788 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fba600a8c50 0x7fba6014b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:57.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.202+0000 7fba5f59e700 1 --2- 172.21.15.17:0/3432300788 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba600b2710 0x7fba6014afd0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:57.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.202+0000 7fba5f59e700 1 -- 172.21.15.17:0/3432300788 >> 172.21.15.17:0/3432300788 conn(0x7fba60060310 msgr2=0x7fba600b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:57.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.202+0000 7fba5f59e700 1 -- 172.21.15.17:0/3432300788 shutdown_connections 2022-08-01T19:35:57.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.202+0000 7fba5f59e700 1 -- 172.21.15.17:0/3432300788 wait complete. 2022-08-01T19:35:57.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.202+0000 7fba5f59e700 1 Processor -- start 2022-08-01T19:35:57.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.206+0000 7fba5f59e700 1 -- start start 2022-08-01T19:35:57.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.206+0000 7fba5f59e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba600a8c50 0x7fba601570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:57.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.206+0000 7fba5f59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fba600b01a0 0x7fba600642d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:57.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.206+0000 7fba5f59e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba600b2710 0x7fba600b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:57.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.206+0000 7fba5f59e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba60157670 con 0x7fba600b2710 2022-08-01T19:35:57.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.206+0000 7fba5f59e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba600b3590 con 0x7fba600a8c50 2022-08-01T19:35:57.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.206+0000 7fba5f59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba600b37f0 con 0x7fba600b01a0 2022-08-01T19:35:57.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.206+0000 7fba5ed9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba600b2710 0x7fba600b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:57.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.206+0000 7fba5ed9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba600b2710 0x7fba600b4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38120/0 (socket says 172.21.15.17:38120) 2022-08-01T19:35:57.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.206+0000 7fba5ed9d700 1 -- 172.21.15.17:0/1068951183 learned_addr learned my addr 172.21.15.17:0/1068951183 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:57.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.206+0000 7fba5e59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba600a8c50 0x7fba601570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:57.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.206+0000 7fba5dd9b700 1 --2- 172.21.15.17:0/1068951183 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fba600b01a0 0x7fba600642d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:57.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.206+0000 7fba5ed9d700 1 -- 172.21.15.17:0/1068951183 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fba600b01a0 msgr2=0x7fba600642d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:57.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.206+0000 7fba5ed9d700 1 --2- 172.21.15.17:0/1068951183 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fba600b01a0 0x7fba600642d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:57.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.206+0000 7fba5ed9d700 1 -- 172.21.15.17:0/1068951183 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba600a8c50 msgr2=0x7fba601570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:57.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.206+0000 7fba5ed9d700 1 --2- 172.21.15.17:0/1068951183 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba600a8c50 0x7fba601570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:57.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.206+0000 7fba5ed9d700 1 -- 172.21.15.17:0/1068951183 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba5000b180 con 0x7fba600b2710 2022-08-01T19:35:57.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.206+0000 7fba5ed9d700 1 --2- 172.21.15.17:0/1068951183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba600b2710 0x7fba600b4dd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7fba5800c000 tx=0x7fba580091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:57.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.206+0000 7fba4f7fe700 1 -- 172.21.15.17:0/1068951183 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba58012600 con 0x7fba600b2710 2022-08-01T19:35:57.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.206+0000 7fba4f7fe700 1 -- 172.21.15.17:0/1068951183 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba58012760 con 0x7fba600b2710 2022-08-01T19:35:57.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.206+0000 7fba4f7fe700 1 -- 172.21.15.17:0/1068951183 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba58012ac0 con 0x7fba600b2710 2022-08-01T19:35:57.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.206+0000 7fba5f59e700 1 -- 172.21.15.17:0/1068951183 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba600b3a90 con 0x7fba600b2710 2022-08-01T19:35:57.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.206+0000 7fba5f59e700 1 -- 172.21.15.17:0/1068951183 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba60064950 con 0x7fba600b2710 2022-08-01T19:35:57.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.206+0000 7fba5f59e700 1 -- 172.21.15.17:0/1068951183 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba600b3c20 con 0x7fba600b2710 2022-08-01T19:35:57.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.210+0000 7fba4f7fe700 1 -- 172.21.15.17:0/1068951183 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fba58022070 con 0x7fba600b2710 2022-08-01T19:35:57.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.210+0000 7fba4f7fe700 1 --2- 172.21.15.17:0/1068951183 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fba3c053b00 0x7fba3c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:57.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.210+0000 7fba4f7fe700 1 -- 172.21.15.17:0/1068951183 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fba5806c320 con 0x7fba600b2710 2022-08-01T19:35:57.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.210+0000 7fba5e59c700 1 --2- 172.21.15.17:0/1068951183 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fba3c053b00 0x7fba3c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:57.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.210+0000 7fba5e59c700 1 --2- 172.21.15.17:0/1068951183 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fba3c053b00 0x7fba3c055fc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7fba5001a000 tx=0x7fba50014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:57.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.218+0000 7fba4f7fe700 1 -- 172.21.15.17:0/1068951183 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fba58043360 con 0x7fba600b2710 2022-08-01T19:35:57.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.382+0000 7fba5f59e700 1 -- 172.21.15.17:0/1068951183 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba600574e0 con 0x7fba600b2710 2022-08-01T19:35:57.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.382+0000 7fba4f7fe700 1 -- 172.21.15.17:0/1068951183 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+1562 (secure 0 0 0) 0x7fba58016020 con 0x7fba600b2710 2022-08-01T19:35:57.388 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:35:57.388 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":15,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2487504686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2487504686},{"type":"v1","addr":"172.21.15.17:6833","nonce":2487504686}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10},{"gid":4885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/4255478434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4255478434},{"type":"v1","addr":"172.21.15.77:6835","nonce":4255478434}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":11},{"gid":5140,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/4154415741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":4154415741},{"type":"v1","addr":"172.21.15.17:6835","nonce":4154415741}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[]} 2022-08-01T19:35:57.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.386+0000 7fba5f59e700 1 -- 172.21.15.17:0/1068951183 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fba3c053b00 msgr2=0x7fba3c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:57.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.386+0000 7fba5f59e700 1 --2- 172.21.15.17:0/1068951183 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fba3c053b00 0x7fba3c055fc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7fba5001a000 tx=0x7fba50014000).stop 2022-08-01T19:35:57.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.386+0000 7fba5f59e700 1 -- 172.21.15.17:0/1068951183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba600b2710 msgr2=0x7fba600b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:57.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.386+0000 7fba5f59e700 1 --2- 172.21.15.17:0/1068951183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba600b2710 0x7fba600b4dd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7fba5800c000 tx=0x7fba580091d0).stop 2022-08-01T19:35:57.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.386+0000 7fba5f59e700 1 -- 172.21.15.17:0/1068951183 shutdown_connections 2022-08-01T19:35:57.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.386+0000 7fba5f59e700 1 --2- 172.21.15.17:0/1068951183 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fba3c053b00 0x7fba3c055fc0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:57.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.386+0000 7fba5f59e700 1 --2- 172.21.15.17:0/1068951183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba600b2710 0x7fba600b4dd0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:57.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.386+0000 7fba5f59e700 1 --2- 172.21.15.17:0/1068951183 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fba600b01a0 0x7fba600642d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:57.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.386+0000 7fba5f59e700 1 --2- 172.21.15.17:0/1068951183 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba600a8c50 0x7fba601570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:57.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.386+0000 7fba5f59e700 1 -- 172.21.15.17:0/1068951183 >> 172.21.15.17:0/1068951183 conn(0x7fba60060310 msgr2=0x7fba600add30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:57.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.386+0000 7fba5f59e700 1 -- 172.21.15.17:0/1068951183 shutdown_connections 2022-08-01T19:35:57.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.386+0000 7fba5f59e700 1 -- 172.21.15.17:0/1068951183 wait complete. 2022-08-01T19:35:57.394 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 15 2022-08-01T19:35:57.407 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:35:57.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.506+0000 7f5043fff700 1 Processor -- start 2022-08-01T19:35:57.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.506+0000 7f5043fff700 1 -- start start 2022-08-01T19:35:57.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.506+0000 7f5043fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f50440b01a0 0x7f50440a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:57.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.506+0000 7f5043fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f50440afd60 0x7f50440a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:57.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.506+0000 7f5043fff700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50440a7250 con 0x7f50440b2710 2022-08-01T19:35:57.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.506+0000 7f5043fff700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50440a7480 con 0x7f50440b01a0 2022-08-01T19:35:57.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.506+0000 7f5043fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50440a7720 con 0x7f50440afd60 2022-08-01T19:35:57.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.506+0000 7f50427fc700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f50440b01a0 0x7f50440a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:57.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.506+0000 7f50427fc700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f50440b01a0 0x7f50440a8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38124/0 (socket says 172.21.15.17:38124) 2022-08-01T19:35:57.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.506+0000 7f50427fc700 1 -- 172.21.15.17:0/739116880 learned_addr learned my addr 172.21.15.17:0/739116880 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:57.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.506+0000 7f50427fc700 1 -- 172.21.15.17:0/739116880 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f50440afd60 msgr2=0x7f50440a8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:57.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.506+0000 7f50437fe700 1 --2- 172.21.15.17:0/739116880 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f50440afd60 0x7f50440a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:57.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.506+0000 7f50427fc700 1 --2- 172.21.15.17:0/739116880 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f50440afd60 0x7f50440a8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:57.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.506+0000 7f50427fc700 1 -- 172.21.15.17:0/739116880 >> v1:172.21.15.17:6789/0 conn(0x7f50440b2710 legacy=0x7f504414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:57.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.506+0000 7f50427fc700 1 -- 172.21.15.17:0/739116880 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50440a79c0 con 0x7f50440b01a0 2022-08-01T19:35:57.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.506+0000 7f50427fc700 1 --2- 172.21.15.17:0/739116880 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50440b01a0 0x7f50440a8540 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f503c0032f0 tx=0x7f503c009290).ready entity=mon.0 client_cookie=97dc600aaadbcb7f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:57.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.506+0000 7f50437fe700 1 --2- 172.21.15.17:0/739116880 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f50440afd60 0x7f50440a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:35:57.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.506+0000 7f5041ffb700 1 -- 172.21.15.17:0/739116880 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f503c00b8a0 con 0x7f50440b01a0 2022-08-01T19:35:57.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.506+0000 7f5041ffb700 1 -- 172.21.15.17:0/739116880 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503c00ba00 con 0x7f50440b01a0 2022-08-01T19:35:57.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.506+0000 7f5041ffb700 1 -- 172.21.15.17:0/739116880 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f503c00bbb0 con 0x7f50440b01a0 2022-08-01T19:35:57.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.506+0000 7f5043fff700 1 -- 172.21.15.17:0/739116880 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50440b01a0 msgr2=0x7f50440a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:57.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.506+0000 7f5043fff700 1 --2- 172.21.15.17:0/739116880 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50440b01a0 0x7f50440a8540 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f503c0032f0 tx=0x7f503c009290).stop 2022-08-01T19:35:57.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.506+0000 7f5043fff700 1 -- 172.21.15.17:0/739116880 shutdown_connections 2022-08-01T19:35:57.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.506+0000 7f5043fff700 1 --2- 172.21.15.17:0/739116880 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f50440afd60 0x7f50440a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:57.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.506+0000 7f5043fff700 1 --2- 172.21.15.17:0/739116880 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50440b01a0 0x7f50440a8540 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:57.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.506+0000 7f5043fff700 1 -- 172.21.15.17:0/739116880 >> 172.21.15.17:0/739116880 conn(0x7f5044060310 msgr2=0x7f50440b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:57.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.506+0000 7f5043fff700 1 -- 172.21.15.17:0/739116880 shutdown_connections 2022-08-01T19:35:57.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.506+0000 7f5043fff700 1 -- 172.21.15.17:0/739116880 wait complete. 2022-08-01T19:35:57.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.510+0000 7f5043fff700 1 Processor -- start 2022-08-01T19:35:57.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.510+0000 7f5043fff700 1 -- start start 2022-08-01T19:35:57.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.510+0000 7f5043fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50440afd60 0x7f5044159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:57.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.510+0000 7f5043fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f50440b01a0 0x7f50440aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:57.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.510+0000 7f5043fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50440b2710 0x7f50440a9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:57.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.510+0000 7f5043fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50440b4e60 con 0x7f50440b2710 2022-08-01T19:35:57.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.510+0000 7f5043fff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50440b5100 con 0x7f50440afd60 2022-08-01T19:35:57.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.510+0000 7f5043fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50440b53a0 con 0x7f50440b01a0 2022-08-01T19:35:57.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.510+0000 7f50437fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50440b2710 0x7f50440a9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:57.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.510+0000 7f50437fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50440b2710 0x7f50440a9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38134/0 (socket says 172.21.15.17:38134) 2022-08-01T19:35:57.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.510+0000 7f50437fe700 1 -- 172.21.15.17:0/921954516 learned_addr learned my addr 172.21.15.17:0/921954516 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:57.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.510+0000 7f5042ffd700 1 --2- 172.21.15.17:0/921954516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50440afd60 0x7f5044159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:57.525 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.510+0000 7f50427fc700 1 --2- 172.21.15.17:0/921954516 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f50440b01a0 0x7f50440aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:57.525 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.510+0000 7f50437fe700 1 -- 172.21.15.17:0/921954516 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f50440b01a0 msgr2=0x7f50440aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:57.525 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.510+0000 7f50437fe700 1 --2- 172.21.15.17:0/921954516 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f50440b01a0 0x7f50440aebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:57.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.510+0000 7f50437fe700 1 -- 172.21.15.17:0/921954516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50440afd60 msgr2=0x7f5044159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:57.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.510+0000 7f50437fe700 1 --2- 172.21.15.17:0/921954516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50440afd60 0x7f5044159bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:57.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.510+0000 7f50437fe700 1 -- 172.21.15.17:0/921954516 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f503c009530 con 0x7f50440b2710 2022-08-01T19:35:57.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.510+0000 7f50437fe700 1 --2- 172.21.15.17:0/921954516 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50440b2710 0x7f50440a9c10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f5034007000 tx=0x7f5034008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:57.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.510+0000 7f503bfff700 1 -- 172.21.15.17:0/921954516 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5034015070 con 0x7f50440b2710 2022-08-01T19:35:57.529 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.510+0000 7f503bfff700 1 -- 172.21.15.17:0/921954516 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5034010580 con 0x7f50440b2710 2022-08-01T19:35:57.529 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.510+0000 7f5043fff700 1 -- 172.21.15.17:0/921954516 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50440a9770 con 0x7f50440b2710 2022-08-01T19:35:57.529 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.510+0000 7f503bfff700 1 -- 172.21.15.17:0/921954516 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50340108e0 con 0x7f50440b2710 2022-08-01T19:35:57.529 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.510+0000 7f5043fff700 1 -- 172.21.15.17:0/921954516 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50440aa100 con 0x7f50440b2710 2022-08-01T19:35:57.530 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.514+0000 7f5043fff700 1 -- 172.21.15.17:0/921954516 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50440574e0 con 0x7f50440b2710 2022-08-01T19:35:57.530 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.514+0000 7f503bfff700 1 -- 172.21.15.17:0/921954516 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5034010a40 con 0x7f50440b2710 2022-08-01T19:35:57.530 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.514+0000 7f503bfff700 1 --2- 172.21.15.17:0/921954516 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5028053950 0x7f5028055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:57.530 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.514+0000 7f503bfff700 1 -- 172.21.15.17:0/921954516 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f503406bc50 con 0x7f50440b2710 2022-08-01T19:35:57.531 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.514+0000 7f5042ffd700 1 --2- 172.21.15.17:0/921954516 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5028053950 0x7f5028055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:57.531 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.514+0000 7f5042ffd700 1 --2- 172.21.15.17:0/921954516 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5028053950 0x7f5028055e10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f502c00d000 tx=0x7f502c007b70).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:57.531 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.522+0000 7f503bfff700 1 -- 172.21.15.17:0/921954516 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5034042be0 con 0x7f50440b2710 2022-08-01T19:35:57.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.678+0000 7f5043fff700 1 -- 172.21.15.17:0/921954516 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5044069a70 con 0x7f50440b2710 2022-08-01T19:35:57.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.678+0000 7f503bfff700 1 -- 172.21.15.17:0/921954516 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+1562 (secure 0 0 0) 0x7f5034046200 con 0x7f50440b2710 2022-08-01T19:35:57.685 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:35:57.685 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":15,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2487504686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2487504686},{"type":"v1","addr":"172.21.15.17:6833","nonce":2487504686}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":10},{"gid":4885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/4255478434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4255478434},{"type":"v1","addr":"172.21.15.77:6835","nonce":4255478434}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":11},{"gid":5140,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/4154415741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":4154415741},{"type":"v1","addr":"172.21.15.17:6835","nonce":4154415741}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":15}],"filesystems":[]} 2022-08-01T19:35:57.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.682+0000 7f5043fff700 1 -- 172.21.15.17:0/921954516 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5028053950 msgr2=0x7f5028055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:57.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.682+0000 7f5043fff700 1 --2- 172.21.15.17:0/921954516 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5028053950 0x7f5028055e10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f502c00d000 tx=0x7f502c007b70).stop 2022-08-01T19:35:57.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.682+0000 7f5043fff700 1 -- 172.21.15.17:0/921954516 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50440b2710 msgr2=0x7f50440a9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:57.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.682+0000 7f5043fff700 1 --2- 172.21.15.17:0/921954516 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50440b2710 0x7f50440a9c10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f5034007000 tx=0x7f5034008040).stop 2022-08-01T19:35:57.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.682+0000 7f5043fff700 1 -- 172.21.15.17:0/921954516 shutdown_connections 2022-08-01T19:35:57.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.682+0000 7f5043fff700 1 --2- 172.21.15.17:0/921954516 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5028053950 0x7f5028055e10 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:57.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.682+0000 7f5043fff700 1 --2- 172.21.15.17:0/921954516 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50440b2710 0x7f50440a9c10 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:57.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.682+0000 7f5043fff700 1 --2- 172.21.15.17:0/921954516 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f50440b01a0 0x7f50440aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:57.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.682+0000 7f5043fff700 1 --2- 172.21.15.17:0/921954516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50440afd60 0x7f5044159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:57.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.682+0000 7f5043fff700 1 -- 172.21.15.17:0/921954516 >> 172.21.15.17:0/921954516 conn(0x7f5044060310 msgr2=0x7f5044146750 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:57.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.682+0000 7f5043fff700 1 -- 172.21.15.17:0/921954516 shutdown_connections 2022-08-01T19:35:57.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.682+0000 7f5043fff700 1 -- 172.21.15.17:0/921954516 wait complete. 2022-08-01T19:35:57.690 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 15 2022-08-01T19:35:57.697 INFO:tasks.ceph.mds.a:Restarting daemon 2022-08-01T19:35:57.697 INFO:tasks.ceph.mds.a:Stopping old one... 2022-08-01T19:35:57.698 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-08-01T19:35:57.698 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:35:57.699 INFO:tasks.ceph.mds.c:Restarting daemon 2022-08-01T19:35:57.699 INFO:tasks.ceph.mds.c:Stopping old one... 2022-08-01T19:35:57.699 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-08-01T19:35:57.700 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:35:57.700 INFO:tasks.ceph.mds.b:Restarting daemon 2022-08-01T19:35:57.700 INFO:tasks.ceph.mds.b:Stopping old one... 2022-08-01T19:35:57.701 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-08-01T19:35:57.701 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:35:57.715 INFO:tasks.ceph.mds.a:Stopped 2022-08-01T19:35:57.715 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-08-01T19:35:57.717 INFO:tasks.ceph.mds.c:Stopped 2022-08-01T19:35:57.717 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-08-01T19:35:57.718 INFO:tasks.ceph.mds.a:Started 2022-08-01T19:35:57.719 DEBUG:teuthology.parallel:result is None 2022-08-01T19:35:57.720 INFO:tasks.ceph.mds.b:Stopped 2022-08-01T19:35:57.720 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-08-01T19:35:57.721 INFO:tasks.ceph.mds.c:Started 2022-08-01T19:35:57.722 DEBUG:teuthology.parallel:result is None 2022-08-01T19:35:57.724 INFO:tasks.ceph.mds.b:Started 2022-08-01T19:35:57.724 DEBUG:teuthology.parallel:result is None 2022-08-01T19:35:57.725 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-08-01T19:35:57.795 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2022-08-01T19:35:57.797 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2022-08-01T19:35:57.799 INFO:tasks.ceph.mds.b.smithi077.stdout:starting mds.b at 2022-08-01T19:35:57.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.814+0000 7f8500bcb700 1 Processor -- start 2022-08-01T19:35:57.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.814+0000 7f8500bcb700 1 -- start start 2022-08-01T19:35:57.822 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.814+0000 7f8500bcb700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f84fc0b1ee0 0x7f84fc0b0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:57.822 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.814+0000 7f8500bcb700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84fc05c620 0x7f84fc0ad930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:57.822 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.814+0000 7f8500bcb700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84fc0b22c0 con 0x7f84fc0ade20 2022-08-01T19:35:57.822 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.814+0000 7f8500bcb700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84fc0b2c10 con 0x7f84fc0b1ee0 2022-08-01T19:35:57.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.814+0000 7f8500bcb700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc0b2eb0 con 0x7f84fc05c620 2022-08-01T19:35:57.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.814+0000 7f84fa59c700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f84fc0b1ee0 0x7f84fc0b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:57.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.814+0000 7f84f3fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84fc05c620 0x7f84fc0ad930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:57.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.814+0000 7f84fad9d700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7f84fc0ade20 0x7f84fc0a5ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:47166/0 (socket says 172.21.15.17:47166) 2022-08-01T19:35:57.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.814+0000 7f84fad9d700 1 -- 172.21.15.17:0/3126011587 learned_addr learned my addr 172.21.15.17:0/3126011587 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:35:57.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.818+0000 7f84fa59c700 1 -- 172.21.15.17:0/3126011587 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84fc05c620 msgr2=0x7f84fc0ad930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:57.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.818+0000 7f84fa59c700 1 --2- 172.21.15.17:0/3126011587 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84fc05c620 0x7f84fc0ad930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:57.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.818+0000 7f84fa59c700 1 -- 172.21.15.17:0/3126011587 >> v1:172.21.15.77:6789/0 conn(0x7f84fc0ade20 legacy=0x7f84fc0a5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:57.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.818+0000 7f84fa59c700 1 -- 172.21.15.17:0/3126011587 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84fc0ad1f0 con 0x7f84fc0b1ee0 2022-08-01T19:35:57.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.818+0000 7f84fa59c700 1 --2- 172.21.15.17:0/3126011587 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84fc0b1ee0 0x7f84fc0b0680 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f84ec002f70 tx=0x7f84ec006df0).ready entity=mon.1 client_cookie=657255e7d503fbb8 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:57.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.826+0000 7f84f9d9b700 1 -- 172.21.15.17:0/3126011587 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84ec00d570 con 0x7f84fc0b1ee0 2022-08-01T19:35:57.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.826+0000 7f84f9d9b700 1 -- 172.21.15.17:0/3126011587 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84ec000e90 con 0x7f84fc0b1ee0 2022-08-01T19:35:57.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.826+0000 7f84f9d9b700 1 -- 172.21.15.17:0/3126011587 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84ec00d8c0 con 0x7f84fc0b1ee0 2022-08-01T19:35:57.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.826+0000 7f8500bcb700 1 -- 172.21.15.17:0/3126011587 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84fc0b1ee0 msgr2=0x7f84fc0b0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:57.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.826+0000 7f8500bcb700 1 --2- 172.21.15.17:0/3126011587 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84fc0b1ee0 0x7f84fc0b0680 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f84ec002f70 tx=0x7f84ec006df0).stop 2022-08-01T19:35:57.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.826+0000 7f8500bcb700 1 -- 172.21.15.17:0/3126011587 shutdown_connections 2022-08-01T19:35:57.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.826+0000 7f8500bcb700 1 --2- 172.21.15.17:0/3126011587 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84fc05c620 0x7f84fc0ad930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:57.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.826+0000 7f8500bcb700 1 --2- 172.21.15.17:0/3126011587 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84fc0b1ee0 0x7f84fc0b0680 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:57.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.826+0000 7f8500bcb700 1 -- 172.21.15.17:0/3126011587 >> 172.21.15.17:0/3126011587 conn(0x7f84fc05ceb0 msgr2=0x7f84fc063130 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:57.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.826+0000 7f8500bcb700 1 -- 172.21.15.17:0/3126011587 shutdown_connections 2022-08-01T19:35:57.834 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.826+0000 7f8500bcb700 1 -- 172.21.15.17:0/3126011587 wait complete. 2022-08-01T19:35:57.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.830+0000 7f8500bcb700 1 Processor -- start 2022-08-01T19:35:57.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.830+0000 7f8500bcb700 1 -- start start 2022-08-01T19:35:57.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.830+0000 7f8500bcb700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84fc05c620 0x7f84fc064da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:57.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.830+0000 7f8500bcb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84fc0ade20 0x7f84fc13c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:57.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.830+0000 7f8500bcb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84fc0b1ee0 0x7f84fc155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:57.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.830+0000 7f8500bcb700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc067c40 con 0x7f84fc0b1ee0 2022-08-01T19:35:57.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.830+0000 7f8500bcb700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc065c40 con 0x7f84fc0ade20 2022-08-01T19:35:57.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.830+0000 7f8500bcb700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc065700 con 0x7f84fc05c620 2022-08-01T19:35:57.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.830+0000 7f84fad9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84fc0b1ee0 0x7f84fc155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:57.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.830+0000 7f84fa59c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84fc05c620 0x7f84fc064da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:57.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.830+0000 7f84fad9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84fc0b1ee0 0x7f84fc155d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38178/0 (socket says 172.21.15.17:38178) 2022-08-01T19:35:57.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.830+0000 7f84fad9d700 1 -- 172.21.15.17:0/2247260702 learned_addr learned my addr 172.21.15.17:0/2247260702 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:57.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.830+0000 7f84f3fff700 1 --2- 172.21.15.17:0/2247260702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84fc0ade20 0x7f84fc13c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:57.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.830+0000 7f84fa59c700 1 -- 172.21.15.17:0/2247260702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84fc0ade20 msgr2=0x7f84fc13c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:57.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.830+0000 7f84fa59c700 1 --2- 172.21.15.17:0/2247260702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84fc0ade20 0x7f84fc13c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:57.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.830+0000 7f84fa59c700 1 -- 172.21.15.17:0/2247260702 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84fc0b1ee0 msgr2=0x7f84fc155d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:57.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.830+0000 7f84fa59c700 1 --2- 172.21.15.17:0/2247260702 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84fc0b1ee0 0x7f84fc155d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:57.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.830+0000 7f84fa59c700 1 -- 172.21.15.17:0/2247260702 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ec00b180 con 0x7f84fc05c620 2022-08-01T19:35:57.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.830+0000 7f84fad9d700 1 --2- 172.21.15.17:0/2247260702 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84fc0b1ee0 0x7f84fc155d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:35:57.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.830+0000 7f84fa59c700 1 --2- 172.21.15.17:0/2247260702 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84fc05c620 0x7f84fc064da0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f84ec002f70 tx=0x7f84ec014000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:57.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.830+0000 7f84f37fe700 1 -- 172.21.15.17:0/2247260702 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84ec00dea0 con 0x7f84fc05c620 2022-08-01T19:35:57.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.830+0000 7f84f37fe700 1 -- 172.21.15.17:0/2247260702 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84ec013e90 con 0x7f84fc05c620 2022-08-01T19:35:57.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.830+0000 7f8500bcb700 1 -- 172.21.15.17:0/2247260702 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84fc0659a0 con 0x7f84fc05c620 2022-08-01T19:35:57.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.830+0000 7f84f37fe700 1 -- 172.21.15.17:0/2247260702 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84ec01e5f0 con 0x7f84fc05c620 2022-08-01T19:35:57.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.830+0000 7f8500bcb700 1 -- 172.21.15.17:0/2247260702 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84fc142590 con 0x7f84fc05c620 2022-08-01T19:35:57.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.834+0000 7f8500bcb700 1 -- 172.21.15.17:0/2247260702 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84fc0574e0 con 0x7f84fc05c620 2022-08-01T19:35:57.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.834+0000 7f84f37fe700 1 -- 172.21.15.17:0/2247260702 <== mon.2 v2:172.21.15.77:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f84ec025030 con 0x7f84fc05c620 2022-08-01T19:35:57.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.834+0000 7f84f37fe700 1 --2- 172.21.15.17:0/2247260702 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f84e0053aa0 0x7f84e0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:57.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.834+0000 7f84f37fe700 1 -- 172.21.15.17:0/2247260702 <== mon.2 v2:172.21.15.77:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f84ec06c910 con 0x7f84fc05c620 2022-08-01T19:35:57.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.834+0000 7f84f3fff700 1 --2- 172.21.15.17:0/2247260702 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f84e0053aa0 0x7f84e0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:57.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.834+0000 7f84f3fff700 1 --2- 172.21.15.17:0/2247260702 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f84e0053aa0 0x7f84e0055f60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f84e400b000 tx=0x7f84e4006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:57.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.842+0000 7f84f37fe700 1 -- 172.21.15.17:0/2247260702 <== mon.2 v2:172.21.15.77:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f84ec0439b0 con 0x7f84fc05c620 2022-08-01T19:35:57.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:57.978+0000 7f8500bcb700 1 -- 172.21.15.17:0/2247260702 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f84fc05d8f0 con 0x7f84fc05c620 2022-08-01T19:35:58.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.834+0000 7f84f37fe700 1 -- 172.21.15.17:0/2247260702 <== mon.2 v2:172.21.15.77:3301/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v41) v1 ==== 97+0+0 (secure 0 0 0) 0x7f84ec023070 con 0x7f84fc05c620 2022-08-01T19:35:58.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.838+0000 7f8500bcb700 1 -- 172.21.15.17:0/2247260702 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f84e0053aa0 msgr2=0x7f84e0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:58.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.838+0000 7f8500bcb700 1 --2- 172.21.15.17:0/2247260702 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f84e0053aa0 0x7f84e0055f60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f84e400b000 tx=0x7f84e4006df0).stop 2022-08-01T19:35:58.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.838+0000 7f8500bcb700 1 -- 172.21.15.17:0/2247260702 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84fc05c620 msgr2=0x7f84fc064da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:58.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.838+0000 7f8500bcb700 1 --2- 172.21.15.17:0/2247260702 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84fc05c620 0x7f84fc064da0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f84ec002f70 tx=0x7f84ec014000).stop 2022-08-01T19:35:58.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.838+0000 7f8500bcb700 1 -- 172.21.15.17:0/2247260702 shutdown_connections 2022-08-01T19:35:58.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.838+0000 7f8500bcb700 1 --2- 172.21.15.17:0/2247260702 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f84e0053aa0 0x7f84e0055f60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:58.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.838+0000 7f8500bcb700 1 --2- 172.21.15.17:0/2247260702 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84fc0b1ee0 0x7f84fc155d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:58.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.838+0000 7f8500bcb700 1 --2- 172.21.15.17:0/2247260702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84fc0ade20 0x7f84fc13c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:58.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.838+0000 7f8500bcb700 1 --2- 172.21.15.17:0/2247260702 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84fc05c620 0x7f84fc064da0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:58.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.838+0000 7f8500bcb700 1 -- 172.21.15.17:0/2247260702 >> 172.21.15.17:0/2247260702 conn(0x7f84fc05ceb0 msgr2=0x7f84fc13d530 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:58.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.838+0000 7f8500bcb700 1 -- 172.21.15.17:0/2247260702 shutdown_connections 2022-08-01T19:35:58.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.842+0000 7f8500bcb700 1 -- 172.21.15.17:0/2247260702 wait complete. 2022-08-01T19:35:58.848 INFO:teuthology.orchestra.run.smithi017.stderr: removed all blacklist entries 2022-08-01T19:35:58.862 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-08-01T19:35:58.863 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-08-01T19:35:58.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.966+0000 7f1db2785700 1 Processor -- start 2022-08-01T19:35:58.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.966+0000 7f1db2785700 1 -- start start 2022-08-01T19:35:58.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.966+0000 7f1db2785700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1dac0b01a0 0x7f1dac0a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:58.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.966+0000 7f1db2785700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f1dac0afd60 0x7f1dac0a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:58.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.966+0000 7f1db2785700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dac0a7250 con 0x7f1dac0b2710 2022-08-01T19:35:58.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.966+0000 7f1db2785700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dac0a7480 con 0x7f1dac0b01a0 2022-08-01T19:35:58.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.966+0000 7f1db2785700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dac0a7720 con 0x7f1dac0afd60 2022-08-01T19:35:58.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.966+0000 7f1db0f82700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1dac0b01a0 0x7f1dac0a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:58.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.966+0000 7f1db0f82700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1dac0b01a0 0x7f1dac0a8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38188/0 (socket says 172.21.15.17:38188) 2022-08-01T19:35:58.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.966+0000 7f1db0f82700 1 -- 172.21.15.17:0/3091789233 learned_addr learned my addr 172.21.15.17:0/3091789233 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:58.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.966+0000 7f1db1f84700 1 --2- 172.21.15.17:0/3091789233 >> v2:172.21.15.77:3300/0 conn(0x7f1dac0afd60 0x7f1dac0a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:58.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.966+0000 7f1db0f82700 1 -- 172.21.15.17:0/3091789233 >> v2:172.21.15.77:3300/0 conn(0x7f1dac0afd60 msgr2=0x7f1dac0a8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:58.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.966+0000 7f1db0f82700 1 --2- 172.21.15.17:0/3091789233 >> v2:172.21.15.77:3300/0 conn(0x7f1dac0afd60 0x7f1dac0a8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:58.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.966+0000 7f1db0f82700 1 -- 172.21.15.17:0/3091789233 >> v1:172.21.15.77:6789/0 conn(0x7f1dac0b2710 legacy=0x7f1dac14afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:58.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.966+0000 7f1db0f82700 1 -- 172.21.15.17:0/3091789233 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dac0a79c0 con 0x7f1dac0b01a0 2022-08-01T19:35:58.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.966+0000 7f1db0f82700 1 --2- 172.21.15.17:0/3091789233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dac0b01a0 0x7f1dac0a8540 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f1da00032f0 tx=0x7f1da0009290).ready entity=mon.0 client_cookie=188c266e502e0246 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:58.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.966+0000 7f1d9ffff700 1 -- 172.21.15.17:0/3091789233 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1da000b870 con 0x7f1dac0b01a0 2022-08-01T19:35:58.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.966+0000 7f1d9ffff700 1 -- 172.21.15.17:0/3091789233 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da000b9d0 con 0x7f1dac0b01a0 2022-08-01T19:35:58.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.966+0000 7f1d9ffff700 1 -- 172.21.15.17:0/3091789233 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1da000bb40 con 0x7f1dac0b01a0 2022-08-01T19:35:58.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.966+0000 7f1db2785700 1 -- 172.21.15.17:0/3091789233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dac0b01a0 msgr2=0x7f1dac0a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:58.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.966+0000 7f1db2785700 1 --2- 172.21.15.17:0/3091789233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dac0b01a0 0x7f1dac0a8540 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f1da00032f0 tx=0x7f1da0009290).stop 2022-08-01T19:35:58.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.966+0000 7f1db2785700 1 -- 172.21.15.17:0/3091789233 shutdown_connections 2022-08-01T19:35:58.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.966+0000 7f1db2785700 1 --2- 172.21.15.17:0/3091789233 >> v2:172.21.15.77:3300/0 conn(0x7f1dac0afd60 0x7f1dac0a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:58.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.966+0000 7f1db2785700 1 --2- 172.21.15.17:0/3091789233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dac0b01a0 0x7f1dac0a8540 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:58.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.966+0000 7f1db2785700 1 -- 172.21.15.17:0/3091789233 >> 172.21.15.17:0/3091789233 conn(0x7f1dac060310 msgr2=0x7f1dac0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:58.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.966+0000 7f1db2785700 1 -- 172.21.15.17:0/3091789233 shutdown_connections 2022-08-01T19:35:58.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.970+0000 7f1db2785700 1 -- 172.21.15.17:0/3091789233 wait complete. 2022-08-01T19:35:58.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.970+0000 7f1db2785700 1 Processor -- start 2022-08-01T19:35:58.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.970+0000 7f1db2785700 1 -- start start 2022-08-01T19:35:58.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.970+0000 7f1db2785700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1dac0afd60 0x7f1dac063840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:58.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.970+0000 7f1db2785700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dac0b01a0 0x7f1dac1565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:58.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.970+0000 7f1db2785700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1dac0b2710 0x7f1dac0667b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:58.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.970+0000 7f1db2785700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac154270 con 0x7f1dac0b01a0 2022-08-01T19:35:58.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.970+0000 7f1db2785700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac066350 con 0x7f1dac0afd60 2022-08-01T19:35:58.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.970+0000 7f1db2785700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac0b60d0 con 0x7f1dac0b2710 2022-08-01T19:35:58.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.970+0000 7f1db0f82700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dac0b01a0 0x7f1dac1565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:58.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.970+0000 7f1db0f82700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dac0b01a0 0x7f1dac1565c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38194/0 (socket says 172.21.15.17:38194) 2022-08-01T19:35:58.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.970+0000 7f1db0f82700 1 -- 172.21.15.17:0/581672582 learned_addr learned my addr 172.21.15.17:0/581672582 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:58.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.970+0000 7f1db1783700 1 --2- 172.21.15.17:0/581672582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1dac0afd60 0x7f1dac063840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:58.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.970+0000 7f1db1f84700 1 --2- 172.21.15.17:0/581672582 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1dac0b2710 0x7f1dac0667b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:58.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.970+0000 7f1db0f82700 1 -- 172.21.15.17:0/581672582 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1dac0b2710 msgr2=0x7f1dac0667b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:58.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.970+0000 7f1db0f82700 1 --2- 172.21.15.17:0/581672582 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1dac0b2710 0x7f1dac0667b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:58.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.970+0000 7f1db0f82700 1 -- 172.21.15.17:0/581672582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1dac0afd60 msgr2=0x7f1dac063840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:58.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.970+0000 7f1db0f82700 1 --2- 172.21.15.17:0/581672582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1dac0afd60 0x7f1dac063840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:58.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.970+0000 7f1db0f82700 1 -- 172.21.15.17:0/581672582 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da0009530 con 0x7f1dac0b01a0 2022-08-01T19:35:58.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.974+0000 7f1db0f82700 1 --2- 172.21.15.17:0/581672582 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dac0b01a0 0x7f1dac1565c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f1da00032f0 tx=0x7f1da0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:58.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.974+0000 7f1d9e7fc700 1 -- 172.21.15.17:0/581672582 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1da000b870 con 0x7f1dac0b01a0 2022-08-01T19:35:58.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.974+0000 7f1d9e7fc700 1 -- 172.21.15.17:0/581672582 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da0021440 con 0x7f1dac0b01a0 2022-08-01T19:35:58.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.974+0000 7f1db2785700 1 -- 172.21.15.17:0/581672582 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dac0b5b90 con 0x7f1dac0b01a0 2022-08-01T19:35:58.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.974+0000 7f1d9e7fc700 1 -- 172.21.15.17:0/581672582 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1da00217a0 con 0x7f1dac0b01a0 2022-08-01T19:35:58.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.974+0000 7f1db2785700 1 -- 172.21.15.17:0/581672582 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dac156ab0 con 0x7f1dac0b01a0 2022-08-01T19:35:58.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.974+0000 7f1db2785700 1 -- 172.21.15.17:0/581672582 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dac0574e0 con 0x7f1dac0b01a0 2022-08-01T19:35:58.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.974+0000 7f1d9e7fc700 1 -- 172.21.15.17:0/581672582 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1da000f070 con 0x7f1dac0b01a0 2022-08-01T19:35:58.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.974+0000 7f1d9e7fc700 1 --2- 172.21.15.17:0/581672582 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1d940539e0 0x7f1d94055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:58.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.974+0000 7f1d9e7fc700 1 -- 172.21.15.17:0/581672582 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f1da006d170 con 0x7f1dac0b01a0 2022-08-01T19:35:58.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.978+0000 7f1db1783700 1 --2- 172.21.15.17:0/581672582 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1d940539e0 0x7f1d94055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:58.991 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.978+0000 7f1db1783700 1 --2- 172.21.15.17:0/581672582 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1d940539e0 0x7f1d94055ea0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f1da400d000 tx=0x7f1da4009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:58.991 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:58.982+0000 7f1d9e7fc700 1 -- 172.21.15.17:0/581672582 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1da0044270 con 0x7f1dac0b01a0 2022-08-01T19:35:59.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.142+0000 7f1db2785700 1 -- 172.21.15.17:0/581672582 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f1dac0664c0 con 0x7f1dac0b01a0 2022-08-01T19:35:59.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.146+0000 7f1d9e7fc700 1 -- 172.21.15.17:0/581672582 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f1da0015020 con 0x7f1dac0b01a0 2022-08-01T19:35:59.151 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:35:59.152 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:35:59.152 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_dump": [ 2022-08-01T19:35:59.153 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:59.153 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.a", 2022-08-01T19:35:59.153 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhijotxORAAVc04rsrlFj0NShsyc6Qbhg==", 2022-08-01T19:35:59.154 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:35:59.154 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:35:59.155 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:35:59.155 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:35:59.155 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:35:59.156 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:59.156 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:59.157 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:59.157 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.b", 2022-08-01T19:35:59.158 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhiud2xBxAAcn6/O5ysdIp/oIbF5Y1dAw==", 2022-08-01T19:35:59.158 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:35:59.158 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:35:59.159 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:35:59.159 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:35:59.160 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:35:59.160 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:59.160 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:59.161 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:59.161 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.c", 2022-08-01T19:35:59.162 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+KuhiMzMbBBAAfdT5mtpYwboNL+7f2B09KQ==", 2022-08-01T19:35:59.162 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:35:59.162 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:35:59.163 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:35:59.163 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:35:59.164 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:35:59.164 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:59.165 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:59.165 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:59.165 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.0", 2022-08-01T19:35:59.166 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhia27/NRAAZXBp+nYelJVJ6Mge9r+3Vg==", 2022-08-01T19:35:59.166 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:35:59.167 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:35:59.167 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:35:59.167 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:35:59.168 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:59.168 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:59.168 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:59.169 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.1", 2022-08-01T19:35:59.169 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCBKuhiOggRGRAAvgA0MxwBJwmpXV6S9DTdjA==", 2022-08-01T19:35:59.169 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:35:59.170 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:35:59.170 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:35:59.171 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:35:59.171 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:59.171 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:59.171 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:59.171 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.2", 2022-08-01T19:35:59.172 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCDKuhi5PgOORAAhcORK4bW2T3dXPeYjcCm+g==", 2022-08-01T19:35:59.172 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:35:59.172 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:35:59.172 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:35:59.173 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:35:59.173 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:59.173 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:59.173 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:59.174 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.3", 2022-08-01T19:35:59.174 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCGKuhiutn6GxAAk2ucLuGbT5YV/R4xXQGh0w==", 2022-08-01T19:35:59.174 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:35:59.174 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:35:59.175 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:35:59.175 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:35:59.175 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:59.175 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:59.175 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:59.176 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.4", 2022-08-01T19:35:59.176 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCLKuhiGHTRHxAArr3TwXA0gsCAAF/Qs38wUQ==", 2022-08-01T19:35:59.176 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:35:59.176 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:35:59.177 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:35:59.177 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:35:59.177 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:59.177 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:59.177 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:59.178 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.5", 2022-08-01T19:35:59.178 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCOKuhiMMDbBBAAvTzBvOcv7pFWm/P/lNqB0w==", 2022-08-01T19:35:59.178 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:35:59.178 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:35:59.179 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:35:59.179 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:35:59.179 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:59.179 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:59.180 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:59.180 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.6", 2022-08-01T19:35:59.180 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCQKuhiG2H3IxAAufUcWqo9ULquxIrUD5vSfA==", 2022-08-01T19:35:59.180 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:35:59.181 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:35:59.181 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:35:59.181 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:35:59.181 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:59.181 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:59.182 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:59.182 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.7", 2022-08-01T19:35:59.182 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCTKuhiU50LChAAXuhrYBbHkk3eKzAPiEJFTg==", 2022-08-01T19:35:59.183 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:35:59.183 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:35:59.183 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:35:59.183 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:35:59.183 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:59.184 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:59.184 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:59.184 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.0", 2022-08-01T19:35:59.184 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhig3EnCxAANM20jSIQf/AKwuRX1qwT/A==", 2022-08-01T19:35:59.185 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:35:59.185 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:35:59.185 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow r", 2022-08-01T19:35:59.185 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T19:35:59.185 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:59.186 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:59.186 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:59.186 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.1", 2022-08-01T19:35:59.186 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+KuhicJR0DhAAivqEAZUhXnSwjVRwWm2bJg==", 2022-08-01T19:35:59.187 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:35:59.187 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:35:59.187 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow r", 2022-08-01T19:35:59.187 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T19:35:59.188 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:59.188 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:59.188 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:59.188 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.admin", 2022-08-01T19:35:59.189 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhiPTTGHxAAgfthCR6RX6j8F1YseFGNeA==", 2022-08-01T19:35:59.189 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:35:59.189 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:35:59.189 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:35:59.190 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:35:59.190 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:35:59.190 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:59.190 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:59.191 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:59.191 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.x", 2022-08-01T19:35:59.191 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9Kuhi2jvCNhAAyUxiGByZ+OZr8Y7dh3Qt5w==", 2022-08-01T19:35:59.191 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:35:59.192 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:35:59.192 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2022-08-01T19:35:59.192 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:35:59.192 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:59.192 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:35:59.193 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:35:59.193 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.y", 2022-08-01T19:35:59.193 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhiDJXpMhAA8aek67ydbRgNO0h+miws+w==", 2022-08-01T19:35:59.194 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:35:59.194 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:35:59.194 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2022-08-01T19:35:59.194 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:35:59.195 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:59.195 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:35:59.195 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:35:59.195 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:35:59.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.146+0000 7f1db2785700 1 -- 172.21.15.17:0/581672582 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1d940539e0 msgr2=0x7f1d94055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:59.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.146+0000 7f1db2785700 1 --2- 172.21.15.17:0/581672582 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1d940539e0 0x7f1d94055ea0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f1da400d000 tx=0x7f1da4009040).stop 2022-08-01T19:35:59.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.146+0000 7f1db2785700 1 -- 172.21.15.17:0/581672582 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dac0b01a0 msgr2=0x7f1dac1565c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:59.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.146+0000 7f1db2785700 1 --2- 172.21.15.17:0/581672582 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dac0b01a0 0x7f1dac1565c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f1da00032f0 tx=0x7f1da0016000).stop 2022-08-01T19:35:59.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.146+0000 7f1db2785700 1 -- 172.21.15.17:0/581672582 shutdown_connections 2022-08-01T19:35:59.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.146+0000 7f1db2785700 1 --2- 172.21.15.17:0/581672582 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1d940539e0 0x7f1d94055ea0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:59.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.146+0000 7f1db2785700 1 --2- 172.21.15.17:0/581672582 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1dac0b2710 0x7f1dac0667b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:59.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.146+0000 7f1db2785700 1 --2- 172.21.15.17:0/581672582 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dac0b01a0 0x7f1dac1565c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:59.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.146+0000 7f1db2785700 1 --2- 172.21.15.17:0/581672582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1dac0afd60 0x7f1dac063840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:59.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.146+0000 7f1db2785700 1 -- 172.21.15.17:0/581672582 >> 172.21.15.17:0/581672582 conn(0x7f1dac060310 msgr2=0x7f1dac1466e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:59.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.146+0000 7f1db2785700 1 -- 172.21.15.17:0/581672582 shutdown_connections 2022-08-01T19:35:59.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.146+0000 7f1db2785700 1 -- 172.21.15.17:0/581672582 wait complete. 2022-08-01T19:35:59.201 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:35:59.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.306+0000 7ff6e37b2700 1 Processor -- start 2022-08-01T19:35:59.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.306+0000 7ff6e37b2700 1 -- start start 2022-08-01T19:35:59.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.306+0000 7ff6e37b2700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ff6dc0b3630 0x7ff6dc143f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:59.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.306+0000 7ff6e37b2700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff6dc0af910 0x7ff6dc1402a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:59.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.306+0000 7ff6e37b2700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6dc0b3a10 con 0x7ff6dc0aa0d0 2022-08-01T19:35:59.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.306+0000 7ff6e37b2700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6dc1443f0 con 0x7ff6dc0af910 2022-08-01T19:35:59.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.306+0000 7ff6e37b2700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6dc146b40 con 0x7ff6dc0b3630 2022-08-01T19:35:59.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.306+0000 7ff6e1d4f700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff6dc0af910 0x7ff6dc1402a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:59.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.306+0000 7ff6e1d4f700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff6dc0af910 0x7ff6dc1402a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38204/0 (socket says 172.21.15.17:38204) 2022-08-01T19:35:59.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.306+0000 7ff6e1d4f700 1 -- 172.21.15.17:0/201569730 learned_addr learned my addr 172.21.15.17:0/201569730 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:59.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.306+0000 7ff6e154e700 1 --2- 172.21.15.17:0/201569730 >> v2:172.21.15.77:3300/0 conn(0x7ff6dc0b3630 0x7ff6dc143f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:59.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.306+0000 7ff6e1d4f700 1 -- 172.21.15.17:0/201569730 >> v2:172.21.15.77:3300/0 conn(0x7ff6dc0b3630 msgr2=0x7ff6dc143f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:59.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.306+0000 7ff6e1d4f700 1 --2- 172.21.15.17:0/201569730 >> v2:172.21.15.77:3300/0 conn(0x7ff6dc0b3630 0x7ff6dc143f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:59.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.306+0000 7ff6e1d4f700 1 -- 172.21.15.17:0/201569730 >> v1:172.21.15.77:6789/0 conn(0x7ff6dc0aa0d0 legacy=0x7ff6dc0af200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:59.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.306+0000 7ff6e1d4f700 1 -- 172.21.15.17:0/201569730 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6dc146de0 con 0x7ff6dc0af910 2022-08-01T19:35:59.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.306+0000 7ff6e1d4f700 1 --2- 172.21.15.17:0/201569730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6dc0af910 0x7ff6dc1402a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7ff6d40032f0 tx=0x7ff6d4009290).ready entity=mon.0 client_cookie=b89c11733a0df071 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:59.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.306+0000 7ff6cbfff700 1 -- 172.21.15.17:0/201569730 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6d400b870 con 0x7ff6dc0af910 2022-08-01T19:35:59.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.306+0000 7ff6cbfff700 1 -- 172.21.15.17:0/201569730 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d400b9d0 con 0x7ff6dc0af910 2022-08-01T19:35:59.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.306+0000 7ff6cbfff700 1 -- 172.21.15.17:0/201569730 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6d400bb40 con 0x7ff6dc0af910 2022-08-01T19:35:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.306+0000 7ff6e37b2700 1 -- 172.21.15.17:0/201569730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6dc0af910 msgr2=0x7ff6dc1402a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.306+0000 7ff6e37b2700 1 --2- 172.21.15.17:0/201569730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6dc0af910 0x7ff6dc1402a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7ff6d40032f0 tx=0x7ff6d4009290).stop 2022-08-01T19:35:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.306+0000 7ff6e37b2700 1 -- 172.21.15.17:0/201569730 shutdown_connections 2022-08-01T19:35:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.306+0000 7ff6e37b2700 1 --2- 172.21.15.17:0/201569730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6dc0af910 0x7ff6dc1402a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.306+0000 7ff6e37b2700 1 --2- 172.21.15.17:0/201569730 >> v2:172.21.15.77:3300/0 conn(0x7ff6dc0b3630 0x7ff6dc143f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.306+0000 7ff6e37b2700 1 -- 172.21.15.17:0/201569730 >> 172.21.15.17:0/201569730 conn(0x7ff6dc060950 msgr2=0x7ff6dc060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.306+0000 7ff6e37b2700 1 -- 172.21.15.17:0/201569730 shutdown_connections 2022-08-01T19:35:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.306+0000 7ff6e37b2700 1 -- 172.21.15.17:0/201569730 wait complete. 2022-08-01T19:35:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.306+0000 7ff6e37b2700 1 Processor -- start 2022-08-01T19:35:59.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.306+0000 7ff6e37b2700 1 -- start start 2022-08-01T19:35:59.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.310+0000 7ff6e37b2700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6dc0aa0d0 0x7ff6dc13e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:59.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.310+0000 7ff6e37b2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6dc0af910 0x7ff6dc13bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:59.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.310+0000 7ff6e37b2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6dc0b3630 0x7ff6dc13e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:59.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.310+0000 7ff6e37b2700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc0b23d0 con 0x7ff6dc0af910 2022-08-01T19:35:59.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.310+0000 7ff6e37b2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc135e80 con 0x7ff6dc0b3630 2022-08-01T19:35:59.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.310+0000 7ff6e37b2700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc13eaf0 con 0x7ff6dc0aa0d0 2022-08-01T19:35:59.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.310+0000 7ff6e0d4d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6dc0af910 0x7ff6dc13bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:59.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.310+0000 7ff6e0d4d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6dc0af910 0x7ff6dc13bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38208/0 (socket says 172.21.15.17:38208) 2022-08-01T19:35:59.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.310+0000 7ff6e0d4d700 1 -- 172.21.15.17:0/2727744926 learned_addr learned my addr 172.21.15.17:0/2727744926 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:59.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.310+0000 7ff6e1d4f700 1 --2- 172.21.15.17:0/2727744926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6dc0b3630 0x7ff6dc13e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:59.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.310+0000 7ff6e154e700 1 --2- 172.21.15.17:0/2727744926 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6dc0aa0d0 0x7ff6dc13e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:59.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.310+0000 7ff6e0d4d700 1 -- 172.21.15.17:0/2727744926 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6dc0aa0d0 msgr2=0x7ff6dc13e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:59.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.310+0000 7ff6e0d4d700 1 --2- 172.21.15.17:0/2727744926 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6dc0aa0d0 0x7ff6dc13e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:59.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.310+0000 7ff6e0d4d700 1 -- 172.21.15.17:0/2727744926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6dc0b3630 msgr2=0x7ff6dc13e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:59.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.310+0000 7ff6e0d4d700 1 --2- 172.21.15.17:0/2727744926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6dc0b3630 0x7ff6dc13e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:59.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.310+0000 7ff6e0d4d700 1 -- 172.21.15.17:0/2727744926 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d4009530 con 0x7ff6dc0af910 2022-08-01T19:35:59.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.310+0000 7ff6e0d4d700 1 --2- 172.21.15.17:0/2727744926 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6dc0af910 0x7ff6dc13bf30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7ff6dc136c10 tx=0x7ff6cc007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:59.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.310+0000 7ff6ca7fc700 1 -- 172.21.15.17:0/2727744926 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6cc007030 con 0x7ff6dc0af910 2022-08-01T19:35:59.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.310+0000 7ff6ca7fc700 1 -- 172.21.15.17:0/2727744926 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6cc007190 con 0x7ff6dc0af910 2022-08-01T19:35:59.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.310+0000 7ff6e37b2700 1 -- 172.21.15.17:0/2727744926 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6dc1360e0 con 0x7ff6dc0af910 2022-08-01T19:35:59.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.310+0000 7ff6ca7fc700 1 -- 172.21.15.17:0/2727744926 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6cc006510 con 0x7ff6dc0af910 2022-08-01T19:35:59.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.310+0000 7ff6e37b2700 1 -- 172.21.15.17:0/2727744926 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6dc148be0 con 0x7ff6dc0af910 2022-08-01T19:35:59.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.310+0000 7ff6e37b2700 1 -- 172.21.15.17:0/2727744926 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6dc148050 con 0x7ff6dc0af910 2022-08-01T19:35:59.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.314+0000 7ff6ca7fc700 1 -- 172.21.15.17:0/2727744926 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff6cc006670 con 0x7ff6dc0af910 2022-08-01T19:35:59.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.314+0000 7ff6ca7fc700 1 --2- 172.21.15.17:0/2727744926 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff6bc053b00 0x7ff6bc055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:59.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.314+0000 7ff6ca7fc700 1 -- 172.21.15.17:0/2727744926 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5600+0+0 (secure 0 0 0) 0x7ff6cc06b600 con 0x7ff6dc0af910 2022-08-01T19:35:59.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.314+0000 7ff6e154e700 1 --2- 172.21.15.17:0/2727744926 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff6bc053b00 0x7ff6bc055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:59.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.314+0000 7ff6e154e700 1 --2- 172.21.15.17:0/2727744926 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff6bc053b00 0x7ff6bc055fc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7ff6d800b000 tx=0x7ff6d8006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:59.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.318+0000 7ff6ca7fc700 1 -- 172.21.15.17:0/2727744926 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff6dc148050 con 0x7ff6dc0af910 2022-08-01T19:35:59.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.446+0000 7ff6e37b2700 1 -- 172.21.15.17:0/2727744926 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6dc148050 con 0x7ff6dc0af910 2022-08-01T19:35:59.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.446+0000 7ff6ca7fc700 1 -- 172.21.15.17:0/2727744926 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+13414 (secure 0 0 0) 0x7ff6dc148050 con 0x7ff6dc0af910 2022-08-01T19:35:59.453 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:35:59.453 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":41,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:35:58.832382+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:35:59.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.450+0000 7ff6e37b2700 1 -- 172.21.15.17:0/2727744926 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff6bc053b00 msgr2=0x7ff6bc055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:59.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.450+0000 7ff6e37b2700 1 --2- 172.21.15.17:0/2727744926 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff6bc053b00 0x7ff6bc055fc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7ff6d800b000 tx=0x7ff6d8006df0).stop 2022-08-01T19:35:59.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.450+0000 7ff6e37b2700 1 -- 172.21.15.17:0/2727744926 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6dc0af910 msgr2=0x7ff6dc13bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:59.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.450+0000 7ff6e37b2700 1 --2- 172.21.15.17:0/2727744926 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6dc0af910 0x7ff6dc13bf30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7ff6dc136c10 tx=0x7ff6cc007bd0).stop 2022-08-01T19:35:59.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.450+0000 7ff6e37b2700 1 -- 172.21.15.17:0/2727744926 shutdown_connections 2022-08-01T19:35:59.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.450+0000 7ff6e37b2700 1 --2- 172.21.15.17:0/2727744926 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff6bc053b00 0x7ff6bc055fc0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:59.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.450+0000 7ff6e37b2700 1 --2- 172.21.15.17:0/2727744926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6dc0b3630 0x7ff6dc13e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:59.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.450+0000 7ff6e37b2700 1 --2- 172.21.15.17:0/2727744926 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6dc0af910 0x7ff6dc13bf30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:59.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.450+0000 7ff6e37b2700 1 --2- 172.21.15.17:0/2727744926 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6dc0aa0d0 0x7ff6dc13e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:59.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.450+0000 7ff6e37b2700 1 -- 172.21.15.17:0/2727744926 >> 172.21.15.17:0/2727744926 conn(0x7ff6dc060950 msgr2=0x7ff6dc1402f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:59.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.450+0000 7ff6e37b2700 1 -- 172.21.15.17:0/2727744926 shutdown_connections 2022-08-01T19:35:59.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.450+0000 7ff6e37b2700 1 -- 172.21.15.17:0/2727744926 wait complete. 2022-08-01T19:35:59.467 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-08-01T19:35:59.468 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:35:59.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.578+0000 7f971ab33700 1 Processor -- start 2022-08-01T19:35:59.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.578+0000 7f971ab33700 1 -- start start 2022-08-01T19:35:59.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f971ab33700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f97140b2730 0x7f97140666b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:59.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f971ab33700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f9714066ba0 0x7f971414dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:59.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f971ab33700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97140654c0 con 0x7f97140b3190 2022-08-01T19:35:59.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f971ab33700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9714062190 con 0x7f9714066ba0 2022-08-01T19:35:59.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f971ab33700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97140b3570 con 0x7f97140b2730 2022-08-01T19:35:59.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f971a332700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f9714066ba0 0x7f971414dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:59.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f9719330700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f97140b2730 0x7f97140666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:59.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f971a332700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f9714066ba0 0x7f971414dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:36140/0 (socket says 172.21.15.17:36140) 2022-08-01T19:35:59.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f9719b31700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7f97140b3190 0x7f9714060ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:47208/0 (socket says 172.21.15.17:47208) 2022-08-01T19:35:59.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f971a332700 1 -- 172.21.15.17:0/4028928645 learned_addr learned my addr 172.21.15.17:0/4028928645 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:59.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f971a332700 1 -- 172.21.15.17:0/4028928645 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f97140b2730 msgr2=0x7f97140666b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:59.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f971a332700 1 --2- 172.21.15.17:0/4028928645 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f97140b2730 0x7f97140666b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:59.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f971a332700 1 -- 172.21.15.17:0/4028928645 >> v1:172.21.15.77:6789/0 conn(0x7f97140b3190 legacy=0x7f9714060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:59.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f9718b2f700 1 -- 172.21.15.17:0/4028928645 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 847147818 0 0) 0x7f97140654c0 con 0x7f97140b3190 2022-08-01T19:35:59.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f971a332700 1 -- 172.21.15.17:0/4028928645 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f971405fbf0 con 0x7f9714066ba0 2022-08-01T19:35:59.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f971a332700 1 --2- 172.21.15.17:0/4028928645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9714066ba0 0x7f971414dab0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f970c0032f0 tx=0x7f970c009290).ready entity=mon.1 client_cookie=6d311a81b0c3a62e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:59.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f9718b2f700 1 -- 172.21.15.17:0/4028928645 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f970c00b8a0 con 0x7f9714066ba0 2022-08-01T19:35:59.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f9718b2f700 1 -- 172.21.15.17:0/4028928645 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f970c00ba00 con 0x7f9714066ba0 2022-08-01T19:35:59.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f9718b2f700 1 -- 172.21.15.17:0/4028928645 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f970c00bb70 con 0x7f9714066ba0 2022-08-01T19:35:59.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f971ab33700 1 -- 172.21.15.17:0/4028928645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9714066ba0 msgr2=0x7f971414dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:59.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f971ab33700 1 --2- 172.21.15.17:0/4028928645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9714066ba0 0x7f971414dab0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f970c0032f0 tx=0x7f970c009290).stop 2022-08-01T19:35:59.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f971ab33700 1 -- 172.21.15.17:0/4028928645 shutdown_connections 2022-08-01T19:35:59.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f971ab33700 1 --2- 172.21.15.17:0/4028928645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9714066ba0 0x7f971414dab0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:59.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f971ab33700 1 --2- 172.21.15.17:0/4028928645 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f97140b2730 0x7f97140666b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:59.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f971ab33700 1 -- 172.21.15.17:0/4028928645 >> 172.21.15.17:0/4028928645 conn(0x7f9714063870 msgr2=0x7f9714063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:59.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f971ab33700 1 -- 172.21.15.17:0/4028928645 shutdown_connections 2022-08-01T19:35:59.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f971ab33700 1 -- 172.21.15.17:0/4028928645 wait complete. 2022-08-01T19:35:59.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f971ab33700 1 Processor -- start 2022-08-01T19:35:59.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f971ab33700 1 -- start start 2022-08-01T19:35:59.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f971ab33700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9714066ba0 0x7f9714156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:59.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f971ab33700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97140b2730 0x7f971413e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:59.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f971ab33700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f97140b3190 0x7f9714148790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:59.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f971ab33700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97140aa830 con 0x7f9714066ba0 2022-08-01T19:35:59.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f971ab33700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9714156640 con 0x7f97140b2730 2022-08-01T19:35:59.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f971ab33700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97140b44d0 con 0x7f97140b3190 2022-08-01T19:35:59.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f9719b31700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9714066ba0 0x7f9714156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:59.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f971a332700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f97140b3190 0x7f9714148790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:59.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f9719330700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97140b2730 0x7f971413e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:59.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f9719b31700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9714066ba0 0x7f9714156150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38222/0 (socket says 172.21.15.17:38222) 2022-08-01T19:35:59.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f9719b31700 1 -- 172.21.15.17:0/3375723643 learned_addr learned my addr 172.21.15.17:0/3375723643 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:59.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f9719330700 1 -- 172.21.15.17:0/3375723643 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f97140b3190 msgr2=0x7f9714148790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:59.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f9719330700 1 --2- 172.21.15.17:0/3375723643 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f97140b3190 0x7f9714148790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:59.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f9719330700 1 -- 172.21.15.17:0/3375723643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9714066ba0 msgr2=0x7f9714156150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:59.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f9719330700 1 --2- 172.21.15.17:0/3375723643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9714066ba0 0x7f9714156150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:59.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f9719330700 1 -- 172.21.15.17:0/3375723643 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f970c009530 con 0x7f97140b2730 2022-08-01T19:35:59.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f9719330700 1 --2- 172.21.15.17:0/3375723643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97140b2730 0x7f971413e2b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f970400c000 tx=0x7f9704006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:59.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f9702ffd700 1 -- 172.21.15.17:0/3375723643 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9704011750 con 0x7f97140b2730 2022-08-01T19:35:59.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f9702ffd700 1 -- 172.21.15.17:0/3375723643 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97040118b0 con 0x7f97140b2730 2022-08-01T19:35:59.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f971ab33700 1 -- 172.21.15.17:0/3375723643 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97140b4770 con 0x7f97140b2730 2022-08-01T19:35:59.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f9702ffd700 1 -- 172.21.15.17:0/3375723643 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9704011c10 con 0x7f97140b2730 2022-08-01T19:35:59.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.582+0000 7f971ab33700 1 -- 172.21.15.17:0/3375723643 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97140b4f50 con 0x7f97140b2730 2022-08-01T19:35:59.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.586+0000 7f971ab33700 1 -- 172.21.15.17:0/3375723643 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97140b4dc0 con 0x7f97140b2730 2022-08-01T19:35:59.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.586+0000 7f9702ffd700 1 -- 172.21.15.17:0/3375723643 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9704011d70 con 0x7f97140b2730 2022-08-01T19:35:59.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.586+0000 7f9702ffd700 1 --2- 172.21.15.17:0/3375723643 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f96f4053a70 0x7f96f4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:59.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.586+0000 7f9702ffd700 1 -- 172.21.15.17:0/3375723643 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f970406bdf0 con 0x7f97140b2730 2022-08-01T19:35:59.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.586+0000 7f9719b31700 1 --2- 172.21.15.17:0/3375723643 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f96f4053a70 0x7f96f4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:59.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.590+0000 7f9719b31700 1 --2- 172.21.15.17:0/3375723643 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f96f4053a70 0x7f96f4055f30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f971000d000 tx=0x7f9710007c80).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:59.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.594+0000 7f9702ffd700 1 -- 172.21.15.17:0/3375723643 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9704042d00 con 0x7f97140b2730 2022-08-01T19:35:59.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.718+0000 7f971ab33700 1 -- 172.21.15.17:0/3375723643 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f971405fe90 con 0x7f97140b2730 2022-08-01T19:35:59.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.718+0000 7f9702ffd700 1 -- 172.21.15.17:0/3375723643 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v41) v1 ==== 110+0+10 (secure 0 0 0) 0x7f9704046320 con 0x7f97140b2730 2022-08-01T19:35:59.724 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:35:59.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.722+0000 7f971ab33700 1 -- 172.21.15.17:0/3375723643 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f96f4053a70 msgr2=0x7f96f4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:59.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.722+0000 7f971ab33700 1 --2- 172.21.15.17:0/3375723643 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f96f4053a70 0x7f96f4055f30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f971000d000 tx=0x7f9710007c80).stop 2022-08-01T19:35:59.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.722+0000 7f971ab33700 1 -- 172.21.15.17:0/3375723643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97140b2730 msgr2=0x7f971413e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:59.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.722+0000 7f971ab33700 1 --2- 172.21.15.17:0/3375723643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97140b2730 0x7f971413e2b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f970400c000 tx=0x7f9704006df0).stop 2022-08-01T19:35:59.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.722+0000 7f971ab33700 1 -- 172.21.15.17:0/3375723643 shutdown_connections 2022-08-01T19:35:59.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.722+0000 7f971ab33700 1 --2- 172.21.15.17:0/3375723643 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f96f4053a70 0x7f96f4055f30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:59.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.722+0000 7f971ab33700 1 --2- 172.21.15.17:0/3375723643 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f97140b3190 0x7f9714148790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:59.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.722+0000 7f971ab33700 1 --2- 172.21.15.17:0/3375723643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97140b2730 0x7f971413e2b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:59.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.722+0000 7f971ab33700 1 --2- 172.21.15.17:0/3375723643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9714066ba0 0x7f9714156150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:59.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.722+0000 7f971ab33700 1 -- 172.21.15.17:0/3375723643 >> 172.21.15.17:0/3375723643 conn(0x7f9714063870 msgr2=0x7f9714149b90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:59.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.722+0000 7f971ab33700 1 -- 172.21.15.17:0/3375723643 shutdown_connections 2022-08-01T19:35:59.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.722+0000 7f971ab33700 1 -- 172.21.15.17:0/3375723643 wait complete. 2022-08-01T19:35:59.737 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:35:59.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.842+0000 7fc91d25d700 1 Processor -- start 2022-08-01T19:35:59.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.842+0000 7fc91d25d700 1 -- start start 2022-08-01T19:35:59.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.842+0000 7fc91d25d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9180614b0 0x7fc918061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:59.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.842+0000 7fc91d25d700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fc9180ad750 0x7fc9180b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:59.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.842+0000 7fc91d25d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc9181405d0 0x7fc918140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:59.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.842+0000 7fc91d25d700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9180aee80 con 0x7fc9181405d0 2022-08-01T19:35:59.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.842+0000 7fc91d25d700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9180b70b0 con 0x7fc9180ad750 2022-08-01T19:35:59.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.842+0000 7fc91d25d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9180af0b0 con 0x7fc9180614b0 2022-08-01T19:35:59.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.842+0000 7fc91ca5c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc9181405d0 0x7fc918140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:59.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.842+0000 7fc91ca5c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc9181405d0 0x7fc918140a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38232/0 (socket says 172.21.15.17:38232) 2022-08-01T19:35:59.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.842+0000 7fc91ca5c700 1 -- 172.21.15.17:0/980497228 learned_addr learned my addr 172.21.15.17:0/980497228 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:59.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.842+0000 7fc917fff700 1 --2- 172.21.15.17:0/980497228 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9180614b0 0x7fc918061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:59.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.842+0000 7fc9177fe700 1 --2- 172.21.15.17:0/980497228 >> v2:172.21.15.77:3300/0 conn(0x7fc9180ad750 0x7fc9180b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:59.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.842+0000 7fc91ca5c700 1 -- 172.21.15.17:0/980497228 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9180614b0 msgr2=0x7fc918061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:59.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.842+0000 7fc91ca5c700 1 --2- 172.21.15.17:0/980497228 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9180614b0 0x7fc918061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:59.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.842+0000 7fc91ca5c700 1 -- 172.21.15.17:0/980497228 >> v2:172.21.15.77:3300/0 conn(0x7fc9180ad750 msgr2=0x7fc9180b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:35:59.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.842+0000 7fc91ca5c700 1 --2- 172.21.15.17:0/980497228 >> v2:172.21.15.77:3300/0 conn(0x7fc9180ad750 0x7fc9180b9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:59.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.842+0000 7fc91ca5c700 1 -- 172.21.15.17:0/980497228 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9180af350 con 0x7fc9181405d0 2022-08-01T19:35:59.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.842+0000 7fc91ca5c700 1 --2- 172.21.15.17:0/980497228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9181405d0 0x7fc918140a80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fc9100032f0 tx=0x7fc910009290).ready entity=mon.0 client_cookie=5cfc35607039355f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:59.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.842+0000 7fc916ffd700 1 -- 172.21.15.17:0/980497228 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc91000b870 con 0x7fc9181405d0 2022-08-01T19:35:59.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.842+0000 7fc916ffd700 1 -- 172.21.15.17:0/980497228 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc91000b9d0 con 0x7fc9181405d0 2022-08-01T19:35:59.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.842+0000 7fc916ffd700 1 -- 172.21.15.17:0/980497228 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc91000bb40 con 0x7fc9181405d0 2022-08-01T19:35:59.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.846+0000 7fc91d25d700 1 -- 172.21.15.17:0/980497228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9181405d0 msgr2=0x7fc918140a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:59.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.846+0000 7fc91d25d700 1 --2- 172.21.15.17:0/980497228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9181405d0 0x7fc918140a80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fc9100032f0 tx=0x7fc910009290).stop 2022-08-01T19:35:59.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.846+0000 7fc91d25d700 1 -- 172.21.15.17:0/980497228 shutdown_connections 2022-08-01T19:35:59.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.846+0000 7fc91d25d700 1 --2- 172.21.15.17:0/980497228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9181405d0 0x7fc918140a80 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:59.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.846+0000 7fc91d25d700 1 --2- 172.21.15.17:0/980497228 >> v2:172.21.15.77:3300/0 conn(0x7fc9180ad750 0x7fc9180b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:59.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.846+0000 7fc91d25d700 1 --2- 172.21.15.17:0/980497228 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9180614b0 0x7fc918061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:59.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.846+0000 7fc91d25d700 1 -- 172.21.15.17:0/980497228 >> 172.21.15.17:0/980497228 conn(0x7fc918066f70 msgr2=0x7fc918067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:59.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.846+0000 7fc91d25d700 1 -- 172.21.15.17:0/980497228 shutdown_connections 2022-08-01T19:35:59.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.846+0000 7fc91d25d700 1 -- 172.21.15.17:0/980497228 wait complete. 2022-08-01T19:35:59.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.846+0000 7fc91d25d700 1 Processor -- start 2022-08-01T19:35:59.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.846+0000 7fc91d25d700 1 -- start start 2022-08-01T19:35:59.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.846+0000 7fc91d25d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9180614b0 0x7fc91809ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:59.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.846+0000 7fc91d25d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9180ad750 0x7fc918152e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:59.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.846+0000 7fc91d25d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9181405d0 0x7fc9180ac730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:59.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.846+0000 7fc91d25d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9180b0d40 con 0x7fc9181405d0 2022-08-01T19:35:59.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.846+0000 7fc91d25d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9180b0360 con 0x7fc9180ad750 2022-08-01T19:35:59.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.846+0000 7fc91d25d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9180b0600 con 0x7fc9180614b0 2022-08-01T19:35:59.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.846+0000 7fc91ca5c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9181405d0 0x7fc9180ac730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:59.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.846+0000 7fc91ca5c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9181405d0 0x7fc9180ac730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38238/0 (socket says 172.21.15.17:38238) 2022-08-01T19:35:59.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.846+0000 7fc91ca5c700 1 -- 172.21.15.17:0/3403846841 learned_addr learned my addr 172.21.15.17:0/3403846841 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:35:59.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.846+0000 7fc9177fe700 1 --2- 172.21.15.17:0/3403846841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9180ad750 0x7fc918152e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:59.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.846+0000 7fc917fff700 1 --2- 172.21.15.17:0/3403846841 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9180614b0 0x7fc91809ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:59.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.846+0000 7fc91ca5c700 1 -- 172.21.15.17:0/3403846841 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9180614b0 msgr2=0x7fc91809ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:59.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.846+0000 7fc91ca5c700 1 --2- 172.21.15.17:0/3403846841 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9180614b0 0x7fc91809ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:59.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.846+0000 7fc91ca5c700 1 -- 172.21.15.17:0/3403846841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9180ad750 msgr2=0x7fc918152e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:59.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.846+0000 7fc91ca5c700 1 --2- 172.21.15.17:0/3403846841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9180ad750 0x7fc918152e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:59.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.846+0000 7fc91ca5c700 1 -- 172.21.15.17:0/3403846841 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc910009530 con 0x7fc9181405d0 2022-08-01T19:35:59.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.846+0000 7fc917fff700 1 --2- 172.21.15.17:0/3403846841 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9180614b0 0x7fc91809ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:35:59.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.846+0000 7fc9177fe700 1 --2- 172.21.15.17:0/3403846841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9180ad750 0x7fc918152e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:35:59.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.846+0000 7fc91ca5c700 1 --2- 172.21.15.17:0/3403846841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9181405d0 0x7fc9180ac730 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7fc9100032f0 tx=0x7fc910016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:59.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.846+0000 7fc9157fa700 1 -- 172.21.15.17:0/3403846841 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc91000b870 con 0x7fc9181405d0 2022-08-01T19:35:59.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.850+0000 7fc9157fa700 1 -- 172.21.15.17:0/3403846841 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9100107a0 con 0x7fc9181405d0 2022-08-01T19:35:59.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.850+0000 7fc91d25d700 1 -- 172.21.15.17:0/3403846841 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9180b08a0 con 0x7fc9181405d0 2022-08-01T19:35:59.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.850+0000 7fc9157fa700 1 -- 172.21.15.17:0/3403846841 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc910010b00 con 0x7fc9181405d0 2022-08-01T19:35:59.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.850+0000 7fc91d25d700 1 -- 172.21.15.17:0/3403846841 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9180b3ef0 con 0x7fc9181405d0 2022-08-01T19:35:59.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.850+0000 7fc91d25d700 1 -- 172.21.15.17:0/3403846841 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9180af5f0 con 0x7fc9181405d0 2022-08-01T19:35:59.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.850+0000 7fc9157fa700 1 -- 172.21.15.17:0/3403846841 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc910027020 con 0x7fc9181405d0 2022-08-01T19:35:59.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.850+0000 7fc9157fa700 1 --2- 172.21.15.17:0/3403846841 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc8fc053a70 0x7fc8fc055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:35:59.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.850+0000 7fc9157fa700 1 -- 172.21.15.17:0/3403846841 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fc91006d0e0 con 0x7fc9181405d0 2022-08-01T19:35:59.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.850+0000 7fc917fff700 1 --2- 172.21.15.17:0/3403846841 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc8fc053a70 0x7fc8fc055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:35:59.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.850+0000 7fc917fff700 1 --2- 172.21.15.17:0/3403846841 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc8fc053a70 0x7fc8fc055f30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7fc90800c000 tx=0x7fc908008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:35:59.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.858+0000 7fc9157fa700 1 -- 172.21.15.17:0/3403846841 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc9100441e0 con 0x7fc9181405d0 2022-08-01T19:35:59.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.982+0000 7fc91d25d700 1 -- 172.21.15.17:0/3403846841 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc9180574e0 con 0x7fc9181405d0 2022-08-01T19:35:59.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.986+0000 7fc9157fa700 1 -- 172.21.15.17:0/3403846841 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v41) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc910015020 con 0x7fc9181405d0 2022-08-01T19:35:59.990 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:35:59.992 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.986+0000 7fc91d25d700 1 -- 172.21.15.17:0/3403846841 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc8fc053a70 msgr2=0x7fc8fc055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:59.992 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.986+0000 7fc91d25d700 1 --2- 172.21.15.17:0/3403846841 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc8fc053a70 0x7fc8fc055f30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7fc90800c000 tx=0x7fc908008040).stop 2022-08-01T19:35:59.992 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.986+0000 7fc91d25d700 1 -- 172.21.15.17:0/3403846841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9181405d0 msgr2=0x7fc9180ac730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:35:59.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.986+0000 7fc91d25d700 1 --2- 172.21.15.17:0/3403846841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9181405d0 0x7fc9180ac730 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7fc9100032f0 tx=0x7fc910016000).stop 2022-08-01T19:35:59.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.986+0000 7fc91d25d700 1 -- 172.21.15.17:0/3403846841 shutdown_connections 2022-08-01T19:35:59.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.986+0000 7fc91d25d700 1 --2- 172.21.15.17:0/3403846841 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc8fc053a70 0x7fc8fc055f30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:59.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.986+0000 7fc91d25d700 1 --2- 172.21.15.17:0/3403846841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9181405d0 0x7fc9180ac730 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:59.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.986+0000 7fc91d25d700 1 --2- 172.21.15.17:0/3403846841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9180ad750 0x7fc918152e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:59.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.986+0000 7fc91d25d700 1 --2- 172.21.15.17:0/3403846841 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9180614b0 0x7fc91809ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:35:59.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.986+0000 7fc91d25d700 1 -- 172.21.15.17:0/3403846841 >> 172.21.15.17:0/3403846841 conn(0x7fc918066f70 msgr2=0x7fc918143900 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:35:59.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.986+0000 7fc91d25d700 1 -- 172.21.15.17:0/3403846841 shutdown_connections 2022-08-01T19:35:59.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:35:59.986+0000 7fc91d25d700 1 -- 172.21.15.17:0/3403846841 wait complete. 2022-08-01T19:36:00.003 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-08-01T19:36:00.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.098+0000 7f0499afc700 1 Processor -- start 2022-08-01T19:36:00.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.098+0000 7f0499afc700 1 -- start start 2022-08-01T19:36:00.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.098+0000 7f0499afc700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f04940672b0 0x7f04940b9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:00.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.098+0000 7f0499afc700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f0494143b40 0x7f04940aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:00.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.098+0000 7f0499afc700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0494140180 0x7f0494140610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:00.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.098+0000 7f0499afc700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0494143210 con 0x7f04940672b0 2022-08-01T19:36:00.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.098+0000 7f0499afc700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04940b9d60 con 0x7f0494143b40 2022-08-01T19:36:00.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.098+0000 7f0499afc700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0494067690 con 0x7f0494140180 2022-08-01T19:36:00.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.098+0000 7f04937fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f04940672b0 0x7f04940b9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:00.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.098+0000 7f04937fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f04940672b0 0x7f04940b9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38244/0 (socket says 172.21.15.17:38244) 2022-08-01T19:36:00.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.098+0000 7f04937fe700 1 -- 172.21.15.17:0/888377392 learned_addr learned my addr 172.21.15.17:0/888377392 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:00.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.098+0000 7f0492ffd700 1 --2- 172.21.15.17:0/888377392 >> v2:172.21.15.77:3300/0 conn(0x7f0494143b40 0x7f04940aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:00.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.098+0000 7f0493fff700 1 --2- 172.21.15.17:0/888377392 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0494140180 0x7f0494140610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:00.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.098+0000 7f0492ffd700 1 -- 172.21.15.17:0/888377392 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0494140180 msgr2=0x7f0494140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:36:00.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.098+0000 7f0492ffd700 1 --2- 172.21.15.17:0/888377392 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0494140180 0x7f0494140610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:00.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.098+0000 7f0492ffd700 1 -- 172.21.15.17:0/888377392 >> v2:172.21.15.17:3300/0 conn(0x7f04940672b0 msgr2=0x7f04940b9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:36:00.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.098+0000 7f0492ffd700 1 --2- 172.21.15.17:0/888377392 >> v2:172.21.15.17:3300/0 conn(0x7f04940672b0 0x7f04940b9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:00.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.098+0000 7f0492ffd700 1 -- 172.21.15.17:0/888377392 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04940af660 con 0x7f0494143b40 2022-08-01T19:36:00.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.098+0000 7f0492ffd700 1 --2- 172.21.15.17:0/888377392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0494143b40 0x7f04940aed90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f0484002f70 tx=0x7f0484006df0).ready entity=mon.1 client_cookie=c0e0a91a377c85d9 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:00.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.098+0000 7f04927fc700 1 -- 172.21.15.17:0/888377392 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f048400d4a0 con 0x7f0494143b40 2022-08-01T19:36:00.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.098+0000 7f04927fc700 1 -- 172.21.15.17:0/888377392 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0484000e90 con 0x7f0494143b40 2022-08-01T19:36:00.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.098+0000 7f04927fc700 1 -- 172.21.15.17:0/888377392 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f048400d7f0 con 0x7f0494143b40 2022-08-01T19:36:00.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.098+0000 7f0499afc700 1 -- 172.21.15.17:0/888377392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0494143b40 msgr2=0x7f04940aed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:00.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.098+0000 7f0499afc700 1 --2- 172.21.15.17:0/888377392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0494143b40 0x7f04940aed90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f0484002f70 tx=0x7f0484006df0).stop 2022-08-01T19:36:00.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.098+0000 7f0499afc700 1 -- 172.21.15.17:0/888377392 shutdown_connections 2022-08-01T19:36:00.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.098+0000 7f0499afc700 1 --2- 172.21.15.17:0/888377392 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0494140180 0x7f0494140610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:00.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.098+0000 7f0499afc700 1 --2- 172.21.15.17:0/888377392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0494143b40 0x7f04940aed90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:00.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.098+0000 7f0499afc700 1 --2- 172.21.15.17:0/888377392 >> v2:172.21.15.17:3300/0 conn(0x7f04940672b0 0x7f04940b9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:00.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.098+0000 7f0499afc700 1 -- 172.21.15.17:0/888377392 >> 172.21.15.17:0/888377392 conn(0x7f049405fb80 msgr2=0x7f0494061830 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:00.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.098+0000 7f0499afc700 1 -- 172.21.15.17:0/888377392 shutdown_connections 2022-08-01T19:36:00.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.098+0000 7f0499afc700 1 -- 172.21.15.17:0/888377392 wait complete. 2022-08-01T19:36:00.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.098+0000 7f0499afc700 1 Processor -- start 2022-08-01T19:36:00.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.098+0000 7f0499afc700 1 -- start start 2022-08-01T19:36:00.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.098+0000 7f0499afc700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f04940672b0 0x7f0494149250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:00.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.098+0000 7f0499afc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0494140180 0x7f04940b7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:00.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.102+0000 7f0499afc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0494143b40 0x7f0494152c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:00.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.102+0000 7f0499afc700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0494145bd0 con 0x7f0494143b40 2022-08-01T19:36:00.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.102+0000 7f0499afc700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0494144a90 con 0x7f0494140180 2022-08-01T19:36:00.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.102+0000 7f0499afc700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0494153180 con 0x7f04940672b0 2022-08-01T19:36:00.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.102+0000 7f0493fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0494143b40 0x7f0494152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:00.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.102+0000 7f0493fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0494143b40 0x7f0494152c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38252/0 (socket says 172.21.15.17:38252) 2022-08-01T19:36:00.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.102+0000 7f0493fff700 1 -- 172.21.15.17:0/2006756523 learned_addr learned my addr 172.21.15.17:0/2006756523 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:00.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.102+0000 7f0492ffd700 1 --2- 172.21.15.17:0/2006756523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0494140180 0x7f04940b7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:00.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.102+0000 7f04937fe700 1 --2- 172.21.15.17:0/2006756523 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f04940672b0 0x7f0494149250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:00.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.102+0000 7f0493fff700 1 -- 172.21.15.17:0/2006756523 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f04940672b0 msgr2=0x7f0494149250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:00.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.102+0000 7f0493fff700 1 --2- 172.21.15.17:0/2006756523 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f04940672b0 0x7f0494149250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:00.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.102+0000 7f0493fff700 1 -- 172.21.15.17:0/2006756523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0494140180 msgr2=0x7f04940b7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:00.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.102+0000 7f0493fff700 1 --2- 172.21.15.17:0/2006756523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0494140180 0x7f04940b7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:00.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.102+0000 7f0493fff700 1 -- 172.21.15.17:0/2006756523 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f048400b180 con 0x7f0494143b40 2022-08-01T19:36:00.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.102+0000 7f0493fff700 1 --2- 172.21.15.17:0/2006756523 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0494143b40 0x7f0494152c90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f048c006e00 tx=0x7f048c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:00.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.102+0000 7f0490ff9700 1 -- 172.21.15.17:0/2006756523 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f048c00eeb0 con 0x7f0494143b40 2022-08-01T19:36:00.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.102+0000 7f0490ff9700 1 -- 172.21.15.17:0/2006756523 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048c00c850 con 0x7f0494143b40 2022-08-01T19:36:00.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.102+0000 7f0499afc700 1 -- 172.21.15.17:0/2006756523 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0494145930 con 0x7f0494143b40 2022-08-01T19:36:00.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.102+0000 7f0490ff9700 1 -- 172.21.15.17:0/2006756523 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f048c00cbb0 con 0x7f0494143b40 2022-08-01T19:36:00.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.102+0000 7f0499afc700 1 -- 172.21.15.17:0/2006756523 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0494155bb0 con 0x7f0494143b40 2022-08-01T19:36:00.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.102+0000 7f0499afc700 1 -- 172.21.15.17:0/2006756523 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0494143400 con 0x7f0494143b40 2022-08-01T19:36:00.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.102+0000 7f0490ff9700 1 -- 172.21.15.17:0/2006756523 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f048c00cd10 con 0x7f0494143b40 2022-08-01T19:36:00.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.106+0000 7f0490ff9700 1 --2- 172.21.15.17:0/2006756523 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0474053a70 0x7f0474055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:00.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.106+0000 7f0490ff9700 1 -- 172.21.15.17:0/2006756523 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f048c06c0c0 con 0x7f0494143b40 2022-08-01T19:36:00.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.106+0000 7f04937fe700 1 --2- 172.21.15.17:0/2006756523 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0474053a70 0x7f0474055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:00.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.106+0000 7f04937fe700 1 --2- 172.21.15.17:0/2006756523 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0474053a70 0x7f0474055f30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f0488008000 tx=0x7f0488008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:00.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.110+0000 7f0490ff9700 1 -- 172.21.15.17:0/2006756523 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f048c0431c0 con 0x7f0494143b40 2022-08-01T19:36:00.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.238+0000 7f0499afc700 1 -- 172.21.15.17:0/2006756523 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f04940574e0 con 0x7f0494143b40 2022-08-01T19:36:00.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.238+0000 7f0490ff9700 1 -- 172.21.15.17:0/2006756523 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v41) v1 ==== 84+0+143 (secure 0 0 0) 0x7f048c0467e0 con 0x7f0494143b40 2022-08-01T19:36:00.245 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:36:00.246 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2022-08-01T19:36:00.246 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:36:00.246 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 1, 2022-08-01T19:36:00.246 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": "device_health_metrics" 2022-08-01T19:36:00.247 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:36:00.247 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:36:00.247 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 2, 2022-08-01T19:36:00.247 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": "rbd" 2022-08-01T19:36:00.247 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:36:00.248 INFO:teuthology.orchestra.run.smithi017.stdout:] 2022-08-01T19:36:00.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.242+0000 7f0499afc700 1 -- 172.21.15.17:0/2006756523 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0474053a70 msgr2=0x7f0474055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:00.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.242+0000 7f0499afc700 1 --2- 172.21.15.17:0/2006756523 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0474053a70 0x7f0474055f30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f0488008000 tx=0x7f0488008040).stop 2022-08-01T19:36:00.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.242+0000 7f0499afc700 1 -- 172.21.15.17:0/2006756523 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0494143b40 msgr2=0x7f0494152c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:00.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.242+0000 7f0499afc700 1 --2- 172.21.15.17:0/2006756523 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0494143b40 0x7f0494152c90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f048c006e00 tx=0x7f048c0091d0).stop 2022-08-01T19:36:00.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.242+0000 7f0499afc700 1 -- 172.21.15.17:0/2006756523 shutdown_connections 2022-08-01T19:36:00.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.242+0000 7f0499afc700 1 --2- 172.21.15.17:0/2006756523 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0474053a70 0x7f0474055f30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:00.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.242+0000 7f0499afc700 1 --2- 172.21.15.17:0/2006756523 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0494143b40 0x7f0494152c90 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:00.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.242+0000 7f0499afc700 1 --2- 172.21.15.17:0/2006756523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0494140180 0x7f04940b7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:00.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.242+0000 7f0499afc700 1 --2- 172.21.15.17:0/2006756523 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f04940672b0 0x7f0494149250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:00.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.242+0000 7f0499afc700 1 -- 172.21.15.17:0/2006756523 >> 172.21.15.17:0/2006756523 conn(0x7f049405fb80 msgr2=0x7f049405cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:00.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.242+0000 7f0499afc700 1 -- 172.21.15.17:0/2006756523 shutdown_connections 2022-08-01T19:36:00.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.242+0000 7f0499afc700 1 -- 172.21.15.17:0/2006756523 wait complete. 2022-08-01T19:36:00.258 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-08-01T19:36:00.259 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-08-01T19:36:00.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.362+0000 7f548f181700 1 Processor -- start 2022-08-01T19:36:00.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.362+0000 7f548f181700 1 -- start start 2022-08-01T19:36:00.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.362+0000 7f548f181700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f54880b3190 0x7f5488060ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:00.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.362+0000 7f548f181700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f54880b2730 0x7f5488061190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:00.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.362+0000 7f548f181700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54880654c0 con 0x7f54880666b0 2022-08-01T19:36:00.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.362+0000 7f548f181700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5488062190 con 0x7f54880b2730 2022-08-01T19:36:00.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.362+0000 7f548f181700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54880b3570 con 0x7f54880b3190 2022-08-01T19:36:00.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.362+0000 7f548d97e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f54880b2730 0x7f5488061190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:00.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.362+0000 7f548d97e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f54880b2730 0x7f5488061190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38258/0 (socket says 172.21.15.17:38258) 2022-08-01T19:36:00.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.362+0000 7f548d97e700 1 -- 172.21.15.17:0/2464567620 learned_addr learned my addr 172.21.15.17:0/2464567620 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:00.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.362+0000 7f548e17f700 1 --2- 172.21.15.17:0/2464567620 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f54880b3190 0x7f5488060ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:00.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.366+0000 7f548d97e700 1 -- 172.21.15.17:0/2464567620 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f54880b3190 msgr2=0x7f5488060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:36:00.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.366+0000 7f548d97e700 1 --2- 172.21.15.17:0/2464567620 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f54880b3190 0x7f5488060ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:00.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.366+0000 7f548d97e700 1 -- 172.21.15.17:0/2464567620 >> v1:172.21.15.77:6789/0 conn(0x7f54880666b0 legacy=0x7f5488066b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:00.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.366+0000 7f548d97e700 1 -- 172.21.15.17:0/2464567620 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548805fbf0 con 0x7f54880b2730 2022-08-01T19:36:00.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.366+0000 7f548d97e700 1 --2- 172.21.15.17:0/2464567620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54880b2730 0x7f5488061190 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f5478002f70 tx=0x7f5478006eb0).ready entity=mon.0 client_cookie=c3f1eaf6ef2d683b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:00.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.366+0000 7f548d17d700 1 -- 172.21.15.17:0/2464567620 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f547800d5c0 con 0x7f54880b2730 2022-08-01T19:36:00.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.366+0000 7f548d17d700 1 -- 172.21.15.17:0/2464567620 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5478000e90 con 0x7f54880b2730 2022-08-01T19:36:00.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.366+0000 7f548d17d700 1 -- 172.21.15.17:0/2464567620 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f547800d950 con 0x7f54880b2730 2022-08-01T19:36:00.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.366+0000 7f548f181700 1 -- 172.21.15.17:0/2464567620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54880b2730 msgr2=0x7f5488061190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:00.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.366+0000 7f548f181700 1 --2- 172.21.15.17:0/2464567620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54880b2730 0x7f5488061190 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f5478002f70 tx=0x7f5478006eb0).stop 2022-08-01T19:36:00.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.366+0000 7f548f181700 1 -- 172.21.15.17:0/2464567620 shutdown_connections 2022-08-01T19:36:00.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.366+0000 7f548f181700 1 --2- 172.21.15.17:0/2464567620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54880b2730 0x7f5488061190 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:00.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.366+0000 7f548f181700 1 --2- 172.21.15.17:0/2464567620 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f54880b3190 0x7f5488060ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:00.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.366+0000 7f548f181700 1 -- 172.21.15.17:0/2464567620 >> 172.21.15.17:0/2464567620 conn(0x7f5488063870 msgr2=0x7f5488063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:00.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.366+0000 7f548f181700 1 -- 172.21.15.17:0/2464567620 shutdown_connections 2022-08-01T19:36:00.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.366+0000 7f548f181700 1 -- 172.21.15.17:0/2464567620 wait complete. 2022-08-01T19:36:00.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.366+0000 7f548f181700 1 Processor -- start 2022-08-01T19:36:00.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.370+0000 7f548f181700 1 -- start start 2022-08-01T19:36:00.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.370+0000 7f548f181700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54880666b0 0x7f54880b6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:00.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.370+0000 7f548f181700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54880b2730 0x7f54880aaa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:00.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.370+0000 7f548f181700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f54880b3190 0x7f54880aaf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:00.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.370+0000 7f548f181700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54880ab400 con 0x7f54880b2730 2022-08-01T19:36:00.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.370+0000 7f548f181700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54880aa350 con 0x7f54880666b0 2022-08-01T19:36:00.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.370+0000 7f548f181700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54880ab8f0 con 0x7f54880b3190 2022-08-01T19:36:00.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.370+0000 7f548d97e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54880b2730 0x7f54880aaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:00.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.370+0000 7f548d97e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54880b2730 0x7f54880aaa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38264/0 (socket says 172.21.15.17:38264) 2022-08-01T19:36:00.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.370+0000 7f548d97e700 1 -- 172.21.15.17:0/102895145 learned_addr learned my addr 172.21.15.17:0/102895145 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:00.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.370+0000 7f548e17f700 1 --2- 172.21.15.17:0/102895145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54880666b0 0x7f54880b6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:00.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.370+0000 7f548e980700 1 --2- 172.21.15.17:0/102895145 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f54880b3190 0x7f54880aaf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:00.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.370+0000 7f548d97e700 1 -- 172.21.15.17:0/102895145 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f54880b3190 msgr2=0x7f54880aaf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:00.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.370+0000 7f548d97e700 1 --2- 172.21.15.17:0/102895145 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f54880b3190 0x7f54880aaf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:00.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.370+0000 7f548d97e700 1 -- 172.21.15.17:0/102895145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54880666b0 msgr2=0x7f54880b6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:00.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.370+0000 7f548d97e700 1 --2- 172.21.15.17:0/102895145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54880666b0 0x7f54880b6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:00.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.370+0000 7f548d97e700 1 -- 172.21.15.17:0/102895145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f547800b1e0 con 0x7f54880b2730 2022-08-01T19:36:00.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.370+0000 7f548d97e700 1 --2- 172.21.15.17:0/102895145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54880b2730 0x7f54880aaa20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f5478002f70 tx=0x7f5478015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:00.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.370+0000 7f54777fe700 1 -- 172.21.15.17:0/102895145 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f547800d940 con 0x7f54880b2730 2022-08-01T19:36:00.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.370+0000 7f54777fe700 1 -- 172.21.15.17:0/102895145 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5478011b80 con 0x7f54880b2730 2022-08-01T19:36:00.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.370+0000 7f548f181700 1 -- 172.21.15.17:0/102895145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54880abb90 con 0x7f54880b2730 2022-08-01T19:36:00.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.370+0000 7f54777fe700 1 -- 172.21.15.17:0/102895145 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f547801e460 con 0x7f54880b2730 2022-08-01T19:36:00.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.370+0000 7f548f181700 1 -- 172.21.15.17:0/102895145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54880aa580 con 0x7f54880b2730 2022-08-01T19:36:00.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.370+0000 7f548f181700 1 -- 172.21.15.17:0/102895145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f548805fe90 con 0x7f54880b2730 2022-08-01T19:36:00.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.374+0000 7f54777fe700 1 -- 172.21.15.17:0/102895145 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f547801a040 con 0x7f54880b2730 2022-08-01T19:36:00.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.374+0000 7f54777fe700 1 --2- 172.21.15.17:0/102895145 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5468053a70 0x7f5468055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:00.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.374+0000 7f54777fe700 1 -- 172.21.15.17:0/102895145 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f547806cb90 con 0x7f54880b2730 2022-08-01T19:36:00.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.374+0000 7f548e17f700 1 --2- 172.21.15.17:0/102895145 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5468053a70 0x7f5468055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:00.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.374+0000 7f548e17f700 1 --2- 172.21.15.17:0/102895145 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5468053a70 0x7f5468055f30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f548400c000 tx=0x7f5484008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:00.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.378+0000 7f54777fe700 1 -- 172.21.15.17:0/102895145 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5478043d10 con 0x7f54880b2730 2022-08-01T19:36:00.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:00.506+0000 7f548f181700 1 -- 172.21.15.17:0/102895145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f54880574e0 con 0x7f54880b2730 2022-08-01T19:36:01.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.470+0000 7f54777fe700 1 -- 172.21.15.17:0/102895145 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v42) v1 ==== 133+0+0 (secure 0 0 0) 0x7f54780472b0 con 0x7f54880b2730 2022-08-01T19:36:01.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.474+0000 7f548f181700 1 -- 172.21.15.17:0/102895145 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5468053a70 msgr2=0x7f5468055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:01.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.474+0000 7f548f181700 1 --2- 172.21.15.17:0/102895145 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5468053a70 0x7f5468055f30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f548400c000 tx=0x7f5484008040).stop 2022-08-01T19:36:01.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.474+0000 7f548f181700 1 -- 172.21.15.17:0/102895145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54880b2730 msgr2=0x7f54880aaa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:01.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.474+0000 7f548f181700 1 --2- 172.21.15.17:0/102895145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54880b2730 0x7f54880aaa20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f5478002f70 tx=0x7f5478015000).stop 2022-08-01T19:36:01.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.478+0000 7f548f181700 1 -- 172.21.15.17:0/102895145 shutdown_connections 2022-08-01T19:36:01.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.478+0000 7f548f181700 1 --2- 172.21.15.17:0/102895145 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5468053a70 0x7f5468055f30 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:01.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.478+0000 7f548f181700 1 --2- 172.21.15.17:0/102895145 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f54880b3190 0x7f54880aaf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:01.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.478+0000 7f548f181700 1 --2- 172.21.15.17:0/102895145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54880b2730 0x7f54880aaa20 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:01.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.478+0000 7f548f181700 1 --2- 172.21.15.17:0/102895145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54880666b0 0x7f54880b6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:01.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.478+0000 7f548f181700 1 -- 172.21.15.17:0/102895145 >> 172.21.15.17:0/102895145 conn(0x7f5488063870 msgr2=0x7f54881437a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:01.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.482+0000 7f548f181700 1 -- 172.21.15.17:0/102895145 shutdown_connections 2022-08-01T19:36:01.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.482+0000 7f548f181700 1 -- 172.21.15.17:0/102895145 wait complete. 2022-08-01T19:36:01.486 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' created 2022-08-01T19:36:01.500 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-08-01T19:36:01.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.610+0000 7fc875b2b700 1 Processor -- start 2022-08-01T19:36:01.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.610+0000 7fc875b2b700 1 -- start start 2022-08-01T19:36:01.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.610+0000 7fc875b2b700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fc8700b2710 0x7fc87014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:01.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.610+0000 7fc875b2b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc8700a8c50 0x7fc87014b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:01.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.610+0000 7fc875b2b700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8700a7250 con 0x7fc8700b01a0 2022-08-01T19:36:01.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.610+0000 7fc875b2b700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8700a7480 con 0x7fc8700a8c50 2022-08-01T19:36:01.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.610+0000 7fc875b2b700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8700a7720 con 0x7fc8700b2710 2022-08-01T19:36:01.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.610+0000 7fc87532a700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc8700a8c50 0x7fc87014b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:01.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.610+0000 7fc87532a700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc8700a8c50 0x7fc87014b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38272/0 (socket says 172.21.15.17:38272) 2022-08-01T19:36:01.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.610+0000 7fc87532a700 1 -- 172.21.15.17:0/363150175 learned_addr learned my addr 172.21.15.17:0/363150175 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:01.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.610+0000 7fc874b29700 1 --2- 172.21.15.17:0/363150175 >> v2:172.21.15.77:3300/0 conn(0x7fc8700b2710 0x7fc87014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:01.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.610+0000 7fc87532a700 1 -- 172.21.15.17:0/363150175 >> v2:172.21.15.77:3300/0 conn(0x7fc8700b2710 msgr2=0x7fc87014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:36:01.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.610+0000 7fc87532a700 1 --2- 172.21.15.17:0/363150175 >> v2:172.21.15.77:3300/0 conn(0x7fc8700b2710 0x7fc87014afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:01.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.610+0000 7fc87532a700 1 -- 172.21.15.17:0/363150175 >> v1:172.21.15.17:6789/0 conn(0x7fc8700b01a0 legacy=0x7fc8700a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:01.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.610+0000 7fc87532a700 1 -- 172.21.15.17:0/363150175 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8700a79c0 con 0x7fc8700a8c50 2022-08-01T19:36:01.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.610+0000 7fc86f7fe700 1 -- 172.21.15.17:0/363150175 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4034546785 0 0) 0x7fc8700a7250 con 0x7fc8700b01a0 2022-08-01T19:36:01.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.610+0000 7fc87532a700 1 --2- 172.21.15.17:0/363150175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8700a8c50 0x7fc87014b4c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7fc868002f70 tx=0x7fc868006eb0).ready entity=mon.0 client_cookie=71c5792c7018894f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:01.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.610+0000 7fc86f7fe700 1 -- 172.21.15.17:0/363150175 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc86800d610 con 0x7fc8700a8c50 2022-08-01T19:36:01.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.610+0000 7fc86f7fe700 1 -- 172.21.15.17:0/363150175 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc868000e90 con 0x7fc8700a8c50 2022-08-01T19:36:01.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.610+0000 7fc86f7fe700 1 -- 172.21.15.17:0/363150175 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc86800d960 con 0x7fc8700a8c50 2022-08-01T19:36:01.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.610+0000 7fc875b2b700 1 -- 172.21.15.17:0/363150175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8700a8c50 msgr2=0x7fc87014b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:01.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.610+0000 7fc875b2b700 1 --2- 172.21.15.17:0/363150175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8700a8c50 0x7fc87014b4c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7fc868002f70 tx=0x7fc868006eb0).stop 2022-08-01T19:36:01.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.610+0000 7fc875b2b700 1 -- 172.21.15.17:0/363150175 shutdown_connections 2022-08-01T19:36:01.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.610+0000 7fc875b2b700 1 --2- 172.21.15.17:0/363150175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8700a8c50 0x7fc87014b4c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:01.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.610+0000 7fc875b2b700 1 --2- 172.21.15.17:0/363150175 >> v2:172.21.15.77:3300/0 conn(0x7fc8700b2710 0x7fc87014afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:01.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.610+0000 7fc875b2b700 1 -- 172.21.15.17:0/363150175 >> 172.21.15.17:0/363150175 conn(0x7fc870060310 msgr2=0x7fc8700b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:01.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.610+0000 7fc875b2b700 1 -- 172.21.15.17:0/363150175 shutdown_connections 2022-08-01T19:36:01.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.610+0000 7fc875b2b700 1 -- 172.21.15.17:0/363150175 wait complete. 2022-08-01T19:36:01.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.614+0000 7fc875b2b700 1 Processor -- start 2022-08-01T19:36:01.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.614+0000 7fc875b2b700 1 -- start start 2022-08-01T19:36:01.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.614+0000 7fc875b2b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc8700a8c50 0x7fc8700ab630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:01.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.614+0000 7fc875b2b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8700b01a0 0x7fc8700ac710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:01.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.614+0000 7fc875b2b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8700b2710 0x7fc87014a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:01.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.614+0000 7fc875b2b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8701571d0 con 0x7fc8700b2710 2022-08-01T19:36:01.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.614+0000 7fc875b2b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8701576a0 con 0x7fc8700b01a0 2022-08-01T19:36:01.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.614+0000 7fc875b2b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc870157400 con 0x7fc8700a8c50 2022-08-01T19:36:01.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.614+0000 7fc87532a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8700b2710 0x7fc87014a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:01.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.614+0000 7fc87532a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8700b2710 0x7fc87014a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38278/0 (socket says 172.21.15.17:38278) 2022-08-01T19:36:01.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.614+0000 7fc87532a700 1 -- 172.21.15.17:0/2497928322 learned_addr learned my addr 172.21.15.17:0/2497928322 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:01.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.614+0000 7fc874b29700 1 --2- 172.21.15.17:0/2497928322 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc8700a8c50 0x7fc8700ab630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:01.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.614+0000 7fc86ffff700 1 --2- 172.21.15.17:0/2497928322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8700b01a0 0x7fc8700ac710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:01.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.614+0000 7fc87532a700 1 -- 172.21.15.17:0/2497928322 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc8700a8c50 msgr2=0x7fc8700ab630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:01.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.614+0000 7fc87532a700 1 --2- 172.21.15.17:0/2497928322 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc8700a8c50 0x7fc8700ab630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:01.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.614+0000 7fc87532a700 1 -- 172.21.15.17:0/2497928322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8700b01a0 msgr2=0x7fc8700ac710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:01.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.614+0000 7fc87532a700 1 --2- 172.21.15.17:0/2497928322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8700b01a0 0x7fc8700ac710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:01.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.614+0000 7fc87532a700 1 -- 172.21.15.17:0/2497928322 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc86800b1e0 con 0x7fc8700b2710 2022-08-01T19:36:01.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.614+0000 7fc87532a700 1 --2- 172.21.15.17:0/2497928322 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8700b2710 0x7fc87014a050 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fc868002f70 tx=0x7fc868015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:01.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.614+0000 7fc86dffb700 1 -- 172.21.15.17:0/2497928322 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc868011760 con 0x7fc8700b2710 2022-08-01T19:36:01.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.614+0000 7fc86dffb700 1 -- 172.21.15.17:0/2497928322 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8680118c0 con 0x7fc8700b2710 2022-08-01T19:36:01.636 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.614+0000 7fc875b2b700 1 -- 172.21.15.17:0/2497928322 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc870063190 con 0x7fc8700b2710 2022-08-01T19:36:01.636 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.614+0000 7fc86dffb700 1 -- 172.21.15.17:0/2497928322 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc868011c20 con 0x7fc8700b2710 2022-08-01T19:36:01.636 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.614+0000 7fc875b2b700 1 -- 172.21.15.17:0/2497928322 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8700634a0 con 0x7fc8700b2710 2022-08-01T19:36:01.638 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.618+0000 7fc875b2b700 1 -- 172.21.15.17:0/2497928322 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8700ab370 con 0x7fc8700b2710 2022-08-01T19:36:01.638 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.618+0000 7fc86dffb700 1 -- 172.21.15.17:0/2497928322 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc868025030 con 0x7fc8700b2710 2022-08-01T19:36:01.638 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.618+0000 7fc86dffb700 1 --2- 172.21.15.17:0/2497928322 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc850053a70 0x7fc850055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:01.639 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.618+0000 7fc86dffb700 1 -- 172.21.15.17:0/2497928322 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5954+0+0 (secure 0 0 0) 0x7fc86806d0a0 con 0x7fc8700b2710 2022-08-01T19:36:01.639 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.618+0000 7fc874b29700 1 --2- 172.21.15.17:0/2497928322 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc850053a70 0x7fc850055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:01.639 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.618+0000 7fc874b29700 1 --2- 172.21.15.17:0/2497928322 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc850053a70 0x7fc850055f30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7fc86400c000 tx=0x7fc864008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:01.639 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.626+0000 7fc86dffb700 1 -- 172.21.15.17:0/2497928322 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc8700ab370 con 0x7fc8700b2710 2022-08-01T19:36:01.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.754+0000 7fc875b2b700 1 -- 172.21.15.17:0/2497928322 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fc8700ab370 con 0x7fc8700b2710 2022-08-01T19:36:01.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.854+0000 7fc86dffb700 1 -- 172.21.15.17:0/2497928322 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v43) v1 ==== 125+0+0 (secure 0 0 0) 0x7fc8700ab370 con 0x7fc8700b2710 2022-08-01T19:36:01.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.858+0000 7fc875b2b700 1 -- 172.21.15.17:0/2497928322 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc850053a70 msgr2=0x7fc850055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:01.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.858+0000 7fc875b2b700 1 --2- 172.21.15.17:0/2497928322 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc850053a70 0x7fc850055f30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7fc86400c000 tx=0x7fc864008040).stop 2022-08-01T19:36:01.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.858+0000 7fc875b2b700 1 -- 172.21.15.17:0/2497928322 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8700b2710 msgr2=0x7fc87014a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:01.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.858+0000 7fc875b2b700 1 --2- 172.21.15.17:0/2497928322 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8700b2710 0x7fc87014a050 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fc868002f70 tx=0x7fc868015000).stop 2022-08-01T19:36:01.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.862+0000 7fc875b2b700 1 -- 172.21.15.17:0/2497928322 shutdown_connections 2022-08-01T19:36:01.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.862+0000 7fc875b2b700 1 --2- 172.21.15.17:0/2497928322 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc850053a70 0x7fc850055f30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:01.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.862+0000 7fc875b2b700 1 --2- 172.21.15.17:0/2497928322 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8700b2710 0x7fc87014a050 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:01.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.862+0000 7fc875b2b700 1 --2- 172.21.15.17:0/2497928322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8700b01a0 0x7fc8700ac710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:01.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.862+0000 7fc875b2b700 1 --2- 172.21.15.17:0/2497928322 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc8700a8c50 0x7fc8700ab630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:01.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.862+0000 7fc875b2b700 1 -- 172.21.15.17:0/2497928322 >> 172.21.15.17:0/2497928322 conn(0x7fc870060310 msgr2=0x7fc87005be30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:01.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.862+0000 7fc875b2b700 1 -- 172.21.15.17:0/2497928322 shutdown_connections 2022-08-01T19:36:01.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.862+0000 7fc875b2b700 1 -- 172.21.15.17:0/2497928322 wait complete. 2022-08-01T19:36:01.870 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' created 2022-08-01T19:36:01.878 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-08-01T19:36:01.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.986+0000 7f493bfff700 1 Processor -- start 2022-08-01T19:36:01.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.986+0000 7f493bfff700 1 -- start start 2022-08-01T19:36:02.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.986+0000 7f493bfff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f493c0a9da0 0x7f493c060950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:02.002 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.986+0000 7f493bfff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f493c1402d0 0x7f493c140760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:02.002 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.986+0000 7f493bfff700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f493c067030 con 0x7f493c0b31d0 2022-08-01T19:36:02.003 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.986+0000 7f493bfff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f493c064a70 con 0x7f493c0a9da0 2022-08-01T19:36:02.003 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.986+0000 7f493bfff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493c061f60 con 0x7f493c1402d0 2022-08-01T19:36:02.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.986+0000 7f493affd700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f493c0b31d0 0x7f493c066020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58708/0 (socket says 172.21.15.17:58708) 2022-08-01T19:36:02.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.986+0000 7f493affd700 1 -- 172.21.15.17:0/501669819 learned_addr learned my addr 172.21.15.17:0/501669819 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:36:02.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.986+0000 7f493b7fe700 1 --2- 172.21.15.17:0/501669819 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f493c1402d0 0x7f493c140760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:02.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.986+0000 7f493a7fc700 1 --2- 172.21.15.17:0/501669819 >> v2:172.21.15.77:3300/0 conn(0x7f493c0a9da0 0x7f493c060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:02.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.986+0000 7f4939ffb700 1 -- 172.21.15.17:0/501669819 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1644310960 0 0) 0x7f493c067030 con 0x7f493c0b31d0 2022-08-01T19:36:02.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.986+0000 7f4939ffb700 1 -- 172.21.15.17:0/501669819 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f491c003260 con 0x7f493c0b31d0 2022-08-01T19:36:02.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.986+0000 7f493a7fc700 1 -- 172.21.15.17:0/501669819 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f493c1402d0 msgr2=0x7f493c140760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:36:02.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.986+0000 7f493a7fc700 1 --2- 172.21.15.17:0/501669819 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f493c1402d0 0x7f493c140760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:02.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.986+0000 7f493a7fc700 1 -- 172.21.15.17:0/501669819 >> v1:172.21.15.17:6789/0 conn(0x7f493c0b31d0 legacy=0x7f493c066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:02.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.986+0000 7f493a7fc700 1 -- 172.21.15.17:0/501669819 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f493c0b35b0 con 0x7f493c0a9da0 2022-08-01T19:36:02.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.986+0000 7f493a7fc700 1 --2- 172.21.15.17:0/501669819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f493c0a9da0 0x7f493c060950 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f493c063230 tx=0x7f4924006df0).ready entity=mon.1 client_cookie=71e3759ad0f00cdf server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:02.007 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.986+0000 7f4939ffb700 1 -- 172.21.15.17:0/501669819 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f492400d570 con 0x7f493c0a9da0 2022-08-01T19:36:02.007 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.986+0000 7f4939ffb700 1 -- 172.21.15.17:0/501669819 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4924000e90 con 0x7f493c0a9da0 2022-08-01T19:36:02.007 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.986+0000 7f4939ffb700 1 -- 172.21.15.17:0/501669819 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f492400d8c0 con 0x7f493c0a9da0 2022-08-01T19:36:02.007 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.986+0000 7f493bfff700 1 -- 172.21.15.17:0/501669819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f493c0a9da0 msgr2=0x7f493c060950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:02.008 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.986+0000 7f493bfff700 1 --2- 172.21.15.17:0/501669819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f493c0a9da0 0x7f493c060950 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f493c063230 tx=0x7f4924006df0).stop 2022-08-01T19:36:02.008 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.986+0000 7f493bfff700 1 -- 172.21.15.17:0/501669819 shutdown_connections 2022-08-01T19:36:02.008 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.986+0000 7f493bfff700 1 --2- 172.21.15.17:0/501669819 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f493c1402d0 0x7f493c140760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:02.008 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.986+0000 7f493bfff700 1 --2- 172.21.15.17:0/501669819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f493c0a9da0 0x7f493c060950 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:02.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.986+0000 7f493bfff700 1 -- 172.21.15.17:0/501669819 >> 172.21.15.17:0/501669819 conn(0x7f493c063870 msgr2=0x7f493c063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:02.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.986+0000 7f493bfff700 1 -- 172.21.15.17:0/501669819 shutdown_connections 2022-08-01T19:36:02.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.986+0000 7f493bfff700 1 -- 172.21.15.17:0/501669819 wait complete. 2022-08-01T19:36:02.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.990+0000 7f493bfff700 1 Processor -- start 2022-08-01T19:36:02.010 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.990+0000 7f493bfff700 1 -- start start 2022-08-01T19:36:02.010 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.990+0000 7f493bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f493c0a9da0 0x7f493c0ae160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:02.010 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.990+0000 7f493bfff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f493c0b31d0 0x7f493c13f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:02.010 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.990+0000 7f493bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f493c1402d0 0x7f493c156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:02.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.990+0000 7f493bfff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493c156510 con 0x7f493c1402d0 2022-08-01T19:36:02.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.990+0000 7f493bfff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493c0adb40 con 0x7f493c0a9da0 2022-08-01T19:36:02.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.990+0000 7f493bfff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493c0adde0 con 0x7f493c0b31d0 2022-08-01T19:36:02.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.990+0000 7f493b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f493c1402d0 0x7f493c156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:02.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.990+0000 7f493b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f493c1402d0 0x7f493c156020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38290/0 (socket says 172.21.15.17:38290) 2022-08-01T19:36:02.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.990+0000 7f493b7fe700 1 -- 172.21.15.17:0/3671785494 learned_addr learned my addr 172.21.15.17:0/3671785494 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:02.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.990+0000 7f493a7fc700 1 --2- 172.21.15.17:0/3671785494 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f493c0b31d0 0x7f493c13f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:02.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.990+0000 7f493affd700 1 --2- 172.21.15.17:0/3671785494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f493c0a9da0 0x7f493c0ae160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:02.013 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.990+0000 7f493b7fe700 1 -- 172.21.15.17:0/3671785494 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f493c0b31d0 msgr2=0x7f493c13f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:02.013 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.990+0000 7f493b7fe700 1 --2- 172.21.15.17:0/3671785494 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f493c0b31d0 0x7f493c13f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:02.013 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.990+0000 7f493b7fe700 1 -- 172.21.15.17:0/3671785494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f493c0a9da0 msgr2=0x7f493c0ae160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:02.013 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.990+0000 7f493b7fe700 1 --2- 172.21.15.17:0/3671785494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f493c0a9da0 0x7f493c0ae160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:02.014 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.990+0000 7f493b7fe700 1 -- 172.21.15.17:0/3671785494 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492400b180 con 0x7f493c1402d0 2022-08-01T19:36:02.014 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.990+0000 7f493b7fe700 1 --2- 172.21.15.17:0/3671785494 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f493c1402d0 0x7f493c156020 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f4934006e00 tx=0x7f49340091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:02.014 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.990+0000 7f4933fff700 1 -- 172.21.15.17:0/3671785494 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f493400c640 con 0x7f493c1402d0 2022-08-01T19:36:02.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.990+0000 7f4933fff700 1 -- 172.21.15.17:0/3671785494 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493400c7a0 con 0x7f493c1402d0 2022-08-01T19:36:02.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.990+0000 7f4933fff700 1 -- 172.21.15.17:0/3671785494 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f493400cb00 con 0x7f493c1402d0 2022-08-01T19:36:02.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.990+0000 7f493bfff700 1 -- 172.21.15.17:0/3671785494 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f493c152350 con 0x7f493c1402d0 2022-08-01T19:36:02.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.990+0000 7f493bfff700 1 -- 172.21.15.17:0/3671785494 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f493c152040 con 0x7f493c1402d0 2022-08-01T19:36:02.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.990+0000 7f493bfff700 1 -- 172.21.15.17:0/3671785494 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f493c1524e0 con 0x7f493c1402d0 2022-08-01T19:36:02.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.994+0000 7f4933fff700 1 -- 172.21.15.17:0/3671785494 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4934022070 con 0x7f493c1402d0 2022-08-01T19:36:02.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.994+0000 7f4933fff700 1 --2- 172.21.15.17:0/3671785494 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4920053950 0x7f4920055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:02.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.994+0000 7f4933fff700 1 -- 172.21.15.17:0/3671785494 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f493406cd40 con 0x7f493c1402d0 2022-08-01T19:36:02.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.994+0000 7f493affd700 1 --2- 172.21.15.17:0/3671785494 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4920053950 0x7f4920055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:02.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.994+0000 7f493affd700 1 --2- 172.21.15.17:0/3671785494 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4920053950 0x7f4920055e10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f492c00d000 tx=0x7f492c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:02.018 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:01.998+0000 7f4933fff700 1 -- 172.21.15.17:0/3671785494 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4934043a70 con 0x7f493c1402d0 2022-08-01T19:36:02.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.162+0000 7f493bfff700 1 -- 172.21.15.17:0/3671785494 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f493c066af0 con 0x7f493c1402d0 2022-08-01T19:36:02.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.174+0000 7f4933fff700 1 -- 172.21.15.17:0/3671785494 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 7 and data pool 8 v17) v1 ==== 187+0+0 (secure 0 0 0) 0x7f4934029020 con 0x7f493c1402d0 2022-08-01T19:36:02.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.182+0000 7f493bfff700 1 -- 172.21.15.17:0/3671785494 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4920053950 msgr2=0x7f4920055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:02.190 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.182+0000 7f493bfff700 1 --2- 172.21.15.17:0/3671785494 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4920053950 0x7f4920055e10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f492c00d000 tx=0x7f492c009040).stop 2022-08-01T19:36:02.190 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.182+0000 7f493bfff700 1 -- 172.21.15.17:0/3671785494 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f493c1402d0 msgr2=0x7f493c156020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:02.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.182+0000 7f493bfff700 1 --2- 172.21.15.17:0/3671785494 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f493c1402d0 0x7f493c156020 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f4934006e00 tx=0x7f49340091d0).stop 2022-08-01T19:36:02.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.182+0000 7f493bfff700 1 -- 172.21.15.17:0/3671785494 shutdown_connections 2022-08-01T19:36:02.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.182+0000 7f493bfff700 1 --2- 172.21.15.17:0/3671785494 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4920053950 0x7f4920055e10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:02.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.182+0000 7f493bfff700 1 --2- 172.21.15.17:0/3671785494 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f493c1402d0 0x7f493c156020 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:02.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.182+0000 7f493bfff700 1 --2- 172.21.15.17:0/3671785494 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f493c0b31d0 0x7f493c13f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:02.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.182+0000 7f493bfff700 1 --2- 172.21.15.17:0/3671785494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f493c0a9da0 0x7f493c0ae160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:02.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.182+0000 7f493bfff700 1 -- 172.21.15.17:0/3671785494 >> 172.21.15.17:0/3671785494 conn(0x7f493c063870 msgr2=0x7f493c0b0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:02.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.186+0000 7f493bfff700 1 -- 172.21.15.17:0/3671785494 shutdown_connections 2022-08-01T19:36:02.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.186+0000 7f493bfff700 1 -- 172.21.15.17:0/3671785494 wait complete. 2022-08-01T19:36:02.197 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 7 and data pool 8 2022-08-01T19:36:02.204 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:36:02.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.294+0000 7f1da1693700 1 Processor -- start 2022-08-01T19:36:02.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.294+0000 7f1da1693700 1 -- start start 2022-08-01T19:36:02.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.294+0000 7f1da1693700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1d9c148530 0x7f1d9c0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:02.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.294+0000 7f1da1693700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1d9c0a7250 0x7f1d9c1448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:02.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.294+0000 7f1da1693700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d9c0b04e0 con 0x7f1d9c0b1f40 2022-08-01T19:36:02.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.294+0000 7f1da1693700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d9c0a9250 con 0x7f1d9c0a7250 2022-08-01T19:36:02.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.294+0000 7f1da1693700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c0b0780 con 0x7f1d9c148530 2022-08-01T19:36:02.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.294+0000 7f1d9b7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1d9c0a7250 0x7f1d9c1448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:02.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.294+0000 7f1d9b7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1d9c0a7250 0x7f1d9c1448b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38308/0 (socket says 172.21.15.17:38308) 2022-08-01T19:36:02.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.294+0000 7f1d9b7fe700 1 -- 172.21.15.17:0/906409608 learned_addr learned my addr 172.21.15.17:0/906409608 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:02.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.294+0000 7f1d93fff700 1 --2- 172.21.15.17:0/906409608 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1d9c148530 0x7f1d9c0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:02.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.294+0000 7f1d9b7fe700 1 -- 172.21.15.17:0/906409608 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1d9c148530 msgr2=0x7f1d9c0b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:36:02.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.294+0000 7f1d9b7fe700 1 --2- 172.21.15.17:0/906409608 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1d9c148530 0x7f1d9c0b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:02.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.294+0000 7f1d9b7fe700 1 -- 172.21.15.17:0/906409608 >> v1:172.21.15.77:6789/0 conn(0x7f1d9c0b1f40 legacy=0x7f1d9c0b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:02.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.294+0000 7f1d9b7fe700 1 -- 172.21.15.17:0/906409608 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d9c0b0a20 con 0x7f1d9c0a7250 2022-08-01T19:36:02.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.294+0000 7f1d93fff700 1 --2- 172.21.15.17:0/906409608 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1d9c148530 0x7f1d9c0b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:36:02.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.294+0000 7f1d9b7fe700 1 --2- 172.21.15.17:0/906409608 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d9c0a7250 0x7f1d9c1448b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f1d940032f0 tx=0x7f1d94009290).ready entity=mon.0 client_cookie=49853d68eef1cd5d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:02.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.294+0000 7f1d9a7fc700 1 -- 172.21.15.17:0/906409608 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d9400b8a0 con 0x7f1d9c0a7250 2022-08-01T19:36:02.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.294+0000 7f1d9a7fc700 1 -- 172.21.15.17:0/906409608 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9400ba00 con 0x7f1d9c0a7250 2022-08-01T19:36:02.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.294+0000 7f1d9a7fc700 1 -- 172.21.15.17:0/906409608 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d9400bb70 con 0x7f1d9c0a7250 2022-08-01T19:36:02.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.294+0000 7f1da1693700 1 -- 172.21.15.17:0/906409608 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d9c0a7250 msgr2=0x7f1d9c1448b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:02.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.294+0000 7f1da1693700 1 --2- 172.21.15.17:0/906409608 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d9c0a7250 0x7f1d9c1448b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f1d940032f0 tx=0x7f1d94009290).stop 2022-08-01T19:36:02.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.294+0000 7f1da1693700 1 -- 172.21.15.17:0/906409608 shutdown_connections 2022-08-01T19:36:02.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.294+0000 7f1da1693700 1 --2- 172.21.15.17:0/906409608 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d9c0a7250 0x7f1d9c1448b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:02.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.294+0000 7f1da1693700 1 --2- 172.21.15.17:0/906409608 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1d9c148530 0x7f1d9c0b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:02.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.294+0000 7f1da1693700 1 -- 172.21.15.17:0/906409608 >> 172.21.15.17:0/906409608 conn(0x7f1d9c05f840 msgr2=0x7f1d9c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:02.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.294+0000 7f1da1693700 1 -- 172.21.15.17:0/906409608 shutdown_connections 2022-08-01T19:36:02.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.294+0000 7f1da1693700 1 -- 172.21.15.17:0/906409608 wait complete. 2022-08-01T19:36:02.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.298+0000 7f1da1693700 1 Processor -- start 2022-08-01T19:36:02.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.298+0000 7f1da1693700 1 -- start start 2022-08-01T19:36:02.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.298+0000 7f1da1693700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d9c0a7250 0x7f1d9c0ae300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:02.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.298+0000 7f1da1693700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d9c0b1f40 0x7f1d9c149ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:02.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.298+0000 7f1da1693700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1d9c148530 0x7f1d9c143a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:02.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.298+0000 7f1da1693700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c0a95d0 con 0x7f1d9c0a7250 2022-08-01T19:36:02.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.298+0000 7f1da1693700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c0a9800 con 0x7f1d9c0b1f40 2022-08-01T19:36:02.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.298+0000 7f1da1693700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c0a9aa0 con 0x7f1d9c148530 2022-08-01T19:36:02.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.298+0000 7f1d9affd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d9c0a7250 0x7f1d9c0ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:02.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.298+0000 7f1d9affd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d9c0a7250 0x7f1d9c0ae300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38310/0 (socket says 172.21.15.17:38310) 2022-08-01T19:36:02.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.298+0000 7f1d9affd700 1 -- 172.21.15.17:0/676541658 learned_addr learned my addr 172.21.15.17:0/676541658 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:02.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.298+0000 7f1d93fff700 1 --2- 172.21.15.17:0/676541658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d9c0b1f40 0x7f1d9c149ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:02.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.298+0000 7f1d9b7fe700 1 --2- 172.21.15.17:0/676541658 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1d9c148530 0x7f1d9c143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:02.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.298+0000 7f1d9affd700 1 -- 172.21.15.17:0/676541658 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1d9c148530 msgr2=0x7f1d9c143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:02.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.298+0000 7f1d9affd700 1 --2- 172.21.15.17:0/676541658 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1d9c148530 0x7f1d9c143a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:02.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.298+0000 7f1d9affd700 1 -- 172.21.15.17:0/676541658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d9c0b1f40 msgr2=0x7f1d9c149ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:02.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.298+0000 7f1d9affd700 1 --2- 172.21.15.17:0/676541658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d9c0b1f40 0x7f1d9c149ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:02.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.298+0000 7f1d9affd700 1 -- 172.21.15.17:0/676541658 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d94009530 con 0x7f1d9c0a7250 2022-08-01T19:36:02.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.298+0000 7f1d9affd700 1 --2- 172.21.15.17:0/676541658 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d9c0a7250 0x7f1d9c0ae300 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f1d9c061530 tx=0x7f1d84007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:02.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.298+0000 7f1d98ff9700 1 -- 172.21.15.17:0/676541658 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d84012400 con 0x7f1d9c0a7250 2022-08-01T19:36:02.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.298+0000 7f1d98ff9700 1 -- 172.21.15.17:0/676541658 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d84012560 con 0x7f1d9c0a7250 2022-08-01T19:36:02.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.298+0000 7f1da1693700 1 -- 172.21.15.17:0/676541658 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d9c0a9d40 con 0x7f1d9c0a7250 2022-08-01T19:36:02.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.298+0000 7f1d98ff9700 1 -- 172.21.15.17:0/676541658 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d840128c0 con 0x7f1d9c0a7250 2022-08-01T19:36:02.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.298+0000 7f1da1693700 1 -- 172.21.15.17:0/676541658 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d9c14a390 con 0x7f1d9c0a7250 2022-08-01T19:36:02.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.302+0000 7f1da1693700 1 -- 172.21.15.17:0/676541658 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d9c0574e0 con 0x7f1d9c0a7250 2022-08-01T19:36:02.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.302+0000 7f1d98ff9700 1 -- 172.21.15.17:0/676541658 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1d84019030 con 0x7f1d9c0a7250 2022-08-01T19:36:02.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.302+0000 7f1d98ff9700 1 --2- 172.21.15.17:0/676541658 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1d74053aa0 0x7f1d74055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:02.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.302+0000 7f1d98ff9700 1 -- 172.21.15.17:0/676541658 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1d8406b360 con 0x7f1d9c0a7250 2022-08-01T19:36:02.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.302+0000 7f1d93fff700 1 --2- 172.21.15.17:0/676541658 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1d74053aa0 0x7f1d74055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:02.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.302+0000 7f1d93fff700 1 --2- 172.21.15.17:0/676541658 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1d74053aa0 0x7f1d74055f60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f1d8c00c000 tx=0x7f1d8c008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:02.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.310+0000 7f1d98ff9700 1 -- 172.21.15.17:0/676541658 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1d84042120 con 0x7f1d9c0a7250 2022-08-01T19:36:02.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.438+0000 7f1da1693700 1 -- 172.21.15.17:0/676541658 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d9c0447e0 con 0x7f1d9c0a7250 2022-08-01T19:36:02.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.442+0000 7f1d98ff9700 1 -- 172.21.15.17:0/676541658 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16297 (secure 0 0 0) 0x7f1d84045740 con 0x7f1d9c0a7250 2022-08-01T19:36:02.449 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:36:02.450 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":44,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:36:02.170418+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:36:00.565402+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2022-08-01T19:36:01.813338+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:36:02.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.446+0000 7f1da1693700 1 -- 172.21.15.17:0/676541658 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1d74053aa0 msgr2=0x7f1d74055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:02.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.446+0000 7f1da1693700 1 --2- 172.21.15.17:0/676541658 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1d74053aa0 0x7f1d74055f60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f1d8c00c000 tx=0x7f1d8c008040).stop 2022-08-01T19:36:02.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.446+0000 7f1da1693700 1 -- 172.21.15.17:0/676541658 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d9c0a7250 msgr2=0x7f1d9c0ae300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:02.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.446+0000 7f1da1693700 1 --2- 172.21.15.17:0/676541658 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d9c0a7250 0x7f1d9c0ae300 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f1d9c061530 tx=0x7f1d84007c50).stop 2022-08-01T19:36:02.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.446+0000 7f1da1693700 1 -- 172.21.15.17:0/676541658 shutdown_connections 2022-08-01T19:36:02.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.446+0000 7f1da1693700 1 --2- 172.21.15.17:0/676541658 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1d74053aa0 0x7f1d74055f60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:02.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.446+0000 7f1da1693700 1 --2- 172.21.15.17:0/676541658 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1d9c148530 0x7f1d9c143a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:02.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.446+0000 7f1da1693700 1 --2- 172.21.15.17:0/676541658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d9c0b1f40 0x7f1d9c149ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:02.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.446+0000 7f1da1693700 1 --2- 172.21.15.17:0/676541658 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d9c0a7250 0x7f1d9c0ae300 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:02.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.446+0000 7f1da1693700 1 -- 172.21.15.17:0/676541658 >> 172.21.15.17:0/676541658 conn(0x7f1d9c05f840 msgr2=0x7f1d9c0b40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:02.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.446+0000 7f1da1693700 1 -- 172.21.15.17:0/676541658 shutdown_connections 2022-08-01T19:36:02.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.446+0000 7f1da1693700 1 -- 172.21.15.17:0/676541658 wait complete. 2022-08-01T19:36:02.465 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:36:02.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.566+0000 7f8f0f59e700 1 Processor -- start 2022-08-01T19:36:02.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.566+0000 7f8f0f59e700 1 -- start start 2022-08-01T19:36:02.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.566+0000 7f8f0f59e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8f100b2710 0x7f8f1014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:02.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.566+0000 7f8f0f59e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f8f100b01a0 0x7f8f1014b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:02.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.566+0000 7f8f0f59e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f100a7250 con 0x7f8f100afd60 2022-08-01T19:36:02.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.566+0000 7f8f0f59e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f100a7480 con 0x7f8f100b2710 2022-08-01T19:36:02.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.566+0000 7f8f0f59e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f100a7720 con 0x7f8f100b01a0 2022-08-01T19:36:02.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.566+0000 7f8f0e59c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8f100b2710 0x7f8f1014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:02.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.566+0000 7f8f0e59c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8f100b2710 0x7f8f1014afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38318/0 (socket says 172.21.15.17:38318) 2022-08-01T19:36:02.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.566+0000 7f8f0e59c700 1 -- 172.21.15.17:0/229188212 learned_addr learned my addr 172.21.15.17:0/229188212 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:02.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.566+0000 7f8f0dd9b700 1 --2- 172.21.15.17:0/229188212 >> v2:172.21.15.77:3300/0 conn(0x7f8f100b01a0 0x7f8f1014b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:02.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.566+0000 7f8f0e59c700 1 -- 172.21.15.17:0/229188212 >> v2:172.21.15.77:3300/0 conn(0x7f8f100b01a0 msgr2=0x7f8f1014b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:36:02.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.566+0000 7f8f0e59c700 1 --2- 172.21.15.17:0/229188212 >> v2:172.21.15.77:3300/0 conn(0x7f8f100b01a0 0x7f8f1014b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:02.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.566+0000 7f8f0e59c700 1 -- 172.21.15.17:0/229188212 >> v1:172.21.15.77:6789/0 conn(0x7f8f100afd60 legacy=0x7f8f100a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:02.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.566+0000 7f8f0e59c700 1 -- 172.21.15.17:0/229188212 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f100a79c0 con 0x7f8f100b2710 2022-08-01T19:36:02.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.566+0000 7f8f0e59c700 1 --2- 172.21.15.17:0/229188212 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f100b2710 0x7f8f1014afd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f8f00002f70 tx=0x7f8f00006eb0).ready entity=mon.0 client_cookie=9723a4b56d98e38a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:02.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.566+0000 7f8f0d59a700 1 -- 172.21.15.17:0/229188212 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f0000d5c0 con 0x7f8f100b2710 2022-08-01T19:36:02.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.566+0000 7f8f0d59a700 1 -- 172.21.15.17:0/229188212 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f00000e90 con 0x7f8f100b2710 2022-08-01T19:36:02.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.566+0000 7f8f0d59a700 1 -- 172.21.15.17:0/229188212 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f0000d950 con 0x7f8f100b2710 2022-08-01T19:36:02.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.566+0000 7f8f0f59e700 1 -- 172.21.15.17:0/229188212 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f100b2710 msgr2=0x7f8f1014afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:02.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.566+0000 7f8f0f59e700 1 --2- 172.21.15.17:0/229188212 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f100b2710 0x7f8f1014afd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f8f00002f70 tx=0x7f8f00006eb0).stop 2022-08-01T19:36:02.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.566+0000 7f8f0f59e700 1 -- 172.21.15.17:0/229188212 shutdown_connections 2022-08-01T19:36:02.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.566+0000 7f8f0f59e700 1 --2- 172.21.15.17:0/229188212 >> v2:172.21.15.77:3300/0 conn(0x7f8f100b01a0 0x7f8f1014b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:02.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.566+0000 7f8f0f59e700 1 --2- 172.21.15.17:0/229188212 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f100b2710 0x7f8f1014afd0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:02.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.566+0000 7f8f0f59e700 1 -- 172.21.15.17:0/229188212 >> 172.21.15.17:0/229188212 conn(0x7f8f10060310 msgr2=0x7f8f100b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:02.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.566+0000 7f8f0f59e700 1 -- 172.21.15.17:0/229188212 shutdown_connections 2022-08-01T19:36:02.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.566+0000 7f8f0f59e700 1 -- 172.21.15.17:0/229188212 wait complete. 2022-08-01T19:36:02.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.570+0000 7f8f0f59e700 1 Processor -- start 2022-08-01T19:36:02.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.570+0000 7f8f0f59e700 1 -- start start 2022-08-01T19:36:02.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.570+0000 7f8f0f59e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f100afd60 0x7f8f10148b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:02.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.570+0000 7f8f0f59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8f100b01a0 0x7f8f101564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:02.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.570+0000 7f8f0f59e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f100b2710 0x7f8f101569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:02.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.570+0000 7f8f0f59e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f100ac1f0 con 0x7f8f100b2710 2022-08-01T19:36:02.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.570+0000 7f8f0f59e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f100ab9c0 con 0x7f8f100afd60 2022-08-01T19:36:02.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.570+0000 7f8f0f59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f100661a0 con 0x7f8f100b01a0 2022-08-01T19:36:02.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.570+0000 7f8f0ed9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f100b2710 0x7f8f101569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:02.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.570+0000 7f8f0ed9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f100b2710 0x7f8f101569e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38326/0 (socket says 172.21.15.17:38326) 2022-08-01T19:36:02.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.570+0000 7f8f0ed9d700 1 -- 172.21.15.17:0/2181753753 learned_addr learned my addr 172.21.15.17:0/2181753753 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:02.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.570+0000 7f8f0e59c700 1 --2- 172.21.15.17:0/2181753753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f100afd60 0x7f8f10148b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:02.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.570+0000 7f8f0dd9b700 1 --2- 172.21.15.17:0/2181753753 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8f100b01a0 0x7f8f101564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:02.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.570+0000 7f8f0ed9d700 1 -- 172.21.15.17:0/2181753753 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8f100b01a0 msgr2=0x7f8f101564f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:02.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.570+0000 7f8f0ed9d700 1 --2- 172.21.15.17:0/2181753753 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8f100b01a0 0x7f8f101564f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:02.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.570+0000 7f8f0ed9d700 1 -- 172.21.15.17:0/2181753753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f100afd60 msgr2=0x7f8f10148b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:02.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.570+0000 7f8f0ed9d700 1 --2- 172.21.15.17:0/2181753753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f100afd60 0x7f8f10148b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:02.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.570+0000 7f8f0ed9d700 1 -- 172.21.15.17:0/2181753753 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0000b1e0 con 0x7f8f100b2710 2022-08-01T19:36:02.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.570+0000 7f8f0dd9b700 1 --2- 172.21.15.17:0/2181753753 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8f100b01a0 0x7f8f101564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:36:02.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.570+0000 7f8f0ed9d700 1 --2- 172.21.15.17:0/2181753753 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f100b2710 0x7f8f101569e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f8f1006b950 tx=0x7f8f08007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:02.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.570+0000 7f8eff7fe700 1 -- 172.21.15.17:0/2181753753 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f0800f470 con 0x7f8f100b2710 2022-08-01T19:36:02.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.570+0000 7f8eff7fe700 1 -- 172.21.15.17:0/2181753753 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f0800f5d0 con 0x7f8f100b2710 2022-08-01T19:36:02.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.570+0000 7f8f0f59e700 1 -- 172.21.15.17:0/2181753753 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f10066440 con 0x7f8f100b2710 2022-08-01T19:36:02.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.570+0000 7f8eff7fe700 1 -- 172.21.15.17:0/2181753753 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f0800f930 con 0x7f8f100b2710 2022-08-01T19:36:02.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.570+0000 7f8f0f59e700 1 -- 172.21.15.17:0/2181753753 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f10066b80 con 0x7f8f100b2710 2022-08-01T19:36:02.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.574+0000 7f8f0f59e700 1 -- 172.21.15.17:0/2181753753 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f100574e0 con 0x7f8f100b2710 2022-08-01T19:36:02.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.574+0000 7f8eff7fe700 1 -- 172.21.15.17:0/2181753753 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8f08019030 con 0x7f8f100b2710 2022-08-01T19:36:02.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.574+0000 7f8eff7fe700 1 --2- 172.21.15.17:0/2181753753 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8eec0539e0 0x7f8eec055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:02.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.574+0000 7f8eff7fe700 1 -- 172.21.15.17:0/2181753753 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8f0806b6d0 con 0x7f8f100b2710 2022-08-01T19:36:02.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.574+0000 7f8f0e59c700 1 --2- 172.21.15.17:0/2181753753 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8eec0539e0 0x7f8eec055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:02.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.574+0000 7f8f0e59c700 1 --2- 172.21.15.17:0/2181753753 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8eec0539e0 0x7f8eec055ea0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f8f0001c000 tx=0x7f8f00015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:02.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.586+0000 7f8eff7fe700 1 -- 172.21.15.17:0/2181753753 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8f08042490 con 0x7f8f100b2710 2022-08-01T19:36:02.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.710+0000 7f8f0f59e700 1 -- 172.21.15.17:0/2181753753 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f10069a70 con 0x7f8f100b2710 2022-08-01T19:36:02.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.710+0000 7f8eff7fe700 1 -- 172.21.15.17:0/2181753753 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16297 (secure 0 0 0) 0x7f8f08045ab0 con 0x7f8f100b2710 2022-08-01T19:36:02.716 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:36:02.716 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":44,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:36:02.170418+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:36:00.565402+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2022-08-01T19:36:01.813338+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:36:02.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.714+0000 7f8f0f59e700 1 -- 172.21.15.17:0/2181753753 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8eec0539e0 msgr2=0x7f8eec055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:02.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.714+0000 7f8f0f59e700 1 --2- 172.21.15.17:0/2181753753 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8eec0539e0 0x7f8eec055ea0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f8f0001c000 tx=0x7f8f00015000).stop 2022-08-01T19:36:02.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.714+0000 7f8f0f59e700 1 -- 172.21.15.17:0/2181753753 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f100b2710 msgr2=0x7f8f101569e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:02.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.714+0000 7f8f0f59e700 1 --2- 172.21.15.17:0/2181753753 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f100b2710 0x7f8f101569e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f8f1006b950 tx=0x7f8f08007f10).stop 2022-08-01T19:36:02.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.714+0000 7f8f0f59e700 1 -- 172.21.15.17:0/2181753753 shutdown_connections 2022-08-01T19:36:02.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.714+0000 7f8f0f59e700 1 --2- 172.21.15.17:0/2181753753 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8eec0539e0 0x7f8eec055ea0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:02.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.714+0000 7f8f0f59e700 1 --2- 172.21.15.17:0/2181753753 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f100b2710 0x7f8f101569e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:02.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.714+0000 7f8f0f59e700 1 --2- 172.21.15.17:0/2181753753 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8f100b01a0 0x7f8f101564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:02.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.714+0000 7f8f0f59e700 1 --2- 172.21.15.17:0/2181753753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f100afd60 0x7f8f10148b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:02.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.714+0000 7f8f0f59e700 1 -- 172.21.15.17:0/2181753753 >> 172.21.15.17:0/2181753753 conn(0x7f8f10060310 msgr2=0x7f8f10149940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:02.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.714+0000 7f8f0f59e700 1 -- 172.21.15.17:0/2181753753 shutdown_connections 2022-08-01T19:36:02.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.714+0000 7f8f0f59e700 1 -- 172.21.15.17:0/2181753753 wait complete. 2022-08-01T19:36:02.736 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-08-01T19:36:02.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.846+0000 7f0856b1b700 1 Processor -- start 2022-08-01T19:36:02.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.846+0000 7f0856b1b700 1 -- start start 2022-08-01T19:36:02.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.846+0000 7f0856b1b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f08500b31f0 0x7f08500b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:02.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.846+0000 7f0856b1b700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08500b2510 con 0x7f08500b7500 2022-08-01T19:36:02.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.846+0000 7f0856b1b700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08501443e0 con 0x7f08500aadb0 2022-08-01T19:36:02.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.846+0000 7f0856b1b700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08500b3700 con 0x7f08500b31f0 2022-08-01T19:36:02.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.846+0000 7f0855318700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f08500b31f0 0x7f08500b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:02.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.846+0000 7f0855b19700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f08500b7500 0x7f085013e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58756/0 (socket says 172.21.15.17:58756) 2022-08-01T19:36:02.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.846+0000 7f0855b19700 1 -- 172.21.15.17:0/3451789476 learned_addr learned my addr 172.21.15.17:0/3451789476 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:36:02.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.846+0000 7f0855318700 1 -- 172.21.15.17:0/3451789476 >> v1:172.21.15.77:6789/0 conn(0x7f08500aadb0 legacy=0x7f0850064d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:02.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.846+0000 7f0855318700 1 -- 172.21.15.17:0/3451789476 >> v1:172.21.15.17:6789/0 conn(0x7f08500b7500 legacy=0x7f085013e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:02.858 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.846+0000 7f0854b17700 1 -- 172.21.15.17:0/3451789476 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1581306283 0 0) 0x7f08500b2510 con 0x7f08500b7500 2022-08-01T19:36:02.858 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.846+0000 7f0855318700 1 -- 172.21.15.17:0/3451789476 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08500b2740 con 0x7f08500b31f0 2022-08-01T19:36:02.858 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.846+0000 7f0855318700 1 --2- 172.21.15.17:0/3451789476 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08500b31f0 0x7f08500b78e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f0844002f70 tx=0x7f0844006df0).ready entity=mon.0 client_cookie=f09e530f12487849 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:02.858 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.846+0000 7f0854b17700 1 -- 172.21.15.17:0/3451789476 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f084400d610 con 0x7f08500b31f0 2022-08-01T19:36:02.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.846+0000 7f0854b17700 1 -- 172.21.15.17:0/3451789476 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0844000e90 con 0x7f08500b31f0 2022-08-01T19:36:02.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.846+0000 7f0854b17700 1 -- 172.21.15.17:0/3451789476 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f084400d960 con 0x7f08500b31f0 2022-08-01T19:36:02.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.846+0000 7f0856b1b700 1 -- 172.21.15.17:0/3451789476 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08500b31f0 msgr2=0x7f08500b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:02.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.850+0000 7f0856b1b700 1 --2- 172.21.15.17:0/3451789476 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08500b31f0 0x7f08500b78e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f0844002f70 tx=0x7f0844006df0).stop 2022-08-01T19:36:02.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.850+0000 7f0856b1b700 1 -- 172.21.15.17:0/3451789476 shutdown_connections 2022-08-01T19:36:02.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.850+0000 7f0856b1b700 1 --2- 172.21.15.17:0/3451789476 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08500b31f0 0x7f08500b78e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:02.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.850+0000 7f0856b1b700 1 -- 172.21.15.17:0/3451789476 >> 172.21.15.17:0/3451789476 conn(0x7f085005f840 msgr2=0x7f085005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:02.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.850+0000 7f0856b1b700 1 -- 172.21.15.17:0/3451789476 shutdown_connections 2022-08-01T19:36:02.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.850+0000 7f0856b1b700 1 -- 172.21.15.17:0/3451789476 wait complete. 2022-08-01T19:36:02.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.850+0000 7f0856b1b700 1 Processor -- start 2022-08-01T19:36:02.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.850+0000 7f0856b1b700 1 -- start start 2022-08-01T19:36:02.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.850+0000 7f0856b1b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f08500aadb0 0x7f08500ae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:02.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.850+0000 7f0856b1b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08500b31f0 0x7f0850061ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:02.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.850+0000 7f0856b1b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08500b7500 0x7f08500ac700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:02.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.850+0000 7f0856b1b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08500b0a70 con 0x7f08500b7500 2022-08-01T19:36:02.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.850+0000 7f0856b1b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08500b0090 con 0x7f08500b31f0 2022-08-01T19:36:02.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.850+0000 7f0856b1b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08500b0330 con 0x7f08500aadb0 2022-08-01T19:36:02.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.850+0000 7f085631a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08500b7500 0x7f08500ac700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:02.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.850+0000 7f085631a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08500b7500 0x7f08500ac700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38340/0 (socket says 172.21.15.17:38340) 2022-08-01T19:36:02.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.850+0000 7f085631a700 1 -- 172.21.15.17:0/3974447573 learned_addr learned my addr 172.21.15.17:0/3974447573 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:02.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.850+0000 7f0855318700 1 --2- 172.21.15.17:0/3974447573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08500b31f0 0x7f0850061ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:02.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.850+0000 7f0855b19700 1 --2- 172.21.15.17:0/3974447573 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f08500aadb0 0x7f08500ae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:02.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.850+0000 7f085631a700 1 -- 172.21.15.17:0/3974447573 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f08500aadb0 msgr2=0x7f08500ae4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:02.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.850+0000 7f085631a700 1 --2- 172.21.15.17:0/3974447573 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f08500aadb0 0x7f08500ae4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:02.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.850+0000 7f085631a700 1 -- 172.21.15.17:0/3974447573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08500b31f0 msgr2=0x7f0850061ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:02.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.850+0000 7f085631a700 1 --2- 172.21.15.17:0/3974447573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08500b31f0 0x7f0850061ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:02.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.850+0000 7f085631a700 1 -- 172.21.15.17:0/3974447573 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f084400b180 con 0x7f08500b7500 2022-08-01T19:36:02.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.850+0000 7f085631a700 1 --2- 172.21.15.17:0/3974447573 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08500b7500 0x7f08500ac700 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f08500b5130 tx=0x7f0848007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:02.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.850+0000 7f0842ffd700 1 -- 172.21.15.17:0/3974447573 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f084800f470 con 0x7f08500b7500 2022-08-01T19:36:02.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.850+0000 7f0842ffd700 1 -- 172.21.15.17:0/3974447573 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f084800f5d0 con 0x7f08500b7500 2022-08-01T19:36:02.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.850+0000 7f0856b1b700 1 -- 172.21.15.17:0/3974447573 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08500627b0 con 0x7f08500b7500 2022-08-01T19:36:02.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.850+0000 7f0842ffd700 1 -- 172.21.15.17:0/3974447573 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f084800f930 con 0x7f08500b7500 2022-08-01T19:36:02.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.850+0000 7f0856b1b700 1 -- 172.21.15.17:0/3974447573 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08500ad8c0 con 0x7f08500b7500 2022-08-01T19:36:02.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.854+0000 7f0856b1b700 1 -- 172.21.15.17:0/3974447573 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08500574e0 con 0x7f08500b7500 2022-08-01T19:36:02.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.854+0000 7f0842ffd700 1 -- 172.21.15.17:0/3974447573 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0848019030 con 0x7f08500b7500 2022-08-01T19:36:02.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.854+0000 7f0842ffd700 1 --2- 172.21.15.17:0/3974447573 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0830053b30 0x7f0830055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:02.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.854+0000 7f0842ffd700 1 -- 172.21.15.17:0/3974447573 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f084806b740 con 0x7f08500b7500 2022-08-01T19:36:02.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.854+0000 7f0855b19700 1 --2- 172.21.15.17:0/3974447573 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0830053b30 0x7f0830055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:02.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.854+0000 7f0855b19700 1 --2- 172.21.15.17:0/3974447573 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0830053b30 0x7f0830055ff0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f084c00d000 tx=0x7f084c007ba0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:02.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:02.862+0000 7f0842ffd700 1 -- 172.21.15.17:0/3974447573 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f08480423f0 con 0x7f08500b7500 2022-08-01T19:36:03.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.022+0000 7f0856b1b700 1 -- 172.21.15.17:0/3974447573 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f0850060330 con 0x7f08500b7500 2022-08-01T19:36:03.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.174+0000 7f0842ffd700 1 -- 172.21.15.17:0/3974447573 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0848045a10 con 0x7f08500b7500 2022-08-01T19:36:03.184 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.178+0000 7f0856b1b700 1 -- 172.21.15.17:0/3974447573 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0830053b30 msgr2=0x7f0830055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:03.184 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.178+0000 7f0856b1b700 1 --2- 172.21.15.17:0/3974447573 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0830053b30 0x7f0830055ff0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f084c00d000 tx=0x7f084c007ba0).stop 2022-08-01T19:36:03.184 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.178+0000 7f0856b1b700 1 -- 172.21.15.17:0/3974447573 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08500b7500 msgr2=0x7f08500ac700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:03.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.178+0000 7f0856b1b700 1 --2- 172.21.15.17:0/3974447573 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08500b7500 0x7f08500ac700 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f08500b5130 tx=0x7f0848007f40).stop 2022-08-01T19:36:03.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.178+0000 7f0856b1b700 1 -- 172.21.15.17:0/3974447573 shutdown_connections 2022-08-01T19:36:03.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.178+0000 7f0856b1b700 1 --2- 172.21.15.17:0/3974447573 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0830053b30 0x7f0830055ff0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:03.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.178+0000 7f0856b1b700 1 --2- 172.21.15.17:0/3974447573 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08500b7500 0x7f08500ac700 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:03.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.178+0000 7f0856b1b700 1 --2- 172.21.15.17:0/3974447573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08500b31f0 0x7f0850061ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:03.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.178+0000 7f0856b1b700 1 --2- 172.21.15.17:0/3974447573 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f08500aadb0 0x7f08500ae4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:03.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.178+0000 7f0856b1b700 1 -- 172.21.15.17:0/3974447573 >> 172.21.15.17:0/3974447573 conn(0x7f085005f840 msgr2=0x7f085005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:03.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.178+0000 7f0856b1b700 1 -- 172.21.15.17:0/3974447573 shutdown_connections 2022-08-01T19:36:03.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.178+0000 7f0856b1b700 1 -- 172.21.15.17:0/3974447573 wait complete. 2022-08-01T19:36:03.196 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:36:03.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.294+0000 7f848b59e700 1 Processor -- start 2022-08-01T19:36:03.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.294+0000 7f848b59e700 1 -- start start 2022-08-01T19:36:03.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.294+0000 7f848b59e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f848c0af450 0x7f848c0af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:03.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.294+0000 7f848b59e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f848c0ad9e0 0x7f848c0a8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:03.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.294+0000 7f848b59e700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f848c0aff30 con 0x7f848c0ad4b0 2022-08-01T19:36:03.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.294+0000 7f848b59e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f848c05c7a0 con 0x7f848c0af450 2022-08-01T19:36:03.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.294+0000 7f848b59e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f848c140990 con 0x7f848c0ad9e0 2022-08-01T19:36:03.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.294+0000 7f848a59c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f848c0af450 0x7f848c0af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:03.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.294+0000 7f848a59c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f848c0af450 0x7f848c0af830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38344/0 (socket says 172.21.15.17:38344) 2022-08-01T19:36:03.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.294+0000 7f848a59c700 1 -- 172.21.15.17:0/3793579567 learned_addr learned my addr 172.21.15.17:0/3793579567 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:03.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.294+0000 7f848ad9d700 1 --2- 172.21.15.17:0/3793579567 >> v2:172.21.15.77:3300/0 conn(0x7f848c0ad9e0 0x7f848c0a8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:03.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.298+0000 7f848959a700 1 -- 172.21.15.17:0/3793579567 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 408316225 0 0) 0x7f848c0aff30 con 0x7f848c0ad4b0 2022-08-01T19:36:03.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.298+0000 7f848a59c700 1 -- 172.21.15.17:0/3793579567 >> v2:172.21.15.77:3300/0 conn(0x7f848c0ad9e0 msgr2=0x7f848c0a8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:36:03.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.298+0000 7f848a59c700 1 --2- 172.21.15.17:0/3793579567 >> v2:172.21.15.77:3300/0 conn(0x7f848c0ad9e0 0x7f848c0a8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:03.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.298+0000 7f848a59c700 1 -- 172.21.15.17:0/3793579567 >> v1:172.21.15.17:6789/0 conn(0x7f848c0ad4b0 legacy=0x7f848c1432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:03.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.298+0000 7f848a59c700 1 -- 172.21.15.17:0/3793579567 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848c1439f0 con 0x7f848c0af450 2022-08-01T19:36:03.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.298+0000 7f848a59c700 1 --2- 172.21.15.17:0/3793579567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f848c0af450 0x7f848c0af830 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f848c0ad900 tx=0x7f847c006eb0).ready entity=mon.0 client_cookie=e3fc8f4aa4584707 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:03.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.298+0000 7f848959a700 1 -- 172.21.15.17:0/3793579567 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f847c00d610 con 0x7f848c0af450 2022-08-01T19:36:03.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.298+0000 7f848959a700 1 -- 172.21.15.17:0/3793579567 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847c000e90 con 0x7f848c0af450 2022-08-01T19:36:03.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.298+0000 7f848959a700 1 -- 172.21.15.17:0/3793579567 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f847c00d960 con 0x7f848c0af450 2022-08-01T19:36:03.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.298+0000 7f848b59e700 1 -- 172.21.15.17:0/3793579567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f848c0af450 msgr2=0x7f848c0af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:03.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.298+0000 7f848b59e700 1 --2- 172.21.15.17:0/3793579567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f848c0af450 0x7f848c0af830 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f848c0ad900 tx=0x7f847c006eb0).stop 2022-08-01T19:36:03.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.298+0000 7f848b59e700 1 -- 172.21.15.17:0/3793579567 shutdown_connections 2022-08-01T19:36:03.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.298+0000 7f848b59e700 1 --2- 172.21.15.17:0/3793579567 >> v2:172.21.15.77:3300/0 conn(0x7f848c0ad9e0 0x7f848c0a8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:03.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.298+0000 7f848b59e700 1 --2- 172.21.15.17:0/3793579567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f848c0af450 0x7f848c0af830 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:03.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.298+0000 7f848b59e700 1 -- 172.21.15.17:0/3793579567 >> 172.21.15.17:0/3793579567 conn(0x7f848c0612d0 msgr2=0x7f848c062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:03.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.298+0000 7f848b59e700 1 -- 172.21.15.17:0/3793579567 shutdown_connections 2022-08-01T19:36:03.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.298+0000 7f848b59e700 1 -- 172.21.15.17:0/3793579567 wait complete. 2022-08-01T19:36:03.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.302+0000 7f848b59e700 1 Processor -- start 2022-08-01T19:36:03.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.302+0000 7f848b59e700 1 -- start start 2022-08-01T19:36:03.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.302+0000 7f848b59e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f848c0ad4b0 0x7f848c0ac1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:03.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.302+0000 7f848b59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f848c0ad9e0 0x7f848c153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:03.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.302+0000 7f848b59e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f848c0af450 0x7f848c154200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:03.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.302+0000 7f848b59e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848c0b6970 con 0x7f848c0ad4b0 2022-08-01T19:36:03.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.302+0000 7f848b59e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848c1546f0 con 0x7f848c0af450 2022-08-01T19:36:03.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.302+0000 7f848b59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f848c0b5400 con 0x7f848c0ad9e0 2022-08-01T19:36:03.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.302+0000 7f848a59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f848c0ad4b0 0x7f848c0ac1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:03.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.302+0000 7f848a59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f848c0ad4b0 0x7f848c0ac1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38350/0 (socket says 172.21.15.17:38350) 2022-08-01T19:36:03.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.302+0000 7f848a59c700 1 -- 172.21.15.17:0/1993435575 learned_addr learned my addr 172.21.15.17:0/1993435575 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:03.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.302+0000 7f848ad9d700 1 --2- 172.21.15.17:0/1993435575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f848c0af450 0x7f848c154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:03.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.302+0000 7f8489d9b700 1 --2- 172.21.15.17:0/1993435575 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f848c0ad9e0 0x7f848c153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:03.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.302+0000 7f848a59c700 1 -- 172.21.15.17:0/1993435575 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f848c0ad9e0 msgr2=0x7f848c153d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:03.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.302+0000 7f848a59c700 1 --2- 172.21.15.17:0/1993435575 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f848c0ad9e0 0x7f848c153d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:03.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.302+0000 7f848a59c700 1 -- 172.21.15.17:0/1993435575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f848c0af450 msgr2=0x7f848c154200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:03.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.302+0000 7f848a59c700 1 --2- 172.21.15.17:0/1993435575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f848c0af450 0x7f848c154200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:03.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.302+0000 7f848a59c700 1 -- 172.21.15.17:0/1993435575 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f847c00b1e0 con 0x7f848c0ad4b0 2022-08-01T19:36:03.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.302+0000 7f848ad9d700 1 --2- 172.21.15.17:0/1993435575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f848c0af450 0x7f848c154200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:36:03.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.302+0000 7f8489d9b700 1 --2- 172.21.15.17:0/1993435575 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f848c0ad9e0 0x7f848c153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:36:03.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.302+0000 7f848a59c700 1 --2- 172.21.15.17:0/1993435575 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f848c0ad4b0 0x7f848c0ac1e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f848c0ad900 tx=0x7f847c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:03.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.302+0000 7f847b7fe700 1 -- 172.21.15.17:0/1993435575 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f847c00de90 con 0x7f848c0ad4b0 2022-08-01T19:36:03.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.302+0000 7f847b7fe700 1 -- 172.21.15.17:0/1993435575 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847c006b40 con 0x7f848c0ad4b0 2022-08-01T19:36:03.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.302+0000 7f848b59e700 1 -- 172.21.15.17:0/1993435575 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f848c143fb0 con 0x7f848c0ad4b0 2022-08-01T19:36:03.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.302+0000 7f847b7fe700 1 -- 172.21.15.17:0/1993435575 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f847c002f50 con 0x7f848c0ad4b0 2022-08-01T19:36:03.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.302+0000 7f848b59e700 1 -- 172.21.15.17:0/1993435575 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f848c0b5be0 con 0x7f848c0ad4b0 2022-08-01T19:36:03.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.306+0000 7f847b7fe700 1 -- 172.21.15.17:0/1993435575 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f847c025030 con 0x7f848c0ad4b0 2022-08-01T19:36:03.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.306+0000 7f848b59e700 1 -- 172.21.15.17:0/1993435575 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f848c0574e0 con 0x7f848c0ad4b0 2022-08-01T19:36:03.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.306+0000 7f847b7fe700 1 --2- 172.21.15.17:0/1993435575 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8468053a10 0x7f8468055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:03.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.306+0000 7f847b7fe700 1 -- 172.21.15.17:0/1993435575 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f847c06d220 con 0x7f848c0ad4b0 2022-08-01T19:36:03.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.306+0000 7f8489d9b700 1 --2- 172.21.15.17:0/1993435575 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8468053a10 0x7f8468055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:03.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.306+0000 7f8489d9b700 1 --2- 172.21.15.17:0/1993435575 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8468053a10 0x7f8468055ed0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f848000b000 tx=0x7f8480009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:03.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.310+0000 7f847b7fe700 1 -- 172.21.15.17:0/1993435575 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f847c043fe0 con 0x7f848c0ad4b0 2022-08-01T19:36:03.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.474+0000 7f848b59e700 1 -- 172.21.15.17:0/1993435575 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f848c061b20 con 0x7f848c0ad4b0 2022-08-01T19:36:03.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.474+0000 7f847b7fe700 1 -- 172.21.15.17:0/1993435575 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+2488 (secure 0 0 0) 0x7f847c023090 con 0x7f848c0ad4b0 2022-08-01T19:36:03.481 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:36:03.481 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/109603607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":109603607},{"type":"v1","addr":"172.21.15.17:6833","nonce":109603607}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":16},{"gid":5230,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/2722431349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2722431349},{"type":"v1","addr":"172.21.15.77:6835","nonce":2722431349}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":17}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:36:02.170790+0000","modified":"2022-08-01T19:36:03.173986+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5238},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5238":{"gid":5238,"name":"c","rank":0,"incarnation":17,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/3980246529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":3980246529},{"type":"v1","addr":"172.21.15.17:6835","nonce":3980246529}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-08-01T19:36:03.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.478+0000 7f848b59e700 1 -- 172.21.15.17:0/1993435575 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8468053a10 msgr2=0x7f8468055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:03.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.478+0000 7f848b59e700 1 --2- 172.21.15.17:0/1993435575 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8468053a10 0x7f8468055ed0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f848000b000 tx=0x7f8480009040).stop 2022-08-01T19:36:03.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.478+0000 7f848b59e700 1 -- 172.21.15.17:0/1993435575 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f848c0ad4b0 msgr2=0x7f848c0ac1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:03.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.478+0000 7f848b59e700 1 --2- 172.21.15.17:0/1993435575 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f848c0ad4b0 0x7f848c0ac1e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f848c0ad900 tx=0x7f847c015000).stop 2022-08-01T19:36:03.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.478+0000 7f848b59e700 1 -- 172.21.15.17:0/1993435575 shutdown_connections 2022-08-01T19:36:03.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.478+0000 7f848b59e700 1 --2- 172.21.15.17:0/1993435575 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8468053a10 0x7f8468055ed0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:03.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.478+0000 7f848b59e700 1 --2- 172.21.15.17:0/1993435575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f848c0af450 0x7f848c154200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:03.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.478+0000 7f848b59e700 1 --2- 172.21.15.17:0/1993435575 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f848c0ad9e0 0x7f848c153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:03.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.478+0000 7f848b59e700 1 --2- 172.21.15.17:0/1993435575 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f848c0ad4b0 0x7f848c0ac1e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:03.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.478+0000 7f848b59e700 1 -- 172.21.15.17:0/1993435575 >> 172.21.15.17:0/1993435575 conn(0x7f848c0612d0 msgr2=0x7f848c0a8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:03.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.478+0000 7f848b59e700 1 -- 172.21.15.17:0/1993435575 shutdown_connections 2022-08-01T19:36:03.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.478+0000 7f848b59e700 1 -- 172.21.15.17:0/1993435575 wait complete. 2022-08-01T19:36:03.487 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 18 2022-08-01T19:36:03.495 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:36:03.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.594+0000 7f9e42616700 1 Processor -- start 2022-08-01T19:36:03.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.594+0000 7f9e42616700 1 -- start start 2022-08-01T19:36:03.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.594+0000 7f9e42616700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9e3c0b2710 0x7f9e3c0abca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:03.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.594+0000 7f9e42616700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f9e3c0b13f0 0x7f9e3c0ac190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:03.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.594+0000 7f9e42616700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e3c0adf10 con 0x7f9e3c0b1920 2022-08-01T19:36:03.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.594+0000 7f9e42616700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e3c0ae1b0 con 0x7f9e3c0b13f0 2022-08-01T19:36:03.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.594+0000 7f9e42616700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c0a9520 con 0x7f9e3c0b2710 2022-08-01T19:36:03.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.594+0000 7f9e40e13700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f9e3c0b13f0 0x7f9e3c0ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:03.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.594+0000 7f9e40e13700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f9e3c0b13f0 0x7f9e3c0ac190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:36280/0 (socket says 172.21.15.17:36280) 2022-08-01T19:36:03.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.594+0000 7f9e40e13700 1 -- 172.21.15.17:0/1822526900 learned_addr learned my addr 172.21.15.17:0/1822526900 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:03.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.594+0000 7f9e41614700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9e3c0b2710 0x7f9e3c0abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:03.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.594+0000 7f9e41e15700 1 --1- 172.21.15.17:0/1822526900 >> v1:172.21.15.77:6789/0 conn(0x7f9e3c0b1920 0x7f9e3c0afd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:47354/0 (socket says 172.21.15.17:47354) 2022-08-01T19:36:03.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.594+0000 7f9e40e13700 1 -- 172.21.15.17:0/1822526900 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9e3c0b2710 msgr2=0x7f9e3c0abca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:36:03.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.594+0000 7f9e40e13700 1 --2- 172.21.15.17:0/1822526900 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9e3c0b2710 0x7f9e3c0abca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:03.602 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.594+0000 7f9e40e13700 1 -- 172.21.15.17:0/1822526900 >> v1:172.21.15.77:6789/0 conn(0x7f9e3c0b1920 legacy=0x7f9e3c0afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:03.602 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.594+0000 7f9e40e13700 1 -- 172.21.15.17:0/1822526900 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c0a99c0 con 0x7f9e3c0b13f0 2022-08-01T19:36:03.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.594+0000 7f9e40e13700 1 --2- 172.21.15.17:0/1822526900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e3c0b13f0 0x7f9e3c0ac190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f9e3c067bc0 tx=0x7f9e2c006eb0).ready entity=mon.1 client_cookie=aa43c99e67c71f28 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:03.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.594+0000 7f9e2bfff700 1 -- 172.21.15.17:0/1822526900 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e2c00d5c0 con 0x7f9e3c0b13f0 2022-08-01T19:36:03.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.594+0000 7f9e2bfff700 1 -- 172.21.15.17:0/1822526900 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2c000e90 con 0x7f9e3c0b13f0 2022-08-01T19:36:03.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.594+0000 7f9e2bfff700 1 -- 172.21.15.17:0/1822526900 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e2c00d950 con 0x7f9e3c0b13f0 2022-08-01T19:36:03.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.594+0000 7f9e42616700 1 -- 172.21.15.17:0/1822526900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e3c0b13f0 msgr2=0x7f9e3c0ac190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:03.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.594+0000 7f9e42616700 1 --2- 172.21.15.17:0/1822526900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e3c0b13f0 0x7f9e3c0ac190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f9e3c067bc0 tx=0x7f9e2c006eb0).stop 2022-08-01T19:36:03.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.594+0000 7f9e42616700 1 -- 172.21.15.17:0/1822526900 shutdown_connections 2022-08-01T19:36:03.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.594+0000 7f9e42616700 1 --2- 172.21.15.17:0/1822526900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e3c0b13f0 0x7f9e3c0ac190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:03.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.594+0000 7f9e42616700 1 --2- 172.21.15.17:0/1822526900 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9e3c0b2710 0x7f9e3c0abca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:03.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.594+0000 7f9e42616700 1 -- 172.21.15.17:0/1822526900 >> 172.21.15.17:0/1822526900 conn(0x7f9e3c060310 msgr2=0x7f9e3c0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:03.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.594+0000 7f9e42616700 1 -- 172.21.15.17:0/1822526900 shutdown_connections 2022-08-01T19:36:03.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.594+0000 7f9e42616700 1 -- 172.21.15.17:0/1822526900 wait complete. 2022-08-01T19:36:03.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.594+0000 7f9e42616700 1 Processor -- start 2022-08-01T19:36:03.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.594+0000 7f9e42616700 1 -- start start 2022-08-01T19:36:03.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.598+0000 7f9e42616700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e3c0b13f0 0x7f9e3c13f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:03.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.598+0000 7f9e42616700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e3c0b1920 0x7f9e3c0aacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:03.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.598+0000 7f9e42616700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9e3c0b2710 0x7f9e3c13a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:03.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.598+0000 7f9e42616700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c13ef40 con 0x7f9e3c0b13f0 2022-08-01T19:36:03.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.598+0000 7f9e42616700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c13f6f0 con 0x7f9e3c0b1920 2022-08-01T19:36:03.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.598+0000 7f9e42616700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c13d500 con 0x7f9e3c0b2710 2022-08-01T19:36:03.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.598+0000 7f9e41614700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e3c0b13f0 0x7f9e3c13f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:03.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.598+0000 7f9e41614700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e3c0b13f0 0x7f9e3c13f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38364/0 (socket says 172.21.15.17:38364) 2022-08-01T19:36:03.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.598+0000 7f9e41614700 1 -- 172.21.15.17:0/1636221867 learned_addr learned my addr 172.21.15.17:0/1636221867 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:03.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.598+0000 7f9e40e13700 1 --2- 172.21.15.17:0/1636221867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e3c0b1920 0x7f9e3c0aacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:03.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.598+0000 7f9e41e15700 1 --2- 172.21.15.17:0/1636221867 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9e3c0b2710 0x7f9e3c13a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:03.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.598+0000 7f9e41614700 1 -- 172.21.15.17:0/1636221867 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9e3c0b2710 msgr2=0x7f9e3c13a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:03.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.598+0000 7f9e41614700 1 --2- 172.21.15.17:0/1636221867 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9e3c0b2710 0x7f9e3c13a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:03.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.598+0000 7f9e41614700 1 -- 172.21.15.17:0/1636221867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e3c0b1920 msgr2=0x7f9e3c0aacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:03.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.598+0000 7f9e41614700 1 --2- 172.21.15.17:0/1636221867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e3c0b1920 0x7f9e3c0aacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:03.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.598+0000 7f9e41614700 1 -- 172.21.15.17:0/1636221867 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e2c00b1e0 con 0x7f9e3c0b13f0 2022-08-01T19:36:03.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.598+0000 7f9e41614700 1 --2- 172.21.15.17:0/1636221867 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e3c0b13f0 0x7f9e3c13f200 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f9e3c13fd70 tx=0x7f9e38006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:03.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.598+0000 7f9e2a7fc700 1 -- 172.21.15.17:0/1636221867 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e38010780 con 0x7f9e3c0b13f0 2022-08-01T19:36:03.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.598+0000 7f9e2a7fc700 1 -- 172.21.15.17:0/1636221867 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e380108e0 con 0x7f9e3c0b13f0 2022-08-01T19:36:03.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.598+0000 7f9e42616700 1 -- 172.21.15.17:0/1636221867 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e3c13bdf0 con 0x7f9e3c0b13f0 2022-08-01T19:36:03.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.598+0000 7f9e2a7fc700 1 -- 172.21.15.17:0/1636221867 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e38010c40 con 0x7f9e3c0b13f0 2022-08-01T19:36:03.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.598+0000 7f9e42616700 1 -- 172.21.15.17:0/1636221867 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e3c13bb20 con 0x7f9e3c0b13f0 2022-08-01T19:36:03.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.602+0000 7f9e42616700 1 -- 172.21.15.17:0/1636221867 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e3c140a30 con 0x7f9e3c0b13f0 2022-08-01T19:36:03.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.602+0000 7f9e2a7fc700 1 -- 172.21.15.17:0/1636221867 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9e3801c460 con 0x7f9e3c0b13f0 2022-08-01T19:36:03.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.602+0000 7f9e2a7fc700 1 --2- 172.21.15.17:0/1636221867 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9e1c053a70 0x7f9e1c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:03.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.602+0000 7f9e2a7fc700 1 -- 172.21.15.17:0/1636221867 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9e3806c4e0 con 0x7f9e3c0b13f0 2022-08-01T19:36:03.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.602+0000 7f9e40e13700 1 --2- 172.21.15.17:0/1636221867 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9e1c053a70 0x7f9e1c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:03.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.602+0000 7f9e40e13700 1 --2- 172.21.15.17:0/1636221867 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9e1c053a70 0x7f9e1c055f30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f9e2c01c000 tx=0x7f9e2c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:03.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.606+0000 7f9e2a7fc700 1 -- 172.21.15.17:0/1636221867 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9e380431a0 con 0x7f9e3c0b13f0 2022-08-01T19:36:03.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.734+0000 7f9e42616700 1 -- 172.21.15.17:0/1636221867 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e3c13a160 con 0x7f9e3c0b13f0 2022-08-01T19:36:03.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.734+0000 7f9e2a7fc700 1 -- 172.21.15.17:0/1636221867 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16271 (secure 0 0 0) 0x7f9e380467c0 con 0x7f9e3c0b13f0 2022-08-01T19:36:03.740 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:36:03.740 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":45,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:36:03.174642+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:36:00.565402+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2022-08-01T19:36:01.813338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:36:03.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.738+0000 7f9e42616700 1 -- 172.21.15.17:0/1636221867 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9e1c053a70 msgr2=0x7f9e1c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:03.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.738+0000 7f9e42616700 1 --2- 172.21.15.17:0/1636221867 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9e1c053a70 0x7f9e1c055f30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f9e2c01c000 tx=0x7f9e2c015000).stop 2022-08-01T19:36:03.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.738+0000 7f9e42616700 1 -- 172.21.15.17:0/1636221867 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e3c0b13f0 msgr2=0x7f9e3c13f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:03.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.738+0000 7f9e42616700 1 --2- 172.21.15.17:0/1636221867 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e3c0b13f0 0x7f9e3c13f200 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f9e3c13fd70 tx=0x7f9e38006df0).stop 2022-08-01T19:36:03.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.738+0000 7f9e42616700 1 -- 172.21.15.17:0/1636221867 shutdown_connections 2022-08-01T19:36:03.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.738+0000 7f9e42616700 1 --2- 172.21.15.17:0/1636221867 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9e1c053a70 0x7f9e1c055f30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:03.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.738+0000 7f9e42616700 1 --2- 172.21.15.17:0/1636221867 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9e3c0b2710 0x7f9e3c13a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:03.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.738+0000 7f9e42616700 1 --2- 172.21.15.17:0/1636221867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e3c0b1920 0x7f9e3c0aacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:03.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.738+0000 7f9e42616700 1 --2- 172.21.15.17:0/1636221867 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e3c0b13f0 0x7f9e3c13f200 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:03.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.738+0000 7f9e42616700 1 -- 172.21.15.17:0/1636221867 >> 172.21.15.17:0/1636221867 conn(0x7f9e3c060310 msgr2=0x7f9e3c0b5360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:03.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.738+0000 7f9e42616700 1 -- 172.21.15.17:0/1636221867 shutdown_connections 2022-08-01T19:36:03.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.738+0000 7f9e42616700 1 -- 172.21.15.17:0/1636221867 wait complete. 2022-08-01T19:36:03.755 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T19:36:03.870 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.866+0000 7fbaaf272700 1 Processor -- start 2022-08-01T19:36:03.870 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.866+0000 7fbaaf272700 1 -- start start 2022-08-01T19:36:03.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.866+0000 7fbaaf272700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fbaa80b6710 0x7fbaa80b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.866+0000 7fbaaf272700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbaa80b6af0 0x7fbaa812bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.866+0000 7fbaaf272700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaa8144240 con 0x7fbaa80b4550 2022-08-01T19:36:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.866+0000 7fbaaf272700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaa8143d00 con 0x7fbaa80b6af0 2022-08-01T19:36:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.866+0000 7fbaaf272700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaa8143fa0 con 0x7fbaa80b6710 2022-08-01T19:36:03.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.866+0000 7fbaad80f700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbaa80b6af0 0x7fbaa812bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:03.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.866+0000 7fbaad80f700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbaa80b6af0 0x7fbaa812bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38376/0 (socket says 172.21.15.17:38376) 2022-08-01T19:36:03.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.866+0000 7fbaad80f700 1 -- 172.21.15.17:0/2803212835 learned_addr learned my addr 172.21.15.17:0/2803212835 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:03.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.866+0000 7fbaad00e700 1 --2- 172.21.15.17:0/2803212835 >> v2:172.21.15.77:3300/0 conn(0x7fbaa80b6710 0x7fbaa80b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:03.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.866+0000 7fbaad80f700 1 -- 172.21.15.17:0/2803212835 >> v2:172.21.15.77:3300/0 conn(0x7fbaa80b6710 msgr2=0x7fbaa80b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:36:03.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.866+0000 7fbaad80f700 1 --2- 172.21.15.17:0/2803212835 >> v2:172.21.15.77:3300/0 conn(0x7fbaa80b6710 0x7fbaa80b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:03.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.866+0000 7fbaad80f700 1 -- 172.21.15.17:0/2803212835 >> v1:172.21.15.77:6789/0 conn(0x7fbaa80b4550 legacy=0x7fbaa8061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:03.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.866+0000 7fbaad80f700 1 -- 172.21.15.17:0/2803212835 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa8061780 con 0x7fbaa80b6af0 2022-08-01T19:36:03.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.866+0000 7fbaad80f700 1 --2- 172.21.15.17:0/2803212835 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbaa80b6af0 0x7fbaa812bb90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7fbaa81432b0 tx=0x7fbaa0009290).ready entity=mon.0 client_cookie=8130a7163d92de6b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:03.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.870+0000 7fba9ffff700 1 -- 172.21.15.17:0/2803212835 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbaa000b860 con 0x7fbaa80b6af0 2022-08-01T19:36:03.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.870+0000 7fba9ffff700 1 -- 172.21.15.17:0/2803212835 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa000b9c0 con 0x7fbaa80b6af0 2022-08-01T19:36:03.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.870+0000 7fba9ffff700 1 -- 172.21.15.17:0/2803212835 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbaa000bb30 con 0x7fbaa80b6af0 2022-08-01T19:36:03.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.870+0000 7fbaaf272700 1 -- 172.21.15.17:0/2803212835 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbaa80b6af0 msgr2=0x7fbaa812bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:03.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.870+0000 7fbaaf272700 1 --2- 172.21.15.17:0/2803212835 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbaa80b6af0 0x7fbaa812bb90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7fbaa81432b0 tx=0x7fbaa0009290).stop 2022-08-01T19:36:03.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.870+0000 7fbaaf272700 1 -- 172.21.15.17:0/2803212835 shutdown_connections 2022-08-01T19:36:03.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.870+0000 7fbaaf272700 1 --2- 172.21.15.17:0/2803212835 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbaa80b6af0 0x7fbaa812bb90 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:03.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.870+0000 7fbaaf272700 1 --2- 172.21.15.17:0/2803212835 >> v2:172.21.15.77:3300/0 conn(0x7fbaa80b6710 0x7fbaa80b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:03.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.870+0000 7fbaaf272700 1 -- 172.21.15.17:0/2803212835 >> 172.21.15.17:0/2803212835 conn(0x7fbaa805c620 msgr2=0x7fbaa805ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:03.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.870+0000 7fbaaf272700 1 -- 172.21.15.17:0/2803212835 shutdown_connections 2022-08-01T19:36:03.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.870+0000 7fbaaf272700 1 -- 172.21.15.17:0/2803212835 wait complete. 2022-08-01T19:36:03.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.870+0000 7fbaaf272700 1 Processor -- start 2022-08-01T19:36:03.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.870+0000 7fbaaf272700 1 -- start start 2022-08-01T19:36:03.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.874+0000 7fbaaf272700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbaa80b4550 0x7fbaa80b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:03.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.874+0000 7fbaaf272700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbaa80b6710 0x7fbaa8146ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:03.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.874+0000 7fbaaf272700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbaa80b6af0 0x7fbaa81473e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:03.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.874+0000 7fbaaf272700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa8148040 con 0x7fbaa80b4550 2022-08-01T19:36:03.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.874+0000 7fbaaf272700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa80ba6d0 con 0x7fbaa80b6710 2022-08-01T19:36:03.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.874+0000 7fbaaf272700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa8148500 con 0x7fbaa80b6af0 2022-08-01T19:36:03.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.874+0000 7fbaad00e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbaa80b4550 0x7fbaa80b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:03.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.874+0000 7fbaad00e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbaa80b4550 0x7fbaa80b78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38378/0 (socket says 172.21.15.17:38378) 2022-08-01T19:36:03.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.874+0000 7fbaad00e700 1 -- 172.21.15.17:0/2323985901 learned_addr learned my addr 172.21.15.17:0/2323985901 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:03.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.874+0000 7fbaad80f700 1 --2- 172.21.15.17:0/2323985901 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbaa80b6af0 0x7fbaa81473e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:03.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.874+0000 7fbaac80d700 1 --2- 172.21.15.17:0/2323985901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbaa80b6710 0x7fbaa8146ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:03.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.874+0000 7fbaad00e700 1 -- 172.21.15.17:0/2323985901 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbaa80b6af0 msgr2=0x7fbaa81473e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:03.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.874+0000 7fbaad00e700 1 --2- 172.21.15.17:0/2323985901 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbaa80b6af0 0x7fbaa81473e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:03.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.874+0000 7fbaad00e700 1 -- 172.21.15.17:0/2323985901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbaa80b6710 msgr2=0x7fbaa8146ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:03.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.874+0000 7fbaad00e700 1 --2- 172.21.15.17:0/2323985901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbaa80b6710 0x7fbaa8146ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:03.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.874+0000 7fbaad00e700 1 -- 172.21.15.17:0/2323985901 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa0009530 con 0x7fbaa80b4550 2022-08-01T19:36:03.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.874+0000 7fbaac80d700 1 --2- 172.21.15.17:0/2323985901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbaa80b6710 0x7fbaa8146ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:36:03.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.874+0000 7fbaad00e700 1 --2- 172.21.15.17:0/2323985901 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbaa80b4550 0x7fbaa80b78e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7fbaa400c000 tx=0x7fbaa4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:03.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.874+0000 7fba9e7fc700 1 -- 172.21.15.17:0/2323985901 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbaa4010780 con 0x7fbaa80b4550 2022-08-01T19:36:03.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.874+0000 7fba9e7fc700 1 -- 172.21.15.17:0/2323985901 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa40108e0 con 0x7fbaa80b4550 2022-08-01T19:36:03.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.874+0000 7fba9e7fc700 1 -- 172.21.15.17:0/2323985901 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbaa4010c40 con 0x7fbaa80b4550 2022-08-01T19:36:03.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.874+0000 7fbaaf272700 1 -- 172.21.15.17:0/2323985901 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaa8147da0 con 0x7fbaa80b4550 2022-08-01T19:36:03.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.874+0000 7fbaaf272700 1 -- 172.21.15.17:0/2323985901 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaa8146bf0 con 0x7fbaa80b4550 2022-08-01T19:36:03.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.874+0000 7fbaaf272700 1 -- 172.21.15.17:0/2323985901 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaa8061a20 con 0x7fbaa80b4550 2022-08-01T19:36:03.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.878+0000 7fba9e7fc700 1 -- 172.21.15.17:0/2323985901 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbaa401c460 con 0x7fbaa80b4550 2022-08-01T19:36:03.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.878+0000 7fba9e7fc700 1 --2- 172.21.15.17:0/2323985901 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fba90053b30 0x7fba90055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:03.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.878+0000 7fba9e7fc700 1 -- 172.21.15.17:0/2323985901 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fbaa406c400 con 0x7fbaa80b4550 2022-08-01T19:36:03.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.878+0000 7fbaac80d700 1 --2- 172.21.15.17:0/2323985901 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fba90053b30 0x7fba90055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:03.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.878+0000 7fbaac80d700 1 --2- 172.21.15.17:0/2323985901 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fba90053b30 0x7fba90055ff0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7fba9800d000 tx=0x7fba98009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:03.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:03.886+0000 7fba9e7fc700 1 -- 172.21.15.17:0/2323985901 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbaa40430c0 con 0x7fbaa80b4550 2022-08-01T19:36:04.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.054+0000 7fbaaf272700 1 -- 172.21.15.17:0/2323985901 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fbaa80574e0 con 0x7fbaa80b4550 2022-08-01T19:36:04.066 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.062+0000 7fba9e7fc700 1 -- 172.21.15.17:0/2323985901 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v17) v1 ==== 184+0+0 (secure 0 0 0) 0x7fbaa40466e0 con 0x7fbaa80b4550 2022-08-01T19:36:04.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.062+0000 7fbaaf272700 1 -- 172.21.15.17:0/2323985901 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fba90053b30 msgr2=0x7fba90055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:04.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.062+0000 7fbaaf272700 1 --2- 172.21.15.17:0/2323985901 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fba90053b30 0x7fba90055ff0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7fba9800d000 tx=0x7fba98009040).stop 2022-08-01T19:36:04.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.062+0000 7fbaaf272700 1 -- 172.21.15.17:0/2323985901 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbaa80b4550 msgr2=0x7fbaa80b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:04.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.062+0000 7fbaaf272700 1 --2- 172.21.15.17:0/2323985901 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbaa80b4550 0x7fbaa80b78e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7fbaa400c000 tx=0x7fbaa4006df0).stop 2022-08-01T19:36:04.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.062+0000 7fbaaf272700 1 -- 172.21.15.17:0/2323985901 shutdown_connections 2022-08-01T19:36:04.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.062+0000 7fbaaf272700 1 --2- 172.21.15.17:0/2323985901 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fba90053b30 0x7fba90055ff0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:04.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.062+0000 7fbaaf272700 1 --2- 172.21.15.17:0/2323985901 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbaa80b6af0 0x7fbaa81473e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:04.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.062+0000 7fbaaf272700 1 --2- 172.21.15.17:0/2323985901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbaa80b6710 0x7fbaa8146ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:04.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.062+0000 7fbaaf272700 1 --2- 172.21.15.17:0/2323985901 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbaa80b4550 0x7fbaa80b78e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:04.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.062+0000 7fbaaf272700 1 -- 172.21.15.17:0/2323985901 >> 172.21.15.17:0/2323985901 conn(0x7fbaa805c620 msgr2=0x7fbaa81420b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:04.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.062+0000 7fbaaf272700 1 -- 172.21.15.17:0/2323985901 shutdown_connections 2022-08-01T19:36:04.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.066+0000 7fbaaf272700 1 -- 172.21.15.17:0/2323985901 wait complete. 2022-08-01T19:36:04.075 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.0 2022-08-01T19:36:04.081 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T19:36:04.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.190+0000 7fc1be63e700 1 Processor -- start 2022-08-01T19:36:04.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.190+0000 7fc1be63e700 1 -- start start 2022-08-01T19:36:04.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.190+0000 7fc1be63e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc1b80b6710 0x7fc1b80b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:04.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.190+0000 7fc1be63e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fc1b80b6af0 0x7fc1b812bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:04.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.190+0000 7fc1be63e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1b8144240 con 0x7fc1b80b4550 2022-08-01T19:36:04.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.190+0000 7fc1be63e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1b8143d00 con 0x7fc1b80b6710 2022-08-01T19:36:04.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.190+0000 7fc1be63e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1b8143fa0 con 0x7fc1b80b6af0 2022-08-01T19:36:04.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.190+0000 7fc1b7fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc1b80b6710 0x7fc1b80b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:04.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.190+0000 7fc1b7fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc1b80b6710 0x7fc1b80b5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38386/0 (socket says 172.21.15.17:38386) 2022-08-01T19:36:04.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.190+0000 7fc1b7fff700 1 -- 172.21.15.17:0/2141826139 learned_addr learned my addr 172.21.15.17:0/2141826139 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:04.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.190+0000 7fc1bcbdb700 1 --2- 172.21.15.17:0/2141826139 >> v2:172.21.15.77:3300/0 conn(0x7fc1b80b6af0 0x7fc1b812bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:04.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.190+0000 7fc1b7fff700 1 -- 172.21.15.17:0/2141826139 >> v2:172.21.15.77:3300/0 conn(0x7fc1b80b6af0 msgr2=0x7fc1b812bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:36:04.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.190+0000 7fc1b7fff700 1 --2- 172.21.15.17:0/2141826139 >> v2:172.21.15.77:3300/0 conn(0x7fc1b80b6af0 0x7fc1b812bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:04.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.190+0000 7fc1b7fff700 1 -- 172.21.15.17:0/2141826139 >> v1:172.21.15.77:6789/0 conn(0x7fc1b80b4550 legacy=0x7fc1b8061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:04.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.190+0000 7fc1b7fff700 1 -- 172.21.15.17:0/2141826139 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b8061780 con 0x7fc1b80b6710 2022-08-01T19:36:04.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.190+0000 7fc1bcbdb700 1 --2- 172.21.15.17:0/2141826139 >> v2:172.21.15.77:3300/0 conn(0x7fc1b80b6af0 0x7fc1b812bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:36:04.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.190+0000 7fc1b7fff700 1 --2- 172.21.15.17:0/2141826139 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1b80b6710 0x7fc1b80b5d80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7fc1b81432b0 tx=0x7fc1a8006eb0).ready entity=mon.0 client_cookie=d391aca433777e8f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:04.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.190+0000 7fc1b77fe700 1 -- 172.21.15.17:0/2141826139 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1a800d610 con 0x7fc1b80b6710 2022-08-01T19:36:04.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.190+0000 7fc1b77fe700 1 -- 172.21.15.17:0/2141826139 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a8000e90 con 0x7fc1b80b6710 2022-08-01T19:36:04.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.190+0000 7fc1b77fe700 1 -- 172.21.15.17:0/2141826139 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1a800d960 con 0x7fc1b80b6710 2022-08-01T19:36:04.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.194+0000 7fc1be63e700 1 -- 172.21.15.17:0/2141826139 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1b80b6710 msgr2=0x7fc1b80b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:04.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.194+0000 7fc1be63e700 1 --2- 172.21.15.17:0/2141826139 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1b80b6710 0x7fc1b80b5d80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7fc1b81432b0 tx=0x7fc1a8006eb0).stop 2022-08-01T19:36:04.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.194+0000 7fc1be63e700 1 -- 172.21.15.17:0/2141826139 shutdown_connections 2022-08-01T19:36:04.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.194+0000 7fc1be63e700 1 --2- 172.21.15.17:0/2141826139 >> v2:172.21.15.77:3300/0 conn(0x7fc1b80b6af0 0x7fc1b812bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:04.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.194+0000 7fc1be63e700 1 --2- 172.21.15.17:0/2141826139 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1b80b6710 0x7fc1b80b5d80 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:04.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.194+0000 7fc1be63e700 1 -- 172.21.15.17:0/2141826139 >> 172.21.15.17:0/2141826139 conn(0x7fc1b805c620 msgr2=0x7fc1b805ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:04.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.194+0000 7fc1be63e700 1 -- 172.21.15.17:0/2141826139 shutdown_connections 2022-08-01T19:36:04.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.194+0000 7fc1be63e700 1 -- 172.21.15.17:0/2141826139 wait complete. 2022-08-01T19:36:04.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.194+0000 7fc1be63e700 1 Processor -- start 2022-08-01T19:36:04.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.194+0000 7fc1be63e700 1 -- start start 2022-08-01T19:36:04.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.194+0000 7fc1be63e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1b80b4550 0x7fc1b8156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:04.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.194+0000 7fc1be63e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1b80b6710 0x7fc1b8147d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:04.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.194+0000 7fc1be63e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc1b80b6af0 0x7fc1b8148270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:04.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.194+0000 7fc1be63e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b8129a90 con 0x7fc1b80b4550 2022-08-01T19:36:04.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.194+0000 7fc1be63e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b8156880 con 0x7fc1b80b6710 2022-08-01T19:36:04.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.194+0000 7fc1be63e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b8129cf0 con 0x7fc1b80b6af0 2022-08-01T19:36:04.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.198+0000 7fc1b7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1b80b4550 0x7fc1b8156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:04.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.198+0000 7fc1affff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1b80b6710 0x7fc1b8147d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:04.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.198+0000 7fc1b7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1b80b4550 0x7fc1b8156390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38392/0 (socket says 172.21.15.17:38392) 2022-08-01T19:36:04.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.198+0000 7fc1b7fff700 1 -- 172.21.15.17:0/668312135 learned_addr learned my addr 172.21.15.17:0/668312135 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:04.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.198+0000 7fc1bcbdb700 1 --2- 172.21.15.17:0/668312135 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc1b80b6af0 0x7fc1b8148270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:04.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.198+0000 7fc1b7fff700 1 -- 172.21.15.17:0/668312135 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc1b80b6af0 msgr2=0x7fc1b8148270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:04.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.198+0000 7fc1b7fff700 1 --2- 172.21.15.17:0/668312135 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc1b80b6af0 0x7fc1b8148270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:04.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.198+0000 7fc1b7fff700 1 -- 172.21.15.17:0/668312135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1b80b6710 msgr2=0x7fc1b8147d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:04.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.198+0000 7fc1b7fff700 1 --2- 172.21.15.17:0/668312135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1b80b6710 0x7fc1b8147d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:04.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.198+0000 7fc1b7fff700 1 -- 172.21.15.17:0/668312135 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a800b1e0 con 0x7fc1b80b4550 2022-08-01T19:36:04.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.198+0000 7fc1b7fff700 1 --2- 172.21.15.17:0/668312135 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1b80b4550 0x7fc1b8156390 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7fc1b81432b0 tx=0x7fc1a8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:04.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.198+0000 7fc1b5ffb700 1 -- 172.21.15.17:0/668312135 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1a80068b0 con 0x7fc1b80b4550 2022-08-01T19:36:04.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.198+0000 7fc1b5ffb700 1 -- 172.21.15.17:0/668312135 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a8002f50 con 0x7fc1b80b4550 2022-08-01T19:36:04.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.198+0000 7fc1b5ffb700 1 -- 172.21.15.17:0/668312135 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1a8011b80 con 0x7fc1b80b4550 2022-08-01T19:36:04.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.198+0000 7fc1be63e700 1 -- 172.21.15.17:0/668312135 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b8129f90 con 0x7fc1b80b4550 2022-08-01T19:36:04.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.198+0000 7fc1be63e700 1 -- 172.21.15.17:0/668312135 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b812b8f0 con 0x7fc1b80b4550 2022-08-01T19:36:04.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.198+0000 7fc1b5ffb700 1 -- 172.21.15.17:0/668312135 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc1a8025030 con 0x7fc1b80b4550 2022-08-01T19:36:04.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.202+0000 7fc1be63e700 1 -- 172.21.15.17:0/668312135 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b8061a20 con 0x7fc1b80b4550 2022-08-01T19:36:04.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.202+0000 7fc1b5ffb700 1 --2- 172.21.15.17:0/668312135 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc19c053aa0 0x7fc19c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:04.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.202+0000 7fc1b5ffb700 1 -- 172.21.15.17:0/668312135 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc1a806d120 con 0x7fc1b80b4550 2022-08-01T19:36:04.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.202+0000 7fc1affff700 1 --2- 172.21.15.17:0/668312135 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc19c053aa0 0x7fc19c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:04.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.202+0000 7fc1affff700 1 --2- 172.21.15.17:0/668312135 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc19c053aa0 0x7fc19c055f60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7fc1a000d000 tx=0x7fc1a0007b90).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:04.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.206+0000 7fc1b5ffb700 1 -- 172.21.15.17:0/668312135 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc1a8043ee0 con 0x7fc1b80b4550 2022-08-01T19:36:04.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.370+0000 7fc1be63e700 1 -- 172.21.15.17:0/668312135 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fc1b80574e0 con 0x7fc1b80b4550 2022-08-01T19:36:04.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.378+0000 7fc1b5ffb700 1 -- 172.21.15.17:0/668312135 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v18) v1 ==== 184+0+0 (secure 0 0 0) 0x7fc1a80230e0 con 0x7fc1b80b4550 2022-08-01T19:36:04.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.382+0000 7fc1be63e700 1 -- 172.21.15.17:0/668312135 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc19c053aa0 msgr2=0x7fc19c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:04.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.382+0000 7fc1be63e700 1 --2- 172.21.15.17:0/668312135 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc19c053aa0 0x7fc19c055f60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7fc1a000d000 tx=0x7fc1a0007b90).stop 2022-08-01T19:36:04.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.382+0000 7fc1be63e700 1 -- 172.21.15.17:0/668312135 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1b80b4550 msgr2=0x7fc1b8156390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:04.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.382+0000 7fc1be63e700 1 --2- 172.21.15.17:0/668312135 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1b80b4550 0x7fc1b8156390 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7fc1b81432b0 tx=0x7fc1a8015000).stop 2022-08-01T19:36:04.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.382+0000 7fc1be63e700 1 -- 172.21.15.17:0/668312135 shutdown_connections 2022-08-01T19:36:04.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.382+0000 7fc1be63e700 1 --2- 172.21.15.17:0/668312135 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc19c053aa0 0x7fc19c055f60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:04.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.382+0000 7fc1be63e700 1 --2- 172.21.15.17:0/668312135 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc1b80b6af0 0x7fc1b8148270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:04.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.382+0000 7fc1be63e700 1 --2- 172.21.15.17:0/668312135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1b80b6710 0x7fc1b8147d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:04.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.382+0000 7fc1be63e700 1 --2- 172.21.15.17:0/668312135 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1b80b4550 0x7fc1b8156390 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:04.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.382+0000 7fc1be63e700 1 -- 172.21.15.17:0/668312135 >> 172.21.15.17:0/668312135 conn(0x7fc1b805c620 msgr2=0x7fc1b81432f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:04.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.382+0000 7fc1be63e700 1 -- 172.21.15.17:0/668312135 shutdown_connections 2022-08-01T19:36:04.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.382+0000 7fc1be63e700 1 -- 172.21.15.17:0/668312135 wait complete. 2022-08-01T19:36:04.392 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.1 2022-08-01T19:36:04.402 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:36:04.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.498+0000 7f8d65211700 1 Processor -- start 2022-08-01T19:36:04.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.498+0000 7f8d65211700 1 -- start start 2022-08-01T19:36:04.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.502+0000 7f8d65211700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8d60060f00 0x7f8d600b6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:04.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.502+0000 7f8d65211700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f8d60145250 0x7f8d600ac780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:04.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.502+0000 7f8d65211700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d600ace20 con 0x7f8d600b6750 2022-08-01T19:36:04.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.502+0000 7f8d65211700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d600ae6f0 con 0x7f8d60060f00 2022-08-01T19:36:04.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.502+0000 7f8d65211700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d60144870 con 0x7f8d60145250 2022-08-01T19:36:04.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.502+0000 7f8d5e59c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8d60060f00 0x7f8d600b6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:04.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.502+0000 7f8d5e59c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8d60060f00 0x7f8d600b6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38402/0 (socket says 172.21.15.17:38402) 2022-08-01T19:36:04.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.502+0000 7f8d5f59e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f8d60145250 0x7f8d600ac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:04.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.502+0000 7f8d5e59c700 1 -- 172.21.15.17:0/3179687323 learned_addr learned my addr 172.21.15.17:0/3179687323 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:04.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.502+0000 7f8d5ed9d700 1 --1- 172.21.15.17:0/3179687323 >> v1:172.21.15.77:6789/0 conn(0x7f8d600b6750 0x7f8d60141190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:47394/0 (socket says 172.21.15.17:47394) 2022-08-01T19:36:04.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.502+0000 7f8d5e59c700 1 -- 172.21.15.17:0/3179687323 >> v2:172.21.15.77:3300/0 conn(0x7f8d60145250 msgr2=0x7f8d600ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:36:04.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.502+0000 7f8d5e59c700 1 --2- 172.21.15.17:0/3179687323 >> v2:172.21.15.77:3300/0 conn(0x7f8d60145250 0x7f8d600ac780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:04.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.502+0000 7f8d5e59c700 1 -- 172.21.15.17:0/3179687323 >> v1:172.21.15.77:6789/0 conn(0x7f8d600b6750 legacy=0x7f8d60141190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:04.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.502+0000 7f8d5e59c700 1 -- 172.21.15.17:0/3179687323 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d60144b10 con 0x7f8d60060f00 2022-08-01T19:36:04.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.502+0000 7f8d5e59c700 1 --2- 172.21.15.17:0/3179687323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d60060f00 0x7f8d600b6b30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f8d54002f70 tx=0x7f8d54006eb0).ready entity=mon.0 client_cookie=8b334ce97d321253 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:04.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.502+0000 7f8d5dd9b700 1 -- 172.21.15.17:0/3179687323 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d5400d5c0 con 0x7f8d60060f00 2022-08-01T19:36:04.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.502+0000 7f8d5dd9b700 1 -- 172.21.15.17:0/3179687323 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d54000e90 con 0x7f8d60060f00 2022-08-01T19:36:04.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.502+0000 7f8d5dd9b700 1 -- 172.21.15.17:0/3179687323 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d5400d950 con 0x7f8d60060f00 2022-08-01T19:36:04.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.506+0000 7f8d65211700 1 -- 172.21.15.17:0/3179687323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d60060f00 msgr2=0x7f8d600b6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:04.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.506+0000 7f8d65211700 1 --2- 172.21.15.17:0/3179687323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d60060f00 0x7f8d600b6b30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f8d54002f70 tx=0x7f8d54006eb0).stop 2022-08-01T19:36:04.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.506+0000 7f8d65211700 1 -- 172.21.15.17:0/3179687323 shutdown_connections 2022-08-01T19:36:04.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.506+0000 7f8d65211700 1 --2- 172.21.15.17:0/3179687323 >> v2:172.21.15.77:3300/0 conn(0x7f8d60145250 0x7f8d600ac780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:04.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.506+0000 7f8d65211700 1 --2- 172.21.15.17:0/3179687323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d60060f00 0x7f8d600b6b30 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:04.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.506+0000 7f8d65211700 1 -- 172.21.15.17:0/3179687323 >> 172.21.15.17:0/3179687323 conn(0x7f8d600b3610 msgr2=0x7f8d600b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:04.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.506+0000 7f8d65211700 1 -- 172.21.15.17:0/3179687323 shutdown_connections 2022-08-01T19:36:04.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.506+0000 7f8d65211700 1 -- 172.21.15.17:0/3179687323 wait complete. 2022-08-01T19:36:04.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.506+0000 7f8d65211700 1 Processor -- start 2022-08-01T19:36:04.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.506+0000 7f8d65211700 1 -- start start 2022-08-01T19:36:04.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.506+0000 7f8d65211700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d60060f00 0x7f8d600b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:04.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.506+0000 7f8d65211700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8d600b6750 0x7f8d6013e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:04.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.506+0000 7f8d65211700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d60145250 0x7f8d6013d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:04.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.506+0000 7f8d65211700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6013e130 con 0x7f8d60145250 2022-08-01T19:36:04.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.506+0000 7f8d65211700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d601408c0 con 0x7f8d60060f00 2022-08-01T19:36:04.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.506+0000 7f8d65211700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d60140b60 con 0x7f8d600b6750 2022-08-01T19:36:04.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.510+0000 7f8d5ed9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d60060f00 0x7f8d600b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:04.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.510+0000 7f8d5f59e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d60145250 0x7f8d6013d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:04.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.510+0000 7f8d5ed9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d60060f00 0x7f8d600b1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:36328/0 (socket says 172.21.15.17:36328) 2022-08-01T19:36:04.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.510+0000 7f8d5e59c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8d600b6750 0x7f8d6013e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:04.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.510+0000 7f8d5ed9d700 1 -- 172.21.15.17:0/4001232646 learned_addr learned my addr 172.21.15.17:0/4001232646 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:04.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.510+0000 7f8d5f59e700 1 -- 172.21.15.17:0/4001232646 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8d600b6750 msgr2=0x7f8d6013e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:04.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.510+0000 7f8d5f59e700 1 --2- 172.21.15.17:0/4001232646 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8d600b6750 0x7f8d6013e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:04.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.510+0000 7f8d5f59e700 1 -- 172.21.15.17:0/4001232646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d60060f00 msgr2=0x7f8d600b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:04.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.510+0000 7f8d5f59e700 1 --2- 172.21.15.17:0/4001232646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d60060f00 0x7f8d600b1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:04.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.510+0000 7f8d5f59e700 1 -- 172.21.15.17:0/4001232646 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d5400b1e0 con 0x7f8d60145250 2022-08-01T19:36:04.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.510+0000 7f8d5ed9d700 1 --2- 172.21.15.17:0/4001232646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d60060f00 0x7f8d600b1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:36:04.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.510+0000 7f8d5f59e700 1 --2- 172.21.15.17:0/4001232646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d60145250 0x7f8d6013d4c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f8d5800c000 tx=0x7f8d580091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:04.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.510+0000 7f8d4ffff700 1 -- 172.21.15.17:0/4001232646 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d58012600 con 0x7f8d60145250 2022-08-01T19:36:04.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.510+0000 7f8d4ffff700 1 -- 172.21.15.17:0/4001232646 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d58012760 con 0x7f8d60145250 2022-08-01T19:36:04.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.510+0000 7f8d65211700 1 -- 172.21.15.17:0/4001232646 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d60140e00 con 0x7f8d60145250 2022-08-01T19:36:04.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.510+0000 7f8d4ffff700 1 -- 172.21.15.17:0/4001232646 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d58012ac0 con 0x7f8d60145250 2022-08-01T19:36:04.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.510+0000 7f8d65211700 1 -- 172.21.15.17:0/4001232646 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d6013d9b0 con 0x7f8d60145250 2022-08-01T19:36:04.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.510+0000 7f8d65211700 1 -- 172.21.15.17:0/4001232646 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d60144db0 con 0x7f8d60145250 2022-08-01T19:36:04.525 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.510+0000 7f8d4ffff700 1 -- 172.21.15.17:0/4001232646 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8d58022070 con 0x7f8d60145250 2022-08-01T19:36:04.525 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.514+0000 7f8d4ffff700 1 --2- 172.21.15.17:0/4001232646 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8d3c053a70 0x7f8d3c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:04.525 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.514+0000 7f8d4ffff700 1 -- 172.21.15.17:0/4001232646 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8d5806c650 con 0x7f8d60145250 2022-08-01T19:36:04.525 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.514+0000 7f8d5ed9d700 1 --2- 172.21.15.17:0/4001232646 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8d3c053a70 0x7f8d3c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:04.526 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.514+0000 7f8d5ed9d700 1 --2- 172.21.15.17:0/4001232646 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8d3c053a70 0x7f8d3c055f30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f8d5000d000 tx=0x7f8d50007c40).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:04.526 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.518+0000 7f8d4ffff700 1 -- 172.21.15.17:0/4001232646 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8d580433b0 con 0x7f8d60145250 2022-08-01T19:36:04.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.678+0000 7f8d65211700 1 -- 172.21.15.17:0/4001232646 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d600574e0 con 0x7f8d60145250 2022-08-01T19:36:04.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.678+0000 7f8d4ffff700 1 -- 172.21.15.17:0/4001232646 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+2488 (secure 0 0 0) 0x7f8d58016020 con 0x7f8d60145250 2022-08-01T19:36:04.685 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:36:04.686 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/109603607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":109603607},{"type":"v1","addr":"172.21.15.17:6833","nonce":109603607}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":16},{"gid":5230,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/2722431349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2722431349},{"type":"v1","addr":"172.21.15.77:6835","nonce":2722431349}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":17}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:36:02.170790+0000","modified":"2022-08-01T19:36:03.173986+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5238},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5238":{"gid":5238,"name":"c","rank":0,"incarnation":17,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/3980246529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":3980246529},{"type":"v1","addr":"172.21.15.17:6835","nonce":3980246529}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-08-01T19:36:04.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.678+0000 7f8d65211700 1 -- 172.21.15.17:0/4001232646 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8d3c053a70 msgr2=0x7f8d3c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:04.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.678+0000 7f8d65211700 1 --2- 172.21.15.17:0/4001232646 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8d3c053a70 0x7f8d3c055f30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f8d5000d000 tx=0x7f8d50007c40).stop 2022-08-01T19:36:04.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.678+0000 7f8d65211700 1 -- 172.21.15.17:0/4001232646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d60145250 msgr2=0x7f8d6013d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:04.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.678+0000 7f8d65211700 1 --2- 172.21.15.17:0/4001232646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d60145250 0x7f8d6013d4c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f8d5800c000 tx=0x7f8d580091d0).stop 2022-08-01T19:36:04.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.682+0000 7f8d65211700 1 -- 172.21.15.17:0/4001232646 shutdown_connections 2022-08-01T19:36:04.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.682+0000 7f8d65211700 1 --2- 172.21.15.17:0/4001232646 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8d3c053a70 0x7f8d3c055f30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:04.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.682+0000 7f8d65211700 1 --2- 172.21.15.17:0/4001232646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d60145250 0x7f8d6013d4c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:04.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.682+0000 7f8d65211700 1 --2- 172.21.15.17:0/4001232646 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8d600b6750 0x7f8d6013e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:04.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.682+0000 7f8d65211700 1 --2- 172.21.15.17:0/4001232646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d60060f00 0x7f8d600b1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:04.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.682+0000 7f8d65211700 1 -- 172.21.15.17:0/4001232646 >> 172.21.15.17:0/4001232646 conn(0x7f8d600b3610 msgr2=0x7f8d600b5210 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:04.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.682+0000 7f8d65211700 1 -- 172.21.15.17:0/4001232646 shutdown_connections 2022-08-01T19:36:04.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:04.682+0000 7f8d65211700 1 -- 172.21.15.17:0/4001232646 wait complete. 2022-08-01T19:36:04.690 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 18 2022-08-01T19:36:05.005 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:36:02.170790+0000', 'modified': '2022-08-01T19:36:03.173986+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5238}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5238': {'gid': 5238, 'name': 'c', 'rank': 0, 'incarnation': 17, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/3980246529', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 3980246529}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 3980246529}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:36:05.006 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:36:05.007 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T19:36:05.124 INFO:teuthology.orchestra.run.smithi017.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":5238,"want_state":"up:active","state":"up:active","rank_uptime":2.942074082,"mdsmap_epoch":18,"osdmap_epoch":45,"osdmap_epoch_barrier":44,"uptime":7.314262909} 2022-08-01T19:36:05.126 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 5238, "mdsmap_epoch": 18, "osdmap_epoch": 45, "osdmap_epoch_barrier": 44, "rank_uptime": 2.942074082, "state": "up:active", "uptime": 7.314262909, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:36:05.127 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:36:05.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.230+0000 7f8e4cb16700 1 Processor -- start 2022-08-01T19:36:05.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.230+0000 7f8e4cb16700 1 -- start start 2022-08-01T19:36:05.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.230+0000 7f8e4cb16700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f8e480b2710 0x7f8e481401e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:05.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.230+0000 7f8e4cb16700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8e480b0270 0x7f8e481406d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:05.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.230+0000 7f8e4cb16700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8e480ac880 0x7f8e48143000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:05.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.230+0000 7f8e4cb16700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e480ad740 con 0x7f8e480b0270 2022-08-01T19:36:05.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.230+0000 7f8e4cb16700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e48067a10 con 0x7f8e480b2710 2022-08-01T19:36:05.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.230+0000 7f8e4cb16700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e48067cb0 con 0x7f8e480ac880 2022-08-01T19:36:05.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.230+0000 7f8e3effd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8e480b0270 0x7f8e481406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:05.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.230+0000 7f8e3effd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8e480b0270 0x7f8e481406d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38414/0 (socket says 172.21.15.17:38414) 2022-08-01T19:36:05.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.230+0000 7f8e3effd700 1 -- 172.21.15.17:0/1823307949 learned_addr learned my addr 172.21.15.17:0/1823307949 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:05.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.230+0000 7f8e47fff700 1 --2- 172.21.15.17:0/1823307949 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8e480ac880 0x7f8e48143000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:05.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.230+0000 7f8e477fe700 1 --2- 172.21.15.17:0/1823307949 >> v2:172.21.15.77:3300/0 conn(0x7f8e480b2710 0x7f8e481401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:05.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.230+0000 7f8e3effd700 1 -- 172.21.15.17:0/1823307949 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8e480ac880 msgr2=0x7f8e48143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:36:05.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.230+0000 7f8e3effd700 1 --2- 172.21.15.17:0/1823307949 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8e480ac880 0x7f8e48143000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:05.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.230+0000 7f8e3effd700 1 -- 172.21.15.17:0/1823307949 >> v2:172.21.15.77:3300/0 conn(0x7f8e480b2710 msgr2=0x7f8e481401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:36:05.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.230+0000 7f8e3effd700 1 --2- 172.21.15.17:0/1823307949 >> v2:172.21.15.77:3300/0 conn(0x7f8e480b2710 0x7f8e481401e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:05.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.230+0000 7f8e3effd700 1 -- 172.21.15.17:0/1823307949 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e48147160 con 0x7f8e480b0270 2022-08-01T19:36:05.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.230+0000 7f8e3effd700 1 --2- 172.21.15.17:0/1823307949 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e480b0270 0x7f8e481406d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f8e30002f70 tx=0x7f8e30006eb0).ready entity=mon.0 client_cookie=80383c965744946c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:05.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.230+0000 7f8e46ffd700 1 -- 172.21.15.17:0/1823307949 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e3000d570 con 0x7f8e480b0270 2022-08-01T19:36:05.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.234+0000 7f8e46ffd700 1 -- 172.21.15.17:0/1823307949 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e30000e90 con 0x7f8e480b0270 2022-08-01T19:36:05.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.234+0000 7f8e46ffd700 1 -- 172.21.15.17:0/1823307949 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e3000d900 con 0x7f8e480b0270 2022-08-01T19:36:05.246 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.234+0000 7f8e4cb16700 1 -- 172.21.15.17:0/1823307949 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e480b0270 msgr2=0x7f8e481406d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:05.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.234+0000 7f8e4cb16700 1 --2- 172.21.15.17:0/1823307949 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e480b0270 0x7f8e481406d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f8e30002f70 tx=0x7f8e30006eb0).stop 2022-08-01T19:36:05.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.234+0000 7f8e4cb16700 1 -- 172.21.15.17:0/1823307949 shutdown_connections 2022-08-01T19:36:05.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.234+0000 7f8e4cb16700 1 --2- 172.21.15.17:0/1823307949 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8e480ac880 0x7f8e48143000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:05.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.234+0000 7f8e4cb16700 1 --2- 172.21.15.17:0/1823307949 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e480b0270 0x7f8e481406d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:05.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.234+0000 7f8e4cb16700 1 --2- 172.21.15.17:0/1823307949 >> v2:172.21.15.77:3300/0 conn(0x7f8e480b2710 0x7f8e481401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:05.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.234+0000 7f8e4cb16700 1 -- 172.21.15.17:0/1823307949 >> 172.21.15.17:0/1823307949 conn(0x7f8e48060310 msgr2=0x7f8e480b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:05.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.234+0000 7f8e4cb16700 1 -- 172.21.15.17:0/1823307949 shutdown_connections 2022-08-01T19:36:05.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.234+0000 7f8e4cb16700 1 -- 172.21.15.17:0/1823307949 wait complete. 2022-08-01T19:36:05.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.234+0000 7f8e4cb16700 1 Processor -- start 2022-08-01T19:36:05.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.234+0000 7f8e4cb16700 1 -- start start 2022-08-01T19:36:05.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.238+0000 7f8e4cb16700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8e480ac880 0x7f8e480ab720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:05.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.238+0000 7f8e4cb16700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e480b0270 0x7f8e4809ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:05.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.238+0000 7f8e4cb16700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e480b2710 0x7f8e481559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:05.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.238+0000 7f8e4cb16700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e48142e60 con 0x7f8e480b2710 2022-08-01T19:36:05.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.238+0000 7f8e4cb16700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e48143100 con 0x7f8e480b0270 2022-08-01T19:36:05.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.238+0000 7f8e4cb16700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4814ffa0 con 0x7f8e480ac880 2022-08-01T19:36:05.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.238+0000 7f8e47fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e480b2710 0x7f8e481559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:05.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.238+0000 7f8e477fe700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8e480ac880 0x7f8e480ab720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:05.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.238+0000 7f8e47fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e480b2710 0x7f8e481559a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38424/0 (socket says 172.21.15.17:38424) 2022-08-01T19:36:05.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.238+0000 7f8e47fff700 1 -- 172.21.15.17:0/1998943027 learned_addr learned my addr 172.21.15.17:0/1998943027 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:05.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.238+0000 7f8e3effd700 1 --2- 172.21.15.17:0/1998943027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e480b0270 0x7f8e4809ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:05.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.238+0000 7f8e47fff700 1 -- 172.21.15.17:0/1998943027 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8e480ac880 msgr2=0x7f8e480ab720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:05.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.238+0000 7f8e47fff700 1 --2- 172.21.15.17:0/1998943027 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8e480ac880 0x7f8e480ab720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:05.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.238+0000 7f8e47fff700 1 -- 172.21.15.17:0/1998943027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e480b0270 msgr2=0x7f8e4809ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:05.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.238+0000 7f8e47fff700 1 --2- 172.21.15.17:0/1998943027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e480b0270 0x7f8e4809ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:05.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.238+0000 7f8e47fff700 1 -- 172.21.15.17:0/1998943027 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e3000b1e0 con 0x7f8e480b2710 2022-08-01T19:36:05.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.238+0000 7f8e47fff700 1 --2- 172.21.15.17:0/1998943027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e480b2710 0x7f8e481559a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f8e4000c000 tx=0x7f8e400091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:05.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.238+0000 7f8e457fa700 1 -- 172.21.15.17:0/1998943027 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e40012600 con 0x7f8e480b2710 2022-08-01T19:36:05.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.238+0000 7f8e457fa700 1 -- 172.21.15.17:0/1998943027 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e40012760 con 0x7f8e480b2710 2022-08-01T19:36:05.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.238+0000 7f8e4cb16700 1 -- 172.21.15.17:0/1998943027 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e480a1440 con 0x7f8e480b2710 2022-08-01T19:36:05.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.238+0000 7f8e457fa700 1 -- 172.21.15.17:0/1998943027 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e40012ac0 con 0x7f8e480b2710 2022-08-01T19:36:05.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.238+0000 7f8e4cb16700 1 -- 172.21.15.17:0/1998943027 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e48143e80 con 0x7f8e480b2710 2022-08-01T19:36:05.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.242+0000 7f8e4cb16700 1 -- 172.21.15.17:0/1998943027 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e481432c0 con 0x7f8e480b2710 2022-08-01T19:36:05.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.242+0000 7f8e457fa700 1 -- 172.21.15.17:0/1998943027 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8e40012c20 con 0x7f8e480b2710 2022-08-01T19:36:05.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.242+0000 7f8e457fa700 1 --2- 172.21.15.17:0/1998943027 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8e2c053950 0x7f8e2c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:05.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.242+0000 7f8e457fa700 1 -- 172.21.15.17:0/1998943027 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8e4006c310 con 0x7f8e480b2710 2022-08-01T19:36:05.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.242+0000 7f8e477fe700 1 --2- 172.21.15.17:0/1998943027 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8e2c053950 0x7f8e2c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:05.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.242+0000 7f8e477fe700 1 --2- 172.21.15.17:0/1998943027 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8e2c053950 0x7f8e2c055e10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f8e3800c000 tx=0x7f8e38008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:05.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.254+0000 7f8e457fa700 1 -- 172.21.15.17:0/1998943027 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8e40042fc0 con 0x7f8e480b2710 2022-08-01T19:36:05.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.374+0000 7f8e4cb16700 1 -- 172.21.15.17:0/1998943027 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e480574e0 con 0x7f8e480b2710 2022-08-01T19:36:05.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.378+0000 7f8e457fa700 1 -- 172.21.15.17:0/1998943027 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16271 (secure 0 0 0) 0x7f8e400465e0 con 0x7f8e480b2710 2022-08-01T19:36:05.383 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:36:05.383 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":45,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:36:03.174642+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:36:00.565402+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2022-08-01T19:36:01.813338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:36:05.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.378+0000 7f8e4cb16700 1 -- 172.21.15.17:0/1998943027 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8e2c053950 msgr2=0x7f8e2c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:05.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.378+0000 7f8e4cb16700 1 --2- 172.21.15.17:0/1998943027 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8e2c053950 0x7f8e2c055e10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f8e3800c000 tx=0x7f8e38008040).stop 2022-08-01T19:36:05.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.378+0000 7f8e4cb16700 1 -- 172.21.15.17:0/1998943027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e480b2710 msgr2=0x7f8e481559a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:05.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.378+0000 7f8e4cb16700 1 --2- 172.21.15.17:0/1998943027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e480b2710 0x7f8e481559a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f8e4000c000 tx=0x7f8e400091d0).stop 2022-08-01T19:36:05.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.378+0000 7f8e4cb16700 1 -- 172.21.15.17:0/1998943027 shutdown_connections 2022-08-01T19:36:05.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.378+0000 7f8e4cb16700 1 --2- 172.21.15.17:0/1998943027 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8e2c053950 0x7f8e2c055e10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:05.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.378+0000 7f8e4cb16700 1 --2- 172.21.15.17:0/1998943027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e480b2710 0x7f8e481559a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:05.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.378+0000 7f8e4cb16700 1 --2- 172.21.15.17:0/1998943027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e480b0270 0x7f8e4809ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:05.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.378+0000 7f8e4cb16700 1 --2- 172.21.15.17:0/1998943027 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8e480ac880 0x7f8e480ab720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:05.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.378+0000 7f8e4cb16700 1 -- 172.21.15.17:0/1998943027 >> 172.21.15.17:0/1998943027 conn(0x7f8e48060310 msgr2=0x7f8e4805cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:05.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.378+0000 7f8e4cb16700 1 -- 172.21.15.17:0/1998943027 shutdown_connections 2022-08-01T19:36:05.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.378+0000 7f8e4cb16700 1 -- 172.21.15.17:0/1998943027 wait complete. 2022-08-01T19:36:05.396 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T19:36:00.565402+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T19:36:01.813338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T19:36:05.397 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:36:05.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.498+0000 7f391ffff700 1 Processor -- start 2022-08-01T19:36:05.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.498+0000 7f391ffff700 1 -- start start 2022-08-01T19:36:05.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.498+0000 7f391ffff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f39200b6420 0x7f3920061600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:05.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.498+0000 7f391ffff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3920142ff0 0x7f3920143480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:05.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.498+0000 7f391ffff700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39200b60e0 con 0x7f39200b41f0 2022-08-01T19:36:05.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.498+0000 7f391ffff700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39200b2100 con 0x7f3920142ff0 2022-08-01T19:36:05.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.498+0000 7f391ffff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39200b23a0 con 0x7f39200b6420 2022-08-01T19:36:05.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.498+0000 7f391f7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3920142ff0 0x7f3920143480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:05.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.498+0000 7f391f7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3920142ff0 0x7f3920143480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38432/0 (socket says 172.21.15.17:38432) 2022-08-01T19:36:05.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.498+0000 7f391f7fe700 1 -- 172.21.15.17:0/2112633260 learned_addr learned my addr 172.21.15.17:0/2112633260 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:05.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.498+0000 7f391effd700 1 --2- 172.21.15.17:0/2112633260 >> v2:172.21.15.77:3300/0 conn(0x7f39200b6420 0x7f3920061600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:05.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.498+0000 7f391f7fe700 1 -- 172.21.15.17:0/2112633260 >> v2:172.21.15.77:3300/0 conn(0x7f39200b6420 msgr2=0x7f3920061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:36:05.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.498+0000 7f391f7fe700 1 --2- 172.21.15.17:0/2112633260 >> v2:172.21.15.77:3300/0 conn(0x7f39200b6420 0x7f3920061600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:05.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.498+0000 7f391f7fe700 1 -- 172.21.15.17:0/2112633260 >> v1:172.21.15.17:6789/0 conn(0x7f39200b41f0 legacy=0x7f3920061af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:05.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.502+0000 7f391f7fe700 1 -- 172.21.15.17:0/2112633260 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39200b2640 con 0x7f3920142ff0 2022-08-01T19:36:05.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.502+0000 7f391f7fe700 1 --2- 172.21.15.17:0/2112633260 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3920142ff0 0x7f3920143480 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f39140012a0 tx=0x7f3918009290).ready entity=mon.0 client_cookie=be5d872c6337a8d2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:05.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.502+0000 7f391dffb700 1 -- 172.21.15.17:0/2112633260 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f391800b730 con 0x7f3920142ff0 2022-08-01T19:36:05.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.502+0000 7f391dffb700 1 -- 172.21.15.17:0/2112633260 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f391800b890 con 0x7f3920142ff0 2022-08-01T19:36:05.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.502+0000 7f391dffb700 1 -- 172.21.15.17:0/2112633260 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f391800ba00 con 0x7f3920142ff0 2022-08-01T19:36:05.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.502+0000 7f391ffff700 1 -- 172.21.15.17:0/2112633260 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3920142ff0 msgr2=0x7f3920143480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:05.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.502+0000 7f391ffff700 1 --2- 172.21.15.17:0/2112633260 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3920142ff0 0x7f3920143480 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f39140012a0 tx=0x7f3918009290).stop 2022-08-01T19:36:05.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.502+0000 7f391ffff700 1 -- 172.21.15.17:0/2112633260 shutdown_connections 2022-08-01T19:36:05.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.502+0000 7f391ffff700 1 --2- 172.21.15.17:0/2112633260 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3920142ff0 0x7f3920143480 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:05.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.502+0000 7f391ffff700 1 --2- 172.21.15.17:0/2112633260 >> v2:172.21.15.77:3300/0 conn(0x7f39200b6420 0x7f3920061600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:05.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.502+0000 7f391ffff700 1 -- 172.21.15.17:0/2112633260 >> 172.21.15.17:0/2112633260 conn(0x7f392005c620 msgr2=0x7f392005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:05.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.502+0000 7f391ffff700 1 -- 172.21.15.17:0/2112633260 shutdown_connections 2022-08-01T19:36:05.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.502+0000 7f391ffff700 1 -- 172.21.15.17:0/2112633260 wait complete. 2022-08-01T19:36:05.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.502+0000 7f391ffff700 1 Processor -- start 2022-08-01T19:36:05.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.502+0000 7f391ffff700 1 -- start start 2022-08-01T19:36:05.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.502+0000 7f391ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39200b41f0 0x7f39200b86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:05.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.502+0000 7f391ffff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f39200b6420 0x7f392012ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:05.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.502+0000 7f391ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3920142ff0 0x7f392012b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:05.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.502+0000 7f391ffff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39200afdc0 con 0x7f39200b41f0 2022-08-01T19:36:05.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.502+0000 7f391ffff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39200b0290 con 0x7f3920142ff0 2022-08-01T19:36:05.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.502+0000 7f391ffff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39200b0530 con 0x7f39200b6420 2022-08-01T19:36:05.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.506+0000 7f391effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39200b41f0 0x7f39200b86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:05.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.506+0000 7f391effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39200b41f0 0x7f39200b86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38434/0 (socket says 172.21.15.17:38434) 2022-08-01T19:36:05.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.506+0000 7f391effd700 1 -- 172.21.15.17:0/1745917421 learned_addr learned my addr 172.21.15.17:0/1745917421 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:05.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.506+0000 7f391f7fe700 1 --2- 172.21.15.17:0/1745917421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3920142ff0 0x7f392012b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:05.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.506+0000 7f391e7fc700 1 --2- 172.21.15.17:0/1745917421 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f39200b6420 0x7f392012ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:05.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.506+0000 7f391effd700 1 -- 172.21.15.17:0/1745917421 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f39200b6420 msgr2=0x7f392012ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:05.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.506+0000 7f391effd700 1 --2- 172.21.15.17:0/1745917421 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f39200b6420 0x7f392012ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:05.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.506+0000 7f391effd700 1 -- 172.21.15.17:0/1745917421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3920142ff0 msgr2=0x7f392012b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:05.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.506+0000 7f391effd700 1 --2- 172.21.15.17:0/1745917421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3920142ff0 0x7f392012b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:05.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.506+0000 7f391effd700 1 -- 172.21.15.17:0/1745917421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3918009530 con 0x7f39200b41f0 2022-08-01T19:36:05.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.506+0000 7f391f7fe700 1 --2- 172.21.15.17:0/1745917421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3920142ff0 0x7f392012b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:36:05.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.506+0000 7f391effd700 1 --2- 172.21.15.17:0/1745917421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39200b41f0 0x7f39200b86c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f39100030b0 tx=0x7f3910008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:05.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.506+0000 7f390ffff700 1 -- 172.21.15.17:0/1745917421 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3910011780 con 0x7f39200b41f0 2022-08-01T19:36:05.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.506+0000 7f390ffff700 1 -- 172.21.15.17:0/1745917421 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39100118e0 con 0x7f39200b41f0 2022-08-01T19:36:05.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.506+0000 7f390ffff700 1 -- 172.21.15.17:0/1745917421 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3910011c40 con 0x7f39200b41f0 2022-08-01T19:36:05.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.506+0000 7f391ffff700 1 -- 172.21.15.17:0/1745917421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39200b07d0 con 0x7f39200b41f0 2022-08-01T19:36:05.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.506+0000 7f391ffff700 1 -- 172.21.15.17:0/1745917421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39200aeaa0 con 0x7f39200b41f0 2022-08-01T19:36:05.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.506+0000 7f390ffff700 1 -- 172.21.15.17:0/1745917421 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f391001c460 con 0x7f39200b41f0 2022-08-01T19:36:05.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.506+0000 7f391ffff700 1 -- 172.21.15.17:0/1745917421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39200af800 con 0x7f39200b41f0 2022-08-01T19:36:05.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.506+0000 7f390ffff700 1 --2- 172.21.15.17:0/1745917421 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f38fc053b00 0x7f38fc055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:05.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.506+0000 7f390ffff700 1 -- 172.21.15.17:0/1745917421 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f391006c210 con 0x7f39200b41f0 2022-08-01T19:36:05.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.510+0000 7f391e7fc700 1 --2- 172.21.15.17:0/1745917421 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f38fc053b00 0x7f38fc055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:05.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.510+0000 7f391e7fc700 1 --2- 172.21.15.17:0/1745917421 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f38fc053b00 0x7f38fc055fc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f391400d000 tx=0x7f3914001a30).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:05.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.514+0000 7f390ffff700 1 -- 172.21.15.17:0/1745917421 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3910046e60 con 0x7f39200b41f0 2022-08-01T19:36:05.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.638+0000 7f391ffff700 1 -- 172.21.15.17:0/1745917421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f39200b28e0 con 0x7f39200b41f0 2022-08-01T19:36:05.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.638+0000 7f390ffff700 1 -- 172.21.15.17:0/1745917421 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v45) v1 ==== 110+0+10 (secure 0 0 0) 0x7f3910042f70 con 0x7f39200b41f0 2022-08-01T19:36:05.643 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:36:05.645 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.638+0000 7f391ffff700 1 -- 172.21.15.17:0/1745917421 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f38fc053b00 msgr2=0x7f38fc055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:05.645 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.638+0000 7f391ffff700 1 --2- 172.21.15.17:0/1745917421 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f38fc053b00 0x7f38fc055fc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f391400d000 tx=0x7f3914001a30).stop 2022-08-01T19:36:05.645 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.638+0000 7f391ffff700 1 -- 172.21.15.17:0/1745917421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39200b41f0 msgr2=0x7f39200b86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:05.645 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.638+0000 7f391ffff700 1 --2- 172.21.15.17:0/1745917421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39200b41f0 0x7f39200b86c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f39100030b0 tx=0x7f3910008040).stop 2022-08-01T19:36:05.646 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.638+0000 7f391ffff700 1 -- 172.21.15.17:0/1745917421 shutdown_connections 2022-08-01T19:36:05.646 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.638+0000 7f391ffff700 1 --2- 172.21.15.17:0/1745917421 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f38fc053b00 0x7f38fc055fc0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:05.646 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.638+0000 7f391ffff700 1 --2- 172.21.15.17:0/1745917421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3920142ff0 0x7f392012b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:05.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.638+0000 7f391ffff700 1 --2- 172.21.15.17:0/1745917421 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f39200b6420 0x7f392012ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:05.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.638+0000 7f391ffff700 1 --2- 172.21.15.17:0/1745917421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39200b41f0 0x7f39200b86c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:05.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.638+0000 7f391ffff700 1 -- 172.21.15.17:0/1745917421 >> 172.21.15.17:0/1745917421 conn(0x7f392005c620 msgr2=0x7f39201415b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:05.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.638+0000 7f391ffff700 1 -- 172.21.15.17:0/1745917421 shutdown_connections 2022-08-01T19:36:05.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.638+0000 7f391ffff700 1 -- 172.21.15.17:0/1745917421 wait complete. 2022-08-01T19:36:05.656 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:36:05.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.758+0000 7f9c13e13700 1 Processor -- start 2022-08-01T19:36:05.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.758+0000 7f9c13e13700 1 -- start start 2022-08-01T19:36:05.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.758+0000 7f9c13e13700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f9c0c0b1460 0x7f9c0c0b0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:05.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.758+0000 7f9c13e13700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c0c0a7960 con 0x7f9c0c0b2430 2022-08-01T19:36:05.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.758+0000 7f9c13e13700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c0c0b07e0 con 0x7f9c0c0b2050 2022-08-01T19:36:05.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.758+0000 7f9c13e13700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c0c0b18d0 con 0x7f9c0c0b1460 2022-08-01T19:36:05.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.758+0000 7f9c123b0700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9c0c0b2430 0x7f9c0c1413e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58872/0 (socket says 172.21.15.17:58872) 2022-08-01T19:36:05.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.758+0000 7f9c123b0700 1 -- 172.21.15.17:0/1796893569 learned_addr learned my addr 172.21.15.17:0/1796893569 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:36:05.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.758+0000 7f9c113ae700 1 --2- 172.21.15.17:0/1796893569 >> v2:172.21.15.77:3300/0 conn(0x7f9c0c0b1460 0x7f9c0c0b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:05.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.758+0000 7f9c10bad700 1 -- 172.21.15.17:0/1796893569 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1847625882 0 0) 0x7f9c0c0a7960 con 0x7f9c0c0b2430 2022-08-01T19:36:05.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.758+0000 7f9c10bad700 1 -- 172.21.15.17:0/1796893569 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bf4003260 con 0x7f9c0c0b2430 2022-08-01T19:36:05.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.758+0000 7f9c10bad700 1 -- 172.21.15.17:0/1796893569 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1795874567 0 0) 0x7f9bf4003260 con 0x7f9c0c0b2430 2022-08-01T19:36:05.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.758+0000 7f9c113ae700 1 -- 172.21.15.17:0/1796893569 >> v1:172.21.15.77:6789/0 conn(0x7f9c0c0b2050 legacy=0x7f9c0c0a7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:05.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.758+0000 7f9c113ae700 1 -- 172.21.15.17:0/1796893569 >> v1:172.21.15.17:6789/0 conn(0x7f9c0c0b2430 legacy=0x7f9c0c1413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:05.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.758+0000 7f9c113ae700 1 -- 172.21.15.17:0/1796893569 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c0c0b0a80 con 0x7f9c0c0b1460 2022-08-01T19:36:05.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.758+0000 7f9c113ae700 1 --2- 172.21.15.17:0/1796893569 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c0c0b1460 0x7f9c0c0b0140 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f9c04001b80 tx=0x7f9c08006df0).ready entity=mon.1 client_cookie=ae4fd0053a86ed3 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:05.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.758+0000 7f9c10bad700 1 -- 172.21.15.17:0/1796893569 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c040019e0 con 0x7f9c0c0b1460 2022-08-01T19:36:05.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.758+0000 7f9c10bad700 1 -- 172.21.15.17:0/1796893569 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c08000e90 con 0x7f9c0c0b1460 2022-08-01T19:36:05.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.758+0000 7f9c10bad700 1 -- 172.21.15.17:0/1796893569 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c08011400 con 0x7f9c0c0b1460 2022-08-01T19:36:05.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.758+0000 7f9c13e13700 1 -- 172.21.15.17:0/1796893569 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c0c0b1460 msgr2=0x7f9c0c0b0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:05.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.758+0000 7f9c13e13700 1 --2- 172.21.15.17:0/1796893569 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c0c0b1460 0x7f9c0c0b0140 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f9c04001b80 tx=0x7f9c08006df0).stop 2022-08-01T19:36:05.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.758+0000 7f9c13e13700 1 -- 172.21.15.17:0/1796893569 shutdown_connections 2022-08-01T19:36:05.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.758+0000 7f9c13e13700 1 --2- 172.21.15.17:0/1796893569 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c0c0b1460 0x7f9c0c0b0140 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:05.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.758+0000 7f9c13e13700 1 -- 172.21.15.17:0/1796893569 >> 172.21.15.17:0/1796893569 conn(0x7f9c0c05f840 msgr2=0x7f9c0c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:05.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.758+0000 7f9c13e13700 1 -- 172.21.15.17:0/1796893569 shutdown_connections 2022-08-01T19:36:05.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.758+0000 7f9c13e13700 1 -- 172.21.15.17:0/1796893569 wait complete. 2022-08-01T19:36:05.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.762+0000 7f9c13e13700 1 Processor -- start 2022-08-01T19:36:05.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.762+0000 7f9c13e13700 1 -- start start 2022-08-01T19:36:05.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.762+0000 7f9c13e13700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9c0c0b1460 0x7f9c0c0af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:05.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.762+0000 7f9c13e13700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c0c0b2050 0x7f9c0c15a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:05.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.762+0000 7f9c13e13700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c0c0b2430 0x7f9c0c15a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:05.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.762+0000 7f9c13e13700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0c149960 con 0x7f9c0c0b2430 2022-08-01T19:36:05.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.762+0000 7f9c13e13700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0c156de0 con 0x7f9c0c0b2050 2022-08-01T19:36:05.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.762+0000 7f9c13e13700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0c15ae60 con 0x7f9c0c0b1460 2022-08-01T19:36:05.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.762+0000 7f9c123b0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c0c0b2430 0x7f9c0c15a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:05.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.762+0000 7f9c123b0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c0c0b2430 0x7f9c0c15a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38450/0 (socket says 172.21.15.17:38450) 2022-08-01T19:36:05.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.762+0000 7f9c123b0700 1 -- 172.21.15.17:0/2903721520 learned_addr learned my addr 172.21.15.17:0/2903721520 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:05.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.762+0000 7f9c113ae700 1 --2- 172.21.15.17:0/2903721520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c0c0b2050 0x7f9c0c15a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:05.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.762+0000 7f9c11baf700 1 --2- 172.21.15.17:0/2903721520 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9c0c0b1460 0x7f9c0c0af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:05.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.762+0000 7f9c123b0700 1 -- 172.21.15.17:0/2903721520 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9c0c0b1460 msgr2=0x7f9c0c0af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:05.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.762+0000 7f9c123b0700 1 --2- 172.21.15.17:0/2903721520 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9c0c0b1460 0x7f9c0c0af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:05.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.762+0000 7f9c123b0700 1 -- 172.21.15.17:0/2903721520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c0c0b2050 msgr2=0x7f9c0c15a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:05.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.762+0000 7f9c123b0700 1 --2- 172.21.15.17:0/2903721520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c0c0b2050 0x7f9c0c15a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:05.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.762+0000 7f9c123b0700 1 -- 172.21.15.17:0/2903721520 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c0800b180 con 0x7f9c0c0b2430 2022-08-01T19:36:05.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.762+0000 7f9c113ae700 1 --2- 172.21.15.17:0/2903721520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c0c0b2050 0x7f9c0c15a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:36:05.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.762+0000 7f9c123b0700 1 --2- 172.21.15.17:0/2903721520 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c0c0b2430 0x7f9c0c15a8a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f9c0c063080 tx=0x7f9c040020c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:05.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.762+0000 7f9bfaffd700 1 -- 172.21.15.17:0/2903721520 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c04004ab0 con 0x7f9c0c0b2430 2022-08-01T19:36:05.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.762+0000 7f9bfaffd700 1 -- 172.21.15.17:0/2903721520 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c04004c10 con 0x7f9c0c0b2430 2022-08-01T19:36:05.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.762+0000 7f9c13e13700 1 -- 172.21.15.17:0/2903721520 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c0c148cc0 con 0x7f9c0c0b2430 2022-08-01T19:36:05.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.762+0000 7f9bfaffd700 1 -- 172.21.15.17:0/2903721520 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c04007170 con 0x7f9c0c0b2430 2022-08-01T19:36:05.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.762+0000 7f9c13e13700 1 -- 172.21.15.17:0/2903721520 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c0c0ac3d0 con 0x7f9c0c0b2430 2022-08-01T19:36:05.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.766+0000 7f9c13e13700 1 -- 172.21.15.17:0/2903721520 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c0c15b290 con 0x7f9c0c0b2430 2022-08-01T19:36:05.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.766+0000 7f9bfaffd700 1 -- 172.21.15.17:0/2903721520 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9c04010070 con 0x7f9c0c0b2430 2022-08-01T19:36:05.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.766+0000 7f9bfaffd700 1 --2- 172.21.15.17:0/2903721520 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9bec053a70 0x7f9bec055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:05.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.766+0000 7f9bfaffd700 1 -- 172.21.15.17:0/2903721520 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9c0406bc50 con 0x7f9c0c0b2430 2022-08-01T19:36:05.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.766+0000 7f9c11baf700 1 --2- 172.21.15.17:0/2903721520 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9bec053a70 0x7f9bec055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:05.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.766+0000 7f9c11baf700 1 --2- 172.21.15.17:0/2903721520 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9bec053a70 0x7f9bec055f30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f9bfc00b000 tx=0x7f9bfc009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:05.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.774+0000 7f9bfaffd700 1 -- 172.21.15.17:0/2903721520 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9c04042880 con 0x7f9c0c0b2430 2022-08-01T19:36:05.909 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.902+0000 7f9c13e13700 1 -- 172.21.15.17:0/2903721520 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9c0c0574e0 con 0x7f9c0c0b2430 2022-08-01T19:36:05.909 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.906+0000 7f9bfaffd700 1 -- 172.21.15.17:0/2903721520 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v45) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9c04019030 con 0x7f9c0c0b2430 2022-08-01T19:36:05.910 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:36:05.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.906+0000 7f9c13e13700 1 -- 172.21.15.17:0/2903721520 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9bec053a70 msgr2=0x7f9bec055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:05.912 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.906+0000 7f9c13e13700 1 --2- 172.21.15.17:0/2903721520 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9bec053a70 0x7f9bec055f30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f9bfc00b000 tx=0x7f9bfc009040).stop 2022-08-01T19:36:05.912 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.906+0000 7f9c13e13700 1 -- 172.21.15.17:0/2903721520 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c0c0b2430 msgr2=0x7f9c0c15a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:05.912 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.906+0000 7f9c13e13700 1 --2- 172.21.15.17:0/2903721520 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c0c0b2430 0x7f9c0c15a8a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f9c0c063080 tx=0x7f9c040020c0).stop 2022-08-01T19:36:05.913 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.906+0000 7f9c13e13700 1 -- 172.21.15.17:0/2903721520 shutdown_connections 2022-08-01T19:36:05.913 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.906+0000 7f9c13e13700 1 --2- 172.21.15.17:0/2903721520 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9bec053a70 0x7f9bec055f30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:05.913 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.906+0000 7f9c13e13700 1 --2- 172.21.15.17:0/2903721520 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c0c0b2430 0x7f9c0c15a8a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:05.913 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.906+0000 7f9c13e13700 1 --2- 172.21.15.17:0/2903721520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c0c0b2050 0x7f9c0c15a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:05.914 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.906+0000 7f9c13e13700 1 --2- 172.21.15.17:0/2903721520 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9c0c0b1460 0x7f9c0c0af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:05.914 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.906+0000 7f9c13e13700 1 -- 172.21.15.17:0/2903721520 >> 172.21.15.17:0/2903721520 conn(0x7f9c0c05f840 msgr2=0x7f9c0c13ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:05.914 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.906+0000 7f9c13e13700 1 -- 172.21.15.17:0/2903721520 shutdown_connections 2022-08-01T19:36:05.915 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:05.906+0000 7f9c13e13700 1 -- 172.21.15.17:0/2903721520 wait complete. 2022-08-01T19:36:05.929 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T19:36:06.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.042+0000 7f3cc91aa700 1 Processor -- start 2022-08-01T19:36:06.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.042+0000 7f3cc91aa700 1 -- start start 2022-08-01T19:36:06.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.042+0000 7f3cc91aa700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3cc40a8c50 0x7f3cc4152d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:06.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.042+0000 7f3cc91aa700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cc40a7250 con 0x7f3cc40b01a0 2022-08-01T19:36:06.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.042+0000 7f3cc91aa700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cc40a7480 con 0x7f3cc40b2710 2022-08-01T19:36:06.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.042+0000 7f3cc91aa700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cc40a7720 con 0x7f3cc40a8c50 2022-08-01T19:36:06.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.042+0000 7f3cc89a9700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3cc40a8c50 0x7f3cc4152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:06.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.042+0000 7f3cc89a9700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3cc40a8c50 0x7f3cc4152d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38460/0 (socket says 172.21.15.17:38460) 2022-08-01T19:36:06.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.042+0000 7f3cc89a9700 1 -- 172.21.15.17:0/1148630581 learned_addr learned my addr 172.21.15.17:0/1148630581 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:06.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.042+0000 7f3cc37fe700 1 --1- 172.21.15.17:0/1148630581 >> v1:172.21.15.17:6789/0 conn(0x7f3cc40b01a0 0x7f3cc40a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58884/0 (socket says 172.21.15.17:58884) 2022-08-01T19:36:06.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.042+0000 7f3cc89a9700 1 -- 172.21.15.17:0/1148630581 >> v1:172.21.15.77:6789/0 conn(0x7f3cc40b2710 legacy=0x7f3cc414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:06.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.042+0000 7f3cc89a9700 1 -- 172.21.15.17:0/1148630581 >> v1:172.21.15.17:6789/0 conn(0x7f3cc40b01a0 legacy=0x7f3cc40a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:06.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.042+0000 7f3cc89a9700 1 -- 172.21.15.17:0/1148630581 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc40a79c0 con 0x7f3cc40a8c50 2022-08-01T19:36:06.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.042+0000 7f3cc89a9700 1 --2- 172.21.15.17:0/1148630581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3cc40a8c50 0x7f3cc4152d70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f3cbc002f70 tx=0x7f3cbc006eb0).ready entity=mon.0 client_cookie=6e422c89bbecf596 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:06.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.042+0000 7f3cc2ffd700 1 -- 172.21.15.17:0/1148630581 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3cbc00d660 con 0x7f3cc40a8c50 2022-08-01T19:36:06.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.042+0000 7f3cc2ffd700 1 -- 172.21.15.17:0/1148630581 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cbc000e90 con 0x7f3cc40a8c50 2022-08-01T19:36:06.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.042+0000 7f3cc2ffd700 1 -- 172.21.15.17:0/1148630581 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3cbc00d9f0 con 0x7f3cc40a8c50 2022-08-01T19:36:06.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.042+0000 7f3cc91aa700 1 -- 172.21.15.17:0/1148630581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3cc40a8c50 msgr2=0x7f3cc4152d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:06.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.042+0000 7f3cc91aa700 1 --2- 172.21.15.17:0/1148630581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3cc40a8c50 0x7f3cc4152d70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f3cbc002f70 tx=0x7f3cbc006eb0).stop 2022-08-01T19:36:06.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.042+0000 7f3cc91aa700 1 -- 172.21.15.17:0/1148630581 shutdown_connections 2022-08-01T19:36:06.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.042+0000 7f3cc91aa700 1 --2- 172.21.15.17:0/1148630581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3cc40a8c50 0x7f3cc4152d70 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:06.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.042+0000 7f3cc91aa700 1 -- 172.21.15.17:0/1148630581 >> 172.21.15.17:0/1148630581 conn(0x7f3cc4060310 msgr2=0x7f3cc40b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:06.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.042+0000 7f3cc91aa700 1 -- 172.21.15.17:0/1148630581 shutdown_connections 2022-08-01T19:36:06.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.042+0000 7f3cc91aa700 1 -- 172.21.15.17:0/1148630581 wait complete. 2022-08-01T19:36:06.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.046+0000 7f3cc91aa700 1 Processor -- start 2022-08-01T19:36:06.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.046+0000 7f3cc91aa700 1 -- start start 2022-08-01T19:36:06.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.046+0000 7f3cc91aa700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3cc40a8c50 0x7f3cc40abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:06.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.046+0000 7f3cc91aa700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3cc40b01a0 0x7f3cc40662a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:06.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.046+0000 7f3cc91aa700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3cc40b2710 0x7f3cc40b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:06.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.046+0000 7f3cc91aa700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc40b59e0 con 0x7f3cc40b01a0 2022-08-01T19:36:06.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.046+0000 7f3cc91aa700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc4065e40 con 0x7f3cc40b2710 2022-08-01T19:36:06.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.046+0000 7f3cc91aa700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc40b5740 con 0x7f3cc40a8c50 2022-08-01T19:36:06.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.046+0000 7f3cc37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3cc40b01a0 0x7f3cc40662a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:06.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.046+0000 7f3cc37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3cc40b01a0 0x7f3cc40662a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38464/0 (socket says 172.21.15.17:38464) 2022-08-01T19:36:06.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.046+0000 7f3cc37fe700 1 -- 172.21.15.17:0/1331749398 learned_addr learned my addr 172.21.15.17:0/1331749398 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:06.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.046+0000 7f3cc89a9700 1 --2- 172.21.15.17:0/1331749398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3cc40b2710 0x7f3cc40b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:06.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.046+0000 7f3cc3fff700 1 --2- 172.21.15.17:0/1331749398 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3cc40a8c50 0x7f3cc40abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:06.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.046+0000 7f3cc37fe700 1 -- 172.21.15.17:0/1331749398 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3cc40a8c50 msgr2=0x7f3cc40abbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:06.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.046+0000 7f3cc37fe700 1 --2- 172.21.15.17:0/1331749398 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3cc40a8c50 0x7f3cc40abbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:06.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.046+0000 7f3cc37fe700 1 -- 172.21.15.17:0/1331749398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3cc40b2710 msgr2=0x7f3cc40b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:06.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.046+0000 7f3cc37fe700 1 --2- 172.21.15.17:0/1331749398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3cc40b2710 0x7f3cc40b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:06.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.046+0000 7f3cc37fe700 1 -- 172.21.15.17:0/1331749398 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cbc00b1e0 con 0x7f3cc40b01a0 2022-08-01T19:36:06.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.046+0000 7f3cc37fe700 1 --2- 172.21.15.17:0/1331749398 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3cc40b01a0 0x7f3cc40662a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7f3cc406b950 tx=0x7f3cac007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:06.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.046+0000 7f3cc17fa700 1 -- 172.21.15.17:0/1331749398 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3cac0070d0 con 0x7f3cc40b01a0 2022-08-01T19:36:06.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.046+0000 7f3cc17fa700 1 -- 172.21.15.17:0/1331749398 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cac007230 con 0x7f3cc40b01a0 2022-08-01T19:36:06.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.046+0000 7f3cc17fa700 1 -- 172.21.15.17:0/1331749398 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3cac005550 con 0x7f3cc40b01a0 2022-08-01T19:36:06.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.046+0000 7f3cc91aa700 1 -- 172.21.15.17:0/1331749398 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cc415a7e0 con 0x7f3cc40b01a0 2022-08-01T19:36:06.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.046+0000 7f3cc91aa700 1 -- 172.21.15.17:0/1331749398 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc40afec0 con 0x7f3cc40b01a0 2022-08-01T19:36:06.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.050+0000 7f3cc17fa700 1 -- 172.21.15.17:0/1331749398 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3cac01a070 con 0x7f3cc40b01a0 2022-08-01T19:36:06.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.050+0000 7f3cc91aa700 1 -- 172.21.15.17:0/1331749398 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cc415a640 con 0x7f3cc40b01a0 2022-08-01T19:36:06.063 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.050+0000 7f3cc17fa700 1 --2- 172.21.15.17:0/1331749398 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3ca8053b00 0x7f3ca8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:06.063 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.050+0000 7f3cc17fa700 1 -- 172.21.15.17:0/1331749398 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f3cac06b190 con 0x7f3cc40b01a0 2022-08-01T19:36:06.063 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.050+0000 7f3cc3fff700 1 --2- 172.21.15.17:0/1331749398 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3ca8053b00 0x7f3ca8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:06.063 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.050+0000 7f3cc3fff700 1 --2- 172.21.15.17:0/1331749398 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3ca8053b00 0x7f3ca8055fc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f3cb400d000 tx=0x7f3cb4007f10).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:06.064 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.058+0000 7f3cc17fa700 1 -- 172.21.15.17:0/1331749398 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3cc415a640 con 0x7f3cc40b01a0 2022-08-01T19:36:06.188 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.182+0000 7f3cc91aa700 1 -- 172.21.15.17:0/1331749398 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3cc415a640 con 0x7f3cc40b01a0 2022-08-01T19:36:06.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.182+0000 7f3cc17fa700 1 -- 172.21.15.17:0/1331749398 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) v1 ==== 104+0+10 (secure 0 0 0) 0x7f3cc415a640 con 0x7f3cc40b01a0 2022-08-01T19:36:06.189 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:36:06.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.186+0000 7f3cc91aa700 1 -- 172.21.15.17:0/1331749398 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3ca8053b00 msgr2=0x7f3ca8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:06.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.186+0000 7f3cc91aa700 1 --2- 172.21.15.17:0/1331749398 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3ca8053b00 0x7f3ca8055fc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f3cb400d000 tx=0x7f3cb4007f10).stop 2022-08-01T19:36:06.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.186+0000 7f3cc91aa700 1 -- 172.21.15.17:0/1331749398 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3cc40b01a0 msgr2=0x7f3cc40662a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:06.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.186+0000 7f3cc91aa700 1 --2- 172.21.15.17:0/1331749398 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3cc40b01a0 0x7f3cc40662a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7f3cc406b950 tx=0x7f3cac007be0).stop 2022-08-01T19:36:06.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.186+0000 7f3cc91aa700 1 -- 172.21.15.17:0/1331749398 shutdown_connections 2022-08-01T19:36:06.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.186+0000 7f3cc91aa700 1 --2- 172.21.15.17:0/1331749398 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3ca8053b00 0x7f3ca8055fc0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:06.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.186+0000 7f3cc91aa700 1 --2- 172.21.15.17:0/1331749398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3cc40b2710 0x7f3cc40b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:06.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.186+0000 7f3cc91aa700 1 --2- 172.21.15.17:0/1331749398 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3cc40b01a0 0x7f3cc40662a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:06.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.186+0000 7f3cc91aa700 1 --2- 172.21.15.17:0/1331749398 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3cc40a8c50 0x7f3cc40abbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:06.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.186+0000 7f3cc91aa700 1 -- 172.21.15.17:0/1331749398 >> 172.21.15.17:0/1331749398 conn(0x7f3cc4060310 msgr2=0x7f3cc405be30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:06.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.186+0000 7f3cc91aa700 1 -- 172.21.15.17:0/1331749398 shutdown_connections 2022-08-01T19:36:06.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.186+0000 7f3cc91aa700 1 -- 172.21.15.17:0/1331749398 wait complete. 2022-08-01T19:36:06.202 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T19:36:06.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.306+0000 7f9690049700 1 Processor -- start 2022-08-01T19:36:06.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.306+0000 7f9690049700 1 -- start start 2022-08-01T19:36:06.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.306+0000 7f9690049700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96880b38a0 0x7f96880b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:06.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.306+0000 7f9690049700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f96880a9da0 0x7f96880b1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:06.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.306+0000 7f9690049700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9688143ab0 con 0x7f96880b0740 2022-08-01T19:36:06.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.310+0000 7f9690049700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9688143d50 con 0x7f96880a9da0 2022-08-01T19:36:06.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.310+0000 7f9690049700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9688143ff0 con 0x7f96880b38a0 2022-08-01T19:36:06.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.310+0000 7f968dde5700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f96880b0740 0x7f968813d6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58896/0 (socket says 172.21.15.17:58896) 2022-08-01T19:36:06.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.310+0000 7f968dde5700 1 -- 172.21.15.17:0/404903425 learned_addr learned my addr 172.21.15.17:0/404903425 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:36:06.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.310+0000 7f968e5e6700 1 --2- 172.21.15.17:0/404903425 >> v2:172.21.15.17:3300/0 conn(0x7f96880a9da0 0x7f96880b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:06.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.310+0000 7f968d5e4700 1 --2- 172.21.15.17:0/404903425 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96880b38a0 0x7f96880b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:06.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.310+0000 7f968cde3700 1 -- 172.21.15.17:0/404903425 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3071523963 0 0) 0x7f9688143ab0 con 0x7f96880b0740 2022-08-01T19:36:06.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.310+0000 7f968cde3700 1 -- 172.21.15.17:0/404903425 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9670003260 con 0x7f96880b0740 2022-08-01T19:36:06.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.310+0000 7f968e5e6700 1 -- 172.21.15.17:0/404903425 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96880b38a0 msgr2=0x7f96880b0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:36:06.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.310+0000 7f968e5e6700 1 --2- 172.21.15.17:0/404903425 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96880b38a0 0x7f96880b0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:06.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.310+0000 7f968e5e6700 1 -- 172.21.15.17:0/404903425 >> v1:172.21.15.17:6789/0 conn(0x7f96880b0740 legacy=0x7f968813d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:06.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.310+0000 7f968e5e6700 1 -- 172.21.15.17:0/404903425 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9688144290 con 0x7f96880a9da0 2022-08-01T19:36:06.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.310+0000 7f968e5e6700 1 --2- 172.21.15.17:0/404903425 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96880a9da0 0x7f96880b1240 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f96800032f0 tx=0x7f96800091d0).ready entity=mon.0 client_cookie=73ce0a8d68f7346f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:06.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.310+0000 7f968cde3700 1 -- 172.21.15.17:0/404903425 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f968000b6c0 con 0x7f96880a9da0 2022-08-01T19:36:06.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.310+0000 7f968cde3700 1 -- 172.21.15.17:0/404903425 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f968000b820 con 0x7f96880a9da0 2022-08-01T19:36:06.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.310+0000 7f968cde3700 1 -- 172.21.15.17:0/404903425 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f968000b990 con 0x7f96880a9da0 2022-08-01T19:36:06.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.310+0000 7f9690049700 1 -- 172.21.15.17:0/404903425 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96880a9da0 msgr2=0x7f96880b1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:06.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.310+0000 7f9690049700 1 --2- 172.21.15.17:0/404903425 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96880a9da0 0x7f96880b1240 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f96800032f0 tx=0x7f96800091d0).stop 2022-08-01T19:36:06.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.310+0000 7f9690049700 1 -- 172.21.15.17:0/404903425 shutdown_connections 2022-08-01T19:36:06.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.310+0000 7f9690049700 1 --2- 172.21.15.17:0/404903425 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96880a9da0 0x7f96880b1240 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:06.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.310+0000 7f9690049700 1 --2- 172.21.15.17:0/404903425 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96880b38a0 0x7f96880b0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:06.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.310+0000 7f9690049700 1 -- 172.21.15.17:0/404903425 >> 172.21.15.17:0/404903425 conn(0x7f9688060950 msgr2=0x7f9688060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:06.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.310+0000 7f9690049700 1 -- 172.21.15.17:0/404903425 shutdown_connections 2022-08-01T19:36:06.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.310+0000 7f9690049700 1 -- 172.21.15.17:0/404903425 wait complete. 2022-08-01T19:36:06.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.314+0000 7f9690049700 1 Processor -- start 2022-08-01T19:36:06.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.314+0000 7f9690049700 1 -- start start 2022-08-01T19:36:06.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.314+0000 7f9690049700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96880a9da0 0x7f96880ae0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:06.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.314+0000 7f9690049700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96880b0740 0x7f968814fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:06.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.314+0000 7f9690049700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96880b38a0 0x7f9688140f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:06.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.314+0000 7f9690049700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9688140af0 con 0x7f96880a9da0 2022-08-01T19:36:06.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.314+0000 7f9690049700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96880ae880 con 0x7f96880b0740 2022-08-01T19:36:06.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.314+0000 7f9690049700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96880ad4d0 con 0x7f96880b38a0 2022-08-01T19:36:06.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.314+0000 7f968dde5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96880a9da0 0x7f96880ae0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:06.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.314+0000 7f968dde5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96880a9da0 0x7f96880ae0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38476/0 (socket says 172.21.15.17:38476) 2022-08-01T19:36:06.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.314+0000 7f968d5e4700 1 --2- 172.21.15.17:0/493924190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96880b0740 0x7f968814fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:06.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.314+0000 7f968dde5700 1 -- 172.21.15.17:0/493924190 learned_addr learned my addr 172.21.15.17:0/493924190 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:06.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.314+0000 7f968e5e6700 1 --2- 172.21.15.17:0/493924190 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96880b38a0 0x7f9688140f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:06.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.314+0000 7f968dde5700 1 -- 172.21.15.17:0/493924190 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96880b38a0 msgr2=0x7f9688140f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:06.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.314+0000 7f968dde5700 1 --2- 172.21.15.17:0/493924190 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96880b38a0 0x7f9688140f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:06.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.314+0000 7f968dde5700 1 -- 172.21.15.17:0/493924190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96880b0740 msgr2=0x7f968814fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:06.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.314+0000 7f968dde5700 1 --2- 172.21.15.17:0/493924190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96880b0740 0x7f968814fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:06.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.314+0000 7f968dde5700 1 -- 172.21.15.17:0/493924190 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9680009470 con 0x7f96880a9da0 2022-08-01T19:36:06.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.314+0000 7f968d5e4700 1 --2- 172.21.15.17:0/493924190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96880b0740 0x7f968814fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:36:06.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.314+0000 7f968e5e6700 1 --2- 172.21.15.17:0/493924190 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96880b38a0 0x7f9688140f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:36:06.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.314+0000 7f968dde5700 1 --2- 172.21.15.17:0/493924190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96880a9da0 0x7f96880ae0e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f9684001110 tx=0x7f9684007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:06.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.314+0000 7f9676ffd700 1 -- 172.21.15.17:0/493924190 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96840070d0 con 0x7f96880a9da0 2022-08-01T19:36:06.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.318+0000 7f9676ffd700 1 -- 172.21.15.17:0/493924190 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9684007230 con 0x7f96880a9da0 2022-08-01T19:36:06.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.318+0000 7f9690049700 1 -- 172.21.15.17:0/493924190 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96880ad770 con 0x7f96880a9da0 2022-08-01T19:36:06.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.318+0000 7f9676ffd700 1 -- 172.21.15.17:0/493924190 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9684005610 con 0x7f96880a9da0 2022-08-01T19:36:06.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.318+0000 7f9690049700 1 -- 172.21.15.17:0/493924190 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9688150130 con 0x7f96880a9da0 2022-08-01T19:36:06.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.318+0000 7f9690049700 1 -- 172.21.15.17:0/493924190 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f968813ddf0 con 0x7f96880a9da0 2022-08-01T19:36:06.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.318+0000 7f9676ffd700 1 -- 172.21.15.17:0/493924190 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9684010070 con 0x7f96880a9da0 2022-08-01T19:36:06.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.318+0000 7f9676ffd700 1 --2- 172.21.15.17:0/493924190 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9668053980 0x7f9668055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:06.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.318+0000 7f9676ffd700 1 -- 172.21.15.17:0/493924190 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f968406b5a0 con 0x7f96880a9da0 2022-08-01T19:36:06.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.318+0000 7f968d5e4700 1 --2- 172.21.15.17:0/493924190 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9668053980 0x7f9668055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:06.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.322+0000 7f968d5e4700 1 --2- 172.21.15.17:0/493924190 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9668053980 0x7f9668055e40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f967800c000 tx=0x7f9678008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:06.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.326+0000 7f9676ffd700 1 -- 172.21.15.17:0/493924190 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9684042360 con 0x7f96880a9da0 2022-08-01T19:36:06.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.454+0000 7f9690049700 1 -- 172.21.15.17:0/493924190 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f96880574e0 con 0x7f96880a9da0 2022-08-01T19:36:06.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.454+0000 7f9676ffd700 1 -- 172.21.15.17:0/493924190 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) v1 ==== 100+0+10 (secure 0 0 0) 0x7f9684045980 con 0x7f96880a9da0 2022-08-01T19:36:06.459 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:36:06.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.458+0000 7f9690049700 1 -- 172.21.15.17:0/493924190 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9668053980 msgr2=0x7f9668055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:06.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.458+0000 7f9690049700 1 --2- 172.21.15.17:0/493924190 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9668053980 0x7f9668055e40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f967800c000 tx=0x7f9678008040).stop 2022-08-01T19:36:06.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.458+0000 7f9690049700 1 -- 172.21.15.17:0/493924190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96880a9da0 msgr2=0x7f96880ae0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:06.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.458+0000 7f9690049700 1 --2- 172.21.15.17:0/493924190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96880a9da0 0x7f96880ae0e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f9684001110 tx=0x7f9684007ad0).stop 2022-08-01T19:36:06.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.458+0000 7f9690049700 1 -- 172.21.15.17:0/493924190 shutdown_connections 2022-08-01T19:36:06.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.458+0000 7f9690049700 1 --2- 172.21.15.17:0/493924190 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9668053980 0x7f9668055e40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:06.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.458+0000 7f9690049700 1 --2- 172.21.15.17:0/493924190 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96880b38a0 0x7f9688140f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:06.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.458+0000 7f9690049700 1 --2- 172.21.15.17:0/493924190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96880b0740 0x7f968814fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:06.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.458+0000 7f9690049700 1 --2- 172.21.15.17:0/493924190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96880a9da0 0x7f96880ae0e0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:06.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.458+0000 7f9690049700 1 -- 172.21.15.17:0/493924190 >> 172.21.15.17:0/493924190 conn(0x7f9688060950 msgr2=0x7f968813d380 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:06.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.458+0000 7f9690049700 1 -- 172.21.15.17:0/493924190 shutdown_connections 2022-08-01T19:36:06.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.458+0000 7f9690049700 1 -- 172.21.15.17:0/493924190 wait complete. 2022-08-01T19:36:06.477 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T19:36:06.478 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:36:06.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.574+0000 7f4fd7fff700 1 Processor -- start 2022-08-01T19:36:06.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.574+0000 7f4fd7fff700 1 -- start start 2022-08-01T19:36:06.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.574+0000 7f4fd7fff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f4fd81489f0 0x7f4fd814ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:06.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.574+0000 7f4fd7fff700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fd8067030 con 0x7f4fd80b31d0 2022-08-01T19:36:06.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.574+0000 7f4fd7fff700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fd80b35b0 con 0x7f4fd80a9da0 2022-08-01T19:36:06.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.574+0000 7f4fd7fff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fd8064a70 con 0x7f4fd81489f0 2022-08-01T19:36:06.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.574+0000 7f4fd6ffd700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f4fd80b31d0 0x7f4fd8066020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58912/0 (socket says 172.21.15.17:58912) 2022-08-01T19:36:06.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.574+0000 7f4fd6ffd700 1 -- 172.21.15.17:0/245833031 learned_addr learned my addr 172.21.15.17:0/245833031 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:36:06.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.574+0000 7f4fd77fe700 1 --2- 172.21.15.17:0/245833031 >> v2:172.21.15.77:3300/0 conn(0x7f4fd81489f0 0x7f4fd814ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:06.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.574+0000 7f4fd5ffb700 1 -- 172.21.15.17:0/245833031 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1281067322 0 0) 0x7f4fd8067030 con 0x7f4fd80b31d0 2022-08-01T19:36:06.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.574+0000 7f4fd5ffb700 1 -- 172.21.15.17:0/245833031 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fbc003260 con 0x7f4fd80b31d0 2022-08-01T19:36:06.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.574+0000 7f4fd77fe700 1 -- 172.21.15.17:0/245833031 >> v1:172.21.15.77:6789/0 conn(0x7f4fd80a9da0 legacy=0x7f4fd8060950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:06.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.574+0000 7f4fd77fe700 1 -- 172.21.15.17:0/245833031 >> v1:172.21.15.17:6789/0 conn(0x7f4fd80b31d0 legacy=0x7f4fd8066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:06.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.574+0000 7f4fd77fe700 1 -- 172.21.15.17:0/245833031 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd8066af0 con 0x7f4fd81489f0 2022-08-01T19:36:06.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.574+0000 7f4fd77fe700 1 --2- 172.21.15.17:0/245833031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fd81489f0 0x7f4fd814ade0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f4fd00032f0 tx=0x7f4fd00091d0).ready entity=mon.1 client_cookie=7a9b640b751e93d5 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:06.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.574+0000 7f4fd5ffb700 1 -- 172.21.15.17:0/245833031 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fd000b710 con 0x7f4fd81489f0 2022-08-01T19:36:06.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.574+0000 7f4fd5ffb700 1 -- 172.21.15.17:0/245833031 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd000b870 con 0x7f4fd81489f0 2022-08-01T19:36:06.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.574+0000 7f4fd5ffb700 1 -- 172.21.15.17:0/245833031 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fd000b9e0 con 0x7f4fd81489f0 2022-08-01T19:36:06.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.574+0000 7f4fd7fff700 1 -- 172.21.15.17:0/245833031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fd81489f0 msgr2=0x7f4fd814ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:06.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.574+0000 7f4fd7fff700 1 --2- 172.21.15.17:0/245833031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fd81489f0 0x7f4fd814ade0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f4fd00032f0 tx=0x7f4fd00091d0).stop 2022-08-01T19:36:06.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.574+0000 7f4fd7fff700 1 -- 172.21.15.17:0/245833031 shutdown_connections 2022-08-01T19:36:06.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.574+0000 7f4fd7fff700 1 --2- 172.21.15.17:0/245833031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fd81489f0 0x7f4fd814ade0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:06.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.574+0000 7f4fd7fff700 1 -- 172.21.15.17:0/245833031 >> 172.21.15.17:0/245833031 conn(0x7f4fd8063870 msgr2=0x7f4fd8063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:06.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.574+0000 7f4fd7fff700 1 -- 172.21.15.17:0/245833031 shutdown_connections 2022-08-01T19:36:06.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.574+0000 7f4fd7fff700 1 -- 172.21.15.17:0/245833031 wait complete. 2022-08-01T19:36:06.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.574+0000 7f4fd7fff700 1 Processor -- start 2022-08-01T19:36:06.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.574+0000 7f4fd7fff700 1 -- start start 2022-08-01T19:36:06.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.578+0000 7f4fd7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fd80a9da0 0x7f4fd814c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:06.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.578+0000 7f4fd7fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4fd80b31d0 0x7f4fd81374d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:06.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.578+0000 7f4fd7fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fd81489f0 0x7f4fd8132090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:06.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.578+0000 7f4fd7fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd81318a0 con 0x7f4fd80a9da0 2022-08-01T19:36:06.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.578+0000 7f4fd7fff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd80b4780 con 0x7f4fd81489f0 2022-08-01T19:36:06.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.578+0000 7f4fd7fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd8153a10 con 0x7f4fd80b31d0 2022-08-01T19:36:06.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.578+0000 7f4fd6ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fd80a9da0 0x7f4fd814c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:06.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.578+0000 7f4fd6ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fd80a9da0 0x7f4fd814c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38490/0 (socket says 172.21.15.17:38490) 2022-08-01T19:36:06.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.578+0000 7f4fd6ffd700 1 -- 172.21.15.17:0/83501405 learned_addr learned my addr 172.21.15.17:0/83501405 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:06.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.578+0000 7f4fd67fc700 1 --2- 172.21.15.17:0/83501405 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4fd80b31d0 0x7f4fd81374d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:06.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.578+0000 7f4fd77fe700 1 --2- 172.21.15.17:0/83501405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fd81489f0 0x7f4fd8132090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:06.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.578+0000 7f4fd6ffd700 1 -- 172.21.15.17:0/83501405 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4fd80b31d0 msgr2=0x7f4fd81374d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:06.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.578+0000 7f4fd6ffd700 1 --2- 172.21.15.17:0/83501405 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4fd80b31d0 0x7f4fd81374d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:06.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.578+0000 7f4fd6ffd700 1 -- 172.21.15.17:0/83501405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fd81489f0 msgr2=0x7f4fd8132090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:06.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.578+0000 7f4fd6ffd700 1 --2- 172.21.15.17:0/83501405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fd81489f0 0x7f4fd8132090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:06.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.578+0000 7f4fd6ffd700 1 -- 172.21.15.17:0/83501405 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd0009470 con 0x7f4fd80a9da0 2022-08-01T19:36:06.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.578+0000 7f4fd67fc700 1 --2- 172.21.15.17:0/83501405 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4fd80b31d0 0x7f4fd81374d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:36:06.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.578+0000 7f4fd6ffd700 1 --2- 172.21.15.17:0/83501405 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fd80a9da0 0x7f4fd814c320 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f4fcc001110 tx=0x7f4fcc007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:06.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.578+0000 7f4fc7fff700 1 -- 172.21.15.17:0/83501405 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fcc00ce70 con 0x7f4fd80a9da0 2022-08-01T19:36:06.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.578+0000 7f4fc7fff700 1 -- 172.21.15.17:0/83501405 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fcc0072e0 con 0x7f4fd80a9da0 2022-08-01T19:36:06.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.578+0000 7f4fd7fff700 1 -- 172.21.15.17:0/83501405 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd814c810 con 0x7f4fd80a9da0 2022-08-01T19:36:06.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.578+0000 7f4fc7fff700 1 -- 172.21.15.17:0/83501405 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fcc005800 con 0x7f4fd80a9da0 2022-08-01T19:36:06.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.578+0000 7f4fd7fff700 1 -- 172.21.15.17:0/83501405 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd8143e90 con 0x7f4fd80a9da0 2022-08-01T19:36:06.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.582+0000 7f4fd7fff700 1 -- 172.21.15.17:0/83501405 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fd8153bd0 con 0x7f4fd80a9da0 2022-08-01T19:36:06.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.582+0000 7f4fc7fff700 1 -- 172.21.15.17:0/83501405 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4fcc010070 con 0x7f4fd80a9da0 2022-08-01T19:36:06.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.582+0000 7f4fc7fff700 1 --2- 172.21.15.17:0/83501405 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4fb4053b00 0x7f4fb4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:06.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.582+0000 7f4fc7fff700 1 -- 172.21.15.17:0/83501405 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4fcc06b7a0 con 0x7f4fd80a9da0 2022-08-01T19:36:06.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.582+0000 7f4fd67fc700 1 --2- 172.21.15.17:0/83501405 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4fb4053b00 0x7f4fb4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:06.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.582+0000 7f4fd67fc700 1 --2- 172.21.15.17:0/83501405 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4fb4053b00 0x7f4fb4055fc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f4fc800b000 tx=0x7f4fc8009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:06.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.594+0000 7f4fc7fff700 1 -- 172.21.15.17:0/83501405 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4fcc042500 con 0x7f4fd80a9da0 2022-08-01T19:36:06.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.754+0000 7f4fd7fff700 1 -- 172.21.15.17:0/83501405 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4fd8066d90 con 0x7f4fd80a9da0 2022-08-01T19:36:06.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.754+0000 7f4fc7fff700 1 -- 172.21.15.17:0/83501405 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+2488 (secure 0 0 0) 0x7f4fcc045b20 con 0x7f4fd80a9da0 2022-08-01T19:36:06.760 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:36:06.761 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/109603607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":109603607},{"type":"v1","addr":"172.21.15.17:6833","nonce":109603607}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":16},{"gid":5230,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/2722431349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2722431349},{"type":"v1","addr":"172.21.15.77:6835","nonce":2722431349}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":17}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:36:02.170790+0000","modified":"2022-08-01T19:36:03.173986+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5238},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5238":{"gid":5238,"name":"c","rank":0,"incarnation":17,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/3980246529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":3980246529},{"type":"v1","addr":"172.21.15.17:6835","nonce":3980246529}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-08-01T19:36:06.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.754+0000 7f4fd7fff700 1 -- 172.21.15.17:0/83501405 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4fb4053b00 msgr2=0x7f4fb4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:06.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.758+0000 7f4fd7fff700 1 --2- 172.21.15.17:0/83501405 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4fb4053b00 0x7f4fb4055fc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f4fc800b000 tx=0x7f4fc8009040).stop 2022-08-01T19:36:06.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.758+0000 7f4fd7fff700 1 -- 172.21.15.17:0/83501405 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fd80a9da0 msgr2=0x7f4fd814c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:06.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.758+0000 7f4fd7fff700 1 --2- 172.21.15.17:0/83501405 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fd80a9da0 0x7f4fd814c320 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f4fcc001110 tx=0x7f4fcc007ad0).stop 2022-08-01T19:36:06.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.758+0000 7f4fd7fff700 1 -- 172.21.15.17:0/83501405 shutdown_connections 2022-08-01T19:36:06.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.758+0000 7f4fd7fff700 1 --2- 172.21.15.17:0/83501405 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4fb4053b00 0x7f4fb4055fc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:06.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.758+0000 7f4fd7fff700 1 --2- 172.21.15.17:0/83501405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fd81489f0 0x7f4fd8132090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:06.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.758+0000 7f4fd7fff700 1 --2- 172.21.15.17:0/83501405 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4fd80b31d0 0x7f4fd81374d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:06.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.758+0000 7f4fd7fff700 1 --2- 172.21.15.17:0/83501405 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fd80a9da0 0x7f4fd814c320 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:06.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.758+0000 7f4fd7fff700 1 -- 172.21.15.17:0/83501405 >> 172.21.15.17:0/83501405 conn(0x7f4fd8063870 msgr2=0x7f4fd80b5520 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:06.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.758+0000 7f4fd7fff700 1 -- 172.21.15.17:0/83501405 shutdown_connections 2022-08-01T19:36:06.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:06.758+0000 7f4fd7fff700 1 -- 172.21.15.17:0/83501405 wait complete. 2022-08-01T19:36:06.768 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 18 2022-08-01T19:36:07.075 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:36:02.170790+0000', 'modified': '2022-08-01T19:36:03.173986+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5238}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5238': {'gid': 5238, 'name': 'c', 'rank': 0, 'incarnation': 17, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/3980246529', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 3980246529}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 3980246529}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:36:07.077 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:36:07.078 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T19:36:07.207 INFO:teuthology.orchestra.run.smithi017.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":5238,"want_state":"up:active","state":"up:active","rank_uptime":5.0250817269999999,"mdsmap_epoch":18,"osdmap_epoch":45,"osdmap_epoch_barrier":44,"uptime":9.3972833819999995} 2022-08-01T19:36:07.208 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 5238, "mdsmap_epoch": 18, "osdmap_epoch": 45, "osdmap_epoch_barrier": 44, "rank_uptime": 5.025081727, "state": "up:active", "uptime": 9.397283382, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:36:07.209 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T19:36:07.209 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-08-01T19:36:07.210 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi085.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-08-01T19:36:07.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:36:07.211 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:36:07.219 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:36:07.277 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:36:07.278 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:36:07.288 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:36:07.288 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:36:07.289 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:36:07.290 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:36:07.295 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T19:36:07.296 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:36:07.338 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:36:07.355 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:36:07.358 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:36:07.370 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:36:07.371 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:36:07.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:36:07.372 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:36:07.412 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:2022-08-01T19:36:07.411+0000 7f24b2ca3080 -1 init, newargv = 0x56140235f990 newargc=9ceph-fuse[22581]: starting ceph client 2022-08-01T19:36:07.412 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr: 2022-08-01T19:36:07.423 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:ceph-fuse[22581]: starting fuse 2022-08-01T19:36:08.519 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:36:08.538 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:36:08.539 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:36:08.596 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:36:08.598 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:36:08.599 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:36:08.599 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:36:08.606 INFO:teuthology.orchestra.run.smithi085.stdout:52 2022-08-01T19:36:08.606 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T19:36:08.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:36:08.608 DEBUG:teuthology.orchestra.run.smithi085:> sudo python3 -c ' 2022-08-01T19:36:08.608 DEBUG:teuthology.orchestra.run.smithi085:> import glob 2022-08-01T19:36:08.608 DEBUG:teuthology.orchestra.run.smithi085:> import re 2022-08-01T19:36:08.609 DEBUG:teuthology.orchestra.run.smithi085:> import os 2022-08-01T19:36:08.609 DEBUG:teuthology.orchestra.run.smithi085:> import subprocess 2022-08-01T19:36:08.610 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:36:08.610 DEBUG:teuthology.orchestra.run.smithi085:> def find_socket(client_name): 2022-08-01T19:36:08.611 DEBUG:teuthology.orchestra.run.smithi085:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-08-01T19:36:08.611 DEBUG:teuthology.orchestra.run.smithi085:> files = glob.glob(asok_path) 2022-08-01T19:36:08.611 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:36:08.612 DEBUG:teuthology.orchestra.run.smithi085:> # Given a non-glob path, it better be there 2022-08-01T19:36:08.612 DEBUG:teuthology.orchestra.run.smithi085:> if "*" not in asok_path: 2022-08-01T19:36:08.613 DEBUG:teuthology.orchestra.run.smithi085:> assert(len(files) == 1) 2022-08-01T19:36:08.613 DEBUG:teuthology.orchestra.run.smithi085:> return files[0] 2022-08-01T19:36:08.613 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:36:08.614 DEBUG:teuthology.orchestra.run.smithi085:> for f in files: 2022-08-01T19:36:08.614 DEBUG:teuthology.orchestra.run.smithi085:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T19:36:08.615 DEBUG:teuthology.orchestra.run.smithi085:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T19:36:08.615 DEBUG:teuthology.orchestra.run.smithi085:> return f 2022-08-01T19:36:08.616 DEBUG:teuthology.orchestra.run.smithi085:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T19:36:08.616 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:36:08.616 DEBUG:teuthology.orchestra.run.smithi085:> print(find_socket("client.0")) 2022-08-01T19:36:08.617 DEBUG:teuthology.orchestra.run.smithi085:> ' 2022-08-01T19:36:08.659 INFO:teuthology.orchestra.run.smithi085.stdout:/var/run/ceph/ceph-client.0.22581.asok 2022-08-01T19:36:08.664 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22581.asok 2022-08-01T19:36:08.665 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:36:08.665 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22581.asok status 2022-08-01T19:36:08.799 INFO:teuthology.orchestra.run.smithi085.stdout:{ 2022-08-01T19:36:08.800 INFO:teuthology.orchestra.run.smithi085.stdout: "metadata": { 2022-08-01T19:36:08.800 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T19:36:08.800 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T19:36:08.800 INFO:teuthology.orchestra.run.smithi085.stdout: "entity_id": "0", 2022-08-01T19:36:08.801 INFO:teuthology.orchestra.run.smithi085.stdout: "hostname": "smithi085", 2022-08-01T19:36:08.801 INFO:teuthology.orchestra.run.smithi085.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-08-01T19:36:08.801 INFO:teuthology.orchestra.run.smithi085.stdout: "pid": "22581", 2022-08-01T19:36:08.801 INFO:teuthology.orchestra.run.smithi085.stdout: "root": "/" 2022-08-01T19:36:08.802 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:36:08.802 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_count": 0, 2022-08-01T19:36:08.802 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_pinned_count": 0, 2022-08-01T19:36:08.802 INFO:teuthology.orchestra.run.smithi085.stdout: "id": 5000, 2022-08-01T19:36:08.803 INFO:teuthology.orchestra.run.smithi085.stdout: "inst": { 2022-08-01T19:36:08.803 INFO:teuthology.orchestra.run.smithi085.stdout: "name": { 2022-08-01T19:36:08.803 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "client", 2022-08-01T19:36:08.803 INFO:teuthology.orchestra.run.smithi085.stdout: "num": 5000 2022-08-01T19:36:08.804 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:36:08.804 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2022-08-01T19:36:08.804 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2022-08-01T19:36:08.805 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2022-08-01T19:36:08.805 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 3329677115 2022-08-01T19:36:08.805 INFO:teuthology.orchestra.run.smithi085.stdout: } 2022-08-01T19:36:08.805 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:36:08.806 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2022-08-01T19:36:08.806 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2022-08-01T19:36:08.806 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2022-08-01T19:36:08.806 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 3329677115 2022-08-01T19:36:08.806 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:36:08.807 INFO:teuthology.orchestra.run.smithi085.stdout: "inst_str": "client.5000 172.21.15.85:0/3329677115", 2022-08-01T19:36:08.807 INFO:teuthology.orchestra.run.smithi085.stdout: "addr_str": "172.21.15.85:0/3329677115", 2022-08-01T19:36:08.807 INFO:teuthology.orchestra.run.smithi085.stdout: "inode_count": 1, 2022-08-01T19:36:08.808 INFO:teuthology.orchestra.run.smithi085.stdout: "mds_epoch": 18, 2022-08-01T19:36:08.808 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch": 45, 2022-08-01T19:36:08.808 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch_barrier": 0, 2022-08-01T19:36:08.808 INFO:teuthology.orchestra.run.smithi085.stdout: "blacklisted": false, 2022-08-01T19:36:08.808 INFO:teuthology.orchestra.run.smithi085.stdout: "fs_name": "cephfs" 2022-08-01T19:36:08.809 INFO:teuthology.orchestra.run.smithi085.stdout:} 2022-08-01T19:36:08.810 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:36:08.810 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:36:08.811 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:36:08.817 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:36:08.818 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:36:08.819 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:36:08.819 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:36:08.878 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:36:08.879 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:36:08.880 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:36:08.886 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:36:08.887 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:36:08.888 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-08-01T19:36:08.889 DEBUG:teuthology.orchestra.run.smithi085:> import sys 2022-08-01T19:36:08.889 DEBUG:teuthology.orchestra.run.smithi085:> import time 2022-08-01T19:36:08.889 DEBUG:teuthology.orchestra.run.smithi085:> import os 2022-08-01T19:36:08.890 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:36:08.890 DEBUG:teuthology.orchestra.run.smithi085:> n = 1000 2022-08-01T19:36:08.891 DEBUG:teuthology.orchestra.run.smithi085:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2022-08-01T19:36:08.891 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:36:08.891 DEBUG:teuthology.orchestra.run.smithi085:> if not os.path.exists(os.path.dirname(abs_path)): 2022-08-01T19:36:08.892 DEBUG:teuthology.orchestra.run.smithi085:> os.makedirs(os.path.dirname(abs_path)) 2022-08-01T19:36:08.892 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:36:08.893 DEBUG:teuthology.orchestra.run.smithi085:> for i in range(0, n): 2022-08-01T19:36:08.893 DEBUG:teuthology.orchestra.run.smithi085:> fname = "{0}_{1}".format(abs_path, i) 2022-08-01T19:36:08.893 DEBUG:teuthology.orchestra.run.smithi085:> with open(fname, '"'"'w'"'"') as f: 2022-08-01T19:36:08.894 DEBUG:teuthology.orchestra.run.smithi085:> f.write('"'"'content'"'"') 2022-08-01T19:36:08.894 DEBUG:teuthology.orchestra.run.smithi085:> if True: 2022-08-01T19:36:08.895 DEBUG:teuthology.orchestra.run.smithi085:> f.flush() 2022-08-01T19:36:08.895 DEBUG:teuthology.orchestra.run.smithi085:> os.fsync(f.fileno()) 2022-08-01T19:36:08.895 DEBUG:teuthology.orchestra.run.smithi085:> ' 2022-08-01T19:36:17.087 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:36:17.090 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:36:17.094 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:36:17.096 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:36:20.011 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:36:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.122+0000 7f95d08e9700 1 Processor -- start 2022-08-01T19:36:20.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.122+0000 7f95d08e9700 1 -- start start 2022-08-01T19:36:20.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.122+0000 7f95d08e9700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f95cc0b6a90 0x7f95cc0acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:20.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.122+0000 7f95d08e9700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95cc0b6e70 con 0x7f95cc060aa0 2022-08-01T19:36:20.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.122+0000 7f95d08e9700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95cc061370 con 0x7f95cc0aa760 2022-08-01T19:36:20.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.122+0000 7f95d08e9700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95cc061610 con 0x7f95cc0b6a90 2022-08-01T19:36:20.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.122+0000 7f95ca59c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f95cc0b6a90 0x7f95cc0acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:20.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.122+0000 7f95c3fff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f95cc060aa0 0x7f95cc0aa050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58928/0 (socket says 172.21.15.17:58928) 2022-08-01T19:36:20.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.122+0000 7f95ca59c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f95cc0b6a90 0x7f95cc0acee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38500/0 (socket says 172.21.15.17:38500) 2022-08-01T19:36:20.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.122+0000 7f95c3fff700 1 -- 172.21.15.17:0/313697925 learned_addr learned my addr 172.21.15.17:0/313697925 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:36:20.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.122+0000 7f95ca59c700 1 -- 172.21.15.17:0/313697925 >> v1:172.21.15.77:6789/0 conn(0x7f95cc0aa760 legacy=0x7f95cc13c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:20.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.122+0000 7f95ca59c700 1 -- 172.21.15.17:0/313697925 >> v1:172.21.15.17:6789/0 conn(0x7f95cc060aa0 legacy=0x7f95cc0aa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.122+0000 7f95ca59c700 1 -- 172.21.15.17:0/313697925 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95cc0618b0 con 0x7f95cc0b6a90 2022-08-01T19:36:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.122+0000 7f95ca59c700 1 --2- 172.21.15.17:0/313697925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95cc0b6a90 0x7f95cc0acee0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f95bc002f70 tx=0x7f95bc006e80).ready entity=mon.0 client_cookie=c4abe7e2d15d4932 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:20.134 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.122+0000 7f95c9d9b700 1 -- 172.21.15.17:0/313697925 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95bc014070 con 0x7f95cc0b6a90 2022-08-01T19:36:20.135 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.122+0000 7f95c9d9b700 1 -- 172.21.15.17:0/313697925 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95bc000e90 con 0x7f95cc0b6a90 2022-08-01T19:36:20.135 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.122+0000 7f95c9d9b700 1 -- 172.21.15.17:0/313697925 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95bc00a8b0 con 0x7f95cc0b6a90 2022-08-01T19:36:20.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.122+0000 7f95d08e9700 1 -- 172.21.15.17:0/313697925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95cc0b6a90 msgr2=0x7f95cc0acee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:20.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.122+0000 7f95d08e9700 1 --2- 172.21.15.17:0/313697925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95cc0b6a90 0x7f95cc0acee0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f95bc002f70 tx=0x7f95bc006e80).stop 2022-08-01T19:36:20.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.122+0000 7f95d08e9700 1 -- 172.21.15.17:0/313697925 shutdown_connections 2022-08-01T19:36:20.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.122+0000 7f95d08e9700 1 --2- 172.21.15.17:0/313697925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95cc0b6a90 0x7f95cc0acee0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:20.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.122+0000 7f95d08e9700 1 -- 172.21.15.17:0/313697925 >> 172.21.15.17:0/313697925 conn(0x7f95cc0b3610 msgr2=0x7f95cc0b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:20.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.122+0000 7f95d08e9700 1 -- 172.21.15.17:0/313697925 shutdown_connections 2022-08-01T19:36:20.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.122+0000 7f95d08e9700 1 -- 172.21.15.17:0/313697925 wait complete. 2022-08-01T19:36:20.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.126+0000 7f95d08e9700 1 Processor -- start 2022-08-01T19:36:20.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.126+0000 7f95d08e9700 1 -- start start 2022-08-01T19:36:20.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.126+0000 7f95d08e9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95cc060aa0 0x7f95cc1431a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:20.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.126+0000 7f95d08e9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95cc0aa760 0x7f95cc13bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:20.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.126+0000 7f95d08e9700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f95cc0b6a90 0x7f95cc139ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.126+0000 7f95d08e9700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc145e70 con 0x7f95cc0aa760 2022-08-01T19:36:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.126+0000 7f95d08e9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc1444a0 con 0x7f95cc060aa0 2022-08-01T19:36:20.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.126+0000 7f95d08e9700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc13c190 con 0x7f95cc0b6a90 2022-08-01T19:36:20.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.126+0000 7f95c3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95cc0aa760 0x7f95cc13bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:20.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.126+0000 7f95c3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95cc0aa760 0x7f95cc13bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38508/0 (socket says 172.21.15.17:38508) 2022-08-01T19:36:20.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.126+0000 7f95c3fff700 1 -- 172.21.15.17:0/2390286448 learned_addr learned my addr 172.21.15.17:0/2390286448 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:20.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.126+0000 7f95ca59c700 1 --2- 172.21.15.17:0/2390286448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95cc060aa0 0x7f95cc1431a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:20.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.126+0000 7f95cad9d700 1 --2- 172.21.15.17:0/2390286448 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f95cc0b6a90 0x7f95cc139ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:20.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.126+0000 7f95c3fff700 1 -- 172.21.15.17:0/2390286448 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f95cc0b6a90 msgr2=0x7f95cc139ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:20.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.126+0000 7f95c3fff700 1 --2- 172.21.15.17:0/2390286448 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f95cc0b6a90 0x7f95cc139ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:20.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.126+0000 7f95c3fff700 1 -- 172.21.15.17:0/2390286448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95cc060aa0 msgr2=0x7f95cc1431a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:20.154 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.126+0000 7f95c3fff700 1 --2- 172.21.15.17:0/2390286448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95cc060aa0 0x7f95cc1431a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:20.154 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.126+0000 7f95c3fff700 1 -- 172.21.15.17:0/2390286448 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95bc00c310 con 0x7f95cc0aa760 2022-08-01T19:36:20.154 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.126+0000 7f95c3fff700 1 --2- 172.21.15.17:0/2390286448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95cc0aa760 0x7f95cc13bca0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f95cc060490 tx=0x7f95b4007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:20.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.126+0000 7f95c37fe700 1 -- 172.21.15.17:0/2390286448 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95b40070d0 con 0x7f95cc0aa760 2022-08-01T19:36:20.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.126+0000 7f95c37fe700 1 -- 172.21.15.17:0/2390286448 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b4007230 con 0x7f95cc0aa760 2022-08-01T19:36:20.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.126+0000 7f95d08e9700 1 -- 172.21.15.17:0/2390286448 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95cc145bd0 con 0x7f95cc0aa760 2022-08-01T19:36:20.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.126+0000 7f95c37fe700 1 -- 172.21.15.17:0/2390286448 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95b40055d0 con 0x7f95cc0aa760 2022-08-01T19:36:20.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.126+0000 7f95d08e9700 1 -- 172.21.15.17:0/2390286448 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95cc13a3d0 con 0x7f95cc0aa760 2022-08-01T19:36:20.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.130+0000 7f95d08e9700 1 -- 172.21.15.17:0/2390286448 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95cc145a30 con 0x7f95cc0aa760 2022-08-01T19:36:20.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.130+0000 7f95c37fe700 1 -- 172.21.15.17:0/2390286448 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f95b4010070 con 0x7f95cc0aa760 2022-08-01T19:36:20.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.130+0000 7f95c37fe700 1 --2- 172.21.15.17:0/2390286448 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f95b0053b30 0x7f95b0055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:20.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.130+0000 7f95c37fe700 1 -- 172.21.15.17:0/2390286448 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f95b406b490 con 0x7f95cc0aa760 2022-08-01T19:36:20.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.130+0000 7f95ca59c700 1 --2- 172.21.15.17:0/2390286448 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f95b0053b30 0x7f95b0055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:20.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.130+0000 7f95ca59c700 1 --2- 172.21.15.17:0/2390286448 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f95b0053b30 0x7f95b0055ff0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f95bc01c000 tx=0x7f95cc05ff40).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:20.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.134+0000 7f95c37fe700 1 -- 172.21.15.17:0/2390286448 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f95cc145a30 con 0x7f95cc0aa760 2022-08-01T19:36:20.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.298+0000 7f95d08e9700 1 -- 172.21.15.17:0/2390286448 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95cc145a30 con 0x7f95cc0aa760 2022-08-01T19:36:20.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.298+0000 7f95c37fe700 1 -- 172.21.15.17:0/2390286448 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+2488 (secure 0 0 0) 0x7f95cc145a30 con 0x7f95cc0aa760 2022-08-01T19:36:20.305 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:36:20.305 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/109603607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":109603607},{"type":"v1","addr":"172.21.15.17:6833","nonce":109603607}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":16},{"gid":5230,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/2722431349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2722431349},{"type":"v1","addr":"172.21.15.77:6835","nonce":2722431349}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":17}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:36:02.170790+0000","modified":"2022-08-01T19:36:03.173986+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5238},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5238":{"gid":5238,"name":"c","rank":0,"incarnation":17,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/3980246529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":3980246529},{"type":"v1","addr":"172.21.15.17:6835","nonce":3980246529}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-08-01T19:36:20.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.302+0000 7f95d08e9700 1 -- 172.21.15.17:0/2390286448 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f95b0053b30 msgr2=0x7f95b0055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:20.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.302+0000 7f95d08e9700 1 --2- 172.21.15.17:0/2390286448 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f95b0053b30 0x7f95b0055ff0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f95bc01c000 tx=0x7f95cc05ff40).stop 2022-08-01T19:36:20.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.302+0000 7f95d08e9700 1 -- 172.21.15.17:0/2390286448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95cc0aa760 msgr2=0x7f95cc13bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:20.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.302+0000 7f95d08e9700 1 --2- 172.21.15.17:0/2390286448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95cc0aa760 0x7f95cc13bca0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f95cc060490 tx=0x7f95b4007c30).stop 2022-08-01T19:36:20.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.302+0000 7f95d08e9700 1 -- 172.21.15.17:0/2390286448 shutdown_connections 2022-08-01T19:36:20.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.302+0000 7f95d08e9700 1 --2- 172.21.15.17:0/2390286448 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f95b0053b30 0x7f95b0055ff0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:20.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.302+0000 7f95d08e9700 1 --2- 172.21.15.17:0/2390286448 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f95cc0b6a90 0x7f95cc139ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:20.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.302+0000 7f95d08e9700 1 --2- 172.21.15.17:0/2390286448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95cc0aa760 0x7f95cc13bca0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:20.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.302+0000 7f95d08e9700 1 --2- 172.21.15.17:0/2390286448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95cc060aa0 0x7f95cc1431a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:20.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.302+0000 7f95d08e9700 1 -- 172.21.15.17:0/2390286448 >> 172.21.15.17:0/2390286448 conn(0x7f95cc0b3610 msgr2=0x7f95cc0b48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:20.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.302+0000 7f95d08e9700 1 -- 172.21.15.17:0/2390286448 shutdown_connections 2022-08-01T19:36:20.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.302+0000 7f95d08e9700 1 -- 172.21.15.17:0/2390286448 wait complete. 2022-08-01T19:36:20.311 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 18 2022-08-01T19:36:20.318 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_recall_max_caps 20 2022-08-01T19:36:20.423 INFO:teuthology.orchestra.run.smithi017.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2022-08-01T19:36:20.424 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2022-08-01T19:36:20.425 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:36:20.530 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.526+0000 7f19c5e8b700 1 Processor -- start 2022-08-01T19:36:20.530 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.526+0000 7f19c5e8b700 1 -- start start 2022-08-01T19:36:20.532 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.526+0000 7f19c5e8b700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f19c00b08a0 0x7f19c0142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:20.532 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.526+0000 7f19c5e8b700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19c0143ab0 con 0x7f19c00b3260 2022-08-01T19:36:20.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.526+0000 7f19c5e8b700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19c00b0c80 con 0x7f19c00ad000 2022-08-01T19:36:20.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.526+0000 7f19c5e8b700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19c00ae810 con 0x7f19c00b08a0 2022-08-01T19:36:20.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.526+0000 7f19bffff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f19c00b3260 0x7f19c01433a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58942/0 (socket says 172.21.15.17:58942) 2022-08-01T19:36:20.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.526+0000 7f19bffff700 1 -- 172.21.15.17:0/754227500 learned_addr learned my addr 172.21.15.17:0/754227500 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:36:20.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.526+0000 7f19c4e89700 1 --2- 172.21.15.17:0/754227500 >> v2:172.21.15.77:3300/0 conn(0x7f19c00b08a0 0x7f19c0142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:20.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.526+0000 7f19bf7fe700 1 -- 172.21.15.17:0/754227500 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 567839041 0 0) 0x7f19c0143ab0 con 0x7f19c00b3260 2022-08-01T19:36:20.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.526+0000 7f19bf7fe700 1 -- 172.21.15.17:0/754227500 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19a0003260 con 0x7f19c00b3260 2022-08-01T19:36:20.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.526+0000 7f19c4e89700 1 -- 172.21.15.17:0/754227500 >> v1:172.21.15.77:6789/0 conn(0x7f19c00ad000 legacy=0x7f19c013fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:20.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.526+0000 7f19c4e89700 1 -- 172.21.15.17:0/754227500 >> v1:172.21.15.17:6789/0 conn(0x7f19c00b3260 legacy=0x7f19c01433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:20.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.526+0000 7f19c4e89700 1 -- 172.21.15.17:0/754227500 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c00aeab0 con 0x7f19c00b08a0 2022-08-01T19:36:20.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.526+0000 7f19c4e89700 1 --2- 172.21.15.17:0/754227500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19c00b08a0 0x7f19c0142eb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f19b00032f0 tx=0x7f19b000a370).ready entity=mon.1 client_cookie=a6da4978eeb4e051 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:20.538 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.530+0000 7f19bf7fe700 1 -- 172.21.15.17:0/754227500 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19b000cb70 con 0x7f19c00b08a0 2022-08-01T19:36:20.539 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.530+0000 7f19bf7fe700 1 -- 172.21.15.17:0/754227500 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b000ccd0 con 0x7f19c00b08a0 2022-08-01T19:36:20.539 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.530+0000 7f19bf7fe700 1 -- 172.21.15.17:0/754227500 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19b0010600 con 0x7f19c00b08a0 2022-08-01T19:36:20.539 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.530+0000 7f19c5e8b700 1 -- 172.21.15.17:0/754227500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19c00b08a0 msgr2=0x7f19c0142eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:20.540 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.530+0000 7f19c5e8b700 1 --2- 172.21.15.17:0/754227500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19c00b08a0 0x7f19c0142eb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f19b00032f0 tx=0x7f19b000a370).stop 2022-08-01T19:36:20.540 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.530+0000 7f19c5e8b700 1 -- 172.21.15.17:0/754227500 shutdown_connections 2022-08-01T19:36:20.540 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.530+0000 7f19c5e8b700 1 --2- 172.21.15.17:0/754227500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19c00b08a0 0x7f19c0142eb0 secure :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0x7f19b00032f0 tx=0x7f19b000a370).stop 2022-08-01T19:36:20.540 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.530+0000 7f19c5e8b700 1 -- 172.21.15.17:0/754227500 >> 172.21.15.17:0/754227500 conn(0x7f19c00612d0 msgr2=0x7f19c0062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:20.540 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.530+0000 7f19c5e8b700 1 -- 172.21.15.17:0/754227500 shutdown_connections 2022-08-01T19:36:20.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.530+0000 7f19c5e8b700 1 -- 172.21.15.17:0/754227500 wait complete. 2022-08-01T19:36:20.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.530+0000 7f19c5e8b700 1 Processor -- start 2022-08-01T19:36:20.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.530+0000 7f19c5e8b700 1 -- start start 2022-08-01T19:36:20.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.530+0000 7f19c5e8b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19c00ad000 0x7f19c00670e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:20.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.530+0000 7f19c5e8b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f19c00b3260 0x7f19c0154000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:20.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.530+0000 7f19c5e8b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19c00a01c0 0x7f19c01544f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:20.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.530+0000 7f19c5e8b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c0140c00 con 0x7f19c00a01c0 2022-08-01T19:36:20.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.530+0000 7f19c5e8b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c01505b0 con 0x7f19c00ad000 2022-08-01T19:36:20.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.530+0000 7f19c5e8b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c00a09e0 con 0x7f19c00b3260 2022-08-01T19:36:20.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.530+0000 7f19c568a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19c00a01c0 0x7f19c01544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:20.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.530+0000 7f19c568a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19c00a01c0 0x7f19c01544f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38524/0 (socket says 172.21.15.17:38524) 2022-08-01T19:36:20.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.530+0000 7f19bffff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f19c00b3260 0x7f19c0154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:20.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.530+0000 7f19c568a700 1 -- 172.21.15.17:0/3414072097 learned_addr learned my addr 172.21.15.17:0/3414072097 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:20.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.530+0000 7f19c4e89700 1 --2- 172.21.15.17:0/3414072097 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19c00ad000 0x7f19c00670e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:20.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.534+0000 7f19c568a700 1 -- 172.21.15.17:0/3414072097 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f19c00b3260 msgr2=0x7f19c0154000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:20.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.534+0000 7f19c568a700 1 --2- 172.21.15.17:0/3414072097 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f19c00b3260 0x7f19c0154000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:20.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.534+0000 7f19c568a700 1 -- 172.21.15.17:0/3414072097 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19c00ad000 msgr2=0x7f19c00670e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:20.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.534+0000 7f19c568a700 1 --2- 172.21.15.17:0/3414072097 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19c00ad000 0x7f19c00670e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:20.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.534+0000 7f19c568a700 1 -- 172.21.15.17:0/3414072097 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b000a610 con 0x7f19c00a01c0 2022-08-01T19:36:20.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.534+0000 7f19c568a700 1 --2- 172.21.15.17:0/3414072097 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19c00a01c0 0x7f19c01544f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f19c00b0420 tx=0x7f19b8007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:20.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.534+0000 7f19bdffb700 1 -- 172.21.15.17:0/3414072097 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19b8012400 con 0x7f19c00a01c0 2022-08-01T19:36:20.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.534+0000 7f19bdffb700 1 -- 172.21.15.17:0/3414072097 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b8012560 con 0x7f19c00a01c0 2022-08-01T19:36:20.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.534+0000 7f19c5e8b700 1 -- 172.21.15.17:0/3414072097 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c00a0c80 con 0x7f19c00a01c0 2022-08-01T19:36:20.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.534+0000 7f19bdffb700 1 -- 172.21.15.17:0/3414072097 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f19b80128c0 con 0x7f19c00a01c0 2022-08-01T19:36:20.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.534+0000 7f19c5e8b700 1 -- 172.21.15.17:0/3414072097 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c00677f0 con 0x7f19c00a01c0 2022-08-01T19:36:20.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.534+0000 7f19c5e8b700 1 -- 172.21.15.17:0/3414072097 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19c00574e0 con 0x7f19c00a01c0 2022-08-01T19:36:20.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.534+0000 7f19bdffb700 1 -- 172.21.15.17:0/3414072097 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f19b8019030 con 0x7f19c00a01c0 2022-08-01T19:36:20.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.538+0000 7f19bdffb700 1 --2- 172.21.15.17:0/3414072097 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f19a4053a70 0x7f19a4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:20.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.538+0000 7f19bdffb700 1 -- 172.21.15.17:0/3414072097 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f19b806b2f0 con 0x7f19c00a01c0 2022-08-01T19:36:20.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.538+0000 7f19c4e89700 1 --2- 172.21.15.17:0/3414072097 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f19a4053a70 0x7f19a4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:20.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.538+0000 7f19c4e89700 1 --2- 172.21.15.17:0/3414072097 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f19a4053a70 0x7f19a4055f30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f19b001b000 tx=0x7f19c00ae730).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:20.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.546+0000 7f19bdffb700 1 -- 172.21.15.17:0/3414072097 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f19b80420b0 con 0x7f19c00a01c0 2022-08-01T19:36:20.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.706+0000 7f19c5e8b700 1 -- 172.21.15.17:0/3414072097 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19c00b2e00 con 0x7f19c00a01c0 2022-08-01T19:36:20.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.706+0000 7f19bdffb700 1 -- 172.21.15.17:0/3414072097 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+2488 (secure 0 0 0) 0x7f19b80456d0 con 0x7f19c00a01c0 2022-08-01T19:36:20.711 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:36:20.712 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/109603607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":109603607},{"type":"v1","addr":"172.21.15.17:6833","nonce":109603607}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":16},{"gid":5230,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/2722431349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2722431349},{"type":"v1","addr":"172.21.15.77:6835","nonce":2722431349}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":17}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:36:02.170790+0000","modified":"2022-08-01T19:36:03.173986+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5238},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5238":{"gid":5238,"name":"c","rank":0,"incarnation":17,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/3980246529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":3980246529},{"type":"v1","addr":"172.21.15.17:6835","nonce":3980246529}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-08-01T19:36:20.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.706+0000 7f19c5e8b700 1 -- 172.21.15.17:0/3414072097 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f19a4053a70 msgr2=0x7f19a4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:20.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.706+0000 7f19c5e8b700 1 --2- 172.21.15.17:0/3414072097 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f19a4053a70 0x7f19a4055f30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f19b001b000 tx=0x7f19c00ae730).stop 2022-08-01T19:36:20.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.706+0000 7f19c5e8b700 1 -- 172.21.15.17:0/3414072097 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19c00a01c0 msgr2=0x7f19c01544f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:20.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.706+0000 7f19c5e8b700 1 --2- 172.21.15.17:0/3414072097 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19c00a01c0 0x7f19c01544f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f19c00b0420 tx=0x7f19b8007bc0).stop 2022-08-01T19:36:20.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.706+0000 7f19c5e8b700 1 -- 172.21.15.17:0/3414072097 shutdown_connections 2022-08-01T19:36:20.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.706+0000 7f19c5e8b700 1 --2- 172.21.15.17:0/3414072097 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f19a4053a70 0x7f19a4055f30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:20.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.706+0000 7f19c5e8b700 1 --2- 172.21.15.17:0/3414072097 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19c00a01c0 0x7f19c01544f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:20.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.706+0000 7f19c5e8b700 1 --2- 172.21.15.17:0/3414072097 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f19c00b3260 0x7f19c0154000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:20.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.706+0000 7f19c5e8b700 1 --2- 172.21.15.17:0/3414072097 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19c00ad000 0x7f19c00670e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:20.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.706+0000 7f19c5e8b700 1 -- 172.21.15.17:0/3414072097 >> 172.21.15.17:0/3414072097 conn(0x7f19c00612d0 msgr2=0x7f19c00b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:20.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.706+0000 7f19c5e8b700 1 -- 172.21.15.17:0/3414072097 shutdown_connections 2022-08-01T19:36:20.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.710+0000 7f19c5e8b700 1 -- 172.21.15.17:0/3414072097 wait complete. 2022-08-01T19:36:20.717 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 18 2022-08-01T19:36:20.725 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_recall_max_decay_threshold 400 2022-08-01T19:36:20.827 INFO:teuthology.orchestra.run.smithi017.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2022-08-01T19:36:20.828 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2022-08-01T19:36:20.829 INFO:tasks.cephfs.fuse_mount:Killing ceph-fuse connection on ubuntu@smithi085.front.sepia.ceph.com... 2022-08-01T19:36:20.848 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:36:20.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.950+0000 7fe1848ec700 1 Processor -- start 2022-08-01T19:36:20.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.950+0000 7fe1848ec700 1 -- start start 2022-08-01T19:36:20.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.954+0000 7fe1848ec700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1800b0630 0x7fe1800adec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:20.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.954+0000 7fe1848ec700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fe1800aa2b0 0x7fe1800ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:20.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.954+0000 7fe1848ec700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe1800b0a10 0x7fe1801438a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:20.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.954+0000 7fe1848ec700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1800b2c80 con 0x7fe1800b0a10 2022-08-01T19:36:20.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.954+0000 7fe1848ec700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1800b2900 con 0x7fe1800aa2b0 2022-08-01T19:36:20.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.954+0000 7fe1848ec700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1800b2eb0 con 0x7fe1800b0630 2022-08-01T19:36:20.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.954+0000 7fe17ed9d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe1800b0a10 0x7fe1801438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:20.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.954+0000 7fe17ed9d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe1800b0a10 0x7fe1801438a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38532/0 (socket says 172.21.15.17:38532) 2022-08-01T19:36:20.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.954+0000 7fe17ed9d700 1 -- 172.21.15.17:0/2214678493 learned_addr learned my addr 172.21.15.17:0/2214678493 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:20.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.954+0000 7fe17e59c700 1 --2- 172.21.15.17:0/2214678493 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1800b0630 0x7fe1800adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:20.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.954+0000 7fe177fff700 1 --2- 172.21.15.17:0/2214678493 >> v2:172.21.15.77:3300/0 conn(0x7fe1800aa2b0 0x7fe1800ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:20.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.954+0000 7fe17ed9d700 1 -- 172.21.15.17:0/2214678493 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1800b0630 msgr2=0x7fe1800adec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:36:20.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.954+0000 7fe17ed9d700 1 --2- 172.21.15.17:0/2214678493 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1800b0630 0x7fe1800adec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:20.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.954+0000 7fe17ed9d700 1 -- 172.21.15.17:0/2214678493 >> v2:172.21.15.77:3300/0 conn(0x7fe1800aa2b0 msgr2=0x7fe1800ae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:36:20.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.954+0000 7fe17ed9d700 1 --2- 172.21.15.17:0/2214678493 >> v2:172.21.15.77:3300/0 conn(0x7fe1800aa2b0 0x7fe1800ae3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:20.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.954+0000 7fe17ed9d700 1 -- 172.21.15.17:0/2214678493 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1800a9d00 con 0x7fe1800b0a10 2022-08-01T19:36:20.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.954+0000 7fe177fff700 1 --2- 172.21.15.17:0/2214678493 >> v2:172.21.15.77:3300/0 conn(0x7fe1800aa2b0 0x7fe1800ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:36:20.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.954+0000 7fe17ed9d700 1 --2- 172.21.15.17:0/2214678493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1800b0a10 0x7fe1801438a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7fe1780032f0 tx=0x7fe178009290).ready entity=mon.0 client_cookie=71f44c40f3a50aec server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:20.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.954+0000 7fe17dd9b700 1 -- 172.21.15.17:0/2214678493 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe17800f8a0 con 0x7fe1800b0a10 2022-08-01T19:36:20.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.954+0000 7fe17dd9b700 1 -- 172.21.15.17:0/2214678493 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17800fa00 con 0x7fe1800b0a10 2022-08-01T19:36:20.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.954+0000 7fe17dd9b700 1 -- 172.21.15.17:0/2214678493 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe17800fd60 con 0x7fe1800b0a10 2022-08-01T19:36:20.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.954+0000 7fe1848ec700 1 -- 172.21.15.17:0/2214678493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1800b0a10 msgr2=0x7fe1801438a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:20.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.954+0000 7fe1848ec700 1 --2- 172.21.15.17:0/2214678493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1800b0a10 0x7fe1801438a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7fe1780032f0 tx=0x7fe178009290).stop 2022-08-01T19:36:20.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.954+0000 7fe1848ec700 1 -- 172.21.15.17:0/2214678493 shutdown_connections 2022-08-01T19:36:20.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.954+0000 7fe1848ec700 1 --2- 172.21.15.17:0/2214678493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1800b0a10 0x7fe1801438a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:20.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.954+0000 7fe1848ec700 1 --2- 172.21.15.17:0/2214678493 >> v2:172.21.15.77:3300/0 conn(0x7fe1800aa2b0 0x7fe1800ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:20.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.954+0000 7fe1848ec700 1 --2- 172.21.15.17:0/2214678493 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1800b0630 0x7fe1800adec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:20.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.954+0000 7fe1848ec700 1 -- 172.21.15.17:0/2214678493 >> 172.21.15.17:0/2214678493 conn(0x7fe180060950 msgr2=0x7fe180060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:20.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.954+0000 7fe1848ec700 1 -- 172.21.15.17:0/2214678493 shutdown_connections 2022-08-01T19:36:20.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.954+0000 7fe1848ec700 1 -- 172.21.15.17:0/2214678493 wait complete. 2022-08-01T19:36:20.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.958+0000 7fe1848ec700 1 Processor -- start 2022-08-01T19:36:20.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.958+0000 7fe1848ec700 1 -- start start 2022-08-01T19:36:20.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.958+0000 7fe1848ec700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1800aa2b0 0x7fe18013d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:20.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.958+0000 7fe1848ec700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1800b0630 0x7fe1800676e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:20.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.958+0000 7fe1848ec700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1800b0a10 0x7fe1801416c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:20.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.958+0000 7fe1848ec700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe180145240 con 0x7fe1800aa2b0 2022-08-01T19:36:20.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.958+0000 7fe1848ec700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe180141bb0 con 0x7fe1800b0630 2022-08-01T19:36:20.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.958+0000 7fe1848ec700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe180063a60 con 0x7fe1800b0a10 2022-08-01T19:36:20.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.958+0000 7fe17e59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1800aa2b0 0x7fe18013d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:20.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.958+0000 7fe17e59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1800aa2b0 0x7fe18013d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38536/0 (socket says 172.21.15.17:38536) 2022-08-01T19:36:20.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.958+0000 7fe17e59c700 1 -- 172.21.15.17:0/4283928534 learned_addr learned my addr 172.21.15.17:0/4283928534 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:20.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.958+0000 7fe177fff700 1 --2- 172.21.15.17:0/4283928534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1800b0630 0x7fe1800676e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:20.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.958+0000 7fe17ed9d700 1 --2- 172.21.15.17:0/4283928534 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1800b0a10 0x7fe1801416c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:20.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.958+0000 7fe17e59c700 1 -- 172.21.15.17:0/4283928534 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1800b0a10 msgr2=0x7fe1801416c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:20.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.958+0000 7fe17e59c700 1 --2- 172.21.15.17:0/4283928534 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1800b0a10 0x7fe1801416c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:20.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.958+0000 7fe17e59c700 1 -- 172.21.15.17:0/4283928534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1800b0630 msgr2=0x7fe1800676e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:20.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.958+0000 7fe17e59c700 1 --2- 172.21.15.17:0/4283928534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1800b0630 0x7fe1800676e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:20.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.958+0000 7fe17e59c700 1 -- 172.21.15.17:0/4283928534 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe16800b180 con 0x7fe1800aa2b0 2022-08-01T19:36:20.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.958+0000 7fe177fff700 1 --2- 172.21.15.17:0/4283928534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1800b0630 0x7fe1800676e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:36:20.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.958+0000 7fe17e59c700 1 --2- 172.21.15.17:0/4283928534 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1800aa2b0 0x7fe18013d720 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7fe17000c000 tx=0x7fe170006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:20.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.958+0000 7fe1777fe700 1 -- 172.21.15.17:0/4283928534 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe170010780 con 0x7fe1800aa2b0 2022-08-01T19:36:20.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.958+0000 7fe1777fe700 1 -- 172.21.15.17:0/4283928534 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1700108e0 con 0x7fe1800aa2b0 2022-08-01T19:36:20.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.958+0000 7fe1848ec700 1 -- 172.21.15.17:0/4283928534 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe178009530 con 0x7fe1800aa2b0 2022-08-01T19:36:20.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.958+0000 7fe1777fe700 1 -- 172.21.15.17:0/4283928534 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe170010c40 con 0x7fe1800aa2b0 2022-08-01T19:36:20.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.958+0000 7fe1848ec700 1 -- 172.21.15.17:0/4283928534 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe180145470 con 0x7fe1800aa2b0 2022-08-01T19:36:20.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.962+0000 7fe1848ec700 1 -- 172.21.15.17:0/4283928534 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1800af480 con 0x7fe1800aa2b0 2022-08-01T19:36:20.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.962+0000 7fe1777fe700 1 -- 172.21.15.17:0/4283928534 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe17001c460 con 0x7fe1800aa2b0 2022-08-01T19:36:20.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.962+0000 7fe1777fe700 1 --2- 172.21.15.17:0/4283928534 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe164053a70 0x7fe164055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:20.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.962+0000 7fe1777fe700 1 -- 172.21.15.17:0/4283928534 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe17006c450 con 0x7fe1800aa2b0 2022-08-01T19:36:20.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.962+0000 7fe177fff700 1 --2- 172.21.15.17:0/4283928534 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe164053a70 0x7fe164055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:20.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.962+0000 7fe177fff700 1 --2- 172.21.15.17:0/4283928534 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe164053a70 0x7fe164055f30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fe16800b000 tx=0x7fe168006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:20.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:20.970+0000 7fe1777fe700 1 -- 172.21.15.17:0/4283928534 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe1700431b0 con 0x7fe1800aa2b0 2022-08-01T19:36:21.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.134+0000 7fe1848ec700 1 -- 172.21.15.17:0/4283928534 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe180063bf0 con 0x7fe1800aa2b0 2022-08-01T19:36:21.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.134+0000 7fe1777fe700 1 -- 172.21.15.17:0/4283928534 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+2488 (secure 0 0 0) 0x7fe1700467d0 con 0x7fe1800aa2b0 2022-08-01T19:36:21.141 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:36:21.142 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/109603607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":109603607},{"type":"v1","addr":"172.21.15.17:6833","nonce":109603607}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":16},{"gid":5230,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/2722431349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2722431349},{"type":"v1","addr":"172.21.15.77:6835","nonce":2722431349}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":17}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:36:02.170790+0000","modified":"2022-08-01T19:36:03.173986+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5238},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5238":{"gid":5238,"name":"c","rank":0,"incarnation":17,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/3980246529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":3980246529},{"type":"v1","addr":"172.21.15.17:6835","nonce":3980246529}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-08-01T19:36:21.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.138+0000 7fe1848ec700 1 -- 172.21.15.17:0/4283928534 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe164053a70 msgr2=0x7fe164055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:21.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.138+0000 7fe1848ec700 1 --2- 172.21.15.17:0/4283928534 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe164053a70 0x7fe164055f30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fe16800b000 tx=0x7fe168006df0).stop 2022-08-01T19:36:21.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.138+0000 7fe1848ec700 1 -- 172.21.15.17:0/4283928534 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1800aa2b0 msgr2=0x7fe18013d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:21.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.138+0000 7fe1848ec700 1 --2- 172.21.15.17:0/4283928534 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1800aa2b0 0x7fe18013d720 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7fe17000c000 tx=0x7fe170006df0).stop 2022-08-01T19:36:21.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.138+0000 7fe1848ec700 1 -- 172.21.15.17:0/4283928534 shutdown_connections 2022-08-01T19:36:21.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.138+0000 7fe1848ec700 1 --2- 172.21.15.17:0/4283928534 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe164053a70 0x7fe164055f30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:21.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.138+0000 7fe1848ec700 1 --2- 172.21.15.17:0/4283928534 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1800b0a10 0x7fe1801416c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:21.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.138+0000 7fe1848ec700 1 --2- 172.21.15.17:0/4283928534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1800b0630 0x7fe1800676e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:21.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.138+0000 7fe1848ec700 1 --2- 172.21.15.17:0/4283928534 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1800aa2b0 0x7fe18013d720 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:21.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.138+0000 7fe1848ec700 1 -- 172.21.15.17:0/4283928534 >> 172.21.15.17:0/4283928534 conn(0x7fe180060950 msgr2=0x7fe180060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:21.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.138+0000 7fe1848ec700 1 -- 172.21.15.17:0/4283928534 shutdown_connections 2022-08-01T19:36:21.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.138+0000 7fe1848ec700 1 -- 172.21.15.17:0/4283928534 wait complete. 2022-08-01T19:36:21.151 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 18 2022-08-01T19:36:21.153 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c cache drop 2022-08-01T19:36:21.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.258+0000 7f7a3359e700 1 Processor -- start 2022-08-01T19:36:21.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.258+0000 7f7a3359e700 1 -- start start 2022-08-01T19:36:21.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.258+0000 7f7a3359e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7a340ad750 0x7f7a34061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:21.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.258+0000 7f7a3359e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f7a341405d0 0x7f7a34140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:21.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.258+0000 7f7a3359e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a340aee80 con 0x7f7a340614b0 2022-08-01T19:36:21.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.258+0000 7f7a3359e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a340b70b0 con 0x7f7a341405d0 2022-08-01T19:36:21.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.258+0000 7f7a3359e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a340af0b0 con 0x7f7a340ad750 2022-08-01T19:36:21.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.258+0000 7f7a31d9b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7a340ad750 0x7f7a34061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:21.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.258+0000 7f7a32d9d700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f7a341405d0 0x7f7a34140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:21.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.258+0000 7f7a31d9b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7a340ad750 0x7f7a34061890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:34642/0 (socket says 172.21.15.17:34642) 2022-08-01T19:36:21.266 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.258+0000 7f7a3259c700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7f7a340614b0 0x7f7a340b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:47538/0 (socket says 172.21.15.17:47538) 2022-08-01T19:36:21.266 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.258+0000 7f7a31d9b700 1 -- 172.21.15.17:0/3097685462 learned_addr learned my addr 172.21.15.17:0/3097685462 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:21.266 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.258+0000 7f7a31d9b700 1 -- 172.21.15.17:0/3097685462 >> v2:172.21.15.77:3300/0 conn(0x7f7a341405d0 msgr2=0x7f7a34140a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:36:21.266 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.258+0000 7f7a31d9b700 1 --2- 172.21.15.17:0/3097685462 >> v2:172.21.15.77:3300/0 conn(0x7f7a341405d0 0x7f7a34140a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:21.266 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.258+0000 7f7a31d9b700 1 -- 172.21.15.17:0/3097685462 >> v1:172.21.15.77:6789/0 conn(0x7f7a340614b0 legacy=0x7f7a340b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:21.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.258+0000 7f7a31d9b700 1 -- 172.21.15.17:0/3097685462 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a340af350 con 0x7f7a340ad750 2022-08-01T19:36:21.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.258+0000 7f7a31d9b700 1 --2- 172.21.15.17:0/3097685462 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7a340ad750 0x7f7a34061890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f7a24002f70 tx=0x7f7a24006eb0).ready entity=mon.2 client_cookie=8f7ac719cd623a3c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:21.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.258+0000 7f7a3159a700 1 -- 172.21.15.17:0/3097685462 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a2400d570 con 0x7f7a340ad750 2022-08-01T19:36:21.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.258+0000 7f7a3159a700 1 -- 172.21.15.17:0/3097685462 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a24000e90 con 0x7f7a340ad750 2022-08-01T19:36:21.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.258+0000 7f7a3159a700 1 -- 172.21.15.17:0/3097685462 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a2400d900 con 0x7f7a340ad750 2022-08-01T19:36:21.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.258+0000 7f7a3359e700 1 -- 172.21.15.17:0/3097685462 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7a340ad750 msgr2=0x7f7a34061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:21.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.258+0000 7f7a3359e700 1 --2- 172.21.15.17:0/3097685462 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7a340ad750 0x7f7a34061890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f7a24002f70 tx=0x7f7a24006eb0).stop 2022-08-01T19:36:21.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.258+0000 7f7a3359e700 1 -- 172.21.15.17:0/3097685462 shutdown_connections 2022-08-01T19:36:21.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.258+0000 7f7a3359e700 1 --2- 172.21.15.17:0/3097685462 >> v2:172.21.15.77:3300/0 conn(0x7f7a341405d0 0x7f7a34140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:21.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.258+0000 7f7a3359e700 1 --2- 172.21.15.17:0/3097685462 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7a340ad750 0x7f7a34061890 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:21.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.258+0000 7f7a3359e700 1 -- 172.21.15.17:0/3097685462 >> 172.21.15.17:0/3097685462 conn(0x7f7a34066f70 msgr2=0x7f7a34067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:21.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.258+0000 7f7a3359e700 1 -- 172.21.15.17:0/3097685462 shutdown_connections 2022-08-01T19:36:21.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.258+0000 7f7a3359e700 1 -- 172.21.15.17:0/3097685462 wait complete. 2022-08-01T19:36:21.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a3359e700 1 Processor -- start 2022-08-01T19:36:21.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a3359e700 1 -- start start 2022-08-01T19:36:21.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a3359e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7a340614b0 0x7f7a34152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:21.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a3359e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a340ad750 0x7f7a340b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:21.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a3359e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a341405d0 0x7f7a34155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:21.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a3359e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a34152690 con 0x7f7a340ad750 2022-08-01T19:36:21.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a3359e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a340b19f0 con 0x7f7a341405d0 2022-08-01T19:36:21.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a3359e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a34152880 con 0x7f7a340614b0 2022-08-01T19:36:21.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a31d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a340ad750 0x7f7a340b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:21.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a31d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a340ad750 0x7f7a340b8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38550/0 (socket says 172.21.15.17:38550) 2022-08-01T19:36:21.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a31d9b700 1 -- 172.21.15.17:0/37209743 learned_addr learned my addr 172.21.15.17:0/37209743 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:21.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a3259c700 1 --2- 172.21.15.17:0/37209743 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7a340614b0 0x7f7a34152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:21.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a32d9d700 1 --2- 172.21.15.17:0/37209743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a341405d0 0x7f7a34155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:21.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a31d9b700 1 -- 172.21.15.17:0/37209743 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7a340614b0 msgr2=0x7f7a34152fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:21.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a31d9b700 1 --2- 172.21.15.17:0/37209743 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7a340614b0 0x7f7a34152fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:21.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a31d9b700 1 -- 172.21.15.17:0/37209743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a341405d0 msgr2=0x7f7a34155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:21.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a31d9b700 1 --2- 172.21.15.17:0/37209743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a341405d0 0x7f7a34155c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:21.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a31d9b700 1 -- 172.21.15.17:0/37209743 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a2400b1e0 con 0x7f7a340ad750 2022-08-01T19:36:21.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a31d9b700 1 --2- 172.21.15.17:0/37209743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a340ad750 0x7f7a340b8b20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f7a24002f70 tx=0x7f7a24015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:21.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a237fe700 1 -- 172.21.15.17:0/37209743 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a240068b0 con 0x7f7a340ad750 2022-08-01T19:36:21.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a237fe700 1 -- 172.21.15.17:0/37209743 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2400de30 con 0x7f7a340ad750 2022-08-01T19:36:21.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a3359e700 1 -- 172.21.15.17:0/37209743 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a34152b20 con 0x7f7a340ad750 2022-08-01T19:36:21.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a237fe700 1 -- 172.21.15.17:0/37209743 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a24011d70 con 0x7f7a340ad750 2022-08-01T19:36:21.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a3359e700 1 -- 172.21.15.17:0/37209743 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a341534b0 con 0x7f7a340ad750 2022-08-01T19:36:21.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a237fe700 1 -- 172.21.15.17:0/37209743 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7a24025030 con 0x7f7a340ad750 2022-08-01T19:36:21.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a42ffb740 1 Processor -- start 2022-08-01T19:36:21.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a42ffb740 1 -- start start 2022-08-01T19:36:21.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a237fe700 1 --2- 172.21.15.17:0/37209743 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7a100539e0 0x7f7a10055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:21.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a237fe700 1 -- 172.21.15.17:0/37209743 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7a2406d240 con 0x7f7a340ad750 2022-08-01T19:36:21.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a42ffb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x14ace00 0x14abe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:21.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a42ffb740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x14ad1e0 0x14ac380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:21.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a42ffb740 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x14df920 0x14dfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:21.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a42ffb740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x140a320 con 0x14ad1e0 2022-08-01T19:36:21.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a42ffb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x141dc50 con 0x14ace00 2022-08-01T19:36:21.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a42ffb740 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x13fe1f0 con 0x14df920 2022-08-01T19:36:21.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a32d9d700 1 --2- 172.21.15.17:0/37209743 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7a100539e0 0x7f7a10055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:21.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a31d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x14ad1e0 0x14ac380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:21.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a31d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x14ad1e0 0x14ac380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38558/0 (socket says 172.21.15.17:38558) 2022-08-01T19:36:21.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a31d9b700 1 -- 172.21.15.17:0/556457799 learned_addr learned my addr 172.21.15.17:0/556457799 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:21.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a3259c700 1 --2- 172.21.15.17:0/556457799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x14ace00 0x14abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:21.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a32d9d700 1 --2- 172.21.15.17:0/556457799 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x14df920 0x14dfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:21.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a31d9b700 1 -- 172.21.15.17:0/556457799 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x14df920 msgr2=0x14dfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:36:21.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a31d9b700 1 --2- 172.21.15.17:0/556457799 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x14df920 0x14dfdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:21.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a31d9b700 1 -- 172.21.15.17:0/556457799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x14ace00 msgr2=0x14abe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:36:21.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a31d9b700 1 --2- 172.21.15.17:0/556457799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x14ace00 0x14abe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:21.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a31d9b700 1 -- 172.21.15.17:0/556457799 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1455ec0 con 0x14ad1e0 2022-08-01T19:36:21.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a31d9b700 1 --2- 172.21.15.17:0/556457799 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x14ad1e0 0x14ac380 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x14b0770 tx=0x7f7a24011000).ready entity=mon.0 client_cookie=63281d0e30783e00 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:21.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a32d9d700 1 --2- 172.21.15.17:0/37209743 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7a100539e0 0x7f7a10055ea0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f7a2c00c000 tx=0x7f7a2c0091d0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:21.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a3359e700 1 -- 172.21.15.17:0/556457799 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a24047920 con 0x14ad1e0 2022-08-01T19:36:21.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a3359e700 1 -- 172.21.15.17:0/556457799 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a240476f0 con 0x14ad1e0 2022-08-01T19:36:21.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.266+0000 7f7a3359e700 1 -- 172.21.15.17:0/556457799 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a24047510 con 0x14ad1e0 2022-08-01T19:36:21.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.270+0000 7f7a42ffb740 1 -- 172.21.15.17:0/556457799 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x14ad1e0 msgr2=0x14ac380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:21.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.270+0000 7f7a42ffb740 1 --2- 172.21.15.17:0/556457799 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x14ad1e0 0x14ac380 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x14b0770 tx=0x7f7a24011000).stop 2022-08-01T19:36:21.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.270+0000 7f7a42ffb740 1 -- 172.21.15.17:0/556457799 shutdown_connections 2022-08-01T19:36:21.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.270+0000 7f7a42ffb740 1 --2- 172.21.15.17:0/556457799 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x14df920 0x14dfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:21.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.270+0000 7f7a42ffb740 1 --2- 172.21.15.17:0/556457799 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x14ad1e0 0x14ac380 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:21.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.270+0000 7f7a42ffb740 1 --2- 172.21.15.17:0/556457799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x14ace00 0x14abe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:21.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.270+0000 7f7a42ffb740 1 -- 172.21.15.17:0/556457799 >> 172.21.15.17:0/556457799 conn(0x14aa180 msgr2=0x14aae20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:21.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.270+0000 7f7a42ffb740 1 -- 172.21.15.17:0/556457799 shutdown_connections 2022-08-01T19:36:21.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.270+0000 7f7a42ffb740 1 -- 172.21.15.17:0/556457799 wait complete. 2022-08-01T19:36:21.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.270+0000 7f7a42ffb740 1 -- start start 2022-08-01T19:36:21.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.270+0000 7f7a42ffb740 1 Processor -- start 2022-08-01T19:36:21.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.270+0000 7f7a42ffb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x14ad1e0 0x14e4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:21.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.270+0000 7f7a42ffb740 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x14df920 0x14e4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:21.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.270+0000 7f7a42ffb740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x14e53f0 0x14f5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:21.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.270+0000 7f7a42ffb740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x14e6a40 con 0x14e53f0 2022-08-01T19:36:21.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.270+0000 7f7a42ffb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x14e5f30 con 0x14ad1e0 2022-08-01T19:36:21.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.270+0000 7f7a42ffb740 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x14f2e60 con 0x14df920 2022-08-01T19:36:21.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.270+0000 7f7a42ffb740 10 client.0 ll_register_callbacks cb 0x1403310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-08-01T19:36:21.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.270+0000 7f7a32d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x14e53f0 0x14f5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:21.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.270+0000 7f7a31d9b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x14df920 0x14e4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:21.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.270+0000 7f7a3259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x14ad1e0 0x14e4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:21.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.270+0000 7f7a31d9b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x14df920 0x14e4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:34664/0 (socket says 172.21.15.17:34664) 2022-08-01T19:36:21.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.270+0000 7f7a31d9b700 1 -- 172.21.15.17:0/1505141535 learned_addr learned my addr 172.21.15.17:0/1505141535 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:21.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.270+0000 7f7a32d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x14e53f0 0x14f5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38566/0 (socket says 172.21.15.17:38566) 2022-08-01T19:36:21.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a32d9d700 1 -- 172.21.15.17:0/1505141535 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x14df920 msgr2=0x14e4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:36:21.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a32d9d700 1 --2- 172.21.15.17:0/1505141535 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x14df920 0x14e4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:21.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a32d9d700 1 -- 172.21.15.17:0/1505141535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x14ad1e0 msgr2=0x14e4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:36:21.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a32d9d700 1 --2- 172.21.15.17:0/1505141535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x14ad1e0 0x14e4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:21.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a32d9d700 1 -- 172.21.15.17:0/1505141535 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a341534b0 con 0x14e53f0 2022-08-01T19:36:21.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a3259c700 1 --2- 172.21.15.17:0/1505141535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x14ad1e0 0x14e4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:36:21.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a32d9d700 1 --2- 172.21.15.17:0/1505141535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x14e53f0 0x14f5430 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f7a2c00a000 tx=0x7f7a2c01a000).ready entity=mon.0 client_cookie=231cb734580d3858 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:21.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a0d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2022-08-01T19:36:21.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a0d7fa700 1 -- 172.21.15.17:0/1505141535 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a2c016520 con 0x14e53f0 2022-08-01T19:36:21.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a42ffb740 1 -- 172.21.15.17:0/1505141535 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1413c70 con 0x14e53f0 2022-08-01T19:36:21.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a0d7fa700 1 -- 172.21.15.17:0/1505141535 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7a2c016680 con 0x14e53f0 2022-08-01T19:36:21.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a0d7fa700 1 -- 172.21.15.17:0/1505141535 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a2c016840 con 0x14e53f0 2022-08-01T19:36:21.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a42ffb740 10 client.5421 fetch_fsmap learned FSMap version 18 2022-08-01T19:36:21.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a42ffb740 1 -- 172.21.15.17:0/1505141535 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x13f0950 con 0x14e53f0 2022-08-01T19:36:21.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a0d7fa700 1 -- 172.21.15.17:0/1505141535 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1413c70 con 0x14e53f0 2022-08-01T19:36:21.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a0d7fa700 1 -- 172.21.15.17:0/1505141535 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 18) v1 ==== 1434+0+0 (secure 0 0 0) 0x7f7a2c0292f0 con 0x14e53f0 2022-08-01T19:36:21.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a42ffb740 10 client.5421 fetch_fsmap finished waiting for FSMap version 18 2022-08-01T19:36:21.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a42ffb740 10 client.5421 resolve_mds: resolved ID 'c' to GID 5238 2022-08-01T19:36:21.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a42ffb740 20 client.5421 populate_metadata read hostname 'smithi017' 2022-08-01T19:36:21.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a42ffb740 1 --2- 172.21.15.17:0/1505141535 >> [v2:172.21.15.17:6834/3980246529,v1:172.21.15.17:6835/3980246529] conn(0x14f5920 0x14f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:21.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a42ffb740 4 client.5421 mds_command: new command op to 5238 tid=0[{"prefix": "get_command_descriptions"}] 2022-08-01T19:36:21.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a42ffb740 1 -- 172.21.15.17:0/1505141535 --> [v2:172.21.15.17:6834/3980246529,v1:172.21.15.17:6835/3980246529] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x140bfe0 con 0x14f5920 2022-08-01T19:36:21.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a3259c700 1 --2- 172.21.15.17:0/1505141535 >> [v2:172.21.15.17:6834/3980246529,v1:172.21.15.17:6835/3980246529] conn(0x14f5920 0x14f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:21.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a0d7fa700 10 client.5421 ms_handle_connect on v2:172.21.15.17:6834/3980246529 2022-08-01T19:36:21.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a3259c700 1 --2- 172.21.15.17:0/1505141535 >> [v2:172.21.15.17:6834/3980246529,v1:172.21.15.17:6835/3980246529] conn(0x14f5920 0x14f5d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=f375a9cc15ea44d5 server_cookie=8d6d5d3fd857111d in_seq=0 out_seq=0 2022-08-01T19:36:21.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a0d7fa700 1 -- 172.21.15.17:0/1505141535 <== mds.0 v2:172.21.15.17:6834/3980246529 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x140bfe0 con 0x14f5920 2022-08-01T19:36:21.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a0d7fa700 10 client.5421 handle_command_reply: tid=0 2022-08-01T19:36:21.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a42ffb740 1 client.5421 shutdown 2022-08-01T19:36:21.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a42ffb740 1 -- 172.21.15.17:0/1505141535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x14e53f0 msgr2=0x14f5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:21.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a42ffb740 1 --2- 172.21.15.17:0/1505141535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x14e53f0 0x14f5430 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f7a2c00a000 tx=0x7f7a2c01a000).stop 2022-08-01T19:36:21.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a42ffb740 1 -- 172.21.15.17:0/1505141535 shutdown_connections 2022-08-01T19:36:21.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a42ffb740 1 --2- 172.21.15.17:0/1505141535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x14e53f0 0x14f5430 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:21.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a42ffb740 1 --2- 172.21.15.17:0/1505141535 >> [v2:172.21.15.17:6834/3980246529,v1:172.21.15.17:6835/3980246529] conn(0x14f5920 0x14f5d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:21.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a42ffb740 1 --2- 172.21.15.17:0/1505141535 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x14df920 0x14e4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:21.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a0d7fa700 0 client.5421 ms_handle_reset on v2:172.21.15.17:6834/3980246529 2022-08-01T19:36:21.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a42ffb740 1 --2- 172.21.15.17:0/1505141535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x14ad1e0 0x14e4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:21.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a42ffb740 1 -- 172.21.15.17:0/1505141535 >> 172.21.15.17:0/1505141535 conn(0x14ace00 msgr2=0x14ab630 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:21.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a42ffb740 1 -- 172.21.15.17:0/1505141535 shutdown_connections 2022-08-01T19:36:21.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.274+0000 7f7a42ffb740 1 -- 172.21.15.17:0/1505141535 wait complete. 2022-08-01T19:36:21.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.278+0000 7f7a42ffb740 20 client.5421 trim_cache size 0 max 16384 2022-08-01T19:36:21.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.290+0000 7f7a42ffb740 1 Processor -- start 2022-08-01T19:36:21.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.290+0000 7f7a42ffb740 1 -- start start 2022-08-01T19:36:21.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.290+0000 7f7a42ffb740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x14ad1e0 0x14e4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:21.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.290+0000 7f7a42ffb740 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x14df920 0x14deb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:21.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.290+0000 7f7a42ffb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x14e53f0 0x14df040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:21.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.290+0000 7f7a42ffb740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x14ae070 con 0x14ad1e0 2022-08-01T19:36:21.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.290+0000 7f7a42ffb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x144ad70 con 0x14e53f0 2022-08-01T19:36:21.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.290+0000 7f7a42ffb740 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x141f530 con 0x14df920 2022-08-01T19:36:21.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.290+0000 7f7a3259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x14ad1e0 0x14e4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:21.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.290+0000 7f7a3259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x14ad1e0 0x14e4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38570/0 (socket says 172.21.15.17:38570) 2022-08-01T19:36:21.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.290+0000 7f7a3259c700 1 -- 172.21.15.17:0/4132461574 learned_addr learned my addr 172.21.15.17:0/4132461574 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:21.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.290+0000 7f7a32d9d700 1 --2- 172.21.15.17:0/4132461574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x14e53f0 0x14df040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:21.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.290+0000 7f7a31d9b700 1 --2- 172.21.15.17:0/4132461574 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x14df920 0x14deb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:21.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.294+0000 7f7a3259c700 1 -- 172.21.15.17:0/4132461574 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x14df920 msgr2=0x14deb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:36:21.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.294+0000 7f7a3259c700 1 --2- 172.21.15.17:0/4132461574 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x14df920 0x14deb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:21.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.294+0000 7f7a3259c700 1 -- 172.21.15.17:0/4132461574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x14e53f0 msgr2=0x14df040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:36:21.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.294+0000 7f7a3259c700 1 --2- 172.21.15.17:0/4132461574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x14e53f0 0x14df040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:21.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.294+0000 7f7a3259c700 1 -- 172.21.15.17:0/4132461574 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x140d550 con 0x14ad1e0 2022-08-01T19:36:21.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.294+0000 7f7a3259c700 1 --2- 172.21.15.17:0/4132461574 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x14ad1e0 0x14e4880 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f7a2800f000 tx=0x7f7a2801c280).ready entity=mon.0 client_cookie=c0c0d970e706c239 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:21.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.294+0000 7f7a0f7fe700 1 -- 172.21.15.17:0/4132461574 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a28004b10 con 0x14ad1e0 2022-08-01T19:36:21.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.294+0000 7f7a0f7fe700 1 -- 172.21.15.17:0/4132461574 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2801c040 con 0x14ad1e0 2022-08-01T19:36:21.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.294+0000 7f7a0f7fe700 1 -- 172.21.15.17:0/4132461574 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a2801e620 con 0x14ad1e0 2022-08-01T19:36:21.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.294+0000 7f7a42ffb740 1 -- 172.21.15.17:0/4132461574 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x14ad1e0 msgr2=0x14e4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:21.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.294+0000 7f7a42ffb740 1 --2- 172.21.15.17:0/4132461574 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x14ad1e0 0x14e4880 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f7a2800f000 tx=0x7f7a2801c280).stop 2022-08-01T19:36:21.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.294+0000 7f7a42ffb740 1 -- 172.21.15.17:0/4132461574 shutdown_connections 2022-08-01T19:36:21.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.294+0000 7f7a42ffb740 1 --2- 172.21.15.17:0/4132461574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x14e53f0 0x14df040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:21.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.294+0000 7f7a42ffb740 1 --2- 172.21.15.17:0/4132461574 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x14df920 0x14deb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:21.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.294+0000 7f7a42ffb740 1 --2- 172.21.15.17:0/4132461574 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x14ad1e0 0x14e4880 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:21.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.294+0000 7f7a42ffb740 1 -- 172.21.15.17:0/4132461574 >> 172.21.15.17:0/4132461574 conn(0x14ace00 msgr2=0x14aade0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:21.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.294+0000 7f7a42ffb740 1 -- 172.21.15.17:0/4132461574 shutdown_connections 2022-08-01T19:36:21.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.294+0000 7f7a42ffb740 1 -- 172.21.15.17:0/4132461574 wait complete. 2022-08-01T19:36:21.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.294+0000 7f7a42ffb740 1 -- start start 2022-08-01T19:36:21.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.294+0000 7f7a42ffb740 1 Processor -- start 2022-08-01T19:36:21.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a42ffb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x14ad1e0 0x1546460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:21.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a42ffb740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x14df920 0x14ea520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:21.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a42ffb740 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x14e53f0 0x14abfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:21.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a42ffb740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x14e9a00 con 0x14df920 2022-08-01T19:36:21.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a42ffb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x14df0a0 con 0x14ad1e0 2022-08-01T19:36:21.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a42ffb740 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x14e8c70 con 0x14e53f0 2022-08-01T19:36:21.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a42ffb740 10 client.0 ll_register_callbacks cb 0x14aca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-08-01T19:36:21.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a31d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x14df920 0x14ea520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:21.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a32d9d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x14e53f0 0x14abfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:21.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a3259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x14ad1e0 0x1546460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:21.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a31d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x14df920 0x14ea520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38580/0 (socket says 172.21.15.17:38580) 2022-08-01T19:36:21.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a31d9b700 1 -- 172.21.15.17:0/1603707104 learned_addr learned my addr 172.21.15.17:0/1603707104 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:21.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a31d9b700 1 -- 172.21.15.17:0/1603707104 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x14e53f0 msgr2=0x14abfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:36:21.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a31d9b700 1 --2- 172.21.15.17:0/1603707104 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x14e53f0 0x14abfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:21.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a31d9b700 1 -- 172.21.15.17:0/1603707104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x14ad1e0 msgr2=0x1546460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:36:21.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a31d9b700 1 --2- 172.21.15.17:0/1603707104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x14ad1e0 0x1546460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:21.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a31d9b700 1 -- 172.21.15.17:0/1603707104 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a340aee80 con 0x14df920 2022-08-01T19:36:21.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a0e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2022-08-01T19:36:21.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a31d9b700 1 --2- 172.21.15.17:0/1603707104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x14df920 0x14ea520 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x1545880 tx=0x7f7a2402e000).ready entity=mon.0 client_cookie=22aa5b26b1e3b37c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:21.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a0e7fc700 1 -- 172.21.15.17:0/1603707104 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a24047920 con 0x14df920 2022-08-01T19:36:21.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a0e7fc700 1 -- 172.21.15.17:0/1603707104 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7a24047b50 con 0x14df920 2022-08-01T19:36:21.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a42ffb740 1 -- 172.21.15.17:0/1603707104 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a240476f0 con 0x14df920 2022-08-01T19:36:21.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a0e7fc700 1 -- 172.21.15.17:0/1603707104 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a24046b10 con 0x14df920 2022-08-01T19:36:21.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a42ffb740 10 client.5427 fetch_fsmap learned FSMap version 18 2022-08-01T19:36:21.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a42ffb740 1 -- 172.21.15.17:0/1603707104 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x1415e50 con 0x14df920 2022-08-01T19:36:21.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a0e7fc700 1 -- 172.21.15.17:0/1603707104 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a240476f0 con 0x14df920 2022-08-01T19:36:21.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a0e7fc700 1 -- 172.21.15.17:0/1603707104 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 18) v1 ==== 1434+0+0 (secure 0 0 0) 0x7f7a2401b030 con 0x14df920 2022-08-01T19:36:21.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a42ffb740 10 client.5427 fetch_fsmap finished waiting for FSMap version 18 2022-08-01T19:36:21.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a42ffb740 10 client.5427 resolve_mds: resolved ID 'c' to GID 5238 2022-08-01T19:36:21.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a42ffb740 20 client.5427 populate_metadata read hostname 'smithi017' 2022-08-01T19:36:21.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a42ffb740 1 --2- 172.21.15.17:0/1603707104 >> [v2:172.21.15.17:6834/3980246529,v1:172.21.15.17:6835/3980246529] conn(0x1542af0 0x1542ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:21.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a42ffb740 4 client.5427 mds_command: new command op to 5238 tid=0[{"prefix": "cache drop"}] 2022-08-01T19:36:21.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a42ffb740 1 -- 172.21.15.17:0/1603707104 --> [v2:172.21.15.17:6834/3980246529,v1:172.21.15.17:6835/3980246529] -- command(tid 0: {"prefix": "cache drop"}) v1 -- 0x1301590 con 0x1542af0 2022-08-01T19:36:21.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a3259c700 1 --2- 172.21.15.17:0/1603707104 >> [v2:172.21.15.17:6834/3980246529,v1:172.21.15.17:6835/3980246529] conn(0x1542af0 0x1542ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:21.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a0e7fc700 10 client.5427 ms_handle_connect on v2:172.21.15.17:6834/3980246529 2022-08-01T19:36:21.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:21.298+0000 7f7a3259c700 1 --2- 172.21.15.17:0/1603707104 >> [v2:172.21.15.17:6834/3980246529,v1:172.21.15.17:6835/3980246529] conn(0x1542af0 0x1542ed0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=88f2de3d5df00e4c server_cookie=c27adf7a97e3a0e3 in_seq=0 out_seq=0 2022-08-01T19:36:32.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.510+0000 7f7a0e7fc700 1 -- 172.21.15.17:0/1603707104 <== mds.0 v2:172.21.15.17:6834/3980246529 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+380 (crc 0 0 0) 0x1301590 con 0x1542af0 2022-08-01T19:36:32.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.510+0000 7f7a0e7fc700 10 client.5427 handle_command_reply: tid=0 2022-08-01T19:36:32.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.510+0000 7f7a42ffb740 1 client.5427 shutdown 2022-08-01T19:36:32.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.510+0000 7f7a42ffb740 1 -- 172.21.15.17:0/1603707104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x14df920 msgr2=0x14ea520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:32.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.510+0000 7f7a42ffb740 1 --2- 172.21.15.17:0/1603707104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x14df920 0x14ea520 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x1545880 tx=0x7f7a2402e000).stop 2022-08-01T19:36:32.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.510+0000 7f7a42ffb740 1 -- 172.21.15.17:0/1603707104 shutdown_connections 2022-08-01T19:36:32.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.510+0000 7f7a42ffb740 1 --2- 172.21.15.17:0/1603707104 >> [v2:172.21.15.17:6834/3980246529,v1:172.21.15.17:6835/3980246529] conn(0x1542af0 0x1542ed0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:32.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.510+0000 7f7a42ffb740 1 --2- 172.21.15.17:0/1603707104 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x14e53f0 0x14abfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:32.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.510+0000 7f7a42ffb740 1 --2- 172.21.15.17:0/1603707104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x14df920 0x14ea520 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:32.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.510+0000 7f7a0e7fc700 0 client.5427 ms_handle_reset on v2:172.21.15.17:6834/3980246529 2022-08-01T19:36:32.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.510+0000 7f7a42ffb740 1 --2- 172.21.15.17:0/1603707104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x14ad1e0 0x1546460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:32.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.510+0000 7f7a42ffb740 1 -- 172.21.15.17:0/1603707104 >> 172.21.15.17:0/1603707104 conn(0x14ace00 msgr2=0x14e4670 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:32.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.510+0000 7f7a42ffb740 1 -- 172.21.15.17:0/1603707104 shutdown_connections 2022-08-01T19:36:32.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.510+0000 7f7a42ffb740 1 -- 172.21.15.17:0/1603707104 wait complete. 2022-08-01T19:36:32.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.510+0000 7f7a42ffb740 20 client.5427 trim_cache size 0 max 16384 2022-08-01T19:36:32.517 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:36:32.518 INFO:teuthology.orchestra.run.smithi017.stdout: "client_recall": { 2022-08-01T19:36:32.518 INFO:teuthology.orchestra.run.smithi017.stdout: "return_code": 0, 2022-08-01T19:36:32.518 INFO:teuthology.orchestra.run.smithi017.stdout: "message": "(0) Success", 2022-08-01T19:36:32.519 INFO:teuthology.orchestra.run.smithi017.stdout: "recalled": 220 2022-08-01T19:36:32.519 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:36:32.519 INFO:teuthology.orchestra.run.smithi017.stdout: "flush_journal": { 2022-08-01T19:36:32.519 INFO:teuthology.orchestra.run.smithi017.stdout: "return_code": 0, 2022-08-01T19:36:32.520 INFO:teuthology.orchestra.run.smithi017.stdout: "message": "" 2022-08-01T19:36:32.520 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:36:32.520 INFO:teuthology.orchestra.run.smithi017.stdout: "trim_cache": { 2022-08-01T19:36:32.520 INFO:teuthology.orchestra.run.smithi017.stdout: "trimmed": 0 2022-08-01T19:36:32.521 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:36:32.521 INFO:teuthology.orchestra.run.smithi017.stdout: "cache": { 2022-08-01T19:36:32.521 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": { 2022-08-01T19:36:32.521 INFO:teuthology.orchestra.run.smithi017.stdout: "items": 19983, 2022-08-01T19:36:32.522 INFO:teuthology.orchestra.run.smithi017.stdout: "bytes": 3099232 2022-08-01T19:36:32.522 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:36:32.522 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:36:32.523 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 11.206896333 2022-08-01T19:36:32.523 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:36:32.526 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.510+0000 7f7a3359e700 1 -- 172.21.15.17:0/37209743 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7a100539e0 msgr2=0x7f7a10055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:32.526 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.510+0000 7f7a3359e700 1 --2- 172.21.15.17:0/37209743 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7a100539e0 0x7f7a10055ea0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f7a2c00c000 tx=0x7f7a2c0091d0).stop 2022-08-01T19:36:32.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.510+0000 7f7a3359e700 1 -- 172.21.15.17:0/37209743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a340ad750 msgr2=0x7f7a340b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:32.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.510+0000 7f7a3359e700 1 --2- 172.21.15.17:0/37209743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a340ad750 0x7f7a340b8b20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f7a24002f70 tx=0x7f7a24015000).stop 2022-08-01T19:36:32.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.510+0000 7f7a3359e700 1 -- 172.21.15.17:0/37209743 shutdown_connections 2022-08-01T19:36:32.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.510+0000 7f7a3359e700 1 --2- 172.21.15.17:0/37209743 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7a100539e0 0x7f7a10055ea0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:32.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.510+0000 7f7a3359e700 1 --2- 172.21.15.17:0/37209743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a341405d0 0x7f7a34155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:32.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.510+0000 7f7a3359e700 1 --2- 172.21.15.17:0/37209743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a340ad750 0x7f7a340b8b20 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:32.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.510+0000 7f7a3359e700 1 --2- 172.21.15.17:0/37209743 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7a340614b0 0x7f7a34152fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:32.529 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.510+0000 7f7a3359e700 1 -- 172.21.15.17:0/37209743 >> 172.21.15.17:0/37209743 conn(0x7f7a34066f70 msgr2=0x7f7a3405cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:32.529 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.510+0000 7f7a3359e700 1 -- 172.21.15.17:0/37209743 shutdown_connections 2022-08-01T19:36:32.529 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.510+0000 7f7a3359e700 1 -- 172.21.15.17:0/37209743 wait complete. 2022-08-01T19:36:32.530 INFO:tasks.cephfs.fuse_mount:Cleaning up killed ceph-fuse connection 2022-08-01T19:36:32.530 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:36:32.531 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:36:32.531 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:36:32.539 INFO:teuthology.orchestra.run.smithi085.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2022-08-01T19:36:32.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:36:32.541 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2022-08-01T19:36:32.542 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi085.front.sepia.ceph.com... 2022-08-01T19:36:32.543 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T19:36:32.544 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:36:32.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:36:32.566 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:36:32.566 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:36:32.574 INFO:teuthology.orchestra.run.smithi085.stdout:ext2/ext3 2022-08-01T19:36:32.575 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T19:36:32.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T19:36:32.576 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:36:32.623 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:36:32.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.726+0000 7efdd759e700 1 Processor -- start 2022-08-01T19:36:32.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.726+0000 7efdd759e700 1 -- start start 2022-08-01T19:36:32.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.730+0000 7efdd759e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7efdd80b31d0 0x7efdd8066020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:32.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.730+0000 7efdd759e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efdd80a9da0 0x7efdd8060950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:32.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.730+0000 7efdd759e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdd8067030 con 0x7efdd81434b0 2022-08-01T19:36:32.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.730+0000 7efdd759e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdd8064a70 con 0x7efdd80a9da0 2022-08-01T19:36:32.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.730+0000 7efdd759e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdd8061f60 con 0x7efdd80b31d0 2022-08-01T19:36:32.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.730+0000 7efdd5d9b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efdd80a9da0 0x7efdd8060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:32.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.730+0000 7efdd5d9b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efdd80a9da0 0x7efdd8060950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38588/0 (socket says 172.21.15.17:38588) 2022-08-01T19:36:32.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.730+0000 7efdd5d9b700 1 -- 172.21.15.17:0/249102175 learned_addr learned my addr 172.21.15.17:0/249102175 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:32.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.730+0000 7efdd659c700 1 --2- 172.21.15.17:0/249102175 >> v2:172.21.15.77:3300/0 conn(0x7efdd80b31d0 0x7efdd8066020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:32.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.730+0000 7efdd5d9b700 1 -- 172.21.15.17:0/249102175 >> v2:172.21.15.77:3300/0 conn(0x7efdd80b31d0 msgr2=0x7efdd8066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:36:32.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.730+0000 7efdd5d9b700 1 --2- 172.21.15.17:0/249102175 >> v2:172.21.15.77:3300/0 conn(0x7efdd80b31d0 0x7efdd8066020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:32.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.730+0000 7efdd5d9b700 1 -- 172.21.15.17:0/249102175 >> v1:172.21.15.77:6789/0 conn(0x7efdd81434b0 legacy=0x7efdd8143900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:32.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.730+0000 7efdd5d9b700 1 -- 172.21.15.17:0/249102175 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd80b35b0 con 0x7efdd80a9da0 2022-08-01T19:36:32.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.730+0000 7efdd5d9b700 1 --2- 172.21.15.17:0/249102175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdd80a9da0 0x7efdd8060950 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7efdcc002f70 tx=0x7efdcc006eb0).ready entity=mon.0 client_cookie=4637445893f459b4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:32.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.730+0000 7efdd559a700 1 -- 172.21.15.17:0/249102175 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efdcc00d5c0 con 0x7efdd80a9da0 2022-08-01T19:36:32.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.730+0000 7efdd559a700 1 -- 172.21.15.17:0/249102175 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdcc000e90 con 0x7efdd80a9da0 2022-08-01T19:36:32.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.730+0000 7efdd559a700 1 -- 172.21.15.17:0/249102175 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efdcc00d950 con 0x7efdd80a9da0 2022-08-01T19:36:32.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.734+0000 7efdd759e700 1 -- 172.21.15.17:0/249102175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdd80a9da0 msgr2=0x7efdd8060950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:32.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.734+0000 7efdd759e700 1 --2- 172.21.15.17:0/249102175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdd80a9da0 0x7efdd8060950 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7efdcc002f70 tx=0x7efdcc006eb0).stop 2022-08-01T19:36:32.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.734+0000 7efdd759e700 1 -- 172.21.15.17:0/249102175 shutdown_connections 2022-08-01T19:36:32.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.734+0000 7efdd759e700 1 --2- 172.21.15.17:0/249102175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdd80a9da0 0x7efdd8060950 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:32.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.734+0000 7efdd759e700 1 --2- 172.21.15.17:0/249102175 >> v2:172.21.15.77:3300/0 conn(0x7efdd80b31d0 0x7efdd8066020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:32.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.734+0000 7efdd759e700 1 -- 172.21.15.17:0/249102175 >> 172.21.15.17:0/249102175 conn(0x7efdd8063870 msgr2=0x7efdd8063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:32.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.734+0000 7efdd759e700 1 -- 172.21.15.17:0/249102175 shutdown_connections 2022-08-01T19:36:32.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.734+0000 7efdd759e700 1 -- 172.21.15.17:0/249102175 wait complete. 2022-08-01T19:36:32.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.734+0000 7efdd759e700 1 Processor -- start 2022-08-01T19:36:32.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.734+0000 7efdd759e700 1 -- start start 2022-08-01T19:36:32.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.734+0000 7efdd759e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdd80a9da0 0x7efdd81466b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:32.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.734+0000 7efdd759e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efdd80b31d0 0x7efdd81422f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:32.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.734+0000 7efdd759e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efdd81434b0 0x7efdd81522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:32.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.734+0000 7efdd759e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd80a99f0 con 0x7efdd80a9da0 2022-08-01T19:36:32.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.734+0000 7efdd759e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd80ab6b0 con 0x7efdd80b31d0 2022-08-01T19:36:32.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.734+0000 7efdd759e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdd80aeec0 con 0x7efdd81434b0 2022-08-01T19:36:32.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.734+0000 7efdd659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdd80a9da0 0x7efdd81466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:32.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.734+0000 7efdd659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdd80a9da0 0x7efdd81466b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38592/0 (socket says 172.21.15.17:38592) 2022-08-01T19:36:32.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.734+0000 7efdd659c700 1 -- 172.21.15.17:0/3579268159 learned_addr learned my addr 172.21.15.17:0/3579268159 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:32.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.734+0000 7efdd5d9b700 1 --2- 172.21.15.17:0/3579268159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efdd80b31d0 0x7efdd81422f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:32.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.738+0000 7efdd6d9d700 1 --2- 172.21.15.17:0/3579268159 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efdd81434b0 0x7efdd81522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:32.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.738+0000 7efdd659c700 1 -- 172.21.15.17:0/3579268159 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efdd81434b0 msgr2=0x7efdd81522e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:32.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.738+0000 7efdd659c700 1 --2- 172.21.15.17:0/3579268159 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efdd81434b0 0x7efdd81522e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:32.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.738+0000 7efdd659c700 1 -- 172.21.15.17:0/3579268159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efdd80b31d0 msgr2=0x7efdd81422f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.738+0000 7efdd659c700 1 --2- 172.21.15.17:0/3579268159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efdd80b31d0 0x7efdd81422f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.738+0000 7efdd659c700 1 -- 172.21.15.17:0/3579268159 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdcc00b1e0 con 0x7efdd80a9da0 2022-08-01T19:36:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.738+0000 7efdd659c700 1 --2- 172.21.15.17:0/3579268159 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdd80a9da0 0x7efdd81466b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7efdc800c000 tx=0x7efdc8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.738+0000 7efdc77fe700 1 -- 172.21.15.17:0/3579268159 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efdc8010780 con 0x7efdd80a9da0 2022-08-01T19:36:32.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.738+0000 7efdc77fe700 1 -- 172.21.15.17:0/3579268159 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdc80108e0 con 0x7efdd80a9da0 2022-08-01T19:36:32.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.738+0000 7efdd759e700 1 -- 172.21.15.17:0/3579268159 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdd80ab8e0 con 0x7efdd80a9da0 2022-08-01T19:36:32.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.738+0000 7efdc77fe700 1 -- 172.21.15.17:0/3579268159 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efdc8010c40 con 0x7efdd80a9da0 2022-08-01T19:36:32.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.738+0000 7efdd759e700 1 -- 172.21.15.17:0/3579268159 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdd80ac2c0 con 0x7efdd80a9da0 2022-08-01T19:36:32.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.738+0000 7efdd759e700 1 -- 172.21.15.17:0/3579268159 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdd8066af0 con 0x7efdd80a9da0 2022-08-01T19:36:32.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.738+0000 7efdc77fe700 1 -- 172.21.15.17:0/3579268159 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7efdc801c460 con 0x7efdd80a9da0 2022-08-01T19:36:32.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.738+0000 7efdc77fe700 1 --2- 172.21.15.17:0/3579268159 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efdb4053a70 0x7efdb4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:32.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.738+0000 7efdc77fe700 1 -- 172.21.15.17:0/3579268159 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7efdc806c560 con 0x7efdd80a9da0 2022-08-01T19:36:32.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.742+0000 7efdd5d9b700 1 --2- 172.21.15.17:0/3579268159 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efdb4053a70 0x7efdb4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:32.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.742+0000 7efdd5d9b700 1 --2- 172.21.15.17:0/3579268159 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efdb4053a70 0x7efdb4055f30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7efdcc01c000 tx=0x7efdcc015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:32.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.754+0000 7efdc77fe700 1 -- 172.21.15.17:0/3579268159 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efdc8043130 con 0x7efdd80a9da0 2022-08-01T19:36:32.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.874+0000 7efdd759e700 1 -- 172.21.15.17:0/3579268159 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efdd80574e0 con 0x7efdd80a9da0 2022-08-01T19:36:32.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.878+0000 7efdc77fe700 1 -- 172.21.15.17:0/3579268159 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16271 (secure 0 0 0) 0x7efdc8046750 con 0x7efdd80a9da0 2022-08-01T19:36:32.883 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:36:32.883 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":45,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:36:03.174642+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:36:00.565402+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2022-08-01T19:36:01.813338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:36:32.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.878+0000 7efdd759e700 1 -- 172.21.15.17:0/3579268159 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efdb4053a70 msgr2=0x7efdb4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:32.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.878+0000 7efdd759e700 1 --2- 172.21.15.17:0/3579268159 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efdb4053a70 0x7efdb4055f30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7efdcc01c000 tx=0x7efdcc015000).stop 2022-08-01T19:36:32.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.878+0000 7efdd759e700 1 -- 172.21.15.17:0/3579268159 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdd80a9da0 msgr2=0x7efdd81466b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:32.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.878+0000 7efdd759e700 1 --2- 172.21.15.17:0/3579268159 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdd80a9da0 0x7efdd81466b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7efdc800c000 tx=0x7efdc8006df0).stop 2022-08-01T19:36:32.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.878+0000 7efdd759e700 1 -- 172.21.15.17:0/3579268159 shutdown_connections 2022-08-01T19:36:32.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.878+0000 7efdd759e700 1 --2- 172.21.15.17:0/3579268159 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efdb4053a70 0x7efdb4055f30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:32.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.878+0000 7efdd759e700 1 --2- 172.21.15.17:0/3579268159 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efdd81434b0 0x7efdd81522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:32.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.878+0000 7efdd759e700 1 --2- 172.21.15.17:0/3579268159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efdd80b31d0 0x7efdd81422f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:32.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.878+0000 7efdd759e700 1 --2- 172.21.15.17:0/3579268159 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdd80a9da0 0x7efdd81466b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:32.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.878+0000 7efdd759e700 1 -- 172.21.15.17:0/3579268159 >> 172.21.15.17:0/3579268159 conn(0x7efdd8063870 msgr2=0x7efdd80ad000 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:32.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.878+0000 7efdd759e700 1 -- 172.21.15.17:0/3579268159 shutdown_connections 2022-08-01T19:36:32.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:32.878+0000 7efdd759e700 1 -- 172.21.15.17:0/3579268159 wait complete. 2022-08-01T19:36:32.895 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T19:36:00.565402+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T19:36:01.813338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T19:36:32.896 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:36:33.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.006+0000 7fd3f289b700 1 Processor -- start 2022-08-01T19:36:33.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.006+0000 7fd3f289b700 1 -- start start 2022-08-01T19:36:33.008 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.006+0000 7fd3f289b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd3ec0b08a0 0x7fd3ec142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:33.008 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.006+0000 7fd3f289b700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fd3ec0b3260 0x7fd3ec1433a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:33.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.006+0000 7fd3f289b700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3ec0b0c80 con 0x7fd3ec0ad000 2022-08-01T19:36:33.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.006+0000 7fd3f289b700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3ec0ae810 con 0x7fd3ec0b3260 2022-08-01T19:36:33.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.006+0000 7fd3f289b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec0aeab0 con 0x7fd3ec0b08a0 2022-08-01T19:36:33.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.006+0000 7fd3f209a700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd3ec0ad000 0x7fd3ec05f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59030/0 (socket says 172.21.15.17:59030) 2022-08-01T19:36:33.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.006+0000 7fd3f209a700 1 -- 172.21.15.17:0/833953759 learned_addr learned my addr 172.21.15.17:0/833953759 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:36:33.010 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.006+0000 7fd3f1098700 1 --2- 172.21.15.17:0/833953759 >> v2:172.21.15.77:3300/0 conn(0x7fd3ec0b3260 0x7fd3ec1433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:33.010 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.006+0000 7fd3f1899700 1 --2- 172.21.15.17:0/833953759 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd3ec0b08a0 0x7fd3ec142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:33.010 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.006+0000 7fd3f0897700 1 -- 172.21.15.17:0/833953759 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3214016499 0 0) 0x7fd3ec0b0c80 con 0x7fd3ec0ad000 2022-08-01T19:36:33.010 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.006+0000 7fd3f0897700 1 -- 172.21.15.17:0/833953759 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3d4003260 con 0x7fd3ec0ad000 2022-08-01T19:36:33.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.006+0000 7fd3f1098700 1 -- 172.21.15.17:0/833953759 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd3ec0b08a0 msgr2=0x7fd3ec142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:36:33.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.006+0000 7fd3f1098700 1 --2- 172.21.15.17:0/833953759 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd3ec0b08a0 0x7fd3ec142eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.006+0000 7fd3f1098700 1 -- 172.21.15.17:0/833953759 >> v1:172.21.15.17:6789/0 conn(0x7fd3ec0ad000 legacy=0x7fd3ec05f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:33.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.006+0000 7fd3f0897700 1 -- 172.21.15.17:0/833953759 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1331085469 0 0) 0x7fd3d4003260 con 0x7fd3ec0ad000 2022-08-01T19:36:33.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.006+0000 7fd3f1098700 1 -- 172.21.15.17:0/833953759 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ec0aed50 con 0x7fd3ec0b3260 2022-08-01T19:36:33.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.006+0000 7fd3f1098700 1 --2- 172.21.15.17:0/833953759 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3ec0b3260 0x7fd3ec1433a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7fd3e4001b80 tx=0x7fd3dc006df0).ready entity=mon.1 client_cookie=5ecc3eaa6b1cc49f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:33.014 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.006+0000 7fd3f0897700 1 -- 172.21.15.17:0/833953759 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3e40019e0 con 0x7fd3ec0b3260 2022-08-01T19:36:33.014 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.006+0000 7fd3f0897700 1 -- 172.21.15.17:0/833953759 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3dc000e90 con 0x7fd3ec0b3260 2022-08-01T19:36:33.014 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.006+0000 7fd3f0897700 1 -- 172.21.15.17:0/833953759 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3dc012460 con 0x7fd3ec0b3260 2022-08-01T19:36:33.014 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.006+0000 7fd3f289b700 1 -- 172.21.15.17:0/833953759 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3ec0b3260 msgr2=0x7fd3ec1433a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:33.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.006+0000 7fd3f289b700 1 --2- 172.21.15.17:0/833953759 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3ec0b3260 0x7fd3ec1433a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7fd3e4001b80 tx=0x7fd3dc006df0).stop 2022-08-01T19:36:33.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.006+0000 7fd3f289b700 1 -- 172.21.15.17:0/833953759 shutdown_connections 2022-08-01T19:36:33.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.006+0000 7fd3f289b700 1 --2- 172.21.15.17:0/833953759 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3ec0b3260 0x7fd3ec1433a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.006+0000 7fd3f289b700 1 --2- 172.21.15.17:0/833953759 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd3ec0b08a0 0x7fd3ec142eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.006+0000 7fd3f289b700 1 -- 172.21.15.17:0/833953759 >> 172.21.15.17:0/833953759 conn(0x7fd3ec0612d0 msgr2=0x7fd3ec062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:33.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.006+0000 7fd3f289b700 1 -- 172.21.15.17:0/833953759 shutdown_connections 2022-08-01T19:36:33.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.006+0000 7fd3f289b700 1 -- 172.21.15.17:0/833953759 wait complete. 2022-08-01T19:36:33.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.010+0000 7fd3f289b700 1 Processor -- start 2022-08-01T19:36:33.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.010+0000 7fd3f289b700 1 -- start start 2022-08-01T19:36:33.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.010+0000 7fd3f289b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd3ec0ad000 0x7fd3ec14ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:33.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.010+0000 7fd3f289b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3ec0b08a0 0x7fd3ec1422b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:33.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.010+0000 7fd3f289b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3ec0b3260 0x7fd3ec0aab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:33.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.010+0000 7fd3f289b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec09edf0 con 0x7fd3ec0b08a0 2022-08-01T19:36:33.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.010+0000 7fd3f289b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec0aca70 con 0x7fd3ec0b3260 2022-08-01T19:36:33.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.010+0000 7fd3f289b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec0676d0 con 0x7fd3ec0ad000 2022-08-01T19:36:33.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.010+0000 7fd3f1098700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3ec0b08a0 0x7fd3ec1422b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:33.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.010+0000 7fd3f209a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3ec0b3260 0x7fd3ec0aab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:33.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.010+0000 7fd3f1899700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd3ec0ad000 0x7fd3ec14ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:33.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.010+0000 7fd3f209a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3ec0b3260 0x7fd3ec0aab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:36532/0 (socket says 172.21.15.17:36532) 2022-08-01T19:36:33.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.010+0000 7fd3f209a700 1 -- 172.21.15.17:0/2692393552 learned_addr learned my addr 172.21.15.17:0/2692393552 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:33.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.010+0000 7fd3f209a700 1 -- 172.21.15.17:0/2692393552 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd3ec0ad000 msgr2=0x7fd3ec14ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:33.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.010+0000 7fd3f209a700 1 --2- 172.21.15.17:0/2692393552 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd3ec0ad000 0x7fd3ec14ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.010+0000 7fd3f209a700 1 -- 172.21.15.17:0/2692393552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3ec0b08a0 msgr2=0x7fd3ec1422b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:33.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.010+0000 7fd3f209a700 1 --2- 172.21.15.17:0/2692393552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3ec0b08a0 0x7fd3ec1422b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.010+0000 7fd3f209a700 1 -- 172.21.15.17:0/2692393552 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3dc00b180 con 0x7fd3ec0b3260 2022-08-01T19:36:33.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.010+0000 7fd3f209a700 1 --2- 172.21.15.17:0/2692393552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3ec0b3260 0x7fd3ec0aab50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fd3e4001c40 tx=0x7fd3e4002180).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:33.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.010+0000 7fd3daffd700 1 -- 172.21.15.17:0/2692393552 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3e4004ab0 con 0x7fd3ec0b3260 2022-08-01T19:36:33.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.010+0000 7fd3daffd700 1 -- 172.21.15.17:0/2692393552 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e4004c10 con 0x7fd3ec0b3260 2022-08-01T19:36:33.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.010+0000 7fd3f289b700 1 -- 172.21.15.17:0/2692393552 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3ec064110 con 0x7fd3ec0b3260 2022-08-01T19:36:33.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.010+0000 7fd3daffd700 1 -- 172.21.15.17:0/2692393552 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3e4007170 con 0x7fd3ec0b3260 2022-08-01T19:36:33.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.010+0000 7fd3f289b700 1 -- 172.21.15.17:0/2692393552 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3ec067930 con 0x7fd3ec0b3260 2022-08-01T19:36:33.026 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.014+0000 7fd3f289b700 1 -- 172.21.15.17:0/2692393552 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3ec0aef30 con 0x7fd3ec0b3260 2022-08-01T19:36:33.026 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.014+0000 7fd3daffd700 1 -- 172.21.15.17:0/2692393552 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd3e401b070 con 0x7fd3ec0b3260 2022-08-01T19:36:33.026 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.014+0000 7fd3daffd700 1 --2- 172.21.15.17:0/2692393552 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd3cc053a10 0x7fd3cc055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:33.026 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.014+0000 7fd3daffd700 1 -- 172.21.15.17:0/2692393552 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd3e406b8e0 con 0x7fd3ec0b3260 2022-08-01T19:36:33.027 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.014+0000 7fd3f1899700 1 --2- 172.21.15.17:0/2692393552 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd3cc053a10 0x7fd3cc055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:33.027 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.014+0000 7fd3f1899700 1 --2- 172.21.15.17:0/2692393552 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd3cc053a10 0x7fd3cc055ed0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7fd3e8008000 tx=0x7fd3e8008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:33.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.030+0000 7fd3daffd700 1 -- 172.21.15.17:0/2692393552 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd3e4042640 con 0x7fd3ec0b3260 2022-08-01T19:36:33.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.150+0000 7fd3f289b700 1 -- 172.21.15.17:0/2692393552 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fd3ec0574e0 con 0x7fd3ec0b3260 2022-08-01T19:36:33.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.150+0000 7fd3daffd700 1 -- 172.21.15.17:0/2692393552 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v45) v1 ==== 110+0+10 (secure 0 0 0) 0x7fd3e4045c60 con 0x7fd3ec0b3260 2022-08-01T19:36:33.157 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:36:33.162 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.154+0000 7fd3f289b700 1 -- 172.21.15.17:0/2692393552 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd3cc053a10 msgr2=0x7fd3cc055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:33.163 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.154+0000 7fd3f289b700 1 --2- 172.21.15.17:0/2692393552 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd3cc053a10 0x7fd3cc055ed0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7fd3e8008000 tx=0x7fd3e8008040).stop 2022-08-01T19:36:33.163 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.154+0000 7fd3f289b700 1 -- 172.21.15.17:0/2692393552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3ec0b3260 msgr2=0x7fd3ec0aab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:33.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.154+0000 7fd3f289b700 1 --2- 172.21.15.17:0/2692393552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3ec0b3260 0x7fd3ec0aab50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fd3e4001c40 tx=0x7fd3e4002180).stop 2022-08-01T19:36:33.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.154+0000 7fd3f289b700 1 -- 172.21.15.17:0/2692393552 shutdown_connections 2022-08-01T19:36:33.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.154+0000 7fd3f289b700 1 --2- 172.21.15.17:0/2692393552 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd3cc053a10 0x7fd3cc055ed0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.154+0000 7fd3f289b700 1 --2- 172.21.15.17:0/2692393552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3ec0b3260 0x7fd3ec0aab50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.154+0000 7fd3f289b700 1 --2- 172.21.15.17:0/2692393552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3ec0b08a0 0x7fd3ec1422b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.154+0000 7fd3f289b700 1 --2- 172.21.15.17:0/2692393552 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd3ec0ad000 0x7fd3ec14ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.154+0000 7fd3f289b700 1 -- 172.21.15.17:0/2692393552 >> 172.21.15.17:0/2692393552 conn(0x7fd3ec0612d0 msgr2=0x7fd3ec141100 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:33.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.154+0000 7fd3f289b700 1 -- 172.21.15.17:0/2692393552 shutdown_connections 2022-08-01T19:36:33.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.154+0000 7fd3f289b700 1 -- 172.21.15.17:0/2692393552 wait complete. 2022-08-01T19:36:33.174 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:36:33.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.278+0000 7fc1bed1d700 1 Processor -- start 2022-08-01T19:36:33.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.278+0000 7fc1bed1d700 1 -- start start 2022-08-01T19:36:33.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.278+0000 7fc1bed1d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc1b8146ae0 0x7fc1b81403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:33.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.278+0000 7fc1bed1d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc1b80b4310 0x7fc1b81408b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:33.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.278+0000 7fc1bed1d700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1b8143cf0 con 0x7fc1b80b4820 2022-08-01T19:36:33.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.278+0000 7fc1bed1d700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1b81476a0 con 0x7fc1b8146ae0 2022-08-01T19:36:33.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.278+0000 7fc1bed1d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b8143f90 con 0x7fc1b80b4310 2022-08-01T19:36:33.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.278+0000 7fc1bcab9700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc1b8146ae0 0x7fc1b81403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:33.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.278+0000 7fc1bcab9700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc1b8146ae0 0x7fc1b81403c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38616/0 (socket says 172.21.15.17:38616) 2022-08-01T19:36:33.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.278+0000 7fc1bcab9700 1 -- 172.21.15.17:0/4064671196 learned_addr learned my addr 172.21.15.17:0/4064671196 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:33.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.278+0000 7fc1b7fff700 1 --2- 172.21.15.17:0/4064671196 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc1b80b4310 0x7fc1b81408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:33.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.278+0000 7fc1b77fe700 1 -- 172.21.15.17:0/4064671196 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 445781146 0 0) 0x7fc1b8143cf0 con 0x7fc1b80b4820 2022-08-01T19:36:33.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.278+0000 7fc1bcab9700 1 -- 172.21.15.17:0/4064671196 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc1b80b4310 msgr2=0x7fc1b81408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:36:33.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.278+0000 7fc1bcab9700 1 --2- 172.21.15.17:0/4064671196 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc1b80b4310 0x7fc1b81408b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.278+0000 7fc1bcab9700 1 -- 172.21.15.17:0/4064671196 >> v1:172.21.15.77:6789/0 conn(0x7fc1b80b4820 legacy=0x7fc1b80b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:33.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.278+0000 7fc1bcab9700 1 -- 172.21.15.17:0/4064671196 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b80b0e20 con 0x7fc1b8146ae0 2022-08-01T19:36:33.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.278+0000 7fc1bcab9700 1 --2- 172.21.15.17:0/4064671196 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1b8146ae0 0x7fc1b81403c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7fc1a8002f70 tx=0x7fc1a8006eb0).ready entity=mon.0 client_cookie=ec80fa6b5d712a4e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:33.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.278+0000 7fc1b77fe700 1 -- 172.21.15.17:0/4064671196 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1a800d570 con 0x7fc1b8146ae0 2022-08-01T19:36:33.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.278+0000 7fc1b77fe700 1 -- 172.21.15.17:0/4064671196 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a8000e90 con 0x7fc1b8146ae0 2022-08-01T19:36:33.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.278+0000 7fc1b77fe700 1 -- 172.21.15.17:0/4064671196 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1a800d8c0 con 0x7fc1b8146ae0 2022-08-01T19:36:33.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.278+0000 7fc1bed1d700 1 -- 172.21.15.17:0/4064671196 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1b8146ae0 msgr2=0x7fc1b81403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:33.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.278+0000 7fc1bed1d700 1 --2- 172.21.15.17:0/4064671196 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1b8146ae0 0x7fc1b81403c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7fc1a8002f70 tx=0x7fc1a8006eb0).stop 2022-08-01T19:36:33.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.278+0000 7fc1bed1d700 1 -- 172.21.15.17:0/4064671196 shutdown_connections 2022-08-01T19:36:33.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.278+0000 7fc1bed1d700 1 --2- 172.21.15.17:0/4064671196 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc1b80b4310 0x7fc1b81408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.278+0000 7fc1bed1d700 1 --2- 172.21.15.17:0/4064671196 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1b8146ae0 0x7fc1b81403c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.278+0000 7fc1bed1d700 1 -- 172.21.15.17:0/4064671196 >> 172.21.15.17:0/4064671196 conn(0x7fc1b805c620 msgr2=0x7fc1b805ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:33.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.278+0000 7fc1bed1d700 1 -- 172.21.15.17:0/4064671196 shutdown_connections 2022-08-01T19:36:33.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.278+0000 7fc1bed1d700 1 -- 172.21.15.17:0/4064671196 wait complete. 2022-08-01T19:36:33.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.282+0000 7fc1bed1d700 1 Processor -- start 2022-08-01T19:36:33.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.282+0000 7fc1bed1d700 1 -- start start 2022-08-01T19:36:33.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.282+0000 7fc1bed1d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1b80b4310 0x7fc1b80b8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:33.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.282+0000 7fc1bed1d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1b80b4820 0x7fc1b8147b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:33.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.282+0000 7fc1bed1d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc1b8146ae0 0x7fc1b8148050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:33.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.282+0000 7fc1bed1d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b8155930 con 0x7fc1b80b4310 2022-08-01T19:36:33.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.282+0000 7fc1bed1d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b80afe70 con 0x7fc1b80b4820 2022-08-01T19:36:33.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.282+0000 7fc1bed1d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b80aefc0 con 0x7fc1b8146ae0 2022-08-01T19:36:33.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.282+0000 7fc1b7fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1b80b4820 0x7fc1b8147b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:33.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.282+0000 7fc1bcab9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1b80b4310 0x7fc1b80b8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:33.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.282+0000 7fc1b7fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1b80b4820 0x7fc1b8147b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:36542/0 (socket says 172.21.15.17:36542) 2022-08-01T19:36:33.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.282+0000 7fc1b7fff700 1 -- 172.21.15.17:0/1247185273 learned_addr learned my addr 172.21.15.17:0/1247185273 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:33.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.282+0000 7fc1bd2ba700 1 --2- 172.21.15.17:0/1247185273 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc1b8146ae0 0x7fc1b8148050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:33.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.282+0000 7fc1b7fff700 1 -- 172.21.15.17:0/1247185273 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc1b8146ae0 msgr2=0x7fc1b8148050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:33.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.282+0000 7fc1b7fff700 1 --2- 172.21.15.17:0/1247185273 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc1b8146ae0 0x7fc1b8148050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.282+0000 7fc1b7fff700 1 -- 172.21.15.17:0/1247185273 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1b80b4310 msgr2=0x7fc1b80b8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:33.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.282+0000 7fc1b7fff700 1 --2- 172.21.15.17:0/1247185273 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1b80b4310 0x7fc1b80b8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.282+0000 7fc1b7fff700 1 -- 172.21.15.17:0/1247185273 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a800b1e0 con 0x7fc1b80b4820 2022-08-01T19:36:33.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.282+0000 7fc1bcab9700 1 --2- 172.21.15.17:0/1247185273 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1b80b4310 0x7fc1b80b8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:36:33.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.282+0000 7fc1b7fff700 1 --2- 172.21.15.17:0/1247185273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1b80b4820 0x7fc1b8147b60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7fc1a000c000 tx=0x7fc1a0006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:33.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.286+0000 7fc1b5ffb700 1 -- 172.21.15.17:0/1247185273 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1a0010780 con 0x7fc1b80b4820 2022-08-01T19:36:33.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.286+0000 7fc1b5ffb700 1 -- 172.21.15.17:0/1247185273 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a000b3b0 con 0x7fc1b80b4820 2022-08-01T19:36:33.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.286+0000 7fc1b5ffb700 1 -- 172.21.15.17:0/1247185273 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1a0010ad0 con 0x7fc1b80b4820 2022-08-01T19:36:33.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.286+0000 7fc1bed1d700 1 -- 172.21.15.17:0/1247185273 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b80af260 con 0x7fc1b80b4820 2022-08-01T19:36:33.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.286+0000 7fc1bed1d700 1 -- 172.21.15.17:0/1247185273 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b80b9160 con 0x7fc1b80b4820 2022-08-01T19:36:33.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.286+0000 7fc1bed1d700 1 -- 172.21.15.17:0/1247185273 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b80af3f0 con 0x7fc1b80b4820 2022-08-01T19:36:33.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.286+0000 7fc1b5ffb700 1 -- 172.21.15.17:0/1247185273 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc1a0010c30 con 0x7fc1b80b4820 2022-08-01T19:36:33.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.286+0000 7fc1b5ffb700 1 --2- 172.21.15.17:0/1247185273 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc19c053a10 0x7fc19c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:33.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.286+0000 7fc1b5ffb700 1 -- 172.21.15.17:0/1247185273 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc1a006c440 con 0x7fc1b80b4820 2022-08-01T19:36:33.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.286+0000 7fc1bcab9700 1 --2- 172.21.15.17:0/1247185273 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc19c053a10 0x7fc19c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:33.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.290+0000 7fc1bcab9700 1 --2- 172.21.15.17:0/1247185273 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc19c053a10 0x7fc19c055ed0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7fc1a801c000 tx=0x7fc1a8015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:33.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.294+0000 7fc1b5ffb700 1 -- 172.21.15.17:0/1247185273 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc1a0043070 con 0x7fc1b80b4820 2022-08-01T19:36:33.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.418+0000 7fc1bed1d700 1 -- 172.21.15.17:0/1247185273 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc1b80574e0 con 0x7fc1b80b4820 2022-08-01T19:36:33.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.426+0000 7fc1b5ffb700 1 -- 172.21.15.17:0/1247185273 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v45) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc1a0046690 con 0x7fc1b80b4820 2022-08-01T19:36:33.427 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:36:33.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.426+0000 7fc1bed1d700 1 -- 172.21.15.17:0/1247185273 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc19c053a10 msgr2=0x7fc19c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:33.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.426+0000 7fc1bed1d700 1 --2- 172.21.15.17:0/1247185273 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc19c053a10 0x7fc19c055ed0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7fc1a801c000 tx=0x7fc1a8015000).stop 2022-08-01T19:36:33.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.426+0000 7fc1bed1d700 1 -- 172.21.15.17:0/1247185273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1b80b4820 msgr2=0x7fc1b8147b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:33.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.426+0000 7fc1bed1d700 1 --2- 172.21.15.17:0/1247185273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1b80b4820 0x7fc1b8147b60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7fc1a000c000 tx=0x7fc1a0006df0).stop 2022-08-01T19:36:33.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.426+0000 7fc1bed1d700 1 -- 172.21.15.17:0/1247185273 shutdown_connections 2022-08-01T19:36:33.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.426+0000 7fc1bed1d700 1 --2- 172.21.15.17:0/1247185273 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc19c053a10 0x7fc19c055ed0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.426+0000 7fc1bed1d700 1 --2- 172.21.15.17:0/1247185273 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc1b8146ae0 0x7fc1b8148050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.426+0000 7fc1bed1d700 1 --2- 172.21.15.17:0/1247185273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1b80b4820 0x7fc1b8147b60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.426+0000 7fc1bed1d700 1 --2- 172.21.15.17:0/1247185273 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1b80b4310 0x7fc1b80b8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.426+0000 7fc1bed1d700 1 -- 172.21.15.17:0/1247185273 >> 172.21.15.17:0/1247185273 conn(0x7fc1b805c620 msgr2=0x7fc1b8143360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:33.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.426+0000 7fc1bed1d700 1 -- 172.21.15.17:0/1247185273 shutdown_connections 2022-08-01T19:36:33.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.426+0000 7fc1bed1d700 1 -- 172.21.15.17:0/1247185273 wait complete. 2022-08-01T19:36:33.438 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T19:36:33.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.542+0000 7fe5a80bb700 1 Processor -- start 2022-08-01T19:36:33.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.542+0000 7fe5a80bb700 1 -- start start 2022-08-01T19:36:33.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.546+0000 7fe5a80bb700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fe5a00aa050 0x7fe5a00aa430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:33.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.546+0000 7fe5a80bb700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe5a013f920 0x7fe5a013fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:33.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.546+0000 7fe5a80bb700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5a0061b20 con 0x7fe5a0060560 2022-08-01T19:36:33.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.546+0000 7fe5a80bb700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5a0140280 con 0x7fe5a00aa050 2022-08-01T19:36:33.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.546+0000 7fe5a80bb700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a0061dc0 con 0x7fe5a013f920 2022-08-01T19:36:33.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.546+0000 7fe5a6658700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe5a013f920 0x7fe5a013fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:33.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.546+0000 7fe5a5e57700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fe5a00aa050 0x7fe5a00aa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:33.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.546+0000 7fe5a5656700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7fe5a0060560 0x7fe5a013ce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:47622/0 (socket says 172.21.15.17:47622) 2022-08-01T19:36:33.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.546+0000 7fe5a5656700 1 -- 172.21.15.17:0/2740869947 learned_addr learned my addr 172.21.15.17:0/2740869947 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:36:33.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.546+0000 7fe5a6658700 1 -- 172.21.15.17:0/2740869947 >> v2:172.21.15.77:3300/0 conn(0x7fe5a00aa050 msgr2=0x7fe5a00aa430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:36:33.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.546+0000 7fe5a6658700 1 --2- 172.21.15.17:0/2740869947 >> v2:172.21.15.77:3300/0 conn(0x7fe5a00aa050 0x7fe5a00aa430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.546+0000 7fe5a6658700 1 -- 172.21.15.17:0/2740869947 >> v1:172.21.15.77:6789/0 conn(0x7fe5a0060560 legacy=0x7fe5a013ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:33.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.546+0000 7fe5a6658700 1 -- 172.21.15.17:0/2740869947 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a0060d20 con 0x7fe5a013f920 2022-08-01T19:36:33.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.546+0000 7fe5a4e55700 1 -- 172.21.15.17:0/2740869947 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3269143915 0 0) 0x7fe5a0061b20 con 0x7fe5a0060560 2022-08-01T19:36:33.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.546+0000 7fe5a6658700 1 --2- 172.21.15.17:0/2740869947 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe5a013f920 0x7fe5a013fd90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fe5980032f0 tx=0x7fe5980091d0).ready entity=mon.2 client_cookie=a879f5651bd65833 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:33.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.546+0000 7fe5a4e55700 1 -- 172.21.15.17:0/2740869947 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe59800b6c0 con 0x7fe5a013f920 2022-08-01T19:36:33.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.546+0000 7fe5a4e55700 1 -- 172.21.15.17:0/2740869947 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe59800b820 con 0x7fe5a013f920 2022-08-01T19:36:33.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.546+0000 7fe5a4e55700 1 -- 172.21.15.17:0/2740869947 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe59800b990 con 0x7fe5a013f920 2022-08-01T19:36:33.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.546+0000 7fe5a80bb700 1 -- 172.21.15.17:0/2740869947 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe5a013f920 msgr2=0x7fe5a013fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:33.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.546+0000 7fe5a80bb700 1 --2- 172.21.15.17:0/2740869947 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe5a013f920 0x7fe5a013fd90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fe5980032f0 tx=0x7fe5980091d0).stop 2022-08-01T19:36:33.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.546+0000 7fe5a80bb700 1 -- 172.21.15.17:0/2740869947 shutdown_connections 2022-08-01T19:36:33.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.546+0000 7fe5a80bb700 1 --2- 172.21.15.17:0/2740869947 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe5a013f920 0x7fe5a013fd90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.546+0000 7fe5a80bb700 1 --2- 172.21.15.17:0/2740869947 >> v2:172.21.15.77:3300/0 conn(0x7fe5a00aa050 0x7fe5a00aa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.546+0000 7fe5a80bb700 1 -- 172.21.15.17:0/2740869947 >> 172.21.15.17:0/2740869947 conn(0x7fe5a00b2b00 msgr2=0x7fe5a00b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:33.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.546+0000 7fe5a80bb700 1 -- 172.21.15.17:0/2740869947 shutdown_connections 2022-08-01T19:36:33.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.546+0000 7fe5a80bb700 1 -- 172.21.15.17:0/2740869947 wait complete. 2022-08-01T19:36:33.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.550+0000 7fe5a80bb700 1 Processor -- start 2022-08-01T19:36:33.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.550+0000 7fe5a80bb700 1 -- start start 2022-08-01T19:36:33.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.550+0000 7fe5a80bb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5a0060560 0x7fe5a0156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:33.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.550+0000 7fe5a80bb700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe5a00aa050 0x7fe5a00a75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:33.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.550+0000 7fe5a80bb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5a013f920 0x7fe5a00af6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:33.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.550+0000 7fe5a80bb700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a00aec70 con 0x7fe5a013f920 2022-08-01T19:36:33.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.550+0000 7fe5a80bb700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a01570d0 con 0x7fe5a0060560 2022-08-01T19:36:33.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.550+0000 7fe5a80bb700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a00ae9d0 con 0x7fe5a00aa050 2022-08-01T19:36:33.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.550+0000 7fe5a6658700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5a013f920 0x7fe5a00af6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:33.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.550+0000 7fe5a5656700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe5a00aa050 0x7fe5a00a75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:33.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.550+0000 7fe5a6658700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5a013f920 0x7fe5a00af6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38638/0 (socket says 172.21.15.17:38638) 2022-08-01T19:36:33.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.550+0000 7fe5a6658700 1 -- 172.21.15.17:0/3095718584 learned_addr learned my addr 172.21.15.17:0/3095718584 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:33.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.550+0000 7fe5a5e57700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5a0060560 0x7fe5a0156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:33.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.550+0000 7fe5a5656700 1 -- 172.21.15.17:0/3095718584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5a0060560 msgr2=0x7fe5a0156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:33.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.550+0000 7fe5a5656700 1 --2- 172.21.15.17:0/3095718584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5a0060560 0x7fe5a0156be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.550+0000 7fe5a5656700 1 -- 172.21.15.17:0/3095718584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5a013f920 msgr2=0x7fe5a00af6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:33.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.550+0000 7fe5a5656700 1 --2- 172.21.15.17:0/3095718584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5a013f920 0x7fe5a00af6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.550+0000 7fe5a5656700 1 -- 172.21.15.17:0/3095718584 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe598009470 con 0x7fe5a00aa050 2022-08-01T19:36:33.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.550+0000 7fe5a5656700 1 --2- 172.21.15.17:0/3095718584 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe5a00aa050 0x7fe5a00a75d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fe590000ed0 tx=0x7fe590007c20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:33.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.550+0000 7fe58effd700 1 -- 172.21.15.17:0/3095718584 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5900070d0 con 0x7fe5a00aa050 2022-08-01T19:36:33.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.550+0000 7fe58effd700 1 -- 172.21.15.17:0/3095718584 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe590007230 con 0x7fe5a00aa050 2022-08-01T19:36:33.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.550+0000 7fe5a80bb700 1 -- 172.21.15.17:0/3095718584 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5a00b0290 con 0x7fe5a00aa050 2022-08-01T19:36:33.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.550+0000 7fe58effd700 1 -- 172.21.15.17:0/3095718584 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe590005610 con 0x7fe5a00aa050 2022-08-01T19:36:33.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.550+0000 7fe5a80bb700 1 -- 172.21.15.17:0/3095718584 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5a0142730 con 0x7fe5a00aa050 2022-08-01T19:36:33.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.554+0000 7fe5a80bb700 1 -- 172.21.15.17:0/3095718584 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5a0060fc0 con 0x7fe5a00aa050 2022-08-01T19:36:33.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.554+0000 7fe58effd700 1 -- 172.21.15.17:0/3095718584 <== mon.2 v2:172.21.15.77:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe59001a060 con 0x7fe5a00aa050 2022-08-01T19:36:33.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.554+0000 7fe58effd700 1 --2- 172.21.15.17:0/3095718584 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe5800539e0 0x7fe580055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:33.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.554+0000 7fe58effd700 1 -- 172.21.15.17:0/3095718584 <== mon.2 v2:172.21.15.77:3301/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe59006b7c0 con 0x7fe5a00aa050 2022-08-01T19:36:33.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.554+0000 7fe5a5e57700 1 --2- 172.21.15.17:0/3095718584 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe5800539e0 0x7fe580055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:33.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.554+0000 7fe5a5e57700 1 --2- 172.21.15.17:0/3095718584 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe5800539e0 0x7fe580055ea0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7fe59c00b000 tx=0x7fe59c006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:33.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.566+0000 7fe58effd700 1 -- 172.21.15.17:0/3095718584 <== mon.2 v2:172.21.15.77:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe590042580 con 0x7fe5a00aa050 2022-08-01T19:36:33.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.690+0000 7fe5a80bb700 1 -- 172.21.15.17:0/3095718584 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe5a00574e0 con 0x7fe5a00aa050 2022-08-01T19:36:33.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.690+0000 7fe58effd700 1 -- 172.21.15.17:0/3095718584 <== mon.2 v2:172.21.15.77:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) v1 ==== 104+0+10 (secure 0 0 0) 0x7fe590018020 con 0x7fe5a00aa050 2022-08-01T19:36:33.695 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:36:33.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.694+0000 7fe5a80bb700 1 -- 172.21.15.17:0/3095718584 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe5800539e0 msgr2=0x7fe580055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:33.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.694+0000 7fe5a80bb700 1 --2- 172.21.15.17:0/3095718584 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe5800539e0 0x7fe580055ea0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7fe59c00b000 tx=0x7fe59c006df0).stop 2022-08-01T19:36:33.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.694+0000 7fe5a80bb700 1 -- 172.21.15.17:0/3095718584 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe5a00aa050 msgr2=0x7fe5a00a75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:33.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.694+0000 7fe5a80bb700 1 --2- 172.21.15.17:0/3095718584 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe5a00aa050 0x7fe5a00a75d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fe590000ed0 tx=0x7fe590007c20).stop 2022-08-01T19:36:33.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.694+0000 7fe5a80bb700 1 -- 172.21.15.17:0/3095718584 shutdown_connections 2022-08-01T19:36:33.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.694+0000 7fe5a80bb700 1 --2- 172.21.15.17:0/3095718584 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe5800539e0 0x7fe580055ea0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.694+0000 7fe5a80bb700 1 --2- 172.21.15.17:0/3095718584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5a013f920 0x7fe5a00af6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.694+0000 7fe5a80bb700 1 --2- 172.21.15.17:0/3095718584 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe5a00aa050 0x7fe5a00a75d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.694+0000 7fe5a80bb700 1 --2- 172.21.15.17:0/3095718584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5a0060560 0x7fe5a0156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.694+0000 7fe5a80bb700 1 -- 172.21.15.17:0/3095718584 >> 172.21.15.17:0/3095718584 conn(0x7fe5a00b2b00 msgr2=0x7fe5a00b1690 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:33.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.694+0000 7fe5a80bb700 1 -- 172.21.15.17:0/3095718584 shutdown_connections 2022-08-01T19:36:33.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.694+0000 7fe5a80bb700 1 -- 172.21.15.17:0/3095718584 wait complete. 2022-08-01T19:36:33.715 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T19:36:33.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.822+0000 7f1e1023f700 1 Processor -- start 2022-08-01T19:36:33.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.822+0000 7f1e1023f700 1 -- start start 2022-08-01T19:36:33.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.822+0000 7f1e1023f700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f1e080aa050 0x7f1e080aa430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:33.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.822+0000 7f1e1023f700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1e0813f920 0x7f1e0813fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:33.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.822+0000 7f1e1023f700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e08061b20 con 0x7f1e08060560 2022-08-01T19:36:33.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.822+0000 7f1e1023f700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e08140280 con 0x7f1e080aa050 2022-08-01T19:36:33.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.822+0000 7f1e1023f700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e08061dc0 con 0x7f1e0813f920 2022-08-01T19:36:33.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.822+0000 7f1e0d7da700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1e08060560 0x7f1e0813ce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59072/0 (socket says 172.21.15.17:59072) 2022-08-01T19:36:33.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.822+0000 7f1e0dfdb700 1 --2- 172.21.15.17:0/1747648476 >> v2:172.21.15.77:3300/0 conn(0x7f1e080aa050 0x7f1e080aa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:33.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.822+0000 7f1e0e7dc700 1 --2- 172.21.15.17:0/1747648476 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1e0813f920 0x7f1e0813fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:33.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.822+0000 7f1e0d7da700 1 -- 172.21.15.17:0/1747648476 learned_addr learned my addr 172.21.15.17:0/1747648476 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:36:33.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.822+0000 7f1e0cfd9700 1 -- 172.21.15.17:0/1747648476 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3706095062 0 0) 0x7f1e08061b20 con 0x7f1e08060560 2022-08-01T19:36:33.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.822+0000 7f1e0e7dc700 1 -- 172.21.15.17:0/1747648476 >> v2:172.21.15.77:3300/0 conn(0x7f1e080aa050 msgr2=0x7f1e080aa430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:36:33.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.822+0000 7f1e0e7dc700 1 --2- 172.21.15.17:0/1747648476 >> v2:172.21.15.77:3300/0 conn(0x7f1e080aa050 0x7f1e080aa430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.822+0000 7f1e0e7dc700 1 -- 172.21.15.17:0/1747648476 >> v1:172.21.15.17:6789/0 conn(0x7f1e08060560 legacy=0x7f1e0813ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:33.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.822+0000 7f1e0e7dc700 1 -- 172.21.15.17:0/1747648476 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e08060d20 con 0x7f1e0813f920 2022-08-01T19:36:33.834 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.822+0000 7f1e0e7dc700 1 --2- 172.21.15.17:0/1747648476 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1e0813f920 0x7f1e0813fd90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f1e000032f0 tx=0x7f1e000091d0).ready entity=mon.2 client_cookie=ee9335cdcb19be67 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:33.834 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.822+0000 7f1e0cfd9700 1 -- 172.21.15.17:0/1747648476 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e0000b6c0 con 0x7f1e0813f920 2022-08-01T19:36:33.834 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.822+0000 7f1e0cfd9700 1 -- 172.21.15.17:0/1747648476 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0000b820 con 0x7f1e0813f920 2022-08-01T19:36:33.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.822+0000 7f1e0cfd9700 1 -- 172.21.15.17:0/1747648476 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e0000b990 con 0x7f1e0813f920 2022-08-01T19:36:33.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.826+0000 7f1e1023f700 1 -- 172.21.15.17:0/1747648476 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1e0813f920 msgr2=0x7f1e0813fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:33.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.826+0000 7f1e1023f700 1 --2- 172.21.15.17:0/1747648476 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1e0813f920 0x7f1e0813fd90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f1e000032f0 tx=0x7f1e000091d0).stop 2022-08-01T19:36:33.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.826+0000 7f1e1023f700 1 -- 172.21.15.17:0/1747648476 shutdown_connections 2022-08-01T19:36:33.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.826+0000 7f1e1023f700 1 --2- 172.21.15.17:0/1747648476 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1e0813f920 0x7f1e0813fd90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.826+0000 7f1e1023f700 1 --2- 172.21.15.17:0/1747648476 >> v2:172.21.15.77:3300/0 conn(0x7f1e080aa050 0x7f1e080aa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.826+0000 7f1e1023f700 1 -- 172.21.15.17:0/1747648476 >> 172.21.15.17:0/1747648476 conn(0x7f1e080b2b00 msgr2=0x7f1e080b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:33.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.826+0000 7f1e1023f700 1 -- 172.21.15.17:0/1747648476 shutdown_connections 2022-08-01T19:36:33.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.826+0000 7f1e1023f700 1 -- 172.21.15.17:0/1747648476 wait complete. 2022-08-01T19:36:33.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.826+0000 7f1e1023f700 1 Processor -- start 2022-08-01T19:36:33.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.826+0000 7f1e1023f700 1 -- start start 2022-08-01T19:36:33.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.826+0000 7f1e1023f700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1e08060560 0x7f1e08156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:33.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.826+0000 7f1e1023f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e080aa050 0x7f1e080afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:33.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.826+0000 7f1e1023f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e0813f920 0x7f1e0813b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:33.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.826+0000 7f1e1023f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e08063570 con 0x7f1e0813f920 2022-08-01T19:36:33.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.826+0000 7f1e1023f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e08156550 con 0x7f1e080aa050 2022-08-01T19:36:33.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.826+0000 7f1e1023f700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e08156780 con 0x7f1e08060560 2022-08-01T19:36:33.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.826+0000 7f1e0e7dc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e0813f920 0x7f1e0813b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:33.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.826+0000 7f1e0e7dc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e0813f920 0x7f1e0813b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38652/0 (socket says 172.21.15.17:38652) 2022-08-01T19:36:33.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.826+0000 7f1e0e7dc700 1 -- 172.21.15.17:0/3226382739 learned_addr learned my addr 172.21.15.17:0/3226382739 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:33.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.826+0000 7f1e0d7da700 1 --2- 172.21.15.17:0/3226382739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e080aa050 0x7f1e080afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:33.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.826+0000 7f1e0dfdb700 1 --2- 172.21.15.17:0/3226382739 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1e08060560 0x7f1e08156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:33.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.826+0000 7f1e0e7dc700 1 -- 172.21.15.17:0/3226382739 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1e08060560 msgr2=0x7f1e08156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:33.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.826+0000 7f1e0e7dc700 1 --2- 172.21.15.17:0/3226382739 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1e08060560 0x7f1e08156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.826+0000 7f1e0e7dc700 1 -- 172.21.15.17:0/3226382739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e080aa050 msgr2=0x7f1e080afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:33.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.826+0000 7f1e0e7dc700 1 --2- 172.21.15.17:0/3226382739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e080aa050 0x7f1e080afd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.826+0000 7f1e0e7dc700 1 -- 172.21.15.17:0/3226382739 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e00009470 con 0x7f1e0813f920 2022-08-01T19:36:33.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.830+0000 7f1e0e7dc700 1 --2- 172.21.15.17:0/3226382739 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e0813f920 0x7f1e0813b4a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f1e000032f0 tx=0x7f1e00016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:33.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.830+0000 7f1df6ffd700 1 -- 172.21.15.17:0/3226382739 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e0000b990 con 0x7f1e0813f920 2022-08-01T19:36:33.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.830+0000 7f1df6ffd700 1 -- 172.21.15.17:0/3226382739 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e00021630 con 0x7f1e0813f920 2022-08-01T19:36:33.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.830+0000 7f1e1023f700 1 -- 172.21.15.17:0/3226382739 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e080b0270 con 0x7f1e0813f920 2022-08-01T19:36:33.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.830+0000 7f1df6ffd700 1 -- 172.21.15.17:0/3226382739 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e0000b6c0 con 0x7f1e0813f920 2022-08-01T19:36:33.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.830+0000 7f1e1023f700 1 -- 172.21.15.17:0/3226382739 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e081569e0 con 0x7f1e0813f920 2022-08-01T19:36:33.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.830+0000 7f1e1023f700 1 -- 172.21.15.17:0/3226382739 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e08060fc0 con 0x7f1e0813f920 2022-08-01T19:36:33.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.830+0000 7f1df6ffd700 1 -- 172.21.15.17:0/3226382739 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1e00027020 con 0x7f1e0813f920 2022-08-01T19:36:33.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.830+0000 7f1df6ffd700 1 --2- 172.21.15.17:0/3226382739 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1de80539e0 0x7f1de8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:33.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.834+0000 7f1df6ffd700 1 -- 172.21.15.17:0/3226382739 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1e0006d550 con 0x7f1e0813f920 2022-08-01T19:36:33.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.834+0000 7f1e0dfdb700 1 --2- 172.21.15.17:0/3226382739 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1de80539e0 0x7f1de8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:33.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.834+0000 7f1e0dfdb700 1 --2- 172.21.15.17:0/3226382739 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1de80539e0 0x7f1de8055ea0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f1e0400b000 tx=0x7f1e04006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:33.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.838+0000 7f1df6ffd700 1 -- 172.21.15.17:0/3226382739 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1e00044270 con 0x7f1e0813f920 2022-08-01T19:36:33.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.966+0000 7f1e1023f700 1 -- 172.21.15.17:0/3226382739 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1e080574e0 con 0x7f1e0813f920 2022-08-01T19:36:33.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.966+0000 7f1df6ffd700 1 -- 172.21.15.17:0/3226382739 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) v1 ==== 100+0+10 (secure 0 0 0) 0x7f1e000150a0 con 0x7f1e0813f920 2022-08-01T19:36:33.972 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:36:33.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.970+0000 7f1e1023f700 1 -- 172.21.15.17:0/3226382739 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1de80539e0 msgr2=0x7f1de8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:33.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.970+0000 7f1e1023f700 1 --2- 172.21.15.17:0/3226382739 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1de80539e0 0x7f1de8055ea0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f1e0400b000 tx=0x7f1e04006df0).stop 2022-08-01T19:36:33.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.970+0000 7f1e1023f700 1 -- 172.21.15.17:0/3226382739 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e0813f920 msgr2=0x7f1e0813b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:33.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.970+0000 7f1e1023f700 1 --2- 172.21.15.17:0/3226382739 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e0813f920 0x7f1e0813b4a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f1e000032f0 tx=0x7f1e00016000).stop 2022-08-01T19:36:33.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.970+0000 7f1e1023f700 1 -- 172.21.15.17:0/3226382739 shutdown_connections 2022-08-01T19:36:33.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.970+0000 7f1e1023f700 1 --2- 172.21.15.17:0/3226382739 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1de80539e0 0x7f1de8055ea0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.970+0000 7f1e1023f700 1 --2- 172.21.15.17:0/3226382739 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e0813f920 0x7f1e0813b4a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.970+0000 7f1e1023f700 1 --2- 172.21.15.17:0/3226382739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e080aa050 0x7f1e080afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.970+0000 7f1e1023f700 1 --2- 172.21.15.17:0/3226382739 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1e08060560 0x7f1e08156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:33.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.970+0000 7f1e1023f700 1 -- 172.21.15.17:0/3226382739 >> 172.21.15.17:0/3226382739 conn(0x7f1e080b2b00 msgr2=0x7f1e080b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:33.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.970+0000 7f1e1023f700 1 -- 172.21.15.17:0/3226382739 shutdown_connections 2022-08-01T19:36:33.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:33.970+0000 7f1e1023f700 1 -- 172.21.15.17:0/3226382739 wait complete. 2022-08-01T19:36:33.990 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T19:36:33.991 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:36:34.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.090+0000 7f11a57bc700 1 Processor -- start 2022-08-01T19:36:34.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.090+0000 7f11a57bc700 1 -- start start 2022-08-01T19:36:34.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.090+0000 7f11a57bc700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f11a00b0630 0x7f11a00adec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:34.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.090+0000 7f11a57bc700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f11a00aa2b0 0x7f11a00ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:34.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.090+0000 7f11a57bc700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11a00b2c80 con 0x7f11a00b0a10 2022-08-01T19:36:34.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.090+0000 7f11a57bc700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11a00b2900 con 0x7f11a00aa2b0 2022-08-01T19:36:34.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.090+0000 7f11a57bc700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a00b2eb0 con 0x7f11a00b0630 2022-08-01T19:36:34.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.090+0000 7f119f7fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f11a00b0a10 0x7f11a01438a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59086/0 (socket says 172.21.15.17:59086) 2022-08-01T19:36:34.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.090+0000 7f119f7fe700 1 -- 172.21.15.17:0/2623790784 learned_addr learned my addr 172.21.15.17:0/2623790784 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:36:34.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.090+0000 7f119effd700 1 --2- 172.21.15.17:0/2623790784 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f11a00b0630 0x7f11a00adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:34.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.090+0000 7f119e7fc700 1 --2- 172.21.15.17:0/2623790784 >> v2:172.21.15.77:3300/0 conn(0x7f11a00aa2b0 0x7f11a00ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:34.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.090+0000 7f119dffb700 1 -- 172.21.15.17:0/2623790784 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1615263221 0 0) 0x7f11a00b2c80 con 0x7f11a00b0a10 2022-08-01T19:36:34.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.090+0000 7f119dffb700 1 -- 172.21.15.17:0/2623790784 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1184003260 con 0x7f11a00b0a10 2022-08-01T19:36:34.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.090+0000 7f119e7fc700 1 -- 172.21.15.17:0/2623790784 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f11a00b0630 msgr2=0x7f11a00adec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:36:34.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.090+0000 7f119dffb700 1 -- 172.21.15.17:0/2623790784 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1325660359 0 0) 0x7f1184003260 con 0x7f11a00b0a10 2022-08-01T19:36:34.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.090+0000 7f119e7fc700 1 --2- 172.21.15.17:0/2623790784 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f11a00b0630 0x7f11a00adec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:34.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.090+0000 7f119e7fc700 1 -- 172.21.15.17:0/2623790784 >> v1:172.21.15.17:6789/0 conn(0x7f11a00b0a10 legacy=0x7f11a01438a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:34.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.090+0000 7f119e7fc700 1 -- 172.21.15.17:0/2623790784 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a00a9d00 con 0x7f11a00aa2b0 2022-08-01T19:36:34.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.090+0000 7f119effd700 1 --2- 172.21.15.17:0/2623790784 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f11a00b0630 0x7f11a00adec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:36:34.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.090+0000 7f119e7fc700 1 --2- 172.21.15.17:0/2623790784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11a00aa2b0 0x7f11a00ae3b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f1198001b80 tx=0x7f1194006df0).ready entity=mon.1 client_cookie=49281e1ead03a1f4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:34.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.090+0000 7f119dffb700 1 -- 172.21.15.17:0/2623790784 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11980019e0 con 0x7f11a00aa2b0 2022-08-01T19:36:34.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.090+0000 7f119dffb700 1 -- 172.21.15.17:0/2623790784 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1194000e90 con 0x7f11a00aa2b0 2022-08-01T19:36:34.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.090+0000 7f119dffb700 1 -- 172.21.15.17:0/2623790784 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1194012460 con 0x7f11a00aa2b0 2022-08-01T19:36:34.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.090+0000 7f11a57bc700 1 -- 172.21.15.17:0/2623790784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11a00aa2b0 msgr2=0x7f11a00ae3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:34.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.090+0000 7f11a57bc700 1 --2- 172.21.15.17:0/2623790784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11a00aa2b0 0x7f11a00ae3b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f1198001b80 tx=0x7f1194006df0).stop 2022-08-01T19:36:34.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.090+0000 7f11a57bc700 1 -- 172.21.15.17:0/2623790784 shutdown_connections 2022-08-01T19:36:34.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.090+0000 7f11a57bc700 1 --2- 172.21.15.17:0/2623790784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11a00aa2b0 0x7f11a00ae3b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:34.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.090+0000 7f11a57bc700 1 --2- 172.21.15.17:0/2623790784 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f11a00b0630 0x7f11a00adec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:34.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.090+0000 7f11a57bc700 1 -- 172.21.15.17:0/2623790784 >> 172.21.15.17:0/2623790784 conn(0x7f11a0060950 msgr2=0x7f11a0060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:34.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.090+0000 7f11a57bc700 1 -- 172.21.15.17:0/2623790784 shutdown_connections 2022-08-01T19:36:34.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.090+0000 7f11a57bc700 1 -- 172.21.15.17:0/2623790784 wait complete. 2022-08-01T19:36:34.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.094+0000 7f11a57bc700 1 Processor -- start 2022-08-01T19:36:34.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.094+0000 7f11a57bc700 1 -- start start 2022-08-01T19:36:34.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.094+0000 7f11a57bc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11a00aa2b0 0x7f11a0139240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:34.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.094+0000 7f11a57bc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11a00b0630 0x7f11a0065770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:34.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.094+0000 7f11a57bc700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f11a00b0a10 0x7f11a00abeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:34.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.094+0000 7f11a57bc700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a0134470 con 0x7f11a00b0630 2022-08-01T19:36:34.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.094+0000 7f11a57bc700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a014bc40 con 0x7f11a00aa2b0 2022-08-01T19:36:34.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.094+0000 7f11a57bc700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a012eec0 con 0x7f11a00b0a10 2022-08-01T19:36:34.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.094+0000 7f119e7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11a00b0630 0x7f11a0065770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:34.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.094+0000 7f119effd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11a00aa2b0 0x7f11a0139240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:34.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.094+0000 7f119e7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11a00b0630 0x7f11a0065770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38666/0 (socket says 172.21.15.17:38666) 2022-08-01T19:36:34.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.094+0000 7f119e7fc700 1 -- 172.21.15.17:0/2969108751 learned_addr learned my addr 172.21.15.17:0/2969108751 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:36:34.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.094+0000 7f119f7fe700 1 --2- 172.21.15.17:0/2969108751 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f11a00b0a10 0x7f11a00abeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:34.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.094+0000 7f119e7fc700 1 -- 172.21.15.17:0/2969108751 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f11a00b0a10 msgr2=0x7f11a00abeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:34.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.094+0000 7f119e7fc700 1 --2- 172.21.15.17:0/2969108751 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f11a00b0a10 0x7f11a00abeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:34.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.094+0000 7f119e7fc700 1 -- 172.21.15.17:0/2969108751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11a00aa2b0 msgr2=0x7f11a0139240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:34.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.094+0000 7f119e7fc700 1 --2- 172.21.15.17:0/2969108751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11a00aa2b0 0x7f11a0139240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:34.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.094+0000 7f119e7fc700 1 -- 172.21.15.17:0/2969108751 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f119000b180 con 0x7f11a00b0630 2022-08-01T19:36:34.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.094+0000 7f119effd700 1 --2- 172.21.15.17:0/2969108751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11a00aa2b0 0x7f11a0139240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:36:34.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.094+0000 7f119e7fc700 1 --2- 172.21.15.17:0/2969108751 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11a00b0630 0x7f11a0065770 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f1198001b80 tx=0x7f1194014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:34.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.094+0000 7f118ffff700 1 -- 172.21.15.17:0/2969108751 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f119400d8a0 con 0x7f11a00b0630 2022-08-01T19:36:34.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.094+0000 7f118ffff700 1 -- 172.21.15.17:0/2969108751 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f119400da00 con 0x7f11a00b0630 2022-08-01T19:36:34.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.094+0000 7f11a57bc700 1 -- 172.21.15.17:0/2969108751 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f119400b180 con 0x7f11a00b0630 2022-08-01T19:36:34.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.094+0000 7f118ffff700 1 -- 172.21.15.17:0/2969108751 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f119400dd60 con 0x7f11a00b0630 2022-08-01T19:36:34.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.094+0000 7f11a57bc700 1 -- 172.21.15.17:0/2969108751 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a012f400 con 0x7f11a00b0630 2022-08-01T19:36:34.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.098+0000 7f11a57bc700 1 -- 172.21.15.17:0/2969108751 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a00ac980 con 0x7f11a00b0630 2022-08-01T19:36:34.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.098+0000 7f118ffff700 1 -- 172.21.15.17:0/2969108751 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1194025030 con 0x7f11a00b0630 2022-08-01T19:36:34.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.098+0000 7f118ffff700 1 --2- 172.21.15.17:0/2969108751 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f117c053aa0 0x7f117c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:36:34.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.098+0000 7f118ffff700 1 -- 172.21.15.17:0/2969108751 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f119406cb60 con 0x7f11a00b0630 2022-08-01T19:36:34.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.098+0000 7f119effd700 1 --2- 172.21.15.17:0/2969108751 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f117c053aa0 0x7f117c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:36:34.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.098+0000 7f119effd700 1 --2- 172.21.15.17:0/2969108751 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f117c053aa0 0x7f117c055f60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f119000b000 tx=0x7f1190006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:36:34.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.102+0000 7f118ffff700 1 -- 172.21.15.17:0/2969108751 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1194043960 con 0x7f11a00b0630 2022-08-01T19:36:34.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.270+0000 7f11a57bc700 1 -- 172.21.15.17:0/2969108751 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11a00574e0 con 0x7f11a00b0630 2022-08-01T19:36:34.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.274+0000 7f118ffff700 1 -- 172.21.15.17:0/2969108751 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+2488 (secure 0 0 0) 0x7f1194023070 con 0x7f11a00b0630 2022-08-01T19:36:34.277 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:36:34.277 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/109603607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":109603607},{"type":"v1","addr":"172.21.15.17:6833","nonce":109603607}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":16},{"gid":5230,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/2722431349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2722431349},{"type":"v1","addr":"172.21.15.77:6835","nonce":2722431349}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":17}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:36:02.170790+0000","modified":"2022-08-01T19:36:03.173986+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5238},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5238":{"gid":5238,"name":"c","rank":0,"incarnation":17,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/3980246529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":3980246529},{"type":"v1","addr":"172.21.15.17:6835","nonce":3980246529}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-08-01T19:36:34.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.274+0000 7f11a57bc700 1 -- 172.21.15.17:0/2969108751 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f117c053aa0 msgr2=0x7f117c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:34.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.274+0000 7f11a57bc700 1 --2- 172.21.15.17:0/2969108751 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f117c053aa0 0x7f117c055f60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f119000b000 tx=0x7f1190006df0).stop 2022-08-01T19:36:34.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.274+0000 7f11a57bc700 1 -- 172.21.15.17:0/2969108751 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11a00b0630 msgr2=0x7f11a0065770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:36:34.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.274+0000 7f11a57bc700 1 --2- 172.21.15.17:0/2969108751 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11a00b0630 0x7f11a0065770 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f1198001b80 tx=0x7f1194014000).stop 2022-08-01T19:36:34.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.274+0000 7f11a57bc700 1 -- 172.21.15.17:0/2969108751 shutdown_connections 2022-08-01T19:36:34.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.274+0000 7f11a57bc700 1 --2- 172.21.15.17:0/2969108751 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f117c053aa0 0x7f117c055f60 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:34.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.274+0000 7f11a57bc700 1 --2- 172.21.15.17:0/2969108751 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f11a00b0a10 0x7f11a00abeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:34.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.274+0000 7f11a57bc700 1 --2- 172.21.15.17:0/2969108751 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11a00b0630 0x7f11a0065770 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:34.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.274+0000 7f11a57bc700 1 --2- 172.21.15.17:0/2969108751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11a00aa2b0 0x7f11a0139240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:36:34.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.274+0000 7f11a57bc700 1 -- 172.21.15.17:0/2969108751 >> 172.21.15.17:0/2969108751 conn(0x7f11a0060950 msgr2=0x7f11a00b5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:36:34.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.274+0000 7f11a57bc700 1 -- 172.21.15.17:0/2969108751 shutdown_connections 2022-08-01T19:36:34.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:36:34.274+0000 7f11a57bc700 1 -- 172.21.15.17:0/2969108751 wait complete. 2022-08-01T19:36:34.282 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 18 2022-08-01T19:36:34.593 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:36:02.170790+0000', 'modified': '2022-08-01T19:36:03.173986+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5238}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5238': {'gid': 5238, 'name': 'c', 'rank': 0, 'incarnation': 17, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/3980246529', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 3980246529}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 3980246529}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:36:34.594 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:36:34.595 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T19:36:34.712 INFO:teuthology.orchestra.run.smithi017.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":5238,"want_state":"up:active","state":"up:active","rank_uptime":32.529938274999999,"mdsmap_epoch":18,"osdmap_epoch":45,"osdmap_epoch_barrier":44,"uptime":36.902136704} 2022-08-01T19:36:34.713 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 5238, "mdsmap_epoch": 18, "osdmap_epoch": 45, "osdmap_epoch_barrier": 44, "rank_uptime": 32.529938275, "state": "up:active", "uptime": 36.902136704, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:36:34.714 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T19:36:34.714 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-08-01T19:36:34.714 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi085.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-08-01T19:36:34.715 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:36:34.715 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:36:34.724 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:36:34.737 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:36:34.738 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:36:34.794 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:36:34.795 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:36:34.796 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:36:34.797 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:36:34.804 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T19:36:34.805 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:36:34.854 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:36:34.872 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:36:34.873 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:36:34.884 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:36:34.885 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:36:34.886 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:36:34.886 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:36:34.927 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:2022-08-01T19:36:34.927+0000 7f5b35789080 -1 init, newargv = 0x555fcb8d8940 newargc=9ceph-fuse[ 2022-08-01T19:36:34.927 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:22662]: starting ceph client 2022-08-01T19:36:34.936 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:ceph-fuse[22662]: starting fuse 2022-08-01T19:36:36.035 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:36:36.061 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:36:36.062 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:36:36.124 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:36:36.125 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:36:36.125 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:36:36.126 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:36:36.134 INFO:teuthology.orchestra.run.smithi085.stdout:52 2022-08-01T19:36:36.135 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T19:36:36.136 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:36:36.137 DEBUG:teuthology.orchestra.run.smithi085:> sudo python3 -c ' 2022-08-01T19:36:36.138 DEBUG:teuthology.orchestra.run.smithi085:> import glob 2022-08-01T19:36:36.139 DEBUG:teuthology.orchestra.run.smithi085:> import re 2022-08-01T19:36:36.139 DEBUG:teuthology.orchestra.run.smithi085:> import os 2022-08-01T19:36:36.140 DEBUG:teuthology.orchestra.run.smithi085:> import subprocess 2022-08-01T19:36:36.141 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:36:36.142 DEBUG:teuthology.orchestra.run.smithi085:> def find_socket(client_name): 2022-08-01T19:36:36.143 DEBUG:teuthology.orchestra.run.smithi085:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-08-01T19:36:36.143 DEBUG:teuthology.orchestra.run.smithi085:> files = glob.glob(asok_path) 2022-08-01T19:36:36.144 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:36:36.145 DEBUG:teuthology.orchestra.run.smithi085:> # Given a non-glob path, it better be there 2022-08-01T19:36:36.146 DEBUG:teuthology.orchestra.run.smithi085:> if "*" not in asok_path: 2022-08-01T19:36:36.147 DEBUG:teuthology.orchestra.run.smithi085:> assert(len(files) == 1) 2022-08-01T19:36:36.147 DEBUG:teuthology.orchestra.run.smithi085:> return files[0] 2022-08-01T19:36:36.148 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:36:36.149 DEBUG:teuthology.orchestra.run.smithi085:> for f in files: 2022-08-01T19:36:36.150 DEBUG:teuthology.orchestra.run.smithi085:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T19:36:36.151 DEBUG:teuthology.orchestra.run.smithi085:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T19:36:36.151 DEBUG:teuthology.orchestra.run.smithi085:> return f 2022-08-01T19:36:36.152 DEBUG:teuthology.orchestra.run.smithi085:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T19:36:36.153 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:36:36.154 DEBUG:teuthology.orchestra.run.smithi085:> print(find_socket("client.0")) 2022-08-01T19:36:36.154 DEBUG:teuthology.orchestra.run.smithi085:> ' 2022-08-01T19:36:36.221 INFO:teuthology.orchestra.run.smithi085.stdout:/var/run/ceph/ceph-client.0.22662.asok 2022-08-01T19:36:36.224 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22662.asok 2022-08-01T19:36:36.225 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:36:36.225 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22662.asok status 2022-08-01T19:36:36.313 INFO:teuthology.orchestra.run.smithi085.stdout:{ 2022-08-01T19:36:36.314 INFO:teuthology.orchestra.run.smithi085.stdout: "metadata": { 2022-08-01T19:36:36.315 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T19:36:36.315 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T19:36:36.316 INFO:teuthology.orchestra.run.smithi085.stdout: "entity_id": "0", 2022-08-01T19:36:36.316 INFO:teuthology.orchestra.run.smithi085.stdout: "hostname": "smithi085", 2022-08-01T19:36:36.317 INFO:teuthology.orchestra.run.smithi085.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-08-01T19:36:36.317 INFO:teuthology.orchestra.run.smithi085.stdout: "pid": "22662", 2022-08-01T19:36:36.318 INFO:teuthology.orchestra.run.smithi085.stdout: "root": "/" 2022-08-01T19:36:36.318 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:36:36.319 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_count": 0, 2022-08-01T19:36:36.319 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_pinned_count": 0, 2022-08-01T19:36:36.320 INFO:teuthology.orchestra.run.smithi085.stdout: "id": 5491, 2022-08-01T19:36:36.320 INFO:teuthology.orchestra.run.smithi085.stdout: "inst": { 2022-08-01T19:36:36.321 INFO:teuthology.orchestra.run.smithi085.stdout: "name": { 2022-08-01T19:36:36.321 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "client", 2022-08-01T19:36:36.322 INFO:teuthology.orchestra.run.smithi085.stdout: "num": 5491 2022-08-01T19:36:36.323 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:36:36.323 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2022-08-01T19:36:36.323 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2022-08-01T19:36:36.324 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2022-08-01T19:36:36.324 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 348291646 2022-08-01T19:36:36.325 INFO:teuthology.orchestra.run.smithi085.stdout: } 2022-08-01T19:36:36.325 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:36:36.326 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2022-08-01T19:36:36.326 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2022-08-01T19:36:36.327 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2022-08-01T19:36:36.327 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 348291646 2022-08-01T19:36:36.327 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:36:36.328 INFO:teuthology.orchestra.run.smithi085.stdout: "inst_str": "client.5491 172.21.15.85:0/348291646", 2022-08-01T19:36:36.328 INFO:teuthology.orchestra.run.smithi085.stdout: "addr_str": "172.21.15.85:0/348291646", 2022-08-01T19:36:36.329 INFO:teuthology.orchestra.run.smithi085.stdout: "inode_count": 1, 2022-08-01T19:36:36.329 INFO:teuthology.orchestra.run.smithi085.stdout: "mds_epoch": 18, 2022-08-01T19:36:36.330 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch": 45, 2022-08-01T19:36:36.330 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch_barrier": 0, 2022-08-01T19:36:36.330 INFO:teuthology.orchestra.run.smithi085.stdout: "blacklisted": false, 2022-08-01T19:36:36.331 INFO:teuthology.orchestra.run.smithi085.stdout: "fs_name": "cephfs" 2022-08-01T19:36:36.331 INFO:teuthology.orchestra.run.smithi085.stdout:} 2022-08-01T19:36:36.334 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:36:36.335 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:36:36.335 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:36:36.345 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:36:36.346 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:36:36.347 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:36:36.348 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:36:47.228 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:36:47.232 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:36:47.235 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:36:47.238 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:37:17.352 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:37:17.356 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:37:17.359 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:37:17.361 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:37:22.810 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T19:37:22.811 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:37:22.831 DEBUG:teuthology.orchestra.run.smithi077:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:37:22.853 DEBUG:teuthology.orchestra.run.smithi085:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:37:22.875 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:37:22.897 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T19:37:22.897 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:37:22.898 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:37:22.898 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:37:22.907 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:37:22.908 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:37:22.909 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi085.front.sepia.ceph.com... 2022-08-01T19:37:22.909 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T19:37:22.910 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:37:22.924 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:ceph-fuse[22662]: fuse finished with error 0 and tester_r 0 2022-08-01T19:37:22.927 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:37:22.928 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:37:22.928 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:37:22.982 INFO:teuthology.orchestra.run.smithi085.stdout:ext2/ext3 2022-08-01T19:37:22.982 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T19:37:22.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T19:37:22.990 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:37:22.997 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:37:22.998 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:37:22.998 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:37:23.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:37:23.007 INFO:teuthology.orchestra.run.smithi190.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T19:37:23.008 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-08-01T19:37:23.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T19:37:23.009 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:37:23.057 INFO:teuthology.orchestra.run.smithi190.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T19:37:23.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:37:23.059 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T19:37:23.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.170+0000 7efc756db700 1 Processor -- start 2022-08-01T19:37:23.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.170+0000 7efc756db700 1 -- start start 2022-08-01T19:37:23.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.170+0000 7efc756db700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efc700b6710 0x7efc700b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:23.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.170+0000 7efc756db700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efc700b6af0 0x7efc7012bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:23.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.170+0000 7efc756db700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc70144240 con 0x7efc700b4550 2022-08-01T19:37:23.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.170+0000 7efc756db700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc70143d00 con 0x7efc700b6af0 2022-08-01T19:37:23.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.170+0000 7efc756db700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc70143fa0 con 0x7efc700b6710 2022-08-01T19:37:23.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.170+0000 7efc6f7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efc700b6af0 0x7efc7012bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:23.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.170+0000 7efc6f7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efc700b6af0 0x7efc7012bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38674/0 (socket says 172.21.15.17:38674) 2022-08-01T19:37:23.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.170+0000 7efc6f7fe700 1 -- 172.21.15.17:0/3123999414 learned_addr learned my addr 172.21.15.17:0/3123999414 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:23.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.170+0000 7efc6effd700 1 --2- 172.21.15.17:0/3123999414 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efc700b6710 0x7efc700b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:23.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.170+0000 7efc6f7fe700 1 -- 172.21.15.17:0/3123999414 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efc700b6710 msgr2=0x7efc700b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:23.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.170+0000 7efc6f7fe700 1 --2- 172.21.15.17:0/3123999414 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efc700b6710 0x7efc700b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:23.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.170+0000 7efc6f7fe700 1 -- 172.21.15.17:0/3123999414 >> v1:172.21.15.77:6789/0 conn(0x7efc700b4550 legacy=0x7efc70061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:23.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.170+0000 7efc6f7fe700 1 -- 172.21.15.17:0/3123999414 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc70061780 con 0x7efc700b6af0 2022-08-01T19:37:23.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.170+0000 7efc6f7fe700 1 --2- 172.21.15.17:0/3123999414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc700b6af0 0x7efc7012bb90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7efc680032f0 tx=0x7efc68009290).ready entity=mon.0 client_cookie=59a11df05ae61b8e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:23.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.170+0000 7efc6e7fc700 1 -- 172.21.15.17:0/3123999414 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc6800b8a0 con 0x7efc700b6af0 2022-08-01T19:37:23.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.170+0000 7efc6e7fc700 1 -- 172.21.15.17:0/3123999414 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc6800ba00 con 0x7efc700b6af0 2022-08-01T19:37:23.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.170+0000 7efc6e7fc700 1 -- 172.21.15.17:0/3123999414 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc6800bb70 con 0x7efc700b6af0 2022-08-01T19:37:23.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.170+0000 7efc756db700 1 -- 172.21.15.17:0/3123999414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc700b6af0 msgr2=0x7efc7012bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:23.188 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.170+0000 7efc756db700 1 --2- 172.21.15.17:0/3123999414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc700b6af0 0x7efc7012bb90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7efc680032f0 tx=0x7efc68009290).stop 2022-08-01T19:37:23.188 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.170+0000 7efc756db700 1 -- 172.21.15.17:0/3123999414 shutdown_connections 2022-08-01T19:37:23.188 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.170+0000 7efc756db700 1 --2- 172.21.15.17:0/3123999414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc700b6af0 0x7efc7012bb90 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:23.188 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.170+0000 7efc756db700 1 --2- 172.21.15.17:0/3123999414 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efc700b6710 0x7efc700b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:23.188 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.170+0000 7efc756db700 1 -- 172.21.15.17:0/3123999414 >> 172.21.15.17:0/3123999414 conn(0x7efc7005c620 msgr2=0x7efc7005ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:23.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.170+0000 7efc756db700 1 -- 172.21.15.17:0/3123999414 shutdown_connections 2022-08-01T19:37:23.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.170+0000 7efc756db700 1 -- 172.21.15.17:0/3123999414 wait complete. 2022-08-01T19:37:23.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.174+0000 7efc756db700 1 Processor -- start 2022-08-01T19:37:23.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.174+0000 7efc756db700 1 -- start start 2022-08-01T19:37:23.190 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.174+0000 7efc756db700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efc700b4550 0x7efc700b3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:23.190 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.174+0000 7efc756db700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc700b6710 0x7efc701449c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:23.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.174+0000 7efc756db700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc700b6af0 0x7efc70124fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:23.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.174+0000 7efc756db700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc701254a0 con 0x7efc700b6710 2022-08-01T19:37:23.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.174+0000 7efc756db700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc700b3470 con 0x7efc700b6af0 2022-08-01T19:37:23.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.174+0000 7efc756db700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc70155ab0 con 0x7efc700b4550 2022-08-01T19:37:23.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.174+0000 7efc667fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc700b6710 0x7efc701449c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:23.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.174+0000 7efc6f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc700b6af0 0x7efc70124fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:23.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.174+0000 7efc667fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc700b6710 0x7efc701449c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38678/0 (socket says 172.21.15.17:38678) 2022-08-01T19:37:23.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.174+0000 7efc667fc700 1 -- 172.21.15.17:0/946839396 learned_addr learned my addr 172.21.15.17:0/946839396 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:23.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.174+0000 7efc6effd700 1 --2- 172.21.15.17:0/946839396 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efc700b4550 0x7efc700b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:23.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.174+0000 7efc667fc700 1 -- 172.21.15.17:0/946839396 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efc700b4550 msgr2=0x7efc700b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:23.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.174+0000 7efc667fc700 1 --2- 172.21.15.17:0/946839396 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efc700b4550 0x7efc700b3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:23.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.174+0000 7efc667fc700 1 -- 172.21.15.17:0/946839396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc700b6af0 msgr2=0x7efc70124fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:23.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.174+0000 7efc667fc700 1 --2- 172.21.15.17:0/946839396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc700b6af0 0x7efc70124fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:23.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.174+0000 7efc667fc700 1 -- 172.21.15.17:0/946839396 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc68009530 con 0x7efc700b6710 2022-08-01T19:37:23.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.174+0000 7efc667fc700 1 --2- 172.21.15.17:0/946839396 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc700b6710 0x7efc701449c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7efc70061100 tx=0x7efc58007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:23.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.174+0000 7efc6cff9700 1 -- 172.21.15.17:0/946839396 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc58007030 con 0x7efc700b6710 2022-08-01T19:37:23.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.174+0000 7efc6cff9700 1 -- 172.21.15.17:0/946839396 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc58007190 con 0x7efc700b6710 2022-08-01T19:37:23.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.174+0000 7efc756db700 1 -- 172.21.15.17:0/946839396 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc70155d50 con 0x7efc700b6710 2022-08-01T19:37:23.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.174+0000 7efc6cff9700 1 -- 172.21.15.17:0/946839396 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc58006510 con 0x7efc700b6710 2022-08-01T19:37:23.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.174+0000 7efc756db700 1 -- 172.21.15.17:0/946839396 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc700b2f10 con 0x7efc700b6710 2022-08-01T19:37:23.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.178+0000 7efc756db700 1 -- 172.21.15.17:0/946839396 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc700b3670 con 0x7efc700b6710 2022-08-01T19:37:23.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.178+0000 7efc6cff9700 1 -- 172.21.15.17:0/946839396 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7efc58010030 con 0x7efc700b6710 2022-08-01T19:37:23.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.178+0000 7efc6cff9700 1 --2- 172.21.15.17:0/946839396 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efc54053a70 0x7efc54055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:23.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.178+0000 7efc6cff9700 1 -- 172.21.15.17:0/946839396 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7efc5806bc50 con 0x7efc700b6710 2022-08-01T19:37:23.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.178+0000 7efc6effd700 1 --2- 172.21.15.17:0/946839396 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efc54053a70 0x7efc54055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:23.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.178+0000 7efc6effd700 1 --2- 172.21.15.17:0/946839396 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efc54053a70 0x7efc54055f30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7efc6000b000 tx=0x7efc60006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:23.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.182+0000 7efc6cff9700 1 -- 172.21.15.17:0/946839396 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efc58042a10 con 0x7efc700b6710 2022-08-01T19:37:23.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.310+0000 7efc756db700 1 -- 172.21.15.17:0/946839396 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7efc70155ee0 con 0x7efc700b6710 2022-08-01T19:37:23.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.314+0000 7efc6cff9700 1 -- 172.21.15.17:0/946839396 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v45) v1 ==== 81+0+34037 (secure 0 0 0) 0x7efc58046030 con 0x7efc700b6710 2022-08-01T19:37:23.319 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:37:23.319 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:37:23.320 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 45, 2022-08-01T19:37:23.320 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", 2022-08-01T19:37:23.320 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2022-08-01T19:33:50.105743+0000", 2022-08-01T19:37:23.320 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2022-08-01T19:36:03.174642+0000", 2022-08-01T19:37:23.321 INFO:teuthology.orchestra.run.smithi017.stdout: "last_up_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:37:23.321 INFO:teuthology.orchestra.run.smithi017.stdout: "last_in_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:37:23.321 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T19:37:23.321 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_num": 5799936, 2022-08-01T19:37:23.322 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_set": [ 2022-08-01T19:37:23.322 INFO:teuthology.orchestra.run.smithi017.stdout: "pglog_hardlimit", 2022-08-01T19:37:23.322 INFO:teuthology.orchestra.run.smithi017.stdout: "purged_snapdirs", 2022-08-01T19:37:23.323 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_deletes", 2022-08-01T19:37:23.323 INFO:teuthology.orchestra.run.smithi017.stdout: "sortbitwise" 2022-08-01T19:37:23.323 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:37:23.323 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_version": 6, 2022-08-01T19:37:23.324 INFO:teuthology.orchestra.run.smithi017.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T19:37:23.324 INFO:teuthology.orchestra.run.smithi017.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T19:37:23.324 INFO:teuthology.orchestra.run.smithi017.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T19:37:23.324 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_snapshot": "", 2022-08-01T19:37:23.325 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_max": 8, 2022-08-01T19:37:23.325 INFO:teuthology.orchestra.run.smithi017.stdout: "max_osd": 8, 2022-08-01T19:37:23.325 INFO:teuthology.orchestra.run.smithi017.stdout: "require_min_compat_client": "jewel", 2022-08-01T19:37:23.325 INFO:teuthology.orchestra.run.smithi017.stdout: "min_compat_client": "jewel", 2022-08-01T19:37:23.326 INFO:teuthology.orchestra.run.smithi017.stdout: "require_osd_release": "octopus", 2022-08-01T19:37:23.326 INFO:teuthology.orchestra.run.smithi017.stdout: "pools": [ 2022-08-01T19:37:23.326 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.326 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 1, 2022-08-01T19:37:23.327 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "device_health_metrics", 2022-08-01T19:37:23.327 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:33:53.001841+0000", 2022-08-01T19:37:23.327 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:37:23.327 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:37:23.328 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:37:23.328 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:37:23.328 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:37:23.328 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:37:23.329 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:37:23.329 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:37:23.329 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 1, 2022-08-01T19:37:23.329 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 1, 2022-08-01T19:37:23.330 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 1, 2022-08-01T19:37:23.330 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 1, 2022-08-01T19:37:23.330 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 1, 2022-08-01T19:37:23.330 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:37:23.331 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:37:23.331 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:37:23.331 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:37:23.331 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:37:23.332 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:37:23.332 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:37:23.332 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.332 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "16", 2022-08-01T19:37:23.333 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:37:23.333 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:37:23.333 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:37:23.333 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:37:23.334 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:37:23.334 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:37:23.334 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:37:23.334 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:37:23.335 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:37:23.335 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:37:23.335 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:37:23.335 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:37:23.336 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:37:23.336 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:37:23.336 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:37:23.337 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:37:23.337 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:37:23.337 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:37:23.337 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:37:23.338 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:37:23.338 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:37:23.338 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:37:23.339 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:37:23.339 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:37:23.339 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:37:23.339 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:37:23.340 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.340 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:37:23.340 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:37:23.340 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:37:23.340 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:37:23.341 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:37:23.341 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:37:23.341 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:37:23.341 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:37:23.342 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:37:23.342 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:37:23.342 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:37:23.342 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:37:23.343 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 1 2022-08-01T19:37:23.343 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.343 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:37:23.344 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_devicehealth": {} 2022-08-01T19:37:23.344 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.344 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.344 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.345 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 2, 2022-08-01T19:37:23.345 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "rbd", 2022-08-01T19:37:23.346 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:34:01.553098+0000", 2022-08-01T19:37:23.346 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:37:23.347 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:37:23.347 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:37:23.348 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:37:23.348 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:37:23.349 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:37:23.349 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:37:23.349 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:37:23.350 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:37:23.350 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:37:23.351 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:37:23.351 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:37:23.352 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:37:23.353 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:37:23.353 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:37:23.353 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:37:23.353 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:37:23.354 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:37:23.354 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:37:23.354 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:37:23.354 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.355 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "19", 2022-08-01T19:37:23.355 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:37:23.355 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:37:23.355 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:37:23.356 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:37:23.356 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:37:23.356 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:37:23.357 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:37:23.357 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:37:23.357 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:37:23.357 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:37:23.357 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:37:23.358 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:37:23.358 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:37:23.358 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:37:23.358 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:37:23.359 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:37:23.359 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:37:23.359 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:37:23.359 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:37:23.360 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:37:23.360 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:37:23.360 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:37:23.360 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:37:23.361 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:37:23.361 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:37:23.362 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:37:23.362 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.362 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:37:23.362 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:37:23.363 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:37:23.363 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:37:23.363 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:37:23.363 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:37:23.364 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:37:23.364 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:37:23.364 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:37:23.364 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:37:23.365 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:37:23.365 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:37:23.365 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:37:23.365 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd": {} 2022-08-01T19:37:23.366 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.366 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.366 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.366 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 7, 2022-08-01T19:37:23.367 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "cephfs_metadata", 2022-08-01T19:37:23.367 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:36:00.565402+0000", 2022-08-01T19:37:23.367 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:37:23.367 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:37:23.368 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:37:23.368 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:37:23.368 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:37:23.368 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:37:23.369 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:37:23.369 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:37:23.369 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:37:23.369 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:37:23.369 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:37:23.370 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:37:23.370 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:37:23.370 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:37:23.370 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:37:23.371 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:37:23.371 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:37:23.371 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:37:23.371 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:37:23.372 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:37:23.372 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.372 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "45", 2022-08-01T19:37:23.372 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:37:23.373 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:37:23.373 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:37:23.373 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:37:23.373 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:37:23.374 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:37:23.374 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:37:23.374 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:37:23.374 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:37:23.375 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:37:23.375 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:37:23.375 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:37:23.375 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:37:23.376 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:37:23.376 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:37:23.376 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:37:23.376 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:37:23.377 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:37:23.377 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:37:23.377 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:37:23.377 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:37:23.378 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:37:23.378 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:37:23.378 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:37:23.379 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:37:23.379 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:37:23.379 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.379 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:37:23.379 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:37:23.380 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:37:23.380 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:37:23.380 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:37:23.380 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:37:23.381 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:37:23.381 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:37:23.381 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:37:23.381 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:37:23.382 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:37:23.382 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:37:23.382 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_bias": 4, 2022-08-01T19:37:23.383 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 16, 2022-08-01T19:37:23.383 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_priority": 5 2022-08-01T19:37:23.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.383 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:37:23.384 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs": { 2022-08-01T19:37:23.384 INFO:teuthology.orchestra.run.smithi017.stdout: "metadata": "cephfs" 2022-08-01T19:37:23.384 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.384 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.384 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.385 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.385 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 8, 2022-08-01T19:37:23.385 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "cephfs_data", 2022-08-01T19:37:23.385 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:36:01.813338+0000", 2022-08-01T19:37:23.386 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:37:23.386 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:37:23.387 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:37:23.387 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:37:23.388 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:37:23.388 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:37:23.389 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:37:23.389 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:37:23.390 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:37:23.390 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:37:23.391 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:37:23.391 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:37:23.392 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:37:23.392 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:37:23.393 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:37:23.393 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:37:23.394 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:37:23.395 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:37:23.395 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:37:23.396 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:37:23.396 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.397 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "45", 2022-08-01T19:37:23.397 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:37:23.397 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:37:23.398 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:37:23.398 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:37:23.398 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:37:23.398 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:37:23.399 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:37:23.399 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:37:23.399 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:37:23.399 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:37:23.400 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:37:23.400 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:37:23.400 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:37:23.400 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:37:23.401 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:37:23.401 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:37:23.401 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:37:23.401 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:37:23.402 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:37:23.402 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:37:23.402 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:37:23.403 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:37:23.403 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:37:23.404 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:37:23.404 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:37:23.405 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:37:23.405 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.406 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:37:23.406 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:37:23.407 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:37:23.407 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:37:23.407 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:37:23.407 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:37:23.408 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:37:23.408 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:37:23.408 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:37:23.409 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:37:23.409 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:37:23.409 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:37:23.409 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:37:23.410 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs": { 2022-08-01T19:37:23.410 INFO:teuthology.orchestra.run.smithi017.stdout: "data": "cephfs" 2022-08-01T19:37:23.410 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.410 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.411 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.411 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:37:23.411 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2022-08-01T19:37:23.411 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.412 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:37:23.412 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a962f14f-cf16-4fa8-84ff-c85325db9132", 2022-08-01T19:37:23.412 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:37:23.412 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:37:23.413 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:37:23.413 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:37:23.413 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:37:23.413 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:37:23.414 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:37:23.414 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 43, 2022-08-01T19:37:23.414 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:37:23.414 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:37:23.415 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:37:23.415 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:23.415 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.415 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:23.416 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6800", 2022-08-01T19:37:23.416 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:37:23.416 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.416 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.417 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:23.417 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6801", 2022-08-01T19:37:23.417 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:37:23.417 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.417 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.418 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.418 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:37:23.418 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:23.418 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.419 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:23.419 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6802", 2022-08-01T19:37:23.419 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:37:23.419 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.420 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.420 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:23.420 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6803", 2022-08-01T19:37:23.420 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:37:23.421 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.421 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.421 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.421 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:37:23.422 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:23.422 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.422 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:23.422 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6806", 2022-08-01T19:37:23.423 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:37:23.423 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.423 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.423 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:23.423 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6807", 2022-08-01T19:37:23.424 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:37:23.424 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.424 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.424 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.425 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:37:23.425 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:23.425 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.425 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:23.426 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6804", 2022-08-01T19:37:23.426 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:37:23.426 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.426 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.427 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:23.427 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6805", 2022-08-01T19:37:23.427 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:37:23.427 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.427 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.428 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.428 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6801/23008", 2022-08-01T19:37:23.428 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6803/23008", 2022-08-01T19:37:23.428 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6807/23008", 2022-08-01T19:37:23.429 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6805/23008", 2022-08-01T19:37:23.429 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:37:23.429 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:37:23.429 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:37:23.430 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.430 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.430 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.430 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:37:23.431 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "805f7051-a527-4e98-8844-b6e406a63248", 2022-08-01T19:37:23.431 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:37:23.431 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:37:23.431 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:37:23.432 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:37:23.432 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:37:23.432 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:37:23.432 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:37:23.432 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 43, 2022-08-01T19:37:23.433 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:37:23.433 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:37:23.433 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:37:23.433 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:23.434 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.434 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:23.434 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6808", 2022-08-01T19:37:23.434 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:37:23.435 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.435 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.435 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:23.435 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6809", 2022-08-01T19:37:23.436 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:37:23.436 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.436 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.436 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.436 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:37:23.437 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:23.437 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.437 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:23.437 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6810", 2022-08-01T19:37:23.438 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:37:23.438 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.438 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.438 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:23.439 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6811", 2022-08-01T19:37:23.439 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:37:23.439 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.439 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.440 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.440 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:37:23.440 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:23.440 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.441 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:23.441 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6814", 2022-08-01T19:37:23.441 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:37:23.441 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.441 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.442 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:23.442 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6815", 2022-08-01T19:37:23.442 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:37:23.442 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.443 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.443 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.443 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:37:23.443 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:23.444 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.444 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:23.444 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6812", 2022-08-01T19:37:23.444 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:37:23.445 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.445 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.445 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:23.445 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6813", 2022-08-01T19:37:23.446 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:37:23.446 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.446 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.446 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.446 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6809/23007", 2022-08-01T19:37:23.447 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6811/23007", 2022-08-01T19:37:23.447 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6815/23007", 2022-08-01T19:37:23.447 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6813/23007", 2022-08-01T19:37:23.447 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:37:23.448 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:37:23.448 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:37:23.448 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.448 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.449 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.449 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:37:23.449 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "89f7f3df-7c10-48e1-8b69-9e4243923a61", 2022-08-01T19:37:23.449 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:37:23.450 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:37:23.450 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:37:23.459 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:37:23.459 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:37:23.459 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:37:23.460 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:37:23.460 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 32, 2022-08-01T19:37:23.460 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:37:23.460 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:37:23.461 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:37:23.461 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:23.461 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.461 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:23.462 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6817", 2022-08-01T19:37:23.462 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:37:23.462 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.462 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.463 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:23.463 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6819", 2022-08-01T19:37:23.463 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:37:23.463 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.464 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.464 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.464 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:37:23.464 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:23.465 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.465 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:23.465 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6822", 2022-08-01T19:37:23.465 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:37:23.465 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.466 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.466 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:23.466 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6824", 2022-08-01T19:37:23.466 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:37:23.467 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.467 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.467 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.467 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:37:23.468 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:23.468 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.468 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:23.468 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6830", 2022-08-01T19:37:23.469 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:37:23.469 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.469 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.469 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:23.470 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6831", 2022-08-01T19:37:23.470 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:37:23.470 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.470 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.471 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.471 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:37:23.471 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:23.471 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.472 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:23.472 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6826", 2022-08-01T19:37:23.472 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:37:23.472 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.473 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.473 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:23.473 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6829", 2022-08-01T19:37:23.473 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:37:23.474 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.474 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.474 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.474 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6819/23010", 2022-08-01T19:37:23.475 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6824/23010", 2022-08-01T19:37:23.475 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6831/23010", 2022-08-01T19:37:23.475 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6829/23010", 2022-08-01T19:37:23.475 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:37:23.475 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:37:23.476 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:37:23.476 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.476 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.477 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.477 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:37:23.477 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "96123468-aabc-4a64-909e-a192c369090d", 2022-08-01T19:37:23.477 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:37:23.478 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:37:23.478 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:37:23.478 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:37:23.478 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:37:23.479 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:37:23.479 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:37:23.479 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 43, 2022-08-01T19:37:23.479 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:37:23.479 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:37:23.480 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:37:23.480 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:23.480 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.480 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:23.481 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6816", 2022-08-01T19:37:23.481 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:37:23.481 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.481 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.482 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:23.482 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6818", 2022-08-01T19:37:23.482 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:37:23.482 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.483 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.483 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.483 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:37:23.483 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:23.484 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.484 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:23.484 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6820", 2022-08-01T19:37:23.484 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:37:23.484 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.485 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.485 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:23.485 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6821", 2022-08-01T19:37:23.485 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:37:23.486 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.486 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.486 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.486 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:37:23.487 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:23.487 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.487 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:23.487 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6827", 2022-08-01T19:37:23.488 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:37:23.488 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.488 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.488 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:23.488 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6828", 2022-08-01T19:37:23.489 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:37:23.489 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.489 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.489 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.490 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:37:23.490 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:23.490 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.490 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:23.491 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6823", 2022-08-01T19:37:23.491 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:37:23.491 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.491 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.492 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:23.492 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6825", 2022-08-01T19:37:23.492 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:37:23.492 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.493 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.493 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.493 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6818/23009", 2022-08-01T19:37:23.493 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6821/23009", 2022-08-01T19:37:23.494 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6828/23009", 2022-08-01T19:37:23.494 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6825/23009", 2022-08-01T19:37:23.494 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:37:23.494 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:37:23.495 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:37:23.495 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.495 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.495 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.495 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:37:23.496 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a6e26665-e04b-46ec-ab66-e912acbf044c", 2022-08-01T19:37:23.496 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:37:23.496 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:37:23.496 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:37:23.497 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:37:23.497 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:37:23.497 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:37:23.497 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:37:23.498 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 43, 2022-08-01T19:37:23.498 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:37:23.498 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:37:23.498 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:37:23.499 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:23.499 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.499 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:23.499 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6810", 2022-08-01T19:37:23.499 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:37:23.500 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.500 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.500 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:23.500 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6811", 2022-08-01T19:37:23.501 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:37:23.501 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.501 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.501 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.502 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:37:23.502 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:23.502 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.502 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:23.503 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6812", 2022-08-01T19:37:23.503 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:37:23.503 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.503 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.503 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:23.504 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6813", 2022-08-01T19:37:23.504 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:37:23.505 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.505 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.505 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.506 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:37:23.506 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:23.507 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.507 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:23.507 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6816", 2022-08-01T19:37:23.508 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:37:23.508 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.508 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.508 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:23.509 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6817", 2022-08-01T19:37:23.509 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:37:23.509 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.510 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.510 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.510 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:37:23.510 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:23.511 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.511 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:23.511 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6814", 2022-08-01T19:37:23.511 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:37:23.511 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.512 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.512 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:23.512 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6815", 2022-08-01T19:37:23.512 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:37:23.513 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.513 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.513 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.513 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6811/23584", 2022-08-01T19:37:23.514 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6813/23584", 2022-08-01T19:37:23.514 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6817/23584", 2022-08-01T19:37:23.514 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6815/23584", 2022-08-01T19:37:23.514 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:37:23.515 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:37:23.515 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:37:23.515 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.515 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.515 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.516 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:37:23.516 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "502c0034-18ee-44ac-87ff-19b9dddb66a0", 2022-08-01T19:37:23.516 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:37:23.516 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:37:23.517 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:37:23.517 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:37:23.517 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:37:23.517 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:37:23.517 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:37:23.518 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 43, 2022-08-01T19:37:23.518 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:37:23.518 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:37:23.519 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:37:23.519 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:23.519 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.520 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:23.520 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6802", 2022-08-01T19:37:23.520 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:37:23.520 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.520 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.521 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:23.521 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6803", 2022-08-01T19:37:23.521 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:37:23.521 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.522 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.522 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.523 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:37:23.523 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:23.523 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.523 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:23.523 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6804", 2022-08-01T19:37:23.524 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:37:23.524 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.525 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.525 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:23.525 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6805", 2022-08-01T19:37:23.526 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:37:23.526 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.527 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.527 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.528 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:37:23.528 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:23.528 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.529 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:23.529 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6808", 2022-08-01T19:37:23.530 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:37:23.530 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.531 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.531 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:23.532 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6809", 2022-08-01T19:37:23.532 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:37:23.533 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.533 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.533 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.534 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:37:23.534 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:23.535 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.535 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:23.536 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6806", 2022-08-01T19:37:23.536 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:37:23.536 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.537 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.537 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:23.537 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6807", 2022-08-01T19:37:23.537 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:37:23.538 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.538 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.538 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.539 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6803/23585", 2022-08-01T19:37:23.539 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6805/23585", 2022-08-01T19:37:23.539 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6809/23585", 2022-08-01T19:37:23.539 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6807/23585", 2022-08-01T19:37:23.540 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:37:23.540 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:37:23.540 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:37:23.540 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.541 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.541 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.541 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:37:23.541 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "ac4e755a-7fe7-46cb-8407-852d5af9ffc5", 2022-08-01T19:37:23.542 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:37:23.542 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:37:23.542 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:37:23.542 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:37:23.543 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:37:23.543 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:37:23.543 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:37:23.543 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 43, 2022-08-01T19:37:23.544 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:37:23.544 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:37:23.544 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:37:23.544 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:23.544 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.545 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:23.545 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6820", 2022-08-01T19:37:23.545 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:37:23.545 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.546 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.546 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:23.546 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6821", 2022-08-01T19:37:23.546 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:37:23.547 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.547 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.547 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.547 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:37:23.548 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:23.548 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.548 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:23.548 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6823", 2022-08-01T19:37:23.548 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:37:23.549 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.549 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.549 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:23.550 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6825", 2022-08-01T19:37:23.550 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:37:23.550 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.550 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.551 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.551 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:37:23.551 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:23.551 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.551 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:23.552 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6831", 2022-08-01T19:37:23.552 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:37:23.552 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.552 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.553 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:23.553 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6833", 2022-08-01T19:37:23.553 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:37:23.553 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.553 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.554 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.554 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:37:23.554 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:23.555 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.555 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:23.556 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6827", 2022-08-01T19:37:23.556 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:37:23.556 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.557 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.557 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:23.558 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6829", 2022-08-01T19:37:23.558 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:37:23.558 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.559 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.559 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.559 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6821/23587", 2022-08-01T19:37:23.559 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6825/23587", 2022-08-01T19:37:23.560 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6833/23587", 2022-08-01T19:37:23.560 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6829/23587", 2022-08-01T19:37:23.560 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:37:23.561 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:37:23.561 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:37:23.561 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.561 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.562 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.562 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:37:23.562 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "874761e2-6410-40fc-b60c-e58abe5ce489", 2022-08-01T19:37:23.562 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:37:23.562 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:37:23.563 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:37:23.563 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:37:23.564 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:37:23.564 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:37:23.564 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:37:23.564 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 42, 2022-08-01T19:37:23.564 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:37:23.565 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:37:23.565 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:37:23.566 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:23.566 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.567 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:23.567 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6818", 2022-08-01T19:37:23.567 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:37:23.568 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.568 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.568 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:23.569 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6819", 2022-08-01T19:37:23.569 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:37:23.570 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.570 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.570 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.570 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:37:23.571 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:23.571 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.571 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:23.571 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6822", 2022-08-01T19:37:23.572 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:37:23.572 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.572 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.573 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:23.573 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6824", 2022-08-01T19:37:23.573 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:37:23.573 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.574 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.574 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.574 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:37:23.574 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:23.575 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.575 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:23.575 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6830", 2022-08-01T19:37:23.575 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:37:23.576 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.576 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.576 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:23.576 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6832", 2022-08-01T19:37:23.576 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:37:23.577 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.577 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.577 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.577 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:37:23.578 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:23.578 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.578 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:23.578 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6826", 2022-08-01T19:37:23.578 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:37:23.579 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.579 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.579 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:23.579 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6828", 2022-08-01T19:37:23.580 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:37:23.580 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.580 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.580 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.581 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6819/23586", 2022-08-01T19:37:23.581 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6824/23586", 2022-08-01T19:37:23.581 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6832/23586", 2022-08-01T19:37:23.581 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6828/23586", 2022-08-01T19:37:23.581 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:37:23.582 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:37:23.582 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:37:23.582 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:23.582 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.583 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:37:23.583 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_xinfo": [ 2022-08-01T19:37:23.583 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.583 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:37:23.584 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:37:23.584 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:37:23.584 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:37:23.584 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:37:23.584 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:37:23.585 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.588817+0000", 2022-08-01T19:37:23.585 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:37:23.585 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.585 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.586 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:37:23.586 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:37:23.586 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:37:23.586 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:37:23.586 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:37:23.587 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:37:23.587 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.782753+0000", 2022-08-01T19:37:23.587 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:37:23.587 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.588 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.588 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:37:23.588 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:37:23.588 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:37:23.588 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:37:23.589 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:37:23.589 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:37:23.589 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.837924+0000", 2022-08-01T19:37:23.589 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:37:23.590 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.590 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.590 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:37:23.590 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:37:23.591 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:37:23.591 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:37:23.591 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:37:23.591 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:37:23.592 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.839948+0000", 2022-08-01T19:37:23.592 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:37:23.592 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.592 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.592 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:37:23.593 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:37:23.593 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:37:23.593 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:37:23.593 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:37:23.594 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:37:23.594 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.112801+0000", 2022-08-01T19:37:23.594 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:37:23.594 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.595 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.595 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:37:23.595 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:37:23.595 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:37:23.596 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:37:23.596 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:37:23.596 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:37:23.596 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.106704+0000", 2022-08-01T19:37:23.596 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:37:23.597 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.597 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.597 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:37:23.597 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:37:23.598 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:37:23.598 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:37:23.598 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:37:23.598 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:37:23.599 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.893170+0000", 2022-08-01T19:37:23.599 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:37:23.599 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.599 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:23.600 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:37:23.600 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:37:23.600 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:37:23.600 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:37:23.601 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:37:23.601 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:37:23.601 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.886947+0000", 2022-08-01T19:37:23.601 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:37:23.601 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.602 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:37:23.602 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap": [], 2022-08-01T19:37:23.602 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap_items": [], 2022-08-01T19:37:23.602 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_temp": [], 2022-08-01T19:37:23.603 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_temp": [], 2022-08-01T19:37:23.603 INFO:teuthology.orchestra.run.smithi017.stdout: "blacklist": {}, 2022-08-01T19:37:23.603 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profiles": { 2022-08-01T19:37:23.603 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_data": { 2022-08-01T19:37:23.604 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-device-class": "", 2022-08-01T19:37:23.604 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:37:23.604 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-root": "default", 2022-08-01T19:37:23.604 INFO:teuthology.orchestra.run.smithi017.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T19:37:23.604 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:37:23.605 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "2", 2022-08-01T19:37:23.605 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:37:23.605 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:37:23.605 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van", 2022-08-01T19:37:23.606 INFO:teuthology.orchestra.run.smithi017.stdout: "w": "8" 2022-08-01T19:37:23.606 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.606 INFO:teuthology.orchestra.run.smithi017.stdout: "default": { 2022-08-01T19:37:23.606 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:37:23.607 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:37:23.607 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "1", 2022-08-01T19:37:23.607 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:37:23.607 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:37:23.607 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van" 2022-08-01T19:37:23.608 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:23.608 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:23.608 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps_queue": [], 2022-08-01T19:37:23.608 INFO:teuthology.orchestra.run.smithi017.stdout: "new_removed_snaps": [], 2022-08-01T19:37:23.609 INFO:teuthology.orchestra.run.smithi017.stdout: "new_purged_snaps": [], 2022-08-01T19:37:23.609 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_node_flags": {}, 2022-08-01T19:37:23.609 INFO:teuthology.orchestra.run.smithi017.stdout: "device_class_flags": {} 2022-08-01T19:37:23.609 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:37:23.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.318+0000 7efc756db700 1 -- 172.21.15.17:0/946839396 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efc54053a70 msgr2=0x7efc54055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:23.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.318+0000 7efc756db700 1 --2- 172.21.15.17:0/946839396 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efc54053a70 0x7efc54055f30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7efc6000b000 tx=0x7efc60006df0).stop 2022-08-01T19:37:23.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.318+0000 7efc756db700 1 -- 172.21.15.17:0/946839396 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc700b6710 msgr2=0x7efc701449c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:23.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.318+0000 7efc756db700 1 --2- 172.21.15.17:0/946839396 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc700b6710 0x7efc701449c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7efc70061100 tx=0x7efc58007be0).stop 2022-08-01T19:37:23.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.318+0000 7efc756db700 1 -- 172.21.15.17:0/946839396 shutdown_connections 2022-08-01T19:37:23.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.318+0000 7efc756db700 1 --2- 172.21.15.17:0/946839396 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efc54053a70 0x7efc54055f30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:23.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.318+0000 7efc756db700 1 --2- 172.21.15.17:0/946839396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc700b6af0 0x7efc70124fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:23.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.318+0000 7efc756db700 1 --2- 172.21.15.17:0/946839396 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc700b6710 0x7efc701449c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:23.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.318+0000 7efc756db700 1 --2- 172.21.15.17:0/946839396 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efc700b4550 0x7efc700b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:23.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.318+0000 7efc756db700 1 -- 172.21.15.17:0/946839396 >> 172.21.15.17:0/946839396 conn(0x7efc7005c620 msgr2=0x7efc70141120 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:23.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.318+0000 7efc756db700 1 -- 172.21.15.17:0/946839396 shutdown_connections 2022-08-01T19:37:23.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.318+0000 7efc756db700 1 -- 172.21.15.17:0/946839396 wait complete. 2022-08-01T19:37:23.615 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:37:23.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.722+0000 7f077ffff700 1 Processor -- start 2022-08-01T19:37:23.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.722+0000 7f077ffff700 1 -- start start 2022-08-01T19:37:23.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.722+0000 7f077ffff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f07800b08a0 0x7f0780142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:23.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.722+0000 7f077ffff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f07800ad000 0x7f07801433a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:23.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.722+0000 7f077ffff700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07800b0c80 con 0x7f07800b3260 2022-08-01T19:37:23.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.722+0000 7f077ffff700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f078005cd30 con 0x7f07800b08a0 2022-08-01T19:37:23.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.722+0000 7f077ffff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07800ae810 con 0x7f07800ad000 2022-08-01T19:37:23.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.722+0000 7f077effd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f07800b08a0 0x7f0780142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:23.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.722+0000 7f077e7fc700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f07800b3260 0x7f078005c620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59112/0 (socket says 172.21.15.17:59112) 2022-08-01T19:37:23.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.722+0000 7f077e7fc700 1 -- 172.21.15.17:0/1301678363 learned_addr learned my addr 172.21.15.17:0/1301678363 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:37:23.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.722+0000 7f077f7fe700 1 --2- 172.21.15.17:0/1301678363 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f07800ad000 0x7f07801433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:23.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.722+0000 7f077dffb700 1 -- 172.21.15.17:0/1301678363 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3837265467 0 0) 0x7f07800b0c80 con 0x7f07800b3260 2022-08-01T19:37:23.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.722+0000 7f077dffb700 1 -- 172.21.15.17:0/1301678363 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0760003260 con 0x7f07800b3260 2022-08-01T19:37:23.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.722+0000 7f077effd700 1 -- 172.21.15.17:0/1301678363 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f07800ad000 msgr2=0x7f07801433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:23.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.722+0000 7f077effd700 1 --2- 172.21.15.17:0/1301678363 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f07800ad000 0x7f07801433a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:23.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.722+0000 7f077effd700 1 -- 172.21.15.17:0/1301678363 >> v1:172.21.15.17:6789/0 conn(0x7f07800b3260 legacy=0x7f078005c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:23.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.722+0000 7f077effd700 1 -- 172.21.15.17:0/1301678363 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07800aeab0 con 0x7f07800b08a0 2022-08-01T19:37:23.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.722+0000 7f077f7fe700 1 --2- 172.21.15.17:0/1301678363 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f07800ad000 0x7f07801433a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:37:23.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.722+0000 7f077effd700 1 --2- 172.21.15.17:0/1301678363 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07800b08a0 0x7f0780142eb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f0768002f70 tx=0x7f0768006df0).ready entity=mon.0 client_cookie=159275a12e87fcf7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:23.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.722+0000 7f077dffb700 1 -- 172.21.15.17:0/1301678363 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f076800d570 con 0x7f07800b08a0 2022-08-01T19:37:23.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.722+0000 7f077dffb700 1 -- 172.21.15.17:0/1301678363 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0768000e90 con 0x7f07800b08a0 2022-08-01T19:37:23.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.722+0000 7f077dffb700 1 -- 172.21.15.17:0/1301678363 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f076800d8c0 con 0x7f07800b08a0 2022-08-01T19:37:23.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.722+0000 7f077ffff700 1 -- 172.21.15.17:0/1301678363 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07800b08a0 msgr2=0x7f0780142eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:23.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.722+0000 7f077ffff700 1 --2- 172.21.15.17:0/1301678363 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07800b08a0 0x7f0780142eb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f0768002f70 tx=0x7f0768006df0).stop 2022-08-01T19:37:23.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.722+0000 7f077ffff700 1 -- 172.21.15.17:0/1301678363 shutdown_connections 2022-08-01T19:37:23.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.722+0000 7f077ffff700 1 --2- 172.21.15.17:0/1301678363 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f07800ad000 0x7f07801433a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:23.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.722+0000 7f077ffff700 1 --2- 172.21.15.17:0/1301678363 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07800b08a0 0x7f0780142eb0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:23.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.722+0000 7f077ffff700 1 -- 172.21.15.17:0/1301678363 >> 172.21.15.17:0/1301678363 conn(0x7f07800612d0 msgr2=0x7f0780062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:23.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.726+0000 7f077ffff700 1 -- 172.21.15.17:0/1301678363 shutdown_connections 2022-08-01T19:37:23.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.726+0000 7f077ffff700 1 -- 172.21.15.17:0/1301678363 wait complete. 2022-08-01T19:37:23.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.726+0000 7f077ffff700 1 Processor -- start 2022-08-01T19:37:23.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.726+0000 7f077ffff700 1 -- start start 2022-08-01T19:37:23.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.726+0000 7f077ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07800ad000 0x7f07800b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:23.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.726+0000 7f077ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07800b08a0 0x7f0780064a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:23.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.726+0000 7f077ffff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f07800b3260 0x7f0780064f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:23.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.726+0000 7f077ffff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0780063ca0 con 0x7f07800ad000 2022-08-01T19:37:23.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.726+0000 7f077ffff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0780065b00 con 0x7f07800b08a0 2022-08-01T19:37:23.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.726+0000 7f077ffff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07800658a0 con 0x7f07800b3260 2022-08-01T19:37:23.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.726+0000 7f077effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07800ad000 0x7f07800b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:23.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.726+0000 7f077effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07800ad000 0x7f07800b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38690/0 (socket says 172.21.15.17:38690) 2022-08-01T19:37:23.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.726+0000 7f077effd700 1 -- 172.21.15.17:0/1244855830 learned_addr learned my addr 172.21.15.17:0/1244855830 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:23.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.726+0000 7f077e7fc700 1 --2- 172.21.15.17:0/1244855830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07800b08a0 0x7f0780064a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:23.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.726+0000 7f077f7fe700 1 --2- 172.21.15.17:0/1244855830 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f07800b3260 0x7f0780064f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:23.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.726+0000 7f077effd700 1 -- 172.21.15.17:0/1244855830 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f07800b3260 msgr2=0x7f0780064f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:23.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.726+0000 7f077effd700 1 --2- 172.21.15.17:0/1244855830 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f07800b3260 0x7f0780064f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:23.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.726+0000 7f077effd700 1 -- 172.21.15.17:0/1244855830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07800b08a0 msgr2=0x7f0780064a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:23.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.726+0000 7f077effd700 1 --2- 172.21.15.17:0/1244855830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07800b08a0 0x7f0780064a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:23.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.726+0000 7f077effd700 1 -- 172.21.15.17:0/1244855830 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f076800b180 con 0x7f07800ad000 2022-08-01T19:37:23.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.726+0000 7f077f7fe700 1 --2- 172.21.15.17:0/1244855830 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f07800b3260 0x7f0780064f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:37:23.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.726+0000 7f077effd700 1 --2- 172.21.15.17:0/1244855830 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07800ad000 0x7f07800b3b00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f0768002f70 tx=0x7f0768014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:23.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.726+0000 7f0777fff700 1 -- 172.21.15.17:0/1244855830 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0768016070 con 0x7f07800ad000 2022-08-01T19:37:23.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.726+0000 7f0777fff700 1 -- 172.21.15.17:0/1244855830 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0768013910 con 0x7f07800ad000 2022-08-01T19:37:23.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.726+0000 7f077ffff700 1 -- 172.21.15.17:0/1244855830 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07800ac270 con 0x7f07800ad000 2022-08-01T19:37:23.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.726+0000 7f0777fff700 1 -- 172.21.15.17:0/1244855830 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0768013c70 con 0x7f07800ad000 2022-08-01T19:37:23.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.726+0000 7f077ffff700 1 -- 172.21.15.17:0/1244855830 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0780065460 con 0x7f07800ad000 2022-08-01T19:37:23.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.730+0000 7f077ffff700 1 -- 172.21.15.17:0/1244855830 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f078005cb40 con 0x7f07800ad000 2022-08-01T19:37:23.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.730+0000 7f0777fff700 1 -- 172.21.15.17:0/1244855830 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f076800f030 con 0x7f07800ad000 2022-08-01T19:37:23.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.730+0000 7f0777fff700 1 --2- 172.21.15.17:0/1244855830 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0764053a70 0x7f0764055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:23.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.730+0000 7f0777fff700 1 -- 172.21.15.17:0/1244855830 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f076806c9c0 con 0x7f07800ad000 2022-08-01T19:37:23.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.730+0000 7f077e7fc700 1 --2- 172.21.15.17:0/1244855830 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0764053a70 0x7f0764055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:23.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.730+0000 7f077e7fc700 1 --2- 172.21.15.17:0/1244855830 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0764053a70 0x7f0764055f30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f077000d000 tx=0x7f0770009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:23.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.734+0000 7f0777fff700 1 -- 172.21.15.17:0/1244855830 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f078005cb40 con 0x7f07800ad000 2022-08-01T19:37:23.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.898+0000 7f077ffff700 1 -- 172.21.15.17:0/1244855830 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f078005cb40 con 0x7f07800ad000 2022-08-01T19:37:23.901 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.898+0000 7f0777fff700 1 -- 172.21.15.17:0/1244855830 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+2488 (secure 0 0 0) 0x7f078005cb40 con 0x7f07800ad000 2022-08-01T19:37:23.902 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:37:23.902 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/109603607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":109603607},{"type":"v1","addr":"172.21.15.17:6833","nonce":109603607}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":16},{"gid":5230,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/2722431349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2722431349},{"type":"v1","addr":"172.21.15.77:6835","nonce":2722431349}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":17}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:36:02.170790+0000","modified":"2022-08-01T19:36:03.173986+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5238},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5238":{"gid":5238,"name":"c","rank":0,"incarnation":17,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/3980246529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":3980246529},{"type":"v1","addr":"172.21.15.17:6835","nonce":3980246529}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-08-01T19:37:23.905 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.902+0000 7f077ffff700 1 -- 172.21.15.17:0/1244855830 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0764053a70 msgr2=0x7f0764055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:23.905 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.902+0000 7f077ffff700 1 --2- 172.21.15.17:0/1244855830 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0764053a70 0x7f0764055f30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f077000d000 tx=0x7f0770009040).stop 2022-08-01T19:37:23.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.902+0000 7f077ffff700 1 -- 172.21.15.17:0/1244855830 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07800ad000 msgr2=0x7f07800b3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:23.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.902+0000 7f077ffff700 1 --2- 172.21.15.17:0/1244855830 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07800ad000 0x7f07800b3b00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f0768002f70 tx=0x7f0768014000).stop 2022-08-01T19:37:23.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.902+0000 7f077ffff700 1 -- 172.21.15.17:0/1244855830 shutdown_connections 2022-08-01T19:37:23.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.902+0000 7f077ffff700 1 --2- 172.21.15.17:0/1244855830 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0764053a70 0x7f0764055f30 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:23.908 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.902+0000 7f077ffff700 1 --2- 172.21.15.17:0/1244855830 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f07800b3260 0x7f0780064f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:23.908 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.902+0000 7f077ffff700 1 --2- 172.21.15.17:0/1244855830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07800b08a0 0x7f0780064a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:23.908 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.902+0000 7f077ffff700 1 --2- 172.21.15.17:0/1244855830 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07800ad000 0x7f07800b3b00 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:23.909 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.902+0000 7f077ffff700 1 -- 172.21.15.17:0/1244855830 >> 172.21.15.17:0/1244855830 conn(0x7f07800612d0 msgr2=0x7f07800a8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:23.909 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.902+0000 7f077ffff700 1 -- 172.21.15.17:0/1244855830 shutdown_connections 2022-08-01T19:37:23.909 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:23.902+0000 7f077ffff700 1 -- 172.21.15.17:0/1244855830 wait complete. 2022-08-01T19:37:23.910 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 18 2022-08-01T19:37:23.921 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-08-01T19:37:24.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.026+0000 7f3fbe480700 1 Processor -- start 2022-08-01T19:37:24.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.026+0000 7f3fbe480700 1 -- start start 2022-08-01T19:37:24.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.026+0000 7f3fbe480700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3fb80b07a0 0x7f3fb80af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:24.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.026+0000 7f3fbe480700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f3fb8147730 0x7f3fb80acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:24.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.026+0000 7f3fbe480700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fb80b5a10 con 0x7f3fb80b5630 2022-08-01T19:37:24.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.026+0000 7f3fbe480700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fb8145470 con 0x7f3fb8147730 2022-08-01T19:37:24.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.026+0000 7f3fbe480700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb80ae180 con 0x7f3fb80b07a0 2022-08-01T19:37:24.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.026+0000 7f3fbd47e700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f3fb80b5630 0x7f3fb8067730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59126/0 (socket says 172.21.15.17:59126) 2022-08-01T19:37:24.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.026+0000 7f3fbd47e700 1 -- 172.21.15.17:0/1973133449 learned_addr learned my addr 172.21.15.17:0/1973133449 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:37:24.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.026+0000 7f3fbcc7d700 1 --2- 172.21.15.17:0/1973133449 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3fb80b07a0 0x7f3fb80af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:24.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.026+0000 7f3fbdc7f700 1 --2- 172.21.15.17:0/1973133449 >> v2:172.21.15.77:3300/0 conn(0x7f3fb8147730 0x7f3fb80acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:24.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.026+0000 7f3fb7fff700 1 -- 172.21.15.17:0/1973133449 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 660367775 0 0) 0x7f3fb80b5a10 con 0x7f3fb80b5630 2022-08-01T19:37:24.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.026+0000 7f3fb7fff700 1 -- 172.21.15.17:0/1973133449 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fa4003260 con 0x7f3fb80b5630 2022-08-01T19:37:24.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.026+0000 7f3fbcc7d700 1 -- 172.21.15.17:0/1973133449 >> v2:172.21.15.77:3300/0 conn(0x7f3fb8147730 msgr2=0x7f3fb80acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:24.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.026+0000 7f3fbcc7d700 1 --2- 172.21.15.17:0/1973133449 >> v2:172.21.15.77:3300/0 conn(0x7f3fb8147730 0x7f3fb80acdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:24.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.026+0000 7f3fbcc7d700 1 -- 172.21.15.17:0/1973133449 >> v1:172.21.15.17:6789/0 conn(0x7f3fb80b5630 legacy=0x7f3fb8067730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:24.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.026+0000 7f3fbcc7d700 1 -- 172.21.15.17:0/1973133449 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb80ac990 con 0x7f3fb80b07a0 2022-08-01T19:37:24.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.026+0000 7f3fbcc7d700 1 --2- 172.21.15.17:0/1973133449 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3fb80b07a0 0x7f3fb80af6e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f3fa80032f0 tx=0x7f3fa80091d0).ready entity=mon.2 client_cookie=180ff86a20bca89f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:24.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.026+0000 7f3fb7fff700 1 -- 172.21.15.17:0/1973133449 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fa800b600 con 0x7f3fb80b07a0 2022-08-01T19:37:24.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.026+0000 7f3fb7fff700 1 -- 172.21.15.17:0/1973133449 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa800b760 con 0x7f3fb80b07a0 2022-08-01T19:37:24.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.026+0000 7f3fb7fff700 1 -- 172.21.15.17:0/1973133449 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fa800b8d0 con 0x7f3fb80b07a0 2022-08-01T19:37:24.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.026+0000 7f3fbe480700 1 -- 172.21.15.17:0/1973133449 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3fb80b07a0 msgr2=0x7f3fb80af6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:24.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.026+0000 7f3fbe480700 1 --2- 172.21.15.17:0/1973133449 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3fb80b07a0 0x7f3fb80af6e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f3fa80032f0 tx=0x7f3fa80091d0).stop 2022-08-01T19:37:24.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.026+0000 7f3fbe480700 1 -- 172.21.15.17:0/1973133449 shutdown_connections 2022-08-01T19:37:24.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.026+0000 7f3fbe480700 1 --2- 172.21.15.17:0/1973133449 >> v2:172.21.15.77:3300/0 conn(0x7f3fb8147730 0x7f3fb80acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:24.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.026+0000 7f3fbe480700 1 --2- 172.21.15.17:0/1973133449 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3fb80b07a0 0x7f3fb80af6e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:24.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.026+0000 7f3fbe480700 1 -- 172.21.15.17:0/1973133449 >> 172.21.15.17:0/1973133449 conn(0x7f3fb80626f0 msgr2=0x7f3fb8062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:24.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.026+0000 7f3fbe480700 1 -- 172.21.15.17:0/1973133449 shutdown_connections 2022-08-01T19:37:24.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.026+0000 7f3fbe480700 1 -- 172.21.15.17:0/1973133449 wait complete. 2022-08-01T19:37:24.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.030+0000 7f3fbe480700 1 Processor -- start 2022-08-01T19:37:24.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.030+0000 7f3fbe480700 1 -- start start 2022-08-01T19:37:24.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.030+0000 7f3fbe480700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3fb80b07a0 0x7f3fb814a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:24.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.030+0000 7f3fbe480700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fb80b5630 0x7f3fb8066d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:24.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.030+0000 7f3fbe480700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3fb8147730 0x7f3fb80abba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:24.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.030+0000 7f3fbe480700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb815aeb0 con 0x7f3fb80b5630 2022-08-01T19:37:24.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.030+0000 7f3fbe480700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb80b44e0 con 0x7f3fb8147730 2022-08-01T19:37:24.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.030+0000 7f3fbe480700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb80633d0 con 0x7f3fb80b07a0 2022-08-01T19:37:24.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.030+0000 7f3fbcc7d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fb80b5630 0x7f3fb8066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:24.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.030+0000 7f3fbcc7d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fb80b5630 0x7f3fb8066d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38706/0 (socket says 172.21.15.17:38706) 2022-08-01T19:37:24.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.030+0000 7f3fbcc7d700 1 -- 172.21.15.17:0/2525880814 learned_addr learned my addr 172.21.15.17:0/2525880814 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:24.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.030+0000 7f3fbdc7f700 1 --2- 172.21.15.17:0/2525880814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3fb8147730 0x7f3fb80abba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:24.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.030+0000 7f3fbd47e700 1 --2- 172.21.15.17:0/2525880814 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3fb80b07a0 0x7f3fb814a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:24.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.030+0000 7f3fbcc7d700 1 -- 172.21.15.17:0/2525880814 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3fb80b07a0 msgr2=0x7f3fb814a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:24.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.030+0000 7f3fbcc7d700 1 --2- 172.21.15.17:0/2525880814 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3fb80b07a0 0x7f3fb814a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:24.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.030+0000 7f3fbcc7d700 1 -- 172.21.15.17:0/2525880814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3fb8147730 msgr2=0x7f3fb80abba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:24.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.030+0000 7f3fbcc7d700 1 --2- 172.21.15.17:0/2525880814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3fb8147730 0x7f3fb80abba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:24.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.030+0000 7f3fbcc7d700 1 -- 172.21.15.17:0/2525880814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa8009470 con 0x7f3fb80b5630 2022-08-01T19:37:24.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.030+0000 7f3fbd47e700 1 --2- 172.21.15.17:0/2525880814 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3fb80b07a0 0x7f3fb814a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:37:24.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.030+0000 7f3fbcc7d700 1 --2- 172.21.15.17:0/2525880814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fb80b5630 0x7f3fb8066d80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f3fa80032f0 tx=0x7f3fa8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:24.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.030+0000 7f3fb67fc700 1 -- 172.21.15.17:0/2525880814 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fa8011cc0 con 0x7f3fb80b5630 2022-08-01T19:37:24.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.030+0000 7f3fb67fc700 1 -- 172.21.15.17:0/2525880814 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa8011e20 con 0x7f3fb80b5630 2022-08-01T19:37:24.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.030+0000 7f3fbe480700 1 -- 172.21.15.17:0/2525880814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fb815a980 con 0x7f3fb80b5630 2022-08-01T19:37:24.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.030+0000 7f3fb67fc700 1 -- 172.21.15.17:0/2525880814 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fa801f5e0 con 0x7f3fb80b5630 2022-08-01T19:37:24.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.030+0000 7f3fbe480700 1 -- 172.21.15.17:0/2525880814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fb815ac10 con 0x7f3fb80b5630 2022-08-01T19:37:24.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.034+0000 7f3fbe480700 1 -- 172.21.15.17:0/2525880814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fb80574e0 con 0x7f3fb80b5630 2022-08-01T19:37:24.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.034+0000 7f3fb67fc700 1 -- 172.21.15.17:0/2525880814 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3fa8026030 con 0x7f3fb80b5630 2022-08-01T19:37:24.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.034+0000 7f3fb67fc700 1 --2- 172.21.15.17:0/2525880814 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3f98053a70 0x7f3f98055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:24.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.034+0000 7f3fb67fc700 1 -- 172.21.15.17:0/2525880814 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f3fa806cc70 con 0x7f3fb80b5630 2022-08-01T19:37:24.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.034+0000 7f3fbd47e700 1 --2- 172.21.15.17:0/2525880814 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3f98053a70 0x7f3f98055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:24.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.034+0000 7f3fbd47e700 1 --2- 172.21.15.17:0/2525880814 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3f98053a70 0x7f3f98055f30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f3fac00b000 tx=0x7f3fac009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:24.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.038+0000 7f3fb67fc700 1 -- 172.21.15.17:0/2525880814 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3fa8043920 con 0x7f3fb80b5630 2022-08-01T19:37:24.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:24.206+0000 7f3fbe480700 1 -- 172.21.15.17:0/2525880814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3fb805f3d0 con 0x7f3fb80b5630 2022-08-01T19:37:25.122 INFO:tasks.ceph.mon.a.smithi017.stderr:2022-08-01T19:37:25.122+0000 7f6a53773700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-08-01T19:37:25.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.134+0000 7f3fb67fc700 1 -- 172.21.15.17:0/2525880814 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3fa8024090 con 0x7f3fb80b5630 2022-08-01T19:37:25.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.138+0000 7f3fbe480700 1 -- 172.21.15.17:0/2525880814 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3f98053a70 msgr2=0x7f3f98055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:25.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.138+0000 7f3fbe480700 1 --2- 172.21.15.17:0/2525880814 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3f98053a70 0x7f3f98055f30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f3fac00b000 tx=0x7f3fac009040).stop 2022-08-01T19:37:25.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.138+0000 7f3fbe480700 1 -- 172.21.15.17:0/2525880814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fb80b5630 msgr2=0x7f3fb8066d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:25.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.138+0000 7f3fbe480700 1 --2- 172.21.15.17:0/2525880814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fb80b5630 0x7f3fb8066d80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f3fa80032f0 tx=0x7f3fa8016000).stop 2022-08-01T19:37:25.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.138+0000 7f3fbe480700 1 -- 172.21.15.17:0/2525880814 shutdown_connections 2022-08-01T19:37:25.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.138+0000 7f3fbe480700 1 --2- 172.21.15.17:0/2525880814 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3f98053a70 0x7f3f98055f30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:25.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.138+0000 7f3fbe480700 1 --2- 172.21.15.17:0/2525880814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3fb8147730 0x7f3fb80abba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:25.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.138+0000 7f3fbe480700 1 --2- 172.21.15.17:0/2525880814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fb80b5630 0x7f3fb8066d80 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:25.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.138+0000 7f3fbe480700 1 --2- 172.21.15.17:0/2525880814 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3fb80b07a0 0x7f3fb814a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:25.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.138+0000 7f3fbe480700 1 -- 172.21.15.17:0/2525880814 >> 172.21.15.17:0/2525880814 conn(0x7f3fb80626f0 msgr2=0x7f3fb80af450 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:25.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.138+0000 7f3fbe480700 1 -- 172.21.15.17:0/2525880814 shutdown_connections 2022-08-01T19:37:25.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.138+0000 7f3fbe480700 1 -- 172.21.15.17:0/2525880814 wait complete. 2022-08-01T19:37:25.145 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-08-01T19:37:25.157 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:37:25.178 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2022-08-01T19:37:25.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.262+0000 7fc929336700 1 Processor -- start 2022-08-01T19:37:25.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.262+0000 7fc929336700 1 -- start start 2022-08-01T19:37:25.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc929336700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fc9240b31d0 0x7fc924066020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:25.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc929336700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc9240a9da0 0x7fc924060950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:25.271 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc929336700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9241434b0 0x7fc924143940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:25.271 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc929336700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc924064a70 con 0x7fc9240a9da0 2022-08-01T19:37:25.271 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc929336700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc924061f60 con 0x7fc9240b31d0 2022-08-01T19:37:25.272 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc929336700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc924067030 con 0x7fc9241434b0 2022-08-01T19:37:25.272 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc9237fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc9240a9da0 0x7fc924060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:25.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc9237fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc9240a9da0 0x7fc924060950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38726/0 (socket says 172.21.15.17:38726) 2022-08-01T19:37:25.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc9237fe700 1 -- 172.21.15.17:0/4133962539 learned_addr learned my addr 172.21.15.17:0/4133962539 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:25.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc928b35700 1 --2- 172.21.15.17:0/4133962539 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9241434b0 0x7fc924143940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:25.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc923fff700 1 --2- 172.21.15.17:0/4133962539 >> v2:172.21.15.77:3300/0 conn(0x7fc9240b31d0 0x7fc924066020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:25.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc9237fe700 1 -- 172.21.15.17:0/4133962539 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9241434b0 msgr2=0x7fc924143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:25.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc9237fe700 1 --2- 172.21.15.17:0/4133962539 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9241434b0 0x7fc924143940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:25.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc9237fe700 1 -- 172.21.15.17:0/4133962539 >> v2:172.21.15.77:3300/0 conn(0x7fc9240b31d0 msgr2=0x7fc924066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:25.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc9237fe700 1 --2- 172.21.15.17:0/4133962539 >> v2:172.21.15.77:3300/0 conn(0x7fc9240b31d0 0x7fc924066020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:25.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc9237fe700 1 -- 172.21.15.17:0/4133962539 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9240b35b0 con 0x7fc9240a9da0 2022-08-01T19:37:25.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc9237fe700 1 --2- 172.21.15.17:0/4133962539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9240a9da0 0x7fc924060950 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7fc90c002f70 tx=0x7fc90c006eb0).ready entity=mon.0 client_cookie=d304f630096d22d1 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:25.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc922ffd700 1 -- 172.21.15.17:0/4133962539 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc90c00d570 con 0x7fc9240a9da0 2022-08-01T19:37:25.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc922ffd700 1 -- 172.21.15.17:0/4133962539 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc90c000e90 con 0x7fc9240a9da0 2022-08-01T19:37:25.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc922ffd700 1 -- 172.21.15.17:0/4133962539 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc90c00d900 con 0x7fc9240a9da0 2022-08-01T19:37:25.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc929336700 1 -- 172.21.15.17:0/4133962539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9240a9da0 msgr2=0x7fc924060950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:25.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc929336700 1 --2- 172.21.15.17:0/4133962539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9240a9da0 0x7fc924060950 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7fc90c002f70 tx=0x7fc90c006eb0).stop 2022-08-01T19:37:25.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc929336700 1 -- 172.21.15.17:0/4133962539 shutdown_connections 2022-08-01T19:37:25.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc929336700 1 --2- 172.21.15.17:0/4133962539 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9241434b0 0x7fc924143940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:25.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc929336700 1 --2- 172.21.15.17:0/4133962539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9240a9da0 0x7fc924060950 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:25.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc929336700 1 --2- 172.21.15.17:0/4133962539 >> v2:172.21.15.77:3300/0 conn(0x7fc9240b31d0 0x7fc924066020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:25.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc929336700 1 -- 172.21.15.17:0/4133962539 >> 172.21.15.17:0/4133962539 conn(0x7fc924063870 msgr2=0x7fc924063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:25.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc929336700 1 -- 172.21.15.17:0/4133962539 shutdown_connections 2022-08-01T19:37:25.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc929336700 1 -- 172.21.15.17:0/4133962539 wait complete. 2022-08-01T19:37:25.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc929336700 1 Processor -- start 2022-08-01T19:37:25.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc929336700 1 -- start start 2022-08-01T19:37:25.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc929336700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9240a9da0 0x7fc92405fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:25.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc929336700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9240b31d0 0x7fc9240b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:25.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc929336700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9241434b0 0x7fc92413a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:25.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc929336700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc924145560 con 0x7fc9240a9da0 2022-08-01T19:37:25.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc929336700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc924145800 con 0x7fc9241434b0 2022-08-01T19:37:25.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc929336700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc924145aa0 con 0x7fc9240b31d0 2022-08-01T19:37:25.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc923fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9240a9da0 0x7fc92405fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:25.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc923fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9240a9da0 0x7fc92405fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38730/0 (socket says 172.21.15.17:38730) 2022-08-01T19:37:25.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc923fff700 1 -- 172.21.15.17:0/431367885 learned_addr learned my addr 172.21.15.17:0/431367885 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:25.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc9237fe700 1 --2- 172.21.15.17:0/431367885 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9240b31d0 0x7fc9240b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:25.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc928b35700 1 --2- 172.21.15.17:0/431367885 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9241434b0 0x7fc92413a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:25.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc923fff700 1 -- 172.21.15.17:0/431367885 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9240b31d0 msgr2=0x7fc9240b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:25.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc923fff700 1 --2- 172.21.15.17:0/431367885 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9240b31d0 0x7fc9240b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:25.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc923fff700 1 -- 172.21.15.17:0/431367885 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9241434b0 msgr2=0x7fc92413a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:25.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc923fff700 1 --2- 172.21.15.17:0/431367885 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9241434b0 0x7fc92413a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:25.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc923fff700 1 -- 172.21.15.17:0/431367885 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc90c00b1e0 con 0x7fc9240a9da0 2022-08-01T19:37:25.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc9237fe700 1 --2- 172.21.15.17:0/431367885 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9240b31d0 0x7fc9240b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:37:25.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc923fff700 1 --2- 172.21.15.17:0/431367885 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9240a9da0 0x7fc92405fe10 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7fc9140030b0 tx=0x7fc914008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:25.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc9217fa700 1 -- 172.21.15.17:0/431367885 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc914011780 con 0x7fc9240a9da0 2022-08-01T19:37:25.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc9217fa700 1 -- 172.21.15.17:0/431367885 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9140118e0 con 0x7fc9240a9da0 2022-08-01T19:37:25.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc929336700 1 -- 172.21.15.17:0/431367885 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc924145d40 con 0x7fc9240a9da0 2022-08-01T19:37:25.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc9217fa700 1 -- 172.21.15.17:0/431367885 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc914011c40 con 0x7fc9240a9da0 2022-08-01T19:37:25.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.266+0000 7fc929336700 1 -- 172.21.15.17:0/431367885 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc924060620 con 0x7fc9240a9da0 2022-08-01T19:37:25.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.270+0000 7fc9217fa700 1 -- 172.21.15.17:0/431367885 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc91401c460 con 0x7fc9240a9da0 2022-08-01T19:37:25.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.270+0000 7fc929336700 1 -- 172.21.15.17:0/431367885 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc924145ed0 con 0x7fc9240a9da0 2022-08-01T19:37:25.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.270+0000 7fc9217fa700 1 --2- 172.21.15.17:0/431367885 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9080539e0 0x7fc908055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:25.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.270+0000 7fc9217fa700 1 -- 172.21.15.17:0/431367885 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fc91406c6a0 con 0x7fc9240a9da0 2022-08-01T19:37:25.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.270+0000 7fc9237fe700 1 --2- 172.21.15.17:0/431367885 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9080539e0 0x7fc908055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:25.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.270+0000 7fc9237fe700 1 --2- 172.21.15.17:0/431367885 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9080539e0 0x7fc908055ea0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7fc90c01a000 tx=0x7fc90c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:25.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.282+0000 7fc9217fa700 1 -- 172.21.15.17:0/431367885 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc9140432a0 con 0x7fc9240a9da0 2022-08-01T19:37:25.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.438+0000 7fc929336700 1 -- 172.21.15.17:0/431367885 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc924066af0 con 0x7fc9240a9da0 2022-08-01T19:37:25.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.438+0000 7fc9217fa700 1 -- 172.21.15.17:0/431367885 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+2127 (secure 0 0 0) 0x7fc9140468c0 con 0x7fc9240a9da0 2022-08-01T19:37:25.443 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:37:25.443 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":19,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/109603607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":109603607},{"type":"v1","addr":"172.21.15.17:6833","nonce":109603607}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":16},{"gid":5230,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/2722431349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2722431349},{"type":"v1","addr":"172.21.15.77:6835","nonce":2722431349}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":17}],"filesystems":[{"mdsmap":{"epoch":19,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:36:02.170790+0000","modified":"2022-08-01T19:37:25.124687+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-08-01T19:37:25.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.438+0000 7fc929336700 1 -- 172.21.15.17:0/431367885 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9080539e0 msgr2=0x7fc908055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:25.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.438+0000 7fc929336700 1 --2- 172.21.15.17:0/431367885 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9080539e0 0x7fc908055ea0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7fc90c01a000 tx=0x7fc90c015000).stop 2022-08-01T19:37:25.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.438+0000 7fc929336700 1 -- 172.21.15.17:0/431367885 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9240a9da0 msgr2=0x7fc92405fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:25.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.438+0000 7fc929336700 1 --2- 172.21.15.17:0/431367885 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9240a9da0 0x7fc92405fe10 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7fc9140030b0 tx=0x7fc914008040).stop 2022-08-01T19:37:25.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.438+0000 7fc929336700 1 -- 172.21.15.17:0/431367885 shutdown_connections 2022-08-01T19:37:25.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.438+0000 7fc929336700 1 --2- 172.21.15.17:0/431367885 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc9080539e0 0x7fc908055ea0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:25.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.438+0000 7fc929336700 1 --2- 172.21.15.17:0/431367885 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9241434b0 0x7fc92413a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:25.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.438+0000 7fc929336700 1 --2- 172.21.15.17:0/431367885 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc9240b31d0 0x7fc9240b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:25.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.438+0000 7fc929336700 1 --2- 172.21.15.17:0/431367885 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9240a9da0 0x7fc92405fe10 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:25.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.438+0000 7fc929336700 1 -- 172.21.15.17:0/431367885 >> 172.21.15.17:0/431367885 conn(0x7fc924063870 msgr2=0x7fc924063c50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:25.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.438+0000 7fc929336700 1 -- 172.21.15.17:0/431367885 shutdown_connections 2022-08-01T19:37:25.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.438+0000 7fc929336700 1 -- 172.21.15.17:0/431367885 wait complete. 2022-08-01T19:37:25.451 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 19 2022-08-01T19:37:25.455 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-08-01T19:37:25.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.558+0000 7f494cb93700 1 Processor -- start 2022-08-01T19:37:25.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.558+0000 7f494cb93700 1 -- start start 2022-08-01T19:37:25.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.562+0000 7f494cb93700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f49480b01a0 0x7f49480a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:25.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.562+0000 7f494cb93700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f49480afd60 0x7f49480a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:25.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.562+0000 7f494cb93700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49480a7250 con 0x7f49480b2710 2022-08-01T19:37:25.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.562+0000 7f494cb93700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49480a7480 con 0x7f49480b01a0 2022-08-01T19:37:25.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.562+0000 7f494cb93700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49480a7720 con 0x7f49480afd60 2022-08-01T19:37:25.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.562+0000 7f49477fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f49480b2710 0x7f494814afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59164/0 (socket says 172.21.15.17:59164) 2022-08-01T19:37:25.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.562+0000 7f49477fe700 1 -- 172.21.15.17:0/2227050135 learned_addr learned my addr 172.21.15.17:0/2227050135 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:37:25.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.562+0000 7f4946ffd700 1 --2- 172.21.15.17:0/2227050135 >> v2:172.21.15.17:3300/0 conn(0x7f49480b01a0 0x7f49480a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:25.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.562+0000 7f4947fff700 1 --2- 172.21.15.17:0/2227050135 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f49480afd60 0x7f49480a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:25.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.562+0000 7f49467fc700 1 -- 172.21.15.17:0/2227050135 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2238692931 0 0) 0x7f49480a7250 con 0x7f49480b2710 2022-08-01T19:37:25.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.562+0000 7f49467fc700 1 -- 172.21.15.17:0/2227050135 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4934003260 con 0x7f49480b2710 2022-08-01T19:37:25.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.562+0000 7f4946ffd700 1 -- 172.21.15.17:0/2227050135 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f49480afd60 msgr2=0x7f49480a8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:25.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.562+0000 7f4946ffd700 1 --2- 172.21.15.17:0/2227050135 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f49480afd60 0x7f49480a8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:25.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.562+0000 7f4946ffd700 1 -- 172.21.15.17:0/2227050135 >> v1:172.21.15.17:6789/0 conn(0x7f49480b2710 legacy=0x7f494814afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:25.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.562+0000 7f4946ffd700 1 -- 172.21.15.17:0/2227050135 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49480a79c0 con 0x7f49480b01a0 2022-08-01T19:37:25.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.562+0000 7f4947fff700 1 --2- 172.21.15.17:0/2227050135 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f49480afd60 0x7f49480a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:37:25.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.562+0000 7f4946ffd700 1 --2- 172.21.15.17:0/2227050135 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49480b01a0 0x7f49480a8540 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f493c002f70 tx=0x7f493c006df0).ready entity=mon.0 client_cookie=2988fd921b5bb01b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:25.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.562+0000 7f49467fc700 1 -- 172.21.15.17:0/2227050135 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f493c00d570 con 0x7f49480b01a0 2022-08-01T19:37:25.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.562+0000 7f49467fc700 1 -- 172.21.15.17:0/2227050135 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493c000e90 con 0x7f49480b01a0 2022-08-01T19:37:25.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.562+0000 7f49467fc700 1 -- 172.21.15.17:0/2227050135 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f493c00d8c0 con 0x7f49480b01a0 2022-08-01T19:37:25.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.562+0000 7f494cb93700 1 -- 172.21.15.17:0/2227050135 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49480b01a0 msgr2=0x7f49480a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:25.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.562+0000 7f494cb93700 1 --2- 172.21.15.17:0/2227050135 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49480b01a0 0x7f49480a8540 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f493c002f70 tx=0x7f493c006df0).stop 2022-08-01T19:37:25.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.562+0000 7f494cb93700 1 -- 172.21.15.17:0/2227050135 shutdown_connections 2022-08-01T19:37:25.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.562+0000 7f494cb93700 1 --2- 172.21.15.17:0/2227050135 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f49480afd60 0x7f49480a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:25.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.562+0000 7f494cb93700 1 --2- 172.21.15.17:0/2227050135 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49480b01a0 0x7f49480a8540 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:25.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.562+0000 7f494cb93700 1 -- 172.21.15.17:0/2227050135 >> 172.21.15.17:0/2227050135 conn(0x7f4948060310 msgr2=0x7f49480b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:25.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.562+0000 7f494cb93700 1 -- 172.21.15.17:0/2227050135 shutdown_connections 2022-08-01T19:37:25.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.562+0000 7f494cb93700 1 -- 172.21.15.17:0/2227050135 wait complete. 2022-08-01T19:37:25.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.562+0000 7f494cb93700 1 Processor -- start 2022-08-01T19:37:25.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.562+0000 7f494cb93700 1 -- start start 2022-08-01T19:37:25.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.566+0000 7f494cb93700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49480afd60 0x7f494805c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:25.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.566+0000 7f494cb93700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49480b01a0 0x7f4948157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:25.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.566+0000 7f494cb93700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f49480b2710 0x7f494815a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:25.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.566+0000 7f494cb93700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49480abc60 con 0x7f49480afd60 2022-08-01T19:37:25.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.566+0000 7f494cb93700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4948159b70 con 0x7f49480b01a0 2022-08-01T19:37:25.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.566+0000 7f494cb93700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4948157500 con 0x7f49480b2710 2022-08-01T19:37:25.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.566+0000 7f49477fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49480afd60 0x7f494805c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:25.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.566+0000 7f49477fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49480afd60 0x7f494805c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38746/0 (socket says 172.21.15.17:38746) 2022-08-01T19:37:25.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.566+0000 7f49477fe700 1 -- 172.21.15.17:0/3294994283 learned_addr learned my addr 172.21.15.17:0/3294994283 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:25.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.566+0000 7f4946ffd700 1 --2- 172.21.15.17:0/3294994283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49480b01a0 0x7f4948157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:25.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.566+0000 7f4947fff700 1 --2- 172.21.15.17:0/3294994283 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f49480b2710 0x7f494815a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:25.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.566+0000 7f49477fe700 1 -- 172.21.15.17:0/3294994283 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f49480b2710 msgr2=0x7f494815a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:25.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.566+0000 7f49477fe700 1 --2- 172.21.15.17:0/3294994283 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f49480b2710 0x7f494815a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:25.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.566+0000 7f49477fe700 1 -- 172.21.15.17:0/3294994283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49480b01a0 msgr2=0x7f4948157010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:25.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.566+0000 7f49477fe700 1 --2- 172.21.15.17:0/3294994283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49480b01a0 0x7f4948157010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:25.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.566+0000 7f49477fe700 1 -- 172.21.15.17:0/3294994283 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f493c00b180 con 0x7f49480afd60 2022-08-01T19:37:25.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.566+0000 7f49477fe700 1 --2- 172.21.15.17:0/3294994283 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49480afd60 0x7f494805c170 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f4948157690 tx=0x7f4940007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:25.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.566+0000 7f4944ff9700 1 -- 172.21.15.17:0/3294994283 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4940005ba0 con 0x7f49480afd60 2022-08-01T19:37:25.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.566+0000 7f4944ff9700 1 -- 172.21.15.17:0/3294994283 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4940005d00 con 0x7f49480afd60 2022-08-01T19:37:25.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.566+0000 7f494cb93700 1 -- 172.21.15.17:0/3294994283 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f494814b490 con 0x7f49480afd60 2022-08-01T19:37:25.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.566+0000 7f4944ff9700 1 -- 172.21.15.17:0/3294994283 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49400072c0 con 0x7f49480afd60 2022-08-01T19:37:25.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.566+0000 7f494cb93700 1 -- 172.21.15.17:0/3294994283 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4948159da0 con 0x7f49480afd60 2022-08-01T19:37:25.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.566+0000 7f494cb93700 1 -- 172.21.15.17:0/3294994283 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49480574e0 con 0x7f49480afd60 2022-08-01T19:37:25.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.570+0000 7f4944ff9700 1 -- 172.21.15.17:0/3294994283 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f494001a070 con 0x7f49480afd60 2022-08-01T19:37:25.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.570+0000 7f4944ff9700 1 --2- 172.21.15.17:0/3294994283 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4928053a70 0x7f4928055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:25.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.570+0000 7f4944ff9700 1 -- 172.21.15.17:0/3294994283 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f494006b6a0 con 0x7f49480afd60 2022-08-01T19:37:25.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.570+0000 7f4946ffd700 1 --2- 172.21.15.17:0/3294994283 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4928053a70 0x7f4928055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:25.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.570+0000 7f4946ffd700 1 --2- 172.21.15.17:0/3294994283 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4928053a70 0x7f4928055f30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f493c01a000 tx=0x7f493c014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:25.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.578+0000 7f4944ff9700 1 -- 172.21.15.17:0/3294994283 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4940042300 con 0x7f49480afd60 2022-08-01T19:37:25.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:25.738+0000 7f494cb93700 1 -- 172.21.15.17:0/3294994283 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4948159fa0 con 0x7f49480afd60 2022-08-01T19:37:26.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.218+0000 7f4944ff9700 1 -- 172.21.15.17:0/3294994283 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4940045920 con 0x7f49480afd60 2022-08-01T19:37:26.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.222+0000 7f494cb93700 1 -- 172.21.15.17:0/3294994283 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4928053a70 msgr2=0x7f4928055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:26.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.222+0000 7f494cb93700 1 --2- 172.21.15.17:0/3294994283 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4928053a70 0x7f4928055f30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f493c01a000 tx=0x7f493c014000).stop 2022-08-01T19:37:26.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.222+0000 7f494cb93700 1 -- 172.21.15.17:0/3294994283 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49480afd60 msgr2=0x7f494805c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:26.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.222+0000 7f494cb93700 1 --2- 172.21.15.17:0/3294994283 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49480afd60 0x7f494805c170 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f4948157690 tx=0x7f4940007e20).stop 2022-08-01T19:37:26.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.222+0000 7f494cb93700 1 -- 172.21.15.17:0/3294994283 shutdown_connections 2022-08-01T19:37:26.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.222+0000 7f494cb93700 1 --2- 172.21.15.17:0/3294994283 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4928053a70 0x7f4928055f30 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:26.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.222+0000 7f494cb93700 1 --2- 172.21.15.17:0/3294994283 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f49480b2710 0x7f494815a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:26.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.222+0000 7f494cb93700 1 --2- 172.21.15.17:0/3294994283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49480b01a0 0x7f4948157010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:26.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.222+0000 7f494cb93700 1 --2- 172.21.15.17:0/3294994283 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49480afd60 0x7f494805c170 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:26.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.222+0000 7f494cb93700 1 -- 172.21.15.17:0/3294994283 >> 172.21.15.17:0/3294994283 conn(0x7f4948060310 msgr2=0x7f49480af220 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:26.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.222+0000 7f494cb93700 1 -- 172.21.15.17:0/3294994283 shutdown_connections 2022-08-01T19:37:26.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.222+0000 7f494cb93700 1 -- 172.21.15.17:0/3294994283 wait complete. 2022-08-01T19:37:26.245 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-08-01T19:37:26.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.362+0000 7f68cb659700 1 Processor -- start 2022-08-01T19:37:26.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.362+0000 7f68cb659700 1 -- start start 2022-08-01T19:37:26.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.362+0000 7f68cb659700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f68c40b6710 0x7f68c40b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:26.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.362+0000 7f68cb659700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f68c40b4550 0x7f68c40b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:26.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.362+0000 7f68cb659700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68c4144240 con 0x7f68c4061ec0 2022-08-01T19:37:26.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.362+0000 7f68cb659700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68c40b6fe0 con 0x7f68c40b6710 2022-08-01T19:37:26.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.362+0000 7f68cb659700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c4143d00 con 0x7f68c40b4550 2022-08-01T19:37:26.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.362+0000 7f68c93f5700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f68c40b6710 0x7f68c40b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:26.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.362+0000 7f68c9bf6700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f68c4061ec0 0x7f68c4146850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59182/0 (socket says 172.21.15.17:59182) 2022-08-01T19:37:26.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.362+0000 7f68c9bf6700 1 -- 172.21.15.17:0/3371499842 learned_addr learned my addr 172.21.15.17:0/3371499842 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:37:26.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.362+0000 7f68c8bf4700 1 --2- 172.21.15.17:0/3371499842 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f68c40b4550 0x7f68c40b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:26.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.362+0000 7f68c93f5700 1 -- 172.21.15.17:0/3371499842 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f68c40b4550 msgr2=0x7f68c40b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:26.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.362+0000 7f68c93f5700 1 --2- 172.21.15.17:0/3371499842 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f68c40b4550 0x7f68c40b6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:26.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.362+0000 7f68c93f5700 1 -- 172.21.15.17:0/3371499842 >> v1:172.21.15.17:6789/0 conn(0x7f68c4061ec0 legacy=0x7f68c4146850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:26.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.362+0000 7f68b7fff700 1 -- 172.21.15.17:0/3371499842 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1300909362 0 0) 0x7f68c4144240 con 0x7f68c4061ec0 2022-08-01T19:37:26.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.362+0000 7f68c93f5700 1 -- 172.21.15.17:0/3371499842 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c4143fa0 con 0x7f68c40b6710 2022-08-01T19:37:26.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.362+0000 7f68c93f5700 1 --2- 172.21.15.17:0/3371499842 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68c40b6710 0x7f68c40b5d80 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f68c0002f70 tx=0x7f68c0006df0).ready entity=mon.0 client_cookie=76db5fed9d820550 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:26.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.362+0000 7f68b7fff700 1 -- 172.21.15.17:0/3371499842 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68c000d570 con 0x7f68c40b6710 2022-08-01T19:37:26.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.362+0000 7f68b7fff700 1 -- 172.21.15.17:0/3371499842 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c0000e90 con 0x7f68c40b6710 2022-08-01T19:37:26.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.362+0000 7f68b7fff700 1 -- 172.21.15.17:0/3371499842 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68c000d8c0 con 0x7f68c40b6710 2022-08-01T19:37:26.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.362+0000 7f68cb659700 1 -- 172.21.15.17:0/3371499842 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68c40b6710 msgr2=0x7f68c40b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:26.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.362+0000 7f68cb659700 1 --2- 172.21.15.17:0/3371499842 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68c40b6710 0x7f68c40b5d80 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f68c0002f70 tx=0x7f68c0006df0).stop 2022-08-01T19:37:26.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.362+0000 7f68cb659700 1 -- 172.21.15.17:0/3371499842 shutdown_connections 2022-08-01T19:37:26.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.362+0000 7f68cb659700 1 --2- 172.21.15.17:0/3371499842 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f68c40b4550 0x7f68c40b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:26.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.362+0000 7f68cb659700 1 --2- 172.21.15.17:0/3371499842 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68c40b6710 0x7f68c40b5d80 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:26.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.362+0000 7f68cb659700 1 -- 172.21.15.17:0/3371499842 >> 172.21.15.17:0/3371499842 conn(0x7f68c405c620 msgr2=0x7f68c405ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:26.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.362+0000 7f68cb659700 1 -- 172.21.15.17:0/3371499842 shutdown_connections 2022-08-01T19:37:26.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.362+0000 7f68cb659700 1 -- 172.21.15.17:0/3371499842 wait complete. 2022-08-01T19:37:26.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.362+0000 7f68cb659700 1 Processor -- start 2022-08-01T19:37:26.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.362+0000 7f68cb659700 1 -- start start 2022-08-01T19:37:26.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.366+0000 7f68cb659700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68c4061ec0 0x7f68c4125130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:26.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.366+0000 7f68cb659700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f68c40b4550 0x7f68c40ad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:26.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.366+0000 7f68cb659700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68c40b6710 0x7f68c4152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:26.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.366+0000 7f68cb659700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c40b9320 con 0x7f68c4061ec0 2022-08-01T19:37:26.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.366+0000 7f68cb659700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c412ae30 con 0x7f68c40b6710 2022-08-01T19:37:26.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.366+0000 7f68cb659700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c40b9d40 con 0x7f68c40b4550 2022-08-01T19:37:26.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.366+0000 7f68c93f5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68c4061ec0 0x7f68c4125130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:26.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.366+0000 7f68c93f5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68c4061ec0 0x7f68c4125130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38758/0 (socket says 172.21.15.17:38758) 2022-08-01T19:37:26.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.366+0000 7f68c93f5700 1 -- 172.21.15.17:0/226938272 learned_addr learned my addr 172.21.15.17:0/226938272 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:26.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.366+0000 7f68c8bf4700 1 --2- 172.21.15.17:0/226938272 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f68c40b4550 0x7f68c40ad7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:26.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.366+0000 7f68c9bf6700 1 --2- 172.21.15.17:0/226938272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68c40b6710 0x7f68c4152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:26.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.366+0000 7f68c93f5700 1 -- 172.21.15.17:0/226938272 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f68c40b4550 msgr2=0x7f68c40ad7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:26.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.366+0000 7f68c93f5700 1 --2- 172.21.15.17:0/226938272 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f68c40b4550 0x7f68c40ad7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:26.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.366+0000 7f68c93f5700 1 -- 172.21.15.17:0/226938272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68c40b6710 msgr2=0x7f68c4152390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:26.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.366+0000 7f68c93f5700 1 --2- 172.21.15.17:0/226938272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68c40b6710 0x7f68c4152390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:26.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.366+0000 7f68c93f5700 1 -- 172.21.15.17:0/226938272 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c000b180 con 0x7f68c4061ec0 2022-08-01T19:37:26.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.366+0000 7f68c93f5700 1 --2- 172.21.15.17:0/226938272 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68c4061ec0 0x7f68c4125130 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f68c0002f70 tx=0x7f68c0014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:26.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.366+0000 7f68b67fc700 1 -- 172.21.15.17:0/226938272 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68c000dea0 con 0x7f68c4061ec0 2022-08-01T19:37:26.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.366+0000 7f68b67fc700 1 -- 172.21.15.17:0/226938272 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c0013e90 con 0x7f68c4061ec0 2022-08-01T19:37:26.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.366+0000 7f68cb659700 1 -- 172.21.15.17:0/226938272 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c41567f0 con 0x7f68c4061ec0 2022-08-01T19:37:26.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.366+0000 7f68b67fc700 1 -- 172.21.15.17:0/226938272 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68c001e5f0 con 0x7f68c4061ec0 2022-08-01T19:37:26.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.366+0000 7f68cb659700 1 -- 172.21.15.17:0/226938272 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c412b060 con 0x7f68c4061ec0 2022-08-01T19:37:26.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.370+0000 7f68cb659700 1 -- 172.21.15.17:0/226938272 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68c40b90c0 con 0x7f68c4061ec0 2022-08-01T19:37:26.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.370+0000 7f68b67fc700 1 -- 172.21.15.17:0/226938272 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f68c0025030 con 0x7f68c4061ec0 2022-08-01T19:37:26.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.370+0000 7f68b67fc700 1 --2- 172.21.15.17:0/226938272 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f68ac053a70 0x7f68ac055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:26.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.370+0000 7f68b67fc700 1 -- 172.21.15.17:0/226938272 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f68c006cee0 con 0x7f68c4061ec0 2022-08-01T19:37:26.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.370+0000 7f68c8bf4700 1 --2- 172.21.15.17:0/226938272 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f68ac053a70 0x7f68ac055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:26.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.370+0000 7f68c8bf4700 1 --2- 172.21.15.17:0/226938272 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f68ac053a70 0x7f68ac055f30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f68b800b000 tx=0x7f68b8006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:26.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.374+0000 7f68b67fc700 1 -- 172.21.15.17:0/226938272 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f68c0043c90 con 0x7f68c4061ec0 2022-08-01T19:37:26.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:26.502+0000 7f68cb659700 1 -- 172.21.15.17:0/226938272 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f68c4061780 con 0x7f68c4061ec0 2022-08-01T19:37:27.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.230+0000 7f68b67fc700 1 -- 172.21.15.17:0/226938272 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v47) v1 ==== 185+0+0 (secure 0 0 0) 0x7f68c0047270 con 0x7f68c4061ec0 2022-08-01T19:37:27.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.234+0000 7f68cb659700 1 -- 172.21.15.17:0/226938272 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f68ac053a70 msgr2=0x7f68ac055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:27.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.234+0000 7f68cb659700 1 --2- 172.21.15.17:0/226938272 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f68ac053a70 0x7f68ac055f30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f68b800b000 tx=0x7f68b8006df0).stop 2022-08-01T19:37:27.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.234+0000 7f68cb659700 1 -- 172.21.15.17:0/226938272 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68c4061ec0 msgr2=0x7f68c4125130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:27.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.234+0000 7f68cb659700 1 --2- 172.21.15.17:0/226938272 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68c4061ec0 0x7f68c4125130 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f68c0002f70 tx=0x7f68c0014000).stop 2022-08-01T19:37:27.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.234+0000 7f68cb659700 1 -- 172.21.15.17:0/226938272 shutdown_connections 2022-08-01T19:37:27.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.234+0000 7f68cb659700 1 --2- 172.21.15.17:0/226938272 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f68ac053a70 0x7f68ac055f30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:27.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.234+0000 7f68cb659700 1 --2- 172.21.15.17:0/226938272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68c40b6710 0x7f68c4152390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:27.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.234+0000 7f68cb659700 1 --2- 172.21.15.17:0/226938272 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f68c40b4550 0x7f68c40ad7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:27.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.234+0000 7f68cb659700 1 --2- 172.21.15.17:0/226938272 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68c4061ec0 0x7f68c4125130 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:27.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.234+0000 7f68cb659700 1 -- 172.21.15.17:0/226938272 >> 172.21.15.17:0/226938272 conn(0x7f68c405c620 msgr2=0x7f68c4127710 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:27.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.234+0000 7f68cb659700 1 -- 172.21.15.17:0/226938272 shutdown_connections 2022-08-01T19:37:27.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.234+0000 7f68cb659700 1 -- 172.21.15.17:0/226938272 wait complete. 2022-08-01T19:37:27.242 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' removed 2022-08-01T19:37:27.253 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-08-01T19:37:27.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.362+0000 7f595a991700 1 Processor -- start 2022-08-01T19:37:27.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.362+0000 7f595a991700 1 -- start start 2022-08-01T19:37:27.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.362+0000 7f595a991700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f59541489f0 0x7f595414ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:27.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.362+0000 7f595a991700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5954067030 con 0x7f59540b31d0 2022-08-01T19:37:27.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.362+0000 7f595a991700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59540b35b0 con 0x7f59540a9da0 2022-08-01T19:37:27.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.362+0000 7f595a991700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5954064a70 con 0x7f59541489f0 2022-08-01T19:37:27.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.362+0000 7f595998f700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f59540b31d0 0x7f5954066020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59192/0 (socket says 172.21.15.17:59192) 2022-08-01T19:37:27.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.362+0000 7f595998f700 1 -- 172.21.15.17:0/3518917430 learned_addr learned my addr 172.21.15.17:0/3518917430 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:37:27.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.362+0000 7f595a190700 1 --2- 172.21.15.17:0/3518917430 >> v2:172.21.15.77:3300/0 conn(0x7f59541489f0 0x7f595414ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:27.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.362+0000 7f595898d700 1 -- 172.21.15.17:0/3518917430 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1076138922 0 0) 0x7f5954067030 con 0x7f59540b31d0 2022-08-01T19:37:27.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.362+0000 7f595898d700 1 -- 172.21.15.17:0/3518917430 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f593c003260 con 0x7f59540b31d0 2022-08-01T19:37:27.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.362+0000 7f595898d700 1 -- 172.21.15.17:0/3518917430 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 337172562 0 0) 0x7f59540b35b0 con 0x7f59540a9da0 2022-08-01T19:37:27.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.362+0000 7f595898d700 1 -- 172.21.15.17:0/3518917430 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5954067030 con 0x7f59540a9da0 2022-08-01T19:37:27.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.362+0000 7f595a190700 1 -- 172.21.15.17:0/3518917430 >> v1:172.21.15.77:6789/0 conn(0x7f59540a9da0 legacy=0x7f5954060950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:27.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.362+0000 7f595a190700 1 -- 172.21.15.17:0/3518917430 >> v1:172.21.15.17:6789/0 conn(0x7f59540b31d0 legacy=0x7f5954066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:27.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.362+0000 7f595a190700 1 -- 172.21.15.17:0/3518917430 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5954066af0 con 0x7f59541489f0 2022-08-01T19:37:27.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.362+0000 7f595a190700 1 --2- 172.21.15.17:0/3518917430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59541489f0 0x7f595414ade0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f594c0032f0 tx=0x7f594c0091d0).ready entity=mon.1 client_cookie=f86bda34ffda0346 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:27.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.362+0000 7f595898d700 1 -- 172.21.15.17:0/3518917430 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f594c00b600 con 0x7f59541489f0 2022-08-01T19:37:27.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.362+0000 7f595898d700 1 -- 172.21.15.17:0/3518917430 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594c00b760 con 0x7f59541489f0 2022-08-01T19:37:27.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.362+0000 7f595898d700 1 -- 172.21.15.17:0/3518917430 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f594c00b8d0 con 0x7f59541489f0 2022-08-01T19:37:27.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.362+0000 7f595a991700 1 -- 172.21.15.17:0/3518917430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59541489f0 msgr2=0x7f595414ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:27.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.362+0000 7f595a991700 1 --2- 172.21.15.17:0/3518917430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59541489f0 0x7f595414ade0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f594c0032f0 tx=0x7f594c0091d0).stop 2022-08-01T19:37:27.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.362+0000 7f595a991700 1 -- 172.21.15.17:0/3518917430 shutdown_connections 2022-08-01T19:37:27.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.362+0000 7f595a991700 1 --2- 172.21.15.17:0/3518917430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59541489f0 0x7f595414ade0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:27.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.362+0000 7f595a991700 1 -- 172.21.15.17:0/3518917430 >> 172.21.15.17:0/3518917430 conn(0x7f5954063870 msgr2=0x7f5954063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:27.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.362+0000 7f595a991700 1 -- 172.21.15.17:0/3518917430 shutdown_connections 2022-08-01T19:37:27.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.362+0000 7f595a991700 1 -- 172.21.15.17:0/3518917430 wait complete. 2022-08-01T19:37:27.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.366+0000 7f595a991700 1 Processor -- start 2022-08-01T19:37:27.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.366+0000 7f595a991700 1 -- start start 2022-08-01T19:37:27.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.366+0000 7f595a991700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59540a9da0 0x7f5954145d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:27.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.366+0000 7f595a991700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59540b31d0 0x7f59541382d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:27.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.366+0000 7f595a991700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f59541489f0 0x7f59540ab340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:27.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.366+0000 7f595a991700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5954146be0 con 0x7f59540a9da0 2022-08-01T19:37:27.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.366+0000 7f595a991700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59541466a0 con 0x7f59540b31d0 2022-08-01T19:37:27.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.366+0000 7f595a991700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5954146940 con 0x7f59541489f0 2022-08-01T19:37:27.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.366+0000 7f595998f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59540a9da0 0x7f5954145d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:27.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.366+0000 7f595998f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59540a9da0 0x7f5954145d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38772/0 (socket says 172.21.15.17:38772) 2022-08-01T19:37:27.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.366+0000 7f595998f700 1 -- 172.21.15.17:0/2715125107 learned_addr learned my addr 172.21.15.17:0/2715125107 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:27.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.366+0000 7f595a190700 1 --2- 172.21.15.17:0/2715125107 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f59541489f0 0x7f59540ab340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:27.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.366+0000 7f595918e700 1 --2- 172.21.15.17:0/2715125107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59540b31d0 0x7f59541382d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:27.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.366+0000 7f595998f700 1 -- 172.21.15.17:0/2715125107 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f59541489f0 msgr2=0x7f59540ab340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:27.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.366+0000 7f595998f700 1 --2- 172.21.15.17:0/2715125107 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f59541489f0 0x7f59540ab340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:27.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.366+0000 7f595998f700 1 -- 172.21.15.17:0/2715125107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59540b31d0 msgr2=0x7f59541382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:27.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.366+0000 7f595998f700 1 --2- 172.21.15.17:0/2715125107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59540b31d0 0x7f59541382d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:27.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.366+0000 7f595998f700 1 -- 172.21.15.17:0/2715125107 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f594c009470 con 0x7f59540a9da0 2022-08-01T19:37:27.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.366+0000 7f595998f700 1 --2- 172.21.15.17:0/2715125107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59540a9da0 0x7f5954145d40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f59500011d0 tx=0x7f5950007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:27.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.366+0000 7f5942ffd700 1 -- 172.21.15.17:0/2715125107 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f595000ce70 con 0x7f59540a9da0 2022-08-01T19:37:27.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.366+0000 7f5942ffd700 1 -- 172.21.15.17:0/2715125107 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59500072e0 con 0x7f59540a9da0 2022-08-01T19:37:27.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.366+0000 7f5942ffd700 1 -- 172.21.15.17:0/2715125107 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5950005800 con 0x7f59540a9da0 2022-08-01T19:37:27.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.366+0000 7f595a991700 1 -- 172.21.15.17:0/2715125107 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59540b18b0 con 0x7f59540a9da0 2022-08-01T19:37:27.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.366+0000 7f595a991700 1 -- 172.21.15.17:0/2715125107 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5954136c20 con 0x7f59540a9da0 2022-08-01T19:37:27.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.370+0000 7f595a991700 1 -- 172.21.15.17:0/2715125107 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5954066d90 con 0x7f59540a9da0 2022-08-01T19:37:27.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.370+0000 7f5942ffd700 1 -- 172.21.15.17:0/2715125107 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5950010070 con 0x7f59540a9da0 2022-08-01T19:37:27.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.370+0000 7f5942ffd700 1 --2- 172.21.15.17:0/2715125107 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5934053a70 0x7f5934055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:27.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.370+0000 7f5942ffd700 1 -- 172.21.15.17:0/2715125107 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f595006b830 con 0x7f59540a9da0 2022-08-01T19:37:27.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.370+0000 7f595918e700 1 --2- 172.21.15.17:0/2715125107 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5934053a70 0x7f5934055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:27.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.370+0000 7f595918e700 1 --2- 172.21.15.17:0/2715125107 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5934053a70 0x7f5934055f30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f594400b000 tx=0x7f5944009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:27.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.378+0000 7f5942ffd700 1 -- 172.21.15.17:0/2715125107 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f59500425c0 con 0x7f59540a9da0 2022-08-01T19:37:27.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:27.502+0000 7f595a991700 1 -- 172.21.15.17:0/2715125107 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f59540574e0 con 0x7f59540a9da0 2022-08-01T19:37:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.230+0000 7f5942ffd700 1 -- 172.21.15.17:0/2715125107 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v48) v1 ==== 173+0+0 (secure 0 0 0) 0x7f5950045be0 con 0x7f59540a9da0 2022-08-01T19:37:28.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.238+0000 7f595a991700 1 -- 172.21.15.17:0/2715125107 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5934053a70 msgr2=0x7f5934055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:28.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.238+0000 7f595a991700 1 --2- 172.21.15.17:0/2715125107 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5934053a70 0x7f5934055f30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f594400b000 tx=0x7f5944009040).stop 2022-08-01T19:37:28.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.238+0000 7f595a991700 1 -- 172.21.15.17:0/2715125107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59540a9da0 msgr2=0x7f5954145d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:28.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.238+0000 7f595a991700 1 --2- 172.21.15.17:0/2715125107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59540a9da0 0x7f5954145d40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f59500011d0 tx=0x7f5950007b50).stop 2022-08-01T19:37:28.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.238+0000 7f595a991700 1 -- 172.21.15.17:0/2715125107 shutdown_connections 2022-08-01T19:37:28.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.238+0000 7f595a991700 1 --2- 172.21.15.17:0/2715125107 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5934053a70 0x7f5934055f30 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:28.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.238+0000 7f595a991700 1 --2- 172.21.15.17:0/2715125107 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f59541489f0 0x7f59540ab340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:28.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.238+0000 7f595a991700 1 --2- 172.21.15.17:0/2715125107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59540b31d0 0x7f59541382d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:28.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.238+0000 7f595a991700 1 --2- 172.21.15.17:0/2715125107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59540a9da0 0x7f5954145d40 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:28.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.238+0000 7f595a991700 1 -- 172.21.15.17:0/2715125107 >> 172.21.15.17:0/2715125107 conn(0x7f5954063870 msgr2=0x7f5954063c50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:28.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.238+0000 7f595a991700 1 -- 172.21.15.17:0/2715125107 shutdown_connections 2022-08-01T19:37:28.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.238+0000 7f595a991700 1 -- 172.21.15.17:0/2715125107 wait complete. 2022-08-01T19:37:28.246 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' removed 2022-08-01T19:37:28.256 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-08-01T19:37:28.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.366+0000 7f61dcf7e700 1 Processor -- start 2022-08-01T19:37:28.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.366+0000 7f61dcf7e700 1 -- start start 2022-08-01T19:37:28.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.366+0000 7f61dcf7e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f61d80614b0 0x7f61d8061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:28.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.366+0000 7f61dcf7e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f61d81405d0 0x7f61d8140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:28.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.366+0000 7f61dcf7e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d80aee80 con 0x7f61d80ad750 2022-08-01T19:37:28.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.366+0000 7f61dcf7e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61d80b70b0 con 0x7f61d81405d0 2022-08-01T19:37:28.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.366+0000 7f61dcf7e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61d80af0b0 con 0x7f61d80614b0 2022-08-01T19:37:28.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.366+0000 7f61d7fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f61d81405d0 0x7f61d8140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:28.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.366+0000 7f61d7fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f61d81405d0 0x7f61d8140a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38782/0 (socket says 172.21.15.17:38782) 2022-08-01T19:37:28.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.366+0000 7f61d7fff700 1 -- 172.21.15.17:0/492030004 learned_addr learned my addr 172.21.15.17:0/492030004 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:28.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.366+0000 7f61d77fe700 1 --2- 172.21.15.17:0/492030004 >> v2:172.21.15.77:3300/0 conn(0x7f61d80614b0 0x7f61d8061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:28.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.366+0000 7f61d7fff700 1 -- 172.21.15.17:0/492030004 >> v2:172.21.15.77:3300/0 conn(0x7f61d80614b0 msgr2=0x7f61d8061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:28.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.366+0000 7f61d7fff700 1 --2- 172.21.15.17:0/492030004 >> v2:172.21.15.77:3300/0 conn(0x7f61d80614b0 0x7f61d8061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:28.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.366+0000 7f61d7fff700 1 -- 172.21.15.17:0/492030004 >> v1:172.21.15.77:6789/0 conn(0x7f61d80ad750 legacy=0x7f61d80b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:28.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.366+0000 7f61d7fff700 1 -- 172.21.15.17:0/492030004 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d80af350 con 0x7f61d81405d0 2022-08-01T19:37:28.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.366+0000 7f61d7fff700 1 --2- 172.21.15.17:0/492030004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61d81405d0 0x7f61d8140a60 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f61d80aed30 tx=0x7f61d0009290).ready entity=mon.0 client_cookie=3e3f4eab638f4339 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:28.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.366+0000 7f61d67fc700 1 -- 172.21.15.17:0/492030004 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61d000b860 con 0x7f61d81405d0 2022-08-01T19:37:28.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.366+0000 7f61d67fc700 1 -- 172.21.15.17:0/492030004 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d000b9c0 con 0x7f61d81405d0 2022-08-01T19:37:28.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.366+0000 7f61d67fc700 1 -- 172.21.15.17:0/492030004 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61d000bb30 con 0x7f61d81405d0 2022-08-01T19:37:28.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.366+0000 7f61dcf7e700 1 -- 172.21.15.17:0/492030004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61d81405d0 msgr2=0x7f61d8140a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:28.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.366+0000 7f61dcf7e700 1 --2- 172.21.15.17:0/492030004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61d81405d0 0x7f61d8140a60 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f61d80aed30 tx=0x7f61d0009290).stop 2022-08-01T19:37:28.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.366+0000 7f61dcf7e700 1 -- 172.21.15.17:0/492030004 shutdown_connections 2022-08-01T19:37:28.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.366+0000 7f61dcf7e700 1 --2- 172.21.15.17:0/492030004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61d81405d0 0x7f61d8140a60 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:28.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.366+0000 7f61dcf7e700 1 --2- 172.21.15.17:0/492030004 >> v2:172.21.15.77:3300/0 conn(0x7f61d80614b0 0x7f61d8061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:28.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.366+0000 7f61dcf7e700 1 -- 172.21.15.17:0/492030004 >> 172.21.15.17:0/492030004 conn(0x7f61d8066f70 msgr2=0x7f61d8067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:28.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.366+0000 7f61dcf7e700 1 -- 172.21.15.17:0/492030004 shutdown_connections 2022-08-01T19:37:28.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.366+0000 7f61dcf7e700 1 -- 172.21.15.17:0/492030004 wait complete. 2022-08-01T19:37:28.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.366+0000 7f61dcf7e700 1 Processor -- start 2022-08-01T19:37:28.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.366+0000 7f61dcf7e700 1 -- start start 2022-08-01T19:37:28.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.370+0000 7f61dcf7e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61d80614b0 0x7f61d80b10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:28.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.370+0000 7f61dcf7e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61d80ad750 0x7f61d80b2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:28.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.370+0000 7f61dcf7e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f61d81405d0 0x7f61d80b2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:28.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.370+0000 7f61dcf7e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d8152870 con 0x7f61d80ad750 2022-08-01T19:37:28.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.370+0000 7f61dcf7e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d80b40e0 con 0x7f61d80614b0 2022-08-01T19:37:28.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.370+0000 7f61dcf7e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d80b3ba0 con 0x7f61d81405d0 2022-08-01T19:37:28.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.370+0000 7f61d6ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61d80ad750 0x7f61d80b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:28.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.370+0000 7f61d77fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61d80614b0 0x7f61d80b10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:28.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.370+0000 7f61d77fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61d80614b0 0x7f61d80b10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:36708/0 (socket says 172.21.15.17:36708) 2022-08-01T19:37:28.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.370+0000 7f61d77fe700 1 -- 172.21.15.17:0/1888012440 learned_addr learned my addr 172.21.15.17:0/1888012440 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:28.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.370+0000 7f61d7fff700 1 --2- 172.21.15.17:0/1888012440 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f61d81405d0 0x7f61d80b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:28.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.370+0000 7f61d77fe700 1 -- 172.21.15.17:0/1888012440 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f61d81405d0 msgr2=0x7f61d80b2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:28.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.370+0000 7f61d77fe700 1 --2- 172.21.15.17:0/1888012440 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f61d81405d0 0x7f61d80b2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:28.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.370+0000 7f61d77fe700 1 -- 172.21.15.17:0/1888012440 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61d80ad750 msgr2=0x7f61d80b2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:28.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.370+0000 7f61d77fe700 1 --2- 172.21.15.17:0/1888012440 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61d80ad750 0x7f61d80b2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:28.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.370+0000 7f61d77fe700 1 -- 172.21.15.17:0/1888012440 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d0009530 con 0x7f61d80614b0 2022-08-01T19:37:28.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.370+0000 7f61d77fe700 1 --2- 172.21.15.17:0/1888012440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61d80614b0 0x7f61d80b10e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f61d80b32c0 tx=0x7f61c8006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:28.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.370+0000 7f61d4ff9700 1 -- 172.21.15.17:0/1888012440 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61c8010780 con 0x7f61d80614b0 2022-08-01T19:37:28.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.370+0000 7f61d4ff9700 1 -- 172.21.15.17:0/1888012440 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c80108e0 con 0x7f61d80614b0 2022-08-01T19:37:28.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.370+0000 7f61dcf7e700 1 -- 172.21.15.17:0/1888012440 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61d80b3e40 con 0x7f61d80614b0 2022-08-01T19:37:28.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.370+0000 7f61d4ff9700 1 -- 172.21.15.17:0/1888012440 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61c8010c40 con 0x7f61d80614b0 2022-08-01T19:37:28.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.370+0000 7f61dcf7e700 1 -- 172.21.15.17:0/1888012440 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d80a2950 con 0x7f61d80614b0 2022-08-01T19:37:28.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.374+0000 7f61dcf7e700 1 -- 172.21.15.17:0/1888012440 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61d80af5f0 con 0x7f61d80614b0 2022-08-01T19:37:28.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.374+0000 7f61d4ff9700 1 -- 172.21.15.17:0/1888012440 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f61c801c460 con 0x7f61d80614b0 2022-08-01T19:37:28.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.374+0000 7f61d4ff9700 1 --2- 172.21.15.17:0/1888012440 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f61b8053a70 0x7f61b8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:28.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.374+0000 7f61d4ff9700 1 -- 172.21.15.17:0/1888012440 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f61c806c1c0 con 0x7f61d80614b0 2022-08-01T19:37:28.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.374+0000 7f61d6ffd700 1 --2- 172.21.15.17:0/1888012440 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f61b8053a70 0x7f61b8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:28.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.374+0000 7f61d6ffd700 1 --2- 172.21.15.17:0/1888012440 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f61b8053a70 0x7f61b8055f30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f61cc00d000 tx=0x7f61cc000bc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:28.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.390+0000 7f61d4ff9700 1 -- 172.21.15.17:0/1888012440 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f61c8043160 con 0x7f61d80614b0 2022-08-01T19:37:28.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.514+0000 7f61dcf7e700 1 -- 172.21.15.17:0/1888012440 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f61d80b2080 con 0x7f61d80614b0 2022-08-01T19:37:28.531 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.526+0000 7f61d4ff9700 1 -- 172.21.15.17:0/1888012440 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v5) v1 ==== 104+0+0 (secure 0 0 0) 0x7f61c8046780 con 0x7f61d80614b0 2022-08-01T19:37:28.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.530+0000 7f61dcf7e700 1 -- 172.21.15.17:0/1888012440 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f61b8053a70 msgr2=0x7f61b8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:28.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.530+0000 7f61dcf7e700 1 --2- 172.21.15.17:0/1888012440 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f61b8053a70 0x7f61b8055f30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f61cc00d000 tx=0x7f61cc000bc0).stop 2022-08-01T19:37:28.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.530+0000 7f61dcf7e700 1 -- 172.21.15.17:0/1888012440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61d80614b0 msgr2=0x7f61d80b10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:28.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.530+0000 7f61dcf7e700 1 --2- 172.21.15.17:0/1888012440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61d80614b0 0x7f61d80b10e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f61d80b32c0 tx=0x7f61c8006df0).stop 2022-08-01T19:37:28.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.530+0000 7f61dcf7e700 1 -- 172.21.15.17:0/1888012440 shutdown_connections 2022-08-01T19:37:28.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.530+0000 7f61dcf7e700 1 --2- 172.21.15.17:0/1888012440 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f61b8053a70 0x7f61b8055f30 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:28.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.530+0000 7f61dcf7e700 1 --2- 172.21.15.17:0/1888012440 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f61d81405d0 0x7f61d80b2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:28.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.530+0000 7f61dcf7e700 1 --2- 172.21.15.17:0/1888012440 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61d80ad750 0x7f61d80b2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:28.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.530+0000 7f61dcf7e700 1 --2- 172.21.15.17:0/1888012440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61d80614b0 0x7f61d80b10e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:28.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.530+0000 7f61dcf7e700 1 -- 172.21.15.17:0/1888012440 >> 172.21.15.17:0/1888012440 conn(0x7f61d8066f70 msgr2=0x7f61d8143840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:28.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.530+0000 7f61dcf7e700 1 -- 172.21.15.17:0/1888012440 shutdown_connections 2022-08-01T19:37:28.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.530+0000 7f61dcf7e700 1 -- 172.21.15.17:0/1888012440 wait complete. 2022-08-01T19:37:28.547 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead' 2022-08-01T19:37:28.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f23190b7700 1 Processor -- start 2022-08-01T19:37:28.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f23190b7700 1 -- start start 2022-08-01T19:37:28.660 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f23190b7700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f23140a8c50 0x7f2314152d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:28.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f23190b7700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23140a7250 con 0x7f23140b2710 2022-08-01T19:37:28.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f23190b7700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23140a7480 con 0x7f23140b01a0 2022-08-01T19:37:28.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f23190b7700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23140a7720 con 0x7f23140a8c50 2022-08-01T19:37:28.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f2313fff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f23140b2710 0x7f231414afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59220/0 (socket says 172.21.15.17:59220) 2022-08-01T19:37:28.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f2313fff700 1 -- 172.21.15.17:0/1218011561 learned_addr learned my addr 172.21.15.17:0/1218011561 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:37:28.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f23188b6700 1 --2- 172.21.15.17:0/1218011561 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f23140a8c50 0x7f2314152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:28.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f2312ffd700 1 -- 172.21.15.17:0/1218011561 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 667295173 0 0) 0x7f23140a7250 con 0x7f23140b2710 2022-08-01T19:37:28.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f2312ffd700 1 -- 172.21.15.17:0/1218011561 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22f8003260 con 0x7f23140b2710 2022-08-01T19:37:28.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f2312ffd700 1 -- 172.21.15.17:0/1218011561 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2077538714 0 0) 0x7f23140a7480 con 0x7f23140b01a0 2022-08-01T19:37:28.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f2312ffd700 1 -- 172.21.15.17:0/1218011561 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23140a7250 con 0x7f23140b01a0 2022-08-01T19:37:28.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f2312ffd700 1 -- 172.21.15.17:0/1218011561 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3631971742 0 0) 0x7f22f8003260 con 0x7f23140b2710 2022-08-01T19:37:28.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f2312ffd700 1 -- 172.21.15.17:0/1218011561 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f23140a8c50 msgr2=0x7f2314152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:28.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f2312ffd700 1 --2- 172.21.15.17:0/1218011561 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f23140a8c50 0x7f2314152d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:28.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f2312ffd700 1 -- 172.21.15.17:0/1218011561 >> v1:172.21.15.77:6789/0 conn(0x7f23140b01a0 legacy=0x7f23140a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:28.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f2312ffd700 1 -- 172.21.15.17:0/1218011561 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23140a79c0 con 0x7f23140b2710 2022-08-01T19:37:28.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f2312ffd700 1 -- 172.21.15.17:0/1218011561 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7f23040019e0 con 0x7f23140b2710 2022-08-01T19:37:28.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f2312ffd700 1 -- 172.21.15.17:0/1218011561 >> v1:172.21.15.17:6789/0 conn(0x7f23140b2710 legacy=0x7f231414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:28.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f2312ffd700 1 --2- 172.21.15.17:0/1218011561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22f8003910 0x7f22f8003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:28.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f2312ffd700 1 --2- 172.21.15.17:0/1218011561 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f23140a8c50 0x7f22f8004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:28.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f2312ffd700 1 --2- 172.21.15.17:0/1218011561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22f80049f0 0x7f22f8004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:28.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f2312ffd700 1 -- 172.21.15.17:0/1218011561 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23140a7720 con 0x7f22f8003910 2022-08-01T19:37:28.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f2312ffd700 1 -- 172.21.15.17:0/1218011561 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f800b780 con 0x7f22f80049f0 2022-08-01T19:37:28.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f2312ffd700 1 -- 172.21.15.17:0/1218011561 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22f800b940 con 0x7f23140a8c50 2022-08-01T19:37:28.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f2313fff700 1 --2- 172.21.15.17:0/1218011561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22f8003910 0x7f22f8003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:28.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f23137fe700 1 --2- 172.21.15.17:0/1218011561 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f23140a8c50 0x7f22f8004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:28.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f23188b6700 1 --2- 172.21.15.17:0/1218011561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22f80049f0 0x7f22f8004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:28.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f2313fff700 1 -- 172.21.15.17:0/1218011561 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f23140a8c50 msgr2=0x7f22f8004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:28.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f2313fff700 1 --2- 172.21.15.17:0/1218011561 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f23140a8c50 0x7f22f8004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:28.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f2313fff700 1 -- 172.21.15.17:0/1218011561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22f80049f0 msgr2=0x7f22f8004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:28.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f2313fff700 1 --2- 172.21.15.17:0/1218011561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22f80049f0 0x7f22f8004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:28.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f2313fff700 1 -- 172.21.15.17:0/1218011561 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f8003260 con 0x7f22f8003910 2022-08-01T19:37:28.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f2313fff700 1 --2- 172.21.15.17:0/1218011561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22f8003910 0x7f22f8003d10 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f23040017b0 tx=0x7f23140b0640).ready entity=mon.0 client_cookie=3e9f217d88246f5 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:28.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f2312ffd700 1 -- 172.21.15.17:0/1218011561 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f230400cca0 con 0x7f22f8003910 2022-08-01T19:37:28.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f2312ffd700 1 -- 172.21.15.17:0/1218011561 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f230400ce00 con 0x7f22f8003910 2022-08-01T19:37:28.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f2312ffd700 1 -- 172.21.15.17:0/1218011561 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2304005b80 con 0x7f22f8003910 2022-08-01T19:37:28.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f23190b7700 1 -- 172.21.15.17:0/1218011561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22f8003910 msgr2=0x7f22f8003d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:28.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.654+0000 7f23190b7700 1 --2- 172.21.15.17:0/1218011561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22f8003910 0x7f22f8003d10 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f23040017b0 tx=0x7f23140b0640).stop 2022-08-01T19:37:28.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.658+0000 7f23188b6700 1 -- 172.21.15.17:0/1218011561 reap_dead start 2022-08-01T19:37:28.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.658+0000 7f23190b7700 1 -- 172.21.15.17:0/1218011561 shutdown_connections 2022-08-01T19:37:28.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.658+0000 7f23190b7700 1 -- 172.21.15.17:0/1218011561 >> 172.21.15.17:0/1218011561 conn(0x7f2314060310 msgr2=0x7f23140b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:28.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.658+0000 7f23190b7700 1 -- 172.21.15.17:0/1218011561 shutdown_connections 2022-08-01T19:37:28.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.658+0000 7f23190b7700 1 -- 172.21.15.17:0/1218011561 wait complete. 2022-08-01T19:37:28.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.658+0000 7f23190b7700 1 Processor -- start 2022-08-01T19:37:28.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.658+0000 7f23190b7700 1 -- start start 2022-08-01T19:37:28.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.658+0000 7f23190b7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22f8003910 0x7f23140aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:28.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.658+0000 7f23190b7700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f23140a9f10 0x7f23140b5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:28.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.658+0000 7f23190b7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23140abba0 0x7f23140ac020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:28.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.658+0000 7f23190b7700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23140af910 con 0x7f22f8003910 2022-08-01T19:37:28.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.658+0000 7f23190b7700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23140ab740 con 0x7f23140abba0 2022-08-01T19:37:28.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.658+0000 7f23190b7700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23140b4f50 con 0x7f23140a9f10 2022-08-01T19:37:28.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.662+0000 7f2313fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22f8003910 0x7f23140aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:28.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.662+0000 7f23188b6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23140abba0 0x7f23140ac020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:28.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.662+0000 7f23137fe700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f23140a9f10 0x7f23140b5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:28.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.662+0000 7f23188b6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23140abba0 0x7f23140ac020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:36732/0 (socket says 172.21.15.17:36732) 2022-08-01T19:37:28.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.662+0000 7f23188b6700 1 -- 172.21.15.17:0/3921772534 learned_addr learned my addr 172.21.15.17:0/3921772534 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:28.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.662+0000 7f2313fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22f8003910 0x7f23140aef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38806/0 (socket says 172.21.15.17:38806) 2022-08-01T19:37:28.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.662+0000 7f2313fff700 1 -- 172.21.15.17:0/3921772534 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f23140a9f10 msgr2=0x7f23140b5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:28.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.662+0000 7f2313fff700 1 --2- 172.21.15.17:0/3921772534 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f23140a9f10 0x7f23140b5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:28.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.662+0000 7f2313fff700 1 -- 172.21.15.17:0/3921772534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23140abba0 msgr2=0x7f23140ac020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:28.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.662+0000 7f2313fff700 1 --2- 172.21.15.17:0/3921772534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23140abba0 0x7f23140ac020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:28.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.662+0000 7f2313fff700 1 -- 172.21.15.17:0/3921772534 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23140ac510 con 0x7f22f8003910 2022-08-01T19:37:28.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.662+0000 7f2313fff700 1 --2- 172.21.15.17:0/3921772534 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22f8003910 0x7f23140aef90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f23040017b0 tx=0x7f2304013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:28.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.662+0000 7f23117fa700 1 -- 172.21.15.17:0/3921772534 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f230400cca0 con 0x7f22f8003910 2022-08-01T19:37:28.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.662+0000 7f23117fa700 1 -- 172.21.15.17:0/3921772534 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23040072e0 con 0x7f22f8003910 2022-08-01T19:37:28.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.662+0000 7f23190b7700 1 -- 172.21.15.17:0/3921772534 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23140b4a10 con 0x7f22f8003910 2022-08-01T19:37:28.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.662+0000 7f23117fa700 1 -- 172.21.15.17:0/3921772534 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2304007640 con 0x7f22f8003910 2022-08-01T19:37:28.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.662+0000 7f23190b7700 1 -- 172.21.15.17:0/3921772534 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23140b5a00 con 0x7f22f8003910 2022-08-01T19:37:28.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.662+0000 7f23117fa700 1 -- 172.21.15.17:0/3921772534 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2304025070 con 0x7f22f8003910 2022-08-01T19:37:28.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.662+0000 7f23190b7700 1 -- 172.21.15.17:0/3921772534 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23140ab8b0 con 0x7f22f8003910 2022-08-01T19:37:28.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.666+0000 7f23117fa700 1 --2- 172.21.15.17:0/3921772534 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f22ec053b00 0x7f22ec055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:28.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.666+0000 7f23117fa700 1 -- 172.21.15.17:0/3921772534 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f230406c370 con 0x7f22f8003910 2022-08-01T19:37:28.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.666+0000 7f23137fe700 1 --2- 172.21.15.17:0/3921772534 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f22ec053b00 0x7f22ec055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:28.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.666+0000 7f23137fe700 1 --2- 172.21.15.17:0/3921772534 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f22ec053b00 0x7f22ec055fc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f22fc003000 tx=0x7f22fc007a00).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:28.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.670+0000 7f23117fa700 1 -- 172.21.15.17:0/3921772534 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2304043300 con 0x7f22f8003910 2022-08-01T19:37:28.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:28.842+0000 7f23190b7700 1 -- 172.21.15.17:0/3921772534 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]} v 0) v1 -- 0x7f23140574e0 con 0x7f22f8003910 2022-08-01T19:37:29.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.234+0000 7f23117fa700 1 -- 172.21.15.17:0/3921772534 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]}]=0 v154) v1 ==== 144+0+0 (secure 0 0 0) 0x7f2304046920 con 0x7f22f8003910 2022-08-01T19:37:29.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.238+0000 7f23190b7700 1 -- 172.21.15.17:0/3921772534 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f22ec053b00 msgr2=0x7f22ec055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:29.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.238+0000 7f23190b7700 1 --2- 172.21.15.17:0/3921772534 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f22ec053b00 0x7f22ec055fc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f22fc003000 tx=0x7f22fc007a00).stop 2022-08-01T19:37:29.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.238+0000 7f23190b7700 1 -- 172.21.15.17:0/3921772534 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22f8003910 msgr2=0x7f23140aef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:29.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.238+0000 7f23190b7700 1 --2- 172.21.15.17:0/3921772534 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22f8003910 0x7f23140aef90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f23040017b0 tx=0x7f2304013000).stop 2022-08-01T19:37:29.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.238+0000 7f23190b7700 1 -- 172.21.15.17:0/3921772534 shutdown_connections 2022-08-01T19:37:29.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.238+0000 7f23190b7700 1 --2- 172.21.15.17:0/3921772534 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f22ec053b00 0x7f22ec055fc0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:29.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.238+0000 7f23190b7700 1 --2- 172.21.15.17:0/3921772534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23140abba0 0x7f23140ac020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:29.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.238+0000 7f23190b7700 1 --2- 172.21.15.17:0/3921772534 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f23140a9f10 0x7f23140b5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:29.246 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.238+0000 7f23190b7700 1 --2- 172.21.15.17:0/3921772534 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22f8003910 0x7f23140aef90 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:29.246 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.238+0000 7f23190b7700 1 -- 172.21.15.17:0/3921772534 >> 172.21.15.17:0/3921772534 conn(0x7f2314060310 msgr2=0x7f2314149af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:29.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.238+0000 7f23190b7700 1 -- 172.21.15.17:0/3921772534 shutdown_connections 2022-08-01T19:37:29.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.238+0000 7f23190b7700 1 -- 172.21.15.17:0/3921772534 wait complete. 2022-08-01T19:37:29.258 INFO:tasks.cephfs_test_runner:test_drop_cache_command_dead (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2022-08-01T19:37:29.259 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command_dead_timeout (tasks.cephfs.test_misc.TestCacheDrop) 2022-08-01T19:37:29.260 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout' 2022-08-01T19:37:29.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.370+0000 7f98b3fff700 1 Processor -- start 2022-08-01T19:37:29.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.370+0000 7f98b3fff700 1 -- start start 2022-08-01T19:37:29.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.370+0000 7f98b3fff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f98b40b2710 0x7f98b40abca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:29.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.370+0000 7f98b3fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f98b40b13f0 0x7f98b40ac190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:29.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.370+0000 7f98b3fff700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98b40adf10 con 0x7f98b40b1920 2022-08-01T19:37:29.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.370+0000 7f98b3fff700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98b40ae1b0 con 0x7f98b40b13f0 2022-08-01T19:37:29.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.370+0000 7f98b3fff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98b40a9520 con 0x7f98b40b2710 2022-08-01T19:37:29.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.370+0000 7f98b27fc700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f98b40b13f0 0x7f98b40ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:29.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.370+0000 7f98b27fc700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f98b40b13f0 0x7f98b40ac190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38816/0 (socket says 172.21.15.17:38816) 2022-08-01T19:37:29.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.370+0000 7f98b27fc700 1 -- 172.21.15.17:0/1195565293 learned_addr learned my addr 172.21.15.17:0/1195565293 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:29.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.370+0000 7f98b2ffd700 1 --2- 172.21.15.17:0/1195565293 >> v2:172.21.15.77:3300/0 conn(0x7f98b40b2710 0x7f98b40abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:29.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.370+0000 7f98b27fc700 1 -- 172.21.15.17:0/1195565293 >> v2:172.21.15.77:3300/0 conn(0x7f98b40b2710 msgr2=0x7f98b40abca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:29.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.370+0000 7f98b27fc700 1 --2- 172.21.15.17:0/1195565293 >> v2:172.21.15.77:3300/0 conn(0x7f98b40b2710 0x7f98b40abca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:29.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.370+0000 7f98b27fc700 1 -- 172.21.15.17:0/1195565293 >> v1:172.21.15.77:6789/0 conn(0x7f98b40b1920 legacy=0x7f98b40afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:29.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.370+0000 7f98b27fc700 1 -- 172.21.15.17:0/1195565293 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b40a99c0 con 0x7f98b40b13f0 2022-08-01T19:37:29.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.370+0000 7f98b1ffb700 1 -- 172.21.15.17:0/1195565293 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3680241690 0 0) 0x7f98b40adf10 con 0x7f98b40b1920 2022-08-01T19:37:29.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.370+0000 7f98b27fc700 1 --2- 172.21.15.17:0/1195565293 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98b40b13f0 0x7f98b40ac190 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f98a4002f70 tx=0x7f98a4006eb0).ready entity=mon.0 client_cookie=5c8d9f14b959209b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:29.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.370+0000 7f98b1ffb700 1 -- 172.21.15.17:0/1195565293 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98a400d5c0 con 0x7f98b40b13f0 2022-08-01T19:37:29.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.370+0000 7f98b1ffb700 1 -- 172.21.15.17:0/1195565293 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a4000e90 con 0x7f98b40b13f0 2022-08-01T19:37:29.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.370+0000 7f98b1ffb700 1 -- 172.21.15.17:0/1195565293 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98a400d910 con 0x7f98b40b13f0 2022-08-01T19:37:29.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.370+0000 7f98b3fff700 1 -- 172.21.15.17:0/1195565293 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98b40b13f0 msgr2=0x7f98b40ac190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:29.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.370+0000 7f98b3fff700 1 --2- 172.21.15.17:0/1195565293 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98b40b13f0 0x7f98b40ac190 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f98a4002f70 tx=0x7f98a4006eb0).stop 2022-08-01T19:37:29.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.370+0000 7f98b3fff700 1 -- 172.21.15.17:0/1195565293 shutdown_connections 2022-08-01T19:37:29.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.370+0000 7f98b3fff700 1 --2- 172.21.15.17:0/1195565293 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98b40b13f0 0x7f98b40ac190 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:29.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.370+0000 7f98b3fff700 1 --2- 172.21.15.17:0/1195565293 >> v2:172.21.15.77:3300/0 conn(0x7f98b40b2710 0x7f98b40abca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:29.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.370+0000 7f98b3fff700 1 -- 172.21.15.17:0/1195565293 >> 172.21.15.17:0/1195565293 conn(0x7f98b4060310 msgr2=0x7f98b40b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:29.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.370+0000 7f98b3fff700 1 -- 172.21.15.17:0/1195565293 shutdown_connections 2022-08-01T19:37:29.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.370+0000 7f98b3fff700 1 -- 172.21.15.17:0/1195565293 wait complete. 2022-08-01T19:37:29.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.374+0000 7f98b3fff700 1 Processor -- start 2022-08-01T19:37:29.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.374+0000 7f98b3fff700 1 -- start start 2022-08-01T19:37:29.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.374+0000 7f98b3fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98b40b13f0 0x7f98b40ac010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:29.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.374+0000 7f98b3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98b40b1920 0x7f98b412f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:29.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.374+0000 7f98b3fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98b40b2710 0x7f98b412f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:29.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.374+0000 7f98b3fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b405f130 con 0x7f98b40b1920 2022-08-01T19:37:29.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.374+0000 7f98b3fff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b4137dd0 con 0x7f98b40b13f0 2022-08-01T19:37:29.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.374+0000 7f98b3fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b412ebd0 con 0x7f98b40b2710 2022-08-01T19:37:29.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.374+0000 7f98b27fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98b40b1920 0x7f98b412f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:29.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.374+0000 7f98b2ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98b40b13f0 0x7f98b40ac010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:29.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.374+0000 7f98b2ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98b40b13f0 0x7f98b40ac010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:36742/0 (socket says 172.21.15.17:36742) 2022-08-01T19:37:29.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.374+0000 7f98b2ffd700 1 -- 172.21.15.17:0/2645584293 learned_addr learned my addr 172.21.15.17:0/2645584293 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:29.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.374+0000 7f98b27fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98b40b1920 0x7f98b412f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38822/0 (socket says 172.21.15.17:38822) 2022-08-01T19:37:29.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.374+0000 7f98b37fe700 1 --2- 172.21.15.17:0/2645584293 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98b40b2710 0x7f98b412f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:29.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.374+0000 7f98b2ffd700 1 -- 172.21.15.17:0/2645584293 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98b40b2710 msgr2=0x7f98b412f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:29.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.374+0000 7f98b2ffd700 1 --2- 172.21.15.17:0/2645584293 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98b40b2710 0x7f98b412f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:29.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.374+0000 7f98b2ffd700 1 -- 172.21.15.17:0/2645584293 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98b40b1920 msgr2=0x7f98b412f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:29.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.374+0000 7f98b2ffd700 1 --2- 172.21.15.17:0/2645584293 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98b40b1920 0x7f98b412f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:29.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.374+0000 7f98b2ffd700 1 -- 172.21.15.17:0/2645584293 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a400b1e0 con 0x7f98b40b13f0 2022-08-01T19:37:29.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.374+0000 7f98b27fc700 1 --2- 172.21.15.17:0/2645584293 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98b40b1920 0x7f98b412f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:37:29.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.374+0000 7f98b2ffd700 1 --2- 172.21.15.17:0/2645584293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98b40b13f0 0x7f98b40ac010 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f98a800c000 tx=0x7f98a8006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:29.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.378+0000 7f98a3fff700 1 -- 172.21.15.17:0/2645584293 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98a8010780 con 0x7f98b40b13f0 2022-08-01T19:37:29.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.378+0000 7f98a3fff700 1 -- 172.21.15.17:0/2645584293 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a80108e0 con 0x7f98b40b13f0 2022-08-01T19:37:29.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.378+0000 7f98b3fff700 1 -- 172.21.15.17:0/2645584293 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98b4138250 con 0x7f98b40b13f0 2022-08-01T19:37:29.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.378+0000 7f98a3fff700 1 -- 172.21.15.17:0/2645584293 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98a8010c40 con 0x7f98b40b13f0 2022-08-01T19:37:29.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.378+0000 7f98b3fff700 1 -- 172.21.15.17:0/2645584293 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98b40ab6e0 con 0x7f98b40b13f0 2022-08-01T19:37:29.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.378+0000 7f98b3fff700 1 -- 172.21.15.17:0/2645584293 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98b40ae450 con 0x7f98b40b13f0 2022-08-01T19:37:29.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.378+0000 7f98a3fff700 1 -- 172.21.15.17:0/2645584293 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f98a801c460 con 0x7f98b40b13f0 2022-08-01T19:37:29.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.378+0000 7f98a3fff700 1 --2- 172.21.15.17:0/2645584293 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f98900539e0 0x7f9890055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:29.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.378+0000 7f98a3fff700 1 -- 172.21.15.17:0/2645584293 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f98a806c1a0 con 0x7f98b40b13f0 2022-08-01T19:37:29.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.378+0000 7f98b27fc700 1 --2- 172.21.15.17:0/2645584293 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f98900539e0 0x7f9890055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:29.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.382+0000 7f98b27fc700 1 --2- 172.21.15.17:0/2645584293 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f98900539e0 0x7f9890055ea0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f98a4020000 tx=0x7f98a4015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:29.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.390+0000 7f98a3fff700 1 -- 172.21.15.17:0/2645584293 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f98b40ae450 con 0x7f98b40b13f0 2022-08-01T19:37:29.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:29.554+0000 7f98b3fff700 1 -- 172.21.15.17:0/2645584293 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]} v 0) v1 -- 0x7f98b40ae450 con 0x7f98b40b13f0 2022-08-01T19:37:30.246 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.238+0000 7f98a3fff700 1 -- 172.21.15.17:0/2645584293 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]}]=0 v155) v1 ==== 155+0+0 (secure 0 0 0) 0x7f98b40ae450 con 0x7f98b40b13f0 2022-08-01T19:37:30.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.242+0000 7f98b3fff700 1 -- 172.21.15.17:0/2645584293 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f98900539e0 msgr2=0x7f9890055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:30.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.242+0000 7f98b3fff700 1 --2- 172.21.15.17:0/2645584293 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f98900539e0 0x7f9890055ea0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f98a4020000 tx=0x7f98a4015000).stop 2022-08-01T19:37:30.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.242+0000 7f98b3fff700 1 -- 172.21.15.17:0/2645584293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98b40b13f0 msgr2=0x7f98b40ac010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:30.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.242+0000 7f98b3fff700 1 --2- 172.21.15.17:0/2645584293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98b40b13f0 0x7f98b40ac010 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f98a800c000 tx=0x7f98a8006df0).stop 2022-08-01T19:37:30.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.242+0000 7f98b3fff700 1 -- 172.21.15.17:0/2645584293 shutdown_connections 2022-08-01T19:37:30.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.242+0000 7f98b3fff700 1 --2- 172.21.15.17:0/2645584293 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f98900539e0 0x7f9890055ea0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:30.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.242+0000 7f98b3fff700 1 --2- 172.21.15.17:0/2645584293 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98b40b2710 0x7f98b412f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:30.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.242+0000 7f98b3fff700 1 --2- 172.21.15.17:0/2645584293 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98b40b1920 0x7f98b412f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:30.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.242+0000 7f98b3fff700 1 --2- 172.21.15.17:0/2645584293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98b40b13f0 0x7f98b40ac010 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:30.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.242+0000 7f98b3fff700 1 -- 172.21.15.17:0/2645584293 >> 172.21.15.17:0/2645584293 conn(0x7f98b4060310 msgr2=0x7f98b40b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:30.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.242+0000 7f98b3fff700 1 -- 172.21.15.17:0/2645584293 shutdown_connections 2022-08-01T19:37:30.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.242+0000 7f98b3fff700 1 -- 172.21.15.17:0/2645584293 wait complete. 2022-08-01T19:37:30.261 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-08-01T19:37:30.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.370+0000 7fb8110fe700 1 Processor -- start 2022-08-01T19:37:30.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.370+0000 7fb8110fe700 1 -- start start 2022-08-01T19:37:30.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.370+0000 7fb8110fe700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb80c0a9af0 0x7fb80c0b1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:30.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.370+0000 7fb8110fe700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fb80c0b0850 0x7fb80c0ab130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:30.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.370+0000 7fb8110fe700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb80c0b1df0 con 0x7fb80c0ac3f0 2022-08-01T19:37:30.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.370+0000 7fb8110fe700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb80c0a9ed0 con 0x7fb80c0b0850 2022-08-01T19:37:30.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.370+0000 7fb8110fe700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c0b2ad0 con 0x7fb80c0a9af0 2022-08-01T19:37:30.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.370+0000 7fb803fff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb80c0ac3f0 0x7fb80c0aedf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59256/0 (socket says 172.21.15.17:59256) 2022-08-01T19:37:30.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.370+0000 7fb803fff700 1 -- 172.21.15.17:0/1524617575 learned_addr learned my addr 172.21.15.17:0/1524617575 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:37:30.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.370+0000 7fb80b59e700 1 --2- 172.21.15.17:0/1524617575 >> v2:172.21.15.77:3300/0 conn(0x7fb80c0b0850 0x7fb80c0ab130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:30.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.370+0000 7fb80ad9d700 1 --2- 172.21.15.17:0/1524617575 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb80c0a9af0 0x7fb80c0b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:30.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.370+0000 7fb80a59c700 1 -- 172.21.15.17:0/1524617575 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2756890800 0 0) 0x7fb7f4001110 con 0x7fb80c0ac3f0 2022-08-01T19:37:30.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.370+0000 7fb80a59c700 1 -- 172.21.15.17:0/1524617575 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7ec003260 con 0x7fb80c0ac3f0 2022-08-01T19:37:30.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.370+0000 7fb80b59e700 1 -- 172.21.15.17:0/1524617575 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb80c0a9af0 msgr2=0x7fb80c0b1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:30.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.370+0000 7fb80b59e700 1 --2- 172.21.15.17:0/1524617575 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb80c0a9af0 0x7fb80c0b1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:30.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.370+0000 7fb80b59e700 1 -- 172.21.15.17:0/1524617575 >> v1:172.21.15.17:6789/0 conn(0x7fb80c0ac3f0 legacy=0x7fb80c0aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:30.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.370+0000 7fb80b59e700 1 -- 172.21.15.17:0/1524617575 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80c0b0250 con 0x7fb80c0b0850 2022-08-01T19:37:30.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.370+0000 7fb80b59e700 1 --2- 172.21.15.17:0/1524617575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb80c0b0850 0x7fb80c0ab130 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7fb8040032f0 tx=0x7fb8040091d0).ready entity=mon.1 client_cookie=f370b94c0d6b3fd0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:30.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.370+0000 7fb80a59c700 1 -- 172.21.15.17:0/1524617575 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb80400b600 con 0x7fb80c0b0850 2022-08-01T19:37:30.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.370+0000 7fb80a59c700 1 -- 172.21.15.17:0/1524617575 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb80400b760 con 0x7fb80c0b0850 2022-08-01T19:37:30.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.370+0000 7fb80a59c700 1 -- 172.21.15.17:0/1524617575 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb80400b8d0 con 0x7fb80c0b0850 2022-08-01T19:37:30.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.370+0000 7fb8110fe700 1 -- 172.21.15.17:0/1524617575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb80c0b0850 msgr2=0x7fb80c0ab130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:30.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.370+0000 7fb8110fe700 1 --2- 172.21.15.17:0/1524617575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb80c0b0850 0x7fb80c0ab130 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7fb8040032f0 tx=0x7fb8040091d0).stop 2022-08-01T19:37:30.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.370+0000 7fb8110fe700 1 -- 172.21.15.17:0/1524617575 shutdown_connections 2022-08-01T19:37:30.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.370+0000 7fb8110fe700 1 --2- 172.21.15.17:0/1524617575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb80c0b0850 0x7fb80c0ab130 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:30.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.370+0000 7fb8110fe700 1 --2- 172.21.15.17:0/1524617575 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb80c0a9af0 0x7fb80c0b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:30.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.370+0000 7fb8110fe700 1 -- 172.21.15.17:0/1524617575 >> 172.21.15.17:0/1524617575 conn(0x7fb80c05f840 msgr2=0x7fb80c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:30.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.370+0000 7fb8110fe700 1 -- 172.21.15.17:0/1524617575 shutdown_connections 2022-08-01T19:37:30.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.370+0000 7fb8110fe700 1 -- 172.21.15.17:0/1524617575 wait complete. 2022-08-01T19:37:30.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.370+0000 7fb8110fe700 1 Processor -- start 2022-08-01T19:37:30.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.370+0000 7fb8110fe700 1 -- start start 2022-08-01T19:37:30.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.374+0000 7fb8110fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb80c0a9af0 0x7fb80c0b5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:30.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.374+0000 7fb8110fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb80c0ac3f0 0x7fb80c150ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:30.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.374+0000 7fb8110fe700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb80c0b0850 0x7fb80c150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:30.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.374+0000 7fb8110fe700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c064b80 con 0x7fb80c0ac3f0 2022-08-01T19:37:30.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.374+0000 7fb8110fe700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c1549d0 con 0x7fb80c0a9af0 2022-08-01T19:37:30.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.374+0000 7fb8110fe700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c064910 con 0x7fb80c0b0850 2022-08-01T19:37:30.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.374+0000 7fb803fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb80c0ac3f0 0x7fb80c150ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:30.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.374+0000 7fb803fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb80c0ac3f0 0x7fb80c150ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38836/0 (socket says 172.21.15.17:38836) 2022-08-01T19:37:30.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.374+0000 7fb80ad9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb80c0a9af0 0x7fb80c0b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:30.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.374+0000 7fb803fff700 1 -- 172.21.15.17:0/963046140 learned_addr learned my addr 172.21.15.17:0/963046140 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:30.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.374+0000 7fb80b59e700 1 --2- 172.21.15.17:0/963046140 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb80c0b0850 0x7fb80c150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:30.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.374+0000 7fb803fff700 1 -- 172.21.15.17:0/963046140 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb80c0b0850 msgr2=0x7fb80c150fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:30.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.374+0000 7fb803fff700 1 --2- 172.21.15.17:0/963046140 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb80c0b0850 0x7fb80c150fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:30.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.374+0000 7fb803fff700 1 -- 172.21.15.17:0/963046140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb80c0a9af0 msgr2=0x7fb80c0b5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:30.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.374+0000 7fb803fff700 1 --2- 172.21.15.17:0/963046140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb80c0a9af0 0x7fb80c0b5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:30.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.374+0000 7fb803fff700 1 -- 172.21.15.17:0/963046140 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb804009470 con 0x7fb80c0ac3f0 2022-08-01T19:37:30.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.374+0000 7fb803fff700 1 --2- 172.21.15.17:0/963046140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb80c0ac3f0 0x7fb80c150ad0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7fb7f40012a0 tx=0x7fb7f4007900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:30.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.374+0000 7fb808d99700 1 -- 172.21.15.17:0/963046140 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7f40070d0 con 0x7fb80c0ac3f0 2022-08-01T19:37:30.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.374+0000 7fb808d99700 1 -- 172.21.15.17:0/963046140 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f4007230 con 0x7fb80c0ac3f0 2022-08-01T19:37:30.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.374+0000 7fb8110fe700 1 -- 172.21.15.17:0/963046140 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb80c154bc0 con 0x7fb80c0ac3f0 2022-08-01T19:37:30.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.374+0000 7fb808d99700 1 -- 172.21.15.17:0/963046140 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7f4005610 con 0x7fb80c0ac3f0 2022-08-01T19:37:30.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.374+0000 7fb8110fe700 1 -- 172.21.15.17:0/963046140 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb80c0b5810 con 0x7fb80c0ac3f0 2022-08-01T19:37:30.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.378+0000 7fb8110fe700 1 -- 172.21.15.17:0/963046140 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb80c0574e0 con 0x7fb80c0ac3f0 2022-08-01T19:37:30.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.378+0000 7fb808d99700 1 -- 172.21.15.17:0/963046140 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb7f401a070 con 0x7fb80c0ac3f0 2022-08-01T19:37:30.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.378+0000 7fb808d99700 1 --2- 172.21.15.17:0/963046140 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb7f0053950 0x7fb7f0055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:30.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.378+0000 7fb808d99700 1 -- 172.21.15.17:0/963046140 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fb7f406ad30 con 0x7fb80c0ac3f0 2022-08-01T19:37:30.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.378+0000 7fb80ad9d700 1 --2- 172.21.15.17:0/963046140 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb7f0053950 0x7fb7f0055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:30.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.378+0000 7fb80ad9d700 1 --2- 172.21.15.17:0/963046140 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb7f0053950 0x7fb7f0055e10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7fb7fc00b000 tx=0x7fb7fc006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:30.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.382+0000 7fb808d99700 1 -- 172.21.15.17:0/963046140 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb7f4041cc0 con 0x7fb80c0ac3f0 2022-08-01T19:37:30.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.510+0000 7fb8110fe700 1 -- 172.21.15.17:0/963046140 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb80c0b1df0 con 0x7fb80c0ac3f0 2022-08-01T19:37:30.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.518+0000 7fb808d99700 1 -- 172.21.15.17:0/963046140 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v6)=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb80c0b1df0 con 0x7fb80c0ac3f0 2022-08-01T19:37:30.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.518+0000 7fb8110fe700 1 -- 172.21.15.17:0/963046140 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb7f0053950 msgr2=0x7fb7f0055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:30.526 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.518+0000 7fb8110fe700 1 --2- 172.21.15.17:0/963046140 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb7f0053950 0x7fb7f0055e10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7fb7fc00b000 tx=0x7fb7fc006df0).stop 2022-08-01T19:37:30.526 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.518+0000 7fb8110fe700 1 -- 172.21.15.17:0/963046140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb80c0ac3f0 msgr2=0x7fb80c150ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:30.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.518+0000 7fb8110fe700 1 --2- 172.21.15.17:0/963046140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb80c0ac3f0 0x7fb80c150ad0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7fb7f40012a0 tx=0x7fb7f4007900).stop 2022-08-01T19:37:30.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.522+0000 7fb8110fe700 1 -- 172.21.15.17:0/963046140 shutdown_connections 2022-08-01T19:37:30.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.522+0000 7fb8110fe700 1 --2- 172.21.15.17:0/963046140 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb7f0053950 0x7fb7f0055e10 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:30.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.522+0000 7fb8110fe700 1 --2- 172.21.15.17:0/963046140 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb80c0b0850 0x7fb80c150fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:30.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.522+0000 7fb8110fe700 1 --2- 172.21.15.17:0/963046140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb80c0ac3f0 0x7fb80c150ad0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:30.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.522+0000 7fb8110fe700 1 --2- 172.21.15.17:0/963046140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb80c0a9af0 0x7fb80c0b5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:30.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.522+0000 7fb8110fe700 1 -- 172.21.15.17:0/963046140 >> 172.21.15.17:0/963046140 conn(0x7fb80c05f840 msgr2=0x7fb80c141c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:30.529 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.522+0000 7fb8110fe700 1 -- 172.21.15.17:0/963046140 shutdown_connections 2022-08-01T19:37:30.529 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.522+0000 7fb8110fe700 1 -- 172.21.15.17:0/963046140 wait complete. 2022-08-01T19:37:30.537 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T19:37:30.537 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:37:30.556 DEBUG:teuthology.orchestra.run.smithi077:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:37:30.576 DEBUG:teuthology.orchestra.run.smithi085:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:37:30.598 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:37:30.620 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T19:37:30.621 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:37:30.621 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:37:30.622 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:37:30.646 INFO:teuthology.orchestra.run.smithi085.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-08-01T19:37:30.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:37:30.647 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-08-01T19:37:30.648 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:37:30.648 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:37:30.649 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:37:30.657 INFO:teuthology.orchestra.run.smithi190.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T19:37:30.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:37:30.660 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-08-01T19:37:30.661 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T19:37:30.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.770+0000 7f4bf580d700 1 Processor -- start 2022-08-01T19:37:30.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.770+0000 7f4bf580d700 1 -- start start 2022-08-01T19:37:30.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.770+0000 7f4bf580d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4bf00af910 0x7f4bf0143f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:30.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.770+0000 7f4bf580d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4bf00aa0d0 0x7f4bf01402a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:30.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.770+0000 7f4bf580d700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf00b3a10 con 0x7f4bf00b3630 2022-08-01T19:37:30.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.770+0000 7f4bf580d700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf01443f0 con 0x7f4bf00aa0d0 2022-08-01T19:37:30.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.770+0000 7f4bf580d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf0146b40 con 0x7f4bf00af910 2022-08-01T19:37:30.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.770+0000 7f4bef7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4bf00aa0d0 0x7f4bf01402a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:30.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.770+0000 7f4bef7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4bf00aa0d0 0x7f4bf01402a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38846/0 (socket says 172.21.15.17:38846) 2022-08-01T19:37:30.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.770+0000 7f4bef7fe700 1 -- 172.21.15.17:0/3568949052 learned_addr learned my addr 172.21.15.17:0/3568949052 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:30.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.770+0000 7f4be67fc700 1 --2- 172.21.15.17:0/3568949052 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4bf00af910 0x7f4bf0143f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:30.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.770+0000 7f4bef7fe700 1 -- 172.21.15.17:0/3568949052 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4bf00af910 msgr2=0x7f4bf0143f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:30.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.770+0000 7f4bef7fe700 1 --2- 172.21.15.17:0/3568949052 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4bf00af910 0x7f4bf0143f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:30.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.770+0000 7f4bef7fe700 1 -- 172.21.15.17:0/3568949052 >> v1:172.21.15.77:6789/0 conn(0x7f4bf00b3630 legacy=0x7f4bf00af200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:30.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.770+0000 7f4bef7fe700 1 -- 172.21.15.17:0/3568949052 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf0146de0 con 0x7f4bf00aa0d0 2022-08-01T19:37:30.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.770+0000 7f4be67fc700 1 --2- 172.21.15.17:0/3568949052 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4bf00af910 0x7f4bf0143f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:37:30.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.774+0000 7f4bef7fe700 1 --2- 172.21.15.17:0/3568949052 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4bf00aa0d0 0x7f4bf01402a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f4be80032f0 tx=0x7f4be8009290).ready entity=mon.0 client_cookie=facf669921780847 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:30.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.774+0000 7f4bee7fc700 1 -- 172.21.15.17:0/3568949052 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4be800b8a0 con 0x7f4bf00aa0d0 2022-08-01T19:37:30.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.774+0000 7f4bee7fc700 1 -- 172.21.15.17:0/3568949052 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be800ba00 con 0x7f4bf00aa0d0 2022-08-01T19:37:30.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.774+0000 7f4bee7fc700 1 -- 172.21.15.17:0/3568949052 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4be800bb70 con 0x7f4bf00aa0d0 2022-08-01T19:37:30.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.774+0000 7f4bf580d700 1 -- 172.21.15.17:0/3568949052 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4bf00aa0d0 msgr2=0x7f4bf01402a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:30.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.774+0000 7f4bf580d700 1 --2- 172.21.15.17:0/3568949052 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4bf00aa0d0 0x7f4bf01402a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f4be80032f0 tx=0x7f4be8009290).stop 2022-08-01T19:37:30.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.774+0000 7f4bf580d700 1 -- 172.21.15.17:0/3568949052 shutdown_connections 2022-08-01T19:37:30.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.774+0000 7f4bf580d700 1 --2- 172.21.15.17:0/3568949052 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4bf00aa0d0 0x7f4bf01402a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:30.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.774+0000 7f4bf580d700 1 --2- 172.21.15.17:0/3568949052 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4bf00af910 0x7f4bf0143f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:30.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.774+0000 7f4bf580d700 1 -- 172.21.15.17:0/3568949052 >> 172.21.15.17:0/3568949052 conn(0x7f4bf0060950 msgr2=0x7f4bf0060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:30.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.774+0000 7f4bf580d700 1 -- 172.21.15.17:0/3568949052 shutdown_connections 2022-08-01T19:37:30.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.774+0000 7f4bf580d700 1 -- 172.21.15.17:0/3568949052 wait complete. 2022-08-01T19:37:30.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.774+0000 7f4bf580d700 1 Processor -- start 2022-08-01T19:37:30.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.774+0000 7f4bf580d700 1 -- start start 2022-08-01T19:37:30.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.774+0000 7f4bf580d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4bf00aa0d0 0x7f4bf014a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:30.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.778+0000 7f4bf580d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4bf00af910 0x7f4bf00b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:30.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.778+0000 7f4bf580d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bf00b3630 0x7f4bf0155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:30.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.778+0000 7f4bf580d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf00b30d0 con 0x7f4bf00af910 2022-08-01T19:37:30.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.778+0000 7f4bf580d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf01427c0 con 0x7f4bf00b3630 2022-08-01T19:37:30.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.778+0000 7f4bf580d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf0142a60 con 0x7f4bf00aa0d0 2022-08-01T19:37:30.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.778+0000 7f4be67fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4bf00af910 0x7f4bf00b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:30.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.778+0000 7f4be67fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4bf00af910 0x7f4bf00b28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38850/0 (socket says 172.21.15.17:38850) 2022-08-01T19:37:30.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.778+0000 7f4be67fc700 1 -- 172.21.15.17:0/1226530774 learned_addr learned my addr 172.21.15.17:0/1226530774 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:30.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.778+0000 7f4bef7fe700 1 --2- 172.21.15.17:0/1226530774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bf00b3630 0x7f4bf0155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:30.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.778+0000 7f4beeffd700 1 --2- 172.21.15.17:0/1226530774 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4bf00aa0d0 0x7f4bf014a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:30.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.778+0000 7f4be67fc700 1 -- 172.21.15.17:0/1226530774 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4bf00aa0d0 msgr2=0x7f4bf014a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:30.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.778+0000 7f4be67fc700 1 --2- 172.21.15.17:0/1226530774 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4bf00aa0d0 0x7f4bf014a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:30.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.778+0000 7f4be67fc700 1 -- 172.21.15.17:0/1226530774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bf00b3630 msgr2=0x7f4bf0155da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:30.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.778+0000 7f4be67fc700 1 --2- 172.21.15.17:0/1226530774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bf00b3630 0x7f4bf0155da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:30.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.778+0000 7f4be67fc700 1 -- 172.21.15.17:0/1226530774 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be8009530 con 0x7f4bf00af910 2022-08-01T19:37:30.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.778+0000 7f4be67fc700 1 --2- 172.21.15.17:0/1226530774 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4bf00af910 0x7f4bf00b28a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f4bd800a000 tx=0x7f4bd8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:30.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.778+0000 7f4becff9700 1 -- 172.21.15.17:0/1226530774 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4bd8011780 con 0x7f4bf00af910 2022-08-01T19:37:30.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.778+0000 7f4becff9700 1 -- 172.21.15.17:0/1226530774 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd80118e0 con 0x7f4bf00af910 2022-08-01T19:37:30.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.778+0000 7f4bf580d700 1 -- 172.21.15.17:0/1226530774 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf0142d00 con 0x7f4bf00af910 2022-08-01T19:37:30.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.778+0000 7f4becff9700 1 -- 172.21.15.17:0/1226530774 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4bd8011c40 con 0x7f4bf00af910 2022-08-01T19:37:30.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.778+0000 7f4bf580d700 1 -- 172.21.15.17:0/1226530774 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf00b2d90 con 0x7f4bf00af910 2022-08-01T19:37:30.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.778+0000 7f4bf580d700 1 -- 172.21.15.17:0/1226530774 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf0142e90 con 0x7f4bf00af910 2022-08-01T19:37:30.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.778+0000 7f4becff9700 1 -- 172.21.15.17:0/1226530774 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4bd801c460 con 0x7f4bf00af910 2022-08-01T19:37:30.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.782+0000 7f4becff9700 1 --2- 172.21.15.17:0/1226530774 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4bd4053a70 0x7f4bd4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:30.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.782+0000 7f4becff9700 1 -- 172.21.15.17:0/1226530774 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f4bd806bf80 con 0x7f4bf00af910 2022-08-01T19:37:30.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.782+0000 7f4beeffd700 1 --2- 172.21.15.17:0/1226530774 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4bd4053a70 0x7f4bd4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:30.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.782+0000 7f4beeffd700 1 --2- 172.21.15.17:0/1226530774 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4bd4053a70 0x7f4bd4055f30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f4be000d000 tx=0x7f4be0007b50).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:30.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.786+0000 7f4becff9700 1 -- 172.21.15.17:0/1226530774 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4bd8042fc0 con 0x7f4bf00af910 2022-08-01T19:37:30.918 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.914+0000 7f4bf580d700 1 -- 172.21.15.17:0/1226530774 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f4bf0147080 con 0x7f4bf00af910 2022-08-01T19:37:30.919 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.914+0000 7f4becff9700 1 -- 172.21.15.17:0/1226530774 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v48) v1 ==== 81+0+29219 (secure 0 0 0) 0x7f4bd80465e0 con 0x7f4bf00af910 2022-08-01T19:37:30.920 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:37:30.921 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:37:30.921 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 48, 2022-08-01T19:37:30.921 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", 2022-08-01T19:37:30.921 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2022-08-01T19:33:50.105743+0000", 2022-08-01T19:37:30.921 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2022-08-01T19:37:28.230594+0000", 2022-08-01T19:37:30.922 INFO:teuthology.orchestra.run.smithi017.stdout: "last_up_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:37:30.922 INFO:teuthology.orchestra.run.smithi017.stdout: "last_in_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:37:30.922 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T19:37:30.922 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_num": 5799936, 2022-08-01T19:37:30.923 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_set": [ 2022-08-01T19:37:30.923 INFO:teuthology.orchestra.run.smithi017.stdout: "pglog_hardlimit", 2022-08-01T19:37:30.923 INFO:teuthology.orchestra.run.smithi017.stdout: "purged_snapdirs", 2022-08-01T19:37:30.923 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_deletes", 2022-08-01T19:37:30.924 INFO:teuthology.orchestra.run.smithi017.stdout: "sortbitwise" 2022-08-01T19:37:30.924 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:37:30.924 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_version": 6, 2022-08-01T19:37:30.924 INFO:teuthology.orchestra.run.smithi017.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T19:37:30.925 INFO:teuthology.orchestra.run.smithi017.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T19:37:30.925 INFO:teuthology.orchestra.run.smithi017.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T19:37:30.925 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_snapshot": "", 2022-08-01T19:37:30.925 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_max": 8, 2022-08-01T19:37:30.925 INFO:teuthology.orchestra.run.smithi017.stdout: "max_osd": 8, 2022-08-01T19:37:30.926 INFO:teuthology.orchestra.run.smithi017.stdout: "require_min_compat_client": "jewel", 2022-08-01T19:37:30.926 INFO:teuthology.orchestra.run.smithi017.stdout: "min_compat_client": "jewel", 2022-08-01T19:37:30.926 INFO:teuthology.orchestra.run.smithi017.stdout: "require_osd_release": "octopus", 2022-08-01T19:37:30.926 INFO:teuthology.orchestra.run.smithi017.stdout: "pools": [ 2022-08-01T19:37:30.927 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:30.927 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 1, 2022-08-01T19:37:30.927 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "device_health_metrics", 2022-08-01T19:37:30.927 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:33:53.001841+0000", 2022-08-01T19:37:30.927 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:37:30.928 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:37:30.928 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:37:30.928 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:37:30.928 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:37:30.929 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:37:30.929 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:37:30.929 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:37:30.929 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 1, 2022-08-01T19:37:30.930 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 1, 2022-08-01T19:37:30.930 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 1, 2022-08-01T19:37:30.930 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 1, 2022-08-01T19:37:30.930 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 1, 2022-08-01T19:37:30.931 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:37:30.931 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:37:30.931 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:37:30.932 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:37:30.932 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:37:30.932 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:37:30.933 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:37:30.933 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:30.933 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "16", 2022-08-01T19:37:30.933 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:37:30.934 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:37:30.934 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:37:30.934 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:37:30.934 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:37:30.935 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:37:30.935 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:37:30.935 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:37:30.935 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:37:30.935 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:37:30.936 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:37:30.936 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:37:30.936 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:37:30.936 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:37:30.937 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:37:30.937 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:37:30.937 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:37:30.937 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:37:30.937 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:37:30.938 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:37:30.938 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:37:30.938 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:37:30.938 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:37:30.939 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:37:30.939 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:37:30.939 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:37:30.939 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:30.939 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:37:30.940 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:37:30.940 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:37:30.940 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:37:30.940 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:37:30.940 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:37:30.941 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:37:30.941 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:37:30.941 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:37:30.941 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:37:30.942 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:37:30.942 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:37:30.942 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 1 2022-08-01T19:37:30.942 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:30.942 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:37:30.943 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_devicehealth": {} 2022-08-01T19:37:30.943 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:30.943 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:30.943 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:30.944 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 2, 2022-08-01T19:37:30.944 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "rbd", 2022-08-01T19:37:30.944 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:34:01.553098+0000", 2022-08-01T19:37:30.944 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:37:30.944 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:37:30.945 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:37:30.945 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:37:30.945 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:37:30.945 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:37:30.946 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:37:30.946 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:37:30.946 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:37:30.946 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:37:30.946 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:37:30.947 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:37:30.947 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:37:30.947 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:37:30.947 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:37:30.948 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:37:30.948 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:37:30.948 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:37:30.948 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:37:30.948 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:37:30.949 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:30.949 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "19", 2022-08-01T19:37:30.949 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:37:30.949 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:37:30.949 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:37:30.950 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:37:30.950 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:37:30.950 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:37:30.950 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:37:30.951 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:37:30.951 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:37:30.951 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:37:30.951 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:37:30.951 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:37:30.952 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:37:30.952 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:37:30.952 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:37:30.952 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:37:30.953 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:37:30.953 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:37:30.953 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:37:30.953 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:37:30.953 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:37:30.954 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:37:30.954 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:37:30.954 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:37:30.954 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:37:30.954 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:37:30.955 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:30.955 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:37:30.955 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:37:30.955 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:37:30.956 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:37:30.956 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:37:30.956 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:37:30.956 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:37:30.956 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:37:30.957 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:37:30.957 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:37:30.957 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:37:30.957 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:37:30.957 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:37:30.958 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd": {} 2022-08-01T19:37:30.958 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:30.958 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:30.958 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:37:30.959 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2022-08-01T19:37:30.959 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:30.959 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:37:30.959 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a962f14f-cf16-4fa8-84ff-c85325db9132", 2022-08-01T19:37:30.960 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:37:30.960 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:37:30.960 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:37:30.960 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:37:30.960 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:37:30.961 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:37:30.961 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:37:30.961 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 43, 2022-08-01T19:37:30.962 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:37:30.962 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:37:30.962 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:37:30.962 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:30.963 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:30.963 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:30.963 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6800", 2022-08-01T19:37:30.963 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:37:30.964 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:30.964 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:30.964 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:30.964 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6801", 2022-08-01T19:37:30.965 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:37:30.965 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:30.965 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:30.965 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:30.966 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:37:30.966 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:30.966 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:30.966 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:30.967 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6802", 2022-08-01T19:37:30.967 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:37:30.967 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:30.967 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:30.968 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:30.968 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6803", 2022-08-01T19:37:30.968 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:37:30.968 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:30.968 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:30.969 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:30.969 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:37:30.969 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:30.969 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:30.969 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:30.970 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6806", 2022-08-01T19:37:30.970 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:37:30.970 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:30.970 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:30.971 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:30.971 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6807", 2022-08-01T19:37:30.971 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:37:30.971 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:30.971 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:30.972 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:30.972 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:37:30.972 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:30.972 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:30.973 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:30.973 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6804", 2022-08-01T19:37:30.973 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:37:30.973 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:30.973 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:30.974 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:30.974 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6805", 2022-08-01T19:37:30.974 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:37:30.974 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:30.974 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:30.975 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:30.975 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6801/23008", 2022-08-01T19:37:30.975 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6803/23008", 2022-08-01T19:37:30.975 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6807/23008", 2022-08-01T19:37:30.976 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6805/23008", 2022-08-01T19:37:30.976 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:37:30.976 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:37:30.976 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:37:30.976 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:30.977 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:30.977 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:30.977 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:37:30.977 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "805f7051-a527-4e98-8844-b6e406a63248", 2022-08-01T19:37:30.978 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:37:30.978 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:37:30.978 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:37:30.978 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:37:30.978 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:37:30.979 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:37:30.979 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:37:30.979 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 43, 2022-08-01T19:37:30.979 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:37:30.980 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:37:30.980 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:37:30.980 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:30.980 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:30.981 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:30.981 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6808", 2022-08-01T19:37:30.981 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:37:30.981 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:30.982 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:30.982 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:30.982 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6809", 2022-08-01T19:37:30.982 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:37:30.983 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:30.983 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:30.983 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:30.983 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:37:30.984 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:30.984 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:30.984 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:30.984 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6810", 2022-08-01T19:37:30.985 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:37:30.985 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:30.985 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:30.985 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:30.985 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6811", 2022-08-01T19:37:30.986 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:37:30.986 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:30.986 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:30.986 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:30.987 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:37:30.987 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:30.987 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:30.987 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:30.988 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6814", 2022-08-01T19:37:30.988 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:37:30.988 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:30.988 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:30.988 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:30.989 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6815", 2022-08-01T19:37:30.989 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:37:30.989 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:30.989 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:30.990 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:30.990 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:37:30.990 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:30.990 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:30.990 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:30.991 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6812", 2022-08-01T19:37:30.991 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:37:30.991 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:30.991 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:30.992 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:30.992 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6813", 2022-08-01T19:37:30.992 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:37:30.992 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:30.992 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:30.993 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:30.993 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6809/23007", 2022-08-01T19:37:30.993 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6811/23007", 2022-08-01T19:37:30.993 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6815/23007", 2022-08-01T19:37:30.993 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6813/23007", 2022-08-01T19:37:30.994 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:37:30.994 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:37:30.994 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:37:30.994 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:30.995 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:30.995 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:30.995 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:37:30.995 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "89f7f3df-7c10-48e1-8b69-9e4243923a61", 2022-08-01T19:37:30.996 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:37:30.996 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:37:30.996 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:37:30.996 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:37:30.996 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:37:30.997 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:37:30.997 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:37:30.997 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 32, 2022-08-01T19:37:30.997 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:37:30.997 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:37:30.998 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:37:30.998 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:30.998 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:30.998 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:30.999 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6817", 2022-08-01T19:37:30.999 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:37:30.999 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:30.999 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.000 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:31.000 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6819", 2022-08-01T19:37:31.000 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:37:31.000 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:31.000 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:31.001 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.001 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:37:31.001 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:31.001 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.002 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:31.002 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6822", 2022-08-01T19:37:31.002 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:37:31.002 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.002 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.003 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:31.003 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6824", 2022-08-01T19:37:31.003 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:37:31.003 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:31.004 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:31.004 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.004 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:37:31.004 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:31.004 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.005 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:31.005 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6830", 2022-08-01T19:37:31.005 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:37:31.005 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.005 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.006 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:31.006 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6831", 2022-08-01T19:37:31.006 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:37:31.006 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:31.007 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:31.007 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.007 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:37:31.007 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:31.007 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.008 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:31.008 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6826", 2022-08-01T19:37:31.008 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:37:31.008 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.009 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.009 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:31.009 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6829", 2022-08-01T19:37:31.009 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:37:31.009 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:31.010 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:31.010 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.010 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6819/23010", 2022-08-01T19:37:31.010 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6824/23010", 2022-08-01T19:37:31.011 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6831/23010", 2022-08-01T19:37:31.011 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6829/23010", 2022-08-01T19:37:31.011 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:37:31.011 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:37:31.011 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:37:31.012 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:31.012 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.012 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.012 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:37:31.013 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "96123468-aabc-4a64-909e-a192c369090d", 2022-08-01T19:37:31.013 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:37:31.013 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:37:31.013 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:37:31.013 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:37:31.014 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:37:31.014 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:37:31.014 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:37:31.014 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 43, 2022-08-01T19:37:31.015 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:37:31.015 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:37:31.015 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:37:31.015 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:31.016 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.016 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:31.016 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6816", 2022-08-01T19:37:31.016 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:37:31.016 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.017 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.017 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:31.017 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6818", 2022-08-01T19:37:31.017 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:37:31.017 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:31.018 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:31.018 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.018 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:37:31.018 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:31.019 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.019 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:31.019 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6820", 2022-08-01T19:37:31.019 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:37:31.020 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.020 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.020 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:31.020 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6821", 2022-08-01T19:37:31.020 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:37:31.021 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:31.021 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:31.021 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.021 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:37:31.022 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:31.022 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.022 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:31.022 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6827", 2022-08-01T19:37:31.022 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:37:31.023 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.023 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.023 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:31.023 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6828", 2022-08-01T19:37:31.024 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:37:31.024 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:31.024 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:31.024 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.024 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:37:31.025 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:31.025 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.025 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:31.025 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6823", 2022-08-01T19:37:31.026 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:37:31.026 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.026 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.026 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:31.027 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6825", 2022-08-01T19:37:31.027 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:37:31.027 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:31.027 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:31.028 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.028 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6818/23009", 2022-08-01T19:37:31.028 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6821/23009", 2022-08-01T19:37:31.028 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6828/23009", 2022-08-01T19:37:31.028 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6825/23009", 2022-08-01T19:37:31.029 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:37:31.029 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:37:31.029 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:37:31.029 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:31.030 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.030 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.030 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:37:31.030 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a6e26665-e04b-46ec-ab66-e912acbf044c", 2022-08-01T19:37:31.030 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:37:31.031 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:37:31.031 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:37:31.031 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:37:31.031 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:37:31.032 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:37:31.032 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:37:31.032 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 43, 2022-08-01T19:37:31.032 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:37:31.032 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:37:31.033 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:37:31.033 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:31.033 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.033 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:31.033 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6810", 2022-08-01T19:37:31.034 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:37:31.034 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.034 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.034 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:31.035 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6811", 2022-08-01T19:37:31.035 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:37:31.035 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:31.035 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:31.035 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.036 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:37:31.036 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:31.036 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.036 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:31.037 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6812", 2022-08-01T19:37:31.037 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:37:31.037 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.037 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.037 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:31.038 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6813", 2022-08-01T19:37:31.038 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:37:31.038 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:31.038 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:31.039 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.039 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:37:31.039 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:31.039 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.039 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:31.040 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6816", 2022-08-01T19:37:31.040 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:37:31.040 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.040 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.041 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:31.041 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6817", 2022-08-01T19:37:31.041 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:37:31.041 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:31.041 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:31.042 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.042 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:37:31.042 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:31.042 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.042 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:31.043 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6814", 2022-08-01T19:37:31.043 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:37:31.043 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.043 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.044 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:31.044 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6815", 2022-08-01T19:37:31.044 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:37:31.044 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:31.044 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:31.045 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.045 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6811/23584", 2022-08-01T19:37:31.045 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6813/23584", 2022-08-01T19:37:31.045 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6817/23584", 2022-08-01T19:37:31.046 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6815/23584", 2022-08-01T19:37:31.046 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:37:31.046 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:37:31.046 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:37:31.046 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:31.047 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.047 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.047 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:37:31.047 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "502c0034-18ee-44ac-87ff-19b9dddb66a0", 2022-08-01T19:37:31.047 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:37:31.048 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:37:31.048 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:37:31.048 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:37:31.048 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:37:31.049 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:37:31.049 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:37:31.049 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 43, 2022-08-01T19:37:31.049 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:37:31.049 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:37:31.050 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:37:31.050 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:31.050 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.050 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:31.051 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6802", 2022-08-01T19:37:31.051 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:37:31.051 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.051 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.051 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:31.052 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6803", 2022-08-01T19:37:31.052 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:37:31.052 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:31.052 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:31.052 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.053 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:37:31.053 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:31.053 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.053 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:31.054 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6804", 2022-08-01T19:37:31.054 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:37:31.054 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.054 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.054 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:31.055 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6805", 2022-08-01T19:37:31.055 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:37:31.055 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:31.055 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:31.056 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.056 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:37:31.056 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:31.056 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.056 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:31.057 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6808", 2022-08-01T19:37:31.057 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:37:31.057 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.057 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.057 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:31.058 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6809", 2022-08-01T19:37:31.058 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:37:31.058 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:31.058 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:31.059 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.059 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:37:31.059 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:31.059 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.059 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:31.060 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6806", 2022-08-01T19:37:31.060 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:37:31.060 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.060 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.061 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:31.061 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6807", 2022-08-01T19:37:31.061 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:37:31.061 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:31.061 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:31.062 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.062 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6803/23585", 2022-08-01T19:37:31.062 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6805/23585", 2022-08-01T19:37:31.062 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6809/23585", 2022-08-01T19:37:31.063 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6807/23585", 2022-08-01T19:37:31.063 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:37:31.063 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:37:31.063 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:37:31.063 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:31.064 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.064 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.064 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:37:31.064 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "ac4e755a-7fe7-46cb-8407-852d5af9ffc5", 2022-08-01T19:37:31.064 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:37:31.065 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:37:31.065 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:37:31.065 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:37:31.065 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:37:31.065 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:37:31.066 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:37:31.066 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 43, 2022-08-01T19:37:31.066 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:37:31.066 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:37:31.067 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:37:31.067 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:31.067 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.067 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:31.067 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6820", 2022-08-01T19:37:31.068 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:37:31.068 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.068 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.068 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:31.069 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6821", 2022-08-01T19:37:31.069 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:37:31.069 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:31.069 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:31.069 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.070 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:37:31.070 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:31.070 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.070 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:31.071 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6823", 2022-08-01T19:37:31.071 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:37:31.071 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.071 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.071 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:31.072 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6825", 2022-08-01T19:37:31.072 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:37:31.072 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:31.072 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:31.072 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.073 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:37:31.073 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:31.073 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.073 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:31.074 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6831", 2022-08-01T19:37:31.074 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:37:31.074 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.074 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.074 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:31.075 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6833", 2022-08-01T19:37:31.075 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:37:31.075 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:31.075 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:31.075 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.076 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:37:31.076 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:31.076 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.076 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:31.077 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6827", 2022-08-01T19:37:31.077 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:37:31.077 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.077 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.077 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:31.078 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6829", 2022-08-01T19:37:31.078 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:37:31.078 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:31.078 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:31.079 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.079 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6821/23587", 2022-08-01T19:37:31.079 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6825/23587", 2022-08-01T19:37:31.079 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6833/23587", 2022-08-01T19:37:31.079 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6829/23587", 2022-08-01T19:37:31.080 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:37:31.080 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:37:31.080 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:37:31.080 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:31.081 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.081 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.081 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:37:31.081 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "874761e2-6410-40fc-b60c-e58abe5ce489", 2022-08-01T19:37:31.082 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:37:31.082 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:37:31.082 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:37:31.082 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:37:31.082 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:37:31.083 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:37:31.083 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:37:31.083 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 42, 2022-08-01T19:37:31.083 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:37:31.084 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:37:31.084 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:37:31.084 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:31.084 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.084 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:31.085 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6818", 2022-08-01T19:37:31.085 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:37:31.085 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.085 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.085 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:31.086 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6819", 2022-08-01T19:37:31.086 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:37:31.086 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:31.086 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:31.087 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.087 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:37:31.087 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:31.087 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.087 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:31.088 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6822", 2022-08-01T19:37:31.088 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:37:31.088 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.088 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.088 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:31.089 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6824", 2022-08-01T19:37:31.089 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:37:31.089 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:31.089 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:31.090 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.090 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:37:31.090 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:31.090 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.090 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:31.091 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6830", 2022-08-01T19:37:31.091 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:37:31.091 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.091 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.092 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:31.092 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6832", 2022-08-01T19:37:31.092 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:37:31.092 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:31.092 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:31.093 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.093 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:37:31.093 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:37:31.093 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.093 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:37:31.094 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6826", 2022-08-01T19:37:31.094 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:37:31.094 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.094 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.095 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:37:31.095 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6828", 2022-08-01T19:37:31.095 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:37:31.095 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:31.095 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:31.096 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.096 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6819/23586", 2022-08-01T19:37:31.096 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6824/23586", 2022-08-01T19:37:31.096 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6832/23586", 2022-08-01T19:37:31.096 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6828/23586", 2022-08-01T19:37:31.097 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:37:31.097 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:37:31.097 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:37:31.097 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:31.098 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:31.098 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:37:31.098 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_xinfo": [ 2022-08-01T19:37:31.098 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.099 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:37:31.099 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:37:31.099 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:37:31.099 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:37:31.099 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:37:31.100 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:37:31.100 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.588817+0000", 2022-08-01T19:37:31.100 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:37:31.100 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.101 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.101 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:37:31.101 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:37:31.101 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:37:31.101 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:37:31.102 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:37:31.102 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:37:31.102 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.782753+0000", 2022-08-01T19:37:31.102 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:37:31.103 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.103 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.103 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:37:31.103 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:37:31.103 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:37:31.104 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:37:31.104 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:37:31.104 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:37:31.104 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.837924+0000", 2022-08-01T19:37:31.104 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:37:31.105 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.105 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.105 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:37:31.105 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:37:31.106 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:37:31.106 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:37:31.106 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:37:31.106 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:37:31.106 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.839948+0000", 2022-08-01T19:37:31.107 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:37:31.107 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.107 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.107 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:37:31.107 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:37:31.108 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:37:31.108 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:37:31.108 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:37:31.108 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:37:31.109 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.112801+0000", 2022-08-01T19:37:31.109 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:37:31.109 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.109 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.109 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:37:31.110 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:37:31.110 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:37:31.110 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:37:31.110 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:37:31.111 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:37:31.111 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.106704+0000", 2022-08-01T19:37:31.111 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:37:31.111 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.111 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.112 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:37:31.112 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:37:31.112 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:37:31.112 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:37:31.112 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:37:31.113 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:37:31.113 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.893170+0000", 2022-08-01T19:37:31.113 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:37:31.113 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.114 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:31.114 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:37:31.114 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:37:31.114 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:37:31.114 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:37:31.115 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:37:31.115 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:37:31.115 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.886947+0000", 2022-08-01T19:37:31.115 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:37:31.116 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:31.116 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:37:31.116 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap": [], 2022-08-01T19:37:31.116 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap_items": [], 2022-08-01T19:37:31.117 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_temp": [], 2022-08-01T19:37:31.117 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_temp": [], 2022-08-01T19:37:31.117 INFO:teuthology.orchestra.run.smithi017.stdout: "blacklist": { 2022-08-01T19:37:31.117 INFO:teuthology.orchestra.run.smithi017.stdout: "172.21.15.17:6835/3980246529": "2022-08-02T19:37:24.214184+0000", 2022-08-01T19:37:31.117 INFO:teuthology.orchestra.run.smithi017.stdout: "172.21.15.17:6834/3980246529": "2022-08-02T19:37:24.214184+0000" 2022-08-01T19:37:31.118 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.118 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profiles": { 2022-08-01T19:37:31.118 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_data": { 2022-08-01T19:37:31.119 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-device-class": "", 2022-08-01T19:37:31.119 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:37:31.119 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-root": "default", 2022-08-01T19:37:31.119 INFO:teuthology.orchestra.run.smithi017.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T19:37:31.119 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:37:31.120 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "2", 2022-08-01T19:37:31.120 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:37:31.120 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:37:31.120 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van", 2022-08-01T19:37:31.121 INFO:teuthology.orchestra.run.smithi017.stdout: "w": "8" 2022-08-01T19:37:31.121 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.121 INFO:teuthology.orchestra.run.smithi017.stdout: "default": { 2022-08-01T19:37:31.121 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:37:31.121 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:37:31.122 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "1", 2022-08-01T19:37:31.122 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:37:31.122 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:37:31.122 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van" 2022-08-01T19:37:31.123 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:31.123 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:31.123 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps_queue": [], 2022-08-01T19:37:31.123 INFO:teuthology.orchestra.run.smithi017.stdout: "new_removed_snaps": [], 2022-08-01T19:37:31.123 INFO:teuthology.orchestra.run.smithi017.stdout: "new_purged_snaps": [], 2022-08-01T19:37:31.124 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_node_flags": {}, 2022-08-01T19:37:31.124 INFO:teuthology.orchestra.run.smithi017.stdout: "device_class_flags": {} 2022-08-01T19:37:31.124 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:37:31.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.918+0000 7f4bf580d700 1 -- 172.21.15.17:0/1226530774 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4bd4053a70 msgr2=0x7f4bd4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:31.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.918+0000 7f4bf580d700 1 --2- 172.21.15.17:0/1226530774 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4bd4053a70 0x7f4bd4055f30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f4be000d000 tx=0x7f4be0007b50).stop 2022-08-01T19:37:31.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.918+0000 7f4bf580d700 1 -- 172.21.15.17:0/1226530774 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4bf00af910 msgr2=0x7f4bf00b28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:31.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.918+0000 7f4bf580d700 1 --2- 172.21.15.17:0/1226530774 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4bf00af910 0x7f4bf00b28a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f4bd800a000 tx=0x7f4bd8008040).stop 2022-08-01T19:37:31.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.918+0000 7f4bf580d700 1 -- 172.21.15.17:0/1226530774 shutdown_connections 2022-08-01T19:37:31.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.918+0000 7f4bf580d700 1 --2- 172.21.15.17:0/1226530774 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4bd4053a70 0x7f4bd4055f30 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:31.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.918+0000 7f4bf580d700 1 --2- 172.21.15.17:0/1226530774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bf00b3630 0x7f4bf0155da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.918+0000 7f4bf580d700 1 --2- 172.21.15.17:0/1226530774 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4bf00af910 0x7f4bf00b28a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.918+0000 7f4bf580d700 1 --2- 172.21.15.17:0/1226530774 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4bf00aa0d0 0x7f4bf014a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.918+0000 7f4bf580d700 1 -- 172.21.15.17:0/1226530774 >> 172.21.15.17:0/1226530774 conn(0x7f4bf0060950 msgr2=0x7f4bf0143750 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.918+0000 7f4bf580d700 1 -- 172.21.15.17:0/1226530774 shutdown_connections 2022-08-01T19:37:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:30.918+0000 7f4bf580d700 1 -- 172.21.15.17:0/1226530774 wait complete. 2022-08-01T19:37:31.129 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:37:31.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.238+0000 7fd820811700 1 Processor -- start 2022-08-01T19:37:31.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.238+0000 7fd820811700 1 -- start start 2022-08-01T19:37:31.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd820811700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd81c0ad750 0x7fd81c061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:31.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd820811700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fd81c1405d0 0x7fd81c140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:31.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd820811700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd81c0aee80 con 0x7fd81c0614b0 2022-08-01T19:37:31.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd820811700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd81c0b70b0 con 0x7fd81c1405d0 2022-08-01T19:37:31.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd820811700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c0af0b0 con 0x7fd81c0ad750 2022-08-01T19:37:31.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd81b7fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd81c0614b0 0x7fd81c0b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59282/0 (socket says 172.21.15.17:59282) 2022-08-01T19:37:31.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd81b7fe700 1 -- 172.21.15.17:0/1587358065 learned_addr learned my addr 172.21.15.17:0/1587358065 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:37:31.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd81bfff700 1 --2- 172.21.15.17:0/1587358065 >> v2:172.21.15.77:3300/0 conn(0x7fd81c1405d0 0x7fd81c140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:31.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd81affd700 1 --2- 172.21.15.17:0/1587358065 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd81c0ad750 0x7fd81c061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:31.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd81a7fc700 1 -- 172.21.15.17:0/1587358065 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 436885964 0 0) 0x7fd81c0aee80 con 0x7fd81c0614b0 2022-08-01T19:37:31.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd81a7fc700 1 -- 172.21.15.17:0/1587358065 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7fc003260 con 0x7fd81c0614b0 2022-08-01T19:37:31.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd81bfff700 1 -- 172.21.15.17:0/1587358065 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd81c0ad750 msgr2=0x7fd81c061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:31.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd81bfff700 1 --2- 172.21.15.17:0/1587358065 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd81c0ad750 0x7fd81c061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:31.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd81bfff700 1 -- 172.21.15.17:0/1587358065 >> v1:172.21.15.17:6789/0 conn(0x7fd81c0614b0 legacy=0x7fd81c0b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:31.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd81bfff700 1 -- 172.21.15.17:0/1587358065 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd81c0af350 con 0x7fd81c1405d0 2022-08-01T19:37:31.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd81bfff700 1 --2- 172.21.15.17:0/1587358065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd81c1405d0 0x7fd81c140a80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fd814002f70 tx=0x7fd814006df0).ready entity=mon.1 client_cookie=2dd9b510a8967a99 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:31.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd81a7fc700 1 -- 172.21.15.17:0/1587358065 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd81400d4a0 con 0x7fd81c1405d0 2022-08-01T19:37:31.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd81a7fc700 1 -- 172.21.15.17:0/1587358065 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd814000e90 con 0x7fd81c1405d0 2022-08-01T19:37:31.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd81a7fc700 1 -- 172.21.15.17:0/1587358065 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd81400d7f0 con 0x7fd81c1405d0 2022-08-01T19:37:31.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd820811700 1 -- 172.21.15.17:0/1587358065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd81c1405d0 msgr2=0x7fd81c140a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:31.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd820811700 1 --2- 172.21.15.17:0/1587358065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd81c1405d0 0x7fd81c140a80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fd814002f70 tx=0x7fd814006df0).stop 2022-08-01T19:37:31.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd820811700 1 -- 172.21.15.17:0/1587358065 shutdown_connections 2022-08-01T19:37:31.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd820811700 1 --2- 172.21.15.17:0/1587358065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd81c1405d0 0x7fd81c140a80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:31.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd820811700 1 --2- 172.21.15.17:0/1587358065 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd81c0ad750 0x7fd81c061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:31.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd820811700 1 -- 172.21.15.17:0/1587358065 >> 172.21.15.17:0/1587358065 conn(0x7fd81c066f70 msgr2=0x7fd81c067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:31.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd820811700 1 -- 172.21.15.17:0/1587358065 shutdown_connections 2022-08-01T19:37:31.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd820811700 1 -- 172.21.15.17:0/1587358065 wait complete. 2022-08-01T19:37:31.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd820811700 1 Processor -- start 2022-08-01T19:37:31.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd820811700 1 -- start start 2022-08-01T19:37:31.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd820811700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd81c0614b0 0x7fd81c13f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:31.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd820811700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd81c0ad750 0x7fd81c144f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:31.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd820811700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd81c1405d0 0x7fd81c145430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:31.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd820811700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c0ad580 con 0x7fd81c1405d0 2022-08-01T19:37:31.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd820811700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c13c6d0 con 0x7fd81c0614b0 2022-08-01T19:37:31.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd820811700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c142440 con 0x7fd81c0ad750 2022-08-01T19:37:31.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd81bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd81c1405d0 0x7fd81c145430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:31.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd81affd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd81c0ad750 0x7fd81c144f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:31.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd81b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd81c0614b0 0x7fd81c13f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:31.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd81bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd81c1405d0 0x7fd81c145430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38866/0 (socket says 172.21.15.17:38866) 2022-08-01T19:37:31.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.246+0000 7fd81bfff700 1 -- 172.21.15.17:0/3981375084 learned_addr learned my addr 172.21.15.17:0/3981375084 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:31.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.250+0000 7fd81bfff700 1 -- 172.21.15.17:0/3981375084 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd81c0ad750 msgr2=0x7fd81c144f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:31.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.250+0000 7fd81bfff700 1 --2- 172.21.15.17:0/3981375084 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd81c0ad750 0x7fd81c144f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:31.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.250+0000 7fd81bfff700 1 -- 172.21.15.17:0/3981375084 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd81c0614b0 msgr2=0x7fd81c13f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:31.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.250+0000 7fd81bfff700 1 --2- 172.21.15.17:0/3981375084 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd81c0614b0 0x7fd81c13f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:31.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.250+0000 7fd81bfff700 1 -- 172.21.15.17:0/3981375084 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd81400b180 con 0x7fd81c1405d0 2022-08-01T19:37:31.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.250+0000 7fd81affd700 1 --2- 172.21.15.17:0/3981375084 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd81c0ad750 0x7fd81c144f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:37:31.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.250+0000 7fd81bfff700 1 --2- 172.21.15.17:0/3981375084 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd81c1405d0 0x7fd81c145430 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7fd81c13daf0 tx=0x7fd814014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:31.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.250+0000 7fd818ff9700 1 -- 172.21.15.17:0/3981375084 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd814016070 con 0x7fd81c1405d0 2022-08-01T19:37:31.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.250+0000 7fd818ff9700 1 -- 172.21.15.17:0/3981375084 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8140137d0 con 0x7fd81c1405d0 2022-08-01T19:37:31.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.250+0000 7fd820811700 1 -- 172.21.15.17:0/3981375084 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd81c145d50 con 0x7fd81c1405d0 2022-08-01T19:37:31.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.250+0000 7fd818ff9700 1 -- 172.21.15.17:0/3981375084 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd814013b30 con 0x7fd81c1405d0 2022-08-01T19:37:31.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.250+0000 7fd820811700 1 -- 172.21.15.17:0/3981375084 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd81c140210 con 0x7fd81c1405d0 2022-08-01T19:37:31.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.250+0000 7fd820811700 1 -- 172.21.15.17:0/3981375084 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd81c0af5f0 con 0x7fd81c1405d0 2022-08-01T19:37:31.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.250+0000 7fd818ff9700 1 -- 172.21.15.17:0/3981375084 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd814025030 con 0x7fd81c1405d0 2022-08-01T19:37:31.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.250+0000 7fd818ff9700 1 --2- 172.21.15.17:0/3981375084 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd800053a10 0x7fd800055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:31.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.250+0000 7fd818ff9700 1 -- 172.21.15.17:0/3981375084 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fd81406cb30 con 0x7fd81c1405d0 2022-08-01T19:37:31.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.250+0000 7fd81b7fe700 1 --2- 172.21.15.17:0/3981375084 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd800053a10 0x7fd800055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:31.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.250+0000 7fd81b7fe700 1 --2- 172.21.15.17:0/3981375084 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd800053a10 0x7fd800055ed0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fd80c00d000 tx=0x7fd80c0079c0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:31.266 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.258+0000 7fd818ff9700 1 -- 172.21.15.17:0/3981375084 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd814043bd0 con 0x7fd81c1405d0 2022-08-01T19:37:31.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.414+0000 7fd820811700 1 -- 172.21.15.17:0/3981375084 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd81c0574e0 con 0x7fd81c1405d0 2022-08-01T19:37:31.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.414+0000 7fd818ff9700 1 -- 172.21.15.17:0/3981375084 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+1559 (secure 0 0 0) 0x7fd814023090 con 0x7fd81c1405d0 2022-08-01T19:37:31.420 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:37:31.420 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":21,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/109603607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":109603607},{"type":"v1","addr":"172.21.15.17:6833","nonce":109603607}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":16},{"gid":5230,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/2722431349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2722431349},{"type":"v1","addr":"172.21.15.77:6835","nonce":2722431349}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":17},{"gid":5496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.17:6835/1803382751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1803382751},{"type":"v1","addr":"172.21.15.17:6835","nonce":1803382751}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[]} 2022-08-01T19:37:31.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.418+0000 7fd820811700 1 -- 172.21.15.17:0/3981375084 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd800053a10 msgr2=0x7fd800055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:31.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.418+0000 7fd820811700 1 --2- 172.21.15.17:0/3981375084 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd800053a10 0x7fd800055ed0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fd80c00d000 tx=0x7fd80c0079c0).stop 2022-08-01T19:37:31.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.418+0000 7fd820811700 1 -- 172.21.15.17:0/3981375084 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd81c1405d0 msgr2=0x7fd81c145430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:31.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.418+0000 7fd820811700 1 --2- 172.21.15.17:0/3981375084 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd81c1405d0 0x7fd81c145430 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7fd81c13daf0 tx=0x7fd814014000).stop 2022-08-01T19:37:31.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.418+0000 7fd820811700 1 -- 172.21.15.17:0/3981375084 shutdown_connections 2022-08-01T19:37:31.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.418+0000 7fd820811700 1 --2- 172.21.15.17:0/3981375084 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd800053a10 0x7fd800055ed0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:31.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.418+0000 7fd820811700 1 --2- 172.21.15.17:0/3981375084 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd81c1405d0 0x7fd81c145430 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:31.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.418+0000 7fd820811700 1 --2- 172.21.15.17:0/3981375084 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd81c0ad750 0x7fd81c144f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:31.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.418+0000 7fd820811700 1 --2- 172.21.15.17:0/3981375084 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd81c0614b0 0x7fd81c13f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:31.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.418+0000 7fd820811700 1 -- 172.21.15.17:0/3981375084 >> 172.21.15.17:0/3981375084 conn(0x7fd81c066f70 msgr2=0x7fd81c0b7940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:31.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.418+0000 7fd820811700 1 -- 172.21.15.17:0/3981375084 shutdown_connections 2022-08-01T19:37:31.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.418+0000 7fd820811700 1 -- 172.21.15.17:0/3981375084 wait complete. 2022-08-01T19:37:31.426 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 21 2022-08-01T19:37:31.433 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:37:31.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02d22c6700 1 Processor -- start 2022-08-01T19:37:31.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02d22c6700 1 -- start start 2022-08-01T19:37:31.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02d22c6700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f02cc0b28b0 0x7f02cc13e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:31.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02d22c6700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02cc0aac20 con 0x7f02cc141920 2022-08-01T19:37:31.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02d22c6700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02cc0b2c90 con 0x7f02cc0b1180 2022-08-01T19:37:31.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02d22c6700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc141db0 con 0x7f02cc0b28b0 2022-08-01T19:37:31.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02d1ac5700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f02cc141920 0x7f02cc1443d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59300/0 (socket says 172.21.15.17:59300) 2022-08-01T19:37:31.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02d1ac5700 1 -- 172.21.15.17:0/1845585372 learned_addr learned my addr 172.21.15.17:0/1845585372 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:37:31.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02d12c4700 1 --2- 172.21.15.17:0/1845585372 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f02cc0b28b0 0x7f02cc13e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:31.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02c3fff700 1 -- 172.21.15.17:0/1845585372 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 857905469 0 0) 0x7f02c8001110 con 0x7f02cc141920 2022-08-01T19:37:31.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02c3fff700 1 -- 172.21.15.17:0/1845585372 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02b0003260 con 0x7f02cc141920 2022-08-01T19:37:31.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02c3fff700 1 -- 172.21.15.17:0/1845585372 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1703823612 0 0) 0x7f02b0003260 con 0x7f02cc141920 2022-08-01T19:37:31.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02d12c4700 1 -- 172.21.15.17:0/1845585372 >> v1:172.21.15.77:6789/0 conn(0x7f02cc0b1180 legacy=0x7f02cc141210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:31.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02d12c4700 1 -- 172.21.15.17:0/1845585372 >> v1:172.21.15.17:6789/0 conn(0x7f02cc141920 legacy=0x7f02cc1443d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:31.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02d12c4700 1 -- 172.21.15.17:0/1845585372 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02cc0aa440 con 0x7f02cc0b28b0 2022-08-01T19:37:31.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02d12c4700 1 --2- 172.21.15.17:0/1845585372 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f02cc0b28b0 0x7f02cc13e1c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f02c8001990 tx=0x7f02c40091d0).ready entity=mon.2 client_cookie=856c73029035daa0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:31.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02c3fff700 1 -- 172.21.15.17:0/1845585372 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02c80017f0 con 0x7f02cc0b28b0 2022-08-01T19:37:31.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02c3fff700 1 -- 172.21.15.17:0/1845585372 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c400b580 con 0x7f02cc0b28b0 2022-08-01T19:37:31.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02c3fff700 1 -- 172.21.15.17:0/1845585372 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02c400b6f0 con 0x7f02cc0b28b0 2022-08-01T19:37:31.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02d22c6700 1 -- 172.21.15.17:0/1845585372 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f02cc0b28b0 msgr2=0x7f02cc13e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:31.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02d22c6700 1 --2- 172.21.15.17:0/1845585372 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f02cc0b28b0 0x7f02cc13e1c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f02c8001990 tx=0x7f02c40091d0).stop 2022-08-01T19:37:31.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02d22c6700 1 -- 172.21.15.17:0/1845585372 shutdown_connections 2022-08-01T19:37:31.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02d22c6700 1 --2- 172.21.15.17:0/1845585372 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f02cc0b28b0 0x7f02cc13e1c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:31.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02d22c6700 1 -- 172.21.15.17:0/1845585372 >> 172.21.15.17:0/1845585372 conn(0x7f02cc060310 msgr2=0x7f02cc0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:31.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02d22c6700 1 -- 172.21.15.17:0/1845585372 shutdown_connections 2022-08-01T19:37:31.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02d22c6700 1 -- 172.21.15.17:0/1845585372 wait complete. 2022-08-01T19:37:31.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02d22c6700 1 Processor -- start 2022-08-01T19:37:31.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02d22c6700 1 -- start start 2022-08-01T19:37:31.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02d22c6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02cc0b1180 0x7f02cc0b4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:31.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02d22c6700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f02cc0b28b0 0x7f02cc152ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:31.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02d22c6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02cc141920 0x7f02cc144c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:31.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02d22c6700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc144010 con 0x7f02cc0b1180 2022-08-01T19:37:31.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02d22c6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc1442b0 con 0x7f02cc141920 2022-08-01T19:37:31.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02d22c6700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc144550 con 0x7f02cc0b28b0 2022-08-01T19:37:31.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02d12c4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02cc0b1180 0x7f02cc0b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:31.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02d12c4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02cc0b1180 0x7f02cc0b4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38876/0 (socket says 172.21.15.17:38876) 2022-08-01T19:37:31.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02d12c4700 1 -- 172.21.15.17:0/2297862794 learned_addr learned my addr 172.21.15.17:0/2297862794 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:31.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.538+0000 7f02d1ac5700 1 --2- 172.21.15.17:0/2297862794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02cc141920 0x7f02cc144c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:31.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.534+0000 7f02d0ac3700 1 --2- 172.21.15.17:0/2297862794 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f02cc0b28b0 0x7f02cc152ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:31.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.538+0000 7f02d12c4700 1 -- 172.21.15.17:0/2297862794 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f02cc0b28b0 msgr2=0x7f02cc152ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:31.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.538+0000 7f02d12c4700 1 --2- 172.21.15.17:0/2297862794 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f02cc0b28b0 0x7f02cc152ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:31.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.538+0000 7f02d12c4700 1 -- 172.21.15.17:0/2297862794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02cc141920 msgr2=0x7f02cc144c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:31.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.538+0000 7f02d12c4700 1 --2- 172.21.15.17:0/2297862794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02cc141920 0x7f02cc144c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:31.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.538+0000 7f02d12c4700 1 -- 172.21.15.17:0/2297862794 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c4009470 con 0x7f02cc0b1180 2022-08-01T19:37:31.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.538+0000 7f02d0ac3700 1 --2- 172.21.15.17:0/2297862794 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f02cc0b28b0 0x7f02cc152ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:37:31.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.538+0000 7f02d12c4700 1 --2- 172.21.15.17:0/2297862794 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02cc0b1180 0x7f02cc0b4230 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f02c8001990 tx=0x7f02c40091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:31.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.538+0000 7f02c27fc700 1 -- 172.21.15.17:0/2297862794 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02c400b6f0 con 0x7f02cc0b1180 2022-08-01T19:37:31.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.538+0000 7f02c27fc700 1 -- 172.21.15.17:0/2297862794 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c40032d0 con 0x7f02cc0b1180 2022-08-01T19:37:31.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.538+0000 7f02d22c6700 1 -- 172.21.15.17:0/2297862794 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02cc1447f0 con 0x7f02cc0b1180 2022-08-01T19:37:31.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.538+0000 7f02c27fc700 1 -- 172.21.15.17:0/2297862794 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02c400be90 con 0x7f02cc0b1180 2022-08-01T19:37:31.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.538+0000 7f02d22c6700 1 -- 172.21.15.17:0/2297862794 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02cc0b4e40 con 0x7f02cc0b1180 2022-08-01T19:37:31.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.538+0000 7f02d22c6700 1 -- 172.21.15.17:0/2297862794 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02cc144980 con 0x7f02cc0b1180 2022-08-01T19:37:31.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.538+0000 7f02c27fc700 1 -- 172.21.15.17:0/2297862794 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f02c400f070 con 0x7f02cc0b1180 2022-08-01T19:37:31.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.538+0000 7f02c27fc700 1 --2- 172.21.15.17:0/2297862794 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f02b40539e0 0x7f02b4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:31.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.538+0000 7f02c27fc700 1 -- 172.21.15.17:0/2297862794 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f02c406d9f0 con 0x7f02cc0b1180 2022-08-01T19:37:31.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.542+0000 7f02d0ac3700 1 --2- 172.21.15.17:0/2297862794 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f02b40539e0 0x7f02b4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:31.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.542+0000 7f02d0ac3700 1 --2- 172.21.15.17:0/2297862794 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f02b40539e0 0x7f02b4055ea0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f02bc00b000 tx=0x7f02bc009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:31.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.546+0000 7f02c27fc700 1 -- 172.21.15.17:0/2297862794 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f02c4044980 con 0x7f02cc0b1180 2022-08-01T19:37:31.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.714+0000 7f02d22c6700 1 -- 172.21.15.17:0/2297862794 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02cc0574e0 con 0x7f02cc0b1180 2022-08-01T19:37:31.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.718+0000 7f02c27fc700 1 -- 172.21.15.17:0/2297862794 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+1559 (secure 0 0 0) 0x7f02c4047fa0 con 0x7f02cc0b1180 2022-08-01T19:37:31.721 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:37:31.721 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":21,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/109603607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":109603607},{"type":"v1","addr":"172.21.15.17:6833","nonce":109603607}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":16},{"gid":5230,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/2722431349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2722431349},{"type":"v1","addr":"172.21.15.77:6835","nonce":2722431349}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":17},{"gid":5496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.17:6835/1803382751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1803382751},{"type":"v1","addr":"172.21.15.17:6835","nonce":1803382751}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":21}],"filesystems":[]} 2022-08-01T19:37:31.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.718+0000 7f02d22c6700 1 -- 172.21.15.17:0/2297862794 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f02b40539e0 msgr2=0x7f02b4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:31.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.718+0000 7f02d22c6700 1 --2- 172.21.15.17:0/2297862794 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f02b40539e0 0x7f02b4055ea0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f02bc00b000 tx=0x7f02bc009040).stop 2022-08-01T19:37:31.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.718+0000 7f02d22c6700 1 -- 172.21.15.17:0/2297862794 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02cc0b1180 msgr2=0x7f02cc0b4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:31.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.718+0000 7f02d22c6700 1 --2- 172.21.15.17:0/2297862794 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02cc0b1180 0x7f02cc0b4230 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f02c8001990 tx=0x7f02c40091d0).stop 2022-08-01T19:37:31.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.718+0000 7f02d22c6700 1 -- 172.21.15.17:0/2297862794 shutdown_connections 2022-08-01T19:37:31.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.718+0000 7f02d22c6700 1 --2- 172.21.15.17:0/2297862794 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f02b40539e0 0x7f02b4055ea0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:31.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.718+0000 7f02d22c6700 1 --2- 172.21.15.17:0/2297862794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02cc141920 0x7f02cc144c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:31.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.718+0000 7f02d22c6700 1 --2- 172.21.15.17:0/2297862794 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f02cc0b28b0 0x7f02cc152ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:31.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.718+0000 7f02d22c6700 1 --2- 172.21.15.17:0/2297862794 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02cc0b1180 0x7f02cc0b4230 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:31.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.718+0000 7f02d22c6700 1 -- 172.21.15.17:0/2297862794 >> 172.21.15.17:0/2297862794 conn(0x7f02cc060310 msgr2=0x7f02cc0b02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:31.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.718+0000 7f02d22c6700 1 -- 172.21.15.17:0/2297862794 shutdown_connections 2022-08-01T19:37:31.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.718+0000 7f02d22c6700 1 -- 172.21.15.17:0/2297862794 wait complete. 2022-08-01T19:37:31.726 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 21 2022-08-01T19:37:31.734 INFO:tasks.ceph.mds.a:Restarting daemon 2022-08-01T19:37:31.734 INFO:tasks.ceph.mds.a:Stopping old one... 2022-08-01T19:37:31.735 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-08-01T19:37:31.735 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:37:31.735 INFO:tasks.ceph.mds.c:Restarting daemon 2022-08-01T19:37:31.736 INFO:tasks.ceph.mds.c:Stopping old one... 2022-08-01T19:37:31.736 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-08-01T19:37:31.736 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:37:31.737 INFO:tasks.ceph.mds.b:Restarting daemon 2022-08-01T19:37:31.737 INFO:tasks.ceph.mds.b:Stopping old one... 2022-08-01T19:37:31.737 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-08-01T19:37:31.738 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:37:31.749 INFO:tasks.ceph.mds.a:Stopped 2022-08-01T19:37:31.750 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-08-01T19:37:31.752 INFO:tasks.ceph.mds.c:Stopped 2022-08-01T19:37:31.752 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-08-01T19:37:31.753 INFO:tasks.ceph.mds.a:Started 2022-08-01T19:37:31.754 DEBUG:teuthology.parallel:result is None 2022-08-01T19:37:31.755 INFO:tasks.ceph.mds.c:Started 2022-08-01T19:37:31.755 DEBUG:teuthology.parallel:result is None 2022-08-01T19:37:31.756 INFO:tasks.ceph.mds.b:Stopped 2022-08-01T19:37:31.756 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-08-01T19:37:31.758 INFO:tasks.ceph.mds.b:Started 2022-08-01T19:37:31.759 DEBUG:teuthology.parallel:result is None 2022-08-01T19:37:31.760 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-08-01T19:37:31.813 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2022-08-01T19:37:31.831 INFO:tasks.ceph.mds.b.smithi077.stdout:starting mds.b at 2022-08-01T19:37:31.835 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2022-08-01T19:37:31.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.874+0000 7f6bdd631700 1 Processor -- start 2022-08-01T19:37:31.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.874+0000 7f6bdd631700 1 -- start start 2022-08-01T19:37:31.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.874+0000 7f6bdd631700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6bd813c870 0x7f6bd813ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:31.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.874+0000 7f6bdd631700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bd80b2dc0 con 0x7f6bd80b29e0 2022-08-01T19:37:31.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.874+0000 7f6bdd631700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bd80b17f0 con 0x7f6bd805c620 2022-08-01T19:37:31.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.874+0000 7f6bdd631700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bd80b1340 con 0x7f6bd813c870 2022-08-01T19:37:31.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.874+0000 7f6bd77fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6bd813c870 0x7f6bd813ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:31.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.874+0000 7f6bd7fff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f6bd80b29e0 0x7f6bd80b0c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59334/0 (socket says 172.21.15.17:59334) 2022-08-01T19:37:31.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.874+0000 7f6bd7fff700 1 -- 172.21.15.17:0/2151149250 learned_addr learned my addr 172.21.15.17:0/2151149250 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:37:31.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.874+0000 7f6bd77fe700 1 --2- 172.21.15.17:0/2151149250 >> v2:172.21.15.17:3300/0 conn(0x7f6bd813c870 0x7f6bd813ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38910/0 (socket says 172.21.15.17:38910) 2022-08-01T19:37:31.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.874+0000 7f6bd77fe700 1 -- 172.21.15.17:0/2151149250 >> v1:172.21.15.77:6789/0 conn(0x7f6bd805c620 legacy=0x7f6bd80ad0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:31.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.874+0000 7f6bd77fe700 1 -- 172.21.15.17:0/2151149250 >> v1:172.21.15.17:6789/0 conn(0x7f6bd80b29e0 legacy=0x7f6bd80b0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:31.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.874+0000 7f6bd77fe700 1 -- 172.21.15.17:0/2151149250 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd80b1a90 con 0x7f6bd813c870 2022-08-01T19:37:31.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.874+0000 7f6bd77fe700 1 --2- 172.21.15.17:0/2151149250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6bd813c870 0x7f6bd813ccb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f6bd8063770 tx=0x7f6bc0006df0).ready entity=mon.0 client_cookie=5c34882d88e00ad4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:31.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.874+0000 7f6bd6ffd700 1 -- 172.21.15.17:0/2151149250 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6bc000d5c0 con 0x7f6bd813c870 2022-08-01T19:37:31.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.874+0000 7f6bd6ffd700 1 -- 172.21.15.17:0/2151149250 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc0000e90 con 0x7f6bd813c870 2022-08-01T19:37:31.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.874+0000 7f6bd6ffd700 1 -- 172.21.15.17:0/2151149250 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6bc000d910 con 0x7f6bd813c870 2022-08-01T19:37:31.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.874+0000 7f6bdd631700 1 -- 172.21.15.17:0/2151149250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6bd813c870 msgr2=0x7f6bd813ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:31.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.874+0000 7f6bdd631700 1 --2- 172.21.15.17:0/2151149250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6bd813c870 0x7f6bd813ccb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f6bd8063770 tx=0x7f6bc0006df0).stop 2022-08-01T19:37:31.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.874+0000 7f6bdd631700 1 -- 172.21.15.17:0/2151149250 shutdown_connections 2022-08-01T19:37:31.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.874+0000 7f6bdd631700 1 --2- 172.21.15.17:0/2151149250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6bd813c870 0x7f6bd813ccb0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:31.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.874+0000 7f6bdd631700 1 -- 172.21.15.17:0/2151149250 >> 172.21.15.17:0/2151149250 conn(0x7f6bd80613c0 msgr2=0x7f6bd8062f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:31.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.874+0000 7f6bdd631700 1 -- 172.21.15.17:0/2151149250 shutdown_connections 2022-08-01T19:37:31.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.874+0000 7f6bdd631700 1 -- 172.21.15.17:0/2151149250 wait complete. 2022-08-01T19:37:31.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.874+0000 7f6bdd631700 1 Processor -- start 2022-08-01T19:37:31.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.874+0000 7f6bdd631700 1 -- start start 2022-08-01T19:37:31.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.878+0000 7f6bdd631700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bd805c620 0x7f6bd813be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:31.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.878+0000 7f6bdd631700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6bd80b29e0 0x7f6bd8150730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:31.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.878+0000 7f6bdd631700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6bd813c870 0x7f6bd81543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:31.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.878+0000 7f6bdd631700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd8146340 con 0x7f6bd80b29e0 2022-08-01T19:37:31.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.878+0000 7f6bdd631700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd81465e0 con 0x7f6bd805c620 2022-08-01T19:37:31.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.878+0000 7f6bdd631700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd8153c80 con 0x7f6bd813c870 2022-08-01T19:37:31.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.878+0000 7f6bd77fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6bd80b29e0 0x7f6bd8150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:31.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.878+0000 7f6bd77fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6bd80b29e0 0x7f6bd8150730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38916/0 (socket says 172.21.15.17:38916) 2022-08-01T19:37:31.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.878+0000 7f6bd77fe700 1 -- 172.21.15.17:0/3488895223 learned_addr learned my addr 172.21.15.17:0/3488895223 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:31.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.878+0000 7f6bd7fff700 1 --2- 172.21.15.17:0/3488895223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bd805c620 0x7f6bd813be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:31.901 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.878+0000 7f6bdce30700 1 --2- 172.21.15.17:0/3488895223 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6bd813c870 0x7f6bd81543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:31.901 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.878+0000 7f6bd77fe700 1 -- 172.21.15.17:0/3488895223 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6bd813c870 msgr2=0x7f6bd81543c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:31.902 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.878+0000 7f6bd77fe700 1 --2- 172.21.15.17:0/3488895223 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6bd813c870 0x7f6bd81543c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:31.902 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.878+0000 7f6bd77fe700 1 -- 172.21.15.17:0/3488895223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bd805c620 msgr2=0x7f6bd813be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:31.902 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.878+0000 7f6bd77fe700 1 --2- 172.21.15.17:0/3488895223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bd805c620 0x7f6bd813be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:31.903 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.878+0000 7f6bd77fe700 1 -- 172.21.15.17:0/3488895223 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc000b180 con 0x7f6bd80b29e0 2022-08-01T19:37:31.903 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.878+0000 7f6bdce30700 1 --2- 172.21.15.17:0/3488895223 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6bd813c870 0x7f6bd81543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:37:31.904 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.878+0000 7f6bd77fe700 1 --2- 172.21.15.17:0/3488895223 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6bd80b29e0 0x7f6bd8150730 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7f6bd8063770 tx=0x7f6bc0012000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:31.904 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.878+0000 7f6bd57fa700 1 -- 172.21.15.17:0/3488895223 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6bc0002f50 con 0x7f6bd80b29e0 2022-08-01T19:37:31.904 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.878+0000 7f6bd57fa700 1 -- 172.21.15.17:0/3488895223 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc0013ca0 con 0x7f6bd80b29e0 2022-08-01T19:37:31.905 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.878+0000 7f6bdd631700 1 -- 172.21.15.17:0/3488895223 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bd8153f20 con 0x7f6bd80b29e0 2022-08-01T19:37:31.905 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.878+0000 7f6bd57fa700 1 -- 172.21.15.17:0/3488895223 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6bc001f450 con 0x7f6bd80b29e0 2022-08-01T19:37:31.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.878+0000 7f6bdd631700 1 -- 172.21.15.17:0/3488895223 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bd8145fc0 con 0x7f6bd80b29e0 2022-08-01T19:37:31.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.878+0000 7f6bdd631700 1 -- 172.21.15.17:0/3488895223 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bd81540b0 con 0x7f6bd80b29e0 2022-08-01T19:37:31.907 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.882+0000 7f6bd57fa700 1 -- 172.21.15.17:0/3488895223 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6bc0026070 con 0x7f6bd80b29e0 2022-08-01T19:37:31.907 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.882+0000 7f6bd57fa700 1 --2- 172.21.15.17:0/3488895223 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6bbc0539e0 0x7f6bbc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:31.907 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.882+0000 7f6bd57fa700 1 -- 172.21.15.17:0/3488895223 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f6bc006c970 con 0x7f6bd80b29e0 2022-08-01T19:37:31.907 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.882+0000 7f6bd7fff700 1 --2- 172.21.15.17:0/3488895223 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6bbc0539e0 0x7f6bbc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:31.908 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.882+0000 7f6bd7fff700 1 --2- 172.21.15.17:0/3488895223 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6bbc0539e0 0x7f6bbc055ea0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f6bc800d000 tx=0x7f6bc8009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:31.908 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:31.886+0000 7f6bd57fa700 1 -- 172.21.15.17:0/3488895223 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6bc0043940 con 0x7f6bd80b29e0 2022-08-01T19:37:31.958 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2022-08-01T19:37:31.958 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2022-08-01T19:37:31.959 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2022-08-01T19:37:32.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.022+0000 7f6bdd631700 1 -- 172.21.15.17:0/3488895223 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f6bd80574e0 con 0x7f6bd80b29e0 2022-08-01T19:37:32.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.850+0000 7f6bd57fa700 1 -- 172.21.15.17:0/3488895223 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v51) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6bc0024020 con 0x7f6bd80b29e0 2022-08-01T19:37:32.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.850+0000 7f6bdd631700 1 -- 172.21.15.17:0/3488895223 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6bbc0539e0 msgr2=0x7f6bbc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:32.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.850+0000 7f6bdd631700 1 --2- 172.21.15.17:0/3488895223 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6bbc0539e0 0x7f6bbc055ea0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f6bc800d000 tx=0x7f6bc8009040).stop 2022-08-01T19:37:32.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.850+0000 7f6bdd631700 1 -- 172.21.15.17:0/3488895223 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6bd80b29e0 msgr2=0x7f6bd8150730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:32.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.850+0000 7f6bdd631700 1 --2- 172.21.15.17:0/3488895223 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6bd80b29e0 0x7f6bd8150730 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7f6bd8063770 tx=0x7f6bc0012000).stop 2022-08-01T19:37:32.858 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.850+0000 7f6bdd631700 1 -- 172.21.15.17:0/3488895223 shutdown_connections 2022-08-01T19:37:32.858 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.850+0000 7f6bdd631700 1 --2- 172.21.15.17:0/3488895223 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6bbc0539e0 0x7f6bbc055ea0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:32.858 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.850+0000 7f6bdd631700 1 --2- 172.21.15.17:0/3488895223 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6bd813c870 0x7f6bd81543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:32.858 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.850+0000 7f6bdd631700 1 --2- 172.21.15.17:0/3488895223 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6bd80b29e0 0x7f6bd8150730 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:32.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.850+0000 7f6bdd631700 1 --2- 172.21.15.17:0/3488895223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bd805c620 0x7f6bd813be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:32.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.850+0000 7f6bdd631700 1 -- 172.21.15.17:0/3488895223 >> 172.21.15.17:0/3488895223 conn(0x7f6bd80613c0 msgr2=0x7f6bd8139010 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:32.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.854+0000 7f6bdd631700 1 -- 172.21.15.17:0/3488895223 shutdown_connections 2022-08-01T19:37:32.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.854+0000 7f6bdd631700 1 -- 172.21.15.17:0/3488895223 wait complete. 2022-08-01T19:37:32.860 INFO:teuthology.orchestra.run.smithi017.stderr: removed all blacklist entries 2022-08-01T19:37:32.873 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-08-01T19:37:32.873 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-08-01T19:37:32.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.982+0000 7f5d31657700 1 Processor -- start 2022-08-01T19:37:32.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.982+0000 7f5d31657700 1 -- start start 2022-08-01T19:37:32.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.982+0000 7f5d31657700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5d2c0614b0 0x7f5d2c061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:32.991 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.982+0000 7f5d31657700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d2c0aee80 con 0x7f5d2c1405d0 2022-08-01T19:37:32.991 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.982+0000 7f5d31657700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d2c0b70b0 con 0x7f5d2c0ad750 2022-08-01T19:37:32.991 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.982+0000 7f5d31657700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d2c0af0b0 con 0x7f5d2c0614b0 2022-08-01T19:37:32.991 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.982+0000 7f5d2bfff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5d2c0614b0 0x7f5d2c061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:32.992 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.982+0000 7f5d30e56700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5d2c1405d0 0x7f5d2c1466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59354/0 (socket says 172.21.15.17:59354) 2022-08-01T19:37:32.992 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.982+0000 7f5d30e56700 1 -- 172.21.15.17:0/3040056701 learned_addr learned my addr 172.21.15.17:0/3040056701 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:37:32.992 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.982+0000 7f5d2affd700 1 -- 172.21.15.17:0/3040056701 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3786213196 0 0) 0x7f5d2c0aee80 con 0x7f5d2c1405d0 2022-08-01T19:37:32.992 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.982+0000 7f5d2affd700 1 -- 172.21.15.17:0/3040056701 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d0c003260 con 0x7f5d2c1405d0 2022-08-01T19:37:32.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.982+0000 7f5d2affd700 1 -- 172.21.15.17:0/3040056701 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1404613310 0 0) 0x7f5d2c0b70b0 con 0x7f5d2c0ad750 2022-08-01T19:37:32.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.982+0000 7f5d2affd700 1 -- 172.21.15.17:0/3040056701 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d2c0aee80 con 0x7f5d2c0ad750 2022-08-01T19:37:32.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.982+0000 7f5d2affd700 1 -- 172.21.15.17:0/3040056701 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2698841882 0 0) 0x7f5d0c003260 con 0x7f5d2c1405d0 2022-08-01T19:37:32.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.982+0000 7f5d2affd700 1 -- 172.21.15.17:0/3040056701 >> v2:172.21.15.17:3300/0 conn(0x7f5d2c0614b0 msgr2=0x7f5d2c061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:32.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.982+0000 7f5d2affd700 1 --2- 172.21.15.17:0/3040056701 >> v2:172.21.15.17:3300/0 conn(0x7f5d2c0614b0 0x7f5d2c061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:32.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.982+0000 7f5d2affd700 1 -- 172.21.15.17:0/3040056701 >> v1:172.21.15.77:6789/0 conn(0x7f5d2c0ad750 legacy=0x7f5d2c0b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:32.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.982+0000 7f5d2affd700 1 -- 172.21.15.17:0/3040056701 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2c0af350 con 0x7f5d2c1405d0 2022-08-01T19:37:32.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.982+0000 7f5d2affd700 1 -- 172.21.15.17:0/3040056701 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7f5d240019e0 con 0x7f5d2c1405d0 2022-08-01T19:37:32.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.982+0000 7f5d2affd700 1 -- 172.21.15.17:0/3040056701 >> v1:172.21.15.17:6789/0 conn(0x7f5d2c1405d0 legacy=0x7f5d2c1466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:32.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.982+0000 7f5d2affd700 1 --2- 172.21.15.17:0/3040056701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d0c0038e0 0x7f5d0c003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:32.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.982+0000 7f5d2affd700 1 --2- 172.21.15.17:0/3040056701 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d0c004500 0x7f5d0c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:32.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.982+0000 7f5d2affd700 1 --2- 172.21.15.17:0/3040056701 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d0c004e70 0x7f5d0c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:32.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.982+0000 7f5d2affd700 1 -- 172.21.15.17:0/3040056701 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c0af0b0 con 0x7f5d0c0038e0 2022-08-01T19:37:32.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.982+0000 7f5d2affd700 1 -- 172.21.15.17:0/3040056701 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c00bc40 con 0x7f5d0c004500 2022-08-01T19:37:32.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.982+0000 7f5d2affd700 1 -- 172.21.15.17:0/3040056701 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c00be00 con 0x7f5d0c004e70 2022-08-01T19:37:32.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.982+0000 7f5d2bfff700 1 --2- 172.21.15.17:0/3040056701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d0c0038e0 0x7f5d0c003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:32.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.986+0000 7f5d30e56700 1 --2- 172.21.15.17:0/3040056701 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d0c004e70 0x7f5d0c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:32.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.986+0000 7f5d2b7fe700 1 --2- 172.21.15.17:0/3040056701 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d0c004500 0x7f5d0c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:32.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.986+0000 7f5d2bfff700 1 -- 172.21.15.17:0/3040056701 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d0c004e70 msgr2=0x7f5d0c00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:32.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.986+0000 7f5d2bfff700 1 --2- 172.21.15.17:0/3040056701 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d0c004e70 0x7f5d0c00b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:32.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.986+0000 7f5d2bfff700 1 -- 172.21.15.17:0/3040056701 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d0c004500 msgr2=0x7f5d0c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:32.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.986+0000 7f5d2bfff700 1 --2- 172.21.15.17:0/3040056701 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d0c004500 0x7f5d0c004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:32.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.986+0000 7f5d30e56700 1 -- 172.21.15.17:0/3040056701 reap_dead start 2022-08-01T19:37:32.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.986+0000 7f5d2bfff700 1 -- 172.21.15.17:0/3040056701 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0c003260 con 0x7f5d0c0038e0 2022-08-01T19:37:32.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.986+0000 7f5d2bfff700 1 --2- 172.21.15.17:0/3040056701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d0c0038e0 0x7f5d0c003cc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f5d1400c000 tx=0x7f5d14006df0).ready entity=mon.0 client_cookie=8895780677a25d06 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:32.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.986+0000 7f5d2affd700 1 -- 172.21.15.17:0/3040056701 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d1400e830 con 0x7f5d0c0038e0 2022-08-01T19:37:32.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.986+0000 7f5d2affd700 1 -- 172.21.15.17:0/3040056701 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1400e990 con 0x7f5d0c0038e0 2022-08-01T19:37:33.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.986+0000 7f5d2affd700 1 -- 172.21.15.17:0/3040056701 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d1400ecf0 con 0x7f5d0c0038e0 2022-08-01T19:37:33.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.986+0000 7f5d31657700 1 -- 172.21.15.17:0/3040056701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d0c0038e0 msgr2=0x7f5d0c003cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:33.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.986+0000 7f5d31657700 1 --2- 172.21.15.17:0/3040056701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d0c0038e0 0x7f5d0c003cc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f5d1400c000 tx=0x7f5d14006df0).stop 2022-08-01T19:37:33.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.986+0000 7f5d31657700 1 -- 172.21.15.17:0/3040056701 shutdown_connections 2022-08-01T19:37:33.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.986+0000 7f5d31657700 1 --2- 172.21.15.17:0/3040056701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d0c0038e0 0x7f5d0c003cc0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:33.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.986+0000 7f5d31657700 1 -- 172.21.15.17:0/3040056701 >> 172.21.15.17:0/3040056701 conn(0x7f5d2c066f70 msgr2=0x7f5d2c067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:33.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.986+0000 7f5d31657700 1 -- 172.21.15.17:0/3040056701 shutdown_connections 2022-08-01T19:37:33.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.986+0000 7f5d31657700 1 -- 172.21.15.17:0/3040056701 wait complete. 2022-08-01T19:37:33.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.986+0000 7f5d31657700 1 Processor -- start 2022-08-01T19:37:33.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.986+0000 7f5d31657700 1 -- start start 2022-08-01T19:37:33.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.990+0000 7f5d31657700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d0c0038e0 0x7f5d2c153440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:33.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.990+0000 7f5d31657700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d2c141000 0x7f5d2c09ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:33.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.990+0000 7f5d31657700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d2c09f400 0x7f5d2c155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:33.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.990+0000 7f5d31657700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c0acdf0 con 0x7f5d2c141000 2022-08-01T19:37:33.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.990+0000 7f5d31657700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c0b35b0 con 0x7f5d0c0038e0 2022-08-01T19:37:33.007 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.990+0000 7f5d31657700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c0b28e0 con 0x7f5d2c09f400 2022-08-01T19:37:33.007 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.990+0000 7f5d2b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d2c141000 0x7f5d2c09ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:33.007 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.990+0000 7f5d2b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d2c141000 0x7f5d2c09ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38938/0 (socket says 172.21.15.17:38938) 2022-08-01T19:37:33.007 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.990+0000 7f5d2b7fe700 1 -- 172.21.15.17:0/1634315666 learned_addr learned my addr 172.21.15.17:0/1634315666 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:33.008 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.990+0000 7f5d30e56700 1 --2- 172.21.15.17:0/1634315666 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d2c09f400 0x7f5d2c155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:33.008 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.990+0000 7f5d2bfff700 1 --2- 172.21.15.17:0/1634315666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d0c0038e0 0x7f5d2c153440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:33.008 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.990+0000 7f5d2b7fe700 1 -- 172.21.15.17:0/1634315666 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d2c09f400 msgr2=0x7f5d2c155fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:33.008 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.990+0000 7f5d2b7fe700 1 --2- 172.21.15.17:0/1634315666 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d2c09f400 0x7f5d2c155fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:33.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.990+0000 7f5d2b7fe700 1 -- 172.21.15.17:0/1634315666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d0c0038e0 msgr2=0x7f5d2c153440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:33.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.990+0000 7f5d2b7fe700 1 --2- 172.21.15.17:0/1634315666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d0c0038e0 0x7f5d2c153440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:33.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.990+0000 7f5d2b7fe700 1 -- 172.21.15.17:0/1634315666 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2c0b39e0 con 0x7f5d2c141000 2022-08-01T19:37:33.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.990+0000 7f5d2b7fe700 1 --2- 172.21.15.17:0/1634315666 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d2c141000 0x7f5d2c09ef10 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f5d2c0b38c0 tx=0x7f5d1c009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:33.010 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.990+0000 7f5d297fa700 1 -- 172.21.15.17:0/1634315666 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d1c014430 con 0x7f5d2c141000 2022-08-01T19:37:33.010 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.990+0000 7f5d297fa700 1 -- 172.21.15.17:0/1634315666 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1c014590 con 0x7f5d2c141000 2022-08-01T19:37:33.010 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.990+0000 7f5d31657700 1 -- 172.21.15.17:0/1634315666 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d2c0ad020 con 0x7f5d2c141000 2022-08-01T19:37:33.010 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.990+0000 7f5d297fa700 1 -- 172.21.15.17:0/1634315666 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d1c0148f0 con 0x7f5d2c141000 2022-08-01T19:37:33.010 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.990+0000 7f5d31657700 1 -- 172.21.15.17:0/1634315666 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d2c153930 con 0x7f5d2c141000 2022-08-01T19:37:33.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.994+0000 7f5d31657700 1 -- 172.21.15.17:0/1634315666 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d2c0af5f0 con 0x7f5d2c141000 2022-08-01T19:37:33.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.994+0000 7f5d297fa700 1 -- 172.21.15.17:0/1634315666 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5d1c014a50 con 0x7f5d2c141000 2022-08-01T19:37:33.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.994+0000 7f5d297fa700 1 --2- 172.21.15.17:0/1634315666 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5d10053a70 0x7f5d10055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:33.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.994+0000 7f5d297fa700 1 -- 172.21.15.17:0/1634315666 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f5d1c06bb70 con 0x7f5d2c141000 2022-08-01T19:37:33.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.994+0000 7f5d2bfff700 1 --2- 172.21.15.17:0/1634315666 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5d10053a70 0x7f5d10055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:33.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.994+0000 7f5d2bfff700 1 --2- 172.21.15.17:0/1634315666 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5d10053a70 0x7f5d10055f30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f5d1401b000 tx=0x7f5d14019000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:33.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:32.998+0000 7f5d297fa700 1 -- 172.21.15.17:0/1634315666 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5d1c042c70 con 0x7f5d2c141000 2022-08-01T19:37:33.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.170+0000 7f5d31657700 1 -- 172.21.15.17:0/1634315666 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5d2c0574e0 con 0x7f5d2c141000 2022-08-01T19:37:33.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.170+0000 7f5d297fa700 1 -- 172.21.15.17:0/1634315666 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f5d1c0230b0 con 0x7f5d2c141000 2022-08-01T19:37:33.176 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:37:33.176 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:37:33.177 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_dump": [ 2022-08-01T19:37:33.177 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:33.178 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.a", 2022-08-01T19:37:33.178 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhijotxORAAVc04rsrlFj0NShsyc6Qbhg==", 2022-08-01T19:37:33.179 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:37:33.179 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:37:33.180 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:37:33.180 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:37:33.181 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:37:33.181 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:33.181 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:33.182 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:33.182 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.b", 2022-08-01T19:37:33.183 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhiud2xBxAAcn6/O5ysdIp/oIbF5Y1dAw==", 2022-08-01T19:37:33.183 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:37:33.184 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:37:33.184 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:37:33.184 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:37:33.185 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:37:33.185 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:33.185 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:33.185 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:33.186 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.c", 2022-08-01T19:37:33.186 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+KuhiMzMbBBAAfdT5mtpYwboNL+7f2B09KQ==", 2022-08-01T19:37:33.186 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:37:33.186 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:37:33.187 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:37:33.187 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:37:33.187 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:37:33.187 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:33.187 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:33.188 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:33.188 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.0", 2022-08-01T19:37:33.188 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhia27/NRAAZXBp+nYelJVJ6Mge9r+3Vg==", 2022-08-01T19:37:33.188 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:37:33.189 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:37:33.189 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:37:33.189 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:37:33.189 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:33.189 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:33.190 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:33.190 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.1", 2022-08-01T19:37:33.190 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCBKuhiOggRGRAAvgA0MxwBJwmpXV6S9DTdjA==", 2022-08-01T19:37:33.190 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:37:33.191 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:37:33.191 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:37:33.191 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:37:33.191 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:33.191 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:33.192 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:33.192 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.2", 2022-08-01T19:37:33.192 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCDKuhi5PgOORAAhcORK4bW2T3dXPeYjcCm+g==", 2022-08-01T19:37:33.192 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:37:33.193 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:37:33.193 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:37:33.193 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:37:33.193 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:33.193 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:33.194 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:33.194 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.3", 2022-08-01T19:37:33.194 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCGKuhiutn6GxAAk2ucLuGbT5YV/R4xXQGh0w==", 2022-08-01T19:37:33.194 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:37:33.194 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:37:33.195 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:37:33.195 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:37:33.195 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:33.195 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:33.196 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:33.196 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.4", 2022-08-01T19:37:33.196 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCLKuhiGHTRHxAArr3TwXA0gsCAAF/Qs38wUQ==", 2022-08-01T19:37:33.196 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:37:33.196 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:37:33.197 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:37:33.197 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:37:33.197 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:33.197 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:33.197 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:33.198 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.5", 2022-08-01T19:37:33.198 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCOKuhiMMDbBBAAvTzBvOcv7pFWm/P/lNqB0w==", 2022-08-01T19:37:33.198 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:37:33.198 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:37:33.199 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:37:33.199 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:37:33.199 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:33.199 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:33.199 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:33.200 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.6", 2022-08-01T19:37:33.200 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCQKuhiG2H3IxAAufUcWqo9ULquxIrUD5vSfA==", 2022-08-01T19:37:33.200 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:37:33.200 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:37:33.201 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:37:33.201 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:37:33.201 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:33.201 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:33.201 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:33.202 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.7", 2022-08-01T19:37:33.202 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCTKuhiU50LChAAXuhrYBbHkk3eKzAPiEJFTg==", 2022-08-01T19:37:33.202 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:37:33.202 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:37:33.202 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:37:33.203 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:37:33.203 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:33.203 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:33.203 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:33.204 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.0", 2022-08-01T19:37:33.204 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhig3EnCxAANM20jSIQf/AKwuRX1qwT/A==", 2022-08-01T19:37:33.204 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:37:33.204 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:37:33.204 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow r", 2022-08-01T19:37:33.205 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T19:37:33.205 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:33.205 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:33.205 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:33.205 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.1", 2022-08-01T19:37:33.206 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+KuhicJR0DhAAivqEAZUhXnSwjVRwWm2bJg==", 2022-08-01T19:37:33.206 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:37:33.206 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:37:33.206 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow r", 2022-08-01T19:37:33.207 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T19:37:33.207 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:33.207 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:33.207 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:33.207 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.admin", 2022-08-01T19:37:33.208 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhiPTTGHxAAgfthCR6RX6j8F1YseFGNeA==", 2022-08-01T19:37:33.208 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:37:33.208 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:37:33.208 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:37:33.208 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:37:33.209 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:37:33.209 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:33.209 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:33.209 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:33.210 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.x", 2022-08-01T19:37:33.210 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9Kuhi2jvCNhAAyUxiGByZ+OZr8Y7dh3Qt5w==", 2022-08-01T19:37:33.210 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:37:33.210 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:37:33.210 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2022-08-01T19:37:33.211 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:37:33.211 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:33.211 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:33.211 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:33.212 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.y", 2022-08-01T19:37:33.212 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhiDJXpMhAA8aek67ydbRgNO0h+miws+w==", 2022-08-01T19:37:33.212 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:37:33.212 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:37:33.212 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2022-08-01T19:37:33.213 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:37:33.213 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:33.213 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:33.213 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:37:33.213 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:37:33.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.174+0000 7f5d31657700 1 -- 172.21.15.17:0/1634315666 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5d10053a70 msgr2=0x7f5d10055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:33.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.174+0000 7f5d31657700 1 --2- 172.21.15.17:0/1634315666 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5d10053a70 0x7f5d10055f30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f5d1401b000 tx=0x7f5d14019000).stop 2022-08-01T19:37:33.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.174+0000 7f5d31657700 1 -- 172.21.15.17:0/1634315666 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d2c141000 msgr2=0x7f5d2c09ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:33.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.174+0000 7f5d31657700 1 --2- 172.21.15.17:0/1634315666 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d2c141000 0x7f5d2c09ef10 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f5d2c0b38c0 tx=0x7f5d1c009040).stop 2022-08-01T19:37:33.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.174+0000 7f5d31657700 1 -- 172.21.15.17:0/1634315666 shutdown_connections 2022-08-01T19:37:33.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.174+0000 7f5d31657700 1 --2- 172.21.15.17:0/1634315666 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5d10053a70 0x7f5d10055f30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:33.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.174+0000 7f5d31657700 1 --2- 172.21.15.17:0/1634315666 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d2c09f400 0x7f5d2c155fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:33.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.174+0000 7f5d31657700 1 --2- 172.21.15.17:0/1634315666 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d2c141000 0x7f5d2c09ef10 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:33.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.174+0000 7f5d31657700 1 --2- 172.21.15.17:0/1634315666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d0c0038e0 0x7f5d2c153440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:33.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.174+0000 7f5d31657700 1 -- 172.21.15.17:0/1634315666 >> 172.21.15.17:0/1634315666 conn(0x7f5d2c066f70 msgr2=0x7f5d2c067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:33.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.174+0000 7f5d31657700 1 -- 172.21.15.17:0/1634315666 shutdown_connections 2022-08-01T19:37:33.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.174+0000 7f5d31657700 1 -- 172.21.15.17:0/1634315666 wait complete. 2022-08-01T19:37:33.219 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:37:33.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06c62e0700 1 Processor -- start 2022-08-01T19:37:33.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06c62e0700 1 -- start start 2022-08-01T19:37:33.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06c62e0700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f06c00b1160 0x7f06c01448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:33.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06c62e0700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06c00b04e0 con 0x7f06c0148530 2022-08-01T19:37:33.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06c62e0700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06c00b0780 con 0x7f06c00b1f40 2022-08-01T19:37:33.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06c62e0700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06c00a9250 con 0x7f06c00b1160 2022-08-01T19:37:33.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06b7fff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f06c0148530 0x7f06c00a7250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59372/0 (socket says 172.21.15.17:59372) 2022-08-01T19:37:33.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06b7fff700 1 -- 172.21.15.17:0/98158655 learned_addr learned my addr 172.21.15.17:0/98158655 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:37:33.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06c487d700 1 --2- 172.21.15.17:0/98158655 >> v2:172.21.15.77:3300/0 conn(0x7f06c00b1160 0x7f06c01448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:33.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06bf7fe700 1 -- 172.21.15.17:0/98158655 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3127876953 0 0) 0x7f06c00b04e0 con 0x7f06c0148530 2022-08-01T19:37:33.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06bf7fe700 1 -- 172.21.15.17:0/98158655 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06a0003260 con 0x7f06c0148530 2022-08-01T19:37:33.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06bf7fe700 1 -- 172.21.15.17:0/98158655 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 270290760 0 0) 0x7f06a0003260 con 0x7f06c0148530 2022-08-01T19:37:33.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06bf7fe700 1 -- 172.21.15.17:0/98158655 >> v2:172.21.15.77:3300/0 conn(0x7f06c00b1160 msgr2=0x7f06c01448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:33.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06bf7fe700 1 --2- 172.21.15.17:0/98158655 >> v2:172.21.15.77:3300/0 conn(0x7f06c00b1160 0x7f06c01448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:33.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06bf7fe700 1 -- 172.21.15.17:0/98158655 >> v1:172.21.15.77:6789/0 conn(0x7f06c00b1f40 legacy=0x7f06c00b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:33.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06bf7fe700 1 -- 172.21.15.17:0/98158655 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c00b0a20 con 0x7f06c0148530 2022-08-01T19:37:33.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06bf7fe700 1 -- 172.21.15.17:0/98158655 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7f06a8001660 con 0x7f06c0148530 2022-08-01T19:37:33.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06bf7fe700 1 -- 172.21.15.17:0/98158655 >> v1:172.21.15.17:6789/0 conn(0x7f06c0148530 legacy=0x7f06c00a7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:33.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06bf7fe700 1 --2- 172.21.15.17:0/98158655 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f06a0003750 0x7f06a0003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:33.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06bf7fe700 1 --2- 172.21.15.17:0/98158655 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06a0004500 0x7f06a0004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:33.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06bf7fe700 1 --2- 172.21.15.17:0/98158655 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06a0004ec0 0x7f06a000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:33.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06bf7fe700 1 -- 172.21.15.17:0/98158655 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a000bd70 con 0x7f06a0004ec0 2022-08-01T19:37:33.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06bf7fe700 1 -- 172.21.15.17:0/98158655 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a000bee0 con 0x7f06a0004500 2022-08-01T19:37:33.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06bf7fe700 1 -- 172.21.15.17:0/98158655 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a000c0a0 con 0x7f06a0003750 2022-08-01T19:37:33.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06b7fff700 1 --2- 172.21.15.17:0/98158655 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06a0004500 0x7f06a0004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:33.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06c487d700 1 --2- 172.21.15.17:0/98158655 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06a0004ec0 0x7f06a000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:33.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06bffff700 1 --2- 172.21.15.17:0/98158655 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f06a0003750 0x7f06a0003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:33.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06b7fff700 1 -- 172.21.15.17:0/98158655 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f06a0003750 msgr2=0x7f06a0003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:33.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06b7fff700 1 --2- 172.21.15.17:0/98158655 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f06a0003750 0x7f06a0003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:33.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06b7fff700 1 -- 172.21.15.17:0/98158655 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06a0004ec0 msgr2=0x7f06a000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:33.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06b7fff700 1 --2- 172.21.15.17:0/98158655 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06a0004ec0 0x7f06a000b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:33.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06b7fff700 1 -- 172.21.15.17:0/98158655 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a0003260 con 0x7f06a0004500 2022-08-01T19:37:33.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06c487d700 1 -- 172.21.15.17:0/98158655 reap_dead start 2022-08-01T19:37:33.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06b7fff700 1 --2- 172.21.15.17:0/98158655 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06a0004500 0x7f06a0004980 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f06a8001370 tx=0x7f06c00b0910).ready entity=mon.1 client_cookie=6433c78ff6180ea7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:33.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06bf7fe700 1 -- 172.21.15.17:0/98158655 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06a80070d0 con 0x7f06a0004500 2022-08-01T19:37:33.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06bf7fe700 1 -- 172.21.15.17:0/98158655 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a8007230 con 0x7f06a0004500 2022-08-01T19:37:33.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06bf7fe700 1 -- 172.21.15.17:0/98158655 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06a800c8c0 con 0x7f06a0004500 2022-08-01T19:37:33.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06c62e0700 1 -- 172.21.15.17:0/98158655 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06a0004500 msgr2=0x7f06a0004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:33.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.322+0000 7f06c62e0700 1 --2- 172.21.15.17:0/98158655 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06a0004500 0x7f06a0004980 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f06a8001370 tx=0x7f06c00b0910).stop 2022-08-01T19:37:33.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.326+0000 7f06c62e0700 1 -- 172.21.15.17:0/98158655 shutdown_connections 2022-08-01T19:37:33.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.326+0000 7f06c62e0700 1 --2- 172.21.15.17:0/98158655 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06a0004500 0x7f06a0004980 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:33.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.326+0000 7f06c62e0700 1 -- 172.21.15.17:0/98158655 >> 172.21.15.17:0/98158655 conn(0x7f06c005f840 msgr2=0x7f06c005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:33.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.326+0000 7f06c62e0700 1 -- 172.21.15.17:0/98158655 shutdown_connections 2022-08-01T19:37:33.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.326+0000 7f06c62e0700 1 -- 172.21.15.17:0/98158655 wait complete. 2022-08-01T19:37:33.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.326+0000 7f06c62e0700 1 Processor -- start 2022-08-01T19:37:33.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.326+0000 7f06c62e0700 1 -- start start 2022-08-01T19:37:33.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.326+0000 7f06c62e0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06a0004500 0x7f06c00a9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:33.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.326+0000 7f06c62e0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06c00a9e00 0x7f06c0159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:33.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.326+0000 7f06c62e0700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f06c014a330 0x7f06c015a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:33.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.326+0000 7f06c62e0700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c014b800 con 0x7f06a0004500 2022-08-01T19:37:33.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.326+0000 7f06c62e0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c014aa30 con 0x7f06c00a9e00 2022-08-01T19:37:33.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.326+0000 7f06c62e0700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c014acd0 con 0x7f06c014a330 2022-08-01T19:37:33.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.326+0000 7f06bffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06a0004500 0x7f06c00a9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:33.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.326+0000 7f06bffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06a0004500 0x7f06c00a9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38956/0 (socket says 172.21.15.17:38956) 2022-08-01T19:37:33.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.326+0000 7f06bffff700 1 -- 172.21.15.17:0/1344959276 learned_addr learned my addr 172.21.15.17:0/1344959276 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:33.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.326+0000 7f06b7fff700 1 --2- 172.21.15.17:0/1344959276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06c00a9e00 0x7f06c0159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:33.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.326+0000 7f06c487d700 1 --2- 172.21.15.17:0/1344959276 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f06c014a330 0x7f06c015a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:33.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.326+0000 7f06bffff700 1 -- 172.21.15.17:0/1344959276 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f06c014a330 msgr2=0x7f06c015a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:33.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.326+0000 7f06bffff700 1 --2- 172.21.15.17:0/1344959276 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f06c014a330 0x7f06c015a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:33.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.326+0000 7f06bffff700 1 -- 172.21.15.17:0/1344959276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06c00a9e00 msgr2=0x7f06c0159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:33.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.326+0000 7f06bffff700 1 --2- 172.21.15.17:0/1344959276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06c00a9e00 0x7f06c0159f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:33.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.326+0000 7f06bffff700 1 -- 172.21.15.17:0/1344959276 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c0149fa0 con 0x7f06a0004500 2022-08-01T19:37:33.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.326+0000 7f06bffff700 1 --2- 172.21.15.17:0/1344959276 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06a0004500 0x7f06c00a9910 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f06b0000f50 tx=0x7f06b0007a00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:33.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.330+0000 7f06bdffb700 1 -- 172.21.15.17:0/1344959276 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06b0011590 con 0x7f06a0004500 2022-08-01T19:37:33.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.330+0000 7f06bdffb700 1 -- 172.21.15.17:0/1344959276 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b00116f0 con 0x7f06a0004500 2022-08-01T19:37:33.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.330+0000 7f06c62e0700 1 -- 172.21.15.17:0/1344959276 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06c015a8f0 con 0x7f06a0004500 2022-08-01T19:37:33.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.330+0000 7f06bdffb700 1 -- 172.21.15.17:0/1344959276 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06b0011a50 con 0x7f06a0004500 2022-08-01T19:37:33.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.330+0000 7f06c62e0700 1 -- 172.21.15.17:0/1344959276 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c0148c70 con 0x7f06a0004500 2022-08-01T19:37:33.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.330+0000 7f06c62e0700 1 -- 172.21.15.17:0/1344959276 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06c014b970 con 0x7f06a0004500 2022-08-01T19:37:33.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.330+0000 7f06bdffb700 1 -- 172.21.15.17:0/1344959276 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f06b001a030 con 0x7f06a0004500 2022-08-01T19:37:33.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.330+0000 7f06bdffb700 1 --2- 172.21.15.17:0/1344959276 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f06a4053950 0x7f06a4055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:33.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.330+0000 7f06bdffb700 1 -- 172.21.15.17:0/1344959276 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f06b006b730 con 0x7f06a0004500 2022-08-01T19:37:33.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.330+0000 7f06b7fff700 1 --2- 172.21.15.17:0/1344959276 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f06a4053950 0x7f06a4055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:33.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.334+0000 7f06b7fff700 1 --2- 172.21.15.17:0/1344959276 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f06a4053950 0x7f06a4055e10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f06a8019000 tx=0x7f06a8013000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:33.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.338+0000 7f06bdffb700 1 -- 172.21.15.17:0/1344959276 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f06c014b970 con 0x7f06a0004500 2022-08-01T19:37:33.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.458+0000 7f06c62e0700 1 -- 172.21.15.17:0/1344959276 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06c014ae90 con 0x7f06a0004500 2022-08-01T19:37:33.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.458+0000 7f06bdffb700 1 -- 172.21.15.17:0/1344959276 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+13414 (secure 0 0 0) 0x7f06b0022090 con 0x7f06a0004500 2022-08-01T19:37:33.464 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:37:33.465 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":51,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:37:32.845736+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:37:33.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.462+0000 7f06c62e0700 1 -- 172.21.15.17:0/1344959276 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f06a4053950 msgr2=0x7f06a4055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:33.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.462+0000 7f06c62e0700 1 --2- 172.21.15.17:0/1344959276 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f06a4053950 0x7f06a4055e10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f06a8019000 tx=0x7f06a8013000).stop 2022-08-01T19:37:33.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.462+0000 7f06c62e0700 1 -- 172.21.15.17:0/1344959276 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06a0004500 msgr2=0x7f06c00a9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:33.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.462+0000 7f06c62e0700 1 --2- 172.21.15.17:0/1344959276 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06a0004500 0x7f06c00a9910 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f06b0000f50 tx=0x7f06b0007a00).stop 2022-08-01T19:37:33.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.462+0000 7f06c62e0700 1 -- 172.21.15.17:0/1344959276 shutdown_connections 2022-08-01T19:37:33.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.462+0000 7f06c62e0700 1 --2- 172.21.15.17:0/1344959276 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f06a4053950 0x7f06a4055e10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:33.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.462+0000 7f06c62e0700 1 --2- 172.21.15.17:0/1344959276 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f06c014a330 0x7f06c015a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:33.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.462+0000 7f06c62e0700 1 --2- 172.21.15.17:0/1344959276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06c00a9e00 0x7f06c0159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:33.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.462+0000 7f06c62e0700 1 --2- 172.21.15.17:0/1344959276 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06a0004500 0x7f06c00a9910 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:33.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.462+0000 7f06c62e0700 1 -- 172.21.15.17:0/1344959276 >> 172.21.15.17:0/1344959276 conn(0x7f06c005f840 msgr2=0x7f06c00b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:33.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.462+0000 7f06c62e0700 1 -- 172.21.15.17:0/1344959276 shutdown_connections 2022-08-01T19:37:33.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.462+0000 7f06c62e0700 1 -- 172.21.15.17:0/1344959276 wait complete. 2022-08-01T19:37:33.477 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-08-01T19:37:33.478 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:37:33.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.590+0000 7f0f54b77700 1 Processor -- start 2022-08-01T19:37:33.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.590+0000 7f0f54b77700 1 -- start start 2022-08-01T19:37:33.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.590+0000 7f0f54b77700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0f500aa050 0x7f0f500aa430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:33.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.590+0000 7f0f54b77700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f0f50060560 0x7f0f5013ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:33.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.590+0000 7f0f54b77700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f50061b20 con 0x7f0f5013d350 2022-08-01T19:37:33.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.590+0000 7f0f54b77700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f50061dc0 con 0x7f0f50060560 2022-08-01T19:37:33.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.590+0000 7f0f54b77700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f50060d20 con 0x7f0f500aa050 2022-08-01T19:37:33.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.590+0000 7f0f4dd9b700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f0f50060560 0x7f0f5013ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:33.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.590+0000 7f0f4dd9b700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f0f50060560 0x7f0f5013ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:36886/0 (socket says 172.21.15.17:36886) 2022-08-01T19:37:33.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.590+0000 7f0f4e59c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0f500aa050 0x7f0f500aa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:33.602 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.590+0000 7f0f4dd9b700 1 -- 172.21.15.17:0/2148864191 learned_addr learned my addr 172.21.15.17:0/2148864191 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:33.602 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.590+0000 7f0f4dd9b700 1 -- 172.21.15.17:0/2148864191 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0f500aa050 msgr2=0x7f0f500aa430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:33.602 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.590+0000 7f0f4dd9b700 1 --2- 172.21.15.17:0/2148864191 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0f500aa050 0x7f0f500aa430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:33.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.590+0000 7f0f4dd9b700 1 -- 172.21.15.17:0/2148864191 >> v1:172.21.15.77:6789/0 conn(0x7f0f5013d350 legacy=0x7f0f5013f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:33.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.590+0000 7f0f4dd9b700 1 -- 172.21.15.17:0/2148864191 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f50060fc0 con 0x7f0f50060560 2022-08-01T19:37:33.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.594+0000 7f0f4dd9b700 1 --2- 172.21.15.17:0/2148864191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f50060560 0x7f0f5013ce60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f0f5005fa00 tx=0x7f0f44006f40).ready entity=mon.1 client_cookie=ea6b911b4e5c8565 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:33.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.594+0000 7f0f4d59a700 1 -- 172.21.15.17:0/2148864191 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f4400c880 con 0x7f0f50060560 2022-08-01T19:37:33.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.594+0000 7f0f4d59a700 1 -- 172.21.15.17:0/2148864191 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f44000e90 con 0x7f0f50060560 2022-08-01T19:37:33.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.594+0000 7f0f4d59a700 1 -- 172.21.15.17:0/2148864191 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f4400cbd0 con 0x7f0f50060560 2022-08-01T19:37:33.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.594+0000 7f0f54b77700 1 -- 172.21.15.17:0/2148864191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f50060560 msgr2=0x7f0f5013ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:33.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.594+0000 7f0f54b77700 1 --2- 172.21.15.17:0/2148864191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f50060560 0x7f0f5013ce60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f0f5005fa00 tx=0x7f0f44006f40).stop 2022-08-01T19:37:33.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.594+0000 7f0f54b77700 1 -- 172.21.15.17:0/2148864191 shutdown_connections 2022-08-01T19:37:33.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.594+0000 7f0f54b77700 1 --2- 172.21.15.17:0/2148864191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f50060560 0x7f0f5013ce60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:33.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.594+0000 7f0f54b77700 1 --2- 172.21.15.17:0/2148864191 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0f500aa050 0x7f0f500aa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:33.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.594+0000 7f0f54b77700 1 -- 172.21.15.17:0/2148864191 >> 172.21.15.17:0/2148864191 conn(0x7f0f500b2b00 msgr2=0x7f0f500b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:33.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.594+0000 7f0f54b77700 1 -- 172.21.15.17:0/2148864191 shutdown_connections 2022-08-01T19:37:33.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.594+0000 7f0f54b77700 1 -- 172.21.15.17:0/2148864191 wait complete. 2022-08-01T19:37:33.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.594+0000 7f0f54b77700 1 Processor -- start 2022-08-01T19:37:33.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.594+0000 7f0f54b77700 1 -- start start 2022-08-01T19:37:33.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.598+0000 7f0f54b77700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f50060560 0x7f0f501402f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:33.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.598+0000 7f0f54b77700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0f500aa050 0x7f0f500ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:33.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.598+0000 7f0f54b77700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f5013d350 0x7f0f501428e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:33.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.598+0000 7f0f54b77700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f501407e0 con 0x7f0f5013d350 2022-08-01T19:37:33.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.598+0000 7f0f54b77700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f500af8f0 con 0x7f0f50060560 2022-08-01T19:37:33.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.598+0000 7f0f54b77700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f500ad7c0 con 0x7f0f500aa050 2022-08-01T19:37:33.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.598+0000 7f0f4ed9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f5013d350 0x7f0f501428e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:33.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.598+0000 7f0f4e59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f50060560 0x7f0f501402f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:33.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.598+0000 7f0f4ed9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f5013d350 0x7f0f501428e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38972/0 (socket says 172.21.15.17:38972) 2022-08-01T19:37:33.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.598+0000 7f0f4ed9d700 1 -- 172.21.15.17:0/1895613405 learned_addr learned my addr 172.21.15.17:0/1895613405 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:33.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.598+0000 7f0f4dd9b700 1 --2- 172.21.15.17:0/1895613405 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0f500aa050 0x7f0f500ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:33.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.598+0000 7f0f4e59c700 1 -- 172.21.15.17:0/1895613405 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0f500aa050 msgr2=0x7f0f500ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:33.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.598+0000 7f0f4e59c700 1 --2- 172.21.15.17:0/1895613405 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0f500aa050 0x7f0f500ad2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:33.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.598+0000 7f0f4e59c700 1 -- 172.21.15.17:0/1895613405 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f5013d350 msgr2=0x7f0f501428e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:33.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.598+0000 7f0f4e59c700 1 --2- 172.21.15.17:0/1895613405 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f5013d350 0x7f0f501428e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:33.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.598+0000 7f0f4e59c700 1 -- 172.21.15.17:0/1895613405 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f4400a360 con 0x7f0f50060560 2022-08-01T19:37:33.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.598+0000 7f0f4e59c700 1 --2- 172.21.15.17:0/1895613405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f50060560 0x7f0f501402f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f0f50140290 tx=0x7f0f400091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:33.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.598+0000 7f0f3f7fe700 1 -- 172.21.15.17:0/1895613405 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f400126d0 con 0x7f0f50060560 2022-08-01T19:37:33.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.598+0000 7f0f3f7fe700 1 -- 172.21.15.17:0/1895613405 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f40012830 con 0x7f0f50060560 2022-08-01T19:37:33.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.598+0000 7f0f54b77700 1 -- 172.21.15.17:0/1895613405 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f500afb40 con 0x7f0f50060560 2022-08-01T19:37:33.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.598+0000 7f0f3f7fe700 1 -- 172.21.15.17:0/1895613405 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f40012b90 con 0x7f0f50060560 2022-08-01T19:37:33.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.598+0000 7f0f54b77700 1 -- 172.21.15.17:0/1895613405 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f50143d20 con 0x7f0f50060560 2022-08-01T19:37:33.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.598+0000 7f0f54b77700 1 -- 172.21.15.17:0/1895613405 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f50061260 con 0x7f0f50060560 2022-08-01T19:37:33.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.598+0000 7f0f3f7fe700 1 -- 172.21.15.17:0/1895613405 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0f40022070 con 0x7f0f50060560 2022-08-01T19:37:33.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.602+0000 7f0f3f7fe700 1 --2- 172.21.15.17:0/1895613405 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0f2c053950 0x7f0f2c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:33.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.602+0000 7f0f3f7fe700 1 -- 172.21.15.17:0/1895613405 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f0f4006c250 con 0x7f0f50060560 2022-08-01T19:37:33.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.602+0000 7f0f4dd9b700 1 --2- 172.21.15.17:0/1895613405 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0f2c053950 0x7f0f2c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:33.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.602+0000 7f0f4dd9b700 1 --2- 172.21.15.17:0/1895613405 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0f2c053950 0x7f0f2c055e10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f0f4401b000 tx=0x7f0f50061cb0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:33.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.614+0000 7f0f3f7fe700 1 -- 172.21.15.17:0/1895613405 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0f40043350 con 0x7f0f50060560 2022-08-01T19:37:33.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.738+0000 7f0f54b77700 1 -- 172.21.15.17:0/1895613405 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f0f500574e0 con 0x7f0f50060560 2022-08-01T19:37:33.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.738+0000 7f0f3f7fe700 1 -- 172.21.15.17:0/1895613405 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v51) v1 ==== 110+0+10 (secure 0 0 0) 0x7f0f40046970 con 0x7f0f50060560 2022-08-01T19:37:33.743 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:37:33.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.738+0000 7f0f54b77700 1 -- 172.21.15.17:0/1895613405 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0f2c053950 msgr2=0x7f0f2c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:33.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.738+0000 7f0f54b77700 1 --2- 172.21.15.17:0/1895613405 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0f2c053950 0x7f0f2c055e10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f0f4401b000 tx=0x7f0f50061cb0).stop 2022-08-01T19:37:33.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.738+0000 7f0f54b77700 1 -- 172.21.15.17:0/1895613405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f50060560 msgr2=0x7f0f501402f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:33.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.738+0000 7f0f54b77700 1 --2- 172.21.15.17:0/1895613405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f50060560 0x7f0f501402f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f0f50140290 tx=0x7f0f400091d0).stop 2022-08-01T19:37:33.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.738+0000 7f0f54b77700 1 -- 172.21.15.17:0/1895613405 shutdown_connections 2022-08-01T19:37:33.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.738+0000 7f0f54b77700 1 --2- 172.21.15.17:0/1895613405 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0f2c053950 0x7f0f2c055e10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:33.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.738+0000 7f0f54b77700 1 --2- 172.21.15.17:0/1895613405 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f5013d350 0x7f0f501428e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:33.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.738+0000 7f0f54b77700 1 --2- 172.21.15.17:0/1895613405 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0f500aa050 0x7f0f500ad2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:33.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.738+0000 7f0f54b77700 1 --2- 172.21.15.17:0/1895613405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f50060560 0x7f0f501402f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:33.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.738+0000 7f0f54b77700 1 -- 172.21.15.17:0/1895613405 >> 172.21.15.17:0/1895613405 conn(0x7f0f500b2b00 msgr2=0x7f0f500b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:33.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.742+0000 7f0f54b77700 1 -- 172.21.15.17:0/1895613405 shutdown_connections 2022-08-01T19:37:33.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.742+0000 7f0f54b77700 1 -- 172.21.15.17:0/1895613405 wait complete. 2022-08-01T19:37:33.756 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:37:33.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.854+0000 7fdff140c700 1 Processor -- start 2022-08-01T19:37:33.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.854+0000 7fdff140c700 1 -- start start 2022-08-01T19:37:33.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.854+0000 7fdff140c700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fdfec0b28b0 0x7fdfec13e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:33.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.854+0000 7fdff140c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdfec141920 0x7fdfec13e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:33.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.854+0000 7fdff140c700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfec0aac20 con 0x7fdfec0b1180 2022-08-01T19:37:33.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.854+0000 7fdff140c700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfec140e20 con 0x7fdfec0b28b0 2022-08-01T19:37:33.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.854+0000 7fdff140c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec0b2c90 con 0x7fdfec141920 2022-08-01T19:37:33.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.854+0000 7fdfeb7fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fdfec0b1180 0x7fdfec141210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59406/0 (socket says 172.21.15.17:59406) 2022-08-01T19:37:33.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.854+0000 7fdfeb7fe700 1 -- 172.21.15.17:0/4254678280 learned_addr learned my addr 172.21.15.17:0/4254678280 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:37:33.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.854+0000 7fdff0c0b700 1 --2- 172.21.15.17:0/4254678280 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdfec141920 0x7fdfec13e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:33.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.854+0000 7fdfebfff700 1 --2- 172.21.15.17:0/4254678280 >> v2:172.21.15.77:3300/0 conn(0x7fdfec0b28b0 0x7fdfec13e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:33.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.854+0000 7fdfeaffd700 1 -- 172.21.15.17:0/4254678280 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2429298888 0 0) 0x7fdfe0001110 con 0x7fdfec0b1180 2022-08-01T19:37:33.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.854+0000 7fdfeaffd700 1 -- 172.21.15.17:0/4254678280 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfd8003260 con 0x7fdfec0b1180 2022-08-01T19:37:33.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.854+0000 7fdff0c0b700 1 -- 172.21.15.17:0/4254678280 >> v2:172.21.15.77:3300/0 conn(0x7fdfec0b28b0 msgr2=0x7fdfec13e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:33.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.854+0000 7fdff0c0b700 1 --2- 172.21.15.17:0/4254678280 >> v2:172.21.15.77:3300/0 conn(0x7fdfec0b28b0 0x7fdfec13e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:33.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.854+0000 7fdff0c0b700 1 -- 172.21.15.17:0/4254678280 >> v1:172.21.15.17:6789/0 conn(0x7fdfec0b1180 legacy=0x7fdfec141210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:33.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.854+0000 7fdff0c0b700 1 -- 172.21.15.17:0/4254678280 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfec0aa440 con 0x7fdfec141920 2022-08-01T19:37:33.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.854+0000 7fdff0c0b700 1 --2- 172.21.15.17:0/4254678280 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdfec141920 0x7fdfec13e6b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fdfe40032f0 tx=0x7fdfe40091d0).ready entity=mon.2 client_cookie=ac66b97e66a01819 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:33.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.854+0000 7fdfeaffd700 1 -- 172.21.15.17:0/4254678280 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdfe400b600 con 0x7fdfec141920 2022-08-01T19:37:33.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.854+0000 7fdfeaffd700 1 -- 172.21.15.17:0/4254678280 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe400b760 con 0x7fdfec141920 2022-08-01T19:37:33.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.854+0000 7fdfeaffd700 1 -- 172.21.15.17:0/4254678280 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdfe400b8d0 con 0x7fdfec141920 2022-08-01T19:37:33.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.854+0000 7fdff140c700 1 -- 172.21.15.17:0/4254678280 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdfec141920 msgr2=0x7fdfec13e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:33.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.854+0000 7fdff140c700 1 --2- 172.21.15.17:0/4254678280 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdfec141920 0x7fdfec13e6b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fdfe40032f0 tx=0x7fdfe40091d0).stop 2022-08-01T19:37:33.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.858+0000 7fdff140c700 1 -- 172.21.15.17:0/4254678280 shutdown_connections 2022-08-01T19:37:33.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.858+0000 7fdff140c700 1 --2- 172.21.15.17:0/4254678280 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdfec141920 0x7fdfec13e6b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:33.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.858+0000 7fdff140c700 1 --2- 172.21.15.17:0/4254678280 >> v2:172.21.15.77:3300/0 conn(0x7fdfec0b28b0 0x7fdfec13e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:33.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.858+0000 7fdff140c700 1 -- 172.21.15.17:0/4254678280 >> 172.21.15.17:0/4254678280 conn(0x7fdfec060310 msgr2=0x7fdfec0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:33.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.858+0000 7fdff140c700 1 -- 172.21.15.17:0/4254678280 shutdown_connections 2022-08-01T19:37:33.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.858+0000 7fdff140c700 1 -- 172.21.15.17:0/4254678280 wait complete. 2022-08-01T19:37:33.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.858+0000 7fdff140c700 1 Processor -- start 2022-08-01T19:37:33.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.858+0000 7fdff140c700 1 -- start start 2022-08-01T19:37:33.870 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.858+0000 7fdff140c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdfec0b1180 0x7fdfec150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:33.870 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.858+0000 7fdff140c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdfec0b28b0 0x7fdfec142ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:33.870 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.858+0000 7fdff140c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfec141920 0x7fdfec1540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:33.870 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.858+0000 7fdff140c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec05c5e0 con 0x7fdfec141920 2022-08-01T19:37:33.870 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.858+0000 7fdff140c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec153970 con 0x7fdfec0b28b0 2022-08-01T19:37:33.871 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.858+0000 7fdff140c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec153c10 con 0x7fdfec0b1180 2022-08-01T19:37:33.871 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.858+0000 7fdff0c0b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfec141920 0x7fdfec1540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:33.871 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.858+0000 7fdff0c0b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfec141920 0x7fdfec1540b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38988/0 (socket says 172.21.15.17:38988) 2022-08-01T19:37:33.871 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.858+0000 7fdff0c0b700 1 -- 172.21.15.17:0/3282097894 learned_addr learned my addr 172.21.15.17:0/3282097894 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:33.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.858+0000 7fdfeb7fe700 1 --2- 172.21.15.17:0/3282097894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdfec0b28b0 0x7fdfec142ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:33.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.858+0000 7fdfebfff700 1 --2- 172.21.15.17:0/3282097894 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdfec0b1180 0x7fdfec150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:33.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.858+0000 7fdff0c0b700 1 -- 172.21.15.17:0/3282097894 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdfec0b1180 msgr2=0x7fdfec150220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:33.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.858+0000 7fdff0c0b700 1 --2- 172.21.15.17:0/3282097894 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdfec0b1180 0x7fdfec150220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:33.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.858+0000 7fdff0c0b700 1 -- 172.21.15.17:0/3282097894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdfec0b28b0 msgr2=0x7fdfec142ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:33.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.858+0000 7fdff0c0b700 1 --2- 172.21.15.17:0/3282097894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdfec0b28b0 0x7fdfec142ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:33.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.858+0000 7fdff0c0b700 1 -- 172.21.15.17:0/3282097894 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe4009470 con 0x7fdfec141920 2022-08-01T19:37:33.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.858+0000 7fdfeb7fe700 1 --2- 172.21.15.17:0/3282097894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdfec0b28b0 0x7fdfec142ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:37:33.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.862+0000 7fdff0c0b700 1 --2- 172.21.15.17:0/3282097894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfec141920 0x7fdfec1540b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7fdfe40032f0 tx=0x7fdfe4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:33.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.862+0000 7fdfe97fa700 1 -- 172.21.15.17:0/3282097894 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdfe4020400 con 0x7fdfec141920 2022-08-01T19:37:33.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.862+0000 7fdfe97fa700 1 -- 172.21.15.17:0/3282097894 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe4020560 con 0x7fdfec141920 2022-08-01T19:37:33.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.862+0000 7fdff140c700 1 -- 172.21.15.17:0/3282097894 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfec150710 con 0x7fdfec141920 2022-08-01T19:37:33.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.862+0000 7fdfe97fa700 1 -- 172.21.15.17:0/3282097894 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdfe40208c0 con 0x7fdfec141920 2022-08-01T19:37:33.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.862+0000 7fdff140c700 1 -- 172.21.15.17:0/3282097894 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfec1545a0 con 0x7fdfec141920 2022-08-01T19:37:33.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.862+0000 7fdff140c700 1 -- 172.21.15.17:0/3282097894 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfec05c7e0 con 0x7fdfec141920 2022-08-01T19:37:33.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.862+0000 7fdfe97fa700 1 -- 172.21.15.17:0/3282097894 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdfe4027030 con 0x7fdfec141920 2022-08-01T19:37:33.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.862+0000 7fdfe97fa700 1 --2- 172.21.15.17:0/3282097894 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdfcc053aa0 0x7fdfcc055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:33.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.862+0000 7fdfe97fa700 1 -- 172.21.15.17:0/3282097894 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fdfe406cb60 con 0x7fdfec141920 2022-08-01T19:37:33.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.862+0000 7fdfebfff700 1 --2- 172.21.15.17:0/3282097894 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdfcc053aa0 0x7fdfcc055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:33.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.866+0000 7fdfebfff700 1 --2- 172.21.15.17:0/3282097894 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdfcc053aa0 0x7fdfcc055f60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7fdfdc00b000 tx=0x7fdfdc008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:33.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.874+0000 7fdfe97fa700 1 -- 172.21.15.17:0/3282097894 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdfe4043c60 con 0x7fdfec141920 2022-08-01T19:37:34.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.998+0000 7fdff140c700 1 -- 172.21.15.17:0/3282097894 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdfec0574e0 con 0x7fdfec141920 2022-08-01T19:37:34.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:33.998+0000 7fdfe97fa700 1 -- 172.21.15.17:0/3282097894 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v51) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdfe4015020 con 0x7fdfec141920 2022-08-01T19:37:34.002 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:37:34.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.002+0000 7fdff140c700 1 -- 172.21.15.17:0/3282097894 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdfcc053aa0 msgr2=0x7fdfcc055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:34.007 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.002+0000 7fdff140c700 1 --2- 172.21.15.17:0/3282097894 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdfcc053aa0 0x7fdfcc055f60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7fdfdc00b000 tx=0x7fdfdc008040).stop 2022-08-01T19:37:34.007 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.002+0000 7fdff140c700 1 -- 172.21.15.17:0/3282097894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfec141920 msgr2=0x7fdfec1540b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:34.007 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.002+0000 7fdff140c700 1 --2- 172.21.15.17:0/3282097894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfec141920 0x7fdfec1540b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7fdfe40032f0 tx=0x7fdfe4016000).stop 2022-08-01T19:37:34.007 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.002+0000 7fdff140c700 1 -- 172.21.15.17:0/3282097894 shutdown_connections 2022-08-01T19:37:34.008 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.002+0000 7fdff140c700 1 --2- 172.21.15.17:0/3282097894 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdfcc053aa0 0x7fdfcc055f60 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:34.008 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.002+0000 7fdff140c700 1 --2- 172.21.15.17:0/3282097894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfec141920 0x7fdfec1540b0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:34.008 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.002+0000 7fdff140c700 1 --2- 172.21.15.17:0/3282097894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdfec0b28b0 0x7fdfec142ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:34.008 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.002+0000 7fdff140c700 1 --2- 172.21.15.17:0/3282097894 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdfec0b1180 0x7fdfec150220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:34.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.002+0000 7fdff140c700 1 -- 172.21.15.17:0/3282097894 >> 172.21.15.17:0/3282097894 conn(0x7fdfec060310 msgr2=0x7fdfec0af550 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:34.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.002+0000 7fdff140c700 1 -- 172.21.15.17:0/3282097894 shutdown_connections 2022-08-01T19:37:34.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.002+0000 7fdff140c700 1 -- 172.21.15.17:0/3282097894 wait complete. 2022-08-01T19:37:34.019 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-08-01T19:37:34.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.118+0000 7fb5c8af9700 1 Processor -- start 2022-08-01T19:37:34.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.118+0000 7fb5c8af9700 1 -- start start 2022-08-01T19:37:34.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.122+0000 7fb5c8af9700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb5c40b5ed0 0x7fb5c40b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:34.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.122+0000 7fb5c8af9700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5c4061d90 con 0x7fb5c40b6ee0 2022-08-01T19:37:34.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.122+0000 7fb5c8af9700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5c40ba5a0 con 0x7fb5c40619b0 2022-08-01T19:37:34.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.122+0000 7fb5c8af9700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5c4143fa0 con 0x7fb5c40b5ed0 2022-08-01T19:37:34.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.122+0000 7fb5c2ffd700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb5c40b6ee0 0x7fb5c40b5190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59418/0 (socket says 172.21.15.17:59418) 2022-08-01T19:37:34.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.122+0000 7fb5c3fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb5c40b5ed0 0x7fb5c40b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:34.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.122+0000 7fb5c2ffd700 1 -- 172.21.15.17:0/581655152 learned_addr learned my addr 172.21.15.17:0/581655152 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:37:34.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.122+0000 7fb5c27fc700 1 -- 172.21.15.17:0/581655152 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 108767737 0 0) 0x7fb5c4061d90 con 0x7fb5c40b6ee0 2022-08-01T19:37:34.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.122+0000 7fb5c27fc700 1 -- 172.21.15.17:0/581655152 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5b0003260 con 0x7fb5c40b6ee0 2022-08-01T19:37:34.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.122+0000 7fb5c3fff700 1 -- 172.21.15.17:0/581655152 >> v1:172.21.15.77:6789/0 conn(0x7fb5c40619b0 legacy=0x7fb5c4143890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:34.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.122+0000 7fb5c27fc700 1 -- 172.21.15.17:0/581655152 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 544480159 0 0) 0x7fb5c40ba5a0 con 0x7fb5c40619b0 2022-08-01T19:37:34.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.122+0000 7fb5c3fff700 1 -- 172.21.15.17:0/581655152 >> v1:172.21.15.17:6789/0 conn(0x7fb5c40b6ee0 legacy=0x7fb5c40b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:34.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.122+0000 7fb5c3fff700 1 -- 172.21.15.17:0/581655152 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c4143150 con 0x7fb5c40b5ed0 2022-08-01T19:37:34.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.122+0000 7fb5c3fff700 1 --2- 172.21.15.17:0/581655152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5c40b5ed0 0x7fb5c40b58a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7fb5bc0032f0 tx=0x7fb5bc0091d0).ready entity=mon.0 client_cookie=65bd7faae11cd763 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:34.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.122+0000 7fb5c27fc700 1 -- 172.21.15.17:0/581655152 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5bc00b6c0 con 0x7fb5c40b5ed0 2022-08-01T19:37:34.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.122+0000 7fb5c27fc700 1 -- 172.21.15.17:0/581655152 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5bc00b820 con 0x7fb5c40b5ed0 2022-08-01T19:37:34.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.122+0000 7fb5c27fc700 1 -- 172.21.15.17:0/581655152 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5bc00b990 con 0x7fb5c40b5ed0 2022-08-01T19:37:34.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.122+0000 7fb5c8af9700 1 -- 172.21.15.17:0/581655152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5c40b5ed0 msgr2=0x7fb5c40b58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:34.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.122+0000 7fb5c8af9700 1 --2- 172.21.15.17:0/581655152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5c40b5ed0 0x7fb5c40b58a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7fb5bc0032f0 tx=0x7fb5bc0091d0).stop 2022-08-01T19:37:34.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.122+0000 7fb5c8af9700 1 -- 172.21.15.17:0/581655152 shutdown_connections 2022-08-01T19:37:34.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.122+0000 7fb5c8af9700 1 --2- 172.21.15.17:0/581655152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5c40b5ed0 0x7fb5c40b58a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:34.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.122+0000 7fb5c8af9700 1 -- 172.21.15.17:0/581655152 >> 172.21.15.17:0/581655152 conn(0x7fb5c406af80 msgr2=0x7fb5c405fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:34.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.122+0000 7fb5c8af9700 1 -- 172.21.15.17:0/581655152 shutdown_connections 2022-08-01T19:37:34.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.122+0000 7fb5c8af9700 1 -- 172.21.15.17:0/581655152 wait complete. 2022-08-01T19:37:34.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.126+0000 7fb5c8af9700 1 Processor -- start 2022-08-01T19:37:34.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.126+0000 7fb5c8af9700 1 -- start start 2022-08-01T19:37:34.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.126+0000 7fb5c8af9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5c40619b0 0x7fb5c413b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:34.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.126+0000 7fb5c8af9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5c40b5ed0 0x7fb5c4148720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:34.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.126+0000 7fb5c8af9700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb5c40b6ee0 0x7fb5c4152db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:34.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.126+0000 7fb5c8af9700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c4152480 con 0x7fb5c40b5ed0 2022-08-01T19:37:34.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.126+0000 7fb5c8af9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c413ba70 con 0x7fb5c40619b0 2022-08-01T19:37:34.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.126+0000 7fb5c8af9700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c40b7bb0 con 0x7fb5c40b6ee0 2022-08-01T19:37:34.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.126+0000 7fb5c2ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5c40b5ed0 0x7fb5c4148720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:34.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.126+0000 7fb5c2ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5c40b5ed0 0x7fb5c4148720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38998/0 (socket says 172.21.15.17:38998) 2022-08-01T19:37:34.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.126+0000 7fb5c2ffd700 1 -- 172.21.15.17:0/2146901884 learned_addr learned my addr 172.21.15.17:0/2146901884 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:34.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.126+0000 7fb5c3fff700 1 --2- 172.21.15.17:0/2146901884 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb5c40b6ee0 0x7fb5c4152db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:34.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.126+0000 7fb5c37fe700 1 --2- 172.21.15.17:0/2146901884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5c40619b0 0x7fb5c413b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:34.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.126+0000 7fb5c2ffd700 1 -- 172.21.15.17:0/2146901884 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb5c40b6ee0 msgr2=0x7fb5c4152db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:34.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.126+0000 7fb5c2ffd700 1 --2- 172.21.15.17:0/2146901884 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb5c40b6ee0 0x7fb5c4152db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:34.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.126+0000 7fb5c2ffd700 1 -- 172.21.15.17:0/2146901884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5c40619b0 msgr2=0x7fb5c413b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:34.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.126+0000 7fb5c2ffd700 1 --2- 172.21.15.17:0/2146901884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5c40619b0 0x7fb5c413b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:34.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.126+0000 7fb5c2ffd700 1 -- 172.21.15.17:0/2146901884 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5bc009470 con 0x7fb5c40b5ed0 2022-08-01T19:37:34.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.126+0000 7fb5c2ffd700 1 --2- 172.21.15.17:0/2146901884 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5c40b5ed0 0x7fb5c4148720 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7fb5b8001110 tx=0x7fb5b8007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:34.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.130+0000 7fb5c0ff9700 1 -- 172.21.15.17:0/2146901884 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5b800ce60 con 0x7fb5c40b5ed0 2022-08-01T19:37:34.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.130+0000 7fb5c0ff9700 1 -- 172.21.15.17:0/2146901884 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b8007110 con 0x7fb5c40b5ed0 2022-08-01T19:37:34.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.130+0000 7fb5c8af9700 1 -- 172.21.15.17:0/2146901884 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5c4152670 con 0x7fb5c40b5ed0 2022-08-01T19:37:34.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.130+0000 7fb5c0ff9700 1 -- 172.21.15.17:0/2146901884 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5b8007470 con 0x7fb5c40b5ed0 2022-08-01T19:37:34.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.130+0000 7fb5c8af9700 1 -- 172.21.15.17:0/2146901884 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c4148c10 con 0x7fb5c40b5ed0 2022-08-01T19:37:34.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.130+0000 7fb5c8af9700 1 -- 172.21.15.17:0/2146901884 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5c40574e0 con 0x7fb5c40b5ed0 2022-08-01T19:37:34.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.130+0000 7fb5c0ff9700 1 -- 172.21.15.17:0/2146901884 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb5b801b070 con 0x7fb5c40b5ed0 2022-08-01T19:37:34.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.130+0000 7fb5c0ff9700 1 --2- 172.21.15.17:0/2146901884 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb5a4053a70 0x7fb5a4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:34.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.130+0000 7fb5c0ff9700 1 -- 172.21.15.17:0/2146901884 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fb5b806b190 con 0x7fb5c40b5ed0 2022-08-01T19:37:34.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.130+0000 7fb5c37fe700 1 --2- 172.21.15.17:0/2146901884 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb5a4053a70 0x7fb5a4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:34.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.134+0000 7fb5c37fe700 1 --2- 172.21.15.17:0/2146901884 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb5a4053a70 0x7fb5a4055f30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7fb5b400b000 tx=0x7fb5b4009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:34.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.138+0000 7fb5c0ff9700 1 -- 172.21.15.17:0/2146901884 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb5b8042290 con 0x7fb5c40b5ed0 2022-08-01T19:37:34.272 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.270+0000 7fb5c8af9700 1 -- 172.21.15.17:0/2146901884 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb5c40b7d70 con 0x7fb5c40b5ed0 2022-08-01T19:37:34.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.270+0000 7fb5c0ff9700 1 -- 172.21.15.17:0/2146901884 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v51) v1 ==== 84+0+143 (secure 0 0 0) 0x7fb5b80458b0 con 0x7fb5c40b5ed0 2022-08-01T19:37:34.274 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:37:34.275 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2022-08-01T19:37:34.276 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:34.276 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 1, 2022-08-01T19:37:34.276 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": "device_health_metrics" 2022-08-01T19:37:34.277 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:37:34.277 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:37:34.278 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 2, 2022-08-01T19:37:34.279 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": "rbd" 2022-08-01T19:37:34.279 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:37:34.280 INFO:teuthology.orchestra.run.smithi017.stdout:] 2022-08-01T19:37:34.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.270+0000 7fb5c8af9700 1 -- 172.21.15.17:0/2146901884 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb5a4053a70 msgr2=0x7fb5a4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:34.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.270+0000 7fb5c8af9700 1 --2- 172.21.15.17:0/2146901884 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb5a4053a70 0x7fb5a4055f30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7fb5b400b000 tx=0x7fb5b4009040).stop 2022-08-01T19:37:34.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.270+0000 7fb5c8af9700 1 -- 172.21.15.17:0/2146901884 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5c40b5ed0 msgr2=0x7fb5c4148720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:34.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.270+0000 7fb5c8af9700 1 --2- 172.21.15.17:0/2146901884 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5c40b5ed0 0x7fb5c4148720 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7fb5b8001110 tx=0x7fb5b8007a40).stop 2022-08-01T19:37:34.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.270+0000 7fb5c8af9700 1 -- 172.21.15.17:0/2146901884 shutdown_connections 2022-08-01T19:37:34.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.270+0000 7fb5c8af9700 1 --2- 172.21.15.17:0/2146901884 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb5a4053a70 0x7fb5a4055f30 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:34.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.270+0000 7fb5c8af9700 1 --2- 172.21.15.17:0/2146901884 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb5c40b6ee0 0x7fb5c4152db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:34.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.270+0000 7fb5c8af9700 1 --2- 172.21.15.17:0/2146901884 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5c40b5ed0 0x7fb5c4148720 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:34.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.270+0000 7fb5c8af9700 1 --2- 172.21.15.17:0/2146901884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5c40619b0 0x7fb5c413b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:34.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.270+0000 7fb5c8af9700 1 -- 172.21.15.17:0/2146901884 >> 172.21.15.17:0/2146901884 conn(0x7fb5c406af80 msgr2=0x7fb5c4141ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:34.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.270+0000 7fb5c8af9700 1 -- 172.21.15.17:0/2146901884 shutdown_connections 2022-08-01T19:37:34.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.270+0000 7fb5c8af9700 1 -- 172.21.15.17:0/2146901884 wait complete. 2022-08-01T19:37:34.288 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-08-01T19:37:34.289 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-08-01T19:37:34.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.394+0000 7fa9c1058700 1 Processor -- start 2022-08-01T19:37:34.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.394+0000 7fa9c1058700 1 -- start start 2022-08-01T19:37:34.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9c1058700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fa9bc146ae0 0x7fa9bc1403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:34.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9c1058700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa9bc0b4820 0x7fa9bc0b5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:34.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9c1058700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9bc143cf0 con 0x7fa9bc0b4310 2022-08-01T19:37:34.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9c1058700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9bc1476a0 con 0x7fa9bc0b4820 2022-08-01T19:37:34.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9c1058700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9bc143f90 con 0x7fa9bc146ae0 2022-08-01T19:37:34.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9bb59e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa9bc0b4820 0x7fa9bc0b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:34.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9ba59c700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa9bc0b4310 0x7fa9bc1408b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59432/0 (socket says 172.21.15.17:59432) 2022-08-01T19:37:34.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9ba59c700 1 -- 172.21.15.17:0/3253920982 learned_addr learned my addr 172.21.15.17:0/3253920982 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:37:34.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9bad9d700 1 --2- 172.21.15.17:0/3253920982 >> v2:172.21.15.77:3300/0 conn(0x7fa9bc146ae0 0x7fa9bc1403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:34.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9b9d9b700 1 -- 172.21.15.17:0/3253920982 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2406054113 0 0) 0x7fa9bc143cf0 con 0x7fa9bc0b4310 2022-08-01T19:37:34.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9b9d9b700 1 -- 172.21.15.17:0/3253920982 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9a0003260 con 0x7fa9bc0b4310 2022-08-01T19:37:34.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9b9d9b700 1 -- 172.21.15.17:0/3253920982 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2760688535 0 0) 0x7fa9a0003260 con 0x7fa9bc0b4310 2022-08-01T19:37:34.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9b9d9b700 1 -- 172.21.15.17:0/3253920982 >> v2:172.21.15.77:3300/0 conn(0x7fa9bc146ae0 msgr2=0x7fa9bc1403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:34.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9b9d9b700 1 --2- 172.21.15.17:0/3253920982 >> v2:172.21.15.77:3300/0 conn(0x7fa9bc146ae0 0x7fa9bc1403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:34.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9b9d9b700 1 -- 172.21.15.17:0/3253920982 >> v2:172.21.15.17:3300/0 conn(0x7fa9bc0b4820 msgr2=0x7fa9bc0b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:34.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9b9d9b700 1 --2- 172.21.15.17:0/3253920982 >> v2:172.21.15.17:3300/0 conn(0x7fa9bc0b4820 0x7fa9bc0b5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:34.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9b9d9b700 1 -- 172.21.15.17:0/3253920982 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9bc0b0e20 con 0x7fa9bc0b4310 2022-08-01T19:37:34.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9b9d9b700 1 -- 172.21.15.17:0/3253920982 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7fa9b0001660 con 0x7fa9bc0b4310 2022-08-01T19:37:34.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9bb59e700 1 --2- 172.21.15.17:0/3253920982 >> v2:172.21.15.17:3300/0 conn(0x7fa9bc0b4820 0x7fa9bc0b5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:37:34.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9b9d9b700 1 -- 172.21.15.17:0/3253920982 >> v1:172.21.15.17:6789/0 conn(0x7fa9bc0b4310 legacy=0x7fa9bc1408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:34.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9b9d9b700 1 --2- 172.21.15.17:0/3253920982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9a0003750 0x7fa9a0003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:34.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9b9d9b700 1 --2- 172.21.15.17:0/3253920982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9a0004500 0x7fa9a0004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:34.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9b9d9b700 1 --2- 172.21.15.17:0/3253920982 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa9a0004e70 0x7fa9a000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:34.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9b9d9b700 1 -- 172.21.15.17:0/3253920982 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc1476a0 con 0x7fa9a0004500 2022-08-01T19:37:34.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9b9d9b700 1 -- 172.21.15.17:0/3253920982 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc143f90 con 0x7fa9a0003750 2022-08-01T19:37:34.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9b9d9b700 1 -- 172.21.15.17:0/3253920982 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a000bc40 con 0x7fa9a0004e70 2022-08-01T19:37:34.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9ba59c700 1 --2- 172.21.15.17:0/3253920982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9a0004500 0x7fa9a0004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:34.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9bad9d700 1 --2- 172.21.15.17:0/3253920982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9a0003750 0x7fa9a0003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:34.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9bb59e700 1 --2- 172.21.15.17:0/3253920982 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa9a0004e70 0x7fa9a000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:34.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9ba59c700 1 -- 172.21.15.17:0/3253920982 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa9a0004e70 msgr2=0x7fa9a000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:34.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9ba59c700 1 --2- 172.21.15.17:0/3253920982 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa9a0004e70 0x7fa9a000b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:34.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9ba59c700 1 -- 172.21.15.17:0/3253920982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9a0003750 msgr2=0x7fa9a0003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:34.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9ba59c700 1 --2- 172.21.15.17:0/3253920982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9a0003750 0x7fa9a0003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:34.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9ba59c700 1 -- 172.21.15.17:0/3253920982 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a0003260 con 0x7fa9a0004500 2022-08-01T19:37:34.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9bad9d700 1 --2- 172.21.15.17:0/3253920982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9a0003750 0x7fa9a0003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:37:34.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9bb59e700 1 --2- 172.21.15.17:0/3253920982 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa9a0004e70 0x7fa9a000b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:37:34.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9bb59e700 1 -- 172.21.15.17:0/3253920982 reap_dead start 2022-08-01T19:37:34.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9ba59c700 1 --2- 172.21.15.17:0/3253920982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9a0004500 0x7fa9a0004980 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7fa9b0001370 tx=0x7fa9bc147200).ready entity=mon.0 client_cookie=4d6d4e78d0c1efff server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:34.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9b9d9b700 1 -- 172.21.15.17:0/3253920982 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9b00070d0 con 0x7fa9a0004500 2022-08-01T19:37:34.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9b9d9b700 1 -- 172.21.15.17:0/3253920982 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9b0007230 con 0x7fa9a0004500 2022-08-01T19:37:34.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9b9d9b700 1 -- 172.21.15.17:0/3253920982 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9b000c860 con 0x7fa9a0004500 2022-08-01T19:37:34.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9c1058700 1 -- 172.21.15.17:0/3253920982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9a0004500 msgr2=0x7fa9a0004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:34.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9c1058700 1 --2- 172.21.15.17:0/3253920982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9a0004500 0x7fa9a0004980 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7fa9b0001370 tx=0x7fa9bc147200).stop 2022-08-01T19:37:34.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9c1058700 1 -- 172.21.15.17:0/3253920982 shutdown_connections 2022-08-01T19:37:34.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9c1058700 1 --2- 172.21.15.17:0/3253920982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9a0004500 0x7fa9a0004980 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:34.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9c1058700 1 -- 172.21.15.17:0/3253920982 >> 172.21.15.17:0/3253920982 conn(0x7fa9bc05c620 msgr2=0x7fa9bc05ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:34.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9c1058700 1 -- 172.21.15.17:0/3253920982 shutdown_connections 2022-08-01T19:37:34.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9c1058700 1 -- 172.21.15.17:0/3253920982 wait complete. 2022-08-01T19:37:34.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9c1058700 1 Processor -- start 2022-08-01T19:37:34.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.402+0000 7fa9c1058700 1 -- start start 2022-08-01T19:37:34.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.406+0000 7fa9c1058700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9a0004500 0x7fa9bc1458a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:34.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.406+0000 7fa9c1058700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9bc0b9420 0x7fa9bc0b8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:34.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.406+0000 7fa9c1058700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa9bc0b5060 0x7fa9bc0b5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:34.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.406+0000 7fa9c1058700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc0adbe0 con 0x7fa9a0004500 2022-08-01T19:37:34.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.406+0000 7fa9c1058700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc0b7570 con 0x7fa9bc0b9420 2022-08-01T19:37:34.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.406+0000 7fa9c1058700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc0b7760 con 0x7fa9bc0b5060 2022-08-01T19:37:34.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.406+0000 7fa9bad9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9a0004500 0x7fa9bc1458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:34.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.406+0000 7fa9bad9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9a0004500 0x7fa9bc1458a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39018/0 (socket says 172.21.15.17:39018) 2022-08-01T19:37:34.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.406+0000 7fa9bad9d700 1 -- 172.21.15.17:0/4105137816 learned_addr learned my addr 172.21.15.17:0/4105137816 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:34.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.406+0000 7fa9ba59c700 1 --2- 172.21.15.17:0/4105137816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9bc0b9420 0x7fa9bc0b8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:34.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.406+0000 7fa9bb59e700 1 --2- 172.21.15.17:0/4105137816 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa9bc0b5060 0x7fa9bc0b5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:34.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.406+0000 7fa9bad9d700 1 -- 172.21.15.17:0/4105137816 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa9bc0b5060 msgr2=0x7fa9bc0b5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:34.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.406+0000 7fa9bad9d700 1 --2- 172.21.15.17:0/4105137816 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa9bc0b5060 0x7fa9bc0b5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:34.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.406+0000 7fa9bad9d700 1 -- 172.21.15.17:0/4105137816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9bc0b9420 msgr2=0x7fa9bc0b8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:34.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.406+0000 7fa9bad9d700 1 --2- 172.21.15.17:0/4105137816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9bc0b9420 0x7fa9bc0b8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:34.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.406+0000 7fa9bad9d700 1 -- 172.21.15.17:0/4105137816 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9bc0b7a00 con 0x7fa9a0004500 2022-08-01T19:37:34.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.406+0000 7fa9bad9d700 1 --2- 172.21.15.17:0/4105137816 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9a0004500 0x7fa9bc1458a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7fa9ac00b000 tx=0x7fa9ac008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:34.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.406+0000 7fa9abfff700 1 -- 172.21.15.17:0/4105137816 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9ac014680 con 0x7fa9a0004500 2022-08-01T19:37:34.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.406+0000 7fa9abfff700 1 -- 172.21.15.17:0/4105137816 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ac0147e0 con 0x7fa9a0004500 2022-08-01T19:37:34.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.406+0000 7fa9c1058700 1 -- 172.21.15.17:0/4105137816 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9bc0b7f30 con 0x7fa9a0004500 2022-08-01T19:37:34.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.406+0000 7fa9abfff700 1 -- 172.21.15.17:0/4105137816 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9ac014b40 con 0x7fa9a0004500 2022-08-01T19:37:34.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.406+0000 7fa9c1058700 1 -- 172.21.15.17:0/4105137816 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9bc0b6400 con 0x7fa9a0004500 2022-08-01T19:37:34.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.410+0000 7fa9c1058700 1 -- 172.21.15.17:0/4105137816 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9bc0afd90 con 0x7fa9a0004500 2022-08-01T19:37:34.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.410+0000 7fa9abfff700 1 -- 172.21.15.17:0/4105137816 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa9ac024070 con 0x7fa9a0004500 2022-08-01T19:37:34.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.410+0000 7fa9abfff700 1 --2- 172.21.15.17:0/4105137816 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa998053a70 0x7fa998055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:34.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.410+0000 7fa9abfff700 1 -- 172.21.15.17:0/4105137816 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fa9ac06c610 con 0x7fa9a0004500 2022-08-01T19:37:34.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.410+0000 7fa9ba59c700 1 --2- 172.21.15.17:0/4105137816 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa998053a70 0x7fa998055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:34.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.410+0000 7fa9ba59c700 1 --2- 172.21.15.17:0/4105137816 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa998053a70 0x7fa998055f30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7fa9b001a000 tx=0x7fa9b0015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:34.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.422+0000 7fa9abfff700 1 -- 172.21.15.17:0/4105137816 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa9ac043580 con 0x7fa9a0004500 2022-08-01T19:37:34.540 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:34.538+0000 7fa9c1058700 1 -- 172.21.15.17:0/4105137816 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fa9bc0b80c0 con 0x7fa9a0004500 2022-08-01T19:37:35.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.542+0000 7fa9abfff700 1 -- 172.21.15.17:0/4105137816 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v52) v1 ==== 133+0+0 (secure 0 0 0) 0x7fa9ac046ba0 con 0x7fa9a0004500 2022-08-01T19:37:35.550 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.546+0000 7fa9c1058700 1 -- 172.21.15.17:0/4105137816 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa998053a70 msgr2=0x7fa998055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:35.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.546+0000 7fa9c1058700 1 --2- 172.21.15.17:0/4105137816 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa998053a70 0x7fa998055f30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7fa9b001a000 tx=0x7fa9b0015000).stop 2022-08-01T19:37:35.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.546+0000 7fa9c1058700 1 -- 172.21.15.17:0/4105137816 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9a0004500 msgr2=0x7fa9bc1458a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:35.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.546+0000 7fa9c1058700 1 --2- 172.21.15.17:0/4105137816 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9a0004500 0x7fa9bc1458a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7fa9ac00b000 tx=0x7fa9ac008040).stop 2022-08-01T19:37:35.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.546+0000 7fa9c1058700 1 -- 172.21.15.17:0/4105137816 shutdown_connections 2022-08-01T19:37:35.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.546+0000 7fa9c1058700 1 --2- 172.21.15.17:0/4105137816 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa998053a70 0x7fa998055f30 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:35.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.546+0000 7fa9c1058700 1 --2- 172.21.15.17:0/4105137816 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa9bc0b5060 0x7fa9bc0b5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:35.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.546+0000 7fa9c1058700 1 --2- 172.21.15.17:0/4105137816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9bc0b9420 0x7fa9bc0b8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:35.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.546+0000 7fa9c1058700 1 --2- 172.21.15.17:0/4105137816 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9a0004500 0x7fa9bc1458a0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:35.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.546+0000 7fa9c1058700 1 -- 172.21.15.17:0/4105137816 >> 172.21.15.17:0/4105137816 conn(0x7fa9bc05c620 msgr2=0x7fa9bc061c20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:35.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.546+0000 7fa9c1058700 1 -- 172.21.15.17:0/4105137816 shutdown_connections 2022-08-01T19:37:35.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.546+0000 7fa9c1058700 1 -- 172.21.15.17:0/4105137816 wait complete. 2022-08-01T19:37:35.555 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' created 2022-08-01T19:37:35.569 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-08-01T19:37:35.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.678+0000 7f67e5662700 1 Processor -- start 2022-08-01T19:37:35.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.678+0000 7f67e5662700 1 -- start start 2022-08-01T19:37:35.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.678+0000 7f67e5662700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67e00614b0 0x7f67e0061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:35.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.678+0000 7f67e5662700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f67e00ad750 0x7f67e00b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:35.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.678+0000 7f67e5662700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e00aee80 con 0x7f67e01405d0 2022-08-01T19:37:35.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.678+0000 7f67e5662700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67e00b70b0 con 0x7f67e00ad750 2022-08-01T19:37:35.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.678+0000 7f67e5662700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e00af0b0 con 0x7f67e00614b0 2022-08-01T19:37:35.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.678+0000 7f67e4e61700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f67e01405d0 0x7f67e01466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59456/0 (socket says 172.21.15.17:59456) 2022-08-01T19:37:35.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.678+0000 7f67e4e61700 1 -- 172.21.15.17:0/3184410692 learned_addr learned my addr 172.21.15.17:0/3184410692 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:37:35.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.678+0000 7f67d7fff700 1 --2- 172.21.15.17:0/3184410692 >> v2:172.21.15.77:3300/0 conn(0x7f67e00ad750 0x7f67e00b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:35.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.678+0000 7f67dffff700 1 --2- 172.21.15.17:0/3184410692 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67e00614b0 0x7f67e0061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:35.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.678+0000 7f67df7fe700 1 -- 172.21.15.17:0/3184410692 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2332626975 0 0) 0x7f67e00aee80 con 0x7f67e01405d0 2022-08-01T19:37:35.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.678+0000 7f67df7fe700 1 -- 172.21.15.17:0/3184410692 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67c0003260 con 0x7f67e01405d0 2022-08-01T19:37:35.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.678+0000 7f67d7fff700 1 -- 172.21.15.17:0/3184410692 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67e00614b0 msgr2=0x7f67e0061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:35.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.678+0000 7f67d7fff700 1 --2- 172.21.15.17:0/3184410692 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67e00614b0 0x7f67e0061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:35.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.678+0000 7f67d7fff700 1 -- 172.21.15.17:0/3184410692 >> v1:172.21.15.17:6789/0 conn(0x7f67e01405d0 legacy=0x7f67e01466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:35.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.678+0000 7f67d7fff700 1 -- 172.21.15.17:0/3184410692 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e00af350 con 0x7f67e00ad750 2022-08-01T19:37:35.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.678+0000 7f67d7fff700 1 --2- 172.21.15.17:0/3184410692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67e00ad750 0x7f67e00b9ff0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f67c8002f70 tx=0x7f67c8006df0).ready entity=mon.1 client_cookie=cf70812d88a304ec server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:35.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.678+0000 7f67df7fe700 1 -- 172.21.15.17:0/3184410692 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f67c800d4a0 con 0x7f67e00ad750 2022-08-01T19:37:35.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.678+0000 7f67df7fe700 1 -- 172.21.15.17:0/3184410692 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c8000e90 con 0x7f67e00ad750 2022-08-01T19:37:35.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.678+0000 7f67df7fe700 1 -- 172.21.15.17:0/3184410692 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f67c800d7f0 con 0x7f67e00ad750 2022-08-01T19:37:35.692 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.682+0000 7f67e5662700 1 -- 172.21.15.17:0/3184410692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67e00ad750 msgr2=0x7f67e00b9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:35.692 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.682+0000 7f67e5662700 1 --2- 172.21.15.17:0/3184410692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67e00ad750 0x7f67e00b9ff0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f67c8002f70 tx=0x7f67c8006df0).stop 2022-08-01T19:37:35.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.682+0000 7f67e5662700 1 -- 172.21.15.17:0/3184410692 shutdown_connections 2022-08-01T19:37:35.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.682+0000 7f67e5662700 1 --2- 172.21.15.17:0/3184410692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67e00ad750 0x7f67e00b9ff0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:35.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.682+0000 7f67e5662700 1 --2- 172.21.15.17:0/3184410692 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67e00614b0 0x7f67e0061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:35.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.682+0000 7f67e5662700 1 -- 172.21.15.17:0/3184410692 >> 172.21.15.17:0/3184410692 conn(0x7f67e0066f70 msgr2=0x7f67e0067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:35.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.682+0000 7f67e5662700 1 -- 172.21.15.17:0/3184410692 shutdown_connections 2022-08-01T19:37:35.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.682+0000 7f67e5662700 1 -- 172.21.15.17:0/3184410692 wait complete. 2022-08-01T19:37:35.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.682+0000 7f67e5662700 1 Processor -- start 2022-08-01T19:37:35.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.682+0000 7f67e5662700 1 -- start start 2022-08-01T19:37:35.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.686+0000 7f67e5662700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67e00614b0 0x7f67e01455a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:35.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.686+0000 7f67e5662700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67e00ad750 0x7f67e0152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:35.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.686+0000 7f67e5662700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67e01405d0 0x7f67e0155960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:35.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.686+0000 7f67e5662700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e01532b0 con 0x7f67e00614b0 2022-08-01T19:37:35.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.686+0000 7f67e5662700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e0145bd0 con 0x7f67e01405d0 2022-08-01T19:37:35.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.686+0000 7f67e5662700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e0152680 con 0x7f67e00ad750 2022-08-01T19:37:35.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.686+0000 7f67dffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67e00614b0 0x7f67e01455a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:35.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.686+0000 7f67dffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67e00614b0 0x7f67e01455a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39032/0 (socket says 172.21.15.17:39032) 2022-08-01T19:37:35.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.686+0000 7f67dffff700 1 -- 172.21.15.17:0/3523443988 learned_addr learned my addr 172.21.15.17:0/3523443988 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:35.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.686+0000 7f67d7fff700 1 --2- 172.21.15.17:0/3523443988 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67e00ad750 0x7f67e0152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:35.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.686+0000 7f67e4e61700 1 --2- 172.21.15.17:0/3523443988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67e01405d0 0x7f67e0155960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:35.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.686+0000 7f67dffff700 1 -- 172.21.15.17:0/3523443988 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67e00ad750 msgr2=0x7f67e0152dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:35.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.686+0000 7f67dffff700 1 --2- 172.21.15.17:0/3523443988 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67e00ad750 0x7f67e0152dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:35.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.686+0000 7f67dffff700 1 -- 172.21.15.17:0/3523443988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67e01405d0 msgr2=0x7f67e0155960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:35.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.686+0000 7f67dffff700 1 --2- 172.21.15.17:0/3523443988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67e01405d0 0x7f67e0155960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:35.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.686+0000 7f67dffff700 1 -- 172.21.15.17:0/3523443988 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c800b180 con 0x7f67e00614b0 2022-08-01T19:37:35.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.686+0000 7f67d7fff700 1 --2- 172.21.15.17:0/3523443988 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67e00ad750 0x7f67e0152dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:37:35.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.686+0000 7f67dffff700 1 --2- 172.21.15.17:0/3523443988 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67e00614b0 0x7f67e01455a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7f67d000c000 tx=0x7f67d0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:35.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.686+0000 7f67ddffb700 1 -- 172.21.15.17:0/3523443988 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f67d0010780 con 0x7f67e00614b0 2022-08-01T19:37:35.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.686+0000 7f67ddffb700 1 -- 172.21.15.17:0/3523443988 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d00108e0 con 0x7f67e00614b0 2022-08-01T19:37:35.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.686+0000 7f67e5662700 1 -- 172.21.15.17:0/3523443988 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e0152920 con 0x7f67e00614b0 2022-08-01T19:37:35.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.686+0000 7f67ddffb700 1 -- 172.21.15.17:0/3523443988 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f67d0010c40 con 0x7f67e00614b0 2022-08-01T19:37:35.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.686+0000 7f67e5662700 1 -- 172.21.15.17:0/3523443988 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e0155e50 con 0x7f67e00614b0 2022-08-01T19:37:35.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.690+0000 7f67e5662700 1 -- 172.21.15.17:0/3523443988 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67e00af5f0 con 0x7f67e00614b0 2022-08-01T19:37:35.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.690+0000 7f67ddffb700 1 -- 172.21.15.17:0/3523443988 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f67d001c460 con 0x7f67e00614b0 2022-08-01T19:37:35.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.690+0000 7f67ddffb700 1 --2- 172.21.15.17:0/3523443988 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f67c4053950 0x7f67c4055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:35.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.690+0000 7f67ddffb700 1 -- 172.21.15.17:0/3523443988 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f67d006c280 con 0x7f67e00614b0 2022-08-01T19:37:35.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.690+0000 7f67d7fff700 1 --2- 172.21.15.17:0/3523443988 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f67c4053950 0x7f67c4055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:35.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.690+0000 7f67d7fff700 1 --2- 172.21.15.17:0/3523443988 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f67c4053950 0x7f67c4055e10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f67c801a000 tx=0x7f67c8014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:35.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.702+0000 7f67ddffb700 1 -- 172.21.15.17:0/3523443988 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f67d00431b0 con 0x7f67e00614b0 2022-08-01T19:37:35.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.830+0000 7f67e5662700 1 -- 172.21.15.17:0/3523443988 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f67e0062290 con 0x7f67e00614b0 2022-08-01T19:37:35.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.890+0000 7f67ddffb700 1 -- 172.21.15.17:0/3523443988 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v53) v1 ==== 125+0+0 (secure 0 0 0) 0x7f67d00467d0 con 0x7f67e00614b0 2022-08-01T19:37:35.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.894+0000 7f67e5662700 1 -- 172.21.15.17:0/3523443988 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f67c4053950 msgr2=0x7f67c4055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:35.901 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.894+0000 7f67e5662700 1 --2- 172.21.15.17:0/3523443988 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f67c4053950 0x7f67c4055e10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f67c801a000 tx=0x7f67c8014000).stop 2022-08-01T19:37:35.901 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.894+0000 7f67e5662700 1 -- 172.21.15.17:0/3523443988 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67e00614b0 msgr2=0x7f67e01455a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:35.901 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.894+0000 7f67e5662700 1 --2- 172.21.15.17:0/3523443988 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67e00614b0 0x7f67e01455a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7f67d000c000 tx=0x7f67d0006df0).stop 2022-08-01T19:37:35.904 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.898+0000 7f67e5662700 1 -- 172.21.15.17:0/3523443988 shutdown_connections 2022-08-01T19:37:35.904 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.898+0000 7f67e5662700 1 --2- 172.21.15.17:0/3523443988 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f67c4053950 0x7f67c4055e10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:35.905 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.898+0000 7f67e5662700 1 --2- 172.21.15.17:0/3523443988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67e01405d0 0x7f67e0155960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:35.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.898+0000 7f67e5662700 1 --2- 172.21.15.17:0/3523443988 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67e00ad750 0x7f67e0152dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:35.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.898+0000 7f67e5662700 1 --2- 172.21.15.17:0/3523443988 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67e00614b0 0x7f67e01455a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:35.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.898+0000 7f67e5662700 1 -- 172.21.15.17:0/3523443988 >> 172.21.15.17:0/3523443988 conn(0x7f67e0066f70 msgr2=0x7f67e01467e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:35.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.898+0000 7f67e5662700 1 -- 172.21.15.17:0/3523443988 shutdown_connections 2022-08-01T19:37:35.907 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:35.898+0000 7f67e5662700 1 -- 172.21.15.17:0/3523443988 wait complete. 2022-08-01T19:37:35.907 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' created 2022-08-01T19:37:35.916 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-08-01T19:37:36.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.014+0000 7fafdbfff700 1 Processor -- start 2022-08-01T19:37:36.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.014+0000 7fafdbfff700 1 -- start start 2022-08-01T19:37:36.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.014+0000 7fafdbfff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fafdc0647d0 0x7fafdc064bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:36.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.014+0000 7fafdbfff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fafdc0b38a0 0x7fafdc1470d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:36.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.014+0000 7fafdbfff700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafdc0b4f60 con 0x7fafdc1436f0 2022-08-01T19:37:36.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.014+0000 7fafdbfff700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafdc1475c0 con 0x7fafdc0b38a0 2022-08-01T19:37:36.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.014+0000 7fafdbfff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafdc0b4a60 con 0x7fafdc0647d0 2022-08-01T19:37:36.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.014+0000 7fafda7fc700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fafdc0b38a0 0x7fafdc1470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:36.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.014+0000 7fafda7fc700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fafdc0b38a0 0x7fafdc1470d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39042/0 (socket says 172.21.15.17:39042) 2022-08-01T19:37:36.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.014+0000 7fafda7fc700 1 -- 172.21.15.17:0/732149807 learned_addr learned my addr 172.21.15.17:0/732149807 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:36.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.014+0000 7fafdaffd700 1 --2- 172.21.15.17:0/732149807 >> v2:172.21.15.77:3300/0 conn(0x7fafdc0647d0 0x7fafdc064bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:36.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.014+0000 7fafda7fc700 1 -- 172.21.15.17:0/732149807 >> v2:172.21.15.77:3300/0 conn(0x7fafdc0647d0 msgr2=0x7fafdc064bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:36.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.014+0000 7fafda7fc700 1 --2- 172.21.15.17:0/732149807 >> v2:172.21.15.77:3300/0 conn(0x7fafdc0647d0 0x7fafdc064bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:36.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.014+0000 7fafda7fc700 1 -- 172.21.15.17:0/732149807 >> v1:172.21.15.77:6789/0 conn(0x7fafdc1436f0 legacy=0x7fafdc1403f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:36.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.014+0000 7fafda7fc700 1 -- 172.21.15.17:0/732149807 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafdc0b4cc0 con 0x7fafdc0b38a0 2022-08-01T19:37:36.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.014+0000 7fafdaffd700 1 --2- 172.21.15.17:0/732149807 >> v2:172.21.15.77:3300/0 conn(0x7fafdc0647d0 0x7fafdc064bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:37:36.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.014+0000 7fafda7fc700 1 --2- 172.21.15.17:0/732149807 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafdc0b38a0 0x7fafdc1470d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7fafc4002f70 tx=0x7fafc4006eb0).ready entity=mon.0 client_cookie=b8bfc1d0879e8277 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:36.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.014+0000 7fafd9ffb700 1 -- 172.21.15.17:0/732149807 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafc400d5c0 con 0x7fafdc0b38a0 2022-08-01T19:37:36.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.014+0000 7fafd9ffb700 1 -- 172.21.15.17:0/732149807 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc4000e90 con 0x7fafdc0b38a0 2022-08-01T19:37:36.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.014+0000 7fafd9ffb700 1 -- 172.21.15.17:0/732149807 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafc400d950 con 0x7fafdc0b38a0 2022-08-01T19:37:36.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.014+0000 7fafdbfff700 1 -- 172.21.15.17:0/732149807 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafdc0b38a0 msgr2=0x7fafdc1470d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:36.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.014+0000 7fafdbfff700 1 --2- 172.21.15.17:0/732149807 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafdc0b38a0 0x7fafdc1470d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7fafc4002f70 tx=0x7fafc4006eb0).stop 2022-08-01T19:37:36.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.014+0000 7fafdbfff700 1 -- 172.21.15.17:0/732149807 shutdown_connections 2022-08-01T19:37:36.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.014+0000 7fafdbfff700 1 --2- 172.21.15.17:0/732149807 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafdc0b38a0 0x7fafdc1470d0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:36.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.014+0000 7fafdbfff700 1 --2- 172.21.15.17:0/732149807 >> v2:172.21.15.77:3300/0 conn(0x7fafdc0647d0 0x7fafdc064bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:36.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.014+0000 7fafdbfff700 1 -- 172.21.15.17:0/732149807 >> 172.21.15.17:0/732149807 conn(0x7fafdc05f840 msgr2=0x7fafdc05fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:36.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.014+0000 7fafdbfff700 1 -- 172.21.15.17:0/732149807 shutdown_connections 2022-08-01T19:37:36.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.014+0000 7fafdbfff700 1 -- 172.21.15.17:0/732149807 wait complete. 2022-08-01T19:37:36.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.018+0000 7fafdbfff700 1 Processor -- start 2022-08-01T19:37:36.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.018+0000 7fafdbfff700 1 -- start start 2022-08-01T19:37:36.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.018+0000 7fafdbfff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fafdc0647d0 0x7fafdc0b0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:36.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.018+0000 7fafdbfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafdc0b38a0 0x7fafdc0af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:36.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.018+0000 7fafdbfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafdc1436f0 0x7fafdc152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:36.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.018+0000 7fafdbfff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc0b8450 con 0x7fafdc1436f0 2022-08-01T19:37:36.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.018+0000 7fafdbfff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc0aea60 con 0x7fafdc0b38a0 2022-08-01T19:37:36.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.018+0000 7fafdbfff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc0aee20 con 0x7fafdc0647d0 2022-08-01T19:37:36.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.018+0000 7fafdb7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafdc1436f0 0x7fafdc152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:36.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.018+0000 7fafda7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafdc0b38a0 0x7fafdc0af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:36.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.018+0000 7fafdaffd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fafdc0647d0 0x7fafdc0b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:36.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.018+0000 7fafdb7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafdc1436f0 0x7fafdc152af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39050/0 (socket says 172.21.15.17:39050) 2022-08-01T19:37:36.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.018+0000 7fafda7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafdc0b38a0 0x7fafdc0af7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:36970/0 (socket says 172.21.15.17:36970) 2022-08-01T19:37:36.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.018+0000 7fafdb7fe700 1 -- 172.21.15.17:0/648659783 learned_addr learned my addr 172.21.15.17:0/648659783 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:36.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.018+0000 7fafdaffd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fafdc0647d0 0x7fafdc0b0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:35146/0 (socket says 172.21.15.17:35146) 2022-08-01T19:37:36.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.022+0000 7fafdb7fe700 1 -- 172.21.15.17:0/648659783 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fafdc0647d0 msgr2=0x7fafdc0b0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:36.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.022+0000 7fafdb7fe700 1 --2- 172.21.15.17:0/648659783 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fafdc0647d0 0x7fafdc0b0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:36.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.022+0000 7fafdb7fe700 1 -- 172.21.15.17:0/648659783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafdc0b38a0 msgr2=0x7fafdc0af7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:36.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.022+0000 7fafdb7fe700 1 --2- 172.21.15.17:0/648659783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafdc0b38a0 0x7fafdc0af7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:36.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.022+0000 7fafdb7fe700 1 -- 172.21.15.17:0/648659783 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafcc009470 con 0x7fafdc1436f0 2022-08-01T19:37:36.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.022+0000 7fafdb7fe700 1 --2- 172.21.15.17:0/648659783 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafdc1436f0 0x7fafdc152af0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7fafdc0665e0 tx=0x7fafd4007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:36.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.022+0000 7fafd3fff700 1 -- 172.21.15.17:0/648659783 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafd4007030 con 0x7fafdc1436f0 2022-08-01T19:37:36.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.022+0000 7fafd3fff700 1 -- 172.21.15.17:0/648659783 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd4007190 con 0x7fafdc1436f0 2022-08-01T19:37:36.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.022+0000 7fafdbfff700 1 -- 172.21.15.17:0/648659783 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc400b1e0 con 0x7fafdc1436f0 2022-08-01T19:37:36.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.022+0000 7fafd3fff700 1 -- 172.21.15.17:0/648659783 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafd4006510 con 0x7fafdc1436f0 2022-08-01T19:37:36.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.022+0000 7fafdbfff700 1 -- 172.21.15.17:0/648659783 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafdc0af320 con 0x7fafdc1436f0 2022-08-01T19:37:36.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.022+0000 7fafdbfff700 1 -- 172.21.15.17:0/648659783 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafdc0af190 con 0x7fafdc1436f0 2022-08-01T19:37:36.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.022+0000 7fafd3fff700 1 -- 172.21.15.17:0/648659783 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fafd4010030 con 0x7fafdc1436f0 2022-08-01T19:37:36.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.022+0000 7fafd3fff700 1 --2- 172.21.15.17:0/648659783 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fafc0053a10 0x7fafc0055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:36.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.022+0000 7fafd3fff700 1 -- 172.21.15.17:0/648659783 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fafd406bb60 con 0x7fafdc1436f0 2022-08-01T19:37:36.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.022+0000 7fafdaffd700 1 --2- 172.21.15.17:0/648659783 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fafc0053a10 0x7fafc0055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:36.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.026+0000 7fafdaffd700 1 --2- 172.21.15.17:0/648659783 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fafc0053a10 0x7fafc0055ed0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7fafcc00b000 tx=0x7fafcc0091d0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:36.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.026+0000 7fafd3fff700 1 -- 172.21.15.17:0/648659783 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fafd4042900 con 0x7fafdc1436f0 2022-08-01T19:37:36.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.190+0000 7fafdbfff700 1 -- 172.21.15.17:0/648659783 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fafdc0574e0 con 0x7fafdc1436f0 2022-08-01T19:37:36.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.202+0000 7fafd3fff700 1 -- 172.21.15.17:0/648659783 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 9 and data pool 10 v23) v1 ==== 188+0+0 (secure 0 0 0) 0x7fafd4045f20 con 0x7fafdc1436f0 2022-08-01T19:37:36.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.202+0000 7fafdbfff700 1 -- 172.21.15.17:0/648659783 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fafc0053a10 msgr2=0x7fafc0055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:36.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.202+0000 7fafdbfff700 1 --2- 172.21.15.17:0/648659783 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fafc0053a10 0x7fafc0055ed0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7fafcc00b000 tx=0x7fafcc0091d0).stop 2022-08-01T19:37:36.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.202+0000 7fafdbfff700 1 -- 172.21.15.17:0/648659783 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafdc1436f0 msgr2=0x7fafdc152af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:36.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.202+0000 7fafdbfff700 1 --2- 172.21.15.17:0/648659783 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafdc1436f0 0x7fafdc152af0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7fafdc0665e0 tx=0x7fafd4007c20).stop 2022-08-01T19:37:36.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.206+0000 7fafdbfff700 1 -- 172.21.15.17:0/648659783 shutdown_connections 2022-08-01T19:37:36.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.206+0000 7fafdbfff700 1 --2- 172.21.15.17:0/648659783 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fafc0053a10 0x7fafc0055ed0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:36.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.206+0000 7fafdbfff700 1 --2- 172.21.15.17:0/648659783 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafdc1436f0 0x7fafdc152af0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:36.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.206+0000 7fafdbfff700 1 --2- 172.21.15.17:0/648659783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafdc0b38a0 0x7fafdc0af7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:36.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.206+0000 7fafdbfff700 1 --2- 172.21.15.17:0/648659783 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fafdc0647d0 0x7fafdc0b0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:36.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.206+0000 7fafdbfff700 1 -- 172.21.15.17:0/648659783 >> 172.21.15.17:0/648659783 conn(0x7fafdc05f840 msgr2=0x7fafdc13e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:36.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.206+0000 7fafdbfff700 1 -- 172.21.15.17:0/648659783 shutdown_connections 2022-08-01T19:37:36.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.206+0000 7fafdbfff700 1 -- 172.21.15.17:0/648659783 wait complete. 2022-08-01T19:37:36.224 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 9 and data pool 10 2022-08-01T19:37:36.226 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:37:36.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.310+0000 7f9bfdcbd700 1 Processor -- start 2022-08-01T19:37:36.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.310+0000 7f9bfdcbd700 1 -- start start 2022-08-01T19:37:36.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.310+0000 7f9bfdcbd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9bf80614b0 0x7f9bf8061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:36.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.310+0000 7f9bfdcbd700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f9bf80ad750 0x7f9bf80b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:36.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.310+0000 7f9bfdcbd700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bf80aee80 con 0x7f9bf81405d0 2022-08-01T19:37:36.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.310+0000 7f9bfdcbd700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bf80b70b0 con 0x7f9bf80614b0 2022-08-01T19:37:36.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.310+0000 7f9bfdcbd700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bf80af0b0 con 0x7f9bf80ad750 2022-08-01T19:37:36.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.310+0000 7f9bfccbb700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9bf80614b0 0x7f9bf8061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:36.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.310+0000 7f9bfccbb700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9bf80614b0 0x7f9bf8061890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39066/0 (socket says 172.21.15.17:39066) 2022-08-01T19:37:36.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.310+0000 7f9bfccbb700 1 -- 172.21.15.17:0/3233350995 learned_addr learned my addr 172.21.15.17:0/3233350995 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:36.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.314+0000 7f9bf7fff700 1 --2- 172.21.15.17:0/3233350995 >> v2:172.21.15.77:3300/0 conn(0x7f9bf80ad750 0x7f9bf80b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:36.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.314+0000 7f9bfccbb700 1 -- 172.21.15.17:0/3233350995 >> v2:172.21.15.77:3300/0 conn(0x7f9bf80ad750 msgr2=0x7f9bf80b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:36.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.314+0000 7f9bfccbb700 1 --2- 172.21.15.17:0/3233350995 >> v2:172.21.15.77:3300/0 conn(0x7f9bf80ad750 0x7f9bf80b9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:36.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.314+0000 7f9bf77fe700 1 -- 172.21.15.17:0/3233350995 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3804068263 0 0) 0x7f9bf80aee80 con 0x7f9bf81405d0 2022-08-01T19:37:36.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.314+0000 7f9bfccbb700 1 -- 172.21.15.17:0/3233350995 >> v1:172.21.15.17:6789/0 conn(0x7f9bf81405d0 legacy=0x7f9bf81466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:36.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.314+0000 7f9bfccbb700 1 -- 172.21.15.17:0/3233350995 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf80af350 con 0x7f9bf80614b0 2022-08-01T19:37:36.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.314+0000 7f9bf7fff700 1 --2- 172.21.15.17:0/3233350995 >> v2:172.21.15.77:3300/0 conn(0x7f9bf80ad750 0x7f9bf80b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:37:36.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.314+0000 7f9bfccbb700 1 --2- 172.21.15.17:0/3233350995 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9bf80614b0 0x7f9bf8061890 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f9bf80aed30 tx=0x7f9bec006eb0).ready entity=mon.0 client_cookie=bce9fd27583132e3 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:36.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.314+0000 7f9bf77fe700 1 -- 172.21.15.17:0/3233350995 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9bec00d610 con 0x7f9bf80614b0 2022-08-01T19:37:36.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.314+0000 7f9bf77fe700 1 -- 172.21.15.17:0/3233350995 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bec000e90 con 0x7f9bf80614b0 2022-08-01T19:37:36.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.314+0000 7f9bf77fe700 1 -- 172.21.15.17:0/3233350995 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9bec00d960 con 0x7f9bf80614b0 2022-08-01T19:37:36.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.314+0000 7f9bfdcbd700 1 -- 172.21.15.17:0/3233350995 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9bf80614b0 msgr2=0x7f9bf8061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:36.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.314+0000 7f9bfdcbd700 1 --2- 172.21.15.17:0/3233350995 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9bf80614b0 0x7f9bf8061890 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f9bf80aed30 tx=0x7f9bec006eb0).stop 2022-08-01T19:37:36.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.314+0000 7f9bfdcbd700 1 -- 172.21.15.17:0/3233350995 shutdown_connections 2022-08-01T19:37:36.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.314+0000 7f9bfdcbd700 1 --2- 172.21.15.17:0/3233350995 >> v2:172.21.15.77:3300/0 conn(0x7f9bf80ad750 0x7f9bf80b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:36.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.314+0000 7f9bfdcbd700 1 --2- 172.21.15.17:0/3233350995 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9bf80614b0 0x7f9bf8061890 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:36.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.314+0000 7f9bfdcbd700 1 -- 172.21.15.17:0/3233350995 >> 172.21.15.17:0/3233350995 conn(0x7f9bf8066f70 msgr2=0x7f9bf8067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:36.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.314+0000 7f9bfdcbd700 1 -- 172.21.15.17:0/3233350995 shutdown_connections 2022-08-01T19:37:36.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.314+0000 7f9bfdcbd700 1 -- 172.21.15.17:0/3233350995 wait complete. 2022-08-01T19:37:36.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.314+0000 7f9bfdcbd700 1 Processor -- start 2022-08-01T19:37:36.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.314+0000 7f9bfdcbd700 1 -- start start 2022-08-01T19:37:36.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.318+0000 7f9bfdcbd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9bf80614b0 0x7f9bf80b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:36.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.318+0000 7f9bfdcbd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9bf80ad750 0x7f9bf80b3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:36.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.318+0000 7f9bfdcbd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9bf81405d0 0x7f9bf80b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:36.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.318+0000 7f9bfdcbd700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf8143e80 con 0x7f9bf80614b0 2022-08-01T19:37:36.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.318+0000 7f9bfdcbd700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf8153460 con 0x7f9bf80ad750 2022-08-01T19:37:36.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.318+0000 7f9bfdcbd700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf8152f50 con 0x7f9bf81405d0 2022-08-01T19:37:36.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.318+0000 7f9bfccbb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9bf80614b0 0x7f9bf80b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:36.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.318+0000 7f9bfccbb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9bf80614b0 0x7f9bf80b0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39072/0 (socket says 172.21.15.17:39072) 2022-08-01T19:37:36.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.318+0000 7f9bfccbb700 1 -- 172.21.15.17:0/2222533326 learned_addr learned my addr 172.21.15.17:0/2222533326 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:36.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.318+0000 7f9bf7fff700 1 --2- 172.21.15.17:0/2222533326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9bf80ad750 0x7f9bf80b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:36.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.318+0000 7f9bfd4bc700 1 --2- 172.21.15.17:0/2222533326 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9bf81405d0 0x7f9bf80b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:36.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.318+0000 7f9bfccbb700 1 -- 172.21.15.17:0/2222533326 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9bf81405d0 msgr2=0x7f9bf80b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:36.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.318+0000 7f9bfccbb700 1 --2- 172.21.15.17:0/2222533326 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9bf81405d0 0x7f9bf80b39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:36.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.318+0000 7f9bfccbb700 1 -- 172.21.15.17:0/2222533326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9bf80ad750 msgr2=0x7f9bf80b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:36.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.318+0000 7f9bfccbb700 1 --2- 172.21.15.17:0/2222533326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9bf80ad750 0x7f9bf80b3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:36.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.318+0000 7f9bfccbb700 1 -- 172.21.15.17:0/2222533326 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bec00b1e0 con 0x7f9bf80614b0 2022-08-01T19:37:36.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.318+0000 7f9bfd4bc700 1 --2- 172.21.15.17:0/2222533326 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9bf81405d0 0x7f9bf80b39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:37:36.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.318+0000 7f9bfccbb700 1 --2- 172.21.15.17:0/2222533326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9bf80614b0 0x7f9bf80b0b20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f9bf80aad00 tx=0x7f9bec015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:36.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.318+0000 7f9bf5ffb700 1 -- 172.21.15.17:0/2222533326 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9bec00d290 con 0x7f9bf80614b0 2022-08-01T19:37:36.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.318+0000 7f9bf5ffb700 1 -- 172.21.15.17:0/2222533326 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bec002f50 con 0x7f9bf80614b0 2022-08-01T19:37:36.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.318+0000 7f9bf5ffb700 1 -- 172.21.15.17:0/2222533326 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9bec011da0 con 0x7f9bf80614b0 2022-08-01T19:37:36.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.318+0000 7f9bfdcbd700 1 -- 172.21.15.17:0/2222533326 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf81531c0 con 0x7f9bf80614b0 2022-08-01T19:37:36.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.318+0000 7f9bfdcbd700 1 -- 172.21.15.17:0/2222533326 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf80ab080 con 0x7f9bf80614b0 2022-08-01T19:37:36.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.318+0000 7f9bfdcbd700 1 -- 172.21.15.17:0/2222533326 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf80ab240 con 0x7f9bf80614b0 2022-08-01T19:37:36.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.318+0000 7f9bf5ffb700 1 -- 172.21.15.17:0/2222533326 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9bec025030 con 0x7f9bf80614b0 2022-08-01T19:37:36.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.322+0000 7f9bf5ffb700 1 --2- 172.21.15.17:0/2222533326 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9bd8053b00 0x7f9bd8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:36.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.322+0000 7f9bf5ffb700 1 -- 172.21.15.17:0/2222533326 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9bec06d2e0 con 0x7f9bf80614b0 2022-08-01T19:37:36.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.322+0000 7f9bf7fff700 1 --2- 172.21.15.17:0/2222533326 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9bd8053b00 0x7f9bd8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:36.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.322+0000 7f9bf7fff700 1 --2- 172.21.15.17:0/2222533326 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9bd8053b00 0x7f9bd8055fc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f9be400c000 tx=0x7f9be4008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:36.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.326+0000 7f9bf5ffb700 1 -- 172.21.15.17:0/2222533326 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9bec043fd0 con 0x7f9bf80614b0 2022-08-01T19:37:36.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.450+0000 7f9bfdcbd700 1 -- 172.21.15.17:0/2222533326 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9bf80574e0 con 0x7f9bf80614b0 2022-08-01T19:37:36.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.450+0000 7f9bf5ffb700 1 -- 172.21.15.17:0/2222533326 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f9bec023090 con 0x7f9bf80614b0 2022-08-01T19:37:36.456 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:37:36.456 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":54,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:37:36.197646+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:37:34.595906+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2022-08-01T19:37:35.889700+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:37:36.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.454+0000 7f9bfdcbd700 1 -- 172.21.15.17:0/2222533326 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9bd8053b00 msgr2=0x7f9bd8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:36.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.454+0000 7f9bfdcbd700 1 --2- 172.21.15.17:0/2222533326 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9bd8053b00 0x7f9bd8055fc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f9be400c000 tx=0x7f9be4008040).stop 2022-08-01T19:37:36.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.454+0000 7f9bfdcbd700 1 -- 172.21.15.17:0/2222533326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9bf80614b0 msgr2=0x7f9bf80b0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:36.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.454+0000 7f9bfdcbd700 1 --2- 172.21.15.17:0/2222533326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9bf80614b0 0x7f9bf80b0b20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f9bf80aad00 tx=0x7f9bec015000).stop 2022-08-01T19:37:36.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.454+0000 7f9bfdcbd700 1 -- 172.21.15.17:0/2222533326 shutdown_connections 2022-08-01T19:37:36.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.454+0000 7f9bfdcbd700 1 --2- 172.21.15.17:0/2222533326 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9bd8053b00 0x7f9bd8055fc0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:36.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.454+0000 7f9bfdcbd700 1 --2- 172.21.15.17:0/2222533326 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9bf81405d0 0x7f9bf80b39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:36.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.454+0000 7f9bfdcbd700 1 --2- 172.21.15.17:0/2222533326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9bf80ad750 0x7f9bf80b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:36.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.454+0000 7f9bfdcbd700 1 --2- 172.21.15.17:0/2222533326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9bf80614b0 0x7f9bf80b0b20 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:36.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.454+0000 7f9bfdcbd700 1 -- 172.21.15.17:0/2222533326 >> 172.21.15.17:0/2222533326 conn(0x7f9bf8066f70 msgr2=0x7f9bf80b8860 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:36.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.454+0000 7f9bfdcbd700 1 -- 172.21.15.17:0/2222533326 shutdown_connections 2022-08-01T19:37:36.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.454+0000 7f9bfdcbd700 1 -- 172.21.15.17:0/2222533326 wait complete. 2022-08-01T19:37:36.470 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:37:36.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.566+0000 7f84c67a0700 1 Processor -- start 2022-08-01T19:37:36.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.566+0000 7f84c67a0700 1 -- start start 2022-08-01T19:37:36.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.566+0000 7f84c67a0700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f84c00b31d0 0x7f84c0066020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:36.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.566+0000 7f84c67a0700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f84c00a9da0 0x7f84c0060950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:36.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.566+0000 7f84c67a0700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84c01434b0 0x7f84c0143940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:36.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.566+0000 7f84c67a0700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84c0064a70 con 0x7f84c00b31d0 2022-08-01T19:37:36.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.566+0000 7f84c67a0700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84c0061f60 con 0x7f84c00a9da0 2022-08-01T19:37:36.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.566+0000 7f84c67a0700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c0067030 con 0x7f84c01434b0 2022-08-01T19:37:36.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.566+0000 7f84c579e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f84c00b31d0 0x7f84c0066020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:36.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.566+0000 7f84c579e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f84c00b31d0 0x7f84c0066020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39080/0 (socket says 172.21.15.17:39080) 2022-08-01T19:37:36.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.566+0000 7f84c579e700 1 -- 172.21.15.17:0/255980499 learned_addr learned my addr 172.21.15.17:0/255980499 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:36.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.566+0000 7f84c4f9d700 1 --2- 172.21.15.17:0/255980499 >> v2:172.21.15.77:3300/0 conn(0x7f84c00a9da0 0x7f84c0060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:36.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.566+0000 7f84c5f9f700 1 --2- 172.21.15.17:0/255980499 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84c01434b0 0x7f84c0143940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:36.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.566+0000 7f84c579e700 1 -- 172.21.15.17:0/255980499 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84c01434b0 msgr2=0x7f84c0143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:36.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.566+0000 7f84c579e700 1 --2- 172.21.15.17:0/255980499 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84c01434b0 0x7f84c0143940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:36.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.566+0000 7f84c579e700 1 -- 172.21.15.17:0/255980499 >> v2:172.21.15.77:3300/0 conn(0x7f84c00a9da0 msgr2=0x7f84c0060950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:36.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.566+0000 7f84c579e700 1 --2- 172.21.15.17:0/255980499 >> v2:172.21.15.77:3300/0 conn(0x7f84c00a9da0 0x7f84c0060950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:36.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.566+0000 7f84c579e700 1 -- 172.21.15.17:0/255980499 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c00b35b0 con 0x7f84c00b31d0 2022-08-01T19:37:36.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.566+0000 7f84c4f9d700 1 --2- 172.21.15.17:0/255980499 >> v2:172.21.15.77:3300/0 conn(0x7f84c00a9da0 0x7f84c0060950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:37:36.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.570+0000 7f84c579e700 1 --2- 172.21.15.17:0/255980499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84c00b31d0 0x7f84c0066020 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f84b80032f0 tx=0x7f84b8009290).ready entity=mon.0 client_cookie=652a61136deeba27 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:36.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.570+0000 7f84bffff700 1 -- 172.21.15.17:0/255980499 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84b800b680 con 0x7f84c00b31d0 2022-08-01T19:37:36.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.570+0000 7f84bffff700 1 -- 172.21.15.17:0/255980499 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b800b7e0 con 0x7f84c00b31d0 2022-08-01T19:37:36.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.570+0000 7f84bffff700 1 -- 172.21.15.17:0/255980499 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84b800b950 con 0x7f84c00b31d0 2022-08-01T19:37:36.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.570+0000 7f84c67a0700 1 -- 172.21.15.17:0/255980499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84c00b31d0 msgr2=0x7f84c0066020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:36.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.570+0000 7f84c67a0700 1 --2- 172.21.15.17:0/255980499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84c00b31d0 0x7f84c0066020 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f84b80032f0 tx=0x7f84b8009290).stop 2022-08-01T19:37:36.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.570+0000 7f84c67a0700 1 -- 172.21.15.17:0/255980499 shutdown_connections 2022-08-01T19:37:36.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.570+0000 7f84c67a0700 1 --2- 172.21.15.17:0/255980499 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84c01434b0 0x7f84c0143940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:36.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.570+0000 7f84c67a0700 1 --2- 172.21.15.17:0/255980499 >> v2:172.21.15.77:3300/0 conn(0x7f84c00a9da0 0x7f84c0060950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:36.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.570+0000 7f84c67a0700 1 --2- 172.21.15.17:0/255980499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84c00b31d0 0x7f84c0066020 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:36.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.570+0000 7f84c67a0700 1 -- 172.21.15.17:0/255980499 >> 172.21.15.17:0/255980499 conn(0x7f84c0063870 msgr2=0x7f84c0063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:36.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.570+0000 7f84c67a0700 1 -- 172.21.15.17:0/255980499 shutdown_connections 2022-08-01T19:37:36.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.570+0000 7f84c67a0700 1 -- 172.21.15.17:0/255980499 wait complete. 2022-08-01T19:37:36.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.570+0000 7f84c67a0700 1 Processor -- start 2022-08-01T19:37:36.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.570+0000 7f84c67a0700 1 -- start start 2022-08-01T19:37:36.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.574+0000 7f84c67a0700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84c00a9da0 0x7f84c0060ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:36.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.574+0000 7f84c67a0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84c00b31d0 0x7f84c01519a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:36.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.574+0000 7f84c67a0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84c01434b0 0x7f84c0151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:36.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.574+0000 7f84c67a0700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c00b5150 con 0x7f84c01434b0 2022-08-01T19:37:36.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.574+0000 7f84c67a0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c00b5380 con 0x7f84c00b31d0 2022-08-01T19:37:36.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.574+0000 7f84c67a0700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c00b5620 con 0x7f84c00a9da0 2022-08-01T19:37:36.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.574+0000 7f84c5f9f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84c01434b0 0x7f84c0151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:36.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.574+0000 7f84c5f9f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84c01434b0 0x7f84c0151e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39088/0 (socket says 172.21.15.17:39088) 2022-08-01T19:37:36.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.574+0000 7f84c5f9f700 1 -- 172.21.15.17:0/4008250662 learned_addr learned my addr 172.21.15.17:0/4008250662 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:36.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.574+0000 7f84c579e700 1 --2- 172.21.15.17:0/4008250662 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84c00a9da0 0x7f84c0060ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:36.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.574+0000 7f84c4f9d700 1 --2- 172.21.15.17:0/4008250662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84c00b31d0 0x7f84c01519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:36.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.574+0000 7f84c579e700 1 -- 172.21.15.17:0/4008250662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84c00b31d0 msgr2=0x7f84c01519a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:36.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.574+0000 7f84c579e700 1 --2- 172.21.15.17:0/4008250662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84c00b31d0 0x7f84c01519a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:36.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.574+0000 7f84c579e700 1 -- 172.21.15.17:0/4008250662 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84c01434b0 msgr2=0x7f84c0151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:36.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.574+0000 7f84c579e700 1 --2- 172.21.15.17:0/4008250662 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84c01434b0 0x7f84c0151e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:36.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.574+0000 7f84c579e700 1 -- 172.21.15.17:0/4008250662 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b400c2e0 con 0x7f84c00a9da0 2022-08-01T19:37:36.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.574+0000 7f84c5f9f700 1 --2- 172.21.15.17:0/4008250662 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84c01434b0 0x7f84c0151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:37:36.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.574+0000 7f84c579e700 1 --2- 172.21.15.17:0/4008250662 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84c00a9da0 0x7f84c0060ab0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f84c00b49f0 tx=0x7f84b8017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:36.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.574+0000 7f84be7fc700 1 -- 172.21.15.17:0/4008250662 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84b800b680 con 0x7f84c00a9da0 2022-08-01T19:37:36.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.574+0000 7f84be7fc700 1 -- 172.21.15.17:0/4008250662 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b801b410 con 0x7f84c00a9da0 2022-08-01T19:37:36.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.574+0000 7f84be7fc700 1 -- 172.21.15.17:0/4008250662 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84b800bee0 con 0x7f84c00a9da0 2022-08-01T19:37:36.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.574+0000 7f84c67a0700 1 -- 172.21.15.17:0/4008250662 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b8009530 con 0x7f84c00a9da0 2022-08-01T19:37:36.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.574+0000 7f84c67a0700 1 -- 172.21.15.17:0/4008250662 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c00aee10 con 0x7f84c00a9da0 2022-08-01T19:37:36.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.574+0000 7f84be7fc700 1 -- 172.21.15.17:0/4008250662 <== mon.2 v2:172.21.15.77:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f84b800f070 con 0x7f84c00a9da0 2022-08-01T19:37:36.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.578+0000 7f84c67a0700 1 -- 172.21.15.17:0/4008250662 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c0151700 con 0x7f84c00a9da0 2022-08-01T19:37:36.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.578+0000 7f84be7fc700 1 --2- 172.21.15.17:0/4008250662 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f84a0053950 0x7f84a0055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:36.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.578+0000 7f84be7fc700 1 -- 172.21.15.17:0/4008250662 <== mon.2 v2:172.21.15.77:3301/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f84b806d440 con 0x7f84c00a9da0 2022-08-01T19:37:36.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.578+0000 7f84c4f9d700 1 --2- 172.21.15.17:0/4008250662 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f84a0053950 0x7f84a0055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:36.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.578+0000 7f84c4f9d700 1 --2- 172.21.15.17:0/4008250662 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f84a0053950 0x7f84a0055e10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f84b4008000 tx=0x7f84b4008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:36.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.582+0000 7f84be7fc700 1 -- 172.21.15.17:0/4008250662 <== mon.2 v2:172.21.15.77:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f84c0151700 con 0x7f84c00a9da0 2022-08-01T19:37:36.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.710+0000 7f84c67a0700 1 -- 172.21.15.17:0/4008250662 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84c0151700 con 0x7f84c00a9da0 2022-08-01T19:37:36.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.710+0000 7f84be7fc700 1 -- 172.21.15.17:0/4008250662 <== mon.2 v2:172.21.15.77:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16299 (secure 0 0 0) 0x7f84c0151700 con 0x7f84c00a9da0 2022-08-01T19:37:36.716 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:37:36.716 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":54,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:37:36.197646+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:37:34.595906+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2022-08-01T19:37:35.889700+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:37:36.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.714+0000 7f84c67a0700 1 -- 172.21.15.17:0/4008250662 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f84a0053950 msgr2=0x7f84a0055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:36.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.714+0000 7f84c67a0700 1 --2- 172.21.15.17:0/4008250662 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f84a0053950 0x7f84a0055e10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f84b4008000 tx=0x7f84b4008040).stop 2022-08-01T19:37:36.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.714+0000 7f84c67a0700 1 -- 172.21.15.17:0/4008250662 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84c00a9da0 msgr2=0x7f84c0060ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:36.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.714+0000 7f84c67a0700 1 --2- 172.21.15.17:0/4008250662 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84c00a9da0 0x7f84c0060ab0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f84c00b49f0 tx=0x7f84b8017000).stop 2022-08-01T19:37:36.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.714+0000 7f84c67a0700 1 -- 172.21.15.17:0/4008250662 shutdown_connections 2022-08-01T19:37:36.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.714+0000 7f84c67a0700 1 --2- 172.21.15.17:0/4008250662 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f84a0053950 0x7f84a0055e10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:36.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.714+0000 7f84c67a0700 1 --2- 172.21.15.17:0/4008250662 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84c01434b0 0x7f84c0151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:36.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.714+0000 7f84c67a0700 1 --2- 172.21.15.17:0/4008250662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84c00b31d0 0x7f84c01519a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:36.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.714+0000 7f84c67a0700 1 --2- 172.21.15.17:0/4008250662 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84c00a9da0 0x7f84c0060ab0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:36.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.714+0000 7f84c67a0700 1 -- 172.21.15.17:0/4008250662 >> 172.21.15.17:0/4008250662 conn(0x7f84c0063870 msgr2=0x7f84c0063c50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:36.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.714+0000 7f84c67a0700 1 -- 172.21.15.17:0/4008250662 shutdown_connections 2022-08-01T19:37:36.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.714+0000 7f84c67a0700 1 -- 172.21.15.17:0/4008250662 wait complete. 2022-08-01T19:37:36.730 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-08-01T19:37:36.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.846+0000 7f1afb59e700 1 Processor -- start 2022-08-01T19:37:36.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.846+0000 7f1afb59e700 1 -- start start 2022-08-01T19:37:36.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.850+0000 7f1afb59e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1afc0b3190 0x7f1afc060ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:36.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.850+0000 7f1afb59e700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1afc0654c0 con 0x7f1afc061190 2022-08-01T19:37:36.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.850+0000 7f1afb59e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1afc0b3570 con 0x7f1afc0b2730 2022-08-01T19:37:36.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.850+0000 7f1afb59e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1afc062190 con 0x7f1afc0b3190 2022-08-01T19:37:36.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.850+0000 7f1afad9d700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1afc061190 0x7f1afc066dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59522/0 (socket says 172.21.15.17:59522) 2022-08-01T19:37:36.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.850+0000 7f1afad9d700 1 -- 172.21.15.17:0/279693222 learned_addr learned my addr 172.21.15.17:0/279693222 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:37:36.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.850+0000 7f1afa59c700 1 --2- 172.21.15.17:0/279693222 >> v2:172.21.15.17:3300/0 conn(0x7f1afc0b3190 0x7f1afc060ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:36.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.850+0000 7f1af9d9b700 1 -- 172.21.15.17:0/279693222 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4129571556 0 0) 0x7f1afc0654c0 con 0x7f1afc061190 2022-08-01T19:37:36.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.850+0000 7f1af9d9b700 1 -- 172.21.15.17:0/279693222 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ae0003260 con 0x7f1afc061190 2022-08-01T19:37:36.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.850+0000 7f1afa59c700 1 -- 172.21.15.17:0/279693222 >> v1:172.21.15.77:6789/0 conn(0x7f1afc0b2730 legacy=0x7f1afc0666b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:36.858 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.850+0000 7f1afa59c700 1 -- 172.21.15.17:0/279693222 >> v1:172.21.15.17:6789/0 conn(0x7f1afc061190 legacy=0x7f1afc066dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:36.858 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.850+0000 7f1afa59c700 1 -- 172.21.15.17:0/279693222 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1afc05fbf0 con 0x7f1afc0b3190 2022-08-01T19:37:36.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.850+0000 7f1afa59c700 1 --2- 172.21.15.17:0/279693222 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1afc0b3190 0x7f1afc060ca0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7f1aec002f70 tx=0x7f1aec006df0).ready entity=mon.0 client_cookie=3849f47f1d7e66bf server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:36.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.850+0000 7f1af9d9b700 1 -- 172.21.15.17:0/279693222 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1aec00d5c0 con 0x7f1afc0b3190 2022-08-01T19:37:36.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.850+0000 7f1af9d9b700 1 -- 172.21.15.17:0/279693222 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aec000e90 con 0x7f1afc0b3190 2022-08-01T19:37:36.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.850+0000 7f1af9d9b700 1 -- 172.21.15.17:0/279693222 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1aec00d910 con 0x7f1afc0b3190 2022-08-01T19:37:36.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.850+0000 7f1afb59e700 1 -- 172.21.15.17:0/279693222 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1afc0b3190 msgr2=0x7f1afc060ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:36.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.850+0000 7f1afb59e700 1 --2- 172.21.15.17:0/279693222 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1afc0b3190 0x7f1afc060ca0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7f1aec002f70 tx=0x7f1aec006df0).stop 2022-08-01T19:37:36.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.850+0000 7f1afb59e700 1 -- 172.21.15.17:0/279693222 shutdown_connections 2022-08-01T19:37:36.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.850+0000 7f1afb59e700 1 --2- 172.21.15.17:0/279693222 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1afc0b3190 0x7f1afc060ca0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:36.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.850+0000 7f1afb59e700 1 -- 172.21.15.17:0/279693222 >> 172.21.15.17:0/279693222 conn(0x7f1afc063870 msgr2=0x7f1afc063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:36.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.850+0000 7f1afb59e700 1 -- 172.21.15.17:0/279693222 shutdown_connections 2022-08-01T19:37:36.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.850+0000 7f1afb59e700 1 -- 172.21.15.17:0/279693222 wait complete. 2022-08-01T19:37:36.870 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.850+0000 7f1afb59e700 1 Processor -- start 2022-08-01T19:37:36.870 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.850+0000 7f1afb59e700 1 -- start start 2022-08-01T19:37:36.871 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.850+0000 7f1afb59e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1afc061190 0x7f1afc141580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:36.871 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.850+0000 7f1afb59e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1afc0b2730 0x7f1afc152350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:36.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.850+0000 7f1afb59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1afc0b3190 0x7f1afc152840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:36.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.850+0000 7f1afb59e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1afc0b48b0 con 0x7f1afc0b2730 2022-08-01T19:37:36.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.850+0000 7f1afb59e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1afc0b4610 con 0x7f1afc061190 2022-08-01T19:37:36.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.850+0000 7f1afb59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1afc0b5bb0 con 0x7f1afc0b3190 2022-08-01T19:37:36.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.850+0000 7f1af3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1afc0b2730 0x7f1afc152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:36.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.850+0000 7f1af3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1afc0b2730 0x7f1afc152350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39102/0 (socket says 172.21.15.17:39102) 2022-08-01T19:37:36.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.850+0000 7f1af3fff700 1 -- 172.21.15.17:0/1270118528 learned_addr learned my addr 172.21.15.17:0/1270118528 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:36.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.850+0000 7f1afad9d700 1 --2- 172.21.15.17:0/1270118528 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1afc0b3190 0x7f1afc152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:36.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.854+0000 7f1afa59c700 1 --2- 172.21.15.17:0/1270118528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1afc061190 0x7f1afc141580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:36.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.854+0000 7f1af3fff700 1 -- 172.21.15.17:0/1270118528 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1afc0b3190 msgr2=0x7f1afc152840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:36.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.854+0000 7f1af3fff700 1 --2- 172.21.15.17:0/1270118528 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1afc0b3190 0x7f1afc152840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:36.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.854+0000 7f1af3fff700 1 -- 172.21.15.17:0/1270118528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1afc061190 msgr2=0x7f1afc141580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:36.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.854+0000 7f1af3fff700 1 --2- 172.21.15.17:0/1270118528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1afc061190 0x7f1afc141580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:36.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.854+0000 7f1af3fff700 1 -- 172.21.15.17:0/1270118528 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aec00b180 con 0x7f1afc0b2730 2022-08-01T19:37:36.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.854+0000 7f1afad9d700 1 --2- 172.21.15.17:0/1270118528 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1afc0b3190 0x7f1afc152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:37:36.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.854+0000 7f1afa59c700 1 --2- 172.21.15.17:0/1270118528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1afc061190 0x7f1afc141580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:37:36.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.854+0000 7f1af3fff700 1 --2- 172.21.15.17:0/1270118528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1afc0b2730 0x7f1afc152350 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7f1afc068010 tx=0x7f1ae4007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:36.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.854+0000 7f1af37fe700 1 -- 172.21.15.17:0/1270118528 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ae400ce30 con 0x7f1afc0b2730 2022-08-01T19:37:36.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.854+0000 7f1af37fe700 1 -- 172.21.15.17:0/1270118528 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae4007240 con 0x7f1afc0b2730 2022-08-01T19:37:36.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.854+0000 7f1af37fe700 1 -- 172.21.15.17:0/1270118528 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ae4006700 con 0x7f1afc0b2730 2022-08-01T19:37:36.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.854+0000 7f1afb59e700 1 -- 172.21.15.17:0/1270118528 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1afc0b5e50 con 0x7f1afc0b2730 2022-08-01T19:37:36.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.854+0000 7f1afb59e700 1 -- 172.21.15.17:0/1270118528 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1afc140440 con 0x7f1afc0b2730 2022-08-01T19:37:36.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.854+0000 7f1af37fe700 1 -- 172.21.15.17:0/1270118528 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1ae4010030 con 0x7f1afc0b2730 2022-08-01T19:37:36.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.854+0000 7f1afb59e700 1 -- 172.21.15.17:0/1270118528 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1afc0b5fe0 con 0x7f1afc0b2730 2022-08-01T19:37:36.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.854+0000 7f1af37fe700 1 --2- 172.21.15.17:0/1270118528 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1ad40539e0 0x7f1ad4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:36.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.854+0000 7f1af37fe700 1 -- 172.21.15.17:0/1270118528 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1ae406bce0 con 0x7f1afc0b2730 2022-08-01T19:37:36.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.858+0000 7f1afa59c700 1 --2- 172.21.15.17:0/1270118528 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1ad40539e0 0x7f1ad4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:36.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.858+0000 7f1afa59c700 1 --2- 172.21.15.17:0/1270118528 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1ad40539e0 0x7f1ad4055ea0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f1aec018000 tx=0x7f1aec014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:36.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:36.862+0000 7f1af37fe700 1 -- 172.21.15.17:0/1270118528 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1ae4042aa0 con 0x7f1afc0b2730 2022-08-01T19:37:37.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.034+0000 7f1afb59e700 1 -- 172.21.15.17:0/1270118528 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1afc0574e0 con 0x7f1afc0b2730 2022-08-01T19:37:37.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.210+0000 7f1af37fe700 1 -- 172.21.15.17:0/1270118528 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1ae40460c0 con 0x7f1afc0b2730 2022-08-01T19:37:37.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.218+0000 7f1afb59e700 1 -- 172.21.15.17:0/1270118528 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1ad40539e0 msgr2=0x7f1ad4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:37.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.218+0000 7f1afb59e700 1 --2- 172.21.15.17:0/1270118528 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1ad40539e0 0x7f1ad4055ea0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f1aec018000 tx=0x7f1aec014000).stop 2022-08-01T19:37:37.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.218+0000 7f1afb59e700 1 -- 172.21.15.17:0/1270118528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1afc0b2730 msgr2=0x7f1afc152350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:37.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.218+0000 7f1afb59e700 1 --2- 172.21.15.17:0/1270118528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1afc0b2730 0x7f1afc152350 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7f1afc068010 tx=0x7f1ae4007be0).stop 2022-08-01T19:37:37.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.218+0000 7f1afb59e700 1 -- 172.21.15.17:0/1270118528 shutdown_connections 2022-08-01T19:37:37.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.218+0000 7f1afb59e700 1 --2- 172.21.15.17:0/1270118528 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1ad40539e0 0x7f1ad4055ea0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:37.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.218+0000 7f1afb59e700 1 --2- 172.21.15.17:0/1270118528 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1afc0b3190 0x7f1afc152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:37.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.218+0000 7f1afb59e700 1 --2- 172.21.15.17:0/1270118528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1afc0b2730 0x7f1afc152350 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:37.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.218+0000 7f1afb59e700 1 --2- 172.21.15.17:0/1270118528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1afc061190 0x7f1afc141580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:37.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.218+0000 7f1afb59e700 1 -- 172.21.15.17:0/1270118528 >> 172.21.15.17:0/1270118528 conn(0x7f1afc063870 msgr2=0x7f1afc066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:37.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.218+0000 7f1afb59e700 1 -- 172.21.15.17:0/1270118528 shutdown_connections 2022-08-01T19:37:37.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.218+0000 7f1afb59e700 1 -- 172.21.15.17:0/1270118528 wait complete. 2022-08-01T19:37:37.232 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:37:37.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.342+0000 7f2fc2c77700 1 Processor -- start 2022-08-01T19:37:37.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.342+0000 7f2fc2c77700 1 -- start start 2022-08-01T19:37:37.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.342+0000 7f2fc2c77700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2fbc0b1f40 0x7f2fbc0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:37.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.342+0000 7f2fc2c77700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2fbc0a7250 0x7f2fbc1448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:37.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.342+0000 7f2fc2c77700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fbc0b04e0 con 0x7f2fbc148530 2022-08-01T19:37:37.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.342+0000 7f2fc2c77700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fbc0a9250 con 0x7f2fbc0b1f40 2022-08-01T19:37:37.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.342+0000 7f2fc2c77700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc0b0780 con 0x7f2fbc0a7250 2022-08-01T19:37:37.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.342+0000 7f2fc0a13700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2fbc0b1f40 0x7f2fbc0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:37.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.342+0000 7f2fbbfff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2fbc148530 0x7f2fbc0b2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59534/0 (socket says 172.21.15.17:59534) 2022-08-01T19:37:37.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.342+0000 7f2fbbfff700 1 -- 172.21.15.17:0/2209626761 learned_addr learned my addr 172.21.15.17:0/2209626761 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:37:37.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.342+0000 7f2fc1214700 1 --2- 172.21.15.17:0/2209626761 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2fbc0a7250 0x7f2fbc1448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:37.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.342+0000 7f2fbb7fe700 1 -- 172.21.15.17:0/2209626761 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3611684265 0 0) 0x7f2fbc0b04e0 con 0x7f2fbc148530 2022-08-01T19:37:37.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.342+0000 7f2fbb7fe700 1 -- 172.21.15.17:0/2209626761 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fa0003260 con 0x7f2fbc148530 2022-08-01T19:37:37.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.342+0000 7f2fc0a13700 1 -- 172.21.15.17:0/2209626761 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2fbc0a7250 msgr2=0x7f2fbc1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:37.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.342+0000 7f2fc0a13700 1 --2- 172.21.15.17:0/2209626761 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2fbc0a7250 0x7f2fbc1448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:37.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.342+0000 7f2fc0a13700 1 -- 172.21.15.17:0/2209626761 >> v1:172.21.15.17:6789/0 conn(0x7f2fbc148530 legacy=0x7f2fbc0b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:37.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.342+0000 7f2fc0a13700 1 -- 172.21.15.17:0/2209626761 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fbc0b0a20 con 0x7f2fbc0b1f40 2022-08-01T19:37:37.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.342+0000 7f2fc1214700 1 --2- 172.21.15.17:0/2209626761 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2fbc0a7250 0x7f2fbc1448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:37:37.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.342+0000 7f2fc0a13700 1 --2- 172.21.15.17:0/2209626761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fbc0b1f40 0x7f2fbc0b1160 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f2fbc1478c0 tx=0x7f2fb0006df0).ready entity=mon.0 client_cookie=530724f727fc7029 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:37.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.342+0000 7f2fbb7fe700 1 -- 172.21.15.17:0/2209626761 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fb000d570 con 0x7f2fbc0b1f40 2022-08-01T19:37:37.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.342+0000 7f2fbb7fe700 1 -- 172.21.15.17:0/2209626761 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb0000e90 con 0x7f2fbc0b1f40 2022-08-01T19:37:37.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.342+0000 7f2fbb7fe700 1 -- 172.21.15.17:0/2209626761 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fb000d8c0 con 0x7f2fbc0b1f40 2022-08-01T19:37:37.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.342+0000 7f2fc2c77700 1 -- 172.21.15.17:0/2209626761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fbc0b1f40 msgr2=0x7f2fbc0b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:37.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.342+0000 7f2fc2c77700 1 --2- 172.21.15.17:0/2209626761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fbc0b1f40 0x7f2fbc0b1160 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f2fbc1478c0 tx=0x7f2fb0006df0).stop 2022-08-01T19:37:37.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.342+0000 7f2fc2c77700 1 -- 172.21.15.17:0/2209626761 shutdown_connections 2022-08-01T19:37:37.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.342+0000 7f2fc2c77700 1 --2- 172.21.15.17:0/2209626761 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2fbc0a7250 0x7f2fbc1448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:37.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.342+0000 7f2fc2c77700 1 --2- 172.21.15.17:0/2209626761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fbc0b1f40 0x7f2fbc0b1160 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:37.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.342+0000 7f2fc2c77700 1 -- 172.21.15.17:0/2209626761 >> 172.21.15.17:0/2209626761 conn(0x7f2fbc05f840 msgr2=0x7f2fbc05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:37.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.342+0000 7f2fc2c77700 1 -- 172.21.15.17:0/2209626761 shutdown_connections 2022-08-01T19:37:37.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.342+0000 7f2fc2c77700 1 -- 172.21.15.17:0/2209626761 wait complete. 2022-08-01T19:37:37.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.346+0000 7f2fc2c77700 1 Processor -- start 2022-08-01T19:37:37.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.346+0000 7f2fc2c77700 1 -- start start 2022-08-01T19:37:37.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.346+0000 7f2fc2c77700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2fbc0a7250 0x7f2fbc0a9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:37.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.346+0000 7f2fc2c77700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2fbc0b1f40 0x7f2fbc143b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:37.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.346+0000 7f2fc2c77700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fbc148530 0x7f2fbc144000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:37.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.346+0000 7f2fc2c77700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc0a9f40 con 0x7f2fbc148530 2022-08-01T19:37:37.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.346+0000 7f2fc2c77700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc1444f0 con 0x7f2fbc0a7250 2022-08-01T19:37:37.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.346+0000 7f2fc2c77700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc0aafd0 con 0x7f2fbc0b1f40 2022-08-01T19:37:37.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.346+0000 7f2fc1214700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fbc148530 0x7f2fbc144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:37.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.346+0000 7f2fc1214700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fbc148530 0x7f2fbc144000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39118/0 (socket says 172.21.15.17:39118) 2022-08-01T19:37:37.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.346+0000 7f2fc1214700 1 -- 172.21.15.17:0/830937510 learned_addr learned my addr 172.21.15.17:0/830937510 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:37.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.346+0000 7f2fbbfff700 1 --2- 172.21.15.17:0/830937510 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2fbc0b1f40 0x7f2fbc143b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:37.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.346+0000 7f2fc0a13700 1 --2- 172.21.15.17:0/830937510 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2fbc0a7250 0x7f2fbc0a9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:37.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.346+0000 7f2fc1214700 1 -- 172.21.15.17:0/830937510 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2fbc0b1f40 msgr2=0x7f2fbc143b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:37.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.346+0000 7f2fc1214700 1 --2- 172.21.15.17:0/830937510 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2fbc0b1f40 0x7f2fbc143b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:37.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.346+0000 7f2fc1214700 1 -- 172.21.15.17:0/830937510 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2fbc0a7250 msgr2=0x7f2fbc0a9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:37.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.346+0000 7f2fc1214700 1 --2- 172.21.15.17:0/830937510 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2fbc0a7250 0x7f2fbc0a9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:37.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.346+0000 7f2fc1214700 1 -- 172.21.15.17:0/830937510 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb000b180 con 0x7f2fbc148530 2022-08-01T19:37:37.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.346+0000 7f2fc1214700 1 --2- 172.21.15.17:0/830937510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fbc148530 0x7f2fbc144000 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f2fb4006cc0 tx=0x7f2fb4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:37.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.346+0000 7f2fb9ffb700 1 -- 172.21.15.17:0/830937510 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fb400f710 con 0x7f2fbc148530 2022-08-01T19:37:37.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.346+0000 7f2fb9ffb700 1 -- 172.21.15.17:0/830937510 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb400f870 con 0x7f2fbc148530 2022-08-01T19:37:37.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.346+0000 7f2fc2c77700 1 -- 172.21.15.17:0/830937510 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fbc0ab270 con 0x7f2fbc148530 2022-08-01T19:37:37.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.346+0000 7f2fb9ffb700 1 -- 172.21.15.17:0/830937510 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fb400fbd0 con 0x7f2fbc148530 2022-08-01T19:37:37.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.346+0000 7f2fc2c77700 1 -- 172.21.15.17:0/830937510 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fbc0ab7b0 con 0x7f2fbc148530 2022-08-01T19:37:37.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.350+0000 7f2fc2c77700 1 -- 172.21.15.17:0/830937510 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fbc0ab940 con 0x7f2fbc148530 2022-08-01T19:37:37.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.350+0000 7f2fb9ffb700 1 -- 172.21.15.17:0/830937510 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2fb401a030 con 0x7f2fbc148530 2022-08-01T19:37:37.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.350+0000 7f2fb9ffb700 1 --2- 172.21.15.17:0/830937510 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2f9c0539e0 0x7f2f9c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:37.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.350+0000 7f2fb9ffb700 1 -- 172.21.15.17:0/830937510 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f2fb406bfc0 con 0x7f2fbc148530 2022-08-01T19:37:37.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.350+0000 7f2fc0a13700 1 --2- 172.21.15.17:0/830937510 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2f9c0539e0 0x7f2f9c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:37.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.350+0000 7f2fc0a13700 1 --2- 172.21.15.17:0/830937510 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2f9c0539e0 0x7f2f9c055ea0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f2fb001a000 tx=0x7f2fb0014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:37.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.358+0000 7f2fb9ffb700 1 -- 172.21.15.17:0/830937510 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2fb4042d80 con 0x7f2fbc148530 2022-08-01T19:37:37.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.518+0000 7f2fc2c77700 1 -- 172.21.15.17:0/830937510 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2fbc0574e0 con 0x7f2fbc148530 2022-08-01T19:37:37.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.522+0000 7f2fb9ffb700 1 -- 172.21.15.17:0/830937510 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f2fb40463a0 con 0x7f2fbc148530 2022-08-01T19:37:37.525 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:37:37.525 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":24,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/1914125901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":1914125901},{"type":"v1","addr":"172.21.15.17:6833","nonce":1914125901}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":22},{"gid":5587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1403688289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1403688289},{"type":"v1","addr":"172.21.15.77:6835","nonce":1403688289}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":22}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:37:36.197818+0000","modified":"2022-08-01T19:37:37.201700+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5589":{"gid":5589,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/3325991310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":3325991310},{"type":"v1","addr":"172.21.15.17:6835","nonce":3325991310}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-08-01T19:37:37.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.522+0000 7f2fc2c77700 1 -- 172.21.15.17:0/830937510 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2f9c0539e0 msgr2=0x7f2f9c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:37.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.522+0000 7f2fc2c77700 1 --2- 172.21.15.17:0/830937510 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2f9c0539e0 0x7f2f9c055ea0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f2fb001a000 tx=0x7f2fb0014000).stop 2022-08-01T19:37:37.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.522+0000 7f2fc2c77700 1 -- 172.21.15.17:0/830937510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fbc148530 msgr2=0x7f2fbc144000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:37.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.522+0000 7f2fc2c77700 1 --2- 172.21.15.17:0/830937510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fbc148530 0x7f2fbc144000 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f2fb4006cc0 tx=0x7f2fb4008040).stop 2022-08-01T19:37:37.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.522+0000 7f2fc2c77700 1 -- 172.21.15.17:0/830937510 shutdown_connections 2022-08-01T19:37:37.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.522+0000 7f2fc2c77700 1 --2- 172.21.15.17:0/830937510 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2f9c0539e0 0x7f2f9c055ea0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:37.529 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.522+0000 7f2fc2c77700 1 --2- 172.21.15.17:0/830937510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fbc148530 0x7f2fbc144000 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:37.529 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.522+0000 7f2fc2c77700 1 --2- 172.21.15.17:0/830937510 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2fbc0b1f40 0x7f2fbc143b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:37.529 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.522+0000 7f2fc2c77700 1 --2- 172.21.15.17:0/830937510 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2fbc0a7250 0x7f2fbc0a9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:37.529 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.522+0000 7f2fc2c77700 1 -- 172.21.15.17:0/830937510 >> 172.21.15.17:0/830937510 conn(0x7f2fbc05f840 msgr2=0x7f2fbc0a79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:37.530 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.522+0000 7f2fc2c77700 1 -- 172.21.15.17:0/830937510 shutdown_connections 2022-08-01T19:37:37.530 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.522+0000 7f2fc2c77700 1 -- 172.21.15.17:0/830937510 wait complete. 2022-08-01T19:37:37.530 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 24 2022-08-01T19:37:37.539 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:37:37.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.642+0000 7f090ebe3700 1 Processor -- start 2022-08-01T19:37:37.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.642+0000 7f090ebe3700 1 -- start start 2022-08-01T19:37:37.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.642+0000 7f090ebe3700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f09080b31d0 0x7f0908066020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:37.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.642+0000 7f090ebe3700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09080a9da0 0x7f0908060950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:37.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.642+0000 7f090ebe3700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0908067030 con 0x7f09081434b0 2022-08-01T19:37:37.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.642+0000 7f090ebe3700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0908064a70 con 0x7f09080b31d0 2022-08-01T19:37:37.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.642+0000 7f090ebe3700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0908061f60 con 0x7f09080a9da0 2022-08-01T19:37:37.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.642+0000 7f090dbe1700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f09080b31d0 0x7f0908066020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:37.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.642+0000 7f090dbe1700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f09080b31d0 0x7f0908066020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39124/0 (socket says 172.21.15.17:39124) 2022-08-01T19:37:37.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.642+0000 7f090dbe1700 1 -- 172.21.15.17:0/3399033852 learned_addr learned my addr 172.21.15.17:0/3399033852 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:37.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.642+0000 7f090d3e0700 1 --2- 172.21.15.17:0/3399033852 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09080a9da0 0x7f0908060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:37.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.642+0000 7f090dbe1700 1 -- 172.21.15.17:0/3399033852 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09080a9da0 msgr2=0x7f0908060950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:37.660 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.642+0000 7f090dbe1700 1 --2- 172.21.15.17:0/3399033852 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09080a9da0 0x7f0908060950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:37.660 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.642+0000 7f090dbe1700 1 -- 172.21.15.17:0/3399033852 >> v1:172.21.15.77:6789/0 conn(0x7f09081434b0 legacy=0x7f0908143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:37.660 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.642+0000 7f090dbe1700 1 -- 172.21.15.17:0/3399033852 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09080b35b0 con 0x7f09080b31d0 2022-08-01T19:37:37.660 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.642+0000 7f090dbe1700 1 --2- 172.21.15.17:0/3399033852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09080b31d0 0x7f0908066020 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f08f8002f70 tx=0x7f08f8006eb0).ready entity=mon.0 client_cookie=b488b5e7a49cd1ca server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:37.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.642+0000 7f090cbdf700 1 -- 172.21.15.17:0/3399033852 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f08f800d570 con 0x7f09080b31d0 2022-08-01T19:37:37.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.642+0000 7f090cbdf700 1 -- 172.21.15.17:0/3399033852 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f8000e90 con 0x7f09080b31d0 2022-08-01T19:37:37.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.642+0000 7f090cbdf700 1 -- 172.21.15.17:0/3399033852 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f08f800d900 con 0x7f09080b31d0 2022-08-01T19:37:37.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.642+0000 7f090ebe3700 1 -- 172.21.15.17:0/3399033852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09080b31d0 msgr2=0x7f0908066020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:37.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.642+0000 7f090ebe3700 1 --2- 172.21.15.17:0/3399033852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09080b31d0 0x7f0908066020 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f08f8002f70 tx=0x7f08f8006eb0).stop 2022-08-01T19:37:37.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.642+0000 7f090ebe3700 1 -- 172.21.15.17:0/3399033852 shutdown_connections 2022-08-01T19:37:37.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.642+0000 7f090ebe3700 1 --2- 172.21.15.17:0/3399033852 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09080a9da0 0x7f0908060950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:37.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.642+0000 7f090ebe3700 1 --2- 172.21.15.17:0/3399033852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09080b31d0 0x7f0908066020 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:37.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.642+0000 7f090ebe3700 1 -- 172.21.15.17:0/3399033852 >> 172.21.15.17:0/3399033852 conn(0x7f0908063870 msgr2=0x7f0908063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:37.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.642+0000 7f090ebe3700 1 -- 172.21.15.17:0/3399033852 shutdown_connections 2022-08-01T19:37:37.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.642+0000 7f090ebe3700 1 -- 172.21.15.17:0/3399033852 wait complete. 2022-08-01T19:37:37.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.646+0000 7f090ebe3700 1 Processor -- start 2022-08-01T19:37:37.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.646+0000 7f090ebe3700 1 -- start start 2022-08-01T19:37:37.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.646+0000 7f090ebe3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09080a9da0 0x7f09080ae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:37.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.646+0000 7f090ebe3700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09080b31d0 0x7f0908140ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:37.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.646+0000 7f090ebe3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09081434b0 0x7f0908140120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:37.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.646+0000 7f090ebe3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0908146a30 con 0x7f09080a9da0 2022-08-01T19:37:37.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.646+0000 7f090ebe3700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0908146cd0 con 0x7f09081434b0 2022-08-01T19:37:37.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.646+0000 7f090ebe3700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0908146f70 con 0x7f09080b31d0 2022-08-01T19:37:37.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.646+0000 7f090dbe1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09080a9da0 0x7f09080ae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:37.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.646+0000 7f090dbe1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09080a9da0 0x7f09080ae7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39130/0 (socket says 172.21.15.17:39130) 2022-08-01T19:37:37.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.646+0000 7f090dbe1700 1 -- 172.21.15.17:0/1198736232 learned_addr learned my addr 172.21.15.17:0/1198736232 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:37.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.646+0000 7f090d3e0700 1 --2- 172.21.15.17:0/1198736232 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09080b31d0 0x7f0908140ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:37.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.646+0000 7f090e3e2700 1 --2- 172.21.15.17:0/1198736232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09081434b0 0x7f0908140120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:37.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.646+0000 7f090dbe1700 1 -- 172.21.15.17:0/1198736232 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09080b31d0 msgr2=0x7f0908140ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:37.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.646+0000 7f090dbe1700 1 --2- 172.21.15.17:0/1198736232 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09080b31d0 0x7f0908140ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:37.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.646+0000 7f090dbe1700 1 -- 172.21.15.17:0/1198736232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09081434b0 msgr2=0x7f0908140120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:37.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.646+0000 7f090dbe1700 1 --2- 172.21.15.17:0/1198736232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09081434b0 0x7f0908140120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:37.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.646+0000 7f090dbe1700 1 -- 172.21.15.17:0/1198736232 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f800b1e0 con 0x7f09080a9da0 2022-08-01T19:37:37.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.646+0000 7f090dbe1700 1 --2- 172.21.15.17:0/1198736232 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09080a9da0 0x7f09080ae7b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f08f8002f70 tx=0x7f08f8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:37.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.646+0000 7f08f6ffd700 1 -- 172.21.15.17:0/1198736232 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f08f8011860 con 0x7f09080a9da0 2022-08-01T19:37:37.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.646+0000 7f08f6ffd700 1 -- 172.21.15.17:0/1198736232 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f80119c0 con 0x7f09080a9da0 2022-08-01T19:37:37.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.646+0000 7f090ebe3700 1 -- 172.21.15.17:0/1198736232 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f090813fc80 con 0x7f09080a9da0 2022-08-01T19:37:37.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.646+0000 7f08f6ffd700 1 -- 172.21.15.17:0/1198736232 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f08f8011d20 con 0x7f09080a9da0 2022-08-01T19:37:37.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.646+0000 7f090ebe3700 1 -- 172.21.15.17:0/1198736232 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f090814b430 con 0x7f09080a9da0 2022-08-01T19:37:37.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.650+0000 7f08f6ffd700 1 -- 172.21.15.17:0/1198736232 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f08f8025030 con 0x7f09080a9da0 2022-08-01T19:37:37.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.650+0000 7f090ebe3700 1 -- 172.21.15.17:0/1198736232 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0908066af0 con 0x7f09080a9da0 2022-08-01T19:37:37.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.650+0000 7f08f6ffd700 1 --2- 172.21.15.17:0/1198736232 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f08e8053950 0x7f08e8055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:37.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.650+0000 7f08f6ffd700 1 -- 172.21.15.17:0/1198736232 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f08f806d360 con 0x7f09080a9da0 2022-08-01T19:37:37.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.650+0000 7f090d3e0700 1 --2- 172.21.15.17:0/1198736232 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f08e8053950 0x7f08e8055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:37.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.650+0000 7f090d3e0700 1 --2- 172.21.15.17:0/1198736232 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f08e8053950 0x7f08e8055e10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f090400b000 tx=0x7f0904006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:37.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.658+0000 7f08f6ffd700 1 -- 172.21.15.17:0/1198736232 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f08f8044010 con 0x7f09080a9da0 2022-08-01T19:37:37.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.786+0000 7f090ebe3700 1 -- 172.21.15.17:0/1198736232 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09080574e0 con 0x7f09080a9da0 2022-08-01T19:37:37.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.790+0000 7f08f6ffd700 1 -- 172.21.15.17:0/1198736232 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16273 (secure 0 0 0) 0x7f08f8023090 con 0x7f09080a9da0 2022-08-01T19:37:37.793 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:37:37.793 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":55,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:37:37.200615+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:37:34.595906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2022-08-01T19:37:35.889700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:37:37.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.790+0000 7f090ebe3700 1 -- 172.21.15.17:0/1198736232 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f08e8053950 msgr2=0x7f08e8055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:37.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.790+0000 7f090ebe3700 1 --2- 172.21.15.17:0/1198736232 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f08e8053950 0x7f08e8055e10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f090400b000 tx=0x7f0904006df0).stop 2022-08-01T19:37:37.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.790+0000 7f090ebe3700 1 -- 172.21.15.17:0/1198736232 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09080a9da0 msgr2=0x7f09080ae7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:37.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.790+0000 7f090ebe3700 1 --2- 172.21.15.17:0/1198736232 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09080a9da0 0x7f09080ae7b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f08f8002f70 tx=0x7f08f8015000).stop 2022-08-01T19:37:37.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.790+0000 7f090ebe3700 1 -- 172.21.15.17:0/1198736232 shutdown_connections 2022-08-01T19:37:37.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.790+0000 7f090ebe3700 1 --2- 172.21.15.17:0/1198736232 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f08e8053950 0x7f08e8055e10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:37.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.790+0000 7f090ebe3700 1 --2- 172.21.15.17:0/1198736232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09081434b0 0x7f0908140120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:37.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.790+0000 7f090ebe3700 1 --2- 172.21.15.17:0/1198736232 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09080b31d0 0x7f0908140ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:37.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.790+0000 7f090ebe3700 1 --2- 172.21.15.17:0/1198736232 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09080a9da0 0x7f09080ae7b0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:37.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.790+0000 7f090ebe3700 1 -- 172.21.15.17:0/1198736232 >> 172.21.15.17:0/1198736232 conn(0x7f0908063870 msgr2=0x7f09080b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:37.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.790+0000 7f090ebe3700 1 -- 172.21.15.17:0/1198736232 shutdown_connections 2022-08-01T19:37:37.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.790+0000 7f090ebe3700 1 -- 172.21.15.17:0/1198736232 wait complete. 2022-08-01T19:37:37.806 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T19:37:37.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.918+0000 7faf65f70700 1 Processor -- start 2022-08-01T19:37:37.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.918+0000 7faf65f70700 1 -- start start 2022-08-01T19:37:37.924 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.918+0000 7faf65f70700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7faf600b0630 0x7faf600adec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:37.924 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.918+0000 7faf65f70700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faf600aa2b0 0x7faf600ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:37.924 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.918+0000 7faf65f70700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf600b2c80 con 0x7faf600b0a10 2022-08-01T19:37:37.924 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.918+0000 7faf65f70700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf600b2900 con 0x7faf600aa2b0 2022-08-01T19:37:37.925 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.918+0000 7faf65f70700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf600b2eb0 con 0x7faf600b0630 2022-08-01T19:37:37.925 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.918+0000 7faf57fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faf600aa2b0 0x7faf600ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:37.925 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.918+0000 7faf57fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faf600aa2b0 0x7faf600ae3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39136/0 (socket says 172.21.15.17:39136) 2022-08-01T19:37:37.925 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.918+0000 7faf57fff700 1 -- 172.21.15.17:0/3662689128 learned_addr learned my addr 172.21.15.17:0/3662689128 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:37.926 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.918+0000 7faf5ffff700 1 --1- 172.21.15.17:0/3662689128 >> v1:172.21.15.17:6789/0 conn(0x7faf600b0a10 0x7faf601438a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59566/0 (socket says 172.21.15.17:59566) 2022-08-01T19:37:37.926 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.918+0000 7faf5f7fe700 1 --2- 172.21.15.17:0/3662689128 >> v2:172.21.15.77:3300/0 conn(0x7faf600b0630 0x7faf600adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:37.926 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.918+0000 7faf57fff700 1 -- 172.21.15.17:0/3662689128 >> v2:172.21.15.77:3300/0 conn(0x7faf600b0630 msgr2=0x7faf600adec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:37.926 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.918+0000 7faf57fff700 1 --2- 172.21.15.17:0/3662689128 >> v2:172.21.15.77:3300/0 conn(0x7faf600b0630 0x7faf600adec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:37.927 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.918+0000 7faf57fff700 1 -- 172.21.15.17:0/3662689128 >> v1:172.21.15.17:6789/0 conn(0x7faf600b0a10 legacy=0x7faf601438a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:37.927 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.918+0000 7faf5effd700 1 -- 172.21.15.17:0/3662689128 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3878968383 0 0) 0x7faf600b2c80 con 0x7faf600b0a10 2022-08-01T19:37:37.927 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.918+0000 7faf57fff700 1 -- 172.21.15.17:0/3662689128 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf600a9d00 con 0x7faf600aa2b0 2022-08-01T19:37:37.927 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.922+0000 7faf57fff700 1 --2- 172.21.15.17:0/3662689128 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf600aa2b0 0x7faf600ae3b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7faf48002f70 tx=0x7faf48006eb0).ready entity=mon.0 client_cookie=684d278f218d4f91 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:37.931 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.922+0000 7faf5effd700 1 -- 172.21.15.17:0/3662689128 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf4800d5c0 con 0x7faf600aa2b0 2022-08-01T19:37:37.931 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.922+0000 7faf5effd700 1 -- 172.21.15.17:0/3662689128 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf48000e90 con 0x7faf600aa2b0 2022-08-01T19:37:37.931 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.922+0000 7faf5effd700 1 -- 172.21.15.17:0/3662689128 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf4800d910 con 0x7faf600aa2b0 2022-08-01T19:37:37.932 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.922+0000 7faf65f70700 1 -- 172.21.15.17:0/3662689128 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf600aa2b0 msgr2=0x7faf600ae3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:37.932 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.922+0000 7faf65f70700 1 --2- 172.21.15.17:0/3662689128 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf600aa2b0 0x7faf600ae3b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7faf48002f70 tx=0x7faf48006eb0).stop 2022-08-01T19:37:37.932 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.922+0000 7faf65f70700 1 -- 172.21.15.17:0/3662689128 shutdown_connections 2022-08-01T19:37:37.932 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.922+0000 7faf65f70700 1 --2- 172.21.15.17:0/3662689128 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf600aa2b0 0x7faf600ae3b0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:37.933 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.922+0000 7faf65f70700 1 --2- 172.21.15.17:0/3662689128 >> v2:172.21.15.77:3300/0 conn(0x7faf600b0630 0x7faf600adec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:37.933 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.922+0000 7faf65f70700 1 -- 172.21.15.17:0/3662689128 >> 172.21.15.17:0/3662689128 conn(0x7faf60060950 msgr2=0x7faf60060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:37.933 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.922+0000 7faf65f70700 1 -- 172.21.15.17:0/3662689128 shutdown_connections 2022-08-01T19:37:37.933 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.922+0000 7faf65f70700 1 -- 172.21.15.17:0/3662689128 wait complete. 2022-08-01T19:37:37.934 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.922+0000 7faf65f70700 1 Processor -- start 2022-08-01T19:37:37.934 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.922+0000 7faf65f70700 1 -- start start 2022-08-01T19:37:37.934 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.926+0000 7faf65f70700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf600aa2b0 0x7faf6009fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:37.934 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.926+0000 7faf65f70700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7faf600b0630 0x7faf600ac650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:37.935 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.926+0000 7faf65f70700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf600b0a10 0x7faf60150c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:37.935 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.926+0000 7faf65f70700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf601505b0 con 0x7faf600aa2b0 2022-08-01T19:37:37.935 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.926+0000 7faf65f70700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf600a01c0 con 0x7faf600b0a10 2022-08-01T19:37:37.935 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.926+0000 7faf65f70700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf600a0460 con 0x7faf600b0630 2022-08-01T19:37:37.936 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.926+0000 7faf5f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf600aa2b0 0x7faf6009fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:37.936 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.926+0000 7faf5f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf600aa2b0 0x7faf6009fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39142/0 (socket says 172.21.15.17:39142) 2022-08-01T19:37:37.936 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.926+0000 7faf5f7fe700 1 -- 172.21.15.17:0/2486333099 learned_addr learned my addr 172.21.15.17:0/2486333099 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:37.936 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.926+0000 7faf57fff700 1 --2- 172.21.15.17:0/2486333099 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7faf600b0630 0x7faf600ac650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:37.937 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.926+0000 7faf5ffff700 1 --2- 172.21.15.17:0/2486333099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf600b0a10 0x7faf60150c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:37.937 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.926+0000 7faf5f7fe700 1 -- 172.21.15.17:0/2486333099 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7faf600b0630 msgr2=0x7faf600ac650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:37.937 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.926+0000 7faf5f7fe700 1 --2- 172.21.15.17:0/2486333099 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7faf600b0630 0x7faf600ac650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:37.938 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.926+0000 7faf5f7fe700 1 -- 172.21.15.17:0/2486333099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf600b0a10 msgr2=0x7faf60150c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:37.938 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.926+0000 7faf5f7fe700 1 --2- 172.21.15.17:0/2486333099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf600b0a10 0x7faf60150c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:37.938 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.926+0000 7faf5f7fe700 1 -- 172.21.15.17:0/2486333099 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4800b1e0 con 0x7faf600aa2b0 2022-08-01T19:37:37.938 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.926+0000 7faf5ffff700 1 --2- 172.21.15.17:0/2486333099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf600b0a10 0x7faf60150c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:37:37.939 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.926+0000 7faf5f7fe700 1 --2- 172.21.15.17:0/2486333099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf600aa2b0 0x7faf6009fb90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7faf5000c000 tx=0x7faf50006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:37.939 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.926+0000 7faf5d7fa700 1 -- 172.21.15.17:0/2486333099 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf50010780 con 0x7faf600aa2b0 2022-08-01T19:37:37.939 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.926+0000 7faf5d7fa700 1 -- 172.21.15.17:0/2486333099 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf500108e0 con 0x7faf600aa2b0 2022-08-01T19:37:37.939 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.926+0000 7faf65f70700 1 -- 172.21.15.17:0/2486333099 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf601507a0 con 0x7faf600aa2b0 2022-08-01T19:37:37.940 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.926+0000 7faf5d7fa700 1 -- 172.21.15.17:0/2486333099 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf50010c40 con 0x7faf600aa2b0 2022-08-01T19:37:37.940 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.926+0000 7faf65f70700 1 -- 172.21.15.17:0/2486333099 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf601451e0 con 0x7faf600aa2b0 2022-08-01T19:37:37.940 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.926+0000 7faf65f70700 1 -- 172.21.15.17:0/2486333099 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf60150930 con 0x7faf600aa2b0 2022-08-01T19:37:37.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.930+0000 7faf5d7fa700 1 -- 172.21.15.17:0/2486333099 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7faf5001c460 con 0x7faf600aa2b0 2022-08-01T19:37:37.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.930+0000 7faf5d7fa700 1 --2- 172.21.15.17:0/2486333099 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7faf440539e0 0x7faf44055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:37.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.930+0000 7faf5d7fa700 1 -- 172.21.15.17:0/2486333099 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6420+0+0 (secure 0 0 0) 0x7faf5006c540 con 0x7faf600aa2b0 2022-08-01T19:37:37.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.930+0000 7faf57fff700 1 --2- 172.21.15.17:0/2486333099 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7faf440539e0 0x7faf44055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:37.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.930+0000 7faf57fff700 1 --2- 172.21.15.17:0/2486333099 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7faf440539e0 0x7faf44055ea0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7faf4801a000 tx=0x7faf48015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:37.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:37.938+0000 7faf5d7fa700 1 -- 172.21.15.17:0/2486333099 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7faf500432a0 con 0x7faf600aa2b0 2022-08-01T19:37:38.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.098+0000 7faf65f70700 1 -- 172.21.15.17:0/2486333099 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7faf600a0620 con 0x7faf600aa2b0 2022-08-01T19:37:38.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.106+0000 7faf5d7fa700 1 -- 172.21.15.17:0/2486333099 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v19) v1 ==== 184+0+0 (secure 0 0 0) 0x7faf500468c0 con 0x7faf600aa2b0 2022-08-01T19:37:38.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.106+0000 7faf65f70700 1 -- 172.21.15.17:0/2486333099 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7faf440539e0 msgr2=0x7faf44055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:38.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.106+0000 7faf65f70700 1 --2- 172.21.15.17:0/2486333099 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7faf440539e0 0x7faf44055ea0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7faf4801a000 tx=0x7faf48015000).stop 2022-08-01T19:37:38.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.106+0000 7faf65f70700 1 -- 172.21.15.17:0/2486333099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf600aa2b0 msgr2=0x7faf6009fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:38.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.106+0000 7faf65f70700 1 --2- 172.21.15.17:0/2486333099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf600aa2b0 0x7faf6009fb90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7faf5000c000 tx=0x7faf50006df0).stop 2022-08-01T19:37:38.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.110+0000 7faf65f70700 1 -- 172.21.15.17:0/2486333099 shutdown_connections 2022-08-01T19:37:38.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.110+0000 7faf65f70700 1 --2- 172.21.15.17:0/2486333099 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7faf440539e0 0x7faf44055ea0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:38.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.110+0000 7faf65f70700 1 --2- 172.21.15.17:0/2486333099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf600b0a10 0x7faf60150c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:38.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.110+0000 7faf65f70700 1 --2- 172.21.15.17:0/2486333099 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7faf600b0630 0x7faf600ac650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:38.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.110+0000 7faf65f70700 1 --2- 172.21.15.17:0/2486333099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf600aa2b0 0x7faf6009fb90 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:38.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.110+0000 7faf65f70700 1 -- 172.21.15.17:0/2486333099 >> 172.21.15.17:0/2486333099 conn(0x7faf60060950 msgr2=0x7faf600b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:38.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.110+0000 7faf65f70700 1 -- 172.21.15.17:0/2486333099 shutdown_connections 2022-08-01T19:37:38.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.110+0000 7faf65f70700 1 -- 172.21.15.17:0/2486333099 wait complete. 2022-08-01T19:37:38.116 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.0 2022-08-01T19:37:38.127 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T19:37:38.234 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.230+0000 7f2929e60700 1 Processor -- start 2022-08-01T19:37:38.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.230+0000 7f2929e60700 1 -- start start 2022-08-01T19:37:38.236 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.234+0000 7f2929e60700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f29240b6510 0x7f2924140500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:38.236 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.234+0000 7f2929e60700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f29240b2aa0 0x7f29241409f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:38.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.234+0000 7f2929e60700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29241467f0 con 0x7f2924061480 2022-08-01T19:37:38.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.234+0000 7f2929e60700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2924062140 con 0x7f29240b2aa0 2022-08-01T19:37:38.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.234+0000 7f2929e60700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29240656b0 con 0x7f29240b6510 2022-08-01T19:37:38.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.234+0000 7f292965f700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2924061480 0x7f29240b0e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59580/0 (socket says 172.21.15.17:59580) 2022-08-01T19:37:38.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.234+0000 7f292965f700 1 -- 172.21.15.17:0/2607545918 learned_addr learned my addr 172.21.15.17:0/2607545918 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:37:38.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.234+0000 7f2928e5e700 1 --2- 172.21.15.17:0/2607545918 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f29240b6510 0x7f2924140500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:38.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.234+0000 7f2923fff700 1 --2- 172.21.15.17:0/2607545918 >> v2:172.21.15.77:3300/0 conn(0x7f29240b2aa0 0x7f29241409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:38.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.234+0000 7f29237fe700 1 -- 172.21.15.17:0/2607545918 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2611679354 0 0) 0x7f29241467f0 con 0x7f2924061480 2022-08-01T19:37:38.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.234+0000 7f29237fe700 1 -- 172.21.15.17:0/2607545918 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2910003260 con 0x7f2924061480 2022-08-01T19:37:38.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.234+0000 7f2923fff700 1 -- 172.21.15.17:0/2607545918 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f29240b6510 msgr2=0x7f2924140500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:38.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.234+0000 7f2923fff700 1 --2- 172.21.15.17:0/2607545918 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f29240b6510 0x7f2924140500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:38.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.234+0000 7f29237fe700 1 -- 172.21.15.17:0/2607545918 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1022309109 0 0) 0x7f2910003260 con 0x7f2924061480 2022-08-01T19:37:38.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.234+0000 7f2923fff700 1 -- 172.21.15.17:0/2607545918 >> v1:172.21.15.17:6789/0 conn(0x7f2924061480 legacy=0x7f29240b0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:38.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.234+0000 7f2923fff700 1 -- 172.21.15.17:0/2607545918 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29241476a0 con 0x7f29240b2aa0 2022-08-01T19:37:38.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.234+0000 7f2923fff700 1 --2- 172.21.15.17:0/2607545918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29240b2aa0 0x7f29241409f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f291c001b80 tx=0x7f2914006df0).ready entity=mon.1 client_cookie=6660974c0559513a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:38.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.234+0000 7f29237fe700 1 -- 172.21.15.17:0/2607545918 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f291c0019e0 con 0x7f29240b2aa0 2022-08-01T19:37:38.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.234+0000 7f29237fe700 1 -- 172.21.15.17:0/2607545918 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2914000e90 con 0x7f29240b2aa0 2022-08-01T19:37:38.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.234+0000 7f29237fe700 1 -- 172.21.15.17:0/2607545918 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2914012460 con 0x7f29240b2aa0 2022-08-01T19:37:38.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.234+0000 7f2929e60700 1 -- 172.21.15.17:0/2607545918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29240b2aa0 msgr2=0x7f29241409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:38.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.234+0000 7f2929e60700 1 --2- 172.21.15.17:0/2607545918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29240b2aa0 0x7f29241409f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f291c001b80 tx=0x7f2914006df0).stop 2022-08-01T19:37:38.246 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.234+0000 7f2929e60700 1 -- 172.21.15.17:0/2607545918 shutdown_connections 2022-08-01T19:37:38.246 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.234+0000 7f2929e60700 1 --2- 172.21.15.17:0/2607545918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29240b2aa0 0x7f29241409f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:38.246 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.234+0000 7f2929e60700 1 --2- 172.21.15.17:0/2607545918 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f29240b6510 0x7f2924140500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:38.246 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.234+0000 7f2929e60700 1 -- 172.21.15.17:0/2607545918 >> 172.21.15.17:0/2607545918 conn(0x7f2924063800 msgr2=0x7f2924063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:38.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.234+0000 7f2929e60700 1 -- 172.21.15.17:0/2607545918 shutdown_connections 2022-08-01T19:37:38.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.234+0000 7f2929e60700 1 -- 172.21.15.17:0/2607545918 wait complete. 2022-08-01T19:37:38.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.238+0000 7f2929e60700 1 Processor -- start 2022-08-01T19:37:38.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.238+0000 7f2929e60700 1 -- start start 2022-08-01T19:37:38.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.238+0000 7f2929e60700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2924061480 0x7f2924142200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:38.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.238+0000 7f2929e60700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29240b2aa0 0x7f2924141720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:38.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.238+0000 7f2929e60700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f29240b6510 0x7f2924145df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:38.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.238+0000 7f2929e60700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2924146810 con 0x7f2924061480 2022-08-01T19:37:38.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.238+0000 7f2929e60700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29241462e0 con 0x7f29240b2aa0 2022-08-01T19:37:38.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.238+0000 7f2929e60700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2924146570 con 0x7f29240b6510 2022-08-01T19:37:38.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.238+0000 7f2928e5e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2924061480 0x7f2924142200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:38.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.238+0000 7f2928e5e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2924061480 0x7f2924142200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39156/0 (socket says 172.21.15.17:39156) 2022-08-01T19:37:38.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.238+0000 7f2928e5e700 1 -- 172.21.15.17:0/3191831581 learned_addr learned my addr 172.21.15.17:0/3191831581 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:38.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.238+0000 7f2923fff700 1 --2- 172.21.15.17:0/3191831581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29240b2aa0 0x7f2924141720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:38.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.238+0000 7f292965f700 1 --2- 172.21.15.17:0/3191831581 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f29240b6510 0x7f2924145df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:38.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.238+0000 7f2928e5e700 1 -- 172.21.15.17:0/3191831581 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f29240b6510 msgr2=0x7f2924145df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:38.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.238+0000 7f2928e5e700 1 --2- 172.21.15.17:0/3191831581 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f29240b6510 0x7f2924145df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:38.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.238+0000 7f2928e5e700 1 -- 172.21.15.17:0/3191831581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29240b2aa0 msgr2=0x7f2924141720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:38.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.238+0000 7f2928e5e700 1 --2- 172.21.15.17:0/3191831581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29240b2aa0 0x7f2924141720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:38.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.238+0000 7f2928e5e700 1 -- 172.21.15.17:0/3191831581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f291400b180 con 0x7f2924061480 2022-08-01T19:37:38.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.238+0000 7f2928e5e700 1 --2- 172.21.15.17:0/3191831581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2924061480 0x7f2924142200 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f291800c000 tx=0x7f2918006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:38.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.238+0000 7f2921ffb700 1 -- 172.21.15.17:0/3191831581 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2918011750 con 0x7f2924061480 2022-08-01T19:37:38.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.238+0000 7f2921ffb700 1 -- 172.21.15.17:0/3191831581 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29180118b0 con 0x7f2924061480 2022-08-01T19:37:38.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.238+0000 7f2929e60700 1 -- 172.21.15.17:0/3191831581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2924152430 con 0x7f2924061480 2022-08-01T19:37:38.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.238+0000 7f2921ffb700 1 -- 172.21.15.17:0/3191831581 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2918011c10 con 0x7f2924061480 2022-08-01T19:37:38.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.238+0000 7f2929e60700 1 -- 172.21.15.17:0/3191831581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2924143a70 con 0x7f2924061480 2022-08-01T19:37:38.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.242+0000 7f2929e60700 1 -- 172.21.15.17:0/3191831581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29241528d0 con 0x7f2924061480 2022-08-01T19:37:38.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.242+0000 7f2921ffb700 1 -- 172.21.15.17:0/3191831581 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2918011d70 con 0x7f2924061480 2022-08-01T19:37:38.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.242+0000 7f2921ffb700 1 --2- 172.21.15.17:0/3191831581 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2904053950 0x7f2904055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:38.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.242+0000 7f2921ffb700 1 -- 172.21.15.17:0/3191831581 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f291806c490 con 0x7f2924061480 2022-08-01T19:37:38.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.242+0000 7f2923fff700 1 --2- 172.21.15.17:0/3191831581 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2904053950 0x7f2904055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:38.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.242+0000 7f2923fff700 1 --2- 172.21.15.17:0/3191831581 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2904053950 0x7f2904055e10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f291401a000 tx=0x7f2914014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:38.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.254+0000 7f2921ffb700 1 -- 172.21.15.17:0/3191831581 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f29241528d0 con 0x7f2924061480 2022-08-01T19:37:38.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.410+0000 7f2929e60700 1 -- 172.21.15.17:0/3191831581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f29241528d0 con 0x7f2924061480 2022-08-01T19:37:38.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.418+0000 7f2921ffb700 1 -- 172.21.15.17:0/3191831581 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v20) v1 ==== 184+0+0 (secure 0 0 0) 0x7f29241528d0 con 0x7f2924061480 2022-08-01T19:37:38.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.418+0000 7f2929e60700 1 -- 172.21.15.17:0/3191831581 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2904053950 msgr2=0x7f2904055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:38.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.418+0000 7f2929e60700 1 --2- 172.21.15.17:0/3191831581 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2904053950 0x7f2904055e10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f291401a000 tx=0x7f2914014000).stop 2022-08-01T19:37:38.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.418+0000 7f2929e60700 1 -- 172.21.15.17:0/3191831581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2924061480 msgr2=0x7f2924142200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:38.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.418+0000 7f2929e60700 1 --2- 172.21.15.17:0/3191831581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2924061480 0x7f2924142200 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f291800c000 tx=0x7f2918006df0).stop 2022-08-01T19:37:38.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.418+0000 7f2929e60700 1 -- 172.21.15.17:0/3191831581 shutdown_connections 2022-08-01T19:37:38.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.418+0000 7f2929e60700 1 --2- 172.21.15.17:0/3191831581 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2904053950 0x7f2904055e10 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:38.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.418+0000 7f2929e60700 1 --2- 172.21.15.17:0/3191831581 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f29240b6510 0x7f2924145df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:38.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.418+0000 7f2929e60700 1 --2- 172.21.15.17:0/3191831581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29240b2aa0 0x7f2924141720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:38.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.418+0000 7f2929e60700 1 --2- 172.21.15.17:0/3191831581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2924061480 0x7f2924142200 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:38.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.418+0000 7f2929e60700 1 -- 172.21.15.17:0/3191831581 >> 172.21.15.17:0/3191831581 conn(0x7f2924063800 msgr2=0x7f29240b4f90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:38.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.418+0000 7f2929e60700 1 -- 172.21.15.17:0/3191831581 shutdown_connections 2022-08-01T19:37:38.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.418+0000 7f2929e60700 1 -- 172.21.15.17:0/3191831581 wait complete. 2022-08-01T19:37:38.431 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.1 2022-08-01T19:37:38.439 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:37:38.547 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.542+0000 7fe9cbfff700 1 Processor -- start 2022-08-01T19:37:38.547 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.542+0000 7fe9cbfff700 1 -- start start 2022-08-01T19:37:38.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.542+0000 7fe9cbfff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe9cc0b3190 0x7fe9cc060ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:38.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.542+0000 7fe9cbfff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe9cc061190 0x7fe9cc066dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:38.550 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.542+0000 7fe9cbfff700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9cc0654c0 con 0x7fe9cc0b2730 2022-08-01T19:37:38.550 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.542+0000 7fe9cbfff700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9cc062190 con 0x7fe9cc0b3190 2022-08-01T19:37:38.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.542+0000 7fe9cbfff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc0615d0 con 0x7fe9cc061190 2022-08-01T19:37:38.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.542+0000 7fe9caffd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe9cc0b3190 0x7fe9cc060ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:38.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.542+0000 7fe9caffd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe9cc0b3190 0x7fe9cc060ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39166/0 (socket says 172.21.15.17:39166) 2022-08-01T19:37:38.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.542+0000 7fe9caffd700 1 -- 172.21.15.17:0/3991831112 learned_addr learned my addr 172.21.15.17:0/3991831112 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:38.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.542+0000 7fe9cb7fe700 1 --2- 172.21.15.17:0/3991831112 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe9cc061190 0x7fe9cc066dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:38.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.546+0000 7fe9c9ffb700 1 -- 172.21.15.17:0/3991831112 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1431838613 0 0) 0x7fe9cc0654c0 con 0x7fe9cc0b2730 2022-08-01T19:37:38.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.546+0000 7fe9caffd700 1 -- 172.21.15.17:0/3991831112 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe9cc061190 msgr2=0x7fe9cc066dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:38.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.546+0000 7fe9caffd700 1 --2- 172.21.15.17:0/3991831112 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe9cc061190 0x7fe9cc066dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:38.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.546+0000 7fe9caffd700 1 -- 172.21.15.17:0/3991831112 >> v1:172.21.15.77:6789/0 conn(0x7fe9cc0b2730 legacy=0x7fe9cc0666b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:38.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.546+0000 7fe9caffd700 1 -- 172.21.15.17:0/3991831112 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9cc0b3570 con 0x7fe9cc0b3190 2022-08-01T19:37:38.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.546+0000 7fe9caffd700 1 --2- 172.21.15.17:0/3991831112 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9cc0b3190 0x7fe9cc060ca0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7fe9cc061710 tx=0x7fe9c0006eb0).ready entity=mon.0 client_cookie=1b7d144c738b6176 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:38.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.546+0000 7fe9c9ffb700 1 -- 172.21.15.17:0/3991831112 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9c000d5c0 con 0x7fe9cc0b3190 2022-08-01T19:37:38.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.546+0000 7fe9c9ffb700 1 -- 172.21.15.17:0/3991831112 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c0000e90 con 0x7fe9cc0b3190 2022-08-01T19:37:38.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.546+0000 7fe9c9ffb700 1 -- 172.21.15.17:0/3991831112 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9c000d910 con 0x7fe9cc0b3190 2022-08-01T19:37:38.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.546+0000 7fe9cbfff700 1 -- 172.21.15.17:0/3991831112 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9cc0b3190 msgr2=0x7fe9cc060ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:38.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.546+0000 7fe9cbfff700 1 --2- 172.21.15.17:0/3991831112 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9cc0b3190 0x7fe9cc060ca0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7fe9cc061710 tx=0x7fe9c0006eb0).stop 2022-08-01T19:37:38.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.546+0000 7fe9cbfff700 1 -- 172.21.15.17:0/3991831112 shutdown_connections 2022-08-01T19:37:38.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.546+0000 7fe9cbfff700 1 --2- 172.21.15.17:0/3991831112 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe9cc061190 0x7fe9cc066dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:38.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.546+0000 7fe9cbfff700 1 --2- 172.21.15.17:0/3991831112 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9cc0b3190 0x7fe9cc060ca0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:38.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.546+0000 7fe9cbfff700 1 -- 172.21.15.17:0/3991831112 >> 172.21.15.17:0/3991831112 conn(0x7fe9cc063870 msgr2=0x7fe9cc063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:38.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.546+0000 7fe9cbfff700 1 -- 172.21.15.17:0/3991831112 shutdown_connections 2022-08-01T19:37:38.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.546+0000 7fe9cbfff700 1 -- 172.21.15.17:0/3991831112 wait complete. 2022-08-01T19:37:38.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.546+0000 7fe9cbfff700 1 Processor -- start 2022-08-01T19:37:38.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.546+0000 7fe9cbfff700 1 -- start start 2022-08-01T19:37:38.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.550+0000 7fe9cbfff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe9cc061190 0x7fe9cc148630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:38.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.550+0000 7fe9cbfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9cc0b2730 0x7fe9cc0ab9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:38.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.550+0000 7fe9cbfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9cc0b3190 0x7fe9cc1415b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:38.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.550+0000 7fe9cbfff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc0af500 con 0x7fe9cc0b2730 2022-08-01T19:37:38.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.550+0000 7fe9cbfff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc0abe90 con 0x7fe9cc0b3190 2022-08-01T19:37:38.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.550+0000 7fe9cbfff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc0af730 con 0x7fe9cc061190 2022-08-01T19:37:38.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.550+0000 7fe9ca7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9cc0b2730 0x7fe9cc0ab9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:38.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.550+0000 7fe9ca7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9cc0b2730 0x7fe9cc0ab9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39174/0 (socket says 172.21.15.17:39174) 2022-08-01T19:37:38.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.550+0000 7fe9ca7fc700 1 -- 172.21.15.17:0/3208447015 learned_addr learned my addr 172.21.15.17:0/3208447015 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:38.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.550+0000 7fe9cb7fe700 1 --2- 172.21.15.17:0/3208447015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9cc0b3190 0x7fe9cc1415b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:38.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.550+0000 7fe9caffd700 1 --2- 172.21.15.17:0/3208447015 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe9cc061190 0x7fe9cc148630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:38.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.550+0000 7fe9ca7fc700 1 -- 172.21.15.17:0/3208447015 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe9cc061190 msgr2=0x7fe9cc148630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:38.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.550+0000 7fe9ca7fc700 1 --2- 172.21.15.17:0/3208447015 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe9cc061190 0x7fe9cc148630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:38.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.550+0000 7fe9ca7fc700 1 -- 172.21.15.17:0/3208447015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9cc0b3190 msgr2=0x7fe9cc1415b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:38.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.550+0000 7fe9ca7fc700 1 --2- 172.21.15.17:0/3208447015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9cc0b3190 0x7fe9cc1415b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:38.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.550+0000 7fe9ca7fc700 1 -- 172.21.15.17:0/3208447015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c000b1e0 con 0x7fe9cc0b2730 2022-08-01T19:37:38.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.550+0000 7fe9ca7fc700 1 --2- 172.21.15.17:0/3208447015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9cc0b2730 0x7fe9cc0ab9a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7fe9bc000ed0 tx=0x7fe9bc007a90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:38.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.550+0000 7fe9bbfff700 1 -- 172.21.15.17:0/3208447015 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9bc012400 con 0x7fe9cc0b2730 2022-08-01T19:37:38.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.550+0000 7fe9cbfff700 1 -- 172.21.15.17:0/3208447015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9cc0af9d0 con 0x7fe9cc0b2730 2022-08-01T19:37:38.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.550+0000 7fe9cbfff700 1 -- 172.21.15.17:0/3208447015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9cc0adca0 con 0x7fe9cc0b2730 2022-08-01T19:37:38.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.550+0000 7fe9bbfff700 1 -- 172.21.15.17:0/3208447015 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9bc012560 con 0x7fe9cc0b2730 2022-08-01T19:37:38.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.550+0000 7fe9bbfff700 1 -- 172.21.15.17:0/3208447015 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9bc012900 con 0x7fe9cc0b2730 2022-08-01T19:37:38.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.550+0000 7fe9cbfff700 1 -- 172.21.15.17:0/3208447015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9cc05fbf0 con 0x7fe9cc0b2730 2022-08-01T19:37:38.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.554+0000 7fe9bbfff700 1 -- 172.21.15.17:0/3208447015 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe9bc010070 con 0x7fe9cc0b2730 2022-08-01T19:37:38.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.554+0000 7fe9bbfff700 1 --2- 172.21.15.17:0/3208447015 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe9a8053b30 0x7fe9a8055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:38.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.554+0000 7fe9bbfff700 1 -- 172.21.15.17:0/3208447015 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe9bc06b880 con 0x7fe9cc0b2730 2022-08-01T19:37:38.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.554+0000 7fe9caffd700 1 --2- 172.21.15.17:0/3208447015 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe9a8053b30 0x7fe9a8055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:38.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.554+0000 7fe9caffd700 1 --2- 172.21.15.17:0/3208447015 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe9a8053b30 0x7fe9a8055ff0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7fe9c001b000 tx=0x7fe9c0015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:38.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.558+0000 7fe9bbfff700 1 -- 172.21.15.17:0/3208447015 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe9bc0425e0 con 0x7fe9cc0b2730 2022-08-01T19:37:38.574 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2022-08-01T19:37:38.575 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2022-08-01T19:37:38.575 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2022-08-01T19:37:38.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.718+0000 7fe9cbfff700 1 -- 172.21.15.17:0/3208447015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9cc0574e0 con 0x7fe9cc0b2730 2022-08-01T19:37:38.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.718+0000 7fe9bbfff700 1 -- 172.21.15.17:0/3208447015 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+2492 (secure 0 0 0) 0x7fe9bc007bb0 con 0x7fe9cc0b2730 2022-08-01T19:37:38.724 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:37:38.724 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":24,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/1914125901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":1914125901},{"type":"v1","addr":"172.21.15.17:6833","nonce":1914125901}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":22},{"gid":5587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1403688289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1403688289},{"type":"v1","addr":"172.21.15.77:6835","nonce":1403688289}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":22}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:37:36.197818+0000","modified":"2022-08-01T19:37:37.201700+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5589":{"gid":5589,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/3325991310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":3325991310},{"type":"v1","addr":"172.21.15.17:6835","nonce":3325991310}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-08-01T19:37:38.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.722+0000 7fe9cbfff700 1 -- 172.21.15.17:0/3208447015 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe9a8053b30 msgr2=0x7fe9a8055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:38.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.722+0000 7fe9cbfff700 1 --2- 172.21.15.17:0/3208447015 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe9a8053b30 0x7fe9a8055ff0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7fe9c001b000 tx=0x7fe9c0015000).stop 2022-08-01T19:37:38.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.722+0000 7fe9cbfff700 1 -- 172.21.15.17:0/3208447015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9cc0b2730 msgr2=0x7fe9cc0ab9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:38.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.722+0000 7fe9cbfff700 1 --2- 172.21.15.17:0/3208447015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9cc0b2730 0x7fe9cc0ab9a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7fe9bc000ed0 tx=0x7fe9bc007a90).stop 2022-08-01T19:37:38.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.722+0000 7fe9cbfff700 1 -- 172.21.15.17:0/3208447015 shutdown_connections 2022-08-01T19:37:38.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.722+0000 7fe9cbfff700 1 --2- 172.21.15.17:0/3208447015 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe9a8053b30 0x7fe9a8055ff0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:38.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.722+0000 7fe9cbfff700 1 --2- 172.21.15.17:0/3208447015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9cc0b3190 0x7fe9cc1415b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:38.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.722+0000 7fe9cbfff700 1 --2- 172.21.15.17:0/3208447015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9cc0b2730 0x7fe9cc0ab9a0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:38.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.722+0000 7fe9cbfff700 1 --2- 172.21.15.17:0/3208447015 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe9cc061190 0x7fe9cc148630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:38.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.722+0000 7fe9cbfff700 1 -- 172.21.15.17:0/3208447015 >> 172.21.15.17:0/3208447015 conn(0x7fe9cc063870 msgr2=0x7fe9cc066510 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:38.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.722+0000 7fe9cbfff700 1 -- 172.21.15.17:0/3208447015 shutdown_connections 2022-08-01T19:37:38.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:38.722+0000 7fe9cbfff700 1 -- 172.21.15.17:0/3208447015 wait complete. 2022-08-01T19:37:38.729 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 24 2022-08-01T19:37:39.040 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:37:36.197818+0000', 'modified': '2022-08-01T19:37:37.201700+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5589}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5589': {'gid': 5589, 'name': 'c', 'rank': 0, 'incarnation': 23, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/3325991310', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 3325991310}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 3325991310}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:37:39.041 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:37:39.041 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T19:37:39.162 INFO:teuthology.orchestra.run.smithi017.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":5589,"want_state":"up:active","state":"up:active","rank_uptime":2.9537422929999999,"mdsmap_epoch":24,"osdmap_epoch":55,"osdmap_epoch_barrier":55,"uptime":7.3147786909999999} 2022-08-01T19:37:39.162 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 5589, "mdsmap_epoch": 24, "osdmap_epoch": 55, "osdmap_epoch_barrier": 55, "rank_uptime": 2.953742293, "state": "up:active", "uptime": 7.314778691, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:37:39.163 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:37:39.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.258+0000 7f32a5371700 1 Processor -- start 2022-08-01T19:37:39.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.258+0000 7f32a5371700 1 -- start start 2022-08-01T19:37:39.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f32a5371700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f32a00b2710 0x7f32a014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:39.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f32a5371700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f32a00b01a0 0x7f32a014b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:39.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f32a5371700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32a00a7250 con 0x7f32a00afd60 2022-08-01T19:37:39.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f32a5371700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32a00a7480 con 0x7f32a00b2710 2022-08-01T19:37:39.272 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f32a5371700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a00a7720 con 0x7f32a00b01a0 2022-08-01T19:37:39.272 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f32a4b70700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f32a00afd60 0x7f32a00a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59608/0 (socket says 172.21.15.17:59608) 2022-08-01T19:37:39.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f32a4b70700 1 -- 172.21.15.17:0/728314874 learned_addr learned my addr 172.21.15.17:0/728314874 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:37:39.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f329f7fe700 1 --2- 172.21.15.17:0/728314874 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f32a00b01a0 0x7f32a014b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:39.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f329ffff700 1 --2- 172.21.15.17:0/728314874 >> v2:172.21.15.77:3300/0 conn(0x7f32a00b2710 0x7f32a014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:39.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f329effd700 1 -- 172.21.15.17:0/728314874 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3474816958 0 0) 0x7f32a00a7250 con 0x7f32a00afd60 2022-08-01T19:37:39.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f329effd700 1 -- 172.21.15.17:0/728314874 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f328c003260 con 0x7f32a00afd60 2022-08-01T19:37:39.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f329effd700 1 -- 172.21.15.17:0/728314874 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2388564073 0 0) 0x7f328c003260 con 0x7f32a00afd60 2022-08-01T19:37:39.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f329effd700 1 -- 172.21.15.17:0/728314874 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f32a00b01a0 msgr2=0x7f32a014b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:39.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f329effd700 1 --2- 172.21.15.17:0/728314874 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f32a00b01a0 0x7f32a014b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:39.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f329effd700 1 -- 172.21.15.17:0/728314874 >> v2:172.21.15.77:3300/0 conn(0x7f32a00b2710 msgr2=0x7f32a014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:39.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f329effd700 1 --2- 172.21.15.17:0/728314874 >> v2:172.21.15.77:3300/0 conn(0x7f32a00b2710 0x7f32a014afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:39.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f329effd700 1 -- 172.21.15.17:0/728314874 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a00a79c0 con 0x7f32a00afd60 2022-08-01T19:37:39.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f329effd700 1 -- 172.21.15.17:0/728314874 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7f32980019e0 con 0x7f32a00afd60 2022-08-01T19:37:39.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f329effd700 1 -- 172.21.15.17:0/728314874 >> v1:172.21.15.17:6789/0 conn(0x7f32a00afd60 legacy=0x7f32a00a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:39.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f329effd700 1 --2- 172.21.15.17:0/728314874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f328c003750 0x7f328c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:39.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f329effd700 1 --2- 172.21.15.17:0/728314874 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f328c004500 0x7f328c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:39.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f329effd700 1 --2- 172.21.15.17:0/728314874 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f32a00b01a0 0x7f328c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:39.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f329effd700 1 -- 172.21.15.17:0/728314874 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a00a7480 con 0x7f328c004500 2022-08-01T19:37:39.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f329effd700 1 -- 172.21.15.17:0/728314874 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a00a7720 con 0x7f328c003750 2022-08-01T19:37:39.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f329effd700 1 -- 172.21.15.17:0/728314874 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f328c00b750 con 0x7f32a00b01a0 2022-08-01T19:37:39.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f329f7fe700 1 --2- 172.21.15.17:0/728314874 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f328c004500 0x7f328c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:39.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f32a4b70700 1 --2- 172.21.15.17:0/728314874 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f32a00b01a0 0x7f328c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:39.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f329f7fe700 1 -- 172.21.15.17:0/728314874 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f32a00b01a0 msgr2=0x7f328c004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:39.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f329f7fe700 1 --2- 172.21.15.17:0/728314874 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f32a00b01a0 0x7f328c004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:39.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f329f7fe700 1 -- 172.21.15.17:0/728314874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f328c003750 msgr2=0x7f328c003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:39.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f329f7fe700 1 --2- 172.21.15.17:0/728314874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f328c003750 0x7f328c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2022-08-01T19:37:39.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f329f7fe700 1 -- 172.21.15.17:0/728314874 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328c003260 con 0x7f328c004500 2022-08-01T19:37:39.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f32a4b70700 1 --2- 172.21.15.17:0/728314874 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f32a00b01a0 0x7f328c004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:37:39.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f329f7fe700 1 --2- 172.21.15.17:0/728314874 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f328c004500 0x7f328c004980 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f329000c000 tx=0x7f3290006df0).ready entity=mon.0 client_cookie=c72b2be129f22ed3 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:39.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f329effd700 1 -- 172.21.15.17:0/728314874 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f329000e830 con 0x7f328c004500 2022-08-01T19:37:39.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f329effd700 1 -- 172.21.15.17:0/728314874 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329000e990 con 0x7f328c004500 2022-08-01T19:37:39.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f329effd700 1 -- 172.21.15.17:0/728314874 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f329000ecf0 con 0x7f328c004500 2022-08-01T19:37:39.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f32a5371700 1 -- 172.21.15.17:0/728314874 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f328c004500 msgr2=0x7f328c004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:39.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f32a5371700 1 --2- 172.21.15.17:0/728314874 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f328c004500 0x7f328c004980 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f329000c000 tx=0x7f3290006df0).stop 2022-08-01T19:37:39.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f32a4b70700 1 -- 172.21.15.17:0/728314874 reap_dead start 2022-08-01T19:37:39.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f32a5371700 1 -- 172.21.15.17:0/728314874 shutdown_connections 2022-08-01T19:37:39.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f32a5371700 1 -- 172.21.15.17:0/728314874 >> 172.21.15.17:0/728314874 conn(0x7f32a0060310 msgr2=0x7f32a00b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:39.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f32a5371700 1 -- 172.21.15.17:0/728314874 shutdown_connections 2022-08-01T19:37:39.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.262+0000 7f32a5371700 1 -- 172.21.15.17:0/728314874 wait complete. 2022-08-01T19:37:39.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.266+0000 7f32a5371700 1 Processor -- start 2022-08-01T19:37:39.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.266+0000 7f32a5371700 1 -- start start 2022-08-01T19:37:39.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.266+0000 7f32a5371700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f328c004500 0x7f32a014cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:39.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.266+0000 7f32a5371700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32a0064230 0x7f32a0146630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:39.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.266+0000 7f32a5371700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f32a0146b20 0x7f32a0159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:39.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.266+0000 7f32a5371700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a00b5c90 con 0x7f32a0064230 2022-08-01T19:37:39.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.266+0000 7f32a5371700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a00b5a30 con 0x7f328c004500 2022-08-01T19:37:39.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.266+0000 7f32a5371700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a00b5180 con 0x7f32a0146b20 2022-08-01T19:37:39.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.266+0000 7f329f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32a0064230 0x7f32a0146630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:39.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.266+0000 7f329f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32a0064230 0x7f32a0146630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39192/0 (socket says 172.21.15.17:39192) 2022-08-01T19:37:39.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.266+0000 7f329f7fe700 1 -- 172.21.15.17:0/1538771867 learned_addr learned my addr 172.21.15.17:0/1538771867 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:39.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.266+0000 7f32a4b70700 1 --2- 172.21.15.17:0/1538771867 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f32a0146b20 0x7f32a0159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:39.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.266+0000 7f329ffff700 1 --2- 172.21.15.17:0/1538771867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f328c004500 0x7f32a014cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:39.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.266+0000 7f329f7fe700 1 -- 172.21.15.17:0/1538771867 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f32a0146b20 msgr2=0x7f32a0159f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:39.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.266+0000 7f329f7fe700 1 --2- 172.21.15.17:0/1538771867 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f32a0146b20 0x7f32a0159f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:39.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.266+0000 7f329f7fe700 1 -- 172.21.15.17:0/1538771867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f328c004500 msgr2=0x7f32a014cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:39.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.266+0000 7f329f7fe700 1 --2- 172.21.15.17:0/1538771867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f328c004500 0x7f32a014cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:39.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.266+0000 7f329f7fe700 1 -- 172.21.15.17:0/1538771867 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a0063c40 con 0x7f32a0064230 2022-08-01T19:37:39.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.266+0000 7f329f7fe700 1 --2- 172.21.15.17:0/1538771867 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32a0064230 0x7f32a0146630 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f329000c000 tx=0x7f3290019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:39.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.266+0000 7f329d7fa700 1 -- 172.21.15.17:0/1538771867 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f329001b820 con 0x7f32a0064230 2022-08-01T19:37:39.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.266+0000 7f329d7fa700 1 -- 172.21.15.17:0/1538771867 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3290000e90 con 0x7f32a0064230 2022-08-01T19:37:39.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.266+0000 7f32a5371700 1 -- 172.21.15.17:0/1538771867 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32a00b5420 con 0x7f32a0064230 2022-08-01T19:37:39.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.266+0000 7f329d7fa700 1 -- 172.21.15.17:0/1538771867 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f329001bb70 con 0x7f32a0064230 2022-08-01T19:37:39.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.266+0000 7f32a5371700 1 -- 172.21.15.17:0/1538771867 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32a00b4210 con 0x7f32a0064230 2022-08-01T19:37:39.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.270+0000 7f329d7fa700 1 -- 172.21.15.17:0/1538771867 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f329002d070 con 0x7f32a0064230 2022-08-01T19:37:39.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.270+0000 7f32a5371700 1 -- 172.21.15.17:0/1538771867 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32a0063e00 con 0x7f32a0064230 2022-08-01T19:37:39.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.270+0000 7f329d7fa700 1 --2- 172.21.15.17:0/1538771867 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3280053b30 0x7f3280055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:39.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.270+0000 7f329d7fa700 1 -- 172.21.15.17:0/1538771867 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f329006d6c0 con 0x7f32a0064230 2022-08-01T19:37:39.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.270+0000 7f329ffff700 1 --2- 172.21.15.17:0/1538771867 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3280053b30 0x7f3280055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:39.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.270+0000 7f329ffff700 1 --2- 172.21.15.17:0/1538771867 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3280053b30 0x7f3280055ff0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f329400b000 tx=0x7f3294008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:39.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.274+0000 7f329d7fa700 1 -- 172.21.15.17:0/1538771867 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3290044480 con 0x7f32a0064230 2022-08-01T19:37:39.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.402+0000 7f32a5371700 1 -- 172.21.15.17:0/1538771867 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32a00574e0 con 0x7f32a0064230 2022-08-01T19:37:39.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.402+0000 7f329d7fa700 1 -- 172.21.15.17:0/1538771867 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16273 (secure 0 0 0) 0x7f3290047aa0 con 0x7f32a0064230 2022-08-01T19:37:39.408 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:37:39.408 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":55,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:37:37.200615+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:37:34.595906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2022-08-01T19:37:35.889700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:37:39.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.406+0000 7f32a5371700 1 -- 172.21.15.17:0/1538771867 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3280053b30 msgr2=0x7f3280055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:39.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.406+0000 7f32a5371700 1 --2- 172.21.15.17:0/1538771867 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3280053b30 0x7f3280055ff0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f329400b000 tx=0x7f3294008040).stop 2022-08-01T19:37:39.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.406+0000 7f32a5371700 1 -- 172.21.15.17:0/1538771867 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32a0064230 msgr2=0x7f32a0146630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:39.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.406+0000 7f32a5371700 1 --2- 172.21.15.17:0/1538771867 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32a0064230 0x7f32a0146630 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f329000c000 tx=0x7f3290019000).stop 2022-08-01T19:37:39.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.406+0000 7f32a5371700 1 -- 172.21.15.17:0/1538771867 shutdown_connections 2022-08-01T19:37:39.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.406+0000 7f32a5371700 1 --2- 172.21.15.17:0/1538771867 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3280053b30 0x7f3280055ff0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:39.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.406+0000 7f32a5371700 1 --2- 172.21.15.17:0/1538771867 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f32a0146b20 0x7f32a0159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:39.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.406+0000 7f32a5371700 1 --2- 172.21.15.17:0/1538771867 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32a0064230 0x7f32a0146630 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:39.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.406+0000 7f32a5371700 1 --2- 172.21.15.17:0/1538771867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f328c004500 0x7f32a014cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:39.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.406+0000 7f32a5371700 1 -- 172.21.15.17:0/1538771867 >> 172.21.15.17:0/1538771867 conn(0x7f32a0060310 msgr2=0x7f32a0149f30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:39.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.406+0000 7f32a5371700 1 -- 172.21.15.17:0/1538771867 shutdown_connections 2022-08-01T19:37:39.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.406+0000 7f32a5371700 1 -- 172.21.15.17:0/1538771867 wait complete. 2022-08-01T19:37:39.421 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T19:37:34.595906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T19:37:35.889700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T19:37:39.422 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:37:39.532 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.530+0000 7f8c1f07d700 1 Processor -- start 2022-08-01T19:37:39.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.530+0000 7f8c1f07d700 1 -- start start 2022-08-01T19:37:39.540 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.530+0000 7f8c1f07d700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f8c180b2710 0x7f8c181401e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:39.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.530+0000 7f8c1f07d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8c180ac880 0x7f8c181406d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:39.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.530+0000 7f8c1f07d700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c18067a10 con 0x7f8c180b0270 2022-08-01T19:37:39.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.530+0000 7f8c1f07d700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c180ad740 con 0x7f8c180ac880 2022-08-01T19:37:39.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.530+0000 7f8c1f07d700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c18067cb0 con 0x7f8c180b2710 2022-08-01T19:37:39.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.530+0000 7f8c1e87c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8c180ac880 0x7f8c181406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:39.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.530+0000 7f8c1e87c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8c180ac880 0x7f8c181406d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39202/0 (socket says 172.21.15.17:39202) 2022-08-01T19:37:39.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.530+0000 7f8c1e87c700 1 -- 172.21.15.17:0/2025661730 learned_addr learned my addr 172.21.15.17:0/2025661730 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:39.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.530+0000 7f8c1e07b700 1 --2- 172.21.15.17:0/2025661730 >> v2:172.21.15.77:3300/0 conn(0x7f8c180b2710 0x7f8c181401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:39.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.530+0000 7f8c1e87c700 1 -- 172.21.15.17:0/2025661730 >> v2:172.21.15.77:3300/0 conn(0x7f8c180b2710 msgr2=0x7f8c181401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:39.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.530+0000 7f8c1e87c700 1 --2- 172.21.15.17:0/2025661730 >> v2:172.21.15.77:3300/0 conn(0x7f8c180b2710 0x7f8c181401e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:39.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.530+0000 7f8c1e87c700 1 -- 172.21.15.17:0/2025661730 >> v1:172.21.15.77:6789/0 conn(0x7f8c180b0270 legacy=0x7f8c18143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:39.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.530+0000 7f8c1d079700 1 -- 172.21.15.17:0/2025661730 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 226279227 0 0) 0x7f8c18067a10 con 0x7f8c180b0270 2022-08-01T19:37:39.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.530+0000 7f8c1e87c700 1 -- 172.21.15.17:0/2025661730 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c18147160 con 0x7f8c180ac880 2022-08-01T19:37:39.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.530+0000 7f8c1e87c700 1 --2- 172.21.15.17:0/2025661730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c180ac880 0x7f8c181406d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f8c100032f0 tx=0x7f8c10009290).ready entity=mon.0 client_cookie=26c99fa4e2f9fe22 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:39.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.530+0000 7f8c1d079700 1 -- 172.21.15.17:0/2025661730 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c1000b870 con 0x7f8c180ac880 2022-08-01T19:37:39.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.530+0000 7f8c1d079700 1 -- 172.21.15.17:0/2025661730 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c1000b9d0 con 0x7f8c180ac880 2022-08-01T19:37:39.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.530+0000 7f8c1d079700 1 -- 172.21.15.17:0/2025661730 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c1000bb40 con 0x7f8c180ac880 2022-08-01T19:37:39.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.530+0000 7f8c1f07d700 1 -- 172.21.15.17:0/2025661730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c180ac880 msgr2=0x7f8c181406d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:39.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.530+0000 7f8c1f07d700 1 --2- 172.21.15.17:0/2025661730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c180ac880 0x7f8c181406d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f8c100032f0 tx=0x7f8c10009290).stop 2022-08-01T19:37:39.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.530+0000 7f8c1f07d700 1 -- 172.21.15.17:0/2025661730 shutdown_connections 2022-08-01T19:37:39.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.530+0000 7f8c1f07d700 1 --2- 172.21.15.17:0/2025661730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c180ac880 0x7f8c181406d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:39.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.530+0000 7f8c1f07d700 1 --2- 172.21.15.17:0/2025661730 >> v2:172.21.15.77:3300/0 conn(0x7f8c180b2710 0x7f8c181401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:39.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.530+0000 7f8c1f07d700 1 -- 172.21.15.17:0/2025661730 >> 172.21.15.17:0/2025661730 conn(0x7f8c18060310 msgr2=0x7f8c180b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:39.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.530+0000 7f8c1f07d700 1 -- 172.21.15.17:0/2025661730 shutdown_connections 2022-08-01T19:37:39.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.530+0000 7f8c1f07d700 1 -- 172.21.15.17:0/2025661730 wait complete. 2022-08-01T19:37:39.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.534+0000 7f8c1f07d700 1 Processor -- start 2022-08-01T19:37:39.547 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.534+0000 7f8c1f07d700 1 -- start start 2022-08-01T19:37:39.547 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.534+0000 7f8c1f07d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8c180ac880 0x7f8c1809f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:39.547 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.534+0000 7f8c1f07d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c180b0270 0x7f8c180a92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:39.547 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.534+0000 7f8c1f07d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c180b2710 0x7f8c18141310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:39.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.534+0000 7f8c1f07d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c180ab600 con 0x7f8c180b2710 2022-08-01T19:37:39.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.534+0000 7f8c1f07d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1809f090 con 0x7f8c180b0270 2022-08-01T19:37:39.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.534+0000 7f8c1f07d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1809f330 con 0x7f8c180ac880 2022-08-01T19:37:39.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.534+0000 7f8c1e87c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c180b2710 0x7f8c18141310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:39.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.534+0000 7f8c1e87c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c180b2710 0x7f8c18141310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39208/0 (socket says 172.21.15.17:39208) 2022-08-01T19:37:39.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.534+0000 7f8c1e07b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8c180ac880 0x7f8c1809f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:39.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.534+0000 7f8c1e87c700 1 -- 172.21.15.17:0/2860296221 learned_addr learned my addr 172.21.15.17:0/2860296221 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:39.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.534+0000 7f8c1d87a700 1 --2- 172.21.15.17:0/2860296221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c180b0270 0x7f8c180a92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:39.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.534+0000 7f8c1e87c700 1 -- 172.21.15.17:0/2860296221 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8c180ac880 msgr2=0x7f8c1809f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:39.550 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.534+0000 7f8c1e87c700 1 --2- 172.21.15.17:0/2860296221 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8c180ac880 0x7f8c1809f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:39.550 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.534+0000 7f8c1e87c700 1 -- 172.21.15.17:0/2860296221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c180b0270 msgr2=0x7f8c180a92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:39.550 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.534+0000 7f8c1e87c700 1 --2- 172.21.15.17:0/2860296221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c180b0270 0x7f8c180a92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:39.550 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.534+0000 7f8c1e87c700 1 -- 172.21.15.17:0/2860296221 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c10009530 con 0x7f8c180b2710 2022-08-01T19:37:39.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.534+0000 7f8c1e87c700 1 --2- 172.21.15.17:0/2860296221 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c180b2710 0x7f8c18141310 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7f8c100032f0 tx=0x7f8c10016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:39.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.534+0000 7f8c077fe700 1 -- 172.21.15.17:0/2860296221 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c1000b870 con 0x7f8c180b2710 2022-08-01T19:37:39.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.538+0000 7f8c1f07d700 1 -- 172.21.15.17:0/2860296221 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c180ab8a0 con 0x7f8c180b2710 2022-08-01T19:37:39.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.538+0000 7f8c1f07d700 1 -- 172.21.15.17:0/2860296221 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c18143da0 con 0x7f8c180b2710 2022-08-01T19:37:39.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.538+0000 7f8c077fe700 1 -- 172.21.15.17:0/2860296221 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c1000bb40 con 0x7f8c180b2710 2022-08-01T19:37:39.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.538+0000 7f8c077fe700 1 -- 172.21.15.17:0/2860296221 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c10021860 con 0x7f8c180b2710 2022-08-01T19:37:39.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.538+0000 7f8c077fe700 1 -- 172.21.15.17:0/2860296221 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8c10028070 con 0x7f8c180b2710 2022-08-01T19:37:39.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.538+0000 7f8c077fe700 1 --2- 172.21.15.17:0/2860296221 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8bf8053950 0x7f8bf8055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:39.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.538+0000 7f8c1f07d700 1 -- 172.21.15.17:0/2860296221 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf4003410 con 0x7f8c180b2710 2022-08-01T19:37:39.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.538+0000 7f8c077fe700 1 -- 172.21.15.17:0/2860296221 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8c1006cdf0 con 0x7f8c180b2710 2022-08-01T19:37:39.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.538+0000 7f8c1e07b700 1 --2- 172.21.15.17:0/2860296221 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8bf8053950 0x7f8bf8055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:39.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.542+0000 7f8c1e07b700 1 --2- 172.21.15.17:0/2860296221 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8bf8053950 0x7f8bf8055e10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f8c1400b000 tx=0x7f8c14006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:39.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.550+0000 7f8c077fe700 1 -- 172.21.15.17:0/2860296221 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8c10047540 con 0x7f8c180b2710 2022-08-01T19:37:39.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.674+0000 7f8c1f07d700 1 -- 172.21.15.17:0/2860296221 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f8bf4003c40 con 0x7f8c180b2710 2022-08-01T19:37:39.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.674+0000 7f8c077fe700 1 -- 172.21.15.17:0/2860296221 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v55) v1 ==== 110+0+10 (secure 0 0 0) 0x7f8c10015070 con 0x7f8c180b2710 2022-08-01T19:37:39.679 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:37:39.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.674+0000 7f8c1f07d700 1 -- 172.21.15.17:0/2860296221 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8bf8053950 msgr2=0x7f8bf8055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:39.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.674+0000 7f8c1f07d700 1 --2- 172.21.15.17:0/2860296221 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8bf8053950 0x7f8bf8055e10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f8c1400b000 tx=0x7f8c14006df0).stop 2022-08-01T19:37:39.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.674+0000 7f8c1f07d700 1 -- 172.21.15.17:0/2860296221 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c180b2710 msgr2=0x7f8c18141310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:39.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.674+0000 7f8c1f07d700 1 --2- 172.21.15.17:0/2860296221 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c180b2710 0x7f8c18141310 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7f8c100032f0 tx=0x7f8c10016000).stop 2022-08-01T19:37:39.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.678+0000 7f8c1f07d700 1 -- 172.21.15.17:0/2860296221 shutdown_connections 2022-08-01T19:37:39.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.678+0000 7f8c1f07d700 1 --2- 172.21.15.17:0/2860296221 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8bf8053950 0x7f8bf8055e10 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:39.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.678+0000 7f8c1f07d700 1 --2- 172.21.15.17:0/2860296221 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c180b2710 0x7f8c18141310 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:39.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.678+0000 7f8c1f07d700 1 --2- 172.21.15.17:0/2860296221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c180b0270 0x7f8c180a92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:39.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.678+0000 7f8c1f07d700 1 --2- 172.21.15.17:0/2860296221 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8c180ac880 0x7f8c1809f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:39.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.678+0000 7f8c1f07d700 1 -- 172.21.15.17:0/2860296221 >> 172.21.15.17:0/2860296221 conn(0x7f8c18060310 msgr2=0x7f8c18140410 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:39.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.678+0000 7f8c1f07d700 1 -- 172.21.15.17:0/2860296221 shutdown_connections 2022-08-01T19:37:39.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.678+0000 7f8c1f07d700 1 -- 172.21.15.17:0/2860296221 wait complete. 2022-08-01T19:37:39.692 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:37:39.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.802+0000 7fc44ee10700 1 Processor -- start 2022-08-01T19:37:39.804 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.802+0000 7fc44ee10700 1 -- start start 2022-08-01T19:37:39.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.802+0000 7fc44ee10700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc4480a9af0 0x7fc4480b1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:39.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.802+0000 7fc44ee10700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fc4480b0850 0x7fc4480ab130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:39.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.802+0000 7fc44ee10700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4480b1df0 con 0x7fc4480ac3f0 2022-08-01T19:37:39.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.802+0000 7fc44ee10700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4480a9ed0 con 0x7fc4480a9af0 2022-08-01T19:37:39.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.802+0000 7fc44ee10700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4480b2ad0 con 0x7fc4480b0850 2022-08-01T19:37:39.807 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.802+0000 7fc44cbac700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc4480a9af0 0x7fc4480b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:39.807 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.802+0000 7fc44cbac700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc4480a9af0 0x7fc4480b1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39214/0 (socket says 172.21.15.17:39214) 2022-08-01T19:37:39.807 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.802+0000 7fc44cbac700 1 -- 172.21.15.17:0/3102210732 learned_addr learned my addr 172.21.15.17:0/3102210732 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:39.807 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.802+0000 7fc44d3ad700 1 --2- 172.21.15.17:0/3102210732 >> v2:172.21.15.77:3300/0 conn(0x7fc4480b0850 0x7fc4480ab130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:39.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.802+0000 7fc44cbac700 1 -- 172.21.15.17:0/3102210732 >> v2:172.21.15.77:3300/0 conn(0x7fc4480b0850 msgr2=0x7fc4480ab130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:39.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.802+0000 7fc44cbac700 1 --2- 172.21.15.17:0/3102210732 >> v2:172.21.15.77:3300/0 conn(0x7fc4480b0850 0x7fc4480ab130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:39.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.802+0000 7fc44cbac700 1 -- 172.21.15.17:0/3102210732 >> v1:172.21.15.77:6789/0 conn(0x7fc4480ac3f0 legacy=0x7fc4480aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:39.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.802+0000 7fc44cbac700 1 -- 172.21.15.17:0/3102210732 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4480b0250 con 0x7fc4480a9af0 2022-08-01T19:37:39.810 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.802+0000 7fc44cbac700 1 --2- 172.21.15.17:0/3102210732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4480a9af0 0x7fc4480b1900 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7fc43c002f70 tx=0x7fc43c006eb0).ready entity=mon.0 client_cookie=d045a515579f1ed7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:39.810 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.802+0000 7fc4477fe700 1 -- 172.21.15.17:0/3102210732 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc43c00d5c0 con 0x7fc4480a9af0 2022-08-01T19:37:39.811 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.802+0000 7fc4477fe700 1 -- 172.21.15.17:0/3102210732 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43c000e90 con 0x7fc4480a9af0 2022-08-01T19:37:39.811 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.802+0000 7fc4477fe700 1 -- 172.21.15.17:0/3102210732 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc43c00d950 con 0x7fc4480a9af0 2022-08-01T19:37:39.811 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.802+0000 7fc44ee10700 1 -- 172.21.15.17:0/3102210732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4480a9af0 msgr2=0x7fc4480b1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:39.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.802+0000 7fc44ee10700 1 --2- 172.21.15.17:0/3102210732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4480a9af0 0x7fc4480b1900 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7fc43c002f70 tx=0x7fc43c006eb0).stop 2022-08-01T19:37:39.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.802+0000 7fc44ee10700 1 -- 172.21.15.17:0/3102210732 shutdown_connections 2022-08-01T19:37:39.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.802+0000 7fc44ee10700 1 --2- 172.21.15.17:0/3102210732 >> v2:172.21.15.77:3300/0 conn(0x7fc4480b0850 0x7fc4480ab130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:39.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.802+0000 7fc44ee10700 1 --2- 172.21.15.17:0/3102210732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4480a9af0 0x7fc4480b1900 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:39.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.802+0000 7fc44ee10700 1 -- 172.21.15.17:0/3102210732 >> 172.21.15.17:0/3102210732 conn(0x7fc44805f840 msgr2=0x7fc44805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:39.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.802+0000 7fc44ee10700 1 -- 172.21.15.17:0/3102210732 shutdown_connections 2022-08-01T19:37:39.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.802+0000 7fc44ee10700 1 -- 172.21.15.17:0/3102210732 wait complete. 2022-08-01T19:37:39.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.806+0000 7fc44ee10700 1 Processor -- start 2022-08-01T19:37:39.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.806+0000 7fc44ee10700 1 -- start start 2022-08-01T19:37:39.816 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.806+0000 7fc44ee10700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4480a9af0 0x7fc448144990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:39.817 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.806+0000 7fc44ee10700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4480ac3f0 0x7fc44813c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:39.817 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.806+0000 7fc44ee10700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4480b0850 0x7fc44813cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:39.817 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.806+0000 7fc44ee10700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4480ada10 con 0x7fc4480ac3f0 2022-08-01T19:37:39.817 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.806+0000 7fc44ee10700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4480ae820 con 0x7fc4480a9af0 2022-08-01T19:37:39.818 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.806+0000 7fc44ee10700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4480aeac0 con 0x7fc4480b0850 2022-08-01T19:37:39.818 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.806+0000 7fc447fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4480ac3f0 0x7fc44813c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:39.818 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.806+0000 7fc447fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4480ac3f0 0x7fc44813c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39220/0 (socket says 172.21.15.17:39220) 2022-08-01T19:37:39.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.806+0000 7fc447fff700 1 -- 172.21.15.17:0/2923626852 learned_addr learned my addr 172.21.15.17:0/2923626852 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:39.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.806+0000 7fc44cbac700 1 --2- 172.21.15.17:0/2923626852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4480a9af0 0x7fc448144990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:39.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.806+0000 7fc44d3ad700 1 --2- 172.21.15.17:0/2923626852 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4480b0850 0x7fc44813cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:39.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.806+0000 7fc447fff700 1 -- 172.21.15.17:0/2923626852 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4480b0850 msgr2=0x7fc44813cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:39.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.806+0000 7fc447fff700 1 --2- 172.21.15.17:0/2923626852 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4480b0850 0x7fc44813cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:39.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.806+0000 7fc447fff700 1 -- 172.21.15.17:0/2923626852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4480a9af0 msgr2=0x7fc448144990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:39.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.806+0000 7fc447fff700 1 --2- 172.21.15.17:0/2923626852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4480a9af0 0x7fc448144990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:39.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.806+0000 7fc447fff700 1 -- 172.21.15.17:0/2923626852 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc43c00b1e0 con 0x7fc4480ac3f0 2022-08-01T19:37:39.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.806+0000 7fc44cbac700 1 --2- 172.21.15.17:0/2923626852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4480a9af0 0x7fc448144990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:37:39.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.806+0000 7fc447fff700 1 --2- 172.21.15.17:0/2923626852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4480ac3f0 0x7fc44813c9d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7fc4480ac9c0 tx=0x7fc438007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:39.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.810+0000 7fc445ffb700 1 -- 172.21.15.17:0/2923626852 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc43800ce30 con 0x7fc4480ac3f0 2022-08-01T19:37:39.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.810+0000 7fc445ffb700 1 -- 172.21.15.17:0/2923626852 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc438007240 con 0x7fc4480ac3f0 2022-08-01T19:37:39.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.810+0000 7fc44ee10700 1 -- 172.21.15.17:0/2923626852 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4480adc40 con 0x7fc4480ac3f0 2022-08-01T19:37:39.822 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.810+0000 7fc445ffb700 1 -- 172.21.15.17:0/2923626852 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc438006700 con 0x7fc4480ac3f0 2022-08-01T19:37:39.822 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.810+0000 7fc44ee10700 1 -- 172.21.15.17:0/2923626852 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc44813d3b0 con 0x7fc4480ac3f0 2022-08-01T19:37:39.822 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.810+0000 7fc44ee10700 1 -- 172.21.15.17:0/2923626852 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4480addd0 con 0x7fc4480ac3f0 2022-08-01T19:37:39.822 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.810+0000 7fc445ffb700 1 -- 172.21.15.17:0/2923626852 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc438010030 con 0x7fc4480ac3f0 2022-08-01T19:37:39.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.810+0000 7fc445ffb700 1 --2- 172.21.15.17:0/2923626852 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc4280539e0 0x7fc428055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:39.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.810+0000 7fc445ffb700 1 -- 172.21.15.17:0/2923626852 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc43806bb40 con 0x7fc4480ac3f0 2022-08-01T19:37:39.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.810+0000 7fc44cbac700 1 --2- 172.21.15.17:0/2923626852 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc4280539e0 0x7fc428055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:39.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.810+0000 7fc44cbac700 1 --2- 172.21.15.17:0/2923626852 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc4280539e0 0x7fc428055ea0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7fc43c01a000 tx=0x7fc43c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:39.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.822+0000 7fc445ffb700 1 -- 172.21.15.17:0/2923626852 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc438042900 con 0x7fc4480ac3f0 2022-08-01T19:37:39.950 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.946+0000 7fc44ee10700 1 -- 172.21.15.17:0/2923626852 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc4480b1df0 con 0x7fc4480ac3f0 2022-08-01T19:37:39.950 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.946+0000 7fc445ffb700 1 -- 172.21.15.17:0/2923626852 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v55) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc4480b1df0 con 0x7fc4480ac3f0 2022-08-01T19:37:39.951 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:37:39.952 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.950+0000 7fc44ee10700 1 -- 172.21.15.17:0/2923626852 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc4280539e0 msgr2=0x7fc428055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:39.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.950+0000 7fc44ee10700 1 --2- 172.21.15.17:0/2923626852 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc4280539e0 0x7fc428055ea0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7fc43c01a000 tx=0x7fc43c015000).stop 2022-08-01T19:37:39.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.950+0000 7fc44ee10700 1 -- 172.21.15.17:0/2923626852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4480ac3f0 msgr2=0x7fc44813c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:39.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.950+0000 7fc44ee10700 1 --2- 172.21.15.17:0/2923626852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4480ac3f0 0x7fc44813c9d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7fc4480ac9c0 tx=0x7fc438007c10).stop 2022-08-01T19:37:39.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.950+0000 7fc44ee10700 1 -- 172.21.15.17:0/2923626852 shutdown_connections 2022-08-01T19:37:39.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.950+0000 7fc44ee10700 1 --2- 172.21.15.17:0/2923626852 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc4280539e0 0x7fc428055ea0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:39.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.950+0000 7fc44ee10700 1 --2- 172.21.15.17:0/2923626852 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4480b0850 0x7fc44813cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:39.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.950+0000 7fc44ee10700 1 --2- 172.21.15.17:0/2923626852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4480ac3f0 0x7fc44813c9d0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:39.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.950+0000 7fc44ee10700 1 --2- 172.21.15.17:0/2923626852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4480a9af0 0x7fc448144990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:39.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.950+0000 7fc44ee10700 1 -- 172.21.15.17:0/2923626852 >> 172.21.15.17:0/2923626852 conn(0x7fc44805f840 msgr2=0x7fc4480b2010 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:39.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.950+0000 7fc44ee10700 1 -- 172.21.15.17:0/2923626852 shutdown_connections 2022-08-01T19:37:39.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:39.950+0000 7fc44ee10700 1 -- 172.21.15.17:0/2923626852 wait complete. 2022-08-01T19:37:39.963 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T19:37:40.076 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.074+0000 7ff455b0c700 1 Processor -- start 2022-08-01T19:37:40.077 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.074+0000 7ff455b0c700 1 -- start start 2022-08-01T19:37:40.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.074+0000 7ff455b0c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff450060290 0x7ff450064ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:40.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.074+0000 7ff455b0c700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ff4501445d0 0x7ff450065390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:40.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.074+0000 7ff455b0c700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff450060670 con 0x7ff4500b3a60 2022-08-01T19:37:40.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.074+0000 7ff455b0c700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff450065880 con 0x7ff4501445d0 2022-08-01T19:37:40.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.074+0000 7ff455b0c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff450065e10 con 0x7ff450060290 2022-08-01T19:37:40.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.074+0000 7ff44ffff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff4500b3a60 0x7ff450144b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59654/0 (socket says 172.21.15.17:59654) 2022-08-01T19:37:40.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.074+0000 7ff44ffff700 1 -- 172.21.15.17:0/3967671049 learned_addr learned my addr 172.21.15.17:0/3967671049 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:37:40.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.074+0000 7ff44effd700 1 --2- 172.21.15.17:0/3967671049 >> v2:172.21.15.77:3300/0 conn(0x7ff4501445d0 0x7ff450065390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:40.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.074+0000 7ff44f7fe700 1 --2- 172.21.15.17:0/3967671049 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff450060290 0x7ff450064ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:40.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.074+0000 7ff44e7fc700 1 -- 172.21.15.17:0/3967671049 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 567119599 0 0) 0x7ff450060670 con 0x7ff4500b3a60 2022-08-01T19:37:40.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.074+0000 7ff44e7fc700 1 -- 172.21.15.17:0/3967671049 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff430003260 con 0x7ff4500b3a60 2022-08-01T19:37:40.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.074+0000 7ff44effd700 1 -- 172.21.15.17:0/3967671049 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff450060290 msgr2=0x7ff450064ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:40.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.074+0000 7ff44effd700 1 --2- 172.21.15.17:0/3967671049 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff450060290 0x7ff450064ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:40.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.074+0000 7ff44effd700 1 -- 172.21.15.17:0/3967671049 >> v1:172.21.15.17:6789/0 conn(0x7ff4500b3a60 legacy=0x7ff450144b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:40.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.074+0000 7ff44effd700 1 -- 172.21.15.17:0/3967671049 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4500b2540 con 0x7ff4501445d0 2022-08-01T19:37:40.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.074+0000 7ff44effd700 1 --2- 172.21.15.17:0/3967671049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4501445d0 0x7ff450065390 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7ff440002f70 tx=0x7ff440006df0).ready entity=mon.1 client_cookie=56b3df7d1b5cf0aa server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:40.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.074+0000 7ff44e7fc700 1 -- 172.21.15.17:0/3967671049 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff44000d4a0 con 0x7ff4501445d0 2022-08-01T19:37:40.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.074+0000 7ff44e7fc700 1 -- 172.21.15.17:0/3967671049 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff440000e90 con 0x7ff4501445d0 2022-08-01T19:37:40.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.074+0000 7ff44e7fc700 1 -- 172.21.15.17:0/3967671049 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff44000d7f0 con 0x7ff4501445d0 2022-08-01T19:37:40.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.078+0000 7ff455b0c700 1 -- 172.21.15.17:0/3967671049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4501445d0 msgr2=0x7ff450065390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:40.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.078+0000 7ff455b0c700 1 --2- 172.21.15.17:0/3967671049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4501445d0 0x7ff450065390 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7ff440002f70 tx=0x7ff440006df0).stop 2022-08-01T19:37:40.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.078+0000 7ff455b0c700 1 -- 172.21.15.17:0/3967671049 shutdown_connections 2022-08-01T19:37:40.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.078+0000 7ff455b0c700 1 --2- 172.21.15.17:0/3967671049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4501445d0 0x7ff450065390 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:40.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.078+0000 7ff455b0c700 1 --2- 172.21.15.17:0/3967671049 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff450060290 0x7ff450064ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:40.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.078+0000 7ff455b0c700 1 -- 172.21.15.17:0/3967671049 >> 172.21.15.17:0/3967671049 conn(0x7ff45005d0f0 msgr2=0x7ff45005f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:40.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.078+0000 7ff455b0c700 1 -- 172.21.15.17:0/3967671049 shutdown_connections 2022-08-01T19:37:40.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.078+0000 7ff455b0c700 1 -- 172.21.15.17:0/3967671049 wait complete. 2022-08-01T19:37:40.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.078+0000 7ff455b0c700 1 Processor -- start 2022-08-01T19:37:40.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.078+0000 7ff455b0c700 1 -- start start 2022-08-01T19:37:40.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.078+0000 7ff455b0c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff450060290 0x7ff4500aa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:40.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.078+0000 7ff455b0c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff4500b3a60 0x7ff45013bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:40.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.078+0000 7ff455b0c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff45013df90 0x7ff4500af840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:40.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.078+0000 7ff455b0c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45013c330 con 0x7ff450060290 2022-08-01T19:37:40.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.078+0000 7ff455b0c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff450060ea0 con 0x7ff45013df90 2022-08-01T19:37:40.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.078+0000 7ff455b0c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4500aa9a0 con 0x7ff4500b3a60 2022-08-01T19:37:40.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.078+0000 7ff44f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff450060290 0x7ff4500aa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:40.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.078+0000 7ff44f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff450060290 0x7ff4500aa4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39232/0 (socket says 172.21.15.17:39232) 2022-08-01T19:37:40.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.078+0000 7ff44f7fe700 1 -- 172.21.15.17:0/4027678358 learned_addr learned my addr 172.21.15.17:0/4027678358 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:40.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.078+0000 7ff44effd700 1 --2- 172.21.15.17:0/4027678358 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff4500b3a60 0x7ff45013bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:40.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.078+0000 7ff44ffff700 1 --2- 172.21.15.17:0/4027678358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff45013df90 0x7ff4500af840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:40.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.082+0000 7ff44f7fe700 1 -- 172.21.15.17:0/4027678358 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff4500b3a60 msgr2=0x7ff45013bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:40.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.082+0000 7ff44f7fe700 1 --2- 172.21.15.17:0/4027678358 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff4500b3a60 0x7ff45013bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:40.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.082+0000 7ff44f7fe700 1 -- 172.21.15.17:0/4027678358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff45013df90 msgr2=0x7ff4500af840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:40.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.082+0000 7ff44f7fe700 1 --2- 172.21.15.17:0/4027678358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff45013df90 0x7ff4500af840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:40.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.082+0000 7ff44f7fe700 1 -- 172.21.15.17:0/4027678358 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44000b180 con 0x7ff450060290 2022-08-01T19:37:40.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.082+0000 7ff44f7fe700 1 --2- 172.21.15.17:0/4027678358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff450060290 0x7ff4500aa4b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7ff45013ea00 tx=0x7ff438006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:40.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.082+0000 7ff44cff9700 1 -- 172.21.15.17:0/4027678358 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff438010780 con 0x7ff450060290 2022-08-01T19:37:40.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.082+0000 7ff44cff9700 1 -- 172.21.15.17:0/4027678358 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4380108e0 con 0x7ff450060290 2022-08-01T19:37:40.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.082+0000 7ff44cff9700 1 -- 172.21.15.17:0/4027678358 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff438010c40 con 0x7ff450060290 2022-08-01T19:37:40.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.082+0000 7ff455b0c700 1 -- 172.21.15.17:0/4027678358 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff450061640 con 0x7ff450060290 2022-08-01T19:37:40.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.082+0000 7ff455b0c700 1 -- 172.21.15.17:0/4027678358 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4500ae300 con 0x7ff450060290 2022-08-01T19:37:40.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.082+0000 7ff455b0c700 1 -- 172.21.15.17:0/4027678358 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff450144b60 con 0x7ff450060290 2022-08-01T19:37:40.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.082+0000 7ff44cff9700 1 -- 172.21.15.17:0/4027678358 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff43801c460 con 0x7ff450060290 2022-08-01T19:37:40.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.082+0000 7ff44cff9700 1 --2- 172.21.15.17:0/4027678358 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff4340539e0 0x7ff434055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:40.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.082+0000 7ff44cff9700 1 -- 172.21.15.17:0/4027678358 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff43806c4a0 con 0x7ff450060290 2022-08-01T19:37:40.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.082+0000 7ff44effd700 1 --2- 172.21.15.17:0/4027678358 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff4340539e0 0x7ff434055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:40.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.086+0000 7ff44effd700 1 --2- 172.21.15.17:0/4027678358 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff4340539e0 0x7ff434055ea0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7ff44001a000 tx=0x7ff440014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:40.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.090+0000 7ff44cff9700 1 -- 172.21.15.17:0/4027678358 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff438043160 con 0x7ff450060290 2022-08-01T19:37:40.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.218+0000 7ff455b0c700 1 -- 172.21.15.17:0/4027678358 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff4500ae500 con 0x7ff450060290 2022-08-01T19:37:40.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.218+0000 7ff44cff9700 1 -- 172.21.15.17:0/4027678358 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+10 (secure 0 0 0) 0x7ff438046780 con 0x7ff450060290 2022-08-01T19:37:40.222 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:37:40.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.218+0000 7ff455b0c700 1 -- 172.21.15.17:0/4027678358 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff4340539e0 msgr2=0x7ff434055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:40.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.218+0000 7ff455b0c700 1 --2- 172.21.15.17:0/4027678358 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff4340539e0 0x7ff434055ea0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7ff44001a000 tx=0x7ff440014000).stop 2022-08-01T19:37:40.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.218+0000 7ff455b0c700 1 -- 172.21.15.17:0/4027678358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff450060290 msgr2=0x7ff4500aa4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:40.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.218+0000 7ff455b0c700 1 --2- 172.21.15.17:0/4027678358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff450060290 0x7ff4500aa4b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7ff45013ea00 tx=0x7ff438006df0).stop 2022-08-01T19:37:40.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.222+0000 7ff455b0c700 1 -- 172.21.15.17:0/4027678358 shutdown_connections 2022-08-01T19:37:40.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.222+0000 7ff455b0c700 1 --2- 172.21.15.17:0/4027678358 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff4340539e0 0x7ff434055ea0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:40.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.222+0000 7ff455b0c700 1 --2- 172.21.15.17:0/4027678358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff45013df90 0x7ff4500af840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:40.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.222+0000 7ff455b0c700 1 --2- 172.21.15.17:0/4027678358 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff4500b3a60 0x7ff45013bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:40.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.222+0000 7ff455b0c700 1 --2- 172.21.15.17:0/4027678358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff450060290 0x7ff4500aa4b0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:40.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.222+0000 7ff455b0c700 1 -- 172.21.15.17:0/4027678358 >> 172.21.15.17:0/4027678358 conn(0x7ff45005d0f0 msgr2=0x7ff4500b6870 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:40.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.222+0000 7ff455b0c700 1 -- 172.21.15.17:0/4027678358 shutdown_connections 2022-08-01T19:37:40.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.222+0000 7ff455b0c700 1 -- 172.21.15.17:0/4027678358 wait complete. 2022-08-01T19:37:40.235 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T19:37:40.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.342+0000 7ff149b8e700 1 Processor -- start 2022-08-01T19:37:40.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.342+0000 7ff149b8e700 1 -- start start 2022-08-01T19:37:40.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.342+0000 7ff149b8e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff1440b1580 0x7ff1440b3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:40.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.342+0000 7ff149b8e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ff1440b1ef0 0x7ff144146d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:40.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.342+0000 7ff149b8e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff144147230 0x7ff144147680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:40.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.342+0000 7ff149b8e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1440ae7e0 con 0x7ff144147230 2022-08-01T19:37:40.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.342+0000 7ff149b8e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1440b1960 con 0x7ff1440b1ef0 2022-08-01T19:37:40.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.342+0000 7ff149b8e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1440b2470 con 0x7ff1440b1580 2022-08-01T19:37:40.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.346+0000 7ff143fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff144147230 0x7ff144147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:40.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.346+0000 7ff143fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff144147230 0x7ff144147680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39244/0 (socket says 172.21.15.17:39244) 2022-08-01T19:37:40.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.346+0000 7ff143fff700 1 -- 172.21.15.17:0/3940616624 learned_addr learned my addr 172.21.15.17:0/3940616624 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:40.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.346+0000 7ff142ffd700 1 --2- 172.21.15.17:0/3940616624 >> v2:172.21.15.77:3300/0 conn(0x7ff1440b1ef0 0x7ff144146d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:40.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.346+0000 7ff1437fe700 1 --2- 172.21.15.17:0/3940616624 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff1440b1580 0x7ff1440b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:40.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.346+0000 7ff143fff700 1 -- 172.21.15.17:0/3940616624 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff1440b1580 msgr2=0x7ff1440b3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:40.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.346+0000 7ff143fff700 1 --2- 172.21.15.17:0/3940616624 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff1440b1580 0x7ff1440b3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:40.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.346+0000 7ff143fff700 1 -- 172.21.15.17:0/3940616624 >> v2:172.21.15.77:3300/0 conn(0x7ff1440b1ef0 msgr2=0x7ff144146d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:40.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.346+0000 7ff143fff700 1 --2- 172.21.15.17:0/3940616624 >> v2:172.21.15.77:3300/0 conn(0x7ff1440b1ef0 0x7ff144146d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:40.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.346+0000 7ff143fff700 1 -- 172.21.15.17:0/3940616624 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1440b2710 con 0x7ff144147230 2022-08-01T19:37:40.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.346+0000 7ff143fff700 1 --2- 172.21.15.17:0/3940616624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff144147230 0x7ff144147680 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7ff13c0032f0 tx=0x7ff13c009290).ready entity=mon.0 client_cookie=a5f2ea33c612c4cc server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:40.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.346+0000 7ff1427fc700 1 -- 172.21.15.17:0/3940616624 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff13c00b870 con 0x7ff144147230 2022-08-01T19:37:40.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.346+0000 7ff1427fc700 1 -- 172.21.15.17:0/3940616624 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff13c00b9d0 con 0x7ff144147230 2022-08-01T19:37:40.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.346+0000 7ff1427fc700 1 -- 172.21.15.17:0/3940616624 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff13c00bb40 con 0x7ff144147230 2022-08-01T19:37:40.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.346+0000 7ff149b8e700 1 -- 172.21.15.17:0/3940616624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff144147230 msgr2=0x7ff144147680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:40.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.346+0000 7ff149b8e700 1 --2- 172.21.15.17:0/3940616624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff144147230 0x7ff144147680 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7ff13c0032f0 tx=0x7ff13c009290).stop 2022-08-01T19:37:40.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.346+0000 7ff149b8e700 1 -- 172.21.15.17:0/3940616624 shutdown_connections 2022-08-01T19:37:40.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.346+0000 7ff149b8e700 1 --2- 172.21.15.17:0/3940616624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff144147230 0x7ff144147680 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:40.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.346+0000 7ff149b8e700 1 --2- 172.21.15.17:0/3940616624 >> v2:172.21.15.77:3300/0 conn(0x7ff1440b1ef0 0x7ff144146d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:40.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.346+0000 7ff149b8e700 1 --2- 172.21.15.17:0/3940616624 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff1440b1580 0x7ff1440b3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:40.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.346+0000 7ff149b8e700 1 -- 172.21.15.17:0/3940616624 >> 172.21.15.17:0/3940616624 conn(0x7ff144060950 msgr2=0x7ff144060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:40.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.346+0000 7ff149b8e700 1 -- 172.21.15.17:0/3940616624 shutdown_connections 2022-08-01T19:37:40.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.346+0000 7ff149b8e700 1 -- 172.21.15.17:0/3940616624 wait complete. 2022-08-01T19:37:40.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.346+0000 7ff149b8e700 1 Processor -- start 2022-08-01T19:37:40.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.350+0000 7ff149b8e700 1 -- start start 2022-08-01T19:37:40.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.350+0000 7ff149b8e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1440b1580 0x7ff1440b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:40.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.350+0000 7ff149b8e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff1440b1ef0 0x7ff1440a95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:40.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.350+0000 7ff149b8e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff144147230 0x7ff1440a9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:40.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.350+0000 7ff149b8e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff144067a40 con 0x7ff1440b1580 2022-08-01T19:37:40.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.350+0000 7ff149b8e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1441526d0 con 0x7ff1440b1ef0 2022-08-01T19:37:40.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.350+0000 7ff149b8e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff144152930 con 0x7ff144147230 2022-08-01T19:37:40.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.350+0000 7ff1437fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1440b1580 0x7ff1440b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:40.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.350+0000 7ff1437fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1440b1580 0x7ff1440b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39248/0 (socket says 172.21.15.17:39248) 2022-08-01T19:37:40.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.350+0000 7ff1437fe700 1 -- 172.21.15.17:0/1722717432 learned_addr learned my addr 172.21.15.17:0/1722717432 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:40.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.350+0000 7ff143fff700 1 --2- 172.21.15.17:0/1722717432 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff144147230 0x7ff1440a9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:40.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.350+0000 7ff142ffd700 1 --2- 172.21.15.17:0/1722717432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff1440b1ef0 0x7ff1440a95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:40.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.350+0000 7ff1437fe700 1 -- 172.21.15.17:0/1722717432 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff144147230 msgr2=0x7ff1440a9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:40.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.350+0000 7ff1437fe700 1 --2- 172.21.15.17:0/1722717432 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff144147230 0x7ff1440a9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:40.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.350+0000 7ff1437fe700 1 -- 172.21.15.17:0/1722717432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff1440b1ef0 msgr2=0x7ff1440a95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:40.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.350+0000 7ff1437fe700 1 --2- 172.21.15.17:0/1722717432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff1440b1ef0 0x7ff1440a95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:40.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.350+0000 7ff1437fe700 1 -- 172.21.15.17:0/1722717432 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff13c009530 con 0x7ff1440b1580 2022-08-01T19:37:40.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.350+0000 7ff1437fe700 1 --2- 172.21.15.17:0/1722717432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1440b1580 0x7ff1440b0430 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7ff12c0030b0 tx=0x7ff12c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:40.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.350+0000 7ff140ff9700 1 -- 172.21.15.17:0/1722717432 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff12c011780 con 0x7ff1440b1580 2022-08-01T19:37:40.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.350+0000 7ff140ff9700 1 -- 172.21.15.17:0/1722717432 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12c0118e0 con 0x7ff1440b1580 2022-08-01T19:37:40.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.350+0000 7ff149b8e700 1 -- 172.21.15.17:0/1722717432 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff144152bd0 con 0x7ff1440b1580 2022-08-01T19:37:40.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.350+0000 7ff140ff9700 1 -- 172.21.15.17:0/1722717432 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff12c011c40 con 0x7ff1440b1580 2022-08-01T19:37:40.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.350+0000 7ff149b8e700 1 -- 172.21.15.17:0/1722717432 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff144153310 con 0x7ff1440b1580 2022-08-01T19:37:40.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.354+0000 7ff140ff9700 1 -- 172.21.15.17:0/1722717432 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff12c012420 con 0x7ff1440b1580 2022-08-01T19:37:40.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.354+0000 7ff149b8e700 1 -- 172.21.15.17:0/1722717432 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1440b29b0 con 0x7ff1440b1580 2022-08-01T19:37:40.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.354+0000 7ff140ff9700 1 --2- 172.21.15.17:0/1722717432 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff128053950 0x7ff128055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:40.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.354+0000 7ff140ff9700 1 -- 172.21.15.17:0/1722717432 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff12c06bff0 con 0x7ff1440b1580 2022-08-01T19:37:40.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.354+0000 7ff142ffd700 1 --2- 172.21.15.17:0/1722717432 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff128053950 0x7ff128055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:40.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.354+0000 7ff142ffd700 1 --2- 172.21.15.17:0/1722717432 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff128053950 0x7ff128055e10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7ff13400b000 tx=0x7ff134006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:40.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.358+0000 7ff140ff9700 1 -- 172.21.15.17:0/1722717432 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff12c042c20 con 0x7ff1440b1580 2022-08-01T19:37:40.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.486+0000 7ff149b8e700 1 -- 172.21.15.17:0/1722717432 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff1440574e0 con 0x7ff1440b1580 2022-08-01T19:37:40.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.486+0000 7ff140ff9700 1 -- 172.21.15.17:0/1722717432 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+10 (secure 0 0 0) 0x7ff12c046240 con 0x7ff1440b1580 2022-08-01T19:37:40.491 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:37:40.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.490+0000 7ff149b8e700 1 -- 172.21.15.17:0/1722717432 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff128053950 msgr2=0x7ff128055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:40.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.490+0000 7ff149b8e700 1 --2- 172.21.15.17:0/1722717432 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff128053950 0x7ff128055e10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7ff13400b000 tx=0x7ff134006df0).stop 2022-08-01T19:37:40.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.490+0000 7ff149b8e700 1 -- 172.21.15.17:0/1722717432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1440b1580 msgr2=0x7ff1440b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:40.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.490+0000 7ff149b8e700 1 --2- 172.21.15.17:0/1722717432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1440b1580 0x7ff1440b0430 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7ff12c0030b0 tx=0x7ff12c008040).stop 2022-08-01T19:37:40.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.490+0000 7ff149b8e700 1 -- 172.21.15.17:0/1722717432 shutdown_connections 2022-08-01T19:37:40.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.490+0000 7ff149b8e700 1 --2- 172.21.15.17:0/1722717432 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff128053950 0x7ff128055e10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:40.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.490+0000 7ff149b8e700 1 --2- 172.21.15.17:0/1722717432 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff144147230 0x7ff1440a9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:40.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.490+0000 7ff149b8e700 1 --2- 172.21.15.17:0/1722717432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff1440b1ef0 0x7ff1440a95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:40.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.490+0000 7ff149b8e700 1 --2- 172.21.15.17:0/1722717432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1440b1580 0x7ff1440b0430 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:40.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.490+0000 7ff149b8e700 1 -- 172.21.15.17:0/1722717432 >> 172.21.15.17:0/1722717432 conn(0x7ff144060950 msgr2=0x7ff1440b53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:40.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.490+0000 7ff149b8e700 1 -- 172.21.15.17:0/1722717432 shutdown_connections 2022-08-01T19:37:40.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.490+0000 7ff149b8e700 1 -- 172.21.15.17:0/1722717432 wait complete. 2022-08-01T19:37:40.508 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T19:37:40.509 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:37:40.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.606+0000 7f07e299f700 1 Processor -- start 2022-08-01T19:37:40.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.606+0000 7f07e299f700 1 -- start start 2022-08-01T19:37:40.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.606+0000 7f07e299f700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f07dc148530 0x7f07dc0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:40.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.606+0000 7f07e299f700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07dc0b04e0 con 0x7f07dc0a7250 2022-08-01T19:37:40.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.606+0000 7f07e299f700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07dc0b0780 con 0x7f07dc0b1f40 2022-08-01T19:37:40.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.606+0000 7f07e299f700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07dc0a9250 con 0x7f07dc148530 2022-08-01T19:37:40.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.606+0000 7f07db7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f07dc148530 0x7f07dc0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:40.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.606+0000 7f07db7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f07dc148530 0x7f07dc0b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39254/0 (socket says 172.21.15.17:39254) 2022-08-01T19:37:40.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.606+0000 7f07db7fe700 1 -- 172.21.15.17:0/1061420439 learned_addr learned my addr 172.21.15.17:0/1061420439 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:40.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.606+0000 7f07e0f3c700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f07dc0a7250 0x7f07dc1448b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59684/0 (socket says 172.21.15.17:59684) 2022-08-01T19:37:40.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.606+0000 7f07db7fe700 1 -- 172.21.15.17:0/1061420439 >> v1:172.21.15.77:6789/0 conn(0x7f07dc0b1f40 legacy=0x7f07dc0b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:40.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.606+0000 7f07db7fe700 1 -- 172.21.15.17:0/1061420439 >> v1:172.21.15.17:6789/0 conn(0x7f07dc0a7250 legacy=0x7f07dc1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:40.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.606+0000 7f07db7fe700 1 -- 172.21.15.17:0/1061420439 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07dc0b0a20 con 0x7f07dc148530 2022-08-01T19:37:40.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.606+0000 7f07db7fe700 1 --2- 172.21.15.17:0/1061420439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07dc148530 0x7f07dc0b1160 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f07dc0b3300 tx=0x7f07cc009290).ready entity=mon.0 client_cookie=443a86edabf7d5bc server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:40.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.606+0000 7f07daffd700 1 -- 172.21.15.17:0/1061420439 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07cc00b820 con 0x7f07dc148530 2022-08-01T19:37:40.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.606+0000 7f07daffd700 1 -- 172.21.15.17:0/1061420439 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07cc00b980 con 0x7f07dc148530 2022-08-01T19:37:40.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.606+0000 7f07daffd700 1 -- 172.21.15.17:0/1061420439 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07cc00baf0 con 0x7f07dc148530 2022-08-01T19:37:40.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.606+0000 7f07e299f700 1 -- 172.21.15.17:0/1061420439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07dc148530 msgr2=0x7f07dc0b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:40.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.606+0000 7f07e299f700 1 --2- 172.21.15.17:0/1061420439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07dc148530 0x7f07dc0b1160 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f07dc0b3300 tx=0x7f07cc009290).stop 2022-08-01T19:37:40.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.606+0000 7f07e299f700 1 -- 172.21.15.17:0/1061420439 shutdown_connections 2022-08-01T19:37:40.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.606+0000 7f07e299f700 1 --2- 172.21.15.17:0/1061420439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07dc148530 0x7f07dc0b1160 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:40.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.606+0000 7f07e299f700 1 -- 172.21.15.17:0/1061420439 >> 172.21.15.17:0/1061420439 conn(0x7f07dc05f840 msgr2=0x7f07dc05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:40.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.606+0000 7f07e299f700 1 -- 172.21.15.17:0/1061420439 shutdown_connections 2022-08-01T19:37:40.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.606+0000 7f07e299f700 1 -- 172.21.15.17:0/1061420439 wait complete. 2022-08-01T19:37:40.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.610+0000 7f07e299f700 1 Processor -- start 2022-08-01T19:37:40.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.610+0000 7f07e299f700 1 -- start start 2022-08-01T19:37:40.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.610+0000 7f07e299f700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f07dc0a7250 0x7f07dc149b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:40.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.610+0000 7f07e299f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07dc0b1f40 0x7f07dc157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:40.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.610+0000 7f07e299f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07dc148530 0x7f07dc0ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:40.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.610+0000 7f07e299f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc15b0b0 con 0x7f07dc0b1f40 2022-08-01T19:37:40.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.610+0000 7f07e299f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc156d40 con 0x7f07dc148530 2022-08-01T19:37:40.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.610+0000 7f07e299f700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc156fe0 con 0x7f07dc0a7250 2022-08-01T19:37:40.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.610+0000 7f07db7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07dc0b1f40 0x7f07dc157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:40.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.610+0000 7f07db7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07dc0b1f40 0x7f07dc157720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39262/0 (socket says 172.21.15.17:39262) 2022-08-01T19:37:40.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.610+0000 7f07db7fe700 1 -- 172.21.15.17:0/1157803402 learned_addr learned my addr 172.21.15.17:0/1157803402 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:40.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.610+0000 7f07dbfff700 1 --2- 172.21.15.17:0/1157803402 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f07dc0a7250 0x7f07dc149b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:40.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.610+0000 7f07e0f3c700 1 --2- 172.21.15.17:0/1157803402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07dc148530 0x7f07dc0ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:40.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.610+0000 7f07db7fe700 1 -- 172.21.15.17:0/1157803402 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f07dc0a7250 msgr2=0x7f07dc149b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:40.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.610+0000 7f07db7fe700 1 --2- 172.21.15.17:0/1157803402 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f07dc0a7250 0x7f07dc149b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:40.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.610+0000 7f07db7fe700 1 -- 172.21.15.17:0/1157803402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07dc148530 msgr2=0x7f07dc0ade90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:40.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.610+0000 7f07db7fe700 1 --2- 172.21.15.17:0/1157803402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07dc148530 0x7f07dc0ade90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:40.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.610+0000 7f07db7fe700 1 -- 172.21.15.17:0/1157803402 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07cc009530 con 0x7f07dc0b1f40 2022-08-01T19:37:40.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.610+0000 7f07db7fe700 1 --2- 172.21.15.17:0/1157803402 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07dc0b1f40 0x7f07dc157720 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f07dc0b3300 tx=0x7f07cc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:40.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.610+0000 7f07d97fa700 1 -- 172.21.15.17:0/1157803402 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07cc01f3f0 con 0x7f07dc0b1f40 2022-08-01T19:37:40.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.614+0000 7f07d97fa700 1 -- 172.21.15.17:0/1157803402 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07cc01f550 con 0x7f07dc0b1f40 2022-08-01T19:37:40.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.614+0000 7f07e299f700 1 -- 172.21.15.17:0/1157803402 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07dc157280 con 0x7f07dc0b1f40 2022-08-01T19:37:40.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.614+0000 7f07d97fa700 1 -- 172.21.15.17:0/1157803402 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07cc01f8b0 con 0x7f07dc0b1f40 2022-08-01T19:37:40.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.614+0000 7f07e299f700 1 -- 172.21.15.17:0/1157803402 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07dc0ae380 con 0x7f07dc0b1f40 2022-08-01T19:37:40.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.614+0000 7f07e299f700 1 -- 172.21.15.17:0/1157803402 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07dc157410 con 0x7f07dc0b1f40 2022-08-01T19:37:40.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.614+0000 7f07d97fa700 1 -- 172.21.15.17:0/1157803402 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f07cc026030 con 0x7f07dc0b1f40 2022-08-01T19:37:40.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.614+0000 7f07d97fa700 1 --2- 172.21.15.17:0/1157803402 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f07bc053aa0 0x7f07bc055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:40.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.614+0000 7f07d97fa700 1 -- 172.21.15.17:0/1157803402 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f07cc06cc70 con 0x7f07dc0b1f40 2022-08-01T19:37:40.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.614+0000 7f07dbfff700 1 --2- 172.21.15.17:0/1157803402 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f07bc053aa0 0x7f07bc055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:40.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.614+0000 7f07dbfff700 1 --2- 172.21.15.17:0/1157803402 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f07bc053aa0 0x7f07bc055f60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f07d000b000 tx=0x7f07d0009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:40.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.622+0000 7f07d97fa700 1 -- 172.21.15.17:0/1157803402 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f07cc043a30 con 0x7f07dc0b1f40 2022-08-01T19:37:40.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.786+0000 7f07e299f700 1 -- 172.21.15.17:0/1157803402 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07dc0b0cc0 con 0x7f07dc0b1f40 2022-08-01T19:37:40.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.790+0000 7f07d97fa700 1 -- 172.21.15.17:0/1157803402 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f07cc024090 con 0x7f07dc0b1f40 2022-08-01T19:37:40.794 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:37:40.795 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":24,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/1914125901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":1914125901},{"type":"v1","addr":"172.21.15.17:6833","nonce":1914125901}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":22},{"gid":5587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1403688289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1403688289},{"type":"v1","addr":"172.21.15.77:6835","nonce":1403688289}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":22}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:37:36.197818+0000","modified":"2022-08-01T19:37:37.201700+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5589":{"gid":5589,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/3325991310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":3325991310},{"type":"v1","addr":"172.21.15.17:6835","nonce":3325991310}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-08-01T19:37:40.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.790+0000 7f07e299f700 1 -- 172.21.15.17:0/1157803402 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f07bc053aa0 msgr2=0x7f07bc055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:40.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.790+0000 7f07e299f700 1 --2- 172.21.15.17:0/1157803402 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f07bc053aa0 0x7f07bc055f60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f07d000b000 tx=0x7f07d0009040).stop 2022-08-01T19:37:40.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.790+0000 7f07e299f700 1 -- 172.21.15.17:0/1157803402 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07dc0b1f40 msgr2=0x7f07dc157720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:40.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.790+0000 7f07e299f700 1 --2- 172.21.15.17:0/1157803402 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07dc0b1f40 0x7f07dc157720 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f07dc0b3300 tx=0x7f07cc016000).stop 2022-08-01T19:37:40.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.790+0000 7f07e299f700 1 -- 172.21.15.17:0/1157803402 shutdown_connections 2022-08-01T19:37:40.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.790+0000 7f07e299f700 1 --2- 172.21.15.17:0/1157803402 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f07bc053aa0 0x7f07bc055f60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:40.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.790+0000 7f07e299f700 1 --2- 172.21.15.17:0/1157803402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07dc148530 0x7f07dc0ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:40.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.790+0000 7f07e299f700 1 --2- 172.21.15.17:0/1157803402 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07dc0b1f40 0x7f07dc157720 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:40.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.790+0000 7f07e299f700 1 --2- 172.21.15.17:0/1157803402 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f07dc0a7250 0x7f07dc149b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:40.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.790+0000 7f07e299f700 1 -- 172.21.15.17:0/1157803402 >> 172.21.15.17:0/1157803402 conn(0x7f07dc05f840 msgr2=0x7f07dc144ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:40.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.790+0000 7f07e299f700 1 -- 172.21.15.17:0/1157803402 shutdown_connections 2022-08-01T19:37:40.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:40.790+0000 7f07e299f700 1 -- 172.21.15.17:0/1157803402 wait complete. 2022-08-01T19:37:40.801 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 24 2022-08-01T19:37:41.108 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:37:36.197818+0000', 'modified': '2022-08-01T19:37:37.201700+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5589}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5589': {'gid': 5589, 'name': 'c', 'rank': 0, 'incarnation': 23, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/3325991310', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 3325991310}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 3325991310}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:37:41.109 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:37:41.110 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T19:37:41.230 INFO:teuthology.orchestra.run.smithi017.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":5589,"want_state":"up:active","state":"up:active","rank_uptime":5.0224536259999999,"mdsmap_epoch":24,"osdmap_epoch":55,"osdmap_epoch_barrier":55,"uptime":9.3834868040000003} 2022-08-01T19:37:41.231 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 5589, "mdsmap_epoch": 24, "osdmap_epoch": 55, "osdmap_epoch_barrier": 55, "rank_uptime": 5.022453626, "state": "up:active", "uptime": 9.383486804, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:37:41.232 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T19:37:41.232 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-08-01T19:37:41.232 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi085.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-08-01T19:37:41.233 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:37:41.233 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:37:41.242 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:37:41.300 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:37:41.301 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:37:41.313 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:37:41.314 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:37:41.315 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:37:41.316 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:37:41.322 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T19:37:41.322 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:37:41.365 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:37:41.385 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:37:41.386 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:37:41.400 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:37:41.401 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:37:41.403 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:37:41.403 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:37:41.439 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:2022-08-01T19:37:41.439+0000 7f959dc8f080 -1 init, newargv = 0x55efc74c3800 newargc=9 2022-08-01T19:37:41.439 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:ceph-fuse[22750]: starting ceph client 2022-08-01T19:37:41.449 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:ceph-fuse[22750]: starting fuse 2022-08-01T19:37:42.547 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:37:42.567 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:37:42.567 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:37:42.626 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:37:42.628 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:37:42.630 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:37:42.631 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:37:42.639 INFO:teuthology.orchestra.run.smithi085.stdout:52 2022-08-01T19:37:42.639 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T19:37:42.639 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:37:42.640 DEBUG:teuthology.orchestra.run.smithi085:> sudo python3 -c ' 2022-08-01T19:37:42.640 DEBUG:teuthology.orchestra.run.smithi085:> import glob 2022-08-01T19:37:42.641 DEBUG:teuthology.orchestra.run.smithi085:> import re 2022-08-01T19:37:42.641 DEBUG:teuthology.orchestra.run.smithi085:> import os 2022-08-01T19:37:42.642 DEBUG:teuthology.orchestra.run.smithi085:> import subprocess 2022-08-01T19:37:42.643 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:37:42.644 DEBUG:teuthology.orchestra.run.smithi085:> def find_socket(client_name): 2022-08-01T19:37:42.645 DEBUG:teuthology.orchestra.run.smithi085:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-08-01T19:37:42.645 DEBUG:teuthology.orchestra.run.smithi085:> files = glob.glob(asok_path) 2022-08-01T19:37:42.646 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:37:42.647 DEBUG:teuthology.orchestra.run.smithi085:> # Given a non-glob path, it better be there 2022-08-01T19:37:42.648 DEBUG:teuthology.orchestra.run.smithi085:> if "*" not in asok_path: 2022-08-01T19:37:42.649 DEBUG:teuthology.orchestra.run.smithi085:> assert(len(files) == 1) 2022-08-01T19:37:42.650 DEBUG:teuthology.orchestra.run.smithi085:> return files[0] 2022-08-01T19:37:42.650 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:37:42.651 DEBUG:teuthology.orchestra.run.smithi085:> for f in files: 2022-08-01T19:37:42.652 DEBUG:teuthology.orchestra.run.smithi085:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T19:37:42.652 DEBUG:teuthology.orchestra.run.smithi085:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T19:37:42.653 DEBUG:teuthology.orchestra.run.smithi085:> return f 2022-08-01T19:37:42.654 DEBUG:teuthology.orchestra.run.smithi085:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T19:37:42.654 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:37:42.655 DEBUG:teuthology.orchestra.run.smithi085:> print(find_socket("client.0")) 2022-08-01T19:37:42.655 DEBUG:teuthology.orchestra.run.smithi085:> ' 2022-08-01T19:37:42.729 INFO:teuthology.orchestra.run.smithi085.stdout:/var/run/ceph/ceph-client.0.22750.asok 2022-08-01T19:37:42.734 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22750.asok 2022-08-01T19:37:42.734 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:37:42.735 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22750.asok status 2022-08-01T19:37:42.821 INFO:teuthology.orchestra.run.smithi085.stdout:{ 2022-08-01T19:37:42.822 INFO:teuthology.orchestra.run.smithi085.stdout: "metadata": { 2022-08-01T19:37:42.822 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T19:37:42.822 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T19:37:42.822 INFO:teuthology.orchestra.run.smithi085.stdout: "entity_id": "0", 2022-08-01T19:37:42.823 INFO:teuthology.orchestra.run.smithi085.stdout: "hostname": "smithi085", 2022-08-01T19:37:42.823 INFO:teuthology.orchestra.run.smithi085.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-08-01T19:37:42.823 INFO:teuthology.orchestra.run.smithi085.stdout: "pid": "22750", 2022-08-01T19:37:42.823 INFO:teuthology.orchestra.run.smithi085.stdout: "root": "/" 2022-08-01T19:37:42.824 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:37:42.824 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_count": 0, 2022-08-01T19:37:42.824 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_pinned_count": 0, 2022-08-01T19:37:42.824 INFO:teuthology.orchestra.run.smithi085.stdout: "id": 5757, 2022-08-01T19:37:42.825 INFO:teuthology.orchestra.run.smithi085.stdout: "inst": { 2022-08-01T19:37:42.825 INFO:teuthology.orchestra.run.smithi085.stdout: "name": { 2022-08-01T19:37:42.825 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "client", 2022-08-01T19:37:42.826 INFO:teuthology.orchestra.run.smithi085.stdout: "num": 5757 2022-08-01T19:37:42.826 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:37:42.826 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2022-08-01T19:37:42.826 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2022-08-01T19:37:42.827 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2022-08-01T19:37:42.827 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 3743997946 2022-08-01T19:37:42.827 INFO:teuthology.orchestra.run.smithi085.stdout: } 2022-08-01T19:37:42.827 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:37:42.828 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2022-08-01T19:37:42.828 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2022-08-01T19:37:42.828 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2022-08-01T19:37:42.828 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 3743997946 2022-08-01T19:37:42.829 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:37:42.829 INFO:teuthology.orchestra.run.smithi085.stdout: "inst_str": "client.5757 172.21.15.85:0/3743997946", 2022-08-01T19:37:42.830 INFO:teuthology.orchestra.run.smithi085.stdout: "addr_str": "172.21.15.85:0/3743997946", 2022-08-01T19:37:42.830 INFO:teuthology.orchestra.run.smithi085.stdout: "inode_count": 1, 2022-08-01T19:37:42.830 INFO:teuthology.orchestra.run.smithi085.stdout: "mds_epoch": 24, 2022-08-01T19:37:42.831 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch": 55, 2022-08-01T19:37:42.831 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch_barrier": 0, 2022-08-01T19:37:42.831 INFO:teuthology.orchestra.run.smithi085.stdout: "blacklisted": false, 2022-08-01T19:37:42.831 INFO:teuthology.orchestra.run.smithi085.stdout: "fs_name": "cephfs" 2022-08-01T19:37:42.831 INFO:teuthology.orchestra.run.smithi085.stdout:} 2022-08-01T19:37:42.833 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:37:42.833 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:37:42.834 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:37:42.840 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:37:42.841 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:37:42.842 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:37:42.842 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:37:42.901 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:37:42.902 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:37:42.902 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:37:42.910 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:37:42.912 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:37:42.913 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-08-01T19:37:42.914 DEBUG:teuthology.orchestra.run.smithi085:> import sys 2022-08-01T19:37:42.914 DEBUG:teuthology.orchestra.run.smithi085:> import time 2022-08-01T19:37:42.915 DEBUG:teuthology.orchestra.run.smithi085:> import os 2022-08-01T19:37:42.916 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:37:42.916 DEBUG:teuthology.orchestra.run.smithi085:> n = 1000 2022-08-01T19:37:42.917 DEBUG:teuthology.orchestra.run.smithi085:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2022-08-01T19:37:42.917 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:37:42.918 DEBUG:teuthology.orchestra.run.smithi085:> if not os.path.exists(os.path.dirname(abs_path)): 2022-08-01T19:37:42.918 DEBUG:teuthology.orchestra.run.smithi085:> os.makedirs(os.path.dirname(abs_path)) 2022-08-01T19:37:42.918 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:37:42.919 DEBUG:teuthology.orchestra.run.smithi085:> for i in range(0, n): 2022-08-01T19:37:42.919 DEBUG:teuthology.orchestra.run.smithi085:> fname = "{0}_{1}".format(abs_path, i) 2022-08-01T19:37:42.920 DEBUG:teuthology.orchestra.run.smithi085:> with open(fname, '"'"'w'"'"') as f: 2022-08-01T19:37:42.920 DEBUG:teuthology.orchestra.run.smithi085:> f.write('"'"'content'"'"') 2022-08-01T19:37:42.921 DEBUG:teuthology.orchestra.run.smithi085:> if True: 2022-08-01T19:37:42.921 DEBUG:teuthology.orchestra.run.smithi085:> f.flush() 2022-08-01T19:37:42.921 DEBUG:teuthology.orchestra.run.smithi085:> os.fsync(f.fileno()) 2022-08-01T19:37:42.922 DEBUG:teuthology.orchestra.run.smithi085:> ' 2022-08-01T19:37:47.415 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:37:47.418 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:37:47.420 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:37:47.423 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:37:54.206 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:37:54.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.318+0000 7f303bfff700 1 Processor -- start 2022-08-01T19:37:54.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.318+0000 7f303bfff700 1 -- start start 2022-08-01T19:37:54.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.318+0000 7f303bfff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f303c0b2710 0x7f303c14afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:54.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.318+0000 7f303bfff700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f303c0a7250 con 0x7f303c0a8c50 2022-08-01T19:37:54.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.318+0000 7f303bfff700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f303c0a7480 con 0x7f303c0b01a0 2022-08-01T19:37:54.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.318+0000 7f303bfff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f303c0a7720 con 0x7f303c0b2710 2022-08-01T19:37:54.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.318+0000 7f303b7fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f303c0a8c50 0x7f303c151d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59698/0 (socket says 172.21.15.17:59698) 2022-08-01T19:37:54.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.318+0000 7f303b7fe700 1 -- 172.21.15.17:0/220566415 learned_addr learned my addr 172.21.15.17:0/220566415 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:37:54.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.318+0000 7f303affd700 1 --2- 172.21.15.17:0/220566415 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f303c0b2710 0x7f303c14afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:54.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.318+0000 7f3039ffb700 1 -- 172.21.15.17:0/220566415 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1336066439 0 0) 0x7f303c0a7480 con 0x7f303c0b01a0 2022-08-01T19:37:54.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.318+0000 7f3039ffb700 1 -- 172.21.15.17:0/220566415 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3020003260 con 0x7f303c0b01a0 2022-08-01T19:37:54.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.318+0000 7f3039ffb700 1 -- 172.21.15.17:0/220566415 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3373367058 0 0) 0x7f303c0a7250 con 0x7f303c0a8c50 2022-08-01T19:37:54.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.318+0000 7f3039ffb700 1 -- 172.21.15.17:0/220566415 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f303c0a7480 con 0x7f303c0a8c50 2022-08-01T19:37:54.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.318+0000 7f303affd700 1 -- 172.21.15.17:0/220566415 >> v1:172.21.15.77:6789/0 conn(0x7f303c0b01a0 legacy=0x7f303c0a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:54.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.318+0000 7f303affd700 1 -- 172.21.15.17:0/220566415 >> v1:172.21.15.17:6789/0 conn(0x7f303c0a8c50 legacy=0x7f303c151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:54.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.318+0000 7f303affd700 1 -- 172.21.15.17:0/220566415 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f303c0a79c0 con 0x7f303c0b2710 2022-08-01T19:37:54.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.318+0000 7f303affd700 1 --2- 172.21.15.17:0/220566415 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f303c0b2710 0x7f303c14afd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f3030002f70 tx=0x7f3030006df0).ready entity=mon.2 client_cookie=77f313328f182761 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:54.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.318+0000 7f3039ffb700 1 -- 172.21.15.17:0/220566415 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f303000d5c0 con 0x7f303c0b2710 2022-08-01T19:37:54.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.318+0000 7f3039ffb700 1 -- 172.21.15.17:0/220566415 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3030000e90 con 0x7f303c0b2710 2022-08-01T19:37:54.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.318+0000 7f3039ffb700 1 -- 172.21.15.17:0/220566415 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f303000d910 con 0x7f303c0b2710 2022-08-01T19:37:54.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.318+0000 7f303bfff700 1 -- 172.21.15.17:0/220566415 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f303c0b2710 msgr2=0x7f303c14afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:54.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.318+0000 7f303bfff700 1 --2- 172.21.15.17:0/220566415 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f303c0b2710 0x7f303c14afd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f3030002f70 tx=0x7f3030006df0).stop 2022-08-01T19:37:54.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.318+0000 7f303bfff700 1 -- 172.21.15.17:0/220566415 shutdown_connections 2022-08-01T19:37:54.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.318+0000 7f303bfff700 1 --2- 172.21.15.17:0/220566415 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f303c0b2710 0x7f303c14afd0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:54.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.318+0000 7f303bfff700 1 -- 172.21.15.17:0/220566415 >> 172.21.15.17:0/220566415 conn(0x7f303c060310 msgr2=0x7f303c0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:54.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.318+0000 7f303bfff700 1 -- 172.21.15.17:0/220566415 shutdown_connections 2022-08-01T19:37:54.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.318+0000 7f303bfff700 1 -- 172.21.15.17:0/220566415 wait complete. 2022-08-01T19:37:54.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.318+0000 7f303bfff700 1 Processor -- start 2022-08-01T19:37:54.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.318+0000 7f303bfff700 1 -- start start 2022-08-01T19:37:54.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.322+0000 7f303bfff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f303c0a8c50 0x7f303c0b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:54.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.322+0000 7f303bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f303c0b01a0 0x7f303c1411f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:54.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.322+0000 7f303bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f303c0b2710 0x7f303c1416e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:54.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.322+0000 7f303bfff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303c0b5fa0 con 0x7f303c0b2710 2022-08-01T19:37:54.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.322+0000 7f303bfff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303c0aa6c0 con 0x7f303c0b01a0 2022-08-01T19:37:54.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.322+0000 7f303bfff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f303c0b4890 con 0x7f303c0a8c50 2022-08-01T19:37:54.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.322+0000 7f303affd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f303c0a8c50 0x7f303c0b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:54.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.322+0000 7f303b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f303c0b2710 0x7f303c1416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:54.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.322+0000 7f303affd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f303c0a8c50 0x7f303c0b4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:35374/0 (socket says 172.21.15.17:35374) 2022-08-01T19:37:54.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.322+0000 7f303affd700 1 -- 172.21.15.17:0/1697695149 learned_addr learned my addr 172.21.15.17:0/1697695149 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:54.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.322+0000 7f303a7fc700 1 --2- 172.21.15.17:0/1697695149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f303c0b01a0 0x7f303c1411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:54.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.322+0000 7f303affd700 1 -- 172.21.15.17:0/1697695149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f303c0b01a0 msgr2=0x7f303c1411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:54.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.322+0000 7f303affd700 1 --2- 172.21.15.17:0/1697695149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f303c0b01a0 0x7f303c1411f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:54.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.322+0000 7f303affd700 1 -- 172.21.15.17:0/1697695149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f303c0b2710 msgr2=0x7f303c1416e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:54.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.322+0000 7f303affd700 1 --2- 172.21.15.17:0/1697695149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f303c0b2710 0x7f303c1416e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:54.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.322+0000 7f303affd700 1 -- 172.21.15.17:0/1697695149 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f303000b180 con 0x7f303c0a8c50 2022-08-01T19:37:54.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.322+0000 7f303affd700 1 --2- 172.21.15.17:0/1697695149 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f303c0a8c50 0x7f303c0b4f40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f3030002f70 tx=0x7f3030014000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:54.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.322+0000 7f302bfff700 1 -- 172.21.15.17:0/1697695149 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3030016070 con 0x7f303c0a8c50 2022-08-01T19:37:54.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.322+0000 7f302bfff700 1 -- 172.21.15.17:0/1697695149 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30300137d0 con 0x7f303c0a8c50 2022-08-01T19:37:54.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.322+0000 7f302bfff700 1 -- 172.21.15.17:0/1697695149 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3030013b30 con 0x7f303c0a8c50 2022-08-01T19:37:54.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.322+0000 7f303bfff700 1 -- 172.21.15.17:0/1697695149 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f303c14d590 con 0x7f303c0a8c50 2022-08-01T19:37:54.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.322+0000 7f303bfff700 1 -- 172.21.15.17:0/1697695149 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f303c0b30b0 con 0x7f303c0a8c50 2022-08-01T19:37:54.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.326+0000 7f303bfff700 1 -- 172.21.15.17:0/1697695149 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f303c0b3270 con 0x7f303c0a8c50 2022-08-01T19:37:54.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.326+0000 7f302bfff700 1 -- 172.21.15.17:0/1697695149 <== mon.2 v2:172.21.15.77:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f303000f030 con 0x7f303c0a8c50 2022-08-01T19:37:54.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.326+0000 7f302bfff700 1 --2- 172.21.15.17:0/1697695149 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3018053a70 0x7f3018055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:54.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.326+0000 7f302bfff700 1 -- 172.21.15.17:0/1697695149 <== mon.2 v2:172.21.15.77:3301/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f303006cb60 con 0x7f303c0a8c50 2022-08-01T19:37:54.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.326+0000 7f303a7fc700 1 --2- 172.21.15.17:0/1697695149 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3018053a70 0x7f3018055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:54.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.326+0000 7f303a7fc700 1 --2- 172.21.15.17:0/1697695149 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3018053a70 0x7f3018055f30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f302c00d000 tx=0x7f302c007dc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:54.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.338+0000 7f302bfff700 1 -- 172.21.15.17:0/1697695149 <== mon.2 v2:172.21.15.77:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f30300438c0 con 0x7f303c0a8c50 2022-08-01T19:37:54.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.502+0000 7f303bfff700 1 -- 172.21.15.17:0/1697695149 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f303c0574e0 con 0x7f303c0a8c50 2022-08-01T19:37:54.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.502+0000 7f302bfff700 1 -- 172.21.15.17:0/1697695149 <== mon.2 v2:172.21.15.77:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f3030023020 con 0x7f303c0a8c50 2022-08-01T19:37:54.507 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:37:54.507 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":24,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/1914125901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":1914125901},{"type":"v1","addr":"172.21.15.17:6833","nonce":1914125901}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":22},{"gid":5587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1403688289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1403688289},{"type":"v1","addr":"172.21.15.77:6835","nonce":1403688289}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":22}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:37:36.197818+0000","modified":"2022-08-01T19:37:37.201700+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5589":{"gid":5589,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/3325991310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":3325991310},{"type":"v1","addr":"172.21.15.17:6835","nonce":3325991310}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-08-01T19:37:54.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.506+0000 7f303bfff700 1 -- 172.21.15.17:0/1697695149 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3018053a70 msgr2=0x7f3018055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:54.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.506+0000 7f303bfff700 1 --2- 172.21.15.17:0/1697695149 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3018053a70 0x7f3018055f30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f302c00d000 tx=0x7f302c007dc0).stop 2022-08-01T19:37:54.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.506+0000 7f303bfff700 1 -- 172.21.15.17:0/1697695149 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f303c0a8c50 msgr2=0x7f303c0b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:54.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.506+0000 7f303bfff700 1 --2- 172.21.15.17:0/1697695149 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f303c0a8c50 0x7f303c0b4f40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f3030002f70 tx=0x7f3030014000).stop 2022-08-01T19:37:54.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.506+0000 7f303bfff700 1 -- 172.21.15.17:0/1697695149 shutdown_connections 2022-08-01T19:37:54.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.506+0000 7f303bfff700 1 --2- 172.21.15.17:0/1697695149 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3018053a70 0x7f3018055f30 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:54.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.506+0000 7f303bfff700 1 --2- 172.21.15.17:0/1697695149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f303c0b2710 0x7f303c1416e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:54.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.506+0000 7f303bfff700 1 --2- 172.21.15.17:0/1697695149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f303c0b01a0 0x7f303c1411f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:54.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.506+0000 7f303bfff700 1 --2- 172.21.15.17:0/1697695149 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f303c0a8c50 0x7f303c0b4f40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:54.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.506+0000 7f303bfff700 1 -- 172.21.15.17:0/1697695149 >> 172.21.15.17:0/1697695149 conn(0x7f303c060310 msgr2=0x7f303c149c60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:54.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.506+0000 7f303bfff700 1 -- 172.21.15.17:0/1697695149 shutdown_connections 2022-08-01T19:37:54.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.506+0000 7f303bfff700 1 -- 172.21.15.17:0/1697695149 wait complete. 2022-08-01T19:37:54.512 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 24 2022-08-01T19:37:54.519 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_recall_max_caps 20 2022-08-01T19:37:54.620 INFO:teuthology.orchestra.run.smithi017.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2022-08-01T19:37:54.621 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2022-08-01T19:37:54.622 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:37:54.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.722+0000 7f1725d94700 1 Processor -- start 2022-08-01T19:37:54.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.722+0000 7f1725d94700 1 -- start start 2022-08-01T19:37:54.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.722+0000 7f1725d94700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f17200b01a0 0x7f17200a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:54.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.722+0000 7f1725d94700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f17200afd60 0x7f17200a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:54.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.722+0000 7f1725d94700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17200a7250 con 0x7f17200b2710 2022-08-01T19:37:54.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.722+0000 7f1725d94700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17200a7480 con 0x7f17200b01a0 2022-08-01T19:37:54.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.722+0000 7f1725d94700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17200a7720 con 0x7f17200afd60 2022-08-01T19:37:54.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.722+0000 7f1724d92700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f17200b2710 0x7f172014afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59708/0 (socket says 172.21.15.17:59708) 2022-08-01T19:37:54.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.722+0000 7f1724d92700 1 -- 172.21.15.17:0/3282828191 learned_addr learned my addr 172.21.15.17:0/3282828191 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:37:54.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.722+0000 7f171ffff700 1 --2- 172.21.15.17:0/3282828191 >> v2:172.21.15.77:3300/0 conn(0x7f17200b01a0 0x7f17200a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:54.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.722+0000 7f1725593700 1 --2- 172.21.15.17:0/3282828191 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f17200afd60 0x7f17200a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:54.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.722+0000 7f171f7fe700 1 -- 172.21.15.17:0/3282828191 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1449856653 0 0) 0x7f17200a7250 con 0x7f17200b2710 2022-08-01T19:37:54.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.722+0000 7f171f7fe700 1 -- 172.21.15.17:0/3282828191 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1700003260 con 0x7f17200b2710 2022-08-01T19:37:54.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.726+0000 7f171ffff700 1 -- 172.21.15.17:0/3282828191 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f17200afd60 msgr2=0x7f17200a8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:54.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.726+0000 7f171ffff700 1 --2- 172.21.15.17:0/3282828191 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f17200afd60 0x7f17200a8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:54.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.726+0000 7f171ffff700 1 -- 172.21.15.17:0/3282828191 >> v1:172.21.15.17:6789/0 conn(0x7f17200b2710 legacy=0x7f172014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:54.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.726+0000 7f171ffff700 1 -- 172.21.15.17:0/3282828191 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17200a79c0 con 0x7f17200b01a0 2022-08-01T19:37:54.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.726+0000 7f171ffff700 1 --2- 172.21.15.17:0/3282828191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17200b01a0 0x7f17200a8540 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f17100032f0 tx=0x7f17100091d0).ready entity=mon.1 client_cookie=e24185096e07c9d1 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:54.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.726+0000 7f171f7fe700 1 -- 172.21.15.17:0/3282828191 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f171000b600 con 0x7f17200b01a0 2022-08-01T19:37:54.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.726+0000 7f171f7fe700 1 -- 172.21.15.17:0/3282828191 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f171000b760 con 0x7f17200b01a0 2022-08-01T19:37:54.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.726+0000 7f171f7fe700 1 -- 172.21.15.17:0/3282828191 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f171000b8d0 con 0x7f17200b01a0 2022-08-01T19:37:54.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.726+0000 7f1725d94700 1 -- 172.21.15.17:0/3282828191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17200b01a0 msgr2=0x7f17200a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:54.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.726+0000 7f1725d94700 1 --2- 172.21.15.17:0/3282828191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17200b01a0 0x7f17200a8540 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f17100032f0 tx=0x7f17100091d0).stop 2022-08-01T19:37:54.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.726+0000 7f1725d94700 1 -- 172.21.15.17:0/3282828191 shutdown_connections 2022-08-01T19:37:54.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.726+0000 7f1725d94700 1 --2- 172.21.15.17:0/3282828191 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f17200afd60 0x7f17200a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:54.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.726+0000 7f1725d94700 1 --2- 172.21.15.17:0/3282828191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17200b01a0 0x7f17200a8540 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:54.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.726+0000 7f1725d94700 1 -- 172.21.15.17:0/3282828191 >> 172.21.15.17:0/3282828191 conn(0x7f1720060310 msgr2=0x7f17200b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:54.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.726+0000 7f1725d94700 1 -- 172.21.15.17:0/3282828191 shutdown_connections 2022-08-01T19:37:54.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.726+0000 7f1725d94700 1 -- 172.21.15.17:0/3282828191 wait complete. 2022-08-01T19:37:54.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.730+0000 7f1725d94700 1 Processor -- start 2022-08-01T19:37:54.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.730+0000 7f1725d94700 1 -- start start 2022-08-01T19:37:54.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.730+0000 7f1725d94700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17200afd60 0x7f17200b4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:54.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.730+0000 7f1725d94700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17200b01a0 0x7f172014a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:54.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.730+0000 7f1725d94700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f17200b2710 0x7f172014aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:54.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.730+0000 7f1725d94700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17200b5e40 con 0x7f17200b01a0 2022-08-01T19:37:54.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.730+0000 7f1725d94700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1720140fc0 con 0x7f17200afd60 2022-08-01T19:37:54.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.730+0000 7f1725d94700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17200b5ba0 con 0x7f17200b2710 2022-08-01T19:37:54.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.730+0000 7f171ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17200b01a0 0x7f172014a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:54.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.730+0000 7f171ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17200b01a0 0x7f172014a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39290/0 (socket says 172.21.15.17:39290) 2022-08-01T19:37:54.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.730+0000 7f171ffff700 1 -- 172.21.15.17:0/131905025 learned_addr learned my addr 172.21.15.17:0/131905025 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:54.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.730+0000 7f1725593700 1 --2- 172.21.15.17:0/131905025 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f17200b2710 0x7f172014aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:54.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.730+0000 7f1724d92700 1 --2- 172.21.15.17:0/131905025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17200afd60 0x7f17200b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:54.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.730+0000 7f171ffff700 1 -- 172.21.15.17:0/131905025 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f17200b2710 msgr2=0x7f172014aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:54.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.730+0000 7f171ffff700 1 --2- 172.21.15.17:0/131905025 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f17200b2710 0x7f172014aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:54.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.730+0000 7f171ffff700 1 -- 172.21.15.17:0/131905025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17200afd60 msgr2=0x7f17200b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:54.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.730+0000 7f171ffff700 1 --2- 172.21.15.17:0/131905025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17200afd60 0x7f17200b4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:54.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.730+0000 7f171ffff700 1 -- 172.21.15.17:0/131905025 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1710009470 con 0x7f17200b01a0 2022-08-01T19:37:54.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.730+0000 7f171ffff700 1 --2- 172.21.15.17:0/131905025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17200b01a0 0x7f172014a5f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f17100032f0 tx=0x7f1710016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:54.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.730+0000 7f171dffb700 1 -- 172.21.15.17:0/131905025 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f171000b8d0 con 0x7f17200b01a0 2022-08-01T19:37:54.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.730+0000 7f171dffb700 1 -- 172.21.15.17:0/131905025 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1710021630 con 0x7f17200b01a0 2022-08-01T19:37:54.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.730+0000 7f1725d94700 1 -- 172.21.15.17:0/131905025 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f172014a150 con 0x7f17200b01a0 2022-08-01T19:37:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.730+0000 7f171dffb700 1 -- 172.21.15.17:0/131905025 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f171000b600 con 0x7f17200b01a0 2022-08-01T19:37:54.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.730+0000 7f1725d94700 1 -- 172.21.15.17:0/131905025 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17200b37f0 con 0x7f17200b01a0 2022-08-01T19:37:54.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.734+0000 7f1725d94700 1 -- 172.21.15.17:0/131905025 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1720149f90 con 0x7f17200b01a0 2022-08-01T19:37:54.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.734+0000 7f171dffb700 1 -- 172.21.15.17:0/131905025 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f171000f070 con 0x7f17200b01a0 2022-08-01T19:37:54.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.734+0000 7f171dffb700 1 --2- 172.21.15.17:0/131905025 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1704053aa0 0x7f1704055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:54.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.734+0000 7f171dffb700 1 -- 172.21.15.17:0/131905025 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f171006d5f0 con 0x7f17200b01a0 2022-08-01T19:37:54.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.734+0000 7f1724d92700 1 --2- 172.21.15.17:0/131905025 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1704053aa0 0x7f1704055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:54.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.734+0000 7f1724d92700 1 --2- 172.21.15.17:0/131905025 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1704053aa0 0x7f1704055f60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f170800d000 tx=0x7f1708009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:54.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.738+0000 7f171dffb700 1 -- 172.21.15.17:0/131905025 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f17100443b0 con 0x7f17200b01a0 2022-08-01T19:37:54.905 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.902+0000 7f1725d94700 1 -- 172.21.15.17:0/131905025 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f172014a2e0 con 0x7f17200b01a0 2022-08-01T19:37:54.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.902+0000 7f171dffb700 1 -- 172.21.15.17:0/131905025 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f17100159d0 con 0x7f17200b01a0 2022-08-01T19:37:54.907 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:37:54.907 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":24,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/1914125901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":1914125901},{"type":"v1","addr":"172.21.15.17:6833","nonce":1914125901}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":22},{"gid":5587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1403688289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1403688289},{"type":"v1","addr":"172.21.15.77:6835","nonce":1403688289}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":22}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:37:36.197818+0000","modified":"2022-08-01T19:37:37.201700+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5589":{"gid":5589,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/3325991310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":3325991310},{"type":"v1","addr":"172.21.15.17:6835","nonce":3325991310}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-08-01T19:37:54.909 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.906+0000 7f1725d94700 1 -- 172.21.15.17:0/131905025 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1704053aa0 msgr2=0x7f1704055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:54.909 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.906+0000 7f1725d94700 1 --2- 172.21.15.17:0/131905025 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1704053aa0 0x7f1704055f60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f170800d000 tx=0x7f1708009040).stop 2022-08-01T19:37:54.909 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.906+0000 7f1725d94700 1 -- 172.21.15.17:0/131905025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17200b01a0 msgr2=0x7f172014a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:54.909 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.906+0000 7f1725d94700 1 --2- 172.21.15.17:0/131905025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17200b01a0 0x7f172014a5f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f17100032f0 tx=0x7f1710016000).stop 2022-08-01T19:37:54.910 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.906+0000 7f1725d94700 1 -- 172.21.15.17:0/131905025 shutdown_connections 2022-08-01T19:37:54.910 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.906+0000 7f1725d94700 1 --2- 172.21.15.17:0/131905025 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1704053aa0 0x7f1704055f60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:54.910 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.906+0000 7f1725d94700 1 --2- 172.21.15.17:0/131905025 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f17200b2710 0x7f172014aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:54.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.906+0000 7f1725d94700 1 --2- 172.21.15.17:0/131905025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17200b01a0 0x7f172014a5f0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:54.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.906+0000 7f1725d94700 1 --2- 172.21.15.17:0/131905025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17200afd60 0x7f17200b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:54.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.906+0000 7f1725d94700 1 -- 172.21.15.17:0/131905025 >> 172.21.15.17:0/131905025 conn(0x7f1720060310 msgr2=0x7f1720148790 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:54.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.906+0000 7f1725d94700 1 -- 172.21.15.17:0/131905025 shutdown_connections 2022-08-01T19:37:54.912 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:54.906+0000 7f1725d94700 1 -- 172.21.15.17:0/131905025 wait complete. 2022-08-01T19:37:54.912 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 24 2022-08-01T19:37:54.919 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_recall_max_decay_threshold 400 2022-08-01T19:37:55.018 INFO:teuthology.orchestra.run.smithi017.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2022-08-01T19:37:55.019 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2022-08-01T19:37:55.019 INFO:tasks.cephfs.fuse_mount:Killing ceph-fuse connection on ubuntu@smithi085.front.sepia.ceph.com... 2022-08-01T19:37:55.038 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:37:55.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.146+0000 7f40b7a54700 1 Processor -- start 2022-08-01T19:37:55.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.146+0000 7f40b7a54700 1 -- start start 2022-08-01T19:37:55.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.146+0000 7f40b7a54700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f40b00b0630 0x7f40b00adec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:55.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.146+0000 7f40b7a54700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f40b00aa2b0 0x7f40b00ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:55.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.146+0000 7f40b7a54700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f40b00b0a10 0x7f40b01438a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:55.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.146+0000 7f40b7a54700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40b00b2c80 con 0x7f40b00aa2b0 2022-08-01T19:37:55.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.146+0000 7f40b7a54700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40b00b2900 con 0x7f40b00b0630 2022-08-01T19:37:55.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.146+0000 7f40b7a54700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b00b2eb0 con 0x7f40b00b0a10 2022-08-01T19:37:55.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.146+0000 7f40b4fef700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f40b00aa2b0 0x7f40b00ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:55.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.146+0000 7f40b5ff1700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f40b00b0a10 0x7f40b01438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:55.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.146+0000 7f40b5ff1700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f40b00b0a10 0x7f40b01438a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:35398/0 (socket says 172.21.15.17:35398) 2022-08-01T19:37:55.162 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.146+0000 7f40b4fef700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f40b00aa2b0 0x7f40b00ae3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39300/0 (socket says 172.21.15.17:39300) 2022-08-01T19:37:55.162 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.146+0000 7f40b5ff1700 1 -- 172.21.15.17:0/1874723818 learned_addr learned my addr 172.21.15.17:0/1874723818 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:55.163 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.146+0000 7f40b57f0700 1 --2- 172.21.15.17:0/1874723818 >> v2:172.21.15.77:3300/0 conn(0x7f40b00b0630 0x7f40b00adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:55.163 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.146+0000 7f40b5ff1700 1 -- 172.21.15.17:0/1874723818 >> v2:172.21.15.77:3300/0 conn(0x7f40b00b0630 msgr2=0x7f40b00adec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:55.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.146+0000 7f40b5ff1700 1 --2- 172.21.15.17:0/1874723818 >> v2:172.21.15.77:3300/0 conn(0x7f40b00b0630 0x7f40b00adec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:55.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.146+0000 7f40b5ff1700 1 -- 172.21.15.17:0/1874723818 >> v2:172.21.15.17:3300/0 conn(0x7f40b00aa2b0 msgr2=0x7f40b00ae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:55.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.146+0000 7f40b5ff1700 1 --2- 172.21.15.17:0/1874723818 >> v2:172.21.15.17:3300/0 conn(0x7f40b00aa2b0 0x7f40b00ae3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:55.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.146+0000 7f40b5ff1700 1 -- 172.21.15.17:0/1874723818 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b00a9d00 con 0x7f40b00b0a10 2022-08-01T19:37:55.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.146+0000 7f40b57f0700 1 --2- 172.21.15.17:0/1874723818 >> v2:172.21.15.77:3300/0 conn(0x7f40b00b0630 0x7f40b00adec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:37:55.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.146+0000 7f40b5ff1700 1 --2- 172.21.15.17:0/1874723818 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f40b00b0a10 0x7f40b01438a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f40a80032f0 tx=0x7f40a8009290).ready entity=mon.2 client_cookie=5b1c86f9ef525652 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:55.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.146+0000 7f409ffff700 1 -- 172.21.15.17:0/1874723818 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40a800b870 con 0x7f40b00b0a10 2022-08-01T19:37:55.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.146+0000 7f409ffff700 1 -- 172.21.15.17:0/1874723818 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a800b9d0 con 0x7f40b00b0a10 2022-08-01T19:37:55.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.146+0000 7f409ffff700 1 -- 172.21.15.17:0/1874723818 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40a800bb40 con 0x7f40b00b0a10 2022-08-01T19:37:55.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.146+0000 7f40b7a54700 1 -- 172.21.15.17:0/1874723818 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f40b00b0a10 msgr2=0x7f40b01438a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:55.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.146+0000 7f40b7a54700 1 --2- 172.21.15.17:0/1874723818 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f40b00b0a10 0x7f40b01438a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f40a80032f0 tx=0x7f40a8009290).stop 2022-08-01T19:37:55.169 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.146+0000 7f40b7a54700 1 -- 172.21.15.17:0/1874723818 shutdown_connections 2022-08-01T19:37:55.169 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.146+0000 7f40b7a54700 1 --2- 172.21.15.17:0/1874723818 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f40b00b0a10 0x7f40b01438a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:55.170 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.146+0000 7f40b7a54700 1 --2- 172.21.15.17:0/1874723818 >> v2:172.21.15.17:3300/0 conn(0x7f40b00aa2b0 0x7f40b00ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:55.170 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.146+0000 7f40b7a54700 1 --2- 172.21.15.17:0/1874723818 >> v2:172.21.15.77:3300/0 conn(0x7f40b00b0630 0x7f40b00adec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:55.171 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.146+0000 7f40b7a54700 1 -- 172.21.15.17:0/1874723818 >> 172.21.15.17:0/1874723818 conn(0x7f40b0060950 msgr2=0x7f40b0060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:55.171 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.146+0000 7f40b7a54700 1 -- 172.21.15.17:0/1874723818 shutdown_connections 2022-08-01T19:37:55.171 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.146+0000 7f40b7a54700 1 -- 172.21.15.17:0/1874723818 wait complete. 2022-08-01T19:37:55.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.150+0000 7f40b7a54700 1 Processor -- start 2022-08-01T19:37:55.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.150+0000 7f40b7a54700 1 -- start start 2022-08-01T19:37:55.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.150+0000 7f40b7a54700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f40b00aa2b0 0x7f40b0147140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:55.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.150+0000 7f40b7a54700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40b00b0630 0x7f40b013dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:55.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.150+0000 7f40b7a54700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40b00b0a10 0x7f40b013e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:55.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.150+0000 7f40b7a54700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b013cb70 con 0x7f40b00b0630 2022-08-01T19:37:55.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.150+0000 7f40b7a54700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b00671e0 con 0x7f40b00b0a10 2022-08-01T19:37:55.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.150+0000 7f40b7a54700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b0060400 con 0x7f40b00aa2b0 2022-08-01T19:37:55.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.150+0000 7f40b4fef700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40b00b0630 0x7f40b013dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:55.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.150+0000 7f40b4fef700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40b00b0630 0x7f40b013dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39302/0 (socket says 172.21.15.17:39302) 2022-08-01T19:37:55.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.150+0000 7f40b4fef700 1 -- 172.21.15.17:0/1655683077 learned_addr learned my addr 172.21.15.17:0/1655683077 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:55.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.150+0000 7f40b57f0700 1 --2- 172.21.15.17:0/1655683077 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f40b00aa2b0 0x7f40b0147140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:55.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.150+0000 7f40b5ff1700 1 --2- 172.21.15.17:0/1655683077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40b00b0a10 0x7f40b013e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:55.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.150+0000 7f40b4fef700 1 -- 172.21.15.17:0/1655683077 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f40b00aa2b0 msgr2=0x7f40b0147140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:55.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.150+0000 7f40b4fef700 1 --2- 172.21.15.17:0/1655683077 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f40b00aa2b0 0x7f40b0147140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:55.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.150+0000 7f40b4fef700 1 -- 172.21.15.17:0/1655683077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40b00b0a10 msgr2=0x7f40b013e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:55.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.150+0000 7f40b4fef700 1 --2- 172.21.15.17:0/1655683077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40b00b0a10 0x7f40b013e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:55.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.150+0000 7f40b4fef700 1 -- 172.21.15.17:0/1655683077 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a8009530 con 0x7f40b00b0630 2022-08-01T19:37:55.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.150+0000 7f40b57f0700 1 --2- 172.21.15.17:0/1655683077 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f40b00aa2b0 0x7f40b0147140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:37:55.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.150+0000 7f40b4fef700 1 --2- 172.21.15.17:0/1655683077 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40b00b0630 0x7f40b013dec0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f40a000c000 tx=0x7f40a0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:55.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.150+0000 7f409e7fc700 1 -- 172.21.15.17:0/1655683077 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40a000c830 con 0x7f40b00b0630 2022-08-01T19:37:55.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.150+0000 7f409e7fc700 1 -- 172.21.15.17:0/1655683077 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a000c990 con 0x7f40b00b0630 2022-08-01T19:37:55.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.150+0000 7f409e7fc700 1 -- 172.21.15.17:0/1655683077 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40a000ccf0 con 0x7f40b00b0630 2022-08-01T19:37:55.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.150+0000 7f40b7a54700 1 -- 172.21.15.17:0/1655683077 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b0064a00 con 0x7f40b00b0630 2022-08-01T19:37:55.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.150+0000 7f40b7a54700 1 -- 172.21.15.17:0/1655683077 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b013d830 con 0x7f40b00b0630 2022-08-01T19:37:55.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.154+0000 7f409e7fc700 1 -- 172.21.15.17:0/1655683077 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f40a001c460 con 0x7f40b00b0630 2022-08-01T19:37:55.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.154+0000 7f40b7a54700 1 -- 172.21.15.17:0/1655683077 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40b00673a0 con 0x7f40b00b0630 2022-08-01T19:37:55.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.154+0000 7f409e7fc700 1 --2- 172.21.15.17:0/1655683077 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f40900539e0 0x7f4090055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:55.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.154+0000 7f409e7fc700 1 -- 172.21.15.17:0/1655683077 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f40a006c670 con 0x7f40b00b0630 2022-08-01T19:37:55.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.154+0000 7f40b57f0700 1 --2- 172.21.15.17:0/1655683077 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f40900539e0 0x7f4090055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:55.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.154+0000 7f40b57f0700 1 --2- 172.21.15.17:0/1655683077 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f40900539e0 0x7f4090055ea0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f40ac00c000 tx=0x7f40ac008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:55.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.158+0000 7f409e7fc700 1 -- 172.21.15.17:0/1655683077 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f40a00432c0 con 0x7f40b00b0630 2022-08-01T19:37:55.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.330+0000 7f40b7a54700 1 -- 172.21.15.17:0/1655683077 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40b00574e0 con 0x7f40b00b0630 2022-08-01T19:37:55.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.330+0000 7f409e7fc700 1 -- 172.21.15.17:0/1655683077 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f40a00468e0 con 0x7f40b00b0630 2022-08-01T19:37:55.334 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:37:55.334 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":24,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/1914125901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":1914125901},{"type":"v1","addr":"172.21.15.17:6833","nonce":1914125901}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":22},{"gid":5587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1403688289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1403688289},{"type":"v1","addr":"172.21.15.77:6835","nonce":1403688289}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":22}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:37:36.197818+0000","modified":"2022-08-01T19:37:37.201700+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5589":{"gid":5589,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/3325991310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":3325991310},{"type":"v1","addr":"172.21.15.17:6835","nonce":3325991310}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-08-01T19:37:55.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.330+0000 7f40b7a54700 1 -- 172.21.15.17:0/1655683077 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f40900539e0 msgr2=0x7f4090055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:55.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.330+0000 7f40b7a54700 1 --2- 172.21.15.17:0/1655683077 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f40900539e0 0x7f4090055ea0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f40ac00c000 tx=0x7f40ac008040).stop 2022-08-01T19:37:55.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.330+0000 7f40b7a54700 1 -- 172.21.15.17:0/1655683077 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40b00b0630 msgr2=0x7f40b013dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:55.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.330+0000 7f40b7a54700 1 --2- 172.21.15.17:0/1655683077 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40b00b0630 0x7f40b013dec0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f40a000c000 tx=0x7f40a0006df0).stop 2022-08-01T19:37:55.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.330+0000 7f40b7a54700 1 -- 172.21.15.17:0/1655683077 shutdown_connections 2022-08-01T19:37:55.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.330+0000 7f40b7a54700 1 --2- 172.21.15.17:0/1655683077 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f40900539e0 0x7f4090055ea0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:55.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.330+0000 7f40b7a54700 1 --2- 172.21.15.17:0/1655683077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40b00b0a10 0x7f40b013e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:55.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.330+0000 7f40b7a54700 1 --2- 172.21.15.17:0/1655683077 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40b00b0630 0x7f40b013dec0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:55.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.330+0000 7f40b7a54700 1 --2- 172.21.15.17:0/1655683077 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f40b00aa2b0 0x7f40b0147140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:55.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.330+0000 7f40b7a54700 1 -- 172.21.15.17:0/1655683077 >> 172.21.15.17:0/1655683077 conn(0x7f40b0060950 msgr2=0x7f40b00ab6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:55.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.334+0000 7f40b7a54700 1 -- 172.21.15.17:0/1655683077 shutdown_connections 2022-08-01T19:37:55.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.334+0000 7f40b7a54700 1 -- 172.21.15.17:0/1655683077 wait complete. 2022-08-01T19:37:55.344 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 24 2022-08-01T19:37:55.348 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c cache drop 5 2022-08-01T19:37:55.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.454+0000 7f878da0f700 1 Processor -- start 2022-08-01T19:37:55.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.454+0000 7f878da0f700 1 -- start start 2022-08-01T19:37:55.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.454+0000 7f878da0f700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f8788066780 0x7f8788066b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:55.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.454+0000 7f878da0f700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f878805f8e0 con 0x7f8788140e40 2022-08-01T19:37:55.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.454+0000 7f878da0f700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87880b1630 con 0x7f87880b1bc0 2022-08-01T19:37:55.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.454+0000 7f878da0f700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f878805fb10 con 0x7f8788066780 2022-08-01T19:37:55.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.454+0000 7f878d20e700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f8788140e40 0x7f878813e2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59740/0 (socket says 172.21.15.17:59740) 2022-08-01T19:37:55.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.454+0000 7f878d20e700 1 -- 172.21.15.17:0/1402428565 learned_addr learned my addr 172.21.15.17:0/1402428565 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:37:55.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.454+0000 7f878ca0d700 1 --2- 172.21.15.17:0/1402428565 >> v2:172.21.15.77:3300/0 conn(0x7f8788066780 0x7f8788066b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:55.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.454+0000 7f87877fe700 1 -- 172.21.15.17:0/1402428565 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3832935718 0 0) 0x7f878805f8e0 con 0x7f8788140e40 2022-08-01T19:37:55.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.454+0000 7f87877fe700 1 -- 172.21.15.17:0/1402428565 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8768003260 con 0x7f8788140e40 2022-08-01T19:37:55.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.454+0000 7f87877fe700 1 -- 172.21.15.17:0/1402428565 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2424291487 0 0) 0x7f87880b1630 con 0x7f87880b1bc0 2022-08-01T19:37:55.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.454+0000 7f87877fe700 1 -- 172.21.15.17:0/1402428565 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f878805f8e0 con 0x7f87880b1bc0 2022-08-01T19:37:55.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.454+0000 7f87877fe700 1 -- 172.21.15.17:0/1402428565 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 625736973 0 0) 0x7f8768003260 con 0x7f8788140e40 2022-08-01T19:37:55.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.454+0000 7f878ca0d700 1 -- 172.21.15.17:0/1402428565 >> v1:172.21.15.77:6789/0 conn(0x7f87880b1bc0 legacy=0x7f878805ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:55.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.454+0000 7f878ca0d700 1 -- 172.21.15.17:0/1402428565 >> v1:172.21.15.17:6789/0 conn(0x7f8788140e40 legacy=0x7f878813e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:55.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.454+0000 7f878ca0d700 1 -- 172.21.15.17:0/1402428565 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87880b18d0 con 0x7f8788066780 2022-08-01T19:37:55.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.454+0000 7f878ca0d700 1 --2- 172.21.15.17:0/1402428565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8788066780 0x7f8788066b60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f8778001780 tx=0x7f8770006df0).ready entity=mon.1 client_cookie=10d5c7c64306a5cd server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:55.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.454+0000 7f87877fe700 1 -- 172.21.15.17:0/1402428565 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87800019e0 con 0x7f8788066780 2022-08-01T19:37:55.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.454+0000 7f87877fe700 1 -- 172.21.15.17:0/1402428565 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8770000e90 con 0x7f8788066780 2022-08-01T19:37:55.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.454+0000 7f87877fe700 1 -- 172.21.15.17:0/1402428565 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87780015e0 con 0x7f8788066780 2022-08-01T19:37:55.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.454+0000 7f878da0f700 1 -- 172.21.15.17:0/1402428565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8788066780 msgr2=0x7f8788066b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:55.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.454+0000 7f878da0f700 1 --2- 172.21.15.17:0/1402428565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8788066780 0x7f8788066b60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f8778001780 tx=0x7f8770006df0).stop 2022-08-01T19:37:55.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.454+0000 7f878da0f700 1 -- 172.21.15.17:0/1402428565 shutdown_connections 2022-08-01T19:37:55.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.454+0000 7f878da0f700 1 --2- 172.21.15.17:0/1402428565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8788066780 0x7f8788066b60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:55.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.454+0000 7f878da0f700 1 -- 172.21.15.17:0/1402428565 >> 172.21.15.17:0/1402428565 conn(0x7f8788063870 msgr2=0x7f8788063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:55.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.454+0000 7f878da0f700 1 -- 172.21.15.17:0/1402428565 shutdown_connections 2022-08-01T19:37:55.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.454+0000 7f878da0f700 1 -- 172.21.15.17:0/1402428565 wait complete. 2022-08-01T19:37:55.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.458+0000 7f878da0f700 1 Processor -- start 2022-08-01T19:37:55.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.458+0000 7f878da0f700 1 -- start start 2022-08-01T19:37:55.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.458+0000 7f878da0f700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8788066780 0x7f87880ae330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:55.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.458+0000 7f878da0f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87880b1bc0 0x7f87880ad880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:55.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.458+0000 7f878da0f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8788140e40 0x7f8788153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:55.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.458+0000 7f878da0f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87880ae820 con 0x7f8788140e40 2022-08-01T19:37:55.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.458+0000 7f878da0f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87880add70 con 0x7f87880b1bc0 2022-08-01T19:37:55.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.458+0000 7f878da0f700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87880aea40 con 0x7f8788066780 2022-08-01T19:37:55.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.458+0000 7f878d20e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8788140e40 0x7f8788153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:55.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.458+0000 7f878d20e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8788140e40 0x7f8788153db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39320/0 (socket says 172.21.15.17:39320) 2022-08-01T19:37:55.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.458+0000 7f878d20e700 1 -- 172.21.15.17:0/3007947178 learned_addr learned my addr 172.21.15.17:0/3007947178 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:55.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.458+0000 7f8787fff700 1 --2- 172.21.15.17:0/3007947178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87880b1bc0 0x7f87880ad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:55.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.458+0000 7f878ca0d700 1 --2- 172.21.15.17:0/3007947178 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8788066780 0x7f87880ae330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:55.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.458+0000 7f878d20e700 1 -- 172.21.15.17:0/3007947178 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8788066780 msgr2=0x7f87880ae330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:55.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.458+0000 7f878d20e700 1 --2- 172.21.15.17:0/3007947178 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8788066780 0x7f87880ae330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:55.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.458+0000 7f878d20e700 1 -- 172.21.15.17:0/3007947178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87880b1bc0 msgr2=0x7f87880ad880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:55.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.458+0000 7f878d20e700 1 --2- 172.21.15.17:0/3007947178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87880b1bc0 0x7f87880ad880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:55.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.458+0000 7f878d20e700 1 -- 172.21.15.17:0/3007947178 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f877000b180 con 0x7f8788140e40 2022-08-01T19:37:55.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.458+0000 7f878d20e700 1 --2- 172.21.15.17:0/3007947178 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8788140e40 0x7f8788153db0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7f8780001bc0 tx=0x7f8780001a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:55.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.458+0000 7f8785ffb700 1 -- 172.21.15.17:0/3007947178 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8780004ab0 con 0x7f8788140e40 2022-08-01T19:37:55.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.458+0000 7f8785ffb700 1 -- 172.21.15.17:0/3007947178 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8780004c10 con 0x7f8788140e40 2022-08-01T19:37:55.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.458+0000 7f878da0f700 1 -- 172.21.15.17:0/3007947178 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87880aed40 con 0x7f8788140e40 2022-08-01T19:37:55.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.458+0000 7f8785ffb700 1 -- 172.21.15.17:0/3007947178 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8780007170 con 0x7f8788140e40 2022-08-01T19:37:55.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.458+0000 7f878da0f700 1 -- 172.21.15.17:0/3007947178 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8788144960 con 0x7f8788140e40 2022-08-01T19:37:55.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.462+0000 7f8785ffb700 1 -- 172.21.15.17:0/3007947178 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f878001b070 con 0x7f8788140e40 2022-08-01T19:37:55.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.462+0000 7f8785ffb700 1 --2- 172.21.15.17:0/3007947178 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f876c053a10 0x7f876c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:55.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.462+0000 7f8785ffb700 1 -- 172.21.15.17:0/3007947178 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f878006b710 con 0x7f8788140e40 2022-08-01T19:37:55.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.462+0000 7f8798ec8740 1 Processor -- start 2022-08-01T19:37:55.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.462+0000 7f8798ec8740 1 -- start start 2022-08-01T19:37:55.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.462+0000 7f878ca0d700 1 --2- 172.21.15.17:0/3007947178 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f876c053a10 0x7f876c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:55.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.462+0000 7f8798ec8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1a8f5c0 0x1a59460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:55.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.462+0000 7f8798ec8740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1a59f60 0x1a59950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:55.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.462+0000 7f8798ec8740 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1a90e30 0x1a912c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:55.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.462+0000 7f8798ec8740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x19b83c0 con 0x1a59f60 2022-08-01T19:37:55.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.462+0000 7f8798ec8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x19cc050 con 0x1a8f5c0 2022-08-01T19:37:55.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.462+0000 7f8798ec8740 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x19ac290 con 0x1a90e30 2022-08-01T19:37:55.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.462+0000 7f878ca0d700 1 --2- 172.21.15.17:0/3007947178 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f876c053a10 0x7f876c055ed0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f8770019000 tx=0x7f87880610c0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:55.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.462+0000 7f8787fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1a59f60 0x1a59950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:55.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.462+0000 7f8787fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1a59f60 0x1a59950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39326/0 (socket says 172.21.15.17:39326) 2022-08-01T19:37:55.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.462+0000 7f8787fff700 1 -- 172.21.15.17:0/547604561 learned_addr learned my addr 172.21.15.17:0/547604561 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:55.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.462+0000 7f878d20e700 1 --2- 172.21.15.17:0/547604561 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1a90e30 0x1a912c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:55.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.462+0000 7f878ca0d700 1 --2- 172.21.15.17:0/547604561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1a8f5c0 0x1a59460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:55.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.466+0000 7f8787fff700 1 -- 172.21.15.17:0/547604561 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1a90e30 msgr2=0x1a912c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:55.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.466+0000 7f8787fff700 1 --2- 172.21.15.17:0/547604561 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1a90e30 0x1a912c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:55.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.466+0000 7f8787fff700 1 -- 172.21.15.17:0/547604561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1a8f5c0 msgr2=0x1a59460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:55.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.466+0000 7f8787fff700 1 --2- 172.21.15.17:0/547604561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1a8f5c0 0x1a59460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:55.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.466+0000 7f8787fff700 1 -- 172.21.15.17:0/547604561 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1a03e90 con 0x1a59f60 2022-08-01T19:37:55.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.466+0000 7f8787fff700 1 --2- 172.21.15.17:0/547604561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1a59f60 0x1a59950 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x19b9230 tx=0x7f8778008040).ready entity=mon.0 client_cookie=21b28b6d1af5ba5f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:55.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.466+0000 7f878da0f700 1 -- 172.21.15.17:0/547604561 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8778006ea0 con 0x1a59f60 2022-08-01T19:37:55.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.466+0000 7f878da0f700 1 -- 172.21.15.17:0/547604561 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8778001ca0 con 0x1a59f60 2022-08-01T19:37:55.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.466+0000 7f878da0f700 1 -- 172.21.15.17:0/547604561 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f877800e8a0 con 0x1a59f60 2022-08-01T19:37:55.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.466+0000 7f8798ec8740 1 -- 172.21.15.17:0/547604561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1a59f60 msgr2=0x1a59950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:55.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.466+0000 7f8798ec8740 1 --2- 172.21.15.17:0/547604561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1a59f60 0x1a59950 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x19b9230 tx=0x7f8778008040).stop 2022-08-01T19:37:55.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.466+0000 7f8798ec8740 1 -- 172.21.15.17:0/547604561 shutdown_connections 2022-08-01T19:37:55.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.466+0000 7f8798ec8740 1 --2- 172.21.15.17:0/547604561 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1a90e30 0x1a912c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:55.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.466+0000 7f8798ec8740 1 --2- 172.21.15.17:0/547604561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1a59f60 0x1a59950 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:55.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.466+0000 7f8798ec8740 1 --2- 172.21.15.17:0/547604561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1a8f5c0 0x1a59460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:55.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.466+0000 7f8798ec8740 1 -- 172.21.15.17:0/547604561 >> 172.21.15.17:0/547604561 conn(0x1a5b3f0 msgr2=0x1a5b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:55.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.466+0000 7f8798ec8740 1 -- 172.21.15.17:0/547604561 shutdown_connections 2022-08-01T19:37:55.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.466+0000 7f8798ec8740 1 -- 172.21.15.17:0/547604561 wait complete. 2022-08-01T19:37:55.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.466+0000 7f8798ec8740 1 -- start start 2022-08-01T19:37:55.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.466+0000 7f8798ec8740 1 Processor -- start 2022-08-01T19:37:55.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.466+0000 7f8798ec8740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1a59f60 0x1a89ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:55.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.466+0000 7f8798ec8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1a8f5c0 0x1a89fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:55.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.466+0000 7f8798ec8740 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1a90e30 0x1a9fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:55.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.466+0000 7f8798ec8740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a5cff0 con 0x1a59f60 2022-08-01T19:37:55.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.466+0000 7f8798ec8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a9d790 con 0x1a8f5c0 2022-08-01T19:37:55.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.466+0000 7f8798ec8740 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x1a8c5b0 con 0x1a90e30 2022-08-01T19:37:55.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.466+0000 7f8798ec8740 10 client.0 ll_register_callbacks cb 0x1a13470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-08-01T19:37:55.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f878ca0d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1a59f60 0x1a89ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:55.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f878ca0d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1a59f60 0x1a89ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39330/0 (socket says 172.21.15.17:39330) 2022-08-01T19:37:55.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f878ca0d700 1 -- 172.21.15.17:0/2505609067 learned_addr learned my addr 172.21.15.17:0/2505609067 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:55.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f8787fff700 1 --2- 172.21.15.17:0/2505609067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1a8f5c0 0x1a89fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:55.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f878d20e700 1 --2- 172.21.15.17:0/2505609067 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1a90e30 0x1a9fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:55.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f878ca0d700 1 -- 172.21.15.17:0/2505609067 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1a90e30 msgr2=0x1a9fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:55.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f878ca0d700 1 --2- 172.21.15.17:0/2505609067 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1a90e30 0x1a9fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:55.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f878ca0d700 1 -- 172.21.15.17:0/2505609067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1a8f5c0 msgr2=0x1a89fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:55.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f878ca0d700 1 --2- 172.21.15.17:0/2505609067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1a8f5c0 0x1a89fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:55.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f878ca0d700 1 -- 172.21.15.17:0/2505609067 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8778009910 con 0x1a59f60 2022-08-01T19:37:55.499 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f878ca0d700 1 --2- 172.21.15.17:0/2505609067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1a59f60 0x1a89ad0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f8770022000 tx=0x7f877001e000).ready entity=mon.0 client_cookie=317c0d68c8b95169 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:55.499 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f87677fe700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2022-08-01T19:37:55.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f87677fe700 1 -- 172.21.15.17:0/2505609067 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f877001edf0 con 0x1a59f60 2022-08-01T19:37:55.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f87677fe700 1 -- 172.21.15.17:0/2505609067 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8770023040 con 0x1a59f60 2022-08-01T19:37:55.501 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f8798ec8740 1 -- 172.21.15.17:0/2505609067 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x19c2000 con 0x1a59f60 2022-08-01T19:37:55.501 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f87677fe700 1 -- 172.21.15.17:0/2505609067 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f877000d3d0 con 0x1a59f60 2022-08-01T19:37:55.501 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f8798ec8740 10 client.5787 fetch_fsmap learned FSMap version 24 2022-08-01T19:37:55.502 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f8798ec8740 1 -- 172.21.15.17:0/2505609067 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x199e9f0 con 0x1a59f60 2022-08-01T19:37:55.502 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f87677fe700 1 -- 172.21.15.17:0/2505609067 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x19c2000 con 0x1a59f60 2022-08-01T19:37:55.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f87677fe700 1 -- 172.21.15.17:0/2505609067 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 1434+0+0 (secure 0 0 0) 0x7f877000db40 con 0x1a59f60 2022-08-01T19:37:55.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f8798ec8740 10 client.5787 fetch_fsmap finished waiting for FSMap version 24 2022-08-01T19:37:55.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f8798ec8740 10 client.5787 resolve_mds: resolved ID 'c' to GID 5589 2022-08-01T19:37:55.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f8798ec8740 20 client.5787 populate_metadata read hostname 'smithi017' 2022-08-01T19:37:55.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f8798ec8740 1 --2- 172.21.15.17:0/2505609067 >> [v2:172.21.15.17:6834/3325991310,v1:172.21.15.17:6835/3325991310] conn(0x1aa0010 0x1aa30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:55.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f8798ec8740 4 client.5787 mds_command: new command op to 5589 tid=0[{"prefix": "get_command_descriptions"}] 2022-08-01T19:37:55.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f8798ec8740 1 -- 172.21.15.17:0/2505609067 --> [v2:172.21.15.17:6834/3325991310,v1:172.21.15.17:6835/3325991310] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x19ba080 con 0x1aa0010 2022-08-01T19:37:55.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f8787fff700 1 --2- 172.21.15.17:0/2505609067 >> [v2:172.21.15.17:6834/3325991310,v1:172.21.15.17:6835/3325991310] conn(0x1aa0010 0x1aa30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:55.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f87677fe700 10 client.5787 ms_handle_connect on v2:172.21.15.17:6834/3325991310 2022-08-01T19:37:55.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f8787fff700 1 --2- 172.21.15.17:0/2505609067 >> [v2:172.21.15.17:6834/3325991310,v1:172.21.15.17:6835/3325991310] conn(0x1aa0010 0x1aa30f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=8b1e5b90bfa3c383 server_cookie=9612f03f126c14b1 in_seq=0 out_seq=0 2022-08-01T19:37:55.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f87677fe700 1 -- 172.21.15.17:0/2505609067 <== mds.0 v2:172.21.15.17:6834/3325991310 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x19ba080 con 0x1aa0010 2022-08-01T19:37:55.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f87677fe700 10 client.5787 handle_command_reply: tid=0 2022-08-01T19:37:55.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f8798ec8740 1 client.5787 shutdown 2022-08-01T19:37:55.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f8798ec8740 1 -- 172.21.15.17:0/2505609067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1a59f60 msgr2=0x1a89ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:55.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f8798ec8740 1 --2- 172.21.15.17:0/2505609067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1a59f60 0x1a89ad0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f8770022000 tx=0x7f877001e000).stop 2022-08-01T19:37:55.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f8798ec8740 1 -- 172.21.15.17:0/2505609067 shutdown_connections 2022-08-01T19:37:55.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f8798ec8740 1 --2- 172.21.15.17:0/2505609067 >> [v2:172.21.15.17:6834/3325991310,v1:172.21.15.17:6835/3325991310] conn(0x1aa0010 0x1aa30f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:55.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f8798ec8740 1 --2- 172.21.15.17:0/2505609067 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1a90e30 0x1a9fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:55.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f8798ec8740 1 --2- 172.21.15.17:0/2505609067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1a8f5c0 0x1a89fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:55.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f87677fe700 0 client.5787 ms_handle_reset on v2:172.21.15.17:6834/3325991310 2022-08-01T19:37:55.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f8798ec8740 1 --2- 172.21.15.17:0/2505609067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1a59f60 0x1a89ad0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:55.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f8798ec8740 1 -- 172.21.15.17:0/2505609067 >> 172.21.15.17:0/2505609067 conn(0x1a5b3f0 msgr2=0x1a912c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:55.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.470+0000 7f8798ec8740 1 -- 172.21.15.17:0/2505609067 shutdown_connections 2022-08-01T19:37:55.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.474+0000 7f8798ec8740 1 -- 172.21.15.17:0/2505609067 wait complete. 2022-08-01T19:37:55.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.474+0000 7f8798ec8740 20 client.5787 trim_cache size 0 max 16384 2022-08-01T19:37:55.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f8798ec8740 1 Processor -- start 2022-08-01T19:37:55.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f8798ec8740 1 -- start start 2022-08-01T19:37:55.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f8798ec8740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1a59f60 0x1a8b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:55.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f8798ec8740 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1a8f5c0 0x1a58840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:55.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f8798ec8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1a90e30 0x1a58d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:55.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f8798ec8740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a90990 con 0x1a59f60 2022-08-01T19:37:55.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f8798ec8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x19ba080 con 0x1a90e30 2022-08-01T19:37:55.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f8798ec8740 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x19f89d0 con 0x1a8f5c0 2022-08-01T19:37:55.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f878ca0d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1a59f60 0x1a8b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:55.525 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f878ca0d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1a59f60 0x1a8b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39340/0 (socket says 172.21.15.17:39340) 2022-08-01T19:37:55.525 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f878ca0d700 1 -- 172.21.15.17:0/2310895002 learned_addr learned my addr 172.21.15.17:0/2310895002 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:55.525 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f878d20e700 1 --2- 172.21.15.17:0/2310895002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1a90e30 0x1a58d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:55.525 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f8787fff700 1 --2- 172.21.15.17:0/2310895002 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1a8f5c0 0x1a58840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:55.526 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f878ca0d700 1 -- 172.21.15.17:0/2310895002 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1a8f5c0 msgr2=0x1a58840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:55.526 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f878ca0d700 1 --2- 172.21.15.17:0/2310895002 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1a8f5c0 0x1a58840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:55.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f878ca0d700 1 -- 172.21.15.17:0/2310895002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1a90e30 msgr2=0x1a58d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:55.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f878ca0d700 1 --2- 172.21.15.17:0/2310895002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1a90e30 0x1a58d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:55.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f878ca0d700 1 -- 172.21.15.17:0/2310895002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x19bb5f0 con 0x1a59f60 2022-08-01T19:37:55.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f878ca0d700 1 --2- 172.21.15.17:0/2310895002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1a59f60 0x1a8b700 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x198f750 tx=0x7f8770029000).ready entity=mon.0 client_cookie=bfa8881ea46207bf server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:55.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f877dffb700 1 -- 172.21.15.17:0/2310895002 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8770032490 con 0x1a59f60 2022-08-01T19:37:55.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f877dffb700 1 -- 172.21.15.17:0/2310895002 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8770032660 con 0x1a59f60 2022-08-01T19:37:55.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f877dffb700 1 -- 172.21.15.17:0/2310895002 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87700329c0 con 0x1a59f60 2022-08-01T19:37:55.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f8798ec8740 1 -- 172.21.15.17:0/2310895002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1a59f60 msgr2=0x1a8b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:37:55.529 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f8798ec8740 1 --2- 172.21.15.17:0/2310895002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1a59f60 0x1a8b700 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x198f750 tx=0x7f8770029000).stop 2022-08-01T19:37:55.529 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f8798ec8740 1 -- 172.21.15.17:0/2310895002 shutdown_connections 2022-08-01T19:37:55.529 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f8798ec8740 1 --2- 172.21.15.17:0/2310895002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1a90e30 0x1a58d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:55.529 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f8798ec8740 1 --2- 172.21.15.17:0/2310895002 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1a8f5c0 0x1a58840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:55.530 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f8798ec8740 1 --2- 172.21.15.17:0/2310895002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1a59f60 0x1a8b700 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:55.530 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f8798ec8740 1 -- 172.21.15.17:0/2310895002 >> 172.21.15.17:0/2310895002 conn(0x1a5b3f0 msgr2=0x1a89720 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:37:55.530 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f8798ec8740 1 -- 172.21.15.17:0/2310895002 shutdown_connections 2022-08-01T19:37:55.530 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f8798ec8740 1 -- 172.21.15.17:0/2310895002 wait complete. 2022-08-01T19:37:55.531 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f8798ec8740 1 -- start start 2022-08-01T19:37:55.531 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f8798ec8740 1 Processor -- start 2022-08-01T19:37:55.531 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f8798ec8740 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1a59f60 0x1a989a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:55.531 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f8798ec8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1a8f5c0 0x1af4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:55.531 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f8798ec8740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1a90e30 0x1af4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:55.532 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f8798ec8740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a942a0 con 0x1a90e30 2022-08-01T19:37:55.532 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.490+0000 7f8798ec8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x1af88c0 con 0x1a8f5c0 2022-08-01T19:37:55.532 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.494+0000 7f8798ec8740 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x1a901b0 con 0x1a59f60 2022-08-01T19:37:55.532 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.494+0000 7f878d20e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1a90e30 0x1af4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:55.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.494+0000 7f878d20e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1a90e30 0x1af4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39348/0 (socket says 172.21.15.17:39348) 2022-08-01T19:37:55.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.494+0000 7f8798ec8740 10 client.0 ll_register_callbacks cb 0x7f8770021070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-08-01T19:37:55.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.494+0000 7f878d20e700 1 -- 172.21.15.17:0/72781034 learned_addr learned my addr 172.21.15.17:0/72781034 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:37:55.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.494+0000 7f878ca0d700 1 --2- 172.21.15.17:0/72781034 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1a59f60 0x1a989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:55.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.494+0000 7f8787fff700 1 --2- 172.21.15.17:0/72781034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1a8f5c0 0x1af4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:55.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.494+0000 7f878d20e700 1 -- 172.21.15.17:0/72781034 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1a59f60 msgr2=0x1a989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:55.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.494+0000 7f878d20e700 1 --2- 172.21.15.17:0/72781034 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1a59f60 0x1a989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:55.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.494+0000 7f878d20e700 1 -- 172.21.15.17:0/72781034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1a8f5c0 msgr2=0x1af4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:37:55.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.494+0000 7f878d20e700 1 --2- 172.21.15.17:0/72781034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1a8f5c0 0x1af4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:37:55.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.494+0000 7f878d20e700 1 -- 172.21.15.17:0/72781034 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x199e9f0 con 0x1a90e30 2022-08-01T19:37:55.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.494+0000 7f878d20e700 1 --2- 172.21.15.17:0/72781034 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1a90e30 0x1af4980 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f8780049000 tx=0x7f878004b040).ready entity=mon.0 client_cookie=c18848832e8355df server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:37:55.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.494+0000 7f877cff9700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2022-08-01T19:37:55.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.494+0000 7f877cff9700 1 -- 172.21.15.17:0/72781034 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87800453d0 con 0x1a90e30 2022-08-01T19:37:55.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.494+0000 7f877cff9700 1 -- 172.21.15.17:0/72781034 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8780015da0 con 0x1a90e30 2022-08-01T19:37:55.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.494+0000 7f8798ec8740 1 -- 172.21.15.17:0/72781034 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8778001ca0 con 0x1a90e30 2022-08-01T19:37:55.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.494+0000 7f877cff9700 1 -- 172.21.15.17:0/72781034 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f878006b110 con 0x1a90e30 2022-08-01T19:37:55.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.494+0000 7f8798ec8740 10 client.5793 fetch_fsmap learned FSMap version 24 2022-08-01T19:37:55.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.494+0000 7f8798ec8740 1 -- 172.21.15.17:0/72781034 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x19c41e0 con 0x1a90e30 2022-08-01T19:37:55.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.494+0000 7f877cff9700 1 -- 172.21.15.17:0/72781034 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878005a020 con 0x1a90e30 2022-08-01T19:37:55.538 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.494+0000 7f877cff9700 1 -- 172.21.15.17:0/72781034 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 1434+0+0 (secure 0 0 0) 0x7f8780053060 con 0x1a90e30 2022-08-01T19:37:55.538 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.494+0000 7f8798ec8740 10 client.5793 fetch_fsmap finished waiting for FSMap version 24 2022-08-01T19:37:55.538 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.494+0000 7f8798ec8740 10 client.5793 resolve_mds: resolved ID 'c' to GID 5589 2022-08-01T19:37:55.538 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.494+0000 7f8798ec8740 20 client.5793 populate_metadata read hostname 'smithi017' 2022-08-01T19:37:55.538 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.494+0000 7f8798ec8740 1 --2- 172.21.15.17:0/72781034 >> [v2:172.21.15.17:6834/3325991310,v1:172.21.15.17:6835/3325991310] conn(0x1af4e70 0x1afa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:37:55.539 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.494+0000 7f8798ec8740 4 client.5793 mds_command: new command op to 5589 tid=0[{"prefix": "cache drop", "timeout": 5}] 2022-08-01T19:37:55.539 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.494+0000 7f8798ec8740 1 -- 172.21.15.17:0/72781034 --> [v2:172.21.15.17:6834/3325991310,v1:172.21.15.17:6835/3325991310] -- command(tid 0: {"prefix": "cache drop", "timeout": 5}) v1 -- 0x18af5e0 con 0x1af4e70 2022-08-01T19:37:55.539 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.494+0000 7f878ca0d700 1 --2- 172.21.15.17:0/72781034 >> [v2:172.21.15.17:6834/3325991310,v1:172.21.15.17:6835/3325991310] conn(0x1af4e70 0x1afa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:37:55.539 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.494+0000 7f877cff9700 10 client.5793 ms_handle_connect on v2:172.21.15.17:6834/3325991310 2022-08-01T19:37:55.540 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:37:55.494+0000 7f878ca0d700 1 --2- 172.21.15.17:0/72781034 >> [v2:172.21.15.17:6834/3325991310,v1:172.21.15.17:6835/3325991310] conn(0x1af4e70 0x1afa9c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=6e614a99b14dd656 server_cookie=906964482cca347c in_seq=0 out_seq=0 2022-08-01T19:38:00.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.698+0000 7f877cff9700 1 -- 172.21.15.17:0/72781034 <== mds.0 v2:172.21.15.17:6834/3325991310 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+404 (crc 0 0 0) 0x18af5e0 con 0x1af4e70 2022-08-01T19:38:00.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.698+0000 7f877cff9700 10 client.5793 handle_command_reply: tid=0 2022-08-01T19:38:00.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.698+0000 7f8798ec8740 1 client.5793 shutdown 2022-08-01T19:38:00.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.698+0000 7f8798ec8740 1 -- 172.21.15.17:0/72781034 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1a90e30 msgr2=0x1af4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:00.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.698+0000 7f8798ec8740 1 --2- 172.21.15.17:0/72781034 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1a90e30 0x1af4980 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f8780049000 tx=0x7f878004b040).stop 2022-08-01T19:38:00.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.698+0000 7f8798ec8740 1 -- 172.21.15.17:0/72781034 shutdown_connections 2022-08-01T19:38:00.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.698+0000 7f8798ec8740 1 --2- 172.21.15.17:0/72781034 >> [v2:172.21.15.17:6834/3325991310,v1:172.21.15.17:6835/3325991310] conn(0x1af4e70 0x1afa9c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:00.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.698+0000 7f8798ec8740 1 --2- 172.21.15.17:0/72781034 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1a90e30 0x1af4980 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:00.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.698+0000 7f8798ec8740 1 --2- 172.21.15.17:0/72781034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1a8f5c0 0x1af4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:00.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.698+0000 7f877cff9700 0 client.5793 ms_handle_reset on v2:172.21.15.17:6834/3325991310 2022-08-01T19:38:00.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.698+0000 7f8798ec8740 1 --2- 172.21.15.17:0/72781034 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1a59f60 0x1a989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:00.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.698+0000 7f8798ec8740 1 -- 172.21.15.17:0/72781034 >> 172.21.15.17:0/72781034 conn(0x1a5b3f0 msgr2=0x1a89f10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:38:00.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.698+0000 7f8798ec8740 1 -- 172.21.15.17:0/72781034 shutdown_connections 2022-08-01T19:38:00.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.698+0000 7f8798ec8740 1 -- 172.21.15.17:0/72781034 wait complete. 2022-08-01T19:38:00.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.698+0000 7f8798ec8740 20 client.5793 trim_cache size 0 max 16384 2022-08-01T19:38:00.714 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:38:00.714 INFO:teuthology.orchestra.run.smithi017.stdout: "client_recall": { 2022-08-01T19:38:00.714 INFO:teuthology.orchestra.run.smithi017.stdout: "return_code": -110, 2022-08-01T19:38:00.714 INFO:teuthology.orchestra.run.smithi017.stdout: "message": "(110) Connection timed out", 2022-08-01T19:38:00.715 INFO:teuthology.orchestra.run.smithi017.stdout: "recalled": 120 2022-08-01T19:38:00.715 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:00.715 INFO:teuthology.orchestra.run.smithi017.stdout: "flush_journal": { 2022-08-01T19:38:00.716 INFO:teuthology.orchestra.run.smithi017.stdout: "return_code": 0, 2022-08-01T19:38:00.716 INFO:teuthology.orchestra.run.smithi017.stdout: "message": "" 2022-08-01T19:38:00.717 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:00.717 INFO:teuthology.orchestra.run.smithi017.stdout: "trim_cache": { 2022-08-01T19:38:00.718 INFO:teuthology.orchestra.run.smithi017.stdout: "trimmed": 0 2022-08-01T19:38:00.718 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:00.719 INFO:teuthology.orchestra.run.smithi017.stdout: "cache": { 2022-08-01T19:38:00.719 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": { 2022-08-01T19:38:00.719 INFO:teuthology.orchestra.run.smithi017.stdout: "items": 19987, 2022-08-01T19:38:00.720 INFO:teuthology.orchestra.run.smithi017.stdout: "bytes": 3099488 2022-08-01T19:38:00.720 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:00.720 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:00.720 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2026249760000001 2022-08-01T19:38:00.721 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:38:00.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.698+0000 7f878da0f700 1 -- 172.21.15.17:0/3007947178 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f876c053a10 msgr2=0x7f876c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:00.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.698+0000 7f878da0f700 1 --2- 172.21.15.17:0/3007947178 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f876c053a10 0x7f876c055ed0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f8770019000 tx=0x7f87880610c0).stop 2022-08-01T19:38:00.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.698+0000 7f878da0f700 1 -- 172.21.15.17:0/3007947178 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8788140e40 msgr2=0x7f8788153db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:00.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.698+0000 7f878da0f700 1 --2- 172.21.15.17:0/3007947178 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8788140e40 0x7f8788153db0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7f8780001bc0 tx=0x7f8780001a40).stop 2022-08-01T19:38:00.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.698+0000 7f878da0f700 1 -- 172.21.15.17:0/3007947178 shutdown_connections 2022-08-01T19:38:00.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.698+0000 7f878da0f700 1 --2- 172.21.15.17:0/3007947178 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f876c053a10 0x7f876c055ed0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:00.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.698+0000 7f878da0f700 1 --2- 172.21.15.17:0/3007947178 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8788140e40 0x7f8788153db0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:00.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.698+0000 7f878da0f700 1 --2- 172.21.15.17:0/3007947178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87880b1bc0 0x7f87880ad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:00.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.698+0000 7f878da0f700 1 --2- 172.21.15.17:0/3007947178 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8788066780 0x7f87880ae330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:00.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.698+0000 7f878da0f700 1 -- 172.21.15.17:0/3007947178 >> 172.21.15.17:0/3007947178 conn(0x7f8788063870 msgr2=0x7f87880b2140 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:38:00.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.698+0000 7f878da0f700 1 -- 172.21.15.17:0/3007947178 shutdown_connections 2022-08-01T19:38:00.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.698+0000 7f878da0f700 1 -- 172.21.15.17:0/3007947178 wait complete. 2022-08-01T19:38:00.731 INFO:tasks.cephfs.fuse_mount:Cleaning up killed ceph-fuse connection 2022-08-01T19:38:00.732 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:38:00.732 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:38:00.733 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:38:00.741 INFO:teuthology.orchestra.run.smithi085.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2022-08-01T19:38:00.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:38:00.744 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2022-08-01T19:38:00.745 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi085.front.sepia.ceph.com... 2022-08-01T19:38:00.746 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T19:38:00.747 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:38:00.769 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:38:00.770 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:38:00.770 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:38:00.817 INFO:teuthology.orchestra.run.smithi085.stdout:ext2/ext3 2022-08-01T19:38:00.819 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T19:38:00.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T19:38:00.821 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:38:00.830 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:38:00.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9112773700 1 Processor -- start 2022-08-01T19:38:00.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9112773700 1 -- start start 2022-08-01T19:38:00.949 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9112773700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f910c0a9da0 0x7f910c060950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:00.949 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9112773700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f910c1402d0 0x7f910c140720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:00.949 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9112773700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f910c067030 con 0x7f910c0b31d0 2022-08-01T19:38:00.950 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9112773700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f910c064a70 con 0x7f910c0a9da0 2022-08-01T19:38:00.950 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9112773700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f910c061f60 con 0x7f910c1402d0 2022-08-01T19:38:00.950 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9110f70700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f910c0a9da0 0x7f910c060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:00.950 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9110f70700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f910c0a9da0 0x7f910c060950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39358/0 (socket says 172.21.15.17:39358) 2022-08-01T19:38:00.951 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9110f70700 1 -- 172.21.15.17:0/1876426038 learned_addr learned my addr 172.21.15.17:0/1876426038 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:38:00.951 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9111771700 1 --1- 172.21.15.17:0/1876426038 >> v1:172.21.15.17:6789/0 conn(0x7f910c0b31d0 0x7f910c066020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59784/0 (socket says 172.21.15.17:59784) 2022-08-01T19:38:00.951 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9111f72700 1 --2- 172.21.15.17:0/1876426038 >> v2:172.21.15.77:3300/0 conn(0x7f910c1402d0 0x7f910c140720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:00.951 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9110f70700 1 -- 172.21.15.17:0/1876426038 >> v2:172.21.15.77:3300/0 conn(0x7f910c1402d0 msgr2=0x7f910c140720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:38:00.952 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9110f70700 1 --2- 172.21.15.17:0/1876426038 >> v2:172.21.15.77:3300/0 conn(0x7f910c1402d0 0x7f910c140720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:00.952 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9110f70700 1 -- 172.21.15.17:0/1876426038 >> v1:172.21.15.17:6789/0 conn(0x7f910c0b31d0 legacy=0x7f910c066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:00.952 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9110f70700 1 -- 172.21.15.17:0/1876426038 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f910c0b35b0 con 0x7f910c0a9da0 2022-08-01T19:38:00.952 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9111f72700 1 --2- 172.21.15.17:0/1876426038 >> v2:172.21.15.77:3300/0 conn(0x7f910c1402d0 0x7f910c140720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:38:00.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9110f70700 1 --2- 172.21.15.17:0/1876426038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f910c0a9da0 0x7f910c060950 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f9108002f70 tx=0x7f9108006eb0).ready entity=mon.0 client_cookie=58152c30cbbd3a1d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:38:00.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f90fffff700 1 -- 172.21.15.17:0/1876426038 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f910800d610 con 0x7f910c0a9da0 2022-08-01T19:38:00.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f90fffff700 1 -- 172.21.15.17:0/1876426038 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9108000e90 con 0x7f910c0a9da0 2022-08-01T19:38:00.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f90fffff700 1 -- 172.21.15.17:0/1876426038 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f910800d9a0 con 0x7f910c0a9da0 2022-08-01T19:38:00.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9112773700 1 -- 172.21.15.17:0/1876426038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f910c0a9da0 msgr2=0x7f910c060950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:00.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9112773700 1 --2- 172.21.15.17:0/1876426038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f910c0a9da0 0x7f910c060950 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f9108002f70 tx=0x7f9108006eb0).stop 2022-08-01T19:38:00.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9112773700 1 -- 172.21.15.17:0/1876426038 shutdown_connections 2022-08-01T19:38:00.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9112773700 1 --2- 172.21.15.17:0/1876426038 >> v2:172.21.15.77:3300/0 conn(0x7f910c1402d0 0x7f910c140720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:00.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9112773700 1 --2- 172.21.15.17:0/1876426038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f910c0a9da0 0x7f910c060950 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:00.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9112773700 1 -- 172.21.15.17:0/1876426038 >> 172.21.15.17:0/1876426038 conn(0x7f910c063870 msgr2=0x7f910c063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:38:00.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9112773700 1 -- 172.21.15.17:0/1876426038 shutdown_connections 2022-08-01T19:38:00.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9112773700 1 -- 172.21.15.17:0/1876426038 wait complete. 2022-08-01T19:38:00.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9112773700 1 Processor -- start 2022-08-01T19:38:00.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9112773700 1 -- start start 2022-08-01T19:38:00.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9112773700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f910c0a9da0 0x7f910c13f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:00.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9112773700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f910c0b31d0 0x7f910c133b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:00.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9112773700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f910c1402d0 0x7f910c134080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:00.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9112773700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910c135f00 con 0x7f910c1402d0 2022-08-01T19:38:00.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9112773700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910c0ae7e0 con 0x7f910c0b31d0 2022-08-01T19:38:00.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9112773700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f910c134570 con 0x7f910c0a9da0 2022-08-01T19:38:00.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9111f72700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f910c1402d0 0x7f910c134080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:00.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9111f72700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f910c1402d0 0x7f910c134080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39368/0 (socket says 172.21.15.17:39368) 2022-08-01T19:38:00.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9111f72700 1 -- 172.21.15.17:0/1565306013 learned_addr learned my addr 172.21.15.17:0/1565306013 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:38:00.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9111771700 1 --2- 172.21.15.17:0/1565306013 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f910c0a9da0 0x7f910c13f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:00.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.942+0000 7f9110f70700 1 --2- 172.21.15.17:0/1565306013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f910c0b31d0 0x7f910c133b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:00.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.946+0000 7f9111f72700 1 -- 172.21.15.17:0/1565306013 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f910c0a9da0 msgr2=0x7f910c13f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:00.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.946+0000 7f9111f72700 1 --2- 172.21.15.17:0/1565306013 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f910c0a9da0 0x7f910c13f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:00.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.946+0000 7f9111f72700 1 -- 172.21.15.17:0/1565306013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f910c0b31d0 msgr2=0x7f910c133b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:00.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.946+0000 7f9111f72700 1 --2- 172.21.15.17:0/1565306013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f910c0b31d0 0x7f910c133b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:00.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.946+0000 7f9111f72700 1 -- 172.21.15.17:0/1565306013 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f910800b1e0 con 0x7f910c1402d0 2022-08-01T19:38:00.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.946+0000 7f9111771700 1 --2- 172.21.15.17:0/1565306013 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f910c0a9da0 0x7f910c13f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:38:00.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.946+0000 7f9110f70700 1 --2- 172.21.15.17:0/1565306013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f910c0b31d0 0x7f910c133b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:38:00.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.946+0000 7f9111f72700 1 --2- 172.21.15.17:0/1565306013 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f910c1402d0 0x7f910c134080 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f910c134700 tx=0x7f9104008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:38:00.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.946+0000 7f90fe7fc700 1 -- 172.21.15.17:0/1565306013 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9104007540 con 0x7f910c1402d0 2022-08-01T19:38:00.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.946+0000 7f90fe7fc700 1 -- 172.21.15.17:0/1565306013 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91040076a0 con 0x7f910c1402d0 2022-08-01T19:38:00.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.946+0000 7f9112773700 1 -- 172.21.15.17:0/1565306013 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f910c136130 con 0x7f910c1402d0 2022-08-01T19:38:00.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.946+0000 7f90fe7fc700 1 -- 172.21.15.17:0/1565306013 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9104007a00 con 0x7f910c1402d0 2022-08-01T19:38:00.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.946+0000 7f9112773700 1 -- 172.21.15.17:0/1565306013 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f910c142220 con 0x7f910c1402d0 2022-08-01T19:38:00.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.946+0000 7f90fe7fc700 1 -- 172.21.15.17:0/1565306013 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9104019030 con 0x7f910c1402d0 2022-08-01T19:38:00.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.946+0000 7f9112773700 1 -- 172.21.15.17:0/1565306013 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f910c0ae950 con 0x7f910c1402d0 2022-08-01T19:38:00.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.946+0000 7f90fe7fc700 1 --2- 172.21.15.17:0/1565306013 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f90f4053950 0x7f90f4055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:00.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.946+0000 7f90fe7fc700 1 -- 172.21.15.17:0/1565306013 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f910406c020 con 0x7f910c1402d0 2022-08-01T19:38:00.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.946+0000 7f9111771700 1 --2- 172.21.15.17:0/1565306013 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f90f4053950 0x7f90f4055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:00.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.946+0000 7f9111771700 1 --2- 172.21.15.17:0/1565306013 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f90f4053950 0x7f90f4055e10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f9100008000 tx=0x7f9100009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:38:00.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:00.950+0000 7f90fe7fc700 1 -- 172.21.15.17:0/1565306013 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f910c0ae950 con 0x7f910c1402d0 2022-08-01T19:38:01.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.082+0000 7f9112773700 1 -- 172.21.15.17:0/1565306013 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f910c0ae950 con 0x7f910c1402d0 2022-08-01T19:38:01.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.082+0000 7f90fe7fc700 1 -- 172.21.15.17:0/1565306013 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16273 (secure 0 0 0) 0x7f910c0ae950 con 0x7f910c1402d0 2022-08-01T19:38:01.088 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:38:01.088 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":55,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:37:37.200615+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:37:34.595906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2022-08-01T19:37:35.889700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:38:01.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.086+0000 7f9112773700 1 -- 172.21.15.17:0/1565306013 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f90f4053950 msgr2=0x7f90f4055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:01.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.086+0000 7f9112773700 1 --2- 172.21.15.17:0/1565306013 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f90f4053950 0x7f90f4055e10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f9100008000 tx=0x7f9100009040).stop 2022-08-01T19:38:01.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.086+0000 7f9112773700 1 -- 172.21.15.17:0/1565306013 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f910c1402d0 msgr2=0x7f910c134080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:01.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.086+0000 7f9112773700 1 --2- 172.21.15.17:0/1565306013 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f910c1402d0 0x7f910c134080 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f910c134700 tx=0x7f9104008040).stop 2022-08-01T19:38:01.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.086+0000 7f9112773700 1 -- 172.21.15.17:0/1565306013 shutdown_connections 2022-08-01T19:38:01.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.086+0000 7f9112773700 1 --2- 172.21.15.17:0/1565306013 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f90f4053950 0x7f90f4055e10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:01.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.086+0000 7f9112773700 1 --2- 172.21.15.17:0/1565306013 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f910c1402d0 0x7f910c134080 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:01.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.086+0000 7f9112773700 1 --2- 172.21.15.17:0/1565306013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f910c0b31d0 0x7f910c133b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:01.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.086+0000 7f9112773700 1 --2- 172.21.15.17:0/1565306013 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f910c0a9da0 0x7f910c13f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:01.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.086+0000 7f9112773700 1 -- 172.21.15.17:0/1565306013 >> 172.21.15.17:0/1565306013 conn(0x7f910c063870 msgr2=0x7f910c0b0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:38:01.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.086+0000 7f9112773700 1 -- 172.21.15.17:0/1565306013 shutdown_connections 2022-08-01T19:38:01.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.086+0000 7f9112773700 1 -- 172.21.15.17:0/1565306013 wait complete. 2022-08-01T19:38:01.102 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T19:37:34.595906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T19:37:35.889700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T19:38:01.103 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:38:01.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.206+0000 7f705bfff700 1 Processor -- start 2022-08-01T19:38:01.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.206+0000 7f705bfff700 1 -- start start 2022-08-01T19:38:01.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f705bfff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f705c0b0d10 0x7f705c146df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:01.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f705bfff700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f705c0b12d0 con 0x7f705c0b0560 2022-08-01T19:38:01.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f705bfff700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f705c0a9700 con 0x7f705c0a9320 2022-08-01T19:38:01.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f705bfff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705c0b1500 con 0x7f705c0b0d10 2022-08-01T19:38:01.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f705b7fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f705c0b0560 0x7f705c140350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59802/0 (socket says 172.21.15.17:59802) 2022-08-01T19:38:01.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f705b7fe700 1 -- 172.21.15.17:0/700193472 learned_addr learned my addr 172.21.15.17:0/700193472 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:38:01.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f705a7fc700 1 --2- 172.21.15.17:0/700193472 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f705c0b0d10 0x7f705c146df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:01.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f7059ffb700 1 -- 172.21.15.17:0/700193472 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2809569584 0 0) 0x7f705c0b12d0 con 0x7f705c0b0560 2022-08-01T19:38:01.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f7059ffb700 1 -- 172.21.15.17:0/700193472 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7040003260 con 0x7f705c0b0560 2022-08-01T19:38:01.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f7059ffb700 1 -- 172.21.15.17:0/700193472 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3341339979 0 0) 0x7f705c0a9700 con 0x7f705c0a9320 2022-08-01T19:38:01.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f7059ffb700 1 -- 172.21.15.17:0/700193472 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f705c0b12d0 con 0x7f705c0a9320 2022-08-01T19:38:01.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f7059ffb700 1 -- 172.21.15.17:0/700193472 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 289104759 0 0) 0x7f7040003260 con 0x7f705c0b0560 2022-08-01T19:38:01.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f7059ffb700 1 -- 172.21.15.17:0/700193472 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f705c0b0d10 msgr2=0x7f705c146df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:38:01.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f7059ffb700 1 --2- 172.21.15.17:0/700193472 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f705c0b0d10 0x7f705c146df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:01.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f7059ffb700 1 -- 172.21.15.17:0/700193472 >> v1:172.21.15.77:6789/0 conn(0x7f705c0a9320 legacy=0x7f705c1466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:01.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f7059ffb700 1 -- 172.21.15.17:0/700193472 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f705c0b17a0 con 0x7f705c0b0560 2022-08-01T19:38:01.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f7059ffb700 1 -- 172.21.15.17:0/700193472 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7f7054001960 con 0x7f705c0b0560 2022-08-01T19:38:01.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f7059ffb700 1 -- 172.21.15.17:0/700193472 >> v1:172.21.15.17:6789/0 conn(0x7f705c0b0560 legacy=0x7f705c140350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:01.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f7059ffb700 1 --2- 172.21.15.17:0/700193472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7040003910 0x7f7040003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:01.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f7059ffb700 1 --2- 172.21.15.17:0/700193472 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f705c0b0d10 0x7f7040004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:01.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f7059ffb700 1 --2- 172.21.15.17:0/700193472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70400049f0 0x7f7040004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:01.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f7059ffb700 1 -- 172.21.15.17:0/700193472 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705c0b1500 con 0x7f7040003910 2022-08-01T19:38:01.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f7059ffb700 1 -- 172.21.15.17:0/700193472 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704000b800 con 0x7f70400049f0 2022-08-01T19:38:01.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f7059ffb700 1 -- 172.21.15.17:0/700193472 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f704000b9c0 con 0x7f705c0b0d10 2022-08-01T19:38:01.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f705affd700 1 --2- 172.21.15.17:0/700193472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7040003910 0x7f7040003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:01.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f705a7fc700 1 --2- 172.21.15.17:0/700193472 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f705c0b0d10 0x7f7040004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:01.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f705affd700 1 -- 172.21.15.17:0/700193472 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f705c0b0d10 msgr2=0x7f7040004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:38:01.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f705affd700 1 --2- 172.21.15.17:0/700193472 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f705c0b0d10 0x7f7040004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:01.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f705affd700 1 -- 172.21.15.17:0/700193472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70400049f0 msgr2=0x7f7040004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:38:01.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f705b7fe700 1 --2- 172.21.15.17:0/700193472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70400049f0 0x7f7040004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:01.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f705affd700 1 --2- 172.21.15.17:0/700193472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70400049f0 0x7f7040004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:01.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f705affd700 1 -- 172.21.15.17:0/700193472 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7040003260 con 0x7f7040003910 2022-08-01T19:38:01.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f705a7fc700 1 --2- 172.21.15.17:0/700193472 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f705c0b0d10 0x7f7040004500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:38:01.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f705b7fe700 1 --2- 172.21.15.17:0/700193472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70400049f0 0x7f7040004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:38:01.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f705affd700 1 --2- 172.21.15.17:0/700193472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7040003910 0x7f7040003d10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f704c001110 tx=0x7f704c007a00).ready entity=mon.0 client_cookie=378d43fbd8a05720 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:38:01.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f7059ffb700 1 -- 172.21.15.17:0/700193472 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f704c007d10 con 0x7f7040003910 2022-08-01T19:38:01.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f7059ffb700 1 -- 172.21.15.17:0/700193472 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704c007030 con 0x7f7040003910 2022-08-01T19:38:01.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f7059ffb700 1 -- 172.21.15.17:0/700193472 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f704c007390 con 0x7f7040003910 2022-08-01T19:38:01.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f705bfff700 1 -- 172.21.15.17:0/700193472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7040003910 msgr2=0x7f7040003d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:01.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f705bfff700 1 --2- 172.21.15.17:0/700193472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7040003910 0x7f7040003d10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f704c001110 tx=0x7f704c007a00).stop 2022-08-01T19:38:01.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f705b7fe700 1 -- 172.21.15.17:0/700193472 reap_dead start 2022-08-01T19:38:01.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f705bfff700 1 -- 172.21.15.17:0/700193472 shutdown_connections 2022-08-01T19:38:01.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f705bfff700 1 -- 172.21.15.17:0/700193472 >> 172.21.15.17:0/700193472 conn(0x7f705c0612d0 msgr2=0x7f705c062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:38:01.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f705bfff700 1 -- 172.21.15.17:0/700193472 shutdown_connections 2022-08-01T19:38:01.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.210+0000 7f705bfff700 1 -- 172.21.15.17:0/700193472 wait complete. 2022-08-01T19:38:01.234 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.214+0000 7f705bfff700 1 Processor -- start 2022-08-01T19:38:01.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.214+0000 7f705bfff700 1 -- start start 2022-08-01T19:38:01.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.218+0000 7f705bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7040003910 0x7f705c13b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:01.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.218+0000 7f705bfff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f705c0ac820 0x7f705c0a99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:01.236 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.218+0000 7f705bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f705c0b38a0 0x7f705c155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:01.236 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.218+0000 7f705bfff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705c067bb0 con 0x7f705c0b38a0 2022-08-01T19:38:01.236 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.218+0000 7f705bfff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705c0a9ec0 con 0x7f7040003910 2022-08-01T19:38:01.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.218+0000 7f705bfff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705c067de0 con 0x7f705c0ac820 2022-08-01T19:38:01.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.218+0000 7f705b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f705c0b38a0 0x7f705c155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:01.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.218+0000 7f705b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f705c0b38a0 0x7f705c155c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39388/0 (socket says 172.21.15.17:39388) 2022-08-01T19:38:01.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.218+0000 7f705affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7040003910 0x7f705c13b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:01.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.218+0000 7f705b7fe700 1 -- 172.21.15.17:0/3169228544 learned_addr learned my addr 172.21.15.17:0/3169228544 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:38:01.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.218+0000 7f705a7fc700 1 --2- 172.21.15.17:0/3169228544 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f705c0ac820 0x7f705c0a99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:01.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.218+0000 7f705b7fe700 1 -- 172.21.15.17:0/3169228544 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f705c0ac820 msgr2=0x7f705c0a99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:01.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.218+0000 7f705b7fe700 1 --2- 172.21.15.17:0/3169228544 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f705c0ac820 0x7f705c0a99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:01.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.218+0000 7f705b7fe700 1 -- 172.21.15.17:0/3169228544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7040003910 msgr2=0x7f705c13b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:01.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.218+0000 7f705b7fe700 1 --2- 172.21.15.17:0/3169228544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7040003910 0x7f705c13b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:01.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.218+0000 7f705b7fe700 1 -- 172.21.15.17:0/3169228544 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f705c0ab190 con 0x7f705c0b38a0 2022-08-01T19:38:01.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.218+0000 7f705a7fc700 1 --2- 172.21.15.17:0/3169228544 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f705c0ac820 0x7f705c0a99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:38:01.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.218+0000 7f705b7fe700 1 --2- 172.21.15.17:0/3169228544 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f705c0b38a0 0x7f705c155c80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f7054001b40 tx=0x7f7054009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:38:01.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.218+0000 7f704bfff700 1 -- 172.21.15.17:0/3169228544 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7054004ab0 con 0x7f705c0b38a0 2022-08-01T19:38:01.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.218+0000 7f704bfff700 1 -- 172.21.15.17:0/3169228544 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7054004c10 con 0x7f705c0b38a0 2022-08-01T19:38:01.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.218+0000 7f705bfff700 1 -- 172.21.15.17:0/3169228544 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f705c145c10 con 0x7f705c0b38a0 2022-08-01T19:38:01.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.218+0000 7f704bfff700 1 -- 172.21.15.17:0/3169228544 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f70540070d0 con 0x7f705c0b38a0 2022-08-01T19:38:01.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.218+0000 7f705bfff700 1 -- 172.21.15.17:0/3169228544 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f705c13b700 con 0x7f705c0b38a0 2022-08-01T19:38:01.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.218+0000 7f705bfff700 1 -- 172.21.15.17:0/3169228544 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f705c0b3ed0 con 0x7f705c0b38a0 2022-08-01T19:38:01.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.222+0000 7f704bfff700 1 -- 172.21.15.17:0/3169228544 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7054007230 con 0x7f705c0b38a0 2022-08-01T19:38:01.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.222+0000 7f704bfff700 1 --2- 172.21.15.17:0/3169228544 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f70380539e0 0x7f7038055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:01.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.222+0000 7f704bfff700 1 -- 172.21.15.17:0/3169228544 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f705406bcb0 con 0x7f705c0b38a0 2022-08-01T19:38:01.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.222+0000 7f705affd700 1 --2- 172.21.15.17:0/3169228544 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f70380539e0 0x7f7038055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:01.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.222+0000 7f705affd700 1 --2- 172.21.15.17:0/3169228544 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f70380539e0 0x7f7038055ea0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f704c01b000 tx=0x7f704c016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:38:01.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.230+0000 7f704bfff700 1 -- 172.21.15.17:0/3169228544 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7054042a70 con 0x7f705c0b38a0 2022-08-01T19:38:01.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.354+0000 7f705bfff700 1 -- 172.21.15.17:0/3169228544 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f705c145da0 con 0x7f705c0b38a0 2022-08-01T19:38:01.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.354+0000 7f704bfff700 1 -- 172.21.15.17:0/3169228544 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v55) v1 ==== 110+0+10 (secure 0 0 0) 0x7f7054046090 con 0x7f705c0b38a0 2022-08-01T19:38:01.359 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:38:01.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.358+0000 7f705bfff700 1 -- 172.21.15.17:0/3169228544 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f70380539e0 msgr2=0x7f7038055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:01.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.358+0000 7f705bfff700 1 --2- 172.21.15.17:0/3169228544 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f70380539e0 0x7f7038055ea0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f704c01b000 tx=0x7f704c016000).stop 2022-08-01T19:38:01.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.358+0000 7f705bfff700 1 -- 172.21.15.17:0/3169228544 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f705c0b38a0 msgr2=0x7f705c155c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:01.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.358+0000 7f705bfff700 1 --2- 172.21.15.17:0/3169228544 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f705c0b38a0 0x7f705c155c80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f7054001b40 tx=0x7f7054009040).stop 2022-08-01T19:38:01.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.358+0000 7f705bfff700 1 -- 172.21.15.17:0/3169228544 shutdown_connections 2022-08-01T19:38:01.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.358+0000 7f705bfff700 1 --2- 172.21.15.17:0/3169228544 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f70380539e0 0x7f7038055ea0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:01.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.358+0000 7f705bfff700 1 --2- 172.21.15.17:0/3169228544 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f705c0b38a0 0x7f705c155c80 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:01.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.358+0000 7f705bfff700 1 --2- 172.21.15.17:0/3169228544 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f705c0ac820 0x7f705c0a99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:01.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.358+0000 7f705bfff700 1 --2- 172.21.15.17:0/3169228544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7040003910 0x7f705c13b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:01.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.358+0000 7f705bfff700 1 -- 172.21.15.17:0/3169228544 >> 172.21.15.17:0/3169228544 conn(0x7f705c0612d0 msgr2=0x7f705c0aedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:38:01.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.358+0000 7f705bfff700 1 -- 172.21.15.17:0/3169228544 shutdown_connections 2022-08-01T19:38:01.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.358+0000 7f705bfff700 1 -- 172.21.15.17:0/3169228544 wait complete. 2022-08-01T19:38:01.377 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:38:01.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.486+0000 7f03a23f5700 1 Processor -- start 2022-08-01T19:38:01.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.486+0000 7f03a23f5700 1 -- start start 2022-08-01T19:38:01.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.490+0000 7f03a23f5700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f039c0ad750 0x7f039c061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:01.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.490+0000 7f03a23f5700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f039c0aee80 con 0x7f039c1405d0 2022-08-01T19:38:01.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.490+0000 7f03a23f5700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f039c0b70b0 con 0x7f039c0614b0 2022-08-01T19:38:01.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.490+0000 7f03a23f5700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f039c0af0b0 con 0x7f039c0ad750 2022-08-01T19:38:01.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.490+0000 7f03a1bf4700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f039c1405d0 0x7f039c1466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59820/0 (socket says 172.21.15.17:59820) 2022-08-01T19:38:01.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.490+0000 7f03a1bf4700 1 -- 172.21.15.17:0/3639584109 learned_addr learned my addr 172.21.15.17:0/3639584109 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:38:01.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.490+0000 7f03a0bf2700 1 --2- 172.21.15.17:0/3639584109 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f039c0ad750 0x7f039c061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:01.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.490+0000 7f038bfff700 1 -- 172.21.15.17:0/3639584109 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2760063806 0 0) 0x7f039c0aee80 con 0x7f039c1405d0 2022-08-01T19:38:01.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.490+0000 7f038bfff700 1 -- 172.21.15.17:0/3639584109 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0384003260 con 0x7f039c1405d0 2022-08-01T19:38:01.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.490+0000 7f03a0bf2700 1 -- 172.21.15.17:0/3639584109 >> v1:172.21.15.77:6789/0 conn(0x7f039c0614b0 legacy=0x7f039c0b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:01.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.490+0000 7f038bfff700 1 -- 172.21.15.17:0/3639584109 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1499322941 0 0) 0x7f039c0b70b0 con 0x7f039c0614b0 2022-08-01T19:38:01.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.490+0000 7f03a0bf2700 1 -- 172.21.15.17:0/3639584109 >> v1:172.21.15.17:6789/0 conn(0x7f039c1405d0 legacy=0x7f039c1466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:01.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.490+0000 7f03a0bf2700 1 -- 172.21.15.17:0/3639584109 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039c0af350 con 0x7f039c0ad750 2022-08-01T19:38:01.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.490+0000 7f03a0bf2700 1 --2- 172.21.15.17:0/3639584109 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f039c0ad750 0x7f039c061890 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f038c002f70 tx=0x7f038c006df0).ready entity=mon.2 client_cookie=40935721902e9d05 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:38:01.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.490+0000 7f038bfff700 1 -- 172.21.15.17:0/3639584109 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f038c00d570 con 0x7f039c0ad750 2022-08-01T19:38:01.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.490+0000 7f038bfff700 1 -- 172.21.15.17:0/3639584109 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f038c000e90 con 0x7f039c0ad750 2022-08-01T19:38:01.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.490+0000 7f038bfff700 1 -- 172.21.15.17:0/3639584109 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f038c00d8c0 con 0x7f039c0ad750 2022-08-01T19:38:01.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.490+0000 7f03a23f5700 1 -- 172.21.15.17:0/3639584109 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f039c0ad750 msgr2=0x7f039c061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:01.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.490+0000 7f03a23f5700 1 --2- 172.21.15.17:0/3639584109 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f039c0ad750 0x7f039c061890 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f038c002f70 tx=0x7f038c006df0).stop 2022-08-01T19:38:01.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.490+0000 7f03a23f5700 1 -- 172.21.15.17:0/3639584109 shutdown_connections 2022-08-01T19:38:01.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.490+0000 7f03a23f5700 1 --2- 172.21.15.17:0/3639584109 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f039c0ad750 0x7f039c061890 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:01.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.490+0000 7f03a23f5700 1 -- 172.21.15.17:0/3639584109 >> 172.21.15.17:0/3639584109 conn(0x7f039c066f70 msgr2=0x7f039c067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:38:01.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.490+0000 7f03a23f5700 1 -- 172.21.15.17:0/3639584109 shutdown_connections 2022-08-01T19:38:01.499 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.490+0000 7f03a23f5700 1 -- 172.21.15.17:0/3639584109 wait complete. 2022-08-01T19:38:01.499 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.490+0000 7f03a23f5700 1 Processor -- start 2022-08-01T19:38:01.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.490+0000 7f03a23f5700 1 -- start start 2022-08-01T19:38:01.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.490+0000 7f03a23f5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f039c0614b0 0x7f039c0b3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:01.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.490+0000 7f03a23f5700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f039c0ad750 0x7f039c13f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:01.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.490+0000 7f03a23f5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f039c1405d0 0x7f039c13fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:01.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.490+0000 7f03a23f5700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039c05d4a0 con 0x7f039c0614b0 2022-08-01T19:38:01.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.490+0000 7f03a23f5700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039c153680 con 0x7f039c1405d0 2022-08-01T19:38:01.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.490+0000 7f03a23f5700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f039c1533e0 con 0x7f039c0ad750 2022-08-01T19:38:01.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.494+0000 7f03a13f3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f039c0614b0 0x7f039c0b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:01.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.494+0000 7f03a0bf2700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f039c0ad750 0x7f039c13f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:01.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.494+0000 7f03a1bf4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f039c1405d0 0x7f039c13fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:01.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.494+0000 7f03a13f3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f039c0614b0 0x7f039c0b3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39398/0 (socket says 172.21.15.17:39398) 2022-08-01T19:38:01.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.494+0000 7f03a13f3700 1 -- 172.21.15.17:0/5987412 learned_addr learned my addr 172.21.15.17:0/5987412 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:38:01.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.494+0000 7f03a0bf2700 1 -- 172.21.15.17:0/5987412 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f039c1405d0 msgr2=0x7f039c13fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:01.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.494+0000 7f03a0bf2700 1 --2- 172.21.15.17:0/5987412 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f039c1405d0 0x7f039c13fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:01.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.494+0000 7f03a0bf2700 1 -- 172.21.15.17:0/5987412 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f039c0614b0 msgr2=0x7f039c0b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:01.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.494+0000 7f03a0bf2700 1 --2- 172.21.15.17:0/5987412 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f039c0614b0 0x7f039c0b3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:01.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.494+0000 7f03a0bf2700 1 -- 172.21.15.17:0/5987412 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f038c00b180 con 0x7f039c0ad750 2022-08-01T19:38:01.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.494+0000 7f03a13f3700 1 --2- 172.21.15.17:0/5987412 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f039c0614b0 0x7f039c0b3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:38:01.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.494+0000 7f03a0bf2700 1 --2- 172.21.15.17:0/5987412 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f039c0ad750 0x7f039c13f920 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f038c002f70 tx=0x7f038c014000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:38:01.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.494+0000 7f038a7fc700 1 -- 172.21.15.17:0/5987412 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f038c013d60 con 0x7f039c0ad750 2022-08-01T19:38:01.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.494+0000 7f038a7fc700 1 -- 172.21.15.17:0/5987412 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f038c01e400 con 0x7f039c0ad750 2022-08-01T19:38:01.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.494+0000 7f03a23f5700 1 -- 172.21.15.17:0/5987412 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f039c0b0ca0 con 0x7f039c0ad750 2022-08-01T19:38:01.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.494+0000 7f038a7fc700 1 -- 172.21.15.17:0/5987412 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f038c01e760 con 0x7f039c0ad750 2022-08-01T19:38:01.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.494+0000 7f03a23f5700 1 -- 172.21.15.17:0/5987412 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f039c140300 con 0x7f039c0ad750 2022-08-01T19:38:01.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.494+0000 7f03a23f5700 1 -- 172.21.15.17:0/5987412 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f039c0af5f0 con 0x7f039c0ad750 2022-08-01T19:38:01.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.498+0000 7f038a7fc700 1 -- 172.21.15.17:0/5987412 <== mon.2 v2:172.21.15.77:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f038c025030 con 0x7f039c0ad750 2022-08-01T19:38:01.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.498+0000 7f038a7fc700 1 --2- 172.21.15.17:0/5987412 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f037c0539e0 0x7f037c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:01.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.498+0000 7f038a7fc700 1 -- 172.21.15.17:0/5987412 <== mon.2 v2:172.21.15.77:3301/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f038c06cde0 con 0x7f039c0ad750 2022-08-01T19:38:01.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.498+0000 7f03a13f3700 1 --2- 172.21.15.17:0/5987412 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f037c0539e0 0x7f037c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:01.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.498+0000 7f03a13f3700 1 --2- 172.21.15.17:0/5987412 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f037c0539e0 0x7f037c055ea0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f039800d000 tx=0x7f0398009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:38:01.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.510+0000 7f038a7fc700 1 -- 172.21.15.17:0/5987412 <== mon.2 v2:172.21.15.77:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f038c043a90 con 0x7f039c0ad750 2022-08-01T19:38:01.637 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.634+0000 7f03a23f5700 1 -- 172.21.15.17:0/5987412 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f039c0574e0 con 0x7f039c0ad750 2022-08-01T19:38:01.638 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.634+0000 7f038a7fc700 1 -- 172.21.15.17:0/5987412 <== mon.2 v2:172.21.15.77:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v55) v1 ==== 92+0+10 (secure 0 0 0) 0x7f038c023070 con 0x7f039c0ad750 2022-08-01T19:38:01.638 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:38:01.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.642+0000 7f03a23f5700 1 -- 172.21.15.17:0/5987412 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f037c0539e0 msgr2=0x7f037c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:01.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.642+0000 7f03a23f5700 1 --2- 172.21.15.17:0/5987412 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f037c0539e0 0x7f037c055ea0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f039800d000 tx=0x7f0398009040).stop 2022-08-01T19:38:01.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.642+0000 7f03a23f5700 1 -- 172.21.15.17:0/5987412 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f039c0ad750 msgr2=0x7f039c13f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:01.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.642+0000 7f03a23f5700 1 --2- 172.21.15.17:0/5987412 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f039c0ad750 0x7f039c13f920 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f038c002f70 tx=0x7f038c014000).stop 2022-08-01T19:38:01.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.642+0000 7f03a23f5700 1 -- 172.21.15.17:0/5987412 shutdown_connections 2022-08-01T19:38:01.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.642+0000 7f03a23f5700 1 --2- 172.21.15.17:0/5987412 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f037c0539e0 0x7f037c055ea0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:01.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.642+0000 7f03a23f5700 1 --2- 172.21.15.17:0/5987412 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f039c1405d0 0x7f039c13fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:01.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.642+0000 7f03a23f5700 1 --2- 172.21.15.17:0/5987412 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f039c0ad750 0x7f039c13f920 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:01.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.642+0000 7f03a23f5700 1 --2- 172.21.15.17:0/5987412 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f039c0614b0 0x7f039c0b3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:01.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.642+0000 7f03a23f5700 1 -- 172.21.15.17:0/5987412 >> 172.21.15.17:0/5987412 conn(0x7f039c066f70 msgr2=0x7f039c0b8a10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:38:01.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.642+0000 7f03a23f5700 1 -- 172.21.15.17:0/5987412 shutdown_connections 2022-08-01T19:38:01.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.642+0000 7f03a23f5700 1 -- 172.21.15.17:0/5987412 wait complete. 2022-08-01T19:38:01.655 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T19:38:01.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.762+0000 7f9b5bfff700 1 Processor -- start 2022-08-01T19:38:01.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.762+0000 7f9b5bfff700 1 -- start start 2022-08-01T19:38:01.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.762+0000 7f9b5bfff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9b5c0b2710 0x7f9b5c14afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:01.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.762+0000 7f9b5bfff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9b5c0b01a0 0x7f9b5c14b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:01.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.762+0000 7f9b5bfff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f9b5c0afd60 0x7f9b5c0a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:01.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.762+0000 7f9b5bfff700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b5c0a7250 con 0x7f9b5c0b2710 2022-08-01T19:38:01.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.762+0000 7f9b5bfff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b5c0a7480 con 0x7f9b5c0afd60 2022-08-01T19:38:01.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.762+0000 7f9b5bfff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c0a7720 con 0x7f9b5c0b01a0 2022-08-01T19:38:01.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.762+0000 7f9b5affd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9b5c0b2710 0x7f9b5c14afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:01.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.762+0000 7f9b5affd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9b5c0b2710 0x7f9b5c14afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39406/0 (socket says 172.21.15.17:39406) 2022-08-01T19:38:01.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.762+0000 7f9b5affd700 1 -- 172.21.15.17:0/1582462717 learned_addr learned my addr 172.21.15.17:0/1582462717 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:38:01.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.762+0000 7f9b5a7fc700 1 --2- 172.21.15.17:0/1582462717 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9b5c0b01a0 0x7f9b5c14b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:01.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.762+0000 7f9b5b7fe700 1 --2- 172.21.15.17:0/1582462717 >> v2:172.21.15.77:3300/0 conn(0x7f9b5c0afd60 0x7f9b5c0a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:01.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.762+0000 7f9b5affd700 1 -- 172.21.15.17:0/1582462717 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9b5c0b01a0 msgr2=0x7f9b5c14b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:38:01.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.762+0000 7f9b5affd700 1 --2- 172.21.15.17:0/1582462717 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9b5c0b01a0 0x7f9b5c14b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:01.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.762+0000 7f9b5affd700 1 -- 172.21.15.17:0/1582462717 >> v2:172.21.15.77:3300/0 conn(0x7f9b5c0afd60 msgr2=0x7f9b5c0a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:38:01.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.762+0000 7f9b5affd700 1 --2- 172.21.15.17:0/1582462717 >> v2:172.21.15.77:3300/0 conn(0x7f9b5c0afd60 0x7f9b5c0a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:01.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.762+0000 7f9b5affd700 1 -- 172.21.15.17:0/1582462717 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b5c0a79c0 con 0x7f9b5c0b2710 2022-08-01T19:38:01.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.762+0000 7f9b5affd700 1 --2- 172.21.15.17:0/1582462717 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b5c0b2710 0x7f9b5c14afd0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f9b50002f70 tx=0x7f9b50006eb0).ready entity=mon.0 client_cookie=176ac1dae60f73fa server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:38:01.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.762+0000 7f9b59ffb700 1 -- 172.21.15.17:0/1582462717 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b5000d570 con 0x7f9b5c0b2710 2022-08-01T19:38:01.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.762+0000 7f9b59ffb700 1 -- 172.21.15.17:0/1582462717 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b50000e90 con 0x7f9b5c0b2710 2022-08-01T19:38:01.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.762+0000 7f9b59ffb700 1 -- 172.21.15.17:0/1582462717 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b5000d900 con 0x7f9b5c0b2710 2022-08-01T19:38:01.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.762+0000 7f9b5bfff700 1 -- 172.21.15.17:0/1582462717 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b5c0b2710 msgr2=0x7f9b5c14afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:01.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.762+0000 7f9b5bfff700 1 --2- 172.21.15.17:0/1582462717 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b5c0b2710 0x7f9b5c14afd0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f9b50002f70 tx=0x7f9b50006eb0).stop 2022-08-01T19:38:01.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.762+0000 7f9b5bfff700 1 -- 172.21.15.17:0/1582462717 shutdown_connections 2022-08-01T19:38:01.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.762+0000 7f9b5bfff700 1 --2- 172.21.15.17:0/1582462717 >> v2:172.21.15.77:3300/0 conn(0x7f9b5c0afd60 0x7f9b5c0a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:01.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.762+0000 7f9b5bfff700 1 --2- 172.21.15.17:0/1582462717 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9b5c0b01a0 0x7f9b5c14b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:01.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.762+0000 7f9b5bfff700 1 --2- 172.21.15.17:0/1582462717 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b5c0b2710 0x7f9b5c14afd0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:01.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.762+0000 7f9b5bfff700 1 -- 172.21.15.17:0/1582462717 >> 172.21.15.17:0/1582462717 conn(0x7f9b5c060310 msgr2=0x7f9b5c0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:38:01.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.762+0000 7f9b5bfff700 1 -- 172.21.15.17:0/1582462717 shutdown_connections 2022-08-01T19:38:01.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.762+0000 7f9b5bfff700 1 -- 172.21.15.17:0/1582462717 wait complete. 2022-08-01T19:38:01.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.762+0000 7f9b5bfff700 1 Processor -- start 2022-08-01T19:38:01.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.762+0000 7f9b5bfff700 1 -- start start 2022-08-01T19:38:01.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.766+0000 7f9b5bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b5c0afd60 0x7f9b5c0abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:01.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.766+0000 7f9b5bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b5c0b01a0 0x7f9b5c0635d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:01.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.766+0000 7f9b5bfff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9b5c063190 0x7f9b5c066380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:01.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.766+0000 7f9b5bfff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c0ac1e0 con 0x7f9b5c0b01a0 2022-08-01T19:38:01.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.766+0000 7f9b5bfff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c067340 con 0x7f9b5c0afd60 2022-08-01T19:38:01.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.766+0000 7f9b5bfff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c1571e0 con 0x7f9b5c063190 2022-08-01T19:38:01.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.766+0000 7f9b5a7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b5c0b01a0 0x7f9b5c0635d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:01.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.766+0000 7f9b5a7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b5c0b01a0 0x7f9b5c0635d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39412/0 (socket says 172.21.15.17:39412) 2022-08-01T19:38:01.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.766+0000 7f9b5a7fc700 1 -- 172.21.15.17:0/3450535403 learned_addr learned my addr 172.21.15.17:0/3450535403 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:38:01.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.766+0000 7f9b5affd700 1 --2- 172.21.15.17:0/3450535403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b5c0afd60 0x7f9b5c0abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:01.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.766+0000 7f9b5b7fe700 1 --2- 172.21.15.17:0/3450535403 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9b5c063190 0x7f9b5c066380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:01.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.766+0000 7f9b5a7fc700 1 -- 172.21.15.17:0/3450535403 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9b5c063190 msgr2=0x7f9b5c066380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:01.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.766+0000 7f9b5a7fc700 1 --2- 172.21.15.17:0/3450535403 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9b5c063190 0x7f9b5c066380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:01.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.766+0000 7f9b5a7fc700 1 -- 172.21.15.17:0/3450535403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b5c0afd60 msgr2=0x7f9b5c0abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:01.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.766+0000 7f9b5a7fc700 1 --2- 172.21.15.17:0/3450535403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b5c0afd60 0x7f9b5c0abcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:01.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.766+0000 7f9b5a7fc700 1 -- 172.21.15.17:0/3450535403 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b5000b1e0 con 0x7f9b5c0b01a0 2022-08-01T19:38:01.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.766+0000 7f9b5affd700 1 --2- 172.21.15.17:0/3450535403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b5c0afd60 0x7f9b5c0abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:38:01.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.766+0000 7f9b5b7fe700 1 --2- 172.21.15.17:0/3450535403 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9b5c063190 0x7f9b5c066380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:38:01.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.766+0000 7f9b5a7fc700 1 --2- 172.21.15.17:0/3450535403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b5c0b01a0 0x7f9b5c0635d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7f9b4c00c000 tx=0x7f9b4c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:38:01.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.766+0000 7f9b4bfff700 1 -- 172.21.15.17:0/3450535403 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b4c010780 con 0x7f9b5c0b01a0 2022-08-01T19:38:01.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.766+0000 7f9b4bfff700 1 -- 172.21.15.17:0/3450535403 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b4c0108e0 con 0x7f9b5c0b01a0 2022-08-01T19:38:01.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.766+0000 7f9b5bfff700 1 -- 172.21.15.17:0/3450535403 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b5c157480 con 0x7f9b5c0b01a0 2022-08-01T19:38:01.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.766+0000 7f9b4bfff700 1 -- 172.21.15.17:0/3450535403 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b4c010c40 con 0x7f9b5c0b01a0 2022-08-01T19:38:01.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.766+0000 7f9b5bfff700 1 -- 172.21.15.17:0/3450535403 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b5c063ac0 con 0x7f9b5c0b01a0 2022-08-01T19:38:01.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.770+0000 7f9b5bfff700 1 -- 172.21.15.17:0/3450535403 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b5c066bb0 con 0x7f9b5c0b01a0 2022-08-01T19:38:01.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.770+0000 7f9b4bfff700 1 -- 172.21.15.17:0/3450535403 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9b4c01c460 con 0x7f9b5c0b01a0 2022-08-01T19:38:01.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.770+0000 7f9b4bfff700 1 --2- 172.21.15.17:0/3450535403 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9b380539e0 0x7f9b38055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:01.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.770+0000 7f9b4bfff700 1 -- 172.21.15.17:0/3450535403 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9b4c06c520 con 0x7f9b5c0b01a0 2022-08-01T19:38:01.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.770+0000 7f9b5affd700 1 --2- 172.21.15.17:0/3450535403 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9b380539e0 0x7f9b38055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:01.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.770+0000 7f9b5affd700 1 --2- 172.21.15.17:0/3450535403 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9b380539e0 0x7f9b38055ea0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f9b50017000 tx=0x7f9b50015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:38:01.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.782+0000 7f9b4bfff700 1 -- 172.21.15.17:0/3450535403 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9b4c069e90 con 0x7f9b5c0b01a0 2022-08-01T19:38:01.910 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.906+0000 7f9b5bfff700 1 -- 172.21.15.17:0/3450535403 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9b5c067510 con 0x7f9b5c0b01a0 2022-08-01T19:38:01.912 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.906+0000 7f9b4bfff700 1 -- 172.21.15.17:0/3450535403 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+10 (secure 0 0 0) 0x7f9b4c043280 con 0x7f9b5c0b01a0 2022-08-01T19:38:01.912 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:38:01.914 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.910+0000 7f9b5bfff700 1 -- 172.21.15.17:0/3450535403 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9b380539e0 msgr2=0x7f9b38055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:01.915 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.910+0000 7f9b5bfff700 1 --2- 172.21.15.17:0/3450535403 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9b380539e0 0x7f9b38055ea0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f9b50017000 tx=0x7f9b50015000).stop 2022-08-01T19:38:01.915 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.910+0000 7f9b5bfff700 1 -- 172.21.15.17:0/3450535403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b5c0b01a0 msgr2=0x7f9b5c0635d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:01.915 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.910+0000 7f9b5bfff700 1 --2- 172.21.15.17:0/3450535403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b5c0b01a0 0x7f9b5c0635d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7f9b4c00c000 tx=0x7f9b4c006df0).stop 2022-08-01T19:38:01.916 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.910+0000 7f9b5bfff700 1 -- 172.21.15.17:0/3450535403 shutdown_connections 2022-08-01T19:38:01.916 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.910+0000 7f9b5bfff700 1 --2- 172.21.15.17:0/3450535403 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9b380539e0 0x7f9b38055ea0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:01.917 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.910+0000 7f9b5bfff700 1 --2- 172.21.15.17:0/3450535403 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9b5c063190 0x7f9b5c066380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:01.917 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.910+0000 7f9b5bfff700 1 --2- 172.21.15.17:0/3450535403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b5c0b01a0 0x7f9b5c0635d0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:01.917 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.910+0000 7f9b5bfff700 1 --2- 172.21.15.17:0/3450535403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b5c0afd60 0x7f9b5c0abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:01.917 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.910+0000 7f9b5bfff700 1 -- 172.21.15.17:0/3450535403 >> 172.21.15.17:0/3450535403 conn(0x7f9b5c060310 msgr2=0x7f9b5c0addd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:38:01.918 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.910+0000 7f9b5bfff700 1 -- 172.21.15.17:0/3450535403 shutdown_connections 2022-08-01T19:38:01.918 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:01.910+0000 7f9b5bfff700 1 -- 172.21.15.17:0/3450535403 wait complete. 2022-08-01T19:38:01.925 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T19:38:02.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.034+0000 7f721efa8700 1 Processor -- start 2022-08-01T19:38:02.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.034+0000 7f721efa8700 1 -- start start 2022-08-01T19:38:02.040 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.034+0000 7f721efa8700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f72180614b0 0x7f7218061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:02.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.034+0000 7f721efa8700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f72181405d0 0x7f7218140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:02.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.034+0000 7f721efa8700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72180aee80 con 0x7f72180ad750 2022-08-01T19:38:02.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.034+0000 7f721efa8700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72180b70b0 con 0x7f72180614b0 2022-08-01T19:38:02.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.034+0000 7f721efa8700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72180af0b0 con 0x7f72181405d0 2022-08-01T19:38:02.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.034+0000 7f721e7a7700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f72181405d0 0x7f7218140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:02.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.034+0000 7f721dfa6700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f72180614b0 0x7f7218061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:02.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.034+0000 7f721e7a7700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f72181405d0 0x7f7218140a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:35524/0 (socket says 172.21.15.17:35524) 2022-08-01T19:38:02.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.034+0000 7f721e7a7700 1 -- 172.21.15.17:0/2343098777 learned_addr learned my addr 172.21.15.17:0/2343098777 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:38:02.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.034+0000 7f721dfa6700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f72180614b0 0x7f7218061890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:37342/0 (socket says 172.21.15.17:37342) 2022-08-01T19:38:02.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.034+0000 7f721d7a5700 1 --1- 172.21.15.17:0/2343098777 >> v1:172.21.15.77:6789/0 conn(0x7f72180ad750 0x7f72180b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:48414/0 (socket says 172.21.15.17:48414) 2022-08-01T19:38:02.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.038+0000 7f721dfa6700 1 -- 172.21.15.17:0/2343098777 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f72181405d0 msgr2=0x7f7218140a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:38:02.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.038+0000 7f721dfa6700 1 --2- 172.21.15.17:0/2343098777 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f72181405d0 0x7f7218140a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:02.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.038+0000 7f721dfa6700 1 -- 172.21.15.17:0/2343098777 >> v1:172.21.15.77:6789/0 conn(0x7f72180ad750 legacy=0x7f72180b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:02.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.038+0000 7f721dfa6700 1 -- 172.21.15.17:0/2343098777 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72180af350 con 0x7f72180614b0 2022-08-01T19:38:02.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.038+0000 7f721e7a7700 1 --2- 172.21.15.17:0/2343098777 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f72181405d0 0x7f7218140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:38:02.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.038+0000 7f721dfa6700 1 --2- 172.21.15.17:0/2343098777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72180614b0 0x7f7218061890 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f72180aed30 tx=0x7f7208006df0).ready entity=mon.1 client_cookie=e7ed3e17715db954 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:38:02.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.038+0000 7f721cfa4700 1 -- 172.21.15.17:0/2343098777 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f720800d570 con 0x7f72180614b0 2022-08-01T19:38:02.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.038+0000 7f721cfa4700 1 -- 172.21.15.17:0/2343098777 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7208000e90 con 0x7f72180614b0 2022-08-01T19:38:02.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.038+0000 7f721cfa4700 1 -- 172.21.15.17:0/2343098777 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f720800d8c0 con 0x7f72180614b0 2022-08-01T19:38:02.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.038+0000 7f721efa8700 1 -- 172.21.15.17:0/2343098777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72180614b0 msgr2=0x7f7218061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:02.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.038+0000 7f721efa8700 1 --2- 172.21.15.17:0/2343098777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72180614b0 0x7f7218061890 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f72180aed30 tx=0x7f7208006df0).stop 2022-08-01T19:38:02.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.038+0000 7f721efa8700 1 -- 172.21.15.17:0/2343098777 shutdown_connections 2022-08-01T19:38:02.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.038+0000 7f721efa8700 1 --2- 172.21.15.17:0/2343098777 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f72181405d0 0x7f7218140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:02.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.038+0000 7f721efa8700 1 --2- 172.21.15.17:0/2343098777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72180614b0 0x7f7218061890 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:02.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.038+0000 7f721efa8700 1 -- 172.21.15.17:0/2343098777 >> 172.21.15.17:0/2343098777 conn(0x7f7218066f70 msgr2=0x7f7218067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:38:02.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.038+0000 7f721efa8700 1 -- 172.21.15.17:0/2343098777 shutdown_connections 2022-08-01T19:38:02.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.038+0000 7f721efa8700 1 -- 172.21.15.17:0/2343098777 wait complete. 2022-08-01T19:38:02.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.038+0000 7f721efa8700 1 Processor -- start 2022-08-01T19:38:02.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.038+0000 7f721efa8700 1 -- start start 2022-08-01T19:38:02.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.042+0000 7f721efa8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72180614b0 0x7f72180b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:02.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.042+0000 7f721efa8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72180ad750 0x7f72181526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:02.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.042+0000 7f721efa8700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f72181405d0 0x7f7218152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:02.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.042+0000 7f721efa8700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7218156ae0 con 0x7f72180614b0 2022-08-01T19:38:02.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.042+0000 7f721efa8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721809ec00 con 0x7f72180ad750 2022-08-01T19:38:02.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.042+0000 7f721efa8700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f721809eea0 con 0x7f72181405d0 2022-08-01T19:38:02.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.042+0000 7f721dfa6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72180614b0 0x7f72180b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:02.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.042+0000 7f721dfa6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72180614b0 0x7f72180b4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39426/0 (socket says 172.21.15.17:39426) 2022-08-01T19:38:02.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.042+0000 7f721dfa6700 1 -- 172.21.15.17:0/135414627 learned_addr learned my addr 172.21.15.17:0/135414627 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:38:02.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.042+0000 7f721d7a5700 1 --2- 172.21.15.17:0/135414627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72180ad750 0x7f72181526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:02.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.042+0000 7f721e7a7700 1 --2- 172.21.15.17:0/135414627 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f72181405d0 0x7f7218152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:02.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.042+0000 7f721dfa6700 1 -- 172.21.15.17:0/135414627 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f72181405d0 msgr2=0x7f7218152bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:02.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.042+0000 7f721dfa6700 1 --2- 172.21.15.17:0/135414627 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f72181405d0 0x7f7218152bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:02.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.042+0000 7f721dfa6700 1 -- 172.21.15.17:0/135414627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72180ad750 msgr2=0x7f72181526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:02.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.042+0000 7f721dfa6700 1 --2- 172.21.15.17:0/135414627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72180ad750 0x7f72181526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:02.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.042+0000 7f721dfa6700 1 -- 172.21.15.17:0/135414627 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7210009530 con 0x7f72180614b0 2022-08-01T19:38:02.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.042+0000 7f721d7a5700 1 --2- 172.21.15.17:0/135414627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72180ad750 0x7f72181526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:38:02.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.042+0000 7f721e7a7700 1 --2- 172.21.15.17:0/135414627 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f72181405d0 0x7f7218152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:38:02.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.042+0000 7f721dfa6700 1 --2- 172.21.15.17:0/135414627 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72180614b0 0x7f72180b4f10 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f72180aed30 tx=0x7f7208014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:38:02.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.042+0000 7f7206ffd700 1 -- 172.21.15.17:0/135414627 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7208013de0 con 0x7f72180614b0 2022-08-01T19:38:02.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.042+0000 7f7206ffd700 1 -- 172.21.15.17:0/135414627 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7208002f70 con 0x7f72180614b0 2022-08-01T19:38:02.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.042+0000 7f721efa8700 1 -- 172.21.15.17:0/135414627 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f720800b180 con 0x7f72180614b0 2022-08-01T19:38:02.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.042+0000 7f7206ffd700 1 -- 172.21.15.17:0/135414627 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f720801e5f0 con 0x7f72180614b0 2022-08-01T19:38:02.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.042+0000 7f721efa8700 1 -- 172.21.15.17:0/135414627 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f721809f5e0 con 0x7f72180614b0 2022-08-01T19:38:02.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.046+0000 7f721efa8700 1 -- 172.21.15.17:0/135414627 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7218156850 con 0x7f72180614b0 2022-08-01T19:38:02.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.046+0000 7f7206ffd700 1 -- 172.21.15.17:0/135414627 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f720801c070 con 0x7f72180614b0 2022-08-01T19:38:02.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.046+0000 7f7206ffd700 1 --2- 172.21.15.17:0/135414627 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f71f8053a10 0x7f71f8055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:02.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.046+0000 7f7206ffd700 1 -- 172.21.15.17:0/135414627 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f720806ca30 con 0x7f72180614b0 2022-08-01T19:38:02.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.046+0000 7f721d7a5700 1 --2- 172.21.15.17:0/135414627 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f71f8053a10 0x7f71f8055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:02.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.046+0000 7f721d7a5700 1 --2- 172.21.15.17:0/135414627 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f71f8053a10 0x7f71f8055ed0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f721400d000 tx=0x7f7214007b80).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:38:02.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.050+0000 7f7206ffd700 1 -- 172.21.15.17:0/135414627 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7208043750 con 0x7f72180614b0 2022-08-01T19:38:02.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.178+0000 7f721efa8700 1 -- 172.21.15.17:0/135414627 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f72180af5f0 con 0x7f72180614b0 2022-08-01T19:38:02.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.178+0000 7f7206ffd700 1 -- 172.21.15.17:0/135414627 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+10 (secure 0 0 0) 0x7f7208023070 con 0x7f72180614b0 2022-08-01T19:38:02.182 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:38:02.183 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.178+0000 7f721efa8700 1 -- 172.21.15.17:0/135414627 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f71f8053a10 msgr2=0x7f71f8055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:02.184 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.178+0000 7f721efa8700 1 --2- 172.21.15.17:0/135414627 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f71f8053a10 0x7f71f8055ed0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f721400d000 tx=0x7f7214007b80).stop 2022-08-01T19:38:02.184 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.178+0000 7f721efa8700 1 -- 172.21.15.17:0/135414627 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72180614b0 msgr2=0x7f72180b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:02.184 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.178+0000 7f721efa8700 1 --2- 172.21.15.17:0/135414627 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72180614b0 0x7f72180b4f10 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f72180aed30 tx=0x7f7208014000).stop 2022-08-01T19:38:02.184 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.178+0000 7f721efa8700 1 -- 172.21.15.17:0/135414627 shutdown_connections 2022-08-01T19:38:02.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.178+0000 7f721efa8700 1 --2- 172.21.15.17:0/135414627 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f71f8053a10 0x7f71f8055ed0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:02.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.178+0000 7f721efa8700 1 --2- 172.21.15.17:0/135414627 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f72181405d0 0x7f7218152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:02.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.178+0000 7f721efa8700 1 --2- 172.21.15.17:0/135414627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72180ad750 0x7f72181526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:02.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.178+0000 7f721efa8700 1 --2- 172.21.15.17:0/135414627 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72180614b0 0x7f72180b4f10 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:02.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.178+0000 7f721efa8700 1 -- 172.21.15.17:0/135414627 >> 172.21.15.17:0/135414627 conn(0x7f7218066f70 msgr2=0x7f7218144f10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:38:02.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.178+0000 7f721efa8700 1 -- 172.21.15.17:0/135414627 shutdown_connections 2022-08-01T19:38:02.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.178+0000 7f721efa8700 1 -- 172.21.15.17:0/135414627 wait complete. 2022-08-01T19:38:02.195 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T19:38:02.195 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:38:02.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7facf60f9700 1 Processor -- start 2022-08-01T19:38:02.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7facf60f9700 1 -- start start 2022-08-01T19:38:02.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7facf60f9700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7facf00b2710 0x7facf014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:02.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7facf60f9700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facf00a7250 con 0x7facf00b01a0 2022-08-01T19:38:02.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7facf60f9700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facf00a7480 con 0x7facf00a8c50 2022-08-01T19:38:02.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7facf60f9700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facf00a7720 con 0x7facf00b2710 2022-08-01T19:38:02.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7facf48f6700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7facf00b01a0 0x7facf00a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59860/0 (socket says 172.21.15.17:59860) 2022-08-01T19:38:02.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7facf48f6700 1 -- 172.21.15.17:0/2457551046 learned_addr learned my addr 172.21.15.17:0/2457551046 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:38:02.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7facf50f7700 1 --2- 172.21.15.17:0/2457551046 >> v2:172.21.15.77:3300/0 conn(0x7facf00b2710 0x7facf014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:02.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7face3fff700 1 -- 172.21.15.17:0/2457551046 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 748251747 0 0) 0x7facf00a7250 con 0x7facf00b01a0 2022-08-01T19:38:02.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7face3fff700 1 -- 172.21.15.17:0/2457551046 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facd4003260 con 0x7facf00b01a0 2022-08-01T19:38:02.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7face3fff700 1 -- 172.21.15.17:0/2457551046 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2047896368 0 0) 0x7facd4003260 con 0x7facf00b01a0 2022-08-01T19:38:02.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7face3fff700 1 -- 172.21.15.17:0/2457551046 >> v2:172.21.15.77:3300/0 conn(0x7facf00b2710 msgr2=0x7facf014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:38:02.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7face3fff700 1 --2- 172.21.15.17:0/2457551046 >> v2:172.21.15.77:3300/0 conn(0x7facf00b2710 0x7facf014afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:02.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7face3fff700 1 -- 172.21.15.17:0/2457551046 >> v1:172.21.15.77:6789/0 conn(0x7facf00a8c50 legacy=0x7facf0151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:02.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7face3fff700 1 -- 172.21.15.17:0/2457551046 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf00a79c0 con 0x7facf00b01a0 2022-08-01T19:38:02.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7face3fff700 1 -- 172.21.15.17:0/2457551046 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7facec0019e0 con 0x7facf00b01a0 2022-08-01T19:38:02.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7face3fff700 1 -- 172.21.15.17:0/2457551046 >> v1:172.21.15.17:6789/0 conn(0x7facf00b01a0 legacy=0x7facf00a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:02.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7face3fff700 1 --2- 172.21.15.17:0/2457551046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facd4003750 0x7facd4003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:02.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7face3fff700 1 --2- 172.21.15.17:0/2457551046 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7facd4004500 0x7facd4004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:02.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7face3fff700 1 --2- 172.21.15.17:0/2457551046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facd4004e70 0x7facd400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:02.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7face3fff700 1 -- 172.21.15.17:0/2457551046 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf00a7720 con 0x7facd4003750 2022-08-01T19:38:02.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7face3fff700 1 -- 172.21.15.17:0/2457551046 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd400bc70 con 0x7facd4004e70 2022-08-01T19:38:02.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7face3fff700 1 -- 172.21.15.17:0/2457551046 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facd400be30 con 0x7facd4004500 2022-08-01T19:38:02.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7facf58f8700 1 --2- 172.21.15.17:0/2457551046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facd4004e70 0x7facd400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:02.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7facf48f6700 1 --2- 172.21.15.17:0/2457551046 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7facd4004500 0x7facd4004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:02.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7facf50f7700 1 --2- 172.21.15.17:0/2457551046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facd4003750 0x7facd4003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:02.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7facf58f8700 1 -- 172.21.15.17:0/2457551046 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7facd4004500 msgr2=0x7facd4004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:38:02.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7facf58f8700 1 --2- 172.21.15.17:0/2457551046 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7facd4004500 0x7facd4004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:02.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7facf58f8700 1 -- 172.21.15.17:0/2457551046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facd4003750 msgr2=0x7facd4003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:38:02.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7facf58f8700 1 --2- 172.21.15.17:0/2457551046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facd4003750 0x7facd4003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:02.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7facf58f8700 1 -- 172.21.15.17:0/2457551046 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd4003260 con 0x7facd4004e70 2022-08-01T19:38:02.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7facf50f7700 1 --2- 172.21.15.17:0/2457551046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facd4003750 0x7facd4003b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:38:02.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7facf58f8700 1 -- 172.21.15.17:0/2457551046 reap_dead start 2022-08-01T19:38:02.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7facf58f8700 1 --2- 172.21.15.17:0/2457551046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facd4004e70 0x7facd400b750 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7face8001110 tx=0x7face8007a00).ready entity=mon.1 client_cookie=5bd943baba7933d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:38:02.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7face3fff700 1 -- 172.21.15.17:0/2457551046 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7face8007d10 con 0x7facd4004e70 2022-08-01T19:38:02.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7face3fff700 1 -- 172.21.15.17:0/2457551046 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face8007030 con 0x7facd4004e70 2022-08-01T19:38:02.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7face3fff700 1 -- 172.21.15.17:0/2457551046 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7face80073b0 con 0x7facd4004e70 2022-08-01T19:38:02.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7facf60f9700 1 -- 172.21.15.17:0/2457551046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facd4004e70 msgr2=0x7facd400b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:02.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7facf60f9700 1 --2- 172.21.15.17:0/2457551046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facd4004e70 0x7facd400b750 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7face8001110 tx=0x7face8007a00).stop 2022-08-01T19:38:02.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7facf60f9700 1 -- 172.21.15.17:0/2457551046 shutdown_connections 2022-08-01T19:38:02.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7facf60f9700 1 --2- 172.21.15.17:0/2457551046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facd4004e70 0x7facd400b750 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:02.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7facf60f9700 1 -- 172.21.15.17:0/2457551046 >> 172.21.15.17:0/2457551046 conn(0x7facf0060310 msgr2=0x7facf00b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:38:02.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7facf60f9700 1 -- 172.21.15.17:0/2457551046 shutdown_connections 2022-08-01T19:38:02.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.294+0000 7facf60f9700 1 -- 172.21.15.17:0/2457551046 wait complete. 2022-08-01T19:38:02.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.298+0000 7facf60f9700 1 Processor -- start 2022-08-01T19:38:02.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.298+0000 7facf60f9700 1 -- start start 2022-08-01T19:38:02.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.298+0000 7facf60f9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facd4004e70 0x7facf0159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:02.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.298+0000 7facf60f9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facf00ab120 0x7facf0063190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:02.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.298+0000 7facf60f9700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7facf00b4e50 0x7facf00ac140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:02.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.298+0000 7facf60f9700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf0065500 con 0x7facf00ab120 2022-08-01T19:38:02.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.298+0000 7facf60f9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf00aa710 con 0x7facd4004e70 2022-08-01T19:38:02.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.298+0000 7facf60f9700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf00aa9b0 con 0x7facf00b4e50 2022-08-01T19:38:02.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.298+0000 7facf48f6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facf00ab120 0x7facf0063190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:02.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.298+0000 7facf48f6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facf00ab120 0x7facf0063190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39448/0 (socket says 172.21.15.17:39448) 2022-08-01T19:38:02.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.298+0000 7facf48f6700 1 -- 172.21.15.17:0/3413406136 learned_addr learned my addr 172.21.15.17:0/3413406136 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:38:02.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.298+0000 7facf50f7700 1 --2- 172.21.15.17:0/3413406136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facd4004e70 0x7facf0159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:02.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.298+0000 7facf58f8700 1 --2- 172.21.15.17:0/3413406136 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7facf00b4e50 0x7facf00ac140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:02.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.298+0000 7facf48f6700 1 -- 172.21.15.17:0/3413406136 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7facf00b4e50 msgr2=0x7facf00ac140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:02.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.298+0000 7facf48f6700 1 --2- 172.21.15.17:0/3413406136 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7facf00b4e50 0x7facf00ac140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:02.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.298+0000 7facf48f6700 1 -- 172.21.15.17:0/3413406136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facd4004e70 msgr2=0x7facf0159ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:02.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.298+0000 7facf48f6700 1 --2- 172.21.15.17:0/3413406136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facd4004e70 0x7facf0159ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:02.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.298+0000 7facf48f6700 1 -- 172.21.15.17:0/3413406136 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf00aac50 con 0x7facf00ab120 2022-08-01T19:38:02.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.298+0000 7facf48f6700 1 --2- 172.21.15.17:0/3413406136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facf00ab120 0x7facf0063190 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7facec00d000 tx=0x7facf00a7b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:38:02.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.298+0000 7face27fc700 1 -- 172.21.15.17:0/3413406136 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7facec011850 con 0x7facf00ab120 2022-08-01T19:38:02.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.298+0000 7face27fc700 1 -- 172.21.15.17:0/3413406136 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facec0119b0 con 0x7facf00ab120 2022-08-01T19:38:02.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.298+0000 7facf60f9700 1 -- 172.21.15.17:0/3413406136 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facf00ac630 con 0x7facf00ab120 2022-08-01T19:38:02.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.298+0000 7face27fc700 1 -- 172.21.15.17:0/3413406136 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7facec011d10 con 0x7facf00ab120 2022-08-01T19:38:02.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.298+0000 7facf60f9700 1 -- 172.21.15.17:0/3413406136 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf0066890 con 0x7facf00ab120 2022-08-01T19:38:02.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.302+0000 7facf60f9700 1 -- 172.21.15.17:0/3413406136 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facf00b5230 con 0x7facf00ab120 2022-08-01T19:38:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.302+0000 7face27fc700 1 -- 172.21.15.17:0/3413406136 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7facec01b030 con 0x7facf00ab120 2022-08-01T19:38:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.302+0000 7face27fc700 1 --2- 172.21.15.17:0/3413406136 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7facd8053b30 0x7facd8055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.302+0000 7face27fc700 1 -- 172.21.15.17:0/3413406136 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6420+0+0 (secure 0 0 0) 0x7facec06bb70 con 0x7facf00ab120 2022-08-01T19:38:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.302+0000 7facf50f7700 1 --2- 172.21.15.17:0/3413406136 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7facd8053b30 0x7facd8055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.302+0000 7facf50f7700 1 --2- 172.21.15.17:0/3413406136 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7facd8053b30 0x7facd8055ff0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7face400b000 tx=0x7face4006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:38:02.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.310+0000 7face27fc700 1 -- 172.21.15.17:0/3413406136 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7facec042930 con 0x7facf00ab120 2022-08-01T19:38:02.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.470+0000 7facf60f9700 1 -- 172.21.15.17:0/3413406136 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7facf00574e0 con 0x7facf00ab120 2022-08-01T19:38:02.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.470+0000 7face27fc700 1 -- 172.21.15.17:0/3413406136 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+2492 (secure 0 0 0) 0x7facec019030 con 0x7facf00ab120 2022-08-01T19:38:02.475 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:38:02.475 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":24,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/1914125901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":1914125901},{"type":"v1","addr":"172.21.15.17:6833","nonce":1914125901}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":22},{"gid":5587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1403688289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1403688289},{"type":"v1","addr":"172.21.15.77:6835","nonce":1403688289}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":22}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:37:36.197818+0000","modified":"2022-08-01T19:37:37.201700+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5589":{"gid":5589,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/3325991310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":3325991310},{"type":"v1","addr":"172.21.15.17:6835","nonce":3325991310}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-08-01T19:38:02.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.474+0000 7facf60f9700 1 -- 172.21.15.17:0/3413406136 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7facd8053b30 msgr2=0x7facd8055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:02.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.474+0000 7facf60f9700 1 --2- 172.21.15.17:0/3413406136 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7facd8053b30 0x7facd8055ff0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7face400b000 tx=0x7face4006df0).stop 2022-08-01T19:38:02.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.474+0000 7facf60f9700 1 -- 172.21.15.17:0/3413406136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facf00ab120 msgr2=0x7facf0063190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:02.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.474+0000 7facf60f9700 1 --2- 172.21.15.17:0/3413406136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facf00ab120 0x7facf0063190 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7facec00d000 tx=0x7facf00a7b50).stop 2022-08-01T19:38:02.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.474+0000 7facf60f9700 1 -- 172.21.15.17:0/3413406136 shutdown_connections 2022-08-01T19:38:02.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.474+0000 7facf60f9700 1 --2- 172.21.15.17:0/3413406136 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7facd8053b30 0x7facd8055ff0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:02.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.474+0000 7facf60f9700 1 --2- 172.21.15.17:0/3413406136 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7facf00b4e50 0x7facf00ac140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:02.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.474+0000 7facf60f9700 1 --2- 172.21.15.17:0/3413406136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facf00ab120 0x7facf0063190 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:02.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.474+0000 7facf60f9700 1 --2- 172.21.15.17:0/3413406136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facd4004e70 0x7facf0159ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:02.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.474+0000 7facf60f9700 1 -- 172.21.15.17:0/3413406136 >> 172.21.15.17:0/3413406136 conn(0x7facf0060310 msgr2=0x7facf014b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:38:02.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.474+0000 7facf60f9700 1 -- 172.21.15.17:0/3413406136 shutdown_connections 2022-08-01T19:38:02.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:02.474+0000 7facf60f9700 1 -- 172.21.15.17:0/3413406136 wait complete. 2022-08-01T19:38:02.482 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 24 2022-08-01T19:38:02.797 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:37:36.197818+0000', 'modified': '2022-08-01T19:37:37.201700+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5589}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5589': {'gid': 5589, 'name': 'c', 'rank': 0, 'incarnation': 23, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/3325991310', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 3325991310}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 3325991310}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:38:02.798 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:38:02.798 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T19:38:02.915 INFO:teuthology.orchestra.run.smithi017.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":5589,"want_state":"up:active","state":"up:active","rank_uptime":26.707593779,"mdsmap_epoch":24,"osdmap_epoch":55,"osdmap_epoch_barrier":55,"uptime":31.068632830999999} 2022-08-01T19:38:02.915 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 5589, "mdsmap_epoch": 24, "osdmap_epoch": 55, "osdmap_epoch_barrier": 55, "rank_uptime": 26.707593779, "state": "up:active", "uptime": 31.068632831, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:38:02.916 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T19:38:02.917 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-08-01T19:38:02.917 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi085.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-08-01T19:38:02.917 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:38:02.918 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:38:02.927 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:38:02.942 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:38:02.942 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:38:02.954 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:38:02.955 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:38:02.956 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:38:02.957 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:38:03.006 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T19:38:03.007 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:38:03.011 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:38:03.030 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:38:03.031 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:38:03.085 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:38:03.086 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:38:03.087 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:38:03.087 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:38:03.088 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:2022-08-01T19:38:03.087+0000 7f165938c080 -1 init, newargv = 0x55ce89ebd870 newargc=9 2022-08-01T19:38:03.089 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:ceph-fuse[22839]: starting ceph client 2022-08-01T19:38:03.097 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:ceph-fuse[22839]: starting fuse 2022-08-01T19:38:04.196 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:38:04.215 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:38:04.216 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:38:04.274 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:38:04.275 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:38:04.276 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:38:04.277 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:38:04.285 INFO:teuthology.orchestra.run.smithi085.stdout:52 2022-08-01T19:38:04.285 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T19:38:04.286 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:38:04.286 DEBUG:teuthology.orchestra.run.smithi085:> sudo python3 -c ' 2022-08-01T19:38:04.287 DEBUG:teuthology.orchestra.run.smithi085:> import glob 2022-08-01T19:38:04.288 DEBUG:teuthology.orchestra.run.smithi085:> import re 2022-08-01T19:38:04.288 DEBUG:teuthology.orchestra.run.smithi085:> import os 2022-08-01T19:38:04.289 DEBUG:teuthology.orchestra.run.smithi085:> import subprocess 2022-08-01T19:38:04.289 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:38:04.290 DEBUG:teuthology.orchestra.run.smithi085:> def find_socket(client_name): 2022-08-01T19:38:04.290 DEBUG:teuthology.orchestra.run.smithi085:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-08-01T19:38:04.291 DEBUG:teuthology.orchestra.run.smithi085:> files = glob.glob(asok_path) 2022-08-01T19:38:04.292 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:38:04.293 DEBUG:teuthology.orchestra.run.smithi085:> # Given a non-glob path, it better be there 2022-08-01T19:38:04.294 DEBUG:teuthology.orchestra.run.smithi085:> if "*" not in asok_path: 2022-08-01T19:38:04.294 DEBUG:teuthology.orchestra.run.smithi085:> assert(len(files) == 1) 2022-08-01T19:38:04.295 DEBUG:teuthology.orchestra.run.smithi085:> return files[0] 2022-08-01T19:38:04.296 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:38:04.297 DEBUG:teuthology.orchestra.run.smithi085:> for f in files: 2022-08-01T19:38:04.297 DEBUG:teuthology.orchestra.run.smithi085:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T19:38:04.298 DEBUG:teuthology.orchestra.run.smithi085:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T19:38:04.298 DEBUG:teuthology.orchestra.run.smithi085:> return f 2022-08-01T19:38:04.299 DEBUG:teuthology.orchestra.run.smithi085:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T19:38:04.299 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:38:04.299 DEBUG:teuthology.orchestra.run.smithi085:> print(find_socket("client.0")) 2022-08-01T19:38:04.300 DEBUG:teuthology.orchestra.run.smithi085:> ' 2022-08-01T19:38:04.345 INFO:teuthology.orchestra.run.smithi085.stdout:/var/run/ceph/ceph-client.0.22839.asok 2022-08-01T19:38:04.350 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22839.asok 2022-08-01T19:38:04.351 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:38:04.351 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22839.asok status 2022-08-01T19:38:04.487 INFO:teuthology.orchestra.run.smithi085.stdout:{ 2022-08-01T19:38:04.487 INFO:teuthology.orchestra.run.smithi085.stdout: "metadata": { 2022-08-01T19:38:04.487 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T19:38:04.488 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T19:38:04.488 INFO:teuthology.orchestra.run.smithi085.stdout: "entity_id": "0", 2022-08-01T19:38:04.488 INFO:teuthology.orchestra.run.smithi085.stdout: "hostname": "smithi085", 2022-08-01T19:38:04.488 INFO:teuthology.orchestra.run.smithi085.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-08-01T19:38:04.489 INFO:teuthology.orchestra.run.smithi085.stdout: "pid": "22839", 2022-08-01T19:38:04.489 INFO:teuthology.orchestra.run.smithi085.stdout: "root": "/" 2022-08-01T19:38:04.489 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:38:04.489 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_count": 0, 2022-08-01T19:38:04.490 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_pinned_count": 0, 2022-08-01T19:38:04.490 INFO:teuthology.orchestra.run.smithi085.stdout: "id": 5821, 2022-08-01T19:38:04.490 INFO:teuthology.orchestra.run.smithi085.stdout: "inst": { 2022-08-01T19:38:04.490 INFO:teuthology.orchestra.run.smithi085.stdout: "name": { 2022-08-01T19:38:04.491 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "client", 2022-08-01T19:38:04.491 INFO:teuthology.orchestra.run.smithi085.stdout: "num": 5821 2022-08-01T19:38:04.491 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:38:04.491 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2022-08-01T19:38:04.491 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2022-08-01T19:38:04.492 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2022-08-01T19:38:04.492 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 2273700995 2022-08-01T19:38:04.492 INFO:teuthology.orchestra.run.smithi085.stdout: } 2022-08-01T19:38:04.492 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:38:04.493 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2022-08-01T19:38:04.493 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2022-08-01T19:38:04.493 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2022-08-01T19:38:04.493 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 2273700995 2022-08-01T19:38:04.493 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:38:04.494 INFO:teuthology.orchestra.run.smithi085.stdout: "inst_str": "client.5821 172.21.15.85:0/2273700995", 2022-08-01T19:38:04.494 INFO:teuthology.orchestra.run.smithi085.stdout: "addr_str": "172.21.15.85:0/2273700995", 2022-08-01T19:38:04.494 INFO:teuthology.orchestra.run.smithi085.stdout: "inode_count": 1, 2022-08-01T19:38:04.494 INFO:teuthology.orchestra.run.smithi085.stdout: "mds_epoch": 24, 2022-08-01T19:38:04.495 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch": 55, 2022-08-01T19:38:04.495 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch_barrier": 0, 2022-08-01T19:38:04.495 INFO:teuthology.orchestra.run.smithi085.stdout: "blacklisted": false, 2022-08-01T19:38:04.495 INFO:teuthology.orchestra.run.smithi085.stdout: "fs_name": "cephfs" 2022-08-01T19:38:04.496 INFO:teuthology.orchestra.run.smithi085.stdout:} 2022-08-01T19:38:04.497 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:38:04.497 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:38:04.498 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:38:04.506 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:38:04.507 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:38:04.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:38:04.508 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:38:17.479 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:38:17.482 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:38:17.484 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:38:17.487 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:38:47.563 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:38:47.566 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:38:47.569 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:38:47.572 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:38:56.848 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T19:38:56.849 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:38:56.869 DEBUG:teuthology.orchestra.run.smithi077:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:38:56.891 DEBUG:teuthology.orchestra.run.smithi085:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:38:56.916 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:38:56.939 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T19:38:56.940 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:38:56.941 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:38:56.942 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:38:56.968 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:38:56.970 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:38:56.970 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi085.front.sepia.ceph.com... 2022-08-01T19:38:56.971 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T19:38:56.972 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:38:56.991 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:ceph-fuse[22839]: fuse finished with error 0 and tester_r 0 2022-08-01T19:38:56.993 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:38:56.994 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:38:56.995 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:38:57.004 INFO:teuthology.orchestra.run.smithi085.stdout:ext2/ext3 2022-08-01T19:38:57.004 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T19:38:57.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T19:38:57.123 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:38:57.130 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:38:57.130 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:38:57.131 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:38:57.138 INFO:teuthology.orchestra.run.smithi190.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T19:38:57.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:38:57.142 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-08-01T19:38:57.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T19:38:57.144 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:38:57.151 INFO:teuthology.orchestra.run.smithi190.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T19:38:57.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:38:57.153 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T19:38:57.266 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.266+0000 7fbc5359e700 1 Processor -- start 2022-08-01T19:38:57.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.266+0000 7fbc5359e700 1 -- start start 2022-08-01T19:38:57.272 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.266+0000 7fbc5359e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fbc540b2710 0x7fbc5414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:57.272 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.266+0000 7fbc5359e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbc540b01a0 0x7fbc5414b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:57.272 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.266+0000 7fbc5359e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc540a7250 con 0x7fbc540afd60 2022-08-01T19:38:57.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.266+0000 7fbc5359e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc540a7480 con 0x7fbc540b2710 2022-08-01T19:38:57.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.266+0000 7fbc5359e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc540a7720 con 0x7fbc540b01a0 2022-08-01T19:38:57.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.266+0000 7fbc51d9b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbc540b01a0 0x7fbc5414b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:57.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.266+0000 7fbc5259c700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fbc540b2710 0x7fbc5414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:57.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.266+0000 7fbc52d9d700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7fbc540afd60 0x7fbc540a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:48452/0 (socket says 172.21.15.17:48452) 2022-08-01T19:38:57.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.266+0000 7fbc52d9d700 1 -- 172.21.15.17:0/775269827 learned_addr learned my addr 172.21.15.17:0/775269827 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:38:57.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.266+0000 7fbc51d9b700 1 --2- 172.21.15.17:0/775269827 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbc540b01a0 0x7fbc5414b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:35554/0 (socket says 172.21.15.17:35554) 2022-08-01T19:38:57.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.266+0000 7fbc5159a700 1 -- 172.21.15.17:0/775269827 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1066744872 0 0) 0x7fbc540a7250 con 0x7fbc540afd60 2022-08-01T19:38:57.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.266+0000 7fbc5159a700 1 -- 172.21.15.17:0/775269827 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc38003260 con 0x7fbc540afd60 2022-08-01T19:38:57.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.266+0000 7fbc51d9b700 1 -- 172.21.15.17:0/775269827 >> v2:172.21.15.77:3300/0 conn(0x7fbc540b2710 msgr2=0x7fbc5414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:38:57.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.266+0000 7fbc51d9b700 1 --2- 172.21.15.17:0/775269827 >> v2:172.21.15.77:3300/0 conn(0x7fbc540b2710 0x7fbc5414afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:57.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.266+0000 7fbc51d9b700 1 -- 172.21.15.17:0/775269827 >> v1:172.21.15.77:6789/0 conn(0x7fbc540afd60 legacy=0x7fbc540a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:57.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.266+0000 7fbc51d9b700 1 -- 172.21.15.17:0/775269827 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc540a79c0 con 0x7fbc540b01a0 2022-08-01T19:38:57.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.266+0000 7fbc51d9b700 1 --2- 172.21.15.17:0/775269827 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbc540b01a0 0x7fbc5414b4c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fbc440032f0 tx=0x7fbc440091d0).ready entity=mon.2 client_cookie=6a13d2e24ea65061 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:38:57.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.266+0000 7fbc5159a700 1 -- 172.21.15.17:0/775269827 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc4400b600 con 0x7fbc540b01a0 2022-08-01T19:38:57.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.266+0000 7fbc5159a700 1 -- 172.21.15.17:0/775269827 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4400b760 con 0x7fbc540b01a0 2022-08-01T19:38:57.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.266+0000 7fbc5159a700 1 -- 172.21.15.17:0/775269827 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc4400b8d0 con 0x7fbc540b01a0 2022-08-01T19:38:57.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.266+0000 7fbc5359e700 1 -- 172.21.15.17:0/775269827 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbc540b01a0 msgr2=0x7fbc5414b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:57.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.266+0000 7fbc5359e700 1 --2- 172.21.15.17:0/775269827 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbc540b01a0 0x7fbc5414b4c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fbc440032f0 tx=0x7fbc440091d0).stop 2022-08-01T19:38:57.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.266+0000 7fbc5359e700 1 -- 172.21.15.17:0/775269827 shutdown_connections 2022-08-01T19:38:57.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.266+0000 7fbc5359e700 1 --2- 172.21.15.17:0/775269827 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbc540b01a0 0x7fbc5414b4c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:57.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.266+0000 7fbc5359e700 1 --2- 172.21.15.17:0/775269827 >> v2:172.21.15.77:3300/0 conn(0x7fbc540b2710 0x7fbc5414afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:57.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.266+0000 7fbc5359e700 1 -- 172.21.15.17:0/775269827 >> 172.21.15.17:0/775269827 conn(0x7fbc54060310 msgr2=0x7fbc540b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:38:57.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.266+0000 7fbc5359e700 1 -- 172.21.15.17:0/775269827 shutdown_connections 2022-08-01T19:38:57.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.266+0000 7fbc5359e700 1 -- 172.21.15.17:0/775269827 wait complete. 2022-08-01T19:38:57.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.266+0000 7fbc5359e700 1 Processor -- start 2022-08-01T19:38:57.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.266+0000 7fbc5359e700 1 -- start start 2022-08-01T19:38:57.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.270+0000 7fbc5359e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc540afd60 0x7fbc5414cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:57.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.270+0000 7fbc5359e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbc540b01a0 0x7fbc54067520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:57.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.270+0000 7fbc5359e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc540b2710 0x7fbc540ab9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:57.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.270+0000 7fbc5359e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc540b52a0 con 0x7fbc540b2710 2022-08-01T19:38:57.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.270+0000 7fbc5359e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc540b3510 con 0x7fbc540afd60 2022-08-01T19:38:57.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.270+0000 7fbc5359e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc540b37b0 con 0x7fbc540b01a0 2022-08-01T19:38:57.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.270+0000 7fbc52d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc540b2710 0x7fbc540ab9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:57.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.270+0000 7fbc5259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc540afd60 0x7fbc5414cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:57.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.270+0000 7fbc51d9b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbc540b01a0 0x7fbc54067520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:57.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.270+0000 7fbc52d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc540b2710 0x7fbc540ab9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39464/0 (socket says 172.21.15.17:39464) 2022-08-01T19:38:57.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.270+0000 7fbc52d9d700 1 -- 172.21.15.17:0/1851924290 learned_addr learned my addr 172.21.15.17:0/1851924290 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:38:57.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.270+0000 7fbc52d9d700 1 -- 172.21.15.17:0/1851924290 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbc540b01a0 msgr2=0x7fbc54067520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:57.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.270+0000 7fbc52d9d700 1 --2- 172.21.15.17:0/1851924290 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbc540b01a0 0x7fbc54067520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:57.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.270+0000 7fbc52d9d700 1 -- 172.21.15.17:0/1851924290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc540afd60 msgr2=0x7fbc5414cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:57.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.270+0000 7fbc52d9d700 1 --2- 172.21.15.17:0/1851924290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc540afd60 0x7fbc5414cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:57.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.270+0000 7fbc52d9d700 1 -- 172.21.15.17:0/1851924290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc44009470 con 0x7fbc540b2710 2022-08-01T19:38:57.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.270+0000 7fbc52d9d700 1 --2- 172.21.15.17:0/1851924290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc540b2710 0x7fbc540ab9b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7fbc4c001110 tx=0x7fbc4c007a80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:38:57.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.270+0000 7fbc437fe700 1 -- 172.21.15.17:0/1851924290 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc4c00ce80 con 0x7fbc540b2710 2022-08-01T19:38:57.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.270+0000 7fbc437fe700 1 -- 172.21.15.17:0/1851924290 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4c0070d0 con 0x7fbc540b2710 2022-08-01T19:38:57.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.270+0000 7fbc5359e700 1 -- 172.21.15.17:0/1851924290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc540b3a50 con 0x7fbc540b2710 2022-08-01T19:38:57.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.270+0000 7fbc437fe700 1 -- 172.21.15.17:0/1851924290 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc4c007430 con 0x7fbc540b2710 2022-08-01T19:38:57.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.270+0000 7fbc5359e700 1 -- 172.21.15.17:0/1851924290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc5414beb0 con 0x7fbc540b2710 2022-08-01T19:38:57.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.274+0000 7fbc5359e700 1 -- 172.21.15.17:0/1851924290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc540574e0 con 0x7fbc540b2710 2022-08-01T19:38:57.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.274+0000 7fbc437fe700 1 -- 172.21.15.17:0/1851924290 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbc4c01a060 con 0x7fbc540b2710 2022-08-01T19:38:57.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.274+0000 7fbc437fe700 1 --2- 172.21.15.17:0/1851924290 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbc30053aa0 0x7fbc30055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:57.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.274+0000 7fbc437fe700 1 -- 172.21.15.17:0/1851924290 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fbc4c06b7e0 con 0x7fbc540b2710 2022-08-01T19:38:57.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.274+0000 7fbc5259c700 1 --2- 172.21.15.17:0/1851924290 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbc30053aa0 0x7fbc30055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:57.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.274+0000 7fbc5259c700 1 --2- 172.21.15.17:0/1851924290 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbc30053aa0 0x7fbc30055f60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7fbc4800b000 tx=0x7fbc48006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:38:57.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.278+0000 7fbc437fe700 1 -- 172.21.15.17:0/1851924290 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbc4c042490 con 0x7fbc540b2710 2022-08-01T19:38:57.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.406+0000 7fbc5359e700 1 -- 172.21.15.17:0/1851924290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fbc54069a70 con 0x7fbc540b2710 2022-08-01T19:38:57.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.410+0000 7fbc437fe700 1 -- 172.21.15.17:0/1851924290 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v55) v1 ==== 81+0+34039 (secure 0 0 0) 0x7fbc4c018070 con 0x7fbc540b2710 2022-08-01T19:38:57.412 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:38:57.412 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:38:57.413 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 55, 2022-08-01T19:38:57.413 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", 2022-08-01T19:38:57.413 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2022-08-01T19:33:50.105743+0000", 2022-08-01T19:38:57.413 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2022-08-01T19:37:37.200615+0000", 2022-08-01T19:38:57.413 INFO:teuthology.orchestra.run.smithi017.stdout: "last_up_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:38:57.414 INFO:teuthology.orchestra.run.smithi017.stdout: "last_in_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:38:57.414 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T19:38:57.414 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_num": 5799936, 2022-08-01T19:38:57.414 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_set": [ 2022-08-01T19:38:57.415 INFO:teuthology.orchestra.run.smithi017.stdout: "pglog_hardlimit", 2022-08-01T19:38:57.415 INFO:teuthology.orchestra.run.smithi017.stdout: "purged_snapdirs", 2022-08-01T19:38:57.415 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_deletes", 2022-08-01T19:38:57.415 INFO:teuthology.orchestra.run.smithi017.stdout: "sortbitwise" 2022-08-01T19:38:57.416 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:38:57.416 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_version": 6, 2022-08-01T19:38:57.416 INFO:teuthology.orchestra.run.smithi017.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T19:38:57.416 INFO:teuthology.orchestra.run.smithi017.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T19:38:57.416 INFO:teuthology.orchestra.run.smithi017.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T19:38:57.417 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_snapshot": "", 2022-08-01T19:38:57.417 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_max": 10, 2022-08-01T19:38:57.417 INFO:teuthology.orchestra.run.smithi017.stdout: "max_osd": 8, 2022-08-01T19:38:57.417 INFO:teuthology.orchestra.run.smithi017.stdout: "require_min_compat_client": "jewel", 2022-08-01T19:38:57.418 INFO:teuthology.orchestra.run.smithi017.stdout: "min_compat_client": "jewel", 2022-08-01T19:38:57.418 INFO:teuthology.orchestra.run.smithi017.stdout: "require_osd_release": "octopus", 2022-08-01T19:38:57.418 INFO:teuthology.orchestra.run.smithi017.stdout: "pools": [ 2022-08-01T19:38:57.418 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.419 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 1, 2022-08-01T19:38:57.419 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "device_health_metrics", 2022-08-01T19:38:57.419 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:33:53.001841+0000", 2022-08-01T19:38:57.419 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:38:57.420 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:38:57.421 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:38:57.421 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:38:57.421 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:38:57.422 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:38:57.422 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:38:57.422 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:38:57.422 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 1, 2022-08-01T19:38:57.423 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 1, 2022-08-01T19:38:57.423 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 1, 2022-08-01T19:38:57.423 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 1, 2022-08-01T19:38:57.423 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 1, 2022-08-01T19:38:57.424 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:38:57.424 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:38:57.424 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:38:57.424 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:38:57.425 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:38:57.425 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:38:57.425 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:38:57.425 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.426 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "16", 2022-08-01T19:38:57.426 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:38:57.426 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:38:57.426 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:38:57.427 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:38:57.427 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:38:57.427 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:38:57.427 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:38:57.428 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:38:57.429 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:38:57.429 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:38:57.429 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:38:57.429 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:38:57.430 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:38:57.430 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:38:57.430 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:38:57.430 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:38:57.431 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:38:57.431 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:38:57.431 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:38:57.431 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:38:57.432 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:38:57.432 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:38:57.432 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:38:57.432 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:38:57.433 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:38:57.433 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:38:57.433 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.433 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:38:57.434 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:38:57.434 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:38:57.434 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:38:57.434 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:38:57.435 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:38:57.435 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:38:57.435 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:38:57.435 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:38:57.435 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:38:57.436 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:38:57.436 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:38:57.436 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 1 2022-08-01T19:38:57.436 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.437 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:38:57.437 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_devicehealth": {} 2022-08-01T19:38:57.437 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.437 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.438 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.438 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 2, 2022-08-01T19:38:57.439 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "rbd", 2022-08-01T19:38:57.439 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:34:01.553098+0000", 2022-08-01T19:38:57.439 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:38:57.440 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:38:57.440 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:38:57.440 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:38:57.440 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:38:57.440 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:38:57.441 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:38:57.441 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:38:57.441 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:38:57.441 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:38:57.442 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:38:57.442 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:38:57.442 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:38:57.442 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:38:57.443 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:38:57.443 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:38:57.443 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:38:57.443 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:38:57.444 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:38:57.444 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:38:57.444 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.444 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "19", 2022-08-01T19:38:57.444 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:38:57.445 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:38:57.445 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:38:57.445 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:38:57.445 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:38:57.446 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:38:57.446 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:38:57.446 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:38:57.446 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:38:57.446 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:38:57.447 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:38:57.447 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:38:57.447 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:38:57.447 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:38:57.448 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:38:57.448 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:38:57.448 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:38:57.448 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:38:57.448 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:38:57.449 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:38:57.449 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:38:57.449 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:38:57.449 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:38:57.450 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:38:57.450 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:38:57.450 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:38:57.450 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.450 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:38:57.451 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:38:57.451 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:38:57.451 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:38:57.451 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:38:57.452 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:38:57.452 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:38:57.452 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:38:57.452 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:38:57.453 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:38:57.453 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:38:57.453 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:38:57.453 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:38:57.453 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd": {} 2022-08-01T19:38:57.454 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.454 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.454 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.454 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 9, 2022-08-01T19:38:57.455 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "cephfs_metadata", 2022-08-01T19:38:57.455 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:37:34.595906+0000", 2022-08-01T19:38:57.455 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:38:57.455 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:38:57.456 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:38:57.456 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:38:57.456 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:38:57.456 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:38:57.457 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:38:57.457 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:38:57.457 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:38:57.457 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:38:57.457 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:38:57.458 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:38:57.458 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:38:57.458 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:38:57.458 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:38:57.459 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:38:57.459 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:38:57.459 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:38:57.459 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:38:57.459 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:38:57.460 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.460 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "55", 2022-08-01T19:38:57.460 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:38:57.460 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:38:57.461 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:38:57.461 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:38:57.461 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:38:57.461 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:38:57.461 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:38:57.462 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:38:57.462 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:38:57.462 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:38:57.462 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:38:57.463 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:38:57.463 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:38:57.463 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:38:57.463 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:38:57.463 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:38:57.464 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:38:57.464 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:38:57.464 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:38:57.464 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:38:57.465 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:38:57.465 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:38:57.465 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:38:57.465 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:38:57.466 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:38:57.466 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:38:57.466 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.466 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:38:57.467 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:38:57.467 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:38:57.467 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:38:57.467 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:38:57.467 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:38:57.468 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:38:57.468 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:38:57.468 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:38:57.468 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:38:57.468 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:38:57.469 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:38:57.469 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_bias": 4, 2022-08-01T19:38:57.469 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 16, 2022-08-01T19:38:57.469 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_priority": 5 2022-08-01T19:38:57.470 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.470 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:38:57.470 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs": { 2022-08-01T19:38:57.470 INFO:teuthology.orchestra.run.smithi017.stdout: "metadata": "cephfs" 2022-08-01T19:38:57.471 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.471 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.471 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.471 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.472 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 10, 2022-08-01T19:38:57.472 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "cephfs_data", 2022-08-01T19:38:57.472 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:37:35.889700+0000", 2022-08-01T19:38:57.472 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:38:57.473 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:38:57.473 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:38:57.473 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:38:57.473 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:38:57.473 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:38:57.474 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:38:57.474 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:38:57.474 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:38:57.474 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:38:57.475 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:38:57.475 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:38:57.475 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:38:57.475 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:38:57.475 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:38:57.476 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:38:57.476 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:38:57.476 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:38:57.476 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:38:57.477 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:38:57.477 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.477 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "55", 2022-08-01T19:38:57.477 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:38:57.478 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:38:57.478 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:38:57.478 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:38:57.478 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:38:57.478 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:38:57.479 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:38:57.479 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:38:57.479 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:38:57.479 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:38:57.480 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:38:57.480 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:38:57.480 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:38:57.480 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:38:57.481 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:38:57.481 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:38:57.481 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:38:57.481 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:38:57.481 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:38:57.482 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:38:57.482 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:38:57.482 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:38:57.483 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:38:57.483 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:38:57.483 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:38:57.483 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:38:57.483 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.484 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:38:57.484 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:38:57.484 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:38:57.484 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:38:57.485 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:38:57.485 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:38:57.485 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:38:57.485 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:38:57.486 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:38:57.486 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:38:57.486 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:38:57.486 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:38:57.486 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:38:57.487 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs": { 2022-08-01T19:38:57.487 INFO:teuthology.orchestra.run.smithi017.stdout: "data": "cephfs" 2022-08-01T19:38:57.487 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.487 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.488 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.488 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:38:57.488 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2022-08-01T19:38:57.488 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.488 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:38:57.489 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a962f14f-cf16-4fa8-84ff-c85325db9132", 2022-08-01T19:38:57.489 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:38:57.489 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:38:57.489 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:38:57.490 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:38:57.490 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:38:57.490 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:38:57.490 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:38:57.490 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 53, 2022-08-01T19:38:57.491 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:38:57.491 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:38:57.491 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:38:57.491 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:38:57.492 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.492 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:38:57.492 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6800", 2022-08-01T19:38:57.492 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:38:57.493 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.493 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.493 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:38:57.493 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6801", 2022-08-01T19:38:57.493 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:38:57.494 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.494 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.494 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.494 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:38:57.495 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:38:57.495 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.495 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:38:57.495 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6802", 2022-08-01T19:38:57.495 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:38:57.496 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.496 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.496 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:38:57.496 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6803", 2022-08-01T19:38:57.497 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:38:57.497 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.497 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.497 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.497 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:38:57.498 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:38:57.498 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.498 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:38:57.498 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6806", 2022-08-01T19:38:57.499 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:38:57.499 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.499 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.499 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:38:57.499 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6807", 2022-08-01T19:38:57.500 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:38:57.500 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.500 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.500 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.501 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:38:57.501 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:38:57.501 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.501 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:38:57.501 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6804", 2022-08-01T19:38:57.502 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:38:57.502 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.502 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.502 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:38:57.503 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6805", 2022-08-01T19:38:57.503 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:38:57.503 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.503 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.503 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.504 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6801/23008", 2022-08-01T19:38:57.504 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6803/23008", 2022-08-01T19:38:57.504 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6807/23008", 2022-08-01T19:38:57.504 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6805/23008", 2022-08-01T19:38:57.505 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:38:57.505 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:38:57.505 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:38:57.505 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.506 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.506 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.506 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:38:57.506 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "805f7051-a527-4e98-8844-b6e406a63248", 2022-08-01T19:38:57.506 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:38:57.507 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:38:57.507 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:38:57.507 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:38:57.507 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:38:57.508 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:38:57.508 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:38:57.508 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 53, 2022-08-01T19:38:57.508 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:38:57.508 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:38:57.509 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:38:57.509 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:38:57.509 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.509 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:38:57.510 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6808", 2022-08-01T19:38:57.510 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:38:57.510 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.510 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.510 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:38:57.511 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6809", 2022-08-01T19:38:57.511 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:38:57.511 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.511 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.512 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.512 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:38:57.512 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:38:57.512 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.512 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:38:57.513 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6810", 2022-08-01T19:38:57.513 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:38:57.513 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.513 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.514 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:38:57.514 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6811", 2022-08-01T19:38:57.514 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:38:57.514 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.514 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.515 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.515 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:38:57.515 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:38:57.515 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.516 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:38:57.516 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6814", 2022-08-01T19:38:57.516 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:38:57.516 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.516 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.517 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:38:57.517 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6815", 2022-08-01T19:38:57.517 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:38:57.517 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.518 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.518 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.518 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:38:57.518 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:38:57.519 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.519 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:38:57.519 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6812", 2022-08-01T19:38:57.519 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:38:57.519 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.520 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.520 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:38:57.520 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6813", 2022-08-01T19:38:57.520 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:38:57.521 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.521 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.521 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.521 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6809/23007", 2022-08-01T19:38:57.522 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6811/23007", 2022-08-01T19:38:57.522 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6815/23007", 2022-08-01T19:38:57.522 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6813/23007", 2022-08-01T19:38:57.522 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:38:57.522 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:38:57.523 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:38:57.523 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.523 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.523 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.523 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:38:57.524 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "89f7f3df-7c10-48e1-8b69-9e4243923a61", 2022-08-01T19:38:57.524 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:38:57.524 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:38:57.524 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:38:57.525 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:38:57.525 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:38:57.525 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:38:57.525 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:38:57.526 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 53, 2022-08-01T19:38:57.526 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:38:57.526 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:38:57.526 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:38:57.526 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:38:57.527 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.527 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:38:57.527 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6817", 2022-08-01T19:38:57.527 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:38:57.528 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.528 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.528 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:38:57.528 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6819", 2022-08-01T19:38:57.529 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:38:57.529 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.529 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.529 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.530 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:38:57.530 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:38:57.530 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.530 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:38:57.531 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6822", 2022-08-01T19:38:57.531 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:38:57.531 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.531 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.531 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:38:57.532 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6824", 2022-08-01T19:38:57.532 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:38:57.532 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.532 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.533 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.533 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:38:57.533 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:38:57.533 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.533 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:38:57.534 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6830", 2022-08-01T19:38:57.534 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:38:57.534 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.534 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.535 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:38:57.535 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6831", 2022-08-01T19:38:57.535 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:38:57.535 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.535 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.536 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.536 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:38:57.536 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:38:57.536 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.537 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:38:57.537 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6826", 2022-08-01T19:38:57.537 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:38:57.537 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.537 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.538 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:38:57.538 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6829", 2022-08-01T19:38:57.538 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:38:57.538 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.538 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.539 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.539 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6819/23010", 2022-08-01T19:38:57.539 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6824/23010", 2022-08-01T19:38:57.539 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6831/23010", 2022-08-01T19:38:57.540 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6829/23010", 2022-08-01T19:38:57.540 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:38:57.540 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:38:57.540 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:38:57.540 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.541 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.541 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.541 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:38:57.541 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "96123468-aabc-4a64-909e-a192c369090d", 2022-08-01T19:38:57.542 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:38:57.542 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:38:57.542 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:38:57.542 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:38:57.542 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:38:57.543 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:38:57.543 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:38:57.543 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 52, 2022-08-01T19:38:57.543 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:38:57.544 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:38:57.544 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:38:57.544 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:38:57.544 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.544 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:38:57.545 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6816", 2022-08-01T19:38:57.545 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:38:57.545 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.545 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.546 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:38:57.546 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6818", 2022-08-01T19:38:57.546 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:38:57.546 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.547 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.547 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.547 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:38:57.547 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:38:57.548 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.548 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:38:57.548 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6820", 2022-08-01T19:38:57.548 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:38:57.549 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.549 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.549 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:38:57.549 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6821", 2022-08-01T19:38:57.549 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:38:57.550 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.550 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.550 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.550 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:38:57.551 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:38:57.551 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.551 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:38:57.551 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6827", 2022-08-01T19:38:57.551 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:38:57.552 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.552 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.552 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:38:57.552 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6828", 2022-08-01T19:38:57.553 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:38:57.553 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.553 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.553 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.553 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:38:57.554 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:38:57.554 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.554 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:38:57.554 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6823", 2022-08-01T19:38:57.555 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:38:57.555 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.555 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.555 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:38:57.555 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6825", 2022-08-01T19:38:57.556 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:38:57.556 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.556 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.556 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.557 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6818/23009", 2022-08-01T19:38:57.557 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6821/23009", 2022-08-01T19:38:57.557 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6828/23009", 2022-08-01T19:38:57.557 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6825/23009", 2022-08-01T19:38:57.557 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:38:57.558 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:38:57.558 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:38:57.558 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.559 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.559 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.559 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:38:57.559 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a6e26665-e04b-46ec-ab66-e912acbf044c", 2022-08-01T19:38:57.560 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:38:57.560 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:38:57.560 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:38:57.560 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:38:57.561 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:38:57.561 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:38:57.561 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:38:57.561 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 53, 2022-08-01T19:38:57.561 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:38:57.562 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:38:57.562 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:38:57.562 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:38:57.562 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.563 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:38:57.563 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6810", 2022-08-01T19:38:57.563 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:38:57.563 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.564 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.564 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:38:57.564 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6811", 2022-08-01T19:38:57.564 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:38:57.564 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.565 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.565 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.565 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:38:57.565 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:38:57.566 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.566 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:38:57.566 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6812", 2022-08-01T19:38:57.566 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:38:57.567 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.567 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.567 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:38:57.567 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6813", 2022-08-01T19:38:57.567 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:38:57.568 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.568 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.568 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.568 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:38:57.569 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:38:57.569 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.569 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:38:57.569 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6816", 2022-08-01T19:38:57.569 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:38:57.570 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.570 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.570 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:38:57.570 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6817", 2022-08-01T19:38:57.571 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:38:57.571 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.571 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.571 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.571 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:38:57.572 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:38:57.572 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.572 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:38:57.572 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6814", 2022-08-01T19:38:57.572 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:38:57.573 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.573 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.573 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:38:57.573 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6815", 2022-08-01T19:38:57.574 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:38:57.574 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.574 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.574 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.574 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6811/23584", 2022-08-01T19:38:57.575 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6813/23584", 2022-08-01T19:38:57.575 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6817/23584", 2022-08-01T19:38:57.575 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6815/23584", 2022-08-01T19:38:57.575 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:38:57.576 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:38:57.576 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:38:57.576 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.576 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.576 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.577 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:38:57.577 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "502c0034-18ee-44ac-87ff-19b9dddb66a0", 2022-08-01T19:38:57.577 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:38:57.577 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:38:57.578 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:38:57.578 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:38:57.578 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:38:57.578 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:38:57.579 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:38:57.579 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 53, 2022-08-01T19:38:57.579 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:38:57.579 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:38:57.579 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:38:57.580 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:38:57.580 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.580 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:38:57.580 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6802", 2022-08-01T19:38:57.581 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:38:57.581 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.581 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.581 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:38:57.581 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6803", 2022-08-01T19:38:57.582 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:38:57.582 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.582 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.582 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.583 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:38:57.583 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:38:57.583 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.583 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:38:57.583 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6804", 2022-08-01T19:38:57.584 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:38:57.584 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.584 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.584 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:38:57.585 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6805", 2022-08-01T19:38:57.585 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:38:57.585 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.585 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.585 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.586 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:38:57.586 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:38:57.586 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.586 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:38:57.586 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6808", 2022-08-01T19:38:57.587 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:38:57.587 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.587 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.587 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:38:57.588 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6809", 2022-08-01T19:38:57.588 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:38:57.588 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.588 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.588 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.589 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:38:57.589 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:38:57.589 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.589 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:38:57.590 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6806", 2022-08-01T19:38:57.590 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:38:57.590 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.590 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.590 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:38:57.591 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6807", 2022-08-01T19:38:57.591 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:38:57.591 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.591 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.592 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.592 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6803/23585", 2022-08-01T19:38:57.592 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6805/23585", 2022-08-01T19:38:57.592 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6809/23585", 2022-08-01T19:38:57.592 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6807/23585", 2022-08-01T19:38:57.593 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:38:57.593 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:38:57.593 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:38:57.593 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.593 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.594 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.594 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:38:57.594 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "ac4e755a-7fe7-46cb-8407-852d5af9ffc5", 2022-08-01T19:38:57.594 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:38:57.595 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:38:57.595 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:38:57.595 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:38:57.595 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:38:57.596 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:38:57.596 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:38:57.596 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 53, 2022-08-01T19:38:57.596 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:38:57.596 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:38:57.597 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:38:57.597 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:38:57.597 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.597 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:38:57.598 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6820", 2022-08-01T19:38:57.598 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:38:57.598 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.598 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.598 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:38:57.599 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6821", 2022-08-01T19:38:57.599 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:38:57.599 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.599 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.600 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.600 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:38:57.600 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:38:57.600 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.600 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:38:57.601 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6823", 2022-08-01T19:38:57.601 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:38:57.601 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.601 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.601 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:38:57.602 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6825", 2022-08-01T19:38:57.602 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:38:57.602 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.602 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.603 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.603 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:38:57.603 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:38:57.603 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.603 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:38:57.604 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6831", 2022-08-01T19:38:57.604 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:38:57.604 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.604 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.604 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:38:57.605 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6833", 2022-08-01T19:38:57.605 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:38:57.605 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.605 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.606 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.606 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:38:57.606 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:38:57.606 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.606 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:38:57.607 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6827", 2022-08-01T19:38:57.607 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:38:57.607 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.607 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.608 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:38:57.608 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6829", 2022-08-01T19:38:57.608 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:38:57.608 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.608 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.609 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.609 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6821/23587", 2022-08-01T19:38:57.609 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6825/23587", 2022-08-01T19:38:57.609 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6833/23587", 2022-08-01T19:38:57.610 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6829/23587", 2022-08-01T19:38:57.610 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:38:57.610 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:38:57.610 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:38:57.610 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.611 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.611 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.611 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:38:57.611 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "874761e2-6410-40fc-b60c-e58abe5ce489", 2022-08-01T19:38:57.612 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:38:57.612 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:38:57.612 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:38:57.612 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:38:57.612 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:38:57.613 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:38:57.613 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:38:57.613 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 52, 2022-08-01T19:38:57.613 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:38:57.614 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:38:57.614 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:38:57.614 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:38:57.614 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.614 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:38:57.615 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6818", 2022-08-01T19:38:57.615 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:38:57.615 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.615 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.616 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:38:57.616 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6819", 2022-08-01T19:38:57.616 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:38:57.616 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.617 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.617 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.617 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:38:57.617 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:38:57.618 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.618 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:38:57.618 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6822", 2022-08-01T19:38:57.618 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:38:57.618 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.619 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.619 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:38:57.619 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6824", 2022-08-01T19:38:57.619 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:38:57.620 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.620 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.620 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.620 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:38:57.620 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:38:57.621 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.621 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:38:57.621 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6830", 2022-08-01T19:38:57.621 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:38:57.622 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.622 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.622 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:38:57.622 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6832", 2022-08-01T19:38:57.622 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:38:57.623 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.623 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.623 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.623 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:38:57.624 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:38:57.624 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.624 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:38:57.625 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6826", 2022-08-01T19:38:57.625 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:38:57.625 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.625 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.625 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:38:57.626 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6828", 2022-08-01T19:38:57.626 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:38:57.626 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.626 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.627 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.627 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6819/23586", 2022-08-01T19:38:57.627 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6824/23586", 2022-08-01T19:38:57.627 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6832/23586", 2022-08-01T19:38:57.627 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6828/23586", 2022-08-01T19:38:57.628 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:38:57.628 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:38:57.628 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:38:57.628 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:38:57.629 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.629 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:38:57.629 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_xinfo": [ 2022-08-01T19:38:57.629 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.629 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:38:57.630 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:38:57.630 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:38:57.630 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:38:57.630 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:38:57.631 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:38:57.631 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.588817+0000", 2022-08-01T19:38:57.631 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:38:57.631 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.632 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.632 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:38:57.632 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:38:57.632 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:38:57.632 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:38:57.633 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:38:57.633 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:38:57.633 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.782753+0000", 2022-08-01T19:38:57.633 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:38:57.634 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.634 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.634 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:38:57.634 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:38:57.634 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:38:57.635 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:38:57.635 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:38:57.635 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:38:57.635 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.837924+0000", 2022-08-01T19:38:57.635 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:38:57.636 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.636 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.636 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:38:57.636 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:38:57.637 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:38:57.637 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:38:57.637 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:38:57.637 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:38:57.637 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.839948+0000", 2022-08-01T19:38:57.638 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:38:57.638 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.638 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.638 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:38:57.639 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:38:57.639 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:38:57.639 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:38:57.639 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:38:57.639 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:38:57.640 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.112801+0000", 2022-08-01T19:38:57.640 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:38:57.640 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.640 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.641 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:38:57.641 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:38:57.641 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:38:57.641 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:38:57.642 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:38:57.642 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:38:57.642 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.106704+0000", 2022-08-01T19:38:57.642 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:38:57.643 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.643 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.643 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:38:57.643 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:38:57.643 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:38:57.644 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:38:57.644 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:38:57.644 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:38:57.644 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.893170+0000", 2022-08-01T19:38:57.644 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:38:57.645 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.645 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:38:57.645 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:38:57.645 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:38:57.646 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:38:57.646 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:38:57.646 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:38:57.646 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:38:57.647 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.886947+0000", 2022-08-01T19:38:57.647 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:38:57.647 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.647 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:38:57.647 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap": [], 2022-08-01T19:38:57.648 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap_items": [], 2022-08-01T19:38:57.648 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_temp": [], 2022-08-01T19:38:57.648 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_temp": [], 2022-08-01T19:38:57.648 INFO:teuthology.orchestra.run.smithi017.stdout: "blacklist": {}, 2022-08-01T19:38:57.648 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profiles": { 2022-08-01T19:38:57.649 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_data": { 2022-08-01T19:38:57.649 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-device-class": "", 2022-08-01T19:38:57.649 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:38:57.649 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-root": "default", 2022-08-01T19:38:57.650 INFO:teuthology.orchestra.run.smithi017.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T19:38:57.650 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:38:57.650 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "2", 2022-08-01T19:38:57.650 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:38:57.650 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:38:57.651 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van", 2022-08-01T19:38:57.651 INFO:teuthology.orchestra.run.smithi017.stdout: "w": "8" 2022-08-01T19:38:57.651 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.651 INFO:teuthology.orchestra.run.smithi017.stdout: "default": { 2022-08-01T19:38:57.652 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:38:57.652 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:38:57.652 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "1", 2022-08-01T19:38:57.652 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:38:57.652 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:38:57.653 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van" 2022-08-01T19:38:57.653 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:38:57.653 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:38:57.653 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps_queue": [], 2022-08-01T19:38:57.653 INFO:teuthology.orchestra.run.smithi017.stdout: "new_removed_snaps": [], 2022-08-01T19:38:57.654 INFO:teuthology.orchestra.run.smithi017.stdout: "new_purged_snaps": [], 2022-08-01T19:38:57.654 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_node_flags": {}, 2022-08-01T19:38:57.654 INFO:teuthology.orchestra.run.smithi017.stdout: "device_class_flags": {} 2022-08-01T19:38:57.654 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:38:57.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.410+0000 7fbc5359e700 1 -- 172.21.15.17:0/1851924290 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbc30053aa0 msgr2=0x7fbc30055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:57.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.410+0000 7fbc5359e700 1 --2- 172.21.15.17:0/1851924290 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbc30053aa0 0x7fbc30055f60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7fbc4800b000 tx=0x7fbc48006df0).stop 2022-08-01T19:38:57.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.410+0000 7fbc5359e700 1 -- 172.21.15.17:0/1851924290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc540b2710 msgr2=0x7fbc540ab9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:57.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.410+0000 7fbc5359e700 1 --2- 172.21.15.17:0/1851924290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc540b2710 0x7fbc540ab9b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7fbc4c001110 tx=0x7fbc4c007a80).stop 2022-08-01T19:38:57.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.410+0000 7fbc5359e700 1 -- 172.21.15.17:0/1851924290 shutdown_connections 2022-08-01T19:38:57.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.410+0000 7fbc5359e700 1 --2- 172.21.15.17:0/1851924290 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbc30053aa0 0x7fbc30055f60 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:57.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.410+0000 7fbc5359e700 1 --2- 172.21.15.17:0/1851924290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc540b2710 0x7fbc540ab9b0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:57.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.410+0000 7fbc5359e700 1 --2- 172.21.15.17:0/1851924290 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbc540b01a0 0x7fbc54067520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:57.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.410+0000 7fbc5359e700 1 --2- 172.21.15.17:0/1851924290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc540afd60 0x7fbc5414cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:57.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.410+0000 7fbc5359e700 1 -- 172.21.15.17:0/1851924290 >> 172.21.15.17:0/1851924290 conn(0x7fbc54060310 msgr2=0x7fbc54144cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:38:57.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.410+0000 7fbc5359e700 1 -- 172.21.15.17:0/1851924290 shutdown_connections 2022-08-01T19:38:57.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.410+0000 7fbc5359e700 1 -- 172.21.15.17:0/1851924290 wait complete. 2022-08-01T19:38:57.660 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:38:57.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.770+0000 7f6b10ff2700 1 Processor -- start 2022-08-01T19:38:57.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.770+0000 7f6b10ff2700 1 -- start start 2022-08-01T19:38:57.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.770+0000 7f6b10ff2700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6b0c0b6710 0x7f6b0c0b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:57.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.770+0000 7f6b10ff2700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6b0c0b6af0 0x7f6b0c12bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:57.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.770+0000 7f6b10ff2700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b0c144240 con 0x7f6b0c0b4550 2022-08-01T19:38:57.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.770+0000 7f6b10ff2700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b0c143d00 con 0x7f6b0c0b6710 2022-08-01T19:38:57.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.770+0000 7f6b10ff2700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c143fa0 con 0x7f6b0c0b6af0 2022-08-01T19:38:57.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.770+0000 7f6b0a59c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6b0c0b6710 0x7f6b0c0b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:57.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.770+0000 7f6b0a59c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6b0c0b6710 0x7f6b0c0b5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39468/0 (socket says 172.21.15.17:39468) 2022-08-01T19:38:57.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.770+0000 7f6b0a59c700 1 -- 172.21.15.17:0/1607520521 learned_addr learned my addr 172.21.15.17:0/1607520521 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:38:57.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.770+0000 7f6b0ad9d700 1 --2- 172.21.15.17:0/1607520521 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6b0c0b6af0 0x7f6b0c12bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:57.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.770+0000 7f6b0a59c700 1 -- 172.21.15.17:0/1607520521 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6b0c0b6af0 msgr2=0x7f6b0c12bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:38:57.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.770+0000 7f6b0a59c700 1 --2- 172.21.15.17:0/1607520521 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6b0c0b6af0 0x7f6b0c12bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:57.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.770+0000 7f6b0959a700 1 -- 172.21.15.17:0/1607520521 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 348730713 0 0) 0x7f6b0c144240 con 0x7f6b0c0b4550 2022-08-01T19:38:57.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.770+0000 7f6b0a59c700 1 -- 172.21.15.17:0/1607520521 >> v1:172.21.15.77:6789/0 conn(0x7f6b0c0b4550 legacy=0x7f6b0c061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:57.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.770+0000 7f6b0a59c700 1 -- 172.21.15.17:0/1607520521 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0c061780 con 0x7f6b0c0b6710 2022-08-01T19:38:57.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.770+0000 7f6b0a59c700 1 --2- 172.21.15.17:0/1607520521 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b0c0b6710 0x7f6b0c0b5d80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f6b0c1432b0 tx=0x7f6afc006eb0).ready entity=mon.0 client_cookie=c9efd66e1a11a56c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:38:57.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.770+0000 7f6b0959a700 1 -- 172.21.15.17:0/1607520521 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6afc00d5c0 con 0x7f6b0c0b6710 2022-08-01T19:38:57.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.770+0000 7f6b0959a700 1 -- 172.21.15.17:0/1607520521 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6afc000e90 con 0x7f6b0c0b6710 2022-08-01T19:38:57.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.770+0000 7f6b0959a700 1 -- 172.21.15.17:0/1607520521 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6afc00d910 con 0x7f6b0c0b6710 2022-08-01T19:38:57.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.770+0000 7f6b10ff2700 1 -- 172.21.15.17:0/1607520521 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b0c0b6710 msgr2=0x7f6b0c0b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:57.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.770+0000 7f6b10ff2700 1 --2- 172.21.15.17:0/1607520521 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b0c0b6710 0x7f6b0c0b5d80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f6b0c1432b0 tx=0x7f6afc006eb0).stop 2022-08-01T19:38:57.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.770+0000 7f6b10ff2700 1 -- 172.21.15.17:0/1607520521 shutdown_connections 2022-08-01T19:38:57.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.770+0000 7f6b10ff2700 1 --2- 172.21.15.17:0/1607520521 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6b0c0b6af0 0x7f6b0c12bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:57.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.770+0000 7f6b10ff2700 1 --2- 172.21.15.17:0/1607520521 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b0c0b6710 0x7f6b0c0b5d80 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:57.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.770+0000 7f6b10ff2700 1 -- 172.21.15.17:0/1607520521 >> 172.21.15.17:0/1607520521 conn(0x7f6b0c05c620 msgr2=0x7f6b0c05ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:38:57.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.770+0000 7f6b10ff2700 1 -- 172.21.15.17:0/1607520521 shutdown_connections 2022-08-01T19:38:57.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.770+0000 7f6b10ff2700 1 -- 172.21.15.17:0/1607520521 wait complete. 2022-08-01T19:38:57.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.774+0000 7f6b10ff2700 1 Processor -- start 2022-08-01T19:38:57.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.774+0000 7f6b10ff2700 1 -- start start 2022-08-01T19:38:57.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.774+0000 7f6b10ff2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b0c0b4550 0x7f6b0c129c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:57.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.778+0000 7f6b10ff2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b0c0b6710 0x7f6b0c12b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:57.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.778+0000 7f6b10ff2700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6b0c0b6af0 0x7f6b0c12bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:57.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.778+0000 7f6b10ff2700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c1566f0 con 0x7f6b0c0b4550 2022-08-01T19:38:57.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.778+0000 7f6b10ff2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c149200 con 0x7f6b0c0b6710 2022-08-01T19:38:57.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.778+0000 7f6b10ff2700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c1494a0 con 0x7f6b0c0b6af0 2022-08-01T19:38:57.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.778+0000 7f6b0a59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b0c0b4550 0x7f6b0c129c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:57.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.778+0000 7f6b0a59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b0c0b4550 0x7f6b0c129c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39474/0 (socket says 172.21.15.17:39474) 2022-08-01T19:38:57.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.778+0000 7f6b0a59c700 1 -- 172.21.15.17:0/3638099372 learned_addr learned my addr 172.21.15.17:0/3638099372 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:38:57.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.778+0000 7f6b0ad9d700 1 --2- 172.21.15.17:0/3638099372 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6b0c0b6af0 0x7f6b0c12bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:57.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.778+0000 7f6b09d9b700 1 --2- 172.21.15.17:0/3638099372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b0c0b6710 0x7f6b0c12b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:57.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.778+0000 7f6b0a59c700 1 -- 172.21.15.17:0/3638099372 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6b0c0b6af0 msgr2=0x7f6b0c12bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:57.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.778+0000 7f6b0a59c700 1 --2- 172.21.15.17:0/3638099372 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6b0c0b6af0 0x7f6b0c12bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:57.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.778+0000 7f6b0a59c700 1 -- 172.21.15.17:0/3638099372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b0c0b6710 msgr2=0x7f6b0c12b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:57.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.778+0000 7f6b0a59c700 1 --2- 172.21.15.17:0/3638099372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b0c0b6710 0x7f6b0c12b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:57.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.778+0000 7f6b0a59c700 1 -- 172.21.15.17:0/3638099372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6afc00b1e0 con 0x7f6b0c0b4550 2022-08-01T19:38:57.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.778+0000 7f6b09d9b700 1 --2- 172.21.15.17:0/3638099372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b0c0b6710 0x7f6b0c12b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:38:57.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.778+0000 7f6b0a59c700 1 --2- 172.21.15.17:0/3638099372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b0c0b4550 0x7f6b0c129c50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f6b0c1432b0 tx=0x7f6afc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:38:57.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.778+0000 7f6afb7fe700 1 -- 172.21.15.17:0/3638099372 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6afc011860 con 0x7f6b0c0b4550 2022-08-01T19:38:57.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.778+0000 7f6afb7fe700 1 -- 172.21.15.17:0/3638099372 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6afc0119c0 con 0x7f6b0c0b4550 2022-08-01T19:38:57.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.778+0000 7f6b10ff2700 1 -- 172.21.15.17:0/3638099372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b0c0b36b0 con 0x7f6b0c0b4550 2022-08-01T19:38:57.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.778+0000 7f6afb7fe700 1 -- 172.21.15.17:0/3638099372 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6afc011d20 con 0x7f6b0c0b4550 2022-08-01T19:38:57.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.778+0000 7f6b10ff2700 1 -- 172.21.15.17:0/3638099372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b0c1563f0 con 0x7f6b0c0b4550 2022-08-01T19:38:57.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.778+0000 7f6b10ff2700 1 -- 172.21.15.17:0/3638099372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b0c061a20 con 0x7f6b0c0b4550 2022-08-01T19:38:57.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.782+0000 7f6afb7fe700 1 -- 172.21.15.17:0/3638099372 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6afc025030 con 0x7f6b0c0b4550 2022-08-01T19:38:57.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.782+0000 7f6afb7fe700 1 --2- 172.21.15.17:0/3638099372 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6ae8053b00 0x7f6ae8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:57.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.782+0000 7f6afb7fe700 1 -- 172.21.15.17:0/3638099372 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6afc06d270 con 0x7f6b0c0b4550 2022-08-01T19:38:57.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.782+0000 7f6b09d9b700 1 --2- 172.21.15.17:0/3638099372 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6ae8053b00 0x7f6ae8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:57.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.782+0000 7f6b09d9b700 1 --2- 172.21.15.17:0/3638099372 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6ae8053b00 0x7f6ae8055fc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f6b0000b000 tx=0x7f6b00009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:38:57.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.786+0000 7f6afb7fe700 1 -- 172.21.15.17:0/3638099372 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6afc043f20 con 0x7f6b0c0b4550 2022-08-01T19:38:57.950 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.946+0000 7f6b10ff2700 1 -- 172.21.15.17:0/3638099372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b0c0574e0 con 0x7f6b0c0b4550 2022-08-01T19:38:57.951 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.950+0000 7f6afb7fe700 1 -- 172.21.15.17:0/3638099372 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f6afc023020 con 0x7f6b0c0b4550 2022-08-01T19:38:57.952 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:38:57.952 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":24,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/1914125901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":1914125901},{"type":"v1","addr":"172.21.15.17:6833","nonce":1914125901}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":22},{"gid":5587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1403688289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1403688289},{"type":"v1","addr":"172.21.15.77:6835","nonce":1403688289}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":22}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:37:36.197818+0000","modified":"2022-08-01T19:37:37.201700+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5589":{"gid":5589,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/3325991310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":3325991310},{"type":"v1","addr":"172.21.15.17:6835","nonce":3325991310}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-08-01T19:38:57.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.950+0000 7f6b10ff2700 1 -- 172.21.15.17:0/3638099372 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6ae8053b00 msgr2=0x7f6ae8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:57.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.950+0000 7f6b10ff2700 1 --2- 172.21.15.17:0/3638099372 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6ae8053b00 0x7f6ae8055fc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f6b0000b000 tx=0x7f6b00009040).stop 2022-08-01T19:38:57.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.950+0000 7f6b10ff2700 1 -- 172.21.15.17:0/3638099372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b0c0b4550 msgr2=0x7f6b0c129c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:57.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.950+0000 7f6b10ff2700 1 --2- 172.21.15.17:0/3638099372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b0c0b4550 0x7f6b0c129c50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f6b0c1432b0 tx=0x7f6afc015000).stop 2022-08-01T19:38:57.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.950+0000 7f6b10ff2700 1 -- 172.21.15.17:0/3638099372 shutdown_connections 2022-08-01T19:38:57.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.950+0000 7f6b10ff2700 1 --2- 172.21.15.17:0/3638099372 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6ae8053b00 0x7f6ae8055fc0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:57.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.950+0000 7f6b10ff2700 1 --2- 172.21.15.17:0/3638099372 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6b0c0b6af0 0x7f6b0c12bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:57.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.950+0000 7f6b10ff2700 1 --2- 172.21.15.17:0/3638099372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b0c0b6710 0x7f6b0c12b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:57.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.950+0000 7f6b10ff2700 1 --2- 172.21.15.17:0/3638099372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b0c0b4550 0x7f6b0c129c50 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:57.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.950+0000 7f6b10ff2700 1 -- 172.21.15.17:0/3638099372 >> 172.21.15.17:0/3638099372 conn(0x7f6b0c05c620 msgr2=0x7f6b0c061de0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:38:57.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.950+0000 7f6b10ff2700 1 -- 172.21.15.17:0/3638099372 shutdown_connections 2022-08-01T19:38:57.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:57.950+0000 7f6b10ff2700 1 -- 172.21.15.17:0/3638099372 wait complete. 2022-08-01T19:38:57.958 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 24 2022-08-01T19:38:57.965 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-08-01T19:38:58.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.066+0000 7f5710efa700 1 Processor -- start 2022-08-01T19:38:58.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.066+0000 7f5710efa700 1 -- start start 2022-08-01T19:38:58.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.066+0000 7f5710efa700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f570c0b07a0 0x7f570c0af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:58.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.066+0000 7f5710efa700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f570c147730 0x7f570c0acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:58.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.066+0000 7f5710efa700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f570c0b5a10 con 0x7f570c0b5630 2022-08-01T19:38:58.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.066+0000 7f5710efa700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f570c145470 con 0x7f570c0b07a0 2022-08-01T19:38:58.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.066+0000 7f5710efa700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f570c0ae180 con 0x7f570c147730 2022-08-01T19:38:58.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.066+0000 7f570b7fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f570c0b5630 0x7f570c067730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59908/0 (socket says 172.21.15.17:59908) 2022-08-01T19:38:58.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.066+0000 7f570b7fe700 1 -- 172.21.15.17:0/1692081620 learned_addr learned my addr 172.21.15.17:0/1692081620 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:38:58.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.066+0000 7f570affd700 1 --2- 172.21.15.17:0/1692081620 >> v2:172.21.15.77:3300/0 conn(0x7f570c0b07a0 0x7f570c0af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:58.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.066+0000 7f570bfff700 1 --2- 172.21.15.17:0/1692081620 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f570c147730 0x7f570c0acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:58.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.066+0000 7f570a7fc700 1 -- 172.21.15.17:0/1692081620 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3472838283 0 0) 0x7f570c0b5a10 con 0x7f570c0b5630 2022-08-01T19:38:58.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.066+0000 7f570a7fc700 1 -- 172.21.15.17:0/1692081620 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56ec003260 con 0x7f570c0b5630 2022-08-01T19:38:58.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.066+0000 7f570affd700 1 -- 172.21.15.17:0/1692081620 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f570c147730 msgr2=0x7f570c0acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:38:58.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.066+0000 7f570affd700 1 --2- 172.21.15.17:0/1692081620 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f570c147730 0x7f570c0acdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:58.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.066+0000 7f570affd700 1 -- 172.21.15.17:0/1692081620 >> v1:172.21.15.17:6789/0 conn(0x7f570c0b5630 legacy=0x7f570c067730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:58.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.066+0000 7f570affd700 1 -- 172.21.15.17:0/1692081620 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f570c0ac990 con 0x7f570c0b07a0 2022-08-01T19:38:58.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.066+0000 7f570affd700 1 --2- 172.21.15.17:0/1692081620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f570c0b07a0 0x7f570c0af6e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f56fc0032f0 tx=0x7f56fc0091d0).ready entity=mon.1 client_cookie=2103b0456e44e81b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:38:58.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.066+0000 7f570a7fc700 1 -- 172.21.15.17:0/1692081620 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56fc00b600 con 0x7f570c0b07a0 2022-08-01T19:38:58.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.066+0000 7f570a7fc700 1 -- 172.21.15.17:0/1692081620 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56fc00b760 con 0x7f570c0b07a0 2022-08-01T19:38:58.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.066+0000 7f570a7fc700 1 -- 172.21.15.17:0/1692081620 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56fc00b910 con 0x7f570c0b07a0 2022-08-01T19:38:58.078 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.070+0000 7f5710efa700 1 -- 172.21.15.17:0/1692081620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f570c0b07a0 msgr2=0x7f570c0af6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:58.079 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.070+0000 7f5710efa700 1 --2- 172.21.15.17:0/1692081620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f570c0b07a0 0x7f570c0af6e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f56fc0032f0 tx=0x7f56fc0091d0).stop 2022-08-01T19:38:58.079 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.070+0000 7f5710efa700 1 -- 172.21.15.17:0/1692081620 shutdown_connections 2022-08-01T19:38:58.079 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.070+0000 7f5710efa700 1 --2- 172.21.15.17:0/1692081620 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f570c147730 0x7f570c0acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:58.079 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.070+0000 7f5710efa700 1 --2- 172.21.15.17:0/1692081620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f570c0b07a0 0x7f570c0af6e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:58.079 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.070+0000 7f5710efa700 1 -- 172.21.15.17:0/1692081620 >> 172.21.15.17:0/1692081620 conn(0x7f570c0626f0 msgr2=0x7f570c062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:38:58.080 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.070+0000 7f5710efa700 1 -- 172.21.15.17:0/1692081620 shutdown_connections 2022-08-01T19:38:58.080 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.070+0000 7f5710efa700 1 -- 172.21.15.17:0/1692081620 wait complete. 2022-08-01T19:38:58.080 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.070+0000 7f5710efa700 1 Processor -- start 2022-08-01T19:38:58.080 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.070+0000 7f5710efa700 1 -- start start 2022-08-01T19:38:58.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.074+0000 7f5710efa700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f570c0b07a0 0x7f570c066de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:58.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.074+0000 7f5710efa700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f570c0b5630 0x7f570c13fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:58.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.074+0000 7f5710efa700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f570c147730 0x7f570c140220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:58.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.074+0000 7f5710efa700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570c064d90 con 0x7f570c147730 2022-08-01T19:38:58.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.074+0000 7f5710efa700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570c144cc0 con 0x7f570c0b07a0 2022-08-01T19:38:58.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.074+0000 7f5710efa700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f570c0b4900 con 0x7f570c0b5630 2022-08-01T19:38:58.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.074+0000 7f570bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f570c147730 0x7f570c140220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:58.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.074+0000 7f570bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f570c147730 0x7f570c140220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39492/0 (socket says 172.21.15.17:39492) 2022-08-01T19:38:58.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.074+0000 7f570b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f570c0b07a0 0x7f570c066de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:58.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.074+0000 7f570bfff700 1 -- 172.21.15.17:0/224069649 learned_addr learned my addr 172.21.15.17:0/224069649 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:38:58.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.074+0000 7f570affd700 1 --2- 172.21.15.17:0/224069649 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f570c0b5630 0x7f570c13fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:58.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.074+0000 7f570bfff700 1 -- 172.21.15.17:0/224069649 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f570c0b5630 msgr2=0x7f570c13fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:58.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.074+0000 7f570bfff700 1 --2- 172.21.15.17:0/224069649 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f570c0b5630 0x7f570c13fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:58.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.074+0000 7f570bfff700 1 -- 172.21.15.17:0/224069649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f570c0b07a0 msgr2=0x7f570c066de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:58.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.074+0000 7f570bfff700 1 --2- 172.21.15.17:0/224069649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f570c0b07a0 0x7f570c066de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:58.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.074+0000 7f570bfff700 1 -- 172.21.15.17:0/224069649 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56fc009470 con 0x7f570c147730 2022-08-01T19:38:58.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.074+0000 7f570bfff700 1 --2- 172.21.15.17:0/224069649 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f570c147730 0x7f570c140220 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f570400c000 tx=0x7f5704006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:38:58.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.074+0000 7f5708ff9700 1 -- 172.21.15.17:0/224069649 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5704011750 con 0x7f570c147730 2022-08-01T19:38:58.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.074+0000 7f5708ff9700 1 -- 172.21.15.17:0/224069649 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57040118b0 con 0x7f570c147730 2022-08-01T19:38:58.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.074+0000 7f5710efa700 1 -- 172.21.15.17:0/224069649 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f570c1466c0 con 0x7f570c147730 2022-08-01T19:38:58.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.074+0000 7f5708ff9700 1 -- 172.21.15.17:0/224069649 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5704011c10 con 0x7f570c147730 2022-08-01T19:38:58.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.074+0000 7f5710efa700 1 -- 172.21.15.17:0/224069649 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f570c0667c0 con 0x7f570c147730 2022-08-01T19:38:58.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.074+0000 7f5710efa700 1 -- 172.21.15.17:0/224069649 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f570c0574e0 con 0x7f570c147730 2022-08-01T19:38:58.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.078+0000 7f5708ff9700 1 -- 172.21.15.17:0/224069649 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5704011d70 con 0x7f570c147730 2022-08-01T19:38:58.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.078+0000 7f5708ff9700 1 --2- 172.21.15.17:0/224069649 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f56f0053b00 0x7f56f0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:58.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.078+0000 7f5708ff9700 1 -- 172.21.15.17:0/224069649 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f570406c440 con 0x7f570c147730 2022-08-01T19:38:58.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.078+0000 7f570b7fe700 1 --2- 172.21.15.17:0/224069649 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f56f0053b00 0x7f56f0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:58.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.078+0000 7f570b7fe700 1 --2- 172.21.15.17:0/224069649 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f56f0053b00 0x7f56f0055fc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f56f400d000 tx=0x7f56f4009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:38:58.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.086+0000 7f5708ff9700 1 -- 172.21.15.17:0/224069649 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f57040430f0 con 0x7f570c147730 2022-08-01T19:38:58.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:58.250+0000 7f5710efa700 1 -- 172.21.15.17:0/224069649 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f570c05f3d0 con 0x7f570c147730 2022-08-01T19:38:59.168 INFO:tasks.ceph.mon.a.smithi017.stderr:2022-08-01T19:38:59.166+0000 7f6a53773700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-08-01T19:38:59.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.178+0000 7f5708ff9700 1 -- 172.21.15.17:0/224069649 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5704046710 con 0x7f570c147730 2022-08-01T19:38:59.184 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.182+0000 7f5710efa700 1 -- 172.21.15.17:0/224069649 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f56f0053b00 msgr2=0x7f56f0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:59.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.182+0000 7f5710efa700 1 --2- 172.21.15.17:0/224069649 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f56f0053b00 0x7f56f0055fc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f56f400d000 tx=0x7f56f4009040).stop 2022-08-01T19:38:59.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.182+0000 7f5710efa700 1 -- 172.21.15.17:0/224069649 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f570c147730 msgr2=0x7f570c140220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:59.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.182+0000 7f5710efa700 1 --2- 172.21.15.17:0/224069649 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f570c147730 0x7f570c140220 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f570400c000 tx=0x7f5704006df0).stop 2022-08-01T19:38:59.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.182+0000 7f5710efa700 1 -- 172.21.15.17:0/224069649 shutdown_connections 2022-08-01T19:38:59.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.182+0000 7f5710efa700 1 --2- 172.21.15.17:0/224069649 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f56f0053b00 0x7f56f0055fc0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:59.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.182+0000 7f5710efa700 1 --2- 172.21.15.17:0/224069649 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f570c147730 0x7f570c140220 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:59.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.182+0000 7f5710efa700 1 --2- 172.21.15.17:0/224069649 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f570c0b5630 0x7f570c13fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:59.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.182+0000 7f5710efa700 1 --2- 172.21.15.17:0/224069649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f570c0b07a0 0x7f570c066de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:59.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.182+0000 7f5710efa700 1 -- 172.21.15.17:0/224069649 >> 172.21.15.17:0/224069649 conn(0x7f570c0626f0 msgr2=0x7f570c147d10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:38:59.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.182+0000 7f5710efa700 1 -- 172.21.15.17:0/224069649 shutdown_connections 2022-08-01T19:38:59.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.182+0000 7f5710efa700 1 -- 172.21.15.17:0/224069649 wait complete. 2022-08-01T19:38:59.187 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-08-01T19:38:59.200 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:38:59.220 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2022-08-01T19:38:59.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2ddeea700 1 Processor -- start 2022-08-01T19:38:59.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2ddeea700 1 -- start start 2022-08-01T19:38:59.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2ddeea700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fd2d80b1f40 0x7fd2d80b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:59.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2ddeea700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2d8148530 0x7fd2d80b2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:59.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2ddeea700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2d80b04e0 con 0x7fd2d80a7250 2022-08-01T19:38:59.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2ddeea700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2d80a9250 con 0x7fd2d80b1f40 2022-08-01T19:38:59.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2ddeea700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d80b0780 con 0x7fd2d8148530 2022-08-01T19:38:59.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2d7fff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd2d80a7250 0x7fd2d81448b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59938/0 (socket says 172.21.15.17:59938) 2022-08-01T19:38:59.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2d7fff700 1 -- 172.21.15.17:0/237452283 learned_addr learned my addr 172.21.15.17:0/237452283 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:38:59.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2d77fe700 1 --2- 172.21.15.17:0/237452283 >> v2:172.21.15.77:3300/0 conn(0x7fd2d80b1f40 0x7fd2d80b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:59.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2d6ffd700 1 --2- 172.21.15.17:0/237452283 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2d8148530 0x7fd2d80b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:59.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2d77fe700 1 -- 172.21.15.17:0/237452283 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2d8148530 msgr2=0x7fd2d80b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:38:59.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2d77fe700 1 --2- 172.21.15.17:0/237452283 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2d8148530 0x7fd2d80b2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:59.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2d77fe700 1 -- 172.21.15.17:0/237452283 >> v1:172.21.15.17:6789/0 conn(0x7fd2d80a7250 legacy=0x7fd2d81448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:59.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2d67fc700 1 -- 172.21.15.17:0/237452283 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2169402352 0 0) 0x7fd2d80b04e0 con 0x7fd2d80a7250 2022-08-01T19:38:59.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2d77fe700 1 -- 172.21.15.17:0/237452283 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d80b0a20 con 0x7fd2d80b1f40 2022-08-01T19:38:59.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2d77fe700 1 --2- 172.21.15.17:0/237452283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2d80b1f40 0x7fd2d80b1160 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7fd2c0002f70 tx=0x7fd2c0006df0).ready entity=mon.1 client_cookie=a2399ec9e0ebb656 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:38:59.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2d67fc700 1 -- 172.21.15.17:0/237452283 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2c000d570 con 0x7fd2d80b1f40 2022-08-01T19:38:59.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2d67fc700 1 -- 172.21.15.17:0/237452283 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c0000e90 con 0x7fd2d80b1f40 2022-08-01T19:38:59.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2d67fc700 1 -- 172.21.15.17:0/237452283 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2c000d8c0 con 0x7fd2d80b1f40 2022-08-01T19:38:59.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2ddeea700 1 -- 172.21.15.17:0/237452283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2d80b1f40 msgr2=0x7fd2d80b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:59.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2ddeea700 1 --2- 172.21.15.17:0/237452283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2d80b1f40 0x7fd2d80b1160 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7fd2c0002f70 tx=0x7fd2c0006df0).stop 2022-08-01T19:38:59.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2ddeea700 1 -- 172.21.15.17:0/237452283 shutdown_connections 2022-08-01T19:38:59.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2ddeea700 1 --2- 172.21.15.17:0/237452283 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2d8148530 0x7fd2d80b2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:59.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2ddeea700 1 --2- 172.21.15.17:0/237452283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2d80b1f40 0x7fd2d80b1160 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:59.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2ddeea700 1 -- 172.21.15.17:0/237452283 >> 172.21.15.17:0/237452283 conn(0x7fd2d805f840 msgr2=0x7fd2d805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:38:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2ddeea700 1 -- 172.21.15.17:0/237452283 shutdown_connections 2022-08-01T19:38:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2ddeea700 1 -- 172.21.15.17:0/237452283 wait complete. 2022-08-01T19:38:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2ddeea700 1 Processor -- start 2022-08-01T19:38:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2ddeea700 1 -- start start 2022-08-01T19:38:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2ddeea700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2d80a7250 0x7fd2d813da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2ddeea700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2d80b1f40 0x7fd2d8156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2ddeea700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2d8148530 0x7fd2d8157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2ddeea700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d814dc30 con 0x7fd2d8148530 2022-08-01T19:38:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2ddeea700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d8140da0 con 0x7fd2d80b1f40 2022-08-01T19:38:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2ddeea700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d8140a20 con 0x7fd2d80a7250 2022-08-01T19:38:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2d7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2d8148530 0x7fd2d8157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2d7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2d8148530 0x7fd2d8157440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39518/0 (socket says 172.21.15.17:39518) 2022-08-01T19:38:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.306+0000 7fd2d7fff700 1 -- 172.21.15.17:0/3228634775 learned_addr learned my addr 172.21.15.17:0/3228634775 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:38:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.310+0000 7fd2d6ffd700 1 --2- 172.21.15.17:0/3228634775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2d80b1f40 0x7fd2d8156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.310+0000 7fd2d77fe700 1 --2- 172.21.15.17:0/3228634775 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2d80a7250 0x7fd2d813da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.310+0000 7fd2d7fff700 1 -- 172.21.15.17:0/3228634775 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2d80a7250 msgr2=0x7fd2d813da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:59.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.310+0000 7fd2d7fff700 1 --2- 172.21.15.17:0/3228634775 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2d80a7250 0x7fd2d813da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:59.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.310+0000 7fd2d7fff700 1 -- 172.21.15.17:0/3228634775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2d80b1f40 msgr2=0x7fd2d8156f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:59.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.310+0000 7fd2d7fff700 1 --2- 172.21.15.17:0/3228634775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2d80b1f40 0x7fd2d8156f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:59.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.310+0000 7fd2d7fff700 1 -- 172.21.15.17:0/3228634775 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c000b180 con 0x7fd2d8148530 2022-08-01T19:38:59.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.310+0000 7fd2d7fff700 1 --2- 172.21.15.17:0/3228634775 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2d8148530 0x7fd2d8157440 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7fd2d0000ed0 tx=0x7fd2d0007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:38:59.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.310+0000 7fd2d4ff9700 1 -- 172.21.15.17:0/3228634775 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2d0007110 con 0x7fd2d8148530 2022-08-01T19:38:59.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.310+0000 7fd2ddeea700 1 -- 172.21.15.17:0/3228634775 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d813ec50 con 0x7fd2d8148530 2022-08-01T19:38:59.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.310+0000 7fd2d4ff9700 1 -- 172.21.15.17:0/3228634775 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d0007270 con 0x7fd2d8148530 2022-08-01T19:38:59.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.310+0000 7fd2d4ff9700 1 -- 172.21.15.17:0/3228634775 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2d0005030 con 0x7fd2d8148530 2022-08-01T19:38:59.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.310+0000 7fd2ddeea700 1 -- 172.21.15.17:0/3228634775 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d8156a80 con 0x7fd2d8148530 2022-08-01T19:38:59.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.310+0000 7fd2d4ff9700 1 -- 172.21.15.17:0/3228634775 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd2d0005190 con 0x7fd2d8148530 2022-08-01T19:38:59.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.310+0000 7fd2d4ff9700 1 --2- 172.21.15.17:0/3228634775 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd2bc053a70 0x7fd2bc055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:59.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.310+0000 7fd2ddeea700 1 -- 172.21.15.17:0/3228634775 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2d8156c70 con 0x7fd2d8148530 2022-08-01T19:38:59.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.310+0000 7fd2d4ff9700 1 -- 172.21.15.17:0/3228634775 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fd2d006b930 con 0x7fd2d8148530 2022-08-01T19:38:59.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.314+0000 7fd2d77fe700 1 --2- 172.21.15.17:0/3228634775 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd2bc053a70 0x7fd2bc055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:59.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.314+0000 7fd2d77fe700 1 --2- 172.21.15.17:0/3228634775 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd2bc053a70 0x7fd2bc055f30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7fd2c001c000 tx=0x7fd2c0015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:38:59.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.322+0000 7fd2d4ff9700 1 -- 172.21.15.17:0/3228634775 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd2d0042600 con 0x7fd2d8148530 2022-08-01T19:38:59.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.482+0000 7fd2ddeea700 1 -- 172.21.15.17:0/3228634775 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2d80b0cc0 con 0x7fd2d8148530 2022-08-01T19:38:59.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.486+0000 7fd2d4ff9700 1 -- 172.21.15.17:0/3228634775 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+2131 (secure 0 0 0) 0x7fd2d0045c20 con 0x7fd2d8148530 2022-08-01T19:38:59.488 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:38:59.488 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/1914125901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":1914125901},{"type":"v1","addr":"172.21.15.17:6833","nonce":1914125901}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":22},{"gid":5587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1403688289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1403688289},{"type":"v1","addr":"172.21.15.77:6835","nonce":1403688289}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":22}],"filesystems":[{"mdsmap":{"epoch":25,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:37:36.197818+0000","modified":"2022-08-01T19:38:59.171676+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-08-01T19:38:59.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.486+0000 7fd2ddeea700 1 -- 172.21.15.17:0/3228634775 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd2bc053a70 msgr2=0x7fd2bc055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:59.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.486+0000 7fd2ddeea700 1 --2- 172.21.15.17:0/3228634775 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd2bc053a70 0x7fd2bc055f30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7fd2c001c000 tx=0x7fd2c0015000).stop 2022-08-01T19:38:59.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.486+0000 7fd2ddeea700 1 -- 172.21.15.17:0/3228634775 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2d8148530 msgr2=0x7fd2d8157440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:59.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.486+0000 7fd2ddeea700 1 --2- 172.21.15.17:0/3228634775 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2d8148530 0x7fd2d8157440 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7fd2d0000ed0 tx=0x7fd2d0007a40).stop 2022-08-01T19:38:59.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.486+0000 7fd2ddeea700 1 -- 172.21.15.17:0/3228634775 shutdown_connections 2022-08-01T19:38:59.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.486+0000 7fd2ddeea700 1 --2- 172.21.15.17:0/3228634775 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd2bc053a70 0x7fd2bc055f30 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:59.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.486+0000 7fd2ddeea700 1 --2- 172.21.15.17:0/3228634775 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2d8148530 0x7fd2d8157440 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:59.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.486+0000 7fd2ddeea700 1 --2- 172.21.15.17:0/3228634775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2d80b1f40 0x7fd2d8156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:59.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.486+0000 7fd2ddeea700 1 --2- 172.21.15.17:0/3228634775 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2d80a7250 0x7fd2d813da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:59.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.486+0000 7fd2ddeea700 1 -- 172.21.15.17:0/3228634775 >> 172.21.15.17:0/3228634775 conn(0x7fd2d805f840 msgr2=0x7fd2d80af890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:38:59.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.486+0000 7fd2ddeea700 1 -- 172.21.15.17:0/3228634775 shutdown_connections 2022-08-01T19:38:59.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.486+0000 7fd2ddeea700 1 -- 172.21.15.17:0/3228634775 wait complete. 2022-08-01T19:38:59.493 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 25 2022-08-01T19:38:59.502 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-08-01T19:38:59.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.606+0000 7ff66ddb8700 1 Processor -- start 2022-08-01T19:38:59.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.606+0000 7ff66ddb8700 1 -- start start 2022-08-01T19:38:59.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff66ddb8700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff668146ae0 0x7ff6681403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:59.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff66ddb8700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6680b4820 0x7ff6680b5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:59.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff66ddb8700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff668143cf0 con 0x7ff6680b4310 2022-08-01T19:38:59.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff66ddb8700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6681476a0 con 0x7ff668146ae0 2022-08-01T19:38:59.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff66ddb8700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff668143f90 con 0x7ff6680b4820 2022-08-01T19:38:59.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff6677fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff668146ae0 0x7ff6681403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:59.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff666ffd700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff6680b4310 0x7ff6681408b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59950/0 (socket says 172.21.15.17:59950) 2022-08-01T19:38:59.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff6677fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff668146ae0 0x7ff6681403c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39522/0 (socket says 172.21.15.17:39522) 2022-08-01T19:38:59.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff666ffd700 1 -- 172.21.15.17:0/1264638036 learned_addr learned my addr 172.21.15.17:0/1264638036 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:38:59.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff667fff700 1 --2- 172.21.15.17:0/1264638036 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6680b4820 0x7ff6680b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:59.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff6677fe700 1 -- 172.21.15.17:0/1264638036 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6680b4820 msgr2=0x7ff6680b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:38:59.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff6677fe700 1 --2- 172.21.15.17:0/1264638036 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6680b4820 0x7ff6680b5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:59.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff6677fe700 1 -- 172.21.15.17:0/1264638036 >> v1:172.21.15.17:6789/0 conn(0x7ff6680b4310 legacy=0x7ff6681408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:59.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff6677fe700 1 -- 172.21.15.17:0/1264638036 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6680b0e20 con 0x7ff668146ae0 2022-08-01T19:38:59.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff667fff700 1 --2- 172.21.15.17:0/1264638036 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6680b4820 0x7ff6680b5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:38:59.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff6677fe700 1 --2- 172.21.15.17:0/1264638036 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff668146ae0 0x7ff6681403c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7ff658002f70 tx=0x7ff658006df0).ready entity=mon.0 client_cookie=c406def9ae197a2f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:38:59.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff6667fc700 1 -- 172.21.15.17:0/1264638036 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff65800d570 con 0x7ff668146ae0 2022-08-01T19:38:59.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff6667fc700 1 -- 172.21.15.17:0/1264638036 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff658000e90 con 0x7ff668146ae0 2022-08-01T19:38:59.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff6667fc700 1 -- 172.21.15.17:0/1264638036 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff65800d8c0 con 0x7ff668146ae0 2022-08-01T19:38:59.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff66ddb8700 1 -- 172.21.15.17:0/1264638036 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff668146ae0 msgr2=0x7ff6681403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:59.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff66ddb8700 1 --2- 172.21.15.17:0/1264638036 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff668146ae0 0x7ff6681403c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7ff658002f70 tx=0x7ff658006df0).stop 2022-08-01T19:38:59.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff66ddb8700 1 -- 172.21.15.17:0/1264638036 shutdown_connections 2022-08-01T19:38:59.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff66ddb8700 1 --2- 172.21.15.17:0/1264638036 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6680b4820 0x7ff6680b5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:59.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff66ddb8700 1 --2- 172.21.15.17:0/1264638036 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff668146ae0 0x7ff6681403c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:59.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff66ddb8700 1 -- 172.21.15.17:0/1264638036 >> 172.21.15.17:0/1264638036 conn(0x7ff66805c620 msgr2=0x7ff66805ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:38:59.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff66ddb8700 1 -- 172.21.15.17:0/1264638036 shutdown_connections 2022-08-01T19:38:59.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff66ddb8700 1 -- 172.21.15.17:0/1264638036 wait complete. 2022-08-01T19:38:59.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff66ddb8700 1 Processor -- start 2022-08-01T19:38:59.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff66ddb8700 1 -- start start 2022-08-01T19:38:59.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff66ddb8700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6680b4310 0x7ff6680b6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:59.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff66ddb8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6680b4820 0x7ff668152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:59.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff66ddb8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff668146ae0 0x7ff668152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:59.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff66ddb8700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff668156910 con 0x7ff668146ae0 2022-08-01T19:38:59.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff66ddb8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6680adcc0 con 0x7ff6680b4820 2022-08-01T19:38:59.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff66ddb8700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6680adf60 con 0x7ff6680b4310 2022-08-01T19:38:59.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff667fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff668146ae0 0x7ff668152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:59.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff667fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff668146ae0 0x7ff668152980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39532/0 (socket says 172.21.15.17:39532) 2022-08-01T19:38:59.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff667fff700 1 -- 172.21.15.17:0/78814248 learned_addr learned my addr 172.21.15.17:0/78814248 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:38:59.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff666ffd700 1 --2- 172.21.15.17:0/78814248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6680b4820 0x7ff668152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:59.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff6677fe700 1 --2- 172.21.15.17:0/78814248 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6680b4310 0x7ff6680b6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:59.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff667fff700 1 -- 172.21.15.17:0/78814248 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6680b4310 msgr2=0x7ff6680b6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:59.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff667fff700 1 --2- 172.21.15.17:0/78814248 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6680b4310 0x7ff6680b6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:59.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff667fff700 1 -- 172.21.15.17:0/78814248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6680b4820 msgr2=0x7ff668152490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:38:59.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff667fff700 1 --2- 172.21.15.17:0/78814248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6680b4820 0x7ff668152490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:38:59.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff667fff700 1 -- 172.21.15.17:0/78814248 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff65800b180 con 0x7ff668146ae0 2022-08-01T19:38:59.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff666ffd700 1 --2- 172.21.15.17:0/78814248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6680b4820 0x7ff668152490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:38:59.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.610+0000 7ff667fff700 1 --2- 172.21.15.17:0/78814248 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff668146ae0 0x7ff668152980 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7ff660006cc0 tx=0x7ff660008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:38:59.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.614+0000 7ff664ff9700 1 -- 172.21.15.17:0/78814248 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff66000c770 con 0x7ff668146ae0 2022-08-01T19:38:59.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.614+0000 7ff664ff9700 1 -- 172.21.15.17:0/78814248 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66000c8d0 con 0x7ff668146ae0 2022-08-01T19:38:59.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.614+0000 7ff66ddb8700 1 -- 172.21.15.17:0/78814248 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6680ae200 con 0x7ff668146ae0 2022-08-01T19:38:59.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.614+0000 7ff664ff9700 1 -- 172.21.15.17:0/78814248 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff66000cc30 con 0x7ff668146ae0 2022-08-01T19:38:59.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.614+0000 7ff66ddb8700 1 -- 172.21.15.17:0/78814248 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff668152e70 con 0x7ff668146ae0 2022-08-01T19:38:59.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.614+0000 7ff66ddb8700 1 -- 172.21.15.17:0/78814248 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6680ae390 con 0x7ff668146ae0 2022-08-01T19:38:59.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.614+0000 7ff664ff9700 1 -- 172.21.15.17:0/78814248 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff660013430 con 0x7ff668146ae0 2022-08-01T19:38:59.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.614+0000 7ff664ff9700 1 --2- 172.21.15.17:0/78814248 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff6480539e0 0x7ff648055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:38:59.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.614+0000 7ff664ff9700 1 -- 172.21.15.17:0/78814248 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6506+0+0 (secure 0 0 0) 0x7ff66006c0e0 con 0x7ff668146ae0 2022-08-01T19:38:59.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.614+0000 7ff6677fe700 1 --2- 172.21.15.17:0/78814248 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff6480539e0 0x7ff648055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:38:59.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.614+0000 7ff6677fe700 1 --2- 172.21.15.17:0/78814248 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff6480539e0 0x7ff648055ea0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7ff65801a000 tx=0x7ff658015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:38:59.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.626+0000 7ff664ff9700 1 -- 172.21.15.17:0/78814248 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff660042d50 con 0x7ff668146ae0 2022-08-01T19:38:59.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:38:59.782+0000 7ff66ddb8700 1 -- 172.21.15.17:0/78814248 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff6680574e0 con 0x7ff668146ae0 2022-08-01T19:39:00.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.262+0000 7ff664ff9700 1 -- 172.21.15.17:0/78814248 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff66000ee60 con 0x7ff668146ae0 2022-08-01T19:39:00.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.266+0000 7ff66ddb8700 1 -- 172.21.15.17:0/78814248 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff6480539e0 msgr2=0x7ff648055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:00.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.266+0000 7ff66ddb8700 1 --2- 172.21.15.17:0/78814248 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff6480539e0 0x7ff648055ea0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7ff65801a000 tx=0x7ff658015000).stop 2022-08-01T19:39:00.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.266+0000 7ff66ddb8700 1 -- 172.21.15.17:0/78814248 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff668146ae0 msgr2=0x7ff668152980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:00.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.266+0000 7ff66ddb8700 1 --2- 172.21.15.17:0/78814248 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff668146ae0 0x7ff668152980 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7ff660006cc0 tx=0x7ff660008040).stop 2022-08-01T19:39:00.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.266+0000 7ff66ddb8700 1 -- 172.21.15.17:0/78814248 shutdown_connections 2022-08-01T19:39:00.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.266+0000 7ff66ddb8700 1 --2- 172.21.15.17:0/78814248 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff6480539e0 0x7ff648055ea0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:00.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.266+0000 7ff66ddb8700 1 --2- 172.21.15.17:0/78814248 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff668146ae0 0x7ff668152980 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:00.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.266+0000 7ff66ddb8700 1 --2- 172.21.15.17:0/78814248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6680b4820 0x7ff668152490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:00.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.266+0000 7ff66ddb8700 1 --2- 172.21.15.17:0/78814248 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6680b4310 0x7ff6680b6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:00.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.266+0000 7ff66ddb8700 1 -- 172.21.15.17:0/78814248 >> 172.21.15.17:0/78814248 conn(0x7ff66805c620 msgr2=0x7ff6680b2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:00.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.266+0000 7ff66ddb8700 1 -- 172.21.15.17:0/78814248 shutdown_connections 2022-08-01T19:39:00.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.266+0000 7ff66ddb8700 1 -- 172.21.15.17:0/78814248 wait complete. 2022-08-01T19:39:00.281 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-08-01T19:39:00.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.398+0000 7ff966fc8700 1 Processor -- start 2022-08-01T19:39:00.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.398+0000 7ff966fc8700 1 -- start start 2022-08-01T19:39:00.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff966fc8700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ff960064b10 0x7ff960144e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:00.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff966fc8700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff9600b36e0 0x7ff960141370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:00.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff966fc8700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff960141860 0x7ff960066ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:00.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff966fc8700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff96013ebf0 con 0x7ff960141860 2022-08-01T19:39:00.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff966fc8700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff960064ef0 con 0x7ff960064b10 2022-08-01T19:39:00.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff966fc8700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff960066690 con 0x7ff9600b36e0 2022-08-01T19:39:00.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff9667c7700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff960141860 0x7ff960066ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:00.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff9667c7700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff960141860 0x7ff960066ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39540/0 (socket says 172.21.15.17:39540) 2022-08-01T19:39:00.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff9667c7700 1 -- 172.21.15.17:0/4089359960 learned_addr learned my addr 172.21.15.17:0/4089359960 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:00.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff9657c5700 1 --2- 172.21.15.17:0/4089359960 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff9600b36e0 0x7ff960141370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:00.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff965fc6700 1 --2- 172.21.15.17:0/4089359960 >> v2:172.21.15.77:3300/0 conn(0x7ff960064b10 0x7ff960144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:00.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff9667c7700 1 -- 172.21.15.17:0/4089359960 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff9600b36e0 msgr2=0x7ff960141370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:00.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff9667c7700 1 --2- 172.21.15.17:0/4089359960 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff9600b36e0 0x7ff960141370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:00.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff9667c7700 1 -- 172.21.15.17:0/4089359960 >> v2:172.21.15.77:3300/0 conn(0x7ff960064b10 msgr2=0x7ff960144e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:00.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff9667c7700 1 --2- 172.21.15.17:0/4089359960 >> v2:172.21.15.77:3300/0 conn(0x7ff960064b10 0x7ff960144e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:00.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff9667c7700 1 -- 172.21.15.17:0/4089359960 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9600b4850 con 0x7ff960141860 2022-08-01T19:39:00.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff9667c7700 1 --2- 172.21.15.17:0/4089359960 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff960141860 0x7ff960066ac0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7ff958002f70 tx=0x7ff958006eb0).ready entity=mon.0 client_cookie=8bfd60141f6a5ceb server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:00.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff964fc4700 1 -- 172.21.15.17:0/4089359960 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff95800d570 con 0x7ff960141860 2022-08-01T19:39:00.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff964fc4700 1 -- 172.21.15.17:0/4089359960 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff958000e90 con 0x7ff960141860 2022-08-01T19:39:00.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff964fc4700 1 -- 172.21.15.17:0/4089359960 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff95800d900 con 0x7ff960141860 2022-08-01T19:39:00.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff966fc8700 1 -- 172.21.15.17:0/4089359960 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff960141860 msgr2=0x7ff960066ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:00.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff966fc8700 1 --2- 172.21.15.17:0/4089359960 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff960141860 0x7ff960066ac0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7ff958002f70 tx=0x7ff958006eb0).stop 2022-08-01T19:39:00.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff966fc8700 1 -- 172.21.15.17:0/4089359960 shutdown_connections 2022-08-01T19:39:00.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff966fc8700 1 --2- 172.21.15.17:0/4089359960 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff960141860 0x7ff960066ac0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:00.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff966fc8700 1 --2- 172.21.15.17:0/4089359960 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff9600b36e0 0x7ff960141370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:00.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff966fc8700 1 --2- 172.21.15.17:0/4089359960 >> v2:172.21.15.77:3300/0 conn(0x7ff960064b10 0x7ff960144e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:00.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff966fc8700 1 -- 172.21.15.17:0/4089359960 >> 172.21.15.17:0/4089359960 conn(0x7ff96005f840 msgr2=0x7ff96005fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:00.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff966fc8700 1 -- 172.21.15.17:0/4089359960 shutdown_connections 2022-08-01T19:39:00.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff966fc8700 1 -- 172.21.15.17:0/4089359960 wait complete. 2022-08-01T19:39:00.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff966fc8700 1 Processor -- start 2022-08-01T19:39:00.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff966fc8700 1 -- start start 2022-08-01T19:39:00.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff966fc8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff960064b10 0x7ff9600b2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:00.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff966fc8700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff9600b36e0 0x7ff9600ae860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:00.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff966fc8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff960141860 0x7ff96013df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:00.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff966fc8700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96013b440 con 0x7ff960141860 2022-08-01T19:39:00.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff966fc8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96013b6a0 con 0x7ff960064b10 2022-08-01T19:39:00.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff966fc8700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff96015a810 con 0x7ff9600b36e0 2022-08-01T19:39:00.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff9667c7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff960141860 0x7ff96013df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:00.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff9667c7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff960141860 0x7ff96013df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39546/0 (socket says 172.21.15.17:39546) 2022-08-01T19:39:00.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff965fc6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff960064b10 0x7ff9600b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:00.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff9667c7700 1 -- 172.21.15.17:0/123658639 learned_addr learned my addr 172.21.15.17:0/123658639 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:00.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff9657c5700 1 --2- 172.21.15.17:0/123658639 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff9600b36e0 0x7ff9600ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:00.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff965fc6700 1 -- 172.21.15.17:0/123658639 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff9600b36e0 msgr2=0x7ff9600ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:00.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff965fc6700 1 --2- 172.21.15.17:0/123658639 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff9600b36e0 0x7ff9600ae860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:00.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff965fc6700 1 -- 172.21.15.17:0/123658639 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff960141860 msgr2=0x7ff96013df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:00.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff965fc6700 1 --2- 172.21.15.17:0/123658639 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff960141860 0x7ff96013df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:00.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff965fc6700 1 -- 172.21.15.17:0/123658639 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff95800b1e0 con 0x7ff960064b10 2022-08-01T19:39:00.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff965fc6700 1 --2- 172.21.15.17:0/123658639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff960064b10 0x7ff9600b2180 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7ff95000c000 tx=0x7ff950006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:00.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff94effd700 1 -- 172.21.15.17:0/123658639 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff950010780 con 0x7ff960064b10 2022-08-01T19:39:00.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff966fc8700 1 -- 172.21.15.17:0/123658639 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff96015aab0 con 0x7ff960064b10 2022-08-01T19:39:00.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff94effd700 1 -- 172.21.15.17:0/123658639 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9500108e0 con 0x7ff960064b10 2022-08-01T19:39:00.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff966fc8700 1 -- 172.21.15.17:0/123658639 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9600a7670 con 0x7ff960064b10 2022-08-01T19:39:00.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff94effd700 1 -- 172.21.15.17:0/123658639 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff950010c40 con 0x7ff960064b10 2022-08-01T19:39:00.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff966fc8700 1 -- 172.21.15.17:0/123658639 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9600b4af0 con 0x7ff960064b10 2022-08-01T19:39:00.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.402+0000 7ff94effd700 1 -- 172.21.15.17:0/123658639 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff9500123f0 con 0x7ff960064b10 2022-08-01T19:39:00.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.406+0000 7ff94effd700 1 --2- 172.21.15.17:0/123658639 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff940053950 0x7ff940055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:00.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.406+0000 7ff94effd700 1 -- 172.21.15.17:0/123658639 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6506+0+0 (secure 0 0 0) 0x7ff95006c5f0 con 0x7ff960064b10 2022-08-01T19:39:00.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.406+0000 7ff9657c5700 1 --2- 172.21.15.17:0/123658639 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff940053950 0x7ff940055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:00.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.406+0000 7ff9657c5700 1 --2- 172.21.15.17:0/123658639 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff940053950 0x7ff940055e10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7ff95c00b000 tx=0x7ff95c0091d0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:00.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.418+0000 7ff94effd700 1 -- 172.21.15.17:0/123658639 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff950043170 con 0x7ff960064b10 2022-08-01T19:39:00.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:00.542+0000 7ff966fc8700 1 -- 172.21.15.17:0/123658639 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff9600574e0 con 0x7ff960064b10 2022-08-01T19:39:01.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.274+0000 7ff94effd700 1 -- 172.21.15.17:0/123658639 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v57) v1 ==== 185+0+0 (secure 0 0 0) 0x7ff950046790 con 0x7ff960064b10 2022-08-01T19:39:01.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.278+0000 7ff966fc8700 1 -- 172.21.15.17:0/123658639 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff940053950 msgr2=0x7ff940055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:01.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.278+0000 7ff966fc8700 1 --2- 172.21.15.17:0/123658639 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff940053950 0x7ff940055e10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7ff95c00b000 tx=0x7ff95c0091d0).stop 2022-08-01T19:39:01.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.278+0000 7ff966fc8700 1 -- 172.21.15.17:0/123658639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff960064b10 msgr2=0x7ff9600b2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:01.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.278+0000 7ff966fc8700 1 --2- 172.21.15.17:0/123658639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff960064b10 0x7ff9600b2180 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7ff95000c000 tx=0x7ff950006df0).stop 2022-08-01T19:39:01.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.278+0000 7ff966fc8700 1 -- 172.21.15.17:0/123658639 shutdown_connections 2022-08-01T19:39:01.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.278+0000 7ff966fc8700 1 --2- 172.21.15.17:0/123658639 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff940053950 0x7ff940055e10 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:01.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.278+0000 7ff966fc8700 1 --2- 172.21.15.17:0/123658639 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff960141860 0x7ff96013df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:01.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.278+0000 7ff966fc8700 1 --2- 172.21.15.17:0/123658639 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff9600b36e0 0x7ff9600ae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:01.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.278+0000 7ff966fc8700 1 --2- 172.21.15.17:0/123658639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff960064b10 0x7ff9600b2180 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:01.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.278+0000 7ff966fc8700 1 -- 172.21.15.17:0/123658639 >> 172.21.15.17:0/123658639 conn(0x7ff96005f840 msgr2=0x7ff96013ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:01.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.278+0000 7ff966fc8700 1 -- 172.21.15.17:0/123658639 shutdown_connections 2022-08-01T19:39:01.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.278+0000 7ff966fc8700 1 -- 172.21.15.17:0/123658639 wait complete. 2022-08-01T19:39:01.288 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' removed 2022-08-01T19:39:01.298 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-08-01T19:39:01.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.406+0000 7fe720fa3700 1 Processor -- start 2022-08-01T19:39:01.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.406+0000 7fe720fa3700 1 -- start start 2022-08-01T19:39:01.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.410+0000 7fe720fa3700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe71c0b0630 0x7fe71c0adec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:01.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.410+0000 7fe720fa3700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe71c0b2c80 con 0x7fe71c0aa2b0 2022-08-01T19:39:01.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.410+0000 7fe720fa3700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe71c0b2900 con 0x7fe71c0b0a10 2022-08-01T19:39:01.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.410+0000 7fe720fa3700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c0b2eb0 con 0x7fe71c0b0630 2022-08-01T19:39:01.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.410+0000 7fe713fff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe71c0aa2b0 0x7fe71c0ae3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59976/0 (socket says 172.21.15.17:59976) 2022-08-01T19:39:01.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.410+0000 7fe713fff700 1 -- 172.21.15.17:0/3114306186 learned_addr learned my addr 172.21.15.17:0/3114306186 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:01.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.410+0000 7fe71a59c700 1 --2- 172.21.15.17:0/3114306186 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe71c0b0630 0x7fe71c0adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:01.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.410+0000 7fe719d9b700 1 -- 172.21.15.17:0/3114306186 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2194379366 0 0) 0x7fe71c0b2c80 con 0x7fe71c0aa2b0 2022-08-01T19:39:01.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.410+0000 7fe719d9b700 1 -- 172.21.15.17:0/3114306186 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6fc003260 con 0x7fe71c0aa2b0 2022-08-01T19:39:01.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.410+0000 7fe719d9b700 1 -- 172.21.15.17:0/3114306186 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1506175121 0 0) 0x7fe71c0b2900 con 0x7fe71c0b0a10 2022-08-01T19:39:01.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.410+0000 7fe719d9b700 1 -- 172.21.15.17:0/3114306186 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe71c0b2c80 con 0x7fe71c0b0a10 2022-08-01T19:39:01.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.410+0000 7fe71a59c700 1 -- 172.21.15.17:0/3114306186 >> v1:172.21.15.77:6789/0 conn(0x7fe71c0b0a10 legacy=0x7fe71c140730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:01.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.410+0000 7fe719d9b700 1 -- 172.21.15.17:0/3114306186 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 107234674 0 0) 0x7fe6fc003260 con 0x7fe71c0aa2b0 2022-08-01T19:39:01.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.410+0000 7fe71a59c700 1 -- 172.21.15.17:0/3114306186 >> v1:172.21.15.17:6789/0 conn(0x7fe71c0aa2b0 legacy=0x7fe71c0ae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:01.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.410+0000 7fe71a59c700 1 -- 172.21.15.17:0/3114306186 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe71c0a9d00 con 0x7fe71c0b0630 2022-08-01T19:39:01.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.410+0000 7fe71a59c700 1 --2- 172.21.15.17:0/3114306186 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe71c0b0630 0x7fe71c0adec0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fe704001800 tx=0x7fe70c006df0).ready entity=mon.2 client_cookie=6ed463d6b8915b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:01.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.410+0000 7fe719d9b700 1 -- 172.21.15.17:0/3114306186 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe704001660 con 0x7fe71c0b0630 2022-08-01T19:39:01.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.410+0000 7fe719d9b700 1 -- 172.21.15.17:0/3114306186 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe70c000e90 con 0x7fe71c0b0630 2022-08-01T19:39:01.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.410+0000 7fe719d9b700 1 -- 172.21.15.17:0/3114306186 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe70c012460 con 0x7fe71c0b0630 2022-08-01T19:39:01.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.414+0000 7fe720fa3700 1 -- 172.21.15.17:0/3114306186 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe71c0b0630 msgr2=0x7fe71c0adec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:01.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.414+0000 7fe720fa3700 1 --2- 172.21.15.17:0/3114306186 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe71c0b0630 0x7fe71c0adec0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fe704001800 tx=0x7fe70c006df0).stop 2022-08-01T19:39:01.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.414+0000 7fe720fa3700 1 -- 172.21.15.17:0/3114306186 shutdown_connections 2022-08-01T19:39:01.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.414+0000 7fe720fa3700 1 --2- 172.21.15.17:0/3114306186 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe71c0b0630 0x7fe71c0adec0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:01.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.414+0000 7fe720fa3700 1 -- 172.21.15.17:0/3114306186 >> 172.21.15.17:0/3114306186 conn(0x7fe71c060950 msgr2=0x7fe71c060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:01.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.414+0000 7fe720fa3700 1 -- 172.21.15.17:0/3114306186 shutdown_connections 2022-08-01T19:39:01.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.414+0000 7fe720fa3700 1 -- 172.21.15.17:0/3114306186 wait complete. 2022-08-01T19:39:01.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.414+0000 7fe720fa3700 1 Processor -- start 2022-08-01T19:39:01.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.414+0000 7fe720fa3700 1 -- start start 2022-08-01T19:39:01.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.418+0000 7fe720fa3700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe71c0aa2b0 0x7fe71c0b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:01.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.418+0000 7fe720fa3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe71c0b0630 0x7fe71c145780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:01.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.418+0000 7fe720fa3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe71c0b0a10 0x7fe71c145c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:01.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.418+0000 7fe720fa3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c14f820 con 0x7fe71c0b0630 2022-08-01T19:39:01.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.418+0000 7fe720fa3700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c14fac0 con 0x7fe71c0b0a10 2022-08-01T19:39:01.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.418+0000 7fe720fa3700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c14fd60 con 0x7fe71c0aa2b0 2022-08-01T19:39:01.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.418+0000 7fe713fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe71c0b0630 0x7fe71c145780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:01.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.418+0000 7fe71a59c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe71c0aa2b0 0x7fe71c0b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:01.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.418+0000 7fe713fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe71c0b0630 0x7fe71c145780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39558/0 (socket says 172.21.15.17:39558) 2022-08-01T19:39:01.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.418+0000 7fe71a59c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe71c0aa2b0 0x7fe71c0b4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:35656/0 (socket says 172.21.15.17:35656) 2022-08-01T19:39:01.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.418+0000 7fe713fff700 1 -- 172.21.15.17:0/2188825311 learned_addr learned my addr 172.21.15.17:0/2188825311 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:01.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.418+0000 7fe71ad9d700 1 --2- 172.21.15.17:0/2188825311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe71c0b0a10 0x7fe71c145c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:01.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.418+0000 7fe713fff700 1 -- 172.21.15.17:0/2188825311 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe71c0aa2b0 msgr2=0x7fe71c0b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:01.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.418+0000 7fe713fff700 1 --2- 172.21.15.17:0/2188825311 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe71c0aa2b0 0x7fe71c0b4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:01.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.418+0000 7fe713fff700 1 -- 172.21.15.17:0/2188825311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe71c0b0a10 msgr2=0x7fe71c145c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:01.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.418+0000 7fe713fff700 1 --2- 172.21.15.17:0/2188825311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe71c0b0a10 0x7fe71c145c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:01.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.418+0000 7fe713fff700 1 -- 172.21.15.17:0/2188825311 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe70c00b180 con 0x7fe71c0b0630 2022-08-01T19:39:01.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.418+0000 7fe713fff700 1 --2- 172.21.15.17:0/2188825311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe71c0b0630 0x7fe71c145780 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7fe704000fe0 tx=0x7fe7040019e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:01.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.418+0000 7fe7137fe700 1 -- 172.21.15.17:0/2188825311 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe704007360 con 0x7fe71c0b0630 2022-08-01T19:39:01.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.418+0000 7fe7137fe700 1 -- 172.21.15.17:0/2188825311 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe704004800 con 0x7fe71c0b0630 2022-08-01T19:39:01.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.418+0000 7fe720fa3700 1 -- 172.21.15.17:0/2188825311 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe71c150000 con 0x7fe71c0b0630 2022-08-01T19:39:01.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.418+0000 7fe7137fe700 1 -- 172.21.15.17:0/2188825311 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe704004b60 con 0x7fe71c0b0630 2022-08-01T19:39:01.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.418+0000 7fe720fa3700 1 -- 172.21.15.17:0/2188825311 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe71c146160 con 0x7fe71c0b0630 2022-08-01T19:39:01.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.418+0000 7fe720fa3700 1 -- 172.21.15.17:0/2188825311 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe71c150190 con 0x7fe71c0b0630 2022-08-01T19:39:01.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.422+0000 7fe7137fe700 1 -- 172.21.15.17:0/2188825311 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe704019030 con 0x7fe71c0b0630 2022-08-01T19:39:01.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.422+0000 7fe7137fe700 1 --2- 172.21.15.17:0/2188825311 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe700053b30 0x7fe700055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:01.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.422+0000 7fe7137fe700 1 -- 172.21.15.17:0/2188825311 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6068+0+0 (secure 0 0 0) 0x7fe70406dea0 con 0x7fe71c0b0630 2022-08-01T19:39:01.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.422+0000 7fe71a59c700 1 --2- 172.21.15.17:0/2188825311 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe700053b30 0x7fe700055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:01.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.422+0000 7fe71a59c700 1 --2- 172.21.15.17:0/2188825311 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe700053b30 0x7fe700055ff0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7fe70c01a000 tx=0x7fe70c014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:01.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.434+0000 7fe7137fe700 1 -- 172.21.15.17:0/2188825311 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe704044d60 con 0x7fe71c0b0630 2022-08-01T19:39:01.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:01.554+0000 7fe720fa3700 1 -- 172.21.15.17:0/2188825311 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe71c0574e0 con 0x7fe71c0b0630 2022-08-01T19:39:02.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.274+0000 7fe7137fe700 1 -- 172.21.15.17:0/2188825311 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v58) v1 ==== 173+0+0 (secure 0 0 0) 0x7fe704048380 con 0x7fe71c0b0630 2022-08-01T19:39:02.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.278+0000 7fe720fa3700 1 -- 172.21.15.17:0/2188825311 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe700053b30 msgr2=0x7fe700055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:02.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.278+0000 7fe720fa3700 1 --2- 172.21.15.17:0/2188825311 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe700053b30 0x7fe700055ff0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7fe70c01a000 tx=0x7fe70c014000).stop 2022-08-01T19:39:02.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.278+0000 7fe720fa3700 1 -- 172.21.15.17:0/2188825311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe71c0b0630 msgr2=0x7fe71c145780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:02.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.278+0000 7fe720fa3700 1 --2- 172.21.15.17:0/2188825311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe71c0b0630 0x7fe71c145780 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7fe704000fe0 tx=0x7fe7040019e0).stop 2022-08-01T19:39:02.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.278+0000 7fe720fa3700 1 -- 172.21.15.17:0/2188825311 shutdown_connections 2022-08-01T19:39:02.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.278+0000 7fe720fa3700 1 --2- 172.21.15.17:0/2188825311 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe700053b30 0x7fe700055ff0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:02.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.278+0000 7fe720fa3700 1 --2- 172.21.15.17:0/2188825311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe71c0b0a10 0x7fe71c145c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:02.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.278+0000 7fe720fa3700 1 --2- 172.21.15.17:0/2188825311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe71c0b0630 0x7fe71c145780 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:02.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.278+0000 7fe720fa3700 1 --2- 172.21.15.17:0/2188825311 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe71c0aa2b0 0x7fe71c0b4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:02.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.278+0000 7fe720fa3700 1 -- 172.21.15.17:0/2188825311 >> 172.21.15.17:0/2188825311 conn(0x7fe71c060950 msgr2=0x7fe71c13f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:02.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.278+0000 7fe720fa3700 1 -- 172.21.15.17:0/2188825311 shutdown_connections 2022-08-01T19:39:02.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.278+0000 7fe720fa3700 1 -- 172.21.15.17:0/2188825311 wait complete. 2022-08-01T19:39:02.285 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' removed 2022-08-01T19:39:02.296 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-08-01T19:39:02.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.410+0000 7f0ef3472700 1 Processor -- start 2022-08-01T19:39:02.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.410+0000 7f0ef3472700 1 -- start start 2022-08-01T19:39:02.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.410+0000 7f0ef3472700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0eec0b2730 0x7f0eec0b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:02.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.410+0000 7f0ef3472700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eec05c620 con 0x7f0eec0ada20 2022-08-01T19:39:02.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.410+0000 7f0ef3472700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eec05c850 con 0x7f0eec143d00 2022-08-01T19:39:02.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.410+0000 7f0ef3472700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eec0ae440 con 0x7f0eec0b2730 2022-08-01T19:39:02.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.410+0000 7f0ef1c6f700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f0eec0ada20 0x7f0eec13fef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59990/0 (socket says 172.21.15.17:59990) 2022-08-01T19:39:02.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.410+0000 7f0ef2470700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0eec0b2730 0x7f0eec0b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:02.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.410+0000 7f0ef1c6f700 1 -- 172.21.15.17:0/3147617107 learned_addr learned my addr 172.21.15.17:0/3147617107 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:02.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.410+0000 7f0ef146e700 1 -- 172.21.15.17:0/3147617107 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 35290718 0 0) 0x7f0eec05c620 con 0x7f0eec0ada20 2022-08-01T19:39:02.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.410+0000 7f0ef146e700 1 -- 172.21.15.17:0/3147617107 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ed4003260 con 0x7f0eec0ada20 2022-08-01T19:39:02.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.410+0000 7f0ef2470700 1 -- 172.21.15.17:0/3147617107 >> v1:172.21.15.77:6789/0 conn(0x7f0eec143d00 legacy=0x7f0eec0a8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:02.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.410+0000 7f0ef146e700 1 -- 172.21.15.17:0/3147617107 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4205452165 0 0) 0x7f0ed4003260 con 0x7f0eec0ada20 2022-08-01T19:39:02.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.410+0000 7f0ef2470700 1 -- 172.21.15.17:0/3147617107 >> v1:172.21.15.17:6789/0 conn(0x7f0eec0ada20 legacy=0x7f0eec13fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:02.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.410+0000 7f0ef2470700 1 -- 172.21.15.17:0/3147617107 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eec1435c0 con 0x7f0eec0b2730 2022-08-01T19:39:02.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.410+0000 7f0ef2470700 1 --2- 172.21.15.17:0/3147617107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0eec0b2730 0x7f0eec0b1fe0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f0ee8001800 tx=0x7f0edc006df0).ready entity=mon.0 client_cookie=bf035f34f0b47571 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:02.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.410+0000 7f0ef146e700 1 -- 172.21.15.17:0/3147617107 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ee8001660 con 0x7f0eec0b2730 2022-08-01T19:39:02.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.410+0000 7f0ef146e700 1 -- 172.21.15.17:0/3147617107 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0edc000e90 con 0x7f0eec0b2730 2022-08-01T19:39:02.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.410+0000 7f0ef146e700 1 -- 172.21.15.17:0/3147617107 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0edc00d4f0 con 0x7f0eec0b2730 2022-08-01T19:39:02.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.410+0000 7f0ef3472700 1 -- 172.21.15.17:0/3147617107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0eec0b2730 msgr2=0x7f0eec0b1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:02.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.410+0000 7f0ef3472700 1 --2- 172.21.15.17:0/3147617107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0eec0b2730 0x7f0eec0b1fe0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f0ee8001800 tx=0x7f0edc006df0).stop 2022-08-01T19:39:02.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.410+0000 7f0ef3472700 1 -- 172.21.15.17:0/3147617107 shutdown_connections 2022-08-01T19:39:02.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.410+0000 7f0ef3472700 1 --2- 172.21.15.17:0/3147617107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0eec0b2730 0x7f0eec0b1fe0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:02.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.410+0000 7f0ef3472700 1 -- 172.21.15.17:0/3147617107 >> 172.21.15.17:0/3147617107 conn(0x7f0eec061360 msgr2=0x7f0eec062f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:02.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.410+0000 7f0ef3472700 1 -- 172.21.15.17:0/3147617107 shutdown_connections 2022-08-01T19:39:02.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.410+0000 7f0ef3472700 1 -- 172.21.15.17:0/3147617107 wait complete. 2022-08-01T19:39:02.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.410+0000 7f0ef3472700 1 Processor -- start 2022-08-01T19:39:02.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.410+0000 7f0ef3472700 1 -- start start 2022-08-01T19:39:02.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.414+0000 7f0ef3472700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0eec0ada20 0x7f0eec141d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:02.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.414+0000 7f0ef3472700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0eec0b2730 0x7f0eec13ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:02.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.414+0000 7f0ef3472700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0eec143d00 0x7f0eec13d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:02.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.414+0000 7f0ef3472700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec142740 con 0x7f0eec0b2730 2022-08-01T19:39:02.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.414+0000 7f0ef3472700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec1429e0 con 0x7f0eec0ada20 2022-08-01T19:39:02.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.414+0000 7f0ef3472700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec142c80 con 0x7f0eec143d00 2022-08-01T19:39:02.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.414+0000 7f0ef1c6f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0eec0b2730 0x7f0eec13ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:02.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.414+0000 7f0ef1c6f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0eec0b2730 0x7f0eec13ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39572/0 (socket says 172.21.15.17:39572) 2022-08-01T19:39:02.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.414+0000 7f0ef1c6f700 1 -- 172.21.15.17:0/556009489 learned_addr learned my addr 172.21.15.17:0/556009489 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:02.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.414+0000 7f0ef2470700 1 --2- 172.21.15.17:0/556009489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0eec0ada20 0x7f0eec141d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:02.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.414+0000 7f0ef2c71700 1 --2- 172.21.15.17:0/556009489 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0eec143d00 0x7f0eec13d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:02.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.414+0000 7f0ef1c6f700 1 -- 172.21.15.17:0/556009489 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0eec143d00 msgr2=0x7f0eec13d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:02.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.414+0000 7f0ef1c6f700 1 --2- 172.21.15.17:0/556009489 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0eec143d00 0x7f0eec13d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:02.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.414+0000 7f0ef1c6f700 1 -- 172.21.15.17:0/556009489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0eec0ada20 msgr2=0x7f0eec141d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:02.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.414+0000 7f0ef1c6f700 1 --2- 172.21.15.17:0/556009489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0eec0ada20 0x7f0eec141d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:02.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.414+0000 7f0ef1c6f700 1 -- 172.21.15.17:0/556009489 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0edc00b180 con 0x7f0eec0b2730 2022-08-01T19:39:02.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.414+0000 7f0ef1c6f700 1 --2- 172.21.15.17:0/556009489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0eec0b2730 0x7f0eec13ce00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f0ee80018c0 tx=0x7f0ee8001d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:02.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.414+0000 7f0edb7fe700 1 -- 172.21.15.17:0/556009489 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ee800ce50 con 0x7f0eec0b2730 2022-08-01T19:39:02.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.414+0000 7f0edb7fe700 1 -- 172.21.15.17:0/556009489 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ee80072e0 con 0x7f0eec0b2730 2022-08-01T19:39:02.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.414+0000 7f0ef3472700 1 -- 172.21.15.17:0/556009489 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eec142f20 con 0x7f0eec0b2730 2022-08-01T19:39:02.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.414+0000 7f0edb7fe700 1 -- 172.21.15.17:0/556009489 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ee8005800 con 0x7f0eec0b2730 2022-08-01T19:39:02.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.414+0000 7f0ef3472700 1 -- 172.21.15.17:0/556009489 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eec1431b0 con 0x7f0eec0b2730 2022-08-01T19:39:02.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.414+0000 7f0ef3472700 1 -- 172.21.15.17:0/556009489 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eec143860 con 0x7f0eec0b2730 2022-08-01T19:39:02.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.418+0000 7f0edb7fe700 1 -- 172.21.15.17:0/556009489 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0ee8010070 con 0x7f0eec0b2730 2022-08-01T19:39:02.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.418+0000 7f0edb7fe700 1 --2- 172.21.15.17:0/556009489 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0ecc053aa0 0x7f0ecc055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:02.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.418+0000 7f0edb7fe700 1 -- 172.21.15.17:0/556009489 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f0ee806b470 con 0x7f0eec0b2730 2022-08-01T19:39:02.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.418+0000 7f0ef2470700 1 --2- 172.21.15.17:0/556009489 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0ecc053aa0 0x7f0ecc055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:02.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.418+0000 7f0ef2470700 1 --2- 172.21.15.17:0/556009489 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0ecc053aa0 0x7f0ecc055f60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f0edc01a000 tx=0x7f0edc014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:02.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.422+0000 7f0edb7fe700 1 -- 172.21.15.17:0/556009489 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0ee8042510 con 0x7f0eec0b2730 2022-08-01T19:39:02.547 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.546+0000 7f0ef3472700 1 -- 172.21.15.17:0/556009489 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0eec0574e0 con 0x7f0eec0b2730 2022-08-01T19:39:02.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.550+0000 7f0edb7fe700 1 -- 172.21.15.17:0/556009489 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v7) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0ee8045b30 con 0x7f0eec0b2730 2022-08-01T19:39:02.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.550+0000 7f0ef3472700 1 -- 172.21.15.17:0/556009489 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0ecc053aa0 msgr2=0x7f0ecc055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:02.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.550+0000 7f0ef3472700 1 --2- 172.21.15.17:0/556009489 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0ecc053aa0 0x7f0ecc055f60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f0edc01a000 tx=0x7f0edc014000).stop 2022-08-01T19:39:02.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.550+0000 7f0ef3472700 1 -- 172.21.15.17:0/556009489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0eec0b2730 msgr2=0x7f0eec13ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:02.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.550+0000 7f0ef3472700 1 --2- 172.21.15.17:0/556009489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0eec0b2730 0x7f0eec13ce00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f0ee80018c0 tx=0x7f0ee8001d80).stop 2022-08-01T19:39:02.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.550+0000 7f0ef3472700 1 -- 172.21.15.17:0/556009489 shutdown_connections 2022-08-01T19:39:02.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.550+0000 7f0ef3472700 1 --2- 172.21.15.17:0/556009489 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0ecc053aa0 0x7f0ecc055f60 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:02.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.550+0000 7f0ef3472700 1 --2- 172.21.15.17:0/556009489 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0eec143d00 0x7f0eec13d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:02.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.550+0000 7f0ef3472700 1 --2- 172.21.15.17:0/556009489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0eec0b2730 0x7f0eec13ce00 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:02.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.550+0000 7f0ef3472700 1 --2- 172.21.15.17:0/556009489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0eec0ada20 0x7f0eec141d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:02.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.550+0000 7f0ef3472700 1 -- 172.21.15.17:0/556009489 >> 172.21.15.17:0/556009489 conn(0x7f0eec061360 msgr2=0x7f0eec149200 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:02.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.550+0000 7f0ef3472700 1 -- 172.21.15.17:0/556009489 shutdown_connections 2022-08-01T19:39:02.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.554+0000 7f0ef3472700 1 -- 172.21.15.17:0/556009489 wait complete. 2022-08-01T19:39:02.567 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout' 2022-08-01T19:39:02.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9ee12b0700 1 Processor -- start 2022-08-01T19:39:02.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9ee12b0700 1 -- start start 2022-08-01T19:39:02.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9ee12b0700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9edc0b7840 0x7f9edc060be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:02.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9ee12b0700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9edc0b2930 0x7f9edc0610d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:02.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9ee12b0700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f9edc146720 0x7f9edc146b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:02.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9ee12b0700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9edc0671e0 con 0x7f9edc0b7840 2022-08-01T19:39:02.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9ee12b0700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9edc065540 con 0x7f9edc146720 2022-08-01T19:39:02.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9ee12b0700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc0b7c20 con 0x7f9edc0b2930 2022-08-01T19:39:02.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9edbfff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9edc0b7840 0x7f9edc060be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:02.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9edbfff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9edc0b7840 0x7f9edc060be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39578/0 (socket says 172.21.15.17:39578) 2022-08-01T19:39:02.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9edbfff700 1 -- 172.21.15.17:0/3351086625 learned_addr learned my addr 172.21.15.17:0/3351086625 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:02.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9edb7fe700 1 --2- 172.21.15.17:0/3351086625 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9edc0b2930 0x7f9edc0610d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:02.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9ee0aaf700 1 --2- 172.21.15.17:0/3351086625 >> v2:172.21.15.77:3300/0 conn(0x7f9edc146720 0x7f9edc146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:02.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9edbfff700 1 -- 172.21.15.17:0/3351086625 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9edc0b2930 msgr2=0x7f9edc0610d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:02.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9edbfff700 1 --2- 172.21.15.17:0/3351086625 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9edc0b2930 0x7f9edc0610d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:02.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9edbfff700 1 -- 172.21.15.17:0/3351086625 >> v2:172.21.15.77:3300/0 conn(0x7f9edc146720 msgr2=0x7f9edc146b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:02.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9edbfff700 1 --2- 172.21.15.17:0/3351086625 >> v2:172.21.15.77:3300/0 conn(0x7f9edc146720 0x7f9edc146b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:02.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9edbfff700 1 -- 172.21.15.17:0/3351086625 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9edc066ce0 con 0x7f9edc0b7840 2022-08-01T19:39:02.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9edbfff700 1 --2- 172.21.15.17:0/3351086625 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9edc0b7840 0x7f9edc060be0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f9ed40032f0 tx=0x7f9ed4009250).ready entity=mon.0 client_cookie=907cf656b62fd841 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:02.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9edaffd700 1 -- 172.21.15.17:0/3351086625 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ed400b870 con 0x7f9edc0b7840 2022-08-01T19:39:02.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9edaffd700 1 -- 172.21.15.17:0/3351086625 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed400b9d0 con 0x7f9edc0b7840 2022-08-01T19:39:02.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9edaffd700 1 -- 172.21.15.17:0/3351086625 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ed400bb80 con 0x7f9edc0b7840 2022-08-01T19:39:02.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9ee12b0700 1 -- 172.21.15.17:0/3351086625 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9edc0b7840 msgr2=0x7f9edc060be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:02.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9ee12b0700 1 --2- 172.21.15.17:0/3351086625 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9edc0b7840 0x7f9edc060be0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f9ed40032f0 tx=0x7f9ed4009250).stop 2022-08-01T19:39:02.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9ee12b0700 1 -- 172.21.15.17:0/3351086625 shutdown_connections 2022-08-01T19:39:02.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9ee12b0700 1 --2- 172.21.15.17:0/3351086625 >> v2:172.21.15.77:3300/0 conn(0x7f9edc146720 0x7f9edc146b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:02.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9ee12b0700 1 --2- 172.21.15.17:0/3351086625 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9edc0b2930 0x7f9edc0610d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:02.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9ee12b0700 1 --2- 172.21.15.17:0/3351086625 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9edc0b7840 0x7f9edc060be0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:02.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9ee12b0700 1 -- 172.21.15.17:0/3351086625 >> 172.21.15.17:0/3351086625 conn(0x7f9edc05f840 msgr2=0x7f9edc05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:02.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9ee12b0700 1 -- 172.21.15.17:0/3351086625 shutdown_connections 2022-08-01T19:39:02.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9ee12b0700 1 -- 172.21.15.17:0/3351086625 wait complete. 2022-08-01T19:39:02.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9ee12b0700 1 Processor -- start 2022-08-01T19:39:02.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9ee12b0700 1 -- start start 2022-08-01T19:39:02.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9ee12b0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9edc0b2930 0x7f9edc0b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:02.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9ee12b0700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9edc0b7840 0x7f9edc0ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:02.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9ee12b0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9edc146720 0x7f9edc0aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:02.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9ee12b0700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc0ada90 con 0x7f9edc146720 2022-08-01T19:39:02.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9ee12b0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc0ad680 con 0x7f9edc0b2930 2022-08-01T19:39:02.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9ee12b0700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc152e70 con 0x7f9edc0b7840 2022-08-01T19:39:02.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9ee0aaf700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9edc146720 0x7f9edc0aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:02.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9ee0aaf700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9edc146720 0x7f9edc0aecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39588/0 (socket says 172.21.15.17:39588) 2022-08-01T19:39:02.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9ee0aaf700 1 -- 172.21.15.17:0/1497556819 learned_addr learned my addr 172.21.15.17:0/1497556819 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:02.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9edbfff700 1 --2- 172.21.15.17:0/1497556819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9edc0b2930 0x7f9edc0b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:02.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.670+0000 7f9edb7fe700 1 --2- 172.21.15.17:0/1497556819 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9edc0b7840 0x7f9edc0ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:02.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.674+0000 7f9ee0aaf700 1 -- 172.21.15.17:0/1497556819 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9edc0b7840 msgr2=0x7f9edc0ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:02.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.674+0000 7f9ee0aaf700 1 --2- 172.21.15.17:0/1497556819 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9edc0b7840 0x7f9edc0ae7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:02.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.674+0000 7f9ee0aaf700 1 -- 172.21.15.17:0/1497556819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9edc0b2930 msgr2=0x7f9edc0b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:02.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.674+0000 7f9ee0aaf700 1 --2- 172.21.15.17:0/1497556819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9edc0b2930 0x7f9edc0b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:02.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.674+0000 7f9ee0aaf700 1 -- 172.21.15.17:0/1497556819 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed40094d0 con 0x7f9edc146720 2022-08-01T19:39:02.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.674+0000 7f9edb7fe700 1 --2- 172.21.15.17:0/1497556819 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9edc0b7840 0x7f9edc0ae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:39:02.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.674+0000 7f9ee0aaf700 1 --2- 172.21.15.17:0/1497556819 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9edc146720 0x7f9edc0aecc0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f9ecc00a000 tx=0x7f9ecc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:02.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.674+0000 7f9ed97fa700 1 -- 172.21.15.17:0/1497556819 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ecc011780 con 0x7f9edc146720 2022-08-01T19:39:02.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.674+0000 7f9ed97fa700 1 -- 172.21.15.17:0/1497556819 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ecc0118e0 con 0x7f9edc146720 2022-08-01T19:39:02.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.674+0000 7f9ed97fa700 1 -- 172.21.15.17:0/1497556819 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ecc011c40 con 0x7f9edc146720 2022-08-01T19:39:02.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.674+0000 7f9ee12b0700 1 -- 172.21.15.17:0/1497556819 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9edc153110 con 0x7f9edc146720 2022-08-01T19:39:02.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.674+0000 7f9ee12b0700 1 -- 172.21.15.17:0/1497556819 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9edc0ad350 con 0x7f9edc146720 2022-08-01T19:39:02.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.674+0000 7f9ee12b0700 1 -- 172.21.15.17:0/1497556819 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9edc1532a0 con 0x7f9edc146720 2022-08-01T19:39:02.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.674+0000 7f9ed97fa700 1 -- 172.21.15.17:0/1497556819 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9ecc012420 con 0x7f9edc146720 2022-08-01T19:39:02.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.674+0000 7f9ed97fa700 1 --2- 172.21.15.17:0/1497556819 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9ebc053950 0x7f9ebc055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:02.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.674+0000 7f9ed97fa700 1 -- 172.21.15.17:0/1497556819 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f9ecc06bb20 con 0x7f9edc146720 2022-08-01T19:39:02.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.674+0000 7f9edbfff700 1 --2- 172.21.15.17:0/1497556819 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9ebc053950 0x7f9ebc055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:02.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.674+0000 7f9edbfff700 1 --2- 172.21.15.17:0/1497556819 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9ebc053950 0x7f9ebc055e10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f9ed401b000 tx=0x7f9ed4016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:02.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.682+0000 7f9ed97fa700 1 -- 172.21.15.17:0/1497556819 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9ecc042b60 con 0x7f9edc146720 2022-08-01T19:39:02.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:02.846+0000 7f9ee12b0700 1 -- 172.21.15.17:0/1497556819 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]} v 0) v1 -- 0x7f9edc0574e0 con 0x7f9edc146720 2022-08-01T19:39:03.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.282+0000 7f9ed97fa700 1 -- 172.21.15.17:0/1497556819 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]}]=0 v214) v1 ==== 152+0+0 (secure 0 0 0) 0x7f9ecc046180 con 0x7f9edc146720 2022-08-01T19:39:03.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.282+0000 7f9ee12b0700 1 -- 172.21.15.17:0/1497556819 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9ebc053950 msgr2=0x7f9ebc055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:03.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.282+0000 7f9ee12b0700 1 --2- 172.21.15.17:0/1497556819 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9ebc053950 0x7f9ebc055e10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f9ed401b000 tx=0x7f9ed4016000).stop 2022-08-01T19:39:03.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.282+0000 7f9ee12b0700 1 -- 172.21.15.17:0/1497556819 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9edc146720 msgr2=0x7f9edc0aecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:03.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.282+0000 7f9ee12b0700 1 --2- 172.21.15.17:0/1497556819 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9edc146720 0x7f9edc0aecc0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f9ecc00a000 tx=0x7f9ecc008040).stop 2022-08-01T19:39:03.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.282+0000 7f9ee12b0700 1 -- 172.21.15.17:0/1497556819 shutdown_connections 2022-08-01T19:39:03.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.282+0000 7f9ee12b0700 1 --2- 172.21.15.17:0/1497556819 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9ebc053950 0x7f9ebc055e10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:03.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.282+0000 7f9ee12b0700 1 --2- 172.21.15.17:0/1497556819 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9edc146720 0x7f9edc0aecc0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:03.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.282+0000 7f9ee12b0700 1 --2- 172.21.15.17:0/1497556819 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9edc0b7840 0x7f9edc0ae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:03.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.282+0000 7f9ee12b0700 1 --2- 172.21.15.17:0/1497556819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9edc0b2930 0x7f9edc0b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:03.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.282+0000 7f9ee12b0700 1 -- 172.21.15.17:0/1497556819 >> 172.21.15.17:0/1497556819 conn(0x7f9edc05f840 msgr2=0x7f9edc0b56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:03.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.282+0000 7f9ee12b0700 1 -- 172.21.15.17:0/1497556819 shutdown_connections 2022-08-01T19:39:03.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.282+0000 7f9ee12b0700 1 -- 172.21.15.17:0/1497556819 wait complete. 2022-08-01T19:39:03.303 INFO:tasks.cephfs_test_runner:test_drop_cache_command_dead_timeout (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2022-08-01T19:39:03.304 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command_timeout (tasks.cephfs.test_misc.TestCacheDrop) 2022-08-01T19:39:03.304 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout' 2022-08-01T19:39:03.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.410+0000 7f925c8b9700 1 Processor -- start 2022-08-01T19:39:03.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.410+0000 7f925c8b9700 1 -- start start 2022-08-01T19:39:03.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.410+0000 7f925c8b9700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f92580b2710 0x7f925814afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:03.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.410+0000 7f925c8b9700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f92580b01a0 0x7f925814b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:03.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.410+0000 7f925c8b9700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f92580afd60 0x7f92580a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:03.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.410+0000 7f925c8b9700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92580a7250 con 0x7f92580b2710 2022-08-01T19:39:03.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.410+0000 7f925c8b9700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92580a7480 con 0x7f92580b01a0 2022-08-01T19:39:03.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.410+0000 7f925c8b9700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92580a7720 con 0x7f92580afd60 2022-08-01T19:39:03.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.410+0000 7f92577fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f92580b2710 0x7f925814afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:03.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.410+0000 7f92577fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f92580b2710 0x7f925814afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39594/0 (socket says 172.21.15.17:39594) 2022-08-01T19:39:03.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.410+0000 7f92577fe700 1 -- 172.21.15.17:0/3242216106 learned_addr learned my addr 172.21.15.17:0/3242216106 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:03.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.410+0000 7f9256ffd700 1 --2- 172.21.15.17:0/3242216106 >> v2:172.21.15.77:3300/0 conn(0x7f92580b01a0 0x7f925814b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:03.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.410+0000 7f9257fff700 1 --2- 172.21.15.17:0/3242216106 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f92580afd60 0x7f92580a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:03.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.410+0000 7f92577fe700 1 -- 172.21.15.17:0/3242216106 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f92580afd60 msgr2=0x7f92580a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:03.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.410+0000 7f92577fe700 1 --2- 172.21.15.17:0/3242216106 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f92580afd60 0x7f92580a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:03.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.410+0000 7f92577fe700 1 -- 172.21.15.17:0/3242216106 >> v2:172.21.15.77:3300/0 conn(0x7f92580b01a0 msgr2=0x7f925814b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:03.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.410+0000 7f92577fe700 1 --2- 172.21.15.17:0/3242216106 >> v2:172.21.15.77:3300/0 conn(0x7f92580b01a0 0x7f925814b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:03.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.410+0000 7f92577fe700 1 -- 172.21.15.17:0/3242216106 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92580a79c0 con 0x7f92580b2710 2022-08-01T19:39:03.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.410+0000 7f9256ffd700 1 --2- 172.21.15.17:0/3242216106 >> v2:172.21.15.77:3300/0 conn(0x7f92580b01a0 0x7f925814b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:39:03.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.410+0000 7f92577fe700 1 --2- 172.21.15.17:0/3242216106 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92580b2710 0x7f925814afd0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f9240002f70 tx=0x7f9240006eb0).ready entity=mon.0 client_cookie=9ec183356a16bc5a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:03.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.410+0000 7f92567fc700 1 -- 172.21.15.17:0/3242216106 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f924000d570 con 0x7f92580b2710 2022-08-01T19:39:03.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.410+0000 7f92567fc700 1 -- 172.21.15.17:0/3242216106 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9240000e90 con 0x7f92580b2710 2022-08-01T19:39:03.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.410+0000 7f92567fc700 1 -- 172.21.15.17:0/3242216106 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f924000d900 con 0x7f92580b2710 2022-08-01T19:39:03.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.414+0000 7f925c8b9700 1 -- 172.21.15.17:0/3242216106 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92580b2710 msgr2=0x7f925814afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:03.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.414+0000 7f925c8b9700 1 --2- 172.21.15.17:0/3242216106 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92580b2710 0x7f925814afd0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f9240002f70 tx=0x7f9240006eb0).stop 2022-08-01T19:39:03.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.414+0000 7f925c8b9700 1 -- 172.21.15.17:0/3242216106 shutdown_connections 2022-08-01T19:39:03.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.414+0000 7f925c8b9700 1 --2- 172.21.15.17:0/3242216106 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f92580afd60 0x7f92580a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:03.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.414+0000 7f925c8b9700 1 --2- 172.21.15.17:0/3242216106 >> v2:172.21.15.77:3300/0 conn(0x7f92580b01a0 0x7f925814b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:03.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.414+0000 7f925c8b9700 1 --2- 172.21.15.17:0/3242216106 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92580b2710 0x7f925814afd0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:03.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.414+0000 7f925c8b9700 1 -- 172.21.15.17:0/3242216106 >> 172.21.15.17:0/3242216106 conn(0x7f9258060310 msgr2=0x7f92580b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:03.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.414+0000 7f925c8b9700 1 -- 172.21.15.17:0/3242216106 shutdown_connections 2022-08-01T19:39:03.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.414+0000 7f925c8b9700 1 -- 172.21.15.17:0/3242216106 wait complete. 2022-08-01T19:39:03.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.414+0000 7f925c8b9700 1 Processor -- start 2022-08-01T19:39:03.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.414+0000 7f925c8b9700 1 -- start start 2022-08-01T19:39:03.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.414+0000 7f925c8b9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92580afd60 0x7f92580aeee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:03.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.414+0000 7f925c8b9700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f92580b01a0 0x7f92581563e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:03.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.414+0000 7f925c8b9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92580b2710 0x7f92580a9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:03.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.414+0000 7f925c8b9700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9258157400 con 0x7f92580b2710 2022-08-01T19:39:03.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.414+0000 7f925c8b9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92580b5a20 con 0x7f92580afd60 2022-08-01T19:39:03.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.414+0000 7f925c8b9700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92581568d0 con 0x7f92580b01a0 2022-08-01T19:39:03.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.414+0000 7f9257fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92580b2710 0x7f92580a9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:03.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.414+0000 7f9257fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92580b2710 0x7f92580a9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39602/0 (socket says 172.21.15.17:39602) 2022-08-01T19:39:03.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.414+0000 7f9257fff700 1 -- 172.21.15.17:0/3959965831 learned_addr learned my addr 172.21.15.17:0/3959965831 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:03.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.414+0000 7f92577fe700 1 --2- 172.21.15.17:0/3959965831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92580afd60 0x7f92580aeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:03.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.414+0000 7f9256ffd700 1 --2- 172.21.15.17:0/3959965831 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f92580b01a0 0x7f92581563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:03.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.414+0000 7f9257fff700 1 -- 172.21.15.17:0/3959965831 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f92580b01a0 msgr2=0x7f92581563e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:03.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.414+0000 7f9257fff700 1 --2- 172.21.15.17:0/3959965831 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f92580b01a0 0x7f92581563e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:03.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.414+0000 7f9257fff700 1 -- 172.21.15.17:0/3959965831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92580afd60 msgr2=0x7f92580aeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:03.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.414+0000 7f9257fff700 1 --2- 172.21.15.17:0/3959965831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92580afd60 0x7f92580aeee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:03.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.414+0000 7f9257fff700 1 -- 172.21.15.17:0/3959965831 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924800b180 con 0x7f92580b2710 2022-08-01T19:39:03.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.414+0000 7f92577fe700 1 --2- 172.21.15.17:0/3959965831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92580afd60 0x7f92580aeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:39:03.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.414+0000 7f9257fff700 1 --2- 172.21.15.17:0/3959965831 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92580b2710 0x7f92580a9ca0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7f9250006e00 tx=0x7f92500091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:03.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.418+0000 7f9254ff9700 1 -- 172.21.15.17:0/3959965831 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f925000c7f0 con 0x7f92580b2710 2022-08-01T19:39:03.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.418+0000 7f9254ff9700 1 -- 172.21.15.17:0/3959965831 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f925000c950 con 0x7f92580b2710 2022-08-01T19:39:03.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.418+0000 7f925c8b9700 1 -- 172.21.15.17:0/3959965831 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f924000b1e0 con 0x7f92580b2710 2022-08-01T19:39:03.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.418+0000 7f9254ff9700 1 -- 172.21.15.17:0/3959965831 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f925000ccb0 con 0x7f92580b2710 2022-08-01T19:39:03.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.418+0000 7f925c8b9700 1 -- 172.21.15.17:0/3959965831 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92580b4040 con 0x7f92580b2710 2022-08-01T19:39:03.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.418+0000 7f925c8b9700 1 -- 172.21.15.17:0/3959965831 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92580a9a00 con 0x7f92580b2710 2022-08-01T19:39:03.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.418+0000 7f9254ff9700 1 -- 172.21.15.17:0/3959965831 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f92500123f0 con 0x7f92580b2710 2022-08-01T19:39:03.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.418+0000 7f9254ff9700 1 --2- 172.21.15.17:0/3959965831 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f923c053b00 0x7f923c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:03.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.422+0000 7f9254ff9700 1 -- 172.21.15.17:0/3959965831 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f925006c0e0 con 0x7f92580b2710 2022-08-01T19:39:03.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.422+0000 7f92577fe700 1 --2- 172.21.15.17:0/3959965831 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f923c053b00 0x7f923c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:03.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.422+0000 7f92577fe700 1 --2- 172.21.15.17:0/3959965831 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f923c053b00 0x7f923c055fc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f924001a000 tx=0x7f9240015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:03.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.426+0000 7f9254ff9700 1 -- 172.21.15.17:0/3959965831 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f92580a9a00 con 0x7f92580b2710 2022-08-01T19:39:03.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:03.590+0000 7f925c8b9700 1 -- 172.21.15.17:0/3959965831 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]} v 0) v1 -- 0x7f92580a9a00 con 0x7f92580b2710 2022-08-01T19:39:04.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.282+0000 7f9254ff9700 1 -- 172.21.15.17:0/3959965831 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]}]=0 v215) v1 ==== 150+0+0 (secure 0 0 0) 0x7f92580a9a00 con 0x7f92580b2710 2022-08-01T19:39:04.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.286+0000 7f925c8b9700 1 -- 172.21.15.17:0/3959965831 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f923c053b00 msgr2=0x7f923c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:04.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.286+0000 7f925c8b9700 1 --2- 172.21.15.17:0/3959965831 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f923c053b00 0x7f923c055fc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f924001a000 tx=0x7f9240015000).stop 2022-08-01T19:39:04.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.286+0000 7f925c8b9700 1 -- 172.21.15.17:0/3959965831 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92580b2710 msgr2=0x7f92580a9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:04.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.286+0000 7f925c8b9700 1 --2- 172.21.15.17:0/3959965831 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92580b2710 0x7f92580a9ca0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7f9250006e00 tx=0x7f92500091d0).stop 2022-08-01T19:39:04.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.286+0000 7f925c8b9700 1 -- 172.21.15.17:0/3959965831 shutdown_connections 2022-08-01T19:39:04.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.286+0000 7f925c8b9700 1 --2- 172.21.15.17:0/3959965831 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f923c053b00 0x7f923c055fc0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:04.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.286+0000 7f925c8b9700 1 --2- 172.21.15.17:0/3959965831 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92580b2710 0x7f92580a9ca0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:04.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.286+0000 7f925c8b9700 1 --2- 172.21.15.17:0/3959965831 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f92580b01a0 0x7f92581563e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:04.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.286+0000 7f925c8b9700 1 --2- 172.21.15.17:0/3959965831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92580afd60 0x7f92580aeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:04.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.286+0000 7f925c8b9700 1 -- 172.21.15.17:0/3959965831 >> 172.21.15.17:0/3959965831 conn(0x7f9258060310 msgr2=0x7f925814ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:04.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.286+0000 7f925c8b9700 1 -- 172.21.15.17:0/3959965831 shutdown_connections 2022-08-01T19:39:04.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.286+0000 7f925c8b9700 1 -- 172.21.15.17:0/3959965831 wait complete. 2022-08-01T19:39:04.308 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-08-01T19:39:04.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.418+0000 7f5cc359e700 1 Processor -- start 2022-08-01T19:39:04.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.418+0000 7f5cc359e700 1 -- start start 2022-08-01T19:39:04.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.418+0000 7f5cc359e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5cc40a9320 0x7f5cc41466e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:04.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.418+0000 7f5cc359e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5cc40b0d10 0x7f5cc4146bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:04.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.418+0000 7f5cc359e700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cc40b12d0 con 0x7f5cc40b0560 2022-08-01T19:39:04.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.418+0000 7f5cc359e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cc40a9700 con 0x7f5cc40a9320 2022-08-01T19:39:04.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.418+0000 7f5cc359e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc40b1500 con 0x7f5cc40b0d10 2022-08-01T19:39:04.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.418+0000 7f5cc2d9d700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5cc40b0560 0x7f5cc4140350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60038/0 (socket says 172.21.15.17:60038) 2022-08-01T19:39:04.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.418+0000 7f5cc2d9d700 1 -- 172.21.15.17:0/1561948181 learned_addr learned my addr 172.21.15.17:0/1561948181 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:04.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.418+0000 7f5cc259c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5cc40a9320 0x7f5cc41466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:04.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.418+0000 7f5cc1d9b700 1 --2- 172.21.15.17:0/1561948181 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5cc40b0d10 0x7f5cc4146bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:04.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.418+0000 7f5cc159a700 1 -- 172.21.15.17:0/1561948181 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 700050072 0 0) 0x7f5cc40b12d0 con 0x7f5cc40b0560 2022-08-01T19:39:04.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.418+0000 7f5cc159a700 1 -- 172.21.15.17:0/1561948181 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ca8003260 con 0x7f5cc40b0560 2022-08-01T19:39:04.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.418+0000 7f5cc259c700 1 -- 172.21.15.17:0/1561948181 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5cc40b0d10 msgr2=0x7f5cc4146bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:04.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.418+0000 7f5cc259c700 1 --2- 172.21.15.17:0/1561948181 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5cc40b0d10 0x7f5cc4146bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:04.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.418+0000 7f5cc259c700 1 -- 172.21.15.17:0/1561948181 >> v1:172.21.15.17:6789/0 conn(0x7f5cc40b0560 legacy=0x7f5cc4140350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:04.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.418+0000 7f5cc259c700 1 -- 172.21.15.17:0/1561948181 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc40b17a0 con 0x7f5cc40a9320 2022-08-01T19:39:04.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.418+0000 7f5cc259c700 1 --2- 172.21.15.17:0/1561948181 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cc40a9320 0x7f5cc41466e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f5cb4002f70 tx=0x7f5cb4006df0).ready entity=mon.0 client_cookie=469f7c8186d190f9 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:04.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.418+0000 7f5cc159a700 1 -- 172.21.15.17:0/1561948181 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5cb400d4a0 con 0x7f5cc40a9320 2022-08-01T19:39:04.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.418+0000 7f5cc159a700 1 -- 172.21.15.17:0/1561948181 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb4000e90 con 0x7f5cc40a9320 2022-08-01T19:39:04.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.418+0000 7f5cc159a700 1 -- 172.21.15.17:0/1561948181 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5cb400d7f0 con 0x7f5cc40a9320 2022-08-01T19:39:04.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.422+0000 7f5cc359e700 1 -- 172.21.15.17:0/1561948181 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cc40a9320 msgr2=0x7f5cc41466e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:04.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.422+0000 7f5cc359e700 1 --2- 172.21.15.17:0/1561948181 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cc40a9320 0x7f5cc41466e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f5cb4002f70 tx=0x7f5cb4006df0).stop 2022-08-01T19:39:04.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.422+0000 7f5cc359e700 1 -- 172.21.15.17:0/1561948181 shutdown_connections 2022-08-01T19:39:04.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.422+0000 7f5cc359e700 1 --2- 172.21.15.17:0/1561948181 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5cc40b0d10 0x7f5cc4146bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:04.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.422+0000 7f5cc359e700 1 --2- 172.21.15.17:0/1561948181 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cc40a9320 0x7f5cc41466e0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:04.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.422+0000 7f5cc359e700 1 -- 172.21.15.17:0/1561948181 >> 172.21.15.17:0/1561948181 conn(0x7f5cc40612d0 msgr2=0x7f5cc4062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:04.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.422+0000 7f5cc359e700 1 -- 172.21.15.17:0/1561948181 shutdown_connections 2022-08-01T19:39:04.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.422+0000 7f5cc359e700 1 -- 172.21.15.17:0/1561948181 wait complete. 2022-08-01T19:39:04.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.422+0000 7f5cc359e700 1 Processor -- start 2022-08-01T19:39:04.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.422+0000 7f5cc359e700 1 -- start start 2022-08-01T19:39:04.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.422+0000 7f5cc359e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5cc40a9320 0x7f5cc40b64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:04.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.422+0000 7f5cc359e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5cc40b0560 0x7f5cc4152fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:04.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.422+0000 7f5cc359e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cc40b0d10 0x7f5cc413efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:04.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.422+0000 7f5cc359e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc40b5830 con 0x7f5cc40b0d10 2022-08-01T19:39:04.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.422+0000 7f5cc359e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc40b5ad0 con 0x7f5cc40b0560 2022-08-01T19:39:04.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.422+0000 7f5cc359e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc40b5d70 con 0x7f5cc40a9320 2022-08-01T19:39:04.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.422+0000 7f5cc2d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cc40b0d10 0x7f5cc413efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:04.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.422+0000 7f5cc2d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cc40b0d10 0x7f5cc413efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39618/0 (socket says 172.21.15.17:39618) 2022-08-01T19:39:04.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.422+0000 7f5cc2d9d700 1 -- 172.21.15.17:0/97153098 learned_addr learned my addr 172.21.15.17:0/97153098 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:04.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.426+0000 7f5cc1d9b700 1 --2- 172.21.15.17:0/97153098 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5cc40b0560 0x7f5cc4152fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:04.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.426+0000 7f5cc259c700 1 --2- 172.21.15.17:0/97153098 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5cc40a9320 0x7f5cc40b64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:04.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.426+0000 7f5cc2d9d700 1 -- 172.21.15.17:0/97153098 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5cc40a9320 msgr2=0x7f5cc40b64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:04.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.426+0000 7f5cc2d9d700 1 --2- 172.21.15.17:0/97153098 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5cc40a9320 0x7f5cc40b64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:04.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.426+0000 7f5cc2d9d700 1 -- 172.21.15.17:0/97153098 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5cc40b0560 msgr2=0x7f5cc4152fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:04.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.426+0000 7f5cc2d9d700 1 --2- 172.21.15.17:0/97153098 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5cc40b0560 0x7f5cc4152fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:04.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.426+0000 7f5cc2d9d700 1 -- 172.21.15.17:0/97153098 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb400b180 con 0x7f5cc40b0d10 2022-08-01T19:39:04.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.426+0000 7f5cc2d9d700 1 --2- 172.21.15.17:0/97153098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cc40b0d10 0x7f5cc413efa0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f5cbc001490 tx=0x7f5cbc007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:04.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.426+0000 7f5cb37fe700 1 -- 172.21.15.17:0/97153098 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5cbc005ba0 con 0x7f5cc40b0d10 2022-08-01T19:39:04.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.426+0000 7f5cb37fe700 1 -- 172.21.15.17:0/97153098 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cbc005d00 con 0x7f5cc40b0d10 2022-08-01T19:39:04.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.426+0000 7f5cb37fe700 1 -- 172.21.15.17:0/97153098 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5cbc0072c0 con 0x7f5cc40b0d10 2022-08-01T19:39:04.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.426+0000 7f5cc359e700 1 -- 172.21.15.17:0/97153098 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc40b6010 con 0x7f5cc40b0d10 2022-08-01T19:39:04.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.426+0000 7f5cc359e700 1 -- 172.21.15.17:0/97153098 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc40ac640 con 0x7f5cc40b0d10 2022-08-01T19:39:04.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.426+0000 7f5cc359e700 1 -- 172.21.15.17:0/97153098 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc40b61a0 con 0x7f5cc40b0d10 2022-08-01T19:39:04.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.426+0000 7f5cb37fe700 1 -- 172.21.15.17:0/97153098 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5cbc010070 con 0x7f5cc40b0d10 2022-08-01T19:39:04.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.430+0000 7f5cb37fe700 1 --2- 172.21.15.17:0/97153098 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5ca0053a10 0x7f5ca0055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:04.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.430+0000 7f5cb37fe700 1 -- 172.21.15.17:0/97153098 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f5cbc06b6f0 con 0x7f5cc40b0d10 2022-08-01T19:39:04.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.430+0000 7f5cc259c700 1 --2- 172.21.15.17:0/97153098 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5ca0053a10 0x7f5ca0055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:04.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.430+0000 7f5cc259c700 1 --2- 172.21.15.17:0/97153098 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5ca0053a10 0x7f5ca0055ed0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f5cb401a000 tx=0x7f5cb4014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:04.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.438+0000 7f5cb37fe700 1 -- 172.21.15.17:0/97153098 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5cbc042680 con 0x7f5cc40b0d10 2022-08-01T19:39:04.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.562+0000 7f5cc359e700 1 -- 172.21.15.17:0/97153098 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5cc40574e0 con 0x7f5cc40b0d10 2022-08-01T19:39:04.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.570+0000 7f5cb37fe700 1 -- 172.21.15.17:0/97153098 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v8)=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5cbc018070 con 0x7f5cc40b0d10 2022-08-01T19:39:04.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.574+0000 7f5cc359e700 1 -- 172.21.15.17:0/97153098 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5ca0053a10 msgr2=0x7f5ca0055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:04.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.574+0000 7f5cc359e700 1 --2- 172.21.15.17:0/97153098 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5ca0053a10 0x7f5ca0055ed0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f5cb401a000 tx=0x7f5cb4014000).stop 2022-08-01T19:39:04.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.574+0000 7f5cc359e700 1 -- 172.21.15.17:0/97153098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cc40b0d10 msgr2=0x7f5cc413efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:04.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.574+0000 7f5cc359e700 1 --2- 172.21.15.17:0/97153098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cc40b0d10 0x7f5cc413efa0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f5cbc001490 tx=0x7f5cbc007e00).stop 2022-08-01T19:39:04.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.574+0000 7f5cc359e700 1 -- 172.21.15.17:0/97153098 shutdown_connections 2022-08-01T19:39:04.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.574+0000 7f5cc359e700 1 --2- 172.21.15.17:0/97153098 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5ca0053a10 0x7f5ca0055ed0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:04.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.574+0000 7f5cc359e700 1 --2- 172.21.15.17:0/97153098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cc40b0d10 0x7f5cc413efa0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:04.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.574+0000 7f5cc359e700 1 --2- 172.21.15.17:0/97153098 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5cc40b0560 0x7f5cc4152fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:04.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.574+0000 7f5cc359e700 1 --2- 172.21.15.17:0/97153098 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5cc40a9320 0x7f5cc40b64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:04.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.574+0000 7f5cc359e700 1 -- 172.21.15.17:0/97153098 >> 172.21.15.17:0/97153098 conn(0x7f5cc40612d0 msgr2=0x7f5cc40ae1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:04.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.574+0000 7f5cc359e700 1 -- 172.21.15.17:0/97153098 shutdown_connections 2022-08-01T19:39:04.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.574+0000 7f5cc359e700 1 -- 172.21.15.17:0/97153098 wait complete. 2022-08-01T19:39:04.595 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T19:39:04.596 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:39:04.615 DEBUG:teuthology.orchestra.run.smithi077:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:39:04.636 DEBUG:teuthology.orchestra.run.smithi085:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:39:04.658 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:39:04.680 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T19:39:04.681 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:39:04.682 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:39:04.682 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:39:04.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:39:04.710 INFO:teuthology.orchestra.run.smithi085.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-08-01T19:39:04.711 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-08-01T19:39:04.711 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:39:04.712 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:39:04.712 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:39:04.719 INFO:teuthology.orchestra.run.smithi190.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T19:39:04.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:39:04.721 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-08-01T19:39:04.722 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T19:39:04.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae43e2700 1 Processor -- start 2022-08-01T19:39:04.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae43e2700 1 -- start start 2022-08-01T19:39:04.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae43e2700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f8adc148530 0x7f8adc0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:04.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae43e2700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8adc0b04e0 con 0x7f8adc0a7250 2022-08-01T19:39:04.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae43e2700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8adc0b0780 con 0x7f8adc0b1f40 2022-08-01T19:39:04.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae43e2700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8adc0a9250 con 0x7f8adc148530 2022-08-01T19:39:04.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae297f700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f8adc0a7250 0x7f8adc1448b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60052/0 (socket says 172.21.15.17:60052) 2022-08-01T19:39:04.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae297f700 1 -- 172.21.15.17:0/239453371 learned_addr learned my addr 172.21.15.17:0/239453371 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:04.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae197d700 1 --2- 172.21.15.17:0/239453371 >> v2:172.21.15.77:3300/0 conn(0x7f8adc148530 0x7f8adc0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:04.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae117c700 1 -- 172.21.15.17:0/239453371 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2288488357 0 0) 0x7f8adc0b04e0 con 0x7f8adc0a7250 2022-08-01T19:39:04.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae117c700 1 -- 172.21.15.17:0/239453371 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ac8003260 con 0x7f8adc0a7250 2022-08-01T19:39:04.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae197d700 1 -- 172.21.15.17:0/239453371 >> v1:172.21.15.77:6789/0 conn(0x7f8adc0b1f40 legacy=0x7f8adc0b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:04.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae197d700 1 -- 172.21.15.17:0/239453371 >> v1:172.21.15.17:6789/0 conn(0x7f8adc0a7250 legacy=0x7f8adc1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:04.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae197d700 1 -- 172.21.15.17:0/239453371 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8adc0b0a20 con 0x7f8adc148530 2022-08-01T19:39:04.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae197d700 1 --2- 172.21.15.17:0/239453371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8adc148530 0x7f8adc0b1160 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f8adc1478c0 tx=0x7f8ad40091d0).ready entity=mon.1 client_cookie=68a4e5e4214fd37d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:04.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae117c700 1 -- 172.21.15.17:0/239453371 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ad400b680 con 0x7f8adc148530 2022-08-01T19:39:04.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae117c700 1 -- 172.21.15.17:0/239453371 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad400b7e0 con 0x7f8adc148530 2022-08-01T19:39:04.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae117c700 1 -- 172.21.15.17:0/239453371 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ad400b950 con 0x7f8adc148530 2022-08-01T19:39:04.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae43e2700 1 -- 172.21.15.17:0/239453371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8adc148530 msgr2=0x7f8adc0b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:04.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae43e2700 1 --2- 172.21.15.17:0/239453371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8adc148530 0x7f8adc0b1160 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f8adc1478c0 tx=0x7f8ad40091d0).stop 2022-08-01T19:39:04.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae43e2700 1 -- 172.21.15.17:0/239453371 shutdown_connections 2022-08-01T19:39:04.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae43e2700 1 --2- 172.21.15.17:0/239453371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8adc148530 0x7f8adc0b1160 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:04.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae43e2700 1 -- 172.21.15.17:0/239453371 >> 172.21.15.17:0/239453371 conn(0x7f8adc05f840 msgr2=0x7f8adc05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:04.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae43e2700 1 -- 172.21.15.17:0/239453371 shutdown_connections 2022-08-01T19:39:04.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae43e2700 1 -- 172.21.15.17:0/239453371 wait complete. 2022-08-01T19:39:04.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae43e2700 1 Processor -- start 2022-08-01T19:39:04.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae43e2700 1 -- start start 2022-08-01T19:39:04.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae43e2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8adc0a7250 0x7f8adc14b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:04.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae43e2700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8adc0b1f40 0x7f8adc0aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:04.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae43e2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8adc148530 0x7f8adc143c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:04.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae43e2700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc14aef0 con 0x7f8adc0a7250 2022-08-01T19:39:04.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae43e2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc14ac50 con 0x7f8adc148530 2022-08-01T19:39:04.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae43e2700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc14a340 con 0x7f8adc0b1f40 2022-08-01T19:39:04.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae217e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8adc0a7250 0x7f8adc14b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:04.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae217e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8adc0a7250 0x7f8adc14b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39628/0 (socket says 172.21.15.17:39628) 2022-08-01T19:39:04.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae217e700 1 -- 172.21.15.17:0/1069360782 learned_addr learned my addr 172.21.15.17:0/1069360782 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:04.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae197d700 1 --2- 172.21.15.17:0/1069360782 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8adc0b1f40 0x7f8adc0aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:04.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae297f700 1 --2- 172.21.15.17:0/1069360782 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8adc148530 0x7f8adc143c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:04.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae217e700 1 -- 172.21.15.17:0/1069360782 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8adc0b1f40 msgr2=0x7f8adc0aeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:04.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae217e700 1 --2- 172.21.15.17:0/1069360782 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8adc0b1f40 0x7f8adc0aeec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:04.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae217e700 1 -- 172.21.15.17:0/1069360782 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8adc148530 msgr2=0x7f8adc143c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:04.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae217e700 1 --2- 172.21.15.17:0/1069360782 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8adc148530 0x7f8adc143c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:04.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae217e700 1 -- 172.21.15.17:0/1069360782 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad4009470 con 0x7f8adc0a7250 2022-08-01T19:39:04.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae217e700 1 --2- 172.21.15.17:0/1069360782 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8adc0a7250 0x7f8adc14b9e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7f8adc0b3300 tx=0x7f8ad8007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:04.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ad37fe700 1 -- 172.21.15.17:0/1069360782 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ad8007030 con 0x7f8adc0a7250 2022-08-01T19:39:04.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae43e2700 1 -- 172.21.15.17:0/1069360782 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8adc14a5e0 con 0x7f8adc0a7250 2022-08-01T19:39:04.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ad37fe700 1 -- 172.21.15.17:0/1069360782 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad8007190 con 0x7f8adc0a7250 2022-08-01T19:39:04.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ad37fe700 1 -- 172.21.15.17:0/1069360782 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ad8006510 con 0x7f8adc0a7250 2022-08-01T19:39:04.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.838+0000 7f8ae43e2700 1 -- 172.21.15.17:0/1069360782 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8adc0a9fa0 con 0x7f8adc0a7250 2022-08-01T19:39:04.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.842+0000 7f8ae43e2700 1 -- 172.21.15.17:0/1069360782 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8adc0b0cc0 con 0x7f8adc0a7250 2022-08-01T19:39:04.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.842+0000 7f8ad37fe700 1 -- 172.21.15.17:0/1069360782 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8ad8010030 con 0x7f8adc0a7250 2022-08-01T19:39:04.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.842+0000 7f8ad37fe700 1 --2- 172.21.15.17:0/1069360782 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8abc053b00 0x7f8abc055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:04.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.842+0000 7f8ad37fe700 1 -- 172.21.15.17:0/1069360782 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f8ad806bad0 con 0x7f8adc0a7250 2022-08-01T19:39:04.858 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.842+0000 7f8ae197d700 1 --2- 172.21.15.17:0/1069360782 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8abc053b00 0x7f8abc055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:04.858 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.846+0000 7f8ae197d700 1 --2- 172.21.15.17:0/1069360782 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8abc053b00 0x7f8abc055fc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f8ad401b000 tx=0x7f8ad4016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:04.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.854+0000 7f8ad37fe700 1 -- 172.21.15.17:0/1069360782 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8ad80429e0 con 0x7f8adc0a7250 2022-08-01T19:39:04.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.974+0000 7f8ae43e2700 1 -- 172.21.15.17:0/1069360782 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f8adc0574e0 con 0x7f8adc0a7250 2022-08-01T19:39:04.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.978+0000 7f8ad37fe700 1 -- 172.21.15.17:0/1069360782 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v58) v1 ==== 81+0+29220 (secure 0 0 0) 0x7f8ad8046000 con 0x7f8adc0a7250 2022-08-01T19:39:04.980 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:39:04.980 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:39:04.981 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 58, 2022-08-01T19:39:04.981 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", 2022-08-01T19:39:04.981 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2022-08-01T19:33:50.105743+0000", 2022-08-01T19:39:04.982 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2022-08-01T19:39:02.273603+0000", 2022-08-01T19:39:04.982 INFO:teuthology.orchestra.run.smithi017.stdout: "last_up_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:39:04.982 INFO:teuthology.orchestra.run.smithi017.stdout: "last_in_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:39:04.982 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T19:39:04.983 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_num": 5799936, 2022-08-01T19:39:04.983 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_set": [ 2022-08-01T19:39:04.983 INFO:teuthology.orchestra.run.smithi017.stdout: "pglog_hardlimit", 2022-08-01T19:39:04.983 INFO:teuthology.orchestra.run.smithi017.stdout: "purged_snapdirs", 2022-08-01T19:39:04.984 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_deletes", 2022-08-01T19:39:04.984 INFO:teuthology.orchestra.run.smithi017.stdout: "sortbitwise" 2022-08-01T19:39:04.984 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:39:04.984 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_version": 6, 2022-08-01T19:39:04.984 INFO:teuthology.orchestra.run.smithi017.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T19:39:04.985 INFO:teuthology.orchestra.run.smithi017.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T19:39:04.985 INFO:teuthology.orchestra.run.smithi017.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T19:39:04.985 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_snapshot": "", 2022-08-01T19:39:04.985 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_max": 10, 2022-08-01T19:39:04.986 INFO:teuthology.orchestra.run.smithi017.stdout: "max_osd": 8, 2022-08-01T19:39:04.986 INFO:teuthology.orchestra.run.smithi017.stdout: "require_min_compat_client": "jewel", 2022-08-01T19:39:04.986 INFO:teuthology.orchestra.run.smithi017.stdout: "min_compat_client": "jewel", 2022-08-01T19:39:04.987 INFO:teuthology.orchestra.run.smithi017.stdout: "require_osd_release": "octopus", 2022-08-01T19:39:04.987 INFO:teuthology.orchestra.run.smithi017.stdout: "pools": [ 2022-08-01T19:39:04.987 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:04.987 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 1, 2022-08-01T19:39:04.988 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "device_health_metrics", 2022-08-01T19:39:04.988 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:33:53.001841+0000", 2022-08-01T19:39:04.988 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:39:04.988 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:39:04.989 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:39:04.989 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:39:04.989 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:39:04.989 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:39:04.990 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:39:04.990 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:39:04.990 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 1, 2022-08-01T19:39:04.990 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 1, 2022-08-01T19:39:04.991 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 1, 2022-08-01T19:39:04.991 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 1, 2022-08-01T19:39:04.991 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 1, 2022-08-01T19:39:04.991 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:39:04.992 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:39:04.992 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:39:04.992 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:39:04.992 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:39:04.993 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:39:04.993 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:39:04.993 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:04.993 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "16", 2022-08-01T19:39:04.994 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:39:04.994 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:39:04.994 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:39:04.995 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:39:04.995 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:39:04.995 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:39:04.995 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:39:04.996 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:39:04.996 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:39:04.996 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:39:04.996 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:39:04.996 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:39:04.997 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:39:04.997 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:39:04.997 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:39:04.997 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:39:04.998 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:39:04.998 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:39:04.998 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:39:04.999 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:39:04.999 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:39:04.999 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:39:04.999 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:39:04.999 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:39:05.000 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:39:05.000 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:39:05.000 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.000 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:39:05.001 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:39:05.001 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:39:05.001 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:39:05.001 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:39:05.002 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:39:05.002 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:39:05.002 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:39:05.003 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:39:05.003 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:39:05.003 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:39:05.003 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:39:05.004 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 1 2022-08-01T19:39:05.004 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.004 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:39:05.004 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_devicehealth": {} 2022-08-01T19:39:05.005 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.005 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.005 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.005 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 2, 2022-08-01T19:39:05.006 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "rbd", 2022-08-01T19:39:05.006 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:34:01.553098+0000", 2022-08-01T19:39:05.006 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:39:05.006 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:39:05.007 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:39:05.007 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:39:05.007 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:39:05.007 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:39:05.008 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:39:05.008 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:39:05.008 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:39:05.008 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:39:05.009 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:39:05.009 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:39:05.009 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:39:05.009 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:39:05.010 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:39:05.010 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:39:05.010 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:39:05.010 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:39:05.011 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:39:05.011 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:39:05.011 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.012 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "19", 2022-08-01T19:39:05.012 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:39:05.012 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:39:05.012 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:39:05.013 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:39:05.013 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:39:05.013 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:39:05.014 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:39:05.014 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:39:05.014 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:39:05.014 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:39:05.015 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:39:05.015 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:39:05.015 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:39:05.016 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:39:05.016 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:39:05.016 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:39:05.016 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:39:05.017 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:39:05.017 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:39:05.017 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:39:05.017 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:39:05.018 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:39:05.018 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:39:05.018 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:39:05.018 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:39:05.019 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:39:05.019 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.019 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:39:05.019 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:39:05.020 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:39:05.020 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:39:05.020 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:39:05.020 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:39:05.021 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:39:05.021 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:39:05.021 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:39:05.021 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:39:05.021 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:39:05.022 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:39:05.022 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:39:05.022 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd": {} 2022-08-01T19:39:05.023 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.023 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.023 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:39:05.023 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2022-08-01T19:39:05.024 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.024 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:39:05.024 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a962f14f-cf16-4fa8-84ff-c85325db9132", 2022-08-01T19:39:05.024 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:39:05.025 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:39:05.025 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:39:05.025 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:39:05.026 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:39:05.026 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:39:05.026 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:39:05.026 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 53, 2022-08-01T19:39:05.027 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:39:05.027 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:39:05.027 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:39:05.027 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:05.028 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.028 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:05.028 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6800", 2022-08-01T19:39:05.028 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:39:05.029 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.029 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.029 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:05.029 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6801", 2022-08-01T19:39:05.030 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:39:05.030 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.030 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.030 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.031 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:39:05.031 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:05.031 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.031 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:05.032 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6802", 2022-08-01T19:39:05.032 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:39:05.032 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.032 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.033 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:05.033 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6803", 2022-08-01T19:39:05.033 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:39:05.033 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.034 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.034 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.034 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:39:05.034 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:05.035 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.035 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:05.035 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6806", 2022-08-01T19:39:05.036 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:39:05.036 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.036 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.037 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:05.037 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6807", 2022-08-01T19:39:05.037 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:39:05.038 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.038 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.038 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.038 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:39:05.039 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:05.039 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.039 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:05.039 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6804", 2022-08-01T19:39:05.040 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:39:05.040 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.040 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.040 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:05.041 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6805", 2022-08-01T19:39:05.041 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:39:05.041 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.041 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.042 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.042 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6801/23008", 2022-08-01T19:39:05.042 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6803/23008", 2022-08-01T19:39:05.042 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6807/23008", 2022-08-01T19:39:05.043 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6805/23008", 2022-08-01T19:39:05.043 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:39:05.043 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:39:05.043 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:39:05.044 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.044 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.044 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.044 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:39:05.045 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "805f7051-a527-4e98-8844-b6e406a63248", 2022-08-01T19:39:05.045 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:39:05.045 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:39:05.046 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:39:05.046 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:39:05.046 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:39:05.046 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:39:05.047 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:39:05.047 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 53, 2022-08-01T19:39:05.047 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:39:05.047 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:39:05.048 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:39:05.048 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:05.048 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.048 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:05.048 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6808", 2022-08-01T19:39:05.049 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:39:05.049 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.049 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.049 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:05.050 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6809", 2022-08-01T19:39:05.050 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:39:05.050 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.050 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.051 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.051 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:39:05.051 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:05.051 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.052 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:05.052 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6810", 2022-08-01T19:39:05.052 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:39:05.052 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.053 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.053 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:05.053 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6811", 2022-08-01T19:39:05.053 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:39:05.054 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.054 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.054 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.054 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:39:05.055 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:05.055 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.055 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:05.056 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6814", 2022-08-01T19:39:05.056 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:39:05.056 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.056 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.057 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:05.057 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6815", 2022-08-01T19:39:05.057 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:39:05.057 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.058 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.058 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.058 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:39:05.058 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:05.059 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.059 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:05.059 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6812", 2022-08-01T19:39:05.060 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:39:05.060 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.060 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.060 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:05.061 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6813", 2022-08-01T19:39:05.061 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:39:05.061 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.061 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.062 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.062 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6809/23007", 2022-08-01T19:39:05.062 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6811/23007", 2022-08-01T19:39:05.062 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6815/23007", 2022-08-01T19:39:05.063 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6813/23007", 2022-08-01T19:39:05.063 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:39:05.063 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:39:05.063 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:39:05.064 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.064 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.064 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.064 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:39:05.065 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "89f7f3df-7c10-48e1-8b69-9e4243923a61", 2022-08-01T19:39:05.065 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:39:05.065 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:39:05.065 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:39:05.066 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:39:05.066 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:39:05.066 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:39:05.066 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:39:05.067 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 53, 2022-08-01T19:39:05.067 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:39:05.067 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:39:05.067 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:39:05.068 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:05.068 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.068 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:05.068 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6817", 2022-08-01T19:39:05.069 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:39:05.069 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.069 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.069 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:05.070 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6819", 2022-08-01T19:39:05.070 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:39:05.070 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.070 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.070 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.071 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:39:05.071 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:05.071 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.071 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:05.072 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6822", 2022-08-01T19:39:05.072 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:39:05.072 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.072 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.073 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:05.073 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6824", 2022-08-01T19:39:05.073 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:39:05.073 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.074 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.074 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.074 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:39:05.074 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:05.075 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.075 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:05.075 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6830", 2022-08-01T19:39:05.075 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:39:05.076 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.076 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.076 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:05.076 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6831", 2022-08-01T19:39:05.077 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:39:05.077 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.077 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.077 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.078 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:39:05.078 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:05.078 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.078 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:05.079 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6826", 2022-08-01T19:39:05.079 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:39:05.079 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.079 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.080 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:05.080 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6829", 2022-08-01T19:39:05.080 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:39:05.080 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.081 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.081 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.081 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6819/23010", 2022-08-01T19:39:05.081 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6824/23010", 2022-08-01T19:39:05.082 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6831/23010", 2022-08-01T19:39:05.082 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6829/23010", 2022-08-01T19:39:05.082 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:39:05.082 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:39:05.083 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:39:05.083 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.083 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.083 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.084 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:39:05.084 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "96123468-aabc-4a64-909e-a192c369090d", 2022-08-01T19:39:05.084 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:39:05.084 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:39:05.084 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:39:05.085 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:39:05.085 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:39:05.085 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:39:05.085 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:39:05.086 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 52, 2022-08-01T19:39:05.086 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:39:05.086 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:39:05.086 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:39:05.087 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:05.087 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.087 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:05.087 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6816", 2022-08-01T19:39:05.088 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:39:05.088 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.088 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.088 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:05.089 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6818", 2022-08-01T19:39:05.089 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:39:05.089 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.089 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.089 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.090 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:39:05.090 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:05.090 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.090 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:05.091 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6820", 2022-08-01T19:39:05.091 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:39:05.091 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.091 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.092 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:05.092 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6821", 2022-08-01T19:39:05.092 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:39:05.092 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.093 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.093 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.093 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:39:05.093 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:05.094 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.094 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:05.094 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6827", 2022-08-01T19:39:05.094 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:39:05.095 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.095 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.095 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:05.095 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6828", 2022-08-01T19:39:05.096 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:39:05.096 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.096 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.096 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.096 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:39:05.097 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:05.097 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.097 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:05.097 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6823", 2022-08-01T19:39:05.098 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:39:05.098 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.098 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.098 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:05.099 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6825", 2022-08-01T19:39:05.099 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:39:05.099 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.099 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.100 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.100 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6818/23009", 2022-08-01T19:39:05.100 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6821/23009", 2022-08-01T19:39:05.100 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6828/23009", 2022-08-01T19:39:05.101 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6825/23009", 2022-08-01T19:39:05.101 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:39:05.101 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:39:05.101 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:39:05.102 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.102 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.102 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.102 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:39:05.103 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a6e26665-e04b-46ec-ab66-e912acbf044c", 2022-08-01T19:39:05.103 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:39:05.103 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:39:05.103 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:39:05.104 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:39:05.104 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:39:05.104 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:39:05.104 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:39:05.105 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 53, 2022-08-01T19:39:05.105 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:39:05.105 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:39:05.105 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:39:05.105 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:05.106 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.106 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:05.106 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6810", 2022-08-01T19:39:05.106 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:39:05.107 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.107 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.107 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:05.107 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6811", 2022-08-01T19:39:05.108 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:39:05.108 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.108 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.108 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.109 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:39:05.109 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:05.109 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.109 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:05.110 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6812", 2022-08-01T19:39:05.110 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:39:05.110 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.110 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.111 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:05.111 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6813", 2022-08-01T19:39:05.111 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:39:05.112 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.112 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.112 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.112 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:39:05.113 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:05.113 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.113 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:05.113 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6816", 2022-08-01T19:39:05.114 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:39:05.114 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.114 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.115 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:05.115 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6817", 2022-08-01T19:39:05.115 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:39:05.115 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.116 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.116 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.116 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:39:05.116 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:05.117 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.117 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:05.117 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6814", 2022-08-01T19:39:05.117 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:39:05.118 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.118 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.118 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:05.118 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6815", 2022-08-01T19:39:05.118 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:39:05.119 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.119 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.119 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.120 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6811/23584", 2022-08-01T19:39:05.120 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6813/23584", 2022-08-01T19:39:05.120 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6817/23584", 2022-08-01T19:39:05.120 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6815/23584", 2022-08-01T19:39:05.121 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:39:05.121 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:39:05.121 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:39:05.122 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.122 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.122 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.122 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:39:05.123 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "502c0034-18ee-44ac-87ff-19b9dddb66a0", 2022-08-01T19:39:05.123 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:39:05.123 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:39:05.123 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:39:05.124 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:39:05.124 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:39:05.124 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:39:05.124 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:39:05.125 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 53, 2022-08-01T19:39:05.125 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:39:05.125 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:39:05.125 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:39:05.126 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:05.126 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.126 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:05.126 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6802", 2022-08-01T19:39:05.127 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:39:05.127 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.127 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.127 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:05.128 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6803", 2022-08-01T19:39:05.128 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:39:05.128 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.128 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.129 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.129 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:39:05.129 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:05.129 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.130 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:05.130 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6804", 2022-08-01T19:39:05.130 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:39:05.130 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.131 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.131 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:05.131 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6805", 2022-08-01T19:39:05.131 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:39:05.132 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.132 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.132 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.132 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:39:05.133 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:05.133 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.133 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:05.133 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6808", 2022-08-01T19:39:05.134 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:39:05.134 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.134 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.134 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:05.135 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6809", 2022-08-01T19:39:05.135 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:39:05.135 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.135 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.136 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.136 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:39:05.136 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:05.136 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.136 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:05.137 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6806", 2022-08-01T19:39:05.137 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:39:05.137 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.138 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.138 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:05.138 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6807", 2022-08-01T19:39:05.138 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:39:05.139 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.139 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.139 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.139 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6803/23585", 2022-08-01T19:39:05.140 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6805/23585", 2022-08-01T19:39:05.140 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6809/23585", 2022-08-01T19:39:05.140 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6807/23585", 2022-08-01T19:39:05.140 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:39:05.141 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:39:05.141 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:39:05.141 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.141 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.142 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.142 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:39:05.142 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "ac4e755a-7fe7-46cb-8407-852d5af9ffc5", 2022-08-01T19:39:05.142 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:39:05.143 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:39:05.143 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:39:05.143 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:39:05.143 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:39:05.144 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:39:05.144 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:39:05.144 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 53, 2022-08-01T19:39:05.144 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:39:05.145 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:39:05.145 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:39:05.145 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:05.145 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.146 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:05.146 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6820", 2022-08-01T19:39:05.146 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:39:05.146 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.147 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.147 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:05.147 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6821", 2022-08-01T19:39:05.148 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:39:05.148 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.148 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.148 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.149 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:39:05.149 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:05.149 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.149 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:05.150 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6823", 2022-08-01T19:39:05.150 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:39:05.150 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.151 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.151 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:05.151 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6825", 2022-08-01T19:39:05.151 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:39:05.151 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.152 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.152 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.152 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:39:05.152 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:05.153 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.153 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:05.153 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6831", 2022-08-01T19:39:05.153 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:39:05.154 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.154 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.154 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:05.154 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6833", 2022-08-01T19:39:05.155 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:39:05.155 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.155 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.155 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.155 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:39:05.156 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:05.156 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.156 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:05.156 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6827", 2022-08-01T19:39:05.157 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:39:05.157 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.157 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.157 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:05.158 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6829", 2022-08-01T19:39:05.158 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:39:05.158 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.158 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.159 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.159 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6821/23587", 2022-08-01T19:39:05.159 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6825/23587", 2022-08-01T19:39:05.159 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6833/23587", 2022-08-01T19:39:05.160 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6829/23587", 2022-08-01T19:39:05.160 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:39:05.160 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:39:05.160 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:39:05.161 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.161 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.161 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.161 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:39:05.162 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "874761e2-6410-40fc-b60c-e58abe5ce489", 2022-08-01T19:39:05.162 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:39:05.162 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:39:05.162 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:39:05.163 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:39:05.163 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:39:05.163 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:39:05.164 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:39:05.164 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 52, 2022-08-01T19:39:05.164 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:39:05.165 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:39:05.165 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:39:05.165 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:05.166 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.166 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:05.167 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6818", 2022-08-01T19:39:05.167 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:39:05.167 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.168 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.168 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:05.168 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6819", 2022-08-01T19:39:05.168 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:39:05.168 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.169 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.169 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.169 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:39:05.169 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:05.170 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.170 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:05.170 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6822", 2022-08-01T19:39:05.170 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:39:05.171 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.171 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.171 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:05.171 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6824", 2022-08-01T19:39:05.172 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:39:05.172 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.172 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.172 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.173 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:39:05.173 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:05.173 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.173 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:05.174 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6830", 2022-08-01T19:39:05.174 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:39:05.174 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.174 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.175 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:05.175 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6832", 2022-08-01T19:39:05.175 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:39:05.175 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.176 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.176 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.176 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:39:05.176 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:05.177 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.177 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:05.177 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6826", 2022-08-01T19:39:05.178 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:39:05.178 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.179 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.179 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:05.180 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6828", 2022-08-01T19:39:05.180 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:39:05.180 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.180 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.181 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.181 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6819/23586", 2022-08-01T19:39:05.181 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6824/23586", 2022-08-01T19:39:05.181 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6832/23586", 2022-08-01T19:39:05.181 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6828/23586", 2022-08-01T19:39:05.182 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:39:05.182 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:39:05.182 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:39:05.182 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:05.183 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.183 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:39:05.183 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_xinfo": [ 2022-08-01T19:39:05.183 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.184 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:39:05.184 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:39:05.184 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:39:05.184 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:39:05.184 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:39:05.185 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:39:05.185 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.588817+0000", 2022-08-01T19:39:05.185 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:39:05.185 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.186 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.186 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:39:05.186 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:39:05.186 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:39:05.187 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:39:05.187 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:39:05.187 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:39:05.187 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.782753+0000", 2022-08-01T19:39:05.188 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:39:05.188 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.188 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.188 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:39:05.188 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:39:05.189 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:39:05.189 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:39:05.189 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:39:05.189 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:39:05.190 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.837924+0000", 2022-08-01T19:39:05.190 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:39:05.190 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.190 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.190 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:39:05.191 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:39:05.191 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:39:05.191 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:39:05.191 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:39:05.192 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:39:05.192 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.839948+0000", 2022-08-01T19:39:05.192 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:39:05.192 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.193 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.193 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:39:05.193 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:39:05.193 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:39:05.193 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:39:05.194 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:39:05.194 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:39:05.194 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.112801+0000", 2022-08-01T19:39:05.194 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:39:05.195 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.195 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.195 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:39:05.195 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:39:05.196 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:39:05.196 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:39:05.196 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:39:05.196 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:39:05.197 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.106704+0000", 2022-08-01T19:39:05.197 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:39:05.197 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.198 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.198 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:39:05.198 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:39:05.198 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:39:05.199 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:39:05.199 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:39:05.199 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:39:05.199 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.893170+0000", 2022-08-01T19:39:05.199 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:39:05.200 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.200 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:05.200 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:39:05.200 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:39:05.201 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:39:05.201 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:39:05.201 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:39:05.201 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:39:05.201 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.886947+0000", 2022-08-01T19:39:05.202 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:39:05.202 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.202 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:39:05.202 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap": [], 2022-08-01T19:39:05.203 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap_items": [], 2022-08-01T19:39:05.203 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_temp": [], 2022-08-01T19:39:05.203 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_temp": [], 2022-08-01T19:39:05.203 INFO:teuthology.orchestra.run.smithi017.stdout: "blacklist": { 2022-08-01T19:39:05.203 INFO:teuthology.orchestra.run.smithi017.stdout: "172.21.15.17:6835/3325991310": "2022-08-02T19:38:58.256217+0000", 2022-08-01T19:39:05.204 INFO:teuthology.orchestra.run.smithi017.stdout: "172.21.15.17:6834/3325991310": "2022-08-02T19:38:58.256217+0000" 2022-08-01T19:39:05.204 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.204 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profiles": { 2022-08-01T19:39:05.204 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_data": { 2022-08-01T19:39:05.204 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-device-class": "", 2022-08-01T19:39:05.205 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:39:05.205 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-root": "default", 2022-08-01T19:39:05.205 INFO:teuthology.orchestra.run.smithi017.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T19:39:05.205 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:39:05.206 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "2", 2022-08-01T19:39:05.206 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:39:05.206 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:39:05.206 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van", 2022-08-01T19:39:05.207 INFO:teuthology.orchestra.run.smithi017.stdout: "w": "8" 2022-08-01T19:39:05.207 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.207 INFO:teuthology.orchestra.run.smithi017.stdout: "default": { 2022-08-01T19:39:05.207 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:39:05.208 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:39:05.208 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "1", 2022-08-01T19:39:05.208 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:39:05.208 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:39:05.209 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van" 2022-08-01T19:39:05.209 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:05.209 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:05.209 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps_queue": [], 2022-08-01T19:39:05.210 INFO:teuthology.orchestra.run.smithi017.stdout: "new_removed_snaps": [], 2022-08-01T19:39:05.210 INFO:teuthology.orchestra.run.smithi017.stdout: "new_purged_snaps": [], 2022-08-01T19:39:05.210 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_node_flags": {}, 2022-08-01T19:39:05.210 INFO:teuthology.orchestra.run.smithi017.stdout: "device_class_flags": {} 2022-08-01T19:39:05.211 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:39:05.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.978+0000 7f8ae43e2700 1 -- 172.21.15.17:0/1069360782 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8abc053b00 msgr2=0x7f8abc055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:05.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.978+0000 7f8ae43e2700 1 --2- 172.21.15.17:0/1069360782 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8abc053b00 0x7f8abc055fc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f8ad401b000 tx=0x7f8ad4016000).stop 2022-08-01T19:39:05.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.978+0000 7f8ae43e2700 1 -- 172.21.15.17:0/1069360782 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8adc0a7250 msgr2=0x7f8adc14b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:05.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.978+0000 7f8ae43e2700 1 --2- 172.21.15.17:0/1069360782 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8adc0a7250 0x7f8adc14b9e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7f8adc0b3300 tx=0x7f8ad8007c10).stop 2022-08-01T19:39:05.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.978+0000 7f8ae43e2700 1 -- 172.21.15.17:0/1069360782 shutdown_connections 2022-08-01T19:39:05.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.978+0000 7f8ae43e2700 1 --2- 172.21.15.17:0/1069360782 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8abc053b00 0x7f8abc055fc0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:05.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.978+0000 7f8ae43e2700 1 --2- 172.21.15.17:0/1069360782 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8adc148530 0x7f8adc143c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:05.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.978+0000 7f8ae43e2700 1 --2- 172.21.15.17:0/1069360782 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8adc0b1f40 0x7f8adc0aeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:05.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.978+0000 7f8ae43e2700 1 --2- 172.21.15.17:0/1069360782 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8adc0a7250 0x7f8adc14b9e0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:05.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.978+0000 7f8ae43e2700 1 -- 172.21.15.17:0/1069360782 >> 172.21.15.17:0/1069360782 conn(0x7f8adc05f840 msgr2=0x7f8adc14e170 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:05.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.978+0000 7f8ae43e2700 1 -- 172.21.15.17:0/1069360782 shutdown_connections 2022-08-01T19:39:05.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:04.978+0000 7f8ae43e2700 1 -- 172.21.15.17:0/1069360782 wait complete. 2022-08-01T19:39:05.217 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:39:05.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.322+0000 7f01d4997700 1 Processor -- start 2022-08-01T19:39:05.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.322+0000 7f01d4997700 1 -- start start 2022-08-01T19:39:05.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.322+0000 7f01d4997700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f01d00b36e0 0x7f01d0141a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:05.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.322+0000 7f01d4997700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f01d0144e60 0x7f01d0066ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:05.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.322+0000 7f01d4997700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01d0064ef0 con 0x7f01d0064b10 2022-08-01T19:39:05.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.322+0000 7f01d4997700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01d013ebf0 con 0x7f01d0144e60 2022-08-01T19:39:05.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.322+0000 7f01d4997700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d01452d0 con 0x7f01d00b36e0 2022-08-01T19:39:05.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.322+0000 7f01cf7fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f01d0064b10 0x7f01d0141370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60062/0 (socket says 172.21.15.17:60062) 2022-08-01T19:39:05.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.322+0000 7f01cf7fe700 1 -- 172.21.15.17:0/3149308849 learned_addr learned my addr 172.21.15.17:0/3149308849 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:05.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.322+0000 7f01cffff700 1 --2- 172.21.15.17:0/3149308849 >> v2:172.21.15.77:3300/0 conn(0x7f01d0144e60 0x7f01d0066ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:05.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.322+0000 7f01ceffd700 1 --2- 172.21.15.17:0/3149308849 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f01d00b36e0 0x7f01d0141a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:05.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.322+0000 7f01ce7fc700 1 -- 172.21.15.17:0/3149308849 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 629317205 0 0) 0x7f01d0064ef0 con 0x7f01d0064b10 2022-08-01T19:39:05.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.322+0000 7f01ce7fc700 1 -- 172.21.15.17:0/3149308849 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01bc003260 con 0x7f01d0064b10 2022-08-01T19:39:05.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.322+0000 7f01ce7fc700 1 -- 172.21.15.17:0/3149308849 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3373141610 0 0) 0x7f01bc003260 con 0x7f01d0064b10 2022-08-01T19:39:05.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.322+0000 7f01cffff700 1 -- 172.21.15.17:0/3149308849 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f01d00b36e0 msgr2=0x7f01d0141a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:05.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.322+0000 7f01cffff700 1 --2- 172.21.15.17:0/3149308849 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f01d00b36e0 0x7f01d0141a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:05.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.322+0000 7f01cffff700 1 -- 172.21.15.17:0/3149308849 >> v1:172.21.15.17:6789/0 conn(0x7f01d0064b10 legacy=0x7f01d0141370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:05.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.322+0000 7f01cffff700 1 -- 172.21.15.17:0/3149308849 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d0066690 con 0x7f01d0144e60 2022-08-01T19:39:05.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.322+0000 7f01cffff700 1 --2- 172.21.15.17:0/3149308849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01d0144e60 0x7f01d0066ac0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f01c0001820 tx=0x7f01c80091d0).ready entity=mon.1 client_cookie=1ead584f124c969a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:05.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.322+0000 7f01ce7fc700 1 -- 172.21.15.17:0/3149308849 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01c0001680 con 0x7f01d0144e60 2022-08-01T19:39:05.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.322+0000 7f01ce7fc700 1 -- 172.21.15.17:0/3149308849 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c800f510 con 0x7f01d0144e60 2022-08-01T19:39:05.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.322+0000 7f01ce7fc700 1 -- 172.21.15.17:0/3149308849 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01c800f680 con 0x7f01d0144e60 2022-08-01T19:39:05.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.322+0000 7f01d4997700 1 -- 172.21.15.17:0/3149308849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01d0144e60 msgr2=0x7f01d0066ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:05.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.322+0000 7f01d4997700 1 --2- 172.21.15.17:0/3149308849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01d0144e60 0x7f01d0066ac0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f01c0001820 tx=0x7f01c80091d0).stop 2022-08-01T19:39:05.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.322+0000 7f01d4997700 1 -- 172.21.15.17:0/3149308849 shutdown_connections 2022-08-01T19:39:05.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.326+0000 7f01d4997700 1 --2- 172.21.15.17:0/3149308849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01d0144e60 0x7f01d0066ac0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:05.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.326+0000 7f01d4997700 1 --2- 172.21.15.17:0/3149308849 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f01d00b36e0 0x7f01d0141a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:05.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.326+0000 7f01d4997700 1 -- 172.21.15.17:0/3149308849 >> 172.21.15.17:0/3149308849 conn(0x7f01d005f840 msgr2=0x7f01d005fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:05.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.326+0000 7f01d4997700 1 -- 172.21.15.17:0/3149308849 shutdown_connections 2022-08-01T19:39:05.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.326+0000 7f01d4997700 1 -- 172.21.15.17:0/3149308849 wait complete. 2022-08-01T19:39:05.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.326+0000 7f01d4997700 1 Processor -- start 2022-08-01T19:39:05.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.326+0000 7f01d4997700 1 -- start start 2022-08-01T19:39:05.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.334+0000 7f01d4997700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01d0064b10 0x7f01d013c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:05.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.334+0000 7f01d4997700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f01d00b36e0 0x7f01d00affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:05.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.334+0000 7f01d4997700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01d0144e60 0x7f01d00b0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:05.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.334+0000 7f01d4997700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d0153c10 con 0x7f01d0144e60 2022-08-01T19:39:05.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.334+0000 7f01d4997700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d0153eb0 con 0x7f01d0064b10 2022-08-01T19:39:05.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.334+0000 7f01d4997700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d0154150 con 0x7f01d00b36e0 2022-08-01T19:39:05.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.334+0000 7f01cffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01d0144e60 0x7f01d00b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:05.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.334+0000 7f01ceffd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f01d00b36e0 0x7f01d00affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:05.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.334+0000 7f01cffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01d0144e60 0x7f01d00b0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39646/0 (socket says 172.21.15.17:39646) 2022-08-01T19:39:05.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.334+0000 7f01cffff700 1 -- 172.21.15.17:0/3211500961 learned_addr learned my addr 172.21.15.17:0/3211500961 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:05.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.334+0000 7f01cf7fe700 1 --2- 172.21.15.17:0/3211500961 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01d0064b10 0x7f01d013c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:05.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.334+0000 7f01cffff700 1 -- 172.21.15.17:0/3211500961 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f01d00b36e0 msgr2=0x7f01d00affa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:05.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.334+0000 7f01cffff700 1 --2- 172.21.15.17:0/3211500961 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f01d00b36e0 0x7f01d00affa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:05.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.334+0000 7f01cffff700 1 -- 172.21.15.17:0/3211500961 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01d0064b10 msgr2=0x7f01d013c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:05.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.334+0000 7f01cffff700 1 --2- 172.21.15.17:0/3211500961 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01d0064b10 0x7f01d013c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:05.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.334+0000 7f01cffff700 1 -- 172.21.15.17:0/3211500961 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c8009470 con 0x7f01d0144e60 2022-08-01T19:39:05.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.334+0000 7f01ceffd700 1 --2- 172.21.15.17:0/3211500961 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f01d00b36e0 0x7f01d00affa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:39:05.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.334+0000 7f01cffff700 1 --2- 172.21.15.17:0/3211500961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01d0144e60 0x7f01d00b0490 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f01c0001820 tx=0x7f01c8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:05.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.334+0000 7f01ccff9700 1 -- 172.21.15.17:0/3211500961 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01c800f680 con 0x7f01d0144e60 2022-08-01T19:39:05.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.334+0000 7f01ccff9700 1 -- 172.21.15.17:0/3211500961 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c80032d0 con 0x7f01d0144e60 2022-08-01T19:39:05.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.334+0000 7f01d4997700 1 -- 172.21.15.17:0/3211500961 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d013cbe0 con 0x7f01d0144e60 2022-08-01T19:39:05.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.334+0000 7f01ccff9700 1 -- 172.21.15.17:0/3211500961 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01c80205e0 con 0x7f01d0144e60 2022-08-01T19:39:05.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.334+0000 7f01d4997700 1 -- 172.21.15.17:0/3211500961 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d00b0980 con 0x7f01d0144e60 2022-08-01T19:39:05.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.334+0000 7f01d4997700 1 -- 172.21.15.17:0/3211500961 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d01547b0 con 0x7f01d0144e60 2022-08-01T19:39:05.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.334+0000 7f01ccff9700 1 -- 172.21.15.17:0/3211500961 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f01c801e070 con 0x7f01d0144e60 2022-08-01T19:39:05.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.334+0000 7f01ccff9700 1 --2- 172.21.15.17:0/3211500961 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f01b0053950 0x7f01b0055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:05.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.334+0000 7f01ccff9700 1 -- 172.21.15.17:0/3211500961 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f01c806c970 con 0x7f01d0144e60 2022-08-01T19:39:05.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.334+0000 7f01cf7fe700 1 --2- 172.21.15.17:0/3211500961 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f01b0053950 0x7f01b0055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:05.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.334+0000 7f01cf7fe700 1 --2- 172.21.15.17:0/3211500961 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f01b0053950 0x7f01b0055e10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f01c000d000 tx=0x7f01c00019a0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:05.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.338+0000 7f01ccff9700 1 -- 172.21.15.17:0/3211500961 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f01c8043a10 con 0x7f01d0144e60 2022-08-01T19:39:05.502 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.498+0000 7f01d4997700 1 -- 172.21.15.17:0/3211500961 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01d00b4850 con 0x7f01d0144e60 2022-08-01T19:39:05.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.498+0000 7f01ccff9700 1 -- 172.21.15.17:0/3211500961 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+1559 (secure 0 0 0) 0x7f01c8015020 con 0x7f01d0144e60 2022-08-01T19:39:05.503 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:39:05.503 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":27,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/1914125901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":1914125901},{"type":"v1","addr":"172.21.15.17:6833","nonce":1914125901}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":22},{"gid":5587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1403688289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1403688289},{"type":"v1","addr":"172.21.15.77:6835","nonce":1403688289}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":22},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.17:6835/260169708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":260169708},{"type":"v1","addr":"172.21.15.17:6835","nonce":260169708}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27}],"filesystems":[]} 2022-08-01T19:39:05.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.502+0000 7f01d4997700 1 -- 172.21.15.17:0/3211500961 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f01b0053950 msgr2=0x7f01b0055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:05.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.502+0000 7f01d4997700 1 --2- 172.21.15.17:0/3211500961 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f01b0053950 0x7f01b0055e10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f01c000d000 tx=0x7f01c00019a0).stop 2022-08-01T19:39:05.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.502+0000 7f01d4997700 1 -- 172.21.15.17:0/3211500961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01d0144e60 msgr2=0x7f01d00b0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:05.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.502+0000 7f01d4997700 1 --2- 172.21.15.17:0/3211500961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01d0144e60 0x7f01d00b0490 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f01c0001820 tx=0x7f01c8016000).stop 2022-08-01T19:39:05.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.502+0000 7f01d4997700 1 -- 172.21.15.17:0/3211500961 shutdown_connections 2022-08-01T19:39:05.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.502+0000 7f01d4997700 1 --2- 172.21.15.17:0/3211500961 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f01b0053950 0x7f01b0055e10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:05.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.502+0000 7f01d4997700 1 --2- 172.21.15.17:0/3211500961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01d0144e60 0x7f01d00b0490 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:05.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.502+0000 7f01d4997700 1 --2- 172.21.15.17:0/3211500961 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f01d00b36e0 0x7f01d00affa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:05.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.502+0000 7f01d4997700 1 --2- 172.21.15.17:0/3211500961 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01d0064b10 0x7f01d013c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:05.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.502+0000 7f01d4997700 1 -- 172.21.15.17:0/3211500961 >> 172.21.15.17:0/3211500961 conn(0x7f01d005f840 msgr2=0x7f01d00b5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:05.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.502+0000 7f01d4997700 1 -- 172.21.15.17:0/3211500961 shutdown_connections 2022-08-01T19:39:05.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.502+0000 7f01d4997700 1 -- 172.21.15.17:0/3211500961 wait complete. 2022-08-01T19:39:05.508 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 27 2022-08-01T19:39:05.516 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:39:05.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.614+0000 7f5a9de9b700 1 Processor -- start 2022-08-01T19:39:05.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.614+0000 7f5a9de9b700 1 -- start start 2022-08-01T19:39:05.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.614+0000 7f5a9de9b700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f5a980ad750 0x7f5a98061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:05.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.614+0000 7f5a9de9b700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a980aee80 con 0x7f5a981405d0 2022-08-01T19:39:05.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.614+0000 7f5a9de9b700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a980b70b0 con 0x7f5a980614b0 2022-08-01T19:39:05.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.614+0000 7f5a9de9b700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a980af0b0 con 0x7f5a980ad750 2022-08-01T19:39:05.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.614+0000 7f5a9d69a700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5a981405d0 0x7f5a981466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60080/0 (socket says 172.21.15.17:60080) 2022-08-01T19:39:05.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.614+0000 7f5a9d69a700 1 -- 172.21.15.17:0/3349457037 learned_addr learned my addr 172.21.15.17:0/3349457037 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:05.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.614+0000 7f5a97fff700 1 --2- 172.21.15.17:0/3349457037 >> v2:172.21.15.77:3300/0 conn(0x7f5a980ad750 0x7f5a98061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:05.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.614+0000 7f5a977fe700 1 -- 172.21.15.17:0/3349457037 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 890336820 0 0) 0x7f5a980aee80 con 0x7f5a981405d0 2022-08-01T19:39:05.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.614+0000 7f5a977fe700 1 -- 172.21.15.17:0/3349457037 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a7c003260 con 0x7f5a981405d0 2022-08-01T19:39:05.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.614+0000 7f5a97fff700 1 -- 172.21.15.17:0/3349457037 >> v1:172.21.15.77:6789/0 conn(0x7f5a980614b0 legacy=0x7f5a980b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:05.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.614+0000 7f5a97fff700 1 -- 172.21.15.17:0/3349457037 >> v1:172.21.15.17:6789/0 conn(0x7f5a981405d0 legacy=0x7f5a981466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:05.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.614+0000 7f5a97fff700 1 -- 172.21.15.17:0/3349457037 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a980af350 con 0x7f5a980ad750 2022-08-01T19:39:05.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.614+0000 7f5a97fff700 1 --2- 172.21.15.17:0/3349457037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a980ad750 0x7f5a98061890 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f5a84002f70 tx=0x7f5a84006df0).ready entity=mon.1 client_cookie=c8da6e3ed5913814 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:05.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.614+0000 7f5a977fe700 1 -- 172.21.15.17:0/3349457037 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a8400d610 con 0x7f5a980ad750 2022-08-01T19:39:05.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.614+0000 7f5a977fe700 1 -- 172.21.15.17:0/3349457037 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a84000e90 con 0x7f5a980ad750 2022-08-01T19:39:05.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.614+0000 7f5a977fe700 1 -- 172.21.15.17:0/3349457037 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a8400d960 con 0x7f5a980ad750 2022-08-01T19:39:05.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.614+0000 7f5a9de9b700 1 -- 172.21.15.17:0/3349457037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a980ad750 msgr2=0x7f5a98061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:05.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.614+0000 7f5a9de9b700 1 --2- 172.21.15.17:0/3349457037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a980ad750 0x7f5a98061890 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f5a84002f70 tx=0x7f5a84006df0).stop 2022-08-01T19:39:05.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.614+0000 7f5a9de9b700 1 -- 172.21.15.17:0/3349457037 shutdown_connections 2022-08-01T19:39:05.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.614+0000 7f5a9de9b700 1 --2- 172.21.15.17:0/3349457037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a980ad750 0x7f5a98061890 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:05.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.614+0000 7f5a9de9b700 1 -- 172.21.15.17:0/3349457037 >> 172.21.15.17:0/3349457037 conn(0x7f5a98066f70 msgr2=0x7f5a98067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:05.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.614+0000 7f5a9de9b700 1 -- 172.21.15.17:0/3349457037 shutdown_connections 2022-08-01T19:39:05.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.618+0000 7f5a9de9b700 1 -- 172.21.15.17:0/3349457037 wait complete. 2022-08-01T19:39:05.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.618+0000 7f5a9de9b700 1 Processor -- start 2022-08-01T19:39:05.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.618+0000 7f5a9de9b700 1 -- start start 2022-08-01T19:39:05.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.618+0000 7f5a9de9b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5a980614b0 0x7f5a98144790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:05.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.618+0000 7f5a9de9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a980ad750 0x7f5a981559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:05.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.618+0000 7f5a9de9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a981405d0 0x7f5a98155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:05.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.618+0000 7f5a9de9b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a980b3d90 con 0x7f5a980ad750 2022-08-01T19:39:05.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.618+0000 7f5a9de9b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a980afbd0 con 0x7f5a981405d0 2022-08-01T19:39:05.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.618+0000 7f5a9de9b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a980ad2c0 con 0x7f5a980614b0 2022-08-01T19:39:05.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.618+0000 7f5a97fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a980ad750 0x7f5a981559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:05.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.618+0000 7f5a97fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a980ad750 0x7f5a981559d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39656/0 (socket says 172.21.15.17:39656) 2022-08-01T19:39:05.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.618+0000 7f5a97fff700 1 -- 172.21.15.17:0/1236872747 learned_addr learned my addr 172.21.15.17:0/1236872747 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:05.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.618+0000 7f5a9d69a700 1 --2- 172.21.15.17:0/1236872747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a981405d0 0x7f5a98155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:05.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.618+0000 7f5a9ce99700 1 --2- 172.21.15.17:0/1236872747 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5a980614b0 0x7f5a98144790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:05.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.618+0000 7f5a97fff700 1 -- 172.21.15.17:0/1236872747 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5a980614b0 msgr2=0x7f5a98144790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:05.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.618+0000 7f5a97fff700 1 --2- 172.21.15.17:0/1236872747 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5a980614b0 0x7f5a98144790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:05.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.618+0000 7f5a97fff700 1 -- 172.21.15.17:0/1236872747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a981405d0 msgr2=0x7f5a98155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:05.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.618+0000 7f5a97fff700 1 --2- 172.21.15.17:0/1236872747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a981405d0 0x7f5a98155ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:05.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.618+0000 7f5a97fff700 1 -- 172.21.15.17:0/1236872747 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a8400b180 con 0x7f5a980ad750 2022-08-01T19:39:05.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.618+0000 7f5a9ce99700 1 --2- 172.21.15.17:0/1236872747 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5a980614b0 0x7f5a98144790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:39:05.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.622+0000 7f5a97fff700 1 --2- 172.21.15.17:0/1236872747 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a980ad750 0x7f5a981559d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f5a84002f70 tx=0x7f5a84014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:05.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.622+0000 7f5a95ffb700 1 -- 172.21.15.17:0/1236872747 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a84016070 con 0x7f5a980ad750 2022-08-01T19:39:05.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.622+0000 7f5a95ffb700 1 -- 172.21.15.17:0/1236872747 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8400de90 con 0x7f5a980ad750 2022-08-01T19:39:05.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.622+0000 7f5a9de9b700 1 -- 172.21.15.17:0/1236872747 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a980ac580 con 0x7f5a980ad750 2022-08-01T19:39:05.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.622+0000 7f5a95ffb700 1 -- 172.21.15.17:0/1236872747 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a84013e50 con 0x7f5a980ad750 2022-08-01T19:39:05.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.622+0000 7f5a9de9b700 1 -- 172.21.15.17:0/1236872747 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a980b40b0 con 0x7f5a980ad750 2022-08-01T19:39:05.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.622+0000 7f5a9de9b700 1 -- 172.21.15.17:0/1236872747 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a980af5f0 con 0x7f5a980ad750 2022-08-01T19:39:05.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.622+0000 7f5a95ffb700 1 -- 172.21.15.17:0/1236872747 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5a84024030 con 0x7f5a980ad750 2022-08-01T19:39:05.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.626+0000 7f5a95ffb700 1 --2- 172.21.15.17:0/1236872747 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5a78053a70 0x7f5a78055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:05.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.626+0000 7f5a95ffb700 1 -- 172.21.15.17:0/1236872747 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f5a8406c960 con 0x7f5a980ad750 2022-08-01T19:39:05.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.626+0000 7f5a9ce99700 1 --2- 172.21.15.17:0/1236872747 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5a78053a70 0x7f5a78055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:05.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.626+0000 7f5a9ce99700 1 --2- 172.21.15.17:0/1236872747 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5a78053a70 0x7f5a78055f30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f5a8c00b000 tx=0x7f5a8c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:05.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.630+0000 7f5a95ffb700 1 -- 172.21.15.17:0/1236872747 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5a84043960 con 0x7f5a980ad750 2022-08-01T19:39:05.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.794+0000 7f5a9de9b700 1 -- 172.21.15.17:0/1236872747 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a980b40b0 con 0x7f5a980ad750 2022-08-01T19:39:05.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.794+0000 7f5a95ffb700 1 -- 172.21.15.17:0/1236872747 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+1559 (secure 0 0 0) 0x7f5a980b40b0 con 0x7f5a980ad750 2022-08-01T19:39:05.799 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:39:05.799 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":27,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/1914125901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":1914125901},{"type":"v1","addr":"172.21.15.17:6833","nonce":1914125901}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":22},{"gid":5587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1403688289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1403688289},{"type":"v1","addr":"172.21.15.77:6835","nonce":1403688289}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":22},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.17:6835/260169708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":260169708},{"type":"v1","addr":"172.21.15.17:6835","nonce":260169708}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":27}],"filesystems":[]} 2022-08-01T19:39:05.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.798+0000 7f5a9de9b700 1 -- 172.21.15.17:0/1236872747 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5a78053a70 msgr2=0x7f5a78055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:05.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.798+0000 7f5a9de9b700 1 --2- 172.21.15.17:0/1236872747 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5a78053a70 0x7f5a78055f30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f5a8c00b000 tx=0x7f5a8c009040).stop 2022-08-01T19:39:05.802 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.798+0000 7f5a9de9b700 1 -- 172.21.15.17:0/1236872747 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a980ad750 msgr2=0x7f5a981559d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:05.802 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.798+0000 7f5a9de9b700 1 --2- 172.21.15.17:0/1236872747 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a980ad750 0x7f5a981559d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f5a84002f70 tx=0x7f5a84014000).stop 2022-08-01T19:39:05.802 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.798+0000 7f5a9de9b700 1 -- 172.21.15.17:0/1236872747 shutdown_connections 2022-08-01T19:39:05.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.798+0000 7f5a9de9b700 1 --2- 172.21.15.17:0/1236872747 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5a78053a70 0x7f5a78055f30 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:05.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.798+0000 7f5a9de9b700 1 --2- 172.21.15.17:0/1236872747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a981405d0 0x7f5a98155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:05.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.798+0000 7f5a9de9b700 1 --2- 172.21.15.17:0/1236872747 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a980ad750 0x7f5a981559d0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:05.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.798+0000 7f5a9de9b700 1 --2- 172.21.15.17:0/1236872747 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5a980614b0 0x7f5a98144790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:05.804 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.798+0000 7f5a9de9b700 1 -- 172.21.15.17:0/1236872747 >> 172.21.15.17:0/1236872747 conn(0x7f5a98066f70 msgr2=0x7f5a9813fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:05.804 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.798+0000 7f5a9de9b700 1 -- 172.21.15.17:0/1236872747 shutdown_connections 2022-08-01T19:39:05.804 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.798+0000 7f5a9de9b700 1 -- 172.21.15.17:0/1236872747 wait complete. 2022-08-01T19:39:05.805 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 27 2022-08-01T19:39:05.812 INFO:tasks.ceph.mds.a:Restarting daemon 2022-08-01T19:39:05.812 INFO:tasks.ceph.mds.a:Stopping old one... 2022-08-01T19:39:05.813 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-08-01T19:39:05.813 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:39:05.813 INFO:tasks.ceph.mds.c:Restarting daemon 2022-08-01T19:39:05.814 INFO:tasks.ceph.mds.c:Stopping old one... 2022-08-01T19:39:05.814 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-08-01T19:39:05.815 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:39:05.815 INFO:tasks.ceph.mds.b:Restarting daemon 2022-08-01T19:39:05.815 INFO:tasks.ceph.mds.b:Stopping old one... 2022-08-01T19:39:05.816 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-08-01T19:39:05.816 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:39:05.823 INFO:tasks.ceph.mds.a:Stopped 2022-08-01T19:39:05.824 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-08-01T19:39:05.826 INFO:tasks.ceph.mds.a:Started 2022-08-01T19:39:05.827 DEBUG:teuthology.parallel:result is None 2022-08-01T19:39:05.828 INFO:tasks.ceph.mds.c:Stopped 2022-08-01T19:39:05.828 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-08-01T19:39:05.830 INFO:tasks.ceph.mds.c:Started 2022-08-01T19:39:05.831 DEBUG:teuthology.parallel:result is None 2022-08-01T19:39:05.832 INFO:tasks.ceph.mds.b:Stopped 2022-08-01T19:39:05.832 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-08-01T19:39:05.835 INFO:tasks.ceph.mds.b:Started 2022-08-01T19:39:05.835 DEBUG:teuthology.parallel:result is None 2022-08-01T19:39:05.836 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-08-01T19:39:05.898 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2022-08-01T19:39:05.912 INFO:tasks.ceph.mds.b.smithi077.stdout:starting mds.b at 2022-08-01T19:39:05.913 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2022-08-01T19:39:05.948 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.946+0000 7fafddb30700 1 Processor -- start 2022-08-01T19:39:05.949 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.946+0000 7fafddb30700 1 -- start start 2022-08-01T19:39:05.952 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.946+0000 7fafddb30700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fafd8143e20 0x7fafd805f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:05.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.946+0000 7fafddb30700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafd80b3dd0 con 0x7fafd8060eb0 2022-08-01T19:39:05.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.946+0000 7fafddb30700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafd80b2720 con 0x7fafd80b2340 2022-08-01T19:39:05.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.946+0000 7fafddb30700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafd8063f80 con 0x7fafd8143e20 2022-08-01T19:39:05.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.946+0000 7fafd7fff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fafd8060eb0 0x7fafd812c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60114/0 (socket says 172.21.15.17:60114) 2022-08-01T19:39:05.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.946+0000 7fafd7fff700 1 -- 172.21.15.17:0/3102184795 learned_addr learned my addr 172.21.15.17:0/3102184795 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:05.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.946+0000 7fafdd32f700 1 --2- 172.21.15.17:0/3102184795 >> v2:172.21.15.77:3300/0 conn(0x7fafd8143e20 0x7fafd805f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:05.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.946+0000 7fafd77fe700 1 -- 172.21.15.17:0/3102184795 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1434756028 0 0) 0x7fafd80b3dd0 con 0x7fafd8060eb0 2022-08-01T19:39:05.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.946+0000 7fafd77fe700 1 -- 172.21.15.17:0/3102184795 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafbc003260 con 0x7fafd8060eb0 2022-08-01T19:39:05.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.946+0000 7fafd77fe700 1 -- 172.21.15.17:0/3102184795 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 119762764 0 0) 0x7fafd80b2720 con 0x7fafd80b2340 2022-08-01T19:39:05.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.946+0000 7fafd77fe700 1 -- 172.21.15.17:0/3102184795 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafd80b3dd0 con 0x7fafd80b2340 2022-08-01T19:39:05.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.946+0000 7fafd77fe700 1 -- 172.21.15.17:0/3102184795 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 875146402 0 0) 0x7fafbc003260 con 0x7fafd8060eb0 2022-08-01T19:39:05.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.946+0000 7fafdd32f700 1 -- 172.21.15.17:0/3102184795 >> v1:172.21.15.77:6789/0 conn(0x7fafd80b2340 legacy=0x7fafd80b36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:05.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.946+0000 7fafdd32f700 1 -- 172.21.15.17:0/3102184795 >> v1:172.21.15.17:6789/0 conn(0x7fafd8060eb0 legacy=0x7fafd812c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:05.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.946+0000 7fafdd32f700 1 -- 172.21.15.17:0/3102184795 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd80b2fc0 con 0x7fafd8143e20 2022-08-01T19:39:05.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.946+0000 7fafdd32f700 1 --2- 172.21.15.17:0/3102184795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafd8143e20 0x7fafd805f840 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7fafc4001110 tx=0x7fafd00091d0).ready entity=mon.1 client_cookie=f5650747a1f17f04 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:05.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.950+0000 7fafd77fe700 1 -- 172.21.15.17:0/3102184795 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafc4001660 con 0x7fafd8143e20 2022-08-01T19:39:05.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.950+0000 7fafd77fe700 1 -- 172.21.15.17:0/3102184795 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd000f510 con 0x7fafd8143e20 2022-08-01T19:39:05.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.950+0000 7fafd77fe700 1 -- 172.21.15.17:0/3102184795 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafd000f680 con 0x7fafd8143e20 2022-08-01T19:39:05.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.950+0000 7fafddb30700 1 -- 172.21.15.17:0/3102184795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafd8143e20 msgr2=0x7fafd805f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:05.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.950+0000 7fafddb30700 1 --2- 172.21.15.17:0/3102184795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafd8143e20 0x7fafd805f840 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7fafc4001110 tx=0x7fafd00091d0).stop 2022-08-01T19:39:05.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.950+0000 7fafddb30700 1 -- 172.21.15.17:0/3102184795 shutdown_connections 2022-08-01T19:39:05.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.950+0000 7fafddb30700 1 --2- 172.21.15.17:0/3102184795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafd8143e20 0x7fafd805f840 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:05.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.950+0000 7fafddb30700 1 -- 172.21.15.17:0/3102184795 >> 172.21.15.17:0/3102184795 conn(0x7fafd805c620 msgr2=0x7fafd805ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:05.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.950+0000 7fafddb30700 1 -- 172.21.15.17:0/3102184795 shutdown_connections 2022-08-01T19:39:05.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.950+0000 7fafddb30700 1 -- 172.21.15.17:0/3102184795 wait complete. 2022-08-01T19:39:05.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.950+0000 7fafddb30700 1 Processor -- start 2022-08-01T19:39:05.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.950+0000 7fafddb30700 1 -- start start 2022-08-01T19:39:05.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.950+0000 7fafddb30700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafd8060eb0 0x7fafd80b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:05.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.950+0000 7fafddb30700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fafd80b2340 0x7fafd81565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:05.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.950+0000 7fafddb30700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafd8143e20 0x7fafd80af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:05.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.950+0000 7fafddb30700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd80aeda0 con 0x7fafd8143e20 2022-08-01T19:39:05.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.950+0000 7fafddb30700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd8155f10 con 0x7fafd8060eb0 2022-08-01T19:39:05.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.950+0000 7fafddb30700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafd8064720 con 0x7fafd80b2340 2022-08-01T19:39:05.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.954+0000 7fafdd32f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafd8143e20 0x7fafd80af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:05.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.954+0000 7fafd7fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fafd80b2340 0x7fafd81565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:05.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.954+0000 7fafdd32f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafd8143e20 0x7fafd80af150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39696/0 (socket says 172.21.15.17:39696) 2022-08-01T19:39:05.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.954+0000 7fafdcb2e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafd8060eb0 0x7fafd80b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:05.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.954+0000 7fafdd32f700 1 -- 172.21.15.17:0/1003399365 learned_addr learned my addr 172.21.15.17:0/1003399365 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:05.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.954+0000 7fafdd32f700 1 -- 172.21.15.17:0/1003399365 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fafd80b2340 msgr2=0x7fafd81565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:05.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.954+0000 7fafdd32f700 1 --2- 172.21.15.17:0/1003399365 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fafd80b2340 0x7fafd81565a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:05.966 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.954+0000 7fafdd32f700 1 -- 172.21.15.17:0/1003399365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafd8060eb0 msgr2=0x7fafd80b56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:05.966 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.954+0000 7fafdd32f700 1 --2- 172.21.15.17:0/1003399365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafd8060eb0 0x7fafd80b56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:05.966 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.954+0000 7fafdd32f700 1 -- 172.21.15.17:0/1003399365 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd0009470 con 0x7fafd8143e20 2022-08-01T19:39:05.966 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.954+0000 7fafdd32f700 1 --2- 172.21.15.17:0/1003399365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafd8143e20 0x7fafd80af150 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7fafc4001110 tx=0x7fafd0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:05.967 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.954+0000 7fafd5ffb700 1 -- 172.21.15.17:0/1003399365 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafd00032d0 con 0x7fafd8143e20 2022-08-01T19:39:05.967 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.954+0000 7fafd5ffb700 1 -- 172.21.15.17:0/1003399365 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd0011b80 con 0x7fafd8143e20 2022-08-01T19:39:05.967 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.954+0000 7fafd5ffb700 1 -- 172.21.15.17:0/1003399365 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafd0021440 con 0x7fafd8143e20 2022-08-01T19:39:05.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.954+0000 7fafddb30700 1 -- 172.21.15.17:0/1003399365 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafd80649c0 con 0x7fafd8143e20 2022-08-01T19:39:05.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.954+0000 7fafddb30700 1 -- 172.21.15.17:0/1003399365 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafd8156100 con 0x7fafd8143e20 2022-08-01T19:39:05.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.954+0000 7fafddb30700 1 -- 172.21.15.17:0/1003399365 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafd81562c0 con 0x7fafd8143e20 2022-08-01T19:39:05.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.954+0000 7fafd5ffb700 1 -- 172.21.15.17:0/1003399365 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fafd0027020 con 0x7fafd8143e20 2022-08-01T19:39:05.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.954+0000 7fafd5ffb700 1 --2- 172.21.15.17:0/1003399365 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fafb8053950 0x7fafb8055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:05.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.954+0000 7fafd5ffb700 1 -- 172.21.15.17:0/1003399365 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fafd006cd50 con 0x7fafd8143e20 2022-08-01T19:39:05.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.954+0000 7fafdcb2e700 1 --2- 172.21.15.17:0/1003399365 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fafb8053950 0x7fafb8055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:05.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.958+0000 7fafdcb2e700 1 --2- 172.21.15.17:0/1003399365 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fafb8053950 0x7fafb8055e10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7fafcc00d000 tx=0x7fafcc007a20).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:05.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:05.962+0000 7fafd5ffb700 1 -- 172.21.15.17:0/1003399365 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fafd0043df0 con 0x7fafd8143e20 2022-08-01T19:39:06.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:06.098+0000 7fafddb30700 1 -- 172.21.15.17:0/1003399365 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fafd8061d60 con 0x7fafd8143e20 2022-08-01T19:39:06.951 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:06.946+0000 7fafd5ffb700 1 -- 172.21.15.17:0/1003399365 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v62) v1 ==== 97+0+0 (secure 0 0 0) 0x7fafd0015020 con 0x7fafd8143e20 2022-08-01T19:39:06.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:06.950+0000 7fafddb30700 1 -- 172.21.15.17:0/1003399365 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fafb8053950 msgr2=0x7fafb8055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:06.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:06.950+0000 7fafddb30700 1 --2- 172.21.15.17:0/1003399365 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fafb8053950 0x7fafb8055e10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7fafcc00d000 tx=0x7fafcc007a20).stop 2022-08-01T19:39:06.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:06.954+0000 7fafddb30700 1 -- 172.21.15.17:0/1003399365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafd8143e20 msgr2=0x7fafd80af150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:06.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:06.954+0000 7fafddb30700 1 --2- 172.21.15.17:0/1003399365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafd8143e20 0x7fafd80af150 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7fafc4001110 tx=0x7fafd0016000).stop 2022-08-01T19:39:06.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:06.954+0000 7fafddb30700 1 -- 172.21.15.17:0/1003399365 shutdown_connections 2022-08-01T19:39:06.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:06.954+0000 7fafddb30700 1 --2- 172.21.15.17:0/1003399365 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fafb8053950 0x7fafb8055e10 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:06.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:06.954+0000 7fafddb30700 1 --2- 172.21.15.17:0/1003399365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafd8143e20 0x7fafd80af150 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:06.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:06.954+0000 7fafddb30700 1 --2- 172.21.15.17:0/1003399365 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fafd80b2340 0x7fafd81565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:06.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:06.954+0000 7fafddb30700 1 --2- 172.21.15.17:0/1003399365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafd8060eb0 0x7fafd80b56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:06.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:06.954+0000 7fafddb30700 1 -- 172.21.15.17:0/1003399365 >> 172.21.15.17:0/1003399365 conn(0x7fafd805c620 msgr2=0x7fafd81430b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:06.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:06.954+0000 7fafddb30700 1 -- 172.21.15.17:0/1003399365 shutdown_connections 2022-08-01T19:39:06.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:06.954+0000 7fafddb30700 1 -- 172.21.15.17:0/1003399365 wait complete. 2022-08-01T19:39:06.959 INFO:teuthology.orchestra.run.smithi017.stderr: removed all blacklist entries 2022-08-01T19:39:06.970 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-08-01T19:39:06.971 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-08-01T19:39:07.080 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc19cb9700 1 Processor -- start 2022-08-01T19:39:07.080 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc19cb9700 1 -- start start 2022-08-01T19:39:07.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc19cb9700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcc140b3070 0x7fcc14065720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:07.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc19cb9700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fcc1414b040 0x7fcc1414b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:07.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc19cb9700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc140b2c20 con 0x7fcc140b7840 2022-08-01T19:39:07.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc19cb9700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc14061830 con 0x7fcc1414b040 2022-08-01T19:39:07.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc19cb9700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc14065c10 con 0x7fcc140b3070 2022-08-01T19:39:07.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc18cb7700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fcc140b7840 0x7fcc14061120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60130/0 (socket says 172.21.15.17:60130) 2022-08-01T19:39:07.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc18cb7700 1 -- 172.21.15.17:0/2447644288 learned_addr learned my addr 172.21.15.17:0/2447644288 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:07.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc13fff700 1 --2- 172.21.15.17:0/2447644288 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcc140b3070 0x7fcc14065720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:07.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc194b8700 1 --2- 172.21.15.17:0/2447644288 >> v2:172.21.15.77:3300/0 conn(0x7fcc1414b040 0x7fcc1414b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:07.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc137fe700 1 -- 172.21.15.17:0/2447644288 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3302683064 0 0) 0x7fcc140b2c20 con 0x7fcc140b7840 2022-08-01T19:39:07.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc137fe700 1 -- 172.21.15.17:0/2447644288 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbf4003260 con 0x7fcc140b7840 2022-08-01T19:39:07.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc13fff700 1 -- 172.21.15.17:0/2447644288 >> v2:172.21.15.77:3300/0 conn(0x7fcc1414b040 msgr2=0x7fcc1414b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:07.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc13fff700 1 --2- 172.21.15.17:0/2447644288 >> v2:172.21.15.77:3300/0 conn(0x7fcc1414b040 0x7fcc1414b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:07.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc13fff700 1 -- 172.21.15.17:0/2447644288 >> v1:172.21.15.17:6789/0 conn(0x7fcc140b7840 legacy=0x7fcc14061120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:07.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc13fff700 1 -- 172.21.15.17:0/2447644288 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc140b7c20 con 0x7fcc140b3070 2022-08-01T19:39:07.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc13fff700 1 --2- 172.21.15.17:0/2447644288 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcc140b3070 0x7fcc14065720 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fcbfc002f70 tx=0x7fcbfc006df0).ready entity=mon.2 client_cookie=1ba7a84cc9966b47 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:07.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc137fe700 1 -- 172.21.15.17:0/2447644288 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcbfc00d4a0 con 0x7fcc140b3070 2022-08-01T19:39:07.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc137fe700 1 -- 172.21.15.17:0/2447644288 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbfc000e90 con 0x7fcc140b3070 2022-08-01T19:39:07.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc137fe700 1 -- 172.21.15.17:0/2447644288 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcbfc00d830 con 0x7fcc140b3070 2022-08-01T19:39:07.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc19cb9700 1 -- 172.21.15.17:0/2447644288 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcc140b3070 msgr2=0x7fcc14065720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:07.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc19cb9700 1 --2- 172.21.15.17:0/2447644288 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcc140b3070 0x7fcc14065720 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fcbfc002f70 tx=0x7fcbfc006df0).stop 2022-08-01T19:39:07.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc19cb9700 1 -- 172.21.15.17:0/2447644288 shutdown_connections 2022-08-01T19:39:07.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc19cb9700 1 --2- 172.21.15.17:0/2447644288 >> v2:172.21.15.77:3300/0 conn(0x7fcc1414b040 0x7fcc1414b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:07.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc19cb9700 1 --2- 172.21.15.17:0/2447644288 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcc140b3070 0x7fcc14065720 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:07.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc19cb9700 1 -- 172.21.15.17:0/2447644288 >> 172.21.15.17:0/2447644288 conn(0x7fcc1405f840 msgr2=0x7fcc1405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:07.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc19cb9700 1 -- 172.21.15.17:0/2447644288 shutdown_connections 2022-08-01T19:39:07.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc19cb9700 1 -- 172.21.15.17:0/2447644288 wait complete. 2022-08-01T19:39:07.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc19cb9700 1 Processor -- start 2022-08-01T19:39:07.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc19cb9700 1 -- start start 2022-08-01T19:39:07.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc19cb9700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcc140b3070 0x7fcc140a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:07.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc19cb9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc140b7840 0x7fcc140aa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:07.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc19cb9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc1414b040 0x7fcc140aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:07.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc19cb9700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc140b0940 con 0x7fcc1414b040 2022-08-01T19:39:07.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc19cb9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc14157460 con 0x7fcc140b7840 2022-08-01T19:39:07.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc19cb9700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc140ab190 con 0x7fcc140b3070 2022-08-01T19:39:07.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc13fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc140b7840 0x7fcc140aa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:07.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc194b8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc1414b040 0x7fcc140aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:07.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc13fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc140b7840 0x7fcc140aa7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:37634/0 (socket says 172.21.15.17:37634) 2022-08-01T19:39:07.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc13fff700 1 -- 172.21.15.17:0/3883144626 learned_addr learned my addr 172.21.15.17:0/3883144626 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:07.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.082+0000 7fcc18cb7700 1 --2- 172.21.15.17:0/3883144626 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcc140b3070 0x7fcc140a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:07.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.086+0000 7fcc13fff700 1 -- 172.21.15.17:0/3883144626 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcc140b3070 msgr2=0x7fcc140a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:07.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.086+0000 7fcc13fff700 1 --2- 172.21.15.17:0/3883144626 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcc140b3070 0x7fcc140a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:07.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.086+0000 7fcc13fff700 1 -- 172.21.15.17:0/3883144626 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc1414b040 msgr2=0x7fcc140aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:07.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.086+0000 7fcc13fff700 1 --2- 172.21.15.17:0/3883144626 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc1414b040 0x7fcc140aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:07.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.086+0000 7fcc13fff700 1 -- 172.21.15.17:0/3883144626 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbfc00b180 con 0x7fcc140b7840 2022-08-01T19:39:07.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.086+0000 7fcc13fff700 1 --2- 172.21.15.17:0/3883144626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc140b7840 0x7fcc140aa7b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7fcbfc002f70 tx=0x7fcbfc014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:07.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.086+0000 7fcc11ffb700 1 -- 172.21.15.17:0/3883144626 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcbfc013e90 con 0x7fcc140b7840 2022-08-01T19:39:07.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.086+0000 7fcc11ffb700 1 -- 172.21.15.17:0/3883144626 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbfc00de50 con 0x7fcc140b7840 2022-08-01T19:39:07.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.086+0000 7fcc11ffb700 1 -- 172.21.15.17:0/3883144626 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcbfc01e5f0 con 0x7fcc140b7840 2022-08-01T19:39:07.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.086+0000 7fcc19cb9700 1 -- 172.21.15.17:0/3883144626 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc1409f050 con 0x7fcc140b7840 2022-08-01T19:39:07.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.086+0000 7fcc19cb9700 1 -- 172.21.15.17:0/3883144626 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc1409ebe0 con 0x7fcc140b7840 2022-08-01T19:39:07.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.086+0000 7fcc19cb9700 1 -- 172.21.15.17:0/3883144626 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc140a25a0 con 0x7fcc140b7840 2022-08-01T19:39:07.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.086+0000 7fcc11ffb700 1 -- 172.21.15.17:0/3883144626 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcbfc01c070 con 0x7fcc140b7840 2022-08-01T19:39:07.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.086+0000 7fcc11ffb700 1 --2- 172.21.15.17:0/3883144626 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcbf8053a70 0x7fcbf8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:07.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.090+0000 7fcc11ffb700 1 -- 172.21.15.17:0/3883144626 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fcbfc06c840 con 0x7fcc140b7840 2022-08-01T19:39:07.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.090+0000 7fcc18cb7700 1 --2- 172.21.15.17:0/3883144626 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcbf8053a70 0x7fcbf8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:07.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.090+0000 7fcc18cb7700 1 --2- 172.21.15.17:0/3883144626 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcbf8053a70 0x7fcbf8055f30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7fcc0400d000 tx=0x7fcc04009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:07.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.098+0000 7fcc11ffb700 1 -- 172.21.15.17:0/3883144626 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcbfc043830 con 0x7fcc140b7840 2022-08-01T19:39:07.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.266+0000 7fcc19cb9700 1 -- 172.21.15.17:0/3883144626 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fcc140b2980 con 0x7fcc140b7840 2022-08-01T19:39:07.272 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.270+0000 7fcc11ffb700 1 -- 172.21.15.17:0/3883144626 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+4255 (secure 0 0 0) 0x7fcbfc023020 con 0x7fcc140b7840 2022-08-01T19:39:07.272 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:39:07.273 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:39:07.273 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_dump": [ 2022-08-01T19:39:07.274 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:07.274 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.a", 2022-08-01T19:39:07.274 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhijotxORAAVc04rsrlFj0NShsyc6Qbhg==", 2022-08-01T19:39:07.275 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:39:07.275 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:39:07.276 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:39:07.276 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:39:07.277 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:39:07.277 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:07.278 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:07.278 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:07.278 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.b", 2022-08-01T19:39:07.279 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhiud2xBxAAcn6/O5ysdIp/oIbF5Y1dAw==", 2022-08-01T19:39:07.279 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:39:07.280 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:39:07.280 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:39:07.280 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:39:07.280 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:39:07.281 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:07.281 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:07.281 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:07.281 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.c", 2022-08-01T19:39:07.282 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+KuhiMzMbBBAAfdT5mtpYwboNL+7f2B09KQ==", 2022-08-01T19:39:07.282 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:39:07.282 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:39:07.282 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:39:07.283 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:39:07.283 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:39:07.283 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:07.283 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:07.284 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:07.284 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.0", 2022-08-01T19:39:07.284 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhia27/NRAAZXBp+nYelJVJ6Mge9r+3Vg==", 2022-08-01T19:39:07.284 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:39:07.285 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:39:07.285 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:39:07.285 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:39:07.285 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:07.286 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:07.286 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:07.286 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.1", 2022-08-01T19:39:07.286 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCBKuhiOggRGRAAvgA0MxwBJwmpXV6S9DTdjA==", 2022-08-01T19:39:07.287 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:39:07.287 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:39:07.287 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:39:07.287 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:39:07.288 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:07.288 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:07.288 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:07.288 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.2", 2022-08-01T19:39:07.288 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCDKuhi5PgOORAAhcORK4bW2T3dXPeYjcCm+g==", 2022-08-01T19:39:07.289 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:39:07.289 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:39:07.289 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:39:07.289 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:39:07.290 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:07.290 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:07.290 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:07.291 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.3", 2022-08-01T19:39:07.291 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCGKuhiutn6GxAAk2ucLuGbT5YV/R4xXQGh0w==", 2022-08-01T19:39:07.291 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:39:07.291 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:39:07.292 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:39:07.292 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:39:07.292 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:07.292 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:07.293 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:07.293 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.4", 2022-08-01T19:39:07.293 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCLKuhiGHTRHxAArr3TwXA0gsCAAF/Qs38wUQ==", 2022-08-01T19:39:07.293 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:39:07.294 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:39:07.294 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:39:07.294 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:39:07.294 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:07.295 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:07.295 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:07.295 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.5", 2022-08-01T19:39:07.295 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCOKuhiMMDbBBAAvTzBvOcv7pFWm/P/lNqB0w==", 2022-08-01T19:39:07.295 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:39:07.296 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:39:07.296 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:39:07.296 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:39:07.297 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:07.297 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:07.297 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:07.297 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.6", 2022-08-01T19:39:07.298 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCQKuhiG2H3IxAAufUcWqo9ULquxIrUD5vSfA==", 2022-08-01T19:39:07.298 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:39:07.298 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:39:07.298 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:39:07.299 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:39:07.299 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:07.299 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:07.299 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:07.300 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.7", 2022-08-01T19:39:07.300 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCTKuhiU50LChAAXuhrYBbHkk3eKzAPiEJFTg==", 2022-08-01T19:39:07.300 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:39:07.300 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:39:07.300 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:39:07.301 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:39:07.301 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:07.301 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:07.301 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:07.302 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.0", 2022-08-01T19:39:07.302 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhig3EnCxAANM20jSIQf/AKwuRX1qwT/A==", 2022-08-01T19:39:07.302 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:39:07.302 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:39:07.302 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow r", 2022-08-01T19:39:07.303 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T19:39:07.303 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:07.303 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:07.303 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:07.304 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.1", 2022-08-01T19:39:07.304 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+KuhicJR0DhAAivqEAZUhXnSwjVRwWm2bJg==", 2022-08-01T19:39:07.304 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:39:07.305 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:39:07.305 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow r", 2022-08-01T19:39:07.305 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T19:39:07.305 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:07.306 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:07.306 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:07.306 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.admin", 2022-08-01T19:39:07.307 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhiPTTGHxAAgfthCR6RX6j8F1YseFGNeA==", 2022-08-01T19:39:07.307 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:39:07.307 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:39:07.307 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:39:07.308 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:39:07.308 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:39:07.308 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:07.308 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:07.308 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:07.309 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.x", 2022-08-01T19:39:07.309 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9Kuhi2jvCNhAAyUxiGByZ+OZr8Y7dh3Qt5w==", 2022-08-01T19:39:07.309 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:39:07.309 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:39:07.310 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2022-08-01T19:39:07.310 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:39:07.310 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:07.310 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:07.311 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:07.311 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.y", 2022-08-01T19:39:07.311 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhiDJXpMhAA8aek67ydbRgNO0h+miws+w==", 2022-08-01T19:39:07.311 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:39:07.311 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:39:07.312 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2022-08-01T19:39:07.312 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:39:07.312 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:07.312 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:07.313 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:07.313 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:39:07.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.270+0000 7fcc19cb9700 1 -- 172.21.15.17:0/3883144626 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcbf8053a70 msgr2=0x7fcbf8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:07.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.270+0000 7fcc19cb9700 1 --2- 172.21.15.17:0/3883144626 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcbf8053a70 0x7fcbf8055f30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7fcc0400d000 tx=0x7fcc04009040).stop 2022-08-01T19:39:07.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.270+0000 7fcc19cb9700 1 -- 172.21.15.17:0/3883144626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc140b7840 msgr2=0x7fcc140aa7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:07.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.270+0000 7fcc19cb9700 1 --2- 172.21.15.17:0/3883144626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc140b7840 0x7fcc140aa7b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7fcbfc002f70 tx=0x7fcbfc014000).stop 2022-08-01T19:39:07.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.270+0000 7fcc19cb9700 1 -- 172.21.15.17:0/3883144626 shutdown_connections 2022-08-01T19:39:07.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.270+0000 7fcc19cb9700 1 --2- 172.21.15.17:0/3883144626 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcbf8053a70 0x7fcbf8055f30 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:07.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.270+0000 7fcc19cb9700 1 --2- 172.21.15.17:0/3883144626 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc1414b040 0x7fcc140aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:07.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.270+0000 7fcc19cb9700 1 --2- 172.21.15.17:0/3883144626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc140b7840 0x7fcc140aa7b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:07.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.270+0000 7fcc19cb9700 1 --2- 172.21.15.17:0/3883144626 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcc140b3070 0x7fcc140a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:07.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.270+0000 7fcc19cb9700 1 -- 172.21.15.17:0/3883144626 >> 172.21.15.17:0/3883144626 conn(0x7fcc1405f840 msgr2=0x7fcc140b47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:07.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.270+0000 7fcc19cb9700 1 -- 172.21.15.17:0/3883144626 shutdown_connections 2022-08-01T19:39:07.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.270+0000 7fcc19cb9700 1 -- 172.21.15.17:0/3883144626 wait complete. 2022-08-01T19:39:07.319 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:39:07.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.422+0000 7fc39c820700 1 Processor -- start 2022-08-01T19:39:07.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.422+0000 7fc39c820700 1 -- start start 2022-08-01T19:39:07.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.422+0000 7fc39c820700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fc3980b2710 0x7fc39814afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:07.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.422+0000 7fc39c820700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc3980b01a0 0x7fc39814b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:07.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.422+0000 7fc39c820700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3980a7250 con 0x7fc3980afd60 2022-08-01T19:39:07.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.422+0000 7fc39c820700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3980a7480 con 0x7fc3980b01a0 2022-08-01T19:39:07.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.422+0000 7fc39c820700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3980a7720 con 0x7fc3980b2710 2022-08-01T19:39:07.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.422+0000 7fc39659c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc3980b01a0 0x7fc39814b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:07.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.422+0000 7fc39659c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc3980b01a0 0x7fc39814b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39718/0 (socket says 172.21.15.17:39718) 2022-08-01T19:39:07.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.422+0000 7fc39659c700 1 -- 172.21.15.17:0/3037395321 learned_addr learned my addr 172.21.15.17:0/3037395321 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:07.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.422+0000 7fc396d9d700 1 --2- 172.21.15.17:0/3037395321 >> v2:172.21.15.77:3300/0 conn(0x7fc3980b2710 0x7fc39814afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:07.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.422+0000 7fc39659c700 1 -- 172.21.15.17:0/3037395321 >> v2:172.21.15.77:3300/0 conn(0x7fc3980b2710 msgr2=0x7fc39814afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:07.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.422+0000 7fc39659c700 1 --2- 172.21.15.17:0/3037395321 >> v2:172.21.15.77:3300/0 conn(0x7fc3980b2710 0x7fc39814afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:07.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.422+0000 7fc39659c700 1 -- 172.21.15.17:0/3037395321 >> v1:172.21.15.77:6789/0 conn(0x7fc3980afd60 legacy=0x7fc3980a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:07.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.422+0000 7fc39659c700 1 -- 172.21.15.17:0/3037395321 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3980a79c0 con 0x7fc3980b01a0 2022-08-01T19:39:07.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.422+0000 7fc396d9d700 1 --2- 172.21.15.17:0/3037395321 >> v2:172.21.15.77:3300/0 conn(0x7fc3980b2710 0x7fc39814afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:39:07.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.422+0000 7fc39659c700 1 --2- 172.21.15.17:0/3037395321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3980b01a0 0x7fc39814b4c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7fc388002f70 tx=0x7fc388006eb0).ready entity=mon.0 client_cookie=a546986b666986a6 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:07.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.422+0000 7fc395d9b700 1 -- 172.21.15.17:0/3037395321 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc38800d610 con 0x7fc3980b01a0 2022-08-01T19:39:07.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.422+0000 7fc395d9b700 1 -- 172.21.15.17:0/3037395321 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc388000e90 con 0x7fc3980b01a0 2022-08-01T19:39:07.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.422+0000 7fc395d9b700 1 -- 172.21.15.17:0/3037395321 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc38800d9a0 con 0x7fc3980b01a0 2022-08-01T19:39:07.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.422+0000 7fc39c820700 1 -- 172.21.15.17:0/3037395321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3980b01a0 msgr2=0x7fc39814b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:07.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.422+0000 7fc39c820700 1 --2- 172.21.15.17:0/3037395321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3980b01a0 0x7fc39814b4c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7fc388002f70 tx=0x7fc388006eb0).stop 2022-08-01T19:39:07.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.422+0000 7fc39c820700 1 -- 172.21.15.17:0/3037395321 shutdown_connections 2022-08-01T19:39:07.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.422+0000 7fc39c820700 1 --2- 172.21.15.17:0/3037395321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3980b01a0 0x7fc39814b4c0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:07.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.422+0000 7fc39c820700 1 --2- 172.21.15.17:0/3037395321 >> v2:172.21.15.77:3300/0 conn(0x7fc3980b2710 0x7fc39814afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:07.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.422+0000 7fc39c820700 1 -- 172.21.15.17:0/3037395321 >> 172.21.15.17:0/3037395321 conn(0x7fc398060310 msgr2=0x7fc3980b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:07.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.422+0000 7fc39c820700 1 -- 172.21.15.17:0/3037395321 shutdown_connections 2022-08-01T19:39:07.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.422+0000 7fc39c820700 1 -- 172.21.15.17:0/3037395321 wait complete. 2022-08-01T19:39:07.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.426+0000 7fc39c820700 1 Processor -- start 2022-08-01T19:39:07.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.426+0000 7fc39c820700 1 -- start start 2022-08-01T19:39:07.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.430+0000 7fc39c820700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3980afd60 0x7fc3980aa980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:07.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.430+0000 7fc39c820700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3980b01a0 0x7fc3980a9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:07.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.430+0000 7fc39c820700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc3980b2710 0x7fc398156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:07.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.430+0000 7fc39c820700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc398159c60 con 0x7fc3980afd60 2022-08-01T19:39:07.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.430+0000 7fc39c820700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc398159f00 con 0x7fc3980b01a0 2022-08-01T19:39:07.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.430+0000 7fc39c820700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc39815a1a0 con 0x7fc3980b2710 2022-08-01T19:39:07.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.430+0000 7fc396d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3980afd60 0x7fc3980aa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:07.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.430+0000 7fc396d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3980afd60 0x7fc3980aa980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39724/0 (socket says 172.21.15.17:39724) 2022-08-01T19:39:07.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.430+0000 7fc396d9d700 1 -- 172.21.15.17:0/1388736292 learned_addr learned my addr 172.21.15.17:0/1388736292 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:07.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.430+0000 7fc39659c700 1 --2- 172.21.15.17:0/1388736292 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3980b01a0 0x7fc3980a9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:07.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.430+0000 7fc39759e700 1 --2- 172.21.15.17:0/1388736292 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc3980b2710 0x7fc398156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:07.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.430+0000 7fc396d9d700 1 -- 172.21.15.17:0/1388736292 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc3980b2710 msgr2=0x7fc398156600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:07.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.430+0000 7fc396d9d700 1 --2- 172.21.15.17:0/1388736292 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc3980b2710 0x7fc398156600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:07.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.430+0000 7fc396d9d700 1 -- 172.21.15.17:0/1388736292 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3980b01a0 msgr2=0x7fc3980a9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:07.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.430+0000 7fc396d9d700 1 --2- 172.21.15.17:0/1388736292 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3980b01a0 0x7fc3980a9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:07.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.430+0000 7fc396d9d700 1 -- 172.21.15.17:0/1388736292 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38800b1e0 con 0x7fc3980afd60 2022-08-01T19:39:07.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.430+0000 7fc396d9d700 1 --2- 172.21.15.17:0/1388736292 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3980afd60 0x7fc3980aa980 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7fc390007000 tx=0x7fc390008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:07.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.430+0000 7fc38ffff700 1 -- 172.21.15.17:0/1388736292 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc390007600 con 0x7fc3980afd60 2022-08-01T19:39:07.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.430+0000 7fc38ffff700 1 -- 172.21.15.17:0/1388736292 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc390007760 con 0x7fc3980afd60 2022-08-01T19:39:07.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.430+0000 7fc39c820700 1 -- 172.21.15.17:0/1388736292 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc39815a440 con 0x7fc3980afd60 2022-08-01T19:39:07.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.430+0000 7fc38ffff700 1 -- 172.21.15.17:0/1388736292 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc390007ac0 con 0x7fc3980afd60 2022-08-01T19:39:07.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.430+0000 7fc39c820700 1 -- 172.21.15.17:0/1388736292 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc398064420 con 0x7fc3980afd60 2022-08-01T19:39:07.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.430+0000 7fc38ffff700 1 -- 172.21.15.17:0/1388736292 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc390007c20 con 0x7fc3980afd60 2022-08-01T19:39:07.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.430+0000 7fc39c820700 1 -- 172.21.15.17:0/1388736292 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc39815a5d0 con 0x7fc3980afd60 2022-08-01T19:39:07.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.430+0000 7fc38ffff700 1 --2- 172.21.15.17:0/1388736292 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc37c053a70 0x7fc37c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:07.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.434+0000 7fc38ffff700 1 -- 172.21.15.17:0/1388736292 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fc39006c300 con 0x7fc3980afd60 2022-08-01T19:39:07.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.434+0000 7fc39659c700 1 --2- 172.21.15.17:0/1388736292 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc37c053a70 0x7fc37c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:07.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.434+0000 7fc39659c700 1 --2- 172.21.15.17:0/1388736292 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc37c053a70 0x7fc37c055f30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7fc38801a000 tx=0x7fc388015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:07.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.438+0000 7fc38ffff700 1 -- 172.21.15.17:0/1388736292 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc3900432f0 con 0x7fc3980afd60 2022-08-01T19:39:07.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.570+0000 7fc39c820700 1 -- 172.21.15.17:0/1388736292 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3980574e0 con 0x7fc3980afd60 2022-08-01T19:39:07.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.570+0000 7fc38ffff700 1 -- 172.21.15.17:0/1388736292 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+13415 (secure 0 0 0) 0x7fc390046910 con 0x7fc3980afd60 2022-08-01T19:39:07.574 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:39:07.574 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":62,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:39:06.946479+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:39:07.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.574+0000 7fc39c820700 1 -- 172.21.15.17:0/1388736292 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc37c053a70 msgr2=0x7fc37c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:07.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.574+0000 7fc39c820700 1 --2- 172.21.15.17:0/1388736292 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc37c053a70 0x7fc37c055f30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7fc38801a000 tx=0x7fc388015000).stop 2022-08-01T19:39:07.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.574+0000 7fc39c820700 1 -- 172.21.15.17:0/1388736292 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3980afd60 msgr2=0x7fc3980aa980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:07.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.574+0000 7fc39c820700 1 --2- 172.21.15.17:0/1388736292 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3980afd60 0x7fc3980aa980 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7fc390007000 tx=0x7fc390008040).stop 2022-08-01T19:39:07.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.574+0000 7fc39c820700 1 -- 172.21.15.17:0/1388736292 shutdown_connections 2022-08-01T19:39:07.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.574+0000 7fc39c820700 1 --2- 172.21.15.17:0/1388736292 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc37c053a70 0x7fc37c055f30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:07.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.574+0000 7fc39c820700 1 --2- 172.21.15.17:0/1388736292 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc3980b2710 0x7fc398156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:07.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.574+0000 7fc39c820700 1 --2- 172.21.15.17:0/1388736292 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3980b01a0 0x7fc3980a9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:07.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.574+0000 7fc39c820700 1 --2- 172.21.15.17:0/1388736292 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3980afd60 0x7fc3980aa980 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:07.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.574+0000 7fc39c820700 1 -- 172.21.15.17:0/1388736292 >> 172.21.15.17:0/1388736292 conn(0x7fc398060310 msgr2=0x7fc39814b150 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:07.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.574+0000 7fc39c820700 1 -- 172.21.15.17:0/1388736292 shutdown_connections 2022-08-01T19:39:07.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.574+0000 7fc39c820700 1 -- 172.21.15.17:0/1388736292 wait complete. 2022-08-01T19:39:07.592 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-08-01T19:39:07.593 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:39:07.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.682+0000 7f167e62d700 1 Processor -- start 2022-08-01T19:39:07.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.682+0000 7f167e62d700 1 -- start start 2022-08-01T19:39:07.692 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f167e62d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f16780b3340 0x7f167813e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:07.692 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f167e62d700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16780ada80 con 0x7f1678145220 2022-08-01T19:39:07.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f167e62d700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16780b0090 con 0x7f16780b3720 2022-08-01T19:39:07.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f167e62d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16781456f0 con 0x7f16780b3340 2022-08-01T19:39:07.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f167cbca700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1678145220 0x7f167814bbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60162/0 (socket says 172.21.15.17:60162) 2022-08-01T19:39:07.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f167cbca700 1 -- 172.21.15.17:0/2555803482 learned_addr learned my addr 172.21.15.17:0/2555803482 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:07.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f1677fff700 1 --2- 172.21.15.17:0/2555803482 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f16780b3340 0x7f167813e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:07.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f1676ffd700 1 -- 172.21.15.17:0/2555803482 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2016194184 0 0) 0x7f16780ada80 con 0x7f1678145220 2022-08-01T19:39:07.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f1676ffd700 1 -- 172.21.15.17:0/2555803482 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1664003260 con 0x7f1678145220 2022-08-01T19:39:07.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f1676ffd700 1 -- 172.21.15.17:0/2555803482 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2546751470 0 0) 0x7f1664003260 con 0x7f1678145220 2022-08-01T19:39:07.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f1676ffd700 1 -- 172.21.15.17:0/2555803482 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f16780b3340 msgr2=0x7f167813e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:07.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f1676ffd700 1 --2- 172.21.15.17:0/2555803482 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f16780b3340 0x7f167813e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:07.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f1676ffd700 1 -- 172.21.15.17:0/2555803482 >> v1:172.21.15.77:6789/0 conn(0x7f16780b3720 legacy=0x7f1678141590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:07.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f1676ffd700 1 -- 172.21.15.17:0/2555803482 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16780b02f0 con 0x7f1678145220 2022-08-01T19:39:07.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f1676ffd700 1 -- 172.21.15.17:0/2555803482 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7f16700019e0 con 0x7f1678145220 2022-08-01T19:39:07.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f1677fff700 1 --2- 172.21.15.17:0/2555803482 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f16780b3340 0x7f167813e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:39:07.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f1676ffd700 1 -- 172.21.15.17:0/2555803482 >> v1:172.21.15.17:6789/0 conn(0x7f1678145220 legacy=0x7f167814bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:07.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f1676ffd700 1 --2- 172.21.15.17:0/2555803482 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1664003780 0x7f1664003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:07.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f1676ffd700 1 --2- 172.21.15.17:0/2555803482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1664004500 0x7f1664004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:07.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f1676ffd700 1 --2- 172.21.15.17:0/2555803482 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f16780b3340 0x7f1664004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:07.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f1676ffd700 1 -- 172.21.15.17:0/2555803482 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16781456f0 con 0x7f1664003780 2022-08-01T19:39:07.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f1676ffd700 1 -- 172.21.15.17:0/2555803482 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166400b750 con 0x7f1664004500 2022-08-01T19:39:07.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f1676ffd700 1 -- 172.21.15.17:0/2555803482 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f166400b8c0 con 0x7f16780b3340 2022-08-01T19:39:07.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f1677fff700 1 --2- 172.21.15.17:0/2555803482 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1664003780 0x7f1664003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:07.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f167cbca700 1 --2- 172.21.15.17:0/2555803482 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f16780b3340 0x7f1664004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:07.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f16777fe700 1 --2- 172.21.15.17:0/2555803482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1664004500 0x7f1664004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:07.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f1677fff700 1 -- 172.21.15.17:0/2555803482 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f16780b3340 msgr2=0x7f1664004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:07.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f1677fff700 1 --2- 172.21.15.17:0/2555803482 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f16780b3340 0x7f1664004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:07.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f1677fff700 1 -- 172.21.15.17:0/2555803482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1664004500 msgr2=0x7f1664004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:07.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f1677fff700 1 --2- 172.21.15.17:0/2555803482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1664004500 0x7f1664004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:07.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f1677fff700 1 -- 172.21.15.17:0/2555803482 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f166800b180 con 0x7f1664003780 2022-08-01T19:39:07.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f1677fff700 1 --2- 172.21.15.17:0/2555803482 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1664003780 0x7f1664003b80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f166800c000 tx=0x7f1668006df0).ready entity=mon.0 client_cookie=6758d230d33d45c5 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:07.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f1676ffd700 1 -- 172.21.15.17:0/2555803482 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f166800e7d0 con 0x7f1664003780 2022-08-01T19:39:07.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f1676ffd700 1 -- 172.21.15.17:0/2555803482 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f166800e930 con 0x7f1664003780 2022-08-01T19:39:07.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f1676ffd700 1 -- 172.21.15.17:0/2555803482 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f166800ec90 con 0x7f1664003780 2022-08-01T19:39:07.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f167e62d700 1 -- 172.21.15.17:0/2555803482 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1664003780 msgr2=0x7f1664003b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:07.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f167e62d700 1 --2- 172.21.15.17:0/2555803482 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1664003780 0x7f1664003b80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f166800c000 tx=0x7f1668006df0).stop 2022-08-01T19:39:07.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f167cbca700 1 -- 172.21.15.17:0/2555803482 reap_dead start 2022-08-01T19:39:07.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f167e62d700 1 -- 172.21.15.17:0/2555803482 shutdown_connections 2022-08-01T19:39:07.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f167e62d700 1 -- 172.21.15.17:0/2555803482 >> 172.21.15.17:0/2555803482 conn(0x7f1678060950 msgr2=0x7f1678060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:07.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f167e62d700 1 -- 172.21.15.17:0/2555803482 shutdown_connections 2022-08-01T19:39:07.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f167e62d700 1 -- 172.21.15.17:0/2555803482 wait complete. 2022-08-01T19:39:07.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f167e62d700 1 Processor -- start 2022-08-01T19:39:07.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f167e62d700 1 -- start start 2022-08-01T19:39:07.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f167e62d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1664003780 0x7f16781401a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:07.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f167e62d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16780aade0 0x7f1678142670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:07.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f167e62d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1678150bc0 0x7f1678066d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:07.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f167e62d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1678150290 con 0x7f1678150bc0 2022-08-01T19:39:07.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f167e62d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1678143930 con 0x7f16780aade0 2022-08-01T19:39:07.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f167e62d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1678144550 con 0x7f1664003780 2022-08-01T19:39:07.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f167cbca700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1678150bc0 0x7f1678066d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:07.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f167cbca700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1678150bc0 0x7f1678066d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39748/0 (socket says 172.21.15.17:39748) 2022-08-01T19:39:07.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f167cbca700 1 -- 172.21.15.17:0/4236098677 learned_addr learned my addr 172.21.15.17:0/4236098677 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:07.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f1677fff700 1 --2- 172.21.15.17:0/4236098677 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1664003780 0x7f16781401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:07.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f16777fe700 1 --2- 172.21.15.17:0/4236098677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16780aade0 0x7f1678142670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:07.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f167cbca700 1 -- 172.21.15.17:0/4236098677 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1664003780 msgr2=0x7f16781401a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:07.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f167cbca700 1 --2- 172.21.15.17:0/4236098677 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1664003780 0x7f16781401a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:07.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f167cbca700 1 -- 172.21.15.17:0/4236098677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16780aade0 msgr2=0x7f1678142670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:07.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f167cbca700 1 --2- 172.21.15.17:0/4236098677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16780aade0 0x7f1678142670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:07.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f167cbca700 1 -- 172.21.15.17:0/4236098677 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1678150480 con 0x7f1678150bc0 2022-08-01T19:39:07.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f167cbca700 1 --2- 172.21.15.17:0/4236098677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1678150bc0 0x7f1678066d50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f1678143a70 tx=0x7f16780b0480).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:07.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f16757fa700 1 -- 172.21.15.17:0/4236098677 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f16700107e0 con 0x7f1678150bc0 2022-08-01T19:39:07.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f16757fa700 1 -- 172.21.15.17:0/4236098677 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1670010940 con 0x7f1678150bc0 2022-08-01T19:39:07.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f167e62d700 1 -- 172.21.15.17:0/4236098677 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1678150720 con 0x7f1678150bc0 2022-08-01T19:39:07.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f16757fa700 1 -- 172.21.15.17:0/4236098677 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1670010ca0 con 0x7f1678150bc0 2022-08-01T19:39:07.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.690+0000 7f167e62d700 1 -- 172.21.15.17:0/4236098677 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1678063a60 con 0x7f1678150bc0 2022-08-01T19:39:07.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.694+0000 7f167e62d700 1 -- 172.21.15.17:0/4236098677 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1678144750 con 0x7f1678150bc0 2022-08-01T19:39:07.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.694+0000 7f16757fa700 1 -- 172.21.15.17:0/4236098677 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f167001c070 con 0x7f1678150bc0 2022-08-01T19:39:07.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.694+0000 7f16757fa700 1 --2- 172.21.15.17:0/4236098677 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1658053b00 0x7f1658055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:07.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.694+0000 7f16757fa700 1 -- 172.21.15.17:0/4236098677 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f167006b3a0 con 0x7f1678150bc0 2022-08-01T19:39:07.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.694+0000 7f1677fff700 1 --2- 172.21.15.17:0/4236098677 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1658053b00 0x7f1658055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:07.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.694+0000 7f1677fff700 1 --2- 172.21.15.17:0/4236098677 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1658053b00 0x7f1658055fc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f166801e000 tx=0x7f1668019000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:07.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.706+0000 7f16757fa700 1 -- 172.21.15.17:0/4236098677 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1670042390 con 0x7f1678150bc0 2022-08-01T19:39:07.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.826+0000 7f167e62d700 1 -- 172.21.15.17:0/4236098677 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f16781508b0 con 0x7f1678150bc0 2022-08-01T19:39:07.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.826+0000 7f16757fa700 1 -- 172.21.15.17:0/4236098677 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v62) v1 ==== 110+0+10 (secure 0 0 0) 0x7f16700459b0 con 0x7f1678150bc0 2022-08-01T19:39:07.832 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:39:07.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.830+0000 7f167e62d700 1 -- 172.21.15.17:0/4236098677 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1658053b00 msgr2=0x7f1658055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:07.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.830+0000 7f167e62d700 1 --2- 172.21.15.17:0/4236098677 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1658053b00 0x7f1658055fc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f166801e000 tx=0x7f1668019000).stop 2022-08-01T19:39:07.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.830+0000 7f167e62d700 1 -- 172.21.15.17:0/4236098677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1678150bc0 msgr2=0x7f1678066d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:07.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.830+0000 7f167e62d700 1 --2- 172.21.15.17:0/4236098677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1678150bc0 0x7f1678066d50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f1678143a70 tx=0x7f16780b0480).stop 2022-08-01T19:39:07.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.830+0000 7f167e62d700 1 -- 172.21.15.17:0/4236098677 shutdown_connections 2022-08-01T19:39:07.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.830+0000 7f167e62d700 1 --2- 172.21.15.17:0/4236098677 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1658053b00 0x7f1658055fc0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:07.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.830+0000 7f167e62d700 1 --2- 172.21.15.17:0/4236098677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1678150bc0 0x7f1678066d50 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:07.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.830+0000 7f167e62d700 1 --2- 172.21.15.17:0/4236098677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16780aade0 0x7f1678142670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:07.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.830+0000 7f167e62d700 1 --2- 172.21.15.17:0/4236098677 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1664003780 0x7f16781401a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:07.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.830+0000 7f167e62d700 1 -- 172.21.15.17:0/4236098677 >> 172.21.15.17:0/4236098677 conn(0x7f1678060950 msgr2=0x7f16780b2720 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:07.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.830+0000 7f167e62d700 1 -- 172.21.15.17:0/4236098677 shutdown_connections 2022-08-01T19:39:07.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.830+0000 7f167e62d700 1 -- 172.21.15.17:0/4236098677 wait complete. 2022-08-01T19:39:07.845 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:39:07.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.954+0000 7fa1f69fa700 1 Processor -- start 2022-08-01T19:39:07.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.954+0000 7fa1f69fa700 1 -- start start 2022-08-01T19:39:07.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f69fa700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa1f005c6f0 0x7fa1f00b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:07.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f69fa700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f00aa630 con 0x7fa1f00af550 2022-08-01T19:39:07.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f69fa700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f005fd10 con 0x7fa1f0060330 2022-08-01T19:39:07.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f69fa700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f00b15e0 con 0x7fa1f005c6f0 2022-08-01T19:39:07.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f59f8700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa1f00af550 0x7fa1f01445e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60178/0 (socket says 172.21.15.17:60178) 2022-08-01T19:39:07.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f59f8700 1 -- 172.21.15.17:0/2373959493 learned_addr learned my addr 172.21.15.17:0/2373959493 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:07.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f61f9700 1 --2- 172.21.15.17:0/2373959493 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa1f005c6f0 0x7fa1f00b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:07.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f49f6700 1 -- 172.21.15.17:0/2373959493 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4074204225 0 0) 0x7fa1f00aa630 con 0x7fa1f00af550 2022-08-01T19:39:07.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f49f6700 1 -- 172.21.15.17:0/2373959493 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1d8003260 con 0x7fa1f00af550 2022-08-01T19:39:07.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f49f6700 1 -- 172.21.15.17:0/2373959493 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2384806186 0 0) 0x7fa1f005fd10 con 0x7fa1f0060330 2022-08-01T19:39:07.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f49f6700 1 -- 172.21.15.17:0/2373959493 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1f00aa630 con 0x7fa1f0060330 2022-08-01T19:39:07.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f49f6700 1 -- 172.21.15.17:0/2373959493 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 757934852 0 0) 0x7fa1d8003260 con 0x7fa1f00af550 2022-08-01T19:39:07.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f49f6700 1 -- 172.21.15.17:0/2373959493 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa1f005c6f0 msgr2=0x7fa1f00b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:07.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f49f6700 1 --2- 172.21.15.17:0/2373959493 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa1f005c6f0 0x7fa1f00b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:07.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f49f6700 1 -- 172.21.15.17:0/2373959493 >> v1:172.21.15.77:6789/0 conn(0x7fa1f0060330 legacy=0x7fa1f00a8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:07.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f49f6700 1 -- 172.21.15.17:0/2373959493 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f005f840 con 0x7fa1f00af550 2022-08-01T19:39:07.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f49f6700 1 -- 172.21.15.17:0/2373959493 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7fa1ec001680 con 0x7fa1f00af550 2022-08-01T19:39:07.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f49f6700 1 -- 172.21.15.17:0/2373959493 >> v1:172.21.15.17:6789/0 conn(0x7fa1f00af550 legacy=0x7fa1f01445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:07.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f49f6700 1 --2- 172.21.15.17:0/2373959493 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa1f005c6f0 0x7fa1d8003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:07.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f49f6700 1 --2- 172.21.15.17:0/2373959493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1d8003e20 0x7fa1d8004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:07.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f49f6700 1 --2- 172.21.15.17:0/2373959493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1d80049f0 0x7fa1d8004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:07.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f49f6700 1 -- 172.21.15.17:0/2373959493 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f00b15e0 con 0x7fa1d80049f0 2022-08-01T19:39:07.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f49f6700 1 -- 172.21.15.17:0/2373959493 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d800b800 con 0x7fa1d8003e20 2022-08-01T19:39:07.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f49f6700 1 -- 172.21.15.17:0/2373959493 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d800b9c0 con 0x7fa1f005c6f0 2022-08-01T19:39:07.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f61f9700 1 --2- 172.21.15.17:0/2373959493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1d80049f0 0x7fa1d8004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:07.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f59f8700 1 --2- 172.21.15.17:0/2373959493 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa1f005c6f0 0x7fa1d8003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:07.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f51f7700 1 --2- 172.21.15.17:0/2373959493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1d8003e20 0x7fa1d8004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:07.966 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f61f9700 1 -- 172.21.15.17:0/2373959493 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa1f005c6f0 msgr2=0x7fa1d8003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:07.966 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f61f9700 1 --2- 172.21.15.17:0/2373959493 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa1f005c6f0 0x7fa1d8003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:07.966 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f61f9700 1 -- 172.21.15.17:0/2373959493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1d8003e20 msgr2=0x7fa1d8004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:07.966 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f61f9700 1 --2- 172.21.15.17:0/2373959493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1d8003e20 0x7fa1d8004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:07.967 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f61f9700 1 -- 172.21.15.17:0/2373959493 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d8003260 con 0x7fa1d80049f0 2022-08-01T19:39:07.967 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f61f9700 1 --2- 172.21.15.17:0/2373959493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1d80049f0 0x7fa1d8004e70 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7fa1e800c000 tx=0x7fa1e80091d0).ready entity=mon.0 client_cookie=823bba7b35b329af server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:07.967 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f49f6700 1 -- 172.21.15.17:0/2373959493 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1e800e7e0 con 0x7fa1d80049f0 2022-08-01T19:39:07.967 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f49f6700 1 -- 172.21.15.17:0/2373959493 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e800e940 con 0x7fa1d80049f0 2022-08-01T19:39:07.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f49f6700 1 -- 172.21.15.17:0/2373959493 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1e800eca0 con 0x7fa1d80049f0 2022-08-01T19:39:07.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f69fa700 1 -- 172.21.15.17:0/2373959493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1d80049f0 msgr2=0x7fa1d8004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:07.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f69fa700 1 --2- 172.21.15.17:0/2373959493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1d80049f0 0x7fa1d8004e70 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7fa1e800c000 tx=0x7fa1e80091d0).stop 2022-08-01T19:39:07.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f61f9700 1 -- 172.21.15.17:0/2373959493 reap_dead start 2022-08-01T19:39:07.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f69fa700 1 -- 172.21.15.17:0/2373959493 shutdown_connections 2022-08-01T19:39:07.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f69fa700 1 -- 172.21.15.17:0/2373959493 >> 172.21.15.17:0/2373959493 conn(0x7fa1f00612d0 msgr2=0x7fa1f0062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:07.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f69fa700 1 -- 172.21.15.17:0/2373959493 shutdown_connections 2022-08-01T19:39:07.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f69fa700 1 -- 172.21.15.17:0/2373959493 wait complete. 2022-08-01T19:39:07.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f69fa700 1 Processor -- start 2022-08-01T19:39:07.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f69fa700 1 -- start start 2022-08-01T19:39:07.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f69fa700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa1f0140860 0x7fa1f0152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:07.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f69fa700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1f0152d60 0x7fa1f01531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:07.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f69fa700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1f0155b00 0x7fa1f0155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:07.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f69fa700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f0140c40 con 0x7fa1f0152d60 2022-08-01T19:39:07.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f69fa700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f0145f50 con 0x7fa1f0155b00 2022-08-01T19:39:07.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f69fa700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f01401a0 con 0x7fa1f0140860 2022-08-01T19:39:07.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f51f7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1f0152d60 0x7fa1f01531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:07.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f51f7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1f0152d60 0x7fa1f01531e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39766/0 (socket says 172.21.15.17:39766) 2022-08-01T19:39:07.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f51f7700 1 -- 172.21.15.17:0/2164122817 learned_addr learned my addr 172.21.15.17:0/2164122817 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:07.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f61f9700 1 --2- 172.21.15.17:0/2164122817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1f0155b00 0x7fa1f0155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:07.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.958+0000 7fa1f59f8700 1 --2- 172.21.15.17:0/2164122817 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa1f0140860 0x7fa1f0152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:07.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.962+0000 7fa1f51f7700 1 -- 172.21.15.17:0/2164122817 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa1f0140860 msgr2=0x7fa1f0152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:07.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.962+0000 7fa1f51f7700 1 --2- 172.21.15.17:0/2164122817 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa1f0140860 0x7fa1f0152870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:07.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.962+0000 7fa1f51f7700 1 -- 172.21.15.17:0/2164122817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1f0155b00 msgr2=0x7fa1f0155f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:07.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.962+0000 7fa1f51f7700 1 --2- 172.21.15.17:0/2164122817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1f0155b00 0x7fa1f0155f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:07.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.962+0000 7fa1f51f7700 1 -- 172.21.15.17:0/2164122817 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f00aefc0 con 0x7fa1f0152d60 2022-08-01T19:39:07.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.962+0000 7fa1f51f7700 1 --2- 172.21.15.17:0/2164122817 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1f0152d60 0x7fa1f01531e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7fa1e000d000 tx=0x7fa1e0007a00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:07.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.962+0000 7fa1deffd700 1 -- 172.21.15.17:0/2164122817 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1e0011590 con 0x7fa1f0152d60 2022-08-01T19:39:07.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.962+0000 7fa1deffd700 1 -- 172.21.15.17:0/2164122817 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e00116f0 con 0x7fa1f0152d60 2022-08-01T19:39:07.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.962+0000 7fa1f69fa700 1 -- 172.21.15.17:0/2164122817 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1f0145a10 con 0x7fa1f0152d60 2022-08-01T19:39:07.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.962+0000 7fa1deffd700 1 -- 172.21.15.17:0/2164122817 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1e0011a50 con 0x7fa1f0152d60 2022-08-01T19:39:07.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.962+0000 7fa1f69fa700 1 -- 172.21.15.17:0/2164122817 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f00ae930 con 0x7fa1f0152d60 2022-08-01T19:39:07.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.962+0000 7fa1f69fa700 1 -- 172.21.15.17:0/2164122817 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1f00574e0 con 0x7fa1f0152d60 2022-08-01T19:39:07.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.962+0000 7fa1deffd700 1 -- 172.21.15.17:0/2164122817 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa1e001a030 con 0x7fa1f0152d60 2022-08-01T19:39:07.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.962+0000 7fa1deffd700 1 --2- 172.21.15.17:0/2164122817 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa1d0053980 0x7fa1d0055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:07.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.962+0000 7fa1deffd700 1 -- 172.21.15.17:0/2164122817 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fa1e006b910 con 0x7fa1f0152d60 2022-08-01T19:39:07.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.962+0000 7fa1f59f8700 1 --2- 172.21.15.17:0/2164122817 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa1d0053980 0x7fa1d0055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:07.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.966+0000 7fa1f59f8700 1 --2- 172.21.15.17:0/2164122817 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa1d0053980 0x7fa1d0055e40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7fa1ec005000 tx=0x7fa1f0061ba0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:07.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:07.970+0000 7fa1deffd700 1 -- 172.21.15.17:0/2164122817 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa1e0042a10 con 0x7fa1f0152d60 2022-08-01T19:39:08.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.098+0000 7fa1f69fa700 1 -- 172.21.15.17:0/2164122817 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa1f00ac9b0 con 0x7fa1f0152d60 2022-08-01T19:39:08.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.098+0000 7fa1deffd700 1 -- 172.21.15.17:0/2164122817 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v62) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa1e0046030 con 0x7fa1f0152d60 2022-08-01T19:39:08.101 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:39:08.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.098+0000 7fa1f69fa700 1 -- 172.21.15.17:0/2164122817 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa1d0053980 msgr2=0x7fa1d0055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:08.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.098+0000 7fa1f69fa700 1 --2- 172.21.15.17:0/2164122817 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa1d0053980 0x7fa1d0055e40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7fa1ec005000 tx=0x7fa1f0061ba0).stop 2022-08-01T19:39:08.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.098+0000 7fa1f69fa700 1 -- 172.21.15.17:0/2164122817 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1f0152d60 msgr2=0x7fa1f01531e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:08.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.098+0000 7fa1f69fa700 1 --2- 172.21.15.17:0/2164122817 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1f0152d60 0x7fa1f01531e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7fa1e000d000 tx=0x7fa1e0007a00).stop 2022-08-01T19:39:08.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.102+0000 7fa1f69fa700 1 -- 172.21.15.17:0/2164122817 shutdown_connections 2022-08-01T19:39:08.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.102+0000 7fa1f69fa700 1 --2- 172.21.15.17:0/2164122817 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa1d0053980 0x7fa1d0055e40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:08.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.102+0000 7fa1f69fa700 1 --2- 172.21.15.17:0/2164122817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1f0155b00 0x7fa1f0155f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:08.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.102+0000 7fa1f69fa700 1 --2- 172.21.15.17:0/2164122817 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1f0152d60 0x7fa1f01531e0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:08.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.102+0000 7fa1f69fa700 1 --2- 172.21.15.17:0/2164122817 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa1f0140860 0x7fa1f0152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:08.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.102+0000 7fa1f69fa700 1 -- 172.21.15.17:0/2164122817 >> 172.21.15.17:0/2164122817 conn(0x7fa1f00612d0 msgr2=0x7fa1f013e280 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:08.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.102+0000 7fa1f69fa700 1 -- 172.21.15.17:0/2164122817 shutdown_connections 2022-08-01T19:39:08.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.102+0000 7fa1f69fa700 1 -- 172.21.15.17:0/2164122817 wait complete. 2022-08-01T19:39:08.115 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-08-01T19:39:08.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.222+0000 7f279c988700 1 Processor -- start 2022-08-01T19:39:08.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.222+0000 7f279c988700 1 -- start start 2022-08-01T19:39:08.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.222+0000 7f279c988700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f27980b07a0 0x7f27980af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:08.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.222+0000 7f279c988700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2798147730 0x7f27980acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:08.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.222+0000 7f279c988700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2798147b80 con 0x7f27980b5630 2022-08-01T19:39:08.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.222+0000 7f279c988700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2798145470 con 0x7f2798147730 2022-08-01T19:39:08.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.222+0000 7f279c988700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27980ae180 con 0x7f27980b07a0 2022-08-01T19:39:08.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.222+0000 7f2797fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2798147730 0x7f27980acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:08.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.222+0000 7f2797fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2798147730 0x7f27980acdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39776/0 (socket says 172.21.15.17:39776) 2022-08-01T19:39:08.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.222+0000 7f2797fff700 1 -- 172.21.15.17:0/3301357337 learned_addr learned my addr 172.21.15.17:0/3301357337 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:08.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.222+0000 7f2796ffd700 1 --2- 172.21.15.17:0/3301357337 >> v2:172.21.15.77:3300/0 conn(0x7f27980b07a0 0x7f27980af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:08.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.222+0000 7f2796ffd700 1 -- 172.21.15.17:0/3301357337 >> v2:172.21.15.17:3300/0 conn(0x7f2798147730 msgr2=0x7f27980acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:08.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.222+0000 7f2796ffd700 1 --2- 172.21.15.17:0/3301357337 >> v2:172.21.15.17:3300/0 conn(0x7f2798147730 0x7f27980acdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:08.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.222+0000 7f2796ffd700 1 -- 172.21.15.17:0/3301357337 >> v1:172.21.15.77:6789/0 conn(0x7f27980b5630 legacy=0x7f2798067730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:08.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.222+0000 7f2796ffd700 1 -- 172.21.15.17:0/3301357337 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27980b5a10 con 0x7f27980b07a0 2022-08-01T19:39:08.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.222+0000 7f2797fff700 1 --2- 172.21.15.17:0/3301357337 >> v2:172.21.15.17:3300/0 conn(0x7f2798147730 0x7f27980acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:39:08.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.222+0000 7f2796ffd700 1 --2- 172.21.15.17:0/3301357337 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27980b07a0 0x7f27980af6e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f27880032f0 tx=0x7f27880091d0).ready entity=mon.1 client_cookie=565f368f3072932d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:08.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.222+0000 7f27967fc700 1 -- 172.21.15.17:0/3301357337 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f278800b680 con 0x7f27980b07a0 2022-08-01T19:39:08.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.222+0000 7f27967fc700 1 -- 172.21.15.17:0/3301357337 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f278800b7e0 con 0x7f27980b07a0 2022-08-01T19:39:08.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.222+0000 7f27967fc700 1 -- 172.21.15.17:0/3301357337 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f278800b950 con 0x7f27980b07a0 2022-08-01T19:39:08.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.222+0000 7f279c988700 1 -- 172.21.15.17:0/3301357337 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27980b07a0 msgr2=0x7f27980af6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:08.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.222+0000 7f279c988700 1 --2- 172.21.15.17:0/3301357337 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27980b07a0 0x7f27980af6e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f27880032f0 tx=0x7f27880091d0).stop 2022-08-01T19:39:08.234 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.222+0000 7f279c988700 1 -- 172.21.15.17:0/3301357337 shutdown_connections 2022-08-01T19:39:08.234 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.222+0000 7f279c988700 1 --2- 172.21.15.17:0/3301357337 >> v2:172.21.15.17:3300/0 conn(0x7f2798147730 0x7f27980acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:08.234 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.222+0000 7f279c988700 1 --2- 172.21.15.17:0/3301357337 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27980b07a0 0x7f27980af6e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:08.234 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.222+0000 7f279c988700 1 -- 172.21.15.17:0/3301357337 >> 172.21.15.17:0/3301357337 conn(0x7f27980626f0 msgr2=0x7f2798062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:08.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.222+0000 7f279c988700 1 -- 172.21.15.17:0/3301357337 shutdown_connections 2022-08-01T19:39:08.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.222+0000 7f279c988700 1 -- 172.21.15.17:0/3301357337 wait complete. 2022-08-01T19:39:08.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.222+0000 7f279c988700 1 Processor -- start 2022-08-01T19:39:08.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.222+0000 7f279c988700 1 -- start start 2022-08-01T19:39:08.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.222+0000 7f279c988700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27980b07a0 0x7f279814a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:08.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.222+0000 7f279c988700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27980b5630 0x7f279815a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:08.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.222+0000 7f279c988700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2798147730 0x7f279815a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:08.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.222+0000 7f279c988700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2798065740 con 0x7f27980b5630 2022-08-01T19:39:08.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.222+0000 7f279c988700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27980b49b0 con 0x7f27980b07a0 2022-08-01T19:39:08.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.226+0000 7f279c988700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27980b40a0 con 0x7f2798147730 2022-08-01T19:39:08.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.226+0000 7f2796ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27980b5630 0x7f279815a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:08.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.226+0000 7f2796ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27980b5630 0x7f279815a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39780/0 (socket says 172.21.15.17:39780) 2022-08-01T19:39:08.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.226+0000 7f2796ffd700 1 -- 172.21.15.17:0/3851750825 learned_addr learned my addr 172.21.15.17:0/3851750825 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:08.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.226+0000 7f27977fe700 1 --2- 172.21.15.17:0/3851750825 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27980b07a0 0x7f279814a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:08.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.226+0000 7f2797fff700 1 --2- 172.21.15.17:0/3851750825 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2798147730 0x7f279815a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:08.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.226+0000 7f2796ffd700 1 -- 172.21.15.17:0/3851750825 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2798147730 msgr2=0x7f279815a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:08.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.226+0000 7f2796ffd700 1 --2- 172.21.15.17:0/3851750825 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2798147730 0x7f279815a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:08.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.226+0000 7f2796ffd700 1 -- 172.21.15.17:0/3851750825 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27980b07a0 msgr2=0x7f279814a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:08.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.226+0000 7f2796ffd700 1 --2- 172.21.15.17:0/3851750825 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27980b07a0 0x7f279814a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:08.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.226+0000 7f2796ffd700 1 -- 172.21.15.17:0/3851750825 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f279000b1e0 con 0x7f27980b5630 2022-08-01T19:39:08.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.226+0000 7f2796ffd700 1 --2- 172.21.15.17:0/3851750825 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27980b5630 0x7f279815a2a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7f27880032f0 tx=0x7f2788016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:08.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.226+0000 7f2794ff9700 1 -- 172.21.15.17:0/3851750825 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f278800b680 con 0x7f27980b5630 2022-08-01T19:39:08.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.226+0000 7f2794ff9700 1 -- 172.21.15.17:0/3851750825 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2788021630 con 0x7f27980b5630 2022-08-01T19:39:08.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.226+0000 7f279c988700 1 -- 172.21.15.17:0/3851750825 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2788009470 con 0x7f27980b5630 2022-08-01T19:39:08.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.226+0000 7f2794ff9700 1 -- 172.21.15.17:0/3851750825 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2788021990 con 0x7f27980b5630 2022-08-01T19:39:08.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.226+0000 7f279c988700 1 -- 172.21.15.17:0/3851750825 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27980b42d0 con 0x7f27980b5630 2022-08-01T19:39:08.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.226+0000 7f279c988700 1 -- 172.21.15.17:0/3851750825 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27980b4710 con 0x7f27980b5630 2022-08-01T19:39:08.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.226+0000 7f2794ff9700 1 -- 172.21.15.17:0/3851750825 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2788027020 con 0x7f27980b5630 2022-08-01T19:39:08.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.230+0000 7f2794ff9700 1 --2- 172.21.15.17:0/3851750825 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2778053a70 0x7f2778055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:08.246 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.230+0000 7f2794ff9700 1 -- 172.21.15.17:0/3851750825 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f278806d190 con 0x7f27980b5630 2022-08-01T19:39:08.246 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.230+0000 7f27977fe700 1 --2- 172.21.15.17:0/3851750825 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2778053a70 0x7f2778055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:08.246 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.230+0000 7f27977fe700 1 --2- 172.21.15.17:0/3851750825 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2778053a70 0x7f2778055f30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f278c00d000 tx=0x7f278c000bc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:08.246 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.238+0000 7f2794ff9700 1 -- 172.21.15.17:0/3851750825 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2788044180 con 0x7f27980b5630 2022-08-01T19:39:08.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.362+0000 7f279c988700 1 -- 172.21.15.17:0/3851750825 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f279805f3d0 con 0x7f27980b5630 2022-08-01T19:39:08.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.366+0000 7f2794ff9700 1 -- 172.21.15.17:0/3851750825 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v62) v1 ==== 84+0+143 (secure 0 0 0) 0x7f2788015090 con 0x7f27980b5630 2022-08-01T19:39:08.368 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:39:08.368 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2022-08-01T19:39:08.368 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:08.369 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 1, 2022-08-01T19:39:08.369 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": "device_health_metrics" 2022-08-01T19:39:08.369 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:08.369 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:08.369 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 2, 2022-08-01T19:39:08.370 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": "rbd" 2022-08-01T19:39:08.370 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:08.370 INFO:teuthology.orchestra.run.smithi017.stdout:] 2022-08-01T19:39:08.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.366+0000 7f279c988700 1 -- 172.21.15.17:0/3851750825 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2778053a70 msgr2=0x7f2778055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:08.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.366+0000 7f279c988700 1 --2- 172.21.15.17:0/3851750825 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2778053a70 0x7f2778055f30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f278c00d000 tx=0x7f278c000bc0).stop 2022-08-01T19:39:08.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.366+0000 7f279c988700 1 -- 172.21.15.17:0/3851750825 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27980b5630 msgr2=0x7f279815a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:08.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.366+0000 7f279c988700 1 --2- 172.21.15.17:0/3851750825 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27980b5630 0x7f279815a2a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7f27880032f0 tx=0x7f2788016000).stop 2022-08-01T19:39:08.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.366+0000 7f279c988700 1 -- 172.21.15.17:0/3851750825 shutdown_connections 2022-08-01T19:39:08.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.366+0000 7f279c988700 1 --2- 172.21.15.17:0/3851750825 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2778053a70 0x7f2778055f30 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:08.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.366+0000 7f279c988700 1 --2- 172.21.15.17:0/3851750825 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2798147730 0x7f279815a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:08.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.366+0000 7f279c988700 1 --2- 172.21.15.17:0/3851750825 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27980b5630 0x7f279815a2a0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:08.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.366+0000 7f279c988700 1 --2- 172.21.15.17:0/3851750825 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27980b07a0 0x7f279814a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:08.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.366+0000 7f279c988700 1 -- 172.21.15.17:0/3851750825 >> 172.21.15.17:0/3851750825 conn(0x7f27980626f0 msgr2=0x7f2798143bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:08.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.366+0000 7f279c988700 1 -- 172.21.15.17:0/3851750825 shutdown_connections 2022-08-01T19:39:08.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.366+0000 7f279c988700 1 -- 172.21.15.17:0/3851750825 wait complete. 2022-08-01T19:39:08.386 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-08-01T19:39:08.387 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-08-01T19:39:08.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.498+0000 7fe23d121700 1 Processor -- start 2022-08-01T19:39:08.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.498+0000 7fe23d121700 1 -- start start 2022-08-01T19:39:08.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.498+0000 7fe23d121700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe2380b3980 0x7fe238147f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:08.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.498+0000 7fe23d121700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe2380b4310 0x7fe2380b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:08.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.498+0000 7fe23d121700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2380b5fd0 con 0x7fe238144830 2022-08-01T19:39:08.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.498+0000 7fe23d121700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2380b3d60 con 0x7fe2380b4310 2022-08-01T19:39:08.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.498+0000 7fe23d121700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2380b6200 con 0x7fe2380b3980 2022-08-01T19:39:08.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.498+0000 7fe23759e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe2380b4310 0x7fe2380b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:08.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.498+0000 7fe23759e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe2380b4310 0x7fe2380b47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39788/0 (socket says 172.21.15.17:39788) 2022-08-01T19:39:08.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.498+0000 7fe23759e700 1 -- 172.21.15.17:0/2864551225 learned_addr learned my addr 172.21.15.17:0/2864551225 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:08.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.498+0000 7fe236d9d700 1 --2- 172.21.15.17:0/2864551225 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe2380b3980 0x7fe238147f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:08.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.498+0000 7fe23759e700 1 -- 172.21.15.17:0/2864551225 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe2380b3980 msgr2=0x7fe238147f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:08.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.498+0000 7fe23759e700 1 --2- 172.21.15.17:0/2864551225 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe2380b3980 0x7fe238147f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:08.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.498+0000 7fe23759e700 1 -- 172.21.15.17:0/2864551225 >> v1:172.21.15.77:6789/0 conn(0x7fe238144830 legacy=0x7fe238144c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:08.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.498+0000 7fe23759e700 1 -- 172.21.15.17:0/2864551225 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2380b64a0 con 0x7fe2380b4310 2022-08-01T19:39:08.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.498+0000 7fe23759e700 1 --2- 172.21.15.17:0/2864551225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2380b4310 0x7fe2380b47a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7fe2300032f0 tx=0x7fe230009290).ready entity=mon.0 client_cookie=f12224742c89af02 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:08.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.498+0000 7fe235d9b700 1 -- 172.21.15.17:0/2864551225 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe23000b8a0 con 0x7fe2380b4310 2022-08-01T19:39:08.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.498+0000 7fe235d9b700 1 -- 172.21.15.17:0/2864551225 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23000ba00 con 0x7fe2380b4310 2022-08-01T19:39:08.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.498+0000 7fe235d9b700 1 -- 172.21.15.17:0/2864551225 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe23000bb70 con 0x7fe2380b4310 2022-08-01T19:39:08.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe23d121700 1 -- 172.21.15.17:0/2864551225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2380b4310 msgr2=0x7fe2380b47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:08.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe23d121700 1 --2- 172.21.15.17:0/2864551225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2380b4310 0x7fe2380b47a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7fe2300032f0 tx=0x7fe230009290).stop 2022-08-01T19:39:08.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe23d121700 1 -- 172.21.15.17:0/2864551225 shutdown_connections 2022-08-01T19:39:08.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe23d121700 1 --2- 172.21.15.17:0/2864551225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2380b4310 0x7fe2380b47a0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:08.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe23d121700 1 --2- 172.21.15.17:0/2864551225 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe2380b3980 0x7fe238147f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:08.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe23d121700 1 -- 172.21.15.17:0/2864551225 >> 172.21.15.17:0/2864551225 conn(0x7fe23805c620 msgr2=0x7fe23805ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:08.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe23d121700 1 -- 172.21.15.17:0/2864551225 shutdown_connections 2022-08-01T19:39:08.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe23d121700 1 -- 172.21.15.17:0/2864551225 wait complete. 2022-08-01T19:39:08.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe23d121700 1 Processor -- start 2022-08-01T19:39:08.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe23d121700 1 -- start start 2022-08-01T19:39:08.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe23d121700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2380b3980 0x7fe238157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:08.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe23d121700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2380b4310 0x7fe2380b7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:08.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe23d121700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe238144830 0x7fe2380af970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:08.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe23d121700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2380629e0 con 0x7fe2380b4310 2022-08-01T19:39:08.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe23d121700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23814dc80 con 0x7fe2380b3980 2022-08-01T19:39:08.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe23d121700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe238062000 con 0x7fe238144830 2022-08-01T19:39:08.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe23659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2380b4310 0x7fe2380b7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:08.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe23659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2380b4310 0x7fe2380b7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39792/0 (socket says 172.21.15.17:39792) 2022-08-01T19:39:08.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe23659c700 1 -- 172.21.15.17:0/1239928304 learned_addr learned my addr 172.21.15.17:0/1239928304 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:08.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe236d9d700 1 --2- 172.21.15.17:0/1239928304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2380b3980 0x7fe238157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:08.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe23759e700 1 --2- 172.21.15.17:0/1239928304 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe238144830 0x7fe2380af970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:08.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe23659c700 1 -- 172.21.15.17:0/1239928304 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe238144830 msgr2=0x7fe2380af970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:08.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe23659c700 1 --2- 172.21.15.17:0/1239928304 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe238144830 0x7fe2380af970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:08.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe23659c700 1 -- 172.21.15.17:0/1239928304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2380b3980 msgr2=0x7fe238157250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:08.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe23659c700 1 --2- 172.21.15.17:0/1239928304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2380b3980 0x7fe238157250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:08.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe23659c700 1 -- 172.21.15.17:0/1239928304 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe230009530 con 0x7fe2380b4310 2022-08-01T19:39:08.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe23659c700 1 --2- 172.21.15.17:0/1239928304 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2380b4310 0x7fe2380b7890 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7fe238067c20 tx=0x7fe220007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:08.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe22ffff700 1 -- 172.21.15.17:0/1239928304 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe220007e70 con 0x7fe2380b4310 2022-08-01T19:39:08.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe22ffff700 1 -- 172.21.15.17:0/1239928304 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe220007030 con 0x7fe2380b4310 2022-08-01T19:39:08.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe23d121700 1 -- 172.21.15.17:0/1239928304 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2380622a0 con 0x7fe2380b4310 2022-08-01T19:39:08.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe22ffff700 1 -- 172.21.15.17:0/1239928304 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe220007390 con 0x7fe2380b4310 2022-08-01T19:39:08.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe23d121700 1 -- 172.21.15.17:0/1239928304 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2380b0aa0 con 0x7fe2380b4310 2022-08-01T19:39:08.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe22ffff700 1 -- 172.21.15.17:0/1239928304 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe220010030 con 0x7fe2380b4310 2022-08-01T19:39:08.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe23d121700 1 -- 172.21.15.17:0/1239928304 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2380574e0 con 0x7fe2380b4310 2022-08-01T19:39:08.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe22ffff700 1 --2- 172.21.15.17:0/1239928304 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe21c053a70 0x7fe21c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:08.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe22ffff700 1 -- 172.21.15.17:0/1239928304 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fe22006b990 con 0x7fe2380b4310 2022-08-01T19:39:08.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe236d9d700 1 --2- 172.21.15.17:0/1239928304 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe21c053a70 0x7fe21c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:08.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.506+0000 7fe236d9d700 1 --2- 172.21.15.17:0/1239928304 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe21c053a70 0x7fe21c055f30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7fe22800c000 tx=0x7fe228008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:08.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.510+0000 7fe22ffff700 1 -- 172.21.15.17:0/1239928304 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe220042900 con 0x7fe2380b4310 2022-08-01T19:39:08.637 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:08.634+0000 7fe23d121700 1 -- 172.21.15.17:0/1239928304 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fe238069a70 con 0x7fe2380b4310 2022-08-01T19:39:09.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.594+0000 7fe22ffff700 1 -- 172.21.15.17:0/1239928304 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v63) v1 ==== 133+0+0 (secure 0 0 0) 0x7fe220045f20 con 0x7fe2380b4310 2022-08-01T19:39:09.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.598+0000 7fe23d121700 1 -- 172.21.15.17:0/1239928304 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe21c053a70 msgr2=0x7fe21c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:09.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.598+0000 7fe23d121700 1 --2- 172.21.15.17:0/1239928304 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe21c053a70 0x7fe21c055f30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7fe22800c000 tx=0x7fe228008040).stop 2022-08-01T19:39:09.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.598+0000 7fe23d121700 1 -- 172.21.15.17:0/1239928304 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2380b4310 msgr2=0x7fe2380b7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:09.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.598+0000 7fe23d121700 1 --2- 172.21.15.17:0/1239928304 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2380b4310 0x7fe2380b7890 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7fe238067c20 tx=0x7fe220007c00).stop 2022-08-01T19:39:09.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.598+0000 7fe23d121700 1 -- 172.21.15.17:0/1239928304 shutdown_connections 2022-08-01T19:39:09.602 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.598+0000 7fe23d121700 1 --2- 172.21.15.17:0/1239928304 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe21c053a70 0x7fe21c055f30 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:09.602 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.598+0000 7fe23d121700 1 --2- 172.21.15.17:0/1239928304 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe238144830 0x7fe2380af970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:09.602 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.598+0000 7fe23d121700 1 --2- 172.21.15.17:0/1239928304 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2380b4310 0x7fe2380b7890 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:09.602 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.598+0000 7fe23d121700 1 --2- 172.21.15.17:0/1239928304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2380b3980 0x7fe238157250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:09.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.598+0000 7fe23d121700 1 -- 172.21.15.17:0/1239928304 >> 172.21.15.17:0/1239928304 conn(0x7fe23805c620 msgr2=0x7fe238146790 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:09.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.598+0000 7fe23d121700 1 -- 172.21.15.17:0/1239928304 shutdown_connections 2022-08-01T19:39:09.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.598+0000 7fe23d121700 1 -- 172.21.15.17:0/1239928304 wait complete. 2022-08-01T19:39:09.604 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' created 2022-08-01T19:39:09.614 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-08-01T19:39:09.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.718+0000 7f9198ddc700 1 Processor -- start 2022-08-01T19:39:09.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.718+0000 7f9198ddc700 1 -- start start 2022-08-01T19:39:09.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.718+0000 7f9198ddc700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f91940614b0 0x7f9194061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:09.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.718+0000 7f9198ddc700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f91940ad750 0x7f91940b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:09.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.718+0000 7f9198ddc700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f91941405d0 0x7f9194140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:09.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.718+0000 7f9198ddc700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91940aee80 con 0x7f91940ad750 2022-08-01T19:39:09.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.718+0000 7f9198ddc700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91940b70b0 con 0x7f91941405d0 2022-08-01T19:39:09.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.718+0000 7f9198ddc700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91940af0b0 con 0x7f91940614b0 2022-08-01T19:39:09.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.718+0000 7f9192ffd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f91940ad750 0x7f91940b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:09.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.718+0000 7f9192ffd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f91940ad750 0x7f91940b9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39800/0 (socket says 172.21.15.17:39800) 2022-08-01T19:39:09.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.718+0000 7f9192ffd700 1 -- 172.21.15.17:0/133523443 learned_addr learned my addr 172.21.15.17:0/133523443 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:09.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.718+0000 7f9193fff700 1 --2- 172.21.15.17:0/133523443 >> v2:172.21.15.77:3300/0 conn(0x7f91941405d0 0x7f9194140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:09.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.718+0000 7f91937fe700 1 --2- 172.21.15.17:0/133523443 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f91940614b0 0x7f9194061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:09.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.718+0000 7f9192ffd700 1 -- 172.21.15.17:0/133523443 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f91940614b0 msgr2=0x7f9194061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:09.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.718+0000 7f9192ffd700 1 --2- 172.21.15.17:0/133523443 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f91940614b0 0x7f9194061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:09.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.718+0000 7f9192ffd700 1 -- 172.21.15.17:0/133523443 >> v2:172.21.15.77:3300/0 conn(0x7f91941405d0 msgr2=0x7f9194140a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:09.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.718+0000 7f9192ffd700 1 --2- 172.21.15.17:0/133523443 >> v2:172.21.15.77:3300/0 conn(0x7f91941405d0 0x7f9194140a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:09.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.718+0000 7f9192ffd700 1 -- 172.21.15.17:0/133523443 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91940af350 con 0x7f91940ad750 2022-08-01T19:39:09.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.718+0000 7f9193fff700 1 --2- 172.21.15.17:0/133523443 >> v2:172.21.15.77:3300/0 conn(0x7f91941405d0 0x7f9194140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:39:09.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.718+0000 7f9192ffd700 1 --2- 172.21.15.17:0/133523443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91940ad750 0x7f91940b9ff0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f9184002f70 tx=0x7f9184006eb0).ready entity=mon.0 client_cookie=951d79e8afe9dcda server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:09.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.718+0000 7f91927fc700 1 -- 172.21.15.17:0/133523443 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f918400d570 con 0x7f91940ad750 2022-08-01T19:39:09.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.718+0000 7f91927fc700 1 -- 172.21.15.17:0/133523443 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9184000e90 con 0x7f91940ad750 2022-08-01T19:39:09.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.718+0000 7f91927fc700 1 -- 172.21.15.17:0/133523443 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f918400d900 con 0x7f91940ad750 2022-08-01T19:39:09.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.718+0000 7f9198ddc700 1 -- 172.21.15.17:0/133523443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91940ad750 msgr2=0x7f91940b9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:09.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.718+0000 7f9198ddc700 1 --2- 172.21.15.17:0/133523443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91940ad750 0x7f91940b9ff0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f9184002f70 tx=0x7f9184006eb0).stop 2022-08-01T19:39:09.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.718+0000 7f9198ddc700 1 -- 172.21.15.17:0/133523443 shutdown_connections 2022-08-01T19:39:09.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.718+0000 7f9198ddc700 1 --2- 172.21.15.17:0/133523443 >> v2:172.21.15.77:3300/0 conn(0x7f91941405d0 0x7f9194140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:09.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.718+0000 7f9198ddc700 1 --2- 172.21.15.17:0/133523443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91940ad750 0x7f91940b9ff0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:09.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.718+0000 7f9198ddc700 1 --2- 172.21.15.17:0/133523443 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f91940614b0 0x7f9194061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:09.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.718+0000 7f9198ddc700 1 -- 172.21.15.17:0/133523443 >> 172.21.15.17:0/133523443 conn(0x7f9194066f70 msgr2=0x7f9194067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:09.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.718+0000 7f9198ddc700 1 -- 172.21.15.17:0/133523443 shutdown_connections 2022-08-01T19:39:09.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.722+0000 7f9198ddc700 1 -- 172.21.15.17:0/133523443 wait complete. 2022-08-01T19:39:09.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.722+0000 7f9198ddc700 1 Processor -- start 2022-08-01T19:39:09.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.722+0000 7f9198ddc700 1 -- start start 2022-08-01T19:39:09.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.722+0000 7f9198ddc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91940614b0 0x7f91941447b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:09.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.722+0000 7f9198ddc700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f91940ad750 0x7f9194152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:09.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.722+0000 7f9198ddc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91941405d0 0x7f9194152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:09.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.722+0000 7f9198ddc700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91940b3960 con 0x7f91940614b0 2022-08-01T19:39:09.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.722+0000 7f9198ddc700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91941532e0 con 0x7f91941405d0 2022-08-01T19:39:09.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.722+0000 7f9198ddc700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9194145410 con 0x7f91940ad750 2022-08-01T19:39:09.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.722+0000 7f91937fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91940614b0 0x7f91941447b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:09.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.722+0000 7f91937fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91940614b0 0x7f91941447b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39806/0 (socket says 172.21.15.17:39806) 2022-08-01T19:39:09.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.722+0000 7f91937fe700 1 -- 172.21.15.17:0/742462334 learned_addr learned my addr 172.21.15.17:0/742462334 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:09.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.722+0000 7f9193fff700 1 --2- 172.21.15.17:0/742462334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91941405d0 0x7f9194152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:09.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.722+0000 7f9192ffd700 1 --2- 172.21.15.17:0/742462334 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f91940ad750 0x7f9194152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:09.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.722+0000 7f91937fe700 1 -- 172.21.15.17:0/742462334 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f91940ad750 msgr2=0x7f9194152900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:09.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.722+0000 7f91937fe700 1 --2- 172.21.15.17:0/742462334 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f91940ad750 0x7f9194152900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:09.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.722+0000 7f91937fe700 1 -- 172.21.15.17:0/742462334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91941405d0 msgr2=0x7f9194152df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:09.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.722+0000 7f91937fe700 1 --2- 172.21.15.17:0/742462334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91941405d0 0x7f9194152df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:09.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.722+0000 7f91937fe700 1 -- 172.21.15.17:0/742462334 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f918400b1e0 con 0x7f91940614b0 2022-08-01T19:39:09.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.722+0000 7f9193fff700 1 --2- 172.21.15.17:0/742462334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91941405d0 0x7f9194152df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:39:09.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.722+0000 7f91937fe700 1 --2- 172.21.15.17:0/742462334 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91940614b0 0x7f91941447b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f9194142d30 tx=0x7f917c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:09.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.722+0000 7f9190ff9700 1 -- 172.21.15.17:0/742462334 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f917c011780 con 0x7f91940614b0 2022-08-01T19:39:09.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.722+0000 7f9190ff9700 1 -- 172.21.15.17:0/742462334 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f917c0118e0 con 0x7f91940614b0 2022-08-01T19:39:09.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.722+0000 7f9198ddc700 1 -- 172.21.15.17:0/742462334 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91940b1a90 con 0x7f91940614b0 2022-08-01T19:39:09.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.722+0000 7f9190ff9700 1 -- 172.21.15.17:0/742462334 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f917c011c40 con 0x7f91940614b0 2022-08-01T19:39:09.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.722+0000 7f9198ddc700 1 -- 172.21.15.17:0/742462334 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91940b1320 con 0x7f91940614b0 2022-08-01T19:39:09.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.726+0000 7f9198ddc700 1 -- 172.21.15.17:0/742462334 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91940af5f0 con 0x7f91940614b0 2022-08-01T19:39:09.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.726+0000 7f9190ff9700 1 -- 172.21.15.17:0/742462334 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f917c012420 con 0x7f91940614b0 2022-08-01T19:39:09.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.730+0000 7f9190ff9700 1 --2- 172.21.15.17:0/742462334 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9178053950 0x7f9178055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:09.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.730+0000 7f9190ff9700 1 -- 172.21.15.17:0/742462334 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f917c06c780 con 0x7f91940614b0 2022-08-01T19:39:09.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.730+0000 7f9192ffd700 1 --2- 172.21.15.17:0/742462334 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9178053950 0x7f9178055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:09.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.730+0000 7f9192ffd700 1 --2- 172.21.15.17:0/742462334 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9178053950 0x7f9178055e10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f918401a000 tx=0x7f9184015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:09.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.738+0000 7f9190ff9700 1 -- 172.21.15.17:0/742462334 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f917c0436b0 con 0x7f91940614b0 2022-08-01T19:39:09.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:09.858+0000 7f9198ddc700 1 -- 172.21.15.17:0/742462334 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f91940574e0 con 0x7f91940614b0 2022-08-01T19:39:10.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.606+0000 7f9190ff9700 1 -- 172.21.15.17:0/742462334 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v64) v1 ==== 125+0+0 (secure 0 0 0) 0x7f917c046cd0 con 0x7f91940614b0 2022-08-01T19:39:10.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.610+0000 7f9198ddc700 1 -- 172.21.15.17:0/742462334 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9178053950 msgr2=0x7f9178055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:10.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.610+0000 7f9198ddc700 1 --2- 172.21.15.17:0/742462334 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9178053950 0x7f9178055e10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f918401a000 tx=0x7f9184015000).stop 2022-08-01T19:39:10.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.610+0000 7f9198ddc700 1 -- 172.21.15.17:0/742462334 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91940614b0 msgr2=0x7f91941447b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:10.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.610+0000 7f9198ddc700 1 --2- 172.21.15.17:0/742462334 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91940614b0 0x7f91941447b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f9194142d30 tx=0x7f917c008040).stop 2022-08-01T19:39:10.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.614+0000 7f9198ddc700 1 -- 172.21.15.17:0/742462334 shutdown_connections 2022-08-01T19:39:10.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.614+0000 7f9198ddc700 1 --2- 172.21.15.17:0/742462334 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9178053950 0x7f9178055e10 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:10.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.614+0000 7f9198ddc700 1 --2- 172.21.15.17:0/742462334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91941405d0 0x7f9194152df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:10.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.614+0000 7f9198ddc700 1 --2- 172.21.15.17:0/742462334 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f91940ad750 0x7f9194152900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:10.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.614+0000 7f9198ddc700 1 --2- 172.21.15.17:0/742462334 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91940614b0 0x7f91941447b0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:10.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.614+0000 7f9198ddc700 1 -- 172.21.15.17:0/742462334 >> 172.21.15.17:0/742462334 conn(0x7f9194066f70 msgr2=0x7f91941467e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:10.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.614+0000 7f9198ddc700 1 -- 172.21.15.17:0/742462334 shutdown_connections 2022-08-01T19:39:10.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.614+0000 7f9198ddc700 1 -- 172.21.15.17:0/742462334 wait complete. 2022-08-01T19:39:10.620 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' created 2022-08-01T19:39:10.632 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-08-01T19:39:10.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.738+0000 7ffbbddbd700 1 Processor -- start 2022-08-01T19:39:10.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.738+0000 7ffbbddbd700 1 -- start start 2022-08-01T19:39:10.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.738+0000 7ffbbddbd700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ffbb80b6710 0x7ffbb80b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:10.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.738+0000 7ffbbddbd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffbb80b4550 0x7ffbb80b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:10.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.738+0000 7ffbbddbd700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbb8144240 con 0x7ffbb8061ec0 2022-08-01T19:39:10.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.738+0000 7ffbbddbd700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb80b6fe0 con 0x7ffbb80b6710 2022-08-01T19:39:10.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.738+0000 7ffbbddbd700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb8143d00 con 0x7ffbb80b4550 2022-08-01T19:39:10.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.742+0000 7ffbb7fff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ffbb8061ec0 0x7ffbb8146850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60244/0 (socket says 172.21.15.17:60244) 2022-08-01T19:39:10.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.742+0000 7ffbb7fff700 1 -- 172.21.15.17:0/772837399 learned_addr learned my addr 172.21.15.17:0/772837399 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:10.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.742+0000 7ffbb77fe700 1 --2- 172.21.15.17:0/772837399 >> v2:172.21.15.77:3300/0 conn(0x7ffbb80b6710 0x7ffbb80b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:10.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.742+0000 7ffbb6ffd700 1 --2- 172.21.15.17:0/772837399 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffbb80b4550 0x7ffbb80b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:10.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.742+0000 7ffbb67fc700 1 -- 172.21.15.17:0/772837399 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2361706308 0 0) 0x7ffbb8144240 con 0x7ffbb8061ec0 2022-08-01T19:39:10.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.742+0000 7ffbb67fc700 1 -- 172.21.15.17:0/772837399 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb98003260 con 0x7ffbb8061ec0 2022-08-01T19:39:10.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.742+0000 7ffbb6ffd700 1 -- 172.21.15.17:0/772837399 >> v2:172.21.15.77:3300/0 conn(0x7ffbb80b6710 msgr2=0x7ffbb80b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:10.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.742+0000 7ffbb6ffd700 1 --2- 172.21.15.17:0/772837399 >> v2:172.21.15.77:3300/0 conn(0x7ffbb80b6710 0x7ffbb80b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:10.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.742+0000 7ffbb6ffd700 1 -- 172.21.15.17:0/772837399 >> v1:172.21.15.17:6789/0 conn(0x7ffbb8061ec0 legacy=0x7ffbb8146850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:10.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.742+0000 7ffbb6ffd700 1 -- 172.21.15.17:0/772837399 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb8143fa0 con 0x7ffbb80b4550 2022-08-01T19:39:10.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.742+0000 7ffbb6ffd700 1 --2- 172.21.15.17:0/772837399 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffbb80b4550 0x7ffbb80b6af0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7ffba8002f70 tx=0x7ffba8006df0).ready entity=mon.2 client_cookie=b93bf447395a4d4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:10.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.742+0000 7ffbb67fc700 1 -- 172.21.15.17:0/772837399 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffba800d4a0 con 0x7ffbb80b4550 2022-08-01T19:39:10.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.742+0000 7ffbb67fc700 1 -- 172.21.15.17:0/772837399 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba8000e90 con 0x7ffbb80b4550 2022-08-01T19:39:10.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.742+0000 7ffbb67fc700 1 -- 172.21.15.17:0/772837399 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffba800d7f0 con 0x7ffbb80b4550 2022-08-01T19:39:10.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.742+0000 7ffbbddbd700 1 -- 172.21.15.17:0/772837399 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffbb80b4550 msgr2=0x7ffbb80b6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:10.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.742+0000 7ffbbddbd700 1 --2- 172.21.15.17:0/772837399 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffbb80b4550 0x7ffbb80b6af0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7ffba8002f70 tx=0x7ffba8006df0).stop 2022-08-01T19:39:10.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.742+0000 7ffbbddbd700 1 -- 172.21.15.17:0/772837399 shutdown_connections 2022-08-01T19:39:10.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.742+0000 7ffbbddbd700 1 --2- 172.21.15.17:0/772837399 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffbb80b4550 0x7ffbb80b6af0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:10.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.742+0000 7ffbbddbd700 1 --2- 172.21.15.17:0/772837399 >> v2:172.21.15.77:3300/0 conn(0x7ffbb80b6710 0x7ffbb80b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:10.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.742+0000 7ffbbddbd700 1 -- 172.21.15.17:0/772837399 >> 172.21.15.17:0/772837399 conn(0x7ffbb805c620 msgr2=0x7ffbb805ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:10.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.742+0000 7ffbbddbd700 1 -- 172.21.15.17:0/772837399 shutdown_connections 2022-08-01T19:39:10.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.742+0000 7ffbbddbd700 1 -- 172.21.15.17:0/772837399 wait complete. 2022-08-01T19:39:10.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.746+0000 7ffbbddbd700 1 Processor -- start 2022-08-01T19:39:10.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.746+0000 7ffbbddbd700 1 -- start start 2022-08-01T19:39:10.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.746+0000 7ffbbddbd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffbb8061ec0 0x7ffbb8127660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:10.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.746+0000 7ffbbddbd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbb80b4550 0x7ffbb813e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:10.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.746+0000 7ffbbddbd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbb80b6710 0x7ffbb813e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:10.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.746+0000 7ffbbddbd700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb8155d10 con 0x7ffbb80b4550 2022-08-01T19:39:10.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.746+0000 7ffbbddbd700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb8155fb0 con 0x7ffbb80b6710 2022-08-01T19:39:10.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.746+0000 7ffbbddbd700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb8156250 con 0x7ffbb8061ec0 2022-08-01T19:39:10.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.746+0000 7ffbb6ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbb80b4550 0x7ffbb813e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:10.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.746+0000 7ffbb6ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbb80b4550 0x7ffbb813e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39822/0 (socket says 172.21.15.17:39822) 2022-08-01T19:39:10.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.746+0000 7ffbb6ffd700 1 -- 172.21.15.17:0/2405527706 learned_addr learned my addr 172.21.15.17:0/2405527706 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:10.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.746+0000 7ffbb7fff700 1 --2- 172.21.15.17:0/2405527706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbb80b6710 0x7ffbb813e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:10.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.746+0000 7ffbb77fe700 1 --2- 172.21.15.17:0/2405527706 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffbb8061ec0 0x7ffbb8127660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:10.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.746+0000 7ffbb6ffd700 1 -- 172.21.15.17:0/2405527706 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffbb8061ec0 msgr2=0x7ffbb8127660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:10.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.746+0000 7ffbb6ffd700 1 --2- 172.21.15.17:0/2405527706 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffbb8061ec0 0x7ffbb8127660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:10.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.746+0000 7ffbb6ffd700 1 -- 172.21.15.17:0/2405527706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbb80b6710 msgr2=0x7ffbb813e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:10.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.746+0000 7ffbb6ffd700 1 --2- 172.21.15.17:0/2405527706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbb80b6710 0x7ffbb813e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:10.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.746+0000 7ffbb6ffd700 1 -- 172.21.15.17:0/2405527706 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba800b180 con 0x7ffbb80b4550 2022-08-01T19:39:10.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.746+0000 7ffbb7fff700 1 --2- 172.21.15.17:0/2405527706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbb80b6710 0x7ffbb813e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:39:10.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.746+0000 7ffbb6ffd700 1 --2- 172.21.15.17:0/2405527706 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbb80b4550 0x7ffbb813e3b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7ffba8002f70 tx=0x7ffba8014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:10.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.746+0000 7ffbb4ff9700 1 -- 172.21.15.17:0/2405527706 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffba8016070 con 0x7ffbb80b4550 2022-08-01T19:39:10.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.746+0000 7ffbb4ff9700 1 -- 172.21.15.17:0/2405527706 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba80137d0 con 0x7ffbb80b4550 2022-08-01T19:39:10.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.746+0000 7ffbbddbd700 1 -- 172.21.15.17:0/2405527706 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb81564f0 con 0x7ffbb80b4550 2022-08-01T19:39:10.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.746+0000 7ffbb4ff9700 1 -- 172.21.15.17:0/2405527706 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffba8013b30 con 0x7ffbb80b4550 2022-08-01T19:39:10.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.746+0000 7ffbbddbd700 1 -- 172.21.15.17:0/2405527706 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb813ed90 con 0x7ffbb80b4550 2022-08-01T19:39:10.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.750+0000 7ffbbddbd700 1 -- 172.21.15.17:0/2405527706 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb8156680 con 0x7ffbb80b4550 2022-08-01T19:39:10.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.750+0000 7ffbb4ff9700 1 -- 172.21.15.17:0/2405527706 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ffba800f030 con 0x7ffbb80b4550 2022-08-01T19:39:10.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.750+0000 7ffbb4ff9700 1 --2- 172.21.15.17:0/2405527706 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ffb9c053b00 0x7ffb9c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:10.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.750+0000 7ffbb4ff9700 1 -- 172.21.15.17:0/2405527706 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6304+0+0 (secure 0 0 0) 0x7ffba806cc40 con 0x7ffbb80b4550 2022-08-01T19:39:10.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.750+0000 7ffbb77fe700 1 --2- 172.21.15.17:0/2405527706 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ffb9c053b00 0x7ffb9c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:10.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.750+0000 7ffbb77fe700 1 --2- 172.21.15.17:0/2405527706 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ffb9c053b00 0x7ffb9c055fc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7ffba000b000 tx=0x7ffba0006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:10.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.758+0000 7ffbb4ff9700 1 -- 172.21.15.17:0/2405527706 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ffba8043970 con 0x7ffbb80b4550 2022-08-01T19:39:10.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.926+0000 7ffbbddbd700 1 -- 172.21.15.17:0/2405527706 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7ffbb8061780 con 0x7ffbb80b4550 2022-08-01T19:39:10.937 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.934+0000 7ffbb4ff9700 1 -- 172.21.15.17:0/2405527706 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 11 and data pool 12 v29) v1 ==== 189+0+0 (secure 0 0 0) 0x7ffba80230d0 con 0x7ffbb80b4550 2022-08-01T19:39:10.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.938+0000 7ffbbddbd700 1 -- 172.21.15.17:0/2405527706 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ffb9c053b00 msgr2=0x7ffb9c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:10.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.938+0000 7ffbbddbd700 1 --2- 172.21.15.17:0/2405527706 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ffb9c053b00 0x7ffb9c055fc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7ffba000b000 tx=0x7ffba0006df0).stop 2022-08-01T19:39:10.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.938+0000 7ffbbddbd700 1 -- 172.21.15.17:0/2405527706 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbb80b4550 msgr2=0x7ffbb813e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:10.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.938+0000 7ffbbddbd700 1 --2- 172.21.15.17:0/2405527706 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbb80b4550 0x7ffbb813e3b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7ffba8002f70 tx=0x7ffba8014000).stop 2022-08-01T19:39:10.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.938+0000 7ffbbddbd700 1 -- 172.21.15.17:0/2405527706 shutdown_connections 2022-08-01T19:39:10.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.938+0000 7ffbbddbd700 1 --2- 172.21.15.17:0/2405527706 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ffb9c053b00 0x7ffb9c055fc0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:10.944 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.938+0000 7ffbbddbd700 1 --2- 172.21.15.17:0/2405527706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffbb80b6710 0x7ffbb813e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:10.944 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.938+0000 7ffbbddbd700 1 --2- 172.21.15.17:0/2405527706 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbb80b4550 0x7ffbb813e3b0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:10.944 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.938+0000 7ffbbddbd700 1 --2- 172.21.15.17:0/2405527706 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffbb8061ec0 0x7ffbb8127660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:10.945 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.938+0000 7ffbbddbd700 1 -- 172.21.15.17:0/2405527706 >> 172.21.15.17:0/2405527706 conn(0x7ffbb805c620 msgr2=0x7ffbb8145520 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:10.945 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.938+0000 7ffbbddbd700 1 -- 172.21.15.17:0/2405527706 shutdown_connections 2022-08-01T19:39:10.945 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:10.938+0000 7ffbbddbd700 1 -- 172.21.15.17:0/2405527706 wait complete. 2022-08-01T19:39:10.946 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 11 and data pool 12 2022-08-01T19:39:10.958 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:39:11.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.050+0000 7f5ef359e700 1 Processor -- start 2022-08-01T19:39:11.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.050+0000 7f5ef359e700 1 -- start start 2022-08-01T19:39:11.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.050+0000 7f5ef359e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f5ef40b2710 0x7f5ef414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:11.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.050+0000 7f5ef359e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5ef40b01a0 0x7f5ef414b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:11.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.050+0000 7f5ef359e700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ef40a7250 con 0x7f5ef40afd60 2022-08-01T19:39:11.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.050+0000 7f5ef359e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ef40a7480 con 0x7f5ef40b01a0 2022-08-01T19:39:11.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.050+0000 7f5ef359e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ef40a7720 con 0x7f5ef40b2710 2022-08-01T19:39:11.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.050+0000 7f5ef1d9b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5ef40b01a0 0x7f5ef414b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:11.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.050+0000 7f5ef2d9d700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5ef40afd60 0x7f5ef40a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60258/0 (socket says 172.21.15.17:60258) 2022-08-01T19:39:11.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.054+0000 7f5ef2d9d700 1 -- 172.21.15.17:0/3261806193 learned_addr learned my addr 172.21.15.17:0/3261806193 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:11.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.054+0000 7f5ef259c700 1 --2- 172.21.15.17:0/3261806193 >> v2:172.21.15.77:3300/0 conn(0x7f5ef40b2710 0x7f5ef414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:11.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.054+0000 7f5ef159a700 1 -- 172.21.15.17:0/3261806193 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3915613210 0 0) 0x7f5ef40a7250 con 0x7f5ef40afd60 2022-08-01T19:39:11.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.054+0000 7f5ef1d9b700 1 -- 172.21.15.17:0/3261806193 >> v2:172.21.15.77:3300/0 conn(0x7f5ef40b2710 msgr2=0x7f5ef414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:11.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.054+0000 7f5ef1d9b700 1 --2- 172.21.15.17:0/3261806193 >> v2:172.21.15.77:3300/0 conn(0x7f5ef40b2710 0x7f5ef414afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:11.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.054+0000 7f5ef1d9b700 1 -- 172.21.15.17:0/3261806193 >> v1:172.21.15.17:6789/0 conn(0x7f5ef40afd60 legacy=0x7f5ef40a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:11.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.054+0000 7f5ef1d9b700 1 -- 172.21.15.17:0/3261806193 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef40a79c0 con 0x7f5ef40b01a0 2022-08-01T19:39:11.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.054+0000 7f5ef1d9b700 1 --2- 172.21.15.17:0/3261806193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ef40b01a0 0x7f5ef414b4c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f5ee40032f0 tx=0x7f5ee40091d0).ready entity=mon.0 client_cookie=b7df3a09613c6d0b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:11.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.054+0000 7f5ef159a700 1 -- 172.21.15.17:0/3261806193 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ee400b680 con 0x7f5ef40b01a0 2022-08-01T19:39:11.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.054+0000 7f5ef159a700 1 -- 172.21.15.17:0/3261806193 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee400b7e0 con 0x7f5ef40b01a0 2022-08-01T19:39:11.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.054+0000 7f5ef159a700 1 -- 172.21.15.17:0/3261806193 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ee400b950 con 0x7f5ef40b01a0 2022-08-01T19:39:11.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.054+0000 7f5ef359e700 1 -- 172.21.15.17:0/3261806193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ef40b01a0 msgr2=0x7f5ef414b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:11.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.054+0000 7f5ef359e700 1 --2- 172.21.15.17:0/3261806193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ef40b01a0 0x7f5ef414b4c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f5ee40032f0 tx=0x7f5ee40091d0).stop 2022-08-01T19:39:11.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.054+0000 7f5ef359e700 1 -- 172.21.15.17:0/3261806193 shutdown_connections 2022-08-01T19:39:11.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.054+0000 7f5ef359e700 1 --2- 172.21.15.17:0/3261806193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ef40b01a0 0x7f5ef414b4c0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:11.063 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.054+0000 7f5ef359e700 1 --2- 172.21.15.17:0/3261806193 >> v2:172.21.15.77:3300/0 conn(0x7f5ef40b2710 0x7f5ef414afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:11.063 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.054+0000 7f5ef359e700 1 -- 172.21.15.17:0/3261806193 >> 172.21.15.17:0/3261806193 conn(0x7f5ef4060310 msgr2=0x7f5ef40b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:11.063 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.054+0000 7f5ef359e700 1 -- 172.21.15.17:0/3261806193 shutdown_connections 2022-08-01T19:39:11.063 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.054+0000 7f5ef359e700 1 -- 172.21.15.17:0/3261806193 wait complete. 2022-08-01T19:39:11.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.058+0000 7f5ef359e700 1 Processor -- start 2022-08-01T19:39:11.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.058+0000 7f5ef359e700 1 -- start start 2022-08-01T19:39:11.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.058+0000 7f5ef359e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ef40afd60 0x7f5ef40aec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:11.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.058+0000 7f5ef359e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5ef40b01a0 0x7f5ef4159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:11.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.058+0000 7f5ef359e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ef40b2710 0x7f5ef415a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:11.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.058+0000 7f5ef359e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef40ac350 con 0x7f5ef40b2710 2022-08-01T19:39:11.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.058+0000 7f5ef359e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef40ac5f0 con 0x7f5ef40afd60 2022-08-01T19:39:11.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.058+0000 7f5ef359e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef40ac890 con 0x7f5ef40b01a0 2022-08-01T19:39:11.069 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.058+0000 7f5ef2d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ef40b2710 0x7f5ef415a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:11.069 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.058+0000 7f5ef2d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ef40b2710 0x7f5ef415a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39838/0 (socket says 172.21.15.17:39838) 2022-08-01T19:39:11.069 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.058+0000 7f5ef2d9d700 1 -- 172.21.15.17:0/1859479895 learned_addr learned my addr 172.21.15.17:0/1859479895 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:11.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.058+0000 7f5ef259c700 1 --2- 172.21.15.17:0/1859479895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ef40afd60 0x7f5ef40aec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:11.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.058+0000 7f5ef1d9b700 1 --2- 172.21.15.17:0/1859479895 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5ef40b01a0 0x7f5ef4159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:11.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.058+0000 7f5ef2d9d700 1 -- 172.21.15.17:0/1859479895 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5ef40b01a0 msgr2=0x7f5ef4159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:11.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.058+0000 7f5ef2d9d700 1 --2- 172.21.15.17:0/1859479895 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5ef40b01a0 0x7f5ef4159c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:11.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.058+0000 7f5ef2d9d700 1 -- 172.21.15.17:0/1859479895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ef40afd60 msgr2=0x7f5ef40aec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:11.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.058+0000 7f5ef2d9d700 1 --2- 172.21.15.17:0/1859479895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ef40afd60 0x7f5ef40aec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:11.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.058+0000 7f5ef2d9d700 1 -- 172.21.15.17:0/1859479895 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee4009470 con 0x7f5ef40b2710 2022-08-01T19:39:11.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.058+0000 7f5ef2d9d700 1 --2- 172.21.15.17:0/1859479895 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ef40b2710 0x7f5ef415a160 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f5eec000bc0 tx=0x7f5eec0079c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:11.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.058+0000 7f5eeb7fe700 1 -- 172.21.15.17:0/1859479895 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5eec00ce00 con 0x7f5ef40b2710 2022-08-01T19:39:11.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.058+0000 7f5eeb7fe700 1 -- 172.21.15.17:0/1859479895 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eec0070d0 con 0x7f5ef40b2710 2022-08-01T19:39:11.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.058+0000 7f5ef359e700 1 -- 172.21.15.17:0/1859479895 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef40acb30 con 0x7f5ef40b2710 2022-08-01T19:39:11.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.062+0000 7f5eeb7fe700 1 -- 172.21.15.17:0/1859479895 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5eec007430 con 0x7f5ef40b2710 2022-08-01T19:39:11.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.062+0000 7f5ef359e700 1 -- 172.21.15.17:0/1859479895 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef40ab090 con 0x7f5ef40b2710 2022-08-01T19:39:11.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.062+0000 7f5ef359e700 1 -- 172.21.15.17:0/1859479895 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef40574e0 con 0x7f5ef40b2710 2022-08-01T19:39:11.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.062+0000 7f5eeb7fe700 1 -- 172.21.15.17:0/1859479895 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5eec01a060 con 0x7f5ef40b2710 2022-08-01T19:39:11.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.062+0000 7f5eeb7fe700 1 --2- 172.21.15.17:0/1859479895 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5ed8053b30 0x7f5ed8055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:11.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.062+0000 7f5eeb7fe700 1 -- 172.21.15.17:0/1859479895 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5eec06b6a0 con 0x7f5ef40b2710 2022-08-01T19:39:11.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.062+0000 7f5ef259c700 1 --2- 172.21.15.17:0/1859479895 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5ed8053b30 0x7f5ed8055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:11.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.062+0000 7f5ef259c700 1 --2- 172.21.15.17:0/1859479895 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5ed8053b30 0x7f5ed8055ff0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f5edc00b000 tx=0x7f5edc006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:11.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.066+0000 7f5eeb7fe700 1 -- 172.21.15.17:0/1859479895 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5eec042460 con 0x7f5ef40b2710 2022-08-01T19:39:11.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.194+0000 7f5ef359e700 1 -- 172.21.15.17:0/1859479895 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ef415a7f0 con 0x7f5ef40b2710 2022-08-01T19:39:11.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.194+0000 7f5eeb7fe700 1 -- 172.21.15.17:0/1859479895 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f5eec018020 con 0x7f5ef40b2710 2022-08-01T19:39:11.199 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:39:11.199 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":65,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:39:10.932581+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:39:08.694582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2022-08-01T19:39:09.921664+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:39:11.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.198+0000 7f5ef359e700 1 -- 172.21.15.17:0/1859479895 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5ed8053b30 msgr2=0x7f5ed8055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:11.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.198+0000 7f5ef359e700 1 --2- 172.21.15.17:0/1859479895 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5ed8053b30 0x7f5ed8055ff0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f5edc00b000 tx=0x7f5edc006df0).stop 2022-08-01T19:39:11.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.198+0000 7f5ef359e700 1 -- 172.21.15.17:0/1859479895 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ef40b2710 msgr2=0x7f5ef415a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:11.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.198+0000 7f5ef359e700 1 --2- 172.21.15.17:0/1859479895 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ef40b2710 0x7f5ef415a160 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f5eec000bc0 tx=0x7f5eec0079c0).stop 2022-08-01T19:39:11.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.198+0000 7f5ef359e700 1 -- 172.21.15.17:0/1859479895 shutdown_connections 2022-08-01T19:39:11.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.198+0000 7f5ef359e700 1 --2- 172.21.15.17:0/1859479895 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5ed8053b30 0x7f5ed8055ff0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:11.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.198+0000 7f5ef359e700 1 --2- 172.21.15.17:0/1859479895 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ef40b2710 0x7f5ef415a160 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:11.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.198+0000 7f5ef359e700 1 --2- 172.21.15.17:0/1859479895 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5ef40b01a0 0x7f5ef4159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:11.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.198+0000 7f5ef359e700 1 --2- 172.21.15.17:0/1859479895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ef40afd60 0x7f5ef40aec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:11.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.198+0000 7f5ef359e700 1 -- 172.21.15.17:0/1859479895 >> 172.21.15.17:0/1859479895 conn(0x7f5ef4060310 msgr2=0x7f5ef4149b80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:11.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.198+0000 7f5ef359e700 1 -- 172.21.15.17:0/1859479895 shutdown_connections 2022-08-01T19:39:11.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.198+0000 7f5ef359e700 1 -- 172.21.15.17:0/1859479895 wait complete. 2022-08-01T19:39:11.212 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:39:11.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.310+0000 7f61f994d700 1 Processor -- start 2022-08-01T19:39:11.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.310+0000 7f61f994d700 1 -- start start 2022-08-01T19:39:11.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.310+0000 7f61f994d700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f61f40b2710 0x7f61f414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:11.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.310+0000 7f61f994d700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f40a7250 con 0x7f61f40b01a0 2022-08-01T19:39:11.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.310+0000 7f61f994d700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f40a7480 con 0x7f61f40a8c50 2022-08-01T19:39:11.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.310+0000 7f61f994d700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61f40a7720 con 0x7f61f40b2710 2022-08-01T19:39:11.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.310+0000 7f61f3fff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f61f40b01a0 0x7f61f40a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60270/0 (socket says 172.21.15.17:60270) 2022-08-01T19:39:11.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.310+0000 7f61f3fff700 1 -- 172.21.15.17:0/4240825604 learned_addr learned my addr 172.21.15.17:0/4240825604 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:11.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.310+0000 7f61f894b700 1 --2- 172.21.15.17:0/4240825604 >> v2:172.21.15.77:3300/0 conn(0x7f61f40b2710 0x7f61f414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:11.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.310+0000 7f61f37fe700 1 -- 172.21.15.17:0/4240825604 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1140404914 0 0) 0x7f61f40a7250 con 0x7f61f40b01a0 2022-08-01T19:39:11.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.310+0000 7f61f37fe700 1 -- 172.21.15.17:0/4240825604 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61d4003260 con 0x7f61f40b01a0 2022-08-01T19:39:11.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.310+0000 7f61f894b700 1 -- 172.21.15.17:0/4240825604 >> v1:172.21.15.77:6789/0 conn(0x7f61f40a8c50 legacy=0x7f61f4151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:11.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.310+0000 7f61f37fe700 1 -- 172.21.15.17:0/4240825604 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 469124096 0 0) 0x7f61f40a7480 con 0x7f61f40a8c50 2022-08-01T19:39:11.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.310+0000 7f61f894b700 1 -- 172.21.15.17:0/4240825604 >> v1:172.21.15.17:6789/0 conn(0x7f61f40b01a0 legacy=0x7f61f40a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:11.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.310+0000 7f61f894b700 1 -- 172.21.15.17:0/4240825604 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f40a79c0 con 0x7f61f40b2710 2022-08-01T19:39:11.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.310+0000 7f61f894b700 1 --2- 172.21.15.17:0/4240825604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61f40b2710 0x7f61f414afd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f61e40032f0 tx=0x7f61e40091d0).ready entity=mon.1 client_cookie=309914bce1dd6ac8 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:11.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.310+0000 7f61f37fe700 1 -- 172.21.15.17:0/4240825604 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61e400b6c0 con 0x7f61f40b2710 2022-08-01T19:39:11.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.310+0000 7f61f37fe700 1 -- 172.21.15.17:0/4240825604 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e400b820 con 0x7f61f40b2710 2022-08-01T19:39:11.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.310+0000 7f61f37fe700 1 -- 172.21.15.17:0/4240825604 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61e400b990 con 0x7f61f40b2710 2022-08-01T19:39:11.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.310+0000 7f61f994d700 1 -- 172.21.15.17:0/4240825604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61f40b2710 msgr2=0x7f61f414afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:11.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.310+0000 7f61f994d700 1 --2- 172.21.15.17:0/4240825604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61f40b2710 0x7f61f414afd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f61e40032f0 tx=0x7f61e40091d0).stop 2022-08-01T19:39:11.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.310+0000 7f61f994d700 1 -- 172.21.15.17:0/4240825604 shutdown_connections 2022-08-01T19:39:11.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.310+0000 7f61f994d700 1 --2- 172.21.15.17:0/4240825604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61f40b2710 0x7f61f414afd0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:11.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.310+0000 7f61f994d700 1 -- 172.21.15.17:0/4240825604 >> 172.21.15.17:0/4240825604 conn(0x7f61f4060310 msgr2=0x7f61f40b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:11.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.310+0000 7f61f994d700 1 -- 172.21.15.17:0/4240825604 shutdown_connections 2022-08-01T19:39:11.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.310+0000 7f61f994d700 1 -- 172.21.15.17:0/4240825604 wait complete. 2022-08-01T19:39:11.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.310+0000 7f61f994d700 1 Processor -- start 2022-08-01T19:39:11.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.310+0000 7f61f994d700 1 -- start start 2022-08-01T19:39:11.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.314+0000 7f61f994d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f61f40a8c50 0x7f61f40637e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:11.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.314+0000 7f61f994d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61f40b01a0 0x7f61f4149a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:11.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.314+0000 7f61f994d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61f40b2710 0x7f61f4149f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:11.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.314+0000 7f61f994d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f4148850 con 0x7f61f40b01a0 2022-08-01T19:39:11.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.314+0000 7f61f994d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f4148f20 con 0x7f61f40b2710 2022-08-01T19:39:11.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.314+0000 7f61f994d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f4157080 con 0x7f61f40a8c50 2022-08-01T19:39:11.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.314+0000 7f61f3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61f40b01a0 0x7f61f4149a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:11.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.314+0000 7f61f3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61f40b01a0 0x7f61f4149a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39850/0 (socket says 172.21.15.17:39850) 2022-08-01T19:39:11.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.314+0000 7f61f3fff700 1 -- 172.21.15.17:0/1800716189 learned_addr learned my addr 172.21.15.17:0/1800716189 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:11.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.314+0000 7f61f894b700 1 --2- 172.21.15.17:0/1800716189 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f61f40a8c50 0x7f61f40637e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:11.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.314+0000 7f61f914c700 1 --2- 172.21.15.17:0/1800716189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61f40b2710 0x7f61f4149f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:11.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.314+0000 7f61f3fff700 1 -- 172.21.15.17:0/1800716189 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f61f40a8c50 msgr2=0x7f61f40637e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:11.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.314+0000 7f61f3fff700 1 --2- 172.21.15.17:0/1800716189 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f61f40a8c50 0x7f61f40637e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:11.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.314+0000 7f61f3fff700 1 -- 172.21.15.17:0/1800716189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61f40b2710 msgr2=0x7f61f4149f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:11.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.314+0000 7f61f3fff700 1 --2- 172.21.15.17:0/1800716189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61f40b2710 0x7f61f4149f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:11.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.314+0000 7f61f3fff700 1 -- 172.21.15.17:0/1800716189 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e4009470 con 0x7f61f40b01a0 2022-08-01T19:39:11.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.314+0000 7f61f894b700 1 --2- 172.21.15.17:0/1800716189 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f61f40a8c50 0x7f61f40637e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:39:11.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.314+0000 7f61f3fff700 1 --2- 172.21.15.17:0/1800716189 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61f40b01a0 0x7f61f4149a10 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f61dc001110 tx=0x7f61dc007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:11.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.314+0000 7f61f1ffb700 1 -- 172.21.15.17:0/1800716189 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61dc0070d0 con 0x7f61f40b01a0 2022-08-01T19:39:11.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.314+0000 7f61f1ffb700 1 -- 172.21.15.17:0/1800716189 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61dc007230 con 0x7f61f40b01a0 2022-08-01T19:39:11.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.314+0000 7f61f1ffb700 1 -- 172.21.15.17:0/1800716189 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61dc005610 con 0x7f61f40b01a0 2022-08-01T19:39:11.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.314+0000 7f61f994d700 1 -- 172.21.15.17:0/1800716189 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f4157320 con 0x7f61f40b01a0 2022-08-01T19:39:11.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.314+0000 7f61f994d700 1 -- 172.21.15.17:0/1800716189 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f414a3f0 con 0x7f61f40b01a0 2022-08-01T19:39:11.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.314+0000 7f61f994d700 1 -- 172.21.15.17:0/1800716189 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f40574e0 con 0x7f61f40b01a0 2022-08-01T19:39:11.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.318+0000 7f61f1ffb700 1 -- 172.21.15.17:0/1800716189 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f61dc01a070 con 0x7f61f40b01a0 2022-08-01T19:39:11.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.318+0000 7f61f1ffb700 1 --2- 172.21.15.17:0/1800716189 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f61d8053aa0 0x7f61d8055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:11.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.318+0000 7f61f1ffb700 1 -- 172.21.15.17:0/1800716189 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f61dc06b280 con 0x7f61f40b01a0 2022-08-01T19:39:11.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.318+0000 7f61f894b700 1 --2- 172.21.15.17:0/1800716189 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f61d8053aa0 0x7f61d8055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:11.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.318+0000 7f61f894b700 1 --2- 172.21.15.17:0/1800716189 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f61d8053aa0 0x7f61d8055f60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f61e4019000 tx=0x7f61e4016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:11.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.326+0000 7f61f1ffb700 1 -- 172.21.15.17:0/1800716189 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f61dc042040 con 0x7f61f40b01a0 2022-08-01T19:39:11.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.446+0000 7f61f994d700 1 -- 172.21.15.17:0/1800716189 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61f4148c80 con 0x7f61f40b01a0 2022-08-01T19:39:11.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.446+0000 7f61f1ffb700 1 -- 172.21.15.17:0/1800716189 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f61dc045660 con 0x7f61f40b01a0 2022-08-01T19:39:11.450 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:39:11.450 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":65,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:39:10.932581+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:39:08.694582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2022-08-01T19:39:09.921664+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:39:11.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.450+0000 7f61f994d700 1 -- 172.21.15.17:0/1800716189 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f61d8053aa0 msgr2=0x7f61d8055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:11.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.450+0000 7f61f994d700 1 --2- 172.21.15.17:0/1800716189 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f61d8053aa0 0x7f61d8055f60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f61e4019000 tx=0x7f61e4016000).stop 2022-08-01T19:39:11.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.450+0000 7f61f994d700 1 -- 172.21.15.17:0/1800716189 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61f40b01a0 msgr2=0x7f61f4149a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:11.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.450+0000 7f61f994d700 1 --2- 172.21.15.17:0/1800716189 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61f40b01a0 0x7f61f4149a10 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f61dc001110 tx=0x7f61dc007a40).stop 2022-08-01T19:39:11.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.450+0000 7f61f994d700 1 -- 172.21.15.17:0/1800716189 shutdown_connections 2022-08-01T19:39:11.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.450+0000 7f61f994d700 1 --2- 172.21.15.17:0/1800716189 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f61d8053aa0 0x7f61d8055f60 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:11.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.450+0000 7f61f994d700 1 --2- 172.21.15.17:0/1800716189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61f40b2710 0x7f61f4149f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:11.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.450+0000 7f61f994d700 1 --2- 172.21.15.17:0/1800716189 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61f40b01a0 0x7f61f4149a10 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:11.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.450+0000 7f61f994d700 1 --2- 172.21.15.17:0/1800716189 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f61f40a8c50 0x7f61f40637e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:11.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.450+0000 7f61f994d700 1 -- 172.21.15.17:0/1800716189 >> 172.21.15.17:0/1800716189 conn(0x7f61f4060310 msgr2=0x7f61f40ad390 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:11.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.450+0000 7f61f994d700 1 -- 172.21.15.17:0/1800716189 shutdown_connections 2022-08-01T19:39:11.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.450+0000 7f61f994d700 1 -- 172.21.15.17:0/1800716189 wait complete. 2022-08-01T19:39:11.463 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-08-01T19:39:11.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.566+0000 7f2304d86700 1 Processor -- start 2022-08-01T19:39:11.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.566+0000 7f2304d86700 1 -- start start 2022-08-01T19:39:11.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.570+0000 7f2304d86700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f2300146ae0 0x7f23001403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:11.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.570+0000 7f2304d86700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f23000b4310 0x7f23001408b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:11.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.570+0000 7f2304d86700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f23000b4820 0x7f23000b5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:11.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.570+0000 7f2304d86700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23001476a0 con 0x7f23000b4310 2022-08-01T19:39:11.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.570+0000 7f2304d86700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2300143cf0 con 0x7f2300146ae0 2022-08-01T19:39:11.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.570+0000 7f2304d86700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2300143f90 con 0x7f23000b4820 2022-08-01T19:39:11.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.570+0000 7f22fdd9b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f23000b4310 0x7f23001408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:11.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.570+0000 7f22fdd9b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f23000b4310 0x7f23001408b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39856/0 (socket says 172.21.15.17:39856) 2022-08-01T19:39:11.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.570+0000 7f22fdd9b700 1 -- 172.21.15.17:0/1026819181 learned_addr learned my addr 172.21.15.17:0/1026819181 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:11.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.570+0000 7f22fed9d700 1 --2- 172.21.15.17:0/1026819181 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f23000b4820 0x7f23000b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:11.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.570+0000 7f22fe59c700 1 --2- 172.21.15.17:0/1026819181 >> v2:172.21.15.77:3300/0 conn(0x7f2300146ae0 0x7f23001403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:11.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.570+0000 7f22fdd9b700 1 -- 172.21.15.17:0/1026819181 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f23000b4820 msgr2=0x7f23000b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:11.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.570+0000 7f22fdd9b700 1 --2- 172.21.15.17:0/1026819181 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f23000b4820 0x7f23000b5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:11.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.570+0000 7f22fdd9b700 1 -- 172.21.15.17:0/1026819181 >> v2:172.21.15.77:3300/0 conn(0x7f2300146ae0 msgr2=0x7f23001403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:11.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.570+0000 7f22fdd9b700 1 --2- 172.21.15.17:0/1026819181 >> v2:172.21.15.77:3300/0 conn(0x7f2300146ae0 0x7f23001403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:11.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.570+0000 7f22fdd9b700 1 -- 172.21.15.17:0/1026819181 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23000b0e20 con 0x7f23000b4310 2022-08-01T19:39:11.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.570+0000 7f22fed9d700 1 --2- 172.21.15.17:0/1026819181 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f23000b4820 0x7f23000b5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:39:11.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.570+0000 7f22fdd9b700 1 --2- 172.21.15.17:0/1026819181 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23000b4310 0x7f23001408b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f22f0002f70 tx=0x7f22f0006eb0).ready entity=mon.0 client_cookie=c5f7c86e0533955a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:11.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.570+0000 7f22fd59a700 1 -- 172.21.15.17:0/1026819181 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f22f000d570 con 0x7f23000b4310 2022-08-01T19:39:11.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.570+0000 7f22fd59a700 1 -- 172.21.15.17:0/1026819181 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f0000e90 con 0x7f23000b4310 2022-08-01T19:39:11.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.570+0000 7f22fd59a700 1 -- 172.21.15.17:0/1026819181 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f22f000d900 con 0x7f23000b4310 2022-08-01T19:39:11.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.570+0000 7f2304d86700 1 -- 172.21.15.17:0/1026819181 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23000b4310 msgr2=0x7f23001408b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:11.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.570+0000 7f2304d86700 1 --2- 172.21.15.17:0/1026819181 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23000b4310 0x7f23001408b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f22f0002f70 tx=0x7f22f0006eb0).stop 2022-08-01T19:39:11.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.570+0000 7f2304d86700 1 -- 172.21.15.17:0/1026819181 shutdown_connections 2022-08-01T19:39:11.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.570+0000 7f2304d86700 1 --2- 172.21.15.17:0/1026819181 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f23000b4820 0x7f23000b5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:11.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.570+0000 7f2304d86700 1 --2- 172.21.15.17:0/1026819181 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23000b4310 0x7f23001408b0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:11.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.570+0000 7f2304d86700 1 --2- 172.21.15.17:0/1026819181 >> v2:172.21.15.77:3300/0 conn(0x7f2300146ae0 0x7f23001403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:11.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.570+0000 7f2304d86700 1 -- 172.21.15.17:0/1026819181 >> 172.21.15.17:0/1026819181 conn(0x7f230005c620 msgr2=0x7f230005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:11.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.570+0000 7f2304d86700 1 -- 172.21.15.17:0/1026819181 shutdown_connections 2022-08-01T19:39:11.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.570+0000 7f2304d86700 1 -- 172.21.15.17:0/1026819181 wait complete. 2022-08-01T19:39:11.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.574+0000 7f2304d86700 1 Processor -- start 2022-08-01T19:39:11.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.574+0000 7f2304d86700 1 -- start start 2022-08-01T19:39:11.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.578+0000 7f2304d86700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23000b4310 0x7f23000b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:11.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.578+0000 7f2304d86700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23000b4820 0x7f2300155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:11.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.578+0000 7f2304d86700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2300146ae0 0x7f2300156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:11.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.578+0000 7f2304d86700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23001558a0 con 0x7f23000b4820 2022-08-01T19:39:11.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.578+0000 7f22fdd9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23000b4820 0x7f2300155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:11.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.578+0000 7f2304d86700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23000b8d40 con 0x7f23000b4310 2022-08-01T19:39:11.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.578+0000 7f2304d86700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23000ade20 con 0x7f2300146ae0 2022-08-01T19:39:11.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.578+0000 7f22fdd9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23000b4820 0x7f2300155fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39864/0 (socket says 172.21.15.17:39864) 2022-08-01T19:39:11.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.578+0000 7f22fdd9b700 1 -- 172.21.15.17:0/681865857 learned_addr learned my addr 172.21.15.17:0/681865857 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:11.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.578+0000 7f22fe59c700 1 --2- 172.21.15.17:0/681865857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23000b4310 0x7f23000b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:11.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.578+0000 7f22fed9d700 1 --2- 172.21.15.17:0/681865857 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2300146ae0 0x7f2300156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:11.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.578+0000 7f22fdd9b700 1 -- 172.21.15.17:0/681865857 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2300146ae0 msgr2=0x7f2300156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:11.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.578+0000 7f22fdd9b700 1 --2- 172.21.15.17:0/681865857 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2300146ae0 0x7f2300156490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:11.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.578+0000 7f22fdd9b700 1 -- 172.21.15.17:0/681865857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23000b4310 msgr2=0x7f23000b00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:11.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.578+0000 7f22fdd9b700 1 --2- 172.21.15.17:0/681865857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23000b4310 0x7f23000b00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:11.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.578+0000 7f22fdd9b700 1 -- 172.21.15.17:0/681865857 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f000b1e0 con 0x7f23000b4820 2022-08-01T19:39:11.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.578+0000 7f22fed9d700 1 --2- 172.21.15.17:0/681865857 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2300146ae0 0x7f2300156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:39:11.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.578+0000 7f22fe59c700 1 --2- 172.21.15.17:0/681865857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23000b4310 0x7f23000b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:39:11.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.578+0000 7f22fdd9b700 1 --2- 172.21.15.17:0/681865857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23000b4820 0x7f2300155fa0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f22f0002f70 tx=0x7f22f0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:11.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.578+0000 7f22f77fe700 1 -- 172.21.15.17:0/681865857 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f22f0011990 con 0x7f23000b4820 2022-08-01T19:39:11.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.578+0000 7f22f77fe700 1 -- 172.21.15.17:0/681865857 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f0011af0 con 0x7f23000b4820 2022-08-01T19:39:11.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.578+0000 7f2304d86700 1 -- 172.21.15.17:0/681865857 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23000ae0c0 con 0x7f23000b4820 2022-08-01T19:39:11.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.578+0000 7f22f77fe700 1 -- 172.21.15.17:0/681865857 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f22f0011e50 con 0x7f23000b4820 2022-08-01T19:39:11.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.578+0000 7f2304d86700 1 -- 172.21.15.17:0/681865857 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23000adb10 con 0x7f23000b4820 2022-08-01T19:39:11.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.578+0000 7f2304d86700 1 -- 172.21.15.17:0/681865857 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23000574e0 con 0x7f23000b4820 2022-08-01T19:39:11.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.578+0000 7f22f77fe700 1 -- 172.21.15.17:0/681865857 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f22f0025030 con 0x7f23000b4820 2022-08-01T19:39:11.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.582+0000 7f22f77fe700 1 --2- 172.21.15.17:0/681865857 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f22e4053a70 0x7f22e4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:11.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.582+0000 7f22f77fe700 1 -- 172.21.15.17:0/681865857 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f22f006d3f0 con 0x7f23000b4820 2022-08-01T19:39:11.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.582+0000 7f22fe59c700 1 --2- 172.21.15.17:0/681865857 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f22e4053a70 0x7f22e4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:11.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.582+0000 7f22fe59c700 1 --2- 172.21.15.17:0/681865857 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f22e4053a70 0x7f22e4055f30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f22e800c000 tx=0x7f22e8008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:11.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.590+0000 7f22f77fe700 1 -- 172.21.15.17:0/681865857 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f22f00440a0 con 0x7f23000b4820 2022-08-01T19:39:11.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.750+0000 7f2304d86700 1 -- 172.21.15.17:0/681865857 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2300155d00 con 0x7f23000b4820 2022-08-01T19:39:11.949 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.950+0000 7f22f77fe700 1 -- 172.21.15.17:0/681865857 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v30) v1 ==== 118+0+0 (secure 0 0 0) 0x7f22f0023020 con 0x7f23000b4820 2022-08-01T19:39:11.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.950+0000 7f2304d86700 1 -- 172.21.15.17:0/681865857 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f22e4053a70 msgr2=0x7f22e4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:11.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.950+0000 7f2304d86700 1 --2- 172.21.15.17:0/681865857 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f22e4053a70 0x7f22e4055f30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f22e800c000 tx=0x7f22e8008040).stop 2022-08-01T19:39:11.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.950+0000 7f2304d86700 1 -- 172.21.15.17:0/681865857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23000b4820 msgr2=0x7f2300155fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:11.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.950+0000 7f2304d86700 1 --2- 172.21.15.17:0/681865857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23000b4820 0x7f2300155fa0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f22f0002f70 tx=0x7f22f0015000).stop 2022-08-01T19:39:11.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.950+0000 7f2304d86700 1 -- 172.21.15.17:0/681865857 shutdown_connections 2022-08-01T19:39:11.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.950+0000 7f2304d86700 1 --2- 172.21.15.17:0/681865857 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f22e4053a70 0x7f22e4055f30 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:11.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.950+0000 7f2304d86700 1 --2- 172.21.15.17:0/681865857 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2300146ae0 0x7f2300156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:11.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.950+0000 7f2304d86700 1 --2- 172.21.15.17:0/681865857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23000b4820 0x7f2300155fa0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:11.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.950+0000 7f2304d86700 1 --2- 172.21.15.17:0/681865857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23000b4310 0x7f23000b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:11.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.950+0000 7f2304d86700 1 -- 172.21.15.17:0/681865857 >> 172.21.15.17:0/681865857 conn(0x7f230005c620 msgr2=0x7f23000b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:11.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.950+0000 7f2304d86700 1 -- 172.21.15.17:0/681865857 shutdown_connections 2022-08-01T19:39:11.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:11.950+0000 7f2304d86700 1 -- 172.21.15.17:0/681865857 wait complete. 2022-08-01T19:39:11.968 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:39:12.078 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.078+0000 7fc13759e700 1 Processor -- start 2022-08-01T19:39:12.079 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.078+0000 7fc13759e700 1 -- start start 2022-08-01T19:39:12.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.082+0000 7fc13759e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc1300b1bc0 0x7fc130066b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:12.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.082+0000 7fc13759e700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc13005f8e0 con 0x7fc130140e40 2022-08-01T19:39:12.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.082+0000 7fc13759e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1300b1630 con 0x7fc130066780 2022-08-01T19:39:12.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.082+0000 7fc13759e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc13005fb10 con 0x7fc1300b1bc0 2022-08-01T19:39:12.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.082+0000 7fc135d9b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc1300b1bc0 0x7fc130066b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:12.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.082+0000 7fc136d9d700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc130140e40 0x7fc13013e2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60300/0 (socket says 172.21.15.17:60300) 2022-08-01T19:39:12.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.082+0000 7fc136d9d700 1 -- 172.21.15.17:0/429637233 learned_addr learned my addr 172.21.15.17:0/429637233 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:12.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.082+0000 7fc13559a700 1 -- 172.21.15.17:0/429637233 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1600552719 0 0) 0x7fc13005f8e0 con 0x7fc130140e40 2022-08-01T19:39:12.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.082+0000 7fc13559a700 1 -- 172.21.15.17:0/429637233 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc118003260 con 0x7fc130140e40 2022-08-01T19:39:12.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.082+0000 7fc135d9b700 1 -- 172.21.15.17:0/429637233 >> v1:172.21.15.77:6789/0 conn(0x7fc130066780 legacy=0x7fc13005ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:12.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.082+0000 7fc13559a700 1 -- 172.21.15.17:0/429637233 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1165594287 0 0) 0x7fc1300b1630 con 0x7fc130066780 2022-08-01T19:39:12.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.082+0000 7fc135d9b700 1 -- 172.21.15.17:0/429637233 >> v1:172.21.15.17:6789/0 conn(0x7fc130140e40 legacy=0x7fc13013e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:12.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.082+0000 7fc135d9b700 1 -- 172.21.15.17:0/429637233 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1300b18d0 con 0x7fc1300b1bc0 2022-08-01T19:39:12.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.082+0000 7fc135d9b700 1 --2- 172.21.15.17:0/429637233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1300b1bc0 0x7fc130066b60 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7fc128002f70 tx=0x7fc128006df0).ready entity=mon.0 client_cookie=33b4c173713ebe39 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:12.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.082+0000 7fc13559a700 1 -- 172.21.15.17:0/429637233 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc12800d610 con 0x7fc1300b1bc0 2022-08-01T19:39:12.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.082+0000 7fc13559a700 1 -- 172.21.15.17:0/429637233 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc128000e90 con 0x7fc1300b1bc0 2022-08-01T19:39:12.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.082+0000 7fc13559a700 1 -- 172.21.15.17:0/429637233 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc12800d960 con 0x7fc1300b1bc0 2022-08-01T19:39:12.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.082+0000 7fc13759e700 1 -- 172.21.15.17:0/429637233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1300b1bc0 msgr2=0x7fc130066b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:12.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.082+0000 7fc13759e700 1 --2- 172.21.15.17:0/429637233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1300b1bc0 0x7fc130066b60 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7fc128002f70 tx=0x7fc128006df0).stop 2022-08-01T19:39:12.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.082+0000 7fc13759e700 1 -- 172.21.15.17:0/429637233 shutdown_connections 2022-08-01T19:39:12.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.082+0000 7fc13759e700 1 --2- 172.21.15.17:0/429637233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1300b1bc0 0x7fc130066b60 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:12.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.082+0000 7fc13759e700 1 -- 172.21.15.17:0/429637233 >> 172.21.15.17:0/429637233 conn(0x7fc130063870 msgr2=0x7fc130063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:12.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.082+0000 7fc13759e700 1 -- 172.21.15.17:0/429637233 shutdown_connections 2022-08-01T19:39:12.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.082+0000 7fc13759e700 1 -- 172.21.15.17:0/429637233 wait complete. 2022-08-01T19:39:12.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.086+0000 7fc13759e700 1 Processor -- start 2022-08-01T19:39:12.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.086+0000 7fc13759e700 1 -- start start 2022-08-01T19:39:12.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.086+0000 7fc13759e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc130066780 0x7fc13013f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:12.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.086+0000 7fc13759e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1300b1bc0 0x7fc1300aa550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:12.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.086+0000 7fc13759e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc130140e40 0x7fc1300ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:12.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.086+0000 7fc13759e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1300ac070 con 0x7fc1300b1bc0 2022-08-01T19:39:12.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.086+0000 7fc13759e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1300ad080 con 0x7fc130140e40 2022-08-01T19:39:12.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.086+0000 7fc13759e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1300ad320 con 0x7fc130066780 2022-08-01T19:39:12.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.086+0000 7fc135d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1300b1bc0 0x7fc1300aa550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:12.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.086+0000 7fc135d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1300b1bc0 0x7fc1300aa550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39876/0 (socket says 172.21.15.17:39876) 2022-08-01T19:39:12.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.086+0000 7fc135d9b700 1 -- 172.21.15.17:0/2292438083 learned_addr learned my addr 172.21.15.17:0/2292438083 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:12.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.086+0000 7fc136d9d700 1 --2- 172.21.15.17:0/2292438083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc130140e40 0x7fc1300ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:12.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.086+0000 7fc13659c700 1 --2- 172.21.15.17:0/2292438083 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc130066780 0x7fc13013f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:12.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.086+0000 7fc135d9b700 1 -- 172.21.15.17:0/2292438083 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc130066780 msgr2=0x7fc13013f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:12.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.086+0000 7fc135d9b700 1 --2- 172.21.15.17:0/2292438083 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc130066780 0x7fc13013f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:12.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.086+0000 7fc135d9b700 1 -- 172.21.15.17:0/2292438083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc130140e40 msgr2=0x7fc1300ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:12.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.086+0000 7fc135d9b700 1 --2- 172.21.15.17:0/2292438083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc130140e40 0x7fc1300ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:12.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.086+0000 7fc135d9b700 1 -- 172.21.15.17:0/2292438083 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc12800b180 con 0x7fc1300b1bc0 2022-08-01T19:39:12.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.086+0000 7fc135d9b700 1 --2- 172.21.15.17:0/2292438083 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1300b1bc0 0x7fc1300aa550 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7fc128002f70 tx=0x7fc128014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:12.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.086+0000 7fc1237fe700 1 -- 172.21.15.17:0/2292438083 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc128016070 con 0x7fc1300b1bc0 2022-08-01T19:39:12.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.086+0000 7fc1237fe700 1 -- 172.21.15.17:0/2292438083 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc12800de90 con 0x7fc1300b1bc0 2022-08-01T19:39:12.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.086+0000 7fc13759e700 1 -- 172.21.15.17:0/2292438083 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc130153420 con 0x7fc1300b1bc0 2022-08-01T19:39:12.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.086+0000 7fc1237fe700 1 -- 172.21.15.17:0/2292438083 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc128013d20 con 0x7fc1300b1bc0 2022-08-01T19:39:12.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.086+0000 7fc13759e700 1 -- 172.21.15.17:0/2292438083 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1300acda0 con 0x7fc1300b1bc0 2022-08-01T19:39:12.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.086+0000 7fc13759e700 1 -- 172.21.15.17:0/2292438083 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc130145210 con 0x7fc1300b1bc0 2022-08-01T19:39:12.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.086+0000 7fc1237fe700 1 -- 172.21.15.17:0/2292438083 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc128025030 con 0x7fc1300b1bc0 2022-08-01T19:39:12.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.086+0000 7fc1237fe700 1 --2- 172.21.15.17:0/2292438083 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc110053a70 0x7fc110055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:12.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.086+0000 7fc1237fe700 1 -- 172.21.15.17:0/2292438083 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc12806cfe0 con 0x7fc1300b1bc0 2022-08-01T19:39:12.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.086+0000 7fc13659c700 1 --2- 172.21.15.17:0/2292438083 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc110053a70 0x7fc110055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:12.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.086+0000 7fc13659c700 1 --2- 172.21.15.17:0/2292438083 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc110053a70 0x7fc110055f30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fc12400d000 tx=0x7fc124007a00).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:12.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.098+0000 7fc1237fe700 1 -- 172.21.15.17:0/2292438083 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc130145210 con 0x7fc1300b1bc0 2022-08-01T19:39:12.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.254+0000 7fc13759e700 1 -- 172.21.15.17:0/2292438083 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc130145210 con 0x7fc1300b1bc0 2022-08-01T19:39:12.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.258+0000 7fc1237fe700 1 -- 172.21.15.17:0/2292438083 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+2493 (secure 0 0 0) 0x7fc130145210 con 0x7fc1300b1bc0 2022-08-01T19:39:12.259 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:39:12.260 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":30,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5414,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1130955771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1130955771},{"type":"v1","addr":"172.21.15.17:6835","nonce":1130955771}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":28},{"gid":5940,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3691532323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3691532323},{"type":"v1","addr":"172.21.15.17:6833","nonce":3691532323}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":28}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:39:10.932933+0000","modified":"2022-08-01T19:39:11.936834+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5949},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5949":{"gid":5949,"name":"b","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6835/2879527612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2879527612},{"type":"v1","addr":"172.21.15.77:6835","nonce":2879527612}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-08-01T19:39:12.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.258+0000 7fc13759e700 1 -- 172.21.15.17:0/2292438083 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc110053a70 msgr2=0x7fc110055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:12.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.258+0000 7fc13759e700 1 --2- 172.21.15.17:0/2292438083 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc110053a70 0x7fc110055f30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fc12400d000 tx=0x7fc124007a00).stop 2022-08-01T19:39:12.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.258+0000 7fc13759e700 1 -- 172.21.15.17:0/2292438083 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1300b1bc0 msgr2=0x7fc1300aa550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:12.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.258+0000 7fc13759e700 1 --2- 172.21.15.17:0/2292438083 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1300b1bc0 0x7fc1300aa550 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7fc128002f70 tx=0x7fc128014000).stop 2022-08-01T19:39:12.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.258+0000 7fc13759e700 1 -- 172.21.15.17:0/2292438083 shutdown_connections 2022-08-01T19:39:12.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.258+0000 7fc13759e700 1 --2- 172.21.15.17:0/2292438083 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc110053a70 0x7fc110055f30 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:12.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.258+0000 7fc13759e700 1 --2- 172.21.15.17:0/2292438083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc130140e40 0x7fc1300ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:12.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.258+0000 7fc13759e700 1 --2- 172.21.15.17:0/2292438083 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1300b1bc0 0x7fc1300aa550 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:12.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.258+0000 7fc13759e700 1 --2- 172.21.15.17:0/2292438083 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc130066780 0x7fc13013f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:12.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.258+0000 7fc13759e700 1 -- 172.21.15.17:0/2292438083 >> 172.21.15.17:0/2292438083 conn(0x7fc130063870 msgr2=0x7fc13013d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:12.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.258+0000 7fc13759e700 1 -- 172.21.15.17:0/2292438083 shutdown_connections 2022-08-01T19:39:12.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.258+0000 7fc13759e700 1 -- 172.21.15.17:0/2292438083 wait complete. 2022-08-01T19:39:12.265 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 30 2022-08-01T19:39:12.273 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:39:12.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb903fff700 1 Processor -- start 2022-08-01T19:39:12.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb903fff700 1 -- start start 2022-08-01T19:39:12.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb903fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb9040b2710 0x7fb90414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:12.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb903fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb9040a8c50 0x7fb90414b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:12.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb903fff700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9040a7250 con 0x7fb9040b01a0 2022-08-01T19:39:12.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb903fff700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9040a7480 con 0x7fb9040a8c50 2022-08-01T19:39:12.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb903fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9040a7720 con 0x7fb9040b2710 2022-08-01T19:39:12.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb9027fc700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb9040b01a0 0x7fb9040a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60310/0 (socket says 172.21.15.17:60310) 2022-08-01T19:39:12.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb9027fc700 1 -- 172.21.15.17:0/364352841 learned_addr learned my addr 172.21.15.17:0/364352841 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:12.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb9037fe700 1 --2- 172.21.15.17:0/364352841 >> v2:172.21.15.17:3300/0 conn(0x7fb9040a8c50 0x7fb90414b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:12.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb902ffd700 1 --2- 172.21.15.17:0/364352841 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb9040b2710 0x7fb90414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:12.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb901ffb700 1 -- 172.21.15.17:0/364352841 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 392635321 0 0) 0x7fb9040a7250 con 0x7fb9040b01a0 2022-08-01T19:39:12.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb901ffb700 1 -- 172.21.15.17:0/364352841 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8e8003260 con 0x7fb9040b01a0 2022-08-01T19:39:12.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb901ffb700 1 -- 172.21.15.17:0/364352841 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 399452561 0 0) 0x7fb8e8003260 con 0x7fb9040b01a0 2022-08-01T19:39:12.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb901ffb700 1 -- 172.21.15.17:0/364352841 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb9040b2710 msgr2=0x7fb90414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:12.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb901ffb700 1 --2- 172.21.15.17:0/364352841 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb9040b2710 0x7fb90414afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:12.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb901ffb700 1 -- 172.21.15.17:0/364352841 >> v2:172.21.15.17:3300/0 conn(0x7fb9040a8c50 msgr2=0x7fb90414b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:12.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb901ffb700 1 --2- 172.21.15.17:0/364352841 >> v2:172.21.15.17:3300/0 conn(0x7fb9040a8c50 0x7fb90414b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:12.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb901ffb700 1 -- 172.21.15.17:0/364352841 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9040a79c0 con 0x7fb9040b01a0 2022-08-01T19:39:12.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb901ffb700 1 -- 172.21.15.17:0/364352841 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7fb8f4001660 con 0x7fb9040b01a0 2022-08-01T19:39:12.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb901ffb700 1 -- 172.21.15.17:0/364352841 >> v1:172.21.15.17:6789/0 conn(0x7fb9040b01a0 legacy=0x7fb9040a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:12.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb901ffb700 1 --2- 172.21.15.17:0/364352841 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb9040b2710 0x7fb8e8003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:12.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb901ffb700 1 --2- 172.21.15.17:0/364352841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8e8003c80 0x7fb8e8004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:12.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb901ffb700 1 --2- 172.21.15.17:0/364352841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb8e80049f0 0x7fb8e8004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:12.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb901ffb700 1 -- 172.21.15.17:0/364352841 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9040a7480 con 0x7fb8e8003c80 2022-08-01T19:39:12.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb901ffb700 1 -- 172.21.15.17:0/364352841 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9040a7720 con 0x7fb8e80049f0 2022-08-01T19:39:12.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb901ffb700 1 -- 172.21.15.17:0/364352841 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e800b750 con 0x7fb9040b2710 2022-08-01T19:39:12.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb9027fc700 1 --2- 172.21.15.17:0/364352841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8e8003c80 0x7fb8e8004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:12.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb9037fe700 1 --2- 172.21.15.17:0/364352841 >> v2:172.21.15.17:3300/0 conn(0x7fb9040a8c50 0x7fb90414b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:39:12.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb902ffd700 1 --2- 172.21.15.17:0/364352841 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb9040b2710 0x7fb8e8003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:12.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb9037fe700 1 --2- 172.21.15.17:0/364352841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb8e80049f0 0x7fb8e8004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:12.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb902ffd700 1 -- 172.21.15.17:0/364352841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb8e80049f0 msgr2=0x7fb8e8004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:12.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb902ffd700 1 --2- 172.21.15.17:0/364352841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb8e80049f0 0x7fb8e8004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:12.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb902ffd700 1 -- 172.21.15.17:0/364352841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8e8003c80 msgr2=0x7fb8e8004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:12.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb902ffd700 1 --2- 172.21.15.17:0/364352841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8e8003c80 0x7fb8e8004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:12.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb902ffd700 1 -- 172.21.15.17:0/364352841 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e8003260 con 0x7fb9040b2710 2022-08-01T19:39:12.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb9027fc700 1 --2- 172.21.15.17:0/364352841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8e8003c80 0x7fb8e8004500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:39:12.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb902ffd700 1 --2- 172.21.15.17:0/364352841 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb9040b2710 0x7fb8e8003750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fb8f800c000 tx=0x7fb8f8006df0).ready entity=mon.2 client_cookie=87b9640219c64557 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:12.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb901ffb700 1 -- 172.21.15.17:0/364352841 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8f800e830 con 0x7fb9040b2710 2022-08-01T19:39:12.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb901ffb700 1 -- 172.21.15.17:0/364352841 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f800e990 con 0x7fb9040b2710 2022-08-01T19:39:12.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.374+0000 7fb901ffb700 1 -- 172.21.15.17:0/364352841 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8f800ecf0 con 0x7fb9040b2710 2022-08-01T19:39:12.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.378+0000 7fb903fff700 1 -- 172.21.15.17:0/364352841 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb9040b2710 msgr2=0x7fb8e8003750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:12.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.378+0000 7fb903fff700 1 --2- 172.21.15.17:0/364352841 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb9040b2710 0x7fb8e8003750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fb8f800c000 tx=0x7fb8f8006df0).stop 2022-08-01T19:39:12.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.378+0000 7fb9037fe700 1 -- 172.21.15.17:0/364352841 reap_dead start 2022-08-01T19:39:12.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.378+0000 7fb903fff700 1 -- 172.21.15.17:0/364352841 shutdown_connections 2022-08-01T19:39:12.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.378+0000 7fb903fff700 1 -- 172.21.15.17:0/364352841 >> 172.21.15.17:0/364352841 conn(0x7fb904060310 msgr2=0x7fb9040b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:12.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.378+0000 7fb903fff700 1 -- 172.21.15.17:0/364352841 shutdown_connections 2022-08-01T19:39:12.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.378+0000 7fb903fff700 1 -- 172.21.15.17:0/364352841 wait complete. 2022-08-01T19:39:12.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.378+0000 7fb903fff700 1 Processor -- start 2022-08-01T19:39:12.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.378+0000 7fb903fff700 1 -- start start 2022-08-01T19:39:12.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.378+0000 7fb903fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb9040b2710 0x7fb90405c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:12.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.378+0000 7fb903fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb904156c40 0x7fb9041570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:12.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.378+0000 7fb903fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb904159bf0 0x7fb90415a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:12.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.378+0000 7fb903fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9040b6060 con 0x7fb904156c40 2022-08-01T19:39:12.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.378+0000 7fb903fff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9040aea70 con 0x7fb904159bf0 2022-08-01T19:39:12.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.378+0000 7fb903fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9041567e0 con 0x7fb9040b2710 2022-08-01T19:39:12.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.378+0000 7fb9027fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb904156c40 0x7fb9041570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:12.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.378+0000 7fb902ffd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb9040b2710 0x7fb90405c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:12.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.378+0000 7fb9027fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb904156c40 0x7fb9041570a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39898/0 (socket says 172.21.15.17:39898) 2022-08-01T19:39:12.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.378+0000 7fb9027fc700 1 -- 172.21.15.17:0/2133764506 learned_addr learned my addr 172.21.15.17:0/2133764506 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:12.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.378+0000 7fb9037fe700 1 --2- 172.21.15.17:0/2133764506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb904159bf0 0x7fb90415a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:12.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.378+0000 7fb902ffd700 1 -- 172.21.15.17:0/2133764506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb904159bf0 msgr2=0x7fb90415a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:12.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.378+0000 7fb902ffd700 1 --2- 172.21.15.17:0/2133764506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb904159bf0 0x7fb90415a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:12.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.378+0000 7fb902ffd700 1 -- 172.21.15.17:0/2133764506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb904156c40 msgr2=0x7fb9041570a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:12.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.378+0000 7fb902ffd700 1 --2- 172.21.15.17:0/2133764506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb904156c40 0x7fb9041570a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:12.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.378+0000 7fb902ffd700 1 -- 172.21.15.17:0/2133764506 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9040b53b0 con 0x7fb9040b2710 2022-08-01T19:39:12.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.378+0000 7fb9027fc700 1 --2- 172.21.15.17:0/2133764506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb904156c40 0x7fb9041570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:39:12.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.382+0000 7fb902ffd700 1 --2- 172.21.15.17:0/2133764506 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb9040b2710 0x7fb90405c170 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fb904156b20 tx=0x7fb8f8019000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:12.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.382+0000 7fb8f3fff700 1 -- 172.21.15.17:0/2133764506 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8f801b820 con 0x7fb9040b2710 2022-08-01T19:39:12.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.382+0000 7fb8f3fff700 1 -- 172.21.15.17:0/2133764506 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f8000e90 con 0x7fb9040b2710 2022-08-01T19:39:12.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.382+0000 7fb903fff700 1 -- 172.21.15.17:0/2133764506 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9040b5110 con 0x7fb9040b2710 2022-08-01T19:39:12.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.382+0000 7fb8f3fff700 1 -- 172.21.15.17:0/2133764506 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8f801bb70 con 0x7fb9040b2710 2022-08-01T19:39:12.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.382+0000 7fb903fff700 1 -- 172.21.15.17:0/2133764506 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9040b2f60 con 0x7fb9040b2710 2022-08-01T19:39:12.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.382+0000 7fb903fff700 1 -- 172.21.15.17:0/2133764506 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9040574e0 con 0x7fb9040b2710 2022-08-01T19:39:12.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.382+0000 7fb8f3fff700 1 -- 172.21.15.17:0/2133764506 <== mon.2 v2:172.21.15.77:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb8f801bcd0 con 0x7fb9040b2710 2022-08-01T19:39:12.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.382+0000 7fb8f3fff700 1 --2- 172.21.15.17:0/2133764506 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb8e0053a10 0x7fb8e0055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:12.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.382+0000 7fb8f3fff700 1 -- 172.21.15.17:0/2133764506 <== mon.2 v2:172.21.15.77:3301/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb8f806d500 con 0x7fb9040b2710 2022-08-01T19:39:12.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.382+0000 7fb9027fc700 1 --2- 172.21.15.17:0/2133764506 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb8e0053a10 0x7fb8e0055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:12.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.386+0000 7fb9027fc700 1 --2- 172.21.15.17:0/2133764506 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb8e0053a10 0x7fb8e0055ed0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7fb8f4007880 tx=0x7fb90406a5f0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:12.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.398+0000 7fb8f3fff700 1 -- 172.21.15.17:0/2133764506 <== mon.2 v2:172.21.15.77:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb8f8044220 con 0x7fb9040b2710 2022-08-01T19:39:12.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.518+0000 7fb903fff700 1 -- 172.21.15.17:0/2133764506 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb904156970 con 0x7fb9040b2710 2022-08-01T19:39:12.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.526+0000 7fb8f3fff700 1 -- 172.21.15.17:0/2133764506 <== mon.2 v2:172.21.15.77:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16274 (secure 0 0 0) 0x7fb8f802a020 con 0x7fb9040b2710 2022-08-01T19:39:12.524 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:39:12.525 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":66,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:39:11.935986+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:39:08.694582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2022-08-01T19:39:09.921664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:39:12.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.526+0000 7fb903fff700 1 -- 172.21.15.17:0/2133764506 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb8e0053a10 msgr2=0x7fb8e0055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:12.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.526+0000 7fb903fff700 1 --2- 172.21.15.17:0/2133764506 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb8e0053a10 0x7fb8e0055ed0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7fb8f4007880 tx=0x7fb90406a5f0).stop 2022-08-01T19:39:12.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.526+0000 7fb903fff700 1 -- 172.21.15.17:0/2133764506 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb9040b2710 msgr2=0x7fb90405c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:12.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.526+0000 7fb903fff700 1 --2- 172.21.15.17:0/2133764506 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb9040b2710 0x7fb90405c170 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fb904156b20 tx=0x7fb8f8019000).stop 2022-08-01T19:39:12.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.526+0000 7fb903fff700 1 -- 172.21.15.17:0/2133764506 shutdown_connections 2022-08-01T19:39:12.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.526+0000 7fb903fff700 1 --2- 172.21.15.17:0/2133764506 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb8e0053a10 0x7fb8e0055ed0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:12.529 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.526+0000 7fb903fff700 1 --2- 172.21.15.17:0/2133764506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb904159bf0 0x7fb90415a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:12.529 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.526+0000 7fb903fff700 1 --2- 172.21.15.17:0/2133764506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb904156c40 0x7fb9041570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:12.529 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.526+0000 7fb903fff700 1 --2- 172.21.15.17:0/2133764506 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb9040b2710 0x7fb90405c170 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:12.529 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.526+0000 7fb903fff700 1 -- 172.21.15.17:0/2133764506 >> 172.21.15.17:0/2133764506 conn(0x7fb904060310 msgr2=0x7fb9040673e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:12.530 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.526+0000 7fb903fff700 1 -- 172.21.15.17:0/2133764506 shutdown_connections 2022-08-01T19:39:12.530 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.526+0000 7fb903fff700 1 -- 172.21.15.17:0/2133764506 wait complete. 2022-08-01T19:39:12.542 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T19:39:12.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.654+0000 7f6c6a27d700 1 Processor -- start 2022-08-01T19:39:12.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.654+0000 7f6c6a27d700 1 -- start start 2022-08-01T19:39:12.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.654+0000 7f6c6a27d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6c640af450 0x7f6c640af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:12.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.654+0000 7f6c6a27d700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c640aff30 con 0x7f6c640ad4b0 2022-08-01T19:39:12.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.654+0000 7f6c6a27d700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c6405c7a0 con 0x7f6c640ad9e0 2022-08-01T19:39:12.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.654+0000 7f6c6a27d700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c64140990 con 0x7f6c640af450 2022-08-01T19:39:12.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.654+0000 7f6c6927b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6c640af450 0x7f6c640af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:12.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.654+0000 7f6c68a7a700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f6c640ad4b0 0x7f6c641432e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60332/0 (socket says 172.21.15.17:60332) 2022-08-01T19:39:12.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.654+0000 7f6c68a7a700 1 -- 172.21.15.17:0/3699404379 learned_addr learned my addr 172.21.15.17:0/3699404379 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:12.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.654+0000 7f6c5bfff700 1 -- 172.21.15.17:0/3699404379 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4079059821 0 0) 0x7f6c640aff30 con 0x7f6c640ad4b0 2022-08-01T19:39:12.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.658+0000 7f6c5bfff700 1 -- 172.21.15.17:0/3699404379 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c48003260 con 0x7f6c640ad4b0 2022-08-01T19:39:12.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.658+0000 7f6c5bfff700 1 -- 172.21.15.17:0/3699404379 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 272334658 0 0) 0x7f6c6405c7a0 con 0x7f6c640ad9e0 2022-08-01T19:39:12.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.658+0000 7f6c6927b700 1 -- 172.21.15.17:0/3699404379 >> v1:172.21.15.77:6789/0 conn(0x7f6c640ad9e0 legacy=0x7f6c640a8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:12.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.658+0000 7f6c6927b700 1 -- 172.21.15.17:0/3699404379 >> v1:172.21.15.17:6789/0 conn(0x7f6c640ad4b0 legacy=0x7f6c641432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:12.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.658+0000 7f6c6927b700 1 -- 172.21.15.17:0/3699404379 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c641439f0 con 0x7f6c640af450 2022-08-01T19:39:12.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.658+0000 7f6c6927b700 1 --2- 172.21.15.17:0/3699404379 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c640af450 0x7f6c640af830 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7f6c640ad900 tx=0x7f6c60006df0).ready entity=mon.0 client_cookie=ef494778aac519ea server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:12.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.658+0000 7f6c5bfff700 1 -- 172.21.15.17:0/3699404379 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c6000d610 con 0x7f6c640af450 2022-08-01T19:39:12.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.658+0000 7f6c5bfff700 1 -- 172.21.15.17:0/3699404379 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c60000e90 con 0x7f6c640af450 2022-08-01T19:39:12.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.658+0000 7f6c5bfff700 1 -- 172.21.15.17:0/3699404379 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c6000d960 con 0x7f6c640af450 2022-08-01T19:39:12.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.658+0000 7f6c6a27d700 1 -- 172.21.15.17:0/3699404379 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c640af450 msgr2=0x7f6c640af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:12.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.658+0000 7f6c6a27d700 1 --2- 172.21.15.17:0/3699404379 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c640af450 0x7f6c640af830 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7f6c640ad900 tx=0x7f6c60006df0).stop 2022-08-01T19:39:12.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.658+0000 7f6c6a27d700 1 -- 172.21.15.17:0/3699404379 shutdown_connections 2022-08-01T19:39:12.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.658+0000 7f6c6a27d700 1 --2- 172.21.15.17:0/3699404379 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c640af450 0x7f6c640af830 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:12.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.658+0000 7f6c6a27d700 1 -- 172.21.15.17:0/3699404379 >> 172.21.15.17:0/3699404379 conn(0x7f6c640612d0 msgr2=0x7f6c64062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:12.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.658+0000 7f6c6a27d700 1 -- 172.21.15.17:0/3699404379 shutdown_connections 2022-08-01T19:39:12.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.658+0000 7f6c6a27d700 1 -- 172.21.15.17:0/3699404379 wait complete. 2022-08-01T19:39:12.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.662+0000 7f6c6a27d700 1 Processor -- start 2022-08-01T19:39:12.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.662+0000 7f6c6a27d700 1 -- start start 2022-08-01T19:39:12.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.662+0000 7f6c6a27d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c640ad4b0 0x7f6c640a9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:12.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.662+0000 7f6c6a27d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6c640ad9e0 0x7f6c64067230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:12.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.662+0000 7f6c6a27d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c640af450 0x7f6c64067720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:12.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.662+0000 7f6c6a27d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c640ab8e0 con 0x7f6c640af450 2022-08-01T19:39:12.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.662+0000 7f6c6a27d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c64064de0 con 0x7f6c640ad4b0 2022-08-01T19:39:12.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.662+0000 7f6c6a27d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c640ab180 con 0x7f6c640ad9e0 2022-08-01T19:39:12.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.662+0000 7f6c69a7c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c640af450 0x7f6c64067720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:12.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.662+0000 7f6c69a7c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c640af450 0x7f6c64067720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39914/0 (socket says 172.21.15.17:39914) 2022-08-01T19:39:12.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.662+0000 7f6c69a7c700 1 -- 172.21.15.17:0/2508171984 learned_addr learned my addr 172.21.15.17:0/2508171984 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:12.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.662+0000 7f6c6927b700 1 --2- 172.21.15.17:0/2508171984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c640ad4b0 0x7f6c640a9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:12.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.662+0000 7f6c68a7a700 1 --2- 172.21.15.17:0/2508171984 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6c640ad9e0 0x7f6c64067230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:12.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.662+0000 7f6c69a7c700 1 -- 172.21.15.17:0/2508171984 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6c640ad9e0 msgr2=0x7f6c64067230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:12.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.662+0000 7f6c69a7c700 1 --2- 172.21.15.17:0/2508171984 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6c640ad9e0 0x7f6c64067230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:12.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.662+0000 7f6c69a7c700 1 -- 172.21.15.17:0/2508171984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c640ad4b0 msgr2=0x7f6c640a9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:12.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.662+0000 7f6c69a7c700 1 --2- 172.21.15.17:0/2508171984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c640ad4b0 0x7f6c640a9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:12.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.662+0000 7f6c69a7c700 1 -- 172.21.15.17:0/2508171984 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6000b180 con 0x7f6c640af450 2022-08-01T19:39:12.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.662+0000 7f6c69a7c700 1 --2- 172.21.15.17:0/2508171984 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c640af450 0x7f6c64067720 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f6c5c001250 tx=0x7f6c5c007e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:12.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.662+0000 7f6c5a7fc700 1 -- 172.21.15.17:0/2508171984 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c5c00c900 con 0x7f6c640af450 2022-08-01T19:39:12.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.662+0000 7f6c5a7fc700 1 -- 172.21.15.17:0/2508171984 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5c00ca60 con 0x7f6c640af450 2022-08-01T19:39:12.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.662+0000 7f6c6a27d700 1 -- 172.21.15.17:0/2508171984 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c640abb10 con 0x7f6c640af450 2022-08-01T19:39:12.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.662+0000 7f6c5a7fc700 1 -- 172.21.15.17:0/2508171984 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c5c00cdc0 con 0x7f6c640af450 2022-08-01T19:39:12.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.662+0000 7f6c6a27d700 1 -- 172.21.15.17:0/2508171984 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c640abf40 con 0x7f6c640af450 2022-08-01T19:39:12.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.666+0000 7f6c6a27d700 1 -- 172.21.15.17:0/2508171984 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c640574e0 con 0x7f6c640af450 2022-08-01T19:39:12.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.666+0000 7f6c5a7fc700 1 -- 172.21.15.17:0/2508171984 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6c5c021070 con 0x7f6c640af450 2022-08-01T19:39:12.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.666+0000 7f6c5a7fc700 1 --2- 172.21.15.17:0/2508171984 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6c4c053b00 0x7f6c4c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:12.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.666+0000 7f6c5a7fc700 1 -- 172.21.15.17:0/2508171984 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6c5c06ef10 con 0x7f6c640af450 2022-08-01T19:39:12.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.666+0000 7f6c6927b700 1 --2- 172.21.15.17:0/2508171984 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6c4c053b00 0x7f6c4c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:12.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.666+0000 7f6c6927b700 1 --2- 172.21.15.17:0/2508171984 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6c4c053b00 0x7f6c4c055fc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f6c60018000 tx=0x7f6c60014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:12.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.670+0000 7f6c5a7fc700 1 -- 172.21.15.17:0/2508171984 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6c5c045cd0 con 0x7f6c640af450 2022-08-01T19:39:12.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.834+0000 7f6c6a27d700 1 -- 172.21.15.17:0/2508171984 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f6c64061b20 con 0x7f6c640af450 2022-08-01T19:39:12.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.842+0000 7f6c5a7fc700 1 -- 172.21.15.17:0/2508171984 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v21) v1 ==== 184+0+0 (secure 0 0 0) 0x7f6c5c01f030 con 0x7f6c640af450 2022-08-01T19:39:12.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.842+0000 7f6c6a27d700 1 -- 172.21.15.17:0/2508171984 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6c4c053b00 msgr2=0x7f6c4c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:12.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.842+0000 7f6c6a27d700 1 --2- 172.21.15.17:0/2508171984 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6c4c053b00 0x7f6c4c055fc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f6c60018000 tx=0x7f6c60014000).stop 2022-08-01T19:39:12.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.842+0000 7f6c6a27d700 1 -- 172.21.15.17:0/2508171984 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c640af450 msgr2=0x7f6c64067720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:12.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.842+0000 7f6c6a27d700 1 --2- 172.21.15.17:0/2508171984 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c640af450 0x7f6c64067720 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f6c5c001250 tx=0x7f6c5c007e50).stop 2022-08-01T19:39:12.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.842+0000 7f6c6a27d700 1 -- 172.21.15.17:0/2508171984 shutdown_connections 2022-08-01T19:39:12.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.842+0000 7f6c6a27d700 1 --2- 172.21.15.17:0/2508171984 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6c4c053b00 0x7f6c4c055fc0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:12.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.842+0000 7f6c6a27d700 1 --2- 172.21.15.17:0/2508171984 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c640af450 0x7f6c64067720 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:12.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.842+0000 7f6c6a27d700 1 --2- 172.21.15.17:0/2508171984 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6c640ad9e0 0x7f6c64067230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:12.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.842+0000 7f6c6a27d700 1 --2- 172.21.15.17:0/2508171984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c640ad4b0 0x7f6c640a9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:12.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.842+0000 7f6c6a27d700 1 -- 172.21.15.17:0/2508171984 >> 172.21.15.17:0/2508171984 conn(0x7f6c640612d0 msgr2=0x7f6c6413da90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:12.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.842+0000 7f6c6a27d700 1 -- 172.21.15.17:0/2508171984 shutdown_connections 2022-08-01T19:39:12.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.842+0000 7f6c6a27d700 1 -- 172.21.15.17:0/2508171984 wait complete. 2022-08-01T19:39:12.850 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.0 2022-08-01T19:39:12.862 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T19:39:12.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.974+0000 7f5f14d14700 1 Processor -- start 2022-08-01T19:39:12.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.974+0000 7f5f14d14700 1 -- start start 2022-08-01T19:39:12.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.974+0000 7f5f14d14700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5f10144190 0x7f5f1014c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:12.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.974+0000 7f5f14d14700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f100aad40 con 0x7f5f10060290 2022-08-01T19:39:12.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.974+0000 7f5f14d14700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f10060670 con 0x7f5f10067490 2022-08-01T19:39:12.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.974+0000 7f5f14d14700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f10067030 con 0x7f5f10144190 2022-08-01T19:39:12.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.974+0000 7f5f0e59c700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5f10060290 0x7f5f100b48e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60344/0 (socket says 172.21.15.17:60344) 2022-08-01T19:39:12.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.974+0000 7f5f0e59c700 1 -- 172.21.15.17:0/3766799018 learned_addr learned my addr 172.21.15.17:0/3766799018 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:12.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.974+0000 7f5f0ed9d700 1 --2- 172.21.15.17:0/3766799018 >> v2:172.21.15.17:3300/0 conn(0x7f5f10144190 0x7f5f1014c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:12.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.974+0000 7f5f0dd9b700 1 -- 172.21.15.17:0/3766799018 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1681651396 0 0) 0x7f5f100aad40 con 0x7f5f10060290 2022-08-01T19:39:12.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.974+0000 7f5f0dd9b700 1 -- 172.21.15.17:0/3766799018 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ef0003260 con 0x7f5f10060290 2022-08-01T19:39:12.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.974+0000 7f5f0dd9b700 1 -- 172.21.15.17:0/3766799018 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2099649 0 0) 0x7f5f10060670 con 0x7f5f10067490 2022-08-01T19:39:12.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.974+0000 7f5f0ed9d700 1 -- 172.21.15.17:0/3766799018 >> v1:172.21.15.77:6789/0 conn(0x7f5f10067490 legacy=0x7f5f10143a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:12.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.974+0000 7f5f0ed9d700 1 -- 172.21.15.17:0/3766799018 >> v1:172.21.15.17:6789/0 conn(0x7f5f10060290 legacy=0x7f5f100b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:12.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.974+0000 7f5f0ed9d700 1 -- 172.21.15.17:0/3766799018 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f100b3540 con 0x7f5f10144190 2022-08-01T19:39:12.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.974+0000 7f5f0ed9d700 1 --2- 172.21.15.17:0/3766799018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f10144190 0x7f5f1014c9f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f5f080032f0 tx=0x7f5f080091d0).ready entity=mon.0 client_cookie=ba13c67f684d6b6a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:12.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.978+0000 7f5f0dd9b700 1 -- 172.21.15.17:0/3766799018 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f0800b6c0 con 0x7f5f10144190 2022-08-01T19:39:12.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.978+0000 7f5f0dd9b700 1 -- 172.21.15.17:0/3766799018 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f0800b820 con 0x7f5f10144190 2022-08-01T19:39:12.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.978+0000 7f5f0dd9b700 1 -- 172.21.15.17:0/3766799018 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f0800b990 con 0x7f5f10144190 2022-08-01T19:39:12.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.978+0000 7f5f14d14700 1 -- 172.21.15.17:0/3766799018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f10144190 msgr2=0x7f5f1014c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:12.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.978+0000 7f5f14d14700 1 --2- 172.21.15.17:0/3766799018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f10144190 0x7f5f1014c9f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f5f080032f0 tx=0x7f5f080091d0).stop 2022-08-01T19:39:12.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.978+0000 7f5f14d14700 1 -- 172.21.15.17:0/3766799018 shutdown_connections 2022-08-01T19:39:12.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.978+0000 7f5f14d14700 1 --2- 172.21.15.17:0/3766799018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f10144190 0x7f5f1014c9f0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:12.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.978+0000 7f5f14d14700 1 -- 172.21.15.17:0/3766799018 >> 172.21.15.17:0/3766799018 conn(0x7f5f1005d0c0 msgr2=0x7f5f1005f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:12.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.978+0000 7f5f14d14700 1 -- 172.21.15.17:0/3766799018 shutdown_connections 2022-08-01T19:39:12.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.978+0000 7f5f14d14700 1 -- 172.21.15.17:0/3766799018 wait complete. 2022-08-01T19:39:12.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.978+0000 7f5f14d14700 1 Processor -- start 2022-08-01T19:39:12.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.978+0000 7f5f14d14700 1 -- start start 2022-08-01T19:39:12.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.978+0000 7f5f14d14700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f10060290 0x7f5f100ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:12.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.978+0000 7f5f14d14700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5f10067490 0x7f5f100afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:12.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.978+0000 7f5f14d14700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f10144190 0x7f5f10060bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:12.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.978+0000 7f5f14d14700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1013ea70 con 0x7f5f10060290 2022-08-01T19:39:12.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.978+0000 7f5f14d14700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f100b0610 con 0x7f5f10144190 2022-08-01T19:39:12.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.978+0000 7f5f14d14700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f100b1320 con 0x7f5f10067490 2022-08-01T19:39:12.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.978+0000 7f5f0e59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f10060290 0x7f5f100ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:12.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.978+0000 7f5f05d9b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5f10067490 0x7f5f100afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:12.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.978+0000 7f5f0ed9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f10144190 0x7f5f10060bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:12.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.978+0000 7f5f0ed9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f10144190 0x7f5f10060bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:37850/0 (socket says 172.21.15.17:37850) 2022-08-01T19:39:12.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.978+0000 7f5f0e59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f10060290 0x7f5f100ae700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39924/0 (socket says 172.21.15.17:39924) 2022-08-01T19:39:12.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.978+0000 7f5f0ed9d700 1 -- 172.21.15.17:0/2804065903 learned_addr learned my addr 172.21.15.17:0/2804065903 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:12.991 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.982+0000 7f5f0ed9d700 1 -- 172.21.15.17:0/2804065903 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5f10067490 msgr2=0x7f5f100afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:12.991 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.982+0000 7f5f0ed9d700 1 --2- 172.21.15.17:0/2804065903 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5f10067490 0x7f5f100afc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:12.991 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.982+0000 7f5f0ed9d700 1 -- 172.21.15.17:0/2804065903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f10060290 msgr2=0x7f5f100ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:12.991 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.982+0000 7f5f0ed9d700 1 --2- 172.21.15.17:0/2804065903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f10060290 0x7f5f100ae700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:12.991 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.982+0000 7f5f0ed9d700 1 -- 172.21.15.17:0/2804065903 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f08009470 con 0x7f5f10144190 2022-08-01T19:39:12.992 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.982+0000 7f5f0ed9d700 1 --2- 172.21.15.17:0/2804065903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f10144190 0x7f5f10060bd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f5f080032f0 tx=0x7f5f08016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:12.992 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.982+0000 7f5f07fff700 1 -- 172.21.15.17:0/2804065903 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f08011d70 con 0x7f5f10144190 2022-08-01T19:39:12.992 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.982+0000 7f5f07fff700 1 -- 172.21.15.17:0/2804065903 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f08021440 con 0x7f5f10144190 2022-08-01T19:39:12.992 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.982+0000 7f5f14d14700 1 -- 172.21.15.17:0/2804065903 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f1013f2f0 con 0x7f5f10144190 2022-08-01T19:39:12.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.982+0000 7f5f07fff700 1 -- 172.21.15.17:0/2804065903 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f080217a0 con 0x7f5f10144190 2022-08-01T19:39:12.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.982+0000 7f5f14d14700 1 -- 172.21.15.17:0/2804065903 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f1013e3d0 con 0x7f5f10144190 2022-08-01T19:39:12.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.982+0000 7f5f14d14700 1 -- 172.21.15.17:0/2804065903 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f100574e0 con 0x7f5f10144190 2022-08-01T19:39:12.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.982+0000 7f5f07fff700 1 -- 172.21.15.17:0/2804065903 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5f08027020 con 0x7f5f10144190 2022-08-01T19:39:12.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.986+0000 7f5f07fff700 1 --2- 172.21.15.17:0/2804065903 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5ef4053aa0 0x7f5ef4055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:12.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.986+0000 7f5f07fff700 1 -- 172.21.15.17:0/2804065903 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5f0806d450 con 0x7f5f10144190 2022-08-01T19:39:12.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.986+0000 7f5f0e59c700 1 --2- 172.21.15.17:0/2804065903 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5ef4053aa0 0x7f5ef4055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:12.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.986+0000 7f5f0e59c700 1 --2- 172.21.15.17:0/2804065903 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5ef4053aa0 0x7f5ef4055f60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f5f0000d000 tx=0x7f5f00009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:12.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:12.994+0000 7f5f07fff700 1 -- 172.21.15.17:0/2804065903 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5f08044100 con 0x7f5f10144190 2022-08-01T19:39:13.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.162+0000 7f5f14d14700 1 -- 172.21.15.17:0/2804065903 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f5f10069a70 con 0x7f5f10144190 2022-08-01T19:39:13.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.170+0000 7f5f07fff700 1 -- 172.21.15.17:0/2804065903 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v22) v1 ==== 184+0+0 (secure 0 0 0) 0x7f5f080150e0 con 0x7f5f10144190 2022-08-01T19:39:13.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.174+0000 7f5f14d14700 1 -- 172.21.15.17:0/2804065903 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5ef4053aa0 msgr2=0x7f5ef4055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:13.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.174+0000 7f5f14d14700 1 --2- 172.21.15.17:0/2804065903 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5ef4053aa0 0x7f5ef4055f60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f5f0000d000 tx=0x7f5f00009040).stop 2022-08-01T19:39:13.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.174+0000 7f5f14d14700 1 -- 172.21.15.17:0/2804065903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f10144190 msgr2=0x7f5f10060bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:13.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.174+0000 7f5f14d14700 1 --2- 172.21.15.17:0/2804065903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f10144190 0x7f5f10060bd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f5f080032f0 tx=0x7f5f08016000).stop 2022-08-01T19:39:13.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.174+0000 7f5f14d14700 1 -- 172.21.15.17:0/2804065903 shutdown_connections 2022-08-01T19:39:13.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.174+0000 7f5f14d14700 1 --2- 172.21.15.17:0/2804065903 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5ef4053aa0 0x7f5ef4055f60 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:13.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.174+0000 7f5f14d14700 1 --2- 172.21.15.17:0/2804065903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f10144190 0x7f5f10060bd0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:13.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.174+0000 7f5f14d14700 1 --2- 172.21.15.17:0/2804065903 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5f10067490 0x7f5f100afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:13.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.174+0000 7f5f14d14700 1 --2- 172.21.15.17:0/2804065903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f10060290 0x7f5f100ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:13.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.174+0000 7f5f14d14700 1 -- 172.21.15.17:0/2804065903 >> 172.21.15.17:0/2804065903 conn(0x7f5f1005d0c0 msgr2=0x7f5f100652b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:13.182 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.174+0000 7f5f14d14700 1 -- 172.21.15.17:0/2804065903 shutdown_connections 2022-08-01T19:39:13.182 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.174+0000 7f5f14d14700 1 -- 172.21.15.17:0/2804065903 wait complete. 2022-08-01T19:39:13.182 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.1 2022-08-01T19:39:13.192 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:39:13.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.294+0000 7fadc09e9700 1 Processor -- start 2022-08-01T19:39:13.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.294+0000 7fadc09e9700 1 -- start start 2022-08-01T19:39:13.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.294+0000 7fadc09e9700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fadbc0b3190 0x7fadbc060ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:13.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.294+0000 7fadc09e9700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fadbc0b2730 0x7fadbc061190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:13.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.294+0000 7fadc09e9700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadbc0654c0 con 0x7fadbc0666b0 2022-08-01T19:39:13.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.294+0000 7fadc09e9700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadbc062190 con 0x7fadbc0b3190 2022-08-01T19:39:13.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.294+0000 7fadc09e9700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc0b3570 con 0x7fadbc0b2730 2022-08-01T19:39:13.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.294+0000 7fadbbfff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fadbc0666b0 0x7fadbc066b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60360/0 (socket says 172.21.15.17:60360) 2022-08-01T19:39:13.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.294+0000 7fadbbfff700 1 -- 172.21.15.17:0/453462990 learned_addr learned my addr 172.21.15.17:0/453462990 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:13.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.294+0000 7fadbb7fe700 1 --2- 172.21.15.17:0/453462990 >> v2:172.21.15.77:3300/0 conn(0x7fadbc0b3190 0x7fadbc060ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:13.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.294+0000 7fadb3fff700 1 --2- 172.21.15.17:0/453462990 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fadbc0b2730 0x7fadbc061190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:13.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.294+0000 7fadbaffd700 1 -- 172.21.15.17:0/453462990 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3393195725 0 0) 0x7fadbc0654c0 con 0x7fadbc0666b0 2022-08-01T19:39:13.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.294+0000 7fadbaffd700 1 -- 172.21.15.17:0/453462990 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad9c003260 con 0x7fadbc0666b0 2022-08-01T19:39:13.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.294+0000 7fadbb7fe700 1 -- 172.21.15.17:0/453462990 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fadbc0b2730 msgr2=0x7fadbc061190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:13.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.294+0000 7fadbb7fe700 1 --2- 172.21.15.17:0/453462990 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fadbc0b2730 0x7fadbc061190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:13.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.294+0000 7fadbb7fe700 1 -- 172.21.15.17:0/453462990 >> v1:172.21.15.17:6789/0 conn(0x7fadbc0666b0 legacy=0x7fadbc066b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:13.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.294+0000 7fadbb7fe700 1 -- 172.21.15.17:0/453462990 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadbc05fbf0 con 0x7fadbc0b3190 2022-08-01T19:39:13.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.294+0000 7fadbb7fe700 1 --2- 172.21.15.17:0/453462990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadbc0b3190 0x7fadbc060ca0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7fadac002f70 tx=0x7fadac006df0).ready entity=mon.1 client_cookie=4523abc7bd380baf server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:13.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.294+0000 7fadbaffd700 1 -- 172.21.15.17:0/453462990 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadac00d4a0 con 0x7fadbc0b3190 2022-08-01T19:39:13.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.294+0000 7fadbaffd700 1 -- 172.21.15.17:0/453462990 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadac000e90 con 0x7fadbc0b3190 2022-08-01T19:39:13.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.294+0000 7fadbaffd700 1 -- 172.21.15.17:0/453462990 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadac00d7f0 con 0x7fadbc0b3190 2022-08-01T19:39:13.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.294+0000 7fadc09e9700 1 -- 172.21.15.17:0/453462990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadbc0b3190 msgr2=0x7fadbc060ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:13.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.294+0000 7fadc09e9700 1 --2- 172.21.15.17:0/453462990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadbc0b3190 0x7fadbc060ca0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7fadac002f70 tx=0x7fadac006df0).stop 2022-08-01T19:39:13.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.294+0000 7fadc09e9700 1 -- 172.21.15.17:0/453462990 shutdown_connections 2022-08-01T19:39:13.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.294+0000 7fadc09e9700 1 --2- 172.21.15.17:0/453462990 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fadbc0b2730 0x7fadbc061190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:13.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.294+0000 7fadc09e9700 1 --2- 172.21.15.17:0/453462990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadbc0b3190 0x7fadbc060ca0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:13.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.294+0000 7fadc09e9700 1 -- 172.21.15.17:0/453462990 >> 172.21.15.17:0/453462990 conn(0x7fadbc063870 msgr2=0x7fadbc063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:13.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.294+0000 7fadc09e9700 1 -- 172.21.15.17:0/453462990 shutdown_connections 2022-08-01T19:39:13.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.294+0000 7fadc09e9700 1 -- 172.21.15.17:0/453462990 wait complete. 2022-08-01T19:39:13.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.298+0000 7fadc09e9700 1 Processor -- start 2022-08-01T19:39:13.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.298+0000 7fadc09e9700 1 -- start start 2022-08-01T19:39:13.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.298+0000 7fadc09e9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc0666b0 0x7fadbc0ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:13.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.298+0000 7fadc09e9700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fadbc0b2730 0x7fadbc145d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:13.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.298+0000 7fadc09e9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadbc0b3190 0x7fadbc151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:13.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.298+0000 7fadc09e9700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc0ac4e0 con 0x7fadbc0666b0 2022-08-01T19:39:13.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.298+0000 7fadc09e9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc0ac780 con 0x7fadbc0b3190 2022-08-01T19:39:13.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.298+0000 7fadc09e9700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc155590 con 0x7fadbc0b2730 2022-08-01T19:39:13.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.298+0000 7fadbb7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc0666b0 0x7fadbc0ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:13.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.298+0000 7fadbb7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc0666b0 0x7fadbc0ae680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39938/0 (socket says 172.21.15.17:39938) 2022-08-01T19:39:13.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.298+0000 7fadbb7fe700 1 -- 172.21.15.17:0/1690340072 learned_addr learned my addr 172.21.15.17:0/1690340072 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:13.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.298+0000 7fadbbfff700 1 --2- 172.21.15.17:0/1690340072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadbc0b3190 0x7fadbc151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:13.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.298+0000 7fadb3fff700 1 --2- 172.21.15.17:0/1690340072 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fadbc0b2730 0x7fadbc145d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:13.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.298+0000 7fadbb7fe700 1 -- 172.21.15.17:0/1690340072 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fadbc0b2730 msgr2=0x7fadbc145d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:13.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.298+0000 7fadbb7fe700 1 --2- 172.21.15.17:0/1690340072 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fadbc0b2730 0x7fadbc145d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:13.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.298+0000 7fadbb7fe700 1 -- 172.21.15.17:0/1690340072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadbc0b3190 msgr2=0x7fadbc151fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:13.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.298+0000 7fadbb7fe700 1 --2- 172.21.15.17:0/1690340072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadbc0b3190 0x7fadbc151fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:13.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.298+0000 7fadbb7fe700 1 -- 172.21.15.17:0/1690340072 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadac00b180 con 0x7fadbc0666b0 2022-08-01T19:39:13.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.298+0000 7fadbb7fe700 1 --2- 172.21.15.17:0/1690340072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc0666b0 0x7fadbc0ae680 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7fadac002f70 tx=0x7fadac014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:13.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.298+0000 7fadb97fa700 1 -- 172.21.15.17:0/1690340072 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadac016070 con 0x7fadbc0666b0 2022-08-01T19:39:13.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.298+0000 7fadb97fa700 1 -- 172.21.15.17:0/1690340072 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadac013a40 con 0x7fadbc0666b0 2022-08-01T19:39:13.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.298+0000 7fadc09e9700 1 -- 172.21.15.17:0/1690340072 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadbc155830 con 0x7fadbc0666b0 2022-08-01T19:39:13.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.298+0000 7fadb97fa700 1 -- 172.21.15.17:0/1690340072 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadac013da0 con 0x7fadbc0666b0 2022-08-01T19:39:13.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.298+0000 7fadc09e9700 1 -- 172.21.15.17:0/1690340072 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadbc0ac160 con 0x7fadbc0666b0 2022-08-01T19:39:13.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.302+0000 7fadc09e9700 1 -- 172.21.15.17:0/1690340072 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadbc05fe90 con 0x7fadbc0666b0 2022-08-01T19:39:13.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.302+0000 7fadb97fa700 1 -- 172.21.15.17:0/1690340072 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fadac025030 con 0x7fadbc0666b0 2022-08-01T19:39:13.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.302+0000 7fadb97fa700 1 --2- 172.21.15.17:0/1690340072 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fada0053a70 0x7fada0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:13.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.302+0000 7fadb97fa700 1 -- 172.21.15.17:0/1690340072 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fadac06cff0 con 0x7fadbc0666b0 2022-08-01T19:39:13.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.302+0000 7fadb3fff700 1 --2- 172.21.15.17:0/1690340072 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fada0053a70 0x7fada0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:13.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.302+0000 7fadb3fff700 1 --2- 172.21.15.17:0/1690340072 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fada0053a70 0x7fada0055f30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7fada400b000 tx=0x7fada4008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:13.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.310+0000 7fadb97fa700 1 -- 172.21.15.17:0/1690340072 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fadac043ca0 con 0x7fadbc0666b0 2022-08-01T19:39:13.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.470+0000 7fadc09e9700 1 -- 172.21.15.17:0/1690340072 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fadbc0574e0 con 0x7fadbc0666b0 2022-08-01T19:39:13.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.474+0000 7fadb97fa700 1 -- 172.21.15.17:0/1690340072 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+2493 (secure 0 0 0) 0x7fadac023090 con 0x7fadbc0666b0 2022-08-01T19:39:13.476 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:39:13.476 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":30,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5414,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1130955771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1130955771},{"type":"v1","addr":"172.21.15.17:6835","nonce":1130955771}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":28},{"gid":5940,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3691532323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3691532323},{"type":"v1","addr":"172.21.15.17:6833","nonce":3691532323}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":28}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:39:10.932933+0000","modified":"2022-08-01T19:39:11.936834+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5949},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5949":{"gid":5949,"name":"b","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6835/2879527612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2879527612},{"type":"v1","addr":"172.21.15.77:6835","nonce":2879527612}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-08-01T19:39:13.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.478+0000 7fadc09e9700 1 -- 172.21.15.17:0/1690340072 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fada0053a70 msgr2=0x7fada0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:13.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.478+0000 7fadc09e9700 1 --2- 172.21.15.17:0/1690340072 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fada0053a70 0x7fada0055f30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7fada400b000 tx=0x7fada4008040).stop 2022-08-01T19:39:13.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.478+0000 7fadc09e9700 1 -- 172.21.15.17:0/1690340072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc0666b0 msgr2=0x7fadbc0ae680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:13.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.478+0000 7fadc09e9700 1 --2- 172.21.15.17:0/1690340072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc0666b0 0x7fadbc0ae680 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7fadac002f70 tx=0x7fadac014000).stop 2022-08-01T19:39:13.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.478+0000 7fadc09e9700 1 -- 172.21.15.17:0/1690340072 shutdown_connections 2022-08-01T19:39:13.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.478+0000 7fadc09e9700 1 --2- 172.21.15.17:0/1690340072 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fada0053a70 0x7fada0055f30 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:13.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.478+0000 7fadc09e9700 1 --2- 172.21.15.17:0/1690340072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadbc0b3190 0x7fadbc151fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:13.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.478+0000 7fadc09e9700 1 --2- 172.21.15.17:0/1690340072 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fadbc0b2730 0x7fadbc145d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:13.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.478+0000 7fadc09e9700 1 --2- 172.21.15.17:0/1690340072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc0666b0 0x7fadbc0ae680 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:13.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.478+0000 7fadc09e9700 1 -- 172.21.15.17:0/1690340072 >> 172.21.15.17:0/1690340072 conn(0x7fadbc063870 msgr2=0x7fadbc149aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:13.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.478+0000 7fadc09e9700 1 -- 172.21.15.17:0/1690340072 shutdown_connections 2022-08-01T19:39:13.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:13.478+0000 7fadc09e9700 1 -- 172.21.15.17:0/1690340072 wait complete. 2022-08-01T19:39:13.484 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 30 2022-08-01T19:39:13.799 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:39:10.932933+0000', 'modified': '2022-08-01T19:39:11.936834+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5949}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5949': {'gid': 5949, 'name': 'b', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.77:6835/2879527612', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6834', 'nonce': 2879527612}, {'type': 'v1', 'addr': '172.21.15.77:6835', 'nonce': 2879527612}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:39:13.800 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:39:13.800 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-08-01T19:39:13.917 INFO:teuthology.orchestra.run.smithi077.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":5949,"want_state":"up:active","state":"up:active","rank_uptime":2.9747398459999999,"mdsmap_epoch":30,"osdmap_epoch":66,"osdmap_epoch_barrier":66,"uptime":7.994822535} 2022-08-01T19:39:13.918 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 5949, "mdsmap_epoch": 30, "osdmap_epoch": 66, "osdmap_epoch_barrier": 66, "rank_uptime": 2.974739846, "state": "up:active", "uptime": 7.994822535, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:39:13.919 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:39:14.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c1b3f2700 1 Processor -- start 2022-08-01T19:39:14.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c1b3f2700 1 -- start start 2022-08-01T19:39:14.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c1b3f2700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f3c140b6510 0x7f3c1413d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:14.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c1b3f2700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c14066af0 con 0x7f3c140b0e60 2022-08-01T19:39:14.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c1b3f2700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c14066d90 con 0x7f3c14064500 2022-08-01T19:39:14.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c1b3f2700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c14067030 con 0x7f3c140b6510 2022-08-01T19:39:14.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c19bef700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f3c140b0e60 0x7f3c140609c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60372/0 (socket says 172.21.15.17:60372) 2022-08-01T19:39:14.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c19bef700 1 -- 172.21.15.17:0/2976063236 learned_addr learned my addr 172.21.15.17:0/2976063236 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:14.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c1a3f0700 1 --2- 172.21.15.17:0/2976063236 >> v2:172.21.15.77:3300/0 conn(0x7f3c140b6510 0x7f3c1413d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:14.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c193ee700 1 -- 172.21.15.17:0/2976063236 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3670034888 0 0) 0x7f3c14066af0 con 0x7f3c140b0e60 2022-08-01T19:39:14.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c193ee700 1 -- 172.21.15.17:0/2976063236 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bfc003260 con 0x7f3c140b0e60 2022-08-01T19:39:14.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c193ee700 1 -- 172.21.15.17:0/2976063236 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 285964856 0 0) 0x7f3bfc003260 con 0x7f3c140b0e60 2022-08-01T19:39:14.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c193ee700 1 -- 172.21.15.17:0/2976063236 >> v2:172.21.15.77:3300/0 conn(0x7f3c140b6510 msgr2=0x7f3c1413d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:14.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c193ee700 1 --2- 172.21.15.17:0/2976063236 >> v2:172.21.15.77:3300/0 conn(0x7f3c140b6510 0x7f3c1413d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c193ee700 1 -- 172.21.15.17:0/2976063236 >> v1:172.21.15.77:6789/0 conn(0x7f3c14064500 legacy=0x7f3c140610d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:14.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c193ee700 1 -- 172.21.15.17:0/2976063236 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c140672d0 con 0x7f3c140b0e60 2022-08-01T19:39:14.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c193ee700 1 -- 172.21.15.17:0/2976063236 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7f3c10001660 con 0x7f3c140b0e60 2022-08-01T19:39:14.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c1a3f0700 1 --2- 172.21.15.17:0/2976063236 >> v2:172.21.15.77:3300/0 conn(0x7f3c140b6510 0x7f3c1413d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:39:14.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c193ee700 1 -- 172.21.15.17:0/2976063236 >> v1:172.21.15.17:6789/0 conn(0x7f3c140b0e60 legacy=0x7f3c140609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:14.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c193ee700 1 --2- 172.21.15.17:0/2976063236 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3bfc003750 0x7f3bfc003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:14.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c193ee700 1 --2- 172.21.15.17:0/2976063236 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bfc004500 0x7f3bfc004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:14.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c193ee700 1 --2- 172.21.15.17:0/2976063236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bfc004e70 0x7f3bfc00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:14.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c193ee700 1 -- 172.21.15.17:0/2976063236 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c14067030 con 0x7f3bfc004500 2022-08-01T19:39:14.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c193ee700 1 -- 172.21.15.17:0/2976063236 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bfc00bd20 con 0x7f3bfc004e70 2022-08-01T19:39:14.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c193ee700 1 -- 172.21.15.17:0/2976063236 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bfc00bee0 con 0x7f3bfc003750 2022-08-01T19:39:14.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c19bef700 1 --2- 172.21.15.17:0/2976063236 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bfc004500 0x7f3bfc004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:14.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c1abf1700 1 --2- 172.21.15.17:0/2976063236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bfc004e70 0x7f3bfc00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:14.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c1a3f0700 1 --2- 172.21.15.17:0/2976063236 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3bfc003750 0x7f3bfc003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:14.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c19bef700 1 -- 172.21.15.17:0/2976063236 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3bfc003750 msgr2=0x7f3bfc003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:14.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c19bef700 1 --2- 172.21.15.17:0/2976063236 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3bfc003750 0x7f3bfc003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c19bef700 1 -- 172.21.15.17:0/2976063236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bfc004e70 msgr2=0x7f3bfc00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:14.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c19bef700 1 --2- 172.21.15.17:0/2976063236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bfc004e70 0x7f3bfc00b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c19bef700 1 -- 172.21.15.17:0/2976063236 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bfc003260 con 0x7f3bfc004500 2022-08-01T19:39:14.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c1abf1700 1 --2- 172.21.15.17:0/2976063236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bfc004e70 0x7f3bfc00b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:39:14.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c1abf1700 1 -- 172.21.15.17:0/2976063236 reap_dead start 2022-08-01T19:39:14.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c19bef700 1 --2- 172.21.15.17:0/2976063236 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bfc004500 0x7f3bfc004980 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7f3c10001370 tx=0x7f3c14065b90).ready entity=mon.0 client_cookie=7a3a3abc20e5359c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:14.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c193ee700 1 -- 172.21.15.17:0/2976063236 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c100070d0 con 0x7f3bfc004500 2022-08-01T19:39:14.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c193ee700 1 -- 172.21.15.17:0/2976063236 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c10007230 con 0x7f3bfc004500 2022-08-01T19:39:14.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.030+0000 7f3c193ee700 1 -- 172.21.15.17:0/2976063236 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c1000c8f0 con 0x7f3bfc004500 2022-08-01T19:39:14.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.034+0000 7f3c1b3f2700 1 -- 172.21.15.17:0/2976063236 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bfc004500 msgr2=0x7f3bfc004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:14.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.034+0000 7f3c1b3f2700 1 --2- 172.21.15.17:0/2976063236 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bfc004500 0x7f3bfc004980 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7f3c10001370 tx=0x7f3c14065b90).stop 2022-08-01T19:39:14.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.034+0000 7f3c1b3f2700 1 -- 172.21.15.17:0/2976063236 shutdown_connections 2022-08-01T19:39:14.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.034+0000 7f3c1b3f2700 1 --2- 172.21.15.17:0/2976063236 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bfc004500 0x7f3bfc004980 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.034+0000 7f3c1b3f2700 1 -- 172.21.15.17:0/2976063236 >> 172.21.15.17:0/2976063236 conn(0x7f3c14063800 msgr2=0x7f3c14063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:14.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.034+0000 7f3c1b3f2700 1 -- 172.21.15.17:0/2976063236 shutdown_connections 2022-08-01T19:39:14.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.034+0000 7f3c1b3f2700 1 -- 172.21.15.17:0/2976063236 wait complete. 2022-08-01T19:39:14.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.034+0000 7f3c1b3f2700 1 Processor -- start 2022-08-01T19:39:14.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.034+0000 7f3c1b3f2700 1 -- start start 2022-08-01T19:39:14.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.034+0000 7f3c1b3f2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bfc004500 0x7f3c14130950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:14.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.034+0000 7f3c1b3f2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c140ad8b0 0x7f3c140add10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:14.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.034+0000 7f3c1b3f2700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3c1405fd80 0x7f3c14060200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:14.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.034+0000 7f3c1b3f2700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c14142480 con 0x7f3bfc004500 2022-08-01T19:39:14.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.034+0000 7f3c1b3f2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c14060ac0 con 0x7f3c140ad8b0 2022-08-01T19:39:14.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.034+0000 7f3c1b3f2700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c14142020 con 0x7f3c1405fd80 2022-08-01T19:39:14.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.034+0000 7f3c1a3f0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bfc004500 0x7f3c14130950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:14.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.034+0000 7f3c1a3f0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bfc004500 0x7f3c14130950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39958/0 (socket says 172.21.15.17:39958) 2022-08-01T19:39:14.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.034+0000 7f3c1a3f0700 1 -- 172.21.15.17:0/4292636558 learned_addr learned my addr 172.21.15.17:0/4292636558 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:14.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.034+0000 7f3c1abf1700 1 --2- 172.21.15.17:0/4292636558 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3c1405fd80 0x7f3c14060200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:14.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.034+0000 7f3c19bef700 1 --2- 172.21.15.17:0/4292636558 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c140ad8b0 0x7f3c140add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:14.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.038+0000 7f3c1a3f0700 1 -- 172.21.15.17:0/4292636558 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3c1405fd80 msgr2=0x7f3c14060200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:14.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.038+0000 7f3c1a3f0700 1 --2- 172.21.15.17:0/4292636558 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3c1405fd80 0x7f3c14060200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.038+0000 7f3c1a3f0700 1 -- 172.21.15.17:0/4292636558 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c140ad8b0 msgr2=0x7f3c140add10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:14.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.038+0000 7f3c1a3f0700 1 --2- 172.21.15.17:0/4292636558 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c140ad8b0 0x7f3c140add10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.038+0000 7f3c1a3f0700 1 -- 172.21.15.17:0/4292636558 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c14061750 con 0x7f3bfc004500 2022-08-01T19:39:14.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.038+0000 7f3c19bef700 1 --2- 172.21.15.17:0/4292636558 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c140ad8b0 0x7f3c140add10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:39:14.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.038+0000 7f3c1a3f0700 1 --2- 172.21.15.17:0/4292636558 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bfc004500 0x7f3c14130950 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f3c0400b000 tx=0x7f3c04008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:14.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.038+0000 7f3c037fe700 1 -- 172.21.15.17:0/4292636558 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c0400e8d0 con 0x7f3bfc004500 2022-08-01T19:39:14.063 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.038+0000 7f3c037fe700 1 -- 172.21.15.17:0/4292636558 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c0400ea30 con 0x7f3bfc004500 2022-08-01T19:39:14.063 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.038+0000 7f3c1b3f2700 1 -- 172.21.15.17:0/4292636558 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c14142b70 con 0x7f3bfc004500 2022-08-01T19:39:14.063 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.038+0000 7f3c037fe700 1 -- 172.21.15.17:0/4292636558 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c0400ed90 con 0x7f3bfc004500 2022-08-01T19:39:14.064 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.038+0000 7f3c1b3f2700 1 -- 172.21.15.17:0/4292636558 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c140614b0 con 0x7f3bfc004500 2022-08-01T19:39:14.064 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.038+0000 7f3c1b3f2700 1 -- 172.21.15.17:0/4292636558 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c140574e0 con 0x7f3bfc004500 2022-08-01T19:39:14.064 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.038+0000 7f3c037fe700 1 -- 172.21.15.17:0/4292636558 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3c04014470 con 0x7f3bfc004500 2022-08-01T19:39:14.065 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.038+0000 7f3c037fe700 1 --2- 172.21.15.17:0/4292636558 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3bf4053aa0 0x7f3bf4055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:14.065 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.038+0000 7f3c037fe700 1 -- 172.21.15.17:0/4292636558 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f3c0406cb60 con 0x7f3bfc004500 2022-08-01T19:39:14.066 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.038+0000 7f3c19bef700 1 --2- 172.21.15.17:0/4292636558 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3bf4053aa0 0x7f3bf4055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:14.066 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.042+0000 7f3c19bef700 1 --2- 172.21.15.17:0/4292636558 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3bf4053aa0 0x7f3bf4055f60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f3c10017000 tx=0x7f3c10013000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:14.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.050+0000 7f3c037fe700 1 -- 172.21.15.17:0/4292636558 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3c040438c0 con 0x7f3bfc004500 2022-08-01T19:39:14.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.170+0000 7f3c1b3f2700 1 -- 172.21.15.17:0/4292636558 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c140b0a50 con 0x7f3bfc004500 2022-08-01T19:39:14.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.170+0000 7f3c037fe700 1 -- 172.21.15.17:0/4292636558 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f3c04046ee0 con 0x7f3bfc004500 2022-08-01T19:39:14.174 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:39:14.175 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":66,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:39:11.935986+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:39:08.694582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2022-08-01T19:39:09.921664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:39:14.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.174+0000 7f3c1b3f2700 1 -- 172.21.15.17:0/4292636558 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3bf4053aa0 msgr2=0x7f3bf4055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:14.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.174+0000 7f3c1b3f2700 1 --2- 172.21.15.17:0/4292636558 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3bf4053aa0 0x7f3bf4055f60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f3c10017000 tx=0x7f3c10013000).stop 2022-08-01T19:39:14.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.174+0000 7f3c1b3f2700 1 -- 172.21.15.17:0/4292636558 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bfc004500 msgr2=0x7f3c14130950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:14.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.174+0000 7f3c1b3f2700 1 --2- 172.21.15.17:0/4292636558 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bfc004500 0x7f3c14130950 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f3c0400b000 tx=0x7f3c04008040).stop 2022-08-01T19:39:14.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.174+0000 7f3c1b3f2700 1 -- 172.21.15.17:0/4292636558 shutdown_connections 2022-08-01T19:39:14.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.174+0000 7f3c1b3f2700 1 --2- 172.21.15.17:0/4292636558 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3bf4053aa0 0x7f3bf4055f60 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.174+0000 7f3c1b3f2700 1 --2- 172.21.15.17:0/4292636558 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3c1405fd80 0x7f3c14060200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.174+0000 7f3c1b3f2700 1 --2- 172.21.15.17:0/4292636558 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c140ad8b0 0x7f3c140add10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.174+0000 7f3c1b3f2700 1 --2- 172.21.15.17:0/4292636558 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bfc004500 0x7f3c14130950 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.174+0000 7f3c1b3f2700 1 -- 172.21.15.17:0/4292636558 >> 172.21.15.17:0/4292636558 conn(0x7f3c14063800 msgr2=0x7f3c1413ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:14.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.174+0000 7f3c1b3f2700 1 -- 172.21.15.17:0/4292636558 shutdown_connections 2022-08-01T19:39:14.182 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.174+0000 7f3c1b3f2700 1 -- 172.21.15.17:0/4292636558 wait complete. 2022-08-01T19:39:14.188 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T19:39:08.694582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T19:39:09.921664+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T19:39:14.189 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:39:14.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.298+0000 7f194e0a5700 1 Processor -- start 2022-08-01T19:39:14.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.298+0000 7f194e0a5700 1 -- start start 2022-08-01T19:39:14.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.298+0000 7f194e0a5700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f19480a9320 0x7f19481466e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:14.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.298+0000 7f194e0a5700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f19480b0d10 0x7f1948146bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:14.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.298+0000 7f194e0a5700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19480b12d0 con 0x7f19480b0560 2022-08-01T19:39:14.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.298+0000 7f194e0a5700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19480a9700 con 0x7f19480b0d10 2022-08-01T19:39:14.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.298+0000 7f194e0a5700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19480b1500 con 0x7f19480a9320 2022-08-01T19:39:14.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.298+0000 7f194c8a2700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f19480b0d10 0x7f1948146bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:14.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.298+0000 7f194c8a2700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f19480b0d10 0x7f1948146bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39968/0 (socket says 172.21.15.17:39968) 2022-08-01T19:39:14.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.298+0000 7f194c8a2700 1 -- 172.21.15.17:0/2996650229 learned_addr learned my addr 172.21.15.17:0/2996650229 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:14.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.298+0000 7f194d8a4700 1 --1- 172.21.15.17:0/2996650229 >> v1:172.21.15.77:6789/0 conn(0x7f19480b0560 0x7f1948140350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:48964/0 (socket says 172.21.15.17:48964) 2022-08-01T19:39:14.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.298+0000 7f194d0a3700 1 --2- 172.21.15.17:0/2996650229 >> v2:172.21.15.77:3300/0 conn(0x7f19480a9320 0x7f19481466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:14.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.302+0000 7f194c8a2700 1 -- 172.21.15.17:0/2996650229 >> v2:172.21.15.77:3300/0 conn(0x7f19480a9320 msgr2=0x7f19481466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:14.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.302+0000 7f194c8a2700 1 --2- 172.21.15.17:0/2996650229 >> v2:172.21.15.77:3300/0 conn(0x7f19480a9320 0x7f19481466e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.302+0000 7f194c8a2700 1 -- 172.21.15.17:0/2996650229 >> v1:172.21.15.77:6789/0 conn(0x7f19480b0560 legacy=0x7f1948140350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:14.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.302+0000 7f194c8a2700 1 -- 172.21.15.17:0/2996650229 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19480b17a0 con 0x7f19480b0d10 2022-08-01T19:39:14.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.302+0000 7f194c8a2700 1 --2- 172.21.15.17:0/2996650229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19480b0d10 0x7f1948146bd0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f1938002f70 tx=0x7f1938006eb0).ready entity=mon.0 client_cookie=82e0b4a0d164a9af server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:14.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.302+0000 7f193ffff700 1 -- 172.21.15.17:0/2996650229 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f193800d5c0 con 0x7f19480b0d10 2022-08-01T19:39:14.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.302+0000 7f193ffff700 1 -- 172.21.15.17:0/2996650229 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1938000e90 con 0x7f19480b0d10 2022-08-01T19:39:14.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.302+0000 7f193ffff700 1 -- 172.21.15.17:0/2996650229 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f193800d950 con 0x7f19480b0d10 2022-08-01T19:39:14.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.302+0000 7f194e0a5700 1 -- 172.21.15.17:0/2996650229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19480b0d10 msgr2=0x7f1948146bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:14.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.302+0000 7f194e0a5700 1 --2- 172.21.15.17:0/2996650229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19480b0d10 0x7f1948146bd0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f1938002f70 tx=0x7f1938006eb0).stop 2022-08-01T19:39:14.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.302+0000 7f194e0a5700 1 -- 172.21.15.17:0/2996650229 shutdown_connections 2022-08-01T19:39:14.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.302+0000 7f194e0a5700 1 --2- 172.21.15.17:0/2996650229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19480b0d10 0x7f1948146bd0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.302+0000 7f194e0a5700 1 --2- 172.21.15.17:0/2996650229 >> v2:172.21.15.77:3300/0 conn(0x7f19480a9320 0x7f19481466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.302+0000 7f194e0a5700 1 -- 172.21.15.17:0/2996650229 >> 172.21.15.17:0/2996650229 conn(0x7f19480612d0 msgr2=0x7f1948062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:14.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.302+0000 7f194e0a5700 1 -- 172.21.15.17:0/2996650229 shutdown_connections 2022-08-01T19:39:14.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.302+0000 7f194e0a5700 1 -- 172.21.15.17:0/2996650229 wait complete. 2022-08-01T19:39:14.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.306+0000 7f194e0a5700 1 Processor -- start 2022-08-01T19:39:14.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.306+0000 7f194e0a5700 1 -- start start 2022-08-01T19:39:14.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.306+0000 7f194e0a5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19480a9320 0x7f19480af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:14.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.306+0000 7f194e0a5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19480b0560 0x7f1948152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:14.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.306+0000 7f194e0a5700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f19480b0d10 0x7f19481530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:14.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.306+0000 7f194e0a5700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1948152530 con 0x7f19480b0560 2022-08-01T19:39:14.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.306+0000 7f194e0a5700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19480ac030 con 0x7f19480a9320 2022-08-01T19:39:14.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.306+0000 7f194e0a5700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1948152720 con 0x7f19480b0d10 2022-08-01T19:39:14.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.306+0000 7f194c8a2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19480b0560 0x7f1948152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:14.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.306+0000 7f194c8a2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19480b0560 0x7f1948152bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39972/0 (socket says 172.21.15.17:39972) 2022-08-01T19:39:14.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.306+0000 7f194c8a2700 1 -- 172.21.15.17:0/570164646 learned_addr learned my addr 172.21.15.17:0/570164646 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:14.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.306+0000 7f194d0a3700 1 --2- 172.21.15.17:0/570164646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19480a9320 0x7f19480af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:14.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.306+0000 7f194d8a4700 1 --2- 172.21.15.17:0/570164646 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f19480b0d10 0x7f19481530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:14.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.306+0000 7f194c8a2700 1 -- 172.21.15.17:0/570164646 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f19480b0d10 msgr2=0x7f19481530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:14.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.306+0000 7f194c8a2700 1 --2- 172.21.15.17:0/570164646 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f19480b0d10 0x7f19481530b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.306+0000 7f194c8a2700 1 -- 172.21.15.17:0/570164646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19480a9320 msgr2=0x7f19480af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:14.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.306+0000 7f194c8a2700 1 --2- 172.21.15.17:0/570164646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19480a9320 0x7f19480af2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.306+0000 7f194c8a2700 1 -- 172.21.15.17:0/570164646 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f193800b1e0 con 0x7f19480b0560 2022-08-01T19:39:14.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.306+0000 7f194c8a2700 1 --2- 172.21.15.17:0/570164646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19480b0560 0x7f1948152bc0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f1938002f70 tx=0x7f1938015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:14.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.306+0000 7f193e7fc700 1 -- 172.21.15.17:0/570164646 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f193800d940 con 0x7f19480b0560 2022-08-01T19:39:14.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.306+0000 7f193e7fc700 1 -- 172.21.15.17:0/570164646 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1938011b80 con 0x7f19480b0560 2022-08-01T19:39:14.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.306+0000 7f194e0a5700 1 -- 172.21.15.17:0/570164646 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19480ab9a0 con 0x7f19480b0560 2022-08-01T19:39:14.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.306+0000 7f193e7fc700 1 -- 172.21.15.17:0/570164646 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f193801e460 con 0x7f19480b0560 2022-08-01T19:39:14.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.306+0000 7f194e0a5700 1 -- 172.21.15.17:0/570164646 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1948067c90 con 0x7f19480b0560 2022-08-01T19:39:14.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.310+0000 7f194e0a5700 1 -- 172.21.15.17:0/570164646 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19481528e0 con 0x7f19480b0560 2022-08-01T19:39:14.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.310+0000 7f193e7fc700 1 -- 172.21.15.17:0/570164646 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1938025030 con 0x7f19480b0560 2022-08-01T19:39:14.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.310+0000 7f193e7fc700 1 --2- 172.21.15.17:0/570164646 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1930053aa0 0x7f1930055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:14.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.310+0000 7f193e7fc700 1 -- 172.21.15.17:0/570164646 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f193806cf80 con 0x7f19480b0560 2022-08-01T19:39:14.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.310+0000 7f194d0a3700 1 --2- 172.21.15.17:0/570164646 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1930053aa0 0x7f1930055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:14.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.310+0000 7f194d0a3700 1 --2- 172.21.15.17:0/570164646 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1930053aa0 0x7f1930055f60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f194400b000 tx=0x7f1944006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:14.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.314+0000 7f193e7fc700 1 -- 172.21.15.17:0/570164646 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1938043e00 con 0x7f19480b0560 2022-08-01T19:39:14.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.442+0000 7f194e0a5700 1 -- 172.21.15.17:0/570164646 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f19480574e0 con 0x7f19480b0560 2022-08-01T19:39:14.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.442+0000 7f193e7fc700 1 -- 172.21.15.17:0/570164646 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v66) v1 ==== 110+0+10 (secure 0 0 0) 0x7f1938023020 con 0x7f19480b0560 2022-08-01T19:39:14.446 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:39:14.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.446+0000 7f194e0a5700 1 -- 172.21.15.17:0/570164646 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1930053aa0 msgr2=0x7f1930055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:14.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.446+0000 7f194e0a5700 1 --2- 172.21.15.17:0/570164646 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1930053aa0 0x7f1930055f60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f194400b000 tx=0x7f1944006df0).stop 2022-08-01T19:39:14.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.446+0000 7f194e0a5700 1 -- 172.21.15.17:0/570164646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19480b0560 msgr2=0x7f1948152bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:14.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.446+0000 7f194e0a5700 1 --2- 172.21.15.17:0/570164646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19480b0560 0x7f1948152bc0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f1938002f70 tx=0x7f1938015000).stop 2022-08-01T19:39:14.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.446+0000 7f194e0a5700 1 -- 172.21.15.17:0/570164646 shutdown_connections 2022-08-01T19:39:14.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.446+0000 7f194e0a5700 1 --2- 172.21.15.17:0/570164646 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1930053aa0 0x7f1930055f60 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.446+0000 7f194e0a5700 1 --2- 172.21.15.17:0/570164646 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f19480b0d10 0x7f19481530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.446+0000 7f194e0a5700 1 --2- 172.21.15.17:0/570164646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19480b0560 0x7f1948152bc0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.446+0000 7f194e0a5700 1 --2- 172.21.15.17:0/570164646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19480a9320 0x7f19480af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.446+0000 7f194e0a5700 1 -- 172.21.15.17:0/570164646 >> 172.21.15.17:0/570164646 conn(0x7f19480612d0 msgr2=0x7f1948141030 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:14.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.446+0000 7f194e0a5700 1 -- 172.21.15.17:0/570164646 shutdown_connections 2022-08-01T19:39:14.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.446+0000 7f194e0a5700 1 -- 172.21.15.17:0/570164646 wait complete. 2022-08-01T19:39:14.459 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:39:14.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.562+0000 7f97e3a55700 1 Processor -- start 2022-08-01T19:39:14.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.562+0000 7f97e3a55700 1 -- start start 2022-08-01T19:39:14.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.562+0000 7f97e3a55700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f97dc0b2050 0x7f97dc0b1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:14.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.562+0000 7f97e3a55700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f97dc0b2430 0x7f97dc1413e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:14.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.562+0000 7f97e3a55700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97dc0b05b0 con 0x7f97dc0b0140 2022-08-01T19:39:14.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.562+0000 7f97e3a55700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97dc0a7960 con 0x7f97dc0b2430 2022-08-01T19:39:14.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.562+0000 7f97e3a55700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97dc0b07e0 con 0x7f97dc0b2050 2022-08-01T19:39:14.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.562+0000 7f97e0ff0700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f97dc0b0140 0x7f97dc0a7250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60406/0 (socket says 172.21.15.17:60406) 2022-08-01T19:39:14.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.562+0000 7f97e0ff0700 1 -- 172.21.15.17:0/1028041249 learned_addr learned my addr 172.21.15.17:0/1028041249 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:14.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.562+0000 7f97e1ff2700 1 --2- 172.21.15.17:0/1028041249 >> v2:172.21.15.17:3300/0 conn(0x7f97dc0b2430 0x7f97dc1413e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:14.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.562+0000 7f97e17f1700 1 --2- 172.21.15.17:0/1028041249 >> v2:172.21.15.77:3300/0 conn(0x7f97dc0b2050 0x7f97dc0b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:14.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.562+0000 7f97d3fff700 1 -- 172.21.15.17:0/1028041249 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2186835244 0 0) 0x7f97dc0b05b0 con 0x7f97dc0b0140 2022-08-01T19:39:14.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.562+0000 7f97d3fff700 1 -- 172.21.15.17:0/1028041249 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97c0003260 con 0x7f97dc0b0140 2022-08-01T19:39:14.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.562+0000 7f97e17f1700 1 -- 172.21.15.17:0/1028041249 >> v2:172.21.15.17:3300/0 conn(0x7f97dc0b2430 msgr2=0x7f97dc1413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:14.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.562+0000 7f97e17f1700 1 --2- 172.21.15.17:0/1028041249 >> v2:172.21.15.17:3300/0 conn(0x7f97dc0b2430 0x7f97dc1413e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.562+0000 7f97e17f1700 1 -- 172.21.15.17:0/1028041249 >> v1:172.21.15.17:6789/0 conn(0x7f97dc0b0140 legacy=0x7f97dc0a7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:14.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.562+0000 7f97e17f1700 1 -- 172.21.15.17:0/1028041249 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97dc0b0a80 con 0x7f97dc0b2050 2022-08-01T19:39:14.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.562+0000 7f97e17f1700 1 --2- 172.21.15.17:0/1028041249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97dc0b2050 0x7f97dc0b1460 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f97d80032f0 tx=0x7f97d80091d0).ready entity=mon.1 client_cookie=c734a13403b791db server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:14.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.562+0000 7f97d3fff700 1 -- 172.21.15.17:0/1028041249 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97d800b600 con 0x7f97dc0b2050 2022-08-01T19:39:14.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.562+0000 7f97d3fff700 1 -- 172.21.15.17:0/1028041249 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d800b760 con 0x7f97dc0b2050 2022-08-01T19:39:14.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.562+0000 7f97d3fff700 1 -- 172.21.15.17:0/1028041249 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97d800b8d0 con 0x7f97dc0b2050 2022-08-01T19:39:14.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.566+0000 7f97e3a55700 1 -- 172.21.15.17:0/1028041249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97dc0b2050 msgr2=0x7f97dc0b1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:14.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.566+0000 7f97e3a55700 1 --2- 172.21.15.17:0/1028041249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97dc0b2050 0x7f97dc0b1460 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f97d80032f0 tx=0x7f97d80091d0).stop 2022-08-01T19:39:14.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.566+0000 7f97e3a55700 1 -- 172.21.15.17:0/1028041249 shutdown_connections 2022-08-01T19:39:14.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.566+0000 7f97e3a55700 1 --2- 172.21.15.17:0/1028041249 >> v2:172.21.15.17:3300/0 conn(0x7f97dc0b2430 0x7f97dc1413e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.566+0000 7f97e3a55700 1 --2- 172.21.15.17:0/1028041249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97dc0b2050 0x7f97dc0b1460 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.566+0000 7f97e3a55700 1 -- 172.21.15.17:0/1028041249 >> 172.21.15.17:0/1028041249 conn(0x7f97dc05f840 msgr2=0x7f97dc05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:14.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.566+0000 7f97e3a55700 1 -- 172.21.15.17:0/1028041249 shutdown_connections 2022-08-01T19:39:14.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.566+0000 7f97e3a55700 1 -- 172.21.15.17:0/1028041249 wait complete. 2022-08-01T19:39:14.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.566+0000 7f97e3a55700 1 Processor -- start 2022-08-01T19:39:14.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.566+0000 7f97e3a55700 1 -- start start 2022-08-01T19:39:14.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.566+0000 7f97e3a55700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97dc0b0140 0x7f97dc14d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:14.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.566+0000 7f97e3a55700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f97dc0b2050 0x7f97dc13a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:14.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.566+0000 7f97e3a55700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f97dc0b2430 0x7f97dc13a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:14.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.566+0000 7f97e3a55700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc140660 con 0x7f97dc0b2430 2022-08-01T19:39:14.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.566+0000 7f97e3a55700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc140900 con 0x7f97dc0b0140 2022-08-01T19:39:14.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.566+0000 7f97e3a55700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc140ba0 con 0x7f97dc0b2050 2022-08-01T19:39:14.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.566+0000 7f97e1ff2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f97dc0b2430 0x7f97dc13a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:14.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.566+0000 7f97e17f1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97dc0b0140 0x7f97dc14d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:14.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.566+0000 7f97e1ff2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f97dc0b2430 0x7f97dc13a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39990/0 (socket says 172.21.15.17:39990) 2022-08-01T19:39:14.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.566+0000 7f97e1ff2700 1 -- 172.21.15.17:0/4013154815 learned_addr learned my addr 172.21.15.17:0/4013154815 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:14.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.566+0000 7f97e0ff0700 1 --2- 172.21.15.17:0/4013154815 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f97dc0b2050 0x7f97dc13a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:14.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.566+0000 7f97e17f1700 1 -- 172.21.15.17:0/4013154815 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f97dc0b2050 msgr2=0x7f97dc13a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:14.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.566+0000 7f97e17f1700 1 --2- 172.21.15.17:0/4013154815 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f97dc0b2050 0x7f97dc13a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.570+0000 7f97e17f1700 1 -- 172.21.15.17:0/4013154815 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f97dc0b2430 msgr2=0x7f97dc13a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:14.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.570+0000 7f97e17f1700 1 --2- 172.21.15.17:0/4013154815 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f97dc0b2430 0x7f97dc13a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.570+0000 7f97e17f1700 1 -- 172.21.15.17:0/4013154815 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d8009470 con 0x7f97dc0b0140 2022-08-01T19:39:14.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.570+0000 7f97e1ff2700 1 --2- 172.21.15.17:0/4013154815 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f97dc0b2430 0x7f97dc13a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:39:14.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.570+0000 7f97e17f1700 1 --2- 172.21.15.17:0/4013154815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97dc0b0140 0x7f97dc14d900 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f97d80032f0 tx=0x7f97d8016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:14.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.570+0000 7f97d27fc700 1 -- 172.21.15.17:0/4013154815 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97d8011b60 con 0x7f97dc0b0140 2022-08-01T19:39:14.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.570+0000 7f97d27fc700 1 -- 172.21.15.17:0/4013154815 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d8011cc0 con 0x7f97dc0b0140 2022-08-01T19:39:14.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.570+0000 7f97e3a55700 1 -- 172.21.15.17:0/4013154815 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97dc13b510 con 0x7f97dc0b0140 2022-08-01T19:39:14.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.570+0000 7f97d27fc700 1 -- 172.21.15.17:0/4013154815 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97d801f5e0 con 0x7f97dc0b0140 2022-08-01T19:39:14.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.570+0000 7f97e3a55700 1 -- 172.21.15.17:0/4013154815 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97dc1433a0 con 0x7f97dc0b0140 2022-08-01T19:39:14.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.570+0000 7f97e3a55700 1 -- 172.21.15.17:0/4013154815 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97dc140d60 con 0x7f97dc0b0140 2022-08-01T19:39:14.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.570+0000 7f97d27fc700 1 -- 172.21.15.17:0/4013154815 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f97d8026030 con 0x7f97dc0b0140 2022-08-01T19:39:14.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.570+0000 7f97d27fc700 1 --2- 172.21.15.17:0/4013154815 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f97c4053950 0x7f97c4055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:14.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.570+0000 7f97d27fc700 1 -- 172.21.15.17:0/4013154815 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f97d806cc50 con 0x7f97dc0b0140 2022-08-01T19:39:14.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.570+0000 7f97e0ff0700 1 --2- 172.21.15.17:0/4013154815 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f97c4053950 0x7f97c4055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:14.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.574+0000 7f97e0ff0700 1 --2- 172.21.15.17:0/4013154815 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f97c4053950 0x7f97c4055e10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f97cc00d000 tx=0x7f97cc009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:14.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.578+0000 7f97d27fc700 1 -- 172.21.15.17:0/4013154815 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f97d8043900 con 0x7f97dc0b0140 2022-08-01T19:39:14.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.706+0000 7f97e3a55700 1 -- 172.21.15.17:0/4013154815 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f97dc0b0d20 con 0x7f97dc0b0140 2022-08-01T19:39:14.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.706+0000 7f97d27fc700 1 -- 172.21.15.17:0/4013154815 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v66) v1 ==== 92+0+10 (secure 0 0 0) 0x7f97d8024020 con 0x7f97dc0b0140 2022-08-01T19:39:14.710 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:39:14.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.710+0000 7f97e3a55700 1 -- 172.21.15.17:0/4013154815 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f97c4053950 msgr2=0x7f97c4055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:14.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.710+0000 7f97e3a55700 1 --2- 172.21.15.17:0/4013154815 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f97c4053950 0x7f97c4055e10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f97cc00d000 tx=0x7f97cc009040).stop 2022-08-01T19:39:14.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.710+0000 7f97e3a55700 1 -- 172.21.15.17:0/4013154815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97dc0b0140 msgr2=0x7f97dc14d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:14.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.710+0000 7f97e3a55700 1 --2- 172.21.15.17:0/4013154815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97dc0b0140 0x7f97dc14d900 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f97d80032f0 tx=0x7f97d8016000).stop 2022-08-01T19:39:14.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.710+0000 7f97e3a55700 1 -- 172.21.15.17:0/4013154815 shutdown_connections 2022-08-01T19:39:14.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.710+0000 7f97e3a55700 1 --2- 172.21.15.17:0/4013154815 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f97c4053950 0x7f97c4055e10 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.710+0000 7f97e3a55700 1 --2- 172.21.15.17:0/4013154815 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f97dc0b2430 0x7f97dc13a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.710+0000 7f97e3a55700 1 --2- 172.21.15.17:0/4013154815 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f97dc0b2050 0x7f97dc13a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.710+0000 7f97e3a55700 1 --2- 172.21.15.17:0/4013154815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97dc0b0140 0x7f97dc14d900 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.710+0000 7f97e3a55700 1 -- 172.21.15.17:0/4013154815 >> 172.21.15.17:0/4013154815 conn(0x7f97dc05f840 msgr2=0x7f97dc146300 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:14.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.710+0000 7f97e3a55700 1 -- 172.21.15.17:0/4013154815 shutdown_connections 2022-08-01T19:39:14.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.710+0000 7f97e3a55700 1 -- 172.21.15.17:0/4013154815 wait complete. 2022-08-01T19:39:14.723 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T19:39:14.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.830+0000 7f7eb244b700 1 Processor -- start 2022-08-01T19:39:14.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.830+0000 7f7eb244b700 1 -- start start 2022-08-01T19:39:14.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.830+0000 7f7eb244b700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f7eac060290 0x7f7eac064ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:14.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.830+0000 7f7eb244b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7eac0b3a60 0x7f7eac065390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:14.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.830+0000 7f7eb244b700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7eac060670 con 0x7f7eac1445d0 2022-08-01T19:39:14.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.830+0000 7f7eb244b700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7eac065880 con 0x7f7eac060290 2022-08-01T19:39:14.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.830+0000 7f7eb244b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eac065e10 con 0x7f7eac0b3a60 2022-08-01T19:39:14.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.830+0000 7f7eab7fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f7eac1445d0 0x7f7eac144b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60420/0 (socket says 172.21.15.17:60420) 2022-08-01T19:39:14.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.830+0000 7f7eab7fe700 1 -- 172.21.15.17:0/3563217311 learned_addr learned my addr 172.21.15.17:0/3563217311 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:14.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.830+0000 7f7eb09e8700 1 --2- 172.21.15.17:0/3563217311 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7eac0b3a60 0x7f7eac065390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:14.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.830+0000 7f7eabfff700 1 --2- 172.21.15.17:0/3563217311 >> v2:172.21.15.77:3300/0 conn(0x7f7eac060290 0x7f7eac064ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:14.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.830+0000 7f7eaaffd700 1 -- 172.21.15.17:0/3563217311 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1827942228 0 0) 0x7f7eac060670 con 0x7f7eac1445d0 2022-08-01T19:39:14.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.830+0000 7f7eaaffd700 1 -- 172.21.15.17:0/3563217311 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e98003260 con 0x7f7eac1445d0 2022-08-01T19:39:14.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.830+0000 7f7eb09e8700 1 -- 172.21.15.17:0/3563217311 >> v2:172.21.15.77:3300/0 conn(0x7f7eac060290 msgr2=0x7f7eac064ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:14.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.830+0000 7f7eb09e8700 1 --2- 172.21.15.17:0/3563217311 >> v2:172.21.15.77:3300/0 conn(0x7f7eac060290 0x7f7eac064ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.830+0000 7f7eb09e8700 1 -- 172.21.15.17:0/3563217311 >> v1:172.21.15.17:6789/0 conn(0x7f7eac1445d0 legacy=0x7f7eac144b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:14.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.830+0000 7f7eb09e8700 1 -- 172.21.15.17:0/3563217311 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eac0b2540 con 0x7f7eac0b3a60 2022-08-01T19:39:14.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.830+0000 7f7eb09e8700 1 --2- 172.21.15.17:0/3563217311 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7eac0b3a60 0x7f7eac065390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f7ea40032f0 tx=0x7f7ea40091d0).ready entity=mon.2 client_cookie=90d10006cf6ff71d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:14.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.830+0000 7f7eaaffd700 1 -- 172.21.15.17:0/3563217311 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ea400b600 con 0x7f7eac0b3a60 2022-08-01T19:39:14.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.830+0000 7f7eaaffd700 1 -- 172.21.15.17:0/3563217311 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea400b760 con 0x7f7eac0b3a60 2022-08-01T19:39:14.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.830+0000 7f7eaaffd700 1 -- 172.21.15.17:0/3563217311 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ea400b8d0 con 0x7f7eac0b3a60 2022-08-01T19:39:14.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.834+0000 7f7eb244b700 1 -- 172.21.15.17:0/3563217311 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7eac0b3a60 msgr2=0x7f7eac065390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:14.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.834+0000 7f7eb244b700 1 --2- 172.21.15.17:0/3563217311 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7eac0b3a60 0x7f7eac065390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f7ea40032f0 tx=0x7f7ea40091d0).stop 2022-08-01T19:39:14.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.834+0000 7f7eb244b700 1 -- 172.21.15.17:0/3563217311 shutdown_connections 2022-08-01T19:39:14.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.834+0000 7f7eb244b700 1 --2- 172.21.15.17:0/3563217311 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7eac0b3a60 0x7f7eac065390 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.834+0000 7f7eb244b700 1 --2- 172.21.15.17:0/3563217311 >> v2:172.21.15.77:3300/0 conn(0x7f7eac060290 0x7f7eac064ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.834+0000 7f7eb244b700 1 -- 172.21.15.17:0/3563217311 >> 172.21.15.17:0/3563217311 conn(0x7f7eac05d0f0 msgr2=0x7f7eac05f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:14.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.834+0000 7f7eb244b700 1 -- 172.21.15.17:0/3563217311 shutdown_connections 2022-08-01T19:39:14.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.834+0000 7f7eb244b700 1 -- 172.21.15.17:0/3563217311 wait complete. 2022-08-01T19:39:14.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.834+0000 7f7eb244b700 1 Processor -- start 2022-08-01T19:39:14.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.834+0000 7f7eb244b700 1 -- start start 2022-08-01T19:39:14.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.834+0000 7f7eb244b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7eac060290 0x7f7eac0af460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:14.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.834+0000 7f7eb244b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7eac0b3a60 0x7f7eac0addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:14.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.834+0000 7f7eb244b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7eac0ae2a0 0x7f7eac150730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:14.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.834+0000 7f7eb244b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eac0ae750 con 0x7f7eac0b3a60 2022-08-01T19:39:14.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.834+0000 7f7eb244b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eac0b1320 con 0x7f7eac0ae2a0 2022-08-01T19:39:14.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.834+0000 7f7eb244b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eac0b15c0 con 0x7f7eac060290 2022-08-01T19:39:14.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.834+0000 7f7eab7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7eac0b3a60 0x7f7eac0addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:14.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.834+0000 7f7eab7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7eac0b3a60 0x7f7eac0addb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40002/0 (socket says 172.21.15.17:40002) 2022-08-01T19:39:14.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.834+0000 7f7eab7fe700 1 -- 172.21.15.17:0/37401374 learned_addr learned my addr 172.21.15.17:0/37401374 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:14.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.834+0000 7f7eb09e8700 1 --2- 172.21.15.17:0/37401374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7eac0ae2a0 0x7f7eac150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:14.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.834+0000 7f7eabfff700 1 --2- 172.21.15.17:0/37401374 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7eac060290 0x7f7eac0af460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:14.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.834+0000 7f7eab7fe700 1 -- 172.21.15.17:0/37401374 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7eac060290 msgr2=0x7f7eac0af460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:14.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.834+0000 7f7eab7fe700 1 --2- 172.21.15.17:0/37401374 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7eac060290 0x7f7eac0af460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.834+0000 7f7eab7fe700 1 -- 172.21.15.17:0/37401374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7eac0ae2a0 msgr2=0x7f7eac150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:14.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.834+0000 7f7eab7fe700 1 --2- 172.21.15.17:0/37401374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7eac0ae2a0 0x7f7eac150730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.834+0000 7f7eab7fe700 1 -- 172.21.15.17:0/37401374 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea4009470 con 0x7f7eac0b3a60 2022-08-01T19:39:14.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.834+0000 7f7eabfff700 1 --2- 172.21.15.17:0/37401374 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7eac060290 0x7f7eac0af460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:39:14.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.838+0000 7f7eab7fe700 1 --2- 172.21.15.17:0/37401374 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7eac0b3a60 0x7f7eac0addb0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7f7e9c001110 tx=0x7f7e9c007a80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:14.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.838+0000 7f7ea97fa700 1 -- 172.21.15.17:0/37401374 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e9c00ce80 con 0x7f7eac0b3a60 2022-08-01T19:39:14.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.838+0000 7f7ea97fa700 1 -- 172.21.15.17:0/37401374 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9c007110 con 0x7f7eac0b3a60 2022-08-01T19:39:14.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.838+0000 7f7eb244b700 1 -- 172.21.15.17:0/37401374 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eac0b0620 con 0x7f7eac0b3a60 2022-08-01T19:39:14.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.838+0000 7f7eb244b700 1 -- 172.21.15.17:0/37401374 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eac0b0b60 con 0x7f7eac0b3a60 2022-08-01T19:39:14.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.838+0000 7f7ea97fa700 1 -- 172.21.15.17:0/37401374 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7e9c007470 con 0x7f7eac0b3a60 2022-08-01T19:39:14.858 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.838+0000 7f7eb244b700 1 -- 172.21.15.17:0/37401374 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eac0574e0 con 0x7f7eac0b3a60 2022-08-01T19:39:14.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.838+0000 7f7ea97fa700 1 -- 172.21.15.17:0/37401374 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7e9c01b070 con 0x7f7eac0b3a60 2022-08-01T19:39:14.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.838+0000 7f7ea97fa700 1 --2- 172.21.15.17:0/37401374 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7e8c053a10 0x7f7e8c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:14.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.838+0000 7f7ea97fa700 1 -- 172.21.15.17:0/37401374 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7e9c06b550 con 0x7f7eac0b3a60 2022-08-01T19:39:14.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.838+0000 7f7eabfff700 1 --2- 172.21.15.17:0/37401374 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7e8c053a10 0x7f7e8c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:14.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.842+0000 7f7eabfff700 1 --2- 172.21.15.17:0/37401374 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7e8c053a10 0x7f7e8c055ed0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f7ea000b000 tx=0x7f7ea0008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:14.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.854+0000 7f7ea97fa700 1 -- 172.21.15.17:0/37401374 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7e9c042270 con 0x7f7eac0b3a60 2022-08-01T19:39:14.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.974+0000 7f7eb244b700 1 -- 172.21.15.17:0/37401374 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7eac0b1100 con 0x7f7eac0b3a60 2022-08-01T19:39:14.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.974+0000 7f7ea97fa700 1 -- 172.21.15.17:0/37401374 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v66) v1 ==== 104+0+10 (secure 0 0 0) 0x7f7e9c045890 con 0x7f7eac0b3a60 2022-08-01T19:39:14.979 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:39:14.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.978+0000 7f7eb244b700 1 -- 172.21.15.17:0/37401374 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7e8c053a10 msgr2=0x7f7e8c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:14.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.978+0000 7f7eb244b700 1 --2- 172.21.15.17:0/37401374 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7e8c053a10 0x7f7e8c055ed0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f7ea000b000 tx=0x7f7ea0008040).stop 2022-08-01T19:39:14.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.978+0000 7f7eb244b700 1 -- 172.21.15.17:0/37401374 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7eac0b3a60 msgr2=0x7f7eac0addb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:14.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.978+0000 7f7eb244b700 1 --2- 172.21.15.17:0/37401374 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7eac0b3a60 0x7f7eac0addb0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7f7e9c001110 tx=0x7f7e9c007a80).stop 2022-08-01T19:39:14.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.978+0000 7f7eb244b700 1 -- 172.21.15.17:0/37401374 shutdown_connections 2022-08-01T19:39:14.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.978+0000 7f7eb244b700 1 --2- 172.21.15.17:0/37401374 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7e8c053a10 0x7f7e8c055ed0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.978+0000 7f7eb244b700 1 --2- 172.21.15.17:0/37401374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7eac0ae2a0 0x7f7eac150730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.978+0000 7f7eb244b700 1 --2- 172.21.15.17:0/37401374 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7eac0b3a60 0x7f7eac0addb0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.978+0000 7f7eb244b700 1 --2- 172.21.15.17:0/37401374 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7eac060290 0x7f7eac0af460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:14.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.978+0000 7f7eb244b700 1 -- 172.21.15.17:0/37401374 >> 172.21.15.17:0/37401374 conn(0x7f7eac05d0f0 msgr2=0x7f7eac140520 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:14.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.978+0000 7f7eb244b700 1 -- 172.21.15.17:0/37401374 shutdown_connections 2022-08-01T19:39:14.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:14.978+0000 7f7eb244b700 1 -- 172.21.15.17:0/37401374 wait complete. 2022-08-01T19:39:14.992 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T19:39:15.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.102+0000 7fbb5b003700 1 Processor -- start 2022-08-01T19:39:15.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.102+0000 7fbb5b003700 1 -- start start 2022-08-01T19:39:15.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.102+0000 7fbb5b003700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbb540b2930 0x7fbb540612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:15.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.102+0000 7fbb5b003700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb540671e0 con 0x7fbb540b7840 2022-08-01T19:39:15.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.102+0000 7fbb5b003700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb540617e0 con 0x7fbb54143400 2022-08-01T19:39:15.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.102+0000 7fbb5b003700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb54065540 con 0x7fbb540b2930 2022-08-01T19:39:15.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.102+0000 7fbb5a001700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fbb540b7840 0x7fbb54060be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60434/0 (socket says 172.21.15.17:60434) 2022-08-01T19:39:15.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.102+0000 7fbb5a001700 1 -- 172.21.15.17:0/537275808 learned_addr learned my addr 172.21.15.17:0/537275808 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:15.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.102+0000 7fbb59800700 1 --2- 172.21.15.17:0/537275808 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbb540b2930 0x7fbb540612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:15.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.102+0000 7fbb58fff700 1 -- 172.21.15.17:0/537275808 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3392340719 0 0) 0x7fbb540671e0 con 0x7fbb540b7840 2022-08-01T19:39:15.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.102+0000 7fbb58fff700 1 -- 172.21.15.17:0/537275808 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb3c003260 con 0x7fbb540b7840 2022-08-01T19:39:15.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.102+0000 7fbb58fff700 1 -- 172.21.15.17:0/537275808 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2737565693 0 0) 0x7fbb540617e0 con 0x7fbb54143400 2022-08-01T19:39:15.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.102+0000 7fbb58fff700 1 -- 172.21.15.17:0/537275808 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb540671e0 con 0x7fbb54143400 2022-08-01T19:39:15.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.102+0000 7fbb59800700 1 -- 172.21.15.17:0/537275808 >> v1:172.21.15.77:6789/0 conn(0x7fbb54143400 legacy=0x7fbb54143850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:15.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.102+0000 7fbb59800700 1 -- 172.21.15.17:0/537275808 >> v1:172.21.15.17:6789/0 conn(0x7fbb540b7840 legacy=0x7fbb54060be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:15.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.102+0000 7fbb59800700 1 -- 172.21.15.17:0/537275808 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb540b7c20 con 0x7fbb540b2930 2022-08-01T19:39:15.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.102+0000 7fbb59800700 1 --2- 172.21.15.17:0/537275808 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbb540b2930 0x7fbb540612f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fbb44002f70 tx=0x7fbb44006df0).ready entity=mon.2 client_cookie=af5c3dfce4652333 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:15.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.102+0000 7fbb58fff700 1 -- 172.21.15.17:0/537275808 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb4400d4f0 con 0x7fbb540b2930 2022-08-01T19:39:15.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.102+0000 7fbb58fff700 1 -- 172.21.15.17:0/537275808 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb44000e90 con 0x7fbb540b2930 2022-08-01T19:39:15.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.102+0000 7fbb58fff700 1 -- 172.21.15.17:0/537275808 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb4400d840 con 0x7fbb540b2930 2022-08-01T19:39:15.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.102+0000 7fbb5b003700 1 -- 172.21.15.17:0/537275808 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbb540b2930 msgr2=0x7fbb540612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:15.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.102+0000 7fbb5b003700 1 --2- 172.21.15.17:0/537275808 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbb540b2930 0x7fbb540612f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fbb44002f70 tx=0x7fbb44006df0).stop 2022-08-01T19:39:15.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.102+0000 7fbb5b003700 1 -- 172.21.15.17:0/537275808 shutdown_connections 2022-08-01T19:39:15.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.102+0000 7fbb5b003700 1 --2- 172.21.15.17:0/537275808 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbb540b2930 0x7fbb540612f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:15.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.102+0000 7fbb5b003700 1 -- 172.21.15.17:0/537275808 >> 172.21.15.17:0/537275808 conn(0x7fbb5405f840 msgr2=0x7fbb5405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:15.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.102+0000 7fbb5b003700 1 -- 172.21.15.17:0/537275808 shutdown_connections 2022-08-01T19:39:15.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.102+0000 7fbb5b003700 1 -- 172.21.15.17:0/537275808 wait complete. 2022-08-01T19:39:15.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.102+0000 7fbb5b003700 1 Processor -- start 2022-08-01T19:39:15.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.102+0000 7fbb5b003700 1 -- start start 2022-08-01T19:39:15.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.106+0000 7fbb5b003700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb540b2930 0x7fbb540b4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:15.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.106+0000 7fbb5b003700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbb540b7840 0x7fbb541523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:15.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.106+0000 7fbb5b003700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb54143400 0x7fbb541528b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:15.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.106+0000 7fbb5b003700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb540aca20 con 0x7fbb54143400 2022-08-01T19:39:15.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.106+0000 7fbb5b003700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb540b0de0 con 0x7fbb540b2930 2022-08-01T19:39:15.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.106+0000 7fbb5b003700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb540b1080 con 0x7fbb540b7840 2022-08-01T19:39:15.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.106+0000 7fbb5a802700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb54143400 0x7fbb541528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:15.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.106+0000 7fbb5a802700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb54143400 0x7fbb541528b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40018/0 (socket says 172.21.15.17:40018) 2022-08-01T19:39:15.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.106+0000 7fbb5a001700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb540b2930 0x7fbb540b4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:15.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.106+0000 7fbb5a802700 1 -- 172.21.15.17:0/549393940 learned_addr learned my addr 172.21.15.17:0/549393940 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:15.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.106+0000 7fbb59800700 1 --2- 172.21.15.17:0/549393940 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbb540b7840 0x7fbb541523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:15.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.106+0000 7fbb59800700 1 -- 172.21.15.17:0/549393940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb540b2930 msgr2=0x7fbb540b4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:15.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.106+0000 7fbb59800700 1 --2- 172.21.15.17:0/549393940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb540b2930 0x7fbb540b4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:15.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.106+0000 7fbb59800700 1 -- 172.21.15.17:0/549393940 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb54143400 msgr2=0x7fbb541528b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:15.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.106+0000 7fbb59800700 1 --2- 172.21.15.17:0/549393940 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb54143400 0x7fbb541528b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:15.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.106+0000 7fbb59800700 1 -- 172.21.15.17:0/549393940 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb4400b180 con 0x7fbb540b7840 2022-08-01T19:39:15.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.106+0000 7fbb5a802700 1 --2- 172.21.15.17:0/549393940 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb54143400 0x7fbb541528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:39:15.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.106+0000 7fbb59800700 1 --2- 172.21.15.17:0/549393940 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbb540b7840 0x7fbb541523c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fbb44002f70 tx=0x7fbb44014000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:15.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.106+0000 7fbb42ffd700 1 -- 172.21.15.17:0/549393940 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb44016070 con 0x7fbb540b7840 2022-08-01T19:39:15.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.106+0000 7fbb42ffd700 1 -- 172.21.15.17:0/549393940 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb44013850 con 0x7fbb540b7840 2022-08-01T19:39:15.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.106+0000 7fbb5b003700 1 -- 172.21.15.17:0/549393940 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb540b1320 con 0x7fbb540b7840 2022-08-01T19:39:15.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.106+0000 7fbb42ffd700 1 -- 172.21.15.17:0/549393940 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb44013bb0 con 0x7fbb540b7840 2022-08-01T19:39:15.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.106+0000 7fbb5b003700 1 -- 172.21.15.17:0/549393940 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb540ab700 con 0x7fbb540b7840 2022-08-01T19:39:15.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.110+0000 7fbb5b003700 1 -- 172.21.15.17:0/549393940 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb540ac460 con 0x7fbb540b7840 2022-08-01T19:39:15.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.110+0000 7fbb42ffd700 1 -- 172.21.15.17:0/549393940 <== mon.2 v2:172.21.15.77:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbb4400f030 con 0x7fbb540b7840 2022-08-01T19:39:15.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.110+0000 7fbb42ffd700 1 --2- 172.21.15.17:0/549393940 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbb34053a70 0x7fbb34055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:15.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.110+0000 7fbb42ffd700 1 -- 172.21.15.17:0/549393940 <== mon.2 v2:172.21.15.77:3301/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fbb4406ccc0 con 0x7fbb540b7840 2022-08-01T19:39:15.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.110+0000 7fbb5a001700 1 --2- 172.21.15.17:0/549393940 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbb34053a70 0x7fbb34055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:15.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.110+0000 7fbb5a001700 1 --2- 172.21.15.17:0/549393940 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbb34053a70 0x7fbb34055f30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7fbb5000d000 tx=0x7fbb50002810).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:15.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.118+0000 7fbb42ffd700 1 -- 172.21.15.17:0/549393940 <== mon.2 v2:172.21.15.77:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbb44073080 con 0x7fbb540b7840 2022-08-01T19:39:15.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.246+0000 7fbb5b003700 1 -- 172.21.15.17:0/549393940 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbb540b14b0 con 0x7fbb540b7840 2022-08-01T19:39:15.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.246+0000 7fbb42ffd700 1 -- 172.21.15.17:0/549393940 <== mon.2 v2:172.21.15.77:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v66) v1 ==== 100+0+10 (secure 0 0 0) 0x7fbb44023020 con 0x7fbb540b7840 2022-08-01T19:39:15.249 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:39:15.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.246+0000 7fbb5b003700 1 -- 172.21.15.17:0/549393940 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbb34053a70 msgr2=0x7fbb34055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:15.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.246+0000 7fbb5b003700 1 --2- 172.21.15.17:0/549393940 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbb34053a70 0x7fbb34055f30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7fbb5000d000 tx=0x7fbb50002810).stop 2022-08-01T19:39:15.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.246+0000 7fbb5b003700 1 -- 172.21.15.17:0/549393940 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbb540b7840 msgr2=0x7fbb541523c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:15.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.246+0000 7fbb5b003700 1 --2- 172.21.15.17:0/549393940 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbb540b7840 0x7fbb541523c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fbb44002f70 tx=0x7fbb44014000).stop 2022-08-01T19:39:15.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.250+0000 7fbb5b003700 1 -- 172.21.15.17:0/549393940 shutdown_connections 2022-08-01T19:39:15.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.250+0000 7fbb5b003700 1 --2- 172.21.15.17:0/549393940 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbb34053a70 0x7fbb34055f30 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:15.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.250+0000 7fbb5b003700 1 --2- 172.21.15.17:0/549393940 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb54143400 0x7fbb541528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:15.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.250+0000 7fbb5b003700 1 --2- 172.21.15.17:0/549393940 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbb540b7840 0x7fbb541523c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:15.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.250+0000 7fbb5b003700 1 --2- 172.21.15.17:0/549393940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb540b2930 0x7fbb540b4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:15.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.250+0000 7fbb5b003700 1 -- 172.21.15.17:0/549393940 >> 172.21.15.17:0/549393940 conn(0x7fbb5405f840 msgr2=0x7fbb5405c040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:15.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.250+0000 7fbb5b003700 1 -- 172.21.15.17:0/549393940 shutdown_connections 2022-08-01T19:39:15.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.250+0000 7fbb5b003700 1 -- 172.21.15.17:0/549393940 wait complete. 2022-08-01T19:39:15.263 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T19:39:15.265 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:39:15.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.374+0000 7f511cd9b700 1 Processor -- start 2022-08-01T19:39:15.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.374+0000 7f511cd9b700 1 -- start start 2022-08-01T19:39:15.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.374+0000 7f511cd9b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f51180b01a0 0x7f51180a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:15.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.374+0000 7f511cd9b700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f51180afd60 0x7f51180a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:15.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.374+0000 7f511cd9b700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51180a7250 con 0x7f51180b2710 2022-08-01T19:39:15.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.374+0000 7f511cd9b700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51180a7480 con 0x7f51180afd60 2022-08-01T19:39:15.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.374+0000 7f511cd9b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51180a7720 con 0x7f51180b01a0 2022-08-01T19:39:15.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.374+0000 7f51177fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f51180b2710 0x7f511814afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60450/0 (socket says 172.21.15.17:60450) 2022-08-01T19:39:15.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.374+0000 7f51177fe700 1 -- 172.21.15.17:0/583086720 learned_addr learned my addr 172.21.15.17:0/583086720 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:15.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.374+0000 7f5116ffd700 1 --2- 172.21.15.17:0/583086720 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f51180b01a0 0x7f51180a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:15.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.374+0000 7f5117fff700 1 --2- 172.21.15.17:0/583086720 >> v2:172.21.15.77:3300/0 conn(0x7f51180afd60 0x7f51180a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:15.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.374+0000 7f5116ffd700 1 -- 172.21.15.17:0/583086720 >> v2:172.21.15.77:3300/0 conn(0x7f51180afd60 msgr2=0x7f51180a8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:15.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.374+0000 7f5116ffd700 1 --2- 172.21.15.17:0/583086720 >> v2:172.21.15.77:3300/0 conn(0x7f51180afd60 0x7f51180a8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:15.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.374+0000 7f5116ffd700 1 -- 172.21.15.17:0/583086720 >> v1:172.21.15.17:6789/0 conn(0x7f51180b2710 legacy=0x7f511814afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:15.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.374+0000 7f5116ffd700 1 -- 172.21.15.17:0/583086720 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51180a79c0 con 0x7f51180b01a0 2022-08-01T19:39:15.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.374+0000 7f51167fc700 1 -- 172.21.15.17:0/583086720 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2429569490 0 0) 0x7f51180a7250 con 0x7f51180b2710 2022-08-01T19:39:15.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.374+0000 7f5116ffd700 1 --2- 172.21.15.17:0/583086720 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f51180b01a0 0x7f51180a8540 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f5108002f70 tx=0x7f5108006df0).ready entity=mon.2 client_cookie=7613c42f3f1cc598 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:15.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.374+0000 7f51167fc700 1 -- 172.21.15.17:0/583086720 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f510800d570 con 0x7f51180b01a0 2022-08-01T19:39:15.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.374+0000 7f51167fc700 1 -- 172.21.15.17:0/583086720 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5108000e90 con 0x7f51180b01a0 2022-08-01T19:39:15.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.374+0000 7f51167fc700 1 -- 172.21.15.17:0/583086720 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f510800d8c0 con 0x7f51180b01a0 2022-08-01T19:39:15.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.374+0000 7f511cd9b700 1 -- 172.21.15.17:0/583086720 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f51180b01a0 msgr2=0x7f51180a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:15.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.374+0000 7f511cd9b700 1 --2- 172.21.15.17:0/583086720 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f51180b01a0 0x7f51180a8540 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f5108002f70 tx=0x7f5108006df0).stop 2022-08-01T19:39:15.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.374+0000 7f511cd9b700 1 -- 172.21.15.17:0/583086720 shutdown_connections 2022-08-01T19:39:15.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.374+0000 7f511cd9b700 1 --2- 172.21.15.17:0/583086720 >> v2:172.21.15.77:3300/0 conn(0x7f51180afd60 0x7f51180a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:15.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.374+0000 7f511cd9b700 1 --2- 172.21.15.17:0/583086720 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f51180b01a0 0x7f51180a8540 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:15.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.374+0000 7f511cd9b700 1 -- 172.21.15.17:0/583086720 >> 172.21.15.17:0/583086720 conn(0x7f5118060310 msgr2=0x7f51180b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:15.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.374+0000 7f511cd9b700 1 -- 172.21.15.17:0/583086720 shutdown_connections 2022-08-01T19:39:15.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.374+0000 7f511cd9b700 1 -- 172.21.15.17:0/583086720 wait complete. 2022-08-01T19:39:15.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.374+0000 7f511cd9b700 1 Processor -- start 2022-08-01T19:39:15.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.374+0000 7f511cd9b700 1 -- start start 2022-08-01T19:39:15.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.374+0000 7f511cd9b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f51180afd60 0x7f51180a9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:15.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.374+0000 7f511cd9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51180b01a0 0x7f511814baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:15.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.374+0000 7f511cd9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51180b2710 0x7f511814bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:15.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.378+0000 7f511cd9b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5118156520 con 0x7f51180b2710 2022-08-01T19:39:15.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.378+0000 7f511cd9b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51181567c0 con 0x7f51180b01a0 2022-08-01T19:39:15.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.378+0000 7f511cd9b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5118156a60 con 0x7f51180afd60 2022-08-01T19:39:15.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.378+0000 7f5117fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51180b2710 0x7f511814bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:15.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.378+0000 7f51177fe700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f51180afd60 0x7f51180a9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:15.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.378+0000 7f5116ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51180b01a0 0x7f511814baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:15.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.378+0000 7f51177fe700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f51180afd60 0x7f51180a9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:36128/0 (socket says 172.21.15.17:36128) 2022-08-01T19:39:15.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.378+0000 7f5117fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51180b2710 0x7f511814bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40032/0 (socket says 172.21.15.17:40032) 2022-08-01T19:39:15.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.378+0000 7f51177fe700 1 -- 172.21.15.17:0/3842964119 learned_addr learned my addr 172.21.15.17:0/3842964119 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:15.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.378+0000 7f5117fff700 1 -- 172.21.15.17:0/3842964119 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f51180afd60 msgr2=0x7f51180a9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:15.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.378+0000 7f5117fff700 1 --2- 172.21.15.17:0/3842964119 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f51180afd60 0x7f51180a9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:15.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.378+0000 7f5117fff700 1 -- 172.21.15.17:0/3842964119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51180b01a0 msgr2=0x7f511814baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:15.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.378+0000 7f5117fff700 1 --2- 172.21.15.17:0/3842964119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51180b01a0 0x7f511814baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:15.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.378+0000 7f5117fff700 1 -- 172.21.15.17:0/3842964119 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f510800b180 con 0x7f51180b2710 2022-08-01T19:39:15.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.378+0000 7f51177fe700 1 --2- 172.21.15.17:0/3842964119 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f51180afd60 0x7f51180a9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:39:15.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.378+0000 7f5116ffd700 1 --2- 172.21.15.17:0/3842964119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51180b01a0 0x7f511814baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:39:15.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.378+0000 7f5117fff700 1 --2- 172.21.15.17:0/3842964119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51180b2710 0x7f511814bf90 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f511000c000 tx=0x7f51100091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:15.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.378+0000 7f5114ff9700 1 -- 172.21.15.17:0/3842964119 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5110012600 con 0x7f51180b2710 2022-08-01T19:39:15.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.378+0000 7f5114ff9700 1 -- 172.21.15.17:0/3842964119 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5110012760 con 0x7f51180b2710 2022-08-01T19:39:15.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.378+0000 7f511cd9b700 1 -- 172.21.15.17:0/3842964119 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5118156d00 con 0x7f51180b2710 2022-08-01T19:39:15.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.378+0000 7f5114ff9700 1 -- 172.21.15.17:0/3842964119 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5110012ac0 con 0x7f51180b2710 2022-08-01T19:39:15.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.378+0000 7f511cd9b700 1 -- 172.21.15.17:0/3842964119 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51180b5f50 con 0x7f51180b2710 2022-08-01T19:39:15.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.378+0000 7f511cd9b700 1 -- 172.21.15.17:0/3842964119 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51180574e0 con 0x7f51180b2710 2022-08-01T19:39:15.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.382+0000 7f5114ff9700 1 -- 172.21.15.17:0/3842964119 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5110022070 con 0x7f51180b2710 2022-08-01T19:39:15.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.382+0000 7f5114ff9700 1 --2- 172.21.15.17:0/3842964119 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f50f8053a10 0x7f50f8055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:15.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.382+0000 7f5114ff9700 1 -- 172.21.15.17:0/3842964119 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f511006c6f0 con 0x7f51180b2710 2022-08-01T19:39:15.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.382+0000 7f51177fe700 1 --2- 172.21.15.17:0/3842964119 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f50f8053a10 0x7f50f8055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:15.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.382+0000 7f51177fe700 1 --2- 172.21.15.17:0/3842964119 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f50f8053a10 0x7f50f8055ed0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f510c00d000 tx=0x7f510c007ce0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:15.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.386+0000 7f5114ff9700 1 -- 172.21.15.17:0/3842964119 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f51100433a0 con 0x7f51180b2710 2022-08-01T19:39:15.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.550+0000 7f511cd9b700 1 -- 172.21.15.17:0/3842964119 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5118069a70 con 0x7f51180b2710 2022-08-01T19:39:15.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.550+0000 7f5114ff9700 1 -- 172.21.15.17:0/3842964119 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+2493 (secure 0 0 0) 0x7f51100169d0 con 0x7f51180b2710 2022-08-01T19:39:15.554 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:39:15.554 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":30,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5414,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1130955771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1130955771},{"type":"v1","addr":"172.21.15.17:6835","nonce":1130955771}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":28},{"gid":5940,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3691532323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3691532323},{"type":"v1","addr":"172.21.15.17:6833","nonce":3691532323}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":28}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:39:10.932933+0000","modified":"2022-08-01T19:39:11.936834+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5949},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5949":{"gid":5949,"name":"b","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6835/2879527612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2879527612},{"type":"v1","addr":"172.21.15.77:6835","nonce":2879527612}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-08-01T19:39:15.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.554+0000 7f511cd9b700 1 -- 172.21.15.17:0/3842964119 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f50f8053a10 msgr2=0x7f50f8055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:15.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.554+0000 7f511cd9b700 1 --2- 172.21.15.17:0/3842964119 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f50f8053a10 0x7f50f8055ed0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f510c00d000 tx=0x7f510c007ce0).stop 2022-08-01T19:39:15.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.554+0000 7f511cd9b700 1 -- 172.21.15.17:0/3842964119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51180b2710 msgr2=0x7f511814bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:15.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.554+0000 7f511cd9b700 1 --2- 172.21.15.17:0/3842964119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51180b2710 0x7f511814bf90 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f511000c000 tx=0x7f51100091d0).stop 2022-08-01T19:39:15.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.554+0000 7f511cd9b700 1 -- 172.21.15.17:0/3842964119 shutdown_connections 2022-08-01T19:39:15.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.554+0000 7f511cd9b700 1 --2- 172.21.15.17:0/3842964119 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f50f8053a10 0x7f50f8055ed0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:15.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.554+0000 7f511cd9b700 1 --2- 172.21.15.17:0/3842964119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51180b2710 0x7f511814bf90 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:15.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.554+0000 7f511cd9b700 1 --2- 172.21.15.17:0/3842964119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51180b01a0 0x7f511814baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:15.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.554+0000 7f511cd9b700 1 --2- 172.21.15.17:0/3842964119 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f51180afd60 0x7f51180a9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:15.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.554+0000 7f511cd9b700 1 -- 172.21.15.17:0/3842964119 >> 172.21.15.17:0/3842964119 conn(0x7f5118060310 msgr2=0x7f5118148ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:15.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.554+0000 7f511cd9b700 1 -- 172.21.15.17:0/3842964119 shutdown_connections 2022-08-01T19:39:15.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:15.554+0000 7f511cd9b700 1 -- 172.21.15.17:0/3842964119 wait complete. 2022-08-01T19:39:15.560 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 30 2022-08-01T19:39:15.870 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:39:10.932933+0000', 'modified': '2022-08-01T19:39:11.936834+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5949}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5949': {'gid': 5949, 'name': 'b', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.77:6835/2879527612', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6834', 'nonce': 2879527612}, {'type': 'v1', 'addr': '172.21.15.77:6835', 'nonce': 2879527612}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:39:15.870 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:39:15.871 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-08-01T19:39:15.985 INFO:teuthology.orchestra.run.smithi077.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":5949,"want_state":"up:active","state":"up:active","rank_uptime":5.042757742,"mdsmap_epoch":30,"osdmap_epoch":66,"osdmap_epoch_barrier":66,"uptime":10.062836476999999} 2022-08-01T19:39:15.985 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 5949, "mdsmap_epoch": 30, "osdmap_epoch": 66, "osdmap_epoch_barrier": 66, "rank_uptime": 5.042757742, "state": "up:active", "uptime": 10.062836477, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:39:15.986 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T19:39:15.986 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-08-01T19:39:15.987 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi085.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-08-01T19:39:15.987 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:39:15.988 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:39:15.996 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:39:16.055 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:39:16.056 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:39:16.067 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:39:16.068 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:39:16.069 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:39:16.070 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:39:16.075 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T19:39:16.076 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:39:16.119 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:39:16.136 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:39:16.137 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:39:16.146 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:39:16.147 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:39:16.148 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:39:16.149 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:39:16.192 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:2022-08-01T19:39:16.195+0000 7f513d6d0080 -1 init, newargv = 0x55e3ed8adc90 newargc=9ceph-fuse[22929]: starting ceph client 2022-08-01T19:39:16.192 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr: 2022-08-01T19:39:16.202 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:ceph-fuse[22929]: starting fuse 2022-08-01T19:39:17.255 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:39:17.276 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:39:17.277 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:39:17.332 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:39:17.333 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:39:17.334 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:39:17.335 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:39:17.340 INFO:teuthology.orchestra.run.smithi085.stdout:52 2022-08-01T19:39:17.341 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T19:39:17.341 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:39:17.342 DEBUG:teuthology.orchestra.run.smithi085:> sudo python3 -c ' 2022-08-01T19:39:17.342 DEBUG:teuthology.orchestra.run.smithi085:> import glob 2022-08-01T19:39:17.343 DEBUG:teuthology.orchestra.run.smithi085:> import re 2022-08-01T19:39:17.343 DEBUG:teuthology.orchestra.run.smithi085:> import os 2022-08-01T19:39:17.344 DEBUG:teuthology.orchestra.run.smithi085:> import subprocess 2022-08-01T19:39:17.344 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:39:17.344 DEBUG:teuthology.orchestra.run.smithi085:> def find_socket(client_name): 2022-08-01T19:39:17.345 DEBUG:teuthology.orchestra.run.smithi085:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-08-01T19:39:17.345 DEBUG:teuthology.orchestra.run.smithi085:> files = glob.glob(asok_path) 2022-08-01T19:39:17.346 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:39:17.346 DEBUG:teuthology.orchestra.run.smithi085:> # Given a non-glob path, it better be there 2022-08-01T19:39:17.347 DEBUG:teuthology.orchestra.run.smithi085:> if "*" not in asok_path: 2022-08-01T19:39:17.347 DEBUG:teuthology.orchestra.run.smithi085:> assert(len(files) == 1) 2022-08-01T19:39:17.348 DEBUG:teuthology.orchestra.run.smithi085:> return files[0] 2022-08-01T19:39:17.348 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:39:17.349 DEBUG:teuthology.orchestra.run.smithi085:> for f in files: 2022-08-01T19:39:17.349 DEBUG:teuthology.orchestra.run.smithi085:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T19:39:17.349 DEBUG:teuthology.orchestra.run.smithi085:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T19:39:17.350 DEBUG:teuthology.orchestra.run.smithi085:> return f 2022-08-01T19:39:17.350 DEBUG:teuthology.orchestra.run.smithi085:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T19:39:17.351 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:39:17.351 DEBUG:teuthology.orchestra.run.smithi085:> print(find_socket("client.0")) 2022-08-01T19:39:17.352 DEBUG:teuthology.orchestra.run.smithi085:> ' 2022-08-01T19:39:17.390 INFO:teuthology.orchestra.run.smithi085.stdout:/var/run/ceph/ceph-client.0.22929.asok 2022-08-01T19:39:17.395 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22929.asok 2022-08-01T19:39:17.396 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:39:17.396 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22929.asok status 2022-08-01T19:39:17.531 INFO:teuthology.orchestra.run.smithi085.stdout:{ 2022-08-01T19:39:17.531 INFO:teuthology.orchestra.run.smithi085.stdout: "metadata": { 2022-08-01T19:39:17.531 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T19:39:17.532 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T19:39:17.532 INFO:teuthology.orchestra.run.smithi085.stdout: "entity_id": "0", 2022-08-01T19:39:17.532 INFO:teuthology.orchestra.run.smithi085.stdout: "hostname": "smithi085", 2022-08-01T19:39:17.532 INFO:teuthology.orchestra.run.smithi085.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-08-01T19:39:17.533 INFO:teuthology.orchestra.run.smithi085.stdout: "pid": "22929", 2022-08-01T19:39:17.533 INFO:teuthology.orchestra.run.smithi085.stdout: "root": "/" 2022-08-01T19:39:17.533 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:39:17.533 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_count": 0, 2022-08-01T19:39:17.534 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_pinned_count": 0, 2022-08-01T19:39:17.534 INFO:teuthology.orchestra.run.smithi085.stdout: "id": 6094, 2022-08-01T19:39:17.534 INFO:teuthology.orchestra.run.smithi085.stdout: "inst": { 2022-08-01T19:39:17.534 INFO:teuthology.orchestra.run.smithi085.stdout: "name": { 2022-08-01T19:39:17.535 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "client", 2022-08-01T19:39:17.535 INFO:teuthology.orchestra.run.smithi085.stdout: "num": 6094 2022-08-01T19:39:17.535 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:39:17.536 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2022-08-01T19:39:17.536 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2022-08-01T19:39:17.536 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2022-08-01T19:39:17.536 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 1612052965 2022-08-01T19:39:17.537 INFO:teuthology.orchestra.run.smithi085.stdout: } 2022-08-01T19:39:17.537 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:39:17.537 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2022-08-01T19:39:17.537 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2022-08-01T19:39:17.538 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2022-08-01T19:39:17.538 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 1612052965 2022-08-01T19:39:17.538 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:39:17.538 INFO:teuthology.orchestra.run.smithi085.stdout: "inst_str": "client.6094 172.21.15.85:0/1612052965", 2022-08-01T19:39:17.538 INFO:teuthology.orchestra.run.smithi085.stdout: "addr_str": "172.21.15.85:0/1612052965", 2022-08-01T19:39:17.539 INFO:teuthology.orchestra.run.smithi085.stdout: "inode_count": 1, 2022-08-01T19:39:17.539 INFO:teuthology.orchestra.run.smithi085.stdout: "mds_epoch": 30, 2022-08-01T19:39:17.539 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch": 66, 2022-08-01T19:39:17.539 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch_barrier": 0, 2022-08-01T19:39:17.540 INFO:teuthology.orchestra.run.smithi085.stdout: "blacklisted": false, 2022-08-01T19:39:17.540 INFO:teuthology.orchestra.run.smithi085.stdout: "fs_name": "cephfs" 2022-08-01T19:39:17.540 INFO:teuthology.orchestra.run.smithi085.stdout:} 2022-08-01T19:39:17.541 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:39:17.542 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:39:17.542 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:39:17.549 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:39:17.551 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:39:17.551 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:39:17.552 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:39:17.612 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:39:17.613 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:39:17.613 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:39:17.620 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:39:17.621 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:39:17.622 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:39:17.623 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-08-01T19:39:17.623 DEBUG:teuthology.orchestra.run.smithi085:> import sys 2022-08-01T19:39:17.624 DEBUG:teuthology.orchestra.run.smithi085:> import time 2022-08-01T19:39:17.624 DEBUG:teuthology.orchestra.run.smithi085:> import os 2022-08-01T19:39:17.624 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:39:17.625 DEBUG:teuthology.orchestra.run.smithi085:> n = 1000 2022-08-01T19:39:17.625 DEBUG:teuthology.orchestra.run.smithi085:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2022-08-01T19:39:17.625 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:39:17.626 DEBUG:teuthology.orchestra.run.smithi085:> if not os.path.exists(os.path.dirname(abs_path)): 2022-08-01T19:39:17.626 DEBUG:teuthology.orchestra.run.smithi085:> os.makedirs(os.path.dirname(abs_path)) 2022-08-01T19:39:17.627 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:39:17.627 DEBUG:teuthology.orchestra.run.smithi085:> for i in range(0, n): 2022-08-01T19:39:17.628 DEBUG:teuthology.orchestra.run.smithi085:> fname = "{0}_{1}".format(abs_path, i) 2022-08-01T19:39:17.628 DEBUG:teuthology.orchestra.run.smithi085:> with open(fname, '"'"'w'"'"') as f: 2022-08-01T19:39:17.628 DEBUG:teuthology.orchestra.run.smithi085:> f.write('"'"'content'"'"') 2022-08-01T19:39:17.629 DEBUG:teuthology.orchestra.run.smithi085:> if True: 2022-08-01T19:39:17.629 DEBUG:teuthology.orchestra.run.smithi085:> f.flush() 2022-08-01T19:39:17.630 DEBUG:teuthology.orchestra.run.smithi085:> os.fsync(f.fileno()) 2022-08-01T19:39:17.631 DEBUG:teuthology.orchestra.run.smithi085:> ' 2022-08-01T19:39:17.632 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:39:17.636 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:39:17.638 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:39:27.914 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:39:28.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.030+0000 7fa7ca412700 1 Processor -- start 2022-08-01T19:39:28.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.030+0000 7fa7ca412700 1 -- start start 2022-08-01T19:39:28.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.030+0000 7fa7ca412700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa7c40b28b0 0x7fa7c413e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:28.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.030+0000 7fa7ca412700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7c40aac20 con 0x7fa7c40b1180 2022-08-01T19:39:28.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.030+0000 7fa7ca412700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7c40b2c90 con 0x7fa7c4141920 2022-08-01T19:39:28.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.030+0000 7fa7ca412700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c4141db0 con 0x7fa7c40b28b0 2022-08-01T19:39:28.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.030+0000 7fa7c8c0f700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa7c40b1180 0x7fa7c4141210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60462/0 (socket says 172.21.15.17:60462) 2022-08-01T19:39:28.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.030+0000 7fa7c8c0f700 1 -- 172.21.15.17:0/2663232256 learned_addr learned my addr 172.21.15.17:0/2663232256 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:28.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.030+0000 7fa7c9410700 1 --2- 172.21.15.17:0/2663232256 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa7c40b28b0 0x7fa7c413e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:28.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.030+0000 7fa7b3fff700 1 -- 172.21.15.17:0/2663232256 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1408144918 0 0) 0x7fa7b4001110 con 0x7fa7c40b1180 2022-08-01T19:39:28.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.030+0000 7fa7b3fff700 1 -- 172.21.15.17:0/2663232256 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7ac003260 con 0x7fa7c40b1180 2022-08-01T19:39:28.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.030+0000 7fa7b3fff700 1 -- 172.21.15.17:0/2663232256 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2357628084 0 0) 0x7fa7c40b2c90 con 0x7fa7c4141920 2022-08-01T19:39:28.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.030+0000 7fa7b3fff700 1 -- 172.21.15.17:0/2663232256 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7b4001110 con 0x7fa7c4141920 2022-08-01T19:39:28.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.030+0000 7fa7c9410700 1 -- 172.21.15.17:0/2663232256 >> v1:172.21.15.77:6789/0 conn(0x7fa7c4141920 legacy=0x7fa7c41443d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:28.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.030+0000 7fa7c9410700 1 -- 172.21.15.17:0/2663232256 >> v1:172.21.15.17:6789/0 conn(0x7fa7c40b1180 legacy=0x7fa7c4141210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:28.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.030+0000 7fa7c9410700 1 -- 172.21.15.17:0/2663232256 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c40aa440 con 0x7fa7c40b28b0 2022-08-01T19:39:28.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.030+0000 7fa7c9410700 1 --2- 172.21.15.17:0/2663232256 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa7c40b28b0 0x7fa7c413e1c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fa7c0002f70 tx=0x7fa7c0006df0).ready entity=mon.2 client_cookie=595acf794c6c87e0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:28.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.030+0000 7fa7b3fff700 1 -- 172.21.15.17:0/2663232256 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7c000d5c0 con 0x7fa7c40b28b0 2022-08-01T19:39:28.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.030+0000 7fa7b3fff700 1 -- 172.21.15.17:0/2663232256 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c0000e90 con 0x7fa7c40b28b0 2022-08-01T19:39:28.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.030+0000 7fa7b3fff700 1 -- 172.21.15.17:0/2663232256 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7c000d910 con 0x7fa7c40b28b0 2022-08-01T19:39:28.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.030+0000 7fa7ca412700 1 -- 172.21.15.17:0/2663232256 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa7c40b28b0 msgr2=0x7fa7c413e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:28.040 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.030+0000 7fa7ca412700 1 --2- 172.21.15.17:0/2663232256 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa7c40b28b0 0x7fa7c413e1c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fa7c0002f70 tx=0x7fa7c0006df0).stop 2022-08-01T19:39:28.040 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.030+0000 7fa7ca412700 1 -- 172.21.15.17:0/2663232256 shutdown_connections 2022-08-01T19:39:28.040 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.030+0000 7fa7ca412700 1 --2- 172.21.15.17:0/2663232256 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa7c40b28b0 0x7fa7c413e1c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:28.040 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.030+0000 7fa7ca412700 1 -- 172.21.15.17:0/2663232256 >> 172.21.15.17:0/2663232256 conn(0x7fa7c4060310 msgr2=0x7fa7c40b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:28.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.030+0000 7fa7ca412700 1 -- 172.21.15.17:0/2663232256 shutdown_connections 2022-08-01T19:39:28.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.030+0000 7fa7ca412700 1 -- 172.21.15.17:0/2663232256 wait complete. 2022-08-01T19:39:28.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.034+0000 7fa7ca412700 1 Processor -- start 2022-08-01T19:39:28.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.034+0000 7fa7ca412700 1 -- start start 2022-08-01T19:39:28.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.034+0000 7fa7ca412700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7c40b1180 0x7fa7c413bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:28.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.034+0000 7fa7ca412700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7c40b28b0 0x7fa7c413f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:28.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.034+0000 7fa7ca412700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa7c4141920 0x7fa7c40b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:28.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.034+0000 7fa7ca412700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c4151120 con 0x7fa7c40b28b0 2022-08-01T19:39:28.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.034+0000 7fa7ca412700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c4150540 con 0x7fa7c40b1180 2022-08-01T19:39:28.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.034+0000 7fa7ca412700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c413f790 con 0x7fa7c4141920 2022-08-01T19:39:28.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.034+0000 7fa7c8c0f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7c40b28b0 0x7fa7c413f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:28.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.034+0000 7fa7c8c0f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7c40b28b0 0x7fa7c413f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40044/0 (socket says 172.21.15.17:40044) 2022-08-01T19:39:28.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.034+0000 7fa7c8c0f700 1 -- 172.21.15.17:0/2485406769 learned_addr learned my addr 172.21.15.17:0/2485406769 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:28.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.034+0000 7fa7c9410700 1 --2- 172.21.15.17:0/2485406769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7c40b1180 0x7fa7c413bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:28.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.034+0000 7fa7c9c11700 1 --2- 172.21.15.17:0/2485406769 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa7c4141920 0x7fa7c40b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:28.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.034+0000 7fa7c8c0f700 1 -- 172.21.15.17:0/2485406769 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa7c4141920 msgr2=0x7fa7c40b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:28.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.034+0000 7fa7c8c0f700 1 --2- 172.21.15.17:0/2485406769 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa7c4141920 0x7fa7c40b3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:28.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.034+0000 7fa7c8c0f700 1 -- 172.21.15.17:0/2485406769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7c40b1180 msgr2=0x7fa7c413bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:28.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.034+0000 7fa7c8c0f700 1 --2- 172.21.15.17:0/2485406769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7c40b1180 0x7fa7c413bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:28.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.034+0000 7fa7c8c0f700 1 -- 172.21.15.17:0/2485406769 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c000b180 con 0x7fa7c40b28b0 2022-08-01T19:39:28.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.034+0000 7fa7c8c0f700 1 --2- 172.21.15.17:0/2485406769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7c40b28b0 0x7fa7c413f2a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7fa7c40b2f90 tx=0x7fa7b4007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:28.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.034+0000 7fa7b27fc700 1 -- 172.21.15.17:0/2485406769 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7b40072e0 con 0x7fa7c40b28b0 2022-08-01T19:39:28.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.034+0000 7fa7b27fc700 1 -- 172.21.15.17:0/2485406769 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b4007440 con 0x7fa7c40b28b0 2022-08-01T19:39:28.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.034+0000 7fa7ca412700 1 -- 172.21.15.17:0/2485406769 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7c40b3070 con 0x7fa7c40b28b0 2022-08-01T19:39:28.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.034+0000 7fa7b27fc700 1 -- 172.21.15.17:0/2485406769 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7b4005800 con 0x7fa7c40b28b0 2022-08-01T19:39:28.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.034+0000 7fa7ca412700 1 -- 172.21.15.17:0/2485406769 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7c40b35b0 con 0x7fa7c40b28b0 2022-08-01T19:39:28.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.038+0000 7fa7ca412700 1 -- 172.21.15.17:0/2485406769 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7c40b3770 con 0x7fa7c40b28b0 2022-08-01T19:39:28.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.038+0000 7fa7b27fc700 1 -- 172.21.15.17:0/2485406769 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa7b401a060 con 0x7fa7c40b28b0 2022-08-01T19:39:28.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.038+0000 7fa7b27fc700 1 --2- 172.21.15.17:0/2485406769 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa7a4053b30 0x7fa7a4055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:28.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.038+0000 7fa7b27fc700 1 -- 172.21.15.17:0/2485406769 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fa7b406b7d0 con 0x7fa7c40b28b0 2022-08-01T19:39:28.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.038+0000 7fa7c9410700 1 --2- 172.21.15.17:0/2485406769 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa7a4053b30 0x7fa7a4055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:28.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.038+0000 7fa7c9410700 1 --2- 172.21.15.17:0/2485406769 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa7a4053b30 0x7fa7a4055ff0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7fa7c001a000 tx=0x7fa7c0014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:28.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.046+0000 7fa7b27fc700 1 -- 172.21.15.17:0/2485406769 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa7b4042480 con 0x7fa7c40b28b0 2022-08-01T19:39:28.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.202+0000 7fa7ca412700 1 -- 172.21.15.17:0/2485406769 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7c40574e0 con 0x7fa7c40b28b0 2022-08-01T19:39:28.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.202+0000 7fa7b27fc700 1 -- 172.21.15.17:0/2485406769 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+2493 (secure 0 0 0) 0x7fa7b4018020 con 0x7fa7c40b28b0 2022-08-01T19:39:28.207 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:39:28.207 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":30,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5414,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1130955771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1130955771},{"type":"v1","addr":"172.21.15.17:6835","nonce":1130955771}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":28},{"gid":5940,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3691532323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3691532323},{"type":"v1","addr":"172.21.15.17:6833","nonce":3691532323}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":28}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:39:10.932933+0000","modified":"2022-08-01T19:39:11.936834+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5949},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5949":{"gid":5949,"name":"b","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6835/2879527612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2879527612},{"type":"v1","addr":"172.21.15.77:6835","nonce":2879527612}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-08-01T19:39:28.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.206+0000 7fa7ca412700 1 -- 172.21.15.17:0/2485406769 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa7a4053b30 msgr2=0x7fa7a4055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:28.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.206+0000 7fa7ca412700 1 --2- 172.21.15.17:0/2485406769 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa7a4053b30 0x7fa7a4055ff0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7fa7c001a000 tx=0x7fa7c0014000).stop 2022-08-01T19:39:28.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.206+0000 7fa7ca412700 1 -- 172.21.15.17:0/2485406769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7c40b28b0 msgr2=0x7fa7c413f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:28.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.206+0000 7fa7ca412700 1 --2- 172.21.15.17:0/2485406769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7c40b28b0 0x7fa7c413f2a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7fa7c40b2f90 tx=0x7fa7b4007b50).stop 2022-08-01T19:39:28.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.206+0000 7fa7ca412700 1 -- 172.21.15.17:0/2485406769 shutdown_connections 2022-08-01T19:39:28.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.206+0000 7fa7ca412700 1 --2- 172.21.15.17:0/2485406769 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa7a4053b30 0x7fa7a4055ff0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:28.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.206+0000 7fa7ca412700 1 --2- 172.21.15.17:0/2485406769 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa7c4141920 0x7fa7c40b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:28.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.206+0000 7fa7ca412700 1 --2- 172.21.15.17:0/2485406769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7c40b28b0 0x7fa7c413f2a0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:28.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.206+0000 7fa7ca412700 1 --2- 172.21.15.17:0/2485406769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7c40b1180 0x7fa7c413bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:28.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.206+0000 7fa7ca412700 1 -- 172.21.15.17:0/2485406769 >> 172.21.15.17:0/2485406769 conn(0x7fa7c4060310 msgr2=0x7fa7c4139d50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:28.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.206+0000 7fa7ca412700 1 -- 172.21.15.17:0/2485406769 shutdown_connections 2022-08-01T19:39:28.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.206+0000 7fa7ca412700 1 -- 172.21.15.17:0/2485406769 wait complete. 2022-08-01T19:39:28.217 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 30 2022-08-01T19:39:28.220 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json config set mds_recall_max_caps 20 2022-08-01T19:39:28.325 INFO:teuthology.orchestra.run.smithi077.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2022-08-01T19:39:28.326 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2022-08-01T19:39:28.326 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:39:28.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.434+0000 7f7833fff700 1 Processor -- start 2022-08-01T19:39:28.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.434+0000 7f7833fff700 1 -- start start 2022-08-01T19:39:28.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.434+0000 7f7833fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f78340ad750 0x7f7834061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:28.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.434+0000 7f7833fff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f78341405d0 0x7f7834140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:28.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.434+0000 7f7833fff700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78340aee80 con 0x7f78340614b0 2022-08-01T19:39:28.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.434+0000 7f7833fff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78340b70b0 con 0x7f78341405d0 2022-08-01T19:39:28.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.434+0000 7f7833fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78340af0b0 con 0x7f78340ad750 2022-08-01T19:39:28.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.434+0000 7f7832ffd700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f78340614b0 0x7f78340b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60476/0 (socket says 172.21.15.17:60476) 2022-08-01T19:39:28.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.434+0000 7f7832ffd700 1 -- 172.21.15.17:0/3899106111 learned_addr learned my addr 172.21.15.17:0/3899106111 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:28.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.434+0000 7f782bfff700 1 --2- 172.21.15.17:0/3899106111 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f78340ad750 0x7f7834061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:28.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.434+0000 7f78337fe700 1 --2- 172.21.15.17:0/3899106111 >> v2:172.21.15.77:3300/0 conn(0x7f78341405d0 0x7f7834140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:28.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.434+0000 7f78327fc700 1 -- 172.21.15.17:0/3899106111 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3075002212 0 0) 0x7f78340aee80 con 0x7f78340614b0 2022-08-01T19:39:28.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.434+0000 7f78327fc700 1 -- 172.21.15.17:0/3899106111 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7814003260 con 0x7f78340614b0 2022-08-01T19:39:28.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.434+0000 7f78327fc700 1 -- 172.21.15.17:0/3899106111 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2677346357 0 0) 0x7f7814003260 con 0x7f78340614b0 2022-08-01T19:39:28.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.434+0000 7f78327fc700 1 -- 172.21.15.17:0/3899106111 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f78340ad750 msgr2=0x7f7834061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:28.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.434+0000 7f78327fc700 1 --2- 172.21.15.17:0/3899106111 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f78340ad750 0x7f7834061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:28.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.434+0000 7f78327fc700 1 -- 172.21.15.17:0/3899106111 >> v2:172.21.15.77:3300/0 conn(0x7f78341405d0 msgr2=0x7f7834140a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:28.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.434+0000 7f78327fc700 1 --2- 172.21.15.17:0/3899106111 >> v2:172.21.15.77:3300/0 conn(0x7f78341405d0 0x7f7834140a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:28.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.434+0000 7f78327fc700 1 -- 172.21.15.17:0/3899106111 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78340af350 con 0x7f78340614b0 2022-08-01T19:39:28.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.434+0000 7f78327fc700 1 -- 172.21.15.17:0/3899106111 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7f7824001660 con 0x7f78340614b0 2022-08-01T19:39:28.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.434+0000 7f78327fc700 1 -- 172.21.15.17:0/3899106111 >> v1:172.21.15.17:6789/0 conn(0x7f78340614b0 legacy=0x7f78340b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:28.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.434+0000 7f78327fc700 1 --2- 172.21.15.17:0/3899106111 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7814003750 0x7f7814003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:28.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.434+0000 7f78327fc700 1 --2- 172.21.15.17:0/3899106111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7814004500 0x7f7814004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:28.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.434+0000 7f78327fc700 1 --2- 172.21.15.17:0/3899106111 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f78340ad750 0x7f7814004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:28.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.434+0000 7f78327fc700 1 -- 172.21.15.17:0/3899106111 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78340b70b0 con 0x7f7814003750 2022-08-01T19:39:28.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.434+0000 7f78327fc700 1 -- 172.21.15.17:0/3899106111 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78340af0b0 con 0x7f7814004500 2022-08-01T19:39:28.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.434+0000 7f78327fc700 1 -- 172.21.15.17:0/3899106111 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f781400b750 con 0x7f78340ad750 2022-08-01T19:39:28.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.434+0000 7f7832ffd700 1 --2- 172.21.15.17:0/3899106111 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7814003750 0x7f7814003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:28.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.434+0000 7f78337fe700 1 --2- 172.21.15.17:0/3899106111 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f78340ad750 0x7f7814004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:28.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.434+0000 7f782bfff700 1 --2- 172.21.15.17:0/3899106111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7814004500 0x7f7814004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:28.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.434+0000 7f78337fe700 1 -- 172.21.15.17:0/3899106111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7814004500 msgr2=0x7f7814004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:28.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.434+0000 7f78337fe700 1 --2- 172.21.15.17:0/3899106111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7814004500 0x7f7814004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:28.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.434+0000 7f78337fe700 1 -- 172.21.15.17:0/3899106111 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7814003750 msgr2=0x7f7814003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:28.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.434+0000 7f78337fe700 1 --2- 172.21.15.17:0/3899106111 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7814003750 0x7f7814003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:28.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.434+0000 7f78337fe700 1 -- 172.21.15.17:0/3899106111 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7814003260 con 0x7f78340ad750 2022-08-01T19:39:28.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.438+0000 7f78337fe700 1 --2- 172.21.15.17:0/3899106111 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f78340ad750 0x7f7814004e70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f782c00c000 tx=0x7f782c0091d0).ready entity=mon.2 client_cookie=a40dbad7f2973fe8 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:28.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.438+0000 7f78327fc700 1 -- 172.21.15.17:0/3899106111 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f782c00e7e0 con 0x7f78340ad750 2022-08-01T19:39:28.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.438+0000 7f78327fc700 1 -- 172.21.15.17:0/3899106111 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f782c00e940 con 0x7f78340ad750 2022-08-01T19:39:28.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.438+0000 7f78327fc700 1 -- 172.21.15.17:0/3899106111 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f782c00eca0 con 0x7f78340ad750 2022-08-01T19:39:28.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.438+0000 7f7833fff700 1 -- 172.21.15.17:0/3899106111 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f78340ad750 msgr2=0x7f7814004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:28.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.438+0000 7f7833fff700 1 --2- 172.21.15.17:0/3899106111 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f78340ad750 0x7f7814004e70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f782c00c000 tx=0x7f782c0091d0).stop 2022-08-01T19:39:28.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.438+0000 7f78337fe700 1 -- 172.21.15.17:0/3899106111 reap_dead start 2022-08-01T19:39:28.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.438+0000 7f7833fff700 1 -- 172.21.15.17:0/3899106111 shutdown_connections 2022-08-01T19:39:28.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.438+0000 7f7833fff700 1 -- 172.21.15.17:0/3899106111 >> 172.21.15.17:0/3899106111 conn(0x7f7834066f70 msgr2=0x7f7834067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:28.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.438+0000 7f7833fff700 1 -- 172.21.15.17:0/3899106111 shutdown_connections 2022-08-01T19:39:28.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.438+0000 7f7833fff700 1 -- 172.21.15.17:0/3899106111 wait complete. 2022-08-01T19:39:28.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.438+0000 7f7833fff700 1 Processor -- start 2022-08-01T19:39:28.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.438+0000 7f7833fff700 1 -- start start 2022-08-01T19:39:28.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.442+0000 7f7833fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f78340ad750 0x7f78341564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:28.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.442+0000 7f7833fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7834146190 0x7f78341457b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:28.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.442+0000 7f7833fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78340b3700 0x7f78340b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:28.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.442+0000 7f7833fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78340b0af0 con 0x7f78340b3700 2022-08-01T19:39:28.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.442+0000 7f7833fff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78340b0d90 con 0x7f7834146190 2022-08-01T19:39:28.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.442+0000 7f7833fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78340b1030 con 0x7f78340ad750 2022-08-01T19:39:28.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.442+0000 7f78337fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78340b3700 0x7f78340b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:28.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.442+0000 7f78337fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78340b3700 0x7f78340b3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40066/0 (socket says 172.21.15.17:40066) 2022-08-01T19:39:28.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.442+0000 7f78337fe700 1 -- 172.21.15.17:0/2020133912 learned_addr learned my addr 172.21.15.17:0/2020133912 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:28.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.442+0000 7f782bfff700 1 --2- 172.21.15.17:0/2020133912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7834146190 0x7f78341457b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:28.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.442+0000 7f7832ffd700 1 --2- 172.21.15.17:0/2020133912 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f78340ad750 0x7f78341564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:28.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.442+0000 7f78337fe700 1 -- 172.21.15.17:0/2020133912 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f78340ad750 msgr2=0x7f78341564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:28.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.442+0000 7f78337fe700 1 --2- 172.21.15.17:0/2020133912 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f78340ad750 0x7f78341564c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:28.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.442+0000 7f78337fe700 1 -- 172.21.15.17:0/2020133912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7834146190 msgr2=0x7f78341457b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:28.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.442+0000 7f78337fe700 1 --2- 172.21.15.17:0/2020133912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7834146190 0x7f78341457b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:28.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.442+0000 7f78337fe700 1 -- 172.21.15.17:0/2020133912 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78340b14d0 con 0x7f78340b3700 2022-08-01T19:39:28.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.442+0000 7f78337fe700 1 --2- 172.21.15.17:0/2020133912 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78340b3700 0x7f78340b3b60 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f78240018c0 tx=0x7f782c01a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:28.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.442+0000 7f7830ff9700 1 -- 172.21.15.17:0/2020133912 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f782c009600 con 0x7f78340b3700 2022-08-01T19:39:28.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.442+0000 7f7830ff9700 1 -- 172.21.15.17:0/2020133912 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f782c01ec80 con 0x7f78340b3700 2022-08-01T19:39:28.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.442+0000 7f7830ff9700 1 -- 172.21.15.17:0/2020133912 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f782c01b450 con 0x7f78340b3700 2022-08-01T19:39:28.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.442+0000 7f7833fff700 1 -- 172.21.15.17:0/2020133912 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78340b4050 con 0x7f78340b3700 2022-08-01T19:39:28.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.442+0000 7f7833fff700 1 -- 172.21.15.17:0/2020133912 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78340a9e50 con 0x7f78340b3700 2022-08-01T19:39:28.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.442+0000 7f7833fff700 1 -- 172.21.15.17:0/2020133912 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78340af5f0 con 0x7f78340b3700 2022-08-01T19:39:28.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.446+0000 7f7830ff9700 1 -- 172.21.15.17:0/2020133912 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f782c01b5b0 con 0x7f78340b3700 2022-08-01T19:39:28.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.446+0000 7f7830ff9700 1 --2- 172.21.15.17:0/2020133912 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7818053a70 0x7f7818055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:28.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.446+0000 7f7830ff9700 1 -- 172.21.15.17:0/2020133912 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f782c06d910 con 0x7f78340b3700 2022-08-01T19:39:28.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.446+0000 7f7832ffd700 1 --2- 172.21.15.17:0/2020133912 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7818053a70 0x7f7818055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:28.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.446+0000 7f7832ffd700 1 --2- 172.21.15.17:0/2020133912 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7818053a70 0x7f7818055f30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f7824005000 tx=0x7f78340af4e0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:28.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.454+0000 7f7830ff9700 1 -- 172.21.15.17:0/2020133912 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f782c044760 con 0x7f78340b3700 2022-08-01T19:39:28.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.610+0000 7f7833fff700 1 -- 172.21.15.17:0/2020133912 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78340574e0 con 0x7f78340b3700 2022-08-01T19:39:28.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.614+0000 7f7830ff9700 1 -- 172.21.15.17:0/2020133912 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+2493 (secure 0 0 0) 0x7f782c0169d0 con 0x7f78340b3700 2022-08-01T19:39:28.615 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:39:28.616 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":30,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5414,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1130955771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1130955771},{"type":"v1","addr":"172.21.15.17:6835","nonce":1130955771}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":28},{"gid":5940,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3691532323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3691532323},{"type":"v1","addr":"172.21.15.17:6833","nonce":3691532323}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":28}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:39:10.932933+0000","modified":"2022-08-01T19:39:11.936834+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5949},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5949":{"gid":5949,"name":"b","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6835/2879527612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2879527612},{"type":"v1","addr":"172.21.15.77:6835","nonce":2879527612}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-08-01T19:39:28.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.614+0000 7f7833fff700 1 -- 172.21.15.17:0/2020133912 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7818053a70 msgr2=0x7f7818055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:28.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.614+0000 7f7833fff700 1 --2- 172.21.15.17:0/2020133912 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7818053a70 0x7f7818055f30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f7824005000 tx=0x7f78340af4e0).stop 2022-08-01T19:39:28.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.614+0000 7f7833fff700 1 -- 172.21.15.17:0/2020133912 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78340b3700 msgr2=0x7f78340b3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:28.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.614+0000 7f7833fff700 1 --2- 172.21.15.17:0/2020133912 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78340b3700 0x7f78340b3b60 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f78240018c0 tx=0x7f782c01a000).stop 2022-08-01T19:39:28.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.614+0000 7f7833fff700 1 -- 172.21.15.17:0/2020133912 shutdown_connections 2022-08-01T19:39:28.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.614+0000 7f7833fff700 1 --2- 172.21.15.17:0/2020133912 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7818053a70 0x7f7818055f30 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:28.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.614+0000 7f7833fff700 1 --2- 172.21.15.17:0/2020133912 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78340b3700 0x7f78340b3b60 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:28.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.614+0000 7f7833fff700 1 --2- 172.21.15.17:0/2020133912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7834146190 0x7f78341457b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:28.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.614+0000 7f7833fff700 1 --2- 172.21.15.17:0/2020133912 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f78340ad750 0x7f78341564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:28.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.614+0000 7f7833fff700 1 -- 172.21.15.17:0/2020133912 >> 172.21.15.17:0/2020133912 conn(0x7f7834066f70 msgr2=0x7f783405be30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:28.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.614+0000 7f7833fff700 1 -- 172.21.15.17:0/2020133912 shutdown_connections 2022-08-01T19:39:28.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.614+0000 7f7833fff700 1 -- 172.21.15.17:0/2020133912 wait complete. 2022-08-01T19:39:28.626 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 30 2022-08-01T19:39:28.629 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json config set mds_recall_max_decay_threshold 400 2022-08-01T19:39:28.741 INFO:teuthology.orchestra.run.smithi077.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2022-08-01T19:39:28.741 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2022-08-01T19:39:28.743 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:39:28.858 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.858+0000 7ff95359e700 1 Processor -- start 2022-08-01T19:39:28.858 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.858+0000 7ff95359e700 1 -- start start 2022-08-01T19:39:28.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.858+0000 7ff95359e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff9540b2710 0x7ff9541401e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:28.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.858+0000 7ff95359e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ff9540b0270 0x7ff9541406d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:28.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.858+0000 7ff95359e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff954067a10 con 0x7ff9540ac880 2022-08-01T19:39:28.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.858+0000 7ff95359e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9540ad740 con 0x7ff9540b0270 2022-08-01T19:39:28.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.858+0000 7ff95359e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff954067cb0 con 0x7ff9540b2710 2022-08-01T19:39:28.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.858+0000 7ff949d9b700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ff9540b0270 0x7ff9541406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:28.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.858+0000 7ff95259c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff9540b2710 0x7ff9541401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:28.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.858+0000 7ff949d9b700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ff9540b0270 0x7ff9541406d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:37992/0 (socket says 172.21.15.17:37992) 2022-08-01T19:39:28.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.858+0000 7ff949d9b700 1 -- 172.21.15.17:0/4130261817 learned_addr learned my addr 172.21.15.17:0/4130261817 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:28.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.858+0000 7ff949d9b700 1 -- 172.21.15.17:0/4130261817 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff9540b2710 msgr2=0x7ff9541401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:28.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.858+0000 7ff949d9b700 1 --2- 172.21.15.17:0/4130261817 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff9540b2710 0x7ff9541401e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:28.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.858+0000 7ff949d9b700 1 -- 172.21.15.17:0/4130261817 >> v1:172.21.15.77:6789/0 conn(0x7ff9540ac880 legacy=0x7ff954143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:28.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.858+0000 7ff949d9b700 1 -- 172.21.15.17:0/4130261817 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff954147160 con 0x7ff9540b0270 2022-08-01T19:39:28.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.858+0000 7ff949d9b700 1 --2- 172.21.15.17:0/4130261817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9540b0270 0x7ff9541406d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7ff93c002f70 tx=0x7ff93c006eb0).ready entity=mon.1 client_cookie=7cc1843096971161 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:28.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.858+0000 7ff951d9b700 1 -- 172.21.15.17:0/4130261817 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff93c00d570 con 0x7ff9540b0270 2022-08-01T19:39:28.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.858+0000 7ff951d9b700 1 -- 172.21.15.17:0/4130261817 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff93c000e90 con 0x7ff9540b0270 2022-08-01T19:39:28.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.858+0000 7ff951d9b700 1 -- 172.21.15.17:0/4130261817 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff93c00d900 con 0x7ff9540b0270 2022-08-01T19:39:28.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.858+0000 7ff95359e700 1 -- 172.21.15.17:0/4130261817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9540b0270 msgr2=0x7ff9541406d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:28.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.858+0000 7ff95359e700 1 --2- 172.21.15.17:0/4130261817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9540b0270 0x7ff9541406d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7ff93c002f70 tx=0x7ff93c006eb0).stop 2022-08-01T19:39:28.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.858+0000 7ff95359e700 1 -- 172.21.15.17:0/4130261817 shutdown_connections 2022-08-01T19:39:28.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.858+0000 7ff95359e700 1 --2- 172.21.15.17:0/4130261817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9540b0270 0x7ff9541406d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:28.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.858+0000 7ff95359e700 1 --2- 172.21.15.17:0/4130261817 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff9540b2710 0x7ff9541401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:28.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.858+0000 7ff95359e700 1 -- 172.21.15.17:0/4130261817 >> 172.21.15.17:0/4130261817 conn(0x7ff954060310 msgr2=0x7ff9540b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:28.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.858+0000 7ff95359e700 1 -- 172.21.15.17:0/4130261817 shutdown_connections 2022-08-01T19:39:28.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.858+0000 7ff95359e700 1 -- 172.21.15.17:0/4130261817 wait complete. 2022-08-01T19:39:28.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.862+0000 7ff95359e700 1 Processor -- start 2022-08-01T19:39:28.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.862+0000 7ff95359e700 1 -- start start 2022-08-01T19:39:28.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.862+0000 7ff95359e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff9540ac880 0x7ff9541461f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:28.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.862+0000 7ff95359e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9540b0270 0x7ff9540a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:28.870 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.862+0000 7ff95359e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9540b2710 0x7ff9540a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:28.870 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.862+0000 7ff95359e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff954152570 con 0x7ff9540b2710 2022-08-01T19:39:28.870 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.862+0000 7ff95359e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95409ea40 con 0x7ff9540b0270 2022-08-01T19:39:28.870 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.862+0000 7ff95359e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff954142e30 con 0x7ff9540ac880 2022-08-01T19:39:28.871 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.862+0000 7ff952d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9540b2710 0x7ff9540a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:28.871 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.862+0000 7ff952d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9540b2710 0x7ff9540a1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40080/0 (socket says 172.21.15.17:40080) 2022-08-01T19:39:28.871 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.862+0000 7ff952d9d700 1 -- 172.21.15.17:0/1492267296 learned_addr learned my addr 172.21.15.17:0/1492267296 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:28.871 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.862+0000 7ff949d9b700 1 --2- 172.21.15.17:0/1492267296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9540b0270 0x7ff9540a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:28.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.862+0000 7ff95259c700 1 --2- 172.21.15.17:0/1492267296 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff9540ac880 0x7ff9541461f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:28.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.862+0000 7ff952d9d700 1 -- 172.21.15.17:0/1492267296 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff9540ac880 msgr2=0x7ff9541461f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:28.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.862+0000 7ff952d9d700 1 --2- 172.21.15.17:0/1492267296 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff9540ac880 0x7ff9541461f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:28.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.862+0000 7ff952d9d700 1 -- 172.21.15.17:0/1492267296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9540b0270 msgr2=0x7ff9540a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:28.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.862+0000 7ff952d9d700 1 --2- 172.21.15.17:0/1492267296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9540b0270 0x7ff9540a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:28.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.862+0000 7ff952d9d700 1 -- 172.21.15.17:0/1492267296 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff93c00b1e0 con 0x7ff9540b2710 2022-08-01T19:39:28.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.862+0000 7ff952d9d700 1 --2- 172.21.15.17:0/1492267296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9540b2710 0x7ff9540a1040 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7ff954061c10 tx=0x7ff94c007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:28.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.862+0000 7ff94bfff700 1 -- 172.21.15.17:0/1492267296 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff94c00f470 con 0x7ff9540b2710 2022-08-01T19:39:28.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.862+0000 7ff94bfff700 1 -- 172.21.15.17:0/1492267296 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff94c00f5d0 con 0x7ff9540b2710 2022-08-01T19:39:28.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.862+0000 7ff95359e700 1 -- 172.21.15.17:0/1492267296 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff95409ec70 con 0x7ff9540b2710 2022-08-01T19:39:28.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.862+0000 7ff94bfff700 1 -- 172.21.15.17:0/1492267296 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff94c00f930 con 0x7ff9540b2710 2022-08-01T19:39:28.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.862+0000 7ff95359e700 1 -- 172.21.15.17:0/1492267296 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9541466e0 con 0x7ff9540b2710 2022-08-01T19:39:28.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.866+0000 7ff95359e700 1 -- 172.21.15.17:0/1492267296 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff95409f0b0 con 0x7ff9540b2710 2022-08-01T19:39:28.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.866+0000 7ff94bfff700 1 -- 172.21.15.17:0/1492267296 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff94c019030 con 0x7ff9540b2710 2022-08-01T19:39:28.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.866+0000 7ff94bfff700 1 --2- 172.21.15.17:0/1492267296 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff938053950 0x7ff938055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:28.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.866+0000 7ff94bfff700 1 -- 172.21.15.17:0/1492267296 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff94c06b820 con 0x7ff9540b2710 2022-08-01T19:39:28.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.866+0000 7ff95259c700 1 --2- 172.21.15.17:0/1492267296 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff938053950 0x7ff938055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:28.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.866+0000 7ff95259c700 1 --2- 172.21.15.17:0/1492267296 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff938053950 0x7ff938055e10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7ff94400b000 tx=0x7ff944006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:28.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:28.878+0000 7ff94bfff700 1 -- 172.21.15.17:0/1492267296 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff94c0425e0 con 0x7ff9540b2710 2022-08-01T19:39:29.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.034+0000 7ff95359e700 1 -- 172.21.15.17:0/1492267296 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9540574e0 con 0x7ff9540b2710 2022-08-01T19:39:29.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.034+0000 7ff94bfff700 1 -- 172.21.15.17:0/1492267296 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+2493 (secure 0 0 0) 0x7ff94c045c00 con 0x7ff9540b2710 2022-08-01T19:39:29.039 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:39:29.039 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":30,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5414,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1130955771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1130955771},{"type":"v1","addr":"172.21.15.17:6835","nonce":1130955771}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":28},{"gid":5940,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3691532323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3691532323},{"type":"v1","addr":"172.21.15.17:6833","nonce":3691532323}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":28}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:39:10.932933+0000","modified":"2022-08-01T19:39:11.936834+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5949},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5949":{"gid":5949,"name":"b","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6835/2879527612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2879527612},{"type":"v1","addr":"172.21.15.77:6835","nonce":2879527612}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-08-01T19:39:29.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.038+0000 7ff95359e700 1 -- 172.21.15.17:0/1492267296 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff938053950 msgr2=0x7ff938055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:29.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.038+0000 7ff95359e700 1 --2- 172.21.15.17:0/1492267296 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff938053950 0x7ff938055e10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7ff94400b000 tx=0x7ff944006df0).stop 2022-08-01T19:39:29.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.038+0000 7ff95359e700 1 -- 172.21.15.17:0/1492267296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9540b2710 msgr2=0x7ff9540a1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:29.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.038+0000 7ff95359e700 1 --2- 172.21.15.17:0/1492267296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9540b2710 0x7ff9540a1040 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7ff954061c10 tx=0x7ff94c007f10).stop 2022-08-01T19:39:29.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.038+0000 7ff95359e700 1 -- 172.21.15.17:0/1492267296 shutdown_connections 2022-08-01T19:39:29.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.038+0000 7ff95359e700 1 --2- 172.21.15.17:0/1492267296 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff938053950 0x7ff938055e10 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:29.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.038+0000 7ff95359e700 1 --2- 172.21.15.17:0/1492267296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9540b2710 0x7ff9540a1040 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:29.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.038+0000 7ff95359e700 1 --2- 172.21.15.17:0/1492267296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9540b0270 0x7ff9540a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:29.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.038+0000 7ff95359e700 1 --2- 172.21.15.17:0/1492267296 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff9540ac880 0x7ff9541461f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:29.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.038+0000 7ff95359e700 1 -- 172.21.15.17:0/1492267296 >> 172.21.15.17:0/1492267296 conn(0x7ff954060310 msgr2=0x7ff9541446d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:29.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.038+0000 7ff95359e700 1 -- 172.21.15.17:0/1492267296 shutdown_connections 2022-08-01T19:39:29.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.038+0000 7ff95359e700 1 -- 172.21.15.17:0/1492267296 wait complete. 2022-08-01T19:39:29.045 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 30 2022-08-01T19:39:29.052 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b cache drop 10 2022-08-01T19:39:29.162 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.158+0000 7fd0588cd700 1 Processor -- start 2022-08-01T19:39:29.162 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.158+0000 7fd0588cd700 1 -- start start 2022-08-01T19:39:29.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.162+0000 7fd0588cd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd0540b1b10 0x7fd0540b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:29.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.162+0000 7fd0588cd700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd054140220 con 0x7fd054060d00 2022-08-01T19:39:29.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.162+0000 7fd0588cd700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0541404c0 con 0x7fd054061860 2022-08-01T19:39:29.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.162+0000 7fd0588cd700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd054061150 con 0x7fd0540b1b10 2022-08-01T19:39:29.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.162+0000 7fd052d9d700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd054060d00 0x7fd05405f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60514/0 (socket says 172.21.15.17:60514) 2022-08-01T19:39:29.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.162+0000 7fd052d9d700 1 -- 172.21.15.17:0/36098577 learned_addr learned my addr 172.21.15.17:0/36098577 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:29.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.162+0000 7fd05259c700 1 --2- 172.21.15.17:0/36098577 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd0540b1b10 0x7fd0540b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:29.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.162+0000 7fd05159a700 1 -- 172.21.15.17:0/36098577 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2667532172 0 0) 0x7fd054140220 con 0x7fd054060d00 2022-08-01T19:39:29.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.162+0000 7fd05159a700 1 -- 172.21.15.17:0/36098577 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd038003260 con 0x7fd054060d00 2022-08-01T19:39:29.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.162+0000 7fd05259c700 1 -- 172.21.15.17:0/36098577 >> v1:172.21.15.77:6789/0 conn(0x7fd054061860 legacy=0x7fd054139e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:29.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.162+0000 7fd05259c700 1 -- 172.21.15.17:0/36098577 >> v1:172.21.15.17:6789/0 conn(0x7fd054060d00 legacy=0x7fd05405f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:29.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.162+0000 7fd05159a700 1 -- 172.21.15.17:0/36098577 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4186900937 0 0) 0x7fd038003260 con 0x7fd054060d00 2022-08-01T19:39:29.169 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.162+0000 7fd05259c700 1 -- 172.21.15.17:0/36098577 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd054140760 con 0x7fd0540b1b10 2022-08-01T19:39:29.169 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.162+0000 7fd05259c700 1 --2- 172.21.15.17:0/36098577 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd0540b1b10 0x7fd0540b1ef0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fd04c001b80 tx=0x7fd044006df0).ready entity=mon.2 client_cookie=654f828df8cc1736 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:29.169 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.162+0000 7fd05159a700 1 -- 172.21.15.17:0/36098577 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd04c0019e0 con 0x7fd0540b1b10 2022-08-01T19:39:29.169 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.162+0000 7fd05159a700 1 -- 172.21.15.17:0/36098577 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd044000e90 con 0x7fd0540b1b10 2022-08-01T19:39:29.170 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.162+0000 7fd05159a700 1 -- 172.21.15.17:0/36098577 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd04400d4a0 con 0x7fd0540b1b10 2022-08-01T19:39:29.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.162+0000 7fd0588cd700 1 -- 172.21.15.17:0/36098577 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd0540b1b10 msgr2=0x7fd0540b1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:29.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.162+0000 7fd0588cd700 1 --2- 172.21.15.17:0/36098577 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd0540b1b10 0x7fd0540b1ef0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fd04c001b80 tx=0x7fd044006df0).stop 2022-08-01T19:39:29.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.162+0000 7fd0588cd700 1 -- 172.21.15.17:0/36098577 shutdown_connections 2022-08-01T19:39:29.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.162+0000 7fd0588cd700 1 --2- 172.21.15.17:0/36098577 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd0540b1b10 0x7fd0540b1ef0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:29.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.162+0000 7fd0588cd700 1 -- 172.21.15.17:0/36098577 >> 172.21.15.17:0/36098577 conn(0x7fd0540b2b00 msgr2=0x7fd0540b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:29.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.166+0000 7fd0588cd700 1 -- 172.21.15.17:0/36098577 shutdown_connections 2022-08-01T19:39:29.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.166+0000 7fd0588cd700 1 -- 172.21.15.17:0/36098577 wait complete. 2022-08-01T19:39:29.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.166+0000 7fd0588cd700 1 Processor -- start 2022-08-01T19:39:29.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.166+0000 7fd0588cd700 1 -- start start 2022-08-01T19:39:29.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.166+0000 7fd0588cd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd054060d00 0x7fd0540ad720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:29.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.166+0000 7fd0588cd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd054061860 0x7fd05414f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:29.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.166+0000 7fd0588cd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd0540b1b10 0x7fd0541525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:29.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.166+0000 7fd0588cd700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0540658e0 con 0x7fd054060d00 2022-08-01T19:39:29.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.166+0000 7fd0588cd700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd054066260 con 0x7fd054061860 2022-08-01T19:39:29.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.166+0000 7fd0588cd700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd054066500 con 0x7fd0540b1b10 2022-08-01T19:39:29.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.166+0000 7fd05259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd054060d00 0x7fd0540ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:29.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.166+0000 7fd05259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd054060d00 0x7fd0540ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40090/0 (socket says 172.21.15.17:40090) 2022-08-01T19:39:29.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.166+0000 7fd05259c700 1 -- 172.21.15.17:0/224238787 learned_addr learned my addr 172.21.15.17:0/224238787 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:29.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.166+0000 7fd051d9b700 1 --2- 172.21.15.17:0/224238787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd054061860 0x7fd05414f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:29.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.166+0000 7fd052d9d700 1 --2- 172.21.15.17:0/224238787 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd0540b1b10 0x7fd0541525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:29.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.166+0000 7fd05259c700 1 -- 172.21.15.17:0/224238787 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd0540b1b10 msgr2=0x7fd0541525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:29.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.166+0000 7fd05259c700 1 --2- 172.21.15.17:0/224238787 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd0540b1b10 0x7fd0541525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:29.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.166+0000 7fd05259c700 1 -- 172.21.15.17:0/224238787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd054061860 msgr2=0x7fd05414f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:29.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.166+0000 7fd05259c700 1 --2- 172.21.15.17:0/224238787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd054061860 0x7fd05414f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:29.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.166+0000 7fd05259c700 1 -- 172.21.15.17:0/224238787 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04400b180 con 0x7fd054060d00 2022-08-01T19:39:29.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.166+0000 7fd05259c700 1 --2- 172.21.15.17:0/224238787 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd054060d00 0x7fd0540ad720 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7fd04c001b80 tx=0x7fd044014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:29.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.166+0000 7fd0437fe700 1 -- 172.21.15.17:0/224238787 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0440139c0 con 0x7fd054060d00 2022-08-01T19:39:29.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.166+0000 7fd0437fe700 1 -- 172.21.15.17:0/224238787 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd044013b20 con 0x7fd054060d00 2022-08-01T19:39:29.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.166+0000 7fd0588cd700 1 -- 172.21.15.17:0/224238787 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd054150210 con 0x7fd054060d00 2022-08-01T19:39:29.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.170+0000 7fd0437fe700 1 -- 172.21.15.17:0/224238787 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd044013e80 con 0x7fd054060d00 2022-08-01T19:39:29.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.170+0000 7fd0588cd700 1 -- 172.21.15.17:0/224238787 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd054064c80 con 0x7fd054060d00 2022-08-01T19:39:29.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.170+0000 7fd0437fe700 1 -- 172.21.15.17:0/224238787 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd044025030 con 0x7fd054060d00 2022-08-01T19:39:29.182 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.170+0000 7fd0437fe700 1 --2- 172.21.15.17:0/224238787 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd030053b00 0x7fd030055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:29.182 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.170+0000 7fd0437fe700 1 -- 172.21.15.17:0/224238787 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd04406cd00 con 0x7fd054060d00 2022-08-01T19:39:29.182 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.170+0000 7fd051d9b700 1 --2- 172.21.15.17:0/224238787 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd030053b00 0x7fd030055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:29.182 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.170+0000 7fd051d9b700 1 --2- 172.21.15.17:0/224238787 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd030053b00 0x7fd030055fc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7fd04800d000 tx=0x7fd048007b90).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:29.190 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.174+0000 7fd062b31740 1 Processor -- start 2022-08-01T19:39:29.190 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.174+0000 7fd062b31740 1 -- start start 2022-08-01T19:39:29.190 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.174+0000 7fd062b31740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1d9fa50 0x1da1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:29.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.174+0000 7fd062b31740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1d9edf0 0x1da1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:29.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.174+0000 7fd062b31740 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1da1710 0x1dd3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:29.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.174+0000 7fd062b31740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x1cfe3c0 con 0x1d9fa50 2022-08-01T19:39:29.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.174+0000 7fd062b31740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d12050 con 0x1d9edf0 2022-08-01T19:39:29.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.174+0000 7fd062b31740 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x1cf2290 con 0x1da1710 2022-08-01T19:39:29.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.174+0000 7fd05259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1d9fa50 0x1da1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:29.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.174+0000 7fd05259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1d9fa50 0x1da1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40098/0 (socket says 172.21.15.17:40098) 2022-08-01T19:39:29.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.174+0000 7fd05259c700 1 -- 172.21.15.17:0/1280735426 learned_addr learned my addr 172.21.15.17:0/1280735426 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:29.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.174+0000 7fd051d9b700 1 --2- 172.21.15.17:0/1280735426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1d9edf0 0x1da1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:29.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.174+0000 7fd052d9d700 1 --2- 172.21.15.17:0/1280735426 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1da1710 0x1dd3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:29.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.174+0000 7fd05259c700 1 -- 172.21.15.17:0/1280735426 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1da1710 msgr2=0x1dd3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:29.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.174+0000 7fd05259c700 1 --2- 172.21.15.17:0/1280735426 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1da1710 0x1dd3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:29.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.174+0000 7fd05259c700 1 -- 172.21.15.17:0/1280735426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1d9edf0 msgr2=0x1da1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:29.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.174+0000 7fd05259c700 1 --2- 172.21.15.17:0/1280735426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1d9edf0 0x1da1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:29.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.174+0000 7fd05259c700 1 -- 172.21.15.17:0/1280735426 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1d49e90 con 0x1d9fa50 2022-08-01T19:39:29.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.174+0000 7fd052d9d700 1 --2- 172.21.15.17:0/1280735426 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1da1710 0x1dd3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:39:29.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.174+0000 7fd05259c700 1 --2- 172.21.15.17:0/1280735426 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1d9fa50 0x1da1fc0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7fd04406b240 tx=0x7fd044010000).ready entity=mon.0 client_cookie=dbc3195e5c6422ba server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:29.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.174+0000 7fd0588cd700 1 -- 172.21.15.17:0/1280735426 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd044047610 con 0x1d9fa50 2022-08-01T19:39:29.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.174+0000 7fd0588cd700 1 -- 172.21.15.17:0/1280735426 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd044045a10 con 0x1d9fa50 2022-08-01T19:39:29.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.174+0000 7fd0588cd700 1 -- 172.21.15.17:0/1280735426 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd044043650 con 0x1d9fa50 2022-08-01T19:39:29.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.174+0000 7fd062b31740 1 -- 172.21.15.17:0/1280735426 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1d9fa50 msgr2=0x1da1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:29.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.174+0000 7fd062b31740 1 --2- 172.21.15.17:0/1280735426 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1d9fa50 0x1da1fc0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7fd04406b240 tx=0x7fd044010000).stop 2022-08-01T19:39:29.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.174+0000 7fd062b31740 1 -- 172.21.15.17:0/1280735426 shutdown_connections 2022-08-01T19:39:29.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.174+0000 7fd062b31740 1 --2- 172.21.15.17:0/1280735426 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1da1710 0x1dd3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:29.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.174+0000 7fd062b31740 1 --2- 172.21.15.17:0/1280735426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1d9edf0 0x1da1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:29.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.174+0000 7fd062b31740 1 --2- 172.21.15.17:0/1280735426 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1d9fa50 0x1da1fc0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:29.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.174+0000 7fd062b31740 1 -- 172.21.15.17:0/1280735426 >> 172.21.15.17:0/1280735426 conn(0x1d9e450 msgr2=0x1d9dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:29.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.174+0000 7fd062b31740 1 -- 172.21.15.17:0/1280735426 shutdown_connections 2022-08-01T19:39:29.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.174+0000 7fd062b31740 1 -- 172.21.15.17:0/1280735426 wait complete. 2022-08-01T19:39:29.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.174+0000 7fd062b31740 1 -- start start 2022-08-01T19:39:29.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd062b31740 1 Processor -- start 2022-08-01T19:39:29.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd062b31740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1d9edf0 0x1dd1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:29.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd062b31740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1d9fa50 0x1de9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:29.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd062b31740 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1da1710 0x1de56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:29.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd062b31740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x1dccda0 con 0x1d9edf0 2022-08-01T19:39:29.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd062b31740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x1de8c70 con 0x1d9fa50 2022-08-01T19:39:29.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd062b31740 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x1dd0550 con 0x1da1710 2022-08-01T19:39:29.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd05259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1d9edf0 0x1dd1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:29.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd062b31740 10 client.0 ll_register_callbacks cb 0x1d59470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-08-01T19:39:29.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd05259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1d9edf0 0x1dd1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40104/0 (socket says 172.21.15.17:40104) 2022-08-01T19:39:29.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd05259c700 1 -- 172.21.15.17:0/1402886462 learned_addr learned my addr 172.21.15.17:0/1402886462 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:29.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd051d9b700 1 --2- 172.21.15.17:0/1402886462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1d9fa50 0x1de9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:29.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd052d9d700 1 --2- 172.21.15.17:0/1402886462 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1da1710 0x1de56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:29.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd05259c700 1 -- 172.21.15.17:0/1402886462 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1da1710 msgr2=0x1de56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:29.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd05259c700 1 --2- 172.21.15.17:0/1402886462 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1da1710 0x1de56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:29.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd05259c700 1 -- 172.21.15.17:0/1402886462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1d9fa50 msgr2=0x1de9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:29.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd05259c700 1 --2- 172.21.15.17:0/1402886462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1d9fa50 0x1de9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:29.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd05259c700 1 -- 172.21.15.17:0/1402886462 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd054064c80 con 0x1d9edf0 2022-08-01T19:39:29.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd051d9b700 1 --2- 172.21.15.17:0/1402886462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1d9fa50 0x1de9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:39:29.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd05259c700 1 --2- 172.21.15.17:0/1402886462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1d9edf0 0x1dd1e00 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7fd04406b240 tx=0x7fd04401f000).ready entity=mon.0 client_cookie=7e7e030b0bfd8dd2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:29.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd02d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2022-08-01T19:39:29.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd02d7fa700 1 -- 172.21.15.17:0/1402886462 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd044006440 con 0x1d9edf0 2022-08-01T19:39:29.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd02d7fa700 1 -- 172.21.15.17:0/1402886462 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd044047610 con 0x1d9edf0 2022-08-01T19:39:29.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd062b31740 1 -- 172.21.15.17:0/1402886462 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1d08000 con 0x1d9edf0 2022-08-01T19:39:29.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd02d7fa700 1 -- 172.21.15.17:0/1402886462 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd04400bed0 con 0x1d9edf0 2022-08-01T19:39:29.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd062b31740 10 client.6141 fetch_fsmap learned FSMap version 30 2022-08-01T19:39:29.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd062b31740 1 -- 172.21.15.17:0/1402886462 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x1ce49f0 con 0x1d9edf0 2022-08-01T19:39:29.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd02d7fa700 1 -- 172.21.15.17:0/1402886462 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1d08000 con 0x1d9edf0 2022-08-01T19:39:29.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd02d7fa700 1 -- 172.21.15.17:0/1402886462 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 30) v1 ==== 1434+0+0 (secure 0 0 0) 0x7fd0440056f0 con 0x1d9edf0 2022-08-01T19:39:29.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd062b31740 10 client.6141 fetch_fsmap finished waiting for FSMap version 30 2022-08-01T19:39:29.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd062b31740 10 client.6141 resolve_mds: resolved ID 'b' to GID 5949 2022-08-01T19:39:29.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd062b31740 20 client.6141 populate_metadata read hostname 'smithi017' 2022-08-01T19:39:29.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd062b31740 1 --2- 172.21.15.17:0/1402886462 >> [v2:172.21.15.77:6834/2879527612,v1:172.21.15.77:6835/2879527612] conn(0x1dd8d70 0x1dd08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:29.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd062b31740 4 client.6141 mds_command: new command op to 5949 tid=0[{"prefix": "get_command_descriptions"}] 2022-08-01T19:39:29.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd062b31740 1 -- 172.21.15.17:0/1402886462 --> [v2:172.21.15.77:6834/2879527612,v1:172.21.15.77:6835/2879527612] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1d00080 con 0x1dd8d70 2022-08-01T19:39:29.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd051d9b700 1 --2- 172.21.15.17:0/1402886462 >> [v2:172.21.15.77:6834/2879527612,v1:172.21.15.77:6835/2879527612] conn(0x1dd8d70 0x1dd08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:29.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd02d7fa700 10 client.6141 ms_handle_connect on v2:172.21.15.77:6834/2879527612 2022-08-01T19:39:29.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.178+0000 7fd051d9b700 1 --2- 172.21.15.17:0/1402886462 >> [v2:172.21.15.77:6834/2879527612,v1:172.21.15.77:6835/2879527612] conn(0x1dd8d70 0x1dd08e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=fe1c775785501c95 server_cookie=b05bcdaa74ca74e0 in_seq=0 out_seq=0 2022-08-01T19:39:29.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.182+0000 7fd02d7fa700 1 -- 172.21.15.17:0/1402886462 <== mds.0 v2:172.21.15.77:6834/2879527612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1d00080 con 0x1dd8d70 2022-08-01T19:39:29.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.182+0000 7fd02d7fa700 10 client.6141 handle_command_reply: tid=0 2022-08-01T19:39:29.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.182+0000 7fd062b31740 1 client.6141 shutdown 2022-08-01T19:39:29.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.182+0000 7fd062b31740 1 -- 172.21.15.17:0/1402886462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1d9edf0 msgr2=0x1dd1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:29.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.182+0000 7fd062b31740 1 --2- 172.21.15.17:0/1402886462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1d9edf0 0x1dd1e00 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7fd04406b240 tx=0x7fd04401f000).stop 2022-08-01T19:39:29.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.182+0000 7fd062b31740 1 -- 172.21.15.17:0/1402886462 shutdown_connections 2022-08-01T19:39:29.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.182+0000 7fd062b31740 1 --2- 172.21.15.17:0/1402886462 >> [v2:172.21.15.77:6834/2879527612,v1:172.21.15.77:6835/2879527612] conn(0x1dd8d70 0x1dd08e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:29.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.182+0000 7fd062b31740 1 --2- 172.21.15.17:0/1402886462 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1da1710 0x1de56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:29.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.182+0000 7fd062b31740 1 --2- 172.21.15.17:0/1402886462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1d9fa50 0x1de9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:29.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.182+0000 7fd062b31740 1 --2- 172.21.15.17:0/1402886462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1d9edf0 0x1dd1e00 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:29.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.182+0000 7fd02d7fa700 0 client.6141 ms_handle_reset on v2:172.21.15.77:6834/2879527612 2022-08-01T19:39:29.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.182+0000 7fd062b31740 1 -- 172.21.15.17:0/1402886462 >> 172.21.15.17:0/1402886462 conn(0x1d9e450 msgr2=0x1da03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:29.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.182+0000 7fd062b31740 1 -- 172.21.15.17:0/1402886462 shutdown_connections 2022-08-01T19:39:29.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.182+0000 7fd062b31740 1 -- 172.21.15.17:0/1402886462 wait complete. 2022-08-01T19:39:29.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.182+0000 7fd062b31740 20 client.6141 trim_cache size 0 max 16384 2022-08-01T19:39:29.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.194+0000 7fd062b31740 1 Processor -- start 2022-08-01T19:39:29.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.194+0000 7fd062b31740 1 -- start start 2022-08-01T19:39:29.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.194+0000 7fd062b31740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1d9edf0 0x1dce610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:29.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.194+0000 7fd062b31740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1d9fa50 0x1dd18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:29.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.194+0000 7fd062b31740 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1da1710 0x1dd1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:29.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.194+0000 7fd062b31740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x1dd46c0 con 0x1d9edf0 2022-08-01T19:39:29.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.194+0000 7fd062b31740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x1dd26b0 con 0x1d9fa50 2022-08-01T19:39:29.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.194+0000 7fd062b31740 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x1d3e9d0 con 0x1da1710 2022-08-01T19:39:29.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.194+0000 7fd05259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1d9edf0 0x1dce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:29.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd05259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1d9edf0 0x1dce610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40112/0 (socket says 172.21.15.17:40112) 2022-08-01T19:39:29.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd05259c700 1 -- 172.21.15.17:0/3412197567 learned_addr learned my addr 172.21.15.17:0/3412197567 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:29.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd051d9b700 1 --2- 172.21.15.17:0/3412197567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1d9fa50 0x1dd18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:29.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd052d9d700 1 --2- 172.21.15.17:0/3412197567 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1da1710 0x1dd1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:29.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd05259c700 1 -- 172.21.15.17:0/3412197567 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1da1710 msgr2=0x1dd1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:29.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd05259c700 1 --2- 172.21.15.17:0/3412197567 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1da1710 0x1dd1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:29.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd05259c700 1 -- 172.21.15.17:0/3412197567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1d9fa50 msgr2=0x1dd18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:29.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd05259c700 1 --2- 172.21.15.17:0/3412197567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1d9fa50 0x1dd18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:29.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd05259c700 1 -- 172.21.15.17:0/3412197567 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1d015f0 con 0x1d9edf0 2022-08-01T19:39:29.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd052d9d700 1 --2- 172.21.15.17:0/3412197567 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1da1710 0x1dd1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:39:29.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd05259c700 1 --2- 172.21.15.17:0/3412197567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1d9edf0 0x1dce610 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7fd04406b240 tx=0x7fd044037000).ready entity=mon.0 client_cookie=9712a5b2b630e540 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:29.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd02f7fe700 1 -- 172.21.15.17:0/3412197567 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd04400bed0 con 0x1d9edf0 2022-08-01T19:39:29.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd02f7fe700 1 -- 172.21.15.17:0/3412197567 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd044005ed0 con 0x1d9edf0 2022-08-01T19:39:29.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd02f7fe700 1 -- 172.21.15.17:0/3412197567 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd044047610 con 0x1d9edf0 2022-08-01T19:39:29.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd062b31740 1 -- 172.21.15.17:0/3412197567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1d9edf0 msgr2=0x1dce610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:29.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd062b31740 1 --2- 172.21.15.17:0/3412197567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1d9edf0 0x1dce610 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7fd04406b240 tx=0x7fd044037000).stop 2022-08-01T19:39:29.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd062b31740 1 -- 172.21.15.17:0/3412197567 shutdown_connections 2022-08-01T19:39:29.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd062b31740 1 --2- 172.21.15.17:0/3412197567 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1da1710 0x1dd1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:29.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd062b31740 1 --2- 172.21.15.17:0/3412197567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1d9fa50 0x1dd18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:29.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd062b31740 1 --2- 172.21.15.17:0/3412197567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1d9edf0 0x1dce610 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:29.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd062b31740 1 -- 172.21.15.17:0/3412197567 >> 172.21.15.17:0/3412197567 conn(0x1d9e450 msgr2=0x1de9430 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:29.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd062b31740 1 -- 172.21.15.17:0/3412197567 shutdown_connections 2022-08-01T19:39:29.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd062b31740 1 -- 172.21.15.17:0/3412197567 wait complete. 2022-08-01T19:39:29.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd062b31740 1 -- start start 2022-08-01T19:39:29.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd062b31740 1 Processor -- start 2022-08-01T19:39:29.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd062b31740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1d9edf0 0x1de20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:29.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd062b31740 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1d9fa50 0x1e351f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:29.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd062b31740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1da1710 0x1e356e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:29.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd062b31740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x1dcfbe0 con 0x1da1710 2022-08-01T19:39:29.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd062b31740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e38030 con 0x1d9edf0 2022-08-01T19:39:29.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd062b31740 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x1de29f0 con 0x1d9fa50 2022-08-01T19:39:29.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd062b31740 10 client.0 ll_register_callbacks cb 0x1dd04b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-08-01T19:39:29.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd052d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1da1710 0x1e356e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:29.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd05259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1d9edf0 0x1de20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:29.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd052d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1da1710 0x1e356e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40122/0 (socket says 172.21.15.17:40122) 2022-08-01T19:39:29.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd051d9b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1d9fa50 0x1e351f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:29.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd05259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1d9edf0 0x1de20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:38040/0 (socket says 172.21.15.17:38040) 2022-08-01T19:39:29.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.198+0000 7fd052d9d700 1 -- 172.21.15.17:0/1968079263 learned_addr learned my addr 172.21.15.17:0/1968079263 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:29.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.202+0000 7fd05259c700 1 -- 172.21.15.17:0/1968079263 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1d9fa50 msgr2=0x1e351f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:29.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.202+0000 7fd05259c700 1 --2- 172.21.15.17:0/1968079263 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1d9fa50 0x1e351f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:29.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.202+0000 7fd05259c700 1 -- 172.21.15.17:0/1968079263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1da1710 msgr2=0x1e356e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:29.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.202+0000 7fd05259c700 1 --2- 172.21.15.17:0/1968079263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1da1710 0x1e356e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:29.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.202+0000 7fd05259c700 1 -- 172.21.15.17:0/1968079263 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1ce49f0 con 0x1d9edf0 2022-08-01T19:39:29.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.202+0000 7fd051d9b700 1 --2- 172.21.15.17:0/1968079263 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1d9fa50 0x1e351f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:39:29.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.202+0000 7fd05259c700 1 --2- 172.21.15.17:0/1968079263 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1d9edf0 0x1de20e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x1de2790 tx=0x7fd044041000).ready entity=mon.1 client_cookie=be23300c77c51956 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:29.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.202+0000 7fd02e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2022-08-01T19:39:29.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.202+0000 7fd02e7fc700 1 -- 172.21.15.17:0/1968079263 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd04400bed0 con 0x1d9edf0 2022-08-01T19:39:29.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.202+0000 7fd062b31740 1 -- 172.21.15.17:0/1968079263 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd044045a10 con 0x1d9edf0 2022-08-01T19:39:29.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.202+0000 7fd02e7fc700 1 -- 172.21.15.17:0/1968079263 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd044047610 con 0x1d9edf0 2022-08-01T19:39:29.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.202+0000 7fd02e7fc700 1 -- 172.21.15.17:0/1968079263 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0440473e0 con 0x1d9edf0 2022-08-01T19:39:29.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.202+0000 7fd062b31740 10 client.6130 fetch_fsmap learned FSMap version 30 2022-08-01T19:39:29.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.202+0000 7fd062b31740 1 -- 172.21.15.17:0/1968079263 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x1d0a1e0 con 0x1d9edf0 2022-08-01T19:39:29.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.202+0000 7fd02e7fc700 1 -- 172.21.15.17:0/1968079263 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd044045a10 con 0x1d9edf0 2022-08-01T19:39:29.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.202+0000 7fd02e7fc700 1 -- 172.21.15.17:0/1968079263 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 30) v1 ==== 1434+0+0 (secure 0 0 0) 0x7fd044006250 con 0x1d9edf0 2022-08-01T19:39:29.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.202+0000 7fd062b31740 10 client.6130 fetch_fsmap finished waiting for FSMap version 30 2022-08-01T19:39:29.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.202+0000 7fd062b31740 10 client.6130 resolve_mds: resolved ID 'b' to GID 5949 2022-08-01T19:39:29.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.202+0000 7fd062b31740 20 client.6130 populate_metadata read hostname 'smithi017' 2022-08-01T19:39:29.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.202+0000 7fd062b31740 1 --2- 172.21.15.17:0/1968079263 >> [v2:172.21.15.77:6834/2879527612,v1:172.21.15.77:6835/2879527612] conn(0x1e38420 0x1e38800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:29.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.202+0000 7fd062b31740 4 client.6130 mds_command: new command op to 5949 tid=0[{"prefix": "cache drop", "timeout": 10}] 2022-08-01T19:39:29.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.202+0000 7fd062b31740 1 -- 172.21.15.17:0/1968079263 --> [v2:172.21.15.77:6834/2879527612,v1:172.21.15.77:6835/2879527612] -- command(tid 0: {"prefix": "cache drop", "timeout": 10}) v1 -- 0x1bf55e0 con 0x1e38420 2022-08-01T19:39:29.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.202+0000 7fd051d9b700 1 --2- 172.21.15.17:0/1968079263 >> [v2:172.21.15.77:6834/2879527612,v1:172.21.15.77:6835/2879527612] conn(0x1e38420 0x1e38800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:29.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.202+0000 7fd02e7fc700 10 client.6130 ms_handle_connect on v2:172.21.15.77:6834/2879527612 2022-08-01T19:39:29.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:29.202+0000 7fd051d9b700 1 --2- 172.21.15.17:0/1968079263 >> [v2:172.21.15.77:6834/2879527612,v1:172.21.15.77:6835/2879527612] conn(0x1e38420 0x1e38800 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=3837d8bcd40e1a99 server_cookie=2902029361e6e680 in_seq=0 out_seq=0 2022-08-01T19:39:39.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:39.510+0000 7fd02e7fc700 1 -- 172.21.15.17:0/1968079263 <== mds.0 v2:172.21.15.77:6834/2879527612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+406 (crc 0 0 0) 0x1bf55e0 con 0x1e38420 2022-08-01T19:39:39.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:39.510+0000 7fd02e7fc700 10 client.6130 handle_command_reply: tid=0 2022-08-01T19:39:39.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:39.510+0000 7fd062b31740 1 client.6130 shutdown 2022-08-01T19:39:39.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:39.510+0000 7fd062b31740 1 -- 172.21.15.17:0/1968079263 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1d9edf0 msgr2=0x1de20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:39.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:39.510+0000 7fd062b31740 1 --2- 172.21.15.17:0/1968079263 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1d9edf0 0x1de20e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x1de2790 tx=0x7fd044041000).stop 2022-08-01T19:39:39.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:39.510+0000 7fd062b31740 1 -- 172.21.15.17:0/1968079263 shutdown_connections 2022-08-01T19:39:39.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:39.510+0000 7fd062b31740 1 --2- 172.21.15.17:0/1968079263 >> [v2:172.21.15.77:6834/2879527612,v1:172.21.15.77:6835/2879527612] conn(0x1e38420 0x1e38800 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:39.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:39.510+0000 7fd062b31740 1 --2- 172.21.15.17:0/1968079263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x1da1710 0x1e356e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:39.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:39.510+0000 7fd062b31740 1 --2- 172.21.15.17:0/1968079263 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x1d9fa50 0x1e351f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:39.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:39.510+0000 7fd062b31740 1 --2- 172.21.15.17:0/1968079263 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x1d9edf0 0x1de20e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:39.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:39.510+0000 7fd02e7fc700 0 client.6130 ms_handle_reset on v2:172.21.15.77:6834/2879527612 2022-08-01T19:39:39.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:39.510+0000 7fd062b31740 1 -- 172.21.15.17:0/1968079263 >> 172.21.15.17:0/1968079263 conn(0x1d9e450 msgr2=0x1de9350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:39.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:39.510+0000 7fd062b31740 1 -- 172.21.15.17:0/1968079263 shutdown_connections 2022-08-01T19:39:39.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:39.510+0000 7fd062b31740 1 -- 172.21.15.17:0/1968079263 wait complete. 2022-08-01T19:39:39.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:39.510+0000 7fd062b31740 20 client.6130 trim_cache size 0 max 16384 2022-08-01T19:39:39.520 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:39:39.521 INFO:teuthology.orchestra.run.smithi017.stdout: "client_recall": { 2022-08-01T19:39:39.521 INFO:teuthology.orchestra.run.smithi017.stdout: "return_code": -110, 2022-08-01T19:39:39.521 INFO:teuthology.orchestra.run.smithi017.stdout: "message": "(110) Connection timed out", 2022-08-01T19:39:39.522 INFO:teuthology.orchestra.run.smithi017.stdout: "recalled": 240 2022-08-01T19:39:39.522 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:39.523 INFO:teuthology.orchestra.run.smithi017.stdout: "flush_journal": { 2022-08-01T19:39:39.523 INFO:teuthology.orchestra.run.smithi017.stdout: "return_code": 0, 2022-08-01T19:39:39.523 INFO:teuthology.orchestra.run.smithi017.stdout: "message": "" 2022-08-01T19:39:39.524 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:39.524 INFO:teuthology.orchestra.run.smithi017.stdout: "trim_cache": { 2022-08-01T19:39:39.524 INFO:teuthology.orchestra.run.smithi017.stdout: "trimmed": 220 2022-08-01T19:39:39.525 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:39.525 INFO:teuthology.orchestra.run.smithi017.stdout: "cache": { 2022-08-01T19:39:39.525 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": { 2022-08-01T19:39:39.526 INFO:teuthology.orchestra.run.smithi017.stdout: "items": 15412, 2022-08-01T19:39:39.526 INFO:teuthology.orchestra.run.smithi017.stdout: "bytes": 2428176 2022-08-01T19:39:39.527 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:39.527 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:39.528 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 10.303162968000001 2022-08-01T19:39:39.528 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:39:39.532 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:39.510+0000 7fd0588cd700 1 -- 172.21.15.17:0/224238787 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd030053b00 msgr2=0x7fd030055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:39.532 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:39.510+0000 7fd0588cd700 1 --2- 172.21.15.17:0/224238787 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd030053b00 0x7fd030055fc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7fd04800d000 tx=0x7fd048007b90).stop 2022-08-01T19:39:39.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:39.510+0000 7fd0588cd700 1 -- 172.21.15.17:0/224238787 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd054060d00 msgr2=0x7fd0540ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:39.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:39.510+0000 7fd0588cd700 1 --2- 172.21.15.17:0/224238787 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd054060d00 0x7fd0540ad720 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7fd04c001b80 tx=0x7fd044014000).stop 2022-08-01T19:39:39.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:39.510+0000 7fd0588cd700 1 -- 172.21.15.17:0/224238787 shutdown_connections 2022-08-01T19:39:39.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:39.510+0000 7fd0588cd700 1 --2- 172.21.15.17:0/224238787 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd030053b00 0x7fd030055fc0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:39.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:39.510+0000 7fd0588cd700 1 --2- 172.21.15.17:0/224238787 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd0540b1b10 0x7fd0541525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:39.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:39.510+0000 7fd0588cd700 1 --2- 172.21.15.17:0/224238787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd054061860 0x7fd05414f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:39.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:39.510+0000 7fd0588cd700 1 --2- 172.21.15.17:0/224238787 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd054060d00 0x7fd0540ad720 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:39.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:39.510+0000 7fd0588cd700 1 -- 172.21.15.17:0/224238787 >> 172.21.15.17:0/224238787 conn(0x7fd0540b2b00 msgr2=0x7fd0541392c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:39.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:39.510+0000 7fd0588cd700 1 -- 172.21.15.17:0/224238787 shutdown_connections 2022-08-01T19:39:39.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:39.510+0000 7fd0588cd700 1 -- 172.21.15.17:0/224238787 wait complete. 2022-08-01T19:39:39.538 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T19:39:39.539 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:39:39.562 DEBUG:teuthology.orchestra.run.smithi077:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:39:39.584 DEBUG:teuthology.orchestra.run.smithi085:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:39:39.606 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:39:39.630 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T19:39:39.631 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:39:39.631 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:39:39.632 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:39:39.641 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:39:39.643 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:39:39.643 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi085.front.sepia.ceph.com... 2022-08-01T19:39:39.644 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T19:39:39.645 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:39:39.662 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:ceph-fuse[22929]: fuse finished with error 0 and tester_r 0 2022-08-01T19:39:39.664 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:39:39.665 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:39:39.665 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:39:39.716 INFO:teuthology.orchestra.run.smithi085.stdout:ext2/ext3 2022-08-01T19:39:39.716 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T19:39:39.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T19:39:39.824 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:39:39.833 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:39:39.835 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:39:39.835 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:39:39.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:39:39.845 INFO:teuthology.orchestra.run.smithi190.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T19:39:39.846 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-08-01T19:39:39.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T19:39:39.848 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:39:39.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:39:39.902 INFO:teuthology.orchestra.run.smithi190.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T19:39:39.903 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T19:39:40.026 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.018+0000 7ff7f3d50700 1 Processor -- start 2022-08-01T19:39:40.026 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.018+0000 7ff7f3d50700 1 -- start start 2022-08-01T19:39:40.027 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.018+0000 7ff7f3d50700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff7ec0b1f40 0x7ff7ec0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:40.027 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.018+0000 7ff7f3d50700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ff7ec0a7250 0x7ff7ec1448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:40.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.018+0000 7ff7f3d50700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7ec0b04e0 con 0x7ff7ec148530 2022-08-01T19:39:40.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.018+0000 7ff7f3d50700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7ec0a9250 con 0x7ff7ec0b1f40 2022-08-01T19:39:40.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.018+0000 7ff7f3d50700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7ec0b0780 con 0x7ff7ec0a7250 2022-08-01T19:39:40.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.018+0000 7ff7f1aec700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff7ec0b1f40 0x7ff7ec0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:40.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.018+0000 7ff7f12eb700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff7ec148530 0x7ff7ec0b2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60554/0 (socket says 172.21.15.17:60554) 2022-08-01T19:39:40.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.018+0000 7ff7f1aec700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff7ec0b1f40 0x7ff7ec0b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40126/0 (socket says 172.21.15.17:40126) 2022-08-01T19:39:40.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.018+0000 7ff7f12eb700 1 -- 172.21.15.17:0/1745187481 learned_addr learned my addr 172.21.15.17:0/1745187481 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:40.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.018+0000 7ff7f22ed700 1 --2- 172.21.15.17:0/1745187481 >> v2:172.21.15.77:3300/0 conn(0x7ff7ec0a7250 0x7ff7ec1448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:40.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.018+0000 7ff7f1aec700 1 -- 172.21.15.17:0/1745187481 >> v2:172.21.15.77:3300/0 conn(0x7ff7ec0a7250 msgr2=0x7ff7ec1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:40.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.018+0000 7ff7f1aec700 1 --2- 172.21.15.17:0/1745187481 >> v2:172.21.15.77:3300/0 conn(0x7ff7ec0a7250 0x7ff7ec1448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:40.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.018+0000 7ff7f1aec700 1 -- 172.21.15.17:0/1745187481 >> v1:172.21.15.17:6789/0 conn(0x7ff7ec148530 legacy=0x7ff7ec0b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:40.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.018+0000 7ff7f1aec700 1 -- 172.21.15.17:0/1745187481 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ec0b0a20 con 0x7ff7ec0b1f40 2022-08-01T19:39:40.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.018+0000 7ff7f1aec700 1 --2- 172.21.15.17:0/1745187481 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7ec0b1f40 0x7ff7ec0b1160 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7ff7ec1478c0 tx=0x7ff7e40091d0).ready entity=mon.0 client_cookie=9832394cb1d95eeb server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:40.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.018+0000 7ff7f0aea700 1 -- 172.21.15.17:0/1745187481 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7e4009ea0 con 0x7ff7ec0b1f40 2022-08-01T19:39:40.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.018+0000 7ff7f0aea700 1 -- 172.21.15.17:0/1745187481 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e400b860 con 0x7ff7ec0b1f40 2022-08-01T19:39:40.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.018+0000 7ff7f0aea700 1 -- 172.21.15.17:0/1745187481 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7e400b9d0 con 0x7ff7ec0b1f40 2022-08-01T19:39:40.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.022+0000 7ff7f3d50700 1 -- 172.21.15.17:0/1745187481 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7ec0b1f40 msgr2=0x7ff7ec0b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:40.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.022+0000 7ff7f3d50700 1 --2- 172.21.15.17:0/1745187481 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7ec0b1f40 0x7ff7ec0b1160 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7ff7ec1478c0 tx=0x7ff7e40091d0).stop 2022-08-01T19:39:40.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.022+0000 7ff7f3d50700 1 -- 172.21.15.17:0/1745187481 shutdown_connections 2022-08-01T19:39:40.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.022+0000 7ff7f3d50700 1 --2- 172.21.15.17:0/1745187481 >> v2:172.21.15.77:3300/0 conn(0x7ff7ec0a7250 0x7ff7ec1448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:40.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.022+0000 7ff7f3d50700 1 --2- 172.21.15.17:0/1745187481 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7ec0b1f40 0x7ff7ec0b1160 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:40.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.022+0000 7ff7f3d50700 1 -- 172.21.15.17:0/1745187481 >> 172.21.15.17:0/1745187481 conn(0x7ff7ec05f840 msgr2=0x7ff7ec05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:40.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.022+0000 7ff7f3d50700 1 -- 172.21.15.17:0/1745187481 shutdown_connections 2022-08-01T19:39:40.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.022+0000 7ff7f3d50700 1 -- 172.21.15.17:0/1745187481 wait complete. 2022-08-01T19:39:40.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.022+0000 7ff7f3d50700 1 Processor -- start 2022-08-01T19:39:40.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.022+0000 7ff7f3d50700 1 -- start start 2022-08-01T19:39:40.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.022+0000 7ff7f3d50700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7ec0a7250 0x7ff7ec14a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:40.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.022+0000 7ff7f3d50700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7ec0b1f40 0x7ff7ec1439d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:40.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.022+0000 7ff7f3d50700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff7ec148530 0x7ff7ec143ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:40.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.022+0000 7ff7f3d50700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ec0aa270 con 0x7ff7ec0a7250 2022-08-01T19:39:40.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.022+0000 7ff7f3d50700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ec0aaa30 con 0x7ff7ec0b1f40 2022-08-01T19:39:40.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.022+0000 7ff7f3d50700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ec14a9c0 con 0x7ff7ec148530 2022-08-01T19:39:40.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.022+0000 7ff7f1aec700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7ec0a7250 0x7ff7ec14a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:40.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.022+0000 7ff7f1aec700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7ec0a7250 0x7ff7ec14a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40132/0 (socket says 172.21.15.17:40132) 2022-08-01T19:39:40.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.022+0000 7ff7f1aec700 1 -- 172.21.15.17:0/2825423217 learned_addr learned my addr 172.21.15.17:0/2825423217 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:40.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.026+0000 7ff7f12eb700 1 --2- 172.21.15.17:0/2825423217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7ec0b1f40 0x7ff7ec1439d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:40.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.026+0000 7ff7f22ed700 1 --2- 172.21.15.17:0/2825423217 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff7ec148530 0x7ff7ec143ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:40.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.026+0000 7ff7f1aec700 1 -- 172.21.15.17:0/2825423217 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff7ec148530 msgr2=0x7ff7ec143ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:40.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.026+0000 7ff7f1aec700 1 --2- 172.21.15.17:0/2825423217 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff7ec148530 0x7ff7ec143ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:40.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.026+0000 7ff7f1aec700 1 -- 172.21.15.17:0/2825423217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7ec0b1f40 msgr2=0x7ff7ec1439d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:40.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.026+0000 7ff7f1aec700 1 --2- 172.21.15.17:0/2825423217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7ec0b1f40 0x7ff7ec1439d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:40.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.026+0000 7ff7f1aec700 1 -- 172.21.15.17:0/2825423217 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e4009470 con 0x7ff7ec0a7250 2022-08-01T19:39:40.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.026+0000 7ff7f12eb700 1 --2- 172.21.15.17:0/2825423217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7ec0b1f40 0x7ff7ec1439d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:39:40.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.026+0000 7ff7f22ed700 1 --2- 172.21.15.17:0/2825423217 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff7ec148530 0x7ff7ec143ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:39:40.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.026+0000 7ff7f1aec700 1 --2- 172.21.15.17:0/2825423217 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7ec0a7250 0x7ff7ec14a4d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7ff7ec1478c0 tx=0x7ff7e4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:40.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.026+0000 7ff7e2ffd700 1 -- 172.21.15.17:0/2825423217 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7e4011cc0 con 0x7ff7ec0a7250 2022-08-01T19:39:40.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.026+0000 7ff7f3d50700 1 -- 172.21.15.17:0/2825423217 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7ec0aaed0 con 0x7ff7ec0a7250 2022-08-01T19:39:40.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.026+0000 7ff7e2ffd700 1 -- 172.21.15.17:0/2825423217 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e4011e20 con 0x7ff7ec0a7250 2022-08-01T19:39:40.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.026+0000 7ff7e2ffd700 1 -- 172.21.15.17:0/2825423217 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7e401f5e0 con 0x7ff7ec0a7250 2022-08-01T19:39:40.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.026+0000 7ff7f3d50700 1 -- 172.21.15.17:0/2825423217 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7ec0a9e00 con 0x7ff7ec0a7250 2022-08-01T19:39:40.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.026+0000 7ff7f3d50700 1 -- 172.21.15.17:0/2825423217 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7ec0aa430 con 0x7ff7ec0a7250 2022-08-01T19:39:40.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.026+0000 7ff7e2ffd700 1 -- 172.21.15.17:0/2825423217 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff7e4026030 con 0x7ff7ec0a7250 2022-08-01T19:39:40.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.026+0000 7ff7e2ffd700 1 --2- 172.21.15.17:0/2825423217 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff7d4053a10 0x7ff7d4055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:40.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.026+0000 7ff7e2ffd700 1 -- 172.21.15.17:0/2825423217 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff7e406cdf0 con 0x7ff7ec0a7250 2022-08-01T19:39:40.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.030+0000 7ff7f12eb700 1 --2- 172.21.15.17:0/2825423217 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff7d4053a10 0x7ff7d4055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:40.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.030+0000 7ff7f12eb700 1 --2- 172.21.15.17:0/2825423217 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff7d4053a10 0x7ff7d4055ed0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7ff7dc008000 tx=0x7ff7dc009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:40.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.042+0000 7ff7e2ffd700 1 -- 172.21.15.17:0/2825423217 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff7e4047540 con 0x7ff7ec0a7250 2022-08-01T19:39:40.163 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.162+0000 7ff7f3d50700 1 -- 172.21.15.17:0/2825423217 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7ff7ec0b0cc0 con 0x7ff7ec0a7250 2022-08-01T19:39:40.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.162+0000 7ff7e2ffd700 1 -- 172.21.15.17:0/2825423217 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v66) v1 ==== 81+0+34040 (secure 0 0 0) 0x7ff7e4015020 con 0x7ff7ec0a7250 2022-08-01T19:39:40.166 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:39:40.166 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:39:40.167 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 66, 2022-08-01T19:39:40.167 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", 2022-08-01T19:39:40.168 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2022-08-01T19:33:50.105743+0000", 2022-08-01T19:39:40.168 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2022-08-01T19:39:11.935986+0000", 2022-08-01T19:39:40.169 INFO:teuthology.orchestra.run.smithi017.stdout: "last_up_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:39:40.169 INFO:teuthology.orchestra.run.smithi017.stdout: "last_in_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:39:40.169 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T19:39:40.169 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_num": 5799936, 2022-08-01T19:39:40.169 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_set": [ 2022-08-01T19:39:40.170 INFO:teuthology.orchestra.run.smithi017.stdout: "pglog_hardlimit", 2022-08-01T19:39:40.170 INFO:teuthology.orchestra.run.smithi017.stdout: "purged_snapdirs", 2022-08-01T19:39:40.171 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_deletes", 2022-08-01T19:39:40.171 INFO:teuthology.orchestra.run.smithi017.stdout: "sortbitwise" 2022-08-01T19:39:40.171 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:39:40.172 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_version": 6, 2022-08-01T19:39:40.172 INFO:teuthology.orchestra.run.smithi017.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T19:39:40.172 INFO:teuthology.orchestra.run.smithi017.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T19:39:40.172 INFO:teuthology.orchestra.run.smithi017.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T19:39:40.172 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_snapshot": "", 2022-08-01T19:39:40.173 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_max": 12, 2022-08-01T19:39:40.173 INFO:teuthology.orchestra.run.smithi017.stdout: "max_osd": 8, 2022-08-01T19:39:40.173 INFO:teuthology.orchestra.run.smithi017.stdout: "require_min_compat_client": "jewel", 2022-08-01T19:39:40.173 INFO:teuthology.orchestra.run.smithi017.stdout: "min_compat_client": "jewel", 2022-08-01T19:39:40.174 INFO:teuthology.orchestra.run.smithi017.stdout: "require_osd_release": "octopus", 2022-08-01T19:39:40.174 INFO:teuthology.orchestra.run.smithi017.stdout: "pools": [ 2022-08-01T19:39:40.175 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.175 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 1, 2022-08-01T19:39:40.175 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "device_health_metrics", 2022-08-01T19:39:40.176 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:33:53.001841+0000", 2022-08-01T19:39:40.176 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:39:40.176 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:39:40.176 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:39:40.177 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:39:40.177 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:39:40.177 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:39:40.177 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:39:40.177 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:39:40.178 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 1, 2022-08-01T19:39:40.178 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 1, 2022-08-01T19:39:40.178 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 1, 2022-08-01T19:39:40.178 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 1, 2022-08-01T19:39:40.179 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 1, 2022-08-01T19:39:40.179 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:39:40.179 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:39:40.179 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:39:40.179 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:39:40.180 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:39:40.180 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:39:40.180 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:39:40.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.181 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "16", 2022-08-01T19:39:40.181 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:39:40.181 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:39:40.181 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:39:40.182 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:39:40.182 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:39:40.182 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:39:40.182 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:39:40.183 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:39:40.183 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:39:40.183 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:39:40.183 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:39:40.184 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:39:40.184 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:39:40.184 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:39:40.185 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:39:40.185 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:39:40.185 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:39:40.185 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:39:40.186 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:39:40.186 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:39:40.186 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:39:40.186 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:39:40.187 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:39:40.187 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:39:40.187 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:39:40.187 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:39:40.187 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.188 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:39:40.188 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:39:40.188 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:39:40.188 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:39:40.189 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:39:40.189 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:39:40.189 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:39:40.189 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:39:40.190 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:39:40.190 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:39:40.190 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:39:40.190 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:39:40.191 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 1 2022-08-01T19:39:40.191 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.191 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:39:40.191 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_devicehealth": {} 2022-08-01T19:39:40.192 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.192 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.192 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.192 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 2, 2022-08-01T19:39:40.192 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "rbd", 2022-08-01T19:39:40.193 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:34:01.553098+0000", 2022-08-01T19:39:40.193 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:39:40.193 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:39:40.193 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:39:40.194 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:39:40.194 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:39:40.194 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:39:40.194 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:39:40.195 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:39:40.195 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:39:40.195 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:39:40.195 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:39:40.196 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:39:40.196 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:39:40.196 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:39:40.196 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:39:40.197 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:39:40.197 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:39:40.197 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:39:40.197 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:39:40.198 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:39:40.198 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.198 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "19", 2022-08-01T19:39:40.198 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:39:40.199 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:39:40.199 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:39:40.199 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:39:40.199 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:39:40.200 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:39:40.200 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:39:40.200 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:39:40.200 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:39:40.201 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:39:40.201 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:39:40.201 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:39:40.201 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:39:40.202 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:39:40.202 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:39:40.202 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:39:40.202 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:39:40.203 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:39:40.203 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:39:40.203 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:39:40.203 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:39:40.204 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:39:40.204 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:39:40.204 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:39:40.204 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:39:40.205 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:39:40.205 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.205 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:39:40.205 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:39:40.206 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:39:40.206 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:39:40.206 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:39:40.206 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:39:40.207 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:39:40.207 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:39:40.207 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:39:40.207 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:39:40.208 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:39:40.208 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:39:40.208 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:39:40.208 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd": {} 2022-08-01T19:39:40.209 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.209 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.209 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.209 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 11, 2022-08-01T19:39:40.209 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "cephfs_metadata", 2022-08-01T19:39:40.210 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:39:08.694582+0000", 2022-08-01T19:39:40.210 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:39:40.210 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:39:40.210 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:39:40.211 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:39:40.211 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:39:40.211 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:39:40.211 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:39:40.212 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:39:40.212 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:39:40.212 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:39:40.212 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:39:40.212 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:39:40.213 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:39:40.213 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:39:40.213 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:39:40.213 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:39:40.213 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:39:40.214 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:39:40.214 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:39:40.214 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:39:40.214 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.215 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "65", 2022-08-01T19:39:40.215 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:39:40.215 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:39:40.215 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:39:40.216 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:39:40.216 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:39:40.216 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:39:40.216 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:39:40.216 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:39:40.217 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:39:40.217 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:39:40.217 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:39:40.217 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:39:40.218 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:39:40.218 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:39:40.218 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:39:40.218 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:39:40.219 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:39:40.219 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:39:40.219 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:39:40.219 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:39:40.219 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:39:40.220 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:39:40.220 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:39:40.220 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:39:40.220 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:39:40.221 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:39:40.221 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.221 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:39:40.221 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:39:40.221 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:39:40.222 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:39:40.222 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:39:40.222 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:39:40.222 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:39:40.223 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:39:40.223 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:39:40.223 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:39:40.223 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:39:40.224 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:39:40.224 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_bias": 4, 2022-08-01T19:39:40.224 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 16, 2022-08-01T19:39:40.224 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_priority": 5 2022-08-01T19:39:40.224 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.225 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:39:40.225 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs": { 2022-08-01T19:39:40.225 INFO:teuthology.orchestra.run.smithi017.stdout: "metadata": "cephfs" 2022-08-01T19:39:40.225 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.226 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.226 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.226 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.226 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 12, 2022-08-01T19:39:40.226 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "cephfs_data", 2022-08-01T19:39:40.227 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:39:09.921664+0000", 2022-08-01T19:39:40.227 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:39:40.227 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:39:40.227 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:39:40.228 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:39:40.228 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:39:40.228 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:39:40.228 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:39:40.229 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:39:40.229 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:39:40.229 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:39:40.229 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:39:40.230 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:39:40.230 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:39:40.230 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:39:40.230 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:39:40.230 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:39:40.231 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:39:40.231 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:39:40.231 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:39:40.231 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:39:40.232 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.232 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "66", 2022-08-01T19:39:40.232 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:39:40.232 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:39:40.232 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:39:40.233 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:39:40.233 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:39:40.233 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:39:40.233 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:39:40.234 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:39:40.234 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:39:40.234 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:39:40.234 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:39:40.235 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:39:40.235 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:39:40.235 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:39:40.235 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:39:40.236 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:39:40.236 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:39:40.236 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:39:40.236 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:39:40.237 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:39:40.237 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:39:40.237 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:39:40.237 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:39:40.237 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:39:40.238 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:39:40.238 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:39:40.238 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.238 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:39:40.239 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:39:40.239 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:39:40.239 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:39:40.239 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:39:40.240 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:39:40.240 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:39:40.240 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:39:40.240 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:39:40.240 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:39:40.241 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:39:40.241 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:39:40.241 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:39:40.241 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs": { 2022-08-01T19:39:40.242 INFO:teuthology.orchestra.run.smithi017.stdout: "data": "cephfs" 2022-08-01T19:39:40.242 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.242 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.242 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.243 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:39:40.243 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2022-08-01T19:39:40.243 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.244 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:39:40.244 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a962f14f-cf16-4fa8-84ff-c85325db9132", 2022-08-01T19:39:40.244 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:39:40.244 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:39:40.244 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:39:40.245 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:39:40.245 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:39:40.245 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:39:40.245 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:39:40.246 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 64, 2022-08-01T19:39:40.246 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:39:40.246 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:39:40.246 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:39:40.247 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:40.247 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.247 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:40.247 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6800", 2022-08-01T19:39:40.247 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:39:40.248 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.248 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.248 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:40.248 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6801", 2022-08-01T19:39:40.249 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:39:40.249 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.249 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.249 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.249 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:39:40.250 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:40.250 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.250 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:40.250 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6802", 2022-08-01T19:39:40.251 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:39:40.251 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.251 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.251 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:40.252 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6803", 2022-08-01T19:39:40.252 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:39:40.252 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.252 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.252 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.253 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:39:40.253 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:40.253 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.253 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:40.254 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6806", 2022-08-01T19:39:40.254 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:39:40.254 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.254 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.254 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:40.255 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6807", 2022-08-01T19:39:40.255 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:39:40.255 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.256 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.256 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.256 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:39:40.256 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:40.257 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.257 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:40.257 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6804", 2022-08-01T19:39:40.257 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:39:40.258 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.258 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.258 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:40.258 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6805", 2022-08-01T19:39:40.259 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:39:40.259 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.259 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.259 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.259 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6801/23008", 2022-08-01T19:39:40.260 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6803/23008", 2022-08-01T19:39:40.260 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6807/23008", 2022-08-01T19:39:40.260 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6805/23008", 2022-08-01T19:39:40.260 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:39:40.261 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:39:40.261 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:39:40.261 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.261 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.262 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.262 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:39:40.262 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "805f7051-a527-4e98-8844-b6e406a63248", 2022-08-01T19:39:40.262 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:39:40.262 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:39:40.263 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:39:40.263 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:39:40.263 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:39:40.263 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:39:40.264 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:39:40.264 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 64, 2022-08-01T19:39:40.264 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:39:40.264 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:39:40.264 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:39:40.265 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:40.265 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.265 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:40.265 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6808", 2022-08-01T19:39:40.266 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:39:40.266 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.266 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.266 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:40.267 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6809", 2022-08-01T19:39:40.267 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:39:40.267 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.267 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.267 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.268 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:39:40.268 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:40.268 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.268 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:40.269 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6810", 2022-08-01T19:39:40.269 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:39:40.269 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.269 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.270 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:40.270 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6811", 2022-08-01T19:39:40.270 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:39:40.270 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.271 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.271 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.271 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:39:40.272 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:40.272 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.272 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:40.272 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6814", 2022-08-01T19:39:40.273 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:39:40.273 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.273 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.273 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:40.274 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6815", 2022-08-01T19:39:40.274 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:39:40.274 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.274 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.275 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.275 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:39:40.275 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:40.276 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.276 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:40.276 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6812", 2022-08-01T19:39:40.276 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:39:40.277 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.277 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.277 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:40.278 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6813", 2022-08-01T19:39:40.278 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:39:40.278 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.278 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.279 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.279 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6809/23007", 2022-08-01T19:39:40.279 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6811/23007", 2022-08-01T19:39:40.279 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6815/23007", 2022-08-01T19:39:40.280 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6813/23007", 2022-08-01T19:39:40.280 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:39:40.280 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:39:40.280 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:39:40.280 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.281 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.281 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.281 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:39:40.281 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "89f7f3df-7c10-48e1-8b69-9e4243923a61", 2022-08-01T19:39:40.282 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:39:40.282 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:39:40.282 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:39:40.282 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:39:40.283 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:39:40.283 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:39:40.283 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:39:40.284 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 63, 2022-08-01T19:39:40.284 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:39:40.285 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:39:40.285 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:39:40.286 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:40.286 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.287 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:40.287 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6817", 2022-08-01T19:39:40.288 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:39:40.288 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.289 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.289 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:40.290 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6819", 2022-08-01T19:39:40.290 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:39:40.291 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.291 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.292 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.292 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:39:40.293 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:40.294 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.294 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:40.295 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6822", 2022-08-01T19:39:40.295 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:39:40.296 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.296 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.296 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:40.297 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6824", 2022-08-01T19:39:40.297 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:39:40.297 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.298 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.298 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.298 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:39:40.298 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:40.299 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.299 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:40.299 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6830", 2022-08-01T19:39:40.299 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:39:40.300 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.300 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.300 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:40.300 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6831", 2022-08-01T19:39:40.301 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:39:40.301 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.301 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.301 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.302 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:39:40.302 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:40.302 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.302 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:40.303 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6826", 2022-08-01T19:39:40.303 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:39:40.303 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.304 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.304 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:40.305 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6829", 2022-08-01T19:39:40.305 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:39:40.305 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.306 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.306 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.307 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6819/23010", 2022-08-01T19:39:40.307 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6824/23010", 2022-08-01T19:39:40.308 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6831/23010", 2022-08-01T19:39:40.308 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6829/23010", 2022-08-01T19:39:40.309 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:39:40.309 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:39:40.310 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:39:40.310 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.311 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.311 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.312 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:39:40.312 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "96123468-aabc-4a64-909e-a192c369090d", 2022-08-01T19:39:40.312 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:39:40.313 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:39:40.313 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:39:40.314 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:39:40.314 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:39:40.314 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:39:40.315 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:39:40.315 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 52, 2022-08-01T19:39:40.316 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:39:40.316 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:39:40.316 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:39:40.317 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:40.317 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.318 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:40.318 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6816", 2022-08-01T19:39:40.319 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:39:40.319 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.319 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.320 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:40.320 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6818", 2022-08-01T19:39:40.321 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:39:40.321 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.321 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.322 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.322 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:39:40.323 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:40.323 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.323 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:40.324 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6820", 2022-08-01T19:39:40.324 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:39:40.325 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.325 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.325 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:40.326 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6821", 2022-08-01T19:39:40.326 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:39:40.327 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.327 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.327 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.328 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:39:40.328 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:40.329 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.329 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:40.329 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6827", 2022-08-01T19:39:40.330 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:39:40.330 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.330 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.331 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:40.331 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6828", 2022-08-01T19:39:40.331 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:39:40.331 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.331 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.332 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.332 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:39:40.332 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:40.332 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.333 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:40.333 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6823", 2022-08-01T19:39:40.333 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:39:40.333 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.333 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.334 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:40.334 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6825", 2022-08-01T19:39:40.334 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:39:40.334 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.335 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.335 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.335 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6818/23009", 2022-08-01T19:39:40.335 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6821/23009", 2022-08-01T19:39:40.336 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6828/23009", 2022-08-01T19:39:40.336 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6825/23009", 2022-08-01T19:39:40.336 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:39:40.336 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:39:40.336 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:39:40.337 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.337 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.337 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.337 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:39:40.337 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a6e26665-e04b-46ec-ab66-e912acbf044c", 2022-08-01T19:39:40.338 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:39:40.338 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:39:40.338 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:39:40.338 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:39:40.339 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:39:40.339 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:39:40.339 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:39:40.339 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 64, 2022-08-01T19:39:40.340 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:39:40.340 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:39:40.340 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:39:40.340 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:40.340 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.341 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:40.341 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6810", 2022-08-01T19:39:40.341 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:39:40.341 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.341 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.342 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:40.342 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6811", 2022-08-01T19:39:40.342 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:39:40.342 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.343 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.343 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.343 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:39:40.343 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:40.344 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.344 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:40.344 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6812", 2022-08-01T19:39:40.344 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:39:40.344 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.345 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.345 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:40.345 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6813", 2022-08-01T19:39:40.345 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:39:40.346 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.346 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.346 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.346 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:39:40.346 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:40.347 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.347 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:40.347 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6816", 2022-08-01T19:39:40.347 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:39:40.348 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.348 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.348 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:40.348 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6817", 2022-08-01T19:39:40.348 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:39:40.349 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.349 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.349 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.349 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:39:40.350 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:40.350 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.350 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:40.350 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6814", 2022-08-01T19:39:40.351 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:39:40.351 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.351 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.351 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:40.352 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6815", 2022-08-01T19:39:40.352 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:39:40.352 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.352 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.352 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.353 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6811/23584", 2022-08-01T19:39:40.353 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6813/23584", 2022-08-01T19:39:40.353 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6817/23584", 2022-08-01T19:39:40.353 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6815/23584", 2022-08-01T19:39:40.354 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:39:40.354 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:39:40.354 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:39:40.354 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.355 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.355 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.355 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:39:40.356 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "502c0034-18ee-44ac-87ff-19b9dddb66a0", 2022-08-01T19:39:40.356 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:39:40.356 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:39:40.357 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:39:40.357 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:39:40.357 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:39:40.357 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:39:40.357 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:39:40.358 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 63, 2022-08-01T19:39:40.358 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:39:40.358 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:39:40.358 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:39:40.359 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:40.359 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.359 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:40.359 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6802", 2022-08-01T19:39:40.360 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:39:40.360 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.360 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.360 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:40.360 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6803", 2022-08-01T19:39:40.361 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:39:40.361 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.361 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.361 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.362 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:39:40.362 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:40.362 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.362 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:40.362 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6804", 2022-08-01T19:39:40.363 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:39:40.363 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.363 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.363 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:40.364 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6805", 2022-08-01T19:39:40.364 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:39:40.364 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.364 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.364 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.365 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:39:40.365 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:40.365 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.365 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:40.366 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6808", 2022-08-01T19:39:40.366 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:39:40.366 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.366 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.367 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:40.367 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6809", 2022-08-01T19:39:40.367 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:39:40.367 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.367 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.368 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.368 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:39:40.368 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:40.368 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.369 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:40.369 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6806", 2022-08-01T19:39:40.369 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:39:40.369 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.369 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.370 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:40.370 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6807", 2022-08-01T19:39:40.370 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:39:40.370 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.371 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.371 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.371 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6803/23585", 2022-08-01T19:39:40.371 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6805/23585", 2022-08-01T19:39:40.371 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6809/23585", 2022-08-01T19:39:40.372 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6807/23585", 2022-08-01T19:39:40.372 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:39:40.372 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:39:40.372 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:39:40.373 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.373 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.373 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.373 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:39:40.373 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "ac4e755a-7fe7-46cb-8407-852d5af9ffc5", 2022-08-01T19:39:40.374 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:39:40.374 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:39:40.374 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:39:40.374 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:39:40.375 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:39:40.375 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:39:40.375 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:39:40.375 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 64, 2022-08-01T19:39:40.375 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:39:40.376 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:39:40.376 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:39:40.376 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:40.376 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.377 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:40.377 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6820", 2022-08-01T19:39:40.377 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:39:40.377 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.377 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.378 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:40.378 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6821", 2022-08-01T19:39:40.378 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:39:40.378 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.379 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.379 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.379 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:39:40.379 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:40.379 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.380 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:40.380 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6823", 2022-08-01T19:39:40.380 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:39:40.380 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.381 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.381 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:40.381 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6825", 2022-08-01T19:39:40.381 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:39:40.381 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.382 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.382 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.382 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:39:40.382 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:40.383 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.383 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:40.383 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6831", 2022-08-01T19:39:40.383 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:39:40.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.384 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.384 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:40.384 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6833", 2022-08-01T19:39:40.384 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:39:40.385 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.385 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.385 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:39:40.385 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:40.386 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.386 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:40.386 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6827", 2022-08-01T19:39:40.386 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:39:40.387 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.387 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.387 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:40.387 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6829", 2022-08-01T19:39:40.388 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:39:40.388 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.388 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.388 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.389 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6821/23587", 2022-08-01T19:39:40.389 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6825/23587", 2022-08-01T19:39:40.389 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6833/23587", 2022-08-01T19:39:40.389 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6829/23587", 2022-08-01T19:39:40.389 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:39:40.390 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:39:40.390 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:39:40.390 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.390 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.391 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.391 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:39:40.391 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "874761e2-6410-40fc-b60c-e58abe5ce489", 2022-08-01T19:39:40.392 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:39:40.392 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:39:40.392 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:39:40.392 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:39:40.393 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:39:40.393 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:39:40.393 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:39:40.394 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 63, 2022-08-01T19:39:40.394 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:39:40.394 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:39:40.394 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:39:40.395 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:40.395 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.395 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:40.396 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6818", 2022-08-01T19:39:40.396 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:39:40.396 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.397 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.397 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:40.397 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6819", 2022-08-01T19:39:40.398 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:39:40.398 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.398 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.399 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.399 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:39:40.400 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:40.400 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.400 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:40.401 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6822", 2022-08-01T19:39:40.401 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:39:40.402 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.402 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.402 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:40.403 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6824", 2022-08-01T19:39:40.403 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:39:40.403 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.403 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.404 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.404 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:39:40.404 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:40.404 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.405 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:40.405 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6830", 2022-08-01T19:39:40.405 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:39:40.405 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.405 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.406 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:40.406 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6832", 2022-08-01T19:39:40.406 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:39:40.407 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.407 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.407 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.408 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:39:40.408 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:40.408 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.408 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:40.408 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6826", 2022-08-01T19:39:40.409 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:39:40.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.409 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:40.410 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6828", 2022-08-01T19:39:40.410 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:39:40.410 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.410 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.411 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.411 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6819/23586", 2022-08-01T19:39:40.411 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6824/23586", 2022-08-01T19:39:40.412 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6832/23586", 2022-08-01T19:39:40.412 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6828/23586", 2022-08-01T19:39:40.412 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:39:40.412 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:39:40.413 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:39:40.413 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:40.413 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.414 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:39:40.414 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_xinfo": [ 2022-08-01T19:39:40.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.414 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:39:40.415 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:39:40.415 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:39:40.415 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:39:40.415 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:39:40.416 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:39:40.416 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.588817+0000", 2022-08-01T19:39:40.416 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:39:40.416 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.417 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.417 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:39:40.418 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:39:40.418 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:39:40.418 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:39:40.419 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:39:40.419 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:39:40.420 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.782753+0000", 2022-08-01T19:39:40.420 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:39:40.421 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.421 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.421 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:39:40.422 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:39:40.422 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:39:40.423 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:39:40.423 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:39:40.424 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:39:40.424 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.837924+0000", 2022-08-01T19:39:40.424 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:39:40.425 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.425 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.426 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:39:40.426 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:39:40.427 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:39:40.427 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:39:40.428 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:39:40.428 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:39:40.428 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.839948+0000", 2022-08-01T19:39:40.429 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:39:40.429 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.429 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.430 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:39:40.430 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:39:40.431 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:39:40.431 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:39:40.432 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:39:40.432 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:39:40.432 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.112801+0000", 2022-08-01T19:39:40.432 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:39:40.433 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.433 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.433 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:39:40.433 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:39:40.433 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:39:40.434 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:39:40.434 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:39:40.435 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:39:40.435 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.106704+0000", 2022-08-01T19:39:40.435 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:39:40.435 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.436 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.436 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:39:40.436 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:39:40.436 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:39:40.436 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:39:40.437 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:39:40.437 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:39:40.437 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.893170+0000", 2022-08-01T19:39:40.437 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:39:40.438 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.438 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:40.438 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:39:40.438 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:39:40.439 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:39:40.439 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:39:40.439 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:39:40.439 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:39:40.439 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.886947+0000", 2022-08-01T19:39:40.440 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:39:40.440 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.440 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:39:40.440 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap": [], 2022-08-01T19:39:40.441 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap_items": [], 2022-08-01T19:39:40.441 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_temp": [], 2022-08-01T19:39:40.441 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_temp": [], 2022-08-01T19:39:40.441 INFO:teuthology.orchestra.run.smithi017.stdout: "blacklist": {}, 2022-08-01T19:39:40.442 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profiles": { 2022-08-01T19:39:40.442 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_data": { 2022-08-01T19:39:40.442 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-device-class": "", 2022-08-01T19:39:40.443 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:39:40.443 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-root": "default", 2022-08-01T19:39:40.444 INFO:teuthology.orchestra.run.smithi017.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T19:39:40.444 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:39:40.444 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "2", 2022-08-01T19:39:40.445 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:39:40.445 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:39:40.446 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van", 2022-08-01T19:39:40.446 INFO:teuthology.orchestra.run.smithi017.stdout: "w": "8" 2022-08-01T19:39:40.447 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.447 INFO:teuthology.orchestra.run.smithi017.stdout: "default": { 2022-08-01T19:39:40.447 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:39:40.448 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:39:40.448 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "1", 2022-08-01T19:39:40.449 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:39:40.449 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:39:40.450 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van" 2022-08-01T19:39:40.450 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:40.451 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:40.451 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps_queue": [], 2022-08-01T19:39:40.451 INFO:teuthology.orchestra.run.smithi017.stdout: "new_removed_snaps": [], 2022-08-01T19:39:40.452 INFO:teuthology.orchestra.run.smithi017.stdout: "new_purged_snaps": [], 2022-08-01T19:39:40.452 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_node_flags": {}, 2022-08-01T19:39:40.453 INFO:teuthology.orchestra.run.smithi017.stdout: "device_class_flags": {} 2022-08-01T19:39:40.453 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:39:40.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.166+0000 7ff7f3d50700 1 -- 172.21.15.17:0/2825423217 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff7d4053a10 msgr2=0x7ff7d4055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:40.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.166+0000 7ff7f3d50700 1 --2- 172.21.15.17:0/2825423217 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff7d4053a10 0x7ff7d4055ed0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7ff7dc008000 tx=0x7ff7dc009040).stop 2022-08-01T19:39:40.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.166+0000 7ff7f3d50700 1 -- 172.21.15.17:0/2825423217 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7ec0a7250 msgr2=0x7ff7ec14a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:40.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.166+0000 7ff7f3d50700 1 --2- 172.21.15.17:0/2825423217 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7ec0a7250 0x7ff7ec14a4d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7ff7ec1478c0 tx=0x7ff7e4016000).stop 2022-08-01T19:39:40.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.166+0000 7ff7f3d50700 1 -- 172.21.15.17:0/2825423217 shutdown_connections 2022-08-01T19:39:40.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.166+0000 7ff7f3d50700 1 --2- 172.21.15.17:0/2825423217 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff7d4053a10 0x7ff7d4055ed0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:40.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.166+0000 7ff7f3d50700 1 --2- 172.21.15.17:0/2825423217 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff7ec148530 0x7ff7ec143ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:40.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.166+0000 7ff7f3d50700 1 --2- 172.21.15.17:0/2825423217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7ec0b1f40 0x7ff7ec1439d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:40.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.166+0000 7ff7f3d50700 1 --2- 172.21.15.17:0/2825423217 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7ec0a7250 0x7ff7ec14a4d0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:40.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.166+0000 7ff7f3d50700 1 -- 172.21.15.17:0/2825423217 >> 172.21.15.17:0/2825423217 conn(0x7ff7ec05f840 msgr2=0x7ff7ec1449c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:40.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.166+0000 7ff7f3d50700 1 -- 172.21.15.17:0/2825423217 shutdown_connections 2022-08-01T19:39:40.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.166+0000 7ff7f3d50700 1 -- 172.21.15.17:0/2825423217 wait complete. 2022-08-01T19:39:40.460 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:39:40.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.574+0000 7fe8b2f11700 1 Processor -- start 2022-08-01T19:39:40.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.574+0000 7fe8b2f11700 1 -- start start 2022-08-01T19:39:40.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.574+0000 7fe8b2f11700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fe8ac067180 0x7fe8ac143a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:40.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.574+0000 7fe8b2f11700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe8ac0b0870 0x7fe8ac0b1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:40.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.574+0000 7fe8b2f11700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8ac067560 con 0x7fe8ac0ad1c0 2022-08-01T19:39:40.602 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.574+0000 7fe8b2f11700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8ac0b0160 con 0x7fe8ac0b0870 2022-08-01T19:39:40.602 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.574+0000 7fe8b2f11700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8ac0b0400 con 0x7fe8ac067180 2022-08-01T19:39:40.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.574+0000 7fe8abfff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe8ac0ad1c0 0x7fe8ac143f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60566/0 (socket says 172.21.15.17:60566) 2022-08-01T19:39:40.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.574+0000 7fe8b14ae700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe8ac0b0870 0x7fe8ac0b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:40.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.574+0000 7fe8abfff700 1 -- 172.21.15.17:0/3955694094 learned_addr learned my addr 172.21.15.17:0/3955694094 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:40.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.574+0000 7fe8b0cad700 1 --2- 172.21.15.17:0/3955694094 >> v2:172.21.15.77:3300/0 conn(0x7fe8ac067180 0x7fe8ac143a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:40.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.574+0000 7fe8b14ae700 1 -- 172.21.15.17:0/3955694094 >> v2:172.21.15.77:3300/0 conn(0x7fe8ac067180 msgr2=0x7fe8ac143a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:40.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.574+0000 7fe8b14ae700 1 --2- 172.21.15.17:0/3955694094 >> v2:172.21.15.77:3300/0 conn(0x7fe8ac067180 0x7fe8ac143a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:40.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.574+0000 7fe8b14ae700 1 -- 172.21.15.17:0/3955694094 >> v1:172.21.15.17:6789/0 conn(0x7fe8ac0ad1c0 legacy=0x7fe8ac143f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:40.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.574+0000 7fe8b14ae700 1 -- 172.21.15.17:0/3955694094 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ac0af530 con 0x7fe8ac0b0870 2022-08-01T19:39:40.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.574+0000 7fe8b0cad700 1 --2- 172.21.15.17:0/3955694094 >> v2:172.21.15.77:3300/0 conn(0x7fe8ac067180 0x7fe8ac143a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:39:40.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.574+0000 7fe8b14ae700 1 --2- 172.21.15.17:0/3955694094 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8ac0b0870 0x7fe8ac0b1070 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7fe8a40032f0 tx=0x7fe8a40091d0).ready entity=mon.0 client_cookie=a2c47aa1178e89b0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:40.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.574+0000 7fe8ab7fe700 1 -- 172.21.15.17:0/3955694094 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8a4009ea0 con 0x7fe8ac0b0870 2022-08-01T19:39:40.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.574+0000 7fe8ab7fe700 1 -- 172.21.15.17:0/3955694094 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a400b870 con 0x7fe8ac0b0870 2022-08-01T19:39:40.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.574+0000 7fe8ab7fe700 1 -- 172.21.15.17:0/3955694094 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8a400b9e0 con 0x7fe8ac0b0870 2022-08-01T19:39:40.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.574+0000 7fe8b2f11700 1 -- 172.21.15.17:0/3955694094 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8ac0b0870 msgr2=0x7fe8ac0b1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:40.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.574+0000 7fe8b2f11700 1 --2- 172.21.15.17:0/3955694094 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8ac0b0870 0x7fe8ac0b1070 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7fe8a40032f0 tx=0x7fe8a40091d0).stop 2022-08-01T19:39:40.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.574+0000 7fe8b2f11700 1 -- 172.21.15.17:0/3955694094 shutdown_connections 2022-08-01T19:39:40.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.574+0000 7fe8b2f11700 1 --2- 172.21.15.17:0/3955694094 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8ac0b0870 0x7fe8ac0b1070 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:40.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.574+0000 7fe8b2f11700 1 --2- 172.21.15.17:0/3955694094 >> v2:172.21.15.77:3300/0 conn(0x7fe8ac067180 0x7fe8ac143a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:40.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.574+0000 7fe8b2f11700 1 -- 172.21.15.17:0/3955694094 >> 172.21.15.17:0/3955694094 conn(0x7fe8ac05f840 msgr2=0x7fe8ac05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:40.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.574+0000 7fe8b2f11700 1 -- 172.21.15.17:0/3955694094 shutdown_connections 2022-08-01T19:39:40.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.574+0000 7fe8b2f11700 1 -- 172.21.15.17:0/3955694094 wait complete. 2022-08-01T19:39:40.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.574+0000 7fe8b2f11700 1 Processor -- start 2022-08-01T19:39:40.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.574+0000 7fe8b2f11700 1 -- start start 2022-08-01T19:39:40.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.578+0000 7fe8b2f11700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8ac067180 0x7fe8ac1568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:40.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.578+0000 7fe8b2f11700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe8ac0b0870 0x7fe8ac0a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:40.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.578+0000 7fe8b2f11700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8ac1532b0 0x7fe8ac0a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:40.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.578+0000 7fe8b2f11700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac1537c0 con 0x7fe8ac067180 2022-08-01T19:39:40.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.578+0000 7fe8b2f11700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac155f20 con 0x7fe8ac1532b0 2022-08-01T19:39:40.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.578+0000 7fe8b2f11700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac156180 con 0x7fe8ac0b0870 2022-08-01T19:39:40.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.578+0000 7fe8b0cad700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8ac067180 0x7fe8ac1568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:40.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.578+0000 7fe8b0cad700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8ac067180 0x7fe8ac1568c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40146/0 (socket says 172.21.15.17:40146) 2022-08-01T19:39:40.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.578+0000 7fe8b0cad700 1 -- 172.21.15.17:0/4101075117 learned_addr learned my addr 172.21.15.17:0/4101075117 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:40.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.578+0000 7fe8b14ae700 1 --2- 172.21.15.17:0/4101075117 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8ac1532b0 0x7fe8ac0a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:40.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.578+0000 7fe8abfff700 1 --2- 172.21.15.17:0/4101075117 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe8ac0b0870 0x7fe8ac0a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:40.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.578+0000 7fe8b0cad700 1 -- 172.21.15.17:0/4101075117 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe8ac0b0870 msgr2=0x7fe8ac0a0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:40.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.578+0000 7fe8b0cad700 1 --2- 172.21.15.17:0/4101075117 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe8ac0b0870 0x7fe8ac0a0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:40.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.578+0000 7fe8b0cad700 1 -- 172.21.15.17:0/4101075117 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8ac1532b0 msgr2=0x7fe8ac0a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:40.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.578+0000 7fe8b0cad700 1 --2- 172.21.15.17:0/4101075117 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8ac1532b0 0x7fe8ac0a17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:40.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.578+0000 7fe8b0cad700 1 -- 172.21.15.17:0/4101075117 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a4009470 con 0x7fe8ac067180 2022-08-01T19:39:40.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.578+0000 7fe8abfff700 1 --2- 172.21.15.17:0/4101075117 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe8ac0b0870 0x7fe8ac0a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:39:40.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.578+0000 7fe8b0cad700 1 --2- 172.21.15.17:0/4101075117 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8ac067180 0x7fe8ac1568c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7fe8a0006940 tx=0x7fe8a0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:40.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.578+0000 7fe8a9ffb700 1 -- 172.21.15.17:0/4101075117 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8a0011850 con 0x7fe8ac067180 2022-08-01T19:39:40.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.578+0000 7fe8a9ffb700 1 -- 172.21.15.17:0/4101075117 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a00119b0 con 0x7fe8ac067180 2022-08-01T19:39:40.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.578+0000 7fe8b2f11700 1 -- 172.21.15.17:0/4101075117 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8ac156420 con 0x7fe8ac067180 2022-08-01T19:39:40.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.578+0000 7fe8a9ffb700 1 -- 172.21.15.17:0/4101075117 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8a0011d10 con 0x7fe8ac067180 2022-08-01T19:39:40.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.578+0000 7fe8b2f11700 1 -- 172.21.15.17:0/4101075117 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8ac0a1cb0 con 0x7fe8ac067180 2022-08-01T19:39:40.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.578+0000 7fe8a9ffb700 1 -- 172.21.15.17:0/4101075117 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe8a0006030 con 0x7fe8ac067180 2022-08-01T19:39:40.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.578+0000 7fe8b2f11700 1 -- 172.21.15.17:0/4101075117 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8ac146fb0 con 0x7fe8ac067180 2022-08-01T19:39:40.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.582+0000 7fe8a9ffb700 1 --2- 172.21.15.17:0/4101075117 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe88c053b00 0x7fe88c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:40.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.582+0000 7fe8a9ffb700 1 -- 172.21.15.17:0/4101075117 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe8a006bfc0 con 0x7fe8ac067180 2022-08-01T19:39:40.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.582+0000 7fe8abfff700 1 --2- 172.21.15.17:0/4101075117 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe88c053b00 0x7fe88c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:40.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.582+0000 7fe8abfff700 1 --2- 172.21.15.17:0/4101075117 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe88c053b00 0x7fe88c055fc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7fe89c00b000 tx=0x7fe89c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:40.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.590+0000 7fe8a9ffb700 1 -- 172.21.15.17:0/4101075117 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe8ac146fb0 con 0x7fe8ac067180 2022-08-01T19:39:40.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.750+0000 7fe8b2f11700 1 -- 172.21.15.17:0/4101075117 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8ac146fb0 con 0x7fe8ac067180 2022-08-01T19:39:40.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.750+0000 7fe8a9ffb700 1 -- 172.21.15.17:0/4101075117 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+2493 (secure 0 0 0) 0x7fe8ac146fb0 con 0x7fe8ac067180 2022-08-01T19:39:40.753 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:39:40.754 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":30,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5414,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1130955771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1130955771},{"type":"v1","addr":"172.21.15.17:6835","nonce":1130955771}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":28},{"gid":5940,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3691532323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3691532323},{"type":"v1","addr":"172.21.15.17:6833","nonce":3691532323}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":28}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:39:10.932933+0000","modified":"2022-08-01T19:39:11.936834+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5949},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5949":{"gid":5949,"name":"b","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6835/2879527612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2879527612},{"type":"v1","addr":"172.21.15.77:6835","nonce":2879527612}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-08-01T19:39:40.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.754+0000 7fe8b2f11700 1 -- 172.21.15.17:0/4101075117 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe88c053b00 msgr2=0x7fe88c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:40.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.754+0000 7fe8b2f11700 1 --2- 172.21.15.17:0/4101075117 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe88c053b00 0x7fe88c055fc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7fe89c00b000 tx=0x7fe89c009040).stop 2022-08-01T19:39:40.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.754+0000 7fe8b2f11700 1 -- 172.21.15.17:0/4101075117 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8ac067180 msgr2=0x7fe8ac1568c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:40.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.754+0000 7fe8b2f11700 1 --2- 172.21.15.17:0/4101075117 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8ac067180 0x7fe8ac1568c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7fe8a0006940 tx=0x7fe8a0008040).stop 2022-08-01T19:39:40.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.754+0000 7fe8b2f11700 1 -- 172.21.15.17:0/4101075117 shutdown_connections 2022-08-01T19:39:40.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.754+0000 7fe8b2f11700 1 --2- 172.21.15.17:0/4101075117 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe88c053b00 0x7fe88c055fc0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:40.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.754+0000 7fe8b2f11700 1 --2- 172.21.15.17:0/4101075117 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8ac1532b0 0x7fe8ac0a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:40.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.754+0000 7fe8b2f11700 1 --2- 172.21.15.17:0/4101075117 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe8ac0b0870 0x7fe8ac0a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:40.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.754+0000 7fe8b2f11700 1 --2- 172.21.15.17:0/4101075117 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8ac067180 0x7fe8ac1568c0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:40.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.754+0000 7fe8b2f11700 1 -- 172.21.15.17:0/4101075117 >> 172.21.15.17:0/4101075117 conn(0x7fe8ac05f840 msgr2=0x7fe8ac143a20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:40.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.754+0000 7fe8b2f11700 1 -- 172.21.15.17:0/4101075117 shutdown_connections 2022-08-01T19:39:40.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.754+0000 7fe8b2f11700 1 -- 172.21.15.17:0/4101075117 wait complete. 2022-08-01T19:39:40.759 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 30 2022-08-01T19:39:40.767 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-08-01T19:39:40.871 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff972aaf700 1 Processor -- start 2022-08-01T19:39:40.871 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff972aaf700 1 -- start start 2022-08-01T19:39:40.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff972aaf700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff96c0af6e0 0x7ff96c147730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:40.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff972aaf700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff96c0b5a10 con 0x7ff96c0b5630 2022-08-01T19:39:40.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff972aaf700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff96c0ac990 con 0x7ff96c0b07a0 2022-08-01T19:39:40.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff972aaf700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c145470 con 0x7ff96c0af6e0 2022-08-01T19:39:40.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff971aad700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff96c0b5630 0x7ff96c067730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60580/0 (socket says 172.21.15.17:60580) 2022-08-01T19:39:40.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff971aad700 1 -- 172.21.15.17:0/194503559 learned_addr learned my addr 172.21.15.17:0/194503559 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:40.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff9722ae700 1 --2- 172.21.15.17:0/194503559 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff96c0af6e0 0x7ff96c147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:40.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff970aab700 1 -- 172.21.15.17:0/194503559 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2417870059 0 0) 0x7ff96c0b5a10 con 0x7ff96c0b5630 2022-08-01T19:39:40.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff970aab700 1 -- 172.21.15.17:0/194503559 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff954003260 con 0x7ff96c0b5630 2022-08-01T19:39:40.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff970aab700 1 -- 172.21.15.17:0/194503559 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3509343763 0 0) 0x7ff96c0ac990 con 0x7ff96c0b07a0 2022-08-01T19:39:40.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff970aab700 1 -- 172.21.15.17:0/194503559 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff96c0b5a10 con 0x7ff96c0b07a0 2022-08-01T19:39:40.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff970aab700 1 -- 172.21.15.17:0/194503559 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2631726192 0 0) 0x7ff954003260 con 0x7ff96c0b5630 2022-08-01T19:39:40.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff970aab700 1 -- 172.21.15.17:0/194503559 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff96c0af6e0 msgr2=0x7ff96c147730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:40.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff970aab700 1 --2- 172.21.15.17:0/194503559 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff96c0af6e0 0x7ff96c147730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:40.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff970aab700 1 -- 172.21.15.17:0/194503559 >> v1:172.21.15.77:6789/0 conn(0x7ff96c0b07a0 legacy=0x7ff96c0acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:40.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff970aab700 1 -- 172.21.15.17:0/194503559 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff96c0b1bc0 con 0x7ff96c0b5630 2022-08-01T19:39:40.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff970aab700 1 -- 172.21.15.17:0/194503559 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7ff968001660 con 0x7ff96c0b5630 2022-08-01T19:39:40.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff9722ae700 1 --2- 172.21.15.17:0/194503559 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff96c0af6e0 0x7ff96c147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:39:40.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff970aab700 1 -- 172.21.15.17:0/194503559 >> v1:172.21.15.17:6789/0 conn(0x7ff96c0b5630 legacy=0x7ff96c067730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:40.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff970aab700 1 --2- 172.21.15.17:0/194503559 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff96c0af6e0 0x7ff954003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:40.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff970aab700 1 --2- 172.21.15.17:0/194503559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff954003e00 0x7ff954004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:40.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff970aab700 1 --2- 172.21.15.17:0/194503559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9540049f0 0x7ff954004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:40.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff970aab700 1 -- 172.21.15.17:0/194503559 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c145470 con 0x7ff9540049f0 2022-08-01T19:39:40.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff970aab700 1 -- 172.21.15.17:0/194503559 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95400b7b0 con 0x7ff954003e00 2022-08-01T19:39:40.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff970aab700 1 -- 172.21.15.17:0/194503559 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff95400b970 con 0x7ff96c0af6e0 2022-08-01T19:39:40.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff971aad700 1 --2- 172.21.15.17:0/194503559 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff96c0af6e0 0x7ff954003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:40.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff9712ac700 1 --2- 172.21.15.17:0/194503559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff954003e00 0x7ff954004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:40.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff9722ae700 1 --2- 172.21.15.17:0/194503559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9540049f0 0x7ff954004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:40.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff971aad700 1 -- 172.21.15.17:0/194503559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff954003e00 msgr2=0x7ff954004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:40.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff971aad700 1 --2- 172.21.15.17:0/194503559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff954003e00 0x7ff954004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:40.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff971aad700 1 -- 172.21.15.17:0/194503559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9540049f0 msgr2=0x7ff954004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:40.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff971aad700 1 --2- 172.21.15.17:0/194503559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9540049f0 0x7ff954004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:40.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff971aad700 1 -- 172.21.15.17:0/194503559 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff964009470 con 0x7ff96c0af6e0 2022-08-01T19:39:40.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff9712ac700 1 --2- 172.21.15.17:0/194503559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff954003e00 0x7ff954004500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:39:40.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff9722ae700 1 --2- 172.21.15.17:0/194503559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9540049f0 0x7ff954004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:39:40.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff971aad700 1 --2- 172.21.15.17:0/194503559 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff96c0af6e0 0x7ff954003910 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7ff968001430 tx=0x7ff968001dd0).ready entity=mon.2 client_cookie=d1c1406e85d89316 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:40.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff970aab700 1 -- 172.21.15.17:0/194503559 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9680070d0 con 0x7ff96c0af6e0 2022-08-01T19:39:40.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff970aab700 1 -- 172.21.15.17:0/194503559 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff968007230 con 0x7ff96c0af6e0 2022-08-01T19:39:40.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff970aab700 1 -- 172.21.15.17:0/194503559 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff96800c8f0 con 0x7ff96c0af6e0 2022-08-01T19:39:40.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff972aaf700 1 -- 172.21.15.17:0/194503559 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff96c0af6e0 msgr2=0x7ff954003910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:40.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff972aaf700 1 --2- 172.21.15.17:0/194503559 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff96c0af6e0 0x7ff954003910 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7ff968001430 tx=0x7ff968001dd0).stop 2022-08-01T19:39:40.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff9722ae700 1 -- 172.21.15.17:0/194503559 reap_dead start 2022-08-01T19:39:40.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff972aaf700 1 -- 172.21.15.17:0/194503559 shutdown_connections 2022-08-01T19:39:40.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff972aaf700 1 -- 172.21.15.17:0/194503559 >> 172.21.15.17:0/194503559 conn(0x7ff96c0626f0 msgr2=0x7ff96c062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:40.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff972aaf700 1 -- 172.21.15.17:0/194503559 shutdown_connections 2022-08-01T19:39:40.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff972aaf700 1 -- 172.21.15.17:0/194503559 wait complete. 2022-08-01T19:39:40.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff972aaf700 1 Processor -- start 2022-08-01T19:39:40.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.874+0000 7ff972aaf700 1 -- start start 2022-08-01T19:39:40.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.878+0000 7ff972aaf700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff96c0af6e0 0x7ff96c065920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:40.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.878+0000 7ff972aaf700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff96c14a120 0x7ff96c143660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:40.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.878+0000 7ff972aaf700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff96c143b50 0x7ff96c156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:40.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.878+0000 7ff972aaf700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c0ae180 con 0x7ff96c14a120 2022-08-01T19:39:40.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.878+0000 7ff972aaf700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c14a830 con 0x7ff96c0af6e0 2022-08-01T19:39:40.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.878+0000 7ff972aaf700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c0a72f0 con 0x7ff96c143b50 2022-08-01T19:39:40.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.878+0000 7ff9712ac700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff96c14a120 0x7ff96c143660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:40.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.878+0000 7ff9712ac700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff96c14a120 0x7ff96c143660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40168/0 (socket says 172.21.15.17:40168) 2022-08-01T19:39:40.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.878+0000 7ff9712ac700 1 -- 172.21.15.17:0/3775631421 learned_addr learned my addr 172.21.15.17:0/3775631421 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:40.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.878+0000 7ff971aad700 1 --2- 172.21.15.17:0/3775631421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff96c0af6e0 0x7ff96c065920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:40.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.878+0000 7ff9722ae700 1 --2- 172.21.15.17:0/3775631421 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff96c143b50 0x7ff96c156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:40.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.878+0000 7ff9712ac700 1 -- 172.21.15.17:0/3775631421 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff96c143b50 msgr2=0x7ff96c156c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:40.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.878+0000 7ff9712ac700 1 --2- 172.21.15.17:0/3775631421 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff96c143b50 0x7ff96c156c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:40.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.878+0000 7ff9712ac700 1 -- 172.21.15.17:0/3775631421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff96c0af6e0 msgr2=0x7ff96c065920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:40.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.878+0000 7ff9712ac700 1 --2- 172.21.15.17:0/3775631421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff96c0af6e0 0x7ff96c065920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:40.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.878+0000 7ff9712ac700 1 -- 172.21.15.17:0/3775631421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff96c0a7590 con 0x7ff96c14a120 2022-08-01T19:39:40.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.878+0000 7ff9712ac700 1 --2- 172.21.15.17:0/3775631421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff96c14a120 0x7ff96c143660 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7ff95c00d000 tx=0x7ff95c0079e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:40.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.878+0000 7ff95affd700 1 -- 172.21.15.17:0/3775631421 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff95c0018a0 con 0x7ff96c14a120 2022-08-01T19:39:40.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.878+0000 7ff95affd700 1 -- 172.21.15.17:0/3775631421 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff95c001a00 con 0x7ff96c14a120 2022-08-01T19:39:40.899 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.878+0000 7ff972aaf700 1 -- 172.21.15.17:0/3775631421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff96c0a7830 con 0x7ff96c14a120 2022-08-01T19:39:40.899 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.878+0000 7ff95affd700 1 -- 172.21.15.17:0/3775631421 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff95c001d60 con 0x7ff96c14a120 2022-08-01T19:39:40.899 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.878+0000 7ff972aaf700 1 -- 172.21.15.17:0/3775631421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff96c0a9b70 con 0x7ff96c14a120 2022-08-01T19:39:40.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.878+0000 7ff972aaf700 1 -- 172.21.15.17:0/3775631421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff96c0b1680 con 0x7ff96c14a120 2022-08-01T19:39:40.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.878+0000 7ff95affd700 1 -- 172.21.15.17:0/3775631421 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff95c01b030 con 0x7ff96c14a120 2022-08-01T19:39:40.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.882+0000 7ff95affd700 1 --2- 172.21.15.17:0/3775631421 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff94c053a70 0x7ff94c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:40.901 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.882+0000 7ff95affd700 1 -- 172.21.15.17:0/3775631421 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff95c06c330 con 0x7ff96c14a120 2022-08-01T19:39:40.901 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.882+0000 7ff971aad700 1 --2- 172.21.15.17:0/3775631421 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff94c053a70 0x7ff94c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:40.902 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.882+0000 7ff971aad700 1 --2- 172.21.15.17:0/3775631421 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff94c053a70 0x7ff94c055f30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7ff968018000 tx=0x7ff968015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:40.902 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:40.890+0000 7ff95affd700 1 -- 172.21.15.17:0/3775631421 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff95c043140 con 0x7ff96c14a120 2022-08-01T19:39:41.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.046+0000 7ff972aaf700 1 -- 172.21.15.17:0/3775631421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff96c0a7a30 con 0x7ff96c14a120 2022-08-01T19:39:41.197 INFO:tasks.ceph.mon.a.smithi017.stderr:2022-08-01T19:39:41.198+0000 7f6a53773700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-08-01T19:39:41.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.202+0000 7ff95affd700 1 -- 172.21.15.17:0/3775631421 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v31) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff95c046760 con 0x7ff96c14a120 2022-08-01T19:39:41.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.206+0000 7ff972aaf700 1 -- 172.21.15.17:0/3775631421 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff94c053a70 msgr2=0x7ff94c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:41.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.206+0000 7ff972aaf700 1 --2- 172.21.15.17:0/3775631421 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff94c053a70 0x7ff94c055f30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7ff968018000 tx=0x7ff968015000).stop 2022-08-01T19:39:41.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.206+0000 7ff972aaf700 1 -- 172.21.15.17:0/3775631421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff96c14a120 msgr2=0x7ff96c143660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:41.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.206+0000 7ff972aaf700 1 --2- 172.21.15.17:0/3775631421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff96c14a120 0x7ff96c143660 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7ff95c00d000 tx=0x7ff95c0079e0).stop 2022-08-01T19:39:41.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.206+0000 7ff972aaf700 1 -- 172.21.15.17:0/3775631421 shutdown_connections 2022-08-01T19:39:41.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.206+0000 7ff972aaf700 1 --2- 172.21.15.17:0/3775631421 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff94c053a70 0x7ff94c055f30 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:41.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.206+0000 7ff972aaf700 1 --2- 172.21.15.17:0/3775631421 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff96c143b50 0x7ff96c156c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:41.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.206+0000 7ff972aaf700 1 --2- 172.21.15.17:0/3775631421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff96c14a120 0x7ff96c143660 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:41.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.206+0000 7ff972aaf700 1 --2- 172.21.15.17:0/3775631421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff96c0af6e0 0x7ff96c065920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:41.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.206+0000 7ff972aaf700 1 -- 172.21.15.17:0/3775631421 >> 172.21.15.17:0/3775631421 conn(0x7ff96c0626f0 msgr2=0x7ff96c148980 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:41.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.206+0000 7ff972aaf700 1 -- 172.21.15.17:0/3775631421 shutdown_connections 2022-08-01T19:39:41.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.206+0000 7ff972aaf700 1 -- 172.21.15.17:0/3775631421 wait complete. 2022-08-01T19:39:41.212 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-08-01T19:39:41.224 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:39:41.241 INFO:tasks.ceph.mds.b.smithi077.stdout:starting mds.b at 2022-08-01T19:39:41.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.330+0000 7f0289a5b700 1 Processor -- start 2022-08-01T19:39:41.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.330+0000 7f0289a5b700 1 -- start start 2022-08-01T19:39:41.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.330+0000 7f0289a5b700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f028405c960 0x7f02840b1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:41.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.330+0000 7f0289a5b700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f028405cd40 con 0x7f028405d240 2022-08-01T19:39:41.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.330+0000 7f0289a5b700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02840b1920 con 0x7f028413f520 2022-08-01T19:39:41.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.330+0000 7f0289a5b700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02840ad6d0 con 0x7f028405c960 2022-08-01T19:39:41.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.330+0000 7f027bfff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f028405d240 0x7f02840acfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60600/0 (socket says 172.21.15.17:60600) 2022-08-01T19:39:41.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.330+0000 7f027bfff700 1 -- 172.21.15.17:0/440699986 learned_addr learned my addr 172.21.15.17:0/440699986 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:41.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.330+0000 7f0288a59700 1 --2- 172.21.15.17:0/440699986 >> v2:172.21.15.77:3300/0 conn(0x7f028405c960 0x7f02840b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:41.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.330+0000 7f027b7fe700 1 -- 172.21.15.17:0/440699986 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 215102813 0 0) 0x7f028405cd40 con 0x7f028405d240 2022-08-01T19:39:41.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.330+0000 7f027b7fe700 1 -- 172.21.15.17:0/440699986 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0270003260 con 0x7f028405d240 2022-08-01T19:39:41.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.330+0000 7f0288a59700 1 -- 172.21.15.17:0/440699986 >> v1:172.21.15.77:6789/0 conn(0x7f028413f520 legacy=0x7f02840aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:41.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.330+0000 7f027b7fe700 1 -- 172.21.15.17:0/440699986 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3382855377 0 0) 0x7f02840b1920 con 0x7f028413f520 2022-08-01T19:39:41.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.330+0000 7f0288a59700 1 -- 172.21.15.17:0/440699986 >> v1:172.21.15.17:6789/0 conn(0x7f028405d240 legacy=0x7f02840acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:41.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.330+0000 7f0288a59700 1 -- 172.21.15.17:0/440699986 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02840b1bc0 con 0x7f028405c960 2022-08-01T19:39:41.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.330+0000 7f0288a59700 1 --2- 172.21.15.17:0/440699986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f028405c960 0x7f02840b1280 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f0274001800 tx=0x7f0280006df0).ready entity=mon.1 client_cookie=178b6ee3280cd211 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:41.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.330+0000 7f027b7fe700 1 -- 172.21.15.17:0/440699986 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0274001660 con 0x7f028405c960 2022-08-01T19:39:41.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.330+0000 7f027b7fe700 1 -- 172.21.15.17:0/440699986 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0280000e90 con 0x7f028405c960 2022-08-01T19:39:41.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.330+0000 7f027b7fe700 1 -- 172.21.15.17:0/440699986 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0280011400 con 0x7f028405c960 2022-08-01T19:39:41.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.330+0000 7f0289a5b700 1 -- 172.21.15.17:0/440699986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f028405c960 msgr2=0x7f02840b1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:41.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.330+0000 7f0289a5b700 1 --2- 172.21.15.17:0/440699986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f028405c960 0x7f02840b1280 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f0274001800 tx=0x7f0280006df0).stop 2022-08-01T19:39:41.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.330+0000 7f0289a5b700 1 -- 172.21.15.17:0/440699986 shutdown_connections 2022-08-01T19:39:41.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.330+0000 7f0289a5b700 1 --2- 172.21.15.17:0/440699986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f028405c960 0x7f02840b1280 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:41.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.330+0000 7f0289a5b700 1 -- 172.21.15.17:0/440699986 >> 172.21.15.17:0/440699986 conn(0x7f0284060310 msgr2=0x7f02840b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:41.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.330+0000 7f0289a5b700 1 -- 172.21.15.17:0/440699986 shutdown_connections 2022-08-01T19:39:41.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.330+0000 7f0289a5b700 1 -- 172.21.15.17:0/440699986 wait complete. 2022-08-01T19:39:41.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.334+0000 7f0289a5b700 1 Processor -- start 2022-08-01T19:39:41.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.334+0000 7f0289a5b700 1 -- start start 2022-08-01T19:39:41.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.338+0000 7f0289a5b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f028405c960 0x7f0284142df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:41.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.338+0000 7f0289a5b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f028405d240 0x7f0284152ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:41.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.338+0000 7f0289a5b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f028413f520 0x7f028413a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:41.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.338+0000 7f0289a5b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0284064810 con 0x7f028405c960 2022-08-01T19:39:41.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.338+0000 7f0289a5b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0284064a40 con 0x7f028405d240 2022-08-01T19:39:41.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.338+0000 7f0289a5b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0284064ce0 con 0x7f028413f520 2022-08-01T19:39:41.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.338+0000 7f0288a59700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f028405c960 0x7f0284142df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:41.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.338+0000 7f0288a59700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f028405c960 0x7f0284142df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40180/0 (socket says 172.21.15.17:40180) 2022-08-01T19:39:41.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.338+0000 7f0288a59700 1 -- 172.21.15.17:0/718123101 learned_addr learned my addr 172.21.15.17:0/718123101 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:41.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.338+0000 7f027bfff700 1 --2- 172.21.15.17:0/718123101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f028405d240 0x7f0284152ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:41.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.338+0000 7f028925a700 1 --2- 172.21.15.17:0/718123101 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f028413f520 0x7f028413a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:41.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.338+0000 7f0288a59700 1 -- 172.21.15.17:0/718123101 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f028413f520 msgr2=0x7f028413a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:41.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.338+0000 7f0288a59700 1 --2- 172.21.15.17:0/718123101 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f028413f520 0x7f028413a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:41.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.338+0000 7f0288a59700 1 -- 172.21.15.17:0/718123101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f028405d240 msgr2=0x7f0284152ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:41.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.338+0000 7f0288a59700 1 --2- 172.21.15.17:0/718123101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f028405d240 0x7f0284152ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:41.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.338+0000 7f0288a59700 1 -- 172.21.15.17:0/718123101 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f028000b180 con 0x7f028405c960 2022-08-01T19:39:41.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.338+0000 7f027bfff700 1 --2- 172.21.15.17:0/718123101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f028405d240 0x7f0284152ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:39:41.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.338+0000 7f028925a700 1 --2- 172.21.15.17:0/718123101 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f028413f520 0x7f028413a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:39:41.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.338+0000 7f0288a59700 1 --2- 172.21.15.17:0/718123101 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f028405c960 0x7f0284142df0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f0274001800 tx=0x7f02840b1ab0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:41.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.338+0000 7f0279ffb700 1 -- 172.21.15.17:0/718123101 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f028000da60 con 0x7f028405c960 2022-08-01T19:39:41.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.338+0000 7f0279ffb700 1 -- 172.21.15.17:0/718123101 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f028000dbc0 con 0x7f028405c960 2022-08-01T19:39:41.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.338+0000 7f0289a5b700 1 -- 172.21.15.17:0/718123101 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0284064f80 con 0x7f028405c960 2022-08-01T19:39:41.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.338+0000 7f0279ffb700 1 -- 172.21.15.17:0/718123101 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f028001d400 con 0x7f028405c960 2022-08-01T19:39:41.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.338+0000 7f0289a5b700 1 -- 172.21.15.17:0/718123101 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02841533d0 con 0x7f028405c960 2022-08-01T19:39:41.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.338+0000 7f0289a5b700 1 -- 172.21.15.17:0/718123101 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0284065110 con 0x7f028405c960 2022-08-01T19:39:41.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.338+0000 7f0279ffb700 1 -- 172.21.15.17:0/718123101 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0280024030 con 0x7f028405c960 2022-08-01T19:39:41.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.342+0000 7f0279ffb700 1 --2- 172.21.15.17:0/718123101 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0264053a70 0x7f0264055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:41.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.342+0000 7f0279ffb700 1 -- 172.21.15.17:0/718123101 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f028006ca70 con 0x7f028405c960 2022-08-01T19:39:41.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.342+0000 7f027bfff700 1 --2- 172.21.15.17:0/718123101 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0264053a70 0x7f0264055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:41.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.342+0000 7f027bfff700 1 --2- 172.21.15.17:0/718123101 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0264053a70 0x7f0264055f30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f0274008000 tx=0x7f0274009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:41.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.346+0000 7f0279ffb700 1 -- 172.21.15.17:0/718123101 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f02800436d0 con 0x7f028405c960 2022-08-01T19:39:41.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.510+0000 7f0289a5b700 1 -- 172.21.15.17:0/718123101 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02840574e0 con 0x7f028405c960 2022-08-01T19:39:41.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.510+0000 7f0279ffb700 1 -- 172.21.15.17:0/718123101 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+2132 (secure 0 0 0) 0x7f0280022070 con 0x7f028405c960 2022-08-01T19:39:41.515 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:39:41.515 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":31,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5414,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1130955771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1130955771},{"type":"v1","addr":"172.21.15.17:6835","nonce":1130955771}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":28},{"gid":5940,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3691532323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3691532323},{"type":"v1","addr":"172.21.15.17:6833","nonce":3691532323}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":28}],"filesystems":[{"mdsmap":{"epoch":31,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:39:10.932933+0000","modified":"2022-08-01T19:39:41.201205+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-08-01T19:39:41.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.514+0000 7f0289a5b700 1 -- 172.21.15.17:0/718123101 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0264053a70 msgr2=0x7f0264055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:41.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.514+0000 7f0289a5b700 1 --2- 172.21.15.17:0/718123101 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0264053a70 0x7f0264055f30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f0274008000 tx=0x7f0274009040).stop 2022-08-01T19:39:41.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.514+0000 7f0289a5b700 1 -- 172.21.15.17:0/718123101 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f028405c960 msgr2=0x7f0284142df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:41.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.514+0000 7f0289a5b700 1 --2- 172.21.15.17:0/718123101 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f028405c960 0x7f0284142df0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f0274001800 tx=0x7f02840b1ab0).stop 2022-08-01T19:39:41.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.514+0000 7f0289a5b700 1 -- 172.21.15.17:0/718123101 shutdown_connections 2022-08-01T19:39:41.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.514+0000 7f0289a5b700 1 --2- 172.21.15.17:0/718123101 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0264053a70 0x7f0264055f30 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:41.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.514+0000 7f0289a5b700 1 --2- 172.21.15.17:0/718123101 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f028413f520 0x7f028413a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:41.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.514+0000 7f0289a5b700 1 --2- 172.21.15.17:0/718123101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f028405d240 0x7f0284152ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:41.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.514+0000 7f0289a5b700 1 --2- 172.21.15.17:0/718123101 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f028405c960 0x7f0284142df0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:41.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.514+0000 7f0289a5b700 1 -- 172.21.15.17:0/718123101 >> 172.21.15.17:0/718123101 conn(0x7f0284060310 msgr2=0x7f028413c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:41.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.514+0000 7f0289a5b700 1 -- 172.21.15.17:0/718123101 shutdown_connections 2022-08-01T19:39:41.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.514+0000 7f0289a5b700 1 -- 172.21.15.17:0/718123101 wait complete. 2022-08-01T19:39:41.525 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 31 2022-08-01T19:39:41.532 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-08-01T19:39:41.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.642+0000 7f279aa5a700 1 Processor -- start 2022-08-01T19:39:41.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.642+0000 7f279aa5a700 1 -- start start 2022-08-01T19:39:41.646 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.642+0000 7f279aa5a700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2794146ae0 0x7f27941403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:41.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.642+0000 7f279aa5a700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f27940b4310 0x7f27941408b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:41.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.642+0000 7f279aa5a700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2794143cf0 con 0x7f27940b4820 2022-08-01T19:39:41.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.642+0000 7f279aa5a700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27941476a0 con 0x7f2794146ae0 2022-08-01T19:39:41.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.642+0000 7f279aa5a700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2794143f90 con 0x7f27940b4310 2022-08-01T19:39:41.648 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.642+0000 7f2793fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2794146ae0 0x7f27941403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:41.648 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.642+0000 7f2798ff7700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f27940b4820 0x7f27940b5060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60618/0 (socket says 172.21.15.17:60618) 2022-08-01T19:39:41.648 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.642+0000 7f2798ff7700 1 -- 172.21.15.17:0/3291004872 learned_addr learned my addr 172.21.15.17:0/3291004872 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:41.648 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.642+0000 7f27937fe700 1 --2- 172.21.15.17:0/3291004872 >> v2:172.21.15.77:3300/0 conn(0x7f27940b4310 0x7f27941408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:41.648 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.642+0000 7f2792ffd700 1 -- 172.21.15.17:0/3291004872 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1916063648 0 0) 0x7f2794143cf0 con 0x7f27940b4820 2022-08-01T19:39:41.649 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.642+0000 7f2792ffd700 1 -- 172.21.15.17:0/3291004872 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2780003260 con 0x7f27940b4820 2022-08-01T19:39:41.649 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.642+0000 7f2793fff700 1 -- 172.21.15.17:0/3291004872 >> v2:172.21.15.77:3300/0 conn(0x7f27940b4310 msgr2=0x7f27941408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:41.649 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.642+0000 7f2793fff700 1 --2- 172.21.15.17:0/3291004872 >> v2:172.21.15.77:3300/0 conn(0x7f27940b4310 0x7f27941408b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:41.649 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.642+0000 7f2793fff700 1 -- 172.21.15.17:0/3291004872 >> v1:172.21.15.17:6789/0 conn(0x7f27940b4820 legacy=0x7f27940b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:41.650 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.642+0000 7f2793fff700 1 -- 172.21.15.17:0/3291004872 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27940b0e20 con 0x7f2794146ae0 2022-08-01T19:39:41.650 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.642+0000 7f2793fff700 1 --2- 172.21.15.17:0/3291004872 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2794146ae0 0x7f27941403c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f2788002f70 tx=0x7f2788006df0).ready entity=mon.0 client_cookie=5efc51281278973d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:41.650 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.642+0000 7f2792ffd700 1 -- 172.21.15.17:0/3291004872 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f278800d4f0 con 0x7f2794146ae0 2022-08-01T19:39:41.650 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.642+0000 7f2792ffd700 1 -- 172.21.15.17:0/3291004872 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2788000e90 con 0x7f2794146ae0 2022-08-01T19:39:41.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.642+0000 7f2792ffd700 1 -- 172.21.15.17:0/3291004872 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f278800d840 con 0x7f2794146ae0 2022-08-01T19:39:41.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.642+0000 7f279aa5a700 1 -- 172.21.15.17:0/3291004872 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2794146ae0 msgr2=0x7f27941403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:41.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.642+0000 7f279aa5a700 1 --2- 172.21.15.17:0/3291004872 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2794146ae0 0x7f27941403c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f2788002f70 tx=0x7f2788006df0).stop 2022-08-01T19:39:41.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.642+0000 7f279aa5a700 1 -- 172.21.15.17:0/3291004872 shutdown_connections 2022-08-01T19:39:41.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.642+0000 7f279aa5a700 1 --2- 172.21.15.17:0/3291004872 >> v2:172.21.15.77:3300/0 conn(0x7f27940b4310 0x7f27941408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:41.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.642+0000 7f279aa5a700 1 --2- 172.21.15.17:0/3291004872 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2794146ae0 0x7f27941403c0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:41.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.642+0000 7f279aa5a700 1 -- 172.21.15.17:0/3291004872 >> 172.21.15.17:0/3291004872 conn(0x7f279405c620 msgr2=0x7f279405ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:41.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.642+0000 7f279aa5a700 1 -- 172.21.15.17:0/3291004872 shutdown_connections 2022-08-01T19:39:41.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.642+0000 7f279aa5a700 1 -- 172.21.15.17:0/3291004872 wait complete. 2022-08-01T19:39:41.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.646+0000 7f279aa5a700 1 Processor -- start 2022-08-01T19:39:41.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.646+0000 7f279aa5a700 1 -- start start 2022-08-01T19:39:41.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.646+0000 7f279aa5a700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f27940b4310 0x7f2794152e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:41.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.646+0000 7f279aa5a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27940b4820 0x7f27940b9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:41.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.646+0000 7f279aa5a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2794146ae0 0x7f27940b8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:41.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.646+0000 7f279aa5a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27940ba0c0 con 0x7f2794146ae0 2022-08-01T19:39:41.660 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.646+0000 7f279aa5a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2794152450 con 0x7f27940b4820 2022-08-01T19:39:41.660 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.646+0000 7f279aa5a700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27941526f0 con 0x7f27940b4310 2022-08-01T19:39:41.660 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.646+0000 7f2798ff7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2794146ae0 0x7f27940b8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:41.660 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.646+0000 7f2798ff7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2794146ae0 0x7f27940b8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40198/0 (socket says 172.21.15.17:40198) 2022-08-01T19:39:41.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.646+0000 7f2798ff7700 1 -- 172.21.15.17:0/3501606943 learned_addr learned my addr 172.21.15.17:0/3501606943 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:41.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.646+0000 7f2793fff700 1 --2- 172.21.15.17:0/3501606943 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f27940b4310 0x7f2794152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:41.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.646+0000 7f27937fe700 1 --2- 172.21.15.17:0/3501606943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27940b4820 0x7f27940b9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:41.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.646+0000 7f2798ff7700 1 -- 172.21.15.17:0/3501606943 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f27940b4310 msgr2=0x7f2794152e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:41.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.646+0000 7f2798ff7700 1 --2- 172.21.15.17:0/3501606943 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f27940b4310 0x7f2794152e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:41.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.646+0000 7f2798ff7700 1 -- 172.21.15.17:0/3501606943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27940b4820 msgr2=0x7f27940b9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:41.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.646+0000 7f2798ff7700 1 --2- 172.21.15.17:0/3501606943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27940b4820 0x7f27940b9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:41.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.646+0000 7f2798ff7700 1 -- 172.21.15.17:0/3501606943 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f278800b180 con 0x7f2794146ae0 2022-08-01T19:39:41.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.646+0000 7f27937fe700 1 --2- 172.21.15.17:0/3501606943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27940b4820 0x7f27940b9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:39:41.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.646+0000 7f2798ff7700 1 --2- 172.21.15.17:0/3501606943 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2794146ae0 0x7f27940b8640 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7f278c001490 tx=0x7f278c007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:41.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.646+0000 7f27917fa700 1 -- 172.21.15.17:0/3501606943 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f278c005ba0 con 0x7f2794146ae0 2022-08-01T19:39:41.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.650+0000 7f27917fa700 1 -- 172.21.15.17:0/3501606943 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f278c005d00 con 0x7f2794146ae0 2022-08-01T19:39:41.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.650+0000 7f279aa5a700 1 -- 172.21.15.17:0/3501606943 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2794152990 con 0x7f2794146ae0 2022-08-01T19:39:41.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.650+0000 7f27917fa700 1 -- 172.21.15.17:0/3501606943 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f278c0072c0 con 0x7f2794146ae0 2022-08-01T19:39:41.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.650+0000 7f279aa5a700 1 -- 172.21.15.17:0/3501606943 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2794145f50 con 0x7f2794146ae0 2022-08-01T19:39:41.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.650+0000 7f279aa5a700 1 -- 172.21.15.17:0/3501606943 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2794152b20 con 0x7f2794146ae0 2022-08-01T19:39:41.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.650+0000 7f27917fa700 1 -- 172.21.15.17:0/3501606943 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f278c010070 con 0x7f2794146ae0 2022-08-01T19:39:41.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.650+0000 7f27917fa700 1 --2- 172.21.15.17:0/3501606943 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2774053a70 0x7f2774055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:41.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.650+0000 7f27917fa700 1 -- 172.21.15.17:0/3501606943 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f278c06b9b0 con 0x7f2794146ae0 2022-08-01T19:39:41.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.650+0000 7f2793fff700 1 --2- 172.21.15.17:0/3501606943 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2774053a70 0x7f2774055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:41.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.650+0000 7f2793fff700 1 --2- 172.21.15.17:0/3501606943 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2774053a70 0x7f2774055f30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f278801a000 tx=0x7f2788014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:41.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.658+0000 7f27917fa700 1 -- 172.21.15.17:0/3501606943 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f278c042680 con 0x7f2794146ae0 2022-08-01T19:39:41.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:41.822+0000 7f279aa5a700 1 -- 172.21.15.17:0/3501606943 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f27940574e0 con 0x7f2794146ae0 2022-08-01T19:39:42.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.222+0000 7f27917fa700 1 -- 172.21.15.17:0/3501606943 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) v1 ==== 104+0+0 (secure 0 0 0) 0x7f278c018020 con 0x7f2794146ae0 2022-08-01T19:39:42.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.226+0000 7f279aa5a700 1 -- 172.21.15.17:0/3501606943 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2774053a70 msgr2=0x7f2774055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:42.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.226+0000 7f279aa5a700 1 --2- 172.21.15.17:0/3501606943 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2774053a70 0x7f2774055f30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f278801a000 tx=0x7f2788014000).stop 2022-08-01T19:39:42.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.226+0000 7f279aa5a700 1 -- 172.21.15.17:0/3501606943 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2794146ae0 msgr2=0x7f27940b8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:42.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.226+0000 7f279aa5a700 1 --2- 172.21.15.17:0/3501606943 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2794146ae0 0x7f27940b8640 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7f278c001490 tx=0x7f278c007e00).stop 2022-08-01T19:39:42.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.226+0000 7f279aa5a700 1 -- 172.21.15.17:0/3501606943 shutdown_connections 2022-08-01T19:39:42.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.226+0000 7f279aa5a700 1 --2- 172.21.15.17:0/3501606943 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2774053a70 0x7f2774055f30 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:42.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.226+0000 7f279aa5a700 1 --2- 172.21.15.17:0/3501606943 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2794146ae0 0x7f27940b8640 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:42.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.226+0000 7f279aa5a700 1 --2- 172.21.15.17:0/3501606943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27940b4820 0x7f27940b9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:42.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.226+0000 7f279aa5a700 1 --2- 172.21.15.17:0/3501606943 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f27940b4310 0x7f2794152e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:42.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.226+0000 7f279aa5a700 1 -- 172.21.15.17:0/3501606943 >> 172.21.15.17:0/3501606943 conn(0x7f279405c620 msgr2=0x7f27940616e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:42.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.226+0000 7f279aa5a700 1 -- 172.21.15.17:0/3501606943 shutdown_connections 2022-08-01T19:39:42.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.226+0000 7f279aa5a700 1 -- 172.21.15.17:0/3501606943 wait complete. 2022-08-01T19:39:42.242 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-08-01T19:39:42.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.362+0000 7f5776c38700 1 Processor -- start 2022-08-01T19:39:42.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.362+0000 7f5776c38700 1 -- start start 2022-08-01T19:39:42.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.366+0000 7f5776c38700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f57700b4550 0x7f57700b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:42.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.366+0000 7f5776c38700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f57700b6af0 0x7f577012bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:42.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.366+0000 7f5776c38700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5770144240 con 0x7f57700b6710 2022-08-01T19:39:42.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.366+0000 7f5776c38700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5770143d00 con 0x7f57700b4550 2022-08-01T19:39:42.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.366+0000 7f5776c38700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5770143fa0 con 0x7f57700b6af0 2022-08-01T19:39:42.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.366+0000 7f576ffff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f57700b4550 0x7f57700b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:42.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.366+0000 7f57749d4700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f57700b6710 0x7f5770061ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60628/0 (socket says 172.21.15.17:60628) 2022-08-01T19:39:42.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.366+0000 7f576ffff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f57700b4550 0x7f57700b5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40204/0 (socket says 172.21.15.17:40204) 2022-08-01T19:39:42.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.366+0000 7f57749d4700 1 -- 172.21.15.17:0/585673296 learned_addr learned my addr 172.21.15.17:0/585673296 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:42.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.366+0000 7f57751d5700 1 --2- 172.21.15.17:0/585673296 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f57700b6af0 0x7f577012bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:42.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.366+0000 7f576ffff700 1 -- 172.21.15.17:0/585673296 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f57700b6af0 msgr2=0x7f577012bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:42.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.366+0000 7f576ffff700 1 --2- 172.21.15.17:0/585673296 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f57700b6af0 0x7f577012bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:42.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.366+0000 7f576ffff700 1 -- 172.21.15.17:0/585673296 >> v1:172.21.15.17:6789/0 conn(0x7f57700b6710 legacy=0x7f5770061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:42.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.366+0000 7f576f7fe700 1 -- 172.21.15.17:0/585673296 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3081963129 0 0) 0x7f5770144240 con 0x7f57700b6710 2022-08-01T19:39:42.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.366+0000 7f576ffff700 1 -- 172.21.15.17:0/585673296 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5770061780 con 0x7f57700b4550 2022-08-01T19:39:42.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.366+0000 7f576ffff700 1 --2- 172.21.15.17:0/585673296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57700b4550 0x7f57700b5d80 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f57701432b0 tx=0x7f5758006df0).ready entity=mon.0 client_cookie=77cc083a89321389 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:42.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.366+0000 7f576f7fe700 1 -- 172.21.15.17:0/585673296 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f575800d570 con 0x7f57700b4550 2022-08-01T19:39:42.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.366+0000 7f576f7fe700 1 -- 172.21.15.17:0/585673296 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5758000e90 con 0x7f57700b4550 2022-08-01T19:39:42.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.366+0000 7f576f7fe700 1 -- 172.21.15.17:0/585673296 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f575800d8c0 con 0x7f57700b4550 2022-08-01T19:39:42.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.366+0000 7f5776c38700 1 -- 172.21.15.17:0/585673296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57700b4550 msgr2=0x7f57700b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:42.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.366+0000 7f5776c38700 1 --2- 172.21.15.17:0/585673296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57700b4550 0x7f57700b5d80 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f57701432b0 tx=0x7f5758006df0).stop 2022-08-01T19:39:42.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.366+0000 7f5776c38700 1 -- 172.21.15.17:0/585673296 shutdown_connections 2022-08-01T19:39:42.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.366+0000 7f5776c38700 1 --2- 172.21.15.17:0/585673296 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f57700b6af0 0x7f577012bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:42.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.366+0000 7f5776c38700 1 --2- 172.21.15.17:0/585673296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57700b4550 0x7f57700b5d80 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:42.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.366+0000 7f5776c38700 1 -- 172.21.15.17:0/585673296 >> 172.21.15.17:0/585673296 conn(0x7f577005c620 msgr2=0x7f577005ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:42.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.366+0000 7f5776c38700 1 -- 172.21.15.17:0/585673296 shutdown_connections 2022-08-01T19:39:42.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.366+0000 7f5776c38700 1 -- 172.21.15.17:0/585673296 wait complete. 2022-08-01T19:39:42.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.370+0000 7f5776c38700 1 Processor -- start 2022-08-01T19:39:42.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.370+0000 7f5776c38700 1 -- start start 2022-08-01T19:39:42.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.370+0000 7f5776c38700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57700b4550 0x7f57701449c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:42.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.370+0000 7f5776c38700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f57700b6710 0x7f57701251e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:42.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.370+0000 7f5776c38700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57700b6af0 0x7f57701256d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:42.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.370+0000 7f5776c38700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5770145440 con 0x7f57700b6af0 2022-08-01T19:39:42.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.370+0000 7f5776c38700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5770126c50 con 0x7f57700b4550 2022-08-01T19:39:42.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.370+0000 7f5776c38700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57700b3950 con 0x7f57700b6710 2022-08-01T19:39:42.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.370+0000 7f57751d5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57700b6af0 0x7f57701256d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:42.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.370+0000 7f57751d5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57700b6af0 0x7f57701256d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40212/0 (socket says 172.21.15.17:40212) 2022-08-01T19:39:42.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.370+0000 7f57751d5700 1 -- 172.21.15.17:0/2605860858 learned_addr learned my addr 172.21.15.17:0/2605860858 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:42.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.370+0000 7f57749d4700 1 --2- 172.21.15.17:0/2605860858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57700b4550 0x7f57701449c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:42.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.370+0000 7f576ffff700 1 --2- 172.21.15.17:0/2605860858 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f57700b6710 0x7f57701251e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:42.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.370+0000 7f57751d5700 1 -- 172.21.15.17:0/2605860858 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f57700b6710 msgr2=0x7f57701251e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:42.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.370+0000 7f57751d5700 1 --2- 172.21.15.17:0/2605860858 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f57700b6710 0x7f57701251e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:42.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.370+0000 7f57751d5700 1 -- 172.21.15.17:0/2605860858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57700b4550 msgr2=0x7f57701449c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:42.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.370+0000 7f57751d5700 1 --2- 172.21.15.17:0/2605860858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57700b4550 0x7f57701449c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:42.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.370+0000 7f57751d5700 1 -- 172.21.15.17:0/2605860858 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f575800b180 con 0x7f57700b6af0 2022-08-01T19:39:42.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.370+0000 7f576ffff700 1 --2- 172.21.15.17:0/2605860858 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f57700b6710 0x7f57701251e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:39:42.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.370+0000 7f57751d5700 1 --2- 172.21.15.17:0/2605860858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57700b6af0 0x7f57701256d0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7f5768006e00 tx=0x7f57680091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:42.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.370+0000 7f576dffb700 1 -- 172.21.15.17:0/2605860858 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f576800c640 con 0x7f57700b6af0 2022-08-01T19:39:42.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.370+0000 7f576dffb700 1 -- 172.21.15.17:0/2605860858 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f576800c7a0 con 0x7f57700b6af0 2022-08-01T19:39:42.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.370+0000 7f5776c38700 1 -- 172.21.15.17:0/2605860858 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57700b3bf0 con 0x7f57700b6af0 2022-08-01T19:39:42.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.370+0000 7f576dffb700 1 -- 172.21.15.17:0/2605860858 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f576800cb00 con 0x7f57700b6af0 2022-08-01T19:39:42.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.370+0000 7f5776c38700 1 -- 172.21.15.17:0/2605860858 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57700b34e0 con 0x7f57700b6af0 2022-08-01T19:39:42.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.374+0000 7f5776c38700 1 -- 172.21.15.17:0/2605860858 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57700b37b0 con 0x7f57700b6af0 2022-08-01T19:39:42.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.374+0000 7f576dffb700 1 -- 172.21.15.17:0/2605860858 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f576800cc60 con 0x7f57700b6af0 2022-08-01T19:39:42.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.374+0000 7f576dffb700 1 --2- 172.21.15.17:0/2605860858 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5754053b30 0x7f5754055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:42.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.374+0000 7f576dffb700 1 -- 172.21.15.17:0/2605860858 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f576806c390 con 0x7f57700b6af0 2022-08-01T19:39:42.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.374+0000 7f57749d4700 1 --2- 172.21.15.17:0/2605860858 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5754053b30 0x7f5754055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:42.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.378+0000 7f57749d4700 1 --2- 172.21.15.17:0/2605860858 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5754053b30 0x7f5754055ff0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f576000d000 tx=0x7f5760007a00).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:42.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.382+0000 7f576dffb700 1 -- 172.21.15.17:0/2605860858 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f57700b37b0 con 0x7f57700b6af0 2022-08-01T19:39:42.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:42.510+0000 7f5776c38700 1 -- 172.21.15.17:0/2605860858 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f57700b37b0 con 0x7f57700b6af0 2022-08-01T19:39:43.234 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.234+0000 7f576dffb700 1 -- 172.21.15.17:0/2605860858 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v68) v1 ==== 185+0+0 (secure 0 0 0) 0x7f57700b37b0 con 0x7f57700b6af0 2022-08-01T19:39:43.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.238+0000 7f5776c38700 1 -- 172.21.15.17:0/2605860858 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5754053b30 msgr2=0x7f5754055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:43.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.238+0000 7f5776c38700 1 --2- 172.21.15.17:0/2605860858 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5754053b30 0x7f5754055ff0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f576000d000 tx=0x7f5760007a00).stop 2022-08-01T19:39:43.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.238+0000 7f5776c38700 1 -- 172.21.15.17:0/2605860858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57700b6af0 msgr2=0x7f57701256d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:43.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.238+0000 7f5776c38700 1 --2- 172.21.15.17:0/2605860858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57700b6af0 0x7f57701256d0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7f5768006e00 tx=0x7f57680091d0).stop 2022-08-01T19:39:43.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.238+0000 7f5776c38700 1 -- 172.21.15.17:0/2605860858 shutdown_connections 2022-08-01T19:39:43.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.238+0000 7f5776c38700 1 --2- 172.21.15.17:0/2605860858 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5754053b30 0x7f5754055ff0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:43.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.238+0000 7f5776c38700 1 --2- 172.21.15.17:0/2605860858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57700b6af0 0x7f57701256d0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:43.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.238+0000 7f5776c38700 1 --2- 172.21.15.17:0/2605860858 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f57700b6710 0x7f57701251e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:43.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.238+0000 7f5776c38700 1 --2- 172.21.15.17:0/2605860858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57700b4550 0x7f57701449c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:43.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.238+0000 7f5776c38700 1 -- 172.21.15.17:0/2605860858 >> 172.21.15.17:0/2605860858 conn(0x7f577005c620 msgr2=0x7f5770141120 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:43.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.238+0000 7f5776c38700 1 -- 172.21.15.17:0/2605860858 shutdown_connections 2022-08-01T19:39:43.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.238+0000 7f5776c38700 1 -- 172.21.15.17:0/2605860858 wait complete. 2022-08-01T19:39:43.242 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' removed 2022-08-01T19:39:43.255 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-08-01T19:39:43.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.370+0000 7f9143078700 1 Processor -- start 2022-08-01T19:39:43.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.370+0000 7f9143078700 1 -- start start 2022-08-01T19:39:43.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.374+0000 7f9143078700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f913c0b31d0 0x7f913c066020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:43.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.374+0000 7f9143078700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f913c0a9da0 0x7f913c060950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:43.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.374+0000 7f9143078700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f913c067030 con 0x7f913c1434b0 2022-08-01T19:39:43.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.374+0000 7f9143078700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f913c064a70 con 0x7f913c0a9da0 2022-08-01T19:39:43.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.374+0000 7f9143078700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913c061f60 con 0x7f913c0b31d0 2022-08-01T19:39:43.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.374+0000 7f9141875700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f913c0a9da0 0x7f913c060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:43.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.374+0000 7f9142076700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f913c0b31d0 0x7f913c066020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:43.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.374+0000 7f9141875700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f913c0a9da0 0x7f913c060950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:38138/0 (socket says 172.21.15.17:38138) 2022-08-01T19:39:43.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.374+0000 7f9141875700 1 -- 172.21.15.17:0/66694255 learned_addr learned my addr 172.21.15.17:0/66694255 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:43.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.374+0000 7f9141875700 1 -- 172.21.15.17:0/66694255 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f913c0b31d0 msgr2=0x7f913c066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:43.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.374+0000 7f9141875700 1 --2- 172.21.15.17:0/66694255 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f913c0b31d0 0x7f913c066020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:43.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.374+0000 7f9141875700 1 -- 172.21.15.17:0/66694255 >> v1:172.21.15.77:6789/0 conn(0x7f913c1434b0 legacy=0x7f913c143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:43.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.374+0000 7f9141875700 1 -- 172.21.15.17:0/66694255 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913c0b35b0 con 0x7f913c0a9da0 2022-08-01T19:39:43.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.374+0000 7f9141875700 1 --2- 172.21.15.17:0/66694255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f913c0a9da0 0x7f913c060950 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f912c002f70 tx=0x7f912c006eb0).ready entity=mon.1 client_cookie=a71ab4b7f8351dc9 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:43.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.374+0000 7f9141074700 1 -- 172.21.15.17:0/66694255 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f912c00f970 con 0x7f913c0a9da0 2022-08-01T19:39:43.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.374+0000 7f9141074700 1 -- 172.21.15.17:0/66694255 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912c000e90 con 0x7f913c0a9da0 2022-08-01T19:39:43.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.374+0000 7f9141074700 1 -- 172.21.15.17:0/66694255 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f912c00fd00 con 0x7f913c0a9da0 2022-08-01T19:39:43.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.374+0000 7f9143078700 1 -- 172.21.15.17:0/66694255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f913c0a9da0 msgr2=0x7f913c060950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:43.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.374+0000 7f9143078700 1 --2- 172.21.15.17:0/66694255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f913c0a9da0 0x7f913c060950 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f912c002f70 tx=0x7f912c006eb0).stop 2022-08-01T19:39:43.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.374+0000 7f9143078700 1 -- 172.21.15.17:0/66694255 shutdown_connections 2022-08-01T19:39:43.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.374+0000 7f9143078700 1 --2- 172.21.15.17:0/66694255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f913c0a9da0 0x7f913c060950 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:43.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.374+0000 7f9143078700 1 --2- 172.21.15.17:0/66694255 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f913c0b31d0 0x7f913c066020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:43.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.374+0000 7f9143078700 1 -- 172.21.15.17:0/66694255 >> 172.21.15.17:0/66694255 conn(0x7f913c063870 msgr2=0x7f913c063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:43.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.374+0000 7f9143078700 1 -- 172.21.15.17:0/66694255 shutdown_connections 2022-08-01T19:39:43.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.374+0000 7f9143078700 1 -- 172.21.15.17:0/66694255 wait complete. 2022-08-01T19:39:43.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.378+0000 7f9143078700 1 Processor -- start 2022-08-01T19:39:43.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.378+0000 7f9143078700 1 -- start start 2022-08-01T19:39:43.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.378+0000 7f9143078700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f913c0a9da0 0x7f913c13f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:43.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.378+0000 7f9143078700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f913c0b31d0 0x7f913c13fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:43.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.378+0000 7f9143078700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f913c1434b0 0x7f913c140300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:43.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.378+0000 7f9143078700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913c13dd70 con 0x7f913c0a9da0 2022-08-01T19:39:43.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.378+0000 7f9143078700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913c0b5b40 con 0x7f913c1434b0 2022-08-01T19:39:43.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.378+0000 7f9143078700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913c13dfa0 con 0x7f913c0b31d0 2022-08-01T19:39:43.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.378+0000 7f9142076700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f913c0a9da0 0x7f913c13f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:43.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.378+0000 7f9142076700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f913c0a9da0 0x7f913c13f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40222/0 (socket says 172.21.15.17:40222) 2022-08-01T19:39:43.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.378+0000 7f9142076700 1 -- 172.21.15.17:0/2705413002 learned_addr learned my addr 172.21.15.17:0/2705413002 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:43.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.378+0000 7f9142877700 1 --2- 172.21.15.17:0/2705413002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f913c1434b0 0x7f913c140300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:43.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.378+0000 7f9141875700 1 --2- 172.21.15.17:0/2705413002 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f913c0b31d0 0x7f913c13fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:43.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.378+0000 7f9142076700 1 -- 172.21.15.17:0/2705413002 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f913c0b31d0 msgr2=0x7f913c13fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:43.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.378+0000 7f9142076700 1 --2- 172.21.15.17:0/2705413002 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f913c0b31d0 0x7f913c13fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:43.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.378+0000 7f9142076700 1 -- 172.21.15.17:0/2705413002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f913c1434b0 msgr2=0x7f913c140300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:43.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.378+0000 7f9142076700 1 --2- 172.21.15.17:0/2705413002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f913c1434b0 0x7f913c140300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:43.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.382+0000 7f9142076700 1 -- 172.21.15.17:0/2705413002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912c00b1e0 con 0x7f913c0a9da0 2022-08-01T19:39:43.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.382+0000 7f9142076700 1 --2- 172.21.15.17:0/2705413002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f913c0a9da0 0x7f913c13f580 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f913800c000 tx=0x7f9138006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:43.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.382+0000 7f912b7fe700 1 -- 172.21.15.17:0/2705413002 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9138010780 con 0x7f913c0a9da0 2022-08-01T19:39:43.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.382+0000 7f912b7fe700 1 -- 172.21.15.17:0/2705413002 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91380108e0 con 0x7f913c0a9da0 2022-08-01T19:39:43.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.382+0000 7f9143078700 1 -- 172.21.15.17:0/2705413002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f913c13e240 con 0x7f913c0a9da0 2022-08-01T19:39:43.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.382+0000 7f912b7fe700 1 -- 172.21.15.17:0/2705413002 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9138010c40 con 0x7f913c0a9da0 2022-08-01T19:39:43.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.382+0000 7f9143078700 1 -- 172.21.15.17:0/2705413002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f913c1407f0 con 0x7f913c0a9da0 2022-08-01T19:39:43.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.382+0000 7f9143078700 1 -- 172.21.15.17:0/2705413002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f913c13e420 con 0x7f913c0a9da0 2022-08-01T19:39:43.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.382+0000 7f912b7fe700 1 -- 172.21.15.17:0/2705413002 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f913801c460 con 0x7f913c0a9da0 2022-08-01T19:39:43.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.382+0000 7f912b7fe700 1 --2- 172.21.15.17:0/2705413002 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f911c053a10 0x7f911c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:43.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.382+0000 7f912b7fe700 1 -- 172.21.15.17:0/2705413002 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f913806c270 con 0x7f913c0a9da0 2022-08-01T19:39:43.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.386+0000 7f9141875700 1 --2- 172.21.15.17:0/2705413002 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f911c053a10 0x7f911c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:43.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.386+0000 7f9141875700 1 --2- 172.21.15.17:0/2705413002 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f911c053a10 0x7f911c055ed0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f912c01e000 tx=0x7f912c017000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:43.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.390+0000 7f912b7fe700 1 -- 172.21.15.17:0/2705413002 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f913c13e420 con 0x7f913c0a9da0 2022-08-01T19:39:43.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:43.518+0000 7f9143078700 1 -- 172.21.15.17:0/2705413002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f913c13e3d0 con 0x7f913c0a9da0 2022-08-01T19:39:44.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.246+0000 7f912b7fe700 1 -- 172.21.15.17:0/2705413002 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v69) v1 ==== 173+0+0 (secure 0 0 0) 0x7f9138043130 con 0x7f913c0a9da0 2022-08-01T19:39:44.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.250+0000 7f9143078700 1 -- 172.21.15.17:0/2705413002 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f911c053a10 msgr2=0x7f911c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:44.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.250+0000 7f9143078700 1 --2- 172.21.15.17:0/2705413002 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f911c053a10 0x7f911c055ed0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f912c01e000 tx=0x7f912c017000).stop 2022-08-01T19:39:44.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.254+0000 7f9143078700 1 -- 172.21.15.17:0/2705413002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f913c0a9da0 msgr2=0x7f913c13f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:44.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.254+0000 7f9143078700 1 --2- 172.21.15.17:0/2705413002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f913c0a9da0 0x7f913c13f580 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f913800c000 tx=0x7f9138006df0).stop 2022-08-01T19:39:44.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.254+0000 7f9143078700 1 -- 172.21.15.17:0/2705413002 shutdown_connections 2022-08-01T19:39:44.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.254+0000 7f9143078700 1 --2- 172.21.15.17:0/2705413002 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f911c053a10 0x7f911c055ed0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:44.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.254+0000 7f9143078700 1 --2- 172.21.15.17:0/2705413002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f913c1434b0 0x7f913c140300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:44.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.254+0000 7f9143078700 1 --2- 172.21.15.17:0/2705413002 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f913c0b31d0 0x7f913c13fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:44.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.254+0000 7f9143078700 1 --2- 172.21.15.17:0/2705413002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f913c0a9da0 0x7f913c13f580 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:44.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.254+0000 7f9143078700 1 -- 172.21.15.17:0/2705413002 >> 172.21.15.17:0/2705413002 conn(0x7f913c063870 msgr2=0x7f913c143b30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:44.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.254+0000 7f9143078700 1 -- 172.21.15.17:0/2705413002 shutdown_connections 2022-08-01T19:39:44.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.254+0000 7f9143078700 1 -- 172.21.15.17:0/2705413002 wait complete. 2022-08-01T19:39:44.258 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' removed 2022-08-01T19:39:44.266 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-08-01T19:39:44.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.382+0000 7f51f1a05700 1 Processor -- start 2022-08-01T19:39:44.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.382+0000 7f51f1a05700 1 -- start start 2022-08-01T19:39:44.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.382+0000 7f51f1a05700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f51ec0614b0 0x7f51ec061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:44.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.382+0000 7f51f1a05700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f51ec0ad750 0x7f51ec0b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:44.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.382+0000 7f51f1a05700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51ec0aee80 con 0x7f51ec1405d0 2022-08-01T19:39:44.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.382+0000 7f51f1a05700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51ec0b70b0 con 0x7f51ec0614b0 2022-08-01T19:39:44.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.382+0000 7f51f1a05700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51ec0af0b0 con 0x7f51ec0ad750 2022-08-01T19:39:44.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.382+0000 7f51f0a03700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f51ec0614b0 0x7f51ec061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:44.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.386+0000 7f51f0a03700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f51ec0614b0 0x7f51ec061890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40232/0 (socket says 172.21.15.17:40232) 2022-08-01T19:39:44.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.386+0000 7f51f0a03700 1 -- 172.21.15.17:0/321481294 learned_addr learned my addr 172.21.15.17:0/321481294 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:44.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.386+0000 7f51ebfff700 1 --2- 172.21.15.17:0/321481294 >> v2:172.21.15.77:3300/0 conn(0x7f51ec0ad750 0x7f51ec0b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:44.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.386+0000 7f51f0a03700 1 -- 172.21.15.17:0/321481294 >> v2:172.21.15.77:3300/0 conn(0x7f51ec0ad750 msgr2=0x7f51ec0b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:44.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.386+0000 7f51f0a03700 1 --2- 172.21.15.17:0/321481294 >> v2:172.21.15.77:3300/0 conn(0x7f51ec0ad750 0x7f51ec0b9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:44.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.386+0000 7f51f0a03700 1 -- 172.21.15.17:0/321481294 >> v1:172.21.15.77:6789/0 conn(0x7f51ec1405d0 legacy=0x7f51ec1466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:44.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.386+0000 7f51f0a03700 1 -- 172.21.15.17:0/321481294 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ec0af350 con 0x7f51ec0614b0 2022-08-01T19:39:44.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.386+0000 7f51f0a03700 1 --2- 172.21.15.17:0/321481294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51ec0614b0 0x7f51ec061890 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f51ec0aed30 tx=0x7f51e0006eb0).ready entity=mon.0 client_cookie=17edc1129486a10b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:44.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.386+0000 7f51eb7fe700 1 -- 172.21.15.17:0/321481294 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51e000d610 con 0x7f51ec0614b0 2022-08-01T19:39:44.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.386+0000 7f51eb7fe700 1 -- 172.21.15.17:0/321481294 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e0000e90 con 0x7f51ec0614b0 2022-08-01T19:39:44.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.386+0000 7f51eb7fe700 1 -- 172.21.15.17:0/321481294 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51e000d9a0 con 0x7f51ec0614b0 2022-08-01T19:39:44.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.386+0000 7f51f1a05700 1 -- 172.21.15.17:0/321481294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51ec0614b0 msgr2=0x7f51ec061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:44.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.386+0000 7f51f1a05700 1 --2- 172.21.15.17:0/321481294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51ec0614b0 0x7f51ec061890 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f51ec0aed30 tx=0x7f51e0006eb0).stop 2022-08-01T19:39:44.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.386+0000 7f51f1a05700 1 -- 172.21.15.17:0/321481294 shutdown_connections 2022-08-01T19:39:44.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.386+0000 7f51f1a05700 1 --2- 172.21.15.17:0/321481294 >> v2:172.21.15.77:3300/0 conn(0x7f51ec0ad750 0x7f51ec0b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:44.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.386+0000 7f51f1a05700 1 --2- 172.21.15.17:0/321481294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51ec0614b0 0x7f51ec061890 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:44.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.386+0000 7f51f1a05700 1 -- 172.21.15.17:0/321481294 >> 172.21.15.17:0/321481294 conn(0x7f51ec066f70 msgr2=0x7f51ec067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:44.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.386+0000 7f51f1a05700 1 -- 172.21.15.17:0/321481294 shutdown_connections 2022-08-01T19:39:44.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.386+0000 7f51f1a05700 1 -- 172.21.15.17:0/321481294 wait complete. 2022-08-01T19:39:44.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.390+0000 7f51f1a05700 1 Processor -- start 2022-08-01T19:39:44.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.390+0000 7f51f1a05700 1 -- start start 2022-08-01T19:39:44.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.390+0000 7f51f1a05700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f51ec0614b0 0x7f51ec0b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:44.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.390+0000 7f51f1a05700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51ec0ad750 0x7f51ec0a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:44.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.390+0000 7f51f1a05700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51ec1405d0 0x7f51ec155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:44.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.390+0000 7f51f1a05700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec0b38e0 con 0x7f51ec1405d0 2022-08-01T19:39:44.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.390+0000 7f51f1a05700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec1537b0 con 0x7f51ec0ad750 2022-08-01T19:39:44.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.390+0000 7f51f1a05700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec153510 con 0x7f51ec0614b0 2022-08-01T19:39:44.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.390+0000 7f51f1204700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51ec1405d0 0x7f51ec155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:44.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.390+0000 7f51f1204700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51ec1405d0 0x7f51ec155dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40238/0 (socket says 172.21.15.17:40238) 2022-08-01T19:39:44.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.390+0000 7f51f1204700 1 -- 172.21.15.17:0/2001665864 learned_addr learned my addr 172.21.15.17:0/2001665864 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:44.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.390+0000 7f51ebfff700 1 --2- 172.21.15.17:0/2001665864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51ec0ad750 0x7f51ec0a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:44.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.390+0000 7f51f0a03700 1 --2- 172.21.15.17:0/2001665864 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f51ec0614b0 0x7f51ec0b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:44.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.390+0000 7f51ebfff700 1 -- 172.21.15.17:0/2001665864 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f51ec0614b0 msgr2=0x7f51ec0b2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:44.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.390+0000 7f51ebfff700 1 --2- 172.21.15.17:0/2001665864 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f51ec0614b0 0x7f51ec0b2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:44.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.390+0000 7f51ebfff700 1 -- 172.21.15.17:0/2001665864 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51ec1405d0 msgr2=0x7f51ec155dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:44.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.390+0000 7f51ebfff700 1 --2- 172.21.15.17:0/2001665864 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51ec1405d0 0x7f51ec155dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:44.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.390+0000 7f51ebfff700 1 -- 172.21.15.17:0/2001665864 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e000b1e0 con 0x7f51ec0ad750 2022-08-01T19:39:44.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.390+0000 7f51ebfff700 1 --2- 172.21.15.17:0/2001665864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51ec0ad750 0x7f51ec0a1590 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f51ec0a1010 tx=0x7f51dc006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:44.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.390+0000 7f51e9ffb700 1 -- 172.21.15.17:0/2001665864 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51dc011750 con 0x7f51ec0ad750 2022-08-01T19:39:44.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.390+0000 7f51e9ffb700 1 -- 172.21.15.17:0/2001665864 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51dc0118b0 con 0x7f51ec0ad750 2022-08-01T19:39:44.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.390+0000 7f51f1a05700 1 -- 172.21.15.17:0/2001665864 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51ec0a10f0 con 0x7f51ec0ad750 2022-08-01T19:39:44.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.390+0000 7f51e9ffb700 1 -- 172.21.15.17:0/2001665864 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51dc011c10 con 0x7f51ec0ad750 2022-08-01T19:39:44.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.390+0000 7f51f1a05700 1 -- 172.21.15.17:0/2001665864 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51ec0a1a80 con 0x7f51ec0ad750 2022-08-01T19:39:44.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.394+0000 7f51f1a05700 1 -- 172.21.15.17:0/2001665864 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51ec0a1280 con 0x7f51ec0ad750 2022-08-01T19:39:44.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.394+0000 7f51e9ffb700 1 -- 172.21.15.17:0/2001665864 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f51dc011d70 con 0x7f51ec0ad750 2022-08-01T19:39:44.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.394+0000 7f51e9ffb700 1 --2- 172.21.15.17:0/2001665864 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f51cc053950 0x7f51cc055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:44.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.394+0000 7f51e9ffb700 1 -- 172.21.15.17:0/2001665864 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f51dc06bf10 con 0x7f51ec0ad750 2022-08-01T19:39:44.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.394+0000 7f51f0a03700 1 --2- 172.21.15.17:0/2001665864 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f51cc053950 0x7f51cc055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:44.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.394+0000 7f51f0a03700 1 --2- 172.21.15.17:0/2001665864 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f51cc053950 0x7f51cc055e10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f51e001c000 tx=0x7f51e0015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:44.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.402+0000 7f51e9ffb700 1 -- 172.21.15.17:0/2001665864 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f51dc042eb0 con 0x7f51ec0ad750 2022-08-01T19:39:44.530 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.530+0000 7f51f1a05700 1 -- 172.21.15.17:0/2001665864 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f51ec0af5f0 con 0x7f51ec0ad750 2022-08-01T19:39:44.539 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.542+0000 7f51e9ffb700 1 -- 172.21.15.17:0/2001665864 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f51dc0464d0 con 0x7f51ec0ad750 2022-08-01T19:39:44.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.542+0000 7f51f1a05700 1 -- 172.21.15.17:0/2001665864 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f51cc053950 msgr2=0x7f51cc055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:44.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.542+0000 7f51f1a05700 1 --2- 172.21.15.17:0/2001665864 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f51cc053950 0x7f51cc055e10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f51e001c000 tx=0x7f51e0015000).stop 2022-08-01T19:39:44.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.542+0000 7f51f1a05700 1 -- 172.21.15.17:0/2001665864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51ec0ad750 msgr2=0x7f51ec0a1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:44.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.542+0000 7f51f1a05700 1 --2- 172.21.15.17:0/2001665864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51ec0ad750 0x7f51ec0a1590 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f51ec0a1010 tx=0x7f51dc006df0).stop 2022-08-01T19:39:44.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.542+0000 7f51f1a05700 1 -- 172.21.15.17:0/2001665864 shutdown_connections 2022-08-01T19:39:44.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.542+0000 7f51f1a05700 1 --2- 172.21.15.17:0/2001665864 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f51cc053950 0x7f51cc055e10 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:44.547 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.542+0000 7f51f1a05700 1 --2- 172.21.15.17:0/2001665864 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51ec1405d0 0x7f51ec155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:44.547 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.542+0000 7f51f1a05700 1 --2- 172.21.15.17:0/2001665864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51ec0ad750 0x7f51ec0a1590 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:44.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.542+0000 7f51f1a05700 1 --2- 172.21.15.17:0/2001665864 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f51ec0614b0 0x7f51ec0b2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:44.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.542+0000 7f51f1a05700 1 -- 172.21.15.17:0/2001665864 >> 172.21.15.17:0/2001665864 conn(0x7f51ec066f70 msgr2=0x7f51ec144f50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:44.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.542+0000 7f51f1a05700 1 -- 172.21.15.17:0/2001665864 shutdown_connections 2022-08-01T19:39:44.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.542+0000 7f51f1a05700 1 -- 172.21.15.17:0/2001665864 wait complete. 2022-08-01T19:39:44.555 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout' 2022-08-01T19:39:44.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.662+0000 7fd4fddc8700 1 Processor -- start 2022-08-01T19:39:44.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.662+0000 7fd4fddc8700 1 -- start start 2022-08-01T19:39:44.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4fddc8700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fd4f80614b0 0x7fd4f8061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:44.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4fddc8700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd4f81405d0 0x7fd4f8140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:44.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4fddc8700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4f80aee80 con 0x7fd4f80ad750 2022-08-01T19:39:44.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4fddc8700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4f80b70b0 con 0x7fd4f80614b0 2022-08-01T19:39:44.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4fddc8700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f80af0b0 con 0x7fd4f81405d0 2022-08-01T19:39:44.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4f7fff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd4f80ad750 0x7fd4f80b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60672/0 (socket says 172.21.15.17:60672) 2022-08-01T19:39:44.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4f7fff700 1 -- 172.21.15.17:0/647252075 learned_addr learned my addr 172.21.15.17:0/647252075 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:44.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4fd5c7700 1 --2- 172.21.15.17:0/647252075 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd4f81405d0 0x7fd4f8140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:44.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4fcdc6700 1 --2- 172.21.15.17:0/647252075 >> v2:172.21.15.77:3300/0 conn(0x7fd4f80614b0 0x7fd4f8061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:44.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4f77fe700 1 -- 172.21.15.17:0/647252075 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2632151848 0 0) 0x7fd4f80aee80 con 0x7fd4f80ad750 2022-08-01T19:39:44.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4f77fe700 1 -- 172.21.15.17:0/647252075 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4e4003260 con 0x7fd4f80ad750 2022-08-01T19:39:44.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4f77fe700 1 -- 172.21.15.17:0/647252075 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 165882543 0 0) 0x7fd4e4003260 con 0x7fd4f80ad750 2022-08-01T19:39:44.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4f77fe700 1 -- 172.21.15.17:0/647252075 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd4f81405d0 msgr2=0x7fd4f8140a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:44.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4f77fe700 1 --2- 172.21.15.17:0/647252075 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd4f81405d0 0x7fd4f8140a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:44.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4f77fe700 1 -- 172.21.15.17:0/647252075 >> v2:172.21.15.77:3300/0 conn(0x7fd4f80614b0 msgr2=0x7fd4f8061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:44.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4f77fe700 1 --2- 172.21.15.17:0/647252075 >> v2:172.21.15.77:3300/0 conn(0x7fd4f80614b0 0x7fd4f8061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:44.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4f77fe700 1 -- 172.21.15.17:0/647252075 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f80af350 con 0x7fd4f80ad750 2022-08-01T19:39:44.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4f77fe700 1 -- 172.21.15.17:0/647252075 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7fd4e8001660 con 0x7fd4f80ad750 2022-08-01T19:39:44.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4f77fe700 1 -- 172.21.15.17:0/647252075 >> v1:172.21.15.17:6789/0 conn(0x7fd4f80ad750 legacy=0x7fd4f80b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:44.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4f77fe700 1 --2- 172.21.15.17:0/647252075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4e4003750 0x7fd4e4003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:44.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4f77fe700 1 --2- 172.21.15.17:0/647252075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4e4004500 0x7fd4e4004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:44.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4f77fe700 1 --2- 172.21.15.17:0/647252075 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd4f81405d0 0x7fd4e4004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:44.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4f77fe700 1 -- 172.21.15.17:0/647252075 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f80b70b0 con 0x7fd4e4004500 2022-08-01T19:39:44.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4f77fe700 1 -- 172.21.15.17:0/647252075 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f80af0b0 con 0x7fd4e4003750 2022-08-01T19:39:44.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4f77fe700 1 -- 172.21.15.17:0/647252075 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e400b750 con 0x7fd4f81405d0 2022-08-01T19:39:44.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4f7fff700 1 --2- 172.21.15.17:0/647252075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4e4004500 0x7fd4e4004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:44.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4fd5c7700 1 --2- 172.21.15.17:0/647252075 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd4f81405d0 0x7fd4e4004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:44.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4fcdc6700 1 --2- 172.21.15.17:0/647252075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4e4003750 0x7fd4e4003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:44.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4fd5c7700 1 -- 172.21.15.17:0/647252075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4e4003750 msgr2=0x7fd4e4003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:44.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4fd5c7700 1 --2- 172.21.15.17:0/647252075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4e4003750 0x7fd4e4003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:44.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4fd5c7700 1 -- 172.21.15.17:0/647252075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4e4004500 msgr2=0x7fd4e4004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:44.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4fd5c7700 1 --2- 172.21.15.17:0/647252075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4e4004500 0x7fd4e4004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:44.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4fd5c7700 1 -- 172.21.15.17:0/647252075 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e4003260 con 0x7fd4f81405d0 2022-08-01T19:39:44.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4f7fff700 1 --2- 172.21.15.17:0/647252075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4e4004500 0x7fd4e4004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:39:44.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4fd5c7700 1 --2- 172.21.15.17:0/647252075 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd4f81405d0 0x7fd4e4004e70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fd4f000c000 tx=0x7fd4f00091d0).ready entity=mon.2 client_cookie=8c33567ecc1c176b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:44.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4f77fe700 1 -- 172.21.15.17:0/647252075 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4f000e7e0 con 0x7fd4f81405d0 2022-08-01T19:39:44.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4f77fe700 1 -- 172.21.15.17:0/647252075 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f000e940 con 0x7fd4f81405d0 2022-08-01T19:39:44.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4f77fe700 1 -- 172.21.15.17:0/647252075 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4f000eca0 con 0x7fd4f81405d0 2022-08-01T19:39:44.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4fddc8700 1 -- 172.21.15.17:0/647252075 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd4f81405d0 msgr2=0x7fd4e4004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:44.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4fddc8700 1 --2- 172.21.15.17:0/647252075 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd4f81405d0 0x7fd4e4004e70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fd4f000c000 tx=0x7fd4f00091d0).stop 2022-08-01T19:39:44.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4fd5c7700 1 -- 172.21.15.17:0/647252075 reap_dead start 2022-08-01T19:39:44.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.666+0000 7fd4fddc8700 1 -- 172.21.15.17:0/647252075 shutdown_connections 2022-08-01T19:39:44.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.670+0000 7fd4fddc8700 1 -- 172.21.15.17:0/647252075 >> 172.21.15.17:0/647252075 conn(0x7fd4f8066f70 msgr2=0x7fd4f8067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:44.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.670+0000 7fd4fddc8700 1 -- 172.21.15.17:0/647252075 shutdown_connections 2022-08-01T19:39:44.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.670+0000 7fd4fddc8700 1 -- 172.21.15.17:0/647252075 wait complete. 2022-08-01T19:39:44.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.670+0000 7fd4fddc8700 1 Processor -- start 2022-08-01T19:39:44.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.670+0000 7fd4fddc8700 1 -- start start 2022-08-01T19:39:44.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.670+0000 7fd4fddc8700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd4f81405d0 0x7fd4f8156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:44.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.670+0000 7fd4fddc8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4f8143d50 0x7fd4f8155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:44.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.670+0000 7fd4fddc8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4f80ad2b0 0x7fd4f80ab2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:44.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.670+0000 7fd4fddc8700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f81437f0 con 0x7fd4f80ad2b0 2022-08-01T19:39:44.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.670+0000 7fd4fddc8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f80aa500 con 0x7fd4f8143d50 2022-08-01T19:39:44.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.670+0000 7fd4fddc8700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f8143a20 con 0x7fd4f81405d0 2022-08-01T19:39:44.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.670+0000 7fd4fd5c7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4f80ad2b0 0x7fd4f80ab2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:44.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.670+0000 7fd4fd5c7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4f80ad2b0 0x7fd4f80ab2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40260/0 (socket says 172.21.15.17:40260) 2022-08-01T19:39:44.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.670+0000 7fd4fd5c7700 1 -- 172.21.15.17:0/223599629 learned_addr learned my addr 172.21.15.17:0/223599629 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:44.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.670+0000 7fd4fcdc6700 1 --2- 172.21.15.17:0/223599629 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd4f81405d0 0x7fd4f8156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:44.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.670+0000 7fd4f7fff700 1 --2- 172.21.15.17:0/223599629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4f8143d50 0x7fd4f8155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:44.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.670+0000 7fd4fd5c7700 1 -- 172.21.15.17:0/223599629 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd4f81405d0 msgr2=0x7fd4f8156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:44.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.670+0000 7fd4fd5c7700 1 --2- 172.21.15.17:0/223599629 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd4f81405d0 0x7fd4f8156720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:44.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.670+0000 7fd4fd5c7700 1 -- 172.21.15.17:0/223599629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4f8143d50 msgr2=0x7fd4f8155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:44.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.670+0000 7fd4fd5c7700 1 --2- 172.21.15.17:0/223599629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4f8143d50 0x7fd4f8155e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:44.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.670+0000 7fd4fd5c7700 1 -- 172.21.15.17:0/223599629 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f80afac0 con 0x7fd4f80ad2b0 2022-08-01T19:39:44.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.670+0000 7fd4fcdc6700 1 --2- 172.21.15.17:0/223599629 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd4f81405d0 0x7fd4f8156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:39:44.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.674+0000 7fd4fd5c7700 1 --2- 172.21.15.17:0/223599629 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4f80ad2b0 0x7fd4f80ab2a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7fd4f80aecc0 tx=0x7fd4f001a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:44.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.674+0000 7fd4f5ffb700 1 -- 172.21.15.17:0/223599629 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4f0009600 con 0x7fd4f80ad2b0 2022-08-01T19:39:44.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.674+0000 7fd4f5ffb700 1 -- 172.21.15.17:0/223599629 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f001dda0 con 0x7fd4f80ad2b0 2022-08-01T19:39:44.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.674+0000 7fd4f5ffb700 1 -- 172.21.15.17:0/223599629 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4f001b640 con 0x7fd4f80ad2b0 2022-08-01T19:39:44.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.674+0000 7fd4fddc8700 1 -- 172.21.15.17:0/223599629 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f80afd60 con 0x7fd4f80ad2b0 2022-08-01T19:39:44.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.674+0000 7fd4fddc8700 1 -- 172.21.15.17:0/223599629 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f8156340 con 0x7fd4f80ad2b0 2022-08-01T19:39:44.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.674+0000 7fd4f5ffb700 1 -- 172.21.15.17:0/223599629 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd4f001b7a0 con 0x7fd4f80ad2b0 2022-08-01T19:39:44.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.674+0000 7fd4fddc8700 1 -- 172.21.15.17:0/223599629 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4f8156c10 con 0x7fd4f80ad2b0 2022-08-01T19:39:44.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.674+0000 7fd4f5ffb700 1 --2- 172.21.15.17:0/223599629 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd4d8053950 0x7fd4d8055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:44.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.674+0000 7fd4f5ffb700 1 -- 172.21.15.17:0/223599629 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fd4f006dbd0 con 0x7fd4f80ad2b0 2022-08-01T19:39:44.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.678+0000 7fd4fcdc6700 1 --2- 172.21.15.17:0/223599629 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd4d8053950 0x7fd4d8055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:44.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.678+0000 7fd4fcdc6700 1 --2- 172.21.15.17:0/223599629 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd4d8053950 0x7fd4d8055e10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7fd4ec00b000 tx=0x7fd4ec008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:44.692 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.678+0000 7fd4f5ffb700 1 -- 172.21.15.17:0/223599629 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd4f0044b70 con 0x7fd4f80ad2b0 2022-08-01T19:39:44.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:44.842+0000 7fd4fddc8700 1 -- 172.21.15.17:0/223599629 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]} v 0) v1 -- 0x7fd4f80af5f0 con 0x7fd4f80ad2b0 2022-08-01T19:39:45.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.250+0000 7fd4f5ffb700 1 -- 172.21.15.17:0/223599629 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]}]=0 v244) v1 ==== 147+0+0 (secure 0 0 0) 0x7fd4f002a020 con 0x7fd4f80ad2b0 2022-08-01T19:39:45.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.254+0000 7fd4fddc8700 1 -- 172.21.15.17:0/223599629 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd4d8053950 msgr2=0x7fd4d8055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:45.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.254+0000 7fd4fddc8700 1 --2- 172.21.15.17:0/223599629 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd4d8053950 0x7fd4d8055e10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7fd4ec00b000 tx=0x7fd4ec008040).stop 2022-08-01T19:39:45.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.254+0000 7fd4fddc8700 1 -- 172.21.15.17:0/223599629 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4f80ad2b0 msgr2=0x7fd4f80ab2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:45.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.254+0000 7fd4fddc8700 1 --2- 172.21.15.17:0/223599629 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4f80ad2b0 0x7fd4f80ab2a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7fd4f80aecc0 tx=0x7fd4f001a000).stop 2022-08-01T19:39:45.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.254+0000 7fd4fddc8700 1 -- 172.21.15.17:0/223599629 shutdown_connections 2022-08-01T19:39:45.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.254+0000 7fd4fddc8700 1 --2- 172.21.15.17:0/223599629 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd4d8053950 0x7fd4d8055e10 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:45.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.254+0000 7fd4fddc8700 1 --2- 172.21.15.17:0/223599629 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4f80ad2b0 0x7fd4f80ab2a0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:45.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.254+0000 7fd4fddc8700 1 --2- 172.21.15.17:0/223599629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4f8143d50 0x7fd4f8155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:45.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.254+0000 7fd4fddc8700 1 --2- 172.21.15.17:0/223599629 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd4f81405d0 0x7fd4f8156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:45.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.254+0000 7fd4fddc8700 1 -- 172.21.15.17:0/223599629 >> 172.21.15.17:0/223599629 conn(0x7fd4f8066f70 msgr2=0x7fd4f805be30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:45.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.254+0000 7fd4fddc8700 1 -- 172.21.15.17:0/223599629 shutdown_connections 2022-08-01T19:39:45.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.254+0000 7fd4fddc8700 1 -- 172.21.15.17:0/223599629 wait complete. 2022-08-01T19:39:45.270 INFO:tasks.cephfs_test_runner:test_drop_cache_command_timeout (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2022-08-01T19:39:45.271 INFO:tasks.cephfs_test_runner:Starting test: test_cap_revoke_nonresponder (tasks.cephfs.test_misc.TestMisc) 2022-08-01T19:39:45.272 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder' 2022-08-01T19:39:45.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.374+0000 7f35e23a9700 1 Processor -- start 2022-08-01T19:39:45.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.374+0000 7f35e23a9700 1 -- start start 2022-08-01T19:39:45.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.374+0000 7f35e23a9700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f35dc0b1f40 0x7f35dc0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:45.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.374+0000 7f35e23a9700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f35dc0a7250 0x7f35dc1448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:45.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.374+0000 7f35e23a9700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35dc0b04e0 con 0x7f35dc148530 2022-08-01T19:39:45.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.374+0000 7f35e23a9700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35dc0a9250 con 0x7f35dc0a7250 2022-08-01T19:39:45.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.374+0000 7f35e23a9700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35dc0b0780 con 0x7f35dc0b1f40 2022-08-01T19:39:45.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.374+0000 7f35e0946700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f35dc0a7250 0x7f35dc1448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:45.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.374+0000 7f35e0946700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f35dc0a7250 0x7f35dc1448b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40268/0 (socket says 172.21.15.17:40268) 2022-08-01T19:39:45.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.374+0000 7f35e0946700 1 -- 172.21.15.17:0/2722540853 learned_addr learned my addr 172.21.15.17:0/2722540853 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:45.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.374+0000 7f35dbfff700 1 --2- 172.21.15.17:0/2722540853 >> v2:172.21.15.77:3300/0 conn(0x7f35dc0b1f40 0x7f35dc0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:45.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.374+0000 7f35e0946700 1 -- 172.21.15.17:0/2722540853 >> v2:172.21.15.77:3300/0 conn(0x7f35dc0b1f40 msgr2=0x7f35dc0b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:45.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.374+0000 7f35e0946700 1 --2- 172.21.15.17:0/2722540853 >> v2:172.21.15.77:3300/0 conn(0x7f35dc0b1f40 0x7f35dc0b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:45.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.374+0000 7f35e0946700 1 -- 172.21.15.17:0/2722540853 >> v1:172.21.15.77:6789/0 conn(0x7f35dc148530 legacy=0x7f35dc0b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:45.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.374+0000 7f35e0946700 1 -- 172.21.15.17:0/2722540853 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35dc0b0a20 con 0x7f35dc0a7250 2022-08-01T19:39:45.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.374+0000 7f35e0946700 1 --2- 172.21.15.17:0/2722540853 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35dc0a7250 0x7f35dc1448b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f35dc1478c0 tx=0x7f35d4009290).ready entity=mon.0 client_cookie=d06c4c2e2a4cf593 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:45.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.374+0000 7f35daffd700 1 -- 172.21.15.17:0/2722540853 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35d400b860 con 0x7f35dc0a7250 2022-08-01T19:39:45.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.374+0000 7f35daffd700 1 -- 172.21.15.17:0/2722540853 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d400b9c0 con 0x7f35dc0a7250 2022-08-01T19:39:45.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.374+0000 7f35daffd700 1 -- 172.21.15.17:0/2722540853 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35d400bb30 con 0x7f35dc0a7250 2022-08-01T19:39:45.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.374+0000 7f35e23a9700 1 -- 172.21.15.17:0/2722540853 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35dc0a7250 msgr2=0x7f35dc1448b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:45.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.374+0000 7f35e23a9700 1 --2- 172.21.15.17:0/2722540853 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35dc0a7250 0x7f35dc1448b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f35dc1478c0 tx=0x7f35d4009290).stop 2022-08-01T19:39:45.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.374+0000 7f35e23a9700 1 -- 172.21.15.17:0/2722540853 shutdown_connections 2022-08-01T19:39:45.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.374+0000 7f35e23a9700 1 --2- 172.21.15.17:0/2722540853 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35dc0a7250 0x7f35dc1448b0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:45.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.374+0000 7f35e23a9700 1 --2- 172.21.15.17:0/2722540853 >> v2:172.21.15.77:3300/0 conn(0x7f35dc0b1f40 0x7f35dc0b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:45.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.374+0000 7f35e23a9700 1 -- 172.21.15.17:0/2722540853 >> 172.21.15.17:0/2722540853 conn(0x7f35dc05f840 msgr2=0x7f35dc05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:45.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.374+0000 7f35e23a9700 1 -- 172.21.15.17:0/2722540853 shutdown_connections 2022-08-01T19:39:45.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.374+0000 7f35e23a9700 1 -- 172.21.15.17:0/2722540853 wait complete. 2022-08-01T19:39:45.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.378+0000 7f35e23a9700 1 Processor -- start 2022-08-01T19:39:45.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.378+0000 7f35e23a9700 1 -- start start 2022-08-01T19:39:45.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.378+0000 7f35e23a9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35dc0a7250 0x7f35dc0aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:45.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.378+0000 7f35e23a9700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35dc0b1f40 0x7f35dc157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:45.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.378+0000 7f35e23a9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35dc148530 0x7f35dc157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:45.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.378+0000 7f35e23a9700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc14ab00 con 0x7f35dc0a7250 2022-08-01T19:39:45.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.378+0000 7f35e23a9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc148c90 con 0x7f35dc148530 2022-08-01T19:39:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.378+0000 7f35e23a9700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc14bbd0 con 0x7f35dc0b1f40 2022-08-01T19:39:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.378+0000 7f35dbfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35dc0a7250 0x7f35dc0aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.378+0000 7f35dbfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35dc0a7250 0x7f35dc0aedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40270/0 (socket says 172.21.15.17:40270) 2022-08-01T19:39:45.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.378+0000 7f35dbfff700 1 -- 172.21.15.17:0/2332437327 learned_addr learned my addr 172.21.15.17:0/2332437327 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:45.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.378+0000 7f35e0946700 1 --2- 172.21.15.17:0/2332437327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35dc148530 0x7f35dc157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:45.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.378+0000 7f35db7fe700 1 --2- 172.21.15.17:0/2332437327 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35dc0b1f40 0x7f35dc157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:45.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.382+0000 7f35dbfff700 1 -- 172.21.15.17:0/2332437327 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35dc0b1f40 msgr2=0x7f35dc157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:45.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.382+0000 7f35dbfff700 1 --2- 172.21.15.17:0/2332437327 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35dc0b1f40 0x7f35dc157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:45.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.382+0000 7f35dbfff700 1 -- 172.21.15.17:0/2332437327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35dc148530 msgr2=0x7f35dc157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:45.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.382+0000 7f35dbfff700 1 --2- 172.21.15.17:0/2332437327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35dc148530 0x7f35dc157640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:45.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.382+0000 7f35dbfff700 1 -- 172.21.15.17:0/2332437327 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d4009530 con 0x7f35dc0a7250 2022-08-01T19:39:45.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.382+0000 7f35dbfff700 1 --2- 172.21.15.17:0/2332437327 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35dc0a7250 0x7f35dc0aedc0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f35d000c000 tx=0x7f35d0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:45.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.382+0000 7f35d97fa700 1 -- 172.21.15.17:0/2332437327 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35d0010780 con 0x7f35dc0a7250 2022-08-01T19:39:45.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.382+0000 7f35d97fa700 1 -- 172.21.15.17:0/2332437327 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d00108e0 con 0x7f35dc0a7250 2022-08-01T19:39:45.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.382+0000 7f35e23a9700 1 -- 172.21.15.17:0/2332437327 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35dc14ad30 con 0x7f35dc0a7250 2022-08-01T19:39:45.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.382+0000 7f35d97fa700 1 -- 172.21.15.17:0/2332437327 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35d0010c40 con 0x7f35dc0a7250 2022-08-01T19:39:45.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.382+0000 7f35e23a9700 1 -- 172.21.15.17:0/2332437327 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35dc156c80 con 0x7f35dc0a7250 2022-08-01T19:39:45.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.382+0000 7f35e23a9700 1 -- 172.21.15.17:0/2332437327 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35dc0b0cc0 con 0x7f35dc0a7250 2022-08-01T19:39:45.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.382+0000 7f35d97fa700 1 -- 172.21.15.17:0/2332437327 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f35d001c460 con 0x7f35dc0a7250 2022-08-01T19:39:45.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.386+0000 7f35d97fa700 1 --2- 172.21.15.17:0/2332437327 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f35bc053950 0x7f35bc055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:45.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.386+0000 7f35d97fa700 1 -- 172.21.15.17:0/2332437327 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f35d006c230 con 0x7f35dc0a7250 2022-08-01T19:39:45.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.386+0000 7f35db7fe700 1 --2- 172.21.15.17:0/2332437327 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f35bc053950 0x7f35bc055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:45.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.386+0000 7f35db7fe700 1 --2- 172.21.15.17:0/2332437327 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f35bc053950 0x7f35bc055e10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f35cc00d000 tx=0x7f35cc007b70).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:45.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.390+0000 7f35d97fa700 1 -- 172.21.15.17:0/2332437327 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f35d0043160 con 0x7f35dc0a7250 2022-08-01T19:39:45.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:45.558+0000 7f35e23a9700 1 -- 172.21.15.17:0/2332437327 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]} v 0) v1 -- 0x7f35dc14bd40 con 0x7f35dc0a7250 2022-08-01T19:39:46.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.254+0000 7f35d97fa700 1 -- 172.21.15.17:0/2332437327 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]}]=0 v245) v1 ==== 142+0+0 (secure 0 0 0) 0x7f35dc14bd40 con 0x7f35dc0a7250 2022-08-01T19:39:46.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.258+0000 7f35e23a9700 1 -- 172.21.15.17:0/2332437327 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f35bc053950 msgr2=0x7f35bc055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:46.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.258+0000 7f35e23a9700 1 --2- 172.21.15.17:0/2332437327 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f35bc053950 0x7f35bc055e10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f35cc00d000 tx=0x7f35cc007b70).stop 2022-08-01T19:39:46.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.258+0000 7f35e23a9700 1 -- 172.21.15.17:0/2332437327 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35dc0a7250 msgr2=0x7f35dc0aedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:46.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.258+0000 7f35e23a9700 1 --2- 172.21.15.17:0/2332437327 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35dc0a7250 0x7f35dc0aedc0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f35d000c000 tx=0x7f35d0006df0).stop 2022-08-01T19:39:46.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.258+0000 7f35e23a9700 1 -- 172.21.15.17:0/2332437327 shutdown_connections 2022-08-01T19:39:46.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.258+0000 7f35e23a9700 1 --2- 172.21.15.17:0/2332437327 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f35bc053950 0x7f35bc055e10 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:46.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.258+0000 7f35e23a9700 1 --2- 172.21.15.17:0/2332437327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35dc148530 0x7f35dc157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:46.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.258+0000 7f35e23a9700 1 --2- 172.21.15.17:0/2332437327 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35dc0b1f40 0x7f35dc157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:46.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.258+0000 7f35e23a9700 1 --2- 172.21.15.17:0/2332437327 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35dc0a7250 0x7f35dc0aedc0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:46.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.258+0000 7f35e23a9700 1 -- 172.21.15.17:0/2332437327 >> 172.21.15.17:0/2332437327 conn(0x7f35dc05f840 msgr2=0x7f35dc0b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:46.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.258+0000 7f35e23a9700 1 -- 172.21.15.17:0/2332437327 shutdown_connections 2022-08-01T19:39:46.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.258+0000 7f35e23a9700 1 -- 172.21.15.17:0/2332437327 wait complete. 2022-08-01T19:39:46.271 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-08-01T19:39:46.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.386+0000 7f7fc53d5700 1 Processor -- start 2022-08-01T19:39:46.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.386+0000 7f7fc53d5700 1 -- start start 2022-08-01T19:39:46.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.390+0000 7f7fc53d5700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7fc00b31d0 0x7f7fc0066020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:46.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.390+0000 7f7fc53d5700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fc0067030 con 0x7f7fc01402d0 2022-08-01T19:39:46.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.390+0000 7f7fc53d5700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fc00b35b0 con 0x7f7fc00a9da0 2022-08-01T19:39:46.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.390+0000 7f7fc53d5700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fc0064a70 con 0x7f7fc00b31d0 2022-08-01T19:39:46.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.390+0000 7f7fbffff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7fc00b31d0 0x7f7fc0066020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:46.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.390+0000 7f7fc4bd4700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f7fc01402d0 0x7f7fc0140760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60706/0 (socket says 172.21.15.17:60706) 2022-08-01T19:39:46.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.390+0000 7f7fc4bd4700 1 -- 172.21.15.17:0/3683220246 learned_addr learned my addr 172.21.15.17:0/3683220246 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:46.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.390+0000 7f7fbffff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7fc00b31d0 0x7f7fc0066020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40278/0 (socket says 172.21.15.17:40278) 2022-08-01T19:39:46.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.390+0000 7f7fbeffd700 1 -- 172.21.15.17:0/3683220246 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 19849293 0 0) 0x7f7fc0067030 con 0x7f7fc01402d0 2022-08-01T19:39:46.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.390+0000 7f7fbeffd700 1 -- 172.21.15.17:0/3683220246 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fac003260 con 0x7f7fc01402d0 2022-08-01T19:39:46.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.390+0000 7f7fbffff700 1 -- 172.21.15.17:0/3683220246 >> v1:172.21.15.77:6789/0 conn(0x7f7fc00a9da0 legacy=0x7f7fc0060950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:46.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.390+0000 7f7fbffff700 1 -- 172.21.15.17:0/3683220246 >> v1:172.21.15.17:6789/0 conn(0x7f7fc01402d0 legacy=0x7f7fc0140760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:46.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.390+0000 7f7fbffff700 1 -- 172.21.15.17:0/3683220246 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc0066af0 con 0x7f7fc00b31d0 2022-08-01T19:39:46.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.390+0000 7f7fbffff700 1 --2- 172.21.15.17:0/3683220246 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fc00b31d0 0x7f7fc0066020 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f7fb0002f70 tx=0x7f7fb0006e80).ready entity=mon.0 client_cookie=e2c6b3a5bb615917 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:46.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.390+0000 7f7fbeffd700 1 -- 172.21.15.17:0/3683220246 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fb000b7d0 con 0x7f7fc00b31d0 2022-08-01T19:39:46.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.390+0000 7f7fbeffd700 1 -- 172.21.15.17:0/3683220246 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb000b930 con 0x7f7fc00b31d0 2022-08-01T19:39:46.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.390+0000 7f7fbeffd700 1 -- 172.21.15.17:0/3683220246 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fb000bc90 con 0x7f7fc00b31d0 2022-08-01T19:39:46.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.390+0000 7f7fc53d5700 1 -- 172.21.15.17:0/3683220246 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fc00b31d0 msgr2=0x7f7fc0066020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:46.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.390+0000 7f7fc53d5700 1 --2- 172.21.15.17:0/3683220246 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fc00b31d0 0x7f7fc0066020 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f7fb0002f70 tx=0x7f7fb0006e80).stop 2022-08-01T19:39:46.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.390+0000 7f7fc53d5700 1 -- 172.21.15.17:0/3683220246 shutdown_connections 2022-08-01T19:39:46.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.390+0000 7f7fc53d5700 1 --2- 172.21.15.17:0/3683220246 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fc00b31d0 0x7f7fc0066020 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:46.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.390+0000 7f7fc53d5700 1 -- 172.21.15.17:0/3683220246 >> 172.21.15.17:0/3683220246 conn(0x7f7fc0063870 msgr2=0x7f7fc0063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:46.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.390+0000 7f7fc53d5700 1 -- 172.21.15.17:0/3683220246 shutdown_connections 2022-08-01T19:39:46.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.390+0000 7f7fc53d5700 1 -- 172.21.15.17:0/3683220246 wait complete. 2022-08-01T19:39:46.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.394+0000 7f7fc53d5700 1 Processor -- start 2022-08-01T19:39:46.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.394+0000 7f7fc53d5700 1 -- start start 2022-08-01T19:39:46.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.394+0000 7f7fc53d5700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7fc00a9da0 0x7f7fc013bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:46.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.394+0000 7f7fc53d5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fc00b31d0 0x7f7fc0137b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:46.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.394+0000 7f7fc53d5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fc01402d0 0x7f7fc014df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:46.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.394+0000 7f7fc53d5700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc00aa860 con 0x7f7fc01402d0 2022-08-01T19:39:46.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.394+0000 7f7fc53d5700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc0137380 con 0x7f7fc00b31d0 2022-08-01T19:39:46.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.394+0000 7f7fc53d5700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc0151580 con 0x7f7fc00a9da0 2022-08-01T19:39:46.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.394+0000 7f7fc4bd4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fc01402d0 0x7f7fc014df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:46.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.394+0000 7f7fc4bd4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fc01402d0 0x7f7fc014df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40288/0 (socket says 172.21.15.17:40288) 2022-08-01T19:39:46.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.394+0000 7f7fc4bd4700 1 -- 172.21.15.17:0/296452218 learned_addr learned my addr 172.21.15.17:0/296452218 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:46.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.394+0000 7f7fbf7fe700 1 --2- 172.21.15.17:0/296452218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fc00b31d0 0x7f7fc0137b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:46.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.394+0000 7f7fbffff700 1 --2- 172.21.15.17:0/296452218 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7fc00a9da0 0x7f7fc013bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:46.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.394+0000 7f7fc4bd4700 1 -- 172.21.15.17:0/296452218 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7fc00a9da0 msgr2=0x7f7fc013bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:46.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.394+0000 7f7fc4bd4700 1 --2- 172.21.15.17:0/296452218 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7fc00a9da0 0x7f7fc013bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:46.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.394+0000 7f7fc4bd4700 1 -- 172.21.15.17:0/296452218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fc00b31d0 msgr2=0x7f7fc0137b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:46.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.394+0000 7f7fc4bd4700 1 --2- 172.21.15.17:0/296452218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fc00b31d0 0x7f7fc0137b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:46.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.394+0000 7f7fc4bd4700 1 -- 172.21.15.17:0/296452218 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb000d410 con 0x7f7fc01402d0 2022-08-01T19:39:46.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.394+0000 7f7fbffff700 1 --2- 172.21.15.17:0/296452218 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7fc00a9da0 0x7f7fc013bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:39:46.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.394+0000 7f7fc4bd4700 1 --2- 172.21.15.17:0/296452218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fc01402d0 0x7f7fc014df10 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f7fb8001490 tx=0x7f7fb8007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:46.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.394+0000 7f7fbd7fa700 1 -- 172.21.15.17:0/296452218 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fb8005ba0 con 0x7f7fc01402d0 2022-08-01T19:39:46.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.394+0000 7f7fbd7fa700 1 -- 172.21.15.17:0/296452218 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb8005d00 con 0x7f7fc01402d0 2022-08-01T19:39:46.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.394+0000 7f7fc53d5700 1 -- 172.21.15.17:0/296452218 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc0151820 con 0x7f7fc01402d0 2022-08-01T19:39:46.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.394+0000 7f7fbd7fa700 1 -- 172.21.15.17:0/296452218 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fb80072c0 con 0x7f7fc01402d0 2022-08-01T19:39:46.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.398+0000 7f7fc53d5700 1 -- 172.21.15.17:0/296452218 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc0151d60 con 0x7f7fc01402d0 2022-08-01T19:39:46.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.398+0000 7f7fc53d5700 1 -- 172.21.15.17:0/296452218 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fc0066d90 con 0x7f7fc01402d0 2022-08-01T19:39:46.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.398+0000 7f7fbd7fa700 1 -- 172.21.15.17:0/296452218 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7fb801a060 con 0x7f7fc01402d0 2022-08-01T19:39:46.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.398+0000 7f7fbd7fa700 1 --2- 172.21.15.17:0/296452218 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7fa0053b30 0x7f7fa0055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:46.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.398+0000 7f7fbd7fa700 1 -- 172.21.15.17:0/296452218 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f7fb806b7e0 con 0x7f7fc01402d0 2022-08-01T19:39:46.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.398+0000 7f7fbffff700 1 --2- 172.21.15.17:0/296452218 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7fa0053b30 0x7f7fa0055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:46.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.402+0000 7f7fbffff700 1 --2- 172.21.15.17:0/296452218 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7fa0053b30 0x7f7fa0055ff0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f7fb0019000 tx=0x7f7fc0067360).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:46.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.406+0000 7f7fbd7fa700 1 -- 172.21.15.17:0/296452218 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7fb8042880 con 0x7f7fc01402d0 2022-08-01T19:39:46.532 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.534+0000 7f7fc53d5700 1 -- 172.21.15.17:0/296452218 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7fc01523a0 con 0x7f7fc01402d0 2022-08-01T19:39:46.539 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.542+0000 7f7fbd7fa700 1 -- 172.21.15.17:0/296452218 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7fb8018030 con 0x7f7fc01402d0 2022-08-01T19:39:46.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.542+0000 7f7fc53d5700 1 -- 172.21.15.17:0/296452218 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7fa0053b30 msgr2=0x7f7fa0055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:46.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.542+0000 7f7fc53d5700 1 --2- 172.21.15.17:0/296452218 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7fa0053b30 0x7f7fa0055ff0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f7fb0019000 tx=0x7f7fc0067360).stop 2022-08-01T19:39:46.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.542+0000 7f7fc53d5700 1 -- 172.21.15.17:0/296452218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fc01402d0 msgr2=0x7f7fc014df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:46.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.542+0000 7f7fc53d5700 1 --2- 172.21.15.17:0/296452218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fc01402d0 0x7f7fc014df10 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f7fb8001490 tx=0x7f7fb8007dc0).stop 2022-08-01T19:39:46.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.542+0000 7f7fc53d5700 1 -- 172.21.15.17:0/296452218 shutdown_connections 2022-08-01T19:39:46.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.542+0000 7f7fc53d5700 1 --2- 172.21.15.17:0/296452218 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7fa0053b30 0x7f7fa0055ff0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:46.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.542+0000 7f7fc53d5700 1 --2- 172.21.15.17:0/296452218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fc01402d0 0x7f7fc014df10 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:46.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.542+0000 7f7fc53d5700 1 --2- 172.21.15.17:0/296452218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fc00b31d0 0x7f7fc0137b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:46.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.542+0000 7f7fc53d5700 1 --2- 172.21.15.17:0/296452218 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7fc00a9da0 0x7f7fc013bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:46.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.542+0000 7f7fc53d5700 1 -- 172.21.15.17:0/296452218 >> 172.21.15.17:0/296452218 conn(0x7f7fc0063870 msgr2=0x7f7fc00b4780 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:46.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.542+0000 7f7fc53d5700 1 -- 172.21.15.17:0/296452218 shutdown_connections 2022-08-01T19:39:46.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.542+0000 7f7fc53d5700 1 -- 172.21.15.17:0/296452218 wait complete. 2022-08-01T19:39:46.554 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T19:39:46.555 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:39:46.571 DEBUG:teuthology.orchestra.run.smithi077:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:39:46.593 DEBUG:teuthology.orchestra.run.smithi085:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:39:46.616 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:39:46.637 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T19:39:46.638 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:39:46.638 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:39:46.639 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:39:46.650 INFO:teuthology.orchestra.run.smithi085.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-08-01T19:39:46.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:39:46.652 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-08-01T19:39:46.652 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:39:46.653 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:39:46.653 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:39:46.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:39:46.691 INFO:teuthology.orchestra.run.smithi190.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T19:39:46.691 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-08-01T19:39:46.692 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T19:39:46.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.806+0000 7f25ff59e700 1 Processor -- start 2022-08-01T19:39:46.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.806+0000 7f25ff59e700 1 -- start start 2022-08-01T19:39:46.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.806+0000 7f25ff59e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f26000b2710 0x7f260014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:46.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.806+0000 7f25ff59e700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26000a7250 con 0x7f26000a8c50 2022-08-01T19:39:46.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.806+0000 7f25ff59e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26000a7480 con 0x7f26000b01a0 2022-08-01T19:39:46.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.806+0000 7f25ff59e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26000a7720 con 0x7f26000b2710 2022-08-01T19:39:46.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.806+0000 7f25fed9d700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f26000a8c50 0x7f2600151d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60722/0 (socket says 172.21.15.17:60722) 2022-08-01T19:39:46.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.806+0000 7f25fed9d700 1 -- 172.21.15.17:0/2223879237 learned_addr learned my addr 172.21.15.17:0/2223879237 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:46.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.806+0000 7f25fe59c700 1 --2- 172.21.15.17:0/2223879237 >> v2:172.21.15.17:3300/0 conn(0x7f26000b2710 0x7f260014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:46.811 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.806+0000 7f25fd59a700 1 -- 172.21.15.17:0/2223879237 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 251153075 0 0) 0x7f26000a7250 con 0x7f26000a8c50 2022-08-01T19:39:46.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.806+0000 7f25fd59a700 1 -- 172.21.15.17:0/2223879237 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25e4003260 con 0x7f26000a8c50 2022-08-01T19:39:46.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.810+0000 7f25fd59a700 1 -- 172.21.15.17:0/2223879237 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1500675655 0 0) 0x7f26000a7480 con 0x7f26000b01a0 2022-08-01T19:39:46.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.810+0000 7f25fe59c700 1 -- 172.21.15.17:0/2223879237 >> v1:172.21.15.77:6789/0 conn(0x7f26000b01a0 legacy=0x7f26000a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:46.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.810+0000 7f25fe59c700 1 -- 172.21.15.17:0/2223879237 >> v1:172.21.15.17:6789/0 conn(0x7f26000a8c50 legacy=0x7f2600151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:46.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.810+0000 7f25fe59c700 1 -- 172.21.15.17:0/2223879237 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26000a79c0 con 0x7f26000b2710 2022-08-01T19:39:46.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.810+0000 7f25fe59c700 1 --2- 172.21.15.17:0/2223879237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26000b2710 0x7f260014afd0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f25f4002f70 tx=0x7f25f4006df0).ready entity=mon.0 client_cookie=3e8c27ad4cd2041c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:46.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.810+0000 7f25fd59a700 1 -- 172.21.15.17:0/2223879237 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25f400d610 con 0x7f26000b2710 2022-08-01T19:39:46.814 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.810+0000 7f25fd59a700 1 -- 172.21.15.17:0/2223879237 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f4000e90 con 0x7f26000b2710 2022-08-01T19:39:46.815 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.810+0000 7f25fd59a700 1 -- 172.21.15.17:0/2223879237 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25f400d960 con 0x7f26000b2710 2022-08-01T19:39:46.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.810+0000 7f25ff59e700 1 -- 172.21.15.17:0/2223879237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26000b2710 msgr2=0x7f260014afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:46.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.810+0000 7f25ff59e700 1 --2- 172.21.15.17:0/2223879237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26000b2710 0x7f260014afd0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f25f4002f70 tx=0x7f25f4006df0).stop 2022-08-01T19:39:46.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.810+0000 7f25ff59e700 1 -- 172.21.15.17:0/2223879237 shutdown_connections 2022-08-01T19:39:46.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.810+0000 7f25ff59e700 1 --2- 172.21.15.17:0/2223879237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26000b2710 0x7f260014afd0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:46.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.810+0000 7f25ff59e700 1 -- 172.21.15.17:0/2223879237 >> 172.21.15.17:0/2223879237 conn(0x7f2600060310 msgr2=0x7f26000b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:46.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.810+0000 7f25ff59e700 1 -- 172.21.15.17:0/2223879237 shutdown_connections 2022-08-01T19:39:46.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.810+0000 7f25ff59e700 1 -- 172.21.15.17:0/2223879237 wait complete. 2022-08-01T19:39:46.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.814+0000 7f25ff59e700 1 Processor -- start 2022-08-01T19:39:46.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.814+0000 7f25ff59e700 1 -- start start 2022-08-01T19:39:46.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.814+0000 7f25ff59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f26000a8c50 0x7f26000636d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:46.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.814+0000 7f25ff59e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26000b01a0 0x7f2600156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:46.822 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.814+0000 7f25ff59e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26000b2710 0x7f2600156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:46.822 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.814+0000 7f25ff59e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26000aa180 con 0x7f26000b2710 2022-08-01T19:39:46.822 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.814+0000 7f25ff59e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26000a9450 con 0x7f26000b01a0 2022-08-01T19:39:46.822 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.814+0000 7f25ff59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26000aa3b0 con 0x7f26000a8c50 2022-08-01T19:39:46.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.814+0000 7f25fed9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26000b2710 0x7f2600156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:46.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.814+0000 7f25fed9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26000b2710 0x7f2600156930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40302/0 (socket says 172.21.15.17:40302) 2022-08-01T19:39:46.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.814+0000 7f25fed9d700 1 -- 172.21.15.17:0/1233371018 learned_addr learned my addr 172.21.15.17:0/1233371018 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:46.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.814+0000 7f25fdd9b700 1 --2- 172.21.15.17:0/1233371018 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26000b01a0 0x7f2600156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:46.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.814+0000 7f25fe59c700 1 --2- 172.21.15.17:0/1233371018 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f26000a8c50 0x7f26000636d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:46.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.814+0000 7f25fed9d700 1 -- 172.21.15.17:0/1233371018 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f26000a8c50 msgr2=0x7f26000636d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:46.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.814+0000 7f25fed9d700 1 --2- 172.21.15.17:0/1233371018 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f26000a8c50 0x7f26000636d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:46.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.814+0000 7f25fed9d700 1 -- 172.21.15.17:0/1233371018 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26000b01a0 msgr2=0x7f2600156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:46.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.814+0000 7f25fed9d700 1 --2- 172.21.15.17:0/1233371018 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26000b01a0 0x7f2600156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:46.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.814+0000 7f25fed9d700 1 -- 172.21.15.17:0/1233371018 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f400b180 con 0x7f26000b2710 2022-08-01T19:39:46.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.814+0000 7f25fe59c700 1 --2- 172.21.15.17:0/1233371018 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f26000a8c50 0x7f26000636d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:39:46.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.814+0000 7f25fed9d700 1 --2- 172.21.15.17:0/1233371018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26000b2710 0x7f2600156930 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f25f8001110 tx=0x7f25f80079e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:46.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.814+0000 7f25ef7fe700 1 -- 172.21.15.17:0/1233371018 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25f80070d0 con 0x7f26000b2710 2022-08-01T19:39:46.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.814+0000 7f25ef7fe700 1 -- 172.21.15.17:0/1233371018 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f8007230 con 0x7f26000b2710 2022-08-01T19:39:46.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.814+0000 7f25ef7fe700 1 -- 172.21.15.17:0/1233371018 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25f8005610 con 0x7f26000b2710 2022-08-01T19:39:46.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.814+0000 7f25ff59e700 1 -- 172.21.15.17:0/1233371018 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26000ab2e0 con 0x7f26000b2710 2022-08-01T19:39:46.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.814+0000 7f25ff59e700 1 -- 172.21.15.17:0/1233371018 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f260014b660 con 0x7f26000b2710 2022-08-01T19:39:46.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.818+0000 7f25ff59e700 1 -- 172.21.15.17:0/1233371018 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26000ab4d0 con 0x7f26000b2710 2022-08-01T19:39:46.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.818+0000 7f25ef7fe700 1 -- 172.21.15.17:0/1233371018 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f25f801a070 con 0x7f26000b2710 2022-08-01T19:39:46.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.818+0000 7f25ef7fe700 1 --2- 172.21.15.17:0/1233371018 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f25dc053aa0 0x7f25dc055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:46.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.818+0000 7f25ef7fe700 1 -- 172.21.15.17:0/1233371018 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f25f806b040 con 0x7f26000b2710 2022-08-01T19:39:46.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.818+0000 7f25fe59c700 1 --2- 172.21.15.17:0/1233371018 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f25dc053aa0 0x7f25dc055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:46.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.818+0000 7f25fe59c700 1 --2- 172.21.15.17:0/1233371018 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f25dc053aa0 0x7f25dc055f60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f25f4018000 tx=0x7f25f4014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:46.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.822+0000 7f25ef7fe700 1 -- 172.21.15.17:0/1233371018 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f26000ab4d0 con 0x7f26000b2710 2022-08-01T19:39:46.949 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.950+0000 7f25ff59e700 1 -- 172.21.15.17:0/1233371018 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f26000ab4d0 con 0x7f26000b2710 2022-08-01T19:39:46.951 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.950+0000 7f25ef7fe700 1 -- 172.21.15.17:0/1233371018 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v69) v1 ==== 81+0+29220 (secure 0 0 0) 0x7f26000ab4d0 con 0x7f26000b2710 2022-08-01T19:39:46.952 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:39:46.952 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:39:46.953 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 69, 2022-08-01T19:39:46.953 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", 2022-08-01T19:39:46.953 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2022-08-01T19:33:50.105743+0000", 2022-08-01T19:39:46.953 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2022-08-01T19:39:44.235129+0000", 2022-08-01T19:39:46.954 INFO:teuthology.orchestra.run.smithi017.stdout: "last_up_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:39:46.954 INFO:teuthology.orchestra.run.smithi017.stdout: "last_in_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:39:46.955 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T19:39:46.956 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_num": 5799936, 2022-08-01T19:39:46.956 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_set": [ 2022-08-01T19:39:46.956 INFO:teuthology.orchestra.run.smithi017.stdout: "pglog_hardlimit", 2022-08-01T19:39:46.956 INFO:teuthology.orchestra.run.smithi017.stdout: "purged_snapdirs", 2022-08-01T19:39:46.957 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_deletes", 2022-08-01T19:39:46.957 INFO:teuthology.orchestra.run.smithi017.stdout: "sortbitwise" 2022-08-01T19:39:46.957 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:39:46.957 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_version": 6, 2022-08-01T19:39:46.957 INFO:teuthology.orchestra.run.smithi017.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T19:39:46.958 INFO:teuthology.orchestra.run.smithi017.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T19:39:46.958 INFO:teuthology.orchestra.run.smithi017.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T19:39:46.958 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_snapshot": "", 2022-08-01T19:39:46.958 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_max": 12, 2022-08-01T19:39:46.959 INFO:teuthology.orchestra.run.smithi017.stdout: "max_osd": 8, 2022-08-01T19:39:46.959 INFO:teuthology.orchestra.run.smithi017.stdout: "require_min_compat_client": "jewel", 2022-08-01T19:39:46.959 INFO:teuthology.orchestra.run.smithi017.stdout: "min_compat_client": "jewel", 2022-08-01T19:39:46.959 INFO:teuthology.orchestra.run.smithi017.stdout: "require_osd_release": "octopus", 2022-08-01T19:39:46.960 INFO:teuthology.orchestra.run.smithi017.stdout: "pools": [ 2022-08-01T19:39:46.960 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:46.960 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 1, 2022-08-01T19:39:46.960 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "device_health_metrics", 2022-08-01T19:39:46.961 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:33:53.001841+0000", 2022-08-01T19:39:46.961 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:39:46.961 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:39:46.961 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:39:46.961 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:39:46.962 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:39:46.962 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:39:46.962 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:39:46.962 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:39:46.963 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 1, 2022-08-01T19:39:46.963 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 1, 2022-08-01T19:39:46.963 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 1, 2022-08-01T19:39:46.963 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 1, 2022-08-01T19:39:46.964 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 1, 2022-08-01T19:39:46.964 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:39:46.964 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:39:46.964 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:39:46.964 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:39:46.965 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:39:46.965 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:39:46.965 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:39:46.965 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:46.966 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "16", 2022-08-01T19:39:46.966 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:39:46.966 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:39:46.966 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:39:46.966 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:39:46.967 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:39:46.967 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:39:46.967 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:39:46.967 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:39:46.968 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:39:46.968 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:39:46.968 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:39:46.968 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:39:46.969 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:39:46.971 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:39:46.971 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:39:46.972 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:39:46.973 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:39:46.973 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:39:46.974 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:39:46.978 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:39:46.978 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:39:46.979 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:39:46.979 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:39:46.980 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:39:46.980 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:39:46.980 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:39:46.981 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:46.981 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:39:46.981 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:39:46.982 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:39:46.982 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:39:46.983 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:39:46.983 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:39:46.983 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:39:46.984 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:39:46.984 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:39:46.984 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:39:46.985 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:39:46.985 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:39:46.985 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 1 2022-08-01T19:39:46.986 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:46.986 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:39:46.987 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_devicehealth": {} 2022-08-01T19:39:46.987 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:46.987 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:46.988 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:46.988 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 2, 2022-08-01T19:39:46.989 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "rbd", 2022-08-01T19:39:46.989 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:34:01.553098+0000", 2022-08-01T19:39:46.989 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:39:46.990 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:39:46.990 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:39:46.991 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:39:46.991 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:39:46.992 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:39:46.992 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:39:46.992 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:39:46.993 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:39:46.993 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:39:46.994 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:39:46.994 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:39:46.995 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:39:46.995 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:39:46.995 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:39:46.996 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:39:46.996 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:39:46.997 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:39:46.997 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:39:46.997 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:39:46.998 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:46.999 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "19", 2022-08-01T19:39:46.999 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:39:46.999 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:39:47.000 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:39:47.000 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:39:47.000 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:39:47.001 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:39:47.001 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:39:47.001 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:39:47.002 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:39:47.002 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:39:47.002 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:39:47.003 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:39:47.003 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:39:47.004 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:39:47.004 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:39:47.004 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:39:47.004 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:39:47.005 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:39:47.005 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:39:47.005 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:39:47.005 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:39:47.005 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:39:47.006 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:39:47.006 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:39:47.006 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:39:47.006 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:39:47.007 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.007 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:39:47.007 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:39:47.007 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:39:47.008 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:39:47.008 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:39:47.008 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:39:47.008 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:39:47.009 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:39:47.009 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:39:47.009 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:39:47.009 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:39:47.009 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:39:47.010 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:39:47.010 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd": {} 2022-08-01T19:39:47.010 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.010 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.011 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:39:47.011 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2022-08-01T19:39:47.011 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.011 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:39:47.012 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a962f14f-cf16-4fa8-84ff-c85325db9132", 2022-08-01T19:39:47.012 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:39:47.012 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:39:47.012 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:39:47.013 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:39:47.013 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:39:47.013 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:39:47.013 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:39:47.014 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 64, 2022-08-01T19:39:47.014 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:39:47.014 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:39:47.014 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:39:47.015 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:47.015 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.015 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:47.015 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6800", 2022-08-01T19:39:47.016 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:39:47.016 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.016 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.016 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:47.017 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6801", 2022-08-01T19:39:47.017 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:39:47.017 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.017 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.018 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.018 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:39:47.018 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:47.019 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.019 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:47.019 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6802", 2022-08-01T19:39:47.019 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:39:47.020 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.020 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.020 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:47.020 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6803", 2022-08-01T19:39:47.020 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:39:47.021 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.021 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.021 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.022 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:39:47.022 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:47.022 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.022 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:47.022 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6806", 2022-08-01T19:39:47.023 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:39:47.023 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.023 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.023 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:47.024 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6807", 2022-08-01T19:39:47.024 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:39:47.024 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.024 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.024 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.025 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:39:47.025 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:47.025 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.026 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:47.026 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6804", 2022-08-01T19:39:47.026 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:39:47.026 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.026 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.027 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:47.027 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6805", 2022-08-01T19:39:47.027 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:39:47.027 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.028 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.028 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.028 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6801/23008", 2022-08-01T19:39:47.028 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6803/23008", 2022-08-01T19:39:47.028 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6807/23008", 2022-08-01T19:39:47.029 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6805/23008", 2022-08-01T19:39:47.029 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:39:47.029 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:39:47.029 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:39:47.030 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.030 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.030 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.030 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:39:47.030 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "805f7051-a527-4e98-8844-b6e406a63248", 2022-08-01T19:39:47.031 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:39:47.031 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:39:47.031 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:39:47.031 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:39:47.032 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:39:47.032 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:39:47.032 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:39:47.032 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 64, 2022-08-01T19:39:47.033 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:39:47.033 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:39:47.033 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:39:47.033 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:47.034 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.034 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:47.034 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6808", 2022-08-01T19:39:47.034 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:39:47.035 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.035 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.035 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:47.035 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6809", 2022-08-01T19:39:47.036 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:39:47.036 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.036 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.036 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.037 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:39:47.037 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:47.037 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.037 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:47.037 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6810", 2022-08-01T19:39:47.038 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:39:47.038 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.038 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.038 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:47.039 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6811", 2022-08-01T19:39:47.039 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:39:47.039 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.039 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.040 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.040 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:39:47.040 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:47.040 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.040 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:47.041 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6814", 2022-08-01T19:39:47.041 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:39:47.041 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.042 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.042 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:47.042 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6815", 2022-08-01T19:39:47.042 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:39:47.043 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.043 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.043 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.043 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:39:47.044 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:47.044 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.044 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:47.044 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6812", 2022-08-01T19:39:47.044 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:39:47.045 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.045 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.045 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:47.045 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6813", 2022-08-01T19:39:47.046 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:39:47.046 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.046 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.047 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.047 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6809/23007", 2022-08-01T19:39:47.047 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6811/23007", 2022-08-01T19:39:47.047 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6815/23007", 2022-08-01T19:39:47.047 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6813/23007", 2022-08-01T19:39:47.048 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:39:47.048 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:39:47.048 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:39:47.048 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.049 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.049 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.049 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:39:47.049 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "89f7f3df-7c10-48e1-8b69-9e4243923a61", 2022-08-01T19:39:47.050 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:39:47.050 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:39:47.051 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:39:47.051 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:39:47.051 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:39:47.051 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:39:47.051 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:39:47.052 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 63, 2022-08-01T19:39:47.052 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:39:47.052 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:39:47.052 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:39:47.053 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:47.053 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.053 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:47.053 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6817", 2022-08-01T19:39:47.053 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:39:47.054 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.054 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.055 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:47.055 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6819", 2022-08-01T19:39:47.055 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:39:47.056 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.056 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.057 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.057 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:39:47.057 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:47.058 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.058 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:47.058 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6822", 2022-08-01T19:39:47.059 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:39:47.059 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.059 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.060 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:47.060 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6824", 2022-08-01T19:39:47.060 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:39:47.061 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.061 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.062 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.062 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:39:47.063 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:47.063 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.063 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:47.064 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6830", 2022-08-01T19:39:47.064 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:39:47.065 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.065 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.066 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:47.066 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6831", 2022-08-01T19:39:47.066 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:39:47.067 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.067 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.068 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.068 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:39:47.069 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:47.069 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.070 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:47.070 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6826", 2022-08-01T19:39:47.070 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:39:47.071 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.071 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.072 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:47.072 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6829", 2022-08-01T19:39:47.073 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:39:47.073 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.074 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.074 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.074 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6819/23010", 2022-08-01T19:39:47.075 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6824/23010", 2022-08-01T19:39:47.075 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6831/23010", 2022-08-01T19:39:47.075 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6829/23010", 2022-08-01T19:39:47.076 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:39:47.076 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:39:47.077 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:39:47.077 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.077 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.078 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.078 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:39:47.079 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "96123468-aabc-4a64-909e-a192c369090d", 2022-08-01T19:39:47.079 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:39:47.080 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:39:47.080 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:39:47.080 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:39:47.081 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:39:47.081 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:39:47.082 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:39:47.082 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 52, 2022-08-01T19:39:47.083 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:39:47.083 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:39:47.084 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:39:47.084 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:47.084 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.084 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:47.085 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6816", 2022-08-01T19:39:47.085 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:39:47.085 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.085 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.085 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:47.086 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6818", 2022-08-01T19:39:47.086 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:39:47.086 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.086 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.087 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.087 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:39:47.087 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:47.087 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.087 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:47.088 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6820", 2022-08-01T19:39:47.088 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:39:47.088 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.088 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.089 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:47.089 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6821", 2022-08-01T19:39:47.089 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:39:47.089 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.089 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.090 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.090 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:39:47.090 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:47.090 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.091 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:47.091 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6827", 2022-08-01T19:39:47.091 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:39:47.091 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.091 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.092 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:47.092 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6828", 2022-08-01T19:39:47.092 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:39:47.092 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.092 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.093 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.093 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:39:47.093 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:47.093 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.094 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:47.094 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6823", 2022-08-01T19:39:47.094 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:39:47.094 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.094 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.095 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:47.095 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6825", 2022-08-01T19:39:47.095 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:39:47.095 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.096 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.096 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.096 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6818/23009", 2022-08-01T19:39:47.096 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6821/23009", 2022-08-01T19:39:47.096 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6828/23009", 2022-08-01T19:39:47.097 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6825/23009", 2022-08-01T19:39:47.097 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:39:47.097 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:39:47.097 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:39:47.097 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.098 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.098 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.098 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:39:47.098 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a6e26665-e04b-46ec-ab66-e912acbf044c", 2022-08-01T19:39:47.099 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:39:47.099 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:39:47.099 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:39:47.099 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:39:47.099 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:39:47.100 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:39:47.100 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:39:47.100 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 64, 2022-08-01T19:39:47.100 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:39:47.101 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:39:47.101 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:39:47.101 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:47.101 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.101 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:47.102 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6810", 2022-08-01T19:39:47.102 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:39:47.102 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.102 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.102 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:47.103 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6811", 2022-08-01T19:39:47.103 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:39:47.103 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.103 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.104 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.104 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:39:47.104 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:47.104 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.105 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:47.105 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6812", 2022-08-01T19:39:47.105 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:39:47.105 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.105 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.106 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:47.106 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6813", 2022-08-01T19:39:47.106 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:39:47.106 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.107 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.107 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.107 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:39:47.107 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:47.107 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.108 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:47.108 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6816", 2022-08-01T19:39:47.108 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:39:47.108 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.109 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.109 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:47.109 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6817", 2022-08-01T19:39:47.109 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:39:47.109 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.110 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.110 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.110 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:39:47.110 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:47.111 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.111 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:47.111 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6814", 2022-08-01T19:39:47.111 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:39:47.111 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.112 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.112 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:47.112 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6815", 2022-08-01T19:39:47.112 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:39:47.112 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.113 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.113 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.113 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6811/23584", 2022-08-01T19:39:47.113 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6813/23584", 2022-08-01T19:39:47.114 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6817/23584", 2022-08-01T19:39:47.114 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6815/23584", 2022-08-01T19:39:47.114 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:39:47.114 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:39:47.114 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:39:47.115 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.115 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.115 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.115 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:39:47.116 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "502c0034-18ee-44ac-87ff-19b9dddb66a0", 2022-08-01T19:39:47.116 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:39:47.116 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:39:47.116 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:39:47.116 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:39:47.117 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:39:47.117 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:39:47.117 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:39:47.117 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 63, 2022-08-01T19:39:47.117 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:39:47.118 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:39:47.118 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:39:47.118 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:47.118 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.119 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:47.119 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6802", 2022-08-01T19:39:47.119 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:39:47.119 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.119 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.120 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:47.120 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6803", 2022-08-01T19:39:47.120 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:39:47.120 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.120 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.121 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.121 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:39:47.121 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:47.121 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.122 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:47.122 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6804", 2022-08-01T19:39:47.122 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:39:47.122 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.122 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.123 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:47.123 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6805", 2022-08-01T19:39:47.123 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:39:47.123 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.124 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.124 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.124 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:39:47.124 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:47.124 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.125 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:47.125 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6808", 2022-08-01T19:39:47.125 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:39:47.125 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.126 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.126 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:47.126 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6809", 2022-08-01T19:39:47.126 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:39:47.126 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.135 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.136 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.136 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:39:47.136 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:47.136 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.137 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:47.137 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6806", 2022-08-01T19:39:47.137 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:39:47.137 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.137 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.138 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:47.138 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6807", 2022-08-01T19:39:47.138 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:39:47.138 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.139 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.139 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.139 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6803/23585", 2022-08-01T19:39:47.139 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6805/23585", 2022-08-01T19:39:47.139 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6809/23585", 2022-08-01T19:39:47.140 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6807/23585", 2022-08-01T19:39:47.140 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:39:47.140 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:39:47.140 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:39:47.141 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.141 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.141 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.141 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:39:47.141 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "ac4e755a-7fe7-46cb-8407-852d5af9ffc5", 2022-08-01T19:39:47.142 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:39:47.142 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:39:47.142 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:39:47.142 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:39:47.143 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:39:47.143 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:39:47.143 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:39:47.143 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 64, 2022-08-01T19:39:47.143 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:39:47.144 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:39:47.144 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:39:47.144 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:47.144 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.145 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:47.145 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6820", 2022-08-01T19:39:47.145 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:39:47.145 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.145 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.146 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:47.146 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6821", 2022-08-01T19:39:47.146 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:39:47.147 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.147 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.148 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.148 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:39:47.148 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:47.149 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.149 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:47.150 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6823", 2022-08-01T19:39:47.150 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:39:47.150 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.151 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.151 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:47.152 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6825", 2022-08-01T19:39:47.152 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:39:47.152 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.153 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.153 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.154 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:39:47.154 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:47.155 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.155 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:47.156 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6831", 2022-08-01T19:39:47.156 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:39:47.156 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.157 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.157 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:47.158 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6833", 2022-08-01T19:39:47.158 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:39:47.159 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.159 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.159 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.160 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:39:47.160 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:47.160 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.161 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:47.161 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6827", 2022-08-01T19:39:47.162 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:39:47.162 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.163 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.163 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:47.164 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6829", 2022-08-01T19:39:47.164 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:39:47.165 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.165 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.165 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.166 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6821/23587", 2022-08-01T19:39:47.166 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6825/23587", 2022-08-01T19:39:47.166 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6833/23587", 2022-08-01T19:39:47.167 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6829/23587", 2022-08-01T19:39:47.167 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:39:47.167 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:39:47.168 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:39:47.168 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.169 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.169 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.169 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:39:47.169 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "874761e2-6410-40fc-b60c-e58abe5ce489", 2022-08-01T19:39:47.170 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:39:47.170 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:39:47.170 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:39:47.170 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:39:47.171 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:39:47.171 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:39:47.172 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:39:47.172 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 63, 2022-08-01T19:39:47.172 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:39:47.173 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:39:47.173 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:39:47.173 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:47.174 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.174 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:47.174 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6818", 2022-08-01T19:39:47.174 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:39:47.175 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.175 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.176 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:47.176 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6819", 2022-08-01T19:39:47.177 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:39:47.177 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.178 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.178 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.179 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:39:47.179 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:47.180 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.180 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:47.181 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6822", 2022-08-01T19:39:47.181 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:39:47.182 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.183 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.183 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:47.184 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6824", 2022-08-01T19:39:47.184 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:39:47.185 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.185 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.185 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.186 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:39:47.186 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:47.187 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.187 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:47.188 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6830", 2022-08-01T19:39:47.189 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:39:47.189 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.190 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.190 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:47.191 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6832", 2022-08-01T19:39:47.191 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:39:47.192 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.192 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.192 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.193 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:39:47.193 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:39:47.194 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.194 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:39:47.195 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6826", 2022-08-01T19:39:47.195 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:39:47.196 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.196 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.197 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:39:47.197 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6828", 2022-08-01T19:39:47.197 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:39:47.198 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.198 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.199 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.199 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6819/23586", 2022-08-01T19:39:47.200 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6824/23586", 2022-08-01T19:39:47.200 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6832/23586", 2022-08-01T19:39:47.200 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6828/23586", 2022-08-01T19:39:47.201 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:39:47.201 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:39:47.202 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:39:47.202 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:47.203 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.203 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:39:47.204 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_xinfo": [ 2022-08-01T19:39:47.204 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.205 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:39:47.205 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:39:47.206 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:39:47.206 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:39:47.207 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:39:47.207 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:39:47.208 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.588817+0000", 2022-08-01T19:39:47.208 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:39:47.209 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.209 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.209 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:39:47.210 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:39:47.210 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:39:47.211 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:39:47.211 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:39:47.212 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:39:47.212 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.782753+0000", 2022-08-01T19:39:47.213 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:39:47.213 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.214 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.214 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:39:47.215 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:39:47.215 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:39:47.215 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:39:47.216 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:39:47.217 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:39:47.217 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.837924+0000", 2022-08-01T19:39:47.218 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:39:47.218 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.219 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.219 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:39:47.220 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:39:47.220 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:39:47.221 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:39:47.222 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:39:47.222 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:39:47.223 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.839948+0000", 2022-08-01T19:39:47.223 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:39:47.223 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.224 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.224 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:39:47.225 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:39:47.225 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:39:47.225 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:39:47.226 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:39:47.226 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:39:47.226 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.112801+0000", 2022-08-01T19:39:47.227 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:39:47.227 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.228 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.228 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:39:47.229 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:39:47.229 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:39:47.230 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:39:47.230 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:39:47.231 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:39:47.231 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.106704+0000", 2022-08-01T19:39:47.231 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:39:47.232 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.232 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.233 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:39:47.233 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:39:47.233 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:39:47.233 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:39:47.234 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:39:47.234 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:39:47.234 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.893170+0000", 2022-08-01T19:39:47.234 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:39:47.235 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.235 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:47.235 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:39:47.235 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:39:47.236 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:39:47.236 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:39:47.236 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:39:47.237 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:39:47.237 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.886947+0000", 2022-08-01T19:39:47.237 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:39:47.237 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.237 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:39:47.238 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap": [], 2022-08-01T19:39:47.238 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap_items": [], 2022-08-01T19:39:47.239 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_temp": [], 2022-08-01T19:39:47.239 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_temp": [], 2022-08-01T19:39:47.240 INFO:teuthology.orchestra.run.smithi017.stdout: "blacklist": { 2022-08-01T19:39:47.240 INFO:teuthology.orchestra.run.smithi017.stdout: "172.21.15.77:6835/2879527612": "2022-08-02T19:39:41.053298+0000", 2022-08-01T19:39:47.241 INFO:teuthology.orchestra.run.smithi017.stdout: "172.21.15.77:6834/2879527612": "2022-08-02T19:39:41.053298+0000" 2022-08-01T19:39:47.241 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.242 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profiles": { 2022-08-01T19:39:47.242 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_data": { 2022-08-01T19:39:47.243 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-device-class": "", 2022-08-01T19:39:47.243 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:39:47.244 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-root": "default", 2022-08-01T19:39:47.244 INFO:teuthology.orchestra.run.smithi017.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T19:39:47.245 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:39:47.245 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "2", 2022-08-01T19:39:47.246 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:39:47.246 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:39:47.247 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van", 2022-08-01T19:39:47.247 INFO:teuthology.orchestra.run.smithi017.stdout: "w": "8" 2022-08-01T19:39:47.248 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.248 INFO:teuthology.orchestra.run.smithi017.stdout: "default": { 2022-08-01T19:39:47.248 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:39:47.249 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:39:47.249 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "1", 2022-08-01T19:39:47.249 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:39:47.249 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:39:47.250 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van" 2022-08-01T19:39:47.250 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:47.250 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:47.250 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps_queue": [], 2022-08-01T19:39:47.251 INFO:teuthology.orchestra.run.smithi017.stdout: "new_removed_snaps": [], 2022-08-01T19:39:47.251 INFO:teuthology.orchestra.run.smithi017.stdout: "new_purged_snaps": [], 2022-08-01T19:39:47.251 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_node_flags": {}, 2022-08-01T19:39:47.252 INFO:teuthology.orchestra.run.smithi017.stdout: "device_class_flags": {} 2022-08-01T19:39:47.252 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:39:47.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.954+0000 7f25ff59e700 1 -- 172.21.15.17:0/1233371018 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f25dc053aa0 msgr2=0x7f25dc055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:47.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.954+0000 7f25ff59e700 1 --2- 172.21.15.17:0/1233371018 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f25dc053aa0 0x7f25dc055f60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f25f4018000 tx=0x7f25f4014000).stop 2022-08-01T19:39:47.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.954+0000 7f25ff59e700 1 -- 172.21.15.17:0/1233371018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26000b2710 msgr2=0x7f2600156930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:47.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.954+0000 7f25ff59e700 1 --2- 172.21.15.17:0/1233371018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26000b2710 0x7f2600156930 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f25f8001110 tx=0x7f25f80079e0).stop 2022-08-01T19:39:47.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.954+0000 7f25ff59e700 1 -- 172.21.15.17:0/1233371018 shutdown_connections 2022-08-01T19:39:47.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.954+0000 7f25ff59e700 1 --2- 172.21.15.17:0/1233371018 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f25dc053aa0 0x7f25dc055f60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:47.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.954+0000 7f25ff59e700 1 --2- 172.21.15.17:0/1233371018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26000b2710 0x7f2600156930 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:47.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.954+0000 7f25ff59e700 1 --2- 172.21.15.17:0/1233371018 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26000b01a0 0x7f2600156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:47.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.954+0000 7f25ff59e700 1 --2- 172.21.15.17:0/1233371018 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f26000a8c50 0x7f26000636d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:47.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.954+0000 7f25ff59e700 1 -- 172.21.15.17:0/1233371018 >> 172.21.15.17:0/1233371018 conn(0x7f2600060310 msgr2=0x7f26000b3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:47.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.954+0000 7f25ff59e700 1 -- 172.21.15.17:0/1233371018 shutdown_connections 2022-08-01T19:39:47.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:46.954+0000 7f25ff59e700 1 -- 172.21.15.17:0/1233371018 wait complete. 2022-08-01T19:39:47.258 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:39:47.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.370+0000 7fdd2359e700 1 Processor -- start 2022-08-01T19:39:47.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.370+0000 7fdd2359e700 1 -- start start 2022-08-01T19:39:47.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.370+0000 7fdd2359e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdd240b2710 0x7fdd2414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:47.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.370+0000 7fdd2359e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fdd240b01a0 0x7fdd2414b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:47.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.370+0000 7fdd2359e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd240a7250 con 0x7fdd240afd60 2022-08-01T19:39:47.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.370+0000 7fdd2359e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd240a7480 con 0x7fdd240b01a0 2022-08-01T19:39:47.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.370+0000 7fdd2359e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd240a7720 con 0x7fdd240b2710 2022-08-01T19:39:47.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.370+0000 7fdd21d9b700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fdd240b01a0 0x7fdd2414b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:47.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.370+0000 7fdd2259c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdd240b2710 0x7fdd2414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:47.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.370+0000 7fdd21d9b700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fdd240b01a0 0x7fdd2414b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:38228/0 (socket says 172.21.15.17:38228) 2022-08-01T19:39:47.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.370+0000 7fdd21d9b700 1 -- 172.21.15.17:0/1458967228 learned_addr learned my addr 172.21.15.17:0/1458967228 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:47.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.370+0000 7fdd21d9b700 1 -- 172.21.15.17:0/1458967228 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdd240b2710 msgr2=0x7fdd2414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:47.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.370+0000 7fdd21d9b700 1 --2- 172.21.15.17:0/1458967228 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdd240b2710 0x7fdd2414afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:47.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.374+0000 7fdd21d9b700 1 -- 172.21.15.17:0/1458967228 >> v1:172.21.15.77:6789/0 conn(0x7fdd240afd60 legacy=0x7fdd240a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:47.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.374+0000 7fdd21d9b700 1 -- 172.21.15.17:0/1458967228 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd240a79c0 con 0x7fdd240b01a0 2022-08-01T19:39:47.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.374+0000 7fdd21d9b700 1 --2- 172.21.15.17:0/1458967228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd240b01a0 0x7fdd2414b4c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7fdd140032f0 tx=0x7fdd14009290).ready entity=mon.1 client_cookie=2287f50d0db3237a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:47.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.374+0000 7fdd2159a700 1 -- 172.21.15.17:0/1458967228 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd1400b8a0 con 0x7fdd240b01a0 2022-08-01T19:39:47.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.374+0000 7fdd2159a700 1 -- 172.21.15.17:0/1458967228 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1400ba00 con 0x7fdd240b01a0 2022-08-01T19:39:47.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.374+0000 7fdd2159a700 1 -- 172.21.15.17:0/1458967228 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd1400bb70 con 0x7fdd240b01a0 2022-08-01T19:39:47.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.374+0000 7fdd2359e700 1 -- 172.21.15.17:0/1458967228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd240b01a0 msgr2=0x7fdd2414b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:47.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.374+0000 7fdd2359e700 1 --2- 172.21.15.17:0/1458967228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd240b01a0 0x7fdd2414b4c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7fdd140032f0 tx=0x7fdd14009290).stop 2022-08-01T19:39:47.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.374+0000 7fdd2359e700 1 -- 172.21.15.17:0/1458967228 shutdown_connections 2022-08-01T19:39:47.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.374+0000 7fdd2359e700 1 --2- 172.21.15.17:0/1458967228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd240b01a0 0x7fdd2414b4c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:47.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.374+0000 7fdd2359e700 1 --2- 172.21.15.17:0/1458967228 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdd240b2710 0x7fdd2414afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:47.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.374+0000 7fdd2359e700 1 -- 172.21.15.17:0/1458967228 >> 172.21.15.17:0/1458967228 conn(0x7fdd24060310 msgr2=0x7fdd240b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:47.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.374+0000 7fdd2359e700 1 -- 172.21.15.17:0/1458967228 shutdown_connections 2022-08-01T19:39:47.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.374+0000 7fdd2359e700 1 -- 172.21.15.17:0/1458967228 wait complete. 2022-08-01T19:39:47.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.374+0000 7fdd2359e700 1 Processor -- start 2022-08-01T19:39:47.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.374+0000 7fdd2359e700 1 -- start start 2022-08-01T19:39:47.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.378+0000 7fdd2359e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdd240afd60 0x7fdd240b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:47.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.378+0000 7fdd2359e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd240b01a0 0x7fdd240b5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:47.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.378+0000 7fdd2359e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd240b2710 0x7fdd240ab2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:47.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.378+0000 7fdd2359e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd240b3920 con 0x7fdd240b2710 2022-08-01T19:39:47.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.378+0000 7fdd2359e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd240ac0a0 con 0x7fdd240b01a0 2022-08-01T19:39:47.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.378+0000 7fdd2359e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd240ac340 con 0x7fdd240afd60 2022-08-01T19:39:47.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.378+0000 7fdd22d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd240b2710 0x7fdd240ab2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:47.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.378+0000 7fdd22d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd240b2710 0x7fdd240ab2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40316/0 (socket says 172.21.15.17:40316) 2022-08-01T19:39:47.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.378+0000 7fdd22d9d700 1 -- 172.21.15.17:0/2256693229 learned_addr learned my addr 172.21.15.17:0/2256693229 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:47.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.378+0000 7fdd21d9b700 1 --2- 172.21.15.17:0/2256693229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd240b01a0 0x7fdd240b5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:47.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.378+0000 7fdd2259c700 1 --2- 172.21.15.17:0/2256693229 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdd240afd60 0x7fdd240b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:47.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.378+0000 7fdd22d9d700 1 -- 172.21.15.17:0/2256693229 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdd240afd60 msgr2=0x7fdd240b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:47.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.378+0000 7fdd22d9d700 1 --2- 172.21.15.17:0/2256693229 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdd240afd60 0x7fdd240b32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:47.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.378+0000 7fdd22d9d700 1 -- 172.21.15.17:0/2256693229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd240b01a0 msgr2=0x7fdd240b5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:47.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.378+0000 7fdd22d9d700 1 --2- 172.21.15.17:0/2256693229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd240b01a0 0x7fdd240b5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:47.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.378+0000 7fdd22d9d700 1 -- 172.21.15.17:0/2256693229 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd14009530 con 0x7fdd240b2710 2022-08-01T19:39:47.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.378+0000 7fdd22d9d700 1 --2- 172.21.15.17:0/2256693229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd240b2710 0x7fdd240ab2c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7fdd24061c10 tx=0x7fdd1c007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:47.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.378+0000 7fdd137fe700 1 -- 172.21.15.17:0/2256693229 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd1c012400 con 0x7fdd240b2710 2022-08-01T19:39:47.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.378+0000 7fdd137fe700 1 -- 172.21.15.17:0/2256693229 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1c012560 con 0x7fdd240b2710 2022-08-01T19:39:47.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.378+0000 7fdd137fe700 1 -- 172.21.15.17:0/2256693229 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd1c0128c0 con 0x7fdd240b2710 2022-08-01T19:39:47.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.378+0000 7fdd2359e700 1 -- 172.21.15.17:0/2256693229 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd240ac5e0 con 0x7fdd240b2710 2022-08-01T19:39:47.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.378+0000 7fdd2359e700 1 -- 172.21.15.17:0/2256693229 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd240b3c00 con 0x7fdd240b2710 2022-08-01T19:39:47.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.378+0000 7fdd2359e700 1 -- 172.21.15.17:0/2256693229 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd240ac770 con 0x7fdd240b2710 2022-08-01T19:39:47.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.378+0000 7fdd137fe700 1 -- 172.21.15.17:0/2256693229 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdd1c019030 con 0x7fdd240b2710 2022-08-01T19:39:47.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.382+0000 7fdd137fe700 1 --2- 172.21.15.17:0/2256693229 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdd00053aa0 0x7fdd00055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:47.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.382+0000 7fdd137fe700 1 -- 172.21.15.17:0/2256693229 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fdd1c06afe0 con 0x7fdd240b2710 2022-08-01T19:39:47.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.382+0000 7fdd2259c700 1 --2- 172.21.15.17:0/2256693229 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdd00053aa0 0x7fdd00055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:47.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.382+0000 7fdd2259c700 1 --2- 172.21.15.17:0/2256693229 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdd00053aa0 0x7fdd00055f60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7fdd1800b000 tx=0x7fdd18006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:47.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.386+0000 7fdd137fe700 1 -- 172.21.15.17:0/2256693229 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdd1c042080 con 0x7fdd240b2710 2022-08-01T19:39:47.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.550+0000 7fdd2359e700 1 -- 172.21.15.17:0/2256693229 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd240574e0 con 0x7fdd240b2710 2022-08-01T19:39:47.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.550+0000 7fdd137fe700 1 -- 172.21.15.17:0/2256693229 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+1559 (secure 0 0 0) 0x7fdd1c0456a0 con 0x7fdd240b2710 2022-08-01T19:39:47.550 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:39:47.550 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":32,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5414,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1130955771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1130955771},{"type":"v1","addr":"172.21.15.17:6835","nonce":1130955771}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":28},{"gid":5940,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3691532323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3691532323},{"type":"v1","addr":"172.21.15.17:6833","nonce":3691532323}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":28},{"gid":6154,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/588062822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":588062822},{"type":"v1","addr":"172.21.15.77:6835","nonce":588062822}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32}],"filesystems":[]} 2022-08-01T19:39:47.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.554+0000 7fdd2359e700 1 -- 172.21.15.17:0/2256693229 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdd00053aa0 msgr2=0x7fdd00055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:47.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.554+0000 7fdd2359e700 1 --2- 172.21.15.17:0/2256693229 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdd00053aa0 0x7fdd00055f60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7fdd1800b000 tx=0x7fdd18006df0).stop 2022-08-01T19:39:47.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.554+0000 7fdd2359e700 1 -- 172.21.15.17:0/2256693229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd240b2710 msgr2=0x7fdd240ab2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:47.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.554+0000 7fdd2359e700 1 --2- 172.21.15.17:0/2256693229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd240b2710 0x7fdd240ab2c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7fdd24061c10 tx=0x7fdd1c007bc0).stop 2022-08-01T19:39:47.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.554+0000 7fdd2359e700 1 -- 172.21.15.17:0/2256693229 shutdown_connections 2022-08-01T19:39:47.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.554+0000 7fdd2359e700 1 --2- 172.21.15.17:0/2256693229 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdd00053aa0 0x7fdd00055f60 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:47.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.554+0000 7fdd2359e700 1 --2- 172.21.15.17:0/2256693229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd240b2710 0x7fdd240ab2c0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:47.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.554+0000 7fdd2359e700 1 --2- 172.21.15.17:0/2256693229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd240b01a0 0x7fdd240b5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:47.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.554+0000 7fdd2359e700 1 --2- 172.21.15.17:0/2256693229 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdd240afd60 0x7fdd240b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:47.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.554+0000 7fdd2359e700 1 -- 172.21.15.17:0/2256693229 >> 172.21.15.17:0/2256693229 conn(0x7fdd24060310 msgr2=0x7fdd24149f70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:47.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.554+0000 7fdd2359e700 1 -- 172.21.15.17:0/2256693229 shutdown_connections 2022-08-01T19:39:47.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.554+0000 7fdd2359e700 1 -- 172.21.15.17:0/2256693229 wait complete. 2022-08-01T19:39:47.557 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 32 2022-08-01T19:39:47.563 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:39:47.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.662+0000 7ff7dca17700 1 Processor -- start 2022-08-01T19:39:47.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.662+0000 7ff7dca17700 1 -- start start 2022-08-01T19:39:47.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.666+0000 7ff7dca17700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ff7d8060330 0x7ff7d80b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:47.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.666+0000 7ff7dca17700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff7d805c6f0 0x7ff7d80a8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:47.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.666+0000 7ff7dca17700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7d80aa630 con 0x7ff7d80af550 2022-08-01T19:39:47.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.666+0000 7ff7dca17700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7d80b15e0 con 0x7ff7d805c6f0 2022-08-01T19:39:47.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.666+0000 7ff7dca17700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7d80af930 con 0x7ff7d8060330 2022-08-01T19:39:47.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.666+0000 7ff7d7fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff7d805c6f0 0x7ff7d80a8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:47.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.666+0000 7ff7d7fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff7d805c6f0 0x7ff7d80a8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40324/0 (socket says 172.21.15.17:40324) 2022-08-01T19:39:47.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.666+0000 7ff7d7fff700 1 -- 172.21.15.17:0/2047952107 learned_addr learned my addr 172.21.15.17:0/2047952107 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:47.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.666+0000 7ff7d6ffd700 1 --2- 172.21.15.17:0/2047952107 >> v2:172.21.15.77:3300/0 conn(0x7ff7d8060330 0x7ff7d80b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:47.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.666+0000 7ff7d7fff700 1 -- 172.21.15.17:0/2047952107 >> v2:172.21.15.77:3300/0 conn(0x7ff7d8060330 msgr2=0x7ff7d80b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:47.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.666+0000 7ff7d7fff700 1 --2- 172.21.15.17:0/2047952107 >> v2:172.21.15.77:3300/0 conn(0x7ff7d8060330 0x7ff7d80b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:47.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.666+0000 7ff7d7fff700 1 -- 172.21.15.17:0/2047952107 >> v1:172.21.15.77:6789/0 conn(0x7ff7d80af550 legacy=0x7ff7d81445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:47.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.666+0000 7ff7d7fff700 1 -- 172.21.15.17:0/2047952107 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d805fd10 con 0x7ff7d805c6f0 2022-08-01T19:39:47.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.666+0000 7ff7d7fff700 1 --2- 172.21.15.17:0/2047952107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7d805c6f0 0x7ff7d80a8d80 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7ff7d00032f0 tx=0x7ff7d0009290).ready entity=mon.0 client_cookie=dda89902d90c37b9 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:47.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.666+0000 7ff7d67fc700 1 -- 172.21.15.17:0/2047952107 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7d000b870 con 0x7ff7d805c6f0 2022-08-01T19:39:47.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.666+0000 7ff7d67fc700 1 -- 172.21.15.17:0/2047952107 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d000b9d0 con 0x7ff7d805c6f0 2022-08-01T19:39:47.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.666+0000 7ff7d67fc700 1 -- 172.21.15.17:0/2047952107 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7d000bb40 con 0x7ff7d805c6f0 2022-08-01T19:39:47.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.666+0000 7ff7dca17700 1 -- 172.21.15.17:0/2047952107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7d805c6f0 msgr2=0x7ff7d80a8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:47.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.666+0000 7ff7dca17700 1 --2- 172.21.15.17:0/2047952107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7d805c6f0 0x7ff7d80a8d80 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7ff7d00032f0 tx=0x7ff7d0009290).stop 2022-08-01T19:39:47.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.666+0000 7ff7dca17700 1 -- 172.21.15.17:0/2047952107 shutdown_connections 2022-08-01T19:39:47.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.666+0000 7ff7dca17700 1 --2- 172.21.15.17:0/2047952107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7d805c6f0 0x7ff7d80a8d80 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:47.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.666+0000 7ff7dca17700 1 --2- 172.21.15.17:0/2047952107 >> v2:172.21.15.77:3300/0 conn(0x7ff7d8060330 0x7ff7d80b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:47.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.666+0000 7ff7dca17700 1 -- 172.21.15.17:0/2047952107 >> 172.21.15.17:0/2047952107 conn(0x7ff7d80612d0 msgr2=0x7ff7d8062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:47.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.666+0000 7ff7dca17700 1 -- 172.21.15.17:0/2047952107 shutdown_connections 2022-08-01T19:39:47.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.666+0000 7ff7dca17700 1 -- 172.21.15.17:0/2047952107 wait complete. 2022-08-01T19:39:47.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.666+0000 7ff7dca17700 1 Processor -- start 2022-08-01T19:39:47.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.666+0000 7ff7dca17700 1 -- start start 2022-08-01T19:39:47.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.670+0000 7ff7dca17700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff7d805c6f0 0x7ff7d8144d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:47.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.670+0000 7ff7dca17700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7d8060330 0x7ff7d8150d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:47.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.670+0000 7ff7dca17700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7d80af550 0x7ff7d81467f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:47.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.670+0000 7ff7dca17700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d8067c70 con 0x7ff7d80af550 2022-08-01T19:39:47.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.670+0000 7ff7dca17700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d80641c0 con 0x7ff7d8060330 2022-08-01T19:39:47.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.670+0000 7ff7dca17700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d8064e90 con 0x7ff7d805c6f0 2022-08-01T19:39:47.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.670+0000 7ff7d7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7d80af550 0x7ff7d81467f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:47.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.670+0000 7ff7d7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7d80af550 0x7ff7d81467f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40330/0 (socket says 172.21.15.17:40330) 2022-08-01T19:39:47.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.670+0000 7ff7d7fff700 1 -- 172.21.15.17:0/2969789928 learned_addr learned my addr 172.21.15.17:0/2969789928 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:47.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.670+0000 7ff7d6ffd700 1 --2- 172.21.15.17:0/2969789928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7d8060330 0x7ff7d8150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:47.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.670+0000 7ff7d77fe700 1 --2- 172.21.15.17:0/2969789928 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff7d805c6f0 0x7ff7d8144d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:47.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.670+0000 7ff7d7fff700 1 -- 172.21.15.17:0/2969789928 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff7d805c6f0 msgr2=0x7ff7d8144d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:47.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.670+0000 7ff7d7fff700 1 --2- 172.21.15.17:0/2969789928 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff7d805c6f0 0x7ff7d8144d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:47.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.670+0000 7ff7d7fff700 1 -- 172.21.15.17:0/2969789928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7d8060330 msgr2=0x7ff7d8150d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:47.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.670+0000 7ff7d7fff700 1 --2- 172.21.15.17:0/2969789928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7d8060330 0x7ff7d8150d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:47.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.670+0000 7ff7d7fff700 1 -- 172.21.15.17:0/2969789928 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d0009530 con 0x7ff7d80af550 2022-08-01T19:39:47.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.670+0000 7ff7d7fff700 1 --2- 172.21.15.17:0/2969789928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7d80af550 0x7ff7d81467f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7ff7d00032f0 tx=0x7ff7d0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:47.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.670+0000 7ff7d4ff9700 1 -- 172.21.15.17:0/2969789928 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7d000b870 con 0x7ff7d80af550 2022-08-01T19:39:47.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.670+0000 7ff7d4ff9700 1 -- 172.21.15.17:0/2969789928 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d0021440 con 0x7ff7d80af550 2022-08-01T19:39:47.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.670+0000 7ff7dca17700 1 -- 172.21.15.17:0/2969789928 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d805f840 con 0x7ff7d80af550 2022-08-01T19:39:47.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.670+0000 7ff7d4ff9700 1 -- 172.21.15.17:0/2969789928 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7d00217a0 con 0x7ff7d80af550 2022-08-01T19:39:47.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.670+0000 7ff7dca17700 1 -- 172.21.15.17:0/2969789928 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d80aa810 con 0x7ff7d80af550 2022-08-01T19:39:47.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.674+0000 7ff7d4ff9700 1 -- 172.21.15.17:0/2969789928 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff7d0027020 con 0x7ff7d80af550 2022-08-01T19:39:47.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.674+0000 7ff7dca17700 1 -- 172.21.15.17:0/2969789928 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7d80aa9d0 con 0x7ff7d80af550 2022-08-01T19:39:47.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.674+0000 7ff7d4ff9700 1 --2- 172.21.15.17:0/2969789928 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff7bc053b00 0x7ff7bc055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:47.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.674+0000 7ff7d4ff9700 1 -- 172.21.15.17:0/2969789928 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5686+0+0 (secure 0 0 0) 0x7ff7d006d2b0 con 0x7ff7d80af550 2022-08-01T19:39:47.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.674+0000 7ff7d77fe700 1 --2- 172.21.15.17:0/2969789928 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff7bc053b00 0x7ff7bc055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:47.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.674+0000 7ff7d77fe700 1 --2- 172.21.15.17:0/2969789928 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff7bc053b00 0x7ff7bc055fc0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7ff7c800d000 tx=0x7ff7c8007b70).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:47.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.678+0000 7ff7d4ff9700 1 -- 172.21.15.17:0/2969789928 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff7d0044350 con 0x7ff7d80af550 2022-08-01T19:39:47.693 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:39:47.696 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:39:47.700 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:39:47.798 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:39:47.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.842+0000 7ff7dca17700 1 -- 172.21.15.17:0/2969789928 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7d805fa70 con 0x7ff7d80af550 2022-08-01T19:39:47.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.842+0000 7ff7d4ff9700 1 -- 172.21.15.17:0/2969789928 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+1559 (secure 0 0 0) 0x7ff7d0015090 con 0x7ff7d80af550 2022-08-01T19:39:47.843 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:39:47.844 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":32,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5414,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1130955771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1130955771},{"type":"v1","addr":"172.21.15.17:6835","nonce":1130955771}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":28},{"gid":5940,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3691532323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3691532323},{"type":"v1","addr":"172.21.15.17:6833","nonce":3691532323}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":28},{"gid":6154,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/588062822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":588062822},{"type":"v1","addr":"172.21.15.77:6835","nonce":588062822}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":32}],"filesystems":[]} 2022-08-01T19:39:47.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.842+0000 7ff7dca17700 1 -- 172.21.15.17:0/2969789928 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff7bc053b00 msgr2=0x7ff7bc055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:47.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.842+0000 7ff7dca17700 1 --2- 172.21.15.17:0/2969789928 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff7bc053b00 0x7ff7bc055fc0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7ff7c800d000 tx=0x7ff7c8007b70).stop 2022-08-01T19:39:47.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.842+0000 7ff7dca17700 1 -- 172.21.15.17:0/2969789928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7d80af550 msgr2=0x7ff7d81467f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:47.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.842+0000 7ff7dca17700 1 --2- 172.21.15.17:0/2969789928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7d80af550 0x7ff7d81467f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7ff7d00032f0 tx=0x7ff7d0016000).stop 2022-08-01T19:39:47.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.846+0000 7ff7dca17700 1 -- 172.21.15.17:0/2969789928 shutdown_connections 2022-08-01T19:39:47.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.846+0000 7ff7dca17700 1 --2- 172.21.15.17:0/2969789928 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff7bc053b00 0x7ff7bc055fc0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:47.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.846+0000 7ff7dca17700 1 --2- 172.21.15.17:0/2969789928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7d80af550 0x7ff7d81467f0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:47.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.846+0000 7ff7dca17700 1 --2- 172.21.15.17:0/2969789928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7d8060330 0x7ff7d8150d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:47.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.846+0000 7ff7dca17700 1 --2- 172.21.15.17:0/2969789928 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff7d805c6f0 0x7ff7d8144d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:47.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.846+0000 7ff7dca17700 1 -- 172.21.15.17:0/2969789928 >> 172.21.15.17:0/2969789928 conn(0x7ff7d80612d0 msgr2=0x7ff7d81410b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:47.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.846+0000 7ff7dca17700 1 -- 172.21.15.17:0/2969789928 shutdown_connections 2022-08-01T19:39:47.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:47.846+0000 7ff7dca17700 1 -- 172.21.15.17:0/2969789928 wait complete. 2022-08-01T19:39:47.848 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 32 2022-08-01T19:39:47.857 INFO:tasks.ceph.mds.a:Restarting daemon 2022-08-01T19:39:47.858 INFO:tasks.ceph.mds.a:Stopping old one... 2022-08-01T19:39:47.858 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-08-01T19:39:47.859 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:39:47.859 INFO:tasks.ceph.mds.c:Restarting daemon 2022-08-01T19:39:47.859 INFO:tasks.ceph.mds.c:Stopping old one... 2022-08-01T19:39:47.860 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-08-01T19:39:47.860 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:39:47.860 INFO:tasks.ceph.mds.b:Restarting daemon 2022-08-01T19:39:47.861 INFO:tasks.ceph.mds.b:Stopping old one... 2022-08-01T19:39:47.861 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-08-01T19:39:47.862 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:39:47.877 INFO:tasks.ceph.mds.b:Stopped 2022-08-01T19:39:47.877 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-08-01T19:39:47.881 INFO:tasks.ceph.mds.b:Started 2022-08-01T19:39:47.882 DEBUG:teuthology.parallel:result is None 2022-08-01T19:39:47.918 INFO:tasks.ceph.mds.c:Stopped 2022-08-01T19:39:47.919 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-08-01T19:39:47.921 INFO:tasks.ceph.mds.a:Stopped 2022-08-01T19:39:47.921 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-08-01T19:39:47.922 INFO:tasks.ceph.mds.c:Started 2022-08-01T19:39:47.923 DEBUG:teuthology.parallel:result is None 2022-08-01T19:39:47.924 INFO:tasks.ceph.mds.a:Started 2022-08-01T19:39:47.924 DEBUG:teuthology.parallel:result is None 2022-08-01T19:39:47.925 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-08-01T19:39:47.953 INFO:tasks.ceph.mds.b.smithi077.stdout:starting mds.b at 2022-08-01T19:39:47.994 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2022-08-01T19:39:48.008 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2022-08-01T19:39:48.018 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.018+0000 7f165a1bd700 1 Processor -- start 2022-08-01T19:39:48.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.018+0000 7f165a1bd700 1 -- start start 2022-08-01T19:39:48.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.018+0000 7f165a1bd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f16540af1b0 0x7f16540a9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:48.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.018+0000 7f165a1bd700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f16540aa390 0x7f16540667a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:48.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.018+0000 7f165a1bd700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16540aadf0 con 0x7f16541437f0 2022-08-01T19:39:48.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.018+0000 7f165a1bd700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f165405f9c0 con 0x7f16540aa390 2022-08-01T19:39:48.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.018+0000 7f165a1bd700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16540b1bd0 con 0x7f16540af1b0 2022-08-01T19:39:48.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.022+0000 7f1652ffd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f16540af1b0 0x7f16540a9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:48.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.022+0000 7f1653fff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f16540aa390 0x7f16540667a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:48.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.022+0000 7f1653fff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f16540aa390 0x7f16540667a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:38284/0 (socket says 172.21.15.17:38284) 2022-08-01T19:39:48.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.022+0000 7f1653fff700 1 -- 172.21.15.17:0/3636862120 learned_addr learned my addr 172.21.15.17:0/3636862120 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:48.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.022+0000 7f1652ffd700 1 -- 172.21.15.17:0/3636862120 >> v2:172.21.15.77:3300/0 conn(0x7f16540aa390 msgr2=0x7f16540667a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:48.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.022+0000 7f1652ffd700 1 --2- 172.21.15.17:0/3636862120 >> v2:172.21.15.77:3300/0 conn(0x7f16540aa390 0x7f16540667a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:48.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.022+0000 7f1652ffd700 1 -- 172.21.15.17:0/3636862120 >> v1:172.21.15.77:6789/0 conn(0x7f16541437f0 legacy=0x7f16540aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:48.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.022+0000 7f1652ffd700 1 -- 172.21.15.17:0/3636862120 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1654143fd0 con 0x7f16540af1b0 2022-08-01T19:39:48.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.022+0000 7f1653fff700 1 --2- 172.21.15.17:0/3636862120 >> v2:172.21.15.77:3300/0 conn(0x7f16540aa390 0x7f16540667a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:39:48.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.022+0000 7f1652ffd700 1 --2- 172.21.15.17:0/3636862120 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f16540af1b0 0x7f16540a9ea0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f1644002f70 tx=0x7f1644006df0).ready entity=mon.2 client_cookie=b009eefcf82d4de5 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:48.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.022+0000 7f16527fc700 1 -- 172.21.15.17:0/3636862120 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f164400d570 con 0x7f16540af1b0 2022-08-01T19:39:48.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.022+0000 7f16527fc700 1 -- 172.21.15.17:0/3636862120 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1644000e90 con 0x7f16540af1b0 2022-08-01T19:39:48.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.022+0000 7f16527fc700 1 -- 172.21.15.17:0/3636862120 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f164400d8c0 con 0x7f16540af1b0 2022-08-01T19:39:48.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.022+0000 7f165a1bd700 1 -- 172.21.15.17:0/3636862120 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f16540af1b0 msgr2=0x7f16540a9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:48.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.022+0000 7f165a1bd700 1 --2- 172.21.15.17:0/3636862120 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f16540af1b0 0x7f16540a9ea0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f1644002f70 tx=0x7f1644006df0).stop 2022-08-01T19:39:48.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.022+0000 7f165a1bd700 1 -- 172.21.15.17:0/3636862120 shutdown_connections 2022-08-01T19:39:48.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.022+0000 7f165a1bd700 1 --2- 172.21.15.17:0/3636862120 >> v2:172.21.15.77:3300/0 conn(0x7f16540aa390 0x7f16540667a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:48.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.022+0000 7f165a1bd700 1 --2- 172.21.15.17:0/3636862120 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f16540af1b0 0x7f16540a9ea0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:48.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.022+0000 7f165a1bd700 1 -- 172.21.15.17:0/3636862120 >> 172.21.15.17:0/3636862120 conn(0x7f16540b5570 msgr2=0x7f16540b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:48.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.022+0000 7f165a1bd700 1 -- 172.21.15.17:0/3636862120 shutdown_connections 2022-08-01T19:39:48.040 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.026+0000 7f165a1bd700 1 -- 172.21.15.17:0/3636862120 wait complete. 2022-08-01T19:39:48.040 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.026+0000 7f165a1bd700 1 Processor -- start 2022-08-01T19:39:48.040 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.026+0000 7f165a1bd700 1 -- start start 2022-08-01T19:39:48.040 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.026+0000 7f165a1bd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16540aa390 0x7f1654152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:48.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.026+0000 7f165a1bd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f16540af1b0 0x7f165413eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:48.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.026+0000 7f165a1bd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16541437f0 0x7f1654138a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:48.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.026+0000 7f165a1bd700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165413f9f0 con 0x7f16540aa390 2022-08-01T19:39:48.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.026+0000 7f165a1bd700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165413fc90 con 0x7f16541437f0 2022-08-01T19:39:48.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.026+0000 7f165a1bd700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1654153420 con 0x7f16540af1b0 2022-08-01T19:39:48.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.026+0000 7f16537fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16540aa390 0x7f1654152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:48.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.026+0000 7f16537fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16540aa390 0x7f1654152de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40364/0 (socket says 172.21.15.17:40364) 2022-08-01T19:39:48.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.026+0000 7f16537fe700 1 -- 172.21.15.17:0/3285341840 learned_addr learned my addr 172.21.15.17:0/3285341840 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:48.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.026+0000 7f1653fff700 1 --2- 172.21.15.17:0/3285341840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16541437f0 0x7f1654138a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:48.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.026+0000 7f1652ffd700 1 --2- 172.21.15.17:0/3285341840 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f16540af1b0 0x7f165413eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:48.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.026+0000 7f16537fe700 1 -- 172.21.15.17:0/3285341840 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f16540af1b0 msgr2=0x7f165413eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:48.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.026+0000 7f16537fe700 1 --2- 172.21.15.17:0/3285341840 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f16540af1b0 0x7f165413eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:48.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.026+0000 7f16537fe700 1 -- 172.21.15.17:0/3285341840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16541437f0 msgr2=0x7f1654138a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:48.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.026+0000 7f16537fe700 1 --2- 172.21.15.17:0/3285341840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16541437f0 0x7f1654138a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:48.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.026+0000 7f16537fe700 1 -- 172.21.15.17:0/3285341840 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f164800b1e0 con 0x7f16540aa390 2022-08-01T19:39:48.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.026+0000 7f16537fe700 1 --2- 172.21.15.17:0/3285341840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16540aa390 0x7f1654152de0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7f16541442e0 tx=0x7f164c00c300).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:48.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.026+0000 7f1650ff9700 1 -- 172.21.15.17:0/3285341840 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f164c001b10 con 0x7f16540aa390 2022-08-01T19:39:48.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.026+0000 7f1650ff9700 1 -- 172.21.15.17:0/3285341840 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f164c001c70 con 0x7f16540aa390 2022-08-01T19:39:48.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.026+0000 7f165a1bd700 1 -- 172.21.15.17:0/3285341840 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f164400b180 con 0x7f16540aa390 2022-08-01T19:39:48.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.026+0000 7f1650ff9700 1 -- 172.21.15.17:0/3285341840 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f164c002010 con 0x7f16540aa390 2022-08-01T19:39:48.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.026+0000 7f165a1bd700 1 -- 172.21.15.17:0/3285341840 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f165413d080 con 0x7f16540aa390 2022-08-01T19:39:48.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.030+0000 7f165a1bd700 1 -- 172.21.15.17:0/3285341840 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16540574e0 con 0x7f16540aa390 2022-08-01T19:39:48.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.030+0000 7f1650ff9700 1 -- 172.21.15.17:0/3285341840 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f164c017030 con 0x7f16540aa390 2022-08-01T19:39:48.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.030+0000 7f1650ff9700 1 --2- 172.21.15.17:0/3285341840 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1634053950 0x7f1634055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:48.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.030+0000 7f1650ff9700 1 -- 172.21.15.17:0/3285341840 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f164c06e7c0 con 0x7f16540aa390 2022-08-01T19:39:48.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.030+0000 7f1652ffd700 1 --2- 172.21.15.17:0/3285341840 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1634053950 0x7f1634055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:48.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.030+0000 7f1652ffd700 1 --2- 172.21.15.17:0/3285341840 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1634053950 0x7f1634055e10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f164401a000 tx=0x7f1644014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:48.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.038+0000 7f1650ff9700 1 -- 172.21.15.17:0/3285341840 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f164c045860 con 0x7f16540aa390 2022-08-01T19:39:48.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:48.170+0000 7f165a1bd700 1 -- 172.21.15.17:0/3285341840 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f1654069a70 con 0x7f16540aa390 2022-08-01T19:39:49.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.038+0000 7f1650ff9700 1 -- 172.21.15.17:0/3285341840 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v72) v1 ==== 97+0+0 (secure 0 0 0) 0x7f164c048e80 con 0x7f16540aa390 2022-08-01T19:39:49.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.042+0000 7f165a1bd700 1 -- 172.21.15.17:0/3285341840 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1634053950 msgr2=0x7f1634055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:49.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.042+0000 7f165a1bd700 1 --2- 172.21.15.17:0/3285341840 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1634053950 0x7f1634055e10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f164401a000 tx=0x7f1644014000).stop 2022-08-01T19:39:49.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.042+0000 7f165a1bd700 1 -- 172.21.15.17:0/3285341840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16540aa390 msgr2=0x7f1654152de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:49.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.042+0000 7f165a1bd700 1 --2- 172.21.15.17:0/3285341840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16540aa390 0x7f1654152de0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7f16541442e0 tx=0x7f164c00c300).stop 2022-08-01T19:39:49.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.042+0000 7f165a1bd700 1 -- 172.21.15.17:0/3285341840 shutdown_connections 2022-08-01T19:39:49.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.042+0000 7f165a1bd700 1 --2- 172.21.15.17:0/3285341840 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1634053950 0x7f1634055e10 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:49.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.042+0000 7f165a1bd700 1 --2- 172.21.15.17:0/3285341840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f16541437f0 0x7f1654138a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:49.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.042+0000 7f165a1bd700 1 --2- 172.21.15.17:0/3285341840 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f16540af1b0 0x7f165413eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:49.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.042+0000 7f165a1bd700 1 --2- 172.21.15.17:0/3285341840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16540aa390 0x7f1654152de0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:49.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.042+0000 7f165a1bd700 1 -- 172.21.15.17:0/3285341840 >> 172.21.15.17:0/3285341840 conn(0x7f16540b5570 msgr2=0x7f1654142ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:49.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.042+0000 7f165a1bd700 1 -- 172.21.15.17:0/3285341840 shutdown_connections 2022-08-01T19:39:49.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.042+0000 7f165a1bd700 1 -- 172.21.15.17:0/3285341840 wait complete. 2022-08-01T19:39:49.052 INFO:teuthology.orchestra.run.smithi017.stderr: removed all blacklist entries 2022-08-01T19:39:49.061 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-08-01T19:39:49.062 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-08-01T19:39:49.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.174+0000 7f3429fa0700 1 Processor -- start 2022-08-01T19:39:49.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.174+0000 7f3429fa0700 1 -- start start 2022-08-01T19:39:49.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.174+0000 7f3429fa0700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f34240b1f40 0x7f34240b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:49.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.174+0000 7f3429fa0700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f34240a7250 0x7f34241448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:49.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.174+0000 7f3429fa0700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34240b04e0 con 0x7f3424148530 2022-08-01T19:39:49.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.174+0000 7f3429fa0700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34240a9250 con 0x7f34240b1f40 2022-08-01T19:39:49.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.174+0000 7f3429fa0700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34240b0780 con 0x7f34240a7250 2022-08-01T19:39:49.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.174+0000 7f34237fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f34240b1f40 0x7f34240b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:49.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.174+0000 7f34237fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f34240b1f40 0x7f34240b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40376/0 (socket says 172.21.15.17:40376) 2022-08-01T19:39:49.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.174+0000 7f34237fe700 1 -- 172.21.15.17:0/321351057 learned_addr learned my addr 172.21.15.17:0/321351057 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:49.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.174+0000 7f3423fff700 1 --2- 172.21.15.17:0/321351057 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f34240a7250 0x7f34241448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:49.182 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.174+0000 7f34237fe700 1 -- 172.21.15.17:0/321351057 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f34240a7250 msgr2=0x7f34241448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:49.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.174+0000 7f34237fe700 1 --2- 172.21.15.17:0/321351057 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f34240a7250 0x7f34241448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:49.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.174+0000 7f34237fe700 1 -- 172.21.15.17:0/321351057 >> v1:172.21.15.77:6789/0 conn(0x7f3424148530 legacy=0x7f34240b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:49.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.174+0000 7f34237fe700 1 -- 172.21.15.17:0/321351057 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34240b0a20 con 0x7f34240b1f40 2022-08-01T19:39:49.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.174+0000 7f34237fe700 1 --2- 172.21.15.17:0/321351057 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34240b1f40 0x7f34240b1160 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f34241478c0 tx=0x7f340c006eb0).ready entity=mon.0 client_cookie=23bacfdc6c5e518a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:49.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.174+0000 7f34227fc700 1 -- 172.21.15.17:0/321351057 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f340c00d5c0 con 0x7f34240b1f40 2022-08-01T19:39:49.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.174+0000 7f34227fc700 1 -- 172.21.15.17:0/321351057 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340c000e90 con 0x7f34240b1f40 2022-08-01T19:39:49.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.174+0000 7f34227fc700 1 -- 172.21.15.17:0/321351057 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f340c00d950 con 0x7f34240b1f40 2022-08-01T19:39:49.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.174+0000 7f3429fa0700 1 -- 172.21.15.17:0/321351057 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34240b1f40 msgr2=0x7f34240b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:49.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.174+0000 7f3429fa0700 1 --2- 172.21.15.17:0/321351057 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34240b1f40 0x7f34240b1160 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f34241478c0 tx=0x7f340c006eb0).stop 2022-08-01T19:39:49.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.174+0000 7f3429fa0700 1 -- 172.21.15.17:0/321351057 shutdown_connections 2022-08-01T19:39:49.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.178+0000 7f3429fa0700 1 --2- 172.21.15.17:0/321351057 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f34240a7250 0x7f34241448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:49.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.178+0000 7f3429fa0700 1 --2- 172.21.15.17:0/321351057 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34240b1f40 0x7f34240b1160 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:49.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.178+0000 7f3429fa0700 1 -- 172.21.15.17:0/321351057 >> 172.21.15.17:0/321351057 conn(0x7f342405f840 msgr2=0x7f342405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:49.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.178+0000 7f3429fa0700 1 -- 172.21.15.17:0/321351057 shutdown_connections 2022-08-01T19:39:49.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.178+0000 7f3429fa0700 1 -- 172.21.15.17:0/321351057 wait complete. 2022-08-01T19:39:49.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.178+0000 7f3429fa0700 1 Processor -- start 2022-08-01T19:39:49.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.178+0000 7f3429fa0700 1 -- start start 2022-08-01T19:39:49.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.182+0000 7f3429fa0700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f34240a7250 0x7f34240ab020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:49.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.182+0000 7f3429fa0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34240b1f40 0x7f34240ab510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:49.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.182+0000 7f3429fa0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3424148530 0x7f3424063650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:49.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.182+0000 7f3429fa0700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34240a9ce0 con 0x7f34240b1f40 2022-08-01T19:39:49.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.182+0000 7f3429fa0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34240a9f80 con 0x7f3424148530 2022-08-01T19:39:49.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.182+0000 7f3429fa0700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34240aa220 con 0x7f34240a7250 2022-08-01T19:39:49.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.182+0000 7f3422ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34240b1f40 0x7f34240ab510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:49.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.182+0000 7f3422ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34240b1f40 0x7f34240ab510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40384/0 (socket says 172.21.15.17:40384) 2022-08-01T19:39:49.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.182+0000 7f3422ffd700 1 -- 172.21.15.17:0/458561498 learned_addr learned my addr 172.21.15.17:0/458561498 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:49.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.182+0000 7f3423fff700 1 --2- 172.21.15.17:0/458561498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3424148530 0x7f3424063650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:49.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.182+0000 7f34237fe700 1 --2- 172.21.15.17:0/458561498 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f34240a7250 0x7f34240ab020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:49.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.182+0000 7f3422ffd700 1 -- 172.21.15.17:0/458561498 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f34240a7250 msgr2=0x7f34240ab020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:49.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.182+0000 7f3422ffd700 1 --2- 172.21.15.17:0/458561498 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f34240a7250 0x7f34240ab020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:49.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.182+0000 7f3422ffd700 1 -- 172.21.15.17:0/458561498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3424148530 msgr2=0x7f3424063650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:49.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.182+0000 7f3422ffd700 1 --2- 172.21.15.17:0/458561498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3424148530 0x7f3424063650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:49.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.182+0000 7f3422ffd700 1 -- 172.21.15.17:0/458561498 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340c00b1e0 con 0x7f34240b1f40 2022-08-01T19:39:49.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.182+0000 7f3422ffd700 1 --2- 172.21.15.17:0/458561498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34240b1f40 0x7f34240ab510 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7f34240b3300 tx=0x7f34140014a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:49.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.182+0000 7f3420ff9700 1 -- 172.21.15.17:0/458561498 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3414010510 con 0x7f34240b1f40 2022-08-01T19:39:49.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.182+0000 7f3420ff9700 1 -- 172.21.15.17:0/458561498 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3414010670 con 0x7f34240b1f40 2022-08-01T19:39:49.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.182+0000 7f3429fa0700 1 -- 172.21.15.17:0/458561498 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34240aa4c0 con 0x7f34240b1f40 2022-08-01T19:39:49.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.182+0000 7f3420ff9700 1 -- 172.21.15.17:0/458561498 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34140107e0 con 0x7f34240b1f40 2022-08-01T19:39:49.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.182+0000 7f3429fa0700 1 -- 172.21.15.17:0/458561498 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34240aa960 con 0x7f34240b1f40 2022-08-01T19:39:49.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.182+0000 7f3420ff9700 1 -- 172.21.15.17:0/458561498 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3414010070 con 0x7f34240b1f40 2022-08-01T19:39:49.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.182+0000 7f3429fa0700 1 -- 172.21.15.17:0/458561498 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34240aa650 con 0x7f34240b1f40 2022-08-01T19:39:49.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.182+0000 7f3420ff9700 1 --2- 172.21.15.17:0/458561498 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3408053a70 0x7f3408055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:49.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.186+0000 7f3420ff9700 1 -- 172.21.15.17:0/458561498 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f341406b480 con 0x7f34240b1f40 2022-08-01T19:39:49.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.186+0000 7f34237fe700 1 --2- 172.21.15.17:0/458561498 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3408053a70 0x7f3408055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:49.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.186+0000 7f34237fe700 1 --2- 172.21.15.17:0/458561498 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3408053a70 0x7f3408055f30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f340c01a000 tx=0x7f340c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:49.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.190+0000 7f3420ff9700 1 -- 172.21.15.17:0/458561498 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3414042580 con 0x7f34240b1f40 2022-08-01T19:39:49.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.362+0000 7f3429fa0700 1 -- 172.21.15.17:0/458561498 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f34240574e0 con 0x7f34240b1f40 2022-08-01T19:39:49.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.366+0000 7f3420ff9700 1 -- 172.21.15.17:0/458561498 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f3414045ba0 con 0x7f34240b1f40 2022-08-01T19:39:49.364 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:39:49.365 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:39:49.365 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_dump": [ 2022-08-01T19:39:49.365 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:49.365 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.a", 2022-08-01T19:39:49.366 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhijotxORAAVc04rsrlFj0NShsyc6Qbhg==", 2022-08-01T19:39:49.366 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:39:49.366 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:39:49.366 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:39:49.367 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:39:49.367 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:39:49.367 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:49.368 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:49.368 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:49.369 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.b", 2022-08-01T19:39:49.369 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhiud2xBxAAcn6/O5ysdIp/oIbF5Y1dAw==", 2022-08-01T19:39:49.370 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:39:49.370 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:39:49.371 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:39:49.371 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:39:49.372 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:39:49.372 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:49.372 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:49.373 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:49.373 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.c", 2022-08-01T19:39:49.374 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+KuhiMzMbBBAAfdT5mtpYwboNL+7f2B09KQ==", 2022-08-01T19:39:49.374 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:39:49.375 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:39:49.375 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:39:49.376 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:39:49.376 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:39:49.377 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:49.377 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:49.378 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:49.378 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.0", 2022-08-01T19:39:49.379 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhia27/NRAAZXBp+nYelJVJ6Mge9r+3Vg==", 2022-08-01T19:39:49.379 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:39:49.380 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:39:49.380 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:39:49.381 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:39:49.381 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:49.382 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:49.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:49.383 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.1", 2022-08-01T19:39:49.383 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCBKuhiOggRGRAAvgA0MxwBJwmpXV6S9DTdjA==", 2022-08-01T19:39:49.384 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:39:49.384 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:39:49.385 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:39:49.385 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:39:49.386 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:49.386 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:49.387 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:49.387 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.2", 2022-08-01T19:39:49.388 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCDKuhi5PgOORAAhcORK4bW2T3dXPeYjcCm+g==", 2022-08-01T19:39:49.388 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:39:49.389 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:39:49.389 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:39:49.390 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:39:49.390 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:49.391 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:49.391 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:49.392 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.3", 2022-08-01T19:39:49.392 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCGKuhiutn6GxAAk2ucLuGbT5YV/R4xXQGh0w==", 2022-08-01T19:39:49.393 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:39:49.393 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:39:49.393 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:39:49.394 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:39:49.394 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:49.395 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:49.395 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:49.396 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.4", 2022-08-01T19:39:49.396 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCLKuhiGHTRHxAArr3TwXA0gsCAAF/Qs38wUQ==", 2022-08-01T19:39:49.397 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:39:49.397 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:39:49.398 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:39:49.407 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:39:49.408 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:49.408 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:49.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:49.409 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.5", 2022-08-01T19:39:49.409 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCOKuhiMMDbBBAAvTzBvOcv7pFWm/P/lNqB0w==", 2022-08-01T19:39:49.410 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:39:49.410 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:39:49.411 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:39:49.411 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:39:49.411 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:49.412 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:49.412 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:49.413 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.6", 2022-08-01T19:39:49.414 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCQKuhiG2H3IxAAufUcWqo9ULquxIrUD5vSfA==", 2022-08-01T19:39:49.414 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:39:49.415 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:39:49.415 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:39:49.416 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:39:49.416 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:49.417 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:49.417 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:49.418 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.7", 2022-08-01T19:39:49.419 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCTKuhiU50LChAAXuhrYBbHkk3eKzAPiEJFTg==", 2022-08-01T19:39:49.420 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:39:49.421 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:39:49.422 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:39:49.422 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:39:49.423 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:49.423 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:49.424 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:49.424 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.0", 2022-08-01T19:39:49.425 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhig3EnCxAANM20jSIQf/AKwuRX1qwT/A==", 2022-08-01T19:39:49.425 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:39:49.425 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:39:49.426 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow r", 2022-08-01T19:39:49.426 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T19:39:49.427 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:49.427 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:49.428 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:49.428 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.1", 2022-08-01T19:39:49.428 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+KuhicJR0DhAAivqEAZUhXnSwjVRwWm2bJg==", 2022-08-01T19:39:49.429 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:39:49.429 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:39:49.429 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow r", 2022-08-01T19:39:49.429 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T19:39:49.430 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:49.430 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:49.430 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:49.431 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.admin", 2022-08-01T19:39:49.431 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhiPTTGHxAAgfthCR6RX6j8F1YseFGNeA==", 2022-08-01T19:39:49.432 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:39:49.432 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:39:49.432 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:39:49.433 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:39:49.433 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:39:49.434 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:49.434 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:49.434 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:49.435 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.x", 2022-08-01T19:39:49.435 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9Kuhi2jvCNhAAyUxiGByZ+OZr8Y7dh3Qt5w==", 2022-08-01T19:39:49.436 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:39:49.436 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:39:49.436 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2022-08-01T19:39:49.437 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:39:49.437 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:49.437 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:49.438 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:49.438 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.y", 2022-08-01T19:39:49.439 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhiDJXpMhAA8aek67ydbRgNO0h+miws+w==", 2022-08-01T19:39:49.439 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:39:49.439 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:39:49.440 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2022-08-01T19:39:49.440 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:39:49.440 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:49.440 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:49.441 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:39:49.441 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:39:49.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.366+0000 7f3429fa0700 1 -- 172.21.15.17:0/458561498 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3408053a70 msgr2=0x7f3408055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:49.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.366+0000 7f3429fa0700 1 --2- 172.21.15.17:0/458561498 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3408053a70 0x7f3408055f30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f340c01a000 tx=0x7f340c015000).stop 2022-08-01T19:39:49.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.366+0000 7f3429fa0700 1 -- 172.21.15.17:0/458561498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34240b1f40 msgr2=0x7f34240ab510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:49.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.366+0000 7f3429fa0700 1 --2- 172.21.15.17:0/458561498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34240b1f40 0x7f34240ab510 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7f34240b3300 tx=0x7f34140014a0).stop 2022-08-01T19:39:49.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.366+0000 7f3429fa0700 1 -- 172.21.15.17:0/458561498 shutdown_connections 2022-08-01T19:39:49.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.366+0000 7f3429fa0700 1 --2- 172.21.15.17:0/458561498 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3408053a70 0x7f3408055f30 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:49.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.366+0000 7f3429fa0700 1 --2- 172.21.15.17:0/458561498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3424148530 0x7f3424063650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:49.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.366+0000 7f3429fa0700 1 --2- 172.21.15.17:0/458561498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34240b1f40 0x7f34240ab510 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:49.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.366+0000 7f3429fa0700 1 --2- 172.21.15.17:0/458561498 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f34240a7250 0x7f34240ab020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:49.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.366+0000 7f3429fa0700 1 -- 172.21.15.17:0/458561498 >> 172.21.15.17:0/458561498 conn(0x7f342405f840 msgr2=0x7f3424144b80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:49.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.366+0000 7f3429fa0700 1 -- 172.21.15.17:0/458561498 shutdown_connections 2022-08-01T19:39:49.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.366+0000 7f3429fa0700 1 -- 172.21.15.17:0/458561498 wait complete. 2022-08-01T19:39:49.447 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:39:49.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.554+0000 7f35b56b3700 1 Processor -- start 2022-08-01T19:39:49.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.554+0000 7f35b56b3700 1 -- start start 2022-08-01T19:39:49.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.558+0000 7f35b56b3700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f35b00b3190 0x7f35b0060ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:49.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.558+0000 7f35b56b3700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f35b00b2730 0x7f35b0061190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:49.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.558+0000 7f35b56b3700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35b00654c0 con 0x7f35b00666b0 2022-08-01T19:39:49.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.558+0000 7f35b56b3700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35b0062190 con 0x7f35b00b2730 2022-08-01T19:39:49.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.558+0000 7f35b56b3700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35b00b3570 con 0x7f35b00b3190 2022-08-01T19:39:49.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.558+0000 7f35af7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f35b00b2730 0x7f35b0061190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:49.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.558+0000 7f35b4eb2700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f35b00666b0 0x7f35b0066b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60820/0 (socket says 172.21.15.17:60820) 2022-08-01T19:39:49.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.558+0000 7f35b4eb2700 1 -- 172.21.15.17:0/2324617253 learned_addr learned my addr 172.21.15.17:0/2324617253 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:49.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.558+0000 7f35affff700 1 --2- 172.21.15.17:0/2324617253 >> v2:172.21.15.77:3300/0 conn(0x7f35b00b3190 0x7f35b0060ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:49.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.558+0000 7f35af7fe700 1 -- 172.21.15.17:0/2324617253 >> v2:172.21.15.77:3300/0 conn(0x7f35b00b3190 msgr2=0x7f35b0060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:49.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.558+0000 7f35af7fe700 1 --2- 172.21.15.17:0/2324617253 >> v2:172.21.15.77:3300/0 conn(0x7f35b00b3190 0x7f35b0060ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:49.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.558+0000 7f35af7fe700 1 -- 172.21.15.17:0/2324617253 >> v1:172.21.15.17:6789/0 conn(0x7f35b00666b0 legacy=0x7f35b0066b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:49.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.558+0000 7f35af7fe700 1 -- 172.21.15.17:0/2324617253 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b005fbf0 con 0x7f35b00b2730 2022-08-01T19:39:49.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.558+0000 7f35af7fe700 1 --2- 172.21.15.17:0/2324617253 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35b00b2730 0x7f35b0061190 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7f35a0002f70 tx=0x7f35a0006df0).ready entity=mon.0 client_cookie=13deee08d23fbfa3 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:49.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.558+0000 7f35aeffd700 1 -- 172.21.15.17:0/2324617253 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35a000d570 con 0x7f35b00b2730 2022-08-01T19:39:49.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.558+0000 7f35aeffd700 1 -- 172.21.15.17:0/2324617253 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a0000e90 con 0x7f35b00b2730 2022-08-01T19:39:49.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.558+0000 7f35aeffd700 1 -- 172.21.15.17:0/2324617253 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35a000d8c0 con 0x7f35b00b2730 2022-08-01T19:39:49.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.562+0000 7f35b56b3700 1 -- 172.21.15.17:0/2324617253 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35b00b2730 msgr2=0x7f35b0061190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:49.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.562+0000 7f35b56b3700 1 --2- 172.21.15.17:0/2324617253 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35b00b2730 0x7f35b0061190 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7f35a0002f70 tx=0x7f35a0006df0).stop 2022-08-01T19:39:49.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.562+0000 7f35b56b3700 1 -- 172.21.15.17:0/2324617253 shutdown_connections 2022-08-01T19:39:49.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.562+0000 7f35b56b3700 1 --2- 172.21.15.17:0/2324617253 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35b00b2730 0x7f35b0061190 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:49.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.562+0000 7f35b56b3700 1 --2- 172.21.15.17:0/2324617253 >> v2:172.21.15.77:3300/0 conn(0x7f35b00b3190 0x7f35b0060ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:49.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.562+0000 7f35b56b3700 1 -- 172.21.15.17:0/2324617253 >> 172.21.15.17:0/2324617253 conn(0x7f35b0063870 msgr2=0x7f35b0063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:49.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.562+0000 7f35b56b3700 1 -- 172.21.15.17:0/2324617253 shutdown_connections 2022-08-01T19:39:49.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.562+0000 7f35b56b3700 1 -- 172.21.15.17:0/2324617253 wait complete. 2022-08-01T19:39:49.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.562+0000 7f35b56b3700 1 Processor -- start 2022-08-01T19:39:49.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.562+0000 7f35b56b3700 1 -- start start 2022-08-01T19:39:49.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.562+0000 7f35b56b3700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35b00666b0 0x7f35b00a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:49.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.562+0000 7f35b56b3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35b00b2730 0x7f35b00a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:49.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.562+0000 7f35b56b3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35b00b3190 0x7f35b00afa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:49.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.562+0000 7f35b56b3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b00a45e0 con 0x7f35b00b3190 2022-08-01T19:39:49.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.562+0000 7f35b56b3700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b0153090 con 0x7f35b00b2730 2022-08-01T19:39:49.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.562+0000 7f35b56b3700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b00adfc0 con 0x7f35b00666b0 2022-08-01T19:39:49.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.562+0000 7f35b4eb2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35b00b3190 0x7f35b00afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:49.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.562+0000 7f35b4eb2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35b00b3190 0x7f35b00afa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40400/0 (socket says 172.21.15.17:40400) 2022-08-01T19:39:49.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.562+0000 7f35b4eb2700 1 -- 172.21.15.17:0/2408487804 learned_addr learned my addr 172.21.15.17:0/2408487804 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:49.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.562+0000 7f35af7fe700 1 --2- 172.21.15.17:0/2408487804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35b00b2730 0x7f35b00a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:49.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.562+0000 7f35affff700 1 --2- 172.21.15.17:0/2408487804 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35b00666b0 0x7f35b00a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:49.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.562+0000 7f35b4eb2700 1 -- 172.21.15.17:0/2408487804 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35b00666b0 msgr2=0x7f35b00a3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:49.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.562+0000 7f35b4eb2700 1 --2- 172.21.15.17:0/2408487804 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35b00666b0 0x7f35b00a3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:49.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.562+0000 7f35b4eb2700 1 -- 172.21.15.17:0/2408487804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35b00b2730 msgr2=0x7f35b00a40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:49.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.562+0000 7f35b4eb2700 1 --2- 172.21.15.17:0/2408487804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35b00b2730 0x7f35b00a40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:49.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.566+0000 7f35b4eb2700 1 -- 172.21.15.17:0/2408487804 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a000b180 con 0x7f35b00b3190 2022-08-01T19:39:49.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.566+0000 7f35b4eb2700 1 --2- 172.21.15.17:0/2408487804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35b00b3190 0x7f35b00afa40 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7f35b00a4720 tx=0x7f35a4007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:49.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.566+0000 7f35ad7fa700 1 -- 172.21.15.17:0/2408487804 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35a40070d0 con 0x7f35b00b3190 2022-08-01T19:39:49.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.566+0000 7f35ad7fa700 1 -- 172.21.15.17:0/2408487804 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a4007230 con 0x7f35b00b3190 2022-08-01T19:39:49.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.566+0000 7f35b56b3700 1 -- 172.21.15.17:0/2408487804 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b01532c0 con 0x7f35b00b3190 2022-08-01T19:39:49.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.566+0000 7f35ad7fa700 1 -- 172.21.15.17:0/2408487804 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35a4005610 con 0x7f35b00b3190 2022-08-01T19:39:49.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.566+0000 7f35b56b3700 1 -- 172.21.15.17:0/2408487804 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b00ae260 con 0x7f35b00b3190 2022-08-01T19:39:49.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.566+0000 7f35b56b3700 1 -- 172.21.15.17:0/2408487804 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35b00ab370 con 0x7f35b00b3190 2022-08-01T19:39:49.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.566+0000 7f35ad7fa700 1 -- 172.21.15.17:0/2408487804 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f35a401a070 con 0x7f35b00b3190 2022-08-01T19:39:49.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.566+0000 7f35ad7fa700 1 --2- 172.21.15.17:0/2408487804 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3590053a70 0x7f3590055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:49.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.566+0000 7f35ad7fa700 1 -- 172.21.15.17:0/2408487804 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f35a406aea0 con 0x7f35b00b3190 2022-08-01T19:39:49.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.566+0000 7f35affff700 1 --2- 172.21.15.17:0/2408487804 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3590053a70 0x7f3590055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:49.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.566+0000 7f35affff700 1 --2- 172.21.15.17:0/2408487804 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3590053a70 0x7f3590055f30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f35a800b000 tx=0x7f35a80091d0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:49.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.574+0000 7f35ad7fa700 1 -- 172.21.15.17:0/2408487804 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f35a4041fa0 con 0x7f35b00b3190 2022-08-01T19:39:49.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.702+0000 7f35b56b3700 1 -- 172.21.15.17:0/2408487804 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35b005fe90 con 0x7f35b00b3190 2022-08-01T19:39:49.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.702+0000 7f35ad7fa700 1 -- 172.21.15.17:0/2408487804 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+13415 (secure 0 0 0) 0x7f35a40455c0 con 0x7f35b00b3190 2022-08-01T19:39:49.703 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:39:49.704 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":72,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:39:49.032670+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:39:49.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.706+0000 7f35b56b3700 1 -- 172.21.15.17:0/2408487804 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3590053a70 msgr2=0x7f3590055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:49.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.706+0000 7f35b56b3700 1 --2- 172.21.15.17:0/2408487804 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3590053a70 0x7f3590055f30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f35a800b000 tx=0x7f35a80091d0).stop 2022-08-01T19:39:49.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.706+0000 7f35b56b3700 1 -- 172.21.15.17:0/2408487804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35b00b3190 msgr2=0x7f35b00afa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:49.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.706+0000 7f35b56b3700 1 --2- 172.21.15.17:0/2408487804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35b00b3190 0x7f35b00afa40 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7f35b00a4720 tx=0x7f35a4007bc0).stop 2022-08-01T19:39:49.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.706+0000 7f35b56b3700 1 -- 172.21.15.17:0/2408487804 shutdown_connections 2022-08-01T19:39:49.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.706+0000 7f35b56b3700 1 --2- 172.21.15.17:0/2408487804 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3590053a70 0x7f3590055f30 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:49.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.706+0000 7f35b56b3700 1 --2- 172.21.15.17:0/2408487804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35b00b3190 0x7f35b00afa40 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:49.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.706+0000 7f35b56b3700 1 --2- 172.21.15.17:0/2408487804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35b00b2730 0x7f35b00a40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:49.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.706+0000 7f35b56b3700 1 --2- 172.21.15.17:0/2408487804 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35b00666b0 0x7f35b00a3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:49.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.706+0000 7f35b56b3700 1 -- 172.21.15.17:0/2408487804 >> 172.21.15.17:0/2408487804 conn(0x7f35b0063870 msgr2=0x7f35b0142bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:49.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.706+0000 7f35b56b3700 1 -- 172.21.15.17:0/2408487804 shutdown_connections 2022-08-01T19:39:49.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.706+0000 7f35b56b3700 1 -- 172.21.15.17:0/2408487804 wait complete. 2022-08-01T19:39:49.719 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-08-01T19:39:49.720 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:39:49.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.834+0000 7fcd1e951700 1 Processor -- start 2022-08-01T19:39:49.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.834+0000 7fcd1e951700 1 -- start start 2022-08-01T19:39:49.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.834+0000 7fcd1e951700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcd180b2710 0x7fcd1814afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:49.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.834+0000 7fcd1e951700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fcd180a8c50 0x7fcd1814b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:49.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.834+0000 7fcd1e951700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd180a7250 con 0x7fcd180b01a0 2022-08-01T19:39:49.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.834+0000 7fcd1e951700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd180a7480 con 0x7fcd180a8c50 2022-08-01T19:39:49.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.834+0000 7fcd1e951700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd180a7720 con 0x7fcd180b2710 2022-08-01T19:39:49.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.838+0000 7fcd1d14e700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fcd180b01a0 0x7fcd180a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60830/0 (socket says 172.21.15.17:60830) 2022-08-01T19:39:49.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.838+0000 7fcd1d14e700 1 -- 172.21.15.17:0/1505093303 learned_addr learned my addr 172.21.15.17:0/1505093303 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:49.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.838+0000 7fcd1e150700 1 --2- 172.21.15.17:0/1505093303 >> v2:172.21.15.77:3300/0 conn(0x7fcd180a8c50 0x7fcd1814b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:49.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.838+0000 7fcd1d94f700 1 --2- 172.21.15.17:0/1505093303 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcd180b2710 0x7fcd1814afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:49.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.838+0000 7fcd1c94d700 1 -- 172.21.15.17:0/1505093303 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 513551173 0 0) 0x7fcd180a7250 con 0x7fcd180b01a0 2022-08-01T19:39:49.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.838+0000 7fcd1c94d700 1 -- 172.21.15.17:0/1505093303 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd00003260 con 0x7fcd180b01a0 2022-08-01T19:39:49.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.838+0000 7fcd1e150700 1 -- 172.21.15.17:0/1505093303 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcd180b2710 msgr2=0x7fcd1814afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:49.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.838+0000 7fcd1e150700 1 --2- 172.21.15.17:0/1505093303 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcd180b2710 0x7fcd1814afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:49.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.838+0000 7fcd1e150700 1 -- 172.21.15.17:0/1505093303 >> v1:172.21.15.17:6789/0 conn(0x7fcd180b01a0 legacy=0x7fcd180a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:49.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.838+0000 7fcd1c94d700 1 -- 172.21.15.17:0/1505093303 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4088638783 0 0) 0x7fcd00003260 con 0x7fcd180b01a0 2022-08-01T19:39:49.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.838+0000 7fcd1e150700 1 -- 172.21.15.17:0/1505093303 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd180a79c0 con 0x7fcd180a8c50 2022-08-01T19:39:49.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.838+0000 7fcd1e150700 1 --2- 172.21.15.17:0/1505093303 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd180a8c50 0x7fcd1814b4c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7fcd14001800 tx=0x7fcd100091d0).ready entity=mon.1 client_cookie=2de49d8d40b56421 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:49.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.838+0000 7fcd1c94d700 1 -- 172.21.15.17:0/1505093303 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd14001660 con 0x7fcd180a8c50 2022-08-01T19:39:49.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.838+0000 7fcd1c94d700 1 -- 172.21.15.17:0/1505093303 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1000f510 con 0x7fcd180a8c50 2022-08-01T19:39:49.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.838+0000 7fcd1c94d700 1 -- 172.21.15.17:0/1505093303 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd1000f680 con 0x7fcd180a8c50 2022-08-01T19:39:49.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.838+0000 7fcd1e951700 1 -- 172.21.15.17:0/1505093303 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd180a8c50 msgr2=0x7fcd1814b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:49.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.838+0000 7fcd1e951700 1 --2- 172.21.15.17:0/1505093303 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd180a8c50 0x7fcd1814b4c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7fcd14001800 tx=0x7fcd100091d0).stop 2022-08-01T19:39:49.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.838+0000 7fcd1e951700 1 -- 172.21.15.17:0/1505093303 shutdown_connections 2022-08-01T19:39:49.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.838+0000 7fcd1e951700 1 --2- 172.21.15.17:0/1505093303 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd180a8c50 0x7fcd1814b4c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:49.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.838+0000 7fcd1e951700 1 --2- 172.21.15.17:0/1505093303 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcd180b2710 0x7fcd1814afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:49.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.838+0000 7fcd1e951700 1 -- 172.21.15.17:0/1505093303 >> 172.21.15.17:0/1505093303 conn(0x7fcd18060310 msgr2=0x7fcd180b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:49.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.838+0000 7fcd1e951700 1 -- 172.21.15.17:0/1505093303 shutdown_connections 2022-08-01T19:39:49.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.838+0000 7fcd1e951700 1 -- 172.21.15.17:0/1505093303 wait complete. 2022-08-01T19:39:49.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.838+0000 7fcd1e951700 1 Processor -- start 2022-08-01T19:39:49.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.842+0000 7fcd1e951700 1 -- start start 2022-08-01T19:39:49.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.842+0000 7fcd1e951700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd180a8c50 0x7fcd1805c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:49.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.842+0000 7fcd1e951700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcd180b01a0 0x7fcd180ac0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:49.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.842+0000 7fcd1e951700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd180b2710 0x7fcd180ac5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:49.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.842+0000 7fcd1e951700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd18063510 con 0x7fcd180a8c50 2022-08-01T19:39:49.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.842+0000 7fcd1e951700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd180abc60 con 0x7fcd180b2710 2022-08-01T19:39:49.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.842+0000 7fcd1e951700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd180acaa0 con 0x7fcd180b01a0 2022-08-01T19:39:49.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.842+0000 7fcd1d94f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd180a8c50 0x7fcd1805c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:49.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.842+0000 7fcd1d94f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd180a8c50 0x7fcd1805c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40410/0 (socket says 172.21.15.17:40410) 2022-08-01T19:39:49.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.842+0000 7fcd1d94f700 1 -- 172.21.15.17:0/1841820531 learned_addr learned my addr 172.21.15.17:0/1841820531 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:49.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.842+0000 7fcd1e150700 1 --2- 172.21.15.17:0/1841820531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd180b2710 0x7fcd180ac5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:49.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.842+0000 7fcd1d14e700 1 --2- 172.21.15.17:0/1841820531 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcd180b01a0 0x7fcd180ac0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:49.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.842+0000 7fcd1e150700 1 -- 172.21.15.17:0/1841820531 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcd180b01a0 msgr2=0x7fcd180ac0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:49.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.842+0000 7fcd1e150700 1 --2- 172.21.15.17:0/1841820531 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcd180b01a0 0x7fcd180ac0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:49.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.842+0000 7fcd1e150700 1 -- 172.21.15.17:0/1841820531 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd180a8c50 msgr2=0x7fcd1805c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:49.870 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.842+0000 7fcd1e150700 1 --2- 172.21.15.17:0/1841820531 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd180a8c50 0x7fcd1805c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:49.870 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.842+0000 7fcd1e150700 1 -- 172.21.15.17:0/1841820531 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd10009470 con 0x7fcd180b2710 2022-08-01T19:39:49.871 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.842+0000 7fcd1d94f700 1 --2- 172.21.15.17:0/1841820531 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd180a8c50 0x7fcd1805c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:39:49.871 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.842+0000 7fcd1e150700 1 --2- 172.21.15.17:0/1841820531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd180b2710 0x7fcd180ac5b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7fcd14001800 tx=0x7fcd10016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:49.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.842+0000 7fcd06ffd700 1 -- 172.21.15.17:0/1841820531 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd100032d0 con 0x7fcd180b2710 2022-08-01T19:39:49.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.842+0000 7fcd06ffd700 1 -- 172.21.15.17:0/1841820531 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd10011b80 con 0x7fcd180b2710 2022-08-01T19:39:49.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.842+0000 7fcd1e951700 1 -- 172.21.15.17:0/1841820531 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd1815a420 con 0x7fcd180b2710 2022-08-01T19:39:49.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.842+0000 7fcd06ffd700 1 -- 172.21.15.17:0/1841820531 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd10021440 con 0x7fcd180b2710 2022-08-01T19:39:49.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.842+0000 7fcd1e951700 1 -- 172.21.15.17:0/1841820531 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd180a92d0 con 0x7fcd180b2710 2022-08-01T19:39:49.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.846+0000 7fcd1e951700 1 -- 172.21.15.17:0/1841820531 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd180aaa40 con 0x7fcd180b2710 2022-08-01T19:39:49.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.846+0000 7fcd06ffd700 1 -- 172.21.15.17:0/1841820531 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcd10027020 con 0x7fcd180b2710 2022-08-01T19:39:49.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.846+0000 7fcd06ffd700 1 --2- 172.21.15.17:0/1841820531 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fccf8053b00 0x7fccf8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:49.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.846+0000 7fcd06ffd700 1 -- 172.21.15.17:0/1841820531 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fcd1006d5d0 con 0x7fcd180b2710 2022-08-01T19:39:49.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.846+0000 7fcd1d94f700 1 --2- 172.21.15.17:0/1841820531 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fccf8053b00 0x7fccf8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:49.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.846+0000 7fcd1d94f700 1 --2- 172.21.15.17:0/1841820531 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fccf8053b00 0x7fccf8055fc0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7fcd0800b000 tx=0x7fcd08006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:49.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.854+0000 7fcd06ffd700 1 -- 172.21.15.17:0/1841820531 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcd100445c0 con 0x7fcd180b2710 2022-08-01T19:39:49.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.986+0000 7fcd1e951700 1 -- 172.21.15.17:0/1841820531 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fcd180abda0 con 0x7fcd180b2710 2022-08-01T19:39:49.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.986+0000 7fcd06ffd700 1 -- 172.21.15.17:0/1841820531 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v72) v1 ==== 110+0+10 (secure 0 0 0) 0x7fcd10015020 con 0x7fcd180b2710 2022-08-01T19:39:49.985 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:39:49.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.986+0000 7fcd1e951700 1 -- 172.21.15.17:0/1841820531 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fccf8053b00 msgr2=0x7fccf8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:49.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.986+0000 7fcd1e951700 1 --2- 172.21.15.17:0/1841820531 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fccf8053b00 0x7fccf8055fc0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7fcd0800b000 tx=0x7fcd08006df0).stop 2022-08-01T19:39:49.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.986+0000 7fcd1e951700 1 -- 172.21.15.17:0/1841820531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd180b2710 msgr2=0x7fcd180ac5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:49.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.986+0000 7fcd1e951700 1 --2- 172.21.15.17:0/1841820531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd180b2710 0x7fcd180ac5b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7fcd14001800 tx=0x7fcd10016000).stop 2022-08-01T19:39:49.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.986+0000 7fcd1e951700 1 -- 172.21.15.17:0/1841820531 shutdown_connections 2022-08-01T19:39:49.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.986+0000 7fcd1e951700 1 --2- 172.21.15.17:0/1841820531 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fccf8053b00 0x7fccf8055fc0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:49.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.986+0000 7fcd1e951700 1 --2- 172.21.15.17:0/1841820531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd180b2710 0x7fcd180ac5b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:49.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.986+0000 7fcd1e951700 1 --2- 172.21.15.17:0/1841820531 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcd180b01a0 0x7fcd180ac0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:49.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.986+0000 7fcd1e951700 1 --2- 172.21.15.17:0/1841820531 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd180a8c50 0x7fcd1805c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:49.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.986+0000 7fcd1e951700 1 -- 172.21.15.17:0/1841820531 >> 172.21.15.17:0/1841820531 conn(0x7fcd18060310 msgr2=0x7fcd18149b70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:49.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.986+0000 7fcd1e951700 1 -- 172.21.15.17:0/1841820531 shutdown_connections 2022-08-01T19:39:49.991 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:49.986+0000 7fcd1e951700 1 -- 172.21.15.17:0/1841820531 wait complete. 2022-08-01T19:39:50.000 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:39:50.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.110+0000 7f918528d700 1 Processor -- start 2022-08-01T19:39:50.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.110+0000 7f918528d700 1 -- start start 2022-08-01T19:39:50.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.110+0000 7f918528d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9180066780 0x7f9180066b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:50.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.110+0000 7f918528d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9180140e40 0x7f918013e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:50.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.110+0000 7f918528d700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f918005f8e0 con 0x7f91800b1bc0 2022-08-01T19:39:50.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.110+0000 7f918528d700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91800b1630 con 0x7f9180066780 2022-08-01T19:39:50.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.110+0000 7f918528d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f918005fb10 con 0x7f9180140e40 2022-08-01T19:39:50.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.110+0000 7f917ffff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9180066780 0x7f9180066b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:50.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.110+0000 7f917ffff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9180066780 0x7f9180066b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40418/0 (socket says 172.21.15.17:40418) 2022-08-01T19:39:50.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.110+0000 7f917ffff700 1 -- 172.21.15.17:0/1762046120 learned_addr learned my addr 172.21.15.17:0/1762046120 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:50.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.110+0000 7f9184a8c700 1 --2- 172.21.15.17:0/1762046120 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9180140e40 0x7f918013e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:50.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.114+0000 7f917ffff700 1 -- 172.21.15.17:0/1762046120 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9180140e40 msgr2=0x7f918013e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:50.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.114+0000 7f917ffff700 1 --2- 172.21.15.17:0/1762046120 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9180140e40 0x7f918013e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:50.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.114+0000 7f917ffff700 1 -- 172.21.15.17:0/1762046120 >> v1:172.21.15.77:6789/0 conn(0x7f91800b1bc0 legacy=0x7f918005ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:50.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.114+0000 7f917ffff700 1 -- 172.21.15.17:0/1762046120 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91800b18d0 con 0x7f9180066780 2022-08-01T19:39:50.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.114+0000 7f917ffff700 1 --2- 172.21.15.17:0/1762046120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9180066780 0x7f9180066b60 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f9170002f70 tx=0x7f9170006eb0).ready entity=mon.0 client_cookie=5799e93894d6b449 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:50.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.114+0000 7f917effd700 1 -- 172.21.15.17:0/1762046120 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f917000d570 con 0x7f9180066780 2022-08-01T19:39:50.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.114+0000 7f917effd700 1 -- 172.21.15.17:0/1762046120 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9170000e90 con 0x7f9180066780 2022-08-01T19:39:50.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.114+0000 7f917effd700 1 -- 172.21.15.17:0/1762046120 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f917000d900 con 0x7f9180066780 2022-08-01T19:39:50.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.114+0000 7f918528d700 1 -- 172.21.15.17:0/1762046120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9180066780 msgr2=0x7f9180066b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:50.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.114+0000 7f918528d700 1 --2- 172.21.15.17:0/1762046120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9180066780 0x7f9180066b60 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f9170002f70 tx=0x7f9170006eb0).stop 2022-08-01T19:39:50.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.114+0000 7f918528d700 1 -- 172.21.15.17:0/1762046120 shutdown_connections 2022-08-01T19:39:50.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.114+0000 7f918528d700 1 --2- 172.21.15.17:0/1762046120 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9180140e40 0x7f918013e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:50.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.114+0000 7f918528d700 1 --2- 172.21.15.17:0/1762046120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9180066780 0x7f9180066b60 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:50.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.114+0000 7f918528d700 1 -- 172.21.15.17:0/1762046120 >> 172.21.15.17:0/1762046120 conn(0x7f9180063870 msgr2=0x7f9180063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:50.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.114+0000 7f918528d700 1 -- 172.21.15.17:0/1762046120 shutdown_connections 2022-08-01T19:39:50.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.114+0000 7f918528d700 1 -- 172.21.15.17:0/1762046120 wait complete. 2022-08-01T19:39:50.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.114+0000 7f918528d700 1 Processor -- start 2022-08-01T19:39:50.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.114+0000 7f918528d700 1 -- start start 2022-08-01T19:39:50.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.118+0000 7f918528d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9180066780 0x7f91800a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:50.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.118+0000 7f918528d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91800b1bc0 0x7f91800a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:50.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.118+0000 7f918528d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9180140e40 0x7f91800afc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:50.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.118+0000 7f918528d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91801531b0 con 0x7f91800b1bc0 2022-08-01T19:39:50.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.118+0000 7f918528d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91800a2250 con 0x7f9180066780 2022-08-01T19:39:50.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.118+0000 7f918528d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9180152530 con 0x7f9180140e40 2022-08-01T19:39:50.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.118+0000 7f917f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91800b1bc0 0x7f91800a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:50.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.118+0000 7f917f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91800b1bc0 0x7f91800a1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40426/0 (socket says 172.21.15.17:40426) 2022-08-01T19:39:50.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.118+0000 7f917f7fe700 1 -- 172.21.15.17:0/3155733361 learned_addr learned my addr 172.21.15.17:0/3155733361 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:50.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.118+0000 7f917ffff700 1 --2- 172.21.15.17:0/3155733361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9180066780 0x7f91800a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:50.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.118+0000 7f9184a8c700 1 --2- 172.21.15.17:0/3155733361 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9180140e40 0x7f91800afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:50.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.118+0000 7f917f7fe700 1 -- 172.21.15.17:0/3155733361 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9180140e40 msgr2=0x7f91800afc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:50.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.118+0000 7f917f7fe700 1 --2- 172.21.15.17:0/3155733361 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9180140e40 0x7f91800afc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:50.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.118+0000 7f917f7fe700 1 -- 172.21.15.17:0/3155733361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9180066780 msgr2=0x7f91800a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:50.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.118+0000 7f917f7fe700 1 --2- 172.21.15.17:0/3155733361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9180066780 0x7f91800a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:50.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.118+0000 7f917f7fe700 1 -- 172.21.15.17:0/3155733361 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f917000b1e0 con 0x7f91800b1bc0 2022-08-01T19:39:50.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.118+0000 7f917f7fe700 1 --2- 172.21.15.17:0/3155733361 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91800b1bc0 0x7f91800a1d60 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f9180063400 tx=0x7f9174007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:50.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.118+0000 7f917d7fa700 1 -- 172.21.15.17:0/3155733361 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9174012400 con 0x7f91800b1bc0 2022-08-01T19:39:50.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.118+0000 7f917d7fa700 1 -- 172.21.15.17:0/3155733361 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9174012560 con 0x7f91800b1bc0 2022-08-01T19:39:50.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.118+0000 7f918528d700 1 -- 172.21.15.17:0/3155733361 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91801527d0 con 0x7f91800b1bc0 2022-08-01T19:39:50.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.118+0000 7f917d7fa700 1 -- 172.21.15.17:0/3155733361 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91740128c0 con 0x7f91800b1bc0 2022-08-01T19:39:50.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.118+0000 7f918528d700 1 -- 172.21.15.17:0/3155733361 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9180152220 con 0x7f91800b1bc0 2022-08-01T19:39:50.134 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.118+0000 7f918528d700 1 -- 172.21.15.17:0/3155733361 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9180155b30 con 0x7f91800b1bc0 2022-08-01T19:39:50.134 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.118+0000 7f917d7fa700 1 -- 172.21.15.17:0/3155733361 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9174019030 con 0x7f91800b1bc0 2022-08-01T19:39:50.135 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.122+0000 7f917d7fa700 1 --2- 172.21.15.17:0/3155733361 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9160053950 0x7f9160055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:50.135 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.122+0000 7f917d7fa700 1 -- 172.21.15.17:0/3155733361 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f917406b010 con 0x7f91800b1bc0 2022-08-01T19:39:50.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.122+0000 7f917ffff700 1 --2- 172.21.15.17:0/3155733361 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9160053950 0x7f9160055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:50.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.122+0000 7f917ffff700 1 --2- 172.21.15.17:0/3155733361 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9160053950 0x7f9160055e10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f917001c000 tx=0x7f9170015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:50.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.130+0000 7f917d7fa700 1 -- 172.21.15.17:0/3155733361 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9180155b30 con 0x7f91800b1bc0 2022-08-01T19:39:50.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.250+0000 7f918528d700 1 -- 172.21.15.17:0/3155733361 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9180155b30 con 0x7f91800b1bc0 2022-08-01T19:39:50.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.254+0000 7f917d7fa700 1 -- 172.21.15.17:0/3155733361 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v72) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9180155b30 con 0x7f91800b1bc0 2022-08-01T19:39:50.253 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:39:50.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.258+0000 7f918528d700 1 -- 172.21.15.17:0/3155733361 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9160053950 msgr2=0x7f9160055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:50.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.258+0000 7f918528d700 1 --2- 172.21.15.17:0/3155733361 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9160053950 0x7f9160055e10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f917001c000 tx=0x7f9170015000).stop 2022-08-01T19:39:50.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.258+0000 7f918528d700 1 -- 172.21.15.17:0/3155733361 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91800b1bc0 msgr2=0x7f91800a1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:50.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.258+0000 7f918528d700 1 --2- 172.21.15.17:0/3155733361 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91800b1bc0 0x7f91800a1d60 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f9180063400 tx=0x7f9174007c50).stop 2022-08-01T19:39:50.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.258+0000 7f918528d700 1 -- 172.21.15.17:0/3155733361 shutdown_connections 2022-08-01T19:39:50.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.258+0000 7f918528d700 1 --2- 172.21.15.17:0/3155733361 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9160053950 0x7f9160055e10 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:50.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.258+0000 7f918528d700 1 --2- 172.21.15.17:0/3155733361 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9180140e40 0x7f91800afc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:50.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.258+0000 7f918528d700 1 --2- 172.21.15.17:0/3155733361 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91800b1bc0 0x7f91800a1d60 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:50.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.258+0000 7f918528d700 1 --2- 172.21.15.17:0/3155733361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9180066780 0x7f91800a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:50.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.258+0000 7f918528d700 1 -- 172.21.15.17:0/3155733361 >> 172.21.15.17:0/3155733361 conn(0x7f9180063870 msgr2=0x7f91801441b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:50.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.258+0000 7f918528d700 1 -- 172.21.15.17:0/3155733361 shutdown_connections 2022-08-01T19:39:50.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.258+0000 7f918528d700 1 -- 172.21.15.17:0/3155733361 wait complete. 2022-08-01T19:39:50.273 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-08-01T19:39:50.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.378+0000 7fc4fca37700 1 Processor -- start 2022-08-01T19:39:50.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.378+0000 7fc4fca37700 1 -- start start 2022-08-01T19:39:50.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.378+0000 7fc4fca37700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fc4f80619b0 0x7fc4f80b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:50.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.378+0000 7fc4fca37700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4f80b5ed0 0x7fc4f80b5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:50.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.378+0000 7fc4fca37700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4f8061d90 con 0x7fc4f80b6ee0 2022-08-01T19:39:50.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.378+0000 7fc4fca37700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4f8143fa0 con 0x7fc4f80619b0 2022-08-01T19:39:50.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.378+0000 7fc4fca37700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f8142f60 con 0x7fc4f80b5ed0 2022-08-01T19:39:50.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.378+0000 7fc4f659c700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc4f80b6ee0 0x7fc4f8143890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60860/0 (socket says 172.21.15.17:60860) 2022-08-01T19:39:50.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.378+0000 7fc4f659c700 1 -- 172.21.15.17:0/1704596933 learned_addr learned my addr 172.21.15.17:0/1704596933 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:50.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.378+0000 7fc4f6d9d700 1 --2- 172.21.15.17:0/1704596933 >> v2:172.21.15.77:3300/0 conn(0x7fc4f80619b0 0x7fc4f80b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:50.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.378+0000 7fc4f759e700 1 --2- 172.21.15.17:0/1704596933 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4f80b5ed0 0x7fc4f80b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:50.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.378+0000 7fc4f5d9b700 1 -- 172.21.15.17:0/1704596933 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3094847836 0 0) 0x7fc4f8061d90 con 0x7fc4f80b6ee0 2022-08-01T19:39:50.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.378+0000 7fc4f5d9b700 1 -- 172.21.15.17:0/1704596933 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4dc003260 con 0x7fc4f80b6ee0 2022-08-01T19:39:50.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.378+0000 7fc4f6d9d700 1 -- 172.21.15.17:0/1704596933 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4f80b5ed0 msgr2=0x7fc4f80b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:50.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.378+0000 7fc4f6d9d700 1 --2- 172.21.15.17:0/1704596933 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4f80b5ed0 0x7fc4f80b5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:50.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.378+0000 7fc4f6d9d700 1 -- 172.21.15.17:0/1704596933 >> v1:172.21.15.17:6789/0 conn(0x7fc4f80b6ee0 legacy=0x7fc4f8143890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:50.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.378+0000 7fc4f5d9b700 1 -- 172.21.15.17:0/1704596933 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1460594750 0 0) 0x7fc4dc003260 con 0x7fc4f80b6ee0 2022-08-01T19:39:50.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.378+0000 7fc4f6d9d700 1 -- 172.21.15.17:0/1704596933 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f80ba5a0 con 0x7fc4f80619b0 2022-08-01T19:39:50.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.378+0000 7fc4f6d9d700 1 --2- 172.21.15.17:0/1704596933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4f80619b0 0x7fc4f80b48a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7fc4ec001820 tx=0x7fc4e8006df0).ready entity=mon.1 client_cookie=bdbfafc8705f3948 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:50.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.378+0000 7fc4f5d9b700 1 -- 172.21.15.17:0/1704596933 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4ec001680 con 0x7fc4f80619b0 2022-08-01T19:39:50.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.378+0000 7fc4f5d9b700 1 -- 172.21.15.17:0/1704596933 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e8000e90 con 0x7fc4f80619b0 2022-08-01T19:39:50.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.378+0000 7fc4f5d9b700 1 -- 172.21.15.17:0/1704596933 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4e8012460 con 0x7fc4f80619b0 2022-08-01T19:39:50.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.378+0000 7fc4fca37700 1 -- 172.21.15.17:0/1704596933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4f80619b0 msgr2=0x7fc4f80b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:50.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.378+0000 7fc4fca37700 1 --2- 172.21.15.17:0/1704596933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4f80619b0 0x7fc4f80b48a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7fc4ec001820 tx=0x7fc4e8006df0).stop 2022-08-01T19:39:50.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.378+0000 7fc4fca37700 1 -- 172.21.15.17:0/1704596933 shutdown_connections 2022-08-01T19:39:50.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.378+0000 7fc4fca37700 1 --2- 172.21.15.17:0/1704596933 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4f80b5ed0 0x7fc4f80b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:50.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.378+0000 7fc4fca37700 1 --2- 172.21.15.17:0/1704596933 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4f80619b0 0x7fc4f80b48a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:50.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.378+0000 7fc4fca37700 1 -- 172.21.15.17:0/1704596933 >> 172.21.15.17:0/1704596933 conn(0x7fc4f806af80 msgr2=0x7fc4f805fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:50.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.382+0000 7fc4fca37700 1 -- 172.21.15.17:0/1704596933 shutdown_connections 2022-08-01T19:39:50.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.382+0000 7fc4fca37700 1 -- 172.21.15.17:0/1704596933 wait complete. 2022-08-01T19:39:50.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.382+0000 7fc4fca37700 1 Processor -- start 2022-08-01T19:39:50.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.382+0000 7fc4fca37700 1 -- start start 2022-08-01T19:39:50.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.382+0000 7fc4fca37700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4f80619b0 0x7fc4f8145990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:50.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.382+0000 7fc4fca37700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4f80b5ed0 0x7fc4f8146120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:50.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.382+0000 7fc4fca37700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4f80b6ee0 0x7fc4f8152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:50.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.382+0000 7fc4fca37700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f8144530 con 0x7fc4f80619b0 2022-08-01T19:39:50.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.382+0000 7fc4fca37700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f8144a90 con 0x7fc4f80b5ed0 2022-08-01T19:39:50.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.382+0000 7fc4fca37700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f8144d30 con 0x7fc4f80b6ee0 2022-08-01T19:39:50.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.382+0000 7fc4f6d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4f80619b0 0x7fc4f8145990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:50.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.382+0000 7fc4f6d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4f80619b0 0x7fc4f8145990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40438/0 (socket says 172.21.15.17:40438) 2022-08-01T19:39:50.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.382+0000 7fc4f6d9d700 1 -- 172.21.15.17:0/2611506009 learned_addr learned my addr 172.21.15.17:0/2611506009 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:50.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.382+0000 7fc4f659c700 1 --2- 172.21.15.17:0/2611506009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4f80b5ed0 0x7fc4f8146120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:50.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.382+0000 7fc4f759e700 1 --2- 172.21.15.17:0/2611506009 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4f80b6ee0 0x7fc4f8152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:50.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.382+0000 7fc4f6d9d700 1 -- 172.21.15.17:0/2611506009 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4f80b6ee0 msgr2=0x7fc4f8152370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:50.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.382+0000 7fc4f6d9d700 1 --2- 172.21.15.17:0/2611506009 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4f80b6ee0 0x7fc4f8152370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:50.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.382+0000 7fc4f6d9d700 1 -- 172.21.15.17:0/2611506009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4f80b5ed0 msgr2=0x7fc4f8146120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:50.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.382+0000 7fc4f6d9d700 1 --2- 172.21.15.17:0/2611506009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4f80b5ed0 0x7fc4f8146120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:50.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.382+0000 7fc4f6d9d700 1 -- 172.21.15.17:0/2611506009 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e800b180 con 0x7fc4f80619b0 2022-08-01T19:39:50.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.382+0000 7fc4f659c700 1 --2- 172.21.15.17:0/2611506009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4f80b5ed0 0x7fc4f8146120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:39:50.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.382+0000 7fc4f6d9d700 1 --2- 172.21.15.17:0/2611506009 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4f80619b0 0x7fc4f8145990 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7fc4ec001820 tx=0x7fc4e8014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:50.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.382+0000 7fc4e7fff700 1 -- 172.21.15.17:0/2611506009 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4e800da20 con 0x7fc4f80619b0 2022-08-01T19:39:50.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.382+0000 7fc4fca37700 1 -- 172.21.15.17:0/2611506009 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4f8144fd0 con 0x7fc4f80619b0 2022-08-01T19:39:50.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.382+0000 7fc4e7fff700 1 -- 172.21.15.17:0/2611506009 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e800db80 con 0x7fc4f80619b0 2022-08-01T19:39:50.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.382+0000 7fc4e7fff700 1 -- 172.21.15.17:0/2611506009 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4e801e3f0 con 0x7fc4f80619b0 2022-08-01T19:39:50.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.382+0000 7fc4fca37700 1 -- 172.21.15.17:0/2611506009 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4f8113ae0 con 0x7fc4f80619b0 2022-08-01T19:39:50.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.386+0000 7fc4e7fff700 1 -- 172.21.15.17:0/2611506009 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc4e801c070 con 0x7fc4f80619b0 2022-08-01T19:39:50.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.386+0000 7fc4fca37700 1 -- 172.21.15.17:0/2611506009 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4f8145730 con 0x7fc4f80619b0 2022-08-01T19:39:50.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.386+0000 7fc4e7fff700 1 --2- 172.21.15.17:0/2611506009 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc4d4053b30 0x7fc4d4055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:50.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.386+0000 7fc4e7fff700 1 -- 172.21.15.17:0/2611506009 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fc4e806c370 con 0x7fc4f80619b0 2022-08-01T19:39:50.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.386+0000 7fc4f659c700 1 --2- 172.21.15.17:0/2611506009 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc4d4053b30 0x7fc4d4055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:50.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.386+0000 7fc4f659c700 1 --2- 172.21.15.17:0/2611506009 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc4d4053b30 0x7fc4d4055ff0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7fc4ec00b000 tx=0x7fc4ec001ce0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:50.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.394+0000 7fc4e7fff700 1 -- 172.21.15.17:0/2611506009 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc4e8071060 con 0x7fc4f80619b0 2022-08-01T19:39:50.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.522+0000 7fc4fca37700 1 -- 172.21.15.17:0/2611506009 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc4f80574e0 con 0x7fc4f80619b0 2022-08-01T19:39:50.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.522+0000 7fc4e7fff700 1 -- 172.21.15.17:0/2611506009 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v72) v1 ==== 84+0+143 (secure 0 0 0) 0x7fc4e8071020 con 0x7fc4f80619b0 2022-08-01T19:39:50.524 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:39:50.525 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2022-08-01T19:39:50.525 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:50.527 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 1, 2022-08-01T19:39:50.527 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": "device_health_metrics" 2022-08-01T19:39:50.527 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:39:50.528 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:39:50.528 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 2, 2022-08-01T19:39:50.528 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": "rbd" 2022-08-01T19:39:50.528 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:39:50.528 INFO:teuthology.orchestra.run.smithi017.stdout:] 2022-08-01T19:39:50.531 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.526+0000 7fc4fca37700 1 -- 172.21.15.17:0/2611506009 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc4d4053b30 msgr2=0x7fc4d4055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:50.532 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.526+0000 7fc4fca37700 1 --2- 172.21.15.17:0/2611506009 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc4d4053b30 0x7fc4d4055ff0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7fc4ec00b000 tx=0x7fc4ec001ce0).stop 2022-08-01T19:39:50.532 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.526+0000 7fc4fca37700 1 -- 172.21.15.17:0/2611506009 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4f80619b0 msgr2=0x7fc4f8145990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:50.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.526+0000 7fc4fca37700 1 --2- 172.21.15.17:0/2611506009 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4f80619b0 0x7fc4f8145990 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7fc4ec001820 tx=0x7fc4e8014000).stop 2022-08-01T19:39:50.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.526+0000 7fc4fca37700 1 -- 172.21.15.17:0/2611506009 shutdown_connections 2022-08-01T19:39:50.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.526+0000 7fc4fca37700 1 --2- 172.21.15.17:0/2611506009 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc4d4053b30 0x7fc4d4055ff0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:50.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.526+0000 7fc4fca37700 1 --2- 172.21.15.17:0/2611506009 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4f80b6ee0 0x7fc4f8152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:50.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.526+0000 7fc4fca37700 1 --2- 172.21.15.17:0/2611506009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4f80b5ed0 0x7fc4f8146120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:50.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.526+0000 7fc4fca37700 1 --2- 172.21.15.17:0/2611506009 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4f80619b0 0x7fc4f8145990 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:50.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.526+0000 7fc4fca37700 1 -- 172.21.15.17:0/2611506009 >> 172.21.15.17:0/2611506009 conn(0x7fc4f806af80 msgr2=0x7fc4f805fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:50.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.526+0000 7fc4fca37700 1 -- 172.21.15.17:0/2611506009 shutdown_connections 2022-08-01T19:39:50.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.526+0000 7fc4fca37700 1 -- 172.21.15.17:0/2611506009 wait complete. 2022-08-01T19:39:50.539 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-08-01T19:39:50.540 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-08-01T19:39:50.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.654+0000 7fddf6a41700 1 Processor -- start 2022-08-01T19:39:50.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.654+0000 7fddf6a41700 1 -- start start 2022-08-01T19:39:50.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.654+0000 7fddf6a41700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fddf00b2730 0x7fddf00666b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:50.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.654+0000 7fddf6a41700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fddf0066ba0 0x7fddf014dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:50.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.654+0000 7fddf6a41700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddf00654c0 con 0x7fddf00b3190 2022-08-01T19:39:50.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.654+0000 7fddf6a41700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddf0062190 con 0x7fddf0066ba0 2022-08-01T19:39:50.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.654+0000 7fddf6a41700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf00b3570 con 0x7fddf00b2730 2022-08-01T19:39:50.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.654+0000 7fddf5a3f700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fddf00b3190 0x7fddf0060ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60872/0 (socket says 172.21.15.17:60872) 2022-08-01T19:39:50.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.654+0000 7fddf6240700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fddf0066ba0 0x7fddf014dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:50.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.654+0000 7fddf5a3f700 1 -- 172.21.15.17:0/3067202522 learned_addr learned my addr 172.21.15.17:0/3067202522 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:50.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.654+0000 7fddf523e700 1 --2- 172.21.15.17:0/3067202522 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fddf00b2730 0x7fddf00666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:50.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.658+0000 7fddf6240700 1 -- 172.21.15.17:0/3067202522 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fddf00b2730 msgr2=0x7fddf00666b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:50.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.658+0000 7fddf6240700 1 --2- 172.21.15.17:0/3067202522 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fddf00b2730 0x7fddf00666b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:50.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.658+0000 7fddf6240700 1 -- 172.21.15.17:0/3067202522 >> v1:172.21.15.17:6789/0 conn(0x7fddf00b3190 legacy=0x7fddf0060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:50.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.658+0000 7fddf6240700 1 -- 172.21.15.17:0/3067202522 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf005fbf0 con 0x7fddf0066ba0 2022-08-01T19:39:50.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.658+0000 7fddf4a3d700 1 -- 172.21.15.17:0/3067202522 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3578223809 0 0) 0x7fddf00654c0 con 0x7fddf00b3190 2022-08-01T19:39:50.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.658+0000 7fddf6240700 1 --2- 172.21.15.17:0/3067202522 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fddf0066ba0 0x7fddf014dab0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7fdde80032f0 tx=0x7fdde80091d0).ready entity=mon.0 client_cookie=cdfebeee675aa03b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:50.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.658+0000 7fddf4a3d700 1 -- 172.21.15.17:0/3067202522 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdde800b6c0 con 0x7fddf0066ba0 2022-08-01T19:39:50.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.658+0000 7fddf4a3d700 1 -- 172.21.15.17:0/3067202522 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde800b820 con 0x7fddf0066ba0 2022-08-01T19:39:50.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.658+0000 7fddf4a3d700 1 -- 172.21.15.17:0/3067202522 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdde800b990 con 0x7fddf0066ba0 2022-08-01T19:39:50.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.658+0000 7fddf6a41700 1 -- 172.21.15.17:0/3067202522 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fddf0066ba0 msgr2=0x7fddf014dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:50.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.658+0000 7fddf6a41700 1 --2- 172.21.15.17:0/3067202522 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fddf0066ba0 0x7fddf014dab0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7fdde80032f0 tx=0x7fdde80091d0).stop 2022-08-01T19:39:50.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.658+0000 7fddf6a41700 1 -- 172.21.15.17:0/3067202522 shutdown_connections 2022-08-01T19:39:50.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.658+0000 7fddf6a41700 1 --2- 172.21.15.17:0/3067202522 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fddf0066ba0 0x7fddf014dab0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:50.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.658+0000 7fddf6a41700 1 --2- 172.21.15.17:0/3067202522 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fddf00b2730 0x7fddf00666b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:50.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.658+0000 7fddf6a41700 1 -- 172.21.15.17:0/3067202522 >> 172.21.15.17:0/3067202522 conn(0x7fddf0063870 msgr2=0x7fddf0063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:50.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.658+0000 7fddf6a41700 1 -- 172.21.15.17:0/3067202522 shutdown_connections 2022-08-01T19:39:50.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.658+0000 7fddf6a41700 1 -- 172.21.15.17:0/3067202522 wait complete. 2022-08-01T19:39:50.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.658+0000 7fddf6a41700 1 Processor -- start 2022-08-01T19:39:50.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.658+0000 7fddf6a41700 1 -- start start 2022-08-01T19:39:50.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.662+0000 7fddf6a41700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddf0066ba0 0x7fddf0148990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:50.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.662+0000 7fddf6a41700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fddf00b2730 0x7fddf00b65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:50.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.662+0000 7fddf6a41700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fddf00b3190 0x7fddf0155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:50.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.662+0000 7fddf6a41700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf0153020 con 0x7fddf00b3190 2022-08-01T19:39:50.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.662+0000 7fddf6a41700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf00abc00 con 0x7fddf0066ba0 2022-08-01T19:39:50.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.662+0000 7fddf6a41700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf0152100 con 0x7fddf00b2730 2022-08-01T19:39:50.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.662+0000 7fddf6240700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fddf00b3190 0x7fddf0155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:50.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.662+0000 7fddf6240700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fddf00b3190 0x7fddf0155690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40456/0 (socket says 172.21.15.17:40456) 2022-08-01T19:39:50.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.662+0000 7fddf6240700 1 -- 172.21.15.17:0/632979354 learned_addr learned my addr 172.21.15.17:0/632979354 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:50.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.662+0000 7fddf523e700 1 --2- 172.21.15.17:0/632979354 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fddf00b2730 0x7fddf00b65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:50.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.662+0000 7fddf5a3f700 1 --2- 172.21.15.17:0/632979354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddf0066ba0 0x7fddf0148990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:50.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.662+0000 7fddf6240700 1 -- 172.21.15.17:0/632979354 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fddf00b2730 msgr2=0x7fddf00b65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:50.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.662+0000 7fddf6240700 1 --2- 172.21.15.17:0/632979354 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fddf00b2730 0x7fddf00b65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:50.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.662+0000 7fddf6240700 1 -- 172.21.15.17:0/632979354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddf0066ba0 msgr2=0x7fddf0148990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:50.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.662+0000 7fddf6240700 1 --2- 172.21.15.17:0/632979354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddf0066ba0 0x7fddf0148990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:50.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.662+0000 7fddf6240700 1 -- 172.21.15.17:0/632979354 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde8009470 con 0x7fddf00b3190 2022-08-01T19:39:50.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.662+0000 7fddf6240700 1 --2- 172.21.15.17:0/632979354 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fddf00b3190 0x7fddf0155690 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7fdde80032f0 tx=0x7fdde8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:50.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.662+0000 7fdddeffd700 1 -- 172.21.15.17:0/632979354 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdde800b6c0 con 0x7fddf00b3190 2022-08-01T19:39:50.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.662+0000 7fdddeffd700 1 -- 172.21.15.17:0/632979354 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde8021630 con 0x7fddf00b3190 2022-08-01T19:39:50.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.662+0000 7fdddeffd700 1 -- 172.21.15.17:0/632979354 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdde8021990 con 0x7fddf00b3190 2022-08-01T19:39:50.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.662+0000 7fddf6a41700 1 -- 172.21.15.17:0/632979354 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddf01523a0 con 0x7fddf00b3190 2022-08-01T19:39:50.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.662+0000 7fddf6a41700 1 -- 172.21.15.17:0/632979354 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddf01528e0 con 0x7fddf00b3190 2022-08-01T19:39:50.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.662+0000 7fddf6a41700 1 -- 172.21.15.17:0/632979354 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddf0152d20 con 0x7fddf00b3190 2022-08-01T19:39:50.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.666+0000 7fdddeffd700 1 -- 172.21.15.17:0/632979354 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdde8027020 con 0x7fddf00b3190 2022-08-01T19:39:50.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.666+0000 7fdddeffd700 1 --2- 172.21.15.17:0/632979354 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fddd0053a70 0x7fddd0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:50.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.666+0000 7fdddeffd700 1 -- 172.21.15.17:0/632979354 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fdde806d0c0 con 0x7fddf00b3190 2022-08-01T19:39:50.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.666+0000 7fddf5a3f700 1 --2- 172.21.15.17:0/632979354 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fddd0053a70 0x7fddd0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:50.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.666+0000 7fddf5a3f700 1 --2- 172.21.15.17:0/632979354 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fddd0053a70 0x7fddd0055f30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7fdde000d000 tx=0x7fdde0007c20).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:50.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.674+0000 7fdddeffd700 1 -- 172.21.15.17:0/632979354 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdde8044200 con 0x7fddf00b3190 2022-08-01T19:39:50.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:50.798+0000 7fddf6a41700 1 -- 172.21.15.17:0/632979354 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fddf00b4550 con 0x7fddf00b3190 2022-08-01T19:39:51.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.573+0000 7fdddeffd700 1 -- 172.21.15.17:0/632979354 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v73) v1 ==== 133+0+0 (secure 0 0 0) 0x7fdde8015020 con 0x7fddf00b3190 2022-08-01T19:39:51.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.577+0000 7fddf6a41700 1 -- 172.21.15.17:0/632979354 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fddd0053a70 msgr2=0x7fddd0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:51.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.577+0000 7fddf6a41700 1 --2- 172.21.15.17:0/632979354 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fddd0053a70 0x7fddd0055f30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7fdde000d000 tx=0x7fdde0007c20).stop 2022-08-01T19:39:51.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.577+0000 7fddf6a41700 1 -- 172.21.15.17:0/632979354 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fddf00b3190 msgr2=0x7fddf0155690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:51.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.577+0000 7fddf6a41700 1 --2- 172.21.15.17:0/632979354 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fddf00b3190 0x7fddf0155690 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7fdde80032f0 tx=0x7fdde8016000).stop 2022-08-01T19:39:51.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.577+0000 7fddf6a41700 1 -- 172.21.15.17:0/632979354 shutdown_connections 2022-08-01T19:39:51.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.577+0000 7fddf6a41700 1 --2- 172.21.15.17:0/632979354 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fddd0053a70 0x7fddd0055f30 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:51.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.577+0000 7fddf6a41700 1 --2- 172.21.15.17:0/632979354 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fddf00b3190 0x7fddf0155690 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:51.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.577+0000 7fddf6a41700 1 --2- 172.21.15.17:0/632979354 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fddf00b2730 0x7fddf00b65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:51.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.577+0000 7fddf6a41700 1 --2- 172.21.15.17:0/632979354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddf0066ba0 0x7fddf0148990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:51.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.577+0000 7fddf6a41700 1 -- 172.21.15.17:0/632979354 >> 172.21.15.17:0/632979354 conn(0x7fddf0063870 msgr2=0x7fddf0146460 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:51.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.577+0000 7fddf6a41700 1 -- 172.21.15.17:0/632979354 shutdown_connections 2022-08-01T19:39:51.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.577+0000 7fddf6a41700 1 -- 172.21.15.17:0/632979354 wait complete. 2022-08-01T19:39:51.587 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' created 2022-08-01T19:39:51.592 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-08-01T19:39:51.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.705+0000 7f6b53571700 1 Processor -- start 2022-08-01T19:39:51.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.705+0000 7f6b53571700 1 -- start start 2022-08-01T19:39:51.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.705+0000 7f6b53571700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6b4c146ae0 0x7f6b4c1403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:51.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.705+0000 7f6b53571700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f6b4c0b4820 0x7f6b4c0b5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:51.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.705+0000 7f6b53571700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b4c143cf0 con 0x7f6b4c0b4310 2022-08-01T19:39:51.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.705+0000 7f6b53571700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b4c1476a0 con 0x7f6b4c0b4820 2022-08-01T19:39:51.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.705+0000 7f6b53571700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c143f90 con 0x7f6b4c146ae0 2022-08-01T19:39:51.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.705+0000 7f6b5130d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6b4c146ae0 0x7f6b4c1403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:51.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.705+0000 7f6b51b0e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f6b4c0b4820 0x7f6b4c0b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:51.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.705+0000 7f6b5130d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6b4c146ae0 0x7f6b4c1403c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:36560/0 (socket says 172.21.15.17:36560) 2022-08-01T19:39:51.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.705+0000 7f6b5130d700 1 -- 172.21.15.17:0/555221284 learned_addr learned my addr 172.21.15.17:0/555221284 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:51.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.705+0000 7f6b50b0c700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7f6b4c0b4310 0x7f6b4c1408b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:49454/0 (socket says 172.21.15.17:49454) 2022-08-01T19:39:51.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.705+0000 7f6b51b0e700 1 -- 172.21.15.17:0/555221284 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6b4c146ae0 msgr2=0x7f6b4c1403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:51.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.705+0000 7f6b51b0e700 1 --2- 172.21.15.17:0/555221284 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6b4c146ae0 0x7f6b4c1403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:51.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.705+0000 7f6b51b0e700 1 -- 172.21.15.17:0/555221284 >> v1:172.21.15.77:6789/0 conn(0x7f6b4c0b4310 legacy=0x7f6b4c1408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:51.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.705+0000 7f6b51b0e700 1 -- 172.21.15.17:0/555221284 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b4c0b0e20 con 0x7f6b4c0b4820 2022-08-01T19:39:51.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.705+0000 7f6b5130d700 1 --2- 172.21.15.17:0/555221284 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6b4c146ae0 0x7f6b4c1403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:39:51.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.705+0000 7f6b51b0e700 1 --2- 172.21.15.17:0/555221284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b4c0b4820 0x7f6b4c0b5060 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f6b440032f0 tx=0x7f6b440091d0).ready entity=mon.1 client_cookie=7aac7fbf3b3db1ef server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:51.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.705+0000 7f6b43fff700 1 -- 172.21.15.17:0/555221284 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b4400b6c0 con 0x7f6b4c0b4820 2022-08-01T19:39:51.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.705+0000 7f6b43fff700 1 -- 172.21.15.17:0/555221284 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b4400b820 con 0x7f6b4c0b4820 2022-08-01T19:39:51.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.705+0000 7f6b43fff700 1 -- 172.21.15.17:0/555221284 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b4400b990 con 0x7f6b4c0b4820 2022-08-01T19:39:51.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.709+0000 7f6b53571700 1 -- 172.21.15.17:0/555221284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b4c0b4820 msgr2=0x7f6b4c0b5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:51.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.709+0000 7f6b53571700 1 --2- 172.21.15.17:0/555221284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b4c0b4820 0x7f6b4c0b5060 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f6b440032f0 tx=0x7f6b440091d0).stop 2022-08-01T19:39:51.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.709+0000 7f6b53571700 1 -- 172.21.15.17:0/555221284 shutdown_connections 2022-08-01T19:39:51.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.709+0000 7f6b53571700 1 --2- 172.21.15.17:0/555221284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b4c0b4820 0x7f6b4c0b5060 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:51.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.709+0000 7f6b53571700 1 --2- 172.21.15.17:0/555221284 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6b4c146ae0 0x7f6b4c1403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:51.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.709+0000 7f6b53571700 1 -- 172.21.15.17:0/555221284 >> 172.21.15.17:0/555221284 conn(0x7f6b4c05c620 msgr2=0x7f6b4c05ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:51.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.709+0000 7f6b53571700 1 -- 172.21.15.17:0/555221284 shutdown_connections 2022-08-01T19:39:51.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.709+0000 7f6b53571700 1 -- 172.21.15.17:0/555221284 wait complete. 2022-08-01T19:39:51.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.709+0000 7f6b53571700 1 Processor -- start 2022-08-01T19:39:51.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.709+0000 7f6b53571700 1 -- start start 2022-08-01T19:39:51.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.709+0000 7f6b53571700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b4c0b4310 0x7f6b4c145de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:51.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.709+0000 7f6b53571700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b4c0b4820 0x7f6b4c0adca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:51.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.709+0000 7f6b53571700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6b4c146ae0 0x7f6b4c0afc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:51.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.709+0000 7f6b53571700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c155b50 con 0x7f6b4c0b4310 2022-08-01T19:39:51.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.709+0000 7f6b53571700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c155df0 con 0x7f6b4c0b4820 2022-08-01T19:39:51.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.709+0000 7f6b53571700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c156090 con 0x7f6b4c146ae0 2022-08-01T19:39:51.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.709+0000 7f6b5130d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b4c0b4310 0x7f6b4c145de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:51.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.709+0000 7f6b5130d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b4c0b4310 0x7f6b4c145de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40466/0 (socket says 172.21.15.17:40466) 2022-08-01T19:39:51.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.713+0000 7f6b5130d700 1 -- 172.21.15.17:0/1449872304 learned_addr learned my addr 172.21.15.17:0/1449872304 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:51.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.713+0000 7f6b51b0e700 1 --2- 172.21.15.17:0/1449872304 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6b4c146ae0 0x7f6b4c0afc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:51.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.713+0000 7f6b50b0c700 1 --2- 172.21.15.17:0/1449872304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b4c0b4820 0x7f6b4c0adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:51.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.713+0000 7f6b5130d700 1 -- 172.21.15.17:0/1449872304 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6b4c146ae0 msgr2=0x7f6b4c0afc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:51.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.713+0000 7f6b5130d700 1 --2- 172.21.15.17:0/1449872304 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6b4c146ae0 0x7f6b4c0afc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:51.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.713+0000 7f6b5130d700 1 -- 172.21.15.17:0/1449872304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b4c0b4820 msgr2=0x7f6b4c0adca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:51.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.713+0000 7f6b5130d700 1 --2- 172.21.15.17:0/1449872304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b4c0b4820 0x7f6b4c0adca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:51.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.713+0000 7f6b5130d700 1 -- 172.21.15.17:0/1449872304 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b4800b1e0 con 0x7f6b4c0b4310 2022-08-01T19:39:51.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.713+0000 7f6b51b0e700 1 --2- 172.21.15.17:0/1449872304 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6b4c146ae0 0x7f6b4c0afc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:39:51.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.713+0000 7f6b5130d700 1 --2- 172.21.15.17:0/1449872304 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b4c0b4310 0x7f6b4c145de0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f6b4800c000 tx=0x7f6b48006eb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:51.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.713+0000 7f6b427fc700 1 -- 172.21.15.17:0/1449872304 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b48012600 con 0x7f6b4c0b4310 2022-08-01T19:39:51.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.713+0000 7f6b427fc700 1 -- 172.21.15.17:0/1449872304 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b48012760 con 0x7f6b4c0b4310 2022-08-01T19:39:51.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.713+0000 7f6b53571700 1 -- 172.21.15.17:0/1449872304 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b44009470 con 0x7f6b4c0b4310 2022-08-01T19:39:51.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.713+0000 7f6b427fc700 1 -- 172.21.15.17:0/1449872304 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b48012ac0 con 0x7f6b4c0b4310 2022-08-01T19:39:51.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.713+0000 7f6b53571700 1 -- 172.21.15.17:0/1449872304 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b4c1462d0 con 0x7f6b4c0b4310 2022-08-01T19:39:51.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.713+0000 7f6b427fc700 1 -- 172.21.15.17:0/1449872304 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6b48012c20 con 0x7f6b4c0b4310 2022-08-01T19:39:51.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.713+0000 7f6b53571700 1 -- 172.21.15.17:0/1449872304 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b4c156530 con 0x7f6b4c0b4310 2022-08-01T19:39:51.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.713+0000 7f6b427fc700 1 --2- 172.21.15.17:0/1449872304 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6b34053a70 0x7f6b34055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:51.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.713+0000 7f6b427fc700 1 -- 172.21.15.17:0/1449872304 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f6b4806bfd0 con 0x7f6b4c0b4310 2022-08-01T19:39:51.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.717+0000 7f6b50b0c700 1 --2- 172.21.15.17:0/1449872304 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6b34053a70 0x7f6b34055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:51.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.717+0000 7f6b50b0c700 1 --2- 172.21.15.17:0/1449872304 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6b34053a70 0x7f6b34055f30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f6b3c00b000 tx=0x7f6b3c007160).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:51.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.721+0000 7f6b427fc700 1 -- 172.21.15.17:0/1449872304 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6b4c156530 con 0x7f6b4c0b4310 2022-08-01T19:39:51.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:51.845+0000 7f6b53571700 1 -- 172.21.15.17:0/1449872304 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f6b4c156530 con 0x7f6b4c0b4310 2022-08-01T19:39:52.008 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.009+0000 7f6b427fc700 1 -- 172.21.15.17:0/1449872304 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v74) v1 ==== 125+0+0 (secure 0 0 0) 0x7f6b4c156530 con 0x7f6b4c0b4310 2022-08-01T19:39:52.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.017+0000 7f6b53571700 1 -- 172.21.15.17:0/1449872304 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6b34053a70 msgr2=0x7f6b34055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:52.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.017+0000 7f6b53571700 1 --2- 172.21.15.17:0/1449872304 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6b34053a70 0x7f6b34055f30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f6b3c00b000 tx=0x7f6b3c007160).stop 2022-08-01T19:39:52.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.017+0000 7f6b53571700 1 -- 172.21.15.17:0/1449872304 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b4c0b4310 msgr2=0x7f6b4c145de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:52.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.017+0000 7f6b53571700 1 --2- 172.21.15.17:0/1449872304 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b4c0b4310 0x7f6b4c145de0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f6b4800c000 tx=0x7f6b48006eb0).stop 2022-08-01T19:39:52.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.021+0000 7f6b53571700 1 -- 172.21.15.17:0/1449872304 shutdown_connections 2022-08-01T19:39:52.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.021+0000 7f6b53571700 1 --2- 172.21.15.17:0/1449872304 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6b34053a70 0x7f6b34055f30 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:52.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.021+0000 7f6b53571700 1 --2- 172.21.15.17:0/1449872304 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6b4c146ae0 0x7f6b4c0afc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:52.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.021+0000 7f6b53571700 1 --2- 172.21.15.17:0/1449872304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b4c0b4820 0x7f6b4c0adca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:52.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.021+0000 7f6b53571700 1 --2- 172.21.15.17:0/1449872304 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b4c0b4310 0x7f6b4c145de0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:52.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.021+0000 7f6b53571700 1 -- 172.21.15.17:0/1449872304 >> 172.21.15.17:0/1449872304 conn(0x7f6b4c05c620 msgr2=0x7f6b4c140190 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:52.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.021+0000 7f6b53571700 1 -- 172.21.15.17:0/1449872304 shutdown_connections 2022-08-01T19:39:52.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.021+0000 7f6b53571700 1 -- 172.21.15.17:0/1449872304 wait complete. 2022-08-01T19:39:52.025 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' created 2022-08-01T19:39:52.032 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-08-01T19:39:52.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.141+0000 7feb9965b700 1 Processor -- start 2022-08-01T19:39:52.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.141+0000 7feb9965b700 1 -- start start 2022-08-01T19:39:52.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.141+0000 7feb9965b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7feb940ad750 0x7feb94061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:52.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.141+0000 7feb9965b700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb940aee80 con 0x7feb941405d0 2022-08-01T19:39:52.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.141+0000 7feb9965b700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb940b70b0 con 0x7feb940614b0 2022-08-01T19:39:52.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.145+0000 7feb9965b700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb940af0b0 con 0x7feb940ad750 2022-08-01T19:39:52.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.145+0000 7feb937fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7feb940ad750 0x7feb94061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:52.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.145+0000 7feb937fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7feb940ad750 0x7feb94061890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40476/0 (socket says 172.21.15.17:40476) 2022-08-01T19:39:52.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.145+0000 7feb937fe700 1 -- 172.21.15.17:0/433980225 learned_addr learned my addr 172.21.15.17:0/433980225 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:52.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.145+0000 7feb98e5a700 1 --1- 172.21.15.17:0/433980225 >> v1:172.21.15.17:6789/0 conn(0x7feb941405d0 0x7feb941466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60904/0 (socket says 172.21.15.17:60904) 2022-08-01T19:39:52.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.145+0000 7feb937fe700 1 -- 172.21.15.17:0/433980225 >> v1:172.21.15.77:6789/0 conn(0x7feb940614b0 legacy=0x7feb940b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:52.162 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.145+0000 7feb937fe700 1 -- 172.21.15.17:0/433980225 >> v1:172.21.15.17:6789/0 conn(0x7feb941405d0 legacy=0x7feb941466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:52.162 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.145+0000 7feb937fe700 1 -- 172.21.15.17:0/433980225 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb940af350 con 0x7feb940ad750 2022-08-01T19:39:52.163 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.145+0000 7feb937fe700 1 --2- 172.21.15.17:0/433980225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb940ad750 0x7feb94061890 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7feb84002f70 tx=0x7feb84006eb0).ready entity=mon.0 client_cookie=7e02025c73cbcdbc server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:52.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.145+0000 7feb92ffd700 1 -- 172.21.15.17:0/433980225 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb8400d660 con 0x7feb940ad750 2022-08-01T19:39:52.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.145+0000 7feb92ffd700 1 -- 172.21.15.17:0/433980225 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb84000e90 con 0x7feb940ad750 2022-08-01T19:39:52.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.145+0000 7feb92ffd700 1 -- 172.21.15.17:0/433980225 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb8400d9f0 con 0x7feb940ad750 2022-08-01T19:39:52.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.145+0000 7feb9965b700 1 -- 172.21.15.17:0/433980225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb940ad750 msgr2=0x7feb94061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:52.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.145+0000 7feb9965b700 1 --2- 172.21.15.17:0/433980225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb940ad750 0x7feb94061890 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7feb84002f70 tx=0x7feb84006eb0).stop 2022-08-01T19:39:52.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.145+0000 7feb9965b700 1 -- 172.21.15.17:0/433980225 shutdown_connections 2022-08-01T19:39:52.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.145+0000 7feb9965b700 1 --2- 172.21.15.17:0/433980225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb940ad750 0x7feb94061890 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:52.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.145+0000 7feb9965b700 1 -- 172.21.15.17:0/433980225 >> 172.21.15.17:0/433980225 conn(0x7feb94066f70 msgr2=0x7feb94067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:52.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.145+0000 7feb9965b700 1 -- 172.21.15.17:0/433980225 shutdown_connections 2022-08-01T19:39:52.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.145+0000 7feb9965b700 1 -- 172.21.15.17:0/433980225 wait complete. 2022-08-01T19:39:52.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.149+0000 7feb9965b700 1 Processor -- start 2022-08-01T19:39:52.169 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.149+0000 7feb9965b700 1 -- start start 2022-08-01T19:39:52.169 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.149+0000 7feb9965b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7feb940614b0 0x7feb940aa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:52.170 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.149+0000 7feb9965b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb940ad750 0x7feb94152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:52.170 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.149+0000 7feb9965b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb941405d0 0x7feb94153010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:52.171 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.149+0000 7feb9965b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb940b3a90 con 0x7feb940ad750 2022-08-01T19:39:52.171 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.149+0000 7feb9965b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb94156580 con 0x7feb941405d0 2022-08-01T19:39:52.171 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.149+0000 7feb9965b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb940ad0d0 con 0x7feb940614b0 2022-08-01T19:39:52.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.149+0000 7feb937fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb940ad750 0x7feb94152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:52.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.149+0000 7feb937fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb940ad750 0x7feb94152b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40482/0 (socket says 172.21.15.17:40482) 2022-08-01T19:39:52.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.149+0000 7feb937fe700 1 -- 172.21.15.17:0/389434977 learned_addr learned my addr 172.21.15.17:0/389434977 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:52.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.149+0000 7feb98e5a700 1 --2- 172.21.15.17:0/389434977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb941405d0 0x7feb94153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:52.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.149+0000 7feb93fff700 1 --2- 172.21.15.17:0/389434977 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7feb940614b0 0x7feb940aa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:52.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.149+0000 7feb937fe700 1 -- 172.21.15.17:0/389434977 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7feb940614b0 msgr2=0x7feb940aa4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:52.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.149+0000 7feb937fe700 1 --2- 172.21.15.17:0/389434977 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7feb940614b0 0x7feb940aa4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:52.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.149+0000 7feb937fe700 1 -- 172.21.15.17:0/389434977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb941405d0 msgr2=0x7feb94153010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:52.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.149+0000 7feb937fe700 1 --2- 172.21.15.17:0/389434977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb941405d0 0x7feb94153010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:52.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.149+0000 7feb937fe700 1 -- 172.21.15.17:0/389434977 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb8400b1e0 con 0x7feb940ad750 2022-08-01T19:39:52.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.149+0000 7feb93fff700 1 --2- 172.21.15.17:0/389434977 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7feb940614b0 0x7feb940aa4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:39:52.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.149+0000 7feb937fe700 1 --2- 172.21.15.17:0/389434977 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb940ad750 0x7feb94152b20 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7feb940ad3a0 tx=0x7feb84015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:52.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.149+0000 7feb917fa700 1 -- 172.21.15.17:0/389434977 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb8400d9e0 con 0x7feb940ad750 2022-08-01T19:39:52.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.149+0000 7feb917fa700 1 -- 172.21.15.17:0/389434977 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb84011bb0 con 0x7feb940ad750 2022-08-01T19:39:52.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.149+0000 7feb9965b700 1 -- 172.21.15.17:0/389434977 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb941567f0 con 0x7feb940ad750 2022-08-01T19:39:52.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.149+0000 7feb917fa700 1 -- 172.21.15.17:0/389434977 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb8401e460 con 0x7feb940ad750 2022-08-01T19:39:52.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.149+0000 7feb9965b700 1 -- 172.21.15.17:0/389434977 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb941525e0 con 0x7feb940ad750 2022-08-01T19:39:52.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.153+0000 7feb9965b700 1 -- 172.21.15.17:0/389434977 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb940af5f0 con 0x7feb940ad750 2022-08-01T19:39:52.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.153+0000 7feb917fa700 1 -- 172.21.15.17:0/389434977 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7feb84025030 con 0x7feb940ad750 2022-08-01T19:39:52.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.153+0000 7feb917fa700 1 --2- 172.21.15.17:0/389434977 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7feb74053980 0x7feb74055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:52.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.153+0000 7feb917fa700 1 -- 172.21.15.17:0/389434977 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6304+0+0 (secure 0 0 0) 0x7feb8406d440 con 0x7feb940ad750 2022-08-01T19:39:52.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.153+0000 7feb93fff700 1 --2- 172.21.15.17:0/389434977 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7feb74053980 0x7feb74055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:52.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.157+0000 7feb93fff700 1 --2- 172.21.15.17:0/389434977 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7feb74053980 0x7feb74055e40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7feb8800b000 tx=0x7feb88009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:52.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.157+0000 7feb917fa700 1 -- 172.21.15.17:0/389434977 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7feb84044170 con 0x7feb940ad750 2022-08-01T19:39:52.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.321+0000 7feb9965b700 1 -- 172.21.15.17:0/389434977 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7feb940574e0 con 0x7feb940ad750 2022-08-01T19:39:52.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.333+0000 7feb917fa700 1 -- 172.21.15.17:0/389434977 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 13 and data pool 14 v34) v1 ==== 189+0+0 (secure 0 0 0) 0x7feb84023020 con 0x7feb940ad750 2022-08-01T19:39:52.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.337+0000 7feb9965b700 1 -- 172.21.15.17:0/389434977 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7feb74053980 msgr2=0x7feb74055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:52.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.337+0000 7feb9965b700 1 --2- 172.21.15.17:0/389434977 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7feb74053980 0x7feb74055e40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7feb8800b000 tx=0x7feb88009040).stop 2022-08-01T19:39:52.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.337+0000 7feb9965b700 1 -- 172.21.15.17:0/389434977 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb940ad750 msgr2=0x7feb94152b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:52.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.337+0000 7feb9965b700 1 --2- 172.21.15.17:0/389434977 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb940ad750 0x7feb94152b20 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7feb940ad3a0 tx=0x7feb84015000).stop 2022-08-01T19:39:52.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.337+0000 7feb9965b700 1 -- 172.21.15.17:0/389434977 shutdown_connections 2022-08-01T19:39:52.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.337+0000 7feb9965b700 1 --2- 172.21.15.17:0/389434977 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7feb74053980 0x7feb74055e40 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:52.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.337+0000 7feb9965b700 1 --2- 172.21.15.17:0/389434977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb941405d0 0x7feb94153010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:52.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.337+0000 7feb9965b700 1 --2- 172.21.15.17:0/389434977 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb940ad750 0x7feb94152b20 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:52.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.337+0000 7feb9965b700 1 --2- 172.21.15.17:0/389434977 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7feb940614b0 0x7feb940aa4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:52.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.337+0000 7feb9965b700 1 -- 172.21.15.17:0/389434977 >> 172.21.15.17:0/389434977 conn(0x7feb94066f70 msgr2=0x7feb9413fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:52.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.337+0000 7feb9965b700 1 -- 172.21.15.17:0/389434977 shutdown_connections 2022-08-01T19:39:52.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.337+0000 7feb9965b700 1 -- 172.21.15.17:0/389434977 wait complete. 2022-08-01T19:39:52.347 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 13 and data pool 14 2022-08-01T19:39:52.352 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:39:52.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.445+0000 7fb5b3fff700 1 Processor -- start 2022-08-01T19:39:52.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.445+0000 7fb5b3fff700 1 -- start start 2022-08-01T19:39:52.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.445+0000 7fb5b3fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb5b40b2710 0x7fb5b414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:52.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.445+0000 7fb5b3fff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fb5b40a8c50 0x7fb5b414b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:52.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.445+0000 7fb5b3fff700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5b40a7250 con 0x7fb5b40b01a0 2022-08-01T19:39:52.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.445+0000 7fb5b3fff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5b40a7480 con 0x7fb5b40a8c50 2022-08-01T19:39:52.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.445+0000 7fb5b3fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b40a7720 con 0x7fb5b40b2710 2022-08-01T19:39:52.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.445+0000 7fb5b27fc700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb5b40b01a0 0x7fb5b40a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60928/0 (socket says 172.21.15.17:60928) 2022-08-01T19:39:52.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.445+0000 7fb5b27fc700 1 -- 172.21.15.17:0/2482910808 learned_addr learned my addr 172.21.15.17:0/2482910808 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:52.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.445+0000 7fb5b37fe700 1 --2- 172.21.15.17:0/2482910808 >> v2:172.21.15.77:3300/0 conn(0x7fb5b40a8c50 0x7fb5b414b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:52.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.445+0000 7fb5b2ffd700 1 --2- 172.21.15.17:0/2482910808 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb5b40b2710 0x7fb5b414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:52.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.445+0000 7fb5b1ffb700 1 -- 172.21.15.17:0/2482910808 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3036979255 0 0) 0x7fb5b40a7250 con 0x7fb5b40b01a0 2022-08-01T19:39:52.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.445+0000 7fb5b1ffb700 1 -- 172.21.15.17:0/2482910808 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb598003260 con 0x7fb5b40b01a0 2022-08-01T19:39:52.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.445+0000 7fb5b1ffb700 1 -- 172.21.15.17:0/2482910808 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1929411290 0 0) 0x7fb598003260 con 0x7fb5b40b01a0 2022-08-01T19:39:52.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.445+0000 7fb5b1ffb700 1 -- 172.21.15.17:0/2482910808 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb5b40b2710 msgr2=0x7fb5b414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:52.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.445+0000 7fb5b1ffb700 1 --2- 172.21.15.17:0/2482910808 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb5b40b2710 0x7fb5b414afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:52.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.445+0000 7fb5b1ffb700 1 -- 172.21.15.17:0/2482910808 >> v2:172.21.15.77:3300/0 conn(0x7fb5b40a8c50 msgr2=0x7fb5b414b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:52.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.445+0000 7fb5b1ffb700 1 --2- 172.21.15.17:0/2482910808 >> v2:172.21.15.77:3300/0 conn(0x7fb5b40a8c50 0x7fb5b414b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:52.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.445+0000 7fb5b1ffb700 1 -- 172.21.15.17:0/2482910808 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b40a79c0 con 0x7fb5b40b01a0 2022-08-01T19:39:52.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.445+0000 7fb5b1ffb700 1 -- 172.21.15.17:0/2482910808 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7fb5a8001660 con 0x7fb5b40b01a0 2022-08-01T19:39:52.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.445+0000 7fb5b1ffb700 1 -- 172.21.15.17:0/2482910808 >> v1:172.21.15.17:6789/0 conn(0x7fb5b40b01a0 legacy=0x7fb5b40a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:52.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.445+0000 7fb5b1ffb700 1 --2- 172.21.15.17:0/2482910808 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb598003750 0x7fb598003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:52.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.445+0000 7fb5b1ffb700 1 --2- 172.21.15.17:0/2482910808 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb5b40b2710 0x7fb598004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:52.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.445+0000 7fb5b1ffb700 1 --2- 172.21.15.17:0/2482910808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5980049f0 0x7fb598004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:52.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.445+0000 7fb5b1ffb700 1 -- 172.21.15.17:0/2482910808 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b40a7480 con 0x7fb598003750 2022-08-01T19:39:52.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.445+0000 7fb5b1ffb700 1 -- 172.21.15.17:0/2482910808 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b40a7720 con 0x7fb5980049f0 2022-08-01T19:39:52.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.445+0000 7fb5b1ffb700 1 -- 172.21.15.17:0/2482910808 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb59800b750 con 0x7fb5b40b2710 2022-08-01T19:39:52.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.445+0000 7fb5b2ffd700 1 --2- 172.21.15.17:0/2482910808 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb598003750 0x7fb598003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:52.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.445+0000 7fb5b27fc700 1 --2- 172.21.15.17:0/2482910808 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb5b40b2710 0x7fb598004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:52.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.445+0000 7fb5b37fe700 1 --2- 172.21.15.17:0/2482910808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5980049f0 0x7fb598004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:52.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.449+0000 7fb5b27fc700 1 -- 172.21.15.17:0/2482910808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5980049f0 msgr2=0x7fb598004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:52.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.449+0000 7fb5b27fc700 1 --2- 172.21.15.17:0/2482910808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5980049f0 0x7fb598004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:52.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.449+0000 7fb5b27fc700 1 -- 172.21.15.17:0/2482910808 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb598003750 msgr2=0x7fb598003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:52.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.449+0000 7fb5b27fc700 1 --2- 172.21.15.17:0/2482910808 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb598003750 0x7fb598003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:52.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.449+0000 7fb5b27fc700 1 -- 172.21.15.17:0/2482910808 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb598003260 con 0x7fb5b40b2710 2022-08-01T19:39:52.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.449+0000 7fb5b27fc700 1 --2- 172.21.15.17:0/2482910808 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb5b40b2710 0x7fb598004500 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fb5a8001370 tx=0x7fb5b406a5f0).ready entity=mon.2 client_cookie=64b7c944a91feadb server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:52.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.449+0000 7fb5b1ffb700 1 -- 172.21.15.17:0/2482910808 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5a80071a0 con 0x7fb5b40b2710 2022-08-01T19:39:52.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.449+0000 7fb5b1ffb700 1 -- 172.21.15.17:0/2482910808 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5a8007300 con 0x7fb5b40b2710 2022-08-01T19:39:52.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.449+0000 7fb5b1ffb700 1 -- 172.21.15.17:0/2482910808 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5a800cbb0 con 0x7fb5b40b2710 2022-08-01T19:39:52.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.449+0000 7fb5b3fff700 1 -- 172.21.15.17:0/2482910808 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb5b40b2710 msgr2=0x7fb598004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:52.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.449+0000 7fb5b3fff700 1 --2- 172.21.15.17:0/2482910808 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb5b40b2710 0x7fb598004500 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fb5a8001370 tx=0x7fb5b406a5f0).stop 2022-08-01T19:39:52.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.449+0000 7fb5b37fe700 1 -- 172.21.15.17:0/2482910808 reap_dead start 2022-08-01T19:39:52.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.449+0000 7fb5b3fff700 1 -- 172.21.15.17:0/2482910808 shutdown_connections 2022-08-01T19:39:52.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.449+0000 7fb5b3fff700 1 -- 172.21.15.17:0/2482910808 >> 172.21.15.17:0/2482910808 conn(0x7fb5b4060310 msgr2=0x7fb5b40b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:52.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.449+0000 7fb5b3fff700 1 -- 172.21.15.17:0/2482910808 shutdown_connections 2022-08-01T19:39:52.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.449+0000 7fb5b3fff700 1 -- 172.21.15.17:0/2482910808 wait complete. 2022-08-01T19:39:52.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.453+0000 7fb5b3fff700 1 Processor -- start 2022-08-01T19:39:52.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.453+0000 7fb5b3fff700 1 -- start start 2022-08-01T19:39:52.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.453+0000 7fb5b3fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5b40b2710 0x7fb5b40ab720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:52.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.453+0000 7fb5b3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5b41465a0 0x7fb5b4146a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:52.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.453+0000 7fb5b3fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb5b4146ef0 0x7fb5b4063190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:52.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.453+0000 7fb5b3fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b40b4c50 con 0x7fb5b41465a0 2022-08-01T19:39:52.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.453+0000 7fb5b3fff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b40b5c10 con 0x7fb5b40b2710 2022-08-01T19:39:52.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.453+0000 7fb5b3fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b40b5eb0 con 0x7fb5b4146ef0 2022-08-01T19:39:52.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.453+0000 7fb5b27fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5b41465a0 0x7fb5b4146a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:52.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.453+0000 7fb5b27fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5b41465a0 0x7fb5b4146a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40512/0 (socket says 172.21.15.17:40512) 2022-08-01T19:39:52.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.453+0000 7fb5b27fc700 1 -- 172.21.15.17:0/688644749 learned_addr learned my addr 172.21.15.17:0/688644749 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:52.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.453+0000 7fb5b2ffd700 1 --2- 172.21.15.17:0/688644749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5b40b2710 0x7fb5b40ab720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:52.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.453+0000 7fb5b37fe700 1 --2- 172.21.15.17:0/688644749 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb5b4146ef0 0x7fb5b4063190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:52.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.453+0000 7fb5b27fc700 1 -- 172.21.15.17:0/688644749 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb5b4146ef0 msgr2=0x7fb5b4063190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:52.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.453+0000 7fb5b27fc700 1 --2- 172.21.15.17:0/688644749 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb5b4146ef0 0x7fb5b4063190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:52.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.453+0000 7fb5b27fc700 1 -- 172.21.15.17:0/688644749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5b40b2710 msgr2=0x7fb5b40ab720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:52.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.453+0000 7fb5b27fc700 1 --2- 172.21.15.17:0/688644749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5b40b2710 0x7fb5b40ab720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:52.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.453+0000 7fb5b27fc700 1 -- 172.21.15.17:0/688644749 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b40b5050 con 0x7fb5b41465a0 2022-08-01T19:39:52.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.453+0000 7fb5b27fc700 1 --2- 172.21.15.17:0/688644749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5b41465a0 0x7fb5b4146a00 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7fb5a8001370 tx=0x7fb5a8013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:52.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.453+0000 7fb5a3fff700 1 -- 172.21.15.17:0/688644749 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5a8001c70 con 0x7fb5b41465a0 2022-08-01T19:39:52.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.453+0000 7fb5a3fff700 1 -- 172.21.15.17:0/688644749 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5a8004920 con 0x7fb5b41465a0 2022-08-01T19:39:52.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.453+0000 7fb5b3fff700 1 -- 172.21.15.17:0/688644749 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5b4065ce0 con 0x7fb5b41465a0 2022-08-01T19:39:52.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.453+0000 7fb5a3fff700 1 -- 172.21.15.17:0/688644749 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5a8004c80 con 0x7fb5b41465a0 2022-08-01T19:39:52.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.453+0000 7fb5b3fff700 1 -- 172.21.15.17:0/688644749 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5b40b4770 con 0x7fb5b41465a0 2022-08-01T19:39:52.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.457+0000 7fb5b3fff700 1 -- 172.21.15.17:0/688644749 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5b40b4dc0 con 0x7fb5b41465a0 2022-08-01T19:39:52.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.457+0000 7fb5a3fff700 1 -- 172.21.15.17:0/688644749 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb5a8025070 con 0x7fb5b41465a0 2022-08-01T19:39:52.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.457+0000 7fb5a3fff700 1 --2- 172.21.15.17:0/688644749 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb590053b00 0x7fb590055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:52.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.457+0000 7fb5a3fff700 1 -- 172.21.15.17:0/688644749 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb5a801a040 con 0x7fb5b41465a0 2022-08-01T19:39:52.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.457+0000 7fb5b2ffd700 1 --2- 172.21.15.17:0/688644749 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb590053b00 0x7fb590055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:52.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.457+0000 7fb5b2ffd700 1 --2- 172.21.15.17:0/688644749 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb590053b00 0x7fb590055fc0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7fb5ac00b000 tx=0x7fb5ac0091d0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:52.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.461+0000 7fb5a3fff700 1 -- 172.21.15.17:0/688644749 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb5b40b4dc0 con 0x7fb5b41465a0 2022-08-01T19:39:52.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.589+0000 7fb5b3fff700 1 -- 172.21.15.17:0/688644749 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5b40b4d90 con 0x7fb5b41465a0 2022-08-01T19:39:52.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.589+0000 7fb5a3fff700 1 -- 172.21.15.17:0/688644749 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16287 (secure 0 0 0) 0x7fb5a8043520 con 0x7fb5b41465a0 2022-08-01T19:39:52.593 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:39:52.593 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":75,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:39:52.325144+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:39:50.853967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2022-08-01T19:39:51.901692+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:39:52.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.593+0000 7fb5b3fff700 1 -- 172.21.15.17:0/688644749 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb590053b00 msgr2=0x7fb590055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:52.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.593+0000 7fb5b3fff700 1 --2- 172.21.15.17:0/688644749 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb590053b00 0x7fb590055fc0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7fb5ac00b000 tx=0x7fb5ac0091d0).stop 2022-08-01T19:39:52.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.593+0000 7fb5b3fff700 1 -- 172.21.15.17:0/688644749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5b41465a0 msgr2=0x7fb5b4146a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:52.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.593+0000 7fb5b3fff700 1 --2- 172.21.15.17:0/688644749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5b41465a0 0x7fb5b4146a00 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7fb5a8001370 tx=0x7fb5a8013000).stop 2022-08-01T19:39:52.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.593+0000 7fb5b3fff700 1 -- 172.21.15.17:0/688644749 shutdown_connections 2022-08-01T19:39:52.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.593+0000 7fb5b3fff700 1 --2- 172.21.15.17:0/688644749 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb590053b00 0x7fb590055fc0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:52.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.593+0000 7fb5b3fff700 1 --2- 172.21.15.17:0/688644749 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb5b4146ef0 0x7fb5b4063190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:52.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.593+0000 7fb5b3fff700 1 --2- 172.21.15.17:0/688644749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5b41465a0 0x7fb5b4146a00 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:52.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.593+0000 7fb5b3fff700 1 --2- 172.21.15.17:0/688644749 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5b40b2710 0x7fb5b40ab720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:52.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.593+0000 7fb5b3fff700 1 -- 172.21.15.17:0/688644749 >> 172.21.15.17:0/688644749 conn(0x7fb5b4060310 msgr2=0x7fb5b4148a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:52.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.593+0000 7fb5b3fff700 1 -- 172.21.15.17:0/688644749 shutdown_connections 2022-08-01T19:39:52.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.593+0000 7fb5b3fff700 1 -- 172.21.15.17:0/688644749 wait complete. 2022-08-01T19:39:52.618 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:39:52.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.737+0000 7f7efb17c700 1 Processor -- start 2022-08-01T19:39:52.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.737+0000 7f7efb17c700 1 -- start start 2022-08-01T19:39:52.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.737+0000 7f7efb17c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7ef40b28b0 0x7f7ef40aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:52.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.737+0000 7f7efb17c700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f7ef40ad4a0 0x7f7ef40af280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:52.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.737+0000 7f7efb17c700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ef40af770 con 0x7f7ef40b0820 2022-08-01T19:39:52.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.737+0000 7f7efb17c700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ef40b2c90 con 0x7f7ef40ad4a0 2022-08-01T19:39:52.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.737+0000 7f7efb17c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef40ac880 con 0x7f7ef40b28b0 2022-08-01T19:39:52.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.737+0000 7f7efa97b700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f7ef40b0820 0x7f7ef405c620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60948/0 (socket says 172.21.15.17:60948) 2022-08-01T19:39:52.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.737+0000 7f7efa97b700 1 -- 172.21.15.17:0/2888024524 learned_addr learned my addr 172.21.15.17:0/2888024524 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:52.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.737+0000 7f7ef9979700 1 --2- 172.21.15.17:0/2888024524 >> v2:172.21.15.77:3300/0 conn(0x7f7ef40ad4a0 0x7f7ef40af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:52.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.737+0000 7f7efa17a700 1 --2- 172.21.15.17:0/2888024524 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7ef40b28b0 0x7f7ef40aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:52.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.737+0000 7f7ef9178700 1 -- 172.21.15.17:0/2888024524 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3273106862 0 0) 0x7f7ef40af770 con 0x7f7ef40b0820 2022-08-01T19:39:52.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.737+0000 7f7ef9178700 1 -- 172.21.15.17:0/2888024524 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7edc003260 con 0x7f7ef40b0820 2022-08-01T19:39:52.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.737+0000 7f7ef9979700 1 -- 172.21.15.17:0/2888024524 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7ef40b28b0 msgr2=0x7f7ef40aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:52.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.737+0000 7f7ef9979700 1 --2- 172.21.15.17:0/2888024524 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7ef40b28b0 0x7f7ef40aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:52.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.737+0000 7f7ef9979700 1 -- 172.21.15.17:0/2888024524 >> v1:172.21.15.17:6789/0 conn(0x7f7ef40b0820 legacy=0x7f7ef405c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:52.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.737+0000 7f7ef9979700 1 -- 172.21.15.17:0/2888024524 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef40acb20 con 0x7f7ef40ad4a0 2022-08-01T19:39:52.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.737+0000 7f7efa17a700 1 --2- 172.21.15.17:0/2888024524 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7ef40b28b0 0x7f7ef40aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:39:52.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.737+0000 7f7ef9979700 1 --2- 172.21.15.17:0/2888024524 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ef40ad4a0 0x7f7ef40af280 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f7ef0002f70 tx=0x7f7ef0006df0).ready entity=mon.1 client_cookie=1beae005c2883206 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:52.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.741+0000 7f7ef9178700 1 -- 172.21.15.17:0/2888024524 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ef000d4a0 con 0x7f7ef40ad4a0 2022-08-01T19:39:52.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.741+0000 7f7ef9178700 1 -- 172.21.15.17:0/2888024524 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef0000e90 con 0x7f7ef40ad4a0 2022-08-01T19:39:52.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.741+0000 7f7ef9178700 1 -- 172.21.15.17:0/2888024524 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ef000d7f0 con 0x7f7ef40ad4a0 2022-08-01T19:39:52.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.741+0000 7f7efb17c700 1 -- 172.21.15.17:0/2888024524 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ef40ad4a0 msgr2=0x7f7ef40af280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:52.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.741+0000 7f7efb17c700 1 --2- 172.21.15.17:0/2888024524 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ef40ad4a0 0x7f7ef40af280 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f7ef0002f70 tx=0x7f7ef0006df0).stop 2022-08-01T19:39:52.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.741+0000 7f7efb17c700 1 -- 172.21.15.17:0/2888024524 shutdown_connections 2022-08-01T19:39:52.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.741+0000 7f7efb17c700 1 --2- 172.21.15.17:0/2888024524 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ef40ad4a0 0x7f7ef40af280 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:52.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.741+0000 7f7efb17c700 1 --2- 172.21.15.17:0/2888024524 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7ef40b28b0 0x7f7ef40aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:52.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.741+0000 7f7efb17c700 1 -- 172.21.15.17:0/2888024524 >> 172.21.15.17:0/2888024524 conn(0x7f7ef4060310 msgr2=0x7f7ef40b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:52.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.741+0000 7f7efb17c700 1 -- 172.21.15.17:0/2888024524 shutdown_connections 2022-08-01T19:39:52.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.741+0000 7f7efb17c700 1 -- 172.21.15.17:0/2888024524 wait complete. 2022-08-01T19:39:52.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.741+0000 7f7efb17c700 1 Processor -- start 2022-08-01T19:39:52.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.741+0000 7f7efb17c700 1 -- start start 2022-08-01T19:39:52.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.741+0000 7f7efb17c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ef40ad4a0 0x7f7ef413eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:52.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.741+0000 7f7efb17c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ef40b0820 0x7f7ef4063240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:52.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.741+0000 7f7efb17c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7ef40b28b0 0x7f7ef41560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:52.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.741+0000 7f7efb17c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef41504f0 con 0x7f7ef40ad4a0 2022-08-01T19:39:52.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.741+0000 7f7efb17c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef41527c0 con 0x7f7ef40b0820 2022-08-01T19:39:52.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.741+0000 7f7efb17c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef4063bf0 con 0x7f7ef40b28b0 2022-08-01T19:39:52.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.741+0000 7f7efa17a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ef40ad4a0 0x7f7ef413eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:52.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.741+0000 7f7efa17a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ef40ad4a0 0x7f7ef413eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40524/0 (socket says 172.21.15.17:40524) 2022-08-01T19:39:52.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.741+0000 7f7efa17a700 1 -- 172.21.15.17:0/1401273731 learned_addr learned my addr 172.21.15.17:0/1401273731 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:52.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.741+0000 7f7efa97b700 1 --2- 172.21.15.17:0/1401273731 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7ef40b28b0 0x7f7ef41560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:52.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.741+0000 7f7ef9979700 1 --2- 172.21.15.17:0/1401273731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ef40b0820 0x7f7ef4063240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:52.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.745+0000 7f7efa17a700 1 -- 172.21.15.17:0/1401273731 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7ef40b28b0 msgr2=0x7f7ef41560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:52.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.745+0000 7f7efa17a700 1 --2- 172.21.15.17:0/1401273731 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7ef40b28b0 0x7f7ef41560b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:52.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.745+0000 7f7efa17a700 1 -- 172.21.15.17:0/1401273731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ef40b0820 msgr2=0x7f7ef4063240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:52.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.745+0000 7f7efa17a700 1 --2- 172.21.15.17:0/1401273731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ef40b0820 0x7f7ef4063240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:52.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.745+0000 7f7efa17a700 1 -- 172.21.15.17:0/1401273731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee400b180 con 0x7f7ef40ad4a0 2022-08-01T19:39:52.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.745+0000 7f7efa17a700 1 --2- 172.21.15.17:0/1401273731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ef40ad4a0 0x7f7ef413eca0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7f7ee400c000 tx=0x7f7ee4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:52.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.745+0000 7f7ee37fe700 1 -- 172.21.15.17:0/1401273731 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ee4010780 con 0x7f7ef40ad4a0 2022-08-01T19:39:52.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.745+0000 7f7ee37fe700 1 -- 172.21.15.17:0/1401273731 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ee40108e0 con 0x7f7ef40ad4a0 2022-08-01T19:39:52.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.745+0000 7f7ee37fe700 1 -- 172.21.15.17:0/1401273731 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ee4010c40 con 0x7f7ef40ad4a0 2022-08-01T19:39:52.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.745+0000 7f7efb17c700 1 -- 172.21.15.17:0/1401273731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ef000b180 con 0x7f7ef40ad4a0 2022-08-01T19:39:52.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.745+0000 7f7efb17c700 1 -- 172.21.15.17:0/1401273731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ef4064130 con 0x7f7ef40ad4a0 2022-08-01T19:39:52.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.745+0000 7f7efb17c700 1 -- 172.21.15.17:0/1401273731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ef4064330 con 0x7f7ef40ad4a0 2022-08-01T19:39:52.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.745+0000 7f7ee37fe700 1 -- 172.21.15.17:0/1401273731 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7ee401c460 con 0x7f7ef40ad4a0 2022-08-01T19:39:52.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.745+0000 7f7ee37fe700 1 --2- 172.21.15.17:0/1401273731 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7ed4053980 0x7f7ed4055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:52.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.745+0000 7f7ee37fe700 1 -- 172.21.15.17:0/1401273731 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7ee406c4a0 con 0x7f7ef40ad4a0 2022-08-01T19:39:52.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.749+0000 7f7ef9979700 1 --2- 172.21.15.17:0/1401273731 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7ed4053980 0x7f7ed4055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:52.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.749+0000 7f7ef9979700 1 --2- 172.21.15.17:0/1401273731 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7ed4053980 0x7f7ed4055e40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f7ef001a000 tx=0x7f7ef0014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:52.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.753+0000 7f7ee37fe700 1 -- 172.21.15.17:0/1401273731 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7ee4043200 con 0x7f7ef40ad4a0 2022-08-01T19:39:52.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.877+0000 7f7efb17c700 1 -- 172.21.15.17:0/1401273731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ef40638d0 con 0x7f7ef40ad4a0 2022-08-01T19:39:52.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.881+0000 7f7ee37fe700 1 -- 172.21.15.17:0/1401273731 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f7ee4046820 con 0x7f7ef40ad4a0 2022-08-01T19:39:52.885 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:39:52.885 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":75,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:39:52.325144+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:39:50.853967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2022-08-01T19:39:51.901692+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:39:52.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.881+0000 7f7efb17c700 1 -- 172.21.15.17:0/1401273731 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7ed4053980 msgr2=0x7f7ed4055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:52.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.881+0000 7f7efb17c700 1 --2- 172.21.15.17:0/1401273731 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7ed4053980 0x7f7ed4055e40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f7ef001a000 tx=0x7f7ef0014000).stop 2022-08-01T19:39:52.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.881+0000 7f7efb17c700 1 -- 172.21.15.17:0/1401273731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ef40ad4a0 msgr2=0x7f7ef413eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:52.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.881+0000 7f7efb17c700 1 --2- 172.21.15.17:0/1401273731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ef40ad4a0 0x7f7ef413eca0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7f7ee400c000 tx=0x7f7ee4006df0).stop 2022-08-01T19:39:52.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.881+0000 7f7efb17c700 1 -- 172.21.15.17:0/1401273731 shutdown_connections 2022-08-01T19:39:52.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.881+0000 7f7efb17c700 1 --2- 172.21.15.17:0/1401273731 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7ed4053980 0x7f7ed4055e40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:52.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.881+0000 7f7efb17c700 1 --2- 172.21.15.17:0/1401273731 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7ef40b28b0 0x7f7ef41560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:52.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.881+0000 7f7efb17c700 1 --2- 172.21.15.17:0/1401273731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ef40b0820 0x7f7ef4063240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:52.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.881+0000 7f7efb17c700 1 --2- 172.21.15.17:0/1401273731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ef40ad4a0 0x7f7ef413eca0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:52.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.881+0000 7f7efb17c700 1 -- 172.21.15.17:0/1401273731 >> 172.21.15.17:0/1401273731 conn(0x7f7ef4060310 msgr2=0x7f7ef40b5340 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:52.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.885+0000 7f7efb17c700 1 -- 172.21.15.17:0/1401273731 shutdown_connections 2022-08-01T19:39:52.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:52.885+0000 7f7efb17c700 1 -- 172.21.15.17:0/1401273731 wait complete. 2022-08-01T19:39:52.901 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-08-01T19:39:53.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.013+0000 7f95d62ad700 1 Processor -- start 2022-08-01T19:39:53.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.013+0000 7f95d62ad700 1 -- start start 2022-08-01T19:39:53.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.017+0000 7f95d62ad700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f95d00b2710 0x7f95d014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:53.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.017+0000 7f95d62ad700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f95d00b01a0 0x7f95d014b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:53.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.017+0000 7f95d62ad700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f95d00afd60 0x7f95d00a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:53.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.017+0000 7f95d62ad700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95d00a7250 con 0x7f95d00afd60 2022-08-01T19:39:53.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.017+0000 7f95d62ad700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95d00a7480 con 0x7f95d00b01a0 2022-08-01T19:39:53.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.017+0000 7f95d62ad700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d00a7720 con 0x7f95d00b2710 2022-08-01T19:39:53.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.017+0000 7f95d5aac700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f95d00afd60 0x7f95d00a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:53.026 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.017+0000 7f95d5aac700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f95d00afd60 0x7f95d00a8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40536/0 (socket says 172.21.15.17:40536) 2022-08-01T19:39:53.026 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.017+0000 7f95d5aac700 1 -- 172.21.15.17:0/211715187 learned_addr learned my addr 172.21.15.17:0/211715187 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:53.026 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.017+0000 7f95d4aaa700 1 --2- 172.21.15.17:0/211715187 >> v2:172.21.15.77:3300/0 conn(0x7f95d00b01a0 0x7f95d014b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:53.027 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.017+0000 7f95d52ab700 1 --2- 172.21.15.17:0/211715187 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f95d00b2710 0x7f95d014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:53.027 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.017+0000 7f95d5aac700 1 -- 172.21.15.17:0/211715187 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f95d00b2710 msgr2=0x7f95d014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:53.027 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.017+0000 7f95d5aac700 1 --2- 172.21.15.17:0/211715187 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f95d00b2710 0x7f95d014afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:53.027 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.017+0000 7f95d5aac700 1 -- 172.21.15.17:0/211715187 >> v2:172.21.15.77:3300/0 conn(0x7f95d00b01a0 msgr2=0x7f95d014b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:53.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.017+0000 7f95d5aac700 1 --2- 172.21.15.17:0/211715187 >> v2:172.21.15.77:3300/0 conn(0x7f95d00b01a0 0x7f95d014b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:53.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.017+0000 7f95d5aac700 1 -- 172.21.15.17:0/211715187 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d00a79c0 con 0x7f95d00afd60 2022-08-01T19:39:53.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.017+0000 7f95d5aac700 1 --2- 172.21.15.17:0/211715187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95d00afd60 0x7f95d00a8540 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7f95c80032f0 tx=0x7f95c8009290).ready entity=mon.0 client_cookie=308f740971d91c5b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:53.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.017+0000 7f95cffff700 1 -- 172.21.15.17:0/211715187 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95c800b680 con 0x7f95d00afd60 2022-08-01T19:39:53.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.017+0000 7f95cffff700 1 -- 172.21.15.17:0/211715187 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c800b7e0 con 0x7f95d00afd60 2022-08-01T19:39:53.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.017+0000 7f95cffff700 1 -- 172.21.15.17:0/211715187 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95c800b950 con 0x7f95d00afd60 2022-08-01T19:39:53.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.017+0000 7f95d62ad700 1 -- 172.21.15.17:0/211715187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95d00afd60 msgr2=0x7f95d00a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:53.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.017+0000 7f95d62ad700 1 --2- 172.21.15.17:0/211715187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95d00afd60 0x7f95d00a8540 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7f95c80032f0 tx=0x7f95c8009290).stop 2022-08-01T19:39:53.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.017+0000 7f95d62ad700 1 -- 172.21.15.17:0/211715187 shutdown_connections 2022-08-01T19:39:53.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.017+0000 7f95d62ad700 1 --2- 172.21.15.17:0/211715187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95d00afd60 0x7f95d00a8540 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:53.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.017+0000 7f95d62ad700 1 --2- 172.21.15.17:0/211715187 >> v2:172.21.15.77:3300/0 conn(0x7f95d00b01a0 0x7f95d014b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:53.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.017+0000 7f95d62ad700 1 --2- 172.21.15.17:0/211715187 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f95d00b2710 0x7f95d014afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:53.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.017+0000 7f95d62ad700 1 -- 172.21.15.17:0/211715187 >> 172.21.15.17:0/211715187 conn(0x7f95d0060310 msgr2=0x7f95d00b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:53.040 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.017+0000 7f95d62ad700 1 -- 172.21.15.17:0/211715187 shutdown_connections 2022-08-01T19:39:53.040 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.017+0000 7f95d62ad700 1 -- 172.21.15.17:0/211715187 wait complete. 2022-08-01T19:39:53.040 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.021+0000 7f95d62ad700 1 Processor -- start 2022-08-01T19:39:53.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.021+0000 7f95d62ad700 1 -- start start 2022-08-01T19:39:53.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.021+0000 7f95d62ad700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95d00afd60 0x7f95d0148b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:53.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.021+0000 7f95d62ad700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f95d00b01a0 0x7f95d0156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:53.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.021+0000 7f95d62ad700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95d00b2710 0x7f95d0156980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:53.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.021+0000 7f95d62ad700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d014d430 con 0x7f95d00afd60 2022-08-01T19:39:53.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.021+0000 7f95d62ad700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d015a5d0 con 0x7f95d00b2710 2022-08-01T19:39:53.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.021+0000 7f95d62ad700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d0156e70 con 0x7f95d00b01a0 2022-08-01T19:39:53.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.025+0000 7f95d52ab700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95d00afd60 0x7f95d0148b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:53.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.025+0000 7f95d52ab700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95d00afd60 0x7f95d0148b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40538/0 (socket says 172.21.15.17:40538) 2022-08-01T19:39:53.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.025+0000 7f95d52ab700 1 -- 172.21.15.17:0/662906951 learned_addr learned my addr 172.21.15.17:0/662906951 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:53.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.025+0000 7f95d4aaa700 1 --2- 172.21.15.17:0/662906951 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f95d00b01a0 0x7f95d0156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:53.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.025+0000 7f95d5aac700 1 --2- 172.21.15.17:0/662906951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95d00b2710 0x7f95d0156980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:53.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.025+0000 7f95d52ab700 1 -- 172.21.15.17:0/662906951 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f95d00b01a0 msgr2=0x7f95d0156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:53.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.025+0000 7f95d52ab700 1 --2- 172.21.15.17:0/662906951 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f95d00b01a0 0x7f95d0156490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:53.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.025+0000 7f95d52ab700 1 -- 172.21.15.17:0/662906951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95d00b2710 msgr2=0x7f95d0156980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:53.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.025+0000 7f95d52ab700 1 --2- 172.21.15.17:0/662906951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95d00b2710 0x7f95d0156980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:53.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.025+0000 7f95d52ab700 1 -- 172.21.15.17:0/662906951 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c8009530 con 0x7f95d00afd60 2022-08-01T19:39:53.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.025+0000 7f95d52ab700 1 --2- 172.21.15.17:0/662906951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95d00afd60 0x7f95d0148b90 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f95c400c000 tx=0x7f95c4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:53.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.025+0000 7f95ce7fc700 1 -- 172.21.15.17:0/662906951 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95c4010780 con 0x7f95d00afd60 2022-08-01T19:39:53.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.025+0000 7f95ce7fc700 1 -- 172.21.15.17:0/662906951 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c40108e0 con 0x7f95d00afd60 2022-08-01T19:39:53.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.025+0000 7f95d62ad700 1 -- 172.21.15.17:0/662906951 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d015a870 con 0x7f95d00afd60 2022-08-01T19:39:53.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.025+0000 7f95ce7fc700 1 -- 172.21.15.17:0/662906951 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95c4010c40 con 0x7f95d00afd60 2022-08-01T19:39:53.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.025+0000 7f95d62ad700 1 -- 172.21.15.17:0/662906951 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d014c8e0 con 0x7f95d00afd60 2022-08-01T19:39:53.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.025+0000 7f95ce7fc700 1 -- 172.21.15.17:0/662906951 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f95c401c460 con 0x7f95d00afd60 2022-08-01T19:39:53.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.025+0000 7f95d62ad700 1 -- 172.21.15.17:0/662906951 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95d015a410 con 0x7f95d00afd60 2022-08-01T19:39:53.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.029+0000 7f95ce7fc700 1 --2- 172.21.15.17:0/662906951 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f95b0053aa0 0x7f95b0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:53.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.029+0000 7f95ce7fc700 1 -- 172.21.15.17:0/662906951 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f95c406c570 con 0x7f95d00afd60 2022-08-01T19:39:53.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.029+0000 7f95d4aaa700 1 --2- 172.21.15.17:0/662906951 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f95b0053aa0 0x7f95b0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:53.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.029+0000 7f95d4aaa700 1 --2- 172.21.15.17:0/662906951 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f95b0053aa0 0x7f95b0055f60 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7f95c000b000 tx=0x7f95c0006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:53.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.033+0000 7f95ce7fc700 1 -- 172.21.15.17:0/662906951 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f95c40431c0 con 0x7f95d00afd60 2022-08-01T19:39:53.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.197+0000 7f95d62ad700 1 -- 172.21.15.17:0/662906951 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f95d00574e0 con 0x7f95d00afd60 2022-08-01T19:39:53.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.333+0000 7f95ce7fc700 1 -- 172.21.15.17:0/662906951 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v35) v1 ==== 118+0+0 (secure 0 0 0) 0x7f95c40467e0 con 0x7f95d00afd60 2022-08-01T19:39:53.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.337+0000 7f95d62ad700 1 -- 172.21.15.17:0/662906951 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f95b0053aa0 msgr2=0x7f95b0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:53.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.337+0000 7f95d62ad700 1 --2- 172.21.15.17:0/662906951 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f95b0053aa0 0x7f95b0055f60 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7f95c000b000 tx=0x7f95c0006df0).stop 2022-08-01T19:39:53.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.337+0000 7f95d62ad700 1 -- 172.21.15.17:0/662906951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95d00afd60 msgr2=0x7f95d0148b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:53.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.337+0000 7f95d62ad700 1 --2- 172.21.15.17:0/662906951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95d00afd60 0x7f95d0148b90 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f95c400c000 tx=0x7f95c4006df0).stop 2022-08-01T19:39:53.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.337+0000 7f95d62ad700 1 -- 172.21.15.17:0/662906951 shutdown_connections 2022-08-01T19:39:53.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.337+0000 7f95d62ad700 1 --2- 172.21.15.17:0/662906951 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f95b0053aa0 0x7f95b0055f60 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:53.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.337+0000 7f95d62ad700 1 --2- 172.21.15.17:0/662906951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95d00b2710 0x7f95d0156980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:53.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.337+0000 7f95d62ad700 1 --2- 172.21.15.17:0/662906951 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f95d00b01a0 0x7f95d0156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:53.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.337+0000 7f95d62ad700 1 --2- 172.21.15.17:0/662906951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95d00afd60 0x7f95d0148b90 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:53.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.337+0000 7f95d62ad700 1 -- 172.21.15.17:0/662906951 >> 172.21.15.17:0/662906951 conn(0x7f95d0060310 msgr2=0x7f95d00676e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:53.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.337+0000 7f95d62ad700 1 -- 172.21.15.17:0/662906951 shutdown_connections 2022-08-01T19:39:53.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.337+0000 7f95d62ad700 1 -- 172.21.15.17:0/662906951 wait complete. 2022-08-01T19:39:53.357 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:39:53.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.465+0000 7fb17ffff700 1 Processor -- start 2022-08-01T19:39:53.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.465+0000 7fb17ffff700 1 -- start start 2022-08-01T19:39:53.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.465+0000 7fb17ffff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb1800614b0 0x7fb180061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:53.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.465+0000 7fb17ffff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fb1801405d0 0x7fb180140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:53.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.465+0000 7fb17ffff700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1800aee80 con 0x7fb1800ad750 2022-08-01T19:39:53.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.465+0000 7fb17ffff700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1800b70b0 con 0x7fb1800614b0 2022-08-01T19:39:53.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.465+0000 7fb17ffff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1800af0b0 con 0x7fb1801405d0 2022-08-01T19:39:53.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.465+0000 7fb17effd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb1800614b0 0x7fb180061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:53.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.465+0000 7fb17effd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb1800614b0 0x7fb180061890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40548/0 (socket says 172.21.15.17:40548) 2022-08-01T19:39:53.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.465+0000 7fb17effd700 1 -- 172.21.15.17:0/2355433715 learned_addr learned my addr 172.21.15.17:0/2355433715 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:53.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.465+0000 7fb17f7fe700 1 --2- 172.21.15.17:0/2355433715 >> v2:172.21.15.77:3300/0 conn(0x7fb1801405d0 0x7fb180140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:53.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.465+0000 7fb17effd700 1 -- 172.21.15.17:0/2355433715 >> v2:172.21.15.77:3300/0 conn(0x7fb1801405d0 msgr2=0x7fb180140a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:53.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.465+0000 7fb17effd700 1 --2- 172.21.15.17:0/2355433715 >> v2:172.21.15.77:3300/0 conn(0x7fb1801405d0 0x7fb180140a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:53.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.465+0000 7fb17effd700 1 -- 172.21.15.17:0/2355433715 >> v1:172.21.15.77:6789/0 conn(0x7fb1800ad750 legacy=0x7fb1800b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:53.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.465+0000 7fb17effd700 1 -- 172.21.15.17:0/2355433715 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1800af350 con 0x7fb1800614b0 2022-08-01T19:39:53.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.465+0000 7fb17effd700 1 --2- 172.21.15.17:0/2355433715 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1800614b0 0x7fb180061890 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7fb1800aed30 tx=0x7fb174006eb0).ready entity=mon.0 client_cookie=7a90964feab82917 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:53.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.465+0000 7fb17dffb700 1 -- 172.21.15.17:0/2355433715 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb17400d610 con 0x7fb1800614b0 2022-08-01T19:39:53.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.465+0000 7fb17dffb700 1 -- 172.21.15.17:0/2355433715 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb174000e90 con 0x7fb1800614b0 2022-08-01T19:39:53.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.465+0000 7fb17dffb700 1 -- 172.21.15.17:0/2355433715 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb17400d9a0 con 0x7fb1800614b0 2022-08-01T19:39:53.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.469+0000 7fb17ffff700 1 -- 172.21.15.17:0/2355433715 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1800614b0 msgr2=0x7fb180061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:53.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.469+0000 7fb17ffff700 1 --2- 172.21.15.17:0/2355433715 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1800614b0 0x7fb180061890 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7fb1800aed30 tx=0x7fb174006eb0).stop 2022-08-01T19:39:53.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.469+0000 7fb17ffff700 1 -- 172.21.15.17:0/2355433715 shutdown_connections 2022-08-01T19:39:53.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.469+0000 7fb17ffff700 1 --2- 172.21.15.17:0/2355433715 >> v2:172.21.15.77:3300/0 conn(0x7fb1801405d0 0x7fb180140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:53.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.469+0000 7fb17ffff700 1 --2- 172.21.15.17:0/2355433715 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1800614b0 0x7fb180061890 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:53.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.469+0000 7fb17ffff700 1 -- 172.21.15.17:0/2355433715 >> 172.21.15.17:0/2355433715 conn(0x7fb180066f70 msgr2=0x7fb180067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:53.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.469+0000 7fb17ffff700 1 -- 172.21.15.17:0/2355433715 shutdown_connections 2022-08-01T19:39:53.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.469+0000 7fb17ffff700 1 -- 172.21.15.17:0/2355433715 wait complete. 2022-08-01T19:39:53.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.469+0000 7fb17ffff700 1 Processor -- start 2022-08-01T19:39:53.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.469+0000 7fb17ffff700 1 -- start start 2022-08-01T19:39:53.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.469+0000 7fb17ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1800614b0 0x7fb1800b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:53.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.469+0000 7fb17ffff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1800ad750 0x7fb180143640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:53.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.473+0000 7fb17ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1801405d0 0x7fb180155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:53.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.473+0000 7fb17ffff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1800b3570 con 0x7fb1800614b0 2022-08-01T19:39:53.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.473+0000 7fb17ffff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb180153600 con 0x7fb1801405d0 2022-08-01T19:39:53.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.473+0000 7fb17ffff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1800b2910 con 0x7fb1800ad750 2022-08-01T19:39:53.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.473+0000 7fb17effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1800614b0 0x7fb1800b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:53.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.473+0000 7fb17effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1800614b0 0x7fb1800b1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40552/0 (socket says 172.21.15.17:40552) 2022-08-01T19:39:53.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.473+0000 7fb17effd700 1 -- 172.21.15.17:0/1809001184 learned_addr learned my addr 172.21.15.17:0/1809001184 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:53.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.473+0000 7fb17f7fe700 1 --2- 172.21.15.17:0/1809001184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1801405d0 0x7fb180155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:53.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.473+0000 7fb17e7fc700 1 --2- 172.21.15.17:0/1809001184 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1800ad750 0x7fb180143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:53.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.473+0000 7fb17effd700 1 -- 172.21.15.17:0/1809001184 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1800ad750 msgr2=0x7fb180143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:53.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.473+0000 7fb17effd700 1 --2- 172.21.15.17:0/1809001184 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1800ad750 0x7fb180143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:53.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.473+0000 7fb17effd700 1 -- 172.21.15.17:0/1809001184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1801405d0 msgr2=0x7fb180155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:53.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.473+0000 7fb17effd700 1 --2- 172.21.15.17:0/1809001184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1801405d0 0x7fb180155b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:53.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.473+0000 7fb17effd700 1 -- 172.21.15.17:0/1809001184 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb17400b1e0 con 0x7fb1800614b0 2022-08-01T19:39:53.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.473+0000 7fb17effd700 1 --2- 172.21.15.17:0/1809001184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1800614b0 0x7fb1800b1b70 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7fb1800aed30 tx=0x7fb174015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:53.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.473+0000 7fb16ffff700 1 -- 172.21.15.17:0/1809001184 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb174011860 con 0x7fb1800614b0 2022-08-01T19:39:53.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.473+0000 7fb16ffff700 1 -- 172.21.15.17:0/1809001184 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1740119c0 con 0x7fb1800614b0 2022-08-01T19:39:53.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.473+0000 7fb17ffff700 1 -- 172.21.15.17:0/1809001184 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1800b21b0 con 0x7fb1800614b0 2022-08-01T19:39:53.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.473+0000 7fb16ffff700 1 -- 172.21.15.17:0/1809001184 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb174011d20 con 0x7fb1800614b0 2022-08-01T19:39:53.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.473+0000 7fb17ffff700 1 -- 172.21.15.17:0/1809001184 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1800b3e10 con 0x7fb1800614b0 2022-08-01T19:39:53.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.473+0000 7fb17ffff700 1 -- 172.21.15.17:0/1809001184 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb180143cd0 con 0x7fb1800614b0 2022-08-01T19:39:53.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.473+0000 7fb16ffff700 1 -- 172.21.15.17:0/1809001184 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb174025030 con 0x7fb1800614b0 2022-08-01T19:39:53.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.473+0000 7fb16ffff700 1 --2- 172.21.15.17:0/1809001184 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb15c053950 0x7fb15c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:53.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.473+0000 7fb16ffff700 1 -- 172.21.15.17:0/1809001184 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb17406d1c0 con 0x7fb1800614b0 2022-08-01T19:39:53.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.477+0000 7fb17e7fc700 1 --2- 172.21.15.17:0/1809001184 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb15c053950 0x7fb15c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:53.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.477+0000 7fb17e7fc700 1 --2- 172.21.15.17:0/1809001184 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb15c053950 0x7fb15c055e10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7fb17000d000 tx=0x7fb170000f40).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:53.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.481+0000 7fb16ffff700 1 -- 172.21.15.17:0/1809001184 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb174043f80 con 0x7fb1800614b0 2022-08-01T19:39:53.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.645+0000 7fb17ffff700 1 -- 172.21.15.17:0/1809001184 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1800574e0 con 0x7fb1800614b0 2022-08-01T19:39:53.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.645+0000 7fb16ffff700 1 -- 172.21.15.17:0/1809001184 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+2493 (secure 0 0 0) 0x7fb174023020 con 0x7fb1800614b0 2022-08-01T19:39:53.653 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:39:53.654 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":35,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5615,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/2284778179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2284778179},{"type":"v1","addr":"172.21.15.77:6835","nonce":2284778179}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":34},{"gid":6261,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/1827111044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":1827111044},{"type":"v1","addr":"172.21.15.17:6833","nonce":1827111044}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":33}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:39:52.325506+0000","modified":"2022-08-01T19:39:53.328610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6267":{"gid":6267,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/3370152344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":3370152344},{"type":"v1","addr":"172.21.15.17:6835","nonce":3370152344}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-08-01T19:39:53.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.645+0000 7fb17ffff700 1 -- 172.21.15.17:0/1809001184 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb15c053950 msgr2=0x7fb15c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:53.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.645+0000 7fb17ffff700 1 --2- 172.21.15.17:0/1809001184 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb15c053950 0x7fb15c055e10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7fb17000d000 tx=0x7fb170000f40).stop 2022-08-01T19:39:53.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.645+0000 7fb17ffff700 1 -- 172.21.15.17:0/1809001184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1800614b0 msgr2=0x7fb1800b1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:53.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.645+0000 7fb17ffff700 1 --2- 172.21.15.17:0/1809001184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1800614b0 0x7fb1800b1b70 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7fb1800aed30 tx=0x7fb174015000).stop 2022-08-01T19:39:53.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.645+0000 7fb17ffff700 1 -- 172.21.15.17:0/1809001184 shutdown_connections 2022-08-01T19:39:53.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.645+0000 7fb17ffff700 1 --2- 172.21.15.17:0/1809001184 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb15c053950 0x7fb15c055e10 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:53.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.645+0000 7fb17ffff700 1 --2- 172.21.15.17:0/1809001184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1801405d0 0x7fb180155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:53.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.645+0000 7fb17ffff700 1 --2- 172.21.15.17:0/1809001184 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1800ad750 0x7fb180143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:53.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.645+0000 7fb17ffff700 1 --2- 172.21.15.17:0/1809001184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1800614b0 0x7fb1800b1b70 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:53.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.645+0000 7fb17ffff700 1 -- 172.21.15.17:0/1809001184 >> 172.21.15.17:0/1809001184 conn(0x7fb180066f70 msgr2=0x7fb1800b8de0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:53.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.645+0000 7fb17ffff700 1 -- 172.21.15.17:0/1809001184 shutdown_connections 2022-08-01T19:39:53.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.645+0000 7fb17ffff700 1 -- 172.21.15.17:0/1809001184 wait complete. 2022-08-01T19:39:53.664 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 35 2022-08-01T19:39:53.666 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:39:53.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.773+0000 7f9ac8f0c700 1 Processor -- start 2022-08-01T19:39:53.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.773+0000 7f9ac8f0c700 1 -- start start 2022-08-01T19:39:53.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.773+0000 7f9ac8f0c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ac40b36e0 0x7f9ac4141a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:53.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.773+0000 7f9ac8f0c700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f9ac4144e60 0x7f9ac4066ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:53.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.773+0000 7f9ac8f0c700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ac4064ef0 con 0x7f9ac4064b10 2022-08-01T19:39:53.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.773+0000 7f9ac8f0c700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ac413ebf0 con 0x7f9ac4144e60 2022-08-01T19:39:53.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.773+0000 7f9ac8f0c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac41452d0 con 0x7f9ac40b36e0 2022-08-01T19:39:53.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.773+0000 7f9ac37fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9ac4064b10 0x7f9ac4141370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60988/0 (socket says 172.21.15.17:60988) 2022-08-01T19:39:53.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.773+0000 7f9ac37fe700 1 -- 172.21.15.17:0/3565373193 learned_addr learned my addr 172.21.15.17:0/3565373193 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:53.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.773+0000 7f9ac3fff700 1 --2- 172.21.15.17:0/3565373193 >> v2:172.21.15.77:3300/0 conn(0x7f9ac4144e60 0x7f9ac4066ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:53.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.773+0000 7f9ac2ffd700 1 --2- 172.21.15.17:0/3565373193 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ac40b36e0 0x7f9ac4141a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:53.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.777+0000 7f9ac27fc700 1 -- 172.21.15.17:0/3565373193 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2987138015 0 0) 0x7f9ac4064ef0 con 0x7f9ac4064b10 2022-08-01T19:39:53.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.777+0000 7f9ac27fc700 1 -- 172.21.15.17:0/3565373193 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ab0003260 con 0x7f9ac4064b10 2022-08-01T19:39:53.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.777+0000 7f9ac2ffd700 1 -- 172.21.15.17:0/3565373193 >> v2:172.21.15.77:3300/0 conn(0x7f9ac4144e60 msgr2=0x7f9ac4066ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:53.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.777+0000 7f9ac2ffd700 1 --2- 172.21.15.17:0/3565373193 >> v2:172.21.15.77:3300/0 conn(0x7f9ac4144e60 0x7f9ac4066ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:53.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.777+0000 7f9ac2ffd700 1 -- 172.21.15.17:0/3565373193 >> v1:172.21.15.17:6789/0 conn(0x7f9ac4064b10 legacy=0x7f9ac4141370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:53.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.777+0000 7f9ac2ffd700 1 -- 172.21.15.17:0/3565373193 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac4066690 con 0x7f9ac40b36e0 2022-08-01T19:39:53.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.777+0000 7f9ac3fff700 1 --2- 172.21.15.17:0/3565373193 >> v2:172.21.15.77:3300/0 conn(0x7f9ac4144e60 0x7f9ac4066ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:39:53.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.777+0000 7f9ac2ffd700 1 --2- 172.21.15.17:0/3565373193 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ac40b36e0 0x7f9ac4141a80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f9ab4002f70 tx=0x7f9ab4006df0).ready entity=mon.2 client_cookie=e7abe5a2ee3689e4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:53.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.777+0000 7f9ac27fc700 1 -- 172.21.15.17:0/3565373193 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ab400d4a0 con 0x7f9ac40b36e0 2022-08-01T19:39:53.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.777+0000 7f9ac27fc700 1 -- 172.21.15.17:0/3565373193 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab4000e90 con 0x7f9ac40b36e0 2022-08-01T19:39:53.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.777+0000 7f9ac27fc700 1 -- 172.21.15.17:0/3565373193 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ab400d7f0 con 0x7f9ac40b36e0 2022-08-01T19:39:53.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.777+0000 7f9ac8f0c700 1 -- 172.21.15.17:0/3565373193 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ac40b36e0 msgr2=0x7f9ac4141a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:53.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.777+0000 7f9ac8f0c700 1 --2- 172.21.15.17:0/3565373193 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ac40b36e0 0x7f9ac4141a80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f9ab4002f70 tx=0x7f9ab4006df0).stop 2022-08-01T19:39:53.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.777+0000 7f9ac8f0c700 1 -- 172.21.15.17:0/3565373193 shutdown_connections 2022-08-01T19:39:53.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.777+0000 7f9ac8f0c700 1 --2- 172.21.15.17:0/3565373193 >> v2:172.21.15.77:3300/0 conn(0x7f9ac4144e60 0x7f9ac4066ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:53.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.777+0000 7f9ac8f0c700 1 --2- 172.21.15.17:0/3565373193 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ac40b36e0 0x7f9ac4141a80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:53.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.777+0000 7f9ac8f0c700 1 -- 172.21.15.17:0/3565373193 >> 172.21.15.17:0/3565373193 conn(0x7f9ac405f840 msgr2=0x7f9ac405fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:53.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.777+0000 7f9ac8f0c700 1 -- 172.21.15.17:0/3565373193 shutdown_connections 2022-08-01T19:39:53.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.777+0000 7f9ac8f0c700 1 -- 172.21.15.17:0/3565373193 wait complete. 2022-08-01T19:39:53.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.781+0000 7f9ac8f0c700 1 Processor -- start 2022-08-01T19:39:53.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.781+0000 7f9ac8f0c700 1 -- start start 2022-08-01T19:39:53.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.781+0000 7f9ac8f0c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ac4064b10 0x7f9ac41425a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:53.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.781+0000 7f9ac8f0c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ac40b36e0 0x7f9ac4062510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:53.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.781+0000 7f9ac8f0c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ac4144e60 0x7f9ac413a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:53.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.781+0000 7f9ac8f0c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac4144160 con 0x7f9ac4144e60 2022-08-01T19:39:53.802 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.781+0000 7f9ac8f0c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac4144400 con 0x7f9ac40b36e0 2022-08-01T19:39:53.802 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.781+0000 7f9ac8f0c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac4146810 con 0x7f9ac4064b10 2022-08-01T19:39:53.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.781+0000 7f9ac3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ac4144e60 0x7f9ac413a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:53.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.781+0000 7f9ac3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ac4144e60 0x7f9ac413a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40570/0 (socket says 172.21.15.17:40570) 2022-08-01T19:39:53.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.781+0000 7f9ac3fff700 1 -- 172.21.15.17:0/375885930 learned_addr learned my addr 172.21.15.17:0/375885930 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:53.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.781+0000 7f9ac2ffd700 1 --2- 172.21.15.17:0/375885930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ac40b36e0 0x7f9ac4062510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:53.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.781+0000 7f9ac37fe700 1 --2- 172.21.15.17:0/375885930 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ac4064b10 0x7f9ac41425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:53.804 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.781+0000 7f9ac3fff700 1 -- 172.21.15.17:0/375885930 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ac4064b10 msgr2=0x7f9ac41425a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:53.804 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.781+0000 7f9ac3fff700 1 --2- 172.21.15.17:0/375885930 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ac4064b10 0x7f9ac41425a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:53.804 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.781+0000 7f9ac3fff700 1 -- 172.21.15.17:0/375885930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ac40b36e0 msgr2=0x7f9ac4062510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:53.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.781+0000 7f9ac3fff700 1 --2- 172.21.15.17:0/375885930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ac40b36e0 0x7f9ac4062510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:53.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.781+0000 7f9ac3fff700 1 -- 172.21.15.17:0/375885930 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9abc009470 con 0x7f9ac4144e60 2022-08-01T19:39:53.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.781+0000 7f9ac3fff700 1 --2- 172.21.15.17:0/375885930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ac4144e60 0x7f9ac413a920 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f9abc00c000 tx=0x7f9abc0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:53.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.781+0000 7f9ac0ff9700 1 -- 172.21.15.17:0/375885930 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9abc00c850 con 0x7f9ac4144e60 2022-08-01T19:39:53.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.781+0000 7f9ac0ff9700 1 -- 172.21.15.17:0/375885930 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9abc00c9b0 con 0x7f9ac4144e60 2022-08-01T19:39:53.807 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.781+0000 7f9ac8f0c700 1 -- 172.21.15.17:0/375885930 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab400b180 con 0x7f9ac4144e60 2022-08-01T19:39:53.807 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.781+0000 7f9ac0ff9700 1 -- 172.21.15.17:0/375885930 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9abc00cd10 con 0x7f9ac4144e60 2022-08-01T19:39:53.807 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.781+0000 7f9ac8f0c700 1 -- 172.21.15.17:0/375885930 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac4146f50 con 0x7f9ac4144e60 2022-08-01T19:39:53.807 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.785+0000 7f9ac8f0c700 1 -- 172.21.15.17:0/375885930 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac40b4850 con 0x7f9ac4144e60 2022-08-01T19:39:53.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.785+0000 7f9ac0ff9700 1 -- 172.21.15.17:0/375885930 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9abc022070 con 0x7f9ac4144e60 2022-08-01T19:39:53.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.785+0000 7f9ac0ff9700 1 --2- 172.21.15.17:0/375885930 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9aa40539e0 0x7f9aa4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:53.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.785+0000 7f9ac0ff9700 1 -- 172.21.15.17:0/375885930 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9abc06c7e0 con 0x7f9ac4144e60 2022-08-01T19:39:53.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.785+0000 7f9ac37fe700 1 --2- 172.21.15.17:0/375885930 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9aa40539e0 0x7f9aa4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:53.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.785+0000 7f9ac37fe700 1 --2- 172.21.15.17:0/375885930 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9aa40539e0 0x7f9aa4055ea0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f9ab800d000 tx=0x7f9ab8009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:53.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.793+0000 7f9ac0ff9700 1 -- 172.21.15.17:0/375885930 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9abc0435a0 con 0x7f9ac4144e60 2022-08-01T19:39:53.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.921+0000 7f9ac8f0c700 1 -- 172.21.15.17:0/375885930 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ac40574e0 con 0x7f9ac4144e60 2022-08-01T19:39:53.925 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.925+0000 7f9ac0ff9700 1 -- 172.21.15.17:0/375885930 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f9abc016090 con 0x7f9ac4144e60 2022-08-01T19:39:53.925 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:39:53.926 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":76,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:39:53.329285+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:39:50.853967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2022-08-01T19:39:51.901692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:39:53.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.925+0000 7f9ac8f0c700 1 -- 172.21.15.17:0/375885930 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9aa40539e0 msgr2=0x7f9aa4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:53.929 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.925+0000 7f9ac8f0c700 1 --2- 172.21.15.17:0/375885930 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9aa40539e0 0x7f9aa4055ea0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f9ab800d000 tx=0x7f9ab8009040).stop 2022-08-01T19:39:53.929 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.925+0000 7f9ac8f0c700 1 -- 172.21.15.17:0/375885930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ac4144e60 msgr2=0x7f9ac413a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:53.930 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.925+0000 7f9ac8f0c700 1 --2- 172.21.15.17:0/375885930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ac4144e60 0x7f9ac413a920 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f9abc00c000 tx=0x7f9abc0091d0).stop 2022-08-01T19:39:53.930 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.925+0000 7f9ac8f0c700 1 -- 172.21.15.17:0/375885930 shutdown_connections 2022-08-01T19:39:53.931 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.925+0000 7f9ac8f0c700 1 --2- 172.21.15.17:0/375885930 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9aa40539e0 0x7f9aa4055ea0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:53.931 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.925+0000 7f9ac8f0c700 1 --2- 172.21.15.17:0/375885930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ac4144e60 0x7f9ac413a920 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:53.932 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.925+0000 7f9ac8f0c700 1 --2- 172.21.15.17:0/375885930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ac40b36e0 0x7f9ac4062510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:53.932 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.925+0000 7f9ac8f0c700 1 --2- 172.21.15.17:0/375885930 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ac4064b10 0x7f9ac41425a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:53.933 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.925+0000 7f9ac8f0c700 1 -- 172.21.15.17:0/375885930 >> 172.21.15.17:0/375885930 conn(0x7f9ac405f840 msgr2=0x7f9ac40b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:53.933 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.925+0000 7f9ac8f0c700 1 -- 172.21.15.17:0/375885930 shutdown_connections 2022-08-01T19:39:53.934 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:53.925+0000 7f9ac8f0c700 1 -- 172.21.15.17:0/375885930 wait complete. 2022-08-01T19:39:53.945 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T19:39:54.065 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.061+0000 7f1418d23700 1 Processor -- start 2022-08-01T19:39:54.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.061+0000 7f1418d23700 1 -- start start 2022-08-01T19:39:54.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.061+0000 7f1418d23700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f14140b2730 0x7f14140666b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:54.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.061+0000 7f1418d23700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1414066ba0 0x7f1414066fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:54.069 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.061+0000 7f1418d23700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14140654c0 con 0x7f14140b3190 2022-08-01T19:39:54.069 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.061+0000 7f1418d23700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1414062190 con 0x7f14140b2730 2022-08-01T19:39:54.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.061+0000 7f1418d23700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14140b3570 con 0x7f1414066ba0 2022-08-01T19:39:54.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.061+0000 7f1412ffd700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f14140b2730 0x7f14140666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:54.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.061+0000 7f1413fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1414066ba0 0x7f1414066fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:54.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.061+0000 7f1412ffd700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f14140b2730 0x7f14140666b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:38498/0 (socket says 172.21.15.17:38498) 2022-08-01T19:39:54.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.061+0000 7f1412ffd700 1 -- 172.21.15.17:0/238797563 learned_addr learned my addr 172.21.15.17:0/238797563 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:54.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.065+0000 7f1412ffd700 1 -- 172.21.15.17:0/238797563 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1414066ba0 msgr2=0x7f1414066fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:54.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.065+0000 7f1412ffd700 1 --2- 172.21.15.17:0/238797563 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1414066ba0 0x7f1414066fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:54.076 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.065+0000 7f1412ffd700 1 -- 172.21.15.17:0/238797563 >> v1:172.21.15.77:6789/0 conn(0x7f14140b3190 legacy=0x7f1414060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:54.076 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.065+0000 7f1412ffd700 1 -- 172.21.15.17:0/238797563 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141405fbf0 con 0x7f14140b2730 2022-08-01T19:39:54.077 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.065+0000 7f1412ffd700 1 --2- 172.21.15.17:0/238797563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14140b2730 0x7f14140666b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f13fc002f70 tx=0x7f13fc006eb0).ready entity=mon.1 client_cookie=49f507adc3c394d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:54.077 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.065+0000 7f14127fc700 1 -- 172.21.15.17:0/238797563 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13fc00d570 con 0x7f14140b2730 2022-08-01T19:39:54.078 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.065+0000 7f14127fc700 1 -- 172.21.15.17:0/238797563 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13fc000e90 con 0x7f14140b2730 2022-08-01T19:39:54.078 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.065+0000 7f14127fc700 1 -- 172.21.15.17:0/238797563 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13fc00d900 con 0x7f14140b2730 2022-08-01T19:39:54.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.065+0000 7f1418d23700 1 -- 172.21.15.17:0/238797563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14140b2730 msgr2=0x7f14140666b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:54.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.065+0000 7f1418d23700 1 --2- 172.21.15.17:0/238797563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14140b2730 0x7f14140666b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f13fc002f70 tx=0x7f13fc006eb0).stop 2022-08-01T19:39:54.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.065+0000 7f1418d23700 1 -- 172.21.15.17:0/238797563 shutdown_connections 2022-08-01T19:39:54.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.065+0000 7f1418d23700 1 --2- 172.21.15.17:0/238797563 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1414066ba0 0x7f1414066fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:54.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.065+0000 7f1418d23700 1 --2- 172.21.15.17:0/238797563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14140b2730 0x7f14140666b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:54.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.065+0000 7f1418d23700 1 -- 172.21.15.17:0/238797563 >> 172.21.15.17:0/238797563 conn(0x7f1414063870 msgr2=0x7f1414063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:54.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.065+0000 7f1418d23700 1 -- 172.21.15.17:0/238797563 shutdown_connections 2022-08-01T19:39:54.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.065+0000 7f1418d23700 1 -- 172.21.15.17:0/238797563 wait complete. 2022-08-01T19:39:54.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.065+0000 7f1418d23700 1 Processor -- start 2022-08-01T19:39:54.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.065+0000 7f1418d23700 1 -- start start 2022-08-01T19:39:54.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.069+0000 7f1418d23700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1414066ba0 0x7f1414145870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:54.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.069+0000 7f1418d23700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14140b2730 0x7f14140aadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:54.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.069+0000 7f1418d23700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14140b3190 0x7f14140ab2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:54.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.069+0000 7f1418d23700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14140ab7d0 con 0x7f14140b2730 2022-08-01T19:39:54.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.069+0000 7f1418d23700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14140b6a10 con 0x7f14140b3190 2022-08-01T19:39:54.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.069+0000 7f1418d23700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14140b6cb0 con 0x7f1414066ba0 2022-08-01T19:39:54.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.069+0000 7f1412ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14140b2730 0x7f14140aadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:54.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.069+0000 7f1412ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14140b2730 0x7f14140aadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40584/0 (socket says 172.21.15.17:40584) 2022-08-01T19:39:54.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.069+0000 7f1412ffd700 1 -- 172.21.15.17:0/2139692176 learned_addr learned my addr 172.21.15.17:0/2139692176 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:54.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.069+0000 7f1413fff700 1 --2- 172.21.15.17:0/2139692176 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14140b3190 0x7f14140ab2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:54.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.069+0000 7f14137fe700 1 --2- 172.21.15.17:0/2139692176 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1414066ba0 0x7f1414145870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:54.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.069+0000 7f1412ffd700 1 -- 172.21.15.17:0/2139692176 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1414066ba0 msgr2=0x7f1414145870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:54.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.069+0000 7f1412ffd700 1 --2- 172.21.15.17:0/2139692176 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1414066ba0 0x7f1414145870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:54.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.069+0000 7f1412ffd700 1 -- 172.21.15.17:0/2139692176 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14140b3190 msgr2=0x7f14140ab2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:54.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.069+0000 7f1412ffd700 1 --2- 172.21.15.17:0/2139692176 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14140b3190 0x7f14140ab2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:54.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.069+0000 7f1412ffd700 1 -- 172.21.15.17:0/2139692176 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13fc00b1e0 con 0x7f14140b2730 2022-08-01T19:39:54.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.069+0000 7f1412ffd700 1 --2- 172.21.15.17:0/2139692176 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14140b2730 0x7f14140aadf0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7f13fc002f70 tx=0x7f13fc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:54.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.069+0000 7f1410ff9700 1 -- 172.21.15.17:0/2139692176 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13fc0068b0 con 0x7f14140b2730 2022-08-01T19:39:54.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.069+0000 7f1410ff9700 1 -- 172.21.15.17:0/2139692176 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13fc011b80 con 0x7f14140b2730 2022-08-01T19:39:54.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.069+0000 7f1410ff9700 1 -- 172.21.15.17:0/2139692176 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13fc01e460 con 0x7f14140b2730 2022-08-01T19:39:54.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.069+0000 7f1418d23700 1 -- 172.21.15.17:0/2139692176 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14140b6f50 con 0x7f14140b2730 2022-08-01T19:39:54.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.069+0000 7f1418d23700 1 -- 172.21.15.17:0/2139692176 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14140ad6d0 con 0x7f14140b2730 2022-08-01T19:39:54.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.073+0000 7f1410ff9700 1 -- 172.21.15.17:0/2139692176 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f13fc01a040 con 0x7f14140b2730 2022-08-01T19:39:54.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.073+0000 7f1418d23700 1 -- 172.21.15.17:0/2139692176 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f141405fe90 con 0x7f14140b2730 2022-08-01T19:39:54.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.073+0000 7f1410ff9700 1 --2- 172.21.15.17:0/2139692176 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f13f8053950 0x7f13f8055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:54.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.073+0000 7f1410ff9700 1 -- 172.21.15.17:0/2139692176 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f13fc06cfb0 con 0x7f14140b2730 2022-08-01T19:39:54.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.073+0000 7f14137fe700 1 --2- 172.21.15.17:0/2139692176 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f13f8053950 0x7f13f8055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:54.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.073+0000 7f14137fe700 1 --2- 172.21.15.17:0/2139692176 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f13f8053950 0x7f13f8055e10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f140400d000 tx=0x7f1404007b90).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:54.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.081+0000 7f1410ff9700 1 -- 172.21.15.17:0/2139692176 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f13fc043d10 con 0x7f14140b2730 2022-08-01T19:39:54.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.245+0000 7f1418d23700 1 -- 172.21.15.17:0/2139692176 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f14140574e0 con 0x7f14140b2730 2022-08-01T19:39:54.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.249+0000 7f1410ff9700 1 -- 172.21.15.17:0/2139692176 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v23) v1 ==== 184+0+0 (secure 0 0 0) 0x7f13fc023070 con 0x7f14140b2730 2022-08-01T19:39:54.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.253+0000 7f1418d23700 1 -- 172.21.15.17:0/2139692176 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f13f8053950 msgr2=0x7f13f8055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:54.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.253+0000 7f1418d23700 1 --2- 172.21.15.17:0/2139692176 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f13f8053950 0x7f13f8055e10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f140400d000 tx=0x7f1404007b90).stop 2022-08-01T19:39:54.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.253+0000 7f1418d23700 1 -- 172.21.15.17:0/2139692176 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14140b2730 msgr2=0x7f14140aadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:54.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.253+0000 7f1418d23700 1 --2- 172.21.15.17:0/2139692176 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14140b2730 0x7f14140aadf0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7f13fc002f70 tx=0x7f13fc015000).stop 2022-08-01T19:39:54.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.253+0000 7f1418d23700 1 -- 172.21.15.17:0/2139692176 shutdown_connections 2022-08-01T19:39:54.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.253+0000 7f1418d23700 1 --2- 172.21.15.17:0/2139692176 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f13f8053950 0x7f13f8055e10 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:54.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.253+0000 7f1418d23700 1 --2- 172.21.15.17:0/2139692176 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14140b3190 0x7f14140ab2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:54.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.253+0000 7f1418d23700 1 --2- 172.21.15.17:0/2139692176 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14140b2730 0x7f14140aadf0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:54.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.253+0000 7f1418d23700 1 --2- 172.21.15.17:0/2139692176 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1414066ba0 0x7f1414145870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:54.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.253+0000 7f1418d23700 1 -- 172.21.15.17:0/2139692176 >> 172.21.15.17:0/2139692176 conn(0x7f1414063870 msgr2=0x7f1414142fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:54.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.253+0000 7f1418d23700 1 -- 172.21.15.17:0/2139692176 shutdown_connections 2022-08-01T19:39:54.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.253+0000 7f1418d23700 1 -- 172.21.15.17:0/2139692176 wait complete. 2022-08-01T19:39:54.258 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.0 2022-08-01T19:39:54.268 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T19:39:54.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.393+0000 7f7600b0c700 1 Processor -- start 2022-08-01T19:39:54.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.393+0000 7f7600b0c700 1 -- start start 2022-08-01T19:39:54.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.393+0000 7f7600b0c700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f75fc0b3630 0x7f75fc143f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:54.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.393+0000 7f7600b0c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f75fc0aa0d0 0x7f75fc0af200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:54.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.393+0000 7f7600b0c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f75fc0af6f0 0x7f75fc05f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:54.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.393+0000 7f7600b0c700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75fc1443f0 con 0x7f75fc0af6f0 2022-08-01T19:39:54.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.393+0000 7f7600b0c700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75fc0b3a10 con 0x7f75fc0b3630 2022-08-01T19:39:54.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.393+0000 7f7600b0c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc0afb80 con 0x7f75fc0aa0d0 2022-08-01T19:39:54.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.393+0000 7f75fad9d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f75fc0af6f0 0x7f75fc05f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:54.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.393+0000 7f75fa59c700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f75fc0b3630 0x7f75fc143f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:54.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.393+0000 7f75fad9d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f75fc0af6f0 0x7f75fc05f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40590/0 (socket says 172.21.15.17:40590) 2022-08-01T19:39:54.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.393+0000 7f75fad9d700 1 -- 172.21.15.17:0/1759936805 learned_addr learned my addr 172.21.15.17:0/1759936805 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:54.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.393+0000 7f75f9d9b700 1 --2- 172.21.15.17:0/1759936805 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f75fc0aa0d0 0x7f75fc0af200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:54.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.397+0000 7f75fad9d700 1 -- 172.21.15.17:0/1759936805 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f75fc0aa0d0 msgr2=0x7f75fc0af200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:54.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.397+0000 7f75fad9d700 1 --2- 172.21.15.17:0/1759936805 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f75fc0aa0d0 0x7f75fc0af200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:54.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.397+0000 7f75fad9d700 1 -- 172.21.15.17:0/1759936805 >> v2:172.21.15.77:3300/0 conn(0x7f75fc0b3630 msgr2=0x7f75fc143f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:54.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.397+0000 7f75fad9d700 1 --2- 172.21.15.17:0/1759936805 >> v2:172.21.15.77:3300/0 conn(0x7f75fc0b3630 0x7f75fc143f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:54.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.397+0000 7f75fad9d700 1 -- 172.21.15.17:0/1759936805 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75fc146b40 con 0x7f75fc0af6f0 2022-08-01T19:39:54.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.397+0000 7f75fa59c700 1 --2- 172.21.15.17:0/1759936805 >> v2:172.21.15.77:3300/0 conn(0x7f75fc0b3630 0x7f75fc143f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:39:54.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.397+0000 7f75fad9d700 1 --2- 172.21.15.17:0/1759936805 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75fc0af6f0 0x7f75fc05f840 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7f75f4002f70 tx=0x7f75f4006eb0).ready entity=mon.0 client_cookie=1475bb13a8c364c6 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:54.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.397+0000 7f75f959a700 1 -- 172.21.15.17:0/1759936805 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75f400d570 con 0x7f75fc0af6f0 2022-08-01T19:39:54.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.397+0000 7f75f959a700 1 -- 172.21.15.17:0/1759936805 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f4000e90 con 0x7f75fc0af6f0 2022-08-01T19:39:54.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.397+0000 7f75f959a700 1 -- 172.21.15.17:0/1759936805 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75f400d900 con 0x7f75fc0af6f0 2022-08-01T19:39:54.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.397+0000 7f7600b0c700 1 -- 172.21.15.17:0/1759936805 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75fc0af6f0 msgr2=0x7f75fc05f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:54.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.397+0000 7f7600b0c700 1 --2- 172.21.15.17:0/1759936805 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75fc0af6f0 0x7f75fc05f840 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7f75f4002f70 tx=0x7f75f4006eb0).stop 2022-08-01T19:39:54.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.397+0000 7f7600b0c700 1 -- 172.21.15.17:0/1759936805 shutdown_connections 2022-08-01T19:39:54.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.397+0000 7f7600b0c700 1 --2- 172.21.15.17:0/1759936805 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75fc0af6f0 0x7f75fc05f840 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:54.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.397+0000 7f7600b0c700 1 --2- 172.21.15.17:0/1759936805 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f75fc0aa0d0 0x7f75fc0af200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:54.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.397+0000 7f7600b0c700 1 --2- 172.21.15.17:0/1759936805 >> v2:172.21.15.77:3300/0 conn(0x7f75fc0b3630 0x7f75fc143f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:54.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.397+0000 7f7600b0c700 1 -- 172.21.15.17:0/1759936805 >> 172.21.15.17:0/1759936805 conn(0x7f75fc060950 msgr2=0x7f75fc060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:54.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.397+0000 7f7600b0c700 1 -- 172.21.15.17:0/1759936805 shutdown_connections 2022-08-01T19:39:54.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.397+0000 7f7600b0c700 1 -- 172.21.15.17:0/1759936805 wait complete. 2022-08-01T19:39:54.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.401+0000 7f7600b0c700 1 Processor -- start 2022-08-01T19:39:54.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.401+0000 7f7600b0c700 1 -- start start 2022-08-01T19:39:54.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.401+0000 7f7600b0c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f75fc0aa0d0 0x7f75fc1464e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:54.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.401+0000 7f7600b0c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75fc0af6f0 0x7f75fc155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:54.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.401+0000 7f7600b0c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f75fc0b3630 0x7f75fc0b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:54.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.401+0000 7f7600b0c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc156090 con 0x7f75fc0af6f0 2022-08-01T19:39:54.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.401+0000 7f7600b0c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc13d440 con 0x7f75fc0b3630 2022-08-01T19:39:54.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.401+0000 7f7600b0c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc13d6e0 con 0x7f75fc0aa0d0 2022-08-01T19:39:54.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.401+0000 7f75f9d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75fc0af6f0 0x7f75fc155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:54.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.401+0000 7f75f9d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75fc0af6f0 0x7f75fc155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40596/0 (socket says 172.21.15.17:40596) 2022-08-01T19:39:54.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.401+0000 7f75f9d9b700 1 -- 172.21.15.17:0/1912288900 learned_addr learned my addr 172.21.15.17:0/1912288900 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:54.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.401+0000 7f75fa59c700 1 --2- 172.21.15.17:0/1912288900 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f75fc0aa0d0 0x7f75fc1464e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:54.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.401+0000 7f75fad9d700 1 --2- 172.21.15.17:0/1912288900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f75fc0b3630 0x7f75fc0b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:54.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.401+0000 7f75f9d9b700 1 -- 172.21.15.17:0/1912288900 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f75fc0aa0d0 msgr2=0x7f75fc1464e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:54.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.401+0000 7f75f9d9b700 1 --2- 172.21.15.17:0/1912288900 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f75fc0aa0d0 0x7f75fc1464e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:54.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.401+0000 7f75f9d9b700 1 -- 172.21.15.17:0/1912288900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f75fc0b3630 msgr2=0x7f75fc0b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:54.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.401+0000 7f75f9d9b700 1 --2- 172.21.15.17:0/1912288900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f75fc0b3630 0x7f75fc0b51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:54.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.401+0000 7f75f9d9b700 1 -- 172.21.15.17:0/1912288900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ec009470 con 0x7f75fc0af6f0 2022-08-01T19:39:54.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.401+0000 7f75f9d9b700 1 --2- 172.21.15.17:0/1912288900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75fc0af6f0 0x7f75fc155ba0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7f75f000c000 tx=0x7f75f0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:54.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.401+0000 7f75eb7fe700 1 -- 172.21.15.17:0/1912288900 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75f0011750 con 0x7f75fc0af6f0 2022-08-01T19:39:54.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.401+0000 7f75eb7fe700 1 -- 172.21.15.17:0/1912288900 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f00118b0 con 0x7f75fc0af6f0 2022-08-01T19:39:54.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.401+0000 7f7600b0c700 1 -- 172.21.15.17:0/1912288900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75f400b1e0 con 0x7f75fc0af6f0 2022-08-01T19:39:54.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.401+0000 7f75eb7fe700 1 -- 172.21.15.17:0/1912288900 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75f0011c10 con 0x7f75fc0af6f0 2022-08-01T19:39:54.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.401+0000 7f7600b0c700 1 -- 172.21.15.17:0/1912288900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75fc0b4d40 con 0x7f75fc0af6f0 2022-08-01T19:39:54.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.405+0000 7f7600b0c700 1 -- 172.21.15.17:0/1912288900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75fc146de0 con 0x7f75fc0af6f0 2022-08-01T19:39:54.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.405+0000 7f75eb7fe700 1 -- 172.21.15.17:0/1912288900 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f75f0011d70 con 0x7f75fc0af6f0 2022-08-01T19:39:54.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.405+0000 7f75eb7fe700 1 --2- 172.21.15.17:0/1912288900 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f75d80539e0 0x7f75d8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:54.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.405+0000 7f75eb7fe700 1 -- 172.21.15.17:0/1912288900 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f75f006c390 con 0x7f75fc0af6f0 2022-08-01T19:39:54.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.405+0000 7f75fa59c700 1 --2- 172.21.15.17:0/1912288900 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f75d80539e0 0x7f75d8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:54.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.405+0000 7f75fa59c700 1 --2- 172.21.15.17:0/1912288900 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f75d80539e0 0x7f75d8055ea0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f75ec00b000 tx=0x7f75ec0091d0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:54.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.413+0000 7f75eb7fe700 1 -- 172.21.15.17:0/1912288900 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f75f0043150 con 0x7f75fc0af6f0 2022-08-01T19:39:54.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.577+0000 7f7600b0c700 1 -- 172.21.15.17:0/1912288900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f75fc0574e0 con 0x7f75fc0af6f0 2022-08-01T19:39:54.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.585+0000 7f75eb7fe700 1 -- 172.21.15.17:0/1912288900 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v24) v1 ==== 184+0+0 (secure 0 0 0) 0x7f75f0046770 con 0x7f75fc0af6f0 2022-08-01T19:39:54.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.589+0000 7f7600b0c700 1 -- 172.21.15.17:0/1912288900 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f75d80539e0 msgr2=0x7f75d8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:54.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.589+0000 7f7600b0c700 1 --2- 172.21.15.17:0/1912288900 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f75d80539e0 0x7f75d8055ea0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f75ec00b000 tx=0x7f75ec0091d0).stop 2022-08-01T19:39:54.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.589+0000 7f7600b0c700 1 -- 172.21.15.17:0/1912288900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75fc0af6f0 msgr2=0x7f75fc155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:54.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.589+0000 7f7600b0c700 1 --2- 172.21.15.17:0/1912288900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75fc0af6f0 0x7f75fc155ba0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7f75f000c000 tx=0x7f75f0006df0).stop 2022-08-01T19:39:54.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.589+0000 7f7600b0c700 1 -- 172.21.15.17:0/1912288900 shutdown_connections 2022-08-01T19:39:54.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.589+0000 7f7600b0c700 1 --2- 172.21.15.17:0/1912288900 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f75d80539e0 0x7f75d8055ea0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:54.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.589+0000 7f7600b0c700 1 --2- 172.21.15.17:0/1912288900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f75fc0b3630 0x7f75fc0b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:54.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.589+0000 7f7600b0c700 1 --2- 172.21.15.17:0/1912288900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75fc0af6f0 0x7f75fc155ba0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:54.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.589+0000 7f7600b0c700 1 --2- 172.21.15.17:0/1912288900 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f75fc0aa0d0 0x7f75fc1464e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:54.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.589+0000 7f7600b0c700 1 -- 172.21.15.17:0/1912288900 >> 172.21.15.17:0/1912288900 conn(0x7f75fc060950 msgr2=0x7f75fc0b06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:54.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.589+0000 7f7600b0c700 1 -- 172.21.15.17:0/1912288900 shutdown_connections 2022-08-01T19:39:54.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.589+0000 7f7600b0c700 1 -- 172.21.15.17:0/1912288900 wait complete. 2022-08-01T19:39:54.595 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.1 2022-08-01T19:39:54.606 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:39:54.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.713+0000 7ff9b0bf0700 1 Processor -- start 2022-08-01T19:39:54.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.713+0000 7ff9b0bf0700 1 -- start start 2022-08-01T19:39:54.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.713+0000 7ff9b0bf0700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff9ac148530 0x7ff9ac0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:54.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.713+0000 7ff9b0bf0700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9ac0b04e0 con 0x7ff9ac0b1f40 2022-08-01T19:39:54.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.713+0000 7ff9b0bf0700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9ac0b0780 con 0x7ff9ac0a7250 2022-08-01T19:39:54.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.713+0000 7ff9b0bf0700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac0a9250 con 0x7ff9ac148530 2022-08-01T19:39:54.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.713+0000 7ff9aa59c700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff9ac0b1f40 0x7ff9ac0b2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:32796/0 (socket says 172.21.15.17:32796) 2022-08-01T19:39:54.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.713+0000 7ff9aa59c700 1 -- 172.21.15.17:0/2390722386 learned_addr learned my addr 172.21.15.17:0/2390722386 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:54.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.713+0000 7ff9a1d9b700 1 --2- 172.21.15.17:0/2390722386 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff9ac148530 0x7ff9ac0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:54.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.713+0000 7ff9a9d9b700 1 -- 172.21.15.17:0/2390722386 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3451975741 0 0) 0x7ff9ac0b04e0 con 0x7ff9ac0b1f40 2022-08-01T19:39:54.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.713+0000 7ff9a9d9b700 1 -- 172.21.15.17:0/2390722386 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff990003260 con 0x7ff9ac0b1f40 2022-08-01T19:39:54.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.713+0000 7ff9a1d9b700 1 -- 172.21.15.17:0/2390722386 >> v1:172.21.15.77:6789/0 conn(0x7ff9ac0a7250 legacy=0x7ff9ac1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:54.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.713+0000 7ff9a1d9b700 1 -- 172.21.15.17:0/2390722386 >> v1:172.21.15.17:6789/0 conn(0x7ff9ac0b1f40 legacy=0x7ff9ac0b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:54.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.713+0000 7ff9a1d9b700 1 -- 172.21.15.17:0/2390722386 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ac0b0a20 con 0x7ff9ac148530 2022-08-01T19:39:54.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.713+0000 7ff9a1d9b700 1 --2- 172.21.15.17:0/2390722386 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff9ac148530 0x7ff9ac0b1160 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7ff99c002f70 tx=0x7ff99c006df0).ready entity=mon.2 client_cookie=9dadff7e9e0c44ff server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:54.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.713+0000 7ff9a9d9b700 1 -- 172.21.15.17:0/2390722386 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff99c00d570 con 0x7ff9ac148530 2022-08-01T19:39:54.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.713+0000 7ff9a9d9b700 1 -- 172.21.15.17:0/2390722386 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99c000e90 con 0x7ff9ac148530 2022-08-01T19:39:54.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.713+0000 7ff9a9d9b700 1 -- 172.21.15.17:0/2390722386 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff99c00d8c0 con 0x7ff9ac148530 2022-08-01T19:39:54.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.713+0000 7ff9b0bf0700 1 -- 172.21.15.17:0/2390722386 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff9ac148530 msgr2=0x7ff9ac0b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:54.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.713+0000 7ff9b0bf0700 1 --2- 172.21.15.17:0/2390722386 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff9ac148530 0x7ff9ac0b1160 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7ff99c002f70 tx=0x7ff99c006df0).stop 2022-08-01T19:39:54.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.713+0000 7ff9b0bf0700 1 -- 172.21.15.17:0/2390722386 shutdown_connections 2022-08-01T19:39:54.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.717+0000 7ff9b0bf0700 1 --2- 172.21.15.17:0/2390722386 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff9ac148530 0x7ff9ac0b1160 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:54.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.717+0000 7ff9b0bf0700 1 -- 172.21.15.17:0/2390722386 >> 172.21.15.17:0/2390722386 conn(0x7ff9ac05f840 msgr2=0x7ff9ac05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:54.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.717+0000 7ff9b0bf0700 1 -- 172.21.15.17:0/2390722386 shutdown_connections 2022-08-01T19:39:54.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.717+0000 7ff9b0bf0700 1 -- 172.21.15.17:0/2390722386 wait complete. 2022-08-01T19:39:54.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.717+0000 7ff9b0bf0700 1 Processor -- start 2022-08-01T19:39:54.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.717+0000 7ff9b0bf0700 1 -- start start 2022-08-01T19:39:54.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.717+0000 7ff9b0bf0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9ac0a7250 0x7ff9ac0af480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:54.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.717+0000 7ff9b0bf0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9ac0b1f40 0x7ff9ac0ac230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:54.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.717+0000 7ff9b0bf0700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff9ac148530 0x7ff9ac0a9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:54.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.717+0000 7ff9b0bf0700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac0a95d0 con 0x7ff9ac0b1f40 2022-08-01T19:39:54.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.717+0000 7ff9b0bf0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac0aa1c0 con 0x7ff9ac0a7250 2022-08-01T19:39:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.717+0000 7ff9b0bf0700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac0ac880 con 0x7ff9ac148530 2022-08-01T19:39:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.717+0000 7ff9a1d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9ac0b1f40 0x7ff9ac0ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.717+0000 7ff9aa59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9ac0a7250 0x7ff9ac0af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.717+0000 7ff9a1d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9ac0b1f40 0x7ff9ac0ac230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40610/0 (socket says 172.21.15.17:40610) 2022-08-01T19:39:54.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.721+0000 7ff9a1d9b700 1 -- 172.21.15.17:0/2617302955 learned_addr learned my addr 172.21.15.17:0/2617302955 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:54.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.721+0000 7ff9aad9d700 1 --2- 172.21.15.17:0/2617302955 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff9ac148530 0x7ff9ac0a9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:54.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.721+0000 7ff9a1d9b700 1 -- 172.21.15.17:0/2617302955 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff9ac148530 msgr2=0x7ff9ac0a9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:54.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.721+0000 7ff9a1d9b700 1 --2- 172.21.15.17:0/2617302955 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff9ac148530 0x7ff9ac0a9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:54.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.721+0000 7ff9a1d9b700 1 -- 172.21.15.17:0/2617302955 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9ac0a7250 msgr2=0x7ff9ac0af480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:54.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.721+0000 7ff9a1d9b700 1 --2- 172.21.15.17:0/2617302955 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9ac0a7250 0x7ff9ac0af480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:54.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.721+0000 7ff9a1d9b700 1 -- 172.21.15.17:0/2617302955 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99c00b180 con 0x7ff9ac0b1f40 2022-08-01T19:39:54.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.721+0000 7ff9aa59c700 1 --2- 172.21.15.17:0/2617302955 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9ac0a7250 0x7ff9ac0af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:39:54.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.721+0000 7ff9a1d9b700 1 --2- 172.21.15.17:0/2617302955 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9ac0b1f40 0x7ff9ac0ac230 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7ff9ac146520 tx=0x7ff99c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:54.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.721+0000 7ff9a3fff700 1 -- 172.21.15.17:0/2617302955 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff99c016070 con 0x7ff9ac0b1f40 2022-08-01T19:39:54.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.721+0000 7ff9a3fff700 1 -- 172.21.15.17:0/2617302955 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99c012670 con 0x7ff9ac0b1f40 2022-08-01T19:39:54.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.721+0000 7ff9b0bf0700 1 -- 172.21.15.17:0/2617302955 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9ac0acb20 con 0x7ff9ac0b1f40 2022-08-01T19:39:54.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.721+0000 7ff9a3fff700 1 -- 172.21.15.17:0/2617302955 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff99c0129d0 con 0x7ff9ac0b1f40 2022-08-01T19:39:54.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.721+0000 7ff9b0bf0700 1 -- 172.21.15.17:0/2617302955 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9ac0abf20 con 0x7ff9ac0b1f40 2022-08-01T19:39:54.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.721+0000 7ff9b0bf0700 1 -- 172.21.15.17:0/2617302955 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9ac0b0cc0 con 0x7ff9ac0b1f40 2022-08-01T19:39:54.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.725+0000 7ff9a3fff700 1 -- 172.21.15.17:0/2617302955 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff99c024030 con 0x7ff9ac0b1f40 2022-08-01T19:39:54.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.725+0000 7ff9a3fff700 1 --2- 172.21.15.17:0/2617302955 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff9840539e0 0x7ff984055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:54.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.725+0000 7ff9a3fff700 1 -- 172.21.15.17:0/2617302955 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff99c06c890 con 0x7ff9ac0b1f40 2022-08-01T19:39:54.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.725+0000 7ff9aa59c700 1 --2- 172.21.15.17:0/2617302955 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff9840539e0 0x7ff984055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:54.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.725+0000 7ff9aa59c700 1 --2- 172.21.15.17:0/2617302955 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff9840539e0 0x7ff984055ea0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7ff99400d000 tx=0x7ff994009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:54.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.729+0000 7ff9a3fff700 1 -- 172.21.15.17:0/2617302955 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff99c043650 con 0x7ff9ac0b1f40 2022-08-01T19:39:54.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.889+0000 7ff9b0bf0700 1 -- 172.21.15.17:0/2617302955 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9ac0574e0 con 0x7ff9ac0b1f40 2022-08-01T19:39:54.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.889+0000 7ff9a3fff700 1 -- 172.21.15.17:0/2617302955 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+2493 (secure 0 0 0) 0x7ff99c022090 con 0x7ff9ac0b1f40 2022-08-01T19:39:54.892 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:39:54.893 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":35,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5615,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/2284778179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2284778179},{"type":"v1","addr":"172.21.15.77:6835","nonce":2284778179}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":34},{"gid":6261,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/1827111044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":1827111044},{"type":"v1","addr":"172.21.15.17:6833","nonce":1827111044}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":33}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:39:52.325506+0000","modified":"2022-08-01T19:39:53.328610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6267":{"gid":6267,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/3370152344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":3370152344},{"type":"v1","addr":"172.21.15.17:6835","nonce":3370152344}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-08-01T19:39:54.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.893+0000 7ff9b0bf0700 1 -- 172.21.15.17:0/2617302955 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff9840539e0 msgr2=0x7ff984055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:54.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.893+0000 7ff9b0bf0700 1 --2- 172.21.15.17:0/2617302955 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff9840539e0 0x7ff984055ea0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7ff99400d000 tx=0x7ff994009040).stop 2022-08-01T19:39:54.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.893+0000 7ff9b0bf0700 1 -- 172.21.15.17:0/2617302955 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9ac0b1f40 msgr2=0x7ff9ac0ac230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:54.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.893+0000 7ff9b0bf0700 1 --2- 172.21.15.17:0/2617302955 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9ac0b1f40 0x7ff9ac0ac230 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7ff9ac146520 tx=0x7ff99c014000).stop 2022-08-01T19:39:54.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.893+0000 7ff9b0bf0700 1 -- 172.21.15.17:0/2617302955 shutdown_connections 2022-08-01T19:39:54.899 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.893+0000 7ff9b0bf0700 1 --2- 172.21.15.17:0/2617302955 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff9840539e0 0x7ff984055ea0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:54.899 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.893+0000 7ff9b0bf0700 1 --2- 172.21.15.17:0/2617302955 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff9ac148530 0x7ff9ac0a9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:54.899 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.893+0000 7ff9b0bf0700 1 --2- 172.21.15.17:0/2617302955 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9ac0b1f40 0x7ff9ac0ac230 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:54.899 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.893+0000 7ff9b0bf0700 1 --2- 172.21.15.17:0/2617302955 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9ac0a7250 0x7ff9ac0af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:54.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.893+0000 7ff9b0bf0700 1 -- 172.21.15.17:0/2617302955 >> 172.21.15.17:0/2617302955 conn(0x7ff9ac05f840 msgr2=0x7ff9ac0b3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:54.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.893+0000 7ff9b0bf0700 1 -- 172.21.15.17:0/2617302955 shutdown_connections 2022-08-01T19:39:54.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:54.893+0000 7ff9b0bf0700 1 -- 172.21.15.17:0/2617302955 wait complete. 2022-08-01T19:39:54.901 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 35 2022-08-01T19:39:55.209 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:39:52.325506+0000', 'modified': '2022-08-01T19:39:53.328610+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6267}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6267': {'gid': 6267, 'name': 'a', 'rank': 0, 'incarnation': 34, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/3370152344', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 3370152344}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 3370152344}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:39:55.210 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:39:55.211 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-08-01T19:39:55.329 INFO:teuthology.orchestra.run.smithi017.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":6267,"want_state":"up:active","state":"up:active","rank_uptime":2.9933395389999999,"mdsmap_epoch":35,"osdmap_epoch":76,"osdmap_epoch_barrier":75,"uptime":7.3100858750000004} 2022-08-01T19:39:55.330 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 6267, "mdsmap_epoch": 35, "osdmap_epoch": 76, "osdmap_epoch_barrier": 75, "rank_uptime": 2.993339539, "state": "up:active", "uptime": 7.310085875, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:39:55.331 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:39:55.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.433+0000 7f5a022de700 1 Processor -- start 2022-08-01T19:39:55.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.433+0000 7f5a022de700 1 -- start start 2022-08-01T19:39:55.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.437+0000 7f5a022de700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f59fc0b7500 0x7f59fc0b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:55.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.437+0000 7f5a022de700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f59fc0aadb0 0x7f59fc064d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:55.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.437+0000 7f5a022de700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59fc0b2510 con 0x7f59fc0b31f0 2022-08-01T19:39:55.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.437+0000 7f5a022de700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59fc1443e0 con 0x7f59fc0b7500 2022-08-01T19:39:55.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.437+0000 7f5a022de700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc0b3700 con 0x7f59fc0aadb0 2022-08-01T19:39:55.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.437+0000 7f5a012dc700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f59fc0b7500 0x7f59fc0b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:55.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.437+0000 7f5a00adb700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7f59fc0b31f0 0x7f59fc13e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:49610/0 (socket says 172.21.15.17:49610) 2022-08-01T19:39:55.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.437+0000 7f5a00adb700 1 -- 172.21.15.17:0/1467999404 learned_addr learned my addr 172.21.15.17:0/1467999404 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:55.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.437+0000 7f5a01add700 1 --2- 172.21.15.17:0/1467999404 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f59fc0aadb0 0x7f59fc064d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:55.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.437+0000 7f5a012dc700 1 -- 172.21.15.17:0/1467999404 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f59fc0aadb0 msgr2=0x7f59fc064d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:55.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.437+0000 7f5a012dc700 1 --2- 172.21.15.17:0/1467999404 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f59fc0aadb0 0x7f59fc064d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:55.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.437+0000 7f5a012dc700 1 -- 172.21.15.17:0/1467999404 >> v1:172.21.15.77:6789/0 conn(0x7f59fc0b31f0 legacy=0x7f59fc13e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:55.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.437+0000 7f59f3fff700 1 -- 172.21.15.17:0/1467999404 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 424082179 0 0) 0x7f59fc0b2510 con 0x7f59fc0b31f0 2022-08-01T19:39:55.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.437+0000 7f5a012dc700 1 -- 172.21.15.17:0/1467999404 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59fc0b2740 con 0x7f59fc0b7500 2022-08-01T19:39:55.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.437+0000 7f5a012dc700 1 --2- 172.21.15.17:0/1467999404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59fc0b7500 0x7f59fc0b78e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7f59ec002f70 tx=0x7f59ec006d90).ready entity=mon.0 client_cookie=480ec54c633d432 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:55.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.437+0000 7f59f3fff700 1 -- 172.21.15.17:0/1467999404 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59ec00d4a0 con 0x7f59fc0b7500 2022-08-01T19:39:55.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.437+0000 7f59f3fff700 1 -- 172.21.15.17:0/1467999404 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59ec000e90 con 0x7f59fc0b7500 2022-08-01T19:39:55.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.437+0000 7f59f3fff700 1 -- 172.21.15.17:0/1467999404 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59ec00d7f0 con 0x7f59fc0b7500 2022-08-01T19:39:55.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.437+0000 7f5a022de700 1 -- 172.21.15.17:0/1467999404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59fc0b7500 msgr2=0x7f59fc0b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:55.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.437+0000 7f5a022de700 1 --2- 172.21.15.17:0/1467999404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59fc0b7500 0x7f59fc0b78e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7f59ec002f70 tx=0x7f59ec006d90).stop 2022-08-01T19:39:55.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.437+0000 7f5a022de700 1 -- 172.21.15.17:0/1467999404 shutdown_connections 2022-08-01T19:39:55.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.437+0000 7f5a022de700 1 --2- 172.21.15.17:0/1467999404 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f59fc0aadb0 0x7f59fc064d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:55.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.437+0000 7f5a022de700 1 --2- 172.21.15.17:0/1467999404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59fc0b7500 0x7f59fc0b78e0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:55.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.437+0000 7f5a022de700 1 -- 172.21.15.17:0/1467999404 >> 172.21.15.17:0/1467999404 conn(0x7f59fc05f840 msgr2=0x7f59fc05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:55.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.437+0000 7f5a022de700 1 -- 172.21.15.17:0/1467999404 shutdown_connections 2022-08-01T19:39:55.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.437+0000 7f5a022de700 1 -- 172.21.15.17:0/1467999404 wait complete. 2022-08-01T19:39:55.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.441+0000 7f5a022de700 1 Processor -- start 2022-08-01T19:39:55.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.441+0000 7f5a022de700 1 -- start start 2022-08-01T19:39:55.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.441+0000 7f5a022de700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59fc0aadb0 0x7f59fc147030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:55.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.441+0000 7f5a022de700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59fc0b31f0 0x7f59fc0af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:55.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.441+0000 7f5a022de700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f59fc0b7500 0x7f59fc0af6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:55.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.441+0000 7f5a022de700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc0ae7b0 con 0x7f59fc0aadb0 2022-08-01T19:39:55.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.441+0000 7f5a022de700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc0abec0 con 0x7f59fc0b31f0 2022-08-01T19:39:55.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.441+0000 7f5a022de700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc0ac0f0 con 0x7f59fc0b7500 2022-08-01T19:39:55.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.441+0000 7f5a012dc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59fc0aadb0 0x7f59fc147030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:55.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.441+0000 7f5a00adb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59fc0b31f0 0x7f59fc0af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:55.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.445+0000 7f5a00adb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59fc0b31f0 0x7f59fc0af1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:38546/0 (socket says 172.21.15.17:38546) 2022-08-01T19:39:55.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.445+0000 7f5a00adb700 1 -- 172.21.15.17:0/2851761081 learned_addr learned my addr 172.21.15.17:0/2851761081 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:55.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.445+0000 7f5a01add700 1 --2- 172.21.15.17:0/2851761081 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f59fc0b7500 0x7f59fc0af6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:55.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.445+0000 7f5a012dc700 1 -- 172.21.15.17:0/2851761081 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f59fc0b7500 msgr2=0x7f59fc0af6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:55.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.445+0000 7f5a012dc700 1 --2- 172.21.15.17:0/2851761081 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f59fc0b7500 0x7f59fc0af6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:55.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.445+0000 7f5a012dc700 1 -- 172.21.15.17:0/2851761081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59fc0b31f0 msgr2=0x7f59fc0af1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:55.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.445+0000 7f5a012dc700 1 --2- 172.21.15.17:0/2851761081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59fc0b31f0 0x7f59fc0af1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:55.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.445+0000 7f5a012dc700 1 -- 172.21.15.17:0/2851761081 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ec00b120 con 0x7f59fc0aadb0 2022-08-01T19:39:55.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.445+0000 7f5a00adb700 1 --2- 172.21.15.17:0/2851761081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59fc0b31f0 0x7f59fc0af1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:39:55.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.445+0000 7f5a012dc700 1 --2- 172.21.15.17:0/2851761081 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59fc0aadb0 0x7f59fc147030 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7f59ec002f70 tx=0x7f59ec014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:55.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.445+0000 7f59f27fc700 1 -- 172.21.15.17:0/2851761081 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59ec020070 con 0x7f59fc0aadb0 2022-08-01T19:39:55.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.445+0000 7f59f27fc700 1 -- 172.21.15.17:0/2851761081 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59ec011840 con 0x7f59fc0aadb0 2022-08-01T19:39:55.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.445+0000 7f5a022de700 1 -- 172.21.15.17:0/2851761081 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59fc0ac390 con 0x7f59fc0aadb0 2022-08-01T19:39:55.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.445+0000 7f59f27fc700 1 -- 172.21.15.17:0/2851761081 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59ec011ba0 con 0x7f59fc0aadb0 2022-08-01T19:39:55.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.445+0000 7f5a022de700 1 -- 172.21.15.17:0/2851761081 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59fc0ad120 con 0x7f59fc0aadb0 2022-08-01T19:39:55.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.445+0000 7f5a022de700 1 -- 172.21.15.17:0/2851761081 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59fc0574e0 con 0x7f59fc0aadb0 2022-08-01T19:39:55.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.449+0000 7f59f27fc700 1 -- 172.21.15.17:0/2851761081 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f59ec00f070 con 0x7f59fc0aadb0 2022-08-01T19:39:55.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.449+0000 7f59f27fc700 1 --2- 172.21.15.17:0/2851761081 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f59dc0539e0 0x7f59dc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:55.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.449+0000 7f59f27fc700 1 -- 172.21.15.17:0/2851761081 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f59ec06cf40 con 0x7f59fc0aadb0 2022-08-01T19:39:55.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.449+0000 7f5a00adb700 1 --2- 172.21.15.17:0/2851761081 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f59dc0539e0 0x7f59dc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:55.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.449+0000 7f5a00adb700 1 --2- 172.21.15.17:0/2851761081 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f59dc0539e0 0x7f59dc055ea0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f59f800d000 tx=0x7f59f8009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:55.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.453+0000 7f59f27fc700 1 -- 172.21.15.17:0/2851761081 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f59ec043d00 con 0x7f59fc0aadb0 2022-08-01T19:39:55.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.577+0000 7f5a022de700 1 -- 172.21.15.17:0/2851761081 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59fc060330 con 0x7f59fc0aadb0 2022-08-01T19:39:55.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.577+0000 7f59f27fc700 1 -- 172.21.15.17:0/2851761081 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f59ec047320 con 0x7f59fc0aadb0 2022-08-01T19:39:55.581 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:39:55.583 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":76,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:39:53.329285+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:39:50.853967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2022-08-01T19:39:51.901692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:39:55.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.581+0000 7f5a022de700 1 -- 172.21.15.17:0/2851761081 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f59dc0539e0 msgr2=0x7f59dc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:55.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.581+0000 7f5a022de700 1 --2- 172.21.15.17:0/2851761081 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f59dc0539e0 0x7f59dc055ea0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f59f800d000 tx=0x7f59f8009040).stop 2022-08-01T19:39:55.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.581+0000 7f5a022de700 1 -- 172.21.15.17:0/2851761081 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59fc0aadb0 msgr2=0x7f59fc147030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:55.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.581+0000 7f5a022de700 1 --2- 172.21.15.17:0/2851761081 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59fc0aadb0 0x7f59fc147030 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7f59ec002f70 tx=0x7f59ec014000).stop 2022-08-01T19:39:55.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.581+0000 7f5a022de700 1 -- 172.21.15.17:0/2851761081 shutdown_connections 2022-08-01T19:39:55.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.581+0000 7f5a022de700 1 --2- 172.21.15.17:0/2851761081 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f59dc0539e0 0x7f59dc055ea0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:55.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.581+0000 7f5a022de700 1 --2- 172.21.15.17:0/2851761081 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f59fc0b7500 0x7f59fc0af6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:55.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.581+0000 7f5a022de700 1 --2- 172.21.15.17:0/2851761081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59fc0b31f0 0x7f59fc0af1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:55.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.581+0000 7f5a022de700 1 --2- 172.21.15.17:0/2851761081 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59fc0aadb0 0x7f59fc147030 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:55.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.581+0000 7f5a022de700 1 -- 172.21.15.17:0/2851761081 >> 172.21.15.17:0/2851761081 conn(0x7f59fc05f840 msgr2=0x7f59fc1479f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:55.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.581+0000 7f5a022de700 1 -- 172.21.15.17:0/2851761081 shutdown_connections 2022-08-01T19:39:55.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.581+0000 7f5a022de700 1 -- 172.21.15.17:0/2851761081 wait complete. 2022-08-01T19:39:55.596 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T19:39:50.853967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T19:39:51.901692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T19:39:55.597 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:39:55.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.705+0000 7f0d754dc700 1 Processor -- start 2022-08-01T19:39:55.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.705+0000 7f0d754dc700 1 -- start start 2022-08-01T19:39:55.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.705+0000 7f0d754dc700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f0d700b2710 0x7f0d7014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:55.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.705+0000 7f0d754dc700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0d700a8c50 0x7f0d7014b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:55.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.705+0000 7f0d754dc700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d700a7250 con 0x7f0d700b01a0 2022-08-01T19:39:55.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.705+0000 7f0d754dc700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d700a7480 con 0x7f0d700b2710 2022-08-01T19:39:55.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.705+0000 7f0d754dc700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d700a7720 con 0x7f0d700a8c50 2022-08-01T19:39:55.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.709+0000 7f0d74cdb700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0d700a8c50 0x7f0d7014b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:55.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.709+0000 7f0d6ffff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f0d700b2710 0x7f0d7014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:55.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.709+0000 7f0d74cdb700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0d700a8c50 0x7f0d7014b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:36732/0 (socket says 172.21.15.17:36732) 2022-08-01T19:39:55.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.709+0000 7f0d6f7fe700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7f0d700b01a0 0x7f0d700a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:49626/0 (socket says 172.21.15.17:49626) 2022-08-01T19:39:55.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.709+0000 7f0d74cdb700 1 -- 172.21.15.17:0/438436567 learned_addr learned my addr 172.21.15.17:0/438436567 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:55.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.709+0000 7f0d74cdb700 1 -- 172.21.15.17:0/438436567 >> v2:172.21.15.77:3300/0 conn(0x7f0d700b2710 msgr2=0x7f0d7014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:55.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.709+0000 7f0d74cdb700 1 --2- 172.21.15.17:0/438436567 >> v2:172.21.15.77:3300/0 conn(0x7f0d700b2710 0x7f0d7014afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:55.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.709+0000 7f0d74cdb700 1 -- 172.21.15.17:0/438436567 >> v1:172.21.15.77:6789/0 conn(0x7f0d700b01a0 legacy=0x7f0d700a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:55.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.709+0000 7f0d74cdb700 1 -- 172.21.15.17:0/438436567 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d700a79c0 con 0x7f0d700a8c50 2022-08-01T19:39:55.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.709+0000 7f0d74cdb700 1 --2- 172.21.15.17:0/438436567 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0d700a8c50 0x7f0d7014b4c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f0d680032f0 tx=0x7f0d68009290).ready entity=mon.2 client_cookie=a218664957679f8f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:55.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.709+0000 7f0d6effd700 1 -- 172.21.15.17:0/438436567 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d6800b8a0 con 0x7f0d700a8c50 2022-08-01T19:39:55.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.709+0000 7f0d6effd700 1 -- 172.21.15.17:0/438436567 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d6800ba00 con 0x7f0d700a8c50 2022-08-01T19:39:55.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.709+0000 7f0d6effd700 1 -- 172.21.15.17:0/438436567 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d6800bb70 con 0x7f0d700a8c50 2022-08-01T19:39:55.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.709+0000 7f0d754dc700 1 -- 172.21.15.17:0/438436567 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0d700a8c50 msgr2=0x7f0d7014b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:55.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.709+0000 7f0d754dc700 1 --2- 172.21.15.17:0/438436567 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0d700a8c50 0x7f0d7014b4c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f0d680032f0 tx=0x7f0d68009290).stop 2022-08-01T19:39:55.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.709+0000 7f0d754dc700 1 -- 172.21.15.17:0/438436567 shutdown_connections 2022-08-01T19:39:55.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.709+0000 7f0d754dc700 1 --2- 172.21.15.17:0/438436567 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0d700a8c50 0x7f0d7014b4c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:55.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.709+0000 7f0d754dc700 1 --2- 172.21.15.17:0/438436567 >> v2:172.21.15.77:3300/0 conn(0x7f0d700b2710 0x7f0d7014afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:55.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.709+0000 7f0d754dc700 1 -- 172.21.15.17:0/438436567 >> 172.21.15.17:0/438436567 conn(0x7f0d70060310 msgr2=0x7f0d700b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:55.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.709+0000 7f0d754dc700 1 -- 172.21.15.17:0/438436567 shutdown_connections 2022-08-01T19:39:55.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.709+0000 7f0d754dc700 1 -- 172.21.15.17:0/438436567 wait complete. 2022-08-01T19:39:55.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.709+0000 7f0d754dc700 1 Processor -- start 2022-08-01T19:39:55.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.709+0000 7f0d754dc700 1 -- start start 2022-08-01T19:39:55.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.713+0000 7f0d754dc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d700a8c50 0x7f0d700a99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:55.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.713+0000 7f0d754dc700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0d700b01a0 0x7f0d70144ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:55.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.713+0000 7f0d754dc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d700b2710 0x7f0d70156750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:55.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.713+0000 7f0d754dc700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d700a92d0 con 0x7f0d700b2710 2022-08-01T19:39:55.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.713+0000 7f0d754dc700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7015a840 con 0x7f0d700a8c50 2022-08-01T19:39:55.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.713+0000 7f0d754dc700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7015a5b0 con 0x7f0d700b01a0 2022-08-01T19:39:55.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.713+0000 7f0d6f7fe700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0d700b01a0 0x7f0d70144ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:55.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.713+0000 7f0d74cdb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d700b2710 0x7f0d70156750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:55.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.713+0000 7f0d6f7fe700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0d700b01a0 0x7f0d70144ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:36736/0 (socket says 172.21.15.17:36736) 2022-08-01T19:39:55.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.713+0000 7f0d6ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d700a8c50 0x7f0d700a99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:55.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.713+0000 7f0d6f7fe700 1 -- 172.21.15.17:0/2251203255 learned_addr learned my addr 172.21.15.17:0/2251203255 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:55.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.713+0000 7f0d6ffff700 1 -- 172.21.15.17:0/2251203255 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0d700b01a0 msgr2=0x7f0d70144ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:55.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.713+0000 7f0d6ffff700 1 --2- 172.21.15.17:0/2251203255 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0d700b01a0 0x7f0d70144ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:55.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.713+0000 7f0d6ffff700 1 -- 172.21.15.17:0/2251203255 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d700b2710 msgr2=0x7f0d70156750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:55.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.713+0000 7f0d6ffff700 1 --2- 172.21.15.17:0/2251203255 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d700b2710 0x7f0d70156750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:55.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.713+0000 7f0d6ffff700 1 -- 172.21.15.17:0/2251203255 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d68009530 con 0x7f0d700a8c50 2022-08-01T19:39:55.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.713+0000 7f0d6f7fe700 1 --2- 172.21.15.17:0/2251203255 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0d700b01a0 0x7f0d70144ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:39:55.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.713+0000 7f0d6ffff700 1 --2- 172.21.15.17:0/2251203255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d700a8c50 0x7f0d700a99a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f0d5800c000 tx=0x7f0d58006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:55.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.713+0000 7f0d6d7fa700 1 -- 172.21.15.17:0/2251203255 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d58011750 con 0x7f0d700a8c50 2022-08-01T19:39:55.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.713+0000 7f0d6d7fa700 1 -- 172.21.15.17:0/2251203255 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d580118b0 con 0x7f0d700a8c50 2022-08-01T19:39:55.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.713+0000 7f0d754dc700 1 -- 172.21.15.17:0/2251203255 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d7015aaa0 con 0x7f0d700a8c50 2022-08-01T19:39:55.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.713+0000 7f0d6d7fa700 1 -- 172.21.15.17:0/2251203255 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d58011c10 con 0x7f0d700a8c50 2022-08-01T19:39:55.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.713+0000 7f0d754dc700 1 -- 172.21.15.17:0/2251203255 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d701453d0 con 0x7f0d700a8c50 2022-08-01T19:39:55.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.713+0000 7f0d754dc700 1 -- 172.21.15.17:0/2251203255 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d700af200 con 0x7f0d700a8c50 2022-08-01T19:39:55.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.717+0000 7f0d6d7fa700 1 -- 172.21.15.17:0/2251203255 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0d58011d70 con 0x7f0d700a8c50 2022-08-01T19:39:55.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.717+0000 7f0d6d7fa700 1 --2- 172.21.15.17:0/2251203255 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0d48053b00 0x7f0d48055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:55.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.717+0000 7f0d6d7fa700 1 -- 172.21.15.17:0/2251203255 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0d5806c470 con 0x7f0d700a8c50 2022-08-01T19:39:55.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.717+0000 7f0d6f7fe700 1 --2- 172.21.15.17:0/2251203255 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0d48053b00 0x7f0d48055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:55.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.717+0000 7f0d6f7fe700 1 --2- 172.21.15.17:0/2251203255 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0d48053b00 0x7f0d48055fc0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f0d6000d000 tx=0x7f0d60007be0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:55.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.725+0000 7f0d6d7fa700 1 -- 172.21.15.17:0/2251203255 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0d580430c0 con 0x7f0d700a8c50 2022-08-01T19:39:55.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.845+0000 7f0d754dc700 1 -- 172.21.15.17:0/2251203255 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f0d700574e0 con 0x7f0d700a8c50 2022-08-01T19:39:55.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.849+0000 7f0d6d7fa700 1 -- 172.21.15.17:0/2251203255 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v76) v1 ==== 110+0+10 (secure 0 0 0) 0x7f0d580466e0 con 0x7f0d700a8c50 2022-08-01T19:39:55.853 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:39:55.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.849+0000 7f0d754dc700 1 -- 172.21.15.17:0/2251203255 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0d48053b00 msgr2=0x7f0d48055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:55.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.849+0000 7f0d754dc700 1 --2- 172.21.15.17:0/2251203255 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0d48053b00 0x7f0d48055fc0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f0d6000d000 tx=0x7f0d60007be0).stop 2022-08-01T19:39:55.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.849+0000 7f0d754dc700 1 -- 172.21.15.17:0/2251203255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d700a8c50 msgr2=0x7f0d700a99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:55.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.849+0000 7f0d754dc700 1 --2- 172.21.15.17:0/2251203255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d700a8c50 0x7f0d700a99a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f0d5800c000 tx=0x7f0d58006df0).stop 2022-08-01T19:39:55.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.849+0000 7f0d754dc700 1 -- 172.21.15.17:0/2251203255 shutdown_connections 2022-08-01T19:39:55.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.849+0000 7f0d754dc700 1 --2- 172.21.15.17:0/2251203255 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0d48053b00 0x7f0d48055fc0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:55.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.849+0000 7f0d754dc700 1 --2- 172.21.15.17:0/2251203255 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d700b2710 0x7f0d70156750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:55.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.849+0000 7f0d754dc700 1 --2- 172.21.15.17:0/2251203255 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0d700b01a0 0x7f0d70144ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:55.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.849+0000 7f0d754dc700 1 --2- 172.21.15.17:0/2251203255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d700a8c50 0x7f0d700a99a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:55.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.849+0000 7f0d754dc700 1 -- 172.21.15.17:0/2251203255 >> 172.21.15.17:0/2251203255 conn(0x7f0d70060310 msgr2=0x7f0d7014b970 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:55.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.849+0000 7f0d754dc700 1 -- 172.21.15.17:0/2251203255 shutdown_connections 2022-08-01T19:39:55.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.849+0000 7f0d754dc700 1 -- 172.21.15.17:0/2251203255 wait complete. 2022-08-01T19:39:55.875 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:39:56.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.993+0000 7f679343c700 1 Processor -- start 2022-08-01T19:39:56.007 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.993+0000 7f679343c700 1 -- start start 2022-08-01T19:39:56.007 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.993+0000 7f679343c700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f678c0b2050 0x7f678c0b1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:56.008 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.993+0000 7f679343c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f678c0b0140 0x7f678c0b2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:56.008 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.993+0000 7f679343c700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f678c0b05b0 con 0x7f678c0a7250 2022-08-01T19:39:56.008 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.993+0000 7f679343c700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f678c0b2920 con 0x7f678c0b0140 2022-08-01T19:39:56.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.993+0000 7f679343c700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f678c0b07e0 con 0x7f678c0b2050 2022-08-01T19:39:56.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.993+0000 7f67909d7700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f678c0b0140 0x7f678c0b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:56.010 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.993+0000 7f67909d7700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f678c0b0140 0x7f678c0b2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40644/0 (socket says 172.21.15.17:40644) 2022-08-01T19:39:56.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.993+0000 7f67909d7700 1 -- 172.21.15.17:0/2859356545 learned_addr learned my addr 172.21.15.17:0/2859356545 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:56.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.993+0000 7f67919d9700 1 --1- 172.21.15.17:0/2859356545 >> v1:172.21.15.17:6789/0 conn(0x7f678c0a7250 0x7f678c1413e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:32842/0 (socket says 172.21.15.17:32842) 2022-08-01T19:39:56.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.993+0000 7f67911d8700 1 --2- 172.21.15.17:0/2859356545 >> v2:172.21.15.77:3300/0 conn(0x7f678c0b2050 0x7f678c0b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:56.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.993+0000 7f67909d7700 1 -- 172.21.15.17:0/2859356545 >> v2:172.21.15.77:3300/0 conn(0x7f678c0b2050 msgr2=0x7f678c0b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:56.013 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.993+0000 7f67909d7700 1 --2- 172.21.15.17:0/2859356545 >> v2:172.21.15.77:3300/0 conn(0x7f678c0b2050 0x7f678c0b1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:56.013 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.993+0000 7f67909d7700 1 -- 172.21.15.17:0/2859356545 >> v1:172.21.15.17:6789/0 conn(0x7f678c0a7250 legacy=0x7f678c1413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:56.013 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.993+0000 7f67909d7700 1 -- 172.21.15.17:0/2859356545 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f678c0b0a80 con 0x7f678c0b0140 2022-08-01T19:39:56.013 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.993+0000 7f67909d7700 1 --2- 172.21.15.17:0/2859356545 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f678c0b0140 0x7f678c0b2430 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f67880032f0 tx=0x7f6788009290).ready entity=mon.0 client_cookie=18c0dbdd58424192 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:56.013 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.993+0000 7f6783fff700 1 -- 172.21.15.17:0/2859356545 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f678800b870 con 0x7f678c0b0140 2022-08-01T19:39:56.014 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.993+0000 7f6783fff700 1 -- 172.21.15.17:0/2859356545 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678800b9d0 con 0x7f678c0b0140 2022-08-01T19:39:56.014 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.993+0000 7f6783fff700 1 -- 172.21.15.17:0/2859356545 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f678800bb40 con 0x7f678c0b0140 2022-08-01T19:39:56.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.997+0000 7f679343c700 1 -- 172.21.15.17:0/2859356545 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f678c0b0140 msgr2=0x7f678c0b2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:56.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.997+0000 7f679343c700 1 --2- 172.21.15.17:0/2859356545 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f678c0b0140 0x7f678c0b2430 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f67880032f0 tx=0x7f6788009290).stop 2022-08-01T19:39:56.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.997+0000 7f679343c700 1 -- 172.21.15.17:0/2859356545 shutdown_connections 2022-08-01T19:39:56.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.997+0000 7f679343c700 1 --2- 172.21.15.17:0/2859356545 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f678c0b0140 0x7f678c0b2430 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:56.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.997+0000 7f679343c700 1 --2- 172.21.15.17:0/2859356545 >> v2:172.21.15.77:3300/0 conn(0x7f678c0b2050 0x7f678c0b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:56.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.997+0000 7f679343c700 1 -- 172.21.15.17:0/2859356545 >> 172.21.15.17:0/2859356545 conn(0x7f678c05f840 msgr2=0x7f678c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:56.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.997+0000 7f679343c700 1 -- 172.21.15.17:0/2859356545 shutdown_connections 2022-08-01T19:39:56.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.997+0000 7f679343c700 1 -- 172.21.15.17:0/2859356545 wait complete. 2022-08-01T19:39:56.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.997+0000 7f679343c700 1 Processor -- start 2022-08-01T19:39:56.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.997+0000 7f679343c700 1 -- start start 2022-08-01T19:39:56.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.997+0000 7f679343c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f678c0a7250 0x7f678c0abe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:56.018 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.997+0000 7f679343c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f678c0b0140 0x7f678c14ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:56.018 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.997+0000 7f679343c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f678c0b2050 0x7f678c14b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:56.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.997+0000 7f679343c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c148cc0 con 0x7f678c0a7250 2022-08-01T19:39:56.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.997+0000 7f679343c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c14b510 con 0x7f678c0b0140 2022-08-01T19:39:56.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.997+0000 7f679343c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678c148ef0 con 0x7f678c0b2050 2022-08-01T19:39:56.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.997+0000 7f67911d8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f678c0a7250 0x7f678c0abe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:56.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.997+0000 7f67911d8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f678c0a7250 0x7f678c0abe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40650/0 (socket says 172.21.15.17:40650) 2022-08-01T19:39:56.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.997+0000 7f67911d8700 1 -- 172.21.15.17:0/308257283 learned_addr learned my addr 172.21.15.17:0/308257283 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:56.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.997+0000 7f67909d7700 1 --2- 172.21.15.17:0/308257283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f678c0b0140 0x7f678c14ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:56.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.997+0000 7f67919d9700 1 --2- 172.21.15.17:0/308257283 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f678c0b2050 0x7f678c14b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:56.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.997+0000 7f67911d8700 1 -- 172.21.15.17:0/308257283 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f678c0b2050 msgr2=0x7f678c14b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:56.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.997+0000 7f67911d8700 1 --2- 172.21.15.17:0/308257283 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f678c0b2050 0x7f678c14b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:56.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.997+0000 7f67911d8700 1 -- 172.21.15.17:0/308257283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f678c0b0140 msgr2=0x7f678c14ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:56.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.997+0000 7f67911d8700 1 --2- 172.21.15.17:0/308257283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f678c0b0140 0x7f678c14ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:56.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.997+0000 7f67911d8700 1 -- 172.21.15.17:0/308257283 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6788009530 con 0x7f678c0a7250 2022-08-01T19:39:56.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.997+0000 7f67909d7700 1 --2- 172.21.15.17:0/308257283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f678c0b0140 0x7f678c14ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:39:56.026 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:55.997+0000 7f67911d8700 1 --2- 172.21.15.17:0/308257283 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f678c0a7250 0x7f678c0abe50 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f677c0030b0 tx=0x7f677c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:56.026 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.001+0000 7f67827fc700 1 -- 172.21.15.17:0/308257283 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f677c011780 con 0x7f678c0a7250 2022-08-01T19:39:56.027 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.001+0000 7f67827fc700 1 -- 172.21.15.17:0/308257283 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f677c0118e0 con 0x7f678c0a7250 2022-08-01T19:39:56.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.001+0000 7f679343c700 1 -- 172.21.15.17:0/308257283 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f678c14a690 con 0x7f678c0a7250 2022-08-01T19:39:56.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.001+0000 7f67827fc700 1 -- 172.21.15.17:0/308257283 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f677c011c40 con 0x7f678c0a7250 2022-08-01T19:39:56.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.001+0000 7f679343c700 1 -- 172.21.15.17:0/308257283 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f678c0a96b0 con 0x7f678c0a7250 2022-08-01T19:39:56.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.001+0000 7f679343c700 1 -- 172.21.15.17:0/308257283 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f678c0b0d20 con 0x7f678c0a7250 2022-08-01T19:39:56.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.001+0000 7f67827fc700 1 -- 172.21.15.17:0/308257283 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f677c01c460 con 0x7f678c0a7250 2022-08-01T19:39:56.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.001+0000 7f67827fc700 1 --2- 172.21.15.17:0/308257283 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6774053b00 0x7f6774055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:56.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.001+0000 7f67827fc700 1 -- 172.21.15.17:0/308257283 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f677c06c1c0 con 0x7f678c0a7250 2022-08-01T19:39:56.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.001+0000 7f67909d7700 1 --2- 172.21.15.17:0/308257283 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6774053b00 0x7f6774055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:56.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.005+0000 7f67909d7700 1 --2- 172.21.15.17:0/308257283 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6774053b00 0x7f6774055fc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f678801b000 tx=0x7f6788016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:56.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.009+0000 7f67827fc700 1 -- 172.21.15.17:0/308257283 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f677c042e10 con 0x7f678c0a7250 2022-08-01T19:39:56.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.133+0000 7f679343c700 1 -- 172.21.15.17:0/308257283 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f678c0574e0 con 0x7f678c0a7250 2022-08-01T19:39:56.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.137+0000 7f67827fc700 1 -- 172.21.15.17:0/308257283 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v76) v1 ==== 92+0+10 (secure 0 0 0) 0x7f677c046430 con 0x7f678c0a7250 2022-08-01T19:39:56.143 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:39:56.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.137+0000 7f679343c700 1 -- 172.21.15.17:0/308257283 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6774053b00 msgr2=0x7f6774055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:56.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.137+0000 7f679343c700 1 --2- 172.21.15.17:0/308257283 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6774053b00 0x7f6774055fc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f678801b000 tx=0x7f6788016000).stop 2022-08-01T19:39:56.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.137+0000 7f679343c700 1 -- 172.21.15.17:0/308257283 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f678c0a7250 msgr2=0x7f678c0abe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:56.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.137+0000 7f679343c700 1 --2- 172.21.15.17:0/308257283 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f678c0a7250 0x7f678c0abe50 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f677c0030b0 tx=0x7f677c008040).stop 2022-08-01T19:39:56.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.137+0000 7f679343c700 1 -- 172.21.15.17:0/308257283 shutdown_connections 2022-08-01T19:39:56.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.137+0000 7f679343c700 1 --2- 172.21.15.17:0/308257283 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6774053b00 0x7f6774055fc0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:56.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.137+0000 7f679343c700 1 --2- 172.21.15.17:0/308257283 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f678c0b2050 0x7f678c14b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:56.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.137+0000 7f679343c700 1 --2- 172.21.15.17:0/308257283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f678c0b0140 0x7f678c14ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:56.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.137+0000 7f679343c700 1 --2- 172.21.15.17:0/308257283 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f678c0a7250 0x7f678c0abe50 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:56.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.137+0000 7f679343c700 1 -- 172.21.15.17:0/308257283 >> 172.21.15.17:0/308257283 conn(0x7f678c05f840 msgr2=0x7f678c0a8020 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:56.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.137+0000 7f679343c700 1 -- 172.21.15.17:0/308257283 shutdown_connections 2022-08-01T19:39:56.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.137+0000 7f679343c700 1 -- 172.21.15.17:0/308257283 wait complete. 2022-08-01T19:39:56.154 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T19:39:56.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.261+0000 7f2f5445d700 1 Processor -- start 2022-08-01T19:39:56.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.261+0000 7f2f5445d700 1 -- start start 2022-08-01T19:39:56.266 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.265+0000 7f2f5445d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2f4c0b4310 0x7f2f4c140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:56.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.265+0000 7f2f5445d700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4c143cf0 con 0x7f2f4c0b4820 2022-08-01T19:39:56.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.265+0000 7f2f5445d700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4c143f90 con 0x7f2f4c146ae0 2022-08-01T19:39:56.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.265+0000 7f2f5445d700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f4c1476a0 con 0x7f2f4c0b4310 2022-08-01T19:39:56.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.265+0000 7f2f529fa700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2f4c0b4820 0x7f2f4c0b5060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:32856/0 (socket says 172.21.15.17:32856) 2022-08-01T19:39:56.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.265+0000 7f2f529fa700 1 -- 172.21.15.17:0/678481012 learned_addr learned my addr 172.21.15.17:0/678481012 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:56.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.265+0000 7f2f519f8700 1 --2- 172.21.15.17:0/678481012 >> v2:172.21.15.17:3300/0 conn(0x7f2f4c0b4310 0x7f2f4c140ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:56.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.265+0000 7f2f511f7700 1 -- 172.21.15.17:0/678481012 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1540513507 0 0) 0x7f2f4c143cf0 con 0x7f2f4c0b4820 2022-08-01T19:39:56.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.265+0000 7f2f511f7700 1 -- 172.21.15.17:0/678481012 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f34003260 con 0x7f2f4c0b4820 2022-08-01T19:39:56.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.265+0000 7f2f511f7700 1 -- 172.21.15.17:0/678481012 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3701160323 0 0) 0x7f2f4c143f90 con 0x7f2f4c146ae0 2022-08-01T19:39:56.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.265+0000 7f2f519f8700 1 -- 172.21.15.17:0/678481012 >> v1:172.21.15.77:6789/0 conn(0x7f2f4c146ae0 legacy=0x7f2f4c1403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:56.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.265+0000 7f2f519f8700 1 -- 172.21.15.17:0/678481012 >> v1:172.21.15.17:6789/0 conn(0x7f2f4c0b4820 legacy=0x7f2f4c0b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:56.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.265+0000 7f2f519f8700 1 -- 172.21.15.17:0/678481012 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4c0b0e20 con 0x7f2f4c0b4310 2022-08-01T19:39:56.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.265+0000 7f2f519f8700 1 --2- 172.21.15.17:0/678481012 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f4c0b4310 0x7f2f4c140ad0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f2f40002f70 tx=0x7f2f40006df0).ready entity=mon.0 client_cookie=7807a5469bce346 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:56.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.265+0000 7f2f511f7700 1 -- 172.21.15.17:0/678481012 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f4000d610 con 0x7f2f4c0b4310 2022-08-01T19:39:56.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.265+0000 7f2f511f7700 1 -- 172.21.15.17:0/678481012 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f40000e90 con 0x7f2f4c0b4310 2022-08-01T19:39:56.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.265+0000 7f2f511f7700 1 -- 172.21.15.17:0/678481012 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f4000d960 con 0x7f2f4c0b4310 2022-08-01T19:39:56.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.265+0000 7f2f5445d700 1 -- 172.21.15.17:0/678481012 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f4c0b4310 msgr2=0x7f2f4c140ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:56.271 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.265+0000 7f2f5445d700 1 --2- 172.21.15.17:0/678481012 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f4c0b4310 0x7f2f4c140ad0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f2f40002f70 tx=0x7f2f40006df0).stop 2022-08-01T19:39:56.271 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.265+0000 7f2f5445d700 1 -- 172.21.15.17:0/678481012 shutdown_connections 2022-08-01T19:39:56.271 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.265+0000 7f2f5445d700 1 --2- 172.21.15.17:0/678481012 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f4c0b4310 0x7f2f4c140ad0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:56.271 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.265+0000 7f2f5445d700 1 -- 172.21.15.17:0/678481012 >> 172.21.15.17:0/678481012 conn(0x7f2f4c05c620 msgr2=0x7f2f4c05ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:56.272 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.265+0000 7f2f5445d700 1 -- 172.21.15.17:0/678481012 shutdown_connections 2022-08-01T19:39:56.272 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.265+0000 7f2f5445d700 1 -- 172.21.15.17:0/678481012 wait complete. 2022-08-01T19:39:56.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.269+0000 7f2f5445d700 1 Processor -- start 2022-08-01T19:39:56.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.269+0000 7f2f5445d700 1 -- start start 2022-08-01T19:39:56.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.269+0000 7f2f5445d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f4c0b4310 0x7f2f4c140190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:56.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.269+0000 7f2f5445d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f4c0b4820 0x7f2f4c13ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:56.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.269+0000 7f2f5445d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2f4c146ae0 0x7f2f4c13b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:56.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.269+0000 7f2f5445d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c13b7c0 con 0x7f2f4c0b4820 2022-08-01T19:39:56.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.269+0000 7f2f5445d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c145450 con 0x7f2f4c0b4310 2022-08-01T19:39:56.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.269+0000 7f2f5445d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c145d30 con 0x7f2f4c146ae0 2022-08-01T19:39:56.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.269+0000 7f2f519f8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f4c0b4820 0x7f2f4c13ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:56.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.269+0000 7f2f519f8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f4c0b4820 0x7f2f4c13ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40666/0 (socket says 172.21.15.17:40666) 2022-08-01T19:39:56.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.269+0000 7f2f519f8700 1 -- 172.21.15.17:0/1862052146 learned_addr learned my addr 172.21.15.17:0/1862052146 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:56.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.269+0000 7f2f521f9700 1 --2- 172.21.15.17:0/1862052146 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f4c0b4310 0x7f2f4c140190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:56.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.269+0000 7f2f529fa700 1 --2- 172.21.15.17:0/1862052146 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2f4c146ae0 0x7f2f4c13b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:56.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.273+0000 7f2f519f8700 1 -- 172.21.15.17:0/1862052146 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2f4c146ae0 msgr2=0x7f2f4c13b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:56.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.273+0000 7f2f519f8700 1 --2- 172.21.15.17:0/1862052146 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2f4c146ae0 0x7f2f4c13b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:56.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.273+0000 7f2f519f8700 1 -- 172.21.15.17:0/1862052146 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f4c0b4310 msgr2=0x7f2f4c140190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:56.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.273+0000 7f2f519f8700 1 --2- 172.21.15.17:0/1862052146 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f4c0b4310 0x7f2f4c140190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:56.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.273+0000 7f2f519f8700 1 -- 172.21.15.17:0/1862052146 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4000b180 con 0x7f2f4c0b4820 2022-08-01T19:39:56.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.273+0000 7f2f521f9700 1 --2- 172.21.15.17:0/1862052146 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f4c0b4310 0x7f2f4c140190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:39:56.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.273+0000 7f2f519f8700 1 --2- 172.21.15.17:0/1862052146 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f4c0b4820 0x7f2f4c13ade0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7f2f4c0b97f0 tx=0x7f2f40014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:56.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.273+0000 7f2f3f7fe700 1 -- 172.21.15.17:0/1862052146 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f40016070 con 0x7f2f4c0b4820 2022-08-01T19:39:56.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.273+0000 7f2f3f7fe700 1 -- 172.21.15.17:0/1862052146 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f4000de90 con 0x7f2f4c0b4820 2022-08-01T19:39:56.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.273+0000 7f2f5445d700 1 -- 172.21.15.17:0/1862052146 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f4c1457f0 con 0x7f2f4c0b4820 2022-08-01T19:39:56.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.273+0000 7f2f3f7fe700 1 -- 172.21.15.17:0/1862052146 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f40013d20 con 0x7f2f4c0b4820 2022-08-01T19:39:56.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.273+0000 7f2f5445d700 1 -- 172.21.15.17:0/1862052146 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f4c0ba320 con 0x7f2f4c0b4820 2022-08-01T19:39:56.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.273+0000 7f2f5445d700 1 -- 172.21.15.17:0/1862052146 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f4c155650 con 0x7f2f4c0b4820 2022-08-01T19:39:56.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.273+0000 7f2f3f7fe700 1 -- 172.21.15.17:0/1862052146 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2f40025030 con 0x7f2f4c0b4820 2022-08-01T19:39:56.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.273+0000 7f2f3f7fe700 1 --2- 172.21.15.17:0/1862052146 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2f2c053980 0x7f2f2c055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:56.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.277+0000 7f2f3f7fe700 1 -- 172.21.15.17:0/1862052146 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f2f4006cf70 con 0x7f2f4c0b4820 2022-08-01T19:39:56.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.277+0000 7f2f521f9700 1 --2- 172.21.15.17:0/1862052146 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2f2c053980 0x7f2f2c055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:56.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.277+0000 7f2f521f9700 1 --2- 172.21.15.17:0/1862052146 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2f2c053980 0x7f2f2c055e40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f2f4800d000 tx=0x7f2f48009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:56.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.281+0000 7f2f3f7fe700 1 -- 172.21.15.17:0/1862052146 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2f40043d30 con 0x7f2f4c0b4820 2022-08-01T19:39:56.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.409+0000 7f2f5445d700 1 -- 172.21.15.17:0/1862052146 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2f4c0574e0 con 0x7f2f4c0b4820 2022-08-01T19:39:56.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.409+0000 7f2f3f7fe700 1 -- 172.21.15.17:0/1862052146 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v76) v1 ==== 104+0+10 (secure 0 0 0) 0x7f2f40047350 con 0x7f2f4c0b4820 2022-08-01T19:39:56.411 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:39:56.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.413+0000 7f2f5445d700 1 -- 172.21.15.17:0/1862052146 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2f2c053980 msgr2=0x7f2f2c055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:56.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.413+0000 7f2f5445d700 1 --2- 172.21.15.17:0/1862052146 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2f2c053980 0x7f2f2c055e40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f2f4800d000 tx=0x7f2f48009040).stop 2022-08-01T19:39:56.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.413+0000 7f2f5445d700 1 -- 172.21.15.17:0/1862052146 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f4c0b4820 msgr2=0x7f2f4c13ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:56.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.413+0000 7f2f5445d700 1 --2- 172.21.15.17:0/1862052146 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f4c0b4820 0x7f2f4c13ade0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7f2f4c0b97f0 tx=0x7f2f40014000).stop 2022-08-01T19:39:56.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.413+0000 7f2f5445d700 1 -- 172.21.15.17:0/1862052146 shutdown_connections 2022-08-01T19:39:56.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.413+0000 7f2f5445d700 1 --2- 172.21.15.17:0/1862052146 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2f2c053980 0x7f2f2c055e40 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:56.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.413+0000 7f2f5445d700 1 --2- 172.21.15.17:0/1862052146 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2f4c146ae0 0x7f2f4c13b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:56.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.413+0000 7f2f5445d700 1 --2- 172.21.15.17:0/1862052146 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f4c0b4820 0x7f2f4c13ade0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:56.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.413+0000 7f2f5445d700 1 --2- 172.21.15.17:0/1862052146 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f4c0b4310 0x7f2f4c140190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:56.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.413+0000 7f2f5445d700 1 -- 172.21.15.17:0/1862052146 >> 172.21.15.17:0/1862052146 conn(0x7f2f4c05c620 msgr2=0x7f2f4c0b5060 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:56.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.413+0000 7f2f5445d700 1 -- 172.21.15.17:0/1862052146 shutdown_connections 2022-08-01T19:39:56.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.413+0000 7f2f5445d700 1 -- 172.21.15.17:0/1862052146 wait complete. 2022-08-01T19:39:56.432 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T19:39:56.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.549+0000 7fe119f44700 1 Processor -- start 2022-08-01T19:39:56.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.549+0000 7fe119f44700 1 -- start start 2022-08-01T19:39:56.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.549+0000 7fe119f44700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1140b6420 0x7fe114061600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:56.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.553+0000 7fe119f44700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe1140b41f0 0x7fe114061af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:56.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.553+0000 7fe119f44700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fe114061fe0 0x7fe1141465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:56.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.553+0000 7fe119f44700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1140b60e0 con 0x7fe1140b41f0 2022-08-01T19:39:56.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.553+0000 7fe119f44700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1140b2100 con 0x7fe114061fe0 2022-08-01T19:39:56.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.553+0000 7fe119f44700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1140b23a0 con 0x7fe1140b6420 2022-08-01T19:39:56.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.553+0000 7fe113fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe1140b41f0 0x7fe114061af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:56.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.553+0000 7fe113fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe1140b41f0 0x7fe114061af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40674/0 (socket says 172.21.15.17:40674) 2022-08-01T19:39:56.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.553+0000 7fe113fff700 1 -- 172.21.15.17:0/2227973140 learned_addr learned my addr 172.21.15.17:0/2227973140 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:56.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.553+0000 7fe119743700 1 --2- 172.21.15.17:0/2227973140 >> v2:172.21.15.77:3300/0 conn(0x7fe114061fe0 0x7fe1141465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:56.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.553+0000 7fe118f42700 1 --2- 172.21.15.17:0/2227973140 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1140b6420 0x7fe114061600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:56.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.553+0000 7fe113fff700 1 -- 172.21.15.17:0/2227973140 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1140b6420 msgr2=0x7fe114061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:56.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.553+0000 7fe113fff700 1 --2- 172.21.15.17:0/2227973140 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1140b6420 0x7fe114061600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:56.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.553+0000 7fe113fff700 1 -- 172.21.15.17:0/2227973140 >> v2:172.21.15.77:3300/0 conn(0x7fe114061fe0 msgr2=0x7fe1141465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:56.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.553+0000 7fe113fff700 1 --2- 172.21.15.17:0/2227973140 >> v2:172.21.15.77:3300/0 conn(0x7fe114061fe0 0x7fe1141465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:56.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.553+0000 7fe113fff700 1 -- 172.21.15.17:0/2227973140 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1140b2640 con 0x7fe1140b41f0 2022-08-01T19:39:56.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.553+0000 7fe113fff700 1 --2- 172.21.15.17:0/2227973140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1140b41f0 0x7fe114061af0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7fe100002f70 tx=0x7fe100006eb0).ready entity=mon.0 client_cookie=6590cd0174c4b75c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:56.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.553+0000 7fe1137fe700 1 -- 172.21.15.17:0/2227973140 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe10000d570 con 0x7fe1140b41f0 2022-08-01T19:39:56.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.553+0000 7fe1137fe700 1 -- 172.21.15.17:0/2227973140 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe100000e90 con 0x7fe1140b41f0 2022-08-01T19:39:56.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.553+0000 7fe1137fe700 1 -- 172.21.15.17:0/2227973140 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe10000d900 con 0x7fe1140b41f0 2022-08-01T19:39:56.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.553+0000 7fe119f44700 1 -- 172.21.15.17:0/2227973140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1140b41f0 msgr2=0x7fe114061af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:56.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.553+0000 7fe119f44700 1 --2- 172.21.15.17:0/2227973140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1140b41f0 0x7fe114061af0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7fe100002f70 tx=0x7fe100006eb0).stop 2022-08-01T19:39:56.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.553+0000 7fe119f44700 1 -- 172.21.15.17:0/2227973140 shutdown_connections 2022-08-01T19:39:56.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.553+0000 7fe119f44700 1 --2- 172.21.15.17:0/2227973140 >> v2:172.21.15.77:3300/0 conn(0x7fe114061fe0 0x7fe1141465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:56.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.553+0000 7fe119f44700 1 --2- 172.21.15.17:0/2227973140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1140b41f0 0x7fe114061af0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:56.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.553+0000 7fe119f44700 1 --2- 172.21.15.17:0/2227973140 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1140b6420 0x7fe114061600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:56.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.553+0000 7fe119f44700 1 -- 172.21.15.17:0/2227973140 >> 172.21.15.17:0/2227973140 conn(0x7fe11405c620 msgr2=0x7fe11405ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:56.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.553+0000 7fe119f44700 1 -- 172.21.15.17:0/2227973140 shutdown_connections 2022-08-01T19:39:56.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.553+0000 7fe119f44700 1 -- 172.21.15.17:0/2227973140 wait complete. 2022-08-01T19:39:56.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.557+0000 7fe119f44700 1 Processor -- start 2022-08-01T19:39:56.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.557+0000 7fe119f44700 1 -- start start 2022-08-01T19:39:56.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.557+0000 7fe119f44700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe114061fe0 0x7fe114145ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:56.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.557+0000 7fe119f44700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1140b6420 0x7fe1140af000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:56.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.557+0000 7fe119f44700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe114146f00 0x7fe1140af4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:56.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.557+0000 7fe119f44700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1140ae550 con 0x7fe114061fe0 2022-08-01T19:39:56.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.557+0000 7fe119f44700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11413e7f0 con 0x7fe114146f00 2022-08-01T19:39:56.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.557+0000 7fe119f44700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe11413f060 con 0x7fe1140b6420 2022-08-01T19:39:56.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.557+0000 7fe118f42700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe114061fe0 0x7fe114145ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:56.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.557+0000 7fe118f42700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe114061fe0 0x7fe114145ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40678/0 (socket says 172.21.15.17:40678) 2022-08-01T19:39:56.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.557+0000 7fe118f42700 1 -- 172.21.15.17:0/2104365041 learned_addr learned my addr 172.21.15.17:0/2104365041 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:56.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.557+0000 7fe113fff700 1 --2- 172.21.15.17:0/2104365041 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1140b6420 0x7fe1140af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:56.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.557+0000 7fe119743700 1 --2- 172.21.15.17:0/2104365041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe114146f00 0x7fe1140af4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:56.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.557+0000 7fe118f42700 1 -- 172.21.15.17:0/2104365041 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1140b6420 msgr2=0x7fe1140af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:56.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.557+0000 7fe118f42700 1 --2- 172.21.15.17:0/2104365041 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1140b6420 0x7fe1140af000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:56.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.557+0000 7fe118f42700 1 -- 172.21.15.17:0/2104365041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe114146f00 msgr2=0x7fe1140af4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:56.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.557+0000 7fe118f42700 1 --2- 172.21.15.17:0/2104365041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe114146f00 0x7fe1140af4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:56.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.557+0000 7fe118f42700 1 -- 172.21.15.17:0/2104365041 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10000b1e0 con 0x7fe114061fe0 2022-08-01T19:39:56.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.557+0000 7fe118f42700 1 --2- 172.21.15.17:0/2104365041 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe114061fe0 0x7fe114145ba0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7fe1140b0d30 tx=0x7fe108008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:56.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.561+0000 7fe111ffb700 1 -- 172.21.15.17:0/2104365041 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe108011780 con 0x7fe114061fe0 2022-08-01T19:39:56.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.561+0000 7fe111ffb700 1 -- 172.21.15.17:0/2104365041 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1080118e0 con 0x7fe114061fe0 2022-08-01T19:39:56.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.561+0000 7fe119f44700 1 -- 172.21.15.17:0/2104365041 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe11413f300 con 0x7fe114061fe0 2022-08-01T19:39:56.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.561+0000 7fe111ffb700 1 -- 172.21.15.17:0/2104365041 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe108011c40 con 0x7fe114061fe0 2022-08-01T19:39:56.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.561+0000 7fe119f44700 1 -- 172.21.15.17:0/2104365041 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe11413fcf0 con 0x7fe114061fe0 2022-08-01T19:39:56.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.561+0000 7fe119f44700 1 -- 172.21.15.17:0/2104365041 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1140b28e0 con 0x7fe114061fe0 2022-08-01T19:39:56.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.561+0000 7fe111ffb700 1 -- 172.21.15.17:0/2104365041 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe10801c460 con 0x7fe114061fe0 2022-08-01T19:39:56.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.561+0000 7fe111ffb700 1 --2- 172.21.15.17:0/2104365041 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe0f4053a70 0x7fe0f4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:56.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.561+0000 7fe111ffb700 1 -- 172.21.15.17:0/2104365041 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe10806c260 con 0x7fe114061fe0 2022-08-01T19:39:56.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.565+0000 7fe113fff700 1 --2- 172.21.15.17:0/2104365041 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe0f4053a70 0x7fe0f4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:56.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.565+0000 7fe113fff700 1 --2- 172.21.15.17:0/2104365041 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe0f4053a70 0x7fe0f4055f30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7fe10001a000 tx=0x7fe100015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:56.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.569+0000 7fe111ffb700 1 -- 172.21.15.17:0/2104365041 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe108042fc0 con 0x7fe114061fe0 2022-08-01T19:39:56.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.701+0000 7fe119f44700 1 -- 172.21.15.17:0/2104365041 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe1140574e0 con 0x7fe114061fe0 2022-08-01T19:39:56.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.701+0000 7fe111ffb700 1 -- 172.21.15.17:0/2104365041 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v76) v1 ==== 100+0+10 (secure 0 0 0) 0x7fe1080465e0 con 0x7fe114061fe0 2022-08-01T19:39:56.713 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:39:56.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.701+0000 7fe119f44700 1 -- 172.21.15.17:0/2104365041 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe0f4053a70 msgr2=0x7fe0f4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:56.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.701+0000 7fe119f44700 1 --2- 172.21.15.17:0/2104365041 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe0f4053a70 0x7fe0f4055f30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7fe10001a000 tx=0x7fe100015000).stop 2022-08-01T19:39:56.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.701+0000 7fe119f44700 1 -- 172.21.15.17:0/2104365041 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe114061fe0 msgr2=0x7fe114145ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:56.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.701+0000 7fe119f44700 1 --2- 172.21.15.17:0/2104365041 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe114061fe0 0x7fe114145ba0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7fe1140b0d30 tx=0x7fe108008040).stop 2022-08-01T19:39:56.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.701+0000 7fe119f44700 1 -- 172.21.15.17:0/2104365041 shutdown_connections 2022-08-01T19:39:56.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.701+0000 7fe119f44700 1 --2- 172.21.15.17:0/2104365041 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe0f4053a70 0x7fe0f4055f30 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:56.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.705+0000 7fe119f44700 1 --2- 172.21.15.17:0/2104365041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe114146f00 0x7fe1140af4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:56.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.705+0000 7fe119f44700 1 --2- 172.21.15.17:0/2104365041 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1140b6420 0x7fe1140af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:56.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.705+0000 7fe119f44700 1 --2- 172.21.15.17:0/2104365041 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe114061fe0 0x7fe114145ba0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:56.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.705+0000 7fe119f44700 1 -- 172.21.15.17:0/2104365041 >> 172.21.15.17:0/2104365041 conn(0x7fe11405c620 msgr2=0x7fe114143520 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:56.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.705+0000 7fe119f44700 1 -- 172.21.15.17:0/2104365041 shutdown_connections 2022-08-01T19:39:56.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.705+0000 7fe119f44700 1 -- 172.21.15.17:0/2104365041 wait complete. 2022-08-01T19:39:56.722 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T19:39:56.723 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:39:56.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.829+0000 7fbf3da76700 1 Processor -- start 2022-08-01T19:39:56.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.829+0000 7fbf3da76700 1 -- start start 2022-08-01T19:39:56.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.829+0000 7fbf3da76700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbf380b01a0 0x7fbf380a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:56.834 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.829+0000 7fbf3da76700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf380a7250 con 0x7fbf380b2710 2022-08-01T19:39:56.834 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.829+0000 7fbf3da76700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf380a7480 con 0x7fbf380afd60 2022-08-01T19:39:56.834 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.829+0000 7fbf3da76700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf380a7720 con 0x7fbf380b01a0 2022-08-01T19:39:56.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.829+0000 7fbf37fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbf380b01a0 0x7fbf380a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:56.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.829+0000 7fbf3ca74700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fbf380b2710 0x7fbf3814afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:32882/0 (socket says 172.21.15.17:32882) 2022-08-01T19:39:56.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.829+0000 7fbf3ca74700 1 -- 172.21.15.17:0/519643711 learned_addr learned my addr 172.21.15.17:0/519643711 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:56.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.833+0000 7fbf37fff700 1 -- 172.21.15.17:0/519643711 >> v1:172.21.15.77:6789/0 conn(0x7fbf380afd60 legacy=0x7fbf38151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:56.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.833+0000 7fbf37fff700 1 -- 172.21.15.17:0/519643711 >> v1:172.21.15.17:6789/0 conn(0x7fbf380b2710 legacy=0x7fbf3814afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:56.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.833+0000 7fbf37fff700 1 -- 172.21.15.17:0/519643711 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf380a79c0 con 0x7fbf380b01a0 2022-08-01T19:39:56.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.833+0000 7fbf37fff700 1 --2- 172.21.15.17:0/519643711 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf380b01a0 0x7fbf380a8540 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7fbf28002f70 tx=0x7fbf28006df0).ready entity=mon.0 client_cookie=7a6a5fc2d1c23d2e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:56.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.833+0000 7fbf377fe700 1 -- 172.21.15.17:0/519643711 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf2800d610 con 0x7fbf380b01a0 2022-08-01T19:39:56.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.833+0000 7fbf377fe700 1 -- 172.21.15.17:0/519643711 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf28000e90 con 0x7fbf380b01a0 2022-08-01T19:39:56.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.833+0000 7fbf377fe700 1 -- 172.21.15.17:0/519643711 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf2800d960 con 0x7fbf380b01a0 2022-08-01T19:39:56.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.833+0000 7fbf3da76700 1 -- 172.21.15.17:0/519643711 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf380b01a0 msgr2=0x7fbf380a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:56.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.833+0000 7fbf3da76700 1 --2- 172.21.15.17:0/519643711 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf380b01a0 0x7fbf380a8540 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7fbf28002f70 tx=0x7fbf28006df0).stop 2022-08-01T19:39:56.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.833+0000 7fbf3da76700 1 -- 172.21.15.17:0/519643711 shutdown_connections 2022-08-01T19:39:56.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.833+0000 7fbf3da76700 1 --2- 172.21.15.17:0/519643711 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf380b01a0 0x7fbf380a8540 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:56.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.833+0000 7fbf3da76700 1 -- 172.21.15.17:0/519643711 >> 172.21.15.17:0/519643711 conn(0x7fbf38060310 msgr2=0x7fbf380b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:56.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.833+0000 7fbf3da76700 1 -- 172.21.15.17:0/519643711 shutdown_connections 2022-08-01T19:39:56.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.833+0000 7fbf3da76700 1 -- 172.21.15.17:0/519643711 wait complete. 2022-08-01T19:39:56.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.833+0000 7fbf3da76700 1 Processor -- start 2022-08-01T19:39:56.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.833+0000 7fbf3da76700 1 -- start start 2022-08-01T19:39:56.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.833+0000 7fbf3da76700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf380afd60 0x7fbf380aadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:56.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.833+0000 7fbf3da76700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbf380b01a0 0x7fbf380aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:56.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.833+0000 7fbf3da76700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf380b2710 0x7fbf38156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:56.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.833+0000 7fbf3da76700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf380aa3e0 con 0x7fbf380b2710 2022-08-01T19:39:56.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.833+0000 7fbf3da76700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf380aa680 con 0x7fbf380afd60 2022-08-01T19:39:56.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.833+0000 7fbf3da76700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf380aa920 con 0x7fbf380b01a0 2022-08-01T19:39:56.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.833+0000 7fbf3d275700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf380b2710 0x7fbf38156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:56.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.833+0000 7fbf3ca74700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf380afd60 0x7fbf380aadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:56.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.833+0000 7fbf3d275700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf380b2710 0x7fbf38156820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40696/0 (socket says 172.21.15.17:40696) 2022-08-01T19:39:56.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.833+0000 7fbf37fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbf380b01a0 0x7fbf380aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:56.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.833+0000 7fbf3ca74700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf380afd60 0x7fbf380aadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:38614/0 (socket says 172.21.15.17:38614) 2022-08-01T19:39:56.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.833+0000 7fbf3d275700 1 -- 172.21.15.17:0/307403267 learned_addr learned my addr 172.21.15.17:0/307403267 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:56.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.837+0000 7fbf37fff700 1 -- 172.21.15.17:0/307403267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf380afd60 msgr2=0x7fbf380aadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:56.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.837+0000 7fbf37fff700 1 --2- 172.21.15.17:0/307403267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf380afd60 0x7fbf380aadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:56.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.837+0000 7fbf37fff700 1 -- 172.21.15.17:0/307403267 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf380b2710 msgr2=0x7fbf38156820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:56.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.837+0000 7fbf37fff700 1 --2- 172.21.15.17:0/307403267 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf380b2710 0x7fbf38156820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:56.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.837+0000 7fbf37fff700 1 -- 172.21.15.17:0/307403267 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2800b180 con 0x7fbf380b01a0 2022-08-01T19:39:56.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.837+0000 7fbf3d275700 1 --2- 172.21.15.17:0/307403267 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf380b2710 0x7fbf38156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:39:56.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.837+0000 7fbf37fff700 1 --2- 172.21.15.17:0/307403267 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbf380b01a0 0x7fbf380aecb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fbf28002f70 tx=0x7fbf28015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:56.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.837+0000 7fbf35ffb700 1 -- 172.21.15.17:0/307403267 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf2800d990 con 0x7fbf380b01a0 2022-08-01T19:39:56.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.837+0000 7fbf35ffb700 1 -- 172.21.15.17:0/307403267 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf28011b40 con 0x7fbf380b01a0 2022-08-01T19:39:56.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.837+0000 7fbf3da76700 1 -- 172.21.15.17:0/307403267 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf38066a90 con 0x7fbf380b01a0 2022-08-01T19:39:56.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.837+0000 7fbf35ffb700 1 -- 172.21.15.17:0/307403267 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf28011ea0 con 0x7fbf380b01a0 2022-08-01T19:39:56.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.837+0000 7fbf3da76700 1 -- 172.21.15.17:0/307403267 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf380abf80 con 0x7fbf380b01a0 2022-08-01T19:39:56.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.837+0000 7fbf3da76700 1 -- 172.21.15.17:0/307403267 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf380574e0 con 0x7fbf380b01a0 2022-08-01T19:39:56.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.837+0000 7fbf35ffb700 1 -- 172.21.15.17:0/307403267 <== mon.2 v2:172.21.15.77:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbf2801a040 con 0x7fbf380b01a0 2022-08-01T19:39:56.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.837+0000 7fbf35ffb700 1 --2- 172.21.15.17:0/307403267 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbf18053b00 0x7fbf18055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:56.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.841+0000 7fbf35ffb700 1 -- 172.21.15.17:0/307403267 <== mon.2 v2:172.21.15.77:3301/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fbf2806dcd0 con 0x7fbf380b01a0 2022-08-01T19:39:56.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.841+0000 7fbf3ca74700 1 --2- 172.21.15.17:0/307403267 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbf18053b00 0x7fbf18055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:56.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.841+0000 7fbf3ca74700 1 --2- 172.21.15.17:0/307403267 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbf18053b00 0x7fbf18055fc0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7fbf2c00d000 tx=0x7fbf2c007be0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:56.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:56.849+0000 7fbf35ffb700 1 -- 172.21.15.17:0/307403267 <== mon.2 v2:172.21.15.77:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbf28044980 con 0x7fbf380b01a0 2022-08-01T19:39:57.014 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:57.013+0000 7fbf3da76700 1 -- 172.21.15.17:0/307403267 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf38069a70 con 0x7fbf380b01a0 2022-08-01T19:39:57.014 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:57.013+0000 7fbf35ffb700 1 -- 172.21.15.17:0/307403267 <== mon.2 v2:172.21.15.77:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+2493 (secure 0 0 0) 0x7fbf28047fa0 con 0x7fbf380b01a0 2022-08-01T19:39:57.015 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:39:57.015 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":35,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5615,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/2284778179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2284778179},{"type":"v1","addr":"172.21.15.77:6835","nonce":2284778179}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":34},{"gid":6261,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/1827111044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":1827111044},{"type":"v1","addr":"172.21.15.17:6833","nonce":1827111044}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":33}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:39:52.325506+0000","modified":"2022-08-01T19:39:53.328610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6267":{"gid":6267,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/3370152344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":3370152344},{"type":"v1","addr":"172.21.15.17:6835","nonce":3370152344}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-08-01T19:39:57.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:57.017+0000 7fbf3da76700 1 -- 172.21.15.17:0/307403267 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbf18053b00 msgr2=0x7fbf18055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:57.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:57.017+0000 7fbf3da76700 1 --2- 172.21.15.17:0/307403267 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbf18053b00 0x7fbf18055fc0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7fbf2c00d000 tx=0x7fbf2c007be0).stop 2022-08-01T19:39:57.018 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:57.017+0000 7fbf3da76700 1 -- 172.21.15.17:0/307403267 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbf380b01a0 msgr2=0x7fbf380aecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:57.018 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:57.017+0000 7fbf3da76700 1 --2- 172.21.15.17:0/307403267 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbf380b01a0 0x7fbf380aecb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fbf28002f70 tx=0x7fbf28015000).stop 2022-08-01T19:39:57.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:57.017+0000 7fbf3da76700 1 -- 172.21.15.17:0/307403267 shutdown_connections 2022-08-01T19:39:57.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:57.017+0000 7fbf3da76700 1 --2- 172.21.15.17:0/307403267 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbf18053b00 0x7fbf18055fc0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:57.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:57.017+0000 7fbf3da76700 1 --2- 172.21.15.17:0/307403267 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf380b2710 0x7fbf38156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:57.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:57.017+0000 7fbf3da76700 1 --2- 172.21.15.17:0/307403267 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbf380b01a0 0x7fbf380aecb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:57.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:57.017+0000 7fbf3da76700 1 --2- 172.21.15.17:0/307403267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf380afd60 0x7fbf380aadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:57.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:57.017+0000 7fbf3da76700 1 -- 172.21.15.17:0/307403267 >> 172.21.15.17:0/307403267 conn(0x7fbf38060310 msgr2=0x7fbf3805be30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:57.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:57.017+0000 7fbf3da76700 1 -- 172.21.15.17:0/307403267 shutdown_connections 2022-08-01T19:39:57.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:57.017+0000 7fbf3da76700 1 -- 172.21.15.17:0/307403267 wait complete. 2022-08-01T19:39:57.023 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 35 2022-08-01T19:39:57.340 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:39:52.325506+0000', 'modified': '2022-08-01T19:39:53.328610+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6267}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6267': {'gid': 6267, 'name': 'a', 'rank': 0, 'incarnation': 34, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/3370152344', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 3370152344}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 3370152344}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:39:57.340 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:39:57.341 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-08-01T19:39:57.459 INFO:teuthology.orchestra.run.smithi017.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":6267,"want_state":"up:active","state":"up:active","rank_uptime":5.1223515319999997,"mdsmap_epoch":35,"osdmap_epoch":76,"osdmap_epoch_barrier":75,"uptime":9.4391044070000003} 2022-08-01T19:39:57.459 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 6267, "mdsmap_epoch": 35, "osdmap_epoch": 76, "osdmap_epoch_barrier": 75, "rank_uptime": 5.122351532, "state": "up:active", "uptime": 9.439104407, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:39:57.460 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T19:39:57.460 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-08-01T19:39:57.461 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi085.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-08-01T19:39:57.461 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:39:57.461 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:39:57.473 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:39:57.533 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:39:57.534 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:39:57.547 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:39:57.548 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:39:57.550 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:39:57.551 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:39:57.558 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T19:39:57.559 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:39:57.602 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:39:57.620 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:39:57.621 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:39:57.632 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:39:57.633 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:39:57.634 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:39:57.635 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:39:57.677 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:2022-08-01T19:39:57.676+0000 7f4c63e70080 -1 init, newargv = 0x55e2ebb2f0d0 newargc=9ceph-fuse[23029]: starting ceph client 2022-08-01T19:39:57.677 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr: 2022-08-01T19:39:57.687 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:ceph-fuse[23029]: starting fuse 2022-08-01T19:39:58.789 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:39:58.809 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:39:58.810 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:39:58.837 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:39:58.839 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:39:58.839 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:39:58.840 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:39:58.896 INFO:teuthology.orchestra.run.smithi085.stdout:52 2022-08-01T19:39:58.897 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T19:39:58.897 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:39:58.898 DEBUG:teuthology.orchestra.run.smithi085:> sudo python3 -c ' 2022-08-01T19:39:58.898 DEBUG:teuthology.orchestra.run.smithi085:> import glob 2022-08-01T19:39:58.907 DEBUG:teuthology.orchestra.run.smithi085:> import re 2022-08-01T19:39:58.907 DEBUG:teuthology.orchestra.run.smithi085:> import os 2022-08-01T19:39:58.908 DEBUG:teuthology.orchestra.run.smithi085:> import subprocess 2022-08-01T19:39:58.908 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:39:58.909 DEBUG:teuthology.orchestra.run.smithi085:> def find_socket(client_name): 2022-08-01T19:39:58.913 DEBUG:teuthology.orchestra.run.smithi085:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-08-01T19:39:58.914 DEBUG:teuthology.orchestra.run.smithi085:> files = glob.glob(asok_path) 2022-08-01T19:39:58.915 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:39:58.915 DEBUG:teuthology.orchestra.run.smithi085:> # Given a non-glob path, it better be there 2022-08-01T19:39:58.916 DEBUG:teuthology.orchestra.run.smithi085:> if "*" not in asok_path: 2022-08-01T19:39:58.917 DEBUG:teuthology.orchestra.run.smithi085:> assert(len(files) == 1) 2022-08-01T19:39:58.918 DEBUG:teuthology.orchestra.run.smithi085:> return files[0] 2022-08-01T19:39:58.919 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:39:58.920 DEBUG:teuthology.orchestra.run.smithi085:> for f in files: 2022-08-01T19:39:58.921 DEBUG:teuthology.orchestra.run.smithi085:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T19:39:58.922 DEBUG:teuthology.orchestra.run.smithi085:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T19:39:58.923 DEBUG:teuthology.orchestra.run.smithi085:> return f 2022-08-01T19:39:58.924 DEBUG:teuthology.orchestra.run.smithi085:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T19:39:58.924 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:39:58.925 DEBUG:teuthology.orchestra.run.smithi085:> print(find_socket("client.0")) 2022-08-01T19:39:58.926 DEBUG:teuthology.orchestra.run.smithi085:> ' 2022-08-01T19:39:58.974 INFO:teuthology.orchestra.run.smithi085.stdout:/var/run/ceph/ceph-client.0.23029.asok 2022-08-01T19:39:58.977 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.23029.asok 2022-08-01T19:39:58.979 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:39:58.979 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.23029.asok status 2022-08-01T19:39:59.114 INFO:teuthology.orchestra.run.smithi085.stdout:{ 2022-08-01T19:39:59.115 INFO:teuthology.orchestra.run.smithi085.stdout: "metadata": { 2022-08-01T19:39:59.115 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T19:39:59.115 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T19:39:59.116 INFO:teuthology.orchestra.run.smithi085.stdout: "entity_id": "0", 2022-08-01T19:39:59.116 INFO:teuthology.orchestra.run.smithi085.stdout: "hostname": "smithi085", 2022-08-01T19:39:59.117 INFO:teuthology.orchestra.run.smithi085.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-08-01T19:39:59.117 INFO:teuthology.orchestra.run.smithi085.stdout: "pid": "23029", 2022-08-01T19:39:59.118 INFO:teuthology.orchestra.run.smithi085.stdout: "root": "/" 2022-08-01T19:39:59.118 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:39:59.118 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_count": 0, 2022-08-01T19:39:59.119 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_pinned_count": 0, 2022-08-01T19:39:59.119 INFO:teuthology.orchestra.run.smithi085.stdout: "id": 5744, 2022-08-01T19:39:59.119 INFO:teuthology.orchestra.run.smithi085.stdout: "inst": { 2022-08-01T19:39:59.120 INFO:teuthology.orchestra.run.smithi085.stdout: "name": { 2022-08-01T19:39:59.120 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "client", 2022-08-01T19:39:59.121 INFO:teuthology.orchestra.run.smithi085.stdout: "num": 5744 2022-08-01T19:39:59.121 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:39:59.121 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2022-08-01T19:39:59.122 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2022-08-01T19:39:59.122 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2022-08-01T19:39:59.123 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 3805297491 2022-08-01T19:39:59.123 INFO:teuthology.orchestra.run.smithi085.stdout: } 2022-08-01T19:39:59.124 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:39:59.124 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2022-08-01T19:39:59.125 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2022-08-01T19:39:59.125 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2022-08-01T19:39:59.125 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 3805297491 2022-08-01T19:39:59.128 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:39:59.128 INFO:teuthology.orchestra.run.smithi085.stdout: "inst_str": "client.5744 172.21.15.85:0/3805297491", 2022-08-01T19:39:59.129 INFO:teuthology.orchestra.run.smithi085.stdout: "addr_str": "172.21.15.85:0/3805297491", 2022-08-01T19:39:59.130 INFO:teuthology.orchestra.run.smithi085.stdout: "inode_count": 1, 2022-08-01T19:39:59.130 INFO:teuthology.orchestra.run.smithi085.stdout: "mds_epoch": 35, 2022-08-01T19:39:59.131 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch": 76, 2022-08-01T19:39:59.131 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch_barrier": 0, 2022-08-01T19:39:59.132 INFO:teuthology.orchestra.run.smithi085.stdout: "blacklisted": false, 2022-08-01T19:39:59.132 INFO:teuthology.orchestra.run.smithi085.stdout: "fs_name": "cephfs" 2022-08-01T19:39:59.132 INFO:teuthology.orchestra.run.smithi085.stdout:} 2022-08-01T19:39:59.135 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:39:59.136 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:39:59.137 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:39:59.156 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:39:59.157 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:39:59.158 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:39:59.159 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:39:59.180 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:39:59.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.289+0000 7f5e8ffff700 1 Processor -- start 2022-08-01T19:39:59.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.289+0000 7f5e8ffff700 1 -- start start 2022-08-01T19:39:59.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.289+0000 7f5e8ffff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5e900b08a0 0x7f5e90142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:59.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.289+0000 7f5e8ffff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f5e900ad000 0x7f5e9013fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:59.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.289+0000 7f5e8ffff700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e90143ab0 con 0x7f5e900b3260 2022-08-01T19:39:59.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.289+0000 7f5e8ffff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e900b0c80 con 0x7f5e900ad000 2022-08-01T19:39:59.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.289+0000 7f5e8ffff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e900ae810 con 0x7f5e900b08a0 2022-08-01T19:39:59.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.289+0000 7f5e8e7fc700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5e900b3260 0x7f5e901433a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:32894/0 (socket says 172.21.15.17:32894) 2022-08-01T19:39:59.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.289+0000 7f5e8e7fc700 1 -- 172.21.15.17:0/2209003445 learned_addr learned my addr 172.21.15.17:0/2209003445 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:39:59.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.289+0000 7f5e8effd700 1 --2- 172.21.15.17:0/2209003445 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5e900b08a0 0x7f5e90142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:59.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.289+0000 7f5e8f7fe700 1 --2- 172.21.15.17:0/2209003445 >> v2:172.21.15.77:3300/0 conn(0x7f5e900ad000 0x7f5e9013fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:59.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.289+0000 7f5e8dffb700 1 -- 172.21.15.17:0/2209003445 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 280926806 0 0) 0x7f5e90143ab0 con 0x7f5e900b3260 2022-08-01T19:39:59.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.289+0000 7f5e8dffb700 1 -- 172.21.15.17:0/2209003445 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e74003260 con 0x7f5e900b3260 2022-08-01T19:39:59.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.293+0000 7f5e8dffb700 1 -- 172.21.15.17:0/2209003445 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1581226925 0 0) 0x7f5e74003260 con 0x7f5e900b3260 2022-08-01T19:39:59.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.293+0000 7f5e8dffb700 1 -- 172.21.15.17:0/2209003445 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5e900b08a0 msgr2=0x7f5e90142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:59.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.293+0000 7f5e8dffb700 1 --2- 172.21.15.17:0/2209003445 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5e900b08a0 0x7f5e90142eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:59.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.293+0000 7f5e8dffb700 1 -- 172.21.15.17:0/2209003445 >> v2:172.21.15.77:3300/0 conn(0x7f5e900ad000 msgr2=0x7f5e9013fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:59.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.293+0000 7f5e8dffb700 1 --2- 172.21.15.17:0/2209003445 >> v2:172.21.15.77:3300/0 conn(0x7f5e900ad000 0x7f5e9013fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:59.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.293+0000 7f5e8dffb700 1 -- 172.21.15.17:0/2209003445 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e900aeab0 con 0x7f5e900b3260 2022-08-01T19:39:59.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.293+0000 7f5e8dffb700 1 -- 172.21.15.17:0/2209003445 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7f5e80001660 con 0x7f5e900b3260 2022-08-01T19:39:59.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.293+0000 7f5e8dffb700 1 -- 172.21.15.17:0/2209003445 >> v1:172.21.15.17:6789/0 conn(0x7f5e900b3260 legacy=0x7f5e901433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:59.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.293+0000 7f5e8dffb700 1 --2- 172.21.15.17:0/2209003445 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5e900b08a0 0x7f5e74003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:59.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.293+0000 7f5e8dffb700 1 --2- 172.21.15.17:0/2209003445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e74003c80 0x7f5e74004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:59.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.293+0000 7f5e8dffb700 1 --2- 172.21.15.17:0/2209003445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e740049f0 0x7f5e74004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:59.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.293+0000 7f5e8dffb700 1 -- 172.21.15.17:0/2209003445 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e900b0c80 con 0x7f5e740049f0 2022-08-01T19:39:59.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.293+0000 7f5e8dffb700 1 -- 172.21.15.17:0/2209003445 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e900ae810 con 0x7f5e74003c80 2022-08-01T19:39:59.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.293+0000 7f5e8dffb700 1 -- 172.21.15.17:0/2209003445 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7400b750 con 0x7f5e900b08a0 2022-08-01T19:39:59.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.293+0000 7f5e8f7fe700 1 --2- 172.21.15.17:0/2209003445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e740049f0 0x7f5e74004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:59.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.293+0000 7f5e8effd700 1 --2- 172.21.15.17:0/2209003445 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5e900b08a0 0x7f5e74003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:59.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.293+0000 7f5e8e7fc700 1 --2- 172.21.15.17:0/2209003445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e74003c80 0x7f5e74004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:59.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.293+0000 7f5e8f7fe700 1 -- 172.21.15.17:0/2209003445 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5e900b08a0 msgr2=0x7f5e74003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:59.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.293+0000 7f5e8f7fe700 1 --2- 172.21.15.17:0/2209003445 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5e900b08a0 0x7f5e74003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:59.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.293+0000 7f5e8f7fe700 1 -- 172.21.15.17:0/2209003445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e74003c80 msgr2=0x7f5e74004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:59.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.293+0000 7f5e8f7fe700 1 --2- 172.21.15.17:0/2209003445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e74003c80 0x7f5e74004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:59.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.293+0000 7f5e8f7fe700 1 -- 172.21.15.17:0/2209003445 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e74003260 con 0x7f5e740049f0 2022-08-01T19:39:59.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.293+0000 7f5e8f7fe700 1 --2- 172.21.15.17:0/2209003445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e740049f0 0x7f5e74004e70 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7f5e740054d0 tx=0x7f5e880091d0).ready entity=mon.0 client_cookie=a560083071f5e658 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.293+0000 7f5e8dffb700 1 -- 172.21.15.17:0/2209003445 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e8800e710 con 0x7f5e740049f0 2022-08-01T19:39:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.293+0000 7f5e8dffb700 1 -- 172.21.15.17:0/2209003445 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e8800e870 con 0x7f5e740049f0 2022-08-01T19:39:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.293+0000 7f5e8dffb700 1 -- 172.21.15.17:0/2209003445 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e8800ebd0 con 0x7f5e740049f0 2022-08-01T19:39:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.293+0000 7f5e8ffff700 1 -- 172.21.15.17:0/2209003445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e740049f0 msgr2=0x7f5e74004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.293+0000 7f5e8ffff700 1 --2- 172.21.15.17:0/2209003445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e740049f0 0x7f5e74004e70 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7f5e740054d0 tx=0x7f5e880091d0).stop 2022-08-01T19:39:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.293+0000 7f5e8f7fe700 1 -- 172.21.15.17:0/2209003445 reap_dead start 2022-08-01T19:39:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.293+0000 7f5e8ffff700 1 -- 172.21.15.17:0/2209003445 shutdown_connections 2022-08-01T19:39:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.293+0000 7f5e8ffff700 1 -- 172.21.15.17:0/2209003445 >> 172.21.15.17:0/2209003445 conn(0x7f5e900612d0 msgr2=0x7f5e90062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:59.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.293+0000 7f5e8ffff700 1 -- 172.21.15.17:0/2209003445 shutdown_connections 2022-08-01T19:39:59.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.293+0000 7f5e8ffff700 1 -- 172.21.15.17:0/2209003445 wait complete. 2022-08-01T19:39:59.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.297+0000 7f5e8ffff700 1 Processor -- start 2022-08-01T19:39:59.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.297+0000 7f5e8ffff700 1 -- start start 2022-08-01T19:39:59.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.297+0000 7f5e8ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e740049f0 0x7f5e900b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:59.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.297+0000 7f5e8ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e900ab9a0 0x7f5e900abde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:59.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.297+0000 7f5e8ffff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5e900a8860 0x7f5e900ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:59.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.297+0000 7f5e8ffff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e901407a0 con 0x7f5e900ab9a0 2022-08-01T19:39:59.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.297+0000 7f5e8ffff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e900aad10 con 0x7f5e740049f0 2022-08-01T19:39:59.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.297+0000 7f5e8ffff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e900aaa70 con 0x7f5e900a8860 2022-08-01T19:39:59.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.297+0000 7f5e8e7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e900ab9a0 0x7f5e900abde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:59.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.297+0000 7f5e8effd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e740049f0 0x7f5e900b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:59.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.297+0000 7f5e8f7fe700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5e900a8860 0x7f5e900ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:59.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.297+0000 7f5e8effd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e740049f0 0x7f5e900b1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:38634/0 (socket says 172.21.15.17:38634) 2022-08-01T19:39:59.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.297+0000 7f5e8effd700 1 -- 172.21.15.17:0/2648083872 learned_addr learned my addr 172.21.15.17:0/2648083872 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:59.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.297+0000 7f5e8f7fe700 1 -- 172.21.15.17:0/2648083872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e740049f0 msgr2=0x7f5e900b1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:59.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.297+0000 7f5e8f7fe700 1 --2- 172.21.15.17:0/2648083872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e740049f0 0x7f5e900b1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:59.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.297+0000 7f5e8f7fe700 1 -- 172.21.15.17:0/2648083872 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e900ab9a0 msgr2=0x7f5e900abde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:59.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.297+0000 7f5e8f7fe700 1 --2- 172.21.15.17:0/2648083872 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e900ab9a0 0x7f5e900abde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:59.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.297+0000 7f5e8f7fe700 1 -- 172.21.15.17:0/2648083872 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e900ab500 con 0x7f5e900a8860 2022-08-01T19:39:59.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.297+0000 7f5e8effd700 1 --2- 172.21.15.17:0/2648083872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e740049f0 0x7f5e900b1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:39:59.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.297+0000 7f5e8f7fe700 1 --2- 172.21.15.17:0/2648083872 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5e900a8860 0x7f5e900ac2d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f5e800017c0 tx=0x7f5e8801a000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:59.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.297+0000 7f5e7ffff700 1 -- 172.21.15.17:0/2648083872 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e8800e710 con 0x7f5e900a8860 2022-08-01T19:39:59.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.297+0000 7f5e8ffff700 1 -- 172.21.15.17:0/2648083872 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e900b24c0 con 0x7f5e900a8860 2022-08-01T19:39:59.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.297+0000 7f5e7ffff700 1 -- 172.21.15.17:0/2648083872 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e8801b850 con 0x7f5e900a8860 2022-08-01T19:39:59.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.297+0000 7f5e7ffff700 1 -- 172.21.15.17:0/2648083872 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e8801bbb0 con 0x7f5e900a8860 2022-08-01T19:39:59.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.297+0000 7f5e8ffff700 1 -- 172.21.15.17:0/2648083872 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e90135990 con 0x7f5e900a8860 2022-08-01T19:39:59.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.301+0000 7f5e8ffff700 1 -- 172.21.15.17:0/2648083872 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e900574e0 con 0x7f5e900a8860 2022-08-01T19:39:59.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.301+0000 7f5e7ffff700 1 -- 172.21.15.17:0/2648083872 <== mon.2 v2:172.21.15.77:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5e8801bd10 con 0x7f5e900a8860 2022-08-01T19:39:59.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.301+0000 7f5e7ffff700 1 --2- 172.21.15.17:0/2648083872 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5e6c0539e0 0x7f5e6c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:59.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.301+0000 7f5e7ffff700 1 -- 172.21.15.17:0/2648083872 <== mon.2 v2:172.21.15.77:3301/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5e8806dc90 con 0x7f5e900a8860 2022-08-01T19:39:59.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.301+0000 7f5e8effd700 1 --2- 172.21.15.17:0/2648083872 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5e6c0539e0 0x7f5e6c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:59.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.301+0000 7f5e8effd700 1 --2- 172.21.15.17:0/2648083872 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5e6c0539e0 0x7f5e6c055ea0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f5e8400b000 tx=0x7f5e84006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:59.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.309+0000 7f5e7ffff700 1 -- 172.21.15.17:0/2648083872 <== mon.2 v2:172.21.15.77:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5e880449b0 con 0x7f5e900a8860 2022-08-01T19:39:59.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.445+0000 7f5e8ffff700 1 -- 172.21.15.17:0/2648083872 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e90069a70 con 0x7f5e900a8860 2022-08-01T19:39:59.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.445+0000 7f5e7ffff700 1 -- 172.21.15.17:0/2648083872 <== mon.2 v2:172.21.15.77:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f5e88016090 con 0x7f5e900a8860 2022-08-01T19:39:59.447 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:39:59.448 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":76,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:39:53.329285+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:39:50.853967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2022-08-01T19:39:51.901692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:39:59.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.449+0000 7f5e8ffff700 1 -- 172.21.15.17:0/2648083872 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5e6c0539e0 msgr2=0x7f5e6c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:59.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.449+0000 7f5e8ffff700 1 --2- 172.21.15.17:0/2648083872 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5e6c0539e0 0x7f5e6c055ea0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f5e8400b000 tx=0x7f5e84006df0).stop 2022-08-01T19:39:59.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.449+0000 7f5e8ffff700 1 -- 172.21.15.17:0/2648083872 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5e900a8860 msgr2=0x7f5e900ac2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:59.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.449+0000 7f5e8ffff700 1 --2- 172.21.15.17:0/2648083872 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5e900a8860 0x7f5e900ac2d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f5e800017c0 tx=0x7f5e8801a000).stop 2022-08-01T19:39:59.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.449+0000 7f5e8ffff700 1 -- 172.21.15.17:0/2648083872 shutdown_connections 2022-08-01T19:39:59.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.449+0000 7f5e8ffff700 1 --2- 172.21.15.17:0/2648083872 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5e6c0539e0 0x7f5e6c055ea0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:59.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.449+0000 7f5e8ffff700 1 --2- 172.21.15.17:0/2648083872 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5e900a8860 0x7f5e900ac2d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:59.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.449+0000 7f5e8ffff700 1 --2- 172.21.15.17:0/2648083872 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e900ab9a0 0x7f5e900abde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:59.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.449+0000 7f5e8ffff700 1 --2- 172.21.15.17:0/2648083872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e740049f0 0x7f5e900b1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:59.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.449+0000 7f5e8ffff700 1 -- 172.21.15.17:0/2648083872 >> 172.21.15.17:0/2648083872 conn(0x7f5e900612d0 msgr2=0x7f5e90140a40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:59.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.449+0000 7f5e8ffff700 1 -- 172.21.15.17:0/2648083872 shutdown_connections 2022-08-01T19:39:59.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.449+0000 7f5e8ffff700 1 -- 172.21.15.17:0/2648083872 wait complete. 2022-08-01T19:39:59.467 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T19:39:50.853967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T19:39:51.901692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T19:39:59.468 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:39:59.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.581+0000 7f35ac647700 1 Processor -- start 2022-08-01T19:39:59.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.581+0000 7f35ac647700 1 -- start start 2022-08-01T19:39:59.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.581+0000 7f35ac647700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35a40b4310 0x7f35a4140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:59.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.581+0000 7f35ac647700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f35a40b4820 0x7f35a40b5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:59.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.581+0000 7f35ac647700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35a4143cf0 con 0x7f35a4146ae0 2022-08-01T19:39:59.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.581+0000 7f35ac647700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35a41476a0 con 0x7f35a40b4820 2022-08-01T19:39:59.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.581+0000 7f35ac647700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a4143f90 con 0x7f35a40b4310 2022-08-01T19:39:59.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.581+0000 7f35a9be2700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35a40b4310 0x7f35a4140ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:59.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.581+0000 7f35a9be2700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35a40b4310 0x7f35a4140ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:36822/0 (socket says 172.21.15.17:36822) 2022-08-01T19:39:59.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.581+0000 7f35a9be2700 1 -- 172.21.15.17:0/1215356782 learned_addr learned my addr 172.21.15.17:0/1215356782 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:59.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.581+0000 7f35aa3e3700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7f35a4146ae0 0x7f35a41403c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:49714/0 (socket says 172.21.15.17:49714) 2022-08-01T19:39:59.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.581+0000 7f35aabe4700 1 --2- 172.21.15.17:0/1215356782 >> v2:172.21.15.77:3300/0 conn(0x7f35a40b4820 0x7f35a40b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:59.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.581+0000 7f35a9be2700 1 -- 172.21.15.17:0/1215356782 >> v2:172.21.15.77:3300/0 conn(0x7f35a40b4820 msgr2=0x7f35a40b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:59.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.581+0000 7f35a9be2700 1 --2- 172.21.15.17:0/1215356782 >> v2:172.21.15.77:3300/0 conn(0x7f35a40b4820 0x7f35a40b5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:59.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.581+0000 7f35a9be2700 1 -- 172.21.15.17:0/1215356782 >> v1:172.21.15.77:6789/0 conn(0x7f35a4146ae0 legacy=0x7f35a41403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:59.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.581+0000 7f35a9be2700 1 -- 172.21.15.17:0/1215356782 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a40b0e20 con 0x7f35a40b4310 2022-08-01T19:39:59.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.581+0000 7f35a9be2700 1 --2- 172.21.15.17:0/1215356782 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35a40b4310 0x7f35a4140ad0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f3594002f70 tx=0x7f3594006eb0).ready entity=mon.2 client_cookie=9ec1a28f7a2ada5e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:59.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.581+0000 7f35a93e1700 1 -- 172.21.15.17:0/1215356782 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f359400d5c0 con 0x7f35a40b4310 2022-08-01T19:39:59.602 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.581+0000 7f35a93e1700 1 -- 172.21.15.17:0/1215356782 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3594000e90 con 0x7f35a40b4310 2022-08-01T19:39:59.602 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.581+0000 7f35a93e1700 1 -- 172.21.15.17:0/1215356782 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f359400d950 con 0x7f35a40b4310 2022-08-01T19:39:59.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.581+0000 7f35ac647700 1 -- 172.21.15.17:0/1215356782 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35a40b4310 msgr2=0x7f35a4140ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:59.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.581+0000 7f35ac647700 1 --2- 172.21.15.17:0/1215356782 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35a40b4310 0x7f35a4140ad0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f3594002f70 tx=0x7f3594006eb0).stop 2022-08-01T19:39:59.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.585+0000 7f35ac647700 1 -- 172.21.15.17:0/1215356782 shutdown_connections 2022-08-01T19:39:59.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.585+0000 7f35ac647700 1 --2- 172.21.15.17:0/1215356782 >> v2:172.21.15.77:3300/0 conn(0x7f35a40b4820 0x7f35a40b5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:59.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.585+0000 7f35ac647700 1 --2- 172.21.15.17:0/1215356782 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35a40b4310 0x7f35a4140ad0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:59.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.585+0000 7f35ac647700 1 -- 172.21.15.17:0/1215356782 >> 172.21.15.17:0/1215356782 conn(0x7f35a405c620 msgr2=0x7f35a405ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:59.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.585+0000 7f35ac647700 1 -- 172.21.15.17:0/1215356782 shutdown_connections 2022-08-01T19:39:59.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.585+0000 7f35ac647700 1 -- 172.21.15.17:0/1215356782 wait complete. 2022-08-01T19:39:59.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.585+0000 7f35ac647700 1 Processor -- start 2022-08-01T19:39:59.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.585+0000 7f35ac647700 1 -- start start 2022-08-01T19:39:59.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.585+0000 7f35ac647700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35a40b4310 0x7f35a4142000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:59.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.585+0000 7f35ac647700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35a40b4820 0x7f35a40ae860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:59.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.585+0000 7f35ac647700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35a4146ae0 0x7f35a4145d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:59.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.585+0000 7f35ac647700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a40aed50 con 0x7f35a40b4820 2022-08-01T19:39:59.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.585+0000 7f35ac647700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a40aefe0 con 0x7f35a4146ae0 2022-08-01T19:39:59.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.585+0000 7f35ac647700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a40af280 con 0x7f35a40b4310 2022-08-01T19:39:59.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.585+0000 7f35a9be2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35a40b4820 0x7f35a40ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:59.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.585+0000 7f35a9be2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35a40b4820 0x7f35a40ae860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40728/0 (socket says 172.21.15.17:40728) 2022-08-01T19:39:59.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.585+0000 7f35a9be2700 1 -- 172.21.15.17:0/47587325 learned_addr learned my addr 172.21.15.17:0/47587325 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:59.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.585+0000 7f35aa3e3700 1 --2- 172.21.15.17:0/47587325 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35a40b4310 0x7f35a4142000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:59.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.585+0000 7f35aabe4700 1 --2- 172.21.15.17:0/47587325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35a4146ae0 0x7f35a4145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:59.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.589+0000 7f35a9be2700 1 -- 172.21.15.17:0/47587325 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35a40b4310 msgr2=0x7f35a4142000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:59.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.589+0000 7f35a9be2700 1 --2- 172.21.15.17:0/47587325 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35a40b4310 0x7f35a4142000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:59.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.589+0000 7f35a9be2700 1 -- 172.21.15.17:0/47587325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35a4146ae0 msgr2=0x7f35a4145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:59.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.589+0000 7f35a9be2700 1 --2- 172.21.15.17:0/47587325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35a4146ae0 0x7f35a4145d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:59.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.589+0000 7f35a9be2700 1 -- 172.21.15.17:0/47587325 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359400b1e0 con 0x7f35a40b4820 2022-08-01T19:39:59.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.589+0000 7f35a9be2700 1 --2- 172.21.15.17:0/47587325 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35a40b4820 0x7f35a40ae860 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7f3594002f70 tx=0x7f3594015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:59.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.589+0000 7f35937fe700 1 -- 172.21.15.17:0/47587325 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f359400d940 con 0x7f35a40b4820 2022-08-01T19:39:59.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.589+0000 7f35937fe700 1 -- 172.21.15.17:0/47587325 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3594011b80 con 0x7f35a40b4820 2022-08-01T19:39:59.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.589+0000 7f35937fe700 1 -- 172.21.15.17:0/47587325 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f359401e460 con 0x7f35a40b4820 2022-08-01T19:39:59.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.589+0000 7f35ac647700 1 -- 172.21.15.17:0/47587325 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35a4152a80 con 0x7f35a40b4820 2022-08-01T19:39:59.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.589+0000 7f35ac647700 1 -- 172.21.15.17:0/47587325 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35a4146200 con 0x7f35a40b4820 2022-08-01T19:39:59.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.589+0000 7f35ac647700 1 -- 172.21.15.17:0/47587325 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35a40b8a60 con 0x7f35a40b4820 2022-08-01T19:39:59.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.589+0000 7f35937fe700 1 -- 172.21.15.17:0/47587325 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f359401a040 con 0x7f35a40b4820 2022-08-01T19:39:59.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.589+0000 7f35937fe700 1 --2- 172.21.15.17:0/47587325 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3584053a10 0x7f3584055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:59.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.589+0000 7f35937fe700 1 -- 172.21.15.17:0/47587325 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f359406cee0 con 0x7f35a40b4820 2022-08-01T19:39:59.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.589+0000 7f35aa3e3700 1 --2- 172.21.15.17:0/47587325 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3584053a10 0x7f3584055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:59.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.593+0000 7f35aa3e3700 1 --2- 172.21.15.17:0/47587325 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3584053a10 0x7f3584055ed0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f35a000d000 tx=0x7f35a0007b80).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:59.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.597+0000 7f35937fe700 1 -- 172.21.15.17:0/47587325 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3594043bd0 con 0x7f35a40b4820 2022-08-01T19:39:59.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.717+0000 7f35ac647700 1 -- 172.21.15.17:0/47587325 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f35a4152c40 con 0x7f35a40b4820 2022-08-01T19:39:59.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.721+0000 7f35937fe700 1 -- 172.21.15.17:0/47587325 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v76) v1 ==== 110+0+10 (secure 0 0 0) 0x7f3594023020 con 0x7f35a40b4820 2022-08-01T19:39:59.721 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:39:59.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.721+0000 7f35ac647700 1 -- 172.21.15.17:0/47587325 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3584053a10 msgr2=0x7f3584055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:59.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.721+0000 7f35ac647700 1 --2- 172.21.15.17:0/47587325 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3584053a10 0x7f3584055ed0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f35a000d000 tx=0x7f35a0007b80).stop 2022-08-01T19:39:59.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.721+0000 7f35ac647700 1 -- 172.21.15.17:0/47587325 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35a40b4820 msgr2=0x7f35a40ae860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:59.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.721+0000 7f35ac647700 1 --2- 172.21.15.17:0/47587325 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35a40b4820 0x7f35a40ae860 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7f3594002f70 tx=0x7f3594015000).stop 2022-08-01T19:39:59.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.721+0000 7f35ac647700 1 -- 172.21.15.17:0/47587325 shutdown_connections 2022-08-01T19:39:59.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.721+0000 7f35ac647700 1 --2- 172.21.15.17:0/47587325 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3584053a10 0x7f3584055ed0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:59.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.721+0000 7f35ac647700 1 --2- 172.21.15.17:0/47587325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35a4146ae0 0x7f35a4145d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:59.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.721+0000 7f35ac647700 1 --2- 172.21.15.17:0/47587325 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35a40b4820 0x7f35a40ae860 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:59.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.721+0000 7f35ac647700 1 --2- 172.21.15.17:0/47587325 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35a40b4310 0x7f35a4142000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:59.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.721+0000 7f35ac647700 1 -- 172.21.15.17:0/47587325 >> 172.21.15.17:0/47587325 conn(0x7f35a405c620 msgr2=0x7f35a4144380 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:59.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.721+0000 7f35ac647700 1 -- 172.21.15.17:0/47587325 shutdown_connections 2022-08-01T19:39:59.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.721+0000 7f35ac647700 1 -- 172.21.15.17:0/47587325 wait complete. 2022-08-01T19:39:59.735 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:39:59.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.841+0000 7ff24ae7b700 1 Processor -- start 2022-08-01T19:39:59.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.841+0000 7ff24ae7b700 1 -- start start 2022-08-01T19:39:59.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.841+0000 7ff24ae7b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff2440b1580 0x7ff2440b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:59.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.841+0000 7ff24ae7b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff2440ae0b0 0x7ff24414b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:59.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.841+0000 7ff24ae7b700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2440b1960 con 0x7ff24405f840 2022-08-01T19:39:59.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.841+0000 7ff24ae7b700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2440b3480 con 0x7ff2440ae0b0 2022-08-01T19:39:59.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.841+0000 7ff24ae7b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2440b3720 con 0x7ff2440b1580 2022-08-01T19:39:59.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.841+0000 7ff243fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff2440ae0b0 0x7ff24414b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:59.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.841+0000 7ff243fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff2440ae0b0 0x7ff24414b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40738/0 (socket says 172.21.15.17:40738) 2022-08-01T19:39:59.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.841+0000 7ff243fff700 1 -- 172.21.15.17:0/105844793 learned_addr learned my addr 172.21.15.17:0/105844793 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:59.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.841+0000 7ff248c17700 1 --2- 172.21.15.17:0/105844793 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff2440b1580 0x7ff2440b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:59.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.841+0000 7ff243fff700 1 -- 172.21.15.17:0/105844793 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff2440b1580 msgr2=0x7ff2440b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:39:59.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.841+0000 7ff243fff700 1 --2- 172.21.15.17:0/105844793 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff2440b1580 0x7ff2440b29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:59.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.841+0000 7ff243fff700 1 -- 172.21.15.17:0/105844793 >> v1:172.21.15.77:6789/0 conn(0x7ff24405f840 legacy=0x7ff24405fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:59.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.841+0000 7ff243fff700 1 -- 172.21.15.17:0/105844793 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2440b39c0 con 0x7ff2440ae0b0 2022-08-01T19:39:59.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.841+0000 7ff243fff700 1 --2- 172.21.15.17:0/105844793 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2440ae0b0 0x7ff24414b800 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7ff2440b0ce0 tx=0x7ff230006eb0).ready entity=mon.0 client_cookie=803a53ca69bee576 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:59.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.841+0000 7ff2437fe700 1 -- 172.21.15.17:0/105844793 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff23000d5c0 con 0x7ff2440ae0b0 2022-08-01T19:39:59.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.841+0000 7ff2437fe700 1 -- 172.21.15.17:0/105844793 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff230000e90 con 0x7ff2440ae0b0 2022-08-01T19:39:59.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.841+0000 7ff2437fe700 1 -- 172.21.15.17:0/105844793 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff23000d950 con 0x7ff2440ae0b0 2022-08-01T19:39:59.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.841+0000 7ff24ae7b700 1 -- 172.21.15.17:0/105844793 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2440ae0b0 msgr2=0x7ff24414b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:59.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.841+0000 7ff24ae7b700 1 --2- 172.21.15.17:0/105844793 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2440ae0b0 0x7ff24414b800 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7ff2440b0ce0 tx=0x7ff230006eb0).stop 2022-08-01T19:39:59.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.841+0000 7ff24ae7b700 1 -- 172.21.15.17:0/105844793 shutdown_connections 2022-08-01T19:39:59.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.841+0000 7ff24ae7b700 1 --2- 172.21.15.17:0/105844793 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2440ae0b0 0x7ff24414b800 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:59.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.841+0000 7ff24ae7b700 1 --2- 172.21.15.17:0/105844793 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff2440b1580 0x7ff2440b29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:59.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.841+0000 7ff24ae7b700 1 -- 172.21.15.17:0/105844793 >> 172.21.15.17:0/105844793 conn(0x7ff244060950 msgr2=0x7ff244060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:59.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.845+0000 7ff24ae7b700 1 -- 172.21.15.17:0/105844793 shutdown_connections 2022-08-01T19:39:59.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.845+0000 7ff24ae7b700 1 -- 172.21.15.17:0/105844793 wait complete. 2022-08-01T19:39:59.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.845+0000 7ff24ae7b700 1 Processor -- start 2022-08-01T19:39:59.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.845+0000 7ff24ae7b700 1 -- start start 2022-08-01T19:39:59.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.845+0000 7ff24ae7b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff24405f840 0x7ff2440678d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:59.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.845+0000 7ff24ae7b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff2440ae0b0 0x7ff24409ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:59.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.845+0000 7ff24ae7b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2440b1580 0x7ff24409f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:59.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.845+0000 7ff24ae7b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff244154650 con 0x7ff2440b1580 2022-08-01T19:39:59.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.845+0000 7ff24ae7b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2441576f0 con 0x7ff24405f840 2022-08-01T19:39:59.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.845+0000 7ff24ae7b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2440a14d0 con 0x7ff2440ae0b0 2022-08-01T19:39:59.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.845+0000 7ff249418700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2440b1580 0x7ff24409f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:59.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.845+0000 7ff249418700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2440b1580 0x7ff24409f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40744/0 (socket says 172.21.15.17:40744) 2022-08-01T19:39:59.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.845+0000 7ff249418700 1 -- 172.21.15.17:0/2350554889 learned_addr learned my addr 172.21.15.17:0/2350554889 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:39:59.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.845+0000 7ff243fff700 1 --2- 172.21.15.17:0/2350554889 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff2440ae0b0 0x7ff24409ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:59.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.845+0000 7ff248c17700 1 --2- 172.21.15.17:0/2350554889 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff24405f840 0x7ff2440678d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:59.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.845+0000 7ff249418700 1 -- 172.21.15.17:0/2350554889 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff2440ae0b0 msgr2=0x7ff24409ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:59.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.845+0000 7ff249418700 1 --2- 172.21.15.17:0/2350554889 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff2440ae0b0 0x7ff24409ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:59.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.845+0000 7ff249418700 1 -- 172.21.15.17:0/2350554889 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff24405f840 msgr2=0x7ff2440678d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:59.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.845+0000 7ff249418700 1 --2- 172.21.15.17:0/2350554889 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff24405f840 0x7ff2440678d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:59.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.845+0000 7ff249418700 1 -- 172.21.15.17:0/2350554889 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23000b1e0 con 0x7ff2440b1580 2022-08-01T19:39:59.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.845+0000 7ff248c17700 1 --2- 172.21.15.17:0/2350554889 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff24405f840 0x7ff2440678d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:39:59.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.845+0000 7ff249418700 1 --2- 172.21.15.17:0/2350554889 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2440b1580 0x7ff24409f300 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7ff24414b6e0 tx=0x7ff23c007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:59.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.845+0000 7ff241ffb700 1 -- 172.21.15.17:0/2350554889 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff23c00e4a0 con 0x7ff2440b1580 2022-08-01T19:39:59.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.845+0000 7ff241ffb700 1 -- 172.21.15.17:0/2350554889 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23c00e600 con 0x7ff2440b1580 2022-08-01T19:39:59.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.845+0000 7ff24ae7b700 1 -- 172.21.15.17:0/2350554889 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2440a1770 con 0x7ff2440b1580 2022-08-01T19:39:59.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.845+0000 7ff241ffb700 1 -- 172.21.15.17:0/2350554889 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff23c00e960 con 0x7ff2440b1580 2022-08-01T19:39:59.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.849+0000 7ff24ae7b700 1 -- 172.21.15.17:0/2350554889 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2441572d0 con 0x7ff2440b1580 2022-08-01T19:39:59.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.849+0000 7ff24ae7b700 1 -- 172.21.15.17:0/2350554889 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2441578b0 con 0x7ff2440b1580 2022-08-01T19:39:59.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.849+0000 7ff241ffb700 1 -- 172.21.15.17:0/2350554889 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff23c019030 con 0x7ff2440b1580 2022-08-01T19:39:59.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.849+0000 7ff241ffb700 1 --2- 172.21.15.17:0/2350554889 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff224053b00 0x7ff224055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:39:59.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.849+0000 7ff241ffb700 1 -- 172.21.15.17:0/2350554889 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff23c06b7d0 con 0x7ff2440b1580 2022-08-01T19:39:59.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.849+0000 7ff248c17700 1 --2- 172.21.15.17:0/2350554889 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff224053b00 0x7ff224055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:39:59.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.849+0000 7ff248c17700 1 --2- 172.21.15.17:0/2350554889 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff224053b00 0x7ff224055fc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7ff23800b000 tx=0x7ff238008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:39:59.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.857+0000 7ff241ffb700 1 -- 172.21.15.17:0/2350554889 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff23c0424f0 con 0x7ff2440b1580 2022-08-01T19:39:59.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.981+0000 7ff24ae7b700 1 -- 172.21.15.17:0/2350554889 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff24414b0c0 con 0x7ff2440b1580 2022-08-01T19:39:59.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.981+0000 7ff241ffb700 1 -- 172.21.15.17:0/2350554889 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v76) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff23c045b10 con 0x7ff2440b1580 2022-08-01T19:39:59.981 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:39:59.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.981+0000 7ff24ae7b700 1 -- 172.21.15.17:0/2350554889 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff224053b00 msgr2=0x7ff224055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:59.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.981+0000 7ff24ae7b700 1 --2- 172.21.15.17:0/2350554889 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff224053b00 0x7ff224055fc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7ff23800b000 tx=0x7ff238008040).stop 2022-08-01T19:39:59.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.981+0000 7ff24ae7b700 1 -- 172.21.15.17:0/2350554889 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2440b1580 msgr2=0x7ff24409f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:39:59.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.981+0000 7ff24ae7b700 1 --2- 172.21.15.17:0/2350554889 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2440b1580 0x7ff24409f300 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7ff24414b6e0 tx=0x7ff23c007f40).stop 2022-08-01T19:39:59.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.981+0000 7ff24ae7b700 1 -- 172.21.15.17:0/2350554889 shutdown_connections 2022-08-01T19:39:59.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.981+0000 7ff24ae7b700 1 --2- 172.21.15.17:0/2350554889 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff224053b00 0x7ff224055fc0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:59.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.981+0000 7ff24ae7b700 1 --2- 172.21.15.17:0/2350554889 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2440b1580 0x7ff24409f300 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:59.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.981+0000 7ff24ae7b700 1 --2- 172.21.15.17:0/2350554889 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff2440ae0b0 0x7ff24409ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:59.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.981+0000 7ff24ae7b700 1 --2- 172.21.15.17:0/2350554889 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff24405f840 0x7ff2440678d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:39:59.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.981+0000 7ff24ae7b700 1 -- 172.21.15.17:0/2350554889 >> 172.21.15.17:0/2350554889 conn(0x7ff244060950 msgr2=0x7ff24405fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:39:59.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.981+0000 7ff24ae7b700 1 -- 172.21.15.17:0/2350554889 shutdown_connections 2022-08-01T19:39:59.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:39:59.981+0000 7ff24ae7b700 1 -- 172.21.15.17:0/2350554889 wait complete. 2022-08-01T19:39:59.995 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T19:40:00.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.109+0000 7f8a59041700 1 Processor -- start 2022-08-01T19:40:00.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.109+0000 7f8a59041700 1 -- start start 2022-08-01T19:40:00.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.113+0000 7f8a59041700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8a54066dc0 0x7f8a5414eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:00.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.113+0000 7f8a59041700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a540654c0 con 0x7f8a540b2730 2022-08-01T19:40:00.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.113+0000 7f8a59041700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a540b3570 con 0x7f8a540b3190 2022-08-01T19:40:00.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.113+0000 7f8a59041700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a54062190 con 0x7f8a54066dc0 2022-08-01T19:40:00.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.113+0000 7f8a58840700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8a54066dc0 0x7f8a5414eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:00.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.113+0000 7f8a4b7fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f8a540b2730 0x7f8a540666b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:32944/0 (socket says 172.21.15.17:32944) 2022-08-01T19:40:00.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.113+0000 7f8a4b7fe700 1 -- 172.21.15.17:0/1472864098 learned_addr learned my addr 172.21.15.17:0/1472864098 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:40:00.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.113+0000 7f8a537fe700 1 -- 172.21.15.17:0/1472864098 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2747386757 0 0) 0x7f8a540b3570 con 0x7f8a540b3190 2022-08-01T19:40:00.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.113+0000 7f8a537fe700 1 -- 172.21.15.17:0/1472864098 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a34003260 con 0x7f8a540b3190 2022-08-01T19:40:00.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.113+0000 7f8a58840700 1 -- 172.21.15.17:0/1472864098 >> v1:172.21.15.77:6789/0 conn(0x7f8a540b3190 legacy=0x7f8a54060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:00.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.113+0000 7f8a58840700 1 -- 172.21.15.17:0/1472864098 >> v1:172.21.15.17:6789/0 conn(0x7f8a540b2730 legacy=0x7f8a540666b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:00.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.113+0000 7f8a58840700 1 -- 172.21.15.17:0/1472864098 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a5405fbf0 con 0x7f8a54066dc0 2022-08-01T19:40:00.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.113+0000 7f8a58840700 1 --2- 172.21.15.17:0/1472864098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a54066dc0 0x7f8a5414eac0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f8a4c0032f0 tx=0x7f8a4c0091d0).ready entity=mon.0 client_cookie=e854ff06a717d91 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:00.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.113+0000 7f8a537fe700 1 -- 172.21.15.17:0/1472864098 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a4c00b6c0 con 0x7f8a54066dc0 2022-08-01T19:40:00.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.113+0000 7f8a537fe700 1 -- 172.21.15.17:0/1472864098 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a4c00b820 con 0x7f8a54066dc0 2022-08-01T19:40:00.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.113+0000 7f8a537fe700 1 -- 172.21.15.17:0/1472864098 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a4c00b990 con 0x7f8a54066dc0 2022-08-01T19:40:00.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.113+0000 7f8a59041700 1 -- 172.21.15.17:0/1472864098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a54066dc0 msgr2=0x7f8a5414eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:00.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.113+0000 7f8a59041700 1 --2- 172.21.15.17:0/1472864098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a54066dc0 0x7f8a5414eac0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f8a4c0032f0 tx=0x7f8a4c0091d0).stop 2022-08-01T19:40:00.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.113+0000 7f8a59041700 1 -- 172.21.15.17:0/1472864098 shutdown_connections 2022-08-01T19:40:00.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.113+0000 7f8a59041700 1 --2- 172.21.15.17:0/1472864098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a54066dc0 0x7f8a5414eac0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:00.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.113+0000 7f8a59041700 1 -- 172.21.15.17:0/1472864098 >> 172.21.15.17:0/1472864098 conn(0x7f8a54063870 msgr2=0x7f8a54063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:00.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.113+0000 7f8a59041700 1 -- 172.21.15.17:0/1472864098 shutdown_connections 2022-08-01T19:40:00.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.113+0000 7f8a59041700 1 -- 172.21.15.17:0/1472864098 wait complete. 2022-08-01T19:40:00.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.117+0000 7f8a59041700 1 Processor -- start 2022-08-01T19:40:00.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.117+0000 7f8a59041700 1 -- start start 2022-08-01T19:40:00.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.117+0000 7f8a59041700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a54066dc0 0x7f8a54140170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:00.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.117+0000 7f8a59041700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8a540b2730 0x7f8a54152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:00.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.117+0000 7f8a59041700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a540b3190 0x7f8a54152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:00.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.117+0000 7f8a59041700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a54152e30 con 0x7f8a54066dc0 2022-08-01T19:40:00.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.117+0000 7f8a59041700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a54140da0 con 0x7f8a540b3190 2022-08-01T19:40:00.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.117+0000 7f8a59041700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a540a9b00 con 0x7f8a540b2730 2022-08-01T19:40:00.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.117+0000 7f8a53fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a54066dc0 0x7f8a54140170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:00.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.117+0000 7f8a53fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a54066dc0 0x7f8a54140170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40754/0 (socket says 172.21.15.17:40754) 2022-08-01T19:40:00.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.117+0000 7f8a53fff700 1 -- 172.21.15.17:0/4262126180 learned_addr learned my addr 172.21.15.17:0/4262126180 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:00.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.117+0000 7f8a4b7fe700 1 --2- 172.21.15.17:0/4262126180 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8a540b2730 0x7f8a54152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:00.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.117+0000 7f8a58840700 1 --2- 172.21.15.17:0/4262126180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a540b3190 0x7f8a54152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:00.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.121+0000 7f8a53fff700 1 -- 172.21.15.17:0/4262126180 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8a540b2730 msgr2=0x7f8a54152450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:00.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.121+0000 7f8a53fff700 1 --2- 172.21.15.17:0/4262126180 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8a540b2730 0x7f8a54152450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:00.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.121+0000 7f8a53fff700 1 -- 172.21.15.17:0/4262126180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a540b3190 msgr2=0x7f8a54152940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:00.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.121+0000 7f8a53fff700 1 --2- 172.21.15.17:0/4262126180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a540b3190 0x7f8a54152940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:00.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.121+0000 7f8a53fff700 1 -- 172.21.15.17:0/4262126180 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a4c009470 con 0x7f8a54066dc0 2022-08-01T19:40:00.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.121+0000 7f8a58840700 1 --2- 172.21.15.17:0/4262126180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a540b3190 0x7f8a54152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:40:00.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.121+0000 7f8a53fff700 1 --2- 172.21.15.17:0/4262126180 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a54066dc0 0x7f8a54140170 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f8a44001110 tx=0x7f8a44007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:00.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.121+0000 7f8a51ffb700 1 -- 172.21.15.17:0/4262126180 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a44012400 con 0x7f8a54066dc0 2022-08-01T19:40:00.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.121+0000 7f8a51ffb700 1 -- 172.21.15.17:0/4262126180 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a44012560 con 0x7f8a54066dc0 2022-08-01T19:40:00.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.121+0000 7f8a59041700 1 -- 172.21.15.17:0/4262126180 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a54145ef0 con 0x7f8a54066dc0 2022-08-01T19:40:00.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.121+0000 7f8a51ffb700 1 -- 172.21.15.17:0/4262126180 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a440128c0 con 0x7f8a54066dc0 2022-08-01T19:40:00.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.121+0000 7f8a59041700 1 -- 172.21.15.17:0/4262126180 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a54145860 con 0x7f8a54066dc0 2022-08-01T19:40:00.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.121+0000 7f8a51ffb700 1 -- 172.21.15.17:0/4262126180 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8a44019030 con 0x7f8a54066dc0 2022-08-01T19:40:00.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.121+0000 7f8a59041700 1 -- 172.21.15.17:0/4262126180 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a540a3200 con 0x7f8a54066dc0 2022-08-01T19:40:00.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.121+0000 7f8a51ffb700 1 --2- 172.21.15.17:0/4262126180 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8a38053b00 0x7f8a38055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:00.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.121+0000 7f8a51ffb700 1 -- 172.21.15.17:0/4262126180 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8a4406b420 con 0x7f8a54066dc0 2022-08-01T19:40:00.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.121+0000 7f8a4b7fe700 1 --2- 172.21.15.17:0/4262126180 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8a38053b00 0x7f8a38055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:00.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.121+0000 7f8a4b7fe700 1 --2- 172.21.15.17:0/4262126180 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8a38053b00 0x7f8a38055fc0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f8a3c00d000 tx=0x7f8a3c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:00.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.129+0000 7f8a51ffb700 1 -- 172.21.15.17:0/4262126180 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8a440421e0 con 0x7f8a54066dc0 2022-08-01T19:40:00.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.253+0000 7f8a59041700 1 -- 172.21.15.17:0/4262126180 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8a5405fe90 con 0x7f8a54066dc0 2022-08-01T19:40:00.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.257+0000 7f8a51ffb700 1 -- 172.21.15.17:0/4262126180 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v76) v1 ==== 104+0+10 (secure 0 0 0) 0x7f8a44045800 con 0x7f8a54066dc0 2022-08-01T19:40:00.257 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:40:00.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.257+0000 7f8a59041700 1 -- 172.21.15.17:0/4262126180 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8a38053b00 msgr2=0x7f8a38055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:00.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.257+0000 7f8a59041700 1 --2- 172.21.15.17:0/4262126180 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8a38053b00 0x7f8a38055fc0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f8a3c00d000 tx=0x7f8a3c009040).stop 2022-08-01T19:40:00.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.257+0000 7f8a59041700 1 -- 172.21.15.17:0/4262126180 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a54066dc0 msgr2=0x7f8a54140170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:00.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.257+0000 7f8a59041700 1 --2- 172.21.15.17:0/4262126180 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a54066dc0 0x7f8a54140170 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f8a44001110 tx=0x7f8a44007ae0).stop 2022-08-01T19:40:00.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.257+0000 7f8a59041700 1 -- 172.21.15.17:0/4262126180 shutdown_connections 2022-08-01T19:40:00.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.257+0000 7f8a59041700 1 --2- 172.21.15.17:0/4262126180 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8a38053b00 0x7f8a38055fc0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:00.266 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.257+0000 7f8a59041700 1 --2- 172.21.15.17:0/4262126180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a540b3190 0x7f8a54152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:00.266 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.257+0000 7f8a59041700 1 --2- 172.21.15.17:0/4262126180 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8a540b2730 0x7f8a54152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:00.266 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.257+0000 7f8a59041700 1 --2- 172.21.15.17:0/4262126180 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a54066dc0 0x7f8a54140170 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:00.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.257+0000 7f8a59041700 1 -- 172.21.15.17:0/4262126180 >> 172.21.15.17:0/4262126180 conn(0x7f8a54063870 msgr2=0x7f8a54066510 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:00.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.257+0000 7f8a59041700 1 -- 172.21.15.17:0/4262126180 shutdown_connections 2022-08-01T19:40:00.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.257+0000 7f8a59041700 1 -- 172.21.15.17:0/4262126180 wait complete. 2022-08-01T19:40:00.274 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T19:40:00.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.389+0000 7f92e6930700 1 Processor -- start 2022-08-01T19:40:00.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.389+0000 7f92e6930700 1 -- start start 2022-08-01T19:40:00.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.393+0000 7f92e6930700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f92e00b3190 0x7f92e0060ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:00.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.393+0000 7f92e6930700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f92e00b2730 0x7f92e0061190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:00.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.393+0000 7f92e6930700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92e00654c0 con 0x7f92e00666b0 2022-08-01T19:40:00.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.393+0000 7f92e6930700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92e0062190 con 0x7f92e00b2730 2022-08-01T19:40:00.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.393+0000 7f92e6930700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e00b3570 con 0x7f92e00b3190 2022-08-01T19:40:00.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.393+0000 7f92e612f700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f92e00666b0 0x7f92e0066b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:32960/0 (socket says 172.21.15.17:32960) 2022-08-01T19:40:00.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.393+0000 7f92e612f700 1 -- 172.21.15.17:0/2261997520 learned_addr learned my addr 172.21.15.17:0/2261997520 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:40:00.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.393+0000 7f92e592e700 1 --2- 172.21.15.17:0/2261997520 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f92e00b3190 0x7f92e0060ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:00.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.393+0000 7f92e512d700 1 --2- 172.21.15.17:0/2261997520 >> v2:172.21.15.77:3300/0 conn(0x7f92e00b2730 0x7f92e0061190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:00.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.393+0000 7f92e492c700 1 -- 172.21.15.17:0/2261997520 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1616958847 0 0) 0x7f92e00654c0 con 0x7f92e00666b0 2022-08-01T19:40:00.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.393+0000 7f92e492c700 1 -- 172.21.15.17:0/2261997520 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92c8003260 con 0x7f92e00666b0 2022-08-01T19:40:00.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.393+0000 7f92e592e700 1 -- 172.21.15.17:0/2261997520 >> v2:172.21.15.77:3300/0 conn(0x7f92e00b2730 msgr2=0x7f92e0061190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:00.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.393+0000 7f92e592e700 1 --2- 172.21.15.17:0/2261997520 >> v2:172.21.15.77:3300/0 conn(0x7f92e00b2730 0x7f92e0061190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:00.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.393+0000 7f92e492c700 1 -- 172.21.15.17:0/2261997520 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4059093976 0 0) 0x7f92c8003260 con 0x7f92e00666b0 2022-08-01T19:40:00.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.393+0000 7f92e592e700 1 -- 172.21.15.17:0/2261997520 >> v1:172.21.15.17:6789/0 conn(0x7f92e00666b0 legacy=0x7f92e0066b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:00.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.393+0000 7f92e592e700 1 -- 172.21.15.17:0/2261997520 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e005fbf0 con 0x7f92e00b3190 2022-08-01T19:40:00.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.393+0000 7f92e512d700 1 --2- 172.21.15.17:0/2261997520 >> v2:172.21.15.77:3300/0 conn(0x7f92e00b2730 0x7f92e0061190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:40:00.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.393+0000 7f92e592e700 1 --2- 172.21.15.17:0/2261997520 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f92e00b3190 0x7f92e0060ca0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f92d8001b80 tx=0x7f92d4006df0).ready entity=mon.2 client_cookie=e9776cff995001f8 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:00.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.393+0000 7f92e492c700 1 -- 172.21.15.17:0/2261997520 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92d80019e0 con 0x7f92e00b3190 2022-08-01T19:40:00.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.393+0000 7f92e492c700 1 -- 172.21.15.17:0/2261997520 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d4000e90 con 0x7f92e00b3190 2022-08-01T19:40:00.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.393+0000 7f92e492c700 1 -- 172.21.15.17:0/2261997520 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92d4012460 con 0x7f92e00b3190 2022-08-01T19:40:00.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.393+0000 7f92e6930700 1 -- 172.21.15.17:0/2261997520 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f92e00b3190 msgr2=0x7f92e0060ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:00.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.393+0000 7f92e6930700 1 --2- 172.21.15.17:0/2261997520 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f92e00b3190 0x7f92e0060ca0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f92d8001b80 tx=0x7f92d4006df0).stop 2022-08-01T19:40:00.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.393+0000 7f92e6930700 1 -- 172.21.15.17:0/2261997520 shutdown_connections 2022-08-01T19:40:00.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.393+0000 7f92e6930700 1 --2- 172.21.15.17:0/2261997520 >> v2:172.21.15.77:3300/0 conn(0x7f92e00b2730 0x7f92e0061190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:00.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.393+0000 7f92e6930700 1 --2- 172.21.15.17:0/2261997520 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f92e00b3190 0x7f92e0060ca0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:00.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.393+0000 7f92e6930700 1 -- 172.21.15.17:0/2261997520 >> 172.21.15.17:0/2261997520 conn(0x7f92e0063870 msgr2=0x7f92e0063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:00.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.393+0000 7f92e6930700 1 -- 172.21.15.17:0/2261997520 shutdown_connections 2022-08-01T19:40:00.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.393+0000 7f92e6930700 1 -- 172.21.15.17:0/2261997520 wait complete. 2022-08-01T19:40:00.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.397+0000 7f92e6930700 1 Processor -- start 2022-08-01T19:40:00.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.397+0000 7f92e6930700 1 -- start start 2022-08-01T19:40:00.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.397+0000 7f92e6930700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92e00666b0 0x7f92e00b0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:00.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.397+0000 7f92e6930700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92e00b2730 0x7f92e0152180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:00.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.397+0000 7f92e6930700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f92e00b3190 0x7f92e0152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:00.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.397+0000 7f92e6930700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e0152b60 con 0x7f92e00b2730 2022-08-01T19:40:00.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.397+0000 7f92e6930700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e0148790 con 0x7f92e00666b0 2022-08-01T19:40:00.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.397+0000 7f92e6930700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e00ac200 con 0x7f92e00b3190 2022-08-01T19:40:00.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.397+0000 7f92e512d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92e00b2730 0x7f92e0152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:00.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.397+0000 7f92e512d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92e00b2730 0x7f92e0152180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40770/0 (socket says 172.21.15.17:40770) 2022-08-01T19:40:00.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.397+0000 7f92e512d700 1 -- 172.21.15.17:0/1271970827 learned_addr learned my addr 172.21.15.17:0/1271970827 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:00.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.397+0000 7f92e612f700 1 --2- 172.21.15.17:0/1271970827 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f92e00b3190 0x7f92e0152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:00.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.397+0000 7f92e592e700 1 --2- 172.21.15.17:0/1271970827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92e00666b0 0x7f92e00b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:00.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.397+0000 7f92e512d700 1 -- 172.21.15.17:0/1271970827 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f92e00b3190 msgr2=0x7f92e0152670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:00.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.397+0000 7f92e512d700 1 --2- 172.21.15.17:0/1271970827 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f92e00b3190 0x7f92e0152670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:00.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.397+0000 7f92e512d700 1 -- 172.21.15.17:0/1271970827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92e00666b0 msgr2=0x7f92e00b0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:00.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.397+0000 7f92e512d700 1 --2- 172.21.15.17:0/1271970827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92e00666b0 0x7f92e00b0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:00.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.397+0000 7f92e512d700 1 -- 172.21.15.17:0/1271970827 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92dc00b180 con 0x7f92e00b2730 2022-08-01T19:40:00.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.397+0000 7f92e512d700 1 --2- 172.21.15.17:0/1271970827 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92e00b2730 0x7f92e0152180 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f92dc00c000 tx=0x7f92dc006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:00.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.401+0000 7f92d2ffd700 1 -- 172.21.15.17:0/1271970827 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92dc010780 con 0x7f92e00b2730 2022-08-01T19:40:00.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.401+0000 7f92d2ffd700 1 -- 172.21.15.17:0/1271970827 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92dc0108e0 con 0x7f92e00b2730 2022-08-01T19:40:00.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.401+0000 7f92e6930700 1 -- 172.21.15.17:0/1271970827 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92d400b180 con 0x7f92e00b2730 2022-08-01T19:40:00.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.401+0000 7f92d2ffd700 1 -- 172.21.15.17:0/1271970827 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92dc010c40 con 0x7f92e00b2730 2022-08-01T19:40:00.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.401+0000 7f92e6930700 1 -- 172.21.15.17:0/1271970827 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e00ae850 con 0x7f92e00b2730 2022-08-01T19:40:00.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.401+0000 7f92e6930700 1 -- 172.21.15.17:0/1271970827 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e00ac720 con 0x7f92e00b2730 2022-08-01T19:40:00.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.401+0000 7f92d2ffd700 1 -- 172.21.15.17:0/1271970827 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f92dc01c460 con 0x7f92e00b2730 2022-08-01T19:40:00.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.401+0000 7f92d2ffd700 1 --2- 172.21.15.17:0/1271970827 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f92c0053950 0x7f92c0055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:00.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.401+0000 7f92d2ffd700 1 -- 172.21.15.17:0/1271970827 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f92dc06c510 con 0x7f92e00b2730 2022-08-01T19:40:00.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.405+0000 7f92e592e700 1 --2- 172.21.15.17:0/1271970827 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f92c0053950 0x7f92c0055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:00.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.405+0000 7f92e592e700 1 --2- 172.21.15.17:0/1271970827 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f92c0053950 0x7f92c0055e10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f92d401a000 tx=0x7f92d4014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:00.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.413+0000 7f92d2ffd700 1 -- 172.21.15.17:0/1271970827 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f92dc043270 con 0x7f92e00b2730 2022-08-01T19:40:00.531 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.529+0000 7f92e6930700 1 -- 172.21.15.17:0/1271970827 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f92e005fe90 con 0x7f92e00b2730 2022-08-01T19:40:00.532 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.529+0000 7f92d2ffd700 1 -- 172.21.15.17:0/1271970827 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v76) v1 ==== 100+0+10 (secure 0 0 0) 0x7f92dc046890 con 0x7f92e00b2730 2022-08-01T19:40:00.532 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:40:00.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.533+0000 7f92e6930700 1 -- 172.21.15.17:0/1271970827 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f92c0053950 msgr2=0x7f92c0055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:00.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.533+0000 7f92e6930700 1 --2- 172.21.15.17:0/1271970827 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f92c0053950 0x7f92c0055e10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f92d401a000 tx=0x7f92d4014000).stop 2022-08-01T19:40:00.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.533+0000 7f92e6930700 1 -- 172.21.15.17:0/1271970827 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92e00b2730 msgr2=0x7f92e0152180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:00.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.533+0000 7f92e6930700 1 --2- 172.21.15.17:0/1271970827 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92e00b2730 0x7f92e0152180 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f92dc00c000 tx=0x7f92dc006df0).stop 2022-08-01T19:40:00.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.533+0000 7f92e6930700 1 -- 172.21.15.17:0/1271970827 shutdown_connections 2022-08-01T19:40:00.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.533+0000 7f92e6930700 1 --2- 172.21.15.17:0/1271970827 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f92c0053950 0x7f92c0055e10 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:00.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.533+0000 7f92e6930700 1 --2- 172.21.15.17:0/1271970827 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f92e00b3190 0x7f92e0152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:00.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.533+0000 7f92e6930700 1 --2- 172.21.15.17:0/1271970827 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92e00b2730 0x7f92e0152180 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:00.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.533+0000 7f92e6930700 1 --2- 172.21.15.17:0/1271970827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92e00666b0 0x7f92e00b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:00.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.533+0000 7f92e6930700 1 -- 172.21.15.17:0/1271970827 >> 172.21.15.17:0/1271970827 conn(0x7f92e0063870 msgr2=0x7f92e01463d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:00.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.533+0000 7f92e6930700 1 -- 172.21.15.17:0/1271970827 shutdown_connections 2022-08-01T19:40:00.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.533+0000 7f92e6930700 1 -- 172.21.15.17:0/1271970827 wait complete. 2022-08-01T19:40:00.545 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T19:40:00.545 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:40:00.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.645+0000 7fcd328eb700 1 Processor -- start 2022-08-01T19:40:00.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.645+0000 7fcd328eb700 1 -- start start 2022-08-01T19:40:00.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.645+0000 7fcd328eb700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fcd2c148530 0x7fcd2c0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:00.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.645+0000 7fcd328eb700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcd2c0a7250 0x7fcd2c1448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:00.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.645+0000 7fcd328eb700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd2c0b04e0 con 0x7fcd2c0b1f40 2022-08-01T19:40:00.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.645+0000 7fcd328eb700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd2c0a9250 con 0x7fcd2c148530 2022-08-01T19:40:00.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.645+0000 7fcd328eb700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c0b0780 con 0x7fcd2c0a7250 2022-08-01T19:40:00.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.649+0000 7fcd2bfff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fcd2c0b1f40 0x7fcd2c0b2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:32972/0 (socket says 172.21.15.17:32972) 2022-08-01T19:40:00.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.649+0000 7fcd2bfff700 1 -- 172.21.15.17:0/2750240998 learned_addr learned my addr 172.21.15.17:0/2750240998 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:40:00.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.649+0000 7fcd2b7fe700 1 --2- 172.21.15.17:0/2750240998 >> v2:172.21.15.77:3300/0 conn(0x7fcd2c148530 0x7fcd2c0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:00.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.649+0000 7fcd30e88700 1 --2- 172.21.15.17:0/2750240998 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcd2c0a7250 0x7fcd2c1448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:00.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.649+0000 7fcd2affd700 1 -- 172.21.15.17:0/2750240998 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3323486182 0 0) 0x7fcd2c0b04e0 con 0x7fcd2c0b1f40 2022-08-01T19:40:00.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.649+0000 7fcd2affd700 1 -- 172.21.15.17:0/2750240998 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd0c003260 con 0x7fcd2c0b1f40 2022-08-01T19:40:00.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.649+0000 7fcd2b7fe700 1 -- 172.21.15.17:0/2750240998 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcd2c0a7250 msgr2=0x7fcd2c1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:00.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.649+0000 7fcd2b7fe700 1 --2- 172.21.15.17:0/2750240998 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcd2c0a7250 0x7fcd2c1448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:00.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.649+0000 7fcd2b7fe700 1 -- 172.21.15.17:0/2750240998 >> v1:172.21.15.17:6789/0 conn(0x7fcd2c0b1f40 legacy=0x7fcd2c0b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:00.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.649+0000 7fcd2b7fe700 1 -- 172.21.15.17:0/2750240998 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd2c0b0a20 con 0x7fcd2c148530 2022-08-01T19:40:00.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.649+0000 7fcd2b7fe700 1 --2- 172.21.15.17:0/2750240998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd2c148530 0x7fcd2c0b1160 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fcd2c1478c0 tx=0x7fcd1c0091d0).ready entity=mon.1 client_cookie=db555cdf1536a894 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:00.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.649+0000 7fcd2affd700 1 -- 172.21.15.17:0/2750240998 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd1c00b600 con 0x7fcd2c148530 2022-08-01T19:40:00.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.649+0000 7fcd2affd700 1 -- 172.21.15.17:0/2750240998 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1c00b760 con 0x7fcd2c148530 2022-08-01T19:40:00.660 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.649+0000 7fcd2affd700 1 -- 172.21.15.17:0/2750240998 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd1c00b8d0 con 0x7fcd2c148530 2022-08-01T19:40:00.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.649+0000 7fcd328eb700 1 -- 172.21.15.17:0/2750240998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd2c148530 msgr2=0x7fcd2c0b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:00.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.649+0000 7fcd328eb700 1 --2- 172.21.15.17:0/2750240998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd2c148530 0x7fcd2c0b1160 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fcd2c1478c0 tx=0x7fcd1c0091d0).stop 2022-08-01T19:40:00.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.649+0000 7fcd328eb700 1 -- 172.21.15.17:0/2750240998 shutdown_connections 2022-08-01T19:40:00.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.649+0000 7fcd328eb700 1 --2- 172.21.15.17:0/2750240998 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcd2c0a7250 0x7fcd2c1448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:00.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.649+0000 7fcd328eb700 1 --2- 172.21.15.17:0/2750240998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd2c148530 0x7fcd2c0b1160 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:00.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.649+0000 7fcd328eb700 1 -- 172.21.15.17:0/2750240998 >> 172.21.15.17:0/2750240998 conn(0x7fcd2c05f840 msgr2=0x7fcd2c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:00.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.649+0000 7fcd328eb700 1 -- 172.21.15.17:0/2750240998 shutdown_connections 2022-08-01T19:40:00.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.649+0000 7fcd328eb700 1 -- 172.21.15.17:0/2750240998 wait complete. 2022-08-01T19:40:00.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.653+0000 7fcd328eb700 1 Processor -- start 2022-08-01T19:40:00.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.653+0000 7fcd328eb700 1 -- start start 2022-08-01T19:40:00.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.653+0000 7fcd328eb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd2c0a7250 0x7fcd2c14b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:00.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.653+0000 7fcd328eb700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcd2c0b1f40 0x7fcd2c0ab2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:00.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.653+0000 7fcd328eb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd2c148530 0x7fcd2c0ab7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:00.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.653+0000 7fcd328eb700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c0aa0c0 con 0x7fcd2c148530 2022-08-01T19:40:00.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.653+0000 7fcd328eb700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c0ae5f0 con 0x7fcd2c0a7250 2022-08-01T19:40:00.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.653+0000 7fcd328eb700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c0ae890 con 0x7fcd2c0b1f40 2022-08-01T19:40:00.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.653+0000 7fcd2b7fe700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcd2c0b1f40 0x7fcd2c0ab2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:00.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.653+0000 7fcd30e88700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd2c148530 0x7fcd2c0ab7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:00.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.653+0000 7fcd2bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd2c0a7250 0x7fcd2c14b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:00.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.653+0000 7fcd2b7fe700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcd2c0b1f40 0x7fcd2c0ab2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:36882/0 (socket says 172.21.15.17:36882) 2022-08-01T19:40:00.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.653+0000 7fcd2b7fe700 1 -- 172.21.15.17:0/3997986649 learned_addr learned my addr 172.21.15.17:0/3997986649 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:00.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.653+0000 7fcd30e88700 1 -- 172.21.15.17:0/3997986649 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcd2c0b1f40 msgr2=0x7fcd2c0ab2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:00.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.653+0000 7fcd30e88700 1 --2- 172.21.15.17:0/3997986649 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcd2c0b1f40 0x7fcd2c0ab2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:00.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.653+0000 7fcd30e88700 1 -- 172.21.15.17:0/3997986649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd2c0a7250 msgr2=0x7fcd2c14b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:00.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.653+0000 7fcd30e88700 1 --2- 172.21.15.17:0/3997986649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd2c0a7250 0x7fcd2c14b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:00.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.653+0000 7fcd30e88700 1 -- 172.21.15.17:0/3997986649 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1c009470 con 0x7fcd2c148530 2022-08-01T19:40:00.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.653+0000 7fcd30e88700 1 --2- 172.21.15.17:0/3997986649 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd2c148530 0x7fcd2c0ab7e0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7fcd2400c000 tx=0x7fcd24006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:00.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.653+0000 7fcd297fa700 1 -- 172.21.15.17:0/3997986649 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd24016070 con 0x7fcd2c148530 2022-08-01T19:40:00.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.653+0000 7fcd297fa700 1 -- 172.21.15.17:0/3997986649 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd24011750 con 0x7fcd2c148530 2022-08-01T19:40:00.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.653+0000 7fcd328eb700 1 -- 172.21.15.17:0/3997986649 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd2c0af250 con 0x7fcd2c148530 2022-08-01T19:40:00.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.653+0000 7fcd297fa700 1 -- 172.21.15.17:0/3997986649 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd24011ab0 con 0x7fcd2c148530 2022-08-01T19:40:00.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.653+0000 7fcd328eb700 1 -- 172.21.15.17:0/3997986649 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd2c14a810 con 0x7fcd2c148530 2022-08-01T19:40:00.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.657+0000 7fcd328eb700 1 -- 172.21.15.17:0/3997986649 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd2c0ab160 con 0x7fcd2c148530 2022-08-01T19:40:00.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.657+0000 7fcd297fa700 1 -- 172.21.15.17:0/3997986649 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcd24011c10 con 0x7fcd2c148530 2022-08-01T19:40:00.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.657+0000 7fcd297fa700 1 --2- 172.21.15.17:0/3997986649 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcd10053a30 0x7fcd10055ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:00.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.657+0000 7fcd297fa700 1 -- 172.21.15.17:0/3997986649 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fcd2406bfe0 con 0x7fcd2c148530 2022-08-01T19:40:00.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.657+0000 7fcd2bfff700 1 --2- 172.21.15.17:0/3997986649 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcd10053a30 0x7fcd10055ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:00.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.661+0000 7fcd2bfff700 1 --2- 172.21.15.17:0/3997986649 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcd10053a30 0x7fcd10055ef0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7fcd1400d000 tx=0x7fcd14009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:00.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.665+0000 7fcd297fa700 1 -- 172.21.15.17:0/3997986649 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcd24042c90 con 0x7fcd2c148530 2022-08-01T19:40:00.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.821+0000 7fcd328eb700 1 -- 172.21.15.17:0/3997986649 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd2c0aec10 con 0x7fcd2c148530 2022-08-01T19:40:00.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.825+0000 7fcd297fa700 1 -- 172.21.15.17:0/3997986649 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+2493 (secure 0 0 0) 0x7fcd240462b0 con 0x7fcd2c148530 2022-08-01T19:40:00.828 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:40:00.829 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":35,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5615,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/2284778179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2284778179},{"type":"v1","addr":"172.21.15.77:6835","nonce":2284778179}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":34},{"gid":6261,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/1827111044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":1827111044},{"type":"v1","addr":"172.21.15.17:6833","nonce":1827111044}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":33}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:39:52.325506+0000","modified":"2022-08-01T19:39:53.328610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6267":{"gid":6267,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/3370152344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":3370152344},{"type":"v1","addr":"172.21.15.17:6835","nonce":3370152344}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-08-01T19:40:00.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.825+0000 7fcd328eb700 1 -- 172.21.15.17:0/3997986649 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcd10053a30 msgr2=0x7fcd10055ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:00.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.825+0000 7fcd328eb700 1 --2- 172.21.15.17:0/3997986649 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcd10053a30 0x7fcd10055ef0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7fcd1400d000 tx=0x7fcd14009040).stop 2022-08-01T19:40:00.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.825+0000 7fcd328eb700 1 -- 172.21.15.17:0/3997986649 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd2c148530 msgr2=0x7fcd2c0ab7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:00.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.825+0000 7fcd328eb700 1 --2- 172.21.15.17:0/3997986649 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd2c148530 0x7fcd2c0ab7e0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7fcd2400c000 tx=0x7fcd24006df0).stop 2022-08-01T19:40:00.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.825+0000 7fcd328eb700 1 -- 172.21.15.17:0/3997986649 shutdown_connections 2022-08-01T19:40:00.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.825+0000 7fcd328eb700 1 --2- 172.21.15.17:0/3997986649 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcd10053a30 0x7fcd10055ef0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:00.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.825+0000 7fcd328eb700 1 --2- 172.21.15.17:0/3997986649 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd2c148530 0x7fcd2c0ab7e0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:00.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.825+0000 7fcd328eb700 1 --2- 172.21.15.17:0/3997986649 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcd2c0b1f40 0x7fcd2c0ab2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:00.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.825+0000 7fcd328eb700 1 --2- 172.21.15.17:0/3997986649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd2c0a7250 0x7fcd2c14b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:00.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.825+0000 7fcd328eb700 1 -- 172.21.15.17:0/3997986649 >> 172.21.15.17:0/3997986649 conn(0x7fcd2c05f840 msgr2=0x7fcd2c0b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:00.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.825+0000 7fcd328eb700 1 -- 172.21.15.17:0/3997986649 shutdown_connections 2022-08-01T19:40:00.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:00.825+0000 7fcd328eb700 1 -- 172.21.15.17:0/3997986649 wait complete. 2022-08-01T19:40:00.833 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 35 2022-08-01T19:40:01.149 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:39:52.325506+0000', 'modified': '2022-08-01T19:39:53.328610+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6267}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6267': {'gid': 6267, 'name': 'a', 'rank': 0, 'incarnation': 34, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/3370152344', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 3370152344}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 3370152344}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:40:01.150 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:40:01.151 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-08-01T19:40:01.271 INFO:teuthology.orchestra.run.smithi017.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":6267,"want_state":"up:active","state":"up:active","rank_uptime":8.9349924999999999,"mdsmap_epoch":35,"osdmap_epoch":76,"osdmap_epoch_barrier":75,"uptime":13.251747959999999} 2022-08-01T19:40:01.271 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 6267, "mdsmap_epoch": 35, "osdmap_epoch": 76, "osdmap_epoch_barrier": 75, "rank_uptime": 8.9349925, "state": "up:active", "uptime": 13.25174796, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:40:01.272 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T19:40:01.273 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-08-01T19:40:01.273 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi190.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-08-01T19:40:01.273 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:40:01.274 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:40:01.290 DEBUG:teuthology.orchestra.run.smithi190:> sudo modprobe fuse 2022-08-01T19:40:01.313 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:40:01.314 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:40:01.375 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:40:01.377 INFO:teuthology.orchestra.run.smithi190.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:40:01.378 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:40:01.378 DEBUG:teuthology.orchestra.run.smithi190:> ls /sys/fs/fuse/connections 2022-08-01T19:40:01.436 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T19:40:01.437 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:40:01.446 DEBUG:teuthology.orchestra.run.smithi190:> sudo modprobe fuse 2022-08-01T19:40:01.469 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:40:01.470 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:40:01.523 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:2022-08-01T19:40:01.524+0000 7fbd762cc080 -1 init, newargv = 0x5598a5844770 newargc=9 2022-08-01T19:40:01.524 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:ceph-fuse[22674]: starting ceph client 2022-08-01T19:40:01.525 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:40:01.525 INFO:teuthology.orchestra.run.smithi190.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:40:01.526 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:40:01.527 DEBUG:teuthology.orchestra.run.smithi190:> ls /sys/fs/fuse/connections 2022-08-01T19:40:01.536 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:ceph-fuse[22674]: starting fuse 2022-08-01T19:40:01.542 INFO:teuthology.orchestra.run.smithi190.stdout:52 2022-08-01T19:40:01.543 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T19:40:01.544 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:40:01.545 DEBUG:teuthology.orchestra.run.smithi190:> sudo python3 -c ' 2022-08-01T19:40:01.546 DEBUG:teuthology.orchestra.run.smithi190:> import glob 2022-08-01T19:40:01.547 DEBUG:teuthology.orchestra.run.smithi190:> import re 2022-08-01T19:40:01.547 DEBUG:teuthology.orchestra.run.smithi190:> import os 2022-08-01T19:40:01.548 DEBUG:teuthology.orchestra.run.smithi190:> import subprocess 2022-08-01T19:40:01.549 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:40:01.551 DEBUG:teuthology.orchestra.run.smithi190:> def find_socket(client_name): 2022-08-01T19:40:01.552 DEBUG:teuthology.orchestra.run.smithi190:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-08-01T19:40:01.553 DEBUG:teuthology.orchestra.run.smithi190:> files = glob.glob(asok_path) 2022-08-01T19:40:01.554 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:40:01.555 DEBUG:teuthology.orchestra.run.smithi190:> # Given a non-glob path, it better be there 2022-08-01T19:40:01.555 DEBUG:teuthology.orchestra.run.smithi190:> if "*" not in asok_path: 2022-08-01T19:40:01.557 DEBUG:teuthology.orchestra.run.smithi190:> assert(len(files) == 1) 2022-08-01T19:40:01.559 DEBUG:teuthology.orchestra.run.smithi190:> return files[0] 2022-08-01T19:40:01.560 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:40:01.561 DEBUG:teuthology.orchestra.run.smithi190:> for f in files: 2022-08-01T19:40:01.562 DEBUG:teuthology.orchestra.run.smithi190:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T19:40:01.562 DEBUG:teuthology.orchestra.run.smithi190:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T19:40:01.563 DEBUG:teuthology.orchestra.run.smithi190:> return f 2022-08-01T19:40:01.564 DEBUG:teuthology.orchestra.run.smithi190:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T19:40:01.564 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:40:01.565 DEBUG:teuthology.orchestra.run.smithi190:> print(find_socket("client.1")) 2022-08-01T19:40:01.566 DEBUG:teuthology.orchestra.run.smithi190:> ' 2022-08-01T19:40:01.641 INFO:teuthology.orchestra.run.smithi190.stdout:/var/run/ceph/ceph-client.1.22674.asok 2022-08-01T19:40:01.641 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.22674.asok 2022-08-01T19:40:01.642 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:40:01.643 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.22674.asok status 2022-08-01T19:40:01.735 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2022-08-01T19:40:01.735 INFO:teuthology.orchestra.run.smithi190.stdout: "metadata": { 2022-08-01T19:40:01.736 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T19:40:01.736 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T19:40:01.736 INFO:teuthology.orchestra.run.smithi190.stdout: "entity_id": "1", 2022-08-01T19:40:01.737 INFO:teuthology.orchestra.run.smithi190.stdout: "hostname": "smithi190", 2022-08-01T19:40:01.737 INFO:teuthology.orchestra.run.smithi190.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-08-01T19:40:01.738 INFO:teuthology.orchestra.run.smithi190.stdout: "pid": "22674", 2022-08-01T19:40:01.739 INFO:teuthology.orchestra.run.smithi190.stdout: "root": "/" 2022-08-01T19:40:01.739 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:40:01.740 INFO:teuthology.orchestra.run.smithi190.stdout: "dentry_count": 0, 2022-08-01T19:40:01.741 INFO:teuthology.orchestra.run.smithi190.stdout: "dentry_pinned_count": 0, 2022-08-01T19:40:01.741 INFO:teuthology.orchestra.run.smithi190.stdout: "id": 5783, 2022-08-01T19:40:01.741 INFO:teuthology.orchestra.run.smithi190.stdout: "inst": { 2022-08-01T19:40:01.747 INFO:teuthology.orchestra.run.smithi190.stdout: "name": { 2022-08-01T19:40:01.748 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "client", 2022-08-01T19:40:01.748 INFO:teuthology.orchestra.run.smithi190.stdout: "num": 5783 2022-08-01T19:40:01.748 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:40:01.749 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": { 2022-08-01T19:40:01.749 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "v1", 2022-08-01T19:40:01.750 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": "172.21.15.190:0", 2022-08-01T19:40:01.750 INFO:teuthology.orchestra.run.smithi190.stdout: "nonce": 1202123108 2022-08-01T19:40:01.750 INFO:teuthology.orchestra.run.smithi190.stdout: } 2022-08-01T19:40:01.751 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:40:01.751 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": { 2022-08-01T19:40:01.751 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "v1", 2022-08-01T19:40:01.752 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": "172.21.15.190:0", 2022-08-01T19:40:01.752 INFO:teuthology.orchestra.run.smithi190.stdout: "nonce": 1202123108 2022-08-01T19:40:01.753 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:40:01.753 INFO:teuthology.orchestra.run.smithi190.stdout: "inst_str": "client.5783 172.21.15.190:0/1202123108", 2022-08-01T19:40:01.754 INFO:teuthology.orchestra.run.smithi190.stdout: "addr_str": "172.21.15.190:0/1202123108", 2022-08-01T19:40:01.755 INFO:teuthology.orchestra.run.smithi190.stdout: "inode_count": 1, 2022-08-01T19:40:01.759 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_epoch": 35, 2022-08-01T19:40:01.759 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_epoch": 76, 2022-08-01T19:40:01.760 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_epoch_barrier": 0, 2022-08-01T19:40:01.760 INFO:teuthology.orchestra.run.smithi190.stdout: "blacklisted": false, 2022-08-01T19:40:01.761 INFO:teuthology.orchestra.run.smithi190.stdout: "fs_name": "cephfs" 2022-08-01T19:40:01.761 INFO:teuthology.orchestra.run.smithi190.stdout:} 2022-08-01T19:40:01.764 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:40:01.765 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:40:01.765 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:40:01.786 INFO:teuthology.orchestra.run.smithi190.stdout:fuseblk 2022-08-01T19:40:01.787 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T19:40:01.788 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:40:01.788 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:40:01.852 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:40:01.967 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.965+0000 7ffba3fff700 1 Processor -- start 2022-08-01T19:40:01.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.965+0000 7ffba3fff700 1 -- start start 2022-08-01T19:40:01.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.965+0000 7ffba3fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffba40b2710 0x7ffba414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:01.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.965+0000 7ffba3fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ffba40b01a0 0x7ffba414b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:01.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.965+0000 7ffba3fff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ffba40afd60 0x7ffba40a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:01.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.965+0000 7ffba3fff700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffba40a7250 con 0x7ffba40b01a0 2022-08-01T19:40:01.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.965+0000 7ffba3fff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffba40a7480 con 0x7ffba40afd60 2022-08-01T19:40:01.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.965+0000 7ffba3fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba40a7720 con 0x7ffba40b2710 2022-08-01T19:40:01.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.965+0000 7ffba27fc700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ffba40b01a0 0x7ffba414b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:01.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.969+0000 7ffba2ffd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffba40b2710 0x7ffba414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:01.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.969+0000 7ffba37fe700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ffba40afd60 0x7ffba40a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:01.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.969+0000 7ffba27fc700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ffba40b01a0 0x7ffba414b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40794/0 (socket says 172.21.15.17:40794) 2022-08-01T19:40:01.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.969+0000 7ffba27fc700 1 -- 172.21.15.17:0/1729055415 learned_addr learned my addr 172.21.15.17:0/1729055415 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:01.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.969+0000 7ffba27fc700 1 -- 172.21.15.17:0/1729055415 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffba40b2710 msgr2=0x7ffba414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:01.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.969+0000 7ffba27fc700 1 --2- 172.21.15.17:0/1729055415 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffba40b2710 0x7ffba414afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:01.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.969+0000 7ffba27fc700 1 -- 172.21.15.17:0/1729055415 >> v2:172.21.15.77:3300/0 conn(0x7ffba40afd60 msgr2=0x7ffba40a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:01.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.969+0000 7ffba27fc700 1 --2- 172.21.15.17:0/1729055415 >> v2:172.21.15.77:3300/0 conn(0x7ffba40afd60 0x7ffba40a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:01.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.969+0000 7ffba27fc700 1 -- 172.21.15.17:0/1729055415 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba40a79c0 con 0x7ffba40b01a0 2022-08-01T19:40:01.992 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.969+0000 7ffba27fc700 1 --2- 172.21.15.17:0/1729055415 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffba40b01a0 0x7ffba414b4c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7ffb98002f70 tx=0x7ffb98006eb0).ready entity=mon.0 client_cookie=1ce12ad091ffb2b1 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:01.992 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.969+0000 7ffba1ffb700 1 -- 172.21.15.17:0/1729055415 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb9800d570 con 0x7ffba40b01a0 2022-08-01T19:40:01.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.969+0000 7ffba1ffb700 1 -- 172.21.15.17:0/1729055415 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb98000e90 con 0x7ffba40b01a0 2022-08-01T19:40:01.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.969+0000 7ffba1ffb700 1 -- 172.21.15.17:0/1729055415 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb9800d900 con 0x7ffba40b01a0 2022-08-01T19:40:01.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.969+0000 7ffba3fff700 1 -- 172.21.15.17:0/1729055415 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffba40b01a0 msgr2=0x7ffba414b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:01.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.969+0000 7ffba3fff700 1 --2- 172.21.15.17:0/1729055415 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffba40b01a0 0x7ffba414b4c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7ffb98002f70 tx=0x7ffb98006eb0).stop 2022-08-01T19:40:01.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.969+0000 7ffba3fff700 1 -- 172.21.15.17:0/1729055415 shutdown_connections 2022-08-01T19:40:01.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.969+0000 7ffba3fff700 1 --2- 172.21.15.17:0/1729055415 >> v2:172.21.15.77:3300/0 conn(0x7ffba40afd60 0x7ffba40a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:01.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.969+0000 7ffba3fff700 1 --2- 172.21.15.17:0/1729055415 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffba40b01a0 0x7ffba414b4c0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:01.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.969+0000 7ffba3fff700 1 --2- 172.21.15.17:0/1729055415 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffba40b2710 0x7ffba414afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:01.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.969+0000 7ffba3fff700 1 -- 172.21.15.17:0/1729055415 >> 172.21.15.17:0/1729055415 conn(0x7ffba4060310 msgr2=0x7ffba40b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:01.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.969+0000 7ffba3fff700 1 -- 172.21.15.17:0/1729055415 shutdown_connections 2022-08-01T19:40:01.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.969+0000 7ffba3fff700 1 -- 172.21.15.17:0/1729055415 wait complete. 2022-08-01T19:40:01.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.969+0000 7ffba3fff700 1 Processor -- start 2022-08-01T19:40:01.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.969+0000 7ffba3fff700 1 -- start start 2022-08-01T19:40:01.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.973+0000 7ffba3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffba40afd60 0x7ffba405c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:02.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.973+0000 7ffba3fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffba40b01a0 0x7ffba40ab6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:02.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.973+0000 7ffba3fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffba40b2710 0x7ffba40abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:02.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.973+0000 7ffba3fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba40b6040 con 0x7ffba40afd60 2022-08-01T19:40:02.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.973+0000 7ffba3fff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba40ab280 con 0x7ffba40b2710 2022-08-01T19:40:02.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.973+0000 7ffba3fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba40aaf30 con 0x7ffba40b01a0 2022-08-01T19:40:02.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.973+0000 7ffba2ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffba40afd60 0x7ffba405c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:02.010 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.973+0000 7ffba2ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffba40afd60 0x7ffba405c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40796/0 (socket says 172.21.15.17:40796) 2022-08-01T19:40:02.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.973+0000 7ffba2ffd700 1 -- 172.21.15.17:0/881563425 learned_addr learned my addr 172.21.15.17:0/881563425 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:02.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.973+0000 7ffba27fc700 1 --2- 172.21.15.17:0/881563425 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffba40b01a0 0x7ffba40ab6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:02.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.973+0000 7ffba37fe700 1 --2- 172.21.15.17:0/881563425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffba40b2710 0x7ffba40abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:02.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.973+0000 7ffba2ffd700 1 -- 172.21.15.17:0/881563425 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffba40b01a0 msgr2=0x7ffba40ab6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:02.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.973+0000 7ffba2ffd700 1 --2- 172.21.15.17:0/881563425 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffba40b01a0 0x7ffba40ab6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:02.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.973+0000 7ffba2ffd700 1 -- 172.21.15.17:0/881563425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffba40b2710 msgr2=0x7ffba40abbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:02.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.973+0000 7ffba2ffd700 1 --2- 172.21.15.17:0/881563425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffba40b2710 0x7ffba40abbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:02.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.973+0000 7ffba2ffd700 1 -- 172.21.15.17:0/881563425 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9800b1e0 con 0x7ffba40afd60 2022-08-01T19:40:02.013 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.973+0000 7ffba37fe700 1 --2- 172.21.15.17:0/881563425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffba40b2710 0x7ffba40abbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:40:02.013 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.973+0000 7ffba2ffd700 1 --2- 172.21.15.17:0/881563425 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffba40afd60 0x7ffba405c170 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7ffb9400c000 tx=0x7ffb94006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:02.013 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.973+0000 7ffb93fff700 1 -- 172.21.15.17:0/881563425 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb94010780 con 0x7ffba40afd60 2022-08-01T19:40:02.013 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.973+0000 7ffb93fff700 1 -- 172.21.15.17:0/881563425 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb940108e0 con 0x7ffba40afd60 2022-08-01T19:40:02.014 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.973+0000 7ffba3fff700 1 -- 172.21.15.17:0/881563425 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffba40b4060 con 0x7ffba40afd60 2022-08-01T19:40:02.014 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.973+0000 7ffb93fff700 1 -- 172.21.15.17:0/881563425 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb94010c40 con 0x7ffba40afd60 2022-08-01T19:40:02.014 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.973+0000 7ffba3fff700 1 -- 172.21.15.17:0/881563425 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba40b3dc0 con 0x7ffba40afd60 2022-08-01T19:40:02.014 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.973+0000 7ffba3fff700 1 -- 172.21.15.17:0/881563425 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffba40574e0 con 0x7ffba40afd60 2022-08-01T19:40:02.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.977+0000 7ffb93fff700 1 -- 172.21.15.17:0/881563425 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ffb9401c460 con 0x7ffba40afd60 2022-08-01T19:40:02.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.977+0000 7ffb93fff700 1 --2- 172.21.15.17:0/881563425 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ffb80053b00 0x7ffb80055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:02.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.977+0000 7ffb93fff700 1 -- 172.21.15.17:0/881563425 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ffb9406c490 con 0x7ffba40afd60 2022-08-01T19:40:02.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.977+0000 7ffba27fc700 1 --2- 172.21.15.17:0/881563425 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ffb80053b00 0x7ffb80055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:02.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.977+0000 7ffba27fc700 1 --2- 172.21.15.17:0/881563425 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ffb80053b00 0x7ffb80055fc0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7ffb9801c000 tx=0x7ffb98015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:02.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:01.981+0000 7ffb93fff700 1 -- 172.21.15.17:0/881563425 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ffb94043150 con 0x7ffba40afd60 2022-08-01T19:40:02.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.141+0000 7ffba3fff700 1 -- 172.21.15.17:0/881563425 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffba40ab3f0 con 0x7ffba40afd60 2022-08-01T19:40:02.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.145+0000 7ffb93fff700 1 -- 172.21.15.17:0/881563425 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+2493 (secure 0 0 0) 0x7ffb94046770 con 0x7ffba40afd60 2022-08-01T19:40:02.146 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:40:02.147 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":35,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5615,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/2284778179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2284778179},{"type":"v1","addr":"172.21.15.77:6835","nonce":2284778179}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":34},{"gid":6261,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/1827111044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":1827111044},{"type":"v1","addr":"172.21.15.17:6833","nonce":1827111044}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":33}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:39:52.325506+0000","modified":"2022-08-01T19:39:53.328610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6267":{"gid":6267,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/3370152344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":3370152344},{"type":"v1","addr":"172.21.15.17:6835","nonce":3370152344}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-08-01T19:40:02.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.145+0000 7ffba3fff700 1 -- 172.21.15.17:0/881563425 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ffb80053b00 msgr2=0x7ffb80055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:02.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.145+0000 7ffba3fff700 1 --2- 172.21.15.17:0/881563425 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ffb80053b00 0x7ffb80055fc0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7ffb9801c000 tx=0x7ffb98015000).stop 2022-08-01T19:40:02.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.145+0000 7ffba3fff700 1 -- 172.21.15.17:0/881563425 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffba40afd60 msgr2=0x7ffba405c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:02.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.145+0000 7ffba3fff700 1 --2- 172.21.15.17:0/881563425 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffba40afd60 0x7ffba405c170 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7ffb9400c000 tx=0x7ffb94006df0).stop 2022-08-01T19:40:02.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.145+0000 7ffba3fff700 1 -- 172.21.15.17:0/881563425 shutdown_connections 2022-08-01T19:40:02.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.145+0000 7ffba3fff700 1 --2- 172.21.15.17:0/881563425 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ffb80053b00 0x7ffb80055fc0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:02.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.145+0000 7ffba3fff700 1 --2- 172.21.15.17:0/881563425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffba40b2710 0x7ffba40abbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:02.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.145+0000 7ffba3fff700 1 --2- 172.21.15.17:0/881563425 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffba40b01a0 0x7ffba40ab6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:02.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.145+0000 7ffba3fff700 1 --2- 172.21.15.17:0/881563425 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffba40afd60 0x7ffba405c170 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:02.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.145+0000 7ffba3fff700 1 -- 172.21.15.17:0/881563425 >> 172.21.15.17:0/881563425 conn(0x7ffba4060310 msgr2=0x7ffba4149e70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:02.162 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.145+0000 7ffba3fff700 1 -- 172.21.15.17:0/881563425 shutdown_connections 2022-08-01T19:40:02.162 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.145+0000 7ffba3fff700 1 -- 172.21.15.17:0/881563425 wait complete. 2022-08-01T19:40:02.167 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 35 2022-08-01T19:40:02.169 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:40:02.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.269+0000 7f7983fff700 1 Processor -- start 2022-08-01T19:40:02.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.269+0000 7f7983fff700 1 -- start start 2022-08-01T19:40:02.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.269+0000 7f7983fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7984066780 0x7f7984066b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:02.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.269+0000 7f7983fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f79840b1bc0 0x7f798405ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:02.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.269+0000 7f7983fff700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f798405f8e0 con 0x7f7984140e40 2022-08-01T19:40:02.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.269+0000 7f7983fff700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79840b1630 con 0x7f79840b1bc0 2022-08-01T19:40:02.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.269+0000 7f7983fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f798405fb10 con 0x7f7984066780 2022-08-01T19:40:02.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.269+0000 7f79827fc700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f79840b1bc0 0x7f798405ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:02.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.269+0000 7f79827fc700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f79840b1bc0 0x7f798405ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40806/0 (socket says 172.21.15.17:40806) 2022-08-01T19:40:02.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.269+0000 7f79827fc700 1 -- 172.21.15.17:0/3438279293 learned_addr learned my addr 172.21.15.17:0/3438279293 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:02.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.269+0000 7f7982ffd700 1 --2- 172.21.15.17:0/3438279293 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7984066780 0x7f7984066b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:02.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.273+0000 7f79827fc700 1 -- 172.21.15.17:0/3438279293 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7984066780 msgr2=0x7f7984066b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:02.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.273+0000 7f79827fc700 1 --2- 172.21.15.17:0/3438279293 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7984066780 0x7f7984066b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:02.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.273+0000 7f79827fc700 1 -- 172.21.15.17:0/3438279293 >> v1:172.21.15.77:6789/0 conn(0x7f7984140e40 legacy=0x7f798413e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:02.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.273+0000 7f79827fc700 1 -- 172.21.15.17:0/3438279293 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79840604a0 con 0x7f79840b1bc0 2022-08-01T19:40:02.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.273+0000 7f79827fc700 1 --2- 172.21.15.17:0/3438279293 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79840b1bc0 0x7f798405ffb0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f7978002f70 tx=0x7f7978006eb0).ready entity=mon.0 client_cookie=d123501b8636a176 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:02.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.273+0000 7f7981ffb700 1 -- 172.21.15.17:0/3438279293 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f797800d570 con 0x7f79840b1bc0 2022-08-01T19:40:02.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.273+0000 7f7981ffb700 1 -- 172.21.15.17:0/3438279293 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7978000e90 con 0x7f79840b1bc0 2022-08-01T19:40:02.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.273+0000 7f7981ffb700 1 -- 172.21.15.17:0/3438279293 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f797800d900 con 0x7f79840b1bc0 2022-08-01T19:40:02.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.273+0000 7f7983fff700 1 -- 172.21.15.17:0/3438279293 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79840b1bc0 msgr2=0x7f798405ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:02.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.273+0000 7f7983fff700 1 --2- 172.21.15.17:0/3438279293 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79840b1bc0 0x7f798405ffb0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f7978002f70 tx=0x7f7978006eb0).stop 2022-08-01T19:40:02.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.273+0000 7f7983fff700 1 -- 172.21.15.17:0/3438279293 shutdown_connections 2022-08-01T19:40:02.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.273+0000 7f7983fff700 1 --2- 172.21.15.17:0/3438279293 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79840b1bc0 0x7f798405ffb0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:02.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.273+0000 7f7983fff700 1 --2- 172.21.15.17:0/3438279293 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7984066780 0x7f7984066b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:02.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.273+0000 7f7983fff700 1 -- 172.21.15.17:0/3438279293 >> 172.21.15.17:0/3438279293 conn(0x7f7984063870 msgr2=0x7f7984063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:02.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.273+0000 7f7983fff700 1 -- 172.21.15.17:0/3438279293 shutdown_connections 2022-08-01T19:40:02.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.273+0000 7f7983fff700 1 -- 172.21.15.17:0/3438279293 wait complete. 2022-08-01T19:40:02.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.277+0000 7f7983fff700 1 Processor -- start 2022-08-01T19:40:02.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.277+0000 7f7983fff700 1 -- start start 2022-08-01T19:40:02.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.277+0000 7f7983fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7984066780 0x7f79840ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:02.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.277+0000 7f7983fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79840b1bc0 0x7f79840ada00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:02.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.277+0000 7f7983fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7984140e40 0x7f79841538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:02.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.277+0000 7f7983fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79840ab9d0 con 0x7f7984066780 2022-08-01T19:40:02.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.277+0000 7f7983fff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79840ae870 con 0x7f7984140e40 2022-08-01T19:40:02.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.277+0000 7f7983fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79840afd10 con 0x7f79840b1bc0 2022-08-01T19:40:02.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.277+0000 7f7982ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7984066780 0x7f79840ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:02.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.277+0000 7f7982ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7984066780 0x7f79840ab4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40812/0 (socket says 172.21.15.17:40812) 2022-08-01T19:40:02.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.277+0000 7f7982ffd700 1 -- 172.21.15.17:0/3687143721 learned_addr learned my addr 172.21.15.17:0/3687143721 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:02.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.277+0000 7f79827fc700 1 --2- 172.21.15.17:0/3687143721 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79840b1bc0 0x7f79840ada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:02.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.277+0000 7f79837fe700 1 --2- 172.21.15.17:0/3687143721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7984140e40 0x7f79841538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:02.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.277+0000 7f7982ffd700 1 -- 172.21.15.17:0/3687143721 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79840b1bc0 msgr2=0x7f79840ada00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:02.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.277+0000 7f7982ffd700 1 --2- 172.21.15.17:0/3687143721 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79840b1bc0 0x7f79840ada00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:02.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.277+0000 7f7982ffd700 1 -- 172.21.15.17:0/3687143721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7984140e40 msgr2=0x7f79841538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:02.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.277+0000 7f7982ffd700 1 --2- 172.21.15.17:0/3687143721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7984140e40 0x7f79841538e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:02.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.277+0000 7f7982ffd700 1 -- 172.21.15.17:0/3687143721 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f797800b1e0 con 0x7f7984066780 2022-08-01T19:40:02.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.277+0000 7f7982ffd700 1 --2- 172.21.15.17:0/3687143721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7984066780 0x7f79840ab4e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7f797400c000 tx=0x7f7974006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:02.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.277+0000 7f7973fff700 1 -- 172.21.15.17:0/3687143721 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7974010780 con 0x7f7984066780 2022-08-01T19:40:02.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.277+0000 7f7973fff700 1 -- 172.21.15.17:0/3687143721 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79740108e0 con 0x7f7984066780 2022-08-01T19:40:02.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.277+0000 7f7983fff700 1 -- 172.21.15.17:0/3687143721 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79840affb0 con 0x7f7984066780 2022-08-01T19:40:02.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.277+0000 7f7973fff700 1 -- 172.21.15.17:0/3687143721 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7974010c40 con 0x7f7984066780 2022-08-01T19:40:02.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.277+0000 7f7983fff700 1 -- 172.21.15.17:0/3687143721 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79840adef0 con 0x7f7984066780 2022-08-01T19:40:02.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.281+0000 7f7983fff700 1 -- 172.21.15.17:0/3687143721 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79840b18d0 con 0x7f7984066780 2022-08-01T19:40:02.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.281+0000 7f7973fff700 1 -- 172.21.15.17:0/3687143721 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f797401c460 con 0x7f7984066780 2022-08-01T19:40:02.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.281+0000 7f7973fff700 1 --2- 172.21.15.17:0/3687143721 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f79600539e0 0x7f7960055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:02.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.281+0000 7f7973fff700 1 -- 172.21.15.17:0/3687143721 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f797406c560 con 0x7f7984066780 2022-08-01T19:40:02.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.281+0000 7f79827fc700 1 --2- 172.21.15.17:0/3687143721 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f79600539e0 0x7f7960055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:02.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.281+0000 7f79827fc700 1 --2- 172.21.15.17:0/3687143721 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f79600539e0 0x7f7960055ea0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f797801c000 tx=0x7f7978015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:02.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.289+0000 7f7973fff700 1 -- 172.21.15.17:0/3687143721 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f79740431b0 con 0x7f7984066780 2022-08-01T19:40:02.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.445+0000 7f7983fff700 1 -- 172.21.15.17:0/3687143721 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79840574e0 con 0x7f7984066780 2022-08-01T19:40:02.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.445+0000 7f7973fff700 1 -- 172.21.15.17:0/3687143721 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+2493 (secure 0 0 0) 0x7f79740467d0 con 0x7f7984066780 2022-08-01T19:40:02.448 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:40:02.448 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":35,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5615,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/2284778179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2284778179},{"type":"v1","addr":"172.21.15.77:6835","nonce":2284778179}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":34},{"gid":6261,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/1827111044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":1827111044},{"type":"v1","addr":"172.21.15.17:6833","nonce":1827111044}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":33}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:39:52.325506+0000","modified":"2022-08-01T19:39:53.328610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6267":{"gid":6267,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/3370152344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":3370152344},{"type":"v1","addr":"172.21.15.17:6835","nonce":3370152344}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-08-01T19:40:02.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.449+0000 7f7983fff700 1 -- 172.21.15.17:0/3687143721 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f79600539e0 msgr2=0x7f7960055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:02.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.449+0000 7f7983fff700 1 --2- 172.21.15.17:0/3687143721 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f79600539e0 0x7f7960055ea0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f797801c000 tx=0x7f7978015000).stop 2022-08-01T19:40:02.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.449+0000 7f7983fff700 1 -- 172.21.15.17:0/3687143721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7984066780 msgr2=0x7f79840ab4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:02.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.449+0000 7f7983fff700 1 --2- 172.21.15.17:0/3687143721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7984066780 0x7f79840ab4e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7f797400c000 tx=0x7f7974006df0).stop 2022-08-01T19:40:02.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.449+0000 7f7983fff700 1 -- 172.21.15.17:0/3687143721 shutdown_connections 2022-08-01T19:40:02.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.449+0000 7f7983fff700 1 --2- 172.21.15.17:0/3687143721 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f79600539e0 0x7f7960055ea0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:02.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.449+0000 7f7983fff700 1 --2- 172.21.15.17:0/3687143721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7984140e40 0x7f79841538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:02.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.449+0000 7f7983fff700 1 --2- 172.21.15.17:0/3687143721 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79840b1bc0 0x7f79840ada00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:02.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.449+0000 7f7983fff700 1 --2- 172.21.15.17:0/3687143721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7984066780 0x7f79840ab4e0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:02.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.449+0000 7f7983fff700 1 -- 172.21.15.17:0/3687143721 >> 172.21.15.17:0/3687143721 conn(0x7f7984063870 msgr2=0x7f7984063c50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:02.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.449+0000 7f7983fff700 1 -- 172.21.15.17:0/3687143721 shutdown_connections 2022-08-01T19:40:02.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:02.449+0000 7f7983fff700 1 -- 172.21.15.17:0/3687143721 wait complete. 2022-08-01T19:40:02.459 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 35 2022-08-01T19:40:02.463 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_cap_revoke_eviction_timeout 30.0 2022-08-01T19:40:02.568 INFO:teuthology.orchestra.run.smithi017.stdout:{"success":"mds_cap_revoke_eviction_timeout = '30.000000' "} 2022-08-01T19:40:02.569 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_cap_revoke_eviction_timeout = '30.000000' " } 2022-08-01T19:40:02.570 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:40:02.571 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:40:02.573 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:40:02.585 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:40:02.586 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:40:02.587 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-08-01T19:40:02.588 DEBUG:teuthology.orchestra.run.smithi085:> import time 2022-08-01T19:40:02.589 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:40:02.590 DEBUG:teuthology.orchestra.run.smithi085:> with open("/home/ubuntu/cephtest/mnt.0/background_file", '"'"'w'"'"') as f: 2022-08-01T19:40:02.590 DEBUG:teuthology.orchestra.run.smithi085:> f.write('"'"'content'"'"') 2022-08-01T19:40:02.591 DEBUG:teuthology.orchestra.run.smithi085:> f.flush() 2022-08-01T19:40:02.592 DEBUG:teuthology.orchestra.run.smithi085:> f.write('"'"'content2'"'"') 2022-08-01T19:40:02.593 DEBUG:teuthology.orchestra.run.smithi085:> while True: 2022-08-01T19:40:02.593 DEBUG:teuthology.orchestra.run.smithi085:> time.sleep(1) 2022-08-01T19:40:02.594 DEBUG:teuthology.orchestra.run.smithi085:> ' 2022-08-01T19:40:02.601 DEBUG:teuthology.orchestra.run.smithi085:> sudo ls /home/ubuntu/cephtest/mnt.0/background_file 2022-08-01T19:40:02.626 DEBUG:teuthology.orchestra.run:got remote process result: 2 2022-08-01T19:40:02.628 INFO:teuthology.orchestra.run.smithi085.stderr:ls: cannot access '/home/ubuntu/cephtest/mnt.0/background_file': No such file or directory 2022-08-01T19:40:03.631 DEBUG:teuthology.orchestra.run.smithi085:> sudo ls /home/ubuntu/cephtest/mnt.0/background_file 2022-08-01T19:40:03.649 INFO:teuthology.orchestra.run.smithi085.stdout:/home/ubuntu/cephtest/mnt.0/background_file 2022-08-01T19:40:03.652 DEBUG:tasks.cephfs.mount:File background_file became visible from 0 after 1s 2022-08-01T19:40:03.653 DEBUG:teuthology.orchestra.run.smithi190:> sudo ls /home/ubuntu/cephtest/mnt.1/background_file 2022-08-01T19:40:03.693 INFO:teuthology.orchestra.run.smithi190.stdout:/home/ubuntu/cephtest/mnt.1/background_file 2022-08-01T19:40:03.696 DEBUG:tasks.cephfs.mount:File background_file became visible from 1 after 0s 2022-08-01T19:40:03.696 INFO:tasks.cephfs.fuse_mount:Killing ceph-fuse connection on ubuntu@smithi085.front.sepia.ceph.com... 2022-08-01T19:40:03.715 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:40:03.717 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:40:03.717 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:40:03.762 INFO:teuthology.orchestra.run.smithi190.stdout:fuseblk 2022-08-01T19:40:03.763 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T19:40:03.764 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-08-01T19:40:03.764 DEBUG:teuthology.orchestra.run.smithi190:> import os 2022-08-01T19:40:03.766 DEBUG:teuthology.orchestra.run.smithi190:> import time 2022-08-01T19:40:03.766 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:40:03.766 DEBUG:teuthology.orchestra.run.smithi190:> fd = os.open("/home/ubuntu/cephtest/mnt.1/background_file", os.O_RDWR | os.O_CREAT, 0o644) 2022-08-01T19:40:03.767 DEBUG:teuthology.orchestra.run.smithi190:> try: 2022-08-01T19:40:03.767 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2022-08-01T19:40:03.768 DEBUG:teuthology.orchestra.run.smithi190:> os.write(fd, b'"'"'content'"'"') 2022-08-01T19:40:03.768 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(1) 2022-08-01T19:40:03.769 DEBUG:teuthology.orchestra.run.smithi190:> if not False: 2022-08-01T19:40:03.769 DEBUG:teuthology.orchestra.run.smithi190:> break 2022-08-01T19:40:03.770 DEBUG:teuthology.orchestra.run.smithi190:> except IOError as e: 2022-08-01T19:40:03.770 DEBUG:teuthology.orchestra.run.smithi190:> pass 2022-08-01T19:40:03.770 DEBUG:teuthology.orchestra.run.smithi190:> os.close(fd) 2022-08-01T19:40:03.771 DEBUG:teuthology.orchestra.run.smithi190:> ' 2022-08-01T19:40:17.850 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:40:17.858 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:40:17.865 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:40:17.874 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:40:44.138 INFO:tasks.cephfs.test_misc:cap_waiter waited 40.32865571975708s 2022-08-01T19:40:44.141 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist ls --format=json 2022-08-01T19:40:44.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.247+0000 7fb7258ad700 1 Processor -- start 2022-08-01T19:40:44.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.247+0000 7fb7258ad700 1 -- start start 2022-08-01T19:40:44.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.251+0000 7fb7258ad700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb7200b31d0 0x7fb720066020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:44.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.251+0000 7fb7258ad700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb7200a9da0 0x7fb720060950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:44.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.251+0000 7fb7258ad700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fb7201434b0 0x7fb720143940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:44.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.251+0000 7fb7258ad700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb720064a70 con 0x7fb7200b31d0 2022-08-01T19:40:44.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.251+0000 7fb7258ad700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb720061f60 con 0x7fb7201434b0 2022-08-01T19:40:44.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.251+0000 7fb7258ad700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb720067030 con 0x7fb7200a9da0 2022-08-01T19:40:44.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.251+0000 7fb7248ab700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb7200b31d0 0x7fb720066020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:44.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.251+0000 7fb7248ab700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb7200b31d0 0x7fb720066020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40822/0 (socket says 172.21.15.17:40822) 2022-08-01T19:40:44.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.251+0000 7fb7248ab700 1 -- 172.21.15.17:0/1053923858 learned_addr learned my addr 172.21.15.17:0/1053923858 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:44.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.251+0000 7fb71ffff700 1 --2- 172.21.15.17:0/1053923858 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb7200a9da0 0x7fb720060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:44.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.251+0000 7fb7250ac700 1 --2- 172.21.15.17:0/1053923858 >> v2:172.21.15.77:3300/0 conn(0x7fb7201434b0 0x7fb720143940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:44.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.251+0000 7fb7248ab700 1 -- 172.21.15.17:0/1053923858 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb7200a9da0 msgr2=0x7fb720060950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:44.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.251+0000 7fb7248ab700 1 --2- 172.21.15.17:0/1053923858 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb7200a9da0 0x7fb720060950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:44.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.251+0000 7fb7248ab700 1 -- 172.21.15.17:0/1053923858 >> v2:172.21.15.77:3300/0 conn(0x7fb7201434b0 msgr2=0x7fb720143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:44.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.251+0000 7fb7248ab700 1 --2- 172.21.15.17:0/1053923858 >> v2:172.21.15.77:3300/0 conn(0x7fb7201434b0 0x7fb720143940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:44.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.251+0000 7fb7248ab700 1 -- 172.21.15.17:0/1053923858 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7200b35b0 con 0x7fb7200b31d0 2022-08-01T19:40:44.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.251+0000 7fb7248ab700 1 --2- 172.21.15.17:0/1053923858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7200b31d0 0x7fb720066020 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7fb714002f70 tx=0x7fb714006eb0).ready entity=mon.0 client_cookie=b5d301cf08ee3a88 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:44.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.251+0000 7fb71f7fe700 1 -- 172.21.15.17:0/1053923858 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb71400d570 con 0x7fb7200b31d0 2022-08-01T19:40:44.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.251+0000 7fb71f7fe700 1 -- 172.21.15.17:0/1053923858 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb714000e90 con 0x7fb7200b31d0 2022-08-01T19:40:44.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.251+0000 7fb71f7fe700 1 -- 172.21.15.17:0/1053923858 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb71400d900 con 0x7fb7200b31d0 2022-08-01T19:40:44.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.251+0000 7fb7258ad700 1 -- 172.21.15.17:0/1053923858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7200b31d0 msgr2=0x7fb720066020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:44.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.251+0000 7fb7258ad700 1 --2- 172.21.15.17:0/1053923858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7200b31d0 0x7fb720066020 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7fb714002f70 tx=0x7fb714006eb0).stop 2022-08-01T19:40:44.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.251+0000 7fb7258ad700 1 -- 172.21.15.17:0/1053923858 shutdown_connections 2022-08-01T19:40:44.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.251+0000 7fb7258ad700 1 --2- 172.21.15.17:0/1053923858 >> v2:172.21.15.77:3300/0 conn(0x7fb7201434b0 0x7fb720143940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:44.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.251+0000 7fb7258ad700 1 --2- 172.21.15.17:0/1053923858 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb7200a9da0 0x7fb720060950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:44.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.251+0000 7fb7258ad700 1 --2- 172.21.15.17:0/1053923858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7200b31d0 0x7fb720066020 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:44.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.251+0000 7fb7258ad700 1 -- 172.21.15.17:0/1053923858 >> 172.21.15.17:0/1053923858 conn(0x7fb720063870 msgr2=0x7fb720063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:44.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.255+0000 7fb7258ad700 1 -- 172.21.15.17:0/1053923858 shutdown_connections 2022-08-01T19:40:44.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.255+0000 7fb7258ad700 1 -- 172.21.15.17:0/1053923858 wait complete. 2022-08-01T19:40:44.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.255+0000 7fb7258ad700 1 Processor -- start 2022-08-01T19:40:44.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.255+0000 7fb7258ad700 1 -- start start 2022-08-01T19:40:44.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.255+0000 7fb7258ad700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7200a9da0 0x7fb7200aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:44.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.255+0000 7fb7258ad700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7200b31d0 0x7fb7200ae190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:44.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.255+0000 7fb7258ad700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb7201434b0 0x7fb7200ab580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:44.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.255+0000 7fb7258ad700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7200ae680 con 0x7fb7200a9da0 2022-08-01T19:40:44.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.255+0000 7fb7258ad700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7200ad520 con 0x7fb7200b31d0 2022-08-01T19:40:44.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.255+0000 7fb7258ad700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7200b02f0 con 0x7fb7201434b0 2022-08-01T19:40:44.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.255+0000 7fb7248ab700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7200a9da0 0x7fb7200aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:44.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.255+0000 7fb7248ab700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7200a9da0 0x7fb7200aeab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40826/0 (socket says 172.21.15.17:40826) 2022-08-01T19:40:44.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.255+0000 7fb7248ab700 1 -- 172.21.15.17:0/3307263677 learned_addr learned my addr 172.21.15.17:0/3307263677 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:44.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.255+0000 7fb71ffff700 1 --2- 172.21.15.17:0/3307263677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7200b31d0 0x7fb7200ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:44.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.255+0000 7fb7250ac700 1 --2- 172.21.15.17:0/3307263677 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb7201434b0 0x7fb7200ab580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:44.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.255+0000 7fb7248ab700 1 -- 172.21.15.17:0/3307263677 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb7201434b0 msgr2=0x7fb7200ab580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:44.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.255+0000 7fb7248ab700 1 --2- 172.21.15.17:0/3307263677 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb7201434b0 0x7fb7200ab580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:44.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.255+0000 7fb7248ab700 1 -- 172.21.15.17:0/3307263677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7200b31d0 msgr2=0x7fb7200ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:44.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.255+0000 7fb7248ab700 1 --2- 172.21.15.17:0/3307263677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7200b31d0 0x7fb7200ae190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:44.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.255+0000 7fb7248ab700 1 -- 172.21.15.17:0/3307263677 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb71400b1e0 con 0x7fb7200a9da0 2022-08-01T19:40:44.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.255+0000 7fb71ffff700 1 --2- 172.21.15.17:0/3307263677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7200b31d0 0x7fb7200ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:40:44.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.259+0000 7fb7248ab700 1 --2- 172.21.15.17:0/3307263677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7200a9da0 0x7fb7200aeab0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7fb714002f70 tx=0x7fb714015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:44.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.259+0000 7fb71dffb700 1 -- 172.21.15.17:0/3307263677 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7140117e0 con 0x7fb7200a9da0 2022-08-01T19:40:44.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.259+0000 7fb71dffb700 1 -- 172.21.15.17:0/3307263677 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb714011940 con 0x7fb7200a9da0 2022-08-01T19:40:44.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.259+0000 7fb7258ad700 1 -- 172.21.15.17:0/3307263677 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7200b0550 con 0x7fb7200a9da0 2022-08-01T19:40:44.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.259+0000 7fb71dffb700 1 -- 172.21.15.17:0/3307263677 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb714011ca0 con 0x7fb7200a9da0 2022-08-01T19:40:44.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.259+0000 7fb7258ad700 1 -- 172.21.15.17:0/3307263677 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7201533d0 con 0x7fb7200a9da0 2022-08-01T19:40:44.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.259+0000 7fb7258ad700 1 -- 172.21.15.17:0/3307263677 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6f8003410 con 0x7fb7200a9da0 2022-08-01T19:40:44.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.259+0000 7fb71dffb700 1 -- 172.21.15.17:0/3307263677 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb714025030 con 0x7fb7200a9da0 2022-08-01T19:40:44.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.259+0000 7fb71dffb700 1 --2- 172.21.15.17:0/3307263677 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb700053980 0x7fb700055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:44.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.259+0000 7fb71dffb700 1 -- 172.21.15.17:0/3307263677 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6463+0+0 (secure 0 0 0) 0x7fb71406d0e0 con 0x7fb7200a9da0 2022-08-01T19:40:44.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.259+0000 7fb71ffff700 1 --2- 172.21.15.17:0/3307263677 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb700053980 0x7fb700055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:44.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.263+0000 7fb71ffff700 1 --2- 172.21.15.17:0/3307263677 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb700053980 0x7fb700055e40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fb71000b000 tx=0x7fb710006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:44.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.267+0000 7fb71dffb700 1 -- 172.21.15.17:0/3307263677 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb714043e80 con 0x7fb7200a9da0 2022-08-01T19:40:44.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.407+0000 7fb7258ad700 1 -- 172.21.15.17:0/3307263677 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blacklist ls", "format": "json"} v 0) v1 -- 0x7fb6f80036c0 con 0x7fb7200a9da0 2022-08-01T19:40:44.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.411+0000 7fb71dffb700 1 -- 172.21.15.17:0/3307263677 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist ls", "format": "json"}]=0 listed 1 entries v77) v1 ==== 98+0+80 (secure 0 0 0) 0x7fb714023090 con 0x7fb7200a9da0 2022-08-01T19:40:44.414 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:40:44.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.411+0000 7fb7258ad700 1 -- 172.21.15.17:0/3307263677 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb700053980 msgr2=0x7fb700055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:44.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.411+0000 7fb7258ad700 1 --2- 172.21.15.17:0/3307263677 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb700053980 0x7fb700055e40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fb71000b000 tx=0x7fb710006df0).stop 2022-08-01T19:40:44.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.411+0000 7fb7258ad700 1 -- 172.21.15.17:0/3307263677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7200a9da0 msgr2=0x7fb7200aeab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:44.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.411+0000 7fb7258ad700 1 --2- 172.21.15.17:0/3307263677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7200a9da0 0x7fb7200aeab0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7fb714002f70 tx=0x7fb714015000).stop 2022-08-01T19:40:44.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.415+0000 7fb7258ad700 1 -- 172.21.15.17:0/3307263677 shutdown_connections 2022-08-01T19:40:44.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.415+0000 7fb7258ad700 1 --2- 172.21.15.17:0/3307263677 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb700053980 0x7fb700055e40 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:44.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.415+0000 7fb7258ad700 1 --2- 172.21.15.17:0/3307263677 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb7201434b0 0x7fb7200ab580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:44.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.415+0000 7fb7258ad700 1 --2- 172.21.15.17:0/3307263677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7200b31d0 0x7fb7200ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:44.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.415+0000 7fb7258ad700 1 --2- 172.21.15.17:0/3307263677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7200a9da0 0x7fb7200aeab0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:44.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.415+0000 7fb7258ad700 1 -- 172.21.15.17:0/3307263677 >> 172.21.15.17:0/3307263677 conn(0x7fb720063870 msgr2=0x7fb720142b10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:44.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.415+0000 7fb7258ad700 1 -- 172.21.15.17:0/3307263677 shutdown_connections 2022-08-01T19:40:44.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.415+0000 7fb7258ad700 1 -- 172.21.15.17:0/3307263677 wait complete. 2022-08-01T19:40:44.428 INFO:teuthology.orchestra.run.smithi017.stderr:listed 1 entries 2022-08-01T19:40:44.434 INFO:teuthology.orchestra.run.smithi017.stdout:[{"addr":"172.21.15.85:0/3805297491","until":"2022-08-01T20:40:38.016590+0000"}] 2022-08-01T19:40:44.451 INFO:tasks.cephfs.fuse_mount:Cleaning up killed ceph-fuse connection 2022-08-01T19:40:44.452 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:40:44.453 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:40:44.453 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:40:44.500 INFO:teuthology.orchestra.run.smithi085.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2022-08-01T19:40:44.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:40:44.502 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2022-08-01T19:40:44.503 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi085.front.sepia.ceph.com... 2022-08-01T19:40:44.503 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T19:40:44.504 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:40:44.523 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:40:44.524 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:40:44.524 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:40:44.533 INFO:teuthology.orchestra.run.smithi085.stdout:ext2/ext3 2022-08-01T19:40:44.534 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T19:40:44.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T19:40:44.535 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:40:44.590 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:40:44.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.699+0000 7fbabbbaf700 1 Processor -- start 2022-08-01T19:40:44.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.699+0000 7fbabbbaf700 1 -- start start 2022-08-01T19:40:44.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.699+0000 7fbabbbaf700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fbab40b5d80 0x7fbab40b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:44.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.699+0000 7fbabbbaf700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab4144240 con 0x7fbab40b4550 2022-08-01T19:40:44.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.699+0000 7fbabbbaf700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab40b6fe0 con 0x7fbab40b6710 2022-08-01T19:40:44.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.699+0000 7fbabbbaf700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbab4143d00 con 0x7fbab40b5d80 2022-08-01T19:40:44.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.699+0000 7fbab914a700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fbab40b4550 0x7fbab412bb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33028/0 (socket says 172.21.15.17:33028) 2022-08-01T19:40:44.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.699+0000 7fbab914a700 1 -- 172.21.15.17:0/3183344569 learned_addr learned my addr 172.21.15.17:0/3183344569 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:40:44.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.699+0000 7fbaba14c700 1 --2- 172.21.15.17:0/3183344569 >> v2:172.21.15.77:3300/0 conn(0x7fbab40b5d80 0x7fbab40b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:44.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.699+0000 7fbab8949700 1 -- 172.21.15.17:0/3183344569 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1417702577 0 0) 0x7fbab4144240 con 0x7fbab40b4550 2022-08-01T19:40:44.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.699+0000 7fbab8949700 1 -- 172.21.15.17:0/3183344569 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba94003260 con 0x7fbab40b4550 2022-08-01T19:40:44.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.699+0000 7fbab8949700 1 -- 172.21.15.17:0/3183344569 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 888482556 0 0) 0x7fbab40b6fe0 con 0x7fbab40b6710 2022-08-01T19:40:44.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.699+0000 7fbab8949700 1 -- 172.21.15.17:0/3183344569 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbab4144240 con 0x7fbab40b6710 2022-08-01T19:40:44.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.699+0000 7fbaba14c700 1 -- 172.21.15.17:0/3183344569 >> v1:172.21.15.77:6789/0 conn(0x7fbab40b6710 legacy=0x7fbab4061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:44.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.699+0000 7fbaba14c700 1 -- 172.21.15.17:0/3183344569 >> v1:172.21.15.17:6789/0 conn(0x7fbab40b4550 legacy=0x7fbab412bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:44.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.699+0000 7fbaba14c700 1 -- 172.21.15.17:0/3183344569 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab4143fa0 con 0x7fbab40b5d80 2022-08-01T19:40:44.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.699+0000 7fbaba14c700 1 --2- 172.21.15.17:0/3183344569 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbab40b5d80 0x7fbab40b6af0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7fbaac002f70 tx=0x7fbaac006df0).ready entity=mon.1 client_cookie=9be252bcdfc50d13 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:44.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.699+0000 7fbab8949700 1 -- 172.21.15.17:0/3183344569 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbaac00d540 con 0x7fbab40b5d80 2022-08-01T19:40:44.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.699+0000 7fbab8949700 1 -- 172.21.15.17:0/3183344569 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaac000e90 con 0x7fbab40b5d80 2022-08-01T19:40:44.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.699+0000 7fbab8949700 1 -- 172.21.15.17:0/3183344569 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbaac00d890 con 0x7fbab40b5d80 2022-08-01T19:40:44.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.703+0000 7fbabbbaf700 1 -- 172.21.15.17:0/3183344569 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbab40b5d80 msgr2=0x7fbab40b6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:44.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.703+0000 7fbabbbaf700 1 --2- 172.21.15.17:0/3183344569 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbab40b5d80 0x7fbab40b6af0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7fbaac002f70 tx=0x7fbaac006df0).stop 2022-08-01T19:40:44.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.703+0000 7fbabbbaf700 1 -- 172.21.15.17:0/3183344569 shutdown_connections 2022-08-01T19:40:44.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.703+0000 7fbabbbaf700 1 --2- 172.21.15.17:0/3183344569 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbab40b5d80 0x7fbab40b6af0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:44.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.703+0000 7fbabbbaf700 1 -- 172.21.15.17:0/3183344569 >> 172.21.15.17:0/3183344569 conn(0x7fbab405c620 msgr2=0x7fbab405ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:44.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.703+0000 7fbabbbaf700 1 -- 172.21.15.17:0/3183344569 shutdown_connections 2022-08-01T19:40:44.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.703+0000 7fbabbbaf700 1 -- 172.21.15.17:0/3183344569 wait complete. 2022-08-01T19:40:44.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.703+0000 7fbabbbaf700 1 Processor -- start 2022-08-01T19:40:44.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.703+0000 7fbabbbaf700 1 -- start start 2022-08-01T19:40:44.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.703+0000 7fbabbbaf700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbab40b4550 0x7fbab4142da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:44.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.703+0000 7fbabbbaf700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbab40b5d80 0x7fbab4128cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:44.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.703+0000 7fbabbbaf700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbab40b6710 0x7fbab4126890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:44.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.707+0000 7fbabbbaf700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab4147e40 con 0x7fbab40b6710 2022-08-01T19:40:44.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.707+0000 7fbabbbaf700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab4142910 con 0x7fbab40b4550 2022-08-01T19:40:44.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.707+0000 7fbabbbaf700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbab4129bf0 con 0x7fbab40b5d80 2022-08-01T19:40:44.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.707+0000 7fbaba14c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbab40b6710 0x7fbab4126890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:44.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.707+0000 7fbaba14c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbab40b6710 0x7fbab4126890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40844/0 (socket says 172.21.15.17:40844) 2022-08-01T19:40:44.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.707+0000 7fbaba14c700 1 -- 172.21.15.17:0/3114362860 learned_addr learned my addr 172.21.15.17:0/3114362860 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:44.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.707+0000 7fbab914a700 1 --2- 172.21.15.17:0/3114362860 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbab40b5d80 0x7fbab4128cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:44.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.707+0000 7fbab994b700 1 --2- 172.21.15.17:0/3114362860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbab40b4550 0x7fbab4142da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:44.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.707+0000 7fbaba14c700 1 -- 172.21.15.17:0/3114362860 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbab40b5d80 msgr2=0x7fbab4128cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:44.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.707+0000 7fbaba14c700 1 --2- 172.21.15.17:0/3114362860 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbab40b5d80 0x7fbab4128cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:44.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.707+0000 7fbaba14c700 1 -- 172.21.15.17:0/3114362860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbab40b4550 msgr2=0x7fbab4142da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:44.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.707+0000 7fbaba14c700 1 --2- 172.21.15.17:0/3114362860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbab40b4550 0x7fbab4142da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:44.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.707+0000 7fbaba14c700 1 -- 172.21.15.17:0/3114362860 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaac00b180 con 0x7fbab40b6710 2022-08-01T19:40:44.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.707+0000 7fbaba14c700 1 --2- 172.21.15.17:0/3114362860 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbab40b6710 0x7fbab4126890 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7fbaac002f70 tx=0x7fbaac014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:44.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.707+0000 7fbab2ffd700 1 -- 172.21.15.17:0/3114362860 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbaac016070 con 0x7fbab40b6710 2022-08-01T19:40:44.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.707+0000 7fbab2ffd700 1 -- 172.21.15.17:0/3114362860 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaac0137d0 con 0x7fbab40b6710 2022-08-01T19:40:44.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.707+0000 7fbabbbaf700 1 -- 172.21.15.17:0/3114362860 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbab4142b00 con 0x7fbab40b6710 2022-08-01T19:40:44.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.707+0000 7fbab2ffd700 1 -- 172.21.15.17:0/3114362860 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbaac013b30 con 0x7fbab40b6710 2022-08-01T19:40:44.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.707+0000 7fbabbbaf700 1 -- 172.21.15.17:0/3114362860 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab4126130 con 0x7fbab40b6710 2022-08-01T19:40:44.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.707+0000 7fbabbbaf700 1 -- 172.21.15.17:0/3114362860 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbab4126f20 con 0x7fbab40b6710 2022-08-01T19:40:44.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.711+0000 7fbab2ffd700 1 -- 172.21.15.17:0/3114362860 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbaac025030 con 0x7fbab40b6710 2022-08-01T19:40:44.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.711+0000 7fbab2ffd700 1 --2- 172.21.15.17:0/3114362860 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fba98053aa0 0x7fba98055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:44.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.711+0000 7fbab2ffd700 1 -- 172.21.15.17:0/3114362860 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6463+0+0 (secure 0 0 0) 0x7fbaac06cef0 con 0x7fbab40b6710 2022-08-01T19:40:44.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.711+0000 7fbab994b700 1 --2- 172.21.15.17:0/3114362860 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fba98053aa0 0x7fba98055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:44.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.711+0000 7fbab994b700 1 --2- 172.21.15.17:0/3114362860 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fba98053aa0 0x7fba98055f60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fba9c00d000 tx=0x7fba9c007a00).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:44.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.715+0000 7fbab2ffd700 1 -- 172.21.15.17:0/3114362860 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbaac043c90 con 0x7fbab40b6710 2022-08-01T19:40:44.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.839+0000 7fbabbbaf700 1 -- 172.21.15.17:0/3114362860 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbab4061780 con 0x7fbab40b6710 2022-08-01T19:40:44.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.843+0000 7fbab2ffd700 1 -- 172.21.15.17:0/3114362860 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+16335 (secure 0 0 0) 0x7fbaac023020 con 0x7fbab40b6710 2022-08-01T19:40:44.853 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:40:44.854 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":77,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:40:38.195204+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:39:50.853967+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2022-08-01T19:39:51.901692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.15.85:0/3805297491":"2022-08-01T20:40:38.016590+0000"},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:40:44.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.843+0000 7fbabbbaf700 1 -- 172.21.15.17:0/3114362860 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fba98053aa0 msgr2=0x7fba98055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:44.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.843+0000 7fbabbbaf700 1 --2- 172.21.15.17:0/3114362860 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fba98053aa0 0x7fba98055f60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fba9c00d000 tx=0x7fba9c007a00).stop 2022-08-01T19:40:44.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.843+0000 7fbabbbaf700 1 -- 172.21.15.17:0/3114362860 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbab40b6710 msgr2=0x7fbab4126890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:44.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.843+0000 7fbabbbaf700 1 --2- 172.21.15.17:0/3114362860 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbab40b6710 0x7fbab4126890 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7fbaac002f70 tx=0x7fbaac014000).stop 2022-08-01T19:40:44.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.843+0000 7fbabbbaf700 1 -- 172.21.15.17:0/3114362860 shutdown_connections 2022-08-01T19:40:44.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.843+0000 7fbabbbaf700 1 --2- 172.21.15.17:0/3114362860 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fba98053aa0 0x7fba98055f60 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:44.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.843+0000 7fbabbbaf700 1 --2- 172.21.15.17:0/3114362860 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbab40b6710 0x7fbab4126890 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:44.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.843+0000 7fbabbbaf700 1 --2- 172.21.15.17:0/3114362860 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbab40b5d80 0x7fbab4128cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:44.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.843+0000 7fbabbbaf700 1 --2- 172.21.15.17:0/3114362860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbab40b4550 0x7fbab4142da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:44.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.843+0000 7fbabbbaf700 1 -- 172.21.15.17:0/3114362860 >> 172.21.15.17:0/3114362860 conn(0x7fbab405c620 msgr2=0x7fbab4061fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:44.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.843+0000 7fbabbbaf700 1 -- 172.21.15.17:0/3114362860 shutdown_connections 2022-08-01T19:40:44.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.843+0000 7fbabbbaf700 1 -- 172.21.15.17:0/3114362860 wait complete. 2022-08-01T19:40:44.865 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T19:39:50.853967+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T19:39:51.901692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T19:40:44.867 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:40:44.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.975+0000 7fab28804700 1 Processor -- start 2022-08-01T19:40:44.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.975+0000 7fab28804700 1 -- start start 2022-08-01T19:40:44.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.975+0000 7fab28804700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fab240acd00 0x7fab240b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:44.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.975+0000 7fab28804700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab240ac950 con 0x7fab24140d30 2022-08-01T19:40:44.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.975+0000 7fab28804700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab240ac1b0 con 0x7fab240b1250 2022-08-01T19:40:44.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.975+0000 7fab28804700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab241411a0 con 0x7fab240acd00 2022-08-01T19:40:44.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.975+0000 7fab22d9d700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fab24140d30 0x7fab240b2670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33046/0 (socket says 172.21.15.17:33046) 2022-08-01T19:40:44.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.975+0000 7fab22d9d700 1 -- 172.21.15.17:0/3293902007 learned_addr learned my addr 172.21.15.17:0/3293902007 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:40:44.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.975+0000 7fab1bfff700 1 --2- 172.21.15.17:0/3293902007 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fab240acd00 0x7fab240b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:44.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.979+0000 7fab21d9b700 1 -- 172.21.15.17:0/3293902007 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1425110260 0 0) 0x7fab240ac950 con 0x7fab24140d30 2022-08-01T19:40:44.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.979+0000 7fab21d9b700 1 -- 172.21.15.17:0/3293902007 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab04003260 con 0x7fab24140d30 2022-08-01T19:40:44.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.979+0000 7fab1bfff700 1 -- 172.21.15.17:0/3293902007 >> v1:172.21.15.77:6789/0 conn(0x7fab240b1250 legacy=0x7fab2413d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:44.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.979+0000 7fab1bfff700 1 -- 172.21.15.17:0/3293902007 >> v1:172.21.15.17:6789/0 conn(0x7fab24140d30 legacy=0x7fab240b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:44.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.979+0000 7fab1bfff700 1 -- 172.21.15.17:0/3293902007 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab240ac410 con 0x7fab240acd00 2022-08-01T19:40:44.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.979+0000 7fab1bfff700 1 --2- 172.21.15.17:0/3293902007 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fab240acd00 0x7fab240b1630 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fab0c002f70 tx=0x7fab0c006df0).ready entity=mon.2 client_cookie=85760aedfbb9473d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:44.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.979+0000 7fab21d9b700 1 -- 172.21.15.17:0/3293902007 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab0c00d5c0 con 0x7fab240acd00 2022-08-01T19:40:44.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.979+0000 7fab21d9b700 1 -- 172.21.15.17:0/3293902007 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0c000e90 con 0x7fab240acd00 2022-08-01T19:40:44.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.979+0000 7fab21d9b700 1 -- 172.21.15.17:0/3293902007 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab0c00d910 con 0x7fab240acd00 2022-08-01T19:40:44.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.979+0000 7fab28804700 1 -- 172.21.15.17:0/3293902007 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fab240acd00 msgr2=0x7fab240b1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:44.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.979+0000 7fab28804700 1 --2- 172.21.15.17:0/3293902007 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fab240acd00 0x7fab240b1630 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fab0c002f70 tx=0x7fab0c006df0).stop 2022-08-01T19:40:44.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.979+0000 7fab28804700 1 -- 172.21.15.17:0/3293902007 shutdown_connections 2022-08-01T19:40:44.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.979+0000 7fab28804700 1 --2- 172.21.15.17:0/3293902007 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fab240acd00 0x7fab240b1630 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:44.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.979+0000 7fab28804700 1 -- 172.21.15.17:0/3293902007 >> 172.21.15.17:0/3293902007 conn(0x7fab2405f840 msgr2=0x7fab2405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:44.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.979+0000 7fab28804700 1 -- 172.21.15.17:0/3293902007 shutdown_connections 2022-08-01T19:40:44.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.979+0000 7fab28804700 1 -- 172.21.15.17:0/3293902007 wait complete. 2022-08-01T19:40:44.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.979+0000 7fab28804700 1 Processor -- start 2022-08-01T19:40:44.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.979+0000 7fab28804700 1 -- start start 2022-08-01T19:40:44.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.983+0000 7fab28804700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fab240acd00 0x7fab240aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:44.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.983+0000 7fab28804700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab240b1250 0x7fab24152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:44.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.983+0000 7fab28804700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab24140d30 0x7fab241533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:44.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.983+0000 7fab28804700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2413aac0 con 0x7fab240b1250 2022-08-01T19:40:44.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.983+0000 7fab28804700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2413acf0 con 0x7fab24140d30 2022-08-01T19:40:44.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.983+0000 7fab28804700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab2413af90 con 0x7fab240acd00 2022-08-01T19:40:44.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.983+0000 7fab1bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab240b1250 0x7fab24152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:44.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.983+0000 7fab1bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab240b1250 0x7fab24152ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40856/0 (socket says 172.21.15.17:40856) 2022-08-01T19:40:44.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.983+0000 7fab1bfff700 1 -- 172.21.15.17:0/3746220070 learned_addr learned my addr 172.21.15.17:0/3746220070 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:44.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.983+0000 7fab22d9d700 1 --2- 172.21.15.17:0/3746220070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab24140d30 0x7fab241533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:44.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.983+0000 7fab2259c700 1 --2- 172.21.15.17:0/3746220070 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fab240acd00 0x7fab240aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:44.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.983+0000 7fab1bfff700 1 -- 172.21.15.17:0/3746220070 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fab240acd00 msgr2=0x7fab240aaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:45.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.983+0000 7fab1bfff700 1 --2- 172.21.15.17:0/3746220070 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fab240acd00 0x7fab240aaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:45.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.983+0000 7fab1bfff700 1 -- 172.21.15.17:0/3746220070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab24140d30 msgr2=0x7fab241533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:45.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.983+0000 7fab1bfff700 1 --2- 172.21.15.17:0/3746220070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab24140d30 0x7fab241533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:45.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.983+0000 7fab1bfff700 1 -- 172.21.15.17:0/3746220070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab0c00b180 con 0x7fab240b1250 2022-08-01T19:40:45.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.983+0000 7fab2259c700 1 --2- 172.21.15.17:0/3746220070 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fab240acd00 0x7fab240aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:40:45.002 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.983+0000 7fab1bfff700 1 --2- 172.21.15.17:0/3746220070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab240b1250 0x7fab24152ef0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7fab0c002f70 tx=0x7fab0c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:45.002 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.983+0000 7fab1b7fe700 1 -- 172.21.15.17:0/3746220070 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab0c013e50 con 0x7fab240b1250 2022-08-01T19:40:45.003 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.983+0000 7fab1b7fe700 1 -- 172.21.15.17:0/3746220070 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0c020070 con 0x7fab240b1250 2022-08-01T19:40:45.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.983+0000 7fab28804700 1 -- 172.21.15.17:0/3746220070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab24152a50 con 0x7fab240b1250 2022-08-01T19:40:45.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.983+0000 7fab1b7fe700 1 -- 172.21.15.17:0/3746220070 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab0c019670 con 0x7fab240b1250 2022-08-01T19:40:45.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.983+0000 7fab28804700 1 -- 172.21.15.17:0/3746220070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab24139f00 con 0x7fab240b1250 2022-08-01T19:40:45.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.987+0000 7fab28804700 1 -- 172.21.15.17:0/3746220070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab2413b920 con 0x7fab240b1250 2022-08-01T19:40:45.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.987+0000 7fab1b7fe700 1 -- 172.21.15.17:0/3746220070 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fab0c02c070 con 0x7fab240b1250 2022-08-01T19:40:45.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.987+0000 7fab1b7fe700 1 --2- 172.21.15.17:0/3746220070 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fab080539e0 0x7fab08055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:45.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.987+0000 7fab1b7fe700 1 -- 172.21.15.17:0/3746220070 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6463+0+0 (secure 0 0 0) 0x7fab0c06fcf0 con 0x7fab240b1250 2022-08-01T19:40:45.007 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.987+0000 7fab2259c700 1 --2- 172.21.15.17:0/3746220070 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fab080539e0 0x7fab08055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:45.007 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.987+0000 7fab2259c700 1 --2- 172.21.15.17:0/3746220070 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fab080539e0 0x7fab08055ea0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7fab1400b000 tx=0x7fab14009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:45.008 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:44.995+0000 7fab1b7fe700 1 -- 172.21.15.17:0/3746220070 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fab2413b920 con 0x7fab240b1250 2022-08-01T19:40:45.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.123+0000 7fab28804700 1 -- 172.21.15.17:0/3746220070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fab2413b920 con 0x7fab240b1250 2022-08-01T19:40:45.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.123+0000 7fab1b7fe700 1 -- 172.21.15.17:0/3746220070 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v77) v1 ==== 110+0+10 (secure 0 0 0) 0x7fab2413b920 con 0x7fab240b1250 2022-08-01T19:40:45.134 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:40:45.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.127+0000 7fab28804700 1 -- 172.21.15.17:0/3746220070 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fab080539e0 msgr2=0x7fab08055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:45.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.127+0000 7fab28804700 1 --2- 172.21.15.17:0/3746220070 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fab080539e0 0x7fab08055ea0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7fab1400b000 tx=0x7fab14009040).stop 2022-08-01T19:40:45.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.127+0000 7fab28804700 1 -- 172.21.15.17:0/3746220070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab240b1250 msgr2=0x7fab24152ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:45.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.127+0000 7fab28804700 1 --2- 172.21.15.17:0/3746220070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab240b1250 0x7fab24152ef0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7fab0c002f70 tx=0x7fab0c014000).stop 2022-08-01T19:40:45.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.127+0000 7fab28804700 1 -- 172.21.15.17:0/3746220070 shutdown_connections 2022-08-01T19:40:45.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.127+0000 7fab28804700 1 --2- 172.21.15.17:0/3746220070 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fab080539e0 0x7fab08055ea0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:45.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.127+0000 7fab28804700 1 --2- 172.21.15.17:0/3746220070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab24140d30 0x7fab241533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:45.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.127+0000 7fab28804700 1 --2- 172.21.15.17:0/3746220070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab240b1250 0x7fab24152ef0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:45.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.127+0000 7fab28804700 1 --2- 172.21.15.17:0/3746220070 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fab240acd00 0x7fab240aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:45.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.127+0000 7fab28804700 1 -- 172.21.15.17:0/3746220070 >> 172.21.15.17:0/3746220070 conn(0x7fab2405f840 msgr2=0x7fab2405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:45.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.127+0000 7fab28804700 1 -- 172.21.15.17:0/3746220070 shutdown_connections 2022-08-01T19:40:45.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.127+0000 7fab28804700 1 -- 172.21.15.17:0/3746220070 wait complete. 2022-08-01T19:40:45.158 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:40:45.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.271+0000 7f5e63fff700 1 Processor -- start 2022-08-01T19:40:45.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.271+0000 7f5e63fff700 1 -- start start 2022-08-01T19:40:45.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.271+0000 7f5e63fff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f5e6405c960 0x7f5e640b1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:45.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.271+0000 7f5e63fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5e6405d240 0x7f5e6413f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:45.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.271+0000 7f5e63fff700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e6413fa10 con 0x7f5e640acfc0 2022-08-01T19:40:45.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.271+0000 7f5e63fff700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e64067f00 con 0x7f5e6405d240 2022-08-01T19:40:45.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.271+0000 7f5e63fff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e640b2c80 con 0x7f5e6405c960 2022-08-01T19:40:45.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.271+0000 7f5e627fc700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5e6405d240 0x7f5e6413f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:45.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.271+0000 7f5e637fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5e640acfc0 0x7f5e640b2570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33060/0 (socket says 172.21.15.17:33060) 2022-08-01T19:40:45.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.271+0000 7f5e637fe700 1 -- 172.21.15.17:0/4023103437 learned_addr learned my addr 172.21.15.17:0/4023103437 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:40:45.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.271+0000 7f5e62ffd700 1 --2- 172.21.15.17:0/4023103437 >> v2:172.21.15.77:3300/0 conn(0x7f5e6405c960 0x7f5e640b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:45.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.271+0000 7f5e627fc700 1 -- 172.21.15.17:0/4023103437 >> v2:172.21.15.77:3300/0 conn(0x7f5e6405c960 msgr2=0x7f5e640b1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:45.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.271+0000 7f5e627fc700 1 --2- 172.21.15.17:0/4023103437 >> v2:172.21.15.77:3300/0 conn(0x7f5e6405c960 0x7f5e640b1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:45.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.271+0000 7f5e627fc700 1 -- 172.21.15.17:0/4023103437 >> v1:172.21.15.17:6789/0 conn(0x7f5e640acfc0 legacy=0x7f5e640b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:45.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.271+0000 7f5e627fc700 1 -- 172.21.15.17:0/4023103437 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e6405cd40 con 0x7f5e6405d240 2022-08-01T19:40:45.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.271+0000 7f5e61ffb700 1 -- 172.21.15.17:0/4023103437 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3320902113 0 0) 0x7f5e5c001490 con 0x7f5e640acfc0 2022-08-01T19:40:45.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.271+0000 7f5e627fc700 1 --2- 172.21.15.17:0/4023103437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e6405d240 0x7f5e6413f520 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7f5e640b2dc0 tx=0x7f5e54006df0).ready entity=mon.0 client_cookie=22ab9945550943ce server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:45.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.271+0000 7f5e61ffb700 1 -- 172.21.15.17:0/4023103437 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e5400d5c0 con 0x7f5e6405d240 2022-08-01T19:40:45.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.271+0000 7f5e61ffb700 1 -- 172.21.15.17:0/4023103437 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e54000e90 con 0x7f5e6405d240 2022-08-01T19:40:45.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.271+0000 7f5e61ffb700 1 -- 172.21.15.17:0/4023103437 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e5400d910 con 0x7f5e6405d240 2022-08-01T19:40:45.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.275+0000 7f5e63fff700 1 -- 172.21.15.17:0/4023103437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e6405d240 msgr2=0x7f5e6413f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:45.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.275+0000 7f5e63fff700 1 --2- 172.21.15.17:0/4023103437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e6405d240 0x7f5e6413f520 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7f5e640b2dc0 tx=0x7f5e54006df0).stop 2022-08-01T19:40:45.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.275+0000 7f5e63fff700 1 -- 172.21.15.17:0/4023103437 shutdown_connections 2022-08-01T19:40:45.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.275+0000 7f5e63fff700 1 --2- 172.21.15.17:0/4023103437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e6405d240 0x7f5e6413f520 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:45.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.275+0000 7f5e63fff700 1 --2- 172.21.15.17:0/4023103437 >> v2:172.21.15.77:3300/0 conn(0x7f5e6405c960 0x7f5e640b1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:45.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.275+0000 7f5e63fff700 1 -- 172.21.15.17:0/4023103437 >> 172.21.15.17:0/4023103437 conn(0x7f5e64060310 msgr2=0x7f5e640b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:45.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.275+0000 7f5e63fff700 1 -- 172.21.15.17:0/4023103437 shutdown_connections 2022-08-01T19:40:45.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.275+0000 7f5e63fff700 1 -- 172.21.15.17:0/4023103437 wait complete. 2022-08-01T19:40:45.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.275+0000 7f5e63fff700 1 Processor -- start 2022-08-01T19:40:45.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.275+0000 7f5e63fff700 1 -- start start 2022-08-01T19:40:45.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.275+0000 7f5e63fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e6405c960 0x7f5e6409ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:45.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.275+0000 7f5e63fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5e6405d240 0x7f5e640aa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:45.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.275+0000 7f5e63fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e640acfc0 0x7f5e640aa8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:45.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.275+0000 7f5e63fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e64139560 con 0x7f5e640acfc0 2022-08-01T19:40:45.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.275+0000 7f5e63fff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e640ac090 con 0x7f5e6405c960 2022-08-01T19:40:45.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.275+0000 7f5e63fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e641530a0 con 0x7f5e6405d240 2022-08-01T19:40:45.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.275+0000 7f5e637fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e640acfc0 0x7f5e640aa8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:45.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.275+0000 7f5e637fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e640acfc0 0x7f5e640aa8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40872/0 (socket says 172.21.15.17:40872) 2022-08-01T19:40:45.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.275+0000 7f5e637fe700 1 -- 172.21.15.17:0/1392955416 learned_addr learned my addr 172.21.15.17:0/1392955416 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:45.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.275+0000 7f5e62ffd700 1 --2- 172.21.15.17:0/1392955416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e6405c960 0x7f5e6409ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:45.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.275+0000 7f5e627fc700 1 --2- 172.21.15.17:0/1392955416 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5e6405d240 0x7f5e640aa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:45.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.279+0000 7f5e637fe700 1 -- 172.21.15.17:0/1392955416 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5e6405d240 msgr2=0x7f5e640aa3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:45.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.279+0000 7f5e637fe700 1 --2- 172.21.15.17:0/1392955416 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5e6405d240 0x7f5e640aa3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:45.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.279+0000 7f5e637fe700 1 -- 172.21.15.17:0/1392955416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e6405c960 msgr2=0x7f5e6409ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:45.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.279+0000 7f5e637fe700 1 --2- 172.21.15.17:0/1392955416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e6405c960 0x7f5e6409ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:45.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.279+0000 7f5e637fe700 1 -- 172.21.15.17:0/1392955416 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e5400b180 con 0x7f5e640acfc0 2022-08-01T19:40:45.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.279+0000 7f5e627fc700 1 --2- 172.21.15.17:0/1392955416 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5e6405d240 0x7f5e640aa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:40:45.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.279+0000 7f5e62ffd700 1 --2- 172.21.15.17:0/1392955416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e6405c960 0x7f5e6409ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:40:45.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.279+0000 7f5e637fe700 1 --2- 172.21.15.17:0/1392955416 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e640acfc0 0x7f5e640aa8a0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0x7f5e5c001250 tx=0x7f5e5c001360).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:45.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.279+0000 7f5e5bfff700 1 -- 172.21.15.17:0/1392955416 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e5c005ba0 con 0x7f5e640acfc0 2022-08-01T19:40:45.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.279+0000 7f5e5bfff700 1 -- 172.21.15.17:0/1392955416 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e5c005d00 con 0x7f5e640acfc0 2022-08-01T19:40:45.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.279+0000 7f5e63fff700 1 -- 172.21.15.17:0/1392955416 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e64153340 con 0x7f5e640acfc0 2022-08-01T19:40:45.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.279+0000 7f5e5bfff700 1 -- 172.21.15.17:0/1392955416 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e5c004a90 con 0x7f5e640acfc0 2022-08-01T19:40:45.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.279+0000 7f5e63fff700 1 -- 172.21.15.17:0/1392955416 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e640ab780 con 0x7f5e640acfc0 2022-08-01T19:40:45.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.279+0000 7f5e63fff700 1 -- 172.21.15.17:0/1392955416 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e641412b0 con 0x7f5e640acfc0 2022-08-01T19:40:45.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.279+0000 7f5e5bfff700 1 -- 172.21.15.17:0/1392955416 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5e5c004bf0 con 0x7f5e640acfc0 2022-08-01T19:40:45.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.283+0000 7f5e5bfff700 1 --2- 172.21.15.17:0/1392955416 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5e48053a70 0x7f5e48055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:45.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.283+0000 7f5e5bfff700 1 -- 172.21.15.17:0/1392955416 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f5e5c06c160 con 0x7f5e640acfc0 2022-08-01T19:40:45.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.283+0000 7f5e62ffd700 1 --2- 172.21.15.17:0/1392955416 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5e48053a70 0x7f5e48055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:45.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.283+0000 7f5e62ffd700 1 --2- 172.21.15.17:0/1392955416 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5e48053a70 0x7f5e48055f30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f5e4c008000 tx=0x7f5e4c008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:45.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.287+0000 7f5e5bfff700 1 -- 172.21.15.17:0/1392955416 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5e5c042df0 con 0x7f5e640acfc0 2022-08-01T19:40:45.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.411+0000 7f5e63fff700 1 -- 172.21.15.17:0/1392955416 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5e6413fa10 con 0x7f5e640acfc0 2022-08-01T19:40:45.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.411+0000 7f5e5bfff700 1 -- 172.21.15.17:0/1392955416 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v77) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5e6413fa10 con 0x7f5e640acfc0 2022-08-01T19:40:45.417 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:40:45.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.415+0000 7f5e63fff700 1 -- 172.21.15.17:0/1392955416 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5e48053a70 msgr2=0x7f5e48055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:45.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.415+0000 7f5e63fff700 1 --2- 172.21.15.17:0/1392955416 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5e48053a70 0x7f5e48055f30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f5e4c008000 tx=0x7f5e4c008040).stop 2022-08-01T19:40:45.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.415+0000 7f5e63fff700 1 -- 172.21.15.17:0/1392955416 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e640acfc0 msgr2=0x7f5e640aa8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:45.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.415+0000 7f5e63fff700 1 --2- 172.21.15.17:0/1392955416 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e640acfc0 0x7f5e640aa8a0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0x7f5e5c001250 tx=0x7f5e5c001360).stop 2022-08-01T19:40:45.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.415+0000 7f5e63fff700 1 -- 172.21.15.17:0/1392955416 shutdown_connections 2022-08-01T19:40:45.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.415+0000 7f5e63fff700 1 --2- 172.21.15.17:0/1392955416 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5e48053a70 0x7f5e48055f30 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:45.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.415+0000 7f5e63fff700 1 --2- 172.21.15.17:0/1392955416 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e640acfc0 0x7f5e640aa8a0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:45.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.415+0000 7f5e63fff700 1 --2- 172.21.15.17:0/1392955416 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5e6405d240 0x7f5e640aa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:45.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.415+0000 7f5e63fff700 1 --2- 172.21.15.17:0/1392955416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e6405c960 0x7f5e6409ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:45.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.415+0000 7f5e63fff700 1 -- 172.21.15.17:0/1392955416 >> 172.21.15.17:0/1392955416 conn(0x7f5e64060310 msgr2=0x7f5e6413d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:45.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.415+0000 7f5e63fff700 1 -- 172.21.15.17:0/1392955416 shutdown_connections 2022-08-01T19:40:45.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.415+0000 7f5e63fff700 1 -- 172.21.15.17:0/1392955416 wait complete. 2022-08-01T19:40:45.435 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T19:40:45.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.539+0000 7f43e3088700 1 Processor -- start 2022-08-01T19:40:45.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.539+0000 7f43e3088700 1 -- start start 2022-08-01T19:40:45.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.539+0000 7f43e3088700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f43dc0647d0 0x7f43dc064bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:45.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.539+0000 7f43e3088700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f43dc0b38a0 0x7f43dc1470d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:45.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.539+0000 7f43e3088700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f43dc1436f0 0x7f43dc143b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:45.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.539+0000 7f43e3088700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43dc0b4f60 con 0x7f43dc1436f0 2022-08-01T19:40:45.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.539+0000 7f43e3088700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43dc1475c0 con 0x7f43dc0b38a0 2022-08-01T19:40:45.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.539+0000 7f43e3088700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc0b4a60 con 0x7f43dc0647d0 2022-08-01T19:40:45.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.539+0000 7f43e2887700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f43dc1436f0 0x7f43dc143b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:45.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.539+0000 7f43e2887700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f43dc1436f0 0x7f43dc143b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40880/0 (socket says 172.21.15.17:40880) 2022-08-01T19:40:45.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.539+0000 7f43e2887700 1 -- 172.21.15.17:0/2540502907 learned_addr learned my addr 172.21.15.17:0/2540502907 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:45.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.543+0000 7f43e1885700 1 --2- 172.21.15.17:0/2540502907 >> v2:172.21.15.77:3300/0 conn(0x7f43dc0b38a0 0x7f43dc1470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:45.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.543+0000 7f43e2086700 1 --2- 172.21.15.17:0/2540502907 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f43dc0647d0 0x7f43dc064bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:45.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.543+0000 7f43e2887700 1 -- 172.21.15.17:0/2540502907 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f43dc0647d0 msgr2=0x7f43dc064bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:45.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.543+0000 7f43e2887700 1 --2- 172.21.15.17:0/2540502907 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f43dc0647d0 0x7f43dc064bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:45.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.543+0000 7f43e2887700 1 -- 172.21.15.17:0/2540502907 >> v2:172.21.15.77:3300/0 conn(0x7f43dc0b38a0 msgr2=0x7f43dc1470d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:45.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.543+0000 7f43e2887700 1 --2- 172.21.15.17:0/2540502907 >> v2:172.21.15.77:3300/0 conn(0x7f43dc0b38a0 0x7f43dc1470d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:45.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.543+0000 7f43e2887700 1 -- 172.21.15.17:0/2540502907 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43dc0b4cc0 con 0x7f43dc1436f0 2022-08-01T19:40:45.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.543+0000 7f43e1885700 1 --2- 172.21.15.17:0/2540502907 >> v2:172.21.15.77:3300/0 conn(0x7f43dc0b38a0 0x7f43dc1470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:40:45.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.543+0000 7f43e2887700 1 --2- 172.21.15.17:0/2540502907 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43dc1436f0 0x7f43dc143b80 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7f43d40032f0 tx=0x7f43d4009290).ready entity=mon.0 client_cookie=bf08b8bfaf1278a5 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:45.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.543+0000 7f43e1084700 1 -- 172.21.15.17:0/2540502907 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43d400b870 con 0x7f43dc1436f0 2022-08-01T19:40:45.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.543+0000 7f43e1084700 1 -- 172.21.15.17:0/2540502907 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d400b9d0 con 0x7f43dc1436f0 2022-08-01T19:40:45.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.543+0000 7f43e1084700 1 -- 172.21.15.17:0/2540502907 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43d400bb40 con 0x7f43dc1436f0 2022-08-01T19:40:45.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.543+0000 7f43e3088700 1 -- 172.21.15.17:0/2540502907 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43dc1436f0 msgr2=0x7f43dc143b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:45.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.543+0000 7f43e3088700 1 --2- 172.21.15.17:0/2540502907 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43dc1436f0 0x7f43dc143b80 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7f43d40032f0 tx=0x7f43d4009290).stop 2022-08-01T19:40:45.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.543+0000 7f43e3088700 1 -- 172.21.15.17:0/2540502907 shutdown_connections 2022-08-01T19:40:45.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.543+0000 7f43e3088700 1 --2- 172.21.15.17:0/2540502907 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43dc1436f0 0x7f43dc143b80 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:45.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.543+0000 7f43e3088700 1 --2- 172.21.15.17:0/2540502907 >> v2:172.21.15.77:3300/0 conn(0x7f43dc0b38a0 0x7f43dc1470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:45.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.543+0000 7f43e3088700 1 --2- 172.21.15.17:0/2540502907 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f43dc0647d0 0x7f43dc064bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:45.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.543+0000 7f43e3088700 1 -- 172.21.15.17:0/2540502907 >> 172.21.15.17:0/2540502907 conn(0x7f43dc05f840 msgr2=0x7f43dc05fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:45.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.543+0000 7f43e3088700 1 -- 172.21.15.17:0/2540502907 shutdown_connections 2022-08-01T19:40:45.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.543+0000 7f43e3088700 1 -- 172.21.15.17:0/2540502907 wait complete. 2022-08-01T19:40:45.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.543+0000 7f43e3088700 1 Processor -- start 2022-08-01T19:40:45.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.543+0000 7f43e3088700 1 -- start start 2022-08-01T19:40:45.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.547+0000 7f43e3088700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43dc0647d0 0x7f43dc0aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:45.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.547+0000 7f43e3088700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43dc0b38a0 0x7f43dc155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:45.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.547+0000 7f43e3088700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f43dc1436f0 0x7f43dc155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:45.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.547+0000 7f43e3088700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc0b11d0 con 0x7f43dc0b38a0 2022-08-01T19:40:45.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.547+0000 7f43e3088700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc0b0420 con 0x7f43dc0647d0 2022-08-01T19:40:45.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.547+0000 7f43e3088700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc0b06c0 con 0x7f43dc1436f0 2022-08-01T19:40:45.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.547+0000 7f43e1885700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43dc0b38a0 0x7f43dc155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:45.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.547+0000 7f43e1885700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43dc0b38a0 0x7f43dc155a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40884/0 (socket says 172.21.15.17:40884) 2022-08-01T19:40:45.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.547+0000 7f43e1885700 1 -- 172.21.15.17:0/3251053310 learned_addr learned my addr 172.21.15.17:0/3251053310 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:45.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.547+0000 7f43e2086700 1 --2- 172.21.15.17:0/3251053310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43dc0647d0 0x7f43dc0aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:45.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.547+0000 7f43e2887700 1 --2- 172.21.15.17:0/3251053310 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f43dc1436f0 0x7f43dc155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:45.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.547+0000 7f43e1885700 1 -- 172.21.15.17:0/3251053310 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f43dc1436f0 msgr2=0x7f43dc155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:45.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.547+0000 7f43e1885700 1 --2- 172.21.15.17:0/3251053310 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f43dc1436f0 0x7f43dc155f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:45.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.547+0000 7f43e1885700 1 -- 172.21.15.17:0/3251053310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43dc0647d0 msgr2=0x7f43dc0aeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:45.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.547+0000 7f43e1885700 1 --2- 172.21.15.17:0/3251053310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43dc0647d0 0x7f43dc0aeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:45.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.547+0000 7f43e1885700 1 -- 172.21.15.17:0/3251053310 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d4009530 con 0x7f43dc0b38a0 2022-08-01T19:40:45.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.547+0000 7f43e2086700 1 --2- 172.21.15.17:0/3251053310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43dc0647d0 0x7f43dc0aeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:40:45.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.547+0000 7f43e1885700 1 --2- 172.21.15.17:0/3251053310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43dc0b38a0 0x7f43dc155a80 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7f43d8006940 tx=0x7f43d8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:45.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.547+0000 7f43cb7fe700 1 -- 172.21.15.17:0/3251053310 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43d8011780 con 0x7f43dc0b38a0 2022-08-01T19:40:45.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.547+0000 7f43cb7fe700 1 -- 172.21.15.17:0/3251053310 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d80118e0 con 0x7f43dc0b38a0 2022-08-01T19:40:45.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.547+0000 7f43cb7fe700 1 -- 172.21.15.17:0/3251053310 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43d8011c40 con 0x7f43dc0b38a0 2022-08-01T19:40:45.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.547+0000 7f43e3088700 1 -- 172.21.15.17:0/3251053310 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43dc0b0960 con 0x7f43dc0b38a0 2022-08-01T19:40:45.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.547+0000 7f43e3088700 1 -- 172.21.15.17:0/3251053310 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43dc1451f0 con 0x7f43dc0b38a0 2022-08-01T19:40:45.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.547+0000 7f43e3088700 1 -- 172.21.15.17:0/3251053310 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43dc143250 con 0x7f43dc0b38a0 2022-08-01T19:40:45.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.551+0000 7f43cb7fe700 1 -- 172.21.15.17:0/3251053310 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f43d8012420 con 0x7f43dc0b38a0 2022-08-01T19:40:45.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.551+0000 7f43cb7fe700 1 --2- 172.21.15.17:0/3251053310 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f43bc053950 0x7f43bc055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:45.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.551+0000 7f43cb7fe700 1 -- 172.21.15.17:0/3251053310 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f43d806c080 con 0x7f43dc0b38a0 2022-08-01T19:40:45.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.551+0000 7f43e2086700 1 --2- 172.21.15.17:0/3251053310 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f43bc053950 0x7f43bc055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:45.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.551+0000 7f43e2086700 1 --2- 172.21.15.17:0/3251053310 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f43bc053950 0x7f43bc055e10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f43cc00c000 tx=0x7f43cc008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:45.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.559+0000 7f43cb7fe700 1 -- 172.21.15.17:0/3251053310 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f43d8042d20 con 0x7f43dc0b38a0 2022-08-01T19:40:45.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.687+0000 7f43e3088700 1 -- 172.21.15.17:0/3251053310 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f43dc0574e0 con 0x7f43dc0b38a0 2022-08-01T19:40:45.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.687+0000 7f43cb7fe700 1 -- 172.21.15.17:0/3251053310 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) v1 ==== 104+0+10 (secure 0 0 0) 0x7f43d8046340 con 0x7f43dc0b38a0 2022-08-01T19:40:45.695 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:40:45.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.687+0000 7f43e3088700 1 -- 172.21.15.17:0/3251053310 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f43bc053950 msgr2=0x7f43bc055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:45.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.687+0000 7f43e3088700 1 --2- 172.21.15.17:0/3251053310 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f43bc053950 0x7f43bc055e10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f43cc00c000 tx=0x7f43cc008040).stop 2022-08-01T19:40:45.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.687+0000 7f43e3088700 1 -- 172.21.15.17:0/3251053310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43dc0b38a0 msgr2=0x7f43dc155a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:45.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.687+0000 7f43e3088700 1 --2- 172.21.15.17:0/3251053310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43dc0b38a0 0x7f43dc155a80 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7f43d8006940 tx=0x7f43d8008040).stop 2022-08-01T19:40:45.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.687+0000 7f43e3088700 1 -- 172.21.15.17:0/3251053310 shutdown_connections 2022-08-01T19:40:45.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.687+0000 7f43e3088700 1 --2- 172.21.15.17:0/3251053310 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f43bc053950 0x7f43bc055e10 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:45.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.687+0000 7f43e3088700 1 --2- 172.21.15.17:0/3251053310 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f43dc1436f0 0x7f43dc155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:45.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.687+0000 7f43e3088700 1 --2- 172.21.15.17:0/3251053310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43dc0b38a0 0x7f43dc155a80 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:45.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.687+0000 7f43e3088700 1 --2- 172.21.15.17:0/3251053310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f43dc0647d0 0x7f43dc0aeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:45.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.687+0000 7f43e3088700 1 -- 172.21.15.17:0/3251053310 >> 172.21.15.17:0/3251053310 conn(0x7f43dc05f840 msgr2=0x7f43dc141a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:45.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.687+0000 7f43e3088700 1 -- 172.21.15.17:0/3251053310 shutdown_connections 2022-08-01T19:40:45.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.687+0000 7f43e3088700 1 -- 172.21.15.17:0/3251053310 wait complete. 2022-08-01T19:40:45.710 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T19:40:45.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.819+0000 7fcebffff700 1 Processor -- start 2022-08-01T19:40:45.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.819+0000 7fcebffff700 1 -- start start 2022-08-01T19:40:45.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.819+0000 7fcebffff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcec00b3190 0x7fcec0060ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:45.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.819+0000 7fcebffff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fcec00b2730 0x7fcec0061190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:45.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.819+0000 7fcebffff700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcec00654c0 con 0x7fcec00666b0 2022-08-01T19:40:45.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.819+0000 7fcebffff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcec0062190 con 0x7fcec00b2730 2022-08-01T19:40:45.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.819+0000 7fcebffff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec00b3570 con 0x7fcec00b3190 2022-08-01T19:40:45.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.819+0000 7fcebeffd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcec00b3190 0x7fcec0060ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:45.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.823+0000 7fcebe7fc700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fcec00b2730 0x7fcec0061190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:45.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.823+0000 7fcebf7fe700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7fcec00666b0 0x7fcec0066b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:49888/0 (socket says 172.21.15.17:49888) 2022-08-01T19:40:45.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.823+0000 7fcebf7fe700 1 -- 172.21.15.17:0/966900558 learned_addr learned my addr 172.21.15.17:0/966900558 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:40:45.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.823+0000 7fcebe7fc700 1 -- 172.21.15.17:0/966900558 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcec00b3190 msgr2=0x7fcec0060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:45.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.823+0000 7fcebe7fc700 1 --2- 172.21.15.17:0/966900558 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcec00b3190 0x7fcec0060ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:45.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.823+0000 7fcebe7fc700 1 -- 172.21.15.17:0/966900558 >> v1:172.21.15.77:6789/0 conn(0x7fcec00666b0 legacy=0x7fcec0066b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:45.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.823+0000 7fcebe7fc700 1 -- 172.21.15.17:0/966900558 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec005fbf0 con 0x7fcec00b2730 2022-08-01T19:40:45.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.823+0000 7fcebdffb700 1 -- 172.21.15.17:0/966900558 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 921209999 0 0) 0x7fcec00654c0 con 0x7fcec00666b0 2022-08-01T19:40:45.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.823+0000 7fcebe7fc700 1 --2- 172.21.15.17:0/966900558 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcec00b2730 0x7fcec0061190 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7fceb0002f70 tx=0x7fceb0006df0).ready entity=mon.1 client_cookie=b572a5f19e11539e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:45.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.823+0000 7fcebdffb700 1 -- 172.21.15.17:0/966900558 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fceb000d570 con 0x7fcec00b2730 2022-08-01T19:40:45.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.823+0000 7fcebdffb700 1 -- 172.21.15.17:0/966900558 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb0000e90 con 0x7fcec00b2730 2022-08-01T19:40:45.834 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.823+0000 7fcebdffb700 1 -- 172.21.15.17:0/966900558 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fceb000d8c0 con 0x7fcec00b2730 2022-08-01T19:40:45.834 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.823+0000 7fcebffff700 1 -- 172.21.15.17:0/966900558 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcec00b2730 msgr2=0x7fcec0061190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:45.834 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.823+0000 7fcebffff700 1 --2- 172.21.15.17:0/966900558 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcec00b2730 0x7fcec0061190 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7fceb0002f70 tx=0x7fceb0006df0).stop 2022-08-01T19:40:45.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.823+0000 7fcebffff700 1 -- 172.21.15.17:0/966900558 shutdown_connections 2022-08-01T19:40:45.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.823+0000 7fcebffff700 1 --2- 172.21.15.17:0/966900558 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcec00b2730 0x7fcec0061190 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:45.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.823+0000 7fcebffff700 1 --2- 172.21.15.17:0/966900558 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcec00b3190 0x7fcec0060ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:45.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.823+0000 7fcebffff700 1 -- 172.21.15.17:0/966900558 >> 172.21.15.17:0/966900558 conn(0x7fcec0063870 msgr2=0x7fcec0063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:45.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.823+0000 7fcebffff700 1 -- 172.21.15.17:0/966900558 shutdown_connections 2022-08-01T19:40:45.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.823+0000 7fcebffff700 1 -- 172.21.15.17:0/966900558 wait complete. 2022-08-01T19:40:45.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.827+0000 7fcebffff700 1 Processor -- start 2022-08-01T19:40:45.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.827+0000 7fcebffff700 1 -- start start 2022-08-01T19:40:45.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.827+0000 7fcebffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcec00666b0 0x7fcec00adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:45.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.827+0000 7fcebffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcec00b2730 0x7fcec0145d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:45.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.827+0000 7fcebffff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcec00b3190 0x7fcec00b0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:45.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.827+0000 7fcebffff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec00acfa0 con 0x7fcec00666b0 2022-08-01T19:40:45.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.827+0000 7fcebffff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec00ad1d0 con 0x7fcec00b2730 2022-08-01T19:40:45.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.827+0000 7fcebffff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec00ad470 con 0x7fcec00b3190 2022-08-01T19:40:45.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.827+0000 7fcebeffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcec00666b0 0x7fcec00adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:45.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.827+0000 7fcebeffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcec00666b0 0x7fcec00adcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40896/0 (socket says 172.21.15.17:40896) 2022-08-01T19:40:45.858 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.827+0000 7fcebeffd700 1 -- 172.21.15.17:0/2463027968 learned_addr learned my addr 172.21.15.17:0/2463027968 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:45.858 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.827+0000 7fcebe7fc700 1 --2- 172.21.15.17:0/2463027968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcec00b2730 0x7fcec0145d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:45.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.827+0000 7fcebf7fe700 1 --2- 172.21.15.17:0/2463027968 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcec00b3190 0x7fcec00b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:45.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.827+0000 7fcebeffd700 1 -- 172.21.15.17:0/2463027968 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcec00b3190 msgr2=0x7fcec00b0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:45.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.827+0000 7fcebeffd700 1 --2- 172.21.15.17:0/2463027968 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcec00b3190 0x7fcec00b0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:45.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.827+0000 7fcebeffd700 1 -- 172.21.15.17:0/2463027968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcec00b2730 msgr2=0x7fcec0145d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:45.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.827+0000 7fcebeffd700 1 --2- 172.21.15.17:0/2463027968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcec00b2730 0x7fcec0145d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:45.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.827+0000 7fcebeffd700 1 -- 172.21.15.17:0/2463027968 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb000b180 con 0x7fcec00666b0 2022-08-01T19:40:45.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.827+0000 7fcebf7fe700 1 --2- 172.21.15.17:0/2463027968 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcec00b3190 0x7fcec00b0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:40:45.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.831+0000 7fcebeffd700 1 --2- 172.21.15.17:0/2463027968 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcec00666b0 0x7fcec00adcf0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7fcea800c000 tx=0x7fcea8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:45.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.831+0000 7fceb7fff700 1 -- 172.21.15.17:0/2463027968 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcea8010780 con 0x7fcec00666b0 2022-08-01T19:40:45.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.831+0000 7fceb7fff700 1 -- 172.21.15.17:0/2463027968 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcea80108e0 con 0x7fcec00666b0 2022-08-01T19:40:45.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.831+0000 7fcebffff700 1 -- 172.21.15.17:0/2463027968 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec00ac3e0 con 0x7fcec00666b0 2022-08-01T19:40:45.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.831+0000 7fceb7fff700 1 -- 172.21.15.17:0/2463027968 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcea8010c40 con 0x7fcec00666b0 2022-08-01T19:40:45.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.831+0000 7fcebffff700 1 -- 172.21.15.17:0/2463027968 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec00ac880 con 0x7fcec00666b0 2022-08-01T19:40:45.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.831+0000 7fcebffff700 1 -- 172.21.15.17:0/2463027968 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec00ac570 con 0x7fcec00666b0 2022-08-01T19:40:45.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.831+0000 7fceb7fff700 1 -- 172.21.15.17:0/2463027968 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcea801c460 con 0x7fcec00666b0 2022-08-01T19:40:45.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.831+0000 7fceb7fff700 1 --2- 172.21.15.17:0/2463027968 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcea4053b00 0x7fcea4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:45.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.831+0000 7fceb7fff700 1 -- 172.21.15.17:0/2463027968 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6463+0+0 (secure 0 0 0) 0x7fcea806c4e0 con 0x7fcec00666b0 2022-08-01T19:40:45.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.831+0000 7fcebe7fc700 1 --2- 172.21.15.17:0/2463027968 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcea4053b00 0x7fcea4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:45.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.835+0000 7fcebe7fc700 1 --2- 172.21.15.17:0/2463027968 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcea4053b00 0x7fcea4055fc0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7fceb001a000 tx=0x7fceb0014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:45.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.835+0000 7fceb7fff700 1 -- 172.21.15.17:0/2463027968 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcea8043110 con 0x7fcec00666b0 2022-08-01T19:40:45.966 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.959+0000 7fcebffff700 1 -- 172.21.15.17:0/2463027968 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcec005fe90 con 0x7fcec00666b0 2022-08-01T19:40:45.967 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.959+0000 7fceb7fff700 1 -- 172.21.15.17:0/2463027968 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) v1 ==== 100+0+10 (secure 0 0 0) 0x7fcea8046730 con 0x7fcec00666b0 2022-08-01T19:40:45.968 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:40:45.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.963+0000 7fcebffff700 1 -- 172.21.15.17:0/2463027968 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcea4053b00 msgr2=0x7fcea4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:45.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.963+0000 7fcebffff700 1 --2- 172.21.15.17:0/2463027968 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcea4053b00 0x7fcea4055fc0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7fceb001a000 tx=0x7fceb0014000).stop 2022-08-01T19:40:45.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.963+0000 7fcebffff700 1 -- 172.21.15.17:0/2463027968 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcec00666b0 msgr2=0x7fcec00adcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:45.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.963+0000 7fcebffff700 1 --2- 172.21.15.17:0/2463027968 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcec00666b0 0x7fcec00adcf0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7fcea800c000 tx=0x7fcea8006df0).stop 2022-08-01T19:40:45.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.963+0000 7fcebffff700 1 -- 172.21.15.17:0/2463027968 shutdown_connections 2022-08-01T19:40:45.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.963+0000 7fcebffff700 1 --2- 172.21.15.17:0/2463027968 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcea4053b00 0x7fcea4055fc0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:45.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.963+0000 7fcebffff700 1 --2- 172.21.15.17:0/2463027968 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcec00b3190 0x7fcec00b0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:45.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.963+0000 7fcebffff700 1 --2- 172.21.15.17:0/2463027968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcec00b2730 0x7fcec0145d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:45.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.963+0000 7fcebffff700 1 --2- 172.21.15.17:0/2463027968 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcec00666b0 0x7fcec00adcf0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:45.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.963+0000 7fcebffff700 1 -- 172.21.15.17:0/2463027968 >> 172.21.15.17:0/2463027968 conn(0x7fcec0063870 msgr2=0x7fcec0149b90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:45.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.963+0000 7fcebffff700 1 -- 172.21.15.17:0/2463027968 shutdown_connections 2022-08-01T19:40:45.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:45.963+0000 7fcebffff700 1 -- 172.21.15.17:0/2463027968 wait complete. 2022-08-01T19:40:45.987 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T19:40:45.988 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:40:46.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.087+0000 7f9ba9e95700 1 Processor -- start 2022-08-01T19:40:46.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.087+0000 7f9ba9e95700 1 -- start start 2022-08-01T19:40:46.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.091+0000 7f9ba9e95700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f9ba40b1f40 0x7f9ba40b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:46.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.091+0000 7f9ba9e95700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9ba4148530 0x7f9ba40b2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:46.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.091+0000 7f9ba9e95700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ba40a7250 0x7f9ba41448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:46.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.091+0000 7f9ba9e95700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ba40a9250 con 0x7f9ba4148530 2022-08-01T19:40:46.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.091+0000 7f9ba9e95700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ba40b04e0 con 0x7f9ba40b1f40 2022-08-01T19:40:46.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.091+0000 7f9ba9e95700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba40b0780 con 0x7f9ba40a7250 2022-08-01T19:40:46.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.091+0000 7f9ba2ffd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9ba4148530 0x7f9ba40b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:46.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.091+0000 7f9ba2ffd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9ba4148530 0x7f9ba40b2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40908/0 (socket says 172.21.15.17:40908) 2022-08-01T19:40:46.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.091+0000 7f9ba2ffd700 1 -- 172.21.15.17:0/626914763 learned_addr learned my addr 172.21.15.17:0/626914763 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:46.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.091+0000 7f9ba37fe700 1 --2- 172.21.15.17:0/626914763 >> v2:172.21.15.77:3300/0 conn(0x7f9ba40b1f40 0x7f9ba40b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:46.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.091+0000 7f9ba3fff700 1 --2- 172.21.15.17:0/626914763 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ba40a7250 0x7f9ba41448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:46.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.091+0000 7f9ba2ffd700 1 -- 172.21.15.17:0/626914763 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ba40a7250 msgr2=0x7f9ba41448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:46.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.091+0000 7f9ba2ffd700 1 --2- 172.21.15.17:0/626914763 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ba40a7250 0x7f9ba41448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:46.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.091+0000 7f9ba2ffd700 1 -- 172.21.15.17:0/626914763 >> v2:172.21.15.77:3300/0 conn(0x7f9ba40b1f40 msgr2=0x7f9ba40b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:46.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.091+0000 7f9ba2ffd700 1 --2- 172.21.15.17:0/626914763 >> v2:172.21.15.77:3300/0 conn(0x7f9ba40b1f40 0x7f9ba40b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:46.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.091+0000 7f9ba2ffd700 1 -- 172.21.15.17:0/626914763 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba40b0a20 con 0x7f9ba4148530 2022-08-01T19:40:46.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.091+0000 7f9ba37fe700 1 --2- 172.21.15.17:0/626914763 >> v2:172.21.15.77:3300/0 conn(0x7f9ba40b1f40 0x7f9ba40b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:40:46.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.091+0000 7f9ba2ffd700 1 --2- 172.21.15.17:0/626914763 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ba4148530 0x7f9ba40b2320 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7f9ba41478c0 tx=0x7f9b8c006eb0).ready entity=mon.0 client_cookie=c2f35d6c632a9dad server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:46.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.091+0000 7f9ba27fc700 1 -- 172.21.15.17:0/626914763 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b8c00e610 con 0x7f9ba4148530 2022-08-01T19:40:46.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.091+0000 7f9ba27fc700 1 -- 172.21.15.17:0/626914763 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8c000e90 con 0x7f9ba4148530 2022-08-01T19:40:46.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.091+0000 7f9ba27fc700 1 -- 172.21.15.17:0/626914763 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b8c00e9a0 con 0x7f9ba4148530 2022-08-01T19:40:46.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.091+0000 7f9ba9e95700 1 -- 172.21.15.17:0/626914763 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ba4148530 msgr2=0x7f9ba40b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:46.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.091+0000 7f9ba9e95700 1 --2- 172.21.15.17:0/626914763 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ba4148530 0x7f9ba40b2320 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7f9ba41478c0 tx=0x7f9b8c006eb0).stop 2022-08-01T19:40:46.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.091+0000 7f9ba9e95700 1 -- 172.21.15.17:0/626914763 shutdown_connections 2022-08-01T19:40:46.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.091+0000 7f9ba9e95700 1 --2- 172.21.15.17:0/626914763 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ba40a7250 0x7f9ba41448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:46.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.091+0000 7f9ba9e95700 1 --2- 172.21.15.17:0/626914763 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ba4148530 0x7f9ba40b2320 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:46.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.091+0000 7f9ba9e95700 1 --2- 172.21.15.17:0/626914763 >> v2:172.21.15.77:3300/0 conn(0x7f9ba40b1f40 0x7f9ba40b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:46.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.091+0000 7f9ba9e95700 1 -- 172.21.15.17:0/626914763 >> 172.21.15.17:0/626914763 conn(0x7f9ba405f840 msgr2=0x7f9ba405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:46.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.091+0000 7f9ba9e95700 1 -- 172.21.15.17:0/626914763 shutdown_connections 2022-08-01T19:40:46.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.091+0000 7f9ba9e95700 1 -- 172.21.15.17:0/626914763 wait complete. 2022-08-01T19:40:46.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.095+0000 7f9ba9e95700 1 Processor -- start 2022-08-01T19:40:46.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.095+0000 7f9ba9e95700 1 -- start start 2022-08-01T19:40:46.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.095+0000 7f9ba9e95700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ba40a7250 0x7f9ba414a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:46.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.095+0000 7f9ba9e95700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ba40b1f40 0x7f9ba4143ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:46.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.095+0000 7f9ba9e95700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ba4148530 0x7f9ba4143fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:46.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.095+0000 7f9ba9e95700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba414abb0 con 0x7f9ba40a7250 2022-08-01T19:40:46.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.095+0000 7f9ba9e95700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba4144490 con 0x7f9ba40b1f40 2022-08-01T19:40:46.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.095+0000 7f9ba9e95700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba40ae220 con 0x7f9ba4148530 2022-08-01T19:40:46.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.095+0000 7f9ba37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ba40a7250 0x7f9ba414a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:46.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.095+0000 7f9ba2ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ba40b1f40 0x7f9ba4143ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:46.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.095+0000 7f9ba37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ba40a7250 0x7f9ba414a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40912/0 (socket says 172.21.15.17:40912) 2022-08-01T19:40:46.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.095+0000 7f9ba37fe700 1 -- 172.21.15.17:0/1422855469 learned_addr learned my addr 172.21.15.17:0/1422855469 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:46.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.095+0000 7f9ba3fff700 1 --2- 172.21.15.17:0/1422855469 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ba4148530 0x7f9ba4143fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:46.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.095+0000 7f9ba37fe700 1 -- 172.21.15.17:0/1422855469 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ba4148530 msgr2=0x7f9ba4143fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:46.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.095+0000 7f9ba37fe700 1 --2- 172.21.15.17:0/1422855469 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ba4148530 0x7f9ba4143fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:46.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.095+0000 7f9ba37fe700 1 -- 172.21.15.17:0/1422855469 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ba40b1f40 msgr2=0x7f9ba4143ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:46.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.095+0000 7f9ba37fe700 1 --2- 172.21.15.17:0/1422855469 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ba40b1f40 0x7f9ba4143ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:46.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.095+0000 7f9ba37fe700 1 -- 172.21.15.17:0/1422855469 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b94009470 con 0x7f9ba40a7250 2022-08-01T19:40:46.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.095+0000 7f9ba3fff700 1 --2- 172.21.15.17:0/1422855469 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ba4148530 0x7f9ba4143fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:40:46.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.095+0000 7f9ba37fe700 1 --2- 172.21.15.17:0/1422855469 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ba40a7250 0x7f9ba414a570 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0x7f9b9400c000 tx=0x7f9b940091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:46.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.095+0000 7f9ba0ff9700 1 -- 172.21.15.17:0/1422855469 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b9400c920 con 0x7f9ba40a7250 2022-08-01T19:40:46.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.095+0000 7f9ba0ff9700 1 -- 172.21.15.17:0/1422855469 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b9400ca80 con 0x7f9ba40a7250 2022-08-01T19:40:46.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.095+0000 7f9ba9e95700 1 -- 172.21.15.17:0/1422855469 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b8c00b1e0 con 0x7f9ba40a7250 2022-08-01T19:40:46.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.095+0000 7f9ba0ff9700 1 -- 172.21.15.17:0/1422855469 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b9400cde0 con 0x7f9ba40a7250 2022-08-01T19:40:46.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.095+0000 7f9ba9e95700 1 -- 172.21.15.17:0/1422855469 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ba40ad310 con 0x7f9ba40a7250 2022-08-01T19:40:46.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.099+0000 7f9ba9e95700 1 -- 172.21.15.17:0/1422855469 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ba40b0cc0 con 0x7f9ba40a7250 2022-08-01T19:40:46.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.099+0000 7f9ba0ff9700 1 -- 172.21.15.17:0/1422855469 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9b94022070 con 0x7f9ba40a7250 2022-08-01T19:40:46.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.099+0000 7f9ba0ff9700 1 --2- 172.21.15.17:0/1422855469 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9b880539e0 0x7f9b88055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:46.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.099+0000 7f9ba0ff9700 1 -- 172.21.15.17:0/1422855469 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f9b9406c970 con 0x7f9ba40a7250 2022-08-01T19:40:46.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.099+0000 7f9ba2ffd700 1 --2- 172.21.15.17:0/1422855469 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9b880539e0 0x7f9b88055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:46.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.099+0000 7f9ba2ffd700 1 --2- 172.21.15.17:0/1422855469 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9b880539e0 0x7f9b88055ea0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f9b8c01c000 tx=0x7f9b8c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:46.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.103+0000 7f9ba0ff9700 1 -- 172.21.15.17:0/1422855469 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9b94043710 con 0x7f9ba40a7250 2022-08-01T19:40:46.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.271+0000 7f9ba9e95700 1 -- 172.21.15.17:0/1422855469 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ba40574e0 con 0x7f9ba40a7250 2022-08-01T19:40:46.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.271+0000 7f9ba0ff9700 1 -- 172.21.15.17:0/1422855469 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+2493 (secure 0 0 0) 0x7f9b94046d30 con 0x7f9ba40a7250 2022-08-01T19:40:46.276 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:40:46.276 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":35,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5615,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/2284778179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2284778179},{"type":"v1","addr":"172.21.15.77:6835","nonce":2284778179}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":34},{"gid":6261,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/1827111044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":1827111044},{"type":"v1","addr":"172.21.15.17:6833","nonce":1827111044}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":33}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:39:52.325506+0000","modified":"2022-08-01T19:39:53.328610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6267":{"gid":6267,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/3370152344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":3370152344},{"type":"v1","addr":"172.21.15.17:6835","nonce":3370152344}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-08-01T19:40:46.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.271+0000 7f9ba9e95700 1 -- 172.21.15.17:0/1422855469 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9b880539e0 msgr2=0x7f9b88055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:46.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.271+0000 7f9ba9e95700 1 --2- 172.21.15.17:0/1422855469 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9b880539e0 0x7f9b88055ea0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f9b8c01c000 tx=0x7f9b8c015000).stop 2022-08-01T19:40:46.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.271+0000 7f9ba9e95700 1 -- 172.21.15.17:0/1422855469 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ba40a7250 msgr2=0x7f9ba414a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:46.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.271+0000 7f9ba9e95700 1 --2- 172.21.15.17:0/1422855469 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ba40a7250 0x7f9ba414a570 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0x7f9b9400c000 tx=0x7f9b940091d0).stop 2022-08-01T19:40:46.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.271+0000 7f9ba9e95700 1 -- 172.21.15.17:0/1422855469 shutdown_connections 2022-08-01T19:40:46.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.271+0000 7f9ba9e95700 1 --2- 172.21.15.17:0/1422855469 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9b880539e0 0x7f9b88055ea0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:46.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.271+0000 7f9ba9e95700 1 --2- 172.21.15.17:0/1422855469 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ba4148530 0x7f9ba4143fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:46.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.271+0000 7f9ba9e95700 1 --2- 172.21.15.17:0/1422855469 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ba40b1f40 0x7f9ba4143ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:46.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.271+0000 7f9ba9e95700 1 --2- 172.21.15.17:0/1422855469 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ba40a7250 0x7f9ba414a570 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:46.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.271+0000 7f9ba9e95700 1 -- 172.21.15.17:0/1422855469 >> 172.21.15.17:0/1422855469 conn(0x7f9ba405f840 msgr2=0x7f9ba40b33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:46.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.271+0000 7f9ba9e95700 1 -- 172.21.15.17:0/1422855469 shutdown_connections 2022-08-01T19:40:46.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:46.271+0000 7f9ba9e95700 1 -- 172.21.15.17:0/1422855469 wait complete. 2022-08-01T19:40:46.283 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 35 2022-08-01T19:40:46.594 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:39:52.325506+0000', 'modified': '2022-08-01T19:39:53.328610+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6267}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6267': {'gid': 6267, 'name': 'a', 'rank': 0, 'incarnation': 34, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/3370152344', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 3370152344}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 3370152344}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:40:46.595 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:40:46.596 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-08-01T19:40:46.718 INFO:teuthology.orchestra.run.smithi017.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":6267,"want_state":"up:active","state":"up:active","rank_uptime":54.378267958000002,"mdsmap_epoch":35,"osdmap_epoch":77,"osdmap_epoch_barrier":77,"uptime":58.695025096000002} 2022-08-01T19:40:46.719 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 6267, "mdsmap_epoch": 35, "osdmap_epoch": 77, "osdmap_epoch_barrier": 77, "rank_uptime": 54.378267958, "state": "up:active", "uptime": 58.695025096, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:40:46.720 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T19:40:46.720 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-08-01T19:40:46.721 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi085.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-08-01T19:40:46.722 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:40:46.722 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:40:46.738 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:40:46.805 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:40:46.806 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:40:46.825 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:40:46.827 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:40:46.830 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:40:46.831 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:40:46.882 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T19:40:46.883 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:40:46.893 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:40:46.915 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:40:46.916 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:40:46.967 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:2022-08-01T19:40:46.965+0000 7f008aa9d080 -1 init, newargv = 0x55d88d64aae0 newargc=9ceph-fuse[ 2022-08-01T19:40:46.967 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:23112]: starting ceph client 2022-08-01T19:40:46.979 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:40:46.981 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:40:46.981 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:40:46.982 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:40:46.984 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:ceph-fuse[23112]: starting fuse 2022-08-01T19:40:47.007 INFO:teuthology.orchestra.run.smithi085.stdout:52 2022-08-01T19:40:47.008 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T19:40:47.008 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:40:47.009 DEBUG:teuthology.orchestra.run.smithi085:> sudo python3 -c ' 2022-08-01T19:40:47.011 DEBUG:teuthology.orchestra.run.smithi085:> import glob 2022-08-01T19:40:47.012 DEBUG:teuthology.orchestra.run.smithi085:> import re 2022-08-01T19:40:47.013 DEBUG:teuthology.orchestra.run.smithi085:> import os 2022-08-01T19:40:47.013 DEBUG:teuthology.orchestra.run.smithi085:> import subprocess 2022-08-01T19:40:47.014 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:40:47.015 DEBUG:teuthology.orchestra.run.smithi085:> def find_socket(client_name): 2022-08-01T19:40:47.015 DEBUG:teuthology.orchestra.run.smithi085:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-08-01T19:40:47.017 DEBUG:teuthology.orchestra.run.smithi085:> files = glob.glob(asok_path) 2022-08-01T19:40:47.017 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:40:47.018 DEBUG:teuthology.orchestra.run.smithi085:> # Given a non-glob path, it better be there 2022-08-01T19:40:47.019 DEBUG:teuthology.orchestra.run.smithi085:> if "*" not in asok_path: 2022-08-01T19:40:47.019 DEBUG:teuthology.orchestra.run.smithi085:> assert(len(files) == 1) 2022-08-01T19:40:47.020 DEBUG:teuthology.orchestra.run.smithi085:> return files[0] 2022-08-01T19:40:47.021 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:40:47.022 DEBUG:teuthology.orchestra.run.smithi085:> for f in files: 2022-08-01T19:40:47.022 DEBUG:teuthology.orchestra.run.smithi085:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T19:40:47.023 DEBUG:teuthology.orchestra.run.smithi085:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T19:40:47.024 DEBUG:teuthology.orchestra.run.smithi085:> return f 2022-08-01T19:40:47.025 DEBUG:teuthology.orchestra.run.smithi085:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T19:40:47.026 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:40:47.027 DEBUG:teuthology.orchestra.run.smithi085:> print(find_socket("client.0")) 2022-08-01T19:40:47.028 DEBUG:teuthology.orchestra.run.smithi085:> ' 2022-08-01T19:40:47.095 INFO:teuthology.orchestra.run.smithi085.stdout:/var/run/ceph/ceph-client.0.23112.asok 2022-08-01T19:40:47.100 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.23112.asok 2022-08-01T19:40:47.100 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:40:47.101 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.23112.asok status 2022-08-01T19:40:47.194 INFO:teuthology.orchestra.run.smithi085.stdout:{ 2022-08-01T19:40:47.195 INFO:teuthology.orchestra.run.smithi085.stdout: "metadata": { 2022-08-01T19:40:47.195 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T19:40:47.196 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T19:40:47.196 INFO:teuthology.orchestra.run.smithi085.stdout: "entity_id": "0", 2022-08-01T19:40:47.197 INFO:teuthology.orchestra.run.smithi085.stdout: "hostname": "smithi085", 2022-08-01T19:40:47.197 INFO:teuthology.orchestra.run.smithi085.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-08-01T19:40:47.198 INFO:teuthology.orchestra.run.smithi085.stdout: "pid": "23112", 2022-08-01T19:40:47.198 INFO:teuthology.orchestra.run.smithi085.stdout: "root": "/" 2022-08-01T19:40:47.199 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:40:47.199 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_count": 0, 2022-08-01T19:40:47.199 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_pinned_count": 0, 2022-08-01T19:40:47.200 INFO:teuthology.orchestra.run.smithi085.stdout: "id": 6519, 2022-08-01T19:40:47.200 INFO:teuthology.orchestra.run.smithi085.stdout: "inst": { 2022-08-01T19:40:47.201 INFO:teuthology.orchestra.run.smithi085.stdout: "name": { 2022-08-01T19:40:47.201 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "client", 2022-08-01T19:40:47.201 INFO:teuthology.orchestra.run.smithi085.stdout: "num": 6519 2022-08-01T19:40:47.203 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:40:47.203 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2022-08-01T19:40:47.204 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2022-08-01T19:40:47.204 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2022-08-01T19:40:47.204 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 3260467583 2022-08-01T19:40:47.205 INFO:teuthology.orchestra.run.smithi085.stdout: } 2022-08-01T19:40:47.205 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:40:47.205 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2022-08-01T19:40:47.206 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2022-08-01T19:40:47.206 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2022-08-01T19:40:47.206 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 3260467583 2022-08-01T19:40:47.206 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:40:47.207 INFO:teuthology.orchestra.run.smithi085.stdout: "inst_str": "client.6519 172.21.15.85:0/3260467583", 2022-08-01T19:40:47.207 INFO:teuthology.orchestra.run.smithi085.stdout: "addr_str": "172.21.15.85:0/3260467583", 2022-08-01T19:40:47.207 INFO:teuthology.orchestra.run.smithi085.stdout: "inode_count": 1, 2022-08-01T19:40:47.207 INFO:teuthology.orchestra.run.smithi085.stdout: "mds_epoch": 35, 2022-08-01T19:40:47.208 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch": 77, 2022-08-01T19:40:47.208 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch_barrier": 0, 2022-08-01T19:40:47.208 INFO:teuthology.orchestra.run.smithi085.stdout: "blacklisted": false, 2022-08-01T19:40:47.208 INFO:teuthology.orchestra.run.smithi085.stdout: "fs_name": "cephfs" 2022-08-01T19:40:47.209 INFO:teuthology.orchestra.run.smithi085.stdout:} 2022-08-01T19:40:47.216 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:40:47.217 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:40:47.217 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:40:47.247 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:40:47.248 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:40:47.248 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:40:47.250 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:40:47.311 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T19:40:47.312 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:40:47.336 DEBUG:teuthology.orchestra.run.smithi077:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:40:47.364 DEBUG:teuthology.orchestra.run.smithi085:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:40:47.389 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:40:47.414 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T19:40:47.415 INFO:tasks.cephfs.mount:Terminating background process 2022-08-01T19:40:47.415 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:40:47.416 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:40:47.416 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:40:47.426 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:40:47.427 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:40:47.427 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi085.front.sepia.ceph.com... 2022-08-01T19:40:47.427 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T19:40:47.428 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:40:47.485 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:ceph-fuse[23112]: fuse finished with error 0 and tester_r 0 2022-08-01T19:40:47.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:40:47.492 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:40:47.492 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:40:47.500 INFO:teuthology.orchestra.run.smithi085.stdout:ext2/ext3 2022-08-01T19:40:47.509 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T19:40:47.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T19:40:47.561 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:40:47.575 INFO:tasks.cephfs.mount:Terminating background process 2022-08-01T19:40:47.576 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:40:47.577 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:40:47.577 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:40:47.607 INFO:teuthology.orchestra.run.smithi190.stdout:fuseblk 2022-08-01T19:40:47.608 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T19:40:47.608 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi190.front.sepia.ceph.com... 2022-08-01T19:40:47.608 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T19:40:47.609 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:40:47.631 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:ceph-fuse[22674]: fuse finished with error 0 and tester_r 0 2022-08-01T19:40:47.633 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:40:47.634 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:40:47.635 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:40:47.680 INFO:teuthology.orchestra.run.smithi190.stdout:ext2/ext3 2022-08-01T19:40:47.682 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T19:40:47.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T19:40:47.801 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:40:47.850 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T19:40:47.945 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:40:47.950 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:40:47.954 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:40:47.963 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:40:47.966 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.963+0000 7ff489b00700 1 Processor -- start 2022-08-01T19:40:47.966 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.963+0000 7ff489b00700 1 -- start start 2022-08-01T19:40:47.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.963+0000 7ff489b00700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ff4840b2350 0x7ff4840acd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:47.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.963+0000 7ff489b00700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4840ad200 con 0x7ff484139e50 2022-08-01T19:40:47.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.963+0000 7ff489b00700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff484060ed0 con 0x7ff48413a230 2022-08-01T19:40:47.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.963+0000 7ff489b00700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4840b28c0 con 0x7ff4840b2350 2022-08-01T19:40:47.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.963+0000 7ff4837fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff484139e50 0x7ff484143680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33112/0 (socket says 172.21.15.17:33112) 2022-08-01T19:40:47.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.963+0000 7ff4837fe700 1 -- 172.21.15.17:0/1315463384 learned_addr learned my addr 172.21.15.17:0/1315463384 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:40:47.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.963+0000 7ff482ffd700 1 --2- 172.21.15.17:0/1315463384 >> v2:172.21.15.77:3300/0 conn(0x7ff4840b2350 0x7ff4840acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:47.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.963+0000 7ff4827fc700 1 -- 172.21.15.17:0/1315463384 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 760066264 0 0) 0x7ff474001110 con 0x7ff484139e50 2022-08-01T19:40:47.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.963+0000 7ff4827fc700 1 -- 172.21.15.17:0/1315463384 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff464003260 con 0x7ff484139e50 2022-08-01T19:40:47.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.963+0000 7ff4827fc700 1 -- 172.21.15.17:0/1315463384 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4271234285 0 0) 0x7ff484060ed0 con 0x7ff48413a230 2022-08-01T19:40:47.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.963+0000 7ff4827fc700 1 -- 172.21.15.17:0/1315463384 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff474001110 con 0x7ff48413a230 2022-08-01T19:40:47.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.963+0000 7ff482ffd700 1 -- 172.21.15.17:0/1315463384 >> v1:172.21.15.77:6789/0 conn(0x7ff48413a230 legacy=0x7ff4840b6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:47.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.963+0000 7ff482ffd700 1 -- 172.21.15.17:0/1315463384 >> v1:172.21.15.17:6789/0 conn(0x7ff484139e50 legacy=0x7ff484143680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:47.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.963+0000 7ff482ffd700 1 -- 172.21.15.17:0/1315463384 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4840b6e60 con 0x7ff4840b2350 2022-08-01T19:40:47.991 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.963+0000 7ff482ffd700 1 --2- 172.21.15.17:0/1315463384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4840b2350 0x7ff4840acd10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7ff46c002f70 tx=0x7ff46c006df0).ready entity=mon.1 client_cookie=7b8d19426ab81bb3 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:47.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.967+0000 7ff4827fc700 1 -- 172.21.15.17:0/1315463384 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff46c00d540 con 0x7ff4840b2350 2022-08-01T19:40:47.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.967+0000 7ff4827fc700 1 -- 172.21.15.17:0/1315463384 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46c000e90 con 0x7ff4840b2350 2022-08-01T19:40:47.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.967+0000 7ff4827fc700 1 -- 172.21.15.17:0/1315463384 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff46c00d890 con 0x7ff4840b2350 2022-08-01T19:40:47.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.967+0000 7ff489b00700 1 -- 172.21.15.17:0/1315463384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4840b2350 msgr2=0x7ff4840acd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:47.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.967+0000 7ff489b00700 1 --2- 172.21.15.17:0/1315463384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4840b2350 0x7ff4840acd10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7ff46c002f70 tx=0x7ff46c006df0).stop 2022-08-01T19:40:47.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.967+0000 7ff489b00700 1 -- 172.21.15.17:0/1315463384 shutdown_connections 2022-08-01T19:40:47.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.967+0000 7ff489b00700 1 --2- 172.21.15.17:0/1315463384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4840b2350 0x7ff4840acd10 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:47.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.967+0000 7ff489b00700 1 -- 172.21.15.17:0/1315463384 >> 172.21.15.17:0/1315463384 conn(0x7ff4840b3610 msgr2=0x7ff4840b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:47.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.967+0000 7ff489b00700 1 -- 172.21.15.17:0/1315463384 shutdown_connections 2022-08-01T19:40:47.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.967+0000 7ff489b00700 1 -- 172.21.15.17:0/1315463384 wait complete. 2022-08-01T19:40:47.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.967+0000 7ff489b00700 1 Processor -- start 2022-08-01T19:40:47.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.967+0000 7ff489b00700 1 -- start start 2022-08-01T19:40:47.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.971+0000 7ff489b00700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff4840b2350 0x7ff484143640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:47.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.971+0000 7ff489b00700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff484139e50 0x7ff48414ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:48.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.971+0000 7ff489b00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff48413a230 0x7ff48413fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:48.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.971+0000 7ff489b00700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4840af890 con 0x7ff484139e50 2022-08-01T19:40:48.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.971+0000 7ff489b00700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48413f490 con 0x7ff48413a230 2022-08-01T19:40:48.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.971+0000 7ff489b00700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff48413e750 con 0x7ff4840b2350 2022-08-01T19:40:48.002 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.971+0000 7ff482ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff484139e50 0x7ff48414ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:48.002 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.971+0000 7ff482ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff484139e50 0x7ff48414ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40926/0 (socket says 172.21.15.17:40926) 2022-08-01T19:40:48.002 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.971+0000 7ff482ffd700 1 -- 172.21.15.17:0/3798917660 learned_addr learned my addr 172.21.15.17:0/3798917660 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:48.003 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.971+0000 7ff483fff700 1 --2- 172.21.15.17:0/3798917660 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff48413a230 0x7ff48413fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:48.003 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.971+0000 7ff4837fe700 1 --2- 172.21.15.17:0/3798917660 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff4840b2350 0x7ff484143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:48.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.971+0000 7ff482ffd700 1 -- 172.21.15.17:0/3798917660 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff4840b2350 msgr2=0x7ff484143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:48.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.971+0000 7ff482ffd700 1 --2- 172.21.15.17:0/3798917660 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff4840b2350 0x7ff484143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:48.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.971+0000 7ff482ffd700 1 -- 172.21.15.17:0/3798917660 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff48413a230 msgr2=0x7ff48413fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:48.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.971+0000 7ff482ffd700 1 --2- 172.21.15.17:0/3798917660 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff48413a230 0x7ff48413fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:48.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.971+0000 7ff482ffd700 1 -- 172.21.15.17:0/3798917660 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff46c00b180 con 0x7ff484139e50 2022-08-01T19:40:48.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.971+0000 7ff482ffd700 1 --2- 172.21.15.17:0/3798917660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff484139e50 0x7ff48414ec40 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7ff46c002f70 tx=0x7ff46c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:48.007 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.971+0000 7ff480ff9700 1 -- 172.21.15.17:0/3798917660 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff46c00d1c0 con 0x7ff484139e50 2022-08-01T19:40:48.008 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.971+0000 7ff480ff9700 1 -- 172.21.15.17:0/3798917660 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46c013bf0 con 0x7ff484139e50 2022-08-01T19:40:48.008 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.971+0000 7ff489b00700 1 -- 172.21.15.17:0/3798917660 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff48413e9f0 con 0x7ff484139e50 2022-08-01T19:40:48.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.971+0000 7ff480ff9700 1 -- 172.21.15.17:0/3798917660 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff46c01e400 con 0x7ff484139e50 2022-08-01T19:40:48.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.971+0000 7ff489b00700 1 -- 172.21.15.17:0/3798917660 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff48413f8f0 con 0x7ff484139e50 2022-08-01T19:40:48.010 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.971+0000 7ff489b00700 1 -- 172.21.15.17:0/3798917660 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff48413f650 con 0x7ff484139e50 2022-08-01T19:40:48.010 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.971+0000 7ff480ff9700 1 -- 172.21.15.17:0/3798917660 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff46c025030 con 0x7ff484139e50 2022-08-01T19:40:48.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.971+0000 7ff480ff9700 1 --2- 172.21.15.17:0/3798917660 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff4680539e0 0x7ff468055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:48.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.971+0000 7ff480ff9700 1 -- 172.21.15.17:0/3798917660 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6463+0+0 (secure 0 0 0) 0x7ff46c06cd30 con 0x7ff484139e50 2022-08-01T19:40:48.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.975+0000 7ff4837fe700 1 --2- 172.21.15.17:0/3798917660 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff4680539e0 0x7ff468055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:48.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.975+0000 7ff4837fe700 1 --2- 172.21.15.17:0/3798917660 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff4680539e0 0x7ff468055ea0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7ff47400d000 tx=0x7ff474009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:48.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:47.979+0000 7ff480ff9700 1 -- 172.21.15.17:0/3798917660 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff48413f650 con 0x7ff484139e50 2022-08-01T19:40:48.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.107+0000 7ff489b00700 1 -- 172.21.15.17:0/3798917660 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7ff48413f650 con 0x7ff484139e50 2022-08-01T19:40:48.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.111+0000 7ff480ff9700 1 -- 172.21.15.17:0/3798917660 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v77) v1 ==== 81+0+34116 (secure 0 0 0) 0x7ff48413f650 con 0x7ff484139e50 2022-08-01T19:40:48.118 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:40:48.118 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:40:48.118 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 77, 2022-08-01T19:40:48.119 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", 2022-08-01T19:40:48.119 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2022-08-01T19:33:50.105743+0000", 2022-08-01T19:40:48.119 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2022-08-01T19:40:38.195204+0000", 2022-08-01T19:40:48.119 INFO:teuthology.orchestra.run.smithi017.stdout: "last_up_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:40:48.120 INFO:teuthology.orchestra.run.smithi017.stdout: "last_in_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:40:48.120 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T19:40:48.120 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_num": 5799936, 2022-08-01T19:40:48.120 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_set": [ 2022-08-01T19:40:48.121 INFO:teuthology.orchestra.run.smithi017.stdout: "pglog_hardlimit", 2022-08-01T19:40:48.121 INFO:teuthology.orchestra.run.smithi017.stdout: "purged_snapdirs", 2022-08-01T19:40:48.121 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_deletes", 2022-08-01T19:40:48.121 INFO:teuthology.orchestra.run.smithi017.stdout: "sortbitwise" 2022-08-01T19:40:48.122 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:40:48.122 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_version": 6, 2022-08-01T19:40:48.122 INFO:teuthology.orchestra.run.smithi017.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T19:40:48.122 INFO:teuthology.orchestra.run.smithi017.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T19:40:48.123 INFO:teuthology.orchestra.run.smithi017.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T19:40:48.123 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_snapshot": "", 2022-08-01T19:40:48.123 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_max": 14, 2022-08-01T19:40:48.123 INFO:teuthology.orchestra.run.smithi017.stdout: "max_osd": 8, 2022-08-01T19:40:48.124 INFO:teuthology.orchestra.run.smithi017.stdout: "require_min_compat_client": "jewel", 2022-08-01T19:40:48.124 INFO:teuthology.orchestra.run.smithi017.stdout: "min_compat_client": "jewel", 2022-08-01T19:40:48.124 INFO:teuthology.orchestra.run.smithi017.stdout: "require_osd_release": "octopus", 2022-08-01T19:40:48.130 INFO:teuthology.orchestra.run.smithi017.stdout: "pools": [ 2022-08-01T19:40:48.130 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.130 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 1, 2022-08-01T19:40:48.131 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "device_health_metrics", 2022-08-01T19:40:48.131 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:33:53.001841+0000", 2022-08-01T19:40:48.131 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:40:48.131 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:40:48.132 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:40:48.132 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:40:48.132 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:40:48.132 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:40:48.133 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:40:48.133 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:40:48.133 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 1, 2022-08-01T19:40:48.133 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 1, 2022-08-01T19:40:48.133 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 1, 2022-08-01T19:40:48.134 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 1, 2022-08-01T19:40:48.134 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 1, 2022-08-01T19:40:48.134 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:40:48.134 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:40:48.135 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:40:48.135 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:40:48.135 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:40:48.135 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:40:48.136 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:40:48.136 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.136 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "16", 2022-08-01T19:40:48.136 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:40:48.137 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:40:48.137 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:40:48.137 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:40:48.137 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:40:48.138 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:40:48.138 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:40:48.138 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:40:48.139 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:40:48.139 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:40:48.139 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:40:48.139 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:40:48.139 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:40:48.140 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:40:48.140 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:40:48.140 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:40:48.140 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:40:48.141 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:40:48.141 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:40:48.141 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:40:48.141 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:40:48.142 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:40:48.142 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:40:48.142 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:40:48.142 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:40:48.143 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:40:48.143 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.143 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:40:48.143 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:40:48.144 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:40:48.144 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:40:48.144 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:40:48.144 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:40:48.145 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:40:48.145 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:40:48.145 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:40:48.145 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:40:48.145 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:40:48.146 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:40:48.146 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 1 2022-08-01T19:40:48.146 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.146 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:40:48.147 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_devicehealth": {} 2022-08-01T19:40:48.147 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.147 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.147 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.148 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 2, 2022-08-01T19:40:48.148 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "rbd", 2022-08-01T19:40:48.148 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:34:01.553098+0000", 2022-08-01T19:40:48.148 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:40:48.149 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:40:48.149 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:40:48.149 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:40:48.149 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:40:48.149 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:40:48.150 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:40:48.150 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:40:48.150 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:40:48.150 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:40:48.151 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:40:48.151 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:40:48.151 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:40:48.151 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:40:48.152 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:40:48.152 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:40:48.152 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:40:48.152 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:40:48.153 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:40:48.153 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:40:48.154 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.154 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "19", 2022-08-01T19:40:48.154 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:40:48.154 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:40:48.155 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:40:48.155 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:40:48.155 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:40:48.155 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:40:48.156 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:40:48.156 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:40:48.156 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:40:48.156 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:40:48.156 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:40:48.157 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:40:48.157 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:40:48.157 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:40:48.158 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:40:48.158 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:40:48.159 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:40:48.159 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:40:48.160 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:40:48.160 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:40:48.161 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:40:48.161 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:40:48.162 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:40:48.162 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:40:48.162 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:40:48.163 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:40:48.163 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.164 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:40:48.166 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:40:48.167 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:40:48.167 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:40:48.168 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:40:48.168 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:40:48.168 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:40:48.168 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:40:48.169 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:40:48.169 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:40:48.169 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:40:48.169 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:40:48.170 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:40:48.170 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd": {} 2022-08-01T19:40:48.170 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.171 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.171 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.171 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 13, 2022-08-01T19:40:48.171 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "cephfs_metadata", 2022-08-01T19:40:48.172 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:39:50.853967+0000", 2022-08-01T19:40:48.172 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:40:48.172 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:40:48.173 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:40:48.173 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:40:48.173 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:40:48.173 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:40:48.174 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:40:48.174 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:40:48.174 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:40:48.174 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:40:48.174 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:40:48.175 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:40:48.175 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:40:48.175 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:40:48.175 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:40:48.176 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:40:48.176 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:40:48.176 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:40:48.176 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:40:48.177 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:40:48.177 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.177 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "75", 2022-08-01T19:40:48.177 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:40:48.178 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:40:48.178 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:40:48.178 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:40:48.178 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:40:48.179 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:40:48.179 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:40:48.179 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:40:48.179 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:40:48.180 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:40:48.180 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:40:48.180 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:40:48.180 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:40:48.181 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:40:48.181 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:40:48.181 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:40:48.181 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:40:48.182 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:40:48.182 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:40:48.182 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:40:48.182 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:40:48.183 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:40:48.183 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:40:48.183 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:40:48.183 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:40:48.183 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:40:48.184 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.184 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:40:48.184 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:40:48.184 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:40:48.185 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:40:48.185 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:40:48.185 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:40:48.185 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:40:48.186 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:40:48.186 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:40:48.186 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:40:48.186 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:40:48.187 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:40:48.187 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_bias": 4, 2022-08-01T19:40:48.187 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 16, 2022-08-01T19:40:48.187 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_priority": 5 2022-08-01T19:40:48.187 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.188 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:40:48.188 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs": { 2022-08-01T19:40:48.188 INFO:teuthology.orchestra.run.smithi017.stdout: "metadata": "cephfs" 2022-08-01T19:40:48.188 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.189 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.189 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.189 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.189 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 14, 2022-08-01T19:40:48.190 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "cephfs_data", 2022-08-01T19:40:48.190 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:39:51.901692+0000", 2022-08-01T19:40:48.190 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:40:48.191 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:40:48.191 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:40:48.191 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:40:48.191 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:40:48.192 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:40:48.192 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:40:48.192 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:40:48.192 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:40:48.193 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:40:48.193 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:40:48.193 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:40:48.193 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:40:48.194 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:40:48.194 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:40:48.194 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:40:48.194 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:40:48.195 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:40:48.195 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:40:48.195 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:40:48.195 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.195 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "76", 2022-08-01T19:40:48.196 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:40:48.196 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:40:48.196 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:40:48.196 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:40:48.197 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:40:48.197 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:40:48.197 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:40:48.197 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:40:48.198 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:40:48.198 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:40:48.198 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:40:48.198 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:40:48.200 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:40:48.201 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:40:48.201 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:40:48.202 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:40:48.202 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:40:48.203 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:40:48.203 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:40:48.203 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:40:48.204 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:40:48.204 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:40:48.205 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:40:48.205 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:40:48.206 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:40:48.206 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:40:48.207 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.207 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:40:48.208 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:40:48.208 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:40:48.209 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:40:48.209 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:40:48.210 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:40:48.210 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:40:48.211 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:40:48.211 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:40:48.212 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:40:48.212 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:40:48.212 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:40:48.213 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:40:48.213 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs": { 2022-08-01T19:40:48.213 INFO:teuthology.orchestra.run.smithi017.stdout: "data": "cephfs" 2022-08-01T19:40:48.214 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.214 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.214 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.215 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:40:48.215 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2022-08-01T19:40:48.216 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.216 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:40:48.216 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a962f14f-cf16-4fa8-84ff-c85325db9132", 2022-08-01T19:40:48.217 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:40:48.217 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:40:48.218 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:40:48.218 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:40:48.218 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:40:48.219 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:40:48.219 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:40:48.219 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 74, 2022-08-01T19:40:48.220 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:40:48.220 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:40:48.221 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:40:48.221 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:48.221 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.221 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:48.226 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6800", 2022-08-01T19:40:48.226 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:40:48.226 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.227 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.227 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:48.227 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6801", 2022-08-01T19:40:48.227 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:40:48.228 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.228 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.228 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.229 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:40:48.229 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:48.229 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.229 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:48.230 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6802", 2022-08-01T19:40:48.230 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:40:48.230 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.230 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.231 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:48.231 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6803", 2022-08-01T19:40:48.231 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:40:48.231 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.232 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.232 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.232 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:40:48.232 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:48.232 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.233 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:48.233 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6806", 2022-08-01T19:40:48.233 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:40:48.234 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.234 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.235 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:48.235 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6807", 2022-08-01T19:40:48.235 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:40:48.236 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.236 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.236 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.236 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:40:48.237 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:48.237 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.237 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:48.237 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6804", 2022-08-01T19:40:48.238 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:40:48.238 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.238 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.238 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:48.239 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6805", 2022-08-01T19:40:48.239 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:40:48.239 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.239 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.240 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.240 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6801/23008", 2022-08-01T19:40:48.240 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6803/23008", 2022-08-01T19:40:48.241 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6807/23008", 2022-08-01T19:40:48.241 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6805/23008", 2022-08-01T19:40:48.241 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:40:48.241 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:40:48.242 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:40:48.242 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.242 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.242 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.243 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:40:48.243 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "805f7051-a527-4e98-8844-b6e406a63248", 2022-08-01T19:40:48.243 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:40:48.244 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:40:48.245 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:40:48.245 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:40:48.245 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:40:48.246 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:40:48.246 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:40:48.246 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 64, 2022-08-01T19:40:48.246 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:40:48.247 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:40:48.247 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:40:48.247 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:48.247 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.248 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:48.248 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6808", 2022-08-01T19:40:48.248 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:40:48.248 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.248 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.249 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:48.249 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6809", 2022-08-01T19:40:48.249 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:40:48.249 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.250 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.250 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.250 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:40:48.250 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:48.251 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.251 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:48.251 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6810", 2022-08-01T19:40:48.251 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:40:48.252 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.252 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.252 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:48.253 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6811", 2022-08-01T19:40:48.253 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:40:48.253 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.253 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.254 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.254 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:40:48.254 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:48.254 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.255 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:48.256 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6814", 2022-08-01T19:40:48.256 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:40:48.257 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.257 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.257 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:48.258 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6815", 2022-08-01T19:40:48.258 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:40:48.259 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.260 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.260 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.261 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:40:48.261 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:48.262 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.262 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:48.263 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6812", 2022-08-01T19:40:48.263 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:40:48.264 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.264 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.265 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:48.265 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6813", 2022-08-01T19:40:48.266 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:40:48.266 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.266 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.267 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.267 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6809/23007", 2022-08-01T19:40:48.268 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6811/23007", 2022-08-01T19:40:48.268 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6815/23007", 2022-08-01T19:40:48.269 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6813/23007", 2022-08-01T19:40:48.269 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:40:48.270 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:40:48.270 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:40:48.271 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.271 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.272 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.272 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:40:48.272 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "89f7f3df-7c10-48e1-8b69-9e4243923a61", 2022-08-01T19:40:48.273 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:40:48.273 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:40:48.274 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:40:48.274 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:40:48.275 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:40:48.275 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:40:48.276 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:40:48.277 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 74, 2022-08-01T19:40:48.277 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:40:48.278 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:40:48.278 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:40:48.279 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:48.279 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.279 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:48.280 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6817", 2022-08-01T19:40:48.280 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:40:48.281 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.282 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.282 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:48.282 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6819", 2022-08-01T19:40:48.283 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:40:48.283 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.283 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.284 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.284 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:40:48.284 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:48.284 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.285 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:48.285 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6822", 2022-08-01T19:40:48.285 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:40:48.285 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.286 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.286 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:48.286 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6824", 2022-08-01T19:40:48.286 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:40:48.287 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.287 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.287 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.287 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:40:48.287 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:48.288 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.288 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:48.288 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6830", 2022-08-01T19:40:48.289 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:40:48.289 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.289 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.289 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:48.289 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6831", 2022-08-01T19:40:48.290 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:40:48.290 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.290 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.290 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.291 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:40:48.291 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:48.291 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.291 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:48.292 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6826", 2022-08-01T19:40:48.292 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:40:48.292 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.292 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.293 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:48.293 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6829", 2022-08-01T19:40:48.293 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:40:48.294 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.294 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.294 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.294 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6819/23010", 2022-08-01T19:40:48.295 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6824/23010", 2022-08-01T19:40:48.297 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6831/23010", 2022-08-01T19:40:48.297 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6829/23010", 2022-08-01T19:40:48.297 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:40:48.297 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:40:48.298 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:40:48.298 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.298 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.298 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.299 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:40:48.299 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "96123468-aabc-4a64-909e-a192c369090d", 2022-08-01T19:40:48.299 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:40:48.299 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:40:48.300 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:40:48.300 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:40:48.300 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:40:48.300 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:40:48.301 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:40:48.301 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 74, 2022-08-01T19:40:48.301 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:40:48.301 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:40:48.302 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:40:48.302 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:48.302 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.302 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:48.303 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6816", 2022-08-01T19:40:48.303 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:40:48.303 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.304 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.304 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:48.304 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6818", 2022-08-01T19:40:48.304 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:40:48.305 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.305 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.305 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.305 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:40:48.306 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:48.306 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.306 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:48.306 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6820", 2022-08-01T19:40:48.307 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:40:48.307 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.307 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.307 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:48.308 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6821", 2022-08-01T19:40:48.308 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:40:48.308 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.309 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.309 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.309 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:40:48.309 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:48.310 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.310 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:48.310 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6827", 2022-08-01T19:40:48.310 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:40:48.311 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.311 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.311 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:48.311 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6828", 2022-08-01T19:40:48.312 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:40:48.312 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.312 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.312 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.313 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:40:48.313 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:48.313 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.313 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:48.314 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6823", 2022-08-01T19:40:48.314 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:40:48.314 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.314 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.315 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:48.315 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6825", 2022-08-01T19:40:48.315 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:40:48.315 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.316 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.316 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.316 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6818/23009", 2022-08-01T19:40:48.317 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6821/23009", 2022-08-01T19:40:48.317 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6828/23009", 2022-08-01T19:40:48.317 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6825/23009", 2022-08-01T19:40:48.317 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:40:48.318 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:40:48.318 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:40:48.318 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.319 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.319 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.319 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:40:48.319 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a6e26665-e04b-46ec-ab66-e912acbf044c", 2022-08-01T19:40:48.320 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:40:48.320 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:40:48.321 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:40:48.321 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:40:48.322 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:40:48.322 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:40:48.323 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:40:48.323 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 73, 2022-08-01T19:40:48.323 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:40:48.323 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:40:48.324 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:40:48.324 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:48.324 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.324 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:48.325 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6810", 2022-08-01T19:40:48.325 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:40:48.325 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.326 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.326 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:48.326 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6811", 2022-08-01T19:40:48.326 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:40:48.327 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.327 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.327 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.327 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:40:48.328 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:48.328 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.328 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:48.329 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6812", 2022-08-01T19:40:48.329 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:40:48.329 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.330 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.330 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:48.330 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6813", 2022-08-01T19:40:48.330 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:40:48.331 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.331 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.331 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.331 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:40:48.332 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:48.332 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.332 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:48.332 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6816", 2022-08-01T19:40:48.333 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:40:48.333 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.333 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.333 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:48.334 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6817", 2022-08-01T19:40:48.334 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:40:48.334 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.334 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.335 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.335 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:40:48.335 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:48.335 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.336 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:48.336 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6814", 2022-08-01T19:40:48.336 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:40:48.336 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.337 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.337 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:48.337 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6815", 2022-08-01T19:40:48.337 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:40:48.338 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.338 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.338 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.338 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6811/23584", 2022-08-01T19:40:48.339 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6813/23584", 2022-08-01T19:40:48.339 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6817/23584", 2022-08-01T19:40:48.339 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6815/23584", 2022-08-01T19:40:48.339 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:40:48.340 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:40:48.340 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:40:48.340 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.340 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.341 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.341 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:40:48.341 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "502c0034-18ee-44ac-87ff-19b9dddb66a0", 2022-08-01T19:40:48.341 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:40:48.342 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:40:48.342 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:40:48.342 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:40:48.342 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:40:48.343 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:40:48.343 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:40:48.343 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 74, 2022-08-01T19:40:48.344 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:40:48.344 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:40:48.344 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:40:48.344 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:48.345 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.345 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:48.345 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6802", 2022-08-01T19:40:48.346 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:40:48.346 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.346 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.346 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:48.347 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6803", 2022-08-01T19:40:48.347 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:40:48.347 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.347 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.348 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.348 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:40:48.348 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:48.348 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.348 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:48.349 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6804", 2022-08-01T19:40:48.349 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:40:48.349 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.349 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.350 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:48.350 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6805", 2022-08-01T19:40:48.350 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:40:48.351 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.351 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.351 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.351 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:40:48.352 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:48.352 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.352 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:48.352 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6808", 2022-08-01T19:40:48.353 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:40:48.353 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.353 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.353 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:48.354 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6809", 2022-08-01T19:40:48.354 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:40:48.354 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.354 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.355 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.355 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:40:48.355 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:48.355 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.356 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:48.356 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6806", 2022-08-01T19:40:48.356 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:40:48.356 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.357 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.357 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:48.357 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6807", 2022-08-01T19:40:48.357 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:40:48.358 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.358 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.358 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.358 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6803/23585", 2022-08-01T19:40:48.359 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6805/23585", 2022-08-01T19:40:48.359 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6809/23585", 2022-08-01T19:40:48.359 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6807/23585", 2022-08-01T19:40:48.359 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:40:48.360 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:40:48.360 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:40:48.360 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.361 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.361 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.361 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:40:48.361 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "ac4e755a-7fe7-46cb-8407-852d5af9ffc5", 2022-08-01T19:40:48.362 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:40:48.362 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:40:48.362 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:40:48.362 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:40:48.363 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:40:48.363 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:40:48.363 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:40:48.363 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 74, 2022-08-01T19:40:48.364 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:40:48.364 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:40:48.364 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:40:48.365 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:48.365 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.365 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:48.365 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6820", 2022-08-01T19:40:48.366 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:40:48.366 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.366 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.366 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:48.367 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6821", 2022-08-01T19:40:48.367 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:40:48.367 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.367 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.368 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.368 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:40:48.368 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:48.368 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.369 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:48.369 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6823", 2022-08-01T19:40:48.369 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:40:48.369 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.370 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.370 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:48.370 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6825", 2022-08-01T19:40:48.370 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:40:48.371 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.371 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.371 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.371 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:40:48.372 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:48.372 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.372 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:48.372 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6831", 2022-08-01T19:40:48.372 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:40:48.373 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.373 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.373 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:48.373 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6833", 2022-08-01T19:40:48.374 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:40:48.374 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.375 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.375 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.375 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:40:48.375 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:48.376 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.376 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:48.376 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6827", 2022-08-01T19:40:48.376 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:40:48.377 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.377 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.377 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:48.377 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6829", 2022-08-01T19:40:48.378 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:40:48.378 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.378 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.378 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.379 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6821/23587", 2022-08-01T19:40:48.379 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6825/23587", 2022-08-01T19:40:48.379 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6833/23587", 2022-08-01T19:40:48.379 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6829/23587", 2022-08-01T19:40:48.380 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:40:48.380 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:40:48.380 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:40:48.380 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.381 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.381 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:40:48.381 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "874761e2-6410-40fc-b60c-e58abe5ce489", 2022-08-01T19:40:48.382 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:40:48.382 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:40:48.382 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:40:48.382 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:40:48.383 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:40:48.383 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:40:48.383 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:40:48.383 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 73, 2022-08-01T19:40:48.383 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:40:48.384 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:40:48.384 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:40:48.384 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:48.384 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.385 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:48.385 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6818", 2022-08-01T19:40:48.385 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:40:48.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.386 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.386 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:48.386 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6819", 2022-08-01T19:40:48.386 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:40:48.387 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.387 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.387 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.387 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:40:48.388 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:48.388 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.388 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:48.388 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6822", 2022-08-01T19:40:48.388 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:40:48.389 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.389 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.389 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:48.389 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6824", 2022-08-01T19:40:48.390 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:40:48.390 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.390 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.390 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.391 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:40:48.391 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:48.391 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.391 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:48.392 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6830", 2022-08-01T19:40:48.392 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:40:48.392 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.392 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.393 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:48.393 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6832", 2022-08-01T19:40:48.393 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:40:48.393 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.394 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.394 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.394 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:40:48.394 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:48.395 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.395 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:48.395 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6826", 2022-08-01T19:40:48.395 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:40:48.396 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.396 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.396 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:48.396 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6828", 2022-08-01T19:40:48.397 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:40:48.397 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.397 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.397 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.398 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6819/23586", 2022-08-01T19:40:48.398 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6824/23586", 2022-08-01T19:40:48.398 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6832/23586", 2022-08-01T19:40:48.398 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6828/23586", 2022-08-01T19:40:48.399 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:40:48.399 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:40:48.399 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:40:48.399 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:48.399 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.400 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:40:48.400 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_xinfo": [ 2022-08-01T19:40:48.400 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.400 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:40:48.401 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:40:48.401 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:40:48.401 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:40:48.401 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:40:48.402 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:40:48.402 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.588817+0000", 2022-08-01T19:40:48.402 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:40:48.402 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.403 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.403 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:40:48.403 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:40:48.403 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:40:48.404 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:40:48.404 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:40:48.404 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:40:48.405 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.782753+0000", 2022-08-01T19:40:48.405 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:40:48.405 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.405 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.406 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:40:48.406 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:40:48.406 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:40:48.406 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:40:48.407 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:40:48.407 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:40:48.407 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.837924+0000", 2022-08-01T19:40:48.407 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:40:48.408 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.408 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.408 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:40:48.408 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:40:48.408 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:40:48.409 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:40:48.409 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:40:48.409 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:40:48.409 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.839948+0000", 2022-08-01T19:40:48.410 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:40:48.410 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.410 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.410 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:40:48.411 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:40:48.411 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:40:48.411 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:40:48.411 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:40:48.412 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:40:48.412 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.112801+0000", 2022-08-01T19:40:48.412 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:40:48.412 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.413 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.413 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:40:48.413 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:40:48.413 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:40:48.414 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:40:48.414 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:40:48.414 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:40:48.414 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.106704+0000", 2022-08-01T19:40:48.415 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:40:48.415 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.415 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.416 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:40:48.416 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:40:48.416 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:40:48.416 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:40:48.417 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:40:48.417 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:40:48.417 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.893170+0000", 2022-08-01T19:40:48.417 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:40:48.418 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.418 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:48.418 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:40:48.419 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:40:48.419 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:40:48.419 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:40:48.419 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:40:48.420 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:40:48.420 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.886947+0000", 2022-08-01T19:40:48.420 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:40:48.420 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.421 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:40:48.421 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap": [], 2022-08-01T19:40:48.421 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap_items": [], 2022-08-01T19:40:48.421 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_temp": [], 2022-08-01T19:40:48.422 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_temp": [], 2022-08-01T19:40:48.422 INFO:teuthology.orchestra.run.smithi017.stdout: "blacklist": { 2022-08-01T19:40:48.422 INFO:teuthology.orchestra.run.smithi017.stdout: "172.21.15.85:0/3805297491": "2022-08-01T20:40:38.016590+0000" 2022-08-01T19:40:48.422 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.423 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profiles": { 2022-08-01T19:40:48.423 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_data": { 2022-08-01T19:40:48.423 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-device-class": "", 2022-08-01T19:40:48.423 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:40:48.424 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-root": "default", 2022-08-01T19:40:48.424 INFO:teuthology.orchestra.run.smithi017.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T19:40:48.424 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:40:48.424 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "2", 2022-08-01T19:40:48.424 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:40:48.425 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:40:48.425 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van", 2022-08-01T19:40:48.425 INFO:teuthology.orchestra.run.smithi017.stdout: "w": "8" 2022-08-01T19:40:48.425 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.426 INFO:teuthology.orchestra.run.smithi017.stdout: "default": { 2022-08-01T19:40:48.426 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:40:48.426 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:40:48.426 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "1", 2022-08-01T19:40:48.427 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:40:48.427 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:40:48.427 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van" 2022-08-01T19:40:48.427 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:48.428 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:48.428 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps_queue": [], 2022-08-01T19:40:48.428 INFO:teuthology.orchestra.run.smithi017.stdout: "new_removed_snaps": [], 2022-08-01T19:40:48.428 INFO:teuthology.orchestra.run.smithi017.stdout: "new_purged_snaps": [], 2022-08-01T19:40:48.429 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_node_flags": {}, 2022-08-01T19:40:48.429 INFO:teuthology.orchestra.run.smithi017.stdout: "device_class_flags": {} 2022-08-01T19:40:48.429 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:40:48.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.111+0000 7ff489b00700 1 -- 172.21.15.17:0/3798917660 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff4680539e0 msgr2=0x7ff468055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:48.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.111+0000 7ff489b00700 1 --2- 172.21.15.17:0/3798917660 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff4680539e0 0x7ff468055ea0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7ff47400d000 tx=0x7ff474009040).stop 2022-08-01T19:40:48.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.111+0000 7ff489b00700 1 -- 172.21.15.17:0/3798917660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff484139e50 msgr2=0x7ff48414ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:48.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.111+0000 7ff489b00700 1 --2- 172.21.15.17:0/3798917660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff484139e50 0x7ff48414ec40 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7ff46c002f70 tx=0x7ff46c014000).stop 2022-08-01T19:40:48.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.111+0000 7ff489b00700 1 -- 172.21.15.17:0/3798917660 shutdown_connections 2022-08-01T19:40:48.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.111+0000 7ff489b00700 1 --2- 172.21.15.17:0/3798917660 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff4680539e0 0x7ff468055ea0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:48.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.111+0000 7ff489b00700 1 --2- 172.21.15.17:0/3798917660 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff48413a230 0x7ff48413fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:48.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.111+0000 7ff489b00700 1 --2- 172.21.15.17:0/3798917660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff484139e50 0x7ff48414ec40 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:48.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.111+0000 7ff489b00700 1 --2- 172.21.15.17:0/3798917660 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff4840b2350 0x7ff484143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:48.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.111+0000 7ff489b00700 1 -- 172.21.15.17:0/3798917660 >> 172.21.15.17:0/3798917660 conn(0x7ff4840b3610 msgr2=0x7ff484144fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:48.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.111+0000 7ff489b00700 1 -- 172.21.15.17:0/3798917660 shutdown_connections 2022-08-01T19:40:48.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.111+0000 7ff489b00700 1 -- 172.21.15.17:0/3798917660 wait complete. 2022-08-01T19:40:48.435 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:40:48.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.543+0000 7f228dcc5700 1 Processor -- start 2022-08-01T19:40:48.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.547+0000 7f228dcc5700 1 -- start start 2022-08-01T19:40:48.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.547+0000 7f228dcc5700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f22880af450 0x7f22880af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:48.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.547+0000 7f228dcc5700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f22880ad4b0 0x7f22881432e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:48.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.547+0000 7f228dcc5700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f22881437d0 0x7f22880a8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:48.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.547+0000 7f228dcc5700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22880aff30 con 0x7f22880af450 2022-08-01T19:40:48.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.547+0000 7f228dcc5700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f228805c7a0 con 0x7f22881437d0 2022-08-01T19:40:48.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.547+0000 7f228dcc5700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2288140990 con 0x7f22880ad4b0 2022-08-01T19:40:48.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.547+0000 7f228ccc3700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f22880af450 0x7f22880af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:48.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.547+0000 7f228ccc3700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f22880af450 0x7f22880af830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40932/0 (socket says 172.21.15.17:40932) 2022-08-01T19:40:48.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.547+0000 7f228ccc3700 1 -- 172.21.15.17:0/3026286342 learned_addr learned my addr 172.21.15.17:0/3026286342 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:48.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.547+0000 7f228d4c4700 1 --2- 172.21.15.17:0/3026286342 >> v2:172.21.15.77:3300/0 conn(0x7f22881437d0 0x7f22880a8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:48.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.547+0000 7f2287fff700 1 --2- 172.21.15.17:0/3026286342 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f22880ad4b0 0x7f22881432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:48.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.547+0000 7f228ccc3700 1 -- 172.21.15.17:0/3026286342 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f22880ad4b0 msgr2=0x7f22881432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:48.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.547+0000 7f228ccc3700 1 --2- 172.21.15.17:0/3026286342 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f22880ad4b0 0x7f22881432e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:48.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.547+0000 7f228ccc3700 1 -- 172.21.15.17:0/3026286342 >> v2:172.21.15.77:3300/0 conn(0x7f22881437d0 msgr2=0x7f22880a8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:48.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.547+0000 7f228ccc3700 1 --2- 172.21.15.17:0/3026286342 >> v2:172.21.15.77:3300/0 conn(0x7f22881437d0 0x7f22880a8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:48.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.547+0000 7f228ccc3700 1 -- 172.21.15.17:0/3026286342 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22880ad9e0 con 0x7f22880af450 2022-08-01T19:40:48.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.547+0000 7f228ccc3700 1 --2- 172.21.15.17:0/3026286342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22880af450 0x7f22880af830 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7f228813fdc0 tx=0x7f227c006eb0).ready entity=mon.0 client_cookie=ff44e4b20af54b14 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:48.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.547+0000 7f22877fe700 1 -- 172.21.15.17:0/3026286342 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f227c00e610 con 0x7f22880af450 2022-08-01T19:40:48.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.547+0000 7f22877fe700 1 -- 172.21.15.17:0/3026286342 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f227c000e90 con 0x7f22880af450 2022-08-01T19:40:48.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.547+0000 7f22877fe700 1 -- 172.21.15.17:0/3026286342 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f227c00e9a0 con 0x7f22880af450 2022-08-01T19:40:48.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.551+0000 7f228dcc5700 1 -- 172.21.15.17:0/3026286342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22880af450 msgr2=0x7f22880af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:48.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.551+0000 7f228dcc5700 1 --2- 172.21.15.17:0/3026286342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22880af450 0x7f22880af830 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7f228813fdc0 tx=0x7f227c006eb0).stop 2022-08-01T19:40:48.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.551+0000 7f228dcc5700 1 -- 172.21.15.17:0/3026286342 shutdown_connections 2022-08-01T19:40:48.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.551+0000 7f228dcc5700 1 --2- 172.21.15.17:0/3026286342 >> v2:172.21.15.77:3300/0 conn(0x7f22881437d0 0x7f22880a8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:48.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.551+0000 7f228dcc5700 1 --2- 172.21.15.17:0/3026286342 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f22880ad4b0 0x7f22881432e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:48.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.551+0000 7f228dcc5700 1 --2- 172.21.15.17:0/3026286342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22880af450 0x7f22880af830 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:48.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.551+0000 7f228dcc5700 1 -- 172.21.15.17:0/3026286342 >> 172.21.15.17:0/3026286342 conn(0x7f22880612d0 msgr2=0x7f2288062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:48.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.551+0000 7f228dcc5700 1 -- 172.21.15.17:0/3026286342 shutdown_connections 2022-08-01T19:40:48.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.551+0000 7f228dcc5700 1 -- 172.21.15.17:0/3026286342 wait complete. 2022-08-01T19:40:48.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.551+0000 7f228dcc5700 1 Processor -- start 2022-08-01T19:40:48.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.551+0000 7f228dcc5700 1 -- start start 2022-08-01T19:40:48.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.551+0000 7f228dcc5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22880ad4b0 0x7f2288152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:48.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.551+0000 7f228dcc5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22880af450 0x7f228805fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:48.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.551+0000 7f228dcc5700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f22881437d0 0x7f22880aa600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:48.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.551+0000 7f228dcc5700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22880a8a30 con 0x7f22880ad4b0 2022-08-01T19:40:48.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.551+0000 7f228dcc5700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22880aaaf0 con 0x7f22880af450 2022-08-01T19:40:48.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.551+0000 7f228dcc5700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22880abca0 con 0x7f22881437d0 2022-08-01T19:40:48.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.551+0000 7f228ccc3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22880ad4b0 0x7f2288152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:48.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.551+0000 7f228ccc3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22880ad4b0 0x7f2288152dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40938/0 (socket says 172.21.15.17:40938) 2022-08-01T19:40:48.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.551+0000 7f228ccc3700 1 -- 172.21.15.17:0/3884501284 learned_addr learned my addr 172.21.15.17:0/3884501284 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:48.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.551+0000 7f2287fff700 1 --2- 172.21.15.17:0/3884501284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22880af450 0x7f228805fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:48.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.551+0000 7f228d4c4700 1 --2- 172.21.15.17:0/3884501284 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f22881437d0 0x7f22880aa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:48.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.551+0000 7f228ccc3700 1 -- 172.21.15.17:0/3884501284 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f22881437d0 msgr2=0x7f22880aa600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:48.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.551+0000 7f228ccc3700 1 --2- 172.21.15.17:0/3884501284 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f22881437d0 0x7f22880aa600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:48.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.551+0000 7f228ccc3700 1 -- 172.21.15.17:0/3884501284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22880af450 msgr2=0x7f228805fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:48.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.551+0000 7f228ccc3700 1 --2- 172.21.15.17:0/3884501284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22880af450 0x7f228805fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:48.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.551+0000 7f228ccc3700 1 -- 172.21.15.17:0/3884501284 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f227c00b1e0 con 0x7f22880ad4b0 2022-08-01T19:40:48.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.551+0000 7f2287fff700 1 --2- 172.21.15.17:0/3884501284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22880af450 0x7f228805fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:40:48.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.551+0000 7f228ccc3700 1 --2- 172.21.15.17:0/3884501284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22880ad4b0 0x7f2288152dd0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7f228813fdc0 tx=0x7f227c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:48.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.551+0000 7f2285ffb700 1 -- 172.21.15.17:0/3884501284 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f227c0117e0 con 0x7f22880ad4b0 2022-08-01T19:40:48.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.551+0000 7f2285ffb700 1 -- 172.21.15.17:0/3884501284 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f227c011940 con 0x7f22880ad4b0 2022-08-01T19:40:48.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.555+0000 7f228dcc5700 1 -- 172.21.15.17:0/3884501284 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22880abf40 con 0x7f22880ad4b0 2022-08-01T19:40:48.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.555+0000 7f2285ffb700 1 -- 172.21.15.17:0/3884501284 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f227c011ca0 con 0x7f22880ad4b0 2022-08-01T19:40:48.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.555+0000 7f228dcc5700 1 -- 172.21.15.17:0/3884501284 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f228809ea40 con 0x7f22880ad4b0 2022-08-01T19:40:48.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.555+0000 7f228dcc5700 1 -- 172.21.15.17:0/3884501284 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22880b6100 con 0x7f22880ad4b0 2022-08-01T19:40:48.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.555+0000 7f2285ffb700 1 -- 172.21.15.17:0/3884501284 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f227c025030 con 0x7f22880ad4b0 2022-08-01T19:40:48.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.555+0000 7f2285ffb700 1 --2- 172.21.15.17:0/3884501284 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2268053980 0x7f2268055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:48.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.555+0000 7f2285ffb700 1 -- 172.21.15.17:0/3884501284 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f227c06d2b0 con 0x7f22880ad4b0 2022-08-01T19:40:48.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.555+0000 7f2287fff700 1 --2- 172.21.15.17:0/3884501284 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2268053980 0x7f2268055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:48.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.555+0000 7f2287fff700 1 --2- 172.21.15.17:0/3884501284 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2268053980 0x7f2268055e40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f227400b000 tx=0x7f2274008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:48.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.563+0000 7f2285ffb700 1 -- 172.21.15.17:0/3884501284 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f227c044050 con 0x7f22880ad4b0 2022-08-01T19:40:48.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.723+0000 7f228dcc5700 1 -- 172.21.15.17:0/3884501284 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22880ac2c0 con 0x7f22880ad4b0 2022-08-01T19:40:48.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.723+0000 7f2285ffb700 1 -- 172.21.15.17:0/3884501284 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+2493 (secure 0 0 0) 0x7f227c023090 con 0x7f22880ad4b0 2022-08-01T19:40:48.728 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:40:48.729 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":35,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5615,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/2284778179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2284778179},{"type":"v1","addr":"172.21.15.77:6835","nonce":2284778179}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":34},{"gid":6261,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/1827111044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":1827111044},{"type":"v1","addr":"172.21.15.17:6833","nonce":1827111044}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":33}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:39:52.325506+0000","modified":"2022-08-01T19:39:53.328610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6267":{"gid":6267,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/3370152344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":3370152344},{"type":"v1","addr":"172.21.15.17:6835","nonce":3370152344}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-08-01T19:40:48.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.727+0000 7f228dcc5700 1 -- 172.21.15.17:0/3884501284 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2268053980 msgr2=0x7f2268055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:48.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.727+0000 7f228dcc5700 1 --2- 172.21.15.17:0/3884501284 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2268053980 0x7f2268055e40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f227400b000 tx=0x7f2274008040).stop 2022-08-01T19:40:48.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.727+0000 7f228dcc5700 1 -- 172.21.15.17:0/3884501284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22880ad4b0 msgr2=0x7f2288152dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:48.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.727+0000 7f228dcc5700 1 --2- 172.21.15.17:0/3884501284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22880ad4b0 0x7f2288152dd0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7f228813fdc0 tx=0x7f227c015000).stop 2022-08-01T19:40:48.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.727+0000 7f228dcc5700 1 -- 172.21.15.17:0/3884501284 shutdown_connections 2022-08-01T19:40:48.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.727+0000 7f228dcc5700 1 --2- 172.21.15.17:0/3884501284 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2268053980 0x7f2268055e40 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:48.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.727+0000 7f228dcc5700 1 --2- 172.21.15.17:0/3884501284 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f22881437d0 0x7f22880aa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:48.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.727+0000 7f228dcc5700 1 --2- 172.21.15.17:0/3884501284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22880af450 0x7f228805fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:48.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.727+0000 7f228dcc5700 1 --2- 172.21.15.17:0/3884501284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22880ad4b0 0x7f2288152dd0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:48.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.727+0000 7f228dcc5700 1 -- 172.21.15.17:0/3884501284 >> 172.21.15.17:0/3884501284 conn(0x7f22880612d0 msgr2=0x7f228813dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:48.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.727+0000 7f228dcc5700 1 -- 172.21.15.17:0/3884501284 shutdown_connections 2022-08-01T19:40:48.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.727+0000 7f228dcc5700 1 -- 172.21.15.17:0/3884501284 wait complete. 2022-08-01T19:40:48.734 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 35 2022-08-01T19:40:48.741 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-08-01T19:40:48.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.843+0000 7fb16cc41700 1 Processor -- start 2022-08-01T19:40:48.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.843+0000 7fb16cc41700 1 -- start start 2022-08-01T19:40:48.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.843+0000 7fb16cc41700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb1680619b0 0x7fb1680b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:48.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.843+0000 7fb16cc41700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1680b5ed0 0x7fb1680b5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:48.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.843+0000 7fb16cc41700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb168061d90 con 0x7fb1680b6ee0 2022-08-01T19:40:48.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.843+0000 7fb16cc41700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb168143fa0 con 0x7fb1680619b0 2022-08-01T19:40:48.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.843+0000 7fb16cc41700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb168142f60 con 0x7fb1680b5ed0 2022-08-01T19:40:48.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.843+0000 7fb1677fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb1680619b0 0x7fb1680b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:48.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.843+0000 7fb1677fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb1680619b0 0x7fb1680b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40946/0 (socket says 172.21.15.17:40946) 2022-08-01T19:40:48.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.843+0000 7fb1677fe700 1 -- 172.21.15.17:0/2452576784 learned_addr learned my addr 172.21.15.17:0/2452576784 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:48.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.843+0000 7fb167fff700 1 --2- 172.21.15.17:0/2452576784 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1680b5ed0 0x7fb1680b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:48.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.843+0000 7fb1677fe700 1 -- 172.21.15.17:0/2452576784 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1680b5ed0 msgr2=0x7fb1680b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:48.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.843+0000 7fb1677fe700 1 --2- 172.21.15.17:0/2452576784 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1680b5ed0 0x7fb1680b5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:48.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.843+0000 7fb1677fe700 1 -- 172.21.15.17:0/2452576784 >> v1:172.21.15.77:6789/0 conn(0x7fb1680b6ee0 legacy=0x7fb168143890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:48.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.843+0000 7fb1667fc700 1 -- 172.21.15.17:0/2452576784 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4131953945 0 0) 0x7fb168061d90 con 0x7fb1680b6ee0 2022-08-01T19:40:48.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.843+0000 7fb1677fe700 1 -- 172.21.15.17:0/2452576784 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1680ba5a0 con 0x7fb1680619b0 2022-08-01T19:40:48.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.843+0000 7fb1677fe700 1 --2- 172.21.15.17:0/2452576784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1680619b0 0x7fb1680b48a0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7fb1680b6880 tx=0x7fb150006eb0).ready entity=mon.0 client_cookie=67be5043e01770ab server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:48.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.843+0000 7fb1667fc700 1 -- 172.21.15.17:0/2452576784 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb15000d5c0 con 0x7fb1680619b0 2022-08-01T19:40:48.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.843+0000 7fb1667fc700 1 -- 172.21.15.17:0/2452576784 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb150000e90 con 0x7fb1680619b0 2022-08-01T19:40:48.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.843+0000 7fb1667fc700 1 -- 172.21.15.17:0/2452576784 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb15000d910 con 0x7fb1680619b0 2022-08-01T19:40:48.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.843+0000 7fb16cc41700 1 -- 172.21.15.17:0/2452576784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1680619b0 msgr2=0x7fb1680b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:48.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.843+0000 7fb16cc41700 1 --2- 172.21.15.17:0/2452576784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1680619b0 0x7fb1680b48a0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7fb1680b6880 tx=0x7fb150006eb0).stop 2022-08-01T19:40:48.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.843+0000 7fb16cc41700 1 -- 172.21.15.17:0/2452576784 shutdown_connections 2022-08-01T19:40:48.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.843+0000 7fb16cc41700 1 --2- 172.21.15.17:0/2452576784 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1680b5ed0 0x7fb1680b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:48.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.843+0000 7fb16cc41700 1 --2- 172.21.15.17:0/2452576784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1680619b0 0x7fb1680b48a0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:48.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.843+0000 7fb16cc41700 1 -- 172.21.15.17:0/2452576784 >> 172.21.15.17:0/2452576784 conn(0x7fb16806af80 msgr2=0x7fb16805fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:48.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.843+0000 7fb16cc41700 1 -- 172.21.15.17:0/2452576784 shutdown_connections 2022-08-01T19:40:48.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.843+0000 7fb16cc41700 1 -- 172.21.15.17:0/2452576784 wait complete. 2022-08-01T19:40:48.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.847+0000 7fb16cc41700 1 Processor -- start 2022-08-01T19:40:48.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.847+0000 7fb16cc41700 1 -- start start 2022-08-01T19:40:48.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.847+0000 7fb16cc41700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1680619b0 0x7fb168146670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:48.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.847+0000 7fb16cc41700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1680b5ed0 0x7fb16813e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:48.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.847+0000 7fb16cc41700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1680b6ee0 0x7fb1680adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:48.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.847+0000 7fb16cc41700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb168145af0 con 0x7fb1680619b0 2022-08-01T19:40:48.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.847+0000 7fb16cc41700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb168112ed0 con 0x7fb1680b6ee0 2022-08-01T19:40:48.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.847+0000 7fb16cc41700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1680adb40 con 0x7fb1680b5ed0 2022-08-01T19:40:48.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.847+0000 7fb1677fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1680619b0 0x7fb168146670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:48.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.847+0000 7fb1677fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1680619b0 0x7fb168146670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40952/0 (socket says 172.21.15.17:40952) 2022-08-01T19:40:48.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.847+0000 7fb1677fe700 1 -- 172.21.15.17:0/1635080978 learned_addr learned my addr 172.21.15.17:0/1635080978 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:48.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.847+0000 7fb167fff700 1 --2- 172.21.15.17:0/1635080978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1680b6ee0 0x7fb1680adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:48.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.847+0000 7fb166ffd700 1 --2- 172.21.15.17:0/1635080978 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1680b5ed0 0x7fb16813e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:48.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.847+0000 7fb1677fe700 1 -- 172.21.15.17:0/1635080978 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1680b5ed0 msgr2=0x7fb16813e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:48.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.847+0000 7fb1677fe700 1 --2- 172.21.15.17:0/1635080978 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1680b5ed0 0x7fb16813e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:48.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.847+0000 7fb1677fe700 1 -- 172.21.15.17:0/1635080978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1680b6ee0 msgr2=0x7fb1680adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:48.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.847+0000 7fb1677fe700 1 --2- 172.21.15.17:0/1635080978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1680b6ee0 0x7fb1680adfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:48.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.847+0000 7fb1677fe700 1 -- 172.21.15.17:0/1635080978 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb15000b1e0 con 0x7fb1680619b0 2022-08-01T19:40:48.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.847+0000 7fb1677fe700 1 --2- 172.21.15.17:0/1635080978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1680619b0 0x7fb168146670 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7fb168146290 tx=0x7fb150015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:48.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.847+0000 7fb164ff9700 1 -- 172.21.15.17:0/1635080978 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1500116e0 con 0x7fb1680619b0 2022-08-01T19:40:48.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.847+0000 7fb164ff9700 1 -- 172.21.15.17:0/1635080978 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb150011840 con 0x7fb1680619b0 2022-08-01T19:40:48.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.847+0000 7fb16cc41700 1 -- 172.21.15.17:0/1635080978 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb168156510 con 0x7fb1680619b0 2022-08-01T19:40:48.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.847+0000 7fb164ff9700 1 -- 172.21.15.17:0/1635080978 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb150011ba0 con 0x7fb1680619b0 2022-08-01T19:40:48.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.847+0000 7fb16cc41700 1 -- 172.21.15.17:0/1635080978 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb168112a30 con 0x7fb1680619b0 2022-08-01T19:40:48.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.847+0000 7fb16cc41700 1 -- 172.21.15.17:0/1635080978 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1680574e0 con 0x7fb1680619b0 2022-08-01T19:40:48.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.851+0000 7fb164ff9700 1 -- 172.21.15.17:0/1635080978 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb15000f070 con 0x7fb1680619b0 2022-08-01T19:40:48.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.851+0000 7fb164ff9700 1 --2- 172.21.15.17:0/1635080978 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb140053a70 0x7fb140055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:48.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.851+0000 7fb164ff9700 1 -- 172.21.15.17:0/1635080978 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6463+0+0 (secure 0 0 0) 0x7fb15006e370 con 0x7fb1680619b0 2022-08-01T19:40:48.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.851+0000 7fb166ffd700 1 --2- 172.21.15.17:0/1635080978 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb140053a70 0x7fb140055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:48.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.851+0000 7fb166ffd700 1 --2- 172.21.15.17:0/1635080978 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb140053a70 0x7fb140055f30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7fb15800d000 tx=0x7fb158007b90).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:48.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:48.859+0000 7fb164ff9700 1 -- 172.21.15.17:0/1635080978 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb150048a30 con 0x7fb1680619b0 2022-08-01T19:40:49.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.023+0000 7fb16cc41700 1 -- 172.21.15.17:0/1635080978 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb1680add00 con 0x7fb1680619b0 2022-08-01T19:40:49.492 INFO:tasks.ceph.mon.a.smithi017.stderr:2022-08-01T19:40:49.487+0000 7f6a53773700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-08-01T19:40:49.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.499+0000 7fb164ff9700 1 -- 172.21.15.17:0/1635080978 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb150048620 con 0x7fb1680619b0 2022-08-01T19:40:49.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.503+0000 7fb16cc41700 1 -- 172.21.15.17:0/1635080978 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb140053a70 msgr2=0x7fb140055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:49.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.503+0000 7fb16cc41700 1 --2- 172.21.15.17:0/1635080978 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb140053a70 0x7fb140055f30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7fb15800d000 tx=0x7fb158007b90).stop 2022-08-01T19:40:49.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.503+0000 7fb16cc41700 1 -- 172.21.15.17:0/1635080978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1680619b0 msgr2=0x7fb168146670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:49.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.503+0000 7fb16cc41700 1 --2- 172.21.15.17:0/1635080978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1680619b0 0x7fb168146670 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7fb168146290 tx=0x7fb150015000).stop 2022-08-01T19:40:49.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.503+0000 7fb16cc41700 1 -- 172.21.15.17:0/1635080978 shutdown_connections 2022-08-01T19:40:49.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.503+0000 7fb16cc41700 1 --2- 172.21.15.17:0/1635080978 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb140053a70 0x7fb140055f30 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:49.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.503+0000 7fb16cc41700 1 --2- 172.21.15.17:0/1635080978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1680b6ee0 0x7fb1680adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:49.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.503+0000 7fb16cc41700 1 --2- 172.21.15.17:0/1635080978 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1680b5ed0 0x7fb16813e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:49.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.503+0000 7fb16cc41700 1 --2- 172.21.15.17:0/1635080978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1680619b0 0x7fb168146670 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:49.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.503+0000 7fb16cc41700 1 -- 172.21.15.17:0/1635080978 >> 172.21.15.17:0/1635080978 conn(0x7fb16806af80 msgr2=0x7fb16805fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:49.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.503+0000 7fb16cc41700 1 -- 172.21.15.17:0/1635080978 shutdown_connections 2022-08-01T19:40:49.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.503+0000 7fb16cc41700 1 -- 172.21.15.17:0/1635080978 wait complete. 2022-08-01T19:40:49.512 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-08-01T19:40:49.523 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:40:49.530 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2022-08-01T19:40:49.645 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.635+0000 7fd66759e700 1 Processor -- start 2022-08-01T19:40:49.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.635+0000 7fd66759e700 1 -- start start 2022-08-01T19:40:49.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.639+0000 7fd66759e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd6680af450 0x7fd6680af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:49.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.639+0000 7fd66759e700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6680aff30 con 0x7fd6680ad4b0 2022-08-01T19:40:49.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.639+0000 7fd66759e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd66805c7a0 con 0x7fd6680ad9e0 2022-08-01T19:40:49.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.639+0000 7fd66759e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd668140990 con 0x7fd6680af450 2022-08-01T19:40:49.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.639+0000 7fd65ffff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd6680ad4b0 0x7fd6681432e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33168/0 (socket says 172.21.15.17:33168) 2022-08-01T19:40:49.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.639+0000 7fd65ffff700 1 -- 172.21.15.17:0/1156668733 learned_addr learned my addr 172.21.15.17:0/1156668733 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:40:49.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.639+0000 7fd66659c700 1 --2- 172.21.15.17:0/1156668733 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd6680af450 0x7fd6680af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:49.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.639+0000 7fd665d9b700 1 -- 172.21.15.17:0/1156668733 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2238241987 0 0) 0x7fd6680aff30 con 0x7fd6680ad4b0 2022-08-01T19:40:49.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.639+0000 7fd66659c700 1 -- 172.21.15.17:0/1156668733 >> v1:172.21.15.77:6789/0 conn(0x7fd6680ad9e0 legacy=0x7fd6680a8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:49.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.639+0000 7fd66659c700 1 -- 172.21.15.17:0/1156668733 >> v1:172.21.15.17:6789/0 conn(0x7fd6680ad4b0 legacy=0x7fd6681432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:49.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.639+0000 7fd66659c700 1 -- 172.21.15.17:0/1156668733 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6681439f0 con 0x7fd6680af450 2022-08-01T19:40:49.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.639+0000 7fd66659c700 1 --2- 172.21.15.17:0/1156668733 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd6680af450 0x7fd6680af830 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fd658002f70 tx=0x7fd658006df0).ready entity=mon.2 client_cookie=c5e4030ce17addb server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:49.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.639+0000 7fd665d9b700 1 -- 172.21.15.17:0/1156668733 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd65800d5c0 con 0x7fd6680af450 2022-08-01T19:40:49.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.639+0000 7fd665d9b700 1 -- 172.21.15.17:0/1156668733 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd658000e90 con 0x7fd6680af450 2022-08-01T19:40:49.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.639+0000 7fd665d9b700 1 -- 172.21.15.17:0/1156668733 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd65800d910 con 0x7fd6680af450 2022-08-01T19:40:49.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.643+0000 7fd66759e700 1 -- 172.21.15.17:0/1156668733 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd6680af450 msgr2=0x7fd6680af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:49.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.643+0000 7fd66759e700 1 --2- 172.21.15.17:0/1156668733 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd6680af450 0x7fd6680af830 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fd658002f70 tx=0x7fd658006df0).stop 2022-08-01T19:40:49.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.643+0000 7fd66759e700 1 -- 172.21.15.17:0/1156668733 shutdown_connections 2022-08-01T19:40:49.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.643+0000 7fd66759e700 1 --2- 172.21.15.17:0/1156668733 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd6680af450 0x7fd6680af830 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:49.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.643+0000 7fd66759e700 1 -- 172.21.15.17:0/1156668733 >> 172.21.15.17:0/1156668733 conn(0x7fd6680612d0 msgr2=0x7fd668062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:49.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.643+0000 7fd66759e700 1 -- 172.21.15.17:0/1156668733 shutdown_connections 2022-08-01T19:40:49.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.643+0000 7fd66759e700 1 -- 172.21.15.17:0/1156668733 wait complete. 2022-08-01T19:40:49.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.643+0000 7fd66759e700 1 Processor -- start 2022-08-01T19:40:49.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.643+0000 7fd66759e700 1 -- start start 2022-08-01T19:40:49.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.643+0000 7fd66759e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6680ad4b0 0x7fd66813f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:49.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.643+0000 7fd66759e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6680ad9e0 0x7fd668152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:49.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.643+0000 7fd66759e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd6680af450 0x7fd668153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:49.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.643+0000 7fd66759e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6680aa030 con 0x7fd6680ad4b0 2022-08-01T19:40:49.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.643+0000 7fd66759e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6681528b0 con 0x7fd6680ad9e0 2022-08-01T19:40:49.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.643+0000 7fd66759e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd668064e10 con 0x7fd6680af450 2022-08-01T19:40:49.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.647+0000 7fd66659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6680ad4b0 0x7fd66813f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:49.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.647+0000 7fd65ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6680ad9e0 0x7fd668152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:49.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.647+0000 7fd666d9d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd6680af450 0x7fd668153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:49.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.647+0000 7fd66659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6680ad4b0 0x7fd66813f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40978/0 (socket says 172.21.15.17:40978) 2022-08-01T19:40:49.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.647+0000 7fd65ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6680ad9e0 0x7fd668152d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:38902/0 (socket says 172.21.15.17:38902) 2022-08-01T19:40:49.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.647+0000 7fd666d9d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd6680af450 0x7fd668153240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:37082/0 (socket says 172.21.15.17:37082) 2022-08-01T19:40:49.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.647+0000 7fd66659c700 1 -- 172.21.15.17:0/3762548795 learned_addr learned my addr 172.21.15.17:0/3762548795 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:49.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.647+0000 7fd66659c700 1 -- 172.21.15.17:0/3762548795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd6680af450 msgr2=0x7fd668153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:49.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.647+0000 7fd66659c700 1 --2- 172.21.15.17:0/3762548795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd6680af450 0x7fd668153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:49.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.647+0000 7fd66659c700 1 -- 172.21.15.17:0/3762548795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6680ad9e0 msgr2=0x7fd668152d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:49.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.647+0000 7fd66659c700 1 --2- 172.21.15.17:0/3762548795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6680ad9e0 0x7fd668152d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:49.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.647+0000 7fd66659c700 1 -- 172.21.15.17:0/3762548795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65800b180 con 0x7fd6680ad4b0 2022-08-01T19:40:49.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.647+0000 7fd66659c700 1 --2- 172.21.15.17:0/3762548795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6680ad4b0 0x7fd66813f400 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7fd658002f70 tx=0x7fd658015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:49.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.647+0000 7fd65f7fe700 1 -- 172.21.15.17:0/3762548795 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd65800de80 con 0x7fd6680ad4b0 2022-08-01T19:40:49.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.647+0000 7fd65f7fe700 1 -- 172.21.15.17:0/3762548795 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd658011b40 con 0x7fd6680ad4b0 2022-08-01T19:40:49.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.647+0000 7fd65f7fe700 1 -- 172.21.15.17:0/3762548795 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd658011ea0 con 0x7fd6680ad4b0 2022-08-01T19:40:49.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.647+0000 7fd66759e700 1 -- 172.21.15.17:0/3762548795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6680a8f10 con 0x7fd6680ad4b0 2022-08-01T19:40:49.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.647+0000 7fd66759e700 1 -- 172.21.15.17:0/3762548795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6680a9ad0 con 0x7fd6680ad4b0 2022-08-01T19:40:49.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.647+0000 7fd66759e700 1 -- 172.21.15.17:0/3762548795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6680a9e90 con 0x7fd6680ad4b0 2022-08-01T19:40:49.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.647+0000 7fd65f7fe700 1 -- 172.21.15.17:0/3762548795 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd65801a040 con 0x7fd6680ad4b0 2022-08-01T19:40:49.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.651+0000 7fd65f7fe700 1 --2- 172.21.15.17:0/3762548795 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd64c053b00 0x7fd64c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:49.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.651+0000 7fd65f7fe700 1 -- 172.21.15.17:0/3762548795 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6549+0+0 (secure 0 0 0) 0x7fd65806d100 con 0x7fd6680ad4b0 2022-08-01T19:40:49.692 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.651+0000 7fd65ffff700 1 --2- 172.21.15.17:0/3762548795 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd64c053b00 0x7fd64c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:49.692 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.651+0000 7fd65ffff700 1 --2- 172.21.15.17:0/3762548795 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd64c053b00 0x7fd64c055fc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7fd65000d000 tx=0x7fd6500011a0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:49.692 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.655+0000 7fd65f7fe700 1 -- 172.21.15.17:0/3762548795 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd6680a9e90 con 0x7fd6680ad4b0 2022-08-01T19:40:49.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.823+0000 7fd66759e700 1 -- 172.21.15.17:0/3762548795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6680a9e90 con 0x7fd6680ad4b0 2022-08-01T19:40:49.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.823+0000 7fd65f7fe700 1 -- 172.21.15.17:0/3762548795 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+2132 (secure 0 0 0) 0x7fd6680a9e90 con 0x7fd6680ad4b0 2022-08-01T19:40:49.830 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:40:49.831 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":36,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5615,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/2284778179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2284778179},{"type":"v1","addr":"172.21.15.77:6835","nonce":2284778179}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":34},{"gid":6261,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/1827111044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":1827111044},{"type":"v1","addr":"172.21.15.17:6833","nonce":1827111044}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:39:52.325506+0000","modified":"2022-08-01T19:40:49.493555+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-08-01T19:40:49.834 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.823+0000 7fd66759e700 1 -- 172.21.15.17:0/3762548795 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd64c053b00 msgr2=0x7fd64c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:49.834 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.823+0000 7fd66759e700 1 --2- 172.21.15.17:0/3762548795 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd64c053b00 0x7fd64c055fc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7fd65000d000 tx=0x7fd6500011a0).stop 2022-08-01T19:40:49.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.823+0000 7fd66759e700 1 -- 172.21.15.17:0/3762548795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6680ad4b0 msgr2=0x7fd66813f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:49.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.823+0000 7fd66759e700 1 --2- 172.21.15.17:0/3762548795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6680ad4b0 0x7fd66813f400 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7fd658002f70 tx=0x7fd658015000).stop 2022-08-01T19:40:49.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.823+0000 7fd66759e700 1 -- 172.21.15.17:0/3762548795 shutdown_connections 2022-08-01T19:40:49.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.823+0000 7fd66759e700 1 --2- 172.21.15.17:0/3762548795 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd64c053b00 0x7fd64c055fc0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:49.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.823+0000 7fd66759e700 1 --2- 172.21.15.17:0/3762548795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd6680af450 0x7fd668153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:49.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.823+0000 7fd66759e700 1 --2- 172.21.15.17:0/3762548795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6680ad9e0 0x7fd668152d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:49.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.823+0000 7fd66759e700 1 --2- 172.21.15.17:0/3762548795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6680ad4b0 0x7fd66813f400 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:49.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.823+0000 7fd66759e700 1 -- 172.21.15.17:0/3762548795 >> 172.21.15.17:0/3762548795 conn(0x7fd6680612d0 msgr2=0x7fd6680b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:49.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.823+0000 7fd66759e700 1 -- 172.21.15.17:0/3762548795 shutdown_connections 2022-08-01T19:40:49.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.823+0000 7fd66759e700 1 -- 172.21.15.17:0/3762548795 wait complete. 2022-08-01T19:40:49.843 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 36 2022-08-01T19:40:49.844 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-08-01T19:40:49.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.951+0000 7fd278802700 1 Processor -- start 2022-08-01T19:40:49.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.951+0000 7fd278802700 1 -- start start 2022-08-01T19:40:49.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.951+0000 7fd278802700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2740b2710 0x7fd27414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:49.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.951+0000 7fd278802700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fd2740a8c50 0x7fd27414b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:49.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.951+0000 7fd278802700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2740a7250 con 0x7fd2740b01a0 2022-08-01T19:40:49.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.951+0000 7fd278802700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2740a7480 con 0x7fd2740a8c50 2022-08-01T19:40:49.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.951+0000 7fd278802700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2740a7720 con 0x7fd2740b2710 2022-08-01T19:40:49.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.951+0000 7fd27259c700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd2740b01a0 0x7fd2740a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33180/0 (socket says 172.21.15.17:33180) 2022-08-01T19:40:49.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.951+0000 7fd27259c700 1 -- 172.21.15.17:0/1344199399 learned_addr learned my addr 172.21.15.17:0/1344199399 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:40:49.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.951+0000 7fd27359e700 1 --2- 172.21.15.17:0/1344199399 >> v2:172.21.15.77:3300/0 conn(0x7fd2740a8c50 0x7fd27414b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:49.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.951+0000 7fd272d9d700 1 --2- 172.21.15.17:0/1344199399 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2740b2710 0x7fd27414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:49.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.951+0000 7fd271d9b700 1 -- 172.21.15.17:0/1344199399 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 525011435 0 0) 0x7fd2740a7250 con 0x7fd2740b01a0 2022-08-01T19:40:49.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.951+0000 7fd271d9b700 1 -- 172.21.15.17:0/1344199399 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd258003260 con 0x7fd2740b01a0 2022-08-01T19:40:49.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.955+0000 7fd27359e700 1 -- 172.21.15.17:0/1344199399 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2740b2710 msgr2=0x7fd27414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:49.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.955+0000 7fd27359e700 1 --2- 172.21.15.17:0/1344199399 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2740b2710 0x7fd27414afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:49.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.955+0000 7fd27359e700 1 -- 172.21.15.17:0/1344199399 >> v1:172.21.15.17:6789/0 conn(0x7fd2740b01a0 legacy=0x7fd2740a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:49.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.955+0000 7fd27359e700 1 -- 172.21.15.17:0/1344199399 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2740a79c0 con 0x7fd2740a8c50 2022-08-01T19:40:49.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.955+0000 7fd272d9d700 1 --2- 172.21.15.17:0/1344199399 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2740b2710 0x7fd27414afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:40:49.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.955+0000 7fd27359e700 1 --2- 172.21.15.17:0/1344199399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2740a8c50 0x7fd27414b4c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7fd26c002f70 tx=0x7fd26c006df0).ready entity=mon.1 client_cookie=3898b0eefeae2f70 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:49.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.955+0000 7fd271d9b700 1 -- 172.21.15.17:0/1344199399 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd26c00d4a0 con 0x7fd2740a8c50 2022-08-01T19:40:49.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.955+0000 7fd271d9b700 1 -- 172.21.15.17:0/1344199399 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd26c000e90 con 0x7fd2740a8c50 2022-08-01T19:40:49.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.955+0000 7fd271d9b700 1 -- 172.21.15.17:0/1344199399 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd26c00d7f0 con 0x7fd2740a8c50 2022-08-01T19:40:49.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.955+0000 7fd278802700 1 -- 172.21.15.17:0/1344199399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2740a8c50 msgr2=0x7fd27414b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:49.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.955+0000 7fd278802700 1 --2- 172.21.15.17:0/1344199399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2740a8c50 0x7fd27414b4c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7fd26c002f70 tx=0x7fd26c006df0).stop 2022-08-01T19:40:49.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.955+0000 7fd278802700 1 -- 172.21.15.17:0/1344199399 shutdown_connections 2022-08-01T19:40:49.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.955+0000 7fd278802700 1 --2- 172.21.15.17:0/1344199399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2740a8c50 0x7fd27414b4c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:49.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.955+0000 7fd278802700 1 --2- 172.21.15.17:0/1344199399 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2740b2710 0x7fd27414afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:49.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.955+0000 7fd278802700 1 -- 172.21.15.17:0/1344199399 >> 172.21.15.17:0/1344199399 conn(0x7fd274060310 msgr2=0x7fd2740b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:49.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.955+0000 7fd278802700 1 -- 172.21.15.17:0/1344199399 shutdown_connections 2022-08-01T19:40:49.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.955+0000 7fd278802700 1 -- 172.21.15.17:0/1344199399 wait complete. 2022-08-01T19:40:49.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.955+0000 7fd278802700 1 Processor -- start 2022-08-01T19:40:49.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.955+0000 7fd278802700 1 -- start start 2022-08-01T19:40:49.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.959+0000 7fd278802700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2740a8c50 0x7fd2740b3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:49.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.959+0000 7fd278802700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2740b01a0 0x7fd2740a9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:49.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.959+0000 7fd278802700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2740b2710 0x7fd2740aa270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:49.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.959+0000 7fd278802700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27414ce20 con 0x7fd2740b01a0 2022-08-01T19:40:49.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.959+0000 7fd278802700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2740b3f90 con 0x7fd2740b2710 2022-08-01T19:40:49.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.959+0000 7fd278802700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd27414c6e0 con 0x7fd2740a8c50 2022-08-01T19:40:49.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.959+0000 7fd27259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2740b01a0 0x7fd2740a9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:49.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.959+0000 7fd27259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2740b01a0 0x7fd2740a9d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40994/0 (socket says 172.21.15.17:40994) 2022-08-01T19:40:49.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.959+0000 7fd27359e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2740b2710 0x7fd2740aa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:49.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.959+0000 7fd27259c700 1 -- 172.21.15.17:0/3000658536 learned_addr learned my addr 172.21.15.17:0/3000658536 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:49.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.959+0000 7fd272d9d700 1 --2- 172.21.15.17:0/3000658536 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2740a8c50 0x7fd2740b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:49.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.959+0000 7fd27259c700 1 -- 172.21.15.17:0/3000658536 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2740a8c50 msgr2=0x7fd2740b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:49.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.959+0000 7fd27259c700 1 --2- 172.21.15.17:0/3000658536 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2740a8c50 0x7fd2740b3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:49.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.959+0000 7fd27259c700 1 -- 172.21.15.17:0/3000658536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2740b2710 msgr2=0x7fd2740aa270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:49.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.959+0000 7fd27259c700 1 --2- 172.21.15.17:0/3000658536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2740b2710 0x7fd2740aa270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:49.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.959+0000 7fd27259c700 1 -- 172.21.15.17:0/3000658536 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd26800b180 con 0x7fd2740b01a0 2022-08-01T19:40:49.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.959+0000 7fd27359e700 1 --2- 172.21.15.17:0/3000658536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2740b2710 0x7fd2740aa270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:40:49.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.959+0000 7fd27259c700 1 --2- 172.21.15.17:0/3000658536 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2740b01a0 0x7fd2740a9d80 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7fd264001490 tx=0x7fd264007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:49.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.959+0000 7fd263fff700 1 -- 172.21.15.17:0/3000658536 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd264005ba0 con 0x7fd2740b01a0 2022-08-01T19:40:49.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.959+0000 7fd263fff700 1 -- 172.21.15.17:0/3000658536 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd264005d00 con 0x7fd2740b01a0 2022-08-01T19:40:49.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.959+0000 7fd278802700 1 -- 172.21.15.17:0/3000658536 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd26c00b180 con 0x7fd2740b01a0 2022-08-01T19:40:49.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.959+0000 7fd263fff700 1 -- 172.21.15.17:0/3000658536 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2640072c0 con 0x7fd2740b01a0 2022-08-01T19:40:49.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.959+0000 7fd278802700 1 -- 172.21.15.17:0/3000658536 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2740b4230 con 0x7fd2740b01a0 2022-08-01T19:40:49.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.959+0000 7fd278802700 1 -- 172.21.15.17:0/3000658536 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2740574e0 con 0x7fd2740b01a0 2022-08-01T19:40:49.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.963+0000 7fd263fff700 1 -- 172.21.15.17:0/3000658536 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd26401a070 con 0x7fd2740b01a0 2022-08-01T19:40:49.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.963+0000 7fd263fff700 1 --2- 172.21.15.17:0/3000658536 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd250053a10 0x7fd250055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:49.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.963+0000 7fd263fff700 1 -- 172.21.15.17:0/3000658536 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6549+0+0 (secure 0 0 0) 0x7fd26406b650 con 0x7fd2740b01a0 2022-08-01T19:40:49.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.963+0000 7fd272d9d700 1 --2- 172.21.15.17:0/3000658536 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd250053a10 0x7fd250055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:49.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.963+0000 7fd272d9d700 1 --2- 172.21.15.17:0/3000658536 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd250053a10 0x7fd250055ed0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7fd26800b000 tx=0x7fd268006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:49.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:49.971+0000 7fd263fff700 1 -- 172.21.15.17:0/3000658536 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd264042390 con 0x7fd2740b01a0 2022-08-01T19:40:50.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.135+0000 7fd278802700 1 -- 172.21.15.17:0/3000658536 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd27414cb80 con 0x7fd2740b01a0 2022-08-01T19:40:50.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.507+0000 7fd263fff700 1 -- 172.21.15.17:0/3000658536 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd264018030 con 0x7fd2740b01a0 2022-08-01T19:40:50.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.511+0000 7fd278802700 1 -- 172.21.15.17:0/3000658536 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd250053a10 msgr2=0x7fd250055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:50.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.511+0000 7fd278802700 1 --2- 172.21.15.17:0/3000658536 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd250053a10 0x7fd250055ed0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7fd26800b000 tx=0x7fd268006df0).stop 2022-08-01T19:40:50.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.511+0000 7fd278802700 1 -- 172.21.15.17:0/3000658536 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2740b01a0 msgr2=0x7fd2740a9d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:50.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.511+0000 7fd278802700 1 --2- 172.21.15.17:0/3000658536 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2740b01a0 0x7fd2740a9d80 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7fd264001490 tx=0x7fd264007e20).stop 2022-08-01T19:40:50.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.511+0000 7fd278802700 1 -- 172.21.15.17:0/3000658536 shutdown_connections 2022-08-01T19:40:50.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.511+0000 7fd278802700 1 --2- 172.21.15.17:0/3000658536 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd250053a10 0x7fd250055ed0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:50.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.511+0000 7fd278802700 1 --2- 172.21.15.17:0/3000658536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2740b2710 0x7fd2740aa270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:50.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.511+0000 7fd278802700 1 --2- 172.21.15.17:0/3000658536 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2740b01a0 0x7fd2740a9d80 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:50.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.511+0000 7fd278802700 1 --2- 172.21.15.17:0/3000658536 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2740a8c50 0x7fd2740b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:50.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.511+0000 7fd278802700 1 -- 172.21.15.17:0/3000658536 >> 172.21.15.17:0/3000658536 conn(0x7fd274060310 msgr2=0x7fd274148850 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:50.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.511+0000 7fd278802700 1 -- 172.21.15.17:0/3000658536 shutdown_connections 2022-08-01T19:40:50.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.511+0000 7fd278802700 1 -- 172.21.15.17:0/3000658536 wait complete. 2022-08-01T19:40:50.557 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-08-01T19:40:50.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.675+0000 7f8edb59e700 1 Processor -- start 2022-08-01T19:40:50.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.675+0000 7f8edb59e700 1 -- start start 2022-08-01T19:40:50.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.679+0000 7f8edb59e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f8edc061890 0x7f8edc1466b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:50.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.679+0000 7f8edb59e700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8edc0aee80 con 0x7f8edc0614b0 2022-08-01T19:40:50.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.679+0000 7f8edb59e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8edc0b70b0 con 0x7f8edc0ad750 2022-08-01T19:40:50.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.679+0000 7f8edb59e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8edc0af0b0 con 0x7f8edc061890 2022-08-01T19:40:50.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.679+0000 7f8eda59c700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f8edc0614b0 0x7f8edc0b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33196/0 (socket says 172.21.15.17:33196) 2022-08-01T19:40:50.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.679+0000 7f8eda59c700 1 -- 172.21.15.17:0/3969455355 learned_addr learned my addr 172.21.15.17:0/3969455355 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:40:50.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.679+0000 7f8edad9d700 1 --2- 172.21.15.17:0/3969455355 >> v2:172.21.15.77:3300/0 conn(0x7f8edc061890 0x7f8edc1466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:50.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.679+0000 7f8ed959a700 1 -- 172.21.15.17:0/3969455355 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2770467798 0 0) 0x7f8edc0aee80 con 0x7f8edc0614b0 2022-08-01T19:40:50.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.679+0000 7f8ed959a700 1 -- 172.21.15.17:0/3969455355 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ebc003260 con 0x7f8edc0614b0 2022-08-01T19:40:50.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.679+0000 7f8edad9d700 1 -- 172.21.15.17:0/3969455355 >> v1:172.21.15.77:6789/0 conn(0x7f8edc0ad750 legacy=0x7f8edc1405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:50.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.679+0000 7f8edad9d700 1 -- 172.21.15.17:0/3969455355 >> v1:172.21.15.17:6789/0 conn(0x7f8edc0614b0 legacy=0x7f8edc0b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:50.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.679+0000 7f8edad9d700 1 -- 172.21.15.17:0/3969455355 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8edc0af350 con 0x7f8edc061890 2022-08-01T19:40:50.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.679+0000 7f8edad9d700 1 --2- 172.21.15.17:0/3969455355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8edc061890 0x7f8edc1466b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f8ed4002f70 tx=0x7f8ed4006df0).ready entity=mon.1 client_cookie=c7b8980d312df81c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:50.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.679+0000 7f8ed959a700 1 -- 172.21.15.17:0/3969455355 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ed400d610 con 0x7f8edc061890 2022-08-01T19:40:50.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.679+0000 7f8ed959a700 1 -- 172.21.15.17:0/3969455355 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed4000e90 con 0x7f8edc061890 2022-08-01T19:40:50.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.679+0000 7f8ed959a700 1 -- 172.21.15.17:0/3969455355 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ed400d960 con 0x7f8edc061890 2022-08-01T19:40:50.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.679+0000 7f8edb59e700 1 -- 172.21.15.17:0/3969455355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8edc061890 msgr2=0x7f8edc1466b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:50.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.679+0000 7f8edb59e700 1 --2- 172.21.15.17:0/3969455355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8edc061890 0x7f8edc1466b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f8ed4002f70 tx=0x7f8ed4006df0).stop 2022-08-01T19:40:50.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.679+0000 7f8edb59e700 1 -- 172.21.15.17:0/3969455355 shutdown_connections 2022-08-01T19:40:50.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.679+0000 7f8edb59e700 1 --2- 172.21.15.17:0/3969455355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8edc061890 0x7f8edc1466b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:50.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.679+0000 7f8edb59e700 1 -- 172.21.15.17:0/3969455355 >> 172.21.15.17:0/3969455355 conn(0x7f8edc066f70 msgr2=0x7f8edc067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:50.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.679+0000 7f8edb59e700 1 -- 172.21.15.17:0/3969455355 shutdown_connections 2022-08-01T19:40:50.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.679+0000 7f8edb59e700 1 -- 172.21.15.17:0/3969455355 wait complete. 2022-08-01T19:40:50.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.683+0000 7f8edb59e700 1 Processor -- start 2022-08-01T19:40:50.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.683+0000 7f8edb59e700 1 -- start start 2022-08-01T19:40:50.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.683+0000 7f8edb59e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8edc0614b0 0x7f8edc05d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:50.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.683+0000 7f8edb59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8edc061890 0x7f8edc1527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:50.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.683+0000 7f8edb59e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8edc0ad750 0x7f8edc152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:50.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.683+0000 7f8edb59e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc0aab10 con 0x7f8edc0ad750 2022-08-01T19:40:50.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.683+0000 7f8edb59e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc0ad1f0 con 0x7f8edc0614b0 2022-08-01T19:40:50.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.683+0000 7f8edb59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc0b0e30 con 0x7f8edc061890 2022-08-01T19:40:50.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.683+0000 7f8edad9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8edc0ad750 0x7f8edc152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:50.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.683+0000 7f8edad9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8edc0ad750 0x7f8edc152cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41012/0 (socket says 172.21.15.17:41012) 2022-08-01T19:40:50.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.683+0000 7f8edad9d700 1 -- 172.21.15.17:0/925895546 learned_addr learned my addr 172.21.15.17:0/925895546 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:50.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.683+0000 7f8eda59c700 1 --2- 172.21.15.17:0/925895546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8edc0614b0 0x7f8edc05d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:50.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.683+0000 7f8ed9d9b700 1 --2- 172.21.15.17:0/925895546 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8edc061890 0x7f8edc1527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:50.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.683+0000 7f8edad9d700 1 -- 172.21.15.17:0/925895546 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8edc061890 msgr2=0x7f8edc1527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:50.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.683+0000 7f8edad9d700 1 --2- 172.21.15.17:0/925895546 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8edc061890 0x7f8edc1527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:50.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.683+0000 7f8edad9d700 1 -- 172.21.15.17:0/925895546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8edc0614b0 msgr2=0x7f8edc05d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:50.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.683+0000 7f8edad9d700 1 --2- 172.21.15.17:0/925895546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8edc0614b0 0x7f8edc05d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:50.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.683+0000 7f8edad9d700 1 -- 172.21.15.17:0/925895546 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed400b180 con 0x7f8edc0ad750 2022-08-01T19:40:50.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.683+0000 7f8ed9d9b700 1 --2- 172.21.15.17:0/925895546 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8edc061890 0x7f8edc1527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:40:50.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.683+0000 7f8edad9d700 1 --2- 172.21.15.17:0/925895546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8edc0ad750 0x7f8edc152cc0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7f8ed4002f70 tx=0x7f8ed4014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:50.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.683+0000 7f8ed37fe700 1 -- 172.21.15.17:0/925895546 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ed400d290 con 0x7f8edc0ad750 2022-08-01T19:40:50.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.687+0000 7f8ed37fe700 1 -- 172.21.15.17:0/925895546 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed4013e90 con 0x7f8edc0ad750 2022-08-01T19:40:50.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.687+0000 7f8ed37fe700 1 -- 172.21.15.17:0/925895546 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ed401e5f0 con 0x7f8edc0ad750 2022-08-01T19:40:50.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.687+0000 7f8edb59e700 1 -- 172.21.15.17:0/925895546 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8edc0b2ed0 con 0x7f8edc0ad750 2022-08-01T19:40:50.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.687+0000 7f8edb59e700 1 -- 172.21.15.17:0/925895546 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8edc0b2be0 con 0x7f8edc0ad750 2022-08-01T19:40:50.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.687+0000 7f8edb59e700 1 -- 172.21.15.17:0/925895546 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8edc0ad360 con 0x7f8edc0ad750 2022-08-01T19:40:50.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.687+0000 7f8ed37fe700 1 -- 172.21.15.17:0/925895546 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8ed4025030 con 0x7f8edc0ad750 2022-08-01T19:40:50.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.687+0000 7f8ed37fe700 1 --2- 172.21.15.17:0/925895546 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8ec0053aa0 0x7f8ec0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:50.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.687+0000 7f8ed37fe700 1 -- 172.21.15.17:0/925895546 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6549+0+0 (secure 0 0 0) 0x7f8ed406cf90 con 0x7f8edc0ad750 2022-08-01T19:40:50.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.691+0000 7f8eda59c700 1 --2- 172.21.15.17:0/925895546 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8ec0053aa0 0x7f8ec0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:50.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.691+0000 7f8eda59c700 1 --2- 172.21.15.17:0/925895546 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8ec0053aa0 0x7f8ec0055f60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f8ecc00d000 tx=0x7f8ecc009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:50.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.695+0000 7f8ed37fe700 1 -- 172.21.15.17:0/925895546 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8ed4043c30 con 0x7f8edc0ad750 2022-08-01T19:40:50.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:50.819+0000 7f8edb59e700 1 -- 172.21.15.17:0/925895546 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8edc0af5f0 con 0x7f8edc0ad750 2022-08-01T19:40:51.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.519+0000 7f8ed37fe700 1 -- 172.21.15.17:0/925895546 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v79) v1 ==== 185+0+0 (secure 0 0 0) 0x7f8ed4023070 con 0x7f8edc0ad750 2022-08-01T19:40:51.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.523+0000 7f8edb59e700 1 -- 172.21.15.17:0/925895546 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8ec0053aa0 msgr2=0x7f8ec0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:51.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.523+0000 7f8edb59e700 1 --2- 172.21.15.17:0/925895546 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8ec0053aa0 0x7f8ec0055f60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f8ecc00d000 tx=0x7f8ecc009040).stop 2022-08-01T19:40:51.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.523+0000 7f8edb59e700 1 -- 172.21.15.17:0/925895546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8edc0ad750 msgr2=0x7f8edc152cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:51.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.523+0000 7f8edb59e700 1 --2- 172.21.15.17:0/925895546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8edc0ad750 0x7f8edc152cc0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7f8ed4002f70 tx=0x7f8ed4014000).stop 2022-08-01T19:40:51.538 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.523+0000 7f8edb59e700 1 -- 172.21.15.17:0/925895546 shutdown_connections 2022-08-01T19:40:51.538 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.523+0000 7f8edb59e700 1 --2- 172.21.15.17:0/925895546 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8ec0053aa0 0x7f8ec0055f60 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:51.539 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.523+0000 7f8edb59e700 1 --2- 172.21.15.17:0/925895546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8edc0ad750 0x7f8edc152cc0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:51.539 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.523+0000 7f8edb59e700 1 --2- 172.21.15.17:0/925895546 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8edc061890 0x7f8edc1527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:51.539 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.523+0000 7f8edb59e700 1 --2- 172.21.15.17:0/925895546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8edc0614b0 0x7f8edc05d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:51.539 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.523+0000 7f8edb59e700 1 -- 172.21.15.17:0/925895546 >> 172.21.15.17:0/925895546 conn(0x7f8edc066f70 msgr2=0x7f8edc142990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:51.539 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.523+0000 7f8edb59e700 1 -- 172.21.15.17:0/925895546 shutdown_connections 2022-08-01T19:40:51.540 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.523+0000 7f8edb59e700 1 -- 172.21.15.17:0/925895546 wait complete. 2022-08-01T19:40:51.540 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' removed 2022-08-01T19:40:51.542 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-08-01T19:40:51.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.651+0000 7fd9df22c700 1 Processor -- start 2022-08-01T19:40:51.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.651+0000 7fd9df22c700 1 -- start start 2022-08-01T19:40:51.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.655+0000 7fd9df22c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd9d80ad750 0x7fd9d8061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:51.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.655+0000 7fd9df22c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd9d81405d0 0x7fd9d8140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:51.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.655+0000 7fd9df22c700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d80aee80 con 0x7fd9d80614b0 2022-08-01T19:40:51.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.655+0000 7fd9df22c700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9d80b70b0 con 0x7fd9d80ad750 2022-08-01T19:40:51.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.655+0000 7fd9df22c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d80af0b0 con 0x7fd9d81405d0 2022-08-01T19:40:51.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.655+0000 7fd9dda29700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd9d80ad750 0x7fd9d8061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:51.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.655+0000 7fd9dda29700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd9d80ad750 0x7fd9d8061890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41020/0 (socket says 172.21.15.17:41020) 2022-08-01T19:40:51.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.655+0000 7fd9dda29700 1 -- 172.21.15.17:0/4027380933 learned_addr learned my addr 172.21.15.17:0/4027380933 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:51.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.655+0000 7fd9de22a700 1 --1- 172.21.15.17:0/4027380933 >> v1:172.21.15.77:6789/0 conn(0x7fd9d80614b0 0x7fd9d80b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:50012/0 (socket says 172.21.15.17:50012) 2022-08-01T19:40:51.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.655+0000 7fd9dea2b700 1 --2- 172.21.15.17:0/4027380933 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd9d81405d0 0x7fd9d8140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:51.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.655+0000 7fd9dda29700 1 -- 172.21.15.17:0/4027380933 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd9d81405d0 msgr2=0x7fd9d8140a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:51.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.655+0000 7fd9dda29700 1 --2- 172.21.15.17:0/4027380933 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd9d81405d0 0x7fd9d8140a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:51.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.655+0000 7fd9dda29700 1 -- 172.21.15.17:0/4027380933 >> v1:172.21.15.77:6789/0 conn(0x7fd9d80614b0 legacy=0x7fd9d80b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:51.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.655+0000 7fd9dda29700 1 -- 172.21.15.17:0/4027380933 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d80af350 con 0x7fd9d80ad750 2022-08-01T19:40:51.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.655+0000 7fd9dd228700 1 -- 172.21.15.17:0/4027380933 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 422291001 0 0) 0x7fd9d80aee80 con 0x7fd9d80614b0 2022-08-01T19:40:51.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.655+0000 7fd9dda29700 1 --2- 172.21.15.17:0/4027380933 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9d80ad750 0x7fd9d8061890 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0x7fd9c8002f70 tx=0x7fd9c8006eb0).ready entity=mon.0 client_cookie=e8861e5cd851dce2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:51.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.655+0000 7fd9dd228700 1 -- 172.21.15.17:0/4027380933 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9c800d570 con 0x7fd9d80ad750 2022-08-01T19:40:51.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.655+0000 7fd9dd228700 1 -- 172.21.15.17:0/4027380933 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c8000e90 con 0x7fd9d80ad750 2022-08-01T19:40:51.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.655+0000 7fd9dd228700 1 -- 172.21.15.17:0/4027380933 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9c800d8c0 con 0x7fd9d80ad750 2022-08-01T19:40:51.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.655+0000 7fd9df22c700 1 -- 172.21.15.17:0/4027380933 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9d80ad750 msgr2=0x7fd9d8061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:51.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.655+0000 7fd9df22c700 1 --2- 172.21.15.17:0/4027380933 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9d80ad750 0x7fd9d8061890 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0x7fd9c8002f70 tx=0x7fd9c8006eb0).stop 2022-08-01T19:40:51.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.655+0000 7fd9df22c700 1 -- 172.21.15.17:0/4027380933 shutdown_connections 2022-08-01T19:40:51.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.655+0000 7fd9df22c700 1 --2- 172.21.15.17:0/4027380933 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd9d81405d0 0x7fd9d8140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:51.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.655+0000 7fd9df22c700 1 --2- 172.21.15.17:0/4027380933 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9d80ad750 0x7fd9d8061890 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:51.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.655+0000 7fd9df22c700 1 -- 172.21.15.17:0/4027380933 >> 172.21.15.17:0/4027380933 conn(0x7fd9d8066f70 msgr2=0x7fd9d8067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:51.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.655+0000 7fd9df22c700 1 -- 172.21.15.17:0/4027380933 shutdown_connections 2022-08-01T19:40:51.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.655+0000 7fd9df22c700 1 -- 172.21.15.17:0/4027380933 wait complete. 2022-08-01T19:40:51.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.659+0000 7fd9df22c700 1 Processor -- start 2022-08-01T19:40:51.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.659+0000 7fd9df22c700 1 -- start start 2022-08-01T19:40:51.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.663+0000 7fd9df22c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd9d80614b0 0x7fd9d805d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:51.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.663+0000 7fd9df22c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9d80ad750 0x7fd9d8152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:51.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.663+0000 7fd9df22c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9d81405d0 0x7fd9d8144460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:51.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.663+0000 7fd9df22c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d80b1070 con 0x7fd9d81405d0 2022-08-01T19:40:51.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.663+0000 7fd9df22c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d80aa520 con 0x7fd9d80ad750 2022-08-01T19:40:51.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.663+0000 7fd9df22c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d81524c0 con 0x7fd9d80614b0 2022-08-01T19:40:51.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.663+0000 7fd9dea2b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9d81405d0 0x7fd9d8144460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:51.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.663+0000 7fd9dea2b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9d81405d0 0x7fd9d8144460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41028/0 (socket says 172.21.15.17:41028) 2022-08-01T19:40:51.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.663+0000 7fd9dea2b700 1 -- 172.21.15.17:0/986075611 learned_addr learned my addr 172.21.15.17:0/986075611 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:51.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.663+0000 7fd9dda29700 1 --2- 172.21.15.17:0/986075611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9d80ad750 0x7fd9d8152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:51.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.663+0000 7fd9de22a700 1 --2- 172.21.15.17:0/986075611 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd9d80614b0 0x7fd9d805d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:51.692 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.663+0000 7fd9dea2b700 1 -- 172.21.15.17:0/986075611 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd9d80614b0 msgr2=0x7fd9d805d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:51.692 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.663+0000 7fd9dea2b700 1 --2- 172.21.15.17:0/986075611 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd9d80614b0 0x7fd9d805d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:51.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.663+0000 7fd9dea2b700 1 -- 172.21.15.17:0/986075611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9d80ad750 msgr2=0x7fd9d8152ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:51.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.663+0000 7fd9dea2b700 1 --2- 172.21.15.17:0/986075611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9d80ad750 0x7fd9d8152ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:51.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.663+0000 7fd9dea2b700 1 -- 172.21.15.17:0/986075611 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c800b1e0 con 0x7fd9d81405d0 2022-08-01T19:40:51.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.663+0000 7fd9dea2b700 1 --2- 172.21.15.17:0/986075611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9d81405d0 0x7fd9d8144460 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 rx=0x7fd9d80b11b0 tx=0x7fd9d00091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:51.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.663+0000 7fd9c77fe700 1 -- 172.21.15.17:0/986075611 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9d0012600 con 0x7fd9d81405d0 2022-08-01T19:40:51.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.663+0000 7fd9c77fe700 1 -- 172.21.15.17:0/986075611 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d0012760 con 0x7fd9d81405d0 2022-08-01T19:40:51.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.663+0000 7fd9df22c700 1 -- 172.21.15.17:0/986075611 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d8152760 con 0x7fd9d81405d0 2022-08-01T19:40:51.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.663+0000 7fd9c77fe700 1 -- 172.21.15.17:0/986075611 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9d0012ac0 con 0x7fd9d81405d0 2022-08-01T19:40:51.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.663+0000 7fd9df22c700 1 -- 172.21.15.17:0/986075611 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d80aa1a0 con 0x7fd9d81405d0 2022-08-01T19:40:51.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.663+0000 7fd9df22c700 1 -- 172.21.15.17:0/986075611 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d80ab160 con 0x7fd9d81405d0 2022-08-01T19:40:51.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.663+0000 7fd9c77fe700 1 -- 172.21.15.17:0/986075611 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd9d0022070 con 0x7fd9d81405d0 2022-08-01T19:40:51.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.667+0000 7fd9c77fe700 1 --2- 172.21.15.17:0/986075611 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd9b80539e0 0x7fd9b8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:51.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.667+0000 7fd9c77fe700 1 -- 172.21.15.17:0/986075611 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6111+0+0 (secure 0 0 0) 0x7fd9d006c470 con 0x7fd9d81405d0 2022-08-01T19:40:51.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.667+0000 7fd9de22a700 1 --2- 172.21.15.17:0/986075611 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd9b80539e0 0x7fd9b8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:51.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.667+0000 7fd9de22a700 1 --2- 172.21.15.17:0/986075611 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd9b80539e0 0x7fd9b8055ea0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7fd9d400d000 tx=0x7fd9d4007a00).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:51.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.671+0000 7fd9c77fe700 1 -- 172.21.15.17:0/986075611 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd9d0043310 con 0x7fd9d81405d0 2022-08-01T19:40:51.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:51.799+0000 7fd9df22c700 1 -- 172.21.15.17:0/986075611 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd9d80af5f0 con 0x7fd9d81405d0 2022-08-01T19:40:52.547 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.543+0000 7fd9c77fe700 1 -- 172.21.15.17:0/986075611 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v80) v1 ==== 173+0+0 (secure 0 0 0) 0x7fd9d0016020 con 0x7fd9d81405d0 2022-08-01T19:40:52.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.543+0000 7fd9df22c700 1 -- 172.21.15.17:0/986075611 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd9b80539e0 msgr2=0x7fd9b8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:52.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.543+0000 7fd9df22c700 1 --2- 172.21.15.17:0/986075611 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd9b80539e0 0x7fd9b8055ea0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7fd9d400d000 tx=0x7fd9d4007a00).stop 2022-08-01T19:40:52.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.543+0000 7fd9df22c700 1 -- 172.21.15.17:0/986075611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9d81405d0 msgr2=0x7fd9d8144460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:52.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.543+0000 7fd9df22c700 1 --2- 172.21.15.17:0/986075611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9d81405d0 0x7fd9d8144460 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 rx=0x7fd9d80b11b0 tx=0x7fd9d00091d0).stop 2022-08-01T19:40:52.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.547+0000 7fd9df22c700 1 -- 172.21.15.17:0/986075611 shutdown_connections 2022-08-01T19:40:52.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.547+0000 7fd9df22c700 1 --2- 172.21.15.17:0/986075611 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd9b80539e0 0x7fd9b8055ea0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:52.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.547+0000 7fd9df22c700 1 --2- 172.21.15.17:0/986075611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9d81405d0 0x7fd9d8144460 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:52.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.547+0000 7fd9df22c700 1 --2- 172.21.15.17:0/986075611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9d80ad750 0x7fd9d8152ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:52.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.547+0000 7fd9df22c700 1 --2- 172.21.15.17:0/986075611 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd9d80614b0 0x7fd9d805d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:52.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.547+0000 7fd9df22c700 1 -- 172.21.15.17:0/986075611 >> 172.21.15.17:0/986075611 conn(0x7fd9d8066f70 msgr2=0x7fd9d8067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:52.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.547+0000 7fd9df22c700 1 -- 172.21.15.17:0/986075611 shutdown_connections 2022-08-01T19:40:52.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.547+0000 7fd9df22c700 1 -- 172.21.15.17:0/986075611 wait complete. 2022-08-01T19:40:52.557 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' removed 2022-08-01T19:40:52.568 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-08-01T19:40:52.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.671+0000 7f2d86d78700 1 Processor -- start 2022-08-01T19:40:52.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.671+0000 7f2d86d78700 1 -- start start 2022-08-01T19:40:52.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.675+0000 7f2d86d78700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2d800a9da0 0x7f2d80060950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:52.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.675+0000 7f2d86d78700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2d801402d0 0x7f2d80140760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:52.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.675+0000 7f2d86d78700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d80067030 con 0x7f2d800b31d0 2022-08-01T19:40:52.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.675+0000 7f2d86d78700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d80064a70 con 0x7f2d800a9da0 2022-08-01T19:40:52.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.675+0000 7f2d86d78700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d80061f60 con 0x7f2d801402d0 2022-08-01T19:40:52.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.675+0000 7f2d85575700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2d800a9da0 0x7f2d80060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:52.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.675+0000 7f2d85d76700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2d800b31d0 0x7f2d80066020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33226/0 (socket says 172.21.15.17:33226) 2022-08-01T19:40:52.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.675+0000 7f2d85d76700 1 -- 172.21.15.17:0/3840792158 learned_addr learned my addr 172.21.15.17:0/3840792158 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:40:52.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.675+0000 7f2d86577700 1 --2- 172.21.15.17:0/3840792158 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2d801402d0 0x7f2d80140760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:52.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.675+0000 7f2d84d74700 1 -- 172.21.15.17:0/3840792158 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2604499247 0 0) 0x7f2d80067030 con 0x7f2d800b31d0 2022-08-01T19:40:52.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.675+0000 7f2d84d74700 1 -- 172.21.15.17:0/3840792158 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d6c003260 con 0x7f2d800b31d0 2022-08-01T19:40:52.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.675+0000 7f2d84d74700 1 -- 172.21.15.17:0/3840792158 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3372836916 0 0) 0x7f2d6c003260 con 0x7f2d800b31d0 2022-08-01T19:40:52.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.675+0000 7f2d85575700 1 -- 172.21.15.17:0/3840792158 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2d801402d0 msgr2=0x7f2d80140760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:52.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.675+0000 7f2d85575700 1 --2- 172.21.15.17:0/3840792158 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2d801402d0 0x7f2d80140760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:52.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.675+0000 7f2d85575700 1 -- 172.21.15.17:0/3840792158 >> v1:172.21.15.17:6789/0 conn(0x7f2d800b31d0 legacy=0x7f2d80066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:52.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.675+0000 7f2d85575700 1 -- 172.21.15.17:0/3840792158 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d800b35b0 con 0x7f2d800a9da0 2022-08-01T19:40:52.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.675+0000 7f2d85575700 1 --2- 172.21.15.17:0/3840792158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d800a9da0 0x7f2d80060950 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7f2d78001b80 tx=0x7f2d74006df0).ready entity=mon.0 client_cookie=d9a84fa10b69d74a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:52.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.675+0000 7f2d84d74700 1 -- 172.21.15.17:0/3840792158 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d780019e0 con 0x7f2d800a9da0 2022-08-01T19:40:52.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.675+0000 7f2d84d74700 1 -- 172.21.15.17:0/3840792158 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d74000e90 con 0x7f2d800a9da0 2022-08-01T19:40:52.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.675+0000 7f2d84d74700 1 -- 172.21.15.17:0/3840792158 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d74012460 con 0x7f2d800a9da0 2022-08-01T19:40:52.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.675+0000 7f2d86d78700 1 -- 172.21.15.17:0/3840792158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d800a9da0 msgr2=0x7f2d80060950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:52.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.675+0000 7f2d86d78700 1 --2- 172.21.15.17:0/3840792158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d800a9da0 0x7f2d80060950 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7f2d78001b80 tx=0x7f2d74006df0).stop 2022-08-01T19:40:52.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.675+0000 7f2d86d78700 1 -- 172.21.15.17:0/3840792158 shutdown_connections 2022-08-01T19:40:52.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.675+0000 7f2d86d78700 1 --2- 172.21.15.17:0/3840792158 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2d801402d0 0x7f2d80140760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:52.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.675+0000 7f2d86d78700 1 --2- 172.21.15.17:0/3840792158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d800a9da0 0x7f2d80060950 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:52.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.675+0000 7f2d86d78700 1 -- 172.21.15.17:0/3840792158 >> 172.21.15.17:0/3840792158 conn(0x7f2d80063870 msgr2=0x7f2d80063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:52.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.675+0000 7f2d86d78700 1 -- 172.21.15.17:0/3840792158 shutdown_connections 2022-08-01T19:40:52.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.675+0000 7f2d86d78700 1 -- 172.21.15.17:0/3840792158 wait complete. 2022-08-01T19:40:52.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.679+0000 7f2d86d78700 1 Processor -- start 2022-08-01T19:40:52.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.679+0000 7f2d86d78700 1 -- start start 2022-08-01T19:40:52.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.679+0000 7f2d86d78700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2d800a9da0 0x7f2d80145ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:52.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.679+0000 7f2d86d78700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d800b31d0 0x7f2d800ae150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:52.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.679+0000 7f2d86d78700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d801402d0 0x7f2d800ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:52.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.679+0000 7f2d86d78700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d800acec0 con 0x7f2d800b31d0 2022-08-01T19:40:52.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.679+0000 7f2d86d78700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d80152540 con 0x7f2d801402d0 2022-08-01T19:40:52.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.679+0000 7f2d86d78700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d800ad830 con 0x7f2d800a9da0 2022-08-01T19:40:52.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.679+0000 7f2d85575700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d800b31d0 0x7f2d800ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:52.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.679+0000 7f2d85575700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d800b31d0 0x7f2d800ae150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41040/0 (socket says 172.21.15.17:41040) 2022-08-01T19:40:52.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.679+0000 7f2d85575700 1 -- 172.21.15.17:0/2066916722 learned_addr learned my addr 172.21.15.17:0/2066916722 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:52.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.683+0000 7f2d86577700 1 --2- 172.21.15.17:0/2066916722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d801402d0 0x7f2d800ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:52.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.683+0000 7f2d85d76700 1 --2- 172.21.15.17:0/2066916722 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2d800a9da0 0x7f2d80145ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:52.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.683+0000 7f2d85575700 1 -- 172.21.15.17:0/2066916722 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2d800a9da0 msgr2=0x7f2d80145ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:52.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.683+0000 7f2d85575700 1 --2- 172.21.15.17:0/2066916722 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2d800a9da0 0x7f2d80145ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:52.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.683+0000 7f2d85575700 1 -- 172.21.15.17:0/2066916722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d801402d0 msgr2=0x7f2d800ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:52.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.683+0000 7f2d85575700 1 --2- 172.21.15.17:0/2066916722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d801402d0 0x7f2d800ae640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:52.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.683+0000 7f2d85575700 1 -- 172.21.15.17:0/2066916722 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d7400b180 con 0x7f2d800b31d0 2022-08-01T19:40:52.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.683+0000 7f2d85575700 1 --2- 172.21.15.17:0/2066916722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d800b31d0 0x7f2d800ae150 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f2d78001b80 tx=0x7f2d74014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:52.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.683+0000 7f2d7effd700 1 -- 172.21.15.17:0/2066916722 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d7400ddb0 con 0x7f2d800b31d0 2022-08-01T19:40:52.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.683+0000 7f2d7effd700 1 -- 172.21.15.17:0/2066916722 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d74002f50 con 0x7f2d800b31d0 2022-08-01T19:40:52.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.683+0000 7f2d86d78700 1 -- 172.21.15.17:0/2066916722 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d800ad0f0 con 0x7f2d800b31d0 2022-08-01T19:40:52.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.683+0000 7f2d7effd700 1 -- 172.21.15.17:0/2066916722 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d7401e5f0 con 0x7f2d800b31d0 2022-08-01T19:40:52.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.683+0000 7f2d86d78700 1 -- 172.21.15.17:0/2066916722 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d800abce0 con 0x7f2d800b31d0 2022-08-01T19:40:52.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.683+0000 7f2d86d78700 1 -- 172.21.15.17:0/2066916722 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d80066af0 con 0x7f2d800b31d0 2022-08-01T19:40:52.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.683+0000 7f2d7effd700 1 -- 172.21.15.17:0/2066916722 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2d74025030 con 0x7f2d800b31d0 2022-08-01T19:40:52.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.687+0000 7f2d7effd700 1 --2- 172.21.15.17:0/2066916722 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2d60053a10 0x7f2d60055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:52.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.687+0000 7f2d7effd700 1 -- 172.21.15.17:0/2066916722 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f2d7406c9e0 con 0x7f2d800b31d0 2022-08-01T19:40:52.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.687+0000 7f2d85d76700 1 --2- 172.21.15.17:0/2066916722 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2d60053a10 0x7f2d60055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:52.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.687+0000 7f2d85d76700 1 --2- 172.21.15.17:0/2066916722 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2d60053a10 0x7f2d60055ed0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f2d7800d000 tx=0x7f2d780021c0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:52.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.691+0000 7f2d7effd700 1 -- 172.21.15.17:0/2066916722 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2d74043a50 con 0x7f2d800b31d0 2022-08-01T19:40:52.816 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.811+0000 7f2d86d78700 1 -- 172.21.15.17:0/2066916722 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f2d800574e0 con 0x7f2d800b31d0 2022-08-01T19:40:52.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.819+0000 7f2d7effd700 1 -- 172.21.15.17:0/2066916722 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2d74023020 con 0x7f2d800b31d0 2022-08-01T19:40:52.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.823+0000 7f2d86d78700 1 -- 172.21.15.17:0/2066916722 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2d60053a10 msgr2=0x7f2d60055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:52.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.823+0000 7f2d86d78700 1 --2- 172.21.15.17:0/2066916722 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2d60053a10 0x7f2d60055ed0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f2d7800d000 tx=0x7f2d780021c0).stop 2022-08-01T19:40:52.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.823+0000 7f2d86d78700 1 -- 172.21.15.17:0/2066916722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d800b31d0 msgr2=0x7f2d800ae150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:52.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.823+0000 7f2d86d78700 1 --2- 172.21.15.17:0/2066916722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d800b31d0 0x7f2d800ae150 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f2d78001b80 tx=0x7f2d74014000).stop 2022-08-01T19:40:52.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.823+0000 7f2d86d78700 1 -- 172.21.15.17:0/2066916722 shutdown_connections 2022-08-01T19:40:52.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.823+0000 7f2d86d78700 1 --2- 172.21.15.17:0/2066916722 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2d60053a10 0x7f2d60055ed0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:52.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.823+0000 7f2d86d78700 1 --2- 172.21.15.17:0/2066916722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d801402d0 0x7f2d800ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:52.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.823+0000 7f2d86d78700 1 --2- 172.21.15.17:0/2066916722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d800b31d0 0x7f2d800ae150 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:52.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.823+0000 7f2d86d78700 1 --2- 172.21.15.17:0/2066916722 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2d800a9da0 0x7f2d80145ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:52.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.823+0000 7f2d86d78700 1 -- 172.21.15.17:0/2066916722 >> 172.21.15.17:0/2066916722 conn(0x7f2d80063870 msgr2=0x7f2d800b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:52.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.823+0000 7f2d86d78700 1 -- 172.21.15.17:0/2066916722 shutdown_connections 2022-08-01T19:40:52.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.823+0000 7f2d86d78700 1 -- 172.21.15.17:0/2066916722 wait complete. 2022-08-01T19:40:52.843 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder' 2022-08-01T19:40:52.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.943+0000 7ff35359e700 1 Processor -- start 2022-08-01T19:40:52.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.943+0000 7ff35359e700 1 -- start start 2022-08-01T19:40:52.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.943+0000 7ff35359e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff3540b2710 0x7ff35414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:52.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.943+0000 7ff35359e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ff3540b01a0 0x7ff35414b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:52.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.943+0000 7ff35359e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3540a7250 con 0x7ff3540afd60 2022-08-01T19:40:52.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.943+0000 7ff35359e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3540a7480 con 0x7ff3540b01a0 2022-08-01T19:40:52.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.943+0000 7ff35359e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3540a7720 con 0x7ff3540b2710 2022-08-01T19:40:52.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.947+0000 7ff351d9b700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ff3540b01a0 0x7ff35414b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:52.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.947+0000 7ff352d9d700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7ff3540afd60 0x7ff3540a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:50044/0 (socket says 172.21.15.17:50044) 2022-08-01T19:40:52.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.947+0000 7ff352d9d700 1 -- 172.21.15.17:0/1807464109 learned_addr learned my addr 172.21.15.17:0/1807464109 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:40:52.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.947+0000 7ff35259c700 1 --2- 172.21.15.17:0/1807464109 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff3540b2710 0x7ff35414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:52.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.947+0000 7ff351d9b700 1 -- 172.21.15.17:0/1807464109 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff3540b2710 msgr2=0x7ff35414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:52.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.947+0000 7ff351d9b700 1 --2- 172.21.15.17:0/1807464109 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff3540b2710 0x7ff35414afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:52.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.947+0000 7ff351d9b700 1 -- 172.21.15.17:0/1807464109 >> v1:172.21.15.77:6789/0 conn(0x7ff3540afd60 legacy=0x7ff3540a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:52.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.947+0000 7ff351d9b700 1 -- 172.21.15.17:0/1807464109 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3540a79c0 con 0x7ff3540b01a0 2022-08-01T19:40:52.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.947+0000 7ff351d9b700 1 --2- 172.21.15.17:0/1807464109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3540b01a0 0x7ff35414b4c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7ff3440032f0 tx=0x7ff3440091d0).ready entity=mon.1 client_cookie=fc7112f455666d41 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:52.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.947+0000 7ff35159a700 1 -- 172.21.15.17:0/1807464109 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff34400b6c0 con 0x7ff3540b01a0 2022-08-01T19:40:52.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.947+0000 7ff35159a700 1 -- 172.21.15.17:0/1807464109 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff34400b820 con 0x7ff3540b01a0 2022-08-01T19:40:52.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.947+0000 7ff35159a700 1 -- 172.21.15.17:0/1807464109 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff34400b990 con 0x7ff3540b01a0 2022-08-01T19:40:52.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.947+0000 7ff35359e700 1 -- 172.21.15.17:0/1807464109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3540b01a0 msgr2=0x7ff35414b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:52.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.947+0000 7ff35359e700 1 --2- 172.21.15.17:0/1807464109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3540b01a0 0x7ff35414b4c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7ff3440032f0 tx=0x7ff3440091d0).stop 2022-08-01T19:40:52.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.947+0000 7ff35359e700 1 -- 172.21.15.17:0/1807464109 shutdown_connections 2022-08-01T19:40:52.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.947+0000 7ff35359e700 1 --2- 172.21.15.17:0/1807464109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3540b01a0 0x7ff35414b4c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:52.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.947+0000 7ff35359e700 1 --2- 172.21.15.17:0/1807464109 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff3540b2710 0x7ff35414afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:52.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.947+0000 7ff35359e700 1 -- 172.21.15.17:0/1807464109 >> 172.21.15.17:0/1807464109 conn(0x7ff354060310 msgr2=0x7ff3540b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:52.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.947+0000 7ff35359e700 1 -- 172.21.15.17:0/1807464109 shutdown_connections 2022-08-01T19:40:52.966 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.947+0000 7ff35359e700 1 -- 172.21.15.17:0/1807464109 wait complete. 2022-08-01T19:40:52.966 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.951+0000 7ff35359e700 1 Processor -- start 2022-08-01T19:40:52.967 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.951+0000 7ff35359e700 1 -- start start 2022-08-01T19:40:52.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.951+0000 7ff35359e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3540afd60 0x7ff3540b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:52.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.951+0000 7ff35359e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff3540b01a0 0x7ff3540b5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:52.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.951+0000 7ff35359e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3540b2710 0x7ff3540ab2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:52.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.951+0000 7ff35359e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3540b3920 con 0x7ff3540b2710 2022-08-01T19:40:52.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.951+0000 7ff35359e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3540ac0a0 con 0x7ff3540afd60 2022-08-01T19:40:52.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.951+0000 7ff35359e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3540ac340 con 0x7ff3540b01a0 2022-08-01T19:40:52.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.951+0000 7ff352d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3540b2710 0x7ff3540ab2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:52.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.951+0000 7ff352d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3540b2710 0x7ff3540ab2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41056/0 (socket says 172.21.15.17:41056) 2022-08-01T19:40:52.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.951+0000 7ff352d9d700 1 -- 172.21.15.17:0/3990946207 learned_addr learned my addr 172.21.15.17:0/3990946207 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:52.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.951+0000 7ff35259c700 1 --2- 172.21.15.17:0/3990946207 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3540afd60 0x7ff3540b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:52.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.951+0000 7ff351d9b700 1 --2- 172.21.15.17:0/3990946207 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff3540b01a0 0x7ff3540b5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:52.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.951+0000 7ff352d9d700 1 -- 172.21.15.17:0/3990946207 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff3540b01a0 msgr2=0x7ff3540b5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:52.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.951+0000 7ff352d9d700 1 --2- 172.21.15.17:0/3990946207 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff3540b01a0 0x7ff3540b5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:52.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.951+0000 7ff352d9d700 1 -- 172.21.15.17:0/3990946207 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3540afd60 msgr2=0x7ff3540b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:52.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.951+0000 7ff352d9d700 1 --2- 172.21.15.17:0/3990946207 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3540afd60 0x7ff3540b32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:52.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.951+0000 7ff352d9d700 1 -- 172.21.15.17:0/3990946207 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff344009470 con 0x7ff3540b2710 2022-08-01T19:40:52.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.951+0000 7ff351d9b700 1 --2- 172.21.15.17:0/3990946207 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff3540b01a0 0x7ff3540b5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:40:52.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.951+0000 7ff352d9d700 1 --2- 172.21.15.17:0/3990946207 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3540b2710 0x7ff3540ab2c0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 rx=0x7ff354061c10 tx=0x7ff34c007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:52.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.951+0000 7ff34b7fe700 1 -- 172.21.15.17:0/3990946207 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff34c0070d0 con 0x7ff3540b2710 2022-08-01T19:40:52.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.951+0000 7ff34b7fe700 1 -- 172.21.15.17:0/3990946207 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff34c007230 con 0x7ff3540b2710 2022-08-01T19:40:52.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.951+0000 7ff35359e700 1 -- 172.21.15.17:0/3990946207 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3540ac5e0 con 0x7ff3540b2710 2022-08-01T19:40:52.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.951+0000 7ff34b7fe700 1 -- 172.21.15.17:0/3990946207 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff34c005610 con 0x7ff3540b2710 2022-08-01T19:40:52.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.951+0000 7ff35359e700 1 -- 172.21.15.17:0/3990946207 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3540b3c00 con 0x7ff3540b2710 2022-08-01T19:40:52.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.955+0000 7ff35359e700 1 -- 172.21.15.17:0/3990946207 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3540ac770 con 0x7ff3540b2710 2022-08-01T19:40:52.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.955+0000 7ff34b7fe700 1 -- 172.21.15.17:0/3990946207 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff34c01a070 con 0x7ff3540b2710 2022-08-01T19:40:52.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.955+0000 7ff34b7fe700 1 --2- 172.21.15.17:0/3990946207 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff338053aa0 0x7ff338055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:52.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.955+0000 7ff34b7fe700 1 -- 172.21.15.17:0/3990946207 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5729+0+0 (secure 0 0 0) 0x7ff34c06ad10 con 0x7ff3540b2710 2022-08-01T19:40:52.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.955+0000 7ff35259c700 1 --2- 172.21.15.17:0/3990946207 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff338053aa0 0x7ff338055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:52.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.955+0000 7ff35259c700 1 --2- 172.21.15.17:0/3990946207 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff338053aa0 0x7ff338055f60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7ff33c00b000 tx=0x7ff33c006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:52.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:52.959+0000 7ff34b7fe700 1 -- 172.21.15.17:0/3990946207 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff34c041d80 con 0x7ff3540b2710 2022-08-01T19:40:53.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.123+0000 7ff35359e700 1 -- 172.21.15.17:0/3990946207 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]} v 0) v1 -- 0x7ff3540574e0 con 0x7ff3540b2710 2022-08-01T19:40:53.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.543+0000 7ff34b7fe700 1 -- 172.21.15.17:0/3990946207 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]}]=0 v292) v1 ==== 139+0+0 (secure 0 0 0) 0x7ff34c018030 con 0x7ff3540b2710 2022-08-01T19:40:53.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.547+0000 7ff35359e700 1 -- 172.21.15.17:0/3990946207 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff338053aa0 msgr2=0x7ff338055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:53.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.547+0000 7ff35359e700 1 --2- 172.21.15.17:0/3990946207 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff338053aa0 0x7ff338055f60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7ff33c00b000 tx=0x7ff33c006df0).stop 2022-08-01T19:40:53.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.547+0000 7ff35359e700 1 -- 172.21.15.17:0/3990946207 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3540b2710 msgr2=0x7ff3540ab2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:53.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.547+0000 7ff35359e700 1 --2- 172.21.15.17:0/3990946207 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3540b2710 0x7ff3540ab2c0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 rx=0x7ff354061c10 tx=0x7ff34c007bc0).stop 2022-08-01T19:40:53.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.547+0000 7ff35359e700 1 -- 172.21.15.17:0/3990946207 shutdown_connections 2022-08-01T19:40:53.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.547+0000 7ff35359e700 1 --2- 172.21.15.17:0/3990946207 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff338053aa0 0x7ff338055f60 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:53.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.547+0000 7ff35359e700 1 --2- 172.21.15.17:0/3990946207 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3540b2710 0x7ff3540ab2c0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:53.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.547+0000 7ff35359e700 1 --2- 172.21.15.17:0/3990946207 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff3540b01a0 0x7ff3540b5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:53.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.547+0000 7ff35359e700 1 --2- 172.21.15.17:0/3990946207 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3540afd60 0x7ff3540b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:53.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.547+0000 7ff35359e700 1 -- 172.21.15.17:0/3990946207 >> 172.21.15.17:0/3990946207 conn(0x7ff354060310 msgr2=0x7ff354149f70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:53.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.547+0000 7ff35359e700 1 -- 172.21.15.17:0/3990946207 shutdown_connections 2022-08-01T19:40:53.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.547+0000 7ff35359e700 1 -- 172.21.15.17:0/3990946207 wait complete. 2022-08-01T19:40:53.571 INFO:tasks.cephfs_test_runner:test_cap_revoke_nonresponder (tasks.cephfs.test_misc.TestMisc) ... ok 2022-08-01T19:40:53.572 INFO:tasks.cephfs_test_runner:Starting test: test_dump_inode (tasks.cephfs.test_misc.TestMisc) 2022-08-01T19:40:53.572 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode' 2022-08-01T19:40:53.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.679+0000 7fd9c82a9700 1 Processor -- start 2022-08-01T19:40:53.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.679+0000 7fd9c82a9700 1 -- start start 2022-08-01T19:40:53.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.679+0000 7fd9c82a9700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fd9c00ab9e0 0x7fd9c00b1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:53.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.679+0000 7fd9c82a9700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd9c013eae0 0x7fd9c0141580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:53.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.679+0000 7fd9c82a9700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9c0144ee0 con 0x7fd9c0144760 2022-08-01T19:40:53.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.679+0000 7fd9c82a9700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9c01456c0 con 0x7fd9c013eae0 2022-08-01T19:40:53.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.679+0000 7fd9c82a9700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9c0141190 con 0x7fd9c00ab9e0 2022-08-01T19:40:53.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.679+0000 7fd9c6846700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd9c013eae0 0x7fd9c0141580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:53.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.679+0000 7fd9c6045700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fd9c00ab9e0 0x7fd9c00b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:53.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.679+0000 7fd9c5844700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7fd9c0144760 0x7fd9c013e3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:50054/0 (socket says 172.21.15.17:50054) 2022-08-01T19:40:53.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.679+0000 7fd9c5844700 1 -- 172.21.15.17:0/692779249 learned_addr learned my addr 172.21.15.17:0/692779249 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:40:53.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.679+0000 7fd9c5043700 1 -- 172.21.15.17:0/692779249 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4271094279 0 0) 0x7fd9c0144ee0 con 0x7fd9c0144760 2022-08-01T19:40:53.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.679+0000 7fd9c5043700 1 -- 172.21.15.17:0/692779249 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9a8003260 con 0x7fd9c0144760 2022-08-01T19:40:53.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.679+0000 7fd9c6846700 1 -- 172.21.15.17:0/692779249 >> v2:172.21.15.77:3300/0 conn(0x7fd9c00ab9e0 msgr2=0x7fd9c00b1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:53.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.679+0000 7fd9c6846700 1 --2- 172.21.15.17:0/692779249 >> v2:172.21.15.77:3300/0 conn(0x7fd9c00ab9e0 0x7fd9c00b1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:53.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.679+0000 7fd9c6846700 1 -- 172.21.15.17:0/692779249 >> v1:172.21.15.77:6789/0 conn(0x7fd9c0144760 legacy=0x7fd9c013e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:53.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.679+0000 7fd9c6846700 1 -- 172.21.15.17:0/692779249 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c0145180 con 0x7fd9c013eae0 2022-08-01T19:40:53.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.679+0000 7fd9c6846700 1 --2- 172.21.15.17:0/692779249 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9c013eae0 0x7fd9c0141580 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0x7fd9b8002f70 tx=0x7fd9b8006df0).ready entity=mon.0 client_cookie=10ce599a7380490d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:53.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.683+0000 7fd9c5043700 1 -- 172.21.15.17:0/692779249 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9b800d4f0 con 0x7fd9c013eae0 2022-08-01T19:40:53.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.683+0000 7fd9c5043700 1 -- 172.21.15.17:0/692779249 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b8000e90 con 0x7fd9c013eae0 2022-08-01T19:40:53.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.683+0000 7fd9c5043700 1 -- 172.21.15.17:0/692779249 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9b800d840 con 0x7fd9c013eae0 2022-08-01T19:40:53.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.683+0000 7fd9c82a9700 1 -- 172.21.15.17:0/692779249 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9c013eae0 msgr2=0x7fd9c0141580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:53.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.683+0000 7fd9c82a9700 1 --2- 172.21.15.17:0/692779249 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9c013eae0 0x7fd9c0141580 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0x7fd9b8002f70 tx=0x7fd9b8006df0).stop 2022-08-01T19:40:53.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.683+0000 7fd9c82a9700 1 -- 172.21.15.17:0/692779249 shutdown_connections 2022-08-01T19:40:53.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.683+0000 7fd9c82a9700 1 --2- 172.21.15.17:0/692779249 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9c013eae0 0x7fd9c0141580 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:53.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.683+0000 7fd9c82a9700 1 --2- 172.21.15.17:0/692779249 >> v2:172.21.15.77:3300/0 conn(0x7fd9c00ab9e0 0x7fd9c00b1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:53.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.683+0000 7fd9c82a9700 1 -- 172.21.15.17:0/692779249 >> 172.21.15.17:0/692779249 conn(0x7fd9c00b5570 msgr2=0x7fd9c00b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:53.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.683+0000 7fd9c82a9700 1 -- 172.21.15.17:0/692779249 shutdown_connections 2022-08-01T19:40:53.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.683+0000 7fd9c82a9700 1 -- 172.21.15.17:0/692779249 wait complete. 2022-08-01T19:40:53.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.683+0000 7fd9c82a9700 1 Processor -- start 2022-08-01T19:40:53.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.683+0000 7fd9c82a9700 1 -- start start 2022-08-01T19:40:53.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.683+0000 7fd9c82a9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9c00ab9e0 0x7fd9c0060be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:53.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.683+0000 7fd9c82a9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9c013eae0 0x7fd9c0153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:53.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.683+0000 7fd9c82a9700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd9c0144760 0x7fd9c0154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:53.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.683+0000 7fd9c82a9700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c00b52d0 con 0x7fd9c00ab9e0 2022-08-01T19:40:53.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.683+0000 7fd9c82a9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c0142910 con 0x7fd9c013eae0 2022-08-01T19:40:53.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.683+0000 7fd9c82a9700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c0142bb0 con 0x7fd9c0144760 2022-08-01T19:40:53.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.683+0000 7fd9c6045700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9c00ab9e0 0x7fd9c0060be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:53.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.683+0000 7fd9c6045700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9c00ab9e0 0x7fd9c0060be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41064/0 (socket says 172.21.15.17:41064) 2022-08-01T19:40:53.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.683+0000 7fd9c6045700 1 -- 172.21.15.17:0/2446987843 learned_addr learned my addr 172.21.15.17:0/2446987843 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:53.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.683+0000 7fd9c5844700 1 --2- 172.21.15.17:0/2446987843 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9c013eae0 0x7fd9c0153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:53.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.683+0000 7fd9c6846700 1 --2- 172.21.15.17:0/2446987843 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd9c0144760 0x7fd9c0154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:53.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.683+0000 7fd9c6045700 1 -- 172.21.15.17:0/2446987843 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd9c0144760 msgr2=0x7fd9c0154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:53.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.683+0000 7fd9c6045700 1 --2- 172.21.15.17:0/2446987843 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd9c0144760 0x7fd9c0154430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:53.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.683+0000 7fd9c6045700 1 -- 172.21.15.17:0/2446987843 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9c013eae0 msgr2=0x7fd9c0153f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:53.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.683+0000 7fd9c6045700 1 --2- 172.21.15.17:0/2446987843 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9c013eae0 0x7fd9c0153f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:53.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.683+0000 7fd9c6045700 1 -- 172.21.15.17:0/2446987843 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b800b180 con 0x7fd9c00ab9e0 2022-08-01T19:40:53.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.687+0000 7fd9c6045700 1 --2- 172.21.15.17:0/2446987843 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9c00ab9e0 0x7fd9c0060be0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0x7fd9b000c000 tx=0x7fd9b0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:53.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.687+0000 7fd9af7fe700 1 -- 172.21.15.17:0/2446987843 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9b0010780 con 0x7fd9c00ab9e0 2022-08-01T19:40:53.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.687+0000 7fd9af7fe700 1 -- 172.21.15.17:0/2446987843 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b00108e0 con 0x7fd9c00ab9e0 2022-08-01T19:40:53.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.687+0000 7fd9c82a9700 1 -- 172.21.15.17:0/2446987843 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c0153aa0 con 0x7fd9c00ab9e0 2022-08-01T19:40:53.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.687+0000 7fd9af7fe700 1 -- 172.21.15.17:0/2446987843 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9b0010c40 con 0x7fd9c00ab9e0 2022-08-01T19:40:53.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.687+0000 7fd9c82a9700 1 -- 172.21.15.17:0/2446987843 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c0146ea0 con 0x7fd9c00ab9e0 2022-08-01T19:40:53.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.687+0000 7fd9c82a9700 1 -- 172.21.15.17:0/2446987843 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9c00ae310 con 0x7fd9c00ab9e0 2022-08-01T19:40:53.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.687+0000 7fd9af7fe700 1 -- 172.21.15.17:0/2446987843 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd9b001c460 con 0x7fd9c00ab9e0 2022-08-01T19:40:53.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.687+0000 7fd9af7fe700 1 --2- 172.21.15.17:0/2446987843 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd9a0053980 0x7fd9a0055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:53.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.687+0000 7fd9af7fe700 1 -- 172.21.15.17:0/2446987843 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5729+0+0 (secure 0 0 0) 0x7fd9b006c290 con 0x7fd9c00ab9e0 2022-08-01T19:40:53.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.687+0000 7fd9c5844700 1 --2- 172.21.15.17:0/2446987843 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd9a0053980 0x7fd9a0055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:53.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.687+0000 7fd9c5844700 1 --2- 172.21.15.17:0/2446987843 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd9a0053980 0x7fd9a0055e40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7fd9bc00d000 tx=0x7fd9bc009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:53.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.691+0000 7fd9af7fe700 1 -- 172.21.15.17:0/2446987843 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd9b0043200 con 0x7fd9c00ab9e0 2022-08-01T19:40:53.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:53.855+0000 7fd9c82a9700 1 -- 172.21.15.17:0/2446987843 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]} v 0) v1 -- 0x7fd9c0153c30 con 0x7fd9c00ab9e0 2022-08-01T19:40:54.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.547+0000 7fd9af7fe700 1 -- 172.21.15.17:0/2446987843 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]}]=0 v293) v1 ==== 129+0+0 (secure 0 0 0) 0x7fd9b0046820 con 0x7fd9c00ab9e0 2022-08-01T19:40:54.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.551+0000 7fd9c82a9700 1 -- 172.21.15.17:0/2446987843 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd9a0053980 msgr2=0x7fd9a0055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:54.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.551+0000 7fd9c82a9700 1 --2- 172.21.15.17:0/2446987843 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd9a0053980 0x7fd9a0055e40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7fd9bc00d000 tx=0x7fd9bc009040).stop 2022-08-01T19:40:54.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.551+0000 7fd9c82a9700 1 -- 172.21.15.17:0/2446987843 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9c00ab9e0 msgr2=0x7fd9c0060be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:54.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.551+0000 7fd9c82a9700 1 --2- 172.21.15.17:0/2446987843 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9c00ab9e0 0x7fd9c0060be0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0x7fd9b000c000 tx=0x7fd9b0006df0).stop 2022-08-01T19:40:54.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.551+0000 7fd9c82a9700 1 -- 172.21.15.17:0/2446987843 shutdown_connections 2022-08-01T19:40:54.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.551+0000 7fd9c82a9700 1 --2- 172.21.15.17:0/2446987843 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd9a0053980 0x7fd9a0055e40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:54.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.551+0000 7fd9c82a9700 1 --2- 172.21.15.17:0/2446987843 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd9c0144760 0x7fd9c0154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:54.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.551+0000 7fd9c82a9700 1 --2- 172.21.15.17:0/2446987843 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9c013eae0 0x7fd9c0153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:54.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.551+0000 7fd9c82a9700 1 --2- 172.21.15.17:0/2446987843 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9c00ab9e0 0x7fd9c0060be0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:54.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.551+0000 7fd9c82a9700 1 -- 172.21.15.17:0/2446987843 >> 172.21.15.17:0/2446987843 conn(0x7fd9c00b5570 msgr2=0x7fd9c013f150 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:54.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.551+0000 7fd9c82a9700 1 -- 172.21.15.17:0/2446987843 shutdown_connections 2022-08-01T19:40:54.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.551+0000 7fd9c82a9700 1 -- 172.21.15.17:0/2446987843 wait complete. 2022-08-01T19:40:54.573 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-08-01T19:40:54.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.687+0000 7f1c46755700 1 Processor -- start 2022-08-01T19:40:54.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.687+0000 7f1c46755700 1 -- start start 2022-08-01T19:40:54.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.687+0000 7f1c46755700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1c400b1580 0x7f1c400b3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:54.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.687+0000 7f1c46755700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1c40147450 0x7f1c40143740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:54.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.687+0000 7f1c46755700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c400ae7e0 con 0x7f1c400b1ef0 2022-08-01T19:40:54.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.687+0000 7f1c46755700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c400b1960 con 0x7f1c400b1580 2022-08-01T19:40:54.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.687+0000 7f1c46755700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c400b2470 con 0x7f1c40147450 2022-08-01T19:40:54.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.687+0000 7f1c3ffff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1c400b1580 0x7f1c400b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:54.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.687+0000 7f1c3ffff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1c400b1580 0x7f1c400b3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41072/0 (socket says 172.21.15.17:41072) 2022-08-01T19:40:54.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.687+0000 7f1c3ffff700 1 -- 172.21.15.17:0/1738898792 learned_addr learned my addr 172.21.15.17:0/1738898792 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:54.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.687+0000 7f1c3f7fe700 1 --1- 172.21.15.17:0/1738898792 >> v1:172.21.15.77:6789/0 conn(0x7f1c400b1ef0 0x7f1c40146d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:50068/0 (socket says 172.21.15.17:50068) 2022-08-01T19:40:54.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.687+0000 7f1c44cf2700 1 --2- 172.21.15.17:0/1738898792 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1c40147450 0x7f1c40143740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:54.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.691+0000 7f1c3ffff700 1 -- 172.21.15.17:0/1738898792 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1c40147450 msgr2=0x7f1c40143740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:54.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.691+0000 7f1c3ffff700 1 --2- 172.21.15.17:0/1738898792 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1c40147450 0x7f1c40143740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:54.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.691+0000 7f1c3ffff700 1 -- 172.21.15.17:0/1738898792 >> v1:172.21.15.77:6789/0 conn(0x7f1c400b1ef0 legacy=0x7f1c40146d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:54.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.691+0000 7f1c3ffff700 1 -- 172.21.15.17:0/1738898792 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c400b2710 con 0x7f1c400b1580 2022-08-01T19:40:54.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.691+0000 7f1c44cf2700 1 --2- 172.21.15.17:0/1738898792 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1c40147450 0x7f1c40143740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:40:54.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.691+0000 7f1c3ffff700 1 --2- 172.21.15.17:0/1738898792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c400b1580 0x7f1c400b3210 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7f1c38002f70 tx=0x7f1c38006eb0).ready entity=mon.0 client_cookie=f57217dfd53654a7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:54.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.691+0000 7f1c3effd700 1 -- 172.21.15.17:0/1738898792 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c3800d5c0 con 0x7f1c400b1580 2022-08-01T19:40:54.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.691+0000 7f1c3effd700 1 -- 172.21.15.17:0/1738898792 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c38000e90 con 0x7f1c400b1580 2022-08-01T19:40:54.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.691+0000 7f1c3effd700 1 -- 172.21.15.17:0/1738898792 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c3800d950 con 0x7f1c400b1580 2022-08-01T19:40:54.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.691+0000 7f1c46755700 1 -- 172.21.15.17:0/1738898792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c400b1580 msgr2=0x7f1c400b3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:54.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.691+0000 7f1c46755700 1 --2- 172.21.15.17:0/1738898792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c400b1580 0x7f1c400b3210 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7f1c38002f70 tx=0x7f1c38006eb0).stop 2022-08-01T19:40:54.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.691+0000 7f1c46755700 1 -- 172.21.15.17:0/1738898792 shutdown_connections 2022-08-01T19:40:54.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.691+0000 7f1c46755700 1 --2- 172.21.15.17:0/1738898792 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1c40147450 0x7f1c40143740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:54.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.691+0000 7f1c46755700 1 --2- 172.21.15.17:0/1738898792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c400b1580 0x7f1c400b3210 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:54.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.691+0000 7f1c46755700 1 -- 172.21.15.17:0/1738898792 >> 172.21.15.17:0/1738898792 conn(0x7f1c40060950 msgr2=0x7f1c40060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:54.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.691+0000 7f1c46755700 1 -- 172.21.15.17:0/1738898792 shutdown_connections 2022-08-01T19:40:54.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.691+0000 7f1c46755700 1 -- 172.21.15.17:0/1738898792 wait complete. 2022-08-01T19:40:54.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.691+0000 7f1c46755700 1 Processor -- start 2022-08-01T19:40:54.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.691+0000 7f1c46755700 1 -- start start 2022-08-01T19:40:54.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.695+0000 7f1c46755700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c400b1580 0x7f1c400b0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:54.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.695+0000 7f1c46755700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c400b1ef0 0x7f1c40156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:54.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.695+0000 7f1c46755700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1c40147450 0x7f1c40064190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:54.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.695+0000 7f1c46755700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c40152980 con 0x7f1c400b1ef0 2022-08-01T19:40:54.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.695+0000 7f1c46755700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c40156c80 con 0x7f1c400b1580 2022-08-01T19:40:54.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.695+0000 7f1c46755700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c400ac620 con 0x7f1c40147450 2022-08-01T19:40:54.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.695+0000 7f1c3f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c400b1ef0 0x7f1c40156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:54.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.695+0000 7f1c3f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c400b1ef0 0x7f1c40156790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41080/0 (socket says 172.21.15.17:41080) 2022-08-01T19:40:54.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.695+0000 7f1c3f7fe700 1 -- 172.21.15.17:0/1512852447 learned_addr learned my addr 172.21.15.17:0/1512852447 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:54.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.695+0000 7f1c3ffff700 1 --2- 172.21.15.17:0/1512852447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c400b1580 0x7f1c400b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:54.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.695+0000 7f1c44cf2700 1 --2- 172.21.15.17:0/1512852447 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1c40147450 0x7f1c40064190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:54.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.695+0000 7f1c3f7fe700 1 -- 172.21.15.17:0/1512852447 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1c40147450 msgr2=0x7f1c40064190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:54.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.695+0000 7f1c3f7fe700 1 --2- 172.21.15.17:0/1512852447 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1c40147450 0x7f1c40064190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:54.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.695+0000 7f1c3f7fe700 1 -- 172.21.15.17:0/1512852447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c400b1580 msgr2=0x7f1c400b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:54.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.695+0000 7f1c3f7fe700 1 --2- 172.21.15.17:0/1512852447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c400b1580 0x7f1c400b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:54.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.695+0000 7f1c3f7fe700 1 -- 172.21.15.17:0/1512852447 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c3800b1e0 con 0x7f1c400b1ef0 2022-08-01T19:40:54.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.695+0000 7f1c3f7fe700 1 --2- 172.21.15.17:0/1512852447 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c400b1ef0 0x7f1c40156790 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 rx=0x7f1c40146c20 tx=0x7f1c28007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:54.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.695+0000 7f1c3d7fa700 1 -- 172.21.15.17:0/1512852447 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c280070d0 con 0x7f1c400b1ef0 2022-08-01T19:40:54.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.695+0000 7f1c3d7fa700 1 -- 172.21.15.17:0/1512852447 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c28007230 con 0x7f1c400b1ef0 2022-08-01T19:40:54.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.695+0000 7f1c46755700 1 -- 172.21.15.17:0/1512852447 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c400ac8c0 con 0x7f1c400b1ef0 2022-08-01T19:40:54.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.695+0000 7f1c3d7fa700 1 -- 172.21.15.17:0/1512852447 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c28005550 con 0x7f1c400b1ef0 2022-08-01T19:40:54.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.695+0000 7f1c46755700 1 -- 172.21.15.17:0/1512852447 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c40063a60 con 0x7f1c400b1ef0 2022-08-01T19:40:54.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.695+0000 7f1c46755700 1 -- 172.21.15.17:0/1512852447 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c400650b0 con 0x7f1c400b1ef0 2022-08-01T19:40:54.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.695+0000 7f1c3d7fa700 1 -- 172.21.15.17:0/1512852447 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1c28010070 con 0x7f1c400b1ef0 2022-08-01T19:40:54.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.699+0000 7f1c3d7fa700 1 --2- 172.21.15.17:0/1512852447 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1c24053b00 0x7f1c24055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:54.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.699+0000 7f1c3d7fa700 1 -- 172.21.15.17:0/1512852447 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f1c2806b330 con 0x7f1c400b1ef0 2022-08-01T19:40:54.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.699+0000 7f1c3ffff700 1 --2- 172.21.15.17:0/1512852447 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1c24053b00 0x7f1c24055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:54.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.699+0000 7f1c3ffff700 1 --2- 172.21.15.17:0/1512852447 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1c24053b00 0x7f1c24055fc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f1c3801c000 tx=0x7f1c38015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:54.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.703+0000 7f1c3d7fa700 1 -- 172.21.15.17:0/1512852447 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1c400650b0 con 0x7f1c400b1ef0 2022-08-01T19:40:54.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.827+0000 7f1c46755700 1 -- 172.21.15.17:0/1512852447 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1c400650b0 con 0x7f1c400b1ef0 2022-08-01T19:40:54.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.831+0000 7f1c3d7fa700 1 -- 172.21.15.17:0/1512852447 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1c400650b0 con 0x7f1c400b1ef0 2022-08-01T19:40:54.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.831+0000 7f1c46755700 1 -- 172.21.15.17:0/1512852447 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1c24053b00 msgr2=0x7f1c24055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:54.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.831+0000 7f1c46755700 1 --2- 172.21.15.17:0/1512852447 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1c24053b00 0x7f1c24055fc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f1c3801c000 tx=0x7f1c38015000).stop 2022-08-01T19:40:54.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.831+0000 7f1c46755700 1 -- 172.21.15.17:0/1512852447 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c400b1ef0 msgr2=0x7f1c40156790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:54.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.831+0000 7f1c46755700 1 --2- 172.21.15.17:0/1512852447 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c400b1ef0 0x7f1c40156790 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 rx=0x7f1c40146c20 tx=0x7f1c28007c40).stop 2022-08-01T19:40:54.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.831+0000 7f1c46755700 1 -- 172.21.15.17:0/1512852447 shutdown_connections 2022-08-01T19:40:54.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.831+0000 7f1c46755700 1 --2- 172.21.15.17:0/1512852447 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1c24053b00 0x7f1c24055fc0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:54.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.831+0000 7f1c46755700 1 --2- 172.21.15.17:0/1512852447 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1c40147450 0x7f1c40064190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:54.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.831+0000 7f1c46755700 1 --2- 172.21.15.17:0/1512852447 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c400b1ef0 0x7f1c40156790 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:54.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.831+0000 7f1c46755700 1 --2- 172.21.15.17:0/1512852447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c400b1580 0x7f1c400b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:54.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.831+0000 7f1c46755700 1 -- 172.21.15.17:0/1512852447 >> 172.21.15.17:0/1512852447 conn(0x7f1c40060950 msgr2=0x7f1c400b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:54.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.835+0000 7f1c46755700 1 -- 172.21.15.17:0/1512852447 shutdown_connections 2022-08-01T19:40:54.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:54.835+0000 7f1c46755700 1 -- 172.21.15.17:0/1512852447 wait complete. 2022-08-01T19:40:54.848 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T19:40:54.849 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:40:54.869 DEBUG:teuthology.orchestra.run.smithi077:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:40:54.892 DEBUG:teuthology.orchestra.run.smithi085:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:40:54.914 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:40:54.938 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T19:40:54.939 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:40:54.940 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:40:54.940 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:40:54.949 INFO:teuthology.orchestra.run.smithi085.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-08-01T19:40:54.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:40:54.952 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-08-01T19:40:54.953 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:40:54.954 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:40:54.955 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:40:54.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:40:54.995 INFO:teuthology.orchestra.run.smithi190.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T19:40:54.996 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-08-01T19:40:54.998 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T19:40:55.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.103+0000 7f95ab59e700 1 Processor -- start 2022-08-01T19:40:55.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.103+0000 7f95ab59e700 1 -- start start 2022-08-01T19:40:55.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95ab59e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f95ac064b10 0x7f95ac144e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:55.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95ab59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f95ac141a80 0x7f95ac066ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:55.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95ab59e700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ac064ef0 con 0x7f95ac0b36e0 2022-08-01T19:40:55.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95ab59e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95ac13ebf0 con 0x7f95ac064b10 2022-08-01T19:40:55.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95ab59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac066690 con 0x7f95ac141a80 2022-08-01T19:40:55.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95a9d9b700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f95ac0b36e0 0x7f95ac141370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33280/0 (socket says 172.21.15.17:33280) 2022-08-01T19:40:55.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95a9d9b700 1 -- 172.21.15.17:0/943165795 learned_addr learned my addr 172.21.15.17:0/943165795 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:40:55.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95aa59c700 1 --2- 172.21.15.17:0/943165795 >> v2:172.21.15.77:3300/0 conn(0x7f95ac064b10 0x7f95ac144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:55.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95aad9d700 1 --2- 172.21.15.17:0/943165795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f95ac141a80 0x7f95ac066ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:55.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95a959a700 1 -- 172.21.15.17:0/943165795 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3439515999 0 0) 0x7f95ac064ef0 con 0x7f95ac0b36e0 2022-08-01T19:40:55.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95a959a700 1 -- 172.21.15.17:0/943165795 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f958c003260 con 0x7f95ac0b36e0 2022-08-01T19:40:55.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95a959a700 1 -- 172.21.15.17:0/943165795 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1420371470 0 0) 0x7f958c003260 con 0x7f95ac0b36e0 2022-08-01T19:40:55.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95a959a700 1 -- 172.21.15.17:0/943165795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f95ac141a80 msgr2=0x7f95ac066ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:55.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95a959a700 1 --2- 172.21.15.17:0/943165795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f95ac141a80 0x7f95ac066ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:55.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95a959a700 1 -- 172.21.15.17:0/943165795 >> v2:172.21.15.77:3300/0 conn(0x7f95ac064b10 msgr2=0x7f95ac144e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:55.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95a959a700 1 --2- 172.21.15.17:0/943165795 >> v2:172.21.15.77:3300/0 conn(0x7f95ac064b10 0x7f95ac144e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:55.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95a959a700 1 -- 172.21.15.17:0/943165795 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ac0b4850 con 0x7f95ac0b36e0 2022-08-01T19:40:55.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95a959a700 1 -- 172.21.15.17:0/943165795 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7f959c001660 con 0x7f95ac0b36e0 2022-08-01T19:40:55.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95a959a700 1 -- 172.21.15.17:0/943165795 >> v1:172.21.15.17:6789/0 conn(0x7f95ac0b36e0 legacy=0x7f95ac141370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:55.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95a959a700 1 --2- 172.21.15.17:0/943165795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f958c003750 0x7f958c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:55.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95a959a700 1 --2- 172.21.15.17:0/943165795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f95ac141a80 0x7f958c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:55.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95a959a700 1 --2- 172.21.15.17:0/943165795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f958c0049f0 0x7f958c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:55.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95a959a700 1 -- 172.21.15.17:0/943165795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac066690 con 0x7f958c0049f0 2022-08-01T19:40:55.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95a959a700 1 -- 172.21.15.17:0/943165795 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958c004330 con 0x7f958c003750 2022-08-01T19:40:55.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95a959a700 1 -- 172.21.15.17:0/943165795 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f958c00b7a0 con 0x7f95ac141a80 2022-08-01T19:40:55.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95aad9d700 1 --2- 172.21.15.17:0/943165795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f958c0049f0 0x7f958c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:55.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95aa59c700 1 --2- 172.21.15.17:0/943165795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f958c003750 0x7f958c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:55.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95a9d9b700 1 --2- 172.21.15.17:0/943165795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f95ac141a80 0x7f958c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:55.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95aa59c700 1 -- 172.21.15.17:0/943165795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f95ac141a80 msgr2=0x7f958c004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:55.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95aa59c700 1 --2- 172.21.15.17:0/943165795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f95ac141a80 0x7f958c004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:55.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95aa59c700 1 -- 172.21.15.17:0/943165795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f958c0049f0 msgr2=0x7f958c004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:55.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95aa59c700 1 --2- 172.21.15.17:0/943165795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f958c0049f0 0x7f958c004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:55.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95aa59c700 1 -- 172.21.15.17:0/943165795 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f958c003260 con 0x7f958c003750 2022-08-01T19:40:55.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95aad9d700 1 --2- 172.21.15.17:0/943165795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f958c0049f0 0x7f958c004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:40:55.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95aa59c700 1 --2- 172.21.15.17:0/943165795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f958c003750 0x7f958c003b70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f959400c000 tx=0x7f9594006df0).ready entity=mon.1 client_cookie=f1a9a6232b11b923 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:55.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95a959a700 1 -- 172.21.15.17:0/943165795 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f959400e830 con 0x7f958c003750 2022-08-01T19:40:55.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95a959a700 1 -- 172.21.15.17:0/943165795 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959400e990 con 0x7f958c003750 2022-08-01T19:40:55.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.107+0000 7f95a959a700 1 -- 172.21.15.17:0/943165795 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f959400ecf0 con 0x7f958c003750 2022-08-01T19:40:55.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.111+0000 7f95ab59e700 1 -- 172.21.15.17:0/943165795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f958c003750 msgr2=0x7f958c003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:55.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.111+0000 7f95ab59e700 1 --2- 172.21.15.17:0/943165795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f958c003750 0x7f958c003b70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f959400c000 tx=0x7f9594006df0).stop 2022-08-01T19:40:55.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.111+0000 7f95aad9d700 1 -- 172.21.15.17:0/943165795 reap_dead start 2022-08-01T19:40:55.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.111+0000 7f95ab59e700 1 -- 172.21.15.17:0/943165795 shutdown_connections 2022-08-01T19:40:55.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.111+0000 7f95ab59e700 1 -- 172.21.15.17:0/943165795 >> 172.21.15.17:0/943165795 conn(0x7f95ac05f840 msgr2=0x7f95ac05fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:55.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.111+0000 7f95ab59e700 1 -- 172.21.15.17:0/943165795 shutdown_connections 2022-08-01T19:40:55.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.111+0000 7f95ab59e700 1 -- 172.21.15.17:0/943165795 wait complete. 2022-08-01T19:40:55.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.111+0000 7f95ab59e700 1 Processor -- start 2022-08-01T19:40:55.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.111+0000 7f95ab59e700 1 -- start start 2022-08-01T19:40:55.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.111+0000 7f95ab59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f958c003750 0x7f95ac061f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:55.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.111+0000 7f95ab59e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95ac0b8120 0x7f95ac060e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:55.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.111+0000 7f95ab59e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95ac0ac310 0x7f95ac0b5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:55.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.111+0000 7f95ab59e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac0ad840 con 0x7f95ac0ac310 2022-08-01T19:40:55.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.111+0000 7f95ab59e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac153490 con 0x7f95ac0b8120 2022-08-01T19:40:55.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.111+0000 7f95ab59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac0ae060 con 0x7f958c003750 2022-08-01T19:40:55.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.111+0000 7f95aad9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95ac0ac310 0x7f95ac0b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:55.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.111+0000 7f95aad9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95ac0ac310 0x7f95ac0b5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41102/0 (socket says 172.21.15.17:41102) 2022-08-01T19:40:55.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.111+0000 7f95aad9d700 1 -- 172.21.15.17:0/1891241025 learned_addr learned my addr 172.21.15.17:0/1891241025 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:55.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.111+0000 7f95a9d9b700 1 --2- 172.21.15.17:0/1891241025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95ac0b8120 0x7f95ac060e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:55.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.111+0000 7f95aa59c700 1 --2- 172.21.15.17:0/1891241025 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f958c003750 0x7f95ac061f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:55.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.115+0000 7f95aad9d700 1 -- 172.21.15.17:0/1891241025 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f958c003750 msgr2=0x7f95ac061f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:55.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.115+0000 7f95aad9d700 1 --2- 172.21.15.17:0/1891241025 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f958c003750 0x7f95ac061f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:55.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.115+0000 7f95aad9d700 1 -- 172.21.15.17:0/1891241025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95ac0b8120 msgr2=0x7f95ac060e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:55.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.115+0000 7f95aad9d700 1 --2- 172.21.15.17:0/1891241025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95ac0b8120 0x7f95ac060e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:55.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.115+0000 7f95aad9d700 1 -- 172.21.15.17:0/1891241025 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ac0ae300 con 0x7f95ac0ac310 2022-08-01T19:40:55.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.115+0000 7f95aad9d700 1 --2- 172.21.15.17:0/1891241025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95ac0ac310 0x7f95ac0b5190 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7f95a400b000 tx=0x7f95a40091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:55.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.115+0000 7f95a37fe700 1 -- 172.21.15.17:0/1891241025 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95a4019070 con 0x7f95ac0ac310 2022-08-01T19:40:55.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.115+0000 7f95a37fe700 1 -- 172.21.15.17:0/1891241025 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a400f910 con 0x7f95ac0ac310 2022-08-01T19:40:55.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.115+0000 7f95ab59e700 1 -- 172.21.15.17:0/1891241025 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95ac0625a0 con 0x7f95ac0ac310 2022-08-01T19:40:55.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.115+0000 7f95a37fe700 1 -- 172.21.15.17:0/1891241025 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95a400fc70 con 0x7f95ac0ac310 2022-08-01T19:40:55.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.115+0000 7f95ab59e700 1 -- 172.21.15.17:0/1891241025 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95ac152f70 con 0x7f95ac0ac310 2022-08-01T19:40:55.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.115+0000 7f95a37fe700 1 -- 172.21.15.17:0/1891241025 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f95a4014440 con 0x7f95ac0ac310 2022-08-01T19:40:55.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.115+0000 7f95ab59e700 1 -- 172.21.15.17:0/1891241025 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95ac1531c0 con 0x7f95ac0ac310 2022-08-01T19:40:55.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.115+0000 7f95a37fe700 1 --2- 172.21.15.17:0/1891241025 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9590053b30 0x7f9590055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:55.154 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.115+0000 7f95a37fe700 1 -- 172.21.15.17:0/1891241025 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f95a406c7f0 con 0x7f95ac0ac310 2022-08-01T19:40:55.154 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.115+0000 7f95aa59c700 1 --2- 172.21.15.17:0/1891241025 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9590053b30 0x7f9590055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:55.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.119+0000 7f95aa59c700 1 --2- 172.21.15.17:0/1891241025 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9590053b30 0x7f9590055ff0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f959401e000 tx=0x7f9594019000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:55.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.123+0000 7f95a37fe700 1 -- 172.21.15.17:0/1891241025 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f95a4043800 con 0x7f95ac0ac310 2022-08-01T19:40:55.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.251+0000 7f95ab59e700 1 -- 172.21.15.17:0/1891241025 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f95ac0574e0 con 0x7f95ac0ac310 2022-08-01T19:40:55.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.251+0000 7f95a37fe700 1 -- 172.21.15.17:0/1891241025 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v80) v1 ==== 81+0+29292 (secure 0 0 0) 0x7f95a4046e20 con 0x7f95ac0ac310 2022-08-01T19:40:55.258 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:40:55.258 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:40:55.258 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 80, 2022-08-01T19:40:55.259 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", 2022-08-01T19:40:55.259 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2022-08-01T19:33:50.105743+0000", 2022-08-01T19:40:55.259 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2022-08-01T19:40:52.530297+0000", 2022-08-01T19:40:55.259 INFO:teuthology.orchestra.run.smithi017.stdout: "last_up_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:40:55.260 INFO:teuthology.orchestra.run.smithi017.stdout: "last_in_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:40:55.260 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T19:40:55.260 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_num": 5799936, 2022-08-01T19:40:55.262 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_set": [ 2022-08-01T19:40:55.262 INFO:teuthology.orchestra.run.smithi017.stdout: "pglog_hardlimit", 2022-08-01T19:40:55.263 INFO:teuthology.orchestra.run.smithi017.stdout: "purged_snapdirs", 2022-08-01T19:40:55.263 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_deletes", 2022-08-01T19:40:55.263 INFO:teuthology.orchestra.run.smithi017.stdout: "sortbitwise" 2022-08-01T19:40:55.263 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:40:55.264 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_version": 6, 2022-08-01T19:40:55.264 INFO:teuthology.orchestra.run.smithi017.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T19:40:55.265 INFO:teuthology.orchestra.run.smithi017.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T19:40:55.265 INFO:teuthology.orchestra.run.smithi017.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T19:40:55.265 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_snapshot": "", 2022-08-01T19:40:55.266 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_max": 14, 2022-08-01T19:40:55.266 INFO:teuthology.orchestra.run.smithi017.stdout: "max_osd": 8, 2022-08-01T19:40:55.266 INFO:teuthology.orchestra.run.smithi017.stdout: "require_min_compat_client": "jewel", 2022-08-01T19:40:55.267 INFO:teuthology.orchestra.run.smithi017.stdout: "min_compat_client": "jewel", 2022-08-01T19:40:55.267 INFO:teuthology.orchestra.run.smithi017.stdout: "require_osd_release": "octopus", 2022-08-01T19:40:55.267 INFO:teuthology.orchestra.run.smithi017.stdout: "pools": [ 2022-08-01T19:40:55.272 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.272 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 1, 2022-08-01T19:40:55.272 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "device_health_metrics", 2022-08-01T19:40:55.272 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:33:53.001841+0000", 2022-08-01T19:40:55.273 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:40:55.273 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:40:55.273 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:40:55.273 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:40:55.273 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:40:55.274 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:40:55.274 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:40:55.274 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:40:55.274 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 1, 2022-08-01T19:40:55.275 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 1, 2022-08-01T19:40:55.275 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 1, 2022-08-01T19:40:55.275 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 1, 2022-08-01T19:40:55.275 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 1, 2022-08-01T19:40:55.276 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:40:55.276 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:40:55.276 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:40:55.276 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:40:55.277 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:40:55.277 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:40:55.277 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:40:55.277 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.283 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "16", 2022-08-01T19:40:55.283 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:40:55.283 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:40:55.284 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:40:55.284 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:40:55.284 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:40:55.284 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:40:55.285 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:40:55.285 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:40:55.285 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:40:55.285 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:40:55.286 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:40:55.286 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:40:55.286 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:40:55.286 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:40:55.287 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:40:55.287 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:40:55.287 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:40:55.287 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:40:55.288 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:40:55.288 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:40:55.288 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:40:55.288 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:40:55.288 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:40:55.289 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:40:55.289 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:40:55.289 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:40:55.289 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.290 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:40:55.290 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:40:55.290 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:40:55.290 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:40:55.291 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:40:55.291 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:40:55.291 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:40:55.291 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:40:55.292 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:40:55.292 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:40:55.292 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:40:55.292 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:40:55.292 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 1 2022-08-01T19:40:55.293 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.293 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:40:55.293 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_devicehealth": {} 2022-08-01T19:40:55.293 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.293 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.294 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.294 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 2, 2022-08-01T19:40:55.294 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "rbd", 2022-08-01T19:40:55.294 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:34:01.553098+0000", 2022-08-01T19:40:55.295 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:40:55.295 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:40:55.295 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:40:55.295 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:40:55.296 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:40:55.296 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:40:55.296 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:40:55.296 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:40:55.296 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:40:55.297 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:40:55.297 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:40:55.297 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:40:55.297 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:40:55.297 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:40:55.298 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:40:55.298 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:40:55.298 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:40:55.298 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:40:55.299 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:40:55.299 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:40:55.299 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.299 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "19", 2022-08-01T19:40:55.300 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:40:55.300 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:40:55.300 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:40:55.300 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:40:55.301 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:40:55.301 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:40:55.301 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:40:55.301 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:40:55.301 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:40:55.302 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:40:55.302 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:40:55.302 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:40:55.302 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:40:55.303 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:40:55.303 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:40:55.303 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:40:55.303 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:40:55.304 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:40:55.304 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:40:55.304 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:40:55.304 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:40:55.305 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:40:55.305 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:40:55.305 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:40:55.305 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:40:55.305 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:40:55.306 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.306 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:40:55.306 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:40:55.307 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:40:55.307 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:40:55.307 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:40:55.307 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:40:55.308 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:40:55.308 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:40:55.308 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:40:55.308 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:40:55.308 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:40:55.309 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:40:55.309 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:40:55.309 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd": {} 2022-08-01T19:40:55.309 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.310 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.310 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:40:55.310 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2022-08-01T19:40:55.310 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.311 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:40:55.311 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a962f14f-cf16-4fa8-84ff-c85325db9132", 2022-08-01T19:40:55.311 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:40:55.311 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:40:55.311 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:40:55.312 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:40:55.312 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:40:55.312 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:40:55.312 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:40:55.313 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 74, 2022-08-01T19:40:55.313 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:40:55.313 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:40:55.313 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:40:55.313 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:55.314 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.314 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:55.314 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6800", 2022-08-01T19:40:55.314 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:40:55.315 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.315 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.315 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:55.315 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6801", 2022-08-01T19:40:55.315 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:40:55.316 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.316 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.316 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.316 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:40:55.317 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:55.317 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.317 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:55.317 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6802", 2022-08-01T19:40:55.318 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:40:55.318 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.318 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.318 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:55.319 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6803", 2022-08-01T19:40:55.319 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:40:55.319 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.319 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.319 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.320 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:40:55.320 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:55.320 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.320 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:55.321 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6806", 2022-08-01T19:40:55.321 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:40:55.321 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.321 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.321 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:55.322 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6807", 2022-08-01T19:40:55.322 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:40:55.322 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.322 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.323 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.323 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:40:55.323 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:55.323 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.323 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:55.324 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6804", 2022-08-01T19:40:55.324 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:40:55.324 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.324 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.324 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:55.325 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6805", 2022-08-01T19:40:55.325 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:40:55.325 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.325 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.326 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.336 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6801/23008", 2022-08-01T19:40:55.336 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6803/23008", 2022-08-01T19:40:55.336 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6807/23008", 2022-08-01T19:40:55.336 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6805/23008", 2022-08-01T19:40:55.337 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:40:55.337 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:40:55.337 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:40:55.337 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.337 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.338 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.338 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:40:55.338 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "805f7051-a527-4e98-8844-b6e406a63248", 2022-08-01T19:40:55.338 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:40:55.339 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:40:55.339 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:40:55.339 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:40:55.339 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:40:55.340 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:40:55.340 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:40:55.340 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 64, 2022-08-01T19:40:55.340 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:40:55.341 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:40:55.341 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:40:55.341 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:55.341 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.341 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:55.342 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6808", 2022-08-01T19:40:55.350 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:40:55.351 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.351 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.352 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:55.352 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6809", 2022-08-01T19:40:55.352 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:40:55.353 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.353 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.354 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.354 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:40:55.355 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:55.355 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.356 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:55.357 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6810", 2022-08-01T19:40:55.357 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:40:55.358 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.358 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.359 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:55.359 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6811", 2022-08-01T19:40:55.360 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:40:55.360 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.361 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.361 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.362 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:40:55.362 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:55.363 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.363 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:55.364 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6814", 2022-08-01T19:40:55.364 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:40:55.365 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.365 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.366 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:55.366 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6815", 2022-08-01T19:40:55.367 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:40:55.367 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.368 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.368 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.368 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:40:55.369 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:55.369 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.370 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:55.370 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6812", 2022-08-01T19:40:55.371 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:40:55.371 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.372 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.372 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:55.373 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6813", 2022-08-01T19:40:55.373 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:40:55.374 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.374 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.375 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.375 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6809/23007", 2022-08-01T19:40:55.376 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6811/23007", 2022-08-01T19:40:55.376 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6815/23007", 2022-08-01T19:40:55.377 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6813/23007", 2022-08-01T19:40:55.377 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:40:55.378 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:40:55.378 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:40:55.379 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.379 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.380 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.380 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:40:55.380 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "89f7f3df-7c10-48e1-8b69-9e4243923a61", 2022-08-01T19:40:55.386 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:40:55.386 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:40:55.387 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:40:55.387 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:40:55.387 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:40:55.387 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:40:55.388 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:40:55.388 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 74, 2022-08-01T19:40:55.388 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:40:55.388 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:40:55.389 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:40:55.389 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:55.389 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.389 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:55.389 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6817", 2022-08-01T19:40:55.390 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:40:55.390 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.390 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.391 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:55.391 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6819", 2022-08-01T19:40:55.391 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:40:55.392 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.392 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.392 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.392 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:40:55.393 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:55.393 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.393 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:55.393 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6822", 2022-08-01T19:40:55.394 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:40:55.394 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.394 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.394 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:55.395 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6824", 2022-08-01T19:40:55.395 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:40:55.395 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.395 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.395 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.396 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:40:55.396 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:55.396 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.397 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:55.397 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6830", 2022-08-01T19:40:55.397 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:40:55.398 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.398 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.399 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:55.399 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6831", 2022-08-01T19:40:55.399 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:40:55.399 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.400 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.400 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.400 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:40:55.400 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:55.400 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.401 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:55.401 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6826", 2022-08-01T19:40:55.401 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:40:55.401 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.402 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.402 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:55.402 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6829", 2022-08-01T19:40:55.402 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:40:55.403 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.403 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.403 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.403 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6819/23010", 2022-08-01T19:40:55.404 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6824/23010", 2022-08-01T19:40:55.404 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6831/23010", 2022-08-01T19:40:55.404 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6829/23010", 2022-08-01T19:40:55.405 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:40:55.405 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:40:55.405 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:40:55.405 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.406 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.406 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.406 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:40:55.406 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "96123468-aabc-4a64-909e-a192c369090d", 2022-08-01T19:40:55.407 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:40:55.407 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:40:55.407 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:40:55.407 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:40:55.408 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:40:55.408 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:40:55.408 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:40:55.408 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 74, 2022-08-01T19:40:55.409 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:40:55.409 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:40:55.409 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:40:55.409 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:55.410 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.410 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:55.410 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6816", 2022-08-01T19:40:55.411 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:40:55.411 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.411 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.411 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:55.411 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6818", 2022-08-01T19:40:55.412 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:40:55.412 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.412 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.412 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.413 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:40:55.413 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:55.413 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.413 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:55.414 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6820", 2022-08-01T19:40:55.414 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:40:55.414 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.415 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:55.415 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6821", 2022-08-01T19:40:55.415 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:40:55.415 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.415 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.416 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.416 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:40:55.416 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:55.416 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.417 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:55.417 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6827", 2022-08-01T19:40:55.417 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:40:55.417 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.418 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.418 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:55.418 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6828", 2022-08-01T19:40:55.418 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:40:55.419 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.419 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.419 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.420 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:40:55.420 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:55.420 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.420 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:55.421 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6823", 2022-08-01T19:40:55.421 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:40:55.421 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.421 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.422 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:55.422 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6825", 2022-08-01T19:40:55.422 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:40:55.423 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.423 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.423 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.424 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6818/23009", 2022-08-01T19:40:55.424 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6821/23009", 2022-08-01T19:40:55.424 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6828/23009", 2022-08-01T19:40:55.424 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6825/23009", 2022-08-01T19:40:55.425 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:40:55.425 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:40:55.425 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:40:55.425 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.426 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.426 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.426 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:40:55.426 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a6e26665-e04b-46ec-ab66-e912acbf044c", 2022-08-01T19:40:55.427 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:40:55.427 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:40:55.427 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:40:55.428 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:40:55.428 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:40:55.428 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:40:55.428 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:40:55.428 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 73, 2022-08-01T19:40:55.429 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:40:55.429 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:40:55.430 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:40:55.430 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:55.431 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.431 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:55.431 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6810", 2022-08-01T19:40:55.431 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:40:55.432 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.432 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.432 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:55.432 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6811", 2022-08-01T19:40:55.432 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:40:55.433 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.433 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.433 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.433 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:40:55.434 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:55.434 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.434 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:55.435 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6812", 2022-08-01T19:40:55.435 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:40:55.436 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.436 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.436 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:55.437 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6813", 2022-08-01T19:40:55.437 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:40:55.437 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.437 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.437 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.438 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:40:55.438 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:55.438 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.438 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:55.439 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6816", 2022-08-01T19:40:55.439 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:40:55.439 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.439 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.440 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:55.440 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6817", 2022-08-01T19:40:55.440 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:40:55.440 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.441 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.441 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.441 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:40:55.441 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:55.442 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.442 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:55.442 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6814", 2022-08-01T19:40:55.443 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:40:55.443 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.443 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.443 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:55.444 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6815", 2022-08-01T19:40:55.444 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:40:55.444 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.444 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.445 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.445 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6811/23584", 2022-08-01T19:40:55.445 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6813/23584", 2022-08-01T19:40:55.445 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6817/23584", 2022-08-01T19:40:55.445 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6815/23584", 2022-08-01T19:40:55.446 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:40:55.446 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:40:55.446 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:40:55.446 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.447 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.447 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.447 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:40:55.447 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "502c0034-18ee-44ac-87ff-19b9dddb66a0", 2022-08-01T19:40:55.448 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:40:55.448 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:40:55.448 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:40:55.448 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:40:55.449 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:40:55.449 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:40:55.449 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:40:55.449 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 74, 2022-08-01T19:40:55.453 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:40:55.454 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:40:55.454 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:40:55.455 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:55.457 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.457 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:55.457 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6802", 2022-08-01T19:40:55.458 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:40:55.459 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.459 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.460 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:55.460 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6803", 2022-08-01T19:40:55.461 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:40:55.462 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.462 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.462 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.463 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:40:55.465 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:55.465 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.466 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:55.466 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6804", 2022-08-01T19:40:55.467 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:40:55.467 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.468 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.468 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:55.468 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6805", 2022-08-01T19:40:55.469 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:40:55.469 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.472 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.472 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.473 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:40:55.473 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:55.474 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.474 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:55.475 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6808", 2022-08-01T19:40:55.475 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:40:55.476 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.476 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.477 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:55.477 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6809", 2022-08-01T19:40:55.479 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:40:55.479 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.480 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.480 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.482 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:40:55.493 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:55.493 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.494 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:55.494 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6806", 2022-08-01T19:40:55.495 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:40:55.495 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.496 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.496 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:55.497 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6807", 2022-08-01T19:40:55.497 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:40:55.499 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.499 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.499 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.500 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6803/23585", 2022-08-01T19:40:55.501 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6805/23585", 2022-08-01T19:40:55.501 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6809/23585", 2022-08-01T19:40:55.501 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6807/23585", 2022-08-01T19:40:55.503 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:40:55.503 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:40:55.503 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:40:55.504 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.504 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.504 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.504 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:40:55.505 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "ac4e755a-7fe7-46cb-8407-852d5af9ffc5", 2022-08-01T19:40:55.505 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:40:55.505 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:40:55.510 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:40:55.511 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:40:55.511 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:40:55.512 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:40:55.512 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:40:55.513 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 74, 2022-08-01T19:40:55.513 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:40:55.513 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:40:55.514 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:40:55.514 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:55.515 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.515 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:55.516 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6820", 2022-08-01T19:40:55.516 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:40:55.517 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.517 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.518 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:55.518 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6821", 2022-08-01T19:40:55.519 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:40:55.519 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.519 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.520 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.520 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:40:55.521 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:55.521 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.521 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:55.522 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6823", 2022-08-01T19:40:55.522 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:40:55.522 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.523 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.523 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:55.524 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6825", 2022-08-01T19:40:55.524 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:40:55.524 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.525 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.525 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.526 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:40:55.526 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:55.526 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.527 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:55.527 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6831", 2022-08-01T19:40:55.528 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:40:55.528 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.529 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.529 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:55.530 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6833", 2022-08-01T19:40:55.530 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:40:55.531 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.531 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.532 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.532 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:40:55.533 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:55.533 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.534 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:55.534 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6827", 2022-08-01T19:40:55.535 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:40:55.535 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.536 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.536 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:55.537 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6829", 2022-08-01T19:40:55.537 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:40:55.538 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.538 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.539 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.539 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6821/23587", 2022-08-01T19:40:55.539 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6825/23587", 2022-08-01T19:40:55.540 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6833/23587", 2022-08-01T19:40:55.540 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6829/23587", 2022-08-01T19:40:55.541 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:40:55.541 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:40:55.542 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:40:55.542 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.542 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.543 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.543 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:40:55.547 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "874761e2-6410-40fc-b60c-e58abe5ce489", 2022-08-01T19:40:55.547 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:40:55.548 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:40:55.548 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:40:55.549 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:40:55.549 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:40:55.550 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:40:55.550 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:40:55.551 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 73, 2022-08-01T19:40:55.551 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:40:55.552 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:40:55.552 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:40:55.552 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:55.553 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.553 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:55.554 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6818", 2022-08-01T19:40:55.554 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:40:55.555 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.555 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.555 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:55.556 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6819", 2022-08-01T19:40:55.556 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:40:55.557 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.557 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.558 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.558 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:40:55.559 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:55.559 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.560 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:55.560 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6822", 2022-08-01T19:40:55.560 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:40:55.561 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.561 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.562 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:55.562 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6824", 2022-08-01T19:40:55.562 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:40:55.563 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.563 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.564 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.564 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:40:55.564 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:55.565 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.565 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:55.566 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6830", 2022-08-01T19:40:55.566 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:40:55.567 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.567 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.567 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:55.568 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6832", 2022-08-01T19:40:55.568 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:40:55.568 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.569 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.569 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.570 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:40:55.570 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:40:55.571 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.571 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:40:55.571 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6826", 2022-08-01T19:40:55.572 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:40:55.572 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.573 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.573 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:40:55.574 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6828", 2022-08-01T19:40:55.574 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:40:55.575 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.575 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.575 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.576 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6819/23586", 2022-08-01T19:40:55.576 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6824/23586", 2022-08-01T19:40:55.577 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6832/23586", 2022-08-01T19:40:55.577 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6828/23586", 2022-08-01T19:40:55.577 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:40:55.578 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:40:55.578 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:40:55.579 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:55.579 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.579 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:40:55.580 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_xinfo": [ 2022-08-01T19:40:55.580 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.581 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:40:55.581 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:40:55.582 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:40:55.582 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:40:55.582 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:40:55.583 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:40:55.583 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.588817+0000", 2022-08-01T19:40:55.584 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:40:55.584 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.584 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.585 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:40:55.585 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:40:55.586 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:40:55.586 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:40:55.587 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:40:55.587 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:40:55.587 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.782753+0000", 2022-08-01T19:40:55.588 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:40:55.588 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.589 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.589 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:40:55.592 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:40:55.593 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:40:55.593 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:40:55.594 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:40:55.594 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:40:55.595 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.837924+0000", 2022-08-01T19:40:55.595 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:40:55.596 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.596 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.596 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:40:55.597 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:40:55.597 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:40:55.598 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:40:55.598 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:40:55.599 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:40:55.599 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.839948+0000", 2022-08-01T19:40:55.600 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:40:55.600 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.601 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.601 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:40:55.602 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:40:55.602 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:40:55.602 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:40:55.603 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:40:55.603 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:40:55.604 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.112801+0000", 2022-08-01T19:40:55.604 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:40:55.605 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.605 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.606 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:40:55.606 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:40:55.607 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:40:55.607 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:40:55.607 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:40:55.608 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:40:55.608 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.106704+0000", 2022-08-01T19:40:55.609 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:40:55.610 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.611 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.611 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:40:55.611 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:40:55.612 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:40:55.612 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:40:55.612 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:40:55.613 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:40:55.613 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.893170+0000", 2022-08-01T19:40:55.614 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:40:55.614 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.614 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:55.615 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:40:55.615 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:40:55.616 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:40:55.616 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:40:55.617 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:40:55.617 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:40:55.618 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.886947+0000", 2022-08-01T19:40:55.618 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:40:55.619 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.619 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:40:55.620 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap": [], 2022-08-01T19:40:55.620 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap_items": [], 2022-08-01T19:40:55.621 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_temp": [], 2022-08-01T19:40:55.621 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_temp": [], 2022-08-01T19:40:55.621 INFO:teuthology.orchestra.run.smithi017.stdout: "blacklist": { 2022-08-01T19:40:55.622 INFO:teuthology.orchestra.run.smithi017.stdout: "172.21.15.17:6835/3370152344": "2022-08-02T19:40:49.030637+0000", 2022-08-01T19:40:55.622 INFO:teuthology.orchestra.run.smithi017.stdout: "172.21.15.17:6834/3370152344": "2022-08-02T19:40:49.030637+0000", 2022-08-01T19:40:55.623 INFO:teuthology.orchestra.run.smithi017.stdout: "172.21.15.85:0/3805297491": "2022-08-01T20:40:38.016590+0000" 2022-08-01T19:40:55.623 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.624 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profiles": { 2022-08-01T19:40:55.624 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_data": { 2022-08-01T19:40:55.624 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-device-class": "", 2022-08-01T19:40:55.625 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:40:55.625 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-root": "default", 2022-08-01T19:40:55.625 INFO:teuthology.orchestra.run.smithi017.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T19:40:55.626 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:40:55.626 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "2", 2022-08-01T19:40:55.626 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:40:55.626 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:40:55.627 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van", 2022-08-01T19:40:55.627 INFO:teuthology.orchestra.run.smithi017.stdout: "w": "8" 2022-08-01T19:40:55.627 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.627 INFO:teuthology.orchestra.run.smithi017.stdout: "default": { 2022-08-01T19:40:55.628 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:40:55.628 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:40:55.628 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "1", 2022-08-01T19:40:55.628 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:40:55.629 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:40:55.629 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van" 2022-08-01T19:40:55.629 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:55.629 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:55.630 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps_queue": [], 2022-08-01T19:40:55.630 INFO:teuthology.orchestra.run.smithi017.stdout: "new_removed_snaps": [], 2022-08-01T19:40:55.630 INFO:teuthology.orchestra.run.smithi017.stdout: "new_purged_snaps": [], 2022-08-01T19:40:55.630 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_node_flags": {}, 2022-08-01T19:40:55.630 INFO:teuthology.orchestra.run.smithi017.stdout: "device_class_flags": {} 2022-08-01T19:40:55.631 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:40:55.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.251+0000 7f95ab59e700 1 -- 172.21.15.17:0/1891241025 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9590053b30 msgr2=0x7f9590055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:55.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.251+0000 7f95ab59e700 1 --2- 172.21.15.17:0/1891241025 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9590053b30 0x7f9590055ff0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f959401e000 tx=0x7f9594019000).stop 2022-08-01T19:40:55.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.255+0000 7f95ab59e700 1 -- 172.21.15.17:0/1891241025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95ac0ac310 msgr2=0x7f95ac0b5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:55.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.255+0000 7f95ab59e700 1 --2- 172.21.15.17:0/1891241025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95ac0ac310 0x7f95ac0b5190 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7f95a400b000 tx=0x7f95a40091d0).stop 2022-08-01T19:40:55.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.255+0000 7f95ab59e700 1 -- 172.21.15.17:0/1891241025 shutdown_connections 2022-08-01T19:40:55.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.255+0000 7f95ab59e700 1 --2- 172.21.15.17:0/1891241025 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9590053b30 0x7f9590055ff0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:55.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.255+0000 7f95ab59e700 1 --2- 172.21.15.17:0/1891241025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95ac0ac310 0x7f95ac0b5190 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:55.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.255+0000 7f95ab59e700 1 --2- 172.21.15.17:0/1891241025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95ac0b8120 0x7f95ac060e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:55.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.255+0000 7f95ab59e700 1 --2- 172.21.15.17:0/1891241025 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f958c003750 0x7f95ac061f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:55.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.255+0000 7f95ab59e700 1 -- 172.21.15.17:0/1891241025 >> 172.21.15.17:0/1891241025 conn(0x7f95ac05f840 msgr2=0x7f95ac140830 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:55.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.255+0000 7f95ab59e700 1 -- 172.21.15.17:0/1891241025 shutdown_connections 2022-08-01T19:40:55.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.255+0000 7f95ab59e700 1 -- 172.21.15.17:0/1891241025 wait complete. 2022-08-01T19:40:55.636 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:40:55.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.747+0000 7fa0a0c7f700 1 Processor -- start 2022-08-01T19:40:55.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.747+0000 7fa0a0c7f700 1 -- start start 2022-08-01T19:40:55.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.747+0000 7fa0a0c7f700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa09c0b1f40 0x7fa09c0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:55.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.747+0000 7fa0a0c7f700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa09c0a7250 0x7fa09c1448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:55.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.747+0000 7fa0a0c7f700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa09c0b04e0 con 0x7fa09c148530 2022-08-01T19:40:55.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.747+0000 7fa0a0c7f700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa09c0a9250 con 0x7fa09c0b1f40 2022-08-01T19:40:55.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.747+0000 7fa0a0c7f700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c0b0780 con 0x7fa09c0a7250 2022-08-01T19:40:55.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.747+0000 7fa09a59c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa09c0b1f40 0x7fa09c0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:55.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.747+0000 7fa091d9b700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa09c148530 0x7fa09c0b2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33300/0 (socket says 172.21.15.17:33300) 2022-08-01T19:40:55.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.747+0000 7fa091d9b700 1 -- 172.21.15.17:0/4031949231 learned_addr learned my addr 172.21.15.17:0/4031949231 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:40:55.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.747+0000 7fa09ad9d700 1 --2- 172.21.15.17:0/4031949231 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa09c0a7250 0x7fa09c1448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:55.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.747+0000 7fa09a59c700 1 -- 172.21.15.17:0/4031949231 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa09c0a7250 msgr2=0x7fa09c1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:55.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.747+0000 7fa09a59c700 1 --2- 172.21.15.17:0/4031949231 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa09c0a7250 0x7fa09c1448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:55.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.747+0000 7fa09a59c700 1 -- 172.21.15.17:0/4031949231 >> v1:172.21.15.17:6789/0 conn(0x7fa09c148530 legacy=0x7fa09c0b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:55.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.747+0000 7fa09a59c700 1 -- 172.21.15.17:0/4031949231 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa09c0b0a20 con 0x7fa09c0b1f40 2022-08-01T19:40:55.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.747+0000 7fa09ad9d700 1 --2- 172.21.15.17:0/4031949231 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa09c0a7250 0x7fa09c1448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:40:55.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.747+0000 7fa09a59c700 1 --2- 172.21.15.17:0/4031949231 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa09c0b1f40 0x7fa09c0b1160 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7fa09c1478c0 tx=0x7fa08c006df0).ready entity=mon.0 client_cookie=80ac0904ba5a3854 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:55.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.747+0000 7fa099d9b700 1 -- 172.21.15.17:0/4031949231 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa08c00d570 con 0x7fa09c0b1f40 2022-08-01T19:40:55.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.747+0000 7fa099d9b700 1 -- 172.21.15.17:0/4031949231 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa08c000e90 con 0x7fa09c0b1f40 2022-08-01T19:40:55.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.747+0000 7fa099d9b700 1 -- 172.21.15.17:0/4031949231 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa08c00d8c0 con 0x7fa09c0b1f40 2022-08-01T19:40:55.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.747+0000 7fa0a0c7f700 1 -- 172.21.15.17:0/4031949231 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa09c0b1f40 msgr2=0x7fa09c0b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:55.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.747+0000 7fa0a0c7f700 1 --2- 172.21.15.17:0/4031949231 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa09c0b1f40 0x7fa09c0b1160 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7fa09c1478c0 tx=0x7fa08c006df0).stop 2022-08-01T19:40:55.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.747+0000 7fa0a0c7f700 1 -- 172.21.15.17:0/4031949231 shutdown_connections 2022-08-01T19:40:55.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.747+0000 7fa0a0c7f700 1 --2- 172.21.15.17:0/4031949231 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa09c0a7250 0x7fa09c1448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:55.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.747+0000 7fa0a0c7f700 1 --2- 172.21.15.17:0/4031949231 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa09c0b1f40 0x7fa09c0b1160 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:55.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.747+0000 7fa0a0c7f700 1 -- 172.21.15.17:0/4031949231 >> 172.21.15.17:0/4031949231 conn(0x7fa09c05f840 msgr2=0x7fa09c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:55.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.747+0000 7fa0a0c7f700 1 -- 172.21.15.17:0/4031949231 shutdown_connections 2022-08-01T19:40:55.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.747+0000 7fa0a0c7f700 1 -- 172.21.15.17:0/4031949231 wait complete. 2022-08-01T19:40:55.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.751+0000 7fa0a0c7f700 1 Processor -- start 2022-08-01T19:40:55.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.751+0000 7fa0a0c7f700 1 -- start start 2022-08-01T19:40:55.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.751+0000 7fa0a0c7f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa09c0a7250 0x7fa09c14d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:55.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.751+0000 7fa0a0c7f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa09c0b1f40 0x7fa09c0abbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:55.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.751+0000 7fa0a0c7f700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa09c148530 0x7fa09c0ac0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:55.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.751+0000 7fa0a0c7f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c14b910 con 0x7fa09c0b1f40 2022-08-01T19:40:55.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.751+0000 7fa0a0c7f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c0aaf00 con 0x7fa09c0a7250 2022-08-01T19:40:55.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.751+0000 7fa0a0c7f700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c0ab1a0 con 0x7fa09c148530 2022-08-01T19:40:55.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.751+0000 7fa091d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa09c0b1f40 0x7fa09c0abbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:55.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.751+0000 7fa091d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa09c0b1f40 0x7fa09c0abbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41114/0 (socket says 172.21.15.17:41114) 2022-08-01T19:40:55.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.751+0000 7fa091d9b700 1 -- 172.21.15.17:0/3568504335 learned_addr learned my addr 172.21.15.17:0/3568504335 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:55.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.751+0000 7fa09a59c700 1 --2- 172.21.15.17:0/3568504335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa09c0a7250 0x7fa09c14d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:55.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.751+0000 7fa09ad9d700 1 --2- 172.21.15.17:0/3568504335 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa09c148530 0x7fa09c0ac0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:55.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.755+0000 7fa091d9b700 1 -- 172.21.15.17:0/3568504335 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa09c148530 msgr2=0x7fa09c0ac0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:55.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.755+0000 7fa091d9b700 1 --2- 172.21.15.17:0/3568504335 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa09c148530 0x7fa09c0ac0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:55.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.755+0000 7fa091d9b700 1 -- 172.21.15.17:0/3568504335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa09c0a7250 msgr2=0x7fa09c14d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:55.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.755+0000 7fa091d9b700 1 --2- 172.21.15.17:0/3568504335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa09c0a7250 0x7fa09c14d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:55.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.755+0000 7fa091d9b700 1 -- 172.21.15.17:0/3568504335 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa08c00b180 con 0x7fa09c0b1f40 2022-08-01T19:40:55.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.755+0000 7fa091d9b700 1 --2- 172.21.15.17:0/3568504335 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa09c0b1f40 0x7fa09c0abbe0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7fa09c0b3300 tx=0x7fa084007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:55.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.755+0000 7fa093fff700 1 -- 172.21.15.17:0/3568504335 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0840070d0 con 0x7fa09c0b1f40 2022-08-01T19:40:55.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.755+0000 7fa093fff700 1 -- 172.21.15.17:0/3568504335 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa084007230 con 0x7fa09c0b1f40 2022-08-01T19:40:55.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.755+0000 7fa0a0c7f700 1 -- 172.21.15.17:0/3568504335 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa09c0ab440 con 0x7fa09c0b1f40 2022-08-01T19:40:55.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.755+0000 7fa093fff700 1 -- 172.21.15.17:0/3568504335 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa084005610 con 0x7fa09c0b1f40 2022-08-01T19:40:55.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.755+0000 7fa0a0c7f700 1 -- 172.21.15.17:0/3568504335 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa09c0ab8e0 con 0x7fa09c0b1f40 2022-08-01T19:40:55.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.755+0000 7fa0a0c7f700 1 -- 172.21.15.17:0/3568504335 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa09c0ab5d0 con 0x7fa09c0b1f40 2022-08-01T19:40:55.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.755+0000 7fa093fff700 1 -- 172.21.15.17:0/3568504335 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa084010070 con 0x7fa09c0b1f40 2022-08-01T19:40:55.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.759+0000 7fa093fff700 1 --2- 172.21.15.17:0/3568504335 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa080053b30 0x7fa080055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:55.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.759+0000 7fa093fff700 1 -- 172.21.15.17:0/3568504335 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5729+0+0 (secure 0 0 0) 0x7fa08406b220 con 0x7fa09c0b1f40 2022-08-01T19:40:55.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.759+0000 7fa09a59c700 1 --2- 172.21.15.17:0/3568504335 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa080053b30 0x7fa080055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:55.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.759+0000 7fa09a59c700 1 --2- 172.21.15.17:0/3568504335 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa080053b30 0x7fa080055ff0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7fa08c01c000 tx=0x7fa08c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:55.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.763+0000 7fa093fff700 1 -- 172.21.15.17:0/3568504335 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa084042210 con 0x7fa09c0b1f40 2022-08-01T19:40:55.934 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.923+0000 7fa0a0c7f700 1 -- 172.21.15.17:0/3568504335 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa09c0574e0 con 0x7fa09c0b1f40 2022-08-01T19:40:55.935 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.923+0000 7fa093fff700 1 -- 172.21.15.17:0/3568504335 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+1559 (secure 0 0 0) 0x7fa084045830 con 0x7fa09c0b1f40 2022-08-01T19:40:55.937 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:40:55.937 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":37,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5615,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/2284778179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2284778179},{"type":"v1","addr":"172.21.15.77:6835","nonce":2284778179}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":34},{"gid":6261,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/1827111044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":1827111044},{"type":"v1","addr":"172.21.15.17:6833","nonce":1827111044}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":33},{"gid":6540,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/441173941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":441173941},{"type":"v1","addr":"172.21.15.17:6835","nonce":441173941}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":37}],"filesystems":[]} 2022-08-01T19:40:55.940 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.927+0000 7fa0a0c7f700 1 -- 172.21.15.17:0/3568504335 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa080053b30 msgr2=0x7fa080055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:55.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.927+0000 7fa0a0c7f700 1 --2- 172.21.15.17:0/3568504335 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa080053b30 0x7fa080055ff0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7fa08c01c000 tx=0x7fa08c015000).stop 2022-08-01T19:40:55.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.927+0000 7fa0a0c7f700 1 -- 172.21.15.17:0/3568504335 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa09c0b1f40 msgr2=0x7fa09c0abbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:55.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.927+0000 7fa0a0c7f700 1 --2- 172.21.15.17:0/3568504335 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa09c0b1f40 0x7fa09c0abbe0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7fa09c0b3300 tx=0x7fa084007bd0).stop 2022-08-01T19:40:55.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.927+0000 7fa0a0c7f700 1 -- 172.21.15.17:0/3568504335 shutdown_connections 2022-08-01T19:40:55.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.927+0000 7fa0a0c7f700 1 --2- 172.21.15.17:0/3568504335 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa080053b30 0x7fa080055ff0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:55.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.927+0000 7fa0a0c7f700 1 --2- 172.21.15.17:0/3568504335 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa09c148530 0x7fa09c0ac0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:55.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.927+0000 7fa0a0c7f700 1 --2- 172.21.15.17:0/3568504335 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa09c0b1f40 0x7fa09c0abbe0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:55.944 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.927+0000 7fa0a0c7f700 1 --2- 172.21.15.17:0/3568504335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa09c0a7250 0x7fa09c14d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:55.944 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.927+0000 7fa0a0c7f700 1 -- 172.21.15.17:0/3568504335 >> 172.21.15.17:0/3568504335 conn(0x7fa09c05f840 msgr2=0x7fa09c144ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:55.944 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.927+0000 7fa0a0c7f700 1 -- 172.21.15.17:0/3568504335 shutdown_connections 2022-08-01T19:40:55.945 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:55.927+0000 7fa0a0c7f700 1 -- 172.21.15.17:0/3568504335 wait complete. 2022-08-01T19:40:55.945 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 37 2022-08-01T19:40:55.950 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:40:56.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.047+0000 7f0553862700 1 Processor -- start 2022-08-01T19:40:56.063 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.047+0000 7f0553862700 1 -- start start 2022-08-01T19:40:56.064 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.047+0000 7f0553862700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f054c0b4550 0x7f054c0b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:56.064 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.047+0000 7f0553862700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f054c0b6af0 0x7f054c12bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:56.064 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.047+0000 7f0553862700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f054c144240 con 0x7f054c0b6710 2022-08-01T19:40:56.064 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.047+0000 7f0553862700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f054c143d00 con 0x7f054c0b6af0 2022-08-01T19:40:56.064 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.047+0000 7f0553862700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054c143fa0 con 0x7f054c0b4550 2022-08-01T19:40:56.065 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.047+0000 7f0551dff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f054c0b6af0 0x7f054c12bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:56.065 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.047+0000 7f0551dff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f054c0b6af0 0x7f054c12bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41122/0 (socket says 172.21.15.17:41122) 2022-08-01T19:40:56.065 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.047+0000 7f0550dfd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f054c0b4550 0x7f054c0b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:56.065 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.047+0000 7f0551dff700 1 -- 172.21.15.17:0/1739803774 learned_addr learned my addr 172.21.15.17:0/1739803774 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:56.066 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.047+0000 7f0551dff700 1 -- 172.21.15.17:0/1739803774 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f054c0b4550 msgr2=0x7f054c0b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:56.066 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.047+0000 7f0551dff700 1 --2- 172.21.15.17:0/1739803774 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f054c0b4550 0x7f054c0b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:56.066 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.047+0000 7f0551dff700 1 -- 172.21.15.17:0/1739803774 >> v1:172.21.15.77:6789/0 conn(0x7f054c0b6710 legacy=0x7f054c061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:56.066 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.047+0000 7f0551dff700 1 -- 172.21.15.17:0/1739803774 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054c061780 con 0x7f054c0b6af0 2022-08-01T19:40:56.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.047+0000 7f0551dff700 1 --2- 172.21.15.17:0/1739803774 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f054c0b6af0 0x7f054c12bb90 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7f053c002f70 tx=0x7f053c006eb0).ready entity=mon.0 client_cookie=8e3257bcdd757109 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:56.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.047+0000 7f054bfff700 1 -- 172.21.15.17:0/1739803774 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f053c00d570 con 0x7f054c0b6af0 2022-08-01T19:40:56.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.047+0000 7f054bfff700 1 -- 172.21.15.17:0/1739803774 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053c000e90 con 0x7f054c0b6af0 2022-08-01T19:40:56.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.047+0000 7f054bfff700 1 -- 172.21.15.17:0/1739803774 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f053c00d900 con 0x7f054c0b6af0 2022-08-01T19:40:56.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.047+0000 7f0553862700 1 -- 172.21.15.17:0/1739803774 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f054c0b6af0 msgr2=0x7f054c12bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:56.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.047+0000 7f0553862700 1 --2- 172.21.15.17:0/1739803774 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f054c0b6af0 0x7f054c12bb90 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7f053c002f70 tx=0x7f053c006eb0).stop 2022-08-01T19:40:56.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.051+0000 7f0553862700 1 -- 172.21.15.17:0/1739803774 shutdown_connections 2022-08-01T19:40:56.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.051+0000 7f0553862700 1 --2- 172.21.15.17:0/1739803774 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f054c0b6af0 0x7f054c12bb90 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:56.069 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.051+0000 7f0553862700 1 --2- 172.21.15.17:0/1739803774 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f054c0b4550 0x7f054c0b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:56.069 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.051+0000 7f0553862700 1 -- 172.21.15.17:0/1739803774 >> 172.21.15.17:0/1739803774 conn(0x7f054c05c620 msgr2=0x7f054c05ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:56.069 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.051+0000 7f0553862700 1 -- 172.21.15.17:0/1739803774 shutdown_connections 2022-08-01T19:40:56.069 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.051+0000 7f0553862700 1 -- 172.21.15.17:0/1739803774 wait complete. 2022-08-01T19:40:56.069 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.051+0000 7f0553862700 1 Processor -- start 2022-08-01T19:40:56.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.051+0000 7f0553862700 1 -- start start 2022-08-01T19:40:56.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.055+0000 7f0553862700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f054c0b4550 0x7f054c12aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:56.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.055+0000 7f0553862700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f054c0b6710 0x7f054c141df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:56.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.055+0000 7f0553862700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f054c0b6af0 0x7f054c0ae850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:56.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.055+0000 7f0553862700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c129de0 con 0x7f054c0b6710 2022-08-01T19:40:56.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.055+0000 7f0553862700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c12a7e0 con 0x7f054c0b6af0 2022-08-01T19:40:56.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.055+0000 7f0553862700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054c0b2e20 con 0x7f054c0b4550 2022-08-01T19:40:56.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.055+0000 7f0550dfd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f054c0b6710 0x7f054c141df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:56.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.055+0000 7f0550dfd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f054c0b6710 0x7f054c141df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41126/0 (socket says 172.21.15.17:41126) 2022-08-01T19:40:56.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.055+0000 7f0550dfd700 1 -- 172.21.15.17:0/1354912425 learned_addr learned my addr 172.21.15.17:0/1354912425 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:56.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.055+0000 7f05515fe700 1 --2- 172.21.15.17:0/1354912425 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f054c0b4550 0x7f054c12aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:56.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.055+0000 7f0551dff700 1 --2- 172.21.15.17:0/1354912425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f054c0b6af0 0x7f054c0ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:56.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.055+0000 7f0550dfd700 1 -- 172.21.15.17:0/1354912425 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f054c0b4550 msgr2=0x7f054c12aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:56.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.055+0000 7f0550dfd700 1 --2- 172.21.15.17:0/1354912425 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f054c0b4550 0x7f054c12aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:56.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.055+0000 7f0550dfd700 1 -- 172.21.15.17:0/1354912425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f054c0b6af0 msgr2=0x7f054c0ae850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:56.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.055+0000 7f0550dfd700 1 --2- 172.21.15.17:0/1354912425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f054c0b6af0 0x7f054c0ae850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:56.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.055+0000 7f0550dfd700 1 -- 172.21.15.17:0/1354912425 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f053c00b1e0 con 0x7f054c0b6710 2022-08-01T19:40:56.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.055+0000 7f0550dfd700 1 --2- 172.21.15.17:0/1354912425 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f054c0b6710 0x7f054c141df0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7f053400c000 tx=0x7f0534006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:56.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.055+0000 7f054a7fc700 1 -- 172.21.15.17:0/1354912425 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0534010780 con 0x7f054c0b6710 2022-08-01T19:40:56.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.055+0000 7f054a7fc700 1 -- 172.21.15.17:0/1354912425 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053400b3b0 con 0x7f054c0b6710 2022-08-01T19:40:56.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.055+0000 7f0553862700 1 -- 172.21.15.17:0/1354912425 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f054c142a20 con 0x7f054c0b6710 2022-08-01T19:40:56.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.055+0000 7f054a7fc700 1 -- 172.21.15.17:0/1354912425 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0534010ad0 con 0x7f054c0b6710 2022-08-01T19:40:56.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.055+0000 7f0553862700 1 -- 172.21.15.17:0/1354912425 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f054c12aa40 con 0x7f054c0b6710 2022-08-01T19:40:56.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.055+0000 7f0553862700 1 -- 172.21.15.17:0/1354912425 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f054c12abd0 con 0x7f054c0b6710 2022-08-01T19:40:56.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.059+0000 7f054a7fc700 1 -- 172.21.15.17:0/1354912425 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0534010c30 con 0x7f054c0b6710 2022-08-01T19:40:56.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.059+0000 7f054a7fc700 1 --2- 172.21.15.17:0/1354912425 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0530053b30 0x7f0530055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:56.076 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.059+0000 7f054a7fc700 1 -- 172.21.15.17:0/1354912425 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5729+0+0 (secure 0 0 0) 0x7f053406cc60 con 0x7f054c0b6710 2022-08-01T19:40:56.076 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.059+0000 7f05515fe700 1 --2- 172.21.15.17:0/1354912425 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0530053b30 0x7f0530055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:56.077 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.059+0000 7f05515fe700 1 --2- 172.21.15.17:0/1354912425 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0530053b30 0x7f0530055ff0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f054400d000 tx=0x7f0544007f10).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:56.077 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.063+0000 7f054a7fc700 1 -- 172.21.15.17:0/1354912425 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f05340430f0 con 0x7f054c0b6710 2022-08-01T19:40:56.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.227+0000 7f0553862700 1 -- 172.21.15.17:0/1354912425 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f054c061a20 con 0x7f054c0b6710 2022-08-01T19:40:56.234 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.227+0000 7f054a7fc700 1 -- 172.21.15.17:0/1354912425 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+1559 (secure 0 0 0) 0x7f0534046710 con 0x7f054c0b6710 2022-08-01T19:40:56.240 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:40:56.242 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":37,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5615,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/2284778179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2284778179},{"type":"v1","addr":"172.21.15.77:6835","nonce":2284778179}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":34},{"gid":6261,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/1827111044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":1827111044},{"type":"v1","addr":"172.21.15.17:6833","nonce":1827111044}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":33},{"gid":6540,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/441173941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":441173941},{"type":"v1","addr":"172.21.15.17:6835","nonce":441173941}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":37}],"filesystems":[]} 2022-08-01T19:40:56.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.227+0000 7f0553862700 1 -- 172.21.15.17:0/1354912425 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0530053b30 msgr2=0x7f0530055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:56.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.227+0000 7f0553862700 1 --2- 172.21.15.17:0/1354912425 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0530053b30 0x7f0530055ff0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f054400d000 tx=0x7f0544007f10).stop 2022-08-01T19:40:56.246 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.227+0000 7f0553862700 1 -- 172.21.15.17:0/1354912425 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f054c0b6710 msgr2=0x7f054c141df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:56.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.227+0000 7f0553862700 1 --2- 172.21.15.17:0/1354912425 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f054c0b6710 0x7f054c141df0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7f053400c000 tx=0x7f0534006df0).stop 2022-08-01T19:40:56.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.227+0000 7f0553862700 1 -- 172.21.15.17:0/1354912425 shutdown_connections 2022-08-01T19:40:56.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.227+0000 7f0553862700 1 --2- 172.21.15.17:0/1354912425 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0530053b30 0x7f0530055ff0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:56.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.227+0000 7f0553862700 1 --2- 172.21.15.17:0/1354912425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f054c0b6af0 0x7f054c0ae850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:56.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.227+0000 7f0553862700 1 --2- 172.21.15.17:0/1354912425 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f054c0b6710 0x7f054c141df0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:56.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.227+0000 7f0553862700 1 --2- 172.21.15.17:0/1354912425 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f054c0b4550 0x7f054c12aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:56.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.227+0000 7f0553862700 1 -- 172.21.15.17:0/1354912425 >> 172.21.15.17:0/1354912425 conn(0x7f054c05c620 msgr2=0x7f054c128f20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:56.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.227+0000 7f0553862700 1 -- 172.21.15.17:0/1354912425 shutdown_connections 2022-08-01T19:40:56.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.227+0000 7f0553862700 1 -- 172.21.15.17:0/1354912425 wait complete. 2022-08-01T19:40:56.252 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 37 2022-08-01T19:40:56.253 INFO:tasks.ceph.mds.a:Restarting daemon 2022-08-01T19:40:56.254 INFO:tasks.ceph.mds.a:Stopping old one... 2022-08-01T19:40:56.255 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-08-01T19:40:56.255 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:40:56.256 INFO:tasks.ceph.mds.c:Restarting daemon 2022-08-01T19:40:56.256 INFO:tasks.ceph.mds.c:Stopping old one... 2022-08-01T19:40:56.256 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-08-01T19:40:56.257 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:40:56.257 INFO:tasks.ceph.mds.b:Restarting daemon 2022-08-01T19:40:56.257 INFO:tasks.ceph.mds.b:Stopping old one... 2022-08-01T19:40:56.258 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-08-01T19:40:56.265 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:40:56.274 INFO:tasks.ceph.mds.c:Stopped 2022-08-01T19:40:56.275 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-08-01T19:40:56.279 INFO:tasks.ceph.mds.a:Stopped 2022-08-01T19:40:56.280 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-08-01T19:40:56.285 INFO:tasks.ceph.mds.c:Started 2022-08-01T19:40:56.286 DEBUG:teuthology.parallel:result is None 2022-08-01T19:40:56.287 INFO:tasks.ceph.mds.b:Stopped 2022-08-01T19:40:56.288 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-08-01T19:40:56.291 INFO:tasks.ceph.mds.a:Started 2022-08-01T19:40:56.292 DEBUG:teuthology.parallel:result is None 2022-08-01T19:40:56.294 INFO:tasks.ceph.mds.b:Started 2022-08-01T19:40:56.296 DEBUG:teuthology.parallel:result is None 2022-08-01T19:40:56.300 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-08-01T19:40:56.351 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2022-08-01T19:40:56.371 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2022-08-01T19:40:56.373 INFO:tasks.ceph.mds.b.smithi077.stdout:starting mds.b at 2022-08-01T19:40:56.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.411+0000 7efeb0f2d700 1 Processor -- start 2022-08-01T19:40:56.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.411+0000 7efeb0f2d700 1 -- start start 2022-08-01T19:40:56.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.415+0000 7efeb0f2d700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7efeac0b1f70 0x7efeac144240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:56.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.415+0000 7efeb0f2d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efeac05c620 0x7efeac0ad1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:56.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.415+0000 7efeb0f2d700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeac0b2410 con 0x7efeac0b27e0 2022-08-01T19:40:56.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.415+0000 7efeb0f2d700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeac13dfd0 con 0x7efeac05c620 2022-08-01T19:40:56.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.415+0000 7efeb0f2d700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeac143860 con 0x7efeac0b1f70 2022-08-01T19:40:56.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.415+0000 7efeaa59c700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7efeac0b27e0 0x7efeac140270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33350/0 (socket says 172.21.15.17:33350) 2022-08-01T19:40:56.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.415+0000 7efeaa59c700 1 -- 172.21.15.17:0/4224941929 learned_addr learned my addr 172.21.15.17:0/4224941929 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:40:56.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.415+0000 7efeaad9d700 1 --2- 172.21.15.17:0/4224941929 >> v2:172.21.15.17:3300/0 conn(0x7efeac05c620 0x7efeac0ad1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:56.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.415+0000 7efea9d9b700 1 --2- 172.21.15.17:0/4224941929 >> v2:172.21.15.77:3300/0 conn(0x7efeac0b1f70 0x7efeac144240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:56.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.415+0000 7efea959a700 1 -- 172.21.15.17:0/4224941929 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4266193908 0 0) 0x7efeac0b2410 con 0x7efeac0b27e0 2022-08-01T19:40:56.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.415+0000 7efea959a700 1 -- 172.21.15.17:0/4224941929 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe8c003260 con 0x7efeac0b27e0 2022-08-01T19:40:56.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.415+0000 7efeaad9d700 1 -- 172.21.15.17:0/4224941929 >> v2:172.21.15.77:3300/0 conn(0x7efeac0b1f70 msgr2=0x7efeac144240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:56.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.415+0000 7efeaad9d700 1 --2- 172.21.15.17:0/4224941929 >> v2:172.21.15.77:3300/0 conn(0x7efeac0b1f70 0x7efeac144240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:56.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.415+0000 7efeaad9d700 1 -- 172.21.15.17:0/4224941929 >> v1:172.21.15.17:6789/0 conn(0x7efeac0b27e0 legacy=0x7efeac140270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:56.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.415+0000 7efeaad9d700 1 -- 172.21.15.17:0/4224941929 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeac143b00 con 0x7efeac05c620 2022-08-01T19:40:56.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.415+0000 7efea9d9b700 1 --2- 172.21.15.17:0/4224941929 >> v2:172.21.15.77:3300/0 conn(0x7efeac0b1f70 0x7efeac144240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:40:56.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.415+0000 7efeaad9d700 1 --2- 172.21.15.17:0/4224941929 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeac05c620 0x7efeac0ad1f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7efea40032f0 tx=0x7efea40091d0).ready entity=mon.0 client_cookie=2c7a20f05a2947f1 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:56.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.415+0000 7efea959a700 1 -- 172.21.15.17:0/4224941929 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efea400b600 con 0x7efeac05c620 2022-08-01T19:40:56.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.415+0000 7efea959a700 1 -- 172.21.15.17:0/4224941929 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea400b760 con 0x7efeac05c620 2022-08-01T19:40:56.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.415+0000 7efea959a700 1 -- 172.21.15.17:0/4224941929 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efea400b8d0 con 0x7efeac05c620 2022-08-01T19:40:56.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.415+0000 7efeb0f2d700 1 -- 172.21.15.17:0/4224941929 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeac05c620 msgr2=0x7efeac0ad1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:56.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.415+0000 7efeb0f2d700 1 --2- 172.21.15.17:0/4224941929 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeac05c620 0x7efeac0ad1f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7efea40032f0 tx=0x7efea40091d0).stop 2022-08-01T19:40:56.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.415+0000 7efeb0f2d700 1 -- 172.21.15.17:0/4224941929 shutdown_connections 2022-08-01T19:40:56.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.415+0000 7efeb0f2d700 1 --2- 172.21.15.17:0/4224941929 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeac05c620 0x7efeac0ad1f0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:56.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.415+0000 7efeb0f2d700 1 --2- 172.21.15.17:0/4224941929 >> v2:172.21.15.77:3300/0 conn(0x7efeac0b1f70 0x7efeac144240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:56.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.415+0000 7efeb0f2d700 1 -- 172.21.15.17:0/4224941929 >> 172.21.15.17:0/4224941929 conn(0x7efeac05ceb0 msgr2=0x7efeac063130 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:56.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.415+0000 7efeb0f2d700 1 -- 172.21.15.17:0/4224941929 shutdown_connections 2022-08-01T19:40:56.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.415+0000 7efeb0f2d700 1 -- 172.21.15.17:0/4224941929 wait complete. 2022-08-01T19:40:56.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.419+0000 7efeb0f2d700 1 Processor -- start 2022-08-01T19:40:56.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.419+0000 7efeb0f2d700 1 -- start start 2022-08-01T19:40:56.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.419+0000 7efeb0f2d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efeac05c620 0x7efeac142d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:56.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.419+0000 7efeb0f2d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efeac0b1f70 0x7efeac14e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:56.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.419+0000 7efeb0f2d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeac0b27e0 0x7efeac14ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:56.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.419+0000 7efeb0f2d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac0b5b00 con 0x7efeac0b27e0 2022-08-01T19:40:56.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.419+0000 7efeb0f2d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac13e110 con 0x7efeac05c620 2022-08-01T19:40:56.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.419+0000 7efeb0f2d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeac14e530 con 0x7efeac0b1f70 2022-08-01T19:40:56.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.419+0000 7efeaad9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeac0b27e0 0x7efeac14ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:56.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.419+0000 7efeaad9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeac0b27e0 0x7efeac14ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41166/0 (socket says 172.21.15.17:41166) 2022-08-01T19:40:56.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.419+0000 7efeaad9d700 1 -- 172.21.15.17:0/441719462 learned_addr learned my addr 172.21.15.17:0/441719462 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:56.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.419+0000 7efeaa59c700 1 --2- 172.21.15.17:0/441719462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efeac05c620 0x7efeac142d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:56.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.419+0000 7efea9d9b700 1 --2- 172.21.15.17:0/441719462 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efeac0b1f70 0x7efeac14e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:56.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.419+0000 7efeaad9d700 1 -- 172.21.15.17:0/441719462 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efeac0b1f70 msgr2=0x7efeac14e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:56.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.419+0000 7efeaad9d700 1 --2- 172.21.15.17:0/441719462 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efeac0b1f70 0x7efeac14e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:56.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.419+0000 7efeaad9d700 1 -- 172.21.15.17:0/441719462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efeac05c620 msgr2=0x7efeac142d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:56.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.419+0000 7efeaad9d700 1 --2- 172.21.15.17:0/441719462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efeac05c620 0x7efeac142d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:56.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.419+0000 7efeaad9d700 1 -- 172.21.15.17:0/441719462 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe9400b180 con 0x7efeac0b27e0 2022-08-01T19:40:56.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.419+0000 7efeaad9d700 1 --2- 172.21.15.17:0/441719462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeac0b27e0 0x7efeac14ee50 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x7efea40032f0 tx=0x7efea4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:56.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.419+0000 7efea37fe700 1 -- 172.21.15.17:0/441719462 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efea4011b60 con 0x7efeac0b27e0 2022-08-01T19:40:56.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.419+0000 7efea37fe700 1 -- 172.21.15.17:0/441719462 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea4011cc0 con 0x7efeac0b27e0 2022-08-01T19:40:56.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.419+0000 7efeb0f2d700 1 -- 172.21.15.17:0/441719462 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efea4009470 con 0x7efeac0b27e0 2022-08-01T19:40:56.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.419+0000 7efea37fe700 1 -- 172.21.15.17:0/441719462 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efea401f5e0 con 0x7efeac0b27e0 2022-08-01T19:40:56.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.419+0000 7efeb0f2d700 1 -- 172.21.15.17:0/441719462 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeac13e340 con 0x7efeac0b27e0 2022-08-01T19:40:56.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.423+0000 7efea37fe700 1 -- 172.21.15.17:0/441719462 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7efea4026030 con 0x7efeac0b27e0 2022-08-01T19:40:56.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.423+0000 7efeb0f2d700 1 -- 172.21.15.17:0/441719462 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeac14e6a0 con 0x7efeac0b27e0 2022-08-01T19:40:56.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.423+0000 7efea37fe700 1 --2- 172.21.15.17:0/441719462 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efe90053950 0x7efe90055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:56.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.423+0000 7efea37fe700 1 -- 172.21.15.17:0/441719462 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5729+0+0 (secure 0 0 0) 0x7efea406c850 con 0x7efeac0b27e0 2022-08-01T19:40:56.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.423+0000 7efeaa59c700 1 --2- 172.21.15.17:0/441719462 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efe90053950 0x7efe90055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:56.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.423+0000 7efeaa59c700 1 --2- 172.21.15.17:0/441719462 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efe90053950 0x7efe90055e10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7efe9c00d000 tx=0x7efe9c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:56.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.427+0000 7efea37fe700 1 -- 172.21.15.17:0/441719462 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efeac14e6a0 con 0x7efeac0b27e0 2022-08-01T19:40:56.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:56.563+0000 7efeb0f2d700 1 -- 172.21.15.17:0/441719462 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7efeac14e6a0 con 0x7efeac0b27e0 2022-08-01T19:40:57.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.415+0000 7efea37fe700 1 -- 172.21.15.17:0/441719462 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v84) v1 ==== 97+0+0 (secure 0 0 0) 0x7efeac14e6a0 con 0x7efeac0b27e0 2022-08-01T19:40:57.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.419+0000 7efeb0f2d700 1 -- 172.21.15.17:0/441719462 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efe90053950 msgr2=0x7efe90055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:57.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.419+0000 7efeb0f2d700 1 --2- 172.21.15.17:0/441719462 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efe90053950 0x7efe90055e10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7efe9c00d000 tx=0x7efe9c009040).stop 2022-08-01T19:40:57.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.419+0000 7efeb0f2d700 1 -- 172.21.15.17:0/441719462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeac0b27e0 msgr2=0x7efeac14ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:57.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.419+0000 7efeb0f2d700 1 --2- 172.21.15.17:0/441719462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeac0b27e0 0x7efeac14ee50 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x7efea40032f0 tx=0x7efea4016000).stop 2022-08-01T19:40:57.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.419+0000 7efeb0f2d700 1 -- 172.21.15.17:0/441719462 shutdown_connections 2022-08-01T19:40:57.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.419+0000 7efeb0f2d700 1 --2- 172.21.15.17:0/441719462 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efe90053950 0x7efe90055e10 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:57.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.419+0000 7efeb0f2d700 1 --2- 172.21.15.17:0/441719462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeac0b27e0 0x7efeac14ee50 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:57.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.419+0000 7efeb0f2d700 1 --2- 172.21.15.17:0/441719462 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efeac0b1f70 0x7efeac14e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:57.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.419+0000 7efeb0f2d700 1 --2- 172.21.15.17:0/441719462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efeac05c620 0x7efeac142d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:57.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.419+0000 7efeb0f2d700 1 -- 172.21.15.17:0/441719462 >> 172.21.15.17:0/441719462 conn(0x7efeac05ceb0 msgr2=0x7efeac13add0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:57.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.423+0000 7efeb0f2d700 1 -- 172.21.15.17:0/441719462 shutdown_connections 2022-08-01T19:40:57.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.423+0000 7efeb0f2d700 1 -- 172.21.15.17:0/441719462 wait complete. 2022-08-01T19:40:57.434 INFO:teuthology.orchestra.run.smithi017.stderr: removed all blacklist entries 2022-08-01T19:40:57.449 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-08-01T19:40:57.450 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-08-01T19:40:57.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.591+0000 7f0e6082d700 1 Processor -- start 2022-08-01T19:40:57.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.591+0000 7f0e6082d700 1 -- start start 2022-08-01T19:40:57.602 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.591+0000 7f0e6082d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0e5c0b7500 0x7f0e5c0b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:57.602 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.591+0000 7f0e6082d700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f0e5c0b31f0 0x7f0e5c13e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:57.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.591+0000 7f0e6082d700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e5c0b2510 con 0x7f0e5c13e8a0 2022-08-01T19:40:57.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.591+0000 7f0e6082d700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e5c1443e0 con 0x7f0e5c0b7500 2022-08-01T19:40:57.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.591+0000 7f0e6082d700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e5c0b3700 con 0x7f0e5c0b31f0 2022-08-01T19:40:57.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.591+0000 7f0e5b7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0e5c0b7500 0x7f0e5c0b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:57.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.591+0000 7f0e5bfff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f0e5c13e8a0 0x7f0e5c064d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33372/0 (socket says 172.21.15.17:33372) 2022-08-01T19:40:57.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.591+0000 7f0e5bfff700 1 -- 172.21.15.17:0/106480252 learned_addr learned my addr 172.21.15.17:0/106480252 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:40:57.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.591+0000 7f0e5affd700 1 --2- 172.21.15.17:0/106480252 >> v2:172.21.15.77:3300/0 conn(0x7f0e5c0b31f0 0x7f0e5c13e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:57.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.591+0000 7f0e5b7fe700 1 -- 172.21.15.17:0/106480252 >> v2:172.21.15.77:3300/0 conn(0x7f0e5c0b31f0 msgr2=0x7f0e5c13e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:57.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.591+0000 7f0e5b7fe700 1 --2- 172.21.15.17:0/106480252 >> v2:172.21.15.77:3300/0 conn(0x7f0e5c0b31f0 0x7f0e5c13e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:57.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.591+0000 7f0e5b7fe700 1 -- 172.21.15.17:0/106480252 >> v1:172.21.15.17:6789/0 conn(0x7f0e5c13e8a0 legacy=0x7f0e5c064d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:57.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.591+0000 7f0e5a7fc700 1 -- 172.21.15.17:0/106480252 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 929371775 0 0) 0x7f0e5c0b2510 con 0x7f0e5c13e8a0 2022-08-01T19:40:57.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.591+0000 7f0e5b7fe700 1 -- 172.21.15.17:0/106480252 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e5c0b2740 con 0x7f0e5c0b7500 2022-08-01T19:40:57.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.591+0000 7f0e5b7fe700 1 --2- 172.21.15.17:0/106480252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e5c0b7500 0x7f0e5c0b78e0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7f0e44002f70 tx=0x7f0e44006d90).ready entity=mon.0 client_cookie=c90f05074ded67bf server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:57.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.595+0000 7f0e5a7fc700 1 -- 172.21.15.17:0/106480252 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e4400d4a0 con 0x7f0e5c0b7500 2022-08-01T19:40:57.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.595+0000 7f0e5a7fc700 1 -- 172.21.15.17:0/106480252 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e44000e90 con 0x7f0e5c0b7500 2022-08-01T19:40:57.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.595+0000 7f0e5a7fc700 1 -- 172.21.15.17:0/106480252 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e4400d7f0 con 0x7f0e5c0b7500 2022-08-01T19:40:57.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.595+0000 7f0e6082d700 1 -- 172.21.15.17:0/106480252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e5c0b7500 msgr2=0x7f0e5c0b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:57.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.595+0000 7f0e6082d700 1 --2- 172.21.15.17:0/106480252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e5c0b7500 0x7f0e5c0b78e0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7f0e44002f70 tx=0x7f0e44006d90).stop 2022-08-01T19:40:57.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.595+0000 7f0e6082d700 1 -- 172.21.15.17:0/106480252 shutdown_connections 2022-08-01T19:40:57.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.595+0000 7f0e6082d700 1 --2- 172.21.15.17:0/106480252 >> v2:172.21.15.77:3300/0 conn(0x7f0e5c0b31f0 0x7f0e5c13e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:57.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.595+0000 7f0e6082d700 1 --2- 172.21.15.17:0/106480252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e5c0b7500 0x7f0e5c0b78e0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:57.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.595+0000 7f0e6082d700 1 -- 172.21.15.17:0/106480252 >> 172.21.15.17:0/106480252 conn(0x7f0e5c05f840 msgr2=0x7f0e5c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:57.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.595+0000 7f0e6082d700 1 -- 172.21.15.17:0/106480252 shutdown_connections 2022-08-01T19:40:57.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.595+0000 7f0e6082d700 1 -- 172.21.15.17:0/106480252 wait complete. 2022-08-01T19:40:57.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.595+0000 7f0e6082d700 1 Processor -- start 2022-08-01T19:40:57.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.595+0000 7f0e6082d700 1 -- start start 2022-08-01T19:40:57.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.595+0000 7f0e6082d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e5c0b31f0 0x7f0e5c0aec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:57.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.595+0000 7f0e6082d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0e5c0b7500 0x7f0e5c13d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:57.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.595+0000 7f0e6082d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e5c13e8a0 0x7f0e5c13dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:57.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.595+0000 7f0e6082d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5c0acf60 con 0x7f0e5c13e8a0 2022-08-01T19:40:57.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.595+0000 7f0e6082d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5c0ac250 con 0x7f0e5c0b31f0 2022-08-01T19:40:57.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.595+0000 7f0e6082d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5c0ad1c0 con 0x7f0e5c0b7500 2022-08-01T19:40:57.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.599+0000 7f0e5bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e5c13e8a0 0x7f0e5c13dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:57.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.599+0000 7f0e5bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e5c13e8a0 0x7f0e5c13dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41184/0 (socket says 172.21.15.17:41184) 2022-08-01T19:40:57.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.599+0000 7f0e5bfff700 1 -- 172.21.15.17:0/2929732505 learned_addr learned my addr 172.21.15.17:0/2929732505 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:57.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.599+0000 7f0e5b7fe700 1 --2- 172.21.15.17:0/2929732505 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e5c0b31f0 0x7f0e5c0aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:57.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.599+0000 7f0e5affd700 1 --2- 172.21.15.17:0/2929732505 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0e5c0b7500 0x7f0e5c13d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:57.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.599+0000 7f0e5bfff700 1 -- 172.21.15.17:0/2929732505 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0e5c0b7500 msgr2=0x7f0e5c13d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:57.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.599+0000 7f0e5bfff700 1 --2- 172.21.15.17:0/2929732505 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0e5c0b7500 0x7f0e5c13d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:57.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.599+0000 7f0e5bfff700 1 -- 172.21.15.17:0/2929732505 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e5c0b31f0 msgr2=0x7f0e5c0aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:57.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.599+0000 7f0e5bfff700 1 --2- 172.21.15.17:0/2929732505 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e5c0b31f0 0x7f0e5c0aec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:57.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.599+0000 7f0e5bfff700 1 -- 172.21.15.17:0/2929732505 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e4400b120 con 0x7f0e5c13e8a0 2022-08-01T19:40:57.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.599+0000 7f0e5bfff700 1 --2- 172.21.15.17:0/2929732505 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e5c13e8a0 0x7f0e5c13dc80 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7f0e54001250 tx=0x7f0e54001360).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:57.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.599+0000 7f0e58ff9700 1 -- 172.21.15.17:0/2929732505 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e54005c70 con 0x7f0e5c13e8a0 2022-08-01T19:40:57.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.599+0000 7f0e58ff9700 1 -- 172.21.15.17:0/2929732505 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e54005dd0 con 0x7f0e5c13e8a0 2022-08-01T19:40:57.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.599+0000 7f0e6082d700 1 -- 172.21.15.17:0/2929732505 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e5c13e170 con 0x7f0e5c13e8a0 2022-08-01T19:40:57.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.599+0000 7f0e58ff9700 1 -- 172.21.15.17:0/2929732505 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e54004a90 con 0x7f0e5c13e8a0 2022-08-01T19:40:57.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.599+0000 7f0e6082d700 1 -- 172.21.15.17:0/2929732505 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e5c0abed0 con 0x7f0e5c13e8a0 2022-08-01T19:40:57.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.599+0000 7f0e6082d700 1 -- 172.21.15.17:0/2929732505 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e5c0af430 con 0x7f0e5c13e8a0 2022-08-01T19:40:57.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.599+0000 7f0e58ff9700 1 -- 172.21.15.17:0/2929732505 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0e54004bf0 con 0x7f0e5c13e8a0 2022-08-01T19:40:57.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.603+0000 7f0e58ff9700 1 --2- 172.21.15.17:0/2929732505 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0e40053950 0x7f0e40055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:57.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.603+0000 7f0e58ff9700 1 -- 172.21.15.17:0/2929732505 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f0e5406b960 con 0x7f0e5c13e8a0 2022-08-01T19:40:57.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.603+0000 7f0e5b7fe700 1 --2- 172.21.15.17:0/2929732505 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0e40053950 0x7f0e40055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:57.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.603+0000 7f0e5b7fe700 1 --2- 172.21.15.17:0/2929732505 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0e40053950 0x7f0e40055e10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f0e4401a000 tx=0x7f0e44014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:57.636 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.607+0000 7f0e58ff9700 1 -- 172.21.15.17:0/2929732505 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0e54042a60 con 0x7f0e5c13e8a0 2022-08-01T19:40:57.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.775+0000 7f0e6082d700 1 -- 172.21.15.17:0/2929732505 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0e5c0ac3c0 con 0x7f0e5c13e8a0 2022-08-01T19:40:57.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.775+0000 7f0e58ff9700 1 -- 172.21.15.17:0/2929732505 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v24) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f0e5c0ac3c0 con 0x7f0e5c13e8a0 2022-08-01T19:40:57.780 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:40:57.781 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:40:57.781 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_dump": [ 2022-08-01T19:40:57.782 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:57.782 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.a", 2022-08-01T19:40:57.782 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhijotxORAAVc04rsrlFj0NShsyc6Qbhg==", 2022-08-01T19:40:57.783 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:40:57.783 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:40:57.783 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:40:57.783 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:40:57.784 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:40:57.784 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:57.784 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:57.784 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:57.784 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.b", 2022-08-01T19:40:57.785 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhiud2xBxAAcn6/O5ysdIp/oIbF5Y1dAw==", 2022-08-01T19:40:57.785 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:40:57.785 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:40:57.785 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:40:57.786 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:40:57.786 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:40:57.787 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:57.787 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:57.787 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:57.787 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.c", 2022-08-01T19:40:57.788 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+KuhiMzMbBBAAfdT5mtpYwboNL+7f2B09KQ==", 2022-08-01T19:40:57.788 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:40:57.788 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:40:57.789 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:40:57.789 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:40:57.789 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:40:57.789 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:57.791 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:57.791 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:57.792 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.0", 2022-08-01T19:40:57.792 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhia27/NRAAZXBp+nYelJVJ6Mge9r+3Vg==", 2022-08-01T19:40:57.792 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:40:57.792 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:40:57.793 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:40:57.793 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:40:57.793 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:57.793 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:57.794 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:57.794 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.1", 2022-08-01T19:40:57.794 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCBKuhiOggRGRAAvgA0MxwBJwmpXV6S9DTdjA==", 2022-08-01T19:40:57.795 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:40:57.795 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:40:57.795 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:40:57.796 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:40:57.796 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:57.796 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:57.796 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:57.797 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.2", 2022-08-01T19:40:57.797 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCDKuhi5PgOORAAhcORK4bW2T3dXPeYjcCm+g==", 2022-08-01T19:40:57.797 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:40:57.797 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:40:57.797 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:40:57.798 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:40:57.798 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:57.798 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:57.798 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:57.799 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.3", 2022-08-01T19:40:57.799 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCGKuhiutn6GxAAk2ucLuGbT5YV/R4xXQGh0w==", 2022-08-01T19:40:57.799 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:40:57.799 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:40:57.800 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:40:57.800 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:40:57.800 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:57.800 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:57.801 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:57.801 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.4", 2022-08-01T19:40:57.801 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCLKuhiGHTRHxAArr3TwXA0gsCAAF/Qs38wUQ==", 2022-08-01T19:40:57.801 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:40:57.801 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:40:57.802 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:40:57.802 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:40:57.803 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:57.803 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:57.803 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:57.803 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.5", 2022-08-01T19:40:57.804 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCOKuhiMMDbBBAAvTzBvOcv7pFWm/P/lNqB0w==", 2022-08-01T19:40:57.804 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:40:57.804 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:40:57.805 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:40:57.805 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:40:57.805 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:57.805 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:57.806 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:57.806 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.6", 2022-08-01T19:40:57.806 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCQKuhiG2H3IxAAufUcWqo9ULquxIrUD5vSfA==", 2022-08-01T19:40:57.807 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:40:57.807 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:40:57.807 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:40:57.808 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:40:57.808 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:57.809 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:57.809 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:57.809 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.7", 2022-08-01T19:40:57.810 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCTKuhiU50LChAAXuhrYBbHkk3eKzAPiEJFTg==", 2022-08-01T19:40:57.810 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:40:57.811 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:40:57.811 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:40:57.811 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:40:57.812 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:57.813 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:57.813 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:57.814 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.0", 2022-08-01T19:40:57.814 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhig3EnCxAANM20jSIQf/AKwuRX1qwT/A==", 2022-08-01T19:40:57.815 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:40:57.815 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:40:57.815 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow r", 2022-08-01T19:40:57.816 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T19:40:57.816 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:57.817 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:57.817 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:57.821 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.1", 2022-08-01T19:40:57.822 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+KuhicJR0DhAAivqEAZUhXnSwjVRwWm2bJg==", 2022-08-01T19:40:57.822 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:40:57.822 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:40:57.822 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow r", 2022-08-01T19:40:57.823 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T19:40:57.823 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:57.823 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:57.823 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:57.824 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.admin", 2022-08-01T19:40:57.824 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhiPTTGHxAAgfthCR6RX6j8F1YseFGNeA==", 2022-08-01T19:40:57.824 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:40:57.824 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:40:57.825 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:40:57.825 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:40:57.825 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:40:57.826 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:57.826 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:57.826 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:57.826 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.x", 2022-08-01T19:40:57.827 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9Kuhi2jvCNhAAyUxiGByZ+OZr8Y7dh3Qt5w==", 2022-08-01T19:40:57.828 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:40:57.828 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:40:57.828 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2022-08-01T19:40:57.829 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:40:57.829 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:57.829 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:57.830 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:57.830 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.y", 2022-08-01T19:40:57.830 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhiDJXpMhAA8aek67ydbRgNO0h+miws+w==", 2022-08-01T19:40:57.831 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:40:57.831 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:40:57.831 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2022-08-01T19:40:57.832 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:40:57.832 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:57.832 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:57.833 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:40:57.833 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:40:57.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.779+0000 7f0e6082d700 1 -- 172.21.15.17:0/2929732505 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0e40053950 msgr2=0x7f0e40055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:57.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.779+0000 7f0e6082d700 1 --2- 172.21.15.17:0/2929732505 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0e40053950 0x7f0e40055e10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f0e4401a000 tx=0x7f0e44014000).stop 2022-08-01T19:40:57.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.779+0000 7f0e6082d700 1 -- 172.21.15.17:0/2929732505 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e5c13e8a0 msgr2=0x7f0e5c13dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:57.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.779+0000 7f0e6082d700 1 --2- 172.21.15.17:0/2929732505 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e5c13e8a0 0x7f0e5c13dc80 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7f0e54001250 tx=0x7f0e54001360).stop 2022-08-01T19:40:57.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.779+0000 7f0e6082d700 1 -- 172.21.15.17:0/2929732505 shutdown_connections 2022-08-01T19:40:57.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.779+0000 7f0e6082d700 1 --2- 172.21.15.17:0/2929732505 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0e40053950 0x7f0e40055e10 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:57.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.779+0000 7f0e6082d700 1 --2- 172.21.15.17:0/2929732505 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e5c13e8a0 0x7f0e5c13dc80 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:57.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.779+0000 7f0e6082d700 1 --2- 172.21.15.17:0/2929732505 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0e5c0b7500 0x7f0e5c13d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:57.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.779+0000 7f0e6082d700 1 --2- 172.21.15.17:0/2929732505 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e5c0b31f0 0x7f0e5c0aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:57.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.779+0000 7f0e6082d700 1 -- 172.21.15.17:0/2929732505 >> 172.21.15.17:0/2929732505 conn(0x7f0e5c05f840 msgr2=0x7f0e5c0b5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:57.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.779+0000 7f0e6082d700 1 -- 172.21.15.17:0/2929732505 shutdown_connections 2022-08-01T19:40:57.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.779+0000 7f0e6082d700 1 -- 172.21.15.17:0/2929732505 wait complete. 2022-08-01T19:40:57.841 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:40:57.949 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.943+0000 7ff37359e700 1 Processor -- start 2022-08-01T19:40:57.950 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.943+0000 7ff37359e700 1 -- start start 2022-08-01T19:40:57.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.947+0000 7ff37359e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff3740b08a0 0x7ff374142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:57.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.947+0000 7ff37359e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff3740b3260 0x7ff3741433a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:57.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.947+0000 7ff37359e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ff3740ad000 0x7ff37405f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:57.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.947+0000 7ff37359e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3740b0c80 con 0x7ff3740b3260 2022-08-01T19:40:57.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.947+0000 7ff37359e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3740ae810 con 0x7ff3740ad000 2022-08-01T19:40:57.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.947+0000 7ff37359e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3740aeab0 con 0x7ff3740b08a0 2022-08-01T19:40:57.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.947+0000 7ff36bfff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff3740b3260 0x7ff3741433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:57.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.947+0000 7ff36bfff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff3740b3260 0x7ff3741433a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41190/0 (socket says 172.21.15.17:41190) 2022-08-01T19:40:57.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.947+0000 7ff36bfff700 1 -- 172.21.15.17:0/1909167611 learned_addr learned my addr 172.21.15.17:0/1909167611 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:57.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.947+0000 7ff372d9d700 1 --2- 172.21.15.17:0/1909167611 >> v2:172.21.15.77:3300/0 conn(0x7ff3740ad000 0x7ff37405f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:57.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.947+0000 7ff37259c700 1 --2- 172.21.15.17:0/1909167611 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff3740b08a0 0x7ff374142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:57.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.947+0000 7ff36bfff700 1 -- 172.21.15.17:0/1909167611 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff3740b08a0 msgr2=0x7ff374142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:57.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.947+0000 7ff36bfff700 1 --2- 172.21.15.17:0/1909167611 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff3740b08a0 0x7ff374142eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:57.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.947+0000 7ff36bfff700 1 -- 172.21.15.17:0/1909167611 >> v2:172.21.15.77:3300/0 conn(0x7ff3740ad000 msgr2=0x7ff37405f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:57.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.947+0000 7ff36bfff700 1 --2- 172.21.15.17:0/1909167611 >> v2:172.21.15.77:3300/0 conn(0x7ff3740ad000 0x7ff37405f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:57.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.947+0000 7ff36bfff700 1 -- 172.21.15.17:0/1909167611 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3740aed50 con 0x7ff3740b3260 2022-08-01T19:40:57.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.947+0000 7ff36bfff700 1 --2- 172.21.15.17:0/1909167611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3740b3260 0x7ff3741433a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7ff35c002f70 tx=0x7ff35c006eb0).ready entity=mon.0 client_cookie=31f73dda761f6736 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:57.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.947+0000 7ff371d9b700 1 -- 172.21.15.17:0/1909167611 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff35c00f970 con 0x7ff3740b3260 2022-08-01T19:40:57.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.947+0000 7ff371d9b700 1 -- 172.21.15.17:0/1909167611 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff35c000e90 con 0x7ff3740b3260 2022-08-01T19:40:57.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.947+0000 7ff371d9b700 1 -- 172.21.15.17:0/1909167611 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff35c00fd00 con 0x7ff3740b3260 2022-08-01T19:40:57.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.947+0000 7ff37359e700 1 -- 172.21.15.17:0/1909167611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3740b3260 msgr2=0x7ff3741433a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:57.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.947+0000 7ff37359e700 1 --2- 172.21.15.17:0/1909167611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3740b3260 0x7ff3741433a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7ff35c002f70 tx=0x7ff35c006eb0).stop 2022-08-01T19:40:57.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.947+0000 7ff37359e700 1 -- 172.21.15.17:0/1909167611 shutdown_connections 2022-08-01T19:40:57.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.947+0000 7ff37359e700 1 --2- 172.21.15.17:0/1909167611 >> v2:172.21.15.77:3300/0 conn(0x7ff3740ad000 0x7ff37405f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:57.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.947+0000 7ff37359e700 1 --2- 172.21.15.17:0/1909167611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3740b3260 0x7ff3741433a0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:57.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.947+0000 7ff37359e700 1 --2- 172.21.15.17:0/1909167611 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff3740b08a0 0x7ff374142eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:57.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.947+0000 7ff37359e700 1 -- 172.21.15.17:0/1909167611 >> 172.21.15.17:0/1909167611 conn(0x7ff3740612d0 msgr2=0x7ff374062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:57.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.951+0000 7ff37359e700 1 -- 172.21.15.17:0/1909167611 shutdown_connections 2022-08-01T19:40:57.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.951+0000 7ff37359e700 1 -- 172.21.15.17:0/1909167611 wait complete. 2022-08-01T19:40:57.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.951+0000 7ff37359e700 1 Processor -- start 2022-08-01T19:40:57.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.951+0000 7ff37359e700 1 -- start start 2022-08-01T19:40:57.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.951+0000 7ff37359e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff3740ad000 0x7ff37413de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:57.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.951+0000 7ff37359e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3740b08a0 0x7ff374065c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:57.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.951+0000 7ff37359e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3740b3260 0x7ff374066120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:57.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.951+0000 7ff37359e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3740aabf0 con 0x7ff3740b3260 2022-08-01T19:40:57.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.951+0000 7ff37359e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3740aae20 con 0x7ff3740b08a0 2022-08-01T19:40:57.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.951+0000 7ff37359e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3740ab0c0 con 0x7ff3740ad000 2022-08-01T19:40:57.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.951+0000 7ff372d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3740b3260 0x7ff374066120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:57.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.951+0000 7ff36bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3740b08a0 0x7ff374065c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:57.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.951+0000 7ff372d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3740b3260 0x7ff374066120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41198/0 (socket says 172.21.15.17:41198) 2022-08-01T19:40:57.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.951+0000 7ff36bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3740b08a0 0x7ff374065c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:39118/0 (socket says 172.21.15.17:39118) 2022-08-01T19:40:57.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.951+0000 7ff37259c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff3740ad000 0x7ff37413de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:57.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.955+0000 7ff372d9d700 1 -- 172.21.15.17:0/2840618825 learned_addr learned my addr 172.21.15.17:0/2840618825 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:57.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.955+0000 7ff372d9d700 1 -- 172.21.15.17:0/2840618825 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff3740ad000 msgr2=0x7ff37413de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:57.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.955+0000 7ff372d9d700 1 --2- 172.21.15.17:0/2840618825 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff3740ad000 0x7ff37413de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:57.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.955+0000 7ff372d9d700 1 -- 172.21.15.17:0/2840618825 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3740b08a0 msgr2=0x7ff374065c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:57.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.955+0000 7ff372d9d700 1 --2- 172.21.15.17:0/2840618825 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3740b08a0 0x7ff374065c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:57.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.955+0000 7ff372d9d700 1 -- 172.21.15.17:0/2840618825 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff35c00b1e0 con 0x7ff3740b3260 2022-08-01T19:40:57.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.955+0000 7ff36bfff700 1 --2- 172.21.15.17:0/2840618825 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3740b08a0 0x7ff374065c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:40:57.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.955+0000 7ff372d9d700 1 --2- 172.21.15.17:0/2840618825 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3740b3260 0x7ff374066120 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7ff36c00c000 tx=0x7ff36c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:57.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.955+0000 7ff36b7fe700 1 -- 172.21.15.17:0/2840618825 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff36c00c670 con 0x7ff3740b3260 2022-08-01T19:40:57.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.955+0000 7ff36b7fe700 1 -- 172.21.15.17:0/2840618825 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff36c00c7d0 con 0x7ff3740b3260 2022-08-01T19:40:57.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.955+0000 7ff37359e700 1 -- 172.21.15.17:0/2840618825 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff37413e800 con 0x7ff3740b3260 2022-08-01T19:40:57.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.955+0000 7ff36b7fe700 1 -- 172.21.15.17:0/2840618825 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff36c00cb30 con 0x7ff3740b3260 2022-08-01T19:40:57.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.955+0000 7ff37359e700 1 -- 172.21.15.17:0/2840618825 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3740ab5e0 con 0x7ff3740b3260 2022-08-01T19:40:57.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.955+0000 7ff37359e700 1 -- 172.21.15.17:0/2840618825 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff374065a00 con 0x7ff3740b3260 2022-08-01T19:40:57.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.959+0000 7ff36b7fe700 1 -- 172.21.15.17:0/2840618825 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff36c022070 con 0x7ff3740b3260 2022-08-01T19:40:57.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.959+0000 7ff36b7fe700 1 --2- 172.21.15.17:0/2840618825 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff3580539e0 0x7ff358055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:57.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.959+0000 7ff36b7fe700 1 -- 172.21.15.17:0/2840618825 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5600+0+0 (secure 0 0 0) 0x7ff36c06c440 con 0x7ff3740b3260 2022-08-01T19:40:57.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.959+0000 7ff37259c700 1 --2- 172.21.15.17:0/2840618825 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff3580539e0 0x7ff358055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:57.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.959+0000 7ff37259c700 1 --2- 172.21.15.17:0/2840618825 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff3580539e0 0x7ff358055ea0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7ff36400b000 tx=0x7ff364006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:57.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:57.967+0000 7ff36b7fe700 1 -- 172.21.15.17:0/2840618825 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff36c043430 con 0x7ff3740b3260 2022-08-01T19:40:58.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.091+0000 7ff37359e700 1 -- 172.21.15.17:0/2840618825 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3740574e0 con 0x7ff3740b3260 2022-08-01T19:40:58.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.095+0000 7ff36b7fe700 1 -- 172.21.15.17:0/2840618825 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+13415 (secure 0 0 0) 0x7ff36c016020 con 0x7ff3740b3260 2022-08-01T19:40:58.099 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:40:58.100 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":84,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:40:57.415027+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:40:58.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.095+0000 7ff37359e700 1 -- 172.21.15.17:0/2840618825 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff3580539e0 msgr2=0x7ff358055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:58.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.095+0000 7ff37359e700 1 --2- 172.21.15.17:0/2840618825 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff3580539e0 0x7ff358055ea0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7ff36400b000 tx=0x7ff364006df0).stop 2022-08-01T19:40:58.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.095+0000 7ff37359e700 1 -- 172.21.15.17:0/2840618825 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3740b3260 msgr2=0x7ff374066120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:58.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.095+0000 7ff37359e700 1 --2- 172.21.15.17:0/2840618825 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3740b3260 0x7ff374066120 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7ff36c00c000 tx=0x7ff36c0091d0).stop 2022-08-01T19:40:58.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.095+0000 7ff37359e700 1 -- 172.21.15.17:0/2840618825 shutdown_connections 2022-08-01T19:40:58.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.095+0000 7ff37359e700 1 --2- 172.21.15.17:0/2840618825 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff3580539e0 0x7ff358055ea0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:58.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.095+0000 7ff37359e700 1 --2- 172.21.15.17:0/2840618825 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3740b3260 0x7ff374066120 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:58.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.095+0000 7ff37359e700 1 --2- 172.21.15.17:0/2840618825 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3740b08a0 0x7ff374065c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:58.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.095+0000 7ff37359e700 1 --2- 172.21.15.17:0/2840618825 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff3740ad000 0x7ff37413de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:58.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.095+0000 7ff37359e700 1 -- 172.21.15.17:0/2840618825 >> 172.21.15.17:0/2840618825 conn(0x7ff3740612d0 msgr2=0x7ff374142dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:58.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.095+0000 7ff37359e700 1 -- 172.21.15.17:0/2840618825 shutdown_connections 2022-08-01T19:40:58.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.095+0000 7ff37359e700 1 -- 172.21.15.17:0/2840618825 wait complete. 2022-08-01T19:40:58.119 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-08-01T19:40:58.120 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:40:58.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.223+0000 7f52e0f20700 1 Processor -- start 2022-08-01T19:40:58.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.223+0000 7f52e0f20700 1 -- start start 2022-08-01T19:40:58.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.223+0000 7f52e0f20700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f52dc0600f0 0x7f52dc1405d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:58.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.223+0000 7f52e0f20700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f52dc061360 0x7f52dc140ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:58.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.223+0000 7f52e0f20700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52dc140fb0 con 0x7f52dc0b28a0 2022-08-01T19:40:58.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.223+0000 7f52e0f20700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52dc064aa0 con 0x7f52dc0600f0 2022-08-01T19:40:58.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.223+0000 7f52e0f20700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52dc064d40 con 0x7f52dc061360 2022-08-01T19:40:58.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.223+0000 7f52d9d9b700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f52dc0b28a0 0x7f52dc1432a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33396/0 (socket says 172.21.15.17:33396) 2022-08-01T19:40:58.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.223+0000 7f52d9d9b700 1 -- 172.21.15.17:0/193304576 learned_addr learned my addr 172.21.15.17:0/193304576 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:40:58.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.223+0000 7f52da59c700 1 --2- 172.21.15.17:0/193304576 >> v2:172.21.15.17:3300/0 conn(0x7f52dc0600f0 0x7f52dc1405d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:58.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.223+0000 7f52dad9d700 1 --2- 172.21.15.17:0/193304576 >> v2:172.21.15.77:3300/0 conn(0x7f52dc061360 0x7f52dc140ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:58.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.227+0000 7f52d959a700 1 -- 172.21.15.17:0/193304576 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3845205852 0 0) 0x7f52dc140fb0 con 0x7f52dc0b28a0 2022-08-01T19:40:58.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.227+0000 7f52d959a700 1 -- 172.21.15.17:0/193304576 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52bc003260 con 0x7f52dc0b28a0 2022-08-01T19:40:58.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.227+0000 7f52da59c700 1 -- 172.21.15.17:0/193304576 >> v2:172.21.15.77:3300/0 conn(0x7f52dc061360 msgr2=0x7f52dc140ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:58.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.227+0000 7f52da59c700 1 --2- 172.21.15.17:0/193304576 >> v2:172.21.15.77:3300/0 conn(0x7f52dc061360 0x7f52dc140ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:58.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.227+0000 7f52da59c700 1 -- 172.21.15.17:0/193304576 >> v1:172.21.15.17:6789/0 conn(0x7f52dc0b28a0 legacy=0x7f52dc1432a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:58.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.227+0000 7f52da59c700 1 -- 172.21.15.17:0/193304576 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52dc064fe0 con 0x7f52dc0600f0 2022-08-01T19:40:58.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.227+0000 7f52da59c700 1 --2- 172.21.15.17:0/193304576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52dc0600f0 0x7f52dc1405d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7f52c4002f70 tx=0x7f52c4006df0).ready entity=mon.0 client_cookie=819b0047c7f4fba7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:58.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.227+0000 7f52d959a700 1 -- 172.21.15.17:0/193304576 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52c400d4f0 con 0x7f52dc0600f0 2022-08-01T19:40:58.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.227+0000 7f52d959a700 1 -- 172.21.15.17:0/193304576 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c4000e90 con 0x7f52dc0600f0 2022-08-01T19:40:58.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.227+0000 7f52d959a700 1 -- 172.21.15.17:0/193304576 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52c400d840 con 0x7f52dc0600f0 2022-08-01T19:40:58.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.227+0000 7f52e0f20700 1 -- 172.21.15.17:0/193304576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52dc0600f0 msgr2=0x7f52dc1405d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:58.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.227+0000 7f52e0f20700 1 --2- 172.21.15.17:0/193304576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52dc0600f0 0x7f52dc1405d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7f52c4002f70 tx=0x7f52c4006df0).stop 2022-08-01T19:40:58.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.227+0000 7f52e0f20700 1 -- 172.21.15.17:0/193304576 shutdown_connections 2022-08-01T19:40:58.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.227+0000 7f52e0f20700 1 --2- 172.21.15.17:0/193304576 >> v2:172.21.15.77:3300/0 conn(0x7f52dc061360 0x7f52dc140ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:58.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.227+0000 7f52e0f20700 1 --2- 172.21.15.17:0/193304576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52dc0600f0 0x7f52dc1405d0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:58.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.227+0000 7f52e0f20700 1 -- 172.21.15.17:0/193304576 >> 172.21.15.17:0/193304576 conn(0x7f52dc05d0f0 msgr2=0x7f52dc05f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:58.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.227+0000 7f52e0f20700 1 -- 172.21.15.17:0/193304576 shutdown_connections 2022-08-01T19:40:58.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.227+0000 7f52e0f20700 1 -- 172.21.15.17:0/193304576 wait complete. 2022-08-01T19:40:58.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.227+0000 7f52e0f20700 1 Processor -- start 2022-08-01T19:40:58.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.227+0000 7f52e0f20700 1 -- start start 2022-08-01T19:40:58.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.231+0000 7f52e0f20700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52dc0600f0 0x7f52dc14a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:58.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.231+0000 7f52e0f20700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f52dc061360 0x7f52dc155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:58.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.231+0000 7f52e0f20700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52dc0b28a0 0x7f52dc156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:58.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.231+0000 7f52e0f20700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc1558a0 con 0x7f52dc0b28a0 2022-08-01T19:40:58.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.231+0000 7f52e0f20700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc141720 con 0x7f52dc0600f0 2022-08-01T19:40:58.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.231+0000 7f52e0f20700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc0ac2b0 con 0x7f52dc061360 2022-08-01T19:40:58.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.231+0000 7f52dad9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52dc0b28a0 0x7f52dc156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:58.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.231+0000 7f52dad9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52dc0b28a0 0x7f52dc156150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41212/0 (socket says 172.21.15.17:41212) 2022-08-01T19:40:58.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.231+0000 7f52da59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52dc0600f0 0x7f52dc14a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:58.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.231+0000 7f52dad9d700 1 -- 172.21.15.17:0/1925481849 learned_addr learned my addr 172.21.15.17:0/1925481849 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:58.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.231+0000 7f52d9d9b700 1 --2- 172.21.15.17:0/1925481849 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f52dc061360 0x7f52dc155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:58.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.231+0000 7f52dad9d700 1 -- 172.21.15.17:0/1925481849 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f52dc061360 msgr2=0x7f52dc155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:58.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.231+0000 7f52dad9d700 1 --2- 172.21.15.17:0/1925481849 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f52dc061360 0x7f52dc155c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:58.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.231+0000 7f52dad9d700 1 -- 172.21.15.17:0/1925481849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52dc0600f0 msgr2=0x7f52dc14a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:58.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.231+0000 7f52dad9d700 1 --2- 172.21.15.17:0/1925481849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52dc0600f0 0x7f52dc14a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:58.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.231+0000 7f52dad9d700 1 -- 172.21.15.17:0/1925481849 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c400b180 con 0x7f52dc0b28a0 2022-08-01T19:40:58.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.231+0000 7f52da59c700 1 --2- 172.21.15.17:0/1925481849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52dc0600f0 0x7f52dc14a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:40:58.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.231+0000 7f52dad9d700 1 --2- 172.21.15.17:0/1925481849 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52dc0b28a0 0x7f52dc156150 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 rx=0x7f52d4006e00 tx=0x7f52d40091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:58.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.231+0000 7f52d37fe700 1 -- 172.21.15.17:0/1925481849 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52d400c640 con 0x7f52dc0b28a0 2022-08-01T19:40:58.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.231+0000 7f52d37fe700 1 -- 172.21.15.17:0/1925481849 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d400c7a0 con 0x7f52dc0b28a0 2022-08-01T19:40:58.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.231+0000 7f52e0f20700 1 -- 172.21.15.17:0/1925481849 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52dc0ae2e0 con 0x7f52dc0b28a0 2022-08-01T19:40:58.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.231+0000 7f52d37fe700 1 -- 172.21.15.17:0/1925481849 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52d400cb00 con 0x7f52dc0b28a0 2022-08-01T19:40:58.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.231+0000 7f52e0f20700 1 -- 172.21.15.17:0/1925481849 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52dc141480 con 0x7f52dc0b28a0 2022-08-01T19:40:58.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.235+0000 7f52d37fe700 1 -- 172.21.15.17:0/1925481849 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f52d4022070 con 0x7f52dc0b28a0 2022-08-01T19:40:58.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.235+0000 7f52e0f20700 1 -- 172.21.15.17:0/1925481849 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52dc0b0b00 con 0x7f52dc0b28a0 2022-08-01T19:40:58.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.235+0000 7f52d37fe700 1 --2- 172.21.15.17:0/1925481849 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f52c0053b00 0x7f52c0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:58.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.235+0000 7f52d37fe700 1 -- 172.21.15.17:0/1925481849 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f52d406c4e0 con 0x7f52dc0b28a0 2022-08-01T19:40:58.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.235+0000 7f52da59c700 1 --2- 172.21.15.17:0/1925481849 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f52c0053b00 0x7f52c0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:58.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.235+0000 7f52da59c700 1 --2- 172.21.15.17:0/1925481849 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f52c0053b00 0x7f52c0055fc0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7f52c401a000 tx=0x7f52c4014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:58.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.243+0000 7f52d37fe700 1 -- 172.21.15.17:0/1925481849 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f52d40434d0 con 0x7f52dc0b28a0 2022-08-01T19:40:58.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.367+0000 7f52e0f20700 1 -- 172.21.15.17:0/1925481849 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f52dc065280 con 0x7f52dc0b28a0 2022-08-01T19:40:58.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.371+0000 7f52d37fe700 1 -- 172.21.15.17:0/1925481849 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v84) v1 ==== 110+0+10 (secure 0 0 0) 0x7f52d4046af0 con 0x7f52dc0b28a0 2022-08-01T19:40:58.375 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:40:58.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.371+0000 7f52e0f20700 1 -- 172.21.15.17:0/1925481849 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f52c0053b00 msgr2=0x7f52c0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:58.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.371+0000 7f52e0f20700 1 --2- 172.21.15.17:0/1925481849 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f52c0053b00 0x7f52c0055fc0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7f52c401a000 tx=0x7f52c4014000).stop 2022-08-01T19:40:58.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.371+0000 7f52e0f20700 1 -- 172.21.15.17:0/1925481849 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52dc0b28a0 msgr2=0x7f52dc156150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:58.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.371+0000 7f52e0f20700 1 --2- 172.21.15.17:0/1925481849 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52dc0b28a0 0x7f52dc156150 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 rx=0x7f52d4006e00 tx=0x7f52d40091d0).stop 2022-08-01T19:40:58.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.371+0000 7f52e0f20700 1 -- 172.21.15.17:0/1925481849 shutdown_connections 2022-08-01T19:40:58.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.371+0000 7f52e0f20700 1 --2- 172.21.15.17:0/1925481849 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f52c0053b00 0x7f52c0055fc0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:58.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.371+0000 7f52e0f20700 1 --2- 172.21.15.17:0/1925481849 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52dc0b28a0 0x7f52dc156150 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:58.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.371+0000 7f52e0f20700 1 --2- 172.21.15.17:0/1925481849 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f52dc061360 0x7f52dc155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:58.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.371+0000 7f52e0f20700 1 --2- 172.21.15.17:0/1925481849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52dc0600f0 0x7f52dc14a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:58.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.371+0000 7f52e0f20700 1 -- 172.21.15.17:0/1925481849 >> 172.21.15.17:0/1925481849 conn(0x7f52dc05d0f0 msgr2=0x7f52dc0b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:58.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.375+0000 7f52e0f20700 1 -- 172.21.15.17:0/1925481849 shutdown_connections 2022-08-01T19:40:58.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.375+0000 7f52e0f20700 1 -- 172.21.15.17:0/1925481849 wait complete. 2022-08-01T19:40:58.392 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:40:58.502 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.499+0000 7fb221c74700 1 Processor -- start 2022-08-01T19:40:58.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.499+0000 7fb221c74700 1 -- start start 2022-08-01T19:40:58.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.499+0000 7fb221c74700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fb21c146ae0 0x7fb21c1403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:58.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.499+0000 7fb221c74700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb21c0b4820 0x7fb21c0b5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:58.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.499+0000 7fb221c74700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb21c143cf0 con 0x7fb21c0b4310 2022-08-01T19:40:58.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.499+0000 7fb221c74700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb21c1476a0 con 0x7fb21c0b4820 2022-08-01T19:40:58.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.499+0000 7fb221c74700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb21c143f90 con 0x7fb21c146ae0 2022-08-01T19:40:58.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.499+0000 7fb21bfff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb21c0b4820 0x7fb21c0b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:58.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.499+0000 7fb21bfff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb21c0b4820 0x7fb21c0b5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41220/0 (socket says 172.21.15.17:41220) 2022-08-01T19:40:58.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.499+0000 7fb21bfff700 1 -- 172.21.15.17:0/4059651917 learned_addr learned my addr 172.21.15.17:0/4059651917 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:58.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.499+0000 7fb21b7fe700 1 --2- 172.21.15.17:0/4059651917 >> v2:172.21.15.77:3300/0 conn(0x7fb21c146ae0 0x7fb21c1403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:58.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.499+0000 7fb21bfff700 1 -- 172.21.15.17:0/4059651917 >> v2:172.21.15.77:3300/0 conn(0x7fb21c146ae0 msgr2=0x7fb21c1403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:58.547 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.499+0000 7fb21bfff700 1 --2- 172.21.15.17:0/4059651917 >> v2:172.21.15.77:3300/0 conn(0x7fb21c146ae0 0x7fb21c1403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:58.547 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.499+0000 7fb21bfff700 1 -- 172.21.15.17:0/4059651917 >> v1:172.21.15.17:6789/0 conn(0x7fb21c0b4310 legacy=0x7fb21c1408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:58.547 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.499+0000 7fb21bfff700 1 -- 172.21.15.17:0/4059651917 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb21c0b0e20 con 0x7fb21c0b4820 2022-08-01T19:40:58.547 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.499+0000 7fb21b7fe700 1 --2- 172.21.15.17:0/4059651917 >> v2:172.21.15.77:3300/0 conn(0x7fb21c146ae0 0x7fb21c1403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:40:58.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.499+0000 7fb21bfff700 1 --2- 172.21.15.17:0/4059651917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb21c0b4820 0x7fb21c0b5060 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7fb2140032f0 tx=0x7fb214009290).ready entity=mon.0 client_cookie=9da4e360f20895e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:58.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.499+0000 7fb21affd700 1 -- 172.21.15.17:0/4059651917 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb21400b870 con 0x7fb21c0b4820 2022-08-01T19:40:58.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.499+0000 7fb21affd700 1 -- 172.21.15.17:0/4059651917 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21400b9d0 con 0x7fb21c0b4820 2022-08-01T19:40:58.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.499+0000 7fb21affd700 1 -- 172.21.15.17:0/4059651917 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb21400bb40 con 0x7fb21c0b4820 2022-08-01T19:40:58.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.499+0000 7fb221c74700 1 -- 172.21.15.17:0/4059651917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb21c0b4820 msgr2=0x7fb21c0b5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:58.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.499+0000 7fb221c74700 1 --2- 172.21.15.17:0/4059651917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb21c0b4820 0x7fb21c0b5060 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7fb2140032f0 tx=0x7fb214009290).stop 2022-08-01T19:40:58.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.503+0000 7fb221c74700 1 -- 172.21.15.17:0/4059651917 shutdown_connections 2022-08-01T19:40:58.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.503+0000 7fb221c74700 1 --2- 172.21.15.17:0/4059651917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb21c0b4820 0x7fb21c0b5060 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:58.550 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.503+0000 7fb221c74700 1 --2- 172.21.15.17:0/4059651917 >> v2:172.21.15.77:3300/0 conn(0x7fb21c146ae0 0x7fb21c1403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:58.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.503+0000 7fb221c74700 1 -- 172.21.15.17:0/4059651917 >> 172.21.15.17:0/4059651917 conn(0x7fb21c05c620 msgr2=0x7fb21c05ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:58.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.503+0000 7fb221c74700 1 -- 172.21.15.17:0/4059651917 shutdown_connections 2022-08-01T19:40:58.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.503+0000 7fb221c74700 1 -- 172.21.15.17:0/4059651917 wait complete. 2022-08-01T19:40:58.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.503+0000 7fb221c74700 1 Processor -- start 2022-08-01T19:40:58.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.503+0000 7fb221c74700 1 -- start start 2022-08-01T19:40:58.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.503+0000 7fb221c74700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb21c0b4310 0x7fb21c0b7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:58.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.503+0000 7fb221c74700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb21c0b4820 0x7fb21c152e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:58.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.503+0000 7fb221c74700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb21c146ae0 0x7fb21c0b95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:58.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.503+0000 7fb221c74700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21c0b88a0 con 0x7fb21c146ae0 2022-08-01T19:40:58.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.503+0000 7fb221c74700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21c0b08b0 con 0x7fb21c0b4820 2022-08-01T19:40:58.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.503+0000 7fb221c74700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb21c0b83a0 con 0x7fb21c0b4310 2022-08-01T19:40:58.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.503+0000 7fb21bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb21c146ae0 0x7fb21c0b95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:58.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.503+0000 7fb21bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb21c146ae0 0x7fb21c0b95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41226/0 (socket says 172.21.15.17:41226) 2022-08-01T19:40:58.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.503+0000 7fb21bfff700 1 -- 172.21.15.17:0/1550586287 learned_addr learned my addr 172.21.15.17:0/1550586287 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:58.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.503+0000 7fb212ffd700 1 --2- 172.21.15.17:0/1550586287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb21c0b4820 0x7fb21c152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:58.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.503+0000 7fb21b7fe700 1 --2- 172.21.15.17:0/1550586287 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb21c0b4310 0x7fb21c0b7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:58.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.503+0000 7fb21bfff700 1 -- 172.21.15.17:0/1550586287 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb21c0b4310 msgr2=0x7fb21c0b7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:58.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.503+0000 7fb21bfff700 1 --2- 172.21.15.17:0/1550586287 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb21c0b4310 0x7fb21c0b7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:58.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.503+0000 7fb21bfff700 1 -- 172.21.15.17:0/1550586287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb21c0b4820 msgr2=0x7fb21c152e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:58.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.503+0000 7fb21bfff700 1 --2- 172.21.15.17:0/1550586287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb21c0b4820 0x7fb21c152e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:58.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.503+0000 7fb21bfff700 1 -- 172.21.15.17:0/1550586287 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb214009530 con 0x7fb21c146ae0 2022-08-01T19:40:58.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.503+0000 7fb212ffd700 1 --2- 172.21.15.17:0/1550586287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb21c0b4820 0x7fb21c152e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:40:58.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.503+0000 7fb21bfff700 1 --2- 172.21.15.17:0/1550586287 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb21c146ae0 0x7fb21c0b95c0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7fb2140032f0 tx=0x7fb214016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:58.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.503+0000 7fb2197fa700 1 -- 172.21.15.17:0/1550586287 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb21400b870 con 0x7fb21c146ae0 2022-08-01T19:40:58.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.503+0000 7fb2197fa700 1 -- 172.21.15.17:0/1550586287 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21400bdb0 con 0x7fb21c146ae0 2022-08-01T19:40:58.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.503+0000 7fb221c74700 1 -- 172.21.15.17:0/1550586287 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb21c153340 con 0x7fb21c146ae0 2022-08-01T19:40:58.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.503+0000 7fb2197fa700 1 -- 172.21.15.17:0/1550586287 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb214020610 con 0x7fb21c146ae0 2022-08-01T19:40:58.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.503+0000 7fb221c74700 1 -- 172.21.15.17:0/1550586287 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb21c0b8600 con 0x7fb21c146ae0 2022-08-01T19:40:58.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.507+0000 7fb221c74700 1 -- 172.21.15.17:0/1550586287 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb21c0b92c0 con 0x7fb21c146ae0 2022-08-01T19:40:58.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.507+0000 7fb2197fa700 1 -- 172.21.15.17:0/1550586287 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb21400f070 con 0x7fb21c146ae0 2022-08-01T19:40:58.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.507+0000 7fb2197fa700 1 --2- 172.21.15.17:0/1550586287 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb200053a10 0x7fb200055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:58.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.507+0000 7fb2197fa700 1 -- 172.21.15.17:0/1550586287 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fb21406c960 con 0x7fb21c146ae0 2022-08-01T19:40:58.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.507+0000 7fb21b7fe700 1 --2- 172.21.15.17:0/1550586287 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb200053a10 0x7fb200055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:58.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.507+0000 7fb21b7fe700 1 --2- 172.21.15.17:0/1550586287 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb200053a10 0x7fb200055ed0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7fb20c00c000 tx=0x7fb20c008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:58.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.511+0000 7fb2197fa700 1 -- 172.21.15.17:0/1550586287 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb214043950 con 0x7fb21c146ae0 2022-08-01T19:40:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.635+0000 7fb221c74700 1 -- 172.21.15.17:0/1550586287 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb21c0574e0 con 0x7fb21c146ae0 2022-08-01T19:40:58.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.639+0000 7fb2197fa700 1 -- 172.21.15.17:0/1550586287 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v84) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb214015070 con 0x7fb21c146ae0 2022-08-01T19:40:58.642 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:40:58.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.639+0000 7fb221c74700 1 -- 172.21.15.17:0/1550586287 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb200053a10 msgr2=0x7fb200055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:58.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.639+0000 7fb221c74700 1 --2- 172.21.15.17:0/1550586287 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb200053a10 0x7fb200055ed0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7fb20c00c000 tx=0x7fb20c008040).stop 2022-08-01T19:40:58.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.639+0000 7fb221c74700 1 -- 172.21.15.17:0/1550586287 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb21c146ae0 msgr2=0x7fb21c0b95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:58.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.639+0000 7fb221c74700 1 --2- 172.21.15.17:0/1550586287 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb21c146ae0 0x7fb21c0b95c0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7fb2140032f0 tx=0x7fb214016000).stop 2022-08-01T19:40:58.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.639+0000 7fb221c74700 1 -- 172.21.15.17:0/1550586287 shutdown_connections 2022-08-01T19:40:58.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.639+0000 7fb221c74700 1 --2- 172.21.15.17:0/1550586287 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb200053a10 0x7fb200055ed0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:58.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.639+0000 7fb221c74700 1 --2- 172.21.15.17:0/1550586287 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb21c146ae0 0x7fb21c0b95c0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:58.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.639+0000 7fb221c74700 1 --2- 172.21.15.17:0/1550586287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb21c0b4820 0x7fb21c152e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:58.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.639+0000 7fb221c74700 1 --2- 172.21.15.17:0/1550586287 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb21c0b4310 0x7fb21c0b7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:58.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.639+0000 7fb221c74700 1 -- 172.21.15.17:0/1550586287 >> 172.21.15.17:0/1550586287 conn(0x7fb21c05c620 msgr2=0x7fb21c13e760 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:58.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.639+0000 7fb221c74700 1 -- 172.21.15.17:0/1550586287 shutdown_connections 2022-08-01T19:40:58.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.639+0000 7fb221c74700 1 -- 172.21.15.17:0/1550586287 wait complete. 2022-08-01T19:40:58.662 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-08-01T19:40:58.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.763+0000 7f307a513700 1 Processor -- start 2022-08-01T19:40:58.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.767+0000 7f307a513700 1 -- start start 2022-08-01T19:40:58.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.767+0000 7f307a513700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f30740b1300 0x7f30741415c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:58.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.767+0000 7f307a513700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f3074141ab0 0x7f3074141f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:58.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.767+0000 7f307a513700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30740b3360 con 0x7f30740b79a0 2022-08-01T19:40:58.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.767+0000 7f307a513700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3074142430 con 0x7f30740b1300 2022-08-01T19:40:58.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.767+0000 7f307a513700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30740b7d80 con 0x7f3074141ab0 2022-08-01T19:40:58.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.767+0000 7f30737fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f30740b1300 0x7f30741415c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:58.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.767+0000 7f3073fff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f30740b79a0 0x7f3074144730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33424/0 (socket says 172.21.15.17:33424) 2022-08-01T19:40:58.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.767+0000 7f3073fff700 1 -- 172.21.15.17:0/2405830004 learned_addr learned my addr 172.21.15.17:0/2405830004 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:40:58.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.767+0000 7f3078ab0700 1 --2- 172.21.15.17:0/2405830004 >> v2:172.21.15.77:3300/0 conn(0x7f3074141ab0 0x7f3074141f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:58.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.767+0000 7f30737fe700 1 -- 172.21.15.17:0/2405830004 >> v2:172.21.15.77:3300/0 conn(0x7f3074141ab0 msgr2=0x7f3074141f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:58.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.767+0000 7f30737fe700 1 --2- 172.21.15.17:0/2405830004 >> v2:172.21.15.77:3300/0 conn(0x7f3074141ab0 0x7f3074141f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:58.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.767+0000 7f30737fe700 1 -- 172.21.15.17:0/2405830004 >> v1:172.21.15.17:6789/0 conn(0x7f30740b79a0 legacy=0x7f3074144730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:58.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.767+0000 7f30737fe700 1 -- 172.21.15.17:0/2405830004 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3074067750 con 0x7f30740b1300 2022-08-01T19:40:58.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.767+0000 7f30737fe700 1 --2- 172.21.15.17:0/2405830004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30740b1300 0x7f30741415c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7f3068002f70 tx=0x7f3068006df0).ready entity=mon.0 client_cookie=5c5107c1a665e95f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:58.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.767+0000 7f3072ffd700 1 -- 172.21.15.17:0/2405830004 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f306800d570 con 0x7f30740b1300 2022-08-01T19:40:58.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.767+0000 7f3072ffd700 1 -- 172.21.15.17:0/2405830004 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3068000e90 con 0x7f30740b1300 2022-08-01T19:40:58.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.767+0000 7f3072ffd700 1 -- 172.21.15.17:0/2405830004 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f306800d8c0 con 0x7f30740b1300 2022-08-01T19:40:58.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.767+0000 7f307a513700 1 -- 172.21.15.17:0/2405830004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30740b1300 msgr2=0x7f30741415c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:58.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.767+0000 7f307a513700 1 --2- 172.21.15.17:0/2405830004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30740b1300 0x7f30741415c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7f3068002f70 tx=0x7f3068006df0).stop 2022-08-01T19:40:58.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.767+0000 7f307a513700 1 -- 172.21.15.17:0/2405830004 shutdown_connections 2022-08-01T19:40:58.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.767+0000 7f307a513700 1 --2- 172.21.15.17:0/2405830004 >> v2:172.21.15.77:3300/0 conn(0x7f3074141ab0 0x7f3074141f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:58.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.767+0000 7f307a513700 1 --2- 172.21.15.17:0/2405830004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30740b1300 0x7f30741415c0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:58.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.767+0000 7f307a513700 1 -- 172.21.15.17:0/2405830004 >> 172.21.15.17:0/2405830004 conn(0x7f307405f840 msgr2=0x7f307405fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:58.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.767+0000 7f307a513700 1 -- 172.21.15.17:0/2405830004 shutdown_connections 2022-08-01T19:40:58.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.767+0000 7f307a513700 1 -- 172.21.15.17:0/2405830004 wait complete. 2022-08-01T19:40:58.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.771+0000 7f307a513700 1 Processor -- start 2022-08-01T19:40:58.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.771+0000 7f307a513700 1 -- start start 2022-08-01T19:40:58.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.771+0000 7f307a513700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30740b1300 0x7f307405be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:58.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.771+0000 7f307a513700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30740b79a0 0x7f30740a9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:58.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.771+0000 7f307a513700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3074141ab0 0x7f30740a9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:58.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.771+0000 7f307a513700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3074154bb0 con 0x7f30740b1300 2022-08-01T19:40:58.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.771+0000 7f307a513700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30740accf0 con 0x7f30740b79a0 2022-08-01T19:40:58.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.771+0000 7f307a513700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30740acf90 con 0x7f3074141ab0 2022-08-01T19:40:58.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.771+0000 7f3073fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30740b1300 0x7f307405be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:58.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.771+0000 7f3073fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30740b1300 0x7f307405be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41236/0 (socket says 172.21.15.17:41236) 2022-08-01T19:40:58.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.771+0000 7f3073fff700 1 -- 172.21.15.17:0/111845492 learned_addr learned my addr 172.21.15.17:0/111845492 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:58.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.771+0000 7f30737fe700 1 --2- 172.21.15.17:0/111845492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30740b79a0 0x7f30740a9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:58.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.771+0000 7f3078ab0700 1 --2- 172.21.15.17:0/111845492 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3074141ab0 0x7f30740a9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:58.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.771+0000 7f3073fff700 1 -- 172.21.15.17:0/111845492 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3074141ab0 msgr2=0x7f30740a9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:58.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.771+0000 7f3073fff700 1 --2- 172.21.15.17:0/111845492 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3074141ab0 0x7f30740a9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:58.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.771+0000 7f3073fff700 1 -- 172.21.15.17:0/111845492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30740b79a0 msgr2=0x7f30740a9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:58.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.771+0000 7f3073fff700 1 --2- 172.21.15.17:0/111845492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30740b79a0 0x7f30740a9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:58.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.771+0000 7f3073fff700 1 -- 172.21.15.17:0/111845492 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306800b180 con 0x7f30740b1300 2022-08-01T19:40:58.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.771+0000 7f30737fe700 1 --2- 172.21.15.17:0/111845492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30740b79a0 0x7f30740a9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:40:58.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.771+0000 7f3073fff700 1 --2- 172.21.15.17:0/111845492 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30740b1300 0x7f307405be30 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7f3074067e30 tx=0x7f3064007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:58.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.771+0000 7f30717fa700 1 -- 172.21.15.17:0/111845492 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f306400ce30 con 0x7f30740b1300 2022-08-01T19:40:58.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.771+0000 7f30717fa700 1 -- 172.21.15.17:0/111845492 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30640072e0 con 0x7f30740b1300 2022-08-01T19:40:58.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.771+0000 7f307a513700 1 -- 172.21.15.17:0/111845492 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30740ad290 con 0x7f30740b1300 2022-08-01T19:40:58.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.771+0000 7f30717fa700 1 -- 172.21.15.17:0/111845492 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3064005800 con 0x7f30740b1300 2022-08-01T19:40:58.802 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.771+0000 7f307a513700 1 -- 172.21.15.17:0/111845492 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f307405c320 con 0x7f30740b1300 2022-08-01T19:40:58.802 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.775+0000 7f307a513700 1 -- 172.21.15.17:0/111845492 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30740ab000 con 0x7f30740b1300 2022-08-01T19:40:58.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.775+0000 7f30717fa700 1 -- 172.21.15.17:0/111845492 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3064010070 con 0x7f30740b1300 2022-08-01T19:40:58.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.775+0000 7f30717fa700 1 --2- 172.21.15.17:0/111845492 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3054053b00 0x7f3054055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:58.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.775+0000 7f30717fa700 1 -- 172.21.15.17:0/111845492 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f306406b230 con 0x7f30740b1300 2022-08-01T19:40:58.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.775+0000 7f30737fe700 1 --2- 172.21.15.17:0/111845492 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3054053b00 0x7f3054055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:58.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.775+0000 7f30737fe700 1 --2- 172.21.15.17:0/111845492 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3054053b00 0x7f3054055fc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f3068018000 tx=0x7f3068014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:58.807 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.779+0000 7f30717fa700 1 -- 172.21.15.17:0/111845492 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3064042330 con 0x7f30740b1300 2022-08-01T19:40:58.915 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.907+0000 7f307a513700 1 -- 172.21.15.17:0/111845492 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f30740574e0 con 0x7f30740b1300 2022-08-01T19:40:58.916 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.907+0000 7f30717fa700 1 -- 172.21.15.17:0/111845492 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v84) v1 ==== 84+0+143 (secure 0 0 0) 0x7f3064045950 con 0x7f30740b1300 2022-08-01T19:40:58.916 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:40:58.916 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2022-08-01T19:40:58.917 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:58.917 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 1, 2022-08-01T19:40:58.917 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": "device_health_metrics" 2022-08-01T19:40:58.917 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:40:58.918 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:40:58.918 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 2, 2022-08-01T19:40:58.918 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": "rbd" 2022-08-01T19:40:58.919 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:40:58.919 INFO:teuthology.orchestra.run.smithi017.stdout:] 2022-08-01T19:40:58.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.911+0000 7f307a513700 1 -- 172.21.15.17:0/111845492 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3054053b00 msgr2=0x7f3054055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:58.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.911+0000 7f307a513700 1 --2- 172.21.15.17:0/111845492 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3054053b00 0x7f3054055fc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f3068018000 tx=0x7f3068014000).stop 2022-08-01T19:40:58.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.911+0000 7f307a513700 1 -- 172.21.15.17:0/111845492 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30740b1300 msgr2=0x7f307405be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:58.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.911+0000 7f307a513700 1 --2- 172.21.15.17:0/111845492 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30740b1300 0x7f307405be30 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7f3074067e30 tx=0x7f3064007c40).stop 2022-08-01T19:40:58.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.911+0000 7f307a513700 1 -- 172.21.15.17:0/111845492 shutdown_connections 2022-08-01T19:40:58.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.911+0000 7f307a513700 1 --2- 172.21.15.17:0/111845492 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3054053b00 0x7f3054055fc0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:58.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.911+0000 7f307a513700 1 --2- 172.21.15.17:0/111845492 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3074141ab0 0x7f30740a9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:58.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.911+0000 7f307a513700 1 --2- 172.21.15.17:0/111845492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30740b79a0 0x7f30740a9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:58.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.911+0000 7f307a513700 1 --2- 172.21.15.17:0/111845492 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30740b1300 0x7f307405be30 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:58.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.911+0000 7f307a513700 1 -- 172.21.15.17:0/111845492 >> 172.21.15.17:0/111845492 conn(0x7f307405f840 msgr2=0x7f3074141260 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:58.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.911+0000 7f307a513700 1 -- 172.21.15.17:0/111845492 shutdown_connections 2022-08-01T19:40:58.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:58.911+0000 7f307a513700 1 -- 172.21.15.17:0/111845492 wait complete. 2022-08-01T19:40:58.926 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-08-01T19:40:58.927 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-08-01T19:40:59.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.027+0000 7f44596c0700 1 Processor -- start 2022-08-01T19:40:59.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.027+0000 7f44596c0700 1 -- start start 2022-08-01T19:40:59.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.027+0000 7f44596c0700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f44540b7840 0x7f4454065720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:59.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.027+0000 7f44596c0700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f445414b040 0x7f445414b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:59.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.027+0000 7f44596c0700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44540b2c20 con 0x7f44540b3070 2022-08-01T19:40:59.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.027+0000 7f44596c0700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4454061830 con 0x7f44540b7840 2022-08-01T19:40:59.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.027+0000 7f44596c0700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4454065c10 con 0x7f445414b040 2022-08-01T19:40:59.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.027+0000 7f4453fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f44540b7840 0x7f4454065720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:59.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.027+0000 7f44537fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f44540b3070 0x7f4454061120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33438/0 (socket says 172.21.15.17:33438) 2022-08-01T19:40:59.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.027+0000 7f44537fe700 1 -- 172.21.15.17:0/3235280321 learned_addr learned my addr 172.21.15.17:0/3235280321 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:40:59.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.027+0000 7f4458ebf700 1 --2- 172.21.15.17:0/3235280321 >> v2:172.21.15.77:3300/0 conn(0x7f445414b040 0x7f445414b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:59.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.027+0000 7f4453fff700 1 -- 172.21.15.17:0/3235280321 >> v2:172.21.15.77:3300/0 conn(0x7f445414b040 msgr2=0x7f445414b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:40:59.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.027+0000 7f4453fff700 1 --2- 172.21.15.17:0/3235280321 >> v2:172.21.15.77:3300/0 conn(0x7f445414b040 0x7f445414b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:59.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.027+0000 7f4453fff700 1 -- 172.21.15.17:0/3235280321 >> v1:172.21.15.17:6789/0 conn(0x7f44540b3070 legacy=0x7f4454061120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:59.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.027+0000 7f4453fff700 1 -- 172.21.15.17:0/3235280321 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44540b7c20 con 0x7f44540b7840 2022-08-01T19:40:59.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.027+0000 7f4453fff700 1 --2- 172.21.15.17:0/3235280321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44540b7840 0x7f4454065720 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0x7f44440032f0 tx=0x7f44440091a0).ready entity=mon.0 client_cookie=430f5f3fc030523d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:59.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.027+0000 7f4452ffd700 1 -- 172.21.15.17:0/3235280321 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4444009ea0 con 0x7f44540b7840 2022-08-01T19:40:59.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.027+0000 7f4452ffd700 1 -- 172.21.15.17:0/3235280321 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444400b870 con 0x7f44540b7840 2022-08-01T19:40:59.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.027+0000 7f4452ffd700 1 -- 172.21.15.17:0/3235280321 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f444400b9e0 con 0x7f44540b7840 2022-08-01T19:40:59.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.031+0000 7f44596c0700 1 -- 172.21.15.17:0/3235280321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44540b7840 msgr2=0x7f4454065720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:59.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.031+0000 7f44596c0700 1 --2- 172.21.15.17:0/3235280321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44540b7840 0x7f4454065720 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0x7f44440032f0 tx=0x7f44440091a0).stop 2022-08-01T19:40:59.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.031+0000 7f44596c0700 1 -- 172.21.15.17:0/3235280321 shutdown_connections 2022-08-01T19:40:59.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.031+0000 7f44596c0700 1 --2- 172.21.15.17:0/3235280321 >> v2:172.21.15.77:3300/0 conn(0x7f445414b040 0x7f445414b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:59.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.031+0000 7f44596c0700 1 --2- 172.21.15.17:0/3235280321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44540b7840 0x7f4454065720 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:59.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.031+0000 7f44596c0700 1 -- 172.21.15.17:0/3235280321 >> 172.21.15.17:0/3235280321 conn(0x7f445405f840 msgr2=0x7f445405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:59.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.031+0000 7f44596c0700 1 -- 172.21.15.17:0/3235280321 shutdown_connections 2022-08-01T19:40:59.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.031+0000 7f44596c0700 1 -- 172.21.15.17:0/3235280321 wait complete. 2022-08-01T19:40:59.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.031+0000 7f44596c0700 1 Processor -- start 2022-08-01T19:40:59.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.031+0000 7f44596c0700 1 -- start start 2022-08-01T19:40:59.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.031+0000 7f44596c0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44540b3070 0x7f44540a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:59.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.031+0000 7f44596c0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44540b7840 0x7f4454156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:59.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.031+0000 7f44596c0700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f445414b040 0x7f4454156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:59.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.031+0000 7f44596c0700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445409f520 con 0x7f44540b7840 2022-08-01T19:40:59.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.031+0000 7f44596c0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44540a10e0 con 0x7f44540b3070 2022-08-01T19:40:59.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.031+0000 7f44596c0700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f445409ec00 con 0x7f445414b040 2022-08-01T19:40:59.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.031+0000 7f44537fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44540b7840 0x7f4454156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:59.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.031+0000 7f44537fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44540b7840 0x7f4454156ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41250/0 (socket says 172.21.15.17:41250) 2022-08-01T19:40:59.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.031+0000 7f44537fe700 1 -- 172.21.15.17:0/2161175349 learned_addr learned my addr 172.21.15.17:0/2161175349 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:40:59.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.031+0000 7f4458ebf700 1 --2- 172.21.15.17:0/2161175349 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f445414b040 0x7f4454156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:59.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.031+0000 7f4453fff700 1 --2- 172.21.15.17:0/2161175349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44540b3070 0x7f44540a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:59.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.035+0000 7f44537fe700 1 -- 172.21.15.17:0/2161175349 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f445414b040 msgr2=0x7f4454156fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:59.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.035+0000 7f44537fe700 1 --2- 172.21.15.17:0/2161175349 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f445414b040 0x7f4454156fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:59.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.035+0000 7f44537fe700 1 -- 172.21.15.17:0/2161175349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44540b3070 msgr2=0x7f44540a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:59.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.035+0000 7f44537fe700 1 --2- 172.21.15.17:0/2161175349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44540b3070 0x7f44540a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:59.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.035+0000 7f44537fe700 1 -- 172.21.15.17:0/2161175349 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4444009420 con 0x7f44540b7840 2022-08-01T19:40:59.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.035+0000 7f44537fe700 1 --2- 172.21.15.17:0/2161175349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44540b7840 0x7f4454156ab0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7f4454069e10 tx=0x7f4448007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:59.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.035+0000 7f44517fa700 1 -- 172.21.15.17:0/2161175349 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44480070d0 con 0x7f44540b7840 2022-08-01T19:40:59.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.035+0000 7f44517fa700 1 -- 172.21.15.17:0/2161175349 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4448007230 con 0x7f44540b7840 2022-08-01T19:40:59.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.035+0000 7f44517fa700 1 -- 172.21.15.17:0/2161175349 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4448005610 con 0x7f44540b7840 2022-08-01T19:40:59.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.035+0000 7f44596c0700 1 -- 172.21.15.17:0/2161175349 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f445409efc0 con 0x7f44540b7840 2022-08-01T19:40:59.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.035+0000 7f44596c0700 1 -- 172.21.15.17:0/2161175349 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44540b15f0 con 0x7f44540b7840 2022-08-01T19:40:59.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.035+0000 7f44596c0700 1 -- 172.21.15.17:0/2161175349 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f445409edc0 con 0x7f44540b7840 2022-08-01T19:40:59.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.035+0000 7f44517fa700 1 -- 172.21.15.17:0/2161175349 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4448010070 con 0x7f44540b7840 2022-08-01T19:40:59.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.039+0000 7f44517fa700 1 --2- 172.21.15.17:0/2161175349 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4434053a30 0x7f4434055ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:40:59.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.039+0000 7f44517fa700 1 -- 172.21.15.17:0/2161175349 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f444806b3d0 con 0x7f44540b7840 2022-08-01T19:40:59.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.039+0000 7f4453fff700 1 --2- 172.21.15.17:0/2161175349 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4434053a30 0x7f4434055ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:40:59.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.039+0000 7f4453fff700 1 --2- 172.21.15.17:0/2161175349 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4434053a30 0x7f4434055ef0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f444401b000 tx=0x7f4444016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:40:59.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.043+0000 7f44517fa700 1 -- 172.21.15.17:0/2161175349 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f44480423c0 con 0x7f44540b7840 2022-08-01T19:40:59.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.171+0000 7f44596c0700 1 -- 172.21.15.17:0/2161175349 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f445409ea40 con 0x7f44540b7840 2022-08-01T19:40:59.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.871+0000 7f44517fa700 1 -- 172.21.15.17:0/2161175349 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v85) v1 ==== 133+0+0 (secure 0 0 0) 0x7f44480459e0 con 0x7f44540b7840 2022-08-01T19:40:59.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.875+0000 7f44596c0700 1 -- 172.21.15.17:0/2161175349 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4434053a30 msgr2=0x7f4434055ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:59.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.875+0000 7f44596c0700 1 --2- 172.21.15.17:0/2161175349 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4434053a30 0x7f4434055ef0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f444401b000 tx=0x7f4444016000).stop 2022-08-01T19:40:59.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.875+0000 7f44596c0700 1 -- 172.21.15.17:0/2161175349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44540b7840 msgr2=0x7f4454156ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:40:59.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.875+0000 7f44596c0700 1 --2- 172.21.15.17:0/2161175349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44540b7840 0x7f4454156ab0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7f4454069e10 tx=0x7f4448007c40).stop 2022-08-01T19:40:59.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.875+0000 7f44596c0700 1 -- 172.21.15.17:0/2161175349 shutdown_connections 2022-08-01T19:40:59.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.875+0000 7f44596c0700 1 --2- 172.21.15.17:0/2161175349 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4434053a30 0x7f4434055ef0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:59.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.875+0000 7f44596c0700 1 --2- 172.21.15.17:0/2161175349 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f445414b040 0x7f4454156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:59.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.875+0000 7f44596c0700 1 --2- 172.21.15.17:0/2161175349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44540b7840 0x7f4454156ab0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:59.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.875+0000 7f44596c0700 1 --2- 172.21.15.17:0/2161175349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44540b3070 0x7f44540a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:40:59.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.875+0000 7f44596c0700 1 -- 172.21.15.17:0/2161175349 >> 172.21.15.17:0/2161175349 conn(0x7f445405f840 msgr2=0x7f445405ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:40:59.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.875+0000 7f44596c0700 1 -- 172.21.15.17:0/2161175349 shutdown_connections 2022-08-01T19:40:59.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:40:59.875+0000 7f44596c0700 1 -- 172.21.15.17:0/2161175349 wait complete. 2022-08-01T19:40:59.897 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' created 2022-08-01T19:40:59.898 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-08-01T19:41:00.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.019+0000 7fe2ea6b3700 1 Processor -- start 2022-08-01T19:41:00.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.019+0000 7fe2ea6b3700 1 -- start start 2022-08-01T19:41:00.027 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.019+0000 7fe2ea6b3700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe2e40b08a0 0x7fe2e4142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:00.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.019+0000 7fe2ea6b3700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fe2e40b3260 0x7fe2e41433a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:00.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.019+0000 7fe2ea6b3700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe2e40ad000 0x7fe2e405f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:00.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.019+0000 7fe2ea6b3700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2e40b0c80 con 0x7fe2e40b08a0 2022-08-01T19:41:00.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.019+0000 7fe2ea6b3700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2e40ae810 con 0x7fe2e40b3260 2022-08-01T19:41:00.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.019+0000 7fe2ea6b3700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e40aeab0 con 0x7fe2e40ad000 2022-08-01T19:41:00.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.019+0000 7fe2e96b1700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe2e40b08a0 0x7fe2e4142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:00.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.019+0000 7fe2e8eb0700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fe2e40b3260 0x7fe2e41433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:00.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.019+0000 7fe2e8eb0700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fe2e40b3260 0x7fe2e41433a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:39178/0 (socket says 172.21.15.17:39178) 2022-08-01T19:41:00.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.019+0000 7fe2e96b1700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe2e40b08a0 0x7fe2e4142eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41258/0 (socket says 172.21.15.17:41258) 2022-08-01T19:41:00.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.019+0000 7fe2e8eb0700 1 -- 172.21.15.17:0/584474393 learned_addr learned my addr 172.21.15.17:0/584474393 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:00.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.019+0000 7fe2e9eb2700 1 --2- 172.21.15.17:0/584474393 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe2e40ad000 0x7fe2e405f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:00.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.019+0000 7fe2e8eb0700 1 -- 172.21.15.17:0/584474393 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe2e40ad000 msgr2=0x7fe2e405f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:00.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.019+0000 7fe2e8eb0700 1 --2- 172.21.15.17:0/584474393 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe2e40ad000 0x7fe2e405f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:00.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.019+0000 7fe2e8eb0700 1 -- 172.21.15.17:0/584474393 >> v2:172.21.15.17:3300/0 conn(0x7fe2e40b08a0 msgr2=0x7fe2e4142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:00.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.019+0000 7fe2e8eb0700 1 --2- 172.21.15.17:0/584474393 >> v2:172.21.15.17:3300/0 conn(0x7fe2e40b08a0 0x7fe2e4142eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:00.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.019+0000 7fe2e8eb0700 1 -- 172.21.15.17:0/584474393 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e40aed50 con 0x7fe2e40b3260 2022-08-01T19:41:00.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.019+0000 7fe2e8eb0700 1 --2- 172.21.15.17:0/584474393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2e40b3260 0x7fe2e41433a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7fe2d4002f70 tx=0x7fe2d4006eb0).ready entity=mon.1 client_cookie=a56d80f7daa8a4eb server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:00.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.019+0000 7fe2d3fff700 1 -- 172.21.15.17:0/584474393 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2d400d570 con 0x7fe2e40b3260 2022-08-01T19:41:00.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.019+0000 7fe2d3fff700 1 -- 172.21.15.17:0/584474393 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d4000e90 con 0x7fe2e40b3260 2022-08-01T19:41:00.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.019+0000 7fe2d3fff700 1 -- 172.21.15.17:0/584474393 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2d400d900 con 0x7fe2e40b3260 2022-08-01T19:41:00.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.023+0000 7fe2ea6b3700 1 -- 172.21.15.17:0/584474393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2e40b3260 msgr2=0x7fe2e41433a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:00.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.023+0000 7fe2ea6b3700 1 --2- 172.21.15.17:0/584474393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2e40b3260 0x7fe2e41433a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7fe2d4002f70 tx=0x7fe2d4006eb0).stop 2022-08-01T19:41:00.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.023+0000 7fe2ea6b3700 1 -- 172.21.15.17:0/584474393 shutdown_connections 2022-08-01T19:41:00.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.023+0000 7fe2ea6b3700 1 --2- 172.21.15.17:0/584474393 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe2e40ad000 0x7fe2e405f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:00.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.023+0000 7fe2ea6b3700 1 --2- 172.21.15.17:0/584474393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2e40b3260 0x7fe2e41433a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:00.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.023+0000 7fe2ea6b3700 1 --2- 172.21.15.17:0/584474393 >> v2:172.21.15.17:3300/0 conn(0x7fe2e40b08a0 0x7fe2e4142eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:00.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.023+0000 7fe2ea6b3700 1 -- 172.21.15.17:0/584474393 >> 172.21.15.17:0/584474393 conn(0x7fe2e40612d0 msgr2=0x7fe2e4062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:00.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.023+0000 7fe2ea6b3700 1 -- 172.21.15.17:0/584474393 shutdown_connections 2022-08-01T19:41:00.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.023+0000 7fe2ea6b3700 1 -- 172.21.15.17:0/584474393 wait complete. 2022-08-01T19:41:00.040 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.023+0000 7fe2ea6b3700 1 Processor -- start 2022-08-01T19:41:00.040 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.023+0000 7fe2ea6b3700 1 -- start start 2022-08-01T19:41:00.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.023+0000 7fe2ea6b3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2e40ad000 0x7fe2e4138870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:00.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.023+0000 7fe2ea6b3700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe2e40b08a0 0x7fe2e4063c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:00.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.023+0000 7fe2ea6b3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2e40b3260 0x7fe2e414d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:00.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.023+0000 7fe2ea6b3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e40b5310 con 0x7fe2e40ad000 2022-08-01T19:41:00.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.023+0000 7fe2ea6b3700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e40a9a70 con 0x7fe2e40b3260 2022-08-01T19:41:00.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.023+0000 7fe2ea6b3700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e40a9d10 con 0x7fe2e40b08a0 2022-08-01T19:41:00.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.023+0000 7fe2e96b1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2e40ad000 0x7fe2e4138870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:00.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.023+0000 7fe2e96b1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2e40ad000 0x7fe2e4138870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41262/0 (socket says 172.21.15.17:41262) 2022-08-01T19:41:00.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.023+0000 7fe2e96b1700 1 -- 172.21.15.17:0/985774376 learned_addr learned my addr 172.21.15.17:0/985774376 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:00.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.023+0000 7fe2e9eb2700 1 --2- 172.21.15.17:0/985774376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2e40b3260 0x7fe2e414d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:00.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.023+0000 7fe2e8eb0700 1 --2- 172.21.15.17:0/985774376 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe2e40b08a0 0x7fe2e4063c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:00.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.027+0000 7fe2e96b1700 1 -- 172.21.15.17:0/985774376 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe2e40b08a0 msgr2=0x7fe2e4063c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:00.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.027+0000 7fe2e96b1700 1 --2- 172.21.15.17:0/985774376 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe2e40b08a0 0x7fe2e4063c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:00.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.027+0000 7fe2e96b1700 1 -- 172.21.15.17:0/985774376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2e40b3260 msgr2=0x7fe2e414d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:00.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.027+0000 7fe2e96b1700 1 --2- 172.21.15.17:0/985774376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2e40b3260 0x7fe2e414d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:00.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.027+0000 7fe2e96b1700 1 -- 172.21.15.17:0/985774376 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d400b1e0 con 0x7fe2e40ad000 2022-08-01T19:41:00.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.027+0000 7fe2e96b1700 1 --2- 172.21.15.17:0/985774376 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2e40ad000 0x7fe2e4138870 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7fe2e000c000 tx=0x7fe2e00091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:00.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.027+0000 7fe2d27fc700 1 -- 172.21.15.17:0/985774376 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2e000c850 con 0x7fe2e40ad000 2022-08-01T19:41:00.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.027+0000 7fe2d27fc700 1 -- 172.21.15.17:0/985774376 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e000c9b0 con 0x7fe2e40ad000 2022-08-01T19:41:00.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.027+0000 7fe2ea6b3700 1 -- 172.21.15.17:0/985774376 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e4154600 con 0x7fe2e40ad000 2022-08-01T19:41:00.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.027+0000 7fe2d27fc700 1 -- 172.21.15.17:0/985774376 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2e000cd10 con 0x7fe2e40ad000 2022-08-01T19:41:00.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.027+0000 7fe2ea6b3700 1 -- 172.21.15.17:0/985774376 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e40b5000 con 0x7fe2e40ad000 2022-08-01T19:41:00.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.027+0000 7fe2ea6b3700 1 -- 172.21.15.17:0/985774376 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e40aef30 con 0x7fe2e40ad000 2022-08-01T19:41:00.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.027+0000 7fe2d27fc700 1 -- 172.21.15.17:0/985774376 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe2e0022070 con 0x7fe2e40ad000 2022-08-01T19:41:00.063 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.031+0000 7fe2d27fc700 1 --2- 172.21.15.17:0/985774376 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe2c40539e0 0x7fe2c4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:00.063 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.031+0000 7fe2d27fc700 1 -- 172.21.15.17:0/985774376 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5954+0+0 (secure 0 0 0) 0x7fe2e006c620 con 0x7fe2e40ad000 2022-08-01T19:41:00.064 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.031+0000 7fe2e8eb0700 1 --2- 172.21.15.17:0/985774376 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe2c40539e0 0x7fe2c4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:00.064 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.031+0000 7fe2e8eb0700 1 --2- 172.21.15.17:0/985774376 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe2c40539e0 0x7fe2c4055ea0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7fe2d401c000 tx=0x7fe2d4015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:00.064 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.039+0000 7fe2d27fc700 1 -- 172.21.15.17:0/985774376 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe2e0043510 con 0x7fe2e40ad000 2022-08-01T19:41:00.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.171+0000 7fe2ea6b3700 1 -- 172.21.15.17:0/985774376 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fe2e40574e0 con 0x7fe2e40ad000 2022-08-01T19:41:00.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.875+0000 7fe2d27fc700 1 -- 172.21.15.17:0/985774376 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v86) v1 ==== 125+0+0 (secure 0 0 0) 0x7fe2e006ae60 con 0x7fe2e40ad000 2022-08-01T19:41:00.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.883+0000 7fe2ea6b3700 1 -- 172.21.15.17:0/985774376 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe2c40539e0 msgr2=0x7fe2c4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:00.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.883+0000 7fe2ea6b3700 1 --2- 172.21.15.17:0/985774376 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe2c40539e0 0x7fe2c4055ea0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7fe2d401c000 tx=0x7fe2d4015000).stop 2022-08-01T19:41:00.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.883+0000 7fe2ea6b3700 1 -- 172.21.15.17:0/985774376 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2e40ad000 msgr2=0x7fe2e4138870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:00.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.883+0000 7fe2ea6b3700 1 --2- 172.21.15.17:0/985774376 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2e40ad000 0x7fe2e4138870 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7fe2e000c000 tx=0x7fe2e00091d0).stop 2022-08-01T19:41:00.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.883+0000 7fe2ea6b3700 1 -- 172.21.15.17:0/985774376 shutdown_connections 2022-08-01T19:41:00.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.883+0000 7fe2ea6b3700 1 --2- 172.21.15.17:0/985774376 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe2c40539e0 0x7fe2c4055ea0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:00.899 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.883+0000 7fe2ea6b3700 1 --2- 172.21.15.17:0/985774376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2e40b3260 0x7fe2e414d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:00.899 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.883+0000 7fe2ea6b3700 1 --2- 172.21.15.17:0/985774376 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe2e40b08a0 0x7fe2e4063c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:00.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.883+0000 7fe2ea6b3700 1 --2- 172.21.15.17:0/985774376 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2e40ad000 0x7fe2e4138870 secure :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 rx=0x7fe2e000c000 tx=0x7fe2e00091d0).stop 2022-08-01T19:41:00.901 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.883+0000 7fe2ea6b3700 1 -- 172.21.15.17:0/985774376 >> 172.21.15.17:0/985774376 conn(0x7fe2e40612d0 msgr2=0x7fe2e4143110 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:00.902 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.883+0000 7fe2ea6b3700 1 -- 172.21.15.17:0/985774376 shutdown_connections 2022-08-01T19:41:00.903 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:00.887+0000 7fe2ea6b3700 1 -- 172.21.15.17:0/985774376 wait complete. 2022-08-01T19:41:00.903 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' created 2022-08-01T19:41:00.906 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-08-01T19:41:01.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.011+0000 7ff68599d700 1 Processor -- start 2022-08-01T19:41:01.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.011+0000 7ff68599d700 1 -- start start 2022-08-01T19:41:01.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.011+0000 7ff68599d700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ff680139e50 0x7ff6800acd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:01.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.011+0000 7ff68599d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff68013a230 0x7ff6800b6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:01.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.011+0000 7ff68599d700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6800ad200 con 0x7ff6800b2350 2022-08-01T19:41:01.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.011+0000 7ff68599d700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6800b28c0 con 0x7ff68013a230 2022-08-01T19:41:01.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.011+0000 7ff68599d700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff680060ed0 con 0x7ff680139e50 2022-08-01T19:41:01.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.011+0000 7ff67f7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff68013a230 0x7ff6800b6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:01.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.011+0000 7ff67effd700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ff680139e50 0x7ff6800acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:01.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.015+0000 7ff67f7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff68013a230 0x7ff6800b6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41274/0 (socket says 172.21.15.17:41274) 2022-08-01T19:41:01.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.015+0000 7ff67e7fc700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7ff6800b2350 0x7ff680143680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:50266/0 (socket says 172.21.15.17:50266) 2022-08-01T19:41:01.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.015+0000 7ff67f7fe700 1 -- 172.21.15.17:0/700911119 learned_addr learned my addr 172.21.15.17:0/700911119 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:01.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.015+0000 7ff67dffb700 1 -- 172.21.15.17:0/700911119 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2248609354 0 0) 0x7ff674001110 con 0x7ff6800b2350 2022-08-01T19:41:01.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.015+0000 7ff67dffb700 1 -- 172.21.15.17:0/700911119 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff664003260 con 0x7ff6800b2350 2022-08-01T19:41:01.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.015+0000 7ff67f7fe700 1 -- 172.21.15.17:0/700911119 >> v2:172.21.15.77:3300/0 conn(0x7ff680139e50 msgr2=0x7ff6800acd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:01.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.015+0000 7ff67f7fe700 1 --2- 172.21.15.17:0/700911119 >> v2:172.21.15.77:3300/0 conn(0x7ff680139e50 0x7ff6800acd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:01.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.015+0000 7ff67f7fe700 1 -- 172.21.15.17:0/700911119 >> v1:172.21.15.77:6789/0 conn(0x7ff6800b2350 legacy=0x7ff680143680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:01.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.015+0000 7ff67f7fe700 1 -- 172.21.15.17:0/700911119 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff680061130 con 0x7ff68013a230 2022-08-01T19:41:01.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.015+0000 7ff67f7fe700 1 --2- 172.21.15.17:0/700911119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff68013a230 0x7ff6800b6750 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 rx=0x7ff6780032f0 tx=0x7ff67800b530).ready entity=mon.0 client_cookie=27b2869454794ecc server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:01.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.015+0000 7ff67dffb700 1 -- 172.21.15.17:0/700911119 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff67800dd40 con 0x7ff68013a230 2022-08-01T19:41:01.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.015+0000 7ff67dffb700 1 -- 172.21.15.17:0/700911119 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff678010470 con 0x7ff68013a230 2022-08-01T19:41:01.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.015+0000 7ff67dffb700 1 -- 172.21.15.17:0/700911119 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff67800dea0 con 0x7ff68013a230 2022-08-01T19:41:01.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.015+0000 7ff68599d700 1 -- 172.21.15.17:0/700911119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff68013a230 msgr2=0x7ff6800b6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:01.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.015+0000 7ff68599d700 1 --2- 172.21.15.17:0/700911119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff68013a230 0x7ff6800b6750 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 rx=0x7ff6780032f0 tx=0x7ff67800b530).stop 2022-08-01T19:41:01.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.015+0000 7ff68599d700 1 -- 172.21.15.17:0/700911119 shutdown_connections 2022-08-01T19:41:01.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.015+0000 7ff68599d700 1 --2- 172.21.15.17:0/700911119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff68013a230 0x7ff6800b6750 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:01.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.015+0000 7ff68599d700 1 --2- 172.21.15.17:0/700911119 >> v2:172.21.15.77:3300/0 conn(0x7ff680139e50 0x7ff6800acd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:01.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.015+0000 7ff68599d700 1 -- 172.21.15.17:0/700911119 >> 172.21.15.17:0/700911119 conn(0x7ff6800b3610 msgr2=0x7ff6800b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:01.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.015+0000 7ff68599d700 1 -- 172.21.15.17:0/700911119 shutdown_connections 2022-08-01T19:41:01.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.015+0000 7ff68599d700 1 -- 172.21.15.17:0/700911119 wait complete. 2022-08-01T19:41:01.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.019+0000 7ff68599d700 1 Processor -- start 2022-08-01T19:41:01.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.019+0000 7ff68599d700 1 -- start start 2022-08-01T19:41:01.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.019+0000 7ff68599d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6800b2350 0x7ff680143c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:01.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.019+0000 7ff68599d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff680139e50 0x7ff68013a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:01.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.019+0000 7ff68599d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff68013a230 0x7ff68013ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:01.040 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.019+0000 7ff68599d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6800aed00 con 0x7ff680139e50 2022-08-01T19:41:01.040 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.019+0000 7ff68599d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68013c830 con 0x7ff68013a230 2022-08-01T19:41:01.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.019+0000 7ff68599d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff68013cad0 con 0x7ff6800b2350 2022-08-01T19:41:01.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.023+0000 7ff67effd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6800b2350 0x7ff680143c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:01.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.023+0000 7ff67e7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff680139e50 0x7ff68013a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:01.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.023+0000 7ff67e7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff680139e50 0x7ff68013a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41278/0 (socket says 172.21.15.17:41278) 2022-08-01T19:41:01.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.023+0000 7ff67e7fc700 1 -- 172.21.15.17:0/2365728932 learned_addr learned my addr 172.21.15.17:0/2365728932 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:01.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.023+0000 7ff67f7fe700 1 --2- 172.21.15.17:0/2365728932 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff68013a230 0x7ff68013ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:01.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.023+0000 7ff67f7fe700 1 -- 172.21.15.17:0/2365728932 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6800b2350 msgr2=0x7ff680143c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:01.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.023+0000 7ff67f7fe700 1 --2- 172.21.15.17:0/2365728932 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6800b2350 0x7ff680143c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:01.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.023+0000 7ff67f7fe700 1 -- 172.21.15.17:0/2365728932 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff680139e50 msgr2=0x7ff68013a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:01.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.023+0000 7ff67f7fe700 1 --2- 172.21.15.17:0/2365728932 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff680139e50 0x7ff68013a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:01.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.023+0000 7ff67f7fe700 1 -- 172.21.15.17:0/2365728932 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67800b7d0 con 0x7ff68013a230 2022-08-01T19:41:01.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.023+0000 7ff67f7fe700 1 --2- 172.21.15.17:0/2365728932 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff68013a230 0x7ff68013ab00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7ff6780032f0 tx=0x7ff6801400b0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:01.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.023+0000 7ff66ffff700 1 -- 172.21.15.17:0/2365728932 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff67801d070 con 0x7ff68013a230 2022-08-01T19:41:01.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.023+0000 7ff66ffff700 1 -- 172.21.15.17:0/2365728932 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff678018800 con 0x7ff68013a230 2022-08-01T19:41:01.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.023+0000 7ff66ffff700 1 -- 172.21.15.17:0/2365728932 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff678018b60 con 0x7ff68013a230 2022-08-01T19:41:01.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.023+0000 7ff68599d700 1 -- 172.21.15.17:0/2365728932 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff68013d4d0 con 0x7ff68013a230 2022-08-01T19:41:01.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.023+0000 7ff68599d700 1 -- 172.21.15.17:0/2365728932 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff68013d190 con 0x7ff68013a230 2022-08-01T19:41:01.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.027+0000 7ff68599d700 1 -- 172.21.15.17:0/2365728932 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff68013cc90 con 0x7ff68013a230 2022-08-01T19:41:01.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.027+0000 7ff66ffff700 1 -- 172.21.15.17:0/2365728932 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff678018cc0 con 0x7ff68013a230 2022-08-01T19:41:01.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.027+0000 7ff66ffff700 1 --2- 172.21.15.17:0/2365728932 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff65c053a70 0x7ff65c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:01.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.027+0000 7ff66ffff700 1 -- 172.21.15.17:0/2365728932 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6304+0+0 (secure 0 0 0) 0x7ff67806d110 con 0x7ff68013a230 2022-08-01T19:41:01.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.027+0000 7ff67effd700 1 --2- 172.21.15.17:0/2365728932 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff65c053a70 0x7ff65c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:01.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.027+0000 7ff67effd700 1 --2- 172.21.15.17:0/2365728932 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff65c053a70 0x7ff65c055f30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7ff67000b000 tx=0x7ff670006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:01.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.035+0000 7ff66ffff700 1 -- 172.21.15.17:0/2365728932 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff678044000 con 0x7ff68013a230 2022-08-01T19:41:01.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.199+0000 7ff68599d700 1 -- 172.21.15.17:0/2365728932 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7ff6800613d0 con 0x7ff68013a230 2022-08-01T19:41:01.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.215+0000 7ff66ffff700 1 -- 172.21.15.17:0/2365728932 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 15 and data pool 16 v39) v1 ==== 189+0+0 (secure 0 0 0) 0x7ff678047620 con 0x7ff68013a230 2022-08-01T19:41:01.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.219+0000 7ff68599d700 1 -- 172.21.15.17:0/2365728932 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff65c053a70 msgr2=0x7ff65c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:01.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.219+0000 7ff68599d700 1 --2- 172.21.15.17:0/2365728932 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff65c053a70 0x7ff65c055f30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7ff67000b000 tx=0x7ff670006df0).stop 2022-08-01T19:41:01.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.219+0000 7ff68599d700 1 -- 172.21.15.17:0/2365728932 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff68013a230 msgr2=0x7ff68013ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:01.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.219+0000 7ff68599d700 1 --2- 172.21.15.17:0/2365728932 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff68013a230 0x7ff68013ab00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7ff6780032f0 tx=0x7ff6801400b0).stop 2022-08-01T19:41:01.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.219+0000 7ff68599d700 1 -- 172.21.15.17:0/2365728932 shutdown_connections 2022-08-01T19:41:01.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.219+0000 7ff68599d700 1 --2- 172.21.15.17:0/2365728932 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff65c053a70 0x7ff65c055f30 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:01.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.219+0000 7ff68599d700 1 --2- 172.21.15.17:0/2365728932 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff68013a230 0x7ff68013ab00 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:01.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.219+0000 7ff68599d700 1 --2- 172.21.15.17:0/2365728932 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff680139e50 0x7ff68013a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:01.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.219+0000 7ff68599d700 1 --2- 172.21.15.17:0/2365728932 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6800b2350 0x7ff680143c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:01.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.219+0000 7ff68599d700 1 -- 172.21.15.17:0/2365728932 >> 172.21.15.17:0/2365728932 conn(0x7ff6800b3610 msgr2=0x7ff6800605b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:01.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.219+0000 7ff68599d700 1 -- 172.21.15.17:0/2365728932 shutdown_connections 2022-08-01T19:41:01.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.219+0000 7ff68599d700 1 -- 172.21.15.17:0/2365728932 wait complete. 2022-08-01T19:41:01.229 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 15 and data pool 16 2022-08-01T19:41:01.241 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:41:01.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b0e2d7700 1 Processor -- start 2022-08-01T19:41:01.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b0e2d7700 1 -- start start 2022-08-01T19:41:01.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b0e2d7700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f7b080b1160 0x7f7b081448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:01.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b0e2d7700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b080b04e0 con 0x7f7b08148530 2022-08-01T19:41:01.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b0e2d7700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b080b0780 con 0x7f7b080b1f40 2022-08-01T19:41:01.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b0e2d7700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b080a9250 con 0x7f7b080b1160 2022-08-01T19:41:01.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b077fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f7b08148530 0x7f7b080a7250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33478/0 (socket says 172.21.15.17:33478) 2022-08-01T19:41:01.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b077fe700 1 -- 172.21.15.17:0/1423967822 learned_addr learned my addr 172.21.15.17:0/1423967822 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:01.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b0c874700 1 --2- 172.21.15.17:0/1423967822 >> v2:172.21.15.77:3300/0 conn(0x7f7b080b1160 0x7f7b081448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:01.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b06ffd700 1 -- 172.21.15.17:0/1423967822 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 310112472 0 0) 0x7f7b080b04e0 con 0x7f7b08148530 2022-08-01T19:41:01.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b06ffd700 1 -- 172.21.15.17:0/1423967822 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ae8003260 con 0x7f7b08148530 2022-08-01T19:41:01.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b06ffd700 1 -- 172.21.15.17:0/1423967822 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3056747537 0 0) 0x7f7ae8003260 con 0x7f7b08148530 2022-08-01T19:41:01.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b06ffd700 1 -- 172.21.15.17:0/1423967822 >> v2:172.21.15.77:3300/0 conn(0x7f7b080b1160 msgr2=0x7f7b081448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:01.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b06ffd700 1 --2- 172.21.15.17:0/1423967822 >> v2:172.21.15.77:3300/0 conn(0x7f7b080b1160 0x7f7b081448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:01.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b06ffd700 1 -- 172.21.15.17:0/1423967822 >> v1:172.21.15.77:6789/0 conn(0x7f7b080b1f40 legacy=0x7f7b080b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:01.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b06ffd700 1 -- 172.21.15.17:0/1423967822 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b080b0a20 con 0x7f7b08148530 2022-08-01T19:41:01.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b06ffd700 1 -- 172.21.15.17:0/1423967822 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7f7b000019e0 con 0x7f7b08148530 2022-08-01T19:41:01.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b06ffd700 1 -- 172.21.15.17:0/1423967822 >> v1:172.21.15.17:6789/0 conn(0x7f7b08148530 legacy=0x7f7b080a7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:01.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b06ffd700 1 --2- 172.21.15.17:0/1423967822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ae8003750 0x7f7ae8003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:01.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b06ffd700 1 --2- 172.21.15.17:0/1423967822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ae8004500 0x7f7ae8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:01.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b06ffd700 1 --2- 172.21.15.17:0/1423967822 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7ae8004e70 0x7f7ae800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:01.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b06ffd700 1 -- 172.21.15.17:0/1423967822 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae800bc40 con 0x7f7ae8003750 2022-08-01T19:41:01.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b06ffd700 1 -- 172.21.15.17:0/1423967822 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae800bdb0 con 0x7f7ae8004500 2022-08-01T19:41:01.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b06ffd700 1 -- 172.21.15.17:0/1423967822 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae800bf70 con 0x7f7ae8004e70 2022-08-01T19:41:01.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b07fff700 1 --2- 172.21.15.17:0/1423967822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ae8003750 0x7f7ae8003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:01.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b077fe700 1 --2- 172.21.15.17:0/1423967822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ae8004500 0x7f7ae8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:01.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b0c874700 1 --2- 172.21.15.17:0/1423967822 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7ae8004e70 0x7f7ae800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:01.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b07fff700 1 -- 172.21.15.17:0/1423967822 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7ae8004e70 msgr2=0x7f7ae800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:01.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b07fff700 1 --2- 172.21.15.17:0/1423967822 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7ae8004e70 0x7f7ae800b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:01.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b07fff700 1 -- 172.21.15.17:0/1423967822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ae8004500 msgr2=0x7f7ae8004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:01.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b07fff700 1 --2- 172.21.15.17:0/1423967822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ae8004500 0x7f7ae8004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:01.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b07fff700 1 -- 172.21.15.17:0/1423967822 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae8003260 con 0x7f7ae8003750 2022-08-01T19:41:01.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b0c874700 1 -- 172.21.15.17:0/1423967822 reap_dead start 2022-08-01T19:41:01.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b07fff700 1 --2- 172.21.15.17:0/1423967822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ae8003750 0x7f7ae8003b30 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7f7af0001020 tx=0x7f7af00079e0).ready entity=mon.0 client_cookie=dcaf2d8c6ef582f8 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:01.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b06ffd700 1 -- 172.21.15.17:0/1423967822 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7af0007cf0 con 0x7f7ae8003750 2022-08-01T19:41:01.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b06ffd700 1 -- 172.21.15.17:0/1423967822 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af0007030 con 0x7f7ae8003750 2022-08-01T19:41:01.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.335+0000 7f7b06ffd700 1 -- 172.21.15.17:0/1423967822 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7af0007390 con 0x7f7ae8003750 2022-08-01T19:41:01.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.339+0000 7f7b0e2d7700 1 -- 172.21.15.17:0/1423967822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ae8003750 msgr2=0x7f7ae8003b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:01.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.339+0000 7f7b0e2d7700 1 --2- 172.21.15.17:0/1423967822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ae8003750 0x7f7ae8003b30 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7f7af0001020 tx=0x7f7af00079e0).stop 2022-08-01T19:41:01.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.339+0000 7f7b0e2d7700 1 -- 172.21.15.17:0/1423967822 shutdown_connections 2022-08-01T19:41:01.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.339+0000 7f7b0e2d7700 1 --2- 172.21.15.17:0/1423967822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ae8003750 0x7f7ae8003b30 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:01.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.339+0000 7f7b0e2d7700 1 -- 172.21.15.17:0/1423967822 >> 172.21.15.17:0/1423967822 conn(0x7f7b0805f840 msgr2=0x7f7b0805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:01.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.339+0000 7f7b0e2d7700 1 -- 172.21.15.17:0/1423967822 shutdown_connections 2022-08-01T19:41:01.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.339+0000 7f7b0e2d7700 1 -- 172.21.15.17:0/1423967822 wait complete. 2022-08-01T19:41:01.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.339+0000 7f7b0e2d7700 1 Processor -- start 2022-08-01T19:41:01.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.339+0000 7f7b0e2d7700 1 -- start start 2022-08-01T19:41:01.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.339+0000 7f7b0e2d7700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7ae8003750 0x7f7b081572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:01.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.339+0000 7f7b0e2d7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b080ad4b0 0x7f7b08149e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:01.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.339+0000 7f7b0e2d7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b0814b590 0x7f7b08145730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:01.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.339+0000 7f7b0e2d7700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0814a910 con 0x7f7b080ad4b0 2022-08-01T19:41:01.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.339+0000 7f7b0e2d7700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0814abb0 con 0x7f7b0814b590 2022-08-01T19:41:01.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.339+0000 7f7b0e2d7700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0814ae50 con 0x7f7ae8003750 2022-08-01T19:41:01.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.339+0000 7f7b077fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b080ad4b0 0x7f7b08149e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:01.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.339+0000 7f7b077fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b080ad4b0 0x7f7b08149e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41298/0 (socket says 172.21.15.17:41298) 2022-08-01T19:41:01.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.339+0000 7f7b077fe700 1 -- 172.21.15.17:0/3588131131 learned_addr learned my addr 172.21.15.17:0/3588131131 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:01.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.339+0000 7f7b0c874700 1 --2- 172.21.15.17:0/3588131131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b0814b590 0x7f7b08145730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:01.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.339+0000 7f7b07fff700 1 --2- 172.21.15.17:0/3588131131 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7ae8003750 0x7f7b081572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:01.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.343+0000 7f7b077fe700 1 -- 172.21.15.17:0/3588131131 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7ae8003750 msgr2=0x7f7b081572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:01.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.343+0000 7f7b077fe700 1 --2- 172.21.15.17:0/3588131131 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7ae8003750 0x7f7b081572a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:01.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.343+0000 7f7b077fe700 1 -- 172.21.15.17:0/3588131131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b0814b590 msgr2=0x7f7b08145730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:01.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.343+0000 7f7b077fe700 1 --2- 172.21.15.17:0/3588131131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b0814b590 0x7f7b08145730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:01.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.343+0000 7f7b077fe700 1 -- 172.21.15.17:0/3588131131 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0814b0f0 con 0x7f7b080ad4b0 2022-08-01T19:41:01.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.343+0000 7f7b0c874700 1 --2- 172.21.15.17:0/3588131131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b0814b590 0x7f7b08145730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:01.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.343+0000 7f7b077fe700 1 --2- 172.21.15.17:0/3588131131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b080ad4b0 0x7f7b08149e40 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7f7b0000b000 tx=0x7f7b080b0910).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:01.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.343+0000 7f7b057fa700 1 -- 172.21.15.17:0/3588131131 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b0000ca10 con 0x7f7b080ad4b0 2022-08-01T19:41:01.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.343+0000 7f7b057fa700 1 -- 172.21.15.17:0/3588131131 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b0000cb70 con 0x7f7b080ad4b0 2022-08-01T19:41:01.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.343+0000 7f7b057fa700 1 -- 172.21.15.17:0/3588131131 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b000019a0 con 0x7f7b080ad4b0 2022-08-01T19:41:01.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.343+0000 7f7b0e2d7700 1 -- 172.21.15.17:0/3588131131 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b08145c20 con 0x7f7b080ad4b0 2022-08-01T19:41:01.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.343+0000 7f7b0e2d7700 1 -- 172.21.15.17:0/3588131131 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b0813fdb0 con 0x7f7b080ad4b0 2022-08-01T19:41:01.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.343+0000 7f7b0e2d7700 1 -- 172.21.15.17:0/3588131131 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b080b0cc0 con 0x7f7b080ad4b0 2022-08-01T19:41:01.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.343+0000 7f7b057fa700 1 -- 172.21.15.17:0/3588131131 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7b00014430 con 0x7f7b080ad4b0 2022-08-01T19:41:01.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.343+0000 7f7b057fa700 1 --2- 172.21.15.17:0/3588131131 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7aec053b00 0x7f7aec055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:01.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.343+0000 7f7b057fa700 1 -- 172.21.15.17:0/3588131131 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f7b0006c1e0 con 0x7f7b080ad4b0 2022-08-01T19:41:01.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.343+0000 7f7b07fff700 1 --2- 172.21.15.17:0/3588131131 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7aec053b00 0x7f7aec055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:01.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.343+0000 7f7b07fff700 1 --2- 172.21.15.17:0/3588131131 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7aec053b00 0x7f7aec055fc0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f7af001b000 tx=0x7f7af0015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:01.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.351+0000 7f7b057fa700 1 -- 172.21.15.17:0/3588131131 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7b00042f40 con 0x7f7b080ad4b0 2022-08-01T19:41:01.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.479+0000 7f7b0e2d7700 1 -- 172.21.15.17:0/3588131131 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b080574e0 con 0x7f7b080ad4b0 2022-08-01T19:41:01.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.479+0000 7f7b057fa700 1 -- 172.21.15.17:0/3588131131 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16287 (secure 0 0 0) 0x7f7b00046560 con 0x7f7b080ad4b0 2022-08-01T19:41:01.486 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:01.487 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":87,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:41:01.207172+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:40:59.229815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2022-08-01T19:41:00.225322+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:41:01.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.483+0000 7f7b0e2d7700 1 -- 172.21.15.17:0/3588131131 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7aec053b00 msgr2=0x7f7aec055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:01.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.483+0000 7f7b0e2d7700 1 --2- 172.21.15.17:0/3588131131 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7aec053b00 0x7f7aec055fc0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f7af001b000 tx=0x7f7af0015000).stop 2022-08-01T19:41:01.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.483+0000 7f7b0e2d7700 1 -- 172.21.15.17:0/3588131131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b080ad4b0 msgr2=0x7f7b08149e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:01.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.483+0000 7f7b0e2d7700 1 --2- 172.21.15.17:0/3588131131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b080ad4b0 0x7f7b08149e40 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7f7b0000b000 tx=0x7f7b080b0910).stop 2022-08-01T19:41:01.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.483+0000 7f7b0e2d7700 1 -- 172.21.15.17:0/3588131131 shutdown_connections 2022-08-01T19:41:01.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.483+0000 7f7b0e2d7700 1 --2- 172.21.15.17:0/3588131131 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7aec053b00 0x7f7aec055fc0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:01.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.483+0000 7f7b0e2d7700 1 --2- 172.21.15.17:0/3588131131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b0814b590 0x7f7b08145730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:01.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.483+0000 7f7b0e2d7700 1 --2- 172.21.15.17:0/3588131131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b080ad4b0 0x7f7b08149e40 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:01.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.483+0000 7f7b0e2d7700 1 --2- 172.21.15.17:0/3588131131 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7ae8003750 0x7f7b081572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:01.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.483+0000 7f7b0e2d7700 1 -- 172.21.15.17:0/3588131131 >> 172.21.15.17:0/3588131131 conn(0x7f7b0805f840 msgr2=0x7f7b08147d70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:01.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.483+0000 7f7b0e2d7700 1 -- 172.21.15.17:0/3588131131 shutdown_connections 2022-08-01T19:41:01.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.483+0000 7f7b0e2d7700 1 -- 172.21.15.17:0/3588131131 wait complete. 2022-08-01T19:41:01.506 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:41:01.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.603+0000 7fdf0eb25700 1 Processor -- start 2022-08-01T19:41:01.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.603+0000 7fdf0eb25700 1 -- start start 2022-08-01T19:41:01.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.603+0000 7fdf0eb25700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdf080b65c0 0x7fdf08061600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:01.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.603+0000 7fdf0eb25700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fdf080b41f0 0x7fdf08061af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:01.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.603+0000 7fdf0eb25700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf080b2100 con 0x7fdf08147d80 2022-08-01T19:41:01.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.603+0000 7fdf0eb25700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf080b1630 con 0x7fdf080b41f0 2022-08-01T19:41:01.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.603+0000 7fdf0eb25700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf08061fe0 con 0x7fdf080b65c0 2022-08-01T19:41:01.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.603+0000 7fdf0db23700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdf080b65c0 0x7fdf08061600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:01.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.603+0000 7fdf0d322700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fdf080b41f0 0x7fdf08061af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:01.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.603+0000 7fdf0db23700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdf080b65c0 0x7fdf08061600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:37404/0 (socket says 172.21.15.17:37404) 2022-08-01T19:41:01.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.603+0000 7fdf0db23700 1 -- 172.21.15.17:0/390225738 learned_addr learned my addr 172.21.15.17:0/390225738 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:01.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.603+0000 7fdf0db23700 1 -- 172.21.15.17:0/390225738 >> v2:172.21.15.77:3300/0 conn(0x7fdf080b41f0 msgr2=0x7fdf08061af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:01.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.603+0000 7fdf0db23700 1 --2- 172.21.15.17:0/390225738 >> v2:172.21.15.77:3300/0 conn(0x7fdf080b41f0 0x7fdf08061af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:01.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.603+0000 7fdf0db23700 1 -- 172.21.15.17:0/390225738 >> v1:172.21.15.77:6789/0 conn(0x7fdf08147d80 legacy=0x7fdf08144630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:01.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.603+0000 7fdf0db23700 1 -- 172.21.15.17:0/390225738 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf080b69a0 con 0x7fdf080b65c0 2022-08-01T19:41:01.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.603+0000 7fdf0db23700 1 --2- 172.21.15.17:0/390225738 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdf080b65c0 0x7fdf08061600 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fdf040032f0 tx=0x7fdf04009290).ready entity=mon.2 client_cookie=4373f97d583260eb server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:01.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.603+0000 7fdf0cb21700 1 -- 172.21.15.17:0/390225738 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf0400b8a0 con 0x7fdf080b65c0 2022-08-01T19:41:01.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.603+0000 7fdf0cb21700 1 -- 172.21.15.17:0/390225738 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf0400ba00 con 0x7fdf080b65c0 2022-08-01T19:41:01.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.603+0000 7fdf0cb21700 1 -- 172.21.15.17:0/390225738 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf0400bb70 con 0x7fdf080b65c0 2022-08-01T19:41:01.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.607+0000 7fdf0eb25700 1 -- 172.21.15.17:0/390225738 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdf080b65c0 msgr2=0x7fdf08061600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:01.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.607+0000 7fdf0eb25700 1 --2- 172.21.15.17:0/390225738 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdf080b65c0 0x7fdf08061600 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fdf040032f0 tx=0x7fdf04009290).stop 2022-08-01T19:41:01.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.607+0000 7fdf0eb25700 1 -- 172.21.15.17:0/390225738 shutdown_connections 2022-08-01T19:41:01.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.607+0000 7fdf0eb25700 1 --2- 172.21.15.17:0/390225738 >> v2:172.21.15.77:3300/0 conn(0x7fdf080b41f0 0x7fdf08061af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:01.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.607+0000 7fdf0eb25700 1 --2- 172.21.15.17:0/390225738 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdf080b65c0 0x7fdf08061600 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:01.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.607+0000 7fdf0eb25700 1 -- 172.21.15.17:0/390225738 >> 172.21.15.17:0/390225738 conn(0x7fdf0805c620 msgr2=0x7fdf0805ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:01.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.607+0000 7fdf0eb25700 1 -- 172.21.15.17:0/390225738 shutdown_connections 2022-08-01T19:41:01.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.607+0000 7fdf0eb25700 1 -- 172.21.15.17:0/390225738 wait complete. 2022-08-01T19:41:01.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.607+0000 7fdf0eb25700 1 Processor -- start 2022-08-01T19:41:01.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.607+0000 7fdf0eb25700 1 -- start start 2022-08-01T19:41:01.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.607+0000 7fdf0eb25700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdf080b41f0 0x7fdf08145f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:01.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.607+0000 7fdf0eb25700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf080b65c0 0x7fdf08156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:01.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.607+0000 7fdf0eb25700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf08147d80 0x7fdf08157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:01.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.607+0000 7fdf0eb25700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf08062c00 con 0x7fdf08147d80 2022-08-01T19:41:01.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.607+0000 7fdf0eb25700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf08062ea0 con 0x7fdf080b65c0 2022-08-01T19:41:01.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.607+0000 7fdf0eb25700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf08063140 con 0x7fdf080b41f0 2022-08-01T19:41:01.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.607+0000 7fdf0db23700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdf080b41f0 0x7fdf08145f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:01.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.607+0000 7fdf0db23700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdf080b41f0 0x7fdf08145f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:37410/0 (socket says 172.21.15.17:37410) 2022-08-01T19:41:01.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.607+0000 7fdf0db23700 1 -- 172.21.15.17:0/2006246666 learned_addr learned my addr 172.21.15.17:0/2006246666 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:01.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.607+0000 7fdf0d322700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf080b65c0 0x7fdf08156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:01.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.607+0000 7fdf0e324700 1 --2- 172.21.15.17:0/2006246666 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf08147d80 0x7fdf08157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:01.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.611+0000 7fdf0e324700 1 -- 172.21.15.17:0/2006246666 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdf080b41f0 msgr2=0x7fdf08145f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:01.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.611+0000 7fdf0e324700 1 --2- 172.21.15.17:0/2006246666 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdf080b41f0 0x7fdf08145f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:01.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.611+0000 7fdf0e324700 1 -- 172.21.15.17:0/2006246666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf080b65c0 msgr2=0x7fdf08156c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:01.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.611+0000 7fdf0e324700 1 --2- 172.21.15.17:0/2006246666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf080b65c0 0x7fdf08156c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:01.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.611+0000 7fdf0e324700 1 -- 172.21.15.17:0/2006246666 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf04009530 con 0x7fdf08147d80 2022-08-01T19:41:01.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.611+0000 7fdf0d322700 1 --2- 172.21.15.17:0/2006246666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf080b65c0 0x7fdf08156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:01.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.611+0000 7fdf0db23700 1 --2- 172.21.15.17:0/2006246666 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdf080b41f0 0x7fdf08145f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:01.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.611+0000 7fdf0e324700 1 --2- 172.21.15.17:0/2006246666 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf08147d80 0x7fdf08157100 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7fdf08062a60 tx=0x7fdf00007b10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:01.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.611+0000 7fdefaffd700 1 -- 172.21.15.17:0/2006246666 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf0000ccd0 con 0x7fdf08147d80 2022-08-01T19:41:01.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.611+0000 7fdefaffd700 1 -- 172.21.15.17:0/2006246666 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf0000ce30 con 0x7fdf08147d80 2022-08-01T19:41:01.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.611+0000 7fdefaffd700 1 -- 172.21.15.17:0/2006246666 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf000125f0 con 0x7fdf08147d80 2022-08-01T19:41:01.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.611+0000 7fdf0eb25700 1 -- 172.21.15.17:0/2006246666 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf08146470 con 0x7fdf08147d80 2022-08-01T19:41:01.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.611+0000 7fdf0eb25700 1 -- 172.21.15.17:0/2006246666 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf08149e20 con 0x7fdf08147d80 2022-08-01T19:41:01.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.611+0000 7fdf0eb25700 1 -- 172.21.15.17:0/2006246666 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf080634c0 con 0x7fdf08147d80 2022-08-01T19:41:01.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.615+0000 7fdefaffd700 1 -- 172.21.15.17:0/2006246666 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdf00019030 con 0x7fdf08147d80 2022-08-01T19:41:01.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.615+0000 7fdefaffd700 1 --2- 172.21.15.17:0/2006246666 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdee8053ac0 0x7fdee8055f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:01.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.615+0000 7fdefaffd700 1 -- 172.21.15.17:0/2006246666 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fdf0006b300 con 0x7fdf08147d80 2022-08-01T19:41:01.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.615+0000 7fdf0db23700 1 --2- 172.21.15.17:0/2006246666 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdee8053ac0 0x7fdee8055f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:01.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.615+0000 7fdf0db23700 1 --2- 172.21.15.17:0/2006246666 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdee8053ac0 0x7fdee8055f80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7fdf0401b000 tx=0x7fdf04016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:01.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.619+0000 7fdefaffd700 1 -- 172.21.15.17:0/2006246666 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdf00041fb0 con 0x7fdf08147d80 2022-08-01T19:41:01.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.747+0000 7fdf0eb25700 1 -- 172.21.15.17:0/2006246666 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf081482a0 con 0x7fdf08147d80 2022-08-01T19:41:01.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.747+0000 7fdefaffd700 1 -- 172.21.15.17:0/2006246666 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16287 (secure 0 0 0) 0x7fdf000455d0 con 0x7fdf08147d80 2022-08-01T19:41:01.753 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:01.754 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":87,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:41:01.207172+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:40:59.229815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2022-08-01T19:41:00.225322+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:41:01.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.751+0000 7fdf0eb25700 1 -- 172.21.15.17:0/2006246666 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdee8053ac0 msgr2=0x7fdee8055f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:01.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.751+0000 7fdf0eb25700 1 --2- 172.21.15.17:0/2006246666 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdee8053ac0 0x7fdee8055f80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7fdf0401b000 tx=0x7fdf04016000).stop 2022-08-01T19:41:01.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.751+0000 7fdf0eb25700 1 -- 172.21.15.17:0/2006246666 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf08147d80 msgr2=0x7fdf08157100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:01.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.751+0000 7fdf0eb25700 1 --2- 172.21.15.17:0/2006246666 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf08147d80 0x7fdf08157100 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7fdf08062a60 tx=0x7fdf00007b10).stop 2022-08-01T19:41:01.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.751+0000 7fdf0eb25700 1 -- 172.21.15.17:0/2006246666 shutdown_connections 2022-08-01T19:41:01.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.751+0000 7fdf0eb25700 1 --2- 172.21.15.17:0/2006246666 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdee8053ac0 0x7fdee8055f80 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:01.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.751+0000 7fdf0eb25700 1 --2- 172.21.15.17:0/2006246666 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf08147d80 0x7fdf08157100 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:01.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.751+0000 7fdf0eb25700 1 --2- 172.21.15.17:0/2006246666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf080b65c0 0x7fdf08156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:01.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.751+0000 7fdf0eb25700 1 --2- 172.21.15.17:0/2006246666 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdf080b41f0 0x7fdf08145f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:01.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.751+0000 7fdf0eb25700 1 -- 172.21.15.17:0/2006246666 >> 172.21.15.17:0/2006246666 conn(0x7fdf0805c620 msgr2=0x7fdf08144840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:01.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.751+0000 7fdf0eb25700 1 -- 172.21.15.17:0/2006246666 shutdown_connections 2022-08-01T19:41:01.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.751+0000 7fdf0eb25700 1 -- 172.21.15.17:0/2006246666 wait complete. 2022-08-01T19:41:01.766 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-08-01T19:41:01.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.871+0000 7f85b30c2700 1 Processor -- start 2022-08-01T19:41:01.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.871+0000 7f85b30c2700 1 -- start start 2022-08-01T19:41:01.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.871+0000 7f85b30c2700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f85ac0b6420 0x7f85ac061600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:01.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.871+0000 7f85b30c2700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f85ac142ff0 0x7f85ac1434a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:01.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.871+0000 7f85b30c2700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85ac0b60e0 con 0x7f85ac0b41f0 2022-08-01T19:41:01.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.871+0000 7f85b30c2700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85ac0b2100 con 0x7f85ac0b6420 2022-08-01T19:41:01.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.871+0000 7f85b30c2700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac0b23a0 con 0x7f85ac142ff0 2022-08-01T19:41:01.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.875+0000 7f85b20c0700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f85ac0b6420 0x7f85ac061600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:01.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.875+0000 7f85b18bf700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f85ac0b41f0 0x7f85ac061af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33514/0 (socket says 172.21.15.17:33514) 2022-08-01T19:41:01.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.875+0000 7f85b18bf700 1 -- 172.21.15.17:0/3497748928 learned_addr learned my addr 172.21.15.17:0/3497748928 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:01.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.875+0000 7f85b28c1700 1 --2- 172.21.15.17:0/3497748928 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f85ac142ff0 0x7f85ac1434a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:01.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.875+0000 7f85b10be700 1 -- 172.21.15.17:0/3497748928 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2374595790 0 0) 0x7f85a0001110 con 0x7f85ac0b41f0 2022-08-01T19:41:01.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.875+0000 7f85b10be700 1 -- 172.21.15.17:0/3497748928 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8598003260 con 0x7f85ac0b41f0 2022-08-01T19:41:01.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.875+0000 7f85b20c0700 1 -- 172.21.15.17:0/3497748928 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f85ac142ff0 msgr2=0x7f85ac1434a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:01.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.875+0000 7f85b20c0700 1 --2- 172.21.15.17:0/3497748928 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f85ac142ff0 0x7f85ac1434a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:01.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.875+0000 7f85b20c0700 1 -- 172.21.15.17:0/3497748928 >> v1:172.21.15.17:6789/0 conn(0x7f85ac0b41f0 legacy=0x7f85ac061af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:01.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.875+0000 7f85b20c0700 1 -- 172.21.15.17:0/3497748928 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85ac0b2640 con 0x7f85ac0b6420 2022-08-01T19:41:01.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.875+0000 7f85b20c0700 1 --2- 172.21.15.17:0/3497748928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85ac0b6420 0x7f85ac061600 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7f859c002f70 tx=0x7f859c006df0).ready entity=mon.0 client_cookie=162fe2feeb00731f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:01.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.875+0000 7f85b10be700 1 -- 172.21.15.17:0/3497748928 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f859c00d4a0 con 0x7f85ac0b6420 2022-08-01T19:41:01.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.875+0000 7f85b10be700 1 -- 172.21.15.17:0/3497748928 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859c000e90 con 0x7f85ac0b6420 2022-08-01T19:41:01.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.875+0000 7f85b10be700 1 -- 172.21.15.17:0/3497748928 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f859c00d7f0 con 0x7f85ac0b6420 2022-08-01T19:41:01.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.875+0000 7f85b30c2700 1 -- 172.21.15.17:0/3497748928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85ac0b6420 msgr2=0x7f85ac061600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:01.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.875+0000 7f85b30c2700 1 --2- 172.21.15.17:0/3497748928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85ac0b6420 0x7f85ac061600 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7f859c002f70 tx=0x7f859c006df0).stop 2022-08-01T19:41:01.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.875+0000 7f85b30c2700 1 -- 172.21.15.17:0/3497748928 shutdown_connections 2022-08-01T19:41:01.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.875+0000 7f85b30c2700 1 --2- 172.21.15.17:0/3497748928 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f85ac142ff0 0x7f85ac1434a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:01.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.875+0000 7f85b30c2700 1 --2- 172.21.15.17:0/3497748928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85ac0b6420 0x7f85ac061600 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:01.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.875+0000 7f85b30c2700 1 -- 172.21.15.17:0/3497748928 >> 172.21.15.17:0/3497748928 conn(0x7f85ac05c620 msgr2=0x7f85ac05ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:01.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.875+0000 7f85b30c2700 1 -- 172.21.15.17:0/3497748928 shutdown_connections 2022-08-01T19:41:01.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.875+0000 7f85b30c2700 1 -- 172.21.15.17:0/3497748928 wait complete. 2022-08-01T19:41:01.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.875+0000 7f85b30c2700 1 Processor -- start 2022-08-01T19:41:01.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.875+0000 7f85b30c2700 1 -- start start 2022-08-01T19:41:01.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.879+0000 7f85b30c2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85ac0b41f0 0x7f85ac0b9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:01.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.879+0000 7f85b30c2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85ac0b6420 0x7f85ac152690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:01.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.879+0000 7f85b30c2700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f85ac142ff0 0x7f85ac152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:01.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.879+0000 7f85b30c2700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac0b0410 con 0x7f85ac0b6420 2022-08-01T19:41:01.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.879+0000 7f85b30c2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac0afed0 con 0x7f85ac0b41f0 2022-08-01T19:41:01.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.879+0000 7f85b30c2700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac0b0170 con 0x7f85ac142ff0 2022-08-01T19:41:01.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.879+0000 7f85b18bf700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85ac0b6420 0x7f85ac152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:01.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.879+0000 7f85b18bf700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85ac0b6420 0x7f85ac152690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41328/0 (socket says 172.21.15.17:41328) 2022-08-01T19:41:01.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.879+0000 7f85b18bf700 1 -- 172.21.15.17:0/2445403060 learned_addr learned my addr 172.21.15.17:0/2445403060 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:01.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.879+0000 7f85b28c1700 1 --2- 172.21.15.17:0/2445403060 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f85ac142ff0 0x7f85ac152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:01.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.879+0000 7f85b20c0700 1 --2- 172.21.15.17:0/2445403060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85ac0b41f0 0x7f85ac0b9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:01.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.879+0000 7f85b18bf700 1 -- 172.21.15.17:0/2445403060 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f85ac142ff0 msgr2=0x7f85ac152b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:01.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.879+0000 7f85b18bf700 1 --2- 172.21.15.17:0/2445403060 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f85ac142ff0 0x7f85ac152b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:01.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.879+0000 7f85b18bf700 1 -- 172.21.15.17:0/2445403060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85ac0b41f0 msgr2=0x7f85ac0b9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:01.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.879+0000 7f85b18bf700 1 --2- 172.21.15.17:0/2445403060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85ac0b41f0 0x7f85ac0b9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:01.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.879+0000 7f85b18bf700 1 -- 172.21.15.17:0/2445403060 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f859c00b180 con 0x7f85ac0b6420 2022-08-01T19:41:01.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.879+0000 7f85b18bf700 1 --2- 172.21.15.17:0/2445403060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85ac0b6420 0x7f85ac152690 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7f85a00012a0 tx=0x7f85a00078e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:01.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.879+0000 7f85ab7fe700 1 -- 172.21.15.17:0/2445403060 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85a00070d0 con 0x7f85ac0b6420 2022-08-01T19:41:01.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.879+0000 7f85ab7fe700 1 -- 172.21.15.17:0/2445403060 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a0007230 con 0x7f85ac0b6420 2022-08-01T19:41:01.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.879+0000 7f85b30c2700 1 -- 172.21.15.17:0/2445403060 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85ac0af550 con 0x7f85ac0b6420 2022-08-01T19:41:01.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.879+0000 7f85ab7fe700 1 -- 172.21.15.17:0/2445403060 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85a0007590 con 0x7f85ac0b6420 2022-08-01T19:41:01.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.879+0000 7f85b30c2700 1 -- 172.21.15.17:0/2445403060 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85ac1403e0 con 0x7f85ac0b6420 2022-08-01T19:41:01.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.883+0000 7f85b30c2700 1 -- 172.21.15.17:0/2445403060 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85ac0b28e0 con 0x7f85ac0b6420 2022-08-01T19:41:01.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.883+0000 7f85ab7fe700 1 -- 172.21.15.17:0/2445403060 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f85a0010070 con 0x7f85ac0b6420 2022-08-01T19:41:01.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.883+0000 7f85ab7fe700 1 --2- 172.21.15.17:0/2445403060 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f858c053aa0 0x7f858c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:01.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.883+0000 7f85ab7fe700 1 -- 172.21.15.17:0/2445403060 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f85a006b350 con 0x7f85ac0b6420 2022-08-01T19:41:01.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.883+0000 7f85b20c0700 1 --2- 172.21.15.17:0/2445403060 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f858c053aa0 0x7f858c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:01.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.883+0000 7f85b20c0700 1 --2- 172.21.15.17:0/2445403060 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f858c053aa0 0x7f858c055f60 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f859c01a000 tx=0x7f859c014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:01.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:01.887+0000 7f85ab7fe700 1 -- 172.21.15.17:0/2445403060 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f85a0042110 con 0x7f85ac0b6420 2022-08-01T19:41:02.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.051+0000 7f85b30c2700 1 -- 172.21.15.17:0/2445403060 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f85ac0b60e0 con 0x7f85ac0b6420 2022-08-01T19:41:02.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.211+0000 7f85ab7fe700 1 -- 172.21.15.17:0/2445403060 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v40) v1 ==== 118+0+0 (secure 0 0 0) 0x7f85ac0b60e0 con 0x7f85ac0b6420 2022-08-01T19:41:02.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.215+0000 7f85b30c2700 1 -- 172.21.15.17:0/2445403060 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f858c053aa0 msgr2=0x7f858c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:02.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.215+0000 7f85b30c2700 1 --2- 172.21.15.17:0/2445403060 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f858c053aa0 0x7f858c055f60 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f859c01a000 tx=0x7f859c014000).stop 2022-08-01T19:41:02.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.215+0000 7f85b30c2700 1 -- 172.21.15.17:0/2445403060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85ac0b6420 msgr2=0x7f85ac152690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:02.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.215+0000 7f85b30c2700 1 --2- 172.21.15.17:0/2445403060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85ac0b6420 0x7f85ac152690 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7f85a00012a0 tx=0x7f85a00078e0).stop 2022-08-01T19:41:02.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.215+0000 7f85b30c2700 1 -- 172.21.15.17:0/2445403060 shutdown_connections 2022-08-01T19:41:02.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.215+0000 7f85b30c2700 1 --2- 172.21.15.17:0/2445403060 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f858c053aa0 0x7f858c055f60 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:02.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.215+0000 7f85b30c2700 1 --2- 172.21.15.17:0/2445403060 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f85ac142ff0 0x7f85ac152b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:02.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.215+0000 7f85b30c2700 1 --2- 172.21.15.17:0/2445403060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85ac0b6420 0x7f85ac152690 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:02.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.215+0000 7f85b30c2700 1 --2- 172.21.15.17:0/2445403060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85ac0b41f0 0x7f85ac0b9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:02.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.215+0000 7f85b30c2700 1 -- 172.21.15.17:0/2445403060 >> 172.21.15.17:0/2445403060 conn(0x7f85ac05c620 msgr2=0x7f85ac142510 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:02.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.215+0000 7f85b30c2700 1 -- 172.21.15.17:0/2445403060 shutdown_connections 2022-08-01T19:41:02.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.215+0000 7f85b30c2700 1 -- 172.21.15.17:0/2445403060 wait complete. 2022-08-01T19:41:02.244 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:41:02.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.355+0000 7ff0564e8700 1 Processor -- start 2022-08-01T19:41:02.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.355+0000 7ff0564e8700 1 -- start start 2022-08-01T19:41:02.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff0564e8700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ff0500b1f40 0x7ff0500b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:02.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff0564e8700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0500b04e0 con 0x7ff050148530 2022-08-01T19:41:02.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff0564e8700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0500b0780 con 0x7ff0500a7250 2022-08-01T19:41:02.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff0564e8700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0500a9250 con 0x7ff0500b1f40 2022-08-01T19:41:02.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff04f7fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff050148530 0x7ff0500b2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33526/0 (socket says 172.21.15.17:33526) 2022-08-01T19:41:02.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff04f7fe700 1 -- 172.21.15.17:0/74742284 learned_addr learned my addr 172.21.15.17:0/74742284 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:02.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff04ffff700 1 --2- 172.21.15.17:0/74742284 >> v2:172.21.15.77:3300/0 conn(0x7ff0500b1f40 0x7ff0500b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:02.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff04effd700 1 -- 172.21.15.17:0/74742284 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3326127215 0 0) 0x7ff0500b04e0 con 0x7ff050148530 2022-08-01T19:41:02.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff04effd700 1 -- 172.21.15.17:0/74742284 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff03c003260 con 0x7ff050148530 2022-08-01T19:41:02.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff04effd700 1 -- 172.21.15.17:0/74742284 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 448913909 0 0) 0x7ff0500b0780 con 0x7ff0500a7250 2022-08-01T19:41:02.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff04effd700 1 -- 172.21.15.17:0/74742284 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0500b04e0 con 0x7ff0500a7250 2022-08-01T19:41:02.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff04effd700 1 -- 172.21.15.17:0/74742284 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2532159783 0 0) 0x7ff03c003260 con 0x7ff050148530 2022-08-01T19:41:02.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff04effd700 1 -- 172.21.15.17:0/74742284 >> v2:172.21.15.77:3300/0 conn(0x7ff0500b1f40 msgr2=0x7ff0500b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:02.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff04effd700 1 --2- 172.21.15.17:0/74742284 >> v2:172.21.15.77:3300/0 conn(0x7ff0500b1f40 0x7ff0500b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:02.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff04effd700 1 -- 172.21.15.17:0/74742284 >> v1:172.21.15.77:6789/0 conn(0x7ff0500a7250 legacy=0x7ff0501448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:02.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff04effd700 1 -- 172.21.15.17:0/74742284 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0500b0a20 con 0x7ff050148530 2022-08-01T19:41:02.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff04effd700 1 -- 172.21.15.17:0/74742284 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7ff0400019e0 con 0x7ff050148530 2022-08-01T19:41:02.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff04effd700 1 -- 172.21.15.17:0/74742284 >> v1:172.21.15.17:6789/0 conn(0x7ff050148530 legacy=0x7ff0500b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:02.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff04effd700 1 --2- 172.21.15.17:0/74742284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff03c0038e0 0x7ff03c003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:02.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff04effd700 1 --2- 172.21.15.17:0/74742284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff03c004500 0x7ff03c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:02.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff04effd700 1 --2- 172.21.15.17:0/74742284 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff03c004e70 0x7ff03c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:02.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff04effd700 1 -- 172.21.15.17:0/74742284 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03c00bc90 con 0x7ff03c0038e0 2022-08-01T19:41:02.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff04effd700 1 -- 172.21.15.17:0/74742284 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03c00be00 con 0x7ff03c004500 2022-08-01T19:41:02.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff04effd700 1 -- 172.21.15.17:0/74742284 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff03c00bfc0 con 0x7ff03c004e70 2022-08-01T19:41:02.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff04ffff700 1 --2- 172.21.15.17:0/74742284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff03c0038e0 0x7ff03c003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:02.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff04f7fe700 1 --2- 172.21.15.17:0/74742284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff03c004500 0x7ff03c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:02.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff054a85700 1 --2- 172.21.15.17:0/74742284 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff03c004e70 0x7ff03c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:02.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff04ffff700 1 -- 172.21.15.17:0/74742284 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff03c004e70 msgr2=0x7ff03c00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:02.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff04ffff700 1 --2- 172.21.15.17:0/74742284 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff03c004e70 0x7ff03c00b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:02.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff04ffff700 1 -- 172.21.15.17:0/74742284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff03c004500 msgr2=0x7ff03c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:02.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff04ffff700 1 --2- 172.21.15.17:0/74742284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff03c004500 0x7ff03c004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:02.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff04ffff700 1 -- 172.21.15.17:0/74742284 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff03c003260 con 0x7ff03c0038e0 2022-08-01T19:41:02.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff054a85700 1 -- 172.21.15.17:0/74742284 reap_dead start 2022-08-01T19:41:02.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff04ffff700 1 --2- 172.21.15.17:0/74742284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff03c0038e0 0x7ff03c003cc0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7ff04400c000 tx=0x7ff044006df0).ready entity=mon.0 client_cookie=febd694f8fa83890 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:02.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff04effd700 1 -- 172.21.15.17:0/74742284 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff04400e830 con 0x7ff03c0038e0 2022-08-01T19:41:02.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff04effd700 1 -- 172.21.15.17:0/74742284 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04400e990 con 0x7ff03c0038e0 2022-08-01T19:41:02.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.359+0000 7ff04effd700 1 -- 172.21.15.17:0/74742284 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff04400ecf0 con 0x7ff03c0038e0 2022-08-01T19:41:02.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.363+0000 7ff0564e8700 1 -- 172.21.15.17:0/74742284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff03c0038e0 msgr2=0x7ff03c003cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:02.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.363+0000 7ff0564e8700 1 --2- 172.21.15.17:0/74742284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff03c0038e0 0x7ff03c003cc0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7ff04400c000 tx=0x7ff044006df0).stop 2022-08-01T19:41:02.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.363+0000 7ff0564e8700 1 -- 172.21.15.17:0/74742284 shutdown_connections 2022-08-01T19:41:02.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.363+0000 7ff0564e8700 1 --2- 172.21.15.17:0/74742284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff03c0038e0 0x7ff03c003cc0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:02.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.363+0000 7ff0564e8700 1 -- 172.21.15.17:0/74742284 >> 172.21.15.17:0/74742284 conn(0x7ff05005f840 msgr2=0x7ff05005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:02.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.363+0000 7ff0564e8700 1 -- 172.21.15.17:0/74742284 shutdown_connections 2022-08-01T19:41:02.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.363+0000 7ff0564e8700 1 -- 172.21.15.17:0/74742284 wait complete. 2022-08-01T19:41:02.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.363+0000 7ff0564e8700 1 Processor -- start 2022-08-01T19:41:02.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.363+0000 7ff0564e8700 1 -- start start 2022-08-01T19:41:02.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.367+0000 7ff0564e8700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff03c0038e0 0x7ff05014a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:02.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.367+0000 7ff0564e8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff05014b9d0 0x7ff0500ab2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:02.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.367+0000 7ff0564e8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0500ac360 0x7ff050143b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:02.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.367+0000 7ff0564e8700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0500aaad0 con 0x7ff05014b9d0 2022-08-01T19:41:02.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.367+0000 7ff0564e8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0500a96e0 con 0x7ff0500ac360 2022-08-01T19:41:02.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.367+0000 7ff0564e8700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0500a9980 con 0x7ff03c0038e0 2022-08-01T19:41:02.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.367+0000 7ff04f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff05014b9d0 0x7ff0500ab2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:02.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.367+0000 7ff04f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff05014b9d0 0x7ff0500ab2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41344/0 (socket says 172.21.15.17:41344) 2022-08-01T19:41:02.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.367+0000 7ff04f7fe700 1 -- 172.21.15.17:0/58753996 learned_addr learned my addr 172.21.15.17:0/58753996 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:02.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.367+0000 7ff054a85700 1 --2- 172.21.15.17:0/58753996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0500ac360 0x7ff050143b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:02.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.367+0000 7ff04ffff700 1 --2- 172.21.15.17:0/58753996 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff03c0038e0 0x7ff05014a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:02.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.367+0000 7ff04f7fe700 1 -- 172.21.15.17:0/58753996 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff03c0038e0 msgr2=0x7ff05014a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:02.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.367+0000 7ff04f7fe700 1 --2- 172.21.15.17:0/58753996 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff03c0038e0 0x7ff05014a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:02.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.367+0000 7ff04f7fe700 1 -- 172.21.15.17:0/58753996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0500ac360 msgr2=0x7ff050143b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:02.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.367+0000 7ff04f7fe700 1 --2- 172.21.15.17:0/58753996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0500ac360 0x7ff050143b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:02.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.367+0000 7ff04f7fe700 1 -- 172.21.15.17:0/58753996 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0500a9c20 con 0x7ff05014b9d0 2022-08-01T19:41:02.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.367+0000 7ff04f7fe700 1 --2- 172.21.15.17:0/58753996 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff05014b9d0 0x7ff0500ab2e0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7ff040008000 tx=0x7ff0500b0910).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:02.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.367+0000 7ff04d7fa700 1 -- 172.21.15.17:0/58753996 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff04000ab20 con 0x7ff05014b9d0 2022-08-01T19:41:02.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.367+0000 7ff04d7fa700 1 -- 172.21.15.17:0/58753996 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04000ac80 con 0x7ff05014b9d0 2022-08-01T19:41:02.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.367+0000 7ff0564e8700 1 -- 172.21.15.17:0/58753996 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff05014b1c0 con 0x7ff05014b9d0 2022-08-01T19:41:02.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.367+0000 7ff04d7fa700 1 -- 172.21.15.17:0/58753996 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0400133b0 con 0x7ff05014b9d0 2022-08-01T19:41:02.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.367+0000 7ff0564e8700 1 -- 172.21.15.17:0/58753996 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff05014b650 con 0x7ff05014b9d0 2022-08-01T19:41:02.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.367+0000 7ff0564e8700 1 -- 172.21.15.17:0/58753996 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0500a9db0 con 0x7ff05014b9d0 2022-08-01T19:41:02.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.367+0000 7ff04d7fa700 1 -- 172.21.15.17:0/58753996 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff04001c030 con 0x7ff05014b9d0 2022-08-01T19:41:02.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.371+0000 7ff04d7fa700 1 --2- 172.21.15.17:0/58753996 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff030053a10 0x7ff030055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:02.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.371+0000 7ff04d7fa700 1 -- 172.21.15.17:0/58753996 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff04006c280 con 0x7ff05014b9d0 2022-08-01T19:41:02.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.371+0000 7ff04ffff700 1 --2- 172.21.15.17:0/58753996 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff030053a10 0x7ff030055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:02.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.371+0000 7ff04ffff700 1 --2- 172.21.15.17:0/58753996 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff030053a10 0x7ff030055ed0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7ff04401b000 tx=0x7ff044019000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:02.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.375+0000 7ff04d7fa700 1 -- 172.21.15.17:0/58753996 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff040043040 con 0x7ff05014b9d0 2022-08-01T19:41:02.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.531+0000 7ff0564e8700 1 -- 172.21.15.17:0/58753996 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0500574e0 con 0x7ff05014b9d0 2022-08-01T19:41:02.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.531+0000 7ff04d7fa700 1 -- 172.21.15.17:0/58753996 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+2493 (secure 0 0 0) 0x7ff04001a030 con 0x7ff05014b9d0 2022-08-01T19:41:02.538 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:02.538 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":40,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6615,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3580428616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3580428616},{"type":"v1","addr":"172.21.15.17:6833","nonce":3580428616}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38},{"gid":6627,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/4048623373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":4048623373},{"type":"v1","addr":"172.21.15.17:6835","nonce":4048623373}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:41:01.207552+0000","modified":"2022-08-01T19:41:02.210147+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6630},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6630":{"gid":6630,"name":"b","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6835/4275403969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4275403969},{"type":"v1","addr":"172.21.15.77:6835","nonce":4275403969}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2022-08-01T19:41:02.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.535+0000 7ff0564e8700 1 -- 172.21.15.17:0/58753996 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff030053a10 msgr2=0x7ff030055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:02.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.535+0000 7ff0564e8700 1 --2- 172.21.15.17:0/58753996 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff030053a10 0x7ff030055ed0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7ff04401b000 tx=0x7ff044019000).stop 2022-08-01T19:41:02.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.535+0000 7ff0564e8700 1 -- 172.21.15.17:0/58753996 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff05014b9d0 msgr2=0x7ff0500ab2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:02.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.535+0000 7ff0564e8700 1 --2- 172.21.15.17:0/58753996 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff05014b9d0 0x7ff0500ab2e0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7ff040008000 tx=0x7ff0500b0910).stop 2022-08-01T19:41:02.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.535+0000 7ff0564e8700 1 -- 172.21.15.17:0/58753996 shutdown_connections 2022-08-01T19:41:02.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.535+0000 7ff0564e8700 1 --2- 172.21.15.17:0/58753996 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff030053a10 0x7ff030055ed0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:02.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.535+0000 7ff0564e8700 1 --2- 172.21.15.17:0/58753996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0500ac360 0x7ff050143b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:02.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.535+0000 7ff0564e8700 1 --2- 172.21.15.17:0/58753996 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff05014b9d0 0x7ff0500ab2e0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:02.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.535+0000 7ff0564e8700 1 --2- 172.21.15.17:0/58753996 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff03c0038e0 0x7ff05014a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:02.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.535+0000 7ff0564e8700 1 -- 172.21.15.17:0/58753996 >> 172.21.15.17:0/58753996 conn(0x7ff05005f840 msgr2=0x7ff0500b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:02.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.535+0000 7ff0564e8700 1 -- 172.21.15.17:0/58753996 shutdown_connections 2022-08-01T19:41:02.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.535+0000 7ff0564e8700 1 -- 172.21.15.17:0/58753996 wait complete. 2022-08-01T19:41:02.545 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 40 2022-08-01T19:41:02.551 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:41:02.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.647+0000 7fe1e11d4700 1 Processor -- start 2022-08-01T19:41:02.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.647+0000 7fe1e11d4700 1 -- start start 2022-08-01T19:41:02.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.647+0000 7fe1e11d4700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fe1dc0b38a0 0x7fe1dc064bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:02.660 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.647+0000 7fe1e11d4700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe1dc1470d0 0x7fe1dc1403f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:02.660 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.647+0000 7fe1e11d4700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1dc0b4f60 con 0x7fe1dc0647d0 2022-08-01T19:41:02.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.647+0000 7fe1e11d4700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1dc0b4a60 con 0x7fe1dc1470d0 2022-08-01T19:41:02.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.647+0000 7fe1e11d4700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1dc0b4cc0 con 0x7fe1dc0b38a0 2022-08-01T19:41:02.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.647+0000 7fe1e09d3700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe1dc1470d0 0x7fe1dc1403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:02.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.651+0000 7fe1e09d3700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe1dc1470d0 0x7fe1dc1403f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41356/0 (socket says 172.21.15.17:41356) 2022-08-01T19:41:02.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.651+0000 7fe1e09d3700 1 -- 172.21.15.17:0/3586047286 learned_addr learned my addr 172.21.15.17:0/3586047286 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:02.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.651+0000 7fe1db7fe700 1 --2- 172.21.15.17:0/3586047286 >> v2:172.21.15.77:3300/0 conn(0x7fe1dc0b38a0 0x7fe1dc064bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:02.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.651+0000 7fe1e09d3700 1 -- 172.21.15.17:0/3586047286 >> v2:172.21.15.77:3300/0 conn(0x7fe1dc0b38a0 msgr2=0x7fe1dc064bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:02.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.651+0000 7fe1e09d3700 1 --2- 172.21.15.17:0/3586047286 >> v2:172.21.15.77:3300/0 conn(0x7fe1dc0b38a0 0x7fe1dc064bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:02.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.651+0000 7fe1e09d3700 1 -- 172.21.15.17:0/3586047286 >> v1:172.21.15.17:6789/0 conn(0x7fe1dc0647d0 legacy=0x7fe1dc1436f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:02.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.651+0000 7fe1e09d3700 1 -- 172.21.15.17:0/3586047286 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1dc143250 con 0x7fe1dc1470d0 2022-08-01T19:41:02.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.651+0000 7fe1e09d3700 1 --2- 172.21.15.17:0/3586047286 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1dc1470d0 0x7fe1dc1403f0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7fe1d40032f0 tx=0x7fe1d4009290).ready entity=mon.0 client_cookie=41983d035d3b75bf server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:02.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.651+0000 7fe1daffd700 1 -- 172.21.15.17:0/3586047286 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1d400b870 con 0x7fe1dc1470d0 2022-08-01T19:41:02.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.651+0000 7fe1daffd700 1 -- 172.21.15.17:0/3586047286 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d400b9d0 con 0x7fe1dc1470d0 2022-08-01T19:41:02.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.651+0000 7fe1daffd700 1 -- 172.21.15.17:0/3586047286 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1d400bb40 con 0x7fe1dc1470d0 2022-08-01T19:41:02.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.651+0000 7fe1e11d4700 1 -- 172.21.15.17:0/3586047286 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1dc1470d0 msgr2=0x7fe1dc1403f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:02.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.651+0000 7fe1e11d4700 1 --2- 172.21.15.17:0/3586047286 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1dc1470d0 0x7fe1dc1403f0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7fe1d40032f0 tx=0x7fe1d4009290).stop 2022-08-01T19:41:02.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.651+0000 7fe1e11d4700 1 -- 172.21.15.17:0/3586047286 shutdown_connections 2022-08-01T19:41:02.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.651+0000 7fe1e11d4700 1 --2- 172.21.15.17:0/3586047286 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1dc1470d0 0x7fe1dc1403f0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:02.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.651+0000 7fe1e11d4700 1 --2- 172.21.15.17:0/3586047286 >> v2:172.21.15.77:3300/0 conn(0x7fe1dc0b38a0 0x7fe1dc064bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:02.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.651+0000 7fe1e11d4700 1 -- 172.21.15.17:0/3586047286 >> 172.21.15.17:0/3586047286 conn(0x7fe1dc05f840 msgr2=0x7fe1dc05fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:02.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.651+0000 7fe1e11d4700 1 -- 172.21.15.17:0/3586047286 shutdown_connections 2022-08-01T19:41:02.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.651+0000 7fe1e11d4700 1 -- 172.21.15.17:0/3586047286 wait complete. 2022-08-01T19:41:02.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.655+0000 7fe1e11d4700 1 Processor -- start 2022-08-01T19:41:02.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.655+0000 7fe1e11d4700 1 -- start start 2022-08-01T19:41:02.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.655+0000 7fe1e11d4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1dc0647d0 0x7fe1dc0b7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:02.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.655+0000 7fe1e11d4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1dc0b38a0 0x7fe1dc0b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:02.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.655+0000 7fe1e11d4700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1dc1470d0 0x7fe1dc0b0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:02.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.655+0000 7fe1e11d4700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc0b6d10 con 0x7fe1dc0647d0 2022-08-01T19:41:02.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.655+0000 7fe1e11d4700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc146480 con 0x7fe1dc0b38a0 2022-08-01T19:41:02.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.655+0000 7fe1e11d4700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc0b54f0 con 0x7fe1dc1470d0 2022-08-01T19:41:02.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.655+0000 7fe1dbfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1dc0647d0 0x7fe1dc0b7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:02.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.655+0000 7fe1dbfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1dc0647d0 0x7fe1dc0b7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41358/0 (socket says 172.21.15.17:41358) 2022-08-01T19:41:02.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.655+0000 7fe1dbfff700 1 -- 172.21.15.17:0/4031665618 learned_addr learned my addr 172.21.15.17:0/4031665618 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:02.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.655+0000 7fe1db7fe700 1 --2- 172.21.15.17:0/4031665618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1dc0b38a0 0x7fe1dc0b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:02.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.655+0000 7fe1e09d3700 1 --2- 172.21.15.17:0/4031665618 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1dc1470d0 0x7fe1dc0b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:02.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.655+0000 7fe1dbfff700 1 -- 172.21.15.17:0/4031665618 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1dc1470d0 msgr2=0x7fe1dc0b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:02.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.655+0000 7fe1dbfff700 1 --2- 172.21.15.17:0/4031665618 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1dc1470d0 0x7fe1dc0b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.655+0000 7fe1dbfff700 1 -- 172.21.15.17:0/4031665618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1dc0b38a0 msgr2=0x7fe1dc0b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.655+0000 7fe1dbfff700 1 --2- 172.21.15.17:0/4031665618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1dc0b38a0 0x7fe1dc0b73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:02.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.655+0000 7fe1dbfff700 1 -- 172.21.15.17:0/4031665618 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d4009530 con 0x7fe1dc0647d0 2022-08-01T19:41:02.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.655+0000 7fe1db7fe700 1 --2- 172.21.15.17:0/4031665618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1dc0b38a0 0x7fe1dc0b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:02.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.655+0000 7fe1dbfff700 1 --2- 172.21.15.17:0/4031665618 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1dc0647d0 0x7fe1dc0b7d50 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7fe1dc0660a0 tx=0x7fe1d0007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.655+0000 7fe1d97fa700 1 -- 172.21.15.17:0/4031665618 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1d000ce30 con 0x7fe1dc0647d0 2022-08-01T19:41:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.655+0000 7fe1e11d4700 1 -- 172.21.15.17:0/4031665618 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1dc0b5790 con 0x7fe1dc0647d0 2022-08-01T19:41:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.655+0000 7fe1d97fa700 1 -- 172.21.15.17:0/4031665618 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d0007240 con 0x7fe1dc0647d0 2022-08-01T19:41:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.655+0000 7fe1d97fa700 1 -- 172.21.15.17:0/4031665618 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1d0006700 con 0x7fe1dc0647d0 2022-08-01T19:41:02.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.655+0000 7fe1e11d4700 1 -- 172.21.15.17:0/4031665618 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1dc146a80 con 0x7fe1dc0647d0 2022-08-01T19:41:02.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.655+0000 7fe1e11d4700 1 -- 172.21.15.17:0/4031665618 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1dc147560 con 0x7fe1dc0647d0 2022-08-01T19:41:02.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.659+0000 7fe1d97fa700 1 -- 172.21.15.17:0/4031665618 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe1d0010030 con 0x7fe1dc0647d0 2022-08-01T19:41:02.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.659+0000 7fe1d97fa700 1 --2- 172.21.15.17:0/4031665618 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe1bc053950 0x7fe1bc055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.659+0000 7fe1d97fa700 1 -- 172.21.15.17:0/4031665618 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe1d006bd10 con 0x7fe1dc0647d0 2022-08-01T19:41:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.659+0000 7fe1db7fe700 1 --2- 172.21.15.17:0/4031665618 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe1bc053950 0x7fe1bc055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.659+0000 7fe1db7fe700 1 --2- 172.21.15.17:0/4031665618 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe1bc053950 0x7fe1bc055e10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7fe1cc00c000 tx=0x7fe1cc008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.667+0000 7fe1d97fa700 1 -- 172.21.15.17:0/4031665618 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe1d0042ad0 con 0x7fe1dc0647d0 2022-08-01T19:41:02.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.791+0000 7fe1e11d4700 1 -- 172.21.15.17:0/4031665618 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1dc0574e0 con 0x7fe1dc0647d0 2022-08-01T19:41:02.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.791+0000 7fe1d97fa700 1 -- 172.21.15.17:0/4031665618 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16274 (secure 0 0 0) 0x7fe1d00460f0 con 0x7fe1dc0647d0 2022-08-01T19:41:02.796 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:02.796 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":88,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:41:02.210932+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:40:59.229815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2022-08-01T19:41:00.225322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:41:02.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.795+0000 7fe1e11d4700 1 -- 172.21.15.17:0/4031665618 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe1bc053950 msgr2=0x7fe1bc055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:02.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.795+0000 7fe1e11d4700 1 --2- 172.21.15.17:0/4031665618 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe1bc053950 0x7fe1bc055e10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7fe1cc00c000 tx=0x7fe1cc008040).stop 2022-08-01T19:41:02.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.795+0000 7fe1e11d4700 1 -- 172.21.15.17:0/4031665618 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1dc0647d0 msgr2=0x7fe1dc0b7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:02.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.795+0000 7fe1e11d4700 1 --2- 172.21.15.17:0/4031665618 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1dc0647d0 0x7fe1dc0b7d50 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7fe1dc0660a0 tx=0x7fe1d0007c10).stop 2022-08-01T19:41:02.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.795+0000 7fe1e11d4700 1 -- 172.21.15.17:0/4031665618 shutdown_connections 2022-08-01T19:41:02.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.795+0000 7fe1e11d4700 1 --2- 172.21.15.17:0/4031665618 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe1bc053950 0x7fe1bc055e10 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:02.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.795+0000 7fe1e11d4700 1 --2- 172.21.15.17:0/4031665618 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1dc1470d0 0x7fe1dc0b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:02.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.795+0000 7fe1e11d4700 1 --2- 172.21.15.17:0/4031665618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1dc0b38a0 0x7fe1dc0b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:02.802 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.795+0000 7fe1e11d4700 1 --2- 172.21.15.17:0/4031665618 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1dc0647d0 0x7fe1dc0b7d50 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:02.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.795+0000 7fe1e11d4700 1 -- 172.21.15.17:0/4031665618 >> 172.21.15.17:0/4031665618 conn(0x7fe1dc05f840 msgr2=0x7fe1dc05c040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:02.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.795+0000 7fe1e11d4700 1 -- 172.21.15.17:0/4031665618 shutdown_connections 2022-08-01T19:41:02.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.795+0000 7fe1e11d4700 1 -- 172.21.15.17:0/4031665618 wait complete. 2022-08-01T19:41:02.815 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T19:41:02.948 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.923+0000 7fadeaf3b700 1 Processor -- start 2022-08-01T19:41:02.948 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.923+0000 7fadeaf3b700 1 -- start start 2022-08-01T19:41:02.949 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.927+0000 7fadeaf3b700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fade40ad750 0x7fade4061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:02.949 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.927+0000 7fadeaf3b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fade41405d0 0x7fade4140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:02.949 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.927+0000 7fadeaf3b700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fade40aee80 con 0x7fade40614b0 2022-08-01T19:41:02.950 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.927+0000 7fadeaf3b700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fade40b70b0 con 0x7fade40ad750 2022-08-01T19:41:02.950 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.927+0000 7fadeaf3b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade40af0b0 con 0x7fade41405d0 2022-08-01T19:41:02.951 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.927+0000 7fade9f39700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fade40614b0 0x7fade40b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33562/0 (socket says 172.21.15.17:33562) 2022-08-01T19:41:02.951 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.927+0000 7fade9f39700 1 -- 172.21.15.17:0/3591021334 learned_addr learned my addr 172.21.15.17:0/3591021334 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:02.952 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.927+0000 7fade9738700 1 --2- 172.21.15.17:0/3591021334 >> v2:172.21.15.77:3300/0 conn(0x7fade40ad750 0x7fade4061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:02.952 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.927+0000 7fadea73a700 1 --2- 172.21.15.17:0/3591021334 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fade41405d0 0x7fade4140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:02.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.927+0000 7fade8f37700 1 -- 172.21.15.17:0/3591021334 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 948577758 0 0) 0x7fade40aee80 con 0x7fade40614b0 2022-08-01T19:41:02.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.927+0000 7fade8f37700 1 -- 172.21.15.17:0/3591021334 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadcc003260 con 0x7fade40614b0 2022-08-01T19:41:02.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.927+0000 7fadea73a700 1 -- 172.21.15.17:0/3591021334 >> v2:172.21.15.77:3300/0 conn(0x7fade40ad750 msgr2=0x7fade4061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:02.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.927+0000 7fadea73a700 1 --2- 172.21.15.17:0/3591021334 >> v2:172.21.15.77:3300/0 conn(0x7fade40ad750 0x7fade4061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:02.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.927+0000 7fadea73a700 1 -- 172.21.15.17:0/3591021334 >> v1:172.21.15.17:6789/0 conn(0x7fade40614b0 legacy=0x7fade40b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:02.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.927+0000 7fadea73a700 1 -- 172.21.15.17:0/3591021334 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade40af350 con 0x7fade41405d0 2022-08-01T19:41:02.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.927+0000 7fade9738700 1 --2- 172.21.15.17:0/3591021334 >> v2:172.21.15.77:3300/0 conn(0x7fade40ad750 0x7fade4061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:02.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.927+0000 7fadea73a700 1 --2- 172.21.15.17:0/3591021334 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fade41405d0 0x7fade4140a80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7faddc0032f0 tx=0x7faddc0091d0).ready entity=mon.2 client_cookie=5933af0d3b85e480 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:02.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.927+0000 7fade8f37700 1 -- 172.21.15.17:0/3591021334 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faddc00b600 con 0x7fade41405d0 2022-08-01T19:41:02.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.927+0000 7fade8f37700 1 -- 172.21.15.17:0/3591021334 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faddc00b760 con 0x7fade41405d0 2022-08-01T19:41:02.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.927+0000 7fade8f37700 1 -- 172.21.15.17:0/3591021334 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faddc00b8d0 con 0x7fade41405d0 2022-08-01T19:41:02.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.927+0000 7fadeaf3b700 1 -- 172.21.15.17:0/3591021334 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fade41405d0 msgr2=0x7fade4140a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:02.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.927+0000 7fadeaf3b700 1 --2- 172.21.15.17:0/3591021334 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fade41405d0 0x7fade4140a80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7faddc0032f0 tx=0x7faddc0091d0).stop 2022-08-01T19:41:02.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.927+0000 7fadeaf3b700 1 -- 172.21.15.17:0/3591021334 shutdown_connections 2022-08-01T19:41:02.966 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.927+0000 7fadeaf3b700 1 --2- 172.21.15.17:0/3591021334 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fade41405d0 0x7fade4140a80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:02.966 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.927+0000 7fadeaf3b700 1 --2- 172.21.15.17:0/3591021334 >> v2:172.21.15.77:3300/0 conn(0x7fade40ad750 0x7fade4061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:02.967 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.927+0000 7fadeaf3b700 1 -- 172.21.15.17:0/3591021334 >> 172.21.15.17:0/3591021334 conn(0x7fade4066f70 msgr2=0x7fade4067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:02.967 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.927+0000 7fadeaf3b700 1 -- 172.21.15.17:0/3591021334 shutdown_connections 2022-08-01T19:41:02.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.927+0000 7fadeaf3b700 1 -- 172.21.15.17:0/3591021334 wait complete. 2022-08-01T19:41:02.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.931+0000 7fadeaf3b700 1 Processor -- start 2022-08-01T19:41:02.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.931+0000 7fadeaf3b700 1 -- start start 2022-08-01T19:41:02.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.931+0000 7fadeaf3b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fade40614b0 0x7fade41422d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:02.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.931+0000 7fadeaf3b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fade40ad750 0x7fade413fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:02.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.931+0000 7fadeaf3b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fade41405d0 0x7fade413ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:02.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.931+0000 7fadeaf3b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade4140460 con 0x7fade41405d0 2022-08-01T19:41:02.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.931+0000 7fadeaf3b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade40b1690 con 0x7fade40ad750 2022-08-01T19:41:02.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.931+0000 7fadeaf3b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade4146110 con 0x7fade40614b0 2022-08-01T19:41:02.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.931+0000 7fadea73a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fade41405d0 0x7fade413ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:02.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.931+0000 7fadea73a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fade41405d0 0x7fade413ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41376/0 (socket says 172.21.15.17:41376) 2022-08-01T19:41:02.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.931+0000 7fade9738700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fade40ad750 0x7fade413fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:02.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.931+0000 7fade9f39700 1 --2- 172.21.15.17:0/89046381 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fade40614b0 0x7fade41422d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:02.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.931+0000 7fadea73a700 1 -- 172.21.15.17:0/89046381 learned_addr learned my addr 172.21.15.17:0/89046381 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:02.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.931+0000 7fadea73a700 1 -- 172.21.15.17:0/89046381 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fade40614b0 msgr2=0x7fade41422d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:02.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.931+0000 7fadea73a700 1 --2- 172.21.15.17:0/89046381 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fade40614b0 0x7fade41422d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:02.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.931+0000 7fadea73a700 1 -- 172.21.15.17:0/89046381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fade40ad750 msgr2=0x7fade413fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:02.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.931+0000 7fadea73a700 1 --2- 172.21.15.17:0/89046381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fade40ad750 0x7fade413fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:02.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.931+0000 7fadea73a700 1 -- 172.21.15.17:0/89046381 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade000b180 con 0x7fade41405d0 2022-08-01T19:41:02.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.931+0000 7fade9738700 1 --2- 172.21.15.17:0/89046381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fade40ad750 0x7fade413fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:02.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.931+0000 7fade9f39700 1 --2- 172.21.15.17:0/89046381 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fade40614b0 0x7fade41422d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:02.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.931+0000 7fadea73a700 1 --2- 172.21.15.17:0/89046381 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fade41405d0 0x7fade413ff70 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7fade4145460 tx=0x7faddc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:02.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.931+0000 7fadd2ffd700 1 -- 172.21.15.17:0/89046381 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faddc00b8d0 con 0x7fade41405d0 2022-08-01T19:41:02.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.931+0000 7fadd2ffd700 1 -- 172.21.15.17:0/89046381 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faddc0205e0 con 0x7fade41405d0 2022-08-01T19:41:02.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.931+0000 7fadd2ffd700 1 -- 172.21.15.17:0/89046381 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faddc00b600 con 0x7fade41405d0 2022-08-01T19:41:02.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.931+0000 7fadeaf3b700 1 -- 172.21.15.17:0/89046381 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faddc009470 con 0x7fade41405d0 2022-08-01T19:41:02.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.935+0000 7fadeaf3b700 1 -- 172.21.15.17:0/89046381 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade40aaba0 con 0x7fade41405d0 2022-08-01T19:41:02.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.935+0000 7fadeaf3b700 1 -- 172.21.15.17:0/89046381 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fade40aaf60 con 0x7fade41405d0 2022-08-01T19:41:02.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.935+0000 7fadd2ffd700 1 -- 172.21.15.17:0/89046381 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7faddc01e070 con 0x7fade41405d0 2022-08-01T19:41:02.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.935+0000 7fadd2ffd700 1 --2- 172.21.15.17:0/89046381 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fadc40539e0 0x7fadc4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:02.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.935+0000 7fadd2ffd700 1 -- 172.21.15.17:0/89046381 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6420+0+0 (secure 0 0 0) 0x7faddc06cd70 con 0x7fade41405d0 2022-08-01T19:41:02.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.935+0000 7fade9f39700 1 --2- 172.21.15.17:0/89046381 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fadc40539e0 0x7fadc4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:02.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.939+0000 7fade9f39700 1 --2- 172.21.15.17:0/89046381 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fadc40539e0 0x7fadc4055ea0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7fadd400d000 tx=0x7fadd4009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:02.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:02.939+0000 7fadd2ffd700 1 -- 172.21.15.17:0/89046381 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7faddc047450 con 0x7fade41405d0 2022-08-01T19:41:03.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.111+0000 7fadeaf3b700 1 -- 172.21.15.17:0/89046381 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fade40aad30 con 0x7fade41405d0 2022-08-01T19:41:03.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.119+0000 7fadd2ffd700 1 -- 172.21.15.17:0/89046381 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v25) v1 ==== 184+0+0 (secure 0 0 0) 0x7faddc015020 con 0x7fade41405d0 2022-08-01T19:41:03.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.119+0000 7fadeaf3b700 1 -- 172.21.15.17:0/89046381 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fadc40539e0 msgr2=0x7fadc4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:03.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.119+0000 7fadeaf3b700 1 --2- 172.21.15.17:0/89046381 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fadc40539e0 0x7fadc4055ea0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7fadd400d000 tx=0x7fadd4009040).stop 2022-08-01T19:41:03.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.119+0000 7fadeaf3b700 1 -- 172.21.15.17:0/89046381 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fade41405d0 msgr2=0x7fade413ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:03.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.119+0000 7fadeaf3b700 1 --2- 172.21.15.17:0/89046381 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fade41405d0 0x7fade413ff70 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7fade4145460 tx=0x7faddc016000).stop 2022-08-01T19:41:03.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.119+0000 7fadeaf3b700 1 -- 172.21.15.17:0/89046381 shutdown_connections 2022-08-01T19:41:03.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.119+0000 7fadeaf3b700 1 --2- 172.21.15.17:0/89046381 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fadc40539e0 0x7fadc4055ea0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:03.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.119+0000 7fadeaf3b700 1 --2- 172.21.15.17:0/89046381 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fade41405d0 0x7fade413ff70 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:03.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.119+0000 7fadeaf3b700 1 --2- 172.21.15.17:0/89046381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fade40ad750 0x7fade413fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:03.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.119+0000 7fadeaf3b700 1 --2- 172.21.15.17:0/89046381 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fade40614b0 0x7fade41422d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:03.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.119+0000 7fadeaf3b700 1 -- 172.21.15.17:0/89046381 >> 172.21.15.17:0/89046381 conn(0x7fade4066f70 msgr2=0x7fade4067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:03.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.119+0000 7fadeaf3b700 1 -- 172.21.15.17:0/89046381 shutdown_connections 2022-08-01T19:41:03.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.119+0000 7fadeaf3b700 1 -- 172.21.15.17:0/89046381 wait complete. 2022-08-01T19:41:03.150 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.0 2022-08-01T19:41:03.151 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T19:41:03.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.271+0000 7f4c1de50700 1 Processor -- start 2022-08-01T19:41:03.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.271+0000 7f4c1de50700 1 -- start start 2022-08-01T19:41:03.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.271+0000 7f4c1de50700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f4c18061890 0x7f4c181466b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:03.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.271+0000 7f4c1de50700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c180aee80 con 0x7f4c180ad750 2022-08-01T19:41:03.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.271+0000 7f4c1de50700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c180b70b0 con 0x7f4c180614b0 2022-08-01T19:41:03.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.271+0000 7f4c1de50700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c180af0b0 con 0x7f4c18061890 2022-08-01T19:41:03.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.271+0000 7f4c17fff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f4c180ad750 0x7f4c181405d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33576/0 (socket says 172.21.15.17:33576) 2022-08-01T19:41:03.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.271+0000 7f4c17fff700 1 -- 172.21.15.17:0/104454264 learned_addr learned my addr 172.21.15.17:0/104454264 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:03.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.271+0000 7f4c1d64f700 1 --2- 172.21.15.17:0/104454264 >> v2:172.21.15.77:3300/0 conn(0x7f4c18061890 0x7f4c181466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:03.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.271+0000 7f4c177fe700 1 -- 172.21.15.17:0/104454264 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3969501997 0 0) 0x7f4c180aee80 con 0x7f4c180ad750 2022-08-01T19:41:03.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.271+0000 7f4c177fe700 1 -- 172.21.15.17:0/104454264 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bfc003260 con 0x7f4c180ad750 2022-08-01T19:41:03.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.271+0000 7f4c177fe700 1 -- 172.21.15.17:0/104454264 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2087110891 0 0) 0x7f4bfc003260 con 0x7f4c180ad750 2022-08-01T19:41:03.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.271+0000 7f4c177fe700 1 -- 172.21.15.17:0/104454264 >> v2:172.21.15.77:3300/0 conn(0x7f4c18061890 msgr2=0x7f4c181466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:03.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.271+0000 7f4c177fe700 1 --2- 172.21.15.17:0/104454264 >> v2:172.21.15.77:3300/0 conn(0x7f4c18061890 0x7f4c181466b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:03.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.271+0000 7f4c177fe700 1 -- 172.21.15.17:0/104454264 >> v1:172.21.15.77:6789/0 conn(0x7f4c180614b0 legacy=0x7f4c180b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:03.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.271+0000 7f4c177fe700 1 -- 172.21.15.17:0/104454264 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c180af350 con 0x7f4c180ad750 2022-08-01T19:41:03.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.271+0000 7f4c177fe700 1 -- 172.21.15.17:0/104454264 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7f4c04001660 con 0x7f4c180ad750 2022-08-01T19:41:03.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.271+0000 7f4c1d64f700 1 --2- 172.21.15.17:0/104454264 >> v2:172.21.15.77:3300/0 conn(0x7f4c18061890 0x7f4c181466b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:03.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.271+0000 7f4c177fe700 1 -- 172.21.15.17:0/104454264 >> v1:172.21.15.17:6789/0 conn(0x7f4c180ad750 legacy=0x7f4c181405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:03.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.271+0000 7f4c177fe700 1 --2- 172.21.15.17:0/104454264 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4bfc003750 0x7f4bfc003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:03.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.271+0000 7f4c177fe700 1 --2- 172.21.15.17:0/104454264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bfc004500 0x7f4bfc004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:03.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.271+0000 7f4c177fe700 1 --2- 172.21.15.17:0/104454264 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4bfc004e70 0x7f4bfc00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:03.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.271+0000 7f4c177fe700 1 -- 172.21.15.17:0/104454264 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c180af0b0 con 0x7f4bfc003750 2022-08-01T19:41:03.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.271+0000 7f4c177fe700 1 -- 172.21.15.17:0/104454264 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc00bd20 con 0x7f4bfc004500 2022-08-01T19:41:03.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.271+0000 7f4c177fe700 1 -- 172.21.15.17:0/104454264 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc00bee0 con 0x7f4bfc004e70 2022-08-01T19:41:03.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.271+0000 7f4c17fff700 1 --2- 172.21.15.17:0/104454264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bfc004500 0x7f4bfc004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:03.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.271+0000 7f4c1d64f700 1 --2- 172.21.15.17:0/104454264 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4bfc004e70 0x7f4bfc00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:03.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.271+0000 7f4c1ce4e700 1 --2- 172.21.15.17:0/104454264 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4bfc003750 0x7f4bfc003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:03.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.275+0000 7f4c1d64f700 1 -- 172.21.15.17:0/104454264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bfc004500 msgr2=0x7f4bfc004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:03.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.275+0000 7f4c1d64f700 1 --2- 172.21.15.17:0/104454264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bfc004500 0x7f4bfc004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:03.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.275+0000 7f4c1d64f700 1 -- 172.21.15.17:0/104454264 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4bfc003750 msgr2=0x7f4bfc003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:03.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.275+0000 7f4c1d64f700 1 --2- 172.21.15.17:0/104454264 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4bfc003750 0x7f4bfc003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:03.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.275+0000 7f4c1d64f700 1 -- 172.21.15.17:0/104454264 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bfc003260 con 0x7f4bfc004e70 2022-08-01T19:41:03.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.275+0000 7f4c1d64f700 1 -- 172.21.15.17:0/104454264 reap_dead start 2022-08-01T19:41:03.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.275+0000 7f4c1d64f700 1 --2- 172.21.15.17:0/104454264 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4bfc004e70 0x7f4bfc00b750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f4c1000c000 tx=0x7f4c100091d0).ready entity=mon.2 client_cookie=476283868bc92833 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:03.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.275+0000 7f4c177fe700 1 -- 172.21.15.17:0/104454264 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c1000e920 con 0x7f4bfc004e70 2022-08-01T19:41:03.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.275+0000 7f4c177fe700 1 -- 172.21.15.17:0/104454264 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1000ea80 con 0x7f4bfc004e70 2022-08-01T19:41:03.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.275+0000 7f4c177fe700 1 -- 172.21.15.17:0/104454264 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c1000ede0 con 0x7f4bfc004e70 2022-08-01T19:41:03.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.275+0000 7f4c1de50700 1 -- 172.21.15.17:0/104454264 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4bfc004e70 msgr2=0x7f4bfc00b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:03.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.275+0000 7f4c1de50700 1 --2- 172.21.15.17:0/104454264 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4bfc004e70 0x7f4bfc00b750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f4c1000c000 tx=0x7f4c100091d0).stop 2022-08-01T19:41:03.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.275+0000 7f4c1de50700 1 -- 172.21.15.17:0/104454264 shutdown_connections 2022-08-01T19:41:03.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.275+0000 7f4c1de50700 1 --2- 172.21.15.17:0/104454264 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4bfc004e70 0x7f4bfc00b750 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:03.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.275+0000 7f4c1de50700 1 -- 172.21.15.17:0/104454264 >> 172.21.15.17:0/104454264 conn(0x7f4c18066f70 msgr2=0x7f4c18067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:03.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.275+0000 7f4c1de50700 1 -- 172.21.15.17:0/104454264 shutdown_connections 2022-08-01T19:41:03.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.275+0000 7f4c1de50700 1 -- 172.21.15.17:0/104454264 wait complete. 2022-08-01T19:41:03.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.275+0000 7f4c1de50700 1 Processor -- start 2022-08-01T19:41:03.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.275+0000 7f4c1de50700 1 -- start start 2022-08-01T19:41:03.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.279+0000 7f4c1de50700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4bfc004e70 0x7f4c180a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:03.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.279+0000 7f4c1de50700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4c180a0580 0x7f4c180b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:03.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.279+0000 7f4c1de50700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c1809ea40 0x7f4c180b1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:03.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.279+0000 7f4c1de50700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c180a0e30 con 0x7f4bfc004e70 2022-08-01T19:41:03.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.279+0000 7f4c1de50700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1809ef50 con 0x7f4c1809ea40 2022-08-01T19:41:03.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.279+0000 7f4c1de50700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1809fb90 con 0x7f4c180a0580 2022-08-01T19:41:03.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.279+0000 7f4c1ce4e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4bfc004e70 0x7f4c180a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:03.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.279+0000 7f4c1ce4e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4bfc004e70 0x7f4c180a0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41392/0 (socket says 172.21.15.17:41392) 2022-08-01T19:41:03.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.279+0000 7f4c1ce4e700 1 -- 172.21.15.17:0/1535025617 learned_addr learned my addr 172.21.15.17:0/1535025617 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:03.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.279+0000 7f4c17fff700 1 --2- 172.21.15.17:0/1535025617 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4c180a0580 0x7f4c180b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:03.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.279+0000 7f4c1d64f700 1 --2- 172.21.15.17:0/1535025617 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c1809ea40 0x7f4c180b1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:03.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.279+0000 7f4c1ce4e700 1 -- 172.21.15.17:0/1535025617 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4c180a0580 msgr2=0x7f4c180b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:03.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.279+0000 7f4c1ce4e700 1 --2- 172.21.15.17:0/1535025617 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4c180a0580 0x7f4c180b1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:03.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.279+0000 7f4c1ce4e700 1 -- 172.21.15.17:0/1535025617 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c1809ea40 msgr2=0x7f4c180b1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:03.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.279+0000 7f4c1ce4e700 1 --2- 172.21.15.17:0/1535025617 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c1809ea40 0x7f4c180b1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:03.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.279+0000 7f4c1ce4e700 1 -- 172.21.15.17:0/1535025617 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c1809f760 con 0x7f4bfc004e70 2022-08-01T19:41:03.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.279+0000 7f4c1ce4e700 1 --2- 172.21.15.17:0/1535025617 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4bfc004e70 0x7f4c180a0090 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7f4c1809fcd0 tx=0x7f4c0c0079e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:03.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.279+0000 7f4c15ffb700 1 -- 172.21.15.17:0/1535025617 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c0c0135e0 con 0x7f4bfc004e70 2022-08-01T19:41:03.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.279+0000 7f4c15ffb700 1 -- 172.21.15.17:0/1535025617 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c0c013740 con 0x7f4bfc004e70 2022-08-01T19:41:03.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.279+0000 7f4c1de50700 1 -- 172.21.15.17:0/1535025617 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c180a0b90 con 0x7f4bfc004e70 2022-08-01T19:41:03.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.279+0000 7f4c15ffb700 1 -- 172.21.15.17:0/1535025617 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c0c013aa0 con 0x7f4bfc004e70 2022-08-01T19:41:03.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.279+0000 7f4c1de50700 1 -- 172.21.15.17:0/1535025617 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c180b30c0 con 0x7f4bfc004e70 2022-08-01T19:41:03.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.283+0000 7f4c1de50700 1 -- 172.21.15.17:0/1535025617 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c180af5f0 con 0x7f4bfc004e70 2022-08-01T19:41:03.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.283+0000 7f4c15ffb700 1 -- 172.21.15.17:0/1535025617 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4c0c013c00 con 0x7f4bfc004e70 2022-08-01T19:41:03.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.283+0000 7f4c15ffb700 1 --2- 172.21.15.17:0/1535025617 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4bf8053b00 0x7f4bf8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:03.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.283+0000 7f4c15ffb700 1 -- 172.21.15.17:0/1535025617 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4c0c06c1d0 con 0x7f4bfc004e70 2022-08-01T19:41:03.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.283+0000 7f4c17fff700 1 --2- 172.21.15.17:0/1535025617 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4bf8053b00 0x7f4bf8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:03.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.283+0000 7f4c17fff700 1 --2- 172.21.15.17:0/1535025617 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4bf8053b00 0x7f4bf8055fc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f4c04005000 tx=0x7f4c180af4e0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:03.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.287+0000 7f4c15ffb700 1 -- 172.21.15.17:0/1535025617 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4c0c042f90 con 0x7f4bfc004e70 2022-08-01T19:41:03.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.455+0000 7f4c1de50700 1 -- 172.21.15.17:0/1535025617 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f4c180574e0 con 0x7f4bfc004e70 2022-08-01T19:41:03.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.463+0000 7f4c15ffb700 1 -- 172.21.15.17:0/1535025617 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v26) v1 ==== 184+0+0 (secure 0 0 0) 0x7f4c0c0465b0 con 0x7f4bfc004e70 2022-08-01T19:41:03.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.463+0000 7f4c1de50700 1 -- 172.21.15.17:0/1535025617 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4bf8053b00 msgr2=0x7f4bf8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:03.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.463+0000 7f4c1de50700 1 --2- 172.21.15.17:0/1535025617 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4bf8053b00 0x7f4bf8055fc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f4c04005000 tx=0x7f4c180af4e0).stop 2022-08-01T19:41:03.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.463+0000 7f4c1de50700 1 -- 172.21.15.17:0/1535025617 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4bfc004e70 msgr2=0x7f4c180a0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:03.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.463+0000 7f4c1de50700 1 --2- 172.21.15.17:0/1535025617 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4bfc004e70 0x7f4c180a0090 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7f4c1809fcd0 tx=0x7f4c0c0079e0).stop 2022-08-01T19:41:03.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.463+0000 7f4c1de50700 1 -- 172.21.15.17:0/1535025617 shutdown_connections 2022-08-01T19:41:03.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.463+0000 7f4c1de50700 1 --2- 172.21.15.17:0/1535025617 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4bf8053b00 0x7f4bf8055fc0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:03.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.463+0000 7f4c1de50700 1 --2- 172.21.15.17:0/1535025617 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c1809ea40 0x7f4c180b1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:03.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.463+0000 7f4c1de50700 1 --2- 172.21.15.17:0/1535025617 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4c180a0580 0x7f4c180b1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:03.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.463+0000 7f4c1de50700 1 --2- 172.21.15.17:0/1535025617 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4bfc004e70 0x7f4c180a0090 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:03.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.463+0000 7f4c1de50700 1 -- 172.21.15.17:0/1535025617 >> 172.21.15.17:0/1535025617 conn(0x7f4c18066f70 msgr2=0x7f4c18141d10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:03.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.463+0000 7f4c1de50700 1 -- 172.21.15.17:0/1535025617 shutdown_connections 2022-08-01T19:41:03.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.463+0000 7f4c1de50700 1 -- 172.21.15.17:0/1535025617 wait complete. 2022-08-01T19:41:03.478 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.1 2022-08-01T19:41:03.481 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:41:03.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.587+0000 7fe08f571700 1 Processor -- start 2022-08-01T19:41:03.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.587+0000 7fe08f571700 1 -- start start 2022-08-01T19:41:03.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.591+0000 7fe08f571700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fe0880b1f40 0x7fe0880b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:03.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.591+0000 7fe08f571700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe088148530 0x7fe0880b2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:03.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.591+0000 7fe08f571700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0880b04e0 con 0x7fe0880a7250 2022-08-01T19:41:03.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.591+0000 7fe08f571700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0880a9250 con 0x7fe088148530 2022-08-01T19:41:03.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.591+0000 7fe08f571700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0880b0780 con 0x7fe0880b1f40 2022-08-01T19:41:03.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.591+0000 7fe08cb0c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe088148530 0x7fe0880b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:03.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.591+0000 7fe08cb0c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe088148530 0x7fe0880b2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41400/0 (socket says 172.21.15.17:41400) 2022-08-01T19:41:03.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.591+0000 7fe08cb0c700 1 -- 172.21.15.17:0/1781858557 learned_addr learned my addr 172.21.15.17:0/1781858557 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:03.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.591+0000 7fe08cb0c700 1 -- 172.21.15.17:0/1781858557 >> v2:172.21.15.77:3300/0 conn(0x7fe0880b1f40 msgr2=0x7fe0880b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:03.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.591+0000 7fe08cb0c700 1 --2- 172.21.15.17:0/1781858557 >> v2:172.21.15.77:3300/0 conn(0x7fe0880b1f40 0x7fe0880b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2022-08-01T19:41:03.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.591+0000 7fe08cb0c700 1 -- 172.21.15.17:0/1781858557 >> v1:172.21.15.77:6789/0 conn(0x7fe0880a7250 legacy=0x7fe0881448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:03.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.591+0000 7fe08cb0c700 1 -- 172.21.15.17:0/1781858557 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0880b0a20 con 0x7fe088148530 2022-08-01T19:41:03.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.591+0000 7fe08cb0c700 1 --2- 172.21.15.17:0/1781858557 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe088148530 0x7fe0880b2320 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7fe0880acff0 tx=0x7fe084006eb0).ready entity=mon.0 client_cookie=23ea291634b6ca7f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:03.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.591+0000 7fe07ffff700 1 -- 172.21.15.17:0/1781858557 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe08400d5c0 con 0x7fe088148530 2022-08-01T19:41:03.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.591+0000 7fe07ffff700 1 -- 172.21.15.17:0/1781858557 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe084000e90 con 0x7fe088148530 2022-08-01T19:41:03.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.591+0000 7fe07ffff700 1 -- 172.21.15.17:0/1781858557 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe08400d950 con 0x7fe088148530 2022-08-01T19:41:03.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.591+0000 7fe08f571700 1 -- 172.21.15.17:0/1781858557 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe088148530 msgr2=0x7fe0880b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:03.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.591+0000 7fe08f571700 1 --2- 172.21.15.17:0/1781858557 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe088148530 0x7fe0880b2320 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7fe0880acff0 tx=0x7fe084006eb0).stop 2022-08-01T19:41:03.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.591+0000 7fe08f571700 1 -- 172.21.15.17:0/1781858557 shutdown_connections 2022-08-01T19:41:03.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.591+0000 7fe08f571700 1 --2- 172.21.15.17:0/1781858557 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe088148530 0x7fe0880b2320 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:03.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.591+0000 7fe08f571700 1 --2- 172.21.15.17:0/1781858557 >> v2:172.21.15.77:3300/0 conn(0x7fe0880b1f40 0x7fe0880b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2022-08-01T19:41:03.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.591+0000 7fe08f571700 1 -- 172.21.15.17:0/1781858557 >> 172.21.15.17:0/1781858557 conn(0x7fe08805f840 msgr2=0x7fe08805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:03.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.591+0000 7fe08f571700 1 -- 172.21.15.17:0/1781858557 shutdown_connections 2022-08-01T19:41:03.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.591+0000 7fe08f571700 1 -- 172.21.15.17:0/1781858557 wait complete. 2022-08-01T19:41:03.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.595+0000 7fe08f571700 1 Processor -- start 2022-08-01T19:41:03.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.595+0000 7fe08f571700 1 -- start start 2022-08-01T19:41:03.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.595+0000 7fe08f571700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0880a7250 0x7fe088145f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:03.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.595+0000 7fe08f571700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0880b1f40 0x7fe0880abd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:03.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.595+0000 7fe08f571700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe088148530 0x7fe0880ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:03.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.595+0000 7fe08f571700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0880ac250 con 0x7fe0880a7250 2022-08-01T19:41:03.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.595+0000 7fe08f571700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0880ae4f0 con 0x7fe0880b1f40 2022-08-01T19:41:03.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.595+0000 7fe08f571700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0880ab080 con 0x7fe088148530 2022-08-01T19:41:03.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.595+0000 7fe08d30d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0880a7250 0x7fe088145f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:03.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.595+0000 7fe08d30d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0880a7250 0x7fe088145f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41406/0 (socket says 172.21.15.17:41406) 2022-08-01T19:41:03.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.595+0000 7fe08d30d700 1 -- 172.21.15.17:0/1195614395 learned_addr learned my addr 172.21.15.17:0/1195614395 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:03.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.595+0000 7fe08cb0c700 1 --2- 172.21.15.17:0/1195614395 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0880b1f40 0x7fe0880abd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:03.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.595+0000 7fe08db0e700 1 --2- 172.21.15.17:0/1195614395 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe088148530 0x7fe0880ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:03.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.595+0000 7fe08d30d700 1 -- 172.21.15.17:0/1195614395 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe088148530 msgr2=0x7fe0880ab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:03.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.595+0000 7fe08d30d700 1 --2- 172.21.15.17:0/1195614395 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe088148530 0x7fe0880ab4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:03.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.595+0000 7fe08d30d700 1 -- 172.21.15.17:0/1195614395 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0880b1f40 msgr2=0x7fe0880abd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:03.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.595+0000 7fe08d30d700 1 --2- 172.21.15.17:0/1195614395 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0880b1f40 0x7fe0880abd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:03.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.595+0000 7fe08d30d700 1 -- 172.21.15.17:0/1195614395 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe08400b1e0 con 0x7fe0880a7250 2022-08-01T19:41:03.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.595+0000 7fe08d30d700 1 --2- 172.21.15.17:0/1195614395 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0880a7250 0x7fe088145f80 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 rx=0x7fe078002f70 tx=0x7fe078006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:03.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.599+0000 7fe07e7fc700 1 -- 172.21.15.17:0/1195614395 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe078009ea0 con 0x7fe0880a7250 2022-08-01T19:41:03.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.599+0000 7fe07e7fc700 1 -- 172.21.15.17:0/1195614395 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe078013460 con 0x7fe0880a7250 2022-08-01T19:41:03.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.599+0000 7fe08f571700 1 -- 172.21.15.17:0/1195614395 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0880ae9c0 con 0x7fe0880a7250 2022-08-01T19:41:03.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.599+0000 7fe07e7fc700 1 -- 172.21.15.17:0/1195614395 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0780137c0 con 0x7fe0880a7250 2022-08-01T19:41:03.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.599+0000 7fe08f571700 1 -- 172.21.15.17:0/1195614395 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0880ae170 con 0x7fe0880a7250 2022-08-01T19:41:03.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.599+0000 7fe07e7fc700 1 -- 172.21.15.17:0/1195614395 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe07800f070 con 0x7fe0880a7250 2022-08-01T19:41:03.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.599+0000 7fe08f571700 1 -- 172.21.15.17:0/1195614395 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0880b0cc0 con 0x7fe0880a7250 2022-08-01T19:41:03.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.599+0000 7fe07e7fc700 1 --2- 172.21.15.17:0/1195614395 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe070053b30 0x7fe070055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:03.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.599+0000 7fe07e7fc700 1 -- 172.21.15.17:0/1195614395 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe07806b880 con 0x7fe0880a7250 2022-08-01T19:41:03.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.599+0000 7fe08cb0c700 1 --2- 172.21.15.17:0/1195614395 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe070053b30 0x7fe070055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:03.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.599+0000 7fe08cb0c700 1 --2- 172.21.15.17:0/1195614395 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe070053b30 0x7fe070055ff0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7fe08401c000 tx=0x7fe084015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:03.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.607+0000 7fe07e7fc700 1 -- 172.21.15.17:0/1195614395 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe0780425a0 con 0x7fe0880a7250 2022-08-01T19:41:03.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.767+0000 7fe08f571700 1 -- 172.21.15.17:0/1195614395 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0880574e0 con 0x7fe0880a7250 2022-08-01T19:41:03.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.767+0000 7fe07e7fc700 1 -- 172.21.15.17:0/1195614395 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+2493 (secure 0 0 0) 0x7fe078045bc0 con 0x7fe0880a7250 2022-08-01T19:41:03.772 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:03.772 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":40,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6615,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3580428616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3580428616},{"type":"v1","addr":"172.21.15.17:6833","nonce":3580428616}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38},{"gid":6627,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/4048623373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":4048623373},{"type":"v1","addr":"172.21.15.17:6835","nonce":4048623373}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:41:01.207552+0000","modified":"2022-08-01T19:41:02.210147+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6630},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6630":{"gid":6630,"name":"b","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6835/4275403969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4275403969},{"type":"v1","addr":"172.21.15.77:6835","nonce":4275403969}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2022-08-01T19:41:03.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.771+0000 7fe08f571700 1 -- 172.21.15.17:0/1195614395 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe070053b30 msgr2=0x7fe070055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:03.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.771+0000 7fe08f571700 1 --2- 172.21.15.17:0/1195614395 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe070053b30 0x7fe070055ff0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7fe08401c000 tx=0x7fe084015000).stop 2022-08-01T19:41:03.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.771+0000 7fe08f571700 1 -- 172.21.15.17:0/1195614395 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0880a7250 msgr2=0x7fe088145f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:03.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.771+0000 7fe08f571700 1 --2- 172.21.15.17:0/1195614395 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0880a7250 0x7fe088145f80 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 rx=0x7fe078002f70 tx=0x7fe078006ec0).stop 2022-08-01T19:41:03.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.771+0000 7fe08f571700 1 -- 172.21.15.17:0/1195614395 shutdown_connections 2022-08-01T19:41:03.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.771+0000 7fe08f571700 1 --2- 172.21.15.17:0/1195614395 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe070053b30 0x7fe070055ff0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:03.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.771+0000 7fe08f571700 1 --2- 172.21.15.17:0/1195614395 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe088148530 0x7fe0880ab4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:03.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.771+0000 7fe08f571700 1 --2- 172.21.15.17:0/1195614395 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0880b1f40 0x7fe0880abd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:03.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.771+0000 7fe08f571700 1 --2- 172.21.15.17:0/1195614395 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0880a7250 0x7fe088145f80 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:03.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.771+0000 7fe08f571700 1 -- 172.21.15.17:0/1195614395 >> 172.21.15.17:0/1195614395 conn(0x7fe08805f840 msgr2=0x7fe088144b20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:03.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.771+0000 7fe08f571700 1 -- 172.21.15.17:0/1195614395 shutdown_connections 2022-08-01T19:41:03.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:03.771+0000 7fe08f571700 1 -- 172.21.15.17:0/1195614395 wait complete. 2022-08-01T19:41:03.783 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 40 2022-08-01T19:41:04.090 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 40, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:41:01.207552+0000', 'modified': '2022-08-01T19:41:02.210147+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6630}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6630': {'gid': 6630, 'name': 'b', 'rank': 0, 'incarnation': 39, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.77:6835/4275403969', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6834', 'nonce': 4275403969}, {'type': 'v1', 'addr': '172.21.15.77:6835', 'nonce': 4275403969}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:41:04.090 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:41:04.091 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-08-01T19:41:04.207 INFO:teuthology.orchestra.run.smithi077.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":6630,"want_state":"up:active","state":"up:active","rank_uptime":2.987776572,"mdsmap_epoch":40,"osdmap_epoch":88,"osdmap_epoch_barrier":87,"uptime":7.8215478679999997} 2022-08-01T19:41:04.208 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 6630, "mdsmap_epoch": 40, "osdmap_epoch": 88, "osdmap_epoch_barrier": 87, "rank_uptime": 2.987776572, "state": "up:active", "uptime": 7.821547868, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:41:04.209 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:41:04.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.319+0000 7f846deab700 1 Processor -- start 2022-08-01T19:41:04.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.319+0000 7f846deab700 1 -- start start 2022-08-01T19:41:04.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.323+0000 7f846deab700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84680b1f40 0x7f84680b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:04.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.323+0000 7f846deab700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84680b04e0 con 0x7f84680a7250 2022-08-01T19:41:04.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.323+0000 7f846deab700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84680b0780 con 0x7f8468148530 2022-08-01T19:41:04.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.323+0000 7f846deab700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84680a9250 con 0x7f84680b1f40 2022-08-01T19:41:04.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.323+0000 7f8467fff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f84680a7250 0x7f84681448b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33612/0 (socket says 172.21.15.17:33612) 2022-08-01T19:41:04.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.323+0000 7f8467fff700 1 -- 172.21.15.17:0/3796923941 learned_addr learned my addr 172.21.15.17:0/3796923941 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:04.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.323+0000 7f84677fe700 1 --2- 172.21.15.17:0/3796923941 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84680b1f40 0x7f84680b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:04.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.323+0000 7f84667fc700 1 -- 172.21.15.17:0/3796923941 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3231542502 0 0) 0x7f84680b04e0 con 0x7f84680a7250 2022-08-01T19:41:04.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.323+0000 7f84667fc700 1 -- 172.21.15.17:0/3796923941 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8454003260 con 0x7f84680a7250 2022-08-01T19:41:04.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.323+0000 7f84667fc700 1 -- 172.21.15.17:0/3796923941 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2540887957 0 0) 0x7f84680b0780 con 0x7f8468148530 2022-08-01T19:41:04.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.323+0000 7f84667fc700 1 -- 172.21.15.17:0/3796923941 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84680b04e0 con 0x7f8468148530 2022-08-01T19:41:04.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.323+0000 7f84677fe700 1 -- 172.21.15.17:0/3796923941 >> v1:172.21.15.77:6789/0 conn(0x7f8468148530 legacy=0x7f84680b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:04.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.323+0000 7f84677fe700 1 -- 172.21.15.17:0/3796923941 >> v1:172.21.15.17:6789/0 conn(0x7f84680a7250 legacy=0x7f84681448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:04.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.323+0000 7f84677fe700 1 -- 172.21.15.17:0/3796923941 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84680b0a20 con 0x7f84680b1f40 2022-08-01T19:41:04.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.323+0000 7f84677fe700 1 --2- 172.21.15.17:0/3796923941 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84680b1f40 0x7f84680b1160 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f845c002f70 tx=0x7f845c006df0).ready entity=mon.2 client_cookie=19201485adc6dc29 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:04.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.323+0000 7f84667fc700 1 -- 172.21.15.17:0/3796923941 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f845c00d5c0 con 0x7f84680b1f40 2022-08-01T19:41:04.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.323+0000 7f84667fc700 1 -- 172.21.15.17:0/3796923941 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f845c000e90 con 0x7f84680b1f40 2022-08-01T19:41:04.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.323+0000 7f84667fc700 1 -- 172.21.15.17:0/3796923941 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f845c00d910 con 0x7f84680b1f40 2022-08-01T19:41:04.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.323+0000 7f846deab700 1 -- 172.21.15.17:0/3796923941 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84680b1f40 msgr2=0x7f84680b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:04.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.323+0000 7f846deab700 1 --2- 172.21.15.17:0/3796923941 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84680b1f40 0x7f84680b1160 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f845c002f70 tx=0x7f845c006df0).stop 2022-08-01T19:41:04.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.323+0000 7f846deab700 1 -- 172.21.15.17:0/3796923941 shutdown_connections 2022-08-01T19:41:04.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.323+0000 7f846deab700 1 --2- 172.21.15.17:0/3796923941 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84680b1f40 0x7f84680b1160 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:04.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.327+0000 7f846deab700 1 -- 172.21.15.17:0/3796923941 >> 172.21.15.17:0/3796923941 conn(0x7f846805f840 msgr2=0x7f846805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:04.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.327+0000 7f846deab700 1 -- 172.21.15.17:0/3796923941 shutdown_connections 2022-08-01T19:41:04.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.327+0000 7f846deab700 1 -- 172.21.15.17:0/3796923941 wait complete. 2022-08-01T19:41:04.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.327+0000 7f846deab700 1 Processor -- start 2022-08-01T19:41:04.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.327+0000 7f846deab700 1 -- start start 2022-08-01T19:41:04.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.331+0000 7f846deab700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84680a7250 0x7f8468156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:04.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.331+0000 7f846deab700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84680b1f40 0x7f8468140640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:04.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.331+0000 7f846deab700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8468148530 0x7f846815a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:04.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.331+0000 7f846deab700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84680ae550 con 0x7f84680b1f40 2022-08-01T19:41:04.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.331+0000 7f846deab700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84680ad8d0 con 0x7f84680a7250 2022-08-01T19:41:04.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.331+0000 7f846deab700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84680adb70 con 0x7f8468148530 2022-08-01T19:41:04.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.331+0000 7f8466ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84680b1f40 0x7f8468140640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:04.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.331+0000 7f8466ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84680b1f40 0x7f8468140640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41422/0 (socket says 172.21.15.17:41422) 2022-08-01T19:41:04.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.331+0000 7f8466ffd700 1 -- 172.21.15.17:0/1646946106 learned_addr learned my addr 172.21.15.17:0/1646946106 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:04.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.331+0000 7f8467fff700 1 --2- 172.21.15.17:0/1646946106 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8468148530 0x7f846815a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:04.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.331+0000 7f84677fe700 1 --2- 172.21.15.17:0/1646946106 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84680a7250 0x7f8468156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:04.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.331+0000 7f8466ffd700 1 -- 172.21.15.17:0/1646946106 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8468148530 msgr2=0x7f846815a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:04.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.331+0000 7f8466ffd700 1 --2- 172.21.15.17:0/1646946106 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8468148530 0x7f846815a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:04.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.331+0000 7f8466ffd700 1 -- 172.21.15.17:0/1646946106 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84680a7250 msgr2=0x7f8468156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:04.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.331+0000 7f8466ffd700 1 --2- 172.21.15.17:0/1646946106 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84680a7250 0x7f8468156d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:04.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.331+0000 7f8466ffd700 1 -- 172.21.15.17:0/1646946106 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f845c00b180 con 0x7f84680b1f40 2022-08-01T19:41:04.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.331+0000 7f8466ffd700 1 --2- 172.21.15.17:0/1646946106 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84680b1f40 0x7f8468140640 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7f8458001490 tx=0x7f8458007e30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:04.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.331+0000 7f8464ff9700 1 -- 172.21.15.17:0/1646946106 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8458002310 con 0x7f84680b1f40 2022-08-01T19:41:04.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.331+0000 7f8464ff9700 1 -- 172.21.15.17:0/1646946106 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8458002470 con 0x7f84680b1f40 2022-08-01T19:41:04.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.331+0000 7f846deab700 1 -- 172.21.15.17:0/1646946106 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84680ade10 con 0x7f84680b1f40 2022-08-01T19:41:04.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.331+0000 7f8464ff9700 1 -- 172.21.15.17:0/1646946106 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84580017a0 con 0x7f84680b1f40 2022-08-01T19:41:04.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.331+0000 7f846deab700 1 -- 172.21.15.17:0/1646946106 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84680ad550 con 0x7f84680b1f40 2022-08-01T19:41:04.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.335+0000 7f846deab700 1 -- 172.21.15.17:0/1646946106 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84680574e0 con 0x7f84680b1f40 2022-08-01T19:41:04.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.335+0000 7f8464ff9700 1 -- 172.21.15.17:0/1646946106 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8458019020 con 0x7f84680b1f40 2022-08-01T19:41:04.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.335+0000 7f8464ff9700 1 --2- 172.21.15.17:0/1646946106 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8448053a70 0x7f8448055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:04.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.335+0000 7f8464ff9700 1 -- 172.21.15.17:0/1646946106 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f845806bee0 con 0x7f84680b1f40 2022-08-01T19:41:04.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.335+0000 7f84677fe700 1 --2- 172.21.15.17:0/1646946106 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8448053a70 0x7f8448055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:04.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.335+0000 7f84677fe700 1 --2- 172.21.15.17:0/1646946106 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8448053a70 0x7f8448055f30 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f845c01a000 tx=0x7f845c014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:04.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.343+0000 7f8464ff9700 1 -- 172.21.15.17:0/1646946106 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8458042b30 con 0x7f84680b1f40 2022-08-01T19:41:04.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.471+0000 7f846deab700 1 -- 172.21.15.17:0/1646946106 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8468148310 con 0x7f84680b1f40 2022-08-01T19:41:04.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.471+0000 7f8464ff9700 1 -- 172.21.15.17:0/1646946106 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f8458046150 con 0x7f84680b1f40 2022-08-01T19:41:04.477 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:04.478 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":88,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:41:02.210932+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:40:59.229815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2022-08-01T19:41:00.225322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:41:04.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.475+0000 7f846deab700 1 -- 172.21.15.17:0/1646946106 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8448053a70 msgr2=0x7f8448055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:04.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.475+0000 7f846deab700 1 --2- 172.21.15.17:0/1646946106 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8448053a70 0x7f8448055f30 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f845c01a000 tx=0x7f845c014000).stop 2022-08-01T19:41:04.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.475+0000 7f846deab700 1 -- 172.21.15.17:0/1646946106 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84680b1f40 msgr2=0x7f8468140640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:04.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.475+0000 7f846deab700 1 --2- 172.21.15.17:0/1646946106 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84680b1f40 0x7f8468140640 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7f8458001490 tx=0x7f8458007e30).stop 2022-08-01T19:41:04.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.475+0000 7f846deab700 1 -- 172.21.15.17:0/1646946106 shutdown_connections 2022-08-01T19:41:04.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.475+0000 7f846deab700 1 --2- 172.21.15.17:0/1646946106 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8448053a70 0x7f8448055f30 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:04.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.475+0000 7f846deab700 1 --2- 172.21.15.17:0/1646946106 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8468148530 0x7f846815a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:04.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.475+0000 7f846deab700 1 --2- 172.21.15.17:0/1646946106 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84680b1f40 0x7f8468140640 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:04.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.475+0000 7f846deab700 1 --2- 172.21.15.17:0/1646946106 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84680a7250 0x7f8468156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:04.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.475+0000 7f846deab700 1 -- 172.21.15.17:0/1646946106 >> 172.21.15.17:0/1646946106 conn(0x7f846805f840 msgr2=0x7f84680b1080 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:04.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.475+0000 7f846deab700 1 -- 172.21.15.17:0/1646946106 shutdown_connections 2022-08-01T19:41:04.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.475+0000 7f846deab700 1 -- 172.21.15.17:0/1646946106 wait complete. 2022-08-01T19:41:04.493 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T19:40:59.229815+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T19:41:00.225322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T19:41:04.494 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:41:04.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.603+0000 7f2da655a700 1 Processor -- start 2022-08-01T19:41:04.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.603+0000 7f2da655a700 1 -- start start 2022-08-01T19:41:04.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.603+0000 7f2da655a700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f2da0146ae0 0x7f2da01403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:04.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.603+0000 7f2da655a700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2da00b4820 0x7f2da00b5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:04.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.603+0000 7f2da655a700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2da0143cf0 con 0x7f2da00b4310 2022-08-01T19:41:04.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.603+0000 7f2da655a700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2da01476a0 con 0x7f2da00b4820 2022-08-01T19:41:04.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.603+0000 7f2da655a700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2da0143f90 con 0x7f2da0146ae0 2022-08-01T19:41:04.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.603+0000 7f2d9f7fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2da00b4310 0x7f2da01408b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33624/0 (socket says 172.21.15.17:33624) 2022-08-01T19:41:04.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.603+0000 7f2d9f7fe700 1 -- 172.21.15.17:0/1934059155 learned_addr learned my addr 172.21.15.17:0/1934059155 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:04.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.603+0000 7f2da4af7700 1 --2- 172.21.15.17:0/1934059155 >> v2:172.21.15.17:3300/0 conn(0x7f2da00b4820 0x7f2da00b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:04.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.603+0000 7f2d9ffff700 1 --2- 172.21.15.17:0/1934059155 >> v2:172.21.15.77:3300/0 conn(0x7f2da0146ae0 0x7f2da01403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:04.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.603+0000 7f2d9effd700 1 -- 172.21.15.17:0/1934059155 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1496234909 0 0) 0x7f2da0143cf0 con 0x7f2da00b4310 2022-08-01T19:41:04.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.603+0000 7f2d9effd700 1 -- 172.21.15.17:0/1934059155 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d8c003260 con 0x7f2da00b4310 2022-08-01T19:41:04.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.603+0000 7f2da4af7700 1 -- 172.21.15.17:0/1934059155 >> v2:172.21.15.77:3300/0 conn(0x7f2da0146ae0 msgr2=0x7f2da01403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:04.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.603+0000 7f2da4af7700 1 --2- 172.21.15.17:0/1934059155 >> v2:172.21.15.77:3300/0 conn(0x7f2da0146ae0 0x7f2da01403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:04.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.603+0000 7f2da4af7700 1 -- 172.21.15.17:0/1934059155 >> v1:172.21.15.17:6789/0 conn(0x7f2da00b4310 legacy=0x7f2da01408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:04.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.603+0000 7f2da4af7700 1 -- 172.21.15.17:0/1934059155 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da00b0e20 con 0x7f2da00b4820 2022-08-01T19:41:04.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.603+0000 7f2da4af7700 1 --2- 172.21.15.17:0/1934059155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2da00b4820 0x7f2da00b5060 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0x7f2d980032f0 tx=0x7f2d980091d0).ready entity=mon.0 client_cookie=3179fccd39a0b04e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:04.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.603+0000 7f2d9effd700 1 -- 172.21.15.17:0/1934059155 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d9800b600 con 0x7f2da00b4820 2022-08-01T19:41:04.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.603+0000 7f2d9effd700 1 -- 172.21.15.17:0/1934059155 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9800b760 con 0x7f2da00b4820 2022-08-01T19:41:04.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.603+0000 7f2d9effd700 1 -- 172.21.15.17:0/1934059155 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d9800b8d0 con 0x7f2da00b4820 2022-08-01T19:41:04.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.607+0000 7f2da655a700 1 -- 172.21.15.17:0/1934059155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2da00b4820 msgr2=0x7f2da00b5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:04.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.607+0000 7f2da655a700 1 --2- 172.21.15.17:0/1934059155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2da00b4820 0x7f2da00b5060 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0x7f2d980032f0 tx=0x7f2d980091d0).stop 2022-08-01T19:41:04.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.607+0000 7f2da655a700 1 -- 172.21.15.17:0/1934059155 shutdown_connections 2022-08-01T19:41:04.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.607+0000 7f2da655a700 1 --2- 172.21.15.17:0/1934059155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2da00b4820 0x7f2da00b5060 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:04.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.607+0000 7f2da655a700 1 --2- 172.21.15.17:0/1934059155 >> v2:172.21.15.77:3300/0 conn(0x7f2da0146ae0 0x7f2da01403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:04.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.607+0000 7f2da655a700 1 -- 172.21.15.17:0/1934059155 >> 172.21.15.17:0/1934059155 conn(0x7f2da005c620 msgr2=0x7f2da005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:04.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.607+0000 7f2da655a700 1 -- 172.21.15.17:0/1934059155 shutdown_connections 2022-08-01T19:41:04.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.607+0000 7f2da655a700 1 -- 172.21.15.17:0/1934059155 wait complete. 2022-08-01T19:41:04.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.607+0000 7f2da655a700 1 Processor -- start 2022-08-01T19:41:04.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.607+0000 7f2da655a700 1 -- start start 2022-08-01T19:41:04.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.607+0000 7f2da655a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2da00b4310 0x7f2da00aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:04.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.607+0000 7f2da655a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2da00b4820 0x7f2da00ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:04.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.607+0000 7f2da655a700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2da0146ae0 0x7f2da01455e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:04.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.607+0000 7f2da655a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da0155b50 con 0x7f2da00b4820 2022-08-01T19:41:04.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.607+0000 7f2da655a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da0155df0 con 0x7f2da00b4310 2022-08-01T19:41:04.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.607+0000 7f2da655a700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da0156090 con 0x7f2da0146ae0 2022-08-01T19:41:04.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.607+0000 7f2d9f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2da00b4820 0x7f2da00ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:04.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.607+0000 7f2d9f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2da00b4820 0x7f2da00ad8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41434/0 (socket says 172.21.15.17:41434) 2022-08-01T19:41:04.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.607+0000 7f2d9f7fe700 1 -- 172.21.15.17:0/3311322060 learned_addr learned my addr 172.21.15.17:0/3311322060 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:04.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.607+0000 7f2d9ffff700 1 --2- 172.21.15.17:0/3311322060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2da00b4310 0x7f2da00aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:04.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.607+0000 7f2da4af7700 1 --2- 172.21.15.17:0/3311322060 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2da0146ae0 0x7f2da01455e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:04.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.607+0000 7f2d9f7fe700 1 -- 172.21.15.17:0/3311322060 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2da0146ae0 msgr2=0x7f2da01455e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:04.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.607+0000 7f2d9f7fe700 1 --2- 172.21.15.17:0/3311322060 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2da0146ae0 0x7f2da01455e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:04.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.607+0000 7f2d9f7fe700 1 -- 172.21.15.17:0/3311322060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2da00b4310 msgr2=0x7f2da00aeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:04.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.607+0000 7f2d9f7fe700 1 --2- 172.21.15.17:0/3311322060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2da00b4310 0x7f2da00aeae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:04.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.607+0000 7f2d9f7fe700 1 -- 172.21.15.17:0/3311322060 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d98009470 con 0x7f2da00b4820 2022-08-01T19:41:04.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.611+0000 7f2d9f7fe700 1 --2- 172.21.15.17:0/3311322060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2da00b4820 0x7f2da00ad8a0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0x7f2d94001110 tx=0x7f2d94007a60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:04.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.611+0000 7f2d9d7fa700 1 -- 172.21.15.17:0/3311322060 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d940070d0 con 0x7f2da00b4820 2022-08-01T19:41:04.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.611+0000 7f2d9d7fa700 1 -- 172.21.15.17:0/3311322060 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d94007230 con 0x7f2da00b4820 2022-08-01T19:41:04.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.611+0000 7f2da655a700 1 -- 172.21.15.17:0/3311322060 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da0156330 con 0x7f2da00b4820 2022-08-01T19:41:04.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.611+0000 7f2d9d7fa700 1 -- 172.21.15.17:0/3311322060 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d94005610 con 0x7f2da00b4820 2022-08-01T19:41:04.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.611+0000 7f2da655a700 1 -- 172.21.15.17:0/3311322060 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da00b8d60 con 0x7f2da00b4820 2022-08-01T19:41:04.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.611+0000 7f2da655a700 1 -- 172.21.15.17:0/3311322060 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2da00adf30 con 0x7f2da00b4820 2022-08-01T19:41:04.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.611+0000 7f2d9d7fa700 1 -- 172.21.15.17:0/3311322060 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2d9401a070 con 0x7f2da00b4820 2022-08-01T19:41:04.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.611+0000 7f2d9d7fa700 1 --2- 172.21.15.17:0/3311322060 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2d80053a70 0x7f2d80055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:04.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.611+0000 7f2d9d7fa700 1 -- 172.21.15.17:0/3311322060 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f2d9406b3f0 con 0x7f2da00b4820 2022-08-01T19:41:04.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.611+0000 7f2d9ffff700 1 --2- 172.21.15.17:0/3311322060 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2d80053a70 0x7f2d80055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:04.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.615+0000 7f2d9ffff700 1 --2- 172.21.15.17:0/3311322060 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2d80053a70 0x7f2d80055f30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f2d9000b000 tx=0x7f2d90008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:04.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.619+0000 7f2d9d7fa700 1 -- 172.21.15.17:0/3311322060 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2d940420a0 con 0x7f2da00b4820 2022-08-01T19:41:04.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.743+0000 7f2da655a700 1 -- 172.21.15.17:0/3311322060 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f2da01564c0 con 0x7f2da00b4820 2022-08-01T19:41:04.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.743+0000 7f2d9d7fa700 1 -- 172.21.15.17:0/3311322060 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v88) v1 ==== 110+0+10 (secure 0 0 0) 0x7f2d940456c0 con 0x7f2da00b4820 2022-08-01T19:41:04.750 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:41:04.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.743+0000 7f2da655a700 1 -- 172.21.15.17:0/3311322060 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2d80053a70 msgr2=0x7f2d80055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:04.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.743+0000 7f2da655a700 1 --2- 172.21.15.17:0/3311322060 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2d80053a70 0x7f2d80055f30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f2d9000b000 tx=0x7f2d90008040).stop 2022-08-01T19:41:04.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.743+0000 7f2da655a700 1 -- 172.21.15.17:0/3311322060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2da00b4820 msgr2=0x7f2da00ad8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:04.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.743+0000 7f2da655a700 1 --2- 172.21.15.17:0/3311322060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2da00b4820 0x7f2da00ad8a0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0x7f2d94001110 tx=0x7f2d94007a60).stop 2022-08-01T19:41:04.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.743+0000 7f2da655a700 1 -- 172.21.15.17:0/3311322060 shutdown_connections 2022-08-01T19:41:04.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.743+0000 7f2da655a700 1 --2- 172.21.15.17:0/3311322060 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2d80053a70 0x7f2d80055f30 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:04.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.743+0000 7f2da655a700 1 --2- 172.21.15.17:0/3311322060 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2da0146ae0 0x7f2da01455e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:04.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.743+0000 7f2da655a700 1 --2- 172.21.15.17:0/3311322060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2da00b4820 0x7f2da00ad8a0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:04.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.743+0000 7f2da655a700 1 --2- 172.21.15.17:0/3311322060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2da00b4310 0x7f2da00aeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:04.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.743+0000 7f2da655a700 1 -- 172.21.15.17:0/3311322060 >> 172.21.15.17:0/3311322060 conn(0x7f2da005c620 msgr2=0x7f2da00b6810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:04.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.743+0000 7f2da655a700 1 -- 172.21.15.17:0/3311322060 shutdown_connections 2022-08-01T19:41:04.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.743+0000 7f2da655a700 1 -- 172.21.15.17:0/3311322060 wait complete. 2022-08-01T19:41:04.769 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:41:04.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.875+0000 7f5b335b9700 1 Processor -- start 2022-08-01T19:41:04.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.875+0000 7f5b335b9700 1 -- start start 2022-08-01T19:41:04.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.875+0000 7f5b335b9700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5b2c0a9af0 0x7f5b2c0b1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:04.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.875+0000 7f5b335b9700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5b2c0b0850 0x7f5b2c0ab130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:04.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.875+0000 7f5b335b9700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b2c0b1df0 con 0x7f5b2c0ac3f0 2022-08-01T19:41:04.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.875+0000 7f5b335b9700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b2c0a9ed0 con 0x7f5b2c0a9af0 2022-08-01T19:41:04.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.875+0000 7f5b335b9700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c0b2ad0 con 0x7f5b2c0b0850 2022-08-01T19:41:04.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.879+0000 7f5b30b54700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5b2c0ac3f0 0x7f5b2c0aedf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33636/0 (socket says 172.21.15.17:33636) 2022-08-01T19:41:04.899 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.879+0000 7f5b30b54700 1 -- 172.21.15.17:0/2401554963 learned_addr learned my addr 172.21.15.17:0/2401554963 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:04.899 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.879+0000 7f5b31355700 1 --2- 172.21.15.17:0/2401554963 >> v2:172.21.15.17:3300/0 conn(0x7f5b2c0a9af0 0x7f5b2c0b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:04.899 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.879+0000 7f5b31b56700 1 --2- 172.21.15.17:0/2401554963 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5b2c0b0850 0x7f5b2c0ab130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:04.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.879+0000 7f5b23fff700 1 -- 172.21.15.17:0/2401554963 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 630081455 0 0) 0x7f5b28001110 con 0x7f5b2c0ac3f0 2022-08-01T19:41:04.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.879+0000 7f5b23fff700 1 -- 172.21.15.17:0/2401554963 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b10003260 con 0x7f5b2c0ac3f0 2022-08-01T19:41:04.901 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.879+0000 7f5b31355700 1 -- 172.21.15.17:0/2401554963 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5b2c0b0850 msgr2=0x7f5b2c0ab130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:04.901 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.879+0000 7f5b31355700 1 --2- 172.21.15.17:0/2401554963 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5b2c0b0850 0x7f5b2c0ab130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:04.901 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.879+0000 7f5b31355700 1 -- 172.21.15.17:0/2401554963 >> v1:172.21.15.17:6789/0 conn(0x7f5b2c0ac3f0 legacy=0x7f5b2c0aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:04.902 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.879+0000 7f5b31355700 1 -- 172.21.15.17:0/2401554963 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b2c0b0250 con 0x7f5b2c0a9af0 2022-08-01T19:41:04.902 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.879+0000 7f5b31355700 1 --2- 172.21.15.17:0/2401554963 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b2c0a9af0 0x7f5b2c0b1900 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7f5b1c002f70 tx=0x7f5b1c006df0).ready entity=mon.0 client_cookie=e25fa0d35ef43054 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:04.903 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.879+0000 7f5b23fff700 1 -- 172.21.15.17:0/2401554963 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b1c00d4a0 con 0x7f5b2c0a9af0 2022-08-01T19:41:04.903 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.879+0000 7f5b23fff700 1 -- 172.21.15.17:0/2401554963 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1c000e90 con 0x7f5b2c0a9af0 2022-08-01T19:41:04.904 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.879+0000 7f5b23fff700 1 -- 172.21.15.17:0/2401554963 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b1c00d7f0 con 0x7f5b2c0a9af0 2022-08-01T19:41:04.904 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.879+0000 7f5b335b9700 1 -- 172.21.15.17:0/2401554963 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b2c0a9af0 msgr2=0x7f5b2c0b1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:04.905 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.879+0000 7f5b335b9700 1 --2- 172.21.15.17:0/2401554963 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b2c0a9af0 0x7f5b2c0b1900 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7f5b1c002f70 tx=0x7f5b1c006df0).stop 2022-08-01T19:41:04.905 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.879+0000 7f5b335b9700 1 -- 172.21.15.17:0/2401554963 shutdown_connections 2022-08-01T19:41:04.905 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.879+0000 7f5b335b9700 1 --2- 172.21.15.17:0/2401554963 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5b2c0b0850 0x7f5b2c0ab130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:04.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.879+0000 7f5b335b9700 1 --2- 172.21.15.17:0/2401554963 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b2c0a9af0 0x7f5b2c0b1900 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:04.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.879+0000 7f5b335b9700 1 -- 172.21.15.17:0/2401554963 >> 172.21.15.17:0/2401554963 conn(0x7f5b2c05f840 msgr2=0x7f5b2c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:04.907 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.879+0000 7f5b335b9700 1 -- 172.21.15.17:0/2401554963 shutdown_connections 2022-08-01T19:41:04.907 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.879+0000 7f5b335b9700 1 -- 172.21.15.17:0/2401554963 wait complete. 2022-08-01T19:41:04.909 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.883+0000 7f5b335b9700 1 Processor -- start 2022-08-01T19:41:04.909 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.883+0000 7f5b335b9700 1 -- start start 2022-08-01T19:41:04.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.883+0000 7f5b335b9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b2c0a9af0 0x7f5b2c1449f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:04.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.883+0000 7f5b335b9700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5b2c0ac3f0 0x7f5b2c13c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:04.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.883+0000 7f5b335b9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b2c0b0850 0x7f5b2c13ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:04.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.883+0000 7f5b335b9700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c064300 con 0x7f5b2c0a9af0 2022-08-01T19:41:04.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.883+0000 7f5b335b9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c13d320 con 0x7f5b2c0b0850 2022-08-01T19:41:04.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.883+0000 7f5b335b9700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c064530 con 0x7f5b2c0ac3f0 2022-08-01T19:41:04.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.883+0000 7f5b31355700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b2c0a9af0 0x7f5b2c1449f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:04.924 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.883+0000 7f5b31355700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b2c0a9af0 0x7f5b2c1449f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41448/0 (socket says 172.21.15.17:41448) 2022-08-01T19:41:04.924 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.883+0000 7f5b31355700 1 -- 172.21.15.17:0/4154072719 learned_addr learned my addr 172.21.15.17:0/4154072719 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:04.925 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.883+0000 7f5b30b54700 1 --2- 172.21.15.17:0/4154072719 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5b2c0ac3f0 0x7f5b2c13c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:04.925 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.883+0000 7f5b31b56700 1 --2- 172.21.15.17:0/4154072719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b2c0b0850 0x7f5b2c13ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:04.925 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.883+0000 7f5b31355700 1 -- 172.21.15.17:0/4154072719 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5b2c0ac3f0 msgr2=0x7f5b2c13c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:04.927 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.883+0000 7f5b31355700 1 --2- 172.21.15.17:0/4154072719 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5b2c0ac3f0 0x7f5b2c13c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:04.927 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.883+0000 7f5b31355700 1 -- 172.21.15.17:0/4154072719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b2c0b0850 msgr2=0x7f5b2c13ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:04.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.883+0000 7f5b31355700 1 --2- 172.21.15.17:0/4154072719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b2c0b0850 0x7f5b2c13ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:04.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.883+0000 7f5b31355700 1 -- 172.21.15.17:0/4154072719 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b1c00b180 con 0x7f5b2c0a9af0 2022-08-01T19:41:04.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.883+0000 7f5b31355700 1 --2- 172.21.15.17:0/4154072719 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b2c0a9af0 0x7f5b2c1449f0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 rx=0x7f5b1c002f70 tx=0x7f5b1c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:04.929 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.883+0000 7f5b227fc700 1 -- 172.21.15.17:0/4154072719 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b1c0139c0 con 0x7f5b2c0a9af0 2022-08-01T19:41:04.929 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.883+0000 7f5b227fc700 1 -- 172.21.15.17:0/4154072719 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1c013b20 con 0x7f5b2c0a9af0 2022-08-01T19:41:04.930 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.883+0000 7f5b335b9700 1 -- 172.21.15.17:0/4154072719 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b2c150f30 con 0x7f5b2c0a9af0 2022-08-01T19:41:04.931 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.883+0000 7f5b227fc700 1 -- 172.21.15.17:0/4154072719 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b1c013e80 con 0x7f5b2c0a9af0 2022-08-01T19:41:04.931 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.883+0000 7f5b335b9700 1 -- 172.21.15.17:0/4154072719 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b2c150c20 con 0x7f5b2c0a9af0 2022-08-01T19:41:04.931 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.887+0000 7f5b335b9700 1 -- 172.21.15.17:0/4154072719 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b2c062950 con 0x7f5b2c0a9af0 2022-08-01T19:41:04.932 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.887+0000 7f5b227fc700 1 -- 172.21.15.17:0/4154072719 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5b1c025030 con 0x7f5b2c0a9af0 2022-08-01T19:41:04.932 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.887+0000 7f5b227fc700 1 --2- 172.21.15.17:0/4154072719 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5b14053a70 0x7f5b14055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:04.933 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.887+0000 7f5b227fc700 1 -- 172.21.15.17:0/4154072719 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5b1c06cd50 con 0x7f5b2c0a9af0 2022-08-01T19:41:04.933 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.887+0000 7f5b30b54700 1 --2- 172.21.15.17:0/4154072719 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5b14053a70 0x7f5b14055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:04.934 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.887+0000 7f5b30b54700 1 --2- 172.21.15.17:0/4154072719 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5b14053a70 0x7f5b14055f30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f5b2800d000 tx=0x7f5b280078a0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:04.938 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:04.895+0000 7f5b227fc700 1 -- 172.21.15.17:0/4154072719 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5b1c043a00 con 0x7f5b2c0a9af0 2022-08-01T19:41:05.026 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.023+0000 7f5b335b9700 1 -- 172.21.15.17:0/4154072719 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5b2c0574e0 con 0x7f5b2c0a9af0 2022-08-01T19:41:05.027 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.023+0000 7f5b227fc700 1 -- 172.21.15.17:0/4154072719 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v88) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5b1c023070 con 0x7f5b2c0a9af0 2022-08-01T19:41:05.027 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:41:05.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.023+0000 7f5b335b9700 1 -- 172.21.15.17:0/4154072719 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5b14053a70 msgr2=0x7f5b14055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:05.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.023+0000 7f5b335b9700 1 --2- 172.21.15.17:0/4154072719 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5b14053a70 0x7f5b14055f30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f5b2800d000 tx=0x7f5b280078a0).stop 2022-08-01T19:41:05.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.023+0000 7f5b335b9700 1 -- 172.21.15.17:0/4154072719 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b2c0a9af0 msgr2=0x7f5b2c1449f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:05.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.023+0000 7f5b335b9700 1 --2- 172.21.15.17:0/4154072719 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b2c0a9af0 0x7f5b2c1449f0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 rx=0x7f5b1c002f70 tx=0x7f5b1c014000).stop 2022-08-01T19:41:05.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.023+0000 7f5b335b9700 1 -- 172.21.15.17:0/4154072719 shutdown_connections 2022-08-01T19:41:05.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.023+0000 7f5b335b9700 1 --2- 172.21.15.17:0/4154072719 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5b14053a70 0x7f5b14055f30 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:05.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.023+0000 7f5b335b9700 1 --2- 172.21.15.17:0/4154072719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b2c0b0850 0x7f5b2c13ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:05.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.023+0000 7f5b335b9700 1 --2- 172.21.15.17:0/4154072719 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5b2c0ac3f0 0x7f5b2c13c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:05.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.023+0000 7f5b335b9700 1 --2- 172.21.15.17:0/4154072719 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b2c0a9af0 0x7f5b2c1449f0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:05.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.023+0000 7f5b335b9700 1 -- 172.21.15.17:0/4154072719 >> 172.21.15.17:0/4154072719 conn(0x7f5b2c05f840 msgr2=0x7f5b2c13dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:05.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.023+0000 7f5b335b9700 1 -- 172.21.15.17:0/4154072719 shutdown_connections 2022-08-01T19:41:05.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.023+0000 7f5b335b9700 1 -- 172.21.15.17:0/4154072719 wait complete. 2022-08-01T19:41:05.052 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T19:41:05.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.163+0000 7f1aeb59e700 1 Processor -- start 2022-08-01T19:41:05.171 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.163+0000 7f1aeb59e700 1 -- start start 2022-08-01T19:41:05.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.163+0000 7f1aeb59e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f1aec0b2710 0x7f1aec14afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:05.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.163+0000 7f1aeb59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1aec0b01a0 0x7f1aec14b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:05.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.163+0000 7f1aeb59e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aec0a7250 con 0x7f1aec0afd60 2022-08-01T19:41:05.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.163+0000 7f1aeb59e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aec0a7480 con 0x7f1aec0b2710 2022-08-01T19:41:05.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.163+0000 7f1aeb59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec0a7720 con 0x7f1aec0b01a0 2022-08-01T19:41:05.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.163+0000 7f1ae9d9b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1aec0b01a0 0x7f1aec14b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:05.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.163+0000 7f1aea59c700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f1aec0b2710 0x7f1aec14afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:05.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.163+0000 7f1ae9d9b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1aec0b01a0 0x7f1aec14b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:37558/0 (socket says 172.21.15.17:37558) 2022-08-01T19:41:05.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.163+0000 7f1ae9d9b700 1 -- 172.21.15.17:0/1289080820 learned_addr learned my addr 172.21.15.17:0/1289080820 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:05.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.163+0000 7f1ae9d9b700 1 -- 172.21.15.17:0/1289080820 >> v2:172.21.15.77:3300/0 conn(0x7f1aec0b2710 msgr2=0x7f1aec14afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:05.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.163+0000 7f1ae9d9b700 1 --2- 172.21.15.17:0/1289080820 >> v2:172.21.15.77:3300/0 conn(0x7f1aec0b2710 0x7f1aec14afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:05.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.163+0000 7f1ae9d9b700 1 -- 172.21.15.17:0/1289080820 >> v1:172.21.15.77:6789/0 conn(0x7f1aec0afd60 legacy=0x7f1aec0a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:05.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.163+0000 7f1ae9d9b700 1 -- 172.21.15.17:0/1289080820 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aec0a79c0 con 0x7f1aec0b01a0 2022-08-01T19:41:05.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.163+0000 7f1ae9d9b700 1 --2- 172.21.15.17:0/1289080820 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1aec0b01a0 0x7f1aec14b4c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f1ae40032f0 tx=0x7f1ae4009290).ready entity=mon.2 client_cookie=efaef338cf0c4178 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:05.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.163+0000 7f1ae959a700 1 -- 172.21.15.17:0/1289080820 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ae400b8a0 con 0x7f1aec0b01a0 2022-08-01T19:41:05.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.163+0000 7f1ae959a700 1 -- 172.21.15.17:0/1289080820 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae400ba00 con 0x7f1aec0b01a0 2022-08-01T19:41:05.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.163+0000 7f1ae959a700 1 -- 172.21.15.17:0/1289080820 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ae400bb70 con 0x7f1aec0b01a0 2022-08-01T19:41:05.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.167+0000 7f1aeb59e700 1 -- 172.21.15.17:0/1289080820 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1aec0b01a0 msgr2=0x7f1aec14b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:05.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.167+0000 7f1aeb59e700 1 --2- 172.21.15.17:0/1289080820 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1aec0b01a0 0x7f1aec14b4c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f1ae40032f0 tx=0x7f1ae4009290).stop 2022-08-01T19:41:05.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.167+0000 7f1aeb59e700 1 -- 172.21.15.17:0/1289080820 shutdown_connections 2022-08-01T19:41:05.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.167+0000 7f1aeb59e700 1 --2- 172.21.15.17:0/1289080820 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1aec0b01a0 0x7f1aec14b4c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:05.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.167+0000 7f1aeb59e700 1 --2- 172.21.15.17:0/1289080820 >> v2:172.21.15.77:3300/0 conn(0x7f1aec0b2710 0x7f1aec14afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:05.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.167+0000 7f1aeb59e700 1 -- 172.21.15.17:0/1289080820 >> 172.21.15.17:0/1289080820 conn(0x7f1aec060310 msgr2=0x7f1aec0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:05.188 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.167+0000 7f1aeb59e700 1 -- 172.21.15.17:0/1289080820 shutdown_connections 2022-08-01T19:41:05.188 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.167+0000 7f1aeb59e700 1 -- 172.21.15.17:0/1289080820 wait complete. 2022-08-01T19:41:05.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.167+0000 7f1aeb59e700 1 Processor -- start 2022-08-01T19:41:05.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.167+0000 7f1aeb59e700 1 -- start start 2022-08-01T19:41:05.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.167+0000 7f1aeb59e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1aec0afd60 0x7f1aec149d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:05.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.167+0000 7f1aeb59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1aec0b01a0 0x7f1aec0abfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:05.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.167+0000 7f1aeb59e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1aec0b2710 0x7f1aec0ac4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:05.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.167+0000 7f1aeb59e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec14c520 con 0x7f1aec0afd60 2022-08-01T19:41:05.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.167+0000 7f1aeb59e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec063540 con 0x7f1aec0b2710 2022-08-01T19:41:05.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.167+0000 7f1aeb59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec0ab3a0 con 0x7f1aec0b01a0 2022-08-01T19:41:05.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.167+0000 7f1aea59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1aec0afd60 0x7f1aec149d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:05.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.167+0000 7f1aea59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1aec0afd60 0x7f1aec149d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41462/0 (socket says 172.21.15.17:41462) 2022-08-01T19:41:05.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.171+0000 7f1aea59c700 1 -- 172.21.15.17:0/3384288869 learned_addr learned my addr 172.21.15.17:0/3384288869 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:05.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.171+0000 7f1ae9d9b700 1 --2- 172.21.15.17:0/3384288869 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1aec0b01a0 0x7f1aec0abfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:05.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.171+0000 7f1aead9d700 1 --2- 172.21.15.17:0/3384288869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1aec0b2710 0x7f1aec0ac4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:05.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.171+0000 7f1aea59c700 1 -- 172.21.15.17:0/3384288869 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1aec0b01a0 msgr2=0x7f1aec0abfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:05.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.171+0000 7f1aea59c700 1 --2- 172.21.15.17:0/3384288869 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1aec0b01a0 0x7f1aec0abfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:05.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.171+0000 7f1aea59c700 1 -- 172.21.15.17:0/3384288869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1aec0b2710 msgr2=0x7f1aec0ac4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:05.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.171+0000 7f1aea59c700 1 --2- 172.21.15.17:0/3384288869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1aec0b2710 0x7f1aec0ac4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:05.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.171+0000 7f1aea59c700 1 -- 172.21.15.17:0/3384288869 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae4009530 con 0x7f1aec0afd60 2022-08-01T19:41:05.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.171+0000 7f1aead9d700 1 --2- 172.21.15.17:0/3384288869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1aec0b2710 0x7f1aec0ac4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:05.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.171+0000 7f1ae9d9b700 1 --2- 172.21.15.17:0/3384288869 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1aec0b01a0 0x7f1aec0abfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:05.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.171+0000 7f1aea59c700 1 --2- 172.21.15.17:0/3384288869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1aec0afd60 0x7f1aec149d50 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 rx=0x7f1aec064310 tx=0x7f1ae0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:05.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.171+0000 7f1adb7fe700 1 -- 172.21.15.17:0/3384288869 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ae0010780 con 0x7f1aec0afd60 2022-08-01T19:41:05.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.171+0000 7f1adb7fe700 1 -- 172.21.15.17:0/3384288869 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae00108e0 con 0x7f1aec0afd60 2022-08-01T19:41:05.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.171+0000 7f1aeb59e700 1 -- 172.21.15.17:0/3384288869 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aec159c50 con 0x7f1aec0afd60 2022-08-01T19:41:05.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.171+0000 7f1adb7fe700 1 -- 172.21.15.17:0/3384288869 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ae0010c40 con 0x7f1aec0afd60 2022-08-01T19:41:05.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.171+0000 7f1aeb59e700 1 -- 172.21.15.17:0/3384288869 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aec065380 con 0x7f1aec0afd60 2022-08-01T19:41:05.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.171+0000 7f1adb7fe700 1 -- 172.21.15.17:0/3384288869 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1ae001c460 con 0x7f1aec0afd60 2022-08-01T19:41:05.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.171+0000 7f1aeb59e700 1 -- 172.21.15.17:0/3384288869 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aec0aa8a0 con 0x7f1aec0afd60 2022-08-01T19:41:05.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.171+0000 7f1adb7fe700 1 --2- 172.21.15.17:0/3384288869 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1ac8053a70 0x7f1ac8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:05.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.175+0000 7f1adb7fe700 1 -- 172.21.15.17:0/3384288869 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1ae006c510 con 0x7f1aec0afd60 2022-08-01T19:41:05.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.175+0000 7f1ae9d9b700 1 --2- 172.21.15.17:0/3384288869 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1ac8053a70 0x7f1ac8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:05.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.175+0000 7f1ae9d9b700 1 --2- 172.21.15.17:0/3384288869 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1ac8053a70 0x7f1ac8055f30 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f1ae4019000 tx=0x7f1ae4016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:05.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.179+0000 7f1adb7fe700 1 -- 172.21.15.17:0/3384288869 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1ae00431d0 con 0x7f1aec0afd60 2022-08-01T19:41:05.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.307+0000 7f1aeb59e700 1 -- 172.21.15.17:0/3384288869 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1aec0574e0 con 0x7f1aec0afd60 2022-08-01T19:41:05.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.307+0000 7f1adb7fe700 1 -- 172.21.15.17:0/3384288869 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+10 (secure 0 0 0) 0x7f1ae00467f0 con 0x7f1aec0afd60 2022-08-01T19:41:05.314 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:41:05.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.307+0000 7f1aeb59e700 1 -- 172.21.15.17:0/3384288869 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1ac8053a70 msgr2=0x7f1ac8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:05.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.307+0000 7f1aeb59e700 1 --2- 172.21.15.17:0/3384288869 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1ac8053a70 0x7f1ac8055f30 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f1ae4019000 tx=0x7f1ae4016000).stop 2022-08-01T19:41:05.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.307+0000 7f1aeb59e700 1 -- 172.21.15.17:0/3384288869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1aec0afd60 msgr2=0x7f1aec149d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:05.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.307+0000 7f1aeb59e700 1 --2- 172.21.15.17:0/3384288869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1aec0afd60 0x7f1aec149d50 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 rx=0x7f1aec064310 tx=0x7f1ae0006df0).stop 2022-08-01T19:41:05.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.307+0000 7f1aeb59e700 1 -- 172.21.15.17:0/3384288869 shutdown_connections 2022-08-01T19:41:05.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.307+0000 7f1aeb59e700 1 --2- 172.21.15.17:0/3384288869 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1ac8053a70 0x7f1ac8055f30 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:05.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.307+0000 7f1aeb59e700 1 --2- 172.21.15.17:0/3384288869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1aec0b2710 0x7f1aec0ac4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:05.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.307+0000 7f1aeb59e700 1 --2- 172.21.15.17:0/3384288869 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1aec0b01a0 0x7f1aec0abfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:05.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.307+0000 7f1aeb59e700 1 --2- 172.21.15.17:0/3384288869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1aec0afd60 0x7f1aec149d50 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:05.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.307+0000 7f1aeb59e700 1 -- 172.21.15.17:0/3384288869 >> 172.21.15.17:0/3384288869 conn(0x7f1aec060310 msgr2=0x7f1aec0ad3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:05.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.307+0000 7f1aeb59e700 1 -- 172.21.15.17:0/3384288869 shutdown_connections 2022-08-01T19:41:05.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.307+0000 7f1aeb59e700 1 -- 172.21.15.17:0/3384288869 wait complete. 2022-08-01T19:41:05.325 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T19:41:05.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.431+0000 7fcef8ffa700 1 Processor -- start 2022-08-01T19:41:05.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.431+0000 7fcef8ffa700 1 -- start start 2022-08-01T19:41:05.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.435+0000 7fcef8ffa700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcef4146ae0 0x7fcef41403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:05.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.435+0000 7fcef8ffa700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcef4143cf0 con 0x7fcef40b4310 2022-08-01T19:41:05.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.435+0000 7fcef8ffa700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcef4143f90 con 0x7fcef40b4820 2022-08-01T19:41:05.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.435+0000 7fcef8ffa700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef41476a0 con 0x7fcef4146ae0 2022-08-01T19:41:05.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.435+0000 7fcef259c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcef4146ae0 0x7fcef41403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:05.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.435+0000 7fcef1d9b700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fcef40b4310 0x7fcef41408b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33664/0 (socket says 172.21.15.17:33664) 2022-08-01T19:41:05.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.435+0000 7fcef259c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcef4146ae0 0x7fcef41403c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41472/0 (socket says 172.21.15.17:41472) 2022-08-01T19:41:05.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.435+0000 7fcef1d9b700 1 -- 172.21.15.17:0/3834885055 learned_addr learned my addr 172.21.15.17:0/3834885055 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:05.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.435+0000 7fcef159a700 1 -- 172.21.15.17:0/3834885055 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2808376294 0 0) 0x7fcef4143cf0 con 0x7fcef40b4310 2022-08-01T19:41:05.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.435+0000 7fcef259c700 1 -- 172.21.15.17:0/3834885055 >> v1:172.21.15.77:6789/0 conn(0x7fcef40b4820 legacy=0x7fcef40b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:05.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.435+0000 7fcef259c700 1 -- 172.21.15.17:0/3834885055 >> v1:172.21.15.17:6789/0 conn(0x7fcef40b4310 legacy=0x7fcef41408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:05.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.435+0000 7fcef259c700 1 -- 172.21.15.17:0/3834885055 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef40b0e20 con 0x7fcef4146ae0 2022-08-01T19:41:05.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.435+0000 7fcef259c700 1 --2- 172.21.15.17:0/3834885055 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcef4146ae0 0x7fcef41403c0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 rx=0x7fcee4002f70 tx=0x7fcee4006df0).ready entity=mon.0 client_cookie=94ed9bb823608f7e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:05.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.435+0000 7fcef159a700 1 -- 172.21.15.17:0/3834885055 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcee400d610 con 0x7fcef4146ae0 2022-08-01T19:41:05.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.435+0000 7fcef159a700 1 -- 172.21.15.17:0/3834885055 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee4000e90 con 0x7fcef4146ae0 2022-08-01T19:41:05.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.435+0000 7fcef159a700 1 -- 172.21.15.17:0/3834885055 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcee400d960 con 0x7fcef4146ae0 2022-08-01T19:41:05.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.435+0000 7fcef8ffa700 1 -- 172.21.15.17:0/3834885055 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcef4146ae0 msgr2=0x7fcef41403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:05.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.435+0000 7fcef8ffa700 1 --2- 172.21.15.17:0/3834885055 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcef4146ae0 0x7fcef41403c0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 rx=0x7fcee4002f70 tx=0x7fcee4006df0).stop 2022-08-01T19:41:05.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.435+0000 7fcef8ffa700 1 -- 172.21.15.17:0/3834885055 shutdown_connections 2022-08-01T19:41:05.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.435+0000 7fcef8ffa700 1 --2- 172.21.15.17:0/3834885055 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcef4146ae0 0x7fcef41403c0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:05.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.435+0000 7fcef8ffa700 1 -- 172.21.15.17:0/3834885055 >> 172.21.15.17:0/3834885055 conn(0x7fcef405c620 msgr2=0x7fcef405ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:05.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.435+0000 7fcef8ffa700 1 -- 172.21.15.17:0/3834885055 shutdown_connections 2022-08-01T19:41:05.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.435+0000 7fcef8ffa700 1 -- 172.21.15.17:0/3834885055 wait complete. 2022-08-01T19:41:05.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.439+0000 7fcef8ffa700 1 Processor -- start 2022-08-01T19:41:05.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.439+0000 7fcef8ffa700 1 -- start start 2022-08-01T19:41:05.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.439+0000 7fcef8ffa700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcef40b4310 0x7fcef41560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:05.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.439+0000 7fcef8ffa700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcef40b4820 0x7fcef40b9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:05.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.439+0000 7fcef8ffa700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcef40ba300 0x7fcef413a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:05.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.439+0000 7fcef8ffa700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef40b8b30 con 0x7fcef40b4310 2022-08-01T19:41:05.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.439+0000 7fcef8ffa700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef41565e0 con 0x7fcef40ba300 2022-08-01T19:41:05.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.439+0000 7fcef8ffa700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef4156810 con 0x7fcef40b4820 2022-08-01T19:41:05.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.439+0000 7fcef259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcef40b4310 0x7fcef41560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:05.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.439+0000 7fcef259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcef40b4310 0x7fcef41560f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41478/0 (socket says 172.21.15.17:41478) 2022-08-01T19:41:05.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.439+0000 7fcef259c700 1 -- 172.21.15.17:0/1067146593 learned_addr learned my addr 172.21.15.17:0/1067146593 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:05.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.439+0000 7fcef1d9b700 1 --2- 172.21.15.17:0/1067146593 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcef40b4820 0x7fcef40b9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:05.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.439+0000 7fcef2d9d700 1 --2- 172.21.15.17:0/1067146593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcef40ba300 0x7fcef413a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:05.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.439+0000 7fcef259c700 1 -- 172.21.15.17:0/1067146593 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcef40b4820 msgr2=0x7fcef40b9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:05.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.439+0000 7fcef259c700 1 --2- 172.21.15.17:0/1067146593 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcef40b4820 0x7fcef40b9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:05.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.439+0000 7fcef259c700 1 -- 172.21.15.17:0/1067146593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcef40ba300 msgr2=0x7fcef413a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:05.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.439+0000 7fcef259c700 1 --2- 172.21.15.17:0/1067146593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcef40ba300 0x7fcef413a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:05.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.439+0000 7fcef259c700 1 -- 172.21.15.17:0/1067146593 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee400b180 con 0x7fcef40b4310 2022-08-01T19:41:05.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.439+0000 7fcef259c700 1 --2- 172.21.15.17:0/1067146593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcef40b4310 0x7fcef41560f0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7fcef40b8c70 tx=0x7fcee4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:05.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.439+0000 7fcee37fe700 1 -- 172.21.15.17:0/1067146593 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcee4011b40 con 0x7fcef40b4310 2022-08-01T19:41:05.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.439+0000 7fcee37fe700 1 -- 172.21.15.17:0/1067146593 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee4011ca0 con 0x7fcef40b4310 2022-08-01T19:41:05.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.439+0000 7fcef8ffa700 1 -- 172.21.15.17:0/1067146593 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef40b8690 con 0x7fcef40b4310 2022-08-01T19:41:05.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.439+0000 7fcee37fe700 1 -- 172.21.15.17:0/1067146593 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcee401e460 con 0x7fcef40b4310 2022-08-01T19:41:05.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.439+0000 7fcef8ffa700 1 -- 172.21.15.17:0/1067146593 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef4150340 con 0x7fcef40b4310 2022-08-01T19:41:05.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.443+0000 7fcef8ffa700 1 -- 172.21.15.17:0/1067146593 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef4156a10 con 0x7fcef40b4310 2022-08-01T19:41:05.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.443+0000 7fcee37fe700 1 -- 172.21.15.17:0/1067146593 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcee4025030 con 0x7fcef40b4310 2022-08-01T19:41:05.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.443+0000 7fcee37fe700 1 --2- 172.21.15.17:0/1067146593 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fced0053a70 0x7fced0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:05.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.443+0000 7fcee37fe700 1 -- 172.21.15.17:0/1067146593 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fcee406d210 con 0x7fcef40b4310 2022-08-01T19:41:05.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.443+0000 7fcef1d9b700 1 --2- 172.21.15.17:0/1067146593 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fced0053a70 0x7fced0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:05.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.443+0000 7fcef1d9b700 1 --2- 172.21.15.17:0/1067146593 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fced0053a70 0x7fced0055f30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7fcee800d000 tx=0x7fcee80011a0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:05.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.447+0000 7fcee37fe700 1 -- 172.21.15.17:0/1067146593 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcee4044010 con 0x7fcef40b4310 2022-08-01T19:41:05.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.571+0000 7fcef8ffa700 1 -- 172.21.15.17:0/1067146593 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcef40574e0 con 0x7fcef40b4310 2022-08-01T19:41:05.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.571+0000 7fcee37fe700 1 -- 172.21.15.17:0/1067146593 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+10 (secure 0 0 0) 0x7fcee40230a0 con 0x7fcef40b4310 2022-08-01T19:41:05.580 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:41:05.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.575+0000 7fcef8ffa700 1 -- 172.21.15.17:0/1067146593 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fced0053a70 msgr2=0x7fced0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:05.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.575+0000 7fcef8ffa700 1 --2- 172.21.15.17:0/1067146593 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fced0053a70 0x7fced0055f30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7fcee800d000 tx=0x7fcee80011a0).stop 2022-08-01T19:41:05.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.575+0000 7fcef8ffa700 1 -- 172.21.15.17:0/1067146593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcef40b4310 msgr2=0x7fcef41560f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:05.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.575+0000 7fcef8ffa700 1 --2- 172.21.15.17:0/1067146593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcef40b4310 0x7fcef41560f0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7fcef40b8c70 tx=0x7fcee4015000).stop 2022-08-01T19:41:05.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.575+0000 7fcef8ffa700 1 -- 172.21.15.17:0/1067146593 shutdown_connections 2022-08-01T19:41:05.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.575+0000 7fcef8ffa700 1 --2- 172.21.15.17:0/1067146593 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fced0053a70 0x7fced0055f30 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:05.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.575+0000 7fcef8ffa700 1 --2- 172.21.15.17:0/1067146593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcef40ba300 0x7fcef413a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:05.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.575+0000 7fcef8ffa700 1 --2- 172.21.15.17:0/1067146593 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcef40b4820 0x7fcef40b9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:05.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.575+0000 7fcef8ffa700 1 --2- 172.21.15.17:0/1067146593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcef40b4310 0x7fcef41560f0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:05.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.575+0000 7fcef8ffa700 1 -- 172.21.15.17:0/1067146593 >> 172.21.15.17:0/1067146593 conn(0x7fcef405c620 msgr2=0x7fcef413f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:05.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.575+0000 7fcef8ffa700 1 -- 172.21.15.17:0/1067146593 shutdown_connections 2022-08-01T19:41:05.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.575+0000 7fcef8ffa700 1 -- 172.21.15.17:0/1067146593 wait complete. 2022-08-01T19:41:05.596 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T19:41:05.596 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:41:05.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.699+0000 7f01713b8700 1 Processor -- start 2022-08-01T19:41:05.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.699+0000 7f01713b8700 1 -- start start 2022-08-01T19:41:05.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.699+0000 7f01713b8700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f016c0b01a0 0x7f016c0a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:05.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.699+0000 7f01713b8700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f016c0afd60 0x7f016c0a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:05.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.699+0000 7f01713b8700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f016c0a7250 con 0x7f016c0b2710 2022-08-01T19:41:05.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.699+0000 7f01713b8700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f016c0a7480 con 0x7f016c0b01a0 2022-08-01T19:41:05.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.699+0000 7f01713b8700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f016c0a7720 con 0x7f016c0afd60 2022-08-01T19:41:05.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.699+0000 7f016b7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f016c0b01a0 0x7f016c0a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:05.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.699+0000 7f016b7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f016c0b01a0 0x7f016c0a8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41484/0 (socket says 172.21.15.17:41484) 2022-08-01T19:41:05.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.699+0000 7f016b7fe700 1 -- 172.21.15.17:0/1433666448 learned_addr learned my addr 172.21.15.17:0/1433666448 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:05.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.699+0000 7f016bfff700 1 --1- 172.21.15.17:0/1433666448 >> v1:172.21.15.17:6789/0 conn(0x7f016c0b2710 0x7f016c14afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33678/0 (socket says 172.21.15.17:33678) 2022-08-01T19:41:05.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.699+0000 7f0170bb7700 1 --2- 172.21.15.17:0/1433666448 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f016c0afd60 0x7f016c0a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:05.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.699+0000 7f016b7fe700 1 -- 172.21.15.17:0/1433666448 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f016c0afd60 msgr2=0x7f016c0a8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:05.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.699+0000 7f016b7fe700 1 --2- 172.21.15.17:0/1433666448 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f016c0afd60 0x7f016c0a8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:05.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.699+0000 7f016b7fe700 1 -- 172.21.15.17:0/1433666448 >> v1:172.21.15.17:6789/0 conn(0x7f016c0b2710 legacy=0x7f016c14afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:05.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.699+0000 7f016b7fe700 1 -- 172.21.15.17:0/1433666448 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f016c0a79c0 con 0x7f016c0b01a0 2022-08-01T19:41:05.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.699+0000 7f0170bb7700 1 --2- 172.21.15.17:0/1433666448 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f016c0afd60 0x7f016c0a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:05.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.699+0000 7f016b7fe700 1 --2- 172.21.15.17:0/1433666448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f016c0b01a0 0x7f016c0a8540 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 rx=0x7f015c0032f0 tx=0x7f015c009290).ready entity=mon.0 client_cookie=7026f4a15a731678 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:05.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.699+0000 7f016affd700 1 -- 172.21.15.17:0/1433666448 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f015c00b8a0 con 0x7f016c0b01a0 2022-08-01T19:41:05.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.699+0000 7f016affd700 1 -- 172.21.15.17:0/1433666448 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f015c00ba00 con 0x7f016c0b01a0 2022-08-01T19:41:05.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.699+0000 7f016affd700 1 -- 172.21.15.17:0/1433666448 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f015c00bb70 con 0x7f016c0b01a0 2022-08-01T19:41:05.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.699+0000 7f01713b8700 1 -- 172.21.15.17:0/1433666448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f016c0b01a0 msgr2=0x7f016c0a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:05.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.699+0000 7f01713b8700 1 --2- 172.21.15.17:0/1433666448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f016c0b01a0 0x7f016c0a8540 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 rx=0x7f015c0032f0 tx=0x7f015c009290).stop 2022-08-01T19:41:05.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.703+0000 7f01713b8700 1 -- 172.21.15.17:0/1433666448 shutdown_connections 2022-08-01T19:41:05.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.703+0000 7f01713b8700 1 --2- 172.21.15.17:0/1433666448 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f016c0afd60 0x7f016c0a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:05.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.703+0000 7f01713b8700 1 --2- 172.21.15.17:0/1433666448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f016c0b01a0 0x7f016c0a8540 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:05.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.703+0000 7f01713b8700 1 -- 172.21.15.17:0/1433666448 >> 172.21.15.17:0/1433666448 conn(0x7f016c060310 msgr2=0x7f016c0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:05.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.703+0000 7f01713b8700 1 -- 172.21.15.17:0/1433666448 shutdown_connections 2022-08-01T19:41:05.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.703+0000 7f01713b8700 1 -- 172.21.15.17:0/1433666448 wait complete. 2022-08-01T19:41:05.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.703+0000 7f01713b8700 1 Processor -- start 2022-08-01T19:41:05.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.703+0000 7f01713b8700 1 -- start start 2022-08-01T19:41:05.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.703+0000 7f01713b8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f016c0afd60 0x7f016c0aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:05.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.703+0000 7f01713b8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f016c0b01a0 0x7f016c0b59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:05.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.703+0000 7f01713b8700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f016c0b2710 0x7f016c0a92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:05.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.703+0000 7f01713b8700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016c0ab800 con 0x7f016c0afd60 2022-08-01T19:41:05.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.703+0000 7f01713b8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016c1571e0 con 0x7f016c0b01a0 2022-08-01T19:41:05.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.703+0000 7f01713b8700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f016c157480 con 0x7f016c0b2710 2022-08-01T19:41:05.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.703+0000 7f016bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f016c0afd60 0x7f016c0aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:05.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.703+0000 7f016bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f016c0afd60 0x7f016c0aeb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41490/0 (socket says 172.21.15.17:41490) 2022-08-01T19:41:05.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.703+0000 7f016bfff700 1 -- 172.21.15.17:0/154643732 learned_addr learned my addr 172.21.15.17:0/154643732 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:05.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.707+0000 7f016b7fe700 1 --2- 172.21.15.17:0/154643732 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f016c0b01a0 0x7f016c0b59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:05.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.707+0000 7f0170bb7700 1 --2- 172.21.15.17:0/154643732 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f016c0b2710 0x7f016c0a92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:05.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.707+0000 7f016bfff700 1 -- 172.21.15.17:0/154643732 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f016c0b2710 msgr2=0x7f016c0a92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:05.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.707+0000 7f016bfff700 1 --2- 172.21.15.17:0/154643732 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f016c0b2710 0x7f016c0a92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:05.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.707+0000 7f016bfff700 1 -- 172.21.15.17:0/154643732 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f016c0b01a0 msgr2=0x7f016c0b59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:05.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.707+0000 7f016bfff700 1 --2- 172.21.15.17:0/154643732 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f016c0b01a0 0x7f016c0b59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:05.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.707+0000 7f016bfff700 1 -- 172.21.15.17:0/154643732 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015c009530 con 0x7f016c0afd60 2022-08-01T19:41:05.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.707+0000 7f0170bb7700 1 --2- 172.21.15.17:0/154643732 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f016c0b2710 0x7f016c0a92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:41:05.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.707+0000 7f016bfff700 1 --2- 172.21.15.17:0/154643732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f016c0afd60 0x7f016c0aeb70 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7f016c06b950 tx=0x7f0160007c60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:05.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.707+0000 7f01697fa700 1 -- 172.21.15.17:0/154643732 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f016000ce30 con 0x7f016c0afd60 2022-08-01T19:41:05.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.707+0000 7f01713b8700 1 -- 172.21.15.17:0/154643732 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f016c0abbc0 con 0x7f016c0afd60 2022-08-01T19:41:05.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.707+0000 7f01697fa700 1 -- 172.21.15.17:0/154643732 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01600072e0 con 0x7f016c0afd60 2022-08-01T19:41:05.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.707+0000 7f01697fa700 1 -- 172.21.15.17:0/154643732 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0160005740 con 0x7f016c0afd60 2022-08-01T19:41:05.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.707+0000 7f01713b8700 1 -- 172.21.15.17:0/154643732 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f016c0b5eb0 con 0x7f016c0afd60 2022-08-01T19:41:05.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.707+0000 7f01713b8700 1 -- 172.21.15.17:0/154643732 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f016c159d80 con 0x7f016c0afd60 2022-08-01T19:41:05.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.707+0000 7f01697fa700 1 -- 172.21.15.17:0/154643732 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0160010070 con 0x7f016c0afd60 2022-08-01T19:41:05.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.707+0000 7f01697fa700 1 --2- 172.21.15.17:0/154643732 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f014c0539e0 0x7f014c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:05.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.707+0000 7f01697fa700 1 -- 172.21.15.17:0/154643732 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f016006b5a0 con 0x7f016c0afd60 2022-08-01T19:41:05.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.711+0000 7f016b7fe700 1 --2- 172.21.15.17:0/154643732 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f014c0539e0 0x7f014c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:05.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.711+0000 7f016b7fe700 1 --2- 172.21.15.17:0/154643732 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f014c0539e0 0x7f014c055ea0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f015c018000 tx=0x7f015c016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:05.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.711+0000 7f01697fa700 1 -- 172.21.15.17:0/154643732 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0160042360 con 0x7f016c0afd60 2022-08-01T19:41:05.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.875+0000 7f01713b8700 1 -- 172.21.15.17:0/154643732 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f016c0574e0 con 0x7f016c0afd60 2022-08-01T19:41:05.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.875+0000 7f01697fa700 1 -- 172.21.15.17:0/154643732 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+2493 (secure 0 0 0) 0x7f0160045980 con 0x7f016c0afd60 2022-08-01T19:41:05.882 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:05.882 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":40,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6615,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3580428616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3580428616},{"type":"v1","addr":"172.21.15.17:6833","nonce":3580428616}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38},{"gid":6627,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/4048623373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":4048623373},{"type":"v1","addr":"172.21.15.17:6835","nonce":4048623373}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:41:01.207552+0000","modified":"2022-08-01T19:41:02.210147+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6630},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6630":{"gid":6630,"name":"b","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6835/4275403969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4275403969},{"type":"v1","addr":"172.21.15.77:6835","nonce":4275403969}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2022-08-01T19:41:05.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.875+0000 7f01713b8700 1 -- 172.21.15.17:0/154643732 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f014c0539e0 msgr2=0x7f014c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:05.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.875+0000 7f01713b8700 1 --2- 172.21.15.17:0/154643732 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f014c0539e0 0x7f014c055ea0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f015c018000 tx=0x7f015c016000).stop 2022-08-01T19:41:05.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.875+0000 7f01713b8700 1 -- 172.21.15.17:0/154643732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f016c0afd60 msgr2=0x7f016c0aeb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:05.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.875+0000 7f01713b8700 1 --2- 172.21.15.17:0/154643732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f016c0afd60 0x7f016c0aeb70 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7f016c06b950 tx=0x7f0160007c60).stop 2022-08-01T19:41:05.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.875+0000 7f01713b8700 1 -- 172.21.15.17:0/154643732 shutdown_connections 2022-08-01T19:41:05.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.875+0000 7f01713b8700 1 --2- 172.21.15.17:0/154643732 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f014c0539e0 0x7f014c055ea0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:05.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.875+0000 7f01713b8700 1 --2- 172.21.15.17:0/154643732 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f016c0b2710 0x7f016c0a92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:05.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.875+0000 7f01713b8700 1 --2- 172.21.15.17:0/154643732 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f016c0b01a0 0x7f016c0b59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:05.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.875+0000 7f01713b8700 1 --2- 172.21.15.17:0/154643732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f016c0afd60 0x7f016c0aeb70 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:05.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.875+0000 7f01713b8700 1 -- 172.21.15.17:0/154643732 >> 172.21.15.17:0/154643732 conn(0x7f016c060310 msgr2=0x7f016c148850 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:05.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.875+0000 7f01713b8700 1 -- 172.21.15.17:0/154643732 shutdown_connections 2022-08-01T19:41:05.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:05.875+0000 7f01713b8700 1 -- 172.21.15.17:0/154643732 wait complete. 2022-08-01T19:41:05.890 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 40 2022-08-01T19:41:06.197 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 40, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:41:01.207552+0000', 'modified': '2022-08-01T19:41:02.210147+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6630}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6630': {'gid': 6630, 'name': 'b', 'rank': 0, 'incarnation': 39, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.77:6835/4275403969', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6834', 'nonce': 4275403969}, {'type': 'v1', 'addr': '172.21.15.77:6835', 'nonce': 4275403969}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:41:06.198 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:41:06.198 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-08-01T19:41:06.317 INFO:teuthology.orchestra.run.smithi077.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":6630,"want_state":"up:active","state":"up:active","rank_uptime":5.0973656099999998,"mdsmap_epoch":40,"osdmap_epoch":88,"osdmap_epoch_barrier":87,"uptime":9.9311397209999992} 2022-08-01T19:41:06.318 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 6630, "mdsmap_epoch": 40, "osdmap_epoch": 88, "osdmap_epoch_barrier": 87, "rank_uptime": 5.09736561, "state": "up:active", "uptime": 9.931139721, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:41:06.319 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T19:41:06.320 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-08-01T19:41:06.320 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi085.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-08-01T19:41:06.321 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:06.322 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:41:06.333 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:41:06.360 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:41:06.361 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:41:06.419 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:41:06.420 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:41:06.421 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:06.422 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:41:06.439 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T19:41:06.442 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:41:06.486 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:41:06.507 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:41:06.508 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:41:06.525 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:41:06.526 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:41:06.526 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:06.527 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:41:06.566 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:2022-08-01T19:41:06.556+0000 7f819fafe080 -1 init, newargv = 0x5617d43f89a0 newargc=9ceph-fuse[23193]: starting ceph client 2022-08-01T19:41:06.567 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr: 2022-08-01T19:41:06.570 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:ceph-fuse[23193]: starting fuse 2022-08-01T19:41:06.575 INFO:teuthology.orchestra.run.smithi085.stdout:52 2022-08-01T19:41:06.576 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T19:41:06.577 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:06.577 DEBUG:teuthology.orchestra.run.smithi085:> sudo python3 -c ' 2022-08-01T19:41:06.578 DEBUG:teuthology.orchestra.run.smithi085:> import glob 2022-08-01T19:41:06.580 DEBUG:teuthology.orchestra.run.smithi085:> import re 2022-08-01T19:41:06.581 DEBUG:teuthology.orchestra.run.smithi085:> import os 2022-08-01T19:41:06.581 DEBUG:teuthology.orchestra.run.smithi085:> import subprocess 2022-08-01T19:41:06.582 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:41:06.583 DEBUG:teuthology.orchestra.run.smithi085:> def find_socket(client_name): 2022-08-01T19:41:06.583 DEBUG:teuthology.orchestra.run.smithi085:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-08-01T19:41:06.584 DEBUG:teuthology.orchestra.run.smithi085:> files = glob.glob(asok_path) 2022-08-01T19:41:06.585 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:41:06.585 DEBUG:teuthology.orchestra.run.smithi085:> # Given a non-glob path, it better be there 2022-08-01T19:41:06.586 DEBUG:teuthology.orchestra.run.smithi085:> if "*" not in asok_path: 2022-08-01T19:41:06.586 DEBUG:teuthology.orchestra.run.smithi085:> assert(len(files) == 1) 2022-08-01T19:41:06.586 DEBUG:teuthology.orchestra.run.smithi085:> return files[0] 2022-08-01T19:41:06.587 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:41:06.587 DEBUG:teuthology.orchestra.run.smithi085:> for f in files: 2022-08-01T19:41:06.588 DEBUG:teuthology.orchestra.run.smithi085:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T19:41:06.588 DEBUG:teuthology.orchestra.run.smithi085:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T19:41:06.589 DEBUG:teuthology.orchestra.run.smithi085:> return f 2022-08-01T19:41:06.589 DEBUG:teuthology.orchestra.run.smithi085:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T19:41:06.590 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:41:06.590 DEBUG:teuthology.orchestra.run.smithi085:> print(find_socket("client.0")) 2022-08-01T19:41:06.591 DEBUG:teuthology.orchestra.run.smithi085:> ' 2022-08-01T19:41:06.675 INFO:teuthology.orchestra.run.smithi085.stdout:/var/run/ceph/ceph-client.0.23193.asok 2022-08-01T19:41:06.676 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.23193.asok 2022-08-01T19:41:06.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:06.678 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.23193.asok status 2022-08-01T19:41:06.812 INFO:teuthology.orchestra.run.smithi085.stdout:{ 2022-08-01T19:41:06.812 INFO:teuthology.orchestra.run.smithi085.stdout: "metadata": { 2022-08-01T19:41:06.813 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T19:41:06.818 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T19:41:06.818 INFO:teuthology.orchestra.run.smithi085.stdout: "entity_id": "0", 2022-08-01T19:41:06.819 INFO:teuthology.orchestra.run.smithi085.stdout: "hostname": "smithi085", 2022-08-01T19:41:06.819 INFO:teuthology.orchestra.run.smithi085.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-08-01T19:41:06.819 INFO:teuthology.orchestra.run.smithi085.stdout: "pid": "23193", 2022-08-01T19:41:06.819 INFO:teuthology.orchestra.run.smithi085.stdout: "root": "/" 2022-08-01T19:41:06.820 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:41:06.820 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_count": 0, 2022-08-01T19:41:06.820 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_pinned_count": 0, 2022-08-01T19:41:06.821 INFO:teuthology.orchestra.run.smithi085.stdout: "id": 6784, 2022-08-01T19:41:06.821 INFO:teuthology.orchestra.run.smithi085.stdout: "inst": { 2022-08-01T19:41:06.821 INFO:teuthology.orchestra.run.smithi085.stdout: "name": { 2022-08-01T19:41:06.821 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "client", 2022-08-01T19:41:06.822 INFO:teuthology.orchestra.run.smithi085.stdout: "num": 6784 2022-08-01T19:41:06.822 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:41:06.822 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2022-08-01T19:41:06.823 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2022-08-01T19:41:06.823 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2022-08-01T19:41:06.823 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 91357527 2022-08-01T19:41:06.824 INFO:teuthology.orchestra.run.smithi085.stdout: } 2022-08-01T19:41:06.824 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:41:06.824 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2022-08-01T19:41:06.825 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2022-08-01T19:41:06.829 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2022-08-01T19:41:06.829 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 91357527 2022-08-01T19:41:06.830 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:41:06.830 INFO:teuthology.orchestra.run.smithi085.stdout: "inst_str": "client.6784 172.21.15.85:0/91357527", 2022-08-01T19:41:06.831 INFO:teuthology.orchestra.run.smithi085.stdout: "addr_str": "172.21.15.85:0/91357527", 2022-08-01T19:41:06.831 INFO:teuthology.orchestra.run.smithi085.stdout: "inode_count": 1, 2022-08-01T19:41:06.832 INFO:teuthology.orchestra.run.smithi085.stdout: "mds_epoch": 40, 2022-08-01T19:41:06.832 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch": 88, 2022-08-01T19:41:06.832 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch_barrier": 0, 2022-08-01T19:41:06.833 INFO:teuthology.orchestra.run.smithi085.stdout: "blacklisted": false, 2022-08-01T19:41:06.833 INFO:teuthology.orchestra.run.smithi085.stdout: "fs_name": "cephfs" 2022-08-01T19:41:06.834 INFO:teuthology.orchestra.run.smithi085.stdout:} 2022-08-01T19:41:06.836 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:06.836 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:41:06.837 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:41:06.846 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:41:06.847 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:41:06.848 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:06.849 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:41:06.869 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:41:06.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.975+0000 7fee2cba1700 1 Processor -- start 2022-08-01T19:41:06.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.975+0000 7fee2cba1700 1 -- start start 2022-08-01T19:41:06.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.979+0000 7fee2cba1700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fee28061890 0x7fee281466b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:06.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.979+0000 7fee2cba1700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee280aee80 con 0x7fee280614b0 2022-08-01T19:41:06.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.979+0000 7fee2cba1700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee280b70b0 con 0x7fee280ad750 2022-08-01T19:41:06.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.979+0000 7fee2cba1700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee280af0b0 con 0x7fee28061890 2022-08-01T19:41:06.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.979+0000 7fee277fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fee280614b0 0x7fee280b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33692/0 (socket says 172.21.15.17:33692) 2022-08-01T19:41:06.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.979+0000 7fee277fe700 1 -- 172.21.15.17:0/2461007097 learned_addr learned my addr 172.21.15.17:0/2461007097 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:06.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.979+0000 7fee27fff700 1 --2- 172.21.15.17:0/2461007097 >> v2:172.21.15.77:3300/0 conn(0x7fee28061890 0x7fee281466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:06.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.979+0000 7fee267fc700 1 -- 172.21.15.17:0/2461007097 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 792569648 0 0) 0x7fee280aee80 con 0x7fee280614b0 2022-08-01T19:41:06.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.979+0000 7fee267fc700 1 -- 172.21.15.17:0/2461007097 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee08003260 con 0x7fee280614b0 2022-08-01T19:41:06.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.979+0000 7fee27fff700 1 -- 172.21.15.17:0/2461007097 >> v1:172.21.15.77:6789/0 conn(0x7fee280ad750 legacy=0x7fee281405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:06.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.979+0000 7fee27fff700 1 -- 172.21.15.17:0/2461007097 >> v1:172.21.15.17:6789/0 conn(0x7fee280614b0 legacy=0x7fee280b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:06.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.979+0000 7fee27fff700 1 -- 172.21.15.17:0/2461007097 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee280af350 con 0x7fee28061890 2022-08-01T19:41:06.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.979+0000 7fee27fff700 1 --2- 172.21.15.17:0/2461007097 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee28061890 0x7fee281466b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7fee200032f0 tx=0x7fee200091d0).ready entity=mon.1 client_cookie=9a8f0abbdfd9b86d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:06.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.979+0000 7fee267fc700 1 -- 172.21.15.17:0/2461007097 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee2000b6c0 con 0x7fee28061890 2022-08-01T19:41:06.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.979+0000 7fee267fc700 1 -- 172.21.15.17:0/2461007097 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2000b820 con 0x7fee28061890 2022-08-01T19:41:06.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.979+0000 7fee267fc700 1 -- 172.21.15.17:0/2461007097 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee2000b990 con 0x7fee28061890 2022-08-01T19:41:06.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.979+0000 7fee2cba1700 1 -- 172.21.15.17:0/2461007097 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee28061890 msgr2=0x7fee281466b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:06.991 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.979+0000 7fee2cba1700 1 --2- 172.21.15.17:0/2461007097 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee28061890 0x7fee281466b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7fee200032f0 tx=0x7fee200091d0).stop 2022-08-01T19:41:06.991 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.979+0000 7fee2cba1700 1 -- 172.21.15.17:0/2461007097 shutdown_connections 2022-08-01T19:41:06.991 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.979+0000 7fee2cba1700 1 --2- 172.21.15.17:0/2461007097 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee28061890 0x7fee281466b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:06.991 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.979+0000 7fee2cba1700 1 -- 172.21.15.17:0/2461007097 >> 172.21.15.17:0/2461007097 conn(0x7fee28066f70 msgr2=0x7fee28067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:06.992 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.979+0000 7fee2cba1700 1 -- 172.21.15.17:0/2461007097 shutdown_connections 2022-08-01T19:41:06.992 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.979+0000 7fee2cba1700 1 -- 172.21.15.17:0/2461007097 wait complete. 2022-08-01T19:41:06.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.983+0000 7fee2cba1700 1 Processor -- start 2022-08-01T19:41:06.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.983+0000 7fee2cba1700 1 -- start start 2022-08-01T19:41:06.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.983+0000 7fee2cba1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee280614b0 0x7fee2805cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:06.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.983+0000 7fee2cba1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee28061890 0x7fee28156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:06.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.983+0000 7fee2cba1700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fee280ad750 0x7fee28152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:06.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.983+0000 7fee2cba1700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee281568f0 con 0x7fee280614b0 2022-08-01T19:41:06.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.983+0000 7fee2cba1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee280b1450 con 0x7fee28061890 2022-08-01T19:41:06.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.983+0000 7fee2cba1700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee28146300 con 0x7fee280ad750 2022-08-01T19:41:06.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.983+0000 7fee277fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee280614b0 0x7fee2805cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:06.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.983+0000 7fee277fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee280614b0 0x7fee2805cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41504/0 (socket says 172.21.15.17:41504) 2022-08-01T19:41:06.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.983+0000 7fee277fe700 1 -- 172.21.15.17:0/1877918366 learned_addr learned my addr 172.21.15.17:0/1877918366 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:06.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.983+0000 7fee26ffd700 1 --2- 172.21.15.17:0/1877918366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee28061890 0x7fee28156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:06.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.983+0000 7fee27fff700 1 --2- 172.21.15.17:0/1877918366 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fee280ad750 0x7fee28152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:07.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.983+0000 7fee277fe700 1 -- 172.21.15.17:0/1877918366 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fee280ad750 msgr2=0x7fee28152940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:07.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.983+0000 7fee277fe700 1 --2- 172.21.15.17:0/1877918366 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fee280ad750 0x7fee28152940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:07.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.983+0000 7fee277fe700 1 -- 172.21.15.17:0/1877918366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee28061890 msgr2=0x7fee28156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:07.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.983+0000 7fee277fe700 1 --2- 172.21.15.17:0/1877918366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee28061890 0x7fee28156400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:07.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.983+0000 7fee277fe700 1 -- 172.21.15.17:0/1877918366 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee20009470 con 0x7fee280614b0 2022-08-01T19:41:07.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.983+0000 7fee277fe700 1 --2- 172.21.15.17:0/1877918366 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee280614b0 0x7fee2805cb40 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0x7fee10001110 tx=0x7fee10007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:07.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.987+0000 7fee24ff9700 1 -- 172.21.15.17:0/1877918366 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee100070d0 con 0x7fee280614b0 2022-08-01T19:41:07.002 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.987+0000 7fee24ff9700 1 -- 172.21.15.17:0/1877918366 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee10007230 con 0x7fee280614b0 2022-08-01T19:41:07.002 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.987+0000 7fee2cba1700 1 -- 172.21.15.17:0/1877918366 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee281524e0 con 0x7fee280614b0 2022-08-01T19:41:07.002 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.987+0000 7fee24ff9700 1 -- 172.21.15.17:0/1877918366 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee10005610 con 0x7fee280614b0 2022-08-01T19:41:07.002 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.987+0000 7fee2cba1700 1 -- 172.21.15.17:0/1877918366 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee281472b0 con 0x7fee280614b0 2022-08-01T19:41:07.003 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.987+0000 7fee2cba1700 1 -- 172.21.15.17:0/1877918366 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee280af5f0 con 0x7fee280614b0 2022-08-01T19:41:07.003 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.987+0000 7fee24ff9700 1 -- 172.21.15.17:0/1877918366 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fee10010070 con 0x7fee280614b0 2022-08-01T19:41:07.003 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.991+0000 7fee24ff9700 1 --2- 172.21.15.17:0/1877918366 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fee0c0539e0 0x7fee0c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:07.003 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.991+0000 7fee24ff9700 1 -- 172.21.15.17:0/1877918366 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fee1006b640 con 0x7fee280614b0 2022-08-01T19:41:07.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.991+0000 7fee26ffd700 1 --2- 172.21.15.17:0/1877918366 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fee0c0539e0 0x7fee0c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:07.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.991+0000 7fee26ffd700 1 --2- 172.21.15.17:0/1877918366 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fee0c0539e0 0x7fee0c055ea0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7fee1800d000 tx=0x7fee18007b90).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:07.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:06.995+0000 7fee24ff9700 1 -- 172.21.15.17:0/1877918366 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fee10042400 con 0x7fee280614b0 2022-08-01T19:41:07.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.123+0000 7fee2cba1700 1 -- 172.21.15.17:0/1877918366 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee280574e0 con 0x7fee280614b0 2022-08-01T19:41:07.134 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.123+0000 7fee24ff9700 1 -- 172.21.15.17:0/1877918366 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16274 (secure 0 0 0) 0x7fee10045a20 con 0x7fee280614b0 2022-08-01T19:41:07.134 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:07.135 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":88,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:41:02.210932+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:40:59.229815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2022-08-01T19:41:00.225322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:41:07.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.127+0000 7fee2cba1700 1 -- 172.21.15.17:0/1877918366 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fee0c0539e0 msgr2=0x7fee0c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:07.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.127+0000 7fee2cba1700 1 --2- 172.21.15.17:0/1877918366 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fee0c0539e0 0x7fee0c055ea0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7fee1800d000 tx=0x7fee18007b90).stop 2022-08-01T19:41:07.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.127+0000 7fee2cba1700 1 -- 172.21.15.17:0/1877918366 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee280614b0 msgr2=0x7fee2805cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:07.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.127+0000 7fee2cba1700 1 --2- 172.21.15.17:0/1877918366 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee280614b0 0x7fee2805cb40 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0x7fee10001110 tx=0x7fee10007ad0).stop 2022-08-01T19:41:07.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.127+0000 7fee2cba1700 1 -- 172.21.15.17:0/1877918366 shutdown_connections 2022-08-01T19:41:07.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.127+0000 7fee2cba1700 1 --2- 172.21.15.17:0/1877918366 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fee0c0539e0 0x7fee0c055ea0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:07.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.127+0000 7fee2cba1700 1 --2- 172.21.15.17:0/1877918366 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fee280ad750 0x7fee28152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:07.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.127+0000 7fee2cba1700 1 --2- 172.21.15.17:0/1877918366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee28061890 0x7fee28156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:07.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.127+0000 7fee2cba1700 1 --2- 172.21.15.17:0/1877918366 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee280614b0 0x7fee2805cb40 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:07.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.127+0000 7fee2cba1700 1 -- 172.21.15.17:0/1877918366 >> 172.21.15.17:0/1877918366 conn(0x7fee28066f70 msgr2=0x7fee28141b50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:07.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.127+0000 7fee2cba1700 1 -- 172.21.15.17:0/1877918366 shutdown_connections 2022-08-01T19:41:07.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.127+0000 7fee2cba1700 1 -- 172.21.15.17:0/1877918366 wait complete. 2022-08-01T19:41:07.147 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T19:40:59.229815+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T19:41:00.225322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T19:41:07.148 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:41:07.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.251+0000 7f6c8cf52700 1 Processor -- start 2022-08-01T19:41:07.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.251+0000 7f6c8cf52700 1 -- start start 2022-08-01T19:41:07.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.255+0000 7f6c8cf52700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6c880b4310 0x7f6c88140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:07.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.255+0000 7f6c8cf52700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c88143cf0 con 0x7f6c880b4820 2022-08-01T19:41:07.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.255+0000 7f6c8cf52700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c88143f90 con 0x7f6c88146ae0 2022-08-01T19:41:07.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.255+0000 7f6c8cf52700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c881476a0 con 0x7f6c880b4310 2022-08-01T19:41:07.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.255+0000 7f6c86d9d700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f6c880b4820 0x7f6c880b5060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33710/0 (socket says 172.21.15.17:33710) 2022-08-01T19:41:07.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.255+0000 7f6c86d9d700 1 -- 172.21.15.17:0/4001137892 learned_addr learned my addr 172.21.15.17:0/4001137892 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:07.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.255+0000 7f6c7dd9b700 1 --2- 172.21.15.17:0/4001137892 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6c880b4310 0x7f6c88140ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:07.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.255+0000 7f6c85d9b700 1 -- 172.21.15.17:0/4001137892 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4066450098 0 0) 0x7f6c88143cf0 con 0x7f6c880b4820 2022-08-01T19:41:07.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.255+0000 7f6c85d9b700 1 -- 172.21.15.17:0/4001137892 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c68003260 con 0x7f6c880b4820 2022-08-01T19:41:07.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.255+0000 7f6c85d9b700 1 -- 172.21.15.17:0/4001137892 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 675826085 0 0) 0x7f6c68003260 con 0x7f6c880b4820 2022-08-01T19:41:07.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.255+0000 7f6c85d9b700 1 -- 172.21.15.17:0/4001137892 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6c880b4310 msgr2=0x7f6c88140ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:07.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.255+0000 7f6c85d9b700 1 --2- 172.21.15.17:0/4001137892 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6c880b4310 0x7f6c88140ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:07.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.255+0000 7f6c85d9b700 1 -- 172.21.15.17:0/4001137892 >> v1:172.21.15.77:6789/0 conn(0x7f6c88146ae0 legacy=0x7f6c881403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:07.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.255+0000 7f6c85d9b700 1 -- 172.21.15.17:0/4001137892 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c880b0e20 con 0x7f6c880b4820 2022-08-01T19:41:07.266 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.255+0000 7f6c85d9b700 1 -- 172.21.15.17:0/4001137892 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7f6c800019e0 con 0x7f6c880b4820 2022-08-01T19:41:07.266 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.255+0000 7f6c85d9b700 1 -- 172.21.15.17:0/4001137892 >> v1:172.21.15.17:6789/0 conn(0x7f6c880b4820 legacy=0x7f6c880b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:07.266 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.255+0000 7f6c85d9b700 1 --2- 172.21.15.17:0/4001137892 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6c880b4310 0x7f6c68003780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:07.266 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.255+0000 7f6c85d9b700 1 --2- 172.21.15.17:0/4001137892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c68003c70 0x7f6c68004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:07.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.255+0000 7f6c85d9b700 1 --2- 172.21.15.17:0/4001137892 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c680049f0 0x7f6c68004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:07.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.255+0000 7f6c85d9b700 1 -- 172.21.15.17:0/4001137892 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c881476a0 con 0x7f6c680049f0 2022-08-01T19:41:07.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.255+0000 7f6c85d9b700 1 -- 172.21.15.17:0/4001137892 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6800b750 con 0x7f6c68003c70 2022-08-01T19:41:07.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.255+0000 7f6c85d9b700 1 -- 172.21.15.17:0/4001137892 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6800b910 con 0x7f6c880b4310 2022-08-01T19:41:07.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.255+0000 7f6c86d9d700 1 --2- 172.21.15.17:0/4001137892 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c680049f0 0x7f6c68004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:07.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.255+0000 7f6c7dd9b700 1 --2- 172.21.15.17:0/4001137892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c68003c70 0x7f6c68004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:07.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.255+0000 7f6c8659c700 1 --2- 172.21.15.17:0/4001137892 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6c880b4310 0x7f6c68003780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:07.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.255+0000 7f6c7dd9b700 1 -- 172.21.15.17:0/4001137892 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6c880b4310 msgr2=0x7f6c68003780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:07.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.255+0000 7f6c7dd9b700 1 --2- 172.21.15.17:0/4001137892 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6c880b4310 0x7f6c68003780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:07.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.255+0000 7f6c7dd9b700 1 -- 172.21.15.17:0/4001137892 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c680049f0 msgr2=0x7f6c68004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:07.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.255+0000 7f6c7dd9b700 1 --2- 172.21.15.17:0/4001137892 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c680049f0 0x7f6c68004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:07.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.255+0000 7f6c7dd9b700 1 -- 172.21.15.17:0/4001137892 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c68003260 con 0x7f6c68003c70 2022-08-01T19:41:07.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.255+0000 7f6c86d9d700 1 --2- 172.21.15.17:0/4001137892 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c680049f0 0x7f6c68004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:07.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.255+0000 7f6c7dd9b700 1 --2- 172.21.15.17:0/4001137892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c68003c70 0x7f6c68004500 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f6c7000c000 tx=0x7f6c70006df0).ready entity=mon.1 client_cookie=884898a98b1d34d5 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:07.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.255+0000 7f6c85d9b700 1 -- 172.21.15.17:0/4001137892 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c7000e800 con 0x7f6c68003c70 2022-08-01T19:41:07.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.255+0000 7f6c85d9b700 1 -- 172.21.15.17:0/4001137892 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c7000e960 con 0x7f6c68003c70 2022-08-01T19:41:07.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.255+0000 7f6c85d9b700 1 -- 172.21.15.17:0/4001137892 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c7000ecc0 con 0x7f6c68003c70 2022-08-01T19:41:07.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.259+0000 7f6c8cf52700 1 -- 172.21.15.17:0/4001137892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c68003c70 msgr2=0x7f6c68004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:07.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.259+0000 7f6c8cf52700 1 --2- 172.21.15.17:0/4001137892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c68003c70 0x7f6c68004500 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f6c7000c000 tx=0x7f6c70006df0).stop 2022-08-01T19:41:07.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.259+0000 7f6c86d9d700 1 -- 172.21.15.17:0/4001137892 reap_dead start 2022-08-01T19:41:07.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.259+0000 7f6c8cf52700 1 -- 172.21.15.17:0/4001137892 shutdown_connections 2022-08-01T19:41:07.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.259+0000 7f6c8cf52700 1 -- 172.21.15.17:0/4001137892 >> 172.21.15.17:0/4001137892 conn(0x7f6c8805c620 msgr2=0x7f6c8805ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:07.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.259+0000 7f6c8cf52700 1 -- 172.21.15.17:0/4001137892 shutdown_connections 2022-08-01T19:41:07.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.259+0000 7f6c8cf52700 1 -- 172.21.15.17:0/4001137892 wait complete. 2022-08-01T19:41:07.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.259+0000 7f6c8cf52700 1 Processor -- start 2022-08-01T19:41:07.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.259+0000 7f6c8cf52700 1 -- start start 2022-08-01T19:41:07.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.259+0000 7f6c8cf52700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c880ae760 0x7f6c880aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:07.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.259+0000 7f6c8cf52700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c880af030 0x7f6c88152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:07.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.259+0000 7f6c8cf52700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6c880b78e0 0x7f6c88152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:07.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.259+0000 7f6c8cf52700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c880b7e40 con 0x7f6c880af030 2022-08-01T19:41:07.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.259+0000 7f6c8cf52700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c881525b0 con 0x7f6c880ae760 2022-08-01T19:41:07.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.259+0000 7f6c8cf52700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c88152850 con 0x7f6c880b78e0 2022-08-01T19:41:07.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.259+0000 7f6c7dd9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c880af030 0x7f6c88152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:07.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.259+0000 7f6c7dd9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c880af030 0x7f6c88152af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41526/0 (socket says 172.21.15.17:41526) 2022-08-01T19:41:07.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.259+0000 7f6c8659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c880ae760 0x7f6c880aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:07.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.259+0000 7f6c7dd9b700 1 -- 172.21.15.17:0/1788388827 learned_addr learned my addr 172.21.15.17:0/1788388827 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:07.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.259+0000 7f6c86d9d700 1 --2- 172.21.15.17:0/1788388827 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6c880b78e0 0x7f6c88152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:07.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.263+0000 7f6c7dd9b700 1 -- 172.21.15.17:0/1788388827 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6c880b78e0 msgr2=0x7f6c88152fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:07.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.263+0000 7f6c7dd9b700 1 --2- 172.21.15.17:0/1788388827 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6c880b78e0 0x7f6c88152fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:07.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.263+0000 7f6c7dd9b700 1 -- 172.21.15.17:0/1788388827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c880ae760 msgr2=0x7f6c880aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:07.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.263+0000 7f6c7dd9b700 1 --2- 172.21.15.17:0/1788388827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c880ae760 0x7f6c880aeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:07.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.263+0000 7f6c7dd9b700 1 -- 172.21.15.17:0/1788388827 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c881534d0 con 0x7f6c880af030 2022-08-01T19:41:07.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.263+0000 7f6c7dd9b700 1 --2- 172.21.15.17:0/1788388827 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c880af030 0x7f6c88152af0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 rx=0x7f6c7000c000 tx=0x7f6c70019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:07.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.263+0000 7f6c7ffff700 1 -- 172.21.15.17:0/1788388827 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c7001cda0 con 0x7f6c880af030 2022-08-01T19:41:07.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.263+0000 7f6c7ffff700 1 -- 172.21.15.17:0/1788388827 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c70000e90 con 0x7f6c880af030 2022-08-01T19:41:07.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.263+0000 7f6c7ffff700 1 -- 172.21.15.17:0/1788388827 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c700245e0 con 0x7f6c880af030 2022-08-01T19:41:07.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.263+0000 7f6c8cf52700 1 -- 172.21.15.17:0/1788388827 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c880b6b80 con 0x7f6c880af030 2022-08-01T19:41:07.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.263+0000 7f6c8cf52700 1 -- 172.21.15.17:0/1788388827 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c880af8c0 con 0x7f6c880af030 2022-08-01T19:41:07.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.263+0000 7f6c8cf52700 1 -- 172.21.15.17:0/1788388827 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c880b7f80 con 0x7f6c880af030 2022-08-01T19:41:07.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.263+0000 7f6c7ffff700 1 -- 172.21.15.17:0/1788388827 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6c7002a020 con 0x7f6c880af030 2022-08-01T19:41:07.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.263+0000 7f6c7ffff700 1 --2- 172.21.15.17:0/1788388827 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6c6c053950 0x7f6c6c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:07.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.263+0000 7f6c7ffff700 1 -- 172.21.15.17:0/1788388827 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6c7006d620 con 0x7f6c880af030 2022-08-01T19:41:07.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.267+0000 7f6c8659c700 1 --2- 172.21.15.17:0/1788388827 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6c6c053950 0x7f6c6c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:07.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.267+0000 7f6c8659c700 1 --2- 172.21.15.17:0/1788388827 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6c6c053950 0x7f6c6c055e10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f6c78003000 tx=0x7f6c780079e0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:07.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.271+0000 7f6c7ffff700 1 -- 172.21.15.17:0/1788388827 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6c700443e0 con 0x7f6c880af030 2022-08-01T19:41:07.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.399+0000 7f6c8cf52700 1 -- 172.21.15.17:0/1788388827 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f6c880574e0 con 0x7f6c880af030 2022-08-01T19:41:07.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.399+0000 7f6c7ffff700 1 -- 172.21.15.17:0/1788388827 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v88) v1 ==== 110+0+10 (secure 0 0 0) 0x7f6c70047a00 con 0x7f6c880af030 2022-08-01T19:41:07.405 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:41:07.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.403+0000 7f6c8cf52700 1 -- 172.21.15.17:0/1788388827 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6c6c053950 msgr2=0x7f6c6c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:07.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.403+0000 7f6c8cf52700 1 --2- 172.21.15.17:0/1788388827 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6c6c053950 0x7f6c6c055e10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f6c78003000 tx=0x7f6c780079e0).stop 2022-08-01T19:41:07.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.403+0000 7f6c8cf52700 1 -- 172.21.15.17:0/1788388827 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c880af030 msgr2=0x7f6c88152af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:07.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.403+0000 7f6c8cf52700 1 --2- 172.21.15.17:0/1788388827 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c880af030 0x7f6c88152af0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 rx=0x7f6c7000c000 tx=0x7f6c70019000).stop 2022-08-01T19:41:07.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.403+0000 7f6c8cf52700 1 -- 172.21.15.17:0/1788388827 shutdown_connections 2022-08-01T19:41:07.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.403+0000 7f6c8cf52700 1 --2- 172.21.15.17:0/1788388827 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6c6c053950 0x7f6c6c055e10 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:07.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.403+0000 7f6c8cf52700 1 --2- 172.21.15.17:0/1788388827 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6c880b78e0 0x7f6c88152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:07.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.403+0000 7f6c8cf52700 1 --2- 172.21.15.17:0/1788388827 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c880af030 0x7f6c88152af0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:07.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.403+0000 7f6c8cf52700 1 --2- 172.21.15.17:0/1788388827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c880ae760 0x7f6c880aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:07.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.403+0000 7f6c8cf52700 1 -- 172.21.15.17:0/1788388827 >> 172.21.15.17:0/1788388827 conn(0x7f6c8805c620 msgr2=0x7f6c8813b280 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:07.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.403+0000 7f6c8cf52700 1 -- 172.21.15.17:0/1788388827 shutdown_connections 2022-08-01T19:41:07.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.403+0000 7f6c8cf52700 1 -- 172.21.15.17:0/1788388827 wait complete. 2022-08-01T19:41:07.427 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:41:07.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.535+0000 7f7025b9a700 1 Processor -- start 2022-08-01T19:41:07.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.535+0000 7f7025b9a700 1 -- start start 2022-08-01T19:41:07.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.539+0000 7f7025b9a700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f70200b4310 0x7f7020140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:07.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.539+0000 7f7025b9a700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7020143cf0 con 0x7f7020146ae0 2022-08-01T19:41:07.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.539+0000 7f7025b9a700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7020143f90 con 0x7f70200b4820 2022-08-01T19:41:07.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.539+0000 7f7025b9a700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70201476a0 con 0x7f70200b4310 2022-08-01T19:41:07.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.539+0000 7f701effd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f70200b4310 0x7f7020140ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:07.550 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.539+0000 7f701effd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f70200b4310 0x7f7020140ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41534/0 (socket says 172.21.15.17:41534) 2022-08-01T19:41:07.550 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.539+0000 7f701f7fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f7020146ae0 0x7f70201403c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33726/0 (socket says 172.21.15.17:33726) 2022-08-01T19:41:07.550 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.539+0000 7f701effd700 1 -- 172.21.15.17:0/3838342803 learned_addr learned my addr 172.21.15.17:0/3838342803 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:07.550 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.539+0000 7f701effd700 1 -- 172.21.15.17:0/3838342803 >> v1:172.21.15.77:6789/0 conn(0x7f70200b4820 legacy=0x7f70200b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:07.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.539+0000 7f701effd700 1 -- 172.21.15.17:0/3838342803 >> v1:172.21.15.17:6789/0 conn(0x7f7020146ae0 legacy=0x7f70201403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:07.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.539+0000 7f701effd700 1 -- 172.21.15.17:0/3838342803 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70200b0e20 con 0x7f70200b4310 2022-08-01T19:41:07.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.539+0000 7f701effd700 1 --2- 172.21.15.17:0/3838342803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70200b4310 0x7f7020140ad0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7f7010002f70 tx=0x7f7010006eb0).ready entity=mon.0 client_cookie=22f140780dbab1d4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:07.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.539+0000 7f701e7fc700 1 -- 172.21.15.17:0/3838342803 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f701000d660 con 0x7f70200b4310 2022-08-01T19:41:07.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.539+0000 7f701e7fc700 1 -- 172.21.15.17:0/3838342803 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7010000e90 con 0x7f70200b4310 2022-08-01T19:41:07.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.539+0000 7f701e7fc700 1 -- 172.21.15.17:0/3838342803 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f701000d9f0 con 0x7f70200b4310 2022-08-01T19:41:07.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.539+0000 7f7025b9a700 1 -- 172.21.15.17:0/3838342803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70200b4310 msgr2=0x7f7020140ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:07.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.539+0000 7f7025b9a700 1 --2- 172.21.15.17:0/3838342803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70200b4310 0x7f7020140ad0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7f7010002f70 tx=0x7f7010006eb0).stop 2022-08-01T19:41:07.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.539+0000 7f7025b9a700 1 -- 172.21.15.17:0/3838342803 shutdown_connections 2022-08-01T19:41:07.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.539+0000 7f7025b9a700 1 --2- 172.21.15.17:0/3838342803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70200b4310 0x7f7020140ad0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:07.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.539+0000 7f7025b9a700 1 -- 172.21.15.17:0/3838342803 >> 172.21.15.17:0/3838342803 conn(0x7f702005c620 msgr2=0x7f702005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:07.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.539+0000 7f7025b9a700 1 -- 172.21.15.17:0/3838342803 shutdown_connections 2022-08-01T19:41:07.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.539+0000 7f7025b9a700 1 -- 172.21.15.17:0/3838342803 wait complete. 2022-08-01T19:41:07.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.543+0000 7f7025b9a700 1 Processor -- start 2022-08-01T19:41:07.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.543+0000 7f7025b9a700 1 -- start start 2022-08-01T19:41:07.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.543+0000 7f7025b9a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70200b4310 0x7f702013e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:07.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.547+0000 7f7025b9a700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70200b4820 0x7f7020144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:07.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.547+0000 7f7025b9a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7020146ae0 0x7f70201450c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:07.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.547+0000 7f7025b9a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70201455b0 con 0x7f7020146ae0 2022-08-01T19:41:07.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.547+0000 7f7025b9a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70200af620 con 0x7f70200b4310 2022-08-01T19:41:07.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.547+0000 7f7025b9a700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70200b06a0 con 0x7f70200b4820 2022-08-01T19:41:07.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.547+0000 7f701ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7020146ae0 0x7f70201450c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:07.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.547+0000 7f701ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7020146ae0 0x7f70201450c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41542/0 (socket says 172.21.15.17:41542) 2022-08-01T19:41:07.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.547+0000 7f701ffff700 1 -- 172.21.15.17:0/2767114469 learned_addr learned my addr 172.21.15.17:0/2767114469 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:07.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.547+0000 7f701f7fe700 1 --2- 172.21.15.17:0/2767114469 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70200b4310 0x7f702013e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:07.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.547+0000 7f701effd700 1 --2- 172.21.15.17:0/2767114469 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70200b4820 0x7f7020144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:07.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.547+0000 7f701ffff700 1 -- 172.21.15.17:0/2767114469 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70200b4820 msgr2=0x7f7020144bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:07.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.547+0000 7f701ffff700 1 --2- 172.21.15.17:0/2767114469 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70200b4820 0x7f7020144bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:07.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.547+0000 7f701ffff700 1 -- 172.21.15.17:0/2767114469 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70200b4310 msgr2=0x7f702013e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:07.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.547+0000 7f701ffff700 1 --2- 172.21.15.17:0/2767114469 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70200b4310 0x7f702013e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:07.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.547+0000 7f701ffff700 1 -- 172.21.15.17:0/2767114469 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f701000b1e0 con 0x7f7020146ae0 2022-08-01T19:41:07.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.547+0000 7f701f7fe700 1 --2- 172.21.15.17:0/2767114469 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70200b4310 0x7f702013e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:07.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.547+0000 7f701ffff700 1 --2- 172.21.15.17:0/2767114469 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7020146ae0 0x7f70201450c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7f7020060e20 tx=0x7f7018007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:07.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.547+0000 7f701cff9700 1 -- 172.21.15.17:0/2767114469 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f701800e4a0 con 0x7f7020146ae0 2022-08-01T19:41:07.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.547+0000 7f701cff9700 1 -- 172.21.15.17:0/2767114469 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f701800e600 con 0x7f7020146ae0 2022-08-01T19:41:07.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.547+0000 7f7025b9a700 1 -- 172.21.15.17:0/2767114469 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70200af850 con 0x7f7020146ae0 2022-08-01T19:41:07.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.547+0000 7f701cff9700 1 -- 172.21.15.17:0/2767114469 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f701800e960 con 0x7f7020146ae0 2022-08-01T19:41:07.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.547+0000 7f7025b9a700 1 -- 172.21.15.17:0/2767114469 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70200b0320 con 0x7f7020146ae0 2022-08-01T19:41:07.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.547+0000 7f7025b9a700 1 -- 172.21.15.17:0/2767114469 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7020145f50 con 0x7f7020146ae0 2022-08-01T19:41:07.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.547+0000 7f701cff9700 1 -- 172.21.15.17:0/2767114469 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7018019030 con 0x7f7020146ae0 2022-08-01T19:41:07.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.547+0000 7f701cff9700 1 --2- 172.21.15.17:0/2767114469 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f70000539e0 0x7f7000055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:07.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.551+0000 7f701cff9700 1 -- 172.21.15.17:0/2767114469 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f701806b7d0 con 0x7f7020146ae0 2022-08-01T19:41:07.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.551+0000 7f701f7fe700 1 --2- 172.21.15.17:0/2767114469 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f70000539e0 0x7f7000055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:07.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.551+0000 7f701f7fe700 1 --2- 172.21.15.17:0/2767114469 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f70000539e0 0x7f7000055ea0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f701400b000 tx=0x7f70140070e0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:07.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.555+0000 7f701cff9700 1 -- 172.21.15.17:0/2767114469 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7018042590 con 0x7f7020146ae0 2022-08-01T19:41:07.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.683+0000 7f7025b9a700 1 -- 172.21.15.17:0/2767114469 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f70200574e0 con 0x7f7020146ae0 2022-08-01T19:41:07.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.683+0000 7f701cff9700 1 -- 172.21.15.17:0/2767114469 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v88) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7018045bb0 con 0x7f7020146ae0 2022-08-01T19:41:07.690 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:41:07.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.687+0000 7f7025b9a700 1 -- 172.21.15.17:0/2767114469 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f70000539e0 msgr2=0x7f7000055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:07.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.687+0000 7f7025b9a700 1 --2- 172.21.15.17:0/2767114469 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f70000539e0 0x7f7000055ea0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f701400b000 tx=0x7f70140070e0).stop 2022-08-01T19:41:07.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.687+0000 7f7025b9a700 1 -- 172.21.15.17:0/2767114469 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7020146ae0 msgr2=0x7f70201450c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:07.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.687+0000 7f7025b9a700 1 --2- 172.21.15.17:0/2767114469 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7020146ae0 0x7f70201450c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7f7020060e20 tx=0x7f7018007f40).stop 2022-08-01T19:41:07.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.687+0000 7f7025b9a700 1 -- 172.21.15.17:0/2767114469 shutdown_connections 2022-08-01T19:41:07.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.687+0000 7f7025b9a700 1 --2- 172.21.15.17:0/2767114469 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f70000539e0 0x7f7000055ea0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:07.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.687+0000 7f7025b9a700 1 --2- 172.21.15.17:0/2767114469 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7020146ae0 0x7f70201450c0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:07.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.687+0000 7f7025b9a700 1 --2- 172.21.15.17:0/2767114469 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70200b4820 0x7f7020144bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:07.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.687+0000 7f7025b9a700 1 --2- 172.21.15.17:0/2767114469 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70200b4310 0x7f702013e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:07.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.687+0000 7f7025b9a700 1 -- 172.21.15.17:0/2767114469 >> 172.21.15.17:0/2767114469 conn(0x7f702005c620 msgr2=0x7f702005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:07.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.687+0000 7f7025b9a700 1 -- 172.21.15.17:0/2767114469 shutdown_connections 2022-08-01T19:41:07.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.687+0000 7f7025b9a700 1 -- 172.21.15.17:0/2767114469 wait complete. 2022-08-01T19:41:07.707 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T19:41:07.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.815+0000 7fb017fff700 1 Processor -- start 2022-08-01T19:41:07.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.815+0000 7fb017fff700 1 -- start start 2022-08-01T19:41:07.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.815+0000 7fb017fff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fb0180b6420 0x7fb018061600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:07.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.815+0000 7fb017fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb0180b41f0 0x7fb018061af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:07.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.815+0000 7fb017fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb018061fe0 0x7fb0181465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:07.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.815+0000 7fb017fff700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0180b60e0 con 0x7fb018061fe0 2022-08-01T19:41:07.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.815+0000 7fb017fff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0180b2100 con 0x7fb0180b6420 2022-08-01T19:41:07.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.815+0000 7fb017fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0180b23a0 con 0x7fb0180b41f0 2022-08-01T19:41:07.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.815+0000 7fb0177fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb018061fe0 0x7fb0181465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:07.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.815+0000 7fb0177fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb018061fe0 0x7fb0181465c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41550/0 (socket says 172.21.15.17:41550) 2022-08-01T19:41:07.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.815+0000 7fb0177fe700 1 -- 172.21.15.17:0/3924807268 learned_addr learned my addr 172.21.15.17:0/3924807268 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:07.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.815+0000 7fb00ffff700 1 --2- 172.21.15.17:0/3924807268 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb0180b41f0 0x7fb018061af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:07.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.815+0000 7fb016ffd700 1 --2- 172.21.15.17:0/3924807268 >> v2:172.21.15.77:3300/0 conn(0x7fb0180b6420 0x7fb018061600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:07.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.815+0000 7fb0177fe700 1 -- 172.21.15.17:0/3924807268 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb0180b41f0 msgr2=0x7fb018061af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:07.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.815+0000 7fb0177fe700 1 --2- 172.21.15.17:0/3924807268 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb0180b41f0 0x7fb018061af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:07.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.815+0000 7fb0177fe700 1 -- 172.21.15.17:0/3924807268 >> v2:172.21.15.77:3300/0 conn(0x7fb0180b6420 msgr2=0x7fb018061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:07.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.815+0000 7fb0177fe700 1 --2- 172.21.15.17:0/3924807268 >> v2:172.21.15.77:3300/0 conn(0x7fb0180b6420 0x7fb018061600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:07.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.815+0000 7fb0177fe700 1 -- 172.21.15.17:0/3924807268 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0180b2640 con 0x7fb018061fe0 2022-08-01T19:41:07.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.815+0000 7fb0177fe700 1 --2- 172.21.15.17:0/3924807268 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb018061fe0 0x7fb0181465c0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 rx=0x7fb0100032f0 tx=0x7fb010009290).ready entity=mon.0 client_cookie=2a854364fc02d91c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:07.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.815+0000 7fb0167fc700 1 -- 172.21.15.17:0/3924807268 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb01000b870 con 0x7fb018061fe0 2022-08-01T19:41:07.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.815+0000 7fb0167fc700 1 -- 172.21.15.17:0/3924807268 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb01000b9d0 con 0x7fb018061fe0 2022-08-01T19:41:07.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.815+0000 7fb0167fc700 1 -- 172.21.15.17:0/3924807268 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb01000bb40 con 0x7fb018061fe0 2022-08-01T19:41:07.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.819+0000 7fb017fff700 1 -- 172.21.15.17:0/3924807268 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb018061fe0 msgr2=0x7fb0181465c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:07.834 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.819+0000 7fb017fff700 1 --2- 172.21.15.17:0/3924807268 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb018061fe0 0x7fb0181465c0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 rx=0x7fb0100032f0 tx=0x7fb010009290).stop 2022-08-01T19:41:07.834 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.819+0000 7fb017fff700 1 -- 172.21.15.17:0/3924807268 shutdown_connections 2022-08-01T19:41:07.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.819+0000 7fb017fff700 1 --2- 172.21.15.17:0/3924807268 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb018061fe0 0x7fb0181465c0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:07.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.819+0000 7fb017fff700 1 --2- 172.21.15.17:0/3924807268 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb0180b41f0 0x7fb018061af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:07.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.819+0000 7fb017fff700 1 --2- 172.21.15.17:0/3924807268 >> v2:172.21.15.77:3300/0 conn(0x7fb0180b6420 0x7fb018061600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:07.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.819+0000 7fb017fff700 1 -- 172.21.15.17:0/3924807268 >> 172.21.15.17:0/3924807268 conn(0x7fb01805c620 msgr2=0x7fb01805ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:07.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.819+0000 7fb017fff700 1 -- 172.21.15.17:0/3924807268 shutdown_connections 2022-08-01T19:41:07.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.819+0000 7fb017fff700 1 -- 172.21.15.17:0/3924807268 wait complete. 2022-08-01T19:41:07.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.819+0000 7fb017fff700 1 Processor -- start 2022-08-01T19:41:07.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.819+0000 7fb017fff700 1 -- start start 2022-08-01T19:41:07.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.819+0000 7fb017fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb018061fe0 0x7fb018143a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:07.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.819+0000 7fb017fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0180b41f0 0x7fb01812b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:07.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.819+0000 7fb017fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb0180b6420 0x7fb01812b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:07.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.819+0000 7fb017fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01813ffc0 con 0x7fb018061fe0 2022-08-01T19:41:07.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.819+0000 7fb017fff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb018140260 con 0x7fb0180b41f0 2022-08-01T19:41:07.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.819+0000 7fb017fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb018140500 con 0x7fb0180b6420 2022-08-01T19:41:07.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.819+0000 7fb016ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb018061fe0 0x7fb018143a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:07.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.819+0000 7fb016ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb018061fe0 0x7fb018143a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41552/0 (socket says 172.21.15.17:41552) 2022-08-01T19:41:07.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.819+0000 7fb016ffd700 1 -- 172.21.15.17:0/3724799453 learned_addr learned my addr 172.21.15.17:0/3724799453 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:07.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.823+0000 7fb00ffff700 1 --2- 172.21.15.17:0/3724799453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0180b41f0 0x7fb01812b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:07.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.823+0000 7fb0177fe700 1 --2- 172.21.15.17:0/3724799453 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb0180b6420 0x7fb01812b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:07.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.823+0000 7fb016ffd700 1 -- 172.21.15.17:0/3724799453 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb0180b6420 msgr2=0x7fb01812b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:07.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.823+0000 7fb016ffd700 1 --2- 172.21.15.17:0/3724799453 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb0180b6420 0x7fb01812b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:07.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.823+0000 7fb016ffd700 1 -- 172.21.15.17:0/3724799453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0180b41f0 msgr2=0x7fb01812b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:07.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.823+0000 7fb016ffd700 1 --2- 172.21.15.17:0/3724799453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0180b41f0 0x7fb01812b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:07.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.823+0000 7fb016ffd700 1 -- 172.21.15.17:0/3724799453 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb010009530 con 0x7fb018061fe0 2022-08-01T19:41:07.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.823+0000 7fb00ffff700 1 --2- 172.21.15.17:0/3724799453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0180b41f0 0x7fb01812b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:07.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.823+0000 7fb016ffd700 1 --2- 172.21.15.17:0/3724799453 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb018061fe0 0x7fb018143a00 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7fb0080030b0 tx=0x7fb008008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:07.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.823+0000 7fb014ff9700 1 -- 172.21.15.17:0/3724799453 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb008011780 con 0x7fb018061fe0 2022-08-01T19:41:07.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.823+0000 7fb014ff9700 1 -- 172.21.15.17:0/3724799453 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0080118e0 con 0x7fb018061fe0 2022-08-01T19:41:07.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.823+0000 7fb017fff700 1 -- 172.21.15.17:0/3724799453 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0180aed60 con 0x7fb018061fe0 2022-08-01T19:41:07.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.823+0000 7fb014ff9700 1 -- 172.21.15.17:0/3724799453 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb008011c40 con 0x7fb018061fe0 2022-08-01T19:41:07.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.823+0000 7fb017fff700 1 -- 172.21.15.17:0/3724799453 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0180b0950 con 0x7fb018061fe0 2022-08-01T19:41:07.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.823+0000 7fb014ff9700 1 -- 172.21.15.17:0/3724799453 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb00801c460 con 0x7fb018061fe0 2022-08-01T19:41:07.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.823+0000 7fb014ff9700 1 --2- 172.21.15.17:0/3724799453 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7faffc053b00 0x7faffc055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:07.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.823+0000 7fb017fff700 1 -- 172.21.15.17:0/3724799453 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb018140ce0 con 0x7fb018061fe0 2022-08-01T19:41:07.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.823+0000 7fb014ff9700 1 -- 172.21.15.17:0/3724799453 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb00806c3c0 con 0x7fb018061fe0 2022-08-01T19:41:07.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.823+0000 7fb00ffff700 1 --2- 172.21.15.17:0/3724799453 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7faffc053b00 0x7faffc055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:07.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.827+0000 7fb00ffff700 1 --2- 172.21.15.17:0/3724799453 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7faffc053b00 0x7faffc055fc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7fb00000c000 tx=0x7fb000008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:07.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.831+0000 7fb014ff9700 1 -- 172.21.15.17:0/3724799453 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb008046a40 con 0x7fb018061fe0 2022-08-01T19:41:07.967 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.963+0000 7fb017fff700 1 -- 172.21.15.17:0/3724799453 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb0180b28e0 con 0x7fb018061fe0 2022-08-01T19:41:07.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.963+0000 7fb014ff9700 1 -- 172.21.15.17:0/3724799453 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+10 (secure 0 0 0) 0x7fb008045330 con 0x7fb018061fe0 2022-08-01T19:41:07.969 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:41:07.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.963+0000 7fb017fff700 1 -- 172.21.15.17:0/3724799453 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7faffc053b00 msgr2=0x7faffc055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:07.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.963+0000 7fb017fff700 1 --2- 172.21.15.17:0/3724799453 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7faffc053b00 0x7faffc055fc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7fb00000c000 tx=0x7fb000008040).stop 2022-08-01T19:41:07.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.963+0000 7fb017fff700 1 -- 172.21.15.17:0/3724799453 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb018061fe0 msgr2=0x7fb018143a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:07.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.963+0000 7fb017fff700 1 --2- 172.21.15.17:0/3724799453 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb018061fe0 0x7fb018143a00 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7fb0080030b0 tx=0x7fb008008040).stop 2022-08-01T19:41:07.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.967+0000 7fb017fff700 1 -- 172.21.15.17:0/3724799453 shutdown_connections 2022-08-01T19:41:07.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.967+0000 7fb017fff700 1 --2- 172.21.15.17:0/3724799453 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7faffc053b00 0x7faffc055fc0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:07.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.967+0000 7fb017fff700 1 --2- 172.21.15.17:0/3724799453 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb0180b6420 0x7fb01812b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:07.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.967+0000 7fb017fff700 1 --2- 172.21.15.17:0/3724799453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0180b41f0 0x7fb01812b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:07.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.967+0000 7fb017fff700 1 --2- 172.21.15.17:0/3724799453 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb018061fe0 0x7fb018143a00 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:07.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.967+0000 7fb017fff700 1 -- 172.21.15.17:0/3724799453 >> 172.21.15.17:0/3724799453 conn(0x7fb01805c620 msgr2=0x7fb0180b73f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:07.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.967+0000 7fb017fff700 1 -- 172.21.15.17:0/3724799453 shutdown_connections 2022-08-01T19:41:07.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:07.967+0000 7fb017fff700 1 -- 172.21.15.17:0/3724799453 wait complete. 2022-08-01T19:41:07.982 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T19:41:08.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.087+0000 7fd1e3fff700 1 Processor -- start 2022-08-01T19:41:08.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.087+0000 7fd1e3fff700 1 -- start start 2022-08-01T19:41:08.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.091+0000 7fd1e3fff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fd1e40b2710 0x7fd1e414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:08.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.091+0000 7fd1e3fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd1e40b01a0 0x7fd1e414b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:08.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.091+0000 7fd1e3fff700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1e40a7250 con 0x7fd1e40afd60 2022-08-01T19:41:08.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.091+0000 7fd1e3fff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1e40a7480 con 0x7fd1e40b2710 2022-08-01T19:41:08.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.091+0000 7fd1e3fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e40a7720 con 0x7fd1e40b01a0 2022-08-01T19:41:08.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.091+0000 7fd1e37fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd1e40afd60 0x7fd1e40a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33758/0 (socket says 172.21.15.17:33758) 2022-08-01T19:41:08.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.091+0000 7fd1e37fe700 1 -- 172.21.15.17:0/3305256925 learned_addr learned my addr 172.21.15.17:0/3305256925 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:08.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.091+0000 7fd1e2ffd700 1 --2- 172.21.15.17:0/3305256925 >> v2:172.21.15.77:3300/0 conn(0x7fd1e40b2710 0x7fd1e414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:08.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.091+0000 7fd1e27fc700 1 --2- 172.21.15.17:0/3305256925 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd1e40b01a0 0x7fd1e414b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:08.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.091+0000 7fd1e1ffb700 1 -- 172.21.15.17:0/3305256925 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3710962591 0 0) 0x7fd1e40a7250 con 0x7fd1e40afd60 2022-08-01T19:41:08.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.091+0000 7fd1e1ffb700 1 -- 172.21.15.17:0/3305256925 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1c4003260 con 0x7fd1e40afd60 2022-08-01T19:41:08.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.091+0000 7fd1e27fc700 1 -- 172.21.15.17:0/3305256925 >> v2:172.21.15.77:3300/0 conn(0x7fd1e40b2710 msgr2=0x7fd1e414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:08.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.091+0000 7fd1e27fc700 1 --2- 172.21.15.17:0/3305256925 >> v2:172.21.15.77:3300/0 conn(0x7fd1e40b2710 0x7fd1e414afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:08.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.091+0000 7fd1e27fc700 1 -- 172.21.15.17:0/3305256925 >> v1:172.21.15.17:6789/0 conn(0x7fd1e40afd60 legacy=0x7fd1e40a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:08.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.091+0000 7fd1e27fc700 1 -- 172.21.15.17:0/3305256925 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e40a79c0 con 0x7fd1e40b01a0 2022-08-01T19:41:08.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.091+0000 7fd1e2ffd700 1 --2- 172.21.15.17:0/3305256925 >> v2:172.21.15.77:3300/0 conn(0x7fd1e40b2710 0x7fd1e414afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:08.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.091+0000 7fd1e27fc700 1 --2- 172.21.15.17:0/3305256925 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd1e40b01a0 0x7fd1e414b4c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fd1d40032f0 tx=0x7fd1d40091d0).ready entity=mon.2 client_cookie=274c5969edd3c240 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:08.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.091+0000 7fd1e1ffb700 1 -- 172.21.15.17:0/3305256925 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1d400b600 con 0x7fd1e40b01a0 2022-08-01T19:41:08.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.091+0000 7fd1e1ffb700 1 -- 172.21.15.17:0/3305256925 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1d400b760 con 0x7fd1e40b01a0 2022-08-01T19:41:08.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.091+0000 7fd1e1ffb700 1 -- 172.21.15.17:0/3305256925 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1d400b8d0 con 0x7fd1e40b01a0 2022-08-01T19:41:08.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.091+0000 7fd1e3fff700 1 -- 172.21.15.17:0/3305256925 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd1e40b01a0 msgr2=0x7fd1e414b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:08.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.091+0000 7fd1e3fff700 1 --2- 172.21.15.17:0/3305256925 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd1e40b01a0 0x7fd1e414b4c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fd1d40032f0 tx=0x7fd1d40091d0).stop 2022-08-01T19:41:08.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.091+0000 7fd1e3fff700 1 -- 172.21.15.17:0/3305256925 shutdown_connections 2022-08-01T19:41:08.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.091+0000 7fd1e3fff700 1 --2- 172.21.15.17:0/3305256925 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd1e40b01a0 0x7fd1e414b4c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:08.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.091+0000 7fd1e3fff700 1 --2- 172.21.15.17:0/3305256925 >> v2:172.21.15.77:3300/0 conn(0x7fd1e40b2710 0x7fd1e414afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:08.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.091+0000 7fd1e3fff700 1 -- 172.21.15.17:0/3305256925 >> 172.21.15.17:0/3305256925 conn(0x7fd1e4060310 msgr2=0x7fd1e40b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:08.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.091+0000 7fd1e3fff700 1 -- 172.21.15.17:0/3305256925 shutdown_connections 2022-08-01T19:41:08.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.091+0000 7fd1e3fff700 1 -- 172.21.15.17:0/3305256925 wait complete. 2022-08-01T19:41:08.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.095+0000 7fd1e3fff700 1 Processor -- start 2022-08-01T19:41:08.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.095+0000 7fd1e3fff700 1 -- start start 2022-08-01T19:41:08.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.095+0000 7fd1e3fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd1e40afd60 0x7fd1e414cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:08.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.095+0000 7fd1e3fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1e40b01a0 0x7fd1e40b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:08.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.095+0000 7fd1e3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1e40b2710 0x7fd1e40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:08.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.095+0000 7fd1e3fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e40b5280 con 0x7fd1e40b2710 2022-08-01T19:41:08.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.095+0000 7fd1e3fff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e415a040 con 0x7fd1e40b01a0 2022-08-01T19:41:08.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.095+0000 7fd1e3fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e415a2e0 con 0x7fd1e40afd60 2022-08-01T19:41:08.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.095+0000 7fd1e37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1e40b2710 0x7fd1e40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:08.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.095+0000 7fd1e37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1e40b2710 0x7fd1e40b4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41568/0 (socket says 172.21.15.17:41568) 2022-08-01T19:41:08.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.095+0000 7fd1e37fe700 1 -- 172.21.15.17:0/3044938423 learned_addr learned my addr 172.21.15.17:0/3044938423 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:08.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.095+0000 7fd1e2ffd700 1 --2- 172.21.15.17:0/3044938423 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd1e40afd60 0x7fd1e414cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:08.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.095+0000 7fd1e27fc700 1 --2- 172.21.15.17:0/3044938423 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1e40b01a0 0x7fd1e40b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:08.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.095+0000 7fd1e37fe700 1 -- 172.21.15.17:0/3044938423 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd1e40afd60 msgr2=0x7fd1e414cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:08.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.095+0000 7fd1e37fe700 1 --2- 172.21.15.17:0/3044938423 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd1e40afd60 0x7fd1e414cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:08.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.095+0000 7fd1e37fe700 1 -- 172.21.15.17:0/3044938423 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1e40b01a0 msgr2=0x7fd1e40b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:08.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.095+0000 7fd1e37fe700 1 --2- 172.21.15.17:0/3044938423 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1e40b01a0 0x7fd1e40b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:08.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.095+0000 7fd1e37fe700 1 -- 172.21.15.17:0/3044938423 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1cc00b180 con 0x7fd1e40b2710 2022-08-01T19:41:08.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.099+0000 7fd1e37fe700 1 --2- 172.21.15.17:0/3044938423 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1e40b2710 0x7fd1e40b4d90 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0x7fd1dc001110 tx=0x7fd1dc007a80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:08.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.099+0000 7fd1dbfff700 1 -- 172.21.15.17:0/3044938423 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1dc00ce80 con 0x7fd1e40b2710 2022-08-01T19:41:08.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.099+0000 7fd1dbfff700 1 -- 172.21.15.17:0/3044938423 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1dc0070d0 con 0x7fd1e40b2710 2022-08-01T19:41:08.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.099+0000 7fd1e3fff700 1 -- 172.21.15.17:0/3044938423 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1d4009470 con 0x7fd1e40b2710 2022-08-01T19:41:08.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.099+0000 7fd1dbfff700 1 -- 172.21.15.17:0/3044938423 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1dc007430 con 0x7fd1e40b2710 2022-08-01T19:41:08.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.099+0000 7fd1e3fff700 1 -- 172.21.15.17:0/3044938423 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e414d0e0 con 0x7fd1e40b2710 2022-08-01T19:41:08.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.099+0000 7fd1e3fff700 1 -- 172.21.15.17:0/3044938423 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1e40574e0 con 0x7fd1e40b2710 2022-08-01T19:41:08.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.099+0000 7fd1dbfff700 1 -- 172.21.15.17:0/3044938423 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd1dc010070 con 0x7fd1e40b2710 2022-08-01T19:41:08.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.099+0000 7fd1dbfff700 1 --2- 172.21.15.17:0/3044938423 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd1c8053b00 0x7fd1c8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:08.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.099+0000 7fd1dbfff700 1 -- 172.21.15.17:0/3044938423 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd1dc06b680 con 0x7fd1e40b2710 2022-08-01T19:41:08.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.099+0000 7fd1e2ffd700 1 --2- 172.21.15.17:0/3044938423 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd1c8053b00 0x7fd1c8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:08.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.103+0000 7fd1e2ffd700 1 --2- 172.21.15.17:0/3044938423 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd1c8053b00 0x7fd1c8055fc0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7fd1cc00b000 tx=0x7fd1cc006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:08.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.107+0000 7fd1dbfff700 1 -- 172.21.15.17:0/3044938423 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd1dc0423a0 con 0x7fd1e40b2710 2022-08-01T19:41:08.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.231+0000 7fd1e3fff700 1 -- 172.21.15.17:0/3044938423 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd1e4142860 con 0x7fd1e40b2710 2022-08-01T19:41:08.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.231+0000 7fd1dbfff700 1 -- 172.21.15.17:0/3044938423 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+10 (secure 0 0 0) 0x7fd1dc0459c0 con 0x7fd1e40b2710 2022-08-01T19:41:08.239 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:41:08.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.235+0000 7fd1e3fff700 1 -- 172.21.15.17:0/3044938423 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd1c8053b00 msgr2=0x7fd1c8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:08.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.235+0000 7fd1e3fff700 1 --2- 172.21.15.17:0/3044938423 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd1c8053b00 0x7fd1c8055fc0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7fd1cc00b000 tx=0x7fd1cc006df0).stop 2022-08-01T19:41:08.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.235+0000 7fd1e3fff700 1 -- 172.21.15.17:0/3044938423 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1e40b2710 msgr2=0x7fd1e40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:08.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.235+0000 7fd1e3fff700 1 --2- 172.21.15.17:0/3044938423 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1e40b2710 0x7fd1e40b4d90 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0x7fd1dc001110 tx=0x7fd1dc007a80).stop 2022-08-01T19:41:08.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.235+0000 7fd1e3fff700 1 -- 172.21.15.17:0/3044938423 shutdown_connections 2022-08-01T19:41:08.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.235+0000 7fd1e3fff700 1 --2- 172.21.15.17:0/3044938423 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd1c8053b00 0x7fd1c8055fc0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:08.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.235+0000 7fd1e3fff700 1 --2- 172.21.15.17:0/3044938423 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1e40b2710 0x7fd1e40b4d90 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:08.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.235+0000 7fd1e3fff700 1 --2- 172.21.15.17:0/3044938423 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1e40b01a0 0x7fd1e40b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:08.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.235+0000 7fd1e3fff700 1 --2- 172.21.15.17:0/3044938423 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd1e40afd60 0x7fd1e414cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:08.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.235+0000 7fd1e3fff700 1 -- 172.21.15.17:0/3044938423 >> 172.21.15.17:0/3044938423 conn(0x7fd1e4060310 msgr2=0x7fd1e41498d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:08.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.235+0000 7fd1e3fff700 1 -- 172.21.15.17:0/3044938423 shutdown_connections 2022-08-01T19:41:08.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.235+0000 7fd1e3fff700 1 -- 172.21.15.17:0/3044938423 wait complete. 2022-08-01T19:41:08.257 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T19:41:08.257 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:41:08.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.355+0000 7f54e6564700 1 Processor -- start 2022-08-01T19:41:08.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.355+0000 7f54e6564700 1 -- start start 2022-08-01T19:41:08.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.355+0000 7f54e6564700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f54e00b2710 0x7f54e014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:08.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.355+0000 7f54e6564700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f54e00a8c50 0x7f54e014b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:08.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.355+0000 7f54e6564700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54e00a7250 con 0x7f54e00b01a0 2022-08-01T19:41:08.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.355+0000 7f54e6564700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54e00a7480 con 0x7f54e00b2710 2022-08-01T19:41:08.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.355+0000 7f54e6564700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54e00a7720 con 0x7f54e00a8c50 2022-08-01T19:41:08.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.355+0000 7f54e5d63700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f54e00a8c50 0x7f54e014b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:08.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.355+0000 7f54e5d63700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f54e00a8c50 0x7f54e014b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:37676/0 (socket says 172.21.15.17:37676) 2022-08-01T19:41:08.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.355+0000 7f54e5d63700 1 -- 172.21.15.17:0/3362279760 learned_addr learned my addr 172.21.15.17:0/3362279760 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:08.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.355+0000 7f54e5562700 1 --2- 172.21.15.17:0/3362279760 >> v2:172.21.15.77:3300/0 conn(0x7f54e00b2710 0x7f54e014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:08.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.355+0000 7f54d7fff700 1 -- 172.21.15.17:0/3362279760 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1028430876 0 0) 0x7f54e00a7250 con 0x7f54e00b01a0 2022-08-01T19:41:08.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.355+0000 7f54d7fff700 1 -- 172.21.15.17:0/3362279760 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54c4003260 con 0x7f54e00b01a0 2022-08-01T19:41:08.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.355+0000 7f54e5d63700 1 -- 172.21.15.17:0/3362279760 >> v2:172.21.15.77:3300/0 conn(0x7f54e00b2710 msgr2=0x7f54e014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:08.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.355+0000 7f54e5d63700 1 --2- 172.21.15.17:0/3362279760 >> v2:172.21.15.77:3300/0 conn(0x7f54e00b2710 0x7f54e014afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:08.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.355+0000 7f54e5d63700 1 -- 172.21.15.17:0/3362279760 >> v1:172.21.15.77:6789/0 conn(0x7f54e00b01a0 legacy=0x7f54e00a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:08.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.355+0000 7f54e5d63700 1 -- 172.21.15.17:0/3362279760 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e00a79c0 con 0x7f54e00a8c50 2022-08-01T19:41:08.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.355+0000 7f54e5d63700 1 --2- 172.21.15.17:0/3362279760 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f54e00a8c50 0x7f54e014b4c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f54dc002f70 tx=0x7f54dc006eb0).ready entity=mon.2 client_cookie=7e6835b5dd9d7284 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:08.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.355+0000 7f54d7fff700 1 -- 172.21.15.17:0/3362279760 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54dc00d5c0 con 0x7f54e00a8c50 2022-08-01T19:41:08.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.355+0000 7f54d7fff700 1 -- 172.21.15.17:0/3362279760 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54dc000e90 con 0x7f54e00a8c50 2022-08-01T19:41:08.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.355+0000 7f54d7fff700 1 -- 172.21.15.17:0/3362279760 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54dc00d950 con 0x7f54e00a8c50 2022-08-01T19:41:08.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.355+0000 7f54e6564700 1 -- 172.21.15.17:0/3362279760 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f54e00a8c50 msgr2=0x7f54e014b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:08.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.355+0000 7f54e6564700 1 --2- 172.21.15.17:0/3362279760 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f54e00a8c50 0x7f54e014b4c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f54dc002f70 tx=0x7f54dc006eb0).stop 2022-08-01T19:41:08.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.355+0000 7f54e6564700 1 -- 172.21.15.17:0/3362279760 shutdown_connections 2022-08-01T19:41:08.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.355+0000 7f54e6564700 1 --2- 172.21.15.17:0/3362279760 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f54e00a8c50 0x7f54e014b4c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:08.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.355+0000 7f54e6564700 1 --2- 172.21.15.17:0/3362279760 >> v2:172.21.15.77:3300/0 conn(0x7f54e00b2710 0x7f54e014afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:08.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.359+0000 7f54e6564700 1 -- 172.21.15.17:0/3362279760 >> 172.21.15.17:0/3362279760 conn(0x7f54e0060310 msgr2=0x7f54e00b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:08.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.359+0000 7f54e6564700 1 -- 172.21.15.17:0/3362279760 shutdown_connections 2022-08-01T19:41:08.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.359+0000 7f54e6564700 1 -- 172.21.15.17:0/3362279760 wait complete. 2022-08-01T19:41:08.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.359+0000 7f54e6564700 1 Processor -- start 2022-08-01T19:41:08.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.359+0000 7f54e6564700 1 -- start start 2022-08-01T19:41:08.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.359+0000 7f54e6564700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54e00a8c50 0x7f54e0143e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:08.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.359+0000 7f54e6564700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f54e00b01a0 0x7f54e0159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:08.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.359+0000 7f54e6564700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54e00b2710 0x7f54e015a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:08.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.359+0000 7f54e6564700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e0159b10 con 0x7f54e00a8c50 2022-08-01T19:41:08.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.359+0000 7f54e6564700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e00b6180 con 0x7f54e00b2710 2022-08-01T19:41:08.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.359+0000 7f54e6564700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54e014cf90 con 0x7f54e00b01a0 2022-08-01T19:41:08.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.359+0000 7f54e5562700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54e00a8c50 0x7f54e0143e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:08.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.359+0000 7f54e5562700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54e00a8c50 0x7f54e0143e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41580/0 (socket says 172.21.15.17:41580) 2022-08-01T19:41:08.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.359+0000 7f54e5562700 1 -- 172.21.15.17:0/161923999 learned_addr learned my addr 172.21.15.17:0/161923999 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:08.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.359+0000 7f54e5d63700 1 --2- 172.21.15.17:0/161923999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54e00b2710 0x7f54e015a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:08.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.359+0000 7f54e4d61700 1 --2- 172.21.15.17:0/161923999 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f54e00b01a0 0x7f54e0159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:08.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.359+0000 7f54e5562700 1 -- 172.21.15.17:0/161923999 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f54e00b01a0 msgr2=0x7f54e0159fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:08.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.359+0000 7f54e5562700 1 --2- 172.21.15.17:0/161923999 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f54e00b01a0 0x7f54e0159fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:08.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.359+0000 7f54e5562700 1 -- 172.21.15.17:0/161923999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54e00b2710 msgr2=0x7f54e015a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:08.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.359+0000 7f54e5562700 1 --2- 172.21.15.17:0/161923999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54e00b2710 0x7f54e015a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:08.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.359+0000 7f54e5562700 1 -- 172.21.15.17:0/161923999 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54dc00b1e0 con 0x7f54e00a8c50 2022-08-01T19:41:08.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.359+0000 7f54e5562700 1 --2- 172.21.15.17:0/161923999 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54e00a8c50 0x7f54e0143e00 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 rx=0x7f54d000c000 tx=0x7f54d0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:08.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.359+0000 7f54d67fc700 1 -- 172.21.15.17:0/161923999 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54d0011750 con 0x7f54e00a8c50 2022-08-01T19:41:08.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.359+0000 7f54d67fc700 1 -- 172.21.15.17:0/161923999 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54d00118b0 con 0x7f54e00a8c50 2022-08-01T19:41:08.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.359+0000 7f54e6564700 1 -- 172.21.15.17:0/161923999 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54e014d230 con 0x7f54e00a8c50 2022-08-01T19:41:08.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.359+0000 7f54d67fc700 1 -- 172.21.15.17:0/161923999 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54d0011c10 con 0x7f54e00a8c50 2022-08-01T19:41:08.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.359+0000 7f54e6564700 1 -- 172.21.15.17:0/161923999 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54e014d770 con 0x7f54e00a8c50 2022-08-01T19:41:08.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.363+0000 7f54e6564700 1 -- 172.21.15.17:0/161923999 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54e00b08a0 con 0x7f54e00a8c50 2022-08-01T19:41:08.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.363+0000 7f54d67fc700 1 -- 172.21.15.17:0/161923999 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f54d0011d70 con 0x7f54e00a8c50 2022-08-01T19:41:08.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.363+0000 7f54d67fc700 1 --2- 172.21.15.17:0/161923999 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f54c8053b30 0x7f54c8055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:08.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.363+0000 7f54d67fc700 1 -- 172.21.15.17:0/161923999 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f54d006c3e0 con 0x7f54e00a8c50 2022-08-01T19:41:08.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.363+0000 7f54e4d61700 1 --2- 172.21.15.17:0/161923999 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f54c8053b30 0x7f54c8055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:08.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.363+0000 7f54e4d61700 1 --2- 172.21.15.17:0/161923999 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f54c8053b30 0x7f54c8055ff0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f54d800d000 tx=0x7f54d8007da0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:08.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.371+0000 7f54d67fc700 1 -- 172.21.15.17:0/161923999 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f54d0043140 con 0x7f54e00a8c50 2022-08-01T19:41:08.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.531+0000 7f54e6564700 1 -- 172.21.15.17:0/161923999 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54e00574e0 con 0x7f54e00a8c50 2022-08-01T19:41:08.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.531+0000 7f54d67fc700 1 -- 172.21.15.17:0/161923999 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+2493 (secure 0 0 0) 0x7f54d0046760 con 0x7f54e00a8c50 2022-08-01T19:41:08.538 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:08.538 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":40,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6615,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3580428616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3580428616},{"type":"v1","addr":"172.21.15.17:6833","nonce":3580428616}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38},{"gid":6627,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/4048623373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":4048623373},{"type":"v1","addr":"172.21.15.17:6835","nonce":4048623373}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:41:01.207552+0000","modified":"2022-08-01T19:41:02.210147+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6630},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6630":{"gid":6630,"name":"b","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6835/4275403969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4275403969},{"type":"v1","addr":"172.21.15.77:6835","nonce":4275403969}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2022-08-01T19:41:08.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.535+0000 7f54e6564700 1 -- 172.21.15.17:0/161923999 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f54c8053b30 msgr2=0x7f54c8055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:08.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.535+0000 7f54e6564700 1 --2- 172.21.15.17:0/161923999 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f54c8053b30 0x7f54c8055ff0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f54d800d000 tx=0x7f54d8007da0).stop 2022-08-01T19:41:08.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.535+0000 7f54e6564700 1 -- 172.21.15.17:0/161923999 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54e00a8c50 msgr2=0x7f54e0143e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:08.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.535+0000 7f54e6564700 1 --2- 172.21.15.17:0/161923999 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54e00a8c50 0x7f54e0143e00 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 rx=0x7f54d000c000 tx=0x7f54d0006df0).stop 2022-08-01T19:41:08.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.535+0000 7f54e6564700 1 -- 172.21.15.17:0/161923999 shutdown_connections 2022-08-01T19:41:08.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.535+0000 7f54e6564700 1 --2- 172.21.15.17:0/161923999 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f54c8053b30 0x7f54c8055ff0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:08.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.535+0000 7f54e6564700 1 --2- 172.21.15.17:0/161923999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54e00b2710 0x7f54e015a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:08.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.535+0000 7f54e6564700 1 --2- 172.21.15.17:0/161923999 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f54e00b01a0 0x7f54e0159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:08.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.535+0000 7f54e6564700 1 --2- 172.21.15.17:0/161923999 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54e00a8c50 0x7f54e0143e00 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:08.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.535+0000 7f54e6564700 1 -- 172.21.15.17:0/161923999 >> 172.21.15.17:0/161923999 conn(0x7f54e0060310 msgr2=0x7f54e0149f30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:08.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.535+0000 7f54e6564700 1 -- 172.21.15.17:0/161923999 shutdown_connections 2022-08-01T19:41:08.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:08.535+0000 7f54e6564700 1 -- 172.21.15.17:0/161923999 wait complete. 2022-08-01T19:41:08.544 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 40 2022-08-01T19:41:08.855 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 40, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:41:01.207552+0000', 'modified': '2022-08-01T19:41:02.210147+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6630}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6630': {'gid': 6630, 'name': 'b', 'rank': 0, 'incarnation': 39, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.77:6835/4275403969', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6834', 'nonce': 4275403969}, {'type': 'v1', 'addr': '172.21.15.77:6835', 'nonce': 4275403969}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:41:08.857 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:41:08.858 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-08-01T19:41:08.975 INFO:teuthology.orchestra.run.smithi077.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":6630,"want_state":"up:active","state":"up:active","rank_uptime":7.7556094089999998,"mdsmap_epoch":40,"osdmap_epoch":88,"osdmap_epoch_barrier":87,"uptime":12.589380929000001} 2022-08-01T19:41:08.975 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 6630, "mdsmap_epoch": 40, "osdmap_epoch": 88, "osdmap_epoch_barrier": 87, "rank_uptime": 7.755609409, "state": "up:active", "uptime": 12.589380929, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:41:08.976 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T19:41:08.978 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-08-01T19:41:08.979 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi190.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-08-01T19:41:08.979 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:08.981 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:41:08.991 DEBUG:teuthology.orchestra.run.smithi190:> sudo modprobe fuse 2022-08-01T19:41:09.049 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:41:09.050 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:41:09.064 INFO:teuthology.orchestra.run.smithi190.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:41:09.064 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:41:09.065 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:09.066 DEBUG:teuthology.orchestra.run.smithi190:> ls /sys/fs/fuse/connections 2022-08-01T19:41:09.073 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T19:41:09.074 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:41:09.119 DEBUG:teuthology.orchestra.run.smithi190:> sudo modprobe fuse 2022-08-01T19:41:09.139 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:41:09.140 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:41:09.154 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:41:09.156 INFO:teuthology.orchestra.run.smithi190.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:41:09.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:09.158 DEBUG:teuthology.orchestra.run.smithi190:> ls /sys/fs/fuse/connections 2022-08-01T19:41:09.196 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:2022-08-01T19:41:09.189+0000 7fb9e2957080 -1 init, newargv = 0x55ab1d183940 newargc=9ceph-fuse[22769]: starting ceph client 2022-08-01T19:41:09.197 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr: 2022-08-01T19:41:09.210 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:ceph-fuse[22769]: starting fuse 2022-08-01T19:41:10.308 DEBUG:teuthology.orchestra.run.smithi190:> sudo modprobe fuse 2022-08-01T19:41:10.329 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:41:10.331 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:41:10.392 INFO:teuthology.orchestra.run.smithi190.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:41:10.393 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:41:10.395 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:10.396 DEBUG:teuthology.orchestra.run.smithi190:> ls /sys/fs/fuse/connections 2022-08-01T19:41:10.404 INFO:teuthology.orchestra.run.smithi190.stdout:52 2022-08-01T19:41:10.404 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T19:41:10.405 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:10.405 DEBUG:teuthology.orchestra.run.smithi190:> sudo python3 -c ' 2022-08-01T19:41:10.406 DEBUG:teuthology.orchestra.run.smithi190:> import glob 2022-08-01T19:41:10.406 DEBUG:teuthology.orchestra.run.smithi190:> import re 2022-08-01T19:41:10.410 DEBUG:teuthology.orchestra.run.smithi190:> import os 2022-08-01T19:41:10.411 DEBUG:teuthology.orchestra.run.smithi190:> import subprocess 2022-08-01T19:41:10.411 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:41:10.412 DEBUG:teuthology.orchestra.run.smithi190:> def find_socket(client_name): 2022-08-01T19:41:10.412 DEBUG:teuthology.orchestra.run.smithi190:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-08-01T19:41:10.413 DEBUG:teuthology.orchestra.run.smithi190:> files = glob.glob(asok_path) 2022-08-01T19:41:10.414 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:41:10.415 DEBUG:teuthology.orchestra.run.smithi190:> # Given a non-glob path, it better be there 2022-08-01T19:41:10.417 DEBUG:teuthology.orchestra.run.smithi190:> if "*" not in asok_path: 2022-08-01T19:41:10.418 DEBUG:teuthology.orchestra.run.smithi190:> assert(len(files) == 1) 2022-08-01T19:41:10.419 DEBUG:teuthology.orchestra.run.smithi190:> return files[0] 2022-08-01T19:41:10.421 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:41:10.421 DEBUG:teuthology.orchestra.run.smithi190:> for f in files: 2022-08-01T19:41:10.422 DEBUG:teuthology.orchestra.run.smithi190:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T19:41:10.423 DEBUG:teuthology.orchestra.run.smithi190:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T19:41:10.425 DEBUG:teuthology.orchestra.run.smithi190:> return f 2022-08-01T19:41:10.425 DEBUG:teuthology.orchestra.run.smithi190:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T19:41:10.426 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:41:10.427 DEBUG:teuthology.orchestra.run.smithi190:> print(find_socket("client.1")) 2022-08-01T19:41:10.428 DEBUG:teuthology.orchestra.run.smithi190:> ' 2022-08-01T19:41:10.493 INFO:teuthology.orchestra.run.smithi190.stdout:/var/run/ceph/ceph-client.1.22769.asok 2022-08-01T19:41:10.498 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.22769.asok 2022-08-01T19:41:10.499 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:10.499 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.22769.asok status 2022-08-01T19:41:10.591 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2022-08-01T19:41:10.591 INFO:teuthology.orchestra.run.smithi190.stdout: "metadata": { 2022-08-01T19:41:10.591 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T19:41:10.592 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T19:41:10.592 INFO:teuthology.orchestra.run.smithi190.stdout: "entity_id": "1", 2022-08-01T19:41:10.592 INFO:teuthology.orchestra.run.smithi190.stdout: "hostname": "smithi190", 2022-08-01T19:41:10.592 INFO:teuthology.orchestra.run.smithi190.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-08-01T19:41:10.593 INFO:teuthology.orchestra.run.smithi190.stdout: "pid": "22769", 2022-08-01T19:41:10.593 INFO:teuthology.orchestra.run.smithi190.stdout: "root": "/" 2022-08-01T19:41:10.593 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:41:10.593 INFO:teuthology.orchestra.run.smithi190.stdout: "dentry_count": 0, 2022-08-01T19:41:10.594 INFO:teuthology.orchestra.run.smithi190.stdout: "dentry_pinned_count": 0, 2022-08-01T19:41:10.594 INFO:teuthology.orchestra.run.smithi190.stdout: "id": 6829, 2022-08-01T19:41:10.594 INFO:teuthology.orchestra.run.smithi190.stdout: "inst": { 2022-08-01T19:41:10.594 INFO:teuthology.orchestra.run.smithi190.stdout: "name": { 2022-08-01T19:41:10.595 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "client", 2022-08-01T19:41:10.595 INFO:teuthology.orchestra.run.smithi190.stdout: "num": 6829 2022-08-01T19:41:10.595 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:41:10.595 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": { 2022-08-01T19:41:10.595 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "v1", 2022-08-01T19:41:10.596 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": "172.21.15.190:0", 2022-08-01T19:41:10.596 INFO:teuthology.orchestra.run.smithi190.stdout: "nonce": 1341318127 2022-08-01T19:41:10.596 INFO:teuthology.orchestra.run.smithi190.stdout: } 2022-08-01T19:41:10.597 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:41:10.597 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": { 2022-08-01T19:41:10.597 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "v1", 2022-08-01T19:41:10.597 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": "172.21.15.190:0", 2022-08-01T19:41:10.597 INFO:teuthology.orchestra.run.smithi190.stdout: "nonce": 1341318127 2022-08-01T19:41:10.598 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:41:10.598 INFO:teuthology.orchestra.run.smithi190.stdout: "inst_str": "client.6829 172.21.15.190:0/1341318127", 2022-08-01T19:41:10.598 INFO:teuthology.orchestra.run.smithi190.stdout: "addr_str": "172.21.15.190:0/1341318127", 2022-08-01T19:41:10.599 INFO:teuthology.orchestra.run.smithi190.stdout: "inode_count": 1, 2022-08-01T19:41:10.599 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_epoch": 40, 2022-08-01T19:41:10.599 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_epoch": 88, 2022-08-01T19:41:10.599 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_epoch_barrier": 0, 2022-08-01T19:41:10.600 INFO:teuthology.orchestra.run.smithi190.stdout: "blacklisted": false, 2022-08-01T19:41:10.601 INFO:teuthology.orchestra.run.smithi190.stdout: "fs_name": "cephfs" 2022-08-01T19:41:10.601 INFO:teuthology.orchestra.run.smithi190.stdout:} 2022-08-01T19:41:10.602 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:10.603 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:41:10.603 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:41:10.612 INFO:teuthology.orchestra.run.smithi190.stdout:fuseblk 2022-08-01T19:41:10.618 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T19:41:10.619 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:10.620 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:41:10.685 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:41:10.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.795+0000 7f795bfff700 1 Processor -- start 2022-08-01T19:41:10.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.795+0000 7f795bfff700 1 -- start start 2022-08-01T19:41:10.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.799+0000 7f795bfff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f795c05c960 0x7f795c0b1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:10.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.799+0000 7f795bfff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f795c13f520 0x7f795c0aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:10.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.799+0000 7f795bfff700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f795c05cd40 con 0x7f795c05d240 2022-08-01T19:41:10.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.799+0000 7f795bfff700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f795c0ad6d0 con 0x7f795c13f520 2022-08-01T19:41:10.814 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.799+0000 7f795bfff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f795c067f00 con 0x7f795c05c960 2022-08-01T19:41:10.814 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.799+0000 7f795a7fc700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f795c05d240 0x7f795c0acfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33782/0 (socket says 172.21.15.17:33782) 2022-08-01T19:41:10.814 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.799+0000 7f795a7fc700 1 -- 172.21.15.17:0/1729836361 learned_addr learned my addr 172.21.15.17:0/1729836361 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:10.814 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.799+0000 7f795b7fe700 1 --2- 172.21.15.17:0/1729836361 >> v2:172.21.15.17:3300/0 conn(0x7f795c13f520 0x7f795c0aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:10.815 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.799+0000 7f795affd700 1 --2- 172.21.15.17:0/1729836361 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f795c05c960 0x7f795c0b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:10.815 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.799+0000 7f7959ffb700 1 -- 172.21.15.17:0/1729836361 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1841815437 0 0) 0x7f795c05cd40 con 0x7f795c05d240 2022-08-01T19:41:10.815 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.799+0000 7f7959ffb700 1 -- 172.21.15.17:0/1729836361 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7940003260 con 0x7f795c05d240 2022-08-01T19:41:10.815 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.799+0000 7f795b7fe700 1 -- 172.21.15.17:0/1729836361 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f795c05c960 msgr2=0x7f795c0b1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:10.816 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.799+0000 7f795b7fe700 1 --2- 172.21.15.17:0/1729836361 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f795c05c960 0x7f795c0b1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:10.816 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.799+0000 7f795b7fe700 1 -- 172.21.15.17:0/1729836361 >> v1:172.21.15.17:6789/0 conn(0x7f795c05d240 legacy=0x7f795c0acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:10.816 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.799+0000 7f795b7fe700 1 -- 172.21.15.17:0/1729836361 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f795c0b1920 con 0x7f795c13f520 2022-08-01T19:41:10.816 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.799+0000 7f795b7fe700 1 --2- 172.21.15.17:0/1729836361 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f795c13f520 0x7f795c0aee70 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 rx=0x7f79540032f0 tx=0x7f79540091d0).ready entity=mon.0 client_cookie=1cc40c2222e188a2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:10.817 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.799+0000 7f7959ffb700 1 -- 172.21.15.17:0/1729836361 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f795400b600 con 0x7f795c13f520 2022-08-01T19:41:10.817 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.799+0000 7f7959ffb700 1 -- 172.21.15.17:0/1729836361 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795400b760 con 0x7f795c13f520 2022-08-01T19:41:10.817 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.799+0000 7f7959ffb700 1 -- 172.21.15.17:0/1729836361 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f795400b8d0 con 0x7f795c13f520 2022-08-01T19:41:10.817 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.799+0000 7f795bfff700 1 -- 172.21.15.17:0/1729836361 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f795c13f520 msgr2=0x7f795c0aee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:10.818 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.799+0000 7f795bfff700 1 --2- 172.21.15.17:0/1729836361 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f795c13f520 0x7f795c0aee70 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 rx=0x7f79540032f0 tx=0x7f79540091d0).stop 2022-08-01T19:41:10.818 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.803+0000 7f795bfff700 1 -- 172.21.15.17:0/1729836361 shutdown_connections 2022-08-01T19:41:10.818 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.803+0000 7f795bfff700 1 --2- 172.21.15.17:0/1729836361 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f795c13f520 0x7f795c0aee70 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:10.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.803+0000 7f795bfff700 1 --2- 172.21.15.17:0/1729836361 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f795c05c960 0x7f795c0b1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:10.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.803+0000 7f795bfff700 1 -- 172.21.15.17:0/1729836361 >> 172.21.15.17:0/1729836361 conn(0x7f795c060310 msgr2=0x7f795c0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:10.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.803+0000 7f795bfff700 1 -- 172.21.15.17:0/1729836361 shutdown_connections 2022-08-01T19:41:10.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.803+0000 7f795bfff700 1 -- 172.21.15.17:0/1729836361 wait complete. 2022-08-01T19:41:10.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.803+0000 7f795bfff700 1 Processor -- start 2022-08-01T19:41:10.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.803+0000 7f795bfff700 1 -- start start 2022-08-01T19:41:10.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.807+0000 7f795bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f795c05c960 0x7f795c0ab820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:10.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.807+0000 7f795bfff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f795c05d240 0x7f795c1314c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:10.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.807+0000 7f795bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f795c13f520 0x7f795c1319b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:10.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.807+0000 7f795bfff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795c136360 con 0x7f795c13f520 2022-08-01T19:41:10.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.807+0000 7f795bfff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795c136f40 con 0x7f795c05c960 2022-08-01T19:41:10.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.807+0000 7f795bfff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f795c1371e0 con 0x7f795c05d240 2022-08-01T19:41:10.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.807+0000 7f795b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f795c13f520 0x7f795c1319b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:10.822 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.807+0000 7f795affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f795c05c960 0x7f795c0ab820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:10.822 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.807+0000 7f795b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f795c13f520 0x7f795c1319b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41598/0 (socket says 172.21.15.17:41598) 2022-08-01T19:41:10.822 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.807+0000 7f795a7fc700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f795c05d240 0x7f795c1314c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:10.822 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.807+0000 7f795b7fe700 1 -- 172.21.15.17:0/3358683716 learned_addr learned my addr 172.21.15.17:0/3358683716 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:10.822 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.807+0000 7f795b7fe700 1 -- 172.21.15.17:0/3358683716 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f795c05d240 msgr2=0x7f795c1314c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:10.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.807+0000 7f795b7fe700 1 --2- 172.21.15.17:0/3358683716 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f795c05d240 0x7f795c1314c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:10.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.807+0000 7f795b7fe700 1 -- 172.21.15.17:0/3358683716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f795c05c960 msgr2=0x7f795c0ab820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:10.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.807+0000 7f795b7fe700 1 --2- 172.21.15.17:0/3358683716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f795c05c960 0x7f795c0ab820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:10.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.807+0000 7f795b7fe700 1 -- 172.21.15.17:0/3358683716 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7954009470 con 0x7f795c13f520 2022-08-01T19:41:10.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.807+0000 7f795b7fe700 1 --2- 172.21.15.17:0/3358683716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f795c13f520 0x7f795c1319b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7f79540032f0 tx=0x7f7954016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:10.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.807+0000 7f794bfff700 1 -- 172.21.15.17:0/3358683716 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f795400b8d0 con 0x7f795c13f520 2022-08-01T19:41:10.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.807+0000 7f794bfff700 1 -- 172.21.15.17:0/3358683716 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79540205f0 con 0x7f795c13f520 2022-08-01T19:41:10.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.807+0000 7f795bfff700 1 -- 172.21.15.17:0/3358683716 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f795c1436d0 con 0x7f795c13f520 2022-08-01T19:41:10.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.807+0000 7f794bfff700 1 -- 172.21.15.17:0/3358683716 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f795400b600 con 0x7f795c13f520 2022-08-01T19:41:10.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.807+0000 7f795bfff700 1 -- 172.21.15.17:0/3358683716 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f795c136900 con 0x7f795c13f520 2022-08-01T19:41:10.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.807+0000 7f795bfff700 1 -- 172.21.15.17:0/3358683716 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f795c1364d0 con 0x7f795c13f520 2022-08-01T19:41:10.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.811+0000 7f794bfff700 1 -- 172.21.15.17:0/3358683716 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7954027030 con 0x7f795c13f520 2022-08-01T19:41:10.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.811+0000 7f794bfff700 1 --2- 172.21.15.17:0/3358683716 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7938053a70 0x7f7938055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:10.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.811+0000 7f794bfff700 1 -- 172.21.15.17:0/3358683716 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f795406d1c0 con 0x7f795c13f520 2022-08-01T19:41:10.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.811+0000 7f795affd700 1 --2- 172.21.15.17:0/3358683716 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7938053a70 0x7f7938055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:10.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.811+0000 7f795affd700 1 --2- 172.21.15.17:0/3358683716 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7938053a70 0x7f7938055f30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f795000b000 tx=0x7f7950006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:10.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.819+0000 7f794bfff700 1 -- 172.21.15.17:0/3358683716 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7954043f20 con 0x7f795c13f520 2022-08-01T19:41:10.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.991+0000 7f795bfff700 1 -- 172.21.15.17:0/3358683716 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f795c1373b0 con 0x7f795c13f520 2022-08-01T19:41:10.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.991+0000 7f794bfff700 1 -- 172.21.15.17:0/3358683716 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+2493 (secure 0 0 0) 0x7f7954025020 con 0x7f795c13f520 2022-08-01T19:41:11.000 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:11.001 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":40,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6615,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3580428616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3580428616},{"type":"v1","addr":"172.21.15.17:6833","nonce":3580428616}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38},{"gid":6627,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/4048623373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":4048623373},{"type":"v1","addr":"172.21.15.17:6835","nonce":4048623373}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:41:01.207552+0000","modified":"2022-08-01T19:41:02.210147+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6630},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6630":{"gid":6630,"name":"b","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6835/4275403969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4275403969},{"type":"v1","addr":"172.21.15.77:6835","nonce":4275403969}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2022-08-01T19:41:11.003 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.995+0000 7f795bfff700 1 -- 172.21.15.17:0/3358683716 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7938053a70 msgr2=0x7f7938055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:11.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.995+0000 7f795bfff700 1 --2- 172.21.15.17:0/3358683716 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7938053a70 0x7f7938055f30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f795000b000 tx=0x7f7950006df0).stop 2022-08-01T19:41:11.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.995+0000 7f795bfff700 1 -- 172.21.15.17:0/3358683716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f795c13f520 msgr2=0x7f795c1319b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:11.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.995+0000 7f795bfff700 1 --2- 172.21.15.17:0/3358683716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f795c13f520 0x7f795c1319b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7f79540032f0 tx=0x7f7954016000).stop 2022-08-01T19:41:11.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.995+0000 7f795bfff700 1 -- 172.21.15.17:0/3358683716 shutdown_connections 2022-08-01T19:41:11.014 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.995+0000 7f795bfff700 1 --2- 172.21.15.17:0/3358683716 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7938053a70 0x7f7938055f30 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:11.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.995+0000 7f795bfff700 1 --2- 172.21.15.17:0/3358683716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f795c13f520 0x7f795c1319b0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:11.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.995+0000 7f795bfff700 1 --2- 172.21.15.17:0/3358683716 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f795c05d240 0x7f795c1314c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:11.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.995+0000 7f795bfff700 1 --2- 172.21.15.17:0/3358683716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f795c05c960 0x7f795c0ab820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:11.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.995+0000 7f795bfff700 1 -- 172.21.15.17:0/3358683716 >> 172.21.15.17:0/3358683716 conn(0x7f795c060310 msgr2=0x7f795c0b4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:11.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.995+0000 7f795bfff700 1 -- 172.21.15.17:0/3358683716 shutdown_connections 2022-08-01T19:41:11.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:10.995+0000 7f795bfff700 1 -- 172.21.15.17:0/3358683716 wait complete. 2022-08-01T19:41:11.016 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 40 2022-08-01T19:41:11.018 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json dump inode 1 2022-08-01T19:41:11.133 INFO:teuthology.orchestra.run.smithi077.stdout:{"path":"/","ino":1,"rdev":0,"ctime":"2022-08-01T19:41:06.866378+0000","btime":"2022-08-01T19:41:01.211474+0000","mode":17407,"uid":0,"gid":0,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":4194304,"stripe_count":1,"object_size":4194304,"pool_id":16,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":0,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2022-08-01T19:41:01.211474+0000","atime":"0.000000","time_warp_seq":0,"change_attr":1,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"0.000000","num_files":0,"num_subdirs":0,"change_attr":0},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"2022-08-01T19:41:06.866378+0000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"2022-08-01T19:41:01.211474+0000"},"version":3,"file_data_version":0,"xattr_version":1,"backtrace_version":3,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":2,"is_frozen":false,"is_freezing":false,"pins":{"request":1,"lock":3,"dirfrag":1,"caps":1,"dirtyparent":1,"dirty":1,"authpin":1},"nref":9,"versionlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":1,"num_xlocks":0,"xlock_by":{}},"authlock":{"gather_set":[],"state":"xlockdone","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":1,"xlock_by":{}},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{"gather_set":[],"state":"sync","is_leased":false,"num_rdlocks":1,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"nestlock":{},"flocklock":{},"policylock":{},"states":["auth","dirty","dirtyparent"],"client_caps":[{"client_id":6784,"pending":"pLsXsFs","issued":"pLsXsFs","wanted":"-","last_sent":2},{"client_id":6829,"pending":"pLsXsFs","issued":"pLsXsFs","wanted":"-","last_sent":2}],"loner":-1,"want_loner":-1,"mds_caps_wanted":[]} 2022-08-01T19:41:11.135 DEBUG:tasks.cephfs.filesystem:_json_asok output { "accounted_rstat": { "rbytes": 0, "rctime": "2022-08-01T19:41:01.211474+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "0.000000", "auth_pins": 2, "auth_state": { "replicas": {} }, "authlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 1, "state": "xlockdone", "xlock_by": {} }, "backtrace_version": 3, "btime": "2022-08-01T19:41:01.211474+0000", "change_attr": 1, "client_caps": [ { "client_id": 6784, "issued": "pLsXsFs", "last_sent": 2, "pending": "pLsXsFs", "wanted": "-" }, { "client_id": 6829, "issued": "pLsXsFs", "last_sent": 2, "pending": "pLsXsFs", "wanted": "-" } ], "client_ranges": [], "ctime": "2022-08-01T19:41:06.866378+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 0, "mtime": "0.000000", "num_files": 0, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 0, "ino": 1, "is_auth": true, "is_freezing": false, "is_frozen": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 4194304, "pool_id": 16, "pool_ns": "", "stripe_count": 1, "stripe_unit": 4194304 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 17407, "mtime": "2022-08-01T19:41:01.211474+0000", "nestlock": {}, "nlink": 1, "nref": 9, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "path": "/", "pins": { "authpin": 1, "caps": 1, "dirfrag": 1, "dirty": 1, "dirtyparent": 1, "lock": 3, "request": 1 }, "policylock": {}, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "2022-08-01T19:41:06.866378+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": { "gather_set": [], "is_leased": false, "num_rdlocks": 1, "num_wrlocks": 0, "num_xlocks": 0, "state": "sync", "xlock_by": {} }, "states": [ "auth", "dirty", "dirtyparent" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 0, "truncate_size": 18446744073709551615, "uid": 0, "version": 3, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 1, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [] } 2022-08-01T19:41:11.135 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T19:41:11.136 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:41:11.158 DEBUG:teuthology.orchestra.run.smithi077:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:41:11.177 DEBUG:teuthology.orchestra.run.smithi085:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:41:11.205 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:41:11.228 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T19:41:11.229 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:11.232 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:41:11.232 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:41:11.241 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:41:11.243 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:41:11.244 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi085.front.sepia.ceph.com... 2022-08-01T19:41:11.244 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T19:41:11.245 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:41:11.311 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:ceph-fuse[23193]: fuse finished with error 0 and tester_r 0 2022-08-01T19:41:11.313 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:11.313 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:41:11.314 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:41:11.324 INFO:teuthology.orchestra.run.smithi085.stdout:ext2/ext3 2022-08-01T19:41:11.325 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T19:41:11.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T19:41:11.362 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:41:11.413 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:11.414 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:41:11.415 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:41:11.425 INFO:teuthology.orchestra.run.smithi190.stdout:fuseblk 2022-08-01T19:41:11.426 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T19:41:11.426 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi190.front.sepia.ceph.com... 2022-08-01T19:41:11.427 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T19:41:11.427 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:41:11.443 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:ceph-fuse[22769]: fuse finished with error 0 and tester_r 0 2022-08-01T19:41:11.446 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:11.447 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:41:11.447 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:41:11.496 INFO:teuthology.orchestra.run.smithi190.stdout:ext2/ext3 2022-08-01T19:41:11.497 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T19:41:11.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T19:41:11.593 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:41:11.612 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T19:41:11.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.723+0000 7f6a63fff700 1 Processor -- start 2022-08-01T19:41:11.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.723+0000 7f6a63fff700 1 -- start start 2022-08-01T19:41:11.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.723+0000 7f6a63fff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f6a640b08a0 0x7f6a64142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:11.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.723+0000 7f6a63fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6a640b3260 0x7f6a6405c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:11.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.723+0000 7f6a63fff700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a64143ab0 con 0x7f6a640ad000 2022-08-01T19:41:11.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.723+0000 7f6a63fff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a640b0c80 con 0x7f6a640b08a0 2022-08-01T19:41:11.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.723+0000 7f6a63fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a640ae810 con 0x7f6a640b3260 2022-08-01T19:41:11.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.723+0000 7f6a627fc700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6a640b3260 0x7f6a6405c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:11.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.727+0000 7f6a627fc700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6a640b3260 0x7f6a6405c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:37702/0 (socket says 172.21.15.17:37702) 2022-08-01T19:41:11.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.727+0000 7f6a627fc700 1 -- 172.21.15.17:0/3730442824 learned_addr learned my addr 172.21.15.17:0/3730442824 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:11.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.727+0000 7f6a62ffd700 1 --2- 172.21.15.17:0/3730442824 >> v2:172.21.15.77:3300/0 conn(0x7f6a640b08a0 0x7f6a64142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:11.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.727+0000 7f6a627fc700 1 -- 172.21.15.17:0/3730442824 >> v2:172.21.15.77:3300/0 conn(0x7f6a640b08a0 msgr2=0x7f6a64142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:11.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.727+0000 7f6a627fc700 1 --2- 172.21.15.17:0/3730442824 >> v2:172.21.15.77:3300/0 conn(0x7f6a640b08a0 0x7f6a64142eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:11.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.727+0000 7f6a627fc700 1 -- 172.21.15.17:0/3730442824 >> v1:172.21.15.77:6789/0 conn(0x7f6a640ad000 legacy=0x7f6a641433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:11.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.727+0000 7f6a627fc700 1 -- 172.21.15.17:0/3730442824 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a640aeab0 con 0x7f6a640b3260 2022-08-01T19:41:11.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.727+0000 7f6a61ffb700 1 -- 172.21.15.17:0/3730442824 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 641137884 0 0) 0x7f6a64143ab0 con 0x7f6a640ad000 2022-08-01T19:41:11.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.727+0000 7f6a627fc700 1 --2- 172.21.15.17:0/3730442824 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6a640b3260 0x7f6a6405c620 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f6a54002f70 tx=0x7f6a54006f40).ready entity=mon.2 client_cookie=f41d64d99b1e3fe8 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:11.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.727+0000 7f6a61ffb700 1 -- 172.21.15.17:0/3730442824 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a5400da80 con 0x7f6a640b3260 2022-08-01T19:41:11.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.727+0000 7f6a61ffb700 1 -- 172.21.15.17:0/3730442824 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a54000e90 con 0x7f6a640b3260 2022-08-01T19:41:11.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.727+0000 7f6a61ffb700 1 -- 172.21.15.17:0/3730442824 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a5400ddd0 con 0x7f6a640b3260 2022-08-01T19:41:11.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.727+0000 7f6a63fff700 1 -- 172.21.15.17:0/3730442824 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6a640b3260 msgr2=0x7f6a6405c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:11.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.727+0000 7f6a63fff700 1 --2- 172.21.15.17:0/3730442824 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6a640b3260 0x7f6a6405c620 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f6a54002f70 tx=0x7f6a54006f40).stop 2022-08-01T19:41:11.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.727+0000 7f6a63fff700 1 -- 172.21.15.17:0/3730442824 shutdown_connections 2022-08-01T19:41:11.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.727+0000 7f6a63fff700 1 --2- 172.21.15.17:0/3730442824 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6a640b3260 0x7f6a6405c620 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:11.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.727+0000 7f6a63fff700 1 --2- 172.21.15.17:0/3730442824 >> v2:172.21.15.77:3300/0 conn(0x7f6a640b08a0 0x7f6a64142eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:11.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.727+0000 7f6a63fff700 1 -- 172.21.15.17:0/3730442824 >> 172.21.15.17:0/3730442824 conn(0x7f6a640612d0 msgr2=0x7f6a64062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:11.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.727+0000 7f6a63fff700 1 -- 172.21.15.17:0/3730442824 shutdown_connections 2022-08-01T19:41:11.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.727+0000 7f6a63fff700 1 -- 172.21.15.17:0/3730442824 wait complete. 2022-08-01T19:41:11.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.727+0000 7f6a63fff700 1 Processor -- start 2022-08-01T19:41:11.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.727+0000 7f6a63fff700 1 -- start start 2022-08-01T19:41:11.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.731+0000 7f6a63fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a640ad000 0x7f6a64142fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:11.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.731+0000 7f6a63fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a640b08a0 0x7f6a640a2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:11.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.731+0000 7f6a63fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6a640b3260 0x7f6a64152d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:11.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.731+0000 7f6a63fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a64063c00 con 0x7f6a640ad000 2022-08-01T19:41:11.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.731+0000 7f6a63fff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a64064290 con 0x7f6a640b08a0 2022-08-01T19:41:11.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.731+0000 7f6a63fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a641434c0 con 0x7f6a640b3260 2022-08-01T19:41:11.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.731+0000 7f6a62ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a640ad000 0x7f6a64142fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:11.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.731+0000 7f6a62ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a640ad000 0x7f6a64142fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41608/0 (socket says 172.21.15.17:41608) 2022-08-01T19:41:11.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.731+0000 7f6a62ffd700 1 -- 172.21.15.17:0/2963946462 learned_addr learned my addr 172.21.15.17:0/2963946462 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:11.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.731+0000 7f6a627fc700 1 --2- 172.21.15.17:0/2963946462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a640b08a0 0x7f6a640a2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:11.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.731+0000 7f6a637fe700 1 --2- 172.21.15.17:0/2963946462 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6a640b3260 0x7f6a64152d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:11.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.731+0000 7f6a62ffd700 1 -- 172.21.15.17:0/2963946462 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6a640b3260 msgr2=0x7f6a64152d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:11.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.731+0000 7f6a62ffd700 1 --2- 172.21.15.17:0/2963946462 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6a640b3260 0x7f6a64152d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:11.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.731+0000 7f6a62ffd700 1 -- 172.21.15.17:0/2963946462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a640b08a0 msgr2=0x7f6a640a2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:11.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.731+0000 7f6a62ffd700 1 --2- 172.21.15.17:0/2963946462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a640b08a0 0x7f6a640a2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:11.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.731+0000 7f6a62ffd700 1 -- 172.21.15.17:0/2963946462 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5400b460 con 0x7f6a640ad000 2022-08-01T19:41:11.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.731+0000 7f6a62ffd700 1 --2- 172.21.15.17:0/2963946462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a640ad000 0x7f6a64142fd0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 rx=0x7f6a4c00c000 tx=0x7f6a4c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:11.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.731+0000 7f6a5bfff700 1 -- 172.21.15.17:0/2963946462 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a4c010780 con 0x7f6a640ad000 2022-08-01T19:41:11.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.731+0000 7f6a5bfff700 1 -- 172.21.15.17:0/2963946462 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4c0108e0 con 0x7f6a640ad000 2022-08-01T19:41:11.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.731+0000 7f6a63fff700 1 -- 172.21.15.17:0/2963946462 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a641567e0 con 0x7f6a640ad000 2022-08-01T19:41:11.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.731+0000 7f6a5bfff700 1 -- 172.21.15.17:0/2963946462 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a4c010c40 con 0x7f6a640ad000 2022-08-01T19:41:11.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.731+0000 7f6a63fff700 1 -- 172.21.15.17:0/2963946462 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a64063df0 con 0x7f6a640ad000 2022-08-01T19:41:11.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.735+0000 7f6a63fff700 1 -- 172.21.15.17:0/2963946462 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a64152900 con 0x7f6a640ad000 2022-08-01T19:41:11.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.735+0000 7f6a5bfff700 1 -- 172.21.15.17:0/2963946462 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6a4c01c460 con 0x7f6a640ad000 2022-08-01T19:41:11.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.735+0000 7f6a5bfff700 1 --2- 172.21.15.17:0/2963946462 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6a48053950 0x7f6a48055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:11.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.735+0000 7f6a5bfff700 1 -- 172.21.15.17:0/2963946462 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6a4c06c410 con 0x7f6a640ad000 2022-08-01T19:41:11.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.735+0000 7f6a627fc700 1 --2- 172.21.15.17:0/2963946462 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6a48053950 0x7f6a48055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:11.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.735+0000 7f6a627fc700 1 --2- 172.21.15.17:0/2963946462 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6a48053950 0x7f6a48055e10 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f6a5401b000 tx=0x7f6a640a95b0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:11.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.739+0000 7f6a5bfff700 1 -- 172.21.15.17:0/2963946462 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6a4c043170 con 0x7f6a640ad000 2022-08-01T19:41:11.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.871+0000 7f6a63fff700 1 -- 172.21.15.17:0/2963946462 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f6a640574e0 con 0x7f6a640ad000 2022-08-01T19:41:11.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.871+0000 7f6a5bfff700 1 -- 172.21.15.17:0/2963946462 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v88) v1 ==== 81+0+34040 (secure 0 0 0) 0x7f6a4c046790 con 0x7f6a640ad000 2022-08-01T19:41:11.879 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:11.879 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:41:11.880 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 88, 2022-08-01T19:41:11.880 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", 2022-08-01T19:41:11.880 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2022-08-01T19:33:50.105743+0000", 2022-08-01T19:41:11.880 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2022-08-01T19:41:02.210932+0000", 2022-08-01T19:41:11.881 INFO:teuthology.orchestra.run.smithi017.stdout: "last_up_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:41:11.881 INFO:teuthology.orchestra.run.smithi017.stdout: "last_in_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:41:11.881 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T19:41:11.881 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_num": 5799936, 2022-08-01T19:41:11.882 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_set": [ 2022-08-01T19:41:11.882 INFO:teuthology.orchestra.run.smithi017.stdout: "pglog_hardlimit", 2022-08-01T19:41:11.882 INFO:teuthology.orchestra.run.smithi017.stdout: "purged_snapdirs", 2022-08-01T19:41:11.882 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_deletes", 2022-08-01T19:41:11.883 INFO:teuthology.orchestra.run.smithi017.stdout: "sortbitwise" 2022-08-01T19:41:11.884 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:41:11.885 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_version": 6, 2022-08-01T19:41:11.885 INFO:teuthology.orchestra.run.smithi017.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T19:41:11.885 INFO:teuthology.orchestra.run.smithi017.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T19:41:11.885 INFO:teuthology.orchestra.run.smithi017.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T19:41:11.886 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_snapshot": "", 2022-08-01T19:41:11.886 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_max": 16, 2022-08-01T19:41:11.886 INFO:teuthology.orchestra.run.smithi017.stdout: "max_osd": 8, 2022-08-01T19:41:11.886 INFO:teuthology.orchestra.run.smithi017.stdout: "require_min_compat_client": "jewel", 2022-08-01T19:41:11.887 INFO:teuthology.orchestra.run.smithi017.stdout: "min_compat_client": "jewel", 2022-08-01T19:41:11.887 INFO:teuthology.orchestra.run.smithi017.stdout: "require_osd_release": "octopus", 2022-08-01T19:41:11.887 INFO:teuthology.orchestra.run.smithi017.stdout: "pools": [ 2022-08-01T19:41:11.888 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:11.888 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 1, 2022-08-01T19:41:11.888 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "device_health_metrics", 2022-08-01T19:41:11.888 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:33:53.001841+0000", 2022-08-01T19:41:11.889 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:41:11.889 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:41:11.889 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:41:11.889 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:41:11.889 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:41:11.890 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:41:11.890 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:41:11.891 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:41:11.891 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 1, 2022-08-01T19:41:11.891 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 1, 2022-08-01T19:41:11.891 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 1, 2022-08-01T19:41:11.892 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 1, 2022-08-01T19:41:11.892 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 1, 2022-08-01T19:41:11.892 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:41:11.892 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:41:11.893 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:41:11.893 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:41:11.893 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:41:11.893 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:41:11.893 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:41:11.894 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:11.894 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "16", 2022-08-01T19:41:11.894 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:41:11.894 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:41:11.895 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:41:11.895 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:41:11.895 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:41:11.895 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:41:11.896 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:41:11.896 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:41:11.896 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:41:11.896 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:41:11.896 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:41:11.897 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:41:11.897 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:41:11.897 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:41:11.897 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:41:11.898 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:41:11.898 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:41:11.898 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:41:11.898 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:41:11.899 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:41:11.899 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:41:11.899 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:41:11.899 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:41:11.900 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:41:11.900 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:41:11.900 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:41:11.900 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:11.900 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:41:11.901 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:41:11.901 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:41:11.901 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:41:11.901 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:41:11.902 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:41:11.902 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:41:11.902 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:41:11.902 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:41:11.903 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:41:11.903 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:41:11.903 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:41:11.903 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 1 2022-08-01T19:41:11.904 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:11.904 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:41:11.904 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_devicehealth": {} 2022-08-01T19:41:11.904 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:11.904 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:11.905 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:11.905 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 2, 2022-08-01T19:41:11.905 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "rbd", 2022-08-01T19:41:11.905 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:34:01.553098+0000", 2022-08-01T19:41:11.906 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:41:11.906 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:41:11.906 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:41:11.906 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:41:11.907 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:41:11.907 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:41:11.907 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:41:11.907 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:41:11.908 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:41:11.908 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:41:11.908 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:41:11.908 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:41:11.909 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:41:11.909 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:41:11.909 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:41:11.909 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:41:11.909 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:41:11.910 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:41:11.910 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:41:11.910 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:41:11.910 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:11.911 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "19", 2022-08-01T19:41:11.911 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:41:11.911 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:41:11.911 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:41:11.912 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:41:11.912 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:41:11.912 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:41:11.913 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:41:11.913 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:41:11.913 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:41:11.913 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:41:11.913 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:41:11.914 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:41:11.914 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:41:11.915 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:41:11.915 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:41:11.915 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:41:11.915 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:41:11.916 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:41:11.916 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:41:11.916 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:41:11.916 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:41:11.916 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:41:11.917 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:41:11.917 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:41:11.917 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:41:11.917 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:41:11.917 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:11.918 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:41:11.918 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:41:11.918 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:41:11.918 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:41:11.919 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:41:11.919 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:41:11.919 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:41:11.919 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:41:11.920 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:41:11.920 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:41:11.920 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:41:11.920 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:41:11.921 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:41:11.921 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd": {} 2022-08-01T19:41:11.921 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:11.921 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:11.921 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:11.922 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 15, 2022-08-01T19:41:11.922 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "cephfs_metadata", 2022-08-01T19:41:11.922 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:40:59.229815+0000", 2022-08-01T19:41:11.922 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:41:11.923 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:41:11.923 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:41:11.923 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:41:11.923 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:41:11.924 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:41:11.924 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:41:11.924 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:41:11.924 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:41:11.925 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:41:11.925 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:41:11.925 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:41:11.925 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:41:11.926 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:41:11.926 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:41:11.926 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:41:11.926 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:41:11.927 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:41:11.927 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:41:11.927 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:41:11.927 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:11.928 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "87", 2022-08-01T19:41:11.928 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:41:11.928 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:41:11.928 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:41:11.928 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:41:11.929 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:41:11.929 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:41:11.929 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:41:11.929 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:41:11.930 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:41:11.930 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:41:11.930 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:41:11.931 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:41:11.931 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:41:11.931 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:41:11.931 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:41:11.931 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:41:11.932 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:41:11.932 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:41:11.932 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:41:11.932 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:41:11.933 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:41:11.933 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:41:11.933 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:41:11.933 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:41:11.933 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:41:11.934 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:41:11.934 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:11.934 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:41:11.934 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:41:11.935 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:41:11.935 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:41:11.935 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:41:11.935 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:41:11.935 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:41:11.936 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:41:11.936 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:41:11.936 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:41:11.936 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:41:11.937 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:41:11.937 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_bias": 4, 2022-08-01T19:41:11.937 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 16, 2022-08-01T19:41:11.937 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_priority": 5 2022-08-01T19:41:11.938 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:11.938 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:41:11.938 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs": { 2022-08-01T19:41:11.938 INFO:teuthology.orchestra.run.smithi017.stdout: "metadata": "cephfs" 2022-08-01T19:41:11.938 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:11.939 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:11.939 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:11.939 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:11.939 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 16, 2022-08-01T19:41:11.940 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "cephfs_data", 2022-08-01T19:41:11.940 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:41:00.225322+0000", 2022-08-01T19:41:11.940 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:41:11.940 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:41:11.941 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:41:11.941 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:41:11.941 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:41:11.941 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:41:11.941 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:41:11.942 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:41:11.942 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:41:11.942 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:41:11.942 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:41:11.942 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:41:11.943 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:41:11.943 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:41:11.943 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:41:11.943 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:41:11.944 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:41:11.944 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:41:11.944 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:41:11.944 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:41:11.945 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:11.945 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "88", 2022-08-01T19:41:11.945 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:41:11.945 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:41:11.945 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:41:11.946 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:41:11.946 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:41:11.946 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:41:11.946 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:41:11.946 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:41:11.947 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:41:11.947 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:41:11.948 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:41:11.948 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:41:11.948 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:41:11.948 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:41:11.949 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:41:11.949 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:41:11.949 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:41:11.949 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:41:11.950 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:41:11.950 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:41:11.950 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:41:11.950 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:41:11.951 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:41:11.951 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:41:11.951 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:41:11.951 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:41:11.952 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:11.952 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:41:11.952 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:41:11.952 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:41:11.953 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:41:11.953 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:41:11.953 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:41:11.953 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:41:11.954 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:41:11.954 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:41:11.954 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:41:11.954 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:41:11.954 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:41:11.955 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:41:11.955 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs": { 2022-08-01T19:41:11.955 INFO:teuthology.orchestra.run.smithi017.stdout: "data": "cephfs" 2022-08-01T19:41:11.955 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:11.956 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:11.956 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:11.956 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:41:11.956 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2022-08-01T19:41:11.956 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:11.957 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:41:11.957 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a962f14f-cf16-4fa8-84ff-c85325db9132", 2022-08-01T19:41:11.957 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:11.957 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:11.957 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:11.958 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:11.958 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:11.958 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:11.958 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:41:11.959 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 86, 2022-08-01T19:41:11.959 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:11.959 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:11.959 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:11.960 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:11.960 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:11.960 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:11.960 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6800", 2022-08-01T19:41:11.960 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:11.961 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:11.961 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:11.961 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:11.961 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6801", 2022-08-01T19:41:11.961 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:11.962 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:11.962 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:11.962 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:11.962 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:11.963 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:11.963 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:11.963 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:11.963 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6802", 2022-08-01T19:41:11.963 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:11.964 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:11.964 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:11.964 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:11.964 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6803", 2022-08-01T19:41:11.964 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:11.965 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:11.965 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:11.965 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:11.965 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:11.966 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:11.966 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:11.966 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:11.966 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6806", 2022-08-01T19:41:11.966 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:11.967 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:11.967 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:11.967 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:11.967 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6807", 2022-08-01T19:41:11.968 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:11.968 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:11.968 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:11.968 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:11.968 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:11.969 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:11.969 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:11.969 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:11.969 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6804", 2022-08-01T19:41:11.970 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:11.970 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:11.970 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:11.970 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:11.970 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6805", 2022-08-01T19:41:11.971 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:11.971 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:11.971 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:11.971 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:11.972 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6801/23008", 2022-08-01T19:41:11.972 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6803/23008", 2022-08-01T19:41:11.972 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6807/23008", 2022-08-01T19:41:11.972 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6805/23008", 2022-08-01T19:41:11.972 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:11.973 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:11.973 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:11.973 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:11.973 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:11.973 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:11.974 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:41:11.974 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "805f7051-a527-4e98-8844-b6e406a63248", 2022-08-01T19:41:11.974 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:11.974 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:11.975 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:11.975 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:11.975 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:11.975 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:11.975 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:41:11.976 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 85, 2022-08-01T19:41:11.976 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:11.976 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:11.976 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:11.977 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:11.977 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:11.977 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:11.978 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6808", 2022-08-01T19:41:11.978 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:11.978 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:11.979 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:11.979 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:11.979 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6809", 2022-08-01T19:41:11.980 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:11.980 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:11.981 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:11.981 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:11.981 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:11.982 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:11.982 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:11.982 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:11.983 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6810", 2022-08-01T19:41:11.983 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:11.983 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:11.983 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:11.983 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:11.985 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6811", 2022-08-01T19:41:11.986 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:11.986 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:11.987 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:11.987 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:11.987 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:11.988 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:11.988 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:11.989 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:11.989 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6814", 2022-08-01T19:41:11.990 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:11.991 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:11.991 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:11.992 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:11.992 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6815", 2022-08-01T19:41:11.993 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:11.993 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:11.994 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:11.994 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:11.995 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:11.995 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:11.996 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:11.996 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:11.997 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6812", 2022-08-01T19:41:11.997 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:11.998 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:11.998 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:11.999 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:11.999 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6813", 2022-08-01T19:41:12.000 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:12.000 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:12.001 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:12.001 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.002 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6809/23007", 2022-08-01T19:41:12.002 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6811/23007", 2022-08-01T19:41:12.002 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6815/23007", 2022-08-01T19:41:12.003 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6813/23007", 2022-08-01T19:41:12.003 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:12.004 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:12.004 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:12.005 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:12.005 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.006 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.006 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:41:12.007 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "89f7f3df-7c10-48e1-8b69-9e4243923a61", 2022-08-01T19:41:12.007 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:12.008 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:12.008 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:12.009 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:12.009 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:12.010 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:12.010 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:41:12.011 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 86, 2022-08-01T19:41:12.011 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:12.012 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:12.012 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:12.013 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:12.013 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.013 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:12.014 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6817", 2022-08-01T19:41:12.014 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:12.015 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.015 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.016 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:12.016 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6819", 2022-08-01T19:41:12.017 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:12.017 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:12.018 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:12.018 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.019 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:12.019 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:12.020 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.020 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:12.021 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6822", 2022-08-01T19:41:12.021 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:12.022 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.023 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.023 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:12.024 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6824", 2022-08-01T19:41:12.024 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:12.025 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:12.025 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:12.025 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.026 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:12.026 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:12.027 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.027 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:12.028 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6830", 2022-08-01T19:41:12.028 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:12.029 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.029 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.029 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:12.030 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6831", 2022-08-01T19:41:12.030 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:12.031 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:12.031 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:12.032 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.032 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:12.033 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:12.033 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.033 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:12.034 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6826", 2022-08-01T19:41:12.035 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:12.035 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.036 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.036 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:12.037 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6829", 2022-08-01T19:41:12.037 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:12.038 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:12.038 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:12.039 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.039 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6819/23010", 2022-08-01T19:41:12.039 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6824/23010", 2022-08-01T19:41:12.040 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6831/23010", 2022-08-01T19:41:12.040 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6829/23010", 2022-08-01T19:41:12.041 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:12.041 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:12.042 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:12.043 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:12.044 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.054 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.055 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:41:12.056 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "96123468-aabc-4a64-909e-a192c369090d", 2022-08-01T19:41:12.056 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:12.057 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:12.057 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:12.058 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:12.058 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:12.058 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:12.062 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:41:12.063 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 86, 2022-08-01T19:41:12.063 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:12.068 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:12.069 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:12.069 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:12.069 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.070 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:12.070 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6816", 2022-08-01T19:41:12.071 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:12.071 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.071 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.071 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:12.072 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6818", 2022-08-01T19:41:12.072 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:12.072 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:12.072 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:12.073 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.073 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:12.073 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:12.073 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.074 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:12.074 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6820", 2022-08-01T19:41:12.074 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:12.074 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.075 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.075 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:12.075 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6821", 2022-08-01T19:41:12.075 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:12.076 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:12.076 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:12.076 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.076 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:12.077 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:12.077 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.077 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:12.077 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6827", 2022-08-01T19:41:12.078 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:12.078 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.078 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.079 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:12.079 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6828", 2022-08-01T19:41:12.080 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:12.080 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:12.080 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:12.080 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.080 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:12.081 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:12.081 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.081 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:12.081 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6823", 2022-08-01T19:41:12.082 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:12.082 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.082 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.082 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:12.083 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6825", 2022-08-01T19:41:12.083 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:12.083 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:12.083 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:12.084 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.084 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6818/23009", 2022-08-01T19:41:12.084 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6821/23009", 2022-08-01T19:41:12.084 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6828/23009", 2022-08-01T19:41:12.085 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6825/23009", 2022-08-01T19:41:12.085 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:12.085 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:12.085 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:12.085 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:12.086 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.086 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.086 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:41:12.086 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a6e26665-e04b-46ec-ab66-e912acbf044c", 2022-08-01T19:41:12.087 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:12.087 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:12.087 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:12.087 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:12.088 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:12.088 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:12.088 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:41:12.088 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 86, 2022-08-01T19:41:12.089 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:12.089 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:12.090 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:12.098 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:12.099 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.099 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:12.100 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6810", 2022-08-01T19:41:12.100 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:12.101 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.101 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.102 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:12.102 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6811", 2022-08-01T19:41:12.103 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:12.103 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:12.104 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:12.104 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.105 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:12.105 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:12.106 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.106 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:12.107 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6812", 2022-08-01T19:41:12.108 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:12.109 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.109 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.109 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:12.110 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6813", 2022-08-01T19:41:12.110 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:12.111 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:12.111 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:12.112 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.113 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:12.113 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:12.114 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.114 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:12.115 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6816", 2022-08-01T19:41:12.116 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:12.116 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.116 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.117 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:12.117 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6817", 2022-08-01T19:41:12.117 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:12.118 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:12.118 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:12.119 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.119 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:12.119 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:12.119 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.120 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:12.120 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6814", 2022-08-01T19:41:12.120 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:12.120 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.121 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.121 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:12.121 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6815", 2022-08-01T19:41:12.121 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:12.122 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:12.122 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:12.122 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.122 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6811/23584", 2022-08-01T19:41:12.123 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6813/23584", 2022-08-01T19:41:12.123 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6817/23584", 2022-08-01T19:41:12.123 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6815/23584", 2022-08-01T19:41:12.123 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:12.124 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:12.124 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:12.124 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:12.124 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.125 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.125 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:41:12.126 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "502c0034-18ee-44ac-87ff-19b9dddb66a0", 2022-08-01T19:41:12.126 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:12.126 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:12.127 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:12.127 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:12.128 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:12.128 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:12.129 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:41:12.129 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 86, 2022-08-01T19:41:12.130 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:12.130 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:12.130 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:12.130 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:12.131 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.131 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:12.131 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6802", 2022-08-01T19:41:12.131 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:12.132 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.132 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.132 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:12.132 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6803", 2022-08-01T19:41:12.132 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:12.133 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:12.133 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:12.133 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.133 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:12.134 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:12.134 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.134 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:12.135 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6804", 2022-08-01T19:41:12.135 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:12.135 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.135 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.136 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:12.136 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6805", 2022-08-01T19:41:12.136 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:12.136 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:12.137 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:12.137 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.137 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:12.137 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:12.137 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.138 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:12.138 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6808", 2022-08-01T19:41:12.138 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:12.138 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.139 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.139 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:12.139 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6809", 2022-08-01T19:41:12.139 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:12.140 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:12.140 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:12.140 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.140 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:12.141 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:12.141 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.141 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:12.141 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6806", 2022-08-01T19:41:12.141 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:12.142 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.142 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.142 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:12.142 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6807", 2022-08-01T19:41:12.143 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:12.143 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:12.143 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:12.143 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.144 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6803/23585", 2022-08-01T19:41:12.144 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6805/23585", 2022-08-01T19:41:12.144 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6809/23585", 2022-08-01T19:41:12.144 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6807/23585", 2022-08-01T19:41:12.145 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:12.145 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:12.145 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:12.145 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:12.145 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.146 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.146 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:41:12.146 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "ac4e755a-7fe7-46cb-8407-852d5af9ffc5", 2022-08-01T19:41:12.146 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:12.147 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:12.147 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:12.147 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:12.147 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:12.148 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:12.148 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:41:12.148 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 86, 2022-08-01T19:41:12.148 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:12.149 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:12.149 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:12.149 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:12.149 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.149 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:12.150 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6820", 2022-08-01T19:41:12.150 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:12.150 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.150 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.151 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:12.151 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6821", 2022-08-01T19:41:12.151 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:12.151 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:12.152 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:12.152 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.152 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:12.152 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:12.152 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.153 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:12.153 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6823", 2022-08-01T19:41:12.153 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:12.153 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.154 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.154 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:12.154 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6825", 2022-08-01T19:41:12.155 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:12.155 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:12.155 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:12.155 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.155 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:12.156 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:12.156 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.156 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:12.156 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6831", 2022-08-01T19:41:12.157 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:12.157 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.157 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.157 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:12.158 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6833", 2022-08-01T19:41:12.158 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:12.158 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:12.158 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:12.159 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.159 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:12.159 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:12.159 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.160 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:12.160 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6827", 2022-08-01T19:41:12.160 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:12.160 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.160 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.161 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:12.161 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6829", 2022-08-01T19:41:12.161 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:12.161 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:12.162 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:12.162 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.162 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6821/23587", 2022-08-01T19:41:12.162 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6825/23587", 2022-08-01T19:41:12.163 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6833/23587", 2022-08-01T19:41:12.163 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6829/23587", 2022-08-01T19:41:12.163 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:12.163 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:12.164 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:12.164 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:12.164 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.164 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.165 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:41:12.165 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "874761e2-6410-40fc-b60c-e58abe5ce489", 2022-08-01T19:41:12.165 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:12.165 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:12.166 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:12.166 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:12.166 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:12.166 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:12.167 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:41:12.167 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 73, 2022-08-01T19:41:12.167 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:12.167 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:12.167 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:12.168 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:12.168 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.168 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:12.168 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6818", 2022-08-01T19:41:12.169 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:12.169 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.169 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.169 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:12.169 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6819", 2022-08-01T19:41:12.170 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:12.170 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:12.171 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:12.171 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.171 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:12.171 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:12.172 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.172 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:12.172 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6822", 2022-08-01T19:41:12.173 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:12.173 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.174 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.174 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:12.174 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6824", 2022-08-01T19:41:12.174 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:12.175 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:12.175 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:12.175 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.175 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:12.175 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:12.176 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.176 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:12.176 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6830", 2022-08-01T19:41:12.176 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:12.177 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.177 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.177 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:12.177 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6832", 2022-08-01T19:41:12.178 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:12.178 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:12.178 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:12.178 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.179 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:12.179 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:12.179 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.179 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:12.179 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6826", 2022-08-01T19:41:12.180 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:12.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.180 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.180 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:12.181 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6828", 2022-08-01T19:41:12.181 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:12.181 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:12.181 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:12.181 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.182 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6819/23586", 2022-08-01T19:41:12.182 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6824/23586", 2022-08-01T19:41:12.182 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6832/23586", 2022-08-01T19:41:12.182 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6828/23586", 2022-08-01T19:41:12.183 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:12.183 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:12.183 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:12.183 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:12.184 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:12.184 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:41:12.184 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_xinfo": [ 2022-08-01T19:41:12.184 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.185 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:41:12.185 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:12.185 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:12.185 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:12.186 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:12.186 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:12.186 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.588817+0000", 2022-08-01T19:41:12.186 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:12.187 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.187 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.187 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:41:12.187 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:12.187 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:12.188 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:12.188 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:12.188 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:12.188 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.782753+0000", 2022-08-01T19:41:12.189 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:12.189 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.189 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.189 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:41:12.190 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:12.190 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:12.190 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:12.190 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:12.191 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:12.191 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.837924+0000", 2022-08-01T19:41:12.191 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:12.191 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.192 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.192 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:41:12.192 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:12.192 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:12.192 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:12.193 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:12.193 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:12.193 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.839948+0000", 2022-08-01T19:41:12.193 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:12.194 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.194 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.194 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:41:12.194 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:12.194 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:12.195 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:12.195 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:12.195 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:12.195 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.112801+0000", 2022-08-01T19:41:12.196 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:12.196 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.196 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.196 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:41:12.196 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:12.197 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:12.197 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:12.197 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:12.197 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:12.198 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.106704+0000", 2022-08-01T19:41:12.198 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:12.198 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.198 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.199 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:41:12.199 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:12.199 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:12.199 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:12.199 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:12.200 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:12.200 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.893170+0000", 2022-08-01T19:41:12.200 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:12.200 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.201 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:12.201 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:41:12.201 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:12.201 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:12.202 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:12.202 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:12.202 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:12.202 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.886947+0000", 2022-08-01T19:41:12.203 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:12.203 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:12.203 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:41:12.203 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap": [], 2022-08-01T19:41:12.203 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap_items": [], 2022-08-01T19:41:12.204 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_temp": [], 2022-08-01T19:41:12.204 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_temp": [], 2022-08-01T19:41:12.204 INFO:teuthology.orchestra.run.smithi017.stdout: "blacklist": {}, 2022-08-01T19:41:12.204 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profiles": { 2022-08-01T19:41:12.205 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_data": { 2022-08-01T19:41:12.205 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-device-class": "", 2022-08-01T19:41:12.205 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:41:12.205 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-root": "default", 2022-08-01T19:41:12.206 INFO:teuthology.orchestra.run.smithi017.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T19:41:12.206 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:41:12.206 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "2", 2022-08-01T19:41:12.206 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:41:12.206 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:41:12.207 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van", 2022-08-01T19:41:12.207 INFO:teuthology.orchestra.run.smithi017.stdout: "w": "8" 2022-08-01T19:41:12.207 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.207 INFO:teuthology.orchestra.run.smithi017.stdout: "default": { 2022-08-01T19:41:12.208 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:41:12.208 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:41:12.208 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "1", 2022-08-01T19:41:12.208 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:41:12.209 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:41:12.209 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van" 2022-08-01T19:41:12.209 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:12.209 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:12.210 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps_queue": [], 2022-08-01T19:41:12.210 INFO:teuthology.orchestra.run.smithi017.stdout: "new_removed_snaps": [], 2022-08-01T19:41:12.210 INFO:teuthology.orchestra.run.smithi017.stdout: "new_purged_snaps": [], 2022-08-01T19:41:12.210 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_node_flags": {}, 2022-08-01T19:41:12.211 INFO:teuthology.orchestra.run.smithi017.stdout: "device_class_flags": {} 2022-08-01T19:41:12.211 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:41:12.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.875+0000 7f6a63fff700 1 -- 172.21.15.17:0/2963946462 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6a48053950 msgr2=0x7f6a48055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:12.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.875+0000 7f6a63fff700 1 --2- 172.21.15.17:0/2963946462 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6a48053950 0x7f6a48055e10 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f6a5401b000 tx=0x7f6a640a95b0).stop 2022-08-01T19:41:12.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.875+0000 7f6a63fff700 1 -- 172.21.15.17:0/2963946462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a640ad000 msgr2=0x7f6a64142fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:12.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.875+0000 7f6a63fff700 1 --2- 172.21.15.17:0/2963946462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a640ad000 0x7f6a64142fd0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 rx=0x7f6a4c00c000 tx=0x7f6a4c006df0).stop 2022-08-01T19:41:12.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.875+0000 7f6a63fff700 1 -- 172.21.15.17:0/2963946462 shutdown_connections 2022-08-01T19:41:12.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.875+0000 7f6a63fff700 1 --2- 172.21.15.17:0/2963946462 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6a48053950 0x7f6a48055e10 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:12.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.875+0000 7f6a63fff700 1 --2- 172.21.15.17:0/2963946462 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6a640b3260 0x7f6a64152d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:12.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.875+0000 7f6a63fff700 1 --2- 172.21.15.17:0/2963946462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a640b08a0 0x7f6a640a2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:12.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.875+0000 7f6a63fff700 1 --2- 172.21.15.17:0/2963946462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a640ad000 0x7f6a64142fd0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:12.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.875+0000 7f6a63fff700 1 -- 172.21.15.17:0/2963946462 >> 172.21.15.17:0/2963946462 conn(0x7f6a640612d0 msgr2=0x7f6a6413f110 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:12.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.875+0000 7f6a63fff700 1 -- 172.21.15.17:0/2963946462 shutdown_connections 2022-08-01T19:41:12.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:11.875+0000 7f6a63fff700 1 -- 172.21.15.17:0/2963946462 wait complete. 2022-08-01T19:41:12.216 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:41:12.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a62562700 1 Processor -- start 2022-08-01T19:41:12.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a62562700 1 -- start start 2022-08-01T19:41:12.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a62562700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f6a5c139c70 0x7f6a5c142f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:12.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a62562700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a5c0ac600 con 0x7f6a5c0b1de0 2022-08-01T19:41:12.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a62562700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a5c1438c0 con 0x7f6a5c143430 2022-08-01T19:41:12.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a62562700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a5c0b31a0 con 0x7f6a5c139c70 2022-08-01T19:41:12.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a60aff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f6a5c0b1de0 0x7f6a5c0a8ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33814/0 (socket says 172.21.15.17:33814) 2022-08-01T19:41:12.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a60aff700 1 -- 172.21.15.17:0/793764087 learned_addr learned my addr 172.21.15.17:0/793764087 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:12.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a5bfff700 1 --2- 172.21.15.17:0/793764087 >> v2:172.21.15.77:3300/0 conn(0x7f6a5c139c70 0x7f6a5c142f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:12.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a5b7fe700 1 -- 172.21.15.17:0/793764087 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3804796667 0 0) 0x7f6a5c0ac600 con 0x7f6a5c0b1de0 2022-08-01T19:41:12.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a5b7fe700 1 -- 172.21.15.17:0/793764087 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a3c003260 con 0x7f6a5c0b1de0 2022-08-01T19:41:12.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a5b7fe700 1 -- 172.21.15.17:0/793764087 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3951417694 0 0) 0x7f6a5c1438c0 con 0x7f6a5c143430 2022-08-01T19:41:12.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a5b7fe700 1 -- 172.21.15.17:0/793764087 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a5c0ac600 con 0x7f6a5c143430 2022-08-01T19:41:12.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a5b7fe700 1 -- 172.21.15.17:0/793764087 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1281066359 0 0) 0x7f6a3c003260 con 0x7f6a5c0b1de0 2022-08-01T19:41:12.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a5b7fe700 1 -- 172.21.15.17:0/793764087 >> v2:172.21.15.77:3300/0 conn(0x7f6a5c139c70 msgr2=0x7f6a5c142f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:12.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a5b7fe700 1 --2- 172.21.15.17:0/793764087 >> v2:172.21.15.77:3300/0 conn(0x7f6a5c139c70 0x7f6a5c142f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:12.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a5b7fe700 1 -- 172.21.15.17:0/793764087 >> v1:172.21.15.77:6789/0 conn(0x7f6a5c143430 legacy=0x7f6a5c0ab6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:12.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a5b7fe700 1 -- 172.21.15.17:0/793764087 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5c143b60 con 0x7f6a5c0b1de0 2022-08-01T19:41:12.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a5b7fe700 1 -- 172.21.15.17:0/793764087 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7f6a540019e0 con 0x7f6a5c0b1de0 2022-08-01T19:41:12.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a5b7fe700 1 -- 172.21.15.17:0/793764087 >> v1:172.21.15.17:6789/0 conn(0x7f6a5c0b1de0 legacy=0x7f6a5c0a8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:12.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a5b7fe700 1 --2- 172.21.15.17:0/793764087 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6a3c0038e0 0x7f6a3c003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:12.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a5b7fe700 1 --2- 172.21.15.17:0/793764087 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a3c004500 0x7f6a3c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:12.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a5b7fe700 1 --2- 172.21.15.17:0/793764087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a3c004e70 0x7f6a3c00d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:12.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a5b7fe700 1 -- 172.21.15.17:0/793764087 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c0b31a0 con 0x7f6a3c004500 2022-08-01T19:41:12.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a5b7fe700 1 -- 172.21.15.17:0/793764087 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c00df00 con 0x7f6a3c004e70 2022-08-01T19:41:12.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a5b7fe700 1 -- 172.21.15.17:0/793764087 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c00e0c0 con 0x7f6a3c0038e0 2022-08-01T19:41:12.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a53fff700 1 --2- 172.21.15.17:0/793764087 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a3c004500 0x7f6a3c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:12.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a60aff700 1 --2- 172.21.15.17:0/793764087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a3c004e70 0x7f6a3c00d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:12.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a5bfff700 1 --2- 172.21.15.17:0/793764087 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6a3c0038e0 0x7f6a3c003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:12.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a60aff700 1 -- 172.21.15.17:0/793764087 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6a3c0038e0 msgr2=0x7f6a3c003ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:12.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a60aff700 1 --2- 172.21.15.17:0/793764087 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6a3c0038e0 0x7f6a3c003ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:12.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a60aff700 1 -- 172.21.15.17:0/793764087 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a3c004500 msgr2=0x7f6a3c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:12.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a60aff700 1 --2- 172.21.15.17:0/793764087 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a3c004500 0x7f6a3c004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:12.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a60aff700 1 -- 172.21.15.17:0/793764087 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a3c003260 con 0x7f6a3c004e70 2022-08-01T19:41:12.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a53fff700 1 --2- 172.21.15.17:0/793764087 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a3c004500 0x7f6a3c004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:12.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a60aff700 1 -- 172.21.15.17:0/793764087 reap_dead start 2022-08-01T19:41:12.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.327+0000 7f6a60aff700 1 --2- 172.21.15.17:0/793764087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a3c004e70 0x7f6a3c00d960 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f6a3c0054d0 tx=0x7f6a5c143a50).ready entity=mon.1 client_cookie=fa3f2125a439d9d0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:12.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.331+0000 7f6a5b7fe700 1 -- 172.21.15.17:0/793764087 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a5400cca0 con 0x7f6a3c004e70 2022-08-01T19:41:12.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.331+0000 7f6a5b7fe700 1 -- 172.21.15.17:0/793764087 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5400ce00 con 0x7f6a3c004e70 2022-08-01T19:41:12.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.331+0000 7f6a5b7fe700 1 -- 172.21.15.17:0/793764087 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a54005b80 con 0x7f6a3c004e70 2022-08-01T19:41:12.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.331+0000 7f6a62562700 1 -- 172.21.15.17:0/793764087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a3c004e70 msgr2=0x7f6a3c00d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:12.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.331+0000 7f6a62562700 1 --2- 172.21.15.17:0/793764087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a3c004e70 0x7f6a3c00d960 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f6a3c0054d0 tx=0x7f6a5c143a50).stop 2022-08-01T19:41:12.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.331+0000 7f6a62562700 1 -- 172.21.15.17:0/793764087 shutdown_connections 2022-08-01T19:41:12.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.331+0000 7f6a62562700 1 --2- 172.21.15.17:0/793764087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a3c004e70 0x7f6a3c00d960 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:12.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.331+0000 7f6a62562700 1 -- 172.21.15.17:0/793764087 >> 172.21.15.17:0/793764087 conn(0x7f6a5c05f840 msgr2=0x7f6a5c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:12.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.331+0000 7f6a62562700 1 -- 172.21.15.17:0/793764087 shutdown_connections 2022-08-01T19:41:12.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.331+0000 7f6a62562700 1 -- 172.21.15.17:0/793764087 wait complete. 2022-08-01T19:41:12.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.331+0000 7f6a62562700 1 Processor -- start 2022-08-01T19:41:12.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.331+0000 7f6a62562700 1 -- start start 2022-08-01T19:41:12.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.331+0000 7f6a62562700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a3c004e70 0x7f6a5c13dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:12.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.331+0000 7f6a62562700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a5c140460 0x7f6a5c151e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:12.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.331+0000 7f6a62562700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6a5c13ae60 0x7f6a5c13eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:12.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.331+0000 7f6a62562700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c151770 con 0x7f6a3c004e70 2022-08-01T19:41:12.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.331+0000 7f6a62562700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c13aa50 con 0x7f6a5c140460 2022-08-01T19:41:12.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.331+0000 7f6a62562700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c14ed00 con 0x7f6a5c13ae60 2022-08-01T19:41:12.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.331+0000 7f6a5bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a3c004e70 0x7f6a5c13dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:12.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.335+0000 7f6a5bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a3c004e70 0x7f6a5c13dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41628/0 (socket says 172.21.15.17:41628) 2022-08-01T19:41:12.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.335+0000 7f6a5bfff700 1 -- 172.21.15.17:0/4040731430 learned_addr learned my addr 172.21.15.17:0/4040731430 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:12.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.335+0000 7f6a60aff700 1 --2- 172.21.15.17:0/4040731430 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6a5c13ae60 0x7f6a5c13eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:12.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.335+0000 7f6a53fff700 1 --2- 172.21.15.17:0/4040731430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a5c140460 0x7f6a5c151e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:12.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.335+0000 7f6a5bfff700 1 -- 172.21.15.17:0/4040731430 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6a5c13ae60 msgr2=0x7f6a5c13eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:12.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.335+0000 7f6a5bfff700 1 --2- 172.21.15.17:0/4040731430 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6a5c13ae60 0x7f6a5c13eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:12.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.335+0000 7f6a5bfff700 1 -- 172.21.15.17:0/4040731430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a5c140460 msgr2=0x7f6a5c151e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:12.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.335+0000 7f6a5bfff700 1 --2- 172.21.15.17:0/4040731430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a5c140460 0x7f6a5c151e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:12.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.335+0000 7f6a5bfff700 1 -- 172.21.15.17:0/4040731430 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5c14efa0 con 0x7f6a3c004e70 2022-08-01T19:41:12.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.335+0000 7f6a53fff700 1 --2- 172.21.15.17:0/4040731430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a5c140460 0x7f6a5c151e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:12.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.335+0000 7f6a5bfff700 1 --2- 172.21.15.17:0/4040731430 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a3c004e70 0x7f6a5c13dc60 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 rx=0x7f6a4c00b000 tx=0x7f6a4c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:12.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.335+0000 7f6a59ffb700 1 -- 172.21.15.17:0/4040731430 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a4c00f730 con 0x7f6a3c004e70 2022-08-01T19:41:12.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.335+0000 7f6a59ffb700 1 -- 172.21.15.17:0/4040731430 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4c00f890 con 0x7f6a3c004e70 2022-08-01T19:41:12.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.335+0000 7f6a62562700 1 -- 172.21.15.17:0/4040731430 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a5c1519a0 con 0x7f6a3c004e70 2022-08-01T19:41:12.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.335+0000 7f6a59ffb700 1 -- 172.21.15.17:0/4040731430 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a4c00fa00 con 0x7f6a3c004e70 2022-08-01T19:41:12.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.335+0000 7f6a62562700 1 -- 172.21.15.17:0/4040731430 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a5c152550 con 0x7f6a3c004e70 2022-08-01T19:41:12.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.335+0000 7f6a62562700 1 -- 172.21.15.17:0/4040731430 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a5c13a3a0 con 0x7f6a3c004e70 2022-08-01T19:41:12.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.335+0000 7f6a59ffb700 1 -- 172.21.15.17:0/4040731430 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6a4c024070 con 0x7f6a3c004e70 2022-08-01T19:41:12.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.335+0000 7f6a59ffb700 1 --2- 172.21.15.17:0/4040731430 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6a40053a10 0x7f6a40055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:12.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.335+0000 7f6a59ffb700 1 -- 172.21.15.17:0/4040731430 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6a4c06cb70 con 0x7f6a3c004e70 2022-08-01T19:41:12.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.335+0000 7f6a53fff700 1 --2- 172.21.15.17:0/4040731430 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6a40053a10 0x7f6a40055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:12.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.339+0000 7f6a53fff700 1 --2- 172.21.15.17:0/4040731430 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6a40053a10 0x7f6a40055ed0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f6a44003000 tx=0x7f6a44009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:12.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.339+0000 7f6a59ffb700 1 -- 172.21.15.17:0/4040731430 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6a5c13a3a0 con 0x7f6a3c004e70 2022-08-01T19:41:12.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.503+0000 7f6a62562700 1 -- 172.21.15.17:0/4040731430 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a5c13a3a0 con 0x7f6a3c004e70 2022-08-01T19:41:12.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.503+0000 7f6a59ffb700 1 -- 172.21.15.17:0/4040731430 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+2493 (secure 0 0 0) 0x7f6a5c13a3a0 con 0x7f6a3c004e70 2022-08-01T19:41:12.510 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:12.510 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":40,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6615,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3580428616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3580428616},{"type":"v1","addr":"172.21.15.17:6833","nonce":3580428616}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38},{"gid":6627,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/4048623373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":4048623373},{"type":"v1","addr":"172.21.15.17:6835","nonce":4048623373}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:41:01.207552+0000","modified":"2022-08-01T19:41:02.210147+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6630},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6630":{"gid":6630,"name":"b","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6835/4275403969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4275403969},{"type":"v1","addr":"172.21.15.77:6835","nonce":4275403969}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2022-08-01T19:41:12.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.507+0000 7f6a62562700 1 -- 172.21.15.17:0/4040731430 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6a40053a10 msgr2=0x7f6a40055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:12.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.507+0000 7f6a62562700 1 --2- 172.21.15.17:0/4040731430 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6a40053a10 0x7f6a40055ed0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f6a44003000 tx=0x7f6a44009040).stop 2022-08-01T19:41:12.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.507+0000 7f6a62562700 1 -- 172.21.15.17:0/4040731430 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a3c004e70 msgr2=0x7f6a5c13dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:12.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.507+0000 7f6a62562700 1 --2- 172.21.15.17:0/4040731430 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a3c004e70 0x7f6a5c13dc60 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 rx=0x7f6a4c00b000 tx=0x7f6a4c006df0).stop 2022-08-01T19:41:12.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.507+0000 7f6a62562700 1 -- 172.21.15.17:0/4040731430 shutdown_connections 2022-08-01T19:41:12.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.507+0000 7f6a62562700 1 --2- 172.21.15.17:0/4040731430 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6a40053a10 0x7f6a40055ed0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:12.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.507+0000 7f6a62562700 1 --2- 172.21.15.17:0/4040731430 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6a5c13ae60 0x7f6a5c13eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:12.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.507+0000 7f6a62562700 1 --2- 172.21.15.17:0/4040731430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a5c140460 0x7f6a5c151e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:12.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.507+0000 7f6a62562700 1 --2- 172.21.15.17:0/4040731430 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a3c004e70 0x7f6a5c13dc60 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:12.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.507+0000 7f6a62562700 1 -- 172.21.15.17:0/4040731430 >> 172.21.15.17:0/4040731430 conn(0x7f6a5c05f840 msgr2=0x7f6a5c0ad040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:12.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.507+0000 7f6a62562700 1 -- 172.21.15.17:0/4040731430 shutdown_connections 2022-08-01T19:41:12.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.507+0000 7f6a62562700 1 -- 172.21.15.17:0/4040731430 wait complete. 2022-08-01T19:41:12.515 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 40 2022-08-01T19:41:12.526 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-08-01T19:41:12.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.631+0000 7f988906b700 1 Processor -- start 2022-08-01T19:41:12.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.631+0000 7f988906b700 1 -- start start 2022-08-01T19:41:12.639 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.631+0000 7f988906b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f98840b5ed0 0x7f98840b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:12.640 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.631+0000 7f988906b700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9884061d90 con 0x7f98840b6ee0 2022-08-01T19:41:12.640 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.631+0000 7f988906b700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98840ba5a0 con 0x7f98840619b0 2022-08-01T19:41:12.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.631+0000 7f988906b700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9884143fa0 con 0x7f98840b5ed0 2022-08-01T19:41:12.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.631+0000 7f98837fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f98840b6ee0 0x7f98840b5190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33832/0 (socket says 172.21.15.17:33832) 2022-08-01T19:41:12.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.631+0000 7f98837fe700 1 -- 172.21.15.17:0/3415376187 learned_addr learned my addr 172.21.15.17:0/3415376187 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:12.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.631+0000 7f988886a700 1 --2- 172.21.15.17:0/3415376187 >> v2:172.21.15.17:3300/0 conn(0x7f98840b5ed0 0x7f98840b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:12.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.631+0000 7f9882ffd700 1 -- 172.21.15.17:0/3415376187 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1328301524 0 0) 0x7f9884061d90 con 0x7f98840b6ee0 2022-08-01T19:41:12.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.631+0000 7f9882ffd700 1 -- 172.21.15.17:0/3415376187 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9864003260 con 0x7f98840b6ee0 2022-08-01T19:41:12.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.631+0000 7f9882ffd700 1 -- 172.21.15.17:0/3415376187 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3132496089 0 0) 0x7f98840ba5a0 con 0x7f98840619b0 2022-08-01T19:41:12.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.631+0000 7f9882ffd700 1 -- 172.21.15.17:0/3415376187 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9884061d90 con 0x7f98840619b0 2022-08-01T19:41:12.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.631+0000 7f988886a700 1 -- 172.21.15.17:0/3415376187 >> v1:172.21.15.77:6789/0 conn(0x7f98840619b0 legacy=0x7f9884143890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:12.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.631+0000 7f9882ffd700 1 -- 172.21.15.17:0/3415376187 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 978496386 0 0) 0x7f9864003260 con 0x7f98840b6ee0 2022-08-01T19:41:12.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.631+0000 7f988886a700 1 -- 172.21.15.17:0/3415376187 >> v1:172.21.15.17:6789/0 conn(0x7f98840b6ee0 legacy=0x7f98840b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:12.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.631+0000 7f988886a700 1 -- 172.21.15.17:0/3415376187 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9884143150 con 0x7f98840b5ed0 2022-08-01T19:41:12.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.631+0000 7f988886a700 1 --2- 172.21.15.17:0/3415376187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98840b5ed0 0x7f98840b58a0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 rx=0x7f9874001490 tx=0x7f987c006df0).ready entity=mon.0 client_cookie=464a6d05e74ee46f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:12.645 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.631+0000 7f9882ffd700 1 -- 172.21.15.17:0/3415376187 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9874001a00 con 0x7f98840b5ed0 2022-08-01T19:41:12.645 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.631+0000 7f9882ffd700 1 -- 172.21.15.17:0/3415376187 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987c000e90 con 0x7f98840b5ed0 2022-08-01T19:41:12.645 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.631+0000 7f9882ffd700 1 -- 172.21.15.17:0/3415376187 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f987c012460 con 0x7f98840b5ed0 2022-08-01T19:41:12.646 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.635+0000 7f988906b700 1 -- 172.21.15.17:0/3415376187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98840b5ed0 msgr2=0x7f98840b58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:12.646 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.635+0000 7f988906b700 1 --2- 172.21.15.17:0/3415376187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98840b5ed0 0x7f98840b58a0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 rx=0x7f9874001490 tx=0x7f987c006df0).stop 2022-08-01T19:41:12.646 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.635+0000 7f988906b700 1 -- 172.21.15.17:0/3415376187 shutdown_connections 2022-08-01T19:41:12.646 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.635+0000 7f988906b700 1 --2- 172.21.15.17:0/3415376187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98840b5ed0 0x7f98840b58a0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:12.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.635+0000 7f988906b700 1 -- 172.21.15.17:0/3415376187 >> 172.21.15.17:0/3415376187 conn(0x7f988406af80 msgr2=0x7f988405fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:12.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.635+0000 7f988906b700 1 -- 172.21.15.17:0/3415376187 shutdown_connections 2022-08-01T19:41:12.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.635+0000 7f988906b700 1 -- 172.21.15.17:0/3415376187 wait complete. 2022-08-01T19:41:12.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.635+0000 7f988906b700 1 Processor -- start 2022-08-01T19:41:12.648 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.635+0000 7f988906b700 1 -- start start 2022-08-01T19:41:12.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.635+0000 7f988906b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98840619b0 0x7f98840b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:12.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.635+0000 7f988906b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98840b5ed0 0x7f9884113400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:12.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.635+0000 7f988906b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98840b6ee0 0x7f9884140080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:12.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.635+0000 7f988906b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98841533f0 con 0x7f98840619b0 2022-08-01T19:41:12.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.635+0000 7f988906b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9884153150 con 0x7f98840b5ed0 2022-08-01T19:41:12.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.635+0000 7f988906b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9884144c30 con 0x7f98840b6ee0 2022-08-01T19:41:12.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.635+0000 7f9883fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98840619b0 0x7f98840b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:12.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.635+0000 7f9883fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98840619b0 0x7f98840b05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41642/0 (socket says 172.21.15.17:41642) 2022-08-01T19:41:12.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.635+0000 7f9883fff700 1 -- 172.21.15.17:0/1945924709 learned_addr learned my addr 172.21.15.17:0/1945924709 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:12.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.635+0000 7f98837fe700 1 --2- 172.21.15.17:0/1945924709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98840b5ed0 0x7f9884113400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:12.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.635+0000 7f988886a700 1 --2- 172.21.15.17:0/1945924709 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98840b6ee0 0x7f9884140080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:12.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.635+0000 7f9883fff700 1 -- 172.21.15.17:0/1945924709 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98840b6ee0 msgr2=0x7f9884140080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:12.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.635+0000 7f9883fff700 1 --2- 172.21.15.17:0/1945924709 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98840b6ee0 0x7f9884140080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:12.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.635+0000 7f9883fff700 1 -- 172.21.15.17:0/1945924709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98840b5ed0 msgr2=0x7f9884113400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:12.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.635+0000 7f9883fff700 1 --2- 172.21.15.17:0/1945924709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98840b5ed0 0x7f9884113400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:12.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.635+0000 7f9883fff700 1 -- 172.21.15.17:0/1945924709 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f987c00b180 con 0x7f98840619b0 2022-08-01T19:41:12.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.635+0000 7f9883fff700 1 --2- 172.21.15.17:0/1945924709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98840619b0 0x7f98840b05e0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 rx=0x7f986c001110 tx=0x7f986c007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:12.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.635+0000 7f98817fa700 1 -- 172.21.15.17:0/1945924709 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f986c012400 con 0x7f98840619b0 2022-08-01T19:41:12.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.635+0000 7f98817fa700 1 -- 172.21.15.17:0/1945924709 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986c012560 con 0x7f98840619b0 2022-08-01T19:41:12.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.635+0000 7f988906b700 1 -- 172.21.15.17:0/1945924709 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9884144ed0 con 0x7f98840619b0 2022-08-01T19:41:12.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.635+0000 7f98817fa700 1 -- 172.21.15.17:0/1945924709 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f986c0128c0 con 0x7f98840619b0 2022-08-01T19:41:12.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.635+0000 7f988906b700 1 -- 172.21.15.17:0/1945924709 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9884144360 con 0x7f98840619b0 2022-08-01T19:41:12.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.639+0000 7f988906b700 1 -- 172.21.15.17:0/1945924709 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9884145060 con 0x7f98840619b0 2022-08-01T19:41:12.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.639+0000 7f98817fa700 1 -- 172.21.15.17:0/1945924709 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f986c019030 con 0x7f98840619b0 2022-08-01T19:41:12.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.639+0000 7f98817fa700 1 --2- 172.21.15.17:0/1945924709 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9868053950 0x7f9868055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:12.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.639+0000 7f98817fa700 1 -- 172.21.15.17:0/1945924709 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f986c06b440 con 0x7f98840619b0 2022-08-01T19:41:12.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.639+0000 7f98837fe700 1 --2- 172.21.15.17:0/1945924709 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9868053950 0x7f9868055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:12.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.639+0000 7f98837fe700 1 --2- 172.21.15.17:0/1945924709 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9868053950 0x7f9868055e10 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f987400d000 tx=0x7f9874000f40).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:12.660 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.643+0000 7f98817fa700 1 -- 172.21.15.17:0/1945924709 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f986c042200 con 0x7f98840619b0 2022-08-01T19:41:12.811 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:12.807+0000 7f988906b700 1 -- 172.21.15.17:0/1945924709 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f98840574e0 con 0x7f98840619b0 2022-08-01T19:41:13.215 INFO:tasks.ceph.mon.a.smithi017.stderr:2022-08-01T19:41:13.211+0000 7f6a53773700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-08-01T19:41:13.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.223+0000 7f98817fa700 1 -- 172.21.15.17:0/1945924709 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v41) v1 ==== 161+0+0 (secure 0 0 0) 0x7f986c045820 con 0x7f98840619b0 2022-08-01T19:41:13.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.227+0000 7f988906b700 1 -- 172.21.15.17:0/1945924709 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9868053950 msgr2=0x7f9868055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:13.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.227+0000 7f988906b700 1 --2- 172.21.15.17:0/1945924709 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9868053950 0x7f9868055e10 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f987400d000 tx=0x7f9874000f40).stop 2022-08-01T19:41:13.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.227+0000 7f988906b700 1 -- 172.21.15.17:0/1945924709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98840619b0 msgr2=0x7f98840b05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:13.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.227+0000 7f988906b700 1 --2- 172.21.15.17:0/1945924709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98840619b0 0x7f98840b05e0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 rx=0x7f986c001110 tx=0x7f986c007ae0).stop 2022-08-01T19:41:13.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.227+0000 7f988906b700 1 -- 172.21.15.17:0/1945924709 shutdown_connections 2022-08-01T19:41:13.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.227+0000 7f988906b700 1 --2- 172.21.15.17:0/1945924709 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9868053950 0x7f9868055e10 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:13.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.227+0000 7f988906b700 1 --2- 172.21.15.17:0/1945924709 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98840b6ee0 0x7f9884140080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:13.236 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.227+0000 7f988906b700 1 --2- 172.21.15.17:0/1945924709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98840b5ed0 0x7f9884113400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:13.236 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.227+0000 7f988906b700 1 --2- 172.21.15.17:0/1945924709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98840619b0 0x7f98840b05e0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:13.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.227+0000 7f988906b700 1 -- 172.21.15.17:0/1945924709 >> 172.21.15.17:0/1945924709 conn(0x7f988406af80 msgr2=0x7f9884147850 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:13.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.227+0000 7f988906b700 1 -- 172.21.15.17:0/1945924709 shutdown_connections 2022-08-01T19:41:13.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.227+0000 7f988906b700 1 -- 172.21.15.17:0/1945924709 wait complete. 2022-08-01T19:41:13.240 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-08-01T19:41:13.247 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:41:13.254 INFO:tasks.ceph.mds.b.smithi077.stdout:starting mds.b at 2022-08-01T19:41:13.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.351+0000 7f0441045700 1 Processor -- start 2022-08-01T19:41:13.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.351+0000 7f0441045700 1 -- start start 2022-08-01T19:41:13.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.351+0000 7f0441045700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f043c066dc0 0x7f043c14eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:13.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.355+0000 7f0441045700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f043c0654c0 con 0x7f043c0b3190 2022-08-01T19:41:13.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.355+0000 7f0441045700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f043c0b3570 con 0x7f043c0b2730 2022-08-01T19:41:13.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.355+0000 7f0441045700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f043c062190 con 0x7f043c066dc0 2022-08-01T19:41:13.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.355+0000 7f043bfff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f043c0b3190 0x7f043c060ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33844/0 (socket says 172.21.15.17:33844) 2022-08-01T19:41:13.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.355+0000 7f043bfff700 1 -- 172.21.15.17:0/2134625043 learned_addr learned my addr 172.21.15.17:0/2134625043 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:13.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.355+0000 7f0440844700 1 --2- 172.21.15.17:0/2134625043 >> v2:172.21.15.77:3300/0 conn(0x7f043c066dc0 0x7f043c14eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:13.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.355+0000 7f043b7fe700 1 -- 172.21.15.17:0/2134625043 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1404318725 0 0) 0x7f043c0654c0 con 0x7f043c0b3190 2022-08-01T19:41:13.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.355+0000 7f043b7fe700 1 -- 172.21.15.17:0/2134625043 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f041c003260 con 0x7f043c0b3190 2022-08-01T19:41:13.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.355+0000 7f0440844700 1 -- 172.21.15.17:0/2134625043 >> v1:172.21.15.77:6789/0 conn(0x7f043c0b2730 legacy=0x7f043c0666b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:13.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.355+0000 7f0440844700 1 -- 172.21.15.17:0/2134625043 >> v1:172.21.15.17:6789/0 conn(0x7f043c0b3190 legacy=0x7f043c060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:13.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.355+0000 7f0440844700 1 -- 172.21.15.17:0/2134625043 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f043c05fbf0 con 0x7f043c066dc0 2022-08-01T19:41:13.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.355+0000 7f0440844700 1 --2- 172.21.15.17:0/2134625043 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f043c066dc0 0x7f043c14eac0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f04340032f0 tx=0x7f04340091d0).ready entity=mon.1 client_cookie=408726dacff45372 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:13.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.355+0000 7f043b7fe700 1 -- 172.21.15.17:0/2134625043 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f043400b6c0 con 0x7f043c066dc0 2022-08-01T19:41:13.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.355+0000 7f043b7fe700 1 -- 172.21.15.17:0/2134625043 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f043400b820 con 0x7f043c066dc0 2022-08-01T19:41:13.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.355+0000 7f043b7fe700 1 -- 172.21.15.17:0/2134625043 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f043400b990 con 0x7f043c066dc0 2022-08-01T19:41:13.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.355+0000 7f0441045700 1 -- 172.21.15.17:0/2134625043 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f043c066dc0 msgr2=0x7f043c14eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:13.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.355+0000 7f0441045700 1 --2- 172.21.15.17:0/2134625043 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f043c066dc0 0x7f043c14eac0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f04340032f0 tx=0x7f04340091d0).stop 2022-08-01T19:41:13.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.355+0000 7f0441045700 1 -- 172.21.15.17:0/2134625043 shutdown_connections 2022-08-01T19:41:13.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.355+0000 7f0441045700 1 --2- 172.21.15.17:0/2134625043 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f043c066dc0 0x7f043c14eac0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:13.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.355+0000 7f0441045700 1 -- 172.21.15.17:0/2134625043 >> 172.21.15.17:0/2134625043 conn(0x7f043c063870 msgr2=0x7f043c063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:13.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.355+0000 7f0441045700 1 -- 172.21.15.17:0/2134625043 shutdown_connections 2022-08-01T19:41:13.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.355+0000 7f0441045700 1 -- 172.21.15.17:0/2134625043 wait complete. 2022-08-01T19:41:13.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.359+0000 7f0441045700 1 Processor -- start 2022-08-01T19:41:13.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.359+0000 7f0441045700 1 -- start start 2022-08-01T19:41:13.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.359+0000 7f0441045700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f043c066dc0 0x7f043c1464d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:13.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.359+0000 7f0441045700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f043c0b2730 0x7f043c0b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:13.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.359+0000 7f0441045700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f043c0b3190 0x7f043c1445a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:13.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.359+0000 7f0441045700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043c153410 con 0x7f043c0b2730 2022-08-01T19:41:13.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.359+0000 7f0441045700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043c1524f0 con 0x7f043c066dc0 2022-08-01T19:41:13.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.359+0000 7f0441045700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f043c152790 con 0x7f043c0b3190 2022-08-01T19:41:13.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.359+0000 7f04337fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f043c0b2730 0x7f043c0b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:13.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.359+0000 7f04337fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f043c0b2730 0x7f043c0b58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41656/0 (socket says 172.21.15.17:41656) 2022-08-01T19:41:13.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.359+0000 7f04337fe700 1 -- 172.21.15.17:0/102052342 learned_addr learned my addr 172.21.15.17:0/102052342 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:13.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.359+0000 7f0440844700 1 --2- 172.21.15.17:0/102052342 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f043c0b3190 0x7f043c1445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:13.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.359+0000 7f043bfff700 1 --2- 172.21.15.17:0/102052342 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f043c066dc0 0x7f043c1464d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:13.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.359+0000 7f04337fe700 1 -- 172.21.15.17:0/102052342 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f043c0b3190 msgr2=0x7f043c1445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:13.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.359+0000 7f04337fe700 1 --2- 172.21.15.17:0/102052342 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f043c0b3190 0x7f043c1445a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:13.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.359+0000 7f04337fe700 1 -- 172.21.15.17:0/102052342 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f043c066dc0 msgr2=0x7f043c1464d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:13.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.359+0000 7f04337fe700 1 --2- 172.21.15.17:0/102052342 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f043c066dc0 0x7f043c1464d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:13.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.359+0000 7f04337fe700 1 -- 172.21.15.17:0/102052342 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0434009470 con 0x7f043c0b2730 2022-08-01T19:41:13.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.359+0000 7f0440844700 1 --2- 172.21.15.17:0/102052342 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f043c0b3190 0x7f043c1445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:13.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.363+0000 7f04337fe700 1 --2- 172.21.15.17:0/102052342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f043c0b2730 0x7f043c0b58f0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 rx=0x7f043c068010 tx=0x7f0424007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:13.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.363+0000 7f0439ffb700 1 -- 172.21.15.17:0/102052342 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0424012400 con 0x7f043c0b2730 2022-08-01T19:41:13.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.363+0000 7f0439ffb700 1 -- 172.21.15.17:0/102052342 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0424012560 con 0x7f043c0b2730 2022-08-01T19:41:13.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.363+0000 7f0441045700 1 -- 172.21.15.17:0/102052342 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f043c152a30 con 0x7f043c0b2730 2022-08-01T19:41:13.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.363+0000 7f0439ffb700 1 -- 172.21.15.17:0/102052342 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04240128c0 con 0x7f043c0b2730 2022-08-01T19:41:13.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.363+0000 7f0441045700 1 -- 172.21.15.17:0/102052342 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f043c152f70 con 0x7f043c0b2730 2022-08-01T19:41:13.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.363+0000 7f0441045700 1 -- 172.21.15.17:0/102052342 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f043c153130 con 0x7f043c0b2730 2022-08-01T19:41:13.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.363+0000 7f0439ffb700 1 -- 172.21.15.17:0/102052342 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0424019030 con 0x7f043c0b2730 2022-08-01T19:41:13.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.363+0000 7f0439ffb700 1 --2- 172.21.15.17:0/102052342 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0420053aa0 0x7f0420055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:13.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.363+0000 7f0439ffb700 1 -- 172.21.15.17:0/102052342 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f042406b3f0 con 0x7f043c0b2730 2022-08-01T19:41:13.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.367+0000 7f043bfff700 1 --2- 172.21.15.17:0/102052342 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0420053aa0 0x7f0420055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:13.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.367+0000 7f043bfff700 1 --2- 172.21.15.17:0/102052342 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0420053aa0 0x7f0420055f60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f042c00d000 tx=0x7f042c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:13.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.375+0000 7f0439ffb700 1 -- 172.21.15.17:0/102052342 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0424042100 con 0x7f043c0b2730 2022-08-01T19:41:13.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.539+0000 7f0441045700 1 -- 172.21.15.17:0/102052342 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f043c05fe90 con 0x7f043c0b2730 2022-08-01T19:41:13.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.539+0000 7f0439ffb700 1 -- 172.21.15.17:0/102052342 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+2132 (secure 0 0 0) 0x7f0424045720 con 0x7f043c0b2730 2022-08-01T19:41:13.544 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:13.545 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":41,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6615,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3580428616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3580428616},{"type":"v1","addr":"172.21.15.17:6833","nonce":3580428616}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38},{"gid":6627,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/4048623373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":4048623373},{"type":"v1","addr":"172.21.15.17:6835","nonce":4048623373}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38}],"filesystems":[{"mdsmap":{"epoch":41,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:41:01.207552+0000","modified":"2022-08-01T19:41:13.214963+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2022-08-01T19:41:13.547 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.539+0000 7f0441045700 1 -- 172.21.15.17:0/102052342 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0420053aa0 msgr2=0x7f0420055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:13.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.539+0000 7f0441045700 1 --2- 172.21.15.17:0/102052342 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0420053aa0 0x7f0420055f60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f042c00d000 tx=0x7f042c009040).stop 2022-08-01T19:41:13.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.539+0000 7f0441045700 1 -- 172.21.15.17:0/102052342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f043c0b2730 msgr2=0x7f043c0b58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:13.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.539+0000 7f0441045700 1 --2- 172.21.15.17:0/102052342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f043c0b2730 0x7f043c0b58f0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 rx=0x7f043c068010 tx=0x7f0424007c20).stop 2022-08-01T19:41:13.550 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.539+0000 7f0441045700 1 -- 172.21.15.17:0/102052342 shutdown_connections 2022-08-01T19:41:13.550 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.539+0000 7f0441045700 1 --2- 172.21.15.17:0/102052342 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0420053aa0 0x7f0420055f60 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:13.550 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.539+0000 7f0441045700 1 --2- 172.21.15.17:0/102052342 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f043c0b3190 0x7f043c1445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:13.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.539+0000 7f0441045700 1 --2- 172.21.15.17:0/102052342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f043c0b2730 0x7f043c0b58f0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:13.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.539+0000 7f0441045700 1 --2- 172.21.15.17:0/102052342 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f043c066dc0 0x7f043c1464d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:13.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.539+0000 7f0441045700 1 -- 172.21.15.17:0/102052342 >> 172.21.15.17:0/102052342 conn(0x7f043c063870 msgr2=0x7f043c066510 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:13.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.539+0000 7f0441045700 1 -- 172.21.15.17:0/102052342 shutdown_connections 2022-08-01T19:41:13.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.539+0000 7f0441045700 1 -- 172.21.15.17:0/102052342 wait complete. 2022-08-01T19:41:13.555 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 41 2022-08-01T19:41:13.557 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-08-01T19:41:13.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.679+0000 7f0cd3fff700 1 Processor -- start 2022-08-01T19:41:13.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.679+0000 7f0cd3fff700 1 -- start start 2022-08-01T19:41:13.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.679+0000 7f0cd3fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0cd40b3190 0x7f0cd4060ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:13.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.679+0000 7f0cd3fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0cd4061190 0x7f0cd4066dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:13.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.679+0000 7f0cd3fff700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd40654c0 con 0x7f0cd40b2730 2022-08-01T19:41:13.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.679+0000 7f0cd3fff700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cd4062190 con 0x7f0cd4061190 2022-08-01T19:41:13.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.679+0000 7f0cd3fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd4061600 con 0x7f0cd40b3190 2022-08-01T19:41:13.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.679+0000 7f0cd37fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0cd4061190 0x7f0cd4066dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:13.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.679+0000 7f0cd37fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0cd4061190 0x7f0cd4066dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41664/0 (socket says 172.21.15.17:41664) 2022-08-01T19:41:13.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.679+0000 7f0cd37fe700 1 -- 172.21.15.17:0/1763048085 learned_addr learned my addr 172.21.15.17:0/1763048085 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:13.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.679+0000 7f0cd27fc700 1 --1- 172.21.15.17:0/1763048085 >> v1:172.21.15.17:6789/0 conn(0x7f0cd40b2730 0x7f0cd40666b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33860/0 (socket says 172.21.15.17:33860) 2022-08-01T19:41:13.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.679+0000 7f0cd2ffd700 1 --2- 172.21.15.17:0/1763048085 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0cd40b3190 0x7f0cd4060ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:13.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.679+0000 7f0cd37fe700 1 -- 172.21.15.17:0/1763048085 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0cd40b3190 msgr2=0x7f0cd4060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:13.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.679+0000 7f0cd37fe700 1 --2- 172.21.15.17:0/1763048085 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0cd40b3190 0x7f0cd4060ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:13.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.679+0000 7f0cd37fe700 1 -- 172.21.15.17:0/1763048085 >> v1:172.21.15.17:6789/0 conn(0x7f0cd40b2730 legacy=0x7f0cd40666b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:13.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.679+0000 7f0cd37fe700 1 -- 172.21.15.17:0/1763048085 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd40b3570 con 0x7f0cd4061190 2022-08-01T19:41:13.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.679+0000 7f0cd37fe700 1 --2- 172.21.15.17:0/1763048085 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cd4061190 0x7f0cd4066dc0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 rx=0x7f0ccc002f70 tx=0x7f0ccc006eb0).ready entity=mon.0 client_cookie=69206d707b61e819 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:13.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.679+0000 7f0cd1ffb700 1 -- 172.21.15.17:0/1763048085 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ccc00d570 con 0x7f0cd4061190 2022-08-01T19:41:13.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.679+0000 7f0cd1ffb700 1 -- 172.21.15.17:0/1763048085 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ccc000e90 con 0x7f0cd4061190 2022-08-01T19:41:13.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.679+0000 7f0cd1ffb700 1 -- 172.21.15.17:0/1763048085 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ccc00d900 con 0x7f0cd4061190 2022-08-01T19:41:13.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.683+0000 7f0cd3fff700 1 -- 172.21.15.17:0/1763048085 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cd4061190 msgr2=0x7f0cd4066dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:13.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.683+0000 7f0cd3fff700 1 --2- 172.21.15.17:0/1763048085 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cd4061190 0x7f0cd4066dc0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 rx=0x7f0ccc002f70 tx=0x7f0ccc006eb0).stop 2022-08-01T19:41:13.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.683+0000 7f0cd3fff700 1 -- 172.21.15.17:0/1763048085 shutdown_connections 2022-08-01T19:41:13.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.683+0000 7f0cd3fff700 1 --2- 172.21.15.17:0/1763048085 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cd4061190 0x7f0cd4066dc0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:13.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.683+0000 7f0cd3fff700 1 --2- 172.21.15.17:0/1763048085 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0cd40b3190 0x7f0cd4060ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:13.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.683+0000 7f0cd3fff700 1 -- 172.21.15.17:0/1763048085 >> 172.21.15.17:0/1763048085 conn(0x7f0cd4063870 msgr2=0x7f0cd4063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:13.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.683+0000 7f0cd3fff700 1 -- 172.21.15.17:0/1763048085 shutdown_connections 2022-08-01T19:41:13.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.683+0000 7f0cd3fff700 1 -- 172.21.15.17:0/1763048085 wait complete. 2022-08-01T19:41:13.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.683+0000 7f0cd3fff700 1 Processor -- start 2022-08-01T19:41:13.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.683+0000 7f0cd3fff700 1 -- start start 2022-08-01T19:41:13.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.683+0000 7f0cd3fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0cd4061190 0x7f0cd40b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:13.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.683+0000 7f0cd3fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0cd40b2730 0x7f0cd40aa350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:13.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.683+0000 7f0cd3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cd40b3190 0x7f0cd40b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:13.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.683+0000 7f0cd3fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd40b0440 con 0x7f0cd40b3190 2022-08-01T19:41:13.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.683+0000 7f0cd3fff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd40aba80 con 0x7f0cd40b2730 2022-08-01T19:41:13.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.683+0000 7f0cd3fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd40abd20 con 0x7f0cd4061190 2022-08-01T19:41:13.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.683+0000 7f0cd37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cd40b3190 0x7f0cd40b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:13.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.683+0000 7f0cd2ffd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0cd4061190 0x7f0cd40b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:13.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.683+0000 7f0cd37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cd40b3190 0x7f0cd40b0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41672/0 (socket says 172.21.15.17:41672) 2022-08-01T19:41:13.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.683+0000 7f0cd37fe700 1 -- 172.21.15.17:0/2195853739 learned_addr learned my addr 172.21.15.17:0/2195853739 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:13.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.683+0000 7f0cd27fc700 1 --2- 172.21.15.17:0/2195853739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0cd40b2730 0x7f0cd40aa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:13.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.687+0000 7f0cd2ffd700 1 -- 172.21.15.17:0/2195853739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0cd40b2730 msgr2=0x7f0cd40aa350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:13.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.687+0000 7f0cd2ffd700 1 --2- 172.21.15.17:0/2195853739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0cd40b2730 0x7f0cd40aa350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:13.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.687+0000 7f0cd2ffd700 1 -- 172.21.15.17:0/2195853739 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cd40b3190 msgr2=0x7f0cd40b0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:13.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.687+0000 7f0cd2ffd700 1 --2- 172.21.15.17:0/2195853739 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cd40b3190 0x7f0cd40b0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:13.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.687+0000 7f0cd2ffd700 1 -- 172.21.15.17:0/2195853739 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ccc00b1e0 con 0x7f0cd4061190 2022-08-01T19:41:13.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.687+0000 7f0cd37fe700 1 --2- 172.21.15.17:0/2195853739 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cd40b3190 0x7f0cd40b0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:13.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.687+0000 7f0cd2ffd700 1 --2- 172.21.15.17:0/2195853739 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0cd4061190 0x7f0cd40b68d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f0cc400c000 tx=0x7f0cc40091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:13.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.687+0000 7f0ccbfff700 1 -- 172.21.15.17:0/2195853739 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cc4012600 con 0x7f0cd4061190 2022-08-01T19:41:13.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.687+0000 7f0ccbfff700 1 -- 172.21.15.17:0/2195853739 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc4012760 con 0x7f0cd4061190 2022-08-01T19:41:13.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.687+0000 7f0cd3fff700 1 -- 172.21.15.17:0/2195853739 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd40abfc0 con 0x7f0cd4061190 2022-08-01T19:41:13.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.687+0000 7f0ccbfff700 1 -- 172.21.15.17:0/2195853739 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0cc4012ac0 con 0x7f0cd4061190 2022-08-01T19:41:13.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.687+0000 7f0cd3fff700 1 -- 172.21.15.17:0/2195853739 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd40671b0 con 0x7f0cd4061190 2022-08-01T19:41:13.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.687+0000 7f0cd3fff700 1 -- 172.21.15.17:0/2195853739 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cd40ac150 con 0x7f0cd4061190 2022-08-01T19:41:13.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.687+0000 7f0ccbfff700 1 -- 172.21.15.17:0/2195853739 <== mon.2 v2:172.21.15.77:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0cc4022070 con 0x7f0cd4061190 2022-08-01T19:41:13.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.687+0000 7f0ccbfff700 1 --2- 172.21.15.17:0/2195853739 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0cb8053aa0 0x7f0cb8055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:13.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.687+0000 7f0ccbfff700 1 -- 172.21.15.17:0/2195853739 <== mon.2 v2:172.21.15.77:3301/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f0cc406caf0 con 0x7f0cd4061190 2022-08-01T19:41:13.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.690+0000 7f0cd27fc700 1 --2- 172.21.15.17:0/2195853739 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0cb8053aa0 0x7f0cb8055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:13.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.690+0000 7f0cd27fc700 1 --2- 172.21.15.17:0/2195853739 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0cb8053aa0 0x7f0cb8055f60 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f0cbc00d000 tx=0x7f0cbc007b80).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:13.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.694+0000 7f0ccbfff700 1 -- 172.21.15.17:0/2195853739 <== mon.2 v2:172.21.15.77:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0cc4043750 con 0x7f0cd4061190 2022-08-01T19:41:13.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:13.854+0000 7f0cd3fff700 1 -- 172.21.15.17:0/2195853739 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0cd40b1180 con 0x7f0cd4061190 2022-08-01T19:41:14.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.226+0000 7f0ccbfff700 1 -- 172.21.15.17:0/2195853739 <== mon.2 v2:172.21.15.77:3301/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v42) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0cc4016020 con 0x7f0cd4061190 2022-08-01T19:41:14.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.230+0000 7f0cd3fff700 1 -- 172.21.15.17:0/2195853739 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0cb8053aa0 msgr2=0x7f0cb8055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:14.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.230+0000 7f0cd3fff700 1 --2- 172.21.15.17:0/2195853739 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0cb8053aa0 0x7f0cb8055f60 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f0cbc00d000 tx=0x7f0cbc007b80).stop 2022-08-01T19:41:14.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.230+0000 7f0cd3fff700 1 -- 172.21.15.17:0/2195853739 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0cd4061190 msgr2=0x7f0cd40b68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:14.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.230+0000 7f0cd3fff700 1 --2- 172.21.15.17:0/2195853739 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0cd4061190 0x7f0cd40b68d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f0cc400c000 tx=0x7f0cc40091d0).stop 2022-08-01T19:41:14.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.230+0000 7f0cd3fff700 1 -- 172.21.15.17:0/2195853739 shutdown_connections 2022-08-01T19:41:14.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.230+0000 7f0cd3fff700 1 --2- 172.21.15.17:0/2195853739 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0cb8053aa0 0x7f0cb8055f60 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:14.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.230+0000 7f0cd3fff700 1 --2- 172.21.15.17:0/2195853739 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cd40b3190 0x7f0cd40b0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:14.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.230+0000 7f0cd3fff700 1 --2- 172.21.15.17:0/2195853739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0cd40b2730 0x7f0cd40aa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:14.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.230+0000 7f0cd3fff700 1 --2- 172.21.15.17:0/2195853739 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0cd4061190 0x7f0cd40b68d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:14.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.230+0000 7f0cd3fff700 1 -- 172.21.15.17:0/2195853739 >> 172.21.15.17:0/2195853739 conn(0x7f0cd4063870 msgr2=0x7f0cd4066510 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:14.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.230+0000 7f0cd3fff700 1 -- 172.21.15.17:0/2195853739 shutdown_connections 2022-08-01T19:41:14.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.230+0000 7f0cd3fff700 1 -- 172.21.15.17:0/2195853739 wait complete. 2022-08-01T19:41:14.253 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-08-01T19:41:14.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.362+0000 7f907759e700 1 Processor -- start 2022-08-01T19:41:14.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.362+0000 7f907759e700 1 -- start start 2022-08-01T19:41:14.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.366+0000 7f907759e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9078064b10 0x7f9078144e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:14.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.366+0000 7f907759e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9078141a80 0x7f9078066ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:14.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.366+0000 7f907759e700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9078064ef0 con 0x7f90780b36e0 2022-08-01T19:41:14.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.366+0000 7f907759e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f907813ebf0 con 0x7f9078141a80 2022-08-01T19:41:14.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.366+0000 7f907759e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9078066690 con 0x7f9078064b10 2022-08-01T19:41:14.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.366+0000 7f9075d9b700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f90780b36e0 0x7f9078141370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33870/0 (socket says 172.21.15.17:33870) 2022-08-01T19:41:14.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.366+0000 7f9075d9b700 1 -- 172.21.15.17:0/3953479946 learned_addr learned my addr 172.21.15.17:0/3953479946 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:14.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.366+0000 7f9076d9d700 1 --2- 172.21.15.17:0/3953479946 >> v2:172.21.15.17:3300/0 conn(0x7f9078141a80 0x7f9078066ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:14.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.366+0000 7f907659c700 1 --2- 172.21.15.17:0/3953479946 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9078064b10 0x7f9078144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:14.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.366+0000 7f907559a700 1 -- 172.21.15.17:0/3953479946 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3016665707 0 0) 0x7f9078064ef0 con 0x7f90780b36e0 2022-08-01T19:41:14.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.366+0000 7f907559a700 1 -- 172.21.15.17:0/3953479946 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f905c003260 con 0x7f90780b36e0 2022-08-01T19:41:14.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.366+0000 7f9076d9d700 1 -- 172.21.15.17:0/3953479946 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9078064b10 msgr2=0x7f9078144e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:14.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.366+0000 7f9076d9d700 1 --2- 172.21.15.17:0/3953479946 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9078064b10 0x7f9078144e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:14.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.366+0000 7f9076d9d700 1 -- 172.21.15.17:0/3953479946 >> v1:172.21.15.17:6789/0 conn(0x7f90780b36e0 legacy=0x7f9078141370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:14.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.366+0000 7f9076d9d700 1 -- 172.21.15.17:0/3953479946 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90780b4850 con 0x7f9078141a80 2022-08-01T19:41:14.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.366+0000 7f9076d9d700 1 --2- 172.21.15.17:0/3953479946 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9078141a80 0x7f9078066ac0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0x7f90700032f0 tx=0x7f90700091d0).ready entity=mon.0 client_cookie=3a48c4609586e47a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:14.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.366+0000 7f907559a700 1 -- 172.21.15.17:0/3953479946 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f907000b600 con 0x7f9078141a80 2022-08-01T19:41:14.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.366+0000 7f907559a700 1 -- 172.21.15.17:0/3953479946 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f907000b760 con 0x7f9078141a80 2022-08-01T19:41:14.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.366+0000 7f907559a700 1 -- 172.21.15.17:0/3953479946 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f907000b8d0 con 0x7f9078141a80 2022-08-01T19:41:14.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.366+0000 7f907759e700 1 -- 172.21.15.17:0/3953479946 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9078141a80 msgr2=0x7f9078066ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:14.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.366+0000 7f907759e700 1 --2- 172.21.15.17:0/3953479946 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9078141a80 0x7f9078066ac0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0x7f90700032f0 tx=0x7f90700091d0).stop 2022-08-01T19:41:14.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.366+0000 7f907759e700 1 -- 172.21.15.17:0/3953479946 shutdown_connections 2022-08-01T19:41:14.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.366+0000 7f907759e700 1 --2- 172.21.15.17:0/3953479946 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9078141a80 0x7f9078066ac0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:14.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.366+0000 7f907759e700 1 --2- 172.21.15.17:0/3953479946 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9078064b10 0x7f9078144e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:14.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.366+0000 7f907759e700 1 -- 172.21.15.17:0/3953479946 >> 172.21.15.17:0/3953479946 conn(0x7f907805f840 msgr2=0x7f907805fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:14.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.366+0000 7f907759e700 1 -- 172.21.15.17:0/3953479946 shutdown_connections 2022-08-01T19:41:14.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.366+0000 7f907759e700 1 -- 172.21.15.17:0/3953479946 wait complete. 2022-08-01T19:41:14.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.370+0000 7f907759e700 1 Processor -- start 2022-08-01T19:41:14.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.370+0000 7f907759e700 1 -- start start 2022-08-01T19:41:14.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.370+0000 7f907759e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9078064b10 0x7f90780afd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:14.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.370+0000 7f907759e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90780b36e0 0x7f907813b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:14.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.370+0000 7f907759e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9078141a80 0x7f907813df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:14.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.370+0000 7f907759e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90780b1140 con 0x7f9078064b10 2022-08-01T19:41:14.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.370+0000 7f907759e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90780b13e0 con 0x7f90780b36e0 2022-08-01T19:41:14.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.370+0000 7f907759e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90780b1680 con 0x7f9078141a80 2022-08-01T19:41:14.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.370+0000 7f907659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9078064b10 0x7f90780afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:14.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.370+0000 7f907659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9078064b10 0x7f90780afd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41682/0 (socket says 172.21.15.17:41682) 2022-08-01T19:41:14.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.370+0000 7f907659c700 1 -- 172.21.15.17:0/117795349 learned_addr learned my addr 172.21.15.17:0/117795349 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:14.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.370+0000 7f9076d9d700 1 --2- 172.21.15.17:0/117795349 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9078141a80 0x7f907813df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:14.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.370+0000 7f9075d9b700 1 --2- 172.21.15.17:0/117795349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90780b36e0 0x7f907813b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:14.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.370+0000 7f907659c700 1 -- 172.21.15.17:0/117795349 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9078141a80 msgr2=0x7f907813df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:14.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.370+0000 7f907659c700 1 --2- 172.21.15.17:0/117795349 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9078141a80 0x7f907813df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:14.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.370+0000 7f907659c700 1 -- 172.21.15.17:0/117795349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90780b36e0 msgr2=0x7f907813b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:14.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.370+0000 7f907659c700 1 --2- 172.21.15.17:0/117795349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90780b36e0 0x7f907813b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:14.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.370+0000 7f907659c700 1 -- 172.21.15.17:0/117795349 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9070009470 con 0x7f9078064b10 2022-08-01T19:41:14.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.370+0000 7f907659c700 1 --2- 172.21.15.17:0/117795349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9078064b10 0x7f90780afd40 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 rx=0x7f906c00c000 tx=0x7f906c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:14.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.370+0000 7f90677fe700 1 -- 172.21.15.17:0/117795349 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f906c011750 con 0x7f9078064b10 2022-08-01T19:41:14.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.370+0000 7f90677fe700 1 -- 172.21.15.17:0/117795349 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f906c0118b0 con 0x7f9078064b10 2022-08-01T19:41:14.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.370+0000 7f907759e700 1 -- 172.21.15.17:0/117795349 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90780b1920 con 0x7f9078064b10 2022-08-01T19:41:14.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.370+0000 7f90677fe700 1 -- 172.21.15.17:0/117795349 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f906c011c10 con 0x7f9078064b10 2022-08-01T19:41:14.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.370+0000 7f907759e700 1 -- 172.21.15.17:0/117795349 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f907813d0f0 con 0x7f9078064b10 2022-08-01T19:41:14.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.374+0000 7f907759e700 1 -- 172.21.15.17:0/117795349 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90780b0c80 con 0x7f9078064b10 2022-08-01T19:41:14.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.374+0000 7f90677fe700 1 -- 172.21.15.17:0/117795349 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f906c011d70 con 0x7f9078064b10 2022-08-01T19:41:14.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.374+0000 7f90677fe700 1 --2- 172.21.15.17:0/117795349 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9054053aa0 0x7f9054055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:14.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.374+0000 7f90677fe700 1 -- 172.21.15.17:0/117795349 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f906c06c460 con 0x7f9078064b10 2022-08-01T19:41:14.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.374+0000 7f9075d9b700 1 --2- 172.21.15.17:0/117795349 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9054053aa0 0x7f9054055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:14.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.374+0000 7f9075d9b700 1 --2- 172.21.15.17:0/117795349 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9054053aa0 0x7f9054055f60 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f906800d000 tx=0x7f9068009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:14.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.378+0000 7f90677fe700 1 -- 172.21.15.17:0/117795349 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f90780b0c80 con 0x7f9078064b10 2022-08-01T19:41:14.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:14.506+0000 7f907759e700 1 -- 172.21.15.17:0/117795349 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f90780b0c80 con 0x7f9078064b10 2022-08-01T19:41:15.234 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.230+0000 7f90677fe700 1 -- 172.21.15.17:0/117795349 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v90) v1 ==== 185+0+0 (secure 0 0 0) 0x7f90780b0c80 con 0x7f9078064b10 2022-08-01T19:41:15.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.230+0000 7f907759e700 1 -- 172.21.15.17:0/117795349 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9054053aa0 msgr2=0x7f9054055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:15.246 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.230+0000 7f907759e700 1 --2- 172.21.15.17:0/117795349 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9054053aa0 0x7f9054055f60 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f906800d000 tx=0x7f9068009040).stop 2022-08-01T19:41:15.246 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.230+0000 7f907759e700 1 -- 172.21.15.17:0/117795349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9078064b10 msgr2=0x7f90780afd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:15.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.230+0000 7f907759e700 1 --2- 172.21.15.17:0/117795349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9078064b10 0x7f90780afd40 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 rx=0x7f906c00c000 tx=0x7f906c006df0).stop 2022-08-01T19:41:15.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.234+0000 7f907759e700 1 -- 172.21.15.17:0/117795349 shutdown_connections 2022-08-01T19:41:15.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.234+0000 7f907759e700 1 --2- 172.21.15.17:0/117795349 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9054053aa0 0x7f9054055f60 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:15.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.234+0000 7f907759e700 1 --2- 172.21.15.17:0/117795349 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9078141a80 0x7f907813df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:15.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.234+0000 7f907759e700 1 --2- 172.21.15.17:0/117795349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90780b36e0 0x7f907813b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:15.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.234+0000 7f907759e700 1 --2- 172.21.15.17:0/117795349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9078064b10 0x7f90780afd40 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:15.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.234+0000 7f907759e700 1 -- 172.21.15.17:0/117795349 >> 172.21.15.17:0/117795349 conn(0x7f907805f840 msgr2=0x7f90780b5190 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:15.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.234+0000 7f907759e700 1 -- 172.21.15.17:0/117795349 shutdown_connections 2022-08-01T19:41:15.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.234+0000 7f907759e700 1 -- 172.21.15.17:0/117795349 wait complete. 2022-08-01T19:41:15.251 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' removed 2022-08-01T19:41:15.263 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-08-01T19:41:15.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.374+0000 7f99f69cb700 1 Processor -- start 2022-08-01T19:41:15.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.374+0000 7f99f69cb700 1 -- start start 2022-08-01T19:41:15.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.378+0000 7f99f69cb700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f99f00b7500 0x7f99f00b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:15.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.378+0000 7f99f69cb700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f99f00b31f0 0x7f99f013e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:15.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.378+0000 7f99f69cb700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f99f013e8a0 0x7f99f0064d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:15.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.378+0000 7f99f69cb700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99f00b2510 con 0x7f99f00b7500 2022-08-01T19:41:15.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.378+0000 7f99f69cb700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99f01443e0 con 0x7f99f013e8a0 2022-08-01T19:41:15.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.378+0000 7f99f69cb700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f00b3700 con 0x7f99f00b31f0 2022-08-01T19:41:15.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.378+0000 7f99f59c9700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f99f00b7500 0x7f99f00b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:15.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.378+0000 7f99f59c9700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f99f00b7500 0x7f99f00b78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41690/0 (socket says 172.21.15.17:41690) 2022-08-01T19:41:15.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.378+0000 7f99f59c9700 1 -- 172.21.15.17:0/1443227262 learned_addr learned my addr 172.21.15.17:0/1443227262 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:15.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.378+0000 7f99f51c8700 1 --2- 172.21.15.17:0/1443227262 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f99f00b31f0 0x7f99f013e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:15.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.378+0000 7f99f61ca700 1 --2- 172.21.15.17:0/1443227262 >> v2:172.21.15.77:3300/0 conn(0x7f99f013e8a0 0x7f99f0064d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:15.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.378+0000 7f99f59c9700 1 -- 172.21.15.17:0/1443227262 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f99f00b31f0 msgr2=0x7f99f013e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:15.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.378+0000 7f99f59c9700 1 --2- 172.21.15.17:0/1443227262 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f99f00b31f0 0x7f99f013e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:15.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.378+0000 7f99f59c9700 1 -- 172.21.15.17:0/1443227262 >> v2:172.21.15.77:3300/0 conn(0x7f99f013e8a0 msgr2=0x7f99f0064d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:15.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.378+0000 7f99f59c9700 1 --2- 172.21.15.17:0/1443227262 >> v2:172.21.15.77:3300/0 conn(0x7f99f013e8a0 0x7f99f0064d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:15.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.378+0000 7f99f59c9700 1 -- 172.21.15.17:0/1443227262 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f00b2740 con 0x7f99f00b7500 2022-08-01T19:41:15.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.378+0000 7f99f61ca700 1 --2- 172.21.15.17:0/1443227262 >> v2:172.21.15.77:3300/0 conn(0x7f99f013e8a0 0x7f99f0064d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:15.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.378+0000 7f99f59c9700 1 --2- 172.21.15.17:0/1443227262 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99f00b7500 0x7f99f00b78e0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 rx=0x7f99e0002f70 tx=0x7f99e0006eb0).ready entity=mon.0 client_cookie=33d2e2c2e5000dd3 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:15.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.378+0000 7f99f49c7700 1 -- 172.21.15.17:0/1443227262 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99e000d570 con 0x7f99f00b7500 2022-08-01T19:41:15.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.378+0000 7f99f49c7700 1 -- 172.21.15.17:0/1443227262 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e0000e90 con 0x7f99f00b7500 2022-08-01T19:41:15.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.378+0000 7f99f49c7700 1 -- 172.21.15.17:0/1443227262 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99e000d900 con 0x7f99f00b7500 2022-08-01T19:41:15.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.378+0000 7f99f69cb700 1 -- 172.21.15.17:0/1443227262 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99f00b7500 msgr2=0x7f99f00b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:15.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.378+0000 7f99f69cb700 1 --2- 172.21.15.17:0/1443227262 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99f00b7500 0x7f99f00b78e0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 rx=0x7f99e0002f70 tx=0x7f99e0006eb0).stop 2022-08-01T19:41:15.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.378+0000 7f99f69cb700 1 -- 172.21.15.17:0/1443227262 shutdown_connections 2022-08-01T19:41:15.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.378+0000 7f99f69cb700 1 --2- 172.21.15.17:0/1443227262 >> v2:172.21.15.77:3300/0 conn(0x7f99f013e8a0 0x7f99f0064d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:15.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.378+0000 7f99f69cb700 1 --2- 172.21.15.17:0/1443227262 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f99f00b31f0 0x7f99f013e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:15.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.378+0000 7f99f69cb700 1 --2- 172.21.15.17:0/1443227262 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99f00b7500 0x7f99f00b78e0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:15.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.378+0000 7f99f69cb700 1 -- 172.21.15.17:0/1443227262 >> 172.21.15.17:0/1443227262 conn(0x7f99f005f840 msgr2=0x7f99f005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:15.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.378+0000 7f99f69cb700 1 -- 172.21.15.17:0/1443227262 shutdown_connections 2022-08-01T19:41:15.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.378+0000 7f99f69cb700 1 -- 172.21.15.17:0/1443227262 wait complete. 2022-08-01T19:41:15.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.382+0000 7f99f69cb700 1 Processor -- start 2022-08-01T19:41:15.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.382+0000 7f99f69cb700 1 -- start start 2022-08-01T19:41:15.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.382+0000 7f99f69cb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99f00b31f0 0x7f99f01431f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:15.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.382+0000 7f99f69cb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99f00b7500 0x7f99f013fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:15.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.382+0000 7f99f69cb700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f99f013e8a0 0x7f99f0140230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:15.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.382+0000 7f99f69cb700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f013add0 con 0x7f99f00b7500 2022-08-01T19:41:15.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.382+0000 7f99f69cb700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f013cb60 con 0x7f99f00b31f0 2022-08-01T19:41:15.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.382+0000 7f99f69cb700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f013f480 con 0x7f99f013e8a0 2022-08-01T19:41:15.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.382+0000 7f99f51c8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99f00b7500 0x7f99f013fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:15.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.382+0000 7f99f51c8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99f00b7500 0x7f99f013fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41698/0 (socket says 172.21.15.17:41698) 2022-08-01T19:41:15.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.382+0000 7f99f51c8700 1 -- 172.21.15.17:0/1081725611 learned_addr learned my addr 172.21.15.17:0/1081725611 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:15.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.382+0000 7f99f61ca700 1 --2- 172.21.15.17:0/1081725611 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f99f013e8a0 0x7f99f0140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:15.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.382+0000 7f99f59c9700 1 --2- 172.21.15.17:0/1081725611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99f00b31f0 0x7f99f01431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:15.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.382+0000 7f99f51c8700 1 -- 172.21.15.17:0/1081725611 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f99f013e8a0 msgr2=0x7f99f0140230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:15.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.382+0000 7f99f51c8700 1 --2- 172.21.15.17:0/1081725611 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f99f013e8a0 0x7f99f0140230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:15.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.382+0000 7f99f51c8700 1 -- 172.21.15.17:0/1081725611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99f00b31f0 msgr2=0x7f99f01431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:15.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.382+0000 7f99f51c8700 1 --2- 172.21.15.17:0/1081725611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99f00b31f0 0x7f99f01431f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:15.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.382+0000 7f99f51c8700 1 -- 172.21.15.17:0/1081725611 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e8009470 con 0x7f99f00b7500 2022-08-01T19:41:15.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.386+0000 7f99f51c8700 1 --2- 172.21.15.17:0/1081725611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99f00b7500 0x7f99f013fd40 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0x7f99ec00c000 tx=0x7f99ec006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:15.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.386+0000 7f99deffd700 1 -- 172.21.15.17:0/1081725611 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99ec010780 con 0x7f99f00b7500 2022-08-01T19:41:15.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.386+0000 7f99deffd700 1 -- 172.21.15.17:0/1081725611 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99ec0108e0 con 0x7f99f00b7500 2022-08-01T19:41:15.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.386+0000 7f99f69cb700 1 -- 172.21.15.17:0/1081725611 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e000b1e0 con 0x7f99f00b7500 2022-08-01T19:41:15.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.386+0000 7f99deffd700 1 -- 172.21.15.17:0/1081725611 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99ec010c40 con 0x7f99f00b7500 2022-08-01T19:41:15.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.386+0000 7f99f69cb700 1 -- 172.21.15.17:0/1081725611 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f0143e30 con 0x7f99f00b7500 2022-08-01T19:41:15.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.386+0000 7f99deffd700 1 -- 172.21.15.17:0/1081725611 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f99ec01c460 con 0x7f99f00b7500 2022-08-01T19:41:15.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.386+0000 7f99f69cb700 1 -- 172.21.15.17:0/1081725611 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99f0141d20 con 0x7f99f00b7500 2022-08-01T19:41:15.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.386+0000 7f99deffd700 1 --2- 172.21.15.17:0/1081725611 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f99d0053a70 0x7f99d0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:15.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.386+0000 7f99deffd700 1 -- 172.21.15.17:0/1081725611 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f99ec06c420 con 0x7f99f00b7500 2022-08-01T19:41:15.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.386+0000 7f99f59c9700 1 --2- 172.21.15.17:0/1081725611 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f99d0053a70 0x7f99d0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:15.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.386+0000 7f99f59c9700 1 --2- 172.21.15.17:0/1081725611 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f99d0053a70 0x7f99d0055f30 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f99e001a000 tx=0x7f99e0014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:15.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.394+0000 7f99deffd700 1 -- 172.21.15.17:0/1081725611 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f99ec043240 con 0x7f99f00b7500 2022-08-01T19:41:15.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:15.522+0000 7f99f69cb700 1 -- 172.21.15.17:0/1081725611 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f99f013cd20 con 0x7f99f00b7500 2022-08-01T19:41:16.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.242+0000 7f99deffd700 1 -- 172.21.15.17:0/1081725611 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v91) v1 ==== 173+0+0 (secure 0 0 0) 0x7f99ec046860 con 0x7f99f00b7500 2022-08-01T19:41:16.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.246+0000 7f99f69cb700 1 -- 172.21.15.17:0/1081725611 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f99d0053a70 msgr2=0x7f99d0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:16.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.246+0000 7f99f69cb700 1 --2- 172.21.15.17:0/1081725611 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f99d0053a70 0x7f99d0055f30 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f99e001a000 tx=0x7f99e0014000).stop 2022-08-01T19:41:16.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.246+0000 7f99f69cb700 1 -- 172.21.15.17:0/1081725611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99f00b7500 msgr2=0x7f99f013fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:16.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.246+0000 7f99f69cb700 1 --2- 172.21.15.17:0/1081725611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99f00b7500 0x7f99f013fd40 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0x7f99ec00c000 tx=0x7f99ec006df0).stop 2022-08-01T19:41:16.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.246+0000 7f99f69cb700 1 -- 172.21.15.17:0/1081725611 shutdown_connections 2022-08-01T19:41:16.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.246+0000 7f99f69cb700 1 --2- 172.21.15.17:0/1081725611 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f99d0053a70 0x7f99d0055f30 secure :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 rx=0x7f99e001a000 tx=0x7f99e0014000).stop 2022-08-01T19:41:16.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.246+0000 7f99f69cb700 1 --2- 172.21.15.17:0/1081725611 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f99f013e8a0 0x7f99f0140230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:16.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.246+0000 7f99f69cb700 1 --2- 172.21.15.17:0/1081725611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99f00b7500 0x7f99f013fd40 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:16.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.246+0000 7f99f69cb700 1 --2- 172.21.15.17:0/1081725611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99f00b31f0 0x7f99f01431f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:16.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.246+0000 7f99f69cb700 1 -- 172.21.15.17:0/1081725611 >> 172.21.15.17:0/1081725611 conn(0x7f99f005f840 msgr2=0x7f99f00b64f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:16.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.246+0000 7f99f69cb700 1 -- 172.21.15.17:0/1081725611 shutdown_connections 2022-08-01T19:41:16.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.246+0000 7f99f69cb700 1 -- 172.21.15.17:0/1081725611 wait complete. 2022-08-01T19:41:16.272 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' removed 2022-08-01T19:41:16.276 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-08-01T19:41:16.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.386+0000 7f905359e700 1 Processor -- start 2022-08-01T19:41:16.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.386+0000 7f905359e700 1 -- start start 2022-08-01T19:41:16.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.390+0000 7f905359e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f90540b0d10 0x7f9054146df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:16.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.390+0000 7f905359e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f90540b0560 0x7f9054140350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:16.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.390+0000 7f905359e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90540b12d0 con 0x7f90540a9320 2022-08-01T19:41:16.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.390+0000 7f905359e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90540a9700 con 0x7f90540b0d10 2022-08-01T19:41:16.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.390+0000 7f905359e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90540b1500 con 0x7f90540b0560 2022-08-01T19:41:16.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.390+0000 7f9051d9b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f90540b0d10 0x7f9054146df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:16.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.390+0000 7f9051d9b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f90540b0d10 0x7f9054146df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41706/0 (socket says 172.21.15.17:41706) 2022-08-01T19:41:16.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.390+0000 7f9051d9b700 1 -- 172.21.15.17:0/4033583709 learned_addr learned my addr 172.21.15.17:0/4033583709 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:16.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.390+0000 7f9052d9d700 1 --2- 172.21.15.17:0/4033583709 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f90540b0560 0x7f9054140350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:16.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.390+0000 7f9051d9b700 1 -- 172.21.15.17:0/4033583709 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f90540b0560 msgr2=0x7f9054140350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:16.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.390+0000 7f9051d9b700 1 --2- 172.21.15.17:0/4033583709 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f90540b0560 0x7f9054140350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:16.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.390+0000 7f9051d9b700 1 -- 172.21.15.17:0/4033583709 >> v1:172.21.15.77:6789/0 conn(0x7f90540a9320 legacy=0x7f90541466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:16.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.390+0000 7f9051d9b700 1 -- 172.21.15.17:0/4033583709 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90540b17a0 con 0x7f90540b0d10 2022-08-01T19:41:16.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.390+0000 7f9051d9b700 1 --2- 172.21.15.17:0/4033583709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90540b0d10 0x7f9054146df0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 rx=0x7f9044002f70 tx=0x7f9044006eb0).ready entity=mon.0 client_cookie=e1cc955ef5964290 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:16.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.390+0000 7f905159a700 1 -- 172.21.15.17:0/4033583709 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f904400d570 con 0x7f90540b0d10 2022-08-01T19:41:16.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.390+0000 7f905159a700 1 -- 172.21.15.17:0/4033583709 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9044000e90 con 0x7f90540b0d10 2022-08-01T19:41:16.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.390+0000 7f905159a700 1 -- 172.21.15.17:0/4033583709 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f904400d900 con 0x7f90540b0d10 2022-08-01T19:41:16.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.390+0000 7f905359e700 1 -- 172.21.15.17:0/4033583709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90540b0d10 msgr2=0x7f9054146df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:16.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.390+0000 7f905359e700 1 --2- 172.21.15.17:0/4033583709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90540b0d10 0x7f9054146df0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 rx=0x7f9044002f70 tx=0x7f9044006eb0).stop 2022-08-01T19:41:16.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.390+0000 7f905359e700 1 -- 172.21.15.17:0/4033583709 shutdown_connections 2022-08-01T19:41:16.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.390+0000 7f905359e700 1 --2- 172.21.15.17:0/4033583709 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f90540b0560 0x7f9054140350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:16.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.390+0000 7f905359e700 1 --2- 172.21.15.17:0/4033583709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90540b0d10 0x7f9054146df0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:16.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.390+0000 7f905359e700 1 -- 172.21.15.17:0/4033583709 >> 172.21.15.17:0/4033583709 conn(0x7f90540612d0 msgr2=0x7f9054062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:16.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.390+0000 7f905359e700 1 -- 172.21.15.17:0/4033583709 shutdown_connections 2022-08-01T19:41:16.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.390+0000 7f905359e700 1 -- 172.21.15.17:0/4033583709 wait complete. 2022-08-01T19:41:16.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.394+0000 7f905359e700 1 Processor -- start 2022-08-01T19:41:16.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.394+0000 7f905359e700 1 -- start start 2022-08-01T19:41:16.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.394+0000 7f905359e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f90540a9320 0x7f90540673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:16.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.394+0000 7f905359e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90540b0560 0x7f90540678b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:16.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.394+0000 7f905359e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90540b0d10 0x7f9054065480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:16.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.394+0000 7f905359e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9054067da0 con 0x7f90540b0560 2022-08-01T19:41:16.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.394+0000 7f905359e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9054153310 con 0x7f90540b0d10 2022-08-01T19:41:16.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.394+0000 7f905359e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90540b3d40 con 0x7f90540a9320 2022-08-01T19:41:16.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.394+0000 7f9051d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90540b0560 0x7f90540678b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:16.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.394+0000 7f9051d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90540b0560 0x7f90540678b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41712/0 (socket says 172.21.15.17:41712) 2022-08-01T19:41:16.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.394+0000 7f9051d9b700 1 -- 172.21.15.17:0/823757843 learned_addr learned my addr 172.21.15.17:0/823757843 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:16.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.394+0000 7f9052d9d700 1 --2- 172.21.15.17:0/823757843 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90540b0d10 0x7f9054065480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:16.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.394+0000 7f905259c700 1 --2- 172.21.15.17:0/823757843 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f90540a9320 0x7f90540673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:16.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.394+0000 7f9051d9b700 1 -- 172.21.15.17:0/823757843 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f90540a9320 msgr2=0x7f90540673c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:16.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.394+0000 7f9051d9b700 1 --2- 172.21.15.17:0/823757843 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f90540a9320 0x7f90540673c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:16.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.394+0000 7f9051d9b700 1 -- 172.21.15.17:0/823757843 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90540b0d10 msgr2=0x7f9054065480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:16.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.394+0000 7f9051d9b700 1 --2- 172.21.15.17:0/823757843 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90540b0d10 0x7f9054065480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:16.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.394+0000 7f9051d9b700 1 -- 172.21.15.17:0/823757843 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f904400b1e0 con 0x7f90540b0560 2022-08-01T19:41:16.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.394+0000 7f9051d9b700 1 --2- 172.21.15.17:0/823757843 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90540b0560 0x7f90540678b0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 rx=0x7f9044002f70 tx=0x7f9044015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:16.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.394+0000 7f90437fe700 1 -- 172.21.15.17:0/823757843 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90440068b0 con 0x7f90540b0560 2022-08-01T19:41:16.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.398+0000 7f905359e700 1 -- 172.21.15.17:0/823757843 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90540b3fe0 con 0x7f90540b0560 2022-08-01T19:41:16.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.398+0000 7f90437fe700 1 -- 172.21.15.17:0/823757843 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904400d8f0 con 0x7f90540b0560 2022-08-01T19:41:16.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.398+0000 7f90437fe700 1 -- 172.21.15.17:0/823757843 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9044011d70 con 0x7f90540b0560 2022-08-01T19:41:16.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.398+0000 7f905359e700 1 -- 172.21.15.17:0/823757843 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9054063960 con 0x7f90540b0560 2022-08-01T19:41:16.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.398+0000 7f905359e700 1 -- 172.21.15.17:0/823757843 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9054153510 con 0x7f90540b0560 2022-08-01T19:41:16.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.398+0000 7f90437fe700 1 -- 172.21.15.17:0/823757843 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f904401a040 con 0x7f90540b0560 2022-08-01T19:41:16.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.398+0000 7f90437fe700 1 --2- 172.21.15.17:0/823757843 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9030053980 0x7f9030055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:16.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.398+0000 7f90437fe700 1 -- 172.21.15.17:0/823757843 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f904406ca00 con 0x7f90540b0560 2022-08-01T19:41:16.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.398+0000 7f905259c700 1 --2- 172.21.15.17:0/823757843 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9030053980 0x7f9030055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:16.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.402+0000 7f905259c700 1 --2- 172.21.15.17:0/823757843 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9030053980 0x7f9030055e40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f904800d000 tx=0x7f9048007b70).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:16.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.406+0000 7f90437fe700 1 -- 172.21.15.17:0/823757843 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9044043ab0 con 0x7f90540b0560 2022-08-01T19:41:16.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.534+0000 7f905359e700 1 -- 172.21.15.17:0/823757843 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f90540574e0 con 0x7f90540b0560 2022-08-01T19:41:16.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.542+0000 7f90437fe700 1 -- 172.21.15.17:0/823757843 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f90440470d0 con 0x7f90540b0560 2022-08-01T19:41:16.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.546+0000 7f905359e700 1 -- 172.21.15.17:0/823757843 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9030053980 msgr2=0x7f9030055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:16.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.546+0000 7f905359e700 1 --2- 172.21.15.17:0/823757843 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9030053980 0x7f9030055e40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f904800d000 tx=0x7f9048007b70).stop 2022-08-01T19:41:16.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.546+0000 7f905359e700 1 -- 172.21.15.17:0/823757843 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90540b0560 msgr2=0x7f90540678b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:16.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.546+0000 7f905359e700 1 --2- 172.21.15.17:0/823757843 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90540b0560 0x7f90540678b0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 rx=0x7f9044002f70 tx=0x7f9044015000).stop 2022-08-01T19:41:16.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.546+0000 7f905359e700 1 -- 172.21.15.17:0/823757843 shutdown_connections 2022-08-01T19:41:16.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.546+0000 7f905359e700 1 --2- 172.21.15.17:0/823757843 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9030053980 0x7f9030055e40 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:16.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.546+0000 7f905359e700 1 --2- 172.21.15.17:0/823757843 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90540b0d10 0x7f9054065480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:16.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.546+0000 7f905359e700 1 --2- 172.21.15.17:0/823757843 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90540b0560 0x7f90540678b0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:16.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.546+0000 7f905359e700 1 --2- 172.21.15.17:0/823757843 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f90540a9320 0x7f90540673c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:16.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.546+0000 7f905359e700 1 -- 172.21.15.17:0/823757843 >> 172.21.15.17:0/823757843 conn(0x7f90540612d0 msgr2=0x7f905413f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:16.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.546+0000 7f905359e700 1 -- 172.21.15.17:0/823757843 shutdown_connections 2022-08-01T19:41:16.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.546+0000 7f905359e700 1 -- 172.21.15.17:0/823757843 wait complete. 2022-08-01T19:41:16.565 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode' 2022-08-01T19:41:16.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.666+0000 7f263bfff700 1 Processor -- start 2022-08-01T19:41:16.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.666+0000 7f263bfff700 1 -- start start 2022-08-01T19:41:16.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.666+0000 7f263bfff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f263c064d30 0x7f263c0b7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:16.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.666+0000 7f263bfff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f263c0b2160 0x7f263c061b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:16.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.666+0000 7f263bfff700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f263c066d90 con 0x7f263c062080 2022-08-01T19:41:16.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.666+0000 7f263bfff700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f263c0662e0 con 0x7f263c064d30 2022-08-01T19:41:16.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.666+0000 7f263bfff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f263c065110 con 0x7f263c0b2160 2022-08-01T19:41:16.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.670+0000 7f263b7fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f263c062080 0x7f263c0b2c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33912/0 (socket says 172.21.15.17:33912) 2022-08-01T19:41:16.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.670+0000 7f263b7fe700 1 -- 172.21.15.17:0/2866111699 learned_addr learned my addr 172.21.15.17:0/2866111699 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:16.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.670+0000 7f263affd700 1 --2- 172.21.15.17:0/2866111699 >> v2:172.21.15.17:3300/0 conn(0x7f263c064d30 0x7f263c0b7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:16.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.670+0000 7f2633fff700 1 --2- 172.21.15.17:0/2866111699 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f263c0b2160 0x7f263c061b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:16.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.670+0000 7f263a7fc700 1 -- 172.21.15.17:0/2866111699 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1276685593 0 0) 0x7f263c066d90 con 0x7f263c062080 2022-08-01T19:41:16.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.670+0000 7f263a7fc700 1 -- 172.21.15.17:0/2866111699 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f261c003260 con 0x7f263c062080 2022-08-01T19:41:16.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.670+0000 7f263affd700 1 -- 172.21.15.17:0/2866111699 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f263c0b2160 msgr2=0x7f263c061b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:16.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.670+0000 7f263affd700 1 --2- 172.21.15.17:0/2866111699 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f263c0b2160 0x7f263c061b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:16.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.670+0000 7f263affd700 1 -- 172.21.15.17:0/2866111699 >> v1:172.21.15.17:6789/0 conn(0x7f263c062080 legacy=0x7f263c0b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:16.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.670+0000 7f263affd700 1 -- 172.21.15.17:0/2866111699 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f263c067030 con 0x7f263c064d30 2022-08-01T19:41:16.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.670+0000 7f263affd700 1 --2- 172.21.15.17:0/2866111699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f263c064d30 0x7f263c0b7fb0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7f262c002f70 tx=0x7f262c006df0).ready entity=mon.0 client_cookie=9dd585a4e31e7e23 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:16.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.670+0000 7f263a7fc700 1 -- 172.21.15.17:0/2866111699 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f262c00d4a0 con 0x7f263c064d30 2022-08-01T19:41:16.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.670+0000 7f263a7fc700 1 -- 172.21.15.17:0/2866111699 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f262c000e90 con 0x7f263c064d30 2022-08-01T19:41:16.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.670+0000 7f263a7fc700 1 -- 172.21.15.17:0/2866111699 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f262c00d7f0 con 0x7f263c064d30 2022-08-01T19:41:16.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.670+0000 7f263bfff700 1 -- 172.21.15.17:0/2866111699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f263c064d30 msgr2=0x7f263c0b7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:16.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.670+0000 7f263bfff700 1 --2- 172.21.15.17:0/2866111699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f263c064d30 0x7f263c0b7fb0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7f262c002f70 tx=0x7f262c006df0).stop 2022-08-01T19:41:16.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.670+0000 7f263bfff700 1 -- 172.21.15.17:0/2866111699 shutdown_connections 2022-08-01T19:41:16.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.670+0000 7f263bfff700 1 --2- 172.21.15.17:0/2866111699 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f263c0b2160 0x7f263c061b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:16.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.670+0000 7f263bfff700 1 --2- 172.21.15.17:0/2866111699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f263c064d30 0x7f263c0b7fb0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:16.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.670+0000 7f263bfff700 1 -- 172.21.15.17:0/2866111699 >> 172.21.15.17:0/2866111699 conn(0x7f263c062d70 msgr2=0x7f263c05f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:16.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.670+0000 7f263bfff700 1 -- 172.21.15.17:0/2866111699 shutdown_connections 2022-08-01T19:41:16.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.670+0000 7f263bfff700 1 -- 172.21.15.17:0/2866111699 wait complete. 2022-08-01T19:41:16.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.674+0000 7f263bfff700 1 Processor -- start 2022-08-01T19:41:16.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.674+0000 7f263bfff700 1 -- start start 2022-08-01T19:41:16.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.674+0000 7f263bfff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f263c062080 0x7f263c0b6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:16.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.674+0000 7f263bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f263c064d30 0x7f263c0b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:16.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.674+0000 7f263bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f263c0b2160 0x7f263c0b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:16.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.674+0000 7f263bfff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263c0adfc0 con 0x7f263c064d30 2022-08-01T19:41:16.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.674+0000 7f263bfff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263c0ae1f0 con 0x7f263c0b2160 2022-08-01T19:41:16.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.674+0000 7f263bfff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f263c0ae490 con 0x7f263c062080 2022-08-01T19:41:16.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.674+0000 7f2633fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f263c064d30 0x7f263c0b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:16.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.674+0000 7f2633fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f263c064d30 0x7f263c0b08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41724/0 (socket says 172.21.15.17:41724) 2022-08-01T19:41:16.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.674+0000 7f2633fff700 1 -- 172.21.15.17:0/2606606789 learned_addr learned my addr 172.21.15.17:0/2606606789 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:16.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.674+0000 7f263affd700 1 --2- 172.21.15.17:0/2606606789 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f263c062080 0x7f263c0b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:16.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.674+0000 7f263b7fe700 1 --2- 172.21.15.17:0/2606606789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f263c0b2160 0x7f263c0b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:16.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.674+0000 7f2633fff700 1 -- 172.21.15.17:0/2606606789 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f263c062080 msgr2=0x7f263c0b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:16.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.674+0000 7f2633fff700 1 --2- 172.21.15.17:0/2606606789 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f263c062080 0x7f263c0b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:16.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.674+0000 7f2633fff700 1 -- 172.21.15.17:0/2606606789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f263c0b2160 msgr2=0x7f263c0b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:16.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.674+0000 7f2633fff700 1 --2- 172.21.15.17:0/2606606789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f263c0b2160 0x7f263c0b0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:16.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.674+0000 7f2633fff700 1 -- 172.21.15.17:0/2606606789 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f262c00b180 con 0x7f263c064d30 2022-08-01T19:41:16.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.674+0000 7f263b7fe700 1 --2- 172.21.15.17:0/2606606789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f263c0b2160 0x7f263c0b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:16.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.674+0000 7f2633fff700 1 --2- 172.21.15.17:0/2606606789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f263c064d30 0x7f263c0b08a0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 rx=0x7f262400c000 tx=0x7f2624006d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:16.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.674+0000 7f2638ff9700 1 -- 172.21.15.17:0/2606606789 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2624011750 con 0x7f263c064d30 2022-08-01T19:41:16.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.674+0000 7f2638ff9700 1 -- 172.21.15.17:0/2606606789 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26240118b0 con 0x7f263c064d30 2022-08-01T19:41:16.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.674+0000 7f2638ff9700 1 -- 172.21.15.17:0/2606606789 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2624011c10 con 0x7f263c064d30 2022-08-01T19:41:16.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.674+0000 7f263bfff700 1 -- 172.21.15.17:0/2606606789 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f263c0ae730 con 0x7f263c064d30 2022-08-01T19:41:16.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.674+0000 7f263bfff700 1 -- 172.21.15.17:0/2606606789 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f263c0ad180 con 0x7f263c064d30 2022-08-01T19:41:16.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.678+0000 7f263bfff700 1 -- 172.21.15.17:0/2606606789 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f263c0b0640 con 0x7f263c064d30 2022-08-01T19:41:16.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.678+0000 7f2638ff9700 1 -- 172.21.15.17:0/2606606789 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2624011d70 con 0x7f263c064d30 2022-08-01T19:41:16.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.678+0000 7f2638ff9700 1 --2- 172.21.15.17:0/2606606789 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2620053950 0x7f2620055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:16.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.678+0000 7f2638ff9700 1 -- 172.21.15.17:0/2606606789 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f262406c100 con 0x7f263c064d30 2022-08-01T19:41:16.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.678+0000 7f263affd700 1 --2- 172.21.15.17:0/2606606789 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2620053950 0x7f2620055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:16.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.678+0000 7f263affd700 1 --2- 172.21.15.17:0/2606606789 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2620053950 0x7f2620055e10 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f262c01a000 tx=0x7f262c014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:16.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.686+0000 7f2638ff9700 1 -- 172.21.15.17:0/2606606789 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2624043070 con 0x7f263c064d30 2022-08-01T19:41:16.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:16.854+0000 7f263bfff700 1 -- 172.21.15.17:0/2606606789 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]} v 0) v1 -- 0x7f263c0672d0 con 0x7f263c064d30 2022-08-01T19:41:17.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.246+0000 7f2638ff9700 1 -- 172.21.15.17:0/2606606789 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]}]=0 v315) v1 ==== 126+0+0 (secure 0 0 0) 0x7f2624046690 con 0x7f263c064d30 2022-08-01T19:41:17.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.250+0000 7f263bfff700 1 -- 172.21.15.17:0/2606606789 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2620053950 msgr2=0x7f2620055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:17.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.250+0000 7f263bfff700 1 --2- 172.21.15.17:0/2606606789 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2620053950 0x7f2620055e10 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f262c01a000 tx=0x7f262c014000).stop 2022-08-01T19:41:17.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.250+0000 7f263bfff700 1 -- 172.21.15.17:0/2606606789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f263c064d30 msgr2=0x7f263c0b08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:17.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.250+0000 7f263bfff700 1 --2- 172.21.15.17:0/2606606789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f263c064d30 0x7f263c0b08a0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 rx=0x7f262400c000 tx=0x7f2624006d90).stop 2022-08-01T19:41:17.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.250+0000 7f263bfff700 1 -- 172.21.15.17:0/2606606789 shutdown_connections 2022-08-01T19:41:17.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.250+0000 7f263bfff700 1 --2- 172.21.15.17:0/2606606789 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2620053950 0x7f2620055e10 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:17.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.250+0000 7f263bfff700 1 --2- 172.21.15.17:0/2606606789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f263c0b2160 0x7f263c0b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:17.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.250+0000 7f263bfff700 1 --2- 172.21.15.17:0/2606606789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f263c064d30 0x7f263c0b08a0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:17.271 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.250+0000 7f263bfff700 1 --2- 172.21.15.17:0/2606606789 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f263c062080 0x7f263c0b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:17.271 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.250+0000 7f263bfff700 1 -- 172.21.15.17:0/2606606789 >> 172.21.15.17:0/2606606789 conn(0x7f263c062d70 msgr2=0x7f263c060950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:17.271 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.250+0000 7f263bfff700 1 -- 172.21.15.17:0/2606606789 shutdown_connections 2022-08-01T19:41:17.271 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.250+0000 7f263bfff700 1 -- 172.21.15.17:0/2606606789 wait complete. 2022-08-01T19:41:17.274 INFO:tasks.cephfs_test_runner:test_dump_inode (tasks.cephfs.test_misc.TestMisc) ... ok 2022-08-01T19:41:17.274 INFO:tasks.cephfs_test_runner:Starting test: test_dump_inode_hexademical (tasks.cephfs.test_misc.TestMisc) 2022-08-01T19:41:17.275 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical' 2022-08-01T19:41:17.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.378+0000 7fccb6c94700 1 Processor -- start 2022-08-01T19:41:17.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.378+0000 7fccb6c94700 1 -- start start 2022-08-01T19:41:17.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.378+0000 7fccb6c94700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fccb0064b10 0x7fccb0144e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:17.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.378+0000 7fccb6c94700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fccb00b36e0 0x7fccb0141370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:17.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.378+0000 7fccb6c94700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fccb0141860 0x7fccb0066ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:17.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.378+0000 7fccb6c94700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccb013ebf0 con 0x7fccb0064b10 2022-08-01T19:41:17.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.378+0000 7fccb6c94700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccb0064ef0 con 0x7fccb00b36e0 2022-08-01T19:41:17.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.378+0000 7fccb6c94700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb0066690 con 0x7fccb0141860 2022-08-01T19:41:17.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.378+0000 7fccb5c92700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fccb0064b10 0x7fccb0144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:17.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.378+0000 7fccb5c92700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fccb0064b10 0x7fccb0144e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41730/0 (socket says 172.21.15.17:41730) 2022-08-01T19:41:17.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.378+0000 7fccb5c92700 1 -- 172.21.15.17:0/3762866238 learned_addr learned my addr 172.21.15.17:0/3762866238 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:17.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.378+0000 7fccb6493700 1 --2- 172.21.15.17:0/3762866238 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fccb0141860 0x7fccb0066ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:17.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.378+0000 7fccb5491700 1 --2- 172.21.15.17:0/3762866238 >> v2:172.21.15.77:3300/0 conn(0x7fccb00b36e0 0x7fccb0141370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:17.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.378+0000 7fccb5c92700 1 -- 172.21.15.17:0/3762866238 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fccb0141860 msgr2=0x7fccb0066ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:17.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.378+0000 7fccb5c92700 1 --2- 172.21.15.17:0/3762866238 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fccb0141860 0x7fccb0066ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:17.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.378+0000 7fccb5c92700 1 -- 172.21.15.17:0/3762866238 >> v2:172.21.15.77:3300/0 conn(0x7fccb00b36e0 msgr2=0x7fccb0141370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:17.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.378+0000 7fccb5c92700 1 --2- 172.21.15.17:0/3762866238 >> v2:172.21.15.77:3300/0 conn(0x7fccb00b36e0 0x7fccb0141370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:17.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.378+0000 7fccb5c92700 1 -- 172.21.15.17:0/3762866238 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb00b4850 con 0x7fccb0064b10 2022-08-01T19:41:17.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.378+0000 7fccb5491700 1 --2- 172.21.15.17:0/3762866238 >> v2:172.21.15.77:3300/0 conn(0x7fccb00b36e0 0x7fccb0141370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:17.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.378+0000 7fccb5c92700 1 --2- 172.21.15.17:0/3762866238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fccb0064b10 0x7fccb0144e60 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0x7fcca80032f0 tx=0x7fcca8009290).ready entity=mon.0 client_cookie=3efe025bb0a28682 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:17.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.378+0000 7fccb4c90700 1 -- 172.21.15.17:0/3762866238 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcca800b870 con 0x7fccb0064b10 2022-08-01T19:41:17.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.378+0000 7fccb4c90700 1 -- 172.21.15.17:0/3762866238 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca800b9d0 con 0x7fccb0064b10 2022-08-01T19:41:17.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.378+0000 7fccb4c90700 1 -- 172.21.15.17:0/3762866238 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcca800bb40 con 0x7fccb0064b10 2022-08-01T19:41:17.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.378+0000 7fccb6c94700 1 -- 172.21.15.17:0/3762866238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fccb0064b10 msgr2=0x7fccb0144e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:17.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.378+0000 7fccb6c94700 1 --2- 172.21.15.17:0/3762866238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fccb0064b10 0x7fccb0144e60 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0x7fcca80032f0 tx=0x7fcca8009290).stop 2022-08-01T19:41:17.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.378+0000 7fccb6c94700 1 -- 172.21.15.17:0/3762866238 shutdown_connections 2022-08-01T19:41:17.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.378+0000 7fccb6c94700 1 --2- 172.21.15.17:0/3762866238 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fccb0141860 0x7fccb0066ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:17.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.378+0000 7fccb6c94700 1 --2- 172.21.15.17:0/3762866238 >> v2:172.21.15.77:3300/0 conn(0x7fccb00b36e0 0x7fccb0141370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:17.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.378+0000 7fccb6c94700 1 --2- 172.21.15.17:0/3762866238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fccb0064b10 0x7fccb0144e60 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:17.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.378+0000 7fccb6c94700 1 -- 172.21.15.17:0/3762866238 >> 172.21.15.17:0/3762866238 conn(0x7fccb005f840 msgr2=0x7fccb005fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:17.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.378+0000 7fccb6c94700 1 -- 172.21.15.17:0/3762866238 shutdown_connections 2022-08-01T19:41:17.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.378+0000 7fccb6c94700 1 -- 172.21.15.17:0/3762866238 wait complete. 2022-08-01T19:41:17.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.382+0000 7fccb6c94700 1 Processor -- start 2022-08-01T19:41:17.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.382+0000 7fccb6c94700 1 -- start start 2022-08-01T19:41:17.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.382+0000 7fccb6c94700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccb0064b10 0x7fccb00acfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:17.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.382+0000 7fccb6c94700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fccb00b36e0 0x7fccb0155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:17.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.382+0000 7fccb6c94700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fccb0141860 0x7fccb0155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:17.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.382+0000 7fccb6c94700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb0143b30 con 0x7fccb00b36e0 2022-08-01T19:41:17.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.382+0000 7fccb6c94700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb0142ff0 con 0x7fccb0064b10 2022-08-01T19:41:17.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.382+0000 7fccb6c94700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb0144ab0 con 0x7fccb0141860 2022-08-01T19:41:17.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.382+0000 7fccb5491700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fccb00b36e0 0x7fccb0155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:17.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.382+0000 7fccb5c92700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccb0064b10 0x7fccb00acfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:17.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.382+0000 7fccb6493700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fccb0141860 0x7fccb0155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:17.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.382+0000 7fccb5491700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fccb00b36e0 0x7fccb0155a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41740/0 (socket says 172.21.15.17:41740) 2022-08-01T19:41:17.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.382+0000 7fccb5c92700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccb0064b10 0x7fccb00acfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:39658/0 (socket says 172.21.15.17:39658) 2022-08-01T19:41:17.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.382+0000 7fccb5491700 1 -- 172.21.15.17:0/1207590339 learned_addr learned my addr 172.21.15.17:0/1207590339 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:17.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.382+0000 7fccb5491700 1 -- 172.21.15.17:0/1207590339 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fccb0141860 msgr2=0x7fccb0155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:17.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.382+0000 7fccb5491700 1 --2- 172.21.15.17:0/1207590339 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fccb0141860 0x7fccb0155f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:17.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.382+0000 7fccb5491700 1 -- 172.21.15.17:0/1207590339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccb0064b10 msgr2=0x7fccb00acfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:17.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.382+0000 7fccb5491700 1 --2- 172.21.15.17:0/1207590339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccb0064b10 0x7fccb00acfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:17.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.382+0000 7fccb5491700 1 -- 172.21.15.17:0/1207590339 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca8009530 con 0x7fccb00b36e0 2022-08-01T19:41:17.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.386+0000 7fccb5491700 1 --2- 172.21.15.17:0/1207590339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fccb00b36e0 0x7fccb0155a80 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 rx=0x7fcca4006940 tx=0x7fcca4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:17.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.386+0000 7fcca37fe700 1 -- 172.21.15.17:0/1207590339 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcca4011780 con 0x7fccb00b36e0 2022-08-01T19:41:17.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.386+0000 7fcca37fe700 1 -- 172.21.15.17:0/1207590339 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca40118e0 con 0x7fccb00b36e0 2022-08-01T19:41:17.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.386+0000 7fccb6c94700 1 -- 172.21.15.17:0/1207590339 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccb013b6a0 con 0x7fccb00b36e0 2022-08-01T19:41:17.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.386+0000 7fcca37fe700 1 -- 172.21.15.17:0/1207590339 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcca4011c40 con 0x7fccb00b36e0 2022-08-01T19:41:17.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.386+0000 7fccb6c94700 1 -- 172.21.15.17:0/1207590339 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb0144610 con 0x7fccb00b36e0 2022-08-01T19:41:17.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.386+0000 7fccb6c94700 1 -- 172.21.15.17:0/1207590339 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccb00b4af0 con 0x7fccb00b36e0 2022-08-01T19:41:17.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.386+0000 7fcca37fe700 1 -- 172.21.15.17:0/1207590339 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcca4012420 con 0x7fccb00b36e0 2022-08-01T19:41:17.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.386+0000 7fcca37fe700 1 --2- 172.21.15.17:0/1207590339 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcc8c053950 0x7fcc8c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:17.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.386+0000 7fcca37fe700 1 -- 172.21.15.17:0/1207590339 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fcca406bc20 con 0x7fccb00b36e0 2022-08-01T19:41:17.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.390+0000 7fccb5c92700 1 --2- 172.21.15.17:0/1207590339 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcc8c053950 0x7fcc8c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:17.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.390+0000 7fccb5c92700 1 --2- 172.21.15.17:0/1207590339 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcc8c053950 0x7fcc8c055e10 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7fcca801a000 tx=0x7fcca8016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:17.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.390+0000 7fcca37fe700 1 -- 172.21.15.17:0/1207590339 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcca4042b50 con 0x7fccb00b36e0 2022-08-01T19:41:17.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:17.562+0000 7fccb6c94700 1 -- 172.21.15.17:0/1207590339 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]} v 0) v1 -- 0x7fccb00574e0 con 0x7fccb00b36e0 2022-08-01T19:41:18.099 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:41:18.105 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:41:18.109 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:41:18.115 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:41:18.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.250+0000 7fcca37fe700 1 -- 172.21.15.17:0/1207590339 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]}]=0 v316) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcca4046170 con 0x7fccb00b36e0 2022-08-01T19:41:18.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.254+0000 7fccb6c94700 1 -- 172.21.15.17:0/1207590339 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcc8c053950 msgr2=0x7fcc8c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:18.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.254+0000 7fccb6c94700 1 --2- 172.21.15.17:0/1207590339 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcc8c053950 0x7fcc8c055e10 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7fcca801a000 tx=0x7fcca8016000).stop 2022-08-01T19:41:18.266 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.254+0000 7fccb6c94700 1 -- 172.21.15.17:0/1207590339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fccb00b36e0 msgr2=0x7fccb0155a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:18.266 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.254+0000 7fccb6c94700 1 --2- 172.21.15.17:0/1207590339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fccb00b36e0 0x7fccb0155a80 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 rx=0x7fcca4006940 tx=0x7fcca4008040).stop 2022-08-01T19:41:18.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.254+0000 7fccb6c94700 1 -- 172.21.15.17:0/1207590339 shutdown_connections 2022-08-01T19:41:18.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.254+0000 7fccb6c94700 1 --2- 172.21.15.17:0/1207590339 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcc8c053950 0x7fcc8c055e10 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:18.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.254+0000 7fccb6c94700 1 --2- 172.21.15.17:0/1207590339 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fccb0141860 0x7fccb0155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:18.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.254+0000 7fccb6c94700 1 --2- 172.21.15.17:0/1207590339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fccb00b36e0 0x7fccb0155a80 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:18.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.254+0000 7fccb6c94700 1 --2- 172.21.15.17:0/1207590339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccb0064b10 0x7fccb00acfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:18.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.254+0000 7fccb6c94700 1 -- 172.21.15.17:0/1207590339 >> 172.21.15.17:0/1207590339 conn(0x7fccb005f840 msgr2=0x7fccb0140500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:18.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.254+0000 7fccb6c94700 1 -- 172.21.15.17:0/1207590339 shutdown_connections 2022-08-01T19:41:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.254+0000 7fccb6c94700 1 -- 172.21.15.17:0/1207590339 wait complete. 2022-08-01T19:41:18.278 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-08-01T19:41:18.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.390+0000 7f4071cf9700 1 Processor -- start 2022-08-01T19:41:18.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.390+0000 7f4071cf9700 1 -- start start 2022-08-01T19:41:18.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.390+0000 7f4071cf9700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f406c0b4310 0x7f406c140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:18.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.390+0000 7f4071cf9700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f406c143cf0 con 0x7f406c0b4820 2022-08-01T19:41:18.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.390+0000 7f4071cf9700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f406c143f90 con 0x7f406c146ae0 2022-08-01T19:41:18.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.390+0000 7f4071cf9700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f406c1476a0 con 0x7f406c0b4310 2022-08-01T19:41:18.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.390+0000 7f406bfff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f406c0b4820 0x7f406c0b5060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33942/0 (socket says 172.21.15.17:33942) 2022-08-01T19:41:18.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.390+0000 7f406bfff700 1 -- 172.21.15.17:0/3543394638 learned_addr learned my addr 172.21.15.17:0/3543394638 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:18.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.390+0000 7f406affd700 1 --2- 172.21.15.17:0/3543394638 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f406c0b4310 0x7f406c140ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:18.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.390+0000 7f406a7fc700 1 -- 172.21.15.17:0/3543394638 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2809718026 0 0) 0x7f406c143cf0 con 0x7f406c0b4820 2022-08-01T19:41:18.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.390+0000 7f406a7fc700 1 -- 172.21.15.17:0/3543394638 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4058003260 con 0x7f406c0b4820 2022-08-01T19:41:18.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.390+0000 7f406a7fc700 1 -- 172.21.15.17:0/3543394638 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2998122811 0 0) 0x7f406c143f90 con 0x7f406c146ae0 2022-08-01T19:41:18.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.390+0000 7f406a7fc700 1 -- 172.21.15.17:0/3543394638 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f406c143cf0 con 0x7f406c146ae0 2022-08-01T19:41:18.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.390+0000 7f406affd700 1 -- 172.21.15.17:0/3543394638 >> v1:172.21.15.77:6789/0 conn(0x7f406c146ae0 legacy=0x7f406c1403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:18.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.390+0000 7f406affd700 1 -- 172.21.15.17:0/3543394638 >> v1:172.21.15.17:6789/0 conn(0x7f406c0b4820 legacy=0x7f406c0b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:18.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.390+0000 7f406affd700 1 -- 172.21.15.17:0/3543394638 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f406c0b0e20 con 0x7f406c0b4310 2022-08-01T19:41:18.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.390+0000 7f406affd700 1 --2- 172.21.15.17:0/3543394638 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f406c0b4310 0x7f406c140ad0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f4060002f70 tx=0x7f4060006df0).ready entity=mon.2 client_cookie=9e2fb8e6256bd3ef server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:18.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.394+0000 7f406a7fc700 1 -- 172.21.15.17:0/3543394638 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f406000d5c0 con 0x7f406c0b4310 2022-08-01T19:41:18.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.394+0000 7f406a7fc700 1 -- 172.21.15.17:0/3543394638 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4060000e90 con 0x7f406c0b4310 2022-08-01T19:41:18.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.394+0000 7f406a7fc700 1 -- 172.21.15.17:0/3543394638 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f406000d910 con 0x7f406c0b4310 2022-08-01T19:41:18.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.394+0000 7f4071cf9700 1 -- 172.21.15.17:0/3543394638 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f406c0b4310 msgr2=0x7f406c140ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:18.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.394+0000 7f4071cf9700 1 --2- 172.21.15.17:0/3543394638 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f406c0b4310 0x7f406c140ad0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f4060002f70 tx=0x7f4060006df0).stop 2022-08-01T19:41:18.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.394+0000 7f4071cf9700 1 -- 172.21.15.17:0/3543394638 shutdown_connections 2022-08-01T19:41:18.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.394+0000 7f4071cf9700 1 --2- 172.21.15.17:0/3543394638 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f406c0b4310 0x7f406c140ad0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:18.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.394+0000 7f4071cf9700 1 -- 172.21.15.17:0/3543394638 >> 172.21.15.17:0/3543394638 conn(0x7f406c05c620 msgr2=0x7f406c05ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:18.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.394+0000 7f4071cf9700 1 -- 172.21.15.17:0/3543394638 shutdown_connections 2022-08-01T19:41:18.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.394+0000 7f4071cf9700 1 -- 172.21.15.17:0/3543394638 wait complete. 2022-08-01T19:41:18.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.394+0000 7f4071cf9700 1 Processor -- start 2022-08-01T19:41:18.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.394+0000 7f4071cf9700 1 -- start start 2022-08-01T19:41:18.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.394+0000 7f4071cf9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406c0b4310 0x7f406c14a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:18.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.394+0000 7f4071cf9700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f406c0b4820 0x7f406c152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:18.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.394+0000 7f4071cf9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f406c146ae0 0x7f406c142210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:18.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.394+0000 7f4071cf9700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406c05f130 con 0x7f406c0b4310 2022-08-01T19:41:18.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.394+0000 7f4071cf9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406c142dd0 con 0x7f406c146ae0 2022-08-01T19:41:18.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.394+0000 7f4071cf9700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f406c143070 con 0x7f406c0b4820 2022-08-01T19:41:18.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.394+0000 7f406b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406c0b4310 0x7f406c14a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:18.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.398+0000 7f406b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406c0b4310 0x7f406c14a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41750/0 (socket says 172.21.15.17:41750) 2022-08-01T19:41:18.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.398+0000 7f406b7fe700 1 -- 172.21.15.17:0/1762881946 learned_addr learned my addr 172.21.15.17:0/1762881946 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:18.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.398+0000 7f406bfff700 1 --2- 172.21.15.17:0/1762881946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f406c146ae0 0x7f406c142210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:18.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.398+0000 7f406affd700 1 --2- 172.21.15.17:0/1762881946 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f406c0b4820 0x7f406c152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:18.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.398+0000 7f406b7fe700 1 -- 172.21.15.17:0/1762881946 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f406c0b4820 msgr2=0x7f406c152450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:18.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.398+0000 7f406b7fe700 1 --2- 172.21.15.17:0/1762881946 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f406c0b4820 0x7f406c152450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:18.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.398+0000 7f406b7fe700 1 -- 172.21.15.17:0/1762881946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f406c146ae0 msgr2=0x7f406c142210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:18.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.398+0000 7f406b7fe700 1 --2- 172.21.15.17:0/1762881946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f406c146ae0 0x7f406c142210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:18.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.398+0000 7f406b7fe700 1 -- 172.21.15.17:0/1762881946 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f406000b180 con 0x7f406c0b4310 2022-08-01T19:41:18.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.398+0000 7f406b7fe700 1 --2- 172.21.15.17:0/1762881946 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406c0b4310 0x7f406c14a290 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7f405c001110 tx=0x7f405c007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:18.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.398+0000 7f4068ff9700 1 -- 172.21.15.17:0/1762881946 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f405c012400 con 0x7f406c0b4310 2022-08-01T19:41:18.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.398+0000 7f4068ff9700 1 -- 172.21.15.17:0/1762881946 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f405c012560 con 0x7f406c0b4310 2022-08-01T19:41:18.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.398+0000 7f4071cf9700 1 -- 172.21.15.17:0/1762881946 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f406c143310 con 0x7f406c0b4310 2022-08-01T19:41:18.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.398+0000 7f4068ff9700 1 -- 172.21.15.17:0/1762881946 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f405c0128c0 con 0x7f406c0b4310 2022-08-01T19:41:18.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.398+0000 7f4071cf9700 1 -- 172.21.15.17:0/1762881946 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f406c142700 con 0x7f406c0b4310 2022-08-01T19:41:18.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.398+0000 7f4071cf9700 1 -- 172.21.15.17:0/1762881946 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f406c0adcc0 con 0x7f406c0b4310 2022-08-01T19:41:18.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.398+0000 7f4068ff9700 1 -- 172.21.15.17:0/1762881946 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f405c019030 con 0x7f406c0b4310 2022-08-01T19:41:18.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.398+0000 7f4068ff9700 1 --2- 172.21.15.17:0/1762881946 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f404c053950 0x7f404c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:18.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.402+0000 7f4068ff9700 1 -- 172.21.15.17:0/1762881946 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f405c06b080 con 0x7f406c0b4310 2022-08-01T19:41:18.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.402+0000 7f406affd700 1 --2- 172.21.15.17:0/1762881946 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f404c053950 0x7f404c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:18.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.402+0000 7f406affd700 1 --2- 172.21.15.17:0/1762881946 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f404c053950 0x7f404c055e10 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f406001a000 tx=0x7f4060014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:18.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.406+0000 7f4068ff9700 1 -- 172.21.15.17:0/1762881946 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f406c0adcc0 con 0x7f406c0b4310 2022-08-01T19:41:18.540 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.534+0000 7f4071cf9700 1 -- 172.21.15.17:0/1762881946 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f406c0adcc0 con 0x7f406c0b4310 2022-08-01T19:41:18.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.538+0000 7f4068ff9700 1 -- 172.21.15.17:0/1762881946 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f406c0adcc0 con 0x7f406c0b4310 2022-08-01T19:41:18.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.542+0000 7f4071cf9700 1 -- 172.21.15.17:0/1762881946 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f404c053950 msgr2=0x7f404c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:18.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.542+0000 7f4071cf9700 1 --2- 172.21.15.17:0/1762881946 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f404c053950 0x7f404c055e10 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f406001a000 tx=0x7f4060014000).stop 2022-08-01T19:41:18.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.542+0000 7f4071cf9700 1 -- 172.21.15.17:0/1762881946 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406c0b4310 msgr2=0x7f406c14a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:18.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.542+0000 7f4071cf9700 1 --2- 172.21.15.17:0/1762881946 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406c0b4310 0x7f406c14a290 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7f405c001110 tx=0x7f405c007ae0).stop 2022-08-01T19:41:18.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.542+0000 7f4071cf9700 1 -- 172.21.15.17:0/1762881946 shutdown_connections 2022-08-01T19:41:18.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.542+0000 7f4071cf9700 1 --2- 172.21.15.17:0/1762881946 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f404c053950 0x7f404c055e10 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:18.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.542+0000 7f4071cf9700 1 --2- 172.21.15.17:0/1762881946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f406c146ae0 0x7f406c142210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:18.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.542+0000 7f4071cf9700 1 --2- 172.21.15.17:0/1762881946 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f406c0b4820 0x7f406c152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:18.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.542+0000 7f4071cf9700 1 --2- 172.21.15.17:0/1762881946 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406c0b4310 0x7f406c14a290 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:18.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.542+0000 7f4071cf9700 1 -- 172.21.15.17:0/1762881946 >> 172.21.15.17:0/1762881946 conn(0x7f406c05c620 msgr2=0x7f406c0b5060 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:18.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.542+0000 7f4071cf9700 1 -- 172.21.15.17:0/1762881946 shutdown_connections 2022-08-01T19:41:18.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.542+0000 7f4071cf9700 1 -- 172.21.15.17:0/1762881946 wait complete. 2022-08-01T19:41:18.559 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T19:41:18.561 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:41:18.584 DEBUG:teuthology.orchestra.run.smithi077:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:41:18.612 DEBUG:teuthology.orchestra.run.smithi085:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:41:18.640 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:41:18.663 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T19:41:18.663 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:18.664 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:41:18.664 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:41:18.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:41:18.679 INFO:teuthology.orchestra.run.smithi085.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-08-01T19:41:18.680 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-08-01T19:41:18.681 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:18.683 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:41:18.684 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:41:18.698 INFO:teuthology.orchestra.run.smithi190.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T19:41:18.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:41:18.699 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-08-01T19:41:18.700 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T19:41:18.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.806+0000 7fda75462700 1 Processor -- start 2022-08-01T19:41:18.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.806+0000 7fda75462700 1 -- start start 2022-08-01T19:41:18.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.810+0000 7fda75462700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fda700b2710 0x7fda701401e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:18.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.810+0000 7fda75462700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fda700b0270 0x7fda701406d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:18.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.810+0000 7fda75462700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda70067a10 con 0x7fda700ac880 2022-08-01T19:41:18.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.810+0000 7fda75462700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda700ad740 con 0x7fda700b0270 2022-08-01T19:41:18.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.810+0000 7fda75462700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda70067cb0 con 0x7fda700b2710 2022-08-01T19:41:18.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.810+0000 7fda74c61700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fda700ac880 0x7fda70143000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33956/0 (socket says 172.21.15.17:33956) 2022-08-01T19:41:18.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.810+0000 7fda74c61700 1 -- 172.21.15.17:0/88810837 learned_addr learned my addr 172.21.15.17:0/88810837 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:18.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.810+0000 7fda677fe700 1 --2- 172.21.15.17:0/88810837 >> v2:172.21.15.77:3300/0 conn(0x7fda700b0270 0x7fda701406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:18.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.810+0000 7fda6ffff700 1 --2- 172.21.15.17:0/88810837 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fda700b2710 0x7fda701401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:18.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.810+0000 7fda6f7fe700 1 -- 172.21.15.17:0/88810837 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 92008215 0 0) 0x7fda70067a10 con 0x7fda700ac880 2022-08-01T19:41:18.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.810+0000 7fda6f7fe700 1 -- 172.21.15.17:0/88810837 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda50003260 con 0x7fda700ac880 2022-08-01T19:41:18.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.810+0000 7fda6ffff700 1 -- 172.21.15.17:0/88810837 >> v2:172.21.15.77:3300/0 conn(0x7fda700b0270 msgr2=0x7fda701406d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:18.834 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.810+0000 7fda6ffff700 1 --2- 172.21.15.17:0/88810837 >> v2:172.21.15.77:3300/0 conn(0x7fda700b0270 0x7fda701406d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:18.834 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.810+0000 7fda6ffff700 1 -- 172.21.15.17:0/88810837 >> v1:172.21.15.17:6789/0 conn(0x7fda700ac880 legacy=0x7fda70143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:18.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.810+0000 7fda6ffff700 1 -- 172.21.15.17:0/88810837 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda70147160 con 0x7fda700b2710 2022-08-01T19:41:18.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.810+0000 7fda677fe700 1 --2- 172.21.15.17:0/88810837 >> v2:172.21.15.77:3300/0 conn(0x7fda700b0270 0x7fda701406d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:18.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.810+0000 7fda6ffff700 1 --2- 172.21.15.17:0/88810837 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fda700b2710 0x7fda701401e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fda60002f70 tx=0x7fda60006df0).ready entity=mon.2 client_cookie=6ffdb41fb283eab9 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:18.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.810+0000 7fda6f7fe700 1 -- 172.21.15.17:0/88810837 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda6000d4a0 con 0x7fda700b2710 2022-08-01T19:41:18.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.810+0000 7fda6f7fe700 1 -- 172.21.15.17:0/88810837 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda60000e90 con 0x7fda700b2710 2022-08-01T19:41:18.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.810+0000 7fda6f7fe700 1 -- 172.21.15.17:0/88810837 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda6000d7f0 con 0x7fda700b2710 2022-08-01T19:41:18.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.810+0000 7fda75462700 1 -- 172.21.15.17:0/88810837 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fda700b2710 msgr2=0x7fda701401e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:18.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.810+0000 7fda75462700 1 --2- 172.21.15.17:0/88810837 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fda700b2710 0x7fda701401e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fda60002f70 tx=0x7fda60006df0).stop 2022-08-01T19:41:18.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.810+0000 7fda75462700 1 -- 172.21.15.17:0/88810837 shutdown_connections 2022-08-01T19:41:18.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.810+0000 7fda75462700 1 --2- 172.21.15.17:0/88810837 >> v2:172.21.15.77:3300/0 conn(0x7fda700b0270 0x7fda701406d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:18.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.810+0000 7fda75462700 1 --2- 172.21.15.17:0/88810837 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fda700b2710 0x7fda701401e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:18.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.810+0000 7fda75462700 1 -- 172.21.15.17:0/88810837 >> 172.21.15.17:0/88810837 conn(0x7fda70060310 msgr2=0x7fda700b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:18.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.810+0000 7fda75462700 1 -- 172.21.15.17:0/88810837 shutdown_connections 2022-08-01T19:41:18.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.810+0000 7fda75462700 1 -- 172.21.15.17:0/88810837 wait complete. 2022-08-01T19:41:18.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.814+0000 7fda75462700 1 Processor -- start 2022-08-01T19:41:18.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.814+0000 7fda75462700 1 -- start start 2022-08-01T19:41:18.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.818+0000 7fda75462700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fda700ac880 0x7fda70143240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:18.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.818+0000 7fda75462700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda700b0270 0x7fda700ac200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:18.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.818+0000 7fda75462700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda700b2710 0x7fda70145860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:18.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.818+0000 7fda75462700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda70143ac0 con 0x7fda700b0270 2022-08-01T19:41:18.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.818+0000 7fda75462700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda700a9b40 con 0x7fda700b2710 2022-08-01T19:41:18.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.818+0000 7fda75462700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda70143cf0 con 0x7fda700ac880 2022-08-01T19:41:18.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.818+0000 7fda677fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda700b0270 0x7fda700ac200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:18.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.818+0000 7fda677fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda700b0270 0x7fda700ac200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41766/0 (socket says 172.21.15.17:41766) 2022-08-01T19:41:18.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.818+0000 7fda677fe700 1 -- 172.21.15.17:0/3376195734 learned_addr learned my addr 172.21.15.17:0/3376195734 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:18.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.818+0000 7fda6ffff700 1 --2- 172.21.15.17:0/3376195734 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fda700ac880 0x7fda70143240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:18.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.818+0000 7fda74c61700 1 --2- 172.21.15.17:0/3376195734 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda700b2710 0x7fda70145860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:18.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.818+0000 7fda677fe700 1 -- 172.21.15.17:0/3376195734 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fda700ac880 msgr2=0x7fda70143240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:18.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.818+0000 7fda677fe700 1 --2- 172.21.15.17:0/3376195734 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fda700ac880 0x7fda70143240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:18.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.818+0000 7fda677fe700 1 -- 172.21.15.17:0/3376195734 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda700b2710 msgr2=0x7fda70145860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:18.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.818+0000 7fda677fe700 1 --2- 172.21.15.17:0/3376195734 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda700b2710 0x7fda70145860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:18.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.818+0000 7fda677fe700 1 -- 172.21.15.17:0/3376195734 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5800b180 con 0x7fda700b0270 2022-08-01T19:41:18.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.818+0000 7fda677fe700 1 --2- 172.21.15.17:0/3376195734 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda700b0270 0x7fda700ac200 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 rx=0x7fda5800c000 tx=0x7fda58006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:18.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.818+0000 7fda6dffb700 1 -- 172.21.15.17:0/3376195734 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda58010780 con 0x7fda700b0270 2022-08-01T19:41:18.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.818+0000 7fda6dffb700 1 -- 172.21.15.17:0/3376195734 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda580108e0 con 0x7fda700b0270 2022-08-01T19:41:18.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.818+0000 7fda75462700 1 -- 172.21.15.17:0/3376195734 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda6000b180 con 0x7fda700b0270 2022-08-01T19:41:18.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.818+0000 7fda6dffb700 1 -- 172.21.15.17:0/3376195734 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda58010c40 con 0x7fda700b0270 2022-08-01T19:41:18.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.818+0000 7fda75462700 1 -- 172.21.15.17:0/3376195734 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda700a9340 con 0x7fda700b0270 2022-08-01T19:41:18.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.818+0000 7fda75462700 1 -- 172.21.15.17:0/3376195734 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda700574e0 con 0x7fda700b0270 2022-08-01T19:41:18.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.818+0000 7fda6dffb700 1 -- 172.21.15.17:0/3376195734 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fda5801c460 con 0x7fda700b0270 2022-08-01T19:41:18.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.822+0000 7fda6dffb700 1 --2- 172.21.15.17:0/3376195734 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fda54053950 0x7fda54055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:18.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.822+0000 7fda6dffb700 1 -- 172.21.15.17:0/3376195734 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fda5806c2a0 con 0x7fda700b0270 2022-08-01T19:41:18.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.822+0000 7fda6ffff700 1 --2- 172.21.15.17:0/3376195734 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fda54053950 0x7fda54055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:18.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.822+0000 7fda6ffff700 1 --2- 172.21.15.17:0/3376195734 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fda54053950 0x7fda54055e10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7fda6001a000 tx=0x7fda60014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:18.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.826+0000 7fda6dffb700 1 -- 172.21.15.17:0/3376195734 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fda580432e0 con 0x7fda700b0270 2022-08-01T19:41:18.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.954+0000 7fda75462700 1 -- 172.21.15.17:0/3376195734 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fda700b1bb0 con 0x7fda700b0270 2022-08-01T19:41:18.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.958+0000 7fda6dffb700 1 -- 172.21.15.17:0/3376195734 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v91) v1 ==== 81+0+29220 (secure 0 0 0) 0x7fda58046900 con 0x7fda700b0270 2022-08-01T19:41:18.969 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:18.970 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:41:18.970 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 91, 2022-08-01T19:41:18.971 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", 2022-08-01T19:41:18.972 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2022-08-01T19:33:50.105743+0000", 2022-08-01T19:41:18.972 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2022-08-01T19:41:16.242321+0000", 2022-08-01T19:41:18.973 INFO:teuthology.orchestra.run.smithi017.stdout: "last_up_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:41:18.973 INFO:teuthology.orchestra.run.smithi017.stdout: "last_in_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:41:18.974 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T19:41:18.974 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_num": 5799936, 2022-08-01T19:41:18.975 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_set": [ 2022-08-01T19:41:18.975 INFO:teuthology.orchestra.run.smithi017.stdout: "pglog_hardlimit", 2022-08-01T19:41:18.976 INFO:teuthology.orchestra.run.smithi017.stdout: "purged_snapdirs", 2022-08-01T19:41:18.976 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_deletes", 2022-08-01T19:41:18.976 INFO:teuthology.orchestra.run.smithi017.stdout: "sortbitwise" 2022-08-01T19:41:18.977 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:41:18.977 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_version": 6, 2022-08-01T19:41:18.983 INFO:teuthology.orchestra.run.smithi017.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T19:41:18.983 INFO:teuthology.orchestra.run.smithi017.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T19:41:18.984 INFO:teuthology.orchestra.run.smithi017.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T19:41:18.984 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_snapshot": "", 2022-08-01T19:41:18.984 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_max": 16, 2022-08-01T19:41:18.985 INFO:teuthology.orchestra.run.smithi017.stdout: "max_osd": 8, 2022-08-01T19:41:18.985 INFO:teuthology.orchestra.run.smithi017.stdout: "require_min_compat_client": "jewel", 2022-08-01T19:41:18.986 INFO:teuthology.orchestra.run.smithi017.stdout: "min_compat_client": "jewel", 2022-08-01T19:41:18.986 INFO:teuthology.orchestra.run.smithi017.stdout: "require_osd_release": "octopus", 2022-08-01T19:41:18.987 INFO:teuthology.orchestra.run.smithi017.stdout: "pools": [ 2022-08-01T19:41:18.987 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:18.988 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 1, 2022-08-01T19:41:18.989 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "device_health_metrics", 2022-08-01T19:41:18.989 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:33:53.001841+0000", 2022-08-01T19:41:18.990 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:41:18.990 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:41:18.991 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:41:18.993 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:41:18.994 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:41:18.994 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:41:18.994 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:41:18.995 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:41:18.995 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 1, 2022-08-01T19:41:18.995 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 1, 2022-08-01T19:41:18.995 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 1, 2022-08-01T19:41:18.996 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 1, 2022-08-01T19:41:18.996 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 1, 2022-08-01T19:41:18.996 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:41:18.996 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:41:18.996 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:41:18.997 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:41:18.997 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:41:18.997 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:41:18.997 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:41:18.998 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:18.998 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "16", 2022-08-01T19:41:18.998 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:41:18.999 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:41:19.000 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:41:19.000 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:41:19.001 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:41:19.001 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:41:19.002 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:41:19.002 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:41:19.002 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:41:19.003 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:41:19.003 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:41:19.004 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:41:19.004 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:41:19.005 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:41:19.006 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:41:19.006 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:41:19.007 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:41:19.008 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:41:19.008 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:41:19.009 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:41:19.010 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:41:19.011 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:41:19.011 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:41:19.012 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:41:19.012 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:41:19.019 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:41:19.020 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.020 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:41:19.021 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:41:19.021 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:41:19.021 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:41:19.023 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:41:19.023 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:41:19.024 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:41:19.024 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:41:19.024 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:41:19.025 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:41:19.025 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:41:19.025 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:41:19.026 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 1 2022-08-01T19:41:19.026 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.027 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:41:19.027 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_devicehealth": {} 2022-08-01T19:41:19.027 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.028 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.028 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.029 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 2, 2022-08-01T19:41:19.029 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "rbd", 2022-08-01T19:41:19.029 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:34:01.553098+0000", 2022-08-01T19:41:19.030 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:41:19.030 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:41:19.031 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:41:19.031 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:41:19.032 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:41:19.032 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:41:19.033 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:41:19.034 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:41:19.035 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:41:19.035 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:41:19.036 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:41:19.036 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:41:19.037 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:41:19.037 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:41:19.038 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:41:19.038 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:41:19.039 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:41:19.039 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:41:19.040 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:41:19.040 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:41:19.041 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.041 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "19", 2022-08-01T19:41:19.042 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:41:19.042 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:41:19.043 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:41:19.043 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:41:19.044 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:41:19.045 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:41:19.046 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:41:19.046 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:41:19.047 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:41:19.047 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:41:19.048 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:41:19.048 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:41:19.049 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:41:19.049 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:41:19.050 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:41:19.050 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:41:19.051 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:41:19.051 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:41:19.052 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:41:19.052 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:41:19.053 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:41:19.054 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:41:19.055 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:41:19.055 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:41:19.056 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:41:19.056 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:41:19.056 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.056 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:41:19.057 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:41:19.058 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:41:19.058 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:41:19.058 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:41:19.059 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:41:19.059 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:41:19.060 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:41:19.060 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:41:19.060 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:41:19.061 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:41:19.061 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:41:19.062 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:41:19.062 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd": {} 2022-08-01T19:41:19.063 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.063 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.063 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:41:19.064 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2022-08-01T19:41:19.064 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.065 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:41:19.065 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a962f14f-cf16-4fa8-84ff-c85325db9132", 2022-08-01T19:41:19.066 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:19.066 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:19.066 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:19.067 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:19.067 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:19.068 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:19.068 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:41:19.069 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 86, 2022-08-01T19:41:19.069 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:19.069 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:19.070 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:19.070 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:19.070 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.071 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:19.071 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6800", 2022-08-01T19:41:19.072 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:19.072 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.073 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.073 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:19.074 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6801", 2022-08-01T19:41:19.074 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:19.075 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.075 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.076 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.076 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:19.076 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:19.077 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.077 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:19.078 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6802", 2022-08-01T19:41:19.078 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:19.079 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.080 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.080 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:19.081 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6803", 2022-08-01T19:41:19.081 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:19.082 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.082 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.083 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.083 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:19.084 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:19.084 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.084 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:19.085 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6806", 2022-08-01T19:41:19.085 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:19.086 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.086 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.087 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:19.087 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6807", 2022-08-01T19:41:19.088 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:19.088 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.088 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.089 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.089 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:19.090 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:19.091 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.091 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:19.092 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6804", 2022-08-01T19:41:19.092 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:19.092 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.093 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.093 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:19.094 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6805", 2022-08-01T19:41:19.094 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:19.094 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.095 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.095 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.096 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6801/23008", 2022-08-01T19:41:19.096 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6803/23008", 2022-08-01T19:41:19.096 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6807/23008", 2022-08-01T19:41:19.097 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6805/23008", 2022-08-01T19:41:19.097 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:19.098 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:19.098 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:19.099 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.099 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.100 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.100 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:41:19.102 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "805f7051-a527-4e98-8844-b6e406a63248", 2022-08-01T19:41:19.102 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:19.103 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:19.103 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:19.104 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:19.104 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:19.105 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:19.105 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:41:19.106 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 85, 2022-08-01T19:41:19.106 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:19.106 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:19.107 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:19.107 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:19.107 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.108 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:19.108 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6808", 2022-08-01T19:41:19.109 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:19.109 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.109 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.110 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:19.114 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6809", 2022-08-01T19:41:19.115 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:19.115 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.116 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.116 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.116 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:19.117 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:19.117 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.118 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:19.119 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6810", 2022-08-01T19:41:19.119 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:19.119 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.120 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.120 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:19.121 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6811", 2022-08-01T19:41:19.121 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:19.122 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.122 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.123 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.123 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:19.124 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:19.124 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.125 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:19.125 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6814", 2022-08-01T19:41:19.129 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:19.130 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.130 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.131 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:19.131 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6815", 2022-08-01T19:41:19.132 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:19.132 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.133 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.133 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.134 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:19.134 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:19.135 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.135 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:19.136 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6812", 2022-08-01T19:41:19.136 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:19.136 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.137 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.137 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:19.138 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6813", 2022-08-01T19:41:19.139 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:19.139 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.139 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.141 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.141 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6809/23007", 2022-08-01T19:41:19.141 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6811/23007", 2022-08-01T19:41:19.141 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6815/23007", 2022-08-01T19:41:19.142 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6813/23007", 2022-08-01T19:41:19.143 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:19.143 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:19.143 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:19.143 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.144 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.144 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.144 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:41:19.144 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "89f7f3df-7c10-48e1-8b69-9e4243923a61", 2022-08-01T19:41:19.145 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:19.145 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:19.145 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:19.145 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:19.146 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:19.146 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:19.148 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:41:19.149 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 86, 2022-08-01T19:41:19.149 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:19.149 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:19.150 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:19.150 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:19.151 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.151 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:19.152 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6817", 2022-08-01T19:41:19.152 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:19.153 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.153 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.154 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:19.154 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6819", 2022-08-01T19:41:19.154 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:19.155 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.155 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.156 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.157 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:19.157 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:19.159 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.159 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:19.160 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6822", 2022-08-01T19:41:19.160 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:19.161 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.161 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.161 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:19.162 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6824", 2022-08-01T19:41:19.162 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:19.163 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.163 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.163 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.164 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:19.165 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:19.165 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.165 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:19.166 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6830", 2022-08-01T19:41:19.166 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:19.167 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.167 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.168 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:19.168 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6831", 2022-08-01T19:41:19.169 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:19.169 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.170 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.170 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.170 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:19.171 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:19.171 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.172 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:19.172 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6826", 2022-08-01T19:41:19.173 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:19.173 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.173 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.174 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:19.174 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6829", 2022-08-01T19:41:19.175 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:19.175 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.176 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.176 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.177 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6819/23010", 2022-08-01T19:41:19.177 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6824/23010", 2022-08-01T19:41:19.178 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6831/23010", 2022-08-01T19:41:19.178 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6829/23010", 2022-08-01T19:41:19.179 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:19.179 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:19.179 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:19.180 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.180 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.181 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:41:19.181 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "96123468-aabc-4a64-909e-a192c369090d", 2022-08-01T19:41:19.182 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:19.182 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:19.183 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:19.183 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:19.184 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:19.184 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:19.184 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:41:19.185 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 86, 2022-08-01T19:41:19.185 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:19.186 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:19.186 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:19.186 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:19.187 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.187 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:19.187 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6816", 2022-08-01T19:41:19.188 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:19.189 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.189 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.190 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:19.190 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6818", 2022-08-01T19:41:19.191 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:19.191 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.192 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.192 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.192 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:19.193 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:19.193 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.194 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:19.194 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6820", 2022-08-01T19:41:19.194 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:19.195 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.195 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.196 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:19.196 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6821", 2022-08-01T19:41:19.198 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:19.198 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.198 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.198 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.199 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:19.199 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:19.199 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.199 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:19.200 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6827", 2022-08-01T19:41:19.200 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:19.200 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.200 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.201 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:19.201 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6828", 2022-08-01T19:41:19.201 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:19.201 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.203 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.203 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.203 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:19.204 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:19.204 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.204 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:19.204 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6823", 2022-08-01T19:41:19.205 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:19.205 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.205 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.206 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:19.206 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6825", 2022-08-01T19:41:19.206 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:19.207 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.207 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.207 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.208 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6818/23009", 2022-08-01T19:41:19.208 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6821/23009", 2022-08-01T19:41:19.208 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6828/23009", 2022-08-01T19:41:19.208 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6825/23009", 2022-08-01T19:41:19.208 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:19.209 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:19.209 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:19.209 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.209 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.210 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.210 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:41:19.210 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a6e26665-e04b-46ec-ab66-e912acbf044c", 2022-08-01T19:41:19.210 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:19.211 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:19.211 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:19.211 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:19.211 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:19.212 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:19.212 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:41:19.212 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 86, 2022-08-01T19:41:19.213 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:19.214 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:19.214 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:19.214 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:19.214 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.215 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:19.215 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6810", 2022-08-01T19:41:19.215 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:19.215 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.216 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.216 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:19.216 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6811", 2022-08-01T19:41:19.216 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:19.216 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.217 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.217 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.217 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:19.218 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:19.218 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.218 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:19.218 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6812", 2022-08-01T19:41:19.218 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:19.219 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.219 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.219 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:19.219 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6813", 2022-08-01T19:41:19.220 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:19.220 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.220 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.221 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.221 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:19.221 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:19.221 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.222 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:19.222 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6816", 2022-08-01T19:41:19.222 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:19.222 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.223 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.223 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:19.223 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6817", 2022-08-01T19:41:19.224 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:19.224 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.224 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.224 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.225 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:19.225 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:19.225 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.226 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:19.226 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6814", 2022-08-01T19:41:19.226 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:19.226 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.227 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.227 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:19.227 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6815", 2022-08-01T19:41:19.227 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:19.228 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.228 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.228 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.228 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6811/23584", 2022-08-01T19:41:19.229 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6813/23584", 2022-08-01T19:41:19.229 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6817/23584", 2022-08-01T19:41:19.229 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6815/23584", 2022-08-01T19:41:19.229 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:19.229 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:19.230 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:19.230 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.230 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.231 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.231 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:41:19.231 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "502c0034-18ee-44ac-87ff-19b9dddb66a0", 2022-08-01T19:41:19.231 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:19.232 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:19.232 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:19.232 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:19.232 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:19.233 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:19.233 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:41:19.233 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 86, 2022-08-01T19:41:19.234 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:19.234 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:19.234 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:19.234 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:19.235 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.235 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:19.235 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6802", 2022-08-01T19:41:19.235 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:19.236 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.236 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.236 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:19.236 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6803", 2022-08-01T19:41:19.237 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:19.237 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.237 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.237 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.237 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:19.238 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:19.238 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.238 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:19.238 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6804", 2022-08-01T19:41:19.239 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:19.239 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.240 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.240 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:19.240 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6805", 2022-08-01T19:41:19.240 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:19.241 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.241 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.241 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.241 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:19.241 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:19.242 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.242 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:19.242 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6808", 2022-08-01T19:41:19.242 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:19.243 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.243 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.243 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:19.243 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6809", 2022-08-01T19:41:19.244 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:19.244 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.244 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.244 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.245 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:19.245 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:19.246 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.246 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:19.246 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6806", 2022-08-01T19:41:19.246 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:19.247 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.247 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.247 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:19.248 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6807", 2022-08-01T19:41:19.248 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:19.248 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.248 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.249 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.249 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6803/23585", 2022-08-01T19:41:19.249 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6805/23585", 2022-08-01T19:41:19.249 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6809/23585", 2022-08-01T19:41:19.250 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6807/23585", 2022-08-01T19:41:19.250 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:19.250 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:19.250 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:19.251 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.251 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.251 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.252 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:41:19.252 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "ac4e755a-7fe7-46cb-8407-852d5af9ffc5", 2022-08-01T19:41:19.252 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:19.252 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:19.253 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:19.253 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:19.253 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:19.253 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:19.254 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:41:19.255 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 86, 2022-08-01T19:41:19.255 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:19.255 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:19.256 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:19.256 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:19.256 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.256 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:19.257 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6820", 2022-08-01T19:41:19.257 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:19.257 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.258 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.258 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:19.258 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6821", 2022-08-01T19:41:19.258 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:19.259 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.259 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.259 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.259 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:19.260 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:19.260 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.260 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:19.260 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6823", 2022-08-01T19:41:19.261 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:19.261 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.261 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.261 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:19.262 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6825", 2022-08-01T19:41:19.262 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:19.262 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.263 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.263 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.263 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:19.263 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:19.264 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.264 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:19.265 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6831", 2022-08-01T19:41:19.265 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:19.266 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.266 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.266 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:19.267 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6833", 2022-08-01T19:41:19.267 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:19.267 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.267 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.268 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.268 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:19.268 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:19.268 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.268 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:19.269 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6827", 2022-08-01T19:41:19.269 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:19.269 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.269 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.270 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:19.270 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6829", 2022-08-01T19:41:19.270 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:19.270 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.271 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.271 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.271 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6821/23587", 2022-08-01T19:41:19.271 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6825/23587", 2022-08-01T19:41:19.272 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6833/23587", 2022-08-01T19:41:19.272 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6829/23587", 2022-08-01T19:41:19.272 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:19.272 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:19.273 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:19.273 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.273 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.273 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.274 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:41:19.274 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "874761e2-6410-40fc-b60c-e58abe5ce489", 2022-08-01T19:41:19.274 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:19.274 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:19.275 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:19.275 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:19.276 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:19.276 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:19.276 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:41:19.276 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 73, 2022-08-01T19:41:19.277 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:19.277 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:19.277 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:19.277 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:19.278 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.278 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:19.278 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6818", 2022-08-01T19:41:19.278 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:19.279 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.279 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.279 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:19.279 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6819", 2022-08-01T19:41:19.280 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:19.280 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.280 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.280 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.281 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:19.281 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:19.281 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.281 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:19.282 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6822", 2022-08-01T19:41:19.282 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:19.283 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.283 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.283 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:19.283 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6824", 2022-08-01T19:41:19.284 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:19.284 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.284 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.284 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.285 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:19.285 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:19.286 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.286 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:19.287 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6830", 2022-08-01T19:41:19.287 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:19.287 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.288 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.288 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:19.289 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6832", 2022-08-01T19:41:19.289 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:19.290 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.290 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.291 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.291 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:19.292 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:19.292 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.293 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:19.294 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6826", 2022-08-01T19:41:19.294 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:19.295 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.295 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.296 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:19.297 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6828", 2022-08-01T19:41:19.297 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:19.298 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.298 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.299 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.299 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6819/23586", 2022-08-01T19:41:19.300 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6824/23586", 2022-08-01T19:41:19.300 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6832/23586", 2022-08-01T19:41:19.300 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6828/23586", 2022-08-01T19:41:19.301 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:19.301 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:19.302 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:19.302 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:19.303 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.303 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:41:19.303 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_xinfo": [ 2022-08-01T19:41:19.303 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.304 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:41:19.304 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:19.304 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:19.304 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:19.305 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:19.305 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:19.305 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.588817+0000", 2022-08-01T19:41:19.306 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:19.307 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.308 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.308 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:41:19.309 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:19.309 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:19.310 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:19.310 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:19.311 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:19.311 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.782753+0000", 2022-08-01T19:41:19.311 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:19.312 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.312 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.312 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:41:19.312 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:19.313 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:19.313 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:19.313 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:19.313 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:19.313 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.837924+0000", 2022-08-01T19:41:19.314 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:19.314 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.314 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.314 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:41:19.315 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:19.315 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:19.315 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:19.315 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:19.316 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:19.316 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.839948+0000", 2022-08-01T19:41:19.316 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:19.316 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.316 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.317 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:41:19.317 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:19.318 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:19.318 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:19.318 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:19.319 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:19.319 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.112801+0000", 2022-08-01T19:41:19.319 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:19.319 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.320 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.320 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:41:19.320 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:19.321 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:19.321 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:19.322 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:19.322 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:19.322 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.106704+0000", 2022-08-01T19:41:19.322 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:19.323 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.323 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.323 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:41:19.323 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:19.324 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:19.324 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:19.324 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:19.324 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:19.325 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.893170+0000", 2022-08-01T19:41:19.325 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:19.325 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.325 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:19.325 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:41:19.326 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:19.326 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:19.326 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:19.326 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:19.327 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:19.327 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.886947+0000", 2022-08-01T19:41:19.327 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:19.327 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.328 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:41:19.328 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap": [], 2022-08-01T19:41:19.329 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap_items": [], 2022-08-01T19:41:19.329 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_temp": [], 2022-08-01T19:41:19.329 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_temp": [], 2022-08-01T19:41:19.329 INFO:teuthology.orchestra.run.smithi017.stdout: "blacklist": { 2022-08-01T19:41:19.330 INFO:teuthology.orchestra.run.smithi017.stdout: "172.21.15.77:6835/4275403969": "2022-08-02T19:41:12.812507+0000", 2022-08-01T19:41:19.330 INFO:teuthology.orchestra.run.smithi017.stdout: "172.21.15.77:6834/4275403969": "2022-08-02T19:41:12.812507+0000" 2022-08-01T19:41:19.330 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.330 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profiles": { 2022-08-01T19:41:19.330 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_data": { 2022-08-01T19:41:19.331 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-device-class": "", 2022-08-01T19:41:19.331 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:41:19.331 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-root": "default", 2022-08-01T19:41:19.331 INFO:teuthology.orchestra.run.smithi017.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T19:41:19.332 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:41:19.332 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "2", 2022-08-01T19:41:19.332 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:41:19.332 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:41:19.333 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van", 2022-08-01T19:41:19.333 INFO:teuthology.orchestra.run.smithi017.stdout: "w": "8" 2022-08-01T19:41:19.333 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.333 INFO:teuthology.orchestra.run.smithi017.stdout: "default": { 2022-08-01T19:41:19.333 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:41:19.334 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:41:19.334 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "1", 2022-08-01T19:41:19.334 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:41:19.334 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:41:19.335 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van" 2022-08-01T19:41:19.335 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:19.335 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:19.335 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps_queue": [], 2022-08-01T19:41:19.336 INFO:teuthology.orchestra.run.smithi017.stdout: "new_removed_snaps": [], 2022-08-01T19:41:19.336 INFO:teuthology.orchestra.run.smithi017.stdout: "new_purged_snaps": [], 2022-08-01T19:41:19.336 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_node_flags": {}, 2022-08-01T19:41:19.336 INFO:teuthology.orchestra.run.smithi017.stdout: "device_class_flags": {} 2022-08-01T19:41:19.337 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:41:19.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.958+0000 7fda75462700 1 -- 172.21.15.17:0/3376195734 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fda54053950 msgr2=0x7fda54055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:19.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.958+0000 7fda75462700 1 --2- 172.21.15.17:0/3376195734 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fda54053950 0x7fda54055e10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7fda6001a000 tx=0x7fda60014000).stop 2022-08-01T19:41:19.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.958+0000 7fda75462700 1 -- 172.21.15.17:0/3376195734 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda700b0270 msgr2=0x7fda700ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:19.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.958+0000 7fda75462700 1 --2- 172.21.15.17:0/3376195734 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda700b0270 0x7fda700ac200 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 rx=0x7fda5800c000 tx=0x7fda58006df0).stop 2022-08-01T19:41:19.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.958+0000 7fda75462700 1 -- 172.21.15.17:0/3376195734 shutdown_connections 2022-08-01T19:41:19.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.958+0000 7fda75462700 1 --2- 172.21.15.17:0/3376195734 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fda54053950 0x7fda54055e10 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:19.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.958+0000 7fda75462700 1 --2- 172.21.15.17:0/3376195734 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda700b2710 0x7fda70145860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:19.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.958+0000 7fda75462700 1 --2- 172.21.15.17:0/3376195734 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda700b0270 0x7fda700ac200 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:19.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.958+0000 7fda75462700 1 --2- 172.21.15.17:0/3376195734 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fda700ac880 0x7fda70143240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:19.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.958+0000 7fda75462700 1 -- 172.21.15.17:0/3376195734 >> 172.21.15.17:0/3376195734 conn(0x7fda70060310 msgr2=0x7fda7005be30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:19.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.958+0000 7fda75462700 1 -- 172.21.15.17:0/3376195734 shutdown_connections 2022-08-01T19:41:19.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:18.962+0000 7fda75462700 1 -- 172.21.15.17:0/3376195734 wait complete. 2022-08-01T19:41:19.342 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:41:19.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.446+0000 7fbf11826700 1 Processor -- start 2022-08-01T19:41:19.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.446+0000 7fbf11826700 1 -- start start 2022-08-01T19:41:19.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.450+0000 7fbf11826700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fbf0c0a9320 0x7fbf0c1466e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:19.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.450+0000 7fbf11826700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbf0c0b0d10 0x7fbf0c146bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:19.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.450+0000 7fbf11826700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf0c0b12d0 con 0x7fbf0c0b0560 2022-08-01T19:41:19.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.450+0000 7fbf11826700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf0c0a9700 con 0x7fbf0c0a9320 2022-08-01T19:41:19.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.450+0000 7fbf11826700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0c0b1500 con 0x7fbf0c0b0d10 2022-08-01T19:41:19.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.450+0000 7fbf11025700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fbf0c0b0560 0x7fbf0c140350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33970/0 (socket says 172.21.15.17:33970) 2022-08-01T19:41:19.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.450+0000 7fbf11025700 1 -- 172.21.15.17:0/2328620439 learned_addr learned my addr 172.21.15.17:0/2328620439 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:19.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.450+0000 7fbf0bfff700 1 --2- 172.21.15.17:0/2328620439 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbf0c0b0d10 0x7fbf0c146bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:19.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.450+0000 7fbf10824700 1 --2- 172.21.15.17:0/2328620439 >> v2:172.21.15.77:3300/0 conn(0x7fbf0c0a9320 0x7fbf0c1466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:19.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.450+0000 7fbf0b7fe700 1 -- 172.21.15.17:0/2328620439 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1815693692 0 0) 0x7fbf0c0b12d0 con 0x7fbf0c0b0560 2022-08-01T19:41:19.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.450+0000 7fbf0b7fe700 1 -- 172.21.15.17:0/2328620439 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbef8003260 con 0x7fbf0c0b0560 2022-08-01T19:41:19.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.450+0000 7fbf0bfff700 1 -- 172.21.15.17:0/2328620439 >> v2:172.21.15.77:3300/0 conn(0x7fbf0c0a9320 msgr2=0x7fbf0c1466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:19.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.450+0000 7fbf0bfff700 1 --2- 172.21.15.17:0/2328620439 >> v2:172.21.15.77:3300/0 conn(0x7fbf0c0a9320 0x7fbf0c1466e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:19.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.450+0000 7fbf0bfff700 1 -- 172.21.15.17:0/2328620439 >> v1:172.21.15.17:6789/0 conn(0x7fbf0c0b0560 legacy=0x7fbf0c140350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:19.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.450+0000 7fbf0bfff700 1 -- 172.21.15.17:0/2328620439 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf0c0b17a0 con 0x7fbf0c0b0d10 2022-08-01T19:41:19.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.450+0000 7fbf10824700 1 --2- 172.21.15.17:0/2328620439 >> v2:172.21.15.77:3300/0 conn(0x7fbf0c0a9320 0x7fbf0c1466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:19.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.450+0000 7fbf0bfff700 1 --2- 172.21.15.17:0/2328620439 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbf0c0b0d10 0x7fbf0c146bd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fbefc002f70 tx=0x7fbefc006df0).ready entity=mon.2 client_cookie=dd3e601c5f39d12b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:19.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.450+0000 7fbf0b7fe700 1 -- 172.21.15.17:0/2328620439 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbefc00d4a0 con 0x7fbf0c0b0d10 2022-08-01T19:41:19.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.450+0000 7fbf0b7fe700 1 -- 172.21.15.17:0/2328620439 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbefc000e90 con 0x7fbf0c0b0d10 2022-08-01T19:41:19.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.450+0000 7fbf0b7fe700 1 -- 172.21.15.17:0/2328620439 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbefc00d7f0 con 0x7fbf0c0b0d10 2022-08-01T19:41:19.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.454+0000 7fbf11826700 1 -- 172.21.15.17:0/2328620439 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbf0c0b0d10 msgr2=0x7fbf0c146bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:19.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.454+0000 7fbf11826700 1 --2- 172.21.15.17:0/2328620439 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbf0c0b0d10 0x7fbf0c146bd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fbefc002f70 tx=0x7fbefc006df0).stop 2022-08-01T19:41:19.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.454+0000 7fbf11826700 1 -- 172.21.15.17:0/2328620439 shutdown_connections 2022-08-01T19:41:19.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.454+0000 7fbf11826700 1 --2- 172.21.15.17:0/2328620439 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbf0c0b0d10 0x7fbf0c146bd0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:19.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.454+0000 7fbf11826700 1 --2- 172.21.15.17:0/2328620439 >> v2:172.21.15.77:3300/0 conn(0x7fbf0c0a9320 0x7fbf0c1466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:19.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.454+0000 7fbf11826700 1 -- 172.21.15.17:0/2328620439 >> 172.21.15.17:0/2328620439 conn(0x7fbf0c0612d0 msgr2=0x7fbf0c062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:19.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.454+0000 7fbf11826700 1 -- 172.21.15.17:0/2328620439 shutdown_connections 2022-08-01T19:41:19.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.454+0000 7fbf11826700 1 -- 172.21.15.17:0/2328620439 wait complete. 2022-08-01T19:41:19.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.454+0000 7fbf11826700 1 Processor -- start 2022-08-01T19:41:19.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.454+0000 7fbf11826700 1 -- start start 2022-08-01T19:41:19.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.454+0000 7fbf11826700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbf0c0a9320 0x7fbf0c0aa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:19.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.454+0000 7fbf11826700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf0c0b0560 0x7fbf0c1431c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:19.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.454+0000 7fbf11826700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf0c0b0d10 0x7fbf0c1436b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:19.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.454+0000 7fbf11826700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0c0ab150 con 0x7fbf0c0b0d10 2022-08-01T19:41:19.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.454+0000 7fbf11826700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0c152f10 con 0x7fbf0c0b0560 2022-08-01T19:41:19.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.454+0000 7fbf11826700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0c1531b0 con 0x7fbf0c0a9320 2022-08-01T19:41:19.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.454+0000 7fbf11025700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf0c0b0d10 0x7fbf0c1436b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:19.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.454+0000 7fbf0bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf0c0b0560 0x7fbf0c1431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:19.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.454+0000 7fbf11025700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf0c0b0d10 0x7fbf0c1436b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41782/0 (socket says 172.21.15.17:41782) 2022-08-01T19:41:19.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.454+0000 7fbf10824700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbf0c0a9320 0x7fbf0c0aa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:19.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.454+0000 7fbf11025700 1 -- 172.21.15.17:0/3370097787 learned_addr learned my addr 172.21.15.17:0/3370097787 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:19.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.458+0000 7fbf11025700 1 -- 172.21.15.17:0/3370097787 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbf0c0a9320 msgr2=0x7fbf0c0aa3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:19.499 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.458+0000 7fbf11025700 1 --2- 172.21.15.17:0/3370097787 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbf0c0a9320 0x7fbf0c0aa3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:19.499 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.458+0000 7fbf11025700 1 -- 172.21.15.17:0/3370097787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf0c0b0560 msgr2=0x7fbf0c1431c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:19.499 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.458+0000 7fbf11025700 1 --2- 172.21.15.17:0/3370097787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf0c0b0560 0x7fbf0c1431c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:19.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.458+0000 7fbf11025700 1 -- 172.21.15.17:0/3370097787 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf0000b180 con 0x7fbf0c0b0d10 2022-08-01T19:41:19.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.458+0000 7fbf11025700 1 --2- 172.21.15.17:0/3370097787 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf0c0b0d10 0x7fbf0c1436b0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 rx=0x7fbf04001490 tx=0x7fbf04007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:19.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.458+0000 7fbf09ffb700 1 -- 172.21.15.17:0/3370097787 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf04005ba0 con 0x7fbf0c0b0d10 2022-08-01T19:41:19.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.458+0000 7fbf09ffb700 1 -- 172.21.15.17:0/3370097787 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf04005d00 con 0x7fbf0c0b0d10 2022-08-01T19:41:19.502 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.458+0000 7fbf11826700 1 -- 172.21.15.17:0/3370097787 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbefc00b180 con 0x7fbf0c0b0d10 2022-08-01T19:41:19.502 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.458+0000 7fbf09ffb700 1 -- 172.21.15.17:0/3370097787 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf040072c0 con 0x7fbf0c0b0d10 2022-08-01T19:41:19.502 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.458+0000 7fbf11826700 1 -- 172.21.15.17:0/3370097787 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf0c143ed0 con 0x7fbf0c0b0d10 2022-08-01T19:41:19.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.458+0000 7fbf11826700 1 -- 172.21.15.17:0/3370097787 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf0c05f440 con 0x7fbf0c0b0d10 2022-08-01T19:41:19.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.458+0000 7fbf09ffb700 1 -- 172.21.15.17:0/3370097787 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbf04010070 con 0x7fbf0c0b0d10 2022-08-01T19:41:19.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.462+0000 7fbf09ffb700 1 --2- 172.21.15.17:0/3370097787 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbeec0539e0 0x7fbeec055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:19.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.462+0000 7fbf09ffb700 1 -- 172.21.15.17:0/3370097787 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fbf0406b760 con 0x7fbf0c0b0d10 2022-08-01T19:41:19.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.462+0000 7fbf10824700 1 --2- 172.21.15.17:0/3370097787 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbeec0539e0 0x7fbeec055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:19.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.462+0000 7fbf10824700 1 --2- 172.21.15.17:0/3370097787 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbeec0539e0 0x7fbeec055ea0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7fbf0000b000 tx=0x7fbf00006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:19.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.470+0000 7fbf09ffb700 1 -- 172.21.15.17:0/3370097787 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbf040426f0 con 0x7fbf0c0b0d10 2022-08-01T19:41:19.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.638+0000 7fbf11826700 1 -- 172.21.15.17:0/3370097787 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf0c153370 con 0x7fbf0c0b0d10 2022-08-01T19:41:19.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.638+0000 7fbf09ffb700 1 -- 172.21.15.17:0/3370097787 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+1562 (secure 0 0 0) 0x7fbf04045d10 con 0x7fbf0c0b0d10 2022-08-01T19:41:19.659 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:19.660 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":42,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6089,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/2260041241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2260041241},{"type":"v1","addr":"172.21.15.77:6835","nonce":2260041241}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42},{"gid":6615,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3580428616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3580428616},{"type":"v1","addr":"172.21.15.17:6833","nonce":3580428616}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38},{"gid":6627,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/4048623373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":4048623373},{"type":"v1","addr":"172.21.15.17:6835","nonce":4048623373}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38}],"filesystems":[]} 2022-08-01T19:41:19.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.642+0000 7fbf11826700 1 -- 172.21.15.17:0/3370097787 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbeec0539e0 msgr2=0x7fbeec055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:19.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.642+0000 7fbf11826700 1 --2- 172.21.15.17:0/3370097787 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbeec0539e0 0x7fbeec055ea0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7fbf0000b000 tx=0x7fbf00006df0).stop 2022-08-01T19:41:19.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.642+0000 7fbf11826700 1 -- 172.21.15.17:0/3370097787 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf0c0b0d10 msgr2=0x7fbf0c1436b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:19.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.642+0000 7fbf11826700 1 --2- 172.21.15.17:0/3370097787 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf0c0b0d10 0x7fbf0c1436b0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 rx=0x7fbf04001490 tx=0x7fbf04007e00).stop 2022-08-01T19:41:19.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.642+0000 7fbf11826700 1 -- 172.21.15.17:0/3370097787 shutdown_connections 2022-08-01T19:41:19.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.642+0000 7fbf11826700 1 --2- 172.21.15.17:0/3370097787 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbeec0539e0 0x7fbeec055ea0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:19.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.642+0000 7fbf11826700 1 --2- 172.21.15.17:0/3370097787 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf0c0b0d10 0x7fbf0c1436b0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:19.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.642+0000 7fbf11826700 1 --2- 172.21.15.17:0/3370097787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf0c0b0560 0x7fbf0c1431c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:19.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.642+0000 7fbf11826700 1 --2- 172.21.15.17:0/3370097787 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbf0c0a9320 0x7fbf0c0aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:19.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.642+0000 7fbf11826700 1 -- 172.21.15.17:0/3370097787 >> 172.21.15.17:0/3370097787 conn(0x7fbf0c0612d0 msgr2=0x7fbf0c0b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:19.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.642+0000 7fbf11826700 1 -- 172.21.15.17:0/3370097787 shutdown_connections 2022-08-01T19:41:19.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.642+0000 7fbf11826700 1 -- 172.21.15.17:0/3370097787 wait complete. 2022-08-01T19:41:19.671 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 42 2022-08-01T19:41:19.672 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:41:19.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.774+0000 7f719ffff700 1 Processor -- start 2022-08-01T19:41:19.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.774+0000 7f719ffff700 1 -- start start 2022-08-01T19:41:19.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.778+0000 7f719ffff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f71a00b2710 0x7f71a014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:19.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.778+0000 7f719ffff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71a00a8c50 0x7f71a014b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:19.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.778+0000 7f719ffff700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71a00a7250 con 0x7f71a00b01a0 2022-08-01T19:41:19.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.778+0000 7f719ffff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71a00a7480 con 0x7f71a00b2710 2022-08-01T19:41:19.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.778+0000 7f719ffff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a00a7720 con 0x7f71a00a8c50 2022-08-01T19:41:19.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.778+0000 7f719f7fe700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71a00a8c50 0x7f71a014b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:19.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.778+0000 7f719e7fc700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7f71a00b01a0 0x7f71a00a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:50780/0 (socket says 172.21.15.17:50780) 2022-08-01T19:41:19.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.778+0000 7f719effd700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f71a00b2710 0x7f71a014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:19.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.778+0000 7f719e7fc700 1 -- 172.21.15.17:0/3191569672 learned_addr learned my addr 172.21.15.17:0/3191569672 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:19.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.778+0000 7f719dffb700 1 -- 172.21.15.17:0/3191569672 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 614155652 0 0) 0x7f71a00a7250 con 0x7f71a00b01a0 2022-08-01T19:41:19.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.778+0000 7f719f7fe700 1 -- 172.21.15.17:0/3191569672 >> v2:172.21.15.77:3300/0 conn(0x7f71a00b2710 msgr2=0x7f71a014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:19.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.778+0000 7f719f7fe700 1 --2- 172.21.15.17:0/3191569672 >> v2:172.21.15.77:3300/0 conn(0x7f71a00b2710 0x7f71a014afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:19.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.778+0000 7f719f7fe700 1 -- 172.21.15.17:0/3191569672 >> v1:172.21.15.77:6789/0 conn(0x7f71a00b01a0 legacy=0x7f71a00a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:19.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.778+0000 7f719f7fe700 1 -- 172.21.15.17:0/3191569672 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a00a79c0 con 0x7f71a00a8c50 2022-08-01T19:41:19.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.778+0000 7f719f7fe700 1 --2- 172.21.15.17:0/3191569672 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71a00a8c50 0x7f71a014b4c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f7198002f70 tx=0x7f7198006df0).ready entity=mon.2 client_cookie=1a2d7b07571ac32c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:19.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.778+0000 7f719dffb700 1 -- 172.21.15.17:0/3191569672 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f719800d570 con 0x7f71a00a8c50 2022-08-01T19:41:19.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.778+0000 7f719dffb700 1 -- 172.21.15.17:0/3191569672 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7198000e90 con 0x7f71a00a8c50 2022-08-01T19:41:19.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.778+0000 7f719dffb700 1 -- 172.21.15.17:0/3191569672 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f719800d8c0 con 0x7f71a00a8c50 2022-08-01T19:41:19.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.778+0000 7f719ffff700 1 -- 172.21.15.17:0/3191569672 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71a00a8c50 msgr2=0x7f71a014b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:19.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.778+0000 7f719ffff700 1 --2- 172.21.15.17:0/3191569672 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71a00a8c50 0x7f71a014b4c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f7198002f70 tx=0x7f7198006df0).stop 2022-08-01T19:41:19.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.778+0000 7f719ffff700 1 -- 172.21.15.17:0/3191569672 shutdown_connections 2022-08-01T19:41:19.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.778+0000 7f719ffff700 1 --2- 172.21.15.17:0/3191569672 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71a00a8c50 0x7f71a014b4c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:19.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.778+0000 7f719ffff700 1 --2- 172.21.15.17:0/3191569672 >> v2:172.21.15.77:3300/0 conn(0x7f71a00b2710 0x7f71a014afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:19.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.778+0000 7f719ffff700 1 -- 172.21.15.17:0/3191569672 >> 172.21.15.17:0/3191569672 conn(0x7f71a0060310 msgr2=0x7f71a00b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:19.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.778+0000 7f719ffff700 1 -- 172.21.15.17:0/3191569672 shutdown_connections 2022-08-01T19:41:19.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.778+0000 7f719ffff700 1 -- 172.21.15.17:0/3191569672 wait complete. 2022-08-01T19:41:19.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.782+0000 7f719ffff700 1 Processor -- start 2022-08-01T19:41:19.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.782+0000 7f719ffff700 1 -- start start 2022-08-01T19:41:19.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.782+0000 7f719ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71a00a8c50 0x7f71a00b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:19.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.782+0000 7f719ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71a00b01a0 0x7f71a0064c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:19.807 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.782+0000 7f719ffff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71a00b2710 0x7f71a0065130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:19.807 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.782+0000 7f719ffff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a00ac0c0 con 0x7f71a00a8c50 2022-08-01T19:41:19.807 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.782+0000 7f719ffff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a00ac320 con 0x7f71a00b01a0 2022-08-01T19:41:19.807 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.782+0000 7f719ffff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a00ab4b0 con 0x7f71a00b2710 2022-08-01T19:41:19.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.782+0000 7f719effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71a00a8c50 0x7f71a00b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:19.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.782+0000 7f719effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71a00a8c50 0x7f71a00b57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41792/0 (socket says 172.21.15.17:41792) 2022-08-01T19:41:19.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.782+0000 7f719effd700 1 -- 172.21.15.17:0/2689063242 learned_addr learned my addr 172.21.15.17:0/2689063242 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:19.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.782+0000 7f719e7fc700 1 --2- 172.21.15.17:0/2689063242 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71a00b01a0 0x7f71a0064c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:19.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.782+0000 7f719f7fe700 1 --2- 172.21.15.17:0/2689063242 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71a00b2710 0x7f71a0065130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:19.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.782+0000 7f719effd700 1 -- 172.21.15.17:0/2689063242 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71a00b2710 msgr2=0x7f71a0065130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:19.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.782+0000 7f719effd700 1 --2- 172.21.15.17:0/2689063242 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71a00b2710 0x7f71a0065130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:19.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.782+0000 7f719effd700 1 -- 172.21.15.17:0/2689063242 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71a00b01a0 msgr2=0x7f71a0064c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:19.810 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.782+0000 7f719effd700 1 --2- 172.21.15.17:0/2689063242 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71a00b01a0 0x7f71a0064c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:19.810 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.782+0000 7f719effd700 1 -- 172.21.15.17:0/2689063242 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f719800b180 con 0x7f71a00a8c50 2022-08-01T19:41:19.810 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.782+0000 7f719effd700 1 --2- 172.21.15.17:0/2689063242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71a00a8c50 0x7f71a00b57c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 rx=0x7f719400c000 tx=0x7f7194006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:19.811 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.782+0000 7f718ffff700 1 -- 172.21.15.17:0/2689063242 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7194011750 con 0x7f71a00a8c50 2022-08-01T19:41:19.811 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.782+0000 7f718ffff700 1 -- 172.21.15.17:0/2689063242 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71940118b0 con 0x7f71a00a8c50 2022-08-01T19:41:19.811 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.782+0000 7f719ffff700 1 -- 172.21.15.17:0/2689063242 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a00ab780 con 0x7f71a00a8c50 2022-08-01T19:41:19.811 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.782+0000 7f718ffff700 1 -- 172.21.15.17:0/2689063242 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7194011c10 con 0x7f71a00a8c50 2022-08-01T19:41:19.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.782+0000 7f719ffff700 1 -- 172.21.15.17:0/2689063242 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a00aa990 con 0x7f71a00a8c50 2022-08-01T19:41:19.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.786+0000 7f719ffff700 1 -- 172.21.15.17:0/2689063242 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71a00574e0 con 0x7f71a00a8c50 2022-08-01T19:41:19.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.786+0000 7f718ffff700 1 -- 172.21.15.17:0/2689063242 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7194011d70 con 0x7f71a00a8c50 2022-08-01T19:41:19.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.786+0000 7f718ffff700 1 --2- 172.21.15.17:0/2689063242 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f717c0539e0 0x7f717c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:19.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.786+0000 7f718ffff700 1 -- 172.21.15.17:0/2689063242 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f719406c1f0 con 0x7f71a00a8c50 2022-08-01T19:41:19.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.786+0000 7f719e7fc700 1 --2- 172.21.15.17:0/2689063242 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f717c0539e0 0x7f717c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:19.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.786+0000 7f719e7fc700 1 --2- 172.21.15.17:0/2689063242 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f717c0539e0 0x7f717c055ea0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f719000d000 tx=0x7f7190007d80).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:19.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.790+0000 7f718ffff700 1 -- 172.21.15.17:0/2689063242 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f71940431b0 con 0x7f71a00a8c50 2022-08-01T19:41:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.950+0000 7f719ffff700 1 -- 172.21.15.17:0/2689063242 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71a0069a70 con 0x7f71a00a8c50 2022-08-01T19:41:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.954+0000 7f718ffff700 1 -- 172.21.15.17:0/2689063242 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+1562 (secure 0 0 0) 0x7f71940467d0 con 0x7f71a00a8c50 2022-08-01T19:41:19.959 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:19.959 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":42,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6089,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/2260041241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2260041241},{"type":"v1","addr":"172.21.15.77:6835","nonce":2260041241}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42},{"gid":6615,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3580428616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3580428616},{"type":"v1","addr":"172.21.15.17:6833","nonce":3580428616}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38},{"gid":6627,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/4048623373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":4048623373},{"type":"v1","addr":"172.21.15.17:6835","nonce":4048623373}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":38}],"filesystems":[]} 2022-08-01T19:41:19.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.954+0000 7f719ffff700 1 -- 172.21.15.17:0/2689063242 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f717c0539e0 msgr2=0x7f717c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:19.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.954+0000 7f719ffff700 1 --2- 172.21.15.17:0/2689063242 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f717c0539e0 0x7f717c055ea0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f719000d000 tx=0x7f7190007d80).stop 2022-08-01T19:41:19.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.954+0000 7f719ffff700 1 -- 172.21.15.17:0/2689063242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71a00a8c50 msgr2=0x7f71a00b57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:19.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.954+0000 7f719ffff700 1 --2- 172.21.15.17:0/2689063242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71a00a8c50 0x7f71a00b57c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 rx=0x7f719400c000 tx=0x7f7194006df0).stop 2022-08-01T19:41:19.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.954+0000 7f719ffff700 1 -- 172.21.15.17:0/2689063242 shutdown_connections 2022-08-01T19:41:19.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.954+0000 7f719ffff700 1 --2- 172.21.15.17:0/2689063242 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f717c0539e0 0x7f717c055ea0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:19.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.954+0000 7f719ffff700 1 --2- 172.21.15.17:0/2689063242 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71a00b2710 0x7f71a0065130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:19.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.954+0000 7f719ffff700 1 --2- 172.21.15.17:0/2689063242 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71a00b01a0 0x7f71a0064c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:19.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.954+0000 7f719ffff700 1 --2- 172.21.15.17:0/2689063242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71a00a8c50 0x7f71a00b57c0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:19.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.954+0000 7f719ffff700 1 -- 172.21.15.17:0/2689063242 >> 172.21.15.17:0/2689063242 conn(0x7f71a0060310 msgr2=0x7f71a01489b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:19.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.954+0000 7f719ffff700 1 -- 172.21.15.17:0/2689063242 shutdown_connections 2022-08-01T19:41:19.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:19.954+0000 7f719ffff700 1 -- 172.21.15.17:0/2689063242 wait complete. 2022-08-01T19:41:19.965 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 42 2022-08-01T19:41:19.975 INFO:tasks.ceph.mds.a:Restarting daemon 2022-08-01T19:41:19.975 INFO:tasks.ceph.mds.a:Stopping old one... 2022-08-01T19:41:19.976 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-08-01T19:41:19.977 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:41:19.977 INFO:tasks.ceph.mds.c:Restarting daemon 2022-08-01T19:41:19.982 INFO:tasks.ceph.mds.c:Stopping old one... 2022-08-01T19:41:19.983 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-08-01T19:41:19.984 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:41:19.984 INFO:tasks.ceph.mds.b:Restarting daemon 2022-08-01T19:41:19.985 INFO:tasks.ceph.mds.b:Stopping old one... 2022-08-01T19:41:19.986 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-08-01T19:41:19.986 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:41:19.991 INFO:tasks.ceph.mds.a:Stopped 2022-08-01T19:41:19.992 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-08-01T19:41:19.997 INFO:tasks.ceph.mds.c:Stopped 2022-08-01T19:41:19.999 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-08-01T19:41:20.009 INFO:tasks.ceph.mds.b:Stopped 2022-08-01T19:41:20.010 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-08-01T19:41:20.038 INFO:tasks.ceph.mds.a:Started 2022-08-01T19:41:20.039 DEBUG:teuthology.parallel:result is None 2022-08-01T19:41:20.040 INFO:tasks.ceph.mds.c:Started 2022-08-01T19:41:20.041 DEBUG:teuthology.parallel:result is None 2022-08-01T19:41:20.054 INFO:tasks.ceph.mds.b:Started 2022-08-01T19:41:20.055 DEBUG:teuthology.parallel:result is None 2022-08-01T19:41:20.055 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-08-01T19:41:20.119 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2022-08-01T19:41:20.119 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2022-08-01T19:41:20.137 INFO:tasks.ceph.mds.b.smithi077.stdout:starting mds.b at 2022-08-01T19:41:20.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.146+0000 7fa8f6a04700 1 Processor -- start 2022-08-01T19:41:20.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.146+0000 7fa8f6a04700 1 -- start start 2022-08-01T19:41:20.158 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.146+0000 7fa8f6a04700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa8f013dad0 0x7fa8f0143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:20.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.146+0000 7fa8f6a04700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8f00a8cf0 con 0x7fa8f01437f0 2022-08-01T19:41:20.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.146+0000 7fa8f6a04700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8f00a8f50 con 0x7fa8f00aad90 2022-08-01T19:41:20.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.146+0000 7fa8f6a04700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f01441d0 con 0x7fa8f013dad0 2022-08-01T19:41:20.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.146+0000 7fa8effff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa8f01437f0 0x7fa8f013d3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34018/0 (socket says 172.21.15.17:34018) 2022-08-01T19:41:20.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.146+0000 7fa8effff700 1 -- 172.21.15.17:0/3037878350 learned_addr learned my addr 172.21.15.17:0/3037878350 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:20.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.146+0000 7fa8f4fa1700 1 --2- 172.21.15.17:0/3037878350 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa8f013dad0 0x7fa8f0143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:20.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.150+0000 7fa8eeffd700 1 -- 172.21.15.17:0/3037878350 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 648150464 0 0) 0x7fa8f00a8cf0 con 0x7fa8f01437f0 2022-08-01T19:41:20.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.150+0000 7fa8eeffd700 1 -- 172.21.15.17:0/3037878350 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8d0003260 con 0x7fa8f01437f0 2022-08-01T19:41:20.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.150+0000 7fa8eeffd700 1 -- 172.21.15.17:0/3037878350 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 742191728 0 0) 0x7fa8f00a8f50 con 0x7fa8f00aad90 2022-08-01T19:41:20.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.150+0000 7fa8eeffd700 1 -- 172.21.15.17:0/3037878350 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8f00a8cf0 con 0x7fa8f00aad90 2022-08-01T19:41:20.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.150+0000 7fa8f4fa1700 1 -- 172.21.15.17:0/3037878350 >> v1:172.21.15.77:6789/0 conn(0x7fa8f00aad90 legacy=0x7fa8f00a9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:20.162 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.150+0000 7fa8eeffd700 1 -- 172.21.15.17:0/3037878350 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2296107898 0 0) 0x7fa8d0003260 con 0x7fa8f01437f0 2022-08-01T19:41:20.162 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.150+0000 7fa8f4fa1700 1 -- 172.21.15.17:0/3037878350 >> v1:172.21.15.17:6789/0 conn(0x7fa8f01437f0 legacy=0x7fa8f013d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:20.163 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.150+0000 7fa8f4fa1700 1 -- 172.21.15.17:0/3037878350 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f005fd50 con 0x7fa8f013dad0 2022-08-01T19:41:20.163 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.150+0000 7fa8f4fa1700 1 --2- 172.21.15.17:0/3037878350 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa8f013dad0 0x7fa8f0143bd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fa8d8001110 tx=0x7fa8e0006df0).ready entity=mon.2 client_cookie=bff9ee90e355fb06 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:20.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.150+0000 7fa8eeffd700 1 -- 172.21.15.17:0/3037878350 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8e0014070 con 0x7fa8f013dad0 2022-08-01T19:41:20.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.150+0000 7fa8eeffd700 1 -- 172.21.15.17:0/3037878350 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e0000e90 con 0x7fa8f013dad0 2022-08-01T19:41:20.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.150+0000 7fa8eeffd700 1 -- 172.21.15.17:0/3037878350 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8d8001660 con 0x7fa8f013dad0 2022-08-01T19:41:20.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.150+0000 7fa8f6a04700 1 -- 172.21.15.17:0/3037878350 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa8f013dad0 msgr2=0x7fa8f0143bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:20.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.150+0000 7fa8f6a04700 1 --2- 172.21.15.17:0/3037878350 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa8f013dad0 0x7fa8f0143bd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fa8d8001110 tx=0x7fa8e0006df0).stop 2022-08-01T19:41:20.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.150+0000 7fa8f6a04700 1 -- 172.21.15.17:0/3037878350 shutdown_connections 2022-08-01T19:41:20.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.150+0000 7fa8f6a04700 1 --2- 172.21.15.17:0/3037878350 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa8f013dad0 0x7fa8f0143bd0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:20.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.150+0000 7fa8f6a04700 1 -- 172.21.15.17:0/3037878350 >> 172.21.15.17:0/3037878350 conn(0x7fa8f00b5570 msgr2=0x7fa8f00b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:20.170 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.150+0000 7fa8f6a04700 1 -- 172.21.15.17:0/3037878350 shutdown_connections 2022-08-01T19:41:20.170 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.150+0000 7fa8f6a04700 1 -- 172.21.15.17:0/3037878350 wait complete. 2022-08-01T19:41:20.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.154+0000 7fa8f6a04700 1 Processor -- start 2022-08-01T19:41:20.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.154+0000 7fa8f6a04700 1 -- start start 2022-08-01T19:41:20.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.154+0000 7fa8f6a04700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8f00aad90 0x7fa8f00b0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:20.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.154+0000 7fa8f6a04700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8f013dad0 0x7fa8f00ae300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:20.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.154+0000 7fa8f6a04700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa8f01437f0 0x7fa8f00ac6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:20.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.154+0000 7fa8f6a04700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f0066660 con 0x7fa8f013dad0 2022-08-01T19:41:20.182 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.154+0000 7fa8f6a04700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f0140e90 con 0x7fa8f00aad90 2022-08-01T19:41:20.182 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.154+0000 7fa8f6a04700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f01525c0 con 0x7fa8f01437f0 2022-08-01T19:41:20.183 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.154+0000 7fa8ef7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8f013dad0 0x7fa8f00ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:20.183 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.154+0000 7fa8effff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8f00aad90 0x7fa8f00b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:20.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.154+0000 7fa8f4fa1700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa8f01437f0 0x7fa8f00ac6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:20.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.154+0000 7fa8ef7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8f013dad0 0x7fa8f00ae300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41832/0 (socket says 172.21.15.17:41832) 2022-08-01T19:41:20.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.154+0000 7fa8f4fa1700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa8f01437f0 0x7fa8f00ac6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:37934/0 (socket says 172.21.15.17:37934) 2022-08-01T19:41:20.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.154+0000 7fa8ef7fe700 1 -- 172.21.15.17:0/2521682572 learned_addr learned my addr 172.21.15.17:0/2521682572 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:20.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.154+0000 7fa8effff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8f00aad90 0x7fa8f00b0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:39752/0 (socket says 172.21.15.17:39752) 2022-08-01T19:41:20.188 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.154+0000 7fa8effff700 1 -- 172.21.15.17:0/2521682572 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa8f01437f0 msgr2=0x7fa8f00ac6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:20.188 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.154+0000 7fa8effff700 1 --2- 172.21.15.17:0/2521682572 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa8f01437f0 0x7fa8f00ac6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:20.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.154+0000 7fa8effff700 1 -- 172.21.15.17:0/2521682572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8f013dad0 msgr2=0x7fa8f00ae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:20.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.154+0000 7fa8effff700 1 --2- 172.21.15.17:0/2521682572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8f013dad0 0x7fa8f00ae300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:20.190 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.154+0000 7fa8effff700 1 -- 172.21.15.17:0/2521682572 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e000b180 con 0x7fa8f00aad90 2022-08-01T19:41:20.190 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.154+0000 7fa8ef7fe700 1 --2- 172.21.15.17:0/2521682572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8f013dad0 0x7fa8f00ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:20.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.154+0000 7fa8effff700 1 --2- 172.21.15.17:0/2521682572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8f00aad90 0x7fa8f00b0620 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7fa8f0140fd0 tx=0x7fa8d80016d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:20.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.154+0000 7fa8ed7fa700 1 -- 172.21.15.17:0/2521682572 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8d800cea0 con 0x7fa8f00aad90 2022-08-01T19:41:20.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.154+0000 7fa8ed7fa700 1 -- 172.21.15.17:0/2521682572 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d8007170 con 0x7fa8f00aad90 2022-08-01T19:41:20.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.154+0000 7fa8ed7fa700 1 -- 172.21.15.17:0/2521682572 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8d8004520 con 0x7fa8f00aad90 2022-08-01T19:41:20.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.154+0000 7fa8f6a04700 1 -- 172.21.15.17:0/2521682572 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8f0152080 con 0x7fa8f00aad90 2022-08-01T19:41:20.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.158+0000 7fa8f6a04700 1 -- 172.21.15.17:0/2521682572 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f00ad930 con 0x7fa8f00aad90 2022-08-01T19:41:20.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.158+0000 7fa8f6a04700 1 -- 172.21.15.17:0/2521682572 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8f005f840 con 0x7fa8f00aad90 2022-08-01T19:41:20.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.158+0000 7fa8ed7fa700 1 -- 172.21.15.17:0/2521682572 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa8d801b070 con 0x7fa8f00aad90 2022-08-01T19:41:20.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.158+0000 7fa8ed7fa700 1 --2- 172.21.15.17:0/2521682572 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa8d4053b00 0x7fa8d4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:20.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.158+0000 7fa8ed7fa700 1 -- 172.21.15.17:0/2521682572 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fa8d806b180 con 0x7fa8f00aad90 2022-08-01T19:41:20.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.158+0000 7fa8ef7fe700 1 --2- 172.21.15.17:0/2521682572 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa8d4053b00 0x7fa8d4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:20.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.162+0000 7fa8ef7fe700 1 --2- 172.21.15.17:0/2521682572 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa8d4053b00 0x7fa8d4055fc0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7fa8e800d000 tx=0x7fa8e8007e60).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:20.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.166+0000 7fa8ed7fa700 1 -- 172.21.15.17:0/2521682572 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa8d8042220 con 0x7fa8f00aad90 2022-08-01T19:41:20.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:20.302+0000 7fa8f6a04700 1 -- 172.21.15.17:0/2521682572 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fa8f00574e0 con 0x7fa8f00aad90 2022-08-01T19:41:21.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.182+0000 7fa8ed7fa700 1 -- 172.21.15.17:0/2521682572 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v94) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa8d8045840 con 0x7fa8f00aad90 2022-08-01T19:41:21.190 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.186+0000 7fa8f6a04700 1 -- 172.21.15.17:0/2521682572 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa8d4053b00 msgr2=0x7fa8d4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:21.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.186+0000 7fa8f6a04700 1 --2- 172.21.15.17:0/2521682572 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa8d4053b00 0x7fa8d4055fc0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7fa8e800d000 tx=0x7fa8e8007e60).stop 2022-08-01T19:41:21.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.186+0000 7fa8f6a04700 1 -- 172.21.15.17:0/2521682572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8f00aad90 msgr2=0x7fa8f00b0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:21.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.186+0000 7fa8f6a04700 1 --2- 172.21.15.17:0/2521682572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8f00aad90 0x7fa8f00b0620 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7fa8f0140fd0 tx=0x7fa8d80016d0).stop 2022-08-01T19:41:21.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.186+0000 7fa8f6a04700 1 -- 172.21.15.17:0/2521682572 shutdown_connections 2022-08-01T19:41:21.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.186+0000 7fa8f6a04700 1 --2- 172.21.15.17:0/2521682572 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa8d4053b00 0x7fa8d4055fc0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:21.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.186+0000 7fa8f6a04700 1 --2- 172.21.15.17:0/2521682572 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa8f01437f0 0x7fa8f00ac6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:21.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.186+0000 7fa8f6a04700 1 --2- 172.21.15.17:0/2521682572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8f013dad0 0x7fa8f00ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:21.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.186+0000 7fa8f6a04700 1 --2- 172.21.15.17:0/2521682572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8f00aad90 0x7fa8f00b0620 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:21.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.186+0000 7fa8f6a04700 1 -- 172.21.15.17:0/2521682572 >> 172.21.15.17:0/2521682572 conn(0x7fa8f00b5570 msgr2=0x7fa8f013af40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:21.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.186+0000 7fa8f6a04700 1 -- 172.21.15.17:0/2521682572 shutdown_connections 2022-08-01T19:41:21.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.186+0000 7fa8f6a04700 1 -- 172.21.15.17:0/2521682572 wait complete. 2022-08-01T19:41:21.194 INFO:teuthology.orchestra.run.smithi017.stderr: removed all blacklist entries 2022-08-01T19:41:21.204 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-08-01T19:41:21.205 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-08-01T19:41:21.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.310+0000 7f355a6a1700 1 Processor -- start 2022-08-01T19:41:21.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.310+0000 7f355a6a1700 1 -- start start 2022-08-01T19:41:21.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.310+0000 7f355a6a1700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35540b01a0 0x7f35540a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:21.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.310+0000 7f355a6a1700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35540a7250 con 0x7f35540afd60 2022-08-01T19:41:21.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.310+0000 7f355a6a1700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35540a7480 con 0x7f35540b2710 2022-08-01T19:41:21.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.310+0000 7f355a6a1700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35540a7720 con 0x7f35540b01a0 2022-08-01T19:41:21.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.314+0000 7f3559ea0700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f35540afd60 0x7f3554151d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34040/0 (socket says 172.21.15.17:34040) 2022-08-01T19:41:21.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.314+0000 7f3559ea0700 1 -- 172.21.15.17:0/2512135971 learned_addr learned my addr 172.21.15.17:0/2512135971 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:21.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.314+0000 7f3558e9e700 1 --2- 172.21.15.17:0/2512135971 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35540b01a0 0x7f35540a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:21.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.314+0000 7f354bfff700 1 -- 172.21.15.17:0/2512135971 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2875413634 0 0) 0x7f35540a7250 con 0x7f35540afd60 2022-08-01T19:41:21.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.314+0000 7f354bfff700 1 -- 172.21.15.17:0/2512135971 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3538003260 con 0x7f35540afd60 2022-08-01T19:41:21.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.314+0000 7f354bfff700 1 -- 172.21.15.17:0/2512135971 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2971715915 0 0) 0x7f35540a7480 con 0x7f35540b2710 2022-08-01T19:41:21.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.314+0000 7f354bfff700 1 -- 172.21.15.17:0/2512135971 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35540a7250 con 0x7f35540b2710 2022-08-01T19:41:21.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.314+0000 7f3558e9e700 1 -- 172.21.15.17:0/2512135971 >> v1:172.21.15.77:6789/0 conn(0x7f35540b2710 legacy=0x7f355414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:21.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.314+0000 7f3558e9e700 1 -- 172.21.15.17:0/2512135971 >> v1:172.21.15.17:6789/0 conn(0x7f35540afd60 legacy=0x7f3554151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:21.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.314+0000 7f3558e9e700 1 -- 172.21.15.17:0/2512135971 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35540a79c0 con 0x7f35540b01a0 2022-08-01T19:41:21.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.314+0000 7f3558e9e700 1 --2- 172.21.15.17:0/2512135971 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35540b01a0 0x7f35540a8540 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f3550002f70 tx=0x7f3550006df0).ready entity=mon.2 client_cookie=356fb10119dd6766 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:21.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.314+0000 7f354bfff700 1 -- 172.21.15.17:0/2512135971 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f355000d5c0 con 0x7f35540b01a0 2022-08-01T19:41:21.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.314+0000 7f354bfff700 1 -- 172.21.15.17:0/2512135971 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3550000e90 con 0x7f35540b01a0 2022-08-01T19:41:21.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.314+0000 7f354bfff700 1 -- 172.21.15.17:0/2512135971 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f355000d910 con 0x7f35540b01a0 2022-08-01T19:41:21.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.314+0000 7f355a6a1700 1 -- 172.21.15.17:0/2512135971 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35540b01a0 msgr2=0x7f35540a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:21.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.314+0000 7f355a6a1700 1 --2- 172.21.15.17:0/2512135971 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35540b01a0 0x7f35540a8540 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f3550002f70 tx=0x7f3550006df0).stop 2022-08-01T19:41:21.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.314+0000 7f355a6a1700 1 -- 172.21.15.17:0/2512135971 shutdown_connections 2022-08-01T19:41:21.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.314+0000 7f355a6a1700 1 --2- 172.21.15.17:0/2512135971 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35540b01a0 0x7f35540a8540 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:21.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.314+0000 7f355a6a1700 1 -- 172.21.15.17:0/2512135971 >> 172.21.15.17:0/2512135971 conn(0x7f3554060310 msgr2=0x7f35540b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:21.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.314+0000 7f355a6a1700 1 -- 172.21.15.17:0/2512135971 shutdown_connections 2022-08-01T19:41:21.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.314+0000 7f355a6a1700 1 -- 172.21.15.17:0/2512135971 wait complete. 2022-08-01T19:41:21.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.314+0000 7f355a6a1700 1 Processor -- start 2022-08-01T19:41:21.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.314+0000 7f355a6a1700 1 -- start start 2022-08-01T19:41:21.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.318+0000 7f355a6a1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35540afd60 0x7f35540b5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:21.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.318+0000 7f355a6a1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35540b01a0 0x7f35541423a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:21.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.318+0000 7f355a6a1700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35540b2710 0x7f35541497e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:21.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.318+0000 7f355a6a1700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35540b5e00 con 0x7f35540b01a0 2022-08-01T19:41:21.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.318+0000 7f355a6a1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3554142890 con 0x7f35540afd60 2022-08-01T19:41:21.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.318+0000 7f355a6a1700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35540b2f60 con 0x7f35540b2710 2022-08-01T19:41:21.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.318+0000 7f3558e9e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35540b01a0 0x7f35541423a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:21.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.318+0000 7f3558e9e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35540b01a0 0x7f35541423a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41850/0 (socket says 172.21.15.17:41850) 2022-08-01T19:41:21.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.318+0000 7f355969f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35540afd60 0x7f35540b5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:21.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.318+0000 7f3558e9e700 1 -- 172.21.15.17:0/886841269 learned_addr learned my addr 172.21.15.17:0/886841269 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:21.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.318+0000 7f3559ea0700 1 --2- 172.21.15.17:0/886841269 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35540b2710 0x7f35541497e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:21.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.318+0000 7f3559ea0700 1 -- 172.21.15.17:0/886841269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35540afd60 msgr2=0x7f35540b5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:21.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.318+0000 7f3559ea0700 1 --2- 172.21.15.17:0/886841269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35540afd60 0x7f35540b5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:21.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.318+0000 7f3559ea0700 1 -- 172.21.15.17:0/886841269 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35540b01a0 msgr2=0x7f35541423a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:21.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.318+0000 7f3559ea0700 1 --2- 172.21.15.17:0/886841269 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35540b01a0 0x7f35541423a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:21.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.318+0000 7f3559ea0700 1 -- 172.21.15.17:0/886841269 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f355000b180 con 0x7f35540b2710 2022-08-01T19:41:21.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.318+0000 7f3558e9e700 1 --2- 172.21.15.17:0/886841269 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35540b01a0 0x7f35541423a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:21.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.318+0000 7f3559ea0700 1 --2- 172.21.15.17:0/886841269 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35540b2710 0x7f35541497e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f354c001490 tx=0x7f354c007d60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:21.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.318+0000 7f354a7fc700 1 -- 172.21.15.17:0/886841269 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f354c005ba0 con 0x7f35540b2710 2022-08-01T19:41:21.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.318+0000 7f354a7fc700 1 -- 172.21.15.17:0/886841269 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354c005d00 con 0x7f35540b2710 2022-08-01T19:41:21.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.318+0000 7f355a6a1700 1 -- 172.21.15.17:0/886841269 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35540b3720 con 0x7f35540b2710 2022-08-01T19:41:21.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.318+0000 7f354a7fc700 1 -- 172.21.15.17:0/886841269 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f354c0072c0 con 0x7f35540b2710 2022-08-01T19:41:21.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.318+0000 7f355a6a1700 1 -- 172.21.15.17:0/886841269 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35540b3980 con 0x7f35540b2710 2022-08-01T19:41:21.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.322+0000 7f355a6a1700 1 -- 172.21.15.17:0/886841269 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35540574e0 con 0x7f35540b2710 2022-08-01T19:41:21.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.322+0000 7f354a7fc700 1 -- 172.21.15.17:0/886841269 <== mon.2 v2:172.21.15.77:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f354c01a060 con 0x7f35540b2710 2022-08-01T19:41:21.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.322+0000 7f354a7fc700 1 --2- 172.21.15.17:0/886841269 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f353c053950 0x7f353c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:21.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.322+0000 7f354a7fc700 1 -- 172.21.15.17:0/886841269 <== mon.2 v2:172.21.15.77:3301/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f354c06b730 con 0x7f35540b2710 2022-08-01T19:41:21.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.322+0000 7f355969f700 1 --2- 172.21.15.17:0/886841269 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f353c053950 0x7f353c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:21.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.322+0000 7f355969f700 1 --2- 172.21.15.17:0/886841269 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f353c053950 0x7f353c055e10 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f354400d000 tx=0x7f3544007a00).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:21.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.330+0000 7f354a7fc700 1 -- 172.21.15.17:0/886841269 <== mon.2 v2:172.21.15.77:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f354c0427b0 con 0x7f35540b2710 2022-08-01T19:41:21.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.502+0000 7f355a6a1700 1 -- 172.21.15.17:0/886841269 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3554069a70 con 0x7f35540b2710 2022-08-01T19:41:21.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.502+0000 7f354a7fc700 1 -- 172.21.15.17:0/886841269 <== mon.2 v2:172.21.15.77:3301/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v26) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f354c018070 con 0x7f35540b2710 2022-08-01T19:41:21.508 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:21.508 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:41:21.508 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_dump": [ 2022-08-01T19:41:21.508 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:21.509 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.a", 2022-08-01T19:41:21.509 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhijotxORAAVc04rsrlFj0NShsyc6Qbhg==", 2022-08-01T19:41:21.509 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:41:21.509 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:41:21.510 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:41:21.510 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:41:21.510 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:41:21.510 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:21.511 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:21.511 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:21.511 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.b", 2022-08-01T19:41:21.511 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhiud2xBxAAcn6/O5ysdIp/oIbF5Y1dAw==", 2022-08-01T19:41:21.512 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:41:21.512 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:41:21.512 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:41:21.512 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:41:21.512 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:41:21.513 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:21.513 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:21.513 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:21.514 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.c", 2022-08-01T19:41:21.514 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+KuhiMzMbBBAAfdT5mtpYwboNL+7f2B09KQ==", 2022-08-01T19:41:21.514 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:41:21.514 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:41:21.514 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:41:21.515 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:41:21.515 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:41:21.515 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:21.515 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:21.516 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:21.516 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.0", 2022-08-01T19:41:21.516 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhia27/NRAAZXBp+nYelJVJ6Mge9r+3Vg==", 2022-08-01T19:41:21.516 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:41:21.517 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:41:21.517 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:41:21.517 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:41:21.518 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:21.518 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:21.518 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:21.518 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.1", 2022-08-01T19:41:21.519 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCBKuhiOggRGRAAvgA0MxwBJwmpXV6S9DTdjA==", 2022-08-01T19:41:21.519 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:41:21.519 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:41:21.519 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:41:21.520 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:41:21.520 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:21.520 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:21.520 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:21.521 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.2", 2022-08-01T19:41:21.521 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCDKuhi5PgOORAAhcORK4bW2T3dXPeYjcCm+g==", 2022-08-01T19:41:21.522 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:41:21.523 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:41:21.523 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:41:21.523 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:41:21.523 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:21.524 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:21.524 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:21.524 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.3", 2022-08-01T19:41:21.524 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCGKuhiutn6GxAAk2ucLuGbT5YV/R4xXQGh0w==", 2022-08-01T19:41:21.525 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:41:21.525 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:41:21.525 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:41:21.525 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:41:21.526 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:21.526 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:21.526 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:21.526 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.4", 2022-08-01T19:41:21.527 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCLKuhiGHTRHxAArr3TwXA0gsCAAF/Qs38wUQ==", 2022-08-01T19:41:21.527 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:41:21.527 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:41:21.527 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:41:21.528 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:41:21.528 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:21.528 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:21.528 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:21.529 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.5", 2022-08-01T19:41:21.529 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCOKuhiMMDbBBAAvTzBvOcv7pFWm/P/lNqB0w==", 2022-08-01T19:41:21.529 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:41:21.529 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:41:21.530 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:41:21.530 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:41:21.530 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:21.530 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:21.531 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:21.531 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.6", 2022-08-01T19:41:21.531 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCQKuhiG2H3IxAAufUcWqo9ULquxIrUD5vSfA==", 2022-08-01T19:41:21.531 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:41:21.532 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:41:21.532 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:41:21.532 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:41:21.532 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:21.533 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:21.533 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:21.533 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.7", 2022-08-01T19:41:21.533 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCTKuhiU50LChAAXuhrYBbHkk3eKzAPiEJFTg==", 2022-08-01T19:41:21.534 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:41:21.534 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:41:21.535 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:41:21.535 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:41:21.535 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:21.535 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:21.536 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:21.536 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.0", 2022-08-01T19:41:21.536 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhig3EnCxAANM20jSIQf/AKwuRX1qwT/A==", 2022-08-01T19:41:21.536 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:41:21.537 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:41:21.537 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow r", 2022-08-01T19:41:21.537 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T19:41:21.537 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:21.537 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:21.538 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:21.538 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.1", 2022-08-01T19:41:21.538 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+KuhicJR0DhAAivqEAZUhXnSwjVRwWm2bJg==", 2022-08-01T19:41:21.538 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:41:21.539 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:41:21.540 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow r", 2022-08-01T19:41:21.540 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T19:41:21.541 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:21.541 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:21.542 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:21.543 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.admin", 2022-08-01T19:41:21.543 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhiPTTGHxAAgfthCR6RX6j8F1YseFGNeA==", 2022-08-01T19:41:21.543 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:41:21.544 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:41:21.544 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:41:21.545 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:41:21.545 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:41:21.545 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:21.546 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:21.546 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:21.546 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.x", 2022-08-01T19:41:21.547 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9Kuhi2jvCNhAAyUxiGByZ+OZr8Y7dh3Qt5w==", 2022-08-01T19:41:21.547 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:41:21.547 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:41:21.547 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2022-08-01T19:41:21.548 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:41:21.548 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:21.548 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:21.549 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:21.549 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.y", 2022-08-01T19:41:21.549 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhiDJXpMhAA8aek67ydbRgNO0h+miws+w==", 2022-08-01T19:41:21.549 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:41:21.550 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:41:21.550 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2022-08-01T19:41:21.550 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:41:21.551 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:21.551 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:21.551 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:21.551 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:41:21.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.506+0000 7f355a6a1700 1 -- 172.21.15.17:0/886841269 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f353c053950 msgr2=0x7f353c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:21.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.506+0000 7f355a6a1700 1 --2- 172.21.15.17:0/886841269 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f353c053950 0x7f353c055e10 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f354400d000 tx=0x7f3544007a00).stop 2022-08-01T19:41:21.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.506+0000 7f355a6a1700 1 -- 172.21.15.17:0/886841269 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35540b2710 msgr2=0x7f35541497e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:21.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.506+0000 7f355a6a1700 1 --2- 172.21.15.17:0/886841269 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35540b2710 0x7f35541497e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f354c001490 tx=0x7f354c007d60).stop 2022-08-01T19:41:21.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.506+0000 7f355a6a1700 1 -- 172.21.15.17:0/886841269 shutdown_connections 2022-08-01T19:41:21.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.506+0000 7f355a6a1700 1 --2- 172.21.15.17:0/886841269 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f353c053950 0x7f353c055e10 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:21.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.506+0000 7f355a6a1700 1 --2- 172.21.15.17:0/886841269 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35540b2710 0x7f35541497e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:21.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.506+0000 7f355a6a1700 1 --2- 172.21.15.17:0/886841269 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35540b01a0 0x7f35541423a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:21.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.506+0000 7f355a6a1700 1 --2- 172.21.15.17:0/886841269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35540afd60 0x7f35540b5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:21.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.506+0000 7f355a6a1700 1 -- 172.21.15.17:0/886841269 >> 172.21.15.17:0/886841269 conn(0x7f3554060310 msgr2=0x7f3554145310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:21.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.506+0000 7f355a6a1700 1 -- 172.21.15.17:0/886841269 shutdown_connections 2022-08-01T19:41:21.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.506+0000 7f355a6a1700 1 -- 172.21.15.17:0/886841269 wait complete. 2022-08-01T19:41:21.557 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:41:21.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.654+0000 7f2542ec5700 1 Processor -- start 2022-08-01T19:41:21.660 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.654+0000 7f2542ec5700 1 -- start start 2022-08-01T19:41:21.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.654+0000 7f2542ec5700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f253c0b31d0 0x7f253c066020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:21.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.654+0000 7f2542ec5700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f253c0a9da0 0x7f253c060950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:21.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.654+0000 7f2542ec5700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f253c067030 con 0x7f253c1434b0 2022-08-01T19:41:21.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.654+0000 7f2542ec5700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f253c064a70 con 0x7f253c0b31d0 2022-08-01T19:41:21.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.654+0000 7f2542ec5700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253c061f60 con 0x7f253c0a9da0 2022-08-01T19:41:21.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.654+0000 7f25426c4700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f253c1434b0 0x7f253c143940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34052/0 (socket says 172.21.15.17:34052) 2022-08-01T19:41:21.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.654+0000 7f25426c4700 1 -- 172.21.15.17:0/2941695352 learned_addr learned my addr 172.21.15.17:0/2941695352 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:21.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.654+0000 7f2541ec3700 1 --2- 172.21.15.17:0/2941695352 >> v2:172.21.15.77:3300/0 conn(0x7f253c0b31d0 0x7f253c066020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:21.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.654+0000 7f25416c2700 1 --2- 172.21.15.17:0/2941695352 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f253c0a9da0 0x7f253c060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:21.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.654+0000 7f2540ec1700 1 -- 172.21.15.17:0/2941695352 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3170655484 0 0) 0x7f253c067030 con 0x7f253c1434b0 2022-08-01T19:41:21.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.654+0000 7f2540ec1700 1 -- 172.21.15.17:0/2941695352 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f251c003260 con 0x7f253c1434b0 2022-08-01T19:41:21.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.654+0000 7f2541ec3700 1 -- 172.21.15.17:0/2941695352 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f253c0a9da0 msgr2=0x7f253c060950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:21.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.654+0000 7f2541ec3700 1 --2- 172.21.15.17:0/2941695352 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f253c0a9da0 0x7f253c060950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:21.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.654+0000 7f2541ec3700 1 -- 172.21.15.17:0/2941695352 >> v1:172.21.15.17:6789/0 conn(0x7f253c1434b0 legacy=0x7f253c143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:21.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.654+0000 7f2541ec3700 1 -- 172.21.15.17:0/2941695352 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253c0b35b0 con 0x7f253c0b31d0 2022-08-01T19:41:21.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.654+0000 7f2541ec3700 1 --2- 172.21.15.17:0/2941695352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f253c0b31d0 0x7f253c066020 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f252c0032f0 tx=0x7f252c0091d0).ready entity=mon.1 client_cookie=d8c2b212496aa16 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:21.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.654+0000 7f2540ec1700 1 -- 172.21.15.17:0/2941695352 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f252c00b600 con 0x7f253c0b31d0 2022-08-01T19:41:21.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.654+0000 7f2540ec1700 1 -- 172.21.15.17:0/2941695352 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f252c00b760 con 0x7f253c0b31d0 2022-08-01T19:41:21.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.654+0000 7f2540ec1700 1 -- 172.21.15.17:0/2941695352 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f252c00b910 con 0x7f253c0b31d0 2022-08-01T19:41:21.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.658+0000 7f2542ec5700 1 -- 172.21.15.17:0/2941695352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f253c0b31d0 msgr2=0x7f253c066020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:21.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.658+0000 7f2542ec5700 1 --2- 172.21.15.17:0/2941695352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f253c0b31d0 0x7f253c066020 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f252c0032f0 tx=0x7f252c0091d0).stop 2022-08-01T19:41:21.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.658+0000 7f2542ec5700 1 -- 172.21.15.17:0/2941695352 shutdown_connections 2022-08-01T19:41:21.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.658+0000 7f2542ec5700 1 --2- 172.21.15.17:0/2941695352 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f253c0a9da0 0x7f253c060950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:21.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.658+0000 7f2542ec5700 1 --2- 172.21.15.17:0/2941695352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f253c0b31d0 0x7f253c066020 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:21.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.658+0000 7f2542ec5700 1 -- 172.21.15.17:0/2941695352 >> 172.21.15.17:0/2941695352 conn(0x7f253c063870 msgr2=0x7f253c063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:21.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.658+0000 7f2542ec5700 1 -- 172.21.15.17:0/2941695352 shutdown_connections 2022-08-01T19:41:21.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.658+0000 7f2542ec5700 1 -- 172.21.15.17:0/2941695352 wait complete. 2022-08-01T19:41:21.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.658+0000 7f2542ec5700 1 Processor -- start 2022-08-01T19:41:21.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.658+0000 7f2542ec5700 1 -- start start 2022-08-01T19:41:21.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.658+0000 7f2542ec5700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f253c0a9da0 0x7f253c0b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:21.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.658+0000 7f2542ec5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f253c0b31d0 0x7f253c13b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:21.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.658+0000 7f2542ec5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f253c1434b0 0x7f253c134c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:21.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.658+0000 7f2542ec5700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c0b4500 con 0x7f253c1434b0 2022-08-01T19:41:21.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.658+0000 7f2542ec5700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c0b4730 con 0x7f253c0b31d0 2022-08-01T19:41:21.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.658+0000 7f2542ec5700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253c136a70 con 0x7f253c0a9da0 2022-08-01T19:41:21.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.658+0000 7f25426c4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f253c1434b0 0x7f253c134c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:21.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.658+0000 7f25426c4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f253c1434b0 0x7f253c134c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41866/0 (socket says 172.21.15.17:41866) 2022-08-01T19:41:21.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.658+0000 7f25426c4700 1 -- 172.21.15.17:0/4157384771 learned_addr learned my addr 172.21.15.17:0/4157384771 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:21.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.658+0000 7f2541ec3700 1 --2- 172.21.15.17:0/4157384771 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f253c0a9da0 0x7f253c0b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:21.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.658+0000 7f25416c2700 1 --2- 172.21.15.17:0/4157384771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f253c0b31d0 0x7f253c13b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:21.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.658+0000 7f25426c4700 1 -- 172.21.15.17:0/4157384771 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f253c0a9da0 msgr2=0x7f253c0b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:21.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.658+0000 7f25426c4700 1 --2- 172.21.15.17:0/4157384771 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f253c0a9da0 0x7f253c0b57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:21.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.658+0000 7f25426c4700 1 -- 172.21.15.17:0/4157384771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f253c0b31d0 msgr2=0x7f253c13b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:21.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.658+0000 7f25426c4700 1 --2- 172.21.15.17:0/4157384771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f253c0b31d0 0x7f253c13b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:21.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.658+0000 7f25426c4700 1 -- 172.21.15.17:0/4157384771 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f252c009470 con 0x7f253c1434b0 2022-08-01T19:41:21.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.662+0000 7f25426c4700 1 --2- 172.21.15.17:0/4157384771 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f253c1434b0 0x7f253c134c70 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 rx=0x7f2534001110 tx=0x7f2534007a20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:21.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.662+0000 7f253affd700 1 -- 172.21.15.17:0/4157384771 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f253400ce00 con 0x7f253c1434b0 2022-08-01T19:41:21.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.662+0000 7f2542ec5700 1 -- 172.21.15.17:0/4157384771 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f253c136f10 con 0x7f253c1434b0 2022-08-01T19:41:21.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.662+0000 7f253affd700 1 -- 172.21.15.17:0/4157384771 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25340070d0 con 0x7f253c1434b0 2022-08-01T19:41:21.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.662+0000 7f253affd700 1 -- 172.21.15.17:0/4157384771 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2534007430 con 0x7f253c1434b0 2022-08-01T19:41:21.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.662+0000 7f2542ec5700 1 -- 172.21.15.17:0/4157384771 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f253c0b42e0 con 0x7f253c1434b0 2022-08-01T19:41:21.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.662+0000 7f2542ec5700 1 -- 172.21.15.17:0/4157384771 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f253c0b4930 con 0x7f253c1434b0 2022-08-01T19:41:21.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.662+0000 7f253affd700 1 -- 172.21.15.17:0/4157384771 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2534010070 con 0x7f253c1434b0 2022-08-01T19:41:21.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.662+0000 7f253affd700 1 --2- 172.21.15.17:0/4157384771 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f25200539e0 0x7f2520055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:21.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.662+0000 7f253affd700 1 -- 172.21.15.17:0/4157384771 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f253406b2f0 con 0x7f253c1434b0 2022-08-01T19:41:21.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.662+0000 7f2541ec3700 1 --2- 172.21.15.17:0/4157384771 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f25200539e0 0x7f2520055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:21.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.666+0000 7f2541ec3700 1 --2- 172.21.15.17:0/4157384771 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f25200539e0 0x7f2520055ea0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f252c01b000 tx=0x7f252c016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:21.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.666+0000 7f253affd700 1 -- 172.21.15.17:0/4157384771 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2534042240 con 0x7f253c1434b0 2022-08-01T19:41:21.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.794+0000 7f2542ec5700 1 -- 172.21.15.17:0/4157384771 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f253c066af0 con 0x7f253c1434b0 2022-08-01T19:41:21.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.794+0000 7f253affd700 1 -- 172.21.15.17:0/4157384771 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+13415 (secure 0 0 0) 0x7f2534018070 con 0x7f253c1434b0 2022-08-01T19:41:21.802 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:21.803 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":94,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:41:21.176618+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:41:21.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.794+0000 7f2542ec5700 1 -- 172.21.15.17:0/4157384771 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f25200539e0 msgr2=0x7f2520055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:21.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.794+0000 7f2542ec5700 1 --2- 172.21.15.17:0/4157384771 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f25200539e0 0x7f2520055ea0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f252c01b000 tx=0x7f252c016000).stop 2022-08-01T19:41:21.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.794+0000 7f2542ec5700 1 -- 172.21.15.17:0/4157384771 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f253c1434b0 msgr2=0x7f253c134c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:21.807 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.794+0000 7f2542ec5700 1 --2- 172.21.15.17:0/4157384771 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f253c1434b0 0x7f253c134c70 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 rx=0x7f2534001110 tx=0x7f2534007a20).stop 2022-08-01T19:41:21.807 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.794+0000 7f2542ec5700 1 -- 172.21.15.17:0/4157384771 shutdown_connections 2022-08-01T19:41:21.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.794+0000 7f2542ec5700 1 --2- 172.21.15.17:0/4157384771 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f25200539e0 0x7f2520055ea0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:21.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.794+0000 7f2542ec5700 1 --2- 172.21.15.17:0/4157384771 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f253c1434b0 0x7f253c134c70 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:21.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.794+0000 7f2542ec5700 1 --2- 172.21.15.17:0/4157384771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f253c0b31d0 0x7f253c13b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:21.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.794+0000 7f2542ec5700 1 --2- 172.21.15.17:0/4157384771 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f253c0a9da0 0x7f253c0b57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:21.818 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.794+0000 7f2542ec5700 1 -- 172.21.15.17:0/4157384771 >> 172.21.15.17:0/4157384771 conn(0x7f253c063870 msgr2=0x7f253c0b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:21.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.794+0000 7f2542ec5700 1 -- 172.21.15.17:0/4157384771 shutdown_connections 2022-08-01T19:41:21.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.794+0000 7f2542ec5700 1 -- 172.21.15.17:0/4157384771 wait complete. 2022-08-01T19:41:21.826 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-08-01T19:41:21.827 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:41:21.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.938+0000 7f8415543700 1 Processor -- start 2022-08-01T19:41:21.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.938+0000 7f8415543700 1 -- start start 2022-08-01T19:41:21.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.938+0000 7f8415543700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84100b4310 0x7f8410140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:21.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.938+0000 7f8415543700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f84100b4820 0x7f84100b5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:21.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.938+0000 7f8415543700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8410143cf0 con 0x7f8410146ae0 2022-08-01T19:41:21.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.938+0000 7f8415543700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84101476a0 con 0x7f84100b4820 2022-08-01T19:41:21.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.938+0000 7f8415543700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8410143f90 con 0x7f84100b4310 2022-08-01T19:41:21.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.938+0000 7f840f7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f84100b4820 0x7f84100b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:21.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.938+0000 7f840f7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f84100b4820 0x7f84100b5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41874/0 (socket says 172.21.15.17:41874) 2022-08-01T19:41:21.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.938+0000 7f840f7fe700 1 -- 172.21.15.17:0/2486515143 learned_addr learned my addr 172.21.15.17:0/2486515143 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:21.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.938+0000 7f840e7fc700 1 --2- 172.21.15.17:0/2486515143 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84100b4310 0x7f8410140ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:21.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.938+0000 7f840f7fe700 1 -- 172.21.15.17:0/2486515143 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84100b4310 msgr2=0x7f8410140ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:21.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.938+0000 7f840f7fe700 1 --2- 172.21.15.17:0/2486515143 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84100b4310 0x7f8410140ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:21.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.938+0000 7f840dffb700 1 -- 172.21.15.17:0/2486515143 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2516273390 0 0) 0x7f8410143cf0 con 0x7f8410146ae0 2022-08-01T19:41:21.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.938+0000 7f840f7fe700 1 -- 172.21.15.17:0/2486515143 >> v1:172.21.15.77:6789/0 conn(0x7f8410146ae0 legacy=0x7f84101403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:21.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.938+0000 7f840f7fe700 1 -- 172.21.15.17:0/2486515143 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84100b0e20 con 0x7f84100b4820 2022-08-01T19:41:21.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.938+0000 7f840f7fe700 1 --2- 172.21.15.17:0/2486515143 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84100b4820 0x7f84100b5060 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 rx=0x7f84080032f0 tx=0x7f8408009290).ready entity=mon.0 client_cookie=ae8783b9cb9701fb server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:21.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.938+0000 7f840dffb700 1 -- 172.21.15.17:0/2486515143 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f840800b8a0 con 0x7f84100b4820 2022-08-01T19:41:21.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.938+0000 7f840dffb700 1 -- 172.21.15.17:0/2486515143 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840800ba00 con 0x7f84100b4820 2022-08-01T19:41:21.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.938+0000 7f840dffb700 1 -- 172.21.15.17:0/2486515143 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f840800bb70 con 0x7f84100b4820 2022-08-01T19:41:21.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.942+0000 7f8415543700 1 -- 172.21.15.17:0/2486515143 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84100b4820 msgr2=0x7f84100b5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:21.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.942+0000 7f8415543700 1 --2- 172.21.15.17:0/2486515143 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84100b4820 0x7f84100b5060 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 rx=0x7f84080032f0 tx=0x7f8408009290).stop 2022-08-01T19:41:21.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.942+0000 7f8415543700 1 -- 172.21.15.17:0/2486515143 shutdown_connections 2022-08-01T19:41:21.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.942+0000 7f8415543700 1 --2- 172.21.15.17:0/2486515143 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84100b4820 0x7f84100b5060 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:21.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.942+0000 7f8415543700 1 --2- 172.21.15.17:0/2486515143 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84100b4310 0x7f8410140ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:21.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.942+0000 7f8415543700 1 -- 172.21.15.17:0/2486515143 >> 172.21.15.17:0/2486515143 conn(0x7f841005c620 msgr2=0x7f841005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:21.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.942+0000 7f8415543700 1 -- 172.21.15.17:0/2486515143 shutdown_connections 2022-08-01T19:41:21.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.942+0000 7f8415543700 1 -- 172.21.15.17:0/2486515143 wait complete. 2022-08-01T19:41:21.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.942+0000 7f8415543700 1 Processor -- start 2022-08-01T19:41:21.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.942+0000 7f8415543700 1 -- start start 2022-08-01T19:41:21.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.942+0000 7f8415543700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84100b4310 0x7f84101424a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:21.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.942+0000 7f8415543700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84100b4820 0x7f8410142f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:21.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.942+0000 7f8415543700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8410146ae0 0x7f84100b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:21.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.942+0000 7f8415543700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84100af890 con 0x7f84100b4310 2022-08-01T19:41:21.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.942+0000 7f8415543700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841013dbb0 con 0x7f84100b4820 2022-08-01T19:41:21.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.942+0000 7f8415543700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84100adc40 con 0x7f8410146ae0 2022-08-01T19:41:21.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.942+0000 7f840effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84100b4310 0x7f84101424a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:21.992 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.942+0000 7f840effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84100b4310 0x7f84101424a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41876/0 (socket says 172.21.15.17:41876) 2022-08-01T19:41:21.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.942+0000 7f840effd700 1 -- 172.21.15.17:0/2605522362 learned_addr learned my addr 172.21.15.17:0/2605522362 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:21.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.942+0000 7f840e7fc700 1 --2- 172.21.15.17:0/2605522362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84100b4820 0x7f8410142f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:21.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.942+0000 7f840f7fe700 1 --2- 172.21.15.17:0/2605522362 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8410146ae0 0x7f84100b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:21.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.942+0000 7f840effd700 1 -- 172.21.15.17:0/2605522362 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8410146ae0 msgr2=0x7f84100b00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:21.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.942+0000 7f840effd700 1 --2- 172.21.15.17:0/2605522362 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8410146ae0 0x7f84100b00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:21.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.942+0000 7f840effd700 1 -- 172.21.15.17:0/2605522362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84100b4820 msgr2=0x7f8410142f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:21.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.942+0000 7f840effd700 1 --2- 172.21.15.17:0/2605522362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84100b4820 0x7f8410142f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:21.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.942+0000 7f840effd700 1 -- 172.21.15.17:0/2605522362 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8408009530 con 0x7f84100b4310 2022-08-01T19:41:21.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.942+0000 7f840e7fc700 1 --2- 172.21.15.17:0/2605522362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84100b4820 0x7f8410142f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:21.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.942+0000 7f840effd700 1 --2- 172.21.15.17:0/2605522362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84100b4310 0x7f84101424a0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 rx=0x7f8400000ed0 tx=0x7f8400007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:21.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.942+0000 7f83fffff700 1 -- 172.21.15.17:0/2605522362 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f840000ce80 con 0x7f84100b4310 2022-08-01T19:41:21.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.942+0000 7f83fffff700 1 -- 172.21.15.17:0/2605522362 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840000f470 con 0x7f84100b4310 2022-08-01T19:41:21.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.942+0000 7f8415543700 1 -- 172.21.15.17:0/2605522362 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84100adee0 con 0x7f84100b4310 2022-08-01T19:41:21.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.942+0000 7f83fffff700 1 -- 172.21.15.17:0/2605522362 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f840000f7d0 con 0x7f84100b4310 2022-08-01T19:41:21.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.942+0000 7f8415543700 1 -- 172.21.15.17:0/2605522362 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f841013e300 con 0x7f84100b4310 2022-08-01T19:41:21.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.946+0000 7f8415543700 1 -- 172.21.15.17:0/2605522362 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84100574e0 con 0x7f84100b4310 2022-08-01T19:41:21.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.946+0000 7f83fffff700 1 -- 172.21.15.17:0/2605522362 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8400019030 con 0x7f84100b4310 2022-08-01T19:41:21.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.946+0000 7f83fffff700 1 --2- 172.21.15.17:0/2605522362 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f83ec053a70 0x7f83ec055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:22.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.946+0000 7f83fffff700 1 -- 172.21.15.17:0/2605522362 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f840006b290 con 0x7f84100b4310 2022-08-01T19:41:22.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.946+0000 7f840e7fc700 1 --2- 172.21.15.17:0/2605522362 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f83ec053a70 0x7f83ec055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:22.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.946+0000 7f840e7fc700 1 --2- 172.21.15.17:0/2605522362 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f83ec053a70 0x7f83ec055f30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f840400b000 tx=0x7f8404008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:22.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:21.950+0000 7f83fffff700 1 -- 172.21.15.17:0/2605522362 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8400068e90 con 0x7f84100b4310 2022-08-01T19:41:22.080 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.074+0000 7f8415543700 1 -- 172.21.15.17:0/2605522362 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f8410069a70 con 0x7f84100b4310 2022-08-01T19:41:22.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.074+0000 7f83fffff700 1 -- 172.21.15.17:0/2605522362 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v94) v1 ==== 110+0+10 (secure 0 0 0) 0x7f8400042280 con 0x7f84100b4310 2022-08-01T19:41:22.081 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:41:22.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.078+0000 7f8415543700 1 -- 172.21.15.17:0/2605522362 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f83ec053a70 msgr2=0x7f83ec055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:22.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.078+0000 7f8415543700 1 --2- 172.21.15.17:0/2605522362 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f83ec053a70 0x7f83ec055f30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f840400b000 tx=0x7f8404008040).stop 2022-08-01T19:41:22.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.078+0000 7f8415543700 1 -- 172.21.15.17:0/2605522362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84100b4310 msgr2=0x7f84101424a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:22.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.078+0000 7f8415543700 1 --2- 172.21.15.17:0/2605522362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84100b4310 0x7f84101424a0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 rx=0x7f8400000ed0 tx=0x7f8400007c50).stop 2022-08-01T19:41:22.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.078+0000 7f8415543700 1 -- 172.21.15.17:0/2605522362 shutdown_connections 2022-08-01T19:41:22.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.078+0000 7f8415543700 1 --2- 172.21.15.17:0/2605522362 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f83ec053a70 0x7f83ec055f30 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:22.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.078+0000 7f8415543700 1 --2- 172.21.15.17:0/2605522362 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8410146ae0 0x7f84100b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:22.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.078+0000 7f8415543700 1 --2- 172.21.15.17:0/2605522362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84100b4820 0x7f8410142f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:22.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.078+0000 7f8415543700 1 --2- 172.21.15.17:0/2605522362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84100b4310 0x7f84101424a0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:22.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.078+0000 7f8415543700 1 -- 172.21.15.17:0/2605522362 >> 172.21.15.17:0/2605522362 conn(0x7f841005c620 msgr2=0x7f84100b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:22.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.078+0000 7f8415543700 1 -- 172.21.15.17:0/2605522362 shutdown_connections 2022-08-01T19:41:22.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.078+0000 7f8415543700 1 -- 172.21.15.17:0/2605522362 wait complete. 2022-08-01T19:41:22.101 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:41:22.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.210+0000 7f8d75f1d700 1 Processor -- start 2022-08-01T19:41:22.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.210+0000 7f8d75f1d700 1 -- start start 2022-08-01T19:41:22.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.210+0000 7f8d75f1d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8d700ad750 0x7f8d70061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:22.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.210+0000 7f8d75f1d700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d700aee80 con 0x7f8d700614b0 2022-08-01T19:41:22.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.210+0000 7f8d75f1d700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d700b70b0 con 0x7f8d701405d0 2022-08-01T19:41:22.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.210+0000 7f8d75f1d700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d700af0b0 con 0x7f8d700ad750 2022-08-01T19:41:22.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.210+0000 7f8d6ffff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8d700ad750 0x7f8d70061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:22.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.210+0000 7f8d6ffff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8d700ad750 0x7f8d70061890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41884/0 (socket says 172.21.15.17:41884) 2022-08-01T19:41:22.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.210+0000 7f8d6ffff700 1 -- 172.21.15.17:0/1066721827 learned_addr learned my addr 172.21.15.17:0/1066721827 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:22.234 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.214+0000 7f8d6ffff700 1 -- 172.21.15.17:0/1066721827 >> v1:172.21.15.77:6789/0 conn(0x7f8d701405d0 legacy=0x7f8d701466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:22.234 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.214+0000 7f8d6ffff700 1 -- 172.21.15.17:0/1066721827 >> v1:172.21.15.17:6789/0 conn(0x7f8d700614b0 legacy=0x7f8d700b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:22.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.214+0000 7f8d6ffff700 1 -- 172.21.15.17:0/1066721827 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d700af350 con 0x7f8d700ad750 2022-08-01T19:41:22.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.214+0000 7f8d6ffff700 1 --2- 172.21.15.17:0/1066721827 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d700ad750 0x7f8d70061890 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 rx=0x7f8d5c002f70 tx=0x7f8d5c006eb0).ready entity=mon.0 client_cookie=9134ff4297f33136 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:22.236 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.214+0000 7f8d6f7fe700 1 -- 172.21.15.17:0/1066721827 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d5c00d660 con 0x7f8d700ad750 2022-08-01T19:41:22.236 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.214+0000 7f8d6f7fe700 1 -- 172.21.15.17:0/1066721827 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d5c000e90 con 0x7f8d700ad750 2022-08-01T19:41:22.236 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.214+0000 7f8d6f7fe700 1 -- 172.21.15.17:0/1066721827 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d5c00d9f0 con 0x7f8d700ad750 2022-08-01T19:41:22.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.214+0000 7f8d75f1d700 1 -- 172.21.15.17:0/1066721827 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d700ad750 msgr2=0x7f8d70061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:22.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.214+0000 7f8d75f1d700 1 --2- 172.21.15.17:0/1066721827 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d700ad750 0x7f8d70061890 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 rx=0x7f8d5c002f70 tx=0x7f8d5c006eb0).stop 2022-08-01T19:41:22.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.214+0000 7f8d75f1d700 1 -- 172.21.15.17:0/1066721827 shutdown_connections 2022-08-01T19:41:22.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.214+0000 7f8d75f1d700 1 --2- 172.21.15.17:0/1066721827 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d700ad750 0x7f8d70061890 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:22.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.214+0000 7f8d75f1d700 1 -- 172.21.15.17:0/1066721827 >> 172.21.15.17:0/1066721827 conn(0x7f8d70066f70 msgr2=0x7f8d70067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:22.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.214+0000 7f8d75f1d700 1 -- 172.21.15.17:0/1066721827 shutdown_connections 2022-08-01T19:41:22.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.214+0000 7f8d75f1d700 1 -- 172.21.15.17:0/1066721827 wait complete. 2022-08-01T19:41:22.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.214+0000 7f8d75f1d700 1 Processor -- start 2022-08-01T19:41:22.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.214+0000 7f8d75f1d700 1 -- start start 2022-08-01T19:41:22.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.218+0000 7f8d75f1d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d700614b0 0x7f8d70144bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:22.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.218+0000 7f8d75f1d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d700ad750 0x7f8d70152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:22.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.218+0000 7f8d75f1d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8d701405d0 0x7f8d70153010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:22.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.218+0000 7f8d75f1d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d700aca60 con 0x7f8d700614b0 2022-08-01T19:41:22.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.218+0000 7f8d75f1d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d70145470 con 0x7f8d700ad750 2022-08-01T19:41:22.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.218+0000 7f8d75f1d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d701525c0 con 0x7f8d701405d0 2022-08-01T19:41:22.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.218+0000 7f8d74f1b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d700614b0 0x7f8d70144bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:22.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.218+0000 7f8d74f1b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d700614b0 0x7f8d70144bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41890/0 (socket says 172.21.15.17:41890) 2022-08-01T19:41:22.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.218+0000 7f8d74f1b700 1 -- 172.21.15.17:0/3567933450 learned_addr learned my addr 172.21.15.17:0/3567933450 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:22.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.218+0000 7f8d7571c700 1 --2- 172.21.15.17:0/3567933450 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8d701405d0 0x7f8d70153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:22.246 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.218+0000 7f8d6ffff700 1 --2- 172.21.15.17:0/3567933450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d700ad750 0x7f8d70152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:22.246 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.218+0000 7f8d74f1b700 1 -- 172.21.15.17:0/3567933450 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8d701405d0 msgr2=0x7f8d70153010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:22.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.218+0000 7f8d74f1b700 1 --2- 172.21.15.17:0/3567933450 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8d701405d0 0x7f8d70153010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:22.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.218+0000 7f8d74f1b700 1 -- 172.21.15.17:0/3567933450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d700ad750 msgr2=0x7f8d70152b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:22.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.218+0000 7f8d74f1b700 1 --2- 172.21.15.17:0/3567933450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d700ad750 0x7f8d70152b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:22.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.218+0000 7f8d74f1b700 1 -- 172.21.15.17:0/3567933450 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d5c00b1e0 con 0x7f8d700614b0 2022-08-01T19:41:22.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.218+0000 7f8d74f1b700 1 --2- 172.21.15.17:0/3567933450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d700614b0 0x7f8d70144bf0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 rx=0x7f8d700ae2b0 tx=0x7f8d64007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:22.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.218+0000 7f8d6dffb700 1 -- 172.21.15.17:0/3567933450 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d6400ce30 con 0x7f8d700614b0 2022-08-01T19:41:22.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.218+0000 7f8d6dffb700 1 -- 172.21.15.17:0/3567933450 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6400f470 con 0x7f8d700614b0 2022-08-01T19:41:22.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.218+0000 7f8d75f1d700 1 -- 172.21.15.17:0/3567933450 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d700accc0 con 0x7f8d700614b0 2022-08-01T19:41:22.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.218+0000 7f8d6dffb700 1 -- 172.21.15.17:0/3567933450 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d6400f7d0 con 0x7f8d700614b0 2022-08-01T19:41:22.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.218+0000 7f8d75f1d700 1 -- 172.21.15.17:0/3567933450 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d70153500 con 0x7f8d700614b0 2022-08-01T19:41:22.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.222+0000 7f8d75f1d700 1 -- 172.21.15.17:0/3567933450 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d700af5f0 con 0x7f8d700614b0 2022-08-01T19:41:22.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.222+0000 7f8d6dffb700 1 -- 172.21.15.17:0/3567933450 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8d64019030 con 0x7f8d700614b0 2022-08-01T19:41:22.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.222+0000 7f8d6dffb700 1 --2- 172.21.15.17:0/3567933450 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8d50053950 0x7f8d50055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:22.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.222+0000 7f8d6dffb700 1 -- 172.21.15.17:0/3567933450 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f8d6406b2a0 con 0x7f8d700614b0 2022-08-01T19:41:22.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.222+0000 7f8d6ffff700 1 --2- 172.21.15.17:0/3567933450 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8d50053950 0x7f8d50055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:22.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.222+0000 7f8d6ffff700 1 --2- 172.21.15.17:0/3567933450 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8d50053950 0x7f8d50055e10 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f8d5c01a000 tx=0x7f8d5c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:22.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.226+0000 7f8d6dffb700 1 -- 172.21.15.17:0/3567933450 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8d64042300 con 0x7f8d700614b0 2022-08-01T19:41:22.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.350+0000 7f8d75f1d700 1 -- 172.21.15.17:0/3567933450 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8d700574e0 con 0x7f8d700614b0 2022-08-01T19:41:22.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.350+0000 7f8d6dffb700 1 -- 172.21.15.17:0/3567933450 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v94) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8d64045920 con 0x7f8d700614b0 2022-08-01T19:41:22.365 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:41:22.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.354+0000 7f8d75f1d700 1 -- 172.21.15.17:0/3567933450 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8d50053950 msgr2=0x7f8d50055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:22.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.354+0000 7f8d75f1d700 1 --2- 172.21.15.17:0/3567933450 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8d50053950 0x7f8d50055e10 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f8d5c01a000 tx=0x7f8d5c015000).stop 2022-08-01T19:41:22.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.354+0000 7f8d75f1d700 1 -- 172.21.15.17:0/3567933450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d700614b0 msgr2=0x7f8d70144bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:22.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.354+0000 7f8d75f1d700 1 --2- 172.21.15.17:0/3567933450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d700614b0 0x7f8d70144bf0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 rx=0x7f8d700ae2b0 tx=0x7f8d64007c50).stop 2022-08-01T19:41:22.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.354+0000 7f8d75f1d700 1 -- 172.21.15.17:0/3567933450 shutdown_connections 2022-08-01T19:41:22.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.354+0000 7f8d75f1d700 1 --2- 172.21.15.17:0/3567933450 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8d50053950 0x7f8d50055e10 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:22.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.354+0000 7f8d75f1d700 1 --2- 172.21.15.17:0/3567933450 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8d701405d0 0x7f8d70153010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:22.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.354+0000 7f8d75f1d700 1 --2- 172.21.15.17:0/3567933450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d700ad750 0x7f8d70152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:22.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.354+0000 7f8d75f1d700 1 --2- 172.21.15.17:0/3567933450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d700614b0 0x7f8d70144bf0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:22.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.354+0000 7f8d75f1d700 1 -- 172.21.15.17:0/3567933450 >> 172.21.15.17:0/3567933450 conn(0x7f8d70066f70 msgr2=0x7f8d700b7780 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:22.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.354+0000 7f8d75f1d700 1 -- 172.21.15.17:0/3567933450 shutdown_connections 2022-08-01T19:41:22.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.354+0000 7f8d75f1d700 1 -- 172.21.15.17:0/3567933450 wait complete. 2022-08-01T19:41:22.374 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-08-01T19:41:22.499 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.474+0000 7f0431954700 1 Processor -- start 2022-08-01T19:41:22.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.474+0000 7f0431954700 1 -- start start 2022-08-01T19:41:22.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.478+0000 7f0431954700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f042c0b9870 0x7f042c0aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:22.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.478+0000 7f0431954700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f042c140180 0x7f042c140610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:22.501 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.478+0000 7f0431954700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f042c067690 con 0x7f042c0672b0 2022-08-01T19:41:22.501 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.478+0000 7f0431954700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f042c143210 con 0x7f042c140180 2022-08-01T19:41:22.501 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.478+0000 7f0431954700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f042c0af660 con 0x7f042c0b9870 2022-08-01T19:41:22.501 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.478+0000 7f042b7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f042c140180 0x7f042c140610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:22.502 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.478+0000 7f042b7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f042c140180 0x7f042c140610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41902/0 (socket says 172.21.15.17:41902) 2022-08-01T19:41:22.502 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.478+0000 7f042b7fe700 1 -- 172.21.15.17:0/3242874875 learned_addr learned my addr 172.21.15.17:0/3242874875 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:22.502 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.478+0000 7f042a7fc700 1 --2- 172.21.15.17:0/3242874875 >> v2:172.21.15.77:3300/0 conn(0x7f042c0b9870 0x7f042c0aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:22.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.478+0000 7f042b7fe700 1 -- 172.21.15.17:0/3242874875 >> v2:172.21.15.77:3300/0 conn(0x7f042c0b9870 msgr2=0x7f042c0aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:22.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.478+0000 7f042b7fe700 1 --2- 172.21.15.17:0/3242874875 >> v2:172.21.15.77:3300/0 conn(0x7f042c0b9870 0x7f042c0aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:22.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.478+0000 7f042b7fe700 1 -- 172.21.15.17:0/3242874875 >> v1:172.21.15.77:6789/0 conn(0x7f042c0672b0 legacy=0x7f042c143b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:22.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.478+0000 7f042b7fe700 1 -- 172.21.15.17:0/3242874875 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f042c143400 con 0x7f042c140180 2022-08-01T19:41:22.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.478+0000 7f042a7fc700 1 --2- 172.21.15.17:0/3242874875 >> v2:172.21.15.77:3300/0 conn(0x7f042c0b9870 0x7f042c0aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:22.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.478+0000 7f042b7fe700 1 --2- 172.21.15.17:0/3242874875 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f042c140180 0x7f042c140610 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7f0424002f70 tx=0x7f0424006eb0).ready entity=mon.0 client_cookie=3b818f127faa932f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:22.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.478+0000 7f0429ffb700 1 -- 172.21.15.17:0/3242874875 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f042400d610 con 0x7f042c140180 2022-08-01T19:41:22.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.478+0000 7f0429ffb700 1 -- 172.21.15.17:0/3242874875 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0424000e90 con 0x7f042c140180 2022-08-01T19:41:22.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.478+0000 7f0429ffb700 1 -- 172.21.15.17:0/3242874875 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f042400d9a0 con 0x7f042c140180 2022-08-01T19:41:22.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.478+0000 7f0431954700 1 -- 172.21.15.17:0/3242874875 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f042c140180 msgr2=0x7f042c140610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:22.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.478+0000 7f0431954700 1 --2- 172.21.15.17:0/3242874875 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f042c140180 0x7f042c140610 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7f0424002f70 tx=0x7f0424006eb0).stop 2022-08-01T19:41:22.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.478+0000 7f0431954700 1 -- 172.21.15.17:0/3242874875 shutdown_connections 2022-08-01T19:41:22.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.478+0000 7f0431954700 1 --2- 172.21.15.17:0/3242874875 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f042c140180 0x7f042c140610 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:22.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.478+0000 7f0431954700 1 --2- 172.21.15.17:0/3242874875 >> v2:172.21.15.77:3300/0 conn(0x7f042c0b9870 0x7f042c0aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:22.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.478+0000 7f0431954700 1 -- 172.21.15.17:0/3242874875 >> 172.21.15.17:0/3242874875 conn(0x7f042c05fb80 msgr2=0x7f042c061830 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:22.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.478+0000 7f0431954700 1 -- 172.21.15.17:0/3242874875 shutdown_connections 2022-08-01T19:41:22.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.478+0000 7f0431954700 1 -- 172.21.15.17:0/3242874875 wait complete. 2022-08-01T19:41:22.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.482+0000 7f0431954700 1 Processor -- start 2022-08-01T19:41:22.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.482+0000 7f0431954700 1 -- start start 2022-08-01T19:41:22.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.482+0000 7f0431954700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f042c0672b0 0x7f042c0ba070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:22.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.482+0000 7f0431954700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f042c0b9870 0x7f042c1529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:22.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.482+0000 7f0431954700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f042c140180 0x7f042c152ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:22.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.482+0000 7f0431954700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042c1533d0 con 0x7f042c0b9870 2022-08-01T19:41:22.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.482+0000 7f0431954700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042c144eb0 con 0x7f042c0672b0 2022-08-01T19:41:22.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.482+0000 7f0431954700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f042c145150 con 0x7f042c140180 2022-08-01T19:41:22.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.482+0000 7f042a7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f042c0b9870 0x7f042c1529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:22.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.482+0000 7f042a7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f042c0b9870 0x7f042c1529f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41906/0 (socket says 172.21.15.17:41906) 2022-08-01T19:41:22.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.482+0000 7f042a7fc700 1 -- 172.21.15.17:0/3115772344 learned_addr learned my addr 172.21.15.17:0/3115772344 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:22.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.482+0000 7f042affd700 1 --2- 172.21.15.17:0/3115772344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f042c0672b0 0x7f042c0ba070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:22.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.482+0000 7f042b7fe700 1 --2- 172.21.15.17:0/3115772344 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f042c140180 0x7f042c152ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:22.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.482+0000 7f042a7fc700 1 -- 172.21.15.17:0/3115772344 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f042c140180 msgr2=0x7f042c152ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:22.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.482+0000 7f042a7fc700 1 --2- 172.21.15.17:0/3115772344 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f042c140180 0x7f042c152ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:22.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.482+0000 7f042a7fc700 1 -- 172.21.15.17:0/3115772344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f042c0672b0 msgr2=0x7f042c0ba070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:22.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.482+0000 7f042a7fc700 1 --2- 172.21.15.17:0/3115772344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f042c0672b0 0x7f042c0ba070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:22.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.482+0000 7f042a7fc700 1 -- 172.21.15.17:0/3115772344 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f042400b1e0 con 0x7f042c0b9870 2022-08-01T19:41:22.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.482+0000 7f042affd700 1 --2- 172.21.15.17:0/3115772344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f042c0672b0 0x7f042c0ba070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:22.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.482+0000 7f042a7fc700 1 --2- 172.21.15.17:0/3115772344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f042c0b9870 0x7f042c1529f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0x7f0414006cc0 tx=0x7f0414008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:22.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.482+0000 7f0423fff700 1 -- 172.21.15.17:0/3115772344 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0414013620 con 0x7f042c0b9870 2022-08-01T19:41:22.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.482+0000 7f0423fff700 1 -- 172.21.15.17:0/3115772344 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0414013780 con 0x7f042c0b9870 2022-08-01T19:41:22.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.482+0000 7f0431954700 1 -- 172.21.15.17:0/3115772344 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f042c1453f0 con 0x7f042c0b9870 2022-08-01T19:41:22.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.482+0000 7f0423fff700 1 -- 172.21.15.17:0/3115772344 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0414013ae0 con 0x7f042c0b9870 2022-08-01T19:41:22.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.482+0000 7f0431954700 1 -- 172.21.15.17:0/3115772344 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f042c0b0540 con 0x7f042c0b9870 2022-08-01T19:41:22.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.486+0000 7f0431954700 1 -- 172.21.15.17:0/3115772344 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f042c1436a0 con 0x7f042c0b9870 2022-08-01T19:41:22.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.486+0000 7f0423fff700 1 -- 172.21.15.17:0/3115772344 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0414013c40 con 0x7f042c0b9870 2022-08-01T19:41:22.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.486+0000 7f0423fff700 1 --2- 172.21.15.17:0/3115772344 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0410053b00 0x7f0410055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:22.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.486+0000 7f0423fff700 1 -- 172.21.15.17:0/3115772344 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f041406bbc0 con 0x7f042c0b9870 2022-08-01T19:41:22.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.486+0000 7f042affd700 1 --2- 172.21.15.17:0/3115772344 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0410053b00 0x7f0410055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:22.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.486+0000 7f042affd700 1 --2- 172.21.15.17:0/3115772344 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0410053b00 0x7f0410055fc0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f041c00d000 tx=0x7f041c007b50).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:22.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.490+0000 7f0423fff700 1 -- 172.21.15.17:0/3115772344 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0414042c60 con 0x7f042c0b9870 2022-08-01T19:41:22.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.614+0000 7f0431954700 1 -- 172.21.15.17:0/3115772344 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f042c0574e0 con 0x7f042c0b9870 2022-08-01T19:41:22.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.614+0000 7f0423fff700 1 -- 172.21.15.17:0/3115772344 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v94) v1 ==== 84+0+143 (secure 0 0 0) 0x7f0414046280 con 0x7f042c0b9870 2022-08-01T19:41:22.626 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:22.627 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2022-08-01T19:41:22.627 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:22.627 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 1, 2022-08-01T19:41:22.628 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": "device_health_metrics" 2022-08-01T19:41:22.628 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:22.629 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:22.629 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 2, 2022-08-01T19:41:22.630 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": "rbd" 2022-08-01T19:41:22.630 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:22.630 INFO:teuthology.orchestra.run.smithi017.stdout:] 2022-08-01T19:41:22.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.618+0000 7f0431954700 1 -- 172.21.15.17:0/3115772344 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0410053b00 msgr2=0x7f0410055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:22.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.618+0000 7f0431954700 1 --2- 172.21.15.17:0/3115772344 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0410053b00 0x7f0410055fc0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f041c00d000 tx=0x7f041c007b50).stop 2022-08-01T19:41:22.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.618+0000 7f0431954700 1 -- 172.21.15.17:0/3115772344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f042c0b9870 msgr2=0x7f042c1529f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:22.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.618+0000 7f0431954700 1 --2- 172.21.15.17:0/3115772344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f042c0b9870 0x7f042c1529f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0x7f0414006cc0 tx=0x7f0414008040).stop 2022-08-01T19:41:22.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.618+0000 7f0431954700 1 -- 172.21.15.17:0/3115772344 shutdown_connections 2022-08-01T19:41:22.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.618+0000 7f0431954700 1 --2- 172.21.15.17:0/3115772344 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0410053b00 0x7f0410055fc0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:22.636 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.618+0000 7f0431954700 1 --2- 172.21.15.17:0/3115772344 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f042c140180 0x7f042c152ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:22.638 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.618+0000 7f0431954700 1 --2- 172.21.15.17:0/3115772344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f042c0b9870 0x7f042c1529f0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:22.638 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.618+0000 7f0431954700 1 --2- 172.21.15.17:0/3115772344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f042c0672b0 0x7f042c0ba070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:22.639 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.618+0000 7f0431954700 1 -- 172.21.15.17:0/3115772344 >> 172.21.15.17:0/3115772344 conn(0x7f042c05fb80 msgr2=0x7f042c0b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:22.639 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.618+0000 7f0431954700 1 -- 172.21.15.17:0/3115772344 shutdown_connections 2022-08-01T19:41:22.639 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.618+0000 7f0431954700 1 -- 172.21.15.17:0/3115772344 wait complete. 2022-08-01T19:41:22.641 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-08-01T19:41:22.642 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-08-01T19:41:22.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.746+0000 7f9d16c54700 1 Processor -- start 2022-08-01T19:41:22.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.746+0000 7f9d16c54700 1 -- start start 2022-08-01T19:41:22.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.746+0000 7f9d16c54700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9d10064b10 0x7f9d10144e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:22.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.746+0000 7f9d16c54700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9d100b36e0 0x7f9d10141370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:22.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.746+0000 7f9d16c54700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d10064ef0 con 0x7f9d10141860 2022-08-01T19:41:22.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.746+0000 7f9d16c54700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d1013ebf0 con 0x7f9d100b36e0 2022-08-01T19:41:22.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.746+0000 7f9d16c54700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d10066690 con 0x7f9d10064b10 2022-08-01T19:41:22.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.746+0000 7f9d15451700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9d100b36e0 0x7f9d10141370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:22.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.746+0000 7f9d16453700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9d10141860 0x7f9d10066ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34108/0 (socket says 172.21.15.17:34108) 2022-08-01T19:41:22.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.746+0000 7f9d16453700 1 -- 172.21.15.17:0/892164195 learned_addr learned my addr 172.21.15.17:0/892164195 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:22.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.746+0000 7f9d15c52700 1 --2- 172.21.15.17:0/892164195 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9d10064b10 0x7f9d10144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:22.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.746+0000 7f9d14c50700 1 -- 172.21.15.17:0/892164195 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1041418244 0 0) 0x7f9d10064ef0 con 0x7f9d10141860 2022-08-01T19:41:22.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.746+0000 7f9d15451700 1 -- 172.21.15.17:0/892164195 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9d10064b10 msgr2=0x7f9d10144e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:22.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.746+0000 7f9d15451700 1 --2- 172.21.15.17:0/892164195 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9d10064b10 0x7f9d10144e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:22.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.746+0000 7f9d15451700 1 -- 172.21.15.17:0/892164195 >> v1:172.21.15.17:6789/0 conn(0x7f9d10141860 legacy=0x7f9d10066ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:22.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.746+0000 7f9d15451700 1 -- 172.21.15.17:0/892164195 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d100b4850 con 0x7f9d100b36e0 2022-08-01T19:41:22.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.750+0000 7f9d15451700 1 --2- 172.21.15.17:0/892164195 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d100b36e0 0x7f9d10141370 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 rx=0x7f9d0c002f70 tx=0x7f9d0c006df0).ready entity=mon.0 client_cookie=d8414ff36ece2533 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:22.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.750+0000 7f9d14c50700 1 -- 172.21.15.17:0/892164195 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d0c00d570 con 0x7f9d100b36e0 2022-08-01T19:41:22.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.750+0000 7f9d14c50700 1 -- 172.21.15.17:0/892164195 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d0c000e90 con 0x7f9d100b36e0 2022-08-01T19:41:22.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.750+0000 7f9d14c50700 1 -- 172.21.15.17:0/892164195 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d0c00d8c0 con 0x7f9d100b36e0 2022-08-01T19:41:22.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.750+0000 7f9d16c54700 1 -- 172.21.15.17:0/892164195 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d100b36e0 msgr2=0x7f9d10141370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:22.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.750+0000 7f9d16c54700 1 --2- 172.21.15.17:0/892164195 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d100b36e0 0x7f9d10141370 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 rx=0x7f9d0c002f70 tx=0x7f9d0c006df0).stop 2022-08-01T19:41:22.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.750+0000 7f9d16c54700 1 -- 172.21.15.17:0/892164195 shutdown_connections 2022-08-01T19:41:22.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.750+0000 7f9d16c54700 1 --2- 172.21.15.17:0/892164195 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d100b36e0 0x7f9d10141370 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:22.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.750+0000 7f9d16c54700 1 --2- 172.21.15.17:0/892164195 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9d10064b10 0x7f9d10144e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:22.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.750+0000 7f9d16c54700 1 -- 172.21.15.17:0/892164195 >> 172.21.15.17:0/892164195 conn(0x7f9d1005f840 msgr2=0x7f9d1005fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:22.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.750+0000 7f9d16c54700 1 -- 172.21.15.17:0/892164195 shutdown_connections 2022-08-01T19:41:22.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.750+0000 7f9d16c54700 1 -- 172.21.15.17:0/892164195 wait complete. 2022-08-01T19:41:22.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.750+0000 7f9d16c54700 1 Processor -- start 2022-08-01T19:41:22.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.750+0000 7f9d16c54700 1 -- start start 2022-08-01T19:41:22.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.750+0000 7f9d16c54700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9d10064b10 0x7f9d10060c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:22.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.750+0000 7f9d16c54700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d100b36e0 0x7f9d10155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:22.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.750+0000 7f9d16c54700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d10141860 0x7f9d10155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:22.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.750+0000 7f9d16c54700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d10142f70 con 0x7f9d10141860 2022-08-01T19:41:22.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.750+0000 7f9d16c54700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d100adff0 con 0x7f9d100b36e0 2022-08-01T19:41:22.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.750+0000 7f9d16c54700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d100ae220 con 0x7f9d10064b10 2022-08-01T19:41:22.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.750+0000 7f9d16453700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d10141860 0x7f9d10155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:22.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.754+0000 7f9d16453700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d10141860 0x7f9d10155f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41920/0 (socket says 172.21.15.17:41920) 2022-08-01T19:41:22.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.754+0000 7f9d16453700 1 -- 172.21.15.17:0/934956281 learned_addr learned my addr 172.21.15.17:0/934956281 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:22.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.754+0000 7f9d15c52700 1 --2- 172.21.15.17:0/934956281 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9d10064b10 0x7f9d10060c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:22.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.754+0000 7f9d15451700 1 --2- 172.21.15.17:0/934956281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d100b36e0 0x7f9d10155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:22.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.754+0000 7f9d16453700 1 -- 172.21.15.17:0/934956281 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9d10064b10 msgr2=0x7f9d10060c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:22.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.754+0000 7f9d16453700 1 --2- 172.21.15.17:0/934956281 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9d10064b10 0x7f9d10060c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:22.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.754+0000 7f9d16453700 1 -- 172.21.15.17:0/934956281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d100b36e0 msgr2=0x7f9d10155a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:22.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.754+0000 7f9d16453700 1 --2- 172.21.15.17:0/934956281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d100b36e0 0x7f9d10155a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:22.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.754+0000 7f9d16453700 1 -- 172.21.15.17:0/934956281 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0c00b180 con 0x7f9d10141860 2022-08-01T19:41:22.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.754+0000 7f9d16453700 1 --2- 172.21.15.17:0/934956281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d10141860 0x7f9d10155f20 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0x7f9d101445c0 tx=0x7f9d08007d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:22.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.754+0000 7f9cfeffd700 1 -- 172.21.15.17:0/934956281 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d08005b80 con 0x7f9d10141860 2022-08-01T19:41:22.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.754+0000 7f9cfeffd700 1 -- 172.21.15.17:0/934956281 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d08005ce0 con 0x7f9d10141860 2022-08-01T19:41:22.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.754+0000 7f9d16c54700 1 -- 172.21.15.17:0/934956281 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d100ad0d0 con 0x7f9d10141860 2022-08-01T19:41:22.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.754+0000 7f9cfeffd700 1 -- 172.21.15.17:0/934956281 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d080072c0 con 0x7f9d10141860 2022-08-01T19:41:22.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.754+0000 7f9d16c54700 1 -- 172.21.15.17:0/934956281 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d100acc30 con 0x7f9d10141860 2022-08-01T19:41:22.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.754+0000 7f9d16c54700 1 -- 172.21.15.17:0/934956281 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d100ad260 con 0x7f9d10141860 2022-08-01T19:41:22.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.754+0000 7f9cfeffd700 1 -- 172.21.15.17:0/934956281 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9d0801a060 con 0x7f9d10141860 2022-08-01T19:41:22.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.754+0000 7f9cfeffd700 1 --2- 172.21.15.17:0/934956281 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9cf0053950 0x7f9cf0055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:22.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.754+0000 7f9cfeffd700 1 -- 172.21.15.17:0/934956281 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f9d0806b7c0 con 0x7f9d10141860 2022-08-01T19:41:22.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.758+0000 7f9d15c52700 1 --2- 172.21.15.17:0/934956281 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9cf0053950 0x7f9cf0055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:22.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.758+0000 7f9d15c52700 1 --2- 172.21.15.17:0/934956281 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9cf0053950 0x7f9cf0055e10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f9d0000b000 tx=0x7f9d00006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:22.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.766+0000 7f9cfeffd700 1 -- 172.21.15.17:0/934956281 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9d08042840 con 0x7f9d10141860 2022-08-01T19:41:22.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:22.890+0000 7f9d16c54700 1 -- 172.21.15.17:0/934956281 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f9d100b4af0 con 0x7f9d10141860 2022-08-01T19:41:23.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.586+0000 7f9cfeffd700 1 -- 172.21.15.17:0/934956281 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v95) v1 ==== 133+0+0 (secure 0 0 0) 0x7f9d08018020 con 0x7f9d10141860 2022-08-01T19:41:23.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.590+0000 7f9d16c54700 1 -- 172.21.15.17:0/934956281 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9cf0053950 msgr2=0x7f9cf0055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:23.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.590+0000 7f9d16c54700 1 --2- 172.21.15.17:0/934956281 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9cf0053950 0x7f9cf0055e10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f9d0000b000 tx=0x7f9d00006df0).stop 2022-08-01T19:41:23.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.590+0000 7f9d16c54700 1 -- 172.21.15.17:0/934956281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d10141860 msgr2=0x7f9d10155f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:23.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.590+0000 7f9d16c54700 1 --2- 172.21.15.17:0/934956281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d10141860 0x7f9d10155f20 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0x7f9d101445c0 tx=0x7f9d08007d60).stop 2022-08-01T19:41:23.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.590+0000 7f9d16c54700 1 -- 172.21.15.17:0/934956281 shutdown_connections 2022-08-01T19:41:23.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.590+0000 7f9d16c54700 1 --2- 172.21.15.17:0/934956281 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9cf0053950 0x7f9cf0055e10 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:23.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.590+0000 7f9d16c54700 1 --2- 172.21.15.17:0/934956281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d10141860 0x7f9d10155f20 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:23.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.590+0000 7f9d16c54700 1 --2- 172.21.15.17:0/934956281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d100b36e0 0x7f9d10155a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:23.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.590+0000 7f9d16c54700 1 --2- 172.21.15.17:0/934956281 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9d10064b10 0x7f9d10060c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:23.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.590+0000 7f9d16c54700 1 -- 172.21.15.17:0/934956281 >> 172.21.15.17:0/934956281 conn(0x7f9d1005f840 msgr2=0x7f9d100b74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:23.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.590+0000 7f9d16c54700 1 -- 172.21.15.17:0/934956281 shutdown_connections 2022-08-01T19:41:23.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.590+0000 7f9d16c54700 1 -- 172.21.15.17:0/934956281 wait complete. 2022-08-01T19:41:23.601 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' created 2022-08-01T19:41:23.616 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-08-01T19:41:23.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.730+0000 7fd2af543700 1 Processor -- start 2022-08-01T19:41:23.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.730+0000 7fd2af543700 1 -- start start 2022-08-01T19:41:23.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.734+0000 7fd2af543700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fd2a80600f0 0x7fd2a81405d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:23.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.734+0000 7fd2af543700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd2a8061360 0x7fd2a8140ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:23.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.734+0000 7fd2af543700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2a8140fb0 con 0x7fd2a80b28a0 2022-08-01T19:41:23.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.734+0000 7fd2af543700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2a8064aa0 con 0x7fd2a8061360 2022-08-01T19:41:23.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.734+0000 7fd2af543700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2a8064d40 con 0x7fd2a80600f0 2022-08-01T19:41:23.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.734+0000 7fd2adae0700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd2a8061360 0x7fd2a8140ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:23.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.734+0000 7fd2adae0700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd2a8061360 0x7fd2a8140ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41928/0 (socket says 172.21.15.17:41928) 2022-08-01T19:41:23.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.734+0000 7fd2adae0700 1 -- 172.21.15.17:0/1729541365 learned_addr learned my addr 172.21.15.17:0/1729541365 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:23.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.734+0000 7fd2ad2df700 1 --2- 172.21.15.17:0/1729541365 >> v2:172.21.15.77:3300/0 conn(0x7fd2a80600f0 0x7fd2a81405d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:23.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.734+0000 7fd2adae0700 1 -- 172.21.15.17:0/1729541365 >> v2:172.21.15.77:3300/0 conn(0x7fd2a80600f0 msgr2=0x7fd2a81405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:23.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.734+0000 7fd2adae0700 1 --2- 172.21.15.17:0/1729541365 >> v2:172.21.15.77:3300/0 conn(0x7fd2a80600f0 0x7fd2a81405d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:23.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.734+0000 7fd2adae0700 1 -- 172.21.15.17:0/1729541365 >> v1:172.21.15.77:6789/0 conn(0x7fd2a80b28a0 legacy=0x7fd2a81432a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:23.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.734+0000 7fd2adae0700 1 -- 172.21.15.17:0/1729541365 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a8064fe0 con 0x7fd2a8061360 2022-08-01T19:41:23.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.734+0000 7fd2ad2df700 1 --2- 172.21.15.17:0/1729541365 >> v2:172.21.15.77:3300/0 conn(0x7fd2a80600f0 0x7fd2a81405d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:23.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.734+0000 7fd2adae0700 1 --2- 172.21.15.17:0/1729541365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2a8061360 0x7fd2a8140ac0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 rx=0x7fd2a00032f0 tx=0x7fd2a0009290).ready entity=mon.0 client_cookie=fdf6a4a39f8f5a0c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:23.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.734+0000 7fd297fff700 1 -- 172.21.15.17:0/1729541365 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2a000b870 con 0x7fd2a8061360 2022-08-01T19:41:23.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.734+0000 7fd297fff700 1 -- 172.21.15.17:0/1729541365 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a000b9d0 con 0x7fd2a8061360 2022-08-01T19:41:23.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.734+0000 7fd297fff700 1 -- 172.21.15.17:0/1729541365 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2a000bb40 con 0x7fd2a8061360 2022-08-01T19:41:23.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.734+0000 7fd2af543700 1 -- 172.21.15.17:0/1729541365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2a8061360 msgr2=0x7fd2a8140ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:23.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.734+0000 7fd2af543700 1 --2- 172.21.15.17:0/1729541365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2a8061360 0x7fd2a8140ac0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 rx=0x7fd2a00032f0 tx=0x7fd2a0009290).stop 2022-08-01T19:41:23.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.734+0000 7fd2af543700 1 -- 172.21.15.17:0/1729541365 shutdown_connections 2022-08-01T19:41:23.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.734+0000 7fd2af543700 1 --2- 172.21.15.17:0/1729541365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2a8061360 0x7fd2a8140ac0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:23.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.734+0000 7fd2af543700 1 --2- 172.21.15.17:0/1729541365 >> v2:172.21.15.77:3300/0 conn(0x7fd2a80600f0 0x7fd2a81405d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:23.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.734+0000 7fd2af543700 1 -- 172.21.15.17:0/1729541365 >> 172.21.15.17:0/1729541365 conn(0x7fd2a805d0f0 msgr2=0x7fd2a805f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:23.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.734+0000 7fd2af543700 1 -- 172.21.15.17:0/1729541365 shutdown_connections 2022-08-01T19:41:23.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.734+0000 7fd2af543700 1 -- 172.21.15.17:0/1729541365 wait complete. 2022-08-01T19:41:23.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.738+0000 7fd2af543700 1 Processor -- start 2022-08-01T19:41:23.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.738+0000 7fd2af543700 1 -- start start 2022-08-01T19:41:23.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.738+0000 7fd2af543700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2a80600f0 0x7fd2a81460a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:23.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.738+0000 7fd2af543700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2a8061360 0x7fd2a8147eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:23.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.738+0000 7fd2af543700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2a80b28a0 0x7fd2a8152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:23.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.738+0000 7fd2af543700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a80b5410 con 0x7fd2a8061360 2022-08-01T19:41:23.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.738+0000 7fd2af543700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a80b56b0 con 0x7fd2a80600f0 2022-08-01T19:41:23.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.738+0000 7fd2af543700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a80b5950 con 0x7fd2a80b28a0 2022-08-01T19:41:23.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.738+0000 7fd2acade700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2a8061360 0x7fd2a8147eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:23.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.738+0000 7fd2acade700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2a8061360 0x7fd2a8147eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41934/0 (socket says 172.21.15.17:41934) 2022-08-01T19:41:23.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.738+0000 7fd2acade700 1 -- 172.21.15.17:0/1657831721 learned_addr learned my addr 172.21.15.17:0/1657831721 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:23.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.742+0000 7fd2ad2df700 1 --2- 172.21.15.17:0/1657831721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2a80600f0 0x7fd2a81460a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:23.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.742+0000 7fd2adae0700 1 --2- 172.21.15.17:0/1657831721 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2a80b28a0 0x7fd2a8152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:23.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.742+0000 7fd2acade700 1 -- 172.21.15.17:0/1657831721 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2a80b28a0 msgr2=0x7fd2a8152850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:23.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.742+0000 7fd2acade700 1 --2- 172.21.15.17:0/1657831721 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2a80b28a0 0x7fd2a8152850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:23.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.742+0000 7fd2acade700 1 -- 172.21.15.17:0/1657831721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2a80600f0 msgr2=0x7fd2a81460a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:23.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.742+0000 7fd2acade700 1 --2- 172.21.15.17:0/1657831721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2a80600f0 0x7fd2a81460a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:23.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.742+0000 7fd2acade700 1 -- 172.21.15.17:0/1657831721 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a0009530 con 0x7fd2a8061360 2022-08-01T19:41:23.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.742+0000 7fd2ad2df700 1 --2- 172.21.15.17:0/1657831721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2a80600f0 0x7fd2a81460a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:23.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.742+0000 7fd2acade700 1 --2- 172.21.15.17:0/1657831721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2a8061360 0x7fd2a8147eb0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7fd2a8062940 tx=0x7fd298007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:23.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.742+0000 7fd2967fc700 1 -- 172.21.15.17:0/1657831721 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd298007e40 con 0x7fd2a8061360 2022-08-01T19:41:23.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.742+0000 7fd2967fc700 1 -- 172.21.15.17:0/1657831721 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd298007030 con 0x7fd2a8061360 2022-08-01T19:41:23.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.742+0000 7fd2af543700 1 -- 172.21.15.17:0/1657831721 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a80b7200 con 0x7fd2a8061360 2022-08-01T19:41:23.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.742+0000 7fd2967fc700 1 -- 172.21.15.17:0/1657831721 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd298007390 con 0x7fd2a8061360 2022-08-01T19:41:23.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.742+0000 7fd2af543700 1 -- 172.21.15.17:0/1657831721 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a80b60c0 con 0x7fd2a8061360 2022-08-01T19:41:23.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.742+0000 7fd2af543700 1 -- 172.21.15.17:0/1657831721 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2a8065280 con 0x7fd2a8061360 2022-08-01T19:41:23.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.746+0000 7fd2967fc700 1 -- 172.21.15.17:0/1657831721 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd298010030 con 0x7fd2a8061360 2022-08-01T19:41:23.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.746+0000 7fd2967fc700 1 --2- 172.21.15.17:0/1657831721 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd288053a70 0x7fd288055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:23.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.746+0000 7fd2967fc700 1 -- 172.21.15.17:0/1657831721 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5954+0+0 (secure 0 0 0) 0x7fd29806bf40 con 0x7fd2a8061360 2022-08-01T19:41:23.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.746+0000 7fd2ad2df700 1 --2- 172.21.15.17:0/1657831721 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd288053a70 0x7fd288055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:23.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.746+0000 7fd2ad2df700 1 --2- 172.21.15.17:0/1657831721 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd288053a70 0x7fd288055f30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7fd2a400c000 tx=0x7fd2a4008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:23.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.754+0000 7fd2967fc700 1 -- 172.21.15.17:0/1657831721 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd298042ed0 con 0x7fd2a8061360 2022-08-01T19:41:23.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:23.882+0000 7fd2af543700 1 -- 172.21.15.17:0/1657831721 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fd2a80574e0 con 0x7fd2a8061360 2022-08-01T19:41:24.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.150+0000 7fd2967fc700 1 -- 172.21.15.17:0/1657831721 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v96) v1 ==== 125+0+0 (secure 0 0 0) 0x7fd2980464f0 con 0x7fd2a8061360 2022-08-01T19:41:24.162 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.162+0000 7fd2af543700 1 -- 172.21.15.17:0/1657831721 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd288053a70 msgr2=0x7fd288055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:24.163 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.162+0000 7fd2af543700 1 --2- 172.21.15.17:0/1657831721 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd288053a70 0x7fd288055f30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7fd2a400c000 tx=0x7fd2a4008040).stop 2022-08-01T19:41:24.163 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.162+0000 7fd2af543700 1 -- 172.21.15.17:0/1657831721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2a8061360 msgr2=0x7fd2a8147eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:24.163 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.162+0000 7fd2af543700 1 --2- 172.21.15.17:0/1657831721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2a8061360 0x7fd2a8147eb0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7fd2a8062940 tx=0x7fd298007bd0).stop 2022-08-01T19:41:24.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.162+0000 7fd2af543700 1 -- 172.21.15.17:0/1657831721 shutdown_connections 2022-08-01T19:41:24.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.162+0000 7fd2af543700 1 --2- 172.21.15.17:0/1657831721 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd288053a70 0x7fd288055f30 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:24.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.162+0000 7fd2af543700 1 --2- 172.21.15.17:0/1657831721 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2a80b28a0 0x7fd2a8152850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:24.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.162+0000 7fd2af543700 1 --2- 172.21.15.17:0/1657831721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2a8061360 0x7fd2a8147eb0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:24.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.162+0000 7fd2af543700 1 --2- 172.21.15.17:0/1657831721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2a80600f0 0x7fd2a81460a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:24.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.162+0000 7fd2af543700 1 -- 172.21.15.17:0/1657831721 >> 172.21.15.17:0/1657831721 conn(0x7fd2a805d0f0 msgr2=0x7fd2a80b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:24.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.162+0000 7fd2af543700 1 -- 172.21.15.17:0/1657831721 shutdown_connections 2022-08-01T19:41:24.169 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.162+0000 7fd2af543700 1 -- 172.21.15.17:0/1657831721 wait complete. 2022-08-01T19:41:24.169 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' created 2022-08-01T19:41:24.181 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-08-01T19:41:24.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.290+0000 7f8daa1e2700 1 Processor -- start 2022-08-01T19:41:24.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.290+0000 7f8daa1e2700 1 -- start start 2022-08-01T19:41:24.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.294+0000 7f8daa1e2700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8da40614b0 0x7f8da4061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:24.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.294+0000 7f8daa1e2700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8da40ad750 0x7f8da40b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:24.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.294+0000 7f8daa1e2700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8da40aee80 con 0x7f8da41405d0 2022-08-01T19:41:24.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.294+0000 7f8daa1e2700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8da40b70b0 con 0x7f8da40ad750 2022-08-01T19:41:24.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.294+0000 7f8daa1e2700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8da40af0b0 con 0x7f8da40614b0 2022-08-01T19:41:24.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.294+0000 7f8da89df700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8da40ad750 0x7f8da40b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:24.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.294+0000 7f8da89df700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8da40ad750 0x7f8da40b9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41940/0 (socket says 172.21.15.17:41940) 2022-08-01T19:41:24.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.294+0000 7f8da89df700 1 -- 172.21.15.17:0/2866045313 learned_addr learned my addr 172.21.15.17:0/2866045313 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:24.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.294+0000 7f8da91e0700 1 --2- 172.21.15.17:0/2866045313 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8da40614b0 0x7f8da4061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:24.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.294+0000 7f8da89df700 1 -- 172.21.15.17:0/2866045313 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8da40614b0 msgr2=0x7f8da4061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:24.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.294+0000 7f8da89df700 1 --2- 172.21.15.17:0/2866045313 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8da40614b0 0x7f8da4061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:24.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.294+0000 7f8da89df700 1 -- 172.21.15.17:0/2866045313 >> v1:172.21.15.17:6789/0 conn(0x7f8da41405d0 legacy=0x7f8da41466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:24.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.294+0000 7f8da89df700 1 -- 172.21.15.17:0/2866045313 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da40af350 con 0x7f8da40ad750 2022-08-01T19:41:24.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.294+0000 7f8da89df700 1 --2- 172.21.15.17:0/2866045313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8da40ad750 0x7f8da40b9ff0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0x7f8d9c001490 tx=0x7f8da0006eb0).ready entity=mon.0 client_cookie=27086fae99973f47 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:24.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.294+0000 7f8d97fff700 1 -- 172.21.15.17:0/2866045313 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8da000d570 con 0x7f8da40ad750 2022-08-01T19:41:24.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.294+0000 7f8d97fff700 1 -- 172.21.15.17:0/2866045313 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da0000e90 con 0x7f8da40ad750 2022-08-01T19:41:24.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.294+0000 7f8d97fff700 1 -- 172.21.15.17:0/2866045313 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8da000d900 con 0x7f8da40ad750 2022-08-01T19:41:24.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.294+0000 7f8daa1e2700 1 -- 172.21.15.17:0/2866045313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8da40ad750 msgr2=0x7f8da40b9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:24.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.294+0000 7f8daa1e2700 1 --2- 172.21.15.17:0/2866045313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8da40ad750 0x7f8da40b9ff0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0x7f8d9c001490 tx=0x7f8da0006eb0).stop 2022-08-01T19:41:24.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.294+0000 7f8daa1e2700 1 -- 172.21.15.17:0/2866045313 shutdown_connections 2022-08-01T19:41:24.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.294+0000 7f8daa1e2700 1 --2- 172.21.15.17:0/2866045313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8da40ad750 0x7f8da40b9ff0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:24.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.294+0000 7f8daa1e2700 1 --2- 172.21.15.17:0/2866045313 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8da40614b0 0x7f8da4061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:24.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.294+0000 7f8daa1e2700 1 -- 172.21.15.17:0/2866045313 >> 172.21.15.17:0/2866045313 conn(0x7f8da4066f70 msgr2=0x7f8da4067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:24.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.294+0000 7f8daa1e2700 1 -- 172.21.15.17:0/2866045313 shutdown_connections 2022-08-01T19:41:24.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.294+0000 7f8daa1e2700 1 -- 172.21.15.17:0/2866045313 wait complete. 2022-08-01T19:41:24.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.298+0000 7f8daa1e2700 1 Processor -- start 2022-08-01T19:41:24.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.298+0000 7f8daa1e2700 1 -- start start 2022-08-01T19:41:24.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.298+0000 7f8daa1e2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8da40614b0 0x7f8da40b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:24.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.298+0000 7f8daa1e2700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8da40ad750 0x7f8da4155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:24.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.298+0000 7f8daa1e2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8da41405d0 0x7f8da413f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:24.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.298+0000 7f8daa1e2700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da40b2490 con 0x7f8da40614b0 2022-08-01T19:41:24.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.298+0000 7f8daa1e2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da40b2710 con 0x7f8da41405d0 2022-08-01T19:41:24.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.298+0000 7f8daa1e2700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8da40b29b0 con 0x7f8da40ad750 2022-08-01T19:41:24.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.298+0000 7f8da91e0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8da40614b0 0x7f8da40b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:24.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.298+0000 7f8da91e0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8da40614b0 0x7f8da40b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41946/0 (socket says 172.21.15.17:41946) 2022-08-01T19:41:24.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.298+0000 7f8da91e0700 1 -- 172.21.15.17:0/3165604472 learned_addr learned my addr 172.21.15.17:0/3165604472 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:24.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.298+0000 7f8da91e0700 1 -- 172.21.15.17:0/3165604472 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8da40ad750 msgr2=0x7f8da4155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:24.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.298+0000 7f8da89df700 1 --2- 172.21.15.17:0/3165604472 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8da40ad750 0x7f8da4155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:24.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.298+0000 7f8da99e1700 1 --2- 172.21.15.17:0/3165604472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8da41405d0 0x7f8da413f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:24.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.298+0000 7f8da91e0700 1 --2- 172.21.15.17:0/3165604472 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8da40ad750 0x7f8da4155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:24.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.298+0000 7f8da91e0700 1 -- 172.21.15.17:0/3165604472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8da41405d0 msgr2=0x7f8da413f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:24.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.298+0000 7f8da91e0700 1 --2- 172.21.15.17:0/3165604472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8da41405d0 0x7f8da413f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:24.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.298+0000 7f8da91e0700 1 -- 172.21.15.17:0/3165604472 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da000b1e0 con 0x7f8da40614b0 2022-08-01T19:41:24.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.298+0000 7f8da99e1700 1 --2- 172.21.15.17:0/3165604472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8da41405d0 0x7f8da413f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:41:24.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.298+0000 7f8da89df700 1 --2- 172.21.15.17:0/3165604472 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8da40ad750 0x7f8da4155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:41:24.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.298+0000 7f8da91e0700 1 --2- 172.21.15.17:0/3165604472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8da40614b0 0x7f8da40b00c0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 rx=0x7f8d980030b0 tx=0x7f8d98008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:24.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.298+0000 7f8d967fc700 1 -- 172.21.15.17:0/3165604472 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d98011780 con 0x7f8da40614b0 2022-08-01T19:41:24.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.298+0000 7f8d967fc700 1 -- 172.21.15.17:0/3165604472 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d980118e0 con 0x7f8da40614b0 2022-08-01T19:41:24.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.298+0000 7f8daa1e2700 1 -- 172.21.15.17:0/3165604472 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da40b0d20 con 0x7f8da40614b0 2022-08-01T19:41:24.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.298+0000 7f8d967fc700 1 -- 172.21.15.17:0/3165604472 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d98011c40 con 0x7f8da40614b0 2022-08-01T19:41:24.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.298+0000 7f8daa1e2700 1 -- 172.21.15.17:0/3165604472 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da4146050 con 0x7f8da40614b0 2022-08-01T19:41:24.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.302+0000 7f8daa1e2700 1 -- 172.21.15.17:0/3165604472 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da40b0eb0 con 0x7f8da40614b0 2022-08-01T19:41:24.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.302+0000 7f8d967fc700 1 -- 172.21.15.17:0/3165604472 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8d9801c460 con 0x7f8da40614b0 2022-08-01T19:41:24.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.302+0000 7f8d967fc700 1 --2- 172.21.15.17:0/3165604472 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8d8c053950 0x7f8d8c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:24.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.302+0000 7f8d967fc700 1 -- 172.21.15.17:0/3165604472 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f8d9806c230 con 0x7f8da40614b0 2022-08-01T19:41:24.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.302+0000 7f8da89df700 1 --2- 172.21.15.17:0/3165604472 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8d8c053950 0x7f8d8c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:24.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.302+0000 7f8da89df700 1 --2- 172.21.15.17:0/3165604472 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8d8c053950 0x7f8d8c055e10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f8da0017000 tx=0x7f8da0015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:24.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.306+0000 7f8d967fc700 1 -- 172.21.15.17:0/3165604472 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8d98043010 con 0x7f8da40614b0 2022-08-01T19:41:24.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.474+0000 7f8daa1e2700 1 -- 172.21.15.17:0/3165604472 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f8da40b0750 con 0x7f8da40614b0 2022-08-01T19:41:24.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.482+0000 7f8d967fc700 1 -- 172.21.15.17:0/3165604472 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 17 and data pool 18 v44) v1 ==== 189+0+0 (secure 0 0 0) 0x7f8d98046630 con 0x7f8da40614b0 2022-08-01T19:41:24.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.490+0000 7f8daa1e2700 1 -- 172.21.15.17:0/3165604472 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8d8c053950 msgr2=0x7f8d8c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:24.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.490+0000 7f8daa1e2700 1 --2- 172.21.15.17:0/3165604472 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8d8c053950 0x7f8d8c055e10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f8da0017000 tx=0x7f8da0015000).stop 2022-08-01T19:41:24.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.490+0000 7f8daa1e2700 1 -- 172.21.15.17:0/3165604472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8da40614b0 msgr2=0x7f8da40b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:24.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.490+0000 7f8daa1e2700 1 --2- 172.21.15.17:0/3165604472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8da40614b0 0x7f8da40b00c0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 rx=0x7f8d980030b0 tx=0x7f8d98008040).stop 2022-08-01T19:41:24.501 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.494+0000 7f8daa1e2700 1 -- 172.21.15.17:0/3165604472 shutdown_connections 2022-08-01T19:41:24.501 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.494+0000 7f8daa1e2700 1 --2- 172.21.15.17:0/3165604472 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8d8c053950 0x7f8d8c055e10 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:24.502 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.494+0000 7f8daa1e2700 1 --2- 172.21.15.17:0/3165604472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8da41405d0 0x7f8da413f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:24.502 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.494+0000 7f8daa1e2700 1 --2- 172.21.15.17:0/3165604472 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8da40ad750 0x7f8da4155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:24.502 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.494+0000 7f8daa1e2700 1 --2- 172.21.15.17:0/3165604472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8da40614b0 0x7f8da40b00c0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:24.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.494+0000 7f8daa1e2700 1 -- 172.21.15.17:0/3165604472 >> 172.21.15.17:0/3165604472 conn(0x7f8da4066f70 msgr2=0x7f8da40b8750 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:24.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.494+0000 7f8daa1e2700 1 -- 172.21.15.17:0/3165604472 shutdown_connections 2022-08-01T19:41:24.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.494+0000 7f8daa1e2700 1 -- 172.21.15.17:0/3165604472 wait complete. 2022-08-01T19:41:24.503 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 17 and data pool 18 2022-08-01T19:41:24.511 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:41:24.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.606+0000 7fefd756c700 1 Processor -- start 2022-08-01T19:41:24.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.606+0000 7fefd756c700 1 -- start start 2022-08-01T19:41:24.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.606+0000 7fefd756c700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fefd0064b10 0x7fefd0144e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:24.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.606+0000 7fefd756c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fefd0141a80 0x7fefd0066ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:24.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.606+0000 7fefd756c700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefd0064ef0 con 0x7fefd00b36e0 2022-08-01T19:41:24.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.606+0000 7fefd756c700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefd013ebf0 con 0x7fefd0064b10 2022-08-01T19:41:24.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.606+0000 7fefd756c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefd0066690 con 0x7fefd0141a80 2022-08-01T19:41:24.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.606+0000 7fefd656a700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fefd0064b10 0x7fefd0144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:24.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.606+0000 7fefd6d6b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fefd0141a80 0x7fefd0066ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:24.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.606+0000 7fefd656a700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fefd0064b10 0x7fefd0144e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:39884/0 (socket says 172.21.15.17:39884) 2022-08-01T19:41:24.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.606+0000 7fefd656a700 1 -- 172.21.15.17:0/2526876746 learned_addr learned my addr 172.21.15.17:0/2526876746 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:24.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.610+0000 7fefd6d6b700 1 -- 172.21.15.17:0/2526876746 >> v2:172.21.15.77:3300/0 conn(0x7fefd0064b10 msgr2=0x7fefd0144e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:24.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.610+0000 7fefd6d6b700 1 --2- 172.21.15.17:0/2526876746 >> v2:172.21.15.77:3300/0 conn(0x7fefd0064b10 0x7fefd0144e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:24.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.610+0000 7fefd6d6b700 1 -- 172.21.15.17:0/2526876746 >> v1:172.21.15.77:6789/0 conn(0x7fefd00b36e0 legacy=0x7fefd0141370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:24.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.610+0000 7fefd6d6b700 1 -- 172.21.15.17:0/2526876746 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd00b4850 con 0x7fefd0141a80 2022-08-01T19:41:24.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.610+0000 7fefd6d6b700 1 --2- 172.21.15.17:0/2526876746 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fefd0141a80 0x7fefd0066ac0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fefc80032f0 tx=0x7fefc80091d0).ready entity=mon.2 client_cookie=1110ffd921d652e2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:24.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.610+0000 7fefd5568700 1 -- 172.21.15.17:0/2526876746 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefc800b6c0 con 0x7fefd0141a80 2022-08-01T19:41:24.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.610+0000 7fefd5568700 1 -- 172.21.15.17:0/2526876746 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc800b820 con 0x7fefd0141a80 2022-08-01T19:41:24.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.610+0000 7fefd5568700 1 -- 172.21.15.17:0/2526876746 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefc800b990 con 0x7fefd0141a80 2022-08-01T19:41:24.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.610+0000 7fefd756c700 1 -- 172.21.15.17:0/2526876746 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fefd0141a80 msgr2=0x7fefd0066ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:24.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.610+0000 7fefd756c700 1 --2- 172.21.15.17:0/2526876746 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fefd0141a80 0x7fefd0066ac0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fefc80032f0 tx=0x7fefc80091d0).stop 2022-08-01T19:41:24.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.610+0000 7fefd756c700 1 -- 172.21.15.17:0/2526876746 shutdown_connections 2022-08-01T19:41:24.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.610+0000 7fefd756c700 1 --2- 172.21.15.17:0/2526876746 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fefd0141a80 0x7fefd0066ac0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:24.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.610+0000 7fefd756c700 1 --2- 172.21.15.17:0/2526876746 >> v2:172.21.15.77:3300/0 conn(0x7fefd0064b10 0x7fefd0144e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:24.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.610+0000 7fefd756c700 1 -- 172.21.15.17:0/2526876746 >> 172.21.15.17:0/2526876746 conn(0x7fefd005f840 msgr2=0x7fefd005fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:24.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.610+0000 7fefd756c700 1 -- 172.21.15.17:0/2526876746 shutdown_connections 2022-08-01T19:41:24.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.610+0000 7fefd756c700 1 -- 172.21.15.17:0/2526876746 wait complete. 2022-08-01T19:41:24.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.610+0000 7fefd756c700 1 Processor -- start 2022-08-01T19:41:24.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.610+0000 7fefd756c700 1 -- start start 2022-08-01T19:41:24.636 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.614+0000 7fefd756c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefd0064b10 0x7fefd00b5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:24.636 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.614+0000 7fefd756c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fefd00b36e0 0x7fefd0150440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:24.637 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.614+0000 7fefd756c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefd0141a80 0x7fefd0150930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:24.637 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.614+0000 7fefd756c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd013d6a0 con 0x7fefd0064b10 2022-08-01T19:41:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.614+0000 7fefd756c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd013bef0 con 0x7fefd0141a80 2022-08-01T19:41:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.614+0000 7fefd756c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefd0150e20 con 0x7fefd00b36e0 2022-08-01T19:41:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.614+0000 7fefd656a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefd0064b10 0x7fefd00b5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.614+0000 7fefd656a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefd0064b10 0x7fefd00b5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41968/0 (socket says 172.21.15.17:41968) 2022-08-01T19:41:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.614+0000 7fefd656a700 1 -- 172.21.15.17:0/3841269073 learned_addr learned my addr 172.21.15.17:0/3841269073 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:24.640 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.614+0000 7fefd5d69700 1 --2- 172.21.15.17:0/3841269073 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fefd00b36e0 0x7fefd0150440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:24.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.614+0000 7fefd6d6b700 1 --2- 172.21.15.17:0/3841269073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefd0141a80 0x7fefd0150930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:24.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.614+0000 7fefd656a700 1 -- 172.21.15.17:0/3841269073 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fefd00b36e0 msgr2=0x7fefd0150440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:24.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.614+0000 7fefd656a700 1 --2- 172.21.15.17:0/3841269073 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fefd00b36e0 0x7fefd0150440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:24.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.614+0000 7fefd656a700 1 -- 172.21.15.17:0/3841269073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefd0141a80 msgr2=0x7fefd0150930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:24.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.614+0000 7fefd656a700 1 --2- 172.21.15.17:0/3841269073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefd0141a80 0x7fefd0150930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:24.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.614+0000 7fefd656a700 1 -- 172.21.15.17:0/3841269073 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc8009470 con 0x7fefd0064b10 2022-08-01T19:41:24.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.614+0000 7fefd5d69700 1 --2- 172.21.15.17:0/3841269073 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fefd00b36e0 0x7fefd0150440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:24.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.614+0000 7fefd656a700 1 --2- 172.21.15.17:0/3841269073 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefd0064b10 0x7fefd00b5e00 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0x7fefc000c000 tx=0x7fefc0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:24.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.614+0000 7fefcf7fe700 1 -- 172.21.15.17:0/3841269073 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefc00136e0 con 0x7fefd0064b10 2022-08-01T19:41:24.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.614+0000 7fefcf7fe700 1 -- 172.21.15.17:0/3841269073 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc0013840 con 0x7fefd0064b10 2022-08-01T19:41:24.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.614+0000 7fefd756c700 1 -- 172.21.15.17:0/3841269073 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefd013c160 con 0x7fefd0064b10 2022-08-01T19:41:24.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.614+0000 7fefcf7fe700 1 -- 172.21.15.17:0/3841269073 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefc0013ba0 con 0x7fefd0064b10 2022-08-01T19:41:24.645 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.614+0000 7fefd756c700 1 -- 172.21.15.17:0/3841269073 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefd013dba0 con 0x7fefd0064b10 2022-08-01T19:41:24.645 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.614+0000 7fefd756c700 1 -- 172.21.15.17:0/3841269073 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefd00574e0 con 0x7fefd0064b10 2022-08-01T19:41:24.646 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.614+0000 7fefcf7fe700 1 -- 172.21.15.17:0/3841269073 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fefc0023070 con 0x7fefd0064b10 2022-08-01T19:41:24.648 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.618+0000 7fefcf7fe700 1 --2- 172.21.15.17:0/3841269073 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fefb0053950 0x7fefb0055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:24.648 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.618+0000 7fefcf7fe700 1 -- 172.21.15.17:0/3841269073 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fefc006c360 con 0x7fefd0064b10 2022-08-01T19:41:24.649 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.618+0000 7fefd5d69700 1 --2- 172.21.15.17:0/3841269073 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fefb0053950 0x7fefb0055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:24.649 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.618+0000 7fefd5d69700 1 --2- 172.21.15.17:0/3841269073 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fefb0053950 0x7fefb0055e10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7fefc400b000 tx=0x7fefc4009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:24.650 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.622+0000 7fefcf7fe700 1 -- 172.21.15.17:0/3841269073 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fefc0043120 con 0x7fefd0064b10 2022-08-01T19:41:24.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.746+0000 7fefd756c700 1 -- 172.21.15.17:0/3841269073 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefd0069d90 con 0x7fefd0064b10 2022-08-01T19:41:24.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.750+0000 7fefcf7fe700 1 -- 172.21.15.17:0/3841269073 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16287 (secure 0 0 0) 0x7fefc0046740 con 0x7fefd0064b10 2022-08-01T19:41:24.754 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:24.755 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":97,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:41:24.477137+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:41:22.953305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2022-08-01T19:41:23.938052+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:41:24.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.754+0000 7fefd756c700 1 -- 172.21.15.17:0/3841269073 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fefb0053950 msgr2=0x7fefb0055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:24.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.754+0000 7fefd756c700 1 --2- 172.21.15.17:0/3841269073 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fefb0053950 0x7fefb0055e10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7fefc400b000 tx=0x7fefc4009040).stop 2022-08-01T19:41:24.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.754+0000 7fefd756c700 1 -- 172.21.15.17:0/3841269073 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefd0064b10 msgr2=0x7fefd00b5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:24.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.754+0000 7fefd756c700 1 --2- 172.21.15.17:0/3841269073 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefd0064b10 0x7fefd00b5e00 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0x7fefc000c000 tx=0x7fefc0008040).stop 2022-08-01T19:41:24.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.754+0000 7fefd756c700 1 -- 172.21.15.17:0/3841269073 shutdown_connections 2022-08-01T19:41:24.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.754+0000 7fefd756c700 1 --2- 172.21.15.17:0/3841269073 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fefb0053950 0x7fefb0055e10 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:24.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.754+0000 7fefd756c700 1 --2- 172.21.15.17:0/3841269073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefd0141a80 0x7fefd0150930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:24.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.754+0000 7fefd756c700 1 --2- 172.21.15.17:0/3841269073 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fefd00b36e0 0x7fefd0150440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:24.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.754+0000 7fefd756c700 1 --2- 172.21.15.17:0/3841269073 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefd0064b10 0x7fefd00b5e00 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:24.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.754+0000 7fefd756c700 1 -- 172.21.15.17:0/3841269073 >> 172.21.15.17:0/3841269073 conn(0x7fefd005f840 msgr2=0x7fefd005cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:24.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.754+0000 7fefd756c700 1 -- 172.21.15.17:0/3841269073 shutdown_connections 2022-08-01T19:41:24.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.754+0000 7fefd756c700 1 -- 172.21.15.17:0/3841269073 wait complete. 2022-08-01T19:41:24.773 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:41:24.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.874+0000 7ff4634d8700 1 Processor -- start 2022-08-01T19:41:24.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.874+0000 7ff4634d8700 1 -- start start 2022-08-01T19:41:24.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.878+0000 7ff4634d8700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ff45c0b2710 0x7ff45c1401e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:24.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.878+0000 7ff4634d8700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff45c0ac880 0x7ff45c1406d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:24.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.878+0000 7ff4634d8700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff45c067a10 con 0x7ff45c0b0270 2022-08-01T19:41:24.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.878+0000 7ff4634d8700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff45c0ad740 con 0x7ff45c0b2710 2022-08-01T19:41:24.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.878+0000 7ff4634d8700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff45c067cb0 con 0x7ff45c0ac880 2022-08-01T19:41:24.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.878+0000 7ff461cd5700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff45c0b0270 0x7ff45c143000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34170/0 (socket says 172.21.15.17:34170) 2022-08-01T19:41:24.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.878+0000 7ff461cd5700 1 -- 172.21.15.17:0/2862041448 learned_addr learned my addr 172.21.15.17:0/2862041448 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:24.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.878+0000 7ff462cd7700 1 --2- 172.21.15.17:0/2862041448 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff45c0ac880 0x7ff45c1406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:24.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.878+0000 7ff4624d6700 1 --2- 172.21.15.17:0/2862041448 >> v2:172.21.15.77:3300/0 conn(0x7ff45c0b2710 0x7ff45c1401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:24.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.878+0000 7ff4614d4700 1 -- 172.21.15.17:0/2862041448 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1086801100 0 0) 0x7ff45c067a10 con 0x7ff45c0b0270 2022-08-01T19:41:24.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.878+0000 7ff4614d4700 1 -- 172.21.15.17:0/2862041448 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff444003260 con 0x7ff45c0b0270 2022-08-01T19:41:24.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.878+0000 7ff462cd7700 1 -- 172.21.15.17:0/2862041448 >> v2:172.21.15.77:3300/0 conn(0x7ff45c0b2710 msgr2=0x7ff45c1401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:24.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.878+0000 7ff462cd7700 1 --2- 172.21.15.17:0/2862041448 >> v2:172.21.15.77:3300/0 conn(0x7ff45c0b2710 0x7ff45c1401e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:24.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.878+0000 7ff462cd7700 1 -- 172.21.15.17:0/2862041448 >> v1:172.21.15.17:6789/0 conn(0x7ff45c0b0270 legacy=0x7ff45c143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:24.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.878+0000 7ff462cd7700 1 -- 172.21.15.17:0/2862041448 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff45c147160 con 0x7ff45c0ac880 2022-08-01T19:41:24.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.878+0000 7ff462cd7700 1 --2- 172.21.15.17:0/2862041448 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff45c0ac880 0x7ff45c1406d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7ff4540032f0 tx=0x7ff4540091d0).ready entity=mon.2 client_cookie=ce8b42397727f579 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:24.901 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.878+0000 7ff4614d4700 1 -- 172.21.15.17:0/2862041448 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff45400b600 con 0x7ff45c0ac880 2022-08-01T19:41:24.902 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.878+0000 7ff4614d4700 1 -- 172.21.15.17:0/2862041448 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45400b760 con 0x7ff45c0ac880 2022-08-01T19:41:24.902 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.878+0000 7ff4614d4700 1 -- 172.21.15.17:0/2862041448 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff45400b8d0 con 0x7ff45c0ac880 2022-08-01T19:41:24.902 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.878+0000 7ff4634d8700 1 -- 172.21.15.17:0/2862041448 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff45c0ac880 msgr2=0x7ff45c1406d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:24.903 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.878+0000 7ff4634d8700 1 --2- 172.21.15.17:0/2862041448 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff45c0ac880 0x7ff45c1406d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7ff4540032f0 tx=0x7ff4540091d0).stop 2022-08-01T19:41:24.903 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.878+0000 7ff4634d8700 1 -- 172.21.15.17:0/2862041448 shutdown_connections 2022-08-01T19:41:24.904 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.878+0000 7ff4634d8700 1 --2- 172.21.15.17:0/2862041448 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff45c0ac880 0x7ff45c1406d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:24.904 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.878+0000 7ff4634d8700 1 --2- 172.21.15.17:0/2862041448 >> v2:172.21.15.77:3300/0 conn(0x7ff45c0b2710 0x7ff45c1401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:24.905 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.878+0000 7ff4634d8700 1 -- 172.21.15.17:0/2862041448 >> 172.21.15.17:0/2862041448 conn(0x7ff45c060310 msgr2=0x7ff45c0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:24.905 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.878+0000 7ff4634d8700 1 -- 172.21.15.17:0/2862041448 shutdown_connections 2022-08-01T19:41:24.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.878+0000 7ff4634d8700 1 -- 172.21.15.17:0/2862041448 wait complete. 2022-08-01T19:41:24.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.882+0000 7ff4634d8700 1 Processor -- start 2022-08-01T19:41:24.908 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.882+0000 7ff4634d8700 1 -- start start 2022-08-01T19:41:24.909 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.882+0000 7ff4634d8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff45c0ac880 0x7ff45c152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:24.909 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.882+0000 7ff4634d8700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff45c0b0270 0x7ff45c09ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:24.910 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.882+0000 7ff4634d8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff45c0b2710 0x7ff45c09f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:24.910 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.882+0000 7ff4634d8700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45c09f7f0 con 0x7ff45c0b2710 2022-08-01T19:41:24.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.882+0000 7ff4634d8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45c0abfe0 con 0x7ff45c0ac880 2022-08-01T19:41:24.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.882+0000 7ff4634d8700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff45c0ac280 con 0x7ff45c0b0270 2022-08-01T19:41:24.912 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.882+0000 7ff462cd7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff45c0b2710 0x7ff45c09f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:24.912 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.882+0000 7ff462cd7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff45c0b2710 0x7ff45c09f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41986/0 (socket says 172.21.15.17:41986) 2022-08-01T19:41:24.913 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.882+0000 7ff462cd7700 1 -- 172.21.15.17:0/2019426609 learned_addr learned my addr 172.21.15.17:0/2019426609 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:24.913 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.882+0000 7ff461cd5700 1 --2- 172.21.15.17:0/2019426609 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff45c0b0270 0x7ff45c09ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:24.914 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.882+0000 7ff4624d6700 1 --2- 172.21.15.17:0/2019426609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff45c0ac880 0x7ff45c152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:24.915 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.882+0000 7ff462cd7700 1 -- 172.21.15.17:0/2019426609 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff45c0b0270 msgr2=0x7ff45c09ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:24.919 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.882+0000 7ff462cd7700 1 --2- 172.21.15.17:0/2019426609 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff45c0b0270 0x7ff45c09ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:24.919 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.882+0000 7ff462cd7700 1 -- 172.21.15.17:0/2019426609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff45c0ac880 msgr2=0x7ff45c152e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:24.919 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.882+0000 7ff462cd7700 1 --2- 172.21.15.17:0/2019426609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff45c0ac880 0x7ff45c152e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:24.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.882+0000 7ff462cd7700 1 -- 172.21.15.17:0/2019426609 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff454009470 con 0x7ff45c0b2710 2022-08-01T19:41:24.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.882+0000 7ff462cd7700 1 --2- 172.21.15.17:0/2019426609 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff45c0b2710 0x7ff45c09f300 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 rx=0x7ff4540032f0 tx=0x7ff454016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:24.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.882+0000 7ff44b7fe700 1 -- 172.21.15.17:0/2019426609 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff45400b8d0 con 0x7ff45c0b2710 2022-08-01T19:41:24.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.886+0000 7ff44b7fe700 1 -- 172.21.15.17:0/2019426609 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45401f3f0 con 0x7ff45c0b2710 2022-08-01T19:41:24.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.886+0000 7ff4634d8700 1 -- 172.21.15.17:0/2019426609 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff45c0ac520 con 0x7ff45c0b2710 2022-08-01T19:41:24.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.886+0000 7ff44b7fe700 1 -- 172.21.15.17:0/2019426609 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff45400b600 con 0x7ff45c0b2710 2022-08-01T19:41:24.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.886+0000 7ff4634d8700 1 -- 172.21.15.17:0/2019426609 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff45c153300 con 0x7ff45c0b2710 2022-08-01T19:41:24.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.886+0000 7ff44b7fe700 1 -- 172.21.15.17:0/2019426609 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff454026030 con 0x7ff45c0b2710 2022-08-01T19:41:24.924 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.886+0000 7ff4634d8700 1 -- 172.21.15.17:0/2019426609 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff45c0574e0 con 0x7ff45c0b2710 2022-08-01T19:41:24.924 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.886+0000 7ff44b7fe700 1 --2- 172.21.15.17:0/2019426609 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff43c053b30 0x7ff43c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:24.925 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.886+0000 7ff44b7fe700 1 -- 172.21.15.17:0/2019426609 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff45406c9f0 con 0x7ff45c0b2710 2022-08-01T19:41:24.925 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.886+0000 7ff4624d6700 1 --2- 172.21.15.17:0/2019426609 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff43c053b30 0x7ff43c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:24.925 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.890+0000 7ff4624d6700 1 --2- 172.21.15.17:0/2019426609 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff43c053b30 0x7ff43c055ff0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7ff45800b000 tx=0x7ff458006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:24.927 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:24.898+0000 7ff44b7fe700 1 -- 172.21.15.17:0/2019426609 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff45406ee60 con 0x7ff45c0b2710 2022-08-01T19:41:25.027 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.022+0000 7ff4634d8700 1 -- 172.21.15.17:0/2019426609 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff45c0ab8d0 con 0x7ff45c0b2710 2022-08-01T19:41:25.027 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.022+0000 7ff44b7fe700 1 -- 172.21.15.17:0/2019426609 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16287 (secure 0 0 0) 0x7ff454024090 con 0x7ff45c0b2710 2022-08-01T19:41:25.028 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:25.028 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":97,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:41:24.477137+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:41:22.953305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2022-08-01T19:41:23.938052+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:41:25.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.026+0000 7ff4634d8700 1 -- 172.21.15.17:0/2019426609 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff43c053b30 msgr2=0x7ff43c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:25.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.026+0000 7ff4634d8700 1 --2- 172.21.15.17:0/2019426609 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff43c053b30 0x7ff43c055ff0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7ff45800b000 tx=0x7ff458006df0).stop 2022-08-01T19:41:25.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.026+0000 7ff4634d8700 1 -- 172.21.15.17:0/2019426609 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff45c0b2710 msgr2=0x7ff45c09f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:25.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.026+0000 7ff4634d8700 1 --2- 172.21.15.17:0/2019426609 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff45c0b2710 0x7ff45c09f300 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 rx=0x7ff4540032f0 tx=0x7ff454016000).stop 2022-08-01T19:41:25.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.026+0000 7ff4634d8700 1 -- 172.21.15.17:0/2019426609 shutdown_connections 2022-08-01T19:41:25.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.026+0000 7ff4634d8700 1 --2- 172.21.15.17:0/2019426609 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff43c053b30 0x7ff43c055ff0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:25.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.026+0000 7ff4634d8700 1 --2- 172.21.15.17:0/2019426609 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff45c0b2710 0x7ff45c09f300 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:25.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.026+0000 7ff4634d8700 1 --2- 172.21.15.17:0/2019426609 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff45c0b0270 0x7ff45c09ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:25.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.026+0000 7ff4634d8700 1 --2- 172.21.15.17:0/2019426609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff45c0ac880 0x7ff45c152e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:25.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.026+0000 7ff4634d8700 1 -- 172.21.15.17:0/2019426609 >> 172.21.15.17:0/2019426609 conn(0x7ff45c060310 msgr2=0x7ff45c1403f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:25.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.026+0000 7ff4634d8700 1 -- 172.21.15.17:0/2019426609 shutdown_connections 2022-08-01T19:41:25.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.026+0000 7ff4634d8700 1 -- 172.21.15.17:0/2019426609 wait complete. 2022-08-01T19:41:25.040 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-08-01T19:41:25.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.174+0000 7f1c05b79700 1 Processor -- start 2022-08-01T19:41:25.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.174+0000 7f1c05b79700 1 -- start start 2022-08-01T19:41:25.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.174+0000 7f1c05b79700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1c0005c960 0x7f1c000b1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:25.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.174+0000 7f1c05b79700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1c0005d240 0x7f1c0013f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:25.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.174+0000 7f1c05b79700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c0013fa10 con 0x7f1c000acfc0 2022-08-01T19:41:25.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.174+0000 7f1c05b79700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c00067f00 con 0x7f1c0005d240 2022-08-01T19:41:25.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.174+0000 7f1c05b79700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0005cd40 con 0x7f1c0005c960 2022-08-01T19:41:25.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.174+0000 7f1bfffff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1c0005d240 0x7f1c0013f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:25.184 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.174+0000 7f1bfffff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1c0005d240 0x7f1c0013f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41990/0 (socket says 172.21.15.17:41990) 2022-08-01T19:41:25.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.174+0000 7f1bfffff700 1 -- 172.21.15.17:0/3719211885 learned_addr learned my addr 172.21.15.17:0/3719211885 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:25.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.174+0000 7f1c04b77700 1 --2- 172.21.15.17:0/3719211885 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1c0005c960 0x7f1c000b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:25.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.174+0000 7f1bfffff700 1 -- 172.21.15.17:0/3719211885 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1c0005c960 msgr2=0x7f1c000b1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:25.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.174+0000 7f1bfffff700 1 --2- 172.21.15.17:0/3719211885 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1c0005c960 0x7f1c000b1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:25.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.174+0000 7f1bfffff700 1 -- 172.21.15.17:0/3719211885 >> v1:172.21.15.77:6789/0 conn(0x7f1c000acfc0 legacy=0x7f1c000aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:25.188 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.174+0000 7f1bfffff700 1 -- 172.21.15.17:0/3719211885 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c000b1920 con 0x7f1c0005d240 2022-08-01T19:41:25.188 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.174+0000 7f1bfffff700 1 --2- 172.21.15.17:0/3719211885 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c0005d240 0x7f1c0013f520 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 rx=0x7f1bec002f70 tx=0x7f1bec006e40).ready entity=mon.0 client_cookie=94cd49818ff6e81c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:25.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.178+0000 7f1bff7fe700 1 -- 172.21.15.17:0/3719211885 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bec00bd20 con 0x7f1c0005d240 2022-08-01T19:41:25.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.178+0000 7f1bff7fe700 1 -- 172.21.15.17:0/3719211885 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bec000e90 con 0x7f1c0005d240 2022-08-01T19:41:25.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.178+0000 7f1bff7fe700 1 -- 172.21.15.17:0/3719211885 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bec0155e0 con 0x7f1c0005d240 2022-08-01T19:41:25.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.178+0000 7f1c05b79700 1 -- 172.21.15.17:0/3719211885 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c0005d240 msgr2=0x7f1c0013f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:25.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.178+0000 7f1c05b79700 1 --2- 172.21.15.17:0/3719211885 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c0005d240 0x7f1c0013f520 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 rx=0x7f1bec002f70 tx=0x7f1bec006e40).stop 2022-08-01T19:41:25.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.182+0000 7f1c05b79700 1 -- 172.21.15.17:0/3719211885 shutdown_connections 2022-08-01T19:41:25.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.182+0000 7f1c05b79700 1 --2- 172.21.15.17:0/3719211885 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c0005d240 0x7f1c0013f520 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:25.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.182+0000 7f1c05b79700 1 --2- 172.21.15.17:0/3719211885 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1c0005c960 0x7f1c000b1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:25.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.182+0000 7f1c05b79700 1 -- 172.21.15.17:0/3719211885 >> 172.21.15.17:0/3719211885 conn(0x7f1c00060310 msgr2=0x7f1c000b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:25.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.182+0000 7f1c05b79700 1 -- 172.21.15.17:0/3719211885 shutdown_connections 2022-08-01T19:41:25.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.182+0000 7f1c05b79700 1 -- 172.21.15.17:0/3719211885 wait complete. 2022-08-01T19:41:25.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.182+0000 7f1c05b79700 1 Processor -- start 2022-08-01T19:41:25.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.182+0000 7f1c05b79700 1 -- start start 2022-08-01T19:41:25.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.186+0000 7f1c05b79700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1c0005c960 0x7f1c001517d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:25.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.186+0000 7f1c05b79700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c0005d240 0x7f1c000663f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:25.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.186+0000 7f1c05b79700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c000acfc0 0x7f1c00141470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:25.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.186+0000 7f1c05b79700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c001435a0 con 0x7f1c0005d240 2022-08-01T19:41:25.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.186+0000 7f1c05b79700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c00152140 con 0x7f1c000acfc0 2022-08-01T19:41:25.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.186+0000 7f1c05b79700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c000a94e0 con 0x7f1c0005c960 2022-08-01T19:41:25.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.186+0000 7f1bfffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c0005d240 0x7f1c000663f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:25.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.186+0000 7f1bfffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c0005d240 0x7f1c000663f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41998/0 (socket says 172.21.15.17:41998) 2022-08-01T19:41:25.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.186+0000 7f1bfffff700 1 -- 172.21.15.17:0/3505078335 learned_addr learned my addr 172.21.15.17:0/3505078335 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:25.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.186+0000 7f1c04b77700 1 --2- 172.21.15.17:0/3505078335 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1c0005c960 0x7f1c001517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:25.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.186+0000 7f1c05378700 1 --2- 172.21.15.17:0/3505078335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c000acfc0 0x7f1c00141470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:25.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.186+0000 7f1bfffff700 1 -- 172.21.15.17:0/3505078335 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1c0005c960 msgr2=0x7f1c001517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:25.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.186+0000 7f1bfffff700 1 --2- 172.21.15.17:0/3505078335 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1c0005c960 0x7f1c001517d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:25.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.186+0000 7f1bfffff700 1 -- 172.21.15.17:0/3505078335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c000acfc0 msgr2=0x7f1c00141470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:25.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.186+0000 7f1bfffff700 1 --2- 172.21.15.17:0/3505078335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c000acfc0 0x7f1c00141470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:25.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.186+0000 7f1bfffff700 1 -- 172.21.15.17:0/3505078335 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bec00d4d0 con 0x7f1c0005d240 2022-08-01T19:41:25.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.186+0000 7f1bfffff700 1 --2- 172.21.15.17:0/3505078335 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c0005d240 0x7f1c000663f0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 rx=0x7f1bec006fc0 tx=0x7f1bec018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:25.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.186+0000 7f1bfdffb700 1 -- 172.21.15.17:0/3505078335 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bec0159c0 con 0x7f1c0005d240 2022-08-01T19:41:25.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.190+0000 7f1bfdffb700 1 -- 172.21.15.17:0/3505078335 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bec015b20 con 0x7f1c0005d240 2022-08-01T19:41:25.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.190+0000 7f1bfdffb700 1 -- 172.21.15.17:0/3505078335 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bec015e80 con 0x7f1c0005d240 2022-08-01T19:41:25.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.190+0000 7f1c05b79700 1 -- 172.21.15.17:0/3505078335 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c00143020 con 0x7f1c0005d240 2022-08-01T19:41:25.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.190+0000 7f1c05b79700 1 -- 172.21.15.17:0/3505078335 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c00142460 con 0x7f1c0005d240 2022-08-01T19:41:25.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.190+0000 7f1bfdffb700 1 -- 172.21.15.17:0/3505078335 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1bec028460 con 0x7f1c0005d240 2022-08-01T19:41:25.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.190+0000 7f1c05b79700 1 -- 172.21.15.17:0/3505078335 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c00143710 con 0x7f1c0005d240 2022-08-01T19:41:25.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.190+0000 7f1bfdffb700 1 --2- 172.21.15.17:0/3505078335 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1be0053950 0x7f1be0055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:25.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.190+0000 7f1bfdffb700 1 -- 172.21.15.17:0/3505078335 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1bec06faf0 con 0x7f1c0005d240 2022-08-01T19:41:25.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.190+0000 7f1c04b77700 1 --2- 172.21.15.17:0/3505078335 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1be0053950 0x7f1be0055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:25.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.194+0000 7f1c04b77700 1 --2- 172.21.15.17:0/3505078335 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1be0053950 0x7f1be0055e10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f1bf400c000 tx=0x7f1bf4008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:25.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.198+0000 7f1bfdffb700 1 -- 172.21.15.17:0/3505078335 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1c00143710 con 0x7f1c0005d240 2022-08-01T19:41:25.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:25.362+0000 7f1c05b79700 1 -- 172.21.15.17:0/3505078335 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1c001436e0 con 0x7f1c0005d240 2022-08-01T19:41:26.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.190+0000 7f1bfdffb700 1 -- 172.21.15.17:0/3505078335 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v46) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1bec046780 con 0x7f1c0005d240 2022-08-01T19:41:26.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.194+0000 7f1c05b79700 1 -- 172.21.15.17:0/3505078335 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1be0053950 msgr2=0x7f1be0055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:26.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.194+0000 7f1c05b79700 1 --2- 172.21.15.17:0/3505078335 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1be0053950 0x7f1be0055e10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f1bf400c000 tx=0x7f1bf4008040).stop 2022-08-01T19:41:26.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.194+0000 7f1c05b79700 1 -- 172.21.15.17:0/3505078335 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c0005d240 msgr2=0x7f1c000663f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:26.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.194+0000 7f1c05b79700 1 --2- 172.21.15.17:0/3505078335 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c0005d240 0x7f1c000663f0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 rx=0x7f1bec006fc0 tx=0x7f1bec018000).stop 2022-08-01T19:41:26.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.194+0000 7f1c05b79700 1 -- 172.21.15.17:0/3505078335 shutdown_connections 2022-08-01T19:41:26.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.194+0000 7f1c05b79700 1 --2- 172.21.15.17:0/3505078335 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1be0053950 0x7f1be0055e10 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:26.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.194+0000 7f1c05b79700 1 --2- 172.21.15.17:0/3505078335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c000acfc0 0x7f1c00141470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:26.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.194+0000 7f1c05b79700 1 --2- 172.21.15.17:0/3505078335 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c0005d240 0x7f1c000663f0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:26.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.194+0000 7f1c05b79700 1 --2- 172.21.15.17:0/3505078335 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1c0005c960 0x7f1c001517d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:26.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.194+0000 7f1c05b79700 1 -- 172.21.15.17:0/3505078335 >> 172.21.15.17:0/3505078335 conn(0x7f1c00060310 msgr2=0x7f1c000b01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:26.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.194+0000 7f1c05b79700 1 -- 172.21.15.17:0/3505078335 shutdown_connections 2022-08-01T19:41:26.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.194+0000 7f1c05b79700 1 -- 172.21.15.17:0/3505078335 wait complete. 2022-08-01T19:41:26.213 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:41:26.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.322+0000 7f79f248e700 1 Processor -- start 2022-08-01T19:41:26.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.322+0000 7f79f248e700 1 -- start start 2022-08-01T19:41:26.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.322+0000 7f79f248e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f79ec0b2710 0x7f79ec14afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:26.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.322+0000 7f79f248e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79ec0b01a0 0x7f79ec14b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:26.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.322+0000 7f79f248e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f79ec0afd60 0x7f79ec0a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:26.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.322+0000 7f79f248e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79ec0a7250 con 0x7f79ec0afd60 2022-08-01T19:41:26.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.322+0000 7f79f248e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79ec0a7480 con 0x7f79ec0b2710 2022-08-01T19:41:26.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.322+0000 7f79f248e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec0a7720 con 0x7f79ec0b01a0 2022-08-01T19:41:26.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.322+0000 7f79f1c8d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f79ec0afd60 0x7f79ec0a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:26.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.322+0000 7f79f1c8d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f79ec0afd60 0x7f79ec0a8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42008/0 (socket says 172.21.15.17:42008) 2022-08-01T19:41:26.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.322+0000 7f79f1c8d700 1 -- 172.21.15.17:0/4122974591 learned_addr learned my addr 172.21.15.17:0/4122974591 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:26.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.322+0000 7f79f148c700 1 --2- 172.21.15.17:0/4122974591 >> v2:172.21.15.77:3300/0 conn(0x7f79ec0b2710 0x7f79ec14afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:26.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.322+0000 7f79f0c8b700 1 --2- 172.21.15.17:0/4122974591 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79ec0b01a0 0x7f79ec14b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:26.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.322+0000 7f79f1c8d700 1 -- 172.21.15.17:0/4122974591 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79ec0b01a0 msgr2=0x7f79ec14b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:26.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.322+0000 7f79f1c8d700 1 --2- 172.21.15.17:0/4122974591 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79ec0b01a0 0x7f79ec14b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:26.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.322+0000 7f79f1c8d700 1 -- 172.21.15.17:0/4122974591 >> v2:172.21.15.77:3300/0 conn(0x7f79ec0b2710 msgr2=0x7f79ec14afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:26.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.322+0000 7f79f1c8d700 1 --2- 172.21.15.17:0/4122974591 >> v2:172.21.15.77:3300/0 conn(0x7f79ec0b2710 0x7f79ec14afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:26.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.322+0000 7f79f1c8d700 1 -- 172.21.15.17:0/4122974591 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ec0a79c0 con 0x7f79ec0afd60 2022-08-01T19:41:26.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.322+0000 7f79f0c8b700 1 --2- 172.21.15.17:0/4122974591 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79ec0b01a0 0x7f79ec14b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:26.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.322+0000 7f79f148c700 1 --2- 172.21.15.17:0/4122974591 >> v2:172.21.15.77:3300/0 conn(0x7f79ec0b2710 0x7f79ec14afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:26.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.322+0000 7f79f1c8d700 1 --2- 172.21.15.17:0/4122974591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79ec0afd60 0x7f79ec0a8540 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0x7f79e4002f70 tx=0x7f79e4006eb0).ready entity=mon.0 client_cookie=4c4310c9dcfb8d37 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:26.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.322+0000 7f79ebfff700 1 -- 172.21.15.17:0/4122974591 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79e400d570 con 0x7f79ec0afd60 2022-08-01T19:41:26.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.326+0000 7f79ebfff700 1 -- 172.21.15.17:0/4122974591 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e4000e90 con 0x7f79ec0afd60 2022-08-01T19:41:26.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.326+0000 7f79ebfff700 1 -- 172.21.15.17:0/4122974591 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79e400d900 con 0x7f79ec0afd60 2022-08-01T19:41:26.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.326+0000 7f79f248e700 1 -- 172.21.15.17:0/4122974591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79ec0afd60 msgr2=0x7f79ec0a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:26.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.326+0000 7f79f248e700 1 --2- 172.21.15.17:0/4122974591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79ec0afd60 0x7f79ec0a8540 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0x7f79e4002f70 tx=0x7f79e4006eb0).stop 2022-08-01T19:41:26.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.326+0000 7f79f248e700 1 -- 172.21.15.17:0/4122974591 shutdown_connections 2022-08-01T19:41:26.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.326+0000 7f79f248e700 1 --2- 172.21.15.17:0/4122974591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79ec0afd60 0x7f79ec0a8540 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:26.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.326+0000 7f79f248e700 1 --2- 172.21.15.17:0/4122974591 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79ec0b01a0 0x7f79ec14b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:26.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.326+0000 7f79f248e700 1 --2- 172.21.15.17:0/4122974591 >> v2:172.21.15.77:3300/0 conn(0x7f79ec0b2710 0x7f79ec14afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:26.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.326+0000 7f79f248e700 1 -- 172.21.15.17:0/4122974591 >> 172.21.15.17:0/4122974591 conn(0x7f79ec060310 msgr2=0x7f79ec0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:26.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.326+0000 7f79f248e700 1 -- 172.21.15.17:0/4122974591 shutdown_connections 2022-08-01T19:41:26.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.326+0000 7f79f248e700 1 -- 172.21.15.17:0/4122974591 wait complete. 2022-08-01T19:41:26.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.326+0000 7f79f248e700 1 Processor -- start 2022-08-01T19:41:26.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.330+0000 7f79f248e700 1 -- start start 2022-08-01T19:41:26.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.330+0000 7f79f248e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79ec0afd60 0x7f79ec0b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:26.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.330+0000 7f79f248e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79ec0b01a0 0x7f79ec144010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:26.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.330+0000 7f79f248e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79ec0b2710 0x7f79ec0b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:26.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.330+0000 7f79f248e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec0a97a0 con 0x7f79ec0b2710 2022-08-01T19:41:26.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.330+0000 7f79f248e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec0a9a40 con 0x7f79ec0afd60 2022-08-01T19:41:26.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.330+0000 7f79f248e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec0a9ce0 con 0x7f79ec0b01a0 2022-08-01T19:41:26.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.330+0000 7f79f1c8d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79ec0b2710 0x7f79ec0b4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:26.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.330+0000 7f79f1c8d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79ec0b2710 0x7f79ec0b4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42014/0 (socket says 172.21.15.17:42014) 2022-08-01T19:41:26.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.330+0000 7f79f1c8d700 1 -- 172.21.15.17:0/310966772 learned_addr learned my addr 172.21.15.17:0/310966772 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:26.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.330+0000 7f79f0c8b700 1 --2- 172.21.15.17:0/310966772 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79ec0b01a0 0x7f79ec144010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:26.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.330+0000 7f79f148c700 1 --2- 172.21.15.17:0/310966772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79ec0afd60 0x7f79ec0b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:26.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.330+0000 7f79f1c8d700 1 -- 172.21.15.17:0/310966772 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79ec0b01a0 msgr2=0x7f79ec144010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:26.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.330+0000 7f79f1c8d700 1 --2- 172.21.15.17:0/310966772 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79ec0b01a0 0x7f79ec144010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:26.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.330+0000 7f79f1c8d700 1 -- 172.21.15.17:0/310966772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79ec0afd60 msgr2=0x7f79ec0b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:26.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.330+0000 7f79f1c8d700 1 --2- 172.21.15.17:0/310966772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79ec0afd60 0x7f79ec0b5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:26.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.330+0000 7f79f1c8d700 1 -- 172.21.15.17:0/310966772 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e400b1e0 con 0x7f79ec0b2710 2022-08-01T19:41:26.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.330+0000 7f79f1c8d700 1 --2- 172.21.15.17:0/310966772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79ec0b2710 0x7f79ec0b4a40 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 rx=0x7f79e4002f70 tx=0x7f79e4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:26.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.330+0000 7f79ea7fc700 1 -- 172.21.15.17:0/310966772 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79e4006be0 con 0x7f79ec0b2710 2022-08-01T19:41:26.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.330+0000 7f79ea7fc700 1 -- 172.21.15.17:0/310966772 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e400d8f0 con 0x7f79ec0b2710 2022-08-01T19:41:26.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.330+0000 7f79f248e700 1 -- 172.21.15.17:0/310966772 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79ec0a9f80 con 0x7f79ec0b2710 2022-08-01T19:41:26.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.330+0000 7f79ea7fc700 1 -- 172.21.15.17:0/310966772 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79e4011d70 con 0x7f79ec0b2710 2022-08-01T19:41:26.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.330+0000 7f79f248e700 1 -- 172.21.15.17:0/310966772 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79ec05c460 con 0x7f79ec0b2710 2022-08-01T19:41:26.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.334+0000 7f79f248e700 1 -- 172.21.15.17:0/310966772 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79ec0aa110 con 0x7f79ec0b2710 2022-08-01T19:41:26.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.334+0000 7f79ea7fc700 1 -- 172.21.15.17:0/310966772 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f79e4025030 con 0x7f79ec0b2710 2022-08-01T19:41:26.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.334+0000 7f79ea7fc700 1 --2- 172.21.15.17:0/310966772 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f79cc053aa0 0x7f79cc055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:26.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.334+0000 7f79ea7fc700 1 -- 172.21.15.17:0/310966772 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f79e406d0e0 con 0x7f79ec0b2710 2022-08-01T19:41:26.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.334+0000 7f79f148c700 1 --2- 172.21.15.17:0/310966772 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f79cc053aa0 0x7f79cc055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:26.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.334+0000 7f79f148c700 1 --2- 172.21.15.17:0/310966772 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f79cc053aa0 0x7f79cc055f60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f79e000c000 tx=0x7f79e0008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:26.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.338+0000 7f79ea7fc700 1 -- 172.21.15.17:0/310966772 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f79e4043ee0 con 0x7f79ec0b2710 2022-08-01T19:41:26.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.506+0000 7f79f248e700 1 -- 172.21.15.17:0/310966772 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79ec0574e0 con 0x7f79ec0b2710 2022-08-01T19:41:26.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.506+0000 7f79ea7fc700 1 -- 172.21.15.17:0/310966772 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f79e4023020 con 0x7f79ec0b2710 2022-08-01T19:41:26.509 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:26.509 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":46,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6943,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/256800053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":256800053},{"type":"v1","addr":"172.21.15.77:6835","nonce":256800053}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44},{"gid":6957,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3629116150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3629116150},{"type":"v1","addr":"172.21.15.17:6833","nonce":3629116150}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:41:24.477490+0000","modified":"2022-08-01T19:41:26.186850+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6960},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6960":{"gid":6960,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/2788758172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2788758172},{"type":"v1","addr":"172.21.15.17:6835","nonce":2788758172}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2022-08-01T19:41:26.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.510+0000 7f79f248e700 1 -- 172.21.15.17:0/310966772 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f79cc053aa0 msgr2=0x7f79cc055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:26.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.510+0000 7f79f248e700 1 --2- 172.21.15.17:0/310966772 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f79cc053aa0 0x7f79cc055f60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f79e000c000 tx=0x7f79e0008040).stop 2022-08-01T19:41:26.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.510+0000 7f79f248e700 1 -- 172.21.15.17:0/310966772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79ec0b2710 msgr2=0x7f79ec0b4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:26.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.510+0000 7f79f248e700 1 --2- 172.21.15.17:0/310966772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79ec0b2710 0x7f79ec0b4a40 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 rx=0x7f79e4002f70 tx=0x7f79e4015000).stop 2022-08-01T19:41:26.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.510+0000 7f79f248e700 1 -- 172.21.15.17:0/310966772 shutdown_connections 2022-08-01T19:41:26.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.510+0000 7f79f248e700 1 --2- 172.21.15.17:0/310966772 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f79cc053aa0 0x7f79cc055f60 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:26.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.510+0000 7f79f248e700 1 --2- 172.21.15.17:0/310966772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79ec0b2710 0x7f79ec0b4a40 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:26.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.510+0000 7f79f248e700 1 --2- 172.21.15.17:0/310966772 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79ec0b01a0 0x7f79ec144010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:26.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.510+0000 7f79f248e700 1 --2- 172.21.15.17:0/310966772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79ec0afd60 0x7f79ec0b5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:26.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.510+0000 7f79f248e700 1 -- 172.21.15.17:0/310966772 >> 172.21.15.17:0/310966772 conn(0x7f79ec060310 msgr2=0x7f79ec148790 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:26.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.510+0000 7f79f248e700 1 -- 172.21.15.17:0/310966772 shutdown_connections 2022-08-01T19:41:26.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.510+0000 7f79f248e700 1 -- 172.21.15.17:0/310966772 wait complete. 2022-08-01T19:41:26.517 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 46 2022-08-01T19:41:26.528 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:41:26.637 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.630+0000 7f386f59e700 1 Processor -- start 2022-08-01T19:41:26.638 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.634+0000 7f386f59e700 1 -- start start 2022-08-01T19:41:26.638 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.634+0000 7f386f59e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f38700b2710 0x7f38700abca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:26.639 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.634+0000 7f386f59e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f38700b13f0 0x7f38700ac190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:26.639 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.634+0000 7f386f59e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38700adf10 con 0x7f38700b1920 2022-08-01T19:41:26.639 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.634+0000 7f386f59e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38700ae1b0 con 0x7f38700b13f0 2022-08-01T19:41:26.639 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.634+0000 7f386f59e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38700a9520 con 0x7f38700b2710 2022-08-01T19:41:26.640 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.634+0000 7f3867fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f38700b13f0 0x7f38700ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:26.640 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.634+0000 7f3867fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f38700b13f0 0x7f38700ac190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42022/0 (socket says 172.21.15.17:42022) 2022-08-01T19:41:26.640 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.634+0000 7f3867fff700 1 -- 172.21.15.17:0/2066504226 learned_addr learned my addr 172.21.15.17:0/2066504226 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:26.640 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.634+0000 7f386e59c700 1 --2- 172.21.15.17:0/2066504226 >> v2:172.21.15.77:3300/0 conn(0x7f38700b2710 0x7f38700abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:26.640 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.634+0000 7f3867fff700 1 -- 172.21.15.17:0/2066504226 >> v2:172.21.15.77:3300/0 conn(0x7f38700b2710 msgr2=0x7f38700abca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:26.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.634+0000 7f3867fff700 1 --2- 172.21.15.17:0/2066504226 >> v2:172.21.15.77:3300/0 conn(0x7f38700b2710 0x7f38700abca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:26.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.634+0000 7f3867fff700 1 -- 172.21.15.17:0/2066504226 >> v1:172.21.15.77:6789/0 conn(0x7f38700b1920 legacy=0x7f38700afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:26.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.634+0000 7f3867fff700 1 -- 172.21.15.17:0/2066504226 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38700a99c0 con 0x7f38700b13f0 2022-08-01T19:41:26.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.634+0000 7f3867fff700 1 --2- 172.21.15.17:0/2066504226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38700b13f0 0x7f38700ac190 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0x7f3858002f70 tx=0x7f3858006eb0).ready entity=mon.0 client_cookie=e2b99664019e5f9b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:26.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.634+0000 7f386dd9b700 1 -- 172.21.15.17:0/2066504226 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f385800f9c0 con 0x7f38700b13f0 2022-08-01T19:41:26.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.634+0000 7f386dd9b700 1 -- 172.21.15.17:0/2066504226 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3858000e90 con 0x7f38700b13f0 2022-08-01T19:41:26.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.634+0000 7f386dd9b700 1 -- 172.21.15.17:0/2066504226 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f385800fd50 con 0x7f38700b13f0 2022-08-01T19:41:26.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.634+0000 7f386f59e700 1 -- 172.21.15.17:0/2066504226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38700b13f0 msgr2=0x7f38700ac190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:26.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.634+0000 7f386f59e700 1 --2- 172.21.15.17:0/2066504226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38700b13f0 0x7f38700ac190 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0x7f3858002f70 tx=0x7f3858006eb0).stop 2022-08-01T19:41:26.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.634+0000 7f386f59e700 1 -- 172.21.15.17:0/2066504226 shutdown_connections 2022-08-01T19:41:26.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.634+0000 7f386f59e700 1 --2- 172.21.15.17:0/2066504226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38700b13f0 0x7f38700ac190 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:26.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.634+0000 7f386f59e700 1 --2- 172.21.15.17:0/2066504226 >> v2:172.21.15.77:3300/0 conn(0x7f38700b2710 0x7f38700abca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:26.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.634+0000 7f386f59e700 1 -- 172.21.15.17:0/2066504226 >> 172.21.15.17:0/2066504226 conn(0x7f3870060310 msgr2=0x7f38700b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:26.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.634+0000 7f386f59e700 1 -- 172.21.15.17:0/2066504226 shutdown_connections 2022-08-01T19:41:26.645 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.634+0000 7f386f59e700 1 -- 172.21.15.17:0/2066504226 wait complete. 2022-08-01T19:41:26.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.638+0000 7f386f59e700 1 Processor -- start 2022-08-01T19:41:26.660 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.638+0000 7f386f59e700 1 -- start start 2022-08-01T19:41:26.660 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.638+0000 7f386f59e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38700b13f0 0x7f3870156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:26.660 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.638+0000 7f386f59e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38700b1920 0x7f387009f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:26.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.638+0000 7f386f59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f38700b2710 0x7f3870152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:26.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.638+0000 7f386f59e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38700ab760 con 0x7f38700b1920 2022-08-01T19:41:26.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.638+0000 7f386f59e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387009edc0 con 0x7f38700b13f0 2022-08-01T19:41:26.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.638+0000 7f386f59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3870155c70 con 0x7f38700b2710 2022-08-01T19:41:26.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.638+0000 7f3867fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38700b1920 0x7f387009f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:26.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.638+0000 7f386e59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38700b13f0 0x7f3870156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:26.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.638+0000 7f3867fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38700b1920 0x7f387009f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42026/0 (socket says 172.21.15.17:42026) 2022-08-01T19:41:26.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.638+0000 7f3867fff700 1 -- 172.21.15.17:0/365175145 learned_addr learned my addr 172.21.15.17:0/365175145 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:26.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.638+0000 7f386ed9d700 1 --2- 172.21.15.17:0/365175145 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f38700b2710 0x7f3870152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:26.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.638+0000 7f386e59c700 1 -- 172.21.15.17:0/365175145 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f38700b2710 msgr2=0x7f3870152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:26.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.638+0000 7f386e59c700 1 --2- 172.21.15.17:0/365175145 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f38700b2710 0x7f3870152610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:26.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.638+0000 7f386e59c700 1 -- 172.21.15.17:0/365175145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38700b1920 msgr2=0x7f387009f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:26.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.638+0000 7f386e59c700 1 --2- 172.21.15.17:0/365175145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38700b1920 0x7f387009f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:26.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.638+0000 7f386e59c700 1 -- 172.21.15.17:0/365175145 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f385800b1e0 con 0x7f38700b13f0 2022-08-01T19:41:26.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.642+0000 7f386e59c700 1 --2- 172.21.15.17:0/365175145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38700b13f0 0x7f3870156650 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f386000c000 tx=0x7f3860006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:26.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.642+0000 7f38677fe700 1 -- 172.21.15.17:0/365175145 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3860010780 con 0x7f38700b13f0 2022-08-01T19:41:26.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.642+0000 7f38677fe700 1 -- 172.21.15.17:0/365175145 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38600108e0 con 0x7f38700b13f0 2022-08-01T19:41:26.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.642+0000 7f386f59e700 1 -- 172.21.15.17:0/365175145 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3870155f10 con 0x7f38700b13f0 2022-08-01T19:41:26.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.642+0000 7f38677fe700 1 -- 172.21.15.17:0/365175145 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3860010c40 con 0x7f38700b13f0 2022-08-01T19:41:26.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.642+0000 7f386f59e700 1 -- 172.21.15.17:0/365175145 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f387009f720 con 0x7f38700b13f0 2022-08-01T19:41:26.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.642+0000 7f386f59e700 1 -- 172.21.15.17:0/365175145 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38700ac160 con 0x7f38700b13f0 2022-08-01T19:41:26.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.642+0000 7f38677fe700 1 -- 172.21.15.17:0/365175145 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f386001c460 con 0x7f38700b13f0 2022-08-01T19:41:26.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.642+0000 7f38677fe700 1 --2- 172.21.15.17:0/365175145 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3854053a70 0x7f3854055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:26.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.642+0000 7f38677fe700 1 -- 172.21.15.17:0/365175145 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f386006c2c0 con 0x7f38700b13f0 2022-08-01T19:41:26.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.642+0000 7f3867fff700 1 --2- 172.21.15.17:0/365175145 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3854053a70 0x7f3854055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:26.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.646+0000 7f3867fff700 1 --2- 172.21.15.17:0/365175145 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3854053a70 0x7f3854055f30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f385801c000 tx=0x7f3858017000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:26.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.654+0000 7f38677fe700 1 -- 172.21.15.17:0/365175145 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3860043020 con 0x7f38700b13f0 2022-08-01T19:41:26.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.778+0000 7f386f59e700 1 -- 172.21.15.17:0/365175145 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38700574e0 con 0x7f38700b13f0 2022-08-01T19:41:26.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.782+0000 7f38677fe700 1 -- 172.21.15.17:0/365175145 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f3860046640 con 0x7f38700b13f0 2022-08-01T19:41:26.785 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:26.800 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":98,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:41:25.480327+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:41:22.953305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2022-08-01T19:41:23.938052+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:41:26.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.782+0000 7f386f59e700 1 -- 172.21.15.17:0/365175145 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3854053a70 msgr2=0x7f3854055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:26.804 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.782+0000 7f386f59e700 1 --2- 172.21.15.17:0/365175145 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3854053a70 0x7f3854055f30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f385801c000 tx=0x7f3858017000).stop 2022-08-01T19:41:26.804 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.782+0000 7f386f59e700 1 -- 172.21.15.17:0/365175145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38700b13f0 msgr2=0x7f3870156650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:26.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.782+0000 7f386f59e700 1 --2- 172.21.15.17:0/365175145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38700b13f0 0x7f3870156650 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f386000c000 tx=0x7f3860006df0).stop 2022-08-01T19:41:26.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.782+0000 7f386f59e700 1 -- 172.21.15.17:0/365175145 shutdown_connections 2022-08-01T19:41:26.814 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.782+0000 7f386f59e700 1 --2- 172.21.15.17:0/365175145 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3854053a70 0x7f3854055f30 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:26.815 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.782+0000 7f386f59e700 1 --2- 172.21.15.17:0/365175145 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f38700b2710 0x7f3870152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:26.815 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.782+0000 7f386f59e700 1 --2- 172.21.15.17:0/365175145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38700b1920 0x7f387009f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:26.816 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.782+0000 7f386f59e700 1 --2- 172.21.15.17:0/365175145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38700b13f0 0x7f3870156650 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:26.816 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.782+0000 7f386f59e700 1 -- 172.21.15.17:0/365175145 >> 172.21.15.17:0/365175145 conn(0x7f3870060310 msgr2=0x7f3870142690 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:26.816 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.782+0000 7f386f59e700 1 -- 172.21.15.17:0/365175145 shutdown_connections 2022-08-01T19:41:26.817 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.782+0000 7f386f59e700 1 -- 172.21.15.17:0/365175145 wait complete. 2022-08-01T19:41:26.818 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T19:41:26.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.950+0000 7f4a07fff700 1 Processor -- start 2022-08-01T19:41:26.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.950+0000 7f4a07fff700 1 -- start start 2022-08-01T19:41:26.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.950+0000 7f4a07fff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f4a080b0d10 0x7f4a08146df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:26.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.950+0000 7f4a07fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4a080b0560 0x7f4a08140350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:26.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.950+0000 7f4a07fff700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a080b12d0 con 0x7f4a080a9320 2022-08-01T19:41:26.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.950+0000 7f4a07fff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a080a9700 con 0x7f4a080b0d10 2022-08-01T19:41:26.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.950+0000 7f4a07fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a080b1500 con 0x7f4a080b0560 2022-08-01T19:41:26.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.954+0000 7f4a067fc700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f4a080b0d10 0x7f4a08146df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:26.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.954+0000 7f4a067fc700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f4a080b0d10 0x7f4a08146df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:39956/0 (socket says 172.21.15.17:39956) 2022-08-01T19:41:26.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.954+0000 7f4a067fc700 1 -- 172.21.15.17:0/4116307828 learned_addr learned my addr 172.21.15.17:0/4116307828 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:26.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.954+0000 7f4a077fe700 1 --2- 172.21.15.17:0/4116307828 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4a080b0560 0x7f4a08140350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:26.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.954+0000 7f4a067fc700 1 -- 172.21.15.17:0/4116307828 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4a080b0560 msgr2=0x7f4a08140350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:26.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.954+0000 7f4a067fc700 1 --2- 172.21.15.17:0/4116307828 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4a080b0560 0x7f4a08140350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:26.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.954+0000 7f4a067fc700 1 -- 172.21.15.17:0/4116307828 >> v1:172.21.15.77:6789/0 conn(0x7f4a080a9320 legacy=0x7f4a081466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:26.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.954+0000 7f4a05ffb700 1 -- 172.21.15.17:0/4116307828 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2266673144 0 0) 0x7f4a080b12d0 con 0x7f4a080a9320 2022-08-01T19:41:26.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.954+0000 7f4a067fc700 1 -- 172.21.15.17:0/4116307828 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a080b17a0 con 0x7f4a080b0d10 2022-08-01T19:41:26.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.954+0000 7f4a067fc700 1 --2- 172.21.15.17:0/4116307828 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a080b0d10 0x7f4a08146df0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f49fc002f70 tx=0x7f49fc006eb0).ready entity=mon.1 client_cookie=90e1993dd2a36614 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:26.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.954+0000 7f4a05ffb700 1 -- 172.21.15.17:0/4116307828 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49fc00d570 con 0x7f4a080b0d10 2022-08-01T19:41:26.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.954+0000 7f4a05ffb700 1 -- 172.21.15.17:0/4116307828 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49fc000e90 con 0x7f4a080b0d10 2022-08-01T19:41:26.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.954+0000 7f4a05ffb700 1 -- 172.21.15.17:0/4116307828 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49fc00d8c0 con 0x7f4a080b0d10 2022-08-01T19:41:26.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.954+0000 7f4a07fff700 1 -- 172.21.15.17:0/4116307828 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a080b0d10 msgr2=0x7f4a08146df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:26.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.954+0000 7f4a07fff700 1 --2- 172.21.15.17:0/4116307828 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a080b0d10 0x7f4a08146df0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f49fc002f70 tx=0x7f49fc006eb0).stop 2022-08-01T19:41:26.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.954+0000 7f4a07fff700 1 -- 172.21.15.17:0/4116307828 shutdown_connections 2022-08-01T19:41:26.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.954+0000 7f4a07fff700 1 --2- 172.21.15.17:0/4116307828 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4a080b0560 0x7f4a08140350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:26.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.954+0000 7f4a07fff700 1 --2- 172.21.15.17:0/4116307828 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a080b0d10 0x7f4a08146df0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:26.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.954+0000 7f4a07fff700 1 -- 172.21.15.17:0/4116307828 >> 172.21.15.17:0/4116307828 conn(0x7f4a080612d0 msgr2=0x7f4a08062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:26.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.954+0000 7f4a07fff700 1 -- 172.21.15.17:0/4116307828 shutdown_connections 2022-08-01T19:41:26.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.954+0000 7f4a07fff700 1 -- 172.21.15.17:0/4116307828 wait complete. 2022-08-01T19:41:26.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.958+0000 7f4a07fff700 1 Processor -- start 2022-08-01T19:41:26.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.958+0000 7f4a07fff700 1 -- start start 2022-08-01T19:41:26.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.958+0000 7f4a07fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a080a9320 0x7f4a080b5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:26.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.958+0000 7f4a07fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a080b0560 0x7f4a080b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:26.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.958+0000 7f4a07fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4a080b0d10 0x7f4a08142c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:26.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.958+0000 7f4a07fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a081427b0 con 0x7f4a080a9320 2022-08-01T19:41:26.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.958+0000 7f4a07fff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a080b4f30 con 0x7f4a080b0560 2022-08-01T19:41:26.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.958+0000 7f4a07fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a080b51d0 con 0x7f4a080b0d10 2022-08-01T19:41:26.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.958+0000 7f4a06ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a080a9320 0x7f4a080b5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:26.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.958+0000 7f4a06ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a080a9320 0x7f4a080b5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42038/0 (socket says 172.21.15.17:42038) 2022-08-01T19:41:26.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.958+0000 7f4a06ffd700 1 -- 172.21.15.17:0/3355687060 learned_addr learned my addr 172.21.15.17:0/3355687060 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:26.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.958+0000 7f4a077fe700 1 --2- 172.21.15.17:0/3355687060 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4a080b0d10 0x7f4a08142c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:26.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.958+0000 7f4a067fc700 1 --2- 172.21.15.17:0/3355687060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a080b0560 0x7f4a080b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:26.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.958+0000 7f4a077fe700 1 -- 172.21.15.17:0/3355687060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a080b0560 msgr2=0x7f4a080b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:26.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.958+0000 7f4a077fe700 1 --2- 172.21.15.17:0/3355687060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a080b0560 0x7f4a080b45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:26.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.958+0000 7f4a077fe700 1 -- 172.21.15.17:0/3355687060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a080a9320 msgr2=0x7f4a080b5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:26.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.958+0000 7f4a077fe700 1 --2- 172.21.15.17:0/3355687060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a080a9320 0x7f4a080b5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:26.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.958+0000 7f4a077fe700 1 -- 172.21.15.17:0/3355687060 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49fc00b1e0 con 0x7f4a080b0d10 2022-08-01T19:41:26.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.962+0000 7f4a077fe700 1 --2- 172.21.15.17:0/3355687060 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4a080b0d10 0x7f4a08142c10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f4a0000c000 tx=0x7f4a000091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:26.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.962+0000 7f49f7fff700 1 -- 172.21.15.17:0/3355687060 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a00012600 con 0x7f4a080b0d10 2022-08-01T19:41:26.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.962+0000 7f49f7fff700 1 -- 172.21.15.17:0/3355687060 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a00012760 con 0x7f4a080b0d10 2022-08-01T19:41:26.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.962+0000 7f4a07fff700 1 -- 172.21.15.17:0/3355687060 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a080b4ae0 con 0x7f4a080b0d10 2022-08-01T19:41:26.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.962+0000 7f49f7fff700 1 -- 172.21.15.17:0/3355687060 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a00012ac0 con 0x7f4a080b0d10 2022-08-01T19:41:26.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.962+0000 7f4a07fff700 1 -- 172.21.15.17:0/3355687060 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a08141350 con 0x7f4a080b0d10 2022-08-01T19:41:26.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.962+0000 7f4a07fff700 1 -- 172.21.15.17:0/3355687060 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a08142920 con 0x7f4a080b0d10 2022-08-01T19:41:26.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.962+0000 7f49f7fff700 1 -- 172.21.15.17:0/3355687060 <== mon.2 v2:172.21.15.77:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4a00022070 con 0x7f4a080b0d10 2022-08-01T19:41:26.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.962+0000 7f49f7fff700 1 --2- 172.21.15.17:0/3355687060 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f49e4053980 0x7f49e4055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:26.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.966+0000 7f49f7fff700 1 -- 172.21.15.17:0/3355687060 <== mon.2 v2:172.21.15.77:3301/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4a0006ca60 con 0x7f4a080b0d10 2022-08-01T19:41:26.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.966+0000 7f4a06ffd700 1 --2- 172.21.15.17:0/3355687060 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f49e4053980 0x7f49e4055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:26.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.966+0000 7f4a06ffd700 1 --2- 172.21.15.17:0/3355687060 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f49e4053980 0x7f49e4055e40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f49f800d000 tx=0x7f49f8007c20).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:26.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:26.970+0000 7f49f7fff700 1 -- 172.21.15.17:0/3355687060 <== mon.2 v2:172.21.15.77:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4a00043820 con 0x7f4a080b0d10 2022-08-01T19:41:27.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.138+0000 7f4a07fff700 1 -- 172.21.15.17:0/3355687060 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f4a080574e0 con 0x7f4a080b0d10 2022-08-01T19:41:27.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.146+0000 7f49f7fff700 1 -- 172.21.15.17:0/3355687060 <== mon.2 v2:172.21.15.77:3301/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v27) v1 ==== 184+0+0 (secure 0 0 0) 0x7f4a000160e0 con 0x7f4a080b0d10 2022-08-01T19:41:27.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.150+0000 7f4a07fff700 1 -- 172.21.15.17:0/3355687060 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f49e4053980 msgr2=0x7f49e4055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:27.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.150+0000 7f4a07fff700 1 --2- 172.21.15.17:0/3355687060 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f49e4053980 0x7f49e4055e40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f49f800d000 tx=0x7f49f8007c20).stop 2022-08-01T19:41:27.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.150+0000 7f4a07fff700 1 -- 172.21.15.17:0/3355687060 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4a080b0d10 msgr2=0x7f4a08142c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:27.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.150+0000 7f4a07fff700 1 --2- 172.21.15.17:0/3355687060 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4a080b0d10 0x7f4a08142c10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f4a0000c000 tx=0x7f4a000091d0).stop 2022-08-01T19:41:27.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.150+0000 7f4a07fff700 1 -- 172.21.15.17:0/3355687060 shutdown_connections 2022-08-01T19:41:27.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.150+0000 7f4a07fff700 1 --2- 172.21.15.17:0/3355687060 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f49e4053980 0x7f49e4055e40 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:27.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.150+0000 7f4a07fff700 1 --2- 172.21.15.17:0/3355687060 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4a080b0d10 0x7f4a08142c10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:27.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.150+0000 7f4a07fff700 1 --2- 172.21.15.17:0/3355687060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a080b0560 0x7f4a080b45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:27.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.150+0000 7f4a07fff700 1 --2- 172.21.15.17:0/3355687060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a080a9320 0x7f4a080b5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:27.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.150+0000 7f4a07fff700 1 -- 172.21.15.17:0/3355687060 >> 172.21.15.17:0/3355687060 conn(0x7f4a080612d0 msgr2=0x7f4a0813c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:27.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.150+0000 7f4a07fff700 1 -- 172.21.15.17:0/3355687060 shutdown_connections 2022-08-01T19:41:27.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.150+0000 7f4a07fff700 1 -- 172.21.15.17:0/3355687060 wait complete. 2022-08-01T19:41:27.162 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.0 2022-08-01T19:41:27.164 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T19:41:27.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.282+0000 7fefc7216700 1 Processor -- start 2022-08-01T19:41:27.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.282+0000 7fefc7216700 1 -- start start 2022-08-01T19:41:27.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.286+0000 7fefc7216700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fefc00b0d10 0x7fefc0146df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:27.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.286+0000 7fefc7216700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fefc00b0560 0x7fefc0140350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:27.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.286+0000 7fefc7216700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefc00b12d0 con 0x7fefc00a9320 2022-08-01T19:41:27.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.286+0000 7fefc7216700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefc00a9700 con 0x7fefc00b0560 2022-08-01T19:41:27.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.286+0000 7fefc7216700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefc00b1500 con 0x7fefc00b0d10 2022-08-01T19:41:27.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.286+0000 7fefc6a15700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fefc00b0560 0x7fefc0140350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:27.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.286+0000 7fefc6a15700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fefc00b0560 0x7fefc0140350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42050/0 (socket says 172.21.15.17:42050) 2022-08-01T19:41:27.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.286+0000 7fefc6a15700 1 -- 172.21.15.17:0/3705510328 learned_addr learned my addr 172.21.15.17:0/3705510328 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:27.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.286+0000 7fefc5a13700 1 --2- 172.21.15.17:0/3705510328 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fefc00b0d10 0x7fefc0146df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:27.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.286+0000 7fefc6a15700 1 -- 172.21.15.17:0/3705510328 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fefc00b0d10 msgr2=0x7fefc0146df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:27.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.286+0000 7fefc6a15700 1 --2- 172.21.15.17:0/3705510328 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fefc00b0d10 0x7fefc0146df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:27.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.286+0000 7fefc6a15700 1 -- 172.21.15.17:0/3705510328 >> v1:172.21.15.77:6789/0 conn(0x7fefc00a9320 legacy=0x7fefc01466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:27.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.286+0000 7fefc6a15700 1 -- 172.21.15.17:0/3705510328 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc00b17a0 con 0x7fefc00b0560 2022-08-01T19:41:27.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.286+0000 7fefc6a15700 1 --2- 172.21.15.17:0/3705510328 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefc00b0560 0x7fefc0140350 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 rx=0x7fefb80032f0 tx=0x7fefb8009290).ready entity=mon.0 client_cookie=d00475a2e97fcffb server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:27.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.286+0000 7fefc5212700 1 -- 172.21.15.17:0/3705510328 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefb800b8a0 con 0x7fefc00b0560 2022-08-01T19:41:27.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.286+0000 7fefc5212700 1 -- 172.21.15.17:0/3705510328 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb800ba00 con 0x7fefc00b0560 2022-08-01T19:41:27.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.286+0000 7fefc5212700 1 -- 172.21.15.17:0/3705510328 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefb800bb70 con 0x7fefc00b0560 2022-08-01T19:41:27.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.286+0000 7fefc7216700 1 -- 172.21.15.17:0/3705510328 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefc00b0560 msgr2=0x7fefc0140350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:27.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.286+0000 7fefc7216700 1 --2- 172.21.15.17:0/3705510328 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefc00b0560 0x7fefc0140350 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 rx=0x7fefb80032f0 tx=0x7fefb8009290).stop 2022-08-01T19:41:27.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.286+0000 7fefc7216700 1 -- 172.21.15.17:0/3705510328 shutdown_connections 2022-08-01T19:41:27.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.286+0000 7fefc7216700 1 --2- 172.21.15.17:0/3705510328 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefc00b0560 0x7fefc0140350 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:27.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.286+0000 7fefc7216700 1 --2- 172.21.15.17:0/3705510328 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fefc00b0d10 0x7fefc0146df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:27.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.286+0000 7fefc7216700 1 -- 172.21.15.17:0/3705510328 >> 172.21.15.17:0/3705510328 conn(0x7fefc00612d0 msgr2=0x7fefc0062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:27.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.286+0000 7fefc7216700 1 -- 172.21.15.17:0/3705510328 shutdown_connections 2022-08-01T19:41:27.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.286+0000 7fefc7216700 1 -- 172.21.15.17:0/3705510328 wait complete. 2022-08-01T19:41:27.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.290+0000 7fefc7216700 1 Processor -- start 2022-08-01T19:41:27.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.290+0000 7fefc7216700 1 -- start start 2022-08-01T19:41:27.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.290+0000 7fefc7216700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefc00a9320 0x7fefc0144850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:27.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.290+0000 7fefc7216700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fefc00b0560 0x7fefc0152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:27.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.290+0000 7fefc7216700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefc00b0d10 0x7fefc0152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:27.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.290+0000 7fefc7216700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc0153040 con 0x7fefc00a9320 2022-08-01T19:41:27.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.290+0000 7fefc7216700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc0142dc0 con 0x7fefc00b0d10 2022-08-01T19:41:27.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.290+0000 7fefc7216700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefc0155cd0 con 0x7fefc00b0560 2022-08-01T19:41:27.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.290+0000 7fefc6214700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefc00a9320 0x7fefc0144850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:27.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.290+0000 7fefc6214700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefc00a9320 0x7fefc0144850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42052/0 (socket says 172.21.15.17:42052) 2022-08-01T19:41:27.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.290+0000 7fefc6214700 1 -- 172.21.15.17:0/2712361967 learned_addr learned my addr 172.21.15.17:0/2712361967 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:27.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.290+0000 7fefc5a13700 1 --2- 172.21.15.17:0/2712361967 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fefc00b0560 0x7fefc0152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:27.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.290+0000 7fefc6a15700 1 --2- 172.21.15.17:0/2712361967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefc00b0d10 0x7fefc0152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:27.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.290+0000 7fefc6214700 1 -- 172.21.15.17:0/2712361967 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fefc00b0560 msgr2=0x7fefc0152660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:27.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.290+0000 7fefc6214700 1 --2- 172.21.15.17:0/2712361967 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fefc00b0560 0x7fefc0152660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:27.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.290+0000 7fefc6214700 1 -- 172.21.15.17:0/2712361967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefc00b0d10 msgr2=0x7fefc0152b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:27.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.290+0000 7fefc6214700 1 --2- 172.21.15.17:0/2712361967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefc00b0d10 0x7fefc0152b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:27.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.290+0000 7fefc6214700 1 -- 172.21.15.17:0/2712361967 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb8009530 con 0x7fefc00a9320 2022-08-01T19:41:27.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.290+0000 7fefc6214700 1 --2- 172.21.15.17:0/2712361967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefc00a9320 0x7fefc0144850 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0x7fefc00b0420 tx=0x7fefbc007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:27.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.290+0000 7fefaf7fe700 1 -- 172.21.15.17:0/2712361967 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefbc012400 con 0x7fefc00a9320 2022-08-01T19:41:27.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.290+0000 7fefaf7fe700 1 -- 172.21.15.17:0/2712361967 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefbc012560 con 0x7fefc00a9320 2022-08-01T19:41:27.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.290+0000 7fefaf7fe700 1 -- 172.21.15.17:0/2712361967 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefbc0128c0 con 0x7fefc00a9320 2022-08-01T19:41:27.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.290+0000 7fefc7216700 1 -- 172.21.15.17:0/2712361967 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefc0155f70 con 0x7fefc00a9320 2022-08-01T19:41:27.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.290+0000 7fefc7216700 1 -- 172.21.15.17:0/2712361967 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefc01564b0 con 0x7fefc00a9320 2022-08-01T19:41:27.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.290+0000 7fefc7216700 1 -- 172.21.15.17:0/2712361967 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefc0156670 con 0x7fefc00a9320 2022-08-01T19:41:27.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.294+0000 7fefaf7fe700 1 -- 172.21.15.17:0/2712361967 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fefbc019030 con 0x7fefc00a9320 2022-08-01T19:41:27.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.294+0000 7fefaf7fe700 1 --2- 172.21.15.17:0/2712361967 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fefa0053a70 0x7fefa0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:27.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.294+0000 7fefaf7fe700 1 -- 172.21.15.17:0/2712361967 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fefbc06b4c0 con 0x7fefc00a9320 2022-08-01T19:41:27.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.294+0000 7fefc5a13700 1 --2- 172.21.15.17:0/2712361967 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fefa0053a70 0x7fefa0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:27.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.294+0000 7fefc5a13700 1 --2- 172.21.15.17:0/2712361967 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fefa0053a70 0x7fefa0055f30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7fefb000b000 tx=0x7fefb0006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:27.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.302+0000 7fefaf7fe700 1 -- 172.21.15.17:0/2712361967 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fefbc0420f0 con 0x7fefc00a9320 2022-08-01T19:41:27.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.470+0000 7fefc7216700 1 -- 172.21.15.17:0/2712361967 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fefc00574e0 con 0x7fefc00a9320 2022-08-01T19:41:27.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.478+0000 7fefaf7fe700 1 -- 172.21.15.17:0/2712361967 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v28) v1 ==== 184+0+0 (secure 0 0 0) 0x7fefbc045710 con 0x7fefc00a9320 2022-08-01T19:41:27.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.478+0000 7fefc7216700 1 -- 172.21.15.17:0/2712361967 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fefa0053a70 msgr2=0x7fefa0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:27.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.478+0000 7fefc7216700 1 --2- 172.21.15.17:0/2712361967 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fefa0053a70 0x7fefa0055f30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7fefb000b000 tx=0x7fefb0006df0).stop 2022-08-01T19:41:27.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.478+0000 7fefc7216700 1 -- 172.21.15.17:0/2712361967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefc00a9320 msgr2=0x7fefc0144850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:27.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.478+0000 7fefc7216700 1 --2- 172.21.15.17:0/2712361967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefc00a9320 0x7fefc0144850 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0x7fefc00b0420 tx=0x7fefbc007c50).stop 2022-08-01T19:41:27.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.478+0000 7fefc7216700 1 -- 172.21.15.17:0/2712361967 shutdown_connections 2022-08-01T19:41:27.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.478+0000 7fefc7216700 1 --2- 172.21.15.17:0/2712361967 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fefa0053a70 0x7fefa0055f30 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:27.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.478+0000 7fefc7216700 1 --2- 172.21.15.17:0/2712361967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefc00b0d10 0x7fefc0152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:27.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.478+0000 7fefc7216700 1 --2- 172.21.15.17:0/2712361967 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fefc00b0560 0x7fefc0152660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:27.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.478+0000 7fefc7216700 1 --2- 172.21.15.17:0/2712361967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefc00a9320 0x7fefc0144850 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:27.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.478+0000 7fefc7216700 1 -- 172.21.15.17:0/2712361967 >> 172.21.15.17:0/2712361967 conn(0x7fefc00612d0 msgr2=0x7fefc013df20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:27.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.478+0000 7fefc7216700 1 -- 172.21.15.17:0/2712361967 shutdown_connections 2022-08-01T19:41:27.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.478+0000 7fefc7216700 1 -- 172.21.15.17:0/2712361967 wait complete. 2022-08-01T19:41:27.487 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.1 2022-08-01T19:41:27.496 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:41:27.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.602+0000 7f13ad6b4700 1 Processor -- start 2022-08-01T19:41:27.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.602+0000 7f13ad6b4700 1 -- start start 2022-08-01T19:41:27.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.602+0000 7f13ad6b4700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f13a80aa2b0 0x7f13a80ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:27.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.602+0000 7f13ad6b4700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f13a80b0a10 0x7f13a8140730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:27.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.602+0000 7f13ad6b4700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13a80b2c80 con 0x7f13a80b0630 2022-08-01T19:41:27.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.602+0000 7f13ad6b4700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13a80b2900 con 0x7f13a80b0a10 2022-08-01T19:41:27.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.602+0000 7f13ad6b4700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a80b2eb0 con 0x7f13a80aa2b0 2022-08-01T19:41:27.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.602+0000 7f13a77fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f13a80b0a10 0x7f13a8140730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:27.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.602+0000 7f13a77fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f13a80b0a10 0x7f13a8140730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42064/0 (socket says 172.21.15.17:42064) 2022-08-01T19:41:27.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.602+0000 7f13a77fe700 1 -- 172.21.15.17:0/1356548174 learned_addr learned my addr 172.21.15.17:0/1356548174 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:27.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.602+0000 7f13a67fc700 1 --2- 172.21.15.17:0/1356548174 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f13a80aa2b0 0x7f13a80ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:27.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.602+0000 7f13a77fe700 1 -- 172.21.15.17:0/1356548174 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f13a80aa2b0 msgr2=0x7f13a80ae5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:27.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.602+0000 7f13a77fe700 1 --2- 172.21.15.17:0/1356548174 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f13a80aa2b0 0x7f13a80ae5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:27.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.602+0000 7f13a77fe700 1 -- 172.21.15.17:0/1356548174 >> v1:172.21.15.77:6789/0 conn(0x7f13a80b0630 legacy=0x7f13a80adec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:27.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.602+0000 7f13a77fe700 1 -- 172.21.15.17:0/1356548174 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a80a9d00 con 0x7f13a80b0a10 2022-08-01T19:41:27.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.602+0000 7f13a77fe700 1 --2- 172.21.15.17:0/1356548174 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13a80b0a10 0x7f13a8140730 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 rx=0x7f1398002f70 tx=0x7f1398006eb0).ready entity=mon.0 client_cookie=78495475cb6c8fbd server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:27.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.602+0000 7f13a5ffb700 1 -- 172.21.15.17:0/1356548174 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f139800d570 con 0x7f13a80b0a10 2022-08-01T19:41:27.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.602+0000 7f13a5ffb700 1 -- 172.21.15.17:0/1356548174 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1398000e90 con 0x7f13a80b0a10 2022-08-01T19:41:27.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.602+0000 7f13a5ffb700 1 -- 172.21.15.17:0/1356548174 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f139800d900 con 0x7f13a80b0a10 2022-08-01T19:41:27.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.602+0000 7f13ad6b4700 1 -- 172.21.15.17:0/1356548174 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13a80b0a10 msgr2=0x7f13a8140730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:27.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.602+0000 7f13ad6b4700 1 --2- 172.21.15.17:0/1356548174 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13a80b0a10 0x7f13a8140730 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 rx=0x7f1398002f70 tx=0x7f1398006eb0).stop 2022-08-01T19:41:27.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.602+0000 7f13ad6b4700 1 -- 172.21.15.17:0/1356548174 shutdown_connections 2022-08-01T19:41:27.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.602+0000 7f13ad6b4700 1 --2- 172.21.15.17:0/1356548174 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13a80b0a10 0x7f13a8140730 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:27.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.602+0000 7f13ad6b4700 1 --2- 172.21.15.17:0/1356548174 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f13a80aa2b0 0x7f13a80ae5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:27.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.602+0000 7f13ad6b4700 1 -- 172.21.15.17:0/1356548174 >> 172.21.15.17:0/1356548174 conn(0x7f13a8060950 msgr2=0x7f13a8060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:27.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.602+0000 7f13ad6b4700 1 -- 172.21.15.17:0/1356548174 shutdown_connections 2022-08-01T19:41:27.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.606+0000 7f13ad6b4700 1 -- 172.21.15.17:0/1356548174 wait complete. 2022-08-01T19:41:27.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.606+0000 7f13ad6b4700 1 Processor -- start 2022-08-01T19:41:27.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.606+0000 7f13ad6b4700 1 -- start start 2022-08-01T19:41:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.606+0000 7f13ad6b4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13a80aa2b0 0x7f13a8135d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.606+0000 7f13ad6b4700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f13a80b0630 0x7f13a80641a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.606+0000 7f13ad6b4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13a80b0a10 0x7f13a8064690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.606+0000 7f13ad6b4700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a8067660 con 0x7f13a80b0a10 2022-08-01T19:41:27.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.606+0000 7f13ad6b4700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a8067900 con 0x7f13a80aa2b0 2022-08-01T19:41:27.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.606+0000 7f13ad6b4700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a8063a60 con 0x7f13a80b0630 2022-08-01T19:41:27.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.606+0000 7f13a77fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13a80b0a10 0x7f13a8064690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:27.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.606+0000 7f13a77fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13a80b0a10 0x7f13a8064690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42070/0 (socket says 172.21.15.17:42070) 2022-08-01T19:41:27.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.606+0000 7f13a77fe700 1 -- 172.21.15.17:0/1094957872 learned_addr learned my addr 172.21.15.17:0/1094957872 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:27.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.606+0000 7f13a6ffd700 1 --2- 172.21.15.17:0/1094957872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13a80aa2b0 0x7f13a8135d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:27.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.606+0000 7f13a67fc700 1 --2- 172.21.15.17:0/1094957872 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f13a80b0630 0x7f13a80641a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:27.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.606+0000 7f13a77fe700 1 -- 172.21.15.17:0/1094957872 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f13a80b0630 msgr2=0x7f13a80641a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:27.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.606+0000 7f13a77fe700 1 --2- 172.21.15.17:0/1094957872 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f13a80b0630 0x7f13a80641a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:27.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.606+0000 7f13a77fe700 1 -- 172.21.15.17:0/1094957872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13a80aa2b0 msgr2=0x7f13a8135d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:27.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.606+0000 7f13a77fe700 1 --2- 172.21.15.17:0/1094957872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13a80aa2b0 0x7f13a8135d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:27.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.606+0000 7f13a77fe700 1 -- 172.21.15.17:0/1094957872 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f139800b1e0 con 0x7f13a80b0a10 2022-08-01T19:41:27.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.606+0000 7f13a77fe700 1 --2- 172.21.15.17:0/1094957872 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13a80b0a10 0x7f13a8064690 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7f1398002f70 tx=0x7f1398015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:27.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.610+0000 7f139ffff700 1 -- 172.21.15.17:0/1094957872 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13980116e0 con 0x7f13a80b0a10 2022-08-01T19:41:27.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.610+0000 7f139ffff700 1 -- 172.21.15.17:0/1094957872 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1398011840 con 0x7f13a80b0a10 2022-08-01T19:41:27.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.610+0000 7f13ad6b4700 1 -- 172.21.15.17:0/1094957872 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13a8063d00 con 0x7f13a80b0a10 2022-08-01T19:41:27.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.610+0000 7f139ffff700 1 -- 172.21.15.17:0/1094957872 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1398011ba0 con 0x7f13a80b0a10 2022-08-01T19:41:27.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.610+0000 7f13ad6b4700 1 -- 172.21.15.17:0/1094957872 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13a80ac910 con 0x7f13a80b0a10 2022-08-01T19:41:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.610+0000 7f13ad6b4700 1 -- 172.21.15.17:0/1094957872 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13a8063e90 con 0x7f13a80b0a10 2022-08-01T19:41:27.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.610+0000 7f139ffff700 1 -- 172.21.15.17:0/1094957872 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1398025030 con 0x7f13a80b0a10 2022-08-01T19:41:27.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.610+0000 7f139ffff700 1 --2- 172.21.15.17:0/1094957872 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f138c053950 0x7f138c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:27.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.610+0000 7f139ffff700 1 -- 172.21.15.17:0/1094957872 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f139806d140 con 0x7f13a80b0a10 2022-08-01T19:41:27.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.610+0000 7f13a6ffd700 1 --2- 172.21.15.17:0/1094957872 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f138c053950 0x7f138c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:27.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.614+0000 7f13a6ffd700 1 --2- 172.21.15.17:0/1094957872 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f138c053950 0x7f138c055e10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f139000d000 tx=0x7f1390007b90).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:27.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.622+0000 7f139ffff700 1 -- 172.21.15.17:0/1094957872 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1398043f00 con 0x7f13a80b0a10 2022-08-01T19:41:27.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.786+0000 7f13ad6b4700 1 -- 172.21.15.17:0/1094957872 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f13a80574e0 con 0x7f13a80b0a10 2022-08-01T19:41:27.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.786+0000 7f139ffff700 1 -- 172.21.15.17:0/1094957872 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f1398023090 con 0x7f13a80b0a10 2022-08-01T19:41:27.791 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:27.791 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":46,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6943,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/256800053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":256800053},{"type":"v1","addr":"172.21.15.77:6835","nonce":256800053}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44},{"gid":6957,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3629116150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3629116150},{"type":"v1","addr":"172.21.15.17:6833","nonce":3629116150}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:41:24.477490+0000","modified":"2022-08-01T19:41:26.186850+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6960},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6960":{"gid":6960,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/2788758172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2788758172},{"type":"v1","addr":"172.21.15.17:6835","nonce":2788758172}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2022-08-01T19:41:27.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.786+0000 7f13ad6b4700 1 -- 172.21.15.17:0/1094957872 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f138c053950 msgr2=0x7f138c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:27.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.790+0000 7f13ad6b4700 1 --2- 172.21.15.17:0/1094957872 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f138c053950 0x7f138c055e10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f139000d000 tx=0x7f1390007b90).stop 2022-08-01T19:41:27.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.790+0000 7f13ad6b4700 1 -- 172.21.15.17:0/1094957872 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13a80b0a10 msgr2=0x7f13a8064690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:27.814 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.790+0000 7f13ad6b4700 1 --2- 172.21.15.17:0/1094957872 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13a80b0a10 0x7f13a8064690 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7f1398002f70 tx=0x7f1398015000).stop 2022-08-01T19:41:27.815 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.790+0000 7f13ad6b4700 1 -- 172.21.15.17:0/1094957872 shutdown_connections 2022-08-01T19:41:27.816 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.790+0000 7f13ad6b4700 1 --2- 172.21.15.17:0/1094957872 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f138c053950 0x7f138c055e10 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:27.816 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.790+0000 7f13ad6b4700 1 --2- 172.21.15.17:0/1094957872 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13a80b0a10 0x7f13a8064690 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:27.817 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.790+0000 7f13ad6b4700 1 --2- 172.21.15.17:0/1094957872 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f13a80b0630 0x7f13a80641a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:27.817 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.790+0000 7f13ad6b4700 1 --2- 172.21.15.17:0/1094957872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13a80aa2b0 0x7f13a8135d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:27.817 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.790+0000 7f13ad6b4700 1 -- 172.21.15.17:0/1094957872 >> 172.21.15.17:0/1094957872 conn(0x7f13a8060950 msgr2=0x7f13a80b5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:27.818 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.790+0000 7f13ad6b4700 1 -- 172.21.15.17:0/1094957872 shutdown_connections 2022-08-01T19:41:27.818 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:27.790+0000 7f13ad6b4700 1 -- 172.21.15.17:0/1094957872 wait complete. 2022-08-01T19:41:27.819 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 46 2022-08-01T19:41:28.130 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:41:24.477490+0000', 'modified': '2022-08-01T19:41:26.186850+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6960}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6960': {'gid': 6960, 'name': 'c', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/2788758172', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 2788758172}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 2788758172}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:41:28.131 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:41:28.131 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T19:41:28.249 INFO:teuthology.orchestra.run.smithi017.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":6960,"want_state":"up:active","state":"up:active","rank_uptime":3.7595584199999998,"mdsmap_epoch":46,"osdmap_epoch":98,"osdmap_epoch_barrier":98,"uptime":8.1274224559999997} 2022-08-01T19:41:28.251 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 6960, "mdsmap_epoch": 46, "osdmap_epoch": 98, "osdmap_epoch_barrier": 98, "rank_uptime": 3.75955842, "state": "up:active", "uptime": 8.127422456, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:41:28.252 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:41:28.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.350+0000 7f212e825700 1 Processor -- start 2022-08-01T19:41:28.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.350+0000 7f212e825700 1 -- start start 2022-08-01T19:41:28.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.350+0000 7f212e825700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f21280b28b0 0x7f212813e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:28.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.350+0000 7f212e825700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f21280b1180 0x7f212813e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:28.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.350+0000 7f212e825700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21280aac20 con 0x7f2128141210 2022-08-01T19:41:28.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.350+0000 7f212e825700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2128141db0 con 0x7f21280b1180 2022-08-01T19:41:28.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.350+0000 7f212e825700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2128140e20 con 0x7f21280b28b0 2022-08-01T19:41:28.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.350+0000 7f212d022700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f21280b1180 0x7f212813e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:28.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.350+0000 7f212e024700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2128141210 0x7f21281416a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34272/0 (socket says 172.21.15.17:34272) 2022-08-01T19:41:28.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.350+0000 7f212e024700 1 -- 172.21.15.17:0/3708601464 learned_addr learned my addr 172.21.15.17:0/3708601464 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:28.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.350+0000 7f212d823700 1 --2- 172.21.15.17:0/3708601464 >> v2:172.21.15.77:3300/0 conn(0x7f21280b28b0 0x7f212813e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:28.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.350+0000 7f212c821700 1 -- 172.21.15.17:0/3708601464 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2704866880 0 0) 0x7f2120001490 con 0x7f2128141210 2022-08-01T19:41:28.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.350+0000 7f212c821700 1 -- 172.21.15.17:0/3708601464 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2110003260 con 0x7f2128141210 2022-08-01T19:41:28.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.350+0000 7f212d022700 1 -- 172.21.15.17:0/3708601464 >> v2:172.21.15.77:3300/0 conn(0x7f21280b28b0 msgr2=0x7f212813e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:28.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.350+0000 7f212d022700 1 --2- 172.21.15.17:0/3708601464 >> v2:172.21.15.77:3300/0 conn(0x7f21280b28b0 0x7f212813e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:28.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.350+0000 7f212c821700 1 -- 172.21.15.17:0/3708601464 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 60894355 0 0) 0x7f2110003260 con 0x7f2128141210 2022-08-01T19:41:28.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.350+0000 7f212d022700 1 -- 172.21.15.17:0/3708601464 >> v1:172.21.15.17:6789/0 conn(0x7f2128141210 legacy=0x7f21281416a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:28.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.350+0000 7f212d022700 1 -- 172.21.15.17:0/3708601464 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21280b2c90 con 0x7f21280b1180 2022-08-01T19:41:28.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.354+0000 7f212d022700 1 --2- 172.21.15.17:0/3708601464 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21280b1180 0x7f212813e6b0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 rx=0x7f2120001d10 tx=0x7f2124006df0).ready entity=mon.0 client_cookie=6e84d020e8e6c626 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:28.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.354+0000 7f212c821700 1 -- 172.21.15.17:0/3708601464 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2120001b70 con 0x7f21280b1180 2022-08-01T19:41:28.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.354+0000 7f212c821700 1 -- 172.21.15.17:0/3708601464 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2124000e90 con 0x7f21280b1180 2022-08-01T19:41:28.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.354+0000 7f212c821700 1 -- 172.21.15.17:0/3708601464 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2124012460 con 0x7f21280b1180 2022-08-01T19:41:28.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.354+0000 7f212e825700 1 -- 172.21.15.17:0/3708601464 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21280b1180 msgr2=0x7f212813e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:28.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.354+0000 7f212e825700 1 --2- 172.21.15.17:0/3708601464 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21280b1180 0x7f212813e6b0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 rx=0x7f2120001d10 tx=0x7f2124006df0).stop 2022-08-01T19:41:28.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.354+0000 7f212e825700 1 -- 172.21.15.17:0/3708601464 shutdown_connections 2022-08-01T19:41:28.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.354+0000 7f212e825700 1 --2- 172.21.15.17:0/3708601464 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21280b1180 0x7f212813e6b0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:28.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.354+0000 7f212e825700 1 --2- 172.21.15.17:0/3708601464 >> v2:172.21.15.77:3300/0 conn(0x7f21280b28b0 0x7f212813e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:28.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.354+0000 7f212e825700 1 -- 172.21.15.17:0/3708601464 >> 172.21.15.17:0/3708601464 conn(0x7f2128060310 msgr2=0x7f21280b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:28.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.354+0000 7f212e825700 1 -- 172.21.15.17:0/3708601464 shutdown_connections 2022-08-01T19:41:28.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.354+0000 7f212e825700 1 -- 172.21.15.17:0/3708601464 wait complete. 2022-08-01T19:41:28.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.354+0000 7f212e825700 1 Processor -- start 2022-08-01T19:41:28.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.354+0000 7f212e825700 1 -- start start 2022-08-01T19:41:28.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.358+0000 7f212e825700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f21280b1180 0x7f2128143230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:28.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.358+0000 7f212e825700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21280b28b0 0x7f2128064570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:28.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.358+0000 7f212e825700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2128141210 0x7f21281535c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:28.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.358+0000 7f212e825700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21280639f0 con 0x7f21280b28b0 2022-08-01T19:41:28.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.358+0000 7f212e825700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2128142870 con 0x7f21280b1180 2022-08-01T19:41:28.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.358+0000 7f212e825700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2128143720 con 0x7f2128141210 2022-08-01T19:41:28.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.358+0000 7f212d022700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21280b28b0 0x7f2128064570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:28.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.358+0000 7f212d022700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21280b28b0 0x7f2128064570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42082/0 (socket says 172.21.15.17:42082) 2022-08-01T19:41:28.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.358+0000 7f212d022700 1 -- 172.21.15.17:0/1229373988 learned_addr learned my addr 172.21.15.17:0/1229373988 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:28.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.358+0000 7f212e024700 1 --2- 172.21.15.17:0/1229373988 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2128141210 0x7f21281535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:28.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.358+0000 7f212d823700 1 --2- 172.21.15.17:0/1229373988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f21280b1180 0x7f2128143230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:28.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.358+0000 7f212d022700 1 -- 172.21.15.17:0/1229373988 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2128141210 msgr2=0x7f21281535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:28.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.358+0000 7f212d022700 1 --2- 172.21.15.17:0/1229373988 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2128141210 0x7f21281535c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:28.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.358+0000 7f212d022700 1 -- 172.21.15.17:0/1229373988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f21280b1180 msgr2=0x7f2128143230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:28.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.358+0000 7f212d022700 1 --2- 172.21.15.17:0/1229373988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f21280b1180 0x7f2128143230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:28.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.358+0000 7f212d022700 1 -- 172.21.15.17:0/1229373988 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f212400b180 con 0x7f21280b28b0 2022-08-01T19:41:28.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.358+0000 7f212d022700 1 --2- 172.21.15.17:0/1229373988 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21280b28b0 0x7f2128064570 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7f2120001d10 tx=0x7f2124014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:28.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.358+0000 7f211affd700 1 -- 172.21.15.17:0/1229373988 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2124016070 con 0x7f21280b28b0 2022-08-01T19:41:28.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.358+0000 7f211affd700 1 -- 172.21.15.17:0/1229373988 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2124002f50 con 0x7f21280b28b0 2022-08-01T19:41:28.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.358+0000 7f212e825700 1 -- 172.21.15.17:0/1229373988 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2128143ca0 con 0x7f21280b28b0 2022-08-01T19:41:28.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.358+0000 7f211affd700 1 -- 172.21.15.17:0/1229373988 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2124012460 con 0x7f21280b28b0 2022-08-01T19:41:28.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.358+0000 7f212e825700 1 -- 172.21.15.17:0/1229373988 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2128150e90 con 0x7f21280b28b0 2022-08-01T19:41:28.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.362+0000 7f212e825700 1 -- 172.21.15.17:0/1229373988 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2128143e30 con 0x7f21280b28b0 2022-08-01T19:41:28.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.362+0000 7f211affd700 1 -- 172.21.15.17:0/1229373988 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2124025030 con 0x7f21280b28b0 2022-08-01T19:41:28.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.362+0000 7f211affd700 1 --2- 172.21.15.17:0/1229373988 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f21080539e0 0x7f2108055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:28.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.362+0000 7f211affd700 1 -- 172.21.15.17:0/1229373988 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f212406cf00 con 0x7f21280b28b0 2022-08-01T19:41:28.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.362+0000 7f212d823700 1 --2- 172.21.15.17:0/1229373988 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f21080539e0 0x7f2108055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:28.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.362+0000 7f212d823700 1 --2- 172.21.15.17:0/1229373988 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f21080539e0 0x7f2108055ea0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f211c00b000 tx=0x7f211c006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:28.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.370+0000 7f211affd700 1 -- 172.21.15.17:0/1229373988 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2124043bb0 con 0x7f21280b28b0 2022-08-01T19:41:28.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.494+0000 7f212e825700 1 -- 172.21.15.17:0/1229373988 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21280574e0 con 0x7f21280b28b0 2022-08-01T19:41:28.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.494+0000 7f211affd700 1 -- 172.21.15.17:0/1229373988 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f2124023070 con 0x7f21280b28b0 2022-08-01T19:41:28.498 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:28.499 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":98,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:41:25.480327+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:41:22.953305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2022-08-01T19:41:23.938052+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:41:28.501 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.498+0000 7f212e825700 1 -- 172.21.15.17:0/1229373988 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f21080539e0 msgr2=0x7f2108055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:28.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.498+0000 7f212e825700 1 --2- 172.21.15.17:0/1229373988 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f21080539e0 0x7f2108055ea0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f211c00b000 tx=0x7f211c006df0).stop 2022-08-01T19:41:28.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.498+0000 7f212e825700 1 -- 172.21.15.17:0/1229373988 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21280b28b0 msgr2=0x7f2128064570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:28.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.498+0000 7f212e825700 1 --2- 172.21.15.17:0/1229373988 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21280b28b0 0x7f2128064570 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7f2120001d10 tx=0x7f2124014000).stop 2022-08-01T19:41:28.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.498+0000 7f212e825700 1 -- 172.21.15.17:0/1229373988 shutdown_connections 2022-08-01T19:41:28.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.498+0000 7f212e825700 1 --2- 172.21.15.17:0/1229373988 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f21080539e0 0x7f2108055ea0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:28.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.498+0000 7f212e825700 1 --2- 172.21.15.17:0/1229373988 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2128141210 0x7f21281535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:28.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.498+0000 7f212e825700 1 --2- 172.21.15.17:0/1229373988 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21280b28b0 0x7f2128064570 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:28.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.498+0000 7f212e825700 1 --2- 172.21.15.17:0/1229373988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f21280b1180 0x7f2128143230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:28.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.498+0000 7f212e825700 1 -- 172.21.15.17:0/1229373988 >> 172.21.15.17:0/1229373988 conn(0x7f2128060310 msgr2=0x7f21281404e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:28.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.498+0000 7f212e825700 1 -- 172.21.15.17:0/1229373988 shutdown_connections 2022-08-01T19:41:28.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.498+0000 7f212e825700 1 -- 172.21.15.17:0/1229373988 wait complete. 2022-08-01T19:41:28.520 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T19:41:22.953305+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T19:41:23.938052+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T19:41:28.521 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:41:28.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.638+0000 7f3f28d85700 1 Processor -- start 2022-08-01T19:41:28.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.638+0000 7f3f28d85700 1 -- start start 2022-08-01T19:41:28.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.638+0000 7f3f28d85700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3f24146ae0 0x7f3f241403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:28.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.638+0000 7f3f28d85700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3f240b4820 0x7f3f240b5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:28.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.638+0000 7f3f28d85700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f24143cf0 con 0x7f3f240b4310 2022-08-01T19:41:28.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.638+0000 7f3f28d85700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f241476a0 con 0x7f3f24146ae0 2022-08-01T19:41:28.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.638+0000 7f3f28d85700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f24143f90 con 0x7f3f240b4820 2022-08-01T19:41:28.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.638+0000 7f3f2259c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3f24146ae0 0x7f3f241403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:28.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.638+0000 7f3f21d9b700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f3f240b4310 0x7f3f241408b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34284/0 (socket says 172.21.15.17:34284) 2022-08-01T19:41:28.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.638+0000 7f3f21d9b700 1 -- 172.21.15.17:0/838584318 learned_addr learned my addr 172.21.15.17:0/838584318 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:28.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.638+0000 7f3f2259c700 1 --2- 172.21.15.17:0/838584318 >> v2:172.21.15.17:3300/0 conn(0x7f3f24146ae0 0x7f3f241403c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42088/0 (socket says 172.21.15.17:42088) 2022-08-01T19:41:28.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.638+0000 7f3f22d9d700 1 --2- 172.21.15.17:0/838584318 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3f240b4820 0x7f3f240b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:28.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.638+0000 7f3f2259c700 1 -- 172.21.15.17:0/838584318 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3f240b4820 msgr2=0x7f3f240b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:28.646 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.638+0000 7f3f2259c700 1 --2- 172.21.15.17:0/838584318 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3f240b4820 0x7f3f240b5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:28.646 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.638+0000 7f3f2259c700 1 -- 172.21.15.17:0/838584318 >> v1:172.21.15.17:6789/0 conn(0x7f3f240b4310 legacy=0x7f3f241408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:28.646 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.638+0000 7f3f2259c700 1 -- 172.21.15.17:0/838584318 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f240b0e20 con 0x7f3f24146ae0 2022-08-01T19:41:28.646 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.638+0000 7f3f2259c700 1 --2- 172.21.15.17:0/838584318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f24146ae0 0x7f3f241403c0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 rx=0x7f3f14002f70 tx=0x7f3f14006df0).ready entity=mon.0 client_cookie=da30e2ee964514bd server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:28.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.638+0000 7f3f2159a700 1 -- 172.21.15.17:0/838584318 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f1400d570 con 0x7f3f24146ae0 2022-08-01T19:41:28.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.638+0000 7f3f2159a700 1 -- 172.21.15.17:0/838584318 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f14000e90 con 0x7f3f24146ae0 2022-08-01T19:41:28.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.638+0000 7f3f2159a700 1 -- 172.21.15.17:0/838584318 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f1400d8c0 con 0x7f3f24146ae0 2022-08-01T19:41:28.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.642+0000 7f3f28d85700 1 -- 172.21.15.17:0/838584318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f24146ae0 msgr2=0x7f3f241403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:28.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.642+0000 7f3f28d85700 1 --2- 172.21.15.17:0/838584318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f24146ae0 0x7f3f241403c0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 rx=0x7f3f14002f70 tx=0x7f3f14006df0).stop 2022-08-01T19:41:28.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.642+0000 7f3f28d85700 1 -- 172.21.15.17:0/838584318 shutdown_connections 2022-08-01T19:41:28.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.642+0000 7f3f28d85700 1 --2- 172.21.15.17:0/838584318 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3f240b4820 0x7f3f240b5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:28.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.642+0000 7f3f28d85700 1 --2- 172.21.15.17:0/838584318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f24146ae0 0x7f3f241403c0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:28.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.642+0000 7f3f28d85700 1 -- 172.21.15.17:0/838584318 >> 172.21.15.17:0/838584318 conn(0x7f3f2405c620 msgr2=0x7f3f2405ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:28.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.642+0000 7f3f28d85700 1 -- 172.21.15.17:0/838584318 shutdown_connections 2022-08-01T19:41:28.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.642+0000 7f3f28d85700 1 -- 172.21.15.17:0/838584318 wait complete. 2022-08-01T19:41:28.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.642+0000 7f3f28d85700 1 Processor -- start 2022-08-01T19:41:28.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.642+0000 7f3f28d85700 1 -- start start 2022-08-01T19:41:28.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.642+0000 7f3f28d85700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3f240b4310 0x7f3f240b7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:28.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.646+0000 7f3f28d85700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f240b4820 0x7f3f241524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:28.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.646+0000 7f3f28d85700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f24146ae0 0x7f3f240b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:28.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.646+0000 7f3f28d85700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f240b9070 con 0x7f3f240b4820 2022-08-01T19:41:28.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.646+0000 7f3f28d85700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f241529a0 con 0x7f3f24146ae0 2022-08-01T19:41:28.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.646+0000 7f3f28d85700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f240b9370 con 0x7f3f240b4310 2022-08-01T19:41:28.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.646+0000 7f3f21d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f240b4820 0x7f3f241524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:28.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.646+0000 7f3f21d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f240b4820 0x7f3f241524b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42098/0 (socket says 172.21.15.17:42098) 2022-08-01T19:41:28.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.646+0000 7f3f21d9b700 1 -- 172.21.15.17:0/2004354786 learned_addr learned my addr 172.21.15.17:0/2004354786 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:28.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.646+0000 7f3f22d9d700 1 --2- 172.21.15.17:0/2004354786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f24146ae0 0x7f3f240b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:28.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.646+0000 7f3f2259c700 1 --2- 172.21.15.17:0/2004354786 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3f240b4310 0x7f3f240b7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:28.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.646+0000 7f3f21d9b700 1 -- 172.21.15.17:0/2004354786 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3f240b4310 msgr2=0x7f3f240b7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:28.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.646+0000 7f3f21d9b700 1 --2- 172.21.15.17:0/2004354786 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3f240b4310 0x7f3f240b7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:28.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.646+0000 7f3f21d9b700 1 -- 172.21.15.17:0/2004354786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f24146ae0 msgr2=0x7f3f240b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:28.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.646+0000 7f3f21d9b700 1 --2- 172.21.15.17:0/2004354786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f24146ae0 0x7f3f240b85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:28.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.646+0000 7f3f21d9b700 1 -- 172.21.15.17:0/2004354786 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1400b180 con 0x7f3f240b4820 2022-08-01T19:41:28.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.646+0000 7f3f21d9b700 1 --2- 172.21.15.17:0/2004354786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f240b4820 0x7f3f241524b0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 rx=0x7f3f24060de0 tx=0x7f3f18007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:28.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.646+0000 7f3f137fe700 1 -- 172.21.15.17:0/2004354786 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f1800ce80 con 0x7f3f240b4820 2022-08-01T19:41:28.660 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.646+0000 7f3f137fe700 1 -- 172.21.15.17:0/2004354786 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f180070d0 con 0x7f3f240b4820 2022-08-01T19:41:28.660 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.646+0000 7f3f28d85700 1 -- 172.21.15.17:0/2004354786 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f240b8dd0 con 0x7f3f240b4820 2022-08-01T19:41:28.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.646+0000 7f3f137fe700 1 -- 172.21.15.17:0/2004354786 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f18007430 con 0x7f3f240b4820 2022-08-01T19:41:28.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.646+0000 7f3f28d85700 1 -- 172.21.15.17:0/2004354786 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f240b8a90 con 0x7f3f240b4820 2022-08-01T19:41:28.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.646+0000 7f3f28d85700 1 -- 172.21.15.17:0/2004354786 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f240b91b0 con 0x7f3f240b4820 2022-08-01T19:41:28.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.646+0000 7f3f137fe700 1 -- 172.21.15.17:0/2004354786 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3f18010070 con 0x7f3f240b4820 2022-08-01T19:41:28.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.650+0000 7f3f137fe700 1 --2- 172.21.15.17:0/2004354786 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3f00053a70 0x7f3f00055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:28.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.650+0000 7f3f137fe700 1 -- 172.21.15.17:0/2004354786 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f3f1806b480 con 0x7f3f240b4820 2022-08-01T19:41:28.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.650+0000 7f3f2259c700 1 --2- 172.21.15.17:0/2004354786 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3f00053a70 0x7f3f00055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:28.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.650+0000 7f3f2259c700 1 --2- 172.21.15.17:0/2004354786 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3f00053a70 0x7f3f00055f30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f3f1401a000 tx=0x7f3f14014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:28.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.654+0000 7f3f137fe700 1 -- 172.21.15.17:0/2004354786 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3f18042240 con 0x7f3f240b4820 2022-08-01T19:41:28.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.778+0000 7f3f28d85700 1 -- 172.21.15.17:0/2004354786 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f3f240574e0 con 0x7f3f240b4820 2022-08-01T19:41:28.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.778+0000 7f3f137fe700 1 -- 172.21.15.17:0/2004354786 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v98) v1 ==== 110+0+10 (secure 0 0 0) 0x7f3f18045860 con 0x7f3f240b4820 2022-08-01T19:41:28.782 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:41:28.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.778+0000 7f3f28d85700 1 -- 172.21.15.17:0/2004354786 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3f00053a70 msgr2=0x7f3f00055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:28.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.778+0000 7f3f28d85700 1 --2- 172.21.15.17:0/2004354786 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3f00053a70 0x7f3f00055f30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f3f1401a000 tx=0x7f3f14014000).stop 2022-08-01T19:41:28.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.778+0000 7f3f28d85700 1 -- 172.21.15.17:0/2004354786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f240b4820 msgr2=0x7f3f241524b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:28.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.778+0000 7f3f28d85700 1 --2- 172.21.15.17:0/2004354786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f240b4820 0x7f3f241524b0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 rx=0x7f3f24060de0 tx=0x7f3f18007c70).stop 2022-08-01T19:41:28.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.778+0000 7f3f28d85700 1 -- 172.21.15.17:0/2004354786 shutdown_connections 2022-08-01T19:41:28.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.778+0000 7f3f28d85700 1 --2- 172.21.15.17:0/2004354786 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3f00053a70 0x7f3f00055f30 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:28.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.778+0000 7f3f28d85700 1 --2- 172.21.15.17:0/2004354786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f24146ae0 0x7f3f240b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:28.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.778+0000 7f3f28d85700 1 --2- 172.21.15.17:0/2004354786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f240b4820 0x7f3f241524b0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:28.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.778+0000 7f3f28d85700 1 --2- 172.21.15.17:0/2004354786 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3f240b4310 0x7f3f240b7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:28.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.778+0000 7f3f28d85700 1 -- 172.21.15.17:0/2004354786 >> 172.21.15.17:0/2004354786 conn(0x7f3f2405c620 msgr2=0x7f3f241402b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:28.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.778+0000 7f3f28d85700 1 -- 172.21.15.17:0/2004354786 shutdown_connections 2022-08-01T19:41:28.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.782+0000 7f3f28d85700 1 -- 172.21.15.17:0/2004354786 wait complete. 2022-08-01T19:41:28.798 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:41:28.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.910+0000 7fbc47ec6700 1 Processor -- start 2022-08-01T19:41:28.912 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.910+0000 7fbc47ec6700 1 -- start start 2022-08-01T19:41:28.917 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.910+0000 7fbc47ec6700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbc40146ae0 0x7fbc401403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:28.918 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.910+0000 7fbc47ec6700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc40143cf0 con 0x7fbc400b4310 2022-08-01T19:41:28.918 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.910+0000 7fbc47ec6700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc40143f90 con 0x7fbc400b4820 2022-08-01T19:41:28.919 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.910+0000 7fbc47ec6700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc401476a0 con 0x7fbc40146ae0 2022-08-01T19:41:28.919 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.910+0000 7fbc45461700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fbc400b4310 0x7fbc401408b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34298/0 (socket says 172.21.15.17:34298) 2022-08-01T19:41:28.919 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.910+0000 7fbc45461700 1 -- 172.21.15.17:0/970073507 learned_addr learned my addr 172.21.15.17:0/970073507 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:28.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.910+0000 7fbc45c62700 1 --2- 172.21.15.17:0/970073507 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbc40146ae0 0x7fbc401403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:28.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.910+0000 7fbc44c60700 1 -- 172.21.15.17:0/970073507 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 808568212 0 0) 0x7fbc40143cf0 con 0x7fbc400b4310 2022-08-01T19:41:28.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.910+0000 7fbc44c60700 1 -- 172.21.15.17:0/970073507 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc28003260 con 0x7fbc400b4310 2022-08-01T19:41:28.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.910+0000 7fbc44c60700 1 -- 172.21.15.17:0/970073507 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3909374490 0 0) 0x7fbc40143f90 con 0x7fbc400b4820 2022-08-01T19:41:28.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.910+0000 7fbc44c60700 1 -- 172.21.15.17:0/970073507 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc40143cf0 con 0x7fbc400b4820 2022-08-01T19:41:28.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.910+0000 7fbc45c62700 1 -- 172.21.15.17:0/970073507 >> v1:172.21.15.77:6789/0 conn(0x7fbc400b4820 legacy=0x7fbc400b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:28.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.910+0000 7fbc45c62700 1 -- 172.21.15.17:0/970073507 >> v1:172.21.15.17:6789/0 conn(0x7fbc400b4310 legacy=0x7fbc401408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:28.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.910+0000 7fbc45c62700 1 -- 172.21.15.17:0/970073507 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc400b0e20 con 0x7fbc40146ae0 2022-08-01T19:41:28.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.910+0000 7fbc45c62700 1 --2- 172.21.15.17:0/970073507 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbc40146ae0 0x7fbc401403c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fbc3c002f70 tx=0x7fbc3c006df0).ready entity=mon.2 client_cookie=db6c68006691c335 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:28.924 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.914+0000 7fbc44c60700 1 -- 172.21.15.17:0/970073507 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc3c00d5c0 con 0x7fbc40146ae0 2022-08-01T19:41:28.926 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.914+0000 7fbc44c60700 1 -- 172.21.15.17:0/970073507 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc3c000e90 con 0x7fbc40146ae0 2022-08-01T19:41:28.926 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.914+0000 7fbc44c60700 1 -- 172.21.15.17:0/970073507 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc3c00d910 con 0x7fbc40146ae0 2022-08-01T19:41:28.931 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.914+0000 7fbc47ec6700 1 -- 172.21.15.17:0/970073507 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbc40146ae0 msgr2=0x7fbc401403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:28.931 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.914+0000 7fbc47ec6700 1 --2- 172.21.15.17:0/970073507 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbc40146ae0 0x7fbc401403c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fbc3c002f70 tx=0x7fbc3c006df0).stop 2022-08-01T19:41:28.931 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.914+0000 7fbc47ec6700 1 -- 172.21.15.17:0/970073507 shutdown_connections 2022-08-01T19:41:28.931 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.914+0000 7fbc47ec6700 1 --2- 172.21.15.17:0/970073507 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbc40146ae0 0x7fbc401403c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:28.932 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.914+0000 7fbc47ec6700 1 -- 172.21.15.17:0/970073507 >> 172.21.15.17:0/970073507 conn(0x7fbc4005c620 msgr2=0x7fbc4005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:28.932 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.914+0000 7fbc47ec6700 1 -- 172.21.15.17:0/970073507 shutdown_connections 2022-08-01T19:41:28.932 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.914+0000 7fbc47ec6700 1 -- 172.21.15.17:0/970073507 wait complete. 2022-08-01T19:41:28.932 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.914+0000 7fbc47ec6700 1 Processor -- start 2022-08-01T19:41:28.933 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.914+0000 7fbc47ec6700 1 -- start start 2022-08-01T19:41:28.937 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.918+0000 7fbc47ec6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc400b4310 0x7fbc400adb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:28.938 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.918+0000 7fbc47ec6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc400b4820 0x7fbc400b9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:28.938 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.918+0000 7fbc47ec6700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbc40146ae0 0x7fbc400b9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:28.938 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.918+0000 7fbc47ec6700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc400b04b0 con 0x7fbc400b4820 2022-08-01T19:41:28.938 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.918+0000 7fbc47ec6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc400b8370 con 0x7fbc400b4310 2022-08-01T19:41:28.939 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.918+0000 7fbc47ec6700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc400b8610 con 0x7fbc40146ae0 2022-08-01T19:41:28.939 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.918+0000 7fbc45461700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc400b4820 0x7fbc400b9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:28.939 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.918+0000 7fbc45461700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc400b4820 0x7fbc400b9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42110/0 (socket says 172.21.15.17:42110) 2022-08-01T19:41:28.939 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.918+0000 7fbc45461700 1 -- 172.21.15.17:0/3574547578 learned_addr learned my addr 172.21.15.17:0/3574547578 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:28.939 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.918+0000 7fbc45c62700 1 --2- 172.21.15.17:0/3574547578 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc400b4310 0x7fbc400adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:28.940 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.918+0000 7fbc46463700 1 --2- 172.21.15.17:0/3574547578 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbc40146ae0 0x7fbc400b9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:28.940 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.918+0000 7fbc45461700 1 -- 172.21.15.17:0/3574547578 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbc40146ae0 msgr2=0x7fbc400b9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:28.940 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.918+0000 7fbc45461700 1 --2- 172.21.15.17:0/3574547578 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbc40146ae0 0x7fbc400b9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:28.940 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.918+0000 7fbc45461700 1 -- 172.21.15.17:0/3574547578 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc400b4310 msgr2=0x7fbc400adb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:28.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.918+0000 7fbc45461700 1 --2- 172.21.15.17:0/3574547578 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc400b4310 0x7fbc400adb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:28.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.918+0000 7fbc45461700 1 -- 172.21.15.17:0/3574547578 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc3c00b180 con 0x7fbc400b4820 2022-08-01T19:41:28.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.918+0000 7fbc45c62700 1 --2- 172.21.15.17:0/3574547578 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc400b4310 0x7fbc400adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:28.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.918+0000 7fbc45461700 1 --2- 172.21.15.17:0/3574547578 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc400b4820 0x7fbc400b9050 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0x7fbc30001110 tx=0x7fbc30007af0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:28.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.918+0000 7fbc2effd700 1 -- 172.21.15.17:0/3574547578 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc300072e0 con 0x7fbc400b4820 2022-08-01T19:41:28.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.918+0000 7fbc47ec6700 1 -- 172.21.15.17:0/3574547578 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc400b88b0 con 0x7fbc400b4820 2022-08-01T19:41:28.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.918+0000 7fbc2effd700 1 -- 172.21.15.17:0/3574547578 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc30007440 con 0x7fbc400b4820 2022-08-01T19:41:28.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.918+0000 7fbc2effd700 1 -- 172.21.15.17:0/3574547578 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc30005800 con 0x7fbc400b4820 2022-08-01T19:41:28.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.918+0000 7fbc47ec6700 1 -- 172.21.15.17:0/3574547578 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc400ae830 con 0x7fbc400b4820 2022-08-01T19:41:28.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.918+0000 7fbc47ec6700 1 -- 172.21.15.17:0/3574547578 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc400b8a40 con 0x7fbc400b4820 2022-08-01T19:41:28.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.918+0000 7fbc2effd700 1 -- 172.21.15.17:0/3574547578 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbc30010070 con 0x7fbc400b4820 2022-08-01T19:41:28.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.922+0000 7fbc2effd700 1 --2- 172.21.15.17:0/3574547578 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbc20053b00 0x7fbc20055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:28.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.922+0000 7fbc2effd700 1 -- 172.21.15.17:0/3574547578 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fbc3006bc70 con 0x7fbc400b4820 2022-08-01T19:41:28.949 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.922+0000 7fbc45c62700 1 --2- 172.21.15.17:0/3574547578 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbc20053b00 0x7fbc20055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:28.950 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.922+0000 7fbc45c62700 1 --2- 172.21.15.17:0/3574547578 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbc20053b00 0x7fbc20055fc0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7fbc3c018000 tx=0x7fbc3c014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:28.950 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:28.926+0000 7fbc2effd700 1 -- 172.21.15.17:0/3574547578 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbc30042a30 con 0x7fbc400b4820 2022-08-01T19:41:29.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.054+0000 7fbc47ec6700 1 -- 172.21.15.17:0/3574547578 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbc400574e0 con 0x7fbc400b4820 2022-08-01T19:41:29.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.054+0000 7fbc2effd700 1 -- 172.21.15.17:0/3574547578 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v98) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbc30046050 con 0x7fbc400b4820 2022-08-01T19:41:29.056 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:41:29.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.054+0000 7fbc47ec6700 1 -- 172.21.15.17:0/3574547578 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbc20053b00 msgr2=0x7fbc20055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:29.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.054+0000 7fbc47ec6700 1 --2- 172.21.15.17:0/3574547578 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbc20053b00 0x7fbc20055fc0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7fbc3c018000 tx=0x7fbc3c014000).stop 2022-08-01T19:41:29.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.054+0000 7fbc47ec6700 1 -- 172.21.15.17:0/3574547578 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc400b4820 msgr2=0x7fbc400b9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:29.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.054+0000 7fbc47ec6700 1 --2- 172.21.15.17:0/3574547578 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc400b4820 0x7fbc400b9050 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0x7fbc30001110 tx=0x7fbc30007af0).stop 2022-08-01T19:41:29.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.054+0000 7fbc47ec6700 1 -- 172.21.15.17:0/3574547578 shutdown_connections 2022-08-01T19:41:29.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.054+0000 7fbc47ec6700 1 --2- 172.21.15.17:0/3574547578 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbc20053b00 0x7fbc20055fc0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:29.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.054+0000 7fbc47ec6700 1 --2- 172.21.15.17:0/3574547578 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbc40146ae0 0x7fbc400b9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:29.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.054+0000 7fbc47ec6700 1 --2- 172.21.15.17:0/3574547578 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc400b4820 0x7fbc400b9050 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:29.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.054+0000 7fbc47ec6700 1 --2- 172.21.15.17:0/3574547578 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc400b4310 0x7fbc400adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:29.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.054+0000 7fbc47ec6700 1 -- 172.21.15.17:0/3574547578 >> 172.21.15.17:0/3574547578 conn(0x7fbc4005c620 msgr2=0x7fbc40144a60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:29.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.054+0000 7fbc47ec6700 1 -- 172.21.15.17:0/3574547578 shutdown_connections 2022-08-01T19:41:29.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.054+0000 7fbc47ec6700 1 -- 172.21.15.17:0/3574547578 wait complete. 2022-08-01T19:41:29.084 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T19:41:29.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.202+0000 7f5781e5f700 1 Processor -- start 2022-08-01T19:41:29.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.202+0000 7f5781e5f700 1 -- start start 2022-08-01T19:41:29.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.202+0000 7f5781e5f700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f577c146ae0 0x7f577c1403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:29.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.202+0000 7f5781e5f700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f577c0b4310 0x7f577c1408b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:29.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.202+0000 7f5781e5f700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f577c0b4820 0x7f577c0b5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:29.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.202+0000 7f5781e5f700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f577c1476a0 con 0x7f577c146ae0 2022-08-01T19:41:29.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.202+0000 7f5781e5f700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f577c143cf0 con 0x7f577c0b4820 2022-08-01T19:41:29.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.202+0000 7f5781e5f700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f577c143f90 con 0x7f577c0b4310 2022-08-01T19:41:29.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.202+0000 7f577b7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f577c146ae0 0x7f577c1403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:29.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.202+0000 7f577b7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f577c146ae0 0x7f577c1403c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42116/0 (socket says 172.21.15.17:42116) 2022-08-01T19:41:29.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.202+0000 7f577b7fe700 1 -- 172.21.15.17:0/2317422712 learned_addr learned my addr 172.21.15.17:0/2317422712 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:29.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.202+0000 7f577bfff700 1 --2- 172.21.15.17:0/2317422712 >> v2:172.21.15.77:3300/0 conn(0x7f577c0b4820 0x7f577c0b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:29.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.202+0000 7f577affd700 1 --2- 172.21.15.17:0/2317422712 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f577c0b4310 0x7f577c1408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:29.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.202+0000 7f577b7fe700 1 -- 172.21.15.17:0/2317422712 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f577c0b4310 msgr2=0x7f577c1408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:29.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.202+0000 7f577b7fe700 1 --2- 172.21.15.17:0/2317422712 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f577c0b4310 0x7f577c1408b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:29.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.202+0000 7f577b7fe700 1 -- 172.21.15.17:0/2317422712 >> v2:172.21.15.77:3300/0 conn(0x7f577c0b4820 msgr2=0x7f577c0b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:29.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.202+0000 7f577b7fe700 1 --2- 172.21.15.17:0/2317422712 >> v2:172.21.15.77:3300/0 conn(0x7f577c0b4820 0x7f577c0b5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:29.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.202+0000 7f577b7fe700 1 -- 172.21.15.17:0/2317422712 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f577c0b0e20 con 0x7f577c146ae0 2022-08-01T19:41:29.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.202+0000 7f577bfff700 1 --2- 172.21.15.17:0/2317422712 >> v2:172.21.15.77:3300/0 conn(0x7f577c0b4820 0x7f577c0b5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:29.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.202+0000 7f577b7fe700 1 --2- 172.21.15.17:0/2317422712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f577c146ae0 0x7f577c1403c0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0x7f576c002f70 tx=0x7f576c006eb0).ready entity=mon.0 client_cookie=c34141c4d587e912 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:29.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.202+0000 7f577a7fc700 1 -- 172.21.15.17:0/2317422712 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f576c00d570 con 0x7f577c146ae0 2022-08-01T19:41:29.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.202+0000 7f577a7fc700 1 -- 172.21.15.17:0/2317422712 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f576c000e90 con 0x7f577c146ae0 2022-08-01T19:41:29.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.202+0000 7f577a7fc700 1 -- 172.21.15.17:0/2317422712 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f576c00d900 con 0x7f577c146ae0 2022-08-01T19:41:29.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.202+0000 7f5781e5f700 1 -- 172.21.15.17:0/2317422712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f577c146ae0 msgr2=0x7f577c1403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:29.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.202+0000 7f5781e5f700 1 --2- 172.21.15.17:0/2317422712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f577c146ae0 0x7f577c1403c0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0x7f576c002f70 tx=0x7f576c006eb0).stop 2022-08-01T19:41:29.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.202+0000 7f5781e5f700 1 -- 172.21.15.17:0/2317422712 shutdown_connections 2022-08-01T19:41:29.234 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.206+0000 7f5781e5f700 1 --2- 172.21.15.17:0/2317422712 >> v2:172.21.15.77:3300/0 conn(0x7f577c0b4820 0x7f577c0b5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:29.234 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.206+0000 7f5781e5f700 1 --2- 172.21.15.17:0/2317422712 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f577c0b4310 0x7f577c1408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:29.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.206+0000 7f5781e5f700 1 --2- 172.21.15.17:0/2317422712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f577c146ae0 0x7f577c1403c0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:29.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.206+0000 7f5781e5f700 1 -- 172.21.15.17:0/2317422712 >> 172.21.15.17:0/2317422712 conn(0x7f577c05c620 msgr2=0x7f577c05ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:29.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.206+0000 7f5781e5f700 1 -- 172.21.15.17:0/2317422712 shutdown_connections 2022-08-01T19:41:29.236 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.206+0000 7f5781e5f700 1 -- 172.21.15.17:0/2317422712 wait complete. 2022-08-01T19:41:29.236 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.206+0000 7f5781e5f700 1 Processor -- start 2022-08-01T19:41:29.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.206+0000 7f5781e5f700 1 -- start start 2022-08-01T19:41:29.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.206+0000 7f5781e5f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f577c0b4310 0x7f577c145200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:29.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.206+0000 7f5781e5f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f577c0b4820 0x7f577c0ad780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:29.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.206+0000 7f5781e5f700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f577c146ae0 0x7f577c0adc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:29.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.206+0000 7f5781e5f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577c153620 con 0x7f577c0b4310 2022-08-01T19:41:29.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.206+0000 7f5781e5f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577c153380 con 0x7f577c0b4820 2022-08-01T19:41:29.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.206+0000 7f5781e5f700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f577c14a3f0 con 0x7f577c146ae0 2022-08-01T19:41:29.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.206+0000 7f577b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f577c0b4310 0x7f577c145200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:29.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.206+0000 7f577affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f577c0b4820 0x7f577c0ad780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:29.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.206+0000 7f577bfff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f577c146ae0 0x7f577c0adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:29.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.206+0000 7f577b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f577c0b4310 0x7f577c145200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42122/0 (socket says 172.21.15.17:42122) 2022-08-01T19:41:29.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.206+0000 7f577b7fe700 1 -- 172.21.15.17:0/3642200408 learned_addr learned my addr 172.21.15.17:0/3642200408 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:29.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.210+0000 7f577b7fe700 1 -- 172.21.15.17:0/3642200408 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f577c146ae0 msgr2=0x7f577c0adc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:29.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.210+0000 7f577b7fe700 1 --2- 172.21.15.17:0/3642200408 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f577c146ae0 0x7f577c0adc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:29.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.210+0000 7f577b7fe700 1 -- 172.21.15.17:0/3642200408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f577c0b4820 msgr2=0x7f577c0ad780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:29.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.210+0000 7f577b7fe700 1 --2- 172.21.15.17:0/3642200408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f577c0b4820 0x7f577c0ad780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:29.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.210+0000 7f577b7fe700 1 -- 172.21.15.17:0/3642200408 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576c00b1e0 con 0x7f577c0b4310 2022-08-01T19:41:29.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.210+0000 7f577b7fe700 1 --2- 172.21.15.17:0/3642200408 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f577c0b4310 0x7f577c145200 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 rx=0x7f576c002f70 tx=0x7f576c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:29.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.210+0000 7f5778ff9700 1 -- 172.21.15.17:0/3642200408 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f576c0116e0 con 0x7f577c0b4310 2022-08-01T19:41:29.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.210+0000 7f5778ff9700 1 -- 172.21.15.17:0/3642200408 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f576c011840 con 0x7f577c0b4310 2022-08-01T19:41:29.246 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.210+0000 7f5778ff9700 1 -- 172.21.15.17:0/3642200408 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f576c011ba0 con 0x7f577c0b4310 2022-08-01T19:41:29.246 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.210+0000 7f5781e5f700 1 -- 172.21.15.17:0/3642200408 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f577c1530a0 con 0x7f577c0b4310 2022-08-01T19:41:29.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.210+0000 7f5781e5f700 1 -- 172.21.15.17:0/3642200408 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f577c14a890 con 0x7f577c0b4310 2022-08-01T19:41:29.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.210+0000 7f5781e5f700 1 -- 172.21.15.17:0/3642200408 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f577c14a5b0 con 0x7f577c0b4310 2022-08-01T19:41:29.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.214+0000 7f5778ff9700 1 -- 172.21.15.17:0/3642200408 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f576c01a040 con 0x7f577c0b4310 2022-08-01T19:41:29.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.214+0000 7f5778ff9700 1 --2- 172.21.15.17:0/3642200408 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f575c053980 0x7f575c055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:29.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.214+0000 7f5778ff9700 1 -- 172.21.15.17:0/3642200408 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f576c06cd20 con 0x7f577c0b4310 2022-08-01T19:41:29.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.214+0000 7f577affd700 1 --2- 172.21.15.17:0/3642200408 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f575c053980 0x7f575c055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:29.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.214+0000 7f577affd700 1 --2- 172.21.15.17:0/3642200408 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f575c053980 0x7f575c055e40 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f577000c000 tx=0x7f5770008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:29.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.218+0000 7f5778ff9700 1 -- 172.21.15.17:0/3642200408 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f576c043b20 con 0x7f577c0b4310 2022-08-01T19:41:29.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.342+0000 7f5781e5f700 1 -- 172.21.15.17:0/3642200408 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f577c0b17e0 con 0x7f577c0b4310 2022-08-01T19:41:29.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.346+0000 7f5778ff9700 1 -- 172.21.15.17:0/3642200408 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) v1 ==== 104+0+10 (secure 0 0 0) 0x7f576c0230a0 con 0x7f577c0b4310 2022-08-01T19:41:29.347 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:41:29.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.346+0000 7f5781e5f700 1 -- 172.21.15.17:0/3642200408 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f575c053980 msgr2=0x7f575c055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:29.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.346+0000 7f5781e5f700 1 --2- 172.21.15.17:0/3642200408 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f575c053980 0x7f575c055e40 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f577000c000 tx=0x7f5770008040).stop 2022-08-01T19:41:29.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.346+0000 7f5781e5f700 1 -- 172.21.15.17:0/3642200408 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f577c0b4310 msgr2=0x7f577c145200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:29.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.346+0000 7f5781e5f700 1 --2- 172.21.15.17:0/3642200408 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f577c0b4310 0x7f577c145200 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 rx=0x7f576c002f70 tx=0x7f576c015000).stop 2022-08-01T19:41:29.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.346+0000 7f5781e5f700 1 -- 172.21.15.17:0/3642200408 shutdown_connections 2022-08-01T19:41:29.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.346+0000 7f5781e5f700 1 --2- 172.21.15.17:0/3642200408 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f575c053980 0x7f575c055e40 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:29.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.346+0000 7f5781e5f700 1 --2- 172.21.15.17:0/3642200408 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f577c146ae0 0x7f577c0adc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:29.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.346+0000 7f5781e5f700 1 --2- 172.21.15.17:0/3642200408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f577c0b4820 0x7f577c0ad780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:29.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.346+0000 7f5781e5f700 1 --2- 172.21.15.17:0/3642200408 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f577c0b4310 0x7f577c145200 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:29.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.346+0000 7f5781e5f700 1 -- 172.21.15.17:0/3642200408 >> 172.21.15.17:0/3642200408 conn(0x7f577c05c620 msgr2=0x7f577c145ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:29.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.346+0000 7f5781e5f700 1 -- 172.21.15.17:0/3642200408 shutdown_connections 2022-08-01T19:41:29.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.346+0000 7f5781e5f700 1 -- 172.21.15.17:0/3642200408 wait complete. 2022-08-01T19:41:29.362 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T19:41:29.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.470+0000 7f01a90a7700 1 Processor -- start 2022-08-01T19:41:29.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.470+0000 7f01a90a7700 1 -- start start 2022-08-01T19:41:29.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.470+0000 7f01a90a7700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f01a40b01a0 0x7f01a40a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:29.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.470+0000 7f01a90a7700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f01a40afd60 0x7f01a40a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:29.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.470+0000 7f01a90a7700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01a40a7250 con 0x7f01a40b2710 2022-08-01T19:41:29.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.470+0000 7f01a90a7700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01a40a7480 con 0x7f01a40b01a0 2022-08-01T19:41:29.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.470+0000 7f01a90a7700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a40a7720 con 0x7f01a40afd60 2022-08-01T19:41:29.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.470+0000 7f01a37fe700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f01a40b01a0 0x7f01a40a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:29.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.470+0000 7f01a88a6700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f01a40afd60 0x7f01a40a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:29.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.470+0000 7f01a37fe700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f01a40b01a0 0x7f01a40a8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:40054/0 (socket says 172.21.15.17:40054) 2022-08-01T19:41:29.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.470+0000 7f01a37fe700 1 -- 172.21.15.17:0/2136995127 learned_addr learned my addr 172.21.15.17:0/2136995127 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:29.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.470+0000 7f01a37fe700 1 -- 172.21.15.17:0/2136995127 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f01a40afd60 msgr2=0x7f01a40a8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:29.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.470+0000 7f01a37fe700 1 --2- 172.21.15.17:0/2136995127 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f01a40afd60 0x7f01a40a8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:29.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.470+0000 7f01a37fe700 1 -- 172.21.15.17:0/2136995127 >> v1:172.21.15.77:6789/0 conn(0x7f01a40b2710 legacy=0x7f01a414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:29.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.470+0000 7f01a37fe700 1 -- 172.21.15.17:0/2136995127 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a40a79c0 con 0x7f01a40b01a0 2022-08-01T19:41:29.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.470+0000 7f01a37fe700 1 --2- 172.21.15.17:0/2136995127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01a40b01a0 0x7f01a40a8540 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f0194002f70 tx=0x7f0194006eb0).ready entity=mon.1 client_cookie=c0d059f5da45f7bb server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:29.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.470+0000 7f01a2ffd700 1 -- 172.21.15.17:0/2136995127 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f019400d570 con 0x7f01a40b01a0 2022-08-01T19:41:29.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.470+0000 7f01a2ffd700 1 -- 172.21.15.17:0/2136995127 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0194000e90 con 0x7f01a40b01a0 2022-08-01T19:41:29.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.470+0000 7f01a2ffd700 1 -- 172.21.15.17:0/2136995127 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f019400d900 con 0x7f01a40b01a0 2022-08-01T19:41:29.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.474+0000 7f01a90a7700 1 -- 172.21.15.17:0/2136995127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01a40b01a0 msgr2=0x7f01a40a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:29.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.474+0000 7f01a90a7700 1 --2- 172.21.15.17:0/2136995127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01a40b01a0 0x7f01a40a8540 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f0194002f70 tx=0x7f0194006eb0).stop 2022-08-01T19:41:29.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.474+0000 7f01a90a7700 1 -- 172.21.15.17:0/2136995127 shutdown_connections 2022-08-01T19:41:29.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.474+0000 7f01a90a7700 1 --2- 172.21.15.17:0/2136995127 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f01a40afd60 0x7f01a40a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:29.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.474+0000 7f01a90a7700 1 --2- 172.21.15.17:0/2136995127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01a40b01a0 0x7f01a40a8540 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:29.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.474+0000 7f01a90a7700 1 -- 172.21.15.17:0/2136995127 >> 172.21.15.17:0/2136995127 conn(0x7f01a4060310 msgr2=0x7f01a40b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:29.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.474+0000 7f01a90a7700 1 -- 172.21.15.17:0/2136995127 shutdown_connections 2022-08-01T19:41:29.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.474+0000 7f01a90a7700 1 -- 172.21.15.17:0/2136995127 wait complete. 2022-08-01T19:41:29.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.474+0000 7f01a90a7700 1 Processor -- start 2022-08-01T19:41:29.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.474+0000 7f01a90a7700 1 -- start start 2022-08-01T19:41:29.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.474+0000 7f01a90a7700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f01a40afd60 0x7f01a40b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:29.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.474+0000 7f01a90a7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01a40b01a0 0x7f01a4159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:29.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.474+0000 7f01a90a7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01a40b2710 0x7f01a40ab860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:29.499 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.474+0000 7f01a90a7700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a40aa750 con 0x7f01a40b2710 2022-08-01T19:41:29.499 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.474+0000 7f01a90a7700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a415a490 con 0x7f01a40b01a0 2022-08-01T19:41:29.499 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.474+0000 7f01a90a7700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a415a6c0 con 0x7f01a40afd60 2022-08-01T19:41:29.499 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.474+0000 7f01a88a6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01a40b2710 0x7f01a40ab860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:29.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.474+0000 7f01a88a6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01a40b2710 0x7f01a40ab860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42140/0 (socket says 172.21.15.17:42140) 2022-08-01T19:41:29.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.474+0000 7f01a3fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f01a40afd60 0x7f01a40b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:29.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.474+0000 7f01a88a6700 1 -- 172.21.15.17:0/4053539539 learned_addr learned my addr 172.21.15.17:0/4053539539 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:29.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.474+0000 7f01a37fe700 1 --2- 172.21.15.17:0/4053539539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01a40b01a0 0x7f01a4159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:29.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.474+0000 7f01a88a6700 1 -- 172.21.15.17:0/4053539539 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f01a40afd60 msgr2=0x7f01a40b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:29.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.474+0000 7f01a88a6700 1 --2- 172.21.15.17:0/4053539539 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f01a40afd60 0x7f01a40b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:29.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.474+0000 7f01a88a6700 1 -- 172.21.15.17:0/4053539539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01a40b01a0 msgr2=0x7f01a4159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:29.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.474+0000 7f01a88a6700 1 --2- 172.21.15.17:0/4053539539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01a40b01a0 0x7f01a4159fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:29.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.474+0000 7f01a88a6700 1 -- 172.21.15.17:0/4053539539 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f019400b1e0 con 0x7f01a40b2710 2022-08-01T19:41:29.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.478+0000 7f01a88a6700 1 --2- 172.21.15.17:0/4053539539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01a40b2710 0x7f01a40ab860 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0x7f01a40aa890 tx=0x7f019c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:29.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.478+0000 7f01a17fa700 1 -- 172.21.15.17:0/4053539539 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f019c00c640 con 0x7f01a40b2710 2022-08-01T19:41:29.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.478+0000 7f01a17fa700 1 -- 172.21.15.17:0/4053539539 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019c00c7a0 con 0x7f01a40b2710 2022-08-01T19:41:29.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.478+0000 7f01a90a7700 1 -- 172.21.15.17:0/4053539539 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a40aa2b0 con 0x7f01a40b2710 2022-08-01T19:41:29.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.478+0000 7f01a17fa700 1 -- 172.21.15.17:0/4053539539 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f019c00cb00 con 0x7f01a40b2710 2022-08-01T19:41:29.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.478+0000 7f01a90a7700 1 -- 172.21.15.17:0/4053539539 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a40abd50 con 0x7f01a40b2710 2022-08-01T19:41:29.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.478+0000 7f01a90a7700 1 -- 172.21.15.17:0/4053539539 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01a40aa440 con 0x7f01a40b2710 2022-08-01T19:41:29.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.478+0000 7f01a17fa700 1 -- 172.21.15.17:0/4053539539 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f019c022070 con 0x7f01a40b2710 2022-08-01T19:41:29.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.478+0000 7f01a17fa700 1 --2- 172.21.15.17:0/4053539539 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0184053b00 0x7f0184055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:29.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.478+0000 7f01a17fa700 1 -- 172.21.15.17:0/4053539539 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f019c06c640 con 0x7f01a40b2710 2022-08-01T19:41:29.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.478+0000 7f01a3fff700 1 --2- 172.21.15.17:0/4053539539 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0184053b00 0x7f0184055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:29.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.482+0000 7f01a3fff700 1 --2- 172.21.15.17:0/4053539539 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0184053b00 0x7f0184055fc0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f019800d000 tx=0x7f0198007b90).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:29.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.486+0000 7f01a17fa700 1 -- 172.21.15.17:0/4053539539 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f019c0433a0 con 0x7f01a40b2710 2022-08-01T19:41:29.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.606+0000 7f01a90a7700 1 -- 172.21.15.17:0/4053539539 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f01a40574e0 con 0x7f01a40b2710 2022-08-01T19:41:29.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.606+0000 7f01a17fa700 1 -- 172.21.15.17:0/4053539539 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) v1 ==== 100+0+10 (secure 0 0 0) 0x7f019c0469c0 con 0x7f01a40b2710 2022-08-01T19:41:29.613 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:41:29.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.610+0000 7f01a90a7700 1 -- 172.21.15.17:0/4053539539 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0184053b00 msgr2=0x7f0184055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:29.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.610+0000 7f01a90a7700 1 --2- 172.21.15.17:0/4053539539 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0184053b00 0x7f0184055fc0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f019800d000 tx=0x7f0198007b90).stop 2022-08-01T19:41:29.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.610+0000 7f01a90a7700 1 -- 172.21.15.17:0/4053539539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01a40b2710 msgr2=0x7f01a40ab860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:29.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.610+0000 7f01a90a7700 1 --2- 172.21.15.17:0/4053539539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01a40b2710 0x7f01a40ab860 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0x7f01a40aa890 tx=0x7f019c0091d0).stop 2022-08-01T19:41:29.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.610+0000 7f01a90a7700 1 -- 172.21.15.17:0/4053539539 shutdown_connections 2022-08-01T19:41:29.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.610+0000 7f01a90a7700 1 --2- 172.21.15.17:0/4053539539 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0184053b00 0x7f0184055fc0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:29.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.610+0000 7f01a90a7700 1 --2- 172.21.15.17:0/4053539539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01a40b2710 0x7f01a40ab860 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:29.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.610+0000 7f01a90a7700 1 --2- 172.21.15.17:0/4053539539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01a40b01a0 0x7f01a4159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:29.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.610+0000 7f01a90a7700 1 --2- 172.21.15.17:0/4053539539 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f01a40afd60 0x7f01a40b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:29.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.610+0000 7f01a90a7700 1 -- 172.21.15.17:0/4053539539 >> 172.21.15.17:0/4053539539 conn(0x7f01a4060310 msgr2=0x7f01a414ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:29.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.610+0000 7f01a90a7700 1 -- 172.21.15.17:0/4053539539 shutdown_connections 2022-08-01T19:41:29.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.610+0000 7f01a90a7700 1 -- 172.21.15.17:0/4053539539 wait complete. 2022-08-01T19:41:29.634 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T19:41:29.650 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:41:29.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.750+0000 7f6ce9b71700 1 Processor -- start 2022-08-01T19:41:29.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.754+0000 7f6ce9b71700 1 -- start start 2022-08-01T19:41:29.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.754+0000 7f6ce9b71700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6ce4148530 0x7f6ce40b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:29.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.754+0000 7f6ce9b71700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ce40b04e0 con 0x7f6ce40a7250 2022-08-01T19:41:29.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.754+0000 7f6ce9b71700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ce40b0780 con 0x7f6ce40b1f40 2022-08-01T19:41:29.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.754+0000 7f6ce9b71700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce40a9250 con 0x7f6ce4148530 2022-08-01T19:41:29.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.754+0000 7f6ce3fff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f6ce40a7250 0x7f6ce41448b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34342/0 (socket says 172.21.15.17:34342) 2022-08-01T19:41:29.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.754+0000 7f6ce3fff700 1 -- 172.21.15.17:0/989908239 learned_addr learned my addr 172.21.15.17:0/989908239 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:29.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.754+0000 7f6ce2ffd700 1 --2- 172.21.15.17:0/989908239 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6ce4148530 0x7f6ce40b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:29.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.754+0000 7f6ce27fc700 1 -- 172.21.15.17:0/989908239 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1211326879 0 0) 0x7f6ce40b04e0 con 0x7f6ce40a7250 2022-08-01T19:41:29.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.754+0000 7f6ce27fc700 1 -- 172.21.15.17:0/989908239 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cd0003260 con 0x7f6ce40a7250 2022-08-01T19:41:29.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.754+0000 7f6ce2ffd700 1 -- 172.21.15.17:0/989908239 >> v1:172.21.15.77:6789/0 conn(0x7f6ce40b1f40 legacy=0x7f6ce40b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:29.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.754+0000 7f6ce2ffd700 1 -- 172.21.15.17:0/989908239 >> v1:172.21.15.17:6789/0 conn(0x7f6ce40a7250 legacy=0x7f6ce41448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:29.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.754+0000 7f6ce2ffd700 1 -- 172.21.15.17:0/989908239 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce40b0a20 con 0x7f6ce4148530 2022-08-01T19:41:29.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.754+0000 7f6ce2ffd700 1 --2- 172.21.15.17:0/989908239 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6ce4148530 0x7f6ce40b1160 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f6cd4002f70 tx=0x7f6cd4006df0).ready entity=mon.2 client_cookie=52670d3d8d91a766 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:29.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.754+0000 7f6ce27fc700 1 -- 172.21.15.17:0/989908239 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6cd400d570 con 0x7f6ce4148530 2022-08-01T19:41:29.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.754+0000 7f6ce27fc700 1 -- 172.21.15.17:0/989908239 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd4000e90 con 0x7f6ce4148530 2022-08-01T19:41:29.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.754+0000 7f6ce27fc700 1 -- 172.21.15.17:0/989908239 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6cd400d8c0 con 0x7f6ce4148530 2022-08-01T19:41:29.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.754+0000 7f6ce9b71700 1 -- 172.21.15.17:0/989908239 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6ce4148530 msgr2=0x7f6ce40b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:29.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.754+0000 7f6ce9b71700 1 --2- 172.21.15.17:0/989908239 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6ce4148530 0x7f6ce40b1160 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f6cd4002f70 tx=0x7f6cd4006df0).stop 2022-08-01T19:41:29.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.754+0000 7f6ce9b71700 1 -- 172.21.15.17:0/989908239 shutdown_connections 2022-08-01T19:41:29.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.754+0000 7f6ce9b71700 1 --2- 172.21.15.17:0/989908239 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6ce4148530 0x7f6ce40b1160 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:29.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.754+0000 7f6ce9b71700 1 -- 172.21.15.17:0/989908239 >> 172.21.15.17:0/989908239 conn(0x7f6ce405f840 msgr2=0x7f6ce405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:29.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.754+0000 7f6ce9b71700 1 -- 172.21.15.17:0/989908239 shutdown_connections 2022-08-01T19:41:29.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.754+0000 7f6ce9b71700 1 -- 172.21.15.17:0/989908239 wait complete. 2022-08-01T19:41:29.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.758+0000 7f6ce9b71700 1 Processor -- start 2022-08-01T19:41:29.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.758+0000 7f6ce9b71700 1 -- start start 2022-08-01T19:41:29.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.758+0000 7f6ce9b71700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ce40a7250 0x7f6ce40a9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:29.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.758+0000 7f6ce9b71700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6ce40b1f40 0x7f6ce414a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:29.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.758+0000 7f6ce9b71700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ce4148530 0x7f6ce40ab6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:29.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.758+0000 7f6ce9b71700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce40ab2e0 con 0x7f6ce4148530 2022-08-01T19:41:29.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.758+0000 7f6ce9b71700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce40ac9f0 con 0x7f6ce40a7250 2022-08-01T19:41:29.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.758+0000 7f6ce9b71700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce40aeac0 con 0x7f6ce40b1f40 2022-08-01T19:41:29.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.758+0000 7f6ce3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ce4148530 0x7f6ce40ab6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:29.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.758+0000 7f6ce3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ce4148530 0x7f6ce40ab6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42154/0 (socket says 172.21.15.17:42154) 2022-08-01T19:41:29.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.758+0000 7f6ce3fff700 1 -- 172.21.15.17:0/632731258 learned_addr learned my addr 172.21.15.17:0/632731258 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:29.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.758+0000 7f6ce2ffd700 1 --2- 172.21.15.17:0/632731258 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6ce40b1f40 0x7f6ce414a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:29.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.758+0000 7f6ce37fe700 1 --2- 172.21.15.17:0/632731258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ce40a7250 0x7f6ce40a9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:29.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.758+0000 7f6ce3fff700 1 -- 172.21.15.17:0/632731258 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6ce40b1f40 msgr2=0x7f6ce414a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:29.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.758+0000 7f6ce3fff700 1 --2- 172.21.15.17:0/632731258 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6ce40b1f40 0x7f6ce414a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:29.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.758+0000 7f6ce3fff700 1 -- 172.21.15.17:0/632731258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ce40a7250 msgr2=0x7f6ce40a9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:29.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.758+0000 7f6ce3fff700 1 --2- 172.21.15.17:0/632731258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ce40a7250 0x7f6ce40a9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:29.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.758+0000 7f6ce3fff700 1 -- 172.21.15.17:0/632731258 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd400b180 con 0x7f6ce4148530 2022-08-01T19:41:29.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.758+0000 7f6ce3fff700 1 --2- 172.21.15.17:0/632731258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ce4148530 0x7f6ce40ab6d0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0x7f6cdc001490 tx=0x7f6cdc007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:29.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.758+0000 7f6ce0ff9700 1 -- 172.21.15.17:0/632731258 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6cdc005ba0 con 0x7f6ce4148530 2022-08-01T19:41:29.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.758+0000 7f6ce0ff9700 1 -- 172.21.15.17:0/632731258 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cdc005d00 con 0x7f6ce4148530 2022-08-01T19:41:29.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.758+0000 7f6ce9b71700 1 -- 172.21.15.17:0/632731258 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ce40aed60 con 0x7f6ce4148530 2022-08-01T19:41:29.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.758+0000 7f6ce0ff9700 1 -- 172.21.15.17:0/632731258 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6cdc0072c0 con 0x7f6ce4148530 2022-08-01T19:41:29.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.758+0000 7f6ce9b71700 1 -- 172.21.15.17:0/632731258 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce414b770 con 0x7f6ce4148530 2022-08-01T19:41:29.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.758+0000 7f6ce9b71700 1 -- 172.21.15.17:0/632731258 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ce40af200 con 0x7f6ce4148530 2022-08-01T19:41:29.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.762+0000 7f6ce0ff9700 1 -- 172.21.15.17:0/632731258 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6cdc010070 con 0x7f6ce4148530 2022-08-01T19:41:29.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.762+0000 7f6ce0ff9700 1 --2- 172.21.15.17:0/632731258 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6cc40539e0 0x7f6cc4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:29.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.762+0000 7f6ce0ff9700 1 -- 172.21.15.17:0/632731258 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6cdc06bc00 con 0x7f6ce4148530 2022-08-01T19:41:29.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.762+0000 7f6ce37fe700 1 --2- 172.21.15.17:0/632731258 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6cc40539e0 0x7f6cc4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:29.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.762+0000 7f6ce37fe700 1 --2- 172.21.15.17:0/632731258 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6cc40539e0 0x7f6cc4055ea0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f6cd800b000 tx=0x7f6cd8009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:29.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.766+0000 7f6ce0ff9700 1 -- 172.21.15.17:0/632731258 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6cdc042830 con 0x7f6ce4148530 2022-08-01T19:41:29.937 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.926+0000 7f6ce9b71700 1 -- 172.21.15.17:0/632731258 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ce40b0cc0 con 0x7f6ce4148530 2022-08-01T19:41:29.937 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.930+0000 7f6ce0ff9700 1 -- 172.21.15.17:0/632731258 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f6cdc045e50 con 0x7f6ce4148530 2022-08-01T19:41:29.938 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:29.938 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":46,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6943,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/256800053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":256800053},{"type":"v1","addr":"172.21.15.77:6835","nonce":256800053}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44},{"gid":6957,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3629116150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3629116150},{"type":"v1","addr":"172.21.15.17:6833","nonce":3629116150}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:41:24.477490+0000","modified":"2022-08-01T19:41:26.186850+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6960},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6960":{"gid":6960,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/2788758172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2788758172},{"type":"v1","addr":"172.21.15.17:6835","nonce":2788758172}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2022-08-01T19:41:29.948 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.930+0000 7f6ce9b71700 1 -- 172.21.15.17:0/632731258 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6cc40539e0 msgr2=0x7f6cc4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:29.949 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.930+0000 7f6ce9b71700 1 --2- 172.21.15.17:0/632731258 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6cc40539e0 0x7f6cc4055ea0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f6cd800b000 tx=0x7f6cd8009040).stop 2022-08-01T19:41:29.949 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.930+0000 7f6ce9b71700 1 -- 172.21.15.17:0/632731258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ce4148530 msgr2=0x7f6ce40ab6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:29.950 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.930+0000 7f6ce9b71700 1 --2- 172.21.15.17:0/632731258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ce4148530 0x7f6ce40ab6d0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0x7f6cdc001490 tx=0x7f6cdc007e00).stop 2022-08-01T19:41:29.951 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.930+0000 7f6ce9b71700 1 -- 172.21.15.17:0/632731258 shutdown_connections 2022-08-01T19:41:29.952 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.930+0000 7f6ce9b71700 1 --2- 172.21.15.17:0/632731258 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6cc40539e0 0x7f6cc4055ea0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.930+0000 7f6ce9b71700 1 --2- 172.21.15.17:0/632731258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ce4148530 0x7f6ce40ab6d0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.930+0000 7f6ce9b71700 1 --2- 172.21.15.17:0/632731258 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6ce40b1f40 0x7f6ce414a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:29.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.930+0000 7f6ce9b71700 1 --2- 172.21.15.17:0/632731258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ce40a7250 0x7f6ce40a9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:29.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.930+0000 7f6ce9b71700 1 -- 172.21.15.17:0/632731258 >> 172.21.15.17:0/632731258 conn(0x7f6ce405f840 msgr2=0x7f6ce4147dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:29.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.930+0000 7f6ce9b71700 1 -- 172.21.15.17:0/632731258 shutdown_connections 2022-08-01T19:41:29.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:29.930+0000 7f6ce9b71700 1 -- 172.21.15.17:0/632731258 wait complete. 2022-08-01T19:41:29.955 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 46 2022-08-01T19:41:30.259 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:41:24.477490+0000', 'modified': '2022-08-01T19:41:26.186850+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6960}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6960': {'gid': 6960, 'name': 'c', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/2788758172', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 2788758172}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 2788758172}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:41:30.260 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:41:30.260 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T19:41:30.379 INFO:teuthology.orchestra.run.smithi017.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":6960,"want_state":"up:active","state":"up:active","rank_uptime":5.8897670150000003,"mdsmap_epoch":46,"osdmap_epoch":98,"osdmap_epoch_barrier":98,"uptime":10.257613299000001} 2022-08-01T19:41:30.380 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 6960, "mdsmap_epoch": 46, "osdmap_epoch": 98, "osdmap_epoch_barrier": 98, "rank_uptime": 5.889767015, "state": "up:active", "uptime": 10.257613299, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:41:30.380 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T19:41:30.381 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-08-01T19:41:30.381 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi085.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-08-01T19:41:30.381 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:30.382 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:41:30.408 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:41:30.433 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:41:30.433 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:41:30.486 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:41:30.487 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:41:30.487 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:30.488 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:41:30.521 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T19:41:30.522 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:41:30.566 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:41:30.589 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:41:30.589 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:41:30.639 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:2022-08-01T19:41:30.631+0000 7f1454afd080 -1 init, newargv = 0x5633c1f5d880 newargc=9 2022-08-01T19:41:30.639 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:ceph-fuse[23272]: starting ceph client 2022-08-01T19:41:30.651 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:41:30.655 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:41:30.656 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:30.656 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:41:30.657 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:ceph-fuse[23272]: starting fuse 2022-08-01T19:41:30.664 INFO:teuthology.orchestra.run.smithi085.stdout:52 2022-08-01T19:41:30.664 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T19:41:30.665 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:30.665 DEBUG:teuthology.orchestra.run.smithi085:> sudo python3 -c ' 2022-08-01T19:41:30.666 DEBUG:teuthology.orchestra.run.smithi085:> import glob 2022-08-01T19:41:30.667 DEBUG:teuthology.orchestra.run.smithi085:> import re 2022-08-01T19:41:30.667 DEBUG:teuthology.orchestra.run.smithi085:> import os 2022-08-01T19:41:30.668 DEBUG:teuthology.orchestra.run.smithi085:> import subprocess 2022-08-01T19:41:30.668 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:41:30.669 DEBUG:teuthology.orchestra.run.smithi085:> def find_socket(client_name): 2022-08-01T19:41:30.669 DEBUG:teuthology.orchestra.run.smithi085:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-08-01T19:41:30.670 DEBUG:teuthology.orchestra.run.smithi085:> files = glob.glob(asok_path) 2022-08-01T19:41:30.670 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:41:30.670 DEBUG:teuthology.orchestra.run.smithi085:> # Given a non-glob path, it better be there 2022-08-01T19:41:30.671 DEBUG:teuthology.orchestra.run.smithi085:> if "*" not in asok_path: 2022-08-01T19:41:30.671 DEBUG:teuthology.orchestra.run.smithi085:> assert(len(files) == 1) 2022-08-01T19:41:30.672 DEBUG:teuthology.orchestra.run.smithi085:> return files[0] 2022-08-01T19:41:30.672 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:41:30.673 DEBUG:teuthology.orchestra.run.smithi085:> for f in files: 2022-08-01T19:41:30.673 DEBUG:teuthology.orchestra.run.smithi085:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T19:41:30.673 DEBUG:teuthology.orchestra.run.smithi085:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T19:41:30.675 DEBUG:teuthology.orchestra.run.smithi085:> return f 2022-08-01T19:41:30.675 DEBUG:teuthology.orchestra.run.smithi085:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T19:41:30.676 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:41:30.676 DEBUG:teuthology.orchestra.run.smithi085:> print(find_socket("client.0")) 2022-08-01T19:41:30.677 DEBUG:teuthology.orchestra.run.smithi085:> ' 2022-08-01T19:41:30.763 INFO:teuthology.orchestra.run.smithi085.stdout:/var/run/ceph/ceph-client.0.23272.asok 2022-08-01T19:41:30.774 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.23272.asok 2022-08-01T19:41:30.775 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:30.776 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.23272.asok status 2022-08-01T19:41:30.869 INFO:teuthology.orchestra.run.smithi085.stdout:{ 2022-08-01T19:41:30.870 INFO:teuthology.orchestra.run.smithi085.stdout: "metadata": { 2022-08-01T19:41:30.870 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T19:41:30.870 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T19:41:30.871 INFO:teuthology.orchestra.run.smithi085.stdout: "entity_id": "0", 2022-08-01T19:41:30.871 INFO:teuthology.orchestra.run.smithi085.stdout: "hostname": "smithi085", 2022-08-01T19:41:30.871 INFO:teuthology.orchestra.run.smithi085.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-08-01T19:41:30.872 INFO:teuthology.orchestra.run.smithi085.stdout: "pid": "23272", 2022-08-01T19:41:30.872 INFO:teuthology.orchestra.run.smithi085.stdout: "root": "/" 2022-08-01T19:41:30.872 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:41:30.872 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_count": 0, 2022-08-01T19:41:30.873 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_pinned_count": 0, 2022-08-01T19:41:30.873 INFO:teuthology.orchestra.run.smithi085.stdout: "id": 6293, 2022-08-01T19:41:30.873 INFO:teuthology.orchestra.run.smithi085.stdout: "inst": { 2022-08-01T19:41:30.874 INFO:teuthology.orchestra.run.smithi085.stdout: "name": { 2022-08-01T19:41:30.874 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "client", 2022-08-01T19:41:30.874 INFO:teuthology.orchestra.run.smithi085.stdout: "num": 6293 2022-08-01T19:41:30.875 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:41:30.875 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2022-08-01T19:41:30.882 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2022-08-01T19:41:30.883 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2022-08-01T19:41:30.883 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 1755377115 2022-08-01T19:41:30.883 INFO:teuthology.orchestra.run.smithi085.stdout: } 2022-08-01T19:41:30.884 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:41:30.884 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2022-08-01T19:41:30.884 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2022-08-01T19:41:30.884 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2022-08-01T19:41:30.884 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 1755377115 2022-08-01T19:41:30.885 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:41:30.887 INFO:teuthology.orchestra.run.smithi085.stdout: "inst_str": "client.6293 172.21.15.85:0/1755377115", 2022-08-01T19:41:30.888 INFO:teuthology.orchestra.run.smithi085.stdout: "addr_str": "172.21.15.85:0/1755377115", 2022-08-01T19:41:30.888 INFO:teuthology.orchestra.run.smithi085.stdout: "inode_count": 1, 2022-08-01T19:41:30.889 INFO:teuthology.orchestra.run.smithi085.stdout: "mds_epoch": 46, 2022-08-01T19:41:30.889 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch": 98, 2022-08-01T19:41:30.889 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch_barrier": 0, 2022-08-01T19:41:30.890 INFO:teuthology.orchestra.run.smithi085.stdout: "blacklisted": false, 2022-08-01T19:41:30.906 INFO:teuthology.orchestra.run.smithi085.stdout: "fs_name": "cephfs" 2022-08-01T19:41:30.907 INFO:teuthology.orchestra.run.smithi085.stdout:} 2022-08-01T19:41:30.910 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:30.911 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:41:30.912 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:41:30.956 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:41:30.957 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:41:30.958 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:30.958 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:41:30.981 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:41:31.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.090+0000 7fc733fff700 1 Processor -- start 2022-08-01T19:41:31.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.090+0000 7fc733fff700 1 -- start start 2022-08-01T19:41:31.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.090+0000 7fc733fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc734140c10 0x7fc7340b1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:31.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.090+0000 7fc733fff700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7341467f0 con 0x7fc7340b2aa0 2022-08-01T19:41:31.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.090+0000 7fc733fff700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7341476a0 con 0x7fc7340b6510 2022-08-01T19:41:31.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.090+0000 7fc733fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc734062140 con 0x7fc734140c10 2022-08-01T19:41:31.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.090+0000 7fc7327fc700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc7340b2aa0 0x7fc7340b0e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34354/0 (socket says 172.21.15.17:34354) 2022-08-01T19:41:31.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.090+0000 7fc7327fc700 1 -- 172.21.15.17:0/662088792 learned_addr learned my addr 172.21.15.17:0/662088792 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:31.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.090+0000 7fc7337fe700 1 --2- 172.21.15.17:0/662088792 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc734140c10 0x7fc7340b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:31.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.090+0000 7fc731ffb700 1 -- 172.21.15.17:0/662088792 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2366167582 0 0) 0x7fc7341467f0 con 0x7fc7340b2aa0 2022-08-01T19:41:31.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.090+0000 7fc731ffb700 1 -- 172.21.15.17:0/662088792 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc718003260 con 0x7fc7340b2aa0 2022-08-01T19:41:31.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.090+0000 7fc7337fe700 1 -- 172.21.15.17:0/662088792 >> v1:172.21.15.77:6789/0 conn(0x7fc7340b6510 legacy=0x7fc734140500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:31.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.090+0000 7fc7337fe700 1 -- 172.21.15.17:0/662088792 >> v1:172.21.15.17:6789/0 conn(0x7fc7340b2aa0 legacy=0x7fc7340b0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:31.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.090+0000 7fc731ffb700 1 -- 172.21.15.17:0/662088792 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3231094248 0 0) 0x7fc718003260 con 0x7fc7340b2aa0 2022-08-01T19:41:31.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.090+0000 7fc7337fe700 1 -- 172.21.15.17:0/662088792 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc734146a20 con 0x7fc734140c10 2022-08-01T19:41:31.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.094+0000 7fc7337fe700 1 --2- 172.21.15.17:0/662088792 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc734140c10 0x7fc7340b1570 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fc724001800 tx=0x7fc72c0091d0).ready entity=mon.2 client_cookie=7cf5d0efe8fafdcc server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:31.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.094+0000 7fc731ffb700 1 -- 172.21.15.17:0/662088792 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc724001660 con 0x7fc734140c10 2022-08-01T19:41:31.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.094+0000 7fc731ffb700 1 -- 172.21.15.17:0/662088792 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72c00b580 con 0x7fc734140c10 2022-08-01T19:41:31.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.094+0000 7fc731ffb700 1 -- 172.21.15.17:0/662088792 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc72c00b6f0 con 0x7fc734140c10 2022-08-01T19:41:31.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.094+0000 7fc733fff700 1 -- 172.21.15.17:0/662088792 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc734140c10 msgr2=0x7fc7340b1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:31.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.094+0000 7fc733fff700 1 --2- 172.21.15.17:0/662088792 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc734140c10 0x7fc7340b1570 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fc724001800 tx=0x7fc72c0091d0).stop 2022-08-01T19:41:31.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.094+0000 7fc733fff700 1 -- 172.21.15.17:0/662088792 shutdown_connections 2022-08-01T19:41:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.094+0000 7fc733fff700 1 --2- 172.21.15.17:0/662088792 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc734140c10 0x7fc7340b1570 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:31.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.094+0000 7fc733fff700 1 -- 172.21.15.17:0/662088792 >> 172.21.15.17:0/662088792 conn(0x7fc734063800 msgr2=0x7fc734063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:31.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.094+0000 7fc733fff700 1 -- 172.21.15.17:0/662088792 shutdown_connections 2022-08-01T19:41:31.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.094+0000 7fc733fff700 1 -- 172.21.15.17:0/662088792 wait complete. 2022-08-01T19:41:31.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.094+0000 7fc733fff700 1 Processor -- start 2022-08-01T19:41:31.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.094+0000 7fc733fff700 1 -- start start 2022-08-01T19:41:31.134 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.098+0000 7fc733fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7340b2aa0 0x7fc7340aa580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:31.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.098+0000 7fc733fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc7340b6510 0x7fc7341530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:31.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.098+0000 7fc733fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc734140c10 0x7fc7340ac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:31.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.098+0000 7fc733fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73413d460 con 0x7fc734140c10 2022-08-01T19:41:31.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.098+0000 7fc733fff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc734141ce0 con 0x7fc7340b2aa0 2022-08-01T19:41:31.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.098+0000 7fc733fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc734141f80 con 0x7fc7340b6510 2022-08-01T19:41:31.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.098+0000 7fc7337fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc734140c10 0x7fc7340ac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:31.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.098+0000 7fc7327fc700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc7340b6510 0x7fc7341530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:31.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.098+0000 7fc7327fc700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc7340b6510 0x7fc7341530c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:38266/0 (socket says 172.21.15.17:38266) 2022-08-01T19:41:31.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.098+0000 7fc7327fc700 1 -- 172.21.15.17:0/955258963 learned_addr learned my addr 172.21.15.17:0/955258963 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:31.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.098+0000 7fc732ffd700 1 --2- 172.21.15.17:0/955258963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7340b2aa0 0x7fc7340aa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:31.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.098+0000 7fc7337fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc734140c10 0x7fc7340ac1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42168/0 (socket says 172.21.15.17:42168) 2022-08-01T19:41:31.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.098+0000 7fc7327fc700 1 -- 172.21.15.17:0/955258963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7340b2aa0 msgr2=0x7fc7340aa580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:31.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.098+0000 7fc7327fc700 1 --2- 172.21.15.17:0/955258963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7340b2aa0 0x7fc7340aa580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:31.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.098+0000 7fc7327fc700 1 -- 172.21.15.17:0/955258963 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc734140c10 msgr2=0x7fc7340ac1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:31.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.098+0000 7fc7327fc700 1 --2- 172.21.15.17:0/955258963 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc734140c10 0x7fc7340ac1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:31.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.098+0000 7fc7327fc700 1 -- 172.21.15.17:0/955258963 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc72c009470 con 0x7fc7340b6510 2022-08-01T19:41:31.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.098+0000 7fc7327fc700 1 --2- 172.21.15.17:0/955258963 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc7340b6510 0x7fc7341530c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fc734141810 tx=0x7fc724001a90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:31.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.098+0000 7fc723fff700 1 -- 172.21.15.17:0/955258963 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc724007170 con 0x7fc7340b6510 2022-08-01T19:41:31.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.098+0000 7fc723fff700 1 -- 172.21.15.17:0/955258963 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7240072d0 con 0x7fc7340b6510 2022-08-01T19:41:31.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.098+0000 7fc733fff700 1 -- 172.21.15.17:0/955258963 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc734142220 con 0x7fc7340b6510 2022-08-01T19:41:31.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.098+0000 7fc723fff700 1 -- 172.21.15.17:0/955258963 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc724004920 con 0x7fc7340b6510 2022-08-01T19:41:31.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.098+0000 7fc733fff700 1 -- 172.21.15.17:0/955258963 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7341535b0 con 0x7fc7340b6510 2022-08-01T19:41:31.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.098+0000 7fc733fff700 1 -- 172.21.15.17:0/955258963 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc734146cc0 con 0x7fc7340b6510 2022-08-01T19:41:31.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.102+0000 7fc723fff700 1 -- 172.21.15.17:0/955258963 <== mon.2 v2:172.21.15.77:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc72401b070 con 0x7fc7340b6510 2022-08-01T19:41:31.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.102+0000 7fc723fff700 1 --2- 172.21.15.17:0/955258963 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc710053a70 0x7fc710055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:31.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.102+0000 7fc723fff700 1 -- 172.21.15.17:0/955258963 <== mon.2 v2:172.21.15.77:3301/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc72406b7d0 con 0x7fc7340b6510 2022-08-01T19:41:31.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.102+0000 7fc732ffd700 1 --2- 172.21.15.17:0/955258963 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc710053a70 0x7fc710055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:31.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.102+0000 7fc732ffd700 1 --2- 172.21.15.17:0/955258963 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc710053a70 0x7fc710055f30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7fc72800d000 tx=0x7fc728007b90).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:31.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.110+0000 7fc723fff700 1 -- 172.21.15.17:0/955258963 <== mon.2 v2:172.21.15.77:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc724042590 con 0x7fc7340b6510 2022-08-01T19:41:31.246 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.242+0000 7fc733fff700 1 -- 172.21.15.17:0/955258963 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7340574e0 con 0x7fc7340b6510 2022-08-01T19:41:31.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.246+0000 7fc723fff700 1 -- 172.21.15.17:0/955258963 <== mon.2 v2:172.21.15.77:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+16274 (secure 0 0 0) 0x7fc724045bb0 con 0x7fc7340b6510 2022-08-01T19:41:31.248 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:31.248 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":98,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:41:25.480327+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:41:22.953305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2022-08-01T19:41:23.938052+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:41:31.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.246+0000 7fc733fff700 1 -- 172.21.15.17:0/955258963 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc710053a70 msgr2=0x7fc710055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:31.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.246+0000 7fc733fff700 1 --2- 172.21.15.17:0/955258963 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc710053a70 0x7fc710055f30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7fc72800d000 tx=0x7fc728007b90).stop 2022-08-01T19:41:31.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.246+0000 7fc733fff700 1 -- 172.21.15.17:0/955258963 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc7340b6510 msgr2=0x7fc7341530c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:31.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.246+0000 7fc733fff700 1 --2- 172.21.15.17:0/955258963 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc7340b6510 0x7fc7341530c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fc734141810 tx=0x7fc724001a90).stop 2022-08-01T19:41:31.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.246+0000 7fc733fff700 1 -- 172.21.15.17:0/955258963 shutdown_connections 2022-08-01T19:41:31.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.246+0000 7fc733fff700 1 --2- 172.21.15.17:0/955258963 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc710053a70 0x7fc710055f30 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:31.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.246+0000 7fc733fff700 1 --2- 172.21.15.17:0/955258963 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc734140c10 0x7fc7340ac1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:31.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.246+0000 7fc733fff700 1 --2- 172.21.15.17:0/955258963 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc7340b6510 0x7fc7341530c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:31.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.246+0000 7fc733fff700 1 --2- 172.21.15.17:0/955258963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7340b2aa0 0x7fc7340aa580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:31.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.246+0000 7fc733fff700 1 -- 172.21.15.17:0/955258963 >> 172.21.15.17:0/955258963 conn(0x7fc734063800 msgr2=0x7fc7341456e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:31.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.246+0000 7fc733fff700 1 -- 172.21.15.17:0/955258963 shutdown_connections 2022-08-01T19:41:31.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.246+0000 7fc733fff700 1 -- 172.21.15.17:0/955258963 wait complete. 2022-08-01T19:41:31.267 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T19:41:22.953305+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T19:41:23.938052+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T19:41:31.268 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:41:31.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.378+0000 7f5dd5a97700 1 Processor -- start 2022-08-01T19:41:31.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.378+0000 7f5dd5a97700 1 -- start start 2022-08-01T19:41:31.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.378+0000 7f5dd5a97700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f5dd00614b0 0x7f5dd0061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:31.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.378+0000 7f5dd5a97700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5dd01405d0 0x7f5dd0140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:31.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.378+0000 7f5dd5a97700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dd00aee80 con 0x7f5dd00ad750 2022-08-01T19:41:31.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.378+0000 7f5dd5a97700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd00b70b0 con 0x7f5dd01405d0 2022-08-01T19:41:31.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.378+0000 7f5dd5a97700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd00af0b0 con 0x7f5dd00614b0 2022-08-01T19:41:31.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.378+0000 7f5dd5296700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5dd01405d0 0x7f5dd0140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:31.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.378+0000 7f5dd5296700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5dd01405d0 0x7f5dd0140a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42174/0 (socket says 172.21.15.17:42174) 2022-08-01T19:41:31.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.382+0000 7f5dd5296700 1 -- 172.21.15.17:0/3643938124 learned_addr learned my addr 172.21.15.17:0/3643938124 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:31.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.382+0000 7f5dd4a95700 1 --2- 172.21.15.17:0/3643938124 >> v2:172.21.15.77:3300/0 conn(0x7f5dd00614b0 0x7f5dd0061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:31.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.382+0000 7f5dd5296700 1 -- 172.21.15.17:0/3643938124 >> v2:172.21.15.77:3300/0 conn(0x7f5dd00614b0 msgr2=0x7f5dd0061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:31.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.382+0000 7f5dd5296700 1 --2- 172.21.15.17:0/3643938124 >> v2:172.21.15.77:3300/0 conn(0x7f5dd00614b0 0x7f5dd0061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:31.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.382+0000 7f5dd5296700 1 -- 172.21.15.17:0/3643938124 >> v1:172.21.15.77:6789/0 conn(0x7f5dd00ad750 legacy=0x7f5dd00b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:31.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.382+0000 7f5dd5296700 1 -- 172.21.15.17:0/3643938124 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd00af350 con 0x7f5dd01405d0 2022-08-01T19:41:31.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.382+0000 7f5dd5296700 1 --2- 172.21.15.17:0/3643938124 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dd01405d0 0x7f5dd0140a60 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 rx=0x7f5dd00aed30 tx=0x7f5dc8009290).ready entity=mon.0 client_cookie=9d2d03a377e2a7e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:31.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.382+0000 7f5dcf7fe700 1 -- 172.21.15.17:0/3643938124 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5dc800b860 con 0x7f5dd01405d0 2022-08-01T19:41:31.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.382+0000 7f5dcf7fe700 1 -- 172.21.15.17:0/3643938124 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc800b9c0 con 0x7f5dd01405d0 2022-08-01T19:41:31.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.382+0000 7f5dcf7fe700 1 -- 172.21.15.17:0/3643938124 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5dc800bb30 con 0x7f5dd01405d0 2022-08-01T19:41:31.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.382+0000 7f5dd5a97700 1 -- 172.21.15.17:0/3643938124 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dd01405d0 msgr2=0x7f5dd0140a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:31.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.382+0000 7f5dd5a97700 1 --2- 172.21.15.17:0/3643938124 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dd01405d0 0x7f5dd0140a60 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 rx=0x7f5dd00aed30 tx=0x7f5dc8009290).stop 2022-08-01T19:41:31.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.382+0000 7f5dd5a97700 1 -- 172.21.15.17:0/3643938124 shutdown_connections 2022-08-01T19:41:31.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.382+0000 7f5dd5a97700 1 --2- 172.21.15.17:0/3643938124 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dd01405d0 0x7f5dd0140a60 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:31.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.382+0000 7f5dd5a97700 1 --2- 172.21.15.17:0/3643938124 >> v2:172.21.15.77:3300/0 conn(0x7f5dd00614b0 0x7f5dd0061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:31.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.382+0000 7f5dd5a97700 1 -- 172.21.15.17:0/3643938124 >> 172.21.15.17:0/3643938124 conn(0x7f5dd0066f70 msgr2=0x7f5dd0067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:31.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.382+0000 7f5dd5a97700 1 -- 172.21.15.17:0/3643938124 shutdown_connections 2022-08-01T19:41:31.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.382+0000 7f5dd5a97700 1 -- 172.21.15.17:0/3643938124 wait complete. 2022-08-01T19:41:31.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.382+0000 7f5dd5a97700 1 Processor -- start 2022-08-01T19:41:31.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.382+0000 7f5dd5a97700 1 -- start start 2022-08-01T19:41:31.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.386+0000 7f5dd5a97700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dd00614b0 0x7f5dd00b10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:31.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.386+0000 7f5dd5a97700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5dd00ad750 0x7f5dd00b2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:31.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.386+0000 7f5dd5a97700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dd01405d0 0x7f5dd00b2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:31.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.386+0000 7f5dd5a97700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd0152870 con 0x7f5dd01405d0 2022-08-01T19:41:31.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.386+0000 7f5dd5a97700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd00b40e0 con 0x7f5dd00614b0 2022-08-01T19:41:31.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.386+0000 7f5dd5a97700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd00b3ba0 con 0x7f5dd00ad750 2022-08-01T19:41:31.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.386+0000 7f5dd5296700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dd01405d0 0x7f5dd00b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:31.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.386+0000 7f5dd5296700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dd01405d0 0x7f5dd00b2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42180/0 (socket says 172.21.15.17:42180) 2022-08-01T19:41:31.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.386+0000 7f5dd5296700 1 -- 172.21.15.17:0/2526559769 learned_addr learned my addr 172.21.15.17:0/2526559769 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:31.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.386+0000 7f5dd4a95700 1 --2- 172.21.15.17:0/2526559769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dd00614b0 0x7f5dd00b10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:31.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.386+0000 7f5dcffff700 1 --2- 172.21.15.17:0/2526559769 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5dd00ad750 0x7f5dd00b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:31.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.386+0000 7f5dd5296700 1 -- 172.21.15.17:0/2526559769 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5dd00ad750 msgr2=0x7f5dd00b2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:31.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.386+0000 7f5dd5296700 1 --2- 172.21.15.17:0/2526559769 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5dd00ad750 0x7f5dd00b2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:31.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.386+0000 7f5dd5296700 1 -- 172.21.15.17:0/2526559769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dd00614b0 msgr2=0x7f5dd00b10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:31.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.386+0000 7f5dd5296700 1 --2- 172.21.15.17:0/2526559769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dd00614b0 0x7f5dd00b10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:31.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.386+0000 7f5dd5296700 1 -- 172.21.15.17:0/2526559769 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc8009530 con 0x7f5dd01405d0 2022-08-01T19:41:31.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.386+0000 7f5dd5296700 1 --2- 172.21.15.17:0/2526559769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dd01405d0 0x7f5dd00b2880 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 rx=0x7f5dd00aed30 tx=0x7f5dc8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:31.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.386+0000 7f5dcdffb700 1 -- 172.21.15.17:0/2526559769 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5dc8021440 con 0x7f5dd01405d0 2022-08-01T19:41:31.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.386+0000 7f5dcdffb700 1 -- 172.21.15.17:0/2526559769 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc80215a0 con 0x7f5dd01405d0 2022-08-01T19:41:31.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.386+0000 7f5dcdffb700 1 -- 172.21.15.17:0/2526559769 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5dc8021900 con 0x7f5dd01405d0 2022-08-01T19:41:31.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.386+0000 7f5dd5a97700 1 -- 172.21.15.17:0/2526559769 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd00b3e40 con 0x7f5dd01405d0 2022-08-01T19:41:31.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.386+0000 7f5dd5a97700 1 -- 172.21.15.17:0/2526559769 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd00a2950 con 0x7f5dd01405d0 2022-08-01T19:41:31.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.386+0000 7f5dd5a97700 1 -- 172.21.15.17:0/2526559769 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dd00af5f0 con 0x7f5dd01405d0 2022-08-01T19:41:31.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.386+0000 7f5dcdffb700 1 -- 172.21.15.17:0/2526559769 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5dc8029070 con 0x7f5dd01405d0 2022-08-01T19:41:31.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.390+0000 7f5dcdffb700 1 --2- 172.21.15.17:0/2526559769 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5db0053a70 0x7f5db0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:31.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.390+0000 7f5dcdffb700 1 -- 172.21.15.17:0/2526559769 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5dc806d5e0 con 0x7f5dd01405d0 2022-08-01T19:41:31.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.390+0000 7f5dd4a95700 1 --2- 172.21.15.17:0/2526559769 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5db0053a70 0x7f5db0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:31.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.390+0000 7f5dd4a95700 1 --2- 172.21.15.17:0/2526559769 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5db0053a70 0x7f5db0055f30 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f5dbc00b000 tx=0x7f5dbc006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:31.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.394+0000 7f5dcdffb700 1 -- 172.21.15.17:0/2526559769 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5dc80442a0 con 0x7f5dd01405d0 2022-08-01T19:41:31.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.518+0000 7f5dd5a97700 1 -- 172.21.15.17:0/2526559769 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f5dd00b20b0 con 0x7f5dd01405d0 2022-08-01T19:41:31.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.518+0000 7f5dcdffb700 1 -- 172.21.15.17:0/2526559769 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v98) v1 ==== 110+0+10 (secure 0 0 0) 0x7f5dc8015070 con 0x7f5dd01405d0 2022-08-01T19:41:31.528 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:41:31.529 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.522+0000 7f5dd5a97700 1 -- 172.21.15.17:0/2526559769 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5db0053a70 msgr2=0x7f5db0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:31.530 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.522+0000 7f5dd5a97700 1 --2- 172.21.15.17:0/2526559769 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5db0053a70 0x7f5db0055f30 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f5dbc00b000 tx=0x7f5dbc006df0).stop 2022-08-01T19:41:31.530 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.522+0000 7f5dd5a97700 1 -- 172.21.15.17:0/2526559769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dd01405d0 msgr2=0x7f5dd00b2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:31.530 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.522+0000 7f5dd5a97700 1 --2- 172.21.15.17:0/2526559769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dd01405d0 0x7f5dd00b2880 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 rx=0x7f5dd00aed30 tx=0x7f5dc8016000).stop 2022-08-01T19:41:31.531 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.522+0000 7f5dd5a97700 1 -- 172.21.15.17:0/2526559769 shutdown_connections 2022-08-01T19:41:31.531 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.522+0000 7f5dd5a97700 1 --2- 172.21.15.17:0/2526559769 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5db0053a70 0x7f5db0055f30 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:31.531 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.522+0000 7f5dd5a97700 1 --2- 172.21.15.17:0/2526559769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dd01405d0 0x7f5dd00b2880 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:31.532 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.522+0000 7f5dd5a97700 1 --2- 172.21.15.17:0/2526559769 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5dd00ad750 0x7f5dd00b2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:31.532 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.522+0000 7f5dd5a97700 1 --2- 172.21.15.17:0/2526559769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dd00614b0 0x7f5dd00b10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:31.532 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.522+0000 7f5dd5a97700 1 -- 172.21.15.17:0/2526559769 >> 172.21.15.17:0/2526559769 conn(0x7f5dd0066f70 msgr2=0x7f5dd0143840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:31.532 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.522+0000 7f5dd5a97700 1 -- 172.21.15.17:0/2526559769 shutdown_connections 2022-08-01T19:41:31.532 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.522+0000 7f5dd5a97700 1 -- 172.21.15.17:0/2526559769 wait complete. 2022-08-01T19:41:31.534 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:41:31.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.642+0000 7f9156607700 1 Processor -- start 2022-08-01T19:41:31.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.642+0000 7f9156607700 1 -- start start 2022-08-01T19:41:31.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.642+0000 7f9156607700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f91500b28a0 0x7f91501432a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:31.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.642+0000 7f9156607700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f9150061360 0x7f9150143790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:31.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.642+0000 7f9156607700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9150064aa0 con 0x7f91500600f0 2022-08-01T19:41:31.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.642+0000 7f9156607700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9150064d40 con 0x7f9150061360 2022-08-01T19:41:31.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.642+0000 7f9156607700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9150064fe0 con 0x7f91500b28a0 2022-08-01T19:41:31.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.642+0000 7f914ffff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f91500600f0 0x7f91501405d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34380/0 (socket says 172.21.15.17:34380) 2022-08-01T19:41:31.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.642+0000 7f914ffff700 1 -- 172.21.15.17:0/3919292371 learned_addr learned my addr 172.21.15.17:0/3919292371 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:31.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.642+0000 7f914f7fe700 1 --2- 172.21.15.17:0/3919292371 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f91500b28a0 0x7f91501432a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:31.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.642+0000 7f9154ba4700 1 --2- 172.21.15.17:0/3919292371 >> v2:172.21.15.77:3300/0 conn(0x7f9150061360 0x7f9150143790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:31.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.642+0000 7f914effd700 1 -- 172.21.15.17:0/3919292371 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1015609479 0 0) 0x7f9150064aa0 con 0x7f91500600f0 2022-08-01T19:41:31.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.642+0000 7f914effd700 1 -- 172.21.15.17:0/3919292371 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9130003260 con 0x7f91500600f0 2022-08-01T19:41:31.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.646+0000 7f914effd700 1 -- 172.21.15.17:0/3919292371 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1872160259 0 0) 0x7f9130003260 con 0x7f91500600f0 2022-08-01T19:41:31.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.646+0000 7f914f7fe700 1 -- 172.21.15.17:0/3919292371 >> v2:172.21.15.77:3300/0 conn(0x7f9150061360 msgr2=0x7f9150143790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:31.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.646+0000 7f914f7fe700 1 --2- 172.21.15.17:0/3919292371 >> v2:172.21.15.77:3300/0 conn(0x7f9150061360 0x7f9150143790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:31.692 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.646+0000 7f914f7fe700 1 -- 172.21.15.17:0/3919292371 >> v1:172.21.15.17:6789/0 conn(0x7f91500600f0 legacy=0x7f91501405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:31.692 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.646+0000 7f914f7fe700 1 -- 172.21.15.17:0/3919292371 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9150065280 con 0x7f91500b28a0 2022-08-01T19:41:31.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.646+0000 7f9154ba4700 1 --2- 172.21.15.17:0/3919292371 >> v2:172.21.15.77:3300/0 conn(0x7f9150061360 0x7f9150143790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:31.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.646+0000 7f914f7fe700 1 --2- 172.21.15.17:0/3919292371 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f91500b28a0 0x7f91501432a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f9138001800 tx=0x7f9140006df0).ready entity=mon.2 client_cookie=36285c024ec09b13 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:31.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.646+0000 7f914effd700 1 -- 172.21.15.17:0/3919292371 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9138001660 con 0x7f91500b28a0 2022-08-01T19:41:31.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.646+0000 7f914effd700 1 -- 172.21.15.17:0/3919292371 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9140000e90 con 0x7f91500b28a0 2022-08-01T19:41:31.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.646+0000 7f914effd700 1 -- 172.21.15.17:0/3919292371 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9140012460 con 0x7f91500b28a0 2022-08-01T19:41:31.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.646+0000 7f9156607700 1 -- 172.21.15.17:0/3919292371 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f91500b28a0 msgr2=0x7f91501432a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:31.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.646+0000 7f9156607700 1 --2- 172.21.15.17:0/3919292371 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f91500b28a0 0x7f91501432a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f9138001800 tx=0x7f9140006df0).stop 2022-08-01T19:41:31.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.646+0000 7f9156607700 1 -- 172.21.15.17:0/3919292371 shutdown_connections 2022-08-01T19:41:31.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.646+0000 7f9156607700 1 --2- 172.21.15.17:0/3919292371 >> v2:172.21.15.77:3300/0 conn(0x7f9150061360 0x7f9150143790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:31.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.646+0000 7f9156607700 1 --2- 172.21.15.17:0/3919292371 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f91500b28a0 0x7f91501432a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:31.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.646+0000 7f9156607700 1 -- 172.21.15.17:0/3919292371 >> 172.21.15.17:0/3919292371 conn(0x7f915005d0f0 msgr2=0x7f915005f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:31.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.646+0000 7f9156607700 1 -- 172.21.15.17:0/3919292371 shutdown_connections 2022-08-01T19:41:31.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.646+0000 7f9156607700 1 -- 172.21.15.17:0/3919292371 wait complete. 2022-08-01T19:41:31.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.646+0000 7f9156607700 1 Processor -- start 2022-08-01T19:41:31.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.646+0000 7f9156607700 1 -- start start 2022-08-01T19:41:31.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.650+0000 7f9156607700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f91500600f0 0x7f91500b68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:31.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.650+0000 7f9156607700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9150061360 0x7f9150066170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:31.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.650+0000 7f9156607700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91500b28a0 0x7f91501529a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:31.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.650+0000 7f9156607700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9150152310 con 0x7f9150061360 2022-08-01T19:41:31.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.650+0000 7f9156607700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91501455d0 con 0x7f91500b28a0 2022-08-01T19:41:31.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.650+0000 7f9156607700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9150145800 con 0x7f91500600f0 2022-08-01T19:41:31.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.650+0000 7f914f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9150061360 0x7f9150066170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:31.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.650+0000 7f914f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9150061360 0x7f9150066170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42194/0 (socket says 172.21.15.17:42194) 2022-08-01T19:41:31.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.650+0000 7f914f7fe700 1 -- 172.21.15.17:0/3219453237 learned_addr learned my addr 172.21.15.17:0/3219453237 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:31.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.650+0000 7f9154ba4700 1 --2- 172.21.15.17:0/3219453237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91500b28a0 0x7f91501529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:31.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.650+0000 7f914ffff700 1 --2- 172.21.15.17:0/3219453237 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f91500600f0 0x7f91500b68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:31.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.650+0000 7f914f7fe700 1 -- 172.21.15.17:0/3219453237 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f91500600f0 msgr2=0x7f91500b68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:31.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.650+0000 7f914f7fe700 1 --2- 172.21.15.17:0/3219453237 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f91500600f0 0x7f91500b68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:31.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.650+0000 7f914f7fe700 1 -- 172.21.15.17:0/3219453237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91500b28a0 msgr2=0x7f91501529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:31.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.650+0000 7f914f7fe700 1 --2- 172.21.15.17:0/3219453237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91500b28a0 0x7f91501529a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:31.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.650+0000 7f914f7fe700 1 -- 172.21.15.17:0/3219453237 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9148009470 con 0x7f9150061360 2022-08-01T19:41:31.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.650+0000 7f9154ba4700 1 --2- 172.21.15.17:0/3219453237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91500b28a0 0x7f91501529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:31.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.650+0000 7f914ffff700 1 --2- 172.21.15.17:0/3219453237 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f91500600f0 0x7f91500b68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:31.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.650+0000 7f914f7fe700 1 --2- 172.21.15.17:0/3219453237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9150061360 0x7f9150066170 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 rx=0x7f9150144e70 tx=0x7f9140014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:31.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.650+0000 7f914d7fa700 1 -- 172.21.15.17:0/3219453237 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9140016070 con 0x7f9150061360 2022-08-01T19:41:31.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.650+0000 7f914d7fa700 1 -- 172.21.15.17:0/3219453237 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9140013e40 con 0x7f9150061360 2022-08-01T19:41:31.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.650+0000 7f914d7fa700 1 -- 172.21.15.17:0/3219453237 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f914000de20 con 0x7f9150061360 2022-08-01T19:41:31.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.650+0000 7f9156607700 1 -- 172.21.15.17:0/3219453237 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f914000b180 con 0x7f9150061360 2022-08-01T19:41:31.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.650+0000 7f9156607700 1 -- 172.21.15.17:0/3219453237 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9150152500 con 0x7f9150061360 2022-08-01T19:41:31.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.650+0000 7f914d7fa700 1 -- 172.21.15.17:0/3219453237 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f914000f030 con 0x7f9150061360 2022-08-01T19:41:31.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.650+0000 7f9156607700 1 -- 172.21.15.17:0/3219453237 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9150145ca0 con 0x7f9150061360 2022-08-01T19:41:31.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.650+0000 7f914d7fa700 1 --2- 172.21.15.17:0/3219453237 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9134053a70 0x7f9134055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:31.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.654+0000 7f914d7fa700 1 -- 172.21.15.17:0/3219453237 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f914006cf60 con 0x7f9150061360 2022-08-01T19:41:31.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.654+0000 7f914ffff700 1 --2- 172.21.15.17:0/3219453237 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9134053a70 0x7f9134055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:31.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.654+0000 7f914ffff700 1 --2- 172.21.15.17:0/3219453237 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9134053a70 0x7f9134055f30 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f913800b000 tx=0x7f9138001d00).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:31.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.658+0000 7f914d7fa700 1 -- 172.21.15.17:0/3219453237 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9140043c10 con 0x7f9150061360 2022-08-01T19:41:31.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.782+0000 7f9156607700 1 -- 172.21.15.17:0/3219453237 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f91500604d0 con 0x7f9150061360 2022-08-01T19:41:31.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.782+0000 7f914d7fa700 1 -- 172.21.15.17:0/3219453237 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v98) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9140023070 con 0x7f9150061360 2022-08-01T19:41:31.791 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:41:31.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.786+0000 7f9156607700 1 -- 172.21.15.17:0/3219453237 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9134053a70 msgr2=0x7f9134055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:31.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.786+0000 7f9156607700 1 --2- 172.21.15.17:0/3219453237 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9134053a70 0x7f9134055f30 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f913800b000 tx=0x7f9138001d00).stop 2022-08-01T19:41:31.810 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.786+0000 7f9156607700 1 -- 172.21.15.17:0/3219453237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9150061360 msgr2=0x7f9150066170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:31.810 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.786+0000 7f9156607700 1 --2- 172.21.15.17:0/3219453237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9150061360 0x7f9150066170 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 rx=0x7f9150144e70 tx=0x7f9140014000).stop 2022-08-01T19:41:31.811 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.786+0000 7f9156607700 1 -- 172.21.15.17:0/3219453237 shutdown_connections 2022-08-01T19:41:31.811 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.786+0000 7f9156607700 1 --2- 172.21.15.17:0/3219453237 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9134053a70 0x7f9134055f30 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:31.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.786+0000 7f9156607700 1 --2- 172.21.15.17:0/3219453237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f91500b28a0 0x7f91501529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:31.814 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.786+0000 7f9156607700 1 --2- 172.21.15.17:0/3219453237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9150061360 0x7f9150066170 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:31.815 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.786+0000 7f9156607700 1 --2- 172.21.15.17:0/3219453237 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f91500600f0 0x7f91500b68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:31.815 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.786+0000 7f9156607700 1 -- 172.21.15.17:0/3219453237 >> 172.21.15.17:0/3219453237 conn(0x7f915005d0f0 msgr2=0x7f915013fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:31.816 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.786+0000 7f9156607700 1 -- 172.21.15.17:0/3219453237 shutdown_connections 2022-08-01T19:41:31.816 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.786+0000 7f9156607700 1 -- 172.21.15.17:0/3219453237 wait complete. 2022-08-01T19:41:31.817 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T19:41:31.947 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.930+0000 7f179e7d9700 1 Processor -- start 2022-08-01T19:41:31.948 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.930+0000 7f179e7d9700 1 -- start start 2022-08-01T19:41:31.948 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.930+0000 7f179e7d9700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f17980a8c50 0x7f1798152d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:31.948 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.930+0000 7f179e7d9700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17980a7250 con 0x7f17980b2710 2022-08-01T19:41:31.949 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.930+0000 7f179e7d9700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17980a7480 con 0x7f17980b01a0 2022-08-01T19:41:31.950 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.930+0000 7f179e7d9700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17980a7720 con 0x7f17980a8c50 2022-08-01T19:41:31.950 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.930+0000 7f179dfd8700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f17980a8c50 0x7f1798152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:31.951 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.930+0000 7f179d7d7700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f17980b2710 0x7f179814afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34394/0 (socket says 172.21.15.17:34394) 2022-08-01T19:41:31.951 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.930+0000 7f179dfd8700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f17980a8c50 0x7f1798152d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42204/0 (socket says 172.21.15.17:42204) 2022-08-01T19:41:31.951 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.930+0000 7f179d7d7700 1 -- 172.21.15.17:0/238850115 learned_addr learned my addr 172.21.15.17:0/238850115 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:31.952 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.930+0000 7f178bfff700 1 -- 172.21.15.17:0/238850115 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2421153349 0 0) 0x7f17980a7250 con 0x7f17980b2710 2022-08-01T19:41:31.952 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.930+0000 7f178bfff700 1 -- 172.21.15.17:0/238850115 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f177c003260 con 0x7f17980b2710 2022-08-01T19:41:31.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.930+0000 7f179dfd8700 1 -- 172.21.15.17:0/238850115 >> v1:172.21.15.77:6789/0 conn(0x7f17980b01a0 legacy=0x7f17980a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:31.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.930+0000 7f179dfd8700 1 -- 172.21.15.17:0/238850115 >> v1:172.21.15.17:6789/0 conn(0x7f17980b2710 legacy=0x7f179814afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:31.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.930+0000 7f179dfd8700 1 -- 172.21.15.17:0/238850115 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17980a79c0 con 0x7f17980a8c50 2022-08-01T19:41:31.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.930+0000 7f179dfd8700 1 --2- 172.21.15.17:0/238850115 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17980a8c50 0x7f1798152d70 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 rx=0x7f17900032f0 tx=0x7f17900091d0).ready entity=mon.0 client_cookie=3a5d3f4b9658ad7d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:31.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.930+0000 7f178bfff700 1 -- 172.21.15.17:0/238850115 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f179000b6c0 con 0x7f17980a8c50 2022-08-01T19:41:31.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.930+0000 7f178bfff700 1 -- 172.21.15.17:0/238850115 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f179000b820 con 0x7f17980a8c50 2022-08-01T19:41:31.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.930+0000 7f178bfff700 1 -- 172.21.15.17:0/238850115 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f179000b990 con 0x7f17980a8c50 2022-08-01T19:41:31.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.930+0000 7f179e7d9700 1 -- 172.21.15.17:0/238850115 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17980a8c50 msgr2=0x7f1798152d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:31.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.930+0000 7f179e7d9700 1 --2- 172.21.15.17:0/238850115 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17980a8c50 0x7f1798152d70 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 rx=0x7f17900032f0 tx=0x7f17900091d0).stop 2022-08-01T19:41:31.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.930+0000 7f179e7d9700 1 -- 172.21.15.17:0/238850115 shutdown_connections 2022-08-01T19:41:31.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.930+0000 7f179e7d9700 1 --2- 172.21.15.17:0/238850115 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17980a8c50 0x7f1798152d70 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:31.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.930+0000 7f179e7d9700 1 -- 172.21.15.17:0/238850115 >> 172.21.15.17:0/238850115 conn(0x7f1798060310 msgr2=0x7f17980b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:31.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.930+0000 7f179e7d9700 1 -- 172.21.15.17:0/238850115 shutdown_connections 2022-08-01T19:41:31.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.930+0000 7f179e7d9700 1 -- 172.21.15.17:0/238850115 wait complete. 2022-08-01T19:41:31.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.934+0000 7f179e7d9700 1 Processor -- start 2022-08-01T19:41:31.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.934+0000 7f179e7d9700 1 -- start start 2022-08-01T19:41:31.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.934+0000 7f179e7d9700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f17980a8c50 0x7f17980aa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:31.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.934+0000 7f179e7d9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17980b01a0 0x7f17980b4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:31.966 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.934+0000 7f179e7d9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17980b2710 0x7f17980b5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:31.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.934+0000 7f179e7d9700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17980b4100 con 0x7f17980b2710 2022-08-01T19:41:31.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.934+0000 7f179e7d9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17980b3200 con 0x7f17980b01a0 2022-08-01T19:41:31.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.934+0000 7f179e7d9700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17980b34a0 con 0x7f17980a8c50 2022-08-01T19:41:31.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.934+0000 7f179dfd8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17980b2710 0x7f17980b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:31.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.934+0000 7f179dfd8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17980b2710 0x7f17980b5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42210/0 (socket says 172.21.15.17:42210) 2022-08-01T19:41:31.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.934+0000 7f179dfd8700 1 -- 172.21.15.17:0/3413822755 learned_addr learned my addr 172.21.15.17:0/3413822755 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:31.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.934+0000 7f179d7d7700 1 --2- 172.21.15.17:0/3413822755 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f17980a8c50 0x7f17980aa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:31.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.934+0000 7f179cfd6700 1 --2- 172.21.15.17:0/3413822755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17980b01a0 0x7f17980b4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:31.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.934+0000 7f179dfd8700 1 -- 172.21.15.17:0/3413822755 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f17980a8c50 msgr2=0x7f17980aa0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:31.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.934+0000 7f179dfd8700 1 --2- 172.21.15.17:0/3413822755 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f17980a8c50 0x7f17980aa0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:31.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.934+0000 7f179dfd8700 1 -- 172.21.15.17:0/3413822755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17980b01a0 msgr2=0x7f17980b4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:31.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.934+0000 7f179dfd8700 1 --2- 172.21.15.17:0/3413822755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17980b01a0 0x7f17980b4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:31.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.934+0000 7f179dfd8700 1 -- 172.21.15.17:0/3413822755 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1790009470 con 0x7f17980b2710 2022-08-01T19:41:31.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.934+0000 7f179dfd8700 1 --2- 172.21.15.17:0/3413822755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17980b2710 0x7f17980b5200 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 rx=0x7f17900032f0 tx=0x7f1790016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:31.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.934+0000 7f178a7fc700 1 -- 172.21.15.17:0/3413822755 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f179000b6c0 con 0x7f17980b2710 2022-08-01T19:41:31.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.934+0000 7f178a7fc700 1 -- 172.21.15.17:0/3413822755 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1790021630 con 0x7f17980b2710 2022-08-01T19:41:31.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.934+0000 7f179e7d9700 1 -- 172.21.15.17:0/3413822755 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17980b3740 con 0x7f17980b2710 2022-08-01T19:41:31.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.934+0000 7f178a7fc700 1 -- 172.21.15.17:0/3413822755 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1790021990 con 0x7f17980b2710 2022-08-01T19:41:31.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.934+0000 7f179e7d9700 1 -- 172.21.15.17:0/3413822755 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17980b56f0 con 0x7f17980b2710 2022-08-01T19:41:31.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.938+0000 7f179e7d9700 1 -- 172.21.15.17:0/3413822755 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17980574e0 con 0x7f17980b2710 2022-08-01T19:41:31.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.938+0000 7f178a7fc700 1 -- 172.21.15.17:0/3413822755 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f179000f070 con 0x7f17980b2710 2022-08-01T19:41:31.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.938+0000 7f178a7fc700 1 --2- 172.21.15.17:0/3413822755 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1780053a70 0x7f1780055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:31.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.938+0000 7f178a7fc700 1 -- 172.21.15.17:0/3413822755 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f179006d460 con 0x7f17980b2710 2022-08-01T19:41:31.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.938+0000 7f179d7d7700 1 --2- 172.21.15.17:0/3413822755 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1780053a70 0x7f1780055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:31.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.938+0000 7f179d7d7700 1 --2- 172.21.15.17:0/3413822755 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1780053a70 0x7f1780055f30 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f178c00d000 tx=0x7f178c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:31.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:31.942+0000 7f178a7fc700 1 -- 172.21.15.17:0/3413822755 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1790044220 con 0x7f17980b2710 2022-08-01T19:41:32.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.070+0000 7f179e7d9700 1 -- 172.21.15.17:0/3413822755 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1798069a70 con 0x7f17980b2710 2022-08-01T19:41:32.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.074+0000 7f178a7fc700 1 -- 172.21.15.17:0/3413822755 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) v1 ==== 104+0+10 (secure 0 0 0) 0x7f1790047840 con 0x7f17980b2710 2022-08-01T19:41:32.076 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:41:32.080 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.074+0000 7f179e7d9700 1 -- 172.21.15.17:0/3413822755 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1780053a70 msgr2=0x7f1780055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:32.080 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.074+0000 7f179e7d9700 1 --2- 172.21.15.17:0/3413822755 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1780053a70 0x7f1780055f30 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f178c00d000 tx=0x7f178c009040).stop 2022-08-01T19:41:32.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.074+0000 7f179e7d9700 1 -- 172.21.15.17:0/3413822755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17980b2710 msgr2=0x7f17980b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:32.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.074+0000 7f179e7d9700 1 --2- 172.21.15.17:0/3413822755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17980b2710 0x7f17980b5200 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 rx=0x7f17900032f0 tx=0x7f1790016000).stop 2022-08-01T19:41:32.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.074+0000 7f179e7d9700 1 -- 172.21.15.17:0/3413822755 shutdown_connections 2022-08-01T19:41:32.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.074+0000 7f179e7d9700 1 --2- 172.21.15.17:0/3413822755 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1780053a70 0x7f1780055f30 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:32.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.074+0000 7f179e7d9700 1 --2- 172.21.15.17:0/3413822755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17980b2710 0x7f17980b5200 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:32.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.074+0000 7f179e7d9700 1 --2- 172.21.15.17:0/3413822755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17980b01a0 0x7f17980b4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:32.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.074+0000 7f179e7d9700 1 --2- 172.21.15.17:0/3413822755 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f17980a8c50 0x7f17980aa0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:32.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.074+0000 7f179e7d9700 1 -- 172.21.15.17:0/3413822755 >> 172.21.15.17:0/3413822755 conn(0x7f1798060310 msgr2=0x7f1798144d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:32.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.074+0000 7f179e7d9700 1 -- 172.21.15.17:0/3413822755 shutdown_connections 2022-08-01T19:41:32.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.074+0000 7f179e7d9700 1 -- 172.21.15.17:0/3413822755 wait complete. 2022-08-01T19:41:32.094 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T19:41:32.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.202+0000 7f6484b4b700 1 Processor -- start 2022-08-01T19:41:32.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.202+0000 7f6484b4b700 1 -- start start 2022-08-01T19:41:32.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.206+0000 7f6484b4b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f64800af550 0x7f64800b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:32.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.206+0000 7f6484b4b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6480060330 0x7f64801445e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:32.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.206+0000 7f6484b4b700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f6480144ad0 0x7f64800a8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:32.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.206+0000 7f6484b4b700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64800b15e0 con 0x7f64800af550 2022-08-01T19:41:32.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.206+0000 7f6484b4b700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64800aa630 con 0x7f6480144ad0 2022-08-01T19:41:32.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.206+0000 7f6484b4b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f648005f840 con 0x7f6480060330 2022-08-01T19:41:32.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.206+0000 7f647f7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f64800af550 0x7f64800b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:32.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.206+0000 7f647effd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6480060330 0x7f64801445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:32.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.206+0000 7f647f7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f64800af550 0x7f64800b0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42214/0 (socket says 172.21.15.17:42214) 2022-08-01T19:41:32.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.206+0000 7f647f7fe700 1 -- 172.21.15.17:0/3397870198 learned_addr learned my addr 172.21.15.17:0/3397870198 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:32.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.206+0000 7f647ffff700 1 --2- 172.21.15.17:0/3397870198 >> v2:172.21.15.77:3300/0 conn(0x7f6480144ad0 0x7f64800a8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:32.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.206+0000 7f647f7fe700 1 -- 172.21.15.17:0/3397870198 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6480060330 msgr2=0x7f64801445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:32.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.206+0000 7f647f7fe700 1 --2- 172.21.15.17:0/3397870198 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6480060330 0x7f64801445e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:32.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.206+0000 7f647f7fe700 1 -- 172.21.15.17:0/3397870198 >> v2:172.21.15.77:3300/0 conn(0x7f6480144ad0 msgr2=0x7f64800a8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:32.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.206+0000 7f647f7fe700 1 --2- 172.21.15.17:0/3397870198 >> v2:172.21.15.77:3300/0 conn(0x7f6480144ad0 0x7f64800a8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:32.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.206+0000 7f647f7fe700 1 -- 172.21.15.17:0/3397870198 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f648005fd10 con 0x7f64800af550 2022-08-01T19:41:32.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.206+0000 7f647effd700 1 --2- 172.21.15.17:0/3397870198 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6480060330 0x7f64801445e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:32.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.206+0000 7f647f7fe700 1 --2- 172.21.15.17:0/3397870198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64800af550 0x7f64800b0e40 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 rx=0x7f6470002f70 tx=0x7f6470006eb0).ready entity=mon.0 client_cookie=6eaa19a9884ee141 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:32.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.206+0000 7f647e7fc700 1 -- 172.21.15.17:0/3397870198 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f647000d570 con 0x7f64800af550 2022-08-01T19:41:32.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.206+0000 7f647e7fc700 1 -- 172.21.15.17:0/3397870198 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6470000e90 con 0x7f64800af550 2022-08-01T19:41:32.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.206+0000 7f647e7fc700 1 -- 172.21.15.17:0/3397870198 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f647000d900 con 0x7f64800af550 2022-08-01T19:41:32.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.206+0000 7f6484b4b700 1 -- 172.21.15.17:0/3397870198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64800af550 msgr2=0x7f64800b0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:32.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.206+0000 7f6484b4b700 1 --2- 172.21.15.17:0/3397870198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64800af550 0x7f64800b0e40 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 rx=0x7f6470002f70 tx=0x7f6470006eb0).stop 2022-08-01T19:41:32.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.206+0000 7f6484b4b700 1 -- 172.21.15.17:0/3397870198 shutdown_connections 2022-08-01T19:41:32.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.206+0000 7f6484b4b700 1 --2- 172.21.15.17:0/3397870198 >> v2:172.21.15.77:3300/0 conn(0x7f6480144ad0 0x7f64800a8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:32.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.206+0000 7f6484b4b700 1 --2- 172.21.15.17:0/3397870198 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6480060330 0x7f64801445e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:32.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.206+0000 7f6484b4b700 1 --2- 172.21.15.17:0/3397870198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64800af550 0x7f64800b0e40 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:32.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.206+0000 7f6484b4b700 1 -- 172.21.15.17:0/3397870198 >> 172.21.15.17:0/3397870198 conn(0x7f64800612d0 msgr2=0x7f6480062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:32.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.206+0000 7f6484b4b700 1 -- 172.21.15.17:0/3397870198 shutdown_connections 2022-08-01T19:41:32.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.206+0000 7f6484b4b700 1 -- 172.21.15.17:0/3397870198 wait complete. 2022-08-01T19:41:32.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.210+0000 7f6484b4b700 1 Processor -- start 2022-08-01T19:41:32.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.210+0000 7f6484b4b700 1 -- start start 2022-08-01T19:41:32.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.210+0000 7f6484b4b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6480060330 0x7f64801437e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:32.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.210+0000 7f6484b4b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64800af550 0x7f648013a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:32.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.210+0000 7f6484b4b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6480144ad0 0x7f64800aae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:32.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.210+0000 7f6484b4b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6480136ad0 con 0x7f6480144ad0 2022-08-01T19:41:32.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.210+0000 7f6484b4b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64800aaa40 con 0x7f64800af550 2022-08-01T19:41:32.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.210+0000 7f6484b4b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6480142f10 con 0x7f6480060330 2022-08-01T19:41:32.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.210+0000 7f647ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6480144ad0 0x7f64800aae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:32.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.210+0000 7f647ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6480144ad0 0x7f64800aae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42224/0 (socket says 172.21.15.17:42224) 2022-08-01T19:41:32.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.210+0000 7f647ffff700 1 -- 172.21.15.17:0/864601648 learned_addr learned my addr 172.21.15.17:0/864601648 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:32.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.210+0000 7f647effd700 1 --2- 172.21.15.17:0/864601648 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64800af550 0x7f648013a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:32.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.210+0000 7f647f7fe700 1 --2- 172.21.15.17:0/864601648 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6480060330 0x7f64801437e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:32.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.210+0000 7f647ffff700 1 -- 172.21.15.17:0/864601648 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6480060330 msgr2=0x7f64801437e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:32.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.210+0000 7f647ffff700 1 --2- 172.21.15.17:0/864601648 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6480060330 0x7f64801437e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:32.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.210+0000 7f647ffff700 1 -- 172.21.15.17:0/864601648 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64800af550 msgr2=0x7f648013a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:32.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.210+0000 7f647ffff700 1 --2- 172.21.15.17:0/864601648 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64800af550 0x7f648013a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:32.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.210+0000 7f647ffff700 1 -- 172.21.15.17:0/864601648 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f646800d590 con 0x7f6480144ad0 2022-08-01T19:41:32.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.210+0000 7f647effd700 1 --2- 172.21.15.17:0/864601648 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64800af550 0x7f648013a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:32.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.210+0000 7f647ffff700 1 --2- 172.21.15.17:0/864601648 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6480144ad0 0x7f64800aae70 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 rx=0x7f647800c000 tx=0x7f64780091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:32.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.210+0000 7f647cff9700 1 -- 172.21.15.17:0/864601648 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f647800c670 con 0x7f6480144ad0 2022-08-01T19:41:32.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.214+0000 7f647cff9700 1 -- 172.21.15.17:0/864601648 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f647800c7d0 con 0x7f6480144ad0 2022-08-01T19:41:32.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.214+0000 7f6484b4b700 1 -- 172.21.15.17:0/864601648 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f647000b1e0 con 0x7f6480144ad0 2022-08-01T19:41:32.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.214+0000 7f647cff9700 1 -- 172.21.15.17:0/864601648 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f647800cb30 con 0x7f6480144ad0 2022-08-01T19:41:32.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.214+0000 7f6484b4b700 1 -- 172.21.15.17:0/864601648 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6480143450 con 0x7f6480144ad0 2022-08-01T19:41:32.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.214+0000 7f6484b4b700 1 -- 172.21.15.17:0/864601648 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6480142c80 con 0x7f6480144ad0 2022-08-01T19:41:32.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.214+0000 7f647cff9700 1 -- 172.21.15.17:0/864601648 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6478022070 con 0x7f6480144ad0 2022-08-01T19:41:32.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.214+0000 7f647cff9700 1 --2- 172.21.15.17:0/864601648 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6464053950 0x7f6464055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:32.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.214+0000 7f647cff9700 1 -- 172.21.15.17:0/864601648 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f647806c710 con 0x7f6480144ad0 2022-08-01T19:41:32.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.214+0000 7f647f7fe700 1 --2- 172.21.15.17:0/864601648 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6464053950 0x7f6464055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:32.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.218+0000 7f647f7fe700 1 --2- 172.21.15.17:0/864601648 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6464053950 0x7f6464055e10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f647001c000 tx=0x7f6470015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:32.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.222+0000 7f647cff9700 1 -- 172.21.15.17:0/864601648 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6478043470 con 0x7f6480144ad0 2022-08-01T19:41:32.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.346+0000 7f6484b4b700 1 -- 172.21.15.17:0/864601648 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f64800574e0 con 0x7f6480144ad0 2022-08-01T19:41:32.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.350+0000 7f647cff9700 1 -- 172.21.15.17:0/864601648 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) v1 ==== 100+0+10 (secure 0 0 0) 0x7f6478016020 con 0x7f6480144ad0 2022-08-01T19:41:32.351 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:41:32.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.350+0000 7f6484b4b700 1 -- 172.21.15.17:0/864601648 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6464053950 msgr2=0x7f6464055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:32.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.350+0000 7f6484b4b700 1 --2- 172.21.15.17:0/864601648 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6464053950 0x7f6464055e10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f647001c000 tx=0x7f6470015000).stop 2022-08-01T19:41:32.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.350+0000 7f6484b4b700 1 -- 172.21.15.17:0/864601648 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6480144ad0 msgr2=0x7f64800aae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:32.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.350+0000 7f6484b4b700 1 --2- 172.21.15.17:0/864601648 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6480144ad0 0x7f64800aae70 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 rx=0x7f647800c000 tx=0x7f64780091d0).stop 2022-08-01T19:41:32.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.350+0000 7f6484b4b700 1 -- 172.21.15.17:0/864601648 shutdown_connections 2022-08-01T19:41:32.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.350+0000 7f6484b4b700 1 --2- 172.21.15.17:0/864601648 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6464053950 0x7f6464055e10 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:32.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.350+0000 7f6484b4b700 1 --2- 172.21.15.17:0/864601648 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6480144ad0 0x7f64800aae70 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:32.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.350+0000 7f6484b4b700 1 --2- 172.21.15.17:0/864601648 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64800af550 0x7f648013a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:32.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.350+0000 7f6484b4b700 1 --2- 172.21.15.17:0/864601648 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6480060330 0x7f64801437e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:32.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.350+0000 7f6484b4b700 1 -- 172.21.15.17:0/864601648 >> 172.21.15.17:0/864601648 conn(0x7f64800612d0 msgr2=0x7f64800ae450 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:32.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.350+0000 7f6484b4b700 1 -- 172.21.15.17:0/864601648 shutdown_connections 2022-08-01T19:41:32.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.350+0000 7f6484b4b700 1 -- 172.21.15.17:0/864601648 wait complete. 2022-08-01T19:41:32.372 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T19:41:32.373 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:41:32.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.474+0000 7f96e7fff700 1 Processor -- start 2022-08-01T19:41:32.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.474+0000 7f96e7fff700 1 -- start start 2022-08-01T19:41:32.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.474+0000 7f96e7fff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f96e80b65c0 0x7f96e8061600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:32.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.474+0000 7f96e7fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96e80b41f0 0x7f96e8061af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:32.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.474+0000 7f96e7fff700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96e80b2100 con 0x7f96e8147d80 2022-08-01T19:41:32.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.474+0000 7f96e7fff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96e80b1630 con 0x7f96e80b65c0 2022-08-01T19:41:32.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.474+0000 7f96e7fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96e8061fe0 con 0x7f96e80b41f0 2022-08-01T19:41:32.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.474+0000 7f96e77fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f96e8147d80 0x7f96e8144630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34426/0 (socket says 172.21.15.17:34426) 2022-08-01T19:41:32.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.474+0000 7f96e77fe700 1 -- 172.21.15.17:0/2539852216 learned_addr learned my addr 172.21.15.17:0/2539852216 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:32.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.474+0000 7f96e67fc700 1 --2- 172.21.15.17:0/2539852216 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96e80b41f0 0x7f96e8061af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:32.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.474+0000 7f96e6ffd700 1 --2- 172.21.15.17:0/2539852216 >> v2:172.21.15.77:3300/0 conn(0x7f96e80b65c0 0x7f96e8061600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:32.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.474+0000 7f96e5ffb700 1 -- 172.21.15.17:0/2539852216 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4278174439 0 0) 0x7f96e80b2100 con 0x7f96e8147d80 2022-08-01T19:41:32.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.474+0000 7f96e67fc700 1 -- 172.21.15.17:0/2539852216 >> v2:172.21.15.77:3300/0 conn(0x7f96e80b65c0 msgr2=0x7f96e8061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:32.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.474+0000 7f96e67fc700 1 --2- 172.21.15.17:0/2539852216 >> v2:172.21.15.77:3300/0 conn(0x7f96e80b65c0 0x7f96e8061600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:32.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.474+0000 7f96e67fc700 1 -- 172.21.15.17:0/2539852216 >> v1:172.21.15.17:6789/0 conn(0x7f96e8147d80 legacy=0x7f96e8144630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:32.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.474+0000 7f96e67fc700 1 -- 172.21.15.17:0/2539852216 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e80b69a0 con 0x7f96e80b41f0 2022-08-01T19:41:32.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.474+0000 7f96e6ffd700 1 --2- 172.21.15.17:0/2539852216 >> v2:172.21.15.77:3300/0 conn(0x7f96e80b65c0 0x7f96e8061600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:32.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.474+0000 7f96e67fc700 1 --2- 172.21.15.17:0/2539852216 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96e80b41f0 0x7f96e8061af0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f96d80032f0 tx=0x7f96d80091d0).ready entity=mon.2 client_cookie=21a86daceacf9f2f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:32.499 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.474+0000 7f96e5ffb700 1 -- 172.21.15.17:0/2539852216 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96d800b6c0 con 0x7f96e80b41f0 2022-08-01T19:41:32.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.474+0000 7f96e5ffb700 1 -- 172.21.15.17:0/2539852216 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d800b820 con 0x7f96e80b41f0 2022-08-01T19:41:32.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.474+0000 7f96e5ffb700 1 -- 172.21.15.17:0/2539852216 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96d800b990 con 0x7f96e80b41f0 2022-08-01T19:41:32.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.478+0000 7f96e7fff700 1 -- 172.21.15.17:0/2539852216 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96e80b41f0 msgr2=0x7f96e8061af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:32.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.478+0000 7f96e7fff700 1 --2- 172.21.15.17:0/2539852216 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96e80b41f0 0x7f96e8061af0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f96d80032f0 tx=0x7f96d80091d0).stop 2022-08-01T19:41:32.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.478+0000 7f96e7fff700 1 -- 172.21.15.17:0/2539852216 shutdown_connections 2022-08-01T19:41:32.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.478+0000 7f96e7fff700 1 --2- 172.21.15.17:0/2539852216 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96e80b41f0 0x7f96e8061af0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:32.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.478+0000 7f96e7fff700 1 --2- 172.21.15.17:0/2539852216 >> v2:172.21.15.77:3300/0 conn(0x7f96e80b65c0 0x7f96e8061600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:32.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.478+0000 7f96e7fff700 1 -- 172.21.15.17:0/2539852216 >> 172.21.15.17:0/2539852216 conn(0x7f96e805c620 msgr2=0x7f96e805ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:32.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.478+0000 7f96e7fff700 1 -- 172.21.15.17:0/2539852216 shutdown_connections 2022-08-01T19:41:32.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.478+0000 7f96e7fff700 1 -- 172.21.15.17:0/2539852216 wait complete. 2022-08-01T19:41:32.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.478+0000 7f96e7fff700 1 Processor -- start 2022-08-01T19:41:32.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.478+0000 7f96e7fff700 1 -- start start 2022-08-01T19:41:32.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.482+0000 7f96e7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96e80b41f0 0x7f96e80b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:32.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.482+0000 7f96e7fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96e80b65c0 0x7f96e81498a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:32.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.482+0000 7f96e7fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96e8147d80 0x7f96e8149d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:32.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.482+0000 7f96e7fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e80af7d0 con 0x7f96e80b41f0 2022-08-01T19:41:32.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.482+0000 7f96e7fff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e814a9e0 con 0x7f96e8147d80 2022-08-01T19:41:32.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.482+0000 7f96e7fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96e814ac10 con 0x7f96e80b65c0 2022-08-01T19:41:32.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.482+0000 7f96e6ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96e80b41f0 0x7f96e80b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:32.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.482+0000 7f96e6ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96e80b41f0 0x7f96e80b0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42234/0 (socket says 172.21.15.17:42234) 2022-08-01T19:41:32.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.482+0000 7f96e6ffd700 1 -- 172.21.15.17:0/998187334 learned_addr learned my addr 172.21.15.17:0/998187334 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:32.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.482+0000 7f96e67fc700 1 --2- 172.21.15.17:0/998187334 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96e80b65c0 0x7f96e81498a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:32.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.482+0000 7f96e77fe700 1 --2- 172.21.15.17:0/998187334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96e8147d80 0x7f96e8149d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:32.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.482+0000 7f96e6ffd700 1 -- 172.21.15.17:0/998187334 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96e80b65c0 msgr2=0x7f96e81498a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:32.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.482+0000 7f96e6ffd700 1 --2- 172.21.15.17:0/998187334 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96e80b65c0 0x7f96e81498a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:32.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.482+0000 7f96e6ffd700 1 -- 172.21.15.17:0/998187334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96e8147d80 msgr2=0x7f96e8149d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:32.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.482+0000 7f96e6ffd700 1 --2- 172.21.15.17:0/998187334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96e8147d80 0x7f96e8149d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:32.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.482+0000 7f96e6ffd700 1 -- 172.21.15.17:0/998187334 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d000b180 con 0x7f96e80b41f0 2022-08-01T19:41:32.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.482+0000 7f96e6ffd700 1 --2- 172.21.15.17:0/998187334 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96e80b41f0 0x7f96e80b0330 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0x7f96e80aecf0 tx=0x7f96d0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:32.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.482+0000 7f96dffff700 1 -- 172.21.15.17:0/998187334 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96d000c7e0 con 0x7f96e80b41f0 2022-08-01T19:41:32.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.482+0000 7f96dffff700 1 -- 172.21.15.17:0/998187334 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d000c940 con 0x7f96e80b41f0 2022-08-01T19:41:32.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.482+0000 7f96dffff700 1 -- 172.21.15.17:0/998187334 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96d000cca0 con 0x7f96e80b41f0 2022-08-01T19:41:32.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.482+0000 7f96e7fff700 1 -- 172.21.15.17:0/998187334 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d8009470 con 0x7f96e80b41f0 2022-08-01T19:41:32.525 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.482+0000 7f96e7fff700 1 -- 172.21.15.17:0/998187334 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e814a5e0 con 0x7f96e80b41f0 2022-08-01T19:41:32.525 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.486+0000 7f96e7fff700 1 -- 172.21.15.17:0/998187334 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96e814a7d0 con 0x7f96e80b41f0 2022-08-01T19:41:32.525 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.486+0000 7f96dffff700 1 -- 172.21.15.17:0/998187334 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f96d001c460 con 0x7f96e80b41f0 2022-08-01T19:41:32.525 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.486+0000 7f96dffff700 1 --2- 172.21.15.17:0/998187334 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f96cc053aa0 0x7f96cc055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:32.525 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.486+0000 7f96dffff700 1 -- 172.21.15.17:0/998187334 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f96d006c620 con 0x7f96e80b41f0 2022-08-01T19:41:32.526 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.486+0000 7f96e67fc700 1 --2- 172.21.15.17:0/998187334 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f96cc053aa0 0x7f96cc055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:32.526 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.486+0000 7f96e67fc700 1 --2- 172.21.15.17:0/998187334 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f96cc053aa0 0x7f96cc055f60 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f96d801b000 tx=0x7f96d8016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:32.526 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.494+0000 7f96dffff700 1 -- 172.21.15.17:0/998187334 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f96d00432e0 con 0x7f96e80b41f0 2022-08-01T19:41:32.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.654+0000 7f96e7fff700 1 -- 172.21.15.17:0/998187334 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96e81482a0 con 0x7f96e80b41f0 2022-08-01T19:41:32.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.658+0000 7f96dffff700 1 -- 172.21.15.17:0/998187334 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f96d000ce00 con 0x7f96e80b41f0 2022-08-01T19:41:32.659 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:32.659 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":46,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6943,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/256800053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":256800053},{"type":"v1","addr":"172.21.15.77:6835","nonce":256800053}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44},{"gid":6957,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3629116150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3629116150},{"type":"v1","addr":"172.21.15.17:6833","nonce":3629116150}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:41:24.477490+0000","modified":"2022-08-01T19:41:26.186850+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6960},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6960":{"gid":6960,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/2788758172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2788758172},{"type":"v1","addr":"172.21.15.17:6835","nonce":2788758172}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2022-08-01T19:41:32.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.658+0000 7f96e7fff700 1 -- 172.21.15.17:0/998187334 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f96cc053aa0 msgr2=0x7f96cc055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:32.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.658+0000 7f96e7fff700 1 --2- 172.21.15.17:0/998187334 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f96cc053aa0 0x7f96cc055f60 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f96d801b000 tx=0x7f96d8016000).stop 2022-08-01T19:41:32.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.658+0000 7f96e7fff700 1 -- 172.21.15.17:0/998187334 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96e80b41f0 msgr2=0x7f96e80b0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:32.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.658+0000 7f96e7fff700 1 --2- 172.21.15.17:0/998187334 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96e80b41f0 0x7f96e80b0330 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0x7f96e80aecf0 tx=0x7f96d0006df0).stop 2022-08-01T19:41:32.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.658+0000 7f96e7fff700 1 -- 172.21.15.17:0/998187334 shutdown_connections 2022-08-01T19:41:32.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.658+0000 7f96e7fff700 1 --2- 172.21.15.17:0/998187334 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f96cc053aa0 0x7f96cc055f60 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:32.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.658+0000 7f96e7fff700 1 --2- 172.21.15.17:0/998187334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96e8147d80 0x7f96e8149d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:32.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.658+0000 7f96e7fff700 1 --2- 172.21.15.17:0/998187334 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96e80b65c0 0x7f96e81498a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:32.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.658+0000 7f96e7fff700 1 --2- 172.21.15.17:0/998187334 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96e80b41f0 0x7f96e80b0330 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:32.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.658+0000 7f96e7fff700 1 -- 172.21.15.17:0/998187334 >> 172.21.15.17:0/998187334 conn(0x7f96e805c620 msgr2=0x7f96e81446b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:32.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.658+0000 7f96e7fff700 1 -- 172.21.15.17:0/998187334 shutdown_connections 2022-08-01T19:41:32.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:32.658+0000 7f96e7fff700 1 -- 172.21.15.17:0/998187334 wait complete. 2022-08-01T19:41:32.669 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 46 2022-08-01T19:41:32.984 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:41:24.477490+0000', 'modified': '2022-08-01T19:41:26.186850+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6960}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6960': {'gid': 6960, 'name': 'c', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/2788758172', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 2788758172}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 2788758172}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:41:32.985 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:41:32.986 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T19:41:33.109 INFO:teuthology.orchestra.run.smithi017.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":6960,"want_state":"up:active","state":"up:active","rank_uptime":8.6184822299999997,"mdsmap_epoch":46,"osdmap_epoch":98,"osdmap_epoch_barrier":98,"uptime":12.986335001} 2022-08-01T19:41:33.110 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 6960, "mdsmap_epoch": 46, "osdmap_epoch": 98, "osdmap_epoch_barrier": 98, "rank_uptime": 8.61848223, "state": "up:active", "uptime": 12.986335001, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:41:33.111 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T19:41:33.112 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-08-01T19:41:33.114 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi190.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-08-01T19:41:33.114 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:33.115 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:41:33.129 DEBUG:teuthology.orchestra.run.smithi190:> sudo modprobe fuse 2022-08-01T19:41:33.190 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:41:33.191 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:41:33.215 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:41:33.220 INFO:teuthology.orchestra.run.smithi190.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:41:33.222 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:33.239 DEBUG:teuthology.orchestra.run.smithi190:> ls /sys/fs/fuse/connections 2022-08-01T19:41:33.276 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T19:41:33.277 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:41:33.291 DEBUG:teuthology.orchestra.run.smithi190:> sudo modprobe fuse 2022-08-01T19:41:33.312 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:41:33.313 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:41:33.360 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:2022-08-01T19:41:33.356+0000 7f6cf6b98080 -1 init, newargv = 0x562074bda830 newargc=9ceph-fuse[22852]: starting ceph client 2022-08-01T19:41:33.361 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr: 2022-08-01T19:41:33.370 INFO:teuthology.orchestra.run.smithi190.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:41:33.376 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:ceph-fuse[22852]: starting fuse 2022-08-01T19:41:33.377 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:41:33.378 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:33.379 DEBUG:teuthology.orchestra.run.smithi190:> ls /sys/fs/fuse/connections 2022-08-01T19:41:33.390 INFO:teuthology.orchestra.run.smithi190.stdout:52 2022-08-01T19:41:33.391 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T19:41:33.392 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:33.392 DEBUG:teuthology.orchestra.run.smithi190:> sudo python3 -c ' 2022-08-01T19:41:33.393 DEBUG:teuthology.orchestra.run.smithi190:> import glob 2022-08-01T19:41:33.393 DEBUG:teuthology.orchestra.run.smithi190:> import re 2022-08-01T19:41:33.394 DEBUG:teuthology.orchestra.run.smithi190:> import os 2022-08-01T19:41:33.394 DEBUG:teuthology.orchestra.run.smithi190:> import subprocess 2022-08-01T19:41:33.395 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:41:33.395 DEBUG:teuthology.orchestra.run.smithi190:> def find_socket(client_name): 2022-08-01T19:41:33.395 DEBUG:teuthology.orchestra.run.smithi190:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-08-01T19:41:33.396 DEBUG:teuthology.orchestra.run.smithi190:> files = glob.glob(asok_path) 2022-08-01T19:41:33.396 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:41:33.397 DEBUG:teuthology.orchestra.run.smithi190:> # Given a non-glob path, it better be there 2022-08-01T19:41:33.397 DEBUG:teuthology.orchestra.run.smithi190:> if "*" not in asok_path: 2022-08-01T19:41:33.402 DEBUG:teuthology.orchestra.run.smithi190:> assert(len(files) == 1) 2022-08-01T19:41:33.403 DEBUG:teuthology.orchestra.run.smithi190:> return files[0] 2022-08-01T19:41:33.404 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:41:33.409 DEBUG:teuthology.orchestra.run.smithi190:> for f in files: 2022-08-01T19:41:33.410 DEBUG:teuthology.orchestra.run.smithi190:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T19:41:33.410 DEBUG:teuthology.orchestra.run.smithi190:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T19:41:33.410 DEBUG:teuthology.orchestra.run.smithi190:> return f 2022-08-01T19:41:33.411 DEBUG:teuthology.orchestra.run.smithi190:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T19:41:33.411 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:41:33.412 DEBUG:teuthology.orchestra.run.smithi190:> print(find_socket("client.1")) 2022-08-01T19:41:33.412 DEBUG:teuthology.orchestra.run.smithi190:> ' 2022-08-01T19:41:33.478 INFO:teuthology.orchestra.run.smithi190.stdout:/var/run/ceph/ceph-client.1.22852.asok 2022-08-01T19:41:33.484 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.22852.asok 2022-08-01T19:41:33.485 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:33.487 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.22852.asok status 2022-08-01T19:41:33.581 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2022-08-01T19:41:33.581 INFO:teuthology.orchestra.run.smithi190.stdout: "metadata": { 2022-08-01T19:41:33.581 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T19:41:33.582 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T19:41:33.582 INFO:teuthology.orchestra.run.smithi190.stdout: "entity_id": "1", 2022-08-01T19:41:33.582 INFO:teuthology.orchestra.run.smithi190.stdout: "hostname": "smithi190", 2022-08-01T19:41:33.582 INFO:teuthology.orchestra.run.smithi190.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-08-01T19:41:33.583 INFO:teuthology.orchestra.run.smithi190.stdout: "pid": "22852", 2022-08-01T19:41:33.583 INFO:teuthology.orchestra.run.smithi190.stdout: "root": "/" 2022-08-01T19:41:33.583 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:41:33.583 INFO:teuthology.orchestra.run.smithi190.stdout: "dentry_count": 0, 2022-08-01T19:41:33.584 INFO:teuthology.orchestra.run.smithi190.stdout: "dentry_pinned_count": 0, 2022-08-01T19:41:33.584 INFO:teuthology.orchestra.run.smithi190.stdout: "id": 7164, 2022-08-01T19:41:33.585 INFO:teuthology.orchestra.run.smithi190.stdout: "inst": { 2022-08-01T19:41:33.585 INFO:teuthology.orchestra.run.smithi190.stdout: "name": { 2022-08-01T19:41:33.585 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "client", 2022-08-01T19:41:33.586 INFO:teuthology.orchestra.run.smithi190.stdout: "num": 7164 2022-08-01T19:41:33.586 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:41:33.586 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": { 2022-08-01T19:41:33.586 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "v1", 2022-08-01T19:41:33.587 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": "172.21.15.190:0", 2022-08-01T19:41:33.587 INFO:teuthology.orchestra.run.smithi190.stdout: "nonce": 3652891302 2022-08-01T19:41:33.587 INFO:teuthology.orchestra.run.smithi190.stdout: } 2022-08-01T19:41:33.587 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:41:33.588 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": { 2022-08-01T19:41:33.588 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "v1", 2022-08-01T19:41:33.588 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": "172.21.15.190:0", 2022-08-01T19:41:33.588 INFO:teuthology.orchestra.run.smithi190.stdout: "nonce": 3652891302 2022-08-01T19:41:33.589 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:41:33.593 INFO:teuthology.orchestra.run.smithi190.stdout: "inst_str": "client.7164 172.21.15.190:0/3652891302", 2022-08-01T19:41:33.593 INFO:teuthology.orchestra.run.smithi190.stdout: "addr_str": "172.21.15.190:0/3652891302", 2022-08-01T19:41:33.594 INFO:teuthology.orchestra.run.smithi190.stdout: "inode_count": 1, 2022-08-01T19:41:33.594 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_epoch": 46, 2022-08-01T19:41:33.595 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_epoch": 98, 2022-08-01T19:41:33.596 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_epoch_barrier": 0, 2022-08-01T19:41:33.596 INFO:teuthology.orchestra.run.smithi190.stdout: "blacklisted": false, 2022-08-01T19:41:33.597 INFO:teuthology.orchestra.run.smithi190.stdout: "fs_name": "cephfs" 2022-08-01T19:41:33.597 INFO:teuthology.orchestra.run.smithi190.stdout:} 2022-08-01T19:41:33.600 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:33.601 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:41:33.603 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:41:33.630 INFO:teuthology.orchestra.run.smithi190.stdout:fuseblk 2022-08-01T19:41:33.630 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T19:41:33.631 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:33.632 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:41:33.697 INFO:teuthology.orchestra.run:Running command with timeout 10800 2022-08-01T19:41:33.699 DEBUG:teuthology.orchestra.run.smithi085:> cd /home/ubuntu/cephtest/mnt.0 && sudo mkdir -p foo 2022-08-01T19:41:33.732 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-08-01T19:41:33.733 DEBUG:teuthology.orchestra.run.smithi085:> import os 2022-08-01T19:41:33.733 DEBUG:teuthology.orchestra.run.smithi085:> import stat 2022-08-01T19:41:33.733 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:41:33.734 DEBUG:teuthology.orchestra.run.smithi085:> print(os.stat("/home/ubuntu/cephtest/mnt.0/foo").st_ino) 2022-08-01T19:41:33.734 DEBUG:teuthology.orchestra.run.smithi085:> ' 2022-08-01T19:41:33.812 INFO:teuthology.orchestra.run.smithi085.stdout:1099511627776 2022-08-01T19:41:34.005 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:41:34.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.122+0000 7ff396f68700 1 Processor -- start 2022-08-01T19:41:34.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.122+0000 7ff396f68700 1 -- start start 2022-08-01T19:41:34.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.122+0000 7ff396f68700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff3900b0870 0x7ff390144170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:34.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.122+0000 7ff396f68700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff390067560 con 0x7ff390067180 2022-08-01T19:41:34.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.122+0000 7ff396f68700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3900b0160 con 0x7ff3900ad1c0 2022-08-01T19:41:34.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.122+0000 7ff396f68700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3900b0400 con 0x7ff3900b0870 2022-08-01T19:41:34.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.122+0000 7ff394d04700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff390067180 0x7ff390143a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34436/0 (socket says 172.21.15.17:34436) 2022-08-01T19:41:34.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.122+0000 7ff394d04700 1 -- 172.21.15.17:0/3372894965 learned_addr learned my addr 172.21.15.17:0/3372894965 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:34.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.122+0000 7ff395505700 1 --2- 172.21.15.17:0/3372894965 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff3900b0870 0x7ff390144170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:34.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.122+0000 7ff38f7fe700 1 -- 172.21.15.17:0/3372894965 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 567422420 0 0) 0x7ff390067560 con 0x7ff390067180 2022-08-01T19:41:34.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.122+0000 7ff38f7fe700 1 -- 172.21.15.17:0/3372894965 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff370003260 con 0x7ff390067180 2022-08-01T19:41:34.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.122+0000 7ff38f7fe700 1 -- 172.21.15.17:0/3372894965 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3839985938 0 0) 0x7ff3900b0160 con 0x7ff3900ad1c0 2022-08-01T19:41:34.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.122+0000 7ff395505700 1 -- 172.21.15.17:0/3372894965 >> v1:172.21.15.77:6789/0 conn(0x7ff3900ad1c0 legacy=0x7ff3900b1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:34.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.122+0000 7ff395505700 1 -- 172.21.15.17:0/3372894965 >> v1:172.21.15.17:6789/0 conn(0x7ff390067180 legacy=0x7ff390143a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:34.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.122+0000 7ff395505700 1 -- 172.21.15.17:0/3372894965 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3900af530 con 0x7ff3900b0870 2022-08-01T19:41:34.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.122+0000 7ff395505700 1 --2- 172.21.15.17:0/3372894965 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff3900b0870 0x7ff390144170 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7ff388002f70 tx=0x7ff388006df0).ready entity=mon.2 client_cookie=675d099d7cd2124c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:34.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.122+0000 7ff38f7fe700 1 -- 172.21.15.17:0/3372894965 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff38800d570 con 0x7ff3900b0870 2022-08-01T19:41:34.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.122+0000 7ff38f7fe700 1 -- 172.21.15.17:0/3372894965 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff388000e90 con 0x7ff3900b0870 2022-08-01T19:41:34.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.122+0000 7ff38f7fe700 1 -- 172.21.15.17:0/3372894965 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff38800d8c0 con 0x7ff3900b0870 2022-08-01T19:41:34.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.122+0000 7ff396f68700 1 -- 172.21.15.17:0/3372894965 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff3900b0870 msgr2=0x7ff390144170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:34.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.122+0000 7ff396f68700 1 --2- 172.21.15.17:0/3372894965 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff3900b0870 0x7ff390144170 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7ff388002f70 tx=0x7ff388006df0).stop 2022-08-01T19:41:34.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.122+0000 7ff396f68700 1 -- 172.21.15.17:0/3372894965 shutdown_connections 2022-08-01T19:41:34.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.122+0000 7ff396f68700 1 --2- 172.21.15.17:0/3372894965 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff3900b0870 0x7ff390144170 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:34.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.122+0000 7ff396f68700 1 -- 172.21.15.17:0/3372894965 >> 172.21.15.17:0/3372894965 conn(0x7ff39005f840 msgr2=0x7ff39005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:34.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.122+0000 7ff396f68700 1 -- 172.21.15.17:0/3372894965 shutdown_connections 2022-08-01T19:41:34.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.122+0000 7ff396f68700 1 -- 172.21.15.17:0/3372894965 wait complete. 2022-08-01T19:41:34.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.126+0000 7ff396f68700 1 Processor -- start 2022-08-01T19:41:34.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.126+0000 7ff396f68700 1 -- start start 2022-08-01T19:41:34.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.126+0000 7ff396f68700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff390067180 0x7ff39009f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:34.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.126+0000 7ff396f68700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff3900b0870 0x7ff3900a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:34.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.126+0000 7ff396f68700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3900a1900 0x7ff390152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:34.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.126+0000 7ff396f68700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3900ab720 con 0x7ff3900a1900 2022-08-01T19:41:34.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.126+0000 7ff396f68700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39009f560 con 0x7ff390067180 2022-08-01T19:41:34.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.126+0000 7ff396f68700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3900ab2c0 con 0x7ff3900b0870 2022-08-01T19:41:34.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.126+0000 7ff395505700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3900a1900 0x7ff390152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:34.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.126+0000 7ff395505700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3900a1900 0x7ff390152760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42252/0 (socket says 172.21.15.17:42252) 2022-08-01T19:41:34.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.126+0000 7ff395505700 1 -- 172.21.15.17:0/1581426358 learned_addr learned my addr 172.21.15.17:0/1581426358 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:34.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.126+0000 7ff394d04700 1 --2- 172.21.15.17:0/1581426358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff390067180 0x7ff39009f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:34.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.126+0000 7ff38ffff700 1 --2- 172.21.15.17:0/1581426358 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff3900b0870 0x7ff3900a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:34.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.126+0000 7ff395505700 1 -- 172.21.15.17:0/1581426358 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff3900b0870 msgr2=0x7ff3900a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:34.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.126+0000 7ff395505700 1 --2- 172.21.15.17:0/1581426358 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff3900b0870 0x7ff3900a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:34.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.126+0000 7ff395505700 1 -- 172.21.15.17:0/1581426358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff390067180 msgr2=0x7ff39009f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:34.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.126+0000 7ff395505700 1 --2- 172.21.15.17:0/1581426358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff390067180 0x7ff39009f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:34.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.126+0000 7ff395505700 1 -- 172.21.15.17:0/1581426358 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff38800b180 con 0x7ff3900a1900 2022-08-01T19:41:34.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.126+0000 7ff394d04700 1 --2- 172.21.15.17:0/1581426358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff390067180 0x7ff39009f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:34.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.126+0000 7ff395505700 1 --2- 172.21.15.17:0/1581426358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3900a1900 0x7ff390152760 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 rx=0x7ff388002f70 tx=0x7ff388014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:34.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.126+0000 7ff38dffb700 1 -- 172.21.15.17:0/1581426358 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff388013d60 con 0x7ff3900a1900 2022-08-01T19:41:34.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.126+0000 7ff38dffb700 1 -- 172.21.15.17:0/1581426358 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff38800de40 con 0x7ff3900a1900 2022-08-01T19:41:34.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.126+0000 7ff396f68700 1 -- 172.21.15.17:0/1581426358 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3900ac060 con 0x7ff3900a1900 2022-08-01T19:41:34.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.130+0000 7ff38dffb700 1 -- 172.21.15.17:0/1581426358 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff38801e5f0 con 0x7ff3900a1900 2022-08-01T19:41:34.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.130+0000 7ff396f68700 1 -- 172.21.15.17:0/1581426358 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff390152df0 con 0x7ff3900a1900 2022-08-01T19:41:34.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.130+0000 7ff396f68700 1 -- 172.21.15.17:0/1581426358 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3900ab8e0 con 0x7ff3900a1900 2022-08-01T19:41:34.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.130+0000 7ff38dffb700 1 -- 172.21.15.17:0/1581426358 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff388025030 con 0x7ff3900a1900 2022-08-01T19:41:34.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.130+0000 7ff38dffb700 1 --2- 172.21.15.17:0/1581426358 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff374053b00 0x7ff374055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:34.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.130+0000 7ff38dffb700 1 -- 172.21.15.17:0/1581426358 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff38806ccc0 con 0x7ff3900a1900 2022-08-01T19:41:34.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.130+0000 7ff394d04700 1 --2- 172.21.15.17:0/1581426358 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff374053b00 0x7ff374055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:34.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.130+0000 7ff394d04700 1 --2- 172.21.15.17:0/1581426358 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff374053b00 0x7ff374055fc0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7ff38000d000 tx=0x7ff380009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:34.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.138+0000 7ff38dffb700 1 -- 172.21.15.17:0/1581426358 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff388043a80 con 0x7ff3900a1900 2022-08-01T19:41:34.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.306+0000 7ff396f68700 1 -- 172.21.15.17:0/1581426358 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3900574e0 con 0x7ff3900a1900 2022-08-01T19:41:34.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.306+0000 7ff38dffb700 1 -- 172.21.15.17:0/1581426358 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+2490 (secure 0 0 0) 0x7ff388023020 con 0x7ff3900a1900 2022-08-01T19:41:34.309 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:34.309 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":46,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6943,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/256800053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":256800053},{"type":"v1","addr":"172.21.15.77:6835","nonce":256800053}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44},{"gid":6957,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3629116150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3629116150},{"type":"v1","addr":"172.21.15.17:6833","nonce":3629116150}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:41:24.477490+0000","modified":"2022-08-01T19:41:26.186850+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6960},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6960":{"gid":6960,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/2788758172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2788758172},{"type":"v1","addr":"172.21.15.17:6835","nonce":2788758172}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2022-08-01T19:41:34.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.306+0000 7ff396f68700 1 -- 172.21.15.17:0/1581426358 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff374053b00 msgr2=0x7ff374055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:34.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.306+0000 7ff396f68700 1 --2- 172.21.15.17:0/1581426358 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff374053b00 0x7ff374055fc0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7ff38000d000 tx=0x7ff380009040).stop 2022-08-01T19:41:34.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.306+0000 7ff396f68700 1 -- 172.21.15.17:0/1581426358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3900a1900 msgr2=0x7ff390152760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:34.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.306+0000 7ff396f68700 1 --2- 172.21.15.17:0/1581426358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3900a1900 0x7ff390152760 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 rx=0x7ff388002f70 tx=0x7ff388014000).stop 2022-08-01T19:41:34.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.310+0000 7ff396f68700 1 -- 172.21.15.17:0/1581426358 shutdown_connections 2022-08-01T19:41:34.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.310+0000 7ff396f68700 1 --2- 172.21.15.17:0/1581426358 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff374053b00 0x7ff374055fc0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:34.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.310+0000 7ff396f68700 1 --2- 172.21.15.17:0/1581426358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3900a1900 0x7ff390152760 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:34.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.310+0000 7ff396f68700 1 --2- 172.21.15.17:0/1581426358 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff3900b0870 0x7ff3900a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:34.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.310+0000 7ff396f68700 1 --2- 172.21.15.17:0/1581426358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff390067180 0x7ff39009f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:34.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.310+0000 7ff396f68700 1 -- 172.21.15.17:0/1581426358 >> 172.21.15.17:0/1581426358 conn(0x7ff39005f840 msgr2=0x7ff3900b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:34.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.310+0000 7ff396f68700 1 -- 172.21.15.17:0/1581426358 shutdown_connections 2022-08-01T19:41:34.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.310+0000 7ff396f68700 1 -- 172.21.15.17:0/1581426358 wait complete. 2022-08-01T19:41:34.315 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 46 2022-08-01T19:41:34.323 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump inode 0x10000000000 2022-08-01T19:41:34.426 INFO:teuthology.orchestra.run.smithi017.stdout:{"path":"/foo","ino":1099511627776,"rdev":0,"ctime":"2022-08-01T19:41:33.722986+0000","btime":"2022-08-01T19:41:33.722986+0000","mode":16877,"uid":0,"gid":0,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2022-08-01T19:41:33.722986+0000","atime":"2022-08-01T19:41:33.722986+0000","time_warp_seq":0,"change_attr":0,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"0.000000","num_files":0,"num_subdirs":0,"change_attr":0},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"0.000000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"0.000000"},"version":2,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"dirfrag":1,"caps":1,"dirtyparent":1,"dirty":1},"nref":4,"versionlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{"gather_set":[],"state":"excl","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"linklock":{},"dirfragtreelock":{},"filelock":{"gather_set":[],"state":"excl","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"xattrlock":{"gather_set":[],"state":"excl","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"states":["auth","dirty","dirtyparent","dirtypool"],"client_caps":[{"client_id":6293,"pending":"pAsxLsXsxFsx","issued":"pAsxLsXsxFsx","wanted":"-","last_sent":1}],"loner":6293,"want_loner":6293,"mds_caps_wanted":[]} 2022-08-01T19:41:34.427 DEBUG:tasks.cephfs.filesystem:_json_asok output { "accounted_rstat": { "rbytes": 0, "rctime": "0.000000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "2022-08-01T19:41:33.722986+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "xlock_by": {} }, "backtrace_version": 2, "btime": "2022-08-01T19:41:33.722986+0000", "change_attr": 0, "client_caps": [ { "client_id": 6293, "issued": "pAsxLsXsxFsx", "last_sent": 1, "pending": "pAsxLsXsxFsx", "wanted": "-" } ], "client_ranges": [], "ctime": "2022-08-01T19:41:33.722986+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 0, "mtime": "0.000000", "num_files": 0, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "xlock_by": {} }, "flocklock": {}, "gid": 0, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": 6293, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2022-08-01T19:41:33.722986+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "nlink": 1, "nref": 4, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "path": "/foo", "pins": { "caps": 1, "dirfrag": 1, "dirty": 1, "dirtyparent": 1, "request": 0 }, "policylock": {}, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "0.000000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 0, "version": 2, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "want_loner": 6293, "xattr_version": 1, "xattrlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "xlock_by": {} }, "xattrs": [] } 2022-08-01T19:41:34.427 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T19:41:34.428 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:41:34.448 DEBUG:teuthology.orchestra.run.smithi077:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:41:34.471 DEBUG:teuthology.orchestra.run.smithi085:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:41:34.494 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:41:34.518 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T19:41:34.519 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:34.520 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:41:34.520 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:41:34.532 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:41:34.533 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:41:34.534 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi085.front.sepia.ceph.com... 2022-08-01T19:41:34.535 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T19:41:34.535 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:41:34.590 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:ceph-fuse[23272]: fuse finished with error 0 and tester_r 0 2022-08-01T19:41:34.593 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:34.593 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:41:34.594 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:41:34.601 INFO:teuthology.orchestra.run.smithi085.stdout:ext2/ext3 2022-08-01T19:41:34.602 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T19:41:34.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T19:41:34.640 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:41:34.652 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:34.653 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:41:34.654 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:41:34.665 INFO:teuthology.orchestra.run.smithi190.stdout:fuseblk 2022-08-01T19:41:34.666 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T19:41:34.667 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi190.front.sepia.ceph.com... 2022-08-01T19:41:34.668 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T19:41:34.668 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:41:34.688 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:ceph-fuse[22852]: fuse finished with error 0 and tester_r 0 2022-08-01T19:41:34.690 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:34.692 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:41:34.692 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:41:34.740 INFO:teuthology.orchestra.run.smithi190.stdout:ext2/ext3 2022-08-01T19:41:34.745 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T19:41:34.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T19:41:34.757 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:41:34.769 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T19:41:34.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.882+0000 7fe63a58c700 1 Processor -- start 2022-08-01T19:41:34.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.882+0000 7fe63a58c700 1 -- start start 2022-08-01T19:41:34.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.882+0000 7fe63a58c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe6340614b0 0x7fe634061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:34.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.882+0000 7fe63a58c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe6340ad750 0x7fe6340b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:34.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.882+0000 7fe63a58c700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6340aee80 con 0x7fe6341405d0 2022-08-01T19:41:34.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.882+0000 7fe63a58c700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6340b70b0 con 0x7fe6340ad750 2022-08-01T19:41:34.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.882+0000 7fe63a58c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6340af0b0 con 0x7fe6340614b0 2022-08-01T19:41:34.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.882+0000 7fe638d89700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe6340ad750 0x7fe6340b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:34.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.882+0000 7fe639d8b700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe6341405d0 0x7fe6341466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34452/0 (socket says 172.21.15.17:34452) 2022-08-01T19:41:34.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.882+0000 7fe639d8b700 1 -- 172.21.15.17:0/601368004 learned_addr learned my addr 172.21.15.17:0/601368004 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:34.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.882+0000 7fe63958a700 1 --2- 172.21.15.17:0/601368004 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe6340614b0 0x7fe634061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:34.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.882+0000 7fe638d89700 1 -- 172.21.15.17:0/601368004 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe6340614b0 msgr2=0x7fe634061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:34.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.882+0000 7fe638d89700 1 --2- 172.21.15.17:0/601368004 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe6340614b0 0x7fe634061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:34.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.882+0000 7fe638d89700 1 -- 172.21.15.17:0/601368004 >> v1:172.21.15.17:6789/0 conn(0x7fe6341405d0 legacy=0x7fe6341466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:34.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.882+0000 7fe633fff700 1 -- 172.21.15.17:0/601368004 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1971872730 0 0) 0x7fe6340aee80 con 0x7fe6341405d0 2022-08-01T19:41:34.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.882+0000 7fe638d89700 1 -- 172.21.15.17:0/601368004 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6340af350 con 0x7fe6340ad750 2022-08-01T19:41:34.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.882+0000 7fe63958a700 1 --2- 172.21.15.17:0/601368004 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe6340614b0 0x7fe634061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:34.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.882+0000 7fe638d89700 1 --2- 172.21.15.17:0/601368004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6340ad750 0x7fe6340b9ff0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 rx=0x7fe61c002f70 tx=0x7fe61c006df0).ready entity=mon.0 client_cookie=7ccc7ac135ee48b1 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:34.899 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.882+0000 7fe633fff700 1 -- 172.21.15.17:0/601368004 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe61c00d570 con 0x7fe6340ad750 2022-08-01T19:41:34.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.882+0000 7fe633fff700 1 -- 172.21.15.17:0/601368004 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe61c000e90 con 0x7fe6340ad750 2022-08-01T19:41:34.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.882+0000 7fe633fff700 1 -- 172.21.15.17:0/601368004 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe61c00d8c0 con 0x7fe6340ad750 2022-08-01T19:41:34.901 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.886+0000 7fe63a58c700 1 -- 172.21.15.17:0/601368004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6340ad750 msgr2=0x7fe6340b9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:34.901 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.886+0000 7fe63a58c700 1 --2- 172.21.15.17:0/601368004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6340ad750 0x7fe6340b9ff0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 rx=0x7fe61c002f70 tx=0x7fe61c006df0).stop 2022-08-01T19:41:34.902 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.886+0000 7fe63a58c700 1 -- 172.21.15.17:0/601368004 shutdown_connections 2022-08-01T19:41:34.902 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.886+0000 7fe63a58c700 1 --2- 172.21.15.17:0/601368004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6340ad750 0x7fe6340b9ff0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:34.903 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.886+0000 7fe63a58c700 1 --2- 172.21.15.17:0/601368004 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe6340614b0 0x7fe634061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:34.903 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.886+0000 7fe63a58c700 1 -- 172.21.15.17:0/601368004 >> 172.21.15.17:0/601368004 conn(0x7fe634066f70 msgr2=0x7fe634067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:34.904 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.886+0000 7fe63a58c700 1 -- 172.21.15.17:0/601368004 shutdown_connections 2022-08-01T19:41:34.904 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.886+0000 7fe63a58c700 1 -- 172.21.15.17:0/601368004 wait complete. 2022-08-01T19:41:34.905 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.886+0000 7fe63a58c700 1 Processor -- start 2022-08-01T19:41:34.905 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.886+0000 7fe63a58c700 1 -- start start 2022-08-01T19:41:34.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.886+0000 7fe63a58c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe6340614b0 0x7fe6341489b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:34.907 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.886+0000 7fe63a58c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6340ad750 0x7fe6340b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:34.907 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.886+0000 7fe63a58c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6341405d0 0x7fe6340afd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:34.908 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.886+0000 7fe63a58c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6340b2710 con 0x7fe6341405d0 2022-08-01T19:41:34.908 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.886+0000 7fe63a58c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe634142030 con 0x7fe6340ad750 2022-08-01T19:41:34.909 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.886+0000 7fe63a58c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe634152f20 con 0x7fe6340614b0 2022-08-01T19:41:34.909 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.886+0000 7fe639d8b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6341405d0 0x7fe6340afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:34.910 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.886+0000 7fe63958a700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe6340614b0 0x7fe6341489b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:34.910 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.886+0000 7fe639d8b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6341405d0 0x7fe6340afd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42266/0 (socket says 172.21.15.17:42266) 2022-08-01T19:41:34.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.886+0000 7fe639d8b700 1 -- 172.21.15.17:0/184950315 learned_addr learned my addr 172.21.15.17:0/184950315 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:34.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.886+0000 7fe638d89700 1 --2- 172.21.15.17:0/184950315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6340ad750 0x7fe6340b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:34.912 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.886+0000 7fe639d8b700 1 -- 172.21.15.17:0/184950315 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe6340614b0 msgr2=0x7fe6341489b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:34.912 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.886+0000 7fe639d8b700 1 --2- 172.21.15.17:0/184950315 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe6340614b0 0x7fe6341489b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:34.912 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.886+0000 7fe639d8b700 1 -- 172.21.15.17:0/184950315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6340ad750 msgr2=0x7fe6340b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:34.913 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.886+0000 7fe639d8b700 1 --2- 172.21.15.17:0/184950315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6340ad750 0x7fe6340b1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:34.913 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.886+0000 7fe639d8b700 1 -- 172.21.15.17:0/184950315 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe624009470 con 0x7fe6341405d0 2022-08-01T19:41:34.913 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.886+0000 7fe638d89700 1 --2- 172.21.15.17:0/184950315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6340ad750 0x7fe6340b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:34.913 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.886+0000 7fe639d8b700 1 --2- 172.21.15.17:0/184950315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6341405d0 0x7fe6340afd70 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 rx=0x7fe62c000ed0 tx=0x7fe62c007c60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:34.914 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.886+0000 7fe6327fc700 1 -- 172.21.15.17:0/184950315 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe62c007110 con 0x7fe6341405d0 2022-08-01T19:41:34.914 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.890+0000 7fe6327fc700 1 -- 172.21.15.17:0/184950315 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe62c007270 con 0x7fe6341405d0 2022-08-01T19:41:34.914 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.890+0000 7fe63a58c700 1 -- 172.21.15.17:0/184950315 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe61c00b180 con 0x7fe6341405d0 2022-08-01T19:41:34.914 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.890+0000 7fe6327fc700 1 -- 172.21.15.17:0/184950315 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe62c005030 con 0x7fe6341405d0 2022-08-01T19:41:34.915 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.890+0000 7fe63a58c700 1 -- 172.21.15.17:0/184950315 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6340b2270 con 0x7fe6341405d0 2022-08-01T19:41:34.915 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.890+0000 7fe63a58c700 1 -- 172.21.15.17:0/184950315 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6340b0400 con 0x7fe6341405d0 2022-08-01T19:41:34.915 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.890+0000 7fe6327fc700 1 -- 172.21.15.17:0/184950315 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe62c005190 con 0x7fe6341405d0 2022-08-01T19:41:34.916 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.890+0000 7fe6327fc700 1 --2- 172.21.15.17:0/184950315 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe618053a70 0x7fe618055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:34.916 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.890+0000 7fe6327fc700 1 -- 172.21.15.17:0/184950315 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe62c06b8f0 con 0x7fe6341405d0 2022-08-01T19:41:34.917 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.890+0000 7fe63958a700 1 --2- 172.21.15.17:0/184950315 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe618053a70 0x7fe618055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:34.917 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.890+0000 7fe63958a700 1 --2- 172.21.15.17:0/184950315 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe618053a70 0x7fe618055f30 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7fe62400b000 tx=0x7fe6240091d0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:34.918 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:34.894+0000 7fe6327fc700 1 -- 172.21.15.17:0/184950315 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe62c0425a0 con 0x7fe6341405d0 2022-08-01T19:41:35.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.022+0000 7fe63a58c700 1 -- 172.21.15.17:0/184950315 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fe6340af5f0 con 0x7fe6341405d0 2022-08-01T19:41:35.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.026+0000 7fe6327fc700 1 -- 172.21.15.17:0/184950315 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v98) v1 ==== 81+0+34040 (secure 0 0 0) 0x7fe62c045bc0 con 0x7fe6341405d0 2022-08-01T19:41:35.030 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:35.031 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:41:35.031 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 98, 2022-08-01T19:41:35.031 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", 2022-08-01T19:41:35.032 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2022-08-01T19:33:50.105743+0000", 2022-08-01T19:41:35.032 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2022-08-01T19:41:25.480327+0000", 2022-08-01T19:41:35.033 INFO:teuthology.orchestra.run.smithi017.stdout: "last_up_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:41:35.033 INFO:teuthology.orchestra.run.smithi017.stdout: "last_in_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:41:35.034 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T19:41:35.034 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_num": 5799936, 2022-08-01T19:41:35.035 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_set": [ 2022-08-01T19:41:35.035 INFO:teuthology.orchestra.run.smithi017.stdout: "pglog_hardlimit", 2022-08-01T19:41:35.036 INFO:teuthology.orchestra.run.smithi017.stdout: "purged_snapdirs", 2022-08-01T19:41:35.036 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_deletes", 2022-08-01T19:41:35.037 INFO:teuthology.orchestra.run.smithi017.stdout: "sortbitwise" 2022-08-01T19:41:35.037 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:41:35.037 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_version": 6, 2022-08-01T19:41:35.038 INFO:teuthology.orchestra.run.smithi017.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T19:41:35.038 INFO:teuthology.orchestra.run.smithi017.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T19:41:35.039 INFO:teuthology.orchestra.run.smithi017.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T19:41:35.039 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_snapshot": "", 2022-08-01T19:41:35.039 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_max": 18, 2022-08-01T19:41:35.040 INFO:teuthology.orchestra.run.smithi017.stdout: "max_osd": 8, 2022-08-01T19:41:35.040 INFO:teuthology.orchestra.run.smithi017.stdout: "require_min_compat_client": "jewel", 2022-08-01T19:41:35.041 INFO:teuthology.orchestra.run.smithi017.stdout: "min_compat_client": "jewel", 2022-08-01T19:41:35.041 INFO:teuthology.orchestra.run.smithi017.stdout: "require_osd_release": "octopus", 2022-08-01T19:41:35.041 INFO:teuthology.orchestra.run.smithi017.stdout: "pools": [ 2022-08-01T19:41:35.042 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.042 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 1, 2022-08-01T19:41:35.043 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "device_health_metrics", 2022-08-01T19:41:35.043 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:33:53.001841+0000", 2022-08-01T19:41:35.043 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:41:35.044 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:41:35.044 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:41:35.045 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:41:35.045 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:41:35.046 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:41:35.046 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:41:35.046 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:41:35.047 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 1, 2022-08-01T19:41:35.047 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 1, 2022-08-01T19:41:35.048 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 1, 2022-08-01T19:41:35.048 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 1, 2022-08-01T19:41:35.049 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 1, 2022-08-01T19:41:35.049 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:41:35.049 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:41:35.050 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:41:35.050 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:41:35.050 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:41:35.051 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:41:35.051 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:41:35.052 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.052 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "16", 2022-08-01T19:41:35.052 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:41:35.053 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:41:35.053 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:41:35.054 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:41:35.054 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:41:35.055 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:41:35.055 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:41:35.055 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:41:35.056 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:41:35.056 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:41:35.057 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:41:35.057 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:41:35.057 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:41:35.058 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:41:35.058 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:41:35.059 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:41:35.059 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:41:35.060 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:41:35.060 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:41:35.060 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:41:35.061 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:41:35.061 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:41:35.062 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:41:35.062 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:41:35.062 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:41:35.063 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:41:35.063 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.064 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:41:35.064 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:41:35.064 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:41:35.065 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:41:35.065 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:41:35.066 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:41:35.066 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:41:35.067 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:41:35.067 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:41:35.068 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:41:35.068 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:41:35.069 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:41:35.069 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 1 2022-08-01T19:41:35.069 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.070 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:41:35.070 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_devicehealth": {} 2022-08-01T19:41:35.070 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.071 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.071 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.072 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 2, 2022-08-01T19:41:35.072 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "rbd", 2022-08-01T19:41:35.073 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:34:01.553098+0000", 2022-08-01T19:41:35.073 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:41:35.073 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:41:35.074 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:41:35.074 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:41:35.075 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:41:35.075 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:41:35.076 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:41:35.076 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:41:35.076 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:41:35.077 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:41:35.077 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:41:35.078 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:41:35.078 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:41:35.078 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:41:35.079 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:41:35.079 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:41:35.079 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:41:35.080 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:41:35.080 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:41:35.081 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:41:35.081 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.081 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "19", 2022-08-01T19:41:35.082 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:41:35.082 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:41:35.083 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:41:35.083 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:41:35.083 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:41:35.084 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:41:35.084 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:41:35.084 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:41:35.085 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:41:35.085 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:41:35.086 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:41:35.086 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:41:35.086 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:41:35.087 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:41:35.087 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:41:35.088 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:41:35.088 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:41:35.088 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:41:35.089 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:41:35.089 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:41:35.090 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:41:35.090 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:41:35.091 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:41:35.091 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:41:35.091 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:41:35.092 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:41:35.092 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.093 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:41:35.093 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:41:35.094 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:41:35.094 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:41:35.095 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:41:35.095 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:41:35.096 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:41:35.096 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:41:35.097 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:41:35.097 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:41:35.098 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:41:35.098 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:41:35.098 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:41:35.099 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd": {} 2022-08-01T19:41:35.099 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.100 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.100 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.100 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 17, 2022-08-01T19:41:35.101 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "cephfs_metadata", 2022-08-01T19:41:35.101 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:41:22.953305+0000", 2022-08-01T19:41:35.102 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:41:35.102 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:41:35.103 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:41:35.103 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:41:35.104 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:41:35.104 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:41:35.105 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:41:35.105 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:41:35.105 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:41:35.106 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:41:35.106 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:41:35.107 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:41:35.107 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:41:35.108 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:41:35.108 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:41:35.109 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:41:35.109 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:41:35.110 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:41:35.110 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:41:35.111 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:41:35.111 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.111 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "97", 2022-08-01T19:41:35.112 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:41:35.113 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:41:35.113 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:41:35.114 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:41:35.114 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:41:35.115 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:41:35.115 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:41:35.116 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:41:35.116 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:41:35.116 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:41:35.117 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:41:35.117 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:41:35.118 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:41:35.118 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:41:35.119 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:41:35.119 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:41:35.120 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:41:35.120 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:41:35.121 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:41:35.121 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:41:35.121 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:41:35.125 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:41:35.125 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:41:35.126 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:41:35.126 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:41:35.126 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:41:35.127 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.127 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:41:35.128 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:41:35.128 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:41:35.128 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:41:35.129 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:41:35.129 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:41:35.130 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:41:35.130 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:41:35.131 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:41:35.131 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:41:35.131 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:41:35.134 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:41:35.134 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_bias": 4, 2022-08-01T19:41:35.135 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 16, 2022-08-01T19:41:35.135 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_priority": 5 2022-08-01T19:41:35.136 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.136 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:41:35.137 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs": { 2022-08-01T19:41:35.137 INFO:teuthology.orchestra.run.smithi017.stdout: "metadata": "cephfs" 2022-08-01T19:41:35.137 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.139 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.139 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.139 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.140 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 18, 2022-08-01T19:41:35.140 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "cephfs_data", 2022-08-01T19:41:35.140 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:41:23.938052+0000", 2022-08-01T19:41:35.141 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:41:35.141 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:41:35.142 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:41:35.142 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:41:35.142 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:41:35.143 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:41:35.143 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:41:35.144 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:41:35.144 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:41:35.144 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:41:35.145 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:41:35.145 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:41:35.145 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:41:35.146 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:41:35.146 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:41:35.146 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:41:35.146 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:41:35.147 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:41:35.147 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:41:35.148 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:41:35.148 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.149 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "98", 2022-08-01T19:41:35.155 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:41:35.155 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:41:35.156 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:41:35.156 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:41:35.156 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:41:35.156 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:41:35.157 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:41:35.157 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:41:35.157 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:41:35.157 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:41:35.157 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:41:35.158 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:41:35.158 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:41:35.158 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:41:35.159 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:41:35.159 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:41:35.159 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:41:35.159 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:41:35.159 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:41:35.160 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:41:35.160 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:41:35.160 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:41:35.160 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:41:35.161 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:41:35.161 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:41:35.161 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:41:35.161 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.161 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:41:35.162 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:41:35.162 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:41:35.162 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:41:35.162 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:41:35.163 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:41:35.163 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:41:35.163 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:41:35.163 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:41:35.164 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:41:35.164 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:41:35.164 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:41:35.164 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:41:35.164 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs": { 2022-08-01T19:41:35.165 INFO:teuthology.orchestra.run.smithi017.stdout: "data": "cephfs" 2022-08-01T19:41:35.165 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.165 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.165 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.166 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:41:35.166 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2022-08-01T19:41:35.167 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.167 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:41:35.167 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a962f14f-cf16-4fa8-84ff-c85325db9132", 2022-08-01T19:41:35.167 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:35.168 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:35.168 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:35.169 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:35.169 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:35.169 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:35.169 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:41:35.170 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 96, 2022-08-01T19:41:35.170 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:35.170 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:35.170 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:35.171 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:35.171 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.171 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:35.171 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6800", 2022-08-01T19:41:35.172 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:35.172 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.172 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.172 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:35.172 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6801", 2022-08-01T19:41:35.173 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:35.173 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.173 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.174 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.174 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:35.174 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:35.174 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.174 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:35.175 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6802", 2022-08-01T19:41:35.175 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:35.175 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.175 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.176 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:35.176 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6803", 2022-08-01T19:41:35.176 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:35.176 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.176 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.177 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.177 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:35.177 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:35.177 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.178 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:35.178 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6806", 2022-08-01T19:41:35.178 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:35.178 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.179 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.179 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:35.179 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6807", 2022-08-01T19:41:35.179 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:35.179 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.180 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.180 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:35.180 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:35.181 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.181 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:35.181 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6804", 2022-08-01T19:41:35.181 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:35.182 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.182 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.182 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:35.182 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6805", 2022-08-01T19:41:35.183 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:35.183 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.183 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.183 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.183 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6801/23008", 2022-08-01T19:41:35.184 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6803/23008", 2022-08-01T19:41:35.184 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6807/23008", 2022-08-01T19:41:35.184 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6805/23008", 2022-08-01T19:41:35.185 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:35.185 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:35.185 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:35.185 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.186 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.186 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.186 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:41:35.186 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "805f7051-a527-4e98-8844-b6e406a63248", 2022-08-01T19:41:35.187 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:35.187 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:35.187 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:35.187 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:35.188 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:35.188 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:35.188 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:41:35.188 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 96, 2022-08-01T19:41:35.188 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:35.189 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:35.189 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:35.189 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:35.189 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.190 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:35.190 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6808", 2022-08-01T19:41:35.190 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:35.190 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.191 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.191 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:35.191 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6809", 2022-08-01T19:41:35.191 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:35.191 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.192 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.192 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.192 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:35.192 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:35.192 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.193 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:35.193 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6810", 2022-08-01T19:41:35.193 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:35.193 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.194 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.194 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:35.194 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6811", 2022-08-01T19:41:35.194 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:35.195 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.195 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.195 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.195 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:35.195 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:35.196 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.196 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:35.196 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6814", 2022-08-01T19:41:35.196 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:35.197 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.197 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.197 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:35.197 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6815", 2022-08-01T19:41:35.197 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:35.198 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.198 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.198 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.199 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:35.199 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:35.199 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.199 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:35.200 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6812", 2022-08-01T19:41:35.200 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:35.200 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.200 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.200 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:35.201 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6813", 2022-08-01T19:41:35.202 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:35.202 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.202 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.202 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.202 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6809/23007", 2022-08-01T19:41:35.203 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6811/23007", 2022-08-01T19:41:35.203 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6815/23007", 2022-08-01T19:41:35.203 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6813/23007", 2022-08-01T19:41:35.206 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:35.215 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:35.215 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:35.216 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.216 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.216 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.216 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:41:35.217 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "89f7f3df-7c10-48e1-8b69-9e4243923a61", 2022-08-01T19:41:35.217 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:35.217 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:35.217 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:35.217 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:35.218 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:35.218 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:35.218 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:41:35.218 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 96, 2022-08-01T19:41:35.219 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:35.219 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:35.219 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:35.219 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:35.220 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.220 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:35.220 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6817", 2022-08-01T19:41:35.220 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:35.221 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.221 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.221 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:35.221 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6819", 2022-08-01T19:41:35.222 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:35.222 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.223 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.223 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.224 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:35.224 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:35.224 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.224 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:35.225 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6822", 2022-08-01T19:41:35.225 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:35.225 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.225 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.226 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:35.226 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6824", 2022-08-01T19:41:35.226 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:35.228 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.228 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.228 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.229 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:35.229 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:35.229 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.229 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:35.230 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6830", 2022-08-01T19:41:35.230 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:35.230 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.231 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.231 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:35.231 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6831", 2022-08-01T19:41:35.231 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:35.232 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.232 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.232 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.233 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:35.233 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:35.233 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.233 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:35.234 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6826", 2022-08-01T19:41:35.235 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:35.235 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.236 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.236 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:35.236 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6829", 2022-08-01T19:41:35.236 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:35.237 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.237 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.237 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.237 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6819/23010", 2022-08-01T19:41:35.238 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6824/23010", 2022-08-01T19:41:35.238 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6831/23010", 2022-08-01T19:41:35.238 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6829/23010", 2022-08-01T19:41:35.238 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:35.239 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:35.239 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:35.239 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.240 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.240 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.240 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:41:35.240 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "96123468-aabc-4a64-909e-a192c369090d", 2022-08-01T19:41:35.240 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:35.241 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:35.241 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:35.241 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:35.241 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:35.242 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:35.242 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:41:35.242 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 96, 2022-08-01T19:41:35.243 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:35.243 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:35.243 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:35.243 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:35.244 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.245 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:35.245 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6816", 2022-08-01T19:41:35.245 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:35.245 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.246 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.246 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:35.246 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6818", 2022-08-01T19:41:35.247 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:35.247 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.248 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.248 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.248 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:35.249 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:35.249 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.250 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:35.250 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6820", 2022-08-01T19:41:35.251 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:35.251 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.252 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.252 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:35.253 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6821", 2022-08-01T19:41:35.253 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:35.253 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.253 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.254 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.254 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:35.254 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:35.254 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.255 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:35.255 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6827", 2022-08-01T19:41:35.255 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:35.256 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.256 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.256 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:35.257 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6828", 2022-08-01T19:41:35.257 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:35.257 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.257 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.258 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.258 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:35.258 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:35.258 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.259 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:35.259 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6823", 2022-08-01T19:41:35.259 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:35.259 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.259 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.260 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:35.260 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6825", 2022-08-01T19:41:35.260 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:35.260 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.261 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.261 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.261 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6818/23009", 2022-08-01T19:41:35.261 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6821/23009", 2022-08-01T19:41:35.261 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6828/23009", 2022-08-01T19:41:35.262 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6825/23009", 2022-08-01T19:41:35.262 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:35.262 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:35.263 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:35.263 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.263 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.263 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.263 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:41:35.264 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a6e26665-e04b-46ec-ab66-e912acbf044c", 2022-08-01T19:41:35.264 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:35.264 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:35.264 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:35.265 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:35.265 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:35.265 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:35.265 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:41:35.265 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 95, 2022-08-01T19:41:35.266 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:35.266 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:35.267 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:35.267 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:35.267 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.267 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:35.268 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6810", 2022-08-01T19:41:35.268 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:35.268 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.268 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.268 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:35.269 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6811", 2022-08-01T19:41:35.269 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:35.269 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.269 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.270 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.270 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:35.270 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:35.270 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.271 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:35.271 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6812", 2022-08-01T19:41:35.271 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:35.271 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.272 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.272 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:35.272 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6813", 2022-08-01T19:41:35.272 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:35.272 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.273 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.273 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.273 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:35.273 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:35.274 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.274 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:35.274 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6816", 2022-08-01T19:41:35.274 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:35.275 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.275 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.275 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:35.275 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6817", 2022-08-01T19:41:35.276 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:35.276 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.276 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.276 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.277 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:35.277 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:35.277 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.277 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:35.278 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6814", 2022-08-01T19:41:35.278 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:35.278 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.278 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.279 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:35.279 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6815", 2022-08-01T19:41:35.279 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:35.279 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.280 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.280 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.280 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6811/23584", 2022-08-01T19:41:35.280 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6813/23584", 2022-08-01T19:41:35.280 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6817/23584", 2022-08-01T19:41:35.281 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6815/23584", 2022-08-01T19:41:35.281 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:35.281 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:35.281 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:35.282 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.282 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.282 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.282 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:41:35.282 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "502c0034-18ee-44ac-87ff-19b9dddb66a0", 2022-08-01T19:41:35.283 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:35.283 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:35.283 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:35.283 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:35.284 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:35.285 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:35.285 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:41:35.285 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 86, 2022-08-01T19:41:35.286 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:35.286 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:35.286 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:35.286 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:35.287 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.287 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:35.287 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6802", 2022-08-01T19:41:35.287 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:35.287 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.288 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.288 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:35.288 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6803", 2022-08-01T19:41:35.288 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:35.289 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.289 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.289 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.289 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:35.290 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:35.290 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.290 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:35.291 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6804", 2022-08-01T19:41:35.292 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:35.292 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.292 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.293 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:35.293 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6805", 2022-08-01T19:41:35.293 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:35.294 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.294 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.294 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.294 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:35.295 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:35.295 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.295 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:35.295 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6808", 2022-08-01T19:41:35.296 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:35.296 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.296 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.297 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:35.297 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6809", 2022-08-01T19:41:35.297 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:35.297 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.298 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.298 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.298 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:35.298 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:35.299 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.299 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:35.299 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6806", 2022-08-01T19:41:35.299 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:35.300 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.300 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.300 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:35.300 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6807", 2022-08-01T19:41:35.300 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:35.301 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.301 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.301 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.301 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6803/23585", 2022-08-01T19:41:35.302 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6805/23585", 2022-08-01T19:41:35.302 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6809/23585", 2022-08-01T19:41:35.303 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6807/23585", 2022-08-01T19:41:35.303 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:35.303 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:35.303 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:35.304 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.304 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.304 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.304 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:41:35.305 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "ac4e755a-7fe7-46cb-8407-852d5af9ffc5", 2022-08-01T19:41:35.305 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:35.305 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:35.305 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:35.305 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:35.306 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:35.306 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:35.306 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:41:35.306 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 95, 2022-08-01T19:41:35.307 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:35.307 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:35.307 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:35.307 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:35.307 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.308 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:35.308 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6820", 2022-08-01T19:41:35.308 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:35.309 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.309 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.309 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:35.309 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6821", 2022-08-01T19:41:35.310 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:35.310 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.310 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.310 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.311 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:35.311 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:35.311 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.311 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:35.312 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6823", 2022-08-01T19:41:35.312 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:35.312 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.312 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.313 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:35.313 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6825", 2022-08-01T19:41:35.313 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:35.314 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.314 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.314 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.314 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:35.315 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:35.315 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.315 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:35.315 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6831", 2022-08-01T19:41:35.315 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:35.316 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.316 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.316 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:35.316 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6833", 2022-08-01T19:41:35.317 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:35.317 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.317 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.317 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.318 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:35.318 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:35.318 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.318 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:35.318 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6827", 2022-08-01T19:41:35.319 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:35.319 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.319 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.319 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:35.320 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6829", 2022-08-01T19:41:35.320 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:35.320 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.320 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.320 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.321 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6821/23587", 2022-08-01T19:41:35.321 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6825/23587", 2022-08-01T19:41:35.321 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6833/23587", 2022-08-01T19:41:35.321 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6829/23587", 2022-08-01T19:41:35.321 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:35.322 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:35.322 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:35.322 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.322 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.323 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.323 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:41:35.324 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "874761e2-6410-40fc-b60c-e58abe5ce489", 2022-08-01T19:41:35.324 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:35.324 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:35.324 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:35.325 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:35.325 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:35.325 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:35.325 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:41:35.325 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 96, 2022-08-01T19:41:35.326 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:35.326 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:35.326 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:35.326 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:35.327 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.327 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:35.327 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6818", 2022-08-01T19:41:35.327 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:35.328 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.328 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.328 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:35.328 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6819", 2022-08-01T19:41:35.328 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:35.330 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.331 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.331 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.331 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:35.332 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:35.332 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.333 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:35.333 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6822", 2022-08-01T19:41:35.334 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:35.334 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.335 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.335 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:35.336 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6824", 2022-08-01T19:41:35.336 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:35.339 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.340 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.340 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.341 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:35.341 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:35.341 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.342 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:35.342 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6830", 2022-08-01T19:41:35.343 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:35.343 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.344 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.344 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:35.344 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6832", 2022-08-01T19:41:35.345 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:35.345 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.346 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.346 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.346 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:35.347 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:35.347 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.348 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:35.348 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6826", 2022-08-01T19:41:35.357 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:35.358 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.358 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.359 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:35.359 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6828", 2022-08-01T19:41:35.360 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:35.360 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.360 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.361 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.361 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6819/23586", 2022-08-01T19:41:35.362 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6824/23586", 2022-08-01T19:41:35.362 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6832/23586", 2022-08-01T19:41:35.363 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6828/23586", 2022-08-01T19:41:35.363 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:35.364 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:35.364 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:35.365 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:35.365 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.366 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:41:35.366 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_xinfo": [ 2022-08-01T19:41:35.367 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.367 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:41:35.368 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:35.368 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:35.369 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:35.369 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:35.370 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:35.370 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.588817+0000", 2022-08-01T19:41:35.371 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:35.371 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.372 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.372 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:41:35.373 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:35.373 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:35.374 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:35.374 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:35.375 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:35.375 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.782753+0000", 2022-08-01T19:41:35.376 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:35.376 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.377 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.377 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:41:35.377 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:35.378 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:35.378 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:35.379 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:35.379 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:35.380 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.837924+0000", 2022-08-01T19:41:35.380 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:35.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.381 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.382 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:41:35.382 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:35.383 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:35.383 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:35.384 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:35.390 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:35.390 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.839948+0000", 2022-08-01T19:41:35.390 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:35.391 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.391 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.391 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:41:35.392 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:35.392 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:35.392 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:35.392 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:35.393 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:35.393 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.112801+0000", 2022-08-01T19:41:35.393 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:35.393 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.393 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.395 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:41:35.395 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:35.395 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:35.396 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:35.396 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:35.396 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:35.396 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.106704+0000", 2022-08-01T19:41:35.397 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:35.397 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.397 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.397 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:41:35.398 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:35.398 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:35.398 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:35.398 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:35.399 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:35.400 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.893170+0000", 2022-08-01T19:41:35.400 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:35.401 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.402 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:35.403 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:41:35.403 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:35.404 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:35.404 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:35.405 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:35.406 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:35.406 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.886947+0000", 2022-08-01T19:41:35.407 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:35.407 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.408 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:41:35.408 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap": [], 2022-08-01T19:41:35.409 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap_items": [], 2022-08-01T19:41:35.409 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_temp": [], 2022-08-01T19:41:35.410 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_temp": [], 2022-08-01T19:41:35.410 INFO:teuthology.orchestra.run.smithi017.stdout: "blacklist": {}, 2022-08-01T19:41:35.410 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profiles": { 2022-08-01T19:41:35.411 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_data": { 2022-08-01T19:41:35.411 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-device-class": "", 2022-08-01T19:41:35.411 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:41:35.412 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-root": "default", 2022-08-01T19:41:35.412 INFO:teuthology.orchestra.run.smithi017.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T19:41:35.413 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:41:35.413 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "2", 2022-08-01T19:41:35.413 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:41:35.414 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:41:35.414 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van", 2022-08-01T19:41:35.414 INFO:teuthology.orchestra.run.smithi017.stdout: "w": "8" 2022-08-01T19:41:35.414 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.415 INFO:teuthology.orchestra.run.smithi017.stdout: "default": { 2022-08-01T19:41:35.415 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:41:35.415 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:41:35.416 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "1", 2022-08-01T19:41:35.416 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:41:35.416 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:41:35.417 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van" 2022-08-01T19:41:35.417 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:35.417 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:35.417 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps_queue": [], 2022-08-01T19:41:35.418 INFO:teuthology.orchestra.run.smithi017.stdout: "new_removed_snaps": [], 2022-08-01T19:41:35.418 INFO:teuthology.orchestra.run.smithi017.stdout: "new_purged_snaps": [], 2022-08-01T19:41:35.419 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_node_flags": {}, 2022-08-01T19:41:35.419 INFO:teuthology.orchestra.run.smithi017.stdout: "device_class_flags": {} 2022-08-01T19:41:35.420 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:41:35.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.026+0000 7fe63a58c700 1 -- 172.21.15.17:0/184950315 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe618053a70 msgr2=0x7fe618055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:35.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.026+0000 7fe63a58c700 1 --2- 172.21.15.17:0/184950315 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe618053a70 0x7fe618055f30 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7fe62400b000 tx=0x7fe6240091d0).stop 2022-08-01T19:41:35.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.026+0000 7fe63a58c700 1 -- 172.21.15.17:0/184950315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6341405d0 msgr2=0x7fe6340afd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:35.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.026+0000 7fe63a58c700 1 --2- 172.21.15.17:0/184950315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6341405d0 0x7fe6340afd70 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 rx=0x7fe62c000ed0 tx=0x7fe62c007c60).stop 2022-08-01T19:41:35.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.026+0000 7fe63a58c700 1 -- 172.21.15.17:0/184950315 shutdown_connections 2022-08-01T19:41:35.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.026+0000 7fe63a58c700 1 --2- 172.21.15.17:0/184950315 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe618053a70 0x7fe618055f30 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:35.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.026+0000 7fe63a58c700 1 --2- 172.21.15.17:0/184950315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6341405d0 0x7fe6340afd70 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:35.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.026+0000 7fe63a58c700 1 --2- 172.21.15.17:0/184950315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6340ad750 0x7fe6340b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:35.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.026+0000 7fe63a58c700 1 --2- 172.21.15.17:0/184950315 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe6340614b0 0x7fe6341489b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:35.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.026+0000 7fe63a58c700 1 -- 172.21.15.17:0/184950315 >> 172.21.15.17:0/184950315 conn(0x7fe634066f70 msgr2=0x7fe634146880 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:35.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.026+0000 7fe63a58c700 1 -- 172.21.15.17:0/184950315 shutdown_connections 2022-08-01T19:41:35.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.026+0000 7fe63a58c700 1 -- 172.21.15.17:0/184950315 wait complete. 2022-08-01T19:41:35.430 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:41:35.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.542+0000 7f385a195700 1 Processor -- start 2022-08-01T19:41:35.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.542+0000 7f385a195700 1 -- start start 2022-08-01T19:41:35.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.542+0000 7f385a195700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3854061060 0x7f38540614d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:35.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.542+0000 7f385a195700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3854060670 con 0x7f38540b38d0 2022-08-01T19:41:35.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.542+0000 7f385a195700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3854065c30 con 0x7f3854060290 2022-08-01T19:41:35.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.542+0000 7f385a195700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3854065ed0 con 0x7f3854061060 2022-08-01T19:41:35.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.542+0000 7f3852ffd700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f38540b38d0 0x7f3854060950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34464/0 (socket says 172.21.15.17:34464) 2022-08-01T19:41:35.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.542+0000 7f3852ffd700 1 -- 172.21.15.17:0/1066493157 learned_addr learned my addr 172.21.15.17:0/1066493157 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:35.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.542+0000 7f3853fff700 1 --2- 172.21.15.17:0/1066493157 >> v2:172.21.15.17:3300/0 conn(0x7f3854061060 0x7f38540614d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:35.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.542+0000 7f3853fff700 1 -- 172.21.15.17:0/1066493157 >> v1:172.21.15.77:6789/0 conn(0x7f3854060290 legacy=0x7f3854066610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:35.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.542+0000 7f3853fff700 1 -- 172.21.15.17:0/1066493157 >> v1:172.21.15.17:6789/0 conn(0x7f38540b38d0 legacy=0x7f3854060950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:35.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.542+0000 7f3853fff700 1 -- 172.21.15.17:0/1066493157 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3854066170 con 0x7f3854061060 2022-08-01T19:41:35.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.542+0000 7f3853fff700 1 --2- 172.21.15.17:0/1066493157 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3854061060 0x7f38540614d0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x7f384c002f70 tx=0x7f384c006df0).ready entity=mon.0 client_cookie=40057a5bcf337937 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:35.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.542+0000 7f38527fc700 1 -- 172.21.15.17:0/1066493157 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f384c00d610 con 0x7f3854061060 2022-08-01T19:41:35.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.542+0000 7f38527fc700 1 -- 172.21.15.17:0/1066493157 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384c000e90 con 0x7f3854061060 2022-08-01T19:41:35.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.542+0000 7f38527fc700 1 -- 172.21.15.17:0/1066493157 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f384c00d960 con 0x7f3854061060 2022-08-01T19:41:35.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.542+0000 7f385a195700 1 -- 172.21.15.17:0/1066493157 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3854061060 msgr2=0x7f38540614d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:35.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.546+0000 7f385a195700 1 --2- 172.21.15.17:0/1066493157 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3854061060 0x7f38540614d0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x7f384c002f70 tx=0x7f384c006df0).stop 2022-08-01T19:41:35.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.546+0000 7f385a195700 1 -- 172.21.15.17:0/1066493157 shutdown_connections 2022-08-01T19:41:35.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.546+0000 7f385a195700 1 --2- 172.21.15.17:0/1066493157 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3854061060 0x7f38540614d0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:35.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.546+0000 7f385a195700 1 -- 172.21.15.17:0/1066493157 >> 172.21.15.17:0/1066493157 conn(0x7f385405d0c0 msgr2=0x7f385405f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:35.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.546+0000 7f385a195700 1 -- 172.21.15.17:0/1066493157 shutdown_connections 2022-08-01T19:41:35.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.546+0000 7f385a195700 1 -- 172.21.15.17:0/1066493157 wait complete. 2022-08-01T19:41:35.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.546+0000 7f385a195700 1 Processor -- start 2022-08-01T19:41:35.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.546+0000 7f385a195700 1 -- start start 2022-08-01T19:41:35.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.546+0000 7f385a195700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3854060290 0x7f38540b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:35.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.546+0000 7f385a195700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3854061060 0x7f3854146310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:35.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.546+0000 7f385a195700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38540b38d0 0x7f38540aac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:35.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.546+0000 7f385a195700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3854145ab0 con 0x7f38540b38d0 2022-08-01T19:41:35.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.546+0000 7f385a195700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3854146af0 con 0x7f3854061060 2022-08-01T19:41:35.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.546+0000 7f385a195700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38540afc30 con 0x7f3854060290 2022-08-01T19:41:35.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.546+0000 7f3853fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38540b38d0 0x7f38540aac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:35.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.546+0000 7f3853fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38540b38d0 0x7f38540aac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42280/0 (socket says 172.21.15.17:42280) 2022-08-01T19:41:35.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.546+0000 7f3853fff700 1 -- 172.21.15.17:0/2581542952 learned_addr learned my addr 172.21.15.17:0/2581542952 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:35.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.546+0000 7f38537fe700 1 --2- 172.21.15.17:0/2581542952 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3854060290 0x7f38540b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:35.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.546+0000 7f3852ffd700 1 --2- 172.21.15.17:0/2581542952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3854061060 0x7f3854146310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:35.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.546+0000 7f3853fff700 1 -- 172.21.15.17:0/2581542952 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3854060290 msgr2=0x7f38540b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:35.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.546+0000 7f3853fff700 1 --2- 172.21.15.17:0/2581542952 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3854060290 0x7f38540b17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:35.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.546+0000 7f3853fff700 1 -- 172.21.15.17:0/2581542952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3854061060 msgr2=0x7f3854146310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:35.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.546+0000 7f3853fff700 1 --2- 172.21.15.17:0/2581542952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3854061060 0x7f3854146310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:35.602 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.546+0000 7f3853fff700 1 -- 172.21.15.17:0/2581542952 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384c00b180 con 0x7f38540b38d0 2022-08-01T19:41:35.602 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.546+0000 7f3853fff700 1 --2- 172.21.15.17:0/2581542952 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38540b38d0 0x7f38540aac00 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 rx=0x7f384c002f70 tx=0x7f384c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:35.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.550+0000 7f3850ff9700 1 -- 172.21.15.17:0/2581542952 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f384c00d990 con 0x7f38540b38d0 2022-08-01T19:41:35.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.550+0000 7f3850ff9700 1 -- 172.21.15.17:0/2581542952 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384c011b80 con 0x7f38540b38d0 2022-08-01T19:41:35.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.550+0000 7f385a195700 1 -- 172.21.15.17:0/2581542952 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38540afeb0 con 0x7f38540b38d0 2022-08-01T19:41:35.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.550+0000 7f3850ff9700 1 -- 172.21.15.17:0/2581542952 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f384c01e460 con 0x7f38540b38d0 2022-08-01T19:41:35.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.550+0000 7f385a195700 1 -- 172.21.15.17:0/2581542952 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38540ab5a0 con 0x7f38540b38d0 2022-08-01T19:41:35.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.550+0000 7f385a195700 1 -- 172.21.15.17:0/2581542952 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3854146f20 con 0x7f38540b38d0 2022-08-01T19:41:35.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.550+0000 7f3850ff9700 1 -- 172.21.15.17:0/2581542952 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f384c025030 con 0x7f38540b38d0 2022-08-01T19:41:35.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.550+0000 7f3850ff9700 1 --2- 172.21.15.17:0/2581542952 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f382c053a70 0x7f382c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:35.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.550+0000 7f3850ff9700 1 -- 172.21.15.17:0/2581542952 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f384c06d460 con 0x7f38540b38d0 2022-08-01T19:41:35.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.550+0000 7f38537fe700 1 --2- 172.21.15.17:0/2581542952 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f382c053a70 0x7f382c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:35.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.554+0000 7f38537fe700 1 --2- 172.21.15.17:0/2581542952 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f382c053a70 0x7f382c055f30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f384400d000 tx=0x7f3844007f10).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:35.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.558+0000 7f3850ff9700 1 -- 172.21.15.17:0/2581542952 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f384c044110 con 0x7f38540b38d0 2022-08-01T19:41:35.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.722+0000 7f385a195700 1 -- 172.21.15.17:0/2581542952 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38540658a0 con 0x7f38540b38d0 2022-08-01T19:41:35.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.722+0000 7f3850ff9700 1 -- 172.21.15.17:0/2581542952 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f384c023090 con 0x7f38540b38d0 2022-08-01T19:41:35.725 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:35.725 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":46,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6943,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/256800053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":256800053},{"type":"v1","addr":"172.21.15.77:6835","nonce":256800053}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44},{"gid":6957,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3629116150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3629116150},{"type":"v1","addr":"172.21.15.17:6833","nonce":3629116150}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:41:24.477490+0000","modified":"2022-08-01T19:41:26.186850+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6960},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6960":{"gid":6960,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/2788758172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2788758172},{"type":"v1","addr":"172.21.15.17:6835","nonce":2788758172}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2022-08-01T19:41:35.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.722+0000 7f385a195700 1 -- 172.21.15.17:0/2581542952 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f382c053a70 msgr2=0x7f382c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:35.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.722+0000 7f385a195700 1 --2- 172.21.15.17:0/2581542952 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f382c053a70 0x7f382c055f30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f384400d000 tx=0x7f3844007f10).stop 2022-08-01T19:41:35.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.722+0000 7f385a195700 1 -- 172.21.15.17:0/2581542952 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38540b38d0 msgr2=0x7f38540aac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:35.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.722+0000 7f385a195700 1 --2- 172.21.15.17:0/2581542952 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38540b38d0 0x7f38540aac00 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 rx=0x7f384c002f70 tx=0x7f384c015000).stop 2022-08-01T19:41:35.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.722+0000 7f385a195700 1 -- 172.21.15.17:0/2581542952 shutdown_connections 2022-08-01T19:41:35.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.722+0000 7f385a195700 1 --2- 172.21.15.17:0/2581542952 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f382c053a70 0x7f382c055f30 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:35.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.722+0000 7f385a195700 1 --2- 172.21.15.17:0/2581542952 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38540b38d0 0x7f38540aac00 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:35.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.722+0000 7f385a195700 1 --2- 172.21.15.17:0/2581542952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3854061060 0x7f3854146310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:35.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.722+0000 7f385a195700 1 --2- 172.21.15.17:0/2581542952 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3854060290 0x7f38540b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:35.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.722+0000 7f385a195700 1 -- 172.21.15.17:0/2581542952 >> 172.21.15.17:0/2581542952 conn(0x7f385405d0c0 msgr2=0x7f38540b7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:35.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.722+0000 7f385a195700 1 -- 172.21.15.17:0/2581542952 shutdown_connections 2022-08-01T19:41:35.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.722+0000 7f385a195700 1 -- 172.21.15.17:0/2581542952 wait complete. 2022-08-01T19:41:35.747 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 46 2022-08-01T19:41:35.748 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-08-01T19:41:35.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.862+0000 7fd5521c1700 1 Processor -- start 2022-08-01T19:41:35.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.862+0000 7fd5521c1700 1 -- start start 2022-08-01T19:41:35.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.866+0000 7fd5521c1700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd54c0b9870 0x7fd54c0aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:35.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.866+0000 7fd5521c1700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd54c067690 con 0x7fd54c140180 2022-08-01T19:41:35.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.866+0000 7fd5521c1700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd54c0af660 con 0x7fd54c0672b0 2022-08-01T19:41:35.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.866+0000 7fd5521c1700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c143210 con 0x7fd54c0b9870 2022-08-01T19:41:35.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.866+0000 7fd54bfff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd54c140180 0x7fd54c140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34482/0 (socket says 172.21.15.17:34482) 2022-08-01T19:41:35.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.866+0000 7fd54bfff700 1 -- 172.21.15.17:0/1486797084 learned_addr learned my addr 172.21.15.17:0/1486797084 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:35.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.866+0000 7fd543fff700 1 --2- 172.21.15.17:0/1486797084 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd54c0b9870 0x7fd54c0aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:35.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.866+0000 7fd54affd700 1 -- 172.21.15.17:0/1486797084 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 856294996 0 0) 0x7fd54c067690 con 0x7fd54c140180 2022-08-01T19:41:35.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.866+0000 7fd54affd700 1 -- 172.21.15.17:0/1486797084 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd52c003260 con 0x7fd54c140180 2022-08-01T19:41:35.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.866+0000 7fd543fff700 1 -- 172.21.15.17:0/1486797084 >> v1:172.21.15.77:6789/0 conn(0x7fd54c0672b0 legacy=0x7fd54c143b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:35.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.866+0000 7fd54affd700 1 -- 172.21.15.17:0/1486797084 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3248326232 0 0) 0x7fd54c0af660 con 0x7fd54c0672b0 2022-08-01T19:41:35.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.866+0000 7fd543fff700 1 -- 172.21.15.17:0/1486797084 >> v1:172.21.15.17:6789/0 conn(0x7fd54c140180 legacy=0x7fd54c140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:35.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.866+0000 7fd543fff700 1 -- 172.21.15.17:0/1486797084 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd54c143400 con 0x7fd54c0b9870 2022-08-01T19:41:35.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.866+0000 7fd543fff700 1 --2- 172.21.15.17:0/1486797084 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd54c0b9870 0x7fd54c0aed90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fd544001b80 tx=0x7fd534006df0).ready entity=mon.2 client_cookie=a538c90ed14f405b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:35.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.866+0000 7fd54affd700 1 -- 172.21.15.17:0/1486797084 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5440019e0 con 0x7fd54c0b9870 2022-08-01T19:41:35.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.866+0000 7fd54affd700 1 -- 172.21.15.17:0/1486797084 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd534000e90 con 0x7fd54c0b9870 2022-08-01T19:41:35.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.866+0000 7fd54affd700 1 -- 172.21.15.17:0/1486797084 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd53400d4a0 con 0x7fd54c0b9870 2022-08-01T19:41:35.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.866+0000 7fd5521c1700 1 -- 172.21.15.17:0/1486797084 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd54c0b9870 msgr2=0x7fd54c0aed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:35.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.866+0000 7fd5521c1700 1 --2- 172.21.15.17:0/1486797084 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd54c0b9870 0x7fd54c0aed90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fd544001b80 tx=0x7fd534006df0).stop 2022-08-01T19:41:35.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.866+0000 7fd5521c1700 1 -- 172.21.15.17:0/1486797084 shutdown_connections 2022-08-01T19:41:35.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.866+0000 7fd5521c1700 1 --2- 172.21.15.17:0/1486797084 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd54c0b9870 0x7fd54c0aed90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:35.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.866+0000 7fd5521c1700 1 -- 172.21.15.17:0/1486797084 >> 172.21.15.17:0/1486797084 conn(0x7fd54c05fb80 msgr2=0x7fd54c061830 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:35.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.866+0000 7fd5521c1700 1 -- 172.21.15.17:0/1486797084 shutdown_connections 2022-08-01T19:41:35.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.866+0000 7fd5521c1700 1 -- 172.21.15.17:0/1486797084 wait complete. 2022-08-01T19:41:35.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.870+0000 7fd5521c1700 1 Processor -- start 2022-08-01T19:41:35.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.870+0000 7fd5521c1700 1 -- start start 2022-08-01T19:41:35.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.870+0000 7fd5521c1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd54c0672b0 0x7fd54c148b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:35.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.870+0000 7fd5521c1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd54c0b9870 0x7fd54c0b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:35.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.870+0000 7fd5521c1700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd54c140180 0x7fd54c0b3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:35.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.870+0000 7fd5521c1700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c0affa0 con 0x7fd54c0b9870 2022-08-01T19:41:35.899 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.870+0000 7fd5521c1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c0b01d0 con 0x7fd54c0672b0 2022-08-01T19:41:35.899 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.870+0000 7fd5521c1700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c0b06b0 con 0x7fd54c140180 2022-08-01T19:41:35.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.870+0000 7fd543fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd54c0b9870 0x7fd54c0b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:35.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.870+0000 7fd543fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd54c0b9870 0x7fd54c0b31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42292/0 (socket says 172.21.15.17:42292) 2022-08-01T19:41:35.905 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.870+0000 7fd543fff700 1 -- 172.21.15.17:0/870200382 learned_addr learned my addr 172.21.15.17:0/870200382 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:35.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.870+0000 7fd54b7fe700 1 --2- 172.21.15.17:0/870200382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd54c0672b0 0x7fd54c148b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:35.910 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.870+0000 7fd54bfff700 1 --2- 172.21.15.17:0/870200382 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd54c140180 0x7fd54c0b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:35.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.870+0000 7fd543fff700 1 -- 172.21.15.17:0/870200382 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd54c140180 msgr2=0x7fd54c0b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:35.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.870+0000 7fd543fff700 1 --2- 172.21.15.17:0/870200382 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd54c140180 0x7fd54c0b3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:35.912 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.870+0000 7fd543fff700 1 -- 172.21.15.17:0/870200382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd54c0672b0 msgr2=0x7fd54c148b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:35.912 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.870+0000 7fd543fff700 1 --2- 172.21.15.17:0/870200382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd54c0672b0 0x7fd54c148b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:35.913 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.870+0000 7fd543fff700 1 -- 172.21.15.17:0/870200382 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd53400b180 con 0x7fd54c0b9870 2022-08-01T19:41:35.914 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.870+0000 7fd54b7fe700 1 --2- 172.21.15.17:0/870200382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd54c0672b0 0x7fd54c148b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:35.914 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.870+0000 7fd543fff700 1 --2- 172.21.15.17:0/870200382 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd54c0b9870 0x7fd54c0b31a0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 rx=0x7fd544001b80 tx=0x7fd54c0b6a00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:35.915 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.870+0000 7fd5497fa700 1 -- 172.21.15.17:0/870200382 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd534015070 con 0x7fd54c0b9870 2022-08-01T19:41:35.916 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.870+0000 7fd5497fa700 1 -- 172.21.15.17:0/870200382 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd534002f50 con 0x7fd54c0b9870 2022-08-01T19:41:35.916 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.870+0000 7fd5521c1700 1 -- 172.21.15.17:0/870200382 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd54c0b0950 con 0x7fd54c0b9870 2022-08-01T19:41:35.917 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.870+0000 7fd5497fa700 1 -- 172.21.15.17:0/870200382 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd53400d4a0 con 0x7fd54c0b9870 2022-08-01T19:41:35.917 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.870+0000 7fd5521c1700 1 -- 172.21.15.17:0/870200382 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd54c0b90c0 con 0x7fd54c0b9870 2022-08-01T19:41:35.918 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.874+0000 7fd5521c1700 1 -- 172.21.15.17:0/870200382 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd54c1436a0 con 0x7fd54c0b9870 2022-08-01T19:41:35.918 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.874+0000 7fd5497fa700 1 -- 172.21.15.17:0/870200382 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd534024030 con 0x7fd54c0b9870 2022-08-01T19:41:35.919 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.874+0000 7fd5497fa700 1 --2- 172.21.15.17:0/870200382 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd530053b00 0x7fd530055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:35.919 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.874+0000 7fd5497fa700 1 -- 172.21.15.17:0/870200382 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd53406ccf0 con 0x7fd54c0b9870 2022-08-01T19:41:35.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.874+0000 7fd54b7fe700 1 --2- 172.21.15.17:0/870200382 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd530053b00 0x7fd530055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:35.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.874+0000 7fd54b7fe700 1 --2- 172.21.15.17:0/870200382 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd530053b00 0x7fd530055fc0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7fd53c00b000 tx=0x7fd53c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:35.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:35.878+0000 7fd5497fa700 1 -- 172.21.15.17:0/870200382 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd534043ab0 con 0x7fd54c0b9870 2022-08-01T19:41:36.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.034+0000 7fd5521c1700 1 -- 172.21.15.17:0/870200382 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd54c0574e0 con 0x7fd54c0b9870 2022-08-01T19:41:36.256 INFO:tasks.ceph.mon.a.smithi017.stderr:2022-08-01T19:41:36.250+0000 7f6a53773700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-08-01T19:41:36.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.262+0000 7fd5497fa700 1 -- 172.21.15.17:0/870200382 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v47) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd534022070 con 0x7fd54c0b9870 2022-08-01T19:41:36.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.266+0000 7fd5521c1700 1 -- 172.21.15.17:0/870200382 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd530053b00 msgr2=0x7fd530055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:36.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.266+0000 7fd5521c1700 1 --2- 172.21.15.17:0/870200382 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd530053b00 0x7fd530055fc0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7fd53c00b000 tx=0x7fd53c009040).stop 2022-08-01T19:41:36.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.266+0000 7fd5521c1700 1 -- 172.21.15.17:0/870200382 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd54c0b9870 msgr2=0x7fd54c0b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:36.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.266+0000 7fd5521c1700 1 --2- 172.21.15.17:0/870200382 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd54c0b9870 0x7fd54c0b31a0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 rx=0x7fd544001b80 tx=0x7fd54c0b6a00).stop 2022-08-01T19:41:36.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.266+0000 7fd5521c1700 1 -- 172.21.15.17:0/870200382 shutdown_connections 2022-08-01T19:41:36.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.266+0000 7fd5521c1700 1 --2- 172.21.15.17:0/870200382 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd530053b00 0x7fd530055fc0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:36.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.266+0000 7fd5521c1700 1 --2- 172.21.15.17:0/870200382 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd54c140180 0x7fd54c0b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:36.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.266+0000 7fd5521c1700 1 --2- 172.21.15.17:0/870200382 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd54c0b9870 0x7fd54c0b31a0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:36.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.266+0000 7fd5521c1700 1 --2- 172.21.15.17:0/870200382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd54c0672b0 0x7fd54c148b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:36.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.266+0000 7fd5521c1700 1 -- 172.21.15.17:0/870200382 >> 172.21.15.17:0/870200382 conn(0x7fd54c05fb80 msgr2=0x7fd54c0b3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:36.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.266+0000 7fd5521c1700 1 -- 172.21.15.17:0/870200382 shutdown_connections 2022-08-01T19:41:36.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.266+0000 7fd5521c1700 1 -- 172.21.15.17:0/870200382 wait complete. 2022-08-01T19:41:36.285 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-08-01T19:41:36.287 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:41:36.307 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2022-08-01T19:41:36.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.394+0000 7ffaebfff700 1 Processor -- start 2022-08-01T19:41:36.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.394+0000 7ffaebfff700 1 -- start start 2022-08-01T19:41:36.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.398+0000 7ffaebfff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ffaec0b31f0 0x7ffaec0b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:36.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.398+0000 7ffaebfff700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaec0b2510 con 0x7ffaec0b7500 2022-08-01T19:41:36.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.398+0000 7ffaebfff700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaec1443e0 con 0x7ffaec0aadb0 2022-08-01T19:41:36.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.398+0000 7ffaebfff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaec0b3700 con 0x7ffaec0b31f0 2022-08-01T19:41:36.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.398+0000 7ffaeaffd700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ffaec0b7500 0x7ffaec13e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34510/0 (socket says 172.21.15.17:34510) 2022-08-01T19:41:36.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.398+0000 7ffaeaffd700 1 -- 172.21.15.17:0/2125009555 learned_addr learned my addr 172.21.15.17:0/2125009555 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:36.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.398+0000 7ffae3fff700 1 --2- 172.21.15.17:0/2125009555 >> v2:172.21.15.77:3300/0 conn(0x7ffaec0b31f0 0x7ffaec0b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:36.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.398+0000 7ffaea7fc700 1 -- 172.21.15.17:0/2125009555 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2412133296 0 0) 0x7ffaec0b2510 con 0x7ffaec0b7500 2022-08-01T19:41:36.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.398+0000 7ffaea7fc700 1 -- 172.21.15.17:0/2125009555 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffacc003260 con 0x7ffaec0b7500 2022-08-01T19:41:36.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.398+0000 7ffaea7fc700 1 -- 172.21.15.17:0/2125009555 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 490191567 0 0) 0x7ffaec1443e0 con 0x7ffaec0aadb0 2022-08-01T19:41:36.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.398+0000 7ffaea7fc700 1 -- 172.21.15.17:0/2125009555 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffaec0b2510 con 0x7ffaec0aadb0 2022-08-01T19:41:36.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.398+0000 7ffae3fff700 1 -- 172.21.15.17:0/2125009555 >> v1:172.21.15.77:6789/0 conn(0x7ffaec0aadb0 legacy=0x7ffaec064d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:36.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.398+0000 7ffae3fff700 1 -- 172.21.15.17:0/2125009555 >> v1:172.21.15.17:6789/0 conn(0x7ffaec0b7500 legacy=0x7ffaec13e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:36.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.398+0000 7ffae3fff700 1 -- 172.21.15.17:0/2125009555 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaec0b2740 con 0x7ffaec0b31f0 2022-08-01T19:41:36.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.398+0000 7ffae3fff700 1 --2- 172.21.15.17:0/2125009555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffaec0b31f0 0x7ffaec0b78e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7ffad4002f70 tx=0x7ffad4006df0).ready entity=mon.1 client_cookie=d57e2948459b73f0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:36.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.398+0000 7ffaea7fc700 1 -- 172.21.15.17:0/2125009555 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffad400d540 con 0x7ffaec0b31f0 2022-08-01T19:41:36.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.398+0000 7ffaea7fc700 1 -- 172.21.15.17:0/2125009555 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad4000e90 con 0x7ffaec0b31f0 2022-08-01T19:41:36.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.398+0000 7ffaea7fc700 1 -- 172.21.15.17:0/2125009555 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffad400d890 con 0x7ffaec0b31f0 2022-08-01T19:41:36.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.398+0000 7ffaebfff700 1 -- 172.21.15.17:0/2125009555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffaec0b31f0 msgr2=0x7ffaec0b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:36.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.398+0000 7ffaebfff700 1 --2- 172.21.15.17:0/2125009555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffaec0b31f0 0x7ffaec0b78e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7ffad4002f70 tx=0x7ffad4006df0).stop 2022-08-01T19:41:36.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.398+0000 7ffaebfff700 1 -- 172.21.15.17:0/2125009555 shutdown_connections 2022-08-01T19:41:36.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.398+0000 7ffaebfff700 1 --2- 172.21.15.17:0/2125009555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffaec0b31f0 0x7ffaec0b78e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:36.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.398+0000 7ffaebfff700 1 -- 172.21.15.17:0/2125009555 >> 172.21.15.17:0/2125009555 conn(0x7ffaec05f840 msgr2=0x7ffaec05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:36.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.398+0000 7ffaebfff700 1 -- 172.21.15.17:0/2125009555 shutdown_connections 2022-08-01T19:41:36.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.398+0000 7ffaebfff700 1 -- 172.21.15.17:0/2125009555 wait complete. 2022-08-01T19:41:36.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.402+0000 7ffaebfff700 1 Processor -- start 2022-08-01T19:41:36.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.402+0000 7ffaebfff700 1 -- start start 2022-08-01T19:41:36.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.402+0000 7ffaebfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffaec0aadb0 0x7ffaec0adc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:36.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.402+0000 7ffaebfff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffaec0b31f0 0x7ffaec0ab800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:36.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.402+0000 7ffaebfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffaec0b7500 0x7ffaec060950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:36.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.402+0000 7ffaebfff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaec142400 con 0x7ffaec0b7500 2022-08-01T19:41:36.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.402+0000 7ffaebfff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaec143720 con 0x7ffaec0aadb0 2022-08-01T19:41:36.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.402+0000 7ffaebfff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaec0ae4b0 con 0x7ffaec0b31f0 2022-08-01T19:41:36.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.402+0000 7ffaeb7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffaec0b7500 0x7ffaec060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:36.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.402+0000 7ffaeaffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffaec0aadb0 0x7ffaec0adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:36.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.402+0000 7ffaeb7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffaec0b7500 0x7ffaec060950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42326/0 (socket says 172.21.15.17:42326) 2022-08-01T19:41:36.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.402+0000 7ffae3fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffaec0b31f0 0x7ffaec0ab800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:36.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.402+0000 7ffaeb7fe700 1 -- 172.21.15.17:0/2066069410 learned_addr learned my addr 172.21.15.17:0/2066069410 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:36.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.402+0000 7ffaeb7fe700 1 -- 172.21.15.17:0/2066069410 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffaec0b31f0 msgr2=0x7ffaec0ab800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:36.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.402+0000 7ffaeb7fe700 1 --2- 172.21.15.17:0/2066069410 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffaec0b31f0 0x7ffaec0ab800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:36.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.402+0000 7ffaeb7fe700 1 -- 172.21.15.17:0/2066069410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffaec0aadb0 msgr2=0x7ffaec0adc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:36.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.402+0000 7ffaeb7fe700 1 --2- 172.21.15.17:0/2066069410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffaec0aadb0 0x7ffaec0adc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:36.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.402+0000 7ffaeb7fe700 1 -- 172.21.15.17:0/2066069410 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad400b180 con 0x7ffaec0b7500 2022-08-01T19:41:36.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.402+0000 7ffaeb7fe700 1 --2- 172.21.15.17:0/2066069410 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffaec0b7500 0x7ffaec060950 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0x7ffae4001490 tx=0x7ffae4007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:36.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.402+0000 7ffae8ff9700 1 -- 172.21.15.17:0/2066069410 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffae4002240 con 0x7ffaec0b7500 2022-08-01T19:41:36.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.402+0000 7ffae8ff9700 1 -- 172.21.15.17:0/2066069410 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae40023a0 con 0x7ffaec0b7500 2022-08-01T19:41:36.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.402+0000 7ffaebfff700 1 -- 172.21.15.17:0/2066069410 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaec0624e0 con 0x7ffaec0b7500 2022-08-01T19:41:36.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.402+0000 7ffae8ff9700 1 -- 172.21.15.17:0/2066069410 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffae4001650 con 0x7ffaec0b7500 2022-08-01T19:41:36.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.406+0000 7ffaebfff700 1 -- 172.21.15.17:0/2066069410 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaec0ae7b0 con 0x7ffaec0b7500 2022-08-01T19:41:36.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.406+0000 7ffaebfff700 1 -- 172.21.15.17:0/2066069410 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaec0ae320 con 0x7ffaec0b7500 2022-08-01T19:41:36.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.406+0000 7ffae8ff9700 1 -- 172.21.15.17:0/2066069410 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ffae4010030 con 0x7ffaec0b7500 2022-08-01T19:41:36.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.406+0000 7ffae8ff9700 1 --2- 172.21.15.17:0/2066069410 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ffad0053a70 0x7ffad0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:36.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.406+0000 7ffae8ff9700 1 -- 172.21.15.17:0/2066069410 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6506+0+0 (secure 0 0 0) 0x7ffae406bf60 con 0x7ffaec0b7500 2022-08-01T19:41:36.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.406+0000 7ffaeaffd700 1 --2- 172.21.15.17:0/2066069410 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ffad0053a70 0x7ffad0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:36.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.410+0000 7ffaeaffd700 1 --2- 172.21.15.17:0/2066069410 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ffad0053a70 0x7ffad0055f30 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7ffadc00d000 tx=0x7ffadc009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:36.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.414+0000 7ffae8ff9700 1 -- 172.21.15.17:0/2066069410 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ffae4042c30 con 0x7ffaec0b7500 2022-08-01T19:41:36.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.574+0000 7ffaebfff700 1 -- 172.21.15.17:0/2066069410 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffaec0574e0 con 0x7ffaec0b7500 2022-08-01T19:41:36.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.574+0000 7ffae8ff9700 1 -- 172.21.15.17:0/2066069410 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+2129 (secure 0 0 0) 0x7ffae4046250 con 0x7ffaec0b7500 2022-08-01T19:41:36.581 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:36.582 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":47,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6943,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/256800053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":256800053},{"type":"v1","addr":"172.21.15.77:6835","nonce":256800053}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44},{"gid":6957,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3629116150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3629116150},{"type":"v1","addr":"172.21.15.17:6833","nonce":3629116150}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43}],"filesystems":[{"mdsmap":{"epoch":47,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:41:24.477490+0000","modified":"2022-08-01T19:41:36.251964+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2022-08-01T19:41:36.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.578+0000 7ffaebfff700 1 -- 172.21.15.17:0/2066069410 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ffad0053a70 msgr2=0x7ffad0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:36.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.578+0000 7ffaebfff700 1 --2- 172.21.15.17:0/2066069410 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ffad0053a70 0x7ffad0055f30 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7ffadc00d000 tx=0x7ffadc009040).stop 2022-08-01T19:41:36.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.578+0000 7ffaebfff700 1 -- 172.21.15.17:0/2066069410 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffaec0b7500 msgr2=0x7ffaec060950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:36.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.578+0000 7ffaebfff700 1 --2- 172.21.15.17:0/2066069410 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffaec0b7500 0x7ffaec060950 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0x7ffae4001490 tx=0x7ffae4007dd0).stop 2022-08-01T19:41:36.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.578+0000 7ffaebfff700 1 -- 172.21.15.17:0/2066069410 shutdown_connections 2022-08-01T19:41:36.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.578+0000 7ffaebfff700 1 --2- 172.21.15.17:0/2066069410 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ffad0053a70 0x7ffad0055f30 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:36.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.578+0000 7ffaebfff700 1 --2- 172.21.15.17:0/2066069410 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffaec0b7500 0x7ffaec060950 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:36.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.578+0000 7ffaebfff700 1 --2- 172.21.15.17:0/2066069410 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffaec0b31f0 0x7ffaec0ab800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:36.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.578+0000 7ffaebfff700 1 --2- 172.21.15.17:0/2066069410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffaec0aadb0 0x7ffaec0adc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:36.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.578+0000 7ffaebfff700 1 -- 172.21.15.17:0/2066069410 >> 172.21.15.17:0/2066069410 conn(0x7ffaec05f840 msgr2=0x7ffaec0b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:36.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.578+0000 7ffaebfff700 1 -- 172.21.15.17:0/2066069410 shutdown_connections 2022-08-01T19:41:36.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.578+0000 7ffaebfff700 1 -- 172.21.15.17:0/2066069410 wait complete. 2022-08-01T19:41:36.591 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 47 2022-08-01T19:41:36.602 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-08-01T19:41:36.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.718+0000 7f2981872700 1 Processor -- start 2022-08-01T19:41:36.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.718+0000 7f2981872700 1 -- start start 2022-08-01T19:41:36.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.718+0000 7f2981872700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f297c05fc30 0x7f297c143a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:36.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.718+0000 7f2981872700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f297c0ac690 0x7f297c143f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:36.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.718+0000 7f2981872700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f297c060010 con 0x7f297c060960 2022-08-01T19:41:36.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.718+0000 7f2981872700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f297c1433d0 con 0x7f297c05fc30 2022-08-01T19:41:36.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.718+0000 7f2981872700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f297c0acdf0 con 0x7f297c0ac690 2022-08-01T19:41:36.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.718+0000 7f297affd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f297c05fc30 0x7f297c143a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:36.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.718+0000 7f297affd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f297c05fc30 0x7f297c143a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42330/0 (socket says 172.21.15.17:42330) 2022-08-01T19:41:36.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.718+0000 7f297affd700 1 -- 172.21.15.17:0/1908914459 learned_addr learned my addr 172.21.15.17:0/1908914459 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:36.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.718+0000 7f297b7fe700 1 --2- 172.21.15.17:0/1908914459 >> v2:172.21.15.77:3300/0 conn(0x7f297c0ac690 0x7f297c143f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:36.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.718+0000 7f297affd700 1 -- 172.21.15.17:0/1908914459 >> v2:172.21.15.77:3300/0 conn(0x7f297c0ac690 msgr2=0x7f297c143f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:36.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.718+0000 7f297affd700 1 --2- 172.21.15.17:0/1908914459 >> v2:172.21.15.77:3300/0 conn(0x7f297c0ac690 0x7f297c143f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:36.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.718+0000 7f297affd700 1 -- 172.21.15.17:0/1908914459 >> v1:172.21.15.77:6789/0 conn(0x7f297c060960 legacy=0x7f297c1469a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:36.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.718+0000 7f297affd700 1 -- 172.21.15.17:0/1908914459 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f297c0ad090 con 0x7f297c05fc30 2022-08-01T19:41:36.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.718+0000 7f297b7fe700 1 --2- 172.21.15.17:0/1908914459 >> v2:172.21.15.77:3300/0 conn(0x7f297c0ac690 0x7f297c143f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:36.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.718+0000 7f297affd700 1 --2- 172.21.15.17:0/1908914459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f297c05fc30 0x7f297c143a60 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 rx=0x7f296c002f70 tx=0x7f296c006eb0).ready entity=mon.0 client_cookie=c1d5134d107b5292 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:36.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.718+0000 7f297a7fc700 1 -- 172.21.15.17:0/1908914459 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f296c00d610 con 0x7f297c05fc30 2022-08-01T19:41:36.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.718+0000 7f297a7fc700 1 -- 172.21.15.17:0/1908914459 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296c000e90 con 0x7f297c05fc30 2022-08-01T19:41:36.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.718+0000 7f297a7fc700 1 -- 172.21.15.17:0/1908914459 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f296c00d9a0 con 0x7f297c05fc30 2022-08-01T19:41:36.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.722+0000 7f2981872700 1 -- 172.21.15.17:0/1908914459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f297c05fc30 msgr2=0x7f297c143a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:36.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.722+0000 7f2981872700 1 --2- 172.21.15.17:0/1908914459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f297c05fc30 0x7f297c143a60 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 rx=0x7f296c002f70 tx=0x7f296c006eb0).stop 2022-08-01T19:41:36.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.722+0000 7f2981872700 1 -- 172.21.15.17:0/1908914459 shutdown_connections 2022-08-01T19:41:36.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.722+0000 7f2981872700 1 --2- 172.21.15.17:0/1908914459 >> v2:172.21.15.77:3300/0 conn(0x7f297c0ac690 0x7f297c143f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:36.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.722+0000 7f2981872700 1 --2- 172.21.15.17:0/1908914459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f297c05fc30 0x7f297c143a60 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:36.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.722+0000 7f2981872700 1 -- 172.21.15.17:0/1908914459 >> 172.21.15.17:0/1908914459 conn(0x7f297c0b2b00 msgr2=0x7f297c0b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:36.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.722+0000 7f2981872700 1 -- 172.21.15.17:0/1908914459 shutdown_connections 2022-08-01T19:41:36.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.722+0000 7f2981872700 1 -- 172.21.15.17:0/1908914459 wait complete. 2022-08-01T19:41:36.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.722+0000 7f2981872700 1 Processor -- start 2022-08-01T19:41:36.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.722+0000 7f2981872700 1 -- start start 2022-08-01T19:41:36.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.722+0000 7f2981872700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f297c05fc30 0x7f297c142720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:36.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.722+0000 7f2981872700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f297c060960 0x7f297c156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:36.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.722+0000 7f2981872700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f297c0ac690 0x7f297c156640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:36.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.722+0000 7f2981872700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297c144eb0 con 0x7f297c05fc30 2022-08-01T19:41:36.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.722+0000 7f2981872700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297c156b30 con 0x7f297c060960 2022-08-01T19:41:36.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.722+0000 7f2981872700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f297c0ae050 con 0x7f297c0ac690 2022-08-01T19:41:36.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.722+0000 7f297affd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f297c05fc30 0x7f297c142720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:36.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.722+0000 7f297affd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f297c05fc30 0x7f297c142720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42336/0 (socket says 172.21.15.17:42336) 2022-08-01T19:41:36.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.722+0000 7f297affd700 1 -- 172.21.15.17:0/27022566 learned_addr learned my addr 172.21.15.17:0/27022566 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:36.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.722+0000 7f297b7fe700 1 --2- 172.21.15.17:0/27022566 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f297c0ac690 0x7f297c156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:36.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.722+0000 7f2973fff700 1 --2- 172.21.15.17:0/27022566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f297c060960 0x7f297c156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:36.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.722+0000 7f297affd700 1 -- 172.21.15.17:0/27022566 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f297c0ac690 msgr2=0x7f297c156640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:36.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.722+0000 7f297affd700 1 --2- 172.21.15.17:0/27022566 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f297c0ac690 0x7f297c156640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:36.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.722+0000 7f297affd700 1 -- 172.21.15.17:0/27022566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f297c060960 msgr2=0x7f297c156150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:36.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.722+0000 7f297affd700 1 --2- 172.21.15.17:0/27022566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f297c060960 0x7f297c156150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:36.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.722+0000 7f297affd700 1 -- 172.21.15.17:0/27022566 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f296c00b1e0 con 0x7f297c05fc30 2022-08-01T19:41:36.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.722+0000 7f2973fff700 1 --2- 172.21.15.17:0/27022566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f297c060960 0x7f297c156150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:41:36.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.722+0000 7f297affd700 1 --2- 172.21.15.17:0/27022566 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f297c05fc30 0x7f297c142720 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 rx=0x7f296c002f70 tx=0x7f296c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:36.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.722+0000 7f2978ff9700 1 -- 172.21.15.17:0/27022566 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f296c011790 con 0x7f297c05fc30 2022-08-01T19:41:36.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.722+0000 7f2978ff9700 1 -- 172.21.15.17:0/27022566 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296c0118f0 con 0x7f297c05fc30 2022-08-01T19:41:36.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.722+0000 7f2981872700 1 -- 172.21.15.17:0/27022566 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f297c0ae2f0 con 0x7f297c05fc30 2022-08-01T19:41:36.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.722+0000 7f2978ff9700 1 -- 172.21.15.17:0/27022566 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f296c011c50 con 0x7f297c05fc30 2022-08-01T19:41:36.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.722+0000 7f2981872700 1 -- 172.21.15.17:0/27022566 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f297c155cb0 con 0x7f297c05fc30 2022-08-01T19:41:36.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.726+0000 7f2981872700 1 -- 172.21.15.17:0/27022566 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f297c155e70 con 0x7f297c05fc30 2022-08-01T19:41:36.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.726+0000 7f2978ff9700 1 -- 172.21.15.17:0/27022566 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f296c01a040 con 0x7f297c05fc30 2022-08-01T19:41:36.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.726+0000 7f2978ff9700 1 --2- 172.21.15.17:0/27022566 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f29600539e0 0x7f2960055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:36.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.726+0000 7f2978ff9700 1 -- 172.21.15.17:0/27022566 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f296c06cf50 con 0x7f297c05fc30 2022-08-01T19:41:36.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.726+0000 7f2973fff700 1 --2- 172.21.15.17:0/27022566 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f29600539e0 0x7f2960055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:36.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.726+0000 7f2973fff700 1 --2- 172.21.15.17:0/27022566 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f29600539e0 0x7f2960055ea0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f2964008000 tx=0x7f2964009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:36.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.734+0000 7f2978ff9700 1 -- 172.21.15.17:0/27022566 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f296c043c70 con 0x7f297c05fc30 2022-08-01T19:41:36.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:36.894+0000 7f2981872700 1 -- 172.21.15.17:0/27022566 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f297c0574e0 con 0x7f297c05fc30 2022-08-01T19:41:37.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.274+0000 7f2978ff9700 1 -- 172.21.15.17:0/27022566 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v48) v1 ==== 104+0+0 (secure 0 0 0) 0x7f296c0230a0 con 0x7f297c05fc30 2022-08-01T19:41:37.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.278+0000 7f2981872700 1 -- 172.21.15.17:0/27022566 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f29600539e0 msgr2=0x7f2960055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:37.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.278+0000 7f2981872700 1 --2- 172.21.15.17:0/27022566 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f29600539e0 0x7f2960055ea0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f2964008000 tx=0x7f2964009040).stop 2022-08-01T19:41:37.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.278+0000 7f2981872700 1 -- 172.21.15.17:0/27022566 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f297c05fc30 msgr2=0x7f297c142720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:37.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.278+0000 7f2981872700 1 --2- 172.21.15.17:0/27022566 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f297c05fc30 0x7f297c142720 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 rx=0x7f296c002f70 tx=0x7f296c015000).stop 2022-08-01T19:41:37.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.278+0000 7f2981872700 1 -- 172.21.15.17:0/27022566 shutdown_connections 2022-08-01T19:41:37.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.278+0000 7f2981872700 1 --2- 172.21.15.17:0/27022566 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f29600539e0 0x7f2960055ea0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:37.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.278+0000 7f2981872700 1 --2- 172.21.15.17:0/27022566 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f297c0ac690 0x7f297c156640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:37.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.278+0000 7f2981872700 1 --2- 172.21.15.17:0/27022566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f297c060960 0x7f297c156150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:37.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.278+0000 7f2981872700 1 --2- 172.21.15.17:0/27022566 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f297c05fc30 0x7f297c142720 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:37.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.278+0000 7f2981872700 1 -- 172.21.15.17:0/27022566 >> 172.21.15.17:0/27022566 conn(0x7f297c0b2b00 msgr2=0x7f297c0b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:37.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.278+0000 7f2981872700 1 -- 172.21.15.17:0/27022566 shutdown_connections 2022-08-01T19:41:37.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.278+0000 7f2981872700 1 -- 172.21.15.17:0/27022566 wait complete. 2022-08-01T19:41:37.296 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-08-01T19:41:37.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.418+0000 7f9f0b38d700 1 Processor -- start 2022-08-01T19:41:37.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.418+0000 7f9f0b38d700 1 -- start start 2022-08-01T19:41:37.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.418+0000 7f9f0b38d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9f040b0630 0x7f9f040adec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:37.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.418+0000 7f9f0b38d700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f9f040b0a10 0x7f9f04140730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:37.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.418+0000 7f9f0b38d700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f040b2c80 con 0x7f9f040aa2b0 2022-08-01T19:41:37.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.418+0000 7f9f0b38d700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f040b2900 con 0x7f9f040b0630 2022-08-01T19:41:37.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.418+0000 7f9f0b38d700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f040b2eb0 con 0x7f9f040b0a10 2022-08-01T19:41:37.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.418+0000 7f9f09129700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9f040b0630 0x7f9f040adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:37.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.418+0000 7f9f09129700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9f040b0630 0x7f9f040adec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42344/0 (socket says 172.21.15.17:42344) 2022-08-01T19:41:37.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.418+0000 7f9f09129700 1 -- 172.21.15.17:0/1774863791 learned_addr learned my addr 172.21.15.17:0/1774863791 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:37.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.418+0000 7f9f0992a700 1 --2- 172.21.15.17:0/1774863791 >> v2:172.21.15.77:3300/0 conn(0x7f9f040b0a10 0x7f9f04140730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:37.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.422+0000 7f9f09129700 1 -- 172.21.15.17:0/1774863791 >> v2:172.21.15.77:3300/0 conn(0x7f9f040b0a10 msgr2=0x7f9f04140730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:37.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.422+0000 7f9f09129700 1 --2- 172.21.15.17:0/1774863791 >> v2:172.21.15.77:3300/0 conn(0x7f9f040b0a10 0x7f9f04140730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:37.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.422+0000 7f9f09129700 1 -- 172.21.15.17:0/1774863791 >> v1:172.21.15.77:6789/0 conn(0x7f9f040aa2b0 legacy=0x7f9f040ae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:37.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.422+0000 7f9f09129700 1 -- 172.21.15.17:0/1774863791 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f040a9d00 con 0x7f9f040b0630 2022-08-01T19:41:37.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.422+0000 7f9f09129700 1 --2- 172.21.15.17:0/1774863791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f040b0630 0x7f9f040adec0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 rx=0x7f9ef4002f70 tx=0x7f9ef4006eb0).ready entity=mon.0 client_cookie=6f45632aabcf242a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:37.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.422+0000 7f9ef3fff700 1 -- 172.21.15.17:0/1774863791 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ef400d5c0 con 0x7f9f040b0630 2022-08-01T19:41:37.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.422+0000 7f9ef3fff700 1 -- 172.21.15.17:0/1774863791 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef4000e90 con 0x7f9f040b0630 2022-08-01T19:41:37.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.422+0000 7f9ef3fff700 1 -- 172.21.15.17:0/1774863791 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ef400d950 con 0x7f9f040b0630 2022-08-01T19:41:37.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.422+0000 7f9f0b38d700 1 -- 172.21.15.17:0/1774863791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f040b0630 msgr2=0x7f9f040adec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:37.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.422+0000 7f9f0b38d700 1 --2- 172.21.15.17:0/1774863791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f040b0630 0x7f9f040adec0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 rx=0x7f9ef4002f70 tx=0x7f9ef4006eb0).stop 2022-08-01T19:41:37.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.422+0000 7f9f0b38d700 1 -- 172.21.15.17:0/1774863791 shutdown_connections 2022-08-01T19:41:37.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.422+0000 7f9f0b38d700 1 --2- 172.21.15.17:0/1774863791 >> v2:172.21.15.77:3300/0 conn(0x7f9f040b0a10 0x7f9f04140730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:37.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.422+0000 7f9f0b38d700 1 --2- 172.21.15.17:0/1774863791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f040b0630 0x7f9f040adec0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:37.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.422+0000 7f9f0b38d700 1 -- 172.21.15.17:0/1774863791 >> 172.21.15.17:0/1774863791 conn(0x7f9f04060950 msgr2=0x7f9f04060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:37.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.422+0000 7f9f0b38d700 1 -- 172.21.15.17:0/1774863791 shutdown_connections 2022-08-01T19:41:37.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.422+0000 7f9f0b38d700 1 -- 172.21.15.17:0/1774863791 wait complete. 2022-08-01T19:41:37.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.422+0000 7f9f0b38d700 1 Processor -- start 2022-08-01T19:41:37.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.422+0000 7f9f0b38d700 1 -- start start 2022-08-01T19:41:37.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.426+0000 7f9f0b38d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f040aa2b0 0x7f9f040ae070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:37.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.426+0000 7f9f0b38d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9f040b0630 0x7f9f04153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:37.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.426+0000 7f9f0b38d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f040b0a10 0x7f9f04154190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:37.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.426+0000 7f9f0b38d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0413cd80 con 0x7f9f040aa2b0 2022-08-01T19:41:37.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.426+0000 7f9f0b38d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f04143370 con 0x7f9f040b0a10 2022-08-01T19:41:37.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.426+0000 7f9f0b38d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f04143610 con 0x7f9f040b0630 2022-08-01T19:41:37.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.426+0000 7f9f09129700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f040aa2b0 0x7f9f040ae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:37.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.426+0000 7f9f09129700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f040aa2b0 0x7f9f040ae070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42352/0 (socket says 172.21.15.17:42352) 2022-08-01T19:41:37.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.426+0000 7f9f09129700 1 -- 172.21.15.17:0/2395857279 learned_addr learned my addr 172.21.15.17:0/2395857279 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:37.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.426+0000 7f9f0992a700 1 --2- 172.21.15.17:0/2395857279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f040b0a10 0x7f9f04154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:37.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.426+0000 7f9f08928700 1 --2- 172.21.15.17:0/2395857279 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9f040b0630 0x7f9f04153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:37.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.426+0000 7f9f09129700 1 -- 172.21.15.17:0/2395857279 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9f040b0630 msgr2=0x7f9f04153ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:37.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.426+0000 7f9f09129700 1 --2- 172.21.15.17:0/2395857279 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9f040b0630 0x7f9f04153ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:37.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.426+0000 7f9f09129700 1 -- 172.21.15.17:0/2395857279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f040b0a10 msgr2=0x7f9f04154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:37.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.426+0000 7f9f09129700 1 --2- 172.21.15.17:0/2395857279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f040b0a10 0x7f9f04154190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:37.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.426+0000 7f9f09129700 1 -- 172.21.15.17:0/2395857279 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef400b1e0 con 0x7f9f040aa2b0 2022-08-01T19:41:37.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.426+0000 7f9f08928700 1 --2- 172.21.15.17:0/2395857279 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9f040b0630 0x7f9f04153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:37.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.426+0000 7f9f09129700 1 --2- 172.21.15.17:0/2395857279 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f040aa2b0 0x7f9f040ae070 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 rx=0x7f9ef4002f70 tx=0x7f9ef4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:37.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.426+0000 7f9ef27fc700 1 -- 172.21.15.17:0/2395857279 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ef4011760 con 0x7f9f040aa2b0 2022-08-01T19:41:37.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.426+0000 7f9ef27fc700 1 -- 172.21.15.17:0/2395857279 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef40118c0 con 0x7f9f040aa2b0 2022-08-01T19:41:37.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.426+0000 7f9f0b38d700 1 -- 172.21.15.17:0/2395857279 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f0413d5b0 con 0x7f9f040aa2b0 2022-08-01T19:41:37.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.426+0000 7f9ef27fc700 1 -- 172.21.15.17:0/2395857279 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ef4011c20 con 0x7f9f040aa2b0 2022-08-01T19:41:37.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.426+0000 7f9f0b38d700 1 -- 172.21.15.17:0/2395857279 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f04142d30 con 0x7f9f040aa2b0 2022-08-01T19:41:37.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.426+0000 7f9f0b38d700 1 -- 172.21.15.17:0/2395857279 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f040574e0 con 0x7f9f040aa2b0 2022-08-01T19:41:37.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.430+0000 7f9ef27fc700 1 -- 172.21.15.17:0/2395857279 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9ef4025030 con 0x7f9f040aa2b0 2022-08-01T19:41:37.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.430+0000 7f9ef27fc700 1 --2- 172.21.15.17:0/2395857279 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9ee4053b00 0x7f9ee4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:37.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.430+0000 7f9ef27fc700 1 -- 172.21.15.17:0/2395857279 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f9ef406d180 con 0x7f9f040aa2b0 2022-08-01T19:41:37.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.430+0000 7f9f08928700 1 --2- 172.21.15.17:0/2395857279 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9ee4053b00 0x7f9ee4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:37.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.430+0000 7f9f08928700 1 --2- 172.21.15.17:0/2395857279 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9ee4053b00 0x7f9ee4055fc0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f9f0000b000 tx=0x7f9f00009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:37.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.434+0000 7f9ef27fc700 1 -- 172.21.15.17:0/2395857279 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9ef4043ef0 con 0x7f9f040aa2b0 2022-08-01T19:41:37.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:37.562+0000 7f9f0b38d700 1 -- 172.21.15.17:0/2395857279 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9f0413d740 con 0x7f9f040aa2b0 2022-08-01T19:41:38.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.274+0000 7f9ef27fc700 1 -- 172.21.15.17:0/2395857279 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v100) v1 ==== 185+0+0 (secure 0 0 0) 0x7f9ef4023020 con 0x7f9f040aa2b0 2022-08-01T19:41:38.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.278+0000 7f9f0b38d700 1 -- 172.21.15.17:0/2395857279 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9ee4053b00 msgr2=0x7f9ee4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:38.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.278+0000 7f9f0b38d700 1 --2- 172.21.15.17:0/2395857279 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9ee4053b00 0x7f9ee4055fc0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f9f0000b000 tx=0x7f9f00009040).stop 2022-08-01T19:41:38.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.278+0000 7f9f0b38d700 1 -- 172.21.15.17:0/2395857279 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f040aa2b0 msgr2=0x7f9f040ae070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:38.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.278+0000 7f9f0b38d700 1 --2- 172.21.15.17:0/2395857279 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f040aa2b0 0x7f9f040ae070 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 rx=0x7f9ef4002f70 tx=0x7f9ef4015000).stop 2022-08-01T19:41:38.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.278+0000 7f9f0b38d700 1 -- 172.21.15.17:0/2395857279 shutdown_connections 2022-08-01T19:41:38.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.278+0000 7f9f0b38d700 1 --2- 172.21.15.17:0/2395857279 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9ee4053b00 0x7f9ee4055fc0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:38.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.278+0000 7f9f0b38d700 1 --2- 172.21.15.17:0/2395857279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f040b0a10 0x7f9f04154190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:38.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.278+0000 7f9f0b38d700 1 --2- 172.21.15.17:0/2395857279 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9f040b0630 0x7f9f04153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:38.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.278+0000 7f9f0b38d700 1 --2- 172.21.15.17:0/2395857279 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f040aa2b0 0x7f9f040ae070 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:38.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.278+0000 7f9f0b38d700 1 -- 172.21.15.17:0/2395857279 >> 172.21.15.17:0/2395857279 conn(0x7f9f04060950 msgr2=0x7f9f04140960 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:38.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.278+0000 7f9f0b38d700 1 -- 172.21.15.17:0/2395857279 shutdown_connections 2022-08-01T19:41:38.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.278+0000 7f9f0b38d700 1 -- 172.21.15.17:0/2395857279 wait complete. 2022-08-01T19:41:38.289 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' removed 2022-08-01T19:41:38.295 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-08-01T19:41:38.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.410+0000 7f4120838700 1 Processor -- start 2022-08-01T19:41:38.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.414+0000 7f4120838700 1 -- start start 2022-08-01T19:41:38.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.414+0000 7f4120838700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f411c0b31d0 0x7f411c066020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:38.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.414+0000 7f4120838700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f411c1402d0 0x7f411c140780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:38.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.414+0000 7f4120838700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f411c067030 con 0x7f411c0a9da0 2022-08-01T19:41:38.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.414+0000 7f4120838700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f411c064a70 con 0x7f411c1402d0 2022-08-01T19:41:38.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.414+0000 7f4120838700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f411c061f60 con 0x7f411c0b31d0 2022-08-01T19:41:38.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.414+0000 7f411ad9d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f411c0b31d0 0x7f411c066020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:38.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.414+0000 7f411b59e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f411c1402d0 0x7f411c140780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:38.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.414+0000 7f411b59e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f411c1402d0 0x7f411c140780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:40280/0 (socket says 172.21.15.17:40280) 2022-08-01T19:41:38.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.414+0000 7f411ad9d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f411c0b31d0 0x7f411c066020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:38458/0 (socket says 172.21.15.17:38458) 2022-08-01T19:41:38.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.414+0000 7f411b59e700 1 -- 172.21.15.17:0/876764757 learned_addr learned my addr 172.21.15.17:0/876764757 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:38.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.414+0000 7f4113fff700 1 --1- 172.21.15.17:0/876764757 >> v1:172.21.15.77:6789/0 conn(0x7f411c0a9da0 0x7f411c060950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:51356/0 (socket says 172.21.15.17:51356) 2022-08-01T19:41:38.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.414+0000 7f411b59e700 1 -- 172.21.15.17:0/876764757 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f411c0b31d0 msgr2=0x7f411c066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:38.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.414+0000 7f411b59e700 1 --2- 172.21.15.17:0/876764757 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f411c0b31d0 0x7f411c066020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:38.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.414+0000 7f411b59e700 1 -- 172.21.15.17:0/876764757 >> v1:172.21.15.77:6789/0 conn(0x7f411c0a9da0 legacy=0x7f411c060950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:38.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.414+0000 7f411b59e700 1 -- 172.21.15.17:0/876764757 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c0b35b0 con 0x7f411c1402d0 2022-08-01T19:41:38.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.414+0000 7f411b59e700 1 --2- 172.21.15.17:0/876764757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f411c1402d0 0x7f411c140780 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7f41140032f0 tx=0x7f4114009290).ready entity=mon.1 client_cookie=2e06fd2fe1957263 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:38.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.414+0000 7f411a59c700 1 -- 172.21.15.17:0/876764757 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f411400b8a0 con 0x7f411c1402d0 2022-08-01T19:41:38.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.414+0000 7f411a59c700 1 -- 172.21.15.17:0/876764757 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411400ba00 con 0x7f411c1402d0 2022-08-01T19:41:38.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.414+0000 7f411a59c700 1 -- 172.21.15.17:0/876764757 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f411400bb70 con 0x7f411c1402d0 2022-08-01T19:41:38.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.414+0000 7f4120838700 1 -- 172.21.15.17:0/876764757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f411c1402d0 msgr2=0x7f411c140780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:38.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.414+0000 7f4120838700 1 --2- 172.21.15.17:0/876764757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f411c1402d0 0x7f411c140780 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7f41140032f0 tx=0x7f4114009290).stop 2022-08-01T19:41:38.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.414+0000 7f4120838700 1 -- 172.21.15.17:0/876764757 shutdown_connections 2022-08-01T19:41:38.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.414+0000 7f4120838700 1 --2- 172.21.15.17:0/876764757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f411c1402d0 0x7f411c140780 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:38.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.414+0000 7f4120838700 1 --2- 172.21.15.17:0/876764757 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f411c0b31d0 0x7f411c066020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:38.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.414+0000 7f4120838700 1 -- 172.21.15.17:0/876764757 >> 172.21.15.17:0/876764757 conn(0x7f411c063870 msgr2=0x7f411c063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:38.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.414+0000 7f4120838700 1 -- 172.21.15.17:0/876764757 shutdown_connections 2022-08-01T19:41:38.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.414+0000 7f4120838700 1 -- 172.21.15.17:0/876764757 wait complete. 2022-08-01T19:41:38.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.418+0000 7f4120838700 1 Processor -- start 2022-08-01T19:41:38.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.418+0000 7f4120838700 1 -- start start 2022-08-01T19:41:38.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.418+0000 7f4120838700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f411c0a9da0 0x7f411c0ac200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:38.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.418+0000 7f4120838700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f411c0b31d0 0x7f411c0ac6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:38.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.418+0000 7f4120838700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f411c1402d0 0x7f411c153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:38.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.418+0000 7f4120838700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c134b70 con 0x7f411c0b31d0 2022-08-01T19:41:38.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.418+0000 7f4120838700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c0aafb0 con 0x7f411c0a9da0 2022-08-01T19:41:38.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.418+0000 7f4120838700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f411c136740 con 0x7f411c1402d0 2022-08-01T19:41:38.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.418+0000 7f4113fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f411c0b31d0 0x7f411c0ac6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:38.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.418+0000 7f411ad9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f411c0a9da0 0x7f411c0ac200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:38.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.418+0000 7f411b59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f411c1402d0 0x7f411c153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:38.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.418+0000 7f411b59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f411c1402d0 0x7f411c153c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:38466/0 (socket says 172.21.15.17:38466) 2022-08-01T19:41:38.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.418+0000 7f411ad9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f411c0a9da0 0x7f411c0ac200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:40286/0 (socket says 172.21.15.17:40286) 2022-08-01T19:41:38.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.418+0000 7f411b59e700 1 -- 172.21.15.17:0/2461406670 learned_addr learned my addr 172.21.15.17:0/2461406670 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:38.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.422+0000 7f4113fff700 1 -- 172.21.15.17:0/2461406670 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f411c1402d0 msgr2=0x7f411c153c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:38.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.422+0000 7f4113fff700 1 --2- 172.21.15.17:0/2461406670 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f411c1402d0 0x7f411c153c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:38.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.422+0000 7f4113fff700 1 -- 172.21.15.17:0/2461406670 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f411c0a9da0 msgr2=0x7f411c0ac200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:38.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.422+0000 7f4113fff700 1 --2- 172.21.15.17:0/2461406670 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f411c0a9da0 0x7f411c0ac200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:38.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.422+0000 7f4113fff700 1 -- 172.21.15.17:0/2461406670 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4114009530 con 0x7f411c0b31d0 2022-08-01T19:41:38.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.422+0000 7f411b59e700 1 --2- 172.21.15.17:0/2461406670 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f411c1402d0 0x7f411c153c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:38.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.422+0000 7f411ad9d700 1 --2- 172.21.15.17:0/2461406670 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f411c0a9da0 0x7f411c0ac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:38.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.422+0000 7f4113fff700 1 --2- 172.21.15.17:0/2461406670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f411c0b31d0 0x7f411c0ac6f0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 rx=0x7f411c065790 tx=0x7f4104007b40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:38.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.422+0000 7f4118d99700 1 -- 172.21.15.17:0/2461406670 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41040070d0 con 0x7f411c0b31d0 2022-08-01T19:41:38.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.422+0000 7f4118d99700 1 -- 172.21.15.17:0/2461406670 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4104007230 con 0x7f411c0b31d0 2022-08-01T19:41:38.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.422+0000 7f4120838700 1 -- 172.21.15.17:0/2461406670 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f411c0b6f20 con 0x7f411c0b31d0 2022-08-01T19:41:38.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.422+0000 7f4118d99700 1 -- 172.21.15.17:0/2461406670 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4104005740 con 0x7f411c0b31d0 2022-08-01T19:41:38.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.422+0000 7f4120838700 1 -- 172.21.15.17:0/2461406670 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f411c0acbe0 con 0x7f411c0b31d0 2022-08-01T19:41:38.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.422+0000 7f4120838700 1 -- 172.21.15.17:0/2461406670 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f411c0b70b0 con 0x7f411c0b31d0 2022-08-01T19:41:38.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.422+0000 7f4118d99700 1 -- 172.21.15.17:0/2461406670 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4104010070 con 0x7f411c0b31d0 2022-08-01T19:41:38.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.426+0000 7f4118d99700 1 --2- 172.21.15.17:0/2461406670 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4100053980 0x7f4100055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:38.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.426+0000 7f4118d99700 1 -- 172.21.15.17:0/2461406670 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f410406b3d0 con 0x7f411c0b31d0 2022-08-01T19:41:38.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.426+0000 7f411ad9d700 1 --2- 172.21.15.17:0/2461406670 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4100053980 0x7f4100055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:38.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.426+0000 7f411ad9d700 1 --2- 172.21.15.17:0/2461406670 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4100053980 0x7f4100055e40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f410c00b000 tx=0x7f410c006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:38.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.430+0000 7f4118d99700 1 -- 172.21.15.17:0/2461406670 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f41040421e0 con 0x7f411c0b31d0 2022-08-01T19:41:38.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:38.562+0000 7f4120838700 1 -- 172.21.15.17:0/2461406670 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f411c066af0 con 0x7f411c0b31d0 2022-08-01T19:41:39.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.286+0000 7f4118d99700 1 -- 172.21.15.17:0/2461406670 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v101) v1 ==== 173+0+0 (secure 0 0 0) 0x7f4104045800 con 0x7f411c0b31d0 2022-08-01T19:41:39.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.290+0000 7f4120838700 1 -- 172.21.15.17:0/2461406670 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4100053980 msgr2=0x7f4100055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:39.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.290+0000 7f4120838700 1 --2- 172.21.15.17:0/2461406670 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4100053980 0x7f4100055e40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f410c00b000 tx=0x7f410c006df0).stop 2022-08-01T19:41:39.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.290+0000 7f4120838700 1 -- 172.21.15.17:0/2461406670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f411c0b31d0 msgr2=0x7f411c0ac6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:39.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.290+0000 7f4120838700 1 --2- 172.21.15.17:0/2461406670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f411c0b31d0 0x7f411c0ac6f0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 rx=0x7f411c065790 tx=0x7f4104007b40).stop 2022-08-01T19:41:39.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.290+0000 7f4120838700 1 -- 172.21.15.17:0/2461406670 shutdown_connections 2022-08-01T19:41:39.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.290+0000 7f4120838700 1 --2- 172.21.15.17:0/2461406670 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4100053980 0x7f4100055e40 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:39.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.290+0000 7f4120838700 1 --2- 172.21.15.17:0/2461406670 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f411c1402d0 0x7f411c153c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:39.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.290+0000 7f4120838700 1 --2- 172.21.15.17:0/2461406670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f411c0b31d0 0x7f411c0ac6f0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:39.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.290+0000 7f4120838700 1 --2- 172.21.15.17:0/2461406670 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f411c0a9da0 0x7f411c0ac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:39.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.290+0000 7f4120838700 1 -- 172.21.15.17:0/2461406670 >> 172.21.15.17:0/2461406670 conn(0x7f411c063870 msgr2=0x7f411c0606d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:39.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.290+0000 7f4120838700 1 -- 172.21.15.17:0/2461406670 shutdown_connections 2022-08-01T19:41:39.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.290+0000 7f4120838700 1 -- 172.21.15.17:0/2461406670 wait complete. 2022-08-01T19:41:39.301 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' removed 2022-08-01T19:41:39.309 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-08-01T19:41:39.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.422+0000 7f253ef9a700 1 Processor -- start 2022-08-01T19:41:39.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.422+0000 7f253ef9a700 1 -- start start 2022-08-01T19:41:39.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.422+0000 7f253ef9a700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f25380b0740 0x7f25380b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:39.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.422+0000 7f253ef9a700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f25380b38a0 0x7f253813d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:39.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.422+0000 7f253ef9a700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2538143ab0 con 0x7f253813dbd0 2022-08-01T19:41:39.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.422+0000 7f253ef9a700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2538143d50 con 0x7f25380b38a0 2022-08-01T19:41:39.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.422+0000 7f253ef9a700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2538143ff0 con 0x7f25380b0740 2022-08-01T19:41:39.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.422+0000 7f253d537700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f253813dbd0 0x7f25380b1240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34570/0 (socket says 172.21.15.17:34570) 2022-08-01T19:41:39.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.422+0000 7f253d537700 1 -- 172.21.15.17:0/87925760 learned_addr learned my addr 172.21.15.17:0/87925760 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:39.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.422+0000 7f252ffff700 1 --2- 172.21.15.17:0/87925760 >> v2:172.21.15.77:3300/0 conn(0x7f25380b38a0 0x7f253813d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:39.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.422+0000 7f253cd36700 1 --2- 172.21.15.17:0/87925760 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f25380b0740 0x7f25380b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:39.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.422+0000 7f252f7fe700 1 -- 172.21.15.17:0/87925760 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3828751522 0 0) 0x7f2538143ab0 con 0x7f253813dbd0 2022-08-01T19:41:39.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.422+0000 7f252f7fe700 1 -- 172.21.15.17:0/87925760 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2524003260 con 0x7f253813dbd0 2022-08-01T19:41:39.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.422+0000 7f252ffff700 1 -- 172.21.15.17:0/87925760 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f25380b0740 msgr2=0x7f25380b0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:39.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.422+0000 7f252ffff700 1 --2- 172.21.15.17:0/87925760 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f25380b0740 0x7f25380b0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:39.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.426+0000 7f252ffff700 1 -- 172.21.15.17:0/87925760 >> v1:172.21.15.17:6789/0 conn(0x7f253813dbd0 legacy=0x7f25380b1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:39.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.426+0000 7f252ffff700 1 -- 172.21.15.17:0/87925760 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2538144290 con 0x7f25380b38a0 2022-08-01T19:41:39.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.426+0000 7f252ffff700 1 --2- 172.21.15.17:0/87925760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25380b38a0 0x7f253813d6e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f2528002f70 tx=0x7f2528006df0).ready entity=mon.1 client_cookie=e85532dc41ab080b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:39.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.426+0000 7f252f7fe700 1 -- 172.21.15.17:0/87925760 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f252800d4a0 con 0x7f25380b38a0 2022-08-01T19:41:39.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.426+0000 7f252f7fe700 1 -- 172.21.15.17:0/87925760 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2528000e90 con 0x7f25380b38a0 2022-08-01T19:41:39.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.426+0000 7f252f7fe700 1 -- 172.21.15.17:0/87925760 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f252800d7f0 con 0x7f25380b38a0 2022-08-01T19:41:39.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.426+0000 7f253ef9a700 1 -- 172.21.15.17:0/87925760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25380b38a0 msgr2=0x7f253813d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:39.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.426+0000 7f253ef9a700 1 --2- 172.21.15.17:0/87925760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25380b38a0 0x7f253813d6e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f2528002f70 tx=0x7f2528006df0).stop 2022-08-01T19:41:39.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.426+0000 7f253ef9a700 1 -- 172.21.15.17:0/87925760 shutdown_connections 2022-08-01T19:41:39.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.426+0000 7f253ef9a700 1 --2- 172.21.15.17:0/87925760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25380b38a0 0x7f253813d6e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:39.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.426+0000 7f253ef9a700 1 --2- 172.21.15.17:0/87925760 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f25380b0740 0x7f25380b0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:39.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.426+0000 7f253ef9a700 1 -- 172.21.15.17:0/87925760 >> 172.21.15.17:0/87925760 conn(0x7f2538060950 msgr2=0x7f2538060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:39.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.426+0000 7f253ef9a700 1 -- 172.21.15.17:0/87925760 shutdown_connections 2022-08-01T19:41:39.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.426+0000 7f253ef9a700 1 -- 172.21.15.17:0/87925760 wait complete. 2022-08-01T19:41:39.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.426+0000 7f253ef9a700 1 Processor -- start 2022-08-01T19:41:39.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.426+0000 7f253ef9a700 1 -- start start 2022-08-01T19:41:39.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.430+0000 7f253ef9a700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f25380b0740 0x7f2538142220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:39.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.430+0000 7f253ef9a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25380b38a0 0x7f2538134b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:39.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.430+0000 7f253ef9a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f253813dbd0 0x7f2538135070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:39.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.430+0000 7f253ef9a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25380ab6b0 con 0x7f253813dbd0 2022-08-01T19:41:39.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.430+0000 7f253ef9a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25380ae2d0 con 0x7f25380b38a0 2022-08-01T19:41:39.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.430+0000 7f253ef9a700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25380ae570 con 0x7f25380b0740 2022-08-01T19:41:39.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.430+0000 7f253d537700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f253813dbd0 0x7f2538135070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:39.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.430+0000 7f253d537700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f253813dbd0 0x7f2538135070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42382/0 (socket says 172.21.15.17:42382) 2022-08-01T19:41:39.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.430+0000 7f253d537700 1 -- 172.21.15.17:0/3083822081 learned_addr learned my addr 172.21.15.17:0/3083822081 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:39.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.430+0000 7f253cd36700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f25380b0740 0x7f2538142220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:39.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.430+0000 7f252ffff700 1 --2- 172.21.15.17:0/3083822081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25380b38a0 0x7f2538134b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:39.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.430+0000 7f253d537700 1 -- 172.21.15.17:0/3083822081 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f25380b0740 msgr2=0x7f2538142220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:39.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.430+0000 7f253d537700 1 --2- 172.21.15.17:0/3083822081 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f25380b0740 0x7f2538142220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:39.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.430+0000 7f253d537700 1 -- 172.21.15.17:0/3083822081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25380b38a0 msgr2=0x7f2538134b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:39.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.430+0000 7f253d537700 1 --2- 172.21.15.17:0/3083822081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25380b38a0 0x7f2538134b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:39.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.430+0000 7f253d537700 1 -- 172.21.15.17:0/3083822081 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f252800b180 con 0x7f253813dbd0 2022-08-01T19:41:39.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.430+0000 7f253d537700 1 --2- 172.21.15.17:0/3083822081 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f253813dbd0 0x7f2538135070 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 rx=0x7f2530001490 tx=0x7f2530007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:39.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.430+0000 7f252dffb700 1 -- 172.21.15.17:0/3083822081 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2530005c70 con 0x7f253813dbd0 2022-08-01T19:41:39.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.430+0000 7f252dffb700 1 -- 172.21.15.17:0/3083822081 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2530005dd0 con 0x7f253813dbd0 2022-08-01T19:41:39.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.430+0000 7f253ef9a700 1 -- 172.21.15.17:0/3083822081 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2538063b00 con 0x7f253813dbd0 2022-08-01T19:41:39.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.430+0000 7f252dffb700 1 -- 172.21.15.17:0/3083822081 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25300072c0 con 0x7f253813dbd0 2022-08-01T19:41:39.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.430+0000 7f253ef9a700 1 -- 172.21.15.17:0/3083822081 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25381355e0 con 0x7f253813dbd0 2022-08-01T19:41:39.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.430+0000 7f253ef9a700 1 -- 172.21.15.17:0/3083822081 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25380643e0 con 0x7f253813dbd0 2022-08-01T19:41:39.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.434+0000 7f252dffb700 1 -- 172.21.15.17:0/3083822081 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2530010070 con 0x7f253813dbd0 2022-08-01T19:41:39.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.434+0000 7f252dffb700 1 --2- 172.21.15.17:0/3083822081 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2518053a10 0x7f2518055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:39.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.434+0000 7f252dffb700 1 -- 172.21.15.17:0/3083822081 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f253006b7f0 con 0x7f253813dbd0 2022-08-01T19:41:39.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.434+0000 7f253cd36700 1 --2- 172.21.15.17:0/3083822081 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2518053a10 0x7f2518055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:39.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.434+0000 7f253cd36700 1 --2- 172.21.15.17:0/3083822081 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2518053a10 0x7f2518055ed0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7f253400b000 tx=0x7f2534006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:39.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.438+0000 7f252dffb700 1 -- 172.21.15.17:0/3083822081 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2530042780 con 0x7f253813dbd0 2022-08-01T19:41:39.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.566+0000 7f253ef9a700 1 -- 172.21.15.17:0/3083822081 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f2538143730 con 0x7f253813dbd0 2022-08-01T19:41:39.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.574+0000 7f252dffb700 1 -- 172.21.15.17:0/3083822081 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2530045da0 con 0x7f253813dbd0 2022-08-01T19:41:39.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.578+0000 7f253ef9a700 1 -- 172.21.15.17:0/3083822081 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2518053a10 msgr2=0x7f2518055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:39.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.578+0000 7f253ef9a700 1 --2- 172.21.15.17:0/3083822081 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2518053a10 0x7f2518055ed0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7f253400b000 tx=0x7f2534006df0).stop 2022-08-01T19:41:39.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.578+0000 7f253ef9a700 1 -- 172.21.15.17:0/3083822081 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f253813dbd0 msgr2=0x7f2538135070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:39.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.578+0000 7f253ef9a700 1 --2- 172.21.15.17:0/3083822081 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f253813dbd0 0x7f2538135070 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 rx=0x7f2530001490 tx=0x7f2530007dc0).stop 2022-08-01T19:41:39.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.578+0000 7f253ef9a700 1 -- 172.21.15.17:0/3083822081 shutdown_connections 2022-08-01T19:41:39.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.578+0000 7f253ef9a700 1 --2- 172.21.15.17:0/3083822081 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2518053a10 0x7f2518055ed0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:39.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.578+0000 7f253ef9a700 1 --2- 172.21.15.17:0/3083822081 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f253813dbd0 0x7f2538135070 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:39.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.578+0000 7f253ef9a700 1 --2- 172.21.15.17:0/3083822081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25380b38a0 0x7f2538134b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:39.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.578+0000 7f253ef9a700 1 --2- 172.21.15.17:0/3083822081 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f25380b0740 0x7f2538142220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:39.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.578+0000 7f253ef9a700 1 -- 172.21.15.17:0/3083822081 >> 172.21.15.17:0/3083822081 conn(0x7f2538060950 msgr2=0x7f25380b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:39.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.578+0000 7f253ef9a700 1 -- 172.21.15.17:0/3083822081 shutdown_connections 2022-08-01T19:41:39.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.578+0000 7f253ef9a700 1 -- 172.21.15.17:0/3083822081 wait complete. 2022-08-01T19:41:39.598 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical' 2022-08-01T19:41:39.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.706+0000 7efcd36f6700 1 Processor -- start 2022-08-01T19:41:39.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.706+0000 7efcd36f6700 1 -- start start 2022-08-01T19:41:39.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.706+0000 7efcd36f6700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efccc0b1f40 0x7efccc0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:39.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.706+0000 7efcd36f6700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efccc148530 0x7efccc0b2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:39.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.706+0000 7efcd36f6700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efccc0b04e0 con 0x7efccc0a7250 2022-08-01T19:41:39.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.706+0000 7efcd36f6700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efccc0a9250 con 0x7efccc0b1f40 2022-08-01T19:41:39.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.706+0000 7efcd36f6700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efccc0b0780 con 0x7efccc148530 2022-08-01T19:41:39.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.706+0000 7efcd1492700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efccc0b1f40 0x7efccc0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:39.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.706+0000 7efcd1492700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efccc0b1f40 0x7efccc0b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42386/0 (socket says 172.21.15.17:42386) 2022-08-01T19:41:39.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.706+0000 7efcd1492700 1 -- 172.21.15.17:0/3772259842 learned_addr learned my addr 172.21.15.17:0/3772259842 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:39.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.706+0000 7efcd0c91700 1 --2- 172.21.15.17:0/3772259842 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efccc148530 0x7efccc0b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:39.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.706+0000 7efcd1492700 1 -- 172.21.15.17:0/3772259842 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efccc148530 msgr2=0x7efccc0b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:39.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.706+0000 7efcd1492700 1 --2- 172.21.15.17:0/3772259842 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efccc148530 0x7efccc0b2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:39.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.706+0000 7efcd1492700 1 -- 172.21.15.17:0/3772259842 >> v1:172.21.15.77:6789/0 conn(0x7efccc0a7250 legacy=0x7efccc1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:39.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.706+0000 7efcd1492700 1 -- 172.21.15.17:0/3772259842 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efccc0b0a20 con 0x7efccc0b1f40 2022-08-01T19:41:39.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.706+0000 7efcd1492700 1 --2- 172.21.15.17:0/3772259842 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efccc0b1f40 0x7efccc0b1160 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 rx=0x7efcc40032f0 tx=0x7efcc4009290).ready entity=mon.0 client_cookie=95ef458d47c3e8be server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:39.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.706+0000 7efccbfff700 1 -- 172.21.15.17:0/3772259842 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efcc400b8a0 con 0x7efccc0b1f40 2022-08-01T19:41:39.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.706+0000 7efccbfff700 1 -- 172.21.15.17:0/3772259842 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcc400ba00 con 0x7efccc0b1f40 2022-08-01T19:41:39.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.706+0000 7efccbfff700 1 -- 172.21.15.17:0/3772259842 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efcc400bb70 con 0x7efccc0b1f40 2022-08-01T19:41:39.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.706+0000 7efcd36f6700 1 -- 172.21.15.17:0/3772259842 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efccc0b1f40 msgr2=0x7efccc0b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:39.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.706+0000 7efcd36f6700 1 --2- 172.21.15.17:0/3772259842 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efccc0b1f40 0x7efccc0b1160 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 rx=0x7efcc40032f0 tx=0x7efcc4009290).stop 2022-08-01T19:41:39.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.706+0000 7efcd36f6700 1 -- 172.21.15.17:0/3772259842 shutdown_connections 2022-08-01T19:41:39.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.706+0000 7efcd36f6700 1 --2- 172.21.15.17:0/3772259842 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efccc148530 0x7efccc0b2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:39.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.706+0000 7efcd36f6700 1 --2- 172.21.15.17:0/3772259842 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efccc0b1f40 0x7efccc0b1160 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:39.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.706+0000 7efcd36f6700 1 -- 172.21.15.17:0/3772259842 >> 172.21.15.17:0/3772259842 conn(0x7efccc05f840 msgr2=0x7efccc05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:39.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.710+0000 7efcd36f6700 1 -- 172.21.15.17:0/3772259842 shutdown_connections 2022-08-01T19:41:39.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.710+0000 7efcd36f6700 1 -- 172.21.15.17:0/3772259842 wait complete. 2022-08-01T19:41:39.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.710+0000 7efcd36f6700 1 Processor -- start 2022-08-01T19:41:39.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.710+0000 7efcd36f6700 1 -- start start 2022-08-01T19:41:39.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.710+0000 7efcd36f6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efccc0a7250 0x7efccc146ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:39.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.710+0000 7efcd36f6700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efccc0b1f40 0x7efccc13fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:39.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.710+0000 7efcd36f6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efccc148530 0x7efccc13ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:39.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.710+0000 7efcd36f6700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efccc0b5200 con 0x7efccc148530 2022-08-01T19:41:39.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.710+0000 7efcd36f6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efccc0b54a0 con 0x7efccc0a7250 2022-08-01T19:41:39.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.710+0000 7efcd36f6700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efccc0b5740 con 0x7efccc0b1f40 2022-08-01T19:41:39.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.710+0000 7efcd1c93700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efccc148530 0x7efccc13ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:39.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.710+0000 7efcd1c93700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efccc148530 0x7efccc13ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42396/0 (socket says 172.21.15.17:42396) 2022-08-01T19:41:39.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.710+0000 7efcd1492700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efccc0a7250 0x7efccc146ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:39.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.710+0000 7efcd1c93700 1 -- 172.21.15.17:0/2574609050 learned_addr learned my addr 172.21.15.17:0/2574609050 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:39.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.710+0000 7efcd0c91700 1 --2- 172.21.15.17:0/2574609050 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efccc0b1f40 0x7efccc13fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:39.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.710+0000 7efcd1c93700 1 -- 172.21.15.17:0/2574609050 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efccc0b1f40 msgr2=0x7efccc13fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:39.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.710+0000 7efcd1c93700 1 --2- 172.21.15.17:0/2574609050 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efccc0b1f40 0x7efccc13fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:39.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.710+0000 7efcd1c93700 1 -- 172.21.15.17:0/2574609050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efccc0a7250 msgr2=0x7efccc146ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:39.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.710+0000 7efcd1c93700 1 --2- 172.21.15.17:0/2574609050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efccc0a7250 0x7efccc146ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:39.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.710+0000 7efcd1c93700 1 -- 172.21.15.17:0/2574609050 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc4009530 con 0x7efccc148530 2022-08-01T19:41:39.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.714+0000 7efcd1c93700 1 --2- 172.21.15.17:0/2574609050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efccc148530 0x7efccc13ff10 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 rx=0x7efccc061530 tx=0x7efcbc007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:39.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.714+0000 7efcca7fc700 1 -- 172.21.15.17:0/2574609050 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efcbc012400 con 0x7efccc148530 2022-08-01T19:41:39.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.714+0000 7efcca7fc700 1 -- 172.21.15.17:0/2574609050 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcbc012560 con 0x7efccc148530 2022-08-01T19:41:39.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.714+0000 7efcd36f6700 1 -- 172.21.15.17:0/2574609050 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efccc0b59e0 con 0x7efccc148530 2022-08-01T19:41:39.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.714+0000 7efcca7fc700 1 -- 172.21.15.17:0/2574609050 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efcbc0128c0 con 0x7efccc148530 2022-08-01T19:41:39.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.714+0000 7efcd36f6700 1 -- 172.21.15.17:0/2574609050 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efccc0ae8f0 con 0x7efccc148530 2022-08-01T19:41:39.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.714+0000 7efcca7fc700 1 -- 172.21.15.17:0/2574609050 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7efcbc019030 con 0x7efccc148530 2022-08-01T19:41:39.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.714+0000 7efcd36f6700 1 -- 172.21.15.17:0/2574609050 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efccc0b0cc0 con 0x7efccc148530 2022-08-01T19:41:39.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.718+0000 7efcca7fc700 1 --2- 172.21.15.17:0/2574609050 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efcb0053a70 0x7efcb0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:39.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.718+0000 7efcca7fc700 1 -- 172.21.15.17:0/2574609050 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5686+0+0 (secure 0 0 0) 0x7efcbc06b030 con 0x7efccc148530 2022-08-01T19:41:39.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.718+0000 7efcd1492700 1 --2- 172.21.15.17:0/2574609050 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efcb0053a70 0x7efcb0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:39.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.718+0000 7efcd1492700 1 --2- 172.21.15.17:0/2574609050 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efcb0053a70 0x7efcb0055f30 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7efcc401b000 tx=0x7efcc4016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:39.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.722+0000 7efcca7fc700 1 -- 172.21.15.17:0/2574609050 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efcbc0420d0 con 0x7efccc148530 2022-08-01T19:41:39.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:39.886+0000 7efcd36f6700 1 -- 172.21.15.17:0/2574609050 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]} v 0) v1 -- 0x7efccc0574e0 con 0x7efccc148530 2022-08-01T19:41:40.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.290+0000 7efcca7fc700 1 -- 172.21.15.17:0/2574609050 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]}]=0 v336) v1 ==== 138+0+0 (secure 0 0 0) 0x7efcbc0456f0 con 0x7efccc148530 2022-08-01T19:41:40.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.290+0000 7efcd36f6700 1 -- 172.21.15.17:0/2574609050 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efcb0053a70 msgr2=0x7efcb0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:40.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.290+0000 7efcd36f6700 1 --2- 172.21.15.17:0/2574609050 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efcb0053a70 0x7efcb0055f30 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7efcc401b000 tx=0x7efcc4016000).stop 2022-08-01T19:41:40.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.290+0000 7efcd36f6700 1 -- 172.21.15.17:0/2574609050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efccc148530 msgr2=0x7efccc13ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:40.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.290+0000 7efcd36f6700 1 --2- 172.21.15.17:0/2574609050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efccc148530 0x7efccc13ff10 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 rx=0x7efccc061530 tx=0x7efcbc007bc0).stop 2022-08-01T19:41:40.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.294+0000 7efcd36f6700 1 -- 172.21.15.17:0/2574609050 shutdown_connections 2022-08-01T19:41:40.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.294+0000 7efcd36f6700 1 --2- 172.21.15.17:0/2574609050 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efcb0053a70 0x7efcb0055f30 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:40.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.294+0000 7efcd36f6700 1 --2- 172.21.15.17:0/2574609050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efccc148530 0x7efccc13ff10 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:40.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.294+0000 7efcd36f6700 1 --2- 172.21.15.17:0/2574609050 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efccc0b1f40 0x7efccc13fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:40.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.294+0000 7efcd36f6700 1 --2- 172.21.15.17:0/2574609050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efccc0a7250 0x7efccc146ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:40.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.294+0000 7efcd36f6700 1 -- 172.21.15.17:0/2574609050 >> 172.21.15.17:0/2574609050 conn(0x7efccc05f840 msgr2=0x7efccc144470 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:40.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.294+0000 7efcd36f6700 1 -- 172.21.15.17:0/2574609050 shutdown_connections 2022-08-01T19:41:40.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.294+0000 7efcd36f6700 1 -- 172.21.15.17:0/2574609050 wait complete. 2022-08-01T19:41:40.313 INFO:tasks.cephfs_test_runner:test_dump_inode_hexademical (tasks.cephfs.test_misc.TestMisc) ... ok 2022-08-01T19:41:40.315 INFO:tasks.cephfs_test_runner:Starting test: test_filtered_df (tasks.cephfs.test_misc.TestMisc) 2022-08-01T19:41:40.315 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_filtered_df' 2022-08-01T19:41:40.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.434+0000 7fbe21051700 1 Processor -- start 2022-08-01T19:41:40.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.434+0000 7fbe21051700 1 -- start start 2022-08-01T19:41:40.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.434+0000 7fbe21051700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbe1c0b2710 0x7fbe1c14afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:40.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.434+0000 7fbe21051700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fbe1c0b01a0 0x7fbe1c14b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:40.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.434+0000 7fbe21051700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe1c0a7250 con 0x7fbe1c0afd60 2022-08-01T19:41:40.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.434+0000 7fbe21051700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe1c0a7480 con 0x7fbe1c0b01a0 2022-08-01T19:41:40.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.434+0000 7fbe21051700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c0a7720 con 0x7fbe1c0b2710 2022-08-01T19:41:40.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.434+0000 7fbe20850700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fbe1c0afd60 0x7fbe1c0a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34598/0 (socket says 172.21.15.17:34598) 2022-08-01T19:41:40.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.434+0000 7fbe20850700 1 -- 172.21.15.17:0/3585705785 learned_addr learned my addr 172.21.15.17:0/3585705785 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:40.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.434+0000 7fbe1b7fe700 1 --2- 172.21.15.17:0/3585705785 >> v2:172.21.15.77:3300/0 conn(0x7fbe1c0b01a0 0x7fbe1c14b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:40.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.434+0000 7fbe1bfff700 1 --2- 172.21.15.17:0/3585705785 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbe1c0b2710 0x7fbe1c14afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:40.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.434+0000 7fbe1affd700 1 -- 172.21.15.17:0/3585705785 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 568776248 0 0) 0x7fbe1c0a7250 con 0x7fbe1c0afd60 2022-08-01T19:41:40.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.434+0000 7fbe1affd700 1 -- 172.21.15.17:0/3585705785 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe08003260 con 0x7fbe1c0afd60 2022-08-01T19:41:40.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.434+0000 7fbe1b7fe700 1 -- 172.21.15.17:0/3585705785 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbe1c0b2710 msgr2=0x7fbe1c14afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:40.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.434+0000 7fbe1b7fe700 1 --2- 172.21.15.17:0/3585705785 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbe1c0b2710 0x7fbe1c14afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:40.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.434+0000 7fbe1b7fe700 1 -- 172.21.15.17:0/3585705785 >> v1:172.21.15.17:6789/0 conn(0x7fbe1c0afd60 legacy=0x7fbe1c0a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:40.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.434+0000 7fbe1b7fe700 1 -- 172.21.15.17:0/3585705785 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe1c0a79c0 con 0x7fbe1c0b01a0 2022-08-01T19:41:40.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.434+0000 7fbe1b7fe700 1 --2- 172.21.15.17:0/3585705785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe1c0b01a0 0x7fbe1c14b4c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7fbe0c002f70 tx=0x7fbe0c006df0).ready entity=mon.1 client_cookie=7d0f75508f8224f1 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:40.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.434+0000 7fbe1affd700 1 -- 172.21.15.17:0/3585705785 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe0c00d4a0 con 0x7fbe1c0b01a0 2022-08-01T19:41:40.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.434+0000 7fbe1affd700 1 -- 172.21.15.17:0/3585705785 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe0c000e90 con 0x7fbe1c0b01a0 2022-08-01T19:41:40.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.438+0000 7fbe1affd700 1 -- 172.21.15.17:0/3585705785 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe0c00d7f0 con 0x7fbe1c0b01a0 2022-08-01T19:41:40.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.438+0000 7fbe21051700 1 -- 172.21.15.17:0/3585705785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe1c0b01a0 msgr2=0x7fbe1c14b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:40.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.438+0000 7fbe21051700 1 --2- 172.21.15.17:0/3585705785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe1c0b01a0 0x7fbe1c14b4c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7fbe0c002f70 tx=0x7fbe0c006df0).stop 2022-08-01T19:41:40.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.438+0000 7fbe21051700 1 -- 172.21.15.17:0/3585705785 shutdown_connections 2022-08-01T19:41:40.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.438+0000 7fbe21051700 1 --2- 172.21.15.17:0/3585705785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe1c0b01a0 0x7fbe1c14b4c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:40.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.438+0000 7fbe21051700 1 --2- 172.21.15.17:0/3585705785 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbe1c0b2710 0x7fbe1c14afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:40.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.438+0000 7fbe21051700 1 -- 172.21.15.17:0/3585705785 >> 172.21.15.17:0/3585705785 conn(0x7fbe1c060310 msgr2=0x7fbe1c0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:40.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.438+0000 7fbe21051700 1 -- 172.21.15.17:0/3585705785 shutdown_connections 2022-08-01T19:41:40.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.438+0000 7fbe21051700 1 -- 172.21.15.17:0/3585705785 wait complete. 2022-08-01T19:41:40.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.438+0000 7fbe21051700 1 Processor -- start 2022-08-01T19:41:40.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.438+0000 7fbe21051700 1 -- start start 2022-08-01T19:41:40.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.438+0000 7fbe21051700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbe1c0afd60 0x7fbe1c063cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:40.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.438+0000 7fbe21051700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe1c0b01a0 0x7fbe1c0b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:40.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.438+0000 7fbe21051700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe1c0b2710 0x7fbe1c0b54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:40.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.438+0000 7fbe21051700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c0b5b70 con 0x7fbe1c0b01a0 2022-08-01T19:41:40.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.438+0000 7fbe21051700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c05c2b0 con 0x7fbe1c0b2710 2022-08-01T19:41:40.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.438+0000 7fbe21051700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c05c550 con 0x7fbe1c0afd60 2022-08-01T19:41:40.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.438+0000 7fbe1b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe1c0b01a0 0x7fbe1c0b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:40.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.438+0000 7fbe1b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe1c0b01a0 0x7fbe1c0b4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42408/0 (socket says 172.21.15.17:42408) 2022-08-01T19:41:40.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.438+0000 7fbe1b7fe700 1 -- 172.21.15.17:0/1368891358 learned_addr learned my addr 172.21.15.17:0/1368891358 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:40.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.438+0000 7fbe1bfff700 1 --2- 172.21.15.17:0/1368891358 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbe1c0afd60 0x7fbe1c063cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:40.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.438+0000 7fbe20850700 1 --2- 172.21.15.17:0/1368891358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe1c0b2710 0x7fbe1c0b54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:40.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.438+0000 7fbe1b7fe700 1 -- 172.21.15.17:0/1368891358 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbe1c0afd60 msgr2=0x7fbe1c063cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:40.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.438+0000 7fbe1b7fe700 1 --2- 172.21.15.17:0/1368891358 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbe1c0afd60 0x7fbe1c063cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:40.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.438+0000 7fbe1b7fe700 1 -- 172.21.15.17:0/1368891358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe1c0b2710 msgr2=0x7fbe1c0b54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:40.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.438+0000 7fbe1b7fe700 1 --2- 172.21.15.17:0/1368891358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe1c0b2710 0x7fbe1c0b54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:40.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.438+0000 7fbe1b7fe700 1 -- 172.21.15.17:0/1368891358 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe0c00b180 con 0x7fbe1c0b01a0 2022-08-01T19:41:40.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.438+0000 7fbe1bfff700 1 --2- 172.21.15.17:0/1368891358 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbe1c0afd60 0x7fbe1c063cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:40.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.438+0000 7fbe1b7fe700 1 --2- 172.21.15.17:0/1368891358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe1c0b01a0 0x7fbe1c0b4c60 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 rx=0x7fbe0c002f70 tx=0x7fbe0c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:40.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.442+0000 7fbe197fa700 1 -- 172.21.15.17:0/1368891358 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe0c012930 con 0x7fbe1c0b01a0 2022-08-01T19:41:40.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.442+0000 7fbe21051700 1 -- 172.21.15.17:0/1368891358 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe1c065620 con 0x7fbe1c0b01a0 2022-08-01T19:41:40.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.442+0000 7fbe197fa700 1 -- 172.21.15.17:0/1368891358 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe0c012a90 con 0x7fbe1c0b01a0 2022-08-01T19:41:40.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.442+0000 7fbe197fa700 1 -- 172.21.15.17:0/1368891358 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe0c012df0 con 0x7fbe1c0b01a0 2022-08-01T19:41:40.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.442+0000 7fbe21051700 1 -- 172.21.15.17:0/1368891358 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe1c065ad0 con 0x7fbe1c0b01a0 2022-08-01T19:41:40.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.442+0000 7fbe21051700 1 -- 172.21.15.17:0/1368891358 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe1c0574e0 con 0x7fbe1c0b01a0 2022-08-01T19:41:40.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.442+0000 7fbe197fa700 1 -- 172.21.15.17:0/1368891358 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbe0c01c070 con 0x7fbe1c0b01a0 2022-08-01T19:41:40.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.442+0000 7fbe197fa700 1 --2- 172.21.15.17:0/1368891358 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbdfc053b30 0x7fbdfc055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:40.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.442+0000 7fbe197fa700 1 -- 172.21.15.17:0/1368891358 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fbe0c06c9d0 con 0x7fbe1c0b01a0 2022-08-01T19:41:40.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.442+0000 7fbe1bfff700 1 --2- 172.21.15.17:0/1368891358 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbdfc053b30 0x7fbdfc055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:40.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.446+0000 7fbe1bfff700 1 --2- 172.21.15.17:0/1368891358 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbdfc053b30 0x7fbdfc055ff0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7fbe1000b000 tx=0x7fbe10006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:40.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.454+0000 7fbe197fa700 1 -- 172.21.15.17:0/1368891358 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbe0c043a80 con 0x7fbe1c0b01a0 2022-08-01T19:41:40.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:40.618+0000 7fbe21051700 1 -- 172.21.15.17:0/1368891358 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]} v 0) v1 -- 0x7fbe1c05d410 con 0x7fbe1c0b01a0 2022-08-01T19:41:41.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.294+0000 7fbe197fa700 1 -- 172.21.15.17:0/1368891358 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]}]=0 v337) v1 ==== 130+0+0 (secure 0 0 0) 0x7fbe0c019070 con 0x7fbe1c0b01a0 2022-08-01T19:41:41.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.298+0000 7fbe21051700 1 -- 172.21.15.17:0/1368891358 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbdfc053b30 msgr2=0x7fbdfc055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:41.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.298+0000 7fbe21051700 1 --2- 172.21.15.17:0/1368891358 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbdfc053b30 0x7fbdfc055ff0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7fbe1000b000 tx=0x7fbe10006df0).stop 2022-08-01T19:41:41.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.298+0000 7fbe21051700 1 -- 172.21.15.17:0/1368891358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe1c0b01a0 msgr2=0x7fbe1c0b4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:41.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.298+0000 7fbe21051700 1 --2- 172.21.15.17:0/1368891358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe1c0b01a0 0x7fbe1c0b4c60 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 rx=0x7fbe0c002f70 tx=0x7fbe0c014000).stop 2022-08-01T19:41:41.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.298+0000 7fbe21051700 1 -- 172.21.15.17:0/1368891358 shutdown_connections 2022-08-01T19:41:41.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.298+0000 7fbe21051700 1 --2- 172.21.15.17:0/1368891358 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbdfc053b30 0x7fbdfc055ff0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:41.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.298+0000 7fbe21051700 1 --2- 172.21.15.17:0/1368891358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe1c0b2710 0x7fbe1c0b54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:41.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.298+0000 7fbe21051700 1 --2- 172.21.15.17:0/1368891358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe1c0b01a0 0x7fbe1c0b4c60 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:41.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.298+0000 7fbe21051700 1 --2- 172.21.15.17:0/1368891358 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbe1c0afd60 0x7fbe1c063cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:41.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.298+0000 7fbe21051700 1 -- 172.21.15.17:0/1368891358 >> 172.21.15.17:0/1368891358 conn(0x7fbe1c060310 msgr2=0x7fbe1c148c90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:41.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.298+0000 7fbe21051700 1 -- 172.21.15.17:0/1368891358 shutdown_connections 2022-08-01T19:41:41.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.298+0000 7fbe21051700 1 -- 172.21.15.17:0/1368891358 wait complete. 2022-08-01T19:41:41.321 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-08-01T19:41:41.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.446+0000 7f48b9653700 1 Processor -- start 2022-08-01T19:41:41.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.446+0000 7f48b9653700 1 -- start start 2022-08-01T19:41:41.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.446+0000 7f48b9653700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f48b40af450 0x7f48b40af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:41.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.446+0000 7f48b9653700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f48b40ad4b0 0x7f48b41432e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:41.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.446+0000 7f48b9653700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48b40aff30 con 0x7f48b41437d0 2022-08-01T19:41:41.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.446+0000 7f48b9653700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48b405c7a0 con 0x7f48b40af450 2022-08-01T19:41:41.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.446+0000 7f48b9653700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b4140990 con 0x7f48b40ad4b0 2022-08-01T19:41:41.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.446+0000 7f48b3fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f48b40af450 0x7f48b40af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:41.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.446+0000 7f48b3fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f48b40af450 0x7f48b40af830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42416/0 (socket says 172.21.15.17:42416) 2022-08-01T19:41:41.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.446+0000 7f48b3fff700 1 -- 172.21.15.17:0/2475113389 learned_addr learned my addr 172.21.15.17:0/2475113389 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:41.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.450+0000 7f48b37fe700 1 --2- 172.21.15.17:0/2475113389 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f48b40ad4b0 0x7f48b41432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:41.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.450+0000 7f48b3fff700 1 -- 172.21.15.17:0/2475113389 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f48b40ad4b0 msgr2=0x7f48b41432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:41.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.450+0000 7f48b3fff700 1 --2- 172.21.15.17:0/2475113389 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f48b40ad4b0 0x7f48b41432e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:41.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.450+0000 7f48b3fff700 1 -- 172.21.15.17:0/2475113389 >> v1:172.21.15.77:6789/0 conn(0x7f48b41437d0 legacy=0x7f48b40a8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:41.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.450+0000 7f48b2ffd700 1 -- 172.21.15.17:0/2475113389 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4085050667 0 0) 0x7f48b40aff30 con 0x7f48b41437d0 2022-08-01T19:41:41.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.450+0000 7f48b3fff700 1 -- 172.21.15.17:0/2475113389 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b40a9550 con 0x7f48b40af450 2022-08-01T19:41:41.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.450+0000 7f48b3fff700 1 --2- 172.21.15.17:0/2475113389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48b40af450 0x7f48b40af830 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 rx=0x7f48b413fdc0 tx=0x7f48a4006eb0).ready entity=mon.0 client_cookie=389e67776cbccc6b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:41.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.450+0000 7f48b2ffd700 1 -- 172.21.15.17:0/2475113389 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48a400d5c0 con 0x7f48b40af450 2022-08-01T19:41:41.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.450+0000 7f48b2ffd700 1 -- 172.21.15.17:0/2475113389 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a4000e90 con 0x7f48b40af450 2022-08-01T19:41:41.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.450+0000 7f48b2ffd700 1 -- 172.21.15.17:0/2475113389 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48a400d910 con 0x7f48b40af450 2022-08-01T19:41:41.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.450+0000 7f48b9653700 1 -- 172.21.15.17:0/2475113389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48b40af450 msgr2=0x7f48b40af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:41.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.450+0000 7f48b9653700 1 --2- 172.21.15.17:0/2475113389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48b40af450 0x7f48b40af830 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 rx=0x7f48b413fdc0 tx=0x7f48a4006eb0).stop 2022-08-01T19:41:41.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.450+0000 7f48b9653700 1 -- 172.21.15.17:0/2475113389 shutdown_connections 2022-08-01T19:41:41.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.450+0000 7f48b9653700 1 --2- 172.21.15.17:0/2475113389 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f48b40ad4b0 0x7f48b41432e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:41.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.450+0000 7f48b9653700 1 --2- 172.21.15.17:0/2475113389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48b40af450 0x7f48b40af830 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:41.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.450+0000 7f48b9653700 1 -- 172.21.15.17:0/2475113389 >> 172.21.15.17:0/2475113389 conn(0x7f48b40612d0 msgr2=0x7f48b4062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:41.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.450+0000 7f48b9653700 1 -- 172.21.15.17:0/2475113389 shutdown_connections 2022-08-01T19:41:41.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.450+0000 7f48b9653700 1 -- 172.21.15.17:0/2475113389 wait complete. 2022-08-01T19:41:41.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.454+0000 7f48b9653700 1 Processor -- start 2022-08-01T19:41:41.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.454+0000 7f48b9653700 1 -- start start 2022-08-01T19:41:41.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.454+0000 7f48b9653700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f48b40ad4b0 0x7f48b405f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:41.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.454+0000 7f48b9653700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48b40af450 0x7f48b40b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:41.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.454+0000 7f48b9653700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48b41437d0 0x7f48b4152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:41.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.454+0000 7f48b9653700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b40b5ce0 con 0x7f48b40af450 2022-08-01T19:41:41.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.454+0000 7f48b9653700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b40b6680 con 0x7f48b41437d0 2022-08-01T19:41:41.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.454+0000 7f48b9653700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b4153210 con 0x7f48b40ad4b0 2022-08-01T19:41:41.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.454+0000 7f48b37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48b40af450 0x7f48b40b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:41.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.454+0000 7f48b37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48b40af450 0x7f48b40b3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42426/0 (socket says 172.21.15.17:42426) 2022-08-01T19:41:41.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.454+0000 7f48b37fe700 1 -- 172.21.15.17:0/924504131 learned_addr learned my addr 172.21.15.17:0/924504131 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:41.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.454+0000 7f48b3fff700 1 --2- 172.21.15.17:0/924504131 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f48b40ad4b0 0x7f48b405f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:41.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.454+0000 7f48b8e52700 1 --2- 172.21.15.17:0/924504131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48b41437d0 0x7f48b4152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:41.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.454+0000 7f48b37fe700 1 -- 172.21.15.17:0/924504131 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f48b40ad4b0 msgr2=0x7f48b405f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:41.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.454+0000 7f48b37fe700 1 --2- 172.21.15.17:0/924504131 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f48b40ad4b0 0x7f48b405f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:41.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.454+0000 7f48b37fe700 1 -- 172.21.15.17:0/924504131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48b41437d0 msgr2=0x7f48b4152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:41.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.454+0000 7f48b37fe700 1 --2- 172.21.15.17:0/924504131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48b41437d0 0x7f48b4152610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:41.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.454+0000 7f48b37fe700 1 -- 172.21.15.17:0/924504131 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a400b1e0 con 0x7f48b40af450 2022-08-01T19:41:41.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.454+0000 7f48b37fe700 1 --2- 172.21.15.17:0/924504131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48b40af450 0x7f48b40b3fb0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 rx=0x7f48a800c000 tx=0x7f48a8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:41.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.454+0000 7f48b17fa700 1 -- 172.21.15.17:0/924504131 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48a8010590 con 0x7f48b40af450 2022-08-01T19:41:41.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.454+0000 7f48b17fa700 1 -- 172.21.15.17:0/924504131 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a80106f0 con 0x7f48b40af450 2022-08-01T19:41:41.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.454+0000 7f48b9653700 1 -- 172.21.15.17:0/924504131 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b41534b0 con 0x7f48b40af450 2022-08-01T19:41:41.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.454+0000 7f48b17fa700 1 -- 172.21.15.17:0/924504131 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48a8010a50 con 0x7f48b40af450 2022-08-01T19:41:41.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.454+0000 7f48b9653700 1 -- 172.21.15.17:0/924504131 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b40ac880 con 0x7f48b40af450 2022-08-01T19:41:41.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.458+0000 7f48b9653700 1 -- 172.21.15.17:0/924504131 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48b40b6880 con 0x7f48b40af450 2022-08-01T19:41:41.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.458+0000 7f48b17fa700 1 -- 172.21.15.17:0/924504131 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f48a8010bb0 con 0x7f48b40af450 2022-08-01T19:41:41.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.458+0000 7f48b17fa700 1 --2- 172.21.15.17:0/924504131 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4894053a70 0x7f4894055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:41.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.458+0000 7f48b17fa700 1 -- 172.21.15.17:0/924504131 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f48a806be00 con 0x7f48b40af450 2022-08-01T19:41:41.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.458+0000 7f48b3fff700 1 --2- 172.21.15.17:0/924504131 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4894053a70 0x7f4894055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:41.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.458+0000 7f48b3fff700 1 --2- 172.21.15.17:0/924504131 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4894053a70 0x7f4894055f30 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7f48a401c000 tx=0x7f48a4015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:41.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.462+0000 7f48b17fa700 1 -- 172.21.15.17:0/924504131 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f48a8042d90 con 0x7f48b40af450 2022-08-01T19:41:41.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.594+0000 7f48b9653700 1 -- 172.21.15.17:0/924504131 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f48b40574e0 con 0x7f48b40af450 2022-08-01T19:41:41.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.602+0000 7f48b17fa700 1 -- 172.21.15.17:0/924504131 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f48a80463b0 con 0x7f48b40af450 2022-08-01T19:41:41.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.606+0000 7f48b9653700 1 -- 172.21.15.17:0/924504131 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4894053a70 msgr2=0x7f4894055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:41.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.606+0000 7f48b9653700 1 --2- 172.21.15.17:0/924504131 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4894053a70 0x7f4894055f30 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7f48a401c000 tx=0x7f48a4015000).stop 2022-08-01T19:41:41.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.606+0000 7f48b9653700 1 -- 172.21.15.17:0/924504131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48b40af450 msgr2=0x7f48b40b3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:41.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.606+0000 7f48b9653700 1 --2- 172.21.15.17:0/924504131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48b40af450 0x7f48b40b3fb0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 rx=0x7f48a800c000 tx=0x7f48a8006df0).stop 2022-08-01T19:41:41.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.606+0000 7f48b9653700 1 -- 172.21.15.17:0/924504131 shutdown_connections 2022-08-01T19:41:41.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.606+0000 7f48b9653700 1 --2- 172.21.15.17:0/924504131 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4894053a70 0x7f4894055f30 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:41.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.606+0000 7f48b9653700 1 --2- 172.21.15.17:0/924504131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f48b41437d0 0x7f48b4152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:41.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.606+0000 7f48b9653700 1 --2- 172.21.15.17:0/924504131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48b40af450 0x7f48b40b3fb0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:41.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.606+0000 7f48b9653700 1 --2- 172.21.15.17:0/924504131 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f48b40ad4b0 0x7f48b405f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:41.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.606+0000 7f48b9653700 1 -- 172.21.15.17:0/924504131 >> 172.21.15.17:0/924504131 conn(0x7f48b40612d0 msgr2=0x7f48b405be30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:41.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.606+0000 7f48b9653700 1 -- 172.21.15.17:0/924504131 shutdown_connections 2022-08-01T19:41:41.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.606+0000 7f48b9653700 1 -- 172.21.15.17:0/924504131 wait complete. 2022-08-01T19:41:41.623 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T19:41:41.624 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:41:41.646 DEBUG:teuthology.orchestra.run.smithi077:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:41:41.668 DEBUG:teuthology.orchestra.run.smithi085:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:41:41.690 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:41:41.719 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T19:41:41.720 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:41.721 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:41:41.722 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:41:41.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:41:41.733 INFO:teuthology.orchestra.run.smithi085.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-08-01T19:41:41.734 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-08-01T19:41:41.734 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:41.735 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:41:41.735 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:41:41.744 INFO:teuthology.orchestra.run.smithi190.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T19:41:41.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:41:41.746 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-08-01T19:41:41.747 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T19:41:41.858 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.858+0000 7f8db221b700 1 Processor -- start 2022-08-01T19:41:41.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.858+0000 7f8db221b700 1 -- start start 2022-08-01T19:41:41.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.858+0000 7f8db221b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8dac0b1f40 0x7f8dac0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:41.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.858+0000 7f8db221b700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f8dac148530 0x7f8dac0b2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:41.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.858+0000 7f8db221b700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dac0b04e0 con 0x7f8dac0a7250 2022-08-01T19:41:41.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.858+0000 7f8db221b700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dac0a9250 con 0x7f8dac148530 2022-08-01T19:41:41.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.858+0000 7f8db221b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac0b0780 con 0x7f8dac0b1f40 2022-08-01T19:41:41.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.858+0000 7f8dabfff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f8dac0a7250 0x7f8dac1448b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34628/0 (socket says 172.21.15.17:34628) 2022-08-01T19:41:41.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.858+0000 7f8dabfff700 1 -- 172.21.15.17:0/78348589 learned_addr learned my addr 172.21.15.17:0/78348589 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:41.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.858+0000 7f8daaffd700 1 --2- 172.21.15.17:0/78348589 >> v2:172.21.15.77:3300/0 conn(0x7f8dac148530 0x7f8dac0b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:41.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.858+0000 7f8dab7fe700 1 --2- 172.21.15.17:0/78348589 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8dac0b1f40 0x7f8dac0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:41.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.858+0000 7f8daa7fc700 1 -- 172.21.15.17:0/78348589 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2420168147 0 0) 0x7f8dac0b04e0 con 0x7f8dac0a7250 2022-08-01T19:41:41.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.858+0000 7f8daa7fc700 1 -- 172.21.15.17:0/78348589 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d8c003260 con 0x7f8dac0a7250 2022-08-01T19:41:41.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.858+0000 7f8daaffd700 1 -- 172.21.15.17:0/78348589 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8dac0b1f40 msgr2=0x7f8dac0b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:41.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.858+0000 7f8daaffd700 1 --2- 172.21.15.17:0/78348589 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8dac0b1f40 0x7f8dac0b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:41.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.858+0000 7f8daaffd700 1 -- 172.21.15.17:0/78348589 >> v1:172.21.15.17:6789/0 conn(0x7f8dac0a7250 legacy=0x7f8dac1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:41.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.858+0000 7f8daaffd700 1 -- 172.21.15.17:0/78348589 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dac0b0a20 con 0x7f8dac148530 2022-08-01T19:41:41.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.858+0000 7f8daaffd700 1 --2- 172.21.15.17:0/78348589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8dac148530 0x7f8dac0b2320 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f8d94002f70 tx=0x7f8d94006df0).ready entity=mon.1 client_cookie=168cf48225ddce02 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:41.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.858+0000 7f8daa7fc700 1 -- 172.21.15.17:0/78348589 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d9400d4a0 con 0x7f8dac148530 2022-08-01T19:41:41.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.858+0000 7f8daa7fc700 1 -- 172.21.15.17:0/78348589 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d94000e90 con 0x7f8dac148530 2022-08-01T19:41:41.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.858+0000 7f8daa7fc700 1 -- 172.21.15.17:0/78348589 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d9400d7f0 con 0x7f8dac148530 2022-08-01T19:41:41.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.862+0000 7f8db221b700 1 -- 172.21.15.17:0/78348589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8dac148530 msgr2=0x7f8dac0b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:41.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.862+0000 7f8db221b700 1 --2- 172.21.15.17:0/78348589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8dac148530 0x7f8dac0b2320 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f8d94002f70 tx=0x7f8d94006df0).stop 2022-08-01T19:41:41.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.862+0000 7f8db221b700 1 -- 172.21.15.17:0/78348589 shutdown_connections 2022-08-01T19:41:41.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.862+0000 7f8db221b700 1 --2- 172.21.15.17:0/78348589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8dac148530 0x7f8dac0b2320 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:41.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.862+0000 7f8db221b700 1 --2- 172.21.15.17:0/78348589 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8dac0b1f40 0x7f8dac0b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:41.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.862+0000 7f8db221b700 1 -- 172.21.15.17:0/78348589 >> 172.21.15.17:0/78348589 conn(0x7f8dac05f840 msgr2=0x7f8dac05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:41.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.862+0000 7f8db221b700 1 -- 172.21.15.17:0/78348589 shutdown_connections 2022-08-01T19:41:41.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.862+0000 7f8db221b700 1 -- 172.21.15.17:0/78348589 wait complete. 2022-08-01T19:41:41.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.862+0000 7f8db221b700 1 Processor -- start 2022-08-01T19:41:41.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.862+0000 7f8db221b700 1 -- start start 2022-08-01T19:41:41.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.866+0000 7f8db221b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8dac0a7250 0x7f8dac145f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:41.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.866+0000 7f8db221b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8dac0b1f40 0x7f8dac14a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:41.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.866+0000 7f8db221b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8dac148530 0x7f8dac1439d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:41.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.866+0000 7f8db221b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac0ab2e0 con 0x7f8dac0b1f40 2022-08-01T19:41:41.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.866+0000 7f8db221b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac0ab580 con 0x7f8dac0a7250 2022-08-01T19:41:41.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.866+0000 7f8db221b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac0ab820 con 0x7f8dac148530 2022-08-01T19:41:41.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.866+0000 7f8daaffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8dac0b1f40 0x7f8dac14a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:41.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.866+0000 7f8daaffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8dac0b1f40 0x7f8dac14a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42438/0 (socket says 172.21.15.17:42438) 2022-08-01T19:41:41.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.866+0000 7f8daaffd700 1 -- 172.21.15.17:0/2317705903 learned_addr learned my addr 172.21.15.17:0/2317705903 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:41.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.866+0000 7f8dabfff700 1 --2- 172.21.15.17:0/2317705903 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8dac148530 0x7f8dac1439d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:41.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.866+0000 7f8dab7fe700 1 --2- 172.21.15.17:0/2317705903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8dac0a7250 0x7f8dac145f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:41.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.866+0000 7f8daaffd700 1 -- 172.21.15.17:0/2317705903 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8dac148530 msgr2=0x7f8dac1439d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:41.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.866+0000 7f8daaffd700 1 --2- 172.21.15.17:0/2317705903 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8dac148530 0x7f8dac1439d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:41.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.866+0000 7f8daaffd700 1 -- 172.21.15.17:0/2317705903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8dac0a7250 msgr2=0x7f8dac145f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:41.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.866+0000 7f8daaffd700 1 --2- 172.21.15.17:0/2317705903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8dac0a7250 0x7f8dac145f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:41.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.866+0000 7f8daaffd700 1 -- 172.21.15.17:0/2317705903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9400b180 con 0x7f8dac0b1f40 2022-08-01T19:41:41.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.866+0000 7f8daaffd700 1 --2- 172.21.15.17:0/2317705903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8dac0b1f40 0x7f8dac14a180 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 rx=0x7f8d94002f70 tx=0x7f8d94014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:41.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.866+0000 7f8da8ff9700 1 -- 172.21.15.17:0/2317705903 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d94012860 con 0x7f8dac0b1f40 2022-08-01T19:41:41.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.866+0000 7f8da8ff9700 1 -- 172.21.15.17:0/2317705903 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d940129c0 con 0x7f8dac0b1f40 2022-08-01T19:41:41.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.866+0000 7f8db221b700 1 -- 172.21.15.17:0/2317705903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dac0abac0 con 0x7f8dac0b1f40 2022-08-01T19:41:41.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.866+0000 7f8da8ff9700 1 -- 172.21.15.17:0/2317705903 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d94012d20 con 0x7f8dac0b1f40 2022-08-01T19:41:41.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.866+0000 7f8db221b700 1 -- 172.21.15.17:0/2317705903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dac14bae0 con 0x7f8dac0b1f40 2022-08-01T19:41:41.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.866+0000 7f8db221b700 1 -- 172.21.15.17:0/2317705903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dac0b0cc0 con 0x7f8dac0b1f40 2022-08-01T19:41:41.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.870+0000 7f8da8ff9700 1 -- 172.21.15.17:0/2317705903 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8d9401c070 con 0x7f8dac0b1f40 2022-08-01T19:41:41.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.870+0000 7f8da8ff9700 1 --2- 172.21.15.17:0/2317705903 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8d90053aa0 0x7f8d90055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:41.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.870+0000 7f8da8ff9700 1 -- 172.21.15.17:0/2317705903 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f8d9406c6f0 con 0x7f8dac0b1f40 2022-08-01T19:41:41.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.870+0000 7f8dab7fe700 1 --2- 172.21.15.17:0/2317705903 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8d90053aa0 0x7f8d90055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:41.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.870+0000 7f8dab7fe700 1 --2- 172.21.15.17:0/2317705903 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8d90053aa0 0x7f8d90055f60 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f8d9c00b000 tx=0x7f8d9c006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:41.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:41.874+0000 7f8da8ff9700 1 -- 172.21.15.17:0/2317705903 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8d94043790 con 0x7f8dac0b1f40 2022-08-01T19:41:42.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.002+0000 7f8db221b700 1 -- 172.21.15.17:0/2317705903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f8dac0574e0 con 0x7f8dac0b1f40 2022-08-01T19:41:42.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.006+0000 7f8da8ff9700 1 -- 172.21.15.17:0/2317705903 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v101) v1 ==== 81+0+29221 (secure 0 0 0) 0x7f8d94019090 con 0x7f8dac0b1f40 2022-08-01T19:41:42.007 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:42.007 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:41:42.008 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 101, 2022-08-01T19:41:42.008 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", 2022-08-01T19:41:42.008 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2022-08-01T19:33:50.105743+0000", 2022-08-01T19:41:42.009 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2022-08-01T19:41:39.274710+0000", 2022-08-01T19:41:42.009 INFO:teuthology.orchestra.run.smithi017.stdout: "last_up_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:41:42.009 INFO:teuthology.orchestra.run.smithi017.stdout: "last_in_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:41:42.009 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T19:41:42.009 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_num": 5799936, 2022-08-01T19:41:42.010 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_set": [ 2022-08-01T19:41:42.010 INFO:teuthology.orchestra.run.smithi017.stdout: "pglog_hardlimit", 2022-08-01T19:41:42.010 INFO:teuthology.orchestra.run.smithi017.stdout: "purged_snapdirs", 2022-08-01T19:41:42.010 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_deletes", 2022-08-01T19:41:42.011 INFO:teuthology.orchestra.run.smithi017.stdout: "sortbitwise" 2022-08-01T19:41:42.011 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:41:42.012 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_version": 6, 2022-08-01T19:41:42.012 INFO:teuthology.orchestra.run.smithi017.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T19:41:42.012 INFO:teuthology.orchestra.run.smithi017.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T19:41:42.012 INFO:teuthology.orchestra.run.smithi017.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T19:41:42.013 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_snapshot": "", 2022-08-01T19:41:42.013 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_max": 18, 2022-08-01T19:41:42.013 INFO:teuthology.orchestra.run.smithi017.stdout: "max_osd": 8, 2022-08-01T19:41:42.014 INFO:teuthology.orchestra.run.smithi017.stdout: "require_min_compat_client": "jewel", 2022-08-01T19:41:42.014 INFO:teuthology.orchestra.run.smithi017.stdout: "min_compat_client": "jewel", 2022-08-01T19:41:42.014 INFO:teuthology.orchestra.run.smithi017.stdout: "require_osd_release": "octopus", 2022-08-01T19:41:42.015 INFO:teuthology.orchestra.run.smithi017.stdout: "pools": [ 2022-08-01T19:41:42.015 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.015 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 1, 2022-08-01T19:41:42.015 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "device_health_metrics", 2022-08-01T19:41:42.016 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:33:53.001841+0000", 2022-08-01T19:41:42.016 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:41:42.016 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:41:42.016 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:41:42.017 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:41:42.017 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:41:42.017 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:41:42.017 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:41:42.017 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:41:42.018 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 1, 2022-08-01T19:41:42.018 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 1, 2022-08-01T19:41:42.018 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 1, 2022-08-01T19:41:42.018 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 1, 2022-08-01T19:41:42.019 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 1, 2022-08-01T19:41:42.019 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:41:42.019 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:41:42.019 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:41:42.020 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:41:42.020 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:41:42.020 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:41:42.020 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:41:42.020 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.021 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "16", 2022-08-01T19:41:42.021 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:41:42.021 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:41:42.021 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:41:42.022 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:41:42.022 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:41:42.022 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:41:42.022 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:41:42.023 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:41:42.023 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:41:42.023 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:41:42.023 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:41:42.023 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:41:42.024 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:41:42.024 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:41:42.025 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:41:42.025 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:41:42.025 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:41:42.025 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:41:42.026 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:41:42.026 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:41:42.026 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:41:42.026 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:41:42.027 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:41:42.027 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:41:42.027 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:41:42.027 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:41:42.028 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.028 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:41:42.028 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:41:42.029 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:41:42.029 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:41:42.029 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:41:42.030 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:41:42.030 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:41:42.030 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:41:42.031 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:41:42.031 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:41:42.031 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:41:42.032 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:41:42.032 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 1 2022-08-01T19:41:42.033 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.033 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:41:42.034 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_devicehealth": {} 2022-08-01T19:41:42.035 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.035 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.037 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.038 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 2, 2022-08-01T19:41:42.038 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "rbd", 2022-08-01T19:41:42.038 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:34:01.553098+0000", 2022-08-01T19:41:42.039 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:41:42.039 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:41:42.040 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:41:42.040 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:41:42.040 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:41:42.041 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:41:42.041 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:41:42.042 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:41:42.042 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:41:42.043 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:41:42.044 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:41:42.045 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:41:42.045 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:41:42.045 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:41:42.046 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:41:42.046 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:41:42.046 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:41:42.047 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:41:42.047 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:41:42.047 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:41:42.047 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.047 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "19", 2022-08-01T19:41:42.048 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:41:42.048 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:41:42.048 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:41:42.048 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:41:42.049 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:41:42.049 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:41:42.049 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:41:42.050 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:41:42.050 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:41:42.050 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:41:42.050 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:41:42.051 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:41:42.051 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:41:42.051 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:41:42.051 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:41:42.052 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:41:42.052 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:41:42.052 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:41:42.052 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:41:42.052 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:41:42.053 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:41:42.053 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:41:42.053 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:41:42.053 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:41:42.054 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:41:42.054 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:41:42.054 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.055 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:41:42.055 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:41:42.055 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:41:42.055 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:41:42.056 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:41:42.056 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:41:42.056 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:41:42.057 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:41:42.057 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:41:42.057 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:41:42.057 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:41:42.058 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:41:42.058 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:41:42.058 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd": {} 2022-08-01T19:41:42.059 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.059 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.060 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:41:42.061 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2022-08-01T19:41:42.061 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.062 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:41:42.062 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a962f14f-cf16-4fa8-84ff-c85325db9132", 2022-08-01T19:41:42.063 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:42.063 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:42.064 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:42.064 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:42.065 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:42.065 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:42.066 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:41:42.066 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 96, 2022-08-01T19:41:42.066 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:42.067 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:42.067 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:42.068 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:42.068 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.068 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:42.069 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6800", 2022-08-01T19:41:42.069 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:42.070 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.070 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.070 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:42.071 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6801", 2022-08-01T19:41:42.071 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:42.071 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.071 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.072 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.072 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:42.072 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:42.072 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.073 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:42.073 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6802", 2022-08-01T19:41:42.073 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:42.073 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.074 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.074 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:42.075 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6803", 2022-08-01T19:41:42.075 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:42.076 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.076 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.076 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.077 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:42.077 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:42.078 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.078 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:42.078 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6806", 2022-08-01T19:41:42.079 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:42.079 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.079 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.079 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:42.080 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6807", 2022-08-01T19:41:42.080 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:42.080 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.081 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.081 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.081 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:42.082 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:42.082 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.083 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:42.083 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6804", 2022-08-01T19:41:42.083 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:42.084 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.084 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.085 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:42.085 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6805", 2022-08-01T19:41:42.086 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:42.086 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.087 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.087 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.088 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6801/23008", 2022-08-01T19:41:42.088 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6803/23008", 2022-08-01T19:41:42.088 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6807/23008", 2022-08-01T19:41:42.089 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6805/23008", 2022-08-01T19:41:42.089 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:42.090 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:42.090 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:42.091 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.091 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.091 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.092 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:41:42.092 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "805f7051-a527-4e98-8844-b6e406a63248", 2022-08-01T19:41:42.092 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:42.092 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:42.092 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:42.093 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:42.093 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:42.093 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:42.093 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:41:42.094 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 96, 2022-08-01T19:41:42.094 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:42.094 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:42.094 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:42.095 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:42.095 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.095 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:42.096 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6808", 2022-08-01T19:41:42.097 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:42.097 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.098 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.098 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:42.098 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6809", 2022-08-01T19:41:42.099 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:42.099 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.100 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.100 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.101 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:42.101 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:42.102 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.102 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:42.103 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6810", 2022-08-01T19:41:42.103 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:42.103 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.104 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.104 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:42.105 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6811", 2022-08-01T19:41:42.105 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:42.105 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.107 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.107 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.108 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:42.108 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:42.108 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.109 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:42.109 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6814", 2022-08-01T19:41:42.109 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:42.110 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.110 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.110 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:42.110 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6815", 2022-08-01T19:41:42.111 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:42.111 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.112 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.112 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.112 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:42.112 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:42.113 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.113 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:42.113 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6812", 2022-08-01T19:41:42.113 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:42.113 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.114 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.114 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:42.115 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6813", 2022-08-01T19:41:42.115 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:42.115 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.116 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.116 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.116 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6809/23007", 2022-08-01T19:41:42.116 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6811/23007", 2022-08-01T19:41:42.117 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6815/23007", 2022-08-01T19:41:42.117 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6813/23007", 2022-08-01T19:41:42.117 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:42.117 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:42.118 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:42.118 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.119 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.119 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.119 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:41:42.120 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "89f7f3df-7c10-48e1-8b69-9e4243923a61", 2022-08-01T19:41:42.120 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:42.121 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:42.121 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:42.122 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:42.122 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:42.123 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:42.123 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:41:42.124 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 96, 2022-08-01T19:41:42.125 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:42.125 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:42.125 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:42.126 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:42.126 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.127 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:42.127 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6817", 2022-08-01T19:41:42.128 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:42.128 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.129 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.130 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:42.131 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6819", 2022-08-01T19:41:42.132 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:42.132 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.132 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.133 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.133 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:42.133 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:42.134 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.134 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:42.134 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6822", 2022-08-01T19:41:42.135 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:42.135 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.136 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.136 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:42.137 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6824", 2022-08-01T19:41:42.137 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:42.138 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.138 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.139 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.139 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:42.139 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:42.140 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.140 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:42.141 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6830", 2022-08-01T19:41:42.141 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:42.142 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.142 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.143 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:42.143 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6831", 2022-08-01T19:41:42.143 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:42.144 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.144 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.145 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.145 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:42.146 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:42.146 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.146 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:42.147 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6826", 2022-08-01T19:41:42.147 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:42.148 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.148 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.149 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:42.149 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6829", 2022-08-01T19:41:42.150 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:42.150 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.151 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.151 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.151 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6819/23010", 2022-08-01T19:41:42.151 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6824/23010", 2022-08-01T19:41:42.152 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6831/23010", 2022-08-01T19:41:42.152 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6829/23010", 2022-08-01T19:41:42.153 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:42.153 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:42.153 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:42.154 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.154 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.155 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.155 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:41:42.155 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "96123468-aabc-4a64-909e-a192c369090d", 2022-08-01T19:41:42.156 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:42.156 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:42.156 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:42.156 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:42.157 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:42.157 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:42.157 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:41:42.158 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 96, 2022-08-01T19:41:42.158 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:42.158 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:42.159 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:42.159 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:42.159 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.159 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:42.160 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6816", 2022-08-01T19:41:42.160 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:42.161 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.161 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.162 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:42.162 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6818", 2022-08-01T19:41:42.163 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:42.163 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.164 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.164 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.164 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:42.165 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:42.165 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.165 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:42.166 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6820", 2022-08-01T19:41:42.166 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:42.167 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.168 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.168 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:42.169 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6821", 2022-08-01T19:41:42.169 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:42.169 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.170 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.170 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.170 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:42.170 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:42.171 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.171 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:42.172 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6827", 2022-08-01T19:41:42.172 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:42.172 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.172 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.173 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:42.173 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6828", 2022-08-01T19:41:42.173 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:42.174 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.174 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.174 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.174 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:42.175 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:42.175 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.175 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:42.176 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6823", 2022-08-01T19:41:42.176 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:42.176 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.177 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.177 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:42.178 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6825", 2022-08-01T19:41:42.179 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:42.179 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.180 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.181 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6818/23009", 2022-08-01T19:41:42.181 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6821/23009", 2022-08-01T19:41:42.182 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6828/23009", 2022-08-01T19:41:42.182 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6825/23009", 2022-08-01T19:41:42.183 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:42.183 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:42.183 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:42.184 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.184 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.185 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.185 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:41:42.185 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a6e26665-e04b-46ec-ab66-e912acbf044c", 2022-08-01T19:41:42.186 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:42.186 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:42.187 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:42.187 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:42.187 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:42.187 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:42.187 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:41:42.188 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 95, 2022-08-01T19:41:42.188 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:42.188 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:42.189 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:42.189 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:42.190 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.190 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:42.191 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6810", 2022-08-01T19:41:42.191 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:42.192 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.192 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.193 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:42.193 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6811", 2022-08-01T19:41:42.193 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:42.194 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.195 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.195 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.196 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:42.196 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:42.197 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.197 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:42.198 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6812", 2022-08-01T19:41:42.198 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:42.198 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.199 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.200 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:42.200 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6813", 2022-08-01T19:41:42.201 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:42.201 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.202 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.202 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.202 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:42.203 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:42.203 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.203 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:42.203 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6816", 2022-08-01T19:41:42.204 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:42.204 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.204 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.204 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:42.205 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6817", 2022-08-01T19:41:42.205 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:42.205 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.205 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.206 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.206 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:42.206 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:42.207 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.207 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:42.207 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6814", 2022-08-01T19:41:42.207 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:42.208 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.208 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.208 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:42.208 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6815", 2022-08-01T19:41:42.209 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:42.209 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.209 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.209 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.209 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6811/23584", 2022-08-01T19:41:42.210 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6813/23584", 2022-08-01T19:41:42.210 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6817/23584", 2022-08-01T19:41:42.211 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6815/23584", 2022-08-01T19:41:42.211 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:42.212 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:42.212 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:42.212 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.213 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.213 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.214 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:41:42.214 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "502c0034-18ee-44ac-87ff-19b9dddb66a0", 2022-08-01T19:41:42.214 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:42.215 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:42.215 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:42.215 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:42.216 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:42.216 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:42.216 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:41:42.216 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 86, 2022-08-01T19:41:42.216 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:42.217 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:42.217 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:42.217 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:42.217 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.218 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:42.218 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6802", 2022-08-01T19:41:42.218 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:42.219 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.219 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.219 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:42.219 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6803", 2022-08-01T19:41:42.219 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:42.220 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.220 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.220 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.220 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:42.221 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:42.221 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.221 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:42.221 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6804", 2022-08-01T19:41:42.222 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:42.222 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.222 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.222 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:42.223 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6805", 2022-08-01T19:41:42.223 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:42.223 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.223 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.224 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.224 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:42.224 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:42.224 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.224 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:42.225 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6808", 2022-08-01T19:41:42.225 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:42.225 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.226 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.226 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:42.226 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6809", 2022-08-01T19:41:42.226 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:42.227 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.227 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.227 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.227 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:42.228 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:42.228 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.228 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:42.228 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6806", 2022-08-01T19:41:42.228 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:42.229 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.229 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.229 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:42.229 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6807", 2022-08-01T19:41:42.230 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:42.230 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.230 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.231 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.231 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6803/23585", 2022-08-01T19:41:42.231 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6805/23585", 2022-08-01T19:41:42.231 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6809/23585", 2022-08-01T19:41:42.232 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6807/23585", 2022-08-01T19:41:42.232 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:42.232 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:42.232 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:42.233 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.233 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.233 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.233 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:41:42.233 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "ac4e755a-7fe7-46cb-8407-852d5af9ffc5", 2022-08-01T19:41:42.234 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:42.234 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:42.235 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:42.235 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:42.235 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:42.235 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:42.236 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:41:42.236 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 95, 2022-08-01T19:41:42.236 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:42.236 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:42.237 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:42.237 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:42.237 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.237 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:42.238 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6820", 2022-08-01T19:41:42.238 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:42.238 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.238 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.239 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:42.239 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6821", 2022-08-01T19:41:42.239 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:42.239 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.239 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.240 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.240 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:42.240 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:42.241 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.241 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:42.241 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6823", 2022-08-01T19:41:42.241 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:42.242 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.242 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.242 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:42.242 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6825", 2022-08-01T19:41:42.243 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:42.243 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.243 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.243 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.244 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:42.244 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:42.244 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.244 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:42.244 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6831", 2022-08-01T19:41:42.245 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:42.245 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.245 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.245 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:42.246 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6833", 2022-08-01T19:41:42.246 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:42.246 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.246 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.247 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.247 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:42.247 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:42.247 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.248 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:42.248 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6827", 2022-08-01T19:41:42.248 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:42.248 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.249 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.249 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:42.249 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6829", 2022-08-01T19:41:42.250 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:42.250 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.250 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.250 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.251 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6821/23587", 2022-08-01T19:41:42.251 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6825/23587", 2022-08-01T19:41:42.251 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6833/23587", 2022-08-01T19:41:42.251 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6829/23587", 2022-08-01T19:41:42.252 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:42.252 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:42.252 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:42.252 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.252 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.253 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.253 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:41:42.253 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "874761e2-6410-40fc-b60c-e58abe5ce489", 2022-08-01T19:41:42.253 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:42.254 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:42.254 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:42.254 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:42.254 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:42.255 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:42.255 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:41:42.255 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 96, 2022-08-01T19:41:42.255 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:42.255 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:42.256 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:42.256 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:42.256 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.256 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:42.257 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6818", 2022-08-01T19:41:42.257 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:42.257 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.257 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.258 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:42.258 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6819", 2022-08-01T19:41:42.258 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:42.258 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.259 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.259 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.259 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:42.259 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:42.259 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.260 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:42.260 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6822", 2022-08-01T19:41:42.260 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:42.260 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.261 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.261 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:42.261 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6824", 2022-08-01T19:41:42.261 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:42.261 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.262 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.262 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.262 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:42.262 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:42.263 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.263 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:42.263 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6830", 2022-08-01T19:41:42.263 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:42.264 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.264 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.264 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:42.264 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6832", 2022-08-01T19:41:42.265 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:42.265 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.265 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.265 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.266 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:42.266 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:42.266 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.266 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:42.266 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6826", 2022-08-01T19:41:42.267 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:42.267 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.267 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.267 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:42.268 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6828", 2022-08-01T19:41:42.268 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:42.268 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.269 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.269 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.269 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6819/23586", 2022-08-01T19:41:42.269 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6824/23586", 2022-08-01T19:41:42.270 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6832/23586", 2022-08-01T19:41:42.270 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6828/23586", 2022-08-01T19:41:42.270 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:42.270 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:42.270 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:42.271 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:42.271 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.271 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:41:42.271 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_xinfo": [ 2022-08-01T19:41:42.272 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.272 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:41:42.272 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:42.272 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:42.273 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:42.273 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:42.273 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:42.273 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.588817+0000", 2022-08-01T19:41:42.273 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:42.274 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.274 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.274 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:41:42.274 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:42.275 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:42.275 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:42.275 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:42.275 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:42.276 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.782753+0000", 2022-08-01T19:41:42.276 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:42.276 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.276 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.276 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:41:42.277 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:42.277 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:42.278 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:42.278 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:42.278 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:42.279 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.837924+0000", 2022-08-01T19:41:42.279 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:42.279 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.279 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.280 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:41:42.280 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:42.280 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:42.281 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:42.281 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:42.282 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:42.282 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.839948+0000", 2022-08-01T19:41:42.283 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:42.283 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.284 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.284 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:41:42.284 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:42.285 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:42.285 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:42.285 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:42.286 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:42.286 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.112801+0000", 2022-08-01T19:41:42.287 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:42.287 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.287 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.288 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:41:42.288 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:42.288 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:42.289 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:42.289 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:42.290 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:42.290 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.106704+0000", 2022-08-01T19:41:42.290 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:42.291 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.291 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.292 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:41:42.292 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:42.293 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:42.293 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:42.294 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:42.294 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:42.294 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.893170+0000", 2022-08-01T19:41:42.294 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:42.295 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.295 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:42.295 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:41:42.295 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:42.296 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:42.296 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:42.296 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:42.296 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:42.297 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.886947+0000", 2022-08-01T19:41:42.297 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:42.298 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.298 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:41:42.298 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap": [], 2022-08-01T19:41:42.299 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap_items": [], 2022-08-01T19:41:42.299 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_temp": [], 2022-08-01T19:41:42.299 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_temp": [], 2022-08-01T19:41:42.299 INFO:teuthology.orchestra.run.smithi017.stdout: "blacklist": { 2022-08-01T19:41:42.300 INFO:teuthology.orchestra.run.smithi017.stdout: "172.21.15.17:6835/2788758172": "2022-08-02T19:41:36.039321+0000", 2022-08-01T19:41:42.300 INFO:teuthology.orchestra.run.smithi017.stdout: "172.21.15.17:6834/2788758172": "2022-08-02T19:41:36.039321+0000" 2022-08-01T19:41:42.300 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.301 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profiles": { 2022-08-01T19:41:42.302 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_data": { 2022-08-01T19:41:42.302 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-device-class": "", 2022-08-01T19:41:42.303 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:41:42.303 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-root": "default", 2022-08-01T19:41:42.304 INFO:teuthology.orchestra.run.smithi017.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T19:41:42.305 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:41:42.305 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "2", 2022-08-01T19:41:42.306 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:41:42.306 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:41:42.307 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van", 2022-08-01T19:41:42.307 INFO:teuthology.orchestra.run.smithi017.stdout: "w": "8" 2022-08-01T19:41:42.308 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.308 INFO:teuthology.orchestra.run.smithi017.stdout: "default": { 2022-08-01T19:41:42.309 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:41:42.310 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:41:42.310 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "1", 2022-08-01T19:41:42.311 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:41:42.311 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:41:42.312 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van" 2022-08-01T19:41:42.312 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:42.313 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:42.313 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps_queue": [], 2022-08-01T19:41:42.313 INFO:teuthology.orchestra.run.smithi017.stdout: "new_removed_snaps": [], 2022-08-01T19:41:42.314 INFO:teuthology.orchestra.run.smithi017.stdout: "new_purged_snaps": [], 2022-08-01T19:41:42.314 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_node_flags": {}, 2022-08-01T19:41:42.315 INFO:teuthology.orchestra.run.smithi017.stdout: "device_class_flags": {} 2022-08-01T19:41:42.316 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:41:42.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.006+0000 7f8db221b700 1 -- 172.21.15.17:0/2317705903 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8d90053aa0 msgr2=0x7f8d90055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:42.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.006+0000 7f8db221b700 1 --2- 172.21.15.17:0/2317705903 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8d90053aa0 0x7f8d90055f60 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f8d9c00b000 tx=0x7f8d9c006df0).stop 2022-08-01T19:41:42.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.006+0000 7f8db221b700 1 -- 172.21.15.17:0/2317705903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8dac0b1f40 msgr2=0x7f8dac14a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:42.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.006+0000 7f8db221b700 1 --2- 172.21.15.17:0/2317705903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8dac0b1f40 0x7f8dac14a180 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 rx=0x7f8d94002f70 tx=0x7f8d94014000).stop 2022-08-01T19:41:42.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.006+0000 7f8db221b700 1 -- 172.21.15.17:0/2317705903 shutdown_connections 2022-08-01T19:41:42.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.006+0000 7f8db221b700 1 --2- 172.21.15.17:0/2317705903 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8d90053aa0 0x7f8d90055f60 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:42.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.006+0000 7f8db221b700 1 --2- 172.21.15.17:0/2317705903 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8dac148530 0x7f8dac1439d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:42.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.006+0000 7f8db221b700 1 --2- 172.21.15.17:0/2317705903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8dac0b1f40 0x7f8dac14a180 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:42.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.006+0000 7f8db221b700 1 --2- 172.21.15.17:0/2317705903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8dac0a7250 0x7f8dac145f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:42.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.006+0000 7f8db221b700 1 -- 172.21.15.17:0/2317705903 >> 172.21.15.17:0/2317705903 conn(0x7f8dac05f840 msgr2=0x7f8dac0b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:42.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.006+0000 7f8db221b700 1 -- 172.21.15.17:0/2317705903 shutdown_connections 2022-08-01T19:41:42.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.006+0000 7f8db221b700 1 -- 172.21.15.17:0/2317705903 wait complete. 2022-08-01T19:41:42.332 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:41:42.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.446+0000 7f4ea34f2700 1 Processor -- start 2022-08-01T19:41:42.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.446+0000 7f4ea34f2700 1 -- start start 2022-08-01T19:41:42.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.450+0000 7f4ea34f2700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f4e9c0ad670 0x7f4e9c0b0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:42.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.450+0000 7f4ea34f2700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e9c0b0c40 con 0x7f4e9c13fa40 2022-08-01T19:41:42.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.450+0000 7f4ea34f2700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e9c0b0ee0 con 0x7f4e9c0af120 2022-08-01T19:41:42.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.450+0000 7f4ea34f2700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e9c0ae9e0 con 0x7f4e9c0ad670 2022-08-01T19:41:42.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.450+0000 7f4ea2cf1700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f4e9c13fa40 0x7f4e9c13fea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34642/0 (socket says 172.21.15.17:34642) 2022-08-01T19:41:42.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.450+0000 7f4ea2cf1700 1 -- 172.21.15.17:0/1168323317 learned_addr learned my addr 172.21.15.17:0/1168323317 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:42.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.450+0000 7f4ea24f0700 1 --2- 172.21.15.17:0/1168323317 >> v2:172.21.15.77:3300/0 conn(0x7f4e9c0ad670 0x7f4e9c0b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:42.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.450+0000 7f4ea14ee700 1 -- 172.21.15.17:0/1168323317 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 319807663 0 0) 0x7f4e9c0b0c40 con 0x7f4e9c13fa40 2022-08-01T19:41:42.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.450+0000 7f4ea14ee700 1 -- 172.21.15.17:0/1168323317 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e84004370 con 0x7f4e9c13fa40 2022-08-01T19:41:42.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.450+0000 7f4ea14ee700 1 -- 172.21.15.17:0/1168323317 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3046421472 0 0) 0x7f4e9c0b0ee0 con 0x7f4e9c0af120 2022-08-01T19:41:42.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.450+0000 7f4ea14ee700 1 -- 172.21.15.17:0/1168323317 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e9c0b0c40 con 0x7f4e9c0af120 2022-08-01T19:41:42.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.450+0000 7f4ea24f0700 1 -- 172.21.15.17:0/1168323317 >> v1:172.21.15.77:6789/0 conn(0x7f4e9c0af120 legacy=0x7f4e9c0b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:42.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.450+0000 7f4ea24f0700 1 -- 172.21.15.17:0/1168323317 >> v1:172.21.15.17:6789/0 conn(0x7f4e9c13fa40 legacy=0x7f4e9c13fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:42.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.450+0000 7f4ea24f0700 1 -- 172.21.15.17:0/1168323317 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9c0aec80 con 0x7f4e9c0ad670 2022-08-01T19:41:42.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.450+0000 7f4ea24f0700 1 --2- 172.21.15.17:0/1168323317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e9c0ad670 0x7f4e9c0b0570 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f4e8c002f70 tx=0x7f4e8c006df0).ready entity=mon.1 client_cookie=4b30f09bc746732c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:42.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.450+0000 7f4ea14ee700 1 -- 172.21.15.17:0/1168323317 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e8c00d540 con 0x7f4e9c0ad670 2022-08-01T19:41:42.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.450+0000 7f4ea14ee700 1 -- 172.21.15.17:0/1168323317 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e8c000e90 con 0x7f4e9c0ad670 2022-08-01T19:41:42.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.450+0000 7f4ea14ee700 1 -- 172.21.15.17:0/1168323317 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e8c00d890 con 0x7f4e9c0ad670 2022-08-01T19:41:42.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.450+0000 7f4ea34f2700 1 -- 172.21.15.17:0/1168323317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e9c0ad670 msgr2=0x7f4e9c0b0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:42.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.450+0000 7f4ea34f2700 1 --2- 172.21.15.17:0/1168323317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e9c0ad670 0x7f4e9c0b0570 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f4e8c002f70 tx=0x7f4e8c006df0).stop 2022-08-01T19:41:42.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.450+0000 7f4ea34f2700 1 -- 172.21.15.17:0/1168323317 shutdown_connections 2022-08-01T19:41:42.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.450+0000 7f4ea34f2700 1 --2- 172.21.15.17:0/1168323317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e9c0ad670 0x7f4e9c0b0570 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:42.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.450+0000 7f4ea34f2700 1 -- 172.21.15.17:0/1168323317 >> 172.21.15.17:0/1168323317 conn(0x7f4e9c061360 msgr2=0x7f4e9c062f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:42.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.450+0000 7f4ea34f2700 1 -- 172.21.15.17:0/1168323317 shutdown_connections 2022-08-01T19:41:42.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.450+0000 7f4ea34f2700 1 -- 172.21.15.17:0/1168323317 wait complete. 2022-08-01T19:41:42.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.454+0000 7f4ea34f2700 1 Processor -- start 2022-08-01T19:41:42.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.454+0000 7f4ea34f2700 1 -- start start 2022-08-01T19:41:42.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.454+0000 7f4ea34f2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e9c0ad670 0x7f4e9c0ac6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:42.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.454+0000 7f4ea34f2700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4e9c0af120 0x7f4e9c0a9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:42.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.454+0000 7f4ea34f2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e9c13fa40 0x7f4e9c064930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:42.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.454+0000 7f4ea34f2700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c066360 con 0x7f4e9c13fa40 2022-08-01T19:41:42.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.454+0000 7f4ea34f2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c0ac120 con 0x7f4e9c0ad670 2022-08-01T19:41:42.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.454+0000 7f4ea34f2700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c066d00 con 0x7f4e9c0af120 2022-08-01T19:41:42.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.454+0000 7f4ea2cf1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e9c13fa40 0x7f4e9c064930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:42.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.454+0000 7f4ea2cf1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e9c13fa40 0x7f4e9c064930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42454/0 (socket says 172.21.15.17:42454) 2022-08-01T19:41:42.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.454+0000 7f4ea2cf1700 1 -- 172.21.15.17:0/2874355016 learned_addr learned my addr 172.21.15.17:0/2874355016 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:42.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.454+0000 7f4ea24f0700 1 --2- 172.21.15.17:0/2874355016 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e9c0ad670 0x7f4e9c0ac6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:42.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.454+0000 7f4ea1cef700 1 --2- 172.21.15.17:0/2874355016 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4e9c0af120 0x7f4e9c0a9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:42.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.458+0000 7f4ea2cf1700 1 -- 172.21.15.17:0/2874355016 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4e9c0af120 msgr2=0x7f4e9c0a9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:42.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.458+0000 7f4ea2cf1700 1 --2- 172.21.15.17:0/2874355016 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4e9c0af120 0x7f4e9c0a9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:42.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.458+0000 7f4ea2cf1700 1 -- 172.21.15.17:0/2874355016 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e9c0ad670 msgr2=0x7f4e9c0ac6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:42.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.458+0000 7f4ea2cf1700 1 --2- 172.21.15.17:0/2874355016 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e9c0ad670 0x7f4e9c0ac6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:42.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.458+0000 7f4ea2cf1700 1 -- 172.21.15.17:0/2874355016 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e8c00b180 con 0x7f4e9c13fa40 2022-08-01T19:41:42.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.458+0000 7f4ea2cf1700 1 --2- 172.21.15.17:0/2874355016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e9c13fa40 0x7f4e9c064930 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 rx=0x7f4e94001490 tx=0x7f4e94007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:42.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.458+0000 7f4e8b7fe700 1 -- 172.21.15.17:0/2874355016 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e94005ba0 con 0x7f4e9c13fa40 2022-08-01T19:41:42.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.458+0000 7f4e8b7fe700 1 -- 172.21.15.17:0/2874355016 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e94005d00 con 0x7f4e9c13fa40 2022-08-01T19:41:42.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.458+0000 7f4ea34f2700 1 -- 172.21.15.17:0/2874355016 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e9c066fa0 con 0x7f4e9c13fa40 2022-08-01T19:41:42.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.458+0000 7f4e8b7fe700 1 -- 172.21.15.17:0/2874355016 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e940072c0 con 0x7f4e9c13fa40 2022-08-01T19:41:42.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.458+0000 7f4ea34f2700 1 -- 172.21.15.17:0/2874355016 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e9c067440 con 0x7f4e9c13fa40 2022-08-01T19:41:42.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.458+0000 7f4ea34f2700 1 -- 172.21.15.17:0/2874355016 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e9c0ac2e0 con 0x7f4e9c13fa40 2022-08-01T19:41:42.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.458+0000 7f4e8b7fe700 1 -- 172.21.15.17:0/2874355016 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4e9401a060 con 0x7f4e9c13fa40 2022-08-01T19:41:42.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.458+0000 7f4e8b7fe700 1 --2- 172.21.15.17:0/2874355016 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4e7c053a70 0x7f4e7c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:42.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.462+0000 7f4e8b7fe700 1 -- 172.21.15.17:0/2874355016 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f4e9406b370 con 0x7f4e9c13fa40 2022-08-01T19:41:42.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.462+0000 7f4ea24f0700 1 --2- 172.21.15.17:0/2874355016 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4e7c053a70 0x7f4e7c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:42.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.462+0000 7f4ea24f0700 1 --2- 172.21.15.17:0/2874355016 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4e7c053a70 0x7f4e7c055f30 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f4e8c018000 tx=0x7f4e8c014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:42.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.466+0000 7f4e8b7fe700 1 -- 172.21.15.17:0/2874355016 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4e94042410 con 0x7f4e9c13fa40 2022-08-01T19:41:42.636 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.634+0000 7f4ea34f2700 1 -- 172.21.15.17:0/2874355016 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e9c067130 con 0x7f4e9c13fa40 2022-08-01T19:41:42.637 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.634+0000 7f4e8b7fe700 1 -- 172.21.15.17:0/2874355016 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+1559 (secure 0 0 0) 0x7f4e94018020 con 0x7f4e9c13fa40 2022-08-01T19:41:42.638 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:42.639 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":49,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6943,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/256800053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":256800053},{"type":"v1","addr":"172.21.15.77:6835","nonce":256800053}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44},{"gid":6957,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3629116150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3629116150},{"type":"v1","addr":"172.21.15.17:6833","nonce":3629116150}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43},{"gid":7200,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.17:6835/4089210135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":4089210135},{"type":"v1","addr":"172.21.15.17:6835","nonce":4089210135}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49}],"filesystems":[]} 2022-08-01T19:41:42.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.638+0000 7f4ea34f2700 1 -- 172.21.15.17:0/2874355016 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4e7c053a70 msgr2=0x7f4e7c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:42.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.638+0000 7f4ea34f2700 1 --2- 172.21.15.17:0/2874355016 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4e7c053a70 0x7f4e7c055f30 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f4e8c018000 tx=0x7f4e8c014000).stop 2022-08-01T19:41:42.645 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.638+0000 7f4ea34f2700 1 -- 172.21.15.17:0/2874355016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e9c13fa40 msgr2=0x7f4e9c064930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:42.645 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.638+0000 7f4ea34f2700 1 --2- 172.21.15.17:0/2874355016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e9c13fa40 0x7f4e9c064930 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 rx=0x7f4e94001490 tx=0x7f4e94007e00).stop 2022-08-01T19:41:42.645 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.638+0000 7f4ea34f2700 1 -- 172.21.15.17:0/2874355016 shutdown_connections 2022-08-01T19:41:42.645 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.638+0000 7f4ea34f2700 1 --2- 172.21.15.17:0/2874355016 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4e7c053a70 0x7f4e7c055f30 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:42.646 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.638+0000 7f4ea34f2700 1 --2- 172.21.15.17:0/2874355016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e9c13fa40 0x7f4e9c064930 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:42.646 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.638+0000 7f4ea34f2700 1 --2- 172.21.15.17:0/2874355016 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4e9c0af120 0x7f4e9c0a9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:42.646 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.638+0000 7f4ea34f2700 1 --2- 172.21.15.17:0/2874355016 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e9c0ad670 0x7f4e9c0ac6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:42.648 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.638+0000 7f4ea34f2700 1 -- 172.21.15.17:0/2874355016 >> 172.21.15.17:0/2874355016 conn(0x7f4e9c061360 msgr2=0x7f4e9c0b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:42.649 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.638+0000 7f4ea34f2700 1 -- 172.21.15.17:0/2874355016 shutdown_connections 2022-08-01T19:41:42.649 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.638+0000 7f4ea34f2700 1 -- 172.21.15.17:0/2874355016 wait complete. 2022-08-01T19:41:42.650 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 49 2022-08-01T19:41:42.652 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:41:42.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.762+0000 7f5d1c420700 1 Processor -- start 2022-08-01T19:41:42.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.762+0000 7f5d1c420700 1 -- start start 2022-08-01T19:41:42.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.762+0000 7f5d1c420700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f5d14148530 0x7f5d140b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:42.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.762+0000 7f5d1c420700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d140a7250 0x7f5d141448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:42.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.762+0000 7f5d1c420700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d140b04e0 con 0x7f5d140b1f40 2022-08-01T19:41:42.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.762+0000 7f5d1c420700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d140a9250 con 0x7f5d14148530 2022-08-01T19:41:42.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.762+0000 7f5d1c420700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d140b0780 con 0x7f5d140a7250 2022-08-01T19:41:42.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.762+0000 7f5d199bb700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f5d14148530 0x7f5d140b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:42.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.762+0000 7f5d1a9bd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d140a7250 0x7f5d141448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:42.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.762+0000 7f5d199bb700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f5d14148530 0x7f5d140b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:40380/0 (socket says 172.21.15.17:40380) 2022-08-01T19:41:42.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.762+0000 7f5d199bb700 1 -- 172.21.15.17:0/422986228 learned_addr learned my addr 172.21.15.17:0/422986228 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:42.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.762+0000 7f5d199bb700 1 -- 172.21.15.17:0/422986228 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d140a7250 msgr2=0x7f5d141448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:42.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.762+0000 7f5d199bb700 1 --2- 172.21.15.17:0/422986228 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d140a7250 0x7f5d141448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:42.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.762+0000 7f5d199bb700 1 -- 172.21.15.17:0/422986228 >> v1:172.21.15.77:6789/0 conn(0x7f5d140b1f40 legacy=0x7f5d140b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:42.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.762+0000 7f5d199bb700 1 -- 172.21.15.17:0/422986228 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d140b0a20 con 0x7f5d14148530 2022-08-01T19:41:42.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.762+0000 7f5d199bb700 1 --2- 172.21.15.17:0/422986228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d14148530 0x7f5d140b1160 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f5d141478c0 tx=0x7f5d04006eb0).ready entity=mon.1 client_cookie=81dea996fb279b68 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:42.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.762+0000 7f5d191ba700 1 -- 172.21.15.17:0/422986228 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d0400d5c0 con 0x7f5d14148530 2022-08-01T19:41:42.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.762+0000 7f5d191ba700 1 -- 172.21.15.17:0/422986228 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d04000e90 con 0x7f5d14148530 2022-08-01T19:41:42.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.762+0000 7f5d191ba700 1 -- 172.21.15.17:0/422986228 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d0400d950 con 0x7f5d14148530 2022-08-01T19:41:42.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.762+0000 7f5d1c420700 1 -- 172.21.15.17:0/422986228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d14148530 msgr2=0x7f5d140b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:42.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.762+0000 7f5d1c420700 1 --2- 172.21.15.17:0/422986228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d14148530 0x7f5d140b1160 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f5d141478c0 tx=0x7f5d04006eb0).stop 2022-08-01T19:41:42.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.762+0000 7f5d1c420700 1 -- 172.21.15.17:0/422986228 shutdown_connections 2022-08-01T19:41:42.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.762+0000 7f5d1c420700 1 --2- 172.21.15.17:0/422986228 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d140a7250 0x7f5d141448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:42.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.762+0000 7f5d1c420700 1 --2- 172.21.15.17:0/422986228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d14148530 0x7f5d140b1160 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:42.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.762+0000 7f5d1c420700 1 -- 172.21.15.17:0/422986228 >> 172.21.15.17:0/422986228 conn(0x7f5d1405f840 msgr2=0x7f5d1405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:42.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.762+0000 7f5d1c420700 1 -- 172.21.15.17:0/422986228 shutdown_connections 2022-08-01T19:41:42.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.762+0000 7f5d1c420700 1 -- 172.21.15.17:0/422986228 wait complete. 2022-08-01T19:41:42.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.766+0000 7f5d1c420700 1 Processor -- start 2022-08-01T19:41:42.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.766+0000 7f5d1c420700 1 -- start start 2022-08-01T19:41:42.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.766+0000 7f5d1c420700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d140a7250 0x7f5d140aea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:42.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.766+0000 7f5d1c420700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d140b1f40 0x7f5d14156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:42.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.766+0000 7f5d1c420700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d14148530 0x7f5d140ac480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:42.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.766+0000 7f5d1c420700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d140ab750 con 0x7f5d14148530 2022-08-01T19:41:42.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.766+0000 7f5d1c420700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d140ab9f0 con 0x7f5d140a7250 2022-08-01T19:41:42.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.766+0000 7f5d1c420700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d140abfe0 con 0x7f5d140b1f40 2022-08-01T19:41:42.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.770+0000 7f5d1a9bd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d14148530 0x7f5d140ac480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:42.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.770+0000 7f5d199bb700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d140b1f40 0x7f5d14156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:42.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.770+0000 7f5d1a9bd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d14148530 0x7f5d140ac480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42468/0 (socket says 172.21.15.17:42468) 2022-08-01T19:41:42.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.770+0000 7f5d1a1bc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d140a7250 0x7f5d140aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:42.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.770+0000 7f5d1a9bd700 1 -- 172.21.15.17:0/2821740206 learned_addr learned my addr 172.21.15.17:0/2821740206 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:42.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.770+0000 7f5d1a9bd700 1 -- 172.21.15.17:0/2821740206 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d140b1f40 msgr2=0x7f5d14156c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:42.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.770+0000 7f5d1a9bd700 1 --2- 172.21.15.17:0/2821740206 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d140b1f40 0x7f5d14156c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:42.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.770+0000 7f5d1a9bd700 1 -- 172.21.15.17:0/2821740206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d140a7250 msgr2=0x7f5d140aea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:42.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.770+0000 7f5d1a9bd700 1 --2- 172.21.15.17:0/2821740206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d140a7250 0x7f5d140aea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:42.802 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.770+0000 7f5d1a9bd700 1 -- 172.21.15.17:0/2821740206 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0400b1e0 con 0x7f5d14148530 2022-08-01T19:41:42.802 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.770+0000 7f5d1a9bd700 1 --2- 172.21.15.17:0/2821740206 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d14148530 0x7f5d140ac480 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0x7f5d0c00c000 tx=0x7f5d0c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:42.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.770+0000 7f5d037fe700 1 -- 172.21.15.17:0/2821740206 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d0c011750 con 0x7f5d14148530 2022-08-01T19:41:42.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.770+0000 7f5d037fe700 1 -- 172.21.15.17:0/2821740206 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d0c0118b0 con 0x7f5d14148530 2022-08-01T19:41:42.804 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.770+0000 7f5d037fe700 1 -- 172.21.15.17:0/2821740206 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d0c011c10 con 0x7f5d14148530 2022-08-01T19:41:42.804 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.770+0000 7f5d1c420700 1 -- 172.21.15.17:0/2821740206 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d140af730 con 0x7f5d14148530 2022-08-01T19:41:42.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.770+0000 7f5d1c420700 1 -- 172.21.15.17:0/2821740206 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d140ac970 con 0x7f5d14148530 2022-08-01T19:41:42.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.770+0000 7f5d1c420700 1 -- 172.21.15.17:0/2821740206 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d140ac1a0 con 0x7f5d14148530 2022-08-01T19:41:42.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.770+0000 7f5d037fe700 1 -- 172.21.15.17:0/2821740206 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5d0c011d70 con 0x7f5d14148530 2022-08-01T19:41:42.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.774+0000 7f5d037fe700 1 --2- 172.21.15.17:0/2821740206 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5cf4053aa0 0x7f5cf4055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:42.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.774+0000 7f5d037fe700 1 -- 172.21.15.17:0/2821740206 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f5d0c06c020 con 0x7f5d14148530 2022-08-01T19:41:42.807 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.774+0000 7f5d1a1bc700 1 --2- 172.21.15.17:0/2821740206 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5cf4053aa0 0x7f5cf4055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:42.807 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.774+0000 7f5d1a1bc700 1 --2- 172.21.15.17:0/2821740206 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5cf4053aa0 0x7f5cf4055f60 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f5d1000d000 tx=0x7f5d10007f10).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:42.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.778+0000 7f5d037fe700 1 -- 172.21.15.17:0/2821740206 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5d0c0430c0 con 0x7f5d14148530 2022-08-01T19:41:42.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.938+0000 7f5d1c420700 1 -- 172.21.15.17:0/2821740206 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d140574e0 con 0x7f5d14148530 2022-08-01T19:41:42.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.942+0000 7f5d037fe700 1 -- 172.21.15.17:0/2821740206 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+1559 (secure 0 0 0) 0x7f5d0c0466e0 con 0x7f5d14148530 2022-08-01T19:41:42.944 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:42.945 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":49,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6943,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/256800053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":256800053},{"type":"v1","addr":"172.21.15.77:6835","nonce":256800053}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44},{"gid":6957,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3629116150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3629116150},{"type":"v1","addr":"172.21.15.17:6833","nonce":3629116150}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43},{"gid":7200,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.17:6835/4089210135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":4089210135},{"type":"v1","addr":"172.21.15.17:6835","nonce":4089210135}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49}],"filesystems":[]} 2022-08-01T19:41:42.948 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.942+0000 7f5d1c420700 1 -- 172.21.15.17:0/2821740206 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5cf4053aa0 msgr2=0x7f5cf4055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:42.948 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.942+0000 7f5d1c420700 1 --2- 172.21.15.17:0/2821740206 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5cf4053aa0 0x7f5cf4055f60 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f5d1000d000 tx=0x7f5d10007f10).stop 2022-08-01T19:41:42.949 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.942+0000 7f5d1c420700 1 -- 172.21.15.17:0/2821740206 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d14148530 msgr2=0x7f5d140ac480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:42.949 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.942+0000 7f5d1c420700 1 --2- 172.21.15.17:0/2821740206 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d14148530 0x7f5d140ac480 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0x7f5d0c00c000 tx=0x7f5d0c006df0).stop 2022-08-01T19:41:42.950 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.942+0000 7f5d1c420700 1 -- 172.21.15.17:0/2821740206 shutdown_connections 2022-08-01T19:41:42.951 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.942+0000 7f5d1c420700 1 --2- 172.21.15.17:0/2821740206 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5cf4053aa0 0x7f5cf4055f60 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:42.952 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.942+0000 7f5d1c420700 1 --2- 172.21.15.17:0/2821740206 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d14148530 0x7f5d140ac480 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:42.952 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.942+0000 7f5d1c420700 1 --2- 172.21.15.17:0/2821740206 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d140b1f40 0x7f5d14156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:42.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.942+0000 7f5d1c420700 1 --2- 172.21.15.17:0/2821740206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d140a7250 0x7f5d140aea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:42.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.942+0000 7f5d1c420700 1 -- 172.21.15.17:0/2821740206 >> 172.21.15.17:0/2821740206 conn(0x7f5d1405f840 msgr2=0x7f5d14144de0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:42.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.942+0000 7f5d1c420700 1 -- 172.21.15.17:0/2821740206 shutdown_connections 2022-08-01T19:41:42.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:42.942+0000 7f5d1c420700 1 -- 172.21.15.17:0/2821740206 wait complete. 2022-08-01T19:41:42.954 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 49 2022-08-01T19:41:42.956 INFO:tasks.ceph.mds.a:Restarting daemon 2022-08-01T19:41:42.957 INFO:tasks.ceph.mds.a:Stopping old one... 2022-08-01T19:41:42.957 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-08-01T19:41:42.957 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:41:42.958 INFO:tasks.ceph.mds.c:Restarting daemon 2022-08-01T19:41:42.958 INFO:tasks.ceph.mds.c:Stopping old one... 2022-08-01T19:41:42.959 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-08-01T19:41:42.960 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:41:42.960 INFO:tasks.ceph.mds.b:Restarting daemon 2022-08-01T19:41:42.960 INFO:tasks.ceph.mds.b:Stopping old one... 2022-08-01T19:41:42.961 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-08-01T19:41:42.961 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:41:42.969 INFO:tasks.ceph.mds.a:Stopped 2022-08-01T19:41:42.970 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-08-01T19:41:42.976 INFO:tasks.ceph.mds.c:Stopped 2022-08-01T19:41:42.977 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-08-01T19:41:42.978 INFO:tasks.ceph.mds.a:Started 2022-08-01T19:41:42.979 DEBUG:teuthology.parallel:result is None 2022-08-01T19:41:42.980 INFO:tasks.ceph.mds.b:Stopped 2022-08-01T19:41:42.980 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-08-01T19:41:42.983 INFO:tasks.ceph.mds.c:Started 2022-08-01T19:41:42.983 DEBUG:teuthology.parallel:result is None 2022-08-01T19:41:42.986 INFO:tasks.ceph.mds.b:Started 2022-08-01T19:41:42.987 DEBUG:teuthology.parallel:result is None 2022-08-01T19:41:42.987 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-08-01T19:41:43.039 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2022-08-01T19:41:43.048 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2022-08-01T19:41:43.070 INFO:tasks.ceph.mds.b.smithi077.stdout:starting mds.b at 2022-08-01T19:41:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.082+0000 7f0fd0c1a700 1 Processor -- start 2022-08-01T19:41:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.082+0000 7f0fd0c1a700 1 -- start start 2022-08-01T19:41:43.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.082+0000 7f0fd0c1a700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0fcc0abe80 0x7f0fcc0aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:43.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.082+0000 7f0fd0c1a700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fcc146b10 con 0x7f0fcc0b0400 2022-08-01T19:41:43.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.086+0000 7f0fd0c1a700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fcc067370 con 0x7f0fcc066f90 2022-08-01T19:41:43.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.086+0000 7f0fd0c1a700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fcc0a99a0 con 0x7f0fcc0abe80 2022-08-01T19:41:43.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.086+0000 7f0fc9d9b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0fcc0abe80 0x7f0fcc0aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:43.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.086+0000 7f0fca59c700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7f0fcc066f90 0x7f0fcc0ab770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:51490/0 (socket says 172.21.15.17:51490) 2022-08-01T19:41:43.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.086+0000 7f0fca59c700 1 -- 172.21.15.17:0/944977070 learned_addr learned my addr 172.21.15.17:0/944977070 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:43.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.086+0000 7f0fcad9d700 1 --1- 172.21.15.17:0/944977070 >> v1:172.21.15.17:6789/0 conn(0x7f0fcc0b0400 0x7f0fcc14e060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34692/0 (socket says 172.21.15.17:34692) 2022-08-01T19:41:43.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.086+0000 7f0fc959a700 1 -- 172.21.15.17:0/944977070 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4218522265 0 0) 0x7f0fcc067370 con 0x7f0fcc066f90 2022-08-01T19:41:43.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.086+0000 7f0fc9d9b700 1 -- 172.21.15.17:0/944977070 >> v1:172.21.15.77:6789/0 conn(0x7f0fcc066f90 legacy=0x7f0fcc0ab770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:43.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.086+0000 7f0fc9d9b700 1 -- 172.21.15.17:0/944977070 >> v1:172.21.15.17:6789/0 conn(0x7f0fcc0b0400 legacy=0x7f0fcc14e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:43.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.086+0000 7f0fc9d9b700 1 -- 172.21.15.17:0/944977070 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fcc0a95e0 con 0x7f0fcc0abe80 2022-08-01T19:41:43.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.086+0000 7f0fc9d9b700 1 --2- 172.21.15.17:0/944977070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0fcc0abe80 0x7f0fcc0aff10 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 rx=0x7f0fc4001490 tx=0x7f0fc0006df0).ready entity=mon.0 client_cookie=e4438b838bee5aca server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:43.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.086+0000 7f0fc959a700 1 -- 172.21.15.17:0/944977070 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fc000d5c0 con 0x7f0fcc0abe80 2022-08-01T19:41:43.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.086+0000 7f0fc959a700 1 -- 172.21.15.17:0/944977070 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc0000e90 con 0x7f0fcc0abe80 2022-08-01T19:41:43.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.086+0000 7f0fc959a700 1 -- 172.21.15.17:0/944977070 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fc000d910 con 0x7f0fcc0abe80 2022-08-01T19:41:43.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.086+0000 7f0fd0c1a700 1 -- 172.21.15.17:0/944977070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0fcc0abe80 msgr2=0x7f0fcc0aff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:43.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.086+0000 7f0fd0c1a700 1 --2- 172.21.15.17:0/944977070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0fcc0abe80 0x7f0fcc0aff10 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 rx=0x7f0fc4001490 tx=0x7f0fc0006df0).stop 2022-08-01T19:41:43.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.086+0000 7f0fd0c1a700 1 -- 172.21.15.17:0/944977070 shutdown_connections 2022-08-01T19:41:43.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.086+0000 7f0fd0c1a700 1 --2- 172.21.15.17:0/944977070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0fcc0abe80 0x7f0fcc0aff10 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:43.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.086+0000 7f0fd0c1a700 1 -- 172.21.15.17:0/944977070 >> 172.21.15.17:0/944977070 conn(0x7f0fcc0b5570 msgr2=0x7f0fcc0b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:43.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.086+0000 7f0fd0c1a700 1 -- 172.21.15.17:0/944977070 shutdown_connections 2022-08-01T19:41:43.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.086+0000 7f0fd0c1a700 1 -- 172.21.15.17:0/944977070 wait complete. 2022-08-01T19:41:43.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.086+0000 7f0fd0c1a700 1 Processor -- start 2022-08-01T19:41:43.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.086+0000 7f0fd0c1a700 1 -- start start 2022-08-01T19:41:43.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.090+0000 7f0fd0c1a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0fcc066f90 0x7f0fcc0ac9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:43.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.090+0000 7f0fd0c1a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0fcc0abe80 0x7f0fcc13f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:43.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.090+0000 7f0fd0c1a700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0fcc0b0400 0x7f0fcc13d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:43.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.090+0000 7f0fd0c1a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc153810 con 0x7f0fcc0abe80 2022-08-01T19:41:43.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.090+0000 7f0fd0c1a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc13fe40 con 0x7f0fcc066f90 2022-08-01T19:41:43.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.090+0000 7f0fd0c1a700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc0ad060 con 0x7f0fcc0b0400 2022-08-01T19:41:43.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.090+0000 7f0fc9d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0fcc0abe80 0x7f0fcc13f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:43.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.090+0000 7f0fc9d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0fcc0abe80 0x7f0fcc13f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42504/0 (socket says 172.21.15.17:42504) 2022-08-01T19:41:43.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.090+0000 7f0fc9d9b700 1 -- 172.21.15.17:0/1430743498 learned_addr learned my addr 172.21.15.17:0/1430743498 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:43.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.090+0000 7f0fcad9d700 1 --2- 172.21.15.17:0/1430743498 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0fcc0b0400 0x7f0fcc13d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:43.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.090+0000 7f0fca59c700 1 --2- 172.21.15.17:0/1430743498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0fcc066f90 0x7f0fcc0ac9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:43.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.090+0000 7f0fc9d9b700 1 -- 172.21.15.17:0/1430743498 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0fcc0b0400 msgr2=0x7f0fcc13d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:43.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.090+0000 7f0fc9d9b700 1 --2- 172.21.15.17:0/1430743498 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0fcc0b0400 0x7f0fcc13d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:43.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.090+0000 7f0fc9d9b700 1 -- 172.21.15.17:0/1430743498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0fcc066f90 msgr2=0x7f0fcc0ac9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:43.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.090+0000 7f0fc9d9b700 1 --2- 172.21.15.17:0/1430743498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0fcc066f90 0x7f0fcc0ac9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:43.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.090+0000 7f0fc9d9b700 1 -- 172.21.15.17:0/1430743498 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc000b180 con 0x7f0fcc0abe80 2022-08-01T19:41:43.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.090+0000 7f0fcad9d700 1 --2- 172.21.15.17:0/1430743498 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0fcc0b0400 0x7f0fcc13d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:43.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.090+0000 7f0fca59c700 1 --2- 172.21.15.17:0/1430743498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0fcc066f90 0x7f0fcc0ac9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:43.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.090+0000 7f0fc9d9b700 1 --2- 172.21.15.17:0/1430743498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0fcc0abe80 0x7f0fcc13f160 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 rx=0x7f0fc4001490 tx=0x7f0fc0014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:43.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.090+0000 7f0fbb7fe700 1 -- 172.21.15.17:0/1430743498 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fc0002f50 con 0x7f0fcc0abe80 2022-08-01T19:41:43.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.090+0000 7f0fbb7fe700 1 -- 172.21.15.17:0/1430743498 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc001e400 con 0x7f0fcc0abe80 2022-08-01T19:41:43.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.090+0000 7f0fd0c1a700 1 -- 172.21.15.17:0/1430743498 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fcc0ad300 con 0x7f0fcc0abe80 2022-08-01T19:41:43.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.090+0000 7f0fbb7fe700 1 -- 172.21.15.17:0/1430743498 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fc001e760 con 0x7f0fcc0abe80 2022-08-01T19:41:43.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.090+0000 7f0fd0c1a700 1 -- 172.21.15.17:0/1430743498 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fcc13f650 con 0x7f0fcc0abe80 2022-08-01T19:41:43.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.094+0000 7f0fd0c1a700 1 -- 172.21.15.17:0/1430743498 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fcc0aaaa0 con 0x7f0fcc0abe80 2022-08-01T19:41:43.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.094+0000 7f0fbb7fe700 1 -- 172.21.15.17:0/1430743498 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0fc001c070 con 0x7f0fcc0abe80 2022-08-01T19:41:43.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.094+0000 7f0fbb7fe700 1 --2- 172.21.15.17:0/1430743498 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0fa8053a10 0x7f0fa8055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:43.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.094+0000 7f0fbb7fe700 1 -- 172.21.15.17:0/1430743498 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f0fc006ca90 con 0x7f0fcc0abe80 2022-08-01T19:41:43.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.094+0000 7f0fca59c700 1 --2- 172.21.15.17:0/1430743498 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0fa8053a10 0x7f0fa8055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:43.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.094+0000 7f0fca59c700 1 --2- 172.21.15.17:0/1430743498 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0fa8053a10 0x7f0fa8055ed0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f0fbc00b000 tx=0x7f0fbc009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:43.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.102+0000 7f0fbb7fe700 1 -- 172.21.15.17:0/1430743498 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0fc00439a0 con 0x7f0fcc0abe80 2022-08-01T19:41:43.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:43.238+0000 7f0fd0c1a700 1 -- 172.21.15.17:0/1430743498 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f0fcc0574e0 con 0x7f0fcc0abe80 2022-08-01T19:41:44.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.082+0000 7f0fbb7fe700 1 -- 172.21.15.17:0/1430743498 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v103) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0fc0023070 con 0x7f0fcc0abe80 2022-08-01T19:41:44.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.086+0000 7f0fd0c1a700 1 -- 172.21.15.17:0/1430743498 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0fa8053a10 msgr2=0x7f0fa8055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:44.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.086+0000 7f0fd0c1a700 1 --2- 172.21.15.17:0/1430743498 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0fa8053a10 0x7f0fa8055ed0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f0fbc00b000 tx=0x7f0fbc009040).stop 2022-08-01T19:41:44.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.086+0000 7f0fd0c1a700 1 -- 172.21.15.17:0/1430743498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0fcc0abe80 msgr2=0x7f0fcc13f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:44.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.086+0000 7f0fd0c1a700 1 --2- 172.21.15.17:0/1430743498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0fcc0abe80 0x7f0fcc13f160 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 rx=0x7f0fc4001490 tx=0x7f0fc0014000).stop 2022-08-01T19:41:44.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.086+0000 7f0fd0c1a700 1 -- 172.21.15.17:0/1430743498 shutdown_connections 2022-08-01T19:41:44.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.086+0000 7f0fd0c1a700 1 --2- 172.21.15.17:0/1430743498 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0fa8053a10 0x7f0fa8055ed0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:44.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.086+0000 7f0fd0c1a700 1 --2- 172.21.15.17:0/1430743498 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0fcc0b0400 0x7f0fcc13d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:44.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.086+0000 7f0fd0c1a700 1 --2- 172.21.15.17:0/1430743498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0fcc0abe80 0x7f0fcc13f160 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:44.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.086+0000 7f0fd0c1a700 1 --2- 172.21.15.17:0/1430743498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0fcc066f90 0x7f0fcc0ac9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:44.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.086+0000 7f0fd0c1a700 1 -- 172.21.15.17:0/1430743498 >> 172.21.15.17:0/1430743498 conn(0x7f0fcc0b5570 msgr2=0x7f0fcc143730 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:44.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.086+0000 7f0fd0c1a700 1 -- 172.21.15.17:0/1430743498 shutdown_connections 2022-08-01T19:41:44.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.086+0000 7f0fd0c1a700 1 -- 172.21.15.17:0/1430743498 wait complete. 2022-08-01T19:41:44.095 INFO:teuthology.orchestra.run.smithi017.stderr: removed all blacklist entries 2022-08-01T19:41:44.111 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-08-01T19:41:44.112 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-08-01T19:41:44.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.222+0000 7f101a71e700 1 Processor -- start 2022-08-01T19:41:44.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.222+0000 7f101a71e700 1 -- start start 2022-08-01T19:41:44.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.222+0000 7f101a71e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f10140b4550 0x7f10140b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:44.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.222+0000 7f101a71e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f10140b6af0 0x7f101412bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:44.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.222+0000 7f101a71e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1014144240 con 0x7f10140b6710 2022-08-01T19:41:44.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.222+0000 7f101a71e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1014143d00 con 0x7f10140b6af0 2022-08-01T19:41:44.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.222+0000 7f101a71e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1014143fa0 con 0x7f10140b4550 2022-08-01T19:41:44.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.222+0000 7f1018cbb700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f10140b6af0 0x7f101412bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:44.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.222+0000 7f1018cbb700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f10140b6af0 0x7f101412bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42514/0 (socket says 172.21.15.17:42514) 2022-08-01T19:41:44.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.222+0000 7f1018cbb700 1 -- 172.21.15.17:0/1803179634 learned_addr learned my addr 172.21.15.17:0/1803179634 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:44.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.222+0000 7f100b7fe700 1 --2- 172.21.15.17:0/1803179634 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f10140b4550 0x7f10140b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:44.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.222+0000 7f1018cbb700 1 -- 172.21.15.17:0/1803179634 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f10140b4550 msgr2=0x7f10140b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:44.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.222+0000 7f1018cbb700 1 --2- 172.21.15.17:0/1803179634 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f10140b4550 0x7f10140b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:44.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.222+0000 7f1018cbb700 1 -- 172.21.15.17:0/1803179634 >> v1:172.21.15.77:6789/0 conn(0x7f10140b6710 legacy=0x7f1014061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:44.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.222+0000 7f1018cbb700 1 -- 172.21.15.17:0/1803179634 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1014061780 con 0x7f10140b6af0 2022-08-01T19:41:44.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.222+0000 7f100b7fe700 1 --2- 172.21.15.17:0/1803179634 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f10140b4550 0x7f10140b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:44.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.222+0000 7f1018cbb700 1 --2- 172.21.15.17:0/1803179634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10140b6af0 0x7f101412bb90 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 rx=0x7f100c0032f0 tx=0x7f100c009290).ready entity=mon.0 client_cookie=26c7bc27eb0586a4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:44.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.222+0000 7f10137fe700 1 -- 172.21.15.17:0/1803179634 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f100c00b8a0 con 0x7f10140b6af0 2022-08-01T19:41:44.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.222+0000 7f10137fe700 1 -- 172.21.15.17:0/1803179634 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100c00ba00 con 0x7f10140b6af0 2022-08-01T19:41:44.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.222+0000 7f10137fe700 1 -- 172.21.15.17:0/1803179634 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f100c00bb70 con 0x7f10140b6af0 2022-08-01T19:41:44.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.222+0000 7f101a71e700 1 -- 172.21.15.17:0/1803179634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10140b6af0 msgr2=0x7f101412bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:44.271 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.222+0000 7f101a71e700 1 --2- 172.21.15.17:0/1803179634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10140b6af0 0x7f101412bb90 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 rx=0x7f100c0032f0 tx=0x7f100c009290).stop 2022-08-01T19:41:44.271 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.222+0000 7f101a71e700 1 -- 172.21.15.17:0/1803179634 shutdown_connections 2022-08-01T19:41:44.272 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.222+0000 7f101a71e700 1 --2- 172.21.15.17:0/1803179634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10140b6af0 0x7f101412bb90 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:44.272 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.222+0000 7f101a71e700 1 --2- 172.21.15.17:0/1803179634 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f10140b4550 0x7f10140b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:44.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.222+0000 7f101a71e700 1 -- 172.21.15.17:0/1803179634 >> 172.21.15.17:0/1803179634 conn(0x7f101405c620 msgr2=0x7f101405ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:44.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.222+0000 7f101a71e700 1 -- 172.21.15.17:0/1803179634 shutdown_connections 2022-08-01T19:41:44.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.222+0000 7f101a71e700 1 -- 172.21.15.17:0/1803179634 wait complete. 2022-08-01T19:41:44.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.226+0000 7f101a71e700 1 Processor -- start 2022-08-01T19:41:44.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.226+0000 7f101a71e700 1 -- start start 2022-08-01T19:41:44.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.230+0000 7f101a71e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10140b4550 0x7f1014148ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:44.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.230+0000 7f101a71e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10140b6710 0x7f10141407f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:44.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.230+0000 7f101a71e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f10140b6af0 0x7f1014140ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:44.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.230+0000 7f101a71e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101412b570 con 0x7f10140b6710 2022-08-01T19:41:44.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.230+0000 7f101a71e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101412ab60 con 0x7f10140b4550 2022-08-01T19:41:44.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.230+0000 7f101a71e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f101412b7a0 con 0x7f10140b6af0 2022-08-01T19:41:44.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.230+0000 7f100b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10140b6710 0x7f10141407f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:44.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.230+0000 7f100b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10140b6710 0x7f10141407f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42518/0 (socket says 172.21.15.17:42518) 2022-08-01T19:41:44.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.230+0000 7f100b7fe700 1 -- 172.21.15.17:0/2956787335 learned_addr learned my addr 172.21.15.17:0/2956787335 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:44.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.230+0000 7f1013fff700 1 --2- 172.21.15.17:0/2956787335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10140b4550 0x7f1014148ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:44.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.230+0000 7f1018cbb700 1 --2- 172.21.15.17:0/2956787335 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f10140b6af0 0x7f1014140ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:44.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.230+0000 7f100b7fe700 1 -- 172.21.15.17:0/2956787335 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f10140b6af0 msgr2=0x7f1014140ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:44.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.230+0000 7f100b7fe700 1 --2- 172.21.15.17:0/2956787335 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f10140b6af0 0x7f1014140ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:44.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.230+0000 7f100b7fe700 1 -- 172.21.15.17:0/2956787335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10140b4550 msgr2=0x7f1014148ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:44.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.230+0000 7f100b7fe700 1 --2- 172.21.15.17:0/2956787335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10140b4550 0x7f1014148ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:44.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.230+0000 7f100b7fe700 1 -- 172.21.15.17:0/2956787335 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f100c009530 con 0x7f10140b6710 2022-08-01T19:41:44.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.230+0000 7f1018cbb700 1 --2- 172.21.15.17:0/2956787335 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f10140b6af0 0x7f1014140ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:44.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.230+0000 7f100b7fe700 1 --2- 172.21.15.17:0/2956787335 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10140b6710 0x7f10141407f0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 rx=0x7f0ffc00a000 tx=0x7f0ffc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:44.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.230+0000 7f1011ffb700 1 -- 172.21.15.17:0/2956787335 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ffc011780 con 0x7f10140b6710 2022-08-01T19:41:44.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.230+0000 7f1011ffb700 1 -- 172.21.15.17:0/2956787335 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ffc0118e0 con 0x7f10140b6710 2022-08-01T19:41:44.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.230+0000 7f101a71e700 1 -- 172.21.15.17:0/2956787335 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f101412ad90 con 0x7f10140b6710 2022-08-01T19:41:44.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.230+0000 7f1011ffb700 1 -- 172.21.15.17:0/2956787335 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ffc011c40 con 0x7f10140b6710 2022-08-01T19:41:44.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.230+0000 7f101a71e700 1 -- 172.21.15.17:0/2956787335 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10140ae380 con 0x7f10140b6710 2022-08-01T19:41:44.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.230+0000 7f1011ffb700 1 -- 172.21.15.17:0/2956787335 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0ffc012420 con 0x7f10140b6710 2022-08-01T19:41:44.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.230+0000 7f101a71e700 1 -- 172.21.15.17:0/2956787335 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f101412af20 con 0x7f10140b6710 2022-08-01T19:41:44.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.230+0000 7f1011ffb700 1 --2- 172.21.15.17:0/2956787335 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0ff8053a70 0x7f0ff8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:44.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.234+0000 7f1011ffb700 1 -- 172.21.15.17:0/2956787335 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f0ffc06ba80 con 0x7f10140b6710 2022-08-01T19:41:44.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.234+0000 7f1013fff700 1 --2- 172.21.15.17:0/2956787335 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0ff8053a70 0x7f0ff8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:44.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.234+0000 7f1013fff700 1 --2- 172.21.15.17:0/2956787335 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0ff8053a70 0x7f0ff8055f30 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7f100400d000 tx=0x7f1004000bc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:44.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.238+0000 7f1011ffb700 1 -- 172.21.15.17:0/2956787335 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0ffc042b20 con 0x7f10140b6710 2022-08-01T19:41:44.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.402+0000 7f101a71e700 1 -- 172.21.15.17:0/2956787335 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f1014061a20 con 0x7f10140b6710 2022-08-01T19:41:44.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.406+0000 7f1011ffb700 1 -- 172.21.15.17:0/2956787335 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v28) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f0ffc046140 con 0x7f10140b6710 2022-08-01T19:41:44.410 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:44.410 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:41:44.410 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_dump": [ 2022-08-01T19:41:44.411 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:44.411 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.a", 2022-08-01T19:41:44.411 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhijotxORAAVc04rsrlFj0NShsyc6Qbhg==", 2022-08-01T19:41:44.411 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:41:44.412 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:41:44.412 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:41:44.412 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:41:44.413 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:41:44.413 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:44.414 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:44.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:44.414 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.b", 2022-08-01T19:41:44.415 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhiud2xBxAAcn6/O5ysdIp/oIbF5Y1dAw==", 2022-08-01T19:41:44.415 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:41:44.415 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:41:44.416 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:41:44.416 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:41:44.417 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:41:44.417 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:44.418 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:44.418 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:44.418 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.c", 2022-08-01T19:41:44.418 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+KuhiMzMbBBAAfdT5mtpYwboNL+7f2B09KQ==", 2022-08-01T19:41:44.419 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:41:44.419 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:41:44.420 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:41:44.420 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:41:44.420 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:41:44.421 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:44.421 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:44.421 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:44.421 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.0", 2022-08-01T19:41:44.422 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhia27/NRAAZXBp+nYelJVJ6Mge9r+3Vg==", 2022-08-01T19:41:44.422 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:41:44.422 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:41:44.422 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:41:44.423 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:41:44.423 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:44.423 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:44.423 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:44.424 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.1", 2022-08-01T19:41:44.424 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCBKuhiOggRGRAAvgA0MxwBJwmpXV6S9DTdjA==", 2022-08-01T19:41:44.424 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:41:44.424 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:41:44.425 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:41:44.425 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:41:44.425 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:44.425 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:44.425 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:44.426 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.2", 2022-08-01T19:41:44.426 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCDKuhi5PgOORAAhcORK4bW2T3dXPeYjcCm+g==", 2022-08-01T19:41:44.426 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:41:44.426 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:41:44.427 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:41:44.427 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:41:44.427 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:44.427 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:44.428 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:44.428 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.3", 2022-08-01T19:41:44.428 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCGKuhiutn6GxAAk2ucLuGbT5YV/R4xXQGh0w==", 2022-08-01T19:41:44.428 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:41:44.429 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:41:44.429 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:41:44.429 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:41:44.429 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:44.430 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:44.430 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:44.430 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.4", 2022-08-01T19:41:44.430 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCLKuhiGHTRHxAArr3TwXA0gsCAAF/Qs38wUQ==", 2022-08-01T19:41:44.431 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:41:44.431 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:41:44.431 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:41:44.431 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:41:44.431 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:44.432 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:44.432 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:44.432 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.5", 2022-08-01T19:41:44.432 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCOKuhiMMDbBBAAvTzBvOcv7pFWm/P/lNqB0w==", 2022-08-01T19:41:44.433 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:41:44.433 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:41:44.433 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:41:44.433 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:41:44.433 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:44.434 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:44.434 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:44.434 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.6", 2022-08-01T19:41:44.434 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCQKuhiG2H3IxAAufUcWqo9ULquxIrUD5vSfA==", 2022-08-01T19:41:44.435 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:41:44.435 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:41:44.435 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:41:44.436 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:41:44.436 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:44.436 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:44.436 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:44.437 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.7", 2022-08-01T19:41:44.437 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCTKuhiU50LChAAXuhrYBbHkk3eKzAPiEJFTg==", 2022-08-01T19:41:44.437 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:41:44.437 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:41:44.437 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:41:44.438 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:41:44.438 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:44.438 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:44.438 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:44.439 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.0", 2022-08-01T19:41:44.439 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhig3EnCxAANM20jSIQf/AKwuRX1qwT/A==", 2022-08-01T19:41:44.439 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:41:44.439 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:41:44.440 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow r", 2022-08-01T19:41:44.440 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T19:41:44.440 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:44.440 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:44.440 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:44.441 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.1", 2022-08-01T19:41:44.441 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+KuhicJR0DhAAivqEAZUhXnSwjVRwWm2bJg==", 2022-08-01T19:41:44.441 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:41:44.441 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:41:44.442 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow r", 2022-08-01T19:41:44.442 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T19:41:44.442 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:44.442 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:44.443 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:44.443 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.admin", 2022-08-01T19:41:44.443 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhiPTTGHxAAgfthCR6RX6j8F1YseFGNeA==", 2022-08-01T19:41:44.443 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:41:44.443 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:41:44.444 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:41:44.448 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:41:44.449 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:41:44.449 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:44.450 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:44.451 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:44.451 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.x", 2022-08-01T19:41:44.452 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9Kuhi2jvCNhAAyUxiGByZ+OZr8Y7dh3Qt5w==", 2022-08-01T19:41:44.452 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:41:44.452 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:41:44.453 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2022-08-01T19:41:44.453 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:41:44.454 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:44.454 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:44.454 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:44.455 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.y", 2022-08-01T19:41:44.455 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhiDJXpMhAA8aek67ydbRgNO0h+miws+w==", 2022-08-01T19:41:44.456 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:41:44.457 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:41:44.457 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2022-08-01T19:41:44.458 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:41:44.458 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:44.459 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:44.459 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:44.460 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:41:44.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.406+0000 7f101a71e700 1 -- 172.21.15.17:0/2956787335 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0ff8053a70 msgr2=0x7f0ff8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:44.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.406+0000 7f101a71e700 1 --2- 172.21.15.17:0/2956787335 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0ff8053a70 0x7f0ff8055f30 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7f100400d000 tx=0x7f1004000bc0).stop 2022-08-01T19:41:44.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.406+0000 7f101a71e700 1 -- 172.21.15.17:0/2956787335 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10140b6710 msgr2=0x7f10141407f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:44.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.406+0000 7f101a71e700 1 --2- 172.21.15.17:0/2956787335 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10140b6710 0x7f10141407f0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 rx=0x7f0ffc00a000 tx=0x7f0ffc008040).stop 2022-08-01T19:41:44.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.410+0000 7f101a71e700 1 -- 172.21.15.17:0/2956787335 shutdown_connections 2022-08-01T19:41:44.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.410+0000 7f101a71e700 1 --2- 172.21.15.17:0/2956787335 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0ff8053a70 0x7f0ff8055f30 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:44.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.410+0000 7f101a71e700 1 --2- 172.21.15.17:0/2956787335 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f10140b6af0 0x7f1014140ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:44.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.410+0000 7f101a71e700 1 --2- 172.21.15.17:0/2956787335 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10140b6710 0x7f10141407f0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:44.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.410+0000 7f101a71e700 1 --2- 172.21.15.17:0/2956787335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10140b4550 0x7f1014148ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:44.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.410+0000 7f101a71e700 1 -- 172.21.15.17:0/2956787335 >> 172.21.15.17:0/2956787335 conn(0x7f101405c620 msgr2=0x7f10141420a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:44.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.410+0000 7f101a71e700 1 -- 172.21.15.17:0/2956787335 shutdown_connections 2022-08-01T19:41:44.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.410+0000 7f101a71e700 1 -- 172.21.15.17:0/2956787335 wait complete. 2022-08-01T19:41:44.470 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:41:44.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.574+0000 7f6ff7316700 1 Processor -- start 2022-08-01T19:41:44.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.574+0000 7f6ff7316700 1 -- start start 2022-08-01T19:41:44.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.574+0000 7f6ff7316700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6ff00b28b0 0x7f6ff013e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:44.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.574+0000 7f6ff7316700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6ff0141920 0x7f6ff013e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:44.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.574+0000 7f6ff7316700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ff00aac20 con 0x7f6ff00b1180 2022-08-01T19:41:44.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.574+0000 7f6ff7316700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ff0141db0 con 0x7f6ff0141920 2022-08-01T19:41:44.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.574+0000 7f6ff7316700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff0140e20 con 0x7f6ff00b28b0 2022-08-01T19:41:44.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.574+0000 7f6ff6b15700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6ff0141920 0x7f6ff013e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:44.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.574+0000 7f6ff5b13700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f6ff00b1180 0x7f6ff0141210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34720/0 (socket says 172.21.15.17:34720) 2022-08-01T19:41:44.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.574+0000 7f6ff5b13700 1 -- 172.21.15.17:0/1472459430 learned_addr learned my addr 172.21.15.17:0/1472459430 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:44.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.574+0000 7f6ff6314700 1 --2- 172.21.15.17:0/1472459430 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6ff00b28b0 0x7f6ff013e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:44.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.578+0000 7f6ff5312700 1 -- 172.21.15.17:0/1472459430 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1987181370 0 0) 0x7f6fec001110 con 0x7f6ff00b1180 2022-08-01T19:41:44.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.578+0000 7f6ff5312700 1 -- 172.21.15.17:0/1472459430 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fd8003260 con 0x7f6ff00b1180 2022-08-01T19:41:44.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.578+0000 7f6ff5312700 1 -- 172.21.15.17:0/1472459430 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 790825039 0 0) 0x7f6fd8003260 con 0x7f6ff00b1180 2022-08-01T19:41:44.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.578+0000 7f6ff6b15700 1 -- 172.21.15.17:0/1472459430 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6ff00b28b0 msgr2=0x7f6ff013e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:44.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.578+0000 7f6ff6b15700 1 --2- 172.21.15.17:0/1472459430 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6ff00b28b0 0x7f6ff013e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:44.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.578+0000 7f6ff6b15700 1 -- 172.21.15.17:0/1472459430 >> v1:172.21.15.17:6789/0 conn(0x7f6ff00b1180 legacy=0x7f6ff0141210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:44.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.578+0000 7f6ff6b15700 1 -- 172.21.15.17:0/1472459430 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff00b2c90 con 0x7f6ff0141920 2022-08-01T19:41:44.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.578+0000 7f6ff6314700 1 --2- 172.21.15.17:0/1472459430 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6ff00b28b0 0x7f6ff013e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:44.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.578+0000 7f6ff6b15700 1 --2- 172.21.15.17:0/1472459430 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ff0141920 0x7f6ff013e6b0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 rx=0x7f6fec001990 tx=0x7f6fe80091d0).ready entity=mon.0 client_cookie=b54cf01b471e419c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:44.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.578+0000 7f6ff5312700 1 -- 172.21.15.17:0/1472459430 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fec0017f0 con 0x7f6ff0141920 2022-08-01T19:41:44.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.578+0000 7f6ff5312700 1 -- 172.21.15.17:0/1472459430 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe800b580 con 0x7f6ff0141920 2022-08-01T19:41:44.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.578+0000 7f6ff5312700 1 -- 172.21.15.17:0/1472459430 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fe800b6f0 con 0x7f6ff0141920 2022-08-01T19:41:44.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.578+0000 7f6ff7316700 1 -- 172.21.15.17:0/1472459430 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ff0141920 msgr2=0x7f6ff013e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:44.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.578+0000 7f6ff7316700 1 --2- 172.21.15.17:0/1472459430 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ff0141920 0x7f6ff013e6b0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 rx=0x7f6fec001990 tx=0x7f6fe80091d0).stop 2022-08-01T19:41:44.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.578+0000 7f6ff7316700 1 -- 172.21.15.17:0/1472459430 shutdown_connections 2022-08-01T19:41:44.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.578+0000 7f6ff7316700 1 --2- 172.21.15.17:0/1472459430 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ff0141920 0x7f6ff013e6b0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:44.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.578+0000 7f6ff7316700 1 --2- 172.21.15.17:0/1472459430 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6ff00b28b0 0x7f6ff013e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:44.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.578+0000 7f6ff7316700 1 -- 172.21.15.17:0/1472459430 >> 172.21.15.17:0/1472459430 conn(0x7f6ff0060310 msgr2=0x7f6ff00b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:44.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.578+0000 7f6ff7316700 1 -- 172.21.15.17:0/1472459430 shutdown_connections 2022-08-01T19:41:44.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.578+0000 7f6ff7316700 1 -- 172.21.15.17:0/1472459430 wait complete. 2022-08-01T19:41:44.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.578+0000 7f6ff7316700 1 Processor -- start 2022-08-01T19:41:44.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.578+0000 7f6ff7316700 1 -- start start 2022-08-01T19:41:44.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.582+0000 7f6ff7316700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6ff00b1180 0x7f6ff0142410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:44.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.582+0000 7f6ff7316700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ff00b28b0 0x7f6ff0144250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:44.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.582+0000 7f6ff7316700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ff0141920 0x7f6ff0144740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:44.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.582+0000 7f6ff7316700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff00ae830 con 0x7f6ff0141920 2022-08-01T19:41:44.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.582+0000 7f6ff7316700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff00b3770 con 0x7f6ff00b28b0 2022-08-01T19:41:44.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.582+0000 7f6ff7316700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff00b3d80 con 0x7f6ff00b1180 2022-08-01T19:41:44.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.582+0000 7f6ff6b15700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ff0141920 0x7f6ff0144740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:44.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.582+0000 7f6ff6314700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6ff00b1180 0x7f6ff0142410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:44.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.582+0000 7f6ff6b15700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ff0141920 0x7f6ff0144740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42536/0 (socket says 172.21.15.17:42536) 2022-08-01T19:41:44.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.582+0000 7f6ff6b15700 1 -- 172.21.15.17:0/126188684 learned_addr learned my addr 172.21.15.17:0/126188684 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:44.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.582+0000 7f6ff5b13700 1 --2- 172.21.15.17:0/126188684 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ff00b28b0 0x7f6ff0144250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:44.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.582+0000 7f6ff6b15700 1 -- 172.21.15.17:0/126188684 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6ff00b1180 msgr2=0x7f6ff0142410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:44.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.582+0000 7f6ff6b15700 1 --2- 172.21.15.17:0/126188684 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6ff00b1180 0x7f6ff0142410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:44.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.582+0000 7f6ff6b15700 1 -- 172.21.15.17:0/126188684 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ff00b28b0 msgr2=0x7f6ff0144250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.582+0000 7f6ff6b15700 1 --2- 172.21.15.17:0/126188684 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ff00b28b0 0x7f6ff0144250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.582+0000 7f6ff6b15700 1 -- 172.21.15.17:0/126188684 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe8009470 con 0x7f6ff0141920 2022-08-01T19:41:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.582+0000 7f6ff6314700 1 --2- 172.21.15.17:0/126188684 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6ff00b1180 0x7f6ff0142410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.582+0000 7f6ff6b15700 1 --2- 172.21.15.17:0/126188684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ff0141920 0x7f6ff0144740 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 rx=0x7f6fec001990 tx=0x7f6fe8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.582+0000 7f6fdf7fe700 1 -- 172.21.15.17:0/126188684 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fe8006ea0 con 0x7f6ff0141920 2022-08-01T19:41:44.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.582+0000 7f6fdf7fe700 1 -- 172.21.15.17:0/126188684 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe800b030 con 0x7f6ff0141920 2022-08-01T19:41:44.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.582+0000 7f6ff7316700 1 -- 172.21.15.17:0/126188684 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff00ac880 con 0x7f6ff0141920 2022-08-01T19:41:44.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.582+0000 7f6fdf7fe700 1 -- 172.21.15.17:0/126188684 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fe8011d30 con 0x7f6ff0141920 2022-08-01T19:41:44.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.582+0000 7f6ff7316700 1 -- 172.21.15.17:0/126188684 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff00b3490 con 0x7f6ff0141920 2022-08-01T19:41:44.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.582+0000 7f6ff7316700 1 -- 172.21.15.17:0/126188684 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ff00b3970 con 0x7f6ff0141920 2022-08-01T19:41:44.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.586+0000 7f6fdf7fe700 1 -- 172.21.15.17:0/126188684 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6fe800f070 con 0x7f6ff0141920 2022-08-01T19:41:44.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.586+0000 7f6fdf7fe700 1 --2- 172.21.15.17:0/126188684 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6fd0053980 0x7f6fd0055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:44.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.586+0000 7f6fdf7fe700 1 -- 172.21.15.17:0/126188684 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f6fe806ca10 con 0x7f6ff0141920 2022-08-01T19:41:44.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.586+0000 7f6ff6314700 1 --2- 172.21.15.17:0/126188684 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6fd0053980 0x7f6fd0055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:44.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.586+0000 7f6ff6314700 1 --2- 172.21.15.17:0/126188684 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6fd0053980 0x7f6fd0055e40 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f6fe000c000 tx=0x7f6fe0008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:44.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.590+0000 7f6fdf7fe700 1 -- 172.21.15.17:0/126188684 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6fe8043b10 con 0x7f6ff0141920 2022-08-01T19:41:44.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.718+0000 7f6ff7316700 1 -- 172.21.15.17:0/126188684 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ff00b3f30 con 0x7f6ff0141920 2022-08-01T19:41:44.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.718+0000 7f6fdf7fe700 1 -- 172.21.15.17:0/126188684 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+13416 (secure 0 0 0) 0x7f6fe8026070 con 0x7f6ff0141920 2022-08-01T19:41:44.725 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:44.725 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":103,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:41:44.070575+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:41:44.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.722+0000 7f6ff7316700 1 -- 172.21.15.17:0/126188684 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6fd0053980 msgr2=0x7f6fd0055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:44.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.722+0000 7f6ff7316700 1 --2- 172.21.15.17:0/126188684 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6fd0053980 0x7f6fd0055e40 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f6fe000c000 tx=0x7f6fe0008040).stop 2022-08-01T19:41:44.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.722+0000 7f6ff7316700 1 -- 172.21.15.17:0/126188684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ff0141920 msgr2=0x7f6ff0144740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:44.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.722+0000 7f6ff7316700 1 --2- 172.21.15.17:0/126188684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ff0141920 0x7f6ff0144740 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 rx=0x7f6fec001990 tx=0x7f6fe8016000).stop 2022-08-01T19:41:44.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.722+0000 7f6ff7316700 1 -- 172.21.15.17:0/126188684 shutdown_connections 2022-08-01T19:41:44.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.722+0000 7f6ff7316700 1 --2- 172.21.15.17:0/126188684 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6fd0053980 0x7f6fd0055e40 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:44.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.722+0000 7f6ff7316700 1 --2- 172.21.15.17:0/126188684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ff0141920 0x7f6ff0144740 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:44.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.722+0000 7f6ff7316700 1 --2- 172.21.15.17:0/126188684 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ff00b28b0 0x7f6ff0144250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:44.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.722+0000 7f6ff7316700 1 --2- 172.21.15.17:0/126188684 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6ff00b1180 0x7f6ff0142410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:44.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.722+0000 7f6ff7316700 1 -- 172.21.15.17:0/126188684 >> 172.21.15.17:0/126188684 conn(0x7f6ff0060310 msgr2=0x7f6ff0140240 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:44.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.722+0000 7f6ff7316700 1 -- 172.21.15.17:0/126188684 shutdown_connections 2022-08-01T19:41:44.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.722+0000 7f6ff7316700 1 -- 172.21.15.17:0/126188684 wait complete. 2022-08-01T19:41:44.736 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-08-01T19:41:44.737 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:41:44.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.842+0000 7f9de4a1b700 1 Processor -- start 2022-08-01T19:41:44.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.842+0000 7f9de4a1b700 1 -- start start 2022-08-01T19:41:44.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.842+0000 7f9de4a1b700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f9de00b2050 0x7f9de00b1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:44.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.842+0000 7f9de4a1b700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9de00b05b0 con 0x7f9de00b0140 2022-08-01T19:41:44.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.846+0000 7f9de4a1b700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9de00a7960 con 0x7f9de00b2430 2022-08-01T19:41:44.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.846+0000 7f9de4a1b700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9de00b07e0 con 0x7f9de00b2050 2022-08-01T19:41:44.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.846+0000 7f9dddd9b700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9de00b0140 0x7f9de00a7250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34734/0 (socket says 172.21.15.17:34734) 2022-08-01T19:41:44.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.846+0000 7f9dddd9b700 1 -- 172.21.15.17:0/103289510 learned_addr learned my addr 172.21.15.17:0/103289510 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:44.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.846+0000 7f9dde59c700 1 --2- 172.21.15.17:0/103289510 >> v2:172.21.15.77:3300/0 conn(0x7f9de00b2050 0x7f9de00b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:44.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.846+0000 7f9ddd59a700 1 -- 172.21.15.17:0/103289510 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3641780813 0 0) 0x7f9de00b05b0 con 0x7f9de00b0140 2022-08-01T19:41:44.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.846+0000 7f9ddd59a700 1 -- 172.21.15.17:0/103289510 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dc4003260 con 0x7f9de00b0140 2022-08-01T19:41:44.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.846+0000 7f9dde59c700 1 -- 172.21.15.17:0/103289510 >> v1:172.21.15.77:6789/0 conn(0x7f9de00b2430 legacy=0x7f9de01413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:44.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.846+0000 7f9ddd59a700 1 -- 172.21.15.17:0/103289510 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2146962271 0 0) 0x7f9dc4003260 con 0x7f9de00b0140 2022-08-01T19:41:44.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.846+0000 7f9dde59c700 1 -- 172.21.15.17:0/103289510 >> v1:172.21.15.17:6789/0 conn(0x7f9de00b0140 legacy=0x7f9de00a7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:44.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.846+0000 7f9dde59c700 1 -- 172.21.15.17:0/103289510 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de00b0a80 con 0x7f9de00b2050 2022-08-01T19:41:44.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.846+0000 7f9dde59c700 1 --2- 172.21.15.17:0/103289510 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9de00b2050 0x7f9de00b1460 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f9dd0001800 tx=0x7f9dd4006df0).ready entity=mon.1 client_cookie=552d149bb0952710 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:44.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.846+0000 7f9ddd59a700 1 -- 172.21.15.17:0/103289510 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9dd0001660 con 0x7f9de00b2050 2022-08-01T19:41:44.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.846+0000 7f9ddd59a700 1 -- 172.21.15.17:0/103289510 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd4000e90 con 0x7f9de00b2050 2022-08-01T19:41:44.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.846+0000 7f9ddd59a700 1 -- 172.21.15.17:0/103289510 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9dd4011400 con 0x7f9de00b2050 2022-08-01T19:41:44.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.846+0000 7f9de4a1b700 1 -- 172.21.15.17:0/103289510 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9de00b2050 msgr2=0x7f9de00b1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:44.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.846+0000 7f9de4a1b700 1 --2- 172.21.15.17:0/103289510 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9de00b2050 0x7f9de00b1460 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f9dd0001800 tx=0x7f9dd4006df0).stop 2022-08-01T19:41:44.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.846+0000 7f9de4a1b700 1 -- 172.21.15.17:0/103289510 shutdown_connections 2022-08-01T19:41:44.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.846+0000 7f9de4a1b700 1 --2- 172.21.15.17:0/103289510 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9de00b2050 0x7f9de00b1460 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:44.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.846+0000 7f9de4a1b700 1 -- 172.21.15.17:0/103289510 >> 172.21.15.17:0/103289510 conn(0x7f9de005f840 msgr2=0x7f9de005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:44.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.846+0000 7f9de4a1b700 1 -- 172.21.15.17:0/103289510 shutdown_connections 2022-08-01T19:41:44.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.846+0000 7f9de4a1b700 1 -- 172.21.15.17:0/103289510 wait complete. 2022-08-01T19:41:44.871 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.850+0000 7f9de4a1b700 1 Processor -- start 2022-08-01T19:41:44.871 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.850+0000 7f9de4a1b700 1 -- start start 2022-08-01T19:41:44.871 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.850+0000 7f9de4a1b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9de00b0140 0x7f9de0140720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:44.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.850+0000 7f9de4a1b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9de00b2050 0x7f9de00ab000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:44.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.850+0000 7f9de4a1b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9de00b2430 0x7f9de00ab4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:44.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.850+0000 7f9de4a1b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de006ac50 con 0x7f9de00b0140 2022-08-01T19:41:44.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.850+0000 7f9de4a1b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de00a9b60 con 0x7f9de00b2430 2022-08-01T19:41:44.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.850+0000 7f9de4a1b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de00b3380 con 0x7f9de00b2050 2022-08-01T19:41:44.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.850+0000 7f9dde59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9de00b0140 0x7f9de0140720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:44.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.850+0000 7f9dde59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9de00b0140 0x7f9de0140720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42546/0 (socket says 172.21.15.17:42546) 2022-08-01T19:41:44.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.850+0000 7f9dde59c700 1 -- 172.21.15.17:0/1942275690 learned_addr learned my addr 172.21.15.17:0/1942275690 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:44.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.850+0000 7f9dded9d700 1 --2- 172.21.15.17:0/1942275690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9de00b2430 0x7f9de00ab4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:44.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.850+0000 7f9dddd9b700 1 --2- 172.21.15.17:0/1942275690 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9de00b2050 0x7f9de00ab000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:44.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.850+0000 7f9dde59c700 1 -- 172.21.15.17:0/1942275690 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9de00b2050 msgr2=0x7f9de00ab000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:44.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.850+0000 7f9dde59c700 1 --2- 172.21.15.17:0/1942275690 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9de00b2050 0x7f9de00ab000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:44.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.850+0000 7f9dde59c700 1 -- 172.21.15.17:0/1942275690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9de00b2430 msgr2=0x7f9de00ab4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:44.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.850+0000 7f9dde59c700 1 --2- 172.21.15.17:0/1942275690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9de00b2430 0x7f9de00ab4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:44.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.850+0000 7f9dde59c700 1 -- 172.21.15.17:0/1942275690 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd400b180 con 0x7f9de00b0140 2022-08-01T19:41:44.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.850+0000 7f9dde59c700 1 --2- 172.21.15.17:0/1942275690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9de00b0140 0x7f9de0140720 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0x7f9dd0001800 tx=0x7f9de00b0c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:44.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.850+0000 7f9dcf7fe700 1 -- 172.21.15.17:0/1942275690 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9dd4015070 con 0x7f9de00b0140 2022-08-01T19:41:44.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.850+0000 7f9dcf7fe700 1 -- 172.21.15.17:0/1942275690 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd400db10 con 0x7f9de00b0140 2022-08-01T19:41:44.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.850+0000 7f9de4a1b700 1 -- 172.21.15.17:0/1942275690 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de005f0c0 con 0x7f9de00b0140 2022-08-01T19:41:44.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.850+0000 7f9dcf7fe700 1 -- 172.21.15.17:0/1942275690 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9dd4011400 con 0x7f9de00b0140 2022-08-01T19:41:44.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.850+0000 7f9de4a1b700 1 -- 172.21.15.17:0/1942275690 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de00ae5c0 con 0x7f9de00b0140 2022-08-01T19:41:44.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.854+0000 7f9de4a1b700 1 -- 172.21.15.17:0/1942275690 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9de00b0d20 con 0x7f9de00b0140 2022-08-01T19:41:44.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.854+0000 7f9dcf7fe700 1 -- 172.21.15.17:0/1942275690 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9dd400f070 con 0x7f9de00b0140 2022-08-01T19:41:44.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.854+0000 7f9dcf7fe700 1 --2- 172.21.15.17:0/1942275690 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9dbc053a70 0x7f9dbc055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:44.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.854+0000 7f9dcf7fe700 1 -- 172.21.15.17:0/1942275690 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f9dd406c380 con 0x7f9de00b0140 2022-08-01T19:41:44.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.854+0000 7f9dddd9b700 1 --2- 172.21.15.17:0/1942275690 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9dbc053a70 0x7f9dbc055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:44.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.854+0000 7f9dddd9b700 1 --2- 172.21.15.17:0/1942275690 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9dbc053a70 0x7f9dbc055f30 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f9dd000d000 tx=0x7f9dd0000bc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:44.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.858+0000 7f9dcf7fe700 1 -- 172.21.15.17:0/1942275690 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9dd4043530 con 0x7f9de00b0140 2022-08-01T19:41:44.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.986+0000 7f9de4a1b700 1 -- 172.21.15.17:0/1942275690 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f9de00574e0 con 0x7f9de00b0140 2022-08-01T19:41:44.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.986+0000 7f9dcf7fe700 1 -- 172.21.15.17:0/1942275690 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v103) v1 ==== 110+0+10 (secure 0 0 0) 0x7f9dd400a020 con 0x7f9de00b0140 2022-08-01T19:41:44.994 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:41:44.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.990+0000 7f9de4a1b700 1 -- 172.21.15.17:0/1942275690 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9dbc053a70 msgr2=0x7f9dbc055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:44.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.990+0000 7f9de4a1b700 1 --2- 172.21.15.17:0/1942275690 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9dbc053a70 0x7f9dbc055f30 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f9dd000d000 tx=0x7f9dd0000bc0).stop 2022-08-01T19:41:44.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.990+0000 7f9de4a1b700 1 -- 172.21.15.17:0/1942275690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9de00b0140 msgr2=0x7f9de0140720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:44.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.990+0000 7f9de4a1b700 1 --2- 172.21.15.17:0/1942275690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9de00b0140 0x7f9de0140720 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0x7f9dd0001800 tx=0x7f9de00b0c10).stop 2022-08-01T19:41:44.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.990+0000 7f9de4a1b700 1 -- 172.21.15.17:0/1942275690 shutdown_connections 2022-08-01T19:41:44.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.990+0000 7f9de4a1b700 1 --2- 172.21.15.17:0/1942275690 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9dbc053a70 0x7f9dbc055f30 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:44.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.990+0000 7f9de4a1b700 1 --2- 172.21.15.17:0/1942275690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9de00b2430 0x7f9de00ab4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:44.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.990+0000 7f9de4a1b700 1 --2- 172.21.15.17:0/1942275690 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9de00b2050 0x7f9de00ab000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:44.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.990+0000 7f9de4a1b700 1 --2- 172.21.15.17:0/1942275690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9de00b0140 0x7f9de0140720 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:44.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.990+0000 7f9de4a1b700 1 -- 172.21.15.17:0/1942275690 >> 172.21.15.17:0/1942275690 conn(0x7f9de005f840 msgr2=0x7f9de01466e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:44.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.990+0000 7f9de4a1b700 1 -- 172.21.15.17:0/1942275690 shutdown_connections 2022-08-01T19:41:44.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:44.990+0000 7f9de4a1b700 1 -- 172.21.15.17:0/1942275690 wait complete. 2022-08-01T19:41:45.005 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:41:45.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.110+0000 7f189168d700 1 Processor -- start 2022-08-01T19:41:45.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.110+0000 7f189168d700 1 -- start start 2022-08-01T19:41:45.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.110+0000 7f189168d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f188c0b2710 0x7f188c14afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:45.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.110+0000 7f189168d700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f188c0a7250 con 0x7f188c0a8c50 2022-08-01T19:41:45.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.110+0000 7f189168d700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f188c0a7480 con 0x7f188c0b01a0 2022-08-01T19:41:45.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.110+0000 7f189168d700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f188c0a7720 con 0x7f188c0b2710 2022-08-01T19:41:45.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.110+0000 7f188bfff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f188c0b2710 0x7f188c14afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:45.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.110+0000 7f1890e8c700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f188c0a8c50 0x7f188c151d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34752/0 (socket says 172.21.15.17:34752) 2022-08-01T19:41:45.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.110+0000 7f1890e8c700 1 -- 172.21.15.17:0/3230593231 learned_addr learned my addr 172.21.15.17:0/3230593231 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:45.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.110+0000 7f188bfff700 1 -- 172.21.15.17:0/3230593231 >> v1:172.21.15.77:6789/0 conn(0x7f188c0b01a0 legacy=0x7f188c0a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:45.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.110+0000 7f188bfff700 1 -- 172.21.15.17:0/3230593231 >> v1:172.21.15.17:6789/0 conn(0x7f188c0a8c50 legacy=0x7f188c151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:45.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.110+0000 7f188affd700 1 -- 172.21.15.17:0/3230593231 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3887598193 0 0) 0x7f188c0a7250 con 0x7f188c0a8c50 2022-08-01T19:41:45.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.110+0000 7f188bfff700 1 -- 172.21.15.17:0/3230593231 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188c0a79c0 con 0x7f188c0b2710 2022-08-01T19:41:45.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.110+0000 7f188bfff700 1 --2- 172.21.15.17:0/3230593231 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f188c0b2710 0x7f188c14afd0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7f1880002f70 tx=0x7f1880006df0).ready entity=mon.0 client_cookie=75c85fa794e01dfa server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:45.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.110+0000 7f188affd700 1 -- 172.21.15.17:0/3230593231 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f188000d610 con 0x7f188c0b2710 2022-08-01T19:41:45.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.110+0000 7f188affd700 1 -- 172.21.15.17:0/3230593231 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1880000e90 con 0x7f188c0b2710 2022-08-01T19:41:45.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.110+0000 7f188affd700 1 -- 172.21.15.17:0/3230593231 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f188000d960 con 0x7f188c0b2710 2022-08-01T19:41:45.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.110+0000 7f189168d700 1 -- 172.21.15.17:0/3230593231 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f188c0b2710 msgr2=0x7f188c14afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:45.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.110+0000 7f189168d700 1 --2- 172.21.15.17:0/3230593231 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f188c0b2710 0x7f188c14afd0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7f1880002f70 tx=0x7f1880006df0).stop 2022-08-01T19:41:45.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.110+0000 7f189168d700 1 -- 172.21.15.17:0/3230593231 shutdown_connections 2022-08-01T19:41:45.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.110+0000 7f189168d700 1 --2- 172.21.15.17:0/3230593231 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f188c0b2710 0x7f188c14afd0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:45.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.110+0000 7f189168d700 1 -- 172.21.15.17:0/3230593231 >> 172.21.15.17:0/3230593231 conn(0x7f188c060310 msgr2=0x7f188c0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:45.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.110+0000 7f189168d700 1 -- 172.21.15.17:0/3230593231 shutdown_connections 2022-08-01T19:41:45.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.110+0000 7f189168d700 1 -- 172.21.15.17:0/3230593231 wait complete. 2022-08-01T19:41:45.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.114+0000 7f189168d700 1 Processor -- start 2022-08-01T19:41:45.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.114+0000 7f189168d700 1 -- start start 2022-08-01T19:41:45.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.114+0000 7f189168d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f188c0a8c50 0x7f188c1573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:45.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.114+0000 7f189168d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f188c0b01a0 0x7f188c066d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:45.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.114+0000 7f189168d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f188c0b2710 0x7f188c067250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:45.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.114+0000 7f189168d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c157890 con 0x7f188c0b01a0 2022-08-01T19:41:45.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.114+0000 7f189168d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c148cb0 con 0x7f188c0b2710 2022-08-01T19:41:45.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.114+0000 7f189168d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f188c148ee0 con 0x7f188c0a8c50 2022-08-01T19:41:45.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.114+0000 7f188b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f188c0b01a0 0x7f188c066d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:45.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.114+0000 7f188b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f188c0b01a0 0x7f188c066d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42562/0 (socket says 172.21.15.17:42562) 2022-08-01T19:41:45.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.114+0000 7f188b7fe700 1 -- 172.21.15.17:0/1936195215 learned_addr learned my addr 172.21.15.17:0/1936195215 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:45.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.114+0000 7f188bfff700 1 --2- 172.21.15.17:0/1936195215 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f188c0a8c50 0x7f188c1573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:45.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.114+0000 7f1890e8c700 1 --2- 172.21.15.17:0/1936195215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f188c0b2710 0x7f188c067250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:45.134 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.114+0000 7f188b7fe700 1 -- 172.21.15.17:0/1936195215 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f188c0a8c50 msgr2=0x7f188c1573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:45.134 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.114+0000 7f188b7fe700 1 --2- 172.21.15.17:0/1936195215 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f188c0a8c50 0x7f188c1573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:45.134 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.114+0000 7f188b7fe700 1 -- 172.21.15.17:0/1936195215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f188c0b2710 msgr2=0x7f188c067250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:45.134 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.114+0000 7f188b7fe700 1 --2- 172.21.15.17:0/1936195215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f188c0b2710 0x7f188c067250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:45.135 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.114+0000 7f188b7fe700 1 -- 172.21.15.17:0/1936195215 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188000b180 con 0x7f188c0b01a0 2022-08-01T19:41:45.135 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.114+0000 7f188bfff700 1 --2- 172.21.15.17:0/1936195215 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f188c0a8c50 0x7f188c1573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:45.135 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.114+0000 7f188b7fe700 1 --2- 172.21.15.17:0/1936195215 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f188c0b01a0 0x7f188c066d60 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 rx=0x7f188c063d60 tx=0x7f187c007bf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:45.135 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.114+0000 7f18897fa700 1 -- 172.21.15.17:0/1936195215 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f187c00ce20 con 0x7f188c0b01a0 2022-08-01T19:41:45.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.114+0000 7f18897fa700 1 -- 172.21.15.17:0/1936195215 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187c00f470 con 0x7f188c0b01a0 2022-08-01T19:41:45.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.114+0000 7f18897fa700 1 -- 172.21.15.17:0/1936195215 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f187c00f7d0 con 0x7f188c0b01a0 2022-08-01T19:41:45.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.114+0000 7f189168d700 1 -- 172.21.15.17:0/1936195215 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f188c149180 con 0x7f188c0b01a0 2022-08-01T19:41:45.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.114+0000 7f189168d700 1 -- 172.21.15.17:0/1936195215 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f188c0641e0 con 0x7f188c0b01a0 2022-08-01T19:41:45.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.118+0000 7f189168d700 1 -- 172.21.15.17:0/1936195215 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f188c0574e0 con 0x7f188c0b01a0 2022-08-01T19:41:45.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.118+0000 7f18897fa700 1 -- 172.21.15.17:0/1936195215 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f187c019030 con 0x7f188c0b01a0 2022-08-01T19:41:45.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.118+0000 7f18897fa700 1 --2- 172.21.15.17:0/1936195215 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f186c053b00 0x7f186c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:45.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.118+0000 7f18897fa700 1 -- 172.21.15.17:0/1936195215 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f187c06b200 con 0x7f188c0b01a0 2022-08-01T19:41:45.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.118+0000 7f188bfff700 1 --2- 172.21.15.17:0/1936195215 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f186c053b00 0x7f186c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:45.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.118+0000 7f188bfff700 1 --2- 172.21.15.17:0/1936195215 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f186c053b00 0x7f186c055fc0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f188001a000 tx=0x7f1880015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:45.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.122+0000 7f18897fa700 1 -- 172.21.15.17:0/1936195215 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f187c042260 con 0x7f188c0b01a0 2022-08-01T19:41:45.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.246+0000 7f189168d700 1 -- 172.21.15.17:0/1936195215 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f188c0b3960 con 0x7f188c0b01a0 2022-08-01T19:41:45.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.250+0000 7f18897fa700 1 -- 172.21.15.17:0/1936195215 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v103) v1 ==== 92+0+10 (secure 0 0 0) 0x7f187c045880 con 0x7f188c0b01a0 2022-08-01T19:41:45.251 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:41:45.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.250+0000 7f189168d700 1 -- 172.21.15.17:0/1936195215 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f186c053b00 msgr2=0x7f186c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:45.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.250+0000 7f189168d700 1 --2- 172.21.15.17:0/1936195215 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f186c053b00 0x7f186c055fc0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f188001a000 tx=0x7f1880015000).stop 2022-08-01T19:41:45.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.250+0000 7f189168d700 1 -- 172.21.15.17:0/1936195215 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f188c0b01a0 msgr2=0x7f188c066d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:45.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.250+0000 7f189168d700 1 --2- 172.21.15.17:0/1936195215 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f188c0b01a0 0x7f188c066d60 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 rx=0x7f188c063d60 tx=0x7f187c007bf0).stop 2022-08-01T19:41:45.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.250+0000 7f189168d700 1 -- 172.21.15.17:0/1936195215 shutdown_connections 2022-08-01T19:41:45.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.250+0000 7f189168d700 1 --2- 172.21.15.17:0/1936195215 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f186c053b00 0x7f186c055fc0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:45.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.250+0000 7f189168d700 1 --2- 172.21.15.17:0/1936195215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f188c0b2710 0x7f188c067250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:45.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.250+0000 7f189168d700 1 --2- 172.21.15.17:0/1936195215 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f188c0b01a0 0x7f188c066d60 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:45.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.250+0000 7f189168d700 1 --2- 172.21.15.17:0/1936195215 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f188c0a8c50 0x7f188c1573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:45.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.250+0000 7f189168d700 1 -- 172.21.15.17:0/1936195215 >> 172.21.15.17:0/1936195215 conn(0x7f188c060310 msgr2=0x7f188c0af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:45.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.250+0000 7f189168d700 1 -- 172.21.15.17:0/1936195215 shutdown_connections 2022-08-01T19:41:45.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.250+0000 7f189168d700 1 -- 172.21.15.17:0/1936195215 wait complete. 2022-08-01T19:41:45.264 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-08-01T19:41:45.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.374+0000 7fc75b59e700 1 Processor -- start 2022-08-01T19:41:45.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.374+0000 7fc75b59e700 1 -- start start 2022-08-01T19:41:45.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.374+0000 7fc75b59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc75c0b2340 0x7fc75c0b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:45.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.374+0000 7fc75b59e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc75c060de0 0x7fc75c060190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:45.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.374+0000 7fc75b59e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc75c0b2720 con 0x7fc75c060680 2022-08-01T19:41:45.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.374+0000 7fc75b59e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc75c061e10 con 0x7fc75c060de0 2022-08-01T19:41:45.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.374+0000 7fc75b59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc75c148330 con 0x7fc75c0b2340 2022-08-01T19:41:45.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.374+0000 7fc759d9b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc75c060de0 0x7fc75c060190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:45.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.374+0000 7fc759d9b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc75c060de0 0x7fc75c060190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42568/0 (socket says 172.21.15.17:42568) 2022-08-01T19:41:45.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.374+0000 7fc759d9b700 1 -- 172.21.15.17:0/3630023780 learned_addr learned my addr 172.21.15.17:0/3630023780 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:45.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.374+0000 7fc75a59c700 1 --2- 172.21.15.17:0/3630023780 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc75c0b2340 0x7fc75c0b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:45.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.378+0000 7fc759d9b700 1 -- 172.21.15.17:0/3630023780 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc75c0b2340 msgr2=0x7fc75c0b2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:45.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.378+0000 7fc759d9b700 1 --2- 172.21.15.17:0/3630023780 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc75c0b2340 0x7fc75c0b2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:45.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.378+0000 7fc759d9b700 1 -- 172.21.15.17:0/3630023780 >> v1:172.21.15.77:6789/0 conn(0x7fc75c060680 legacy=0x7fc75c1447c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:45.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.378+0000 7fc759d9b700 1 -- 172.21.15.17:0/3630023780 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc75c05fa50 con 0x7fc75c060de0 2022-08-01T19:41:45.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.378+0000 7fc759d9b700 1 --2- 172.21.15.17:0/3630023780 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc75c060de0 0x7fc75c060190 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 rx=0x7fc74c002f70 tx=0x7fc74c006eb0).ready entity=mon.0 client_cookie=41d8a90ee72493d4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:45.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.378+0000 7fc75959a700 1 -- 172.21.15.17:0/3630023780 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc74c00d570 con 0x7fc75c060de0 2022-08-01T19:41:45.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.378+0000 7fc75959a700 1 -- 172.21.15.17:0/3630023780 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74c000e90 con 0x7fc75c060de0 2022-08-01T19:41:45.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.378+0000 7fc75959a700 1 -- 172.21.15.17:0/3630023780 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc74c00d900 con 0x7fc75c060de0 2022-08-01T19:41:45.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.378+0000 7fc75b59e700 1 -- 172.21.15.17:0/3630023780 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc75c060de0 msgr2=0x7fc75c060190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:45.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.378+0000 7fc75b59e700 1 --2- 172.21.15.17:0/3630023780 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc75c060de0 0x7fc75c060190 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 rx=0x7fc74c002f70 tx=0x7fc74c006eb0).stop 2022-08-01T19:41:45.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.378+0000 7fc75b59e700 1 -- 172.21.15.17:0/3630023780 shutdown_connections 2022-08-01T19:41:45.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.378+0000 7fc75b59e700 1 --2- 172.21.15.17:0/3630023780 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc75c060de0 0x7fc75c060190 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:45.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.378+0000 7fc75b59e700 1 --2- 172.21.15.17:0/3630023780 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc75c0b2340 0x7fc75c0b2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:45.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.378+0000 7fc75b59e700 1 -- 172.21.15.17:0/3630023780 >> 172.21.15.17:0/3630023780 conn(0x7fc75c05c620 msgr2=0x7fc75c05ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:45.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.378+0000 7fc75b59e700 1 -- 172.21.15.17:0/3630023780 shutdown_connections 2022-08-01T19:41:45.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.378+0000 7fc75b59e700 1 -- 172.21.15.17:0/3630023780 wait complete. 2022-08-01T19:41:45.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.382+0000 7fc75b59e700 1 Processor -- start 2022-08-01T19:41:45.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.382+0000 7fc75b59e700 1 -- start start 2022-08-01T19:41:45.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.382+0000 7fc75b59e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc75c060680 0x7fc75c15ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:45.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.382+0000 7fc75b59e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc75c060de0 0x7fc75c14e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:45.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.382+0000 7fc75b59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc75c0b2340 0x7fc75c156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:45.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.382+0000 7fc75b59e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75c135660 con 0x7fc75c060de0 2022-08-01T19:41:45.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.382+0000 7fc75b59e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75c13fb40 con 0x7fc75c060680 2022-08-01T19:41:45.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.382+0000 7fc75b59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc75c15a230 con 0x7fc75c0b2340 2022-08-01T19:41:45.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.382+0000 7fc759d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc75c060de0 0x7fc75c14e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:45.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.382+0000 7fc759d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc75c060de0 0x7fc75c14e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42576/0 (socket says 172.21.15.17:42576) 2022-08-01T19:41:45.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.382+0000 7fc759d9b700 1 -- 172.21.15.17:0/3474356001 learned_addr learned my addr 172.21.15.17:0/3474356001 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:45.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.382+0000 7fc75a59c700 1 --2- 172.21.15.17:0/3474356001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc75c060680 0x7fc75c15ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:45.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.382+0000 7fc75ad9d700 1 --2- 172.21.15.17:0/3474356001 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc75c0b2340 0x7fc75c156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:45.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.382+0000 7fc759d9b700 1 -- 172.21.15.17:0/3474356001 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc75c0b2340 msgr2=0x7fc75c156bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:45.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.382+0000 7fc759d9b700 1 --2- 172.21.15.17:0/3474356001 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc75c0b2340 0x7fc75c156bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:45.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.382+0000 7fc759d9b700 1 -- 172.21.15.17:0/3474356001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc75c060680 msgr2=0x7fc75c15ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:45.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.382+0000 7fc759d9b700 1 --2- 172.21.15.17:0/3474356001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc75c060680 0x7fc75c15ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:45.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.382+0000 7fc759d9b700 1 -- 172.21.15.17:0/3474356001 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc74c00b1e0 con 0x7fc75c060de0 2022-08-01T19:41:45.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.382+0000 7fc759d9b700 1 --2- 172.21.15.17:0/3474356001 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc75c060de0 0x7fc75c14e370 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 rx=0x7fc75c1357a0 tx=0x7fc74c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:45.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.382+0000 7fc74b7fe700 1 -- 172.21.15.17:0/3474356001 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc74c011910 con 0x7fc75c060de0 2022-08-01T19:41:45.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.382+0000 7fc74b7fe700 1 -- 172.21.15.17:0/3474356001 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74c011a70 con 0x7fc75c060de0 2022-08-01T19:41:45.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.382+0000 7fc75b59e700 1 -- 172.21.15.17:0/3474356001 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc75c15a4d0 con 0x7fc75c060de0 2022-08-01T19:41:45.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.386+0000 7fc74b7fe700 1 -- 172.21.15.17:0/3474356001 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc74c011dd0 con 0x7fc75c060de0 2022-08-01T19:41:45.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.386+0000 7fc75b59e700 1 -- 172.21.15.17:0/3474356001 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc75c14e860 con 0x7fc75c060de0 2022-08-01T19:41:45.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.386+0000 7fc75b59e700 1 -- 172.21.15.17:0/3474356001 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc75c05fcf0 con 0x7fc75c060de0 2022-08-01T19:41:45.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.386+0000 7fc74b7fe700 1 -- 172.21.15.17:0/3474356001 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc74c025030 con 0x7fc75c060de0 2022-08-01T19:41:45.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.386+0000 7fc74b7fe700 1 --2- 172.21.15.17:0/3474356001 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc738053a70 0x7fc738055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:45.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.386+0000 7fc74b7fe700 1 -- 172.21.15.17:0/3474356001 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fc74c06d1a0 con 0x7fc75c060de0 2022-08-01T19:41:45.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.386+0000 7fc75a59c700 1 --2- 172.21.15.17:0/3474356001 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc738053a70 0x7fc738055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:45.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.390+0000 7fc75a59c700 1 --2- 172.21.15.17:0/3474356001 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc738053a70 0x7fc738055f30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7fc75000b000 tx=0x7fc750008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:45.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.394+0000 7fc74b7fe700 1 -- 172.21.15.17:0/3474356001 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc74c044190 con 0x7fc75c060de0 2022-08-01T19:41:45.526 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.526+0000 7fc75b59e700 1 -- 172.21.15.17:0/3474356001 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc75c0574e0 con 0x7fc75c060de0 2022-08-01T19:41:45.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.526+0000 7fc74b7fe700 1 -- 172.21.15.17:0/3474356001 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v103) v1 ==== 84+0+143 (secure 0 0 0) 0x7fc74c023090 con 0x7fc75c060de0 2022-08-01T19:41:45.528 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:45.529 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2022-08-01T19:41:45.529 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:45.530 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 1, 2022-08-01T19:41:45.530 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": "device_health_metrics" 2022-08-01T19:41:45.531 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:45.531 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:45.531 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 2, 2022-08-01T19:41:45.532 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": "rbd" 2022-08-01T19:41:45.532 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:45.532 INFO:teuthology.orchestra.run.smithi017.stdout:] 2022-08-01T19:41:45.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.526+0000 7fc75b59e700 1 -- 172.21.15.17:0/3474356001 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc738053a70 msgr2=0x7fc738055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:45.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.526+0000 7fc75b59e700 1 --2- 172.21.15.17:0/3474356001 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc738053a70 0x7fc738055f30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7fc75000b000 tx=0x7fc750008040).stop 2022-08-01T19:41:45.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.526+0000 7fc75b59e700 1 -- 172.21.15.17:0/3474356001 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc75c060de0 msgr2=0x7fc75c14e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:45.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.526+0000 7fc75b59e700 1 --2- 172.21.15.17:0/3474356001 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc75c060de0 0x7fc75c14e370 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 rx=0x7fc75c1357a0 tx=0x7fc74c015000).stop 2022-08-01T19:41:45.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.526+0000 7fc75b59e700 1 -- 172.21.15.17:0/3474356001 shutdown_connections 2022-08-01T19:41:45.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.526+0000 7fc75b59e700 1 --2- 172.21.15.17:0/3474356001 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc738053a70 0x7fc738055f30 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:45.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.526+0000 7fc75b59e700 1 --2- 172.21.15.17:0/3474356001 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc75c0b2340 0x7fc75c156bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:45.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.526+0000 7fc75b59e700 1 --2- 172.21.15.17:0/3474356001 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc75c060de0 0x7fc75c14e370 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:45.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.526+0000 7fc75b59e700 1 --2- 172.21.15.17:0/3474356001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc75c060680 0x7fc75c15ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:45.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.526+0000 7fc75b59e700 1 -- 172.21.15.17:0/3474356001 >> 172.21.15.17:0/3474356001 conn(0x7fc75c05c620 msgr2=0x7fc75c065c90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:45.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.526+0000 7fc75b59e700 1 -- 172.21.15.17:0/3474356001 shutdown_connections 2022-08-01T19:41:45.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.526+0000 7fc75b59e700 1 -- 172.21.15.17:0/3474356001 wait complete. 2022-08-01T19:41:45.547 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-08-01T19:41:45.549 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-08-01T19:41:45.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.662+0000 7f4fed26a700 1 Processor -- start 2022-08-01T19:41:45.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.662+0000 7f4fed26a700 1 -- start start 2022-08-01T19:41:45.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.666+0000 7f4fed26a700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4fe80b31d0 0x7f4fe8066020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:45.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.666+0000 7f4fed26a700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f4fe80a9da0 0x7f4fe8060950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:45.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.666+0000 7f4fed26a700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fe8067030 con 0x7f4fe81434b0 2022-08-01T19:41:45.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.666+0000 7f4fed26a700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fe8064a70 con 0x7f4fe80a9da0 2022-08-01T19:41:45.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.666+0000 7f4fed26a700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe8061f60 con 0x7f4fe80b31d0 2022-08-01T19:41:45.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.666+0000 7f4fe7fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4fe80b31d0 0x7f4fe8066020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:45.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.666+0000 7f4fdffff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f4fe80a9da0 0x7f4fe8060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:45.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.666+0000 7f4fe7fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4fe80b31d0 0x7f4fe8066020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:38682/0 (socket says 172.21.15.17:38682) 2022-08-01T19:41:45.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.666+0000 7f4fe7fff700 1 -- 172.21.15.17:0/1449817281 learned_addr learned my addr 172.21.15.17:0/1449817281 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:45.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.666+0000 7f4fe77fe700 1 -- 172.21.15.17:0/1449817281 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2132580160 0 0) 0x7f4fe8067030 con 0x7f4fe81434b0 2022-08-01T19:41:45.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.666+0000 7f4fe77fe700 1 -- 172.21.15.17:0/1449817281 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fc8003260 con 0x7f4fe81434b0 2022-08-01T19:41:45.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.666+0000 7f4fe7fff700 1 -- 172.21.15.17:0/1449817281 >> v2:172.21.15.77:3300/0 conn(0x7f4fe80a9da0 msgr2=0x7f4fe8060950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:45.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.666+0000 7f4fe7fff700 1 --2- 172.21.15.17:0/1449817281 >> v2:172.21.15.77:3300/0 conn(0x7f4fe80a9da0 0x7f4fe8060950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:45.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.666+0000 7f4fe7fff700 1 -- 172.21.15.17:0/1449817281 >> v1:172.21.15.77:6789/0 conn(0x7f4fe81434b0 legacy=0x7f4fe8143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:45.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.666+0000 7f4fe7fff700 1 -- 172.21.15.17:0/1449817281 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe80b35b0 con 0x7f4fe80b31d0 2022-08-01T19:41:45.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.666+0000 7f4fdffff700 1 --2- 172.21.15.17:0/1449817281 >> v2:172.21.15.77:3300/0 conn(0x7f4fe80a9da0 0x7f4fe8060950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:45.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.666+0000 7f4fe7fff700 1 --2- 172.21.15.17:0/1449817281 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4fe80b31d0 0x7f4fe8066020 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f4fd8002f70 tx=0x7f4fd8006eb0).ready entity=mon.2 client_cookie=b9b4151754dbeaf8 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:45.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.666+0000 7f4fe77fe700 1 -- 172.21.15.17:0/1449817281 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fd800d5c0 con 0x7f4fe80b31d0 2022-08-01T19:41:45.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.666+0000 7f4fe77fe700 1 -- 172.21.15.17:0/1449817281 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd8000e90 con 0x7f4fe80b31d0 2022-08-01T19:41:45.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.666+0000 7f4fe77fe700 1 -- 172.21.15.17:0/1449817281 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fd800d950 con 0x7f4fe80b31d0 2022-08-01T19:41:45.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.666+0000 7f4fed26a700 1 -- 172.21.15.17:0/1449817281 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4fe80b31d0 msgr2=0x7f4fe8066020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:45.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.666+0000 7f4fed26a700 1 --2- 172.21.15.17:0/1449817281 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4fe80b31d0 0x7f4fe8066020 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f4fd8002f70 tx=0x7f4fd8006eb0).stop 2022-08-01T19:41:45.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.666+0000 7f4fed26a700 1 -- 172.21.15.17:0/1449817281 shutdown_connections 2022-08-01T19:41:45.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.666+0000 7f4fed26a700 1 --2- 172.21.15.17:0/1449817281 >> v2:172.21.15.77:3300/0 conn(0x7f4fe80a9da0 0x7f4fe8060950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:45.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.666+0000 7f4fed26a700 1 --2- 172.21.15.17:0/1449817281 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4fe80b31d0 0x7f4fe8066020 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:45.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.666+0000 7f4fed26a700 1 -- 172.21.15.17:0/1449817281 >> 172.21.15.17:0/1449817281 conn(0x7f4fe8063870 msgr2=0x7f4fe8063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:45.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.666+0000 7f4fed26a700 1 -- 172.21.15.17:0/1449817281 shutdown_connections 2022-08-01T19:41:45.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.666+0000 7f4fed26a700 1 -- 172.21.15.17:0/1449817281 wait complete. 2022-08-01T19:41:45.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.670+0000 7f4fed26a700 1 Processor -- start 2022-08-01T19:41:45.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.670+0000 7f4fed26a700 1 -- start start 2022-08-01T19:41:45.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.670+0000 7f4fed26a700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4fe80a9da0 0x7f4fe80af940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:45.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.670+0000 7f4fed26a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fe80b31d0 0x7f4fe80aef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:45.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.670+0000 7f4fed26a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fe81434b0 0x7f4fe80aa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:45.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.670+0000 7f4fed26a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe80ac620 con 0x7f4fe81434b0 2022-08-01T19:41:45.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.670+0000 7f4fed26a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe80ae070 con 0x7f4fe80b31d0 2022-08-01T19:41:45.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.670+0000 7f4fed26a700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe80af3f0 con 0x7f4fe80a9da0 2022-08-01T19:41:45.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.674+0000 7f4feca69700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fe81434b0 0x7f4fe80aa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:45.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.674+0000 7f4fe7fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4fe80a9da0 0x7f4fe80af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:45.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.674+0000 7f4fdffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fe80b31d0 0x7f4fe80aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:45.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.674+0000 7f4feca69700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fe81434b0 0x7f4fe80aa5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42592/0 (socket says 172.21.15.17:42592) 2022-08-01T19:41:45.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.674+0000 7f4feca69700 1 -- 172.21.15.17:0/1003124103 learned_addr learned my addr 172.21.15.17:0/1003124103 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:45.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.674+0000 7f4fe7fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4fe80a9da0 0x7f4fe80af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:38688/0 (socket says 172.21.15.17:38688) 2022-08-01T19:41:45.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.674+0000 7f4feca69700 1 -- 172.21.15.17:0/1003124103 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4fe80a9da0 msgr2=0x7f4fe80af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:45.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.674+0000 7f4feca69700 1 --2- 172.21.15.17:0/1003124103 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4fe80a9da0 0x7f4fe80af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:45.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.674+0000 7f4feca69700 1 -- 172.21.15.17:0/1003124103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fe80b31d0 msgr2=0x7f4fe80aef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:45.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.674+0000 7f4feca69700 1 --2- 172.21.15.17:0/1003124103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fe80b31d0 0x7f4fe80aef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:45.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.674+0000 7f4feca69700 1 -- 172.21.15.17:0/1003124103 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd000d410 con 0x7f4fe81434b0 2022-08-01T19:41:45.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.674+0000 7f4fdffff700 1 --2- 172.21.15.17:0/1003124103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fe80b31d0 0x7f4fe80aef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:45.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.674+0000 7f4feca69700 1 --2- 172.21.15.17:0/1003124103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fe81434b0 0x7f4fe80aa5b0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 rx=0x7f4fe80af580 tx=0x7f4fe0007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:45.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.674+0000 7f4fe5ffb700 1 -- 172.21.15.17:0/1003124103 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fe0002310 con 0x7f4fe81434b0 2022-08-01T19:41:45.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.674+0000 7f4fe5ffb700 1 -- 172.21.15.17:0/1003124103 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe0002470 con 0x7f4fe81434b0 2022-08-01T19:41:45.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.674+0000 7f4fe5ffb700 1 -- 172.21.15.17:0/1003124103 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fe00017a0 con 0x7f4fe81434b0 2022-08-01T19:41:45.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.674+0000 7f4fed26a700 1 -- 172.21.15.17:0/1003124103 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd800b1e0 con 0x7f4fe81434b0 2022-08-01T19:41:45.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.674+0000 7f4fed26a700 1 -- 172.21.15.17:0/1003124103 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe80ae2a0 con 0x7f4fe81434b0 2022-08-01T19:41:45.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.674+0000 7f4fed26a700 1 -- 172.21.15.17:0/1003124103 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe8066af0 con 0x7f4fe81434b0 2022-08-01T19:41:45.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.674+0000 7f4fe5ffb700 1 -- 172.21.15.17:0/1003124103 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4fe0010030 con 0x7f4fe81434b0 2022-08-01T19:41:45.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.674+0000 7f4fe5ffb700 1 --2- 172.21.15.17:0/1003124103 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4fcc0539e0 0x7f4fcc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:45.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.674+0000 7f4fe5ffb700 1 -- 172.21.15.17:0/1003124103 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f4fe006ba40 con 0x7f4fe81434b0 2022-08-01T19:41:45.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.678+0000 7f4fe7fff700 1 --2- 172.21.15.17:0/1003124103 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4fcc0539e0 0x7f4fcc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:45.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.678+0000 7f4fe7fff700 1 --2- 172.21.15.17:0/1003124103 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4fcc0539e0 0x7f4fcc055ea0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f4fd801c000 tx=0x7f4fd8015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:45.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.686+0000 7f4fe5ffb700 1 -- 172.21.15.17:0/1003124103 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4fe0042aa0 con 0x7f4fe81434b0 2022-08-01T19:41:45.810 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:45.810+0000 7f4fed26a700 1 -- 172.21.15.17:0/1003124103 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f4fe80574e0 con 0x7f4fe81434b0 2022-08-01T19:41:46.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.334+0000 7f4fe5ffb700 1 -- 172.21.15.17:0/1003124103 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v104) v1 ==== 133+0+0 (secure 0 0 0) 0x7f4fe00460c0 con 0x7f4fe81434b0 2022-08-01T19:41:46.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.338+0000 7f4fed26a700 1 -- 172.21.15.17:0/1003124103 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4fcc0539e0 msgr2=0x7f4fcc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:46.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.338+0000 7f4fed26a700 1 --2- 172.21.15.17:0/1003124103 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4fcc0539e0 0x7f4fcc055ea0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f4fd801c000 tx=0x7f4fd8015000).stop 2022-08-01T19:41:46.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.338+0000 7f4fed26a700 1 -- 172.21.15.17:0/1003124103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fe81434b0 msgr2=0x7f4fe80aa5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:46.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.338+0000 7f4fed26a700 1 --2- 172.21.15.17:0/1003124103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fe81434b0 0x7f4fe80aa5b0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 rx=0x7f4fe80af580 tx=0x7f4fe0007dd0).stop 2022-08-01T19:41:46.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.338+0000 7f4fed26a700 1 -- 172.21.15.17:0/1003124103 shutdown_connections 2022-08-01T19:41:46.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.338+0000 7f4fed26a700 1 --2- 172.21.15.17:0/1003124103 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4fcc0539e0 0x7f4fcc055ea0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:46.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.338+0000 7f4fed26a700 1 --2- 172.21.15.17:0/1003124103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fe81434b0 0x7f4fe80aa5b0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:46.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.338+0000 7f4fed26a700 1 --2- 172.21.15.17:0/1003124103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fe80b31d0 0x7f4fe80aef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:46.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.338+0000 7f4fed26a700 1 --2- 172.21.15.17:0/1003124103 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4fe80a9da0 0x7f4fe80af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:46.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.338+0000 7f4fed26a700 1 -- 172.21.15.17:0/1003124103 >> 172.21.15.17:0/1003124103 conn(0x7f4fe8063870 msgr2=0x7f4fe80b62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:46.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.338+0000 7f4fed26a700 1 -- 172.21.15.17:0/1003124103 shutdown_connections 2022-08-01T19:41:46.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.338+0000 7f4fed26a700 1 -- 172.21.15.17:0/1003124103 wait complete. 2022-08-01T19:41:46.343 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' created 2022-08-01T19:41:46.359 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-08-01T19:41:46.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.474+0000 7f1fdcf75700 1 Processor -- start 2022-08-01T19:41:46.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.474+0000 7f1fdcf75700 1 -- start start 2022-08-01T19:41:46.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.474+0000 7f1fdcf75700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f1fd80b2050 0x7f1fd80b1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:46.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.474+0000 7f1fdcf75700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fd80b05b0 con 0x7f1fd80b2430 2022-08-01T19:41:46.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.474+0000 7f1fdcf75700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fd80a7960 con 0x7f1fd80b0140 2022-08-01T19:41:46.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.474+0000 7f1fdcf75700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fd80b07e0 con 0x7f1fd80b2050 2022-08-01T19:41:46.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.474+0000 7f1fd6d9d700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1fd80b2430 0x7f1fd81413e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34794/0 (socket says 172.21.15.17:34794) 2022-08-01T19:41:46.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.474+0000 7f1fd6d9d700 1 -- 172.21.15.17:0/3096621664 learned_addr learned my addr 172.21.15.17:0/3096621664 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:46.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.474+0000 7f1fd659c700 1 --2- 172.21.15.17:0/3096621664 >> v2:172.21.15.77:3300/0 conn(0x7f1fd80b2050 0x7f1fd80b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:46.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.474+0000 7f1fd559a700 1 -- 172.21.15.17:0/3096621664 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 811905101 0 0) 0x7f1fd80a7960 con 0x7f1fd80b0140 2022-08-01T19:41:46.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.474+0000 7f1fd559a700 1 -- 172.21.15.17:0/3096621664 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fbc003260 con 0x7f1fd80b0140 2022-08-01T19:41:46.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.474+0000 7f1fd559a700 1 -- 172.21.15.17:0/3096621664 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2687827374 0 0) 0x7f1fd80b05b0 con 0x7f1fd80b2430 2022-08-01T19:41:46.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.474+0000 7f1fd559a700 1 -- 172.21.15.17:0/3096621664 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fd80a7960 con 0x7f1fd80b2430 2022-08-01T19:41:46.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.474+0000 7f1fd659c700 1 -- 172.21.15.17:0/3096621664 >> v1:172.21.15.77:6789/0 conn(0x7f1fd80b0140 legacy=0x7f1fd80a7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:46.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.474+0000 7f1fd659c700 1 -- 172.21.15.17:0/3096621664 >> v1:172.21.15.17:6789/0 conn(0x7f1fd80b2430 legacy=0x7f1fd81413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:46.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.474+0000 7f1fd659c700 1 -- 172.21.15.17:0/3096621664 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd80b0a80 con 0x7f1fd80b2050 2022-08-01T19:41:46.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.474+0000 7f1fd659c700 1 --2- 172.21.15.17:0/3096621664 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1fd80b2050 0x7f1fd80b1460 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f1fcc002f70 tx=0x7f1fcc006df0).ready entity=mon.1 client_cookie=4548d7c24d77f1be server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:46.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.474+0000 7f1fd559a700 1 -- 172.21.15.17:0/3096621664 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1fcc00d540 con 0x7f1fd80b2050 2022-08-01T19:41:46.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.474+0000 7f1fd559a700 1 -- 172.21.15.17:0/3096621664 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fcc000e90 con 0x7f1fd80b2050 2022-08-01T19:41:46.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.474+0000 7f1fd559a700 1 -- 172.21.15.17:0/3096621664 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1fcc00d890 con 0x7f1fd80b2050 2022-08-01T19:41:46.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.478+0000 7f1fdcf75700 1 -- 172.21.15.17:0/3096621664 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1fd80b2050 msgr2=0x7f1fd80b1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:46.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.478+0000 7f1fdcf75700 1 --2- 172.21.15.17:0/3096621664 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1fd80b2050 0x7f1fd80b1460 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f1fcc002f70 tx=0x7f1fcc006df0).stop 2022-08-01T19:41:46.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.478+0000 7f1fdcf75700 1 -- 172.21.15.17:0/3096621664 shutdown_connections 2022-08-01T19:41:46.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.478+0000 7f1fdcf75700 1 --2- 172.21.15.17:0/3096621664 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1fd80b2050 0x7f1fd80b1460 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:46.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.478+0000 7f1fdcf75700 1 -- 172.21.15.17:0/3096621664 >> 172.21.15.17:0/3096621664 conn(0x7f1fd805f840 msgr2=0x7f1fd805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:46.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.478+0000 7f1fdcf75700 1 -- 172.21.15.17:0/3096621664 shutdown_connections 2022-08-01T19:41:46.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.478+0000 7f1fdcf75700 1 -- 172.21.15.17:0/3096621664 wait complete. 2022-08-01T19:41:46.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.478+0000 7f1fdcf75700 1 Processor -- start 2022-08-01T19:41:46.502 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.478+0000 7f1fdcf75700 1 -- start start 2022-08-01T19:41:46.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.482+0000 7f1fdcf75700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1fd80b0140 0x7f1fd80b5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:46.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.482+0000 7f1fdcf75700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1fd80b2050 0x7f1fd815a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:46.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.482+0000 7f1fdcf75700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fd80b2430 0x7f1fd815aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:46.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.482+0000 7f1fdcf75700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd80b4bf0 con 0x7f1fd80b2430 2022-08-01T19:41:46.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.482+0000 7f1fdcf75700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd80b4ee0 con 0x7f1fd80b2050 2022-08-01T19:41:46.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.482+0000 7f1fdcf75700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd80b4750 con 0x7f1fd80b0140 2022-08-01T19:41:46.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.482+0000 7f1fd6d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fd80b2430 0x7f1fd815aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:46.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.482+0000 7f1fd659c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1fd80b0140 0x7f1fd80b5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:46.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.482+0000 7f1fd659c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1fd80b0140 0x7f1fd80b5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:38702/0 (socket says 172.21.15.17:38702) 2022-08-01T19:41:46.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.482+0000 7f1fd659c700 1 -- 172.21.15.17:0/1308264209 learned_addr learned my addr 172.21.15.17:0/1308264209 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:46.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.482+0000 7f1fd5d9b700 1 --2- 172.21.15.17:0/1308264209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1fd80b2050 0x7f1fd815a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:46.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.482+0000 7f1fd6d9d700 1 -- 172.21.15.17:0/1308264209 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1fd80b0140 msgr2=0x7f1fd80b5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:46.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.482+0000 7f1fd6d9d700 1 --2- 172.21.15.17:0/1308264209 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1fd80b0140 0x7f1fd80b5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:46.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.482+0000 7f1fd6d9d700 1 -- 172.21.15.17:0/1308264209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1fd80b2050 msgr2=0x7f1fd815a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:46.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.482+0000 7f1fd6d9d700 1 --2- 172.21.15.17:0/1308264209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1fd80b2050 0x7f1fd815a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:46.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.482+0000 7f1fd6d9d700 1 -- 172.21.15.17:0/1308264209 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fcc00b180 con 0x7f1fd80b2430 2022-08-01T19:41:46.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.482+0000 7f1fd6d9d700 1 --2- 172.21.15.17:0/1308264209 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fd80b2430 0x7f1fd815aad0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0x7f1fd0001490 tx=0x7f1fd0007d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:46.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.482+0000 7f1fc77fe700 1 -- 172.21.15.17:0/1308264209 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1fd000ce70 con 0x7f1fd80b2430 2022-08-01T19:41:46.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.482+0000 7f1fc77fe700 1 -- 172.21.15.17:0/1308264209 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd0005ba0 con 0x7f1fd80b2430 2022-08-01T19:41:46.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.482+0000 7f1fdcf75700 1 -- 172.21.15.17:0/1308264209 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fd80b40c0 con 0x7f1fd80b2430 2022-08-01T19:41:46.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.482+0000 7f1fc77fe700 1 -- 172.21.15.17:0/1308264209 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1fd00070d0 con 0x7f1fd80b2430 2022-08-01T19:41:46.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.482+0000 7f1fdcf75700 1 -- 172.21.15.17:0/1308264209 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd80ac350 con 0x7f1fd80b2430 2022-08-01T19:41:46.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.486+0000 7f1fdcf75700 1 -- 172.21.15.17:0/1308264209 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fd80b4910 con 0x7f1fd80b2430 2022-08-01T19:41:46.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.486+0000 7f1fc77fe700 1 -- 172.21.15.17:0/1308264209 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1fd001a060 con 0x7f1fd80b2430 2022-08-01T19:41:46.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.486+0000 7f1fc77fe700 1 --2- 172.21.15.17:0/1308264209 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1fb4053b30 0x7f1fb4055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:46.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.486+0000 7f1fc77fe700 1 -- 172.21.15.17:0/1308264209 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f1fd006b850 con 0x7f1fd80b2430 2022-08-01T19:41:46.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.486+0000 7f1fd659c700 1 --2- 172.21.15.17:0/1308264209 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1fb4053b30 0x7f1fb4055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:46.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.486+0000 7f1fd659c700 1 --2- 172.21.15.17:0/1308264209 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1fb4053b30 0x7f1fb4055ff0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f1fcc018000 tx=0x7f1fcc014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:46.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.490+0000 7f1fc77fe700 1 -- 172.21.15.17:0/1308264209 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1fd00426d0 con 0x7f1fd80b2430 2022-08-01T19:41:46.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:46.618+0000 7f1fdcf75700 1 -- 172.21.15.17:0/1308264209 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f1fd815b160 con 0x7f1fd80b2430 2022-08-01T19:41:47.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.094+0000 7f1fc77fe700 1 -- 172.21.15.17:0/1308264209 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v105) v1 ==== 125+0+0 (secure 0 0 0) 0x7f1fd0018070 con 0x7f1fd80b2430 2022-08-01T19:41:47.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.102+0000 7f1fdcf75700 1 -- 172.21.15.17:0/1308264209 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1fb4053b30 msgr2=0x7f1fb4055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.102+0000 7f1fdcf75700 1 --2- 172.21.15.17:0/1308264209 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1fb4053b30 0x7f1fb4055ff0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f1fcc018000 tx=0x7f1fcc014000).stop 2022-08-01T19:41:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.102+0000 7f1fdcf75700 1 -- 172.21.15.17:0/1308264209 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fd80b2430 msgr2=0x7f1fd815aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.102+0000 7f1fdcf75700 1 --2- 172.21.15.17:0/1308264209 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fd80b2430 0x7f1fd815aad0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0x7f1fd0001490 tx=0x7f1fd0007d40).stop 2022-08-01T19:41:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.102+0000 7f1fdcf75700 1 -- 172.21.15.17:0/1308264209 shutdown_connections 2022-08-01T19:41:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.102+0000 7f1fdcf75700 1 --2- 172.21.15.17:0/1308264209 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1fb4053b30 0x7f1fb4055ff0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.102+0000 7f1fdcf75700 1 --2- 172.21.15.17:0/1308264209 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fd80b2430 0x7f1fd815aad0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.102+0000 7f1fdcf75700 1 --2- 172.21.15.17:0/1308264209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1fd80b2050 0x7f1fd815a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.102+0000 7f1fdcf75700 1 --2- 172.21.15.17:0/1308264209 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1fd80b0140 0x7f1fd80b5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:47.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.102+0000 7f1fdcf75700 1 -- 172.21.15.17:0/1308264209 >> 172.21.15.17:0/1308264209 conn(0x7f1fd805f840 msgr2=0x7f1fd80af6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:47.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.102+0000 7f1fdcf75700 1 -- 172.21.15.17:0/1308264209 shutdown_connections 2022-08-01T19:41:47.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.102+0000 7f1fdcf75700 1 -- 172.21.15.17:0/1308264209 wait complete. 2022-08-01T19:41:47.108 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' created 2022-08-01T19:41:47.119 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-08-01T19:41:47.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.230+0000 7f2c7341d700 1 Processor -- start 2022-08-01T19:41:47.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.230+0000 7f2c7341d700 1 -- start start 2022-08-01T19:41:47.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.234+0000 7f2c7341d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2c6c0ad750 0x7f2c6c061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:47.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.234+0000 7f2c7341d700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f2c6c1405d0 0x7f2c6c140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:47.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.234+0000 7f2c7341d700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c6c0aee80 con 0x7f2c6c0614b0 2022-08-01T19:41:47.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.234+0000 7f2c7341d700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c6c0b70b0 con 0x7f2c6c1405d0 2022-08-01T19:41:47.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.234+0000 7f2c7341d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c0af0b0 con 0x7f2c6c0ad750 2022-08-01T19:41:47.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.234+0000 7f2c71c1a700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2c6c0ad750 0x7f2c6c061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:47.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.234+0000 7f2c72c1c700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f2c6c1405d0 0x7f2c6c140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:47.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.234+0000 7f2c72c1c700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f2c6c1405d0 0x7f2c6c140a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:40536/0 (socket says 172.21.15.17:40536) 2022-08-01T19:41:47.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.234+0000 7f2c71c1a700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2c6c0ad750 0x7f2c6c061890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:38712/0 (socket says 172.21.15.17:38712) 2022-08-01T19:41:47.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.234+0000 7f2c71c1a700 1 -- 172.21.15.17:0/725598169 learned_addr learned my addr 172.21.15.17:0/725598169 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:47.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.234+0000 7f2c72c1c700 1 -- 172.21.15.17:0/725598169 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2c6c0ad750 msgr2=0x7f2c6c061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:47.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.234+0000 7f2c72c1c700 1 --2- 172.21.15.17:0/725598169 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2c6c0ad750 0x7f2c6c061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:47.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.234+0000 7f2c72c1c700 1 -- 172.21.15.17:0/725598169 >> v1:172.21.15.77:6789/0 conn(0x7f2c6c0614b0 legacy=0x7f2c6c0b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:47.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.234+0000 7f2c72c1c700 1 -- 172.21.15.17:0/725598169 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c6c0af350 con 0x7f2c6c1405d0 2022-08-01T19:41:47.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.234+0000 7f2c72c1c700 1 --2- 172.21.15.17:0/725598169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c6c1405d0 0x7f2c6c140a80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f2c640032f0 tx=0x7f2c640091d0).ready entity=mon.1 client_cookie=3bd7008cd16f1070 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:47.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.234+0000 7f2c71419700 1 -- 172.21.15.17:0/725598169 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c6400b6c0 con 0x7f2c6c1405d0 2022-08-01T19:41:47.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.234+0000 7f2c71419700 1 -- 172.21.15.17:0/725598169 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c6400b820 con 0x7f2c6c1405d0 2022-08-01T19:41:47.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.234+0000 7f2c71419700 1 -- 172.21.15.17:0/725598169 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c6400b990 con 0x7f2c6c1405d0 2022-08-01T19:41:47.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.234+0000 7f2c7341d700 1 -- 172.21.15.17:0/725598169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c6c1405d0 msgr2=0x7f2c6c140a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:47.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.234+0000 7f2c7341d700 1 --2- 172.21.15.17:0/725598169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c6c1405d0 0x7f2c6c140a80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f2c640032f0 tx=0x7f2c640091d0).stop 2022-08-01T19:41:47.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.234+0000 7f2c7341d700 1 -- 172.21.15.17:0/725598169 shutdown_connections 2022-08-01T19:41:47.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.234+0000 7f2c7341d700 1 --2- 172.21.15.17:0/725598169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c6c1405d0 0x7f2c6c140a80 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:47.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.234+0000 7f2c7341d700 1 --2- 172.21.15.17:0/725598169 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2c6c0ad750 0x7f2c6c061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:47.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.234+0000 7f2c7341d700 1 -- 172.21.15.17:0/725598169 >> 172.21.15.17:0/725598169 conn(0x7f2c6c066f70 msgr2=0x7f2c6c067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:47.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.234+0000 7f2c7341d700 1 -- 172.21.15.17:0/725598169 shutdown_connections 2022-08-01T19:41:47.246 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.234+0000 7f2c7341d700 1 -- 172.21.15.17:0/725598169 wait complete. 2022-08-01T19:41:47.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.238+0000 7f2c7341d700 1 Processor -- start 2022-08-01T19:41:47.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.238+0000 7f2c7341d700 1 -- start start 2022-08-01T19:41:47.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.238+0000 7f2c7341d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c6c0614b0 0x7f2c6c0a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:47.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.238+0000 7f2c7341d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2c6c0ad750 0x7f2c6c143a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:47.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.238+0000 7f2c7341d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c6c1405d0 0x7f2c6c1524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:47.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.238+0000 7f2c7341d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c0ad0d0 con 0x7f2c6c1405d0 2022-08-01T19:41:47.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.238+0000 7f2c7341d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c1565e0 con 0x7f2c6c0614b0 2022-08-01T19:41:47.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.238+0000 7f2c7341d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c156880 con 0x7f2c6c0ad750 2022-08-01T19:41:47.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.238+0000 7f2c72c1c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c6c1405d0 0x7f2c6c1524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:47.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.238+0000 7f2c72c1c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c6c1405d0 0x7f2c6c1524f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42620/0 (socket says 172.21.15.17:42620) 2022-08-01T19:41:47.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.238+0000 7f2c72c1c700 1 -- 172.21.15.17:0/93381611 learned_addr learned my addr 172.21.15.17:0/93381611 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:47.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.238+0000 7f2c7241b700 1 --2- 172.21.15.17:0/93381611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c6c0614b0 0x7f2c6c0a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:47.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.238+0000 7f2c71c1a700 1 --2- 172.21.15.17:0/93381611 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2c6c0ad750 0x7f2c6c143a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:47.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.238+0000 7f2c72c1c700 1 -- 172.21.15.17:0/93381611 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2c6c0ad750 msgr2=0x7f2c6c143a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:47.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.238+0000 7f2c72c1c700 1 --2- 172.21.15.17:0/93381611 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2c6c0ad750 0x7f2c6c143a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:47.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.238+0000 7f2c72c1c700 1 -- 172.21.15.17:0/93381611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c6c0614b0 msgr2=0x7f2c6c0a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:47.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.238+0000 7f2c72c1c700 1 --2- 172.21.15.17:0/93381611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c6c0614b0 0x7f2c6c0a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:47.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.238+0000 7f2c72c1c700 1 -- 172.21.15.17:0/93381611 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c64009470 con 0x7f2c6c1405d0 2022-08-01T19:41:47.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.238+0000 7f2c72c1c700 1 --2- 172.21.15.17:0/93381611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c6c1405d0 0x7f2c6c1524f0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 rx=0x7f2c640032f0 tx=0x7f2c64016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:47.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.242+0000 7f2c5f7fe700 1 -- 172.21.15.17:0/93381611 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c64011e20 con 0x7f2c6c1405d0 2022-08-01T19:41:47.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.242+0000 7f2c5f7fe700 1 -- 172.21.15.17:0/93381611 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c64021440 con 0x7f2c6c1405d0 2022-08-01T19:41:47.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.242+0000 7f2c7341d700 1 -- 172.21.15.17:0/93381611 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c6c0b2ed0 con 0x7f2c6c1405d0 2022-08-01T19:41:47.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.242+0000 7f2c5f7fe700 1 -- 172.21.15.17:0/93381611 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c640217a0 con 0x7f2c6c1405d0 2022-08-01T19:41:47.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.242+0000 7f2c7341d700 1 -- 172.21.15.17:0/93381611 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c6c0b37e0 con 0x7f2c6c1405d0 2022-08-01T19:41:47.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.242+0000 7f2c7341d700 1 -- 172.21.15.17:0/93381611 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c6c0af5f0 con 0x7f2c6c1405d0 2022-08-01T19:41:47.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.242+0000 7f2c5f7fe700 1 -- 172.21.15.17:0/93381611 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2c64027020 con 0x7f2c6c1405d0 2022-08-01T19:41:47.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.242+0000 7f2c5f7fe700 1 --2- 172.21.15.17:0/93381611 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2c4c053950 0x7f2c4c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:47.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.242+0000 7f2c5f7fe700 1 -- 172.21.15.17:0/93381611 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f2c6406d3e0 con 0x7f2c6c1405d0 2022-08-01T19:41:47.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.242+0000 7f2c7241b700 1 --2- 172.21.15.17:0/93381611 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2c4c053950 0x7f2c4c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:47.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.246+0000 7f2c7241b700 1 --2- 172.21.15.17:0/93381611 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2c4c053950 0x7f2c4c055e10 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7f2c6000d000 tx=0x7f2c60007b90).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:47.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.250+0000 7f2c5f7fe700 1 -- 172.21.15.17:0/93381611 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2c64044220 con 0x7f2c6c1405d0 2022-08-01T19:41:47.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.418+0000 7f2c7341d700 1 -- 172.21.15.17:0/93381611 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f2c6c0574e0 con 0x7f2c6c1405d0 2022-08-01T19:41:47.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.430+0000 7f2c5f7fe700 1 -- 172.21.15.17:0/93381611 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 19 and data pool 20 v52) v1 ==== 189+0+0 (secure 0 0 0) 0x7f2c640150d0 con 0x7f2c6c1405d0 2022-08-01T19:41:47.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.434+0000 7f2c7341d700 1 -- 172.21.15.17:0/93381611 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2c4c053950 msgr2=0x7f2c4c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:47.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.434+0000 7f2c7341d700 1 --2- 172.21.15.17:0/93381611 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2c4c053950 0x7f2c4c055e10 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7f2c6000d000 tx=0x7f2c60007b90).stop 2022-08-01T19:41:47.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.434+0000 7f2c7341d700 1 -- 172.21.15.17:0/93381611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c6c1405d0 msgr2=0x7f2c6c1524f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:47.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.434+0000 7f2c7341d700 1 --2- 172.21.15.17:0/93381611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c6c1405d0 0x7f2c6c1524f0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 rx=0x7f2c640032f0 tx=0x7f2c64016000).stop 2022-08-01T19:41:47.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.438+0000 7f2c7341d700 1 -- 172.21.15.17:0/93381611 shutdown_connections 2022-08-01T19:41:47.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.438+0000 7f2c7341d700 1 --2- 172.21.15.17:0/93381611 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2c4c053950 0x7f2c4c055e10 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:47.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.438+0000 7f2c7341d700 1 --2- 172.21.15.17:0/93381611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c6c1405d0 0x7f2c6c1524f0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:47.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.438+0000 7f2c7341d700 1 --2- 172.21.15.17:0/93381611 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2c6c0ad750 0x7f2c6c143a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:47.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.438+0000 7f2c7341d700 1 --2- 172.21.15.17:0/93381611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c6c0614b0 0x7f2c6c0a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:47.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.438+0000 7f2c7341d700 1 -- 172.21.15.17:0/93381611 >> 172.21.15.17:0/93381611 conn(0x7f2c6c066f70 msgr2=0x7f2c6c0b7960 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:47.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.438+0000 7f2c7341d700 1 -- 172.21.15.17:0/93381611 shutdown_connections 2022-08-01T19:41:47.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.438+0000 7f2c7341d700 1 -- 172.21.15.17:0/93381611 wait complete. 2022-08-01T19:41:47.443 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 19 and data pool 20 2022-08-01T19:41:47.454 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:41:47.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.550+0000 7fb12447b700 1 Processor -- start 2022-08-01T19:41:47.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.550+0000 7fb12447b700 1 -- start start 2022-08-01T19:41:47.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.554+0000 7fb12447b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb11c060aa0 0x7fb11c0acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:47.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.554+0000 7fb12447b700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fb11c0aa760 0x7fb11c13c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:47.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.554+0000 7fb12447b700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb11c0b6e70 con 0x7fb11c0b6a90 2022-08-01T19:41:47.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.554+0000 7fb12447b700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb11c061370 con 0x7fb11c060aa0 2022-08-01T19:41:47.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.554+0000 7fb12447b700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb11c061610 con 0x7fb11c0aa760 2022-08-01T19:41:47.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.554+0000 7fb121a16700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb11c060aa0 0x7fb11c0acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:47.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.554+0000 7fb121a16700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb11c060aa0 0x7fb11c0acee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42640/0 (socket says 172.21.15.17:42640) 2022-08-01T19:41:47.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.554+0000 7fb121a16700 1 -- 172.21.15.17:0/1493873128 learned_addr learned my addr 172.21.15.17:0/1493873128 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:47.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.554+0000 7fb122a18700 1 --2- 172.21.15.17:0/1493873128 >> v2:172.21.15.77:3300/0 conn(0x7fb11c0aa760 0x7fb11c13c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:47.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.554+0000 7fb121a16700 1 -- 172.21.15.17:0/1493873128 >> v2:172.21.15.77:3300/0 conn(0x7fb11c0aa760 msgr2=0x7fb11c13c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:47.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.554+0000 7fb121a16700 1 --2- 172.21.15.17:0/1493873128 >> v2:172.21.15.77:3300/0 conn(0x7fb11c0aa760 0x7fb11c13c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:47.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.554+0000 7fb121a16700 1 -- 172.21.15.17:0/1493873128 >> v1:172.21.15.77:6789/0 conn(0x7fb11c0b6a90 legacy=0x7fb11c0aa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:47.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.554+0000 7fb121a16700 1 -- 172.21.15.17:0/1493873128 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb11c0618b0 con 0x7fb11c060aa0 2022-08-01T19:41:47.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.554+0000 7fb121a16700 1 --2- 172.21.15.17:0/1493873128 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb11c060aa0 0x7fb11c0acee0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 rx=0x7fb10c002f70 tx=0x7fb10c006f40).ready entity=mon.0 client_cookie=967e914244ed6aa3 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:47.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.554+0000 7fb121215700 1 -- 172.21.15.17:0/1493873128 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb10c00c880 con 0x7fb11c060aa0 2022-08-01T19:41:47.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.554+0000 7fb121215700 1 -- 172.21.15.17:0/1493873128 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10c000e90 con 0x7fb11c060aa0 2022-08-01T19:41:47.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.554+0000 7fb121215700 1 -- 172.21.15.17:0/1493873128 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb10c00cc10 con 0x7fb11c060aa0 2022-08-01T19:41:47.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.554+0000 7fb12447b700 1 -- 172.21.15.17:0/1493873128 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb11c060aa0 msgr2=0x7fb11c0acee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:47.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.554+0000 7fb12447b700 1 --2- 172.21.15.17:0/1493873128 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb11c060aa0 0x7fb11c0acee0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 rx=0x7fb10c002f70 tx=0x7fb10c006f40).stop 2022-08-01T19:41:47.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.554+0000 7fb12447b700 1 -- 172.21.15.17:0/1493873128 shutdown_connections 2022-08-01T19:41:47.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.554+0000 7fb12447b700 1 --2- 172.21.15.17:0/1493873128 >> v2:172.21.15.77:3300/0 conn(0x7fb11c0aa760 0x7fb11c13c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:47.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.554+0000 7fb12447b700 1 --2- 172.21.15.17:0/1493873128 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb11c060aa0 0x7fb11c0acee0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:47.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.554+0000 7fb12447b700 1 -- 172.21.15.17:0/1493873128 >> 172.21.15.17:0/1493873128 conn(0x7fb11c0b3610 msgr2=0x7fb11c0b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:47.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.554+0000 7fb12447b700 1 -- 172.21.15.17:0/1493873128 shutdown_connections 2022-08-01T19:41:47.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.554+0000 7fb12447b700 1 -- 172.21.15.17:0/1493873128 wait complete. 2022-08-01T19:41:47.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.558+0000 7fb12447b700 1 Processor -- start 2022-08-01T19:41:47.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.558+0000 7fb12447b700 1 -- start start 2022-08-01T19:41:47.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.558+0000 7fb12447b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb11c060aa0 0x7fb11c151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:47.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.558+0000 7fb12447b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb11c0aa760 0x7fb11c063ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:47.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.558+0000 7fb12447b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb11c0b6a90 0x7fb11c0ace90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:47.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.558+0000 7fb12447b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c0643d0 con 0x7fb11c0aa760 2022-08-01T19:41:47.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.558+0000 7fb12447b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c0afb40 con 0x7fb11c060aa0 2022-08-01T19:41:47.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.558+0000 7fb12447b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c152970 con 0x7fb11c0b6a90 2022-08-01T19:41:47.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.558+0000 7fb121a16700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb11c0aa760 0x7fb11c063ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:47.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.558+0000 7fb121a16700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb11c0aa760 0x7fb11c063ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42646/0 (socket says 172.21.15.17:42646) 2022-08-01T19:41:47.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.558+0000 7fb121a16700 1 -- 172.21.15.17:0/4153504287 learned_addr learned my addr 172.21.15.17:0/4153504287 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:47.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.558+0000 7fb122217700 1 --2- 172.21.15.17:0/4153504287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb11c060aa0 0x7fb11c151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:47.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.558+0000 7fb122a18700 1 --2- 172.21.15.17:0/4153504287 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb11c0b6a90 0x7fb11c0ace90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:47.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.558+0000 7fb122217700 1 -- 172.21.15.17:0/4153504287 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb11c0b6a90 msgr2=0x7fb11c0ace90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:47.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.558+0000 7fb122217700 1 --2- 172.21.15.17:0/4153504287 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb11c0b6a90 0x7fb11c0ace90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:47.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.558+0000 7fb122217700 1 -- 172.21.15.17:0/4153504287 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb11c0aa760 msgr2=0x7fb11c063ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:47.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.558+0000 7fb122217700 1 --2- 172.21.15.17:0/4153504287 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb11c0aa760 0x7fb11c063ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:47.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.558+0000 7fb122217700 1 -- 172.21.15.17:0/4153504287 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10c00a360 con 0x7fb11c060aa0 2022-08-01T19:41:47.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.558+0000 7fb121a16700 1 --2- 172.21.15.17:0/4153504287 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb11c0aa760 0x7fb11c063ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:47.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.558+0000 7fb122217700 1 --2- 172.21.15.17:0/4153504287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb11c060aa0 0x7fb11c151d80 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7fb11c060490 tx=0x7fb118007b50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:47.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.558+0000 7fb10b7fe700 1 -- 172.21.15.17:0/4153504287 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb118007030 con 0x7fb11c060aa0 2022-08-01T19:41:47.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.558+0000 7fb10b7fe700 1 -- 172.21.15.17:0/4153504287 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb118007190 con 0x7fb11c060aa0 2022-08-01T19:41:47.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.558+0000 7fb12447b700 1 -- 172.21.15.17:0/4153504287 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb11c152430 con 0x7fb11c060aa0 2022-08-01T19:41:47.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.558+0000 7fb10b7fe700 1 -- 172.21.15.17:0/4153504287 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb118006510 con 0x7fb11c060aa0 2022-08-01T19:41:47.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.558+0000 7fb12447b700 1 -- 172.21.15.17:0/4153504287 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb11c0af7c0 con 0x7fb11c060aa0 2022-08-01T19:41:47.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.562+0000 7fb12447b700 1 -- 172.21.15.17:0/4153504287 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb11c063be0 con 0x7fb11c060aa0 2022-08-01T19:41:47.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.562+0000 7fb10b7fe700 1 -- 172.21.15.17:0/4153504287 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb118019030 con 0x7fb11c060aa0 2022-08-01T19:41:47.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.562+0000 7fb10b7fe700 1 --2- 172.21.15.17:0/4153504287 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb0fc0539e0 0x7fb0fc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:47.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.562+0000 7fb10b7fe700 1 -- 172.21.15.17:0/4153504287 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb11806d8a0 con 0x7fb11c060aa0 2022-08-01T19:41:47.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.562+0000 7fb121a16700 1 --2- 172.21.15.17:0/4153504287 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb0fc0539e0 0x7fb0fc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:47.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.562+0000 7fb121a16700 1 --2- 172.21.15.17:0/4153504287 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb0fc0539e0 0x7fb0fc055ea0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7fb10c01d000 tx=0x7fb11c05ff40).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:47.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.570+0000 7fb10b7fe700 1 -- 172.21.15.17:0/4153504287 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb1180445e0 con 0x7fb11c060aa0 2022-08-01T19:41:47.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.698+0000 7fb12447b700 1 -- 172.21.15.17:0/4153504287 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb11c061b50 con 0x7fb11c060aa0 2022-08-01T19:41:47.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.702+0000 7fb10b7fe700 1 -- 172.21.15.17:0/4153504287 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16285 (secure 0 0 0) 0x7fb118047c00 con 0x7fb11c060aa0 2022-08-01T19:41:47.704 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:47.705 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":107,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:41:47.421949+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:41:45.865183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2022-08-01T19:41:46.670831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:41:47.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.702+0000 7fb12447b700 1 -- 172.21.15.17:0/4153504287 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb0fc0539e0 msgr2=0x7fb0fc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:47.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.702+0000 7fb12447b700 1 --2- 172.21.15.17:0/4153504287 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb0fc0539e0 0x7fb0fc055ea0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7fb10c01d000 tx=0x7fb11c05ff40).stop 2022-08-01T19:41:47.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.702+0000 7fb12447b700 1 -- 172.21.15.17:0/4153504287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb11c060aa0 msgr2=0x7fb11c151d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:47.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.702+0000 7fb12447b700 1 --2- 172.21.15.17:0/4153504287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb11c060aa0 0x7fb11c151d80 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7fb11c060490 tx=0x7fb118007b50).stop 2022-08-01T19:41:47.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.702+0000 7fb12447b700 1 -- 172.21.15.17:0/4153504287 shutdown_connections 2022-08-01T19:41:47.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.702+0000 7fb12447b700 1 --2- 172.21.15.17:0/4153504287 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb0fc0539e0 0x7fb0fc055ea0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:47.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.702+0000 7fb12447b700 1 --2- 172.21.15.17:0/4153504287 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb11c0b6a90 0x7fb11c0ace90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:47.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.702+0000 7fb12447b700 1 --2- 172.21.15.17:0/4153504287 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb11c0aa760 0x7fb11c063ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:47.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.702+0000 7fb12447b700 1 --2- 172.21.15.17:0/4153504287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb11c060aa0 0x7fb11c151d80 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:47.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.702+0000 7fb12447b700 1 -- 172.21.15.17:0/4153504287 >> 172.21.15.17:0/4153504287 conn(0x7fb11c0b3610 msgr2=0x7fb11c05c580 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:47.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.702+0000 7fb12447b700 1 -- 172.21.15.17:0/4153504287 shutdown_connections 2022-08-01T19:41:47.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.702+0000 7fb12447b700 1 -- 172.21.15.17:0/4153504287 wait complete. 2022-08-01T19:41:47.722 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:41:47.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.826+0000 7f678bfff700 1 Processor -- start 2022-08-01T19:41:47.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.826+0000 7f678bfff700 1 -- start start 2022-08-01T19:41:47.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.826+0000 7f678bfff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f678c0b1ea0 0x7f678c147600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:47.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.826+0000 7f678bfff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f678c05c620 0x7f678c0a8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:47.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.826+0000 7f678bfff700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f678c14ba70 con 0x7f678c0b2930 2022-08-01T19:41:47.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.826+0000 7f678bfff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f678c145340 con 0x7f678c05c620 2022-08-01T19:41:47.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.826+0000 7f678bfff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678c0b2d10 con 0x7f678c0b1ea0 2022-08-01T19:41:47.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.826+0000 7f678a7fc700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f678c0b1ea0 0x7f678c147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:47.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.826+0000 7f678affd700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7f678c0b2930 0x7f678c14b360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:51646/0 (socket says 172.21.15.17:51646) 2022-08-01T19:41:47.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.826+0000 7f678affd700 1 -- 172.21.15.17:0/1893849200 learned_addr learned my addr 172.21.15.17:0/1893849200 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:47.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.826+0000 7f678b7fe700 1 --2- 172.21.15.17:0/1893849200 >> v2:172.21.15.77:3300/0 conn(0x7f678c05c620 0x7f678c0a8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:47.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.826+0000 7f678a7fc700 1 -- 172.21.15.17:0/1893849200 >> v2:172.21.15.77:3300/0 conn(0x7f678c05c620 msgr2=0x7f678c0a8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:47.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.826+0000 7f678a7fc700 1 --2- 172.21.15.17:0/1893849200 >> v2:172.21.15.77:3300/0 conn(0x7f678c05c620 0x7f678c0a8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:47.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.826+0000 7f678a7fc700 1 -- 172.21.15.17:0/1893849200 >> v1:172.21.15.77:6789/0 conn(0x7f678c0b2930 legacy=0x7f678c14b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:47.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.826+0000 7f678a7fc700 1 -- 172.21.15.17:0/1893849200 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f678c14ac60 con 0x7f678c0b1ea0 2022-08-01T19:41:47.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.826+0000 7f678a7fc700 1 --2- 172.21.15.17:0/1893849200 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f678c0b1ea0 0x7f678c147600 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f6774002f70 tx=0x7f6774006df0).ready entity=mon.2 client_cookie=20c4605876118658 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:47.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.826+0000 7f6789ffb700 1 -- 172.21.15.17:0/1893849200 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f677400d570 con 0x7f678c0b1ea0 2022-08-01T19:41:47.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.830+0000 7f6789ffb700 1 -- 172.21.15.17:0/1893849200 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6774000e90 con 0x7f678c0b1ea0 2022-08-01T19:41:47.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.830+0000 7f6789ffb700 1 -- 172.21.15.17:0/1893849200 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f677400d8c0 con 0x7f678c0b1ea0 2022-08-01T19:41:47.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.830+0000 7f678bfff700 1 -- 172.21.15.17:0/1893849200 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f678c0b1ea0 msgr2=0x7f678c147600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:47.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.830+0000 7f678bfff700 1 --2- 172.21.15.17:0/1893849200 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f678c0b1ea0 0x7f678c147600 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f6774002f70 tx=0x7f6774006df0).stop 2022-08-01T19:41:47.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.830+0000 7f678bfff700 1 -- 172.21.15.17:0/1893849200 shutdown_connections 2022-08-01T19:41:47.858 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.830+0000 7f678bfff700 1 --2- 172.21.15.17:0/1893849200 >> v2:172.21.15.77:3300/0 conn(0x7f678c05c620 0x7f678c0a8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:47.858 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.830+0000 7f678bfff700 1 --2- 172.21.15.17:0/1893849200 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f678c0b1ea0 0x7f678c147600 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:47.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.830+0000 7f678bfff700 1 -- 172.21.15.17:0/1893849200 >> 172.21.15.17:0/1893849200 conn(0x7f678c0612d0 msgr2=0x7f678c062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:47.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.830+0000 7f678bfff700 1 -- 172.21.15.17:0/1893849200 shutdown_connections 2022-08-01T19:41:47.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.830+0000 7f678bfff700 1 -- 172.21.15.17:0/1893849200 wait complete. 2022-08-01T19:41:47.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.830+0000 7f678bfff700 1 Processor -- start 2022-08-01T19:41:47.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.830+0000 7f678bfff700 1 -- start start 2022-08-01T19:41:47.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.834+0000 7f678bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f678c05c620 0x7f678c0af390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:47.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.834+0000 7f678bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f678c0b1ea0 0x7f678c064010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:47.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.834+0000 7f678bfff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f678c0b2930 0x7f678c159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:47.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.834+0000 7f678bfff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c0abee0 con 0x7f678c0b1ea0 2022-08-01T19:41:47.871 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.834+0000 7f678bfff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c1573e0 con 0x7f678c05c620 2022-08-01T19:41:47.871 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.834+0000 7f678bfff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678c157680 con 0x7f678c0b2930 2022-08-01T19:41:47.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.834+0000 7f678a7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f678c0b1ea0 0x7f678c064010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:47.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.834+0000 7f678a7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f678c0b1ea0 0x7f678c064010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42660/0 (socket says 172.21.15.17:42660) 2022-08-01T19:41:47.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.834+0000 7f678a7fc700 1 -- 172.21.15.17:0/1332999133 learned_addr learned my addr 172.21.15.17:0/1332999133 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:47.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.834+0000 7f678affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f678c05c620 0x7f678c0af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:47.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.834+0000 7f678b7fe700 1 --2- 172.21.15.17:0/1332999133 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f678c0b2930 0x7f678c159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:47.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.834+0000 7f678a7fc700 1 -- 172.21.15.17:0/1332999133 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f678c0b2930 msgr2=0x7f678c159c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:47.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.834+0000 7f678a7fc700 1 --2- 172.21.15.17:0/1332999133 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f678c0b2930 0x7f678c159c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:47.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.834+0000 7f678a7fc700 1 -- 172.21.15.17:0/1332999133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f678c05c620 msgr2=0x7f678c0af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:47.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.834+0000 7f678a7fc700 1 --2- 172.21.15.17:0/1332999133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f678c05c620 0x7f678c0af390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:47.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.834+0000 7f678a7fc700 1 -- 172.21.15.17:0/1332999133 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f677400b180 con 0x7f678c0b1ea0 2022-08-01T19:41:47.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.834+0000 7f678a7fc700 1 --2- 172.21.15.17:0/1332999133 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f678c0b1ea0 0x7f678c064010 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 rx=0x7f6774002f70 tx=0x7f6774014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:47.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.834+0000 7f6783fff700 1 -- 172.21.15.17:0/1332999133 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f677400dea0 con 0x7f678c0b1ea0 2022-08-01T19:41:47.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.834+0000 7f6783fff700 1 -- 172.21.15.17:0/1332999133 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6774013e90 con 0x7f678c0b1ea0 2022-08-01T19:41:47.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.834+0000 7f678bfff700 1 -- 172.21.15.17:0/1332999133 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f678c063b70 con 0x7f678c0b1ea0 2022-08-01T19:41:47.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.834+0000 7f6783fff700 1 -- 172.21.15.17:0/1332999133 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f677401e5f0 con 0x7f678c0b1ea0 2022-08-01T19:41:47.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.834+0000 7f678bfff700 1 -- 172.21.15.17:0/1332999133 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f678c064500 con 0x7f678c0b1ea0 2022-08-01T19:41:47.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.834+0000 7f678bfff700 1 -- 172.21.15.17:0/1332999133 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f678c063d00 con 0x7f678c0b1ea0 2022-08-01T19:41:47.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.838+0000 7f6783fff700 1 -- 172.21.15.17:0/1332999133 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6774025030 con 0x7f678c0b1ea0 2022-08-01T19:41:47.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.838+0000 7f6783fff700 1 --2- 172.21.15.17:0/1332999133 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6770053aa0 0x7f6770055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:47.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.838+0000 7f6783fff700 1 -- 172.21.15.17:0/1332999133 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f677406ce70 con 0x7f678c0b1ea0 2022-08-01T19:41:47.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.838+0000 7f678affd700 1 --2- 172.21.15.17:0/1332999133 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6770053aa0 0x7f6770055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:47.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.838+0000 7f678affd700 1 --2- 172.21.15.17:0/1332999133 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6770053aa0 0x7f6770055f60 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f677c00d000 tx=0x7f677c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:47.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.846+0000 7f6783fff700 1 -- 172.21.15.17:0/1332999133 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6774043b20 con 0x7f678c0b1ea0 2022-08-01T19:41:47.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.970+0000 7f678bfff700 1 -- 172.21.15.17:0/1332999133 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f678c0574e0 con 0x7f678c0b1ea0 2022-08-01T19:41:47.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.974+0000 7f6783fff700 1 -- 172.21.15.17:0/1332999133 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f6774023070 con 0x7f678c0b1ea0 2022-08-01T19:41:47.974 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:47.975 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":107,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:41:47.421949+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:41:45.865183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2022-08-01T19:41:46.670831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:41:47.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.974+0000 7f678bfff700 1 -- 172.21.15.17:0/1332999133 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6770053aa0 msgr2=0x7f6770055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:47.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.974+0000 7f678bfff700 1 --2- 172.21.15.17:0/1332999133 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6770053aa0 0x7f6770055f60 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f677c00d000 tx=0x7f677c009040).stop 2022-08-01T19:41:47.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.974+0000 7f678bfff700 1 -- 172.21.15.17:0/1332999133 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f678c0b1ea0 msgr2=0x7f678c064010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:47.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.974+0000 7f678bfff700 1 --2- 172.21.15.17:0/1332999133 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f678c0b1ea0 0x7f678c064010 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 rx=0x7f6774002f70 tx=0x7f6774014000).stop 2022-08-01T19:41:47.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.974+0000 7f678bfff700 1 -- 172.21.15.17:0/1332999133 shutdown_connections 2022-08-01T19:41:47.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.974+0000 7f678bfff700 1 --2- 172.21.15.17:0/1332999133 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6770053aa0 0x7f6770055f60 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:47.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.974+0000 7f678bfff700 1 --2- 172.21.15.17:0/1332999133 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f678c0b2930 0x7f678c159c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:47.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.974+0000 7f678bfff700 1 --2- 172.21.15.17:0/1332999133 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f678c0b1ea0 0x7f678c064010 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:47.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.974+0000 7f678bfff700 1 --2- 172.21.15.17:0/1332999133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f678c05c620 0x7f678c0af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:47.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.974+0000 7f678bfff700 1 -- 172.21.15.17:0/1332999133 >> 172.21.15.17:0/1332999133 conn(0x7f678c0612d0 msgr2=0x7f678c0b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:47.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.974+0000 7f678bfff700 1 -- 172.21.15.17:0/1332999133 shutdown_connections 2022-08-01T19:41:47.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:47.974+0000 7f678bfff700 1 -- 172.21.15.17:0/1332999133 wait complete. 2022-08-01T19:41:47.994 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-08-01T19:41:48.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.102+0000 7fabd8d82700 1 Processor -- start 2022-08-01T19:41:48.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.102+0000 7fabd8d82700 1 -- start start 2022-08-01T19:41:48.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.102+0000 7fabd8d82700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fabd40b0470 0x7fabd414b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:48.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.102+0000 7fabd8d82700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabd40adf10 con 0x7fabd40b2710 2022-08-01T19:41:48.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.102+0000 7fabd8d82700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabd40ae1b0 con 0x7fabd40b13f0 2022-08-01T19:41:48.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.102+0000 7fabd8d82700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabd40a9520 con 0x7fabd40b0470 2022-08-01T19:41:48.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.102+0000 7fabd37fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fabd40b2710 0x7fabd40abca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34860/0 (socket says 172.21.15.17:34860) 2022-08-01T19:41:48.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.102+0000 7fabd37fe700 1 -- 172.21.15.17:0/816809598 learned_addr learned my addr 172.21.15.17:0/816809598 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:48.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.106+0000 7fabd3fff700 1 --2- 172.21.15.17:0/816809598 >> v2:172.21.15.77:3300/0 conn(0x7fabd40b0470 0x7fabd414b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:48.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.106+0000 7fabd2ffd700 1 -- 172.21.15.17:0/816809598 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4127100124 0 0) 0x7fabd40adf10 con 0x7fabd40b2710 2022-08-01T19:41:48.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.106+0000 7fabd2ffd700 1 -- 172.21.15.17:0/816809598 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabb8003260 con 0x7fabd40b2710 2022-08-01T19:41:48.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.106+0000 7fabd3fff700 1 -- 172.21.15.17:0/816809598 >> v1:172.21.15.77:6789/0 conn(0x7fabd40b13f0 legacy=0x7fabd40afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:48.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.106+0000 7fabd3fff700 1 -- 172.21.15.17:0/816809598 >> v1:172.21.15.17:6789/0 conn(0x7fabd40b2710 legacy=0x7fabd40abca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:48.135 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.106+0000 7fabd3fff700 1 -- 172.21.15.17:0/816809598 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd40a99c0 con 0x7fabd40b0470 2022-08-01T19:41:48.135 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.106+0000 7fabd3fff700 1 --2- 172.21.15.17:0/816809598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabd40b0470 0x7fabd414b640 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7fabcc0032f0 tx=0x7fabcc0091d0).ready entity=mon.1 client_cookie=a1613ece69424445 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:48.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.106+0000 7fabd2ffd700 1 -- 172.21.15.17:0/816809598 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fabcc00b6c0 con 0x7fabd40b0470 2022-08-01T19:41:48.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.106+0000 7fabd2ffd700 1 -- 172.21.15.17:0/816809598 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabcc00b820 con 0x7fabd40b0470 2022-08-01T19:41:48.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.106+0000 7fabd2ffd700 1 -- 172.21.15.17:0/816809598 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fabcc00b990 con 0x7fabd40b0470 2022-08-01T19:41:48.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.106+0000 7fabd8d82700 1 -- 172.21.15.17:0/816809598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabd40b0470 msgr2=0x7fabd414b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:48.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.106+0000 7fabd8d82700 1 --2- 172.21.15.17:0/816809598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabd40b0470 0x7fabd414b640 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7fabcc0032f0 tx=0x7fabcc0091d0).stop 2022-08-01T19:41:48.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.106+0000 7fabd8d82700 1 -- 172.21.15.17:0/816809598 shutdown_connections 2022-08-01T19:41:48.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.106+0000 7fabd8d82700 1 --2- 172.21.15.17:0/816809598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabd40b0470 0x7fabd414b640 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:48.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.106+0000 7fabd8d82700 1 -- 172.21.15.17:0/816809598 >> 172.21.15.17:0/816809598 conn(0x7fabd4060310 msgr2=0x7fabd40b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:48.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.106+0000 7fabd8d82700 1 -- 172.21.15.17:0/816809598 shutdown_connections 2022-08-01T19:41:48.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.106+0000 7fabd8d82700 1 -- 172.21.15.17:0/816809598 wait complete. 2022-08-01T19:41:48.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.106+0000 7fabd8d82700 1 Processor -- start 2022-08-01T19:41:48.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.106+0000 7fabd8d82700 1 -- start start 2022-08-01T19:41:48.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.110+0000 7fabd8d82700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabd40b0470 0x7fabd4064c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:48.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.110+0000 7fabd8d82700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabd40b13f0 0x7fabd413ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:48.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.110+0000 7fabd8d82700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fabd40b2710 0x7fabd4063190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:48.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.110+0000 7fabd8d82700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd414fe80 con 0x7fabd40b13f0 2022-08-01T19:41:48.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.110+0000 7fabd8d82700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd40a85f0 con 0x7fabd40b0470 2022-08-01T19:41:48.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.110+0000 7fabd8d82700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd40a8890 con 0x7fabd40b2710 2022-08-01T19:41:48.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.110+0000 7fabd37fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabd40b0470 0x7fabd4064c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:48.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.110+0000 7fabcbfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabd40b13f0 0x7fabd413ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:48.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.110+0000 7fabd37fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabd40b0470 0x7fabd4064c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:40594/0 (socket says 172.21.15.17:40594) 2022-08-01T19:41:48.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.110+0000 7fabd37fe700 1 -- 172.21.15.17:0/4107561445 learned_addr learned my addr 172.21.15.17:0/4107561445 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:48.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.110+0000 7fabd3fff700 1 --2- 172.21.15.17:0/4107561445 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fabd40b2710 0x7fabd4063190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:48.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.110+0000 7fabd37fe700 1 -- 172.21.15.17:0/4107561445 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fabd40b2710 msgr2=0x7fabd4063190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:48.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.110+0000 7fabd37fe700 1 --2- 172.21.15.17:0/4107561445 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fabd40b2710 0x7fabd4063190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:48.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.110+0000 7fabd37fe700 1 -- 172.21.15.17:0/4107561445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabd40b13f0 msgr2=0x7fabd413ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:48.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.110+0000 7fabd37fe700 1 --2- 172.21.15.17:0/4107561445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabd40b13f0 0x7fabd413ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:48.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.110+0000 7fabd37fe700 1 -- 172.21.15.17:0/4107561445 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabcc009470 con 0x7fabd40b0470 2022-08-01T19:41:48.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.110+0000 7fabcbfff700 1 --2- 172.21.15.17:0/4107561445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabd40b13f0 0x7fabd413ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:48.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.110+0000 7fabd37fe700 1 --2- 172.21.15.17:0/4107561445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabd40b0470 0x7fabd4064c40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7fabd409f1a0 tx=0x7fabc4007b60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:48.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.110+0000 7fabd17fa700 1 -- 172.21.15.17:0/4107561445 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fabc40072e0 con 0x7fabd40b0470 2022-08-01T19:41:48.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.110+0000 7fabd17fa700 1 -- 172.21.15.17:0/4107561445 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc4007440 con 0x7fabd40b0470 2022-08-01T19:41:48.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.110+0000 7fabd8d82700 1 -- 172.21.15.17:0/4107561445 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd41408b0 con 0x7fabd40b0470 2022-08-01T19:41:48.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.110+0000 7fabd17fa700 1 -- 172.21.15.17:0/4107561445 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fabc4005800 con 0x7fabd40b0470 2022-08-01T19:41:48.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.110+0000 7fabd8d82700 1 -- 172.21.15.17:0/4107561445 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd414fb80 con 0x7fabd40b0470 2022-08-01T19:41:48.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.110+0000 7fabd8d82700 1 -- 172.21.15.17:0/4107561445 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabd40a8a90 con 0x7fabd40b0470 2022-08-01T19:41:48.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.114+0000 7fabd17fa700 1 -- 172.21.15.17:0/4107561445 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fabc4005960 con 0x7fabd40b0470 2022-08-01T19:41:48.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.114+0000 7fabd17fa700 1 --2- 172.21.15.17:0/4107561445 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fabac053a70 0x7fabac055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:48.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.114+0000 7fabd17fa700 1 -- 172.21.15.17:0/4107561445 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fabc406e3d0 con 0x7fabd40b0470 2022-08-01T19:41:48.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.114+0000 7fabcbfff700 1 --2- 172.21.15.17:0/4107561445 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fabac053a70 0x7fabac055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:48.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.114+0000 7fabcbfff700 1 --2- 172.21.15.17:0/4107561445 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fabac053a70 0x7fabac055f30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7fabbc00d000 tx=0x7fabbc009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:48.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.122+0000 7fabd17fa700 1 -- 172.21.15.17:0/4107561445 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fabc4045190 con 0x7fabd40b0470 2022-08-01T19:41:48.176 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:41:48.181 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:41:48.184 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:41:48.190 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:41:48.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.286+0000 7fabd8d82700 1 -- 172.21.15.17:0/4107561445 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fabd40574e0 con 0x7fabd40b0470 2022-08-01T19:41:48.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.434+0000 7fabd17fa700 1 -- 172.21.15.17:0/4107561445 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v53) v1 ==== 118+0+0 (secure 0 0 0) 0x7fabc4018300 con 0x7fabd40b0470 2022-08-01T19:41:48.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.438+0000 7fabd8d82700 1 -- 172.21.15.17:0/4107561445 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fabac053a70 msgr2=0x7fabac055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:48.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.438+0000 7fabd8d82700 1 --2- 172.21.15.17:0/4107561445 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fabac053a70 0x7fabac055f30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7fabbc00d000 tx=0x7fabbc009040).stop 2022-08-01T19:41:48.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.438+0000 7fabd8d82700 1 -- 172.21.15.17:0/4107561445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabd40b0470 msgr2=0x7fabd4064c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:48.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.438+0000 7fabd8d82700 1 --2- 172.21.15.17:0/4107561445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabd40b0470 0x7fabd4064c40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7fabd409f1a0 tx=0x7fabc4007b60).stop 2022-08-01T19:41:48.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.438+0000 7fabd8d82700 1 -- 172.21.15.17:0/4107561445 shutdown_connections 2022-08-01T19:41:48.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.438+0000 7fabd8d82700 1 --2- 172.21.15.17:0/4107561445 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fabac053a70 0x7fabac055f30 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:48.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.438+0000 7fabd8d82700 1 --2- 172.21.15.17:0/4107561445 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fabd40b2710 0x7fabd4063190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:48.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.438+0000 7fabd8d82700 1 --2- 172.21.15.17:0/4107561445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabd40b13f0 0x7fabd413ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:48.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.438+0000 7fabd8d82700 1 --2- 172.21.15.17:0/4107561445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabd40b0470 0x7fabd4064c40 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:48.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.438+0000 7fabd8d82700 1 -- 172.21.15.17:0/4107561445 >> 172.21.15.17:0/4107561445 conn(0x7fabd4060310 msgr2=0x7fabd40b4940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:48.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.438+0000 7fabd8d82700 1 -- 172.21.15.17:0/4107561445 shutdown_connections 2022-08-01T19:41:48.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.438+0000 7fabd8d82700 1 -- 172.21.15.17:0/4107561445 wait complete. 2022-08-01T19:41:48.456 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:41:48.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.562+0000 7f5d2cc45700 1 Processor -- start 2022-08-01T19:41:48.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.562+0000 7f5d2cc45700 1 -- start start 2022-08-01T19:41:48.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.566+0000 7f5d2cc45700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d28148530 0x7f5d280b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:48.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.566+0000 7f5d2cc45700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d280b04e0 con 0x7f5d280a7250 2022-08-01T19:41:48.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.566+0000 7f5d2cc45700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d280b0780 con 0x7f5d280b1f40 2022-08-01T19:41:48.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.566+0000 7f5d2cc45700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d280a9250 con 0x7f5d28148530 2022-08-01T19:41:48.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.566+0000 7f5d26d9d700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5d280a7250 0x7f5d281448b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34878/0 (socket says 172.21.15.17:34878) 2022-08-01T19:41:48.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.566+0000 7f5d26d9d700 1 -- 172.21.15.17:0/3320039755 learned_addr learned my addr 172.21.15.17:0/3320039755 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:48.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.566+0000 7f5d25d9b700 1 --2- 172.21.15.17:0/3320039755 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d28148530 0x7f5d280b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:48.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.566+0000 7f5d2559a700 1 -- 172.21.15.17:0/3320039755 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4223973746 0 0) 0x7f5d280b04e0 con 0x7f5d280a7250 2022-08-01T19:41:48.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.566+0000 7f5d2559a700 1 -- 172.21.15.17:0/3320039755 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d0c003260 con 0x7f5d280a7250 2022-08-01T19:41:48.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.566+0000 7f5d25d9b700 1 -- 172.21.15.17:0/3320039755 >> v1:172.21.15.77:6789/0 conn(0x7f5d280b1f40 legacy=0x7f5d280b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:48.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.566+0000 7f5d2559a700 1 -- 172.21.15.17:0/3320039755 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4211962509 0 0) 0x7f5d280b0780 con 0x7f5d280b1f40 2022-08-01T19:41:48.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.566+0000 7f5d25d9b700 1 -- 172.21.15.17:0/3320039755 >> v1:172.21.15.17:6789/0 conn(0x7f5d280a7250 legacy=0x7f5d281448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:48.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.566+0000 7f5d25d9b700 1 -- 172.21.15.17:0/3320039755 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d280b0a20 con 0x7f5d28148530 2022-08-01T19:41:48.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.566+0000 7f5d25d9b700 1 --2- 172.21.15.17:0/3320039755 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d28148530 0x7f5d280b1160 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f5d10002f70 tx=0x7f5d10006df0).ready entity=mon.2 client_cookie=fcec75bb1c9cd046 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:48.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.566+0000 7f5d2559a700 1 -- 172.21.15.17:0/3320039755 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d1000d570 con 0x7f5d28148530 2022-08-01T19:41:48.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.566+0000 7f5d2559a700 1 -- 172.21.15.17:0/3320039755 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d10000e90 con 0x7f5d28148530 2022-08-01T19:41:48.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.566+0000 7f5d2559a700 1 -- 172.21.15.17:0/3320039755 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d1000d8c0 con 0x7f5d28148530 2022-08-01T19:41:48.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.566+0000 7f5d2cc45700 1 -- 172.21.15.17:0/3320039755 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d28148530 msgr2=0x7f5d280b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:48.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.566+0000 7f5d2cc45700 1 --2- 172.21.15.17:0/3320039755 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d28148530 0x7f5d280b1160 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f5d10002f70 tx=0x7f5d10006df0).stop 2022-08-01T19:41:48.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.566+0000 7f5d2cc45700 1 -- 172.21.15.17:0/3320039755 shutdown_connections 2022-08-01T19:41:48.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.566+0000 7f5d2cc45700 1 --2- 172.21.15.17:0/3320039755 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d28148530 0x7f5d280b1160 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:48.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.566+0000 7f5d2cc45700 1 -- 172.21.15.17:0/3320039755 >> 172.21.15.17:0/3320039755 conn(0x7f5d2805f840 msgr2=0x7f5d2805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:48.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.566+0000 7f5d2cc45700 1 -- 172.21.15.17:0/3320039755 shutdown_connections 2022-08-01T19:41:48.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.566+0000 7f5d2cc45700 1 -- 172.21.15.17:0/3320039755 wait complete. 2022-08-01T19:41:48.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.570+0000 7f5d2cc45700 1 Processor -- start 2022-08-01T19:41:48.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.570+0000 7f5d2cc45700 1 -- start start 2022-08-01T19:41:48.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.570+0000 7f5d2cc45700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d280a7250 0x7f5d2813bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:48.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.570+0000 7f5d2cc45700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d280b1f40 0x7f5d2813de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:48.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.570+0000 7f5d2cc45700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d28148530 0x7f5d28159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:48.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.570+0000 7f5d2cc45700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2813b760 con 0x7f5d280b1f40 2022-08-01T19:41:48.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.570+0000 7f5d2cc45700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2813c200 con 0x7f5d280a7250 2022-08-01T19:41:48.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.570+0000 7f5d2cc45700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d28147100 con 0x7f5d28148530 2022-08-01T19:41:48.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.570+0000 7f5d25d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d280b1f40 0x7f5d2813de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:48.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.570+0000 7f5d25d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d280b1f40 0x7f5d2813de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42688/0 (socket says 172.21.15.17:42688) 2022-08-01T19:41:48.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.570+0000 7f5d25d9b700 1 -- 172.21.15.17:0/2718464191 learned_addr learned my addr 172.21.15.17:0/2718464191 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:48.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.570+0000 7f5d26d9d700 1 --2- 172.21.15.17:0/2718464191 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d28148530 0x7f5d28159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:48.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.570+0000 7f5d2659c700 1 --2- 172.21.15.17:0/2718464191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d280a7250 0x7f5d2813bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:48.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.570+0000 7f5d25d9b700 1 -- 172.21.15.17:0/2718464191 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d28148530 msgr2=0x7f5d28159e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:48.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.570+0000 7f5d25d9b700 1 --2- 172.21.15.17:0/2718464191 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d28148530 0x7f5d28159e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:48.602 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.570+0000 7f5d25d9b700 1 -- 172.21.15.17:0/2718464191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d280a7250 msgr2=0x7f5d2813bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:48.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.570+0000 7f5d25d9b700 1 --2- 172.21.15.17:0/2718464191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d280a7250 0x7f5d2813bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:48.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.570+0000 7f5d25d9b700 1 -- 172.21.15.17:0/2718464191 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d1000b180 con 0x7f5d280b1f40 2022-08-01T19:41:48.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.570+0000 7f5d25d9b700 1 --2- 172.21.15.17:0/2718464191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d280b1f40 0x7f5d2813de80 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 rx=0x7f5d10002f70 tx=0x7f5d10014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:48.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.570+0000 7f5d1f7fe700 1 -- 172.21.15.17:0/2718464191 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d10016070 con 0x7f5d280b1f40 2022-08-01T19:41:48.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.570+0000 7f5d1f7fe700 1 -- 172.21.15.17:0/2718464191 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d10013780 con 0x7f5d280b1f40 2022-08-01T19:41:48.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.570+0000 7f5d2cc45700 1 -- 172.21.15.17:0/2718464191 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d281473a0 con 0x7f5d280b1f40 2022-08-01T19:41:48.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.570+0000 7f5d1f7fe700 1 -- 172.21.15.17:0/2718464191 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d10013ae0 con 0x7f5d280b1f40 2022-08-01T19:41:48.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.570+0000 7f5d2cc45700 1 -- 172.21.15.17:0/2718464191 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d2813c980 con 0x7f5d280b1f40 2022-08-01T19:41:48.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.574+0000 7f5d2cc45700 1 -- 172.21.15.17:0/2718464191 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d280574e0 con 0x7f5d280b1f40 2022-08-01T19:41:48.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.574+0000 7f5d1f7fe700 1 -- 172.21.15.17:0/2718464191 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5d1000f030 con 0x7f5d280b1f40 2022-08-01T19:41:48.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.574+0000 7f5d1f7fe700 1 --2- 172.21.15.17:0/2718464191 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5d00053a70 0x7f5d00055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:48.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.574+0000 7f5d1f7fe700 1 -- 172.21.15.17:0/2718464191 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5d1006cc60 con 0x7f5d280b1f40 2022-08-01T19:41:48.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.574+0000 7f5d2659c700 1 --2- 172.21.15.17:0/2718464191 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5d00053a70 0x7f5d00055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:48.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.574+0000 7f5d2659c700 1 --2- 172.21.15.17:0/2718464191 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5d00053a70 0x7f5d00055f30 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f5d1800b000 tx=0x7f5d18009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:48.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.578+0000 7f5d1f7fe700 1 -- 172.21.15.17:0/2718464191 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5d10043980 con 0x7f5d280b1f40 2022-08-01T19:41:48.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.738+0000 7f5d2cc45700 1 -- 172.21.15.17:0/2718464191 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d280447e0 con 0x7f5d280b1f40 2022-08-01T19:41:48.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.742+0000 7f5d1f7fe700 1 -- 172.21.15.17:0/2718464191 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f5d10023070 con 0x7f5d280b1f40 2022-08-01T19:41:48.752 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:48.753 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":53,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7252,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/1455566688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1455566688},{"type":"v1","addr":"172.21.15.77:6835","nonce":1455566688}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":51},{"gid":7266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.17:6833/249859043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":249859043},{"type":"v1","addr":"172.21.15.17:6833","nonce":249859043}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":51}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:41:47.422244+0000","modified":"2022-08-01T19:41:48.425195+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7272":{"gid":7272,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/2991628239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2991628239},{"type":"v1","addr":"172.21.15.17:6835","nonce":2991628239}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2022-08-01T19:41:48.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.742+0000 7f5d2cc45700 1 -- 172.21.15.17:0/2718464191 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5d00053a70 msgr2=0x7f5d00055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:48.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.742+0000 7f5d2cc45700 1 --2- 172.21.15.17:0/2718464191 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5d00053a70 0x7f5d00055f30 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f5d1800b000 tx=0x7f5d18009040).stop 2022-08-01T19:41:48.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.742+0000 7f5d2cc45700 1 -- 172.21.15.17:0/2718464191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d280b1f40 msgr2=0x7f5d2813de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:48.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.742+0000 7f5d2cc45700 1 --2- 172.21.15.17:0/2718464191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d280b1f40 0x7f5d2813de80 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 rx=0x7f5d10002f70 tx=0x7f5d10014000).stop 2022-08-01T19:41:48.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.742+0000 7f5d2cc45700 1 -- 172.21.15.17:0/2718464191 shutdown_connections 2022-08-01T19:41:48.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.742+0000 7f5d2cc45700 1 --2- 172.21.15.17:0/2718464191 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5d00053a70 0x7f5d00055f30 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:48.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.742+0000 7f5d2cc45700 1 --2- 172.21.15.17:0/2718464191 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5d28148530 0x7f5d28159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:48.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.742+0000 7f5d2cc45700 1 --2- 172.21.15.17:0/2718464191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d280b1f40 0x7f5d2813de80 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:48.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.742+0000 7f5d2cc45700 1 --2- 172.21.15.17:0/2718464191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d280a7250 0x7f5d2813bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:48.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.742+0000 7f5d2cc45700 1 -- 172.21.15.17:0/2718464191 >> 172.21.15.17:0/2718464191 conn(0x7f5d2805f840 msgr2=0x7f5d280b4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:48.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.742+0000 7f5d2cc45700 1 -- 172.21.15.17:0/2718464191 shutdown_connections 2022-08-01T19:41:48.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.742+0000 7f5d2cc45700 1 -- 172.21.15.17:0/2718464191 wait complete. 2022-08-01T19:41:48.762 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 53 2022-08-01T19:41:48.763 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:41:48.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.874+0000 7ff8ef542700 1 Processor -- start 2022-08-01T19:41:48.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.874+0000 7ff8ef542700 1 -- start start 2022-08-01T19:41:48.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.878+0000 7ff8ef542700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ff8e80b2710 0x7ff8e81401e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:48.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.878+0000 7ff8ef542700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff8e80b0270 0x7ff8e81406d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:48.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.878+0000 7ff8ef542700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8e8067a10 con 0x7ff8e80ac880 2022-08-01T19:41:48.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.878+0000 7ff8ef542700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8e80ad740 con 0x7ff8e80b2710 2022-08-01T19:41:48.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.878+0000 7ff8ef542700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e8067cb0 con 0x7ff8e80b0270 2022-08-01T19:41:48.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.878+0000 7ff8eed41700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff8e80ac880 0x7ff8e8143000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34890/0 (socket says 172.21.15.17:34890) 2022-08-01T19:41:48.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.878+0000 7ff8eed41700 1 -- 172.21.15.17:0/2254562079 learned_addr learned my addr 172.21.15.17:0/2254562079 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:48.901 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.878+0000 7ff8edd3f700 1 --2- 172.21.15.17:0/2254562079 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff8e80b0270 0x7ff8e81406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:48.901 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.878+0000 7ff8ee540700 1 --2- 172.21.15.17:0/2254562079 >> v2:172.21.15.77:3300/0 conn(0x7ff8e80b2710 0x7ff8e81401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:48.902 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.878+0000 7ff8ed53e700 1 -- 172.21.15.17:0/2254562079 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2499802407 0 0) 0x7ff8e8067a10 con 0x7ff8e80ac880 2022-08-01T19:41:48.902 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.878+0000 7ff8ed53e700 1 -- 172.21.15.17:0/2254562079 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8d0003260 con 0x7ff8e80ac880 2022-08-01T19:41:48.903 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.878+0000 7ff8edd3f700 1 -- 172.21.15.17:0/2254562079 >> v2:172.21.15.77:3300/0 conn(0x7ff8e80b2710 msgr2=0x7ff8e81401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:48.903 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.878+0000 7ff8edd3f700 1 --2- 172.21.15.17:0/2254562079 >> v2:172.21.15.77:3300/0 conn(0x7ff8e80b2710 0x7ff8e81401e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:48.903 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.878+0000 7ff8edd3f700 1 -- 172.21.15.17:0/2254562079 >> v1:172.21.15.17:6789/0 conn(0x7ff8e80ac880 legacy=0x7ff8e8143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:48.904 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.878+0000 7ff8edd3f700 1 -- 172.21.15.17:0/2254562079 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e8147160 con 0x7ff8e80b0270 2022-08-01T19:41:48.904 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.878+0000 7ff8edd3f700 1 --2- 172.21.15.17:0/2254562079 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff8e80b0270 0x7ff8e81406d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7ff8e4002f70 tx=0x7ff8e4006df0).ready entity=mon.2 client_cookie=41d0f26e7275a7e7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:48.905 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.878+0000 7ff8ed53e700 1 -- 172.21.15.17:0/2254562079 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8e400d4a0 con 0x7ff8e80b0270 2022-08-01T19:41:48.905 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.878+0000 7ff8ed53e700 1 -- 172.21.15.17:0/2254562079 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e4000e90 con 0x7ff8e80b0270 2022-08-01T19:41:48.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.878+0000 7ff8ed53e700 1 -- 172.21.15.17:0/2254562079 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8e400d7f0 con 0x7ff8e80b0270 2022-08-01T19:41:48.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.878+0000 7ff8ef542700 1 -- 172.21.15.17:0/2254562079 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff8e80b0270 msgr2=0x7ff8e81406d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:48.907 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.878+0000 7ff8ef542700 1 --2- 172.21.15.17:0/2254562079 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff8e80b0270 0x7ff8e81406d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7ff8e4002f70 tx=0x7ff8e4006df0).stop 2022-08-01T19:41:48.907 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.878+0000 7ff8ef542700 1 -- 172.21.15.17:0/2254562079 shutdown_connections 2022-08-01T19:41:48.908 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.878+0000 7ff8ef542700 1 --2- 172.21.15.17:0/2254562079 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff8e80b0270 0x7ff8e81406d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:48.908 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.878+0000 7ff8ef542700 1 --2- 172.21.15.17:0/2254562079 >> v2:172.21.15.77:3300/0 conn(0x7ff8e80b2710 0x7ff8e81401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:48.908 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.878+0000 7ff8ef542700 1 -- 172.21.15.17:0/2254562079 >> 172.21.15.17:0/2254562079 conn(0x7ff8e8060310 msgr2=0x7ff8e80b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:48.909 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.878+0000 7ff8ef542700 1 -- 172.21.15.17:0/2254562079 shutdown_connections 2022-08-01T19:41:48.909 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.878+0000 7ff8ef542700 1 -- 172.21.15.17:0/2254562079 wait complete. 2022-08-01T19:41:48.909 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.878+0000 7ff8ef542700 1 Processor -- start 2022-08-01T19:41:48.910 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.882+0000 7ff8ef542700 1 -- start start 2022-08-01T19:41:48.910 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.882+0000 7ff8ef542700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff8e80ac880 0x7ff8e8153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:48.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.882+0000 7ff8ef542700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8e80b0270 0x7ff8e8144750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:48.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.882+0000 7ff8ef542700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8e80b2710 0x7ff8e80a97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:48.912 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.882+0000 7ff8ef542700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e8145230 con 0x7ff8e80b2710 2022-08-01T19:41:48.912 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.882+0000 7ff8ef542700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e8152870 con 0x7ff8e80b0270 2022-08-01T19:41:48.915 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.882+0000 7ff8ef542700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e8152b10 con 0x7ff8e80ac880 2022-08-01T19:41:48.916 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.882+0000 7ff8eed41700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8e80b2710 0x7ff8e80a97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:48.916 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.882+0000 7ff8eed41700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8e80b2710 0x7ff8e80a97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42704/0 (socket says 172.21.15.17:42704) 2022-08-01T19:41:48.917 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.882+0000 7ff8eed41700 1 -- 172.21.15.17:0/1289109339 learned_addr learned my addr 172.21.15.17:0/1289109339 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:48.917 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.882+0000 7ff8ee540700 1 --2- 172.21.15.17:0/1289109339 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff8e80ac880 0x7ff8e8153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:48.918 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.882+0000 7ff8edd3f700 1 --2- 172.21.15.17:0/1289109339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8e80b0270 0x7ff8e8144750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:48.918 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.882+0000 7ff8eed41700 1 -- 172.21.15.17:0/1289109339 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff8e80ac880 msgr2=0x7ff8e8153250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:48.919 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.882+0000 7ff8eed41700 1 --2- 172.21.15.17:0/1289109339 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff8e80ac880 0x7ff8e8153250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:48.919 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.882+0000 7ff8eed41700 1 -- 172.21.15.17:0/1289109339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8e80b0270 msgr2=0x7ff8e8144750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:48.919 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.882+0000 7ff8eed41700 1 --2- 172.21.15.17:0/1289109339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8e80b0270 0x7ff8e8144750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:48.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.882+0000 7ff8eed41700 1 -- 172.21.15.17:0/1289109339 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e400b180 con 0x7ff8e80b2710 2022-08-01T19:41:48.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.882+0000 7ff8eed41700 1 --2- 172.21.15.17:0/1289109339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8e80b2710 0x7ff8e80a97e0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 rx=0x7ff8e0001490 tx=0x7ff8e0007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:48.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.882+0000 7ff8d77fe700 1 -- 172.21.15.17:0/1289109339 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8e0005c70 con 0x7ff8e80b2710 2022-08-01T19:41:48.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.882+0000 7ff8d77fe700 1 -- 172.21.15.17:0/1289109339 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e0005dd0 con 0x7ff8e80b2710 2022-08-01T19:41:48.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.882+0000 7ff8ef542700 1 -- 172.21.15.17:0/1289109339 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e8152db0 con 0x7ff8e80b2710 2022-08-01T19:41:48.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.882+0000 7ff8d77fe700 1 -- 172.21.15.17:0/1289109339 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8e00072c0 con 0x7ff8e80b2710 2022-08-01T19:41:48.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.882+0000 7ff8ef542700 1 -- 172.21.15.17:0/1289109339 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e8144f20 con 0x7ff8e80b2710 2022-08-01T19:41:48.936 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.882+0000 7ff8ef542700 1 -- 172.21.15.17:0/1289109339 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e80574e0 con 0x7ff8e80b2710 2022-08-01T19:41:48.937 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.886+0000 7ff8d77fe700 1 -- 172.21.15.17:0/1289109339 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff8e001a060 con 0x7ff8e80b2710 2022-08-01T19:41:48.937 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.886+0000 7ff8d77fe700 1 --2- 172.21.15.17:0/1289109339 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff8c8053b00 0x7ff8c8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:48.938 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.886+0000 7ff8d77fe700 1 -- 172.21.15.17:0/1289109339 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff8e006bb50 con 0x7ff8e80b2710 2022-08-01T19:41:48.938 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.886+0000 7ff8ee540700 1 --2- 172.21.15.17:0/1289109339 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff8c8053b00 0x7ff8c8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:48.939 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.886+0000 7ff8ee540700 1 --2- 172.21.15.17:0/1289109339 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff8c8053b00 0x7ff8c8055fc0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7ff8d800b000 tx=0x7ff8d8006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:48.939 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:48.894+0000 7ff8d77fe700 1 -- 172.21.15.17:0/1289109339 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff8e0042800 con 0x7ff8e80b2710 2022-08-01T19:41:49.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.018+0000 7ff8ef542700 1 -- 172.21.15.17:0/1289109339 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8e80aece0 con 0x7ff8e80b2710 2022-08-01T19:41:49.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.022+0000 7ff8d77fe700 1 -- 172.21.15.17:0/1289109339 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16285 (secure 0 0 0) 0x7ff8e0018070 con 0x7ff8e80b2710 2022-08-01T19:41:49.024 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:49.024 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":107,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:41:47.421949+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:41:45.865183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2022-08-01T19:41:46.670831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:41:49.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.026+0000 7ff8ef542700 1 -- 172.21.15.17:0/1289109339 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff8c8053b00 msgr2=0x7ff8c8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:49.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.026+0000 7ff8ef542700 1 --2- 172.21.15.17:0/1289109339 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff8c8053b00 0x7ff8c8055fc0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7ff8d800b000 tx=0x7ff8d8006df0).stop 2022-08-01T19:41:49.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.026+0000 7ff8ef542700 1 -- 172.21.15.17:0/1289109339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8e80b2710 msgr2=0x7ff8e80a97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:49.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.026+0000 7ff8ef542700 1 --2- 172.21.15.17:0/1289109339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8e80b2710 0x7ff8e80a97e0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 rx=0x7ff8e0001490 tx=0x7ff8e0007dc0).stop 2022-08-01T19:41:49.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.026+0000 7ff8ef542700 1 -- 172.21.15.17:0/1289109339 shutdown_connections 2022-08-01T19:41:49.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.026+0000 7ff8ef542700 1 --2- 172.21.15.17:0/1289109339 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff8c8053b00 0x7ff8c8055fc0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:49.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.026+0000 7ff8ef542700 1 --2- 172.21.15.17:0/1289109339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8e80b2710 0x7ff8e80a97e0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:49.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.026+0000 7ff8ef542700 1 --2- 172.21.15.17:0/1289109339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8e80b0270 0x7ff8e8144750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:49.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.026+0000 7ff8ef542700 1 --2- 172.21.15.17:0/1289109339 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff8e80ac880 0x7ff8e8153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:49.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.026+0000 7ff8ef542700 1 -- 172.21.15.17:0/1289109339 >> 172.21.15.17:0/1289109339 conn(0x7ff8e8060310 msgr2=0x7ff8e80adef0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:49.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.026+0000 7ff8ef542700 1 -- 172.21.15.17:0/1289109339 shutdown_connections 2022-08-01T19:41:49.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.026+0000 7ff8ef542700 1 -- 172.21.15.17:0/1289109339 wait complete. 2022-08-01T19:41:49.051 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T19:41:49.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.170+0000 7f0764a27700 1 Processor -- start 2022-08-01T19:41:49.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.170+0000 7f0764a27700 1 -- start start 2022-08-01T19:41:49.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.170+0000 7f0764a27700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0760061890 0x7f07601466b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:49.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.170+0000 7f0764a27700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07600aee80 con 0x7f07600614b0 2022-08-01T19:41:49.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.170+0000 7f0764a27700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07600b70b0 con 0x7f07600ad750 2022-08-01T19:41:49.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.170+0000 7f0764a27700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07600af0b0 con 0x7f0760061890 2022-08-01T19:41:49.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.170+0000 7f075ed9d700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f07600614b0 0x7f07600b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34902/0 (socket says 172.21.15.17:34902) 2022-08-01T19:41:49.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.170+0000 7f075ed9d700 1 -- 172.21.15.17:0/1093939432 learned_addr learned my addr 172.21.15.17:0/1093939432 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:49.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.170+0000 7f075f59e700 1 --2- 172.21.15.17:0/1093939432 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0760061890 0x7f07601466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:49.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.170+0000 7f075dd9b700 1 -- 172.21.15.17:0/1093939432 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 715610105 0 0) 0x7f07600aee80 con 0x7f07600614b0 2022-08-01T19:41:49.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.170+0000 7f075dd9b700 1 -- 172.21.15.17:0/1093939432 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0740003260 con 0x7f07600614b0 2022-08-01T19:41:49.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.170+0000 7f075dd9b700 1 -- 172.21.15.17:0/1093939432 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 440493687 0 0) 0x7f07600b70b0 con 0x7f07600ad750 2022-08-01T19:41:49.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.170+0000 7f075dd9b700 1 -- 172.21.15.17:0/1093939432 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07600aee80 con 0x7f07600ad750 2022-08-01T19:41:49.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.170+0000 7f075dd9b700 1 -- 172.21.15.17:0/1093939432 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 209537719 0 0) 0x7f0740003260 con 0x7f07600614b0 2022-08-01T19:41:49.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.170+0000 7f075dd9b700 1 -- 172.21.15.17:0/1093939432 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0760061890 msgr2=0x7f07601466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:49.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.170+0000 7f075dd9b700 1 --2- 172.21.15.17:0/1093939432 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0760061890 0x7f07601466b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:49.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.170+0000 7f075dd9b700 1 -- 172.21.15.17:0/1093939432 >> v1:172.21.15.77:6789/0 conn(0x7f07600ad750 legacy=0x7f07601405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:49.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.170+0000 7f075dd9b700 1 -- 172.21.15.17:0/1093939432 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07600af350 con 0x7f07600614b0 2022-08-01T19:41:49.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.170+0000 7f075dd9b700 1 -- 172.21.15.17:0/1093939432 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7f0748001660 con 0x7f07600614b0 2022-08-01T19:41:49.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.170+0000 7f075dd9b700 1 -- 172.21.15.17:0/1093939432 >> v1:172.21.15.17:6789/0 conn(0x7f07600614b0 legacy=0x7f07600b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:49.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.174+0000 7f075dd9b700 1 --2- 172.21.15.17:0/1093939432 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0760061890 0x7f0740003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:49.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.174+0000 7f075dd9b700 1 --2- 172.21.15.17:0/1093939432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0740003e20 0x7f0740004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:49.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.174+0000 7f075dd9b700 1 --2- 172.21.15.17:0/1093939432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07400049f0 0x7f0740004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:49.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.174+0000 7f075dd9b700 1 -- 172.21.15.17:0/1093939432 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07600af0b0 con 0x7f0740003e20 2022-08-01T19:41:49.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.174+0000 7f075e59c700 1 --2- 172.21.15.17:0/1093939432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0740003e20 0x7f0740004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:49.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.174+0000 7f075dd9b700 1 -- 172.21.15.17:0/1093939432 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074000b800 con 0x7f07400049f0 2022-08-01T19:41:49.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.174+0000 7f075dd9b700 1 -- 172.21.15.17:0/1093939432 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f074000b9c0 con 0x7f0760061890 2022-08-01T19:41:49.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.174+0000 7f075ed9d700 1 --2- 172.21.15.17:0/1093939432 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0760061890 0x7f0740003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:49.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.174+0000 7f075f59e700 1 --2- 172.21.15.17:0/1093939432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07400049f0 0x7f0740004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:49.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.174+0000 7f075e59c700 1 -- 172.21.15.17:0/1093939432 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0760061890 msgr2=0x7f0740003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:49.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.174+0000 7f075e59c700 1 --2- 172.21.15.17:0/1093939432 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0760061890 0x7f0740003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:49.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.174+0000 7f075e59c700 1 -- 172.21.15.17:0/1093939432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07400049f0 msgr2=0x7f0740004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:49.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.174+0000 7f075e59c700 1 --2- 172.21.15.17:0/1093939432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07400049f0 0x7f0740004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:49.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.174+0000 7f075e59c700 1 -- 172.21.15.17:0/1093939432 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0740003260 con 0x7f0740003e20 2022-08-01T19:41:49.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.174+0000 7f075e59c700 1 --2- 172.21.15.17:0/1093939432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0740003e20 0x7f0740004500 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 rx=0x7f0750001110 tx=0x7f0750007a00).ready entity=mon.0 client_cookie=85e03f30732c77a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:49.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.174+0000 7f075dd9b700 1 -- 172.21.15.17:0/1093939432 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0750007d10 con 0x7f0740003e20 2022-08-01T19:41:49.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.174+0000 7f075dd9b700 1 -- 172.21.15.17:0/1093939432 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0750007030 con 0x7f0740003e20 2022-08-01T19:41:49.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.174+0000 7f075dd9b700 1 -- 172.21.15.17:0/1093939432 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0750007390 con 0x7f0740003e20 2022-08-01T19:41:49.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.174+0000 7f0764a27700 1 -- 172.21.15.17:0/1093939432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0740003e20 msgr2=0x7f0740004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:49.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.174+0000 7f0764a27700 1 --2- 172.21.15.17:0/1093939432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0740003e20 0x7f0740004500 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 rx=0x7f0750001110 tx=0x7f0750007a00).stop 2022-08-01T19:41:49.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.174+0000 7f075f59e700 1 -- 172.21.15.17:0/1093939432 reap_dead start 2022-08-01T19:41:49.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.174+0000 7f0764a27700 1 -- 172.21.15.17:0/1093939432 shutdown_connections 2022-08-01T19:41:49.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.174+0000 7f0764a27700 1 -- 172.21.15.17:0/1093939432 >> 172.21.15.17:0/1093939432 conn(0x7f0760066f70 msgr2=0x7f0760067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:49.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.174+0000 7f0764a27700 1 -- 172.21.15.17:0/1093939432 shutdown_connections 2022-08-01T19:41:49.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.174+0000 7f0764a27700 1 -- 172.21.15.17:0/1093939432 wait complete. 2022-08-01T19:41:49.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.178+0000 7f0764a27700 1 Processor -- start 2022-08-01T19:41:49.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.178+0000 7f0764a27700 1 -- start start 2022-08-01T19:41:49.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.178+0000 7f0764a27700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0740003e20 0x7f07600b0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:49.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.178+0000 7f0764a27700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f07601566b0 0x7f07600b7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:49.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.178+0000 7f0764a27700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0760152f20 0x7f07600a8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:49.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.178+0000 7f0764a27700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07601503b0 con 0x7f0760152f20 2022-08-01T19:41:49.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.178+0000 7f0764a27700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07600b08b0 con 0x7f0740003e20 2022-08-01T19:41:49.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.178+0000 7f0764a27700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07600b0b50 con 0x7f07601566b0 2022-08-01T19:41:49.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.178+0000 7f075f59e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0760152f20 0x7f07600a8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:49.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.178+0000 7f075ed9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0740003e20 0x7f07600b0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:49.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.178+0000 7f075f59e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0760152f20 0x7f07600a8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42724/0 (socket says 172.21.15.17:42724) 2022-08-01T19:41:49.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.178+0000 7f075f59e700 1 -- 172.21.15.17:0/3005912602 learned_addr learned my addr 172.21.15.17:0/3005912602 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:49.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.178+0000 7f075e59c700 1 --2- 172.21.15.17:0/3005912602 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f07601566b0 0x7f07600b7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:49.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.178+0000 7f075f59e700 1 -- 172.21.15.17:0/3005912602 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f07601566b0 msgr2=0x7f07600b7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:49.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.178+0000 7f075f59e700 1 --2- 172.21.15.17:0/3005912602 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f07601566b0 0x7f07600b7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:49.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.178+0000 7f075f59e700 1 -- 172.21.15.17:0/3005912602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0740003e20 msgr2=0x7f07600b0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:49.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.178+0000 7f075f59e700 1 --2- 172.21.15.17:0/3005912602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0740003e20 0x7f07600b0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:49.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.178+0000 7f075f59e700 1 -- 172.21.15.17:0/3005912602 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07601527e0 con 0x7f0760152f20 2022-08-01T19:41:49.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.178+0000 7f075f59e700 1 --2- 172.21.15.17:0/3005912602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0760152f20 0x7f07600a8e60 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 rx=0x7f075800b000 tx=0x7f07580091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:49.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.178+0000 7f0757fff700 1 -- 172.21.15.17:0/3005912602 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0758013750 con 0x7f0760152f20 2022-08-01T19:41:49.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.178+0000 7f0757fff700 1 -- 172.21.15.17:0/3005912602 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07580138b0 con 0x7f0760152f20 2022-08-01T19:41:49.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.178+0000 7f0764a27700 1 -- 172.21.15.17:0/3005912602 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0760152a80 con 0x7f0760152f20 2022-08-01T19:41:49.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.178+0000 7f0757fff700 1 -- 172.21.15.17:0/3005912602 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0758013c10 con 0x7f0760152f20 2022-08-01T19:41:49.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.178+0000 7f0764a27700 1 -- 172.21.15.17:0/3005912602 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07600b83b0 con 0x7f0760152f20 2022-08-01T19:41:49.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.182+0000 7f0764a27700 1 -- 172.21.15.17:0/3005912602 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0760152c10 con 0x7f0760152f20 2022-08-01T19:41:49.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.182+0000 7f0757fff700 1 -- 172.21.15.17:0/3005912602 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0758013d70 con 0x7f0760152f20 2022-08-01T19:41:49.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.182+0000 7f0757fff700 1 --2- 172.21.15.17:0/3005912602 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0744053b00 0x7f0744055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:49.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.182+0000 7f0757fff700 1 -- 172.21.15.17:0/3005912602 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f075806c750 con 0x7f0760152f20 2022-08-01T19:41:49.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.182+0000 7f075ed9d700 1 --2- 172.21.15.17:0/3005912602 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0744053b00 0x7f0744055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:49.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.182+0000 7f075ed9d700 1 --2- 172.21.15.17:0/3005912602 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0744053b00 0x7f0744055fc0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f0748005000 tx=0x7f07600af4e0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:49.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.186+0000 7f0757fff700 1 -- 172.21.15.17:0/3005912602 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0758043510 con 0x7f0760152f20 2022-08-01T19:41:49.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.354+0000 7f0764a27700 1 -- 172.21.15.17:0/3005912602 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f07600af5f0 con 0x7f0760152f20 2022-08-01T19:41:49.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.362+0000 7f0757fff700 1 -- 172.21.15.17:0/3005912602 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v29) v1 ==== 184+0+0 (secure 0 0 0) 0x7f0758018070 con 0x7f0760152f20 2022-08-01T19:41:49.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.366+0000 7f0764a27700 1 -- 172.21.15.17:0/3005912602 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0744053b00 msgr2=0x7f0744055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:49.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.366+0000 7f0764a27700 1 --2- 172.21.15.17:0/3005912602 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0744053b00 0x7f0744055fc0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f0748005000 tx=0x7f07600af4e0).stop 2022-08-01T19:41:49.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.366+0000 7f0764a27700 1 -- 172.21.15.17:0/3005912602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0760152f20 msgr2=0x7f07600a8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:49.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.366+0000 7f0764a27700 1 --2- 172.21.15.17:0/3005912602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0760152f20 0x7f07600a8e60 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 rx=0x7f075800b000 tx=0x7f07580091d0).stop 2022-08-01T19:41:49.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.366+0000 7f0764a27700 1 -- 172.21.15.17:0/3005912602 shutdown_connections 2022-08-01T19:41:49.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.366+0000 7f0764a27700 1 --2- 172.21.15.17:0/3005912602 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0744053b00 0x7f0744055fc0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:49.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.366+0000 7f0764a27700 1 --2- 172.21.15.17:0/3005912602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0760152f20 0x7f07600a8e60 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:49.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.366+0000 7f0764a27700 1 --2- 172.21.15.17:0/3005912602 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f07601566b0 0x7f07600b7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:49.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.366+0000 7f0764a27700 1 --2- 172.21.15.17:0/3005912602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0740003e20 0x7f07600b0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:49.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.366+0000 7f0764a27700 1 -- 172.21.15.17:0/3005912602 >> 172.21.15.17:0/3005912602 conn(0x7f0760066f70 msgr2=0x7f07600b8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:49.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.366+0000 7f0764a27700 1 -- 172.21.15.17:0/3005912602 shutdown_connections 2022-08-01T19:41:49.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.366+0000 7f0764a27700 1 -- 172.21.15.17:0/3005912602 wait complete. 2022-08-01T19:41:49.377 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.0 2022-08-01T19:41:49.381 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T19:41:49.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.502+0000 7f2f87219700 1 Processor -- start 2022-08-01T19:41:49.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.502+0000 7f2f87219700 1 -- start start 2022-08-01T19:41:49.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.502+0000 7f2f87219700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2f800614b0 0x7f2f80061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:49.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.502+0000 7f2f87219700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f2f800ad750 0x7f2f800b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:49.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.502+0000 7f2f87219700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f800aee80 con 0x7f2f801405d0 2022-08-01T19:41:49.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.502+0000 7f2f87219700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f800b70b0 con 0x7f2f800614b0 2022-08-01T19:41:49.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.502+0000 7f2f87219700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f800af0b0 con 0x7f2f800ad750 2022-08-01T19:41:49.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.502+0000 7f2f86217700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2f800614b0 0x7f2f80061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:49.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.502+0000 7f2f86a18700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2f801405d0 0x7f2f801466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34924/0 (socket says 172.21.15.17:34924) 2022-08-01T19:41:49.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.502+0000 7f2f86a18700 1 -- 172.21.15.17:0/1248323033 learned_addr learned my addr 172.21.15.17:0/1248323033 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:49.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.502+0000 7f2f85a16700 1 --2- 172.21.15.17:0/1248323033 >> v2:172.21.15.77:3300/0 conn(0x7f2f800ad750 0x7f2f800b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:49.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.502+0000 7f2f86217700 1 -- 172.21.15.17:0/1248323033 >> v2:172.21.15.77:3300/0 conn(0x7f2f800ad750 msgr2=0x7f2f800b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:49.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.502+0000 7f2f86217700 1 --2- 172.21.15.17:0/1248323033 >> v2:172.21.15.77:3300/0 conn(0x7f2f800ad750 0x7f2f800b9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:49.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.502+0000 7f2f86217700 1 -- 172.21.15.17:0/1248323033 >> v1:172.21.15.17:6789/0 conn(0x7f2f801405d0 legacy=0x7f2f801466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:49.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.502+0000 7f2f86217700 1 -- 172.21.15.17:0/1248323033 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f800af350 con 0x7f2f800614b0 2022-08-01T19:41:49.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.502+0000 7f2f85a16700 1 --2- 172.21.15.17:0/1248323033 >> v2:172.21.15.77:3300/0 conn(0x7f2f800ad750 0x7f2f800b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:49.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.502+0000 7f2f86217700 1 --2- 172.21.15.17:0/1248323033 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f800614b0 0x7f2f80061890 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 rx=0x7f2f800aed30 tx=0x7f2f7c006df0).ready entity=mon.0 client_cookie=da6fbf4bd334a085 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:49.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.502+0000 7f2f85215700 1 -- 172.21.15.17:0/1248323033 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f7c00d5c0 con 0x7f2f800614b0 2022-08-01T19:41:49.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.502+0000 7f2f85215700 1 -- 172.21.15.17:0/1248323033 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f7c000e90 con 0x7f2f800614b0 2022-08-01T19:41:49.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.502+0000 7f2f85215700 1 -- 172.21.15.17:0/1248323033 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f7c00d910 con 0x7f2f800614b0 2022-08-01T19:41:49.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.506+0000 7f2f87219700 1 -- 172.21.15.17:0/1248323033 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f800614b0 msgr2=0x7f2f80061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:49.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.506+0000 7f2f87219700 1 --2- 172.21.15.17:0/1248323033 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f800614b0 0x7f2f80061890 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 rx=0x7f2f800aed30 tx=0x7f2f7c006df0).stop 2022-08-01T19:41:49.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.506+0000 7f2f87219700 1 -- 172.21.15.17:0/1248323033 shutdown_connections 2022-08-01T19:41:49.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.506+0000 7f2f87219700 1 --2- 172.21.15.17:0/1248323033 >> v2:172.21.15.77:3300/0 conn(0x7f2f800ad750 0x7f2f800b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:49.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.506+0000 7f2f87219700 1 --2- 172.21.15.17:0/1248323033 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f800614b0 0x7f2f80061890 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:49.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.506+0000 7f2f87219700 1 -- 172.21.15.17:0/1248323033 >> 172.21.15.17:0/1248323033 conn(0x7f2f80066f70 msgr2=0x7f2f80067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:49.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.506+0000 7f2f87219700 1 -- 172.21.15.17:0/1248323033 shutdown_connections 2022-08-01T19:41:49.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.506+0000 7f2f87219700 1 -- 172.21.15.17:0/1248323033 wait complete. 2022-08-01T19:41:49.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.506+0000 7f2f87219700 1 Processor -- start 2022-08-01T19:41:49.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.506+0000 7f2f87219700 1 -- start start 2022-08-01T19:41:49.530 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.510+0000 7f2f87219700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f800614b0 0x7f2f800b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:49.531 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.510+0000 7f2f87219700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f800ad750 0x7f2f800a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:49.532 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.510+0000 7f2f87219700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2f801405d0 0x7f2f80155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:49.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.510+0000 7f2f87219700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f800b2de0 con 0x7f2f800ad750 2022-08-01T19:41:49.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.510+0000 7f2f87219700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f80153050 con 0x7f2f800614b0 2022-08-01T19:41:49.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.510+0000 7f2f87219700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f800a0e50 con 0x7f2f801405d0 2022-08-01T19:41:49.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.510+0000 7f2f85a16700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f800ad750 0x7f2f800a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:49.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.510+0000 7f2f85a16700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f800ad750 0x7f2f800a12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42736/0 (socket says 172.21.15.17:42736) 2022-08-01T19:41:49.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.510+0000 7f2f85a16700 1 -- 172.21.15.17:0/1568901972 learned_addr learned my addr 172.21.15.17:0/1568901972 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:49.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.510+0000 7f2f86217700 1 --2- 172.21.15.17:0/1568901972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f800614b0 0x7f2f800b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:49.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.510+0000 7f2f86a18700 1 --2- 172.21.15.17:0/1568901972 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2f801405d0 0x7f2f80155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:49.538 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.510+0000 7f2f85a16700 1 -- 172.21.15.17:0/1568901972 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2f801405d0 msgr2=0x7f2f80155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:49.540 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.510+0000 7f2f85a16700 1 --2- 172.21.15.17:0/1568901972 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2f801405d0 0x7f2f80155cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:49.540 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.510+0000 7f2f85a16700 1 -- 172.21.15.17:0/1568901972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f800614b0 msgr2=0x7f2f800b31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:49.540 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.510+0000 7f2f85a16700 1 --2- 172.21.15.17:0/1568901972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f800614b0 0x7f2f800b31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:49.540 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.510+0000 7f2f85a16700 1 -- 172.21.15.17:0/1568901972 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f7c00b180 con 0x7f2f800ad750 2022-08-01T19:41:49.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.510+0000 7f2f85a16700 1 --2- 172.21.15.17:0/1568901972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f800ad750 0x7f2f800a12b0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 rx=0x7f2f7000a000 tx=0x7f2f70008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:49.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.510+0000 7f2f6f7fe700 1 -- 172.21.15.17:0/1568901972 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f70011780 con 0x7f2f800ad750 2022-08-01T19:41:49.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.510+0000 7f2f6f7fe700 1 -- 172.21.15.17:0/1568901972 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f700118e0 con 0x7f2f800ad750 2022-08-01T19:41:49.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.510+0000 7f2f87219700 1 -- 172.21.15.17:0/1568901972 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f800b2b40 con 0x7f2f800ad750 2022-08-01T19:41:49.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.510+0000 7f2f6f7fe700 1 -- 172.21.15.17:0/1568901972 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f70011c40 con 0x7f2f800ad750 2022-08-01T19:41:49.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.510+0000 7f2f87219700 1 -- 172.21.15.17:0/1568901972 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f800b36e0 con 0x7f2f800ad750 2022-08-01T19:41:49.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.514+0000 7f2f87219700 1 -- 172.21.15.17:0/1568901972 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f800b28e0 con 0x7f2f800ad750 2022-08-01T19:41:49.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.514+0000 7f2f6f7fe700 1 -- 172.21.15.17:0/1568901972 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2f70012420 con 0x7f2f800ad750 2022-08-01T19:41:49.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.514+0000 7f2f6f7fe700 1 --2- 172.21.15.17:0/1568901972 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2f60053950 0x7f2f60055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:49.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.514+0000 7f2f6f7fe700 1 -- 172.21.15.17:0/1568901972 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f2f7006bea0 con 0x7f2f800ad750 2022-08-01T19:41:49.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.514+0000 7f2f86217700 1 --2- 172.21.15.17:0/1568901972 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2f60053950 0x7f2f60055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:49.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.514+0000 7f2f86217700 1 --2- 172.21.15.17:0/1568901972 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2f60053950 0x7f2f60055e10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f2f7c019000 tx=0x7f2f7c012000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:49.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.518+0000 7f2f6f7fe700 1 -- 172.21.15.17:0/1568901972 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2f70042af0 con 0x7f2f800ad750 2022-08-01T19:41:49.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.686+0000 7f2f87219700 1 -- 172.21.15.17:0/1568901972 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f2f800af5f0 con 0x7f2f800ad750 2022-08-01T19:41:49.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.694+0000 7f2f6f7fe700 1 -- 172.21.15.17:0/1568901972 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v30) v1 ==== 184+0+0 (secure 0 0 0) 0x7f2f70046110 con 0x7f2f800ad750 2022-08-01T19:41:49.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.698+0000 7f2f87219700 1 -- 172.21.15.17:0/1568901972 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2f60053950 msgr2=0x7f2f60055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:49.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.698+0000 7f2f87219700 1 --2- 172.21.15.17:0/1568901972 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2f60053950 0x7f2f60055e10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f2f7c019000 tx=0x7f2f7c012000).stop 2022-08-01T19:41:49.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.698+0000 7f2f87219700 1 -- 172.21.15.17:0/1568901972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f800ad750 msgr2=0x7f2f800a12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:49.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.698+0000 7f2f87219700 1 --2- 172.21.15.17:0/1568901972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f800ad750 0x7f2f800a12b0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 rx=0x7f2f7000a000 tx=0x7f2f70008040).stop 2022-08-01T19:41:49.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.698+0000 7f2f87219700 1 -- 172.21.15.17:0/1568901972 shutdown_connections 2022-08-01T19:41:49.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.698+0000 7f2f87219700 1 --2- 172.21.15.17:0/1568901972 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2f60053950 0x7f2f60055e10 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:49.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.698+0000 7f2f87219700 1 --2- 172.21.15.17:0/1568901972 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2f801405d0 0x7f2f80155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:49.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.698+0000 7f2f87219700 1 --2- 172.21.15.17:0/1568901972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f800ad750 0x7f2f800a12b0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:49.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.698+0000 7f2f87219700 1 --2- 172.21.15.17:0/1568901972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f800614b0 0x7f2f800b31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:49.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.698+0000 7f2f87219700 1 -- 172.21.15.17:0/1568901972 >> 172.21.15.17:0/1568901972 conn(0x7f2f80066f70 msgr2=0x7f2f800b8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:49.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.698+0000 7f2f87219700 1 -- 172.21.15.17:0/1568901972 shutdown_connections 2022-08-01T19:41:49.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.698+0000 7f2f87219700 1 -- 172.21.15.17:0/1568901972 wait complete. 2022-08-01T19:41:49.709 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.1 2022-08-01T19:41:49.723 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:41:49.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.846+0000 7fdd5da6e700 1 Processor -- start 2022-08-01T19:41:49.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.846+0000 7fdd5da6e700 1 -- start start 2022-08-01T19:41:49.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.846+0000 7fdd5da6e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fdd580b1f40 0x7fdd580b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:49.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.846+0000 7fdd5da6e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdd58148530 0x7fdd580b2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:49.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.846+0000 7fdd5da6e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd580b04e0 con 0x7fdd580a7250 2022-08-01T19:41:49.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.846+0000 7fdd5da6e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd580a9250 con 0x7fdd580b1f40 2022-08-01T19:41:49.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.846+0000 7fdd5da6e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd580b0780 con 0x7fdd58148530 2022-08-01T19:41:49.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.846+0000 7fdd56ffd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdd58148530 0x7fdd580b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:49.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.846+0000 7fdd577fe700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fdd580b1f40 0x7fdd580b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:49.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.846+0000 7fdd56ffd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdd58148530 0x7fdd580b2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:38844/0 (socket says 172.21.15.17:38844) 2022-08-01T19:41:49.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.846+0000 7fdd56ffd700 1 -- 172.21.15.17:0/691568376 learned_addr learned my addr 172.21.15.17:0/691568376 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:49.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.846+0000 7fdd56ffd700 1 -- 172.21.15.17:0/691568376 >> v2:172.21.15.77:3300/0 conn(0x7fdd580b1f40 msgr2=0x7fdd580b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:49.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.846+0000 7fdd56ffd700 1 --2- 172.21.15.17:0/691568376 >> v2:172.21.15.77:3300/0 conn(0x7fdd580b1f40 0x7fdd580b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:49.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.846+0000 7fdd56ffd700 1 -- 172.21.15.17:0/691568376 >> v1:172.21.15.77:6789/0 conn(0x7fdd580a7250 legacy=0x7fdd581448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:49.858 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.846+0000 7fdd56ffd700 1 -- 172.21.15.17:0/691568376 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd580b0a20 con 0x7fdd58148530 2022-08-01T19:41:49.858 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.846+0000 7fdd56ffd700 1 --2- 172.21.15.17:0/691568376 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdd58148530 0x7fdd580b2320 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fdd500032f0 tx=0x7fdd50009290).ready entity=mon.2 client_cookie=17d1a6242a9f0e23 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:49.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.846+0000 7fdd567fc700 1 -- 172.21.15.17:0/691568376 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd5000b8a0 con 0x7fdd58148530 2022-08-01T19:41:49.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.846+0000 7fdd567fc700 1 -- 172.21.15.17:0/691568376 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5000ba00 con 0x7fdd58148530 2022-08-01T19:41:49.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.846+0000 7fdd567fc700 1 -- 172.21.15.17:0/691568376 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd5000bb70 con 0x7fdd58148530 2022-08-01T19:41:49.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.846+0000 7fdd5da6e700 1 -- 172.21.15.17:0/691568376 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdd58148530 msgr2=0x7fdd580b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:49.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.846+0000 7fdd5da6e700 1 --2- 172.21.15.17:0/691568376 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdd58148530 0x7fdd580b2320 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fdd500032f0 tx=0x7fdd50009290).stop 2022-08-01T19:41:49.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.846+0000 7fdd5da6e700 1 -- 172.21.15.17:0/691568376 shutdown_connections 2022-08-01T19:41:49.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.846+0000 7fdd5da6e700 1 --2- 172.21.15.17:0/691568376 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdd58148530 0x7fdd580b2320 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:49.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.846+0000 7fdd5da6e700 1 --2- 172.21.15.17:0/691568376 >> v2:172.21.15.77:3300/0 conn(0x7fdd580b1f40 0x7fdd580b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:49.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.846+0000 7fdd5da6e700 1 -- 172.21.15.17:0/691568376 >> 172.21.15.17:0/691568376 conn(0x7fdd5805f840 msgr2=0x7fdd5805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:49.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.846+0000 7fdd5da6e700 1 -- 172.21.15.17:0/691568376 shutdown_connections 2022-08-01T19:41:49.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.846+0000 7fdd5da6e700 1 -- 172.21.15.17:0/691568376 wait complete. 2022-08-01T19:41:49.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.850+0000 7fdd5da6e700 1 Processor -- start 2022-08-01T19:41:49.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.850+0000 7fdd5da6e700 1 -- start start 2022-08-01T19:41:49.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.850+0000 7fdd5da6e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdd580a7250 0x7fdd5814ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:49.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.850+0000 7fdd5da6e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd580b1f40 0x7fdd58159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:49.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.850+0000 7fdd5da6e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd58148530 0x7fdd5815a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:49.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.850+0000 7fdd5da6e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd580aa5c0 con 0x7fdd58148530 2022-08-01T19:41:49.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.850+0000 7fdd5da6e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd580ab120 con 0x7fdd580b1f40 2022-08-01T19:41:49.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.850+0000 7fdd5da6e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd580ab350 con 0x7fdd580a7250 2022-08-01T19:41:49.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.850+0000 7fdd577fe700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdd580a7250 0x7fdd5814ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:49.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.850+0000 7fdd57fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd58148530 0x7fdd5815a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:49.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.850+0000 7fdd577fe700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdd580a7250 0x7fdd5814ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:38848/0 (socket says 172.21.15.17:38848) 2022-08-01T19:41:49.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.850+0000 7fdd56ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd580b1f40 0x7fdd58159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:49.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.850+0000 7fdd577fe700 1 -- 172.21.15.17:0/1743264017 learned_addr learned my addr 172.21.15.17:0/1743264017 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:49.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.850+0000 7fdd56ffd700 1 -- 172.21.15.17:0/1743264017 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdd580a7250 msgr2=0x7fdd5814ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:49.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.850+0000 7fdd56ffd700 1 --2- 172.21.15.17:0/1743264017 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdd580a7250 0x7fdd5814ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:49.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.850+0000 7fdd56ffd700 1 -- 172.21.15.17:0/1743264017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd58148530 msgr2=0x7fdd5815a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:49.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.850+0000 7fdd56ffd700 1 --2- 172.21.15.17:0/1743264017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd58148530 0x7fdd5815a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:49.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.850+0000 7fdd56ffd700 1 -- 172.21.15.17:0/1743264017 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd50009530 con 0x7fdd580b1f40 2022-08-01T19:41:49.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.850+0000 7fdd57fff700 1 --2- 172.21.15.17:0/1743264017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd58148530 0x7fdd5815a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:49.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.854+0000 7fdd56ffd700 1 --2- 172.21.15.17:0/1743264017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd580b1f40 0x7fdd58159f90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7fdd500032f0 tx=0x7fdd50016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:49.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.854+0000 7fdd54ff9700 1 -- 172.21.15.17:0/1743264017 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd5000b8a0 con 0x7fdd580b1f40 2022-08-01T19:41:49.899 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.854+0000 7fdd54ff9700 1 -- 172.21.15.17:0/1743264017 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd50021440 con 0x7fdd580b1f40 2022-08-01T19:41:49.899 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.854+0000 7fdd5da6e700 1 -- 172.21.15.17:0/1743264017 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd580ab5f0 con 0x7fdd580b1f40 2022-08-01T19:41:49.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.854+0000 7fdd54ff9700 1 -- 172.21.15.17:0/1743264017 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd500217a0 con 0x7fdd580b1f40 2022-08-01T19:41:49.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.854+0000 7fdd5da6e700 1 -- 172.21.15.17:0/1743264017 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5814b550 con 0x7fdd580b1f40 2022-08-01T19:41:49.901 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.854+0000 7fdd5da6e700 1 -- 172.21.15.17:0/1743264017 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd580b0cc0 con 0x7fdd580b1f40 2022-08-01T19:41:49.901 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.854+0000 7fdd54ff9700 1 -- 172.21.15.17:0/1743264017 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdd50027020 con 0x7fdd580b1f40 2022-08-01T19:41:49.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.854+0000 7fdd54ff9700 1 --2- 172.21.15.17:0/1743264017 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdd3c0539e0 0x7fdd3c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:49.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.854+0000 7fdd54ff9700 1 -- 172.21.15.17:0/1743264017 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fdd5006d540 con 0x7fdd580b1f40 2022-08-01T19:41:49.907 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.854+0000 7fdd577fe700 1 --2- 172.21.15.17:0/1743264017 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdd3c0539e0 0x7fdd3c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:49.907 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.858+0000 7fdd577fe700 1 --2- 172.21.15.17:0/1743264017 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdd3c0539e0 0x7fdd3c055ea0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7fdd4000b000 tx=0x7fdd40006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:49.907 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:49.862+0000 7fdd54ff9700 1 -- 172.21.15.17:0/1743264017 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdd50044260 con 0x7fdd580b1f40 2022-08-01T19:41:50.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.026+0000 7fdd5da6e700 1 -- 172.21.15.17:0/1743264017 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd580574e0 con 0x7fdd580b1f40 2022-08-01T19:41:50.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.026+0000 7fdd54ff9700 1 -- 172.21.15.17:0/1743264017 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+2490 (secure 0 0 0) 0x7fdd50015020 con 0x7fdd580b1f40 2022-08-01T19:41:50.031 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:50.032 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":53,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7252,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/1455566688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1455566688},{"type":"v1","addr":"172.21.15.77:6835","nonce":1455566688}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":51},{"gid":7266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.17:6833/249859043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":249859043},{"type":"v1","addr":"172.21.15.17:6833","nonce":249859043}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":51}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:41:47.422244+0000","modified":"2022-08-01T19:41:48.425195+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7272":{"gid":7272,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/2991628239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2991628239},{"type":"v1","addr":"172.21.15.17:6835","nonce":2991628239}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2022-08-01T19:41:50.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.030+0000 7fdd5da6e700 1 -- 172.21.15.17:0/1743264017 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdd3c0539e0 msgr2=0x7fdd3c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:50.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.030+0000 7fdd5da6e700 1 --2- 172.21.15.17:0/1743264017 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdd3c0539e0 0x7fdd3c055ea0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7fdd4000b000 tx=0x7fdd40006df0).stop 2022-08-01T19:41:50.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.030+0000 7fdd5da6e700 1 -- 172.21.15.17:0/1743264017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd580b1f40 msgr2=0x7fdd58159f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:50.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.030+0000 7fdd5da6e700 1 --2- 172.21.15.17:0/1743264017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd580b1f40 0x7fdd58159f90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7fdd500032f0 tx=0x7fdd50016000).stop 2022-08-01T19:41:50.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.030+0000 7fdd5da6e700 1 -- 172.21.15.17:0/1743264017 shutdown_connections 2022-08-01T19:41:50.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.030+0000 7fdd5da6e700 1 --2- 172.21.15.17:0/1743264017 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdd3c0539e0 0x7fdd3c055ea0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:50.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.030+0000 7fdd5da6e700 1 --2- 172.21.15.17:0/1743264017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd58148530 0x7fdd5815a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:50.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.030+0000 7fdd5da6e700 1 --2- 172.21.15.17:0/1743264017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd580b1f40 0x7fdd58159f90 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:50.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.030+0000 7fdd5da6e700 1 --2- 172.21.15.17:0/1743264017 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdd580a7250 0x7fdd5814ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:50.040 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.030+0000 7fdd5da6e700 1 -- 172.21.15.17:0/1743264017 >> 172.21.15.17:0/1743264017 conn(0x7fdd5805f840 msgr2=0x7fdd580b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:50.040 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.030+0000 7fdd5da6e700 1 -- 172.21.15.17:0/1743264017 shutdown_connections 2022-08-01T19:41:50.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.030+0000 7fdd5da6e700 1 -- 172.21.15.17:0/1743264017 wait complete. 2022-08-01T19:41:50.041 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 53 2022-08-01T19:41:50.360 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 53, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:41:47.422244+0000', 'modified': '2022-08-01T19:41:48.425195+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7272}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7272': {'gid': 7272, 'name': 'c', 'rank': 0, 'incarnation': 52, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/2991628239', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 2991628239}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 2991628239}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:41:50.360 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:41:50.361 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T19:41:50.480 INFO:teuthology.orchestra.run.smithi017.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":7272,"want_state":"up:active","state":"up:active","rank_uptime":3.0436295360000001,"mdsmap_epoch":53,"osdmap_epoch":107,"osdmap_epoch_barrier":107,"uptime":7.4185805670000002} 2022-08-01T19:41:50.480 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 7272, "mdsmap_epoch": 53, "osdmap_epoch": 107, "osdmap_epoch_barrier": 107, "rank_uptime": 3.043629536, "state": "up:active", "uptime": 7.418580567, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:41:50.481 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:41:50.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.590+0000 7f6ff4773700 1 Processor -- start 2022-08-01T19:41:50.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.590+0000 7f6ff4773700 1 -- start start 2022-08-01T19:41:50.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.590+0000 7f6ff4773700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6fec0b1f40 0x7f6fec0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:50.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.590+0000 7f6ff4773700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f6fec0a7250 0x7f6fec1448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:50.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.590+0000 7f6ff4773700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fec0b04e0 con 0x7f6fec148530 2022-08-01T19:41:50.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.590+0000 7f6ff4773700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fec0a9250 con 0x7f6fec0b1f40 2022-08-01T19:41:50.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.590+0000 7f6ff4773700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fec0b0780 con 0x7f6fec0a7250 2022-08-01T19:41:50.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.590+0000 7f6ff250f700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6fec0b1f40 0x7f6fec0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:50.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.594+0000 7f6ff250f700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6fec0b1f40 0x7f6fec0b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42758/0 (socket says 172.21.15.17:42758) 2022-08-01T19:41:50.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.594+0000 7f6ff250f700 1 -- 172.21.15.17:0/2492332166 learned_addr learned my addr 172.21.15.17:0/2492332166 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:50.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.594+0000 7f6ff2d10700 1 --2- 172.21.15.17:0/2492332166 >> v2:172.21.15.77:3300/0 conn(0x7f6fec0a7250 0x7f6fec1448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:50.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.594+0000 7f6ff250f700 1 -- 172.21.15.17:0/2492332166 >> v2:172.21.15.77:3300/0 conn(0x7f6fec0a7250 msgr2=0x7f6fec1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:50.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.594+0000 7f6ff250f700 1 --2- 172.21.15.17:0/2492332166 >> v2:172.21.15.77:3300/0 conn(0x7f6fec0a7250 0x7f6fec1448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:50.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.594+0000 7f6ff250f700 1 -- 172.21.15.17:0/2492332166 >> v1:172.21.15.77:6789/0 conn(0x7f6fec148530 legacy=0x7f6fec0b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:50.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.594+0000 7f6ff250f700 1 -- 172.21.15.17:0/2492332166 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fec0b0a20 con 0x7f6fec0b1f40 2022-08-01T19:41:50.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.594+0000 7f6ff250f700 1 --2- 172.21.15.17:0/2492332166 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fec0b1f40 0x7f6fec0b1160 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 rx=0x7f6fec1478c0 tx=0x7f6fdc006eb0).ready entity=mon.0 client_cookie=faf4d23052576713 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:50.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.594+0000 7f6ff150d700 1 -- 172.21.15.17:0/2492332166 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fdc00d610 con 0x7f6fec0b1f40 2022-08-01T19:41:50.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.594+0000 7f6ff150d700 1 -- 172.21.15.17:0/2492332166 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fdc000e90 con 0x7f6fec0b1f40 2022-08-01T19:41:50.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.594+0000 7f6ff150d700 1 -- 172.21.15.17:0/2492332166 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fdc00d9a0 con 0x7f6fec0b1f40 2022-08-01T19:41:50.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.594+0000 7f6ff4773700 1 -- 172.21.15.17:0/2492332166 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fec0b1f40 msgr2=0x7f6fec0b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:50.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.594+0000 7f6ff4773700 1 --2- 172.21.15.17:0/2492332166 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fec0b1f40 0x7f6fec0b1160 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 rx=0x7f6fec1478c0 tx=0x7f6fdc006eb0).stop 2022-08-01T19:41:50.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.594+0000 7f6ff4773700 1 -- 172.21.15.17:0/2492332166 shutdown_connections 2022-08-01T19:41:50.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.594+0000 7f6ff4773700 1 --2- 172.21.15.17:0/2492332166 >> v2:172.21.15.77:3300/0 conn(0x7f6fec0a7250 0x7f6fec1448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:50.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.594+0000 7f6ff4773700 1 --2- 172.21.15.17:0/2492332166 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fec0b1f40 0x7f6fec0b1160 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:50.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.594+0000 7f6ff4773700 1 -- 172.21.15.17:0/2492332166 >> 172.21.15.17:0/2492332166 conn(0x7f6fec05f840 msgr2=0x7f6fec05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:50.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.594+0000 7f6ff4773700 1 -- 172.21.15.17:0/2492332166 shutdown_connections 2022-08-01T19:41:50.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.594+0000 7f6ff4773700 1 -- 172.21.15.17:0/2492332166 wait complete. 2022-08-01T19:41:50.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.598+0000 7f6ff4773700 1 Processor -- start 2022-08-01T19:41:50.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.598+0000 7f6ff4773700 1 -- start start 2022-08-01T19:41:50.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.598+0000 7f6ff4773700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6fec0a7250 0x7f6fec14a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:50.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.598+0000 7f6ff4773700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fec0b1f40 0x7f6fec1439d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:50.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.598+0000 7f6ff4773700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fec148530 0x7f6fec143ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:50.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.598+0000 7f6ff4773700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec0aa270 con 0x7f6fec0b1f40 2022-08-01T19:41:50.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.598+0000 7f6ff4773700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec0aaa30 con 0x7f6fec148530 2022-08-01T19:41:50.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.598+0000 7f6ff4773700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec14a9c0 con 0x7f6fec0a7250 2022-08-01T19:41:50.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.598+0000 7f6ff1d0e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fec0b1f40 0x7f6fec1439d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:50.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.598+0000 7f6ff250f700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6fec0a7250 0x7f6fec14a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:50.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.598+0000 7f6ff1d0e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fec0b1f40 0x7f6fec1439d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42764/0 (socket says 172.21.15.17:42764) 2022-08-01T19:41:50.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.598+0000 7f6ff2d10700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fec148530 0x7f6fec143ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:50.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.598+0000 7f6ff1d0e700 1 -- 172.21.15.17:0/2090350290 learned_addr learned my addr 172.21.15.17:0/2090350290 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:50.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.598+0000 7f6ff1d0e700 1 -- 172.21.15.17:0/2090350290 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6fec0a7250 msgr2=0x7f6fec14a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:50.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.598+0000 7f6ff1d0e700 1 --2- 172.21.15.17:0/2090350290 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6fec0a7250 0x7f6fec14a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:50.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.598+0000 7f6ff1d0e700 1 -- 172.21.15.17:0/2090350290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fec148530 msgr2=0x7f6fec143ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:50.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.598+0000 7f6ff1d0e700 1 --2- 172.21.15.17:0/2090350290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fec148530 0x7f6fec143ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:50.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.598+0000 7f6ff1d0e700 1 -- 172.21.15.17:0/2090350290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fdc00b1e0 con 0x7f6fec0b1f40 2022-08-01T19:41:50.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.598+0000 7f6ff1d0e700 1 --2- 172.21.15.17:0/2090350290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fec0b1f40 0x7f6fec1439d0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 rx=0x7f6fec0acff0 tx=0x7f6fe8001400).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:50.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.598+0000 7f6fdb7fe700 1 -- 172.21.15.17:0/2090350290 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fe800f470 con 0x7f6fec0b1f40 2022-08-01T19:41:50.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.598+0000 7f6fdb7fe700 1 -- 172.21.15.17:0/2090350290 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe800f5d0 con 0x7f6fec0b1f40 2022-08-01T19:41:50.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.598+0000 7f6ff4773700 1 -- 172.21.15.17:0/2090350290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fec0aaed0 con 0x7f6fec0b1f40 2022-08-01T19:41:50.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.598+0000 7f6fdb7fe700 1 -- 172.21.15.17:0/2090350290 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fe800f930 con 0x7f6fec0b1f40 2022-08-01T19:41:50.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.602+0000 7f6ff4773700 1 -- 172.21.15.17:0/2090350290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fec0a9e00 con 0x7f6fec0b1f40 2022-08-01T19:41:50.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.602+0000 7f6ff4773700 1 -- 172.21.15.17:0/2090350290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fec0aa430 con 0x7f6fec0b1f40 2022-08-01T19:41:50.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.602+0000 7f6fdb7fe700 1 -- 172.21.15.17:0/2090350290 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6fe8019030 con 0x7f6fec0b1f40 2022-08-01T19:41:50.636 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.602+0000 7f6fdb7fe700 1 --2- 172.21.15.17:0/2090350290 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6fcc053a10 0x7f6fcc055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:50.636 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.602+0000 7f6fdb7fe700 1 -- 172.21.15.17:0/2090350290 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6fe806b7c0 con 0x7f6fec0b1f40 2022-08-01T19:41:50.636 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.602+0000 7f6ff250f700 1 --2- 172.21.15.17:0/2090350290 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6fcc053a10 0x7f6fcc055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:50.636 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.602+0000 7f6ff250f700 1 --2- 172.21.15.17:0/2090350290 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6fcc053a10 0x7f6fcc055ed0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f6fdc01c000 tx=0x7f6fdc015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:50.637 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.606+0000 7f6fdb7fe700 1 -- 172.21.15.17:0/2090350290 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6fe8042470 con 0x7f6fec0b1f40 2022-08-01T19:41:50.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.734+0000 7f6ff4773700 1 -- 172.21.15.17:0/2090350290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6fec0b0cc0 con 0x7f6fec0b1f40 2022-08-01T19:41:50.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.734+0000 7f6fdb7fe700 1 -- 172.21.15.17:0/2090350290 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f6fe8045a90 con 0x7f6fec0b1f40 2022-08-01T19:41:50.737 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:50.737 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":107,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:41:47.421949+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:41:45.865183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2022-08-01T19:41:46.670831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:41:50.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.738+0000 7f6ff4773700 1 -- 172.21.15.17:0/2090350290 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6fcc053a10 msgr2=0x7f6fcc055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:50.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.738+0000 7f6ff4773700 1 --2- 172.21.15.17:0/2090350290 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6fcc053a10 0x7f6fcc055ed0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f6fdc01c000 tx=0x7f6fdc015000).stop 2022-08-01T19:41:50.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.738+0000 7f6ff4773700 1 -- 172.21.15.17:0/2090350290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fec0b1f40 msgr2=0x7f6fec1439d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:50.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.738+0000 7f6ff4773700 1 --2- 172.21.15.17:0/2090350290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fec0b1f40 0x7f6fec1439d0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 rx=0x7f6fec0acff0 tx=0x7f6fe8001400).stop 2022-08-01T19:41:50.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.738+0000 7f6ff4773700 1 -- 172.21.15.17:0/2090350290 shutdown_connections 2022-08-01T19:41:50.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.738+0000 7f6ff4773700 1 --2- 172.21.15.17:0/2090350290 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6fcc053a10 0x7f6fcc055ed0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:50.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.738+0000 7f6ff4773700 1 --2- 172.21.15.17:0/2090350290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fec148530 0x7f6fec143ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:50.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.738+0000 7f6ff4773700 1 --2- 172.21.15.17:0/2090350290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fec0b1f40 0x7f6fec1439d0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:50.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.738+0000 7f6ff4773700 1 --2- 172.21.15.17:0/2090350290 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6fec0a7250 0x7f6fec14a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:50.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.738+0000 7f6ff4773700 1 -- 172.21.15.17:0/2090350290 >> 172.21.15.17:0/2090350290 conn(0x7f6fec05f840 msgr2=0x7f6fec1449c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:50.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.738+0000 7f6ff4773700 1 -- 172.21.15.17:0/2090350290 shutdown_connections 2022-08-01T19:41:50.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.738+0000 7f6ff4773700 1 -- 172.21.15.17:0/2090350290 wait complete. 2022-08-01T19:41:50.751 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T19:41:45.865183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T19:41:46.670831+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T19:41:50.753 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:41:50.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.866+0000 7f71e2d63700 1 Processor -- start 2022-08-01T19:41:50.870 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.866+0000 7f71e2d63700 1 -- start start 2022-08-01T19:41:50.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.870+0000 7f71e2d63700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f71dc066780 0x7f71dc066b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:50.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.870+0000 7f71e2d63700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f71dc0b1bc0 0x7f71dc05ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:50.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.870+0000 7f71e2d63700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71dc05f8e0 con 0x7f71dc140e40 2022-08-01T19:41:50.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.870+0000 7f71e2d63700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71dc0b1630 con 0x7f71dc066780 2022-08-01T19:41:50.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.870+0000 7f71e2d63700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71dc05fb10 con 0x7f71dc0b1bc0 2022-08-01T19:41:50.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.870+0000 7f71e1d61700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f71dc066780 0x7f71dc066b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:50.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.870+0000 7f71e1d61700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f71dc066780 0x7f71dc066b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42770/0 (socket says 172.21.15.17:42770) 2022-08-01T19:41:50.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.870+0000 7f71e1d61700 1 -- 172.21.15.17:0/3258234036 learned_addr learned my addr 172.21.15.17:0/3258234036 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:50.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.870+0000 7f71e1560700 1 --2- 172.21.15.17:0/3258234036 >> v2:172.21.15.77:3300/0 conn(0x7f71dc0b1bc0 0x7f71dc05ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:50.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.870+0000 7f71e1d61700 1 -- 172.21.15.17:0/3258234036 >> v2:172.21.15.77:3300/0 conn(0x7f71dc0b1bc0 msgr2=0x7f71dc05ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:50.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.870+0000 7f71e1d61700 1 --2- 172.21.15.17:0/3258234036 >> v2:172.21.15.77:3300/0 conn(0x7f71dc0b1bc0 0x7f71dc05ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:50.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.870+0000 7f71e1d61700 1 -- 172.21.15.17:0/3258234036 >> v1:172.21.15.77:6789/0 conn(0x7f71dc140e40 legacy=0x7f71dc13e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:50.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.870+0000 7f71e1d61700 1 -- 172.21.15.17:0/3258234036 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71dc0604a0 con 0x7f71dc066780 2022-08-01T19:41:50.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.870+0000 7f71e1d61700 1 --2- 172.21.15.17:0/3258234036 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71dc066780 0x7f71dc066b60 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 rx=0x7f71d8002f70 tx=0x7f71d8006eb0).ready entity=mon.0 client_cookie=c9ec79a3e09d7505 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:50.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.870+0000 7f71e0d5f700 1 -- 172.21.15.17:0/3258234036 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71d800d5c0 con 0x7f71dc066780 2022-08-01T19:41:50.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.870+0000 7f71e0d5f700 1 -- 172.21.15.17:0/3258234036 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d8000e90 con 0x7f71dc066780 2022-08-01T19:41:50.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.870+0000 7f71e0d5f700 1 -- 172.21.15.17:0/3258234036 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71d800d950 con 0x7f71dc066780 2022-08-01T19:41:50.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.870+0000 7f71e2d63700 1 -- 172.21.15.17:0/3258234036 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71dc066780 msgr2=0x7f71dc066b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:50.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.870+0000 7f71e2d63700 1 --2- 172.21.15.17:0/3258234036 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71dc066780 0x7f71dc066b60 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 rx=0x7f71d8002f70 tx=0x7f71d8006eb0).stop 2022-08-01T19:41:50.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.870+0000 7f71e2d63700 1 -- 172.21.15.17:0/3258234036 shutdown_connections 2022-08-01T19:41:50.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.870+0000 7f71e2d63700 1 --2- 172.21.15.17:0/3258234036 >> v2:172.21.15.77:3300/0 conn(0x7f71dc0b1bc0 0x7f71dc05ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:50.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.870+0000 7f71e2d63700 1 --2- 172.21.15.17:0/3258234036 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71dc066780 0x7f71dc066b60 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:50.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.870+0000 7f71e2d63700 1 -- 172.21.15.17:0/3258234036 >> 172.21.15.17:0/3258234036 conn(0x7f71dc063870 msgr2=0x7f71dc063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:50.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.870+0000 7f71e2d63700 1 -- 172.21.15.17:0/3258234036 shutdown_connections 2022-08-01T19:41:50.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.870+0000 7f71e2d63700 1 -- 172.21.15.17:0/3258234036 wait complete. 2022-08-01T19:41:50.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.874+0000 7f71e2d63700 1 Processor -- start 2022-08-01T19:41:50.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.874+0000 7f71e2d63700 1 -- start start 2022-08-01T19:41:50.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.874+0000 7f71e2d63700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71dc066780 0x7f71dc0b2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:50.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.878+0000 7f71e2d63700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71dc0b1bc0 0x7f71dc1502e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:50.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.878+0000 7f71e2d63700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71dc140e40 0x7f71dc1507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:50.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.878+0000 7f71e2d63700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc1538b0 con 0x7f71dc066780 2022-08-01T19:41:50.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.878+0000 7f71e2d63700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc153b50 con 0x7f71dc140e40 2022-08-01T19:41:50.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.878+0000 7f71e2d63700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc153df0 con 0x7f71dc0b1bc0 2022-08-01T19:41:50.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.878+0000 7f71e1d61700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71dc066780 0x7f71dc0b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:50.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.878+0000 7f71e2562700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71dc140e40 0x7f71dc1507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:50.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.878+0000 7f71e1d61700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71dc066780 0x7f71dc0b2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42776/0 (socket says 172.21.15.17:42776) 2022-08-01T19:41:50.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.878+0000 7f71e1d61700 1 -- 172.21.15.17:0/1455449568 learned_addr learned my addr 172.21.15.17:0/1455449568 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:50.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.878+0000 7f71e1560700 1 --2- 172.21.15.17:0/1455449568 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71dc0b1bc0 0x7f71dc1502e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:50.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.878+0000 7f71e1d61700 1 -- 172.21.15.17:0/1455449568 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71dc0b1bc0 msgr2=0x7f71dc1502e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:50.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.878+0000 7f71e1d61700 1 --2- 172.21.15.17:0/1455449568 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71dc0b1bc0 0x7f71dc1502e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:50.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.878+0000 7f71e1d61700 1 -- 172.21.15.17:0/1455449568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71dc140e40 msgr2=0x7f71dc1507d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:50.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.878+0000 7f71e1d61700 1 --2- 172.21.15.17:0/1455449568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71dc140e40 0x7f71dc1507d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:50.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.878+0000 7f71e1d61700 1 -- 172.21.15.17:0/1455449568 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d800b1e0 con 0x7f71dc066780 2022-08-01T19:41:50.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.878+0000 7f71e2562700 1 --2- 172.21.15.17:0/1455449568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71dc140e40 0x7f71dc1507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:50.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.878+0000 7f71e1560700 1 --2- 172.21.15.17:0/1455449568 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71dc0b1bc0 0x7f71dc1502e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:50.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.878+0000 7f71e1d61700 1 --2- 172.21.15.17:0/1455449568 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71dc066780 0x7f71dc0b2030 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 rx=0x7f71d8002f70 tx=0x7f71d8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:50.901 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.878+0000 7f71ceffd700 1 -- 172.21.15.17:0/1455449568 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71d8011860 con 0x7f71dc066780 2022-08-01T19:41:50.901 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.878+0000 7f71ceffd700 1 -- 172.21.15.17:0/1455449568 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d80119c0 con 0x7f71dc066780 2022-08-01T19:41:50.901 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.878+0000 7f71e2d63700 1 -- 172.21.15.17:0/1455449568 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71dc13e890 con 0x7f71dc066780 2022-08-01T19:41:50.901 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.878+0000 7f71ceffd700 1 -- 172.21.15.17:0/1455449568 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71d8011d20 con 0x7f71dc066780 2022-08-01T19:41:50.901 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.878+0000 7f71e2d63700 1 -- 172.21.15.17:0/1455449568 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71dc0b2520 con 0x7f71dc066780 2022-08-01T19:41:50.902 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.878+0000 7f71ceffd700 1 -- 172.21.15.17:0/1455449568 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f71d8025030 con 0x7f71dc066780 2022-08-01T19:41:50.902 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.882+0000 7f71ceffd700 1 --2- 172.21.15.17:0/1455449568 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f71bc053950 0x7f71bc055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:50.902 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.882+0000 7f71e2d63700 1 -- 172.21.15.17:0/1455449568 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71dc0b18d0 con 0x7f71dc066780 2022-08-01T19:41:50.902 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.882+0000 7f71ceffd700 1 -- 172.21.15.17:0/1455449568 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f71d806d280 con 0x7f71dc066780 2022-08-01T19:41:50.903 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.882+0000 7f71e1560700 1 --2- 172.21.15.17:0/1455449568 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f71bc053950 0x7f71bc055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:50.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.882+0000 7f71e1560700 1 --2- 172.21.15.17:0/1455449568 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f71bc053950 0x7f71bc055e10 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f71d000b000 tx=0x7f71d0006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:50.907 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:50.886+0000 7f71ceffd700 1 -- 172.21.15.17:0/1455449568 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f71d8023070 con 0x7f71dc066780 2022-08-01T19:41:51.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.010+0000 7f71e2d63700 1 -- 172.21.15.17:0/1455449568 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f71dc0574e0 con 0x7f71dc066780 2022-08-01T19:41:51.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.010+0000 7f71ceffd700 1 -- 172.21.15.17:0/1455449568 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v107) v1 ==== 110+0+10 (secure 0 0 0) 0x7f71d8044040 con 0x7f71dc066780 2022-08-01T19:41:51.024 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:41:51.027 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.014+0000 7f71e2d63700 1 -- 172.21.15.17:0/1455449568 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f71bc053950 msgr2=0x7f71bc055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:51.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.014+0000 7f71e2d63700 1 --2- 172.21.15.17:0/1455449568 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f71bc053950 0x7f71bc055e10 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f71d000b000 tx=0x7f71d0006df0).stop 2022-08-01T19:41:51.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.014+0000 7f71e2d63700 1 -- 172.21.15.17:0/1455449568 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71dc066780 msgr2=0x7f71dc0b2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:51.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.014+0000 7f71e2d63700 1 --2- 172.21.15.17:0/1455449568 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71dc066780 0x7f71dc0b2030 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 rx=0x7f71d8002f70 tx=0x7f71d8015000).stop 2022-08-01T19:41:51.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.014+0000 7f71e2d63700 1 -- 172.21.15.17:0/1455449568 shutdown_connections 2022-08-01T19:41:51.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.014+0000 7f71e2d63700 1 --2- 172.21.15.17:0/1455449568 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f71bc053950 0x7f71bc055e10 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:51.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.014+0000 7f71e2d63700 1 --2- 172.21.15.17:0/1455449568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71dc140e40 0x7f71dc1507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:51.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.014+0000 7f71e2d63700 1 --2- 172.21.15.17:0/1455449568 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71dc0b1bc0 0x7f71dc1502e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:51.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.014+0000 7f71e2d63700 1 --2- 172.21.15.17:0/1455449568 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71dc066780 0x7f71dc0b2030 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:51.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.014+0000 7f71e2d63700 1 -- 172.21.15.17:0/1455449568 >> 172.21.15.17:0/1455449568 conn(0x7f71dc063870 msgr2=0x7f71dc144210 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:51.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.014+0000 7f71e2d63700 1 -- 172.21.15.17:0/1455449568 shutdown_connections 2022-08-01T19:41:51.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.014+0000 7f71e2d63700 1 -- 172.21.15.17:0/1455449568 wait complete. 2022-08-01T19:41:51.034 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:41:51.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.146+0000 7f889a8f8700 1 Processor -- start 2022-08-01T19:41:51.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.146+0000 7f889a8f8700 1 -- start start 2022-08-01T19:41:51.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.146+0000 7f889a8f8700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f88940b38a0 0x7f88940b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:51.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.146+0000 7f889a8f8700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8894143ab0 con 0x7f88940a9da0 2022-08-01T19:41:51.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.146+0000 7f889a8f8700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8894143d50 con 0x7f88940b0740 2022-08-01T19:41:51.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.146+0000 7f889a8f8700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8894143ff0 con 0x7f88940b38a0 2022-08-01T19:41:51.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.146+0000 7f8898e95700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f88940a9da0 0x7f88940b1240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34982/0 (socket says 172.21.15.17:34982) 2022-08-01T19:41:51.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.146+0000 7f8898e95700 1 -- 172.21.15.17:0/2442954426 learned_addr learned my addr 172.21.15.17:0/2442954426 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:51.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.146+0000 7f88937fe700 1 --2- 172.21.15.17:0/2442954426 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f88940b38a0 0x7f88940b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:51.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.146+0000 7f8892ffd700 1 -- 172.21.15.17:0/2442954426 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 336222985 0 0) 0x7f8894143ab0 con 0x7f88940a9da0 2022-08-01T19:41:51.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.146+0000 7f8892ffd700 1 -- 172.21.15.17:0/2442954426 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8880003260 con 0x7f88940a9da0 2022-08-01T19:41:51.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.146+0000 7f8892ffd700 1 -- 172.21.15.17:0/2442954426 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 99867785 0 0) 0x7f8894143d50 con 0x7f88940b0740 2022-08-01T19:41:51.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.146+0000 7f8892ffd700 1 -- 172.21.15.17:0/2442954426 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8894143ab0 con 0x7f88940b0740 2022-08-01T19:41:51.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.146+0000 7f8892ffd700 1 -- 172.21.15.17:0/2442954426 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3248655570 0 0) 0x7f8880003260 con 0x7f88940a9da0 2022-08-01T19:41:51.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.146+0000 7f88937fe700 1 -- 172.21.15.17:0/2442954426 >> v1:172.21.15.77:6789/0 conn(0x7f88940b0740 legacy=0x7f889413d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:51.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.146+0000 7f88937fe700 1 -- 172.21.15.17:0/2442954426 >> v1:172.21.15.17:6789/0 conn(0x7f88940a9da0 legacy=0x7f88940b1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:51.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.146+0000 7f88937fe700 1 -- 172.21.15.17:0/2442954426 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8894144290 con 0x7f88940b38a0 2022-08-01T19:41:51.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.146+0000 7f88937fe700 1 --2- 172.21.15.17:0/2442954426 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f88940b38a0 0x7f88940b0b20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f888c001490 tx=0x7f8888006df0).ready entity=mon.2 client_cookie=c308e25e6c6ae1b7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:51.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.146+0000 7f8892ffd700 1 -- 172.21.15.17:0/2442954426 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f888c0019e0 con 0x7f88940b38a0 2022-08-01T19:41:51.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.146+0000 7f8892ffd700 1 -- 172.21.15.17:0/2442954426 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8888000e90 con 0x7f88940b38a0 2022-08-01T19:41:51.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.146+0000 7f8892ffd700 1 -- 172.21.15.17:0/2442954426 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8888012460 con 0x7f88940b38a0 2022-08-01T19:41:51.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.146+0000 7f889a8f8700 1 -- 172.21.15.17:0/2442954426 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f88940b38a0 msgr2=0x7f88940b0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:51.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.146+0000 7f889a8f8700 1 --2- 172.21.15.17:0/2442954426 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f88940b38a0 0x7f88940b0b20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f888c001490 tx=0x7f8888006df0).stop 2022-08-01T19:41:51.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.146+0000 7f889a8f8700 1 -- 172.21.15.17:0/2442954426 shutdown_connections 2022-08-01T19:41:51.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.146+0000 7f889a8f8700 1 --2- 172.21.15.17:0/2442954426 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f88940b38a0 0x7f88940b0b20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:51.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.146+0000 7f889a8f8700 1 -- 172.21.15.17:0/2442954426 >> 172.21.15.17:0/2442954426 conn(0x7f8894060950 msgr2=0x7f8894060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:51.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.146+0000 7f889a8f8700 1 -- 172.21.15.17:0/2442954426 shutdown_connections 2022-08-01T19:41:51.162 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.146+0000 7f889a8f8700 1 -- 172.21.15.17:0/2442954426 wait complete. 2022-08-01T19:41:51.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.150+0000 7f889a8f8700 1 Processor -- start 2022-08-01T19:41:51.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.150+0000 7f889a8f8700 1 -- start start 2022-08-01T19:41:51.169 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.150+0000 7f889a8f8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88940a9da0 0x7f88940b1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:51.169 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.150+0000 7f889a8f8700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f88940b0740 0x7f8894147bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:51.169 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.150+0000 7f889a8f8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88940b38a0 0x7f88941480e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:51.170 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.150+0000 7f889a8f8700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8894140580 con 0x7f88940b38a0 2022-08-01T19:41:51.170 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.150+0000 7f889a8f8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88941528a0 con 0x7f88940a9da0 2022-08-01T19:41:51.171 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.150+0000 7f889a8f8700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8894152b40 con 0x7f88940b0740 2022-08-01T19:41:51.171 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.150+0000 7f8898e95700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88940b38a0 0x7f88941480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:51.171 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.150+0000 7f8898e95700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88940b38a0 0x7f88941480e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42794/0 (socket says 172.21.15.17:42794) 2022-08-01T19:41:51.171 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.150+0000 7f8898e95700 1 -- 172.21.15.17:0/3456287393 learned_addr learned my addr 172.21.15.17:0/3456287393 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:51.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.150+0000 7f88937fe700 1 --2- 172.21.15.17:0/3456287393 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f88940b0740 0x7f8894147bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:51.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.150+0000 7f8893fff700 1 --2- 172.21.15.17:0/3456287393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88940a9da0 0x7f88940b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:51.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.150+0000 7f8898e95700 1 -- 172.21.15.17:0/3456287393 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f88940b0740 msgr2=0x7f8894147bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:51.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.150+0000 7f8898e95700 1 --2- 172.21.15.17:0/3456287393 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f88940b0740 0x7f8894147bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:51.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.150+0000 7f8898e95700 1 -- 172.21.15.17:0/3456287393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88940a9da0 msgr2=0x7f88940b1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:51.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.150+0000 7f8898e95700 1 --2- 172.21.15.17:0/3456287393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88940a9da0 0x7f88940b1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:51.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.150+0000 7f8898e95700 1 -- 172.21.15.17:0/3456287393 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f888800b180 con 0x7f88940b38a0 2022-08-01T19:41:51.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.150+0000 7f8898e95700 1 --2- 172.21.15.17:0/3456287393 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88940b38a0 0x7f88941480e0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 rx=0x7f888c001bc0 tx=0x7f888c001a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:51.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.150+0000 7f88917fa700 1 -- 172.21.15.17:0/3456287393 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f888c004ab0 con 0x7f88940b38a0 2022-08-01T19:41:51.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.150+0000 7f88917fa700 1 -- 172.21.15.17:0/3456287393 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f888c004c10 con 0x7f88940b38a0 2022-08-01T19:41:51.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.150+0000 7f889a8f8700 1 -- 172.21.15.17:0/3456287393 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8894152de0 con 0x7f88940b38a0 2022-08-01T19:41:51.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.150+0000 7f88917fa700 1 -- 172.21.15.17:0/3456287393 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f888c007170 con 0x7f88940b38a0 2022-08-01T19:41:51.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.150+0000 7f889a8f8700 1 -- 172.21.15.17:0/3456287393 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8894152590 con 0x7f88940b38a0 2022-08-01T19:41:51.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.154+0000 7f889a8f8700 1 -- 172.21.15.17:0/3456287393 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8894152f70 con 0x7f88940b38a0 2022-08-01T19:41:51.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.154+0000 7f88917fa700 1 -- 172.21.15.17:0/3456287393 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f888c01b070 con 0x7f88940b38a0 2022-08-01T19:41:51.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.154+0000 7f88917fa700 1 --2- 172.21.15.17:0/3456287393 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8874053a70 0x7f8874055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:51.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.154+0000 7f88917fa700 1 -- 172.21.15.17:0/3456287393 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f888c06ba50 con 0x7f88940b38a0 2022-08-01T19:41:51.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.154+0000 7f8893fff700 1 --2- 172.21.15.17:0/3456287393 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8874053a70 0x7f8874055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:51.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.154+0000 7f8893fff700 1 --2- 172.21.15.17:0/3456287393 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8874053a70 0x7f8874055f30 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f888400d000 tx=0x7f8884007a20).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:51.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.162+0000 7f88917fa700 1 -- 172.21.15.17:0/3456287393 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f888c042770 con 0x7f88940b38a0 2022-08-01T19:41:51.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.282+0000 7f889a8f8700 1 -- 172.21.15.17:0/3456287393 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f889413ddf0 con 0x7f88940b38a0 2022-08-01T19:41:51.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.282+0000 7f88917fa700 1 -- 172.21.15.17:0/3456287393 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v107) v1 ==== 92+0+10 (secure 0 0 0) 0x7f888c045d90 con 0x7f88940b38a0 2022-08-01T19:41:51.295 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:41:51.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.286+0000 7f889a8f8700 1 -- 172.21.15.17:0/3456287393 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8874053a70 msgr2=0x7f8874055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:51.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.286+0000 7f889a8f8700 1 --2- 172.21.15.17:0/3456287393 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8874053a70 0x7f8874055f30 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f888400d000 tx=0x7f8884007a20).stop 2022-08-01T19:41:51.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.286+0000 7f889a8f8700 1 -- 172.21.15.17:0/3456287393 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88940b38a0 msgr2=0x7f88941480e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:51.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.286+0000 7f889a8f8700 1 --2- 172.21.15.17:0/3456287393 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88940b38a0 0x7f88941480e0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 rx=0x7f888c001bc0 tx=0x7f888c001a40).stop 2022-08-01T19:41:51.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.286+0000 7f889a8f8700 1 -- 172.21.15.17:0/3456287393 shutdown_connections 2022-08-01T19:41:51.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.286+0000 7f889a8f8700 1 --2- 172.21.15.17:0/3456287393 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8874053a70 0x7f8874055f30 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:51.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.286+0000 7f889a8f8700 1 --2- 172.21.15.17:0/3456287393 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88940b38a0 0x7f88941480e0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:51.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.286+0000 7f889a8f8700 1 --2- 172.21.15.17:0/3456287393 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f88940b0740 0x7f8894147bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:51.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.286+0000 7f889a8f8700 1 --2- 172.21.15.17:0/3456287393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88940a9da0 0x7f88940b1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:51.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.286+0000 7f889a8f8700 1 -- 172.21.15.17:0/3456287393 >> 172.21.15.17:0/3456287393 conn(0x7f8894060950 msgr2=0x7f8894142b60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:51.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.286+0000 7f889a8f8700 1 -- 172.21.15.17:0/3456287393 shutdown_connections 2022-08-01T19:41:51.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.286+0000 7f889a8f8700 1 -- 172.21.15.17:0/3456287393 wait complete. 2022-08-01T19:41:51.306 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T19:41:51.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.418+0000 7f8c433a2700 1 Processor -- start 2022-08-01T19:41:51.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.418+0000 7f8c433a2700 1 -- start start 2022-08-01T19:41:51.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.418+0000 7f8c433a2700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8c3c0ab930 0x7f8c3c0abd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:51.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.418+0000 7f8c433a2700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f8c3c144890 0x7f8c3c144d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:51.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.418+0000 7f8c433a2700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8c3c0ac750 0x7f8c3c0acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:51.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.418+0000 7f8c433a2700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c3c148120 con 0x7f8c3c0ab930 2022-08-01T19:41:51.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.418+0000 7f8c433a2700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c3c145200 con 0x7f8c3c144890 2022-08-01T19:41:51.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.418+0000 7f8c433a2700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c060990 con 0x7f8c3c0ac750 2022-08-01T19:41:51.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.418+0000 7f8c4113e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8c3c0ab930 0x7f8c3c0abd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:51.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.418+0000 7f8c4113e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8c3c0ab930 0x7f8c3c0abd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42798/0 (socket says 172.21.15.17:42798) 2022-08-01T19:41:51.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.418+0000 7f8c4113e700 1 -- 172.21.15.17:0/4154561463 learned_addr learned my addr 172.21.15.17:0/4154561463 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:51.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.418+0000 7f8c4193f700 1 --2- 172.21.15.17:0/4154561463 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8c3c0ac750 0x7f8c3c0acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:51.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.418+0000 7f8c4093d700 1 --2- 172.21.15.17:0/4154561463 >> v2:172.21.15.77:3300/0 conn(0x7f8c3c144890 0x7f8c3c144d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:51.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.418+0000 7f8c4113e700 1 -- 172.21.15.17:0/4154561463 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8c3c0ac750 msgr2=0x7f8c3c0acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:51.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.418+0000 7f8c4113e700 1 --2- 172.21.15.17:0/4154561463 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8c3c0ac750 0x7f8c3c0acbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:51.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.418+0000 7f8c4113e700 1 -- 172.21.15.17:0/4154561463 >> v2:172.21.15.77:3300/0 conn(0x7f8c3c144890 msgr2=0x7f8c3c144d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:51.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.418+0000 7f8c4113e700 1 --2- 172.21.15.17:0/4154561463 >> v2:172.21.15.77:3300/0 conn(0x7f8c3c144890 0x7f8c3c144d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:51.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.418+0000 7f8c4113e700 1 -- 172.21.15.17:0/4154561463 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c3c060050 con 0x7f8c3c0ab930 2022-08-01T19:41:51.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.418+0000 7f8c4193f700 1 --2- 172.21.15.17:0/4154561463 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8c3c0ac750 0x7f8c3c0acbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:51.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.418+0000 7f8c4113e700 1 --2- 172.21.15.17:0/4154561463 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c3c0ab930 0x7f8c3c0abd10 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 rx=0x7f8c3c0ab540 tx=0x7f8c38006eb0).ready entity=mon.0 client_cookie=9f469c33ac5a31f4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:51.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.418+0000 7f8c33fff700 1 -- 172.21.15.17:0/4154561463 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c3800d570 con 0x7f8c3c0ab930 2022-08-01T19:41:51.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.418+0000 7f8c33fff700 1 -- 172.21.15.17:0/4154561463 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c38000e90 con 0x7f8c3c0ab930 2022-08-01T19:41:51.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.418+0000 7f8c33fff700 1 -- 172.21.15.17:0/4154561463 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c3800d900 con 0x7f8c3c0ab930 2022-08-01T19:41:51.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.422+0000 7f8c433a2700 1 -- 172.21.15.17:0/4154561463 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c3c0ab930 msgr2=0x7f8c3c0abd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:51.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.422+0000 7f8c433a2700 1 --2- 172.21.15.17:0/4154561463 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c3c0ab930 0x7f8c3c0abd10 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 rx=0x7f8c3c0ab540 tx=0x7f8c38006eb0).stop 2022-08-01T19:41:51.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.422+0000 7f8c433a2700 1 -- 172.21.15.17:0/4154561463 shutdown_connections 2022-08-01T19:41:51.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.422+0000 7f8c433a2700 1 --2- 172.21.15.17:0/4154561463 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8c3c0ac750 0x7f8c3c0acbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:51.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.422+0000 7f8c433a2700 1 --2- 172.21.15.17:0/4154561463 >> v2:172.21.15.77:3300/0 conn(0x7f8c3c144890 0x7f8c3c144d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:51.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.422+0000 7f8c433a2700 1 --2- 172.21.15.17:0/4154561463 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c3c0ab930 0x7f8c3c0abd10 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:51.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.422+0000 7f8c433a2700 1 -- 172.21.15.17:0/4154561463 >> 172.21.15.17:0/4154561463 conn(0x7f8c3c0b2b00 msgr2=0x7f8c3c0b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:51.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.422+0000 7f8c433a2700 1 -- 172.21.15.17:0/4154561463 shutdown_connections 2022-08-01T19:41:51.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.422+0000 7f8c433a2700 1 -- 172.21.15.17:0/4154561463 wait complete. 2022-08-01T19:41:51.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.422+0000 7f8c433a2700 1 Processor -- start 2022-08-01T19:41:51.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.422+0000 7f8c433a2700 1 -- start start 2022-08-01T19:41:51.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.422+0000 7f8c433a2700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8c3c0ab930 0x7f8c3c066720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:51.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.422+0000 7f8c433a2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c3c0ac750 0x7f8c3c0af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:51.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.422+0000 7f8c433a2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c3c144890 0x7f8c3c15a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:51.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.422+0000 7f8c433a2700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c157470 con 0x7f8c3c0ac750 2022-08-01T19:41:51.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.422+0000 7f8c433a2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c157700 con 0x7f8c3c144890 2022-08-01T19:41:51.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.422+0000 7f8c433a2700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c1579a0 con 0x7f8c3c0ab930 2022-08-01T19:41:51.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.422+0000 7f8c4093d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c3c0ac750 0x7f8c3c0af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:51.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.422+0000 7f8c4093d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c3c0ac750 0x7f8c3c0af5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42806/0 (socket says 172.21.15.17:42806) 2022-08-01T19:41:51.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.422+0000 7f8c4093d700 1 -- 172.21.15.17:0/3258860290 learned_addr learned my addr 172.21.15.17:0/3258860290 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:51.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.422+0000 7f8c4113e700 1 --2- 172.21.15.17:0/3258860290 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8c3c0ab930 0x7f8c3c066720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:51.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.422+0000 7f8c4193f700 1 --2- 172.21.15.17:0/3258860290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c3c144890 0x7f8c3c15a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:51.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.422+0000 7f8c4093d700 1 -- 172.21.15.17:0/3258860290 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8c3c0ab930 msgr2=0x7f8c3c066720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:51.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.422+0000 7f8c4093d700 1 --2- 172.21.15.17:0/3258860290 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8c3c0ab930 0x7f8c3c066720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:51.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.422+0000 7f8c4093d700 1 -- 172.21.15.17:0/3258860290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c3c144890 msgr2=0x7f8c3c15a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:51.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.422+0000 7f8c4093d700 1 --2- 172.21.15.17:0/3258860290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c3c144890 0x7f8c3c15a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:51.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.422+0000 7f8c4093d700 1 -- 172.21.15.17:0/3258860290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c3800b1e0 con 0x7f8c3c0ac750 2022-08-01T19:41:51.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.422+0000 7f8c4093d700 1 --2- 172.21.15.17:0/3258860290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c3c0ac750 0x7f8c3c0af5d0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 rx=0x7f8c2c0030b0 tx=0x7f8c2c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:51.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.422+0000 7f8c327fc700 1 -- 172.21.15.17:0/3258860290 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c2c011780 con 0x7f8c3c0ac750 2022-08-01T19:41:51.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.422+0000 7f8c327fc700 1 -- 172.21.15.17:0/3258860290 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c2c0118e0 con 0x7f8c3c0ac750 2022-08-01T19:41:51.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.422+0000 7f8c327fc700 1 -- 172.21.15.17:0/3258860290 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c2c011c40 con 0x7f8c3c0ac750 2022-08-01T19:41:51.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.422+0000 7f8c433a2700 1 -- 172.21.15.17:0/3258860290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c3c065fe0 con 0x7f8c3c0ac750 2022-08-01T19:41:51.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.422+0000 7f8c433a2700 1 -- 172.21.15.17:0/3258860290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c3c066f60 con 0x7f8c3c0ac750 2022-08-01T19:41:51.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.426+0000 7f8c433a2700 1 -- 172.21.15.17:0/3258860290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c3c148340 con 0x7f8c3c0ac750 2022-08-01T19:41:51.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.426+0000 7f8c327fc700 1 -- 172.21.15.17:0/3258860290 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8c2c01c460 con 0x7f8c3c0ac750 2022-08-01T19:41:51.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.426+0000 7f8c327fc700 1 --2- 172.21.15.17:0/3258860290 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8c24053a10 0x7f8c24055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:51.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.426+0000 7f8c327fc700 1 -- 172.21.15.17:0/3258860290 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8c2c06c470 con 0x7f8c3c0ac750 2022-08-01T19:41:51.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.426+0000 7f8c4113e700 1 --2- 172.21.15.17:0/3258860290 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8c24053a10 0x7f8c24055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:51.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.426+0000 7f8c4113e700 1 --2- 172.21.15.17:0/3258860290 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8c24053a10 0x7f8c24055ed0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f8c3801b000 tx=0x7f8c38015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:51.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.430+0000 7f8c327fc700 1 -- 172.21.15.17:0/3258860290 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8c2c043130 con 0x7f8c3c0ac750 2022-08-01T19:41:51.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.558+0000 7f8c433a2700 1 -- 172.21.15.17:0/3258860290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8c3c0574e0 con 0x7f8c3c0ac750 2022-08-01T19:41:51.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.558+0000 7f8c327fc700 1 -- 172.21.15.17:0/3258860290 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v107) v1 ==== 104+0+10 (secure 0 0 0) 0x7f8c2c046750 con 0x7f8c3c0ac750 2022-08-01T19:41:51.562 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:41:51.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.558+0000 7f8c433a2700 1 -- 172.21.15.17:0/3258860290 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8c24053a10 msgr2=0x7f8c24055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:51.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.558+0000 7f8c433a2700 1 --2- 172.21.15.17:0/3258860290 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8c24053a10 0x7f8c24055ed0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f8c3801b000 tx=0x7f8c38015000).stop 2022-08-01T19:41:51.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.558+0000 7f8c433a2700 1 -- 172.21.15.17:0/3258860290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c3c0ac750 msgr2=0x7f8c3c0af5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:51.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.558+0000 7f8c433a2700 1 --2- 172.21.15.17:0/3258860290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c3c0ac750 0x7f8c3c0af5d0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 rx=0x7f8c2c0030b0 tx=0x7f8c2c008040).stop 2022-08-01T19:41:51.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.558+0000 7f8c433a2700 1 -- 172.21.15.17:0/3258860290 shutdown_connections 2022-08-01T19:41:51.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.558+0000 7f8c433a2700 1 --2- 172.21.15.17:0/3258860290 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8c24053a10 0x7f8c24055ed0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:51.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.558+0000 7f8c433a2700 1 --2- 172.21.15.17:0/3258860290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c3c144890 0x7f8c3c15a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:51.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.558+0000 7f8c433a2700 1 --2- 172.21.15.17:0/3258860290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c3c0ac750 0x7f8c3c0af5d0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:51.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.558+0000 7f8c433a2700 1 --2- 172.21.15.17:0/3258860290 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8c3c0ab930 0x7f8c3c066720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:51.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.558+0000 7f8c433a2700 1 -- 172.21.15.17:0/3258860290 >> 172.21.15.17:0/3258860290 conn(0x7f8c3c0b2b00 msgr2=0x7f8c3c146860 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:51.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.562+0000 7f8c433a2700 1 -- 172.21.15.17:0/3258860290 shutdown_connections 2022-08-01T19:41:51.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.562+0000 7f8c433a2700 1 -- 172.21.15.17:0/3258860290 wait complete. 2022-08-01T19:41:51.574 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T19:41:51.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.682+0000 7f5b8ab98700 1 Processor -- start 2022-08-01T19:41:51.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.682+0000 7f5b8ab98700 1 -- start start 2022-08-01T19:41:51.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.682+0000 7f5b8ab98700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5b840b7840 0x7f5b84060be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:51.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.682+0000 7f5b8ab98700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5b84143400 0x7f5b84143860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:51.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.686+0000 7f5b8ab98700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b840671e0 con 0x7f5b840b2930 2022-08-01T19:41:51.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.686+0000 7f5b8ab98700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b84065540 con 0x7f5b840b7840 2022-08-01T19:41:51.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.686+0000 7f5b8ab98700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b840b7c20 con 0x7f5b84143400 2022-08-01T19:41:51.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.686+0000 7f5b89b96700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5b840b7840 0x7f5b84060be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:51.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.686+0000 7f5b89b96700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5b840b7840 0x7f5b84060be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42816/0 (socket says 172.21.15.17:42816) 2022-08-01T19:41:51.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.686+0000 7f5b89b96700 1 -- 172.21.15.17:0/2861076089 learned_addr learned my addr 172.21.15.17:0/2861076089 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:51.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.686+0000 7f5b8a397700 1 --2- 172.21.15.17:0/2861076089 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5b84143400 0x7f5b84143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:51.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.686+0000 7f5b89b96700 1 -- 172.21.15.17:0/2861076089 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5b84143400 msgr2=0x7f5b84143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:51.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.686+0000 7f5b89b96700 1 --2- 172.21.15.17:0/2861076089 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5b84143400 0x7f5b84143860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:51.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.686+0000 7f5b89b96700 1 -- 172.21.15.17:0/2861076089 >> v1:172.21.15.77:6789/0 conn(0x7f5b840b2930 legacy=0x7f5b840610d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:51.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.686+0000 7f5b89b96700 1 -- 172.21.15.17:0/2861076089 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b840617e0 con 0x7f5b840b7840 2022-08-01T19:41:51.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.686+0000 7f5b89b96700 1 --2- 172.21.15.17:0/2861076089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b840b7840 0x7f5b84060be0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 rx=0x7f5b80002f70 tx=0x7f5b80006e50).ready entity=mon.0 client_cookie=84013cb24a6cf466 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:51.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.686+0000 7f5b88b94700 1 -- 172.21.15.17:0/2861076089 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b8000d5c0 con 0x7f5b840b7840 2022-08-01T19:41:51.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.686+0000 7f5b88b94700 1 -- 172.21.15.17:0/2861076089 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b80000e90 con 0x7f5b840b7840 2022-08-01T19:41:51.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.686+0000 7f5b88b94700 1 -- 172.21.15.17:0/2861076089 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b8000d950 con 0x7f5b840b7840 2022-08-01T19:41:51.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.686+0000 7f5b8ab98700 1 -- 172.21.15.17:0/2861076089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b840b7840 msgr2=0x7f5b84060be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:51.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.686+0000 7f5b8ab98700 1 --2- 172.21.15.17:0/2861076089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b840b7840 0x7f5b84060be0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 rx=0x7f5b80002f70 tx=0x7f5b80006e50).stop 2022-08-01T19:41:51.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.686+0000 7f5b8ab98700 1 -- 172.21.15.17:0/2861076089 shutdown_connections 2022-08-01T19:41:51.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.686+0000 7f5b8ab98700 1 --2- 172.21.15.17:0/2861076089 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5b84143400 0x7f5b84143860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:51.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.686+0000 7f5b8ab98700 1 --2- 172.21.15.17:0/2861076089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b840b7840 0x7f5b84060be0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:51.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.686+0000 7f5b8ab98700 1 -- 172.21.15.17:0/2861076089 >> 172.21.15.17:0/2861076089 conn(0x7f5b8405f840 msgr2=0x7f5b8405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:51.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.686+0000 7f5b8ab98700 1 -- 172.21.15.17:0/2861076089 shutdown_connections 2022-08-01T19:41:51.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.686+0000 7f5b8ab98700 1 -- 172.21.15.17:0/2861076089 wait complete. 2022-08-01T19:41:51.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.690+0000 7f5b8ab98700 1 Processor -- start 2022-08-01T19:41:51.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.690+0000 7f5b8ab98700 1 -- start start 2022-08-01T19:41:51.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.690+0000 7f5b8ab98700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b840b2930 0x7f5b8414a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:51.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.690+0000 7f5b8ab98700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5b840b7840 0x7f5b840af170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:51.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.690+0000 7f5b8ab98700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b84143400 0x7f5b840af660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:51.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.690+0000 7f5b8ab98700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b840ae760 con 0x7f5b840b2930 2022-08-01T19:41:51.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.690+0000 7f5b8ab98700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b840abd60 con 0x7f5b84143400 2022-08-01T19:41:51.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.690+0000 7f5b8ab98700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b840ab870 con 0x7f5b840b7840 2022-08-01T19:41:51.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.690+0000 7f5b89b96700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b840b2930 0x7f5b8414a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:51.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.690+0000 7f5b89395700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5b840b7840 0x7f5b840af170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:51.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.690+0000 7f5b89b96700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b840b2930 0x7f5b8414a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42818/0 (socket says 172.21.15.17:42818) 2022-08-01T19:41:51.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.690+0000 7f5b89b96700 1 -- 172.21.15.17:0/2366750292 learned_addr learned my addr 172.21.15.17:0/2366750292 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:51.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.690+0000 7f5b8a397700 1 --2- 172.21.15.17:0/2366750292 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b84143400 0x7f5b840af660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:51.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.690+0000 7f5b89b96700 1 -- 172.21.15.17:0/2366750292 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5b840b7840 msgr2=0x7f5b840af170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:51.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.690+0000 7f5b89b96700 1 --2- 172.21.15.17:0/2366750292 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5b840b7840 0x7f5b840af170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:51.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.690+0000 7f5b89b96700 1 -- 172.21.15.17:0/2366750292 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b84143400 msgr2=0x7f5b840af660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:51.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.690+0000 7f5b89b96700 1 --2- 172.21.15.17:0/2366750292 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b84143400 0x7f5b840af660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:51.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.690+0000 7f5b89b96700 1 -- 172.21.15.17:0/2366750292 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b8000b1e0 con 0x7f5b840b2930 2022-08-01T19:41:51.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.690+0000 7f5b89b96700 1 --2- 172.21.15.17:0/2366750292 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b840b2930 0x7f5b8414a720 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 rx=0x7f5b80002f70 tx=0x7f5b80014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:51.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.690+0000 7f5b72ffd700 1 -- 172.21.15.17:0/2366750292 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b80013de0 con 0x7f5b840b2930 2022-08-01T19:41:51.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.690+0000 7f5b72ffd700 1 -- 172.21.15.17:0/2366750292 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b80006850 con 0x7f5b840b2930 2022-08-01T19:41:51.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.690+0000 7f5b72ffd700 1 -- 172.21.15.17:0/2366750292 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b8001e5f0 con 0x7f5b840b2930 2022-08-01T19:41:51.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.690+0000 7f5b8ab98700 1 -- 172.21.15.17:0/2366750292 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b840abfc0 con 0x7f5b840b2930 2022-08-01T19:41:51.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.690+0000 7f5b8ab98700 1 -- 172.21.15.17:0/2366750292 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b84144cb0 con 0x7f5b840b2930 2022-08-01T19:41:51.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.694+0000 7f5b8ab98700 1 -- 172.21.15.17:0/2366750292 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b840aef70 con 0x7f5b840b2930 2022-08-01T19:41:51.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.694+0000 7f5b72ffd700 1 -- 172.21.15.17:0/2366750292 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5b80025030 con 0x7f5b840b2930 2022-08-01T19:41:51.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.694+0000 7f5b72ffd700 1 --2- 172.21.15.17:0/2366750292 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5b64053aa0 0x7f5b64055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:51.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.694+0000 7f5b72ffd700 1 -- 172.21.15.17:0/2366750292 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5b8006ce80 con 0x7f5b840b2930 2022-08-01T19:41:51.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.694+0000 7f5b89395700 1 --2- 172.21.15.17:0/2366750292 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5b64053aa0 0x7f5b64055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:51.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.694+0000 7f5b89395700 1 --2- 172.21.15.17:0/2366750292 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5b64053aa0 0x7f5b64055f60 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f5b7400d000 tx=0x7f5b74007b90).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:51.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.706+0000 7f5b72ffd700 1 -- 172.21.15.17:0/2366750292 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5b80043b30 con 0x7f5b840b2930 2022-08-01T19:41:51.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.830+0000 7f5b8ab98700 1 -- 172.21.15.17:0/2366750292 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5b840574e0 con 0x7f5b840b2930 2022-08-01T19:41:51.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.834+0000 7f5b72ffd700 1 -- 172.21.15.17:0/2366750292 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v107) v1 ==== 100+0+10 (secure 0 0 0) 0x7f5b80023020 con 0x7f5b840b2930 2022-08-01T19:41:51.841 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:41:51.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.834+0000 7f5b8ab98700 1 -- 172.21.15.17:0/2366750292 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5b64053aa0 msgr2=0x7f5b64055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:51.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.834+0000 7f5b8ab98700 1 --2- 172.21.15.17:0/2366750292 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5b64053aa0 0x7f5b64055f60 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f5b7400d000 tx=0x7f5b74007b90).stop 2022-08-01T19:41:51.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.834+0000 7f5b8ab98700 1 -- 172.21.15.17:0/2366750292 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b840b2930 msgr2=0x7f5b8414a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:51.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.834+0000 7f5b8ab98700 1 --2- 172.21.15.17:0/2366750292 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b840b2930 0x7f5b8414a720 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 rx=0x7f5b80002f70 tx=0x7f5b80014000).stop 2022-08-01T19:41:51.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.834+0000 7f5b8ab98700 1 -- 172.21.15.17:0/2366750292 shutdown_connections 2022-08-01T19:41:51.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.834+0000 7f5b8ab98700 1 --2- 172.21.15.17:0/2366750292 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5b64053aa0 0x7f5b64055f60 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:51.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.834+0000 7f5b8ab98700 1 --2- 172.21.15.17:0/2366750292 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b84143400 0x7f5b840af660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:51.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.834+0000 7f5b8ab98700 1 --2- 172.21.15.17:0/2366750292 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5b840b7840 0x7f5b840af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:51.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.834+0000 7f5b8ab98700 1 --2- 172.21.15.17:0/2366750292 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b840b2930 0x7f5b8414a720 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:51.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.834+0000 7f5b8ab98700 1 -- 172.21.15.17:0/2366750292 >> 172.21.15.17:0/2366750292 conn(0x7f5b8405f840 msgr2=0x7f5b840b4920 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:51.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.834+0000 7f5b8ab98700 1 -- 172.21.15.17:0/2366750292 shutdown_connections 2022-08-01T19:41:51.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.834+0000 7f5b8ab98700 1 -- 172.21.15.17:0/2366750292 wait complete. 2022-08-01T19:41:51.853 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T19:41:51.854 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:41:51.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.958+0000 7f9a75bd6700 1 Processor -- start 2022-08-01T19:41:51.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.958+0000 7f9a75bd6700 1 -- start start 2022-08-01T19:41:51.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.958+0000 7f9a75bd6700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9a700b1f40 0x7f9a700b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:51.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.958+0000 7f9a75bd6700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9a70148530 0x7f9a700b2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:51.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.958+0000 7f9a75bd6700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a700b04e0 con 0x7f9a700a7250 2022-08-01T19:41:51.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.958+0000 7f9a75bd6700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a700a9250 con 0x7f9a700b1f40 2022-08-01T19:41:51.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.958+0000 7f9a75bd6700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a700b0780 con 0x7f9a70148530 2022-08-01T19:41:51.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.958+0000 7f9a6ffff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9a700a7250 0x7f9a701448b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35024/0 (socket says 172.21.15.17:35024) 2022-08-01T19:41:51.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.958+0000 7f9a6f7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9a700b1f40 0x7f9a700b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:51.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.958+0000 7f9a6ffff700 1 -- 172.21.15.17:0/3495765399 learned_addr learned my addr 172.21.15.17:0/3495765399 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:51.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.958+0000 7f9a6effd700 1 --2- 172.21.15.17:0/3495765399 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9a70148530 0x7f9a700b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:51.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.958+0000 7f9a6e7fc700 1 -- 172.21.15.17:0/3495765399 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3187916482 0 0) 0x7f9a700b04e0 con 0x7f9a700a7250 2022-08-01T19:41:51.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.958+0000 7f9a6e7fc700 1 -- 172.21.15.17:0/3495765399 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a5c003260 con 0x7f9a700a7250 2022-08-01T19:41:51.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.958+0000 7f9a6effd700 1 -- 172.21.15.17:0/3495765399 >> v2:172.21.15.17:3300/0 conn(0x7f9a700b1f40 msgr2=0x7f9a700b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:51.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.958+0000 7f9a6effd700 1 --2- 172.21.15.17:0/3495765399 >> v2:172.21.15.17:3300/0 conn(0x7f9a700b1f40 0x7f9a700b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:51.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.958+0000 7f9a6effd700 1 -- 172.21.15.17:0/3495765399 >> v1:172.21.15.17:6789/0 conn(0x7f9a700a7250 legacy=0x7f9a701448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:51.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.958+0000 7f9a6effd700 1 -- 172.21.15.17:0/3495765399 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a700b0a20 con 0x7f9a70148530 2022-08-01T19:41:51.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.958+0000 7f9a6f7fe700 1 --2- 172.21.15.17:0/3495765399 >> v2:172.21.15.17:3300/0 conn(0x7f9a700b1f40 0x7f9a700b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:51.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.958+0000 7f9a6effd700 1 --2- 172.21.15.17:0/3495765399 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9a70148530 0x7f9a700b2320 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f9a60002f70 tx=0x7f9a60006df0).ready entity=mon.2 client_cookie=5d4286d4803b1d2f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:51.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.958+0000 7f9a6e7fc700 1 -- 172.21.15.17:0/3495765399 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a6000d4a0 con 0x7f9a70148530 2022-08-01T19:41:51.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.958+0000 7f9a6e7fc700 1 -- 172.21.15.17:0/3495765399 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a60000e90 con 0x7f9a70148530 2022-08-01T19:41:51.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.958+0000 7f9a6e7fc700 1 -- 172.21.15.17:0/3495765399 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a6000d7f0 con 0x7f9a70148530 2022-08-01T19:41:51.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.962+0000 7f9a75bd6700 1 -- 172.21.15.17:0/3495765399 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9a70148530 msgr2=0x7f9a700b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:51.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.962+0000 7f9a75bd6700 1 --2- 172.21.15.17:0/3495765399 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9a70148530 0x7f9a700b2320 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f9a60002f70 tx=0x7f9a60006df0).stop 2022-08-01T19:41:51.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.962+0000 7f9a75bd6700 1 -- 172.21.15.17:0/3495765399 shutdown_connections 2022-08-01T19:41:51.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.962+0000 7f9a75bd6700 1 --2- 172.21.15.17:0/3495765399 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9a70148530 0x7f9a700b2320 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:51.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.962+0000 7f9a75bd6700 1 --2- 172.21.15.17:0/3495765399 >> v2:172.21.15.17:3300/0 conn(0x7f9a700b1f40 0x7f9a700b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:51.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.962+0000 7f9a75bd6700 1 -- 172.21.15.17:0/3495765399 >> 172.21.15.17:0/3495765399 conn(0x7f9a7005f840 msgr2=0x7f9a7005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:51.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.962+0000 7f9a75bd6700 1 -- 172.21.15.17:0/3495765399 shutdown_connections 2022-08-01T19:41:51.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.962+0000 7f9a75bd6700 1 -- 172.21.15.17:0/3495765399 wait complete. 2022-08-01T19:41:51.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.962+0000 7f9a75bd6700 1 Processor -- start 2022-08-01T19:41:51.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.962+0000 7f9a75bd6700 1 -- start start 2022-08-01T19:41:51.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.962+0000 7f9a75bd6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a700a7250 0x7f9a7014ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:51.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.962+0000 7f9a75bd6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a700b1f40 0x7f9a7015a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:51.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.962+0000 7f9a75bd6700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9a70148530 0x7f9a7015a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:51.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.962+0000 7f9a75bd6700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a700aa890 con 0x7f9a700a7250 2022-08-01T19:41:51.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.962+0000 7f9a75bd6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a70156930 con 0x7f9a700b1f40 2022-08-01T19:41:51.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.962+0000 7f9a75bd6700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a700aaac0 con 0x7f9a70148530 2022-08-01T19:41:51.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.966+0000 7f9a6f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a700a7250 0x7f9a7014ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:51.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.966+0000 7f9a6f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a700a7250 0x7f9a7014ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42832/0 (socket says 172.21.15.17:42832) 2022-08-01T19:41:51.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.966+0000 7f9a6f7fe700 1 -- 172.21.15.17:0/2401817105 learned_addr learned my addr 172.21.15.17:0/2401817105 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:51.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.966+0000 7f9a6effd700 1 --2- 172.21.15.17:0/2401817105 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a700b1f40 0x7f9a7015a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:51.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.966+0000 7f9a6ffff700 1 --2- 172.21.15.17:0/2401817105 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9a70148530 0x7f9a7015a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:52.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.966+0000 7f9a6f7fe700 1 -- 172.21.15.17:0/2401817105 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9a70148530 msgr2=0x7f9a7015a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:52.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.966+0000 7f9a6f7fe700 1 --2- 172.21.15.17:0/2401817105 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9a70148530 0x7f9a7015a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:52.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.966+0000 7f9a6f7fe700 1 -- 172.21.15.17:0/2401817105 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a700b1f40 msgr2=0x7f9a7015a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:52.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.966+0000 7f9a6f7fe700 1 --2- 172.21.15.17:0/2401817105 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a700b1f40 0x7f9a7015a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:52.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.966+0000 7f9a6f7fe700 1 -- 172.21.15.17:0/2401817105 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a6400b180 con 0x7f9a700a7250 2022-08-01T19:41:52.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.966+0000 7f9a6effd700 1 --2- 172.21.15.17:0/2401817105 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a700b1f40 0x7f9a7015a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:52.002 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.966+0000 7f9a6ffff700 1 --2- 172.21.15.17:0/2401817105 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9a70148530 0x7f9a7015a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:52.002 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.966+0000 7f9a6f7fe700 1 --2- 172.21.15.17:0/2401817105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a700a7250 0x7f9a7014ae10 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 rx=0x7f9a6400c000 tx=0x7f9a64006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:52.002 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.966+0000 7f9a6cff9700 1 -- 172.21.15.17:0/2401817105 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a6400ca40 con 0x7f9a700a7250 2022-08-01T19:41:52.002 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.966+0000 7f9a6cff9700 1 -- 172.21.15.17:0/2401817105 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6400cba0 con 0x7f9a700a7250 2022-08-01T19:41:52.003 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.966+0000 7f9a6cff9700 1 -- 172.21.15.17:0/2401817105 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a640123f0 con 0x7f9a700a7250 2022-08-01T19:41:52.003 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.966+0000 7f9a75bd6700 1 -- 172.21.15.17:0/2401817105 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a6000b180 con 0x7f9a700a7250 2022-08-01T19:41:52.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.966+0000 7f9a75bd6700 1 -- 172.21.15.17:0/2401817105 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a7015ac60 con 0x7f9a700a7250 2022-08-01T19:41:52.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.966+0000 7f9a75bd6700 1 -- 172.21.15.17:0/2401817105 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a700b0cc0 con 0x7f9a700a7250 2022-08-01T19:41:52.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.966+0000 7f9a6cff9700 1 -- 172.21.15.17:0/2401817105 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9a64012550 con 0x7f9a700a7250 2022-08-01T19:41:52.013 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.966+0000 7f9a6cff9700 1 --2- 172.21.15.17:0/2401817105 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9a50053b30 0x7f9a50055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:52.013 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.966+0000 7f9a6cff9700 1 -- 172.21.15.17:0/2401817105 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9a6406c760 con 0x7f9a700a7250 2022-08-01T19:41:52.014 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.970+0000 7f9a6effd700 1 --2- 172.21.15.17:0/2401817105 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9a50053b30 0x7f9a50055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:52.014 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.970+0000 7f9a6effd700 1 --2- 172.21.15.17:0/2401817105 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9a50053b30 0x7f9a50055ff0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f9a60018000 tx=0x7f9a60014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:52.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:51.974+0000 7f9a6cff9700 1 -- 172.21.15.17:0/2401817105 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9a640434c0 con 0x7f9a700a7250 2022-08-01T19:41:52.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:52.134+0000 7f9a75bd6700 1 -- 172.21.15.17:0/2401817105 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a700574e0 con 0x7f9a700a7250 2022-08-01T19:41:52.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:52.138+0000 7f9a6cff9700 1 -- 172.21.15.17:0/2401817105 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f9a64046ae0 con 0x7f9a700a7250 2022-08-01T19:41:52.141 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:52.143 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":53,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7252,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/1455566688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1455566688},{"type":"v1","addr":"172.21.15.77:6835","nonce":1455566688}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":51},{"gid":7266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.17:6833/249859043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":249859043},{"type":"v1","addr":"172.21.15.17:6833","nonce":249859043}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":51}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:41:47.422244+0000","modified":"2022-08-01T19:41:48.425195+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7272":{"gid":7272,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/2991628239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2991628239},{"type":"v1","addr":"172.21.15.17:6835","nonce":2991628239}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2022-08-01T19:41:52.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:52.138+0000 7f9a75bd6700 1 -- 172.21.15.17:0/2401817105 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9a50053b30 msgr2=0x7f9a50055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:52.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:52.138+0000 7f9a75bd6700 1 --2- 172.21.15.17:0/2401817105 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9a50053b30 0x7f9a50055ff0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f9a60018000 tx=0x7f9a60014000).stop 2022-08-01T19:41:52.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:52.138+0000 7f9a75bd6700 1 -- 172.21.15.17:0/2401817105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a700a7250 msgr2=0x7f9a7014ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:52.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:52.138+0000 7f9a75bd6700 1 --2- 172.21.15.17:0/2401817105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a700a7250 0x7f9a7014ae10 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 rx=0x7f9a6400c000 tx=0x7f9a64006df0).stop 2022-08-01T19:41:52.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:52.138+0000 7f9a75bd6700 1 -- 172.21.15.17:0/2401817105 shutdown_connections 2022-08-01T19:41:52.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:52.138+0000 7f9a75bd6700 1 --2- 172.21.15.17:0/2401817105 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9a50053b30 0x7f9a50055ff0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:52.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:52.138+0000 7f9a75bd6700 1 --2- 172.21.15.17:0/2401817105 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9a70148530 0x7f9a7015a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:52.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:52.138+0000 7f9a75bd6700 1 --2- 172.21.15.17:0/2401817105 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9a700b1f40 0x7f9a7015a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:52.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:52.138+0000 7f9a75bd6700 1 --2- 172.21.15.17:0/2401817105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a700a7250 0x7f9a7014ae10 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:52.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:52.138+0000 7f9a75bd6700 1 -- 172.21.15.17:0/2401817105 >> 172.21.15.17:0/2401817105 conn(0x7f9a7005f840 msgr2=0x7f9a700b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:52.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:52.138+0000 7f9a75bd6700 1 -- 172.21.15.17:0/2401817105 shutdown_connections 2022-08-01T19:41:52.158 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:52.138+0000 7f9a75bd6700 1 -- 172.21.15.17:0/2401817105 wait complete. 2022-08-01T19:41:52.159 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 53 2022-08-01T19:41:52.468 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 53, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:41:47.422244+0000', 'modified': '2022-08-01T19:41:48.425195+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7272}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7272': {'gid': 7272, 'name': 'c', 'rank': 0, 'incarnation': 52, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/2991628239', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 2991628239}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 2991628239}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:41:52.469 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:41:52.470 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T19:41:52.590 INFO:teuthology.orchestra.run.smithi017.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":7272,"want_state":"up:active","state":"up:active","rank_uptime":5.1539897879999996,"mdsmap_epoch":53,"osdmap_epoch":107,"osdmap_epoch_barrier":107,"uptime":9.5289356020000007} 2022-08-01T19:41:52.591 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 7272, "mdsmap_epoch": 53, "osdmap_epoch": 107, "osdmap_epoch_barrier": 107, "rank_uptime": 5.153989788, "state": "up:active", "uptime": 9.528935602, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:41:52.591 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T19:41:52.592 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-08-01T19:41:52.593 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi085.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-08-01T19:41:52.600 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:52.601 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:41:52.611 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:41:52.678 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:41:52.678 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:41:52.691 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:41:52.692 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:41:52.693 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:52.694 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:41:52.746 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T19:41:52.746 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:41:52.750 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:41:52.769 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:41:52.770 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:41:52.823 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:2022-08-01T19:41:52.819+0000 7f87806a4080 -1 init, newargv = 0x55b55f484660 newargc=9 2022-08-01T19:41:52.824 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:ceph-fuse[23358]: starting ceph client 2022-08-01T19:41:52.833 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:41:52.843 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:41:52.844 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:52.845 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:41:52.847 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:ceph-fuse[23358]: starting fuse 2022-08-01T19:41:52.856 INFO:teuthology.orchestra.run.smithi085.stdout:52 2022-08-01T19:41:52.858 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T19:41:52.859 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:52.860 DEBUG:teuthology.orchestra.run.smithi085:> sudo python3 -c ' 2022-08-01T19:41:52.861 DEBUG:teuthology.orchestra.run.smithi085:> import glob 2022-08-01T19:41:52.861 DEBUG:teuthology.orchestra.run.smithi085:> import re 2022-08-01T19:41:52.861 DEBUG:teuthology.orchestra.run.smithi085:> import os 2022-08-01T19:41:52.862 DEBUG:teuthology.orchestra.run.smithi085:> import subprocess 2022-08-01T19:41:52.862 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:41:52.863 DEBUG:teuthology.orchestra.run.smithi085:> def find_socket(client_name): 2022-08-01T19:41:52.863 DEBUG:teuthology.orchestra.run.smithi085:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-08-01T19:41:52.864 DEBUG:teuthology.orchestra.run.smithi085:> files = glob.glob(asok_path) 2022-08-01T19:41:52.864 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:41:52.865 DEBUG:teuthology.orchestra.run.smithi085:> # Given a non-glob path, it better be there 2022-08-01T19:41:52.865 DEBUG:teuthology.orchestra.run.smithi085:> if "*" not in asok_path: 2022-08-01T19:41:52.866 DEBUG:teuthology.orchestra.run.smithi085:> assert(len(files) == 1) 2022-08-01T19:41:52.867 DEBUG:teuthology.orchestra.run.smithi085:> return files[0] 2022-08-01T19:41:52.867 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:41:52.867 DEBUG:teuthology.orchestra.run.smithi085:> for f in files: 2022-08-01T19:41:52.868 DEBUG:teuthology.orchestra.run.smithi085:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T19:41:52.868 DEBUG:teuthology.orchestra.run.smithi085:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T19:41:52.869 DEBUG:teuthology.orchestra.run.smithi085:> return f 2022-08-01T19:41:52.869 DEBUG:teuthology.orchestra.run.smithi085:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T19:41:52.869 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:41:52.871 DEBUG:teuthology.orchestra.run.smithi085:> print(find_socket("client.0")) 2022-08-01T19:41:52.871 DEBUG:teuthology.orchestra.run.smithi085:> ' 2022-08-01T19:41:52.945 INFO:teuthology.orchestra.run.smithi085.stdout:/var/run/ceph/ceph-client.0.23358.asok 2022-08-01T19:41:52.960 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.23358.asok 2022-08-01T19:41:52.961 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:52.963 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.23358.asok status 2022-08-01T19:41:53.054 INFO:teuthology.orchestra.run.smithi085.stdout:{ 2022-08-01T19:41:53.055 INFO:teuthology.orchestra.run.smithi085.stdout: "metadata": { 2022-08-01T19:41:53.056 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T19:41:53.056 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T19:41:53.056 INFO:teuthology.orchestra.run.smithi085.stdout: "entity_id": "0", 2022-08-01T19:41:53.057 INFO:teuthology.orchestra.run.smithi085.stdout: "hostname": "smithi085", 2022-08-01T19:41:53.057 INFO:teuthology.orchestra.run.smithi085.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-08-01T19:41:53.060 INFO:teuthology.orchestra.run.smithi085.stdout: "pid": "23358", 2022-08-01T19:41:53.061 INFO:teuthology.orchestra.run.smithi085.stdout: "root": "/" 2022-08-01T19:41:53.062 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:41:53.062 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_count": 0, 2022-08-01T19:41:53.063 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_pinned_count": 0, 2022-08-01T19:41:53.063 INFO:teuthology.orchestra.run.smithi085.stdout: "id": 7425, 2022-08-01T19:41:53.064 INFO:teuthology.orchestra.run.smithi085.stdout: "inst": { 2022-08-01T19:41:53.064 INFO:teuthology.orchestra.run.smithi085.stdout: "name": { 2022-08-01T19:41:53.064 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "client", 2022-08-01T19:41:53.065 INFO:teuthology.orchestra.run.smithi085.stdout: "num": 7425 2022-08-01T19:41:53.065 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:41:53.066 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2022-08-01T19:41:53.066 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2022-08-01T19:41:53.066 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2022-08-01T19:41:53.067 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 3681758530 2022-08-01T19:41:53.067 INFO:teuthology.orchestra.run.smithi085.stdout: } 2022-08-01T19:41:53.068 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:41:53.069 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2022-08-01T19:41:53.069 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2022-08-01T19:41:53.069 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2022-08-01T19:41:53.070 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 3681758530 2022-08-01T19:41:53.070 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:41:53.070 INFO:teuthology.orchestra.run.smithi085.stdout: "inst_str": "client.7425 172.21.15.85:0/3681758530", 2022-08-01T19:41:53.070 INFO:teuthology.orchestra.run.smithi085.stdout: "addr_str": "172.21.15.85:0/3681758530", 2022-08-01T19:41:53.071 INFO:teuthology.orchestra.run.smithi085.stdout: "inode_count": 1, 2022-08-01T19:41:53.071 INFO:teuthology.orchestra.run.smithi085.stdout: "mds_epoch": 53, 2022-08-01T19:41:53.071 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch": 107, 2022-08-01T19:41:53.071 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch_barrier": 0, 2022-08-01T19:41:53.072 INFO:teuthology.orchestra.run.smithi085.stdout: "blacklisted": false, 2022-08-01T19:41:53.072 INFO:teuthology.orchestra.run.smithi085.stdout: "fs_name": "cephfs" 2022-08-01T19:41:53.072 INFO:teuthology.orchestra.run.smithi085.stdout:} 2022-08-01T19:41:53.074 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:53.075 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:41:53.075 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:41:53.092 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:41:53.093 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:41:53.093 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:53.094 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:41:53.154 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:41:53.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.274+0000 7fed33fff700 1 Processor -- start 2022-08-01T19:41:53.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.274+0000 7fed33fff700 1 -- start start 2022-08-01T19:41:53.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.278+0000 7fed33fff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fed340b2730 0x7fed340666b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:53.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.278+0000 7fed33fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fed34066ba0 0x7fed3414dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:53.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.278+0000 7fed33fff700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed340654c0 con 0x7fed340b3190 2022-08-01T19:41:53.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.278+0000 7fed33fff700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed34062190 con 0x7fed34066ba0 2022-08-01T19:41:53.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.278+0000 7fed33fff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed340b3570 con 0x7fed340b2730 2022-08-01T19:41:53.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.278+0000 7fed337fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fed34066ba0 0x7fed3414dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:53.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.278+0000 7fed337fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fed34066ba0 0x7fed3414dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42844/0 (socket says 172.21.15.17:42844) 2022-08-01T19:41:53.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.278+0000 7fed337fe700 1 -- 172.21.15.17:0/490312495 learned_addr learned my addr 172.21.15.17:0/490312495 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:53.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.278+0000 7fed2bfff700 1 --2- 172.21.15.17:0/490312495 >> v2:172.21.15.77:3300/0 conn(0x7fed340b2730 0x7fed340666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:53.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.278+0000 7fed337fe700 1 -- 172.21.15.17:0/490312495 >> v2:172.21.15.77:3300/0 conn(0x7fed340b2730 msgr2=0x7fed340666b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:53.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.278+0000 7fed337fe700 1 --2- 172.21.15.17:0/490312495 >> v2:172.21.15.77:3300/0 conn(0x7fed340b2730 0x7fed340666b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:53.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.278+0000 7fed337fe700 1 -- 172.21.15.17:0/490312495 >> v1:172.21.15.77:6789/0 conn(0x7fed340b3190 legacy=0x7fed34060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:53.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.278+0000 7fed337fe700 1 -- 172.21.15.17:0/490312495 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed3405fbf0 con 0x7fed34066ba0 2022-08-01T19:41:53.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.278+0000 7fed2bfff700 1 --2- 172.21.15.17:0/490312495 >> v2:172.21.15.77:3300/0 conn(0x7fed340b2730 0x7fed340666b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:53.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.278+0000 7fed337fe700 1 --2- 172.21.15.17:0/490312495 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed34066ba0 0x7fed3414dab0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 rx=0x7fed2c0032f0 tx=0x7fed2c009290).ready entity=mon.0 client_cookie=cd93f14b2140840a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:53.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.278+0000 7fed327fc700 1 -- 172.21.15.17:0/490312495 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed2c00b870 con 0x7fed34066ba0 2022-08-01T19:41:53.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.278+0000 7fed327fc700 1 -- 172.21.15.17:0/490312495 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed2c00b9d0 con 0x7fed34066ba0 2022-08-01T19:41:53.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.278+0000 7fed327fc700 1 -- 172.21.15.17:0/490312495 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed2c00bb40 con 0x7fed34066ba0 2022-08-01T19:41:53.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.278+0000 7fed33fff700 1 -- 172.21.15.17:0/490312495 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed34066ba0 msgr2=0x7fed3414dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:53.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.278+0000 7fed33fff700 1 --2- 172.21.15.17:0/490312495 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed34066ba0 0x7fed3414dab0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 rx=0x7fed2c0032f0 tx=0x7fed2c009290).stop 2022-08-01T19:41:53.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.278+0000 7fed33fff700 1 -- 172.21.15.17:0/490312495 shutdown_connections 2022-08-01T19:41:53.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.278+0000 7fed33fff700 1 --2- 172.21.15.17:0/490312495 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed34066ba0 0x7fed3414dab0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:53.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.278+0000 7fed33fff700 1 --2- 172.21.15.17:0/490312495 >> v2:172.21.15.77:3300/0 conn(0x7fed340b2730 0x7fed340666b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:53.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.278+0000 7fed33fff700 1 -- 172.21.15.17:0/490312495 >> 172.21.15.17:0/490312495 conn(0x7fed34063870 msgr2=0x7fed34063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:53.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.278+0000 7fed33fff700 1 -- 172.21.15.17:0/490312495 shutdown_connections 2022-08-01T19:41:53.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.278+0000 7fed33fff700 1 -- 172.21.15.17:0/490312495 wait complete. 2022-08-01T19:41:53.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.282+0000 7fed33fff700 1 Processor -- start 2022-08-01T19:41:53.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.282+0000 7fed33fff700 1 -- start start 2022-08-01T19:41:53.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.282+0000 7fed33fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed34066ba0 0x7fed340a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:53.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.282+0000 7fed33fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fed340b2730 0x7fed340a2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:53.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.282+0000 7fed33fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed340b3190 0x7fed34143530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:53.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.282+0000 7fed33fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed340a41a0 con 0x7fed34066ba0 2022-08-01T19:41:53.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.282+0000 7fed33fff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed340aa0b0 con 0x7fed340b3190 2022-08-01T19:41:53.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.282+0000 7fed33fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed340a43d0 con 0x7fed340b2730 2022-08-01T19:41:53.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.282+0000 7fed32ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed34066ba0 0x7fed340a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:53.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.282+0000 7fed32ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed34066ba0 0x7fed340a5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42846/0 (socket says 172.21.15.17:42846) 2022-08-01T19:41:53.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.282+0000 7fed32ffd700 1 -- 172.21.15.17:0/2683312988 learned_addr learned my addr 172.21.15.17:0/2683312988 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:53.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.282+0000 7fed2bfff700 1 --2- 172.21.15.17:0/2683312988 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fed340b2730 0x7fed340a2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:53.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.282+0000 7fed337fe700 1 --2- 172.21.15.17:0/2683312988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed340b3190 0x7fed34143530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:53.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.282+0000 7fed32ffd700 1 -- 172.21.15.17:0/2683312988 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fed340b2730 msgr2=0x7fed340a2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:53.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.282+0000 7fed32ffd700 1 --2- 172.21.15.17:0/2683312988 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fed340b2730 0x7fed340a2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:53.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.282+0000 7fed32ffd700 1 -- 172.21.15.17:0/2683312988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed340b3190 msgr2=0x7fed34143530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:53.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.282+0000 7fed32ffd700 1 --2- 172.21.15.17:0/2683312988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed340b3190 0x7fed34143530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:53.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.282+0000 7fed32ffd700 1 -- 172.21.15.17:0/2683312988 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2c009530 con 0x7fed34066ba0 2022-08-01T19:41:53.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.282+0000 7fed2bfff700 1 --2- 172.21.15.17:0/2683312988 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fed340b2730 0x7fed340a2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:53.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.282+0000 7fed32ffd700 1 --2- 172.21.15.17:0/2683312988 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed34066ba0 0x7fed340a5e10 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 rx=0x7fed340634e0 tx=0x7fed24007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:53.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.286+0000 7fed30ff9700 1 -- 172.21.15.17:0/2683312988 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed24012400 con 0x7fed34066ba0 2022-08-01T19:41:53.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.286+0000 7fed30ff9700 1 -- 172.21.15.17:0/2683312988 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed24012560 con 0x7fed34066ba0 2022-08-01T19:41:53.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.286+0000 7fed33fff700 1 -- 172.21.15.17:0/2683312988 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed340a4670 con 0x7fed34066ba0 2022-08-01T19:41:53.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.286+0000 7fed30ff9700 1 -- 172.21.15.17:0/2683312988 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed240128c0 con 0x7fed34066ba0 2022-08-01T19:41:53.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.286+0000 7fed33fff700 1 -- 172.21.15.17:0/2683312988 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed340a3490 con 0x7fed34066ba0 2022-08-01T19:41:53.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.286+0000 7fed33fff700 1 -- 172.21.15.17:0/2683312988 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed3405fe90 con 0x7fed34066ba0 2022-08-01T19:41:53.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.286+0000 7fed30ff9700 1 -- 172.21.15.17:0/2683312988 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fed24019030 con 0x7fed34066ba0 2022-08-01T19:41:53.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.286+0000 7fed30ff9700 1 --2- 172.21.15.17:0/2683312988 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fed18053b00 0x7fed18055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:53.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.286+0000 7fed30ff9700 1 -- 172.21.15.17:0/2683312988 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fed2406b330 con 0x7fed34066ba0 2022-08-01T19:41:53.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.286+0000 7fed2bfff700 1 --2- 172.21.15.17:0/2683312988 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fed18053b00 0x7fed18055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:53.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.290+0000 7fed2bfff700 1 --2- 172.21.15.17:0/2683312988 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fed18053b00 0x7fed18055fc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7fed1c00c000 tx=0x7fed1c008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:53.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.294+0000 7fed30ff9700 1 -- 172.21.15.17:0/2683312988 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fed24042050 con 0x7fed34066ba0 2022-08-01T19:41:53.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.422+0000 7fed33fff700 1 -- 172.21.15.17:0/2683312988 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed340574e0 con 0x7fed34066ba0 2022-08-01T19:41:53.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.422+0000 7fed30ff9700 1 -- 172.21.15.17:0/2683312988 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16285 (secure 0 0 0) 0x7fed24045670 con 0x7fed34066ba0 2022-08-01T19:41:53.428 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:53.428 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":107,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:41:47.421949+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:41:45.865183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2022-08-01T19:41:46.670831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:41:53.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.422+0000 7fed33fff700 1 -- 172.21.15.17:0/2683312988 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fed18053b00 msgr2=0x7fed18055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:53.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.422+0000 7fed33fff700 1 --2- 172.21.15.17:0/2683312988 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fed18053b00 0x7fed18055fc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7fed1c00c000 tx=0x7fed1c008040).stop 2022-08-01T19:41:53.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.422+0000 7fed33fff700 1 -- 172.21.15.17:0/2683312988 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed34066ba0 msgr2=0x7fed340a5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:53.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.422+0000 7fed33fff700 1 --2- 172.21.15.17:0/2683312988 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed34066ba0 0x7fed340a5e10 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 rx=0x7fed340634e0 tx=0x7fed24007c50).stop 2022-08-01T19:41:53.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.426+0000 7fed33fff700 1 -- 172.21.15.17:0/2683312988 shutdown_connections 2022-08-01T19:41:53.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.426+0000 7fed33fff700 1 --2- 172.21.15.17:0/2683312988 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fed18053b00 0x7fed18055fc0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:53.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.426+0000 7fed33fff700 1 --2- 172.21.15.17:0/2683312988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed340b3190 0x7fed34143530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:53.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.426+0000 7fed33fff700 1 --2- 172.21.15.17:0/2683312988 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fed340b2730 0x7fed340a2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:53.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.426+0000 7fed33fff700 1 --2- 172.21.15.17:0/2683312988 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed34066ba0 0x7fed340a5e10 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:53.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.426+0000 7fed33fff700 1 -- 172.21.15.17:0/2683312988 >> 172.21.15.17:0/2683312988 conn(0x7fed34063870 msgr2=0x7fed34147870 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:53.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.426+0000 7fed33fff700 1 -- 172.21.15.17:0/2683312988 shutdown_connections 2022-08-01T19:41:53.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.426+0000 7fed33fff700 1 -- 172.21.15.17:0/2683312988 wait complete. 2022-08-01T19:41:53.439 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T19:41:45.865183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T19:41:46.670831+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T19:41:53.441 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:41:53.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.554+0000 7f621e215700 1 Processor -- start 2022-08-01T19:41:53.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.554+0000 7f621e215700 1 -- start start 2022-08-01T19:41:53.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.554+0000 7f621e215700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f62180b28a0 0x7f62181432a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:53.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.554+0000 7f621e215700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6218061360 0x7f6218143790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:53.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.554+0000 7f621e215700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6218064aa0 con 0x7f62180600f0 2022-08-01T19:41:53.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.554+0000 7f621e215700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6218064d40 con 0x7f6218061360 2022-08-01T19:41:53.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.554+0000 7f621e215700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6218064fe0 con 0x7f62180b28a0 2022-08-01T19:41:53.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.554+0000 7f6217fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6218061360 0x7f6218143790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:53.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.554+0000 7f62177fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f62180600f0 0x7f62181405d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35048/0 (socket says 172.21.15.17:35048) 2022-08-01T19:41:53.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.554+0000 7f62177fe700 1 -- 172.21.15.17:0/1289660680 learned_addr learned my addr 172.21.15.17:0/1289660680 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:53.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.554+0000 7f6216ffd700 1 --2- 172.21.15.17:0/1289660680 >> v2:172.21.15.77:3300/0 conn(0x7f62180b28a0 0x7f62181432a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:53.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.554+0000 7f6217fff700 1 -- 172.21.15.17:0/1289660680 >> v2:172.21.15.77:3300/0 conn(0x7f62180b28a0 msgr2=0x7f62181432a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:53.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.554+0000 7f6217fff700 1 --2- 172.21.15.17:0/1289660680 >> v2:172.21.15.77:3300/0 conn(0x7f62180b28a0 0x7f62181432a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:53.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.554+0000 7f6217fff700 1 -- 172.21.15.17:0/1289660680 >> v1:172.21.15.17:6789/0 conn(0x7f62180600f0 legacy=0x7f62181405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:53.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.554+0000 7f6217fff700 1 -- 172.21.15.17:0/1289660680 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6218065280 con 0x7f6218061360 2022-08-01T19:41:53.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.554+0000 7f6217fff700 1 --2- 172.21.15.17:0/1289660680 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6218061360 0x7f6218143790 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 rx=0x7f62100032f0 tx=0x7f62100091d0).ready entity=mon.0 client_cookie=d9ad09ebf81c11fd server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:53.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.558+0000 7f62167fc700 1 -- 172.21.15.17:0/1289660680 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6210009ea0 con 0x7f6218061360 2022-08-01T19:41:53.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.558+0000 7f62167fc700 1 -- 172.21.15.17:0/1289660680 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621000b870 con 0x7f6218061360 2022-08-01T19:41:53.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.558+0000 7f62167fc700 1 -- 172.21.15.17:0/1289660680 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f621000b9e0 con 0x7f6218061360 2022-08-01T19:41:53.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.558+0000 7f621e215700 1 -- 172.21.15.17:0/1289660680 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6218061360 msgr2=0x7f6218143790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:53.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.558+0000 7f621e215700 1 --2- 172.21.15.17:0/1289660680 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6218061360 0x7f6218143790 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 rx=0x7f62100032f0 tx=0x7f62100091d0).stop 2022-08-01T19:41:53.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.558+0000 7f621e215700 1 -- 172.21.15.17:0/1289660680 shutdown_connections 2022-08-01T19:41:53.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.558+0000 7f621e215700 1 --2- 172.21.15.17:0/1289660680 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6218061360 0x7f6218143790 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:53.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.558+0000 7f621e215700 1 --2- 172.21.15.17:0/1289660680 >> v2:172.21.15.77:3300/0 conn(0x7f62180b28a0 0x7f62181432a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:53.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.558+0000 7f621e215700 1 -- 172.21.15.17:0/1289660680 >> 172.21.15.17:0/1289660680 conn(0x7f621805d0f0 msgr2=0x7f621805f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:53.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.558+0000 7f621e215700 1 -- 172.21.15.17:0/1289660680 shutdown_connections 2022-08-01T19:41:53.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.558+0000 7f621e215700 1 -- 172.21.15.17:0/1289660680 wait complete. 2022-08-01T19:41:53.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.558+0000 7f621e215700 1 Processor -- start 2022-08-01T19:41:53.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.558+0000 7f621e215700 1 -- start start 2022-08-01T19:41:53.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.562+0000 7f621e215700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62180600f0 0x7f62181412c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:53.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.562+0000 7f621e215700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6218061360 0x7f62181417b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:53.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.562+0000 7f621e215700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62180b28a0 0x7f62180acff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:53.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.562+0000 7f621e215700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62180ac400 con 0x7f62180600f0 2022-08-01T19:41:53.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.562+0000 7f621e215700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62180aca10 con 0x7f62180b28a0 2022-08-01T19:41:53.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.562+0000 7f621e215700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62180b1b60 con 0x7f6218061360 2022-08-01T19:41:53.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.562+0000 7f62177fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62180600f0 0x7f62181412c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:53.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.562+0000 7f62177fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62180600f0 0x7f62181412c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42860/0 (socket says 172.21.15.17:42860) 2022-08-01T19:41:53.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.562+0000 7f62177fe700 1 -- 172.21.15.17:0/2314155070 learned_addr learned my addr 172.21.15.17:0/2314155070 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:53.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.562+0000 7f6217fff700 1 --2- 172.21.15.17:0/2314155070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62180b28a0 0x7f62180acff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:53.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.562+0000 7f6216ffd700 1 --2- 172.21.15.17:0/2314155070 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6218061360 0x7f62181417b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:53.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.562+0000 7f62177fe700 1 -- 172.21.15.17:0/2314155070 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6218061360 msgr2=0x7f62181417b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:53.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.562+0000 7f62177fe700 1 --2- 172.21.15.17:0/2314155070 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6218061360 0x7f62181417b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:53.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.562+0000 7f62177fe700 1 -- 172.21.15.17:0/2314155070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62180b28a0 msgr2=0x7f62180acff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:53.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.562+0000 7f62177fe700 1 --2- 172.21.15.17:0/2314155070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62180b28a0 0x7f62180acff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:53.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.562+0000 7f62177fe700 1 -- 172.21.15.17:0/2314155070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6210009470 con 0x7f62180600f0 2022-08-01T19:41:53.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.562+0000 7f6217fff700 1 --2- 172.21.15.17:0/2314155070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62180b28a0 0x7f62180acff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:53.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.562+0000 7f62177fe700 1 --2- 172.21.15.17:0/2314155070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62180600f0 0x7f62181412c0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 rx=0x7f6218062940 tx=0x7f620c007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:53.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.562+0000 7f6214ff9700 1 -- 172.21.15.17:0/2314155070 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f620c00ce30 con 0x7f62180600f0 2022-08-01T19:41:53.602 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.562+0000 7f6214ff9700 1 -- 172.21.15.17:0/2314155070 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f620c0072e0 con 0x7f62180600f0 2022-08-01T19:41:53.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.562+0000 7f621e215700 1 -- 172.21.15.17:0/2314155070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6218143930 con 0x7f62180600f0 2022-08-01T19:41:53.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.562+0000 7f621e215700 1 -- 172.21.15.17:0/2314155070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6218144140 con 0x7f62180600f0 2022-08-01T19:41:53.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.562+0000 7f6214ff9700 1 -- 172.21.15.17:0/2314155070 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f620c007450 con 0x7f62180600f0 2022-08-01T19:41:53.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.562+0000 7f621e215700 1 -- 172.21.15.17:0/2314155070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62180ac600 con 0x7f62180600f0 2022-08-01T19:41:53.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.562+0000 7f6214ff9700 1 -- 172.21.15.17:0/2314155070 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f620c010070 con 0x7f62180600f0 2022-08-01T19:41:53.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.566+0000 7f6214ff9700 1 --2- 172.21.15.17:0/2314155070 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f61f8053b30 0x7f61f8055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:53.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.566+0000 7f6214ff9700 1 -- 172.21.15.17:0/2314155070 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f620c06b5f0 con 0x7f62180600f0 2022-08-01T19:41:53.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.566+0000 7f6216ffd700 1 --2- 172.21.15.17:0/2314155070 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f61f8053b30 0x7f61f8055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:53.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.566+0000 7f6216ffd700 1 --2- 172.21.15.17:0/2314155070 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f61f8053b30 0x7f61f8055ff0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f620800c000 tx=0x7f6208008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:53.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.570+0000 7f6214ff9700 1 -- 172.21.15.17:0/2314155070 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f620c042220 con 0x7f62180600f0 2022-08-01T19:41:53.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.698+0000 7f621e215700 1 -- 172.21.15.17:0/2314155070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f62180604d0 con 0x7f62180600f0 2022-08-01T19:41:53.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.698+0000 7f6214ff9700 1 -- 172.21.15.17:0/2314155070 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v107) v1 ==== 110+0+10 (secure 0 0 0) 0x7f620c045840 con 0x7f62180600f0 2022-08-01T19:41:53.710 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:41:53.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.698+0000 7f621e215700 1 -- 172.21.15.17:0/2314155070 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f61f8053b30 msgr2=0x7f61f8055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:53.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.698+0000 7f621e215700 1 --2- 172.21.15.17:0/2314155070 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f61f8053b30 0x7f61f8055ff0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f620800c000 tx=0x7f6208008040).stop 2022-08-01T19:41:53.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.698+0000 7f621e215700 1 -- 172.21.15.17:0/2314155070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62180600f0 msgr2=0x7f62181412c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:53.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.698+0000 7f621e215700 1 --2- 172.21.15.17:0/2314155070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62180600f0 0x7f62181412c0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 rx=0x7f6218062940 tx=0x7f620c007c00).stop 2022-08-01T19:41:53.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.702+0000 7f621e215700 1 -- 172.21.15.17:0/2314155070 shutdown_connections 2022-08-01T19:41:53.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.702+0000 7f621e215700 1 --2- 172.21.15.17:0/2314155070 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f61f8053b30 0x7f61f8055ff0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:53.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.702+0000 7f621e215700 1 --2- 172.21.15.17:0/2314155070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62180b28a0 0x7f62180acff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:53.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.702+0000 7f621e215700 1 --2- 172.21.15.17:0/2314155070 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6218061360 0x7f62181417b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:53.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.702+0000 7f621e215700 1 --2- 172.21.15.17:0/2314155070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62180600f0 0x7f62181412c0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:53.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.702+0000 7f621e215700 1 -- 172.21.15.17:0/2314155070 >> 172.21.15.17:0/2314155070 conn(0x7f621805d0f0 msgr2=0x7f6218066190 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:53.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.702+0000 7f621e215700 1 -- 172.21.15.17:0/2314155070 shutdown_connections 2022-08-01T19:41:53.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.702+0000 7f621e215700 1 -- 172.21.15.17:0/2314155070 wait complete. 2022-08-01T19:41:53.724 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:41:53.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.834+0000 7f2464eac700 1 Processor -- start 2022-08-01T19:41:53.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.834+0000 7f2464eac700 1 -- start start 2022-08-01T19:41:53.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.834+0000 7f2464eac700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f24600ad750 0x7f2460061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:53.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.834+0000 7f2464eac700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24601405d0 0x7f2460140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:53.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.834+0000 7f2464eac700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24600aee80 con 0x7f24600614b0 2022-08-01T19:41:53.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.834+0000 7f2464eac700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24600b70b0 con 0x7f24600ad750 2022-08-01T19:41:53.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.834+0000 7f2464eac700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24600af0b0 con 0x7f24601405d0 2022-08-01T19:41:53.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.834+0000 7f245effd700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f24600ad750 0x7f2460061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:53.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.834+0000 7f245ffff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24601405d0 0x7f2460140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:53.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.834+0000 7f245effd700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f24600ad750 0x7f2460061890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:40792/0 (socket says 172.21.15.17:40792) 2022-08-01T19:41:53.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.834+0000 7f245f7fe700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7f24600614b0 0x7f24600b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:51862/0 (socket says 172.21.15.17:51862) 2022-08-01T19:41:53.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.834+0000 7f245effd700 1 -- 172.21.15.17:0/1626072032 learned_addr learned my addr 172.21.15.17:0/1626072032 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:53.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.838+0000 7f245effd700 1 -- 172.21.15.17:0/1626072032 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24601405d0 msgr2=0x7f2460140a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:53.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.838+0000 7f245effd700 1 --2- 172.21.15.17:0/1626072032 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24601405d0 0x7f2460140a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:53.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.838+0000 7f245effd700 1 -- 172.21.15.17:0/1626072032 >> v1:172.21.15.77:6789/0 conn(0x7f24600614b0 legacy=0x7f24600b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:53.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.838+0000 7f245effd700 1 -- 172.21.15.17:0/1626072032 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24600af350 con 0x7f24600ad750 2022-08-01T19:41:53.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.838+0000 7f245ffff700 1 --2- 172.21.15.17:0/1626072032 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24601405d0 0x7f2460140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:53.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.838+0000 7f245effd700 1 --2- 172.21.15.17:0/1626072032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24600ad750 0x7f2460061890 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f2448002f70 tx=0x7f2448006eb0).ready entity=mon.1 client_cookie=d8cda380661f7385 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:53.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.838+0000 7f245e7fc700 1 -- 172.21.15.17:0/1626072032 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f244800d570 con 0x7f24600ad750 2022-08-01T19:41:53.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.838+0000 7f245e7fc700 1 -- 172.21.15.17:0/1626072032 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2448000e90 con 0x7f24600ad750 2022-08-01T19:41:53.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.838+0000 7f245e7fc700 1 -- 172.21.15.17:0/1626072032 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f244800d900 con 0x7f24600ad750 2022-08-01T19:41:53.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.838+0000 7f2464eac700 1 -- 172.21.15.17:0/1626072032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24600ad750 msgr2=0x7f2460061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:53.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.838+0000 7f2464eac700 1 --2- 172.21.15.17:0/1626072032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24600ad750 0x7f2460061890 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f2448002f70 tx=0x7f2448006eb0).stop 2022-08-01T19:41:53.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.838+0000 7f2464eac700 1 -- 172.21.15.17:0/1626072032 shutdown_connections 2022-08-01T19:41:53.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.838+0000 7f2464eac700 1 --2- 172.21.15.17:0/1626072032 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24601405d0 0x7f2460140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:53.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.838+0000 7f2464eac700 1 --2- 172.21.15.17:0/1626072032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24600ad750 0x7f2460061890 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:53.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.838+0000 7f2464eac700 1 -- 172.21.15.17:0/1626072032 >> 172.21.15.17:0/1626072032 conn(0x7f2460066f70 msgr2=0x7f2460067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:53.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.838+0000 7f2464eac700 1 -- 172.21.15.17:0/1626072032 shutdown_connections 2022-08-01T19:41:53.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.838+0000 7f2464eac700 1 -- 172.21.15.17:0/1626072032 wait complete. 2022-08-01T19:41:53.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.838+0000 7f2464eac700 1 Processor -- start 2022-08-01T19:41:53.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.838+0000 7f2464eac700 1 -- start start 2022-08-01T19:41:53.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.842+0000 7f2464eac700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24600614b0 0x7f246005d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:53.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.842+0000 7f2464eac700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24600ad750 0x7f2460152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:53.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.842+0000 7f2464eac700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24601405d0 0x7f2460152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:53.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.842+0000 7f2464eac700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24600aac80 con 0x7f24600614b0 2022-08-01T19:41:53.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.842+0000 7f2464eac700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24600aa2b0 con 0x7f24601405d0 2022-08-01T19:41:53.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.842+0000 7f2464eac700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2460152650 con 0x7f24600ad750 2022-08-01T19:41:53.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.842+0000 7f245f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24600614b0 0x7f246005d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:53.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.842+0000 7f245f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24600614b0 0x7f246005d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42874/0 (socket says 172.21.15.17:42874) 2022-08-01T19:41:53.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.842+0000 7f245f7fe700 1 -- 172.21.15.17:0/826515818 learned_addr learned my addr 172.21.15.17:0/826515818 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:53.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.842+0000 7f245effd700 1 --2- 172.21.15.17:0/826515818 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24600ad750 0x7f2460152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:53.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.842+0000 7f245ffff700 1 --2- 172.21.15.17:0/826515818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24601405d0 0x7f2460152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:53.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.842+0000 7f245f7fe700 1 -- 172.21.15.17:0/826515818 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24600ad750 msgr2=0x7f2460152af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:53.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.842+0000 7f245f7fe700 1 --2- 172.21.15.17:0/826515818 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24600ad750 0x7f2460152af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:53.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.842+0000 7f245f7fe700 1 -- 172.21.15.17:0/826515818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24601405d0 msgr2=0x7f2460152fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:53.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.842+0000 7f245f7fe700 1 --2- 172.21.15.17:0/826515818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24601405d0 0x7f2460152fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:53.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.842+0000 7f245f7fe700 1 -- 172.21.15.17:0/826515818 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2458009470 con 0x7f24600614b0 2022-08-01T19:41:53.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.842+0000 7f245f7fe700 1 --2- 172.21.15.17:0/826515818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24600614b0 0x7f246005d100 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 rx=0x7f24600af970 tx=0x7f2450007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:53.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.842+0000 7f245cff9700 1 -- 172.21.15.17:0/826515818 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24500070d0 con 0x7f24600614b0 2022-08-01T19:41:53.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.842+0000 7f245cff9700 1 -- 172.21.15.17:0/826515818 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2450007230 con 0x7f24600614b0 2022-08-01T19:41:53.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.842+0000 7f2464eac700 1 -- 172.21.15.17:0/826515818 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f244800b1e0 con 0x7f24600614b0 2022-08-01T19:41:53.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.842+0000 7f245cff9700 1 -- 172.21.15.17:0/826515818 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2450005550 con 0x7f24600614b0 2022-08-01T19:41:53.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.842+0000 7f2464eac700 1 -- 172.21.15.17:0/826515818 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24601456e0 con 0x7f24600614b0 2022-08-01T19:41:53.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.842+0000 7f2464eac700 1 -- 172.21.15.17:0/826515818 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24600aa4b0 con 0x7f24600614b0 2022-08-01T19:41:53.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.846+0000 7f245cff9700 1 -- 172.21.15.17:0/826515818 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2450010070 con 0x7f24600614b0 2022-08-01T19:41:53.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.846+0000 7f245cff9700 1 --2- 172.21.15.17:0/826515818 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2444053950 0x7f2444055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:53.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.846+0000 7f245cff9700 1 -- 172.21.15.17:0/826515818 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f245006b760 con 0x7f24600614b0 2022-08-01T19:41:53.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.846+0000 7f245effd700 1 --2- 172.21.15.17:0/826515818 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2444053950 0x7f2444055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:53.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.846+0000 7f245effd700 1 --2- 172.21.15.17:0/826515818 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2444053950 0x7f2444055e10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f244801c000 tx=0x7f2448015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:53.858 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.850+0000 7f245cff9700 1 -- 172.21.15.17:0/826515818 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f24500424a0 con 0x7f24600614b0 2022-08-01T19:41:53.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.974+0000 7f2464eac700 1 -- 172.21.15.17:0/826515818 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2460155c00 con 0x7f24600614b0 2022-08-01T19:41:53.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.974+0000 7f245cff9700 1 -- 172.21.15.17:0/826515818 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v107) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2450045ac0 con 0x7f24600614b0 2022-08-01T19:41:53.979 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:41:53.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.978+0000 7f2464eac700 1 -- 172.21.15.17:0/826515818 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2444053950 msgr2=0x7f2444055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:53.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.978+0000 7f2464eac700 1 --2- 172.21.15.17:0/826515818 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2444053950 0x7f2444055e10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f244801c000 tx=0x7f2448015000).stop 2022-08-01T19:41:53.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.978+0000 7f2464eac700 1 -- 172.21.15.17:0/826515818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24600614b0 msgr2=0x7f246005d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:53.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.978+0000 7f2464eac700 1 --2- 172.21.15.17:0/826515818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24600614b0 0x7f246005d100 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 rx=0x7f24600af970 tx=0x7f2450007c40).stop 2022-08-01T19:41:53.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.978+0000 7f2464eac700 1 -- 172.21.15.17:0/826515818 shutdown_connections 2022-08-01T19:41:53.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.978+0000 7f2464eac700 1 --2- 172.21.15.17:0/826515818 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2444053950 0x7f2444055e10 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:53.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.978+0000 7f2464eac700 1 --2- 172.21.15.17:0/826515818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24601405d0 0x7f2460152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:53.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.978+0000 7f2464eac700 1 --2- 172.21.15.17:0/826515818 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24600ad750 0x7f2460152af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:53.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.978+0000 7f2464eac700 1 --2- 172.21.15.17:0/826515818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24600614b0 0x7f246005d100 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:53.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.978+0000 7f2464eac700 1 -- 172.21.15.17:0/826515818 >> 172.21.15.17:0/826515818 conn(0x7f2460066f70 msgr2=0x7f24600b7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:53.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.978+0000 7f2464eac700 1 -- 172.21.15.17:0/826515818 shutdown_connections 2022-08-01T19:41:54.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:53.978+0000 7f2464eac700 1 -- 172.21.15.17:0/826515818 wait complete. 2022-08-01T19:41:54.007 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T19:41:54.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.122+0000 7f96ac823700 1 Processor -- start 2022-08-01T19:41:54.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.122+0000 7f96ac823700 1 -- start start 2022-08-01T19:41:54.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.122+0000 7f96ac823700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f96a8060330 0x7f96a80b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:54.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.122+0000 7f96ac823700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96a805c6f0 0x7f96a80a8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:54.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.122+0000 7f96ac823700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96a80aa630 con 0x7f96a80af550 2022-08-01T19:41:54.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.122+0000 7f96ac823700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96a80b15e0 con 0x7f96a8060330 2022-08-01T19:41:54.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.122+0000 7f96ac823700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96a80af930 con 0x7f96a805c6f0 2022-08-01T19:41:54.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.122+0000 7f96a7fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96a805c6f0 0x7f96a80a8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:54.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.122+0000 7f96a77fe700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7f96a80af550 0x7f96a81445e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:51876/0 (socket says 172.21.15.17:51876) 2022-08-01T19:41:54.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.122+0000 7f96a77fe700 1 -- 172.21.15.17:0/4008481875 learned_addr learned my addr 172.21.15.17:0/4008481875 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:54.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.122+0000 7f96a6ffd700 1 --2- 172.21.15.17:0/4008481875 >> v2:172.21.15.77:3300/0 conn(0x7f96a8060330 0x7f96a80b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:54.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.122+0000 7f96a7fff700 1 -- 172.21.15.17:0/4008481875 >> v2:172.21.15.77:3300/0 conn(0x7f96a8060330 msgr2=0x7f96a80b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:54.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.122+0000 7f96a7fff700 1 --2- 172.21.15.17:0/4008481875 >> v2:172.21.15.77:3300/0 conn(0x7f96a8060330 0x7f96a80b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:54.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.122+0000 7f96a67fc700 1 -- 172.21.15.17:0/4008481875 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2325014553 0 0) 0x7f96a80aa630 con 0x7f96a80af550 2022-08-01T19:41:54.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.122+0000 7f96a7fff700 1 -- 172.21.15.17:0/4008481875 >> v1:172.21.15.77:6789/0 conn(0x7f96a80af550 legacy=0x7f96a81445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:54.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.122+0000 7f96a7fff700 1 -- 172.21.15.17:0/4008481875 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a805fd10 con 0x7f96a805c6f0 2022-08-01T19:41:54.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.122+0000 7f96a7fff700 1 --2- 172.21.15.17:0/4008481875 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96a805c6f0 0x7f96a80a8d80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f96a00032f0 tx=0x7f96a00091d0).ready entity=mon.2 client_cookie=ed5af06d25550c1c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:54.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.122+0000 7f96a67fc700 1 -- 172.21.15.17:0/4008481875 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96a000b6c0 con 0x7f96a805c6f0 2022-08-01T19:41:54.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.126+0000 7f96a67fc700 1 -- 172.21.15.17:0/4008481875 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a000b820 con 0x7f96a805c6f0 2022-08-01T19:41:54.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.126+0000 7f96a67fc700 1 -- 172.21.15.17:0/4008481875 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96a000b990 con 0x7f96a805c6f0 2022-08-01T19:41:54.154 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.126+0000 7f96ac823700 1 -- 172.21.15.17:0/4008481875 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96a805c6f0 msgr2=0x7f96a80a8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:54.154 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.126+0000 7f96ac823700 1 --2- 172.21.15.17:0/4008481875 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96a805c6f0 0x7f96a80a8d80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f96a00032f0 tx=0x7f96a00091d0).stop 2022-08-01T19:41:54.154 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.126+0000 7f96ac823700 1 -- 172.21.15.17:0/4008481875 shutdown_connections 2022-08-01T19:41:54.154 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.126+0000 7f96ac823700 1 --2- 172.21.15.17:0/4008481875 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96a805c6f0 0x7f96a80a8d80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:54.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.126+0000 7f96ac823700 1 --2- 172.21.15.17:0/4008481875 >> v2:172.21.15.77:3300/0 conn(0x7f96a8060330 0x7f96a80b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:54.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.126+0000 7f96ac823700 1 -- 172.21.15.17:0/4008481875 >> 172.21.15.17:0/4008481875 conn(0x7f96a80612d0 msgr2=0x7f96a8062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:54.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.126+0000 7f96ac823700 1 -- 172.21.15.17:0/4008481875 shutdown_connections 2022-08-01T19:41:54.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.126+0000 7f96ac823700 1 -- 172.21.15.17:0/4008481875 wait complete. 2022-08-01T19:41:54.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.126+0000 7f96ac823700 1 Processor -- start 2022-08-01T19:41:54.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.126+0000 7f96ac823700 1 -- start start 2022-08-01T19:41:54.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.130+0000 7f96ac823700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96a805c6f0 0x7f96a80adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:54.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.130+0000 7f96ac823700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96a8060330 0x7f96a8140940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:54.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.130+0000 7f96ac823700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96a80af550 0x7f96a8140e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:54.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.130+0000 7f96ac823700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a80ad390 con 0x7f96a80af550 2022-08-01T19:41:54.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.130+0000 7f96ac823700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a8144880 con 0x7f96a8060330 2022-08-01T19:41:54.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.130+0000 7f96ac823700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96a8144b20 con 0x7f96a805c6f0 2022-08-01T19:41:54.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.130+0000 7f96a7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96a80af550 0x7f96a8140e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:54.158 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.130+0000 7f96a7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96a80af550 0x7f96a8140e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42892/0 (socket says 172.21.15.17:42892) 2022-08-01T19:41:54.158 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.130+0000 7f96a7fff700 1 -- 172.21.15.17:0/3798884977 learned_addr learned my addr 172.21.15.17:0/3798884977 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:54.158 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.130+0000 7f96a6ffd700 1 --2- 172.21.15.17:0/3798884977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96a8060330 0x7f96a8140940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:54.158 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.130+0000 7f96a77fe700 1 --2- 172.21.15.17:0/3798884977 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96a805c6f0 0x7f96a80adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:54.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.130+0000 7f96a7fff700 1 -- 172.21.15.17:0/3798884977 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96a805c6f0 msgr2=0x7f96a80adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:54.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.130+0000 7f96a7fff700 1 --2- 172.21.15.17:0/3798884977 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96a805c6f0 0x7f96a80adcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:54.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.130+0000 7f96a7fff700 1 -- 172.21.15.17:0/3798884977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96a8060330 msgr2=0x7f96a8140940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:54.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.130+0000 7f96a7fff700 1 --2- 172.21.15.17:0/3798884977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96a8060330 0x7f96a8140940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:54.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.130+0000 7f96a7fff700 1 -- 172.21.15.17:0/3798884977 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a0009470 con 0x7f96a80af550 2022-08-01T19:41:54.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.130+0000 7f96a6ffd700 1 --2- 172.21.15.17:0/3798884977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96a8060330 0x7f96a8140940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:54.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.130+0000 7f96a7fff700 1 --2- 172.21.15.17:0/3798884977 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96a80af550 0x7f96a8140e30 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 rx=0x7f96a00032f0 tx=0x7f96a0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:54.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.130+0000 7f96a4ff9700 1 -- 172.21.15.17:0/3798884977 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96a000b6c0 con 0x7f96a80af550 2022-08-01T19:41:54.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.130+0000 7f96a4ff9700 1 -- 172.21.15.17:0/3798884977 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a0021630 con 0x7f96a80af550 2022-08-01T19:41:54.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.130+0000 7f96a4ff9700 1 -- 172.21.15.17:0/3798884977 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96a0021990 con 0x7f96a80af550 2022-08-01T19:41:54.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.130+0000 7f96ac823700 1 -- 172.21.15.17:0/3798884977 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96a805f840 con 0x7f96a80af550 2022-08-01T19:41:54.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.130+0000 7f96ac823700 1 -- 172.21.15.17:0/3798884977 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a8145060 con 0x7f96a80af550 2022-08-01T19:41:54.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.130+0000 7f96a4ff9700 1 -- 172.21.15.17:0/3798884977 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f96a000f070 con 0x7f96a80af550 2022-08-01T19:41:54.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.130+0000 7f96ac823700 1 -- 172.21.15.17:0/3798884977 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96a8145260 con 0x7f96a80af550 2022-08-01T19:41:54.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.134+0000 7f96a4ff9700 1 --2- 172.21.15.17:0/3798884977 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9688053a70 0x7f9688055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:54.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.134+0000 7f96a4ff9700 1 -- 172.21.15.17:0/3798884977 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f96a006cd10 con 0x7f96a80af550 2022-08-01T19:41:54.170 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.134+0000 7f96a77fe700 1 --2- 172.21.15.17:0/3798884977 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9688053a70 0x7f9688055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:54.170 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.134+0000 7f96a77fe700 1 --2- 172.21.15.17:0/3798884977 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9688053a70 0x7f9688055f30 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f969800d000 tx=0x7f9698009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:54.170 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.138+0000 7f96a4ff9700 1 -- 172.21.15.17:0/3798884977 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f96a8145260 con 0x7f96a80af550 2022-08-01T19:41:54.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.262+0000 7f96ac823700 1 -- 172.21.15.17:0/3798884977 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f96a8145260 con 0x7f96a80af550 2022-08-01T19:41:54.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.266+0000 7f96a4ff9700 1 -- 172.21.15.17:0/3798884977 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v107) v1 ==== 104+0+10 (secure 0 0 0) 0x7f96a8145260 con 0x7f96a80af550 2022-08-01T19:41:54.268 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:41:54.272 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.266+0000 7f96ac823700 1 -- 172.21.15.17:0/3798884977 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9688053a70 msgr2=0x7f9688055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:54.272 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.266+0000 7f96ac823700 1 --2- 172.21.15.17:0/3798884977 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9688053a70 0x7f9688055f30 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f969800d000 tx=0x7f9698009040).stop 2022-08-01T19:41:54.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.266+0000 7f96ac823700 1 -- 172.21.15.17:0/3798884977 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96a80af550 msgr2=0x7f96a8140e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:54.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.266+0000 7f96ac823700 1 --2- 172.21.15.17:0/3798884977 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96a80af550 0x7f96a8140e30 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 rx=0x7f96a00032f0 tx=0x7f96a0016000).stop 2022-08-01T19:41:54.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.266+0000 7f96ac823700 1 -- 172.21.15.17:0/3798884977 shutdown_connections 2022-08-01T19:41:54.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.266+0000 7f96ac823700 1 --2- 172.21.15.17:0/3798884977 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9688053a70 0x7f9688055f30 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:54.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.266+0000 7f96ac823700 1 --2- 172.21.15.17:0/3798884977 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96a80af550 0x7f96a8140e30 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:54.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.266+0000 7f96ac823700 1 --2- 172.21.15.17:0/3798884977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96a8060330 0x7f96a8140940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:54.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.266+0000 7f96ac823700 1 --2- 172.21.15.17:0/3798884977 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f96a805c6f0 0x7f96a80adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:54.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.266+0000 7f96ac823700 1 -- 172.21.15.17:0/3798884977 >> 172.21.15.17:0/3798884977 conn(0x7f96a80612d0 msgr2=0x7f96a80b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:54.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.266+0000 7f96ac823700 1 -- 172.21.15.17:0/3798884977 shutdown_connections 2022-08-01T19:41:54.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.270+0000 7f96ac823700 1 -- 172.21.15.17:0/3798884977 wait complete. 2022-08-01T19:41:54.285 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T19:41:54.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.386+0000 7f4ffbfff700 1 Processor -- start 2022-08-01T19:41:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.386+0000 7f4ffbfff700 1 -- start start 2022-08-01T19:41:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.390+0000 7f4ffbfff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4ffc0ad750 0x7f4ffc061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:54.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.390+0000 7f4ffbfff700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ffc0aee80 con 0x7f4ffc0614b0 2022-08-01T19:41:54.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.390+0000 7f4ffbfff700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ffc0b70b0 con 0x7f4ffc1405d0 2022-08-01T19:41:54.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.390+0000 7f4ffbfff700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ffc0af0b0 con 0x7f4ffc0ad750 2022-08-01T19:41:54.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.390+0000 7f4ff27fc700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4ffc0ad750 0x7f4ffc061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:54.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.390+0000 7f4ffaffd700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f4ffc0614b0 0x7f4ffc0b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35090/0 (socket says 172.21.15.17:35090) 2022-08-01T19:41:54.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.390+0000 7f4ffaffd700 1 -- 172.21.15.17:0/1304549247 learned_addr learned my addr 172.21.15.17:0/1304549247 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:54.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.390+0000 7f4ffa7fc700 1 -- 172.21.15.17:0/1304549247 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 253954298 0 0) 0x7f4ffc0aee80 con 0x7f4ffc0614b0 2022-08-01T19:41:54.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.390+0000 7f4ffa7fc700 1 -- 172.21.15.17:0/1304549247 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fdc003260 con 0x7f4ffc0614b0 2022-08-01T19:41:54.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.390+0000 7f4ff27fc700 1 -- 172.21.15.17:0/1304549247 >> v1:172.21.15.77:6789/0 conn(0x7f4ffc1405d0 legacy=0x7f4ffc1466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:54.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.390+0000 7f4ff27fc700 1 -- 172.21.15.17:0/1304549247 >> v1:172.21.15.17:6789/0 conn(0x7f4ffc0614b0 legacy=0x7f4ffc0b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:54.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.390+0000 7f4ff27fc700 1 -- 172.21.15.17:0/1304549247 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ffc0af350 con 0x7f4ffc0ad750 2022-08-01T19:41:54.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.390+0000 7f4ff27fc700 1 --2- 172.21.15.17:0/1304549247 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ffc0ad750 0x7f4ffc061890 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 rx=0x7f4fe4002f70 tx=0x7f4fe4006df0).ready entity=mon.0 client_cookie=a398d200fda5ec4a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:54.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.390+0000 7f4ffa7fc700 1 -- 172.21.15.17:0/1304549247 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fe400d610 con 0x7f4ffc0ad750 2022-08-01T19:41:54.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.390+0000 7f4ffa7fc700 1 -- 172.21.15.17:0/1304549247 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe4000e90 con 0x7f4ffc0ad750 2022-08-01T19:41:54.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.390+0000 7f4ffa7fc700 1 -- 172.21.15.17:0/1304549247 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fe400d960 con 0x7f4ffc0ad750 2022-08-01T19:41:54.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.390+0000 7f4ffbfff700 1 -- 172.21.15.17:0/1304549247 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ffc0ad750 msgr2=0x7f4ffc061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:54.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.390+0000 7f4ffbfff700 1 --2- 172.21.15.17:0/1304549247 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ffc0ad750 0x7f4ffc061890 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 rx=0x7f4fe4002f70 tx=0x7f4fe4006df0).stop 2022-08-01T19:41:54.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.390+0000 7f4ffbfff700 1 -- 172.21.15.17:0/1304549247 shutdown_connections 2022-08-01T19:41:54.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.390+0000 7f4ffbfff700 1 --2- 172.21.15.17:0/1304549247 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ffc0ad750 0x7f4ffc061890 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:54.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.390+0000 7f4ffbfff700 1 -- 172.21.15.17:0/1304549247 >> 172.21.15.17:0/1304549247 conn(0x7f4ffc066f70 msgr2=0x7f4ffc067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:54.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.390+0000 7f4ffbfff700 1 -- 172.21.15.17:0/1304549247 shutdown_connections 2022-08-01T19:41:54.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.390+0000 7f4ffbfff700 1 -- 172.21.15.17:0/1304549247 wait complete. 2022-08-01T19:41:54.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.394+0000 7f4ffbfff700 1 Processor -- start 2022-08-01T19:41:54.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.394+0000 7f4ffbfff700 1 -- start start 2022-08-01T19:41:54.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.394+0000 7f4ffbfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ffc0614b0 0x7f4ffc09ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:54.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.394+0000 7f4ffbfff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4ffc0ad750 0x7f4ffc09f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:54.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.394+0000 7f4ffbfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ffc1405d0 0x7f4ffc0b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:54.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.394+0000 7f4ffbfff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc0a9f70 con 0x7f4ffc0614b0 2022-08-01T19:41:54.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.394+0000 7f4ffbfff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc0b0af0 con 0x7f4ffc1405d0 2022-08-01T19:41:54.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.394+0000 7f4ffbfff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc0b0f20 con 0x7f4ffc0ad750 2022-08-01T19:41:54.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.394+0000 7f4ffaffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ffc0614b0 0x7f4ffc09ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:54.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.394+0000 7f4ffaffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ffc0614b0 0x7f4ffc09ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42904/0 (socket says 172.21.15.17:42904) 2022-08-01T19:41:54.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.394+0000 7f4ffaffd700 1 -- 172.21.15.17:0/3430412824 learned_addr learned my addr 172.21.15.17:0/3430412824 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:54.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.394+0000 7f4ffb7fe700 1 --2- 172.21.15.17:0/3430412824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ffc1405d0 0x7f4ffc0b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:54.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.394+0000 7f4ff27fc700 1 --2- 172.21.15.17:0/3430412824 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4ffc0ad750 0x7f4ffc09f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:54.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.394+0000 7f4ffaffd700 1 -- 172.21.15.17:0/3430412824 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4ffc0ad750 msgr2=0x7f4ffc09f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:54.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.394+0000 7f4ffaffd700 1 --2- 172.21.15.17:0/3430412824 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4ffc0ad750 0x7f4ffc09f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:54.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.394+0000 7f4ffaffd700 1 -- 172.21.15.17:0/3430412824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ffc1405d0 msgr2=0x7f4ffc0b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:54.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.394+0000 7f4ffaffd700 1 --2- 172.21.15.17:0/3430412824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ffc1405d0 0x7f4ffc0b1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:54.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.394+0000 7f4ffaffd700 1 -- 172.21.15.17:0/3430412824 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe400b180 con 0x7f4ffc0614b0 2022-08-01T19:41:54.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.394+0000 7f4ffaffd700 1 --2- 172.21.15.17:0/3430412824 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ffc0614b0 0x7f4ffc09ebe0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 rx=0x7f4fec001110 tx=0x7f4fec007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:54.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.394+0000 7f4ff8ff9700 1 -- 172.21.15.17:0/3430412824 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fec0070d0 con 0x7f4ffc0614b0 2022-08-01T19:41:54.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.394+0000 7f4ff8ff9700 1 -- 172.21.15.17:0/3430412824 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fec007230 con 0x7f4ffc0614b0 2022-08-01T19:41:54.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.394+0000 7f4ffbfff700 1 -- 172.21.15.17:0/3430412824 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ffc153010 con 0x7f4ffc0614b0 2022-08-01T19:41:54.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.394+0000 7f4ff8ff9700 1 -- 172.21.15.17:0/3430412824 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fec005610 con 0x7f4ffc0614b0 2022-08-01T19:41:54.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.394+0000 7f4ffbfff700 1 -- 172.21.15.17:0/3430412824 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ffc0b0770 con 0x7f4ffc0614b0 2022-08-01T19:41:54.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.398+0000 7f4ffbfff700 1 -- 172.21.15.17:0/3430412824 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ffc0af5f0 con 0x7f4ffc0614b0 2022-08-01T19:41:54.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.398+0000 7f4ff8ff9700 1 -- 172.21.15.17:0/3430412824 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4fec010070 con 0x7f4ffc0614b0 2022-08-01T19:41:54.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.398+0000 7f4ff8ff9700 1 --2- 172.21.15.17:0/3430412824 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4fe00539e0 0x7f4fe0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:54.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.398+0000 7f4ff8ff9700 1 -- 172.21.15.17:0/3430412824 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4fec06b790 con 0x7f4ffc0614b0 2022-08-01T19:41:54.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.398+0000 7f4ff27fc700 1 --2- 172.21.15.17:0/3430412824 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4fe00539e0 0x7f4fe0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:54.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.398+0000 7f4ff27fc700 1 --2- 172.21.15.17:0/3430412824 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4fe00539e0 0x7f4fe0055ea0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7f4fe401a000 tx=0x7f4fe4014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:54.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.406+0000 7f4ff8ff9700 1 -- 172.21.15.17:0/3430412824 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4fec042440 con 0x7f4ffc0614b0 2022-08-01T19:41:54.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.530+0000 7f4ffbfff700 1 -- 172.21.15.17:0/3430412824 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4ffc0574e0 con 0x7f4ffc0614b0 2022-08-01T19:41:54.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.530+0000 7f4ff8ff9700 1 -- 172.21.15.17:0/3430412824 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v107) v1 ==== 100+0+10 (secure 0 0 0) 0x7f4fec045a60 con 0x7f4ffc0614b0 2022-08-01T19:41:54.536 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:41:54.538 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.534+0000 7f4ffbfff700 1 -- 172.21.15.17:0/3430412824 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4fe00539e0 msgr2=0x7f4fe0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:54.538 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.534+0000 7f4ffbfff700 1 --2- 172.21.15.17:0/3430412824 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4fe00539e0 0x7f4fe0055ea0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7f4fe401a000 tx=0x7f4fe4014000).stop 2022-08-01T19:41:54.539 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.534+0000 7f4ffbfff700 1 -- 172.21.15.17:0/3430412824 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ffc0614b0 msgr2=0x7f4ffc09ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:54.539 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.534+0000 7f4ffbfff700 1 --2- 172.21.15.17:0/3430412824 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ffc0614b0 0x7f4ffc09ebe0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 rx=0x7f4fec001110 tx=0x7f4fec007ad0).stop 2022-08-01T19:41:54.539 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.534+0000 7f4ffbfff700 1 -- 172.21.15.17:0/3430412824 shutdown_connections 2022-08-01T19:41:54.540 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.534+0000 7f4ffbfff700 1 --2- 172.21.15.17:0/3430412824 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4fe00539e0 0x7f4fe0055ea0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:54.540 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.534+0000 7f4ffbfff700 1 --2- 172.21.15.17:0/3430412824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ffc1405d0 0x7f4ffc0b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:54.540 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.534+0000 7f4ffbfff700 1 --2- 172.21.15.17:0/3430412824 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4ffc0ad750 0x7f4ffc09f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:54.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.534+0000 7f4ffbfff700 1 --2- 172.21.15.17:0/3430412824 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ffc0614b0 0x7f4ffc09ebe0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:54.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.534+0000 7f4ffbfff700 1 -- 172.21.15.17:0/3430412824 >> 172.21.15.17:0/3430412824 conn(0x7f4ffc066f70 msgr2=0x7f4ffc146420 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:54.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.534+0000 7f4ffbfff700 1 -- 172.21.15.17:0/3430412824 shutdown_connections 2022-08-01T19:41:54.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.534+0000 7f4ffbfff700 1 -- 172.21.15.17:0/3430412824 wait complete. 2022-08-01T19:41:54.549 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T19:41:54.549 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:41:54.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.650+0000 7f0a9ffff700 1 Processor -- start 2022-08-01T19:41:54.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.650+0000 7f0a9ffff700 1 -- start start 2022-08-01T19:41:54.660 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.654+0000 7f0a9ffff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f0aa00b31d0 0x7f0aa0066020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:54.660 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.654+0000 7f0a9ffff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0aa01402d0 0x7f0aa0140780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:54.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.654+0000 7f0a9ffff700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa0067030 con 0x7f0aa00a9da0 2022-08-01T19:41:54.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.654+0000 7f0a9ffff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0aa0064a70 con 0x7f0aa00b31d0 2022-08-01T19:41:54.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.654+0000 7f0a9ffff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa0061f60 con 0x7f0aa01402d0 2022-08-01T19:41:54.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.654+0000 7f0a9f7fe700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0aa01402d0 0x7f0aa0140780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:54.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.654+0000 7f0a9effd700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f0aa00b31d0 0x7f0aa0066020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:54.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.654+0000 7f0a9e7fc700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7f0aa00a9da0 0x7f0aa0060950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:51904/0 (socket says 172.21.15.17:51904) 2022-08-01T19:41:54.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.654+0000 7f0a9f7fe700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0aa01402d0 0x7f0aa0140780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:39014/0 (socket says 172.21.15.17:39014) 2022-08-01T19:41:54.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.654+0000 7f0a9e7fc700 1 -- 172.21.15.17:0/2573949246 learned_addr learned my addr 172.21.15.17:0/2573949246 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:54.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.654+0000 7f0a9f7fe700 1 -- 172.21.15.17:0/2573949246 >> v2:172.21.15.77:3300/0 conn(0x7f0aa00b31d0 msgr2=0x7f0aa0066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:54.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.654+0000 7f0a9f7fe700 1 --2- 172.21.15.17:0/2573949246 >> v2:172.21.15.77:3300/0 conn(0x7f0aa00b31d0 0x7f0aa0066020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:54.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.654+0000 7f0a9f7fe700 1 -- 172.21.15.17:0/2573949246 >> v1:172.21.15.77:6789/0 conn(0x7f0aa00a9da0 legacy=0x7f0aa0060950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:54.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.654+0000 7f0a9f7fe700 1 -- 172.21.15.17:0/2573949246 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa00b35b0 con 0x7f0aa01402d0 2022-08-01T19:41:54.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.654+0000 7f0a9effd700 1 --2- 172.21.15.17:0/2573949246 >> v2:172.21.15.77:3300/0 conn(0x7f0aa00b31d0 0x7f0aa0066020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:54.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.654+0000 7f0a9f7fe700 1 --2- 172.21.15.17:0/2573949246 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0aa01402d0 0x7f0aa0140780 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f0a98002f70 tx=0x7f0a98006df0).ready entity=mon.2 client_cookie=ebd4445e4a5ba135 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:54.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.654+0000 7f0a9dffb700 1 -- 172.21.15.17:0/2573949246 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a9800d570 con 0x7f0aa01402d0 2022-08-01T19:41:54.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.654+0000 7f0a9dffb700 1 -- 172.21.15.17:0/2573949246 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a98000e90 con 0x7f0aa01402d0 2022-08-01T19:41:54.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.654+0000 7f0a9dffb700 1 -- 172.21.15.17:0/2573949246 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a9800d8c0 con 0x7f0aa01402d0 2022-08-01T19:41:54.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.654+0000 7f0a9ffff700 1 -- 172.21.15.17:0/2573949246 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0aa01402d0 msgr2=0x7f0aa0140780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:54.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.654+0000 7f0a9ffff700 1 --2- 172.21.15.17:0/2573949246 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0aa01402d0 0x7f0aa0140780 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f0a98002f70 tx=0x7f0a98006df0).stop 2022-08-01T19:41:54.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.654+0000 7f0a9ffff700 1 -- 172.21.15.17:0/2573949246 shutdown_connections 2022-08-01T19:41:54.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.654+0000 7f0a9ffff700 1 --2- 172.21.15.17:0/2573949246 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0aa01402d0 0x7f0aa0140780 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:54.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.654+0000 7f0a9ffff700 1 --2- 172.21.15.17:0/2573949246 >> v2:172.21.15.77:3300/0 conn(0x7f0aa00b31d0 0x7f0aa0066020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:54.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.654+0000 7f0a9ffff700 1 -- 172.21.15.17:0/2573949246 >> 172.21.15.17:0/2573949246 conn(0x7f0aa0063870 msgr2=0x7f0aa0063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:54.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.654+0000 7f0a9ffff700 1 -- 172.21.15.17:0/2573949246 shutdown_connections 2022-08-01T19:41:54.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.654+0000 7f0a9ffff700 1 -- 172.21.15.17:0/2573949246 wait complete. 2022-08-01T19:41:54.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.658+0000 7f0a9ffff700 1 Processor -- start 2022-08-01T19:41:54.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.658+0000 7f0a9ffff700 1 -- start start 2022-08-01T19:41:54.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.658+0000 7f0a9ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0aa00a9da0 0x7f0aa014e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:54.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.658+0000 7f0a9ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0aa00b31d0 0x7f0aa013d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:54.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.658+0000 7f0a9ffff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0aa01402d0 0x7f0aa00b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:54.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.658+0000 7f0a9ffff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa014e610 con 0x7f0aa00a9da0 2022-08-01T19:41:54.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.658+0000 7f0a9ffff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa00adbe0 con 0x7f0aa00b31d0 2022-08-01T19:41:54.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.658+0000 7f0a9ffff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa00aee30 con 0x7f0aa01402d0 2022-08-01T19:41:54.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.658+0000 7f0a9effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0aa00a9da0 0x7f0aa014e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:54.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.658+0000 7f0a9effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0aa00a9da0 0x7f0aa014e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42916/0 (socket says 172.21.15.17:42916) 2022-08-01T19:41:54.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.658+0000 7f0a9effd700 1 -- 172.21.15.17:0/55478737 learned_addr learned my addr 172.21.15.17:0/55478737 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:54.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.658+0000 7f0a9f7fe700 1 --2- 172.21.15.17:0/55478737 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0aa01402d0 0x7f0aa00b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:54.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.658+0000 7f0a9e7fc700 1 --2- 172.21.15.17:0/55478737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0aa00b31d0 0x7f0aa013d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:54.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.658+0000 7f0a9effd700 1 -- 172.21.15.17:0/55478737 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0aa01402d0 msgr2=0x7f0aa00b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:54.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.658+0000 7f0a9effd700 1 --2- 172.21.15.17:0/55478737 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0aa01402d0 0x7f0aa00b18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:54.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.658+0000 7f0a9effd700 1 -- 172.21.15.17:0/55478737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0aa00b31d0 msgr2=0x7f0aa013d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:54.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.658+0000 7f0a9effd700 1 --2- 172.21.15.17:0/55478737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0aa00b31d0 0x7f0aa013d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:54.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.658+0000 7f0a9effd700 1 -- 172.21.15.17:0/55478737 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9000d410 con 0x7f0aa00a9da0 2022-08-01T19:41:54.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.658+0000 7f0a9effd700 1 --2- 172.21.15.17:0/55478737 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0aa00a9da0 0x7f0aa014e120 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 rx=0x7f0a9000e000 tx=0x7f0a90006e80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:54.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.658+0000 7f0a97fff700 1 -- 172.21.15.17:0/55478737 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a90010e90 con 0x7f0aa00a9da0 2022-08-01T19:41:54.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.658+0000 7f0a97fff700 1 -- 172.21.15.17:0/55478737 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9001a470 con 0x7f0aa00a9da0 2022-08-01T19:41:54.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.658+0000 7f0a9ffff700 1 -- 172.21.15.17:0/55478737 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a9800b180 con 0x7f0aa00a9da0 2022-08-01T19:41:54.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.658+0000 7f0a97fff700 1 -- 172.21.15.17:0/55478737 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a9001a7d0 con 0x7f0aa00a9da0 2022-08-01T19:41:54.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.658+0000 7f0a9ffff700 1 -- 172.21.15.17:0/55478737 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa013cf60 con 0x7f0aa00a9da0 2022-08-01T19:41:54.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.662+0000 7f0a9ffff700 1 -- 172.21.15.17:0/55478737 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aa00af030 con 0x7f0aa00a9da0 2022-08-01T19:41:54.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.662+0000 7f0a97fff700 1 -- 172.21.15.17:0/55478737 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0a90021060 con 0x7f0aa00a9da0 2022-08-01T19:41:54.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.662+0000 7f0a97fff700 1 --2- 172.21.15.17:0/55478737 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0a84053980 0x7f0a84055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:54.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.662+0000 7f0a97fff700 1 -- 172.21.15.17:0/55478737 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0a9006f100 con 0x7f0aa00a9da0 2022-08-01T19:41:54.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.662+0000 7f0a9e7fc700 1 --2- 172.21.15.17:0/55478737 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0a84053980 0x7f0a84055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:54.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.662+0000 7f0a9e7fc700 1 --2- 172.21.15.17:0/55478737 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0a84053980 0x7f0a84055e40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f0a8800d000 tx=0x7f0a88009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:54.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.670+0000 7f0a97fff700 1 -- 172.21.15.17:0/55478737 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0a90045ec0 con 0x7f0aa00a9da0 2022-08-01T19:41:54.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.830+0000 7f0a9ffff700 1 -- 172.21.15.17:0/55478737 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0aa013d120 con 0x7f0aa00a9da0 2022-08-01T19:41:54.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.830+0000 7f0a97fff700 1 -- 172.21.15.17:0/55478737 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f0a9001f070 con 0x7f0aa00a9da0 2022-08-01T19:41:54.840 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:54.841 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":53,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7252,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/1455566688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1455566688},{"type":"v1","addr":"172.21.15.77:6835","nonce":1455566688}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":51},{"gid":7266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.17:6833/249859043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":249859043},{"type":"v1","addr":"172.21.15.17:6833","nonce":249859043}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":51}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:41:47.422244+0000","modified":"2022-08-01T19:41:48.425195+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7272":{"gid":7272,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/2991628239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2991628239},{"type":"v1","addr":"172.21.15.17:6835","nonce":2991628239}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2022-08-01T19:41:54.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.830+0000 7f0a9ffff700 1 -- 172.21.15.17:0/55478737 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0a84053980 msgr2=0x7f0a84055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:54.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.830+0000 7f0a9ffff700 1 --2- 172.21.15.17:0/55478737 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0a84053980 0x7f0a84055e40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f0a8800d000 tx=0x7f0a88009040).stop 2022-08-01T19:41:54.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.830+0000 7f0a9ffff700 1 -- 172.21.15.17:0/55478737 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0aa00a9da0 msgr2=0x7f0aa014e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:54.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.830+0000 7f0a9ffff700 1 --2- 172.21.15.17:0/55478737 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0aa00a9da0 0x7f0aa014e120 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 rx=0x7f0a9000e000 tx=0x7f0a90006e80).stop 2022-08-01T19:41:54.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.830+0000 7f0a9ffff700 1 -- 172.21.15.17:0/55478737 shutdown_connections 2022-08-01T19:41:54.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.830+0000 7f0a9ffff700 1 --2- 172.21.15.17:0/55478737 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0a84053980 0x7f0a84055e40 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:54.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.830+0000 7f0a9ffff700 1 --2- 172.21.15.17:0/55478737 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0aa01402d0 0x7f0aa00b18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:54.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.830+0000 7f0a9ffff700 1 --2- 172.21.15.17:0/55478737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0aa00b31d0 0x7f0aa013d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:54.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.830+0000 7f0a9ffff700 1 --2- 172.21.15.17:0/55478737 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0aa00a9da0 0x7f0aa014e120 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:54.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.830+0000 7f0a9ffff700 1 -- 172.21.15.17:0/55478737 >> 172.21.15.17:0/55478737 conn(0x7f0aa0063870 msgr2=0x7f0aa005f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:54.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.834+0000 7f0a9ffff700 1 -- 172.21.15.17:0/55478737 shutdown_connections 2022-08-01T19:41:54.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:54.834+0000 7f0a9ffff700 1 -- 172.21.15.17:0/55478737 wait complete. 2022-08-01T19:41:54.856 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 53 2022-08-01T19:41:55.163 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 53, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:41:47.422244+0000', 'modified': '2022-08-01T19:41:48.425195+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7272}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7272': {'gid': 7272, 'name': 'c', 'rank': 0, 'incarnation': 52, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/2991628239', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 2991628239}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 2991628239}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:41:55.164 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:41:55.164 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T19:41:55.300 INFO:teuthology.orchestra.run.smithi017.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":7272,"want_state":"up:active","state":"up:active","rank_uptime":7.85793974,"mdsmap_epoch":53,"osdmap_epoch":107,"osdmap_epoch_barrier":107,"uptime":12.232893598} 2022-08-01T19:41:55.301 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 7272, "mdsmap_epoch": 53, "osdmap_epoch": 107, "osdmap_epoch_barrier": 107, "rank_uptime": 7.85793974, "state": "up:active", "uptime": 12.232893598, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:41:55.301 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T19:41:55.306 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-08-01T19:41:55.306 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi190.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-08-01T19:41:55.307 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:55.308 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:41:55.367 DEBUG:teuthology.orchestra.run.smithi190:> sudo modprobe fuse 2022-08-01T19:41:55.436 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:41:55.437 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:41:55.502 INFO:teuthology.orchestra.run.smithi190.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:41:55.504 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:41:55.505 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:55.507 DEBUG:teuthology.orchestra.run.smithi190:> ls /sys/fs/fuse/connections 2022-08-01T19:41:55.564 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T19:41:55.565 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:41:55.570 DEBUG:teuthology.orchestra.run.smithi190:> sudo modprobe fuse 2022-08-01T19:41:55.598 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:41:55.599 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:41:55.647 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:2022-08-01T19:41:55.643+0000 7f65da826080 -1 init, newargv = 0x55b82ed3d870 newargc=9 2022-08-01T19:41:55.648 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:ceph-fuse[22932]: starting ceph client 2022-08-01T19:41:55.652 INFO:teuthology.orchestra.run.smithi190.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:41:55.653 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:41:55.655 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:55.656 DEBUG:teuthology.orchestra.run.smithi190:> ls /sys/fs/fuse/connections 2022-08-01T19:41:55.658 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:ceph-fuse[22932]: starting fuse 2022-08-01T19:41:55.663 INFO:teuthology.orchestra.run.smithi190.stdout:52 2022-08-01T19:41:55.664 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T19:41:55.664 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:55.666 DEBUG:teuthology.orchestra.run.smithi190:> sudo python3 -c ' 2022-08-01T19:41:55.667 DEBUG:teuthology.orchestra.run.smithi190:> import glob 2022-08-01T19:41:55.667 DEBUG:teuthology.orchestra.run.smithi190:> import re 2022-08-01T19:41:55.668 DEBUG:teuthology.orchestra.run.smithi190:> import os 2022-08-01T19:41:55.670 DEBUG:teuthology.orchestra.run.smithi190:> import subprocess 2022-08-01T19:41:55.671 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:41:55.672 DEBUG:teuthology.orchestra.run.smithi190:> def find_socket(client_name): 2022-08-01T19:41:55.673 DEBUG:teuthology.orchestra.run.smithi190:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-08-01T19:41:55.673 DEBUG:teuthology.orchestra.run.smithi190:> files = glob.glob(asok_path) 2022-08-01T19:41:55.674 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:41:55.674 DEBUG:teuthology.orchestra.run.smithi190:> # Given a non-glob path, it better be there 2022-08-01T19:41:55.674 DEBUG:teuthology.orchestra.run.smithi190:> if "*" not in asok_path: 2022-08-01T19:41:55.675 DEBUG:teuthology.orchestra.run.smithi190:> assert(len(files) == 1) 2022-08-01T19:41:55.675 DEBUG:teuthology.orchestra.run.smithi190:> return files[0] 2022-08-01T19:41:55.676 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:41:55.676 DEBUG:teuthology.orchestra.run.smithi190:> for f in files: 2022-08-01T19:41:55.676 DEBUG:teuthology.orchestra.run.smithi190:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T19:41:55.677 DEBUG:teuthology.orchestra.run.smithi190:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T19:41:55.677 DEBUG:teuthology.orchestra.run.smithi190:> return f 2022-08-01T19:41:55.678 DEBUG:teuthology.orchestra.run.smithi190:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T19:41:55.678 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:41:55.679 DEBUG:teuthology.orchestra.run.smithi190:> print(find_socket("client.1")) 2022-08-01T19:41:55.679 DEBUG:teuthology.orchestra.run.smithi190:> ' 2022-08-01T19:41:55.730 INFO:teuthology.orchestra.run.smithi190.stdout:/var/run/ceph/ceph-client.1.22932.asok 2022-08-01T19:41:55.739 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.22932.asok 2022-08-01T19:41:55.740 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:55.741 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.22932.asok status 2022-08-01T19:41:55.874 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2022-08-01T19:41:55.874 INFO:teuthology.orchestra.run.smithi190.stdout: "metadata": { 2022-08-01T19:41:55.875 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T19:41:55.875 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T19:41:55.875 INFO:teuthology.orchestra.run.smithi190.stdout: "entity_id": "1", 2022-08-01T19:41:55.875 INFO:teuthology.orchestra.run.smithi190.stdout: "hostname": "smithi190", 2022-08-01T19:41:55.876 INFO:teuthology.orchestra.run.smithi190.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-08-01T19:41:55.876 INFO:teuthology.orchestra.run.smithi190.stdout: "pid": "22932", 2022-08-01T19:41:55.876 INFO:teuthology.orchestra.run.smithi190.stdout: "root": "/" 2022-08-01T19:41:55.876 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:41:55.878 INFO:teuthology.orchestra.run.smithi190.stdout: "dentry_count": 0, 2022-08-01T19:41:55.878 INFO:teuthology.orchestra.run.smithi190.stdout: "dentry_pinned_count": 0, 2022-08-01T19:41:55.879 INFO:teuthology.orchestra.run.smithi190.stdout: "id": 7477, 2022-08-01T19:41:55.879 INFO:teuthology.orchestra.run.smithi190.stdout: "inst": { 2022-08-01T19:41:55.879 INFO:teuthology.orchestra.run.smithi190.stdout: "name": { 2022-08-01T19:41:55.880 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "client", 2022-08-01T19:41:55.880 INFO:teuthology.orchestra.run.smithi190.stdout: "num": 7477 2022-08-01T19:41:55.880 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:41:55.880 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": { 2022-08-01T19:41:55.881 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "v1", 2022-08-01T19:41:55.881 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": "172.21.15.190:0", 2022-08-01T19:41:55.881 INFO:teuthology.orchestra.run.smithi190.stdout: "nonce": 3125911857 2022-08-01T19:41:55.881 INFO:teuthology.orchestra.run.smithi190.stdout: } 2022-08-01T19:41:55.884 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:41:55.885 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": { 2022-08-01T19:41:55.885 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "v1", 2022-08-01T19:41:55.885 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": "172.21.15.190:0", 2022-08-01T19:41:55.886 INFO:teuthology.orchestra.run.smithi190.stdout: "nonce": 3125911857 2022-08-01T19:41:55.886 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:41:55.886 INFO:teuthology.orchestra.run.smithi190.stdout: "inst_str": "client.7477 172.21.15.190:0/3125911857", 2022-08-01T19:41:55.886 INFO:teuthology.orchestra.run.smithi190.stdout: "addr_str": "172.21.15.190:0/3125911857", 2022-08-01T19:41:55.887 INFO:teuthology.orchestra.run.smithi190.stdout: "inode_count": 1, 2022-08-01T19:41:55.887 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_epoch": 53, 2022-08-01T19:41:55.887 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_epoch": 107, 2022-08-01T19:41:55.887 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_epoch_barrier": 0, 2022-08-01T19:41:55.888 INFO:teuthology.orchestra.run.smithi190.stdout: "blacklisted": false, 2022-08-01T19:41:55.888 INFO:teuthology.orchestra.run.smithi190.stdout: "fs_name": "cephfs" 2022-08-01T19:41:55.888 INFO:teuthology.orchestra.run.smithi190.stdout:} 2022-08-01T19:41:55.890 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:55.891 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:41:55.891 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:41:55.903 INFO:teuthology.orchestra.run.smithi190.stdout:fuseblk 2022-08-01T19:41:55.903 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T19:41:55.904 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:55.904 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:41:55.930 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2022-08-01T19:41:56.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.046+0000 7f47360b1700 1 Processor -- start 2022-08-01T19:41:56.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.046+0000 7f47360b1700 1 -- start start 2022-08-01T19:41:56.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.046+0000 7f47360b1700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f47300b2710 0x7f473014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:56.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.046+0000 7f47360b1700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f47300a8c50 0x7f473014b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:56.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.046+0000 7f47360b1700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47300a7250 con 0x7f47300b01a0 2022-08-01T19:41:56.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.046+0000 7f47360b1700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47300a7480 con 0x7f47300b2710 2022-08-01T19:41:56.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.046+0000 7f47360b1700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47300a7720 con 0x7f47300a8c50 2022-08-01T19:41:56.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.046+0000 7f47348ae700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f47300b01a0 0x7f47300a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35120/0 (socket says 172.21.15.17:35120) 2022-08-01T19:41:56.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.046+0000 7f47348ae700 1 -- 172.21.15.17:0/2187567072 learned_addr learned my addr 172.21.15.17:0/2187567072 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:56.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.046+0000 7f47350af700 1 --2- 172.21.15.17:0/2187567072 >> v2:172.21.15.17:3300/0 conn(0x7f47300b2710 0x7f473014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:56.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.046+0000 7f47358b0700 1 --2- 172.21.15.17:0/2187567072 >> v2:172.21.15.77:3300/0 conn(0x7f47300a8c50 0x7f473014b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:56.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.046+0000 7f4727fff700 1 -- 172.21.15.17:0/2187567072 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1668052665 0 0) 0x7f47300a7250 con 0x7f47300b01a0 2022-08-01T19:41:56.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.046+0000 7f4727fff700 1 -- 172.21.15.17:0/2187567072 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4714003260 con 0x7f47300b01a0 2022-08-01T19:41:56.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.046+0000 7f47350af700 1 -- 172.21.15.17:0/2187567072 >> v2:172.21.15.77:3300/0 conn(0x7f47300a8c50 msgr2=0x7f473014b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:56.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.046+0000 7f47350af700 1 --2- 172.21.15.17:0/2187567072 >> v2:172.21.15.77:3300/0 conn(0x7f47300a8c50 0x7f473014b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:56.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.046+0000 7f47350af700 1 -- 172.21.15.17:0/2187567072 >> v1:172.21.15.17:6789/0 conn(0x7f47300b01a0 legacy=0x7f47300a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:56.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.046+0000 7f47350af700 1 -- 172.21.15.17:0/2187567072 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47300a79c0 con 0x7f47300b2710 2022-08-01T19:41:56.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.046+0000 7f47358b0700 1 --2- 172.21.15.17:0/2187567072 >> v2:172.21.15.77:3300/0 conn(0x7f47300a8c50 0x7f473014b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:56.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.046+0000 7f47350af700 1 --2- 172.21.15.17:0/2187567072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47300b2710 0x7f473014afd0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 rx=0x7f472c002f70 tx=0x7f472c006df0).ready entity=mon.0 client_cookie=e2e316d2ca560b36 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:56.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.046+0000 7f4727fff700 1 -- 172.21.15.17:0/2187567072 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f472c00d4f0 con 0x7f47300b2710 2022-08-01T19:41:56.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.046+0000 7f4727fff700 1 -- 172.21.15.17:0/2187567072 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f472c000e90 con 0x7f47300b2710 2022-08-01T19:41:56.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.046+0000 7f4727fff700 1 -- 172.21.15.17:0/2187567072 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f472c00d840 con 0x7f47300b2710 2022-08-01T19:41:56.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.046+0000 7f47360b1700 1 -- 172.21.15.17:0/2187567072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47300b2710 msgr2=0x7f473014afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:56.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.046+0000 7f47360b1700 1 --2- 172.21.15.17:0/2187567072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47300b2710 0x7f473014afd0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 rx=0x7f472c002f70 tx=0x7f472c006df0).stop 2022-08-01T19:41:56.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.046+0000 7f47360b1700 1 -- 172.21.15.17:0/2187567072 shutdown_connections 2022-08-01T19:41:56.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.046+0000 7f47360b1700 1 --2- 172.21.15.17:0/2187567072 >> v2:172.21.15.77:3300/0 conn(0x7f47300a8c50 0x7f473014b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:56.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.046+0000 7f47360b1700 1 --2- 172.21.15.17:0/2187567072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47300b2710 0x7f473014afd0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:56.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.046+0000 7f47360b1700 1 -- 172.21.15.17:0/2187567072 >> 172.21.15.17:0/2187567072 conn(0x7f4730060310 msgr2=0x7f47300b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:56.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.046+0000 7f47360b1700 1 -- 172.21.15.17:0/2187567072 shutdown_connections 2022-08-01T19:41:56.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.046+0000 7f47360b1700 1 -- 172.21.15.17:0/2187567072 wait complete. 2022-08-01T19:41:56.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.050+0000 7f47360b1700 1 Processor -- start 2022-08-01T19:41:56.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.050+0000 7f47360b1700 1 -- start start 2022-08-01T19:41:56.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.050+0000 7f47360b1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47300a8c50 0x7f47300b33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:56.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.050+0000 7f47360b1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47300b01a0 0x7f47300ac1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:56.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.050+0000 7f47360b1700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f47300b2710 0x7f4730156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:56.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.050+0000 7f47360b1700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47300b31a0 con 0x7f47300a8c50 2022-08-01T19:41:56.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.050+0000 7f47360b1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47301571d0 con 0x7f47300b01a0 2022-08-01T19:41:56.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.050+0000 7f47360b1700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4730157470 con 0x7f47300b2710 2022-08-01T19:41:56.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.050+0000 7f47350af700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47300a8c50 0x7f47300b33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:56.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.050+0000 7f47350af700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47300a8c50 0x7f47300b33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42932/0 (socket says 172.21.15.17:42932) 2022-08-01T19:41:56.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.050+0000 7f47350af700 1 -- 172.21.15.17:0/1992275601 learned_addr learned my addr 172.21.15.17:0/1992275601 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:56.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.050+0000 7f47348ae700 1 --2- 172.21.15.17:0/1992275601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47300b01a0 0x7f47300ac1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:56.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.050+0000 7f47358b0700 1 --2- 172.21.15.17:0/1992275601 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f47300b2710 0x7f4730156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:56.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.050+0000 7f47350af700 1 -- 172.21.15.17:0/1992275601 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f47300b2710 msgr2=0x7f4730156a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:56.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.050+0000 7f47350af700 1 --2- 172.21.15.17:0/1992275601 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f47300b2710 0x7f4730156a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:56.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.050+0000 7f47350af700 1 -- 172.21.15.17:0/1992275601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47300b01a0 msgr2=0x7f47300ac1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:56.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.050+0000 7f47350af700 1 --2- 172.21.15.17:0/1992275601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47300b01a0 0x7f47300ac1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:56.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.050+0000 7f47350af700 1 -- 172.21.15.17:0/1992275601 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4728009470 con 0x7f47300a8c50 2022-08-01T19:41:56.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.050+0000 7f47348ae700 1 --2- 172.21.15.17:0/1992275601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47300b01a0 0x7f47300ac1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:56.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.050+0000 7f47358b0700 1 --2- 172.21.15.17:0/1992275601 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f47300b2710 0x7f4730156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:56.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.050+0000 7f47350af700 1 --2- 172.21.15.17:0/1992275601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47300a8c50 0x7f47300b33a0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 rx=0x7f472c002f70 tx=0x7f472c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:56.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.050+0000 7f47267fc700 1 -- 172.21.15.17:0/1992275601 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f472c016070 con 0x7f47300a8c50 2022-08-01T19:41:56.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.050+0000 7f47267fc700 1 -- 172.21.15.17:0/1992275601 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f472c013a00 con 0x7f47300a8c50 2022-08-01T19:41:56.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.050+0000 7f47360b1700 1 -- 172.21.15.17:0/1992275601 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f472c00b180 con 0x7f47300a8c50 2022-08-01T19:41:56.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.050+0000 7f47267fc700 1 -- 172.21.15.17:0/1992275601 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f472c013d60 con 0x7f47300a8c50 2022-08-01T19:41:56.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.050+0000 7f47360b1700 1 -- 172.21.15.17:0/1992275601 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f473014b810 con 0x7f47300a8c50 2022-08-01T19:41:56.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.054+0000 7f47360b1700 1 -- 172.21.15.17:0/1992275601 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47300574e0 con 0x7f47300a8c50 2022-08-01T19:41:56.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.054+0000 7f47267fc700 1 -- 172.21.15.17:0/1992275601 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f472c025030 con 0x7f47300a8c50 2022-08-01T19:41:56.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.054+0000 7f47267fc700 1 --2- 172.21.15.17:0/1992275601 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4718053a70 0x7f4718055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:56.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.054+0000 7f47267fc700 1 -- 172.21.15.17:0/1992275601 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f472c06cfe0 con 0x7f47300a8c50 2022-08-01T19:41:56.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.054+0000 7f47348ae700 1 --2- 172.21.15.17:0/1992275601 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4718053a70 0x7f4718055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:56.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.054+0000 7f47348ae700 1 --2- 172.21.15.17:0/1992275601 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4718053a70 0x7f4718055f30 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f472000b000 tx=0x7f4720009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:56.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.058+0000 7f47267fc700 1 -- 172.21.15.17:0/1992275601 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f472c043da0 con 0x7f47300a8c50 2022-08-01T19:41:56.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.230+0000 7f47360b1700 1 -- 172.21.15.17:0/1992275601 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7f47300623b0 con 0x7f47300a8c50 2022-08-01T19:41:56.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.230+0000 7f47267fc700 1 -- 172.21.15.17:0/1992275601 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1936 (secure 0 0 0) 0x7f472c023020 con 0x7f47300a8c50 2022-08-01T19:41:56.237 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:56.239 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:41:56.240 INFO:teuthology.orchestra.run.smithi017.stdout: "stats": { 2022-08-01T19:41:56.240 INFO:teuthology.orchestra.run.smithi017.stdout: "total_bytes": 773094113280, 2022-08-01T19:41:56.241 INFO:teuthology.orchestra.run.smithi017.stdout: "total_avail_bytes": 764493373440, 2022-08-01T19:41:56.241 INFO:teuthology.orchestra.run.smithi017.stdout: "total_used_bytes": 10805248, 2022-08-01T19:41:56.243 INFO:teuthology.orchestra.run.smithi017.stdout: "total_used_raw_bytes": 8600739840, 2022-08-01T19:41:56.243 INFO:teuthology.orchestra.run.smithi017.stdout: "total_used_raw_ratio": 0.011125087738037109, 2022-08-01T19:41:56.243 INFO:teuthology.orchestra.run.smithi017.stdout: "num_osds": 8, 2022-08-01T19:41:56.244 INFO:teuthology.orchestra.run.smithi017.stdout: "num_per_pool_osds": 8, 2022-08-01T19:41:56.244 INFO:teuthology.orchestra.run.smithi017.stdout: "num_per_pool_omap_osds": 8 2022-08-01T19:41:56.245 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:56.246 INFO:teuthology.orchestra.run.smithi017.stdout: "stats_by_class": { 2022-08-01T19:41:56.246 INFO:teuthology.orchestra.run.smithi017.stdout: "ssd": { 2022-08-01T19:41:56.246 INFO:teuthology.orchestra.run.smithi017.stdout: "total_bytes": 773094113280, 2022-08-01T19:41:56.246 INFO:teuthology.orchestra.run.smithi017.stdout: "total_avail_bytes": 764493373440, 2022-08-01T19:41:56.247 INFO:teuthology.orchestra.run.smithi017.stdout: "total_used_bytes": 10805248, 2022-08-01T19:41:56.247 INFO:teuthology.orchestra.run.smithi017.stdout: "total_used_raw_bytes": 8600739840, 2022-08-01T19:41:56.247 INFO:teuthology.orchestra.run.smithi017.stdout: "total_used_raw_ratio": 0.011125087738037109 2022-08-01T19:41:56.247 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:56.248 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:56.248 INFO:teuthology.orchestra.run.smithi017.stdout: "pools": [ 2022-08-01T19:41:56.248 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:56.248 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "device_health_metrics", 2022-08-01T19:41:56.249 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 1, 2022-08-01T19:41:56.249 INFO:teuthology.orchestra.run.smithi017.stdout: "stats": { 2022-08-01T19:41:56.249 INFO:teuthology.orchestra.run.smithi017.stdout: "stored": 0, 2022-08-01T19:41:56.249 INFO:teuthology.orchestra.run.smithi017.stdout: "objects": 2, 2022-08-01T19:41:56.249 INFO:teuthology.orchestra.run.smithi017.stdout: "kb_used": 0, 2022-08-01T19:41:56.263 INFO:teuthology.orchestra.run.smithi017.stdout: "bytes_used": 0, 2022-08-01T19:41:56.270 INFO:teuthology.orchestra.run.smithi017.stdout: "percent_used": 0, 2022-08-01T19:41:56.271 INFO:teuthology.orchestra.run.smithi017.stdout: "max_avail": 362919297024 2022-08-01T19:41:56.272 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:56.272 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:56.272 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:56.273 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rbd", 2022-08-01T19:41:56.273 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 2, 2022-08-01T19:41:56.273 INFO:teuthology.orchestra.run.smithi017.stdout: "stats": { 2022-08-01T19:41:56.273 INFO:teuthology.orchestra.run.smithi017.stdout: "stored": 0, 2022-08-01T19:41:56.274 INFO:teuthology.orchestra.run.smithi017.stdout: "objects": 0, 2022-08-01T19:41:56.274 INFO:teuthology.orchestra.run.smithi017.stdout: "kb_used": 0, 2022-08-01T19:41:56.274 INFO:teuthology.orchestra.run.smithi017.stdout: "bytes_used": 0, 2022-08-01T19:41:56.274 INFO:teuthology.orchestra.run.smithi017.stdout: "percent_used": 0, 2022-08-01T19:41:56.275 INFO:teuthology.orchestra.run.smithi017.stdout: "max_avail": 362919297024 2022-08-01T19:41:56.275 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:56.275 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:56.275 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:56.276 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "cephfs_metadata", 2022-08-01T19:41:56.276 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 19, 2022-08-01T19:41:56.276 INFO:teuthology.orchestra.run.smithi017.stdout: "stats": { 2022-08-01T19:41:56.276 INFO:teuthology.orchestra.run.smithi017.stdout: "stored": 2742, 2022-08-01T19:41:56.277 INFO:teuthology.orchestra.run.smithi017.stdout: "objects": 22, 2022-08-01T19:41:56.277 INFO:teuthology.orchestra.run.smithi017.stdout: "kb_used": 64, 2022-08-01T19:41:56.277 INFO:teuthology.orchestra.run.smithi017.stdout: "bytes_used": 65536, 2022-08-01T19:41:56.277 INFO:teuthology.orchestra.run.smithi017.stdout: "percent_used": 9.0290036780515948e-08, 2022-08-01T19:41:56.278 INFO:teuthology.orchestra.run.smithi017.stdout: "max_avail": 362919297024 2022-08-01T19:41:56.278 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:56.278 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:56.278 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:56.279 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "cephfs_data", 2022-08-01T19:41:56.279 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 20, 2022-08-01T19:41:56.279 INFO:teuthology.orchestra.run.smithi017.stdout: "stats": { 2022-08-01T19:41:56.279 INFO:teuthology.orchestra.run.smithi017.stdout: "stored": 0, 2022-08-01T19:41:56.280 INFO:teuthology.orchestra.run.smithi017.stdout: "objects": 0, 2022-08-01T19:41:56.280 INFO:teuthology.orchestra.run.smithi017.stdout: "kb_used": 0, 2022-08-01T19:41:56.280 INFO:teuthology.orchestra.run.smithi017.stdout: "bytes_used": 0, 2022-08-01T19:41:56.280 INFO:teuthology.orchestra.run.smithi017.stdout: "percent_used": 0, 2022-08-01T19:41:56.281 INFO:teuthology.orchestra.run.smithi017.stdout: "max_avail": 362919297024 2022-08-01T19:41:56.289 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:56.289 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:56.289 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:56.290 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:41:56.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.230+0000 7f47360b1700 1 -- 172.21.15.17:0/1992275601 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4718053a70 msgr2=0x7f4718055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:56.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.230+0000 7f47360b1700 1 --2- 172.21.15.17:0/1992275601 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4718053a70 0x7f4718055f30 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f472000b000 tx=0x7f4720009040).stop 2022-08-01T19:41:56.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.230+0000 7f47360b1700 1 -- 172.21.15.17:0/1992275601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47300a8c50 msgr2=0x7f47300b33a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:56.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.230+0000 7f47360b1700 1 --2- 172.21.15.17:0/1992275601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47300a8c50 0x7f47300b33a0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 rx=0x7f472c002f70 tx=0x7f472c014000).stop 2022-08-01T19:41:56.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.234+0000 7f47360b1700 1 -- 172.21.15.17:0/1992275601 shutdown_connections 2022-08-01T19:41:56.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.234+0000 7f47360b1700 1 --2- 172.21.15.17:0/1992275601 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4718053a70 0x7f4718055f30 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:56.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.234+0000 7f47360b1700 1 --2- 172.21.15.17:0/1992275601 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f47300b2710 0x7f4730156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:56.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.234+0000 7f47360b1700 1 --2- 172.21.15.17:0/1992275601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f47300b01a0 0x7f47300ac1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:56.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.234+0000 7f47360b1700 1 --2- 172.21.15.17:0/1992275601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47300a8c50 0x7f47300b33a0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:56.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.234+0000 7f47360b1700 1 -- 172.21.15.17:0/1992275601 >> 172.21.15.17:0/1992275601 conn(0x7f4730060310 msgr2=0x7f4730144cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:56.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.234+0000 7f47360b1700 1 -- 172.21.15.17:0/1992275601 shutdown_connections 2022-08-01T19:41:56.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.234+0000 7f47360b1700 1 -- 172.21.15.17:0/1992275601 wait complete. 2022-08-01T19:41:56.296 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data size -f json-pretty 2022-08-01T19:41:56.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.426+0000 7fd714948700 1 Processor -- start 2022-08-01T19:41:56.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.426+0000 7fd714948700 1 -- start start 2022-08-01T19:41:56.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.430+0000 7fd714948700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fd7100b01a0 0x7fd7100a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:56.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.430+0000 7fd714948700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd7100afd60 0x7fd7100a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:56.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.430+0000 7fd714948700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7100a7250 con 0x7fd7100b2710 2022-08-01T19:41:56.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.430+0000 7fd714948700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7100a7480 con 0x7fd7100b01a0 2022-08-01T19:41:56.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.430+0000 7fd714948700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7100a7720 con 0x7fd7100afd60 2022-08-01T19:41:56.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.430+0000 7fd70e59c700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fd7100b01a0 0x7fd7100a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:56.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.430+0000 7fd70f59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd7100afd60 0x7fd7100a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:56.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.430+0000 7fd70e59c700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fd7100b01a0 0x7fd7100a8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:40862/0 (socket says 172.21.15.17:40862) 2022-08-01T19:41:56.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.430+0000 7fd70e59c700 1 -- 172.21.15.17:0/2173650781 learned_addr learned my addr 172.21.15.17:0/2173650781 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:56.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.430+0000 7fd70ed9d700 1 --1- 172.21.15.17:0/2173650781 >> v1:172.21.15.77:6789/0 conn(0x7fd7100b2710 0x7fd71014afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:51932/0 (socket says 172.21.15.17:51932) 2022-08-01T19:41:56.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.430+0000 7fd70e59c700 1 -- 172.21.15.17:0/2173650781 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd7100afd60 msgr2=0x7fd7100a8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:56.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.430+0000 7fd70e59c700 1 --2- 172.21.15.17:0/2173650781 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd7100afd60 0x7fd7100a8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:56.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.430+0000 7fd70e59c700 1 -- 172.21.15.17:0/2173650781 >> v1:172.21.15.77:6789/0 conn(0x7fd7100b2710 legacy=0x7fd71014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:56.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.430+0000 7fd70e59c700 1 -- 172.21.15.17:0/2173650781 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7100a79c0 con 0x7fd7100b01a0 2022-08-01T19:41:56.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.430+0000 7fd70e59c700 1 --2- 172.21.15.17:0/2173650781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7100b01a0 0x7fd7100a8540 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7fd700002f70 tx=0x7fd700006eb0).ready entity=mon.1 client_cookie=fd72f91d98ba32c9 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:56.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.430+0000 7fd70dd9b700 1 -- 172.21.15.17:0/2173650781 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd70000d570 con 0x7fd7100b01a0 2022-08-01T19:41:56.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.430+0000 7fd70dd9b700 1 -- 172.21.15.17:0/2173650781 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd700000e90 con 0x7fd7100b01a0 2022-08-01T19:41:56.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.430+0000 7fd70dd9b700 1 -- 172.21.15.17:0/2173650781 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd70000d900 con 0x7fd7100b01a0 2022-08-01T19:41:56.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.430+0000 7fd714948700 1 -- 172.21.15.17:0/2173650781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7100b01a0 msgr2=0x7fd7100a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:56.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.430+0000 7fd714948700 1 --2- 172.21.15.17:0/2173650781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7100b01a0 0x7fd7100a8540 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7fd700002f70 tx=0x7fd700006eb0).stop 2022-08-01T19:41:56.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.430+0000 7fd714948700 1 -- 172.21.15.17:0/2173650781 shutdown_connections 2022-08-01T19:41:56.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.430+0000 7fd714948700 1 --2- 172.21.15.17:0/2173650781 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd7100afd60 0x7fd7100a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:56.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.430+0000 7fd714948700 1 --2- 172.21.15.17:0/2173650781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7100b01a0 0x7fd7100a8540 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:56.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.430+0000 7fd714948700 1 -- 172.21.15.17:0/2173650781 >> 172.21.15.17:0/2173650781 conn(0x7fd710060310 msgr2=0x7fd7100b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:56.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.430+0000 7fd714948700 1 -- 172.21.15.17:0/2173650781 shutdown_connections 2022-08-01T19:41:56.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.430+0000 7fd714948700 1 -- 172.21.15.17:0/2173650781 wait complete. 2022-08-01T19:41:56.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.434+0000 7fd714948700 1 Processor -- start 2022-08-01T19:41:56.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.434+0000 7fd714948700 1 -- start start 2022-08-01T19:41:56.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.434+0000 7fd714948700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd7100afd60 0x7fd7100b54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:56.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.434+0000 7fd714948700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7100b01a0 0x7fd71014aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:56.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.434+0000 7fd714948700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7100b2710 0x7fd710159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:56.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.434+0000 7fd714948700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7100674e0 con 0x7fd7100b01a0 2022-08-01T19:41:56.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.434+0000 7fd714948700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7100b40e0 con 0x7fd7100b2710 2022-08-01T19:41:56.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.434+0000 7fd714948700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7100b5100 con 0x7fd7100afd60 2022-08-01T19:41:56.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.434+0000 7fd70e59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7100b01a0 0x7fd71014aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:56.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.434+0000 7fd70e59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7100b01a0 0x7fd71014aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42946/0 (socket says 172.21.15.17:42946) 2022-08-01T19:41:56.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.434+0000 7fd70e59c700 1 -- 172.21.15.17:0/2400539696 learned_addr learned my addr 172.21.15.17:0/2400539696 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:56.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.434+0000 7fd70ed9d700 1 --2- 172.21.15.17:0/2400539696 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd7100afd60 0x7fd7100b54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:56.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.434+0000 7fd70f59e700 1 --2- 172.21.15.17:0/2400539696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7100b2710 0x7fd710159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:56.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.438+0000 7fd70ed9d700 1 -- 172.21.15.17:0/2400539696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7100b2710 msgr2=0x7fd710159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:56.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.438+0000 7fd70ed9d700 1 --2- 172.21.15.17:0/2400539696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7100b2710 0x7fd710159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:56.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.438+0000 7fd70ed9d700 1 -- 172.21.15.17:0/2400539696 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7100b01a0 msgr2=0x7fd71014aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:56.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.438+0000 7fd70ed9d700 1 --2- 172.21.15.17:0/2400539696 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7100b01a0 0x7fd71014aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:56.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.438+0000 7fd70ed9d700 1 -- 172.21.15.17:0/2400539696 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd70000b1e0 con 0x7fd7100afd60 2022-08-01T19:41:56.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.438+0000 7fd70ed9d700 1 --2- 172.21.15.17:0/2400539696 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd7100afd60 0x7fd7100b54d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fd71006b950 tx=0x7fd704007b80).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:56.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.438+0000 7fd6fffff700 1 -- 172.21.15.17:0/2400539696 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7040070d0 con 0x7fd7100afd60 2022-08-01T19:41:56.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.438+0000 7fd6fffff700 1 -- 172.21.15.17:0/2400539696 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd704007230 con 0x7fd7100afd60 2022-08-01T19:41:56.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.438+0000 7fd714948700 1 -- 172.21.15.17:0/2400539696 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7100b30e0 con 0x7fd7100afd60 2022-08-01T19:41:56.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.438+0000 7fd6fffff700 1 -- 172.21.15.17:0/2400539696 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd704005550 con 0x7fd7100afd60 2022-08-01T19:41:56.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.438+0000 7fd714948700 1 -- 172.21.15.17:0/2400539696 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7100b3a10 con 0x7fd7100afd60 2022-08-01T19:41:56.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.438+0000 7fd714948700 1 -- 172.21.15.17:0/2400539696 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7100574e0 con 0x7fd7100afd60 2022-08-01T19:41:56.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.438+0000 7fd6fffff700 1 -- 172.21.15.17:0/2400539696 <== mon.2 v2:172.21.15.77:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd70401a060 con 0x7fd7100afd60 2022-08-01T19:41:56.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.442+0000 7fd6fffff700 1 --2- 172.21.15.17:0/2400539696 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd6ec053aa0 0x7fd6ec055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:56.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.442+0000 7fd6fffff700 1 -- 172.21.15.17:0/2400539696 <== mon.2 v2:172.21.15.77:3301/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd70406b1a0 con 0x7fd7100afd60 2022-08-01T19:41:56.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.442+0000 7fd70e59c700 1 --2- 172.21.15.17:0/2400539696 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd6ec053aa0 0x7fd6ec055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:56.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.442+0000 7fd70e59c700 1 --2- 172.21.15.17:0/2400539696 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd6ec053aa0 0x7fd6ec055f60 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7fd70001c000 tx=0x7fd700015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:56.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.450+0000 7fd6fffff700 1 -- 172.21.15.17:0/2400539696 <== mon.2 v2:172.21.15.77:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd704041f60 con 0x7fd7100afd60 2022-08-01T19:41:56.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.578+0000 7fd714948700 1 -- 172.21.15.17:0/2400539696 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "size", "format": "json-pretty"} v 0) v1 -- 0x7fd710069a70 con 0x7fd7100afd60 2022-08-01T19:41:56.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.578+0000 7fd6fffff700 1 -- 172.21.15.17:0/2400539696 <== mon.2 v2:172.21.15.77:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "size", "format": "json-pretty"}]=0 v107) v1 ==== 123+0+64 (secure 0 0 0) 0x7fd704018020 con 0x7fd7100afd60 2022-08-01T19:41:56.583 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:56.583 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:41:56.584 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": "cephfs_data", 2022-08-01T19:41:56.584 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_id": 20, 2022-08-01T19:41:56.585 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2 2022-08-01T19:41:56.585 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:41:56.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.582+0000 7fd714948700 1 -- 172.21.15.17:0/2400539696 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd6ec053aa0 msgr2=0x7fd6ec055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:56.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.582+0000 7fd714948700 1 --2- 172.21.15.17:0/2400539696 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd6ec053aa0 0x7fd6ec055f60 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7fd70001c000 tx=0x7fd700015000).stop 2022-08-01T19:41:56.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.582+0000 7fd714948700 1 -- 172.21.15.17:0/2400539696 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd7100afd60 msgr2=0x7fd7100b54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:56.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.582+0000 7fd714948700 1 --2- 172.21.15.17:0/2400539696 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd7100afd60 0x7fd7100b54d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fd71006b950 tx=0x7fd704007b80).stop 2022-08-01T19:41:56.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.582+0000 7fd714948700 1 -- 172.21.15.17:0/2400539696 shutdown_connections 2022-08-01T19:41:56.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.582+0000 7fd714948700 1 --2- 172.21.15.17:0/2400539696 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd6ec053aa0 0x7fd6ec055f60 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:56.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.582+0000 7fd714948700 1 --2- 172.21.15.17:0/2400539696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7100b2710 0x7fd710159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:56.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.582+0000 7fd714948700 1 --2- 172.21.15.17:0/2400539696 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7100b01a0 0x7fd71014aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:56.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.582+0000 7fd714948700 1 --2- 172.21.15.17:0/2400539696 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd7100afd60 0x7fd7100b54d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:56.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.582+0000 7fd714948700 1 -- 172.21.15.17:0/2400539696 >> 172.21.15.17:0/2400539696 conn(0x7fd710060310 msgr2=0x7fd71014d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:56.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.582+0000 7fd714948700 1 -- 172.21.15.17:0/2400539696 shutdown_connections 2022-08-01T19:41:56.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:56.582+0000 7fd714948700 1 -- 172.21.15.17:0/2400539696 wait complete. 2022-08-01T19:41:56.599 INFO:teuthology.orchestra.run:Running command with timeout 10800 2022-08-01T19:41:56.600 DEBUG:teuthology.orchestra.run.smithi085:> cd /home/ubuntu/cephtest/mnt.0 && sudo df . 2022-08-01T19:41:56.637 INFO:teuthology.orchestra.run.smithi085.stdout:Filesystem 1K-blocks Used Available Use% Mounted on 2022-08-01T19:41:56.638 INFO:teuthology.orchestra.run.smithi085.stdout:ceph-fuse 354410496 0 354410496 0% /home/ubuntu/cephtest/mnt.0 2022-08-01T19:41:56.639 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T19:41:56.640 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:41:56.665 DEBUG:teuthology.orchestra.run.smithi077:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:41:56.699 DEBUG:teuthology.orchestra.run.smithi085:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:41:56.740 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:41:56.766 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T19:41:56.769 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:56.770 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:41:56.770 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:41:56.790 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:41:56.792 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:41:56.793 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi085.front.sepia.ceph.com... 2022-08-01T19:41:56.793 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T19:41:56.794 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:41:56.820 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:ceph-fuse[23358]: fuse finished with error 0 and tester_r 0 2022-08-01T19:41:56.822 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:56.823 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:41:56.825 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:41:56.879 INFO:teuthology.orchestra.run.smithi085.stdout:ext2/ext3 2022-08-01T19:41:56.879 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T19:41:57.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T19:41:57.006 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:41:57.024 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:57.025 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:41:57.025 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:41:57.036 INFO:teuthology.orchestra.run.smithi190.stdout:fuseblk 2022-08-01T19:41:57.038 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T19:41:57.039 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi190.front.sepia.ceph.com... 2022-08-01T19:41:57.040 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T19:41:57.042 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:41:57.101 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:ceph-fuse[22932]: fuse finished with error 0 and tester_r 0 2022-08-01T19:41:57.104 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:41:57.104 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:41:57.105 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:41:57.114 INFO:teuthology.orchestra.run.smithi190.stdout:ext2/ext3 2022-08-01T19:41:57.116 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T19:41:57.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T19:41:57.242 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:41:57.252 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T19:41:57.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.362+0000 7f65658e9700 1 Processor -- start 2022-08-01T19:41:57.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.362+0000 7f65658e9700 1 -- start start 2022-08-01T19:41:57.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.366+0000 7f65658e9700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f65600b2730 0x7f65600666b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:57.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.366+0000 7f65658e9700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6560066ba0 0x7f6560066fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:57.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.366+0000 7f65658e9700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65600654c0 con 0x7f65600b3190 2022-08-01T19:41:57.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.366+0000 7f65658e9700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6560062190 con 0x7f65600b2730 2022-08-01T19:41:57.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.366+0000 7f65658e9700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65600b3570 con 0x7f6560066ba0 2022-08-01T19:41:57.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.366+0000 7f65650e8700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6560066ba0 0x7f6560066fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:57.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.366+0000 7f655ffff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f65600b2730 0x7f65600666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:57.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.366+0000 7f655ffff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f65600b2730 0x7f65600666b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:40874/0 (socket says 172.21.15.17:40874) 2022-08-01T19:41:57.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.366+0000 7f655ffff700 1 -- 172.21.15.17:0/189300605 learned_addr learned my addr 172.21.15.17:0/189300605 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:57.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.366+0000 7f65650e8700 1 -- 172.21.15.17:0/189300605 >> v2:172.21.15.77:3300/0 conn(0x7f65600b2730 msgr2=0x7f65600666b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:57.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.366+0000 7f65650e8700 1 --2- 172.21.15.17:0/189300605 >> v2:172.21.15.77:3300/0 conn(0x7f65600b2730 0x7f65600666b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:57.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.366+0000 7f65650e8700 1 -- 172.21.15.17:0/189300605 >> v1:172.21.15.77:6789/0 conn(0x7f65600b3190 legacy=0x7f6560060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:57.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.366+0000 7f65650e8700 1 -- 172.21.15.17:0/189300605 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656005fbf0 con 0x7f6560066ba0 2022-08-01T19:41:57.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.366+0000 7f65650e8700 1 --2- 172.21.15.17:0/189300605 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6560066ba0 0x7f6560066fe0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f65580032f0 tx=0x7f65580091d0).ready entity=mon.2 client_cookie=b2853cde48ac3820 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:57.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.366+0000 7f655f7fe700 1 -- 172.21.15.17:0/189300605 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f655800b6c0 con 0x7f6560066ba0 2022-08-01T19:41:57.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.366+0000 7f655f7fe700 1 -- 172.21.15.17:0/189300605 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f655800b820 con 0x7f6560066ba0 2022-08-01T19:41:57.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.366+0000 7f655f7fe700 1 -- 172.21.15.17:0/189300605 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f655800b990 con 0x7f6560066ba0 2022-08-01T19:41:57.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.366+0000 7f65658e9700 1 -- 172.21.15.17:0/189300605 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6560066ba0 msgr2=0x7f6560066fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:57.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.366+0000 7f65658e9700 1 --2- 172.21.15.17:0/189300605 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6560066ba0 0x7f6560066fe0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f65580032f0 tx=0x7f65580091d0).stop 2022-08-01T19:41:57.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.366+0000 7f65658e9700 1 -- 172.21.15.17:0/189300605 shutdown_connections 2022-08-01T19:41:57.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.366+0000 7f65658e9700 1 --2- 172.21.15.17:0/189300605 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6560066ba0 0x7f6560066fe0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:57.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.366+0000 7f65658e9700 1 --2- 172.21.15.17:0/189300605 >> v2:172.21.15.77:3300/0 conn(0x7f65600b2730 0x7f65600666b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:57.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.366+0000 7f65658e9700 1 -- 172.21.15.17:0/189300605 >> 172.21.15.17:0/189300605 conn(0x7f6560063870 msgr2=0x7f6560063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:57.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.366+0000 7f65658e9700 1 -- 172.21.15.17:0/189300605 shutdown_connections 2022-08-01T19:41:57.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.366+0000 7f65658e9700 1 -- 172.21.15.17:0/189300605 wait complete. 2022-08-01T19:41:57.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.370+0000 7f65658e9700 1 Processor -- start 2022-08-01T19:41:57.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.370+0000 7f65658e9700 1 -- start start 2022-08-01T19:41:57.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.370+0000 7f65658e9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6560066ba0 0x7f656013e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:57.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.370+0000 7f65658e9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65600b2730 0x7f656014a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:57.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.370+0000 7f65658e9700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f65600b3190 0x7f6560155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:57.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.370+0000 7f65658e9700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65600aba50 con 0x7f65600b2730 2022-08-01T19:41:57.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.370+0000 7f65658e9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65600b4820 con 0x7f6560066ba0 2022-08-01T19:41:57.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.370+0000 7f65658e9700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6560155940 con 0x7f65600b3190 2022-08-01T19:41:57.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.370+0000 7f655ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65600b2730 0x7f656014a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:57.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.370+0000 7f655ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65600b2730 0x7f656014a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42958/0 (socket says 172.21.15.17:42958) 2022-08-01T19:41:57.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.370+0000 7f655ffff700 1 -- 172.21.15.17:0/2264017307 learned_addr learned my addr 172.21.15.17:0/2264017307 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:57.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.370+0000 7f65650e8700 1 --2- 172.21.15.17:0/2264017307 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f65600b3190 0x7f6560155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:57.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.370+0000 7f65648e7700 1 --2- 172.21.15.17:0/2264017307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6560066ba0 0x7f656013e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:57.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.370+0000 7f655ffff700 1 -- 172.21.15.17:0/2264017307 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f65600b3190 msgr2=0x7f6560155de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:57.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.370+0000 7f655ffff700 1 --2- 172.21.15.17:0/2264017307 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f65600b3190 0x7f6560155de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:57.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.370+0000 7f655ffff700 1 -- 172.21.15.17:0/2264017307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6560066ba0 msgr2=0x7f656013e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:57.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.370+0000 7f655ffff700 1 --2- 172.21.15.17:0/2264017307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6560066ba0 0x7f656013e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:57.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.370+0000 7f655ffff700 1 -- 172.21.15.17:0/2264017307 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6558009470 con 0x7f65600b2730 2022-08-01T19:41:57.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.370+0000 7f65648e7700 1 --2- 172.21.15.17:0/2264017307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6560066ba0 0x7f656013e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:57.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.370+0000 7f655ffff700 1 --2- 172.21.15.17:0/2264017307 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65600b2730 0x7f656014a1e0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 rx=0x7f654c00c000 tx=0x7f654c006eb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:57.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.370+0000 7f655dffb700 1 -- 172.21.15.17:0/2264017307 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f654c0126d0 con 0x7f65600b2730 2022-08-01T19:41:57.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.370+0000 7f655dffb700 1 -- 172.21.15.17:0/2264017307 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654c012830 con 0x7f65600b2730 2022-08-01T19:41:57.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.370+0000 7f65658e9700 1 -- 172.21.15.17:0/2264017307 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65600b4ac0 con 0x7f65600b2730 2022-08-01T19:41:57.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.370+0000 7f655dffb700 1 -- 172.21.15.17:0/2264017307 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f654c012b90 con 0x7f65600b2730 2022-08-01T19:41:57.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.370+0000 7f65658e9700 1 -- 172.21.15.17:0/2264017307 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6560152350 con 0x7f65600b2730 2022-08-01T19:41:57.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.374+0000 7f65658e9700 1 -- 172.21.15.17:0/2264017307 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f656005fe90 con 0x7f65600b2730 2022-08-01T19:41:57.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.374+0000 7f655dffb700 1 -- 172.21.15.17:0/2264017307 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f654c022070 con 0x7f65600b2730 2022-08-01T19:41:57.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.374+0000 7f655dffb700 1 --2- 172.21.15.17:0/2264017307 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f65400539e0 0x7f6540055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:57.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.374+0000 7f655dffb700 1 -- 172.21.15.17:0/2264017307 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f654c06c4f0 con 0x7f65600b2730 2022-08-01T19:41:57.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.374+0000 7f65648e7700 1 --2- 172.21.15.17:0/2264017307 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f65400539e0 0x7f6540055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:57.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.374+0000 7f65648e7700 1 --2- 172.21.15.17:0/2264017307 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f65400539e0 0x7f6540055ea0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f655400b000 tx=0x7f6554009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:57.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.382+0000 7f655dffb700 1 -- 172.21.15.17:0/2264017307 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f654c043140 con 0x7f65600b2730 2022-08-01T19:41:57.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.506+0000 7f65658e9700 1 -- 172.21.15.17:0/2264017307 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f65600574e0 con 0x7f65600b2730 2022-08-01T19:41:57.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.510+0000 7f655dffb700 1 -- 172.21.15.17:0/2264017307 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v107) v1 ==== 81+0+34051 (secure 0 0 0) 0x7f654c016020 con 0x7f65600b2730 2022-08-01T19:41:57.515 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:57.515 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:41:57.516 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 107, 2022-08-01T19:41:57.517 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", 2022-08-01T19:41:57.517 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2022-08-01T19:33:50.105743+0000", 2022-08-01T19:41:57.518 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2022-08-01T19:41:47.421949+0000", 2022-08-01T19:41:57.526 INFO:teuthology.orchestra.run.smithi017.stdout: "last_up_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:41:57.527 INFO:teuthology.orchestra.run.smithi017.stdout: "last_in_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:41:57.527 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T19:41:57.528 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_num": 5799936, 2022-08-01T19:41:57.528 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_set": [ 2022-08-01T19:41:57.529 INFO:teuthology.orchestra.run.smithi017.stdout: "pglog_hardlimit", 2022-08-01T19:41:57.529 INFO:teuthology.orchestra.run.smithi017.stdout: "purged_snapdirs", 2022-08-01T19:41:57.529 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_deletes", 2022-08-01T19:41:57.533 INFO:teuthology.orchestra.run.smithi017.stdout: "sortbitwise" 2022-08-01T19:41:57.533 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:41:57.534 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_version": 6, 2022-08-01T19:41:57.534 INFO:teuthology.orchestra.run.smithi017.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T19:41:57.534 INFO:teuthology.orchestra.run.smithi017.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T19:41:57.534 INFO:teuthology.orchestra.run.smithi017.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T19:41:57.535 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_snapshot": "", 2022-08-01T19:41:57.535 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_max": 20, 2022-08-01T19:41:57.535 INFO:teuthology.orchestra.run.smithi017.stdout: "max_osd": 8, 2022-08-01T19:41:57.535 INFO:teuthology.orchestra.run.smithi017.stdout: "require_min_compat_client": "jewel", 2022-08-01T19:41:57.536 INFO:teuthology.orchestra.run.smithi017.stdout: "min_compat_client": "jewel", 2022-08-01T19:41:57.536 INFO:teuthology.orchestra.run.smithi017.stdout: "require_osd_release": "octopus", 2022-08-01T19:41:57.536 INFO:teuthology.orchestra.run.smithi017.stdout: "pools": [ 2022-08-01T19:41:57.537 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.537 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 1, 2022-08-01T19:41:57.538 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "device_health_metrics", 2022-08-01T19:41:57.539 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:33:53.001841+0000", 2022-08-01T19:41:57.539 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:41:57.540 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:41:57.540 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:41:57.541 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:41:57.541 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:41:57.541 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:41:57.543 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:41:57.544 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:41:57.544 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 1, 2022-08-01T19:41:57.545 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 1, 2022-08-01T19:41:57.547 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 1, 2022-08-01T19:41:57.547 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 1, 2022-08-01T19:41:57.548 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 1, 2022-08-01T19:41:57.548 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:41:57.549 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:41:57.549 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:41:57.550 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:41:57.550 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:41:57.551 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:41:57.551 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:41:57.552 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.552 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "16", 2022-08-01T19:41:57.553 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:41:57.553 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:41:57.553 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:41:57.555 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:41:57.555 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:41:57.555 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:41:57.557 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:41:57.557 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:41:57.557 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:41:57.557 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:41:57.558 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:41:57.559 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:41:57.559 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:41:57.560 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:41:57.560 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:41:57.560 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:41:57.561 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:41:57.561 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:41:57.561 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:41:57.561 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:41:57.562 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:41:57.562 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:41:57.562 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:41:57.563 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:41:57.563 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:41:57.564 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:41:57.564 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.564 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:41:57.565 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:41:57.565 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:41:57.566 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:41:57.566 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:41:57.567 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:41:57.567 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:41:57.568 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:41:57.568 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:41:57.569 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:41:57.569 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:41:57.569 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:41:57.570 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 1 2022-08-01T19:41:57.570 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.571 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:41:57.571 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_devicehealth": {} 2022-08-01T19:41:57.572 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.572 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.573 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.573 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 2, 2022-08-01T19:41:57.574 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "rbd", 2022-08-01T19:41:57.574 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:34:01.553098+0000", 2022-08-01T19:41:57.575 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:41:57.575 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:41:57.576 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:41:57.576 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:41:57.576 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:41:57.577 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:41:57.577 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:41:57.578 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:41:57.579 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:41:57.580 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:41:57.581 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:41:57.581 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:41:57.582 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:41:57.583 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:41:57.583 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:41:57.584 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:41:57.584 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:41:57.584 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:41:57.585 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:41:57.585 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:41:57.585 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.585 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "19", 2022-08-01T19:41:57.585 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:41:57.586 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:41:57.586 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:41:57.586 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:41:57.587 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:41:57.587 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:41:57.587 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:41:57.588 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:41:57.588 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:41:57.588 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:41:57.589 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:41:57.589 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:41:57.590 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:41:57.590 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:41:57.591 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:41:57.591 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:41:57.591 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:41:57.592 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:41:57.592 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:41:57.593 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:41:57.593 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:41:57.594 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:41:57.594 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:41:57.595 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:41:57.595 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:41:57.595 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:41:57.596 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.596 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:41:57.597 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:41:57.597 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:41:57.598 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:41:57.598 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:41:57.599 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:41:57.599 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:41:57.599 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:41:57.600 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:41:57.600 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:41:57.601 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:41:57.601 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:41:57.602 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:41:57.602 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd": {} 2022-08-01T19:41:57.602 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.603 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.603 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.604 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 19, 2022-08-01T19:41:57.604 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "cephfs_metadata", 2022-08-01T19:41:57.605 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:41:45.865183+0000", 2022-08-01T19:41:57.606 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:41:57.606 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:41:57.607 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:41:57.607 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:41:57.608 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:41:57.608 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:41:57.609 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:41:57.609 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:41:57.610 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:41:57.610 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:41:57.610 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:41:57.611 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:41:57.611 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:41:57.611 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:41:57.612 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:41:57.612 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:41:57.612 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:41:57.612 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:41:57.613 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:41:57.613 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:41:57.613 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.613 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "107", 2022-08-01T19:41:57.613 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:41:57.614 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:41:57.614 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:41:57.614 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:41:57.615 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:41:57.615 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:41:57.615 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:41:57.615 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:41:57.615 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:41:57.616 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:41:57.616 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:41:57.616 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:41:57.616 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:41:57.617 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:41:57.617 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:41:57.617 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:41:57.617 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:41:57.617 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:41:57.618 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:41:57.618 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:41:57.618 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:41:57.618 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:41:57.619 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:41:57.619 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:41:57.619 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:41:57.619 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:41:57.620 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.620 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:41:57.620 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:41:57.620 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:41:57.620 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:41:57.621 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:41:57.621 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:41:57.621 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:41:57.621 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:41:57.622 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:41:57.622 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:41:57.622 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:41:57.622 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:41:57.623 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_bias": 4, 2022-08-01T19:41:57.623 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 16, 2022-08-01T19:41:57.623 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_priority": 5 2022-08-01T19:41:57.623 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.623 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:41:57.624 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs": { 2022-08-01T19:41:57.624 INFO:teuthology.orchestra.run.smithi017.stdout: "metadata": "cephfs" 2022-08-01T19:41:57.624 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.624 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.625 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.625 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.625 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 20, 2022-08-01T19:41:57.625 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "cephfs_data", 2022-08-01T19:41:57.626 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:41:46.670831+0000", 2022-08-01T19:41:57.626 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:41:57.626 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:41:57.626 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:41:57.627 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:41:57.627 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:41:57.627 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:41:57.628 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:41:57.628 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:41:57.628 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:41:57.628 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:41:57.629 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:41:57.629 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:41:57.629 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:41:57.629 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:41:57.630 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:41:57.630 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:41:57.630 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:41:57.630 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:41:57.631 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:41:57.631 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:41:57.631 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.631 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "107", 2022-08-01T19:41:57.632 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:41:57.632 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:41:57.632 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:41:57.632 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:41:57.633 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:41:57.633 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:41:57.633 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:41:57.634 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:41:57.634 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:41:57.635 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:41:57.635 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:41:57.636 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:41:57.636 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:41:57.637 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:41:57.637 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:41:57.637 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:41:57.638 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:41:57.638 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:41:57.639 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:41:57.640 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:41:57.640 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:41:57.641 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:41:57.641 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:41:57.642 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:41:57.643 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:41:57.643 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:41:57.644 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.644 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:41:57.644 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:41:57.645 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:41:57.645 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:41:57.645 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:41:57.645 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:41:57.646 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:41:57.646 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:41:57.646 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:41:57.647 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:41:57.647 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:41:57.647 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:41:57.648 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:41:57.648 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs": { 2022-08-01T19:41:57.649 INFO:teuthology.orchestra.run.smithi017.stdout: "data": "cephfs" 2022-08-01T19:41:57.649 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.650 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.650 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.651 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:41:57.651 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2022-08-01T19:41:57.651 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.652 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:41:57.652 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a962f14f-cf16-4fa8-84ff-c85325db9132", 2022-08-01T19:41:57.653 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:57.653 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:57.654 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:57.654 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:57.655 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:57.655 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:57.655 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:41:57.656 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 105, 2022-08-01T19:41:57.656 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:57.657 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:57.657 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:57.658 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:57.658 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.659 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:57.659 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6800", 2022-08-01T19:41:57.660 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:57.660 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.661 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.661 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:57.661 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6801", 2022-08-01T19:41:57.662 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:57.662 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.663 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.663 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.664 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:57.664 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:57.665 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.665 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:57.667 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6802", 2022-08-01T19:41:57.667 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:57.668 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.668 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.670 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:57.670 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6803", 2022-08-01T19:41:57.671 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:57.671 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.672 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.672 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.673 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:57.673 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:57.673 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.674 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:57.674 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6806", 2022-08-01T19:41:57.675 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:57.675 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.676 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.676 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:57.677 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6807", 2022-08-01T19:41:57.677 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:57.678 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.678 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.679 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.679 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:57.679 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:57.680 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.680 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:57.681 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6804", 2022-08-01T19:41:57.681 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:57.682 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.682 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.683 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:57.683 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6805", 2022-08-01T19:41:57.683 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:41:57.684 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.684 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.684 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.685 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6801/23008", 2022-08-01T19:41:57.685 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6803/23008", 2022-08-01T19:41:57.685 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6807/23008", 2022-08-01T19:41:57.685 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6805/23008", 2022-08-01T19:41:57.686 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:57.686 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:57.686 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:57.687 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.687 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.687 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.687 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:41:57.688 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "805f7051-a527-4e98-8844-b6e406a63248", 2022-08-01T19:41:57.688 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:57.688 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:57.688 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:57.689 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:57.689 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:57.689 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:57.689 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:41:57.690 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 105, 2022-08-01T19:41:57.690 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:57.690 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:57.691 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:57.691 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:57.691 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.692 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:57.692 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6808", 2022-08-01T19:41:57.693 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:57.693 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.693 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.693 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:57.693 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6809", 2022-08-01T19:41:57.694 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:57.694 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.694 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.694 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.695 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:57.695 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:57.695 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.695 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:57.696 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6810", 2022-08-01T19:41:57.696 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:57.696 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.696 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.697 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:57.697 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6811", 2022-08-01T19:41:57.697 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:57.698 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.698 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.699 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.699 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:57.700 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:57.700 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.700 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:57.701 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6814", 2022-08-01T19:41:57.701 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:57.702 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.702 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.703 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:57.703 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6815", 2022-08-01T19:41:57.704 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:57.704 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.705 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.705 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.705 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:57.706 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:57.706 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.707 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:57.707 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6812", 2022-08-01T19:41:57.707 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:57.708 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.708 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.709 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:57.709 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6813", 2022-08-01T19:41:57.710 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:41:57.710 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.711 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.711 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.711 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6809/23007", 2022-08-01T19:41:57.711 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6811/23007", 2022-08-01T19:41:57.712 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6815/23007", 2022-08-01T19:41:57.712 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6813/23007", 2022-08-01T19:41:57.712 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:57.713 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:57.713 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:57.713 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.714 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.714 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.714 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:41:57.715 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "89f7f3df-7c10-48e1-8b69-9e4243923a61", 2022-08-01T19:41:57.715 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:57.716 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:57.716 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:57.717 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:57.717 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:57.717 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:57.718 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:41:57.718 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 104, 2022-08-01T19:41:57.719 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:57.719 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:57.720 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:57.720 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:57.721 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.721 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:57.721 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6817", 2022-08-01T19:41:57.722 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:57.722 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.723 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.723 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:57.724 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6819", 2022-08-01T19:41:57.724 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:57.725 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.725 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.725 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.726 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:57.726 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:57.727 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.727 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:57.728 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6822", 2022-08-01T19:41:57.728 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:57.729 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.729 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.730 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:57.730 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6824", 2022-08-01T19:41:57.731 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:57.731 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.731 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.732 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.732 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:57.733 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:57.733 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.734 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:57.734 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6830", 2022-08-01T19:41:57.735 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:57.735 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.736 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.736 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:57.736 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6831", 2022-08-01T19:41:57.737 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:57.737 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.738 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.738 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.739 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:57.739 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:57.740 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.740 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:57.741 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6826", 2022-08-01T19:41:57.741 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:57.741 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.742 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.742 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:57.743 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6829", 2022-08-01T19:41:57.743 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:41:57.743 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.744 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.745 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.745 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6819/23010", 2022-08-01T19:41:57.745 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6824/23010", 2022-08-01T19:41:57.746 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6831/23010", 2022-08-01T19:41:57.746 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6829/23010", 2022-08-01T19:41:57.747 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:57.747 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:57.748 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:57.748 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.749 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.749 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.750 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:41:57.750 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "96123468-aabc-4a64-909e-a192c369090d", 2022-08-01T19:41:57.751 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:57.752 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:57.752 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:57.753 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:57.753 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:57.756 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:57.756 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:41:57.757 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 105, 2022-08-01T19:41:57.758 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:57.758 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:57.759 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:57.759 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:57.760 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.760 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:57.761 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6816", 2022-08-01T19:41:57.761 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:57.762 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.762 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.763 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:57.763 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6818", 2022-08-01T19:41:57.764 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:57.764 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.766 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.767 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.767 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:57.768 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:57.768 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.769 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:57.769 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6820", 2022-08-01T19:41:57.769 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:57.770 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.770 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.771 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:57.771 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6821", 2022-08-01T19:41:57.772 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:57.772 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.773 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.773 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.773 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:57.773 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:57.774 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.774 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:57.774 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6827", 2022-08-01T19:41:57.774 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:57.775 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.775 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.775 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:57.775 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6828", 2022-08-01T19:41:57.776 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:57.776 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.777 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.777 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.777 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:57.777 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:57.778 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.778 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:57.778 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6823", 2022-08-01T19:41:57.778 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:57.779 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.780 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.780 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:57.781 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6825", 2022-08-01T19:41:57.781 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:41:57.781 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.782 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.782 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.782 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6818/23009", 2022-08-01T19:41:57.782 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6821/23009", 2022-08-01T19:41:57.782 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6828/23009", 2022-08-01T19:41:57.783 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6825/23009", 2022-08-01T19:41:57.783 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:57.783 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:57.784 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:57.784 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.784 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.784 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.785 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:41:57.785 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a6e26665-e04b-46ec-ab66-e912acbf044c", 2022-08-01T19:41:57.785 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:57.785 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:57.786 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:57.786 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:57.786 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:57.786 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:57.786 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:41:57.787 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 104, 2022-08-01T19:41:57.787 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:57.788 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:57.788 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:57.788 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:57.788 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.789 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:57.789 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6810", 2022-08-01T19:41:57.789 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:57.789 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.790 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.790 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:57.790 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6811", 2022-08-01T19:41:57.790 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:57.791 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.791 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.791 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.792 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:57.792 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:57.792 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.792 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:57.793 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6812", 2022-08-01T19:41:57.793 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:57.793 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.794 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.794 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:57.794 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6813", 2022-08-01T19:41:57.794 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:57.795 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.795 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.795 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.795 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:57.795 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:57.796 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.796 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:57.796 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6816", 2022-08-01T19:41:57.796 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:57.797 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.797 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.797 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:57.798 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6817", 2022-08-01T19:41:57.798 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:57.798 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.799 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.799 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.799 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:57.799 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:57.800 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.800 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:57.800 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6814", 2022-08-01T19:41:57.800 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:57.800 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.801 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.801 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:57.801 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6815", 2022-08-01T19:41:57.801 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:41:57.802 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.802 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.802 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.802 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6811/23584", 2022-08-01T19:41:57.803 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6813/23584", 2022-08-01T19:41:57.803 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6817/23584", 2022-08-01T19:41:57.803 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6815/23584", 2022-08-01T19:41:57.803 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:57.803 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:57.804 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:57.804 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.804 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.804 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.805 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:41:57.805 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "502c0034-18ee-44ac-87ff-19b9dddb66a0", 2022-08-01T19:41:57.805 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:57.805 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:57.806 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:57.806 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:57.806 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:57.806 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:57.806 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:41:57.807 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 104, 2022-08-01T19:41:57.807 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:57.807 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:57.807 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:57.808 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:57.808 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.809 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:57.809 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6802", 2022-08-01T19:41:57.810 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:57.810 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.811 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.812 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:57.812 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6803", 2022-08-01T19:41:57.812 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:57.812 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.813 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.813 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.813 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:57.814 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:57.814 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.814 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:57.814 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6804", 2022-08-01T19:41:57.815 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:57.815 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.815 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.815 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:57.816 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6805", 2022-08-01T19:41:57.816 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:57.816 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.816 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.817 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.817 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:57.817 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:57.817 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.817 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:57.827 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6808", 2022-08-01T19:41:57.827 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:57.827 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.827 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.828 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:57.828 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6809", 2022-08-01T19:41:57.829 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:57.829 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.830 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.830 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.831 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:57.831 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:57.832 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.832 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:57.832 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6806", 2022-08-01T19:41:57.833 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:57.833 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.834 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.834 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:57.834 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6807", 2022-08-01T19:41:57.834 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:41:57.835 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.835 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.836 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.836 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6803/23585", 2022-08-01T19:41:57.836 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6805/23585", 2022-08-01T19:41:57.836 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6809/23585", 2022-08-01T19:41:57.837 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6807/23585", 2022-08-01T19:41:57.837 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:57.837 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:57.837 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:57.838 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.838 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.838 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.838 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:41:57.839 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "ac4e755a-7fe7-46cb-8407-852d5af9ffc5", 2022-08-01T19:41:57.839 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:57.839 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:57.839 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:57.840 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:57.840 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:57.840 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:57.840 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:41:57.841 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 105, 2022-08-01T19:41:57.841 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:57.841 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:57.842 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:57.842 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:57.842 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.842 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:57.843 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6820", 2022-08-01T19:41:57.844 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:57.845 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.845 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.845 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:57.845 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6821", 2022-08-01T19:41:57.846 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:57.846 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.846 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.847 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.847 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:57.847 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:57.847 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.848 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:57.848 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6823", 2022-08-01T19:41:57.848 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:57.849 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.849 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.849 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:57.850 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6825", 2022-08-01T19:41:57.851 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:57.852 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.853 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.853 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.854 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:57.854 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:57.855 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.855 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:57.855 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6831", 2022-08-01T19:41:57.856 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:57.856 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.856 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.857 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:57.857 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6833", 2022-08-01T19:41:57.857 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:57.857 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.858 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.858 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.858 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:57.858 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:57.859 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.859 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:57.859 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6827", 2022-08-01T19:41:57.859 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:57.860 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.860 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.860 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:57.860 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6829", 2022-08-01T19:41:57.862 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:41:57.862 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.862 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.862 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.863 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6821/23587", 2022-08-01T19:41:57.863 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6825/23587", 2022-08-01T19:41:57.863 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6833/23587", 2022-08-01T19:41:57.863 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6829/23587", 2022-08-01T19:41:57.864 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:57.865 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:57.865 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:57.866 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.866 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.866 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.867 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:41:57.867 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "874761e2-6410-40fc-b60c-e58abe5ce489", 2022-08-01T19:41:57.867 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:41:57.867 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:41:57.868 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:41:57.868 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:41:57.868 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:41:57.868 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:41:57.869 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:41:57.869 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 105, 2022-08-01T19:41:57.869 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:41:57.869 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:41:57.870 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:41:57.870 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:57.871 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.871 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:57.871 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6818", 2022-08-01T19:41:57.872 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:57.872 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.872 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.872 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:57.873 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6819", 2022-08-01T19:41:57.873 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:57.874 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.874 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.874 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.875 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:41:57.875 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:57.875 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.875 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:57.879 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6822", 2022-08-01T19:41:57.880 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:57.880 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.881 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.881 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:57.883 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6824", 2022-08-01T19:41:57.883 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:57.884 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.884 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.885 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.886 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:41:57.886 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:57.887 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.887 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:57.888 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6830", 2022-08-01T19:41:57.888 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:57.888 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.889 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.889 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:57.890 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6832", 2022-08-01T19:41:57.891 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:57.891 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.891 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.892 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.892 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:41:57.893 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:41:57.893 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.894 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:41:57.895 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6826", 2022-08-01T19:41:57.895 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:57.896 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.896 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.896 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:41:57.897 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6828", 2022-08-01T19:41:57.897 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:41:57.897 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.898 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.898 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.899 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6819/23586", 2022-08-01T19:41:57.899 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6824/23586", 2022-08-01T19:41:57.899 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6832/23586", 2022-08-01T19:41:57.899 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6828/23586", 2022-08-01T19:41:57.900 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:41:57.900 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:41:57.900 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:41:57.900 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:41:57.901 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.901 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:41:57.901 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_xinfo": [ 2022-08-01T19:41:57.901 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.902 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:41:57.902 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:57.902 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:57.903 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:57.903 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:57.903 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:57.903 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.588817+0000", 2022-08-01T19:41:57.904 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:57.904 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.904 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.904 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:41:57.905 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:57.905 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:57.905 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:57.905 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:57.906 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:57.906 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.782753+0000", 2022-08-01T19:41:57.907 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:57.907 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.907 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.907 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:41:57.908 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:57.908 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:57.908 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:57.908 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:57.909 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:57.909 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.837924+0000", 2022-08-01T19:41:57.909 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:57.909 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.910 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.911 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:41:57.911 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:57.911 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:57.911 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:57.912 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:57.912 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:57.912 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.839948+0000", 2022-08-01T19:41:57.912 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:57.913 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.913 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.913 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:41:57.913 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:57.914 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:57.914 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:57.914 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:57.914 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:57.914 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.112801+0000", 2022-08-01T19:41:57.915 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:57.915 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.915 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.915 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:41:57.917 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:57.917 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:57.918 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:57.918 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:57.919 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:57.919 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.106704+0000", 2022-08-01T19:41:57.919 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:57.920 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.920 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.922 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:41:57.922 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:57.923 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:57.924 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:57.924 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:57.925 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:57.925 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.893170+0000", 2022-08-01T19:41:57.926 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:57.926 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.926 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:41:57.928 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:41:57.928 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:41:57.928 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:41:57.929 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:41:57.929 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:41:57.929 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:41:57.930 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.886947+0000", 2022-08-01T19:41:57.930 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:41:57.930 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.930 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:41:57.931 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap": [], 2022-08-01T19:41:57.931 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap_items": [], 2022-08-01T19:41:57.931 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_temp": [], 2022-08-01T19:41:57.932 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_temp": [], 2022-08-01T19:41:57.932 INFO:teuthology.orchestra.run.smithi017.stdout: "blacklist": {}, 2022-08-01T19:41:57.932 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profiles": { 2022-08-01T19:41:57.932 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_data": { 2022-08-01T19:41:57.933 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-device-class": "", 2022-08-01T19:41:57.933 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:41:57.933 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-root": "default", 2022-08-01T19:41:57.933 INFO:teuthology.orchestra.run.smithi017.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T19:41:57.934 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:41:57.934 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "2", 2022-08-01T19:41:57.934 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:41:57.934 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:41:57.935 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van", 2022-08-01T19:41:57.935 INFO:teuthology.orchestra.run.smithi017.stdout: "w": "8" 2022-08-01T19:41:57.935 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.935 INFO:teuthology.orchestra.run.smithi017.stdout: "default": { 2022-08-01T19:41:57.936 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:41:57.936 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:41:57.936 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "1", 2022-08-01T19:41:57.936 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:41:57.936 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:41:57.937 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van" 2022-08-01T19:41:57.937 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:41:57.937 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:41:57.937 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps_queue": [], 2022-08-01T19:41:57.938 INFO:teuthology.orchestra.run.smithi017.stdout: "new_removed_snaps": [], 2022-08-01T19:41:57.938 INFO:teuthology.orchestra.run.smithi017.stdout: "new_purged_snaps": [], 2022-08-01T19:41:57.938 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_node_flags": {}, 2022-08-01T19:41:57.938 INFO:teuthology.orchestra.run.smithi017.stdout: "device_class_flags": {} 2022-08-01T19:41:57.939 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:41:57.940 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.510+0000 7f65658e9700 1 -- 172.21.15.17:0/2264017307 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f65400539e0 msgr2=0x7f6540055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:57.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.510+0000 7f65658e9700 1 --2- 172.21.15.17:0/2264017307 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f65400539e0 0x7f6540055ea0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f655400b000 tx=0x7f6554009040).stop 2022-08-01T19:41:57.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.510+0000 7f65658e9700 1 -- 172.21.15.17:0/2264017307 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65600b2730 msgr2=0x7f656014a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:57.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.510+0000 7f65658e9700 1 --2- 172.21.15.17:0/2264017307 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65600b2730 0x7f656014a1e0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 rx=0x7f654c00c000 tx=0x7f654c006eb0).stop 2022-08-01T19:41:57.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.510+0000 7f65658e9700 1 -- 172.21.15.17:0/2264017307 shutdown_connections 2022-08-01T19:41:57.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.510+0000 7f65658e9700 1 --2- 172.21.15.17:0/2264017307 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f65400539e0 0x7f6540055ea0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:57.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.510+0000 7f65658e9700 1 --2- 172.21.15.17:0/2264017307 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f65600b3190 0x7f6560155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:57.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.510+0000 7f65658e9700 1 --2- 172.21.15.17:0/2264017307 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65600b2730 0x7f656014a1e0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:57.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.510+0000 7f65658e9700 1 --2- 172.21.15.17:0/2264017307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6560066ba0 0x7f656013e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:57.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.510+0000 7f65658e9700 1 -- 172.21.15.17:0/2264017307 >> 172.21.15.17:0/2264017307 conn(0x7f6560063870 msgr2=0x7f6560147870 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:57.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.510+0000 7f65658e9700 1 -- 172.21.15.17:0/2264017307 shutdown_connections 2022-08-01T19:41:57.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:57.510+0000 7f65658e9700 1 -- 172.21.15.17:0/2264017307 wait complete. 2022-08-01T19:41:57.944 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:41:58.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.058+0000 7fe1d0e89700 1 Processor -- start 2022-08-01T19:41:58.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.058+0000 7fe1d0e89700 1 -- start start 2022-08-01T19:41:58.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.058+0000 7fe1d0e89700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fe1cc0b01a0 0x7fe1cc0a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:58.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.058+0000 7fe1d0e89700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1cc0afd60 0x7fe1cc0a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:58.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.058+0000 7fe1d0e89700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1cc0a7250 con 0x7fe1cc0b2710 2022-08-01T19:41:58.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.058+0000 7fe1d0e89700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1cc0a7480 con 0x7fe1cc0b01a0 2022-08-01T19:41:58.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.058+0000 7fe1d0e89700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc0a7720 con 0x7fe1cc0afd60 2022-08-01T19:41:58.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.058+0000 7fe1cb7fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe1cc0b2710 0x7fe1cc14afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35160/0 (socket says 172.21.15.17:35160) 2022-08-01T19:41:58.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.058+0000 7fe1cb7fe700 1 -- 172.21.15.17:0/3294235044 learned_addr learned my addr 172.21.15.17:0/3294235044 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:58.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.058+0000 7fe1caffd700 1 --2- 172.21.15.17:0/3294235044 >> v2:172.21.15.77:3300/0 conn(0x7fe1cc0b01a0 0x7fe1cc0a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:58.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.058+0000 7fe1cbfff700 1 --2- 172.21.15.17:0/3294235044 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1cc0afd60 0x7fe1cc0a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:58.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.058+0000 7fe1ca7fc700 1 -- 172.21.15.17:0/3294235044 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 397608515 0 0) 0x7fe1cc0a7250 con 0x7fe1cc0b2710 2022-08-01T19:41:58.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.058+0000 7fe1ca7fc700 1 -- 172.21.15.17:0/3294235044 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1ac003260 con 0x7fe1cc0b2710 2022-08-01T19:41:58.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.058+0000 7fe1caffd700 1 -- 172.21.15.17:0/3294235044 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1cc0afd60 msgr2=0x7fe1cc0a8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:58.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.058+0000 7fe1caffd700 1 --2- 172.21.15.17:0/3294235044 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1cc0afd60 0x7fe1cc0a8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:58.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.058+0000 7fe1caffd700 1 -- 172.21.15.17:0/3294235044 >> v1:172.21.15.17:6789/0 conn(0x7fe1cc0b2710 legacy=0x7fe1cc14afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:58.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.058+0000 7fe1caffd700 1 -- 172.21.15.17:0/3294235044 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1cc0a79c0 con 0x7fe1cc0b01a0 2022-08-01T19:41:58.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.058+0000 7fe1cbfff700 1 --2- 172.21.15.17:0/3294235044 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1cc0afd60 0x7fe1cc0a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:41:58.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.058+0000 7fe1caffd700 1 --2- 172.21.15.17:0/3294235044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1cc0b01a0 0x7fe1cc0a8540 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7fe1bc0032f0 tx=0x7fe1bc0091d0).ready entity=mon.1 client_cookie=7253017985c3646 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:58.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.058+0000 7fe1ca7fc700 1 -- 172.21.15.17:0/3294235044 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1bc00b600 con 0x7fe1cc0b01a0 2022-08-01T19:41:58.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.058+0000 7fe1ca7fc700 1 -- 172.21.15.17:0/3294235044 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1bc00b760 con 0x7fe1cc0b01a0 2022-08-01T19:41:58.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.058+0000 7fe1ca7fc700 1 -- 172.21.15.17:0/3294235044 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1bc00b8d0 con 0x7fe1cc0b01a0 2022-08-01T19:41:58.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.062+0000 7fe1d0e89700 1 -- 172.21.15.17:0/3294235044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1cc0b01a0 msgr2=0x7fe1cc0a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:58.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.062+0000 7fe1d0e89700 1 --2- 172.21.15.17:0/3294235044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1cc0b01a0 0x7fe1cc0a8540 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7fe1bc0032f0 tx=0x7fe1bc0091d0).stop 2022-08-01T19:41:58.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.062+0000 7fe1d0e89700 1 -- 172.21.15.17:0/3294235044 shutdown_connections 2022-08-01T19:41:58.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.062+0000 7fe1d0e89700 1 --2- 172.21.15.17:0/3294235044 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1cc0afd60 0x7fe1cc0a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:58.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.062+0000 7fe1d0e89700 1 --2- 172.21.15.17:0/3294235044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1cc0b01a0 0x7fe1cc0a8540 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:58.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.062+0000 7fe1d0e89700 1 -- 172.21.15.17:0/3294235044 >> 172.21.15.17:0/3294235044 conn(0x7fe1cc060310 msgr2=0x7fe1cc0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:58.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.062+0000 7fe1d0e89700 1 -- 172.21.15.17:0/3294235044 shutdown_connections 2022-08-01T19:41:58.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.062+0000 7fe1d0e89700 1 -- 172.21.15.17:0/3294235044 wait complete. 2022-08-01T19:41:58.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.062+0000 7fe1d0e89700 1 Processor -- start 2022-08-01T19:41:58.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.062+0000 7fe1d0e89700 1 -- start start 2022-08-01T19:41:58.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.062+0000 7fe1d0e89700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1cc0afd60 0x7fe1cc05ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:58.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.062+0000 7fe1d0e89700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1cc0b01a0 0x7fe1cc0aa500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:58.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.062+0000 7fe1d0e89700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1cc0b2710 0x7fe1cc0aa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:58.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.062+0000 7fe1d0e89700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc14b230 con 0x7fe1cc0b01a0 2022-08-01T19:41:58.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.062+0000 7fe1d0e89700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc0a9bc0 con 0x7fe1cc0b2710 2022-08-01T19:41:58.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.062+0000 7fe1d0e89700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc0aaee0 con 0x7fe1cc0afd60 2022-08-01T19:41:58.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.062+0000 7fe1caffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1cc0b01a0 0x7fe1cc0aa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:58.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.062+0000 7fe1caffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1cc0b01a0 0x7fe1cc0aa500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42972/0 (socket says 172.21.15.17:42972) 2022-08-01T19:41:58.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.062+0000 7fe1caffd700 1 -- 172.21.15.17:0/3309215389 learned_addr learned my addr 172.21.15.17:0/3309215389 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:58.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.062+0000 7fe1cb7fe700 1 --2- 172.21.15.17:0/3309215389 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1cc0afd60 0x7fe1cc05ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:58.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.066+0000 7fe1cbfff700 1 --2- 172.21.15.17:0/3309215389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1cc0b2710 0x7fe1cc0aa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:58.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.066+0000 7fe1caffd700 1 -- 172.21.15.17:0/3309215389 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1cc0afd60 msgr2=0x7fe1cc05ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:58.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.066+0000 7fe1caffd700 1 --2- 172.21.15.17:0/3309215389 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1cc0afd60 0x7fe1cc05ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:58.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.066+0000 7fe1caffd700 1 -- 172.21.15.17:0/3309215389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1cc0b2710 msgr2=0x7fe1cc0aa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:58.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.066+0000 7fe1caffd700 1 --2- 172.21.15.17:0/3309215389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1cc0b2710 0x7fe1cc0aa9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:58.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.066+0000 7fe1caffd700 1 -- 172.21.15.17:0/3309215389 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c400b180 con 0x7fe1cc0b01a0 2022-08-01T19:41:58.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.066+0000 7fe1cbfff700 1 --2- 172.21.15.17:0/3309215389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1cc0b2710 0x7fe1cc0aa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:58.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.066+0000 7fe1caffd700 1 --2- 172.21.15.17:0/3309215389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1cc0b01a0 0x7fe1cc0aa500 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 rx=0x7fe1bc0032f0 tx=0x7fe1bc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:58.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.066+0000 7fe1c8ff9700 1 -- 172.21.15.17:0/3309215389 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1bc00b8d0 con 0x7fe1cc0b01a0 2022-08-01T19:41:58.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.066+0000 7fe1c8ff9700 1 -- 172.21.15.17:0/3309215389 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1bc021630 con 0x7fe1cc0b01a0 2022-08-01T19:41:58.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.066+0000 7fe1d0e89700 1 -- 172.21.15.17:0/3309215389 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1bc009470 con 0x7fe1cc0b01a0 2022-08-01T19:41:58.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.066+0000 7fe1c8ff9700 1 -- 172.21.15.17:0/3309215389 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1bc00b600 con 0x7fe1cc0b01a0 2022-08-01T19:41:58.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.066+0000 7fe1d0e89700 1 -- 172.21.15.17:0/3309215389 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1cc0b38d0 con 0x7fe1cc0b01a0 2022-08-01T19:41:58.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.066+0000 7fe1d0e89700 1 -- 172.21.15.17:0/3309215389 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1cc0574e0 con 0x7fe1cc0b01a0 2022-08-01T19:41:58.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.066+0000 7fe1c8ff9700 1 -- 172.21.15.17:0/3309215389 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe1bc028070 con 0x7fe1cc0b01a0 2022-08-01T19:41:58.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.066+0000 7fe1c8ff9700 1 --2- 172.21.15.17:0/3309215389 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe1b0053b30 0x7fe1b0055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:58.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.066+0000 7fe1c8ff9700 1 -- 172.21.15.17:0/3309215389 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe1bc06d1d0 con 0x7fe1cc0b01a0 2022-08-01T19:41:58.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.070+0000 7fe1cb7fe700 1 --2- 172.21.15.17:0/3309215389 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe1b0053b30 0x7fe1b0055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:58.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.070+0000 7fe1cb7fe700 1 --2- 172.21.15.17:0/3309215389 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe1b0053b30 0x7fe1b0055ff0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7fe1b400d000 tx=0x7fe1b4009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:58.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.074+0000 7fe1c8ff9700 1 -- 172.21.15.17:0/3309215389 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe1bc043e90 con 0x7fe1cc0b01a0 2022-08-01T19:41:58.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.238+0000 7fe1d0e89700 1 -- 172.21.15.17:0/3309215389 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1cc0b3d10 con 0x7fe1cc0b01a0 2022-08-01T19:41:58.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.238+0000 7fe1c8ff9700 1 -- 172.21.15.17:0/3309215389 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+2490 (secure 0 0 0) 0x7fe1bc026090 con 0x7fe1cc0b01a0 2022-08-01T19:41:58.240 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:58.240 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":53,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7252,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/1455566688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1455566688},{"type":"v1","addr":"172.21.15.77:6835","nonce":1455566688}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":51},{"gid":7266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.17:6833/249859043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":249859043},{"type":"v1","addr":"172.21.15.17:6833","nonce":249859043}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":51}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:41:47.422244+0000","modified":"2022-08-01T19:41:48.425195+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7272":{"gid":7272,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/2991628239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2991628239},{"type":"v1","addr":"172.21.15.17:6835","nonce":2991628239}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2022-08-01T19:41:58.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.238+0000 7fe1d0e89700 1 -- 172.21.15.17:0/3309215389 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe1b0053b30 msgr2=0x7fe1b0055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:58.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.238+0000 7fe1d0e89700 1 --2- 172.21.15.17:0/3309215389 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe1b0053b30 0x7fe1b0055ff0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7fe1b400d000 tx=0x7fe1b4009040).stop 2022-08-01T19:41:58.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.238+0000 7fe1d0e89700 1 -- 172.21.15.17:0/3309215389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1cc0b01a0 msgr2=0x7fe1cc0aa500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:58.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.238+0000 7fe1d0e89700 1 --2- 172.21.15.17:0/3309215389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1cc0b01a0 0x7fe1cc0aa500 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 rx=0x7fe1bc0032f0 tx=0x7fe1bc016000).stop 2022-08-01T19:41:58.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.242+0000 7fe1d0e89700 1 -- 172.21.15.17:0/3309215389 shutdown_connections 2022-08-01T19:41:58.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.242+0000 7fe1d0e89700 1 --2- 172.21.15.17:0/3309215389 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe1b0053b30 0x7fe1b0055ff0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:58.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.242+0000 7fe1d0e89700 1 --2- 172.21.15.17:0/3309215389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1cc0b2710 0x7fe1cc0aa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:58.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.242+0000 7fe1d0e89700 1 --2- 172.21.15.17:0/3309215389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1cc0b01a0 0x7fe1cc0aa500 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:58.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.242+0000 7fe1d0e89700 1 --2- 172.21.15.17:0/3309215389 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1cc0afd60 0x7fe1cc05ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:58.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.242+0000 7fe1d0e89700 1 -- 172.21.15.17:0/3309215389 >> 172.21.15.17:0/3309215389 conn(0x7fe1cc060310 msgr2=0x7fe1cc0ad390 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:58.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.242+0000 7fe1d0e89700 1 -- 172.21.15.17:0/3309215389 shutdown_connections 2022-08-01T19:41:58.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.242+0000 7fe1d0e89700 1 -- 172.21.15.17:0/3309215389 wait complete. 2022-08-01T19:41:58.245 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 53 2022-08-01T19:41:58.254 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-08-01T19:41:58.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.358+0000 7fbd38a06700 1 Processor -- start 2022-08-01T19:41:58.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.358+0000 7fbd38a06700 1 -- start start 2022-08-01T19:41:58.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.362+0000 7fbd38a06700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fbd340619b0 0x7fbd340b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:58.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.362+0000 7fbd38a06700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbd340b6ee0 0x7fbd34143890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:58.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.362+0000 7fbd38a06700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbd340b5ed0 0x7fbd340b5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:58.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.362+0000 7fbd38a06700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd34142f60 con 0x7fbd340b5ed0 2022-08-01T19:41:58.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.362+0000 7fbd38a06700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd34061d90 con 0x7fbd340619b0 2022-08-01T19:41:58.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.362+0000 7fbd38a06700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd340ba5a0 con 0x7fbd340b6ee0 2022-08-01T19:41:58.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.362+0000 7fbd3359e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbd340b5ed0 0x7fbd340b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:58.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.362+0000 7fbd3359e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbd340b5ed0 0x7fbd340b5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42984/0 (socket says 172.21.15.17:42984) 2022-08-01T19:41:58.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.362+0000 7fbd3259c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbd340b6ee0 0x7fbd34143890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:58.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.362+0000 7fbd3359e700 1 -- 172.21.15.17:0/1967886297 learned_addr learned my addr 172.21.15.17:0/1967886297 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:58.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.362+0000 7fbd32d9d700 1 --2- 172.21.15.17:0/1967886297 >> v2:172.21.15.77:3300/0 conn(0x7fbd340619b0 0x7fbd340b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:58.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.362+0000 7fbd3259c700 1 -- 172.21.15.17:0/1967886297 >> v2:172.21.15.77:3300/0 conn(0x7fbd340619b0 msgr2=0x7fbd340b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:58.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.362+0000 7fbd3259c700 1 --2- 172.21.15.17:0/1967886297 >> v2:172.21.15.77:3300/0 conn(0x7fbd340619b0 0x7fbd340b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:58.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.362+0000 7fbd3259c700 1 -- 172.21.15.17:0/1967886297 >> v2:172.21.15.17:3300/0 conn(0x7fbd340b5ed0 msgr2=0x7fbd340b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:41:58.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.362+0000 7fbd3259c700 1 --2- 172.21.15.17:0/1967886297 >> v2:172.21.15.17:3300/0 conn(0x7fbd340b5ed0 0x7fbd340b5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:58.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.362+0000 7fbd3259c700 1 -- 172.21.15.17:0/1967886297 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd34143150 con 0x7fbd340b6ee0 2022-08-01T19:41:58.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.362+0000 7fbd3259c700 1 --2- 172.21.15.17:0/1967886297 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbd340b6ee0 0x7fbd34143890 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fbd24002f70 tx=0x7fbd24006df0).ready entity=mon.2 client_cookie=1b896b6f47b37ab server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:58.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.362+0000 7fbd31d9b700 1 -- 172.21.15.17:0/1967886297 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd2400d4a0 con 0x7fbd340b6ee0 2022-08-01T19:41:58.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.362+0000 7fbd31d9b700 1 -- 172.21.15.17:0/1967886297 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd24000e90 con 0x7fbd340b6ee0 2022-08-01T19:41:58.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.362+0000 7fbd31d9b700 1 -- 172.21.15.17:0/1967886297 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd2400d7f0 con 0x7fbd340b6ee0 2022-08-01T19:41:58.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.362+0000 7fbd38a06700 1 -- 172.21.15.17:0/1967886297 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbd340b6ee0 msgr2=0x7fbd34143890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:58.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.362+0000 7fbd38a06700 1 --2- 172.21.15.17:0/1967886297 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbd340b6ee0 0x7fbd34143890 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fbd24002f70 tx=0x7fbd24006df0).stop 2022-08-01T19:41:58.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.362+0000 7fbd38a06700 1 -- 172.21.15.17:0/1967886297 shutdown_connections 2022-08-01T19:41:58.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.362+0000 7fbd38a06700 1 --2- 172.21.15.17:0/1967886297 >> v2:172.21.15.17:3300/0 conn(0x7fbd340b5ed0 0x7fbd340b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:58.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.362+0000 7fbd38a06700 1 --2- 172.21.15.17:0/1967886297 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbd340b6ee0 0x7fbd34143890 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:58.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.362+0000 7fbd38a06700 1 --2- 172.21.15.17:0/1967886297 >> v2:172.21.15.77:3300/0 conn(0x7fbd340619b0 0x7fbd340b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:58.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.362+0000 7fbd38a06700 1 -- 172.21.15.17:0/1967886297 >> 172.21.15.17:0/1967886297 conn(0x7fbd3406af80 msgr2=0x7fbd3405fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:58.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.362+0000 7fbd38a06700 1 -- 172.21.15.17:0/1967886297 shutdown_connections 2022-08-01T19:41:58.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.362+0000 7fbd38a06700 1 -- 172.21.15.17:0/1967886297 wait complete. 2022-08-01T19:41:58.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.366+0000 7fbd38a06700 1 Processor -- start 2022-08-01T19:41:58.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.366+0000 7fbd38a06700 1 -- start start 2022-08-01T19:41:58.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.366+0000 7fbd38a06700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd340619b0 0x7fbd340b9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:58.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.366+0000 7fbd38a06700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbd340b5ed0 0x7fbd34144870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:58.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.366+0000 7fbd38a06700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd340b6ee0 0x7fbd34144d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:58.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.366+0000 7fbd38a06700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3414a240 con 0x7fbd340619b0 2022-08-01T19:41:58.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.366+0000 7fbd38a06700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd34149880 con 0x7fbd340b6ee0 2022-08-01T19:41:58.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.366+0000 7fbd38a06700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd34149fa0 con 0x7fbd340b5ed0 2022-08-01T19:41:58.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.366+0000 7fbd32d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd340619b0 0x7fbd340b9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:58.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.366+0000 7fbd3259c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbd340b5ed0 0x7fbd34144870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:58.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.366+0000 7fbd32d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd340619b0 0x7fbd340b9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42988/0 (socket says 172.21.15.17:42988) 2022-08-01T19:41:58.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.366+0000 7fbd32d9d700 1 -- 172.21.15.17:0/4100408158 learned_addr learned my addr 172.21.15.17:0/4100408158 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:58.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.366+0000 7fbd3359e700 1 --2- 172.21.15.17:0/4100408158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd340b6ee0 0x7fbd34144d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:58.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.366+0000 7fbd3259c700 1 -- 172.21.15.17:0/4100408158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd340b6ee0 msgr2=0x7fbd34144d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:58.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.366+0000 7fbd3259c700 1 --2- 172.21.15.17:0/4100408158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd340b6ee0 0x7fbd34144d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:58.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.366+0000 7fbd3259c700 1 -- 172.21.15.17:0/4100408158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd340619b0 msgr2=0x7fbd340b9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:58.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.366+0000 7fbd3259c700 1 --2- 172.21.15.17:0/4100408158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd340619b0 0x7fbd340b9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:58.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.366+0000 7fbd3259c700 1 -- 172.21.15.17:0/4100408158 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd2400b180 con 0x7fbd340b5ed0 2022-08-01T19:41:58.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.366+0000 7fbd3259c700 1 --2- 172.21.15.17:0/4100408158 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbd340b5ed0 0x7fbd34144870 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fbd24002f70 tx=0x7fbd24014000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:58.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.366+0000 7fbd2bfff700 1 -- 172.21.15.17:0/4100408158 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd24013da0 con 0x7fbd340b5ed0 2022-08-01T19:41:58.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.366+0000 7fbd2bfff700 1 -- 172.21.15.17:0/4100408158 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd2400de50 con 0x7fbd340b5ed0 2022-08-01T19:41:58.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.366+0000 7fbd2bfff700 1 -- 172.21.15.17:0/4100408158 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd2401e5f0 con 0x7fbd340b5ed0 2022-08-01T19:41:58.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.366+0000 7fbd38a06700 1 -- 172.21.15.17:0/4100408158 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd34149300 con 0x7fbd340b5ed0 2022-08-01T19:41:58.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.366+0000 7fbd38a06700 1 -- 172.21.15.17:0/4100408158 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd340ad9b0 con 0x7fbd340b5ed0 2022-08-01T19:41:58.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.370+0000 7fbd38a06700 1 -- 172.21.15.17:0/4100408158 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd34144610 con 0x7fbd340b5ed0 2022-08-01T19:41:58.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.370+0000 7fbd2bfff700 1 -- 172.21.15.17:0/4100408158 <== mon.2 v2:172.21.15.77:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbd2401c070 con 0x7fbd340b5ed0 2022-08-01T19:41:58.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.370+0000 7fbd2bfff700 1 --2- 172.21.15.17:0/4100408158 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbd18053b00 0x7fbd18055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:58.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.370+0000 7fbd2bfff700 1 -- 172.21.15.17:0/4100408158 <== mon.2 v2:172.21.15.77:3301/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fbd2406c9c0 con 0x7fbd340b5ed0 2022-08-01T19:41:58.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.370+0000 7fbd32d9d700 1 --2- 172.21.15.17:0/4100408158 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbd18053b00 0x7fbd18055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:58.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.370+0000 7fbd32d9d700 1 --2- 172.21.15.17:0/4100408158 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbd18053b00 0x7fbd18055fc0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7fbd1c00b000 tx=0x7fbd1c006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:58.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.378+0000 7fbd2bfff700 1 -- 172.21.15.17:0/4100408158 <== mon.2 v2:172.21.15.77:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbd24043670 con 0x7fbd340b5ed0 2022-08-01T19:41:58.550 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:58.542+0000 7fbd38a06700 1 -- 172.21.15.17:0/4100408158 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fbd340574e0 con 0x7fbd340b5ed0 2022-08-01T19:41:59.495 INFO:tasks.ceph.mon.a.smithi017.stderr:2022-08-01T19:41:59.494+0000 7f6a53773700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-08-01T19:41:59.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.506+0000 7fbd2bfff700 1 -- 172.21.15.17:0/4100408158 <== mon.2 v2:172.21.15.77:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v54) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbd24023090 con 0x7fbd340b5ed0 2022-08-01T19:41:59.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.510+0000 7fbd38a06700 1 -- 172.21.15.17:0/4100408158 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbd18053b00 msgr2=0x7fbd18055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:59.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.510+0000 7fbd38a06700 1 --2- 172.21.15.17:0/4100408158 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbd18053b00 0x7fbd18055fc0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7fbd1c00b000 tx=0x7fbd1c006df0).stop 2022-08-01T19:41:59.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.510+0000 7fbd38a06700 1 -- 172.21.15.17:0/4100408158 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbd340b5ed0 msgr2=0x7fbd34144870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:59.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.510+0000 7fbd38a06700 1 --2- 172.21.15.17:0/4100408158 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbd340b5ed0 0x7fbd34144870 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fbd24002f70 tx=0x7fbd24014000).stop 2022-08-01T19:41:59.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.514+0000 7fbd38a06700 1 -- 172.21.15.17:0/4100408158 shutdown_connections 2022-08-01T19:41:59.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.514+0000 7fbd38a06700 1 --2- 172.21.15.17:0/4100408158 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbd18053b00 0x7fbd18055fc0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:59.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.514+0000 7fbd38a06700 1 --2- 172.21.15.17:0/4100408158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd340b6ee0 0x7fbd34144d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:59.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.514+0000 7fbd38a06700 1 --2- 172.21.15.17:0/4100408158 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbd340b5ed0 0x7fbd34144870 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:59.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.514+0000 7fbd38a06700 1 --2- 172.21.15.17:0/4100408158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd340619b0 0x7fbd340b9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:59.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.514+0000 7fbd38a06700 1 -- 172.21.15.17:0/4100408158 >> 172.21.15.17:0/4100408158 conn(0x7fbd3406af80 msgr2=0x7fbd341405d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:59.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.514+0000 7fbd38a06700 1 -- 172.21.15.17:0/4100408158 shutdown_connections 2022-08-01T19:41:59.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.514+0000 7fbd38a06700 1 -- 172.21.15.17:0/4100408158 wait complete. 2022-08-01T19:41:59.521 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-08-01T19:41:59.551 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:41:59.552 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2022-08-01T19:41:59.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.654+0000 7f1d6f3de700 1 Processor -- start 2022-08-01T19:41:59.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.654+0000 7f1d6f3de700 1 -- start start 2022-08-01T19:41:59.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.658+0000 7f1d6f3de700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1d680b1160 0x7f1d681448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:59.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.658+0000 7f1d6f3de700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d680b04e0 con 0x7f1d680b1f40 2022-08-01T19:41:59.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.658+0000 7f1d6f3de700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d680b0780 con 0x7f1d68148530 2022-08-01T19:41:59.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.658+0000 7f1d6f3de700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d680a9250 con 0x7f1d680b1160 2022-08-01T19:41:59.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.658+0000 7f1d6d17a700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1d680b1f40 0x7f1d680b2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35200/0 (socket says 172.21.15.17:35200) 2022-08-01T19:41:59.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.658+0000 7f1d6d17a700 1 -- 172.21.15.17:0/664568445 learned_addr learned my addr 172.21.15.17:0/664568445 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:41:59.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.658+0000 7f1d6d97b700 1 --2- 172.21.15.17:0/664568445 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1d680b1160 0x7f1d681448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:59.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.658+0000 7f1d5ffff700 1 -- 172.21.15.17:0/664568445 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3004037818 0 0) 0x7f1d680b0780 con 0x7f1d68148530 2022-08-01T19:41:59.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.658+0000 7f1d5ffff700 1 -- 172.21.15.17:0/664568445 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d4c003260 con 0x7f1d68148530 2022-08-01T19:41:59.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.658+0000 7f1d5ffff700 1 -- 172.21.15.17:0/664568445 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2068813965 0 0) 0x7f1d680b04e0 con 0x7f1d680b1f40 2022-08-01T19:41:59.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.658+0000 7f1d5ffff700 1 -- 172.21.15.17:0/664568445 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d680b0780 con 0x7f1d680b1f40 2022-08-01T19:41:59.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.658+0000 7f1d6d97b700 1 -- 172.21.15.17:0/664568445 >> v1:172.21.15.77:6789/0 conn(0x7f1d68148530 legacy=0x7f1d680a7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:59.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.658+0000 7f1d6d97b700 1 -- 172.21.15.17:0/664568445 >> v1:172.21.15.17:6789/0 conn(0x7f1d680b1f40 legacy=0x7f1d680b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:59.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.658+0000 7f1d6d97b700 1 -- 172.21.15.17:0/664568445 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d680b0a20 con 0x7f1d680b1160 2022-08-01T19:41:59.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.658+0000 7f1d6d97b700 1 --2- 172.21.15.17:0/664568445 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1d680b1160 0x7f1d681448b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f1d681478c0 tx=0x7f1d600091d0).ready entity=mon.2 client_cookie=9625e821ccc0e563 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:59.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.658+0000 7f1d5ffff700 1 -- 172.21.15.17:0/664568445 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d6000b600 con 0x7f1d680b1160 2022-08-01T19:41:59.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.658+0000 7f1d5ffff700 1 -- 172.21.15.17:0/664568445 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6000b760 con 0x7f1d680b1160 2022-08-01T19:41:59.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.658+0000 7f1d5ffff700 1 -- 172.21.15.17:0/664568445 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d6000b8d0 con 0x7f1d680b1160 2022-08-01T19:41:59.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.658+0000 7f1d6f3de700 1 -- 172.21.15.17:0/664568445 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1d680b1160 msgr2=0x7f1d681448b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:59.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.658+0000 7f1d6f3de700 1 --2- 172.21.15.17:0/664568445 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1d680b1160 0x7f1d681448b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f1d681478c0 tx=0x7f1d600091d0).stop 2022-08-01T19:41:59.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.658+0000 7f1d6f3de700 1 -- 172.21.15.17:0/664568445 shutdown_connections 2022-08-01T19:41:59.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.658+0000 7f1d6f3de700 1 --2- 172.21.15.17:0/664568445 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1d680b1160 0x7f1d681448b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:59.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.658+0000 7f1d6f3de700 1 -- 172.21.15.17:0/664568445 >> 172.21.15.17:0/664568445 conn(0x7f1d6805f840 msgr2=0x7f1d6805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:59.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.658+0000 7f1d6f3de700 1 -- 172.21.15.17:0/664568445 shutdown_connections 2022-08-01T19:41:59.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.658+0000 7f1d6f3de700 1 -- 172.21.15.17:0/664568445 wait complete. 2022-08-01T19:41:59.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.662+0000 7f1d6f3de700 1 Processor -- start 2022-08-01T19:41:59.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.662+0000 7f1d6f3de700 1 -- start start 2022-08-01T19:41:59.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.662+0000 7f1d6f3de700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1d680b1160 0x7f1d680aaea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:59.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.662+0000 7f1d6f3de700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d680b1f40 0x7f1d6814b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:59.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.662+0000 7f1d6f3de700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d68148530 0x7f1d6814a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:59.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.662+0000 7f1d6f3de700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d681547f0 con 0x7f1d680b1f40 2022-08-01T19:41:59.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.662+0000 7f1d6f3de700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d680aa4b0 con 0x7f1d68148530 2022-08-01T19:41:59.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.662+0000 7f1d6f3de700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d680aa6e0 con 0x7f1d680b1160 2022-08-01T19:41:59.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.662+0000 7f1d6c979700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d680b1f40 0x7f1d6814b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:59.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.662+0000 7f1d6c979700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d680b1f40 0x7f1d6814b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43014/0 (socket says 172.21.15.17:43014) 2022-08-01T19:41:59.692 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.662+0000 7f1d6c979700 1 -- 172.21.15.17:0/3588010079 learned_addr learned my addr 172.21.15.17:0/3588010079 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:41:59.692 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.662+0000 7f1d6d97b700 1 --2- 172.21.15.17:0/3588010079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d68148530 0x7f1d6814a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:59.692 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.662+0000 7f1d6d17a700 1 --2- 172.21.15.17:0/3588010079 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1d680b1160 0x7f1d680aaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:59.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.662+0000 7f1d6c979700 1 -- 172.21.15.17:0/3588010079 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1d680b1160 msgr2=0x7f1d680aaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:59.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.662+0000 7f1d6c979700 1 --2- 172.21.15.17:0/3588010079 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1d680b1160 0x7f1d680aaea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:59.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.662+0000 7f1d6c979700 1 -- 172.21.15.17:0/3588010079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d68148530 msgr2=0x7f1d6814a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:59.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.662+0000 7f1d6c979700 1 --2- 172.21.15.17:0/3588010079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d68148530 0x7f1d6814a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:59.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.662+0000 7f1d6c979700 1 -- 172.21.15.17:0/3588010079 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d60009470 con 0x7f1d680b1f40 2022-08-01T19:41:59.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.662+0000 7f1d6d17a700 1 --2- 172.21.15.17:0/3588010079 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1d680b1160 0x7f1d680aaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:41:59.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.662+0000 7f1d6c979700 1 --2- 172.21.15.17:0/3588010079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d680b1f40 0x7f1d6814b100 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 rx=0x7f1d68148cc0 tx=0x7f1d64007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:59.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.662+0000 7f1d5e7fc700 1 -- 172.21.15.17:0/3588010079 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d64012400 con 0x7f1d680b1f40 2022-08-01T19:41:59.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.662+0000 7f1d6f3de700 1 -- 172.21.15.17:0/3588010079 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d68148da0 con 0x7f1d680b1f40 2022-08-01T19:41:59.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.662+0000 7f1d5e7fc700 1 -- 172.21.15.17:0/3588010079 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d64012560 con 0x7f1d680b1f40 2022-08-01T19:41:59.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.662+0000 7f1d5e7fc700 1 -- 172.21.15.17:0/3588010079 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d640128c0 con 0x7f1d680b1f40 2022-08-01T19:41:59.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.662+0000 7f1d6f3de700 1 -- 172.21.15.17:0/3588010079 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d680aa040 con 0x7f1d680b1f40 2022-08-01T19:41:59.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.666+0000 7f1d6f3de700 1 -- 172.21.15.17:0/3588010079 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d680574e0 con 0x7f1d680b1f40 2022-08-01T19:41:59.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.666+0000 7f1d5e7fc700 1 -- 172.21.15.17:0/3588010079 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1d64019030 con 0x7f1d680b1f40 2022-08-01T19:41:59.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.666+0000 7f1d5e7fc700 1 --2- 172.21.15.17:0/3588010079 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1d50053a70 0x7f1d50055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:41:59.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.666+0000 7f1d5e7fc700 1 -- 172.21.15.17:0/3588010079 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f1d6406b4c0 con 0x7f1d680b1f40 2022-08-01T19:41:59.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.666+0000 7f1d6d17a700 1 --2- 172.21.15.17:0/3588010079 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1d50053a70 0x7f1d50055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:41:59.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.666+0000 7f1d6d17a700 1 --2- 172.21.15.17:0/3588010079 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1d50053a70 0x7f1d50055f30 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f1d5800b000 tx=0x7f1d58009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:41:59.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.674+0000 7f1d5e7fc700 1 -- 172.21.15.17:0/3588010079 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1d64042190 con 0x7f1d680b1f40 2022-08-01T19:41:59.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.838+0000 7f1d6f3de700 1 -- 172.21.15.17:0/3588010079 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d680447e0 con 0x7f1d680b1f40 2022-08-01T19:41:59.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.838+0000 7f1d5e7fc700 1 -- 172.21.15.17:0/3588010079 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+2130 (secure 0 0 0) 0x7f1d640457b0 con 0x7f1d680b1f40 2022-08-01T19:41:59.848 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:41:59.849 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":54,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7252,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/1455566688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1455566688},{"type":"v1","addr":"172.21.15.77:6835","nonce":1455566688}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":51},{"gid":7266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.17:6833/249859043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":249859043},{"type":"v1","addr":"172.21.15.17:6833","nonce":249859043}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":51}],"filesystems":[{"mdsmap":{"epoch":54,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:41:47.422244+0000","modified":"2022-08-01T19:41:59.495072+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2022-08-01T19:41:59.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.838+0000 7f1d6f3de700 1 -- 172.21.15.17:0/3588010079 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1d50053a70 msgr2=0x7f1d50055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:59.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.838+0000 7f1d6f3de700 1 --2- 172.21.15.17:0/3588010079 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1d50053a70 0x7f1d50055f30 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f1d5800b000 tx=0x7f1d58009040).stop 2022-08-01T19:41:59.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.838+0000 7f1d6f3de700 1 -- 172.21.15.17:0/3588010079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d680b1f40 msgr2=0x7f1d6814b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:41:59.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.838+0000 7f1d6f3de700 1 --2- 172.21.15.17:0/3588010079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d680b1f40 0x7f1d6814b100 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 rx=0x7f1d68148cc0 tx=0x7f1d64007ad0).stop 2022-08-01T19:41:59.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.838+0000 7f1d6f3de700 1 -- 172.21.15.17:0/3588010079 shutdown_connections 2022-08-01T19:41:59.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.838+0000 7f1d6f3de700 1 --2- 172.21.15.17:0/3588010079 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1d50053a70 0x7f1d50055f30 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:59.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.838+0000 7f1d6f3de700 1 --2- 172.21.15.17:0/3588010079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d68148530 0x7f1d6814a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:59.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.838+0000 7f1d6f3de700 1 --2- 172.21.15.17:0/3588010079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d680b1f40 0x7f1d6814b100 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:59.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.838+0000 7f1d6f3de700 1 --2- 172.21.15.17:0/3588010079 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1d680b1160 0x7f1d680aaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:41:59.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.838+0000 7f1d6f3de700 1 -- 172.21.15.17:0/3588010079 >> 172.21.15.17:0/3588010079 conn(0x7f1d6805f840 msgr2=0x7f1d680b4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:41:59.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.838+0000 7f1d6f3de700 1 -- 172.21.15.17:0/3588010079 shutdown_connections 2022-08-01T19:41:59.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.838+0000 7f1d6f3de700 1 -- 172.21.15.17:0/3588010079 wait complete. 2022-08-01T19:41:59.856 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 54 2022-08-01T19:41:59.858 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-08-01T19:42:00.018 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.966+0000 7f040de6d700 1 Processor -- start 2022-08-01T19:42:00.018 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.966+0000 7f040de6d700 1 -- start start 2022-08-01T19:42:00.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.966+0000 7f040de6d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f04080a9da0 0x7f04080b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:00.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.966+0000 7f040de6d700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0408143ab0 con 0x7f04080b38a0 2022-08-01T19:42:00.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.966+0000 7f040de6d700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0408143d50 con 0x7f04080b0740 2022-08-01T19:42:00.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.966+0000 7f040de6d700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0408143ff0 con 0x7f04080a9da0 2022-08-01T19:42:00.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.966+0000 7f0407fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f04080a9da0 0x7f04080b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:00.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.966+0000 7f0406ffd700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f04080b38a0 0x7f04080b1240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35214/0 (socket says 172.21.15.17:35214) 2022-08-01T19:42:00.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.966+0000 7f0406ffd700 1 -- 172.21.15.17:0/2918764297 learned_addr learned my addr 172.21.15.17:0/2918764297 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:00.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.966+0000 7f04067fc700 1 -- 172.21.15.17:0/2918764297 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3861797866 0 0) 0x7f0408143ab0 con 0x7f04080b38a0 2022-08-01T19:42:00.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.966+0000 7f04067fc700 1 -- 172.21.15.17:0/2918764297 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03e8003260 con 0x7f04080b38a0 2022-08-01T19:42:00.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.966+0000 7f04067fc700 1 -- 172.21.15.17:0/2918764297 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2622401590 0 0) 0x7f03e8003260 con 0x7f04080b38a0 2022-08-01T19:42:00.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.966+0000 7f04067fc700 1 -- 172.21.15.17:0/2918764297 >> v2:172.21.15.17:3300/0 conn(0x7f04080a9da0 msgr2=0x7f04080b0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:00.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.966+0000 7f04067fc700 1 --2- 172.21.15.17:0/2918764297 >> v2:172.21.15.17:3300/0 conn(0x7f04080a9da0 0x7f04080b0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:00.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.966+0000 7f04067fc700 1 -- 172.21.15.17:0/2918764297 >> v1:172.21.15.77:6789/0 conn(0x7f04080b0740 legacy=0x7f040813d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:00.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.966+0000 7f04067fc700 1 -- 172.21.15.17:0/2918764297 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0408144290 con 0x7f04080b38a0 2022-08-01T19:42:00.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.966+0000 7f04067fc700 1 -- 172.21.15.17:0/2918764297 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7f03f8001660 con 0x7f04080b38a0 2022-08-01T19:42:00.027 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.966+0000 7f04067fc700 1 -- 172.21.15.17:0/2918764297 >> v1:172.21.15.17:6789/0 conn(0x7f04080b38a0 legacy=0x7f04080b1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:00.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.966+0000 7f04067fc700 1 --2- 172.21.15.17:0/2918764297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03e8003750 0x7f03e8003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:00.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.966+0000 7f04067fc700 1 --2- 172.21.15.17:0/2918764297 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f03e8004500 0x7f03e8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:00.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.966+0000 7f04067fc700 1 --2- 172.21.15.17:0/2918764297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03e8004e70 0x7f03e800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:00.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.966+0000 7f04067fc700 1 -- 172.21.15.17:0/2918764297 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0408143ff0 con 0x7f03e8003750 2022-08-01T19:42:00.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.966+0000 7f04067fc700 1 -- 172.21.15.17:0/2918764297 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e800bdd0 con 0x7f03e8004e70 2022-08-01T19:42:00.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.966+0000 7f04067fc700 1 -- 172.21.15.17:0/2918764297 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e800bf60 con 0x7f03e8004500 2022-08-01T19:42:00.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.966+0000 7f0407fff700 1 --2- 172.21.15.17:0/2918764297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03e8004e70 0x7f03e800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:00.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.966+0000 7f0406ffd700 1 --2- 172.21.15.17:0/2918764297 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f03e8004500 0x7f03e8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:00.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.966+0000 7f04077fe700 1 --2- 172.21.15.17:0/2918764297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03e8003750 0x7f03e8003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:00.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.970+0000 7f0406ffd700 1 -- 172.21.15.17:0/2918764297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03e8004e70 msgr2=0x7f03e800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:00.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.970+0000 7f0406ffd700 1 --2- 172.21.15.17:0/2918764297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03e8004e70 0x7f03e800b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:00.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.970+0000 7f0406ffd700 1 -- 172.21.15.17:0/2918764297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03e8003750 msgr2=0x7f03e8003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:00.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.970+0000 7f0406ffd700 1 --2- 172.21.15.17:0/2918764297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03e8003750 0x7f03e8003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:00.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.970+0000 7f0406ffd700 1 -- 172.21.15.17:0/2918764297 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e8003260 con 0x7f03e8004500 2022-08-01T19:42:00.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.970+0000 7f0407fff700 1 --2- 172.21.15.17:0/2918764297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03e8004e70 0x7f03e800b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:42:00.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.970+0000 7f04077fe700 1 --2- 172.21.15.17:0/2918764297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03e8003750 0x7f03e8003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:42:00.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.970+0000 7f0407fff700 1 -- 172.21.15.17:0/2918764297 reap_dead start 2022-08-01T19:42:00.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.970+0000 7f0406ffd700 1 --2- 172.21.15.17:0/2918764297 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f03e8004500 0x7f03e8004980 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f03f8001370 tx=0x7f0408143ee0).ready entity=mon.2 client_cookie=3b98b858607b5383 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:00.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.970+0000 7f04067fc700 1 -- 172.21.15.17:0/2918764297 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03f80070d0 con 0x7f03e8004500 2022-08-01T19:42:00.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.970+0000 7f04067fc700 1 -- 172.21.15.17:0/2918764297 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f8007230 con 0x7f03e8004500 2022-08-01T19:42:00.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.970+0000 7f04067fc700 1 -- 172.21.15.17:0/2918764297 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03f800c8c0 con 0x7f03e8004500 2022-08-01T19:42:00.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.970+0000 7f040de6d700 1 -- 172.21.15.17:0/2918764297 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f03e8004500 msgr2=0x7f03e8004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:00.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.970+0000 7f040de6d700 1 --2- 172.21.15.17:0/2918764297 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f03e8004500 0x7f03e8004980 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f03f8001370 tx=0x7f0408143ee0).stop 2022-08-01T19:42:00.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.970+0000 7f040de6d700 1 -- 172.21.15.17:0/2918764297 shutdown_connections 2022-08-01T19:42:00.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.970+0000 7f040de6d700 1 --2- 172.21.15.17:0/2918764297 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f03e8004500 0x7f03e8004980 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:00.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.970+0000 7f040de6d700 1 -- 172.21.15.17:0/2918764297 >> 172.21.15.17:0/2918764297 conn(0x7f0408060950 msgr2=0x7f0408060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:00.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.970+0000 7f040de6d700 1 -- 172.21.15.17:0/2918764297 shutdown_connections 2022-08-01T19:42:00.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.970+0000 7f040de6d700 1 -- 172.21.15.17:0/2918764297 wait complete. 2022-08-01T19:42:00.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.970+0000 7f040de6d700 1 Processor -- start 2022-08-01T19:42:00.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.970+0000 7f040de6d700 1 -- start start 2022-08-01T19:42:00.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.974+0000 7f040de6d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03e8004500 0x7f04080b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:00.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.974+0000 7f040de6d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0408156790 0x7f0408152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:00.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.974+0000 7f040de6d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0408153150 0x7f0408066af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:00.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.974+0000 7f040de6d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0408156c10 con 0x7f0408153150 2022-08-01T19:42:00.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.974+0000 7f040de6d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0408139a60 con 0x7f03e8004500 2022-08-01T19:42:00.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.974+0000 7f040de6d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0408152800 con 0x7f0408156790 2022-08-01T19:42:00.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.974+0000 7f0407fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0408153150 0x7f0408066af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:00.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.974+0000 7f04077fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03e8004500 0x7f04080b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:00.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.974+0000 7f0407fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0408153150 0x7f0408066af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43036/0 (socket says 172.21.15.17:43036) 2022-08-01T19:42:00.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.974+0000 7f0407fff700 1 -- 172.21.15.17:0/2101915750 learned_addr learned my addr 172.21.15.17:0/2101915750 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:00.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.974+0000 7f0406ffd700 1 --2- 172.21.15.17:0/2101915750 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0408156790 0x7f0408152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:00.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.974+0000 7f04077fe700 1 -- 172.21.15.17:0/2101915750 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0408156790 msgr2=0x7f0408152c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:00.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.974+0000 7f04077fe700 1 --2- 172.21.15.17:0/2101915750 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0408156790 0x7f0408152c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:00.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.974+0000 7f04077fe700 1 -- 172.21.15.17:0/2101915750 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0408153150 msgr2=0x7f0408066af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:00.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.974+0000 7f04077fe700 1 --2- 172.21.15.17:0/2101915750 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0408153150 0x7f0408066af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:00.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.974+0000 7f04077fe700 1 -- 172.21.15.17:0/2101915750 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f040813be80 con 0x7f03e8004500 2022-08-01T19:42:00.063 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.974+0000 7f0406ffd700 1 --2- 172.21.15.17:0/2101915750 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0408156790 0x7f0408152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:42:00.063 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.974+0000 7f0407fff700 1 --2- 172.21.15.17:0/2101915750 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0408153150 0x7f0408066af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:42:00.064 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.974+0000 7f04077fe700 1 --2- 172.21.15.17:0/2101915750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03e8004500 0x7f04080b1090 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f03f000d000 tx=0x7f03f0009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:00.064 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.974+0000 7f0404ff9700 1 -- 172.21.15.17:0/2101915750 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03f00116b0 con 0x7f03e8004500 2022-08-01T19:42:00.064 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.974+0000 7f0404ff9700 1 -- 172.21.15.17:0/2101915750 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f0011810 con 0x7f03e8004500 2022-08-01T19:42:00.065 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.974+0000 7f040de6d700 1 -- 172.21.15.17:0/2101915750 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f040813c120 con 0x7f03e8004500 2022-08-01T19:42:00.065 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.974+0000 7f0404ff9700 1 -- 172.21.15.17:0/2101915750 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03f0011b70 con 0x7f03e8004500 2022-08-01T19:42:00.065 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.974+0000 7f040de6d700 1 -- 172.21.15.17:0/2101915750 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04080aed20 con 0x7f03e8004500 2022-08-01T19:42:00.065 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.978+0000 7f040de6d700 1 -- 172.21.15.17:0/2101915750 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0408153770 con 0x7f03e8004500 2022-08-01T19:42:00.066 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.978+0000 7f0404ff9700 1 -- 172.21.15.17:0/2101915750 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f03f001a030 con 0x7f03e8004500 2022-08-01T19:42:00.066 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.978+0000 7f0404ff9700 1 --2- 172.21.15.17:0/2101915750 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f03ec053b30 0x7f03ec055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:00.066 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.978+0000 7f0404ff9700 1 -- 172.21.15.17:0/2101915750 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f03f006ba60 con 0x7f03e8004500 2022-08-01T19:42:00.066 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.978+0000 7f0406ffd700 1 --2- 172.21.15.17:0/2101915750 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f03ec053b30 0x7f03ec055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:00.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.978+0000 7f0406ffd700 1 --2- 172.21.15.17:0/2101915750 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f03ec053b30 0x7f03ec055ff0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f03f8019000 tx=0x7f03f8013000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:00.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:41:59.986+0000 7f0404ff9700 1 -- 172.21.15.17:0/2101915750 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0408153770 con 0x7f03e8004500 2022-08-01T19:42:00.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.150+0000 7f040de6d700 1 -- 172.21.15.17:0/2101915750 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0408153770 con 0x7f03e8004500 2022-08-01T19:42:00.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.558+0000 7f0404ff9700 1 -- 172.21.15.17:0/2101915750 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v55) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0408153770 con 0x7f03e8004500 2022-08-01T19:42:00.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.558+0000 7f040de6d700 1 -- 172.21.15.17:0/2101915750 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f03ec053b30 msgr2=0x7f03ec055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:00.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.558+0000 7f040de6d700 1 --2- 172.21.15.17:0/2101915750 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f03ec053b30 0x7f03ec055ff0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f03f8019000 tx=0x7f03f8013000).stop 2022-08-01T19:42:00.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.558+0000 7f040de6d700 1 -- 172.21.15.17:0/2101915750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03e8004500 msgr2=0x7f04080b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:00.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.558+0000 7f040de6d700 1 --2- 172.21.15.17:0/2101915750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03e8004500 0x7f04080b1090 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f03f000d000 tx=0x7f03f0009040).stop 2022-08-01T19:42:00.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.558+0000 7f040de6d700 1 -- 172.21.15.17:0/2101915750 shutdown_connections 2022-08-01T19:42:00.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.558+0000 7f040de6d700 1 --2- 172.21.15.17:0/2101915750 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f03ec053b30 0x7f03ec055ff0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:00.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.562+0000 7f040de6d700 1 --2- 172.21.15.17:0/2101915750 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0408153150 0x7f0408066af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:00.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.562+0000 7f040de6d700 1 --2- 172.21.15.17:0/2101915750 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0408156790 0x7f0408152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:00.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.562+0000 7f040de6d700 1 --2- 172.21.15.17:0/2101915750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03e8004500 0x7f04080b1090 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:00.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.562+0000 7f040de6d700 1 -- 172.21.15.17:0/2101915750 >> 172.21.15.17:0/2101915750 conn(0x7f0408060950 msgr2=0x7f040813da40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:00.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.562+0000 7f040de6d700 1 -- 172.21.15.17:0/2101915750 shutdown_connections 2022-08-01T19:42:00.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.562+0000 7f040de6d700 1 -- 172.21.15.17:0/2101915750 wait complete. 2022-08-01T19:42:00.582 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-08-01T19:42:00.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.706+0000 7fac2544e700 1 Processor -- start 2022-08-01T19:42:00.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.706+0000 7fac2544e700 1 -- start start 2022-08-01T19:42:00.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.706+0000 7fac2544e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fac200b6710 0x7fac200b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:00.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.706+0000 7fac2544e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac200b6af0 0x7fac2012bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:00.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.706+0000 7fac2544e700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac20144240 con 0x7fac200b4550 2022-08-01T19:42:00.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.706+0000 7fac2544e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac20143d00 con 0x7fac200b6710 2022-08-01T19:42:00.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.706+0000 7fac2544e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac20143fa0 con 0x7fac200b6af0 2022-08-01T19:42:00.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.706+0000 7fac1effd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fac200b6710 0x7fac200b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:00.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.706+0000 7fac1effd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fac200b6710 0x7fac200b5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43042/0 (socket says 172.21.15.17:43042) 2022-08-01T19:42:00.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.706+0000 7fac1e7fc700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fac200b4550 0x7fac20061ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35236/0 (socket says 172.21.15.17:35236) 2022-08-01T19:42:00.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.706+0000 7fac1effd700 1 -- 172.21.15.17:0/1117240948 learned_addr learned my addr 172.21.15.17:0/1117240948 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:00.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.706+0000 7fac1f7fe700 1 --2- 172.21.15.17:0/1117240948 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac200b6af0 0x7fac2012bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:00.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.710+0000 7fac1dffb700 1 -- 172.21.15.17:0/1117240948 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2363664043 0 0) 0x7fac20144240 con 0x7fac200b4550 2022-08-01T19:42:00.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.710+0000 7fac1dffb700 1 -- 172.21.15.17:0/1117240948 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac00003260 con 0x7fac200b4550 2022-08-01T19:42:00.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.710+0000 7fac1effd700 1 -- 172.21.15.17:0/1117240948 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac200b6af0 msgr2=0x7fac2012bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:00.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.710+0000 7fac1effd700 1 --2- 172.21.15.17:0/1117240948 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac200b6af0 0x7fac2012bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:00.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.710+0000 7fac1effd700 1 -- 172.21.15.17:0/1117240948 >> v1:172.21.15.17:6789/0 conn(0x7fac200b4550 legacy=0x7fac20061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:00.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.710+0000 7fac1effd700 1 -- 172.21.15.17:0/1117240948 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac20061780 con 0x7fac200b6710 2022-08-01T19:42:00.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.710+0000 7fac1effd700 1 --2- 172.21.15.17:0/1117240948 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac200b6710 0x7fac200b5d80 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 rx=0x7fac201432b0 tx=0x7fac08006eb0).ready entity=mon.0 client_cookie=4ad29c8a66fd72e2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:00.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.710+0000 7fac1dffb700 1 -- 172.21.15.17:0/1117240948 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac0800d5c0 con 0x7fac200b6710 2022-08-01T19:42:00.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.710+0000 7fac1dffb700 1 -- 172.21.15.17:0/1117240948 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac08000e90 con 0x7fac200b6710 2022-08-01T19:42:00.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.710+0000 7fac1dffb700 1 -- 172.21.15.17:0/1117240948 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac0800d950 con 0x7fac200b6710 2022-08-01T19:42:00.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.710+0000 7fac2544e700 1 -- 172.21.15.17:0/1117240948 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac200b6710 msgr2=0x7fac200b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:00.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.710+0000 7fac2544e700 1 --2- 172.21.15.17:0/1117240948 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac200b6710 0x7fac200b5d80 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 rx=0x7fac201432b0 tx=0x7fac08006eb0).stop 2022-08-01T19:42:00.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.710+0000 7fac2544e700 1 -- 172.21.15.17:0/1117240948 shutdown_connections 2022-08-01T19:42:00.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.710+0000 7fac2544e700 1 --2- 172.21.15.17:0/1117240948 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac200b6af0 0x7fac2012bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:00.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.710+0000 7fac2544e700 1 --2- 172.21.15.17:0/1117240948 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac200b6710 0x7fac200b5d80 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:00.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.710+0000 7fac2544e700 1 -- 172.21.15.17:0/1117240948 >> 172.21.15.17:0/1117240948 conn(0x7fac2005c620 msgr2=0x7fac2005ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:00.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.710+0000 7fac2544e700 1 -- 172.21.15.17:0/1117240948 shutdown_connections 2022-08-01T19:42:00.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.710+0000 7fac2544e700 1 -- 172.21.15.17:0/1117240948 wait complete. 2022-08-01T19:42:00.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.710+0000 7fac2544e700 1 Processor -- start 2022-08-01T19:42:00.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.710+0000 7fac2544e700 1 -- start start 2022-08-01T19:42:00.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.714+0000 7fac2544e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac200b4550 0x7fac2012af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:00.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.714+0000 7fac2544e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac200b6710 0x7fac200b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:00.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.714+0000 7fac2544e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac200b6af0 0x7fac20144d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:00.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.714+0000 7fac2544e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac200b3220 con 0x7fac200b6af0 2022-08-01T19:42:00.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.714+0000 7fac2544e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac201481b0 con 0x7fac200b6710 2022-08-01T19:42:00.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.714+0000 7fac2544e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac20148450 con 0x7fac200b4550 2022-08-01T19:42:00.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.714+0000 7fac1f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac200b6af0 0x7fac20144d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:00.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.714+0000 7fac1f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac200b6af0 0x7fac20144d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43052/0 (socket says 172.21.15.17:43052) 2022-08-01T19:42:00.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.714+0000 7fac1f7fe700 1 -- 172.21.15.17:0/3661454795 learned_addr learned my addr 172.21.15.17:0/3661454795 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:00.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.714+0000 7fac1effd700 1 --2- 172.21.15.17:0/3661454795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac200b4550 0x7fac2012af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:00.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.714+0000 7fac1e7fc700 1 --2- 172.21.15.17:0/3661454795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac200b6710 0x7fac200b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:00.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.714+0000 7fac1f7fe700 1 -- 172.21.15.17:0/3661454795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac200b4550 msgr2=0x7fac2012af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:00.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.714+0000 7fac1f7fe700 1 --2- 172.21.15.17:0/3661454795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac200b4550 0x7fac2012af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:00.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.714+0000 7fac1f7fe700 1 -- 172.21.15.17:0/3661454795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac200b6710 msgr2=0x7fac200b2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:00.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.714+0000 7fac1f7fe700 1 --2- 172.21.15.17:0/3661454795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac200b6710 0x7fac200b2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:00.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.714+0000 7fac1f7fe700 1 -- 172.21.15.17:0/3661454795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac0800b1e0 con 0x7fac200b6af0 2022-08-01T19:42:00.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.714+0000 7fac1e7fc700 1 --2- 172.21.15.17:0/3661454795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac200b6710 0x7fac200b2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:42:00.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.714+0000 7fac1f7fe700 1 --2- 172.21.15.17:0/3661454795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac200b6af0 0x7fac20144d30 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 rx=0x7fac18006e00 tx=0x7fac180091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:00.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.714+0000 7fac17fff700 1 -- 172.21.15.17:0/3661454795 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac1800eeb0 con 0x7fac200b6af0 2022-08-01T19:42:00.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.714+0000 7fac17fff700 1 -- 172.21.15.17:0/3661454795 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1800c850 con 0x7fac200b6af0 2022-08-01T19:42:00.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.714+0000 7fac2544e700 1 -- 172.21.15.17:0/3661454795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac200b2890 con 0x7fac200b6af0 2022-08-01T19:42:00.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.714+0000 7fac17fff700 1 -- 172.21.15.17:0/3661454795 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac1800cbb0 con 0x7fac200b6af0 2022-08-01T19:42:00.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.714+0000 7fac2544e700 1 -- 172.21.15.17:0/3661454795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac20147de0 con 0x7fac200b6af0 2022-08-01T19:42:00.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.714+0000 7fac17fff700 1 -- 172.21.15.17:0/3661454795 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fac18022070 con 0x7fac200b6af0 2022-08-01T19:42:00.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.718+0000 7fac2544e700 1 -- 172.21.15.17:0/3661454795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac20148650 con 0x7fac200b6af0 2022-08-01T19:42:00.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.718+0000 7fac17fff700 1 --2- 172.21.15.17:0/3661454795 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fac04053b30 0x7fac04055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:00.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.718+0000 7fac17fff700 1 -- 172.21.15.17:0/3661454795 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fac1806c800 con 0x7fac200b6af0 2022-08-01T19:42:00.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.718+0000 7fac1effd700 1 --2- 172.21.15.17:0/3661454795 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fac04053b30 0x7fac04055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:00.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.718+0000 7fac1effd700 1 --2- 172.21.15.17:0/3661454795 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fac04053b30 0x7fac04055ff0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7fac0801c000 tx=0x7fac08015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:00.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.726+0000 7fac17fff700 1 -- 172.21.15.17:0/3661454795 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fac18046f00 con 0x7fac200b6af0 2022-08-01T19:42:00.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:00.854+0000 7fac2544e700 1 -- 172.21.15.17:0/3661454795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fac200b2a20 con 0x7fac200b6af0 2022-08-01T19:42:01.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.566+0000 7fac17fff700 1 -- 172.21.15.17:0/3661454795 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v109) v1 ==== 185+0+0 (secure 0 0 0) 0x7fac18046af0 con 0x7fac200b6af0 2022-08-01T19:42:01.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.574+0000 7fac2544e700 1 -- 172.21.15.17:0/3661454795 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fac04053b30 msgr2=0x7fac04055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:01.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.574+0000 7fac2544e700 1 --2- 172.21.15.17:0/3661454795 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fac04053b30 0x7fac04055ff0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7fac0801c000 tx=0x7fac08015000).stop 2022-08-01T19:42:01.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.574+0000 7fac2544e700 1 -- 172.21.15.17:0/3661454795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac200b6af0 msgr2=0x7fac20144d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:01.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.574+0000 7fac2544e700 1 --2- 172.21.15.17:0/3661454795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac200b6af0 0x7fac20144d30 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 rx=0x7fac18006e00 tx=0x7fac180091d0).stop 2022-08-01T19:42:01.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.574+0000 7fac2544e700 1 -- 172.21.15.17:0/3661454795 shutdown_connections 2022-08-01T19:42:01.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.574+0000 7fac2544e700 1 --2- 172.21.15.17:0/3661454795 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fac04053b30 0x7fac04055ff0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:01.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.574+0000 7fac2544e700 1 --2- 172.21.15.17:0/3661454795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac200b6af0 0x7fac20144d30 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:01.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.574+0000 7fac2544e700 1 --2- 172.21.15.17:0/3661454795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac200b6710 0x7fac200b2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:01.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.574+0000 7fac2544e700 1 --2- 172.21.15.17:0/3661454795 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac200b4550 0x7fac2012af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:01.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.574+0000 7fac2544e700 1 -- 172.21.15.17:0/3661454795 >> 172.21.15.17:0/3661454795 conn(0x7fac2005c620 msgr2=0x7fac201423b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:01.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.574+0000 7fac2544e700 1 -- 172.21.15.17:0/3661454795 shutdown_connections 2022-08-01T19:42:01.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.574+0000 7fac2544e700 1 -- 172.21.15.17:0/3661454795 wait complete. 2022-08-01T19:42:01.599 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' removed 2022-08-01T19:42:01.600 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-08-01T19:42:01.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.714+0000 7f2c2464d700 1 Processor -- start 2022-08-01T19:42:01.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.714+0000 7f2c2464d700 1 -- start start 2022-08-01T19:42:01.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.718+0000 7f2c2464d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2c1c0b6710 0x7f2c1c0b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:01.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.718+0000 7f2c2464d700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f2c1c0b4550 0x7f2c1c0b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:01.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.718+0000 7f2c2464d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2c1c061ec0 0x7f2c1c062330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:01.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.718+0000 7f2c2464d700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c1c144240 con 0x7f2c1c061ec0 2022-08-01T19:42:01.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.718+0000 7f2c2464d700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c1c0b6fe0 con 0x7f2c1c0b4550 2022-08-01T19:42:01.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.718+0000 7f2c2464d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1c143d00 con 0x7f2c1c0b6710 2022-08-01T19:42:01.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.718+0000 7f2c22bea700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2c1c061ec0 0x7f2c1c062330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:01.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.718+0000 7f2c22bea700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2c1c061ec0 0x7f2c1c062330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43060/0 (socket says 172.21.15.17:43060) 2022-08-01T19:42:01.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.718+0000 7f2c223e9700 1 --2- 172.21.15.17:0/263137168 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2c1c0b6710 0x7f2c1c0b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:01.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.718+0000 7f2c21be8700 1 --2- 172.21.15.17:0/263137168 >> v2:172.21.15.77:3300/0 conn(0x7f2c1c0b4550 0x7f2c1c0b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:01.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.718+0000 7f2c22bea700 1 -- 172.21.15.17:0/263137168 learned_addr learned my addr 172.21.15.17:0/263137168 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:01.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.718+0000 7f2c22bea700 1 -- 172.21.15.17:0/263137168 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2c1c0b6710 msgr2=0x7f2c1c0b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:01.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.718+0000 7f2c22bea700 1 --2- 172.21.15.17:0/263137168 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2c1c0b6710 0x7f2c1c0b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:01.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.718+0000 7f2c22bea700 1 -- 172.21.15.17:0/263137168 >> v2:172.21.15.77:3300/0 conn(0x7f2c1c0b4550 msgr2=0x7f2c1c0b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:01.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.718+0000 7f2c22bea700 1 --2- 172.21.15.17:0/263137168 >> v2:172.21.15.77:3300/0 conn(0x7f2c1c0b4550 0x7f2c1c0b6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:01.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.718+0000 7f2c22bea700 1 -- 172.21.15.17:0/263137168 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c1c143fa0 con 0x7f2c1c061ec0 2022-08-01T19:42:01.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.718+0000 7f2c22bea700 1 --2- 172.21.15.17:0/263137168 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c1c061ec0 0x7f2c1c062330 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 rx=0x7f2c140032f0 tx=0x7f2c14009290).ready entity=mon.0 client_cookie=a0ac23c588131997 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:01.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.718+0000 7f2c213e7700 1 -- 172.21.15.17:0/263137168 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c1400b680 con 0x7f2c1c061ec0 2022-08-01T19:42:01.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.718+0000 7f2c213e7700 1 -- 172.21.15.17:0/263137168 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c1400b7e0 con 0x7f2c1c061ec0 2022-08-01T19:42:01.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.718+0000 7f2c213e7700 1 -- 172.21.15.17:0/263137168 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c1400b950 con 0x7f2c1c061ec0 2022-08-01T19:42:01.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.718+0000 7f2c2464d700 1 -- 172.21.15.17:0/263137168 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c1c061ec0 msgr2=0x7f2c1c062330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:01.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.718+0000 7f2c2464d700 1 --2- 172.21.15.17:0/263137168 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c1c061ec0 0x7f2c1c062330 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 rx=0x7f2c140032f0 tx=0x7f2c14009290).stop 2022-08-01T19:42:01.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.718+0000 7f2c2464d700 1 -- 172.21.15.17:0/263137168 shutdown_connections 2022-08-01T19:42:01.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.718+0000 7f2c2464d700 1 --2- 172.21.15.17:0/263137168 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c1c061ec0 0x7f2c1c062330 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:01.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.718+0000 7f2c2464d700 1 --2- 172.21.15.17:0/263137168 >> v2:172.21.15.77:3300/0 conn(0x7f2c1c0b4550 0x7f2c1c0b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:01.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.718+0000 7f2c2464d700 1 --2- 172.21.15.17:0/263137168 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2c1c0b6710 0x7f2c1c0b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:01.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.718+0000 7f2c2464d700 1 -- 172.21.15.17:0/263137168 >> 172.21.15.17:0/263137168 conn(0x7f2c1c05c620 msgr2=0x7f2c1c05ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:01.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.718+0000 7f2c2464d700 1 -- 172.21.15.17:0/263137168 shutdown_connections 2022-08-01T19:42:01.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.718+0000 7f2c2464d700 1 -- 172.21.15.17:0/263137168 wait complete. 2022-08-01T19:42:01.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.722+0000 7f2c2464d700 1 Processor -- start 2022-08-01T19:42:01.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.722+0000 7f2c2464d700 1 -- start start 2022-08-01T19:42:01.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.722+0000 7f2c2464d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2c1c061ec0 0x7f2c1c127250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:01.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.722+0000 7f2c2464d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c1c0b4550 0x7f2c1c152950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:01.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.722+0000 7f2c2464d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c1c0b6710 0x7f2c1c152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:01.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.722+0000 7f2c2464d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1c146550 con 0x7f2c1c0b4550 2022-08-01T19:42:01.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.722+0000 7f2c2464d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1c153330 con 0x7f2c1c0b6710 2022-08-01T19:42:01.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.722+0000 7f2c2464d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1c148160 con 0x7f2c1c061ec0 2022-08-01T19:42:01.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.722+0000 7f2c21be8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c1c0b4550 0x7f2c1c152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:01.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.722+0000 7f2c223e9700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2c1c061ec0 0x7f2c1c127250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:01.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.722+0000 7f2c21be8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c1c0b4550 0x7f2c1c152950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43066/0 (socket says 172.21.15.17:43066) 2022-08-01T19:42:01.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.722+0000 7f2c21be8700 1 -- 172.21.15.17:0/1782630429 learned_addr learned my addr 172.21.15.17:0/1782630429 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:01.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.722+0000 7f2c22bea700 1 --2- 172.21.15.17:0/1782630429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c1c0b6710 0x7f2c1c152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:01.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.726+0000 7f2c223e9700 1 -- 172.21.15.17:0/1782630429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c1c0b6710 msgr2=0x7f2c1c152e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:01.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.726+0000 7f2c223e9700 1 --2- 172.21.15.17:0/1782630429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c1c0b6710 0x7f2c1c152e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:01.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.726+0000 7f2c223e9700 1 -- 172.21.15.17:0/1782630429 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c1c0b4550 msgr2=0x7f2c1c152950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:01.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.726+0000 7f2c223e9700 1 --2- 172.21.15.17:0/1782630429 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c1c0b4550 0x7f2c1c152950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:01.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.726+0000 7f2c223e9700 1 -- 172.21.15.17:0/1782630429 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c14009530 con 0x7f2c1c061ec0 2022-08-01T19:42:01.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.726+0000 7f2c223e9700 1 --2- 172.21.15.17:0/1782630429 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2c1c061ec0 0x7f2c1c127250 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f2c1c153470 tx=0x7f2c18006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:01.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.726+0000 7f2c0b7fe700 1 -- 172.21.15.17:0/1782630429 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c18011750 con 0x7f2c1c061ec0 2022-08-01T19:42:01.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.726+0000 7f2c0b7fe700 1 -- 172.21.15.17:0/1782630429 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c180118b0 con 0x7f2c1c061ec0 2022-08-01T19:42:01.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.726+0000 7f2c2464d700 1 -- 172.21.15.17:0/1782630429 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c1c148400 con 0x7f2c1c061ec0 2022-08-01T19:42:01.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.726+0000 7f2c0b7fe700 1 -- 172.21.15.17:0/1782630429 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c18011c10 con 0x7f2c1c061ec0 2022-08-01T19:42:01.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.726+0000 7f2c2464d700 1 -- 172.21.15.17:0/1782630429 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c1c127740 con 0x7f2c1c061ec0 2022-08-01T19:42:01.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.726+0000 7f2c2464d700 1 -- 172.21.15.17:0/1782630429 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c1c061780 con 0x7f2c1c061ec0 2022-08-01T19:42:01.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.726+0000 7f2c0b7fe700 1 -- 172.21.15.17:0/1782630429 <== mon.2 v2:172.21.15.77:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2c18011d70 con 0x7f2c1c061ec0 2022-08-01T19:42:01.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.726+0000 7f2c0b7fe700 1 --2- 172.21.15.17:0/1782630429 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2bfc053aa0 0x7f2bfc055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:01.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.726+0000 7f2c0b7fe700 1 -- 172.21.15.17:0/1782630429 <== mon.2 v2:172.21.15.77:3301/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f2c1806c470 con 0x7f2c1c061ec0 2022-08-01T19:42:01.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.726+0000 7f2c21be8700 1 --2- 172.21.15.17:0/1782630429 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2bfc053aa0 0x7f2bfc055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:01.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.730+0000 7f2c21be8700 1 --2- 172.21.15.17:0/1782630429 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2bfc053aa0 0x7f2bfc055f60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7f2c0c00b000 tx=0x7f2c0c006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:01.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.738+0000 7f2c0b7fe700 1 -- 172.21.15.17:0/1782630429 <== mon.2 v2:172.21.15.77:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2c18043200 con 0x7f2c1c061ec0 2022-08-01T19:42:01.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:01.866+0000 7f2c2464d700 1 -- 172.21.15.17:0/1782630429 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2c1c0574e0 con 0x7f2c1c061ec0 2022-08-01T19:42:02.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.578+0000 7f2c0b7fe700 1 -- 172.21.15.17:0/1782630429 <== mon.2 v2:172.21.15.77:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v110) v1 ==== 173+0+0 (secure 0 0 0) 0x7f2c18046820 con 0x7f2c1c061ec0 2022-08-01T19:42:02.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.582+0000 7f2c2464d700 1 -- 172.21.15.17:0/1782630429 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2bfc053aa0 msgr2=0x7f2bfc055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:02.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.582+0000 7f2c2464d700 1 --2- 172.21.15.17:0/1782630429 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2bfc053aa0 0x7f2bfc055f60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7f2c0c00b000 tx=0x7f2c0c006df0).stop 2022-08-01T19:42:02.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.582+0000 7f2c2464d700 1 -- 172.21.15.17:0/1782630429 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2c1c061ec0 msgr2=0x7f2c1c127250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:02.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.582+0000 7f2c2464d700 1 --2- 172.21.15.17:0/1782630429 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2c1c061ec0 0x7f2c1c127250 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f2c1c153470 tx=0x7f2c18006df0).stop 2022-08-01T19:42:02.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.582+0000 7f2c2464d700 1 -- 172.21.15.17:0/1782630429 shutdown_connections 2022-08-01T19:42:02.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.582+0000 7f2c2464d700 1 --2- 172.21.15.17:0/1782630429 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2bfc053aa0 0x7f2bfc055f60 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:02.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.582+0000 7f2c2464d700 1 --2- 172.21.15.17:0/1782630429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c1c0b6710 0x7f2c1c152e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:02.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.582+0000 7f2c2464d700 1 --2- 172.21.15.17:0/1782630429 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c1c0b4550 0x7f2c1c152950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:02.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.582+0000 7f2c2464d700 1 --2- 172.21.15.17:0/1782630429 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2c1c061ec0 0x7f2c1c127250 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:02.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.582+0000 7f2c2464d700 1 -- 172.21.15.17:0/1782630429 >> 172.21.15.17:0/1782630429 conn(0x7f2c1c05c620 msgr2=0x7f2c1c148950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:02.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.582+0000 7f2c2464d700 1 -- 172.21.15.17:0/1782630429 shutdown_connections 2022-08-01T19:42:02.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.582+0000 7f2c2464d700 1 -- 172.21.15.17:0/1782630429 wait complete. 2022-08-01T19:42:02.591 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' removed 2022-08-01T19:42:02.601 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-08-01T19:42:02.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.718+0000 7fbcd41af700 1 Processor -- start 2022-08-01T19:42:02.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.718+0000 7fbcd41af700 1 -- start start 2022-08-01T19:42:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.718+0000 7fbcd41af700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fbccc0b1ef0 0x7fbccc0b3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:02.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.718+0000 7fbcd41af700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbccc147450 0x7fbccc143740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:02.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.718+0000 7fbcd41af700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbccc0ae7e0 con 0x7fbccc0b1580 2022-08-01T19:42:02.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.718+0000 7fbcd41af700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbccc0b1960 con 0x7fbccc0b1ef0 2022-08-01T19:42:02.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.718+0000 7fbcd41af700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc0b2470 con 0x7fbccc147450 2022-08-01T19:42:02.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.718+0000 7fbcd174a700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fbccc0b1ef0 0x7fbccc0b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:02.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.718+0000 7fbcd274c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbccc147450 0x7fbccc143740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:02.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.718+0000 7fbcd174a700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fbccc0b1ef0 0x7fbccc0b3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:40994/0 (socket says 172.21.15.17:40994) 2022-08-01T19:42:02.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.718+0000 7fbcd174a700 1 -- 172.21.15.17:0/2139435155 learned_addr learned my addr 172.21.15.17:0/2139435155 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:02.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.718+0000 7fbcd1f4b700 1 --1- 172.21.15.17:0/2139435155 >> v1:172.21.15.77:6789/0 conn(0x7fbccc0b1580 0x7fbccc146d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:52064/0 (socket says 172.21.15.17:52064) 2022-08-01T19:42:02.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.718+0000 7fbcd174a700 1 -- 172.21.15.17:0/2139435155 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbccc147450 msgr2=0x7fbccc143740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:02.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.718+0000 7fbcd174a700 1 --2- 172.21.15.17:0/2139435155 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbccc147450 0x7fbccc143740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:02.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.718+0000 7fbcd174a700 1 -- 172.21.15.17:0/2139435155 >> v1:172.21.15.77:6789/0 conn(0x7fbccc0b1580 legacy=0x7fbccc146d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:02.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.718+0000 7fbcd174a700 1 -- 172.21.15.17:0/2139435155 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbccc0b2710 con 0x7fbccc0b1ef0 2022-08-01T19:42:02.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.722+0000 7fbcd174a700 1 --2- 172.21.15.17:0/2139435155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbccc0b1ef0 0x7fbccc0b3210 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7fbcc8002f70 tx=0x7fbcc8006eb0).ready entity=mon.1 client_cookie=fcdc83eb6f437a15 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:02.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.722+0000 7fbcd0f49700 1 -- 172.21.15.17:0/2139435155 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbcc800d570 con 0x7fbccc0b1ef0 2022-08-01T19:42:02.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.722+0000 7fbcd0f49700 1 -- 172.21.15.17:0/2139435155 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc8000e90 con 0x7fbccc0b1ef0 2022-08-01T19:42:02.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.722+0000 7fbcd0f49700 1 -- 172.21.15.17:0/2139435155 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbcc800d900 con 0x7fbccc0b1ef0 2022-08-01T19:42:02.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.722+0000 7fbcd41af700 1 -- 172.21.15.17:0/2139435155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbccc0b1ef0 msgr2=0x7fbccc0b3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:02.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.722+0000 7fbcd41af700 1 --2- 172.21.15.17:0/2139435155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbccc0b1ef0 0x7fbccc0b3210 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7fbcc8002f70 tx=0x7fbcc8006eb0).stop 2022-08-01T19:42:02.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.722+0000 7fbcd41af700 1 -- 172.21.15.17:0/2139435155 shutdown_connections 2022-08-01T19:42:02.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.722+0000 7fbcd41af700 1 --2- 172.21.15.17:0/2139435155 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbccc147450 0x7fbccc143740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:02.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.722+0000 7fbcd41af700 1 --2- 172.21.15.17:0/2139435155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbccc0b1ef0 0x7fbccc0b3210 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:02.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.722+0000 7fbcd41af700 1 -- 172.21.15.17:0/2139435155 >> 172.21.15.17:0/2139435155 conn(0x7fbccc060950 msgr2=0x7fbccc060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:02.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.722+0000 7fbcd41af700 1 -- 172.21.15.17:0/2139435155 shutdown_connections 2022-08-01T19:42:02.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.722+0000 7fbcd41af700 1 -- 172.21.15.17:0/2139435155 wait complete. 2022-08-01T19:42:02.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.722+0000 7fbcd41af700 1 Processor -- start 2022-08-01T19:42:02.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.722+0000 7fbcd41af700 1 -- start start 2022-08-01T19:42:02.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.726+0000 7fbcd41af700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbccc0b1580 0x7fbccc1449e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:02.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.726+0000 7fbcd41af700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbccc0b1ef0 0x7fbccc143470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:02.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.726+0000 7fbcd41af700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbccc147450 0x7fbccc0acb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:02.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.726+0000 7fbcd41af700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc140970 con 0x7fbccc0b1ef0 2022-08-01T19:42:02.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.726+0000 7fbcd41af700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc153490 con 0x7fbccc147450 2022-08-01T19:42:02.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.726+0000 7fbcd41af700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc153730 con 0x7fbccc0b1580 2022-08-01T19:42:02.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.726+0000 7fbcd174a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbccc0b1ef0 0x7fbccc143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:02.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.726+0000 7fbcd174a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbccc0b1ef0 0x7fbccc143470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43076/0 (socket says 172.21.15.17:43076) 2022-08-01T19:42:02.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.726+0000 7fbcd174a700 1 -- 172.21.15.17:0/2834503610 learned_addr learned my addr 172.21.15.17:0/2834503610 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:02.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.726+0000 7fbcd1f4b700 1 --2- 172.21.15.17:0/2834503610 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbccc0b1580 0x7fbccc1449e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:02.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.726+0000 7fbcd274c700 1 --2- 172.21.15.17:0/2834503610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbccc147450 0x7fbccc0acb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:02.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.726+0000 7fbcd174a700 1 -- 172.21.15.17:0/2834503610 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbccc0b1580 msgr2=0x7fbccc1449e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:02.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.726+0000 7fbcd174a700 1 --2- 172.21.15.17:0/2834503610 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbccc0b1580 0x7fbccc1449e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:02.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.726+0000 7fbcd174a700 1 -- 172.21.15.17:0/2834503610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbccc147450 msgr2=0x7fbccc0acb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:02.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.726+0000 7fbcd174a700 1 --2- 172.21.15.17:0/2834503610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbccc147450 0x7fbccc0acb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:02.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.726+0000 7fbcd174a700 1 -- 172.21.15.17:0/2834503610 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc800b1e0 con 0x7fbccc0b1ef0 2022-08-01T19:42:02.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.726+0000 7fbcd174a700 1 --2- 172.21.15.17:0/2834503610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbccc0b1ef0 0x7fbccc143470 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 rx=0x7fbcc8002f70 tx=0x7fbcc8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:02.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.726+0000 7fbcbaffd700 1 -- 172.21.15.17:0/2834503610 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbcc8011610 con 0x7fbccc0b1ef0 2022-08-01T19:42:02.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.726+0000 7fbcbaffd700 1 -- 172.21.15.17:0/2834503610 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc8011770 con 0x7fbccc0b1ef0 2022-08-01T19:42:02.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.726+0000 7fbcd41af700 1 -- 172.21.15.17:0/2834503610 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbccc140ba0 con 0x7fbccc0b1ef0 2022-08-01T19:42:02.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.726+0000 7fbcbaffd700 1 -- 172.21.15.17:0/2834503610 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbcc8011ad0 con 0x7fbccc0b1ef0 2022-08-01T19:42:02.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.726+0000 7fbcd41af700 1 -- 172.21.15.17:0/2834503610 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbccc143b80 con 0x7fbccc0b1ef0 2022-08-01T19:42:02.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.730+0000 7fbcd41af700 1 -- 172.21.15.17:0/2834503610 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbccc146790 con 0x7fbccc0b1ef0 2022-08-01T19:42:02.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.730+0000 7fbcbaffd700 1 -- 172.21.15.17:0/2834503610 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbcc8025030 con 0x7fbccc0b1ef0 2022-08-01T19:42:02.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.730+0000 7fbcbaffd700 1 --2- 172.21.15.17:0/2834503610 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbcac053a70 0x7fbcac055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:02.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.730+0000 7fbcbaffd700 1 -- 172.21.15.17:0/2834503610 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fbcc806cd70 con 0x7fbccc0b1ef0 2022-08-01T19:42:02.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.730+0000 7fbcd1f4b700 1 --2- 172.21.15.17:0/2834503610 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbcac053a70 0x7fbcac055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:02.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.730+0000 7fbcd1f4b700 1 --2- 172.21.15.17:0/2834503610 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbcac053a70 0x7fbcac055f30 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7fbcbc00d000 tx=0x7fbcbc007b80).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:02.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.734+0000 7fbcbaffd700 1 -- 172.21.15.17:0/2834503610 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbccc146790 con 0x7fbccc0b1ef0 2022-08-01T19:42:02.858 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.854+0000 7fbcd41af700 1 -- 172.21.15.17:0/2834503610 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fbccc146790 con 0x7fbccc0b1ef0 2022-08-01T19:42:02.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.862+0000 7fbcbaffd700 1 -- 172.21.15.17:0/2834503610 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbccc146790 con 0x7fbccc0b1ef0 2022-08-01T19:42:02.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.866+0000 7fbcd41af700 1 -- 172.21.15.17:0/2834503610 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbcac053a70 msgr2=0x7fbcac055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:02.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.866+0000 7fbcd41af700 1 --2- 172.21.15.17:0/2834503610 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbcac053a70 0x7fbcac055f30 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7fbcbc00d000 tx=0x7fbcbc007b80).stop 2022-08-01T19:42:02.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.866+0000 7fbcd41af700 1 -- 172.21.15.17:0/2834503610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbccc0b1ef0 msgr2=0x7fbccc143470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:02.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.866+0000 7fbcd41af700 1 --2- 172.21.15.17:0/2834503610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbccc0b1ef0 0x7fbccc143470 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 rx=0x7fbcc8002f70 tx=0x7fbcc8015000).stop 2022-08-01T19:42:02.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.866+0000 7fbcd41af700 1 -- 172.21.15.17:0/2834503610 shutdown_connections 2022-08-01T19:42:02.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.866+0000 7fbcd41af700 1 --2- 172.21.15.17:0/2834503610 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbcac053a70 0x7fbcac055f30 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:02.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.866+0000 7fbcd41af700 1 --2- 172.21.15.17:0/2834503610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbccc147450 0x7fbccc0acb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:02.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.866+0000 7fbcd41af700 1 --2- 172.21.15.17:0/2834503610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbccc0b1ef0 0x7fbccc143470 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:02.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.866+0000 7fbcd41af700 1 --2- 172.21.15.17:0/2834503610 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbccc0b1580 0x7fbccc1449e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:02.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.866+0000 7fbcd41af700 1 -- 172.21.15.17:0/2834503610 >> 172.21.15.17:0/2834503610 conn(0x7fbccc060950 msgr2=0x7fbccc0b6880 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:02.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.866+0000 7fbcd41af700 1 -- 172.21.15.17:0/2834503610 shutdown_connections 2022-08-01T19:42:02.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.866+0000 7fbcd41af700 1 -- 172.21.15.17:0/2834503610 wait complete. 2022-08-01T19:42:02.882 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_filtered_df' 2022-08-01T19:42:02.991 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.990+0000 7f6fa678b700 1 Processor -- start 2022-08-01T19:42:02.991 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.990+0000 7f6fa678b700 1 -- start start 2022-08-01T19:42:02.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.990+0000 7f6fa678b700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f6fa0063a60 0x7f6fa0143ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:02.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.990+0000 7f6fa678b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6fa0061ab0 0x7f6fa0140b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:02.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.990+0000 7f6fa678b700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fa0063e40 con 0x7f6fa00b1460 2022-08-01T19:42:02.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.990+0000 7f6fa678b700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fa013ff40 con 0x7f6fa0063a60 2022-08-01T19:42:02.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.990+0000 7f6fa678b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa0062180 con 0x7f6fa0061ab0 2022-08-01T19:42:02.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.990+0000 7f6fa5789700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f6fa0063a60 0x7f6fa0143ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:02.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.990+0000 7f6fa4f88700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6fa0061ab0 0x7f6fa0140b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:02.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.990+0000 7f6fa5789700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f6fa0063a60 0x7f6fa0143ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:41006/0 (socket says 172.21.15.17:41006) 2022-08-01T19:42:02.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.990+0000 7f6fa4f88700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6fa0061ab0 0x7f6fa0140b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:39186/0 (socket says 172.21.15.17:39186) 2022-08-01T19:42:02.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.990+0000 7f6fa5789700 1 -- 172.21.15.17:0/3199480928 learned_addr learned my addr 172.21.15.17:0/3199480928 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:03.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.990+0000 7f6fa5789700 1 -- 172.21.15.17:0/3199480928 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6fa0061ab0 msgr2=0x7f6fa0140b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:03.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.990+0000 7f6fa5789700 1 --2- 172.21.15.17:0/3199480928 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6fa0061ab0 0x7f6fa0140b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:03.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.990+0000 7f6fa5789700 1 -- 172.21.15.17:0/3199480928 >> v1:172.21.15.77:6789/0 conn(0x7f6fa00b1460 legacy=0x7f6fa0128e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:03.002 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.990+0000 7f6fa5789700 1 -- 172.21.15.17:0/3199480928 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa0140130 con 0x7f6fa0063a60 2022-08-01T19:42:03.002 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.990+0000 7f6fa4f88700 1 --2- 172.21.15.17:0/3199480928 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6fa0061ab0 0x7f6fa0140b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:42:03.003 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.990+0000 7f6fa5789700 1 --2- 172.21.15.17:0/3199480928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fa0063a60 0x7f6fa0143ee0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f6f9c0032f0 tx=0x7f6f9c00b530).ready entity=mon.1 client_cookie=cb8578b5215ce492 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:03.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.990+0000 7f6f93fff700 1 -- 172.21.15.17:0/3199480928 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f9c00de20 con 0x7f6fa0063a60 2022-08-01T19:42:03.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.990+0000 7f6f93fff700 1 -- 172.21.15.17:0/3199480928 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9c010470 con 0x7f6fa0063a60 2022-08-01T19:42:03.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.990+0000 7f6f93fff700 1 -- 172.21.15.17:0/3199480928 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f9c0107d0 con 0x7f6fa0063a60 2022-08-01T19:42:03.014 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.994+0000 7f6fa678b700 1 -- 172.21.15.17:0/3199480928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fa0063a60 msgr2=0x7f6fa0143ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:03.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.994+0000 7f6fa678b700 1 --2- 172.21.15.17:0/3199480928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fa0063a60 0x7f6fa0143ee0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f6f9c0032f0 tx=0x7f6f9c00b530).stop 2022-08-01T19:42:03.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.994+0000 7f6fa678b700 1 -- 172.21.15.17:0/3199480928 shutdown_connections 2022-08-01T19:42:03.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.994+0000 7f6fa678b700 1 --2- 172.21.15.17:0/3199480928 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6fa0061ab0 0x7f6fa0140b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:03.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.994+0000 7f6fa678b700 1 --2- 172.21.15.17:0/3199480928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fa0063a60 0x7f6fa0143ee0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:03.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.994+0000 7f6fa678b700 1 -- 172.21.15.17:0/3199480928 >> 172.21.15.17:0/3199480928 conn(0x7f6fa005c620 msgr2=0x7f6fa005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:03.027 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.994+0000 7f6fa678b700 1 -- 172.21.15.17:0/3199480928 shutdown_connections 2022-08-01T19:42:03.027 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.994+0000 7f6fa678b700 1 -- 172.21.15.17:0/3199480928 wait complete. 2022-08-01T19:42:03.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.994+0000 7f6fa678b700 1 Processor -- start 2022-08-01T19:42:03.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.994+0000 7f6fa678b700 1 -- start start 2022-08-01T19:42:03.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.994+0000 7f6fa678b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fa0061ab0 0x7f6fa00b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:03.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.994+0000 7f6fa678b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6fa0063a60 0x7f6fa00aee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:03.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.994+0000 7f6fa678b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fa00b1460 0x7f6fa0141340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:03.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.994+0000 7f6fa678b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa0064800 con 0x7f6fa00b1460 2022-08-01T19:42:03.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.994+0000 7f6fa678b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa00b38d0 con 0x7f6fa0061ab0 2022-08-01T19:42:03.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.994+0000 7f6fa678b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa0143030 con 0x7f6fa0063a60 2022-08-01T19:42:03.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.998+0000 7f6fa5f8a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fa00b1460 0x7f6fa0141340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:03.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.998+0000 7f6fa5f8a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fa00b1460 0x7f6fa0141340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43094/0 (socket says 172.21.15.17:43094) 2022-08-01T19:42:03.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.998+0000 7f6fa5f8a700 1 -- 172.21.15.17:0/4133201604 learned_addr learned my addr 172.21.15.17:0/4133201604 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:03.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.998+0000 7f6fa4f88700 1 --2- 172.21.15.17:0/4133201604 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6fa0063a60 0x7f6fa00aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:03.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.998+0000 7f6fa5789700 1 --2- 172.21.15.17:0/4133201604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fa0061ab0 0x7f6fa00b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:03.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.998+0000 7f6fa5f8a700 1 -- 172.21.15.17:0/4133201604 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6fa0063a60 msgr2=0x7f6fa00aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:03.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.998+0000 7f6fa5f8a700 1 --2- 172.21.15.17:0/4133201604 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6fa0063a60 0x7f6fa00aee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:03.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.998+0000 7f6fa5f8a700 1 -- 172.21.15.17:0/4133201604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fa0061ab0 msgr2=0x7f6fa00b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:03.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.998+0000 7f6fa5f8a700 1 --2- 172.21.15.17:0/4133201604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fa0061ab0 0x7f6fa00b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:03.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.998+0000 7f6fa5f8a700 1 -- 172.21.15.17:0/4133201604 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9400b180 con 0x7f6fa00b1460 2022-08-01T19:42:03.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.998+0000 7f6fa5f8a700 1 --2- 172.21.15.17:0/4133201604 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fa00b1460 0x7f6fa0141340 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 rx=0x7f6fa00af9e0 tx=0x7f6f98007ba0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:03.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.998+0000 7f6f92ffd700 1 -- 172.21.15.17:0/4133201604 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f9800cc40 con 0x7f6fa00b1460 2022-08-01T19:42:03.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.998+0000 7f6f92ffd700 1 -- 172.21.15.17:0/4133201604 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9800cda0 con 0x7f6fa00b1460 2022-08-01T19:42:03.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.998+0000 7f6fa678b700 1 -- 172.21.15.17:0/4133201604 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f9c00b7d0 con 0x7f6fa00b1460 2022-08-01T19:42:03.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.998+0000 7f6f92ffd700 1 -- 172.21.15.17:0/4133201604 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f98013640 con 0x7f6fa00b1460 2022-08-01T19:42:03.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.998+0000 7f6fa678b700 1 -- 172.21.15.17:0/4133201604 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fa00b5eb0 con 0x7f6fa00b1460 2022-08-01T19:42:03.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.998+0000 7f6f92ffd700 1 -- 172.21.15.17:0/4133201604 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6f98010070 con 0x7f6fa00b1460 2022-08-01T19:42:03.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.998+0000 7f6fa678b700 1 -- 172.21.15.17:0/4133201604 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fa00574e0 con 0x7f6fa00b1460 2022-08-01T19:42:03.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.998+0000 7f6f92ffd700 1 --2- 172.21.15.17:0/4133201604 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6f7c053a70 0x7f6f7c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:03.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:02.998+0000 7f6f92ffd700 1 -- 172.21.15.17:0/4133201604 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f6f9806b170 con 0x7f6fa00b1460 2022-08-01T19:42:03.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.002+0000 7f6fa5789700 1 --2- 172.21.15.17:0/4133201604 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6f7c053a70 0x7f6f7c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:03.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.002+0000 7f6fa5789700 1 --2- 172.21.15.17:0/4133201604 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6f7c053a70 0x7f6f7c055f30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f6f9c01b000 tx=0x7f6fa00aea60).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:03.040 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.006+0000 7f6f92ffd700 1 -- 172.21.15.17:0/4133201604 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6f98042210 con 0x7f6fa00b1460 2022-08-01T19:42:03.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.178+0000 7f6fa678b700 1 -- 172.21.15.17:0/4133201604 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]} v 0) v1 -- 0x7f6fa0067530 con 0x7f6fa00b1460 2022-08-01T19:42:03.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.582+0000 7f6f92ffd700 1 -- 172.21.15.17:0/4133201604 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]}]=0 v359) v1 ==== 127+0+0 (secure 0 0 0) 0x7f6f98045830 con 0x7f6fa00b1460 2022-08-01T19:42:03.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.582+0000 7f6fa678b700 1 -- 172.21.15.17:0/4133201604 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6f7c053a70 msgr2=0x7f6f7c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:03.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.582+0000 7f6fa678b700 1 --2- 172.21.15.17:0/4133201604 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6f7c053a70 0x7f6f7c055f30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f6f9c01b000 tx=0x7f6fa00aea60).stop 2022-08-01T19:42:03.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.582+0000 7f6fa678b700 1 -- 172.21.15.17:0/4133201604 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fa00b1460 msgr2=0x7f6fa0141340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:03.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.582+0000 7f6fa678b700 1 --2- 172.21.15.17:0/4133201604 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fa00b1460 0x7f6fa0141340 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 rx=0x7f6fa00af9e0 tx=0x7f6f98007ba0).stop 2022-08-01T19:42:03.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.586+0000 7f6fa678b700 1 -- 172.21.15.17:0/4133201604 shutdown_connections 2022-08-01T19:42:03.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.586+0000 7f6fa678b700 1 --2- 172.21.15.17:0/4133201604 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6f7c053a70 0x7f6f7c055f30 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:03.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.586+0000 7f6fa678b700 1 --2- 172.21.15.17:0/4133201604 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fa00b1460 0x7f6fa0141340 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:03.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.586+0000 7f6fa678b700 1 --2- 172.21.15.17:0/4133201604 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6fa0063a60 0x7f6fa00aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:03.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.586+0000 7f6fa678b700 1 --2- 172.21.15.17:0/4133201604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fa0061ab0 0x7f6fa00b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:03.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.586+0000 7f6fa678b700 1 -- 172.21.15.17:0/4133201604 >> 172.21.15.17:0/4133201604 conn(0x7f6fa005c620 msgr2=0x7f6fa005f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:03.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.586+0000 7f6fa678b700 1 -- 172.21.15.17:0/4133201604 shutdown_connections 2022-08-01T19:42:03.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.586+0000 7f6fa678b700 1 -- 172.21.15.17:0/4133201604 wait complete. 2022-08-01T19:42:03.607 INFO:tasks.cephfs_test_runner:test_filtered_df (tasks.cephfs.test_misc.TestMisc) ... ok 2022-08-01T19:42:03.619 INFO:tasks.cephfs_test_runner:Starting test: test_fs_new (tasks.cephfs.test_misc.TestMisc) 2022-08-01T19:42:03.619 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_fs_new' 2022-08-01T19:42:03.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.742+0000 7f6ad69a8700 1 Processor -- start 2022-08-01T19:42:03.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.742+0000 7f6ad69a8700 1 -- start start 2022-08-01T19:42:03.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.742+0000 7f6ad69a8700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6ad00b01a0 0x7f6ad00a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:03.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.742+0000 7f6ad69a8700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ad00a7250 con 0x7f6ad00afd60 2022-08-01T19:42:03.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.742+0000 7f6ad69a8700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ad00a7480 con 0x7f6ad00b2710 2022-08-01T19:42:03.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.742+0000 7f6ad69a8700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ad00a7720 con 0x7f6ad00b01a0 2022-08-01T19:42:03.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.742+0000 7f6ad51a5700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6ad00b01a0 0x7f6ad00a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:03.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.742+0000 7f6ad51a5700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6ad00b01a0 0x7f6ad00a8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43098/0 (socket says 172.21.15.17:43098) 2022-08-01T19:42:03.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.742+0000 7f6ad51a5700 1 -- 172.21.15.17:0/1656041727 learned_addr learned my addr 172.21.15.17:0/1656041727 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:03.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.742+0000 7f6ad51a5700 1 -- 172.21.15.17:0/1656041727 >> v1:172.21.15.77:6789/0 conn(0x7f6ad00b2710 legacy=0x7f6ad014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:03.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.742+0000 7f6ad51a5700 1 -- 172.21.15.17:0/1656041727 >> v1:172.21.15.17:6789/0 conn(0x7f6ad00afd60 legacy=0x7f6ad0151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:03.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.742+0000 7f6ad51a5700 1 -- 172.21.15.17:0/1656041727 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad00a79c0 con 0x7f6ad00b01a0 2022-08-01T19:42:03.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.742+0000 7f6ad51a5700 1 --2- 172.21.15.17:0/1656041727 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ad00b01a0 0x7f6ad00a8540 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 rx=0x7f6ad0061c10 tx=0x7f6ac0006eb0).ready entity=mon.0 client_cookie=78538f398a7396ca server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:03.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.742+0000 7f6ad49a4700 1 -- 172.21.15.17:0/1656041727 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ac000d610 con 0x7f6ad00b01a0 2022-08-01T19:42:03.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.742+0000 7f6ad49a4700 1 -- 172.21.15.17:0/1656041727 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac0000e90 con 0x7f6ad00b01a0 2022-08-01T19:42:03.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.742+0000 7f6ad49a4700 1 -- 172.21.15.17:0/1656041727 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ac000d960 con 0x7f6ad00b01a0 2022-08-01T19:42:03.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.742+0000 7f6ad69a8700 1 -- 172.21.15.17:0/1656041727 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ad00b01a0 msgr2=0x7f6ad00a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:03.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.742+0000 7f6ad69a8700 1 --2- 172.21.15.17:0/1656041727 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ad00b01a0 0x7f6ad00a8540 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 rx=0x7f6ad0061c10 tx=0x7f6ac0006eb0).stop 2022-08-01T19:42:03.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.742+0000 7f6ad69a8700 1 -- 172.21.15.17:0/1656041727 shutdown_connections 2022-08-01T19:42:03.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.742+0000 7f6ad69a8700 1 --2- 172.21.15.17:0/1656041727 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ad00b01a0 0x7f6ad00a8540 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:03.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.742+0000 7f6ad69a8700 1 -- 172.21.15.17:0/1656041727 >> 172.21.15.17:0/1656041727 conn(0x7f6ad0060310 msgr2=0x7f6ad00b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:03.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.746+0000 7f6ad69a8700 1 -- 172.21.15.17:0/1656041727 shutdown_connections 2022-08-01T19:42:03.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.746+0000 7f6ad69a8700 1 -- 172.21.15.17:0/1656041727 wait complete. 2022-08-01T19:42:03.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.746+0000 7f6ad69a8700 1 Processor -- start 2022-08-01T19:42:03.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.746+0000 7f6ad69a8700 1 -- start start 2022-08-01T19:42:03.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.746+0000 7f6ad69a8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ad00afd60 0x7f6ad00634d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:03.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.746+0000 7f6ad69a8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ad00b01a0 0x7f6ad011cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:03.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.746+0000 7f6ad69a8700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6ad00b2710 0x7f6ad0156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:03.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.746+0000 7f6ad69a8700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad0064300 con 0x7f6ad00b01a0 2022-08-01T19:42:03.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.746+0000 7f6ad69a8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad0063dc0 con 0x7f6ad00afd60 2022-08-01T19:42:03.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.746+0000 7f6ad69a8700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad0064060 con 0x7f6ad00b2710 2022-08-01T19:42:03.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.746+0000 7f6ad51a5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ad00b01a0 0x7f6ad011cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:03.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.746+0000 7f6ad51a5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ad00b01a0 0x7f6ad011cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43106/0 (socket says 172.21.15.17:43106) 2022-08-01T19:42:03.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.746+0000 7f6ad51a5700 1 -- 172.21.15.17:0/401673103 learned_addr learned my addr 172.21.15.17:0/401673103 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:03.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.746+0000 7f6ad59a6700 1 --2- 172.21.15.17:0/401673103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ad00afd60 0x7f6ad00634d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:03.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.746+0000 7f6ad61a7700 1 --2- 172.21.15.17:0/401673103 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6ad00b2710 0x7f6ad0156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:03.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.750+0000 7f6ad51a5700 1 -- 172.21.15.17:0/401673103 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6ad00b2710 msgr2=0x7f6ad0156870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:03.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.750+0000 7f6ad51a5700 1 --2- 172.21.15.17:0/401673103 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6ad00b2710 0x7f6ad0156870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:03.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.750+0000 7f6ad51a5700 1 -- 172.21.15.17:0/401673103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ad00afd60 msgr2=0x7f6ad00634d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:03.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.750+0000 7f6ad51a5700 1 --2- 172.21.15.17:0/401673103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ad00afd60 0x7f6ad00634d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:03.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.750+0000 7f6ad51a5700 1 -- 172.21.15.17:0/401673103 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac000b1e0 con 0x7f6ad00b01a0 2022-08-01T19:42:03.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.750+0000 7f6ad51a5700 1 --2- 172.21.15.17:0/401673103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ad00b01a0 0x7f6ad011cfb0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 rx=0x7f6ad0061c10 tx=0x7f6ac0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:03.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.750+0000 7f6abeffd700 1 -- 172.21.15.17:0/401673103 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ac000bec0 con 0x7f6ad00b01a0 2022-08-01T19:42:03.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.750+0000 7f6abeffd700 1 -- 172.21.15.17:0/401673103 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac000d970 con 0x7f6ad00b01a0 2022-08-01T19:42:03.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.750+0000 7f6ad69a8700 1 -- 172.21.15.17:0/401673103 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ad00abdc0 con 0x7f6ad00b01a0 2022-08-01T19:42:03.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.750+0000 7f6abeffd700 1 -- 172.21.15.17:0/401673103 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ac0011e20 con 0x7f6ad00b01a0 2022-08-01T19:42:03.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.750+0000 7f6ad69a8700 1 -- 172.21.15.17:0/401673103 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ad0066e50 con 0x7f6ad00b01a0 2022-08-01T19:42:03.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.750+0000 7f6ad69a8700 1 -- 172.21.15.17:0/401673103 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ad00574e0 con 0x7f6ad00b01a0 2022-08-01T19:42:03.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.754+0000 7f6abeffd700 1 -- 172.21.15.17:0/401673103 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6ac0025030 con 0x7f6ad00b01a0 2022-08-01T19:42:03.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.754+0000 7f6abeffd700 1 --2- 172.21.15.17:0/401673103 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6ab0053b00 0x7f6ab0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:03.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.754+0000 7f6abeffd700 1 -- 172.21.15.17:0/401673103 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f6ac006cf10 con 0x7f6ad00b01a0 2022-08-01T19:42:03.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.754+0000 7f6ad59a6700 1 --2- 172.21.15.17:0/401673103 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6ab0053b00 0x7f6ab0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:03.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.754+0000 7f6ad59a6700 1 --2- 172.21.15.17:0/401673103 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6ab0053b00 0x7f6ab0055fc0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f6acc00c000 tx=0x7f6acc008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:03.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.762+0000 7f6abeffd700 1 -- 172.21.15.17:0/401673103 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6ac0043fb0 con 0x7f6ad00b01a0 2022-08-01T19:42:03.933 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:03.930+0000 7f6ad69a8700 1 -- 172.21.15.17:0/401673103 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_fs_new"]} v 0) v1 -- 0x7f6ad0069a70 con 0x7f6ad00b01a0 2022-08-01T19:42:04.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.582+0000 7f6abeffd700 1 -- 172.21.15.17:0/401673103 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_fs_new"]}]=0 v360) v1 ==== 125+0+0 (secure 0 0 0) 0x7f6ac0023020 con 0x7f6ad00b01a0 2022-08-01T19:42:04.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.586+0000 7f6ad69a8700 1 -- 172.21.15.17:0/401673103 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6ab0053b00 msgr2=0x7f6ab0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:04.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.586+0000 7f6ad69a8700 1 --2- 172.21.15.17:0/401673103 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6ab0053b00 0x7f6ab0055fc0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f6acc00c000 tx=0x7f6acc008040).stop 2022-08-01T19:42:04.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.586+0000 7f6ad69a8700 1 -- 172.21.15.17:0/401673103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ad00b01a0 msgr2=0x7f6ad011cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:04.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.586+0000 7f6ad69a8700 1 --2- 172.21.15.17:0/401673103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ad00b01a0 0x7f6ad011cfb0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 rx=0x7f6ad0061c10 tx=0x7f6ac0015000).stop 2022-08-01T19:42:04.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.586+0000 7f6ad69a8700 1 -- 172.21.15.17:0/401673103 shutdown_connections 2022-08-01T19:42:04.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.586+0000 7f6ad69a8700 1 --2- 172.21.15.17:0/401673103 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6ab0053b00 0x7f6ab0055fc0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:04.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.586+0000 7f6ad69a8700 1 --2- 172.21.15.17:0/401673103 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6ad00b2710 0x7f6ad0156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:04.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.586+0000 7f6ad69a8700 1 --2- 172.21.15.17:0/401673103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ad00b01a0 0x7f6ad011cfb0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:04.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.586+0000 7f6ad69a8700 1 --2- 172.21.15.17:0/401673103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ad00afd60 0x7f6ad00634d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:04.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.586+0000 7f6ad69a8700 1 -- 172.21.15.17:0/401673103 >> 172.21.15.17:0/401673103 conn(0x7f6ad0060310 msgr2=0x7f6ad00af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:04.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.590+0000 7f6ad69a8700 1 -- 172.21.15.17:0/401673103 shutdown_connections 2022-08-01T19:42:04.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.590+0000 7f6ad69a8700 1 -- 172.21.15.17:0/401673103 wait complete. 2022-08-01T19:42:04.613 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-08-01T19:42:04.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.730+0000 7f7eed86a700 1 Processor -- start 2022-08-01T19:42:04.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.730+0000 7f7eed86a700 1 -- start start 2022-08-01T19:42:04.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.730+0000 7f7eed86a700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7ee80b4310 0x7f7ee8140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:04.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.730+0000 7f7eed86a700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ee8143cf0 con 0x7f7ee8146ae0 2022-08-01T19:42:04.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.730+0000 7f7eed86a700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ee8143f90 con 0x7f7ee80b4820 2022-08-01T19:42:04.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.730+0000 7f7eed86a700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ee81476a0 con 0x7f7ee80b4310 2022-08-01T19:42:04.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.730+0000 7f7ee67fc700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7ee80b4310 0x7f7ee8140ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:04.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.730+0000 7f7ee67fc700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7ee80b4310 0x7f7ee8140ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43114/0 (socket says 172.21.15.17:43114) 2022-08-01T19:42:04.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.730+0000 7f7ee67fc700 1 -- 172.21.15.17:0/2694485392 learned_addr learned my addr 172.21.15.17:0/2694485392 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:04.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.730+0000 7f7ee6ffd700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f7ee8146ae0 0x7f7ee81403c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35306/0 (socket says 172.21.15.17:35306) 2022-08-01T19:42:04.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.730+0000 7f7ee67fc700 1 -- 172.21.15.17:0/2694485392 >> v1:172.21.15.77:6789/0 conn(0x7f7ee80b4820 legacy=0x7f7ee80b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:04.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.730+0000 7f7ee67fc700 1 -- 172.21.15.17:0/2694485392 >> v1:172.21.15.17:6789/0 conn(0x7f7ee8146ae0 legacy=0x7f7ee81403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:04.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.730+0000 7f7ee5ffb700 1 -- 172.21.15.17:0/2694485392 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1602637958 0 0) 0x7f7ee8143f90 con 0x7f7ee80b4820 2022-08-01T19:42:04.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.730+0000 7f7ee67fc700 1 -- 172.21.15.17:0/2694485392 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee80b0e20 con 0x7f7ee80b4310 2022-08-01T19:42:04.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.730+0000 7f7ee67fc700 1 --2- 172.21.15.17:0/2694485392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ee80b4310 0x7f7ee8140ad0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7f7ed8001110 tx=0x7f7ed0006eb0).ready entity=mon.0 client_cookie=1a3e6e7388916475 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:04.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.730+0000 7f7ee5ffb700 1 -- 172.21.15.17:0/2694485392 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ed000d610 con 0x7f7ee80b4310 2022-08-01T19:42:04.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.730+0000 7f7ee5ffb700 1 -- 172.21.15.17:0/2694485392 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed0000e90 con 0x7f7ee80b4310 2022-08-01T19:42:04.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.730+0000 7f7ee5ffb700 1 -- 172.21.15.17:0/2694485392 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ed000d960 con 0x7f7ee80b4310 2022-08-01T19:42:04.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.734+0000 7f7eed86a700 1 -- 172.21.15.17:0/2694485392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ee80b4310 msgr2=0x7f7ee8140ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:04.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.734+0000 7f7eed86a700 1 --2- 172.21.15.17:0/2694485392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ee80b4310 0x7f7ee8140ad0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7f7ed8001110 tx=0x7f7ed0006eb0).stop 2022-08-01T19:42:04.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.734+0000 7f7eed86a700 1 -- 172.21.15.17:0/2694485392 shutdown_connections 2022-08-01T19:42:04.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.734+0000 7f7eed86a700 1 --2- 172.21.15.17:0/2694485392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ee80b4310 0x7f7ee8140ad0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:04.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.734+0000 7f7eed86a700 1 -- 172.21.15.17:0/2694485392 >> 172.21.15.17:0/2694485392 conn(0x7f7ee805c620 msgr2=0x7f7ee805ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:04.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.734+0000 7f7eed86a700 1 -- 172.21.15.17:0/2694485392 shutdown_connections 2022-08-01T19:42:04.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.734+0000 7f7eed86a700 1 -- 172.21.15.17:0/2694485392 wait complete. 2022-08-01T19:42:04.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.734+0000 7f7eed86a700 1 Processor -- start 2022-08-01T19:42:04.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.734+0000 7f7eed86a700 1 -- start start 2022-08-01T19:42:04.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.734+0000 7f7eed86a700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7ee80b4310 0x7f7ee80b8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:04.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.734+0000 7f7eed86a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ee80b4820 0x7f7ee80b6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:04.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.734+0000 7f7eed86a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ee8146ae0 0x7f7ee8156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:04.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.734+0000 7f7eed86a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee813fd80 con 0x7f7ee80b4820 2022-08-01T19:42:04.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.734+0000 7f7eed86a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee80b7b60 con 0x7f7ee8146ae0 2022-08-01T19:42:04.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.734+0000 7f7eed86a700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee80b7e00 con 0x7f7ee80b4310 2022-08-01T19:42:04.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.734+0000 7f7ee67fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ee80b4820 0x7f7ee80b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:04.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.734+0000 7f7ee67fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ee80b4820 0x7f7ee80b6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43120/0 (socket says 172.21.15.17:43120) 2022-08-01T19:42:04.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.734+0000 7f7ee67fc700 1 -- 172.21.15.17:0/3332416350 learned_addr learned my addr 172.21.15.17:0/3332416350 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:04.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.734+0000 7f7ee77fe700 1 --2- 172.21.15.17:0/3332416350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ee8146ae0 0x7f7ee8156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:04.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.734+0000 7f7ee6ffd700 1 --2- 172.21.15.17:0/3332416350 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7ee80b4310 0x7f7ee80b8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:04.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.734+0000 7f7ee67fc700 1 -- 172.21.15.17:0/3332416350 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7ee80b4310 msgr2=0x7f7ee80b8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:04.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.734+0000 7f7ee67fc700 1 --2- 172.21.15.17:0/3332416350 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7ee80b4310 0x7f7ee80b8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:04.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.734+0000 7f7ee67fc700 1 -- 172.21.15.17:0/3332416350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ee8146ae0 msgr2=0x7f7ee8156410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:04.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.734+0000 7f7ee67fc700 1 --2- 172.21.15.17:0/3332416350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ee8146ae0 0x7f7ee8156410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:04.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.734+0000 7f7ee67fc700 1 -- 172.21.15.17:0/3332416350 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed000b1e0 con 0x7f7ee80b4820 2022-08-01T19:42:04.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.734+0000 7f7ee67fc700 1 --2- 172.21.15.17:0/3332416350 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ee80b4820 0x7f7ee80b6520 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 rx=0x7f7ed8001110 tx=0x7f7ed0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:04.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.734+0000 7f7edffff700 1 -- 172.21.15.17:0/3332416350 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ed000de90 con 0x7f7ee80b4820 2022-08-01T19:42:04.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.734+0000 7f7edffff700 1 -- 172.21.15.17:0/3332416350 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed0002f50 con 0x7f7ee80b4820 2022-08-01T19:42:04.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.734+0000 7f7eed86a700 1 -- 172.21.15.17:0/3332416350 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ee8152580 con 0x7f7ee80b4820 2022-08-01T19:42:04.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.734+0000 7f7edffff700 1 -- 172.21.15.17:0/3332416350 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ed0011e20 con 0x7f7ee80b4820 2022-08-01T19:42:04.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.734+0000 7f7eed86a700 1 -- 172.21.15.17:0/3332416350 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ee8156900 con 0x7f7ee80b4820 2022-08-01T19:42:04.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.738+0000 7f7eed86a700 1 -- 172.21.15.17:0/3332416350 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ee80574e0 con 0x7f7ee80b4820 2022-08-01T19:42:04.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.738+0000 7f7edffff700 1 -- 172.21.15.17:0/3332416350 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7ed0025030 con 0x7f7ee80b4820 2022-08-01T19:42:04.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.738+0000 7f7edffff700 1 --2- 172.21.15.17:0/3332416350 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7ecc053aa0 0x7f7ecc055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:04.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.738+0000 7f7edffff700 1 -- 172.21.15.17:0/3332416350 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f7ed006d0c0 con 0x7f7ee80b4820 2022-08-01T19:42:04.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.738+0000 7f7ee6ffd700 1 --2- 172.21.15.17:0/3332416350 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7ecc053aa0 0x7f7ecc055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:04.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.738+0000 7f7ee6ffd700 1 --2- 172.21.15.17:0/3332416350 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7ecc053aa0 0x7f7ecc055f60 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f7ed800b000 tx=0x7f7ed8007160).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:04.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.742+0000 7f7edffff700 1 -- 172.21.15.17:0/3332416350 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7ed0044090 con 0x7f7ee80b4820 2022-08-01T19:42:04.871 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.870+0000 7f7eed86a700 1 -- 172.21.15.17:0/3332416350 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7ee8069a70 con 0x7f7ee80b4820 2022-08-01T19:42:04.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.878+0000 7f7edffff700 1 -- 172.21.15.17:0/3332416350 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7ed0047670 con 0x7f7ee80b4820 2022-08-01T19:42:04.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.878+0000 7f7eed86a700 1 -- 172.21.15.17:0/3332416350 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7ecc053aa0 msgr2=0x7f7ecc055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:04.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.878+0000 7f7eed86a700 1 --2- 172.21.15.17:0/3332416350 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7ecc053aa0 0x7f7ecc055f60 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f7ed800b000 tx=0x7f7ed8007160).stop 2022-08-01T19:42:04.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.878+0000 7f7eed86a700 1 -- 172.21.15.17:0/3332416350 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ee80b4820 msgr2=0x7f7ee80b6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:04.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.878+0000 7f7eed86a700 1 --2- 172.21.15.17:0/3332416350 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ee80b4820 0x7f7ee80b6520 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 rx=0x7f7ed8001110 tx=0x7f7ed0015000).stop 2022-08-01T19:42:04.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.878+0000 7f7eed86a700 1 -- 172.21.15.17:0/3332416350 shutdown_connections 2022-08-01T19:42:04.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.878+0000 7f7eed86a700 1 --2- 172.21.15.17:0/3332416350 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7ecc053aa0 0x7f7ecc055f60 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:04.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.878+0000 7f7eed86a700 1 --2- 172.21.15.17:0/3332416350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ee8146ae0 0x7f7ee8156410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:04.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.878+0000 7f7eed86a700 1 --2- 172.21.15.17:0/3332416350 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ee80b4820 0x7f7ee80b6520 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:04.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.878+0000 7f7eed86a700 1 --2- 172.21.15.17:0/3332416350 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7ee80b4310 0x7f7ee80b8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:04.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.878+0000 7f7eed86a700 1 -- 172.21.15.17:0/3332416350 >> 172.21.15.17:0/3332416350 conn(0x7f7ee805c620 msgr2=0x7f7ee813e550 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:04.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.878+0000 7f7eed86a700 1 -- 172.21.15.17:0/3332416350 shutdown_connections 2022-08-01T19:42:04.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:04.878+0000 7f7eed86a700 1 -- 172.21.15.17:0/3332416350 wait complete. 2022-08-01T19:42:04.896 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T19:42:04.897 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:42:04.915 DEBUG:teuthology.orchestra.run.smithi077:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:42:04.941 DEBUG:teuthology.orchestra.run.smithi085:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:42:04.964 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:42:04.996 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T19:42:04.997 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:42:04.997 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:42:04.998 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:42:05.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:42:05.017 INFO:teuthology.orchestra.run.smithi085.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-08-01T19:42:05.018 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-08-01T19:42:05.019 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:42:05.019 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:42:05.019 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:42:05.027 INFO:teuthology.orchestra.run.smithi190.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T19:42:05.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:42:05.029 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-08-01T19:42:05.034 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T19:42:05.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.146+0000 7fac1d1a2700 1 Processor -- start 2022-08-01T19:42:05.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.146+0000 7fac1d1a2700 1 -- start start 2022-08-01T19:42:05.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.146+0000 7fac1d1a2700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac1813d9b0 0x7fac180609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:05.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.146+0000 7fac1d1a2700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fac180b0e60 0x7fac18060eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:05.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.146+0000 7fac1d1a2700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac18066af0 con 0x7fac180b6510 2022-08-01T19:42:05.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.146+0000 7fac1d1a2700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac18066d90 con 0x7fac180b0e60 2022-08-01T19:42:05.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.146+0000 7fac1d1a2700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac18067030 con 0x7fac1813d9b0 2022-08-01T19:42:05.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.146+0000 7fac1c9a1700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fac180b0e60 0x7fac18060eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:05.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.146+0000 7fac17fff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fac180b6510 0x7fac1813d2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35322/0 (socket says 172.21.15.17:35322) 2022-08-01T19:42:05.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.146+0000 7fac17fff700 1 -- 172.21.15.17:0/2524324785 learned_addr learned my addr 172.21.15.17:0/2524324785 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:05.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.146+0000 7fac177fe700 1 --2- 172.21.15.17:0/2524324785 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac1813d9b0 0x7fac180609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:05.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.146+0000 7fac1c9a1700 1 -- 172.21.15.17:0/2524324785 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac1813d9b0 msgr2=0x7fac180609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:05.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.146+0000 7fac1c9a1700 1 --2- 172.21.15.17:0/2524324785 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac1813d9b0 0x7fac180609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:05.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.146+0000 7fac1c9a1700 1 -- 172.21.15.17:0/2524324785 >> v1:172.21.15.17:6789/0 conn(0x7fac180b6510 legacy=0x7fac1813d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:05.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.146+0000 7fac1c9a1700 1 -- 172.21.15.17:0/2524324785 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac180672d0 con 0x7fac180b0e60 2022-08-01T19:42:05.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.146+0000 7fac1c9a1700 1 --2- 172.21.15.17:0/2524324785 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac180b0e60 0x7fac18060eb0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 rx=0x7fac100032f0 tx=0x7fac100091d0).ready entity=mon.0 client_cookie=5f6686738a71a1fd server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:05.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.150+0000 7fac16ffd700 1 -- 172.21.15.17:0/2524324785 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac1000b6c0 con 0x7fac180b0e60 2022-08-01T19:42:05.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.150+0000 7fac16ffd700 1 -- 172.21.15.17:0/2524324785 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1000b820 con 0x7fac180b0e60 2022-08-01T19:42:05.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.150+0000 7fac16ffd700 1 -- 172.21.15.17:0/2524324785 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac1000b990 con 0x7fac180b0e60 2022-08-01T19:42:05.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.150+0000 7fac1d1a2700 1 -- 172.21.15.17:0/2524324785 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac180b0e60 msgr2=0x7fac18060eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:05.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.150+0000 7fac1d1a2700 1 --2- 172.21.15.17:0/2524324785 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac180b0e60 0x7fac18060eb0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 rx=0x7fac100032f0 tx=0x7fac100091d0).stop 2022-08-01T19:42:05.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.150+0000 7fac1d1a2700 1 -- 172.21.15.17:0/2524324785 shutdown_connections 2022-08-01T19:42:05.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.150+0000 7fac1d1a2700 1 --2- 172.21.15.17:0/2524324785 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac180b0e60 0x7fac18060eb0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:05.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.150+0000 7fac1d1a2700 1 --2- 172.21.15.17:0/2524324785 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac1813d9b0 0x7fac180609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:05.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.150+0000 7fac1d1a2700 1 -- 172.21.15.17:0/2524324785 >> 172.21.15.17:0/2524324785 conn(0x7fac18063800 msgr2=0x7fac18063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:05.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.150+0000 7fac1d1a2700 1 -- 172.21.15.17:0/2524324785 shutdown_connections 2022-08-01T19:42:05.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.150+0000 7fac1d1a2700 1 -- 172.21.15.17:0/2524324785 wait complete. 2022-08-01T19:42:05.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.150+0000 7fac1d1a2700 1 Processor -- start 2022-08-01T19:42:05.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.150+0000 7fac1d1a2700 1 -- start start 2022-08-01T19:42:05.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.154+0000 7fac1d1a2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac180b0e60 0x7fac180ad0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:05.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.154+0000 7fac1d1a2700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac180b6510 0x7fac180add50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:05.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.154+0000 7fac1d1a2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac1813d9b0 0x7fac180ae240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:05.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.154+0000 7fac1d1a2700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac18140870 con 0x7fac180b0e60 2022-08-01T19:42:05.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.154+0000 7fac1d1a2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac18140b10 con 0x7fac1813d9b0 2022-08-01T19:42:05.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.154+0000 7fac1d1a2700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac18140db0 con 0x7fac180b6510 2022-08-01T19:42:05.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.154+0000 7fac17fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac180b0e60 0x7fac180ad0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:05.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.154+0000 7fac17fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac180b0e60 0x7fac180ad0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43132/0 (socket says 172.21.15.17:43132) 2022-08-01T19:42:05.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.154+0000 7fac177fe700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac180b6510 0x7fac180add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:05.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.154+0000 7fac17fff700 1 -- 172.21.15.17:0/1801555398 learned_addr learned my addr 172.21.15.17:0/1801555398 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:05.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.154+0000 7fac1c9a1700 1 --2- 172.21.15.17:0/1801555398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac1813d9b0 0x7fac180ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:05.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.154+0000 7fac17fff700 1 -- 172.21.15.17:0/1801555398 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac180b6510 msgr2=0x7fac180add50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:05.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.154+0000 7fac17fff700 1 --2- 172.21.15.17:0/1801555398 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac180b6510 0x7fac180add50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:05.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.154+0000 7fac17fff700 1 -- 172.21.15.17:0/1801555398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac1813d9b0 msgr2=0x7fac180ae240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:05.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.154+0000 7fac17fff700 1 --2- 172.21.15.17:0/1801555398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac1813d9b0 0x7fac180ae240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:05.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.154+0000 7fac17fff700 1 -- 172.21.15.17:0/1801555398 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac10009470 con 0x7fac180b0e60 2022-08-01T19:42:05.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.154+0000 7fac17fff700 1 --2- 172.21.15.17:0/1801555398 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac180b0e60 0x7fac180ad0d0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 rx=0x7fac18067700 tx=0x7fac0c007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:05.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.154+0000 7fac157fa700 1 -- 172.21.15.17:0/1801555398 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac0c0070d0 con 0x7fac180b0e60 2022-08-01T19:42:05.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.154+0000 7fac157fa700 1 -- 172.21.15.17:0/1801555398 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac0c007230 con 0x7fac180b0e60 2022-08-01T19:42:05.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.154+0000 7fac1d1a2700 1 -- 172.21.15.17:0/1801555398 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac180ad8b0 con 0x7fac180b0e60 2022-08-01T19:42:05.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.154+0000 7fac157fa700 1 -- 172.21.15.17:0/1801555398 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac0c005610 con 0x7fac180b0e60 2022-08-01T19:42:05.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.154+0000 7fac1d1a2700 1 -- 172.21.15.17:0/1801555398 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac180ab900 con 0x7fac180b0e60 2022-08-01T19:42:05.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.158+0000 7fac1d1a2700 1 -- 172.21.15.17:0/1801555398 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac180574e0 con 0x7fac180b0e60 2022-08-01T19:42:05.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.158+0000 7fac157fa700 1 -- 172.21.15.17:0/1801555398 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fac0c010070 con 0x7fac180b0e60 2022-08-01T19:42:05.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.158+0000 7fac157fa700 1 --2- 172.21.15.17:0/1801555398 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fabf8053950 0x7fabf8055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:05.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.158+0000 7fac157fa700 1 -- 172.21.15.17:0/1801555398 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fac0c06b360 con 0x7fac180b0e60 2022-08-01T19:42:05.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.158+0000 7fac177fe700 1 --2- 172.21.15.17:0/1801555398 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fabf8053950 0x7fabf8055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:05.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.158+0000 7fac177fe700 1 --2- 172.21.15.17:0/1801555398 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fabf8053950 0x7fabf8055e10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7fac0800b000 tx=0x7fac08006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:05.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.162+0000 7fac157fa700 1 -- 172.21.15.17:0/1801555398 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fac0c0422f0 con 0x7fac180b0e60 2022-08-01T19:42:05.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.290+0000 7fac1d1a2700 1 -- 172.21.15.17:0/1801555398 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fac1806ac80 con 0x7fac180b0e60 2022-08-01T19:42:05.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.290+0000 7fac157fa700 1 -- 172.21.15.17:0/1801555398 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v110) v1 ==== 81+0+29229 (secure 0 0 0) 0x7fac0c018070 con 0x7fac180b0e60 2022-08-01T19:42:05.295 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:42:05.296 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:42:05.296 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 110, 2022-08-01T19:42:05.296 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", 2022-08-01T19:42:05.297 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2022-08-01T19:33:50.105743+0000", 2022-08-01T19:42:05.297 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2022-08-01T19:42:02.564125+0000", 2022-08-01T19:42:05.298 INFO:teuthology.orchestra.run.smithi017.stdout: "last_up_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:42:05.306 INFO:teuthology.orchestra.run.smithi017.stdout: "last_in_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:42:05.306 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T19:42:05.307 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_num": 5799936, 2022-08-01T19:42:05.307 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_set": [ 2022-08-01T19:42:05.307 INFO:teuthology.orchestra.run.smithi017.stdout: "pglog_hardlimit", 2022-08-01T19:42:05.307 INFO:teuthology.orchestra.run.smithi017.stdout: "purged_snapdirs", 2022-08-01T19:42:05.308 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_deletes", 2022-08-01T19:42:05.308 INFO:teuthology.orchestra.run.smithi017.stdout: "sortbitwise" 2022-08-01T19:42:05.308 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:42:05.308 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_version": 6, 2022-08-01T19:42:05.309 INFO:teuthology.orchestra.run.smithi017.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T19:42:05.309 INFO:teuthology.orchestra.run.smithi017.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T19:42:05.309 INFO:teuthology.orchestra.run.smithi017.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T19:42:05.309 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_snapshot": "", 2022-08-01T19:42:05.309 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_max": 20, 2022-08-01T19:42:05.315 INFO:teuthology.orchestra.run.smithi017.stdout: "max_osd": 8, 2022-08-01T19:42:05.315 INFO:teuthology.orchestra.run.smithi017.stdout: "require_min_compat_client": "jewel", 2022-08-01T19:42:05.315 INFO:teuthology.orchestra.run.smithi017.stdout: "min_compat_client": "jewel", 2022-08-01T19:42:05.316 INFO:teuthology.orchestra.run.smithi017.stdout: "require_osd_release": "octopus", 2022-08-01T19:42:05.316 INFO:teuthology.orchestra.run.smithi017.stdout: "pools": [ 2022-08-01T19:42:05.316 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.316 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 1, 2022-08-01T19:42:05.317 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "device_health_metrics", 2022-08-01T19:42:05.317 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:33:53.001841+0000", 2022-08-01T19:42:05.317 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:42:05.317 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:42:05.317 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:42:05.319 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:42:05.319 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:42:05.319 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:42:05.319 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:42:05.320 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:42:05.320 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 1, 2022-08-01T19:42:05.320 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 1, 2022-08-01T19:42:05.320 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 1, 2022-08-01T19:42:05.321 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 1, 2022-08-01T19:42:05.321 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 1, 2022-08-01T19:42:05.321 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:42:05.325 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:42:05.326 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:42:05.326 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:42:05.327 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:42:05.327 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:42:05.328 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:42:05.338 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.338 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "16", 2022-08-01T19:42:05.339 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:42:05.339 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:42:05.339 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:42:05.339 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:42:05.339 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:42:05.340 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:42:05.340 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:42:05.340 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:42:05.340 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:42:05.341 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:42:05.341 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:42:05.341 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:42:05.341 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:42:05.343 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:42:05.343 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:42:05.343 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:42:05.344 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:42:05.344 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:42:05.344 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:42:05.344 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:42:05.345 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:42:05.345 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:42:05.346 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:42:05.346 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:42:05.346 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:42:05.346 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:42:05.347 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.347 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:42:05.347 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:42:05.347 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:42:05.347 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:42:05.348 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:42:05.348 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:42:05.348 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:42:05.348 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:42:05.350 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:42:05.350 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:42:05.350 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:42:05.350 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:42:05.351 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 1 2022-08-01T19:42:05.351 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.351 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:42:05.351 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_devicehealth": {} 2022-08-01T19:42:05.352 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.352 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.352 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.352 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 2, 2022-08-01T19:42:05.353 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "rbd", 2022-08-01T19:42:05.353 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:34:01.553098+0000", 2022-08-01T19:42:05.353 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:42:05.353 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:42:05.353 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:42:05.354 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:42:05.354 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:42:05.354 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:42:05.355 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:42:05.355 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:42:05.355 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:42:05.355 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:42:05.356 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:42:05.356 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:42:05.356 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:42:05.357 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:42:05.357 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:42:05.357 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:42:05.357 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:42:05.358 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:42:05.358 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:42:05.358 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:42:05.358 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.358 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "19", 2022-08-01T19:42:05.359 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:42:05.359 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:42:05.359 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:42:05.359 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:42:05.360 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:42:05.360 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:42:05.360 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:42:05.360 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:42:05.361 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:42:05.361 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:42:05.361 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:42:05.362 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:42:05.362 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:42:05.362 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:42:05.362 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:42:05.363 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:42:05.363 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:42:05.363 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:42:05.363 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:42:05.363 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:42:05.364 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:42:05.364 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:42:05.364 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:42:05.364 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:42:05.365 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:42:05.365 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:42:05.365 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.365 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:42:05.369 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:42:05.369 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:42:05.370 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:42:05.370 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:42:05.370 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:42:05.370 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:42:05.371 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:42:05.371 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:42:05.371 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:42:05.371 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:42:05.372 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:42:05.374 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:42:05.375 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd": {} 2022-08-01T19:42:05.375 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.376 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.376 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:42:05.377 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2022-08-01T19:42:05.377 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.378 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:42:05.378 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a962f14f-cf16-4fa8-84ff-c85325db9132", 2022-08-01T19:42:05.382 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:42:05.383 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:42:05.383 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:42:05.383 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:42:05.384 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:42:05.384 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:42:05.385 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:42:05.385 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 105, 2022-08-01T19:42:05.385 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:42:05.386 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:42:05.386 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:42:05.386 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:05.387 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.387 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:05.387 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6800", 2022-08-01T19:42:05.388 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:42:05.388 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.388 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.389 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:05.389 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6801", 2022-08-01T19:42:05.389 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:42:05.390 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.390 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.390 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.391 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:42:05.391 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:05.391 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.392 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:05.392 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6802", 2022-08-01T19:42:05.392 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:42:05.393 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.393 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.393 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:05.395 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6803", 2022-08-01T19:42:05.396 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:42:05.396 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.397 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.398 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.398 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:42:05.399 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:05.399 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.400 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:05.400 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6806", 2022-08-01T19:42:05.401 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:42:05.401 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.402 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.402 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:05.402 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6807", 2022-08-01T19:42:05.403 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:42:05.403 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.404 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.404 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.405 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:42:05.405 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:05.406 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.406 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:05.407 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6804", 2022-08-01T19:42:05.407 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:42:05.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.410 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:05.410 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6805", 2022-08-01T19:42:05.411 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:42:05.411 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.411 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.412 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.412 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6801/23008", 2022-08-01T19:42:05.412 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6803/23008", 2022-08-01T19:42:05.413 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6807/23008", 2022-08-01T19:42:05.413 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6805/23008", 2022-08-01T19:42:05.414 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:42:05.415 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:42:05.415 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:42:05.416 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.416 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.417 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.417 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:42:05.418 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "805f7051-a527-4e98-8844-b6e406a63248", 2022-08-01T19:42:05.418 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:42:05.418 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:42:05.419 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:42:05.419 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:42:05.420 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:42:05.420 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:42:05.421 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:42:05.421 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 105, 2022-08-01T19:42:05.430 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:42:05.430 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:42:05.431 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:42:05.431 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:05.432 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.432 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:05.433 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6808", 2022-08-01T19:42:05.434 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:42:05.434 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.435 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.435 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:05.436 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6809", 2022-08-01T19:42:05.436 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:42:05.436 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.437 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.437 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.438 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:42:05.438 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:05.439 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.439 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:05.440 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6810", 2022-08-01T19:42:05.440 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:42:05.441 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.441 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.442 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:05.442 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6811", 2022-08-01T19:42:05.442 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:42:05.443 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.443 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.444 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.444 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:42:05.445 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:05.445 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.446 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:05.446 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6814", 2022-08-01T19:42:05.446 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:42:05.447 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.447 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.448 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:05.448 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6815", 2022-08-01T19:42:05.449 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:42:05.449 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.450 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.451 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.451 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:42:05.451 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:05.452 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.452 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:05.453 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6812", 2022-08-01T19:42:05.453 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:42:05.454 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.454 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.455 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:05.455 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6813", 2022-08-01T19:42:05.455 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:42:05.456 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.456 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.456 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.457 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6809/23007", 2022-08-01T19:42:05.458 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6811/23007", 2022-08-01T19:42:05.458 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6815/23007", 2022-08-01T19:42:05.458 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6813/23007", 2022-08-01T19:42:05.459 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:42:05.460 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:42:05.460 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:42:05.460 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.461 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.461 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.461 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:42:05.461 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "89f7f3df-7c10-48e1-8b69-9e4243923a61", 2022-08-01T19:42:05.462 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:42:05.462 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:42:05.462 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:42:05.462 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:42:05.463 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:42:05.463 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:42:05.463 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:42:05.463 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 104, 2022-08-01T19:42:05.464 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:42:05.464 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:42:05.464 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:42:05.464 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:05.464 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.465 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:05.465 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6817", 2022-08-01T19:42:05.465 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:42:05.465 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.465 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.466 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:05.466 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6819", 2022-08-01T19:42:05.467 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:42:05.467 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.467 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.467 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.468 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:42:05.468 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:05.468 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.468 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:05.468 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6822", 2022-08-01T19:42:05.469 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:42:05.469 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.469 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.469 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:05.470 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6824", 2022-08-01T19:42:05.470 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:42:05.470 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.470 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.470 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.471 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:42:05.471 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:05.471 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.471 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:05.472 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6830", 2022-08-01T19:42:05.472 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:42:05.472 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.472 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.472 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:05.473 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6831", 2022-08-01T19:42:05.473 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:42:05.473 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.473 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.474 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.474 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:42:05.474 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:05.474 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.474 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:05.475 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6826", 2022-08-01T19:42:05.475 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:42:05.475 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.475 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.476 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:05.476 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6829", 2022-08-01T19:42:05.476 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:42:05.476 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.476 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.477 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.477 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6819/23010", 2022-08-01T19:42:05.477 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6824/23010", 2022-08-01T19:42:05.477 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6831/23010", 2022-08-01T19:42:05.478 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6829/23010", 2022-08-01T19:42:05.478 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:42:05.478 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:42:05.479 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:42:05.479 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.479 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.479 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.480 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:42:05.480 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "96123468-aabc-4a64-909e-a192c369090d", 2022-08-01T19:42:05.480 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:42:05.480 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:42:05.482 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:42:05.483 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:42:05.483 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:42:05.483 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:42:05.484 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:42:05.484 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 105, 2022-08-01T19:42:05.484 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:42:05.484 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:42:05.485 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:42:05.485 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:05.485 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.485 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:05.486 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6816", 2022-08-01T19:42:05.486 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:42:05.486 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.486 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.487 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:05.487 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6818", 2022-08-01T19:42:05.487 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:42:05.488 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.488 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.488 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.489 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:42:05.489 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:05.490 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.490 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:05.490 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6820", 2022-08-01T19:42:05.490 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:42:05.491 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.491 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.492 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:05.492 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6821", 2022-08-01T19:42:05.492 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:42:05.492 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.493 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.493 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.493 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:42:05.494 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:05.494 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.494 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:05.494 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6827", 2022-08-01T19:42:05.495 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:42:05.495 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.495 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.495 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:05.496 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6828", 2022-08-01T19:42:05.496 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:42:05.496 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.496 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.497 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.497 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:42:05.497 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:05.497 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.498 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:05.498 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6823", 2022-08-01T19:42:05.498 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:42:05.498 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.499 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.499 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:05.499 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6825", 2022-08-01T19:42:05.499 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:42:05.500 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.500 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.500 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.500 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6818/23009", 2022-08-01T19:42:05.501 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6821/23009", 2022-08-01T19:42:05.501 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6828/23009", 2022-08-01T19:42:05.501 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6825/23009", 2022-08-01T19:42:05.501 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:42:05.502 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:42:05.502 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:42:05.502 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.502 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.503 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.503 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:42:05.503 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a6e26665-e04b-46ec-ab66-e912acbf044c", 2022-08-01T19:42:05.504 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:42:05.504 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:42:05.504 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:42:05.504 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:42:05.505 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:42:05.505 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:42:05.505 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:42:05.505 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 104, 2022-08-01T19:42:05.505 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:42:05.506 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:42:05.507 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:42:05.507 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:05.507 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.507 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:05.508 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6810", 2022-08-01T19:42:05.508 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:42:05.508 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.508 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.509 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:05.509 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6811", 2022-08-01T19:42:05.509 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:42:05.509 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.510 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.510 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.510 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:42:05.511 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:05.511 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.511 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:05.511 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6812", 2022-08-01T19:42:05.512 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:42:05.512 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.512 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.512 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:05.513 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6813", 2022-08-01T19:42:05.513 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:42:05.513 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.513 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.514 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.514 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:42:05.514 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:05.514 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.515 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:05.515 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6816", 2022-08-01T19:42:05.515 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:42:05.515 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.516 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.516 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:05.516 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6817", 2022-08-01T19:42:05.516 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:42:05.516 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.517 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.517 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.517 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:42:05.517 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:05.518 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.518 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:05.518 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6814", 2022-08-01T19:42:05.519 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:42:05.519 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.519 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.519 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:05.519 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6815", 2022-08-01T19:42:05.520 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:42:05.520 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.520 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.520 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.521 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6811/23584", 2022-08-01T19:42:05.521 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6813/23584", 2022-08-01T19:42:05.521 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6817/23584", 2022-08-01T19:42:05.521 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6815/23584", 2022-08-01T19:42:05.522 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:42:05.522 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:42:05.522 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:42:05.522 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.523 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.523 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.524 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:42:05.524 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "502c0034-18ee-44ac-87ff-19b9dddb66a0", 2022-08-01T19:42:05.524 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:42:05.525 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:42:05.525 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:42:05.525 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:42:05.525 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:42:05.525 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:42:05.526 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:42:05.526 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 104, 2022-08-01T19:42:05.526 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:42:05.526 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:42:05.527 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:42:05.527 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:05.527 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.527 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:05.528 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6802", 2022-08-01T19:42:05.528 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:42:05.528 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.528 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.529 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:05.529 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6803", 2022-08-01T19:42:05.529 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:42:05.529 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.529 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.534 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.534 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:42:05.537 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:05.538 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.538 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:05.540 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6804", 2022-08-01T19:42:05.541 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:42:05.541 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.542 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.542 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:05.543 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6805", 2022-08-01T19:42:05.543 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:42:05.543 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.544 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.545 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.545 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:42:05.550 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:05.550 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.550 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:05.551 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6808", 2022-08-01T19:42:05.551 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:42:05.551 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.551 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.552 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:05.552 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6809", 2022-08-01T19:42:05.552 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:42:05.552 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.553 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.553 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.553 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:42:05.553 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:05.553 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.554 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:05.554 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6806", 2022-08-01T19:42:05.554 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:42:05.555 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.555 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.555 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:05.555 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6807", 2022-08-01T19:42:05.556 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:42:05.556 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.556 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.556 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.557 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6803/23585", 2022-08-01T19:42:05.557 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6805/23585", 2022-08-01T19:42:05.557 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6809/23585", 2022-08-01T19:42:05.557 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6807/23585", 2022-08-01T19:42:05.558 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:42:05.558 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:42:05.558 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:42:05.558 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.559 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.559 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.559 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:42:05.560 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "ac4e755a-7fe7-46cb-8407-852d5af9ffc5", 2022-08-01T19:42:05.561 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:42:05.561 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:42:05.561 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:42:05.561 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:42:05.562 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:42:05.562 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:42:05.562 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:42:05.562 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 105, 2022-08-01T19:42:05.563 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:42:05.563 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:42:05.564 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:42:05.564 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:05.564 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.564 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:05.564 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6820", 2022-08-01T19:42:05.565 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:42:05.566 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.566 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.566 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:05.567 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6821", 2022-08-01T19:42:05.567 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:42:05.567 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.567 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.568 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.568 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:42:05.568 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:05.568 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.568 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:05.569 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6823", 2022-08-01T19:42:05.569 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:42:05.569 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.570 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.570 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:05.571 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6825", 2022-08-01T19:42:05.571 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:42:05.572 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.572 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.572 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.577 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:42:05.577 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:05.578 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.578 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:05.578 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6831", 2022-08-01T19:42:05.578 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:42:05.579 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.579 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.579 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:05.579 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6833", 2022-08-01T19:42:05.580 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:42:05.581 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.582 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.582 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.582 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:42:05.583 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:05.583 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.583 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:05.583 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6827", 2022-08-01T19:42:05.584 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:42:05.590 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.590 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.591 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:05.591 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6829", 2022-08-01T19:42:05.592 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:42:05.592 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.593 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.593 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.593 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6821/23587", 2022-08-01T19:42:05.593 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6825/23587", 2022-08-01T19:42:05.594 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6833/23587", 2022-08-01T19:42:05.594 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6829/23587", 2022-08-01T19:42:05.594 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:42:05.595 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:42:05.595 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:42:05.595 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.595 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.596 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.596 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:42:05.596 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "874761e2-6410-40fc-b60c-e58abe5ce489", 2022-08-01T19:42:05.596 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:42:05.597 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:42:05.597 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:42:05.597 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:42:05.598 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:42:05.598 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:42:05.598 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:42:05.598 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 105, 2022-08-01T19:42:05.598 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:42:05.599 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:42:05.599 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:42:05.599 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:05.600 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.600 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:05.600 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6818", 2022-08-01T19:42:05.600 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:42:05.600 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.601 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.601 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:05.601 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6819", 2022-08-01T19:42:05.601 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:42:05.604 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.604 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.604 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.604 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:42:05.605 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:05.605 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.605 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:05.605 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6822", 2022-08-01T19:42:05.606 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:42:05.606 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.606 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.606 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:05.607 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6824", 2022-08-01T19:42:05.607 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:42:05.607 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.607 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.608 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.608 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:42:05.608 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:05.608 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.608 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:05.609 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6830", 2022-08-01T19:42:05.609 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:42:05.609 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.609 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.610 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:05.610 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6832", 2022-08-01T19:42:05.610 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:42:05.610 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.611 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.611 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.611 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:42:05.611 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:05.612 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.612 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:05.613 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6826", 2022-08-01T19:42:05.613 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:42:05.613 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.614 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.614 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:05.614 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6828", 2022-08-01T19:42:05.614 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:42:05.615 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.615 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.615 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.615 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6819/23586", 2022-08-01T19:42:05.615 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6824/23586", 2022-08-01T19:42:05.616 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6832/23586", 2022-08-01T19:42:05.616 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6828/23586", 2022-08-01T19:42:05.616 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:42:05.616 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:42:05.617 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:42:05.617 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:05.617 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.617 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:42:05.619 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_xinfo": [ 2022-08-01T19:42:05.619 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.626 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:42:05.626 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:42:05.627 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:42:05.627 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:42:05.627 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:42:05.628 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:42:05.628 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.588817+0000", 2022-08-01T19:42:05.629 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:42:05.629 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.630 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.630 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:42:05.631 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:42:05.631 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:42:05.632 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:42:05.632 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:42:05.632 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:42:05.633 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.782753+0000", 2022-08-01T19:42:05.633 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:42:05.634 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.634 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.634 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:42:05.635 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:42:05.635 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:42:05.635 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:42:05.635 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:42:05.636 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:42:05.636 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.837924+0000", 2022-08-01T19:42:05.636 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:42:05.637 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.637 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.637 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:42:05.638 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:42:05.638 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:42:05.638 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:42:05.639 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:42:05.639 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:42:05.639 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.839948+0000", 2022-08-01T19:42:05.640 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:42:05.640 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.640 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.641 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:42:05.641 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:42:05.641 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:42:05.642 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:42:05.642 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:42:05.643 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:42:05.643 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.112801+0000", 2022-08-01T19:42:05.643 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:42:05.643 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.644 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.644 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:42:05.645 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:42:05.645 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:42:05.646 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:42:05.646 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:42:05.647 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:42:05.647 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.106704+0000", 2022-08-01T19:42:05.649 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:42:05.650 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.650 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.651 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:42:05.653 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:42:05.653 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:42:05.654 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:42:05.654 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:42:05.654 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:42:05.655 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.893170+0000", 2022-08-01T19:42:05.655 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:42:05.655 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.655 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:05.656 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:42:05.656 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:42:05.656 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:42:05.656 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:42:05.656 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:42:05.657 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:42:05.658 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.886947+0000", 2022-08-01T19:42:05.658 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:42:05.658 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.659 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:42:05.659 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap": [], 2022-08-01T19:42:05.659 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap_items": [], 2022-08-01T19:42:05.659 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_temp": [], 2022-08-01T19:42:05.660 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_temp": [], 2022-08-01T19:42:05.660 INFO:teuthology.orchestra.run.smithi017.stdout: "blacklist": { 2022-08-01T19:42:05.660 INFO:teuthology.orchestra.run.smithi017.stdout: "172.21.15.17:6835/2991628239": "2022-08-02T19:41:58.546544+0000", 2022-08-01T19:42:05.660 INFO:teuthology.orchestra.run.smithi017.stdout: "172.21.15.17:6834/2991628239": "2022-08-02T19:41:58.546544+0000" 2022-08-01T19:42:05.660 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.661 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profiles": { 2022-08-01T19:42:05.662 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_data": { 2022-08-01T19:42:05.662 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-device-class": "", 2022-08-01T19:42:05.662 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:42:05.662 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-root": "default", 2022-08-01T19:42:05.663 INFO:teuthology.orchestra.run.smithi017.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T19:42:05.663 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:42:05.663 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "2", 2022-08-01T19:42:05.663 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:42:05.664 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:42:05.664 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van", 2022-08-01T19:42:05.664 INFO:teuthology.orchestra.run.smithi017.stdout: "w": "8" 2022-08-01T19:42:05.664 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.665 INFO:teuthology.orchestra.run.smithi017.stdout: "default": { 2022-08-01T19:42:05.665 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:42:05.665 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:42:05.665 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "1", 2022-08-01T19:42:05.665 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:42:05.666 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:42:05.666 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van" 2022-08-01T19:42:05.667 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:05.667 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:05.667 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps_queue": [], 2022-08-01T19:42:05.668 INFO:teuthology.orchestra.run.smithi017.stdout: "new_removed_snaps": [], 2022-08-01T19:42:05.668 INFO:teuthology.orchestra.run.smithi017.stdout: "new_purged_snaps": [], 2022-08-01T19:42:05.668 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_node_flags": {}, 2022-08-01T19:42:05.668 INFO:teuthology.orchestra.run.smithi017.stdout: "device_class_flags": {} 2022-08-01T19:42:05.669 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:42:05.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.294+0000 7fac1d1a2700 1 -- 172.21.15.17:0/1801555398 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fabf8053950 msgr2=0x7fabf8055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:05.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.294+0000 7fac1d1a2700 1 --2- 172.21.15.17:0/1801555398 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fabf8053950 0x7fabf8055e10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7fac0800b000 tx=0x7fac08006df0).stop 2022-08-01T19:42:05.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.294+0000 7fac1d1a2700 1 -- 172.21.15.17:0/1801555398 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac180b0e60 msgr2=0x7fac180ad0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:05.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.294+0000 7fac1d1a2700 1 --2- 172.21.15.17:0/1801555398 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac180b0e60 0x7fac180ad0d0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 rx=0x7fac18067700 tx=0x7fac0c007c40).stop 2022-08-01T19:42:05.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.294+0000 7fac1d1a2700 1 -- 172.21.15.17:0/1801555398 shutdown_connections 2022-08-01T19:42:05.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.294+0000 7fac1d1a2700 1 --2- 172.21.15.17:0/1801555398 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fabf8053950 0x7fabf8055e10 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:05.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.294+0000 7fac1d1a2700 1 --2- 172.21.15.17:0/1801555398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac1813d9b0 0x7fac180ae240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:05.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.294+0000 7fac1d1a2700 1 --2- 172.21.15.17:0/1801555398 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac180b6510 0x7fac180add50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:05.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.294+0000 7fac1d1a2700 1 --2- 172.21.15.17:0/1801555398 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac180b0e60 0x7fac180ad0d0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:05.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.294+0000 7fac1d1a2700 1 -- 172.21.15.17:0/1801555398 >> 172.21.15.17:0/1801555398 conn(0x7fac18063800 msgr2=0x7fac1805faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:05.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.294+0000 7fac1d1a2700 1 -- 172.21.15.17:0/1801555398 shutdown_connections 2022-08-01T19:42:05.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.294+0000 7fac1d1a2700 1 -- 172.21.15.17:0/1801555398 wait complete. 2022-08-01T19:42:05.674 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:42:05.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.786+0000 7fb614a1c700 1 Processor -- start 2022-08-01T19:42:05.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.786+0000 7fb614a1c700 1 -- start start 2022-08-01T19:42:05.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.786+0000 7fb614a1c700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fb6100b2710 0x7fb61014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:05.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.786+0000 7fb614a1c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb6100b01a0 0x7fb61014b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:05.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.786+0000 7fb614a1c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb6100afd60 0x7fb6100a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:05.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.786+0000 7fb614a1c700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6100a7250 con 0x7fb6100b01a0 2022-08-01T19:42:05.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.786+0000 7fb614a1c700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6100a7480 con 0x7fb6100b2710 2022-08-01T19:42:05.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.786+0000 7fb614a1c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6100a7720 con 0x7fb6100afd60 2022-08-01T19:42:05.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.786+0000 7fb60e59c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb6100b01a0 0x7fb61014b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:05.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.786+0000 7fb60e59c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb6100b01a0 0x7fb61014b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43140/0 (socket says 172.21.15.17:43140) 2022-08-01T19:42:05.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.786+0000 7fb60e59c700 1 -- 172.21.15.17:0/1408032337 learned_addr learned my addr 172.21.15.17:0/1408032337 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:05.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.786+0000 7fb60f59e700 1 --2- 172.21.15.17:0/1408032337 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb6100afd60 0x7fb6100a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:05.810 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.786+0000 7fb60ed9d700 1 --2- 172.21.15.17:0/1408032337 >> v2:172.21.15.77:3300/0 conn(0x7fb6100b2710 0x7fb61014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:05.810 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.786+0000 7fb60e59c700 1 -- 172.21.15.17:0/1408032337 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb6100afd60 msgr2=0x7fb6100a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:05.810 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.786+0000 7fb60e59c700 1 --2- 172.21.15.17:0/1408032337 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb6100afd60 0x7fb6100a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:05.810 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.786+0000 7fb60e59c700 1 -- 172.21.15.17:0/1408032337 >> v2:172.21.15.77:3300/0 conn(0x7fb6100b2710 msgr2=0x7fb61014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:05.811 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.786+0000 7fb60e59c700 1 --2- 172.21.15.17:0/1408032337 >> v2:172.21.15.77:3300/0 conn(0x7fb6100b2710 0x7fb61014afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:05.811 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.786+0000 7fb60e59c700 1 -- 172.21.15.17:0/1408032337 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6100a79c0 con 0x7fb6100b01a0 2022-08-01T19:42:05.811 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.786+0000 7fb60ed9d700 1 --2- 172.21.15.17:0/1408032337 >> v2:172.21.15.77:3300/0 conn(0x7fb6100b2710 0x7fb61014afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:42:05.811 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.786+0000 7fb60e59c700 1 --2- 172.21.15.17:0/1408032337 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6100b01a0 0x7fb61014b4c0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 rx=0x7fb6000032f0 tx=0x7fb600009290).ready entity=mon.0 client_cookie=8971d9f27d727020 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:05.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.786+0000 7fb60dd9b700 1 -- 172.21.15.17:0/1408032337 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb60000b870 con 0x7fb6100b01a0 2022-08-01T19:42:05.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.786+0000 7fb60dd9b700 1 -- 172.21.15.17:0/1408032337 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb60000b9d0 con 0x7fb6100b01a0 2022-08-01T19:42:05.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.786+0000 7fb60dd9b700 1 -- 172.21.15.17:0/1408032337 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb60000bb40 con 0x7fb6100b01a0 2022-08-01T19:42:05.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.786+0000 7fb614a1c700 1 -- 172.21.15.17:0/1408032337 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6100b01a0 msgr2=0x7fb61014b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:05.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.786+0000 7fb614a1c700 1 --2- 172.21.15.17:0/1408032337 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6100b01a0 0x7fb61014b4c0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 rx=0x7fb6000032f0 tx=0x7fb600009290).stop 2022-08-01T19:42:05.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.786+0000 7fb614a1c700 1 -- 172.21.15.17:0/1408032337 shutdown_connections 2022-08-01T19:42:05.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.786+0000 7fb614a1c700 1 --2- 172.21.15.17:0/1408032337 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb6100afd60 0x7fb6100a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:05.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.786+0000 7fb614a1c700 1 --2- 172.21.15.17:0/1408032337 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6100b01a0 0x7fb61014b4c0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:05.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.786+0000 7fb614a1c700 1 --2- 172.21.15.17:0/1408032337 >> v2:172.21.15.77:3300/0 conn(0x7fb6100b2710 0x7fb61014afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:05.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.786+0000 7fb614a1c700 1 -- 172.21.15.17:0/1408032337 >> 172.21.15.17:0/1408032337 conn(0x7fb610060310 msgr2=0x7fb6100b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:05.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.786+0000 7fb614a1c700 1 -- 172.21.15.17:0/1408032337 shutdown_connections 2022-08-01T19:42:05.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.786+0000 7fb614a1c700 1 -- 172.21.15.17:0/1408032337 wait complete. 2022-08-01T19:42:05.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.790+0000 7fb614a1c700 1 Processor -- start 2022-08-01T19:42:05.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.790+0000 7fb614a1c700 1 -- start start 2022-08-01T19:42:05.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.790+0000 7fb614a1c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6100afd60 0x7fb6100ac0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:05.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.790+0000 7fb614a1c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb6100b01a0 0x7fb6100b3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:05.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.790+0000 7fb614a1c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6100b2710 0x7fb6100ab2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:05.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.790+0000 7fb614a1c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6100ac6f0 con 0x7fb6100b2710 2022-08-01T19:42:05.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.790+0000 7fb614a1c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6100b52e0 con 0x7fb6100afd60 2022-08-01T19:42:05.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.790+0000 7fb614a1c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6100b5580 con 0x7fb6100b01a0 2022-08-01T19:42:05.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.790+0000 7fb60f59e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6100b2710 0x7fb6100ab2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:05.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.790+0000 7fb60e59c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb6100b01a0 0x7fb6100b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:05.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.790+0000 7fb60f59e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6100b2710 0x7fb6100ab2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43150/0 (socket says 172.21.15.17:43150) 2022-08-01T19:42:05.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.790+0000 7fb60f59e700 1 -- 172.21.15.17:0/223032042 learned_addr learned my addr 172.21.15.17:0/223032042 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:05.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.790+0000 7fb60ed9d700 1 --2- 172.21.15.17:0/223032042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6100afd60 0x7fb6100ac0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:05.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.790+0000 7fb60f59e700 1 -- 172.21.15.17:0/223032042 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb6100b01a0 msgr2=0x7fb6100b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:05.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.790+0000 7fb60f59e700 1 --2- 172.21.15.17:0/223032042 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb6100b01a0 0x7fb6100b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:05.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.790+0000 7fb60f59e700 1 -- 172.21.15.17:0/223032042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6100afd60 msgr2=0x7fb6100ac0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:05.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.790+0000 7fb60f59e700 1 --2- 172.21.15.17:0/223032042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6100afd60 0x7fb6100ac0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:05.834 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.790+0000 7fb60f59e700 1 -- 172.21.15.17:0/223032042 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb600009530 con 0x7fb6100b2710 2022-08-01T19:42:05.834 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.790+0000 7fb60f59e700 1 --2- 172.21.15.17:0/223032042 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6100b2710 0x7fb6100ab2b0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7fb60800a000 tx=0x7fb608008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:05.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.790+0000 7fb607fff700 1 -- 172.21.15.17:0/223032042 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb608011780 con 0x7fb6100b2710 2022-08-01T19:42:05.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.790+0000 7fb607fff700 1 -- 172.21.15.17:0/223032042 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6080118e0 con 0x7fb6100b2710 2022-08-01T19:42:05.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.790+0000 7fb614a1c700 1 -- 172.21.15.17:0/223032042 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6100b5820 con 0x7fb6100b2710 2022-08-01T19:42:05.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.790+0000 7fb607fff700 1 -- 172.21.15.17:0/223032042 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb608011c40 con 0x7fb6100b2710 2022-08-01T19:42:05.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.790+0000 7fb614a1c700 1 -- 172.21.15.17:0/223032042 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb61014d550 con 0x7fb6100b2710 2022-08-01T19:42:05.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.794+0000 7fb614a1c700 1 -- 172.21.15.17:0/223032042 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6100b59b0 con 0x7fb6100b2710 2022-08-01T19:42:05.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.794+0000 7fb607fff700 1 -- 172.21.15.17:0/223032042 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb608012420 con 0x7fb6100b2710 2022-08-01T19:42:05.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.794+0000 7fb607fff700 1 --2- 172.21.15.17:0/223032042 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb5f4053b30 0x7fb5f4055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:05.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.794+0000 7fb607fff700 1 -- 172.21.15.17:0/223032042 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fb60806bb80 con 0x7fb6100b2710 2022-08-01T19:42:05.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.794+0000 7fb60ed9d700 1 --2- 172.21.15.17:0/223032042 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb5f4053b30 0x7fb5f4055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:05.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.794+0000 7fb60ed9d700 1 --2- 172.21.15.17:0/223032042 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb5f4053b30 0x7fb5f4055ff0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7fb5f800c000 tx=0x7fb5f8008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:05.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.802+0000 7fb607fff700 1 -- 172.21.15.17:0/223032042 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb608042ab0 con 0x7fb6100b2710 2022-08-01T19:42:05.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.970+0000 7fb614a1c700 1 -- 172.21.15.17:0/223032042 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6100574e0 con 0x7fb6100b2710 2022-08-01T19:42:05.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.970+0000 7fb607fff700 1 -- 172.21.15.17:0/223032042 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+1559 (secure 0 0 0) 0x7fb60806be60 con 0x7fb6100b2710 2022-08-01T19:42:05.975 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:42:05.975 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":55,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6602,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2562202884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2562202884},{"type":"v1","addr":"172.21.15.17:6835","nonce":2562202884}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":55},{"gid":7252,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/1455566688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1455566688},{"type":"v1","addr":"172.21.15.77:6835","nonce":1455566688}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":51},{"gid":7266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.17:6833/249859043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":249859043},{"type":"v1","addr":"172.21.15.17:6833","nonce":249859043}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":51}],"filesystems":[]} 2022-08-01T19:42:05.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.970+0000 7fb614a1c700 1 -- 172.21.15.17:0/223032042 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb5f4053b30 msgr2=0x7fb5f4055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:05.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.970+0000 7fb614a1c700 1 --2- 172.21.15.17:0/223032042 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb5f4053b30 0x7fb5f4055ff0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7fb5f800c000 tx=0x7fb5f8008040).stop 2022-08-01T19:42:05.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.970+0000 7fb614a1c700 1 -- 172.21.15.17:0/223032042 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6100b2710 msgr2=0x7fb6100ab2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:05.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.970+0000 7fb614a1c700 1 --2- 172.21.15.17:0/223032042 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6100b2710 0x7fb6100ab2b0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7fb60800a000 tx=0x7fb608008040).stop 2022-08-01T19:42:05.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.970+0000 7fb614a1c700 1 -- 172.21.15.17:0/223032042 shutdown_connections 2022-08-01T19:42:05.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.970+0000 7fb614a1c700 1 --2- 172.21.15.17:0/223032042 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb5f4053b30 0x7fb5f4055ff0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:05.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.970+0000 7fb614a1c700 1 --2- 172.21.15.17:0/223032042 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6100b2710 0x7fb6100ab2b0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:05.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.970+0000 7fb614a1c700 1 --2- 172.21.15.17:0/223032042 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb6100b01a0 0x7fb6100b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:05.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.970+0000 7fb614a1c700 1 --2- 172.21.15.17:0/223032042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6100afd60 0x7fb6100ac0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:05.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.970+0000 7fb614a1c700 1 -- 172.21.15.17:0/223032042 >> 172.21.15.17:0/223032042 conn(0x7fb610060310 msgr2=0x7fb610148850 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:05.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.970+0000 7fb614a1c700 1 -- 172.21.15.17:0/223032042 shutdown_connections 2022-08-01T19:42:05.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:05.970+0000 7fb614a1c700 1 -- 172.21.15.17:0/223032042 wait complete. 2022-08-01T19:42:05.981 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 55 2022-08-01T19:42:05.986 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:42:06.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.086+0000 7ff273635700 1 Processor -- start 2022-08-01T19:42:06.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.086+0000 7ff273635700 1 -- start start 2022-08-01T19:42:06.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.090+0000 7ff273635700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff26c148530 0x7ff26c0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:06.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.090+0000 7ff273635700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff26c0b04e0 con 0x7ff26c0a7250 2022-08-01T19:42:06.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.090+0000 7ff273635700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff26c0b0780 con 0x7ff26c0b1f40 2022-08-01T19:42:06.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.090+0000 7ff273635700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff26c0a9250 con 0x7ff26c148530 2022-08-01T19:42:06.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.090+0000 7ff270bd0700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff26c148530 0x7ff26c0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:06.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.090+0000 7ff271bd2700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff26c0a7250 0x7ff26c1448b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35352/0 (socket says 172.21.15.17:35352) 2022-08-01T19:42:06.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.090+0000 7ff271bd2700 1 -- 172.21.15.17:0/2325680629 learned_addr learned my addr 172.21.15.17:0/2325680629 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:06.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.090+0000 7ff263fff700 1 -- 172.21.15.17:0/2325680629 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 721675696 0 0) 0x7ff26c0b04e0 con 0x7ff26c0a7250 2022-08-01T19:42:06.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.090+0000 7ff263fff700 1 -- 172.21.15.17:0/2325680629 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff250003260 con 0x7ff26c0a7250 2022-08-01T19:42:06.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.090+0000 7ff270bd0700 1 -- 172.21.15.17:0/2325680629 >> v1:172.21.15.77:6789/0 conn(0x7ff26c0b1f40 legacy=0x7ff26c0b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:06.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.090+0000 7ff270bd0700 1 -- 172.21.15.17:0/2325680629 >> v1:172.21.15.17:6789/0 conn(0x7ff26c0a7250 legacy=0x7ff26c1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:06.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.090+0000 7ff263fff700 1 -- 172.21.15.17:0/2325680629 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2850552702 0 0) 0x7ff250003260 con 0x7ff26c0a7250 2022-08-01T19:42:06.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.090+0000 7ff270bd0700 1 -- 172.21.15.17:0/2325680629 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26c0b0a20 con 0x7ff26c148530 2022-08-01T19:42:06.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.090+0000 7ff270bd0700 1 --2- 172.21.15.17:0/2325680629 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff26c148530 0x7ff26c0b1160 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 rx=0x7ff264001b80 tx=0x7ff268006df0).ready entity=mon.0 client_cookie=a0d24c6319e36fc5 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:06.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.090+0000 7ff263fff700 1 -- 172.21.15.17:0/2325680629 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2640019e0 con 0x7ff26c148530 2022-08-01T19:42:06.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.090+0000 7ff263fff700 1 -- 172.21.15.17:0/2325680629 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff268000e90 con 0x7ff26c148530 2022-08-01T19:42:06.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.090+0000 7ff263fff700 1 -- 172.21.15.17:0/2325680629 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff26800d4f0 con 0x7ff26c148530 2022-08-01T19:42:06.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.090+0000 7ff273635700 1 -- 172.21.15.17:0/2325680629 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff26c148530 msgr2=0x7ff26c0b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:06.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.090+0000 7ff273635700 1 --2- 172.21.15.17:0/2325680629 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff26c148530 0x7ff26c0b1160 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 rx=0x7ff264001b80 tx=0x7ff268006df0).stop 2022-08-01T19:42:06.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.090+0000 7ff273635700 1 -- 172.21.15.17:0/2325680629 shutdown_connections 2022-08-01T19:42:06.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.090+0000 7ff273635700 1 --2- 172.21.15.17:0/2325680629 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff26c148530 0x7ff26c0b1160 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:06.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.090+0000 7ff273635700 1 -- 172.21.15.17:0/2325680629 >> 172.21.15.17:0/2325680629 conn(0x7ff26c05f840 msgr2=0x7ff26c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:06.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.090+0000 7ff273635700 1 -- 172.21.15.17:0/2325680629 shutdown_connections 2022-08-01T19:42:06.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.090+0000 7ff273635700 1 -- 172.21.15.17:0/2325680629 wait complete. 2022-08-01T19:42:06.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.094+0000 7ff273635700 1 Processor -- start 2022-08-01T19:42:06.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.094+0000 7ff273635700 1 -- start start 2022-08-01T19:42:06.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.094+0000 7ff273635700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff26c0a7250 0x7ff26c0aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:06.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.094+0000 7ff273635700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff26c0b1f40 0x7ff26c14b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:06.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.094+0000 7ff273635700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff26c148530 0x7ff26c148bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:06.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.094+0000 7ff273635700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c156d10 con 0x7ff26c148530 2022-08-01T19:42:06.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.094+0000 7ff273635700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c156fb0 con 0x7ff26c0b1f40 2022-08-01T19:42:06.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.094+0000 7ff273635700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c157250 con 0x7ff26c0a7250 2022-08-01T19:42:06.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.094+0000 7ff271bd2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff26c148530 0x7ff26c148bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:06.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.094+0000 7ff271bd2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff26c148530 0x7ff26c148bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43164/0 (socket says 172.21.15.17:43164) 2022-08-01T19:42:06.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.094+0000 7ff271bd2700 1 -- 172.21.15.17:0/3145334709 learned_addr learned my addr 172.21.15.17:0/3145334709 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:06.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.098+0000 7ff2713d1700 1 --2- 172.21.15.17:0/3145334709 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff26c0a7250 0x7ff26c0aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:06.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.098+0000 7ff270bd0700 1 --2- 172.21.15.17:0/3145334709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff26c0b1f40 0x7ff26c14b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:06.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.098+0000 7ff271bd2700 1 -- 172.21.15.17:0/3145334709 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff26c0a7250 msgr2=0x7ff26c0aeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:06.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.098+0000 7ff271bd2700 1 --2- 172.21.15.17:0/3145334709 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff26c0a7250 0x7ff26c0aeae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:06.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.098+0000 7ff271bd2700 1 -- 172.21.15.17:0/3145334709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff26c0b1f40 msgr2=0x7ff26c14b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:06.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.098+0000 7ff271bd2700 1 --2- 172.21.15.17:0/3145334709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff26c0b1f40 0x7ff26c14b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:06.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.098+0000 7ff271bd2700 1 -- 172.21.15.17:0/3145334709 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26800b180 con 0x7ff26c148530 2022-08-01T19:42:06.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.098+0000 7ff271bd2700 1 --2- 172.21.15.17:0/3145334709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff26c148530 0x7ff26c148bd0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 rx=0x7ff264001c40 tx=0x7ff2640021c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:06.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.098+0000 7ff2627fc700 1 -- 172.21.15.17:0/3145334709 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff264004ab0 con 0x7ff26c148530 2022-08-01T19:42:06.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.098+0000 7ff2627fc700 1 -- 172.21.15.17:0/3145334709 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff264004c10 con 0x7ff26c148530 2022-08-01T19:42:06.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.098+0000 7ff273635700 1 -- 172.21.15.17:0/3145334709 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff26c1574f0 con 0x7ff26c148530 2022-08-01T19:42:06.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.098+0000 7ff2627fc700 1 -- 172.21.15.17:0/3145334709 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff264007170 con 0x7ff26c148530 2022-08-01T19:42:06.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.098+0000 7ff273635700 1 -- 172.21.15.17:0/3145334709 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff26c156920 con 0x7ff26c148530 2022-08-01T19:42:06.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.098+0000 7ff273635700 1 -- 172.21.15.17:0/3145334709 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff26c157680 con 0x7ff26c148530 2022-08-01T19:42:06.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.098+0000 7ff2627fc700 1 -- 172.21.15.17:0/3145334709 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff26401b070 con 0x7ff26c148530 2022-08-01T19:42:06.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.102+0000 7ff2627fc700 1 --2- 172.21.15.17:0/3145334709 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff254053a70 0x7ff254055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:06.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.102+0000 7ff2627fc700 1 -- 172.21.15.17:0/3145334709 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5686+0+0 (secure 0 0 0) 0x7ff26406b5f0 con 0x7ff26c148530 2022-08-01T19:42:06.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.102+0000 7ff2713d1700 1 --2- 172.21.15.17:0/3145334709 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff254053a70 0x7ff254055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:06.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.102+0000 7ff2713d1700 1 --2- 172.21.15.17:0/3145334709 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff254053a70 0x7ff254055f30 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7ff25c00d000 tx=0x7ff25c007b90).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:06.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.106+0000 7ff2627fc700 1 -- 172.21.15.17:0/3145334709 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff264042630 con 0x7ff26c148530 2022-08-01T19:42:06.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.266+0000 7ff273635700 1 -- 172.21.15.17:0/3145334709 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff26c0b0cc0 con 0x7ff26c148530 2022-08-01T19:42:06.271 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.266+0000 7ff2627fc700 1 -- 172.21.15.17:0/3145334709 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+1559 (secure 0 0 0) 0x7ff264045c50 con 0x7ff26c148530 2022-08-01T19:42:06.271 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:42:06.272 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":55,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6602,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2562202884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2562202884},{"type":"v1","addr":"172.21.15.17:6835","nonce":2562202884}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":55},{"gid":7252,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/1455566688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1455566688},{"type":"v1","addr":"172.21.15.77:6835","nonce":1455566688}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":51},{"gid":7266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.17:6833/249859043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":249859043},{"type":"v1","addr":"172.21.15.17:6833","nonce":249859043}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":51}],"filesystems":[]} 2022-08-01T19:42:06.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.270+0000 7ff273635700 1 -- 172.21.15.17:0/3145334709 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff254053a70 msgr2=0x7ff254055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:06.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.270+0000 7ff273635700 1 --2- 172.21.15.17:0/3145334709 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff254053a70 0x7ff254055f30 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7ff25c00d000 tx=0x7ff25c007b90).stop 2022-08-01T19:42:06.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.270+0000 7ff273635700 1 -- 172.21.15.17:0/3145334709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff26c148530 msgr2=0x7ff26c148bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:06.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.270+0000 7ff273635700 1 --2- 172.21.15.17:0/3145334709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff26c148530 0x7ff26c148bd0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 rx=0x7ff264001c40 tx=0x7ff2640021c0).stop 2022-08-01T19:42:06.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.270+0000 7ff273635700 1 -- 172.21.15.17:0/3145334709 shutdown_connections 2022-08-01T19:42:06.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.270+0000 7ff273635700 1 --2- 172.21.15.17:0/3145334709 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff254053a70 0x7ff254055f30 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:06.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.270+0000 7ff273635700 1 --2- 172.21.15.17:0/3145334709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff26c148530 0x7ff26c148bd0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:06.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.270+0000 7ff273635700 1 --2- 172.21.15.17:0/3145334709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff26c0b1f40 0x7ff26c14b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:06.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.270+0000 7ff273635700 1 --2- 172.21.15.17:0/3145334709 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff26c0a7250 0x7ff26c0aeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:06.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.270+0000 7ff273635700 1 -- 172.21.15.17:0/3145334709 >> 172.21.15.17:0/3145334709 conn(0x7ff26c05f840 msgr2=0x7ff26c147900 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:06.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.270+0000 7ff273635700 1 -- 172.21.15.17:0/3145334709 shutdown_connections 2022-08-01T19:42:06.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.270+0000 7ff273635700 1 -- 172.21.15.17:0/3145334709 wait complete. 2022-08-01T19:42:06.294 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 55 2022-08-01T19:42:06.295 INFO:tasks.ceph.mds.a:Restarting daemon 2022-08-01T19:42:06.296 INFO:tasks.ceph.mds.a:Stopping old one... 2022-08-01T19:42:06.297 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-08-01T19:42:06.297 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:42:06.298 INFO:tasks.ceph.mds.c:Restarting daemon 2022-08-01T19:42:06.299 INFO:tasks.ceph.mds.c:Stopping old one... 2022-08-01T19:42:06.299 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-08-01T19:42:06.300 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:42:06.302 INFO:tasks.ceph.mds.b:Restarting daemon 2022-08-01T19:42:06.302 INFO:tasks.ceph.mds.b:Stopping old one... 2022-08-01T19:42:06.303 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-08-01T19:42:06.332 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:42:06.335 INFO:tasks.ceph.mds.b:Stopped 2022-08-01T19:42:06.453 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-08-01T19:42:06.455 INFO:tasks.ceph.mds.a:Stopped 2022-08-01T19:42:06.455 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-08-01T19:42:06.456 INFO:tasks.ceph.mds.c:Stopped 2022-08-01T19:42:06.457 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-08-01T19:42:06.467 INFO:tasks.ceph.mds.a:Started 2022-08-01T19:42:06.468 INFO:tasks.ceph.mds.c:Started 2022-08-01T19:42:06.468 DEBUG:teuthology.parallel:result is None 2022-08-01T19:42:06.469 DEBUG:teuthology.parallel:result is None 2022-08-01T19:42:06.472 INFO:tasks.ceph.mds.b:Started 2022-08-01T19:42:06.473 DEBUG:teuthology.parallel:result is None 2022-08-01T19:42:06.474 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-08-01T19:42:06.536 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2022-08-01T19:42:06.545 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2022-08-01T19:42:06.551 INFO:tasks.ceph.mds.b.smithi077.stdout:starting mds.b at 2022-08-01T19:42:06.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.578+0000 7f94f5f32700 1 Processor -- start 2022-08-01T19:42:06.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.578+0000 7f94f5f32700 1 -- start start 2022-08-01T19:42:06.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.578+0000 7f94f5f32700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f94f0066f90 0x7f94f00ab770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:06.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.578+0000 7f94f5f32700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f94f0143cc0 0x7f94f014e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:06.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.578+0000 7f94f5f32700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94f0146b10 con 0x7f94f00abc60 2022-08-01T19:42:06.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.578+0000 7f94f5f32700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94f00a99a0 con 0x7f94f0143cc0 2022-08-01T19:42:06.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.578+0000 7f94f5f32700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94f0067370 con 0x7f94f0066f90 2022-08-01T19:42:06.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.578+0000 7f94effff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f94f0143cc0 0x7f94f014e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:06.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.578+0000 7f94eeffd700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f94f00abc60 0x7f94f01435b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35388/0 (socket says 172.21.15.17:35388) 2022-08-01T19:42:06.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.578+0000 7f94effff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f94f0143cc0 0x7f94f014e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43196/0 (socket says 172.21.15.17:43196) 2022-08-01T19:42:06.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.578+0000 7f94eeffd700 1 -- 172.21.15.17:0/3160178004 learned_addr learned my addr 172.21.15.17:0/3160178004 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:06.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.578+0000 7f94ef7fe700 1 --2- 172.21.15.17:0/3160178004 >> v2:172.21.15.77:3300/0 conn(0x7f94f0066f90 0x7f94f00ab770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:06.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.578+0000 7f94effff700 1 -- 172.21.15.17:0/3160178004 >> v2:172.21.15.77:3300/0 conn(0x7f94f0066f90 msgr2=0x7f94f00ab770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:06.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.578+0000 7f94effff700 1 --2- 172.21.15.17:0/3160178004 >> v2:172.21.15.77:3300/0 conn(0x7f94f0066f90 0x7f94f00ab770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:06.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.578+0000 7f94ee7fc700 1 -- 172.21.15.17:0/3160178004 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 96592729 0 0) 0x7f94f0146b10 con 0x7f94f00abc60 2022-08-01T19:42:06.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.578+0000 7f94effff700 1 -- 172.21.15.17:0/3160178004 >> v1:172.21.15.17:6789/0 conn(0x7f94f00abc60 legacy=0x7f94f01435b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:06.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.578+0000 7f94effff700 1 -- 172.21.15.17:0/3160178004 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f00a95e0 con 0x7f94f0143cc0 2022-08-01T19:42:06.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.578+0000 7f94effff700 1 --2- 172.21.15.17:0/3160178004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94f0143cc0 0x7f94f014e060 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 rx=0x7f94e80032f0 tx=0x7f94e80091d0).ready entity=mon.0 client_cookie=151682141d122d10 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:06.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.578+0000 7f94ee7fc700 1 -- 172.21.15.17:0/3160178004 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94e800b680 con 0x7f94f0143cc0 2022-08-01T19:42:06.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.578+0000 7f94ee7fc700 1 -- 172.21.15.17:0/3160178004 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e800b7e0 con 0x7f94f0143cc0 2022-08-01T19:42:06.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.578+0000 7f94ee7fc700 1 -- 172.21.15.17:0/3160178004 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94e800b950 con 0x7f94f0143cc0 2022-08-01T19:42:06.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.578+0000 7f94f5f32700 1 -- 172.21.15.17:0/3160178004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94f0143cc0 msgr2=0x7f94f014e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:06.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.578+0000 7f94f5f32700 1 --2- 172.21.15.17:0/3160178004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94f0143cc0 0x7f94f014e060 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 rx=0x7f94e80032f0 tx=0x7f94e80091d0).stop 2022-08-01T19:42:06.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.578+0000 7f94f5f32700 1 -- 172.21.15.17:0/3160178004 shutdown_connections 2022-08-01T19:42:06.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.578+0000 7f94f5f32700 1 --2- 172.21.15.17:0/3160178004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94f0143cc0 0x7f94f014e060 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:06.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.578+0000 7f94f5f32700 1 --2- 172.21.15.17:0/3160178004 >> v2:172.21.15.77:3300/0 conn(0x7f94f0066f90 0x7f94f00ab770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:06.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.578+0000 7f94f5f32700 1 -- 172.21.15.17:0/3160178004 >> 172.21.15.17:0/3160178004 conn(0x7f94f00b5570 msgr2=0x7f94f00b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:06.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.582+0000 7f94f5f32700 1 -- 172.21.15.17:0/3160178004 shutdown_connections 2022-08-01T19:42:06.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.582+0000 7f94f5f32700 1 -- 172.21.15.17:0/3160178004 wait complete. 2022-08-01T19:42:06.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.582+0000 7f94f5f32700 1 Processor -- start 2022-08-01T19:42:06.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.582+0000 7f94f5f32700 1 -- start start 2022-08-01T19:42:06.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.582+0000 7f94f5f32700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94f0066f90 0x7f94f0149050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:06.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.582+0000 7f94f5f32700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f94f00abc60 0x7f94f0157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:06.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.582+0000 7f94f5f32700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94f0143cc0 0x7f94f00a7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:06.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.582+0000 7f94f5f32700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f0156ac0 con 0x7f94f0143cc0 2022-08-01T19:42:06.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.582+0000 7f94f5f32700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f00af760 con 0x7f94f0066f90 2022-08-01T19:42:06.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.582+0000 7f94f5f32700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f015ab10 con 0x7f94f00abc60 2022-08-01T19:42:06.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.582+0000 7f94effff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94f0143cc0 0x7f94f00a7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:06.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.582+0000 7f94effff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94f0143cc0 0x7f94f00a7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43202/0 (socket says 172.21.15.17:43202) 2022-08-01T19:42:06.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.582+0000 7f94effff700 1 -- 172.21.15.17:0/1228354574 learned_addr learned my addr 172.21.15.17:0/1228354574 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:06.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.582+0000 7f94ef7fe700 1 --2- 172.21.15.17:0/1228354574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94f0066f90 0x7f94f0149050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:06.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.582+0000 7f94eeffd700 1 --2- 172.21.15.17:0/1228354574 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f94f00abc60 0x7f94f0157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:06.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.582+0000 7f94effff700 1 -- 172.21.15.17:0/1228354574 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f94f00abc60 msgr2=0x7f94f0157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:06.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.582+0000 7f94effff700 1 --2- 172.21.15.17:0/1228354574 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f94f00abc60 0x7f94f0157190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:06.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.582+0000 7f94effff700 1 -- 172.21.15.17:0/1228354574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94f0066f90 msgr2=0x7f94f0149050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:06.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.582+0000 7f94effff700 1 --2- 172.21.15.17:0/1228354574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94f0066f90 0x7f94f0149050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:06.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.582+0000 7f94effff700 1 -- 172.21.15.17:0/1228354574 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e8009470 con 0x7f94f0143cc0 2022-08-01T19:42:06.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.586+0000 7f94effff700 1 --2- 172.21.15.17:0/1228354574 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94f0143cc0 0x7f94f00a7250 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 rx=0x7f94e80032f0 tx=0x7f94e8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:06.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.586+0000 7f94ecff9700 1 -- 172.21.15.17:0/1228354574 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94e800b950 con 0x7f94f0143cc0 2022-08-01T19:42:06.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.586+0000 7f94ecff9700 1 -- 172.21.15.17:0/1228354574 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e8021630 con 0x7f94f0143cc0 2022-08-01T19:42:06.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.586+0000 7f94ecff9700 1 -- 172.21.15.17:0/1228354574 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94e800b680 con 0x7f94f0143cc0 2022-08-01T19:42:06.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.586+0000 7f94f5f32700 1 -- 172.21.15.17:0/1228354574 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f0156cf0 con 0x7f94f0143cc0 2022-08-01T19:42:06.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.586+0000 7f94f5f32700 1 -- 172.21.15.17:0/1228354574 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f00aeac0 con 0x7f94f0143cc0 2022-08-01T19:42:06.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.586+0000 7f94f5f32700 1 -- 172.21.15.17:0/1228354574 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94f00aaaa0 con 0x7f94f0143cc0 2022-08-01T19:42:06.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.586+0000 7f94ecff9700 1 -- 172.21.15.17:0/1228354574 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f94e800f070 con 0x7f94f0143cc0 2022-08-01T19:42:06.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.586+0000 7f94ecff9700 1 --2- 172.21.15.17:0/1228354574 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f94d0053a70 0x7f94d0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:06.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.586+0000 7f94ecff9700 1 -- 172.21.15.17:0/1228354574 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f94e806cac0 con 0x7f94f0143cc0 2022-08-01T19:42:06.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.590+0000 7f94ef7fe700 1 --2- 172.21.15.17:0/1228354574 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f94d0053a70 0x7f94d0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:06.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.590+0000 7f94ef7fe700 1 --2- 172.21.15.17:0/1228354574 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f94d0053a70 0x7f94d0055f30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7f94e000b000 tx=0x7f94e0006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:06.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.590+0000 7f94ecff9700 1 -- 172.21.15.17:0/1228354574 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f94e8043a50 con 0x7f94f0143cc0 2022-08-01T19:42:06.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:06.726+0000 7f94f5f32700 1 -- 172.21.15.17:0/1228354574 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f94f00574e0 con 0x7f94f0143cc0 2022-08-01T19:42:07.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.570+0000 7f94ecff9700 1 -- 172.21.15.17:0/1228354574 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v113) v1 ==== 97+0+0 (secure 0 0 0) 0x7f94e8015070 con 0x7f94f0143cc0 2022-08-01T19:42:07.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.570+0000 7f94f5f32700 1 -- 172.21.15.17:0/1228354574 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f94d0053a70 msgr2=0x7f94d0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:07.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.570+0000 7f94f5f32700 1 --2- 172.21.15.17:0/1228354574 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f94d0053a70 0x7f94d0055f30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7f94e000b000 tx=0x7f94e0006df0).stop 2022-08-01T19:42:07.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.570+0000 7f94f5f32700 1 -- 172.21.15.17:0/1228354574 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94f0143cc0 msgr2=0x7f94f00a7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:07.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.570+0000 7f94f5f32700 1 --2- 172.21.15.17:0/1228354574 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94f0143cc0 0x7f94f00a7250 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 rx=0x7f94e80032f0 tx=0x7f94e8016000).stop 2022-08-01T19:42:07.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.570+0000 7f94f5f32700 1 -- 172.21.15.17:0/1228354574 shutdown_connections 2022-08-01T19:42:07.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.570+0000 7f94f5f32700 1 --2- 172.21.15.17:0/1228354574 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f94d0053a70 0x7f94d0055f30 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:07.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.570+0000 7f94f5f32700 1 --2- 172.21.15.17:0/1228354574 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94f0143cc0 0x7f94f00a7250 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:07.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.570+0000 7f94f5f32700 1 --2- 172.21.15.17:0/1228354574 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f94f00abc60 0x7f94f0157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:07.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.570+0000 7f94f5f32700 1 --2- 172.21.15.17:0/1228354574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94f0066f90 0x7f94f0149050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:07.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.574+0000 7f94f5f32700 1 -- 172.21.15.17:0/1228354574 >> 172.21.15.17:0/1228354574 conn(0x7f94f00b5570 msgr2=0x7f94f00b4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:07.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.574+0000 7f94f5f32700 1 -- 172.21.15.17:0/1228354574 shutdown_connections 2022-08-01T19:42:07.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.574+0000 7f94f5f32700 1 -- 172.21.15.17:0/1228354574 wait complete. 2022-08-01T19:42:07.579 INFO:teuthology.orchestra.run.smithi017.stderr: removed all blacklist entries 2022-08-01T19:42:07.591 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-08-01T19:42:07.592 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-08-01T19:42:07.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.702+0000 7fe01a9fe700 1 Processor -- start 2022-08-01T19:42:07.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.702+0000 7fe01a9fe700 1 -- start start 2022-08-01T19:42:07.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.702+0000 7fe01a9fe700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe0140b01a0 0x7fe0140a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:07.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.702+0000 7fe01a9fe700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0140a7250 con 0x7fe0140afd60 2022-08-01T19:42:07.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.702+0000 7fe01a9fe700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0140a7480 con 0x7fe0140b2710 2022-08-01T19:42:07.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.702+0000 7fe01a9fe700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0140a7720 con 0x7fe0140b01a0 2022-08-01T19:42:07.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.702+0000 7fe01a1fd700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe0140afd60 0x7fe014151d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35406/0 (socket says 172.21.15.17:35406) 2022-08-01T19:42:07.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.702+0000 7fe01a1fd700 1 -- 172.21.15.17:0/1992168111 learned_addr learned my addr 172.21.15.17:0/1992168111 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:07.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.702+0000 7fe0191fb700 1 --2- 172.21.15.17:0/1992168111 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe0140b01a0 0x7fe0140a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:07.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.706+0000 7fe0189fa700 1 -- 172.21.15.17:0/1992168111 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 840528292 0 0) 0x7fe0140a7250 con 0x7fe0140afd60 2022-08-01T19:42:07.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.706+0000 7fe0189fa700 1 -- 172.21.15.17:0/1992168111 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdffc003260 con 0x7fe0140afd60 2022-08-01T19:42:07.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.706+0000 7fe0191fb700 1 -- 172.21.15.17:0/1992168111 >> v1:172.21.15.77:6789/0 conn(0x7fe0140b2710 legacy=0x7fe01414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:07.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.706+0000 7fe0191fb700 1 -- 172.21.15.17:0/1992168111 >> v1:172.21.15.17:6789/0 conn(0x7fe0140afd60 legacy=0x7fe014151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:07.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.706+0000 7fe0189fa700 1 -- 172.21.15.17:0/1992168111 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3468592624 0 0) 0x7fe0140a7480 con 0x7fe0140b2710 2022-08-01T19:42:07.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.706+0000 7fe0191fb700 1 -- 172.21.15.17:0/1992168111 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0140a79c0 con 0x7fe0140b01a0 2022-08-01T19:42:07.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.706+0000 7fe0191fb700 1 --2- 172.21.15.17:0/1992168111 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe0140b01a0 0x7fe0140a8540 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7fe00c001b80 tx=0x7fe010006df0).ready entity=mon.2 client_cookie=112deff779135751 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:07.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.706+0000 7fe0189fa700 1 -- 172.21.15.17:0/1992168111 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe00c0019e0 con 0x7fe0140b01a0 2022-08-01T19:42:07.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.706+0000 7fe0189fa700 1 -- 172.21.15.17:0/1992168111 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe010000e90 con 0x7fe0140b01a0 2022-08-01T19:42:07.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.706+0000 7fe01a9fe700 1 -- 172.21.15.17:0/1992168111 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe0140b01a0 msgr2=0x7fe0140a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:07.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.706+0000 7fe01a9fe700 1 --2- 172.21.15.17:0/1992168111 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe0140b01a0 0x7fe0140a8540 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7fe00c001b80 tx=0x7fe010006df0).stop 2022-08-01T19:42:07.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.706+0000 7fe01a9fe700 1 -- 172.21.15.17:0/1992168111 shutdown_connections 2022-08-01T19:42:07.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.706+0000 7fe01a9fe700 1 --2- 172.21.15.17:0/1992168111 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe0140b01a0 0x7fe0140a8540 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:07.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.706+0000 7fe01a9fe700 1 -- 172.21.15.17:0/1992168111 >> 172.21.15.17:0/1992168111 conn(0x7fe014060310 msgr2=0x7fe0140b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:07.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.706+0000 7fe01a9fe700 1 -- 172.21.15.17:0/1992168111 shutdown_connections 2022-08-01T19:42:07.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.706+0000 7fe01a9fe700 1 -- 172.21.15.17:0/1992168111 wait complete. 2022-08-01T19:42:07.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.706+0000 7fe01a9fe700 1 Processor -- start 2022-08-01T19:42:07.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.706+0000 7fe01a9fe700 1 -- start start 2022-08-01T19:42:07.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.710+0000 7fe01a9fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0140afd60 0x7fe0140a95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:07.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.710+0000 7fe01a9fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0140b01a0 0x7fe014066a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:07.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.710+0000 7fe01a9fe700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe0140b2710 0x7fe014066f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:07.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.710+0000 7fe01a9fe700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0140aaa00 con 0x7fe0140b01a0 2022-08-01T19:42:07.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.710+0000 7fe01a9fe700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0140aac80 con 0x7fe0140afd60 2022-08-01T19:42:07.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.710+0000 7fe01a9fe700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0140aaf20 con 0x7fe0140b2710 2022-08-01T19:42:07.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.710+0000 7fe0191fb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0140b01a0 0x7fe014066a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:07.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.710+0000 7fe0191fb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0140b01a0 0x7fe014066a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43216/0 (socket says 172.21.15.17:43216) 2022-08-01T19:42:07.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.710+0000 7fe0191fb700 1 -- 172.21.15.17:0/2638654011 learned_addr learned my addr 172.21.15.17:0/2638654011 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:07.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.710+0000 7fe01a1fd700 1 --2- 172.21.15.17:0/2638654011 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe0140b2710 0x7fe014066f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:07.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.710+0000 7fe0199fc700 1 --2- 172.21.15.17:0/2638654011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0140afd60 0x7fe0140a95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:07.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.710+0000 7fe01a1fd700 1 -- 172.21.15.17:0/2638654011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0140afd60 msgr2=0x7fe0140a95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:07.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.710+0000 7fe01a1fd700 1 --2- 172.21.15.17:0/2638654011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0140afd60 0x7fe0140a95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:07.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.710+0000 7fe01a1fd700 1 -- 172.21.15.17:0/2638654011 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0140b01a0 msgr2=0x7fe014066a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:07.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.710+0000 7fe01a1fd700 1 --2- 172.21.15.17:0/2638654011 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0140b01a0 0x7fe014066a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:07.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.710+0000 7fe01a1fd700 1 -- 172.21.15.17:0/2638654011 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01000b180 con 0x7fe0140b2710 2022-08-01T19:42:07.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.710+0000 7fe0191fb700 1 --2- 172.21.15.17:0/2638654011 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0140b01a0 0x7fe014066a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:42:07.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.710+0000 7fe01a1fd700 1 --2- 172.21.15.17:0/2638654011 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe0140b2710 0x7fe014066f70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fe00c0017b0 tx=0x7fe00c001f10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:07.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.710+0000 7fe002ffd700 1 -- 172.21.15.17:0/2638654011 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe00c00ce70 con 0x7fe0140b2710 2022-08-01T19:42:07.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.710+0000 7fe002ffd700 1 -- 172.21.15.17:0/2638654011 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe00c004ab0 con 0x7fe0140b2710 2022-08-01T19:42:07.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.710+0000 7fe01a9fe700 1 -- 172.21.15.17:0/2638654011 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0140ab1c0 con 0x7fe0140b2710 2022-08-01T19:42:07.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.710+0000 7fe002ffd700 1 -- 172.21.15.17:0/2638654011 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe00c004e10 con 0x7fe0140b2710 2022-08-01T19:42:07.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.710+0000 7fe01a9fe700 1 -- 172.21.15.17:0/2638654011 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0140644e0 con 0x7fe0140b2710 2022-08-01T19:42:07.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.714+0000 7fe01a9fe700 1 -- 172.21.15.17:0/2638654011 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0140574e0 con 0x7fe0140b2710 2022-08-01T19:42:07.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.714+0000 7fe002ffd700 1 -- 172.21.15.17:0/2638654011 <== mon.2 v2:172.21.15.77:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe00c01b060 con 0x7fe0140b2710 2022-08-01T19:42:07.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.714+0000 7fe002ffd700 1 --2- 172.21.15.17:0/2638654011 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdff40539e0 0x7fdff4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:07.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.714+0000 7fe002ffd700 1 -- 172.21.15.17:0/2638654011 <== mon.2 v2:172.21.15.77:3301/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fe00c06bc80 con 0x7fe0140b2710 2022-08-01T19:42:07.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.714+0000 7fe0199fc700 1 --2- 172.21.15.17:0/2638654011 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdff40539e0 0x7fdff4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:07.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.714+0000 7fe0199fc700 1 --2- 172.21.15.17:0/2638654011 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdff40539e0 0x7fdff4055ea0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7fe00400d000 tx=0x7fe004007b90).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:07.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.722+0000 7fe002ffd700 1 -- 172.21.15.17:0/2638654011 <== mon.2 v2:172.21.15.77:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe00c042d80 con 0x7fe0140b2710 2022-08-01T19:42:07.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.890+0000 7fe01a9fe700 1 -- 172.21.15.17:0/2638654011 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe014069a70 con 0x7fe0140b2710 2022-08-01T19:42:07.904 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.902+0000 7fe002ffd700 1 -- 172.21.15.17:0/2638654011 <== mon.2 v2:172.21.15.77:3301/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v31) v1 ==== 80+0+4255 (secure 0 0 0) 0x7fe00c019030 con 0x7fe0140b2710 2022-08-01T19:42:07.905 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:42:07.906 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:42:07.906 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_dump": [ 2022-08-01T19:42:07.907 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:07.907 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.a", 2022-08-01T19:42:07.907 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhijotxORAAVc04rsrlFj0NShsyc6Qbhg==", 2022-08-01T19:42:07.908 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:42:07.909 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:42:07.913 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:42:07.913 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:42:07.913 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:42:07.914 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:07.914 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:07.914 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:07.914 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.b", 2022-08-01T19:42:07.915 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhiud2xBxAAcn6/O5ysdIp/oIbF5Y1dAw==", 2022-08-01T19:42:07.915 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:42:07.915 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:42:07.915 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:42:07.916 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:42:07.916 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:42:07.916 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:07.916 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:07.917 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:07.917 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.c", 2022-08-01T19:42:07.917 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+KuhiMzMbBBAAfdT5mtpYwboNL+7f2B09KQ==", 2022-08-01T19:42:07.917 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:42:07.918 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:42:07.918 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:42:07.918 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:42:07.918 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:42:07.919 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:07.919 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:07.919 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:07.919 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.0", 2022-08-01T19:42:07.920 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhia27/NRAAZXBp+nYelJVJ6Mge9r+3Vg==", 2022-08-01T19:42:07.920 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:42:07.920 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:42:07.920 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:42:07.921 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:42:07.921 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:07.921 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:07.921 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:07.922 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.1", 2022-08-01T19:42:07.922 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCBKuhiOggRGRAAvgA0MxwBJwmpXV6S9DTdjA==", 2022-08-01T19:42:07.922 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:42:07.922 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:42:07.923 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:42:07.923 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:42:07.923 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:07.924 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:07.924 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:07.924 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.2", 2022-08-01T19:42:07.924 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCDKuhi5PgOORAAhcORK4bW2T3dXPeYjcCm+g==", 2022-08-01T19:42:07.925 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:42:07.925 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:42:07.925 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:42:07.925 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:42:07.926 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:07.926 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:07.926 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:07.927 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.3", 2022-08-01T19:42:07.927 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCGKuhiutn6GxAAk2ucLuGbT5YV/R4xXQGh0w==", 2022-08-01T19:42:07.927 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:42:07.927 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:42:07.928 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:42:07.928 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:42:07.928 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:07.929 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:07.929 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:07.929 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.4", 2022-08-01T19:42:07.929 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCLKuhiGHTRHxAArr3TwXA0gsCAAF/Qs38wUQ==", 2022-08-01T19:42:07.930 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:42:07.930 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:42:07.930 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:42:07.930 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:42:07.931 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:07.931 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:07.931 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:07.931 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.5", 2022-08-01T19:42:07.932 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCOKuhiMMDbBBAAvTzBvOcv7pFWm/P/lNqB0w==", 2022-08-01T19:42:07.932 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:42:07.932 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:42:07.932 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:42:07.933 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:42:07.933 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:07.933 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:07.933 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:07.934 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.6", 2022-08-01T19:42:07.934 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCQKuhiG2H3IxAAufUcWqo9ULquxIrUD5vSfA==", 2022-08-01T19:42:07.934 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:42:07.935 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:42:07.935 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:42:07.936 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:42:07.936 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:07.936 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:07.937 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:07.938 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.7", 2022-08-01T19:42:07.938 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCTKuhiU50LChAAXuhrYBbHkk3eKzAPiEJFTg==", 2022-08-01T19:42:07.939 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:42:07.939 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:42:07.940 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:42:07.940 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:42:07.941 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:07.941 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:07.941 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:07.942 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.0", 2022-08-01T19:42:07.942 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhig3EnCxAANM20jSIQf/AKwuRX1qwT/A==", 2022-08-01T19:42:07.942 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:42:07.942 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:42:07.943 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow r", 2022-08-01T19:42:07.943 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T19:42:07.943 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:07.943 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:07.944 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:07.944 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.1", 2022-08-01T19:42:07.944 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+KuhicJR0DhAAivqEAZUhXnSwjVRwWm2bJg==", 2022-08-01T19:42:07.945 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:42:07.945 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:42:07.945 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow r", 2022-08-01T19:42:07.945 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T19:42:07.946 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:07.946 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:07.946 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:07.946 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.admin", 2022-08-01T19:42:07.947 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhiPTTGHxAAgfthCR6RX6j8F1YseFGNeA==", 2022-08-01T19:42:07.947 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:42:07.947 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:42:07.947 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:42:07.948 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:42:07.948 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:42:07.948 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:07.949 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:07.949 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:07.949 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.x", 2022-08-01T19:42:07.949 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9Kuhi2jvCNhAAyUxiGByZ+OZr8Y7dh3Qt5w==", 2022-08-01T19:42:07.950 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:42:07.950 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:42:07.950 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2022-08-01T19:42:07.950 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:42:07.951 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:07.951 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:07.951 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:07.951 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.y", 2022-08-01T19:42:07.952 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhiDJXpMhAA8aek67ydbRgNO0h+miws+w==", 2022-08-01T19:42:07.952 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:42:07.952 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:42:07.952 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2022-08-01T19:42:07.953 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:42:07.953 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:07.953 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:07.953 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:07.954 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:42:07.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.902+0000 7fe01a9fe700 1 -- 172.21.15.17:0/2638654011 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdff40539e0 msgr2=0x7fdff4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:07.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.902+0000 7fe01a9fe700 1 --2- 172.21.15.17:0/2638654011 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdff40539e0 0x7fdff4055ea0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7fe00400d000 tx=0x7fe004007b90).stop 2022-08-01T19:42:07.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.902+0000 7fe01a9fe700 1 -- 172.21.15.17:0/2638654011 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe0140b2710 msgr2=0x7fe014066f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:07.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.902+0000 7fe01a9fe700 1 --2- 172.21.15.17:0/2638654011 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe0140b2710 0x7fe014066f70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fe00c0017b0 tx=0x7fe00c001f10).stop 2022-08-01T19:42:07.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.902+0000 7fe01a9fe700 1 -- 172.21.15.17:0/2638654011 shutdown_connections 2022-08-01T19:42:07.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.902+0000 7fe01a9fe700 1 --2- 172.21.15.17:0/2638654011 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdff40539e0 0x7fdff4055ea0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:07.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.902+0000 7fe01a9fe700 1 --2- 172.21.15.17:0/2638654011 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe0140b2710 0x7fe014066f70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:07.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.902+0000 7fe01a9fe700 1 --2- 172.21.15.17:0/2638654011 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0140b01a0 0x7fe014066a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:07.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.902+0000 7fe01a9fe700 1 --2- 172.21.15.17:0/2638654011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0140afd60 0x7fe0140a95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:07.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.902+0000 7fe01a9fe700 1 -- 172.21.15.17:0/2638654011 >> 172.21.15.17:0/2638654011 conn(0x7fe014060310 msgr2=0x7fe01405cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:07.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.906+0000 7fe01a9fe700 1 -- 172.21.15.17:0/2638654011 shutdown_connections 2022-08-01T19:42:07.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:07.906+0000 7fe01a9fe700 1 -- 172.21.15.17:0/2638654011 wait complete. 2022-08-01T19:42:07.959 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:42:08.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.066+0000 7fb1c3408700 1 Processor -- start 2022-08-01T19:42:08.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.066+0000 7fb1c3408700 1 -- start start 2022-08-01T19:42:08.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.070+0000 7fb1c3408700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb1bc0b01a0 0x7fb1bc0a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:08.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.070+0000 7fb1c3408700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1bc0afd60 0x7fb1bc0a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:08.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.070+0000 7fb1c3408700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1bc0a7250 con 0x7fb1bc0b2710 2022-08-01T19:42:08.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.070+0000 7fb1c3408700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1bc0a7480 con 0x7fb1bc0b01a0 2022-08-01T19:42:08.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.070+0000 7fb1c3408700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc0a7720 con 0x7fb1bc0afd60 2022-08-01T19:42:08.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.070+0000 7fb1c1c05700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb1bc0b01a0 0x7fb1bc0a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:08.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.070+0000 7fb1c1c05700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb1bc0b01a0 0x7fb1bc0a8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43222/0 (socket says 172.21.15.17:43222) 2022-08-01T19:42:08.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.070+0000 7fb1c1c05700 1 -- 172.21.15.17:0/733508876 learned_addr learned my addr 172.21.15.17:0/733508876 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:08.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.070+0000 7fb1c2c07700 1 --2- 172.21.15.17:0/733508876 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1bc0afd60 0x7fb1bc0a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:08.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.070+0000 7fb1c1c05700 1 -- 172.21.15.17:0/733508876 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1bc0afd60 msgr2=0x7fb1bc0a8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:08.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.070+0000 7fb1c1c05700 1 --2- 172.21.15.17:0/733508876 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1bc0afd60 0x7fb1bc0a8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:08.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.070+0000 7fb1c1c05700 1 -- 172.21.15.17:0/733508876 >> v1:172.21.15.77:6789/0 conn(0x7fb1bc0b2710 legacy=0x7fb1bc14afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:08.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.070+0000 7fb1c1c05700 1 -- 172.21.15.17:0/733508876 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1bc0a79c0 con 0x7fb1bc0b01a0 2022-08-01T19:42:08.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.070+0000 7fb1c2c07700 1 --2- 172.21.15.17:0/733508876 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1bc0afd60 0x7fb1bc0a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:42:08.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.070+0000 7fb1c1c05700 1 --2- 172.21.15.17:0/733508876 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1bc0b01a0 0x7fb1bc0a8540 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0x7fb1bc06b950 tx=0x7fb1b8006eb0).ready entity=mon.0 client_cookie=7d695268259f1498 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:08.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.070+0000 7fb1c1404700 1 -- 172.21.15.17:0/733508876 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1b800d570 con 0x7fb1bc0b01a0 2022-08-01T19:42:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.070+0000 7fb1c1404700 1 -- 172.21.15.17:0/733508876 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b8000e90 con 0x7fb1bc0b01a0 2022-08-01T19:42:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.070+0000 7fb1c1404700 1 -- 172.21.15.17:0/733508876 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1b800d8c0 con 0x7fb1bc0b01a0 2022-08-01T19:42:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.070+0000 7fb1c3408700 1 -- 172.21.15.17:0/733508876 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1bc0b01a0 msgr2=0x7fb1bc0a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.070+0000 7fb1c3408700 1 --2- 172.21.15.17:0/733508876 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1bc0b01a0 0x7fb1bc0a8540 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0x7fb1bc06b950 tx=0x7fb1b8006eb0).stop 2022-08-01T19:42:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.070+0000 7fb1c3408700 1 -- 172.21.15.17:0/733508876 shutdown_connections 2022-08-01T19:42:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.070+0000 7fb1c3408700 1 --2- 172.21.15.17:0/733508876 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1bc0afd60 0x7fb1bc0a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:08.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.070+0000 7fb1c3408700 1 --2- 172.21.15.17:0/733508876 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1bc0b01a0 0x7fb1bc0a8540 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:08.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.070+0000 7fb1c3408700 1 -- 172.21.15.17:0/733508876 >> 172.21.15.17:0/733508876 conn(0x7fb1bc060310 msgr2=0x7fb1bc0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:08.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.070+0000 7fb1c3408700 1 -- 172.21.15.17:0/733508876 shutdown_connections 2022-08-01T19:42:08.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.070+0000 7fb1c3408700 1 -- 172.21.15.17:0/733508876 wait complete. 2022-08-01T19:42:08.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.070+0000 7fb1c3408700 1 Processor -- start 2022-08-01T19:42:08.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.070+0000 7fb1c3408700 1 -- start start 2022-08-01T19:42:08.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.074+0000 7fb1c3408700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1bc0afd60 0x7fb1bc157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:08.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.074+0000 7fb1c3408700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1bc0b01a0 0x7fb1bc0634d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:08.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.074+0000 7fb1c3408700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1bc0b2710 0x7fb1bc066f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:08.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.074+0000 7fb1c3408700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc1575e0 con 0x7fb1bc0b01a0 2022-08-01T19:42:08.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.074+0000 7fb1c3408700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc0b58a0 con 0x7fb1bc0afd60 2022-08-01T19:42:08.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.074+0000 7fb1c3408700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc0b5b40 con 0x7fb1bc0b2710 2022-08-01T19:42:08.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.074+0000 7fb1c1c05700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1bc0b01a0 0x7fb1bc0634d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:08.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.074+0000 7fb1c1c05700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1bc0b01a0 0x7fb1bc0634d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43230/0 (socket says 172.21.15.17:43230) 2022-08-01T19:42:08.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.074+0000 7fb1c1c05700 1 -- 172.21.15.17:0/335336062 learned_addr learned my addr 172.21.15.17:0/335336062 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:08.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.074+0000 7fb1c2c07700 1 --2- 172.21.15.17:0/335336062 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1bc0b2710 0x7fb1bc066f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:08.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.074+0000 7fb1c2406700 1 --2- 172.21.15.17:0/335336062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1bc0afd60 0x7fb1bc157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:08.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.074+0000 7fb1c1c05700 1 -- 172.21.15.17:0/335336062 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1bc0b2710 msgr2=0x7fb1bc066f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:08.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.074+0000 7fb1c1c05700 1 --2- 172.21.15.17:0/335336062 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1bc0b2710 0x7fb1bc066f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:08.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.074+0000 7fb1c1c05700 1 -- 172.21.15.17:0/335336062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1bc0afd60 msgr2=0x7fb1bc157050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:08.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.074+0000 7fb1c1c05700 1 --2- 172.21.15.17:0/335336062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1bc0afd60 0x7fb1bc157050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:08.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.074+0000 7fb1c1c05700 1 -- 172.21.15.17:0/335336062 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b800b1e0 con 0x7fb1bc0b01a0 2022-08-01T19:42:08.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.074+0000 7fb1c1c05700 1 --2- 172.21.15.17:0/335336062 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1bc0b01a0 0x7fb1bc0634d0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 rx=0x7fb1bc06b950 tx=0x7fb1b8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:08.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.074+0000 7fb1ab7fe700 1 -- 172.21.15.17:0/335336062 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1b800bec0 con 0x7fb1bc0b01a0 2022-08-01T19:42:08.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.074+0000 7fb1ab7fe700 1 -- 172.21.15.17:0/335336062 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b800d8f0 con 0x7fb1bc0b01a0 2022-08-01T19:42:08.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.074+0000 7fb1c3408700 1 -- 172.21.15.17:0/335336062 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1bc0b5de0 con 0x7fb1bc0b01a0 2022-08-01T19:42:08.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.074+0000 7fb1ab7fe700 1 -- 172.21.15.17:0/335336062 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1b801e460 con 0x7fb1bc0b01a0 2022-08-01T19:42:08.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.074+0000 7fb1c3408700 1 -- 172.21.15.17:0/335336062 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1bc0b4c40 con 0x7fb1bc0b01a0 2022-08-01T19:42:08.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.074+0000 7fb1c3408700 1 -- 172.21.15.17:0/335336062 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1bc0574e0 con 0x7fb1bc0b01a0 2022-08-01T19:42:08.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.074+0000 7fb1ab7fe700 1 -- 172.21.15.17:0/335336062 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb1b8025030 con 0x7fb1bc0b01a0 2022-08-01T19:42:08.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.078+0000 7fb1ab7fe700 1 --2- 172.21.15.17:0/335336062 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb19c053a10 0x7fb19c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:08.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.078+0000 7fb1ab7fe700 1 -- 172.21.15.17:0/335336062 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fb1b806cfe0 con 0x7fb1bc0b01a0 2022-08-01T19:42:08.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.078+0000 7fb1c2406700 1 --2- 172.21.15.17:0/335336062 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb19c053a10 0x7fb19c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:08.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.078+0000 7fb1c2406700 1 --2- 172.21.15.17:0/335336062 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb19c053a10 0x7fb19c055ed0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7fb1ac00a000 tx=0x7fb1ac008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:08.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.082+0000 7fb1ab7fe700 1 -- 172.21.15.17:0/335336062 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb1b80440e0 con 0x7fb1bc0b01a0 2022-08-01T19:42:08.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.206+0000 7fb1c3408700 1 -- 172.21.15.17:0/335336062 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1bc069a70 con 0x7fb1bc0b01a0 2022-08-01T19:42:08.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.206+0000 7fb1ab7fe700 1 -- 172.21.15.17:0/335336062 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+13424 (secure 0 0 0) 0x7fb1b8023020 con 0x7fb1bc0b01a0 2022-08-01T19:42:08.212 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:42:08.213 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":113,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:42:07.563142+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:42:08.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.210+0000 7fb1c3408700 1 -- 172.21.15.17:0/335336062 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb19c053a10 msgr2=0x7fb19c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:08.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.210+0000 7fb1c3408700 1 --2- 172.21.15.17:0/335336062 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb19c053a10 0x7fb19c055ed0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7fb1ac00a000 tx=0x7fb1ac008040).stop 2022-08-01T19:42:08.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.210+0000 7fb1c3408700 1 -- 172.21.15.17:0/335336062 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1bc0b01a0 msgr2=0x7fb1bc0634d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:08.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.210+0000 7fb1c3408700 1 --2- 172.21.15.17:0/335336062 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1bc0b01a0 0x7fb1bc0634d0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 rx=0x7fb1bc06b950 tx=0x7fb1b8015000).stop 2022-08-01T19:42:08.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.210+0000 7fb1c3408700 1 -- 172.21.15.17:0/335336062 shutdown_connections 2022-08-01T19:42:08.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.210+0000 7fb1c3408700 1 --2- 172.21.15.17:0/335336062 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb19c053a10 0x7fb19c055ed0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:08.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.210+0000 7fb1c3408700 1 --2- 172.21.15.17:0/335336062 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1bc0b2710 0x7fb1bc066f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:08.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.210+0000 7fb1c3408700 1 --2- 172.21.15.17:0/335336062 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1bc0b01a0 0x7fb1bc0634d0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:08.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.210+0000 7fb1c3408700 1 --2- 172.21.15.17:0/335336062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1bc0afd60 0x7fb1bc157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:08.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.210+0000 7fb1c3408700 1 -- 172.21.15.17:0/335336062 >> 172.21.15.17:0/335336062 conn(0x7fb1bc060310 msgr2=0x7fb1bc1467b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:08.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.210+0000 7fb1c3408700 1 -- 172.21.15.17:0/335336062 shutdown_connections 2022-08-01T19:42:08.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.210+0000 7fb1c3408700 1 -- 172.21.15.17:0/335336062 wait complete. 2022-08-01T19:42:08.228 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-08-01T19:42:08.229 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:42:08.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.342+0000 7fc59ad36700 1 Processor -- start 2022-08-01T19:42:08.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.342+0000 7fc59ad36700 1 -- start start 2022-08-01T19:42:08.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.342+0000 7fc59ad36700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc594146ae0 0x7fc5941403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:08.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.342+0000 7fc59ad36700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fc5940b4820 0x7fc5940b5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:08.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.342+0000 7fc59ad36700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc594143cf0 con 0x7fc5940b4310 2022-08-01T19:42:08.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.342+0000 7fc59ad36700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5941476a0 con 0x7fc5940b4820 2022-08-01T19:42:08.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.342+0000 7fc59ad36700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc594143f90 con 0x7fc594146ae0 2022-08-01T19:42:08.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.342+0000 7fc593fff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc5940b4310 0x7fc5941408b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35434/0 (socket says 172.21.15.17:35434) 2022-08-01T19:42:08.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.342+0000 7fc593fff700 1 -- 172.21.15.17:0/2854114158 learned_addr learned my addr 172.21.15.17:0/2854114158 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:08.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.342+0000 7fc598ad2700 1 --2- 172.21.15.17:0/2854114158 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc594146ae0 0x7fc5941403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:08.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.342+0000 7fc5992d3700 1 --2- 172.21.15.17:0/2854114158 >> v2:172.21.15.77:3300/0 conn(0x7fc5940b4820 0x7fc5940b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:08.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.342+0000 7fc5937fe700 1 -- 172.21.15.17:0/2854114158 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 874982812 0 0) 0x7fc594143cf0 con 0x7fc5940b4310 2022-08-01T19:42:08.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.342+0000 7fc5937fe700 1 -- 172.21.15.17:0/2854114158 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc578003260 con 0x7fc5940b4310 2022-08-01T19:42:08.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.342+0000 7fc5992d3700 1 -- 172.21.15.17:0/2854114158 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc594146ae0 msgr2=0x7fc5941403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:08.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.342+0000 7fc5992d3700 1 --2- 172.21.15.17:0/2854114158 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc594146ae0 0x7fc5941403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:08.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.342+0000 7fc5992d3700 1 -- 172.21.15.17:0/2854114158 >> v1:172.21.15.17:6789/0 conn(0x7fc5940b4310 legacy=0x7fc5941408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:08.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.342+0000 7fc5992d3700 1 -- 172.21.15.17:0/2854114158 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5940b0e20 con 0x7fc5940b4820 2022-08-01T19:42:08.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.342+0000 7fc598ad2700 1 --2- 172.21.15.17:0/2854114158 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc594146ae0 0x7fc5941403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:42:08.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.342+0000 7fc5992d3700 1 --2- 172.21.15.17:0/2854114158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5940b4820 0x7fc5940b5060 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7fc58c0032f0 tx=0x7fc58c0091d0).ready entity=mon.1 client_cookie=61eb20f01fd4020f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:08.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.342+0000 7fc5937fe700 1 -- 172.21.15.17:0/2854114158 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc58c00b600 con 0x7fc5940b4820 2022-08-01T19:42:08.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.342+0000 7fc5937fe700 1 -- 172.21.15.17:0/2854114158 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc58c00b760 con 0x7fc5940b4820 2022-08-01T19:42:08.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.342+0000 7fc5937fe700 1 -- 172.21.15.17:0/2854114158 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc58c00b8d0 con 0x7fc5940b4820 2022-08-01T19:42:08.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.346+0000 7fc59ad36700 1 -- 172.21.15.17:0/2854114158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5940b4820 msgr2=0x7fc5940b5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:08.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.346+0000 7fc59ad36700 1 --2- 172.21.15.17:0/2854114158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5940b4820 0x7fc5940b5060 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7fc58c0032f0 tx=0x7fc58c0091d0).stop 2022-08-01T19:42:08.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.346+0000 7fc59ad36700 1 -- 172.21.15.17:0/2854114158 shutdown_connections 2022-08-01T19:42:08.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.346+0000 7fc59ad36700 1 --2- 172.21.15.17:0/2854114158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5940b4820 0x7fc5940b5060 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:08.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.346+0000 7fc59ad36700 1 --2- 172.21.15.17:0/2854114158 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc594146ae0 0x7fc5941403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:08.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.346+0000 7fc59ad36700 1 -- 172.21.15.17:0/2854114158 >> 172.21.15.17:0/2854114158 conn(0x7fc59405c620 msgr2=0x7fc59405ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:08.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.346+0000 7fc59ad36700 1 -- 172.21.15.17:0/2854114158 shutdown_connections 2022-08-01T19:42:08.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.346+0000 7fc59ad36700 1 -- 172.21.15.17:0/2854114158 wait complete. 2022-08-01T19:42:08.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.346+0000 7fc59ad36700 1 Processor -- start 2022-08-01T19:42:08.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.346+0000 7fc59ad36700 1 -- start start 2022-08-01T19:42:08.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.346+0000 7fc59ad36700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc5940b4310 0x7fc59413e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:08.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.346+0000 7fc59ad36700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5940b4820 0x7fc59413b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:08.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.346+0000 7fc59ad36700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc594146ae0 0x7fc59413b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:08.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.346+0000 7fc59ad36700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5940aef30 con 0x7fc5940b4820 2022-08-01T19:42:08.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.346+0000 7fc59ad36700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc594146f30 con 0x7fc594146ae0 2022-08-01T19:42:08.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.346+0000 7fc59ad36700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc594141070 con 0x7fc5940b4310 2022-08-01T19:42:08.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.346+0000 7fc593fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5940b4820 0x7fc59413b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:08.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.346+0000 7fc598ad2700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc5940b4310 0x7fc59413e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:08.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.346+0000 7fc593fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5940b4820 0x7fc59413b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43244/0 (socket says 172.21.15.17:43244) 2022-08-01T19:42:08.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.346+0000 7fc593fff700 1 -- 172.21.15.17:0/1964034184 learned_addr learned my addr 172.21.15.17:0/1964034184 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:08.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.346+0000 7fc5992d3700 1 --2- 172.21.15.17:0/1964034184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc594146ae0 0x7fc59413b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:08.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.350+0000 7fc593fff700 1 -- 172.21.15.17:0/1964034184 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc5940b4310 msgr2=0x7fc59413e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:08.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.350+0000 7fc593fff700 1 --2- 172.21.15.17:0/1964034184 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc5940b4310 0x7fc59413e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:08.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.350+0000 7fc593fff700 1 -- 172.21.15.17:0/1964034184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc594146ae0 msgr2=0x7fc59413b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:08.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.350+0000 7fc593fff700 1 --2- 172.21.15.17:0/1964034184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc594146ae0 0x7fc59413b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:08.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.350+0000 7fc593fff700 1 -- 172.21.15.17:0/1964034184 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc58800b180 con 0x7fc5940b4820 2022-08-01T19:42:08.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.350+0000 7fc5992d3700 1 --2- 172.21.15.17:0/1964034184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc594146ae0 0x7fc59413b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:42:08.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.350+0000 7fc598ad2700 1 --2- 172.21.15.17:0/1964034184 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc5940b4310 0x7fc59413e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:42:08.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.350+0000 7fc593fff700 1 --2- 172.21.15.17:0/1964034184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5940b4820 0x7fc59413b010 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7fc580001110 tx=0x7fc580007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:08.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.350+0000 7fc591ffb700 1 -- 172.21.15.17:0/1964034184 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc58000ce50 con 0x7fc5940b4820 2022-08-01T19:42:08.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.350+0000 7fc591ffb700 1 -- 172.21.15.17:0/1964034184 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5800070d0 con 0x7fc5940b4820 2022-08-01T19:42:08.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.350+0000 7fc59ad36700 1 -- 172.21.15.17:0/1964034184 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc58c009470 con 0x7fc5940b4820 2022-08-01T19:42:08.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.350+0000 7fc591ffb700 1 -- 172.21.15.17:0/1964034184 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc580007430 con 0x7fc5940b4820 2022-08-01T19:42:08.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.350+0000 7fc59ad36700 1 -- 172.21.15.17:0/1964034184 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5941427f0 con 0x7fc5940b4820 2022-08-01T19:42:08.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.350+0000 7fc59ad36700 1 -- 172.21.15.17:0/1964034184 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc594155ae0 con 0x7fc5940b4820 2022-08-01T19:42:08.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.350+0000 7fc591ffb700 1 -- 172.21.15.17:0/1964034184 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc58001a060 con 0x7fc5940b4820 2022-08-01T19:42:08.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.350+0000 7fc591ffb700 1 --2- 172.21.15.17:0/1964034184 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc574053a70 0x7fc574055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:08.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.350+0000 7fc591ffb700 1 -- 172.21.15.17:0/1964034184 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fc58006b180 con 0x7fc5940b4820 2022-08-01T19:42:08.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.354+0000 7fc598ad2700 1 --2- 172.21.15.17:0/1964034184 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc574053a70 0x7fc574055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:08.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.354+0000 7fc598ad2700 1 --2- 172.21.15.17:0/1964034184 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc574053a70 0x7fc574055f30 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7fc58800b000 tx=0x7fc588006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:08.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.358+0000 7fc591ffb700 1 -- 172.21.15.17:0/1964034184 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc580042170 con 0x7fc5940b4820 2022-08-01T19:42:08.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.482+0000 7fc59ad36700 1 -- 172.21.15.17:0/1964034184 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc5940574e0 con 0x7fc5940b4820 2022-08-01T19:42:08.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.482+0000 7fc591ffb700 1 -- 172.21.15.17:0/1964034184 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v113) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc580018020 con 0x7fc5940b4820 2022-08-01T19:42:08.488 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:42:08.501 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.486+0000 7fc59ad36700 1 -- 172.21.15.17:0/1964034184 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc574053a70 msgr2=0x7fc574055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:08.501 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.486+0000 7fc59ad36700 1 --2- 172.21.15.17:0/1964034184 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc574053a70 0x7fc574055f30 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7fc58800b000 tx=0x7fc588006df0).stop 2022-08-01T19:42:08.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.486+0000 7fc59ad36700 1 -- 172.21.15.17:0/1964034184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5940b4820 msgr2=0x7fc59413b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:08.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.486+0000 7fc59ad36700 1 --2- 172.21.15.17:0/1964034184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5940b4820 0x7fc59413b010 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7fc580001110 tx=0x7fc580007aa0).stop 2022-08-01T19:42:08.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.486+0000 7fc59ad36700 1 -- 172.21.15.17:0/1964034184 shutdown_connections 2022-08-01T19:42:08.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.486+0000 7fc59ad36700 1 --2- 172.21.15.17:0/1964034184 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc574053a70 0x7fc574055f30 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:08.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.486+0000 7fc59ad36700 1 --2- 172.21.15.17:0/1964034184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc594146ae0 0x7fc59413b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:08.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.486+0000 7fc59ad36700 1 --2- 172.21.15.17:0/1964034184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5940b4820 0x7fc59413b010 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:08.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.486+0000 7fc59ad36700 1 --2- 172.21.15.17:0/1964034184 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc5940b4310 0x7fc59413e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:08.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.486+0000 7fc59ad36700 1 -- 172.21.15.17:0/1964034184 >> 172.21.15.17:0/1964034184 conn(0x7fc59405c620 msgr2=0x7fc5940b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:08.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.486+0000 7fc59ad36700 1 -- 172.21.15.17:0/1964034184 shutdown_connections 2022-08-01T19:42:08.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.486+0000 7fc59ad36700 1 -- 172.21.15.17:0/1964034184 wait complete. 2022-08-01T19:42:08.510 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:42:08.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.638+0000 7f7014fb4700 1 Processor -- start 2022-08-01T19:42:08.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.638+0000 7f7014fb4700 1 -- start start 2022-08-01T19:42:08.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.642+0000 7f7014fb4700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f70100b6510 0x7f7010140500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:08.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.642+0000 7f7014fb4700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70101467f0 con 0x7f70100b2aa0 2022-08-01T19:42:08.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.642+0000 7f7014fb4700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70101476a0 con 0x7f7010061480 2022-08-01T19:42:08.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.642+0000 7f7014fb4700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7010062140 con 0x7f70100b6510 2022-08-01T19:42:08.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.642+0000 7f700f7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f70100b6510 0x7f7010140500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:08.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.642+0000 7f700f7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f70100b6510 0x7f7010140500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43250/0 (socket says 172.21.15.17:43250) 2022-08-01T19:42:08.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.642+0000 7f700f7fe700 1 -- 172.21.15.17:0/2434106200 learned_addr learned my addr 172.21.15.17:0/2434106200 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:08.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.642+0000 7f700effd700 1 --1- 172.21.15.17:0/2434106200 >> v1:172.21.15.17:6789/0 conn(0x7f70100b2aa0 0x7f70100b0e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35448/0 (socket says 172.21.15.17:35448) 2022-08-01T19:42:08.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.642+0000 7f700f7fe700 1 -- 172.21.15.17:0/2434106200 >> v1:172.21.15.77:6789/0 conn(0x7f7010061480 legacy=0x7f7010066830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:08.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.642+0000 7f700f7fe700 1 -- 172.21.15.17:0/2434106200 >> v1:172.21.15.17:6789/0 conn(0x7f70100b2aa0 legacy=0x7f70100b0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:08.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.642+0000 7f700f7fe700 1 -- 172.21.15.17:0/2434106200 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7010146a20 con 0x7f70100b6510 2022-08-01T19:42:08.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.642+0000 7f700f7fe700 1 --2- 172.21.15.17:0/2434106200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70100b6510 0x7f7010140500 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 rx=0x7f6ff8001110 tx=0x7f7000006eb0).ready entity=mon.0 client_cookie=e93a0dbf1fd47794 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:08.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.642+0000 7f700e7fc700 1 -- 172.21.15.17:0/2434106200 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f700000d610 con 0x7f70100b6510 2022-08-01T19:42:08.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.642+0000 7f700e7fc700 1 -- 172.21.15.17:0/2434106200 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7000000e90 con 0x7f70100b6510 2022-08-01T19:42:08.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.642+0000 7f700e7fc700 1 -- 172.21.15.17:0/2434106200 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f700000d9a0 con 0x7f70100b6510 2022-08-01T19:42:08.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.642+0000 7f7014fb4700 1 -- 172.21.15.17:0/2434106200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70100b6510 msgr2=0x7f7010140500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:08.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.642+0000 7f7014fb4700 1 --2- 172.21.15.17:0/2434106200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70100b6510 0x7f7010140500 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 rx=0x7f6ff8001110 tx=0x7f7000006eb0).stop 2022-08-01T19:42:08.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.642+0000 7f7014fb4700 1 -- 172.21.15.17:0/2434106200 shutdown_connections 2022-08-01T19:42:08.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.642+0000 7f7014fb4700 1 --2- 172.21.15.17:0/2434106200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70100b6510 0x7f7010140500 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:08.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.642+0000 7f7014fb4700 1 -- 172.21.15.17:0/2434106200 >> 172.21.15.17:0/2434106200 conn(0x7f7010063800 msgr2=0x7f7010063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:08.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.642+0000 7f7014fb4700 1 -- 172.21.15.17:0/2434106200 shutdown_connections 2022-08-01T19:42:08.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.642+0000 7f7014fb4700 1 -- 172.21.15.17:0/2434106200 wait complete. 2022-08-01T19:42:08.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.646+0000 7f7014fb4700 1 Processor -- start 2022-08-01T19:42:08.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.646+0000 7f7014fb4700 1 -- start start 2022-08-01T19:42:08.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.646+0000 7f7014fb4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7010061480 0x7f7010143aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:08.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.646+0000 7f7014fb4700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70100b2aa0 0x7f7010156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:08.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.646+0000 7f7014fb4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70100b6510 0x7f7010141070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:08.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.646+0000 7f7014fb4700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70100b1b40 con 0x7f70100b6510 2022-08-01T19:42:08.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.646+0000 7f7014fb4700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7010155ad0 con 0x7f7010061480 2022-08-01T19:42:08.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.646+0000 7f7014fb4700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7010156960 con 0x7f70100b2aa0 2022-08-01T19:42:08.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.646+0000 7f700ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70100b6510 0x7f7010141070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:08.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.646+0000 7f700ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70100b6510 0x7f7010141070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43260/0 (socket says 172.21.15.17:43260) 2022-08-01T19:42:08.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.646+0000 7f700ffff700 1 -- 172.21.15.17:0/3141952003 learned_addr learned my addr 172.21.15.17:0/3141952003 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:08.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.646+0000 7f700f7fe700 1 --2- 172.21.15.17:0/3141952003 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7010061480 0x7f7010143aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:08.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.646+0000 7f700effd700 1 --2- 172.21.15.17:0/3141952003 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70100b2aa0 0x7f7010156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:08.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.646+0000 7f700ffff700 1 -- 172.21.15.17:0/3141952003 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70100b2aa0 msgr2=0x7f7010156470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:08.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.646+0000 7f700ffff700 1 --2- 172.21.15.17:0/3141952003 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70100b2aa0 0x7f7010156470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:08.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.646+0000 7f700ffff700 1 -- 172.21.15.17:0/3141952003 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7010061480 msgr2=0x7f7010143aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:08.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.646+0000 7f700ffff700 1 --2- 172.21.15.17:0/3141952003 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7010061480 0x7f7010143aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:08.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.646+0000 7f700ffff700 1 -- 172.21.15.17:0/3141952003 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700000b1e0 con 0x7f70100b6510 2022-08-01T19:42:08.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.646+0000 7f700effd700 1 --2- 172.21.15.17:0/3141952003 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70100b2aa0 0x7f7010156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:42:08.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.646+0000 7f700ffff700 1 --2- 172.21.15.17:0/3141952003 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70100b6510 0x7f7010141070 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 rx=0x7f701006ac40 tx=0x7f7008007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:08.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.646+0000 7f700cff9700 1 -- 172.21.15.17:0/3141952003 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f700800e4a0 con 0x7f70100b6510 2022-08-01T19:42:08.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.646+0000 7f700cff9700 1 -- 172.21.15.17:0/3141952003 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700800e600 con 0x7f70100b6510 2022-08-01T19:42:08.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.646+0000 7f7014fb4700 1 -- 172.21.15.17:0/3141952003 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70100b1d70 con 0x7f70100b6510 2022-08-01T19:42:08.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.646+0000 7f700cff9700 1 -- 172.21.15.17:0/3141952003 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f700800e960 con 0x7f70100b6510 2022-08-01T19:42:08.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.646+0000 7f7014fb4700 1 -- 172.21.15.17:0/3141952003 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7010155fd0 con 0x7f70100b6510 2022-08-01T19:42:08.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.650+0000 7f7014fb4700 1 -- 172.21.15.17:0/3141952003 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7010156190 con 0x7f70100b6510 2022-08-01T19:42:08.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.650+0000 7f700cff9700 1 -- 172.21.15.17:0/3141952003 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7008019030 con 0x7f70100b6510 2022-08-01T19:42:08.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.650+0000 7f700cff9700 1 --2- 172.21.15.17:0/3141952003 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6ff4053b00 0x7f6ff4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:08.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.650+0000 7f700cff9700 1 -- 172.21.15.17:0/3141952003 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f700806b540 con 0x7f70100b6510 2022-08-01T19:42:08.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.650+0000 7f700f7fe700 1 --2- 172.21.15.17:0/3141952003 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6ff4053b00 0x7f6ff4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:08.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.650+0000 7f700f7fe700 1 --2- 172.21.15.17:0/3141952003 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6ff4053b00 0x7f6ff4055fc0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f700001c000 tx=0x7f7000015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:08.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.654+0000 7f700cff9700 1 -- 172.21.15.17:0/3141952003 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7008042450 con 0x7f70100b6510 2022-08-01T19:42:08.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.774+0000 7f7014fb4700 1 -- 172.21.15.17:0/3141952003 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7010146cc0 con 0x7f70100b6510 2022-08-01T19:42:08.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.778+0000 7f700cff9700 1 -- 172.21.15.17:0/3141952003 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v113) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7008045a70 con 0x7f70100b6510 2022-08-01T19:42:08.779 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:42:08.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.778+0000 7f7014fb4700 1 -- 172.21.15.17:0/3141952003 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6ff4053b00 msgr2=0x7f6ff4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:08.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.778+0000 7f7014fb4700 1 --2- 172.21.15.17:0/3141952003 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6ff4053b00 0x7f6ff4055fc0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f700001c000 tx=0x7f7000015000).stop 2022-08-01T19:42:08.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.778+0000 7f7014fb4700 1 -- 172.21.15.17:0/3141952003 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70100b6510 msgr2=0x7f7010141070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:08.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.778+0000 7f7014fb4700 1 --2- 172.21.15.17:0/3141952003 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70100b6510 0x7f7010141070 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 rx=0x7f701006ac40 tx=0x7f7008007f40).stop 2022-08-01T19:42:08.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.778+0000 7f7014fb4700 1 -- 172.21.15.17:0/3141952003 shutdown_connections 2022-08-01T19:42:08.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.778+0000 7f7014fb4700 1 --2- 172.21.15.17:0/3141952003 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6ff4053b00 0x7f6ff4055fc0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:08.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.778+0000 7f7014fb4700 1 --2- 172.21.15.17:0/3141952003 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70100b6510 0x7f7010141070 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:08.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.778+0000 7f7014fb4700 1 --2- 172.21.15.17:0/3141952003 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70100b2aa0 0x7f7010156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:08.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.778+0000 7f7014fb4700 1 --2- 172.21.15.17:0/3141952003 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7010061480 0x7f7010143aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:08.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.778+0000 7f7014fb4700 1 -- 172.21.15.17:0/3141952003 >> 172.21.15.17:0/3141952003 conn(0x7f7010063800 msgr2=0x7f7010140610 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:08.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.778+0000 7f7014fb4700 1 -- 172.21.15.17:0/3141952003 shutdown_connections 2022-08-01T19:42:08.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.782+0000 7f7014fb4700 1 -- 172.21.15.17:0/3141952003 wait complete. 2022-08-01T19:42:08.810 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-08-01T19:42:08.915 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.914+0000 7f3dd0d38700 1 Processor -- start 2022-08-01T19:42:08.916 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.914+0000 7f3dd0d38700 1 -- start start 2022-08-01T19:42:08.946 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.914+0000 7f3dd0d38700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f3dcc0b5ed0 0x7f3dcc0b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:08.947 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.914+0000 7f3dd0d38700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dcc061d90 con 0x7f3dcc0619b0 2022-08-01T19:42:08.948 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.914+0000 7f3dd0d38700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dcc0ba5a0 con 0x7f3dcc0b6ee0 2022-08-01T19:42:08.948 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.914+0000 7f3dd0d38700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dcc143fa0 con 0x7f3dcc0b5ed0 2022-08-01T19:42:08.949 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.914+0000 7f3dcb7fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f3dcc0619b0 0x7f3dcc143890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35458/0 (socket says 172.21.15.17:35458) 2022-08-01T19:42:08.949 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.914+0000 7f3dcb7fe700 1 -- 172.21.15.17:0/354076301 learned_addr learned my addr 172.21.15.17:0/354076301 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:08.949 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.914+0000 7f3dcbfff700 1 --2- 172.21.15.17:0/354076301 >> v2:172.21.15.77:3300/0 conn(0x7f3dcc0b5ed0 0x7f3dcc0b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:08.950 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.914+0000 7f3dca7fc700 1 -- 172.21.15.17:0/354076301 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4183722537 0 0) 0x7f3dcc061d90 con 0x7f3dcc0619b0 2022-08-01T19:42:08.950 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.914+0000 7f3dca7fc700 1 -- 172.21.15.17:0/354076301 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3db8003260 con 0x7f3dcc0619b0 2022-08-01T19:42:08.951 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.914+0000 7f3dca7fc700 1 -- 172.21.15.17:0/354076301 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1470457625 0 0) 0x7f3db8003260 con 0x7f3dcc0619b0 2022-08-01T19:42:08.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.914+0000 7f3dcbfff700 1 -- 172.21.15.17:0/354076301 >> v1:172.21.15.77:6789/0 conn(0x7f3dcc0b6ee0 legacy=0x7f3dcc0b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:08.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.914+0000 7f3dcbfff700 1 -- 172.21.15.17:0/354076301 >> v1:172.21.15.17:6789/0 conn(0x7f3dcc0619b0 legacy=0x7f3dcc143890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:08.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.914+0000 7f3dcbfff700 1 -- 172.21.15.17:0/354076301 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dcc143150 con 0x7f3dcc0b5ed0 2022-08-01T19:42:08.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.914+0000 7f3dcbfff700 1 --2- 172.21.15.17:0/354076301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3dcc0b5ed0 0x7f3dcc0b58a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7f3dbc001820 tx=0x7f3dc40091d0).ready entity=mon.1 client_cookie=d39bcf43e61d1f34 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:08.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.914+0000 7f3dca7fc700 1 -- 172.21.15.17:0/354076301 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dbc001680 con 0x7f3dcc0b5ed0 2022-08-01T19:42:08.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.914+0000 7f3dca7fc700 1 -- 172.21.15.17:0/354076301 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc400b580 con 0x7f3dcc0b5ed0 2022-08-01T19:42:08.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.914+0000 7f3dca7fc700 1 -- 172.21.15.17:0/354076301 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dc400b6f0 con 0x7f3dcc0b5ed0 2022-08-01T19:42:08.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.918+0000 7f3dd0d38700 1 -- 172.21.15.17:0/354076301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3dcc0b5ed0 msgr2=0x7f3dcc0b58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:08.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.918+0000 7f3dd0d38700 1 --2- 172.21.15.17:0/354076301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3dcc0b5ed0 0x7f3dcc0b58a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7f3dbc001820 tx=0x7f3dc40091d0).stop 2022-08-01T19:42:08.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.918+0000 7f3dd0d38700 1 -- 172.21.15.17:0/354076301 shutdown_connections 2022-08-01T19:42:08.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.918+0000 7f3dd0d38700 1 --2- 172.21.15.17:0/354076301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3dcc0b5ed0 0x7f3dcc0b58a0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:08.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.918+0000 7f3dd0d38700 1 -- 172.21.15.17:0/354076301 >> 172.21.15.17:0/354076301 conn(0x7f3dcc06af80 msgr2=0x7f3dcc05fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:08.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.918+0000 7f3dd0d38700 1 -- 172.21.15.17:0/354076301 shutdown_connections 2022-08-01T19:42:08.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.918+0000 7f3dd0d38700 1 -- 172.21.15.17:0/354076301 wait complete. 2022-08-01T19:42:08.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.918+0000 7f3dd0d38700 1 Processor -- start 2022-08-01T19:42:08.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.918+0000 7f3dd0d38700 1 -- start start 2022-08-01T19:42:08.966 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.918+0000 7f3dd0d38700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3dcc0619b0 0x7f3dcc0b0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:08.966 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.918+0000 7f3dd0d38700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dcc0b5ed0 0x7f3dcc146310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:08.967 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.918+0000 7f3dd0d38700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3dcc0b6ee0 0x7f3dcc141ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:08.967 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.918+0000 7f3dd0d38700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc0aed80 con 0x7f3dcc0b5ed0 2022-08-01T19:42:08.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.918+0000 7f3dd0d38700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc146fb0 con 0x7f3dcc0b6ee0 2022-08-01T19:42:08.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.918+0000 7f3dd0d38700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc13fbb0 con 0x7f3dcc0619b0 2022-08-01T19:42:08.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.918+0000 7f3dcaffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dcc0b5ed0 0x7f3dcc146310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:08.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.918+0000 7f3dcaffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dcc0b5ed0 0x7f3dcc146310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43272/0 (socket says 172.21.15.17:43272) 2022-08-01T19:42:08.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.918+0000 7f3dcaffd700 1 -- 172.21.15.17:0/2120682897 learned_addr learned my addr 172.21.15.17:0/2120682897 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:08.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.922+0000 7f3dcb7fe700 1 --2- 172.21.15.17:0/2120682897 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3dcc0619b0 0x7f3dcc0b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:08.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.922+0000 7f3dcbfff700 1 --2- 172.21.15.17:0/2120682897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3dcc0b6ee0 0x7f3dcc141ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:08.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.922+0000 7f3dcaffd700 1 -- 172.21.15.17:0/2120682897 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3dcc0619b0 msgr2=0x7f3dcc0b0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:08.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.922+0000 7f3dcaffd700 1 --2- 172.21.15.17:0/2120682897 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3dcc0619b0 0x7f3dcc0b0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:08.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.922+0000 7f3dcaffd700 1 -- 172.21.15.17:0/2120682897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3dcc0b6ee0 msgr2=0x7f3dcc141ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:08.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.922+0000 7f3dcaffd700 1 --2- 172.21.15.17:0/2120682897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3dcc0b6ee0 0x7f3dcc141ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:08.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.922+0000 7f3dcaffd700 1 -- 172.21.15.17:0/2120682897 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc4009470 con 0x7f3dcc0b5ed0 2022-08-01T19:42:08.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.922+0000 7f3dcaffd700 1 --2- 172.21.15.17:0/2120682897 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dcc0b5ed0 0x7f3dcc146310 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 rx=0x7f3dcc062290 tx=0x7f3dc0007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:08.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.922+0000 7f3dc8ff9700 1 -- 172.21.15.17:0/2120682897 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dc000ccd0 con 0x7f3dcc0b5ed0 2022-08-01T19:42:08.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.922+0000 7f3dc8ff9700 1 -- 172.21.15.17:0/2120682897 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc000ce30 con 0x7f3dcc0b5ed0 2022-08-01T19:42:08.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.922+0000 7f3dd0d38700 1 -- 172.21.15.17:0/2120682897 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dcc1457c0 con 0x7f3dcc0b5ed0 2022-08-01T19:42:08.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.922+0000 7f3dc8ff9700 1 -- 172.21.15.17:0/2120682897 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dc00125f0 con 0x7f3dcc0b5ed0 2022-08-01T19:42:08.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.922+0000 7f3dd0d38700 1 -- 172.21.15.17:0/2120682897 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dcc1423b0 con 0x7f3dcc0b5ed0 2022-08-01T19:42:08.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.922+0000 7f3dd0d38700 1 -- 172.21.15.17:0/2120682897 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dcc0aef80 con 0x7f3dcc0b5ed0 2022-08-01T19:42:08.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.926+0000 7f3dc8ff9700 1 -- 172.21.15.17:0/2120682897 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3dc0019030 con 0x7f3dcc0b5ed0 2022-08-01T19:42:08.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.926+0000 7f3dc8ff9700 1 --2- 172.21.15.17:0/2120682897 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3dac053b30 0x7f3dac055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:08.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.926+0000 7f3dc8ff9700 1 -- 172.21.15.17:0/2120682897 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f3dc006af10 con 0x7f3dcc0b5ed0 2022-08-01T19:42:08.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.926+0000 7f3dcb7fe700 1 --2- 172.21.15.17:0/2120682897 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3dac053b30 0x7f3dac055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:08.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.926+0000 7f3dcb7fe700 1 --2- 172.21.15.17:0/2120682897 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3dac053b30 0x7f3dac055ff0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f3dbc00d000 tx=0x7f3dbc000bc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:08.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:08.930+0000 7f3dc8ff9700 1 -- 172.21.15.17:0/2120682897 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3dc0042010 con 0x7f3dcc0b5ed0 2022-08-01T19:42:09.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.058+0000 7f3dd0d38700 1 -- 172.21.15.17:0/2120682897 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3dcc147170 con 0x7f3dcc0b5ed0 2022-08-01T19:42:09.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.058+0000 7f3dc8ff9700 1 -- 172.21.15.17:0/2120682897 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v113) v1 ==== 84+0+143 (secure 0 0 0) 0x7f3dc0045630 con 0x7f3dcc0b5ed0 2022-08-01T19:42:09.062 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:42:09.062 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2022-08-01T19:42:09.062 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:09.063 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 1, 2022-08-01T19:42:09.063 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": "device_health_metrics" 2022-08-01T19:42:09.063 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:09.063 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:09.063 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 2, 2022-08-01T19:42:09.064 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": "rbd" 2022-08-01T19:42:09.064 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:09.064 INFO:teuthology.orchestra.run.smithi017.stdout:] 2022-08-01T19:42:09.065 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.058+0000 7f3dd0d38700 1 -- 172.21.15.17:0/2120682897 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3dac053b30 msgr2=0x7f3dac055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:09.066 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.058+0000 7f3dd0d38700 1 --2- 172.21.15.17:0/2120682897 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3dac053b30 0x7f3dac055ff0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f3dbc00d000 tx=0x7f3dbc000bc0).stop 2022-08-01T19:42:09.066 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.058+0000 7f3dd0d38700 1 -- 172.21.15.17:0/2120682897 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dcc0b5ed0 msgr2=0x7f3dcc146310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:09.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.058+0000 7f3dd0d38700 1 --2- 172.21.15.17:0/2120682897 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dcc0b5ed0 0x7f3dcc146310 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 rx=0x7f3dcc062290 tx=0x7f3dc0007c20).stop 2022-08-01T19:42:09.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.058+0000 7f3dd0d38700 1 -- 172.21.15.17:0/2120682897 shutdown_connections 2022-08-01T19:42:09.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.058+0000 7f3dd0d38700 1 --2- 172.21.15.17:0/2120682897 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3dac053b30 0x7f3dac055ff0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:09.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.058+0000 7f3dd0d38700 1 --2- 172.21.15.17:0/2120682897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3dcc0b6ee0 0x7f3dcc141ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:09.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.058+0000 7f3dd0d38700 1 --2- 172.21.15.17:0/2120682897 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dcc0b5ed0 0x7f3dcc146310 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:09.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.058+0000 7f3dd0d38700 1 --2- 172.21.15.17:0/2120682897 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3dcc0619b0 0x7f3dcc0b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:09.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.058+0000 7f3dd0d38700 1 -- 172.21.15.17:0/2120682897 >> 172.21.15.17:0/2120682897 conn(0x7f3dcc06af80 msgr2=0x7f3dcc05fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:09.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.058+0000 7f3dd0d38700 1 -- 172.21.15.17:0/2120682897 shutdown_connections 2022-08-01T19:42:09.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.058+0000 7f3dd0d38700 1 -- 172.21.15.17:0/2120682897 wait complete. 2022-08-01T19:42:09.073 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-08-01T19:42:09.074 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-08-01T19:42:09.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.202+0000 7f9bc6425700 1 Processor -- start 2022-08-01T19:42:09.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.202+0000 7f9bc6425700 1 -- start start 2022-08-01T19:42:09.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.202+0000 7f9bc6425700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9bc00ad000 0x7f9bc01435c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:09.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.202+0000 7f9bc6425700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bc0143ab0 con 0x7f9bc00b3260 2022-08-01T19:42:09.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.202+0000 7f9bc6425700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bc00b0c80 con 0x7f9bc00b08a0 2022-08-01T19:42:09.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.202+0000 7f9bc6425700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc00ae810 con 0x7f9bc00ad000 2022-08-01T19:42:09.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.202+0000 7f9bc4c22700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9bc00b3260 0x7f9bc013fa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35474/0 (socket says 172.21.15.17:35474) 2022-08-01T19:42:09.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.202+0000 7f9bc4c22700 1 -- 172.21.15.17:0/218493466 learned_addr learned my addr 172.21.15.17:0/218493466 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:09.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.202+0000 7f9bc5c24700 1 --2- 172.21.15.17:0/218493466 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9bc00ad000 0x7f9bc01435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:09.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.202+0000 7f9bb7fff700 1 -- 172.21.15.17:0/218493466 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 117047651 0 0) 0x7f9bc0143ab0 con 0x7f9bc00b3260 2022-08-01T19:42:09.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.202+0000 7f9bb7fff700 1 -- 172.21.15.17:0/218493466 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ba4003260 con 0x7f9bc00b3260 2022-08-01T19:42:09.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.202+0000 7f9bb7fff700 1 -- 172.21.15.17:0/218493466 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 865179957 0 0) 0x7f9bc00b0c80 con 0x7f9bc00b08a0 2022-08-01T19:42:09.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.202+0000 7f9bb7fff700 1 -- 172.21.15.17:0/218493466 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bc0143ab0 con 0x7f9bc00b08a0 2022-08-01T19:42:09.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.202+0000 7f9bb7fff700 1 -- 172.21.15.17:0/218493466 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1217780062 0 0) 0x7f9ba4003260 con 0x7f9bc00b3260 2022-08-01T19:42:09.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.202+0000 7f9bc5c24700 1 -- 172.21.15.17:0/218493466 >> v1:172.21.15.77:6789/0 conn(0x7f9bc00b08a0 legacy=0x7f9bc0142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:09.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.202+0000 7f9bc5c24700 1 -- 172.21.15.17:0/218493466 >> v1:172.21.15.17:6789/0 conn(0x7f9bc00b3260 legacy=0x7f9bc013fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:09.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.202+0000 7f9bc5c24700 1 -- 172.21.15.17:0/218493466 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc00aeab0 con 0x7f9bc00ad000 2022-08-01T19:42:09.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.202+0000 7f9bc5c24700 1 --2- 172.21.15.17:0/218493466 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9bc00ad000 0x7f9bc01435c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f9bbc001110 tx=0x7f9bb80091d0).ready entity=mon.2 client_cookie=fd6bd6696fe7cd14 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:09.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.202+0000 7f9bb7fff700 1 -- 172.21.15.17:0/218493466 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9bbc001660 con 0x7f9bc00ad000 2022-08-01T19:42:09.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.202+0000 7f9bb7fff700 1 -- 172.21.15.17:0/218493466 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb800f510 con 0x7f9bc00ad000 2022-08-01T19:42:09.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.202+0000 7f9bb7fff700 1 -- 172.21.15.17:0/218493466 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9bb800f680 con 0x7f9bc00ad000 2022-08-01T19:42:09.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.206+0000 7f9bc6425700 1 -- 172.21.15.17:0/218493466 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9bc00ad000 msgr2=0x7f9bc01435c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:09.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.206+0000 7f9bc6425700 1 --2- 172.21.15.17:0/218493466 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9bc00ad000 0x7f9bc01435c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f9bbc001110 tx=0x7f9bb80091d0).stop 2022-08-01T19:42:09.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.206+0000 7f9bc6425700 1 -- 172.21.15.17:0/218493466 shutdown_connections 2022-08-01T19:42:09.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.206+0000 7f9bc6425700 1 --2- 172.21.15.17:0/218493466 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9bc00ad000 0x7f9bc01435c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:09.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.206+0000 7f9bc6425700 1 -- 172.21.15.17:0/218493466 >> 172.21.15.17:0/218493466 conn(0x7f9bc00612d0 msgr2=0x7f9bc0062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:09.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.206+0000 7f9bc6425700 1 -- 172.21.15.17:0/218493466 shutdown_connections 2022-08-01T19:42:09.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.206+0000 7f9bc6425700 1 -- 172.21.15.17:0/218493466 wait complete. 2022-08-01T19:42:09.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.206+0000 7f9bc6425700 1 Processor -- start 2022-08-01T19:42:09.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.206+0000 7f9bc6425700 1 -- start start 2022-08-01T19:42:09.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.206+0000 7f9bc6425700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9bc00ad000 0x7f9bc0141710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:09.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.206+0000 7f9bc6425700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9bc00b08a0 0x7f9bc00a3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:09.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.206+0000 7f9bc6425700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9bc00b3260 0x7f9bc00a4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:09.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.206+0000 7f9bc6425700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc0066fb0 con 0x7f9bc00b08a0 2022-08-01T19:42:09.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.206+0000 7f9bc6425700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc013eac0 con 0x7f9bc00ad000 2022-08-01T19:42:09.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.206+0000 7f9bc6425700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc0064eb0 con 0x7f9bc00b3260 2022-08-01T19:42:09.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.206+0000 7f9bc4c22700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9bc00b08a0 0x7f9bc00a3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:09.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.206+0000 7f9bc4c22700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9bc00b08a0 0x7f9bc00a3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43286/0 (socket says 172.21.15.17:43286) 2022-08-01T19:42:09.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.206+0000 7f9bc4c22700 1 -- 172.21.15.17:0/945665267 learned_addr learned my addr 172.21.15.17:0/945665267 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:09.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.206+0000 7f9bc5c24700 1 --2- 172.21.15.17:0/945665267 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9bc00b3260 0x7f9bc00a4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:09.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.206+0000 7f9bc5423700 1 --2- 172.21.15.17:0/945665267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9bc00ad000 0x7f9bc0141710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:09.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.210+0000 7f9bc4c22700 1 -- 172.21.15.17:0/945665267 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9bc00b3260 msgr2=0x7f9bc00a4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:09.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.210+0000 7f9bc4c22700 1 --2- 172.21.15.17:0/945665267 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9bc00b3260 0x7f9bc00a4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:09.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.210+0000 7f9bc4c22700 1 -- 172.21.15.17:0/945665267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9bc00ad000 msgr2=0x7f9bc0141710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:09.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.210+0000 7f9bc4c22700 1 --2- 172.21.15.17:0/945665267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9bc00ad000 0x7f9bc0141710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:09.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.210+0000 7f9bc4c22700 1 -- 172.21.15.17:0/945665267 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb8009470 con 0x7f9bc00b08a0 2022-08-01T19:42:09.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.210+0000 7f9bc5c24700 1 --2- 172.21.15.17:0/945665267 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9bc00b3260 0x7f9bc00a4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:42:09.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.210+0000 7f9bc4c22700 1 --2- 172.21.15.17:0/945665267 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9bc00b08a0 0x7f9bc00a3c60 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7f9bc00a3740 tx=0x7f9bbc001b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:09.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.210+0000 7f9bb67fc700 1 -- 172.21.15.17:0/945665267 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9bbc007170 con 0x7f9bc00b08a0 2022-08-01T19:42:09.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.210+0000 7f9bb67fc700 1 -- 172.21.15.17:0/945665267 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bbc0072d0 con 0x7f9bc00b08a0 2022-08-01T19:42:09.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.210+0000 7f9bc6425700 1 -- 172.21.15.17:0/945665267 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bc0065fc0 con 0x7f9bc00b08a0 2022-08-01T19:42:09.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.210+0000 7f9bb67fc700 1 -- 172.21.15.17:0/945665267 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9bbc004920 con 0x7f9bc00b08a0 2022-08-01T19:42:09.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.210+0000 7f9bc6425700 1 -- 172.21.15.17:0/945665267 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bc0066260 con 0x7f9bc00b08a0 2022-08-01T19:42:09.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.210+0000 7f9bc6425700 1 -- 172.21.15.17:0/945665267 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bc0155c90 con 0x7f9bc00b08a0 2022-08-01T19:42:09.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.210+0000 7f9bb67fc700 1 -- 172.21.15.17:0/945665267 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9bbc004a80 con 0x7f9bc00b08a0 2022-08-01T19:42:09.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.214+0000 7f9bb67fc700 1 --2- 172.21.15.17:0/945665267 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9ba8053a70 0x7f9ba8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:09.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.214+0000 7f9bb67fc700 1 -- 172.21.15.17:0/945665267 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f9bbc06bab0 con 0x7f9bc00b08a0 2022-08-01T19:42:09.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.214+0000 7f9bc5423700 1 --2- 172.21.15.17:0/945665267 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9ba8053a70 0x7f9ba8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:09.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.214+0000 7f9bc5423700 1 --2- 172.21.15.17:0/945665267 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9ba8053a70 0x7f9ba8055f30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f9bb000b000 tx=0x7f9bb0009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:09.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.218+0000 7f9bb67fc700 1 -- 172.21.15.17:0/945665267 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9bbc042aa0 con 0x7f9bc00b08a0 2022-08-01T19:42:09.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.346+0000 7f9bc6425700 1 -- 172.21.15.17:0/945665267 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f9bc00574e0 con 0x7f9bc00b08a0 2022-08-01T19:42:09.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.918+0000 7f9bb67fc700 1 -- 172.21.15.17:0/945665267 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v114) v1 ==== 133+0+0 (secure 0 0 0) 0x7f9bbc0460c0 con 0x7f9bc00b08a0 2022-08-01T19:42:09.926 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.922+0000 7f9bc6425700 1 -- 172.21.15.17:0/945665267 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9ba8053a70 msgr2=0x7f9ba8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:09.927 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.922+0000 7f9bc6425700 1 --2- 172.21.15.17:0/945665267 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9ba8053a70 0x7f9ba8055f30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f9bb000b000 tx=0x7f9bb0009040).stop 2022-08-01T19:42:09.927 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.922+0000 7f9bc6425700 1 -- 172.21.15.17:0/945665267 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9bc00b08a0 msgr2=0x7f9bc00a3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:09.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.922+0000 7f9bc6425700 1 --2- 172.21.15.17:0/945665267 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9bc00b08a0 0x7f9bc00a3c60 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7f9bc00a3740 tx=0x7f9bbc001b30).stop 2022-08-01T19:42:09.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.922+0000 7f9bc6425700 1 -- 172.21.15.17:0/945665267 shutdown_connections 2022-08-01T19:42:09.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.922+0000 7f9bc6425700 1 --2- 172.21.15.17:0/945665267 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9ba8053a70 0x7f9ba8055f30 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:09.929 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.922+0000 7f9bc6425700 1 --2- 172.21.15.17:0/945665267 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9bc00b3260 0x7f9bc00a4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:09.929 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.922+0000 7f9bc6425700 1 --2- 172.21.15.17:0/945665267 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9bc00b08a0 0x7f9bc00a3c60 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:09.929 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.922+0000 7f9bc6425700 1 --2- 172.21.15.17:0/945665267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9bc00ad000 0x7f9bc0141710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:09.930 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.922+0000 7f9bc6425700 1 -- 172.21.15.17:0/945665267 >> 172.21.15.17:0/945665267 conn(0x7f9bc00612d0 msgr2=0x7f9bc00b24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:09.930 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.922+0000 7f9bc6425700 1 -- 172.21.15.17:0/945665267 shutdown_connections 2022-08-01T19:42:09.930 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:09.926+0000 7f9bc6425700 1 -- 172.21.15.17:0/945665267 wait complete. 2022-08-01T19:42:09.931 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' created 2022-08-01T19:42:09.950 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-08-01T19:42:10.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.066+0000 7f4e8bcc4700 1 Processor -- start 2022-08-01T19:42:10.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.066+0000 7f4e8bcc4700 1 -- start start 2022-08-01T19:42:10.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.070+0000 7f4e8bcc4700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4e84146ae0 0x7f4e841403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:10.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.070+0000 7f4e8bcc4700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f4e840b4820 0x7f4e840b5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:10.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.070+0000 7f4e8bcc4700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e84143cf0 con 0x7f4e840b4310 2022-08-01T19:42:10.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.070+0000 7f4e8bcc4700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e841476a0 con 0x7f4e84146ae0 2022-08-01T19:42:10.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.070+0000 7f4e8bcc4700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e84143f90 con 0x7f4e840b4820 2022-08-01T19:42:10.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.070+0000 7f4e89a60700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4e84146ae0 0x7f4e841403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:10.076 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.070+0000 7f4e8925f700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f4e840b4310 0x7f4e841408b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35486/0 (socket says 172.21.15.17:35486) 2022-08-01T19:42:10.076 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.070+0000 7f4e8925f700 1 -- 172.21.15.17:0/746311758 learned_addr learned my addr 172.21.15.17:0/746311758 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:10.076 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.070+0000 7f4e89a60700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4e84146ae0 0x7f4e841403c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43292/0 (socket says 172.21.15.17:43292) 2022-08-01T19:42:10.076 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.070+0000 7f4e8a261700 1 --2- 172.21.15.17:0/746311758 >> v2:172.21.15.77:3300/0 conn(0x7f4e840b4820 0x7f4e840b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:10.077 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.070+0000 7f4e89a60700 1 -- 172.21.15.17:0/746311758 >> v2:172.21.15.77:3300/0 conn(0x7f4e840b4820 msgr2=0x7f4e840b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:10.077 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.070+0000 7f4e89a60700 1 --2- 172.21.15.17:0/746311758 >> v2:172.21.15.77:3300/0 conn(0x7f4e840b4820 0x7f4e840b5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:10.077 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.070+0000 7f4e89a60700 1 -- 172.21.15.17:0/746311758 >> v1:172.21.15.17:6789/0 conn(0x7f4e840b4310 legacy=0x7f4e841408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:10.077 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.070+0000 7f4e88a5e700 1 -- 172.21.15.17:0/746311758 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1315555873 0 0) 0x7f4e84143cf0 con 0x7f4e840b4310 2022-08-01T19:42:10.078 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.070+0000 7f4e89a60700 1 -- 172.21.15.17:0/746311758 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e840b0e20 con 0x7f4e84146ae0 2022-08-01T19:42:10.078 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.070+0000 7f4e89a60700 1 --2- 172.21.15.17:0/746311758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e84146ae0 0x7f4e841403c0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 rx=0x7f4e7c002f70 tx=0x7f4e7c006df0).ready entity=mon.0 client_cookie=a2cb313d88f16874 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:10.078 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.070+0000 7f4e88a5e700 1 -- 172.21.15.17:0/746311758 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e7c00d570 con 0x7f4e84146ae0 2022-08-01T19:42:10.078 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.070+0000 7f4e88a5e700 1 -- 172.21.15.17:0/746311758 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7c000e90 con 0x7f4e84146ae0 2022-08-01T19:42:10.079 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.070+0000 7f4e88a5e700 1 -- 172.21.15.17:0/746311758 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e7c00d8c0 con 0x7f4e84146ae0 2022-08-01T19:42:10.079 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.070+0000 7f4e8bcc4700 1 -- 172.21.15.17:0/746311758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e84146ae0 msgr2=0x7f4e841403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:10.080 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.070+0000 7f4e8bcc4700 1 --2- 172.21.15.17:0/746311758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e84146ae0 0x7f4e841403c0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 rx=0x7f4e7c002f70 tx=0x7f4e7c006df0).stop 2022-08-01T19:42:10.080 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.070+0000 7f4e8bcc4700 1 -- 172.21.15.17:0/746311758 shutdown_connections 2022-08-01T19:42:10.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.070+0000 7f4e8bcc4700 1 --2- 172.21.15.17:0/746311758 >> v2:172.21.15.77:3300/0 conn(0x7f4e840b4820 0x7f4e840b5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:10.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.070+0000 7f4e8bcc4700 1 --2- 172.21.15.17:0/746311758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e84146ae0 0x7f4e841403c0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:10.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.070+0000 7f4e8bcc4700 1 -- 172.21.15.17:0/746311758 >> 172.21.15.17:0/746311758 conn(0x7f4e8405c620 msgr2=0x7f4e8405ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:10.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.070+0000 7f4e8bcc4700 1 -- 172.21.15.17:0/746311758 shutdown_connections 2022-08-01T19:42:10.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.070+0000 7f4e8bcc4700 1 -- 172.21.15.17:0/746311758 wait complete. 2022-08-01T19:42:10.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.074+0000 7f4e8bcc4700 1 Processor -- start 2022-08-01T19:42:10.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.074+0000 7f4e8bcc4700 1 -- start start 2022-08-01T19:42:10.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.074+0000 7f4e8bcc4700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4e840b4310 0x7f4e840ae820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:10.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.074+0000 7f4e8bcc4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e840b4820 0x7f4e84155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:10.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.074+0000 7f4e8bcc4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e84146ae0 0x7f4e84145550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:10.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.074+0000 7f4e8bcc4700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e84144ef0 con 0x7f4e84146ae0 2022-08-01T19:42:10.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.074+0000 7f4e8bcc4700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e84156ab0 con 0x7f4e840b4820 2022-08-01T19:42:10.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.074+0000 7f4e8bcc4700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e840b8020 con 0x7f4e840b4310 2022-08-01T19:42:10.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.074+0000 7f4e8a261700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e84146ae0 0x7f4e84145550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:10.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.074+0000 7f4e8a261700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e84146ae0 0x7f4e84145550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43300/0 (socket says 172.21.15.17:43300) 2022-08-01T19:42:10.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.074+0000 7f4e8a261700 1 -- 172.21.15.17:0/1729751803 learned_addr learned my addr 172.21.15.17:0/1729751803 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:10.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.074+0000 7f4e8925f700 1 --2- 172.21.15.17:0/1729751803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e840b4820 0x7f4e84155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:10.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.074+0000 7f4e89a60700 1 --2- 172.21.15.17:0/1729751803 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4e840b4310 0x7f4e840ae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:10.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.074+0000 7f4e8a261700 1 -- 172.21.15.17:0/1729751803 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4e840b4310 msgr2=0x7f4e840ae820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:10.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.074+0000 7f4e8a261700 1 --2- 172.21.15.17:0/1729751803 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4e840b4310 0x7f4e840ae820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:10.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.074+0000 7f4e8a261700 1 -- 172.21.15.17:0/1729751803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e840b4820 msgr2=0x7f4e84155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:10.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.074+0000 7f4e8a261700 1 --2- 172.21.15.17:0/1729751803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e840b4820 0x7f4e84155eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:10.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.074+0000 7f4e8a261700 1 -- 172.21.15.17:0/1729751803 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e7c00b180 con 0x7f4e84146ae0 2022-08-01T19:42:10.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.074+0000 7f4e8a261700 1 --2- 172.21.15.17:0/1729751803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e84146ae0 0x7f4e84145550 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 rx=0x7f4e840adb60 tx=0x7f4e740091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:10.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.074+0000 7f4e82ffd700 1 -- 172.21.15.17:0/1729751803 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e7400ee60 con 0x7f4e84146ae0 2022-08-01T19:42:10.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.074+0000 7f4e8bcc4700 1 -- 172.21.15.17:0/1729751803 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e84156790 con 0x7f4e84146ae0 2022-08-01T19:42:10.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.074+0000 7f4e8bcc4700 1 -- 172.21.15.17:0/1729751803 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e84146230 con 0x7f4e84146ae0 2022-08-01T19:42:10.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.074+0000 7f4e82ffd700 1 -- 172.21.15.17:0/1729751803 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7400c850 con 0x7f4e84146ae0 2022-08-01T19:42:10.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.074+0000 7f4e82ffd700 1 -- 172.21.15.17:0/1729751803 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e7400cbb0 con 0x7f4e84146ae0 2022-08-01T19:42:10.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.078+0000 7f4e8bcc4700 1 -- 172.21.15.17:0/1729751803 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e840b7dc0 con 0x7f4e84146ae0 2022-08-01T19:42:10.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.078+0000 7f4e82ffd700 1 -- 172.21.15.17:0/1729751803 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4e7400cd10 con 0x7f4e84146ae0 2022-08-01T19:42:10.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.078+0000 7f4e82ffd700 1 --2- 172.21.15.17:0/1729751803 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4e64053a70 0x7f4e64055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:10.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.078+0000 7f4e82ffd700 1 -- 172.21.15.17:0/1729751803 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f4e7406c8a0 con 0x7f4e84146ae0 2022-08-01T19:42:10.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.078+0000 7f4e89a60700 1 --2- 172.21.15.17:0/1729751803 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4e64053a70 0x7f4e64055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:10.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.078+0000 7f4e89a60700 1 --2- 172.21.15.17:0/1729751803 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4e64053a70 0x7f4e64055f30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f4e7c01a000 tx=0x7f4e7c014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:10.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.082+0000 7f4e82ffd700 1 -- 172.21.15.17:0/1729751803 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4e74043830 con 0x7f4e84146ae0 2022-08-01T19:42:10.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.206+0000 7f4e8bcc4700 1 -- 172.21.15.17:0/1729751803 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f4e840574e0 con 0x7f4e84146ae0 2022-08-01T19:42:10.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.566+0000 7f4e82ffd700 1 -- 172.21.15.17:0/1729751803 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v115) v1 ==== 125+0+0 (secure 0 0 0) 0x7f4e74016020 con 0x7f4e84146ae0 2022-08-01T19:42:10.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.574+0000 7f4e8bcc4700 1 -- 172.21.15.17:0/1729751803 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4e64053a70 msgr2=0x7f4e64055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:10.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.574+0000 7f4e8bcc4700 1 --2- 172.21.15.17:0/1729751803 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4e64053a70 0x7f4e64055f30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f4e7c01a000 tx=0x7f4e7c014000).stop 2022-08-01T19:42:10.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.574+0000 7f4e8bcc4700 1 -- 172.21.15.17:0/1729751803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e84146ae0 msgr2=0x7f4e84145550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:10.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.574+0000 7f4e8bcc4700 1 --2- 172.21.15.17:0/1729751803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e84146ae0 0x7f4e84145550 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 rx=0x7f4e840adb60 tx=0x7f4e740091d0).stop 2022-08-01T19:42:10.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.574+0000 7f4e8bcc4700 1 -- 172.21.15.17:0/1729751803 shutdown_connections 2022-08-01T19:42:10.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.574+0000 7f4e8bcc4700 1 --2- 172.21.15.17:0/1729751803 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4e64053a70 0x7f4e64055f30 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:10.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.574+0000 7f4e8bcc4700 1 --2- 172.21.15.17:0/1729751803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e84146ae0 0x7f4e84145550 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:10.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.574+0000 7f4e8bcc4700 1 --2- 172.21.15.17:0/1729751803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e840b4820 0x7f4e84155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:10.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.574+0000 7f4e8bcc4700 1 --2- 172.21.15.17:0/1729751803 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4e840b4310 0x7f4e840ae820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:10.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.574+0000 7f4e8bcc4700 1 -- 172.21.15.17:0/1729751803 >> 172.21.15.17:0/1729751803 conn(0x7f4e8405c620 msgr2=0x7f4e8413e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:10.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.574+0000 7f4e8bcc4700 1 -- 172.21.15.17:0/1729751803 shutdown_connections 2022-08-01T19:42:10.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.574+0000 7f4e8bcc4700 1 -- 172.21.15.17:0/1729751803 wait complete. 2022-08-01T19:42:10.589 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' created 2022-08-01T19:42:10.599 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-08-01T19:42:10.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.710+0000 7f30e6576700 1 Processor -- start 2022-08-01T19:42:10.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.710+0000 7f30e6576700 1 -- start start 2022-08-01T19:42:10.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.714+0000 7f30e6576700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f30e00b41f0 0x7f30e0061d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:10.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.714+0000 7f30e6576700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f30e0142ff0 0x7f30e0143480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.714+0000 7f30e6576700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30e00b60e0 con 0x7f30e00b6420 2022-08-01T19:42:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.714+0000 7f30e6576700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30e00b2100 con 0x7f30e0142ff0 2022-08-01T19:42:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.714+0000 7f30e6576700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30e00b23a0 con 0x7f30e00b41f0 2022-08-01T19:42:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.714+0000 7f30e5d75700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f30e0142ff0 0x7f30e0143480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:10.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.714+0000 7f30e4d73700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f30e00b41f0 0x7f30e0061d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:10.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.714+0000 7f30e5d75700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f30e0142ff0 0x7f30e0143480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43310/0 (socket says 172.21.15.17:43310) 2022-08-01T19:42:10.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.714+0000 7f30e5d75700 1 -- 172.21.15.17:0/2394811020 learned_addr learned my addr 172.21.15.17:0/2394811020 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:10.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.714+0000 7f30e5574700 1 --1- 172.21.15.17:0/2394811020 >> v1:172.21.15.77:6789/0 conn(0x7f30e00b6420 0x7f30e0061600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:52300/0 (socket says 172.21.15.17:52300) 2022-08-01T19:42:10.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.714+0000 7f30e5d75700 1 -- 172.21.15.17:0/2394811020 >> v2:172.21.15.77:3300/0 conn(0x7f30e00b41f0 msgr2=0x7f30e0061d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:10.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.714+0000 7f30e5d75700 1 --2- 172.21.15.17:0/2394811020 >> v2:172.21.15.77:3300/0 conn(0x7f30e00b41f0 0x7f30e0061d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:10.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.714+0000 7f30e5d75700 1 -- 172.21.15.17:0/2394811020 >> v1:172.21.15.77:6789/0 conn(0x7f30e00b6420 legacy=0x7f30e0061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:10.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.714+0000 7f30e5d75700 1 -- 172.21.15.17:0/2394811020 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e00b2640 con 0x7f30e0142ff0 2022-08-01T19:42:10.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.714+0000 7f30e5d75700 1 --2- 172.21.15.17:0/2394811020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30e0142ff0 0x7f30e0143480 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 rx=0x7f30d80032f0 tx=0x7f30d8009290).ready entity=mon.0 client_cookie=210cfcfeb3380244 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:10.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.714+0000 7f30cffff700 1 -- 172.21.15.17:0/2394811020 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30d800b870 con 0x7f30e0142ff0 2022-08-01T19:42:10.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.714+0000 7f30cffff700 1 -- 172.21.15.17:0/2394811020 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d800b9d0 con 0x7f30e0142ff0 2022-08-01T19:42:10.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.714+0000 7f30cffff700 1 -- 172.21.15.17:0/2394811020 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30d800bb40 con 0x7f30e0142ff0 2022-08-01T19:42:10.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.714+0000 7f30e6576700 1 -- 172.21.15.17:0/2394811020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30e0142ff0 msgr2=0x7f30e0143480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:10.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.714+0000 7f30e6576700 1 --2- 172.21.15.17:0/2394811020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30e0142ff0 0x7f30e0143480 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 rx=0x7f30d80032f0 tx=0x7f30d8009290).stop 2022-08-01T19:42:10.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.714+0000 7f30e6576700 1 -- 172.21.15.17:0/2394811020 shutdown_connections 2022-08-01T19:42:10.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.714+0000 7f30e6576700 1 --2- 172.21.15.17:0/2394811020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30e0142ff0 0x7f30e0143480 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:10.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.714+0000 7f30e6576700 1 --2- 172.21.15.17:0/2394811020 >> v2:172.21.15.77:3300/0 conn(0x7f30e00b41f0 0x7f30e0061d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:10.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.714+0000 7f30e6576700 1 -- 172.21.15.17:0/2394811020 >> 172.21.15.17:0/2394811020 conn(0x7f30e005c620 msgr2=0x7f30e005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:10.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.714+0000 7f30e6576700 1 -- 172.21.15.17:0/2394811020 shutdown_connections 2022-08-01T19:42:10.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.714+0000 7f30e6576700 1 -- 172.21.15.17:0/2394811020 wait complete. 2022-08-01T19:42:10.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.718+0000 7f30e6576700 1 Processor -- start 2022-08-01T19:42:10.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.718+0000 7f30e6576700 1 -- start start 2022-08-01T19:42:10.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.718+0000 7f30e6576700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30e00b41f0 0x7f30e00b3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:10.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.718+0000 7f30e6576700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30e00b6420 0x7f30e0155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:10.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.718+0000 7f30e6576700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f30e0142ff0 0x7f30e0156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:10.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.718+0000 7f30e6576700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e00aff70 con 0x7f30e00b6420 2022-08-01T19:42:10.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.718+0000 7f30e6576700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e00afd10 con 0x7f30e00b41f0 2022-08-01T19:42:10.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.718+0000 7f30e6576700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e00b0470 con 0x7f30e0142ff0 2022-08-01T19:42:10.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.722+0000 7f30e4d73700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30e00b6420 0x7f30e0155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:10.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.722+0000 7f30e5574700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30e00b41f0 0x7f30e00b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:10.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.722+0000 7f30e4d73700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30e00b6420 0x7f30e0155d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43314/0 (socket says 172.21.15.17:43314) 2022-08-01T19:42:10.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.722+0000 7f30e4d73700 1 -- 172.21.15.17:0/2701771879 learned_addr learned my addr 172.21.15.17:0/2701771879 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:10.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.722+0000 7f30e5d75700 1 --2- 172.21.15.17:0/2701771879 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f30e0142ff0 0x7f30e0156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:10.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.722+0000 7f30e5574700 1 -- 172.21.15.17:0/2701771879 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f30e0142ff0 msgr2=0x7f30e0156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:10.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.722+0000 7f30e5574700 1 --2- 172.21.15.17:0/2701771879 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f30e0142ff0 0x7f30e0156230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:10.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.722+0000 7f30e5574700 1 -- 172.21.15.17:0/2701771879 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30e00b6420 msgr2=0x7f30e0155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:10.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.722+0000 7f30e5574700 1 --2- 172.21.15.17:0/2701771879 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30e00b6420 0x7f30e0155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:10.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.722+0000 7f30e5574700 1 -- 172.21.15.17:0/2701771879 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d8009530 con 0x7f30e00b41f0 2022-08-01T19:42:10.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.722+0000 7f30e5574700 1 --2- 172.21.15.17:0/2701771879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30e00b41f0 0x7f30e00b3720 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f30e00b2c60 tx=0x7f30dc007b40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:10.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.722+0000 7f30ce7fc700 1 -- 172.21.15.17:0/2701771879 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30dc00cdd0 con 0x7f30e00b41f0 2022-08-01T19:42:10.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.722+0000 7f30ce7fc700 1 -- 172.21.15.17:0/2701771879 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30dc007170 con 0x7f30e00b41f0 2022-08-01T19:42:10.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.722+0000 7f30ce7fc700 1 -- 172.21.15.17:0/2701771879 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30dc004520 con 0x7f30e00b41f0 2022-08-01T19:42:10.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.722+0000 7f30e6576700 1 -- 172.21.15.17:0/2701771879 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30e00b0710 con 0x7f30e00b41f0 2022-08-01T19:42:10.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.722+0000 7f30e6576700 1 -- 172.21.15.17:0/2701771879 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e0140bb0 con 0x7f30e00b41f0 2022-08-01T19:42:10.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.722+0000 7f30e6576700 1 -- 172.21.15.17:0/2701771879 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30e00b28e0 con 0x7f30e00b41f0 2022-08-01T19:42:10.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.722+0000 7f30ce7fc700 1 -- 172.21.15.17:0/2701771879 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f30dc004680 con 0x7f30e00b41f0 2022-08-01T19:42:10.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.726+0000 7f30ce7fc700 1 --2- 172.21.15.17:0/2701771879 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f30c0053b00 0x7f30c0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:10.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.726+0000 7f30ce7fc700 1 -- 172.21.15.17:0/2701771879 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f30dc06b590 con 0x7f30e00b41f0 2022-08-01T19:42:10.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.726+0000 7f30e4d73700 1 --2- 172.21.15.17:0/2701771879 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f30c0053b00 0x7f30c0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:10.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.726+0000 7f30e4d73700 1 --2- 172.21.15.17:0/2701771879 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f30c0053b00 0x7f30c0055fc0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f30d000b000 tx=0x7f30d0006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:10.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.734+0000 7f30ce7fc700 1 -- 172.21.15.17:0/2701771879 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f30dc042370 con 0x7f30e00b41f0 2022-08-01T19:42:10.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.902+0000 7f30e6576700 1 -- 172.21.15.17:0/2701771879 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f30e00b60e0 con 0x7f30e00b41f0 2022-08-01T19:42:10.917 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.914+0000 7f30ce7fc700 1 -- 172.21.15.17:0/2701771879 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 21 and data pool 22 v57) v1 ==== 189+0+0 (secure 0 0 0) 0x7f30e00b60e0 con 0x7f30e00b41f0 2022-08-01T19:42:10.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.918+0000 7f30e6576700 1 -- 172.21.15.17:0/2701771879 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f30c0053b00 msgr2=0x7f30c0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:10.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.918+0000 7f30e6576700 1 --2- 172.21.15.17:0/2701771879 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f30c0053b00 0x7f30c0055fc0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f30d000b000 tx=0x7f30d0006df0).stop 2022-08-01T19:42:10.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.918+0000 7f30e6576700 1 -- 172.21.15.17:0/2701771879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30e00b41f0 msgr2=0x7f30e00b3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:10.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.918+0000 7f30e6576700 1 --2- 172.21.15.17:0/2701771879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30e00b41f0 0x7f30e00b3720 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f30e00b2c60 tx=0x7f30dc007b40).stop 2022-08-01T19:42:10.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.918+0000 7f30e6576700 1 -- 172.21.15.17:0/2701771879 shutdown_connections 2022-08-01T19:42:10.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.918+0000 7f30e6576700 1 --2- 172.21.15.17:0/2701771879 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f30c0053b00 0x7f30c0055fc0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:10.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.918+0000 7f30e6576700 1 --2- 172.21.15.17:0/2701771879 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f30e0142ff0 0x7f30e0156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:10.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.918+0000 7f30e6576700 1 --2- 172.21.15.17:0/2701771879 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30e00b6420 0x7f30e0155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:10.924 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.918+0000 7f30e6576700 1 --2- 172.21.15.17:0/2701771879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30e00b41f0 0x7f30e00b3720 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:10.924 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.918+0000 7f30e6576700 1 -- 172.21.15.17:0/2701771879 >> 172.21.15.17:0/2701771879 conn(0x7f30e005c620 msgr2=0x7f30e00b89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:10.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.918+0000 7f30e6576700 1 -- 172.21.15.17:0/2701771879 shutdown_connections 2022-08-01T19:42:10.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:10.922+0000 7f30e6576700 1 -- 172.21.15.17:0/2701771879 wait complete. 2022-08-01T19:42:10.929 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 21 and data pool 22 2022-08-01T19:42:10.943 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:42:11.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.054+0000 7f6834da4700 1 Processor -- start 2022-08-01T19:42:11.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.054+0000 7f6834da4700 1 -- start start 2022-08-01T19:42:11.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.054+0000 7f6834da4700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f68300a9320 0x7f68301466e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:11.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.054+0000 7f6834da4700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f68300b0d10 0x7f6830146bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:11.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.054+0000 7f6834da4700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68300b12d0 con 0x7f68300b0560 2022-08-01T19:42:11.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.054+0000 7f6834da4700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68300a9700 con 0x7f68300a9320 2022-08-01T19:42:11.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.054+0000 7f6834da4700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68300b1500 con 0x7f68300b0d10 2022-08-01T19:42:11.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.054+0000 7f682ffff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f68300b0560 0x7f6830140350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35520/0 (socket says 172.21.15.17:35520) 2022-08-01T19:42:11.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.054+0000 7f682ffff700 1 -- 172.21.15.17:0/1399480333 learned_addr learned my addr 172.21.15.17:0/1399480333 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:11.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.054+0000 7f682f7fe700 1 --2- 172.21.15.17:0/1399480333 >> v2:172.21.15.77:3300/0 conn(0x7f68300a9320 0x7f68301466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:11.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.054+0000 7f6827fff700 1 --2- 172.21.15.17:0/1399480333 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f68300b0d10 0x7f6830146bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:11.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.054+0000 7f682effd700 1 -- 172.21.15.17:0/1399480333 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 966258698 0 0) 0x7f68300b12d0 con 0x7f68300b0560 2022-08-01T19:42:11.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.054+0000 7f682effd700 1 -- 172.21.15.17:0/1399480333 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6810003260 con 0x7f68300b0560 2022-08-01T19:42:11.076 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.054+0000 7f682f7fe700 1 -- 172.21.15.17:0/1399480333 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f68300b0d10 msgr2=0x7f6830146bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:11.076 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.054+0000 7f682f7fe700 1 --2- 172.21.15.17:0/1399480333 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f68300b0d10 0x7f6830146bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:11.076 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.054+0000 7f682f7fe700 1 -- 172.21.15.17:0/1399480333 >> v1:172.21.15.17:6789/0 conn(0x7f68300b0560 legacy=0x7f6830140350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:11.077 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.054+0000 7f682f7fe700 1 -- 172.21.15.17:0/1399480333 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68300b17a0 con 0x7f68300a9320 2022-08-01T19:42:11.077 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.054+0000 7f682f7fe700 1 --2- 172.21.15.17:0/1399480333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68300a9320 0x7f68301466e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f6820002f70 tx=0x7f6820006df0).ready entity=mon.1 client_cookie=14991001c76ffd5c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:11.078 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.058+0000 7f682effd700 1 -- 172.21.15.17:0/1399480333 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f682000d4a0 con 0x7f68300a9320 2022-08-01T19:42:11.079 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.058+0000 7f682effd700 1 -- 172.21.15.17:0/1399480333 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6820000e90 con 0x7f68300a9320 2022-08-01T19:42:11.079 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.058+0000 7f682effd700 1 -- 172.21.15.17:0/1399480333 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f682000d7f0 con 0x7f68300a9320 2022-08-01T19:42:11.079 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.058+0000 7f6834da4700 1 -- 172.21.15.17:0/1399480333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68300a9320 msgr2=0x7f68301466e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:11.080 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.058+0000 7f6834da4700 1 --2- 172.21.15.17:0/1399480333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68300a9320 0x7f68301466e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f6820002f70 tx=0x7f6820006df0).stop 2022-08-01T19:42:11.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.058+0000 7f6834da4700 1 -- 172.21.15.17:0/1399480333 shutdown_connections 2022-08-01T19:42:11.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.058+0000 7f6834da4700 1 --2- 172.21.15.17:0/1399480333 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f68300b0d10 0x7f6830146bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:11.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.058+0000 7f6834da4700 1 --2- 172.21.15.17:0/1399480333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68300a9320 0x7f68301466e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:11.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.058+0000 7f6834da4700 1 -- 172.21.15.17:0/1399480333 >> 172.21.15.17:0/1399480333 conn(0x7f68300612d0 msgr2=0x7f6830062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:11.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.058+0000 7f6834da4700 1 -- 172.21.15.17:0/1399480333 shutdown_connections 2022-08-01T19:42:11.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.058+0000 7f6834da4700 1 -- 172.21.15.17:0/1399480333 wait complete. 2022-08-01T19:42:11.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.058+0000 7f6834da4700 1 Processor -- start 2022-08-01T19:42:11.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.058+0000 7f6834da4700 1 -- start start 2022-08-01T19:42:11.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.062+0000 7f6834da4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68300a9320 0x7f68300673d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:11.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.062+0000 7f6834da4700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f68300b0560 0x7f68301559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:11.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.062+0000 7f6834da4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68300b0d10 0x7f6830155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:11.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.062+0000 7f6834da4700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6830152ef0 con 0x7f68300b0d10 2022-08-01T19:42:11.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.062+0000 7f6834da4700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68300b5ea0 con 0x7f68300a9320 2022-08-01T19:42:11.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.062+0000 7f6834da4700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6830153120 con 0x7f68300b0560 2022-08-01T19:42:11.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.062+0000 7f682ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68300b0d10 0x7f6830155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:11.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.062+0000 7f682ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68300b0d10 0x7f6830155e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43332/0 (socket says 172.21.15.17:43332) 2022-08-01T19:42:11.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.062+0000 7f682ffff700 1 -- 172.21.15.17:0/834180273 learned_addr learned my addr 172.21.15.17:0/834180273 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:11.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.062+0000 7f6827fff700 1 --2- 172.21.15.17:0/834180273 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f68300b0560 0x7f68301559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:11.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.062+0000 7f682f7fe700 1 --2- 172.21.15.17:0/834180273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68300a9320 0x7f68300673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:11.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.062+0000 7f682ffff700 1 -- 172.21.15.17:0/834180273 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f68300b0560 msgr2=0x7f68301559a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:11.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.062+0000 7f682ffff700 1 --2- 172.21.15.17:0/834180273 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f68300b0560 0x7f68301559a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:11.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.062+0000 7f682ffff700 1 -- 172.21.15.17:0/834180273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68300a9320 msgr2=0x7f68300673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:11.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.062+0000 7f682ffff700 1 --2- 172.21.15.17:0/834180273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68300a9320 0x7f68300673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:11.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.062+0000 7f682ffff700 1 -- 172.21.15.17:0/834180273 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f682000b180 con 0x7f68300b0d10 2022-08-01T19:42:11.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.062+0000 7f682ffff700 1 --2- 172.21.15.17:0/834180273 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68300b0d10 0x7f6830155e90 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 rx=0x7f6828001490 tx=0x7f6828007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:11.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.062+0000 7f682d7fa700 1 -- 172.21.15.17:0/834180273 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6828005ba0 con 0x7f68300b0d10 2022-08-01T19:42:11.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.062+0000 7f682d7fa700 1 -- 172.21.15.17:0/834180273 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6828005d00 con 0x7f68300b0d10 2022-08-01T19:42:11.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.062+0000 7f6834da4700 1 -- 172.21.15.17:0/834180273 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6830067c10 con 0x7f68300b0d10 2022-08-01T19:42:11.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.062+0000 7f682d7fa700 1 -- 172.21.15.17:0/834180273 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68280072c0 con 0x7f68300b0d10 2022-08-01T19:42:11.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.062+0000 7f6834da4700 1 -- 172.21.15.17:0/834180273 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68300b58e0 con 0x7f68300b0d10 2022-08-01T19:42:11.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.066+0000 7f6834da4700 1 -- 172.21.15.17:0/834180273 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68301532e0 con 0x7f68300b0d10 2022-08-01T19:42:11.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.066+0000 7f682d7fa700 1 -- 172.21.15.17:0/834180273 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6828010070 con 0x7f68300b0d10 2022-08-01T19:42:11.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.066+0000 7f682d7fa700 1 --2- 172.21.15.17:0/834180273 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f68140539e0 0x7f6814055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:11.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.066+0000 7f682d7fa700 1 -- 172.21.15.17:0/834180273 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f682806bf70 con 0x7f68300b0d10 2022-08-01T19:42:11.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.066+0000 7f682f7fe700 1 --2- 172.21.15.17:0/834180273 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f68140539e0 0x7f6814055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:11.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.066+0000 7f682f7fe700 1 --2- 172.21.15.17:0/834180273 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f68140539e0 0x7f6814055ea0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f682001a000 tx=0x7f6820014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:11.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.070+0000 7f682d7fa700 1 -- 172.21.15.17:0/834180273 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6828042d30 con 0x7f68300b0d10 2022-08-01T19:42:11.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.198+0000 7f6834da4700 1 -- 172.21.15.17:0/834180273 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6830063b00 con 0x7f68300b0d10 2022-08-01T19:42:11.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.198+0000 7f682d7fa700 1 -- 172.21.15.17:0/834180273 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f6828046350 con 0x7f68300b0d10 2022-08-01T19:42:11.203 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:42:11.204 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":116,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:42:10.905772+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:42:09.404661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2022-08-01T19:42:10.264672+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:42:11.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.202+0000 7f6834da4700 1 -- 172.21.15.17:0/834180273 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f68140539e0 msgr2=0x7f6814055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:11.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.202+0000 7f6834da4700 1 --2- 172.21.15.17:0/834180273 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f68140539e0 0x7f6814055ea0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f682001a000 tx=0x7f6820014000).stop 2022-08-01T19:42:11.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.202+0000 7f6834da4700 1 -- 172.21.15.17:0/834180273 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68300b0d10 msgr2=0x7f6830155e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:11.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.202+0000 7f6834da4700 1 --2- 172.21.15.17:0/834180273 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68300b0d10 0x7f6830155e90 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 rx=0x7f6828001490 tx=0x7f6828007e00).stop 2022-08-01T19:42:11.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.202+0000 7f6834da4700 1 -- 172.21.15.17:0/834180273 shutdown_connections 2022-08-01T19:42:11.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.202+0000 7f6834da4700 1 --2- 172.21.15.17:0/834180273 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f68140539e0 0x7f6814055ea0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:11.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.202+0000 7f6834da4700 1 --2- 172.21.15.17:0/834180273 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68300b0d10 0x7f6830155e90 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:11.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.202+0000 7f6834da4700 1 --2- 172.21.15.17:0/834180273 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f68300b0560 0x7f68301559a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:11.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.202+0000 7f6834da4700 1 --2- 172.21.15.17:0/834180273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68300a9320 0x7f68300673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:11.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.202+0000 7f6834da4700 1 -- 172.21.15.17:0/834180273 >> 172.21.15.17:0/834180273 conn(0x7f68300612d0 msgr2=0x7f683013dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:11.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.202+0000 7f6834da4700 1 -- 172.21.15.17:0/834180273 shutdown_connections 2022-08-01T19:42:11.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.206+0000 7f6834da4700 1 -- 172.21.15.17:0/834180273 wait complete. 2022-08-01T19:42:11.237 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:42:11.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.346+0000 7f06de364700 1 Processor -- start 2022-08-01T19:42:11.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.346+0000 7f06de364700 1 -- start start 2022-08-01T19:42:11.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.346+0000 7f06de364700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f06d80b0d10 0x7f06d8146df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:11.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.346+0000 7f06de364700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06d80b12d0 con 0x7f06d80b0560 2022-08-01T19:42:11.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.346+0000 7f06de364700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06d80a9700 con 0x7f06d80a9320 2022-08-01T19:42:11.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.346+0000 7f06de364700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06d80b1500 con 0x7f06d80b0d10 2022-08-01T19:42:11.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.350+0000 7f06dcb61700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f06d80b0d10 0x7f06d8146df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:11.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.350+0000 7f06dcb61700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f06d80b0d10 0x7f06d8146df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43338/0 (socket says 172.21.15.17:43338) 2022-08-01T19:42:11.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.350+0000 7f06ddb63700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f06d80b0560 0x7f06d8140350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35534/0 (socket says 172.21.15.17:35534) 2022-08-01T19:42:11.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.350+0000 7f06dcb61700 1 -- 172.21.15.17:0/3046453540 learned_addr learned my addr 172.21.15.17:0/3046453540 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:11.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.350+0000 7f06dcb61700 1 -- 172.21.15.17:0/3046453540 >> v1:172.21.15.77:6789/0 conn(0x7f06d80a9320 legacy=0x7f06d81466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:11.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.350+0000 7f06dcb61700 1 -- 172.21.15.17:0/3046453540 >> v1:172.21.15.17:6789/0 conn(0x7f06d80b0560 legacy=0x7f06d8140350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:11.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.350+0000 7f06dcb61700 1 -- 172.21.15.17:0/3046453540 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d80b17a0 con 0x7f06d80b0d10 2022-08-01T19:42:11.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.350+0000 7f06dcb61700 1 --2- 172.21.15.17:0/3046453540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06d80b0d10 0x7f06d8146df0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 rx=0x7f06c8002f70 tx=0x7f06c8006eb0).ready entity=mon.0 client_cookie=8766e6ea4bead412 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:11.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.350+0000 7f06c7fff700 1 -- 172.21.15.17:0/3046453540 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06c800d660 con 0x7f06d80b0d10 2022-08-01T19:42:11.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.350+0000 7f06c7fff700 1 -- 172.21.15.17:0/3046453540 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c8000e90 con 0x7f06d80b0d10 2022-08-01T19:42:11.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.350+0000 7f06c7fff700 1 -- 172.21.15.17:0/3046453540 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06c800d9f0 con 0x7f06d80b0d10 2022-08-01T19:42:11.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.350+0000 7f06de364700 1 -- 172.21.15.17:0/3046453540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06d80b0d10 msgr2=0x7f06d8146df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:11.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.350+0000 7f06de364700 1 --2- 172.21.15.17:0/3046453540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06d80b0d10 0x7f06d8146df0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 rx=0x7f06c8002f70 tx=0x7f06c8006eb0).stop 2022-08-01T19:42:11.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.350+0000 7f06de364700 1 -- 172.21.15.17:0/3046453540 shutdown_connections 2022-08-01T19:42:11.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.350+0000 7f06de364700 1 --2- 172.21.15.17:0/3046453540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06d80b0d10 0x7f06d8146df0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:11.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.350+0000 7f06de364700 1 -- 172.21.15.17:0/3046453540 >> 172.21.15.17:0/3046453540 conn(0x7f06d80612d0 msgr2=0x7f06d8062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:11.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.350+0000 7f06de364700 1 -- 172.21.15.17:0/3046453540 shutdown_connections 2022-08-01T19:42:11.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.350+0000 7f06de364700 1 -- 172.21.15.17:0/3046453540 wait complete. 2022-08-01T19:42:11.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.350+0000 7f06de364700 1 Processor -- start 2022-08-01T19:42:11.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.350+0000 7f06de364700 1 -- start start 2022-08-01T19:42:11.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.354+0000 7f06de364700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06d80a9320 0x7f06d80b6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:11.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.354+0000 7f06de364700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06d80b0560 0x7f06d813f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:11.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.354+0000 7f06de364700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f06d80b0d10 0x7f06d80b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:11.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.354+0000 7f06de364700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d80b59b0 con 0x7f06d80a9320 2022-08-01T19:42:11.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.354+0000 7f06de364700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d81463c0 con 0x7f06d80b0560 2022-08-01T19:42:11.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.354+0000 7f06de364700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d80b5be0 con 0x7f06d80b0d10 2022-08-01T19:42:11.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.354+0000 7f06dd362700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06d80a9320 0x7f06d80b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:11.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.354+0000 7f06dd362700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06d80a9320 0x7f06d80b6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43342/0 (socket says 172.21.15.17:43342) 2022-08-01T19:42:11.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.354+0000 7f06dd362700 1 -- 172.21.15.17:0/140499194 learned_addr learned my addr 172.21.15.17:0/140499194 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:11.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.354+0000 7f06ddb63700 1 --2- 172.21.15.17:0/140499194 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f06d80b0d10 0x7f06d80b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:11.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.354+0000 7f06dcb61700 1 --2- 172.21.15.17:0/140499194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06d80b0560 0x7f06d813f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:11.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.354+0000 7f06dd362700 1 -- 172.21.15.17:0/140499194 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f06d80b0d10 msgr2=0x7f06d80b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:11.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.354+0000 7f06dd362700 1 --2- 172.21.15.17:0/140499194 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f06d80b0d10 0x7f06d80b37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:11.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.354+0000 7f06dd362700 1 -- 172.21.15.17:0/140499194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06d80b0560 msgr2=0x7f06d813f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:11.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.354+0000 7f06dd362700 1 --2- 172.21.15.17:0/140499194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06d80b0560 0x7f06d813f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:11.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.354+0000 7f06dd362700 1 -- 172.21.15.17:0/140499194 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c800b1e0 con 0x7f06d80a9320 2022-08-01T19:42:11.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.354+0000 7f06dcb61700 1 --2- 172.21.15.17:0/140499194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06d80b0560 0x7f06d813f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:42:11.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.354+0000 7f06dd362700 1 --2- 172.21.15.17:0/140499194 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06d80a9320 0x7f06d80b6690 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 rx=0x7f06d80b0420 tx=0x7f06d4007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:11.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.354+0000 7f06c67fc700 1 -- 172.21.15.17:0/140499194 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06d400ce30 con 0x7f06d80a9320 2022-08-01T19:42:11.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.354+0000 7f06c67fc700 1 -- 172.21.15.17:0/140499194 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d400f470 con 0x7f06d80a9320 2022-08-01T19:42:11.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.354+0000 7f06de364700 1 -- 172.21.15.17:0/140499194 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06d80b5e80 con 0x7f06d80a9320 2022-08-01T19:42:11.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.354+0000 7f06c67fc700 1 -- 172.21.15.17:0/140499194 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06d400f7d0 con 0x7f06d80a9320 2022-08-01T19:42:11.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.354+0000 7f06de364700 1 -- 172.21.15.17:0/140499194 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06d80ab1e0 con 0x7f06d80a9320 2022-08-01T19:42:11.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.354+0000 7f06de364700 1 -- 172.21.15.17:0/140499194 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06d80b6320 con 0x7f06d80a9320 2022-08-01T19:42:11.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.358+0000 7f06c67fc700 1 -- 172.21.15.17:0/140499194 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f06d4019030 con 0x7f06d80a9320 2022-08-01T19:42:11.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.358+0000 7f06c67fc700 1 --2- 172.21.15.17:0/140499194 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f06b80539e0 0x7f06b8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:11.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.358+0000 7f06c67fc700 1 -- 172.21.15.17:0/140499194 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f06d406b3e0 con 0x7f06d80a9320 2022-08-01T19:42:11.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.358+0000 7f06dcb61700 1 --2- 172.21.15.17:0/140499194 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f06b80539e0 0x7f06b8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:11.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.358+0000 7f06dcb61700 1 --2- 172.21.15.17:0/140499194 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f06b80539e0 0x7f06b8055ea0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f06c8017000 tx=0x7f06c8015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:11.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.362+0000 7f06c67fc700 1 -- 172.21.15.17:0/140499194 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f06d80b6320 con 0x7f06d80a9320 2022-08-01T19:42:11.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.494+0000 7f06de364700 1 -- 172.21.15.17:0/140499194 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06d80574e0 con 0x7f06d80a9320 2022-08-01T19:42:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.498+0000 7f06c67fc700 1 -- 172.21.15.17:0/140499194 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f06d40421a0 con 0x7f06d80a9320 2022-08-01T19:42:11.500 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:42:11.501 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":116,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:42:10.905772+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:42:09.404661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2022-08-01T19:42:10.264672+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:42:11.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.498+0000 7f06de364700 1 -- 172.21.15.17:0/140499194 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f06b80539e0 msgr2=0x7f06b8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:11.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.498+0000 7f06de364700 1 --2- 172.21.15.17:0/140499194 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f06b80539e0 0x7f06b8055ea0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f06c8017000 tx=0x7f06c8015000).stop 2022-08-01T19:42:11.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.498+0000 7f06de364700 1 -- 172.21.15.17:0/140499194 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06d80a9320 msgr2=0x7f06d80b6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:11.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.498+0000 7f06de364700 1 --2- 172.21.15.17:0/140499194 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06d80a9320 0x7f06d80b6690 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 rx=0x7f06d80b0420 tx=0x7f06d4007c50).stop 2022-08-01T19:42:11.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.498+0000 7f06de364700 1 -- 172.21.15.17:0/140499194 shutdown_connections 2022-08-01T19:42:11.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.498+0000 7f06de364700 1 --2- 172.21.15.17:0/140499194 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f06b80539e0 0x7f06b8055ea0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:11.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.498+0000 7f06de364700 1 --2- 172.21.15.17:0/140499194 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f06d80b0d10 0x7f06d80b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:11.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.498+0000 7f06de364700 1 --2- 172.21.15.17:0/140499194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06d80b0560 0x7f06d813f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:11.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.498+0000 7f06de364700 1 --2- 172.21.15.17:0/140499194 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06d80a9320 0x7f06d80b6690 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:11.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.498+0000 7f06de364700 1 -- 172.21.15.17:0/140499194 >> 172.21.15.17:0/140499194 conn(0x7f06d80612d0 msgr2=0x7f06d8141d40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:11.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.498+0000 7f06de364700 1 -- 172.21.15.17:0/140499194 shutdown_connections 2022-08-01T19:42:11.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.498+0000 7f06de364700 1 -- 172.21.15.17:0/140499194 wait complete. 2022-08-01T19:42:11.517 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-08-01T19:42:11.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.634+0000 7f460c897700 1 Processor -- start 2022-08-01T19:42:11.636 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.634+0000 7f460c897700 1 -- start start 2022-08-01T19:42:11.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.634+0000 7f460c897700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46080a9320 0x7f46081466e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:11.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.634+0000 7f460c897700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f46080b0d10 0x7f4608146bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:11.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.634+0000 7f460c897700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46080b12d0 con 0x7f46080b0560 2022-08-01T19:42:11.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.634+0000 7f460c897700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46080a9700 con 0x7f46080b0d10 2022-08-01T19:42:11.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.634+0000 7f460c897700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46080b1500 con 0x7f46080a9320 2022-08-01T19:42:11.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.634+0000 7f460759e700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f46080b0560 0x7f4608140350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35548/0 (socket says 172.21.15.17:35548) 2022-08-01T19:42:11.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.634+0000 7f460759e700 1 -- 172.21.15.17:0/625280565 learned_addr learned my addr 172.21.15.17:0/625280565 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:11.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.634+0000 7f4606d9d700 1 --2- 172.21.15.17:0/625280565 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46080a9320 0x7f46081466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:11.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.634+0000 7f460659c700 1 --2- 172.21.15.17:0/625280565 >> v2:172.21.15.77:3300/0 conn(0x7f46080b0d10 0x7f4608146bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:11.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.634+0000 7f4605d9b700 1 -- 172.21.15.17:0/625280565 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2173968741 0 0) 0x7f46080b12d0 con 0x7f46080b0560 2022-08-01T19:42:11.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.634+0000 7f4605d9b700 1 -- 172.21.15.17:0/625280565 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45ec003260 con 0x7f46080b0560 2022-08-01T19:42:11.645 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.634+0000 7f460659c700 1 -- 172.21.15.17:0/625280565 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46080a9320 msgr2=0x7f46081466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:11.645 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.634+0000 7f460659c700 1 --2- 172.21.15.17:0/625280565 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46080a9320 0x7f46081466e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:11.645 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.634+0000 7f460659c700 1 -- 172.21.15.17:0/625280565 >> v1:172.21.15.17:6789/0 conn(0x7f46080b0560 legacy=0x7f4608140350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:11.645 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.634+0000 7f460659c700 1 -- 172.21.15.17:0/625280565 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46080b17a0 con 0x7f46080b0d10 2022-08-01T19:42:11.646 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.634+0000 7f460659c700 1 --2- 172.21.15.17:0/625280565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46080b0d10 0x7f4608146bd0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f45fc002f70 tx=0x7f45fc006df0).ready entity=mon.1 client_cookie=53000ba783fb70a7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:11.646 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.638+0000 7f4605d9b700 1 -- 172.21.15.17:0/625280565 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45fc00d4a0 con 0x7f46080b0d10 2022-08-01T19:42:11.646 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.638+0000 7f4605d9b700 1 -- 172.21.15.17:0/625280565 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45fc000e90 con 0x7f46080b0d10 2022-08-01T19:42:11.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.638+0000 7f4605d9b700 1 -- 172.21.15.17:0/625280565 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45fc00d7f0 con 0x7f46080b0d10 2022-08-01T19:42:11.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.638+0000 7f460c897700 1 -- 172.21.15.17:0/625280565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46080b0d10 msgr2=0x7f4608146bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:11.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.638+0000 7f460c897700 1 --2- 172.21.15.17:0/625280565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46080b0d10 0x7f4608146bd0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f45fc002f70 tx=0x7f45fc006df0).stop 2022-08-01T19:42:11.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.638+0000 7f460c897700 1 -- 172.21.15.17:0/625280565 shutdown_connections 2022-08-01T19:42:11.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.638+0000 7f460c897700 1 --2- 172.21.15.17:0/625280565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46080b0d10 0x7f4608146bd0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:11.648 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.638+0000 7f460c897700 1 --2- 172.21.15.17:0/625280565 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46080a9320 0x7f46081466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:11.648 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.638+0000 7f460c897700 1 -- 172.21.15.17:0/625280565 >> 172.21.15.17:0/625280565 conn(0x7f46080612d0 msgr2=0x7f4608062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:11.648 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.638+0000 7f460c897700 1 -- 172.21.15.17:0/625280565 shutdown_connections 2022-08-01T19:42:11.649 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.638+0000 7f460c897700 1 -- 172.21.15.17:0/625280565 wait complete. 2022-08-01T19:42:11.649 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.638+0000 7f460c897700 1 Processor -- start 2022-08-01T19:42:11.649 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.638+0000 7f460c897700 1 -- start start 2022-08-01T19:42:11.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.642+0000 7f460c897700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46080a9320 0x7f46080af5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:11.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.642+0000 7f460c897700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46080b0560 0x7f4608152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:11.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.642+0000 7f460c897700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46080b0d10 0x7f46080a99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:11.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.642+0000 7f460c897700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4608143990 con 0x7f46080b0560 2022-08-01T19:42:11.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.642+0000 7f460c897700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4608152360 con 0x7f46080b0d10 2022-08-01T19:42:11.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.642+0000 7f460c897700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4608152600 con 0x7f46080a9320 2022-08-01T19:42:11.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.642+0000 7f460659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46080b0560 0x7f4608152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:11.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.642+0000 7f460659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46080b0560 0x7f4608152d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43358/0 (socket says 172.21.15.17:43358) 2022-08-01T19:42:11.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.642+0000 7f460659c700 1 -- 172.21.15.17:0/479026716 learned_addr learned my addr 172.21.15.17:0/479026716 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:11.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.642+0000 7f4606d9d700 1 --2- 172.21.15.17:0/479026716 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46080a9320 0x7f46080af5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:11.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.642+0000 7f460759e700 1 --2- 172.21.15.17:0/479026716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46080b0d10 0x7f46080a99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:11.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.642+0000 7f460659c700 1 -- 172.21.15.17:0/479026716 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46080a9320 msgr2=0x7f46080af5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:11.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.642+0000 7f460659c700 1 --2- 172.21.15.17:0/479026716 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46080a9320 0x7f46080af5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:11.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.642+0000 7f460659c700 1 -- 172.21.15.17:0/479026716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46080b0d10 msgr2=0x7f46080a99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:11.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.642+0000 7f460659c700 1 --2- 172.21.15.17:0/479026716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46080b0d10 0x7f46080a99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:11.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.642+0000 7f460659c700 1 -- 172.21.15.17:0/479026716 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45fc00b180 con 0x7f46080b0560 2022-08-01T19:42:11.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.642+0000 7f460659c700 1 --2- 172.21.15.17:0/479026716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46080b0560 0x7f4608152d40 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 rx=0x7f45fc002f70 tx=0x7f45fc014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:11.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.642+0000 7f45f7fff700 1 -- 172.21.15.17:0/479026716 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45fc016070 con 0x7f46080b0560 2022-08-01T19:42:11.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.642+0000 7f45f7fff700 1 -- 172.21.15.17:0/479026716 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45fc0137d0 con 0x7f46080b0560 2022-08-01T19:42:11.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.642+0000 7f460c897700 1 -- 172.21.15.17:0/479026716 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46081528a0 con 0x7f46080b0560 2022-08-01T19:42:11.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.642+0000 7f45f7fff700 1 -- 172.21.15.17:0/479026716 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45fc013b30 con 0x7f46080b0560 2022-08-01T19:42:11.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.642+0000 7f460c897700 1 -- 172.21.15.17:0/479026716 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46080aae40 con 0x7f46080b0560 2022-08-01T19:42:11.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.646+0000 7f45f7fff700 1 -- 172.21.15.17:0/479026716 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f45fc025030 con 0x7f46080b0560 2022-08-01T19:42:11.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.646+0000 7f460c897700 1 -- 172.21.15.17:0/479026716 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46080574e0 con 0x7f46080b0560 2022-08-01T19:42:11.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.646+0000 7f45f7fff700 1 --2- 172.21.15.17:0/479026716 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f45e4053950 0x7f45e4055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:11.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.646+0000 7f45f7fff700 1 -- 172.21.15.17:0/479026716 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f45fc06cf50 con 0x7f46080b0560 2022-08-01T19:42:11.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.646+0000 7f4606d9d700 1 --2- 172.21.15.17:0/479026716 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f45e4053950 0x7f45e4055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:11.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.646+0000 7f4606d9d700 1 --2- 172.21.15.17:0/479026716 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f45e4053950 0x7f45e4055e10 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f45f800b000 tx=0x7f45f80091d0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:11.660 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.654+0000 7f45f7fff700 1 -- 172.21.15.17:0/479026716 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f45fc043d10 con 0x7f46080b0560 2022-08-01T19:42:11.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.818+0000 7f460c897700 1 -- 172.21.15.17:0/479026716 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f46081475c0 con 0x7f46080b0560 2022-08-01T19:42:11.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.938+0000 7f45f7fff700 1 -- 172.21.15.17:0/479026716 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v58) v1 ==== 118+0+0 (secure 0 0 0) 0x7f45fc0230b0 con 0x7f46080b0560 2022-08-01T19:42:11.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.938+0000 7f460c897700 1 -- 172.21.15.17:0/479026716 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f45e4053950 msgr2=0x7f45e4055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:11.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.938+0000 7f460c897700 1 --2- 172.21.15.17:0/479026716 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f45e4053950 0x7f45e4055e10 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f45f800b000 tx=0x7f45f80091d0).stop 2022-08-01T19:42:11.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.938+0000 7f460c897700 1 -- 172.21.15.17:0/479026716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46080b0560 msgr2=0x7f4608152d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:11.944 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.938+0000 7f460c897700 1 --2- 172.21.15.17:0/479026716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46080b0560 0x7f4608152d40 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 rx=0x7f45fc002f70 tx=0x7f45fc014000).stop 2022-08-01T19:42:11.944 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.938+0000 7f460c897700 1 -- 172.21.15.17:0/479026716 shutdown_connections 2022-08-01T19:42:11.945 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.938+0000 7f460c897700 1 --2- 172.21.15.17:0/479026716 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f45e4053950 0x7f45e4055e10 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:11.946 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.938+0000 7f460c897700 1 --2- 172.21.15.17:0/479026716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46080b0d10 0x7f46080a99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:11.946 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.938+0000 7f460c897700 1 --2- 172.21.15.17:0/479026716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46080b0560 0x7f4608152d40 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:11.947 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.938+0000 7f460c897700 1 --2- 172.21.15.17:0/479026716 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46080a9320 0x7f46080af5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:11.947 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.938+0000 7f460c897700 1 -- 172.21.15.17:0/479026716 >> 172.21.15.17:0/479026716 conn(0x7f46080612d0 msgr2=0x7f4608146520 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:11.947 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.938+0000 7f460c897700 1 -- 172.21.15.17:0/479026716 shutdown_connections 2022-08-01T19:42:11.948 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:11.938+0000 7f460c897700 1 -- 172.21.15.17:0/479026716 wait complete. 2022-08-01T19:42:11.955 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:42:12.077 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.062+0000 7fd394964700 1 Processor -- start 2022-08-01T19:42:12.079 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.062+0000 7fd394964700 1 -- start start 2022-08-01T19:42:12.079 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.066+0000 7fd394964700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fd3900b3190 0x7fd390060ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:12.080 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.066+0000 7fd394964700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd390061190 0x7fd390066dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:12.080 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.066+0000 7fd394964700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3900654c0 con 0x7fd3900b2730 2022-08-01T19:42:12.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.066+0000 7fd394964700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd390062190 con 0x7fd390061190 2022-08-01T19:42:12.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.066+0000 7fd394964700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd390061600 con 0x7fd3900b3190 2022-08-01T19:42:12.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.066+0000 7fd38f59e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd390061190 0x7fd390066dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:12.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.066+0000 7fd38f59e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd390061190 0x7fd390066dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43368/0 (socket says 172.21.15.17:43368) 2022-08-01T19:42:12.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.066+0000 7fd38f59e700 1 -- 172.21.15.17:0/445241949 learned_addr learned my addr 172.21.15.17:0/445241949 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:12.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.066+0000 7fd38ed9d700 1 --2- 172.21.15.17:0/445241949 >> v2:172.21.15.77:3300/0 conn(0x7fd3900b3190 0x7fd390060ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:12.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.066+0000 7fd38f59e700 1 -- 172.21.15.17:0/445241949 >> v2:172.21.15.77:3300/0 conn(0x7fd3900b3190 msgr2=0x7fd390060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:12.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.066+0000 7fd38f59e700 1 --2- 172.21.15.17:0/445241949 >> v2:172.21.15.77:3300/0 conn(0x7fd3900b3190 0x7fd390060ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:12.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.066+0000 7fd38f59e700 1 -- 172.21.15.17:0/445241949 >> v1:172.21.15.77:6789/0 conn(0x7fd3900b2730 legacy=0x7fd3900666b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:12.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.066+0000 7fd38e59c700 1 -- 172.21.15.17:0/445241949 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2549724191 0 0) 0x7fd3900654c0 con 0x7fd3900b2730 2022-08-01T19:42:12.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.066+0000 7fd38f59e700 1 -- 172.21.15.17:0/445241949 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3900b3570 con 0x7fd390061190 2022-08-01T19:42:12.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.066+0000 7fd38f59e700 1 --2- 172.21.15.17:0/445241949 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd390061190 0x7fd390066dc0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 rx=0x7fd3880032f0 tx=0x7fd388009290).ready entity=mon.0 client_cookie=bebea8efb3927639 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:12.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.066+0000 7fd38e59c700 1 -- 172.21.15.17:0/445241949 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd38800b870 con 0x7fd390061190 2022-08-01T19:42:12.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.066+0000 7fd38e59c700 1 -- 172.21.15.17:0/445241949 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd38800b9d0 con 0x7fd390061190 2022-08-01T19:42:12.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.066+0000 7fd38e59c700 1 -- 172.21.15.17:0/445241949 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd38800bb40 con 0x7fd390061190 2022-08-01T19:42:12.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.066+0000 7fd394964700 1 -- 172.21.15.17:0/445241949 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd390061190 msgr2=0x7fd390066dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:12.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.066+0000 7fd394964700 1 --2- 172.21.15.17:0/445241949 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd390061190 0x7fd390066dc0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 rx=0x7fd3880032f0 tx=0x7fd388009290).stop 2022-08-01T19:42:12.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.066+0000 7fd394964700 1 -- 172.21.15.17:0/445241949 shutdown_connections 2022-08-01T19:42:12.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.066+0000 7fd394964700 1 --2- 172.21.15.17:0/445241949 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd390061190 0x7fd390066dc0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:12.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.066+0000 7fd394964700 1 --2- 172.21.15.17:0/445241949 >> v2:172.21.15.77:3300/0 conn(0x7fd3900b3190 0x7fd390060ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:12.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.066+0000 7fd394964700 1 -- 172.21.15.17:0/445241949 >> 172.21.15.17:0/445241949 conn(0x7fd390063870 msgr2=0x7fd390063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:12.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.066+0000 7fd394964700 1 -- 172.21.15.17:0/445241949 shutdown_connections 2022-08-01T19:42:12.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.066+0000 7fd394964700 1 -- 172.21.15.17:0/445241949 wait complete. 2022-08-01T19:42:12.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.070+0000 7fd394964700 1 Processor -- start 2022-08-01T19:42:12.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.070+0000 7fd394964700 1 -- start start 2022-08-01T19:42:12.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.070+0000 7fd394964700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd390061190 0x7fd390152e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:12.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.070+0000 7fd394964700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3900b2730 0x7fd390141130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:12.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.070+0000 7fd394964700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd3900b3190 0x7fd39013e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:12.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.070+0000 7fd394964700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3900a5910 con 0x7fd390061190 2022-08-01T19:42:12.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.070+0000 7fd394964700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3900a5bb0 con 0x7fd3900b2730 2022-08-01T19:42:12.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.070+0000 7fd394964700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3900a5e50 con 0x7fd3900b3190 2022-08-01T19:42:12.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.070+0000 7fd38ed9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd390061190 0x7fd390152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:12.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.070+0000 7fd38ed9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd390061190 0x7fd390152e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43370/0 (socket says 172.21.15.17:43370) 2022-08-01T19:42:12.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.070+0000 7fd38ed9d700 1 -- 172.21.15.17:0/2766603894 learned_addr learned my addr 172.21.15.17:0/2766603894 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:12.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.070+0000 7fd387fff700 1 --2- 172.21.15.17:0/2766603894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3900b2730 0x7fd390141130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:12.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.070+0000 7fd38f59e700 1 --2- 172.21.15.17:0/2766603894 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd3900b3190 0x7fd39013e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:12.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.070+0000 7fd38ed9d700 1 -- 172.21.15.17:0/2766603894 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd3900b3190 msgr2=0x7fd39013e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:12.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.070+0000 7fd38ed9d700 1 --2- 172.21.15.17:0/2766603894 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd3900b3190 0x7fd39013e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:12.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.070+0000 7fd38ed9d700 1 -- 172.21.15.17:0/2766603894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3900b2730 msgr2=0x7fd390141130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:12.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.070+0000 7fd38ed9d700 1 --2- 172.21.15.17:0/2766603894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3900b2730 0x7fd390141130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:12.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.070+0000 7fd38ed9d700 1 -- 172.21.15.17:0/2766603894 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd388009530 con 0x7fd390061190 2022-08-01T19:42:12.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.070+0000 7fd38f59e700 1 --2- 172.21.15.17:0/2766603894 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd3900b3190 0x7fd39013e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:42:12.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.070+0000 7fd38ed9d700 1 --2- 172.21.15.17:0/2766603894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd390061190 0x7fd390152e30 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 rx=0x7fd38000c000 tx=0x7fd380006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:12.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.070+0000 7fd38cd99700 1 -- 172.21.15.17:0/2766603894 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd380010780 con 0x7fd390061190 2022-08-01T19:42:12.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.070+0000 7fd38cd99700 1 -- 172.21.15.17:0/2766603894 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3800108e0 con 0x7fd390061190 2022-08-01T19:42:12.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.070+0000 7fd394964700 1 -- 172.21.15.17:0/2766603894 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3900a62f0 con 0x7fd390061190 2022-08-01T19:42:12.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.070+0000 7fd38cd99700 1 -- 172.21.15.17:0/2766603894 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd380010c40 con 0x7fd390061190 2022-08-01T19:42:12.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.070+0000 7fd394964700 1 -- 172.21.15.17:0/2766603894 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd390153320 con 0x7fd390061190 2022-08-01T19:42:12.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.070+0000 7fd394964700 1 -- 172.21.15.17:0/2766603894 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd39005fbf0 con 0x7fd390061190 2022-08-01T19:42:12.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.074+0000 7fd38cd99700 1 -- 172.21.15.17:0/2766603894 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd38001c460 con 0x7fd390061190 2022-08-01T19:42:12.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.074+0000 7fd38cd99700 1 --2- 172.21.15.17:0/2766603894 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd374053a70 0x7fd374055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:12.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.074+0000 7fd38cd99700 1 -- 172.21.15.17:0/2766603894 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd38006c440 con 0x7fd390061190 2022-08-01T19:42:12.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.074+0000 7fd387fff700 1 --2- 172.21.15.17:0/2766603894 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd374053a70 0x7fd374055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:12.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.074+0000 7fd387fff700 1 --2- 172.21.15.17:0/2766603894 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd374053a70 0x7fd374055f30 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7fd37800d000 tx=0x7fd378007a00).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:12.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.078+0000 7fd38cd99700 1 -- 172.21.15.17:0/2766603894 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd3800431a0 con 0x7fd390061190 2022-08-01T19:42:12.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.242+0000 7fd394964700 1 -- 172.21.15.17:0/2766603894 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3900574e0 con 0x7fd390061190 2022-08-01T19:42:12.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.242+0000 7fd38cd99700 1 -- 172.21.15.17:0/2766603894 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+2492 (secure 0 0 0) 0x7fd3800467c0 con 0x7fd390061190 2022-08-01T19:42:12.248 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:42:12.249 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":58,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7581,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/1744640648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":1744640648},{"type":"v1","addr":"172.21.15.17:6833","nonce":1744640648}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":56},{"gid":7594,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.17:6835/346594655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":346594655},{"type":"v1","addr":"172.21.15.17:6835","nonce":346594655}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":57}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:42:10.906134+0000","modified":"2022-08-01T19:42:11.922099+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7600},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7600":{"gid":7600,"name":"b","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6835/2286085186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2286085186},{"type":"v1","addr":"172.21.15.77:6835","nonce":2286085186}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2022-08-01T19:42:12.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.246+0000 7fd394964700 1 -- 172.21.15.17:0/2766603894 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd374053a70 msgr2=0x7fd374055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:12.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.246+0000 7fd394964700 1 --2- 172.21.15.17:0/2766603894 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd374053a70 0x7fd374055f30 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7fd37800d000 tx=0x7fd378007a00).stop 2022-08-01T19:42:12.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.246+0000 7fd394964700 1 -- 172.21.15.17:0/2766603894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd390061190 msgr2=0x7fd390152e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:12.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.246+0000 7fd394964700 1 --2- 172.21.15.17:0/2766603894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd390061190 0x7fd390152e30 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 rx=0x7fd38000c000 tx=0x7fd380006df0).stop 2022-08-01T19:42:12.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.246+0000 7fd394964700 1 -- 172.21.15.17:0/2766603894 shutdown_connections 2022-08-01T19:42:12.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.246+0000 7fd394964700 1 --2- 172.21.15.17:0/2766603894 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd374053a70 0x7fd374055f30 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:12.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.246+0000 7fd394964700 1 --2- 172.21.15.17:0/2766603894 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd3900b3190 0x7fd39013e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:12.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.246+0000 7fd394964700 1 --2- 172.21.15.17:0/2766603894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3900b2730 0x7fd390141130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:12.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.246+0000 7fd394964700 1 --2- 172.21.15.17:0/2766603894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd390061190 0x7fd390152e30 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:12.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.246+0000 7fd394964700 1 -- 172.21.15.17:0/2766603894 >> 172.21.15.17:0/2766603894 conn(0x7fd390063870 msgr2=0x7fd390143a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:12.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.246+0000 7fd394964700 1 -- 172.21.15.17:0/2766603894 shutdown_connections 2022-08-01T19:42:12.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.246+0000 7fd394964700 1 -- 172.21.15.17:0/2766603894 wait complete. 2022-08-01T19:42:12.257 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 58 2022-08-01T19:42:12.261 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:42:12.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.366+0000 7f46abfff700 1 Processor -- start 2022-08-01T19:42:12.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.366+0000 7f46abfff700 1 -- start start 2022-08-01T19:42:12.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.366+0000 7f46abfff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f46ac0b2710 0x7f46ac14afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:12.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.366+0000 7f46abfff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46ac0a8c50 0x7f46ac14b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:12.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.366+0000 7f46abfff700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46ac0a7250 con 0x7f46ac0b01a0 2022-08-01T19:42:12.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.366+0000 7f46abfff700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46ac0a7480 con 0x7f46ac0b2710 2022-08-01T19:42:12.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.366+0000 7f46abfff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac0a7720 con 0x7f46ac0a8c50 2022-08-01T19:42:12.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.366+0000 7f46aa7fc700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f46ac0b01a0 0x7f46ac0a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35572/0 (socket says 172.21.15.17:35572) 2022-08-01T19:42:12.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.366+0000 7f46aa7fc700 1 -- 172.21.15.17:0/1735054445 learned_addr learned my addr 172.21.15.17:0/1735054445 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:12.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.366+0000 7f46aaffd700 1 --2- 172.21.15.17:0/1735054445 >> v2:172.21.15.17:3300/0 conn(0x7f46ac0b2710 0x7f46ac14afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:12.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.366+0000 7f46ab7fe700 1 --2- 172.21.15.17:0/1735054445 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46ac0a8c50 0x7f46ac14b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:12.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.366+0000 7f46a9ffb700 1 -- 172.21.15.17:0/1735054445 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2264739292 0 0) 0x7f46ac0a7250 con 0x7f46ac0b01a0 2022-08-01T19:42:12.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.366+0000 7f46a9ffb700 1 -- 172.21.15.17:0/1735054445 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4690003260 con 0x7f46ac0b01a0 2022-08-01T19:42:12.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.366+0000 7f46ab7fe700 1 -- 172.21.15.17:0/1735054445 >> v2:172.21.15.17:3300/0 conn(0x7f46ac0b2710 msgr2=0x7f46ac14afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:12.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.366+0000 7f46ab7fe700 1 --2- 172.21.15.17:0/1735054445 >> v2:172.21.15.17:3300/0 conn(0x7f46ac0b2710 0x7f46ac14afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:12.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.366+0000 7f46ab7fe700 1 -- 172.21.15.17:0/1735054445 >> v1:172.21.15.17:6789/0 conn(0x7f46ac0b01a0 legacy=0x7f46ac0a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:12.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.366+0000 7f46ab7fe700 1 -- 172.21.15.17:0/1735054445 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ac0a79c0 con 0x7f46ac0a8c50 2022-08-01T19:42:12.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.366+0000 7f46aaffd700 1 --2- 172.21.15.17:0/1735054445 >> v2:172.21.15.17:3300/0 conn(0x7f46ac0b2710 0x7f46ac14afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:42:12.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.366+0000 7f46ab7fe700 1 --2- 172.21.15.17:0/1735054445 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46ac0a8c50 0x7f46ac14b4c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f46a4002f70 tx=0x7f46a4006df0).ready entity=mon.2 client_cookie=97d3d89bcb16d860 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:12.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.366+0000 7f46a9ffb700 1 -- 172.21.15.17:0/1735054445 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46a400d4a0 con 0x7f46ac0a8c50 2022-08-01T19:42:12.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.366+0000 7f46a9ffb700 1 -- 172.21.15.17:0/1735054445 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a4000e90 con 0x7f46ac0a8c50 2022-08-01T19:42:12.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.366+0000 7f46a9ffb700 1 -- 172.21.15.17:0/1735054445 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46a400d7f0 con 0x7f46ac0a8c50 2022-08-01T19:42:12.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.370+0000 7f46abfff700 1 -- 172.21.15.17:0/1735054445 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46ac0a8c50 msgr2=0x7f46ac14b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:12.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.370+0000 7f46abfff700 1 --2- 172.21.15.17:0/1735054445 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46ac0a8c50 0x7f46ac14b4c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f46a4002f70 tx=0x7f46a4006df0).stop 2022-08-01T19:42:12.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.370+0000 7f46abfff700 1 -- 172.21.15.17:0/1735054445 shutdown_connections 2022-08-01T19:42:12.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.370+0000 7f46abfff700 1 --2- 172.21.15.17:0/1735054445 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46ac0a8c50 0x7f46ac14b4c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:12.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.370+0000 7f46abfff700 1 --2- 172.21.15.17:0/1735054445 >> v2:172.21.15.17:3300/0 conn(0x7f46ac0b2710 0x7f46ac14afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:12.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.370+0000 7f46abfff700 1 -- 172.21.15.17:0/1735054445 >> 172.21.15.17:0/1735054445 conn(0x7f46ac060310 msgr2=0x7f46ac0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:12.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.370+0000 7f46abfff700 1 -- 172.21.15.17:0/1735054445 shutdown_connections 2022-08-01T19:42:12.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.370+0000 7f46abfff700 1 -- 172.21.15.17:0/1735054445 wait complete. 2022-08-01T19:42:12.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.370+0000 7f46abfff700 1 Processor -- start 2022-08-01T19:42:12.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.370+0000 7f46abfff700 1 -- start start 2022-08-01T19:42:12.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.374+0000 7f46abfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46ac0a8c50 0x7f46ac0b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:12.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.374+0000 7f46abfff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46ac0b01a0 0x7f46ac0b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:12.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.374+0000 7f46abfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46ac0b2710 0x7f46ac0b54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:12.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.374+0000 7f46abfff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac0b4b60 con 0x7f46ac0a8c50 2022-08-01T19:42:12.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.374+0000 7f46abfff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac0b4080 con 0x7f46ac0b2710 2022-08-01T19:42:12.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.374+0000 7f46abfff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac0b3a80 con 0x7f46ac0b01a0 2022-08-01T19:42:12.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.374+0000 7f46aaffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46ac0a8c50 0x7f46ac0b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:12.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.374+0000 7f46aaffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46ac0a8c50 0x7f46ac0b32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43384/0 (socket says 172.21.15.17:43384) 2022-08-01T19:42:12.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.374+0000 7f46aaffd700 1 -- 172.21.15.17:0/3593456535 learned_addr learned my addr 172.21.15.17:0/3593456535 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:12.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.374+0000 7f46aa7fc700 1 --2- 172.21.15.17:0/3593456535 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46ac0b01a0 0x7f46ac0b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:12.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.374+0000 7f46ab7fe700 1 --2- 172.21.15.17:0/3593456535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46ac0b2710 0x7f46ac0b54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:12.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.374+0000 7f46aaffd700 1 -- 172.21.15.17:0/3593456535 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46ac0b01a0 msgr2=0x7f46ac0b4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:12.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.374+0000 7f46aaffd700 1 --2- 172.21.15.17:0/3593456535 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46ac0b01a0 0x7f46ac0b4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:12.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.374+0000 7f46aaffd700 1 -- 172.21.15.17:0/3593456535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46ac0b2710 msgr2=0x7f46ac0b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:12.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.374+0000 7f46aaffd700 1 --2- 172.21.15.17:0/3593456535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46ac0b2710 0x7f46ac0b54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:12.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.374+0000 7f46aaffd700 1 -- 172.21.15.17:0/3593456535 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a000b180 con 0x7f46ac0a8c50 2022-08-01T19:42:12.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.374+0000 7f46ab7fe700 1 --2- 172.21.15.17:0/3593456535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46ac0b2710 0x7f46ac0b54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:42:12.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.374+0000 7f46aaffd700 1 --2- 172.21.15.17:0/3593456535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46ac0a8c50 0x7f46ac0b32a0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 rx=0x7f46a000c000 tx=0x7f46a0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:12.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.374+0000 7f469bfff700 1 -- 172.21.15.17:0/3593456535 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46a0010590 con 0x7f46ac0a8c50 2022-08-01T19:42:12.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.374+0000 7f469bfff700 1 -- 172.21.15.17:0/3593456535 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a00106f0 con 0x7f46ac0a8c50 2022-08-01T19:42:12.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.374+0000 7f469bfff700 1 -- 172.21.15.17:0/3593456535 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46a0010a50 con 0x7f46ac0a8c50 2022-08-01T19:42:12.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.374+0000 7f46abfff700 1 -- 172.21.15.17:0/3593456535 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a400b180 con 0x7f46ac0a8c50 2022-08-01T19:42:12.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.374+0000 7f46abfff700 1 -- 172.21.15.17:0/3593456535 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46ac0abc90 con 0x7f46ac0a8c50 2022-08-01T19:42:12.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.378+0000 7f469bfff700 1 -- 172.21.15.17:0/3593456535 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f46a0010bb0 con 0x7f46ac0a8c50 2022-08-01T19:42:12.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.378+0000 7f46abfff700 1 -- 172.21.15.17:0/3593456535 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46ac0574e0 con 0x7f46ac0a8c50 2022-08-01T19:42:12.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.378+0000 7f469bfff700 1 --2- 172.21.15.17:0/3593456535 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4688053a70 0x7f4688055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:12.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.378+0000 7f469bfff700 1 -- 172.21.15.17:0/3593456535 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f46a006c760 con 0x7f46ac0a8c50 2022-08-01T19:42:12.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.378+0000 7f46aa7fc700 1 --2- 172.21.15.17:0/3593456535 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4688053a70 0x7f4688055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:12.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.378+0000 7f46aa7fc700 1 --2- 172.21.15.17:0/3593456535 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4688053a70 0x7f4688055f30 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f469c00d000 tx=0x7f469c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:12.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.382+0000 7f469bfff700 1 -- 172.21.15.17:0/3593456535 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f46a0043390 con 0x7f46ac0a8c50 2022-08-01T19:42:12.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.510+0000 7f46abfff700 1 -- 172.21.15.17:0/3593456535 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46ac069a70 con 0x7f46ac0a8c50 2022-08-01T19:42:12.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.510+0000 7f469bfff700 1 -- 172.21.15.17:0/3593456535 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f46a00469b0 con 0x7f46ac0a8c50 2022-08-01T19:42:12.513 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:42:12.514 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":117,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:42:11.920791+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:42:09.404661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2022-08-01T19:42:10.264672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:42:12.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.514+0000 7f46abfff700 1 -- 172.21.15.17:0/3593456535 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4688053a70 msgr2=0x7f4688055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:12.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.514+0000 7f46abfff700 1 --2- 172.21.15.17:0/3593456535 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4688053a70 0x7f4688055f30 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f469c00d000 tx=0x7f469c009040).stop 2022-08-01T19:42:12.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.514+0000 7f46abfff700 1 -- 172.21.15.17:0/3593456535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46ac0a8c50 msgr2=0x7f46ac0b32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:12.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.514+0000 7f46abfff700 1 --2- 172.21.15.17:0/3593456535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46ac0a8c50 0x7f46ac0b32a0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 rx=0x7f46a000c000 tx=0x7f46a0006df0).stop 2022-08-01T19:42:12.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.514+0000 7f46abfff700 1 -- 172.21.15.17:0/3593456535 shutdown_connections 2022-08-01T19:42:12.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.514+0000 7f46abfff700 1 --2- 172.21.15.17:0/3593456535 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4688053a70 0x7f4688055f30 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:12.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.514+0000 7f46abfff700 1 --2- 172.21.15.17:0/3593456535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46ac0b2710 0x7f46ac0b54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:12.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.514+0000 7f46abfff700 1 --2- 172.21.15.17:0/3593456535 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46ac0b01a0 0x7f46ac0b4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:12.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.514+0000 7f46abfff700 1 --2- 172.21.15.17:0/3593456535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46ac0a8c50 0x7f46ac0b32a0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:12.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.514+0000 7f46abfff700 1 -- 172.21.15.17:0/3593456535 >> 172.21.15.17:0/3593456535 conn(0x7f46ac060310 msgr2=0x7f46ac146030 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:12.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.514+0000 7f46abfff700 1 -- 172.21.15.17:0/3593456535 shutdown_connections 2022-08-01T19:42:12.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.514+0000 7f46abfff700 1 -- 172.21.15.17:0/3593456535 wait complete. 2022-08-01T19:42:12.530 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T19:42:12.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.646+0000 7f9217dc1700 1 Processor -- start 2022-08-01T19:42:12.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.646+0000 7f9217dc1700 1 -- start start 2022-08-01T19:42:12.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.646+0000 7f9217dc1700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9210144190 0x7f921014c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:12.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.646+0000 7f9217dc1700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92100aad40 con 0x7f9210067490 2022-08-01T19:42:12.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.646+0000 7f9217dc1700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9210060670 con 0x7f9210060290 2022-08-01T19:42:12.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.646+0000 7f9217dc1700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9210067030 con 0x7f9210144190 2022-08-01T19:42:12.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.646+0000 7f921535c700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9210067490 0x7f9210143a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35588/0 (socket says 172.21.15.17:35588) 2022-08-01T19:42:12.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.646+0000 7f921535c700 1 -- 172.21.15.17:0/711415804 learned_addr learned my addr 172.21.15.17:0/711415804 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:12.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.646+0000 7f921635e700 1 --2- 172.21.15.17:0/711415804 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9210144190 0x7f921014c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:12.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.646+0000 7f9214b5b700 1 -- 172.21.15.17:0/711415804 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2989339730 0 0) 0x7f92100aad40 con 0x7f9210067490 2022-08-01T19:42:12.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.646+0000 7f9214b5b700 1 -- 172.21.15.17:0/711415804 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91f8003260 con 0x7f9210067490 2022-08-01T19:42:12.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.646+0000 7f921635e700 1 -- 172.21.15.17:0/711415804 >> v1:172.21.15.77:6789/0 conn(0x7f9210060290 legacy=0x7f92100b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:12.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.646+0000 7f921635e700 1 -- 172.21.15.17:0/711415804 >> v1:172.21.15.17:6789/0 conn(0x7f9210067490 legacy=0x7f9210143a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:12.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.646+0000 7f9214b5b700 1 -- 172.21.15.17:0/711415804 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1536430531 0 0) 0x7f9210060670 con 0x7f9210060290 2022-08-01T19:42:12.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.646+0000 7f921635e700 1 -- 172.21.15.17:0/711415804 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92100b3540 con 0x7f9210144190 2022-08-01T19:42:12.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.646+0000 7f921635e700 1 --2- 172.21.15.17:0/711415804 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9210144190 0x7f921014c9f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f9208002f70 tx=0x7f9208006df0).ready entity=mon.2 client_cookie=62c6b46fbbc6e44b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:12.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.646+0000 7f9214b5b700 1 -- 172.21.15.17:0/711415804 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f920800d570 con 0x7f9210144190 2022-08-01T19:42:12.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.646+0000 7f9214b5b700 1 -- 172.21.15.17:0/711415804 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9208000e90 con 0x7f9210144190 2022-08-01T19:42:12.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.646+0000 7f9214b5b700 1 -- 172.21.15.17:0/711415804 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f920800d8c0 con 0x7f9210144190 2022-08-01T19:42:12.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.650+0000 7f9217dc1700 1 -- 172.21.15.17:0/711415804 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9210144190 msgr2=0x7f921014c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:12.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.650+0000 7f9217dc1700 1 --2- 172.21.15.17:0/711415804 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9210144190 0x7f921014c9f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f9208002f70 tx=0x7f9208006df0).stop 2022-08-01T19:42:12.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.650+0000 7f9217dc1700 1 -- 172.21.15.17:0/711415804 shutdown_connections 2022-08-01T19:42:12.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.650+0000 7f9217dc1700 1 --2- 172.21.15.17:0/711415804 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9210144190 0x7f921014c9f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:12.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.650+0000 7f9217dc1700 1 -- 172.21.15.17:0/711415804 >> 172.21.15.17:0/711415804 conn(0x7f921005d0c0 msgr2=0x7f921005f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:12.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.650+0000 7f9217dc1700 1 -- 172.21.15.17:0/711415804 shutdown_connections 2022-08-01T19:42:12.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.650+0000 7f9217dc1700 1 -- 172.21.15.17:0/711415804 wait complete. 2022-08-01T19:42:12.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.650+0000 7f9217dc1700 1 Processor -- start 2022-08-01T19:42:12.684 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.650+0000 7f9217dc1700 1 -- start start 2022-08-01T19:42:12.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.650+0000 7f9217dc1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9210060290 0x7f92100b6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:12.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.650+0000 7f9217dc1700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9210067490 0x7f9210153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:12.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.650+0000 7f9217dc1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9210144190 0x7f92100ad6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:12.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.650+0000 7f9217dc1700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921009f2f0 con 0x7f9210060290 2022-08-01T19:42:12.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.650+0000 7f9217dc1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9210153890 con 0x7f9210144190 2022-08-01T19:42:12.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.650+0000 7f9217dc1700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f921009f520 con 0x7f9210067490 2022-08-01T19:42:12.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.650+0000 7f9215b5d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9210060290 0x7f92100b6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:12.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.650+0000 7f9215b5d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9210060290 0x7f92100b6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43398/0 (socket says 172.21.15.17:43398) 2022-08-01T19:42:12.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.650+0000 7f9215b5d700 1 -- 172.21.15.17:0/4289929145 learned_addr learned my addr 172.21.15.17:0/4289929145 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:12.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.650+0000 7f921635e700 1 --2- 172.21.15.17:0/4289929145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9210144190 0x7f92100ad6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:12.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.650+0000 7f921535c700 1 --2- 172.21.15.17:0/4289929145 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9210067490 0x7f9210153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:12.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.650+0000 7f9215b5d700 1 -- 172.21.15.17:0/4289929145 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9210067490 msgr2=0x7f9210153f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:12.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.650+0000 7f9215b5d700 1 --2- 172.21.15.17:0/4289929145 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9210067490 0x7f9210153f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:12.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.650+0000 7f9215b5d700 1 -- 172.21.15.17:0/4289929145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9210144190 msgr2=0x7f92100ad6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:12.692 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.650+0000 7f9215b5d700 1 --2- 172.21.15.17:0/4289929145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9210144190 0x7f92100ad6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:12.692 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.650+0000 7f9215b5d700 1 -- 172.21.15.17:0/4289929145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f920800b180 con 0x7f9210060290 2022-08-01T19:42:12.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.650+0000 7f921535c700 1 --2- 172.21.15.17:0/4289929145 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9210067490 0x7f9210153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:42:12.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.650+0000 7f9215b5d700 1 --2- 172.21.15.17:0/4289929145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9210060290 0x7f92100b6730 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 rx=0x7f9204000ed0 tx=0x7f9204007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:12.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.650+0000 7f9202ffd700 1 -- 172.21.15.17:0/4289929145 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9204012400 con 0x7f9210060290 2022-08-01T19:42:12.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.650+0000 7f9202ffd700 1 -- 172.21.15.17:0/4289929145 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9204012560 con 0x7f9210060290 2022-08-01T19:42:12.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.650+0000 7f9217dc1700 1 -- 172.21.15.17:0/4289929145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f921009f7c0 con 0x7f9210060290 2022-08-01T19:42:12.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.650+0000 7f9202ffd700 1 -- 172.21.15.17:0/4289929145 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92040128c0 con 0x7f9210060290 2022-08-01T19:42:12.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.650+0000 7f9217dc1700 1 -- 172.21.15.17:0/4289929145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92100b6c20 con 0x7f9210060290 2022-08-01T19:42:12.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.654+0000 7f9217dc1700 1 -- 172.21.15.17:0/4289929145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92100574e0 con 0x7f9210060290 2022-08-01T19:42:12.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.654+0000 7f9202ffd700 1 -- 172.21.15.17:0/4289929145 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9204019030 con 0x7f9210060290 2022-08-01T19:42:12.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.654+0000 7f9202ffd700 1 --2- 172.21.15.17:0/4289929145 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f91f0053aa0 0x7f91f0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:12.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.654+0000 7f9202ffd700 1 -- 172.21.15.17:0/4289929145 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f920406b4a0 con 0x7f9210060290 2022-08-01T19:42:12.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.654+0000 7f921535c700 1 --2- 172.21.15.17:0/4289929145 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f91f0053aa0 0x7f91f0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:12.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.654+0000 7f921535c700 1 --2- 172.21.15.17:0/4289929145 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f91f0053aa0 0x7f91f0055f60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7f920c00b000 tx=0x7f920c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:12.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.662+0000 7f9202ffd700 1 -- 172.21.15.17:0/4289929145 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9204042120 con 0x7f9210060290 2022-08-01T19:42:12.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.826+0000 7f9217dc1700 1 -- 172.21.15.17:0/4289929145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f9210153c10 con 0x7f9210060290 2022-08-01T19:42:12.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.834+0000 7f9202ffd700 1 -- 172.21.15.17:0/4289929145 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v32) v1 ==== 184+0+0 (secure 0 0 0) 0x7f9204045740 con 0x7f9210060290 2022-08-01T19:42:12.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.838+0000 7f9217dc1700 1 -- 172.21.15.17:0/4289929145 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f91f0053aa0 msgr2=0x7f91f0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:12.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.838+0000 7f9217dc1700 1 --2- 172.21.15.17:0/4289929145 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f91f0053aa0 0x7f91f0055f60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7f920c00b000 tx=0x7f920c009040).stop 2022-08-01T19:42:12.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.838+0000 7f9217dc1700 1 -- 172.21.15.17:0/4289929145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9210060290 msgr2=0x7f92100b6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:12.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.838+0000 7f9217dc1700 1 --2- 172.21.15.17:0/4289929145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9210060290 0x7f92100b6730 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 rx=0x7f9204000ed0 tx=0x7f9204007ac0).stop 2022-08-01T19:42:12.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.838+0000 7f9217dc1700 1 -- 172.21.15.17:0/4289929145 shutdown_connections 2022-08-01T19:42:12.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.838+0000 7f9217dc1700 1 --2- 172.21.15.17:0/4289929145 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f91f0053aa0 0x7f91f0055f60 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:12.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.838+0000 7f9217dc1700 1 --2- 172.21.15.17:0/4289929145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9210144190 0x7f92100ad6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:12.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.838+0000 7f9217dc1700 1 --2- 172.21.15.17:0/4289929145 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9210067490 0x7f9210153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:12.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.838+0000 7f9217dc1700 1 --2- 172.21.15.17:0/4289929145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9210060290 0x7f92100b6730 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:12.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.838+0000 7f9217dc1700 1 -- 172.21.15.17:0/4289929145 >> 172.21.15.17:0/4289929145 conn(0x7f921005d0c0 msgr2=0x7f9210141e70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:12.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.838+0000 7f9217dc1700 1 -- 172.21.15.17:0/4289929145 shutdown_connections 2022-08-01T19:42:12.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.838+0000 7f9217dc1700 1 -- 172.21.15.17:0/4289929145 wait complete. 2022-08-01T19:42:12.849 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.0 2022-08-01T19:42:12.852 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T19:42:12.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.966+0000 7f69833e3700 1 Processor -- start 2022-08-01T19:42:12.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.966+0000 7f69833e3700 1 -- start start 2022-08-01T19:42:12.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.966+0000 7f69833e3700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f697c0b0e60 0x7f697c13dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:12.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.966+0000 7f69833e3700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f697c05fe20 0x7f697c13e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:12.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.966+0000 7f69833e3700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f697c0b1700 con 0x7f697c066a50 2022-08-01T19:42:12.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.966+0000 7f69833e3700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f697c066e30 con 0x7f697c05fe20 2022-08-01T19:42:12.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.966+0000 7f69833e3700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f697c0b1500 con 0x7f697c0b0e60 2022-08-01T19:42:12.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.966+0000 7f6982be2700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f697c05fe20 0x7f697c13e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:12.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.966+0000 7f6982be2700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f697c05fe20 0x7f697c13e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43410/0 (socket says 172.21.15.17:43410) 2022-08-01T19:42:12.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.966+0000 7f6982be2700 1 -- 172.21.15.17:0/3954472833 learned_addr learned my addr 172.21.15.17:0/3954472833 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:12.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.966+0000 7f6981be0700 1 --2- 172.21.15.17:0/3954472833 >> v2:172.21.15.77:3300/0 conn(0x7f697c0b0e60 0x7f697c13dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:12.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.966+0000 7f6982be2700 1 -- 172.21.15.17:0/3954472833 >> v2:172.21.15.77:3300/0 conn(0x7f697c0b0e60 msgr2=0x7f697c13dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:12.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.966+0000 7f6982be2700 1 --2- 172.21.15.17:0/3954472833 >> v2:172.21.15.77:3300/0 conn(0x7f697c0b0e60 0x7f697c13dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:12.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.966+0000 7f6982be2700 1 -- 172.21.15.17:0/3954472833 >> v1:172.21.15.77:6789/0 conn(0x7f697c066a50 legacy=0x7f697c0b2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:12.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.966+0000 7f6982be2700 1 -- 172.21.15.17:0/3954472833 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697c05f9c0 con 0x7f697c05fe20 2022-08-01T19:42:12.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.966+0000 7f6981be0700 1 --2- 172.21.15.17:0/3954472833 >> v2:172.21.15.77:3300/0 conn(0x7f697c0b0e60 0x7f697c13dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:42:12.991 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.966+0000 7f6982be2700 1 --2- 172.21.15.17:0/3954472833 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f697c05fe20 0x7f697c13e170 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 rx=0x7f697c140e70 tx=0x7f6974009290).ready entity=mon.0 client_cookie=5c34aaa19a8dab4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:12.991 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.966+0000 7f69813df700 1 -- 172.21.15.17:0/3954472833 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f697400b820 con 0x7f697c05fe20 2022-08-01T19:42:12.992 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.966+0000 7f69813df700 1 -- 172.21.15.17:0/3954472833 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f697400b980 con 0x7f697c05fe20 2022-08-01T19:42:12.992 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.966+0000 7f69813df700 1 -- 172.21.15.17:0/3954472833 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f697400bb30 con 0x7f697c05fe20 2022-08-01T19:42:12.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.970+0000 7f69833e3700 1 -- 172.21.15.17:0/3954472833 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f697c05fe20 msgr2=0x7f697c13e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:12.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.970+0000 7f69833e3700 1 --2- 172.21.15.17:0/3954472833 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f697c05fe20 0x7f697c13e170 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 rx=0x7f697c140e70 tx=0x7f6974009290).stop 2022-08-01T19:42:12.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.970+0000 7f69833e3700 1 -- 172.21.15.17:0/3954472833 shutdown_connections 2022-08-01T19:42:12.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.970+0000 7f69833e3700 1 --2- 172.21.15.17:0/3954472833 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f697c05fe20 0x7f697c13e170 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:12.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.970+0000 7f69833e3700 1 --2- 172.21.15.17:0/3954472833 >> v2:172.21.15.77:3300/0 conn(0x7f697c0b0e60 0x7f697c13dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:12.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.970+0000 7f69833e3700 1 -- 172.21.15.17:0/3954472833 >> 172.21.15.17:0/3954472833 conn(0x7f697c063800 msgr2=0x7f697c063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:12.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.970+0000 7f69833e3700 1 -- 172.21.15.17:0/3954472833 shutdown_connections 2022-08-01T19:42:12.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.970+0000 7f69833e3700 1 -- 172.21.15.17:0/3954472833 wait complete. 2022-08-01T19:42:12.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.970+0000 7f69833e3700 1 Processor -- start 2022-08-01T19:42:12.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.970+0000 7f69833e3700 1 -- start start 2022-08-01T19:42:12.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.970+0000 7f69833e3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f697c05fe20 0x7f697c1371e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:12.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.970+0000 7f69833e3700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f697c066a50 0x7f697c0ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:12.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.970+0000 7f69833e3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f697c0b0e60 0x7f697c13c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:13.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.970+0000 7f69833e3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c064a40 con 0x7f697c0b0e60 2022-08-01T19:42:13.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.970+0000 7f69833e3700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c13bdc0 con 0x7f697c05fe20 2022-08-01T19:42:13.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.970+0000 7f69833e3700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f697c0afb60 con 0x7f697c066a50 2022-08-01T19:42:13.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.970+0000 7f6982be2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f697c0b0e60 0x7f697c13c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:13.003 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.970+0000 7f6982be2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f697c0b0e60 0x7f697c13c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43416/0 (socket says 172.21.15.17:43416) 2022-08-01T19:42:13.003 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.970+0000 7f6982be2700 1 -- 172.21.15.17:0/1687217 learned_addr learned my addr 172.21.15.17:0/1687217 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:13.003 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.970+0000 7f69823e1700 1 --2- 172.21.15.17:0/1687217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f697c05fe20 0x7f697c1371e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:13.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.970+0000 7f6981be0700 1 --2- 172.21.15.17:0/1687217 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f697c066a50 0x7f697c0ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:13.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.974+0000 7f6982be2700 1 -- 172.21.15.17:0/1687217 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f697c066a50 msgr2=0x7f697c0ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:13.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.974+0000 7f6982be2700 1 --2- 172.21.15.17:0/1687217 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f697c066a50 0x7f697c0ae7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:13.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.974+0000 7f6982be2700 1 -- 172.21.15.17:0/1687217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f697c05fe20 msgr2=0x7f697c1371e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:13.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.974+0000 7f6982be2700 1 --2- 172.21.15.17:0/1687217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f697c05fe20 0x7f697c1371e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:13.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.974+0000 7f6982be2700 1 -- 172.21.15.17:0/1687217 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6974009530 con 0x7f697c0b0e60 2022-08-01T19:42:13.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.974+0000 7f6982be2700 1 --2- 172.21.15.17:0/1687217 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f697c0b0e60 0x7f697c13c060 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 rx=0x7f697c140e70 tx=0x7f6974016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:13.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.974+0000 7f696b7fe700 1 -- 172.21.15.17:0/1687217 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69740032f0 con 0x7f697c0b0e60 2022-08-01T19:42:13.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.974+0000 7f696b7fe700 1 -- 172.21.15.17:0/1687217 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6974021440 con 0x7f697c0b0e60 2022-08-01T19:42:13.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.974+0000 7f69833e3700 1 -- 172.21.15.17:0/1687217 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f697c066320 con 0x7f697c0b0e60 2022-08-01T19:42:13.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.974+0000 7f696b7fe700 1 -- 172.21.15.17:0/1687217 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69740217a0 con 0x7f697c0b0e60 2022-08-01T19:42:13.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.974+0000 7f69833e3700 1 -- 172.21.15.17:0/1687217 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f697c066080 con 0x7f697c0b0e60 2022-08-01T19:42:13.007 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.974+0000 7f69833e3700 1 -- 172.21.15.17:0/1687217 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f697c0aff60 con 0x7f697c0b0e60 2022-08-01T19:42:13.007 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.974+0000 7f696b7fe700 1 -- 172.21.15.17:0/1687217 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6974027020 con 0x7f697c0b0e60 2022-08-01T19:42:13.007 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.974+0000 7f696b7fe700 1 --2- 172.21.15.17:0/1687217 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f695c053a70 0x7f695c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:13.007 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.974+0000 7f696b7fe700 1 -- 172.21.15.17:0/1687217 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f697406d600 con 0x7f697c0b0e60 2022-08-01T19:42:13.008 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.978+0000 7f69823e1700 1 --2- 172.21.15.17:0/1687217 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f695c053a70 0x7f695c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:13.008 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.978+0000 7f69823e1700 1 --2- 172.21.15.17:0/1687217 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f695c053a70 0x7f695c055f30 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f696c00d000 tx=0x7f696c007b70).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:13.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:12.982+0000 7f696b7fe700 1 -- 172.21.15.17:0/1687217 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f69740443c0 con 0x7f697c0b0e60 2022-08-01T19:42:13.154 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.146+0000 7f69833e3700 1 -- 172.21.15.17:0/1687217 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f697c0b1930 con 0x7f697c0b0e60 2022-08-01T19:42:13.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.154+0000 7f696b7fe700 1 -- 172.21.15.17:0/1687217 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v33) v1 ==== 184+0+0 (secure 0 0 0) 0x7f6974015020 con 0x7f697c0b0e60 2022-08-01T19:42:13.163 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.158+0000 7f69833e3700 1 -- 172.21.15.17:0/1687217 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f695c053a70 msgr2=0x7f695c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:13.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.158+0000 7f69833e3700 1 --2- 172.21.15.17:0/1687217 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f695c053a70 0x7f695c055f30 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f696c00d000 tx=0x7f696c007b70).stop 2022-08-01T19:42:13.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.158+0000 7f69833e3700 1 -- 172.21.15.17:0/1687217 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f697c0b0e60 msgr2=0x7f697c13c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:13.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.158+0000 7f69833e3700 1 --2- 172.21.15.17:0/1687217 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f697c0b0e60 0x7f697c13c060 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 rx=0x7f697c140e70 tx=0x7f6974016000).stop 2022-08-01T19:42:13.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.158+0000 7f69833e3700 1 -- 172.21.15.17:0/1687217 shutdown_connections 2022-08-01T19:42:13.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.158+0000 7f69833e3700 1 --2- 172.21.15.17:0/1687217 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f695c053a70 0x7f695c055f30 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:13.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.158+0000 7f69833e3700 1 --2- 172.21.15.17:0/1687217 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f697c0b0e60 0x7f697c13c060 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:13.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.158+0000 7f69833e3700 1 --2- 172.21.15.17:0/1687217 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f697c066a50 0x7f697c0ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:13.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.158+0000 7f69833e3700 1 --2- 172.21.15.17:0/1687217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f697c05fe20 0x7f697c1371e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:13.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.158+0000 7f69833e3700 1 -- 172.21.15.17:0/1687217 >> 172.21.15.17:0/1687217 conn(0x7f697c063800 msgr2=0x7f697c1417d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:13.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.158+0000 7f69833e3700 1 -- 172.21.15.17:0/1687217 shutdown_connections 2022-08-01T19:42:13.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.158+0000 7f69833e3700 1 -- 172.21.15.17:0/1687217 wait complete. 2022-08-01T19:42:13.167 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.1 2022-08-01T19:42:13.172 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:42:13.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.274+0000 7f4656734700 1 Processor -- start 2022-08-01T19:42:13.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.274+0000 7f4656734700 1 -- start start 2022-08-01T19:42:13.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.274+0000 7f4656734700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46500b1160 0x7f46501448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:13.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.274+0000 7f4656734700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46500b04e0 con 0x7f46500b1f40 2022-08-01T19:42:13.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.274+0000 7f4656734700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46500b0780 con 0x7f4650148530 2022-08-01T19:42:13.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.274+0000 7f4656734700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46500a9250 con 0x7f46500b1160 2022-08-01T19:42:13.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.274+0000 7f464ffff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f46500b1f40 0x7f46500b2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35614/0 (socket says 172.21.15.17:35614) 2022-08-01T19:42:13.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.274+0000 7f464ffff700 1 -- 172.21.15.17:0/4112411728 learned_addr learned my addr 172.21.15.17:0/4112411728 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:13.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.274+0000 7f4654cd1700 1 --2- 172.21.15.17:0/4112411728 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46500b1160 0x7f46501448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:13.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.274+0000 7f464effd700 1 -- 172.21.15.17:0/4112411728 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3194564746 0 0) 0x7f46500b04e0 con 0x7f46500b1f40 2022-08-01T19:42:13.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.274+0000 7f464effd700 1 -- 172.21.15.17:0/4112411728 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4634003260 con 0x7f46500b1f40 2022-08-01T19:42:13.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.274+0000 7f464effd700 1 -- 172.21.15.17:0/4112411728 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3636432237 0 0) 0x7f46500b0780 con 0x7f4650148530 2022-08-01T19:42:13.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.274+0000 7f464effd700 1 -- 172.21.15.17:0/4112411728 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46500b04e0 con 0x7f4650148530 2022-08-01T19:42:13.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.274+0000 7f464effd700 1 -- 172.21.15.17:0/4112411728 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3307134879 0 0) 0x7f4634003260 con 0x7f46500b1f40 2022-08-01T19:42:13.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.274+0000 7f464effd700 1 -- 172.21.15.17:0/4112411728 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46500b1160 msgr2=0x7f46501448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:13.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.274+0000 7f464effd700 1 --2- 172.21.15.17:0/4112411728 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46500b1160 0x7f46501448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:13.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.274+0000 7f464effd700 1 -- 172.21.15.17:0/4112411728 >> v1:172.21.15.77:6789/0 conn(0x7f4650148530 legacy=0x7f46500a7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:13.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.274+0000 7f464effd700 1 -- 172.21.15.17:0/4112411728 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46500b0a20 con 0x7f46500b1f40 2022-08-01T19:42:13.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.274+0000 7f464effd700 1 -- 172.21.15.17:0/4112411728 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7f4638001660 con 0x7f46500b1f40 2022-08-01T19:42:13.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.274+0000 7f464effd700 1 -- 172.21.15.17:0/4112411728 >> v1:172.21.15.17:6789/0 conn(0x7f46500b1f40 legacy=0x7f46500b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:13.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.274+0000 7f464effd700 1 --2- 172.21.15.17:0/4112411728 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46500b1160 0x7f4634003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:13.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.274+0000 7f464effd700 1 --2- 172.21.15.17:0/4112411728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4634003e20 0x7f4634004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:13.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.274+0000 7f464effd700 1 --2- 172.21.15.17:0/4112411728 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46340049f0 0x7f4634004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:13.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.274+0000 7f464effd700 1 -- 172.21.15.17:0/4112411728 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463400b800 con 0x7f46340049f0 2022-08-01T19:42:13.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.274+0000 7f464effd700 1 -- 172.21.15.17:0/4112411728 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463400b940 con 0x7f4634003e20 2022-08-01T19:42:13.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.274+0000 7f464effd700 1 -- 172.21.15.17:0/4112411728 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f463400bb00 con 0x7f46500b1160 2022-08-01T19:42:13.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.274+0000 7f4654cd1700 1 --2- 172.21.15.17:0/4112411728 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46340049f0 0x7f4634004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:13.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.274+0000 7f464ffff700 1 --2- 172.21.15.17:0/4112411728 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46500b1160 0x7f4634003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:13.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.274+0000 7f464f7fe700 1 --2- 172.21.15.17:0/4112411728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4634003e20 0x7f4634004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:13.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.274+0000 7f4654cd1700 1 -- 172.21.15.17:0/4112411728 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46500b1160 msgr2=0x7f4634003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:13.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.274+0000 7f4654cd1700 1 --2- 172.21.15.17:0/4112411728 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46500b1160 0x7f4634003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:13.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.274+0000 7f4654cd1700 1 -- 172.21.15.17:0/4112411728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4634003e20 msgr2=0x7f4634004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:13.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.274+0000 7f4654cd1700 1 --2- 172.21.15.17:0/4112411728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4634003e20 0x7f4634004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:13.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.274+0000 7f4654cd1700 1 -- 172.21.15.17:0/4112411728 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4634003260 con 0x7f46340049f0 2022-08-01T19:42:13.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.278+0000 7f4654cd1700 1 --2- 172.21.15.17:0/4112411728 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46340049f0 0x7f4634004e70 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 rx=0x7f464000c000 tx=0x7f4640006df0).ready entity=mon.0 client_cookie=758cd8c98da78791 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:13.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.278+0000 7f464effd700 1 -- 172.21.15.17:0/4112411728 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f464000e830 con 0x7f46340049f0 2022-08-01T19:42:13.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.278+0000 7f464effd700 1 -- 172.21.15.17:0/4112411728 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f464000e990 con 0x7f46340049f0 2022-08-01T19:42:13.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.278+0000 7f464effd700 1 -- 172.21.15.17:0/4112411728 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f464000ecf0 con 0x7f46340049f0 2022-08-01T19:42:13.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.278+0000 7f4656734700 1 -- 172.21.15.17:0/4112411728 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46340049f0 msgr2=0x7f4634004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:13.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.278+0000 7f4656734700 1 --2- 172.21.15.17:0/4112411728 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46340049f0 0x7f4634004e70 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 rx=0x7f464000c000 tx=0x7f4640006df0).stop 2022-08-01T19:42:13.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.278+0000 7f4654cd1700 1 -- 172.21.15.17:0/4112411728 reap_dead start 2022-08-01T19:42:13.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.278+0000 7f4656734700 1 -- 172.21.15.17:0/4112411728 shutdown_connections 2022-08-01T19:42:13.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.278+0000 7f4656734700 1 -- 172.21.15.17:0/4112411728 >> 172.21.15.17:0/4112411728 conn(0x7f465005f840 msgr2=0x7f465005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:13.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.278+0000 7f4656734700 1 -- 172.21.15.17:0/4112411728 shutdown_connections 2022-08-01T19:42:13.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.278+0000 7f4656734700 1 -- 172.21.15.17:0/4112411728 wait complete. 2022-08-01T19:42:13.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.278+0000 7f4656734700 1 Processor -- start 2022-08-01T19:42:13.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.278+0000 7f4656734700 1 -- start start 2022-08-01T19:42:13.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.282+0000 7f4656734700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46340049f0 0x7f46500a7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:13.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.282+0000 7f4656734700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4650141590 0x7f4650149130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:13.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.282+0000 7f4656734700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f465014db30 0x7f4650149620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:13.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.282+0000 7f4656734700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4650141e60 con 0x7f465014db30 2022-08-01T19:42:13.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.282+0000 7f4656734700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4650140c60 con 0x7f4650141590 2022-08-01T19:42:13.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.282+0000 7f4656734700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4650141bc0 con 0x7f46340049f0 2022-08-01T19:42:13.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.282+0000 7f4654cd1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f465014db30 0x7f4650149620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:13.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.282+0000 7f4654cd1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f465014db30 0x7f4650149620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43436/0 (socket says 172.21.15.17:43436) 2022-08-01T19:42:13.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.282+0000 7f4654cd1700 1 -- 172.21.15.17:0/3006463179 learned_addr learned my addr 172.21.15.17:0/3006463179 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:13.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.282+0000 7f464ffff700 1 --2- 172.21.15.17:0/3006463179 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46340049f0 0x7f46500a7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:13.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.282+0000 7f464f7fe700 1 --2- 172.21.15.17:0/3006463179 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4650141590 0x7f4650149130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:13.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.282+0000 7f4654cd1700 1 -- 172.21.15.17:0/3006463179 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46340049f0 msgr2=0x7f46500a7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:13.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.282+0000 7f4654cd1700 1 --2- 172.21.15.17:0/3006463179 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46340049f0 0x7f46500a7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:13.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.282+0000 7f4654cd1700 1 -- 172.21.15.17:0/3006463179 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4650141590 msgr2=0x7f4650149130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:13.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.282+0000 7f4654cd1700 1 --2- 172.21.15.17:0/3006463179 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4650141590 0x7f4650149130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:13.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.282+0000 7f4654cd1700 1 -- 172.21.15.17:0/3006463179 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4650140e50 con 0x7f465014db30 2022-08-01T19:42:13.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.282+0000 7f4654cd1700 1 --2- 172.21.15.17:0/3006463179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f465014db30 0x7f4650149620 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 rx=0x7f4638001840 tx=0x7f4640019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:13.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.282+0000 7f464d7fa700 1 -- 172.21.15.17:0/3006463179 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4640012c20 con 0x7f465014db30 2022-08-01T19:42:13.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.282+0000 7f464d7fa700 1 -- 172.21.15.17:0/3006463179 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4640022070 con 0x7f465014db30 2022-08-01T19:42:13.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.282+0000 7f4656734700 1 -- 172.21.15.17:0/3006463179 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46501410f0 con 0x7f465014db30 2022-08-01T19:42:13.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.282+0000 7f464d7fa700 1 -- 172.21.15.17:0/3006463179 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f464000be60 con 0x7f465014db30 2022-08-01T19:42:13.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.282+0000 7f4656734700 1 -- 172.21.15.17:0/3006463179 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f465014d4e0 con 0x7f465014db30 2022-08-01T19:42:13.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.282+0000 7f4656734700 1 -- 172.21.15.17:0/3006463179 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f465014d790 con 0x7f465014db30 2022-08-01T19:42:13.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.282+0000 7f464d7fa700 1 -- 172.21.15.17:0/3006463179 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f464002e070 con 0x7f465014db30 2022-08-01T19:42:13.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.286+0000 7f464d7fa700 1 --2- 172.21.15.17:0/3006463179 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4628053b00 0x7f4628055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:13.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.286+0000 7f464d7fa700 1 -- 172.21.15.17:0/3006463179 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4640000e40 con 0x7f465014db30 2022-08-01T19:42:13.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.286+0000 7f464ffff700 1 --2- 172.21.15.17:0/3006463179 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4628053b00 0x7f4628055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:13.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.286+0000 7f464ffff700 1 --2- 172.21.15.17:0/3006463179 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4628053b00 0x7f4628055fc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f4638005000 tx=0x7f46500b0670).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:13.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.290+0000 7f464d7fa700 1 -- 172.21.15.17:0/3006463179 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4640044930 con 0x7f465014db30 2022-08-01T19:42:13.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.450+0000 7f4656734700 1 -- 172.21.15.17:0/3006463179 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46500574e0 con 0x7f465014db30 2022-08-01T19:42:13.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.450+0000 7f464d7fa700 1 -- 172.21.15.17:0/3006463179 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f464002c070 con 0x7f465014db30 2022-08-01T19:42:13.456 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:42:13.456 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":58,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7581,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/1744640648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":1744640648},{"type":"v1","addr":"172.21.15.17:6833","nonce":1744640648}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":56},{"gid":7594,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.17:6835/346594655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":346594655},{"type":"v1","addr":"172.21.15.17:6835","nonce":346594655}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":57}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:42:10.906134+0000","modified":"2022-08-01T19:42:11.922099+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7600},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7600":{"gid":7600,"name":"b","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6835/2286085186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2286085186},{"type":"v1","addr":"172.21.15.77:6835","nonce":2286085186}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2022-08-01T19:42:13.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.454+0000 7f4656734700 1 -- 172.21.15.17:0/3006463179 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4628053b00 msgr2=0x7f4628055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:13.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.454+0000 7f4656734700 1 --2- 172.21.15.17:0/3006463179 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4628053b00 0x7f4628055fc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f4638005000 tx=0x7f46500b0670).stop 2022-08-01T19:42:13.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.454+0000 7f4656734700 1 -- 172.21.15.17:0/3006463179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f465014db30 msgr2=0x7f4650149620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:13.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.454+0000 7f4656734700 1 --2- 172.21.15.17:0/3006463179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f465014db30 0x7f4650149620 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 rx=0x7f4638001840 tx=0x7f4640019000).stop 2022-08-01T19:42:13.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.454+0000 7f4656734700 1 -- 172.21.15.17:0/3006463179 shutdown_connections 2022-08-01T19:42:13.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.454+0000 7f4656734700 1 --2- 172.21.15.17:0/3006463179 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4628053b00 0x7f4628055fc0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:13.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.454+0000 7f4656734700 1 --2- 172.21.15.17:0/3006463179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f465014db30 0x7f4650149620 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:13.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.454+0000 7f4656734700 1 --2- 172.21.15.17:0/3006463179 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4650141590 0x7f4650149130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:13.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.454+0000 7f4656734700 1 --2- 172.21.15.17:0/3006463179 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46340049f0 0x7f46500a7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:13.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.454+0000 7f4656734700 1 -- 172.21.15.17:0/3006463179 >> 172.21.15.17:0/3006463179 conn(0x7f465005f840 msgr2=0x7f4650143310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:13.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.454+0000 7f4656734700 1 -- 172.21.15.17:0/3006463179 shutdown_connections 2022-08-01T19:42:13.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:13.454+0000 7f4656734700 1 -- 172.21.15.17:0/3006463179 wait complete. 2022-08-01T19:42:13.463 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 58 2022-08-01T19:42:13.775 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 58, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:42:10.906134+0000', 'modified': '2022-08-01T19:42:11.922099+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7600}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7600': {'gid': 7600, 'name': 'b', 'rank': 0, 'incarnation': 57, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.77:6835/2286085186', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6834', 'nonce': 2286085186}, {'type': 'v1', 'addr': '172.21.15.77:6835', 'nonce': 2286085186}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:42:13.776 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:42:13.776 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-08-01T19:42:13.890 INFO:teuthology.orchestra.run.smithi077.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":7600,"want_state":"up:active","state":"up:active","rank_uptime":2.969562351,"mdsmap_epoch":58,"osdmap_epoch":117,"osdmap_epoch_barrier":117,"uptime":7.3273082330000001} 2022-08-01T19:42:13.891 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 7600, "mdsmap_epoch": 58, "osdmap_epoch": 117, "osdmap_epoch_barrier": 117, "rank_uptime": 2.969562351, "state": "up:active", "uptime": 7.327308233, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:42:13.893 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:42:14.013 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.010+0000 7f3ffeda1700 1 Processor -- start 2022-08-01T19:42:14.014 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.010+0000 7f3ffeda1700 1 -- start start 2022-08-01T19:42:14.014 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.010+0000 7f3ffeda1700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3ff80b2710 0x7f3ff814afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:14.014 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.010+0000 7f3ffeda1700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3ff80b01a0 0x7f3ff814b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:14.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.010+0000 7f3ffeda1700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f3ff80afd60 0x7f3ff80a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:14.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.010+0000 7f3ffeda1700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ff80a7250 con 0x7f3ff80b2710 2022-08-01T19:42:14.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.010+0000 7f3ffeda1700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ff80a7480 con 0x7f3ff80afd60 2022-08-01T19:42:14.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.010+0000 7f3ffeda1700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff80a7720 con 0x7f3ff80b01a0 2022-08-01T19:42:14.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.010+0000 7f3ffdd9f700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3ff80b2710 0x7f3ff814afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:14.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.010+0000 7f3ffe5a0700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f3ff80afd60 0x7f3ff80a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:14.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.010+0000 7f3ffd59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3ff80b01a0 0x7f3ff814b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:14.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.010+0000 7f3ffdd9f700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3ff80b2710 0x7f3ff814afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43444/0 (socket says 172.21.15.17:43444) 2022-08-01T19:42:14.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.010+0000 7f3ffdd9f700 1 -- 172.21.15.17:0/187422456 learned_addr learned my addr 172.21.15.17:0/187422456 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:14.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.010+0000 7f3ffe5a0700 1 -- 172.21.15.17:0/187422456 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3ff80b01a0 msgr2=0x7f3ff814b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:14.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.010+0000 7f3ffe5a0700 1 --2- 172.21.15.17:0/187422456 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3ff80b01a0 0x7f3ff814b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:14.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.010+0000 7f3ffe5a0700 1 -- 172.21.15.17:0/187422456 >> v2:172.21.15.17:3300/0 conn(0x7f3ff80b2710 msgr2=0x7f3ff814afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:14.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.010+0000 7f3ffe5a0700 1 --2- 172.21.15.17:0/187422456 >> v2:172.21.15.17:3300/0 conn(0x7f3ff80b2710 0x7f3ff814afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:14.018 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.010+0000 7f3ffe5a0700 1 -- 172.21.15.17:0/187422456 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff80a79c0 con 0x7f3ff80afd60 2022-08-01T19:42:14.018 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.010+0000 7f3ffdd9f700 1 --2- 172.21.15.17:0/187422456 >> v2:172.21.15.17:3300/0 conn(0x7f3ff80b2710 0x7f3ff814afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:42:14.018 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.010+0000 7f3ffe5a0700 1 --2- 172.21.15.17:0/187422456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ff80afd60 0x7f3ff80a8540 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f3ff00032f0 tx=0x7f3ff00091d0).ready entity=mon.1 client_cookie=ed30bfc5823d4506 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:14.018 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.010+0000 7f3ffcd9d700 1 -- 172.21.15.17:0/187422456 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ff000b600 con 0x7f3ff80afd60 2022-08-01T19:42:14.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.010+0000 7f3ffcd9d700 1 -- 172.21.15.17:0/187422456 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff000b760 con 0x7f3ff80afd60 2022-08-01T19:42:14.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.010+0000 7f3ffcd9d700 1 -- 172.21.15.17:0/187422456 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ff000b8d0 con 0x7f3ff80afd60 2022-08-01T19:42:14.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.014+0000 7f3ffeda1700 1 -- 172.21.15.17:0/187422456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ff80afd60 msgr2=0x7f3ff80a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:14.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.014+0000 7f3ffeda1700 1 --2- 172.21.15.17:0/187422456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ff80afd60 0x7f3ff80a8540 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f3ff00032f0 tx=0x7f3ff00091d0).stop 2022-08-01T19:42:14.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.014+0000 7f3ffeda1700 1 -- 172.21.15.17:0/187422456 shutdown_connections 2022-08-01T19:42:14.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.014+0000 7f3ffeda1700 1 --2- 172.21.15.17:0/187422456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ff80afd60 0x7f3ff80a8540 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:14.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.014+0000 7f3ffeda1700 1 --2- 172.21.15.17:0/187422456 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3ff80b01a0 0x7f3ff814b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:14.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.014+0000 7f3ffeda1700 1 --2- 172.21.15.17:0/187422456 >> v2:172.21.15.17:3300/0 conn(0x7f3ff80b2710 0x7f3ff814afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:14.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.014+0000 7f3ffeda1700 1 -- 172.21.15.17:0/187422456 >> 172.21.15.17:0/187422456 conn(0x7f3ff8060310 msgr2=0x7f3ff80b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:14.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.014+0000 7f3ffeda1700 1 -- 172.21.15.17:0/187422456 shutdown_connections 2022-08-01T19:42:14.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.014+0000 7f3ffeda1700 1 -- 172.21.15.17:0/187422456 wait complete. 2022-08-01T19:42:14.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.014+0000 7f3ffeda1700 1 Processor -- start 2022-08-01T19:42:14.040 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.014+0000 7f3ffeda1700 1 -- start start 2022-08-01T19:42:14.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.014+0000 7f3ffeda1700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3ff80afd60 0x7f3ff80b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:14.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.014+0000 7f3ffeda1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ff80b01a0 0x7f3ff814cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:14.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.014+0000 7f3ffeda1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ff80b2710 0x7f3ff80a9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:14.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.014+0000 7f3ffeda1700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff80b4af0 con 0x7f3ff80b2710 2022-08-01T19:42:14.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.014+0000 7f3ffeda1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff8144590 con 0x7f3ff80b01a0 2022-08-01T19:42:14.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.014+0000 7f3ffeda1700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff80b4eb0 con 0x7f3ff80afd60 2022-08-01T19:42:14.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.014+0000 7f3ffe5a0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ff80b2710 0x7f3ff80a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:14.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.014+0000 7f3ffd59e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ff80b01a0 0x7f3ff814cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:14.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.014+0000 7f3ffe5a0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ff80b2710 0x7f3ff80a9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43450/0 (socket says 172.21.15.17:43450) 2022-08-01T19:42:14.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.014+0000 7f3ffd59e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ff80b01a0 0x7f3ff814cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:41368/0 (socket says 172.21.15.17:41368) 2022-08-01T19:42:14.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.014+0000 7f3ffe5a0700 1 -- 172.21.15.17:0/3070032428 learned_addr learned my addr 172.21.15.17:0/3070032428 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:14.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.014+0000 7f3ffdd9f700 1 --2- 172.21.15.17:0/3070032428 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3ff80afd60 0x7f3ff80b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:14.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.014+0000 7f3ffe5a0700 1 -- 172.21.15.17:0/3070032428 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3ff80afd60 msgr2=0x7f3ff80b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:14.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.014+0000 7f3ffe5a0700 1 --2- 172.21.15.17:0/3070032428 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3ff80afd60 0x7f3ff80b3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:14.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.014+0000 7f3ffe5a0700 1 -- 172.21.15.17:0/3070032428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ff80b01a0 msgr2=0x7f3ff814cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:14.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.014+0000 7f3ffe5a0700 1 --2- 172.21.15.17:0/3070032428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ff80b01a0 0x7f3ff814cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:14.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.018+0000 7f3ffe5a0700 1 -- 172.21.15.17:0/3070032428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe800b1e0 con 0x7f3ff80b2710 2022-08-01T19:42:14.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.018+0000 7f3ffe5a0700 1 --2- 172.21.15.17:0/3070032428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ff80b2710 0x7f3ff80a9490 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 rx=0x7f3ff00032f0 tx=0x7f3ff0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:14.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.018+0000 7f3fe6ffd700 1 -- 172.21.15.17:0/3070032428 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ff000b600 con 0x7f3ff80b2710 2022-08-01T19:42:14.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.018+0000 7f3fe6ffd700 1 -- 172.21.15.17:0/3070032428 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff001f5e0 con 0x7f3ff80b2710 2022-08-01T19:42:14.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.018+0000 7f3ffeda1700 1 -- 172.21.15.17:0/3070032428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff0009470 con 0x7f3ff80b2710 2022-08-01T19:42:14.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.018+0000 7f3fe6ffd700 1 -- 172.21.15.17:0/3070032428 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ff001f940 con 0x7f3ff80b2710 2022-08-01T19:42:14.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.018+0000 7f3ffeda1700 1 -- 172.21.15.17:0/3070032428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff80b3470 con 0x7f3ff80b2710 2022-08-01T19:42:14.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.018+0000 7f3ffeda1700 1 -- 172.21.15.17:0/3070032428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ff80b3800 con 0x7f3ff80b2710 2022-08-01T19:42:14.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.018+0000 7f3fe6ffd700 1 -- 172.21.15.17:0/3070032428 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3ff0026030 con 0x7f3ff80b2710 2022-08-01T19:42:14.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.018+0000 7f3fe6ffd700 1 --2- 172.21.15.17:0/3070032428 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3fd8053a70 0x7f3fd8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:14.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.018+0000 7f3fe6ffd700 1 -- 172.21.15.17:0/3070032428 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f3ff006cb20 con 0x7f3ff80b2710 2022-08-01T19:42:14.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.018+0000 7f3ffdd9f700 1 --2- 172.21.15.17:0/3070032428 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3fd8053a70 0x7f3fd8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:14.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.022+0000 7f3ffdd9f700 1 --2- 172.21.15.17:0/3070032428 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3fd8053a70 0x7f3fd8055f30 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f3fe800b000 tx=0x7f3fe8008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:14.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.026+0000 7f3fe6ffd700 1 -- 172.21.15.17:0/3070032428 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3ff00438e0 con 0x7f3ff80b2710 2022-08-01T19:42:14.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.154+0000 7f3ffeda1700 1 -- 172.21.15.17:0/3070032428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ff80574e0 con 0x7f3ff80b2710 2022-08-01T19:42:14.162 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.158+0000 7f3fe6ffd700 1 -- 172.21.15.17:0/3070032428 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f3ff0024090 con 0x7f3ff80b2710 2022-08-01T19:42:14.163 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:42:14.163 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":117,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:42:11.920791+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:42:09.404661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2022-08-01T19:42:10.264672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:42:14.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.158+0000 7f3ffeda1700 1 -- 172.21.15.17:0/3070032428 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3fd8053a70 msgr2=0x7f3fd8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:14.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.158+0000 7f3ffeda1700 1 --2- 172.21.15.17:0/3070032428 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3fd8053a70 0x7f3fd8055f30 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f3fe800b000 tx=0x7f3fe8008040).stop 2022-08-01T19:42:14.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.158+0000 7f3ffeda1700 1 -- 172.21.15.17:0/3070032428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ff80b2710 msgr2=0x7f3ff80a9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:14.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.158+0000 7f3ffeda1700 1 --2- 172.21.15.17:0/3070032428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ff80b2710 0x7f3ff80a9490 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 rx=0x7f3ff00032f0 tx=0x7f3ff0016000).stop 2022-08-01T19:42:14.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.158+0000 7f3ffeda1700 1 -- 172.21.15.17:0/3070032428 shutdown_connections 2022-08-01T19:42:14.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.158+0000 7f3ffeda1700 1 --2- 172.21.15.17:0/3070032428 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3fd8053a70 0x7f3fd8055f30 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:14.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.158+0000 7f3ffeda1700 1 --2- 172.21.15.17:0/3070032428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ff80b2710 0x7f3ff80a9490 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:14.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.158+0000 7f3ffeda1700 1 --2- 172.21.15.17:0/3070032428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ff80b01a0 0x7f3ff814cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:14.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.158+0000 7f3ffeda1700 1 --2- 172.21.15.17:0/3070032428 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3ff80afd60 0x7f3ff80b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:14.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.158+0000 7f3ffeda1700 1 -- 172.21.15.17:0/3070032428 >> 172.21.15.17:0/3070032428 conn(0x7f3ff8060310 msgr2=0x7f3ff8148850 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:14.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.158+0000 7f3ffeda1700 1 -- 172.21.15.17:0/3070032428 shutdown_connections 2022-08-01T19:42:14.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.158+0000 7f3ffeda1700 1 -- 172.21.15.17:0/3070032428 wait complete. 2022-08-01T19:42:14.173 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T19:42:09.404661+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T19:42:10.264672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T19:42:14.175 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:42:14.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.282+0000 7ff171f6e700 1 Processor -- start 2022-08-01T19:42:14.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.282+0000 7ff171f6e700 1 -- start start 2022-08-01T19:42:14.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.282+0000 7ff171f6e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ff16c0b2710 0x7ff16c14afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:14.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.282+0000 7ff171f6e700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff16c0a7250 con 0x7ff16c0a8c50 2022-08-01T19:42:14.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.282+0000 7ff171f6e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff16c0a7480 con 0x7ff16c0b01a0 2022-08-01T19:42:14.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.282+0000 7ff171f6e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff16c0a7720 con 0x7ff16c0b2710 2022-08-01T19:42:14.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.282+0000 7ff17176d700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff16c0a8c50 0x7ff16c151d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35652/0 (socket says 172.21.15.17:35652) 2022-08-01T19:42:14.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.282+0000 7ff17176d700 1 -- 172.21.15.17:0/2155464761 learned_addr learned my addr 172.21.15.17:0/2155464761 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:14.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.286+0000 7ff170f6c700 1 --2- 172.21.15.17:0/2155464761 >> v2:172.21.15.77:3300/0 conn(0x7ff16c0b2710 0x7ff16c14afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:14.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.286+0000 7ff16b7fe700 1 -- 172.21.15.17:0/2155464761 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3025903122 0 0) 0x7ff16c0a7480 con 0x7ff16c0b01a0 2022-08-01T19:42:14.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.286+0000 7ff16b7fe700 1 -- 172.21.15.17:0/2155464761 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff158003260 con 0x7ff16c0b01a0 2022-08-01T19:42:14.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.286+0000 7ff16b7fe700 1 -- 172.21.15.17:0/2155464761 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3037234688 0 0) 0x7ff16c0a7250 con 0x7ff16c0a8c50 2022-08-01T19:42:14.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.286+0000 7ff16b7fe700 1 -- 172.21.15.17:0/2155464761 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff16c0a7480 con 0x7ff16c0a8c50 2022-08-01T19:42:14.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.286+0000 7ff170f6c700 1 -- 172.21.15.17:0/2155464761 >> v1:172.21.15.77:6789/0 conn(0x7ff16c0b01a0 legacy=0x7ff16c0a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:14.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.286+0000 7ff170f6c700 1 -- 172.21.15.17:0/2155464761 >> v1:172.21.15.17:6789/0 conn(0x7ff16c0a8c50 legacy=0x7ff16c151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:14.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.286+0000 7ff170f6c700 1 -- 172.21.15.17:0/2155464761 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff16c0a79c0 con 0x7ff16c0b2710 2022-08-01T19:42:14.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.286+0000 7ff170f6c700 1 --2- 172.21.15.17:0/2155464761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff16c0b2710 0x7ff16c14afd0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7ff160002f70 tx=0x7ff160006df0).ready entity=mon.1 client_cookie=549121f6b65a2ae server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:14.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.286+0000 7ff16b7fe700 1 -- 172.21.15.17:0/2155464761 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff16000d540 con 0x7ff16c0b2710 2022-08-01T19:42:14.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.286+0000 7ff16b7fe700 1 -- 172.21.15.17:0/2155464761 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff160000e90 con 0x7ff16c0b2710 2022-08-01T19:42:14.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.286+0000 7ff16b7fe700 1 -- 172.21.15.17:0/2155464761 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff16000d890 con 0x7ff16c0b2710 2022-08-01T19:42:14.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.286+0000 7ff171f6e700 1 -- 172.21.15.17:0/2155464761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff16c0b2710 msgr2=0x7ff16c14afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:14.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.286+0000 7ff171f6e700 1 --2- 172.21.15.17:0/2155464761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff16c0b2710 0x7ff16c14afd0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7ff160002f70 tx=0x7ff160006df0).stop 2022-08-01T19:42:14.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.286+0000 7ff171f6e700 1 -- 172.21.15.17:0/2155464761 shutdown_connections 2022-08-01T19:42:14.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.286+0000 7ff171f6e700 1 --2- 172.21.15.17:0/2155464761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff16c0b2710 0x7ff16c14afd0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:14.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.286+0000 7ff171f6e700 1 -- 172.21.15.17:0/2155464761 >> 172.21.15.17:0/2155464761 conn(0x7ff16c060310 msgr2=0x7ff16c0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:14.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.286+0000 7ff171f6e700 1 -- 172.21.15.17:0/2155464761 shutdown_connections 2022-08-01T19:42:14.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.286+0000 7ff171f6e700 1 -- 172.21.15.17:0/2155464761 wait complete. 2022-08-01T19:42:14.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.290+0000 7ff171f6e700 1 Processor -- start 2022-08-01T19:42:14.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.290+0000 7ff171f6e700 1 -- start start 2022-08-01T19:42:14.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.290+0000 7ff171f6e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff16c0a8c50 0x7ff16c0b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:14.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.290+0000 7ff171f6e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff16c0b01a0 0x7ff16c0b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:14.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.290+0000 7ff171f6e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff16c0b2710 0x7ff16c159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:14.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.290+0000 7ff171f6e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c14b9a0 con 0x7ff16c0b2710 2022-08-01T19:42:14.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.290+0000 7ff171f6e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c14bc00 con 0x7ff16c0a8c50 2022-08-01T19:42:14.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.290+0000 7ff171f6e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c14c8c0 con 0x7ff16c0b01a0 2022-08-01T19:42:14.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.290+0000 7ff17176d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff16c0b2710 0x7ff16c159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:14.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.290+0000 7ff17176d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff16c0b2710 0x7ff16c159f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43464/0 (socket says 172.21.15.17:43464) 2022-08-01T19:42:14.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.290+0000 7ff17176d700 1 -- 172.21.15.17:0/1416033695 learned_addr learned my addr 172.21.15.17:0/1416033695 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:14.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.290+0000 7ff170f6c700 1 --2- 172.21.15.17:0/1416033695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff16c0a8c50 0x7ff16c0b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:14.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.290+0000 7ff16bfff700 1 --2- 172.21.15.17:0/1416033695 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff16c0b01a0 0x7ff16c0b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:14.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.290+0000 7ff17176d700 1 -- 172.21.15.17:0/1416033695 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff16c0b01a0 msgr2=0x7ff16c0b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:14.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.290+0000 7ff17176d700 1 --2- 172.21.15.17:0/1416033695 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff16c0b01a0 0x7ff16c0b3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:14.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.290+0000 7ff17176d700 1 -- 172.21.15.17:0/1416033695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff16c0a8c50 msgr2=0x7ff16c0b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:14.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.290+0000 7ff17176d700 1 --2- 172.21.15.17:0/1416033695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff16c0a8c50 0x7ff16c0b36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:14.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.290+0000 7ff17176d700 1 -- 172.21.15.17:0/1416033695 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff16000b180 con 0x7ff16c0b2710 2022-08-01T19:42:14.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.290+0000 7ff17176d700 1 --2- 172.21.15.17:0/1416033695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff16c0b2710 0x7ff16c159f00 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 rx=0x7ff164001490 tx=0x7ff164007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:14.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.290+0000 7ff169ffb700 1 -- 172.21.15.17:0/1416033695 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff164005c70 con 0x7ff16c0b2710 2022-08-01T19:42:14.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.290+0000 7ff169ffb700 1 -- 172.21.15.17:0/1416033695 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff164005dd0 con 0x7ff16c0b2710 2022-08-01T19:42:14.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.290+0000 7ff171f6e700 1 -- 172.21.15.17:0/1416033695 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff16c14cb60 con 0x7ff16c0b2710 2022-08-01T19:42:14.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.290+0000 7ff169ffb700 1 -- 172.21.15.17:0/1416033695 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1640072c0 con 0x7ff16c0b2710 2022-08-01T19:42:14.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.290+0000 7ff171f6e700 1 -- 172.21.15.17:0/1416033695 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff16c0b40d0 con 0x7ff16c0b2710 2022-08-01T19:42:14.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.294+0000 7ff169ffb700 1 -- 172.21.15.17:0/1416033695 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff164010070 con 0x7ff16c0b2710 2022-08-01T19:42:14.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.294+0000 7ff171f6e700 1 -- 172.21.15.17:0/1416033695 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff16c14ccf0 con 0x7ff16c0b2710 2022-08-01T19:42:14.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.294+0000 7ff169ffb700 1 --2- 172.21.15.17:0/1416033695 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff14c053a10 0x7ff14c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:14.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.294+0000 7ff169ffb700 1 -- 172.21.15.17:0/1416033695 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff16406baa0 con 0x7ff16c0b2710 2022-08-01T19:42:14.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.294+0000 7ff170f6c700 1 --2- 172.21.15.17:0/1416033695 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff14c053a10 0x7ff14c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:14.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.294+0000 7ff170f6c700 1 --2- 172.21.15.17:0/1416033695 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff14c053a10 0x7ff14c055ed0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7ff16001a000 tx=0x7ff160014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:14.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.298+0000 7ff169ffb700 1 -- 172.21.15.17:0/1416033695 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff1640426d0 con 0x7ff16c0b2710 2022-08-01T19:42:14.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.422+0000 7ff171f6e700 1 -- 172.21.15.17:0/1416033695 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ff16c0574e0 con 0x7ff16c0b2710 2022-08-01T19:42:14.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.422+0000 7ff169ffb700 1 -- 172.21.15.17:0/1416033695 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v117) v1 ==== 110+0+10 (secure 0 0 0) 0x7ff164045cf0 con 0x7ff16c0b2710 2022-08-01T19:42:14.425 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:42:14.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.426+0000 7ff171f6e700 1 -- 172.21.15.17:0/1416033695 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff14c053a10 msgr2=0x7ff14c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:14.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.426+0000 7ff171f6e700 1 --2- 172.21.15.17:0/1416033695 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff14c053a10 0x7ff14c055ed0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7ff16001a000 tx=0x7ff160014000).stop 2022-08-01T19:42:14.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.426+0000 7ff171f6e700 1 -- 172.21.15.17:0/1416033695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff16c0b2710 msgr2=0x7ff16c159f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:14.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.426+0000 7ff171f6e700 1 --2- 172.21.15.17:0/1416033695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff16c0b2710 0x7ff16c159f00 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 rx=0x7ff164001490 tx=0x7ff164007dc0).stop 2022-08-01T19:42:14.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.426+0000 7ff171f6e700 1 -- 172.21.15.17:0/1416033695 shutdown_connections 2022-08-01T19:42:14.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.426+0000 7ff171f6e700 1 --2- 172.21.15.17:0/1416033695 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff14c053a10 0x7ff14c055ed0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:14.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.426+0000 7ff171f6e700 1 --2- 172.21.15.17:0/1416033695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff16c0b2710 0x7ff16c159f00 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:14.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.426+0000 7ff171f6e700 1 --2- 172.21.15.17:0/1416033695 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff16c0b01a0 0x7ff16c0b3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:14.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.426+0000 7ff171f6e700 1 --2- 172.21.15.17:0/1416033695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff16c0a8c50 0x7ff16c0b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:14.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.426+0000 7ff171f6e700 1 -- 172.21.15.17:0/1416033695 >> 172.21.15.17:0/1416033695 conn(0x7ff16c060310 msgr2=0x7ff16c05ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:14.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.426+0000 7ff171f6e700 1 -- 172.21.15.17:0/1416033695 shutdown_connections 2022-08-01T19:42:14.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.426+0000 7ff171f6e700 1 -- 172.21.15.17:0/1416033695 wait complete. 2022-08-01T19:42:14.442 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:42:14.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.566+0000 7f99b6f1d700 1 Processor -- start 2022-08-01T19:42:14.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.566+0000 7f99b6f1d700 1 -- start start 2022-08-01T19:42:14.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.566+0000 7f99b6f1d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f99b0146ae0 0x7f99b01403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:14.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.566+0000 7f99b6f1d700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99b0143cf0 con 0x7f99b00b4310 2022-08-01T19:42:14.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.566+0000 7f99b6f1d700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99b0143f90 con 0x7f99b00b4820 2022-08-01T19:42:14.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.566+0000 7f99b6f1d700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99b01476a0 con 0x7f99b0146ae0 2022-08-01T19:42:14.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.566+0000 7f99b4cb9700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f99b0146ae0 0x7f99b01403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:14.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.566+0000 7f99affff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f99b00b4310 0x7f99b01408b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35662/0 (socket says 172.21.15.17:35662) 2022-08-01T19:42:14.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.566+0000 7f99affff700 1 -- 172.21.15.17:0/661396384 learned_addr learned my addr 172.21.15.17:0/661396384 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:14.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.566+0000 7f99b4cb9700 1 -- 172.21.15.17:0/661396384 >> v1:172.21.15.77:6789/0 conn(0x7f99b00b4820 legacy=0x7f99b00b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:14.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.566+0000 7f99af7fe700 1 -- 172.21.15.17:0/661396384 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 933961403 0 0) 0x7f99b0143f90 con 0x7f99b00b4820 2022-08-01T19:42:14.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.566+0000 7f99b4cb9700 1 -- 172.21.15.17:0/661396384 >> v1:172.21.15.17:6789/0 conn(0x7f99b00b4310 legacy=0x7f99b01408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:14.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.570+0000 7f99b4cb9700 1 -- 172.21.15.17:0/661396384 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b00b0e20 con 0x7f99b0146ae0 2022-08-01T19:42:14.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.570+0000 7f99b4cb9700 1 --2- 172.21.15.17:0/661396384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99b0146ae0 0x7f99b01403c0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 rx=0x7f99a0001110 tx=0x7f9998006df0).ready entity=mon.0 client_cookie=8017928916cec589 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:14.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.570+0000 7f99af7fe700 1 -- 172.21.15.17:0/661396384 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f999800d5c0 con 0x7f99b0146ae0 2022-08-01T19:42:14.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.570+0000 7f99af7fe700 1 -- 172.21.15.17:0/661396384 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9998000e90 con 0x7f99b0146ae0 2022-08-01T19:42:14.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.570+0000 7f99af7fe700 1 -- 172.21.15.17:0/661396384 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f999800d910 con 0x7f99b0146ae0 2022-08-01T19:42:14.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.570+0000 7f99b6f1d700 1 -- 172.21.15.17:0/661396384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99b0146ae0 msgr2=0x7f99b01403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:14.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.570+0000 7f99b6f1d700 1 --2- 172.21.15.17:0/661396384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99b0146ae0 0x7f99b01403c0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 rx=0x7f99a0001110 tx=0x7f9998006df0).stop 2022-08-01T19:42:14.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.570+0000 7f99b6f1d700 1 -- 172.21.15.17:0/661396384 shutdown_connections 2022-08-01T19:42:14.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.570+0000 7f99b6f1d700 1 --2- 172.21.15.17:0/661396384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99b0146ae0 0x7f99b01403c0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:14.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.570+0000 7f99b6f1d700 1 -- 172.21.15.17:0/661396384 >> 172.21.15.17:0/661396384 conn(0x7f99b005c620 msgr2=0x7f99b005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:14.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.570+0000 7f99b6f1d700 1 -- 172.21.15.17:0/661396384 shutdown_connections 2022-08-01T19:42:14.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.570+0000 7f99b6f1d700 1 -- 172.21.15.17:0/661396384 wait complete. 2022-08-01T19:42:14.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.574+0000 7f99b6f1d700 1 Processor -- start 2022-08-01T19:42:14.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.574+0000 7f99b6f1d700 1 -- start start 2022-08-01T19:42:14.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.574+0000 7f99b6f1d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99b00b4310 0x7f99b00b7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:14.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.574+0000 7f99b6f1d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99b00b4820 0x7f99b0152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:14.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.574+0000 7f99b6f1d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f99b0146ae0 0x7f99b0145510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:14.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.574+0000 7f99b6f1d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b00ae990 con 0x7f99b00b4310 2022-08-01T19:42:14.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.574+0000 7f99b6f1d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b00aec30 con 0x7f99b00b4820 2022-08-01T19:42:14.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.574+0000 7f99b6f1d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99b00aeed0 con 0x7f99b0146ae0 2022-08-01T19:42:14.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.574+0000 7f99b4cb9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99b00b4310 0x7f99b00b7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:14.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.574+0000 7f99b4cb9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99b00b4310 0x7f99b00b7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43474/0 (socket says 172.21.15.17:43474) 2022-08-01T19:42:14.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.574+0000 7f99b4cb9700 1 -- 172.21.15.17:0/1067020400 learned_addr learned my addr 172.21.15.17:0/1067020400 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:14.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.574+0000 7f99affff700 1 --2- 172.21.15.17:0/1067020400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99b00b4820 0x7f99b0152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:14.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.574+0000 7f99b54ba700 1 --2- 172.21.15.17:0/1067020400 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f99b0146ae0 0x7f99b0145510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:14.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.574+0000 7f99b4cb9700 1 -- 172.21.15.17:0/1067020400 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f99b0146ae0 msgr2=0x7f99b0145510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:14.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.574+0000 7f99b4cb9700 1 --2- 172.21.15.17:0/1067020400 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f99b0146ae0 0x7f99b0145510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:14.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.574+0000 7f99b4cb9700 1 -- 172.21.15.17:0/1067020400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99b00b4820 msgr2=0x7f99b0152df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:14.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.574+0000 7f99b4cb9700 1 --2- 172.21.15.17:0/1067020400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99b00b4820 0x7f99b0152df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:14.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.574+0000 7f99b4cb9700 1 -- 172.21.15.17:0/1067020400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f999800b180 con 0x7f99b00b4310 2022-08-01T19:42:14.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.574+0000 7f99b54ba700 1 --2- 172.21.15.17:0/1067020400 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f99b0146ae0 0x7f99b0145510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:42:14.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.574+0000 7f99b4cb9700 1 --2- 172.21.15.17:0/1067020400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99b00b4310 0x7f99b00b7af0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 rx=0x7f99a0001110 tx=0x7f9998014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:14.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.574+0000 7f99adffb700 1 -- 172.21.15.17:0/1067020400 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9998013e90 con 0x7f99b00b4310 2022-08-01T19:42:14.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.574+0000 7f99adffb700 1 -- 172.21.15.17:0/1067020400 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9998002f50 con 0x7f99b00b4310 2022-08-01T19:42:14.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.574+0000 7f99b6f1d700 1 -- 172.21.15.17:0/1067020400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99b00af220 con 0x7f99b00b4310 2022-08-01T19:42:14.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.574+0000 7f99adffb700 1 -- 172.21.15.17:0/1067020400 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f999801e5f0 con 0x7f99b00b4310 2022-08-01T19:42:14.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.574+0000 7f99b6f1d700 1 -- 172.21.15.17:0/1067020400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99b00ae480 con 0x7f99b00b4310 2022-08-01T19:42:14.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.578+0000 7f99adffb700 1 -- 172.21.15.17:0/1067020400 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9998025030 con 0x7f99b00b4310 2022-08-01T19:42:14.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.578+0000 7f99b6f1d700 1 -- 172.21.15.17:0/1067020400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99b00af3b0 con 0x7f99b00b4310 2022-08-01T19:42:14.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.578+0000 7f99adffb700 1 --2- 172.21.15.17:0/1067020400 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9994053b00 0x7f9994055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:14.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.578+0000 7f99adffb700 1 -- 172.21.15.17:0/1067020400 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f999806cdf0 con 0x7f99b00b4310 2022-08-01T19:42:14.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.578+0000 7f99affff700 1 --2- 172.21.15.17:0/1067020400 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9994053b00 0x7f9994055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:14.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.578+0000 7f99affff700 1 --2- 172.21.15.17:0/1067020400 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9994053b00 0x7f9994055fc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f99a000d000 tx=0x7f99a0009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:14.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.582+0000 7f99adffb700 1 -- 172.21.15.17:0/1067020400 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9998043bb0 con 0x7f99b00b4310 2022-08-01T19:42:14.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.710+0000 7f99b6f1d700 1 -- 172.21.15.17:0/1067020400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f99b00574e0 con 0x7f99b00b4310 2022-08-01T19:42:14.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.710+0000 7f99adffb700 1 -- 172.21.15.17:0/1067020400 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v117) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9998023070 con 0x7f99b00b4310 2022-08-01T19:42:14.712 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:42:14.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.710+0000 7f99b6f1d700 1 -- 172.21.15.17:0/1067020400 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9994053b00 msgr2=0x7f9994055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:14.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.710+0000 7f99b6f1d700 1 --2- 172.21.15.17:0/1067020400 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9994053b00 0x7f9994055fc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f99a000d000 tx=0x7f99a0009040).stop 2022-08-01T19:42:14.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.710+0000 7f99b6f1d700 1 -- 172.21.15.17:0/1067020400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99b00b4310 msgr2=0x7f99b00b7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:14.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.710+0000 7f99b6f1d700 1 --2- 172.21.15.17:0/1067020400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99b00b4310 0x7f99b00b7af0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 rx=0x7f99a0001110 tx=0x7f9998014000).stop 2022-08-01T19:42:14.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.710+0000 7f99b6f1d700 1 -- 172.21.15.17:0/1067020400 shutdown_connections 2022-08-01T19:42:14.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.710+0000 7f99b6f1d700 1 --2- 172.21.15.17:0/1067020400 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9994053b00 0x7f9994055fc0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:14.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.710+0000 7f99b6f1d700 1 --2- 172.21.15.17:0/1067020400 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f99b0146ae0 0x7f99b0145510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:14.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.710+0000 7f99b6f1d700 1 --2- 172.21.15.17:0/1067020400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99b00b4820 0x7f99b0152df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:14.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.710+0000 7f99b6f1d700 1 --2- 172.21.15.17:0/1067020400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99b00b4310 0x7f99b00b7af0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:14.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.710+0000 7f99b6f1d700 1 -- 172.21.15.17:0/1067020400 >> 172.21.15.17:0/1067020400 conn(0x7f99b005c620 msgr2=0x7f99b013b250 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:14.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.710+0000 7f99b6f1d700 1 -- 172.21.15.17:0/1067020400 shutdown_connections 2022-08-01T19:42:14.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.710+0000 7f99b6f1d700 1 -- 172.21.15.17:0/1067020400 wait complete. 2022-08-01T19:42:14.724 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T19:42:14.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.830+0000 7f4b1a2c3700 1 Processor -- start 2022-08-01T19:42:14.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.830+0000 7f4b1a2c3700 1 -- start start 2022-08-01T19:42:14.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.830+0000 7f4b1a2c3700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4b140614b0 0x7f4b14061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:14.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.830+0000 7f4b1a2c3700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4b141405d0 0x7f4b14140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:14.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.830+0000 7f4b1a2c3700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b140aee80 con 0x7f4b140ad750 2022-08-01T19:42:14.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.830+0000 7f4b1a2c3700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b140b70b0 con 0x7f4b141405d0 2022-08-01T19:42:14.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.830+0000 7f4b1a2c3700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b140af0b0 con 0x7f4b140614b0 2022-08-01T19:42:14.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.830+0000 7f4b19ac2700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4b141405d0 0x7f4b14140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:14.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.830+0000 7f4b18ac0700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f4b140ad750 0x7f4b140b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35678/0 (socket says 172.21.15.17:35678) 2022-08-01T19:42:14.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.830+0000 7f4b18ac0700 1 -- 172.21.15.17:0/517549602 learned_addr learned my addr 172.21.15.17:0/517549602 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:14.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.830+0000 7f4b19ac2700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4b141405d0 0x7f4b14140a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43486/0 (socket says 172.21.15.17:43486) 2022-08-01T19:42:14.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.830+0000 7f4b192c1700 1 --2- 172.21.15.17:0/517549602 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4b140614b0 0x7f4b14061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:14.858 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.834+0000 7f4b19ac2700 1 -- 172.21.15.17:0/517549602 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4b140614b0 msgr2=0x7f4b14061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:14.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.834+0000 7f4b19ac2700 1 --2- 172.21.15.17:0/517549602 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4b140614b0 0x7f4b14061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:14.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.834+0000 7f4b19ac2700 1 -- 172.21.15.17:0/517549602 >> v1:172.21.15.17:6789/0 conn(0x7f4b140ad750 legacy=0x7f4b140b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:14.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.834+0000 7f4b19ac2700 1 -- 172.21.15.17:0/517549602 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b140af350 con 0x7f4b141405d0 2022-08-01T19:42:14.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.834+0000 7f4b19ac2700 1 --2- 172.21.15.17:0/517549602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b141405d0 0x7f4b14140a60 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 rx=0x7f4b0c0032f0 tx=0x7f4b0c0091d0).ready entity=mon.0 client_cookie=4196305850284b4f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:14.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.834+0000 7f4b0bfff700 1 -- 172.21.15.17:0/517549602 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b0c009ea0 con 0x7f4b141405d0 2022-08-01T19:42:14.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.834+0000 7f4b0bfff700 1 -- 172.21.15.17:0/517549602 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0c00b8a0 con 0x7f4b141405d0 2022-08-01T19:42:14.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.834+0000 7f4b0bfff700 1 -- 172.21.15.17:0/517549602 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b0c00ba10 con 0x7f4b141405d0 2022-08-01T19:42:14.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.834+0000 7f4b1a2c3700 1 -- 172.21.15.17:0/517549602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b141405d0 msgr2=0x7f4b14140a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:14.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.834+0000 7f4b1a2c3700 1 --2- 172.21.15.17:0/517549602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b141405d0 0x7f4b14140a60 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 rx=0x7f4b0c0032f0 tx=0x7f4b0c0091d0).stop 2022-08-01T19:42:14.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.834+0000 7f4b1a2c3700 1 -- 172.21.15.17:0/517549602 shutdown_connections 2022-08-01T19:42:14.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.834+0000 7f4b1a2c3700 1 --2- 172.21.15.17:0/517549602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b141405d0 0x7f4b14140a60 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:14.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.834+0000 7f4b1a2c3700 1 --2- 172.21.15.17:0/517549602 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4b140614b0 0x7f4b14061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:14.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.834+0000 7f4b1a2c3700 1 -- 172.21.15.17:0/517549602 >> 172.21.15.17:0/517549602 conn(0x7f4b14066f70 msgr2=0x7f4b14067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:14.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.834+0000 7f4b1a2c3700 1 -- 172.21.15.17:0/517549602 shutdown_connections 2022-08-01T19:42:14.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.834+0000 7f4b1a2c3700 1 -- 172.21.15.17:0/517549602 wait complete. 2022-08-01T19:42:14.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.834+0000 7f4b1a2c3700 1 Processor -- start 2022-08-01T19:42:14.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.834+0000 7f4b1a2c3700 1 -- start start 2022-08-01T19:42:14.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.838+0000 7f4b1a2c3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b140614b0 0x7f4b140b3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:14.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.838+0000 7f4b1a2c3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b140ad750 0x7f4b140b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:14.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.838+0000 7f4b1a2c3700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4b141405d0 0x7f4b140b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:14.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.838+0000 7f4b1a2c3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b14153360 con 0x7f4b140614b0 2022-08-01T19:42:14.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.838+0000 7f4b1a2c3700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b140aaa40 con 0x7f4b140ad750 2022-08-01T19:42:14.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.838+0000 7f4b1a2c3700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b140aace0 con 0x7f4b141405d0 2022-08-01T19:42:14.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.838+0000 7f4b192c1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b140614b0 0x7f4b140b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:14.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.838+0000 7f4b192c1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b140614b0 0x7f4b140b3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43488/0 (socket says 172.21.15.17:43488) 2022-08-01T19:42:14.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.838+0000 7f4b192c1700 1 -- 172.21.15.17:0/3735387639 learned_addr learned my addr 172.21.15.17:0/3735387639 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:14.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.838+0000 7f4b19ac2700 1 --2- 172.21.15.17:0/3735387639 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4b141405d0 0x7f4b140b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:14.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.838+0000 7f4b18ac0700 1 --2- 172.21.15.17:0/3735387639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b140ad750 0x7f4b140b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:14.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.838+0000 7f4b192c1700 1 -- 172.21.15.17:0/3735387639 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4b141405d0 msgr2=0x7f4b140b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:14.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.838+0000 7f4b192c1700 1 --2- 172.21.15.17:0/3735387639 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4b141405d0 0x7f4b140b0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:14.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.838+0000 7f4b192c1700 1 -- 172.21.15.17:0/3735387639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b140ad750 msgr2=0x7f4b140b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:14.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.838+0000 7f4b192c1700 1 --2- 172.21.15.17:0/3735387639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b140ad750 0x7f4b140b3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:14.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.838+0000 7f4b192c1700 1 -- 172.21.15.17:0/3735387639 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b0c009470 con 0x7f4b140614b0 2022-08-01T19:42:14.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.838+0000 7f4b192c1700 1 --2- 172.21.15.17:0/3735387639 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b140614b0 0x7f4b140b3730 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 rx=0x7f4b040030b0 tx=0x7f4b04008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:14.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.838+0000 7f4b0a7fc700 1 -- 172.21.15.17:0/3735387639 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b04011780 con 0x7f4b140614b0 2022-08-01T19:42:14.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.838+0000 7f4b0a7fc700 1 -- 172.21.15.17:0/3735387639 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b040118e0 con 0x7f4b140614b0 2022-08-01T19:42:14.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.838+0000 7f4b0a7fc700 1 -- 172.21.15.17:0/3735387639 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b04011c40 con 0x7f4b140614b0 2022-08-01T19:42:14.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.838+0000 7f4b1a2c3700 1 -- 172.21.15.17:0/3735387639 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b14152f60 con 0x7f4b140614b0 2022-08-01T19:42:14.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.838+0000 7f4b1a2c3700 1 -- 172.21.15.17:0/3735387639 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b140ab9d0 con 0x7f4b140614b0 2022-08-01T19:42:14.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.838+0000 7f4b1a2c3700 1 -- 172.21.15.17:0/3735387639 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b140af5f0 con 0x7f4b140614b0 2022-08-01T19:42:14.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.838+0000 7f4b0a7fc700 1 -- 172.21.15.17:0/3735387639 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4b0401c460 con 0x7f4b140614b0 2022-08-01T19:42:14.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.842+0000 7f4b0a7fc700 1 --2- 172.21.15.17:0/3735387639 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4afc053950 0x7f4afc055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:14.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.842+0000 7f4b0a7fc700 1 -- 172.21.15.17:0/3735387639 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4b0406c1d0 con 0x7f4b140614b0 2022-08-01T19:42:14.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.842+0000 7f4b18ac0700 1 --2- 172.21.15.17:0/3735387639 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4afc053950 0x7f4afc055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:14.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.842+0000 7f4b18ac0700 1 --2- 172.21.15.17:0/3735387639 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4afc053950 0x7f4afc055e10 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f4b1000d000 tx=0x7f4b10009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:14.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.846+0000 7f4b0a7fc700 1 -- 172.21.15.17:0/3735387639 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4b04042f30 con 0x7f4b140614b0 2022-08-01T19:42:14.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.974+0000 7f4b1a2c3700 1 -- 172.21.15.17:0/3735387639 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4b140574e0 con 0x7f4b140614b0 2022-08-01T19:42:14.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.974+0000 7f4b0a7fc700 1 -- 172.21.15.17:0/3735387639 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v117) v1 ==== 104+0+10 (secure 0 0 0) 0x7f4b04046550 con 0x7f4b140614b0 2022-08-01T19:42:14.981 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:42:14.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.978+0000 7f4b1a2c3700 1 -- 172.21.15.17:0/3735387639 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4afc053950 msgr2=0x7f4afc055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:14.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.978+0000 7f4b1a2c3700 1 --2- 172.21.15.17:0/3735387639 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4afc053950 0x7f4afc055e10 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f4b1000d000 tx=0x7f4b10009040).stop 2022-08-01T19:42:14.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.978+0000 7f4b1a2c3700 1 -- 172.21.15.17:0/3735387639 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b140614b0 msgr2=0x7f4b140b3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:14.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.978+0000 7f4b1a2c3700 1 --2- 172.21.15.17:0/3735387639 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b140614b0 0x7f4b140b3730 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 rx=0x7f4b040030b0 tx=0x7f4b04008040).stop 2022-08-01T19:42:14.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.978+0000 7f4b1a2c3700 1 -- 172.21.15.17:0/3735387639 shutdown_connections 2022-08-01T19:42:14.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.978+0000 7f4b1a2c3700 1 --2- 172.21.15.17:0/3735387639 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4afc053950 0x7f4afc055e10 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:14.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.978+0000 7f4b1a2c3700 1 --2- 172.21.15.17:0/3735387639 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4b141405d0 0x7f4b140b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:14.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.978+0000 7f4b1a2c3700 1 --2- 172.21.15.17:0/3735387639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b140ad750 0x7f4b140b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:14.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.978+0000 7f4b1a2c3700 1 --2- 172.21.15.17:0/3735387639 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b140614b0 0x7f4b140b3730 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:14.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.978+0000 7f4b1a2c3700 1 -- 172.21.15.17:0/3735387639 >> 172.21.15.17:0/3735387639 conn(0x7f4b14066f70 msgr2=0x7f4b140b8990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:14.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.978+0000 7f4b1a2c3700 1 -- 172.21.15.17:0/3735387639 shutdown_connections 2022-08-01T19:42:14.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:14.978+0000 7f4b1a2c3700 1 -- 172.21.15.17:0/3735387639 wait complete. 2022-08-01T19:42:14.996 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T19:42:15.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.110+0000 7f5bd9179700 1 Processor -- start 2022-08-01T19:42:15.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.110+0000 7f5bd9179700 1 -- start start 2022-08-01T19:42:15.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.114+0000 7f5bd9179700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5bd40b01a0 0x7f5bd40a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:15.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.114+0000 7f5bd9179700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f5bd40afd60 0x7f5bd40a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:15.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.114+0000 7f5bd9179700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bd40a7250 con 0x7f5bd40b2710 2022-08-01T19:42:15.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.114+0000 7f5bd9179700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bd40a7480 con 0x7f5bd40b01a0 2022-08-01T19:42:15.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.114+0000 7f5bd9179700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bd40a7720 con 0x7f5bd40afd60 2022-08-01T19:42:15.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.114+0000 7f5bd37fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5bd40b01a0 0x7f5bd40a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:15.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.114+0000 7f5bd37fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5bd40b01a0 0x7f5bd40a8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43498/0 (socket says 172.21.15.17:43498) 2022-08-01T19:42:15.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.114+0000 7f5bd37fe700 1 -- 172.21.15.17:0/3806731178 learned_addr learned my addr 172.21.15.17:0/3806731178 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:15.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.114+0000 7f5bd8978700 1 --2- 172.21.15.17:0/3806731178 >> v2:172.21.15.77:3300/0 conn(0x7f5bd40afd60 0x7f5bd40a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:15.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.114+0000 7f5bd37fe700 1 -- 172.21.15.17:0/3806731178 >> v2:172.21.15.77:3300/0 conn(0x7f5bd40afd60 msgr2=0x7f5bd40a8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:15.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.114+0000 7f5bd37fe700 1 --2- 172.21.15.17:0/3806731178 >> v2:172.21.15.77:3300/0 conn(0x7f5bd40afd60 0x7f5bd40a8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:15.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.114+0000 7f5bd37fe700 1 -- 172.21.15.17:0/3806731178 >> v1:172.21.15.77:6789/0 conn(0x7f5bd40b2710 legacy=0x7f5bd414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:15.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.114+0000 7f5bd37fe700 1 -- 172.21.15.17:0/3806731178 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd40a79c0 con 0x7f5bd40b01a0 2022-08-01T19:42:15.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.114+0000 7f5bd37fe700 1 --2- 172.21.15.17:0/3806731178 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bd40b01a0 0x7f5bd40a8540 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 rx=0x7f5bc40032f0 tx=0x7f5bc4009290).ready entity=mon.0 client_cookie=d953cf90b574a254 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:15.169 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.114+0000 7f5bd2ffd700 1 -- 172.21.15.17:0/3806731178 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bc400b870 con 0x7f5bd40b01a0 2022-08-01T19:42:15.169 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.114+0000 7f5bd2ffd700 1 -- 172.21.15.17:0/3806731178 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc400b9d0 con 0x7f5bd40b01a0 2022-08-01T19:42:15.171 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.114+0000 7f5bd2ffd700 1 -- 172.21.15.17:0/3806731178 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bc400bb40 con 0x7f5bd40b01a0 2022-08-01T19:42:15.171 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.114+0000 7f5bd9179700 1 -- 172.21.15.17:0/3806731178 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bd40b01a0 msgr2=0x7f5bd40a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:15.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.114+0000 7f5bd9179700 1 --2- 172.21.15.17:0/3806731178 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bd40b01a0 0x7f5bd40a8540 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 rx=0x7f5bc40032f0 tx=0x7f5bc4009290).stop 2022-08-01T19:42:15.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.114+0000 7f5bd9179700 1 -- 172.21.15.17:0/3806731178 shutdown_connections 2022-08-01T19:42:15.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.114+0000 7f5bd9179700 1 --2- 172.21.15.17:0/3806731178 >> v2:172.21.15.77:3300/0 conn(0x7f5bd40afd60 0x7f5bd40a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:15.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.114+0000 7f5bd9179700 1 --2- 172.21.15.17:0/3806731178 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bd40b01a0 0x7f5bd40a8540 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:15.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.114+0000 7f5bd9179700 1 -- 172.21.15.17:0/3806731178 >> 172.21.15.17:0/3806731178 conn(0x7f5bd4060310 msgr2=0x7f5bd40b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:15.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.114+0000 7f5bd9179700 1 -- 172.21.15.17:0/3806731178 shutdown_connections 2022-08-01T19:42:15.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.114+0000 7f5bd9179700 1 -- 172.21.15.17:0/3806731178 wait complete. 2022-08-01T19:42:15.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.114+0000 7f5bd9179700 1 Processor -- start 2022-08-01T19:42:15.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.114+0000 7f5bd9179700 1 -- start start 2022-08-01T19:42:15.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.118+0000 7f5bd9179700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5bd40afd60 0x7f5bd4063840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:15.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.118+0000 7f5bd9179700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5bd40b01a0 0x7f5bd41565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:15.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.118+0000 7f5bd9179700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bd40b2710 0x7f5bd40667b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:15.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.118+0000 7f5bd9179700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd4154270 con 0x7f5bd40b2710 2022-08-01T19:42:15.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.118+0000 7f5bd9179700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd4066350 con 0x7f5bd40b01a0 2022-08-01T19:42:15.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.118+0000 7f5bd9179700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd40b60d0 con 0x7f5bd40afd60 2022-08-01T19:42:15.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.118+0000 7f5bd8978700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bd40b2710 0x7f5bd40667b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:15.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.118+0000 7f5bd8978700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bd40b2710 0x7f5bd40667b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43506/0 (socket says 172.21.15.17:43506) 2022-08-01T19:42:15.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.118+0000 7f5bd8978700 1 -- 172.21.15.17:0/1993913992 learned_addr learned my addr 172.21.15.17:0/1993913992 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:15.188 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.118+0000 7f5bd37fe700 1 --2- 172.21.15.17:0/1993913992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5bd40b01a0 0x7f5bd41565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:15.188 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.118+0000 7f5bd3fff700 1 --2- 172.21.15.17:0/1993913992 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5bd40afd60 0x7f5bd4063840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:15.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.118+0000 7f5bd8978700 1 -- 172.21.15.17:0/1993913992 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5bd40afd60 msgr2=0x7f5bd4063840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:15.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.118+0000 7f5bd8978700 1 --2- 172.21.15.17:0/1993913992 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5bd40afd60 0x7f5bd4063840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:15.190 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.118+0000 7f5bd8978700 1 -- 172.21.15.17:0/1993913992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5bd40b01a0 msgr2=0x7f5bd41565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:15.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.118+0000 7f5bd8978700 1 --2- 172.21.15.17:0/1993913992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5bd40b01a0 0x7f5bd41565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:15.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.118+0000 7f5bd8978700 1 -- 172.21.15.17:0/1993913992 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc4009530 con 0x7f5bd40b2710 2022-08-01T19:42:15.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.118+0000 7f5bd37fe700 1 --2- 172.21.15.17:0/1993913992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5bd40b01a0 0x7f5bd41565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:42:15.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.118+0000 7f5bd8978700 1 --2- 172.21.15.17:0/1993913992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bd40b2710 0x7f5bd40667b0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 rx=0x7f5bcc00a000 tx=0x7f5bcc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:15.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.118+0000 7f5bd17fa700 1 -- 172.21.15.17:0/1993913992 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bcc012610 con 0x7f5bd40b2710 2022-08-01T19:42:15.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.118+0000 7f5bd17fa700 1 -- 172.21.15.17:0/1993913992 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bcc012770 con 0x7f5bd40b2710 2022-08-01T19:42:15.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.118+0000 7f5bd17fa700 1 -- 172.21.15.17:0/1993913992 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bcc012ad0 con 0x7f5bd40b2710 2022-08-01T19:42:15.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.118+0000 7f5bd9179700 1 -- 172.21.15.17:0/1993913992 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bd40b5b90 con 0x7f5bd40b2710 2022-08-01T19:42:15.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.118+0000 7f5bd9179700 1 -- 172.21.15.17:0/1993913992 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd4156ab0 con 0x7f5bd40b2710 2022-08-01T19:42:15.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.122+0000 7f5bd9179700 1 -- 172.21.15.17:0/1993913992 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bd40664c0 con 0x7f5bd40b2710 2022-08-01T19:42:15.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.122+0000 7f5bd17fa700 1 -- 172.21.15.17:0/1993913992 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5bcc012c30 con 0x7f5bd40b2710 2022-08-01T19:42:15.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.122+0000 7f5bd17fa700 1 --2- 172.21.15.17:0/1993913992 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5bb40539e0 0x7f5bb4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:15.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.122+0000 7f5bd17fa700 1 -- 172.21.15.17:0/1993913992 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5bcc06bec0 con 0x7f5bd40b2710 2022-08-01T19:42:15.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.122+0000 7f5bd3fff700 1 --2- 172.21.15.17:0/1993913992 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5bb40539e0 0x7f5bb4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:15.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.122+0000 7f5bd3fff700 1 --2- 172.21.15.17:0/1993913992 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5bb40539e0 0x7f5bb4055ea0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f5bc800d000 tx=0x7f5bc8009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:15.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.130+0000 7f5bd17fa700 1 -- 172.21.15.17:0/1993913992 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5bcc042b70 con 0x7f5bd40b2710 2022-08-01T19:42:15.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.258+0000 7f5bd9179700 1 -- 172.21.15.17:0/1993913992 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5bd40574e0 con 0x7f5bd40b2710 2022-08-01T19:42:15.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.258+0000 7f5bd17fa700 1 -- 172.21.15.17:0/1993913992 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v117) v1 ==== 100+0+10 (secure 0 0 0) 0x7f5bcc046190 con 0x7f5bd40b2710 2022-08-01T19:42:15.265 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:42:15.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.258+0000 7f5bd9179700 1 -- 172.21.15.17:0/1993913992 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5bb40539e0 msgr2=0x7f5bb4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:15.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.258+0000 7f5bd9179700 1 --2- 172.21.15.17:0/1993913992 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5bb40539e0 0x7f5bb4055ea0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f5bc800d000 tx=0x7f5bc8009040).stop 2022-08-01T19:42:15.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.258+0000 7f5bd9179700 1 -- 172.21.15.17:0/1993913992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bd40b2710 msgr2=0x7f5bd40667b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:15.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.258+0000 7f5bd9179700 1 --2- 172.21.15.17:0/1993913992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bd40b2710 0x7f5bd40667b0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 rx=0x7f5bcc00a000 tx=0x7f5bcc008040).stop 2022-08-01T19:42:15.271 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.258+0000 7f5bd9179700 1 -- 172.21.15.17:0/1993913992 shutdown_connections 2022-08-01T19:42:15.271 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.258+0000 7f5bd9179700 1 --2- 172.21.15.17:0/1993913992 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5bb40539e0 0x7f5bb4055ea0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:15.272 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.258+0000 7f5bd9179700 1 --2- 172.21.15.17:0/1993913992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bd40b2710 0x7f5bd40667b0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:15.272 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.258+0000 7f5bd9179700 1 --2- 172.21.15.17:0/1993913992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5bd40b01a0 0x7f5bd41565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:15.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.258+0000 7f5bd9179700 1 --2- 172.21.15.17:0/1993913992 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5bd40afd60 0x7f5bd4063840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:15.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.258+0000 7f5bd9179700 1 -- 172.21.15.17:0/1993913992 >> 172.21.15.17:0/1993913992 conn(0x7f5bd4060310 msgr2=0x7f5bd41466e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:15.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.258+0000 7f5bd9179700 1 -- 172.21.15.17:0/1993913992 shutdown_connections 2022-08-01T19:42:15.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.258+0000 7f5bd9179700 1 -- 172.21.15.17:0/1993913992 wait complete. 2022-08-01T19:42:15.278 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T19:42:15.279 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:42:15.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.378+0000 7f36b0946700 1 Processor -- start 2022-08-01T19:42:15.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.378+0000 7f36b0946700 1 -- start start 2022-08-01T19:42:15.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.378+0000 7f36b0946700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f36ac060290 0x7f36ac0b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:15.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.378+0000 7f36b0946700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f36ac0b4dd0 0x7f36ac14b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:15.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.378+0000 7f36b0946700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36ac0aad40 con 0x7f36ac067490 2022-08-01T19:42:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.378+0000 7f36b0946700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36ac060670 con 0x7f36ac060290 2022-08-01T19:42:15.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.378+0000 7f36b0946700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac067030 con 0x7f36ac0b4dd0 2022-08-01T19:42:15.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.378+0000 7f36aa59c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f36ac060290 0x7f36ac0b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:15.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.378+0000 7f36aa59c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f36ac060290 0x7f36ac0b48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43512/0 (socket says 172.21.15.17:43512) 2022-08-01T19:42:15.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.378+0000 7f36aa59c700 1 -- 172.21.15.17:0/3504957644 learned_addr learned my addr 172.21.15.17:0/3504957644 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:15.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.378+0000 7f36aad9d700 1 --2- 172.21.15.17:0/3504957644 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f36ac0b4dd0 0x7f36ac14b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:15.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.378+0000 7f36aa59c700 1 -- 172.21.15.17:0/3504957644 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f36ac0b4dd0 msgr2=0x7f36ac14b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:15.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.378+0000 7f36aa59c700 1 --2- 172.21.15.17:0/3504957644 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f36ac0b4dd0 0x7f36ac14b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:15.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.378+0000 7f36aa59c700 1 -- 172.21.15.17:0/3504957644 >> v1:172.21.15.77:6789/0 conn(0x7f36ac067490 legacy=0x7f36ac143a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:15.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.382+0000 7f36aa59c700 1 -- 172.21.15.17:0/3504957644 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ac0b3540 con 0x7f36ac060290 2022-08-01T19:42:15.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.382+0000 7f36aa59c700 1 --2- 172.21.15.17:0/3504957644 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36ac060290 0x7f36ac0b48e0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 rx=0x7f36a0002f70 tx=0x7f36a0006eb0).ready entity=mon.0 client_cookie=bd29593a569f89a2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:15.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.382+0000 7f36a959a700 1 -- 172.21.15.17:0/3504957644 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36a000d570 con 0x7f36ac060290 2022-08-01T19:42:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.382+0000 7f36a959a700 1 -- 172.21.15.17:0/3504957644 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a0000e90 con 0x7f36ac060290 2022-08-01T19:42:15.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.382+0000 7f36a959a700 1 -- 172.21.15.17:0/3504957644 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36a000d900 con 0x7f36ac060290 2022-08-01T19:42:15.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.382+0000 7f36b0946700 1 -- 172.21.15.17:0/3504957644 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36ac060290 msgr2=0x7f36ac0b48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:15.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.382+0000 7f36b0946700 1 --2- 172.21.15.17:0/3504957644 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36ac060290 0x7f36ac0b48e0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 rx=0x7f36a0002f70 tx=0x7f36a0006eb0).stop 2022-08-01T19:42:15.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.382+0000 7f36b0946700 1 -- 172.21.15.17:0/3504957644 shutdown_connections 2022-08-01T19:42:15.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.382+0000 7f36b0946700 1 --2- 172.21.15.17:0/3504957644 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f36ac0b4dd0 0x7f36ac14b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:15.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.382+0000 7f36b0946700 1 --2- 172.21.15.17:0/3504957644 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36ac060290 0x7f36ac0b48e0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:15.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.382+0000 7f36b0946700 1 -- 172.21.15.17:0/3504957644 >> 172.21.15.17:0/3504957644 conn(0x7f36ac05d0c0 msgr2=0x7f36ac05f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:15.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.382+0000 7f36b0946700 1 -- 172.21.15.17:0/3504957644 shutdown_connections 2022-08-01T19:42:15.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.382+0000 7f36b0946700 1 -- 172.21.15.17:0/3504957644 wait complete. 2022-08-01T19:42:15.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.382+0000 7f36b0946700 1 Processor -- start 2022-08-01T19:42:15.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.382+0000 7f36b0946700 1 -- start start 2022-08-01T19:42:15.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.386+0000 7f36b0946700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36ac060290 0x7f36ac0a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:15.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.386+0000 7f36b0946700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f36ac067490 0x7f36ac14d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:15.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.386+0000 7f36b0946700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36ac0b4dd0 0x7f36ac14dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:15.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.386+0000 7f36b0946700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac09f880 con 0x7f36ac0b4dd0 2022-08-01T19:42:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.386+0000 7f36b0946700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac09eee0 con 0x7f36ac060290 2022-08-01T19:42:15.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.386+0000 7f36b0946700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac1511a0 con 0x7f36ac067490 2022-08-01T19:42:15.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.386+0000 7f36aad9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36ac0b4dd0 0x7f36ac14dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:15.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.386+0000 7f36aad9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36ac0b4dd0 0x7f36ac14dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43520/0 (socket says 172.21.15.17:43520) 2022-08-01T19:42:15.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.386+0000 7f36aad9d700 1 -- 172.21.15.17:0/3967129104 learned_addr learned my addr 172.21.15.17:0/3967129104 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:15.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.386+0000 7f36aa59c700 1 --2- 172.21.15.17:0/3967129104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36ac060290 0x7f36ac0a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:15.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.386+0000 7f36a9d9b700 1 --2- 172.21.15.17:0/3967129104 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f36ac067490 0x7f36ac14d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:15.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.386+0000 7f36aad9d700 1 -- 172.21.15.17:0/3967129104 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f36ac067490 msgr2=0x7f36ac14d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:15.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.386+0000 7f36aad9d700 1 --2- 172.21.15.17:0/3967129104 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f36ac067490 0x7f36ac14d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:15.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.386+0000 7f36aad9d700 1 -- 172.21.15.17:0/3967129104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36ac060290 msgr2=0x7f36ac0a2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:15.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.386+0000 7f36aad9d700 1 --2- 172.21.15.17:0/3967129104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36ac060290 0x7f36ac0a2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:15.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.386+0000 7f36aad9d700 1 -- 172.21.15.17:0/3967129104 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a000b1e0 con 0x7f36ac0b4dd0 2022-08-01T19:42:15.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.386+0000 7f36a9d9b700 1 --2- 172.21.15.17:0/3967129104 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f36ac067490 0x7f36ac14d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:42:15.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.386+0000 7f36aa59c700 1 --2- 172.21.15.17:0/3967129104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36ac060290 0x7f36ac0a2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:42:15.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.386+0000 7f36aad9d700 1 --2- 172.21.15.17:0/3967129104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36ac0b4dd0 0x7f36ac14dd10 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 rx=0x7f36a400c000 tx=0x7f36a40091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:15.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.386+0000 7f369b7fe700 1 -- 172.21.15.17:0/3967129104 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36a400ee70 con 0x7f36ac0b4dd0 2022-08-01T19:42:15.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.386+0000 7f369b7fe700 1 -- 172.21.15.17:0/3967129104 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a400c850 con 0x7f36ac0b4dd0 2022-08-01T19:42:15.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.386+0000 7f36b0946700 1 -- 172.21.15.17:0/3967129104 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36ac09f3c0 con 0x7f36ac0b4dd0 2022-08-01T19:42:15.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.386+0000 7f369b7fe700 1 -- 172.21.15.17:0/3967129104 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36a400cbb0 con 0x7f36ac0b4dd0 2022-08-01T19:42:15.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.386+0000 7f36b0946700 1 -- 172.21.15.17:0/3967129104 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36ac150ea0 con 0x7f36ac0b4dd0 2022-08-01T19:42:15.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.386+0000 7f36b0946700 1 -- 172.21.15.17:0/3967129104 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36ac151360 con 0x7f36ac0b4dd0 2022-08-01T19:42:15.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.390+0000 7f369b7fe700 1 -- 172.21.15.17:0/3967129104 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f36a4022070 con 0x7f36ac0b4dd0 2022-08-01T19:42:15.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.390+0000 7f369b7fe700 1 --2- 172.21.15.17:0/3967129104 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3688053980 0x7f3688055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:15.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.390+0000 7f369b7fe700 1 -- 172.21.15.17:0/3967129104 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f36a406c800 con 0x7f36ac0b4dd0 2022-08-01T19:42:15.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.390+0000 7f36aa59c700 1 --2- 172.21.15.17:0/3967129104 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3688053980 0x7f3688055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:15.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.390+0000 7f36aa59c700 1 --2- 172.21.15.17:0/3967129104 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3688053980 0x7f3688055e40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f36a001a000 tx=0x7f36a0015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:15.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.398+0000 7f369b7fe700 1 -- 172.21.15.17:0/3967129104 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f36a40434b0 con 0x7f36ac0b4dd0 2022-08-01T19:42:15.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.562+0000 7f36b0946700 1 -- 172.21.15.17:0/3967129104 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36ac0574e0 con 0x7f36ac0b4dd0 2022-08-01T19:42:15.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.562+0000 7f369b7fe700 1 -- 172.21.15.17:0/3967129104 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f36a4016070 con 0x7f36ac0b4dd0 2022-08-01T19:42:15.565 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:42:15.566 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":58,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7581,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/1744640648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":1744640648},{"type":"v1","addr":"172.21.15.17:6833","nonce":1744640648}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":56},{"gid":7594,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.17:6835/346594655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":346594655},{"type":"v1","addr":"172.21.15.17:6835","nonce":346594655}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":57}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:42:10.906134+0000","modified":"2022-08-01T19:42:11.922099+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7600},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7600":{"gid":7600,"name":"b","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6835/2286085186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2286085186},{"type":"v1","addr":"172.21.15.77:6835","nonce":2286085186}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2022-08-01T19:42:15.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.566+0000 7f36b0946700 1 -- 172.21.15.17:0/3967129104 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3688053980 msgr2=0x7f3688055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:15.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.566+0000 7f36b0946700 1 --2- 172.21.15.17:0/3967129104 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3688053980 0x7f3688055e40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f36a001a000 tx=0x7f36a0015000).stop 2022-08-01T19:42:15.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.566+0000 7f36b0946700 1 -- 172.21.15.17:0/3967129104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36ac0b4dd0 msgr2=0x7f36ac14dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:15.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.566+0000 7f36b0946700 1 --2- 172.21.15.17:0/3967129104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36ac0b4dd0 0x7f36ac14dd10 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 rx=0x7f36a400c000 tx=0x7f36a40091d0).stop 2022-08-01T19:42:15.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.566+0000 7f36b0946700 1 -- 172.21.15.17:0/3967129104 shutdown_connections 2022-08-01T19:42:15.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.566+0000 7f36b0946700 1 --2- 172.21.15.17:0/3967129104 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3688053980 0x7f3688055e40 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:15.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.566+0000 7f36b0946700 1 --2- 172.21.15.17:0/3967129104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36ac0b4dd0 0x7f36ac14dd10 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:15.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.566+0000 7f36b0946700 1 --2- 172.21.15.17:0/3967129104 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f36ac067490 0x7f36ac14d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:15.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.566+0000 7f36b0946700 1 --2- 172.21.15.17:0/3967129104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36ac060290 0x7f36ac0a2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:15.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.566+0000 7f36b0946700 1 -- 172.21.15.17:0/3967129104 >> 172.21.15.17:0/3967129104 conn(0x7f36ac05d0c0 msgr2=0x7f36ac143850 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:15.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.566+0000 7f36b0946700 1 -- 172.21.15.17:0/3967129104 shutdown_connections 2022-08-01T19:42:15.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:15.566+0000 7f36b0946700 1 -- 172.21.15.17:0/3967129104 wait complete. 2022-08-01T19:42:15.573 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 58 2022-08-01T19:42:15.883 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 58, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:42:10.906134+0000', 'modified': '2022-08-01T19:42:11.922099+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7600}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7600': {'gid': 7600, 'name': 'b', 'rank': 0, 'incarnation': 57, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.77:6835/2286085186', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6834', 'nonce': 2286085186}, {'type': 'v1', 'addr': '172.21.15.77:6835', 'nonce': 2286085186}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:42:15.884 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:42:15.884 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-08-01T19:42:16.007 INFO:teuthology.orchestra.run.smithi077.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":7600,"want_state":"up:active","state":"up:active","rank_uptime":5.0863309330000002,"mdsmap_epoch":58,"osdmap_epoch":117,"osdmap_epoch_barrier":117,"uptime":9.4440756629999996} 2022-08-01T19:42:16.007 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 7600, "mdsmap_epoch": 58, "osdmap_epoch": 117, "osdmap_epoch_barrier": 117, "rank_uptime": 5.086330933, "state": "up:active", "uptime": 9.444075663, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:42:16.009 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T19:42:16.010 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-08-01T19:42:16.011 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi085.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-08-01T19:42:16.012 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:42:16.013 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:42:16.031 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:42:16.090 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:42:16.091 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:42:16.117 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:42:16.119 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:42:16.122 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:42:16.123 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:42:16.169 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T19:42:16.171 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:42:16.176 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:42:16.208 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:42:16.209 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:42:16.265 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:2022-08-01T19:42:16.250+0000 7fc5918df080 -1 init, newargv = 0x55b4de5a1900 newargc=9 2022-08-01T19:42:16.265 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:ceph-fuse[23439]: starting ceph client 2022-08-01T19:42:16.266 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:ceph-fuse[23439]: starting fuse 2022-08-01T19:42:16.282 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:42:16.284 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:42:16.284 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:42:16.286 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:42:16.362 INFO:teuthology.orchestra.run.smithi085.stdout:52 2022-08-01T19:42:16.363 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T19:42:16.363 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:42:16.364 DEBUG:teuthology.orchestra.run.smithi085:> sudo python3 -c ' 2022-08-01T19:42:16.364 DEBUG:teuthology.orchestra.run.smithi085:> import glob 2022-08-01T19:42:16.365 DEBUG:teuthology.orchestra.run.smithi085:> import re 2022-08-01T19:42:16.366 DEBUG:teuthology.orchestra.run.smithi085:> import os 2022-08-01T19:42:16.367 DEBUG:teuthology.orchestra.run.smithi085:> import subprocess 2022-08-01T19:42:16.368 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:42:16.368 DEBUG:teuthology.orchestra.run.smithi085:> def find_socket(client_name): 2022-08-01T19:42:16.370 DEBUG:teuthology.orchestra.run.smithi085:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-08-01T19:42:16.370 DEBUG:teuthology.orchestra.run.smithi085:> files = glob.glob(asok_path) 2022-08-01T19:42:16.371 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:42:16.372 DEBUG:teuthology.orchestra.run.smithi085:> # Given a non-glob path, it better be there 2022-08-01T19:42:16.373 DEBUG:teuthology.orchestra.run.smithi085:> if "*" not in asok_path: 2022-08-01T19:42:16.377 DEBUG:teuthology.orchestra.run.smithi085:> assert(len(files) == 1) 2022-08-01T19:42:16.378 DEBUG:teuthology.orchestra.run.smithi085:> return files[0] 2022-08-01T19:42:16.379 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:42:16.380 DEBUG:teuthology.orchestra.run.smithi085:> for f in files: 2022-08-01T19:42:16.381 DEBUG:teuthology.orchestra.run.smithi085:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T19:42:16.381 DEBUG:teuthology.orchestra.run.smithi085:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T19:42:16.383 DEBUG:teuthology.orchestra.run.smithi085:> return f 2022-08-01T19:42:16.384 DEBUG:teuthology.orchestra.run.smithi085:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T19:42:16.385 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:42:16.385 DEBUG:teuthology.orchestra.run.smithi085:> print(find_socket("client.0")) 2022-08-01T19:42:16.386 DEBUG:teuthology.orchestra.run.smithi085:> ' 2022-08-01T19:42:16.445 INFO:teuthology.orchestra.run.smithi085.stdout:/var/run/ceph/ceph-client.0.23439.asok 2022-08-01T19:42:16.462 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.23439.asok 2022-08-01T19:42:16.463 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:42:16.464 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.23439.asok status 2022-08-01T19:42:16.572 INFO:teuthology.orchestra.run.smithi085.stdout:{ 2022-08-01T19:42:16.572 INFO:teuthology.orchestra.run.smithi085.stdout: "metadata": { 2022-08-01T19:42:16.572 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T19:42:16.572 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T19:42:16.573 INFO:teuthology.orchestra.run.smithi085.stdout: "entity_id": "0", 2022-08-01T19:42:16.573 INFO:teuthology.orchestra.run.smithi085.stdout: "hostname": "smithi085", 2022-08-01T19:42:16.573 INFO:teuthology.orchestra.run.smithi085.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-08-01T19:42:16.573 INFO:teuthology.orchestra.run.smithi085.stdout: "pid": "23439", 2022-08-01T19:42:16.579 INFO:teuthology.orchestra.run.smithi085.stdout: "root": "/" 2022-08-01T19:42:16.580 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:42:16.580 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_count": 0, 2022-08-01T19:42:16.581 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_pinned_count": 0, 2022-08-01T19:42:16.586 INFO:teuthology.orchestra.run.smithi085.stdout: "id": 7758, 2022-08-01T19:42:16.586 INFO:teuthology.orchestra.run.smithi085.stdout: "inst": { 2022-08-01T19:42:16.587 INFO:teuthology.orchestra.run.smithi085.stdout: "name": { 2022-08-01T19:42:16.588 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "client", 2022-08-01T19:42:16.590 INFO:teuthology.orchestra.run.smithi085.stdout: "num": 7758 2022-08-01T19:42:16.590 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:42:16.591 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2022-08-01T19:42:16.592 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2022-08-01T19:42:16.593 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2022-08-01T19:42:16.593 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 3031706315 2022-08-01T19:42:16.593 INFO:teuthology.orchestra.run.smithi085.stdout: } 2022-08-01T19:42:16.594 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:42:16.594 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2022-08-01T19:42:16.595 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2022-08-01T19:42:16.595 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2022-08-01T19:42:16.596 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 3031706315 2022-08-01T19:42:16.597 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:42:16.597 INFO:teuthology.orchestra.run.smithi085.stdout: "inst_str": "client.7758 172.21.15.85:0/3031706315", 2022-08-01T19:42:16.598 INFO:teuthology.orchestra.run.smithi085.stdout: "addr_str": "172.21.15.85:0/3031706315", 2022-08-01T19:42:16.598 INFO:teuthology.orchestra.run.smithi085.stdout: "inode_count": 1, 2022-08-01T19:42:16.599 INFO:teuthology.orchestra.run.smithi085.stdout: "mds_epoch": 58, 2022-08-01T19:42:16.599 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch": 117, 2022-08-01T19:42:16.600 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch_barrier": 0, 2022-08-01T19:42:16.600 INFO:teuthology.orchestra.run.smithi085.stdout: "blacklisted": false, 2022-08-01T19:42:16.601 INFO:teuthology.orchestra.run.smithi085.stdout: "fs_name": "cephfs" 2022-08-01T19:42:16.601 INFO:teuthology.orchestra.run.smithi085.stdout:} 2022-08-01T19:42:16.604 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:42:16.605 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:42:16.606 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:42:16.620 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:42:16.622 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:42:16.622 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:42:16.634 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:42:16.683 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:42:16.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.794+0000 7ffa4b217700 1 Processor -- start 2022-08-01T19:42:16.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.794+0000 7ffa4b217700 1 -- start start 2022-08-01T19:42:16.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.794+0000 7ffa4b217700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffa440b1f40 0x7ffa440b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:16.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.794+0000 7ffa4b217700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa440b04e0 con 0x7ffa440a7250 2022-08-01T19:42:16.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.794+0000 7ffa4b217700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa440b0780 con 0x7ffa44148530 2022-08-01T19:42:16.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.794+0000 7ffa4b217700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa440a9250 con 0x7ffa440b1f40 2022-08-01T19:42:16.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.794+0000 7ffa497b4700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ffa440a7250 0x7ffa441448b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35722/0 (socket says 172.21.15.17:35722) 2022-08-01T19:42:16.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.794+0000 7ffa497b4700 1 -- 172.21.15.17:0/1042120309 learned_addr learned my addr 172.21.15.17:0/1042120309 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:16.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.794+0000 7ffa48fb3700 1 --2- 172.21.15.17:0/1042120309 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffa440b1f40 0x7ffa440b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:16.802 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.794+0000 7ffa437fe700 1 -- 172.21.15.17:0/1042120309 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 102003812 0 0) 0x7ffa440b04e0 con 0x7ffa440a7250 2022-08-01T19:42:16.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.794+0000 7ffa437fe700 1 -- 172.21.15.17:0/1042120309 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa28003260 con 0x7ffa440a7250 2022-08-01T19:42:16.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.794+0000 7ffa437fe700 1 -- 172.21.15.17:0/1042120309 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3039741255 0 0) 0x7ffa440b0780 con 0x7ffa44148530 2022-08-01T19:42:16.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.794+0000 7ffa48fb3700 1 -- 172.21.15.17:0/1042120309 >> v1:172.21.15.77:6789/0 conn(0x7ffa44148530 legacy=0x7ffa440b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:16.804 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.794+0000 7ffa48fb3700 1 -- 172.21.15.17:0/1042120309 >> v1:172.21.15.17:6789/0 conn(0x7ffa440a7250 legacy=0x7ffa441448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:16.804 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.794+0000 7ffa48fb3700 1 -- 172.21.15.17:0/1042120309 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa440b0a20 con 0x7ffa440b1f40 2022-08-01T19:42:16.804 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.794+0000 7ffa48fb3700 1 --2- 172.21.15.17:0/1042120309 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffa440b1f40 0x7ffa440b1160 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7ffa2c002f70 tx=0x7ffa2c006df0).ready entity=mon.2 client_cookie=857315ecf77ef1f0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:16.804 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.794+0000 7ffa437fe700 1 -- 172.21.15.17:0/1042120309 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa2c00d570 con 0x7ffa440b1f40 2022-08-01T19:42:16.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.794+0000 7ffa437fe700 1 -- 172.21.15.17:0/1042120309 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa2c000e90 con 0x7ffa440b1f40 2022-08-01T19:42:16.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.794+0000 7ffa437fe700 1 -- 172.21.15.17:0/1042120309 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa2c00d8c0 con 0x7ffa440b1f40 2022-08-01T19:42:16.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.794+0000 7ffa4b217700 1 -- 172.21.15.17:0/1042120309 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffa440b1f40 msgr2=0x7ffa440b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:16.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.794+0000 7ffa4b217700 1 --2- 172.21.15.17:0/1042120309 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffa440b1f40 0x7ffa440b1160 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7ffa2c002f70 tx=0x7ffa2c006df0).stop 2022-08-01T19:42:16.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.798+0000 7ffa4b217700 1 -- 172.21.15.17:0/1042120309 shutdown_connections 2022-08-01T19:42:16.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.798+0000 7ffa4b217700 1 --2- 172.21.15.17:0/1042120309 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffa440b1f40 0x7ffa440b1160 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:16.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.798+0000 7ffa4b217700 1 -- 172.21.15.17:0/1042120309 >> 172.21.15.17:0/1042120309 conn(0x7ffa4405f840 msgr2=0x7ffa4405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:16.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.798+0000 7ffa4b217700 1 -- 172.21.15.17:0/1042120309 shutdown_connections 2022-08-01T19:42:16.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.798+0000 7ffa4b217700 1 -- 172.21.15.17:0/1042120309 wait complete. 2022-08-01T19:42:16.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.798+0000 7ffa4b217700 1 Processor -- start 2022-08-01T19:42:16.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.798+0000 7ffa4b217700 1 -- start start 2022-08-01T19:42:16.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.798+0000 7ffa4b217700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffa440a7250 0x7ffa44143170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:16.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.798+0000 7ffa4b217700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa440b1f40 0x7ffa4413dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:16.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.798+0000 7ffa4b217700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa44148530 0x7ffa4413e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:16.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.798+0000 7ffa4b217700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa440ad3e0 con 0x7ffa440b1f40 2022-08-01T19:42:16.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.798+0000 7ffa4b217700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa44146ce0 con 0x7ffa44148530 2022-08-01T19:42:16.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.798+0000 7ffa4b217700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa44146a80 con 0x7ffa440a7250 2022-08-01T19:42:16.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.798+0000 7ffa43fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa440b1f40 0x7ffa4413dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:16.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.798+0000 7ffa43fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa440b1f40 0x7ffa4413dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43534/0 (socket says 172.21.15.17:43534) 2022-08-01T19:42:16.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.798+0000 7ffa48fb3700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffa440a7250 0x7ffa44143170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:16.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.798+0000 7ffa43fff700 1 -- 172.21.15.17:0/3316313712 learned_addr learned my addr 172.21.15.17:0/3316313712 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:16.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.798+0000 7ffa497b4700 1 --2- 172.21.15.17:0/3316313712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa44148530 0x7ffa4413e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:16.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.798+0000 7ffa43fff700 1 -- 172.21.15.17:0/3316313712 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffa440a7250 msgr2=0x7ffa44143170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:16.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.798+0000 7ffa43fff700 1 --2- 172.21.15.17:0/3316313712 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffa440a7250 0x7ffa44143170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:16.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.798+0000 7ffa43fff700 1 -- 172.21.15.17:0/3316313712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa44148530 msgr2=0x7ffa4413e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:16.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.798+0000 7ffa43fff700 1 --2- 172.21.15.17:0/3316313712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa44148530 0x7ffa4413e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:16.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.798+0000 7ffa43fff700 1 -- 172.21.15.17:0/3316313712 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa2c00b180 con 0x7ffa440b1f40 2022-08-01T19:42:16.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.798+0000 7ffa497b4700 1 --2- 172.21.15.17:0/3316313712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa44148530 0x7ffa4413e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:42:16.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.798+0000 7ffa43fff700 1 --2- 172.21.15.17:0/3316313712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa440b1f40 0x7ffa4413dc80 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 rx=0x7ffa441569b0 tx=0x7ffa340013f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:16.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.802+0000 7ffa41ffb700 1 -- 172.21.15.17:0/3316313712 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa3400f470 con 0x7ffa440b1f40 2022-08-01T19:42:16.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.802+0000 7ffa4b217700 1 -- 172.21.15.17:0/3316313712 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa440ad140 con 0x7ffa440b1f40 2022-08-01T19:42:16.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.802+0000 7ffa4b217700 1 -- 172.21.15.17:0/3316313712 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa440ad690 con 0x7ffa440b1f40 2022-08-01T19:42:16.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.802+0000 7ffa41ffb700 1 -- 172.21.15.17:0/3316313712 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3400f5d0 con 0x7ffa440b1f40 2022-08-01T19:42:16.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.802+0000 7ffa41ffb700 1 -- 172.21.15.17:0/3316313712 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa3400f930 con 0x7ffa440b1f40 2022-08-01T19:42:16.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.802+0000 7ffa4b217700 1 -- 172.21.15.17:0/3316313712 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa441468e0 con 0x7ffa440b1f40 2022-08-01T19:42:16.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.802+0000 7ffa41ffb700 1 -- 172.21.15.17:0/3316313712 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ffa34019030 con 0x7ffa440b1f40 2022-08-01T19:42:16.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.802+0000 7ffa41ffb700 1 --2- 172.21.15.17:0/3316313712 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ffa1c053aa0 0x7ffa1c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:16.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.806+0000 7ffa41ffb700 1 -- 172.21.15.17:0/3316313712 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ffa3406bbd0 con 0x7ffa440b1f40 2022-08-01T19:42:16.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.806+0000 7ffa48fb3700 1 --2- 172.21.15.17:0/3316313712 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ffa1c053aa0 0x7ffa1c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:16.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.806+0000 7ffa48fb3700 1 --2- 172.21.15.17:0/3316313712 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ffa1c053aa0 0x7ffa1c055f60 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7ffa2c018000 tx=0x7ffa2c014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:16.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.810+0000 7ffa41ffb700 1 -- 172.21.15.17:0/3316313712 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ffa34042990 con 0x7ffa440b1f40 2022-08-01T19:42:16.939 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.934+0000 7ffa4b217700 1 -- 172.21.15.17:0/3316313712 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa441468e0 con 0x7ffa440b1f40 2022-08-01T19:42:16.940 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.934+0000 7ffa41ffb700 1 -- 172.21.15.17:0/3316313712 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+16285 (secure 0 0 0) 0x7ffa441468e0 con 0x7ffa440b1f40 2022-08-01T19:42:16.941 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:42:16.941 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":117,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:42:11.920791+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:42:09.404661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2022-08-01T19:42:10.264672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:42:16.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.938+0000 7ffa4b217700 1 -- 172.21.15.17:0/3316313712 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ffa1c053aa0 msgr2=0x7ffa1c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:16.944 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.938+0000 7ffa4b217700 1 --2- 172.21.15.17:0/3316313712 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ffa1c053aa0 0x7ffa1c055f60 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7ffa2c018000 tx=0x7ffa2c014000).stop 2022-08-01T19:42:16.944 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.938+0000 7ffa4b217700 1 -- 172.21.15.17:0/3316313712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa440b1f40 msgr2=0x7ffa4413dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:16.944 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.938+0000 7ffa4b217700 1 --2- 172.21.15.17:0/3316313712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa440b1f40 0x7ffa4413dc80 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 rx=0x7ffa441569b0 tx=0x7ffa340013f0).stop 2022-08-01T19:42:16.945 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.938+0000 7ffa4b217700 1 -- 172.21.15.17:0/3316313712 shutdown_connections 2022-08-01T19:42:16.945 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.938+0000 7ffa4b217700 1 --2- 172.21.15.17:0/3316313712 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ffa1c053aa0 0x7ffa1c055f60 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:16.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.938+0000 7ffa4b217700 1 --2- 172.21.15.17:0/3316313712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa44148530 0x7ffa4413e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:16.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.938+0000 7ffa4b217700 1 --2- 172.21.15.17:0/3316313712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa440b1f40 0x7ffa4413dc80 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:16.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.938+0000 7ffa4b217700 1 --2- 172.21.15.17:0/3316313712 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ffa440a7250 0x7ffa44143170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:16.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.938+0000 7ffa4b217700 1 -- 172.21.15.17:0/3316313712 >> 172.21.15.17:0/3316313712 conn(0x7ffa4405f840 msgr2=0x7ffa440b4d40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:16.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.938+0000 7ffa4b217700 1 -- 172.21.15.17:0/3316313712 shutdown_connections 2022-08-01T19:42:16.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:16.938+0000 7ffa4b217700 1 -- 172.21.15.17:0/3316313712 wait complete. 2022-08-01T19:42:16.959 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T19:42:09.404661+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T19:42:10.264672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T19:42:16.961 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:42:17.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.078+0000 7f0e2e96f700 1 Processor -- start 2022-08-01T19:42:17.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.078+0000 7f0e2e96f700 1 -- start start 2022-08-01T19:42:17.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.078+0000 7f0e2e96f700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0e280b31d0 0x7f0e28066020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:17.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.078+0000 7f0e2e96f700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f0e280a9da0 0x7f0e28060950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:17.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.078+0000 7f0e2e96f700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0e281434b0 0x7f0e28143940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:17.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.078+0000 7f0e2e96f700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e28064a70 con 0x7f0e281434b0 2022-08-01T19:42:17.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.078+0000 7f0e2e96f700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e28061f60 con 0x7f0e280a9da0 2022-08-01T19:42:17.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.078+0000 7f0e2e96f700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e28067030 con 0x7f0e280b31d0 2022-08-01T19:42:17.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.078+0000 7f0e2e16e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0e281434b0 0x7f0e28143940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:17.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.078+0000 7f0e2e16e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0e281434b0 0x7f0e28143940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43542/0 (socket says 172.21.15.17:43542) 2022-08-01T19:42:17.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.078+0000 7f0e2e16e700 1 -- 172.21.15.17:0/2434503844 learned_addr learned my addr 172.21.15.17:0/2434503844 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:17.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.078+0000 7f0e2d16c700 1 --2- 172.21.15.17:0/2434503844 >> v2:172.21.15.77:3300/0 conn(0x7f0e280a9da0 0x7f0e28060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:17.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.078+0000 7f0e2d96d700 1 --2- 172.21.15.17:0/2434503844 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0e280b31d0 0x7f0e28066020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:17.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.078+0000 7f0e2e16e700 1 -- 172.21.15.17:0/2434503844 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0e280b31d0 msgr2=0x7f0e28066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:17.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.078+0000 7f0e2e16e700 1 --2- 172.21.15.17:0/2434503844 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0e280b31d0 0x7f0e28066020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:17.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.078+0000 7f0e2e16e700 1 -- 172.21.15.17:0/2434503844 >> v2:172.21.15.77:3300/0 conn(0x7f0e280a9da0 msgr2=0x7f0e28060950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:17.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.078+0000 7f0e2e16e700 1 --2- 172.21.15.17:0/2434503844 >> v2:172.21.15.77:3300/0 conn(0x7f0e280a9da0 0x7f0e28060950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:17.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.078+0000 7f0e2e16e700 1 -- 172.21.15.17:0/2434503844 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e280b35b0 con 0x7f0e281434b0 2022-08-01T19:42:17.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.078+0000 7f0e2d16c700 1 --2- 172.21.15.17:0/2434503844 >> v2:172.21.15.77:3300/0 conn(0x7f0e280a9da0 0x7f0e28060950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:42:17.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.078+0000 7f0e2e16e700 1 --2- 172.21.15.17:0/2434503844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e281434b0 0x7f0e28143940 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 rx=0x7f0e200032f0 tx=0x7f0e20009290).ready entity=mon.0 client_cookie=9bf748ca1fbd1590 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:17.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.078+0000 7f0e2c96b700 1 -- 172.21.15.17:0/2434503844 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e2000b870 con 0x7f0e281434b0 2022-08-01T19:42:17.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.078+0000 7f0e2c96b700 1 -- 172.21.15.17:0/2434503844 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2000b9d0 con 0x7f0e281434b0 2022-08-01T19:42:17.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.078+0000 7f0e2c96b700 1 -- 172.21.15.17:0/2434503844 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e2000bb40 con 0x7f0e281434b0 2022-08-01T19:42:17.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.082+0000 7f0e2e96f700 1 -- 172.21.15.17:0/2434503844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e281434b0 msgr2=0x7f0e28143940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:17.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.082+0000 7f0e2e96f700 1 --2- 172.21.15.17:0/2434503844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e281434b0 0x7f0e28143940 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 rx=0x7f0e200032f0 tx=0x7f0e20009290).stop 2022-08-01T19:42:17.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.082+0000 7f0e2e96f700 1 -- 172.21.15.17:0/2434503844 shutdown_connections 2022-08-01T19:42:17.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.082+0000 7f0e2e96f700 1 --2- 172.21.15.17:0/2434503844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e281434b0 0x7f0e28143940 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:17.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.082+0000 7f0e2e96f700 1 --2- 172.21.15.17:0/2434503844 >> v2:172.21.15.77:3300/0 conn(0x7f0e280a9da0 0x7f0e28060950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:17.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.082+0000 7f0e2e96f700 1 --2- 172.21.15.17:0/2434503844 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0e280b31d0 0x7f0e28066020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:17.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.082+0000 7f0e2e96f700 1 -- 172.21.15.17:0/2434503844 >> 172.21.15.17:0/2434503844 conn(0x7f0e28063870 msgr2=0x7f0e28063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:17.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.082+0000 7f0e2e96f700 1 -- 172.21.15.17:0/2434503844 shutdown_connections 2022-08-01T19:42:17.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.082+0000 7f0e2e96f700 1 -- 172.21.15.17:0/2434503844 wait complete. 2022-08-01T19:42:17.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.082+0000 7f0e2e96f700 1 Processor -- start 2022-08-01T19:42:17.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.082+0000 7f0e2e96f700 1 -- start start 2022-08-01T19:42:17.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.082+0000 7f0e2e96f700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0e280a9da0 0x7f0e281405d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:17.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.082+0000 7f0e2e96f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e280b31d0 0x7f0e28130530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:17.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.082+0000 7f0e2e96f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e281434b0 0x7f0e28130a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:17.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.082+0000 7f0e2e96f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e280ac120 con 0x7f0e280b31d0 2022-08-01T19:42:17.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.082+0000 7f0e2e96f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e280b1090 con 0x7f0e281434b0 2022-08-01T19:42:17.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.082+0000 7f0e2e96f700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e280b1330 con 0x7f0e280a9da0 2022-08-01T19:42:17.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.086+0000 7f0e2d16c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e280b31d0 0x7f0e28130530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:17.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.086+0000 7f0e2d16c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e280b31d0 0x7f0e28130530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43546/0 (socket says 172.21.15.17:43546) 2022-08-01T19:42:17.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.086+0000 7f0e2d16c700 1 -- 172.21.15.17:0/2945600480 learned_addr learned my addr 172.21.15.17:0/2945600480 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:17.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.086+0000 7f0e2d96d700 1 --2- 172.21.15.17:0/2945600480 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0e280a9da0 0x7f0e281405d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:17.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.086+0000 7f0e2e16e700 1 --2- 172.21.15.17:0/2945600480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e281434b0 0x7f0e28130a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:17.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.086+0000 7f0e2d16c700 1 -- 172.21.15.17:0/2945600480 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0e280a9da0 msgr2=0x7f0e281405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:17.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.086+0000 7f0e2d16c700 1 --2- 172.21.15.17:0/2945600480 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0e280a9da0 0x7f0e281405d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:17.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.086+0000 7f0e2d16c700 1 -- 172.21.15.17:0/2945600480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e281434b0 msgr2=0x7f0e28130a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:17.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.086+0000 7f0e2d16c700 1 --2- 172.21.15.17:0/2945600480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e281434b0 0x7f0e28130a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:17.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.086+0000 7f0e2d16c700 1 -- 172.21.15.17:0/2945600480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e20009530 con 0x7f0e280b31d0 2022-08-01T19:42:17.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.086+0000 7f0e2d96d700 1 --2- 172.21.15.17:0/2945600480 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0e280a9da0 0x7f0e281405d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:42:17.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.086+0000 7f0e2d16c700 1 --2- 172.21.15.17:0/2945600480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e280b31d0 0x7f0e28130530 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 rx=0x7f0e18006940 tx=0x7f0e18008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:17.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.086+0000 7f0e1effd700 1 -- 172.21.15.17:0/2945600480 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e18011780 con 0x7f0e280b31d0 2022-08-01T19:42:17.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.086+0000 7f0e1effd700 1 -- 172.21.15.17:0/2945600480 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e180118e0 con 0x7f0e280b31d0 2022-08-01T19:42:17.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.086+0000 7f0e2e96f700 1 -- 172.21.15.17:0/2945600480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e280b15d0 con 0x7f0e280b31d0 2022-08-01T19:42:17.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.086+0000 7f0e1effd700 1 -- 172.21.15.17:0/2945600480 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e18011c40 con 0x7f0e280b31d0 2022-08-01T19:42:17.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.086+0000 7f0e2e96f700 1 -- 172.21.15.17:0/2945600480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e28130f10 con 0x7f0e280b31d0 2022-08-01T19:42:17.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.086+0000 7f0e1effd700 1 -- 172.21.15.17:0/2945600480 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0e18012420 con 0x7f0e280b31d0 2022-08-01T19:42:17.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.086+0000 7f0e2e96f700 1 -- 172.21.15.17:0/2945600480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e28066af0 con 0x7f0e280b31d0 2022-08-01T19:42:17.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.086+0000 7f0e1effd700 1 --2- 172.21.15.17:0/2945600480 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0e10053a10 0x7f0e10055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:17.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.086+0000 7f0e1effd700 1 -- 172.21.15.17:0/2945600480 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0e1806c060 con 0x7f0e280b31d0 2022-08-01T19:42:17.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.086+0000 7f0e2d96d700 1 --2- 172.21.15.17:0/2945600480 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0e10053a10 0x7f0e10055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:17.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.090+0000 7f0e2d96d700 1 --2- 172.21.15.17:0/2945600480 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0e10053a10 0x7f0e10055ed0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7f0e2400c000 tx=0x7f0e24008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:17.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.094+0000 7f0e1effd700 1 -- 172.21.15.17:0/2945600480 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0e18042cb0 con 0x7f0e280b31d0 2022-08-01T19:42:17.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.222+0000 7f0e2e96f700 1 -- 172.21.15.17:0/2945600480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f0e280574e0 con 0x7f0e280b31d0 2022-08-01T19:42:17.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.222+0000 7f0e1effd700 1 -- 172.21.15.17:0/2945600480 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v117) v1 ==== 110+0+10 (secure 0 0 0) 0x7f0e180462d0 con 0x7f0e280b31d0 2022-08-01T19:42:17.230 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:42:17.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.226+0000 7f0e2e96f700 1 -- 172.21.15.17:0/2945600480 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0e10053a10 msgr2=0x7f0e10055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:17.234 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.226+0000 7f0e2e96f700 1 --2- 172.21.15.17:0/2945600480 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0e10053a10 0x7f0e10055ed0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7f0e2400c000 tx=0x7f0e24008040).stop 2022-08-01T19:42:17.234 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.226+0000 7f0e2e96f700 1 -- 172.21.15.17:0/2945600480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e280b31d0 msgr2=0x7f0e28130530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:17.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.226+0000 7f0e2e96f700 1 --2- 172.21.15.17:0/2945600480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e280b31d0 0x7f0e28130530 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 rx=0x7f0e18006940 tx=0x7f0e18008040).stop 2022-08-01T19:42:17.246 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.226+0000 7f0e2e96f700 1 -- 172.21.15.17:0/2945600480 shutdown_connections 2022-08-01T19:42:17.246 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.226+0000 7f0e2e96f700 1 --2- 172.21.15.17:0/2945600480 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0e10053a10 0x7f0e10055ed0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:17.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.226+0000 7f0e2e96f700 1 --2- 172.21.15.17:0/2945600480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e281434b0 0x7f0e28130a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:17.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.226+0000 7f0e2e96f700 1 --2- 172.21.15.17:0/2945600480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e280b31d0 0x7f0e28130530 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:17.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.226+0000 7f0e2e96f700 1 --2- 172.21.15.17:0/2945600480 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0e280a9da0 0x7f0e281405d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:17.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.226+0000 7f0e2e96f700 1 -- 172.21.15.17:0/2945600480 >> 172.21.15.17:0/2945600480 conn(0x7f0e28063870 msgr2=0x7f0e28142aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:17.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.226+0000 7f0e2e96f700 1 -- 172.21.15.17:0/2945600480 shutdown_connections 2022-08-01T19:42:17.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.226+0000 7f0e2e96f700 1 -- 172.21.15.17:0/2945600480 wait complete. 2022-08-01T19:42:17.253 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:42:17.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.362+0000 7f63107fa700 1 Processor -- start 2022-08-01T19:42:17.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.362+0000 7f63107fa700 1 -- start start 2022-08-01T19:42:17.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.362+0000 7f63107fa700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6308146ae0 0x7f63081403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:17.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.362+0000 7f63107fa700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6308143cf0 con 0x7f63080b4310 2022-08-01T19:42:17.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.362+0000 7f63107fa700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6308143f90 con 0x7f63080b4820 2022-08-01T19:42:17.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.362+0000 7f63107fa700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63081476a0 con 0x7f6308146ae0 2022-08-01T19:42:17.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.362+0000 7f630dd95700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f63080b4310 0x7f63081408b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35746/0 (socket says 172.21.15.17:35746) 2022-08-01T19:42:17.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.362+0000 7f630dd95700 1 -- 172.21.15.17:0/3180438883 learned_addr learned my addr 172.21.15.17:0/3180438883 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:17.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.362+0000 7f630e596700 1 --2- 172.21.15.17:0/3180438883 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6308146ae0 0x7f63081403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:17.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.362+0000 7f630d594700 1 -- 172.21.15.17:0/3180438883 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1657950123 0 0) 0x7f6308143cf0 con 0x7f63080b4310 2022-08-01T19:42:17.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.362+0000 7f630d594700 1 -- 172.21.15.17:0/3180438883 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62f0003260 con 0x7f63080b4310 2022-08-01T19:42:17.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f630d594700 1 -- 172.21.15.17:0/3180438883 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3188493358 0 0) 0x7f62f0003260 con 0x7f63080b4310 2022-08-01T19:42:17.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f630d594700 1 -- 172.21.15.17:0/3180438883 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6308146ae0 msgr2=0x7f63081403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:17.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f630d594700 1 --2- 172.21.15.17:0/3180438883 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6308146ae0 0x7f63081403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:17.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f630d594700 1 -- 172.21.15.17:0/3180438883 >> v1:172.21.15.77:6789/0 conn(0x7f63080b4820 legacy=0x7f63080b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:17.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f630d594700 1 -- 172.21.15.17:0/3180438883 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63080b0e20 con 0x7f63080b4310 2022-08-01T19:42:17.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f630d594700 1 -- 172.21.15.17:0/3180438883 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7f62fc001660 con 0x7f63080b4310 2022-08-01T19:42:17.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f630d594700 1 -- 172.21.15.17:0/3180438883 >> v1:172.21.15.17:6789/0 conn(0x7f63080b4310 legacy=0x7f63081408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:17.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f630d594700 1 --2- 172.21.15.17:0/3180438883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62f0003780 0x7f62f0003b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:17.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f630d594700 1 --2- 172.21.15.17:0/3180438883 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6308146ae0 0x7f62f0004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:17.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f630d594700 1 --2- 172.21.15.17:0/3180438883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62f00049f0 0x7f62f0004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:17.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f630d594700 1 -- 172.21.15.17:0/3180438883 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63081476a0 con 0x7f62f0003780 2022-08-01T19:42:17.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f630d594700 1 -- 172.21.15.17:0/3180438883 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f000b750 con 0x7f62f00049f0 2022-08-01T19:42:17.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f630d594700 1 -- 172.21.15.17:0/3180438883 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62f000b8c0 con 0x7f6308146ae0 2022-08-01T19:42:17.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f630e596700 1 --2- 172.21.15.17:0/3180438883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62f0003780 0x7f62f0003b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:17.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f630dd95700 1 --2- 172.21.15.17:0/3180438883 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6308146ae0 0x7f62f0004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:17.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f630ed97700 1 --2- 172.21.15.17:0/3180438883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62f00049f0 0x7f62f0004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:17.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f630e596700 1 -- 172.21.15.17:0/3180438883 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6308146ae0 msgr2=0x7f62f0004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:17.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f630e596700 1 --2- 172.21.15.17:0/3180438883 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6308146ae0 0x7f62f0004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:17.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f630e596700 1 -- 172.21.15.17:0/3180438883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62f00049f0 msgr2=0x7f62f0004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:17.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f630e596700 1 --2- 172.21.15.17:0/3180438883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62f00049f0 0x7f62f0004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:17.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f630e596700 1 -- 172.21.15.17:0/3180438883 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f0003260 con 0x7f62f0003780 2022-08-01T19:42:17.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f630ed97700 1 --2- 172.21.15.17:0/3180438883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62f00049f0 0x7f62f0004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:42:17.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f630e596700 1 --2- 172.21.15.17:0/3180438883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62f0003780 0x7f62f0003b60 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 rx=0x7f630400c000 tx=0x7f6304006df0).ready entity=mon.0 client_cookie=789d40435559e8a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:17.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f630d594700 1 -- 172.21.15.17:0/3180438883 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f630400e7d0 con 0x7f62f0003780 2022-08-01T19:42:17.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f630d594700 1 -- 172.21.15.17:0/3180438883 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630400e930 con 0x7f62f0003780 2022-08-01T19:42:17.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f630d594700 1 -- 172.21.15.17:0/3180438883 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f630400ec90 con 0x7f62f0003780 2022-08-01T19:42:17.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f63107fa700 1 -- 172.21.15.17:0/3180438883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62f0003780 msgr2=0x7f62f0003b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:17.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f63107fa700 1 --2- 172.21.15.17:0/3180438883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62f0003780 0x7f62f0003b60 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 rx=0x7f630400c000 tx=0x7f6304006df0).stop 2022-08-01T19:42:17.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f630ed97700 1 -- 172.21.15.17:0/3180438883 reap_dead start 2022-08-01T19:42:17.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f63107fa700 1 -- 172.21.15.17:0/3180438883 shutdown_connections 2022-08-01T19:42:17.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f63107fa700 1 -- 172.21.15.17:0/3180438883 >> 172.21.15.17:0/3180438883 conn(0x7f630805c620 msgr2=0x7f630805ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:17.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f63107fa700 1 -- 172.21.15.17:0/3180438883 shutdown_connections 2022-08-01T19:42:17.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f63107fa700 1 -- 172.21.15.17:0/3180438883 wait complete. 2022-08-01T19:42:17.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f63107fa700 1 Processor -- start 2022-08-01T19:42:17.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f63107fa700 1 -- start start 2022-08-01T19:42:17.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f63107fa700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62f0003780 0x7f6308148ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:17.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f63107fa700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63081522e0 0x7f63080addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:17.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f63107fa700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f63080ae2a0 0x7f63080afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:17.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f63107fa700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630813f030 con 0x7f63081522e0 2022-08-01T19:42:17.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f63107fa700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63080ae720 con 0x7f62f0003780 2022-08-01T19:42:17.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f63107fa700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f630813f8d0 con 0x7f63080ae2a0 2022-08-01T19:42:17.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.366+0000 7f630dd95700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63081522e0 0x7f63080addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:17.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.370+0000 7f630dd95700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63081522e0 0x7f63080addb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43564/0 (socket says 172.21.15.17:43564) 2022-08-01T19:42:17.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.370+0000 7f630dd95700 1 -- 172.21.15.17:0/1763387804 learned_addr learned my addr 172.21.15.17:0/1763387804 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:17.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.370+0000 7f630ed97700 1 --2- 172.21.15.17:0/1763387804 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f63080ae2a0 0x7f63080afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:17.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.370+0000 7f630e596700 1 --2- 172.21.15.17:0/1763387804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62f0003780 0x7f6308148ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:17.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.370+0000 7f630dd95700 1 -- 172.21.15.17:0/1763387804 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f63080ae2a0 msgr2=0x7f63080afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:17.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.370+0000 7f630dd95700 1 --2- 172.21.15.17:0/1763387804 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f63080ae2a0 0x7f63080afb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:17.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.370+0000 7f630dd95700 1 -- 172.21.15.17:0/1763387804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62f0003780 msgr2=0x7f6308148ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:17.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.370+0000 7f630dd95700 1 --2- 172.21.15.17:0/1763387804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62f0003780 0x7f6308148ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:17.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.370+0000 7f630dd95700 1 -- 172.21.15.17:0/1763387804 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63080b8590 con 0x7f63081522e0 2022-08-01T19:42:17.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.370+0000 7f630dd95700 1 --2- 172.21.15.17:0/1763387804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63081522e0 0x7f63080addb0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 rx=0x7f63080ae860 tx=0x7f63081431d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:17.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.370+0000 7f62fb7fe700 1 -- 172.21.15.17:0/1763387804 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62fc010640 con 0x7f63081522e0 2022-08-01T19:42:17.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.370+0000 7f62fb7fe700 1 -- 172.21.15.17:0/1763387804 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62fc0107a0 con 0x7f63081522e0 2022-08-01T19:42:17.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.370+0000 7f63107fa700 1 -- 172.21.15.17:0/1763387804 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f630813e2b0 con 0x7f63081522e0 2022-08-01T19:42:17.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.370+0000 7f62fb7fe700 1 -- 172.21.15.17:0/1763387804 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62fc010b00 con 0x7f63081522e0 2022-08-01T19:42:17.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.370+0000 7f63107fa700 1 -- 172.21.15.17:0/1763387804 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f630813fb70 con 0x7f63081522e0 2022-08-01T19:42:17.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.370+0000 7f63107fa700 1 -- 172.21.15.17:0/1763387804 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f630813f300 con 0x7f63081522e0 2022-08-01T19:42:17.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.370+0000 7f62fb7fe700 1 -- 172.21.15.17:0/1763387804 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f62fc010c60 con 0x7f63081522e0 2022-08-01T19:42:17.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.374+0000 7f62fb7fe700 1 --2- 172.21.15.17:0/1763387804 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f62e80539e0 0x7f62e8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:17.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.374+0000 7f62fb7fe700 1 -- 172.21.15.17:0/1763387804 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f62fc06c4d0 con 0x7f63081522e0 2022-08-01T19:42:17.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.374+0000 7f630e596700 1 --2- 172.21.15.17:0/1763387804 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f62e80539e0 0x7f62e8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:17.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.374+0000 7f630e596700 1 --2- 172.21.15.17:0/1763387804 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f62e80539e0 0x7f62e8055ea0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f630401a000 tx=0x7f6304019000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:17.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.378+0000 7f62fb7fe700 1 -- 172.21.15.17:0/1763387804 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f62fc043230 con 0x7f63081522e0 2022-08-01T19:42:17.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.506+0000 7f63107fa700 1 -- 172.21.15.17:0/1763387804 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f63080574e0 con 0x7f63081522e0 2022-08-01T19:42:17.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.506+0000 7f62fb7fe700 1 -- 172.21.15.17:0/1763387804 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v117) v1 ==== 92+0+10 (secure 0 0 0) 0x7f62fc046850 con 0x7f63081522e0 2022-08-01T19:42:17.512 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:42:17.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.506+0000 7f63107fa700 1 -- 172.21.15.17:0/1763387804 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f62e80539e0 msgr2=0x7f62e8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:17.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.506+0000 7f63107fa700 1 --2- 172.21.15.17:0/1763387804 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f62e80539e0 0x7f62e8055ea0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f630401a000 tx=0x7f6304019000).stop 2022-08-01T19:42:17.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.506+0000 7f63107fa700 1 -- 172.21.15.17:0/1763387804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63081522e0 msgr2=0x7f63080addb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:17.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.506+0000 7f63107fa700 1 --2- 172.21.15.17:0/1763387804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63081522e0 0x7f63080addb0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 rx=0x7f63080ae860 tx=0x7f63081431d0).stop 2022-08-01T19:42:17.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.506+0000 7f63107fa700 1 -- 172.21.15.17:0/1763387804 shutdown_connections 2022-08-01T19:42:17.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.510+0000 7f63107fa700 1 --2- 172.21.15.17:0/1763387804 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f62e80539e0 0x7f62e8055ea0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:17.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.510+0000 7f63107fa700 1 --2- 172.21.15.17:0/1763387804 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f63080ae2a0 0x7f63080afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:17.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.510+0000 7f63107fa700 1 --2- 172.21.15.17:0/1763387804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63081522e0 0x7f63080addb0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:17.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.510+0000 7f63107fa700 1 --2- 172.21.15.17:0/1763387804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f62f0003780 0x7f6308148ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:17.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.510+0000 7f63107fa700 1 -- 172.21.15.17:0/1763387804 >> 172.21.15.17:0/1763387804 conn(0x7f630805c620 msgr2=0x7f63080b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:17.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.510+0000 7f63107fa700 1 -- 172.21.15.17:0/1763387804 shutdown_connections 2022-08-01T19:42:17.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.510+0000 7f63107fa700 1 -- 172.21.15.17:0/1763387804 wait complete. 2022-08-01T19:42:17.524 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T19:42:17.648 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.646+0000 7f1ab7fff700 1 Processor -- start 2022-08-01T19:42:17.648 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.646+0000 7f1ab7fff700 1 -- start start 2022-08-01T19:42:17.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.646+0000 7f1ab7fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1ab80b3190 0x7f1ab8060ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:17.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.646+0000 7f1ab7fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1ab80b2730 0x7f1ab8061190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:17.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.646+0000 7f1ab7fff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f1ab80666b0 0x7f1ab8066b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:17.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.646+0000 7f1ab7fff700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ab8062190 con 0x7f1ab80b2730 2022-08-01T19:42:17.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.646+0000 7f1ab7fff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ab80654c0 con 0x7f1ab80666b0 2022-08-01T19:42:17.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.646+0000 7f1ab7fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab80b3570 con 0x7f1ab80b3190 2022-08-01T19:42:17.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.650+0000 7f1aaffff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1ab80b2730 0x7f1ab8061190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:17.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.650+0000 7f1aaffff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1ab80b2730 0x7f1ab8061190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43572/0 (socket says 172.21.15.17:43572) 2022-08-01T19:42:17.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.650+0000 7f1aaffff700 1 -- 172.21.15.17:0/2611768806 learned_addr learned my addr 172.21.15.17:0/2611768806 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:17.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.650+0000 7f1ab6ffd700 1 --2- 172.21.15.17:0/2611768806 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1ab80b3190 0x7f1ab8060ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:17.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.650+0000 7f1ab77fe700 1 --2- 172.21.15.17:0/2611768806 >> v2:172.21.15.77:3300/0 conn(0x7f1ab80666b0 0x7f1ab8066b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:17.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.650+0000 7f1aaffff700 1 -- 172.21.15.17:0/2611768806 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1ab80b3190 msgr2=0x7f1ab8060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:17.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.650+0000 7f1aaffff700 1 --2- 172.21.15.17:0/2611768806 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1ab80b3190 0x7f1ab8060ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:17.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.650+0000 7f1aaffff700 1 -- 172.21.15.17:0/2611768806 >> v2:172.21.15.77:3300/0 conn(0x7f1ab80666b0 msgr2=0x7f1ab8066b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:17.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.650+0000 7f1aaffff700 1 --2- 172.21.15.17:0/2611768806 >> v2:172.21.15.77:3300/0 conn(0x7f1ab80666b0 0x7f1ab8066b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:17.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.650+0000 7f1aaffff700 1 -- 172.21.15.17:0/2611768806 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab805fbf0 con 0x7f1ab80b2730 2022-08-01T19:42:17.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.650+0000 7f1aaffff700 1 --2- 172.21.15.17:0/2611768806 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ab80b2730 0x7f1ab8061190 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 rx=0x7f1aa0002f70 tx=0x7f1aa0006eb0).ready entity=mon.0 client_cookie=770032487fcb64c4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:17.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.650+0000 7f1ab67fc700 1 -- 172.21.15.17:0/2611768806 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1aa000d570 con 0x7f1ab80b2730 2022-08-01T19:42:17.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.650+0000 7f1ab67fc700 1 -- 172.21.15.17:0/2611768806 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa0000e90 con 0x7f1ab80b2730 2022-08-01T19:42:17.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.650+0000 7f1ab67fc700 1 -- 172.21.15.17:0/2611768806 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1aa000d900 con 0x7f1ab80b2730 2022-08-01T19:42:17.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.650+0000 7f1ab7fff700 1 -- 172.21.15.17:0/2611768806 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ab80b2730 msgr2=0x7f1ab8061190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:17.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.650+0000 7f1ab7fff700 1 --2- 172.21.15.17:0/2611768806 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ab80b2730 0x7f1ab8061190 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 rx=0x7f1aa0002f70 tx=0x7f1aa0006eb0).stop 2022-08-01T19:42:17.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.650+0000 7f1ab7fff700 1 -- 172.21.15.17:0/2611768806 shutdown_connections 2022-08-01T19:42:17.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.650+0000 7f1ab7fff700 1 --2- 172.21.15.17:0/2611768806 >> v2:172.21.15.77:3300/0 conn(0x7f1ab80666b0 0x7f1ab8066b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:17.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.650+0000 7f1ab7fff700 1 --2- 172.21.15.17:0/2611768806 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ab80b2730 0x7f1ab8061190 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:17.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.650+0000 7f1ab7fff700 1 --2- 172.21.15.17:0/2611768806 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1ab80b3190 0x7f1ab8060ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:17.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.650+0000 7f1ab7fff700 1 -- 172.21.15.17:0/2611768806 >> 172.21.15.17:0/2611768806 conn(0x7f1ab8063870 msgr2=0x7f1ab8063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:17.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.650+0000 7f1ab7fff700 1 -- 172.21.15.17:0/2611768806 shutdown_connections 2022-08-01T19:42:17.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.650+0000 7f1ab7fff700 1 -- 172.21.15.17:0/2611768806 wait complete. 2022-08-01T19:42:17.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.654+0000 7f1ab7fff700 1 Processor -- start 2022-08-01T19:42:17.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.654+0000 7f1ab7fff700 1 -- start start 2022-08-01T19:42:17.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.654+0000 7f1ab7fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1ab80666b0 0x7f1ab81457b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:17.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.654+0000 7f1ab7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ab80b2730 0x7f1ab8155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:17.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.654+0000 7f1ab7fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ab80b3190 0x7f1ab8156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:17.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.654+0000 7f1ab7fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab8156580 con 0x7f1ab80b2730 2022-08-01T19:42:17.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.654+0000 7f1ab7fff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab80ac7c0 con 0x7f1ab80b3190 2022-08-01T19:42:17.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.654+0000 7f1ab7fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab80aca60 con 0x7f1ab80666b0 2022-08-01T19:42:17.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.654+0000 7f1aaffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ab80b2730 0x7f1ab8155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:17.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.654+0000 7f1aaffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ab80b2730 0x7f1ab8155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43578/0 (socket says 172.21.15.17:43578) 2022-08-01T19:42:17.672 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.654+0000 7f1aaffff700 1 -- 172.21.15.17:0/3485227015 learned_addr learned my addr 172.21.15.17:0/3485227015 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:17.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.654+0000 7f1ab6ffd700 1 --2- 172.21.15.17:0/3485227015 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1ab80666b0 0x7f1ab81457b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:17.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.654+0000 7f1ab77fe700 1 --2- 172.21.15.17:0/3485227015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ab80b3190 0x7f1ab8156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:17.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.658+0000 7f1aaffff700 1 -- 172.21.15.17:0/3485227015 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1ab80666b0 msgr2=0x7f1ab81457b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:17.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.658+0000 7f1aaffff700 1 --2- 172.21.15.17:0/3485227015 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1ab80666b0 0x7f1ab81457b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:17.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.658+0000 7f1aaffff700 1 -- 172.21.15.17:0/3485227015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ab80b3190 msgr2=0x7f1ab8156090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:17.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.658+0000 7f1aaffff700 1 --2- 172.21.15.17:0/3485227015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ab80b3190 0x7f1ab8156090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:17.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.658+0000 7f1aaffff700 1 -- 172.21.15.17:0/3485227015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa000b1e0 con 0x7f1ab80b2730 2022-08-01T19:42:17.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.658+0000 7f1ab77fe700 1 --2- 172.21.15.17:0/3485227015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ab80b3190 0x7f1ab8156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:42:17.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.658+0000 7f1aaffff700 1 --2- 172.21.15.17:0/3485227015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ab80b2730 0x7f1ab8155ba0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 rx=0x7f1aa0002f70 tx=0x7f1aa0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:17.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.658+0000 7f1ab4ff9700 1 -- 172.21.15.17:0/3485227015 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1aa0011860 con 0x7f1ab80b2730 2022-08-01T19:42:17.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.658+0000 7f1ab4ff9700 1 -- 172.21.15.17:0/3485227015 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa00119c0 con 0x7f1ab80b2730 2022-08-01T19:42:17.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.658+0000 7f1ab7fff700 1 -- 172.21.15.17:0/3485227015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ab80acd00 con 0x7f1ab80b2730 2022-08-01T19:42:17.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.658+0000 7f1ab4ff9700 1 -- 172.21.15.17:0/3485227015 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1aa0011d20 con 0x7f1ab80b2730 2022-08-01T19:42:17.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.658+0000 7f1ab7fff700 1 -- 172.21.15.17:0/3485227015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ab8152170 con 0x7f1ab80b2730 2022-08-01T19:42:17.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.658+0000 7f1ab7fff700 1 -- 172.21.15.17:0/3485227015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ab805fe90 con 0x7f1ab80b2730 2022-08-01T19:42:17.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.658+0000 7f1ab4ff9700 1 -- 172.21.15.17:0/3485227015 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1aa0025030 con 0x7f1ab80b2730 2022-08-01T19:42:17.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.658+0000 7f1ab4ff9700 1 --2- 172.21.15.17:0/3485227015 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1a900539e0 0x7f1a90055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:17.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.658+0000 7f1ab4ff9700 1 -- 172.21.15.17:0/3485227015 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1aa006d1c0 con 0x7f1ab80b2730 2022-08-01T19:42:17.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.658+0000 7f1ab6ffd700 1 --2- 172.21.15.17:0/3485227015 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1a900539e0 0x7f1a90055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:17.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.662+0000 7f1ab6ffd700 1 --2- 172.21.15.17:0/3485227015 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1a900539e0 0x7f1a90055ea0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f1aa800c000 tx=0x7f1aa8008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:17.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.666+0000 7f1ab4ff9700 1 -- 172.21.15.17:0/3485227015 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1aa0043f80 con 0x7f1ab80b2730 2022-08-01T19:42:17.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.798+0000 7f1ab7fff700 1 -- 172.21.15.17:0/3485227015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1ab80574e0 con 0x7f1ab80b2730 2022-08-01T19:42:17.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.798+0000 7f1ab4ff9700 1 -- 172.21.15.17:0/3485227015 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v117) v1 ==== 104+0+10 (secure 0 0 0) 0x7f1aa00230a0 con 0x7f1ab80b2730 2022-08-01T19:42:17.800 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:42:17.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.802+0000 7f1ab7fff700 1 -- 172.21.15.17:0/3485227015 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1a900539e0 msgr2=0x7f1a90055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:17.804 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.802+0000 7f1ab7fff700 1 --2- 172.21.15.17:0/3485227015 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1a900539e0 0x7f1a90055ea0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f1aa800c000 tx=0x7f1aa8008040).stop 2022-08-01T19:42:17.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.802+0000 7f1ab7fff700 1 -- 172.21.15.17:0/3485227015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ab80b2730 msgr2=0x7f1ab8155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:17.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.802+0000 7f1ab7fff700 1 --2- 172.21.15.17:0/3485227015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ab80b2730 0x7f1ab8155ba0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 rx=0x7f1aa0002f70 tx=0x7f1aa0015000).stop 2022-08-01T19:42:17.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.802+0000 7f1ab7fff700 1 -- 172.21.15.17:0/3485227015 shutdown_connections 2022-08-01T19:42:17.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.802+0000 7f1ab7fff700 1 --2- 172.21.15.17:0/3485227015 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1a900539e0 0x7f1a90055ea0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:17.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.802+0000 7f1ab7fff700 1 --2- 172.21.15.17:0/3485227015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ab80b3190 0x7f1ab8156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:17.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.802+0000 7f1ab7fff700 1 --2- 172.21.15.17:0/3485227015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ab80b2730 0x7f1ab8155ba0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:17.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.802+0000 7f1ab7fff700 1 --2- 172.21.15.17:0/3485227015 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1ab80666b0 0x7f1ab81457b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:17.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.802+0000 7f1ab7fff700 1 -- 172.21.15.17:0/3485227015 >> 172.21.15.17:0/3485227015 conn(0x7f1ab8063870 msgr2=0x7f1ab81465d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:17.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.802+0000 7f1ab7fff700 1 -- 172.21.15.17:0/3485227015 shutdown_connections 2022-08-01T19:42:17.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.802+0000 7f1ab7fff700 1 -- 172.21.15.17:0/3485227015 wait complete. 2022-08-01T19:42:17.840 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T19:42:17.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.954+0000 7ff64d6f1700 1 Processor -- start 2022-08-01T19:42:17.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.954+0000 7ff64d6f1700 1 -- start start 2022-08-01T19:42:17.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.958+0000 7ff64d6f1700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff6480b2710 0x7ff6480abca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:17.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.958+0000 7ff64d6f1700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6480b13f0 0x7ff6480ac190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:17.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.958+0000 7ff64d6f1700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6480adf10 con 0x7ff6480b1920 2022-08-01T19:42:17.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.958+0000 7ff64d6f1700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6480ae1b0 con 0x7ff6480b2710 2022-08-01T19:42:17.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.958+0000 7ff64d6f1700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6480a9520 con 0x7ff6480b13f0 2022-08-01T19:42:18.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.958+0000 7ff647fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff6480b2710 0x7ff6480abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:18.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.958+0000 7ff647fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff6480b2710 0x7ff6480abca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43584/0 (socket says 172.21.15.17:43584) 2022-08-01T19:42:18.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.958+0000 7ff63ffff700 1 --2- 172.21.15.17:0/1362560909 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6480b13f0 0x7ff6480ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:18.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.958+0000 7ff647fff700 1 -- 172.21.15.17:0/1362560909 learned_addr learned my addr 172.21.15.17:0/1362560909 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:18.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.958+0000 7ff647fff700 1 -- 172.21.15.17:0/1362560909 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6480b13f0 msgr2=0x7ff6480ac190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:18.002 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.958+0000 7ff6477fe700 1 -- 172.21.15.17:0/1362560909 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 447034428 0 0) 0x7ff6480adf10 con 0x7ff6480b1920 2022-08-01T19:42:18.002 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.958+0000 7ff647fff700 1 --2- 172.21.15.17:0/1362560909 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6480b13f0 0x7ff6480ac190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:18.003 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.958+0000 7ff647fff700 1 -- 172.21.15.17:0/1362560909 >> v1:172.21.15.77:6789/0 conn(0x7ff6480b1920 legacy=0x7ff6480afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:18.003 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.958+0000 7ff647fff700 1 -- 172.21.15.17:0/1362560909 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6480a99c0 con 0x7ff6480b2710 2022-08-01T19:42:18.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.958+0000 7ff647fff700 1 --2- 172.21.15.17:0/1362560909 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6480b2710 0x7ff6480abca0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 rx=0x7ff648067bc0 tx=0x7ff638006eb0).ready entity=mon.0 client_cookie=1bcfe9406e75cc53 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:18.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.958+0000 7ff6477fe700 1 -- 172.21.15.17:0/1362560909 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff63800d5c0 con 0x7ff6480b2710 2022-08-01T19:42:18.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.958+0000 7ff6477fe700 1 -- 172.21.15.17:0/1362560909 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff638000e90 con 0x7ff6480b2710 2022-08-01T19:42:18.007 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.958+0000 7ff6477fe700 1 -- 172.21.15.17:0/1362560909 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff63800d910 con 0x7ff6480b2710 2022-08-01T19:42:18.007 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.962+0000 7ff64d6f1700 1 -- 172.21.15.17:0/1362560909 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6480b2710 msgr2=0x7ff6480abca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:18.008 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.962+0000 7ff64d6f1700 1 --2- 172.21.15.17:0/1362560909 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6480b2710 0x7ff6480abca0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 rx=0x7ff648067bc0 tx=0x7ff638006eb0).stop 2022-08-01T19:42:18.008 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.962+0000 7ff64d6f1700 1 -- 172.21.15.17:0/1362560909 shutdown_connections 2022-08-01T19:42:18.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.962+0000 7ff64d6f1700 1 --2- 172.21.15.17:0/1362560909 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6480b13f0 0x7ff6480ac190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:18.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.962+0000 7ff64d6f1700 1 --2- 172.21.15.17:0/1362560909 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6480b2710 0x7ff6480abca0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:18.010 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.962+0000 7ff64d6f1700 1 -- 172.21.15.17:0/1362560909 >> 172.21.15.17:0/1362560909 conn(0x7ff648060310 msgr2=0x7ff6480b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:18.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.962+0000 7ff64d6f1700 1 -- 172.21.15.17:0/1362560909 shutdown_connections 2022-08-01T19:42:18.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.962+0000 7ff64d6f1700 1 -- 172.21.15.17:0/1362560909 wait complete. 2022-08-01T19:42:18.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.962+0000 7ff64d6f1700 1 Processor -- start 2022-08-01T19:42:18.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.962+0000 7ff64d6f1700 1 -- start start 2022-08-01T19:42:18.013 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.966+0000 7ff64d6f1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6480b13f0 0x7ff6480a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:18.013 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.966+0000 7ff64d6f1700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6480b1920 0x7ff64809f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:18.014 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.966+0000 7ff64d6f1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6480b2710 0x7ff64809f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:18.014 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.966+0000 7ff64d6f1700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff648067500 con 0x7ff6480b2710 2022-08-01T19:42:18.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.966+0000 7ff64d6f1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff648066c70 con 0x7ff6480b13f0 2022-08-01T19:42:18.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.966+0000 7ff64d6f1700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff648146500 con 0x7ff6480b1920 2022-08-01T19:42:18.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.966+0000 7ff64cef0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6480b2710 0x7ff64809f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:18.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.966+0000 7ff63ffff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6480b1920 0x7ff64809f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:18.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.966+0000 7ff647fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6480b13f0 0x7ff6480a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:18.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.966+0000 7ff64cef0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6480b2710 0x7ff64809f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43594/0 (socket says 172.21.15.17:43594) 2022-08-01T19:42:18.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.966+0000 7ff647fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6480b13f0 0x7ff6480a1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:41512/0 (socket says 172.21.15.17:41512) 2022-08-01T19:42:18.018 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.966+0000 7ff64cef0700 1 -- 172.21.15.17:0/2837303750 learned_addr learned my addr 172.21.15.17:0/2837303750 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:18.018 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.966+0000 7ff64cef0700 1 -- 172.21.15.17:0/2837303750 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6480b1920 msgr2=0x7ff64809f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:18.018 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.966+0000 7ff64cef0700 1 --2- 172.21.15.17:0/2837303750 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6480b1920 0x7ff64809f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:18.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.966+0000 7ff64cef0700 1 -- 172.21.15.17:0/2837303750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6480b13f0 msgr2=0x7ff6480a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:18.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.966+0000 7ff64cef0700 1 --2- 172.21.15.17:0/2837303750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6480b13f0 0x7ff6480a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:18.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.966+0000 7ff64cef0700 1 -- 172.21.15.17:0/2837303750 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff63800b1e0 con 0x7ff6480b2710 2022-08-01T19:42:18.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.966+0000 7ff647fff700 1 --2- 172.21.15.17:0/2837303750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6480b13f0 0x7ff6480a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:42:18.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.966+0000 7ff64cef0700 1 --2- 172.21.15.17:0/2837303750 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6480b2710 0x7ff64809f610 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 rx=0x7ff640001250 tx=0x7ff640007d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:18.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.970+0000 7ff645ffb700 1 -- 172.21.15.17:0/2837303750 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff64000e4a0 con 0x7ff6480b2710 2022-08-01T19:42:18.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.970+0000 7ff645ffb700 1 -- 172.21.15.17:0/2837303750 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff64000e600 con 0x7ff6480b2710 2022-08-01T19:42:18.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.970+0000 7ff64d6f1700 1 -- 172.21.15.17:0/2837303750 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff64814e9d0 con 0x7ff6480b2710 2022-08-01T19:42:18.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.970+0000 7ff645ffb700 1 -- 172.21.15.17:0/2837303750 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff64000e960 con 0x7ff6480b2710 2022-08-01T19:42:18.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.970+0000 7ff64d6f1700 1 -- 172.21.15.17:0/2837303750 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff648066960 con 0x7ff6480b2710 2022-08-01T19:42:18.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.970+0000 7ff64d6f1700 1 -- 172.21.15.17:0/2837303750 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff64814eb90 con 0x7ff6480b2710 2022-08-01T19:42:18.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.970+0000 7ff645ffb700 1 -- 172.21.15.17:0/2837303750 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff640019030 con 0x7ff6480b2710 2022-08-01T19:42:18.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.974+0000 7ff645ffb700 1 --2- 172.21.15.17:0/2837303750 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff620053a70 0x7ff620055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:18.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.974+0000 7ff645ffb700 1 -- 172.21.15.17:0/2837303750 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff64006b790 con 0x7ff6480b2710 2022-08-01T19:42:18.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.974+0000 7ff647fff700 1 --2- 172.21.15.17:0/2837303750 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff620053a70 0x7ff620055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:18.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.974+0000 7ff647fff700 1 --2- 172.21.15.17:0/2837303750 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff620053a70 0x7ff620055f30 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7ff63801c000 tx=0x7ff638015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:18.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:17.982+0000 7ff645ffb700 1 -- 172.21.15.17:0/2837303750 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff64814eb90 con 0x7ff6480b2710 2022-08-01T19:42:18.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.106+0000 7ff64d6f1700 1 -- 172.21.15.17:0/2837303750 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff64814eb60 con 0x7ff6480b2710 2022-08-01T19:42:18.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.106+0000 7ff645ffb700 1 -- 172.21.15.17:0/2837303750 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v117) v1 ==== 100+0+10 (secure 0 0 0) 0x7ff640042550 con 0x7ff6480b2710 2022-08-01T19:42:18.110 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:42:18.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.110+0000 7ff64d6f1700 1 -- 172.21.15.17:0/2837303750 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff620053a70 msgr2=0x7ff620055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:18.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.110+0000 7ff64d6f1700 1 --2- 172.21.15.17:0/2837303750 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff620053a70 0x7ff620055f30 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7ff63801c000 tx=0x7ff638015000).stop 2022-08-01T19:42:18.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.110+0000 7ff64d6f1700 1 -- 172.21.15.17:0/2837303750 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6480b2710 msgr2=0x7ff64809f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:18.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.110+0000 7ff64d6f1700 1 --2- 172.21.15.17:0/2837303750 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6480b2710 0x7ff64809f610 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 rx=0x7ff640001250 tx=0x7ff640007d90).stop 2022-08-01T19:42:18.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.110+0000 7ff64d6f1700 1 -- 172.21.15.17:0/2837303750 shutdown_connections 2022-08-01T19:42:18.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.110+0000 7ff64d6f1700 1 --2- 172.21.15.17:0/2837303750 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff620053a70 0x7ff620055f30 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:18.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.110+0000 7ff64d6f1700 1 --2- 172.21.15.17:0/2837303750 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6480b2710 0x7ff64809f610 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.110+0000 7ff64d6f1700 1 --2- 172.21.15.17:0/2837303750 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6480b1920 0x7ff64809f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.110+0000 7ff64d6f1700 1 --2- 172.21.15.17:0/2837303750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6480b13f0 0x7ff6480a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:18.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.110+0000 7ff64d6f1700 1 -- 172.21.15.17:0/2837303750 >> 172.21.15.17:0/2837303750 conn(0x7ff648060310 msgr2=0x7ff6480b4080 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:18.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.110+0000 7ff64d6f1700 1 -- 172.21.15.17:0/2837303750 shutdown_connections 2022-08-01T19:42:18.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.110+0000 7ff64d6f1700 1 -- 172.21.15.17:0/2837303750 wait complete. 2022-08-01T19:42:18.124 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T19:42:18.125 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:42:18.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.238+0000 7fce7ebdc700 1 Processor -- start 2022-08-01T19:42:18.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.238+0000 7fce7ebdc700 1 -- start start 2022-08-01T19:42:18.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.238+0000 7fce7ebdc700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fce780b4550 0x7fce780b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:18.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.238+0000 7fce7ebdc700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fce780b6af0 0x7fce7812bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:18.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.238+0000 7fce7ebdc700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce78144240 con 0x7fce780b6710 2022-08-01T19:42:18.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.238+0000 7fce7ebdc700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce78143d00 con 0x7fce780b4550 2022-08-01T19:42:18.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.238+0000 7fce7ebdc700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce78143fa0 con 0x7fce780b6af0 2022-08-01T19:42:18.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.238+0000 7fce7c978700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fce780b6710 0x7fce78061ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35792/0 (socket says 172.21.15.17:35792) 2022-08-01T19:42:18.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.238+0000 7fce7c978700 1 -- 172.21.15.17:0/2061656331 learned_addr learned my addr 172.21.15.17:0/2061656331 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:18.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.238+0000 7fce77fff700 1 --2- 172.21.15.17:0/2061656331 >> v2:172.21.15.17:3300/0 conn(0x7fce780b4550 0x7fce780b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:18.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.238+0000 7fce777fe700 1 -- 172.21.15.17:0/2061656331 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1542783851 0 0) 0x7fce78144240 con 0x7fce780b6710 2022-08-01T19:42:18.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.238+0000 7fce777fe700 1 -- 172.21.15.17:0/2061656331 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce58003260 con 0x7fce780b6710 2022-08-01T19:42:18.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.238+0000 7fce7d179700 1 --2- 172.21.15.17:0/2061656331 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fce780b6af0 0x7fce7812bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:18.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.238+0000 7fce77fff700 1 -- 172.21.15.17:0/2061656331 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fce780b6af0 msgr2=0x7fce7812bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:18.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.238+0000 7fce77fff700 1 --2- 172.21.15.17:0/2061656331 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fce780b6af0 0x7fce7812bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:18.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.238+0000 7fce777fe700 1 -- 172.21.15.17:0/2061656331 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 318828371 0 0) 0x7fce58003260 con 0x7fce780b6710 2022-08-01T19:42:18.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.238+0000 7fce77fff700 1 -- 172.21.15.17:0/2061656331 >> v1:172.21.15.17:6789/0 conn(0x7fce780b6710 legacy=0x7fce78061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:18.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.238+0000 7fce77fff700 1 -- 172.21.15.17:0/2061656331 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce78061780 con 0x7fce780b4550 2022-08-01T19:42:18.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.238+0000 7fce77fff700 1 --2- 172.21.15.17:0/2061656331 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce780b4550 0x7fce780b5d80 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 rx=0x7fce70001b80 tx=0x7fce60006df0).ready entity=mon.0 client_cookie=698c587c53dabc5a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:18.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.238+0000 7fce777fe700 1 -- 172.21.15.17:0/2061656331 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce700019e0 con 0x7fce780b4550 2022-08-01T19:42:18.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.238+0000 7fce777fe700 1 -- 172.21.15.17:0/2061656331 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce60000e90 con 0x7fce780b4550 2022-08-01T19:42:18.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.238+0000 7fce777fe700 1 -- 172.21.15.17:0/2061656331 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce6000d4a0 con 0x7fce780b4550 2022-08-01T19:42:18.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.242+0000 7fce7ebdc700 1 -- 172.21.15.17:0/2061656331 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce780b4550 msgr2=0x7fce780b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:18.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.242+0000 7fce7ebdc700 1 --2- 172.21.15.17:0/2061656331 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce780b4550 0x7fce780b5d80 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 rx=0x7fce70001b80 tx=0x7fce60006df0).stop 2022-08-01T19:42:18.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.242+0000 7fce7ebdc700 1 -- 172.21.15.17:0/2061656331 shutdown_connections 2022-08-01T19:42:18.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.242+0000 7fce7ebdc700 1 --2- 172.21.15.17:0/2061656331 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fce780b6af0 0x7fce7812bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:18.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.242+0000 7fce7ebdc700 1 --2- 172.21.15.17:0/2061656331 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce780b4550 0x7fce780b5d80 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:18.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.242+0000 7fce7ebdc700 1 -- 172.21.15.17:0/2061656331 >> 172.21.15.17:0/2061656331 conn(0x7fce7805c620 msgr2=0x7fce7805ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:18.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.242+0000 7fce7ebdc700 1 -- 172.21.15.17:0/2061656331 shutdown_connections 2022-08-01T19:42:18.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.242+0000 7fce7ebdc700 1 -- 172.21.15.17:0/2061656331 wait complete. 2022-08-01T19:42:18.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.242+0000 7fce7ebdc700 1 Processor -- start 2022-08-01T19:42:18.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.242+0000 7fce7ebdc700 1 -- start start 2022-08-01T19:42:18.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.242+0000 7fce7ebdc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce780b4550 0x7fce780b81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:18.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.242+0000 7fce7ebdc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce780b6710 0x7fce78155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:18.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.242+0000 7fce7ebdc700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fce780b6af0 0x7fce78156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:18.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.242+0000 7fce7ebdc700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce780b8800 con 0x7fce780b6710 2022-08-01T19:42:18.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.242+0000 7fce7ebdc700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce780b7cd0 con 0x7fce780b4550 2022-08-01T19:42:18.272 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.242+0000 7fce7ebdc700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce780b20b0 con 0x7fce780b6af0 2022-08-01T19:42:18.272 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.242+0000 7fce77fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce780b6710 0x7fce78155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:18.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.242+0000 7fce7d179700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fce780b6af0 0x7fce78156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:18.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.242+0000 7fce7d179700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fce780b6af0 0x7fce78156210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:39706/0 (socket says 172.21.15.17:39706) 2022-08-01T19:42:18.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.242+0000 7fce7d179700 1 -- 172.21.15.17:0/777124983 learned_addr learned my addr 172.21.15.17:0/777124983 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:18.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.242+0000 7fce7c978700 1 --2- 172.21.15.17:0/777124983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce780b4550 0x7fce780b81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:18.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.246+0000 7fce7d179700 1 -- 172.21.15.17:0/777124983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce780b4550 msgr2=0x7fce780b81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:18.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.246+0000 7fce7d179700 1 --2- 172.21.15.17:0/777124983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce780b4550 0x7fce780b81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:18.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.246+0000 7fce7d179700 1 -- 172.21.15.17:0/777124983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce780b6710 msgr2=0x7fce78155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:18.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.246+0000 7fce7d179700 1 --2- 172.21.15.17:0/777124983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce780b6710 0x7fce78155d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:18.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.246+0000 7fce7d179700 1 -- 172.21.15.17:0/777124983 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6000b180 con 0x7fce780b6af0 2022-08-01T19:42:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.246+0000 7fce77fff700 1 --2- 172.21.15.17:0/777124983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce780b6710 0x7fce78155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:42:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.246+0000 7fce7d179700 1 --2- 172.21.15.17:0/777124983 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fce780b6af0 0x7fce78156210 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fce6800a000 tx=0x7fce68008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.246+0000 7fce75ffb700 1 -- 172.21.15.17:0/777124983 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce68011780 con 0x7fce780b6af0 2022-08-01T19:42:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.246+0000 7fce75ffb700 1 -- 172.21.15.17:0/777124983 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce680118e0 con 0x7fce780b6af0 2022-08-01T19:42:18.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.246+0000 7fce7ebdc700 1 -- 172.21.15.17:0/777124983 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce780b8a30 con 0x7fce780b6af0 2022-08-01T19:42:18.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.246+0000 7fce75ffb700 1 -- 172.21.15.17:0/777124983 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce68011c40 con 0x7fce780b6af0 2022-08-01T19:42:18.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.246+0000 7fce7ebdc700 1 -- 172.21.15.17:0/777124983 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce780b3d20 con 0x7fce780b6af0 2022-08-01T19:42:18.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.246+0000 7fce7ebdc700 1 -- 172.21.15.17:0/777124983 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce78061a20 con 0x7fce780b6af0 2022-08-01T19:42:18.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.246+0000 7fce75ffb700 1 -- 172.21.15.17:0/777124983 <== mon.2 v2:172.21.15.77:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fce6801c460 con 0x7fce780b6af0 2022-08-01T19:42:18.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.246+0000 7fce75ffb700 1 --2- 172.21.15.17:0/777124983 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fce5c053b30 0x7fce5c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:18.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.246+0000 7fce75ffb700 1 -- 172.21.15.17:0/777124983 <== mon.2 v2:172.21.15.77:3301/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fce6806c0c0 con 0x7fce780b6af0 2022-08-01T19:42:18.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.246+0000 7fce7c978700 1 --2- 172.21.15.17:0/777124983 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fce5c053b30 0x7fce5c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:18.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.250+0000 7fce7c978700 1 --2- 172.21.15.17:0/777124983 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fce5c053b30 0x7fce5c055ff0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7fce7000d000 tx=0x7fce700021c0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:18.281 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:42:18.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.258+0000 7fce75ffb700 1 -- 172.21.15.17:0/777124983 <== mon.2 v2:172.21.15.77:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fce68042e20 con 0x7fce780b6af0 2022-08-01T19:42:18.285 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:42:18.288 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:42:18.312 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:42:18.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.418+0000 7fce7ebdc700 1 -- 172.21.15.17:0/777124983 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce780b7e10 con 0x7fce780b6af0 2022-08-01T19:42:18.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.418+0000 7fce75ffb700 1 -- 172.21.15.17:0/777124983 <== mon.2 v2:172.21.15.77:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+2492 (secure 0 0 0) 0x7fce68046440 con 0x7fce780b6af0 2022-08-01T19:42:18.422 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:42:18.423 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":58,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7581,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/1744640648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":1744640648},{"type":"v1","addr":"172.21.15.17:6833","nonce":1744640648}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":56},{"gid":7594,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.17:6835/346594655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":346594655},{"type":"v1","addr":"172.21.15.17:6835","nonce":346594655}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":57}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:42:10.906134+0000","modified":"2022-08-01T19:42:11.922099+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7600},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7600":{"gid":7600,"name":"b","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6835/2286085186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2286085186},{"type":"v1","addr":"172.21.15.77:6835","nonce":2286085186}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2022-08-01T19:42:18.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.422+0000 7fce7ebdc700 1 -- 172.21.15.17:0/777124983 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fce5c053b30 msgr2=0x7fce5c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:18.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.422+0000 7fce7ebdc700 1 --2- 172.21.15.17:0/777124983 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fce5c053b30 0x7fce5c055ff0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7fce7000d000 tx=0x7fce700021c0).stop 2022-08-01T19:42:18.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.422+0000 7fce7ebdc700 1 -- 172.21.15.17:0/777124983 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fce780b6af0 msgr2=0x7fce78156210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:18.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.422+0000 7fce7ebdc700 1 --2- 172.21.15.17:0/777124983 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fce780b6af0 0x7fce78156210 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fce6800a000 tx=0x7fce68008040).stop 2022-08-01T19:42:18.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.422+0000 7fce7ebdc700 1 -- 172.21.15.17:0/777124983 shutdown_connections 2022-08-01T19:42:18.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.422+0000 7fce7ebdc700 1 --2- 172.21.15.17:0/777124983 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fce5c053b30 0x7fce5c055ff0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:18.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.422+0000 7fce7ebdc700 1 --2- 172.21.15.17:0/777124983 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fce780b6af0 0x7fce78156210 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:18.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.422+0000 7fce7ebdc700 1 --2- 172.21.15.17:0/777124983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce780b6710 0x7fce78155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:18.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.422+0000 7fce7ebdc700 1 --2- 172.21.15.17:0/777124983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fce780b4550 0x7fce780b81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:18.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.422+0000 7fce7ebdc700 1 -- 172.21.15.17:0/777124983 >> 172.21.15.17:0/777124983 conn(0x7fce7805c620 msgr2=0x7fce781432b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:18.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.422+0000 7fce7ebdc700 1 -- 172.21.15.17:0/777124983 shutdown_connections 2022-08-01T19:42:18.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:18.422+0000 7fce7ebdc700 1 -- 172.21.15.17:0/777124983 wait complete. 2022-08-01T19:42:18.437 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 58 2022-08-01T19:42:18.741 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 58, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:42:10.906134+0000', 'modified': '2022-08-01T19:42:11.922099+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7600}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7600': {'gid': 7600, 'name': 'b', 'rank': 0, 'incarnation': 57, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.77:6835/2286085186', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6834', 'nonce': 2286085186}, {'type': 'v1', 'addr': '172.21.15.77:6835', 'nonce': 2286085186}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:42:18.742 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:42:18.743 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-08-01T19:42:18.858 INFO:teuthology.orchestra.run.smithi077.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":7600,"want_state":"up:active","state":"up:active","rank_uptime":7.937635695,"mdsmap_epoch":58,"osdmap_epoch":117,"osdmap_epoch_barrier":117,"uptime":12.295389828999999} 2022-08-01T19:42:18.859 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 7600, "mdsmap_epoch": 58, "osdmap_epoch": 117, "osdmap_epoch_barrier": 117, "rank_uptime": 7.937635695, "state": "up:active", "uptime": 12.295389829, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:42:18.859 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T19:42:18.860 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-08-01T19:42:18.861 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi190.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-08-01T19:42:18.861 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:42:18.862 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:42:18.876 DEBUG:teuthology.orchestra.run.smithi190:> sudo modprobe fuse 2022-08-01T19:42:18.933 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:42:18.934 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:42:18.946 INFO:teuthology.orchestra.run.smithi190.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:42:18.947 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:42:18.948 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:42:18.949 DEBUG:teuthology.orchestra.run.smithi190:> ls /sys/fs/fuse/connections 2022-08-01T19:42:19.000 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T19:42:19.001 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:42:19.008 DEBUG:teuthology.orchestra.run.smithi190:> sudo modprobe fuse 2022-08-01T19:42:19.025 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:42:19.026 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:42:19.083 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:2022-08-01T19:42:19.074+0000 7f2cfbd73080 -1 init, newargv = 0x55594e9007a0 newargc=9ceph-fuse[23012 2022-08-01T19:42:19.084 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:]: starting ceph client 2022-08-01T19:42:19.090 INFO:teuthology.orchestra.run.smithi190.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:42:19.091 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:42:19.092 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:42:19.093 DEBUG:teuthology.orchestra.run.smithi190:> ls /sys/fs/fuse/connections 2022-08-01T19:42:19.093 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:ceph-fuse[23012]: starting fuse 2022-08-01T19:42:19.098 INFO:teuthology.orchestra.run.smithi190.stdout:52 2022-08-01T19:42:19.101 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T19:42:19.102 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:42:19.103 DEBUG:teuthology.orchestra.run.smithi190:> sudo python3 -c ' 2022-08-01T19:42:19.103 DEBUG:teuthology.orchestra.run.smithi190:> import glob 2022-08-01T19:42:19.104 DEBUG:teuthology.orchestra.run.smithi190:> import re 2022-08-01T19:42:19.104 DEBUG:teuthology.orchestra.run.smithi190:> import os 2022-08-01T19:42:19.105 DEBUG:teuthology.orchestra.run.smithi190:> import subprocess 2022-08-01T19:42:19.106 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:42:19.106 DEBUG:teuthology.orchestra.run.smithi190:> def find_socket(client_name): 2022-08-01T19:42:19.107 DEBUG:teuthology.orchestra.run.smithi190:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-08-01T19:42:19.107 DEBUG:teuthology.orchestra.run.smithi190:> files = glob.glob(asok_path) 2022-08-01T19:42:19.108 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:42:19.108 DEBUG:teuthology.orchestra.run.smithi190:> # Given a non-glob path, it better be there 2022-08-01T19:42:19.109 DEBUG:teuthology.orchestra.run.smithi190:> if "*" not in asok_path: 2022-08-01T19:42:19.109 DEBUG:teuthology.orchestra.run.smithi190:> assert(len(files) == 1) 2022-08-01T19:42:19.110 DEBUG:teuthology.orchestra.run.smithi190:> return files[0] 2022-08-01T19:42:19.110 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:42:19.111 DEBUG:teuthology.orchestra.run.smithi190:> for f in files: 2022-08-01T19:42:19.111 DEBUG:teuthology.orchestra.run.smithi190:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T19:42:19.112 DEBUG:teuthology.orchestra.run.smithi190:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T19:42:19.113 DEBUG:teuthology.orchestra.run.smithi190:> return f 2022-08-01T19:42:19.114 DEBUG:teuthology.orchestra.run.smithi190:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T19:42:19.114 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:42:19.115 DEBUG:teuthology.orchestra.run.smithi190:> print(find_socket("client.1")) 2022-08-01T19:42:19.116 DEBUG:teuthology.orchestra.run.smithi190:> ' 2022-08-01T19:42:19.164 INFO:teuthology.orchestra.run.smithi190.stdout:/var/run/ceph/ceph-client.1.23012.asok 2022-08-01T19:42:19.169 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.23012.asok 2022-08-01T19:42:19.170 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:42:19.171 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.23012.asok status 2022-08-01T19:42:19.309 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2022-08-01T19:42:19.310 INFO:teuthology.orchestra.run.smithi190.stdout: "metadata": { 2022-08-01T19:42:19.312 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T19:42:19.313 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T19:42:19.313 INFO:teuthology.orchestra.run.smithi190.stdout: "entity_id": "1", 2022-08-01T19:42:19.313 INFO:teuthology.orchestra.run.smithi190.stdout: "hostname": "smithi190", 2022-08-01T19:42:19.313 INFO:teuthology.orchestra.run.smithi190.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-08-01T19:42:19.314 INFO:teuthology.orchestra.run.smithi190.stdout: "pid": "23012", 2022-08-01T19:42:19.314 INFO:teuthology.orchestra.run.smithi190.stdout: "root": "/" 2022-08-01T19:42:19.314 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:42:19.314 INFO:teuthology.orchestra.run.smithi190.stdout: "dentry_count": 0, 2022-08-01T19:42:19.315 INFO:teuthology.orchestra.run.smithi190.stdout: "dentry_pinned_count": 0, 2022-08-01T19:42:19.315 INFO:teuthology.orchestra.run.smithi190.stdout: "id": 7783, 2022-08-01T19:42:19.315 INFO:teuthology.orchestra.run.smithi190.stdout: "inst": { 2022-08-01T19:42:19.315 INFO:teuthology.orchestra.run.smithi190.stdout: "name": { 2022-08-01T19:42:19.316 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "client", 2022-08-01T19:42:19.316 INFO:teuthology.orchestra.run.smithi190.stdout: "num": 7783 2022-08-01T19:42:19.317 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:42:19.317 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": { 2022-08-01T19:42:19.320 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "v1", 2022-08-01T19:42:19.322 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": "172.21.15.190:0", 2022-08-01T19:42:19.322 INFO:teuthology.orchestra.run.smithi190.stdout: "nonce": 2584123114 2022-08-01T19:42:19.323 INFO:teuthology.orchestra.run.smithi190.stdout: } 2022-08-01T19:42:19.323 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:42:19.323 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": { 2022-08-01T19:42:19.324 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "v1", 2022-08-01T19:42:19.324 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": "172.21.15.190:0", 2022-08-01T19:42:19.324 INFO:teuthology.orchestra.run.smithi190.stdout: "nonce": 2584123114 2022-08-01T19:42:19.325 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:42:19.325 INFO:teuthology.orchestra.run.smithi190.stdout: "inst_str": "client.7783 172.21.15.190:0/2584123114", 2022-08-01T19:42:19.326 INFO:teuthology.orchestra.run.smithi190.stdout: "addr_str": "172.21.15.190:0/2584123114", 2022-08-01T19:42:19.326 INFO:teuthology.orchestra.run.smithi190.stdout: "inode_count": 1, 2022-08-01T19:42:19.326 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_epoch": 58, 2022-08-01T19:42:19.327 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_epoch": 117, 2022-08-01T19:42:19.327 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_epoch_barrier": 0, 2022-08-01T19:42:19.327 INFO:teuthology.orchestra.run.smithi190.stdout: "blacklisted": false, 2022-08-01T19:42:19.328 INFO:teuthology.orchestra.run.smithi190.stdout: "fs_name": "cephfs" 2022-08-01T19:42:19.328 INFO:teuthology.orchestra.run.smithi190.stdout:} 2022-08-01T19:42:19.329 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:42:19.330 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:42:19.331 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:42:19.343 INFO:teuthology.orchestra.run.smithi190.stdout:fuseblk 2022-08-01T19:42:19.344 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T19:42:19.345 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:42:19.351 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:42:19.371 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:42:19.373 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:42:19.374 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:42:19.396 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:42:19.397 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:42:19.398 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:42:19.398 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:42:19.399 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:42:19.415 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:42:19.416 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:42:19.416 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi085.front.sepia.ceph.com... 2022-08-01T19:42:19.417 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T19:42:19.417 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:42:19.439 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:ceph-fuse[23439]: fuse finished with error 0 and tester_r 0 2022-08-01T19:42:19.449 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:42:19.449 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:42:19.450 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:42:19.507 INFO:teuthology.orchestra.run.smithi085.stdout:ext2/ext3 2022-08-01T19:42:19.510 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T19:42:19.511 INFO:teuthology.orchestra.run:waiting for 900 2022-08-01T19:42:25.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T19:42:25.615 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:42:25.636 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:42:25.638 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:42:25.639 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:42:25.653 INFO:teuthology.orchestra.run.smithi190.stdout:fuseblk 2022-08-01T19:42:25.653 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T19:42:25.654 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:42:25.655 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:42:25.656 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:42:25.669 INFO:teuthology.orchestra.run.smithi190.stdout:fuseblk 2022-08-01T19:42:25.670 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T19:42:25.671 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi190.front.sepia.ceph.com... 2022-08-01T19:42:25.671 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T19:42:25.671 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:42:25.730 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:ceph-fuse[23012]: fuse finished with error 0 and tester_r 0 2022-08-01T19:42:25.733 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:42:25.736 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:42:25.737 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:42:25.746 INFO:teuthology.orchestra.run.smithi190.stdout:ext2/ext3 2022-08-01T19:42:25.749 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T19:42:25.750 INFO:teuthology.orchestra.run:waiting for 900 2022-08-01T19:42:31.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T19:42:31.856 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:42:31.866 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-08-01T19:42:31.867 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:42:31.868 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-08-01T19:42:31.869 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:42:31.869 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-08-01T19:42:31.870 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:42:31.890 INFO:tasks.ceph.mds.b:Stopped 2022-08-01T19:42:31.891 DEBUG:teuthology.parallel:result is None 2022-08-01T19:42:31.892 INFO:tasks.ceph.mds.c:Stopped 2022-08-01T19:42:31.893 DEBUG:teuthology.parallel:result is None 2022-08-01T19:42:31.894 INFO:tasks.ceph.mds.a:Stopped 2022-08-01T19:42:31.894 DEBUG:teuthology.parallel:result is None 2022-08-01T19:42:31.895 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail a 2022-08-01T19:42:31.896 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail c 2022-08-01T19:42:31.908 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail b 2022-08-01T19:42:32.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.006+0000 7f03f33b1700 1 Processor -- start 2022-08-01T19:42:32.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.006+0000 7f03f33b1700 1 -- start start 2022-08-01T19:42:32.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.006+0000 7f03f33b1700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f03ec0b6a90 0x7f03ec0acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:32.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.006+0000 7f03f33b1700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f03ec060aa0 0x7f03ec0aa050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:32.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.006+0000 7f03f33b1700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03ec0b6e70 con 0x7f03ec0aa540 2022-08-01T19:42:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.006+0000 7f03f33b1700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03ec061370 con 0x7f03ec060aa0 2022-08-01T19:42:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.006+0000 7f03f33b1700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03ec061610 con 0x7f03ec0b6a90 2022-08-01T19:42:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.006+0000 7f03f094c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f03ec060aa0 0x7f03ec0aa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.006+0000 7f03f194e700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f03ec0aa540 0x7f03ec0b1a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35810/0 (socket says 172.21.15.17:35810) 2022-08-01T19:42:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.006+0000 7f03f194e700 1 -- 172.21.15.17:0/1439792534 learned_addr learned my addr 172.21.15.17:0/1439792534 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.006+0000 7f03f114d700 1 --2- 172.21.15.17:0/1439792534 >> v2:172.21.15.77:3300/0 conn(0x7f03ec0b6a90 0x7f03ec0acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:32.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.006+0000 7f03dbfff700 1 -- 172.21.15.17:0/1439792534 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1862401419 0 0) 0x7f03ec0b6e70 con 0x7f03ec0aa540 2022-08-01T19:42:32.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.006+0000 7f03dbfff700 1 -- 172.21.15.17:0/1439792534 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03d4003260 con 0x7f03ec0aa540 2022-08-01T19:42:32.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.006+0000 7f03f094c700 1 -- 172.21.15.17:0/1439792534 >> v2:172.21.15.77:3300/0 conn(0x7f03ec0b6a90 msgr2=0x7f03ec0acee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:32.040 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.006+0000 7f03f094c700 1 --2- 172.21.15.17:0/1439792534 >> v2:172.21.15.77:3300/0 conn(0x7f03ec0b6a90 0x7f03ec0acee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:32.040 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.006+0000 7f03f094c700 1 -- 172.21.15.17:0/1439792534 >> v1:172.21.15.17:6789/0 conn(0x7f03ec0aa540 legacy=0x7f03ec0b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:32.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.006+0000 7f03f094c700 1 -- 172.21.15.17:0/1439792534 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ec0618b0 con 0x7f03ec060aa0 2022-08-01T19:42:32.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.006+0000 7f03f094c700 1 --2- 172.21.15.17:0/1439792534 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03ec060aa0 0x7f03ec0aa050 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 rx=0x7f03dc002f70 tx=0x7f03dc006df0).ready entity=mon.0 client_cookie=3b4315ac757c4973 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:32.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.006+0000 7f03dbfff700 1 -- 172.21.15.17:0/1439792534 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03dc00d4f0 con 0x7f03ec060aa0 2022-08-01T19:42:32.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.006+0000 7f03dbfff700 1 -- 172.21.15.17:0/1439792534 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03dc000e90 con 0x7f03ec060aa0 2022-08-01T19:42:32.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.006+0000 7f03dbfff700 1 -- 172.21.15.17:0/1439792534 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03dc00d840 con 0x7f03ec060aa0 2022-08-01T19:42:32.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.006+0000 7f03f33b1700 1 -- 172.21.15.17:0/1439792534 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03ec060aa0 msgr2=0x7f03ec0aa050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:32.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.006+0000 7f03f33b1700 1 --2- 172.21.15.17:0/1439792534 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03ec060aa0 0x7f03ec0aa050 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 rx=0x7f03dc002f70 tx=0x7f03dc006df0).stop 2022-08-01T19:42:32.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.006+0000 7f03f33b1700 1 -- 172.21.15.17:0/1439792534 shutdown_connections 2022-08-01T19:42:32.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.006+0000 7f03f33b1700 1 --2- 172.21.15.17:0/1439792534 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03ec060aa0 0x7f03ec0aa050 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:32.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.006+0000 7f03f33b1700 1 --2- 172.21.15.17:0/1439792534 >> v2:172.21.15.77:3300/0 conn(0x7f03ec0b6a90 0x7f03ec0acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:32.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.006+0000 7f03f33b1700 1 -- 172.21.15.17:0/1439792534 >> 172.21.15.17:0/1439792534 conn(0x7f03ec0b3610 msgr2=0x7f03ec0b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:32.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.006+0000 7f03f33b1700 1 -- 172.21.15.17:0/1439792534 shutdown_connections 2022-08-01T19:42:32.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.006+0000 7f03f33b1700 1 -- 172.21.15.17:0/1439792534 wait complete. 2022-08-01T19:42:32.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f03f33b1700 1 Processor -- start 2022-08-01T19:42:32.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f03f33b1700 1 -- start start 2022-08-01T19:42:32.065 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f84d6125700 1 Processor -- start 2022-08-01T19:42:32.065 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f84d6125700 1 -- start start 2022-08-01T19:42:32.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f03f33b1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03ec060aa0 0x7f03ec0b5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:32.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f03f33b1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03ec0aa540 0x7f03ec13e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:32.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f03f33b1700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f03ec0b6a90 0x7f03ec13e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:32.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f03f33b1700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec13f710 con 0x7f03ec0aa540 2022-08-01T19:42:32.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f03f33b1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec134390 con 0x7f03ec060aa0 2022-08-01T19:42:32.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f03f33b1700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec133680 con 0x7f03ec0b6a90 2022-08-01T19:42:32.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f03f094c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03ec0aa540 0x7f03ec13e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:32.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f03f094c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03ec0aa540 0x7f03ec13e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43622/0 (socket says 172.21.15.17:43622) 2022-08-01T19:42:32.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f03f094c700 1 -- 172.21.15.17:0/2224498193 learned_addr learned my addr 172.21.15.17:0/2224498193 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:32.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f03f114d700 1 --2- 172.21.15.17:0/2224498193 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03ec060aa0 0x7f03ec0b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:32.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f03f194e700 1 --2- 172.21.15.17:0/2224498193 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f03ec0b6a90 0x7f03ec13e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:32.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f03f094c700 1 -- 172.21.15.17:0/2224498193 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f03ec0b6a90 msgr2=0x7f03ec13e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:32.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f03f094c700 1 --2- 172.21.15.17:0/2224498193 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f03ec0b6a90 0x7f03ec13e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:32.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f03f094c700 1 -- 172.21.15.17:0/2224498193 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03ec060aa0 msgr2=0x7f03ec0b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:32.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f03f094c700 1 --2- 172.21.15.17:0/2224498193 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03ec060aa0 0x7f03ec0b5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:32.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f03f094c700 1 -- 172.21.15.17:0/2224498193 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03dc00b180 con 0x7f03ec0aa540 2022-08-01T19:42:32.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f03f094c700 1 --2- 172.21.15.17:0/2224498193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03ec0aa540 0x7f03ec13e0b0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 rx=0x7f03ec1337c0 tx=0x7f03dc014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:32.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f84d6125700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84d00b6420 0x7f84d0061600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:32.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f84d6125700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f84d0142ff0 0x7f84d01434a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:32.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f84d6125700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84d00b60e0 con 0x7f84d00b41f0 2022-08-01T19:42:32.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f84d6125700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84d00b2100 con 0x7f84d0142ff0 2022-08-01T19:42:32.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f84d6125700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84d00b23a0 con 0x7f84d00b6420 2022-08-01T19:42:32.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f03da7fc700 1 -- 172.21.15.17:0/2224498193 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03dc013da0 con 0x7f03ec0aa540 2022-08-01T19:42:32.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f03da7fc700 1 -- 172.21.15.17:0/2224498193 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03dc00de60 con 0x7f03ec0aa540 2022-08-01T19:42:32.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f03f33b1700 1 -- 172.21.15.17:0/2224498193 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03ec134ca0 con 0x7f03ec0aa540 2022-08-01T19:42:32.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f84d4922700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7f84d00b41f0 0x7f84d0061af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:52620/0 (socket says 172.21.15.17:52620) 2022-08-01T19:42:32.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f84d4922700 1 -- 172.21.15.17:0/2490910328 learned_addr learned my addr 172.21.15.17:0/2490910328 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:32.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f03da7fc700 1 -- 172.21.15.17:0/2224498193 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f03dc01e5f0 con 0x7f03ec0aa540 2022-08-01T19:42:32.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f84d5123700 1 --2- 172.21.15.17:0/2490910328 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84d00b6420 0x7f84d0061600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:32.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f84d5924700 1 --2- 172.21.15.17:0/2490910328 >> v2:172.21.15.77:3300/0 conn(0x7f84d0142ff0 0x7f84d01434a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:32.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f84bffff700 1 -- 172.21.15.17:0/2490910328 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 704254 0 0) 0x7f84c0001110 con 0x7f84d00b41f0 2022-08-01T19:42:32.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f84bffff700 1 -- 172.21.15.17:0/2490910328 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84b8003260 con 0x7f84d00b41f0 2022-08-01T19:42:32.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f03f33b1700 1 -- 172.21.15.17:0/2224498193 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03ec13c0b0 con 0x7f03ec0aa540 2022-08-01T19:42:32.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f84d5123700 1 -- 172.21.15.17:0/2490910328 >> v2:172.21.15.77:3300/0 conn(0x7f84d0142ff0 msgr2=0x7f84d01434a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:32.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f84d5123700 1 --2- 172.21.15.17:0/2490910328 >> v2:172.21.15.77:3300/0 conn(0x7f84d0142ff0 0x7f84d01434a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:32.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f84d5123700 1 -- 172.21.15.17:0/2490910328 >> v1:172.21.15.77:6789/0 conn(0x7f84d00b41f0 legacy=0x7f84d0061af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:32.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f84d5123700 1 -- 172.21.15.17:0/2490910328 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d00b2640 con 0x7f84d00b6420 2022-08-01T19:42:32.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.010+0000 7f84d5123700 1 --2- 172.21.15.17:0/2490910328 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84d00b6420 0x7f84d0061600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f84cc002f70 tx=0x7f84cc006df0).ready entity=mon.2 client_cookie=aa390fb7e78c6efc server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:32.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f84bffff700 1 -- 172.21.15.17:0/2490910328 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84cc00d4a0 con 0x7f84d00b6420 2022-08-01T19:42:32.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f84bffff700 1 -- 172.21.15.17:0/2490910328 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84cc000e90 con 0x7f84d00b6420 2022-08-01T19:42:32.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f84bffff700 1 -- 172.21.15.17:0/2490910328 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84cc00d7f0 con 0x7f84d00b6420 2022-08-01T19:42:32.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f03f33b1700 1 -- 172.21.15.17:0/2224498193 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03c8003410 con 0x7f03ec0aa540 2022-08-01T19:42:32.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f56613e5700 1 Processor -- start 2022-08-01T19:42:32.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f56613e5700 1 -- start start 2022-08-01T19:42:32.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f84d6125700 1 -- 172.21.15.17:0/2490910328 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84d00b6420 msgr2=0x7f84d0061600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:32.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f84d6125700 1 --2- 172.21.15.17:0/2490910328 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84d00b6420 0x7f84d0061600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f84cc002f70 tx=0x7f84cc006df0).stop 2022-08-01T19:42:32.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f84d6125700 1 -- 172.21.15.17:0/2490910328 shutdown_connections 2022-08-01T19:42:32.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f84d6125700 1 --2- 172.21.15.17:0/2490910328 >> v2:172.21.15.77:3300/0 conn(0x7f84d0142ff0 0x7f84d01434a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:32.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f84d6125700 1 --2- 172.21.15.17:0/2490910328 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84d00b6420 0x7f84d0061600 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:32.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f84d6125700 1 -- 172.21.15.17:0/2490910328 >> 172.21.15.17:0/2490910328 conn(0x7f84d005c620 msgr2=0x7f84d005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:32.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f84d6125700 1 -- 172.21.15.17:0/2490910328 shutdown_connections 2022-08-01T19:42:32.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f84d6125700 1 -- 172.21.15.17:0/2490910328 wait complete. 2022-08-01T19:42:32.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f03da7fc700 1 -- 172.21.15.17:0/2224498193 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f03dc025030 con 0x7f03ec0aa540 2022-08-01T19:42:32.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f56613e5700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f565c0b6710 0x7f565c0b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:32.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f56613e5700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f565c0b4550 0x7f565c0b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:32.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f56613e5700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f565c061ec0 0x7f565c062330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:32.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f56613e5700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f565c144240 con 0x7f565c0b6710 2022-08-01T19:42:32.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f56613e5700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f565c0b6fe0 con 0x7f565c061ec0 2022-08-01T19:42:32.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f56613e5700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f565c143d00 con 0x7f565c0b4550 2022-08-01T19:42:32.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f03da7fc700 1 --2- 172.21.15.17:0/2224498193 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f03cc053b00 0x7f03cc055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:32.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f565affd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f565c0b6710 0x7f565c0b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:32.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f565affd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f565c0b6710 0x7f565c0b5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43630/0 (socket says 172.21.15.17:43630) 2022-08-01T19:42:32.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f565affd700 1 -- 172.21.15.17:0/2447089710 learned_addr learned my addr 172.21.15.17:0/2447089710 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:32.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f565b7fe700 1 --2- 172.21.15.17:0/2447089710 >> v2:172.21.15.77:3300/0 conn(0x7f565c061ec0 0x7f565c062330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:32.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f03da7fc700 1 -- 172.21.15.17:0/2224498193 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f03dc06cd00 con 0x7f03ec0aa540 2022-08-01T19:42:32.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f565a7fc700 1 --2- 172.21.15.17:0/2447089710 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f565c0b4550 0x7f565c0b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:32.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f03f114d700 1 --2- 172.21.15.17:0/2224498193 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f03cc053b00 0x7f03cc055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:32.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f565affd700 1 -- 172.21.15.17:0/2447089710 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f565c0b4550 msgr2=0x7f565c0b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:32.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f565affd700 1 --2- 172.21.15.17:0/2447089710 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f565c0b4550 0x7f565c0b6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:32.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f565affd700 1 -- 172.21.15.17:0/2447089710 >> v2:172.21.15.77:3300/0 conn(0x7f565c061ec0 msgr2=0x7f565c062330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:32.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f565affd700 1 --2- 172.21.15.17:0/2447089710 >> v2:172.21.15.77:3300/0 conn(0x7f565c061ec0 0x7f565c062330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:32.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f565affd700 1 -- 172.21.15.17:0/2447089710 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f565c143fa0 con 0x7f565c0b6710 2022-08-01T19:42:32.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f565a7fc700 1 --2- 172.21.15.17:0/2447089710 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f565c0b4550 0x7f565c0b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:42:32.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f565affd700 1 --2- 172.21.15.17:0/2447089710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f565c0b6710 0x7f565c0b5d80 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 rx=0x7f564c002f70 tx=0x7f564c006eb0).ready entity=mon.0 client_cookie=cd567f139e0b1940 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:32.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f84d6125700 1 Processor -- start 2022-08-01T19:42:32.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f84d6125700 1 -- start start 2022-08-01T19:42:32.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.014+0000 7f03f114d700 1 --2- 172.21.15.17:0/2224498193 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f03cc053b00 0x7f03cc055fc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7f03e8008000 tx=0x7f03e8008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:32.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.018+0000 7f84d6125700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84d00b41f0 0x7f84d0142700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:32.158 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.018+0000 7f84d6125700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84d00b6420 0x7f84d01565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:32.158 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.018+0000 7f84d6125700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84d0142ff0 0x7f84d012ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:32.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.018+0000 7f84d6125700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d013d170 con 0x7f84d0142ff0 2022-08-01T19:42:32.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.018+0000 7f84d6125700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d00b00c0 con 0x7f84d00b6420 2022-08-01T19:42:32.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.018+0000 7f84d6125700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84d00b0360 con 0x7f84d00b41f0 2022-08-01T19:42:32.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.018+0000 7f84d5924700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84d0142ff0 0x7f84d012ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:32.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.018+0000 7f84d5924700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84d0142ff0 0x7f84d012ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43642/0 (socket says 172.21.15.17:43642) 2022-08-01T19:42:32.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.018+0000 7f84d5924700 1 -- 172.21.15.17:0/792209706 learned_addr learned my addr 172.21.15.17:0/792209706 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:32.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.018+0000 7f84d5123700 1 --2- 172.21.15.17:0/792209706 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84d00b41f0 0x7f84d0142700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:32.162 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.018+0000 7f84d4922700 1 --2- 172.21.15.17:0/792209706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84d00b6420 0x7f84d01565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:32.162 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.018+0000 7f84d5924700 1 -- 172.21.15.17:0/792209706 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84d00b41f0 msgr2=0x7f84d0142700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:32.163 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.018+0000 7f84d5924700 1 --2- 172.21.15.17:0/792209706 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84d00b41f0 0x7f84d0142700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:32.163 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.018+0000 7f84d5924700 1 -- 172.21.15.17:0/792209706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84d00b6420 msgr2=0x7f84d01565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:32.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.018+0000 7f84d5924700 1 --2- 172.21.15.17:0/792209706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84d00b6420 0x7f84d01565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:32.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.018+0000 7f84d5924700 1 -- 172.21.15.17:0/792209706 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84cc00b180 con 0x7f84d0142ff0 2022-08-01T19:42:32.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.018+0000 7f84d4922700 1 --2- 172.21.15.17:0/792209706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84d00b6420 0x7f84d01565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:42:32.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.018+0000 7f84d5123700 1 --2- 172.21.15.17:0/792209706 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84d00b41f0 0x7f84d0142700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:42:32.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.018+0000 7f84d5924700 1 --2- 172.21.15.17:0/792209706 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84d0142ff0 0x7f84d012ae40 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 rx=0x7f84c8006e00 tx=0x7f84c80091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:32.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.018+0000 7f84be7fc700 1 -- 172.21.15.17:0/792209706 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84c800c7f0 con 0x7f84d0142ff0 2022-08-01T19:42:32.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.018+0000 7f84d6125700 1 -- 172.21.15.17:0/792209706 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d00b0600 con 0x7f84d0142ff0 2022-08-01T19:42:32.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.018+0000 7f5659ffb700 1 -- 172.21.15.17:0/2447089710 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f564c00d570 con 0x7f565c0b6710 2022-08-01T19:42:32.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.018+0000 7f5659ffb700 1 -- 172.21.15.17:0/2447089710 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564c000e90 con 0x7f565c0b6710 2022-08-01T19:42:32.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.018+0000 7f5659ffb700 1 -- 172.21.15.17:0/2447089710 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f564c00d900 con 0x7f565c0b6710 2022-08-01T19:42:32.169 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.018+0000 7f84d6125700 1 -- 172.21.15.17:0/792209706 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d0156ae0 con 0x7f84d0142ff0 2022-08-01T19:42:32.170 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.018+0000 7f03da7fc700 1 -- 172.21.15.17:0/2224498193 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f03dc070030 con 0x7f03ec0aa540 2022-08-01T19:42:32.171 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.022+0000 7f84be7fc700 1 -- 172.21.15.17:0/792209706 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c800c950 con 0x7f84d0142ff0 2022-08-01T19:42:32.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.022+0000 7f84be7fc700 1 -- 172.21.15.17:0/792209706 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f84c800ccb0 con 0x7f84d0142ff0 2022-08-01T19:42:32.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.022+0000 7f56613e5700 1 -- 172.21.15.17:0/2447089710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f565c0b6710 msgr2=0x7f565c0b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:32.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.022+0000 7f56613e5700 1 --2- 172.21.15.17:0/2447089710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f565c0b6710 0x7f565c0b5d80 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 rx=0x7f564c002f70 tx=0x7f564c006eb0).stop 2022-08-01T19:42:32.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.022+0000 7f56613e5700 1 -- 172.21.15.17:0/2447089710 shutdown_connections 2022-08-01T19:42:32.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.022+0000 7f56613e5700 1 --2- 172.21.15.17:0/2447089710 >> v2:172.21.15.77:3300/0 conn(0x7f565c061ec0 0x7f565c062330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:32.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.022+0000 7f56613e5700 1 --2- 172.21.15.17:0/2447089710 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f565c0b4550 0x7f565c0b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:32.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.022+0000 7f56613e5700 1 --2- 172.21.15.17:0/2447089710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f565c0b6710 0x7f565c0b5d80 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:32.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.022+0000 7f56613e5700 1 -- 172.21.15.17:0/2447089710 >> 172.21.15.17:0/2447089710 conn(0x7f565c05c620 msgr2=0x7f565c05ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:32.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.022+0000 7f56613e5700 1 -- 172.21.15.17:0/2447089710 shutdown_connections 2022-08-01T19:42:32.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.022+0000 7f56613e5700 1 -- 172.21.15.17:0/2447089710 wait complete. 2022-08-01T19:42:32.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.022+0000 7f84d6125700 1 -- 172.21.15.17:0/792209706 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84ac003410 con 0x7f84d0142ff0 2022-08-01T19:42:32.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.022+0000 7f84be7fc700 1 -- 172.21.15.17:0/792209706 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f84c8022070 con 0x7f84d0142ff0 2022-08-01T19:42:32.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.022+0000 7f84be7fc700 1 --2- 172.21.15.17:0/792209706 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f84b0053b30 0x7f84b0055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:32.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.022+0000 7f84be7fc700 1 -- 172.21.15.17:0/792209706 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f84c806c690 con 0x7f84d0142ff0 2022-08-01T19:42:32.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.022+0000 7f84d5123700 1 --2- 172.21.15.17:0/792209706 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f84b0053b30 0x7f84b0055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:32.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.022+0000 7f56613e5700 1 Processor -- start 2022-08-01T19:42:32.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.022+0000 7f56613e5700 1 -- start start 2022-08-01T19:42:32.182 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.022+0000 7f84d5123700 1 --2- 172.21.15.17:0/792209706 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f84b0053b30 0x7f84b0055ff0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f84cc018000 tx=0x7f84cc014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:32.184 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.022+0000 7f56613e5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f565c061ec0 0x7f565c128e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:32.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.022+0000 7f56613e5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f565c0b4550 0x7f565c156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:32.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.022+0000 7f56613e5700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f565c0b6710 0x7f565c156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:32.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.022+0000 7f56613e5700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565c141900 con 0x7f565c0b4550 2022-08-01T19:42:32.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.022+0000 7f56613e5700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565c152610 con 0x7f565c061ec0 2022-08-01T19:42:32.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.022+0000 7f56613e5700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f565c155b80 con 0x7f565c0b6710 2022-08-01T19:42:32.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.022+0000 7f565a7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f565c0b4550 0x7f565c156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:32.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.022+0000 7f565a7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f565c0b4550 0x7f565c156080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43648/0 (socket says 172.21.15.17:43648) 2022-08-01T19:42:32.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.022+0000 7f565a7fc700 1 -- 172.21.15.17:0/1069198731 learned_addr learned my addr 172.21.15.17:0/1069198731 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:32.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.022+0000 7f565affd700 1 --2- 172.21.15.17:0/1069198731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f565c061ec0 0x7f565c128e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:32.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.026+0000 7f565b7fe700 1 --2- 172.21.15.17:0/1069198731 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f565c0b6710 0x7f565c156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:32.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.026+0000 7f565a7fc700 1 -- 172.21.15.17:0/1069198731 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f565c0b6710 msgr2=0x7f565c156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:32.188 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.026+0000 7f565a7fc700 1 --2- 172.21.15.17:0/1069198731 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f565c0b6710 0x7f565c156570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:32.188 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.026+0000 7f565a7fc700 1 -- 172.21.15.17:0/1069198731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f565c061ec0 msgr2=0x7f565c128e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:32.188 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.026+0000 7f565a7fc700 1 --2- 172.21.15.17:0/1069198731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f565c061ec0 0x7f565c128e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:32.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.026+0000 7f565a7fc700 1 -- 172.21.15.17:0/1069198731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f564c00b1e0 con 0x7f565c0b4550 2022-08-01T19:42:32.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.026+0000 7f565a7fc700 1 --2- 172.21.15.17:0/1069198731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f565c0b4550 0x7f565c156080 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 rx=0x7f565c0b3da0 tx=0x7f5650008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:32.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.026+0000 7f564bfff700 1 -- 172.21.15.17:0/1069198731 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5650010580 con 0x7f565c0b4550 2022-08-01T19:42:32.190 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.026+0000 7f564bfff700 1 -- 172.21.15.17:0/1069198731 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56500106e0 con 0x7f565c0b4550 2022-08-01T19:42:32.190 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.026+0000 7f56613e5700 1 -- 172.21.15.17:0/1069198731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f565c156a60 con 0x7f565c0b4550 2022-08-01T19:42:32.190 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.026+0000 7f564bfff700 1 -- 172.21.15.17:0/1069198731 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5650010a40 con 0x7f565c0b4550 2022-08-01T19:42:32.190 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.026+0000 7f56613e5700 1 -- 172.21.15.17:0/1069198731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f565c126670 con 0x7f565c0b4550 2022-08-01T19:42:32.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.026+0000 7f84be7fc700 1 -- 172.21.15.17:0/792209706 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f84c8043450 con 0x7f84d0142ff0 2022-08-01T19:42:32.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.026+0000 7f564bfff700 1 -- 172.21.15.17:0/1069198731 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5650010ba0 con 0x7f565c0b4550 2022-08-01T19:42:32.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.026+0000 7f56613e5700 1 -- 172.21.15.17:0/1069198731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f565c061780 con 0x7f565c0b4550 2022-08-01T19:42:32.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.026+0000 7f564bfff700 1 --2- 172.21.15.17:0/1069198731 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5640053af0 0x7f5640055fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:32.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.026+0000 7f564bfff700 1 -- 172.21.15.17:0/1069198731 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f565006be50 con 0x7f565c0b4550 2022-08-01T19:42:32.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.030+0000 7f565affd700 1 --2- 172.21.15.17:0/1069198731 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5640053af0 0x7f5640055fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:32.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.030+0000 7f565affd700 1 --2- 172.21.15.17:0/1069198731 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5640053af0 0x7f5640055fb0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f564c01c000 tx=0x7f564c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:32.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.034+0000 7f564bfff700 1 -- 172.21.15.17:0/1069198731 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5650042bb0 con 0x7f565c0b4550 2022-08-01T19:42:32.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.186+0000 7f03f33b1700 1 -- 172.21.15.17:0/2224498193 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) v1 -- 0x7f03c80036c0 con 0x7f03ec0aa540 2022-08-01T19:42:32.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.214+0000 7f84d6125700 1 -- 172.21.15.17:0/792209706 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) v1 -- 0x7f84ac0036c0 con 0x7f84d0142ff0 2022-08-01T19:42:32.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.222+0000 7f56613e5700 1 -- 172.21.15.17:0/1069198731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "c"} v 0) v1 -- 0x7f565c0574e0 con 0x7f565c0b4550 2022-08-01T19:42:32.250 INFO:tasks.ceph.mon.a.smithi017.stderr:2022-08-01T19:42:32.242+0000 7f6a53773700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-08-01T19:42:32.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.246+0000 7f03da7fc700 1 -- 172.21.15.17:0/2224498193 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 7594 v59) v1 ==== 95+0+0 (secure 0 0 0) 0x7f03dc023070 con 0x7f03ec0aa540 2022-08-01T19:42:32.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.246+0000 7f84be7fc700 1 -- 172.21.15.17:0/792209706 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 7600 v59) v1 ==== 95+0+0 (secure 0 0 0) 0x7f84c8016020 con 0x7f84d0142ff0 2022-08-01T19:42:32.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.246+0000 7f564bfff700 1 -- 172.21.15.17:0/1069198731 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "c"}]=0 failed mds gid 7581 v59) v1 ==== 95+0+0 (secure 0 0 0) 0x7f56500461d0 con 0x7f565c0b4550 2022-08-01T19:42:32.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f84d6125700 1 -- 172.21.15.17:0/792209706 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f84b0053b30 msgr2=0x7f84b0055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:32.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f84d6125700 1 --2- 172.21.15.17:0/792209706 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f84b0053b30 0x7f84b0055ff0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f84cc018000 tx=0x7f84cc014000).stop 2022-08-01T19:42:32.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f84d6125700 1 -- 172.21.15.17:0/792209706 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84d0142ff0 msgr2=0x7f84d012ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:32.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f84d6125700 1 --2- 172.21.15.17:0/792209706 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84d0142ff0 0x7f84d012ae40 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 rx=0x7f84c8006e00 tx=0x7f84c80091d0).stop 2022-08-01T19:42:32.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f84d6125700 1 -- 172.21.15.17:0/792209706 shutdown_connections 2022-08-01T19:42:32.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f84d6125700 1 --2- 172.21.15.17:0/792209706 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f84b0053b30 0x7f84b0055ff0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:32.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f84d6125700 1 --2- 172.21.15.17:0/792209706 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84d0142ff0 0x7f84d012ae40 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:32.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f84d6125700 1 --2- 172.21.15.17:0/792209706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84d00b6420 0x7f84d01565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:32.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f84d6125700 1 --2- 172.21.15.17:0/792209706 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f84d00b41f0 0x7f84d0142700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:32.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f84d6125700 1 -- 172.21.15.17:0/792209706 >> 172.21.15.17:0/792209706 conn(0x7f84d005c620 msgr2=0x7f84d00b8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:32.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f84d6125700 1 -- 172.21.15.17:0/792209706 shutdown_connections 2022-08-01T19:42:32.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f84d6125700 1 -- 172.21.15.17:0/792209706 wait complete. 2022-08-01T19:42:32.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f03f33b1700 1 -- 172.21.15.17:0/2224498193 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f03cc053b00 msgr2=0x7f03cc055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:32.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f03f33b1700 1 --2- 172.21.15.17:0/2224498193 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f03cc053b00 0x7f03cc055fc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7f03e8008000 tx=0x7f03e8008040).stop 2022-08-01T19:42:32.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f03f33b1700 1 -- 172.21.15.17:0/2224498193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03ec0aa540 msgr2=0x7f03ec13e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:32.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f03f33b1700 1 --2- 172.21.15.17:0/2224498193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03ec0aa540 0x7f03ec13e0b0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 rx=0x7f03ec1337c0 tx=0x7f03dc014000).stop 2022-08-01T19:42:32.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f03f33b1700 1 -- 172.21.15.17:0/2224498193 shutdown_connections 2022-08-01T19:42:32.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f03f33b1700 1 --2- 172.21.15.17:0/2224498193 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f03cc053b00 0x7f03cc055fc0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:32.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f03f33b1700 1 --2- 172.21.15.17:0/2224498193 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f03ec0b6a90 0x7f03ec13e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:32.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f03f33b1700 1 --2- 172.21.15.17:0/2224498193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03ec0aa540 0x7f03ec13e0b0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:32.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f03f33b1700 1 --2- 172.21.15.17:0/2224498193 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03ec060aa0 0x7f03ec0b5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:32.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f03f33b1700 1 -- 172.21.15.17:0/2224498193 >> 172.21.15.17:0/2224498193 conn(0x7f03ec0b3610 msgr2=0x7f03ec138960 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:32.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f03f33b1700 1 -- 172.21.15.17:0/2224498193 shutdown_connections 2022-08-01T19:42:32.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f03f33b1700 1 -- 172.21.15.17:0/2224498193 wait complete. 2022-08-01T19:42:32.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f56613e5700 1 -- 172.21.15.17:0/1069198731 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5640053af0 msgr2=0x7f5640055fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:32.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f56613e5700 1 --2- 172.21.15.17:0/1069198731 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5640053af0 0x7f5640055fb0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f564c01c000 tx=0x7f564c015000).stop 2022-08-01T19:42:32.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f56613e5700 1 -- 172.21.15.17:0/1069198731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f565c0b4550 msgr2=0x7f565c156080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:32.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f56613e5700 1 --2- 172.21.15.17:0/1069198731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f565c0b4550 0x7f565c156080 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 rx=0x7f565c0b3da0 tx=0x7f5650008040).stop 2022-08-01T19:42:32.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f56613e5700 1 -- 172.21.15.17:0/1069198731 shutdown_connections 2022-08-01T19:42:32.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f56613e5700 1 --2- 172.21.15.17:0/1069198731 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5640053af0 0x7f5640055fb0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:32.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f56613e5700 1 --2- 172.21.15.17:0/1069198731 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f565c0b6710 0x7f565c156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:32.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f56613e5700 1 --2- 172.21.15.17:0/1069198731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f565c0b4550 0x7f565c156080 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:32.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f56613e5700 1 --2- 172.21.15.17:0/1069198731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f565c061ec0 0x7f565c128e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:32.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f56613e5700 1 -- 172.21.15.17:0/1069198731 >> 172.21.15.17:0/1069198731 conn(0x7f565c05c620 msgr2=0x7f565c1270d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:32.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f56613e5700 1 -- 172.21.15.17:0/1069198731 shutdown_connections 2022-08-01T19:42:32.266 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.250+0000 7f56613e5700 1 -- 172.21.15.17:0/1069198731 wait complete. 2022-08-01T19:42:32.266 INFO:teuthology.orchestra.run.smithi017.stderr:failed mds gid 7600 2022-08-01T19:42:32.267 INFO:teuthology.orchestra.run.smithi017.stderr:failed mds gid 7594 2022-08-01T19:42:32.267 INFO:teuthology.orchestra.run.smithi017.stderr:failed mds gid 7581 2022-08-01T19:42:32.274 DEBUG:teuthology.parallel:result is '' 2022-08-01T19:42:32.276 DEBUG:teuthology.parallel:result is '' 2022-08-01T19:42:32.276 DEBUG:teuthology.parallel:result is '' 2022-08-01T19:42:32.277 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-08-01T19:42:32.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.378+0000 7fd72bfff700 1 Processor -- start 2022-08-01T19:42:32.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.378+0000 7fd72bfff700 1 -- start start 2022-08-01T19:42:32.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd72bfff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fd72c0ad750 0x7fd72c061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:32.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd72bfff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd72c1405d0 0x7fd72c140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:32.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd72bfff700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd72c0aee80 con 0x7fd72c0614b0 2022-08-01T19:42:32.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd72bfff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd72c0b70b0 con 0x7fd72c0ad750 2022-08-01T19:42:32.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd72bfff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c0af0b0 con 0x7fd72c1405d0 2022-08-01T19:42:32.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd72affd700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd72c0614b0 0x7fd72c0b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35848/0 (socket says 172.21.15.17:35848) 2022-08-01T19:42:32.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd72affd700 1 -- 172.21.15.17:0/1923085364 learned_addr learned my addr 172.21.15.17:0/1923085364 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:32.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd72a7fc700 1 --2- 172.21.15.17:0/1923085364 >> v2:172.21.15.77:3300/0 conn(0x7fd72c0ad750 0x7fd72c061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:32.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd72b7fe700 1 --2- 172.21.15.17:0/1923085364 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd72c1405d0 0x7fd72c140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:32.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd729ffb700 1 -- 172.21.15.17:0/1923085364 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 755760055 0 0) 0x7fd72c0aee80 con 0x7fd72c0614b0 2022-08-01T19:42:32.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd729ffb700 1 -- 172.21.15.17:0/1923085364 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd710003260 con 0x7fd72c0614b0 2022-08-01T19:42:32.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd729ffb700 1 -- 172.21.15.17:0/1923085364 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 788666762 0 0) 0x7fd710003260 con 0x7fd72c0614b0 2022-08-01T19:42:32.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd729ffb700 1 -- 172.21.15.17:0/1923085364 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd72c1405d0 msgr2=0x7fd72c140a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:32.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd729ffb700 1 --2- 172.21.15.17:0/1923085364 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd72c1405d0 0x7fd72c140a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:32.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd729ffb700 1 -- 172.21.15.17:0/1923085364 >> v2:172.21.15.77:3300/0 conn(0x7fd72c0ad750 msgr2=0x7fd72c061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:32.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd729ffb700 1 --2- 172.21.15.17:0/1923085364 >> v2:172.21.15.77:3300/0 conn(0x7fd72c0ad750 0x7fd72c061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:32.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd729ffb700 1 -- 172.21.15.17:0/1923085364 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd72c0af350 con 0x7fd72c0614b0 2022-08-01T19:42:32.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd729ffb700 1 -- 172.21.15.17:0/1923085364 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7fd71c001660 con 0x7fd72c0614b0 2022-08-01T19:42:32.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd729ffb700 1 -- 172.21.15.17:0/1923085364 >> v1:172.21.15.17:6789/0 conn(0x7fd72c0614b0 legacy=0x7fd72c0b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:32.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd729ffb700 1 --2- 172.21.15.17:0/1923085364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd710003750 0x7fd710003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:32.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd729ffb700 1 --2- 172.21.15.17:0/1923085364 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd710004500 0x7fd710004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:32.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd729ffb700 1 --2- 172.21.15.17:0/1923085364 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd710004e70 0x7fd71000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:32.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd729ffb700 1 -- 172.21.15.17:0/1923085364 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c0b70b0 con 0x7fd710004e70 2022-08-01T19:42:32.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd729ffb700 1 -- 172.21.15.17:0/1923085364 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c0af0b0 con 0x7fd710003750 2022-08-01T19:42:32.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd729ffb700 1 -- 172.21.15.17:0/1923085364 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd71000bc40 con 0x7fd710004500 2022-08-01T19:42:32.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd72b7fe700 1 --2- 172.21.15.17:0/1923085364 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd72c1405d0 0x7fd72c140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:42:32.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd72a7fc700 1 --2- 172.21.15.17:0/1923085364 >> v2:172.21.15.77:3300/0 conn(0x7fd72c0ad750 0x7fd72c061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:42:32.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd72b7fe700 1 --2- 172.21.15.17:0/1923085364 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd710004e70 0x7fd71000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:32.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd72affd700 1 --2- 172.21.15.17:0/1923085364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd710003750 0x7fd710003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:32.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd72a7fc700 1 --2- 172.21.15.17:0/1923085364 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd710004500 0x7fd710004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:32.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd72b7fe700 1 -- 172.21.15.17:0/1923085364 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd710004500 msgr2=0x7fd710004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:32.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd72b7fe700 1 --2- 172.21.15.17:0/1923085364 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd710004500 0x7fd710004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:32.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd72b7fe700 1 -- 172.21.15.17:0/1923085364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd710003750 msgr2=0x7fd710003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:32.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd72b7fe700 1 --2- 172.21.15.17:0/1923085364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd710003750 0x7fd710003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:32.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd72b7fe700 1 -- 172.21.15.17:0/1923085364 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd710003260 con 0x7fd710004e70 2022-08-01T19:42:32.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd72b7fe700 1 --2- 172.21.15.17:0/1923085364 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd710004e70 0x7fd71000b750 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 rx=0x7fd72400c000 tx=0x7fd7240091d0).ready entity=mon.0 client_cookie=45b436895e012cb2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:32.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd729ffb700 1 -- 172.21.15.17:0/1923085364 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd724009700 con 0x7fd710004e70 2022-08-01T19:42:32.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd729ffb700 1 -- 172.21.15.17:0/1923085364 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd72400e6e0 con 0x7fd710004e70 2022-08-01T19:42:32.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.382+0000 7fd729ffb700 1 -- 172.21.15.17:0/1923085364 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd72400ea40 con 0x7fd710004e70 2022-08-01T19:42:32.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.386+0000 7fd72bfff700 1 -- 172.21.15.17:0/1923085364 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd710004e70 msgr2=0x7fd71000b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:32.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.386+0000 7fd72bfff700 1 --2- 172.21.15.17:0/1923085364 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd710004e70 0x7fd71000b750 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 rx=0x7fd72400c000 tx=0x7fd7240091d0).stop 2022-08-01T19:42:32.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.386+0000 7fd72b7fe700 1 -- 172.21.15.17:0/1923085364 reap_dead start 2022-08-01T19:42:32.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.386+0000 7fd72bfff700 1 -- 172.21.15.17:0/1923085364 shutdown_connections 2022-08-01T19:42:32.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.386+0000 7fd72bfff700 1 -- 172.21.15.17:0/1923085364 >> 172.21.15.17:0/1923085364 conn(0x7fd72c066f70 msgr2=0x7fd72c067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:32.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.386+0000 7fd72bfff700 1 -- 172.21.15.17:0/1923085364 shutdown_connections 2022-08-01T19:42:32.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.386+0000 7fd72bfff700 1 -- 172.21.15.17:0/1923085364 wait complete. 2022-08-01T19:42:32.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.386+0000 7fd72bfff700 1 Processor -- start 2022-08-01T19:42:32.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.386+0000 7fd72bfff700 1 -- start start 2022-08-01T19:42:32.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.386+0000 7fd72bfff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd72c0afc60 0x7fd72c155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:32.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.386+0000 7fd72bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd72c0ad1d0 0x7fd72c1560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:32.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.386+0000 7fd72bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd72c153320 0x7fd72c1565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:32.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.386+0000 7fd72bfff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c156ad0 con 0x7fd72c153320 2022-08-01T19:42:32.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.386+0000 7fd72bfff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c152940 con 0x7fd72c0ad1d0 2022-08-01T19:42:32.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.386+0000 7fd72bfff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c152be0 con 0x7fd72c0afc60 2022-08-01T19:42:32.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.386+0000 7fd72b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd72c153320 0x7fd72c1565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:32.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.386+0000 7fd72b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd72c153320 0x7fd72c1565e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43670/0 (socket says 172.21.15.17:43670) 2022-08-01T19:42:32.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.386+0000 7fd72b7fe700 1 -- 172.21.15.17:0/3259277885 learned_addr learned my addr 172.21.15.17:0/3259277885 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:32.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.386+0000 7fd72a7fc700 1 --2- 172.21.15.17:0/3259277885 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd72c0ad1d0 0x7fd72c1560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:32.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.386+0000 7fd72affd700 1 --2- 172.21.15.17:0/3259277885 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd72c0afc60 0x7fd72c155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:32.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.390+0000 7fd72b7fe700 1 -- 172.21.15.17:0/3259277885 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd72c0afc60 msgr2=0x7fd72c155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:32.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.390+0000 7fd72b7fe700 1 --2- 172.21.15.17:0/3259277885 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd72c0afc60 0x7fd72c155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:32.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.390+0000 7fd72b7fe700 1 -- 172.21.15.17:0/3259277885 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd72c0ad1d0 msgr2=0x7fd72c1560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:32.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.390+0000 7fd72b7fe700 1 --2- 172.21.15.17:0/3259277885 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd72c0ad1d0 0x7fd72c1560f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:32.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.390+0000 7fd72b7fe700 1 -- 172.21.15.17:0/3259277885 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd72c152e80 con 0x7fd72c153320 2022-08-01T19:42:32.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.390+0000 7fd72b7fe700 1 --2- 172.21.15.17:0/3259277885 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd72c153320 0x7fd72c1565e0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 rx=0x7fd72c0aa160 tx=0x7fd72401a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:32.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.390+0000 7fd71bfff700 1 -- 172.21.15.17:0/3259277885 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd72401db00 con 0x7fd72c153320 2022-08-01T19:42:32.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.390+0000 7fd71bfff700 1 -- 172.21.15.17:0/3259277885 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd72401dc60 con 0x7fd72c153320 2022-08-01T19:42:32.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.390+0000 7fd72bfff700 1 -- 172.21.15.17:0/3259277885 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd72c152630 con 0x7fd72c153320 2022-08-01T19:42:32.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.390+0000 7fd71bfff700 1 -- 172.21.15.17:0/3259277885 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd72401b450 con 0x7fd72c153320 2022-08-01T19:42:32.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.390+0000 7fd72bfff700 1 -- 172.21.15.17:0/3259277885 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd72c0b0040 con 0x7fd72c153320 2022-08-01T19:42:32.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.390+0000 7fd72bfff700 1 -- 172.21.15.17:0/3259277885 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd72c153010 con 0x7fd72c153320 2022-08-01T19:42:32.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.390+0000 7fd71bfff700 1 -- 172.21.15.17:0/3259277885 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd72401b5b0 con 0x7fd72c153320 2022-08-01T19:42:32.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.390+0000 7fd71bfff700 1 --2- 172.21.15.17:0/3259277885 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd7080539e0 0x7fd708055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:32.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.390+0000 7fd71bfff700 1 -- 172.21.15.17:0/3259277885 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fd72406d4f0 con 0x7fd72c153320 2022-08-01T19:42:32.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.390+0000 7fd72affd700 1 --2- 172.21.15.17:0/3259277885 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd7080539e0 0x7fd708055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:32.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.394+0000 7fd72affd700 1 --2- 172.21.15.17:0/3259277885 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd7080539e0 0x7fd708055ea0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7fd71c005000 tx=0x7fd72c0af4e0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:32.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.398+0000 7fd71bfff700 1 -- 172.21.15.17:0/3259277885 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd7240442f0 con 0x7fd72c153320 2022-08-01T19:42:32.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:32.566+0000 7fd72bfff700 1 -- 172.21.15.17:0/3259277885 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd72c0574e0 con 0x7fd72c153320 2022-08-01T19:42:33.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.262+0000 7fd71bfff700 1 -- 172.21.15.17:0/3259277885 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v60) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd7240478d0 con 0x7fd72c153320 2022-08-01T19:42:33.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.266+0000 7fd72bfff700 1 -- 172.21.15.17:0/3259277885 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd7080539e0 msgr2=0x7fd708055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:33.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.266+0000 7fd72bfff700 1 --2- 172.21.15.17:0/3259277885 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd7080539e0 0x7fd708055ea0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7fd71c005000 tx=0x7fd72c0af4e0).stop 2022-08-01T19:42:33.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.266+0000 7fd72bfff700 1 -- 172.21.15.17:0/3259277885 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd72c153320 msgr2=0x7fd72c1565e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:33.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.266+0000 7fd72bfff700 1 --2- 172.21.15.17:0/3259277885 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd72c153320 0x7fd72c1565e0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 rx=0x7fd72c0aa160 tx=0x7fd72401a000).stop 2022-08-01T19:42:33.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.266+0000 7fd72bfff700 1 -- 172.21.15.17:0/3259277885 shutdown_connections 2022-08-01T19:42:33.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.266+0000 7fd72bfff700 1 --2- 172.21.15.17:0/3259277885 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd7080539e0 0x7fd708055ea0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:33.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.266+0000 7fd72bfff700 1 --2- 172.21.15.17:0/3259277885 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd72c153320 0x7fd72c1565e0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:33.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.266+0000 7fd72bfff700 1 --2- 172.21.15.17:0/3259277885 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd72c0ad1d0 0x7fd72c1560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:33.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.266+0000 7fd72bfff700 1 --2- 172.21.15.17:0/3259277885 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd72c0afc60 0x7fd72c155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:33.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.266+0000 7fd72bfff700 1 -- 172.21.15.17:0/3259277885 >> 172.21.15.17:0/3259277885 conn(0x7fd72c066f70 msgr2=0x7fd72c0b8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:33.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.266+0000 7fd72bfff700 1 -- 172.21.15.17:0/3259277885 shutdown_connections 2022-08-01T19:42:33.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.266+0000 7fd72bfff700 1 -- 172.21.15.17:0/3259277885 wait complete. 2022-08-01T19:42:33.284 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-08-01T19:42:33.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.402+0000 7f94e94aa700 1 Processor -- start 2022-08-01T19:42:33.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.402+0000 7f94e94aa700 1 -- start start 2022-08-01T19:42:33.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.402+0000 7f94e94aa700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f94e40b0250 0x7f94e40b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:33.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.402+0000 7f94e94aa700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f94e40a9d00 0x7f94e40b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:33.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.402+0000 7f94e94aa700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94e40aa520 con 0x7f94e40b20a0 2022-08-01T19:42:33.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.402+0000 7f94e94aa700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94e40ab3f0 con 0x7f94e40b0250 2022-08-01T19:42:33.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.402+0000 7f94e94aa700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e40b0630 con 0x7f94e40a9d00 2022-08-01T19:42:33.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.406+0000 7f94e37fe700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f94e40a9d00 0x7f94e40b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:33.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.406+0000 7f94e2ffd700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f94e40b0250 0x7f94e40b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:33.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.406+0000 7f94e37fe700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f94e40a9d00 0x7f94e40b2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:39778/0 (socket says 172.21.15.17:39778) 2022-08-01T19:42:33.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.406+0000 7f94e37fe700 1 -- 172.21.15.17:0/681346483 learned_addr learned my addr 172.21.15.17:0/681346483 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:33.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.406+0000 7f94e27fc700 1 --1- 172.21.15.17:0/681346483 >> v1:172.21.15.77:6789/0 conn(0x7f94e40b20a0 0x7f94e40aace0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:52668/0 (socket says 172.21.15.17:52668) 2022-08-01T19:42:33.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.406+0000 7f94e37fe700 1 -- 172.21.15.17:0/681346483 >> v2:172.21.15.77:3300/0 conn(0x7f94e40b0250 msgr2=0x7f94e40b1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:33.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.406+0000 7f94e37fe700 1 --2- 172.21.15.17:0/681346483 >> v2:172.21.15.77:3300/0 conn(0x7f94e40b0250 0x7f94e40b1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:33.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.406+0000 7f94e37fe700 1 -- 172.21.15.17:0/681346483 >> v1:172.21.15.77:6789/0 conn(0x7f94e40b20a0 legacy=0x7f94e40aace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:33.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.406+0000 7f94e37fe700 1 -- 172.21.15.17:0/681346483 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e40aa280 con 0x7f94e40a9d00 2022-08-01T19:42:33.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.406+0000 7f94e2ffd700 1 --2- 172.21.15.17:0/681346483 >> v2:172.21.15.77:3300/0 conn(0x7f94e40b0250 0x7f94e40b1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:42:33.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.406+0000 7f94e37fe700 1 --2- 172.21.15.17:0/681346483 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f94e40a9d00 0x7f94e40b2fc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f94dc0032f0 tx=0x7f94dc009290).ready entity=mon.2 client_cookie=89dff0a972e4a677 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:33.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.406+0000 7f94e1ffb700 1 -- 172.21.15.17:0/681346483 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94dc00b8a0 con 0x7f94e40a9d00 2022-08-01T19:42:33.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.406+0000 7f94e1ffb700 1 -- 172.21.15.17:0/681346483 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94dc00ba00 con 0x7f94e40a9d00 2022-08-01T19:42:33.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.406+0000 7f94e1ffb700 1 -- 172.21.15.17:0/681346483 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94dc00bb70 con 0x7f94e40a9d00 2022-08-01T19:42:33.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.406+0000 7f94e94aa700 1 -- 172.21.15.17:0/681346483 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f94e40a9d00 msgr2=0x7f94e40b2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:33.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.406+0000 7f94e94aa700 1 --2- 172.21.15.17:0/681346483 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f94e40a9d00 0x7f94e40b2fc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f94dc0032f0 tx=0x7f94dc009290).stop 2022-08-01T19:42:33.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.406+0000 7f94e94aa700 1 -- 172.21.15.17:0/681346483 shutdown_connections 2022-08-01T19:42:33.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.406+0000 7f94e94aa700 1 --2- 172.21.15.17:0/681346483 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f94e40a9d00 0x7f94e40b2fc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:33.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.406+0000 7f94e94aa700 1 --2- 172.21.15.17:0/681346483 >> v2:172.21.15.77:3300/0 conn(0x7f94e40b0250 0x7f94e40b1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:33.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.406+0000 7f94e94aa700 1 -- 172.21.15.17:0/681346483 >> 172.21.15.17:0/681346483 conn(0x7f94e4060950 msgr2=0x7f94e4060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:33.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.406+0000 7f94e94aa700 1 -- 172.21.15.17:0/681346483 shutdown_connections 2022-08-01T19:42:33.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.406+0000 7f94e94aa700 1 -- 172.21.15.17:0/681346483 wait complete. 2022-08-01T19:42:33.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.410+0000 7f94e94aa700 1 Processor -- start 2022-08-01T19:42:33.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.410+0000 7f94e94aa700 1 -- start start 2022-08-01T19:42:33.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.410+0000 7f94e94aa700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94e40a9d00 0x7f94e40b3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:33.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.410+0000 7f94e94aa700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94e40b0250 0x7f94e413fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:33.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.410+0000 7f94e94aa700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f94e40b20a0 0x7f94e4153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:33.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.410+0000 7f94e94aa700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e41402a0 con 0x7f94e40a9d00 2022-08-01T19:42:33.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.410+0000 7f94e94aa700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e4142a40 con 0x7f94e40b0250 2022-08-01T19:42:33.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.410+0000 7f94e94aa700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e413f910 con 0x7f94e40b20a0 2022-08-01T19:42:33.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.410+0000 7f94e2ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94e40a9d00 0x7f94e40b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:33.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.410+0000 7f94e37fe700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f94e40b20a0 0x7f94e4153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:33.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.410+0000 7f94e27fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94e40b0250 0x7f94e413fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:33.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.410+0000 7f94e37fe700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f94e40b20a0 0x7f94e4153b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:39784/0 (socket says 172.21.15.17:39784) 2022-08-01T19:42:33.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.410+0000 7f94e37fe700 1 -- 172.21.15.17:0/649521650 learned_addr learned my addr 172.21.15.17:0/649521650 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:33.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.410+0000 7f94e37fe700 1 -- 172.21.15.17:0/649521650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94e40b0250 msgr2=0x7f94e413fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:33.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.410+0000 7f94e37fe700 1 --2- 172.21.15.17:0/649521650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94e40b0250 0x7f94e413fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:33.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.410+0000 7f94e37fe700 1 -- 172.21.15.17:0/649521650 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94e40a9d00 msgr2=0x7f94e40b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:33.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.410+0000 7f94e37fe700 1 --2- 172.21.15.17:0/649521650 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94e40a9d00 0x7f94e40b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:33.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.410+0000 7f94e37fe700 1 -- 172.21.15.17:0/649521650 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d800b180 con 0x7f94e40b20a0 2022-08-01T19:42:33.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.410+0000 7f94e27fc700 1 --2- 172.21.15.17:0/649521650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94e40b0250 0x7f94e413fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:42:33.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.410+0000 7f94e37fe700 1 --2- 172.21.15.17:0/649521650 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f94e40b20a0 0x7f94e4153b00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f94dc0032f0 tx=0x7f94dc016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:33.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.410+0000 7f94d3fff700 1 -- 172.21.15.17:0/649521650 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94dc00b8a0 con 0x7f94e40b20a0 2022-08-01T19:42:33.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.410+0000 7f94d3fff700 1 -- 172.21.15.17:0/649521650 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94dc021440 con 0x7f94e40b20a0 2022-08-01T19:42:33.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.410+0000 7f94e94aa700 1 -- 172.21.15.17:0/649521650 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94dc009530 con 0x7f94e40b20a0 2022-08-01T19:42:33.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.410+0000 7f94d3fff700 1 -- 172.21.15.17:0/649521650 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94dc0217a0 con 0x7f94e40b20a0 2022-08-01T19:42:33.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.410+0000 7f94e94aa700 1 -- 172.21.15.17:0/649521650 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94e4153ff0 con 0x7f94e40b20a0 2022-08-01T19:42:33.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.414+0000 7f94e94aa700 1 -- 172.21.15.17:0/649521650 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94e4143260 con 0x7f94e40b20a0 2022-08-01T19:42:33.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.414+0000 7f94d3fff700 1 -- 172.21.15.17:0/649521650 <== mon.2 v2:172.21.15.77:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f94dc027020 con 0x7f94e40b20a0 2022-08-01T19:42:33.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.414+0000 7f94d3fff700 1 --2- 172.21.15.17:0/649521650 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f94c0053a70 0x7f94c0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:33.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.414+0000 7f94d3fff700 1 -- 172.21.15.17:0/649521650 <== mon.2 v2:172.21.15.77:3301/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f94dc06d5f0 con 0x7f94e40b20a0 2022-08-01T19:42:33.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.414+0000 7f94e2ffd700 1 --2- 172.21.15.17:0/649521650 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f94c0053a70 0x7f94c0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:33.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.414+0000 7f94e2ffd700 1 --2- 172.21.15.17:0/649521650 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f94c0053a70 0x7f94c0055f30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f94d800b000 tx=0x7f94d8006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:33.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.422+0000 7f94d3fff700 1 -- 172.21.15.17:0/649521650 <== mon.2 v2:172.21.15.77:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f94dc044360 con 0x7f94e40b20a0 2022-08-01T19:42:33.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:33.550+0000 7f94e94aa700 1 -- 172.21.15.17:0/649521650 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f94e40574e0 con 0x7f94e40b20a0 2022-08-01T19:42:34.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.274+0000 7f94d3fff700 1 -- 172.21.15.17:0/649521650 <== mon.2 v2:172.21.15.77:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v121) v1 ==== 185+0+0 (secure 0 0 0) 0x7f94dc015020 con 0x7f94e40b20a0 2022-08-01T19:42:34.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.278+0000 7f94e94aa700 1 -- 172.21.15.17:0/649521650 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f94c0053a70 msgr2=0x7f94c0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:34.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.278+0000 7f94e94aa700 1 --2- 172.21.15.17:0/649521650 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f94c0053a70 0x7f94c0055f30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f94d800b000 tx=0x7f94d8006df0).stop 2022-08-01T19:42:34.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.278+0000 7f94e94aa700 1 -- 172.21.15.17:0/649521650 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f94e40b20a0 msgr2=0x7f94e4153b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:34.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.278+0000 7f94e94aa700 1 --2- 172.21.15.17:0/649521650 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f94e40b20a0 0x7f94e4153b00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f94dc0032f0 tx=0x7f94dc016000).stop 2022-08-01T19:42:34.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.282+0000 7f94e94aa700 1 -- 172.21.15.17:0/649521650 shutdown_connections 2022-08-01T19:42:34.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.282+0000 7f94e94aa700 1 --2- 172.21.15.17:0/649521650 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f94c0053a70 0x7f94c0055f30 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:34.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.282+0000 7f94e94aa700 1 --2- 172.21.15.17:0/649521650 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f94e40b20a0 0x7f94e4153b00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:34.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.282+0000 7f94e94aa700 1 --2- 172.21.15.17:0/649521650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94e40b0250 0x7f94e413fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:34.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.282+0000 7f94e94aa700 1 --2- 172.21.15.17:0/649521650 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94e40a9d00 0x7f94e40b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:34.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.282+0000 7f94e94aa700 1 -- 172.21.15.17:0/649521650 >> 172.21.15.17:0/649521650 conn(0x7f94e4060950 msgr2=0x7f94e40b5cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:34.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.282+0000 7f94e94aa700 1 -- 172.21.15.17:0/649521650 shutdown_connections 2022-08-01T19:42:34.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.282+0000 7f94e94aa700 1 -- 172.21.15.17:0/649521650 wait complete. 2022-08-01T19:42:34.302 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' removed 2022-08-01T19:42:34.303 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-08-01T19:42:34.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.426+0000 7f096e3aa700 1 Processor -- start 2022-08-01T19:42:34.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.426+0000 7f096e3aa700 1 -- start start 2022-08-01T19:42:34.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.426+0000 7f096e3aa700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09680b01a0 0x7f09680a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:34.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.426+0000 7f096e3aa700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09680a7250 con 0x7f09680afd60 2022-08-01T19:42:34.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.426+0000 7f096e3aa700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09680a7480 con 0x7f09680b2710 2022-08-01T19:42:34.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.426+0000 7f096e3aa700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09680a7720 con 0x7f09680b01a0 2022-08-01T19:42:34.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.426+0000 7f096dba9700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f09680afd60 0x7f0968151d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35886/0 (socket says 172.21.15.17:35886) 2022-08-01T19:42:34.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.426+0000 7f096dba9700 1 -- 172.21.15.17:0/646613677 learned_addr learned my addr 172.21.15.17:0/646613677 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:34.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.426+0000 7f096cba7700 1 --2- 172.21.15.17:0/646613677 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09680b01a0 0x7f09680a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:34.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.426+0000 7f095ffff700 1 -- 172.21.15.17:0/646613677 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3679999925 0 0) 0x7f09680a7250 con 0x7f09680afd60 2022-08-01T19:42:34.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.426+0000 7f095ffff700 1 -- 172.21.15.17:0/646613677 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f094c003260 con 0x7f09680afd60 2022-08-01T19:42:34.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.426+0000 7f095ffff700 1 -- 172.21.15.17:0/646613677 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2905812159 0 0) 0x7f09680a7480 con 0x7f09680b2710 2022-08-01T19:42:34.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.426+0000 7f095ffff700 1 -- 172.21.15.17:0/646613677 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09680a7250 con 0x7f09680b2710 2022-08-01T19:42:34.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.426+0000 7f095ffff700 1 -- 172.21.15.17:0/646613677 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2192594279 0 0) 0x7f094c003260 con 0x7f09680afd60 2022-08-01T19:42:34.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.426+0000 7f095ffff700 1 -- 172.21.15.17:0/646613677 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09680b01a0 msgr2=0x7f09680a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:34.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.426+0000 7f095ffff700 1 --2- 172.21.15.17:0/646613677 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09680b01a0 0x7f09680a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:34.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.426+0000 7f095ffff700 1 -- 172.21.15.17:0/646613677 >> v1:172.21.15.77:6789/0 conn(0x7f09680b2710 legacy=0x7f096814afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:34.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.426+0000 7f095ffff700 1 -- 172.21.15.17:0/646613677 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09680a79c0 con 0x7f09680afd60 2022-08-01T19:42:34.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.426+0000 7f095ffff700 1 -- 172.21.15.17:0/646613677 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7f09600019e0 con 0x7f09680afd60 2022-08-01T19:42:34.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.426+0000 7f095ffff700 1 -- 172.21.15.17:0/646613677 >> v1:172.21.15.17:6789/0 conn(0x7f09680afd60 legacy=0x7f0968151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:34.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.426+0000 7f095ffff700 1 --2- 172.21.15.17:0/646613677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f094c003910 0x7f094c003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:34.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.426+0000 7f095ffff700 1 --2- 172.21.15.17:0/646613677 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09680b01a0 0x7f094c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:34.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.426+0000 7f095ffff700 1 --2- 172.21.15.17:0/646613677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f094c0049f0 0x7f094c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:34.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.426+0000 7f095ffff700 1 -- 172.21.15.17:0/646613677 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09680a7720 con 0x7f094c0049f0 2022-08-01T19:42:34.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.426+0000 7f095ffff700 1 -- 172.21.15.17:0/646613677 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f094c00b800 con 0x7f094c003910 2022-08-01T19:42:34.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.426+0000 7f095ffff700 1 -- 172.21.15.17:0/646613677 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f094c00b970 con 0x7f09680b01a0 2022-08-01T19:42:34.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.426+0000 7f096dba9700 1 --2- 172.21.15.17:0/646613677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f094c0049f0 0x7f094c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:34.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.430+0000 7f096d3a8700 1 --2- 172.21.15.17:0/646613677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f094c003910 0x7f094c003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:34.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.430+0000 7f096cba7700 1 --2- 172.21.15.17:0/646613677 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09680b01a0 0x7f094c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:34.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.430+0000 7f096dba9700 1 -- 172.21.15.17:0/646613677 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09680b01a0 msgr2=0x7f094c004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:34.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.430+0000 7f096dba9700 1 --2- 172.21.15.17:0/646613677 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09680b01a0 0x7f094c004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:34.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.430+0000 7f096dba9700 1 -- 172.21.15.17:0/646613677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f094c003910 msgr2=0x7f094c003d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:34.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.430+0000 7f096dba9700 1 --2- 172.21.15.17:0/646613677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f094c003910 0x7f094c003d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:34.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.430+0000 7f096dba9700 1 -- 172.21.15.17:0/646613677 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f094c003260 con 0x7f094c0049f0 2022-08-01T19:42:34.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.430+0000 7f096cba7700 1 --2- 172.21.15.17:0/646613677 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09680b01a0 0x7f094c004500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:42:34.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.430+0000 7f096dba9700 1 --2- 172.21.15.17:0/646613677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f094c0049f0 0x7f094c004e70 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 rx=0x7f09600017b0 tx=0x7f096806a5f0).ready entity=mon.0 client_cookie=7f6fb15d7e0de87d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:34.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.430+0000 7f095ffff700 1 -- 172.21.15.17:0/646613677 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f096000cbb0 con 0x7f094c0049f0 2022-08-01T19:42:34.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.430+0000 7f095ffff700 1 -- 172.21.15.17:0/646613677 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f096000cd10 con 0x7f094c0049f0 2022-08-01T19:42:34.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.430+0000 7f095ffff700 1 -- 172.21.15.17:0/646613677 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0960005b80 con 0x7f094c0049f0 2022-08-01T19:42:34.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.430+0000 7f096e3aa700 1 -- 172.21.15.17:0/646613677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f094c0049f0 msgr2=0x7f094c004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:34.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.430+0000 7f096e3aa700 1 --2- 172.21.15.17:0/646613677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f094c0049f0 0x7f094c004e70 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 rx=0x7f09600017b0 tx=0x7f096806a5f0).stop 2022-08-01T19:42:34.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.430+0000 7f096dba9700 1 -- 172.21.15.17:0/646613677 reap_dead start 2022-08-01T19:42:34.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.430+0000 7f096e3aa700 1 -- 172.21.15.17:0/646613677 shutdown_connections 2022-08-01T19:42:34.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.430+0000 7f096e3aa700 1 -- 172.21.15.17:0/646613677 >> 172.21.15.17:0/646613677 conn(0x7f0968060310 msgr2=0x7f09680b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:34.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.430+0000 7f096e3aa700 1 -- 172.21.15.17:0/646613677 shutdown_connections 2022-08-01T19:42:34.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.430+0000 7f096e3aa700 1 -- 172.21.15.17:0/646613677 wait complete. 2022-08-01T19:42:34.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.434+0000 7f096e3aa700 1 Processor -- start 2022-08-01T19:42:34.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.434+0000 7f096e3aa700 1 -- start start 2022-08-01T19:42:34.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.434+0000 7f096e3aa700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f094c0049f0 0x7f09680af0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:34.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.434+0000 7f096e3aa700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09680b2f60 0x7f09680b3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:34.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.434+0000 7f096e3aa700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f096814c520 0x7f0968159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:34.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.434+0000 7f096e3aa700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0968156800 con 0x7f094c0049f0 2022-08-01T19:42:34.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.434+0000 7f096e3aa700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096814bb40 con 0x7f09680b2f60 2022-08-01T19:42:34.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.434+0000 7f096d3a8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f094c0049f0 0x7f09680af0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:34.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.434+0000 7f096e3aa700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096814bde0 con 0x7f096814c520 2022-08-01T19:42:34.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.434+0000 7f096d3a8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f094c0049f0 0x7f09680af0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43700/0 (socket says 172.21.15.17:43700) 2022-08-01T19:42:34.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.434+0000 7f096d3a8700 1 -- 172.21.15.17:0/2602037683 learned_addr learned my addr 172.21.15.17:0/2602037683 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:34.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.434+0000 7f096cba7700 1 --2- 172.21.15.17:0/2602037683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09680b2f60 0x7f09680b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:34.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.434+0000 7f096dba9700 1 --2- 172.21.15.17:0/2602037683 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f096814c520 0x7f0968159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:34.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.434+0000 7f096d3a8700 1 -- 172.21.15.17:0/2602037683 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f096814c520 msgr2=0x7f0968159a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:34.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.434+0000 7f096d3a8700 1 --2- 172.21.15.17:0/2602037683 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f096814c520 0x7f0968159a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:34.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.434+0000 7f096d3a8700 1 -- 172.21.15.17:0/2602037683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09680b2f60 msgr2=0x7f09680b3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:34.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.434+0000 7f096d3a8700 1 --2- 172.21.15.17:0/2602037683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09680b2f60 0x7f09680b3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:34.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.434+0000 7f096d3a8700 1 -- 172.21.15.17:0/2602037683 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096814c080 con 0x7f094c0049f0 2022-08-01T19:42:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.438+0000 7f096d3a8700 1 --2- 172.21.15.17:0/2602037683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f094c0049f0 0x7f09680af0e0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 rx=0x7f096400d000 tx=0x7f0964007a00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.438+0000 7f095e7fc700 1 -- 172.21.15.17:0/2602037683 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0964011780 con 0x7f094c0049f0 2022-08-01T19:42:34.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.438+0000 7f095e7fc700 1 -- 172.21.15.17:0/2602037683 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09640118e0 con 0x7f094c0049f0 2022-08-01T19:42:34.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.438+0000 7f096e3aa700 1 -- 172.21.15.17:0/2602037683 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f096814d670 con 0x7f094c0049f0 2022-08-01T19:42:34.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.438+0000 7f095e7fc700 1 -- 172.21.15.17:0/2602037683 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0964011c40 con 0x7f094c0049f0 2022-08-01T19:42:34.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.438+0000 7f096e3aa700 1 -- 172.21.15.17:0/2602037683 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0968156480 con 0x7f094c0049f0 2022-08-01T19:42:34.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.438+0000 7f096e3aa700 1 -- 172.21.15.17:0/2602037683 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f096814c210 con 0x7f094c0049f0 2022-08-01T19:42:34.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.438+0000 7f095e7fc700 1 -- 172.21.15.17:0/2602037683 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f096401b030 con 0x7f094c0049f0 2022-08-01T19:42:34.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.438+0000 7f095e7fc700 1 --2- 172.21.15.17:0/2602037683 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0950053b30 0x7f0950055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:34.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.438+0000 7f095e7fc700 1 -- 172.21.15.17:0/2602037683 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f096406bf90 con 0x7f094c0049f0 2022-08-01T19:42:34.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.438+0000 7f096cba7700 1 --2- 172.21.15.17:0/2602037683 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0950053b30 0x7f0950055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:34.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.438+0000 7f096cba7700 1 --2- 172.21.15.17:0/2602037683 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0950053b30 0x7f0950055ff0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7f095800b000 tx=0x7f0958008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:34.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.446+0000 7f095e7fc700 1 -- 172.21.15.17:0/2602037683 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0964042e50 con 0x7f094c0049f0 2022-08-01T19:42:34.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:34.574+0000 7f096e3aa700 1 -- 172.21.15.17:0/2602037683 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f09680574e0 con 0x7f094c0049f0 2022-08-01T19:42:35.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:35.290+0000 7f095e7fc700 1 -- 172.21.15.17:0/2602037683 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v122) v1 ==== 133+0+0 (secure 0 0 0) 0x7f0964019070 con 0x7f094c0049f0 2022-08-01T19:42:35.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:35.294+0000 7f096e3aa700 1 -- 172.21.15.17:0/2602037683 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0950053b30 msgr2=0x7f0950055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:35.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:35.294+0000 7f096e3aa700 1 --2- 172.21.15.17:0/2602037683 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0950053b30 0x7f0950055ff0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7f095800b000 tx=0x7f0958008040).stop 2022-08-01T19:42:35.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:35.294+0000 7f096e3aa700 1 -- 172.21.15.17:0/2602037683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f094c0049f0 msgr2=0x7f09680af0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:35.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:35.294+0000 7f096e3aa700 1 --2- 172.21.15.17:0/2602037683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f094c0049f0 0x7f09680af0e0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 rx=0x7f096400d000 tx=0x7f0964007a00).stop 2022-08-01T19:42:35.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:35.294+0000 7f096e3aa700 1 -- 172.21.15.17:0/2602037683 shutdown_connections 2022-08-01T19:42:35.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:35.294+0000 7f096e3aa700 1 --2- 172.21.15.17:0/2602037683 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0950053b30 0x7f0950055ff0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:35.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:35.294+0000 7f096e3aa700 1 --2- 172.21.15.17:0/2602037683 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f096814c520 0x7f0968159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:35.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:35.294+0000 7f096e3aa700 1 --2- 172.21.15.17:0/2602037683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09680b2f60 0x7f09680b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:35.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:35.294+0000 7f096e3aa700 1 --2- 172.21.15.17:0/2602037683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f094c0049f0 0x7f09680af0e0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:35.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:35.294+0000 7f096e3aa700 1 -- 172.21.15.17:0/2602037683 >> 172.21.15.17:0/2602037683 conn(0x7f0968060310 msgr2=0x7f0968145480 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:35.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:35.294+0000 7f096e3aa700 1 -- 172.21.15.17:0/2602037683 shutdown_connections 2022-08-01T19:42:35.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:35.294+0000 7f096e3aa700 1 -- 172.21.15.17:0/2602037683 wait complete. 2022-08-01T19:42:35.299 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' created 2022-08-01T19:42:35.319 DEBUG:teuthology.orchestra.run.smithi085:> sudo rados -p cephfs_metadata put key /etc/fstab 2022-08-01T19:42:35.374 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.366+0000 7f6892e62b80 1 Processor -- start 2022-08-01T19:42:35.375 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.366+0000 7f6892e62b80 1 -- start start 2022-08-01T19:42:35.375 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.366+0000 7f6892e62b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560572470400 0x56057247eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:35.375 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6892e62b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560572470f90 0x560572478110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:35.376 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6892e62b80 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x5605724762b0 0x560572478600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:35.376 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6892e62b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56057236f810 con 0x560572470f90 2022-08-01T19:42:35.376 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6892e62b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x56057235f6f0 con 0x560572470400 2022-08-01T19:42:35.376 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6892e62b80 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x560572485660 con 0x5605724762b0 2022-08-01T19:42:35.377 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6890a9e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560572470f90 0x560572478110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:35.377 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f689129f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560572470400 0x56057247eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:35.377 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6890a9e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560572470f90 0x560572478110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.85:41774/0 (socket says 172.21.15.85:41774) 2022-08-01T19:42:35.377 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6891aa0700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x5605724762b0 0x560572478600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:35.378 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6890a9e700 1 -- 172.21.15.85:0/1657665373 learned_addr learned my addr 172.21.15.85:0/1657665373 (peer_addr_for_me v2:172.21.15.85:0/0) 2022-08-01T19:42:35.380 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f689129f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560572470400 0x56057247eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.85:41036/0 (socket says 172.21.15.85:41036) 2022-08-01T19:42:35.395 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6890a9e700 1 -- 172.21.15.85:0/1657665373 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x5605724762b0 msgr2=0x560572478600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:35.395 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6890a9e700 1 --2- 172.21.15.85:0/1657665373 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x5605724762b0 0x560572478600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:35.396 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6890a9e700 1 -- 172.21.15.85:0/1657665373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560572470400 msgr2=0x56057247eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:35.396 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6890a9e700 1 --2- 172.21.15.85:0/1657665373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560572470400 0x56057247eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:35.397 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6890a9e700 1 -- 172.21.15.85:0/1657665373 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6884005cd0 con 0x560572470f90 2022-08-01T19:42:35.397 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f689129f700 1 --2- 172.21.15.85:0/1657665373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560572470400 0x56057247eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:42:35.398 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6890a9e700 1 --2- 172.21.15.85:0/1657665373 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560572470f90 0x560572478110 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 rx=0x7f687c001550 tx=0x7f687c006a80).ready entity=mon.0 client_cookie=e034b56ec817b58e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:35.398 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f688affd700 1 -- 172.21.15.85:0/1657665373 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f687c006110 con 0x560572470f90 2022-08-01T19:42:35.398 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f688affd700 1 -- 172.21.15.85:0/1657665373 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f687c006270 con 0x560572470f90 2022-08-01T19:42:35.399 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f688affd700 1 -- 172.21.15.85:0/1657665373 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f687c004730 con 0x560572470f90 2022-08-01T19:42:35.399 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6892e62b80 1 -- 172.21.15.85:0/1657665373 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560572470f90 msgr2=0x560572478110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:35.399 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6892e62b80 1 --2- 172.21.15.85:0/1657665373 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560572470f90 0x560572478110 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 rx=0x7f687c001550 tx=0x7f687c006a80).stop 2022-08-01T19:42:35.399 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6892e62b80 1 -- 172.21.15.85:0/1657665373 shutdown_connections 2022-08-01T19:42:35.400 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6892e62b80 1 --2- 172.21.15.85:0/1657665373 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x5605724762b0 0x560572478600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:35.400 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6892e62b80 1 --2- 172.21.15.85:0/1657665373 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560572470f90 0x560572478110 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:35.400 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6892e62b80 1 --2- 172.21.15.85:0/1657665373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560572470400 0x56057247eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:35.401 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6892e62b80 1 -- 172.21.15.85:0/1657665373 >> 172.21.15.85:0/1657665373 conn(0x5605723e3640 msgr2=0x560572471c60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:35.401 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6892e62b80 1 -- 172.21.15.85:0/1657665373 shutdown_connections 2022-08-01T19:42:35.401 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6892e62b80 1 -- 172.21.15.85:0/1657665373 wait complete. 2022-08-01T19:42:35.402 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6892e62b80 1 Processor -- start 2022-08-01T19:42:35.402 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6892e62b80 1 -- start start 2022-08-01T19:42:35.403 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6892e62b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560572470400 0x56057247eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:35.403 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6892e62b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560572470f90 0x560572479d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:35.403 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6892e62b80 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x5605724762b0 0x56057247a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:35.403 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6892e62b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x560572473070 con 0x560572470f90 2022-08-01T19:42:35.404 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6892e62b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x56057247a7d0 con 0x560572470400 2022-08-01T19:42:35.404 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6892e62b80 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x56057247a990 con 0x5605724762b0 2022-08-01T19:42:35.404 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6890a9e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560572470f90 0x560572479d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:35.404 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6890a9e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560572470f90 0x560572479d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.85:41782/0 (socket says 172.21.15.85:41782) 2022-08-01T19:42:35.405 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6890a9e700 1 -- 172.21.15.85:0/1864620802 learned_addr learned my addr 172.21.15.85:0/1864620802 (peer_addr_for_me v2:172.21.15.85:0/0) 2022-08-01T19:42:35.405 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f689129f700 1 --2- 172.21.15.85:0/1864620802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560572470400 0x56057247eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:35.405 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.370+0000 7f6891aa0700 1 --2- 172.21.15.85:0/1864620802 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x5605724762b0 0x56057247a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:35.409 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.374+0000 7f6890a9e700 1 -- 172.21.15.85:0/1864620802 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x5605724762b0 msgr2=0x56057247a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:35.410 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.374+0000 7f6890a9e700 1 --2- 172.21.15.85:0/1864620802 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x5605724762b0 0x56057247a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:35.410 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.374+0000 7f6890a9e700 1 -- 172.21.15.85:0/1864620802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560572470400 msgr2=0x56057247eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:35.410 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.374+0000 7f6890a9e700 1 --2- 172.21.15.85:0/1864620802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560572470400 0x56057247eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:35.411 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.374+0000 7f6890a9e700 1 -- 172.21.15.85:0/1864620802 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605723845d0 con 0x560572470f90 2022-08-01T19:42:35.411 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.374+0000 7f689129f700 1 --2- 172.21.15.85:0/1864620802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560572470400 0x56057247eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:42:35.412 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.374+0000 7f6890a9e700 1 --2- 172.21.15.85:0/1864620802 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560572470f90 0x560572479d00 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 rx=0x7f687c001550 tx=0x7f687c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:35.412 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.374+0000 7f688a7fc700 1 -- 172.21.15.85:0/1864620802 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f687c013810 con 0x560572470f90 2022-08-01T19:42:35.413 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.374+0000 7f688a7fc700 1 -- 172.21.15.85:0/1864620802 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f687c013970 con 0x560572470f90 2022-08-01T19:42:35.413 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.374+0000 7f6892e62b80 1 -- 172.21.15.85:0/1864620802 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6880001e10 con 0x560572470f90 2022-08-01T19:42:35.413 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.374+0000 7f688a7fc700 1 -- 172.21.15.85:0/1864620802 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f687c013cd0 con 0x560572470f90 2022-08-01T19:42:35.414 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.374+0000 7f6892e62b80 1 -- 172.21.15.85:0/1864620802 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560572358270 con 0x560572470f90 2022-08-01T19:42:35.414 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.378+0000 7f688a7fc700 1 -- 172.21.15.85:0/1864620802 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f687c021470 con 0x560572470f90 2022-08-01T19:42:35.415 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.378+0000 7f688a7fc700 1 --2- 172.21.15.85:0/1864620802 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f68740525f0 0x7f6874054ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:35.415 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.378+0000 7f688a7fc700 1 -- 172.21.15.85:0/1864620802 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6422+0+0 (secure 0 0 0) 0x7f687c01b040 con 0x560572470f90 2022-08-01T19:42:35.415 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.378+0000 7f689129f700 1 --2- 172.21.15.85:0/1864620802 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f68740525f0 0x7f6874054ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:35.416 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.378+0000 7f6892e62b80 1 --2- 172.21.15.85:0/1864620802 >> [v2:172.21.15.17:6800/23008,v1:172.21.15.17:6801/23008] conn(0x56057248d9c0 0x56057248fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:35.416 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.378+0000 7f6892e62b80 1 -- 172.21.15.85:0/1864620802 --> [v2:172.21.15.17:6800/23008,v1:172.21.15.17:6801/23008] -- osd_op(unknown.0.0:1 23.0 23:0620e6f9:::key:head [writefull 0~617 in=617b] snapc 0=[] ondisk+write+known_if_redirected e122) v8 -- 0x560572490340 con 0x56057248d9c0 2022-08-01T19:42:35.416 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.378+0000 7f6891aa0700 1 --2- 172.21.15.85:0/1864620802 >> [v2:172.21.15.17:6800/23008,v1:172.21.15.17:6801/23008] conn(0x56057248d9c0 0x56057248fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:35.417 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.378+0000 7f689129f700 1 --2- 172.21.15.85:0/1864620802 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f68740525f0 0x7f6874054ab0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f6884016000 tx=0x7f6884014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:35.418 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:35.378+0000 7f6891aa0700 1 --2- 172.21.15.85:0/1864620802 >> [v2:172.21.15.17:6800/23008,v1:172.21.15.17:6801/23008] conn(0x56057248d9c0 0x56057248fdb0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:36.570 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:36.566+0000 7f688a7fc700 1 -- 172.21.15.85:0/1864620802 <== osd.0 v2:172.21.15.17:6800/23008 1 ==== osd_map(123..123 src has 1..123) v4 ==== 320+0+0 (crc 0 0 0) 0x7f688c00f8e0 con 0x56057248d9c0 2022-08-01T19:42:36.585 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:36.570+0000 7f6891aa0700 1 -- 172.21.15.85:0/1864620802 <== osd.0 v2:172.21.15.17:6800/23008 2 ==== osd_op_reply(1 key [writefull 0~617] v123'1 uv1 ondisk = 0) v8 ==== 147+0+0 (crc 0 0 0) 0x7f688c00faa0 con 0x56057248d9c0 2022-08-01T19:42:36.590 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:36.570+0000 7f6892e62b80 1 -- 172.21.15.85:0/1864620802 >> [v2:172.21.15.17:6800/23008,v1:172.21.15.17:6801/23008] conn(0x56057248d9c0 msgr2=0x56057248fdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:36.590 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:36.570+0000 7f6892e62b80 1 --2- 172.21.15.85:0/1864620802 >> [v2:172.21.15.17:6800/23008,v1:172.21.15.17:6801/23008] conn(0x56057248d9c0 0x56057248fdb0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:36.590 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:36.570+0000 7f6892e62b80 1 -- 172.21.15.85:0/1864620802 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f68740525f0 msgr2=0x7f6874054ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:36.596 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:36.570+0000 7f6892e62b80 1 --2- 172.21.15.85:0/1864620802 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f68740525f0 0x7f6874054ab0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f6884016000 tx=0x7f6884014000).stop 2022-08-01T19:42:36.596 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:36.570+0000 7f6892e62b80 1 -- 172.21.15.85:0/1864620802 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560572470f90 msgr2=0x560572479d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:36.597 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:36.570+0000 7f6892e62b80 1 --2- 172.21.15.85:0/1864620802 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560572470f90 0x560572479d00 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 rx=0x7f687c001550 tx=0x7f687c014000).stop 2022-08-01T19:42:36.597 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:36.570+0000 7f6891aa0700 1 -- 172.21.15.85:0/1864620802 reap_dead start 2022-08-01T19:42:36.597 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:36.570+0000 7f6892e62b80 1 -- 172.21.15.85:0/1864620802 shutdown_connections 2022-08-01T19:42:36.597 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:36.570+0000 7f6892e62b80 1 -- 172.21.15.85:0/1864620802 >> 172.21.15.85:0/1864620802 conn(0x5605723e3640 msgr2=0x560572471c60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:36.598 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:36.570+0000 7f6892e62b80 1 -- 172.21.15.85:0/1864620802 shutdown_connections 2022-08-01T19:42:36.598 INFO:teuthology.orchestra.run.smithi085.stderr:2022-08-01T19:42:36.570+0000 7f6892e62b80 1 -- 172.21.15.85:0/1864620802 wait complete. 2022-08-01T19:42:36.598 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2022-08-01T19:42:36.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.706+0000 7f5cd8c74700 1 Processor -- start 2022-08-01T19:42:36.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.706+0000 7f5cd8c74700 1 -- start start 2022-08-01T19:42:36.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.710+0000 7f5cd8c74700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5cd40ad4b0 0x7f5cd40af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:36.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.710+0000 7f5cd8c74700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cd40aff30 con 0x7f5cd40af450 2022-08-01T19:42:36.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.710+0000 7f5cd8c74700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cd405c7a0 con 0x7f5cd40ad9e0 2022-08-01T19:42:36.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.710+0000 7f5cd8c74700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cd4140990 con 0x7f5cd40ad4b0 2022-08-01T19:42:36.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.710+0000 7f5cd2ffd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5cd40ad4b0 0x7f5cd40af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:36.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.710+0000 7f5cd37fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5cd40af450 0x7f5cd41432e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35902/0 (socket says 172.21.15.17:35902) 2022-08-01T19:42:36.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.710+0000 7f5cd37fe700 1 -- 172.21.15.17:0/2836524179 learned_addr learned my addr 172.21.15.17:0/2836524179 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:36.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.710+0000 7f5cd2ffd700 1 -- 172.21.15.17:0/2836524179 >> v1:172.21.15.77:6789/0 conn(0x7f5cd40ad9e0 legacy=0x7f5cd40a8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:36.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.710+0000 7f5cd2ffd700 1 -- 172.21.15.17:0/2836524179 >> v1:172.21.15.17:6789/0 conn(0x7f5cd40af450 legacy=0x7f5cd41432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:36.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.710+0000 7f5cd2ffd700 1 -- 172.21.15.17:0/2836524179 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd41439f0 con 0x7f5cd40ad4b0 2022-08-01T19:42:36.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.710+0000 7f5cd2ffd700 1 --2- 172.21.15.17:0/2836524179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cd40ad4b0 0x7f5cd40af830 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 rx=0x7f5cd4063780 tx=0x7f5cc8006e80).ready entity=mon.0 client_cookie=5015175df10f33ea server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:36.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.710+0000 7f5cd27fc700 1 -- 172.21.15.17:0/2836524179 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5cc800bba0 con 0x7f5cd40ad4b0 2022-08-01T19:42:36.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.710+0000 7f5cd27fc700 1 -- 172.21.15.17:0/2836524179 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc8000e90 con 0x7f5cd40ad4b0 2022-08-01T19:42:36.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.710+0000 7f5cd27fc700 1 -- 172.21.15.17:0/2836524179 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5cc800de70 con 0x7f5cd40ad4b0 2022-08-01T19:42:36.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.710+0000 7f5cd8c74700 1 -- 172.21.15.17:0/2836524179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cd40ad4b0 msgr2=0x7f5cd40af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:36.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.710+0000 7f5cd8c74700 1 --2- 172.21.15.17:0/2836524179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cd40ad4b0 0x7f5cd40af830 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 rx=0x7f5cd4063780 tx=0x7f5cc8006e80).stop 2022-08-01T19:42:36.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.710+0000 7f5cd8c74700 1 -- 172.21.15.17:0/2836524179 shutdown_connections 2022-08-01T19:42:36.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.710+0000 7f5cd8c74700 1 --2- 172.21.15.17:0/2836524179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cd40ad4b0 0x7f5cd40af830 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:36.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.710+0000 7f5cd8c74700 1 -- 172.21.15.17:0/2836524179 >> 172.21.15.17:0/2836524179 conn(0x7f5cd40612d0 msgr2=0x7f5cd4062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:36.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.710+0000 7f5cd8c74700 1 -- 172.21.15.17:0/2836524179 shutdown_connections 2022-08-01T19:42:36.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.710+0000 7f5cd8c74700 1 -- 172.21.15.17:0/2836524179 wait complete. 2022-08-01T19:42:36.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.714+0000 7f5cd8c74700 1 Processor -- start 2022-08-01T19:42:36.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.714+0000 7f5cd8c74700 1 -- start start 2022-08-01T19:42:36.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.714+0000 7f5cd8c74700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5cd40ad4b0 0x7f5cd40aa650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:36.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.714+0000 7f5cd8c74700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cd40ad9e0 0x7f5cd413e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:36.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.714+0000 7f5cd8c74700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5cd40af450 0x7f5cd40b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:36.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.714+0000 7f5cd8c74700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd40b3bf0 con 0x7f5cd40ad9e0 2022-08-01T19:42:36.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.714+0000 7f5cd8c74700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd40a9f80 con 0x7f5cd40ad4b0 2022-08-01T19:42:36.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.714+0000 7f5cd8c74700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd40b4490 con 0x7f5cd40af450 2022-08-01T19:42:36.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.714+0000 7f5cd2ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cd40ad9e0 0x7f5cd413e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:36.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.714+0000 7f5cd2ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cd40ad9e0 0x7f5cd413e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43714/0 (socket says 172.21.15.17:43714) 2022-08-01T19:42:36.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.714+0000 7f5cd2ffd700 1 -- 172.21.15.17:0/3454920223 learned_addr learned my addr 172.21.15.17:0/3454920223 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:36.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.714+0000 7f5cd37fe700 1 --2- 172.21.15.17:0/3454920223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5cd40ad4b0 0x7f5cd40aa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:36.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.714+0000 7f5cd3fff700 1 --2- 172.21.15.17:0/3454920223 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5cd40af450 0x7f5cd40b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:36.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.714+0000 7f5cd2ffd700 1 -- 172.21.15.17:0/3454920223 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5cd40af450 msgr2=0x7f5cd40b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:36.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.714+0000 7f5cd2ffd700 1 --2- 172.21.15.17:0/3454920223 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5cd40af450 0x7f5cd40b5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:36.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.714+0000 7f5cd2ffd700 1 -- 172.21.15.17:0/3454920223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5cd40ad4b0 msgr2=0x7f5cd40aa650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:36.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.714+0000 7f5cd2ffd700 1 --2- 172.21.15.17:0/3454920223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5cd40ad4b0 0x7f5cd40aa650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:36.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.714+0000 7f5cd2ffd700 1 -- 172.21.15.17:0/3454920223 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc800d410 con 0x7f5cd40ad9e0 2022-08-01T19:42:36.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.714+0000 7f5cd2ffd700 1 --2- 172.21.15.17:0/3454920223 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cd40ad9e0 0x7f5cd413e340 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 rx=0x7f5cd4063780 tx=0x7f5cd4061990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:36.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.714+0000 7f5cd0ff9700 1 -- 172.21.15.17:0/3454920223 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5cc800de70 con 0x7f5cd40ad9e0 2022-08-01T19:42:36.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.714+0000 7f5cd0ff9700 1 -- 172.21.15.17:0/3454920223 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc8024450 con 0x7f5cd40ad9e0 2022-08-01T19:42:36.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.714+0000 7f5cd8c74700 1 -- 172.21.15.17:0/3454920223 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd40aa1b0 con 0x7f5cd40ad9e0 2022-08-01T19:42:36.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.714+0000 7f5cd0ff9700 1 -- 172.21.15.17:0/3454920223 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5cc80247b0 con 0x7f5cd40ad9e0 2022-08-01T19:42:36.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.714+0000 7f5cd8c74700 1 -- 172.21.15.17:0/3454920223 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd4142250 con 0x7f5cd40ad9e0 2022-08-01T19:42:36.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.718+0000 7f5cd8c74700 1 -- 172.21.15.17:0/3454920223 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cd40aa340 con 0x7f5cd40ad9e0 2022-08-01T19:42:36.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.718+0000 7f5cd0ff9700 1 -- 172.21.15.17:0/3454920223 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5cc8021070 con 0x7f5cd40ad9e0 2022-08-01T19:42:36.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.718+0000 7f5cd0ff9700 1 --2- 172.21.15.17:0/3454920223 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5cb4053a70 0x7f5cb4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:36.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.718+0000 7f5cd0ff9700 1 -- 172.21.15.17:0/3454920223 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6422+0+0 (secure 0 0 0) 0x7f5cc806fb00 con 0x7f5cd40ad9e0 2022-08-01T19:42:36.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.718+0000 7f5cd37fe700 1 --2- 172.21.15.17:0/3454920223 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5cb4053a70 0x7f5cb4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:36.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.718+0000 7f5cd37fe700 1 --2- 172.21.15.17:0/3454920223 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5cb4053a70 0x7f5cb4055f30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f5cc400d000 tx=0x7f5cc4007be0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:36.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.726+0000 7f5cd0ff9700 1 -- 172.21.15.17:0/3454920223 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5cc8071e60 con 0x7f5cd40ad9e0 2022-08-01T19:42:36.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.898+0000 7f5cd8c74700 1 -- 172.21.15.17:0/3454920223 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7f5cd40574e0 con 0x7f5cd40ad9e0 2022-08-01T19:42:36.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.898+0000 7f5cd0ff9700 1 -- 172.21.15.17:0/3454920223 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1588 (secure 0 0 0) 0x7f5cc8046860 con 0x7f5cd40ad9e0 2022-08-01T19:42:36.901 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:42:36.902 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:42:36.902 INFO:teuthology.orchestra.run.smithi017.stdout: "stats": { 2022-08-01T19:42:36.903 INFO:teuthology.orchestra.run.smithi017.stdout: "total_bytes": 773094113280, 2022-08-01T19:42:36.903 INFO:teuthology.orchestra.run.smithi017.stdout: "total_avail_bytes": 764492066816, 2022-08-01T19:42:36.904 INFO:teuthology.orchestra.run.smithi017.stdout: "total_used_bytes": 12111872, 2022-08-01T19:42:36.904 INFO:teuthology.orchestra.run.smithi017.stdout: "total_used_raw_bytes": 8602046464, 2022-08-01T19:42:36.905 INFO:teuthology.orchestra.run.smithi017.stdout: "total_used_raw_ratio": 0.011126778088510036, 2022-08-01T19:42:36.905 INFO:teuthology.orchestra.run.smithi017.stdout: "num_osds": 8, 2022-08-01T19:42:36.905 INFO:teuthology.orchestra.run.smithi017.stdout: "num_per_pool_osds": 8, 2022-08-01T19:42:36.916 INFO:teuthology.orchestra.run.smithi017.stdout: "num_per_pool_omap_osds": 8 2022-08-01T19:42:36.916 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:36.917 INFO:teuthology.orchestra.run.smithi017.stdout: "stats_by_class": { 2022-08-01T19:42:36.917 INFO:teuthology.orchestra.run.smithi017.stdout: "ssd": { 2022-08-01T19:42:36.918 INFO:teuthology.orchestra.run.smithi017.stdout: "total_bytes": 773094113280, 2022-08-01T19:42:36.918 INFO:teuthology.orchestra.run.smithi017.stdout: "total_avail_bytes": 764492066816, 2022-08-01T19:42:36.918 INFO:teuthology.orchestra.run.smithi017.stdout: "total_used_bytes": 12111872, 2022-08-01T19:42:36.918 INFO:teuthology.orchestra.run.smithi017.stdout: "total_used_raw_bytes": 8602046464, 2022-08-01T19:42:36.919 INFO:teuthology.orchestra.run.smithi017.stdout: "total_used_raw_ratio": 0.011126778088510036 2022-08-01T19:42:36.919 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:36.919 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:36.920 INFO:teuthology.orchestra.run.smithi017.stdout: "pools": [ 2022-08-01T19:42:36.921 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:36.921 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "device_health_metrics", 2022-08-01T19:42:36.921 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 1, 2022-08-01T19:42:36.922 INFO:teuthology.orchestra.run.smithi017.stdout: "stats": { 2022-08-01T19:42:36.922 INFO:teuthology.orchestra.run.smithi017.stdout: "stored": 0, 2022-08-01T19:42:36.922 INFO:teuthology.orchestra.run.smithi017.stdout: "objects": 2, 2022-08-01T19:42:36.923 INFO:teuthology.orchestra.run.smithi017.stdout: "kb_used": 0, 2022-08-01T19:42:36.923 INFO:teuthology.orchestra.run.smithi017.stdout: "bytes_used": 0, 2022-08-01T19:42:36.923 INFO:teuthology.orchestra.run.smithi017.stdout: "percent_used": 0, 2022-08-01T19:42:36.923 INFO:teuthology.orchestra.run.smithi017.stdout: "max_avail": 362918576128 2022-08-01T19:42:36.924 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:36.924 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:36.925 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:36.926 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rbd", 2022-08-01T19:42:36.927 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 2, 2022-08-01T19:42:36.928 INFO:teuthology.orchestra.run.smithi017.stdout: "stats": { 2022-08-01T19:42:36.928 INFO:teuthology.orchestra.run.smithi017.stdout: "stored": 0, 2022-08-01T19:42:36.928 INFO:teuthology.orchestra.run.smithi017.stdout: "objects": 0, 2022-08-01T19:42:36.929 INFO:teuthology.orchestra.run.smithi017.stdout: "kb_used": 0, 2022-08-01T19:42:36.929 INFO:teuthology.orchestra.run.smithi017.stdout: "bytes_used": 0, 2022-08-01T19:42:36.931 INFO:teuthology.orchestra.run.smithi017.stdout: "percent_used": 0, 2022-08-01T19:42:36.931 INFO:teuthology.orchestra.run.smithi017.stdout: "max_avail": 362918576128 2022-08-01T19:42:36.931 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:36.932 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:36.932 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:36.933 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "cephfs_data", 2022-08-01T19:42:36.933 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 22, 2022-08-01T19:42:36.934 INFO:teuthology.orchestra.run.smithi017.stdout: "stats": { 2022-08-01T19:42:36.934 INFO:teuthology.orchestra.run.smithi017.stdout: "stored": 0, 2022-08-01T19:42:36.935 INFO:teuthology.orchestra.run.smithi017.stdout: "objects": 0, 2022-08-01T19:42:36.936 INFO:teuthology.orchestra.run.smithi017.stdout: "kb_used": 0, 2022-08-01T19:42:36.936 INFO:teuthology.orchestra.run.smithi017.stdout: "bytes_used": 0, 2022-08-01T19:42:36.937 INFO:teuthology.orchestra.run.smithi017.stdout: "percent_used": 0, 2022-08-01T19:42:36.937 INFO:teuthology.orchestra.run.smithi017.stdout: "max_avail": 362918576128 2022-08-01T19:42:36.937 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:36.938 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:36.938 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:36.939 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:42:36.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.898+0000 7f5cd8c74700 1 -- 172.21.15.17:0/3454920223 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5cb4053a70 msgr2=0x7f5cb4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:36.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.898+0000 7f5cd8c74700 1 --2- 172.21.15.17:0/3454920223 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5cb4053a70 0x7f5cb4055f30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f5cc400d000 tx=0x7f5cc4007be0).stop 2022-08-01T19:42:36.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.898+0000 7f5cd8c74700 1 -- 172.21.15.17:0/3454920223 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cd40ad9e0 msgr2=0x7f5cd413e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:36.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.898+0000 7f5cd8c74700 1 --2- 172.21.15.17:0/3454920223 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cd40ad9e0 0x7f5cd413e340 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 rx=0x7f5cd4063780 tx=0x7f5cd4061990).stop 2022-08-01T19:42:36.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.902+0000 7f5cd8c74700 1 -- 172.21.15.17:0/3454920223 shutdown_connections 2022-08-01T19:42:36.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.902+0000 7f5cd8c74700 1 --2- 172.21.15.17:0/3454920223 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5cb4053a70 0x7f5cb4055f30 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:36.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.902+0000 7f5cd8c74700 1 --2- 172.21.15.17:0/3454920223 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5cd40af450 0x7f5cd40b5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:36.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.902+0000 7f5cd8c74700 1 --2- 172.21.15.17:0/3454920223 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cd40ad9e0 0x7f5cd413e340 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:36.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.902+0000 7f5cd8c74700 1 --2- 172.21.15.17:0/3454920223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5cd40ad4b0 0x7f5cd40aa650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:36.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.902+0000 7f5cd8c74700 1 -- 172.21.15.17:0/3454920223 >> 172.21.15.17:0/3454920223 conn(0x7f5cd40612d0 msgr2=0x7f5cd4137bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:36.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.902+0000 7f5cd8c74700 1 -- 172.21.15.17:0/3454920223 shutdown_connections 2022-08-01T19:42:36.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:36.902+0000 7f5cd8c74700 1 -- 172.21.15.17:0/3454920223 wait complete. 2022-08-01T19:42:36.960 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30)... 2022-08-01T19:42:41.961 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2022-08-01T19:42:42.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.073+0000 7f987a042700 1 Processor -- start 2022-08-01T19:42:42.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.073+0000 7f987a042700 1 -- start start 2022-08-01T19:42:42.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.073+0000 7f987a042700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98740b01a0 0x7f98740a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:42.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.073+0000 7f987a042700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98740a7250 con 0x7f98740afd60 2022-08-01T19:42:42.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.073+0000 7f987a042700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98740a7480 con 0x7f98740b2710 2022-08-01T19:42:42.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.073+0000 7f987a042700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98740a7720 con 0x7f98740b01a0 2022-08-01T19:42:42.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.073+0000 7f9879841700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f98740afd60 0x7f9874151d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35920/0 (socket says 172.21.15.17:35920) 2022-08-01T19:42:42.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.073+0000 7f9879841700 1 -- 172.21.15.17:0/788443488 learned_addr learned my addr 172.21.15.17:0/788443488 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:42.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.073+0000 7f9879040700 1 --1- 172.21.15.17:0/788443488 >> v1:172.21.15.77:6789/0 conn(0x7f98740b2710 0x7f987414afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:52716/0 (socket says 172.21.15.17:52716) 2022-08-01T19:42:42.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.073+0000 7f987883f700 1 --2- 172.21.15.17:0/788443488 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98740b01a0 0x7f98740a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:42.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.073+0000 7f986bfff700 1 -- 172.21.15.17:0/788443488 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3868997468 0 0) 0x7f98740a7250 con 0x7f98740afd60 2022-08-01T19:42:42.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.073+0000 7f986bfff700 1 -- 172.21.15.17:0/788443488 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9858003260 con 0x7f98740afd60 2022-08-01T19:42:42.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.073+0000 7f986bfff700 1 -- 172.21.15.17:0/788443488 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2617836116 0 0) 0x7f98740a7480 con 0x7f98740b2710 2022-08-01T19:42:42.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.073+0000 7f987883f700 1 -- 172.21.15.17:0/788443488 >> v1:172.21.15.77:6789/0 conn(0x7f98740b2710 legacy=0x7f987414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:42.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.073+0000 7f987883f700 1 -- 172.21.15.17:0/788443488 >> v1:172.21.15.17:6789/0 conn(0x7f98740afd60 legacy=0x7f9874151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:42.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.073+0000 7f987883f700 1 -- 172.21.15.17:0/788443488 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98740a79c0 con 0x7f98740b01a0 2022-08-01T19:42:42.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.073+0000 7f987883f700 1 --2- 172.21.15.17:0/788443488 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98740b01a0 0x7f98740a8540 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f9864002f70 tx=0x7f9864006df0).ready entity=mon.2 client_cookie=6fd2191bd596510a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:42.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.073+0000 7f986bfff700 1 -- 172.21.15.17:0/788443488 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f986400d570 con 0x7f98740b01a0 2022-08-01T19:42:42.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.073+0000 7f986bfff700 1 -- 172.21.15.17:0/788443488 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9864000e90 con 0x7f98740b01a0 2022-08-01T19:42:42.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.073+0000 7f986bfff700 1 -- 172.21.15.17:0/788443488 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f986400d8c0 con 0x7f98740b01a0 2022-08-01T19:42:42.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.073+0000 7f987a042700 1 -- 172.21.15.17:0/788443488 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98740b01a0 msgr2=0x7f98740a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:42.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.073+0000 7f987a042700 1 --2- 172.21.15.17:0/788443488 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98740b01a0 0x7f98740a8540 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f9864002f70 tx=0x7f9864006df0).stop 2022-08-01T19:42:42.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.073+0000 7f987a042700 1 -- 172.21.15.17:0/788443488 shutdown_connections 2022-08-01T19:42:42.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.073+0000 7f987a042700 1 --2- 172.21.15.17:0/788443488 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98740b01a0 0x7f98740a8540 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:42.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.073+0000 7f987a042700 1 -- 172.21.15.17:0/788443488 >> 172.21.15.17:0/788443488 conn(0x7f9874060310 msgr2=0x7f98740b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:42.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.073+0000 7f987a042700 1 -- 172.21.15.17:0/788443488 shutdown_connections 2022-08-01T19:42:42.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.073+0000 7f987a042700 1 -- 172.21.15.17:0/788443488 wait complete. 2022-08-01T19:42:42.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.077+0000 7f987a042700 1 Processor -- start 2022-08-01T19:42:42.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.077+0000 7f987a042700 1 -- start start 2022-08-01T19:42:42.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.077+0000 7f987a042700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98740afd60 0x7f98740b3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:42.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.077+0000 7f987a042700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98740b01a0 0x7f98740aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:42.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.077+0000 7f987a042700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98740b2710 0x7f98740abb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:42.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.077+0000 7f987a042700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9874159f50 con 0x7f98740afd60 2022-08-01T19:42:42.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.077+0000 7f987a042700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987415a1f0 con 0x7f98740b2710 2022-08-01T19:42:42.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.077+0000 7f987a042700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f987415a490 con 0x7f98740b01a0 2022-08-01T19:42:42.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.077+0000 7f9879040700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98740afd60 0x7f98740b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:42.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.077+0000 7f9879040700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98740afd60 0x7f98740b3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43728/0 (socket says 172.21.15.17:43728) 2022-08-01T19:42:42.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.077+0000 7f9879040700 1 -- 172.21.15.17:0/535118315 learned_addr learned my addr 172.21.15.17:0/535118315 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:42.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.077+0000 7f987883f700 1 --2- 172.21.15.17:0/535118315 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98740b01a0 0x7f98740aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:42.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.077+0000 7f9879841700 1 --2- 172.21.15.17:0/535118315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98740b2710 0x7f98740abb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:42.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.077+0000 7f9879040700 1 -- 172.21.15.17:0/535118315 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98740b01a0 msgr2=0x7f98740aec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:42.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.077+0000 7f9879040700 1 --2- 172.21.15.17:0/535118315 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98740b01a0 0x7f98740aec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:42.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.077+0000 7f9879040700 1 -- 172.21.15.17:0/535118315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98740b2710 msgr2=0x7f98740abb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:42.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.077+0000 7f9879040700 1 --2- 172.21.15.17:0/535118315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98740b2710 0x7f98740abb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:42.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.077+0000 7f9879040700 1 -- 172.21.15.17:0/535118315 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f986400b180 con 0x7f98740afd60 2022-08-01T19:42:42.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.077+0000 7f9879040700 1 --2- 172.21.15.17:0/535118315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98740afd60 0x7f98740b3210 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 rx=0x7f9870000f40 tx=0x7f98700013c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:42.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.077+0000 7f986a7fc700 1 -- 172.21.15.17:0/535118315 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9870005ae0 con 0x7f98740afd60 2022-08-01T19:42:42.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.077+0000 7f986a7fc700 1 -- 172.21.15.17:0/535118315 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9870005c40 con 0x7f98740afd60 2022-08-01T19:42:42.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.077+0000 7f987a042700 1 -- 172.21.15.17:0/535118315 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f987415a730 con 0x7f98740afd60 2022-08-01T19:42:42.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.077+0000 7f986a7fc700 1 -- 172.21.15.17:0/535118315 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98700070d0 con 0x7f98740afd60 2022-08-01T19:42:42.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.077+0000 7f987a042700 1 -- 172.21.15.17:0/535118315 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98740b3700 con 0x7f98740afd60 2022-08-01T19:42:42.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.081+0000 7f987a042700 1 -- 172.21.15.17:0/535118315 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f987415a910 con 0x7f98740afd60 2022-08-01T19:42:42.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.081+0000 7f986a7fc700 1 -- 172.21.15.17:0/535118315 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f987001a060 con 0x7f98740afd60 2022-08-01T19:42:42.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.081+0000 7f986a7fc700 1 --2- 172.21.15.17:0/535118315 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f985c053980 0x7f985c055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:42.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.081+0000 7f986a7fc700 1 -- 172.21.15.17:0/535118315 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6422+0+0 (secure 0 0 0) 0x7f987006bf70 con 0x7f98740afd60 2022-08-01T19:42:42.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.081+0000 7f987883f700 1 --2- 172.21.15.17:0/535118315 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f985c053980 0x7f985c055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:42.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.081+0000 7f987883f700 1 --2- 172.21.15.17:0/535118315 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f985c053980 0x7f985c055e40 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f9864018000 tx=0x7f9864014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:42.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.085+0000 7f986a7fc700 1 -- 172.21.15.17:0/535118315 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9870042cb0 con 0x7f98740afd60 2022-08-01T19:42:42.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.253+0000 7f987a042700 1 -- 172.21.15.17:0/535118315 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7f987415a8c0 con 0x7f98740afd60 2022-08-01T19:42:42.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.253+0000 7f986a7fc700 1 -- 172.21.15.17:0/535118315 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1932 (secure 0 0 0) 0x7f9870018030 con 0x7f98740afd60 2022-08-01T19:42:42.257 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:42:42.257 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:42:42.257 INFO:teuthology.orchestra.run.smithi017.stdout: "stats": { 2022-08-01T19:42:42.258 INFO:teuthology.orchestra.run.smithi017.stdout: "total_bytes": 773094113280, 2022-08-01T19:42:42.258 INFO:teuthology.orchestra.run.smithi017.stdout: "total_avail_bytes": 764491722752, 2022-08-01T19:42:42.259 INFO:teuthology.orchestra.run.smithi017.stdout: "total_used_bytes": 12455936, 2022-08-01T19:42:42.259 INFO:teuthology.orchestra.run.smithi017.stdout: "total_used_raw_bytes": 8602390528, 2022-08-01T19:42:42.259 INFO:teuthology.orchestra.run.smithi017.stdout: "total_used_raw_ratio": 0.011127223260700703, 2022-08-01T19:42:42.260 INFO:teuthology.orchestra.run.smithi017.stdout: "num_osds": 8, 2022-08-01T19:42:42.261 INFO:teuthology.orchestra.run.smithi017.stdout: "num_per_pool_osds": 8, 2022-08-01T19:42:42.261 INFO:teuthology.orchestra.run.smithi017.stdout: "num_per_pool_omap_osds": 8 2022-08-01T19:42:42.262 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:42.262 INFO:teuthology.orchestra.run.smithi017.stdout: "stats_by_class": { 2022-08-01T19:42:42.264 INFO:teuthology.orchestra.run.smithi017.stdout: "ssd": { 2022-08-01T19:42:42.264 INFO:teuthology.orchestra.run.smithi017.stdout: "total_bytes": 773094113280, 2022-08-01T19:42:42.265 INFO:teuthology.orchestra.run.smithi017.stdout: "total_avail_bytes": 764491722752, 2022-08-01T19:42:42.265 INFO:teuthology.orchestra.run.smithi017.stdout: "total_used_bytes": 12455936, 2022-08-01T19:42:42.266 INFO:teuthology.orchestra.run.smithi017.stdout: "total_used_raw_bytes": 8602390528, 2022-08-01T19:42:42.274 INFO:teuthology.orchestra.run.smithi017.stdout: "total_used_raw_ratio": 0.011127223260700703 2022-08-01T19:42:42.275 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:42.275 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:42.275 INFO:teuthology.orchestra.run.smithi017.stdout: "pools": [ 2022-08-01T19:42:42.276 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:42.276 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "device_health_metrics", 2022-08-01T19:42:42.276 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 1, 2022-08-01T19:42:42.276 INFO:teuthology.orchestra.run.smithi017.stdout: "stats": { 2022-08-01T19:42:42.277 INFO:teuthology.orchestra.run.smithi017.stdout: "stored": 0, 2022-08-01T19:42:42.277 INFO:teuthology.orchestra.run.smithi017.stdout: "objects": 2, 2022-08-01T19:42:42.277 INFO:teuthology.orchestra.run.smithi017.stdout: "kb_used": 0, 2022-08-01T19:42:42.277 INFO:teuthology.orchestra.run.smithi017.stdout: "bytes_used": 0, 2022-08-01T19:42:42.277 INFO:teuthology.orchestra.run.smithi017.stdout: "percent_used": 0, 2022-08-01T19:42:42.278 INFO:teuthology.orchestra.run.smithi017.stdout: "max_avail": 362918445056 2022-08-01T19:42:42.278 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:42.278 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:42.278 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:42.279 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "rbd", 2022-08-01T19:42:42.279 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 2, 2022-08-01T19:42:42.279 INFO:teuthology.orchestra.run.smithi017.stdout: "stats": { 2022-08-01T19:42:42.279 INFO:teuthology.orchestra.run.smithi017.stdout: "stored": 0, 2022-08-01T19:42:42.280 INFO:teuthology.orchestra.run.smithi017.stdout: "objects": 0, 2022-08-01T19:42:42.280 INFO:teuthology.orchestra.run.smithi017.stdout: "kb_used": 0, 2022-08-01T19:42:42.280 INFO:teuthology.orchestra.run.smithi017.stdout: "bytes_used": 0, 2022-08-01T19:42:42.280 INFO:teuthology.orchestra.run.smithi017.stdout: "percent_used": 0, 2022-08-01T19:42:42.281 INFO:teuthology.orchestra.run.smithi017.stdout: "max_avail": 362918445056 2022-08-01T19:42:42.281 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:42.281 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:42.281 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:42.282 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "cephfs_data", 2022-08-01T19:42:42.282 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 22, 2022-08-01T19:42:42.282 INFO:teuthology.orchestra.run.smithi017.stdout: "stats": { 2022-08-01T19:42:42.282 INFO:teuthology.orchestra.run.smithi017.stdout: "stored": 0, 2022-08-01T19:42:42.283 INFO:teuthology.orchestra.run.smithi017.stdout: "objects": 0, 2022-08-01T19:42:42.283 INFO:teuthology.orchestra.run.smithi017.stdout: "kb_used": 0, 2022-08-01T19:42:42.283 INFO:teuthology.orchestra.run.smithi017.stdout: "bytes_used": 0, 2022-08-01T19:42:42.283 INFO:teuthology.orchestra.run.smithi017.stdout: "percent_used": 0, 2022-08-01T19:42:42.284 INFO:teuthology.orchestra.run.smithi017.stdout: "max_avail": 362918445056 2022-08-01T19:42:42.284 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:42.284 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:42.284 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:42.285 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "cephfs_metadata", 2022-08-01T19:42:42.285 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 23, 2022-08-01T19:42:42.285 INFO:teuthology.orchestra.run.smithi017.stdout: "stats": { 2022-08-01T19:42:42.285 INFO:teuthology.orchestra.run.smithi017.stdout: "stored": 617, 2022-08-01T19:42:42.286 INFO:teuthology.orchestra.run.smithi017.stdout: "objects": 1, 2022-08-01T19:42:42.287 INFO:teuthology.orchestra.run.smithi017.stdout: "kb_used": 8, 2022-08-01T19:42:42.287 INFO:teuthology.orchestra.run.smithi017.stdout: "bytes_used": 8192, 2022-08-01T19:42:42.287 INFO:teuthology.orchestra.run.smithi017.stdout: "percent_used": 1.1286282131095504e-08, 2022-08-01T19:42:42.287 INFO:teuthology.orchestra.run.smithi017.stdout: "max_avail": 362918445056 2022-08-01T19:42:42.288 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:42.288 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:42.288 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:42.289 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:42:42.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.257+0000 7f987a042700 1 -- 172.21.15.17:0/535118315 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f985c053980 msgr2=0x7f985c055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:42.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.257+0000 7f987a042700 1 --2- 172.21.15.17:0/535118315 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f985c053980 0x7f985c055e40 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f9864018000 tx=0x7f9864014000).stop 2022-08-01T19:42:42.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.257+0000 7f987a042700 1 -- 172.21.15.17:0/535118315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98740afd60 msgr2=0x7f98740b3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:42.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.257+0000 7f987a042700 1 --2- 172.21.15.17:0/535118315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98740afd60 0x7f98740b3210 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 rx=0x7f9870000f40 tx=0x7f98700013c0).stop 2022-08-01T19:42:42.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.257+0000 7f987a042700 1 -- 172.21.15.17:0/535118315 shutdown_connections 2022-08-01T19:42:42.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.257+0000 7f987a042700 1 --2- 172.21.15.17:0/535118315 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f985c053980 0x7f985c055e40 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:42.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.257+0000 7f987a042700 1 --2- 172.21.15.17:0/535118315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98740b2710 0x7f98740abb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:42.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.257+0000 7f987a042700 1 --2- 172.21.15.17:0/535118315 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f98740b01a0 0x7f98740aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:42.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.257+0000 7f987a042700 1 --2- 172.21.15.17:0/535118315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98740afd60 0x7f98740b3210 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:42.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.257+0000 7f987a042700 1 -- 172.21.15.17:0/535118315 >> 172.21.15.17:0/535118315 conn(0x7f9874060310 msgr2=0x7f9874145ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:42.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.257+0000 7f987a042700 1 -- 172.21.15.17:0/535118315 shutdown_connections 2022-08-01T19:42:42.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.257+0000 7f987a042700 1 -- 172.21.15.17:0/535118315 wait complete. 2022-08-01T19:42:42.295 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s 2022-08-01T19:42:42.296 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2022-08-01T19:42:42.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.405+0000 7f5886fa6700 1 Processor -- start 2022-08-01T19:42:42.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.405+0000 7f5886fa6700 1 -- start start 2022-08-01T19:42:42.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.405+0000 7f5886fa6700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f58800b31d0 0x7f5880066020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:42.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.405+0000 7f5886fa6700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f58801402d0 0x7f5880140780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:42.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.405+0000 7f5886fa6700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5880067030 con 0x7f58800a9da0 2022-08-01T19:42:42.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.405+0000 7f5886fa6700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5880064a70 con 0x7f58800b31d0 2022-08-01T19:42:42.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.405+0000 7f5886fa6700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5880061f60 con 0x7f58801402d0 2022-08-01T19:42:42.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.405+0000 7f58857a3700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f58800a9da0 0x7f5880060950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35932/0 (socket says 172.21.15.17:35932) 2022-08-01T19:42:42.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.405+0000 7f58857a3700 1 -- 172.21.15.17:0/2876766101 learned_addr learned my addr 172.21.15.17:0/2876766101 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:42.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.405+0000 7f5885fa4700 1 --2- 172.21.15.17:0/2876766101 >> v2:172.21.15.77:3300/0 conn(0x7f58800b31d0 0x7f5880066020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:42.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.405+0000 7f58867a5700 1 --2- 172.21.15.17:0/2876766101 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f58801402d0 0x7f5880140780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:42.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.409+0000 7f5884fa2700 1 -- 172.21.15.17:0/2876766101 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4278504626 0 0) 0x7f5880067030 con 0x7f58800a9da0 2022-08-01T19:42:42.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.409+0000 7f5884fa2700 1 -- 172.21.15.17:0/2876766101 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5868003260 con 0x7f58800a9da0 2022-08-01T19:42:42.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.409+0000 7f5885fa4700 1 -- 172.21.15.17:0/2876766101 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f58801402d0 msgr2=0x7f5880140780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:42.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.409+0000 7f5885fa4700 1 --2- 172.21.15.17:0/2876766101 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f58801402d0 0x7f5880140780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:42.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.409+0000 7f5884fa2700 1 -- 172.21.15.17:0/2876766101 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 263172810 0 0) 0x7f5868003260 con 0x7f58800a9da0 2022-08-01T19:42:42.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.409+0000 7f5885fa4700 1 -- 172.21.15.17:0/2876766101 >> v1:172.21.15.17:6789/0 conn(0x7f58800a9da0 legacy=0x7f5880060950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:42.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.409+0000 7f5885fa4700 1 -- 172.21.15.17:0/2876766101 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58800b35b0 con 0x7f58800b31d0 2022-08-01T19:42:42.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.409+0000 7f58867a5700 1 --2- 172.21.15.17:0/2876766101 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f58801402d0 0x7f5880140780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:42:42.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.409+0000 7f5885fa4700 1 --2- 172.21.15.17:0/2876766101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58800b31d0 0x7f5880066020 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f5870001800 tx=0x7f587c006df0).ready entity=mon.1 client_cookie=3b68e2f9d8784a81 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:42.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.409+0000 7f5884fa2700 1 -- 172.21.15.17:0/2876766101 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5870001660 con 0x7f58800b31d0 2022-08-01T19:42:42.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.409+0000 7f5884fa2700 1 -- 172.21.15.17:0/2876766101 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587c000e90 con 0x7f58800b31d0 2022-08-01T19:42:42.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.409+0000 7f5884fa2700 1 -- 172.21.15.17:0/2876766101 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f587c012460 con 0x7f58800b31d0 2022-08-01T19:42:42.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.409+0000 7f5886fa6700 1 -- 172.21.15.17:0/2876766101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58800b31d0 msgr2=0x7f5880066020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:42.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.409+0000 7f5886fa6700 1 --2- 172.21.15.17:0/2876766101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58800b31d0 0x7f5880066020 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f5870001800 tx=0x7f587c006df0).stop 2022-08-01T19:42:42.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.409+0000 7f5886fa6700 1 -- 172.21.15.17:0/2876766101 shutdown_connections 2022-08-01T19:42:42.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.409+0000 7f5886fa6700 1 --2- 172.21.15.17:0/2876766101 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f58801402d0 0x7f5880140780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:42.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.409+0000 7f5886fa6700 1 --2- 172.21.15.17:0/2876766101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58800b31d0 0x7f5880066020 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:42.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.409+0000 7f5886fa6700 1 -- 172.21.15.17:0/2876766101 >> 172.21.15.17:0/2876766101 conn(0x7f5880063870 msgr2=0x7f5880063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:42.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.409+0000 7f5886fa6700 1 -- 172.21.15.17:0/2876766101 shutdown_connections 2022-08-01T19:42:42.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.409+0000 7f5886fa6700 1 -- 172.21.15.17:0/2876766101 wait complete. 2022-08-01T19:42:42.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.413+0000 7f5886fa6700 1 Processor -- start 2022-08-01T19:42:42.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.413+0000 7f5886fa6700 1 -- start start 2022-08-01T19:42:42.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.413+0000 7f5886fa6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58800a9da0 0x7f5880132bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:42.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.413+0000 7f5886fa6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58800b31d0 0x7f5880143070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:42.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.413+0000 7f5886fa6700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f58801402d0 0x7f5880143560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:42.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.413+0000 7f5886fa6700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58800ab2a0 con 0x7f58800a9da0 2022-08-01T19:42:42.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.413+0000 7f5886fa6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58800ab500 con 0x7f58800b31d0 2022-08-01T19:42:42.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.413+0000 7f5886fa6700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58800ab7a0 con 0x7f58801402d0 2022-08-01T19:42:42.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.413+0000 7f5885fa4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58800a9da0 0x7f5880132bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:42.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.413+0000 7f5885fa4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58800a9da0 0x7f5880132bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43742/0 (socket says 172.21.15.17:43742) 2022-08-01T19:42:42.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.413+0000 7f5885fa4700 1 -- 172.21.15.17:0/674730679 learned_addr learned my addr 172.21.15.17:0/674730679 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:42.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.413+0000 7f58857a3700 1 --2- 172.21.15.17:0/674730679 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58800b31d0 0x7f5880143070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:42.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.413+0000 7f58867a5700 1 --2- 172.21.15.17:0/674730679 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f58801402d0 0x7f5880143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:42.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.413+0000 7f5885fa4700 1 -- 172.21.15.17:0/674730679 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f58801402d0 msgr2=0x7f5880143560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:42.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.413+0000 7f5885fa4700 1 --2- 172.21.15.17:0/674730679 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f58801402d0 0x7f5880143560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:42.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.413+0000 7f5885fa4700 1 -- 172.21.15.17:0/674730679 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58800b31d0 msgr2=0x7f5880143070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:42.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.413+0000 7f5885fa4700 1 --2- 172.21.15.17:0/674730679 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58800b31d0 0x7f5880143070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:42.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.413+0000 7f5885fa4700 1 -- 172.21.15.17:0/674730679 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5878009470 con 0x7f58800a9da0 2022-08-01T19:42:42.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.413+0000 7f58857a3700 1 --2- 172.21.15.17:0/674730679 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58800b31d0 0x7f5880143070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:42:42.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.413+0000 7f58867a5700 1 --2- 172.21.15.17:0/674730679 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f58801402d0 0x7f5880143560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:42:42.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.413+0000 7f5885fa4700 1 --2- 172.21.15.17:0/674730679 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58800a9da0 0x7f5880132bb0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 rx=0x7f5870001800 tx=0x7f587c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:42.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.413+0000 7f586effd700 1 -- 172.21.15.17:0/674730679 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f587c00da20 con 0x7f58800a9da0 2022-08-01T19:42:42.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.413+0000 7f586effd700 1 -- 172.21.15.17:0/674730679 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587c00db80 con 0x7f58800a9da0 2022-08-01T19:42:42.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.413+0000 7f5886fa6700 1 -- 172.21.15.17:0/674730679 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f587c00b180 con 0x7f58800a9da0 2022-08-01T19:42:42.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.413+0000 7f586effd700 1 -- 172.21.15.17:0/674730679 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f587c01e400 con 0x7f58800a9da0 2022-08-01T19:42:42.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.413+0000 7f5886fa6700 1 -- 172.21.15.17:0/674730679 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5880141790 con 0x7f58800a9da0 2022-08-01T19:42:42.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.417+0000 7f5886fa6700 1 -- 172.21.15.17:0/674730679 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58800abc40 con 0x7f58800a9da0 2022-08-01T19:42:42.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.417+0000 7f586effd700 1 -- 172.21.15.17:0/674730679 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f587c025030 con 0x7f58800a9da0 2022-08-01T19:42:42.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.417+0000 7f586effd700 1 --2- 172.21.15.17:0/674730679 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5860053b00 0x7f5860055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:42.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.417+0000 7f586effd700 1 -- 172.21.15.17:0/674730679 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6422+0+0 (secure 0 0 0) 0x7f587c06ce90 con 0x7f58800a9da0 2022-08-01T19:42:42.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.417+0000 7f58857a3700 1 --2- 172.21.15.17:0/674730679 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5860053b00 0x7f5860055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:42.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.417+0000 7f58857a3700 1 --2- 172.21.15.17:0/674730679 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5860053b00 0x7f5860055fc0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7f5870008000 tx=0x7f5870009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:42.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.425+0000 7f586effd700 1 -- 172.21.15.17:0/674730679 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f58800abc40 con 0x7f58800a9da0 2022-08-01T19:42:42.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.589+0000 7f5886fa6700 1 -- 172.21.15.17:0/674730679 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f58800abc40 con 0x7f58800a9da0 2022-08-01T19:42:42.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.589+0000 7f586effd700 1 -- 172.21.15.17:0/674730679 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=-22 pool 'cephfs_metadata' already contains some objects. Use an empty pool instead. v60) v1 ==== 209+0+0 (secure 0 0 0) 0x7f587c06e020 con 0x7f58800a9da0 2022-08-01T19:42:42.595 INFO:teuthology.orchestra.run.smithi017.stderr:Error EINVAL: pool 'cephfs_metadata' already contains some objects. Use an empty pool instead. 2022-08-01T19:42:42.601 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-08-01T19:42:42.612 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-08-01T19:42:42.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.729+0000 7f49b0b93700 1 Processor -- start 2022-08-01T19:42:42.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.729+0000 7f49b0b93700 1 -- start start 2022-08-01T19:42:42.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.733+0000 7f49b0b93700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f49ac0b2710 0x7f49ac14afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:42.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.733+0000 7f49b0b93700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f49ac0b01a0 0x7f49ac14b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:42.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.733+0000 7f49b0b93700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f49ac0afd60 0x7f49ac0a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:42.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.733+0000 7f49b0b93700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49ac0a7250 con 0x7f49ac0b01a0 2022-08-01T19:42:42.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.733+0000 7f49b0b93700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49ac0a7480 con 0x7f49ac0b2710 2022-08-01T19:42:42.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.733+0000 7f49b0b93700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac0a7720 con 0x7f49ac0afd60 2022-08-01T19:42:42.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.733+0000 7f49aaffd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f49ac0b01a0 0x7f49ac14b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:42.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.733+0000 7f49aaffd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f49ac0b01a0 0x7f49ac14b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43752/0 (socket says 172.21.15.17:43752) 2022-08-01T19:42:42.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.733+0000 7f49aaffd700 1 -- 172.21.15.17:0/3327071675 learned_addr learned my addr 172.21.15.17:0/3327071675 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:42.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.733+0000 7f49ab7fe700 1 --2- 172.21.15.17:0/3327071675 >> v2:172.21.15.77:3300/0 conn(0x7f49ac0b2710 0x7f49ac14afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:42.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.733+0000 7f49abfff700 1 --2- 172.21.15.17:0/3327071675 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f49ac0afd60 0x7f49ac0a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:42.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.733+0000 7f49aaffd700 1 -- 172.21.15.17:0/3327071675 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f49ac0afd60 msgr2=0x7f49ac0a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:42.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.733+0000 7f49aaffd700 1 --2- 172.21.15.17:0/3327071675 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f49ac0afd60 0x7f49ac0a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:42.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.733+0000 7f49aaffd700 1 -- 172.21.15.17:0/3327071675 >> v2:172.21.15.77:3300/0 conn(0x7f49ac0b2710 msgr2=0x7f49ac14afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:42.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.733+0000 7f49aaffd700 1 --2- 172.21.15.17:0/3327071675 >> v2:172.21.15.77:3300/0 conn(0x7f49ac0b2710 0x7f49ac14afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:42.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.733+0000 7f49aaffd700 1 -- 172.21.15.17:0/3327071675 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ac0a79c0 con 0x7f49ac0b01a0 2022-08-01T19:42:42.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.733+0000 7f49aaffd700 1 --2- 172.21.15.17:0/3327071675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49ac0b01a0 0x7f49ac14b4c0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 rx=0x7f49a0002f70 tx=0x7f49a0006eb0).ready entity=mon.0 client_cookie=931205ce3c30bb98 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:42.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.733+0000 7f49aa7fc700 1 -- 172.21.15.17:0/3327071675 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49a000d570 con 0x7f49ac0b01a0 2022-08-01T19:42:42.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.733+0000 7f49aa7fc700 1 -- 172.21.15.17:0/3327071675 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a0000e90 con 0x7f49ac0b01a0 2022-08-01T19:42:42.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.733+0000 7f49aa7fc700 1 -- 172.21.15.17:0/3327071675 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49a000d900 con 0x7f49ac0b01a0 2022-08-01T19:42:42.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.733+0000 7f49b0b93700 1 -- 172.21.15.17:0/3327071675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49ac0b01a0 msgr2=0x7f49ac14b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:42.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.733+0000 7f49b0b93700 1 --2- 172.21.15.17:0/3327071675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49ac0b01a0 0x7f49ac14b4c0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 rx=0x7f49a0002f70 tx=0x7f49a0006eb0).stop 2022-08-01T19:42:42.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.733+0000 7f49b0b93700 1 -- 172.21.15.17:0/3327071675 shutdown_connections 2022-08-01T19:42:42.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.733+0000 7f49b0b93700 1 --2- 172.21.15.17:0/3327071675 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f49ac0afd60 0x7f49ac0a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:42.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.733+0000 7f49b0b93700 1 --2- 172.21.15.17:0/3327071675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49ac0b01a0 0x7f49ac14b4c0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:42.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.733+0000 7f49b0b93700 1 --2- 172.21.15.17:0/3327071675 >> v2:172.21.15.77:3300/0 conn(0x7f49ac0b2710 0x7f49ac14afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:42.804 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.733+0000 7f49b0b93700 1 -- 172.21.15.17:0/3327071675 >> 172.21.15.17:0/3327071675 conn(0x7f49ac060310 msgr2=0x7f49ac0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:42.804 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.733+0000 7f49b0b93700 1 -- 172.21.15.17:0/3327071675 shutdown_connections 2022-08-01T19:42:42.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.733+0000 7f49b0b93700 1 -- 172.21.15.17:0/3327071675 wait complete. 2022-08-01T19:42:42.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.737+0000 7f49b0b93700 1 Processor -- start 2022-08-01T19:42:42.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.737+0000 7f49b0b93700 1 -- start start 2022-08-01T19:42:42.810 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.737+0000 7f49b0b93700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49ac0afd60 0x7f49ac0aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:42.811 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.737+0000 7f49b0b93700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49ac0b01a0 0x7f49ac0aa500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:42.811 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.737+0000 7f49b0b93700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f49ac0b2710 0x7f49ac0aa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:42.811 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.737+0000 7f49b0b93700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac0a9d50 con 0x7f49ac0afd60 2022-08-01T19:42:42.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.737+0000 7f49b0b93700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac067ea0 con 0x7f49ac0b01a0 2022-08-01T19:42:42.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.737+0000 7f49b0b93700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac0b5550 con 0x7f49ac0b2710 2022-08-01T19:42:42.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.737+0000 7f49ab7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49ac0afd60 0x7f49ac0aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:42.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.737+0000 7f49ab7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49ac0afd60 0x7f49ac0aefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43756/0 (socket says 172.21.15.17:43756) 2022-08-01T19:42:42.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.737+0000 7f49ab7fe700 1 -- 172.21.15.17:0/4148772175 learned_addr learned my addr 172.21.15.17:0/4148772175 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:42.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.737+0000 7f49aaffd700 1 --2- 172.21.15.17:0/4148772175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49ac0b01a0 0x7f49ac0aa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:42.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.737+0000 7f49abfff700 1 --2- 172.21.15.17:0/4148772175 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f49ac0b2710 0x7f49ac0aa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:42.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.737+0000 7f49ab7fe700 1 -- 172.21.15.17:0/4148772175 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f49ac0b2710 msgr2=0x7f49ac0aa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:42.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.737+0000 7f49ab7fe700 1 --2- 172.21.15.17:0/4148772175 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f49ac0b2710 0x7f49ac0aa9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:42.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.737+0000 7f49ab7fe700 1 -- 172.21.15.17:0/4148772175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49ac0b01a0 msgr2=0x7f49ac0aa500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:42.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.737+0000 7f49ab7fe700 1 --2- 172.21.15.17:0/4148772175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49ac0b01a0 0x7f49ac0aa500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:42.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.737+0000 7f49ab7fe700 1 -- 172.21.15.17:0/4148772175 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a000b1e0 con 0x7f49ac0afd60 2022-08-01T19:42:42.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.737+0000 7f49ab7fe700 1 --2- 172.21.15.17:0/4148772175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49ac0afd60 0x7f49ac0aefd0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 rx=0x7f49a400c000 tx=0x7f49a40091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:42.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.737+0000 7f49a8ff9700 1 -- 172.21.15.17:0/4148772175 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49a400c6a0 con 0x7f49ac0afd60 2022-08-01T19:42:42.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.737+0000 7f49a8ff9700 1 -- 172.21.15.17:0/4148772175 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a400c800 con 0x7f49ac0afd60 2022-08-01T19:42:42.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.737+0000 7f49a8ff9700 1 -- 172.21.15.17:0/4148772175 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49a400cb60 con 0x7f49ac0afd60 2022-08-01T19:42:42.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.737+0000 7f49b0b93700 1 -- 172.21.15.17:0/4148772175 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49ac0b35b0 con 0x7f49ac0afd60 2022-08-01T19:42:42.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.741+0000 7f49b0b93700 1 -- 172.21.15.17:0/4148772175 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49ac0b3d90 con 0x7f49ac0afd60 2022-08-01T19:42:42.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.741+0000 7f49b0b93700 1 -- 172.21.15.17:0/4148772175 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49ac0b3c00 con 0x7f49ac0afd60 2022-08-01T19:42:42.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.741+0000 7f49a8ff9700 1 -- 172.21.15.17:0/4148772175 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f49a4022070 con 0x7f49ac0afd60 2022-08-01T19:42:42.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.741+0000 7f49a8ff9700 1 --2- 172.21.15.17:0/4148772175 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f498c053a10 0x7f498c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:42.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.741+0000 7f49a8ff9700 1 -- 172.21.15.17:0/4148772175 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6422+0+0 (secure 0 0 0) 0x7f49a406c700 con 0x7f49ac0afd60 2022-08-01T19:42:42.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.741+0000 7f49aaffd700 1 --2- 172.21.15.17:0/4148772175 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f498c053a10 0x7f498c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:42.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.745+0000 7f49aaffd700 1 --2- 172.21.15.17:0/4148772175 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f498c053a10 0x7f498c055ed0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f49a001c000 tx=0x7f49a0015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:42.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.749+0000 7f49a8ff9700 1 -- 172.21.15.17:0/4148772175 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f49a4043440 con 0x7f49ac0afd60 2022-08-01T19:42:42.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.913+0000 7f49b0b93700 1 -- 172.21.15.17:0/4148772175 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f49ac0574e0 con 0x7f49ac0afd60 2022-08-01T19:42:42.923 INFO:tasks.ceph.mon.a.smithi017.stderr:2022-08-01T19:42:42.913+0000 7f6a50f6e700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-08-01T19:42:42.924 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.921+0000 7f49a8ff9700 1 -- 172.21.15.17:0/4148772175 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 23 and data pool 22 v61) v1 ==== 189+0+0 (secure 0 0 0) 0x7f49a4046a60 con 0x7f49ac0afd60 2022-08-01T19:42:42.930 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.921+0000 7f49b0b93700 1 -- 172.21.15.17:0/4148772175 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f498c053a10 msgr2=0x7f498c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:42.930 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.921+0000 7f49b0b93700 1 --2- 172.21.15.17:0/4148772175 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f498c053a10 0x7f498c055ed0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f49a001c000 tx=0x7f49a0015000).stop 2022-08-01T19:42:42.931 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.921+0000 7f49b0b93700 1 -- 172.21.15.17:0/4148772175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49ac0afd60 msgr2=0x7f49ac0aefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:42.931 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.921+0000 7f49b0b93700 1 --2- 172.21.15.17:0/4148772175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49ac0afd60 0x7f49ac0aefd0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 rx=0x7f49a400c000 tx=0x7f49a40091d0).stop 2022-08-01T19:42:42.931 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.921+0000 7f49b0b93700 1 -- 172.21.15.17:0/4148772175 shutdown_connections 2022-08-01T19:42:42.931 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.921+0000 7f49b0b93700 1 --2- 172.21.15.17:0/4148772175 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f498c053a10 0x7f498c055ed0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:42.932 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.921+0000 7f49b0b93700 1 --2- 172.21.15.17:0/4148772175 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f49ac0b2710 0x7f49ac0aa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:42.932 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.921+0000 7f49b0b93700 1 --2- 172.21.15.17:0/4148772175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49ac0b01a0 0x7f49ac0aa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:42.932 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.921+0000 7f49b0b93700 1 --2- 172.21.15.17:0/4148772175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49ac0afd60 0x7f49ac0aefd0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:42.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.921+0000 7f49b0b93700 1 -- 172.21.15.17:0/4148772175 >> 172.21.15.17:0/4148772175 conn(0x7f49ac060310 msgr2=0x7f49ac146070 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:42.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.921+0000 7f49b0b93700 1 -- 172.21.15.17:0/4148772175 shutdown_connections 2022-08-01T19:42:42.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:42.921+0000 7f49b0b93700 1 -- 172.21.15.17:0/4148772175 wait complete. 2022-08-01T19:42:42.942 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 23 and data pool 22 2022-08-01T19:42:42.944 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-08-01T19:42:43.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.069+0000 7f7096b13700 1 Processor -- start 2022-08-01T19:42:43.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.069+0000 7f7096b13700 1 -- start start 2022-08-01T19:42:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.069+0000 7f7096b13700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f70900b2710 0x7f709014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.069+0000 7f7096b13700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f70900b01a0 0x7f709014b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.069+0000 7f7096b13700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70900afd60 0x7f70900a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.069+0000 7f7096b13700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70900a7250 con 0x7f70900b2710 2022-08-01T19:42:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.069+0000 7f7096b13700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70900a7480 con 0x7f70900b01a0 2022-08-01T19:42:43.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.069+0000 7f7096b13700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70900a7720 con 0x7f70900afd60 2022-08-01T19:42:43.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.069+0000 7f7095b11700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f70900b2710 0x7f709014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:43.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.069+0000 7f7095b11700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f70900b2710 0x7f709014afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43764/0 (socket says 172.21.15.17:43764) 2022-08-01T19:42:43.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.069+0000 7f7095b11700 1 -- 172.21.15.17:0/1754401915 learned_addr learned my addr 172.21.15.17:0/1754401915 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:43.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.069+0000 7f7095310700 1 --2- 172.21.15.17:0/1754401915 >> v2:172.21.15.77:3300/0 conn(0x7f70900b01a0 0x7f709014b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:43.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.069+0000 7f7096312700 1 --2- 172.21.15.17:0/1754401915 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70900afd60 0x7f70900a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:43.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.069+0000 7f7095b11700 1 -- 172.21.15.17:0/1754401915 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70900afd60 msgr2=0x7f70900a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:43.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.069+0000 7f7095b11700 1 --2- 172.21.15.17:0/1754401915 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70900afd60 0x7f70900a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:43.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.069+0000 7f7095b11700 1 -- 172.21.15.17:0/1754401915 >> v2:172.21.15.77:3300/0 conn(0x7f70900b01a0 msgr2=0x7f709014b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:43.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.069+0000 7f7095b11700 1 --2- 172.21.15.17:0/1754401915 >> v2:172.21.15.77:3300/0 conn(0x7f70900b01a0 0x7f709014b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:43.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.069+0000 7f7095b11700 1 -- 172.21.15.17:0/1754401915 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70900a79c0 con 0x7f70900b2710 2022-08-01T19:42:43.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.069+0000 7f7095b11700 1 --2- 172.21.15.17:0/1754401915 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70900b2710 0x7f709014afd0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 rx=0x7f7084002f70 tx=0x7f7084006eb0).ready entity=mon.0 client_cookie=6f92899ebcc99e01 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:43.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.069+0000 7f7094b0f700 1 -- 172.21.15.17:0/1754401915 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f708400d570 con 0x7f70900b2710 2022-08-01T19:42:43.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.069+0000 7f7094b0f700 1 -- 172.21.15.17:0/1754401915 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7084000e90 con 0x7f70900b2710 2022-08-01T19:42:43.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.069+0000 7f7094b0f700 1 -- 172.21.15.17:0/1754401915 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f708400d900 con 0x7f70900b2710 2022-08-01T19:42:43.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.069+0000 7f7096b13700 1 -- 172.21.15.17:0/1754401915 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70900b2710 msgr2=0x7f709014afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:43.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.069+0000 7f7096b13700 1 --2- 172.21.15.17:0/1754401915 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70900b2710 0x7f709014afd0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 rx=0x7f7084002f70 tx=0x7f7084006eb0).stop 2022-08-01T19:42:43.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.069+0000 7f7096b13700 1 -- 172.21.15.17:0/1754401915 shutdown_connections 2022-08-01T19:42:43.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.069+0000 7f7096b13700 1 --2- 172.21.15.17:0/1754401915 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70900afd60 0x7f70900a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:43.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.069+0000 7f7096b13700 1 --2- 172.21.15.17:0/1754401915 >> v2:172.21.15.77:3300/0 conn(0x7f70900b01a0 0x7f709014b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:43.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.069+0000 7f7096b13700 1 --2- 172.21.15.17:0/1754401915 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70900b2710 0x7f709014afd0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:43.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.069+0000 7f7096b13700 1 -- 172.21.15.17:0/1754401915 >> 172.21.15.17:0/1754401915 conn(0x7f7090060310 msgr2=0x7f70900b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:43.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.073+0000 7f7096b13700 1 -- 172.21.15.17:0/1754401915 shutdown_connections 2022-08-01T19:42:43.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.073+0000 7f7096b13700 1 -- 172.21.15.17:0/1754401915 wait complete. 2022-08-01T19:42:43.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.073+0000 7f7096b13700 1 Processor -- start 2022-08-01T19:42:43.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.073+0000 7f7096b13700 1 -- start start 2022-08-01T19:42:43.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.073+0000 7f7096b13700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70900afd60 0x7f70900af430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:43.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.073+0000 7f7096b13700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70900b01a0 0x7f70900a9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:43.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.073+0000 7f7096b13700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70900b2710 0x7f70900aa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:43.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.073+0000 7f7096b13700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709014d730 con 0x7f70900b2710 2022-08-01T19:42:43.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.073+0000 7f7096b13700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70900b4150 con 0x7f70900afd60 2022-08-01T19:42:43.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.073+0000 7f7096b13700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70900b3010 con 0x7f70900b01a0 2022-08-01T19:42:43.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.073+0000 7f7096312700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70900b2710 0x7f70900aa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:43.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.073+0000 7f7096312700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70900b2710 0x7f70900aa1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43774/0 (socket says 172.21.15.17:43774) 2022-08-01T19:42:43.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.073+0000 7f7096312700 1 -- 172.21.15.17:0/2657504468 learned_addr learned my addr 172.21.15.17:0/2657504468 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:43.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.073+0000 7f7095310700 1 --2- 172.21.15.17:0/2657504468 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70900b01a0 0x7f70900a9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:43.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.073+0000 7f7095b11700 1 --2- 172.21.15.17:0/2657504468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70900afd60 0x7f70900af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:43.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.073+0000 7f7096312700 1 -- 172.21.15.17:0/2657504468 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70900b01a0 msgr2=0x7f70900a9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:43.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.073+0000 7f7096312700 1 --2- 172.21.15.17:0/2657504468 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70900b01a0 0x7f70900a9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:43.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.073+0000 7f7096312700 1 -- 172.21.15.17:0/2657504468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70900afd60 msgr2=0x7f70900af430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:43.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.073+0000 7f7096312700 1 --2- 172.21.15.17:0/2657504468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70900afd60 0x7f70900af430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:43.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.073+0000 7f7096312700 1 -- 172.21.15.17:0/2657504468 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f708400b1e0 con 0x7f70900b2710 2022-08-01T19:42:43.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.073+0000 7f7095b11700 1 --2- 172.21.15.17:0/2657504468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70900afd60 0x7f70900af430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:42:43.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.073+0000 7f7096312700 1 --2- 172.21.15.17:0/2657504468 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70900b2710 0x7f70900aa1a0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 rx=0x7f70900b4290 tx=0x7f7088008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:43.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.073+0000 7f7082ffd700 1 -- 172.21.15.17:0/2657504468 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7088011780 con 0x7f70900b2710 2022-08-01T19:42:43.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.077+0000 7f7082ffd700 1 -- 172.21.15.17:0/2657504468 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70880118e0 con 0x7f70900b2710 2022-08-01T19:42:43.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.077+0000 7f7096b13700 1 -- 172.21.15.17:0/2657504468 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70900b3590 con 0x7f70900b2710 2022-08-01T19:42:43.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.077+0000 7f7082ffd700 1 -- 172.21.15.17:0/2657504468 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7088011c40 con 0x7f70900b2710 2022-08-01T19:42:43.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.077+0000 7f7096b13700 1 -- 172.21.15.17:0/2657504468 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70900aa690 con 0x7f70900b2710 2022-08-01T19:42:43.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.077+0000 7f7096b13700 1 -- 172.21.15.17:0/2657504468 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70900b49a0 con 0x7f70900b2710 2022-08-01T19:42:43.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.077+0000 7f7082ffd700 1 -- 172.21.15.17:0/2657504468 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7088012420 con 0x7f70900b2710 2022-08-01T19:42:43.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.077+0000 7f7082ffd700 1 --2- 172.21.15.17:0/2657504468 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f70780539e0 0x7f7078055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:43.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.077+0000 7f7082ffd700 1 -- 172.21.15.17:0/2657504468 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f708806be50 con 0x7f70900b2710 2022-08-01T19:42:43.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.077+0000 7f7095b11700 1 --2- 172.21.15.17:0/2657504468 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f70780539e0 0x7f7078055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:43.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.081+0000 7f7095b11700 1 --2- 172.21.15.17:0/2657504468 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f70780539e0 0x7f7078055ea0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f708401a000 tx=0x7f7084015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:43.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.085+0000 7f7082ffd700 1 -- 172.21.15.17:0/2657504468 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7088042b60 con 0x7f70900b2710 2022-08-01T19:42:43.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.249+0000 7f7096b13700 1 -- 172.21.15.17:0/2657504468 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f70900b0880 con 0x7f70900b2710 2022-08-01T19:42:43.933 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.929+0000 7f7082ffd700 1 -- 172.21.15.17:0/2657504468 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v62) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7088046180 con 0x7f70900b2710 2022-08-01T19:42:43.939 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.933+0000 7f7096b13700 1 -- 172.21.15.17:0/2657504468 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f70780539e0 msgr2=0x7f7078055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:43.940 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.933+0000 7f7096b13700 1 --2- 172.21.15.17:0/2657504468 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f70780539e0 0x7f7078055ea0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f708401a000 tx=0x7f7084015000).stop 2022-08-01T19:42:43.944 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.933+0000 7f7096b13700 1 -- 172.21.15.17:0/2657504468 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70900b2710 msgr2=0x7f70900aa1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:43.945 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.933+0000 7f7096b13700 1 --2- 172.21.15.17:0/2657504468 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70900b2710 0x7f70900aa1a0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 rx=0x7f70900b4290 tx=0x7f7088008040).stop 2022-08-01T19:42:43.946 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.933+0000 7f7096b13700 1 -- 172.21.15.17:0/2657504468 shutdown_connections 2022-08-01T19:42:43.946 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.933+0000 7f7096b13700 1 --2- 172.21.15.17:0/2657504468 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f70780539e0 0x7f7078055ea0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:43.947 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.933+0000 7f7096b13700 1 --2- 172.21.15.17:0/2657504468 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70900b2710 0x7f70900aa1a0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:43.947 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.933+0000 7f7096b13700 1 --2- 172.21.15.17:0/2657504468 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f70900b01a0 0x7f70900a9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:43.948 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.933+0000 7f7096b13700 1 --2- 172.21.15.17:0/2657504468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70900afd60 0x7f70900af430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:43.948 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.933+0000 7f7096b13700 1 -- 172.21.15.17:0/2657504468 >> 172.21.15.17:0/2657504468 conn(0x7f7090060310 msgr2=0x7f7090148850 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:43.949 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.933+0000 7f7096b13700 1 -- 172.21.15.17:0/2657504468 shutdown_connections 2022-08-01T19:42:43.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:43.933+0000 7f7096b13700 1 -- 172.21.15.17:0/2657504468 wait complete. 2022-08-01T19:42:43.959 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-08-01T19:42:44.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.081+0000 7f5ab9d9f700 1 Processor -- start 2022-08-01T19:42:44.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.081+0000 7f5ab9d9f700 1 -- start start 2022-08-01T19:42:44.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.081+0000 7f5ab9d9f700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5ab40b6a90 0x7f5ab40acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:44.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.081+0000 7f5ab9d9f700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f5ab40aa760 0x7f5ab413c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:44.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.081+0000 7f5ab9d9f700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ab40b6e70 con 0x7f5ab4060aa0 2022-08-01T19:42:44.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.081+0000 7f5ab9d9f700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ab4061370 con 0x7f5ab40aa760 2022-08-01T19:42:44.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.081+0000 7f5ab9d9f700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab4061610 con 0x7f5ab40b6a90 2022-08-01T19:42:44.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.081+0000 7f5ab2ffd700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5ab4060aa0 0x7f5ab40aa050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35976/0 (socket says 172.21.15.17:35976) 2022-08-01T19:42:44.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.081+0000 7f5ab2ffd700 1 -- 172.21.15.17:0/2386494503 learned_addr learned my addr 172.21.15.17:0/2386494503 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:44.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.081+0000 7f5ab3fff700 1 --2- 172.21.15.17:0/2386494503 >> v2:172.21.15.77:3300/0 conn(0x7f5ab40aa760 0x7f5ab413c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:44.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.081+0000 7f5ab37fe700 1 --2- 172.21.15.17:0/2386494503 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5ab40b6a90 0x7f5ab40acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:44.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.081+0000 7f5ab27fc700 1 -- 172.21.15.17:0/2386494503 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 685797492 0 0) 0x7f5ab40b6e70 con 0x7f5ab4060aa0 2022-08-01T19:42:44.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.081+0000 7f5ab27fc700 1 -- 172.21.15.17:0/2386494503 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a94003260 con 0x7f5ab4060aa0 2022-08-01T19:42:44.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.081+0000 7f5ab3fff700 1 -- 172.21.15.17:0/2386494503 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5ab40b6a90 msgr2=0x7f5ab40acee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:44.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.081+0000 7f5ab3fff700 1 --2- 172.21.15.17:0/2386494503 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5ab40b6a90 0x7f5ab40acee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:44.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.081+0000 7f5ab3fff700 1 -- 172.21.15.17:0/2386494503 >> v1:172.21.15.17:6789/0 conn(0x7f5ab4060aa0 legacy=0x7f5ab40aa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:44.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.081+0000 7f5ab3fff700 1 -- 172.21.15.17:0/2386494503 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab40618b0 con 0x7f5ab40aa760 2022-08-01T19:42:44.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.081+0000 7f5ab37fe700 1 --2- 172.21.15.17:0/2386494503 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5ab40b6a90 0x7f5ab40acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:42:44.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.081+0000 7f5ab3fff700 1 --2- 172.21.15.17:0/2386494503 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ab40aa760 0x7f5ab413c600 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7f5ab413d6d0 tx=0x7f5aac006df0).ready entity=mon.1 client_cookie=96b380456a890ab7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:44.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.081+0000 7f5ab27fc700 1 -- 172.21.15.17:0/2386494503 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5aac00d570 con 0x7f5ab40aa760 2022-08-01T19:42:44.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.081+0000 7f5ab27fc700 1 -- 172.21.15.17:0/2386494503 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aac000e90 con 0x7f5ab40aa760 2022-08-01T19:42:44.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.081+0000 7f5ab27fc700 1 -- 172.21.15.17:0/2386494503 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5aac00d8c0 con 0x7f5ab40aa760 2022-08-01T19:42:44.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.081+0000 7f5ab9d9f700 1 -- 172.21.15.17:0/2386494503 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ab40aa760 msgr2=0x7f5ab413c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:44.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.081+0000 7f5ab9d9f700 1 --2- 172.21.15.17:0/2386494503 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ab40aa760 0x7f5ab413c600 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7f5ab413d6d0 tx=0x7f5aac006df0).stop 2022-08-01T19:42:44.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.085+0000 7f5ab9d9f700 1 -- 172.21.15.17:0/2386494503 shutdown_connections 2022-08-01T19:42:44.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.085+0000 7f5ab9d9f700 1 --2- 172.21.15.17:0/2386494503 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ab40aa760 0x7f5ab413c600 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:44.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.085+0000 7f5ab9d9f700 1 --2- 172.21.15.17:0/2386494503 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5ab40b6a90 0x7f5ab40acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:44.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.085+0000 7f5ab9d9f700 1 -- 172.21.15.17:0/2386494503 >> 172.21.15.17:0/2386494503 conn(0x7f5ab40b3610 msgr2=0x7f5ab40b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:44.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.085+0000 7f5ab9d9f700 1 -- 172.21.15.17:0/2386494503 shutdown_connections 2022-08-01T19:42:44.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.085+0000 7f5ab9d9f700 1 -- 172.21.15.17:0/2386494503 wait complete. 2022-08-01T19:42:44.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.085+0000 7f5ab9d9f700 1 Processor -- start 2022-08-01T19:42:44.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.085+0000 7f5ab9d9f700 1 -- start start 2022-08-01T19:42:44.134 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.085+0000 7f5ab9d9f700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5ab4060aa0 0x7f5ab41301c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:44.134 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.085+0000 7f5ab9d9f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ab40aa760 0x7f5ab4154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:44.135 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.085+0000 7f5ab9d9f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ab40b6a90 0x7f5ab4154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:44.135 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.085+0000 7f5ab9d9f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab40af050 con 0x7f5ab40aa760 2022-08-01T19:42:44.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.085+0000 7f5ab9d9f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab40aedb0 con 0x7f5ab40b6a90 2022-08-01T19:42:44.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.085+0000 7f5ab9d9f700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab4153df0 con 0x7f5ab4060aa0 2022-08-01T19:42:44.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.085+0000 7f5ab2ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ab40aa760 0x7f5ab4154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:44.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.085+0000 7f5ab2ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ab40aa760 0x7f5ab4154290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43786/0 (socket says 172.21.15.17:43786) 2022-08-01T19:42:44.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.085+0000 7f5ab2ffd700 1 -- 172.21.15.17:0/2839743818 learned_addr learned my addr 172.21.15.17:0/2839743818 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:44.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.085+0000 7f5ab3fff700 1 --2- 172.21.15.17:0/2839743818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ab40b6a90 0x7f5ab4154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:44.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.085+0000 7f5ab37fe700 1 --2- 172.21.15.17:0/2839743818 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5ab4060aa0 0x7f5ab41301c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:44.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.085+0000 7f5ab2ffd700 1 -- 172.21.15.17:0/2839743818 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5ab4060aa0 msgr2=0x7f5ab41301c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:44.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.085+0000 7f5ab2ffd700 1 --2- 172.21.15.17:0/2839743818 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5ab4060aa0 0x7f5ab41301c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:44.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.085+0000 7f5ab2ffd700 1 -- 172.21.15.17:0/2839743818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ab40b6a90 msgr2=0x7f5ab4154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:44.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.085+0000 7f5ab2ffd700 1 --2- 172.21.15.17:0/2839743818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ab40b6a90 0x7f5ab4154780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:44.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.085+0000 7f5ab2ffd700 1 -- 172.21.15.17:0/2839743818 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa4009470 con 0x7f5ab40aa760 2022-08-01T19:42:44.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.085+0000 7f5ab37fe700 1 --2- 172.21.15.17:0/2839743818 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5ab4060aa0 0x7f5ab41301c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:42:44.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.085+0000 7f5ab3fff700 1 --2- 172.21.15.17:0/2839743818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ab40b6a90 0x7f5ab4154780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:42:44.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.085+0000 7f5ab2ffd700 1 --2- 172.21.15.17:0/2839743818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ab40aa760 0x7f5ab4154290 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 rx=0x7f5a9c001110 tx=0x7f5a9c007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:44.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.085+0000 7f5ab0ff9700 1 -- 172.21.15.17:0/2839743818 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a9c007e00 con 0x7f5ab40aa760 2022-08-01T19:42:44.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.085+0000 7f5ab9d9f700 1 -- 172.21.15.17:0/2839743818 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aac00b180 con 0x7f5ab40aa760 2022-08-01T19:42:44.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.085+0000 7f5ab0ff9700 1 -- 172.21.15.17:0/2839743818 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9c0072e0 con 0x7f5ab40aa760 2022-08-01T19:42:44.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.085+0000 7f5ab0ff9700 1 -- 172.21.15.17:0/2839743818 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a9c005800 con 0x7f5ab40aa760 2022-08-01T19:42:44.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.085+0000 7f5ab9d9f700 1 -- 172.21.15.17:0/2839743818 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab41308b0 con 0x7f5ab40aa760 2022-08-01T19:42:44.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.089+0000 7f5ab9d9f700 1 -- 172.21.15.17:0/2839743818 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ab40ad5a0 con 0x7f5ab40aa760 2022-08-01T19:42:44.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.089+0000 7f5ab0ff9700 1 -- 172.21.15.17:0/2839743818 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5a9c010070 con 0x7f5ab40aa760 2022-08-01T19:42:44.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.089+0000 7f5ab0ff9700 1 --2- 172.21.15.17:0/2839743818 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5a98053a70 0x7f5a98055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:44.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.089+0000 7f5ab0ff9700 1 -- 172.21.15.17:0/2839743818 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f5a9c06b900 con 0x7f5ab40aa760 2022-08-01T19:42:44.154 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.089+0000 7f5ab37fe700 1 --2- 172.21.15.17:0/2839743818 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5a98053a70 0x7f5a98055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:44.154 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.093+0000 7f5ab37fe700 1 --2- 172.21.15.17:0/2839743818 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5a98053a70 0x7f5a98055f30 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7f5aa400b000 tx=0x7f5aa4008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:44.154 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.097+0000 7f5ab0ff9700 1 -- 172.21.15.17:0/2839743818 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5a9c042480 con 0x7f5ab40aa760 2022-08-01T19:42:44.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.229+0000 7f5ab9d9f700 1 -- 172.21.15.17:0/2839743818 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5ab4061b50 con 0x7f5ab40aa760 2022-08-01T19:42:44.938 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.933+0000 7f5ab0ff9700 1 -- 172.21.15.17:0/2839743818 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v126) v1 ==== 185+0+0 (secure 0 0 0) 0x7f5a9c018020 con 0x7f5ab40aa760 2022-08-01T19:42:44.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.941+0000 7f5ab9d9f700 1 -- 172.21.15.17:0/2839743818 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5a98053a70 msgr2=0x7f5a98055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:44.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.941+0000 7f5ab9d9f700 1 --2- 172.21.15.17:0/2839743818 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5a98053a70 0x7f5a98055f30 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7f5aa400b000 tx=0x7f5aa4008040).stop 2022-08-01T19:42:44.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.941+0000 7f5ab9d9f700 1 -- 172.21.15.17:0/2839743818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ab40aa760 msgr2=0x7f5ab4154290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:44.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.941+0000 7f5ab9d9f700 1 --2- 172.21.15.17:0/2839743818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ab40aa760 0x7f5ab4154290 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 rx=0x7f5a9c001110 tx=0x7f5a9c007aa0).stop 2022-08-01T19:42:44.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.941+0000 7f5ab9d9f700 1 -- 172.21.15.17:0/2839743818 shutdown_connections 2022-08-01T19:42:44.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.941+0000 7f5ab9d9f700 1 --2- 172.21.15.17:0/2839743818 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5a98053a70 0x7f5a98055f30 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:44.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.941+0000 7f5ab9d9f700 1 --2- 172.21.15.17:0/2839743818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ab40b6a90 0x7f5ab4154780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:44.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.941+0000 7f5ab9d9f700 1 --2- 172.21.15.17:0/2839743818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ab40aa760 0x7f5ab4154290 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:44.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.941+0000 7f5ab9d9f700 1 --2- 172.21.15.17:0/2839743818 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5ab4060aa0 0x7f5ab41301c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:44.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.941+0000 7f5ab9d9f700 1 -- 172.21.15.17:0/2839743818 >> 172.21.15.17:0/2839743818 conn(0x7f5ab40b3610 msgr2=0x7f5ab40b4bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:44.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.941+0000 7f5ab9d9f700 1 -- 172.21.15.17:0/2839743818 shutdown_connections 2022-08-01T19:42:44.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:44.941+0000 7f5ab9d9f700 1 -- 172.21.15.17:0/2839743818 wait complete. 2022-08-01T19:42:44.971 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' removed 2022-08-01T19:42:44.972 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-08-01T19:42:45.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.089+0000 7f7942450700 1 Processor -- start 2022-08-01T19:42:45.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.089+0000 7f7942450700 1 -- start start 2022-08-01T19:42:45.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.089+0000 7f7942450700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f793c0b2710 0x7f793c14afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:45.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.089+0000 7f7942450700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f793c0b01a0 0x7f793c14b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:45.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.089+0000 7f7942450700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f793c0a7250 con 0x7f793c0afd60 2022-08-01T19:42:45.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.089+0000 7f7942450700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f793c0a7480 con 0x7f793c0b2710 2022-08-01T19:42:45.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.089+0000 7f7942450700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f793c0a7720 con 0x7f793c0b01a0 2022-08-01T19:42:45.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.089+0000 7f794144e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f793c0b2710 0x7f793c14afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:45.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.089+0000 7f794144e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f793c0b2710 0x7f793c14afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43792/0 (socket says 172.21.15.17:43792) 2022-08-01T19:42:45.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.089+0000 7f794144e700 1 -- 172.21.15.17:0/2230857905 learned_addr learned my addr 172.21.15.17:0/2230857905 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:45.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.089+0000 7f7940c4d700 1 --2- 172.21.15.17:0/2230857905 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f793c0b01a0 0x7f793c14b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:45.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.089+0000 7f794144e700 1 -- 172.21.15.17:0/2230857905 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f793c0b01a0 msgr2=0x7f793c14b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:45.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.089+0000 7f794144e700 1 --2- 172.21.15.17:0/2230857905 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f793c0b01a0 0x7f793c14b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:45.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.089+0000 7f794144e700 1 -- 172.21.15.17:0/2230857905 >> v1:172.21.15.77:6789/0 conn(0x7f793c0afd60 legacy=0x7f793c0a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:45.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.089+0000 7f794144e700 1 -- 172.21.15.17:0/2230857905 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793c0a79c0 con 0x7f793c0b2710 2022-08-01T19:42:45.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.089+0000 7f794144e700 1 --2- 172.21.15.17:0/2230857905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f793c0b2710 0x7f793c14afd0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 rx=0x7f792c002f70 tx=0x7f792c006eb0).ready entity=mon.0 client_cookie=567851b71099a331 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:45.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.089+0000 7f793bfff700 1 -- 172.21.15.17:0/2230857905 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f792c00d5c0 con 0x7f793c0b2710 2022-08-01T19:42:45.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.089+0000 7f793bfff700 1 -- 172.21.15.17:0/2230857905 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f792c000e90 con 0x7f793c0b2710 2022-08-01T19:42:45.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.089+0000 7f793bfff700 1 -- 172.21.15.17:0/2230857905 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f792c00d950 con 0x7f793c0b2710 2022-08-01T19:42:45.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.093+0000 7f7942450700 1 -- 172.21.15.17:0/2230857905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f793c0b2710 msgr2=0x7f793c14afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:45.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.093+0000 7f7942450700 1 --2- 172.21.15.17:0/2230857905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f793c0b2710 0x7f793c14afd0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 rx=0x7f792c002f70 tx=0x7f792c006eb0).stop 2022-08-01T19:42:45.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.093+0000 7f7942450700 1 -- 172.21.15.17:0/2230857905 shutdown_connections 2022-08-01T19:42:45.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.093+0000 7f7942450700 1 --2- 172.21.15.17:0/2230857905 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f793c0b01a0 0x7f793c14b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:45.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.093+0000 7f7942450700 1 --2- 172.21.15.17:0/2230857905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f793c0b2710 0x7f793c14afd0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:45.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.093+0000 7f7942450700 1 -- 172.21.15.17:0/2230857905 >> 172.21.15.17:0/2230857905 conn(0x7f793c060310 msgr2=0x7f793c0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:45.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.093+0000 7f7942450700 1 -- 172.21.15.17:0/2230857905 shutdown_connections 2022-08-01T19:42:45.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.093+0000 7f7942450700 1 -- 172.21.15.17:0/2230857905 wait complete. 2022-08-01T19:42:45.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.093+0000 7f7942450700 1 Processor -- start 2022-08-01T19:42:45.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.093+0000 7f7942450700 1 -- start start 2022-08-01T19:42:45.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.093+0000 7f7942450700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f793c0afd60 0x7f793c0aab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:45.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.093+0000 7f7942450700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f793c0b01a0 0x7f793c0af000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:45.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.093+0000 7f7942450700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f793c0b2710 0x7f793c0ab100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:45.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.093+0000 7f7942450700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793c0ac370 con 0x7f793c0b01a0 2022-08-01T19:42:45.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.093+0000 7f7942450700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793c0b3940 con 0x7f793c0afd60 2022-08-01T19:42:45.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.093+0000 7f7942450700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f793c0b3b70 con 0x7f793c0b2710 2022-08-01T19:42:45.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.097+0000 7f7940c4d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f793c0b01a0 0x7f793c0af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:45.135 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.097+0000 7f7940c4d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f793c0b01a0 0x7f793c0af000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43800/0 (socket says 172.21.15.17:43800) 2022-08-01T19:42:45.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.097+0000 7f7940c4d700 1 -- 172.21.15.17:0/3366505294 learned_addr learned my addr 172.21.15.17:0/3366505294 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:45.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.097+0000 7f794144e700 1 --2- 172.21.15.17:0/3366505294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f793c0afd60 0x7f793c0aab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:45.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.097+0000 7f7941c4f700 1 --2- 172.21.15.17:0/3366505294 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f793c0b2710 0x7f793c0ab100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:45.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.097+0000 7f7940c4d700 1 -- 172.21.15.17:0/3366505294 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f793c0b2710 msgr2=0x7f793c0ab100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:45.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.097+0000 7f7940c4d700 1 --2- 172.21.15.17:0/3366505294 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f793c0b2710 0x7f793c0ab100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:45.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.097+0000 7f7940c4d700 1 -- 172.21.15.17:0/3366505294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f793c0afd60 msgr2=0x7f793c0aab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:45.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.097+0000 7f7940c4d700 1 --2- 172.21.15.17:0/3366505294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f793c0afd60 0x7f793c0aab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:45.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.097+0000 7f7940c4d700 1 -- 172.21.15.17:0/3366505294 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f792c00b1e0 con 0x7f793c0b01a0 2022-08-01T19:42:45.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.097+0000 7f7940c4d700 1 --2- 172.21.15.17:0/3366505294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f793c0b01a0 0x7f793c0af000 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 rx=0x7f793c0ac4b0 tx=0x7f7930008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:45.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.097+0000 7f793a7fc700 1 -- 172.21.15.17:0/3366505294 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7930013620 con 0x7f793c0b01a0 2022-08-01T19:42:45.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.097+0000 7f793a7fc700 1 -- 172.21.15.17:0/3366505294 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7930013780 con 0x7f793c0b01a0 2022-08-01T19:42:45.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.097+0000 7f793a7fc700 1 -- 172.21.15.17:0/3366505294 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7930013ae0 con 0x7f793c0b01a0 2022-08-01T19:42:45.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.097+0000 7f7942450700 1 -- 172.21.15.17:0/3366505294 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f793c0aa420 con 0x7f793c0b01a0 2022-08-01T19:42:45.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.097+0000 7f7942450700 1 -- 172.21.15.17:0/3366505294 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f793c0af630 con 0x7f793c0b01a0 2022-08-01T19:42:45.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.097+0000 7f7942450700 1 -- 172.21.15.17:0/3366505294 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f793c0574e0 con 0x7f793c0b01a0 2022-08-01T19:42:45.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.097+0000 7f793a7fc700 1 -- 172.21.15.17:0/3366505294 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7930013c40 con 0x7f793c0b01a0 2022-08-01T19:42:45.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.101+0000 7f793a7fc700 1 --2- 172.21.15.17:0/3366505294 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f791c053a70 0x7f791c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:45.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.101+0000 7f793a7fc700 1 -- 172.21.15.17:0/3366505294 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f793006c340 con 0x7f793c0b01a0 2022-08-01T19:42:45.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.101+0000 7f794144e700 1 --2- 172.21.15.17:0/3366505294 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f791c053a70 0x7f791c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:45.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.101+0000 7f794144e700 1 --2- 172.21.15.17:0/3366505294 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f791c053a70 0x7f791c055f30 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7f792c01c000 tx=0x7f792c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:45.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.105+0000 7f793a7fc700 1 -- 172.21.15.17:0/3366505294 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7930043150 con 0x7f793c0b01a0 2022-08-01T19:42:45.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:45.233+0000 7f7942450700 1 -- 172.21.15.17:0/3366505294 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f793c0aa8c0 con 0x7f793c0b01a0 2022-08-01T19:42:46.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.229+0000 7f793a7fc700 1 -- 172.21.15.17:0/3366505294 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v127) v1 ==== 133+0+0 (secure 0 0 0) 0x7f7930046770 con 0x7f793c0b01a0 2022-08-01T19:42:46.236 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.233+0000 7f7942450700 1 -- 172.21.15.17:0/3366505294 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f791c053a70 msgr2=0x7f791c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:46.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.233+0000 7f7942450700 1 --2- 172.21.15.17:0/3366505294 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f791c053a70 0x7f791c055f30 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7f792c01c000 tx=0x7f792c015000).stop 2022-08-01T19:42:46.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.233+0000 7f7942450700 1 -- 172.21.15.17:0/3366505294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f793c0b01a0 msgr2=0x7f793c0af000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:46.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.233+0000 7f7942450700 1 --2- 172.21.15.17:0/3366505294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f793c0b01a0 0x7f793c0af000 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 rx=0x7f793c0ac4b0 tx=0x7f7930008040).stop 2022-08-01T19:42:46.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.233+0000 7f7942450700 1 -- 172.21.15.17:0/3366505294 shutdown_connections 2022-08-01T19:42:46.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.233+0000 7f7942450700 1 --2- 172.21.15.17:0/3366505294 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f791c053a70 0x7f791c055f30 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:46.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.233+0000 7f7942450700 1 --2- 172.21.15.17:0/3366505294 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f793c0b2710 0x7f793c0ab100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:46.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.233+0000 7f7942450700 1 --2- 172.21.15.17:0/3366505294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f793c0b01a0 0x7f793c0af000 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:46.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.233+0000 7f7942450700 1 --2- 172.21.15.17:0/3366505294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f793c0afd60 0x7f793c0aab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:46.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.233+0000 7f7942450700 1 -- 172.21.15.17:0/3366505294 >> 172.21.15.17:0/3366505294 conn(0x7f793c060310 msgr2=0x7f793c14a090 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:46.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.233+0000 7f7942450700 1 -- 172.21.15.17:0/3366505294 shutdown_connections 2022-08-01T19:42:46.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.233+0000 7f7942450700 1 -- 172.21.15.17:0/3366505294 wait complete. 2022-08-01T19:42:46.242 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' created 2022-08-01T19:42:46.263 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2022-08-01T19:42:46.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.389+0000 7f454b6fc700 1 Processor -- start 2022-08-01T19:42:46.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.389+0000 7f454b6fc700 1 -- start start 2022-08-01T19:42:46.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.389+0000 7f454b6fc700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f45440b1580 0x7f45440b3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:46.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.389+0000 7f454b6fc700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f4544147450 0x7f4544143740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:46.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.389+0000 7f454b6fc700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45440ae7e0 con 0x7f45440b1ef0 2022-08-01T19:42:46.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.389+0000 7f454b6fc700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45440b1960 con 0x7f45440b1580 2022-08-01T19:42:46.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.389+0000 7f454b6fc700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45440b2470 con 0x7f4544147450 2022-08-01T19:42:46.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.389+0000 7f4549498700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f45440b1580 0x7f45440b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:46.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.389+0000 7f4549498700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f45440b1580 0x7f45440b3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43806/0 (socket says 172.21.15.17:43806) 2022-08-01T19:42:46.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.389+0000 7f4549498700 1 -- 172.21.15.17:0/1913767836 learned_addr learned my addr 172.21.15.17:0/1913767836 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:46.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.389+0000 7f4549c99700 1 --2- 172.21.15.17:0/1913767836 >> v2:172.21.15.77:3300/0 conn(0x7f4544147450 0x7f4544143740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:46.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.393+0000 7f4549498700 1 -- 172.21.15.17:0/1913767836 >> v2:172.21.15.77:3300/0 conn(0x7f4544147450 msgr2=0x7f4544143740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:46.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.393+0000 7f4549498700 1 --2- 172.21.15.17:0/1913767836 >> v2:172.21.15.77:3300/0 conn(0x7f4544147450 0x7f4544143740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:46.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.393+0000 7f4549498700 1 -- 172.21.15.17:0/1913767836 >> v1:172.21.15.77:6789/0 conn(0x7f45440b1ef0 legacy=0x7f4544146d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:46.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.393+0000 7f4549498700 1 -- 172.21.15.17:0/1913767836 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45440b2710 con 0x7f45440b1580 2022-08-01T19:42:46.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.393+0000 7f4549498700 1 --2- 172.21.15.17:0/1913767836 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45440b1580 0x7f45440b3210 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 rx=0x7f4534002f70 tx=0x7f4534006eb0).ready entity=mon.0 client_cookie=8d97b77690bf0889 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:46.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.393+0000 7f4533fff700 1 -- 172.21.15.17:0/1913767836 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f453400d5c0 con 0x7f45440b1580 2022-08-01T19:42:46.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.393+0000 7f4533fff700 1 -- 172.21.15.17:0/1913767836 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4534000e90 con 0x7f45440b1580 2022-08-01T19:42:46.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.393+0000 7f4533fff700 1 -- 172.21.15.17:0/1913767836 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f453400d950 con 0x7f45440b1580 2022-08-01T19:42:46.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.393+0000 7f454b6fc700 1 -- 172.21.15.17:0/1913767836 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45440b1580 msgr2=0x7f45440b3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:46.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.393+0000 7f454b6fc700 1 --2- 172.21.15.17:0/1913767836 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45440b1580 0x7f45440b3210 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 rx=0x7f4534002f70 tx=0x7f4534006eb0).stop 2022-08-01T19:42:46.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.393+0000 7f454b6fc700 1 -- 172.21.15.17:0/1913767836 shutdown_connections 2022-08-01T19:42:46.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.393+0000 7f454b6fc700 1 --2- 172.21.15.17:0/1913767836 >> v2:172.21.15.77:3300/0 conn(0x7f4544147450 0x7f4544143740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:46.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.393+0000 7f454b6fc700 1 --2- 172.21.15.17:0/1913767836 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45440b1580 0x7f45440b3210 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:46.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.393+0000 7f454b6fc700 1 -- 172.21.15.17:0/1913767836 >> 172.21.15.17:0/1913767836 conn(0x7f4544060950 msgr2=0x7f4544060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:46.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.393+0000 7f454b6fc700 1 -- 172.21.15.17:0/1913767836 shutdown_connections 2022-08-01T19:42:46.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.393+0000 7f454b6fc700 1 -- 172.21.15.17:0/1913767836 wait complete. 2022-08-01T19:42:46.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.397+0000 7f454b6fc700 1 Processor -- start 2022-08-01T19:42:46.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.397+0000 7f454b6fc700 1 -- start start 2022-08-01T19:42:46.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.397+0000 7f454b6fc700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f45440b1580 0x7f4544067850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:46.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.397+0000 7f454b6fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45440b1ef0 0x7f45440aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:46.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.397+0000 7f454b6fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4544147450 0x7f45440aaba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:46.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.397+0000 7f454b6fc700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45440673b0 con 0x7f4544147450 2022-08-01T19:42:46.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.397+0000 7f454b6fc700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4544156040 con 0x7f45440b1ef0 2022-08-01T19:42:46.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.397+0000 7f454b6fc700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45441562e0 con 0x7f45440b1580 2022-08-01T19:42:46.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.397+0000 7f4549c99700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4544147450 0x7f45440aaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:46.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.397+0000 7f4549498700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f45440b1580 0x7f4544067850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:46.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.397+0000 7f4548c97700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45440b1ef0 0x7f45440aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:46.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.397+0000 7f4549c99700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4544147450 0x7f45440aaba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43816/0 (socket says 172.21.15.17:43816) 2022-08-01T19:42:46.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.397+0000 7f4549c99700 1 -- 172.21.15.17:0/3278828748 learned_addr learned my addr 172.21.15.17:0/3278828748 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:46.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.397+0000 7f4549c99700 1 -- 172.21.15.17:0/3278828748 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f45440b1580 msgr2=0x7f4544067850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:46.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.397+0000 7f4549c99700 1 --2- 172.21.15.17:0/3278828748 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f45440b1580 0x7f4544067850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:46.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.397+0000 7f4549c99700 1 -- 172.21.15.17:0/3278828748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45440b1ef0 msgr2=0x7f45440aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:46.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.401+0000 7f4549c99700 1 --2- 172.21.15.17:0/3278828748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45440b1ef0 0x7f45440aa6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:46.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.401+0000 7f4549c99700 1 -- 172.21.15.17:0/3278828748 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f453400b1e0 con 0x7f4544147450 2022-08-01T19:42:46.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.401+0000 7f4548c97700 1 --2- 172.21.15.17:0/3278828748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45440b1ef0 0x7f45440aa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:42:46.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.401+0000 7f4549c99700 1 --2- 172.21.15.17:0/3278828748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4544147450 0x7f45440aaba0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 rx=0x7f453c006e00 tx=0x7f453c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:46.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.401+0000 7f45327fc700 1 -- 172.21.15.17:0/3278828748 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f453c012600 con 0x7f4544147450 2022-08-01T19:42:46.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.401+0000 7f45327fc700 1 -- 172.21.15.17:0/3278828748 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453c012760 con 0x7f4544147450 2022-08-01T19:42:46.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.401+0000 7f45327fc700 1 -- 172.21.15.17:0/3278828748 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f453c012ac0 con 0x7f4544147450 2022-08-01T19:42:46.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.401+0000 7f454b6fc700 1 -- 172.21.15.17:0/3278828748 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4544156580 con 0x7f4544147450 2022-08-01T19:42:46.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.401+0000 7f454b6fc700 1 -- 172.21.15.17:0/3278828748 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45440ac190 con 0x7f4544147450 2022-08-01T19:42:46.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.401+0000 7f454b6fc700 1 -- 172.21.15.17:0/3278828748 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45440b29b0 con 0x7f4544147450 2022-08-01T19:42:46.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.401+0000 7f45327fc700 1 -- 172.21.15.17:0/3278828748 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f453c022070 con 0x7f4544147450 2022-08-01T19:42:46.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.401+0000 7f45327fc700 1 --2- 172.21.15.17:0/3278828748 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4524053b00 0x7f4524055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:46.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.401+0000 7f45327fc700 1 -- 172.21.15.17:0/3278828748 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6422+0+0 (secure 0 0 0) 0x7f453c06c5a0 con 0x7f4544147450 2022-08-01T19:42:46.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.405+0000 7f4549498700 1 --2- 172.21.15.17:0/3278828748 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4524053b00 0x7f4524055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:46.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.405+0000 7f4549498700 1 --2- 172.21.15.17:0/3278828748 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4524053b00 0x7f4524055fc0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f453401c000 tx=0x7f4534015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:46.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.409+0000 7f45327fc700 1 -- 172.21.15.17:0/3278828748 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f453c0432c0 con 0x7f4544147450 2022-08-01T19:42:46.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.573+0000 7f454b6fc700 1 -- 172.21.15.17:0/3278828748 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4544156e60 con 0x7f4544147450 2022-08-01T19:42:46.576 INFO:tasks.ceph.mon.a.smithi017.stderr:2022-08-01T19:42:46.573+0000 7f6a50f6e700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-08-01T19:42:46.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.581+0000 7f45327fc700 1 -- 172.21.15.17:0/3278828748 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 22 v63) v1 ==== 174+0+0 (secure 0 0 0) 0x7f453c0160c0 con 0x7f4544147450 2022-08-01T19:42:46.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.585+0000 7f454b6fc700 1 -- 172.21.15.17:0/3278828748 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4524053b00 msgr2=0x7f4524055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:46.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.585+0000 7f454b6fc700 1 --2- 172.21.15.17:0/3278828748 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4524053b00 0x7f4524055fc0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f453401c000 tx=0x7f4534015000).stop 2022-08-01T19:42:46.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.585+0000 7f454b6fc700 1 -- 172.21.15.17:0/3278828748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4544147450 msgr2=0x7f45440aaba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:46.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.585+0000 7f454b6fc700 1 --2- 172.21.15.17:0/3278828748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4544147450 0x7f45440aaba0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 rx=0x7f453c006e00 tx=0x7f453c0091d0).stop 2022-08-01T19:42:46.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.585+0000 7f454b6fc700 1 -- 172.21.15.17:0/3278828748 shutdown_connections 2022-08-01T19:42:46.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.585+0000 7f454b6fc700 1 --2- 172.21.15.17:0/3278828748 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4524053b00 0x7f4524055fc0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:46.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.585+0000 7f454b6fc700 1 --2- 172.21.15.17:0/3278828748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4544147450 0x7f45440aaba0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:46.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.585+0000 7f454b6fc700 1 --2- 172.21.15.17:0/3278828748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45440b1ef0 0x7f45440aa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:46.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.585+0000 7f454b6fc700 1 --2- 172.21.15.17:0/3278828748 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f45440b1580 0x7f4544067850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:46.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.585+0000 7f454b6fc700 1 -- 172.21.15.17:0/3278828748 >> 172.21.15.17:0/3278828748 conn(0x7f4544060950 msgr2=0x7f4544143d40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:46.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.585+0000 7f454b6fc700 1 -- 172.21.15.17:0/3278828748 shutdown_connections 2022-08-01T19:42:46.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.585+0000 7f454b6fc700 1 -- 172.21.15.17:0/3278828748 wait complete. 2022-08-01T19:42:46.602 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 24 and data pool 22 2022-08-01T19:42:46.606 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T19:42:46.608 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:42:46.625 DEBUG:teuthology.orchestra.run.smithi077:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:42:46.672 DEBUG:teuthology.orchestra.run.smithi085:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:42:46.704 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:42:46.731 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T19:42:46.732 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:42:46.732 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:42:46.733 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:42:46.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:42:46.760 INFO:teuthology.orchestra.run.smithi085.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-08-01T19:42:46.761 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-08-01T19:42:46.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T19:42:46.764 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:42:46.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:42:46.778 INFO:teuthology.orchestra.run.smithi085.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-08-01T19:42:46.779 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:42:46.779 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:42:46.780 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:42:46.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:42:46.809 INFO:teuthology.orchestra.run.smithi190.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T19:42:46.814 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-08-01T19:42:46.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T19:42:46.817 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:42:46.826 INFO:teuthology.orchestra.run.smithi190.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T19:42:46.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:42:46.829 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T19:42:46.951 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.945+0000 7f63630ba700 1 Processor -- start 2022-08-01T19:42:46.951 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.945+0000 7f63630ba700 1 -- start start 2022-08-01T19:42:46.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.945+0000 7f63630ba700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f635c060aa0 0x7f635c0acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:46.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.945+0000 7f63630ba700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f635c0b6e70 con 0x7f635c0aa760 2022-08-01T19:42:46.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.945+0000 7f63630ba700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f635c061370 con 0x7f635c0b6a90 2022-08-01T19:42:46.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.945+0000 7f63630ba700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f635c061610 con 0x7f635c060aa0 2022-08-01T19:42:46.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.945+0000 7f6361657700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f635c0aa760 0x7f635c13c600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36018/0 (socket says 172.21.15.17:36018) 2022-08-01T19:42:46.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.945+0000 7f6361657700 1 -- 172.21.15.17:0/3785949447 learned_addr learned my addr 172.21.15.17:0/3785949447 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:46.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.945+0000 7f6353fff700 1 --2- 172.21.15.17:0/3785949447 >> v2:172.21.15.77:3300/0 conn(0x7f635c060aa0 0x7f635c0acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:46.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.945+0000 7f63537fe700 1 -- 172.21.15.17:0/3785949447 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1972231849 0 0) 0x7f635c0b6e70 con 0x7f635c0aa760 2022-08-01T19:42:46.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.945+0000 7f63537fe700 1 -- 172.21.15.17:0/3785949447 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6348003260 con 0x7f635c0aa760 2022-08-01T19:42:46.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.945+0000 7f6353fff700 1 -- 172.21.15.17:0/3785949447 >> v1:172.21.15.77:6789/0 conn(0x7f635c0b6a90 legacy=0x7f635c0aa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:46.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.945+0000 7f6353fff700 1 -- 172.21.15.17:0/3785949447 >> v1:172.21.15.17:6789/0 conn(0x7f635c0aa760 legacy=0x7f635c13c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:46.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.945+0000 7f6353fff700 1 -- 172.21.15.17:0/3785949447 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f635c0618b0 con 0x7f635c060aa0 2022-08-01T19:42:46.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.945+0000 7f6353fff700 1 --2- 172.21.15.17:0/3785949447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f635c060aa0 0x7f635c0acee0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f634c002f70 tx=0x7f634c006df0).ready entity=mon.1 client_cookie=4770992345192a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:46.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.945+0000 7f63537fe700 1 -- 172.21.15.17:0/3785949447 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f634c00fab0 con 0x7f635c060aa0 2022-08-01T19:42:46.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.945+0000 7f63537fe700 1 -- 172.21.15.17:0/3785949447 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f634c000e90 con 0x7f635c060aa0 2022-08-01T19:42:46.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.945+0000 7f63537fe700 1 -- 172.21.15.17:0/3785949447 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f634c00fe00 con 0x7f635c060aa0 2022-08-01T19:42:46.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.945+0000 7f63630ba700 1 -- 172.21.15.17:0/3785949447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f635c060aa0 msgr2=0x7f635c0acee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:46.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.945+0000 7f63630ba700 1 --2- 172.21.15.17:0/3785949447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f635c060aa0 0x7f635c0acee0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f634c002f70 tx=0x7f634c006df0).stop 2022-08-01T19:42:46.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.945+0000 7f63630ba700 1 -- 172.21.15.17:0/3785949447 shutdown_connections 2022-08-01T19:42:46.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.945+0000 7f63630ba700 1 --2- 172.21.15.17:0/3785949447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f635c060aa0 0x7f635c0acee0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:46.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.945+0000 7f63630ba700 1 -- 172.21.15.17:0/3785949447 >> 172.21.15.17:0/3785949447 conn(0x7f635c0b3610 msgr2=0x7f635c0b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:46.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.945+0000 7f63630ba700 1 -- 172.21.15.17:0/3785949447 shutdown_connections 2022-08-01T19:42:46.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.945+0000 7f63630ba700 1 -- 172.21.15.17:0/3785949447 wait complete. 2022-08-01T19:42:46.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.949+0000 7f63630ba700 1 Processor -- start 2022-08-01T19:42:46.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.949+0000 7f63630ba700 1 -- start start 2022-08-01T19:42:46.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.949+0000 7f63630ba700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f635c060aa0 0x7f635c13e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:46.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.949+0000 7f63630ba700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f635c0aa760 0x7f635c1340b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:46.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.949+0000 7f63630ba700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f635c0b6a90 0x7f635c1345a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:46.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.949+0000 7f63630ba700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635c141fa0 con 0x7f635c0b6a90 2022-08-01T19:42:46.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.949+0000 7f63630ba700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635c13ef00 con 0x7f635c060aa0 2022-08-01T19:42:46.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.949+0000 7f63630ba700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f635c0b04e0 con 0x7f635c0aa760 2022-08-01T19:42:46.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.949+0000 7f6361657700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f635c0b6a90 0x7f635c1345a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:46.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.949+0000 7f6361657700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f635c0b6a90 0x7f635c1345a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43830/0 (socket says 172.21.15.17:43830) 2022-08-01T19:42:46.991 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.949+0000 7f6361657700 1 -- 172.21.15.17:0/961486215 learned_addr learned my addr 172.21.15.17:0/961486215 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:46.991 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.949+0000 7f6360e56700 1 --2- 172.21.15.17:0/961486215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f635c060aa0 0x7f635c13e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:46.992 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.949+0000 7f6353fff700 1 --2- 172.21.15.17:0/961486215 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f635c0aa760 0x7f635c1340b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:46.992 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.949+0000 7f6361657700 1 -- 172.21.15.17:0/961486215 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f635c0aa760 msgr2=0x7f635c1340b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:46.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.949+0000 7f6361657700 1 --2- 172.21.15.17:0/961486215 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f635c0aa760 0x7f635c1340b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:46.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.949+0000 7f6361657700 1 -- 172.21.15.17:0/961486215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f635c060aa0 msgr2=0x7f635c13e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:46.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.949+0000 7f6361657700 1 --2- 172.21.15.17:0/961486215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f635c060aa0 0x7f635c13e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:46.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.949+0000 7f6361657700 1 -- 172.21.15.17:0/961486215 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f634c00b180 con 0x7f635c0b6a90 2022-08-01T19:42:46.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.949+0000 7f6361657700 1 --2- 172.21.15.17:0/961486215 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f635c0b6a90 0x7f635c1345a0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 rx=0x7f6354001490 tx=0x7f6354007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:46.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.949+0000 7f6351ffb700 1 -- 172.21.15.17:0/961486215 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6354005ba0 con 0x7f635c0b6a90 2022-08-01T19:42:46.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.949+0000 7f6351ffb700 1 -- 172.21.15.17:0/961486215 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6354005d00 con 0x7f635c0b6a90 2022-08-01T19:42:46.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.949+0000 7f6351ffb700 1 -- 172.21.15.17:0/961486215 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63540072c0 con 0x7f635c0b6a90 2022-08-01T19:42:46.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.949+0000 7f63630ba700 1 -- 172.21.15.17:0/961486215 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f635c13e000 con 0x7f635c0b6a90 2022-08-01T19:42:46.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.949+0000 7f63630ba700 1 -- 172.21.15.17:0/961486215 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f635c13f2a0 con 0x7f635c0b6a90 2022-08-01T19:42:46.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.953+0000 7f63630ba700 1 -- 172.21.15.17:0/961486215 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f635c061b50 con 0x7f635c0b6a90 2022-08-01T19:42:46.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.953+0000 7f6351ffb700 1 -- 172.21.15.17:0/961486215 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6354010070 con 0x7f635c0b6a90 2022-08-01T19:42:46.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.953+0000 7f6351ffb700 1 --2- 172.21.15.17:0/961486215 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f633c0539e0 0x7f633c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:46.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.953+0000 7f6351ffb700 1 -- 172.21.15.17:0/961486215 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f635406c250 con 0x7f635c0b6a90 2022-08-01T19:42:46.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.953+0000 7f6360e56700 1 --2- 172.21.15.17:0/961486215 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f633c0539e0 0x7f633c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:47.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.953+0000 7f6360e56700 1 --2- 172.21.15.17:0/961486215 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f633c0539e0 0x7f633c055ea0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f635800d000 tx=0x7f6358009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:47.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:46.961+0000 7f6351ffb700 1 -- 172.21.15.17:0/961486215 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6354042eb0 con 0x7f635c0b6a90 2022-08-01T19:42:47.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:47.093+0000 7f63630ba700 1 -- 172.21.15.17:0/961486215 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f635c0574e0 con 0x7f635c0b6a90 2022-08-01T19:42:47.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:47.093+0000 7f6351ffb700 1 -- 172.21.15.17:0/961486215 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v128) v1 ==== 81+0+34218 (secure 0 0 0) 0x7f63540464d0 con 0x7f635c0b6a90 2022-08-01T19:42:47.100 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:42:47.100 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:42:47.100 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 128, 2022-08-01T19:42:47.101 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", 2022-08-01T19:42:47.101 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2022-08-01T19:33:50.105743+0000", 2022-08-01T19:42:47.102 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2022-08-01T19:42:46.576288+0000", 2022-08-01T19:42:47.102 INFO:teuthology.orchestra.run.smithi017.stdout: "last_up_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:42:47.102 INFO:teuthology.orchestra.run.smithi017.stdout: "last_in_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:42:47.103 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T19:42:47.104 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_num": 5799936, 2022-08-01T19:42:47.105 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_set": [ 2022-08-01T19:42:47.105 INFO:teuthology.orchestra.run.smithi017.stdout: "pglog_hardlimit", 2022-08-01T19:42:47.106 INFO:teuthology.orchestra.run.smithi017.stdout: "purged_snapdirs", 2022-08-01T19:42:47.106 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_deletes", 2022-08-01T19:42:47.107 INFO:teuthology.orchestra.run.smithi017.stdout: "sortbitwise" 2022-08-01T19:42:47.107 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:42:47.107 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_version": 6, 2022-08-01T19:42:47.108 INFO:teuthology.orchestra.run.smithi017.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T19:42:47.114 INFO:teuthology.orchestra.run.smithi017.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T19:42:47.114 INFO:teuthology.orchestra.run.smithi017.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T19:42:47.115 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_snapshot": "", 2022-08-01T19:42:47.115 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_max": 24, 2022-08-01T19:42:47.116 INFO:teuthology.orchestra.run.smithi017.stdout: "max_osd": 8, 2022-08-01T19:42:47.116 INFO:teuthology.orchestra.run.smithi017.stdout: "require_min_compat_client": "jewel", 2022-08-01T19:42:47.121 INFO:teuthology.orchestra.run.smithi017.stdout: "min_compat_client": "jewel", 2022-08-01T19:42:47.121 INFO:teuthology.orchestra.run.smithi017.stdout: "require_osd_release": "octopus", 2022-08-01T19:42:47.122 INFO:teuthology.orchestra.run.smithi017.stdout: "pools": [ 2022-08-01T19:42:47.122 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.123 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 1, 2022-08-01T19:42:47.124 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "device_health_metrics", 2022-08-01T19:42:47.124 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:33:53.001841+0000", 2022-08-01T19:42:47.125 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:42:47.125 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:42:47.126 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:42:47.126 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:42:47.127 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:42:47.127 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:42:47.128 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:42:47.128 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:42:47.129 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 1, 2022-08-01T19:42:47.129 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 1, 2022-08-01T19:42:47.130 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 1, 2022-08-01T19:42:47.130 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 1, 2022-08-01T19:42:47.134 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 1, 2022-08-01T19:42:47.134 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:42:47.135 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:42:47.135 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:42:47.135 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:42:47.135 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:42:47.136 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:42:47.136 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:42:47.136 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.136 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "16", 2022-08-01T19:42:47.137 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:42:47.137 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:42:47.137 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:42:47.137 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:42:47.138 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:42:47.138 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:42:47.138 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:42:47.138 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:42:47.139 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:42:47.139 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:42:47.139 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:42:47.139 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:42:47.139 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:42:47.140 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:42:47.140 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:42:47.140 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:42:47.140 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:42:47.141 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:42:47.141 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:42:47.141 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:42:47.141 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:42:47.142 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:42:47.142 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:42:47.142 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:42:47.142 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:42:47.143 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:42:47.143 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.143 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:42:47.143 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:42:47.144 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:42:47.144 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:42:47.144 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:42:47.144 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:42:47.144 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:42:47.145 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:42:47.145 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:42:47.145 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:42:47.145 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:42:47.146 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:42:47.146 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 1 2022-08-01T19:42:47.146 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.146 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:42:47.147 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_devicehealth": {} 2022-08-01T19:42:47.147 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.147 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.147 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.148 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 2, 2022-08-01T19:42:47.155 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "rbd", 2022-08-01T19:42:47.155 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:34:01.553098+0000", 2022-08-01T19:42:47.155 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:42:47.156 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:42:47.156 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:42:47.156 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:42:47.156 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:42:47.157 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:42:47.157 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:42:47.157 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:42:47.157 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:42:47.158 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:42:47.158 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:42:47.158 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:42:47.159 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:42:47.159 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:42:47.159 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:42:47.159 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:42:47.159 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:42:47.160 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:42:47.160 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:42:47.161 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:42:47.161 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.161 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "19", 2022-08-01T19:42:47.163 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:42:47.163 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:42:47.164 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:42:47.164 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:42:47.164 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:42:47.165 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:42:47.165 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:42:47.166 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:42:47.166 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:42:47.167 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:42:47.168 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:42:47.169 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:42:47.169 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:42:47.170 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:42:47.170 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:42:47.171 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:42:47.172 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:42:47.176 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:42:47.176 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:42:47.177 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:42:47.178 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:42:47.179 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:42:47.179 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:42:47.180 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:42:47.193 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:42:47.195 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:42:47.195 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.196 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:42:47.196 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:42:47.197 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:42:47.197 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:42:47.198 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:42:47.207 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:42:47.208 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:42:47.208 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:42:47.209 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:42:47.209 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:42:47.210 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:42:47.210 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:42:47.211 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:42:47.211 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd": {} 2022-08-01T19:42:47.212 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.212 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.213 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.213 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 22, 2022-08-01T19:42:47.214 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "cephfs_data", 2022-08-01T19:42:47.215 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:42:10.264672+0000", 2022-08-01T19:42:47.215 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:42:47.215 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:42:47.216 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:42:47.216 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:42:47.216 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:42:47.217 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:42:47.233 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:42:47.234 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:42:47.234 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:42:47.235 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:42:47.235 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:42:47.236 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:42:47.236 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:42:47.236 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:42:47.237 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:42:47.237 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:42:47.258 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:42:47.258 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:42:47.259 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:42:47.271 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:42:47.271 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.272 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "128", 2022-08-01T19:42:47.272 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:42:47.273 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:42:47.273 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:42:47.278 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:42:47.279 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:42:47.279 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:42:47.280 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:42:47.280 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:42:47.281 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:42:47.281 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:42:47.282 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:42:47.283 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:42:47.283 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:42:47.284 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:42:47.284 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:42:47.284 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:42:47.285 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:42:47.285 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:42:47.286 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:42:47.286 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:42:47.287 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:42:47.287 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:42:47.288 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:42:47.288 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:42:47.289 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:42:47.289 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:42:47.291 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.291 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:42:47.292 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:42:47.292 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:42:47.293 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:42:47.293 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:42:47.294 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:42:47.295 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:42:47.295 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:42:47.296 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:42:47.296 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:42:47.297 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:42:47.297 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:42:47.297 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:42:47.298 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs": { 2022-08-01T19:42:47.298 INFO:teuthology.orchestra.run.smithi017.stdout: "data": "cephfs" 2022-08-01T19:42:47.299 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.299 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.300 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.300 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.300 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 24, 2022-08-01T19:42:47.301 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "cephfs_metadata", 2022-08-01T19:42:47.301 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:42:45.283265+0000", 2022-08-01T19:42:47.321 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 32769, 2022-08-01T19:42:47.322 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool,creating", 2022-08-01T19:42:47.322 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:42:47.323 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:42:47.323 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:42:47.324 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:42:47.324 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:42:47.325 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:42:47.325 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:42:47.325 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:42:47.326 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:42:47.326 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:42:47.327 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:42:47.327 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:42:47.328 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:42:47.328 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:42:47.329 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:42:47.329 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:42:47.330 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:42:47.330 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:42:47.330 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.339 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "128", 2022-08-01T19:42:47.339 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:42:47.340 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:42:47.340 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:42:47.340 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:42:47.340 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:42:47.341 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:42:47.341 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:42:47.341 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:42:47.342 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:42:47.342 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:42:47.342 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:42:47.342 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:42:47.343 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:42:47.343 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:42:47.343 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:42:47.343 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:42:47.344 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:42:47.344 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:42:47.345 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:42:47.345 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:42:47.345 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:42:47.346 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:42:47.346 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:42:47.346 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:42:47.347 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:42:47.347 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:42:47.347 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.348 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:42:47.349 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:42:47.349 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:42:47.349 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:42:47.349 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:42:47.350 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:42:47.355 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:42:47.355 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:42:47.356 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:42:47.356 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:42:47.356 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:42:47.357 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:42:47.357 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_bias": 4, 2022-08-01T19:42:47.357 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 16, 2022-08-01T19:42:47.358 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_priority": 5 2022-08-01T19:42:47.358 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.359 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:42:47.360 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs": { 2022-08-01T19:42:47.360 INFO:teuthology.orchestra.run.smithi017.stdout: "metadata": "cephfs" 2022-08-01T19:42:47.361 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.361 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.362 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.362 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:42:47.362 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2022-08-01T19:42:47.363 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.363 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:42:47.364 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a962f14f-cf16-4fa8-84ff-c85325db9132", 2022-08-01T19:42:47.364 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:42:47.364 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:42:47.366 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:42:47.366 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:42:47.367 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:42:47.367 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:42:47.367 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:42:47.368 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 127, 2022-08-01T19:42:47.368 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:42:47.368 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:42:47.369 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:42:47.370 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:47.371 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.371 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:47.371 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6800", 2022-08-01T19:42:47.372 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:42:47.372 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.373 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.373 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:47.373 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6801", 2022-08-01T19:42:47.374 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:42:47.374 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.375 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.375 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.376 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:42:47.376 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:47.376 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.377 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:47.377 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6802", 2022-08-01T19:42:47.377 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:42:47.378 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.379 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.379 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:47.380 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6803", 2022-08-01T19:42:47.381 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:42:47.381 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.381 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.382 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:42:47.382 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:47.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.382 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:47.383 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6806", 2022-08-01T19:42:47.383 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:42:47.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.383 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.384 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:47.384 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6807", 2022-08-01T19:42:47.395 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:42:47.395 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.396 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.396 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.396 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:42:47.396 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:47.397 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.397 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:47.397 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6804", 2022-08-01T19:42:47.397 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:42:47.398 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.398 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.401 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:47.401 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6805", 2022-08-01T19:42:47.401 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:42:47.402 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.402 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.403 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.403 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6801/23008", 2022-08-01T19:42:47.404 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6803/23008", 2022-08-01T19:42:47.404 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6807/23008", 2022-08-01T19:42:47.405 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6805/23008", 2022-08-01T19:42:47.406 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:42:47.406 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:42:47.407 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:42:47.407 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.408 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.408 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.408 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:42:47.409 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "805f7051-a527-4e98-8844-b6e406a63248", 2022-08-01T19:42:47.409 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:42:47.412 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:42:47.413 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:42:47.413 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:42:47.424 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:42:47.425 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:42:47.426 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:42:47.426 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 127, 2022-08-01T19:42:47.427 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:42:47.427 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:42:47.428 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:42:47.428 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:47.428 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.429 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:47.429 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6808", 2022-08-01T19:42:47.430 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:42:47.430 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.443 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.444 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:47.444 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6809", 2022-08-01T19:42:47.445 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:42:47.445 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.446 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.446 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.447 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:42:47.447 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:47.448 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.448 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:47.449 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6810", 2022-08-01T19:42:47.449 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:42:47.470 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.478 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.478 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:47.483 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6811", 2022-08-01T19:42:47.484 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:42:47.484 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.485 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.485 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.485 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:42:47.486 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:47.486 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.515 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:47.516 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6814", 2022-08-01T19:42:47.516 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:42:47.517 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.517 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.519 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:47.519 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6815", 2022-08-01T19:42:47.519 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:42:47.520 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.520 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.521 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.521 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:42:47.522 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:47.523 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.523 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:47.524 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6812", 2022-08-01T19:42:47.524 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:42:47.525 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.525 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.526 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:47.527 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6813", 2022-08-01T19:42:47.527 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:42:47.528 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.528 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.529 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.529 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6809/23007", 2022-08-01T19:42:47.530 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6811/23007", 2022-08-01T19:42:47.531 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6815/23007", 2022-08-01T19:42:47.532 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6813/23007", 2022-08-01T19:42:47.532 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:42:47.533 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:42:47.533 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:42:47.534 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.534 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.535 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.535 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:42:47.536 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "89f7f3df-7c10-48e1-8b69-9e4243923a61", 2022-08-01T19:42:47.536 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:42:47.537 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:42:47.537 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:42:47.538 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:42:47.538 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:42:47.539 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:42:47.539 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:42:47.540 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 122, 2022-08-01T19:42:47.550 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:42:47.551 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:42:47.558 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:42:47.559 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:47.559 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.560 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:47.560 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6817", 2022-08-01T19:42:47.561 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:42:47.570 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.570 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.571 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:47.571 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6819", 2022-08-01T19:42:47.572 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:42:47.573 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.573 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.574 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.574 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:42:47.575 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:47.575 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.576 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:47.576 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6822", 2022-08-01T19:42:47.587 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:42:47.588 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.588 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.588 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:47.589 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6824", 2022-08-01T19:42:47.589 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:42:47.590 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.590 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.591 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.591 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:42:47.592 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:47.592 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.592 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:47.593 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6830", 2022-08-01T19:42:47.593 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:42:47.594 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.594 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.595 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:47.595 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6831", 2022-08-01T19:42:47.595 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:42:47.596 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.596 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.597 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.597 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:42:47.598 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:47.598 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.598 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:47.599 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6826", 2022-08-01T19:42:47.600 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:42:47.600 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.601 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.601 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:47.602 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6829", 2022-08-01T19:42:47.602 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:42:47.603 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.604 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.604 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.605 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6819/23010", 2022-08-01T19:42:47.605 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6824/23010", 2022-08-01T19:42:47.606 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6831/23010", 2022-08-01T19:42:47.606 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6829/23010", 2022-08-01T19:42:47.607 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:42:47.607 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:42:47.607 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:42:47.608 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.608 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.609 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.609 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:42:47.610 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "96123468-aabc-4a64-909e-a192c369090d", 2022-08-01T19:42:47.610 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:42:47.611 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:42:47.611 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:42:47.612 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:42:47.612 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:42:47.612 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:42:47.613 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:42:47.613 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 127, 2022-08-01T19:42:47.614 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:42:47.615 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:42:47.615 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:42:47.616 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:47.616 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.617 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:47.617 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6816", 2022-08-01T19:42:47.618 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:42:47.618 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.619 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.619 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:47.620 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6818", 2022-08-01T19:42:47.620 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:42:47.620 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.621 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.621 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.627 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:42:47.628 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:47.636 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.636 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:47.636 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6820", 2022-08-01T19:42:47.642 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:42:47.642 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.643 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.643 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:47.644 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6821", 2022-08-01T19:42:47.644 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:42:47.644 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.645 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.645 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.651 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:42:47.651 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:47.651 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.652 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:47.652 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6827", 2022-08-01T19:42:47.653 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:42:47.653 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.653 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.654 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:47.654 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6828", 2022-08-01T19:42:47.655 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:42:47.655 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.656 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.658 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.658 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:42:47.659 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:47.659 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.660 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:47.660 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6823", 2022-08-01T19:42:47.661 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:42:47.661 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.661 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.668 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:47.669 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6825", 2022-08-01T19:42:47.669 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:42:47.670 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.670 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.671 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.671 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6818/23009", 2022-08-01T19:42:47.671 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6821/23009", 2022-08-01T19:42:47.671 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6828/23009", 2022-08-01T19:42:47.672 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6825/23009", 2022-08-01T19:42:47.672 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:42:47.672 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:42:47.673 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:42:47.673 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.674 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.674 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.674 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:42:47.675 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a6e26665-e04b-46ec-ab66-e912acbf044c", 2022-08-01T19:42:47.675 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:42:47.676 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:42:47.677 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:42:47.677 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:42:47.677 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:42:47.679 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:42:47.679 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:42:47.679 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 122, 2022-08-01T19:42:47.680 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:42:47.680 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:42:47.681 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:42:47.681 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:47.681 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.682 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:47.682 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6810", 2022-08-01T19:42:47.683 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:42:47.684 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.684 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.684 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:47.685 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6811", 2022-08-01T19:42:47.685 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:42:47.686 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.686 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.686 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.695 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:42:47.695 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:47.696 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.696 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:47.697 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6812", 2022-08-01T19:42:47.698 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:42:47.699 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.699 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.700 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:47.700 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6813", 2022-08-01T19:42:47.701 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:42:47.701 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.701 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.701 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.702 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:42:47.702 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:47.703 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.703 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:47.703 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6816", 2022-08-01T19:42:47.704 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:42:47.704 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.705 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.705 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:47.705 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6817", 2022-08-01T19:42:47.713 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:42:47.713 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.714 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.714 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.715 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:42:47.715 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:47.716 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.716 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:47.716 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6814", 2022-08-01T19:42:47.717 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:42:47.717 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.717 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.718 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:47.718 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6815", 2022-08-01T19:42:47.719 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:42:47.720 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.720 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.720 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.721 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6811/23584", 2022-08-01T19:42:47.721 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6813/23584", 2022-08-01T19:42:47.721 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6817/23584", 2022-08-01T19:42:47.721 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6815/23584", 2022-08-01T19:42:47.723 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:42:47.723 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:42:47.723 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:42:47.724 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.724 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.724 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.724 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:42:47.725 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "502c0034-18ee-44ac-87ff-19b9dddb66a0", 2022-08-01T19:42:47.725 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:42:47.725 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:42:47.725 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:42:47.726 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:42:47.726 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:42:47.727 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:42:47.727 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:42:47.727 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 127, 2022-08-01T19:42:47.727 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:42:47.728 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:42:47.728 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:42:47.730 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:47.731 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.731 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:47.733 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6802", 2022-08-01T19:42:47.733 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:42:47.734 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.734 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.734 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:47.734 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6803", 2022-08-01T19:42:47.735 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:42:47.735 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.735 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.735 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.736 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:42:47.736 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:47.736 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.736 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:47.737 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6804", 2022-08-01T19:42:47.737 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:42:47.737 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.737 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.738 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:47.738 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6805", 2022-08-01T19:42:47.738 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:42:47.738 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.739 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.739 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.741 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:42:47.741 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:47.741 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.742 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:47.744 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6808", 2022-08-01T19:42:47.744 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:42:47.744 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.745 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.745 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:47.746 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6809", 2022-08-01T19:42:47.746 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:42:47.747 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.747 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.747 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.748 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:42:47.748 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:47.749 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.749 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:47.750 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6806", 2022-08-01T19:42:47.752 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:42:47.752 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.753 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.753 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:47.753 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6807", 2022-08-01T19:42:47.754 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:42:47.754 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.754 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.755 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.755 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6803/23585", 2022-08-01T19:42:47.755 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6805/23585", 2022-08-01T19:42:47.756 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6809/23585", 2022-08-01T19:42:47.756 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6807/23585", 2022-08-01T19:42:47.756 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:42:47.756 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:42:47.756 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:42:47.757 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.757 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.757 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.757 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:42:47.758 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "ac4e755a-7fe7-46cb-8407-852d5af9ffc5", 2022-08-01T19:42:47.758 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:42:47.759 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:42:47.759 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:42:47.759 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:42:47.760 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:42:47.760 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:42:47.762 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:42:47.763 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 127, 2022-08-01T19:42:47.763 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:42:47.764 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:42:47.764 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:42:47.765 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:47.765 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.765 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:47.766 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6820", 2022-08-01T19:42:47.766 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:42:47.767 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.768 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.768 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:47.769 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6821", 2022-08-01T19:42:47.781 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:42:47.781 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.782 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.782 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.783 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:42:47.783 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:47.784 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.798 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:47.799 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6823", 2022-08-01T19:42:47.800 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:42:47.800 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.801 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.801 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:47.802 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6825", 2022-08-01T19:42:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:42:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.804 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:42:47.804 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:47.804 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.804 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:47.805 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6831", 2022-08-01T19:42:47.805 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:42:47.805 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.806 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.806 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:47.806 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6833", 2022-08-01T19:42:47.807 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:42:47.807 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.808 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.808 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.808 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:42:47.809 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:47.809 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.810 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:47.810 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6827", 2022-08-01T19:42:47.810 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:42:47.811 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.811 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.812 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:47.813 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6829", 2022-08-01T19:42:47.813 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:42:47.813 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.814 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.814 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.814 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6821/23587", 2022-08-01T19:42:47.815 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6825/23587", 2022-08-01T19:42:47.815 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6833/23587", 2022-08-01T19:42:47.818 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6829/23587", 2022-08-01T19:42:47.818 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:42:47.819 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:42:47.819 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:42:47.819 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.819 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.820 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.820 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:42:47.820 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "874761e2-6410-40fc-b60c-e58abe5ce489", 2022-08-01T19:42:47.821 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:42:47.821 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:42:47.822 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:42:47.822 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:42:47.823 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:42:47.823 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:42:47.824 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:42:47.824 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 114, 2022-08-01T19:42:47.825 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:42:47.825 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:42:47.826 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:42:47.826 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:47.827 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.827 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:47.827 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6818", 2022-08-01T19:42:47.828 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:42:47.828 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.829 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.829 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:47.830 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6819", 2022-08-01T19:42:47.830 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:42:47.831 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.831 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.832 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.832 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:42:47.832 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:47.833 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.833 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:47.834 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6822", 2022-08-01T19:42:47.834 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:42:47.834 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.835 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.835 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:47.835 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6824", 2022-08-01T19:42:47.836 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:42:47.836 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.836 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.836 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.837 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:42:47.837 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:47.837 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.837 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:47.838 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6830", 2022-08-01T19:42:47.838 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:42:47.838 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.838 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.839 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:47.839 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6832", 2022-08-01T19:42:47.839 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:42:47.839 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.840 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.840 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.840 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:42:47.840 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:47.841 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.841 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:47.842 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6826", 2022-08-01T19:42:47.842 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:42:47.842 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.842 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.843 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:47.843 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6828", 2022-08-01T19:42:47.843 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:42:47.843 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.844 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.844 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.844 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6819/23586", 2022-08-01T19:42:47.844 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6824/23586", 2022-08-01T19:42:47.845 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6832/23586", 2022-08-01T19:42:47.845 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6828/23586", 2022-08-01T19:42:47.845 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:42:47.845 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:42:47.846 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:42:47.846 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:47.846 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.847 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:42:47.847 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_xinfo": [ 2022-08-01T19:42:47.847 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.847 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:42:47.847 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:42:47.848 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:42:47.848 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:42:47.848 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:42:47.849 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:42:47.849 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.588817+0000", 2022-08-01T19:42:47.850 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:42:47.850 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.851 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.851 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:42:47.851 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:42:47.852 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:42:47.852 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:42:47.853 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:42:47.853 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:42:47.854 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.782753+0000", 2022-08-01T19:42:47.854 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:42:47.855 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.855 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.856 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:42:47.856 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:42:47.856 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:42:47.857 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:42:47.857 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:42:47.858 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:42:47.859 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.837924+0000", 2022-08-01T19:42:47.859 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:42:47.860 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.860 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.860 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:42:47.861 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:42:47.861 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:42:47.862 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:42:47.862 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:42:47.863 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:42:47.863 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.839948+0000", 2022-08-01T19:42:47.863 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:42:47.864 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.864 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.864 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:42:47.865 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:42:47.865 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:42:47.866 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:42:47.866 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:42:47.866 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:42:47.867 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.112801+0000", 2022-08-01T19:42:47.867 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:42:47.868 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.868 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.868 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:42:47.869 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:42:47.869 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:42:47.870 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:42:47.870 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:42:47.871 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:42:47.871 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.106704+0000", 2022-08-01T19:42:47.872 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:42:47.872 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.873 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.873 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:42:47.879 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:42:47.879 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:42:47.880 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:42:47.880 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:42:47.881 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:42:47.881 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.893170+0000", 2022-08-01T19:42:47.883 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:42:47.883 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.884 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:47.884 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:42:47.884 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:42:47.885 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:42:47.886 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:42:47.886 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:42:47.887 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:42:47.887 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.886947+0000", 2022-08-01T19:42:47.888 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:42:47.888 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.888 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:42:47.889 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap": [], 2022-08-01T19:42:47.889 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap_items": [], 2022-08-01T19:42:47.896 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_temp": [], 2022-08-01T19:42:47.897 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_temp": [], 2022-08-01T19:42:47.897 INFO:teuthology.orchestra.run.smithi017.stdout: "blacklist": { 2022-08-01T19:42:47.901 INFO:teuthology.orchestra.run.smithi017.stdout: "172.21.15.77:6835/2286085186": "2022-08-02T19:42:32.216900+0000", 2022-08-01T19:42:47.902 INFO:teuthology.orchestra.run.smithi017.stdout: "172.21.15.77:6834/2286085186": "2022-08-02T19:42:32.216900+0000" 2022-08-01T19:42:47.902 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.903 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profiles": { 2022-08-01T19:42:47.903 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_data": { 2022-08-01T19:42:47.904 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-device-class": "", 2022-08-01T19:42:47.904 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:42:47.905 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-root": "default", 2022-08-01T19:42:47.905 INFO:teuthology.orchestra.run.smithi017.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T19:42:47.906 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:42:47.925 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "2", 2022-08-01T19:42:47.925 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:42:47.926 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:42:47.927 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van", 2022-08-01T19:42:47.928 INFO:teuthology.orchestra.run.smithi017.stdout: "w": "8" 2022-08-01T19:42:47.928 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.929 INFO:teuthology.orchestra.run.smithi017.stdout: "default": { 2022-08-01T19:42:47.929 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:42:47.930 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:42:47.930 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "1", 2022-08-01T19:42:47.931 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:42:47.931 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:42:47.931 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van" 2022-08-01T19:42:47.932 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:47.932 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:47.933 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps_queue": [], 2022-08-01T19:42:47.933 INFO:teuthology.orchestra.run.smithi017.stdout: "new_removed_snaps": [], 2022-08-01T19:42:47.933 INFO:teuthology.orchestra.run.smithi017.stdout: "new_purged_snaps": [], 2022-08-01T19:42:47.934 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_node_flags": {}, 2022-08-01T19:42:47.935 INFO:teuthology.orchestra.run.smithi017.stdout: "device_class_flags": {} 2022-08-01T19:42:47.935 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:42:47.937 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:47.097+0000 7f63630ba700 1 -- 172.21.15.17:0/961486215 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f633c0539e0 msgr2=0x7f633c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:47.938 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:47.097+0000 7f63630ba700 1 --2- 172.21.15.17:0/961486215 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f633c0539e0 0x7f633c055ea0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f635800d000 tx=0x7f6358009040).stop 2022-08-01T19:42:47.938 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:47.097+0000 7f63630ba700 1 -- 172.21.15.17:0/961486215 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f635c0b6a90 msgr2=0x7f635c1345a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:47.938 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:47.097+0000 7f63630ba700 1 --2- 172.21.15.17:0/961486215 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f635c0b6a90 0x7f635c1345a0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 rx=0x7f6354001490 tx=0x7f6354007dc0).stop 2022-08-01T19:42:47.938 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:47.097+0000 7f63630ba700 1 -- 172.21.15.17:0/961486215 shutdown_connections 2022-08-01T19:42:47.938 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:47.097+0000 7f63630ba700 1 --2- 172.21.15.17:0/961486215 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f633c0539e0 0x7f633c055ea0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:47.939 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:47.097+0000 7f63630ba700 1 --2- 172.21.15.17:0/961486215 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f635c0b6a90 0x7f635c1345a0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:47.939 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:47.097+0000 7f63630ba700 1 --2- 172.21.15.17:0/961486215 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f635c0aa760 0x7f635c1340b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:47.939 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:47.097+0000 7f63630ba700 1 --2- 172.21.15.17:0/961486215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f635c060aa0 0x7f635c13e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:47.939 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:47.097+0000 7f63630ba700 1 -- 172.21.15.17:0/961486215 >> 172.21.15.17:0/961486215 conn(0x7f635c0b3610 msgr2=0x7f635c0b6020 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:47.940 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:47.097+0000 7f63630ba700 1 -- 172.21.15.17:0/961486215 shutdown_connections 2022-08-01T19:42:47.940 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:47.097+0000 7f63630ba700 1 -- 172.21.15.17:0/961486215 wait complete. 2022-08-01T19:42:47.941 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:42:48.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.057+0000 7fbbad37f700 1 Processor -- start 2022-08-01T19:42:48.063 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.061+0000 7fbbad37f700 1 -- start start 2022-08-01T19:42:48.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.061+0000 7fbbad37f700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbba80b0560 0x7fbba8146df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:48.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.061+0000 7fbbad37f700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbba80b12d0 con 0x7fbba80b0d10 2022-08-01T19:42:48.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.061+0000 7fbbad37f700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbba80a9700 con 0x7fbba80a9320 2022-08-01T19:42:48.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.061+0000 7fbbad37f700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba80b1500 con 0x7fbba80b0560 2022-08-01T19:42:48.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.061+0000 7fbba77fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fbba80b0d10 0x7fbba8140350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36030/0 (socket says 172.21.15.17:36030) 2022-08-01T19:42:48.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.061+0000 7fbba77fe700 1 -- 172.21.15.17:0/566802060 learned_addr learned my addr 172.21.15.17:0/566802060 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:48.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.061+0000 7fbbacb7e700 1 --2- 172.21.15.17:0/566802060 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbba80b0560 0x7fbba8146df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:48.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.061+0000 7fbba6ffd700 1 -- 172.21.15.17:0/566802060 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 736406461 0 0) 0x7fbba80b12d0 con 0x7fbba80b0d10 2022-08-01T19:42:48.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.061+0000 7fbba6ffd700 1 -- 172.21.15.17:0/566802060 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb88003260 con 0x7fbba80b0d10 2022-08-01T19:42:48.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.061+0000 7fbba6ffd700 1 -- 172.21.15.17:0/566802060 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 59425775 0 0) 0x7fbba80a9700 con 0x7fbba80a9320 2022-08-01T19:42:48.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.061+0000 7fbba6ffd700 1 -- 172.21.15.17:0/566802060 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbba80b12d0 con 0x7fbba80a9320 2022-08-01T19:42:48.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.061+0000 7fbba6ffd700 1 -- 172.21.15.17:0/566802060 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4282785136 0 0) 0x7fbb88003260 con 0x7fbba80b0d10 2022-08-01T19:42:48.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.061+0000 7fbba6ffd700 1 -- 172.21.15.17:0/566802060 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbba80b0560 msgr2=0x7fbba8146df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:48.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.061+0000 7fbba6ffd700 1 --2- 172.21.15.17:0/566802060 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbba80b0560 0x7fbba8146df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:48.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.061+0000 7fbba6ffd700 1 -- 172.21.15.17:0/566802060 >> v1:172.21.15.77:6789/0 conn(0x7fbba80a9320 legacy=0x7fbba81466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:48.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.061+0000 7fbba6ffd700 1 -- 172.21.15.17:0/566802060 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba80b17a0 con 0x7fbba80b0d10 2022-08-01T19:42:48.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.061+0000 7fbba6ffd700 1 -- 172.21.15.17:0/566802060 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7fbb90001680 con 0x7fbba80b0d10 2022-08-01T19:42:48.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.061+0000 7fbba6ffd700 1 -- 172.21.15.17:0/566802060 >> v1:172.21.15.17:6789/0 conn(0x7fbba80b0d10 legacy=0x7fbba8140350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:48.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.061+0000 7fbba6ffd700 1 --2- 172.21.15.17:0/566802060 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbba80b0560 0x7fbb88003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:48.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.061+0000 7fbba6ffd700 1 --2- 172.21.15.17:0/566802060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb88003e20 0x7fbb88004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:48.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.061+0000 7fbba6ffd700 1 --2- 172.21.15.17:0/566802060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb880049f0 0x7fbb88004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:48.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.061+0000 7fbba6ffd700 1 -- 172.21.15.17:0/566802060 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba80b1500 con 0x7fbb880049f0 2022-08-01T19:42:48.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.061+0000 7fbba6ffd700 1 -- 172.21.15.17:0/566802060 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8800b800 con 0x7fbb88003e20 2022-08-01T19:42:48.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.061+0000 7fbba6ffd700 1 -- 172.21.15.17:0/566802060 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8800b9c0 con 0x7fbba80b0560 2022-08-01T19:42:48.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.061+0000 7fbba7fff700 1 --2- 172.21.15.17:0/566802060 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbba80b0560 0x7fbb88003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:48.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.061+0000 7fbbacb7e700 1 --2- 172.21.15.17:0/566802060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb880049f0 0x7fbb88004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:48.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.061+0000 7fbba77fe700 1 --2- 172.21.15.17:0/566802060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb88003e20 0x7fbb88004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:48.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.061+0000 7fbba7fff700 1 -- 172.21.15.17:0/566802060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb88003e20 msgr2=0x7fbb88004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:48.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.061+0000 7fbba7fff700 1 --2- 172.21.15.17:0/566802060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb88003e20 0x7fbb88004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:48.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.061+0000 7fbba7fff700 1 -- 172.21.15.17:0/566802060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb880049f0 msgr2=0x7fbb88004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:48.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.061+0000 7fbba7fff700 1 --2- 172.21.15.17:0/566802060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb880049f0 0x7fbb88004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:48.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.061+0000 7fbba7fff700 1 -- 172.21.15.17:0/566802060 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb88003260 con 0x7fbba80b0560 2022-08-01T19:42:48.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.061+0000 7fbbacb7e700 1 --2- 172.21.15.17:0/566802060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb880049f0 0x7fbb88004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:42:48.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.065+0000 7fbba7fff700 1 --2- 172.21.15.17:0/566802060 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbba80b0560 0x7fbb88003910 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fbba0001490 tx=0x7fbba0007d80).ready entity=mon.2 client_cookie=99e9976406005853 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:48.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.065+0000 7fbba6ffd700 1 -- 172.21.15.17:0/566802060 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbba0001780 con 0x7fbba80b0560 2022-08-01T19:42:48.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.065+0000 7fbba6ffd700 1 -- 172.21.15.17:0/566802060 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba0006890 con 0x7fbba80b0560 2022-08-01T19:42:48.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.065+0000 7fbba6ffd700 1 -- 172.21.15.17:0/566802060 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbba0006bf0 con 0x7fbba80b0560 2022-08-01T19:42:48.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.065+0000 7fbbad37f700 1 -- 172.21.15.17:0/566802060 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbba80b0560 msgr2=0x7fbb88003910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:48.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.065+0000 7fbbad37f700 1 --2- 172.21.15.17:0/566802060 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbba80b0560 0x7fbb88003910 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fbba0001490 tx=0x7fbba0007d80).stop 2022-08-01T19:42:48.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.065+0000 7fbbacb7e700 1 -- 172.21.15.17:0/566802060 reap_dead start 2022-08-01T19:42:48.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.065+0000 7fbbad37f700 1 -- 172.21.15.17:0/566802060 shutdown_connections 2022-08-01T19:42:48.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.065+0000 7fbbad37f700 1 -- 172.21.15.17:0/566802060 >> 172.21.15.17:0/566802060 conn(0x7fbba80612d0 msgr2=0x7fbba8062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:48.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.065+0000 7fbbad37f700 1 -- 172.21.15.17:0/566802060 shutdown_connections 2022-08-01T19:42:48.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.065+0000 7fbbad37f700 1 -- 172.21.15.17:0/566802060 wait complete. 2022-08-01T19:42:48.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.069+0000 7fbbad37f700 1 Processor -- start 2022-08-01T19:42:48.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.069+0000 7fbbad37f700 1 -- start start 2022-08-01T19:42:48.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.069+0000 7fbbad37f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbba80b0560 0x7fbba8145720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:48.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.069+0000 7fbbad37f700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbba813ade0 0x7fbba813b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:48.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.069+0000 7fbbad37f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbba8147b40 0x7fbba8147fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:48.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.069+0000 7fbbad37f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba8146460 con 0x7fbba8147b40 2022-08-01T19:42:48.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.069+0000 7fbbad37f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba8139e50 con 0x7fbba80b0560 2022-08-01T19:42:48.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.069+0000 7fbbad37f700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba813a0f0 con 0x7fbba813ade0 2022-08-01T19:42:48.134 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.069+0000 7fbbacb7e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbba8147b40 0x7fbba8147fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:48.134 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.069+0000 7fbba7fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbba80b0560 0x7fbba8145720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:48.135 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.069+0000 7fbbacb7e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbba8147b40 0x7fbba8147fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43850/0 (socket says 172.21.15.17:43850) 2022-08-01T19:42:48.135 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.069+0000 7fbbacb7e700 1 -- 172.21.15.17:0/835547125 learned_addr learned my addr 172.21.15.17:0/835547125 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:48.135 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.069+0000 7fbba77fe700 1 --2- 172.21.15.17:0/835547125 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbba813ade0 0x7fbba813b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:48.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.069+0000 7fbbacb7e700 1 -- 172.21.15.17:0/835547125 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbba813ade0 msgr2=0x7fbba813b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:48.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.069+0000 7fbbacb7e700 1 --2- 172.21.15.17:0/835547125 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbba813ade0 0x7fbba813b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:48.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.069+0000 7fbbacb7e700 1 -- 172.21.15.17:0/835547125 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbba80b0560 msgr2=0x7fbba8145720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:48.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.069+0000 7fbbacb7e700 1 --2- 172.21.15.17:0/835547125 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbba80b0560 0x7fbba8145720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:48.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.069+0000 7fbbacb7e700 1 -- 172.21.15.17:0/835547125 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba8149400 con 0x7fbba8147b40 2022-08-01T19:42:48.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.069+0000 7fbba77fe700 1 --2- 172.21.15.17:0/835547125 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbba813ade0 0x7fbba813b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:42:48.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.069+0000 7fbbacb7e700 1 --2- 172.21.15.17:0/835547125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbba8147b40 0x7fbba8147fc0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 rx=0x7fbb9800b000 tx=0x7fbb98006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:48.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.069+0000 7fbba57fa700 1 -- 172.21.15.17:0/835547125 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb98018070 con 0x7fbba8147b40 2022-08-01T19:42:48.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.069+0000 7fbba57fa700 1 -- 172.21.15.17:0/835547125 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb98000e90 con 0x7fbba8147b40 2022-08-01T19:42:48.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.069+0000 7fbbad37f700 1 -- 172.21.15.17:0/835547125 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbba813b730 con 0x7fbba8147b40 2022-08-01T19:42:48.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.069+0000 7fbba57fa700 1 -- 172.21.15.17:0/835547125 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb9800fb00 con 0x7fbba8147b40 2022-08-01T19:42:48.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.069+0000 7fbbad37f700 1 -- 172.21.15.17:0/835547125 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbba8141030 con 0x7fbba8147b40 2022-08-01T19:42:48.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.073+0000 7fbbad37f700 1 -- 172.21.15.17:0/835547125 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbba8149590 con 0x7fbba8147b40 2022-08-01T19:42:48.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.073+0000 7fbba57fa700 1 -- 172.21.15.17:0/835547125 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbb9800fc60 con 0x7fbba8147b40 2022-08-01T19:42:48.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.073+0000 7fbba57fa700 1 --2- 172.21.15.17:0/835547125 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbb8c053a70 0x7fbb8c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:48.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.073+0000 7fbba57fa700 1 -- 172.21.15.17:0/835547125 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fbb9806c8c0 con 0x7fbba8147b40 2022-08-01T19:42:48.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.073+0000 7fbba7fff700 1 --2- 172.21.15.17:0/835547125 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbb8c053a70 0x7fbb8c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:48.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.077+0000 7fbba7fff700 1 --2- 172.21.15.17:0/835547125 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbb8c053a70 0x7fbb8c055f30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7fbba001b000 tx=0x7fbba0016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:48.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.077+0000 7fbba57fa700 1 -- 172.21.15.17:0/835547125 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbb980434a0 con 0x7fbba8147b40 2022-08-01T19:42:48.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.245+0000 7fbbad37f700 1 -- 172.21.15.17:0/835547125 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbba80574e0 con 0x7fbba8147b40 2022-08-01T19:42:48.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.245+0000 7fbba57fa700 1 -- 172.21.15.17:0/835547125 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+1426 (secure 0 0 0) 0x7fbb98046ac0 con 0x7fbba8147b40 2022-08-01T19:42:48.250 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:42:48.251 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":63,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:42:46.576641+0000","modified":"2022-08-01T19:42:46.576763+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[],"info":{},"data_pools":[22],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2022-08-01T19:42:48.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.249+0000 7fbbad37f700 1 -- 172.21.15.17:0/835547125 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbb8c053a70 msgr2=0x7fbb8c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:48.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.249+0000 7fbbad37f700 1 --2- 172.21.15.17:0/835547125 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbb8c053a70 0x7fbb8c055f30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7fbba001b000 tx=0x7fbba0016000).stop 2022-08-01T19:42:48.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.249+0000 7fbbad37f700 1 -- 172.21.15.17:0/835547125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbba8147b40 msgr2=0x7fbba8147fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:48.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.249+0000 7fbbad37f700 1 --2- 172.21.15.17:0/835547125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbba8147b40 0x7fbba8147fc0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 rx=0x7fbb9800b000 tx=0x7fbb98006df0).stop 2022-08-01T19:42:48.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.249+0000 7fbbad37f700 1 -- 172.21.15.17:0/835547125 shutdown_connections 2022-08-01T19:42:48.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.249+0000 7fbbad37f700 1 --2- 172.21.15.17:0/835547125 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbb8c053a70 0x7fbb8c055f30 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:48.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.249+0000 7fbbad37f700 1 --2- 172.21.15.17:0/835547125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbba8147b40 0x7fbba8147fc0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:48.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.249+0000 7fbbad37f700 1 --2- 172.21.15.17:0/835547125 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbba813ade0 0x7fbba813b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:48.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.249+0000 7fbbad37f700 1 --2- 172.21.15.17:0/835547125 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbba80b0560 0x7fbba8145720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:48.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.249+0000 7fbbad37f700 1 -- 172.21.15.17:0/835547125 >> 172.21.15.17:0/835547125 conn(0x7fbba80612d0 msgr2=0x7fbba8146b70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:48.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.249+0000 7fbbad37f700 1 -- 172.21.15.17:0/835547125 shutdown_connections 2022-08-01T19:42:48.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.249+0000 7fbbad37f700 1 -- 172.21.15.17:0/835547125 wait complete. 2022-08-01T19:42:48.264 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 63 2022-08-01T19:42:48.268 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-08-01T19:42:48.391 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:42:48.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.373+0000 7fdd0933e700 1 Processor -- start 2022-08-01T19:42:48.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.373+0000 7fdd0933e700 1 -- start start 2022-08-01T19:42:48.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.373+0000 7fdd0933e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdd040ab2f0 0x7fdd04067490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:48.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.373+0000 7fdd0933e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fdd040b4ee0 0x7fdd040b76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:48.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.373+0000 7fdd0933e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd04065af0 con 0x7fdd040aaf10 2022-08-01T19:42:48.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.373+0000 7fdd0933e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd040650a0 con 0x7fdd040ab2f0 2022-08-01T19:42:48.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.373+0000 7fdd0933e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd040b3e70 con 0x7fdd040b4ee0 2022-08-01T19:42:48.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.373+0000 7fdcfb7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdd040ab2f0 0x7fdd04067490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:48.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.373+0000 7fdcfb7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdd040ab2f0 0x7fdd04067490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43856/0 (socket says 172.21.15.17:43856) 2022-08-01T19:42:48.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.373+0000 7fdcfb7fe700 1 -- 172.21.15.17:0/3885844807 learned_addr learned my addr 172.21.15.17:0/3885844807 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:48.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.373+0000 7fdd08b3d700 1 --2- 172.21.15.17:0/3885844807 >> v2:172.21.15.77:3300/0 conn(0x7fdd040b4ee0 0x7fdd040b76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:48.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.373+0000 7fdcfb7fe700 1 -- 172.21.15.17:0/3885844807 >> v2:172.21.15.77:3300/0 conn(0x7fdd040b4ee0 msgr2=0x7fdd040b76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:48.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.373+0000 7fdcfb7fe700 1 --2- 172.21.15.17:0/3885844807 >> v2:172.21.15.77:3300/0 conn(0x7fdd040b4ee0 0x7fdd040b76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:48.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.373+0000 7fdcfb7fe700 1 -- 172.21.15.17:0/3885844807 >> v1:172.21.15.77:6789/0 conn(0x7fdd040aaf10 legacy=0x7fdd04064990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:48.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.373+0000 7fdcfb7fe700 1 -- 172.21.15.17:0/3885844807 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd04066ff0 con 0x7fdd040ab2f0 2022-08-01T19:42:48.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.373+0000 7fdcfb7fe700 1 --2- 172.21.15.17:0/3885844807 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd040ab2f0 0x7fdd04067490 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 rx=0x7fdcec002f70 tx=0x7fdcec006e50).ready entity=mon.0 client_cookie=f329556570a96b66 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:48.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.373+0000 7fdd037fe700 1 -- 172.21.15.17:0/3885844807 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdcec00d610 con 0x7fdd040ab2f0 2022-08-01T19:42:48.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.373+0000 7fdd037fe700 1 -- 172.21.15.17:0/3885844807 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcec000e90 con 0x7fdd040ab2f0 2022-08-01T19:42:48.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.373+0000 7fdd037fe700 1 -- 172.21.15.17:0/3885844807 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdcec00d9a0 con 0x7fdd040ab2f0 2022-08-01T19:42:48.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.373+0000 7fdd0933e700 1 -- 172.21.15.17:0/3885844807 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd040ab2f0 msgr2=0x7fdd04067490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:48.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.373+0000 7fdd0933e700 1 --2- 172.21.15.17:0/3885844807 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd040ab2f0 0x7fdd04067490 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 rx=0x7fdcec002f70 tx=0x7fdcec006e50).stop 2022-08-01T19:42:48.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.373+0000 7fdd0933e700 1 -- 172.21.15.17:0/3885844807 shutdown_connections 2022-08-01T19:42:48.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.373+0000 7fdd0933e700 1 --2- 172.21.15.17:0/3885844807 >> v2:172.21.15.77:3300/0 conn(0x7fdd040b4ee0 0x7fdd040b76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:48.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.373+0000 7fdd0933e700 1 --2- 172.21.15.17:0/3885844807 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd040ab2f0 0x7fdd04067490 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:48.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.373+0000 7fdd0933e700 1 -- 172.21.15.17:0/3885844807 >> 172.21.15.17:0/3885844807 conn(0x7fdd04062d70 msgr2=0x7fdd0405f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:48.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.373+0000 7fdd0933e700 1 -- 172.21.15.17:0/3885844807 shutdown_connections 2022-08-01T19:42:48.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.373+0000 7fdd0933e700 1 -- 172.21.15.17:0/3885844807 wait complete. 2022-08-01T19:42:48.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.377+0000 7fdd0933e700 1 Processor -- start 2022-08-01T19:42:48.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.377+0000 7fdd0933e700 1 -- start start 2022-08-01T19:42:48.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.377+0000 7fdd0933e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd040aaf10 0x7fdd040abc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:48.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.377+0000 7fdd0933e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd040ab2f0 0x7fdd04144c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:48.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.377+0000 7fdd0933e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdd040b4ee0 0x7fdd040ac440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:48.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.377+0000 7fdd0933e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd04144350 con 0x7fdd040aaf10 2022-08-01T19:42:48.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.377+0000 7fdd0933e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd041502a0 con 0x7fdd040ab2f0 2022-08-01T19:42:48.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.377+0000 7fdd0933e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd04150540 con 0x7fdd040b4ee0 2022-08-01T19:42:48.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.377+0000 7fdd03fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd040aaf10 0x7fdd040abc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:48.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.377+0000 7fdcfb7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd040ab2f0 0x7fdd04144c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:48.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.377+0000 7fdd03fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd040aaf10 0x7fdd040abc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43860/0 (socket says 172.21.15.17:43860) 2022-08-01T19:42:48.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.377+0000 7fdd08b3d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdd040b4ee0 0x7fdd040ac440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:48.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.377+0000 7fdd03fff700 1 -- 172.21.15.17:0/126812948 learned_addr learned my addr 172.21.15.17:0/126812948 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:48.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.377+0000 7fdcfb7fe700 1 -- 172.21.15.17:0/126812948 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdd040b4ee0 msgr2=0x7fdd040ac440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:48.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.377+0000 7fdcfb7fe700 1 --2- 172.21.15.17:0/126812948 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdd040b4ee0 0x7fdd040ac440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:48.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.377+0000 7fdcfb7fe700 1 -- 172.21.15.17:0/126812948 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd040aaf10 msgr2=0x7fdd040abc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:48.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.377+0000 7fdcfb7fe700 1 --2- 172.21.15.17:0/126812948 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd040aaf10 0x7fdd040abc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:48.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.377+0000 7fdcfb7fe700 1 -- 172.21.15.17:0/126812948 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcec00b1e0 con 0x7fdd040ab2f0 2022-08-01T19:42:48.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.377+0000 7fdd03fff700 1 --2- 172.21.15.17:0/126812948 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd040aaf10 0x7fdd040abc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:42:48.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.377+0000 7fdcfb7fe700 1 --2- 172.21.15.17:0/126812948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd040ab2f0 0x7fdd04144c70 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7fdcec002f70 tx=0x7fdcec015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:48.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.377+0000 7fdd01ffb700 1 -- 172.21.15.17:0/126812948 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdcec00d290 con 0x7fdd040ab2f0 2022-08-01T19:42:48.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.377+0000 7fdd01ffb700 1 -- 172.21.15.17:0/126812948 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcec00d910 con 0x7fdd040ab2f0 2022-08-01T19:42:48.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.377+0000 7fdd0933e700 1 -- 172.21.15.17:0/126812948 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd041507e0 con 0x7fdd040ab2f0 2022-08-01T19:42:48.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.377+0000 7fdd01ffb700 1 -- 172.21.15.17:0/126812948 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdcec011d70 con 0x7fdd040ab2f0 2022-08-01T19:42:48.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.377+0000 7fdd0933e700 1 -- 172.21.15.17:0/126812948 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd040ac160 con 0x7fdd040ab2f0 2022-08-01T19:42:48.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.381+0000 7fdd01ffb700 1 -- 172.21.15.17:0/126812948 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdcec01a040 con 0x7fdd040ab2f0 2022-08-01T19:42:48.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.381+0000 7fdd0933e700 1 -- 172.21.15.17:0/126812948 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd040574e0 con 0x7fdd040ab2f0 2022-08-01T19:42:48.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.381+0000 7fdd01ffb700 1 --2- 172.21.15.17:0/126812948 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdce8053a10 0x7fdce8055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:48.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.381+0000 7fdd01ffb700 1 -- 172.21.15.17:0/126812948 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fdcec06d2a0 con 0x7fdd040ab2f0 2022-08-01T19:42:48.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.381+0000 7fdd03fff700 1 --2- 172.21.15.17:0/126812948 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdce8053a10 0x7fdce8055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:48.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.381+0000 7fdd03fff700 1 --2- 172.21.15.17:0/126812948 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdce8053a10 0x7fdce8055ed0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7fdcf400d000 tx=0x7fdcf4000bc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:48.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.393+0000 7fdd01ffb700 1 -- 172.21.15.17:0/126812948 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdcec043fb0 con 0x7fdd040ab2f0 2022-08-01T19:42:48.416 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:42:48.420 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:42:48.425 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:42:48.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:48.557+0000 7fdd0933e700 1 -- 172.21.15.17:0/126812948 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fdd0405db60 con 0x7fdd040ab2f0 2022-08-01T19:42:49.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.237+0000 7fdd01ffb700 1 -- 172.21.15.17:0/126812948 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v64) v1 ==== 161+0+0 (secure 0 0 0) 0x7fdcec0475d0 con 0x7fdd040ab2f0 2022-08-01T19:42:49.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.241+0000 7fdd0933e700 1 -- 172.21.15.17:0/126812948 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdce8053a10 msgr2=0x7fdce8055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:49.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.241+0000 7fdd0933e700 1 --2- 172.21.15.17:0/126812948 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdce8053a10 0x7fdce8055ed0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7fdcf400d000 tx=0x7fdcf4000bc0).stop 2022-08-01T19:42:49.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.241+0000 7fdd0933e700 1 -- 172.21.15.17:0/126812948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd040ab2f0 msgr2=0x7fdd04144c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:49.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.241+0000 7fdd0933e700 1 --2- 172.21.15.17:0/126812948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd040ab2f0 0x7fdd04144c70 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7fdcec002f70 tx=0x7fdcec015000).stop 2022-08-01T19:42:49.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.241+0000 7fdd0933e700 1 -- 172.21.15.17:0/126812948 shutdown_connections 2022-08-01T19:42:49.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.241+0000 7fdd0933e700 1 --2- 172.21.15.17:0/126812948 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdce8053a10 0x7fdce8055ed0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:49.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.241+0000 7fdd0933e700 1 --2- 172.21.15.17:0/126812948 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdd040b4ee0 0x7fdd040ac440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:49.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.241+0000 7fdd0933e700 1 --2- 172.21.15.17:0/126812948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdd040ab2f0 0x7fdd04144c70 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:49.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.241+0000 7fdd0933e700 1 --2- 172.21.15.17:0/126812948 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd040aaf10 0x7fdd040abc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:49.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.241+0000 7fdd0933e700 1 -- 172.21.15.17:0/126812948 >> 172.21.15.17:0/126812948 conn(0x7fdd04062d70 msgr2=0x7fdd0413db10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:49.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.241+0000 7fdd0933e700 1 -- 172.21.15.17:0/126812948 shutdown_connections 2022-08-01T19:42:49.266 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.241+0000 7fdd0933e700 1 -- 172.21.15.17:0/126812948 wait complete. 2022-08-01T19:42:49.266 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-08-01T19:42:49.268 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:42:49.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.385+0000 7f7d32f15700 1 Processor -- start 2022-08-01T19:42:49.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.385+0000 7f7d32f15700 1 -- start start 2022-08-01T19:42:49.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.389+0000 7f7d32f15700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7d2c0af450 0x7f7d2c0af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:49.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.389+0000 7f7d32f15700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7d2c0ad9e0 0x7f7d2c0a8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:49.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.389+0000 7f7d32f15700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d2c0aff30 con 0x7f7d2c0ad4b0 2022-08-01T19:42:49.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.389+0000 7f7d32f15700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d2c05c7a0 con 0x7f7d2c0ad9e0 2022-08-01T19:42:49.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.389+0000 7f7d32f15700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c140990 con 0x7f7d2c0af450 2022-08-01T19:42:49.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.389+0000 7f7d32714700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7d2c0ad9e0 0x7f7d2c0a8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:49.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.389+0000 7f7d32714700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7d2c0ad9e0 0x7f7d2c0a8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43872/0 (socket says 172.21.15.17:43872) 2022-08-01T19:42:49.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.389+0000 7f7d32714700 1 -- 172.21.15.17:0/3575437076 learned_addr learned my addr 172.21.15.17:0/3575437076 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:49.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.389+0000 7f7d31f13700 1 --2- 172.21.15.17:0/3575437076 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7d2c0af450 0x7f7d2c0af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:49.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.389+0000 7f7d32714700 1 -- 172.21.15.17:0/3575437076 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7d2c0af450 msgr2=0x7f7d2c0af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:49.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.389+0000 7f7d32714700 1 --2- 172.21.15.17:0/3575437076 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7d2c0af450 0x7f7d2c0af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:49.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.389+0000 7f7d32714700 1 -- 172.21.15.17:0/3575437076 >> v1:172.21.15.77:6789/0 conn(0x7f7d2c0ad4b0 legacy=0x7f7d2c1432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:49.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.389+0000 7f7d32714700 1 -- 172.21.15.17:0/3575437076 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d2c1439f0 con 0x7f7d2c0ad9e0 2022-08-01T19:42:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.389+0000 7f7d32714700 1 --2- 172.21.15.17:0/3575437076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d2c0ad9e0 0x7f7d2c0a8e40 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 rx=0x7f7d240032f0 tx=0x7f7d2400b530).ready entity=mon.0 client_cookie=e84842279ec11651 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.389+0000 7f7d30f11700 1 -- 172.21.15.17:0/3575437076 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d24010660 con 0x7f7d2c0ad9e0 2022-08-01T19:42:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.389+0000 7f7d30f11700 1 -- 172.21.15.17:0/3575437076 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d240107c0 con 0x7f7d2c0ad9e0 2022-08-01T19:42:49.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.389+0000 7f7d30f11700 1 -- 172.21.15.17:0/3575437076 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d24010b20 con 0x7f7d2c0ad9e0 2022-08-01T19:42:49.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.389+0000 7f7d32f15700 1 -- 172.21.15.17:0/3575437076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d2c0ad9e0 msgr2=0x7f7d2c0a8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.389+0000 7f7d32f15700 1 --2- 172.21.15.17:0/3575437076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d2c0ad9e0 0x7f7d2c0a8e40 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 rx=0x7f7d240032f0 tx=0x7f7d2400b530).stop 2022-08-01T19:42:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.389+0000 7f7d32f15700 1 -- 172.21.15.17:0/3575437076 shutdown_connections 2022-08-01T19:42:49.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.389+0000 7f7d32f15700 1 --2- 172.21.15.17:0/3575437076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d2c0ad9e0 0x7f7d2c0a8e40 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:49.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.389+0000 7f7d32f15700 1 --2- 172.21.15.17:0/3575437076 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7d2c0af450 0x7f7d2c0af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:49.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.389+0000 7f7d32f15700 1 -- 172.21.15.17:0/3575437076 >> 172.21.15.17:0/3575437076 conn(0x7f7d2c0612d0 msgr2=0x7f7d2c062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:49.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.389+0000 7f7d32f15700 1 -- 172.21.15.17:0/3575437076 shutdown_connections 2022-08-01T19:42:49.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.389+0000 7f7d32f15700 1 -- 172.21.15.17:0/3575437076 wait complete. 2022-08-01T19:42:49.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.393+0000 7f7d32f15700 1 Processor -- start 2022-08-01T19:42:49.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.393+0000 7f7d32f15700 1 -- start start 2022-08-01T19:42:49.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.393+0000 7f7d32f15700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7d2c0ad4b0 0x7f7d2c13f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:49.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.393+0000 7f7d32f15700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d2c0ad9e0 0x7f7d2c151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:49.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.393+0000 7f7d32f15700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d2c0af450 0x7f7d2c152180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:49.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.393+0000 7f7d32f15700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c0aca80 con 0x7f7d2c0af450 2022-08-01T19:42:49.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.393+0000 7f7d32f15700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c0ac540 con 0x7f7d2c0ad9e0 2022-08-01T19:42:49.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.393+0000 7f7d32f15700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c0ac7e0 con 0x7f7d2c0ad4b0 2022-08-01T19:42:49.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.393+0000 7f7d32714700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d2c0af450 0x7f7d2c152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:49.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.393+0000 7f7d32714700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d2c0af450 0x7f7d2c152180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43878/0 (socket says 172.21.15.17:43878) 2022-08-01T19:42:49.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.393+0000 7f7d32714700 1 -- 172.21.15.17:0/2669309896 learned_addr learned my addr 172.21.15.17:0/2669309896 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:49.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.393+0000 7f7d31f13700 1 --2- 172.21.15.17:0/2669309896 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7d2c0ad4b0 0x7f7d2c13f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:49.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.393+0000 7f7d31712700 1 --2- 172.21.15.17:0/2669309896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d2c0ad9e0 0x7f7d2c151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:49.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.393+0000 7f7d32714700 1 -- 172.21.15.17:0/2669309896 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7d2c0ad4b0 msgr2=0x7f7d2c13f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:49.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.393+0000 7f7d32714700 1 --2- 172.21.15.17:0/2669309896 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7d2c0ad4b0 0x7f7d2c13f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:49.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.393+0000 7f7d32714700 1 -- 172.21.15.17:0/2669309896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d2c0ad9e0 msgr2=0x7f7d2c151c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:49.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.393+0000 7f7d32714700 1 --2- 172.21.15.17:0/2669309896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d2c0ad9e0 0x7f7d2c151c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:49.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.393+0000 7f7d32714700 1 -- 172.21.15.17:0/2669309896 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d2400b7d0 con 0x7f7d2c0af450 2022-08-01T19:42:49.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.393+0000 7f7d31f13700 1 --2- 172.21.15.17:0/2669309896 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7d2c0ad4b0 0x7f7d2c13f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:42:49.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.393+0000 7f7d31712700 1 --2- 172.21.15.17:0/2669309896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d2c0ad9e0 0x7f7d2c151c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:42:49.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.393+0000 7f7d32714700 1 --2- 172.21.15.17:0/2669309896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d2c0af450 0x7f7d2c152180 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 rx=0x7f7d240032f0 tx=0x7f7d2c0af030).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:49.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.397+0000 7f7d1affd700 1 -- 172.21.15.17:0/2669309896 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d2400de40 con 0x7f7d2c0af450 2022-08-01T19:42:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.397+0000 7f7d1affd700 1 -- 172.21.15.17:0/2669309896 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d24010930 con 0x7f7d2c0af450 2022-08-01T19:42:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.397+0000 7f7d32f15700 1 -- 172.21.15.17:0/2669309896 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d2c1517f0 con 0x7f7d2c0af450 2022-08-01T19:42:49.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.397+0000 7f7d1affd700 1 -- 172.21.15.17:0/2669309896 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d24010660 con 0x7f7d2c0af450 2022-08-01T19:42:49.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.397+0000 7f7d32f15700 1 -- 172.21.15.17:0/2669309896 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d2c0abfc0 con 0x7f7d2c0af450 2022-08-01T19:42:49.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.397+0000 7f7d32f15700 1 -- 172.21.15.17:0/2669309896 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d2c151980 con 0x7f7d2c0af450 2022-08-01T19:42:49.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.397+0000 7f7d1affd700 1 -- 172.21.15.17:0/2669309896 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7d24021020 con 0x7f7d2c0af450 2022-08-01T19:42:49.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.397+0000 7f7d1affd700 1 --2- 172.21.15.17:0/2669309896 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7d0c053950 0x7f7d0c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:49.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.397+0000 7f7d1affd700 1 -- 172.21.15.17:0/2669309896 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f7d24070330 con 0x7f7d2c0af450 2022-08-01T19:42:49.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.397+0000 7f7d31f13700 1 --2- 172.21.15.17:0/2669309896 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7d0c053950 0x7f7d0c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:49.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.397+0000 7f7d31f13700 1 --2- 172.21.15.17:0/2669309896 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7d0c053950 0x7f7d0c055e10 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f7d2800c000 tx=0x7f7d28008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:49.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.401+0000 7f7d1affd700 1 -- 172.21.15.17:0/2669309896 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7d240470a0 con 0x7f7d2c0af450 2022-08-01T19:42:49.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.561+0000 7f7d32f15700 1 -- 172.21.15.17:0/2669309896 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d2c0574e0 con 0x7f7d2c0af450 2022-08-01T19:42:49.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.561+0000 7f7d1affd700 1 -- 172.21.15.17:0/2669309896 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+1426 (secure 0 0 0) 0x7f7d2404a6c0 con 0x7f7d2c0af450 2022-08-01T19:42:49.566 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:42:49.566 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":64,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":64,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:42:46.576641+0000","modified":"2022-08-01T19:42:49.231948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[],"info":{},"data_pools":[22],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2022-08-01T19:42:49.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.565+0000 7f7d32f15700 1 -- 172.21.15.17:0/2669309896 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7d0c053950 msgr2=0x7f7d0c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:49.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.565+0000 7f7d32f15700 1 --2- 172.21.15.17:0/2669309896 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7d0c053950 0x7f7d0c055e10 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f7d2800c000 tx=0x7f7d28008040).stop 2022-08-01T19:42:49.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.565+0000 7f7d32f15700 1 -- 172.21.15.17:0/2669309896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d2c0af450 msgr2=0x7f7d2c152180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:49.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.565+0000 7f7d32f15700 1 --2- 172.21.15.17:0/2669309896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d2c0af450 0x7f7d2c152180 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 rx=0x7f7d240032f0 tx=0x7f7d2c0af030).stop 2022-08-01T19:42:49.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.565+0000 7f7d32f15700 1 -- 172.21.15.17:0/2669309896 shutdown_connections 2022-08-01T19:42:49.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.565+0000 7f7d32f15700 1 --2- 172.21.15.17:0/2669309896 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7d0c053950 0x7f7d0c055e10 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:49.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.565+0000 7f7d32f15700 1 --2- 172.21.15.17:0/2669309896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d2c0af450 0x7f7d2c152180 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:49.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.565+0000 7f7d32f15700 1 --2- 172.21.15.17:0/2669309896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d2c0ad9e0 0x7f7d2c151c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:49.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.565+0000 7f7d32f15700 1 --2- 172.21.15.17:0/2669309896 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7d2c0ad4b0 0x7f7d2c13f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:49.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.565+0000 7f7d32f15700 1 -- 172.21.15.17:0/2669309896 >> 172.21.15.17:0/2669309896 conn(0x7f7d2c0612d0 msgr2=0x7f7d2c13cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:49.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.565+0000 7f7d32f15700 1 -- 172.21.15.17:0/2669309896 shutdown_connections 2022-08-01T19:42:49.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.565+0000 7f7d32f15700 1 -- 172.21.15.17:0/2669309896 wait complete. 2022-08-01T19:42:49.576 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 64 2022-08-01T19:42:49.600 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-08-01T19:42:49.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.713+0000 7f1f00853700 1 Processor -- start 2022-08-01T19:42:49.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.713+0000 7f1f00853700 1 -- start start 2022-08-01T19:42:49.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.713+0000 7f1f00853700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1efc05ff50 0x7f1efc0b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:49.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.713+0000 7f1f00853700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1efc140220 con 0x7f1efc0b1b10 2022-08-01T19:42:49.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.713+0000 7f1f00853700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1efc1404c0 con 0x7f1efc061860 2022-08-01T19:42:49.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.713+0000 7f1f00853700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1efc140760 con 0x7f1efc05ff50 2022-08-01T19:42:49.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.713+0000 7f1efa59c700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1efc0b1b10 0x7f1efc139e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36076/0 (socket says 172.21.15.17:36076) 2022-08-01T19:42:49.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.713+0000 7f1efa59c700 1 -- 172.21.15.17:0/2305246908 learned_addr learned my addr 172.21.15.17:0/2305246908 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:49.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.713+0000 7f1efad9d700 1 --2- 172.21.15.17:0/2305246908 >> v2:172.21.15.17:3300/0 conn(0x7f1efc05ff50 0x7f1efc0b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:49.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.713+0000 7f1ef959a700 1 -- 172.21.15.17:0/2305246908 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 458298093 0 0) 0x7f1efc140220 con 0x7f1efc0b1b10 2022-08-01T19:42:49.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.713+0000 7f1ef959a700 1 -- 172.21.15.17:0/2305246908 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1edc003260 con 0x7f1efc0b1b10 2022-08-01T19:42:49.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.713+0000 7f1ef959a700 1 -- 172.21.15.17:0/2305246908 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1018515015 0 0) 0x7f1edc003260 con 0x7f1efc0b1b10 2022-08-01T19:42:49.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.713+0000 7f1ef959a700 1 -- 172.21.15.17:0/2305246908 >> v2:172.21.15.17:3300/0 conn(0x7f1efc05ff50 msgr2=0x7f1efc0b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:49.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.713+0000 7f1ef959a700 1 --2- 172.21.15.17:0/2305246908 >> v2:172.21.15.17:3300/0 conn(0x7f1efc05ff50 0x7f1efc0b1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:49.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.713+0000 7f1ef959a700 1 -- 172.21.15.17:0/2305246908 >> v1:172.21.15.77:6789/0 conn(0x7f1efc061860 legacy=0x7f1efc05f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:49.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.713+0000 7f1ef959a700 1 -- 172.21.15.17:0/2305246908 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1efc140a00 con 0x7f1efc0b1b10 2022-08-01T19:42:49.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.717+0000 7f1ef959a700 1 -- 172.21.15.17:0/2305246908 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7f1eec0019e0 con 0x7f1efc0b1b10 2022-08-01T19:42:49.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.717+0000 7f1ef959a700 1 -- 172.21.15.17:0/2305246908 >> v1:172.21.15.17:6789/0 conn(0x7f1efc0b1b10 legacy=0x7f1efc139e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:49.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.717+0000 7f1ef959a700 1 --2- 172.21.15.17:0/2305246908 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1edc003750 0x7f1edc003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:49.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.717+0000 7f1ef959a700 1 --2- 172.21.15.17:0/2305246908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1edc004500 0x7f1edc004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:49.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.717+0000 7f1ef959a700 1 --2- 172.21.15.17:0/2305246908 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1edc004e70 0x7f1edc00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:49.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.717+0000 7f1ef959a700 1 -- 172.21.15.17:0/2305246908 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc140760 con 0x7f1edc004e70 2022-08-01T19:42:49.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.717+0000 7f1ef959a700 1 -- 172.21.15.17:0/2305246908 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1edc00bd20 con 0x7f1edc004500 2022-08-01T19:42:49.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.717+0000 7f1ef959a700 1 -- 172.21.15.17:0/2305246908 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1edc00bee0 con 0x7f1edc003750 2022-08-01T19:42:49.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.717+0000 7f1efad9d700 1 --2- 172.21.15.17:0/2305246908 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1edc004e70 0x7f1edc00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:49.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.717+0000 7f1ef9d9b700 1 --2- 172.21.15.17:0/2305246908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1edc004500 0x7f1edc004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:49.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.717+0000 7f1efa59c700 1 --2- 172.21.15.17:0/2305246908 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1edc003750 0x7f1edc003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:49.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.717+0000 7f1efad9d700 1 -- 172.21.15.17:0/2305246908 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1edc003750 msgr2=0x7f1edc003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:49.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.717+0000 7f1efad9d700 1 --2- 172.21.15.17:0/2305246908 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1edc003750 0x7f1edc003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:49.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.717+0000 7f1efad9d700 1 -- 172.21.15.17:0/2305246908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1edc004500 msgr2=0x7f1edc004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:49.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.717+0000 7f1efad9d700 1 --2- 172.21.15.17:0/2305246908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1edc004500 0x7f1edc004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:49.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.717+0000 7f1efad9d700 1 -- 172.21.15.17:0/2305246908 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1edc003260 con 0x7f1edc004e70 2022-08-01T19:42:49.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.717+0000 7f1efad9d700 1 -- 172.21.15.17:0/2305246908 reap_dead start 2022-08-01T19:42:49.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.717+0000 7f1efad9d700 1 --2- 172.21.15.17:0/2305246908 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1edc004e70 0x7f1edc00b750 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 rx=0x7f1ef400c000 tx=0x7f1ef4006df0).ready entity=mon.0 client_cookie=1407fe27615e041e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:49.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.717+0000 7f1ef959a700 1 -- 172.21.15.17:0/2305246908 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ef400e830 con 0x7f1edc004e70 2022-08-01T19:42:49.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.717+0000 7f1ef959a700 1 -- 172.21.15.17:0/2305246908 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef400e990 con 0x7f1edc004e70 2022-08-01T19:42:49.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.717+0000 7f1ef959a700 1 -- 172.21.15.17:0/2305246908 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ef400ecf0 con 0x7f1edc004e70 2022-08-01T19:42:49.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.717+0000 7f1f00853700 1 -- 172.21.15.17:0/2305246908 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1edc004e70 msgr2=0x7f1edc00b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:49.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.717+0000 7f1f00853700 1 --2- 172.21.15.17:0/2305246908 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1edc004e70 0x7f1edc00b750 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 rx=0x7f1ef400c000 tx=0x7f1ef4006df0).stop 2022-08-01T19:42:49.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.717+0000 7f1f00853700 1 -- 172.21.15.17:0/2305246908 shutdown_connections 2022-08-01T19:42:49.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.717+0000 7f1f00853700 1 --2- 172.21.15.17:0/2305246908 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1edc004e70 0x7f1edc00b750 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:49.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.717+0000 7f1f00853700 1 -- 172.21.15.17:0/2305246908 >> 172.21.15.17:0/2305246908 conn(0x7f1efc0b2b00 msgr2=0x7f1efc0b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:49.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.717+0000 7f1f00853700 1 -- 172.21.15.17:0/2305246908 shutdown_connections 2022-08-01T19:42:49.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.717+0000 7f1f00853700 1 -- 172.21.15.17:0/2305246908 wait complete. 2022-08-01T19:42:49.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.717+0000 7f1f00853700 1 Processor -- start 2022-08-01T19:42:49.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.717+0000 7f1f00853700 1 -- start start 2022-08-01T19:42:49.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.721+0000 7f1f00853700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1edc004e70 0x7f1efc130810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:49.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.721+0000 7f1f00853700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1efc12b130 0x7f1efc12b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:49.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.721+0000 7f1f00853700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1efc142be0 0x7f1efc143060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:49.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.721+0000 7f1f00853700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc13cd80 con 0x7f1efc12b130 2022-08-01T19:42:49.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.721+0000 7f1f00853700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc13d020 con 0x7f1efc142be0 2022-08-01T19:42:49.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.721+0000 7f1f00853700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc13d2c0 con 0x7f1edc004e70 2022-08-01T19:42:49.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.721+0000 7f1ef9d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1efc12b130 0x7f1efc12b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:49.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.721+0000 7f1ef9d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1efc12b130 0x7f1efc12b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43896/0 (socket says 172.21.15.17:43896) 2022-08-01T19:42:49.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.721+0000 7f1ef9d9b700 1 -- 172.21.15.17:0/2367366438 learned_addr learned my addr 172.21.15.17:0/2367366438 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:49.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.721+0000 7f1efa59c700 1 --2- 172.21.15.17:0/2367366438 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1edc004e70 0x7f1efc130810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:49.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.721+0000 7f1efad9d700 1 --2- 172.21.15.17:0/2367366438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1efc142be0 0x7f1efc143060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:49.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.721+0000 7f1ef9d9b700 1 -- 172.21.15.17:0/2367366438 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1edc004e70 msgr2=0x7f1efc130810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:49.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.721+0000 7f1ef9d9b700 1 --2- 172.21.15.17:0/2367366438 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1edc004e70 0x7f1efc130810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:49.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.721+0000 7f1ef9d9b700 1 -- 172.21.15.17:0/2367366438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1efc142be0 msgr2=0x7f1efc143060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:49.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.721+0000 7f1ef9d9b700 1 --2- 172.21.15.17:0/2367366438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1efc142be0 0x7f1efc143060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:49.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.721+0000 7f1ef9d9b700 1 -- 172.21.15.17:0/2367366438 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1efc131060 con 0x7f1efc12b130 2022-08-01T19:42:49.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.721+0000 7f1ef9d9b700 1 --2- 172.21.15.17:0/2367366438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1efc12b130 0x7f1efc12b590 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 rx=0x7f1ee400d000 tx=0x7f1ee4007a00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:49.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.721+0000 7f1ef37fe700 1 -- 172.21.15.17:0/2367366438 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ee4002e90 con 0x7f1efc12b130 2022-08-01T19:42:49.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.721+0000 7f1ef37fe700 1 -- 172.21.15.17:0/2367366438 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee40114c0 con 0x7f1efc12b130 2022-08-01T19:42:49.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.721+0000 7f1f00853700 1 -- 172.21.15.17:0/2367366438 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1efc0b33c0 con 0x7f1efc12b130 2022-08-01T19:42:49.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.721+0000 7f1ef37fe700 1 -- 172.21.15.17:0/2367366438 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ee4011820 con 0x7f1efc12b130 2022-08-01T19:42:49.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.721+0000 7f1f00853700 1 -- 172.21.15.17:0/2367366438 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1efc141780 con 0x7f1efc12b130 2022-08-01T19:42:49.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.721+0000 7f1ef37fe700 1 -- 172.21.15.17:0/2367366438 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1ee401a030 con 0x7f1efc12b130 2022-08-01T19:42:49.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.721+0000 7f1f00853700 1 -- 172.21.15.17:0/2367366438 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1efc0ad110 con 0x7f1efc12b130 2022-08-01T19:42:49.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.721+0000 7f1ef37fe700 1 --2- 172.21.15.17:0/2367366438 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1ee0053b00 0x7f1ee0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:49.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.721+0000 7f1ef37fe700 1 -- 172.21.15.17:0/2367366438 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f1ee406b9d0 con 0x7f1efc12b130 2022-08-01T19:42:49.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.725+0000 7f1efa59c700 1 --2- 172.21.15.17:0/2367366438 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1ee0053b00 0x7f1ee0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:49.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.725+0000 7f1efa59c700 1 --2- 172.21.15.17:0/2367366438 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1ee0053b00 0x7f1ee0055fc0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f1eec005000 tx=0x7f1efc140650).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:49.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.725+0000 7f1ef37fe700 1 -- 172.21.15.17:0/2367366438 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1ee4042750 con 0x7f1efc12b130 2022-08-01T19:42:49.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:49.889+0000 7f1f00853700 1 -- 172.21.15.17:0/2367366438 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1efc0574e0 con 0x7f1efc12b130 2022-08-01T19:42:50.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.601+0000 7f1ef37fe700 1 -- 172.21.15.17:0/2367366438 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v65) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1ee4045d70 con 0x7f1efc12b130 2022-08-01T19:42:50.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.601+0000 7f1f00853700 1 -- 172.21.15.17:0/2367366438 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1ee0053b00 msgr2=0x7f1ee0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:50.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.601+0000 7f1f00853700 1 --2- 172.21.15.17:0/2367366438 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1ee0053b00 0x7f1ee0055fc0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f1eec005000 tx=0x7f1efc140650).stop 2022-08-01T19:42:50.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.601+0000 7f1f00853700 1 -- 172.21.15.17:0/2367366438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1efc12b130 msgr2=0x7f1efc12b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:50.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.601+0000 7f1f00853700 1 --2- 172.21.15.17:0/2367366438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1efc12b130 0x7f1efc12b590 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 rx=0x7f1ee400d000 tx=0x7f1ee4007a00).stop 2022-08-01T19:42:50.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.605+0000 7f1f00853700 1 -- 172.21.15.17:0/2367366438 shutdown_connections 2022-08-01T19:42:50.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.605+0000 7f1f00853700 1 --2- 172.21.15.17:0/2367366438 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1ee0053b00 0x7f1ee0055fc0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:50.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.605+0000 7f1f00853700 1 --2- 172.21.15.17:0/2367366438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1efc142be0 0x7f1efc143060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:50.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.605+0000 7f1f00853700 1 --2- 172.21.15.17:0/2367366438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1efc12b130 0x7f1efc12b590 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:50.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.605+0000 7f1f00853700 1 --2- 172.21.15.17:0/2367366438 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1edc004e70 0x7f1efc130810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:50.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.605+0000 7f1f00853700 1 -- 172.21.15.17:0/2367366438 >> 172.21.15.17:0/2367366438 conn(0x7f1efc0b2b00 msgr2=0x7f1efc13f890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:50.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.605+0000 7f1f00853700 1 -- 172.21.15.17:0/2367366438 shutdown_connections 2022-08-01T19:42:50.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.605+0000 7f1f00853700 1 -- 172.21.15.17:0/2367366438 wait complete. 2022-08-01T19:42:50.638 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-08-01T19:42:50.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.773+0000 7f120759e700 1 Processor -- start 2022-08-01T19:42:50.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.773+0000 7f120759e700 1 -- start start 2022-08-01T19:42:50.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.777+0000 7f120759e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f12080b7500 0x7f12080b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:50.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.777+0000 7f120759e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f12080aadb0 0x7f1208064d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:50.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.777+0000 7f120759e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12080b2510 con 0x7f12080b31f0 2022-08-01T19:42:50.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.777+0000 7f120759e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12081443e0 con 0x7f12080aadb0 2022-08-01T19:42:50.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.777+0000 7f120759e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12080b3700 con 0x7f12080b7500 2022-08-01T19:42:50.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.777+0000 7f1206d9d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f12080aadb0 0x7f1208064d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:50.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.777+0000 7f1206d9d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f12080aadb0 0x7f1208064d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43906/0 (socket says 172.21.15.17:43906) 2022-08-01T19:42:50.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.777+0000 7f1206d9d700 1 -- 172.21.15.17:0/3274447337 learned_addr learned my addr 172.21.15.17:0/3274447337 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:50.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.777+0000 7f120659c700 1 --2- 172.21.15.17:0/3274447337 >> v2:172.21.15.77:3300/0 conn(0x7f12080b7500 0x7f12080b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:50.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.777+0000 7f1206d9d700 1 -- 172.21.15.17:0/3274447337 >> v2:172.21.15.77:3300/0 conn(0x7f12080b7500 msgr2=0x7f12080b78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:50.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.777+0000 7f1206d9d700 1 --2- 172.21.15.17:0/3274447337 >> v2:172.21.15.77:3300/0 conn(0x7f12080b7500 0x7f12080b78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:50.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.777+0000 7f1206d9d700 1 -- 172.21.15.17:0/3274447337 >> v1:172.21.15.77:6789/0 conn(0x7f12080b31f0 legacy=0x7f120813e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:50.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.777+0000 7f1206d9d700 1 -- 172.21.15.17:0/3274447337 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12080b2740 con 0x7f12080aadb0 2022-08-01T19:42:50.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.777+0000 7f1206d9d700 1 --2- 172.21.15.17:0/3274447337 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12080aadb0 0x7f1208064d30 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 rx=0x7f12000032f0 tx=0x7f1200009290).ready entity=mon.0 client_cookie=c068163061744fb9 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:50.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.777+0000 7f120559a700 1 -- 172.21.15.17:0/3274447337 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f120000b870 con 0x7f12080aadb0 2022-08-01T19:42:50.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.777+0000 7f120559a700 1 -- 172.21.15.17:0/3274447337 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f120000b9d0 con 0x7f12080aadb0 2022-08-01T19:42:50.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.777+0000 7f120559a700 1 -- 172.21.15.17:0/3274447337 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f120000bb40 con 0x7f12080aadb0 2022-08-01T19:42:50.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.777+0000 7f120759e700 1 -- 172.21.15.17:0/3274447337 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12080aadb0 msgr2=0x7f1208064d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:50.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.777+0000 7f120759e700 1 --2- 172.21.15.17:0/3274447337 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12080aadb0 0x7f1208064d30 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 rx=0x7f12000032f0 tx=0x7f1200009290).stop 2022-08-01T19:42:50.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.777+0000 7f120759e700 1 -- 172.21.15.17:0/3274447337 shutdown_connections 2022-08-01T19:42:50.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.777+0000 7f120759e700 1 --2- 172.21.15.17:0/3274447337 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12080aadb0 0x7f1208064d30 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:50.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.777+0000 7f120759e700 1 --2- 172.21.15.17:0/3274447337 >> v2:172.21.15.77:3300/0 conn(0x7f12080b7500 0x7f12080b78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:50.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.777+0000 7f120759e700 1 -- 172.21.15.17:0/3274447337 >> 172.21.15.17:0/3274447337 conn(0x7f120805f840 msgr2=0x7f120805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:50.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.777+0000 7f120759e700 1 -- 172.21.15.17:0/3274447337 shutdown_connections 2022-08-01T19:42:50.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.777+0000 7f120759e700 1 -- 172.21.15.17:0/3274447337 wait complete. 2022-08-01T19:42:50.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.781+0000 7f120759e700 1 Processor -- start 2022-08-01T19:42:50.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.781+0000 7f120759e700 1 -- start start 2022-08-01T19:42:50.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.781+0000 7f120759e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12080aadb0 0x7f12080af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:50.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.781+0000 7f120759e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f12080b31f0 0x7f1208146d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:50.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.781+0000 7f120759e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12080b7500 0x7f1208150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:50.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.781+0000 7f120759e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1208154150 con 0x7f12080b7500 2022-08-01T19:42:50.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.781+0000 7f120759e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1208154380 con 0x7f12080aadb0 2022-08-01T19:42:50.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.781+0000 7f120759e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12080aea40 con 0x7f12080b31f0 2022-08-01T19:42:50.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.781+0000 7f120659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12080aadb0 0x7f12080af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:50.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.781+0000 7f1206d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12080b7500 0x7f1208150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:50.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.781+0000 7f120659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12080aadb0 0x7f12080af180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:41830/0 (socket says 172.21.15.17:41830) 2022-08-01T19:42:50.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.781+0000 7f1205d9b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f12080b31f0 0x7f1208146d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:50.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.781+0000 7f120659c700 1 -- 172.21.15.17:0/3759113687 learned_addr learned my addr 172.21.15.17:0/3759113687 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:50.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.781+0000 7f1206d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12080b7500 0x7f1208150280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43912/0 (socket says 172.21.15.17:43912) 2022-08-01T19:42:50.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.785+0000 7f120659c700 1 -- 172.21.15.17:0/3759113687 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f12080b31f0 msgr2=0x7f1208146d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:50.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.785+0000 7f120659c700 1 --2- 172.21.15.17:0/3759113687 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f12080b31f0 0x7f1208146d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:50.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.785+0000 7f120659c700 1 -- 172.21.15.17:0/3759113687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12080b7500 msgr2=0x7f1208150280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:50.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.785+0000 7f120659c700 1 --2- 172.21.15.17:0/3759113687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12080b7500 0x7f1208150280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:50.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.785+0000 7f120659c700 1 -- 172.21.15.17:0/3759113687 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1200009530 con 0x7f12080aadb0 2022-08-01T19:42:50.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.785+0000 7f120659c700 1 --2- 172.21.15.17:0/3759113687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12080aadb0 0x7f12080af180 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f11f000a000 tx=0x7f11f0008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:50.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.785+0000 7f11ff7fe700 1 -- 172.21.15.17:0/3759113687 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11f0011590 con 0x7f12080aadb0 2022-08-01T19:42:50.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.785+0000 7f11ff7fe700 1 -- 172.21.15.17:0/3759113687 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f00116f0 con 0x7f12080aadb0 2022-08-01T19:42:50.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.785+0000 7f11ff7fe700 1 -- 172.21.15.17:0/3759113687 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11f0011a50 con 0x7f12080aadb0 2022-08-01T19:42:50.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.785+0000 7f120759e700 1 -- 172.21.15.17:0/3759113687 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12080aece0 con 0x7f12080aadb0 2022-08-01T19:42:50.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.785+0000 7f120759e700 1 -- 172.21.15.17:0/3759113687 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1208154620 con 0x7f12080aadb0 2022-08-01T19:42:50.802 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.785+0000 7f120759e700 1 -- 172.21.15.17:0/3759113687 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12080aee70 con 0x7f12080aadb0 2022-08-01T19:42:50.802 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.785+0000 7f11ff7fe700 1 -- 172.21.15.17:0/3759113687 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f11f0011bb0 con 0x7f12080aadb0 2022-08-01T19:42:50.802 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.785+0000 7f11ff7fe700 1 --2- 172.21.15.17:0/3759113687 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f11ec053980 0x7f11ec055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:50.802 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.785+0000 7f11ff7fe700 1 -- 172.21.15.17:0/3759113687 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f11f006c450 con 0x7f12080aadb0 2022-08-01T19:42:50.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.789+0000 7f1205d9b700 1 --2- 172.21.15.17:0/3759113687 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f11ec053980 0x7f11ec055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:50.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.789+0000 7f1205d9b700 1 --2- 172.21.15.17:0/3759113687 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f11ec053980 0x7f11ec055e40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7f11f800d000 tx=0x7f11f8007b90).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:50.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.797+0000 7f11ff7fe700 1 -- 172.21.15.17:0/3759113687 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f11f0042fd0 con 0x7f12080aadb0 2022-08-01T19:42:50.932 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:50.925+0000 7f120759e700 1 -- 172.21.15.17:0/3759113687 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f12080574e0 con 0x7f12080aadb0 2022-08-01T19:42:51.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.613+0000 7f11ff7fe700 1 -- 172.21.15.17:0/3759113687 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v131) v1 ==== 185+0+0 (secure 0 0 0) 0x7f11f00465f0 con 0x7f12080aadb0 2022-08-01T19:42:51.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.617+0000 7f120759e700 1 -- 172.21.15.17:0/3759113687 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f11ec053980 msgr2=0x7f11ec055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:51.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.617+0000 7f120759e700 1 --2- 172.21.15.17:0/3759113687 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f11ec053980 0x7f11ec055e40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7f11f800d000 tx=0x7f11f8007b90).stop 2022-08-01T19:42:51.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.617+0000 7f120759e700 1 -- 172.21.15.17:0/3759113687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12080aadb0 msgr2=0x7f12080af180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:51.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.617+0000 7f120759e700 1 --2- 172.21.15.17:0/3759113687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12080aadb0 0x7f12080af180 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f11f000a000 tx=0x7f11f0008040).stop 2022-08-01T19:42:51.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.617+0000 7f120759e700 1 -- 172.21.15.17:0/3759113687 shutdown_connections 2022-08-01T19:42:51.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.617+0000 7f120759e700 1 --2- 172.21.15.17:0/3759113687 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f11ec053980 0x7f11ec055e40 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:51.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.617+0000 7f120759e700 1 --2- 172.21.15.17:0/3759113687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12080b7500 0x7f1208150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:51.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.617+0000 7f120759e700 1 --2- 172.21.15.17:0/3759113687 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f12080b31f0 0x7f1208146d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:51.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.617+0000 7f120759e700 1 --2- 172.21.15.17:0/3759113687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12080aadb0 0x7f12080af180 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:51.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.617+0000 7f120759e700 1 -- 172.21.15.17:0/3759113687 >> 172.21.15.17:0/3759113687 conn(0x7f120805f840 msgr2=0x7f12080b1730 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:51.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.617+0000 7f120759e700 1 -- 172.21.15.17:0/3759113687 shutdown_connections 2022-08-01T19:42:51.640 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.617+0000 7f120759e700 1 -- 172.21.15.17:0/3759113687 wait complete. 2022-08-01T19:42:51.640 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' removed 2022-08-01T19:42:51.642 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-08-01T19:42:51.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.761+0000 7f186abb4700 1 Processor -- start 2022-08-01T19:42:51.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.761+0000 7f186abb4700 1 -- start start 2022-08-01T19:42:51.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.761+0000 7f186abb4700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f18640617e0 0x7f1864066760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:51.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.761+0000 7f186abb4700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1864066c50 0x7f186414db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:51.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.761+0000 7f186abb4700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1864060670 con 0x7f1864060290 2022-08-01T19:42:51.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.765+0000 7f186abb4700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1864060e00 con 0x7f18640617e0 2022-08-01T19:42:51.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.765+0000 7f186abb4700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18640610a0 con 0x7f1864066c50 2022-08-01T19:42:51.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.765+0000 7f185bfff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f18640617e0 0x7f1864066760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:51.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.765+0000 7f185bfff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f18640617e0 0x7f1864066760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43918/0 (socket says 172.21.15.17:43918) 2022-08-01T19:42:51.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.765+0000 7f185bfff700 1 -- 172.21.15.17:0/2187895702 learned_addr learned my addr 172.21.15.17:0/2187895702 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:51.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.765+0000 7f1869151700 1 --2- 172.21.15.17:0/2187895702 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1864066c50 0x7f186414db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:51.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.765+0000 7f185bfff700 1 -- 172.21.15.17:0/2187895702 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1864066c50 msgr2=0x7f186414db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:51.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.765+0000 7f185bfff700 1 --2- 172.21.15.17:0/2187895702 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1864066c50 0x7f186414db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:51.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.765+0000 7f185bfff700 1 -- 172.21.15.17:0/2187895702 >> v1:172.21.15.77:6789/0 conn(0x7f1864060290 legacy=0x7f1864065b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:51.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.765+0000 7f185bfff700 1 -- 172.21.15.17:0/2187895702 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1864061340 con 0x7f18640617e0 2022-08-01T19:42:51.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.765+0000 7f185b7fe700 1 -- 172.21.15.17:0/2187895702 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 705030413 0 0) 0x7f1864060670 con 0x7f1864060290 2022-08-01T19:42:51.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.765+0000 7f185bfff700 1 --2- 172.21.15.17:0/2187895702 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18640617e0 0x7f1864066760 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 rx=0x7f1854002f70 tx=0x7f1854006eb0).ready entity=mon.0 client_cookie=572228432a7b8dfd server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:51.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.765+0000 7f185b7fe700 1 -- 172.21.15.17:0/2187895702 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f185400d570 con 0x7f18640617e0 2022-08-01T19:42:51.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.765+0000 7f185b7fe700 1 -- 172.21.15.17:0/2187895702 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1854000e90 con 0x7f18640617e0 2022-08-01T19:42:51.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.765+0000 7f185b7fe700 1 -- 172.21.15.17:0/2187895702 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f185400d8c0 con 0x7f18640617e0 2022-08-01T19:42:51.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.765+0000 7f186abb4700 1 -- 172.21.15.17:0/2187895702 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18640617e0 msgr2=0x7f1864066760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:51.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.765+0000 7f186abb4700 1 --2- 172.21.15.17:0/2187895702 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18640617e0 0x7f1864066760 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 rx=0x7f1854002f70 tx=0x7f1854006eb0).stop 2022-08-01T19:42:51.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.765+0000 7f186abb4700 1 -- 172.21.15.17:0/2187895702 shutdown_connections 2022-08-01T19:42:51.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.765+0000 7f186abb4700 1 --2- 172.21.15.17:0/2187895702 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1864066c50 0x7f186414db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:51.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.765+0000 7f186abb4700 1 --2- 172.21.15.17:0/2187895702 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18640617e0 0x7f1864066760 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:51.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.765+0000 7f186abb4700 1 -- 172.21.15.17:0/2187895702 >> 172.21.15.17:0/2187895702 conn(0x7f186405d0c0 msgr2=0x7f186405f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:51.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.765+0000 7f186abb4700 1 -- 172.21.15.17:0/2187895702 shutdown_connections 2022-08-01T19:42:51.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.765+0000 7f186abb4700 1 -- 172.21.15.17:0/2187895702 wait complete. 2022-08-01T19:42:51.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.769+0000 7f186abb4700 1 Processor -- start 2022-08-01T19:42:51.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.769+0000 7f186abb4700 1 -- start start 2022-08-01T19:42:51.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.769+0000 7f186abb4700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1864060290 0x7f18640b7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:51.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.769+0000 7f186abb4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f18640617e0 0x7f1864145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:51.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.769+0000 7f186abb4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1864066c50 0x7f18641561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:51.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.769+0000 7f186abb4700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18640b7d00 con 0x7f1864066c50 2022-08-01T19:42:51.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.769+0000 7f186abb4700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18640a5e10 con 0x7f18640617e0 2022-08-01T19:42:51.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.769+0000 7f186abb4700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18640b7f30 con 0x7f1864060290 2022-08-01T19:42:51.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.769+0000 7f1869151700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1864066c50 0x7f18641561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:51.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.769+0000 7f1868950700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1864060290 0x7f18640b7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:51.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.769+0000 7f1868950700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1864060290 0x7f18640b7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:40022/0 (socket says 172.21.15.17:40022) 2022-08-01T19:42:51.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.769+0000 7f1868950700 1 -- 172.21.15.17:0/559127215 learned_addr learned my addr 172.21.15.17:0/559127215 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:51.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.769+0000 7f185bfff700 1 --2- 172.21.15.17:0/559127215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f18640617e0 0x7f1864145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:51.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.773+0000 7f1868950700 1 -- 172.21.15.17:0/559127215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f18640617e0 msgr2=0x7f1864145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:51.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.773+0000 7f1868950700 1 --2- 172.21.15.17:0/559127215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f18640617e0 0x7f1864145bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:51.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.773+0000 7f1868950700 1 -- 172.21.15.17:0/559127215 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1864066c50 msgr2=0x7f18641561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:51.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.773+0000 7f1868950700 1 --2- 172.21.15.17:0/559127215 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1864066c50 0x7f18641561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:51.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.773+0000 7f1868950700 1 -- 172.21.15.17:0/559127215 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185400b1e0 con 0x7f1864060290 2022-08-01T19:42:51.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.773+0000 7f1868950700 1 --2- 172.21.15.17:0/559127215 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1864060290 0x7f18640b7390 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f1860000ed0 tx=0x7f1860007ab0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:51.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.773+0000 7f1859ffb700 1 -- 172.21.15.17:0/559127215 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1860012400 con 0x7f1864060290 2022-08-01T19:42:51.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.773+0000 7f1859ffb700 1 -- 172.21.15.17:0/559127215 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1860012560 con 0x7f1864060290 2022-08-01T19:42:51.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.773+0000 7f186abb4700 1 -- 172.21.15.17:0/559127215 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18640b81d0 con 0x7f1864060290 2022-08-01T19:42:51.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.773+0000 7f1859ffb700 1 -- 172.21.15.17:0/559127215 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18600128c0 con 0x7f1864060290 2022-08-01T19:42:51.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.773+0000 7f186abb4700 1 -- 172.21.15.17:0/559127215 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18641566c0 con 0x7f1864060290 2022-08-01T19:42:51.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.773+0000 7f186abb4700 1 -- 172.21.15.17:0/559127215 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18640b8360 con 0x7f1864060290 2022-08-01T19:42:51.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.777+0000 7f1859ffb700 1 -- 172.21.15.17:0/559127215 <== mon.2 v2:172.21.15.77:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1860019030 con 0x7f1864060290 2022-08-01T19:42:51.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.777+0000 7f1859ffb700 1 --2- 172.21.15.17:0/559127215 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1844053a70 0x7f1844055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:51.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.777+0000 7f1859ffb700 1 -- 172.21.15.17:0/559127215 <== mon.2 v2:172.21.15.77:3301/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f186006b3e0 con 0x7f1864060290 2022-08-01T19:42:51.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.777+0000 7f185bfff700 1 --2- 172.21.15.17:0/559127215 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1844053a70 0x7f1844055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:51.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.777+0000 7f185bfff700 1 --2- 172.21.15.17:0/559127215 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1844053a70 0x7f1844055f30 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7f185401c000 tx=0x7f1854015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:51.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.781+0000 7f1859ffb700 1 -- 172.21.15.17:0/559127215 <== mon.2 v2:172.21.15.77:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f18600423b0 con 0x7f1864060290 2022-08-01T19:42:51.910 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:51.909+0000 7f186abb4700 1 -- 172.21.15.17:0/559127215 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f18640574e0 con 0x7f1864060290 2022-08-01T19:42:52.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.617+0000 7f1859ffb700 1 -- 172.21.15.17:0/559127215 <== mon.2 v2:172.21.15.77:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v132) v1 ==== 173+0+0 (secure 0 0 0) 0x7f18600459d0 con 0x7f1864060290 2022-08-01T19:42:52.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.621+0000 7f186abb4700 1 -- 172.21.15.17:0/559127215 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1844053a70 msgr2=0x7f1844055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:52.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.621+0000 7f186abb4700 1 --2- 172.21.15.17:0/559127215 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1844053a70 0x7f1844055f30 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7f185401c000 tx=0x7f1854015000).stop 2022-08-01T19:42:52.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.621+0000 7f186abb4700 1 -- 172.21.15.17:0/559127215 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1864060290 msgr2=0x7f18640b7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:52.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.621+0000 7f186abb4700 1 --2- 172.21.15.17:0/559127215 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1864060290 0x7f18640b7390 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f1860000ed0 tx=0x7f1860007ab0).stop 2022-08-01T19:42:52.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.621+0000 7f186abb4700 1 -- 172.21.15.17:0/559127215 shutdown_connections 2022-08-01T19:42:52.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.621+0000 7f186abb4700 1 --2- 172.21.15.17:0/559127215 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1844053a70 0x7f1844055f30 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:52.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.621+0000 7f186abb4700 1 --2- 172.21.15.17:0/559127215 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1864066c50 0x7f18641561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:52.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.621+0000 7f186abb4700 1 --2- 172.21.15.17:0/559127215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f18640617e0 0x7f1864145bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:52.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.621+0000 7f186abb4700 1 --2- 172.21.15.17:0/559127215 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1864060290 0x7f18640b7390 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:52.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.621+0000 7f186abb4700 1 -- 172.21.15.17:0/559127215 >> 172.21.15.17:0/559127215 conn(0x7f186405d0c0 msgr2=0x7f1864143e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:52.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.621+0000 7f186abb4700 1 -- 172.21.15.17:0/559127215 shutdown_connections 2022-08-01T19:42:52.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.621+0000 7f186abb4700 1 -- 172.21.15.17:0/559127215 wait complete. 2022-08-01T19:42:52.632 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' removed 2022-08-01T19:42:52.639 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-08-01T19:42:52.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.749+0000 7f519a778700 1 Processor -- start 2022-08-01T19:42:52.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.749+0000 7f519a778700 1 -- start start 2022-08-01T19:42:52.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.749+0000 7f519a778700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f5194144ad0 0x7f51940abea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:52.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.749+0000 7f519a778700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f51940aa050 0x7f51940b0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:52.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.753+0000 7f519a778700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5194144eb0 con 0x7f51940b0ef0 2022-08-01T19:42:52.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.753+0000 7f519a778700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51940ac390 con 0x7f51940aa050 2022-08-01T19:42:52.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.753+0000 7f519a778700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5194145110 con 0x7f5194144ad0 2022-08-01T19:42:52.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.753+0000 7f51937fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f51940aa050 0x7f51940b0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:52.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.753+0000 7f51937fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f51940aa050 0x7f51940b0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43934/0 (socket says 172.21.15.17:43934) 2022-08-01T19:42:52.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.753+0000 7f5193fff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f5194144ad0 0x7f51940abea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:52.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.753+0000 7f51937fe700 1 -- 172.21.15.17:0/3466138765 learned_addr learned my addr 172.21.15.17:0/3466138765 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:52.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.753+0000 7f5198d15700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7f51940b0ef0 0x7f519413e2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:52926/0 (socket says 172.21.15.17:52926) 2022-08-01T19:42:52.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.753+0000 7f51937fe700 1 -- 172.21.15.17:0/3466138765 >> v2:172.21.15.77:3300/0 conn(0x7f5194144ad0 msgr2=0x7f51940abea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:52.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.753+0000 7f51937fe700 1 --2- 172.21.15.17:0/3466138765 >> v2:172.21.15.77:3300/0 conn(0x7f5194144ad0 0x7f51940abea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:52.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.753+0000 7f51937fe700 1 -- 172.21.15.17:0/3466138765 >> v1:172.21.15.77:6789/0 conn(0x7f51940b0ef0 legacy=0x7f519413e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:52.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.753+0000 7f51937fe700 1 -- 172.21.15.17:0/3466138765 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51941453b0 con 0x7f51940aa050 2022-08-01T19:42:52.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.753+0000 7f51937fe700 1 --2- 172.21.15.17:0/3466138765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51940aa050 0x7f51940b0a00 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 rx=0x7f5188002f70 tx=0x7f5188006eb0).ready entity=mon.0 client_cookie=6a2dc0959ac7e0df server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:52.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.753+0000 7f5192ffd700 1 -- 172.21.15.17:0/3466138765 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f518800d5c0 con 0x7f51940aa050 2022-08-01T19:42:52.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.753+0000 7f5192ffd700 1 -- 172.21.15.17:0/3466138765 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5188000e90 con 0x7f51940aa050 2022-08-01T19:42:52.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.753+0000 7f5192ffd700 1 -- 172.21.15.17:0/3466138765 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f518800d950 con 0x7f51940aa050 2022-08-01T19:42:52.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.753+0000 7f519a778700 1 -- 172.21.15.17:0/3466138765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51940aa050 msgr2=0x7f51940b0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:52.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.753+0000 7f519a778700 1 --2- 172.21.15.17:0/3466138765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51940aa050 0x7f51940b0a00 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 rx=0x7f5188002f70 tx=0x7f5188006eb0).stop 2022-08-01T19:42:52.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.753+0000 7f519a778700 1 -- 172.21.15.17:0/3466138765 shutdown_connections 2022-08-01T19:42:52.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.753+0000 7f519a778700 1 --2- 172.21.15.17:0/3466138765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51940aa050 0x7f51940b0a00 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:52.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.753+0000 7f519a778700 1 --2- 172.21.15.17:0/3466138765 >> v2:172.21.15.77:3300/0 conn(0x7f5194144ad0 0x7f51940abea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:52.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.753+0000 7f519a778700 1 -- 172.21.15.17:0/3466138765 >> 172.21.15.17:0/3466138765 conn(0x7f51940b2b00 msgr2=0x7f51940b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:52.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.753+0000 7f519a778700 1 -- 172.21.15.17:0/3466138765 shutdown_connections 2022-08-01T19:42:52.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.753+0000 7f519a778700 1 -- 172.21.15.17:0/3466138765 wait complete. 2022-08-01T19:42:52.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.757+0000 7f519a778700 1 Processor -- start 2022-08-01T19:42:52.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.757+0000 7f519a778700 1 -- start start 2022-08-01T19:42:52.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.757+0000 7f519a778700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51940aa050 0x7f519413c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:52.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.757+0000 7f519a778700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51940b0ef0 0x7f51941528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:52.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.757+0000 7f519a778700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5194144ad0 0x7f5194152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:52.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.757+0000 7f519a778700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51940b4a20 con 0x7f51940b0ef0 2022-08-01T19:42:52.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.757+0000 7f519a778700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51940b4040 con 0x7f51940aa050 2022-08-01T19:42:52.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.757+0000 7f519a778700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51940b42e0 con 0x7f5194144ad0 2022-08-01T19:42:52.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.757+0000 7f51937fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51940b0ef0 0x7f51941528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:52.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.757+0000 7f51937fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51940b0ef0 0x7f51941528d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43936/0 (socket says 172.21.15.17:43936) 2022-08-01T19:42:52.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.757+0000 7f51937fe700 1 -- 172.21.15.17:0/4222897159 learned_addr learned my addr 172.21.15.17:0/4222897159 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:52.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.757+0000 7f5193fff700 1 --2- 172.21.15.17:0/4222897159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51940aa050 0x7f519413c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:52.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.757+0000 7f5198d15700 1 --2- 172.21.15.17:0/4222897159 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5194144ad0 0x7f5194152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:52.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.757+0000 7f51937fe700 1 -- 172.21.15.17:0/4222897159 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5194144ad0 msgr2=0x7f5194152dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:52.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.757+0000 7f51937fe700 1 --2- 172.21.15.17:0/4222897159 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5194144ad0 0x7f5194152dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:52.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.757+0000 7f51937fe700 1 -- 172.21.15.17:0/4222897159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51940aa050 msgr2=0x7f519413c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:52.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.757+0000 7f51937fe700 1 --2- 172.21.15.17:0/4222897159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51940aa050 0x7f519413c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:52.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.757+0000 7f51937fe700 1 -- 172.21.15.17:0/4222897159 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518800b1e0 con 0x7f51940b0ef0 2022-08-01T19:42:52.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.757+0000 7f5198d15700 1 --2- 172.21.15.17:0/4222897159 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5194144ad0 0x7f5194152dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:42:52.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.757+0000 7f51937fe700 1 --2- 172.21.15.17:0/4222897159 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51940b0ef0 0x7f51941528d0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 rx=0x7f5188002f70 tx=0x7f5188015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:52.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.757+0000 7f51917fa700 1 -- 172.21.15.17:0/4222897159 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5188011860 con 0x7f51940b0ef0 2022-08-01T19:42:52.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.757+0000 7f51917fa700 1 -- 172.21.15.17:0/4222897159 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51880119c0 con 0x7f51940b0ef0 2022-08-01T19:42:52.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.757+0000 7f519a778700 1 -- 172.21.15.17:0/4222897159 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51940b4580 con 0x7f51940b0ef0 2022-08-01T19:42:52.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.757+0000 7f51917fa700 1 -- 172.21.15.17:0/4222897159 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5188011d20 con 0x7f51940b0ef0 2022-08-01T19:42:52.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.757+0000 7f519a778700 1 -- 172.21.15.17:0/4222897159 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51941532b0 con 0x7f51940b0ef0 2022-08-01T19:42:52.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.761+0000 7f519a778700 1 -- 172.21.15.17:0/4222897159 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5194145650 con 0x7f51940b0ef0 2022-08-01T19:42:52.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.761+0000 7f51917fa700 1 -- 172.21.15.17:0/4222897159 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5188025030 con 0x7f51940b0ef0 2022-08-01T19:42:52.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.761+0000 7f51917fa700 1 --2- 172.21.15.17:0/4222897159 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5174053b30 0x7f5174055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:52.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.761+0000 7f51917fa700 1 -- 172.21.15.17:0/4222897159 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f518806cf10 con 0x7f51940b0ef0 2022-08-01T19:42:52.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.761+0000 7f5193fff700 1 --2- 172.21.15.17:0/4222897159 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5174053b30 0x7f5174055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:52.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.765+0000 7f5193fff700 1 --2- 172.21.15.17:0/4222897159 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5174053b30 0x7f5174055ff0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f518400b000 tx=0x7f51840091d0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:52.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.769+0000 7f51917fa700 1 -- 172.21.15.17:0/4222897159 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5188043f30 con 0x7f51940b0ef0 2022-08-01T19:42:52.903 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.901+0000 7f519a778700 1 -- 172.21.15.17:0/4222897159 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f51940574e0 con 0x7f51940b0ef0 2022-08-01T19:42:52.914 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.909+0000 7f51917fa700 1 -- 172.21.15.17:0/4222897159 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5188047550 con 0x7f51940b0ef0 2022-08-01T19:42:52.915 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.909+0000 7f519a778700 1 -- 172.21.15.17:0/4222897159 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5174053b30 msgr2=0x7f5174055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:52.915 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.909+0000 7f519a778700 1 --2- 172.21.15.17:0/4222897159 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5174053b30 0x7f5174055ff0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f518400b000 tx=0x7f51840091d0).stop 2022-08-01T19:42:52.915 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.909+0000 7f519a778700 1 -- 172.21.15.17:0/4222897159 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51940b0ef0 msgr2=0x7f51941528d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:52.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.909+0000 7f519a778700 1 --2- 172.21.15.17:0/4222897159 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51940b0ef0 0x7f51941528d0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 rx=0x7f5188002f70 tx=0x7f5188015000).stop 2022-08-01T19:42:52.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.909+0000 7f519a778700 1 -- 172.21.15.17:0/4222897159 shutdown_connections 2022-08-01T19:42:52.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.909+0000 7f519a778700 1 --2- 172.21.15.17:0/4222897159 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5174053b30 0x7f5174055ff0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:52.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.909+0000 7f519a778700 1 --2- 172.21.15.17:0/4222897159 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5194144ad0 0x7f5194152dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:52.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.909+0000 7f519a778700 1 --2- 172.21.15.17:0/4222897159 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51940b0ef0 0x7f51941528d0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:52.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.909+0000 7f519a778700 1 --2- 172.21.15.17:0/4222897159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51940aa050 0x7f519413c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:52.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.909+0000 7f519a778700 1 -- 172.21.15.17:0/4222897159 >> 172.21.15.17:0/4222897159 conn(0x7f51940b2b00 msgr2=0x7f5194143d20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:52.924 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.909+0000 7f519a778700 1 -- 172.21.15.17:0/4222897159 shutdown_connections 2022-08-01T19:42:52.924 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:52.909+0000 7f519a778700 1 -- 172.21.15.17:0/4222897159 wait complete. 2022-08-01T19:42:52.925 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_fs_new' 2022-08-01T19:42:53.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.025+0000 7fadc0868700 1 Processor -- start 2022-08-01T19:42:53.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.025+0000 7fadc0868700 1 -- start start 2022-08-01T19:42:53.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.025+0000 7fadc0868700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fadbc0b08a0 0x7fadbc142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:53.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.025+0000 7fadc0868700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fadbc0b3260 0x7fadbc05c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:53.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.025+0000 7fadc0868700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadbc143ab0 con 0x7fadbc0ad000 2022-08-01T19:42:53.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.025+0000 7fadc0868700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadbc0b0c80 con 0x7fadbc0b08a0 2022-08-01T19:42:53.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.025+0000 7fadc0868700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadbc0ae810 con 0x7fadbc0b3260 2022-08-01T19:42:53.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.025+0000 7fadbb7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fadbc0b08a0 0x7fadbc142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:53.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.025+0000 7fadbb7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fadbc0b08a0 0x7fadbc142eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43946/0 (socket says 172.21.15.17:43946) 2022-08-01T19:42:53.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.025+0000 7fadbb7fe700 1 -- 172.21.15.17:0/3359522927 learned_addr learned my addr 172.21.15.17:0/3359522927 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:53.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.025+0000 7fadbaffd700 1 --2- 172.21.15.17:0/3359522927 >> v2:172.21.15.77:3300/0 conn(0x7fadbc0b3260 0x7fadbc05c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:53.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.029+0000 7fadbb7fe700 1 -- 172.21.15.17:0/3359522927 >> v2:172.21.15.77:3300/0 conn(0x7fadbc0b3260 msgr2=0x7fadbc05c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:53.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.029+0000 7fadbb7fe700 1 --2- 172.21.15.17:0/3359522927 >> v2:172.21.15.77:3300/0 conn(0x7fadbc0b3260 0x7fadbc05c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:53.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.029+0000 7fadba7fc700 1 -- 172.21.15.17:0/3359522927 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3881852257 0 0) 0x7fadbc143ab0 con 0x7fadbc0ad000 2022-08-01T19:42:53.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.029+0000 7fadbb7fe700 1 -- 172.21.15.17:0/3359522927 >> v1:172.21.15.17:6789/0 conn(0x7fadbc0ad000 legacy=0x7fadbc1433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:53.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.029+0000 7fadbb7fe700 1 -- 172.21.15.17:0/3359522927 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadbc0aeab0 con 0x7fadbc0b08a0 2022-08-01T19:42:53.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.029+0000 7fadbb7fe700 1 --2- 172.21.15.17:0/3359522927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc0b08a0 0x7fadbc142eb0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 rx=0x7fadb4002f70 tx=0x7fadb4006eb0).ready entity=mon.0 client_cookie=7d110f98e1eb95c2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:53.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.029+0000 7fadba7fc700 1 -- 172.21.15.17:0/3359522927 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadb400d5c0 con 0x7fadbc0b08a0 2022-08-01T19:42:53.063 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.029+0000 7fadba7fc700 1 -- 172.21.15.17:0/3359522927 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb4000e90 con 0x7fadbc0b08a0 2022-08-01T19:42:53.063 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.029+0000 7fadba7fc700 1 -- 172.21.15.17:0/3359522927 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadb400d910 con 0x7fadbc0b08a0 2022-08-01T19:42:53.064 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.029+0000 7fadc0868700 1 -- 172.21.15.17:0/3359522927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc0b08a0 msgr2=0x7fadbc142eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:53.064 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.029+0000 7fadc0868700 1 --2- 172.21.15.17:0/3359522927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc0b08a0 0x7fadbc142eb0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 rx=0x7fadb4002f70 tx=0x7fadb4006eb0).stop 2022-08-01T19:42:53.065 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.029+0000 7fadc0868700 1 -- 172.21.15.17:0/3359522927 shutdown_connections 2022-08-01T19:42:53.065 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.029+0000 7fadc0868700 1 --2- 172.21.15.17:0/3359522927 >> v2:172.21.15.77:3300/0 conn(0x7fadbc0b3260 0x7fadbc05c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:53.066 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.029+0000 7fadc0868700 1 --2- 172.21.15.17:0/3359522927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc0b08a0 0x7fadbc142eb0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:53.066 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.029+0000 7fadc0868700 1 -- 172.21.15.17:0/3359522927 >> 172.21.15.17:0/3359522927 conn(0x7fadbc0612d0 msgr2=0x7fadbc062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:53.066 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.029+0000 7fadc0868700 1 -- 172.21.15.17:0/3359522927 shutdown_connections 2022-08-01T19:42:53.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.029+0000 7fadc0868700 1 -- 172.21.15.17:0/3359522927 wait complete. 2022-08-01T19:42:53.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.029+0000 7fadc0868700 1 Processor -- start 2022-08-01T19:42:53.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.029+0000 7fadc0868700 1 -- start start 2022-08-01T19:42:53.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.033+0000 7fadc0868700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc0ad000 0x7fadbc0b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:53.069 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.033+0000 7fadc0868700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadbc0b08a0 0x7fadbc153850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:53.069 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.033+0000 7fadc0868700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fadbc0b3260 0x7fadbc066a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:53.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.033+0000 7fadc0868700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc0665f0 con 0x7fadbc0ad000 2022-08-01T19:42:53.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.033+0000 7fadc0868700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc0abd30 con 0x7fadbc0b08a0 2022-08-01T19:42:53.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.033+0000 7fadc0868700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc0ab7f0 con 0x7fadbc0b3260 2022-08-01T19:42:53.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.033+0000 7fadbb7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc0ad000 0x7fadbc0b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:53.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.033+0000 7fadbaffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadbc0b08a0 0x7fadbc153850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:53.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.033+0000 7fadbb7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc0ad000 0x7fadbc0b4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43950/0 (socket says 172.21.15.17:43950) 2022-08-01T19:42:53.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.033+0000 7fadbb7fe700 1 -- 172.21.15.17:0/356966441 learned_addr learned my addr 172.21.15.17:0/356966441 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:53.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.033+0000 7fadbbfff700 1 --2- 172.21.15.17:0/356966441 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fadbc0b3260 0x7fadbc066a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:53.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.033+0000 7fadbaffd700 1 -- 172.21.15.17:0/356966441 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fadbc0b3260 msgr2=0x7fadbc066a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:53.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.033+0000 7fadbaffd700 1 --2- 172.21.15.17:0/356966441 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fadbc0b3260 0x7fadbc066a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:53.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.033+0000 7fadbaffd700 1 -- 172.21.15.17:0/356966441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc0ad000 msgr2=0x7fadbc0b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:53.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.033+0000 7fadbaffd700 1 --2- 172.21.15.17:0/356966441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc0ad000 0x7fadbc0b4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:53.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.033+0000 7fadbaffd700 1 -- 172.21.15.17:0/356966441 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb400b1e0 con 0x7fadbc0b08a0 2022-08-01T19:42:53.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.033+0000 7fadbbfff700 1 --2- 172.21.15.17:0/356966441 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fadbc0b3260 0x7fadbc066a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:42:53.076 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.033+0000 7fadbb7fe700 1 --2- 172.21.15.17:0/356966441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc0ad000 0x7fadbc0b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:42:53.077 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.033+0000 7fadbaffd700 1 --2- 172.21.15.17:0/356966441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadbc0b08a0 0x7fadbc153850 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7fada400c000 tx=0x7fada40091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:53.077 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.033+0000 7fadb8ff9700 1 -- 172.21.15.17:0/356966441 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fada40126d0 con 0x7fadbc0b08a0 2022-08-01T19:42:53.077 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.033+0000 7fadb8ff9700 1 -- 172.21.15.17:0/356966441 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada4012830 con 0x7fadbc0b08a0 2022-08-01T19:42:53.078 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.033+0000 7fadc0868700 1 -- 172.21.15.17:0/356966441 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadbc0aba90 con 0x7fadbc0b08a0 2022-08-01T19:42:53.078 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.033+0000 7fadb8ff9700 1 -- 172.21.15.17:0/356966441 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fada4012b90 con 0x7fadbc0b08a0 2022-08-01T19:42:53.079 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.033+0000 7fadc0868700 1 -- 172.21.15.17:0/356966441 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadbc153d40 con 0x7fadbc0b08a0 2022-08-01T19:42:53.080 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.037+0000 7fadc0868700 1 -- 172.21.15.17:0/356966441 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadbc066790 con 0x7fadbc0b08a0 2022-08-01T19:42:53.080 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.037+0000 7fadb8ff9700 1 -- 172.21.15.17:0/356966441 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fada4022070 con 0x7fadbc0b08a0 2022-08-01T19:42:53.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.037+0000 7fadb8ff9700 1 --2- 172.21.15.17:0/356966441 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fad94053b00 0x7fad94055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:53.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.037+0000 7fadb8ff9700 1 -- 172.21.15.17:0/356966441 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fada406c390 con 0x7fadbc0b08a0 2022-08-01T19:42:53.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.037+0000 7fadbb7fe700 1 --2- 172.21.15.17:0/356966441 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fad94053b00 0x7fad94055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:53.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.037+0000 7fadbb7fe700 1 --2- 172.21.15.17:0/356966441 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fad94053b00 0x7fad94055fc0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7fadb401c000 tx=0x7fadb4015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:53.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.045+0000 7fadb8ff9700 1 -- 172.21.15.17:0/356966441 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fadbc066790 con 0x7fadbc0b08a0 2022-08-01T19:42:53.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.217+0000 7fadc0868700 1 -- 172.21.15.17:0/356966441 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_fs_new"]} v 0) v1 -- 0x7fadbc066790 con 0x7fadbc0b08a0 2022-08-01T19:42:53.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.617+0000 7fadb8ff9700 1 -- 172.21.15.17:0/356966441 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_fs_new"]}]=0 v401) v1 ==== 122+0+0 (secure 0 0 0) 0x7fadbc066790 con 0x7fadbc0b08a0 2022-08-01T19:42:53.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.621+0000 7fadc0868700 1 -- 172.21.15.17:0/356966441 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fad94053b00 msgr2=0x7fad94055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:53.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.621+0000 7fadc0868700 1 --2- 172.21.15.17:0/356966441 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fad94053b00 0x7fad94055fc0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7fadb401c000 tx=0x7fadb4015000).stop 2022-08-01T19:42:53.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.621+0000 7fadc0868700 1 -- 172.21.15.17:0/356966441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadbc0b08a0 msgr2=0x7fadbc153850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:53.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.621+0000 7fadc0868700 1 --2- 172.21.15.17:0/356966441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadbc0b08a0 0x7fadbc153850 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7fada400c000 tx=0x7fada40091d0).stop 2022-08-01T19:42:53.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.621+0000 7fadc0868700 1 -- 172.21.15.17:0/356966441 shutdown_connections 2022-08-01T19:42:53.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.621+0000 7fadc0868700 1 --2- 172.21.15.17:0/356966441 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fad94053b00 0x7fad94055fc0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:53.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.621+0000 7fadc0868700 1 --2- 172.21.15.17:0/356966441 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fadbc0b3260 0x7fadbc066a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:53.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.621+0000 7fadc0868700 1 --2- 172.21.15.17:0/356966441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadbc0b08a0 0x7fadbc153850 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:53.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.621+0000 7fadc0868700 1 --2- 172.21.15.17:0/356966441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc0ad000 0x7fadbc0b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:53.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.621+0000 7fadc0868700 1 -- 172.21.15.17:0/356966441 >> 172.21.15.17:0/356966441 conn(0x7fadbc0612d0 msgr2=0x7fadbc142dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:53.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.621+0000 7fadc0868700 1 -- 172.21.15.17:0/356966441 shutdown_connections 2022-08-01T19:42:53.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.621+0000 7fadc0868700 1 -- 172.21.15.17:0/356966441 wait complete. 2022-08-01T19:42:53.641 INFO:tasks.cephfs_test_runner:test_fs_new (tasks.cephfs.test_misc.TestMisc) ... ok 2022-08-01T19:42:53.642 INFO:tasks.cephfs_test_runner:Starting test: test_getattr_caps (tasks.cephfs.test_misc.TestMisc) 2022-08-01T19:42:53.643 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_getattr_caps' 2022-08-01T19:42:53.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.753+0000 7f12e012c700 1 Processor -- start 2022-08-01T19:42:53.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.753+0000 7f12e012c700 1 -- start start 2022-08-01T19:42:53.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.753+0000 7f12e012c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f12d8148530 0x7f12d80b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:53.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.753+0000 7f12e012c700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d80b04e0 con 0x7f12d80a7250 2022-08-01T19:42:53.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.753+0000 7f12e012c700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d80b0780 con 0x7f12d80b1f40 2022-08-01T19:42:53.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.753+0000 7f12e012c700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12d80a9250 con 0x7f12d8148530 2022-08-01T19:42:53.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.753+0000 7f12dd6c7700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f12d8148530 0x7f12d80b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:53.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.753+0000 7f12dd6c7700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f12d8148530 0x7f12d80b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43960/0 (socket says 172.21.15.17:43960) 2022-08-01T19:42:53.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.753+0000 7f12dd6c7700 1 -- 172.21.15.17:0/763943367 learned_addr learned my addr 172.21.15.17:0/763943367 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:53.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.753+0000 7f12dd6c7700 1 -- 172.21.15.17:0/763943367 >> v1:172.21.15.77:6789/0 conn(0x7f12d80b1f40 legacy=0x7f12d80b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:53.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.753+0000 7f12dd6c7700 1 -- 172.21.15.17:0/763943367 >> v1:172.21.15.17:6789/0 conn(0x7f12d80a7250 legacy=0x7f12d81448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:53.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.753+0000 7f12dd6c7700 1 -- 172.21.15.17:0/763943367 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d80b0a20 con 0x7f12d8148530 2022-08-01T19:42:53.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.753+0000 7f12dd6c7700 1 --2- 172.21.15.17:0/763943367 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12d8148530 0x7f12d80b1160 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 rx=0x7f12d80b3300 tx=0x7f12d0009290).ready entity=mon.0 client_cookie=c0427b7a986ab69c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:53.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.753+0000 7f12dcec6700 1 -- 172.21.15.17:0/763943367 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12d000b820 con 0x7f12d8148530 2022-08-01T19:42:53.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.753+0000 7f12dcec6700 1 -- 172.21.15.17:0/763943367 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d000b980 con 0x7f12d8148530 2022-08-01T19:42:53.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.753+0000 7f12dcec6700 1 -- 172.21.15.17:0/763943367 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12d000baf0 con 0x7f12d8148530 2022-08-01T19:42:53.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.753+0000 7f12e012c700 1 -- 172.21.15.17:0/763943367 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12d8148530 msgr2=0x7f12d80b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:53.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.753+0000 7f12e012c700 1 --2- 172.21.15.17:0/763943367 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12d8148530 0x7f12d80b1160 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 rx=0x7f12d80b3300 tx=0x7f12d0009290).stop 2022-08-01T19:42:53.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.757+0000 7f12e012c700 1 -- 172.21.15.17:0/763943367 shutdown_connections 2022-08-01T19:42:53.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.757+0000 7f12e012c700 1 --2- 172.21.15.17:0/763943367 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12d8148530 0x7f12d80b1160 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:53.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.757+0000 7f12e012c700 1 -- 172.21.15.17:0/763943367 >> 172.21.15.17:0/763943367 conn(0x7f12d805f840 msgr2=0x7f12d805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:53.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.757+0000 7f12e012c700 1 -- 172.21.15.17:0/763943367 shutdown_connections 2022-08-01T19:42:53.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.757+0000 7f12e012c700 1 -- 172.21.15.17:0/763943367 wait complete. 2022-08-01T19:42:53.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.757+0000 7f12e012c700 1 Processor -- start 2022-08-01T19:42:53.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.757+0000 7f12e012c700 1 -- start start 2022-08-01T19:42:53.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.761+0000 7f12e012c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f12d80a7250 0x7f12d8149b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:53.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.761+0000 7f12e012c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12d80b1f40 0x7f12d8157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:53.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.761+0000 7f12e012c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12d8148530 0x7f12d80ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:53.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.761+0000 7f12e012c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d815b0b0 con 0x7f12d8148530 2022-08-01T19:42:53.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.761+0000 7f12e012c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d8156d40 con 0x7f12d80b1f40 2022-08-01T19:42:53.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.761+0000 7f12e012c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d8156fe0 con 0x7f12d80a7250 2022-08-01T19:42:53.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.761+0000 7f12de6c9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12d8148530 0x7f12d80ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:53.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.761+0000 7f12dd6c7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12d80b1f40 0x7f12d8157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:53.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.761+0000 7f12de6c9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12d8148530 0x7f12d80ade90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43968/0 (socket says 172.21.15.17:43968) 2022-08-01T19:42:53.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.761+0000 7f12de6c9700 1 -- 172.21.15.17:0/3016656814 learned_addr learned my addr 172.21.15.17:0/3016656814 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:53.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.761+0000 7f12ddec8700 1 --2- 172.21.15.17:0/3016656814 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f12d80a7250 0x7f12d8149b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:53.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.761+0000 7f12de6c9700 1 -- 172.21.15.17:0/3016656814 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f12d80a7250 msgr2=0x7f12d8149b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:53.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.761+0000 7f12de6c9700 1 --2- 172.21.15.17:0/3016656814 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f12d80a7250 0x7f12d8149b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:53.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.761+0000 7f12de6c9700 1 -- 172.21.15.17:0/3016656814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12d80b1f40 msgr2=0x7f12d8157720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:53.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.761+0000 7f12de6c9700 1 --2- 172.21.15.17:0/3016656814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12d80b1f40 0x7f12d8157720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:53.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.761+0000 7f12de6c9700 1 -- 172.21.15.17:0/3016656814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d0009530 con 0x7f12d8148530 2022-08-01T19:42:53.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.761+0000 7f12de6c9700 1 --2- 172.21.15.17:0/3016656814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12d8148530 0x7f12d80ade90 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 rx=0x7f12d80acff0 tx=0x7f12c8007b80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:53.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.761+0000 7f12ceffd700 1 -- 172.21.15.17:0/3016656814 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12c8012400 con 0x7f12d8148530 2022-08-01T19:42:53.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.761+0000 7f12ceffd700 1 -- 172.21.15.17:0/3016656814 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c8012560 con 0x7f12d8148530 2022-08-01T19:42:53.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.761+0000 7f12e012c700 1 -- 172.21.15.17:0/3016656814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d8157280 con 0x7f12d8148530 2022-08-01T19:42:53.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.761+0000 7f12ceffd700 1 -- 172.21.15.17:0/3016656814 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12c80128c0 con 0x7f12d8148530 2022-08-01T19:42:53.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.761+0000 7f12e012c700 1 -- 172.21.15.17:0/3016656814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d80ae380 con 0x7f12d8148530 2022-08-01T19:42:53.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.765+0000 7f12e012c700 1 -- 172.21.15.17:0/3016656814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d8157410 con 0x7f12d8148530 2022-08-01T19:42:53.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.765+0000 7f12ceffd700 1 -- 172.21.15.17:0/3016656814 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f12c8019030 con 0x7f12d8148530 2022-08-01T19:42:53.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.765+0000 7f12ceffd700 1 --2- 172.21.15.17:0/3016656814 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f12b8053aa0 0x7f12b8055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:53.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.765+0000 7f12ceffd700 1 -- 172.21.15.17:0/3016656814 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f12c806afd0 con 0x7f12d8148530 2022-08-01T19:42:53.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.765+0000 7f12ddec8700 1 --2- 172.21.15.17:0/3016656814 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f12b8053aa0 0x7f12b8055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:53.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.765+0000 7f12ddec8700 1 --2- 172.21.15.17:0/3016656814 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f12b8053aa0 0x7f12b8055f60 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f12d400d000 tx=0x7f12d4007b90).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:53.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.769+0000 7f12ceffd700 1 -- 172.21.15.17:0/3016656814 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f12c8042070 con 0x7f12d8148530 2022-08-01T19:42:53.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:53.941+0000 7f12e012c700 1 -- 172.21.15.17:0/3016656814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]} v 0) v1 -- 0x7f12d80b0cc0 con 0x7f12d8148530 2022-08-01T19:42:54.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.621+0000 7f12ceffd700 1 -- 172.21.15.17:0/3016656814 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]}]=0 v402) v1 ==== 131+0+0 (secure 0 0 0) 0x7f12c8017070 con 0x7f12d8148530 2022-08-01T19:42:54.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.621+0000 7f12e012c700 1 -- 172.21.15.17:0/3016656814 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f12b8053aa0 msgr2=0x7f12b8055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:54.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.621+0000 7f12e012c700 1 --2- 172.21.15.17:0/3016656814 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f12b8053aa0 0x7f12b8055f60 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f12d400d000 tx=0x7f12d4007b90).stop 2022-08-01T19:42:54.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.621+0000 7f12e012c700 1 -- 172.21.15.17:0/3016656814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12d8148530 msgr2=0x7f12d80ade90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:54.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.621+0000 7f12e012c700 1 --2- 172.21.15.17:0/3016656814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12d8148530 0x7f12d80ade90 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 rx=0x7f12d80acff0 tx=0x7f12c8007b80).stop 2022-08-01T19:42:54.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.621+0000 7f12e012c700 1 -- 172.21.15.17:0/3016656814 shutdown_connections 2022-08-01T19:42:54.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.621+0000 7f12e012c700 1 --2- 172.21.15.17:0/3016656814 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f12b8053aa0 0x7f12b8055f60 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:54.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.621+0000 7f12e012c700 1 --2- 172.21.15.17:0/3016656814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12d8148530 0x7f12d80ade90 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:54.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.621+0000 7f12e012c700 1 --2- 172.21.15.17:0/3016656814 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12d80b1f40 0x7f12d8157720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:54.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.621+0000 7f12e012c700 1 --2- 172.21.15.17:0/3016656814 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f12d80a7250 0x7f12d8149b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:54.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.625+0000 7f12e012c700 1 -- 172.21.15.17:0/3016656814 >> 172.21.15.17:0/3016656814 conn(0x7f12d805f840 msgr2=0x7f12d8144ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:54.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.625+0000 7f12e012c700 1 -- 172.21.15.17:0/3016656814 shutdown_connections 2022-08-01T19:42:54.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.625+0000 7f12e012c700 1 -- 172.21.15.17:0/3016656814 wait complete. 2022-08-01T19:42:54.644 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-08-01T19:42:54.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.765+0000 7ff392c22700 1 Processor -- start 2022-08-01T19:42:54.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.765+0000 7ff392c22700 1 -- start start 2022-08-01T19:42:54.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.765+0000 7ff392c22700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ff38c146ae0 0x7ff38c1403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:54.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.765+0000 7ff392c22700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff38c0b4310 0x7ff38c1408b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:54.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.765+0000 7ff392c22700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff38c143cf0 con 0x7ff38c0b4820 2022-08-01T19:42:54.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.765+0000 7ff392c22700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff38c1476a0 con 0x7ff38c0b4310 2022-08-01T19:42:54.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.765+0000 7ff392c22700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff38c143f90 con 0x7ff38c146ae0 2022-08-01T19:42:54.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.765+0000 7ff38bfff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff38c0b4310 0x7ff38c1408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:54.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.765+0000 7ff38bfff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff38c0b4310 0x7ff38c1408b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43976/0 (socket says 172.21.15.17:43976) 2022-08-01T19:42:54.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.765+0000 7ff38bfff700 1 -- 172.21.15.17:0/2112783152 learned_addr learned my addr 172.21.15.17:0/2112783152 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:54.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.765+0000 7ff3909be700 1 --2- 172.21.15.17:0/2112783152 >> v2:172.21.15.77:3300/0 conn(0x7ff38c146ae0 0x7ff38c1403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:54.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.765+0000 7ff38bfff700 1 -- 172.21.15.17:0/2112783152 >> v2:172.21.15.77:3300/0 conn(0x7ff38c146ae0 msgr2=0x7ff38c1403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:54.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.765+0000 7ff38bfff700 1 --2- 172.21.15.17:0/2112783152 >> v2:172.21.15.77:3300/0 conn(0x7ff38c146ae0 0x7ff38c1403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:54.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.765+0000 7ff38bfff700 1 -- 172.21.15.17:0/2112783152 >> v1:172.21.15.77:6789/0 conn(0x7ff38c0b4820 legacy=0x7ff38c0b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:54.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.765+0000 7ff38bfff700 1 -- 172.21.15.17:0/2112783152 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff38c0b0e20 con 0x7ff38c0b4310 2022-08-01T19:42:54.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.765+0000 7ff38bfff700 1 --2- 172.21.15.17:0/2112783152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff38c0b4310 0x7ff38c1408b0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 rx=0x7ff378002f70 tx=0x7ff378006eb0).ready entity=mon.0 client_cookie=46fefda0017f7fac server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:54.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.765+0000 7ff38b7fe700 1 -- 172.21.15.17:0/2112783152 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff37800d610 con 0x7ff38c0b4310 2022-08-01T19:42:54.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.765+0000 7ff38b7fe700 1 -- 172.21.15.17:0/2112783152 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff378000e90 con 0x7ff38c0b4310 2022-08-01T19:42:54.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.765+0000 7ff38b7fe700 1 -- 172.21.15.17:0/2112783152 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff37800d9a0 con 0x7ff38c0b4310 2022-08-01T19:42:54.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.769+0000 7ff392c22700 1 -- 172.21.15.17:0/2112783152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff38c0b4310 msgr2=0x7ff38c1408b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:54.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.769+0000 7ff392c22700 1 --2- 172.21.15.17:0/2112783152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff38c0b4310 0x7ff38c1408b0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 rx=0x7ff378002f70 tx=0x7ff378006eb0).stop 2022-08-01T19:42:54.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.769+0000 7ff392c22700 1 -- 172.21.15.17:0/2112783152 shutdown_connections 2022-08-01T19:42:54.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.769+0000 7ff392c22700 1 --2- 172.21.15.17:0/2112783152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff38c0b4310 0x7ff38c1408b0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:54.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.769+0000 7ff392c22700 1 --2- 172.21.15.17:0/2112783152 >> v2:172.21.15.77:3300/0 conn(0x7ff38c146ae0 0x7ff38c1403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:54.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.769+0000 7ff392c22700 1 -- 172.21.15.17:0/2112783152 >> 172.21.15.17:0/2112783152 conn(0x7ff38c05c620 msgr2=0x7ff38c05ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:54.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.769+0000 7ff392c22700 1 -- 172.21.15.17:0/2112783152 shutdown_connections 2022-08-01T19:42:54.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.769+0000 7ff392c22700 1 -- 172.21.15.17:0/2112783152 wait complete. 2022-08-01T19:42:54.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.769+0000 7ff392c22700 1 Processor -- start 2022-08-01T19:42:54.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.769+0000 7ff392c22700 1 -- start start 2022-08-01T19:42:54.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.769+0000 7ff392c22700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff38c0b4310 0x7ff38c0b6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:54.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.769+0000 7ff392c22700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff38c0b4820 0x7ff38c137100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:54.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.769+0000 7ff392c22700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff38c146ae0 0x7ff38c1375f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:54.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.769+0000 7ff392c22700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c0aea90 con 0x7ff38c0b4820 2022-08-01T19:42:54.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.769+0000 7ff392c22700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c0ae700 con 0x7ff38c0b4310 2022-08-01T19:42:54.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.769+0000 7ff392c22700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c0aecc0 con 0x7ff38c146ae0 2022-08-01T19:42:54.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.769+0000 7ff38bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff38c0b4820 0x7ff38c137100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:54.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.769+0000 7ff3911bf700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff38c146ae0 0x7ff38c1375f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:54.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.769+0000 7ff3909be700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff38c0b4310 0x7ff38c0b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:54.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.769+0000 7ff38bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff38c0b4820 0x7ff38c137100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43982/0 (socket says 172.21.15.17:43982) 2022-08-01T19:42:54.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.769+0000 7ff38bfff700 1 -- 172.21.15.17:0/1167615845 learned_addr learned my addr 172.21.15.17:0/1167615845 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:54.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.769+0000 7ff38bfff700 1 -- 172.21.15.17:0/1167615845 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff38c146ae0 msgr2=0x7ff38c1375f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:54.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.769+0000 7ff38bfff700 1 --2- 172.21.15.17:0/1167615845 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff38c146ae0 0x7ff38c1375f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:54.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.769+0000 7ff38bfff700 1 -- 172.21.15.17:0/1167615845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff38c0b4310 msgr2=0x7ff38c0b6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:54.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.769+0000 7ff38bfff700 1 --2- 172.21.15.17:0/1167615845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff38c0b4310 0x7ff38c0b6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:54.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.769+0000 7ff38bfff700 1 -- 172.21.15.17:0/1167615845 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff37800b1e0 con 0x7ff38c0b4820 2022-08-01T19:42:54.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.773+0000 7ff38bfff700 1 --2- 172.21.15.17:0/1167615845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff38c0b4820 0x7ff38c137100 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 rx=0x7ff378002f70 tx=0x7ff378015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:54.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.773+0000 7ff389ffb700 1 -- 172.21.15.17:0/1167615845 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff378011860 con 0x7ff38c0b4820 2022-08-01T19:42:54.802 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.773+0000 7ff389ffb700 1 -- 172.21.15.17:0/1167615845 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3780119c0 con 0x7ff38c0b4820 2022-08-01T19:42:54.802 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.773+0000 7ff392c22700 1 -- 172.21.15.17:0/1167615845 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff38c0aef60 con 0x7ff38c0b4820 2022-08-01T19:42:54.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.773+0000 7ff389ffb700 1 -- 172.21.15.17:0/1167615845 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff378011d20 con 0x7ff38c0b4820 2022-08-01T19:42:54.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.773+0000 7ff392c22700 1 -- 172.21.15.17:0/1167615845 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff38c137ae0 con 0x7ff38c0b4820 2022-08-01T19:42:54.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.773+0000 7ff392c22700 1 -- 172.21.15.17:0/1167615845 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff38c0af0f0 con 0x7ff38c0b4820 2022-08-01T19:42:54.804 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.773+0000 7ff389ffb700 1 -- 172.21.15.17:0/1167615845 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff37801a040 con 0x7ff38c0b4820 2022-08-01T19:42:54.804 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.773+0000 7ff389ffb700 1 --2- 172.21.15.17:0/1167615845 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff36c053a10 0x7ff36c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:54.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.777+0000 7ff389ffb700 1 -- 172.21.15.17:0/1167615845 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5686+0+0 (secure 0 0 0) 0x7ff37806cc30 con 0x7ff38c0b4820 2022-08-01T19:42:54.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.777+0000 7ff3909be700 1 --2- 172.21.15.17:0/1167615845 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff36c053a10 0x7ff36c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:54.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.777+0000 7ff3909be700 1 --2- 172.21.15.17:0/1167615845 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff36c053a10 0x7ff36c055ed0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7ff38000c000 tx=0x7ff380008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:54.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.781+0000 7ff389ffb700 1 -- 172.21.15.17:0/1167615845 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff378043c70 con 0x7ff38c0b4820 2022-08-01T19:42:54.913 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.909+0000 7ff392c22700 1 -- 172.21.15.17:0/1167615845 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff38c0574e0 con 0x7ff38c0b4820 2022-08-01T19:42:54.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.917+0000 7ff389ffb700 1 -- 172.21.15.17:0/1167615845 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff378047250 con 0x7ff38c0b4820 2022-08-01T19:42:54.924 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.921+0000 7ff392c22700 1 -- 172.21.15.17:0/1167615845 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff36c053a10 msgr2=0x7ff36c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:54.925 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.921+0000 7ff392c22700 1 --2- 172.21.15.17:0/1167615845 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff36c053a10 0x7ff36c055ed0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7ff38000c000 tx=0x7ff380008040).stop 2022-08-01T19:42:54.925 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.921+0000 7ff392c22700 1 -- 172.21.15.17:0/1167615845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff38c0b4820 msgr2=0x7ff38c137100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:54.926 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.921+0000 7ff392c22700 1 --2- 172.21.15.17:0/1167615845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff38c0b4820 0x7ff38c137100 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 rx=0x7ff378002f70 tx=0x7ff378015000).stop 2022-08-01T19:42:54.926 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.921+0000 7ff392c22700 1 -- 172.21.15.17:0/1167615845 shutdown_connections 2022-08-01T19:42:54.926 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.921+0000 7ff392c22700 1 --2- 172.21.15.17:0/1167615845 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff36c053a10 0x7ff36c055ed0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:54.927 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.921+0000 7ff392c22700 1 --2- 172.21.15.17:0/1167615845 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff38c146ae0 0x7ff38c1375f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:54.927 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.921+0000 7ff392c22700 1 --2- 172.21.15.17:0/1167615845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff38c0b4820 0x7ff38c137100 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:54.927 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.921+0000 7ff392c22700 1 --2- 172.21.15.17:0/1167615845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff38c0b4310 0x7ff38c0b6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:54.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.921+0000 7ff392c22700 1 -- 172.21.15.17:0/1167615845 >> 172.21.15.17:0/1167615845 conn(0x7ff38c05c620 msgr2=0x7ff38c13f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:54.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.921+0000 7ff392c22700 1 -- 172.21.15.17:0/1167615845 shutdown_connections 2022-08-01T19:42:54.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:54.921+0000 7ff392c22700 1 -- 172.21.15.17:0/1167615845 wait complete. 2022-08-01T19:42:54.936 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T19:42:54.937 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:42:54.957 DEBUG:teuthology.orchestra.run.smithi077:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:42:54.982 DEBUG:teuthology.orchestra.run.smithi085:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:42:55.005 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:42:55.027 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T19:42:55.027 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:42:55.028 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:42:55.028 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:42:55.053 INFO:teuthology.orchestra.run.smithi085.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-08-01T19:42:55.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:42:55.056 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-08-01T19:42:55.057 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:42:55.058 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:42:55.059 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:42:55.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:42:55.078 INFO:teuthology.orchestra.run.smithi190.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T19:42:55.079 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-08-01T19:42:55.081 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T19:42:55.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.193+0000 7f5ea2be8700 1 Processor -- start 2022-08-01T19:42:55.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.193+0000 7f5ea2be8700 1 -- start start 2022-08-01T19:42:55.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.193+0000 7f5ea2be8700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f5e9c0b28b0 0x7f5e9c13e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:55.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.193+0000 7f5ea2be8700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5e9c141920 0x7f5e9c13e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:55.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.193+0000 7f5ea2be8700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e9c0aac20 con 0x7f5e9c0b1180 2022-08-01T19:42:55.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.193+0000 7f5ea2be8700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e9c141db0 con 0x7f5e9c141920 2022-08-01T19:42:55.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.193+0000 7f5ea2be8700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e9c140e20 con 0x7f5e9c0b28b0 2022-08-01T19:42:55.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.197+0000 7f5ea23e7700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5e9c141920 0x7f5e9c13e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:55.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.197+0000 7f5ea13e5700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5e9c0b1180 0x7f5e9c141210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36184/0 (socket says 172.21.15.17:36184) 2022-08-01T19:42:55.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.197+0000 7f5ea13e5700 1 -- 172.21.15.17:0/164350925 learned_addr learned my addr 172.21.15.17:0/164350925 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:55.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.197+0000 7f5ea1be6700 1 --2- 172.21.15.17:0/164350925 >> v2:172.21.15.77:3300/0 conn(0x7f5e9c0b28b0 0x7f5e9c13e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:55.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.197+0000 7f5ea0be4700 1 -- 172.21.15.17:0/164350925 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 959536378 0 0) 0x7f5e8c001110 con 0x7f5e9c0b1180 2022-08-01T19:42:55.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.197+0000 7f5ea0be4700 1 -- 172.21.15.17:0/164350925 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e84003260 con 0x7f5e9c0b1180 2022-08-01T19:42:55.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.197+0000 7f5ea1be6700 1 -- 172.21.15.17:0/164350925 >> v2:172.21.15.17:3300/0 conn(0x7f5e9c141920 msgr2=0x7f5e9c13e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:55.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.197+0000 7f5ea1be6700 1 --2- 172.21.15.17:0/164350925 >> v2:172.21.15.17:3300/0 conn(0x7f5e9c141920 0x7f5e9c13e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:55.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.197+0000 7f5ea0be4700 1 -- 172.21.15.17:0/164350925 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3636933808 0 0) 0x7f5e84003260 con 0x7f5e9c0b1180 2022-08-01T19:42:55.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.197+0000 7f5ea1be6700 1 -- 172.21.15.17:0/164350925 >> v1:172.21.15.17:6789/0 conn(0x7f5e9c0b1180 legacy=0x7f5e9c141210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:55.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.197+0000 7f5ea1be6700 1 -- 172.21.15.17:0/164350925 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e9c0b2c90 con 0x7f5e9c0b28b0 2022-08-01T19:42:55.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.197+0000 7f5ea23e7700 1 --2- 172.21.15.17:0/164350925 >> v2:172.21.15.17:3300/0 conn(0x7f5e9c141920 0x7f5e9c13e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:42:55.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.197+0000 7f5ea1be6700 1 --2- 172.21.15.17:0/164350925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e9c0b28b0 0x7f5e9c13e1c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f5e8c001990 tx=0x7f5e98006df0).ready entity=mon.1 client_cookie=95832b1ad6fbabb6 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:55.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.197+0000 7f5ea0be4700 1 -- 172.21.15.17:0/164350925 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e8c0017f0 con 0x7f5e9c0b28b0 2022-08-01T19:42:55.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.197+0000 7f5ea0be4700 1 -- 172.21.15.17:0/164350925 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e98000e90 con 0x7f5e9c0b28b0 2022-08-01T19:42:55.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.197+0000 7f5ea0be4700 1 -- 172.21.15.17:0/164350925 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e98012460 con 0x7f5e9c0b28b0 2022-08-01T19:42:55.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.197+0000 7f5ea2be8700 1 -- 172.21.15.17:0/164350925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e9c0b28b0 msgr2=0x7f5e9c13e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:55.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.197+0000 7f5ea2be8700 1 --2- 172.21.15.17:0/164350925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e9c0b28b0 0x7f5e9c13e1c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f5e8c001990 tx=0x7f5e98006df0).stop 2022-08-01T19:42:55.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.197+0000 7f5ea2be8700 1 -- 172.21.15.17:0/164350925 shutdown_connections 2022-08-01T19:42:55.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.197+0000 7f5ea2be8700 1 --2- 172.21.15.17:0/164350925 >> v2:172.21.15.17:3300/0 conn(0x7f5e9c141920 0x7f5e9c13e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:55.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.197+0000 7f5ea2be8700 1 --2- 172.21.15.17:0/164350925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e9c0b28b0 0x7f5e9c13e1c0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:55.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.197+0000 7f5ea2be8700 1 -- 172.21.15.17:0/164350925 >> 172.21.15.17:0/164350925 conn(0x7f5e9c060310 msgr2=0x7f5e9c0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:55.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.197+0000 7f5ea2be8700 1 -- 172.21.15.17:0/164350925 shutdown_connections 2022-08-01T19:42:55.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.197+0000 7f5ea2be8700 1 -- 172.21.15.17:0/164350925 wait complete. 2022-08-01T19:42:55.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.201+0000 7f5ea2be8700 1 Processor -- start 2022-08-01T19:42:55.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.201+0000 7f5ea2be8700 1 -- start start 2022-08-01T19:42:55.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.201+0000 7f5ea2be8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e9c0b1180 0x7f5e9c1423a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:55.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.201+0000 7f5ea2be8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e9c0b28b0 0x7f5e9c142890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:55.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.201+0000 7f5ea2be8700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5e9c141920 0x7f5e9c0b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:55.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.201+0000 7f5ea2be8700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9c0b3b30 con 0x7f5e9c0b1180 2022-08-01T19:42:55.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.201+0000 7f5ea2be8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9c144b90 con 0x7f5e9c0b28b0 2022-08-01T19:42:55.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.201+0000 7f5ea2be8700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9c0b3890 con 0x7f5e9c141920 2022-08-01T19:42:55.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.201+0000 7f5ea1be6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e9c0b1180 0x7f5e9c1423a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:55.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.201+0000 7f5ea1be6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e9c0b1180 0x7f5e9c1423a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43992/0 (socket says 172.21.15.17:43992) 2022-08-01T19:42:55.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.201+0000 7f5ea1be6700 1 -- 172.21.15.17:0/3945418908 learned_addr learned my addr 172.21.15.17:0/3945418908 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:55.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.201+0000 7f5ea13e5700 1 --2- 172.21.15.17:0/3945418908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e9c0b28b0 0x7f5e9c142890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:55.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.201+0000 7f5ea23e7700 1 --2- 172.21.15.17:0/3945418908 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5e9c141920 0x7f5e9c0b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:55.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.201+0000 7f5ea1be6700 1 -- 172.21.15.17:0/3945418908 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5e9c141920 msgr2=0x7f5e9c0b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.201+0000 7f5ea1be6700 1 --2- 172.21.15.17:0/3945418908 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5e9c141920 0x7f5e9c0b4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.201+0000 7f5ea1be6700 1 -- 172.21.15.17:0/3945418908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e9c0b28b0 msgr2=0x7f5e9c142890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:55.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.201+0000 7f5ea1be6700 1 --2- 172.21.15.17:0/3945418908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e9c0b28b0 0x7f5e9c142890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:55.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.201+0000 7f5ea1be6700 1 -- 172.21.15.17:0/3945418908 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e94009470 con 0x7f5e9c0b1180 2022-08-01T19:42:55.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.201+0000 7f5ea13e5700 1 --2- 172.21.15.17:0/3945418908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e9c0b28b0 0x7f5e9c142890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:42:55.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.201+0000 7f5ea1be6700 1 --2- 172.21.15.17:0/3945418908 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e9c0b1180 0x7f5e9c1423a0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 rx=0x7f5e8c001990 tx=0x7f5e98014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:55.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.201+0000 7f5e8affd700 1 -- 172.21.15.17:0/3945418908 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e9800da20 con 0x7f5e9c0b1180 2022-08-01T19:42:55.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.201+0000 7f5e8affd700 1 -- 172.21.15.17:0/3945418908 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9800db80 con 0x7f5e9c0b1180 2022-08-01T19:42:55.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.201+0000 7f5ea2be8700 1 -- 172.21.15.17:0/3945418908 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e9800b180 con 0x7f5e9c0b1180 2022-08-01T19:42:55.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.201+0000 7f5e8affd700 1 -- 172.21.15.17:0/3945418908 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e9801e400 con 0x7f5e9c0b1180 2022-08-01T19:42:55.234 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.201+0000 7f5ea2be8700 1 -- 172.21.15.17:0/3945418908 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e9c0ae360 con 0x7f5e9c0b1180 2022-08-01T19:42:55.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.201+0000 7f5ea2be8700 1 -- 172.21.15.17:0/3945418908 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e9c0574e0 con 0x7f5e9c0b1180 2022-08-01T19:42:55.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.201+0000 7f5e8affd700 1 -- 172.21.15.17:0/3945418908 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5e98025030 con 0x7f5e9c0b1180 2022-08-01T19:42:55.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.205+0000 7f5e8affd700 1 --2- 172.21.15.17:0/3945418908 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5e7c053980 0x7f5e7c055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:55.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.205+0000 7f5e8affd700 1 -- 172.21.15.17:0/3945418908 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f5e9806cb20 con 0x7f5e9c0b1180 2022-08-01T19:42:55.236 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.205+0000 7f5ea13e5700 1 --2- 172.21.15.17:0/3945418908 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5e7c053980 0x7f5e7c055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:55.236 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.205+0000 7f5ea13e5700 1 --2- 172.21.15.17:0/3945418908 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5e7c053980 0x7f5e7c055e40 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f5e8c00b000 tx=0x7f5e8c0011f0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:55.236 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.205+0000 7f5e8affd700 1 -- 172.21.15.17:0/3945418908 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5e98043ab0 con 0x7f5e9c0b1180 2022-08-01T19:42:55.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.337+0000 7f5ea2be8700 1 -- 172.21.15.17:0/3945418908 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f5e9c069a70 con 0x7f5e9c0b1180 2022-08-01T19:42:55.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.337+0000 7f5e8affd700 1 -- 172.21.15.17:0/3945418908 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v132) v1 ==== 81+0+29229 (secure 0 0 0) 0x7f5e98023070 con 0x7f5e9c0b1180 2022-08-01T19:42:55.343 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:42:55.344 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:42:55.344 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 132, 2022-08-01T19:42:55.345 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", 2022-08-01T19:42:55.345 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2022-08-01T19:33:50.105743+0000", 2022-08-01T19:42:55.346 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2022-08-01T19:42:52.608674+0000", 2022-08-01T19:42:55.347 INFO:teuthology.orchestra.run.smithi017.stdout: "last_up_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:42:55.347 INFO:teuthology.orchestra.run.smithi017.stdout: "last_in_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:42:55.348 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T19:42:55.348 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_num": 5799936, 2022-08-01T19:42:55.348 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_set": [ 2022-08-01T19:42:55.349 INFO:teuthology.orchestra.run.smithi017.stdout: "pglog_hardlimit", 2022-08-01T19:42:55.349 INFO:teuthology.orchestra.run.smithi017.stdout: "purged_snapdirs", 2022-08-01T19:42:55.349 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_deletes", 2022-08-01T19:42:55.349 INFO:teuthology.orchestra.run.smithi017.stdout: "sortbitwise" 2022-08-01T19:42:55.350 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:42:55.350 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_version": 6, 2022-08-01T19:42:55.350 INFO:teuthology.orchestra.run.smithi017.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T19:42:55.350 INFO:teuthology.orchestra.run.smithi017.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T19:42:55.351 INFO:teuthology.orchestra.run.smithi017.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T19:42:55.351 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_snapshot": "", 2022-08-01T19:42:55.351 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_max": 24, 2022-08-01T19:42:55.351 INFO:teuthology.orchestra.run.smithi017.stdout: "max_osd": 8, 2022-08-01T19:42:55.351 INFO:teuthology.orchestra.run.smithi017.stdout: "require_min_compat_client": "jewel", 2022-08-01T19:42:55.352 INFO:teuthology.orchestra.run.smithi017.stdout: "min_compat_client": "jewel", 2022-08-01T19:42:55.352 INFO:teuthology.orchestra.run.smithi017.stdout: "require_osd_release": "octopus", 2022-08-01T19:42:55.352 INFO:teuthology.orchestra.run.smithi017.stdout: "pools": [ 2022-08-01T19:42:55.352 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.353 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 1, 2022-08-01T19:42:55.353 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "device_health_metrics", 2022-08-01T19:42:55.353 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:33:53.001841+0000", 2022-08-01T19:42:55.353 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:42:55.354 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:42:55.354 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:42:55.354 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:42:55.355 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:42:55.355 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:42:55.355 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:42:55.355 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:42:55.356 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 1, 2022-08-01T19:42:55.356 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 1, 2022-08-01T19:42:55.356 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 1, 2022-08-01T19:42:55.356 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 1, 2022-08-01T19:42:55.357 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 1, 2022-08-01T19:42:55.357 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:42:55.357 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:42:55.357 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:42:55.357 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:42:55.358 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:42:55.358 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:42:55.358 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:42:55.359 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.359 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "16", 2022-08-01T19:42:55.359 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:42:55.359 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:42:55.359 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:42:55.360 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:42:55.360 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:42:55.360 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:42:55.360 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:42:55.361 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:42:55.361 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:42:55.361 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:42:55.361 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:42:55.362 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:42:55.362 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:42:55.362 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:42:55.362 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:42:55.363 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:42:55.363 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:42:55.363 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:42:55.363 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:42:55.363 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:42:55.364 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:42:55.364 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:42:55.364 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:42:55.364 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:42:55.365 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:42:55.365 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:42:55.365 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.366 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:42:55.366 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:42:55.366 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:42:55.367 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:42:55.367 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:42:55.367 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:42:55.367 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:42:55.368 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:42:55.368 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:42:55.368 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:42:55.368 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:42:55.369 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:42:55.369 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 1 2022-08-01T19:42:55.369 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.369 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:42:55.370 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_devicehealth": {} 2022-08-01T19:42:55.370 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.370 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.371 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.371 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 2, 2022-08-01T19:42:55.371 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "rbd", 2022-08-01T19:42:55.371 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:34:01.553098+0000", 2022-08-01T19:42:55.371 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:42:55.372 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:42:55.372 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:42:55.372 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:42:55.372 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:42:55.373 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:42:55.373 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:42:55.373 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:42:55.373 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:42:55.374 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:42:55.374 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:42:55.374 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:42:55.375 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:42:55.375 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:42:55.375 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:42:55.375 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:42:55.376 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:42:55.376 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:42:55.376 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:42:55.376 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:42:55.377 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.377 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "19", 2022-08-01T19:42:55.377 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:42:55.377 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:42:55.377 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:42:55.378 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:42:55.378 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:42:55.378 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:42:55.378 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:42:55.379 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:42:55.379 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:42:55.379 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:42:55.379 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:42:55.380 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:42:55.380 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:42:55.380 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:42:55.380 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:42:55.381 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:42:55.381 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:42:55.381 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:42:55.381 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:42:55.381 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:42:55.382 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:42:55.382 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:42:55.382 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:42:55.382 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:42:55.383 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:42:55.383 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:42:55.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.383 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:42:55.384 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:42:55.384 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:42:55.384 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:42:55.384 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:42:55.385 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:42:55.385 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:42:55.385 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:42:55.385 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:42:55.385 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:42:55.386 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:42:55.386 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:42:55.386 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:42:55.387 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd": {} 2022-08-01T19:42:55.387 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.387 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.387 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:42:55.388 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2022-08-01T19:42:55.388 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.388 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:42:55.388 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a962f14f-cf16-4fa8-84ff-c85325db9132", 2022-08-01T19:42:55.389 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:42:55.389 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:42:55.389 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:42:55.389 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:42:55.390 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:42:55.390 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:42:55.390 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:42:55.390 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 127, 2022-08-01T19:42:55.391 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:42:55.391 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:42:55.391 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:42:55.391 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:55.392 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.392 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:55.392 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6800", 2022-08-01T19:42:55.392 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:42:55.392 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.393 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.393 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:55.393 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6801", 2022-08-01T19:42:55.393 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:42:55.394 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.394 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.394 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.394 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:42:55.395 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:55.395 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.395 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:55.395 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6802", 2022-08-01T19:42:55.396 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:42:55.396 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.396 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.396 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:55.397 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6803", 2022-08-01T19:42:55.397 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:42:55.397 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.397 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.397 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.398 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:42:55.398 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:55.399 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.399 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:55.399 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6806", 2022-08-01T19:42:55.399 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:42:55.399 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.400 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.400 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:55.400 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6807", 2022-08-01T19:42:55.400 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:42:55.401 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.401 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.401 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.402 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:42:55.402 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:55.402 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.402 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:55.403 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6804", 2022-08-01T19:42:55.403 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:42:55.403 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.403 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.404 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:55.404 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6805", 2022-08-01T19:42:55.404 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:42:55.404 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.405 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.405 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.405 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6801/23008", 2022-08-01T19:42:55.406 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6803/23008", 2022-08-01T19:42:55.407 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6807/23008", 2022-08-01T19:42:55.408 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6805/23008", 2022-08-01T19:42:55.408 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:42:55.408 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:42:55.409 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:42:55.409 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.410 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.410 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.411 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:42:55.411 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "805f7051-a527-4e98-8844-b6e406a63248", 2022-08-01T19:42:55.412 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:42:55.412 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:42:55.413 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:42:55.413 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:42:55.414 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:42:55.414 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:42:55.414 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:42:55.415 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 127, 2022-08-01T19:42:55.415 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:42:55.415 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:42:55.416 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:42:55.416 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:55.416 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.417 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:55.417 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6808", 2022-08-01T19:42:55.418 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:42:55.418 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.419 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.419 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:55.420 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6809", 2022-08-01T19:42:55.420 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:42:55.421 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.421 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.422 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.422 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:42:55.422 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:55.423 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.423 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:55.424 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6810", 2022-08-01T19:42:55.424 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:42:55.424 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.426 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.426 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:55.427 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6811", 2022-08-01T19:42:55.427 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:42:55.428 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.428 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.428 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.429 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:42:55.429 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:55.429 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.429 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:55.430 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6814", 2022-08-01T19:42:55.430 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:42:55.430 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.431 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.431 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:55.431 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6815", 2022-08-01T19:42:55.431 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:42:55.431 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.432 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.432 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.432 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:42:55.433 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:55.433 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.433 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:55.433 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6812", 2022-08-01T19:42:55.434 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:42:55.434 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.434 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.434 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:55.435 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6813", 2022-08-01T19:42:55.435 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:42:55.435 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.436 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.436 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.436 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6809/23007", 2022-08-01T19:42:55.437 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6811/23007", 2022-08-01T19:42:55.437 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6815/23007", 2022-08-01T19:42:55.437 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6813/23007", 2022-08-01T19:42:55.438 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:42:55.438 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:42:55.438 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:42:55.439 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.439 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.439 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.439 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:42:55.440 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "89f7f3df-7c10-48e1-8b69-9e4243923a61", 2022-08-01T19:42:55.440 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:42:55.440 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:42:55.440 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:42:55.441 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:42:55.441 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:42:55.442 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:42:55.442 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:42:55.443 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 122, 2022-08-01T19:42:55.443 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:42:55.443 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:42:55.444 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:42:55.444 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:55.445 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.445 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:55.446 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6817", 2022-08-01T19:42:55.447 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:42:55.447 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.448 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.448 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:55.449 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6819", 2022-08-01T19:42:55.449 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:42:55.450 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.450 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.451 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.451 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:42:55.451 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:55.452 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.452 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:55.453 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6822", 2022-08-01T19:42:55.453 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:42:55.454 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.454 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.455 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:55.455 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6824", 2022-08-01T19:42:55.455 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:42:55.455 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.456 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.456 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.457 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:42:55.457 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:55.457 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.458 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:55.458 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6830", 2022-08-01T19:42:55.459 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:42:55.459 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.460 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.460 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:55.461 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6831", 2022-08-01T19:42:55.461 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:42:55.462 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.462 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.463 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.463 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:42:55.463 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:55.464 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.464 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:55.465 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6826", 2022-08-01T19:42:55.466 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:42:55.466 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.467 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.467 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:55.467 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6829", 2022-08-01T19:42:55.467 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:42:55.468 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.468 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.468 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.468 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6819/23010", 2022-08-01T19:42:55.468 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6824/23010", 2022-08-01T19:42:55.469 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6831/23010", 2022-08-01T19:42:55.469 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6829/23010", 2022-08-01T19:42:55.474 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:42:55.478 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:42:55.478 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:42:55.479 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.479 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.479 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.480 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:42:55.480 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "96123468-aabc-4a64-909e-a192c369090d", 2022-08-01T19:42:55.481 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:42:55.481 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:42:55.481 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:42:55.482 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:42:55.482 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:42:55.483 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:42:55.483 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:42:55.484 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 127, 2022-08-01T19:42:55.484 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:42:55.484 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:42:55.486 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:42:55.487 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:55.487 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.488 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:55.488 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6816", 2022-08-01T19:42:55.489 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:42:55.489 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.490 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.490 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:55.491 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6818", 2022-08-01T19:42:55.492 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:42:55.492 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.493 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.493 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.494 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:42:55.494 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:55.495 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.495 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:55.496 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6820", 2022-08-01T19:42:55.496 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:42:55.496 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.497 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.497 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:55.498 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6821", 2022-08-01T19:42:55.498 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:42:55.499 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.499 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.500 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.500 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:42:55.501 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:55.502 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.502 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:55.503 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6827", 2022-08-01T19:42:55.503 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:42:55.504 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.504 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.505 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:55.505 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6828", 2022-08-01T19:42:55.506 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:42:55.506 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.507 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.507 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.508 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:42:55.508 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:55.509 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.509 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:55.510 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6823", 2022-08-01T19:42:55.510 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:42:55.511 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.511 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.512 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:55.512 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6825", 2022-08-01T19:42:55.512 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:42:55.513 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.513 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.514 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.514 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6818/23009", 2022-08-01T19:42:55.515 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6821/23009", 2022-08-01T19:42:55.515 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6828/23009", 2022-08-01T19:42:55.515 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6825/23009", 2022-08-01T19:42:55.515 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:42:55.516 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:42:55.516 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:42:55.516 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.517 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.517 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.517 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:42:55.517 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a6e26665-e04b-46ec-ab66-e912acbf044c", 2022-08-01T19:42:55.517 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:42:55.518 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:42:55.518 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:42:55.519 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:42:55.519 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:42:55.520 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:42:55.520 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:42:55.521 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 122, 2022-08-01T19:42:55.521 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:42:55.521 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:42:55.522 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:42:55.522 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:55.523 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.523 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:55.523 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6810", 2022-08-01T19:42:55.524 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:42:55.524 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.525 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.525 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:55.526 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6811", 2022-08-01T19:42:55.526 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:42:55.527 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.527 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.528 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.528 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:42:55.529 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:55.530 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.530 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:55.531 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6812", 2022-08-01T19:42:55.531 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:42:55.532 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.532 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.533 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:55.533 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6813", 2022-08-01T19:42:55.534 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:42:55.534 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.535 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.535 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.536 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:42:55.536 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:55.537 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.537 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:55.538 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6816", 2022-08-01T19:42:55.539 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:42:55.539 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.540 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.540 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:55.541 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6817", 2022-08-01T19:42:55.542 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:42:55.542 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.543 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.543 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.544 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:42:55.544 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:55.545 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.545 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:55.545 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6814", 2022-08-01T19:42:55.546 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:42:55.547 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.547 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.547 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:55.548 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6815", 2022-08-01T19:42:55.548 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:42:55.549 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.549 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.550 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.550 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6811/23584", 2022-08-01T19:42:55.551 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6813/23584", 2022-08-01T19:42:55.551 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6817/23584", 2022-08-01T19:42:55.552 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6815/23584", 2022-08-01T19:42:55.552 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:42:55.553 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:42:55.553 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:42:55.554 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.554 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.555 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.555 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:42:55.556 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "502c0034-18ee-44ac-87ff-19b9dddb66a0", 2022-08-01T19:42:55.556 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:42:55.557 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:42:55.557 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:42:55.558 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:42:55.558 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:42:55.559 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:42:55.559 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:42:55.560 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 127, 2022-08-01T19:42:55.560 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:42:55.560 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:42:55.561 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:42:55.561 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:55.562 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.562 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:55.563 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6802", 2022-08-01T19:42:55.563 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:42:55.564 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.564 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.565 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:55.565 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6803", 2022-08-01T19:42:55.566 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:42:55.566 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.567 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.569 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.570 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:42:55.570 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:55.570 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.571 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:55.571 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6804", 2022-08-01T19:42:55.571 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:42:55.572 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.572 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.573 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:55.573 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6805", 2022-08-01T19:42:55.573 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:42:55.574 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.574 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.575 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.575 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:42:55.576 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:55.576 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.577 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:55.577 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6808", 2022-08-01T19:42:55.577 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:42:55.578 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.580 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.580 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:55.581 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6809", 2022-08-01T19:42:55.581 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:42:55.581 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.581 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.582 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.582 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:42:55.582 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:55.583 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.584 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:55.584 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6806", 2022-08-01T19:42:55.584 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:42:55.585 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.585 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.585 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:55.585 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6807", 2022-08-01T19:42:55.586 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:42:55.586 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.586 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.586 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.587 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6803/23585", 2022-08-01T19:42:55.587 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6805/23585", 2022-08-01T19:42:55.587 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6809/23585", 2022-08-01T19:42:55.587 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6807/23585", 2022-08-01T19:42:55.588 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:42:55.588 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:42:55.588 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:42:55.588 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.589 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.589 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.589 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:42:55.590 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "ac4e755a-7fe7-46cb-8407-852d5af9ffc5", 2022-08-01T19:42:55.590 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:42:55.590 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:42:55.591 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:42:55.591 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:42:55.591 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:42:55.591 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:42:55.592 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:42:55.592 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 127, 2022-08-01T19:42:55.592 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:42:55.592 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:42:55.594 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:42:55.594 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:55.594 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.594 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:55.595 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6820", 2022-08-01T19:42:55.595 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:42:55.595 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.596 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.596 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:55.596 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6821", 2022-08-01T19:42:55.596 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:42:55.596 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.597 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.597 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.597 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:42:55.597 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:55.598 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.598 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:55.598 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6823", 2022-08-01T19:42:55.599 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:42:55.599 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.599 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.600 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:55.600 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6825", 2022-08-01T19:42:55.600 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:42:55.600 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.601 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.601 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.601 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:42:55.601 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:55.602 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.602 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:55.602 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6831", 2022-08-01T19:42:55.602 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:42:55.603 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.603 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.603 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:55.604 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6833", 2022-08-01T19:42:55.604 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:42:55.604 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.605 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.605 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.605 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:42:55.605 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:55.606 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.606 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:55.606 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6827", 2022-08-01T19:42:55.606 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:42:55.607 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.607 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.607 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:55.608 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6829", 2022-08-01T19:42:55.608 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:42:55.608 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.608 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.608 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.609 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6821/23587", 2022-08-01T19:42:55.609 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6825/23587", 2022-08-01T19:42:55.609 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6833/23587", 2022-08-01T19:42:55.609 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6829/23587", 2022-08-01T19:42:55.610 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:42:55.610 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:42:55.611 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:42:55.611 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.611 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.612 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.612 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:42:55.612 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "874761e2-6410-40fc-b60c-e58abe5ce489", 2022-08-01T19:42:55.612 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:42:55.612 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:42:55.613 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:42:55.613 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:42:55.613 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:42:55.614 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:42:55.614 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:42:55.614 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 128, 2022-08-01T19:42:55.615 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:42:55.615 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:42:55.615 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:42:55.615 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:55.616 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.616 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:55.616 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6818", 2022-08-01T19:42:55.616 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:42:55.617 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.617 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.617 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:55.617 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6819", 2022-08-01T19:42:55.617 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:42:55.618 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.618 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.618 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.618 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:42:55.619 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:55.619 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.619 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:55.619 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6822", 2022-08-01T19:42:55.620 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:42:55.620 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.620 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.620 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:55.621 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6824", 2022-08-01T19:42:55.621 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:42:55.621 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.621 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.621 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.622 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:42:55.622 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:55.623 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.623 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:55.623 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6830", 2022-08-01T19:42:55.624 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:42:55.624 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.624 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.624 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:55.625 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6832", 2022-08-01T19:42:55.625 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:42:55.625 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.625 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.626 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.626 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:42:55.626 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:42:55.626 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.627 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:42:55.627 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6826", 2022-08-01T19:42:55.627 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:42:55.628 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.628 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.628 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:42:55.628 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6828", 2022-08-01T19:42:55.629 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:42:55.629 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.629 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.629 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.630 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6819/23586", 2022-08-01T19:42:55.630 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6824/23586", 2022-08-01T19:42:55.630 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6832/23586", 2022-08-01T19:42:55.630 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6828/23586", 2022-08-01T19:42:55.631 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:42:55.631 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:42:55.631 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:42:55.631 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:55.632 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.632 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:42:55.632 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_xinfo": [ 2022-08-01T19:42:55.632 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.633 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:42:55.633 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:42:55.633 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:42:55.634 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:42:55.634 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:42:55.634 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:42:55.635 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.588817+0000", 2022-08-01T19:42:55.635 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:42:55.635 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.636 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.636 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:42:55.636 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:42:55.637 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:42:55.637 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:42:55.637 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:42:55.638 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:42:55.638 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.782753+0000", 2022-08-01T19:42:55.638 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:42:55.639 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.639 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.639 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:42:55.640 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:42:55.640 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:42:55.642 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:42:55.642 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:42:55.642 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:42:55.643 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.837924+0000", 2022-08-01T19:42:55.643 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:42:55.643 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.643 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.644 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:42:55.644 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:42:55.644 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:42:55.645 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:42:55.645 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:42:55.645 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:42:55.645 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.839948+0000", 2022-08-01T19:42:55.646 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:42:55.646 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.646 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.646 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:42:55.647 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:42:55.647 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:42:55.647 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:42:55.647 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:42:55.648 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:42:55.648 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.112801+0000", 2022-08-01T19:42:55.648 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:42:55.648 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.649 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.649 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:42:55.649 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:42:55.649 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:42:55.649 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:42:55.650 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:42:55.650 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:42:55.650 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.106704+0000", 2022-08-01T19:42:55.650 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:42:55.651 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.651 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.651 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:42:55.651 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:42:55.652 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:42:55.652 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:42:55.652 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:42:55.652 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:42:55.653 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.893170+0000", 2022-08-01T19:42:55.653 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:42:55.653 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.654 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:55.654 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:42:55.654 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:42:55.654 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:42:55.655 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:42:55.657 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:42:55.657 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:42:55.658 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.886947+0000", 2022-08-01T19:42:55.658 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:42:55.659 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.659 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:42:55.659 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap": [], 2022-08-01T19:42:55.660 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap_items": [], 2022-08-01T19:42:55.660 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_temp": [], 2022-08-01T19:42:55.660 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_temp": [], 2022-08-01T19:42:55.661 INFO:teuthology.orchestra.run.smithi017.stdout: "blacklist": { 2022-08-01T19:42:55.661 INFO:teuthology.orchestra.run.smithi017.stdout: "172.21.15.77:6835/2286085186": "2022-08-02T19:42:32.216900+0000", 2022-08-01T19:42:55.662 INFO:teuthology.orchestra.run.smithi017.stdout: "172.21.15.77:6834/2286085186": "2022-08-02T19:42:32.216900+0000" 2022-08-01T19:42:55.662 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.662 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profiles": { 2022-08-01T19:42:55.663 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_data": { 2022-08-01T19:42:55.663 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-device-class": "", 2022-08-01T19:42:55.664 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:42:55.664 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-root": "default", 2022-08-01T19:42:55.664 INFO:teuthology.orchestra.run.smithi017.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T19:42:55.665 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:42:55.665 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "2", 2022-08-01T19:42:55.666 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:42:55.666 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:42:55.667 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van", 2022-08-01T19:42:55.667 INFO:teuthology.orchestra.run.smithi017.stdout: "w": "8" 2022-08-01T19:42:55.668 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.668 INFO:teuthology.orchestra.run.smithi017.stdout: "default": { 2022-08-01T19:42:55.669 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:42:55.669 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:42:55.670 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "1", 2022-08-01T19:42:55.670 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:42:55.671 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:42:55.671 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van" 2022-08-01T19:42:55.672 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:55.672 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:55.673 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps_queue": [], 2022-08-01T19:42:55.673 INFO:teuthology.orchestra.run.smithi017.stdout: "new_removed_snaps": [], 2022-08-01T19:42:55.674 INFO:teuthology.orchestra.run.smithi017.stdout: "new_purged_snaps": [], 2022-08-01T19:42:55.674 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_node_flags": {}, 2022-08-01T19:42:55.675 INFO:teuthology.orchestra.run.smithi017.stdout: "device_class_flags": {} 2022-08-01T19:42:55.675 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:42:55.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.341+0000 7f5ea2be8700 1 -- 172.21.15.17:0/3945418908 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5e7c053980 msgr2=0x7f5e7c055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:55.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.341+0000 7f5ea2be8700 1 --2- 172.21.15.17:0/3945418908 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5e7c053980 0x7f5e7c055e40 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f5e8c00b000 tx=0x7f5e8c0011f0).stop 2022-08-01T19:42:55.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.341+0000 7f5ea2be8700 1 -- 172.21.15.17:0/3945418908 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e9c0b1180 msgr2=0x7f5e9c1423a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:55.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.341+0000 7f5ea2be8700 1 --2- 172.21.15.17:0/3945418908 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e9c0b1180 0x7f5e9c1423a0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 rx=0x7f5e8c001990 tx=0x7f5e98014000).stop 2022-08-01T19:42:55.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.341+0000 7f5ea2be8700 1 -- 172.21.15.17:0/3945418908 shutdown_connections 2022-08-01T19:42:55.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.341+0000 7f5ea2be8700 1 --2- 172.21.15.17:0/3945418908 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5e7c053980 0x7f5e7c055e40 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:55.680 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.341+0000 7f5ea2be8700 1 --2- 172.21.15.17:0/3945418908 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5e9c141920 0x7f5e9c0b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:55.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.341+0000 7f5ea2be8700 1 --2- 172.21.15.17:0/3945418908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e9c0b28b0 0x7f5e9c142890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:55.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.341+0000 7f5ea2be8700 1 --2- 172.21.15.17:0/3945418908 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e9c0b1180 0x7f5e9c1423a0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:55.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.341+0000 7f5ea2be8700 1 -- 172.21.15.17:0/3945418908 >> 172.21.15.17:0/3945418908 conn(0x7f5e9c060310 msgr2=0x7f5e9c140550 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:55.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.341+0000 7f5ea2be8700 1 -- 172.21.15.17:0/3945418908 shutdown_connections 2022-08-01T19:42:55.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.341+0000 7f5ea2be8700 1 -- 172.21.15.17:0/3945418908 wait complete. 2022-08-01T19:42:55.684 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:42:55.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.797+0000 7f5aa28f6700 1 Processor -- start 2022-08-01T19:42:55.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.797+0000 7f5aa28f6700 1 -- start start 2022-08-01T19:42:55.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.797+0000 7f5aa28f6700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5a9c0b08a0 0x7f5a9c142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:55.802 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.797+0000 7f5aa28f6700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f5a9c0b3260 0x7f5a9c1433a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:55.802 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.797+0000 7f5aa28f6700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a9c0b0c80 con 0x7f5a9c0ad000 2022-08-01T19:42:55.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.797+0000 7f5aa28f6700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a9c0ae810 con 0x7f5a9c0b08a0 2022-08-01T19:42:55.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.797+0000 7f5aa28f6700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a9c0aeab0 con 0x7f5a9c0b3260 2022-08-01T19:42:55.804 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.797+0000 7f5aa18f4700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5a9c0b08a0 0x7f5a9c142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:55.804 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.797+0000 7f5aa18f4700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5a9c0b08a0 0x7f5a9c142eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44000/0 (socket says 172.21.15.17:44000) 2022-08-01T19:42:55.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.797+0000 7f5aa18f4700 1 -- 172.21.15.17:0/1800373845 learned_addr learned my addr 172.21.15.17:0/1800373845 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:55.814 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.797+0000 7f5aa10f3700 1 --2- 172.21.15.17:0/1800373845 >> v2:172.21.15.77:3300/0 conn(0x7f5a9c0b3260 0x7f5a9c1433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:55.815 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.797+0000 7f5aa18f4700 1 -- 172.21.15.17:0/1800373845 >> v2:172.21.15.77:3300/0 conn(0x7f5a9c0b3260 msgr2=0x7f5a9c1433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:55.815 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.797+0000 7f5aa18f4700 1 --2- 172.21.15.17:0/1800373845 >> v2:172.21.15.77:3300/0 conn(0x7f5a9c0b3260 0x7f5a9c1433a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:55.816 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.797+0000 7f5aa18f4700 1 -- 172.21.15.17:0/1800373845 >> v1:172.21.15.77:6789/0 conn(0x7f5a9c0ad000 legacy=0x7f5a9c05f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:55.817 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.797+0000 7f5aa18f4700 1 -- 172.21.15.17:0/1800373845 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a9c0aed50 con 0x7f5a9c0b08a0 2022-08-01T19:42:55.817 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.797+0000 7f5aa18f4700 1 --2- 172.21.15.17:0/1800373845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a9c0b08a0 0x7f5a9c142eb0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 rx=0x7f5a940032f0 tx=0x7f5a94009290).ready entity=mon.0 client_cookie=4b71cea18ba3b32b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:55.818 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.797+0000 7f5aa08f2700 1 -- 172.21.15.17:0/1800373845 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a9400b870 con 0x7f5a9c0b08a0 2022-08-01T19:42:55.818 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.797+0000 7f5aa08f2700 1 -- 172.21.15.17:0/1800373845 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9400b9d0 con 0x7f5a9c0b08a0 2022-08-01T19:42:55.818 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.797+0000 7f5aa08f2700 1 -- 172.21.15.17:0/1800373845 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a9400bb40 con 0x7f5a9c0b08a0 2022-08-01T19:42:55.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.801+0000 7f5aa28f6700 1 -- 172.21.15.17:0/1800373845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a9c0b08a0 msgr2=0x7f5a9c142eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:55.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.801+0000 7f5aa28f6700 1 --2- 172.21.15.17:0/1800373845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a9c0b08a0 0x7f5a9c142eb0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 rx=0x7f5a940032f0 tx=0x7f5a94009290).stop 2022-08-01T19:42:55.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.801+0000 7f5aa28f6700 1 -- 172.21.15.17:0/1800373845 shutdown_connections 2022-08-01T19:42:55.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.801+0000 7f5aa28f6700 1 --2- 172.21.15.17:0/1800373845 >> v2:172.21.15.77:3300/0 conn(0x7f5a9c0b3260 0x7f5a9c1433a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:55.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.801+0000 7f5aa28f6700 1 --2- 172.21.15.17:0/1800373845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a9c0b08a0 0x7f5a9c142eb0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:55.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.801+0000 7f5aa28f6700 1 -- 172.21.15.17:0/1800373845 >> 172.21.15.17:0/1800373845 conn(0x7f5a9c0612d0 msgr2=0x7f5a9c062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:55.822 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.801+0000 7f5aa28f6700 1 -- 172.21.15.17:0/1800373845 shutdown_connections 2022-08-01T19:42:55.822 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.801+0000 7f5aa28f6700 1 -- 172.21.15.17:0/1800373845 wait complete. 2022-08-01T19:42:55.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.801+0000 7f5aa28f6700 1 Processor -- start 2022-08-01T19:42:55.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.801+0000 7f5aa28f6700 1 -- start start 2022-08-01T19:42:55.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.805+0000 7f5aa28f6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a9c0ad000 0x7f5a9c066cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:55.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.805+0000 7f5aa28f6700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5a9c0b08a0 0x7f5a9c0abc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:55.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.805+0000 7f5aa28f6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a9c0b3260 0x7f5a9c152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:55.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.805+0000 7f5aa28f6700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c152850 con 0x7f5a9c0ad000 2022-08-01T19:42:55.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.805+0000 7f5aa28f6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c065ac0 con 0x7f5a9c0b3260 2022-08-01T19:42:55.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.805+0000 7f5aa28f6700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c152a80 con 0x7f5a9c0b08a0 2022-08-01T19:42:55.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.805+0000 7f5aa18f4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a9c0ad000 0x7f5a9c066cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:55.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.805+0000 7f5aa18f4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a9c0ad000 0x7f5a9c066cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44006/0 (socket says 172.21.15.17:44006) 2022-08-01T19:42:55.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.805+0000 7f5aa18f4700 1 -- 172.21.15.17:0/3675472315 learned_addr learned my addr 172.21.15.17:0/3675472315 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:55.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.805+0000 7f5aa10f3700 1 --2- 172.21.15.17:0/3675472315 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5a9c0b08a0 0x7f5a9c0abc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:55.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.805+0000 7f5aa20f5700 1 --2- 172.21.15.17:0/3675472315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a9c0b3260 0x7f5a9c152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:55.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.805+0000 7f5aa18f4700 1 -- 172.21.15.17:0/3675472315 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5a9c0b08a0 msgr2=0x7f5a9c0abc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:55.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.805+0000 7f5aa18f4700 1 --2- 172.21.15.17:0/3675472315 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5a9c0b08a0 0x7f5a9c0abc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:55.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.805+0000 7f5aa18f4700 1 -- 172.21.15.17:0/3675472315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a9c0b3260 msgr2=0x7f5a9c152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:55.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.805+0000 7f5aa18f4700 1 --2- 172.21.15.17:0/3675472315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a9c0b3260 0x7f5a9c152f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:55.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.805+0000 7f5aa18f4700 1 -- 172.21.15.17:0/3675472315 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a94009530 con 0x7f5a9c0ad000 2022-08-01T19:42:55.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.805+0000 7f5aa20f5700 1 --2- 172.21.15.17:0/3675472315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a9c0b3260 0x7f5a9c152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:42:55.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.805+0000 7f5aa18f4700 1 --2- 172.21.15.17:0/3675472315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a9c0ad000 0x7f5a9c066cc0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 rx=0x7f5a940032f0 tx=0x7f5a94016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:55.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.805+0000 7f5a9affd700 1 -- 172.21.15.17:0/3675472315 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a94021440 con 0x7f5a9c0ad000 2022-08-01T19:42:55.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.805+0000 7f5a9affd700 1 -- 172.21.15.17:0/3675472315 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a940215a0 con 0x7f5a9c0ad000 2022-08-01T19:42:55.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.805+0000 7f5aa28f6700 1 -- 172.21.15.17:0/3675472315 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a9c0ac170 con 0x7f5a9c0ad000 2022-08-01T19:42:55.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.805+0000 7f5a9affd700 1 -- 172.21.15.17:0/3675472315 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a94021900 con 0x7f5a9c0ad000 2022-08-01T19:42:55.834 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.805+0000 7f5aa28f6700 1 -- 172.21.15.17:0/3675472315 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a9c065740 con 0x7f5a9c0ad000 2022-08-01T19:42:55.834 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.809+0000 7f5aa28f6700 1 -- 172.21.15.17:0/3675472315 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a9c152c40 con 0x7f5a9c0ad000 2022-08-01T19:42:55.834 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.809+0000 7f5a9affd700 1 -- 172.21.15.17:0/3675472315 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5a9400f070 con 0x7f5a9c0ad000 2022-08-01T19:42:55.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.809+0000 7f5a9affd700 1 --2- 172.21.15.17:0/3675472315 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5a80053b00 0x7f5a80055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:55.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.809+0000 7f5a9affd700 1 -- 172.21.15.17:0/3675472315 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f5a9406d820 con 0x7f5a9c0ad000 2022-08-01T19:42:55.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.809+0000 7f5aa10f3700 1 --2- 172.21.15.17:0/3675472315 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5a80053b00 0x7f5a80055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:55.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.809+0000 7f5aa10f3700 1 --2- 172.21.15.17:0/3675472315 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5a80053b00 0x7f5a80055fc0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f5a8400c000 tx=0x7f5a84008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:55.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.813+0000 7f5a9affd700 1 -- 172.21.15.17:0/3675472315 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5a94044830 con 0x7f5a9c0ad000 2022-08-01T19:42:55.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.981+0000 7f5aa28f6700 1 -- 172.21.15.17:0/3675472315 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a9c0574e0 con 0x7f5a9c0ad000 2022-08-01T19:42:55.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.981+0000 7f5a9affd700 1 -- 172.21.15.17:0/3675472315 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+504 (secure 0 0 0) 0x7f5a94015070 con 0x7f5a9c0ad000 2022-08-01T19:42:55.986 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:42:55.986 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":65,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[]} 2022-08-01T19:42:55.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.981+0000 7f5aa28f6700 1 -- 172.21.15.17:0/3675472315 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5a80053b00 msgr2=0x7f5a80055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:55.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.981+0000 7f5aa28f6700 1 --2- 172.21.15.17:0/3675472315 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5a80053b00 0x7f5a80055fc0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f5a8400c000 tx=0x7f5a84008040).stop 2022-08-01T19:42:55.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.981+0000 7f5aa28f6700 1 -- 172.21.15.17:0/3675472315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a9c0ad000 msgr2=0x7f5a9c066cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:55.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.981+0000 7f5aa28f6700 1 --2- 172.21.15.17:0/3675472315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a9c0ad000 0x7f5a9c066cc0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 rx=0x7f5a940032f0 tx=0x7f5a94016000).stop 2022-08-01T19:42:55.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.981+0000 7f5aa28f6700 1 -- 172.21.15.17:0/3675472315 shutdown_connections 2022-08-01T19:42:55.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.981+0000 7f5aa28f6700 1 --2- 172.21.15.17:0/3675472315 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5a80053b00 0x7f5a80055fc0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:55.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.981+0000 7f5aa28f6700 1 --2- 172.21.15.17:0/3675472315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a9c0b3260 0x7f5a9c152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:55.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.981+0000 7f5aa28f6700 1 --2- 172.21.15.17:0/3675472315 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5a9c0b08a0 0x7f5a9c0abc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:55.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.981+0000 7f5aa28f6700 1 --2- 172.21.15.17:0/3675472315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a9c0ad000 0x7f5a9c066cc0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:55.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.981+0000 7f5aa28f6700 1 -- 172.21.15.17:0/3675472315 >> 172.21.15.17:0/3675472315 conn(0x7f5a9c0612d0 msgr2=0x7f5a9c0b6400 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:55.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.981+0000 7f5aa28f6700 1 -- 172.21.15.17:0/3675472315 shutdown_connections 2022-08-01T19:42:55.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:55.981+0000 7f5aa28f6700 1 -- 172.21.15.17:0/3675472315 wait complete. 2022-08-01T19:42:55.991 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 65 2022-08-01T19:42:56.000 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:42:56.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.105+0000 7f241a9ff700 1 Processor -- start 2022-08-01T19:42:56.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.105+0000 7f241a9ff700 1 -- start start 2022-08-01T19:42:56.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f241a9ff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24140614b0 0x7f2414061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:56.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f241a9ff700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24140aee80 con 0x7f24140ad750 2022-08-01T19:42:56.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f241a9ff700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24140b70b0 con 0x7f24141405d0 2022-08-01T19:42:56.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f241a9ff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24140af0b0 con 0x7f24140614b0 2022-08-01T19:42:56.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f24191fc700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f24140ad750 0x7f24140b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36208/0 (socket says 172.21.15.17:36208) 2022-08-01T19:42:56.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f24191fc700 1 -- 172.21.15.17:0/1621237972 learned_addr learned my addr 172.21.15.17:0/1621237972 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:56.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f24199fd700 1 --2- 172.21.15.17:0/1621237972 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24140614b0 0x7f2414061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:56.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f24189fb700 1 -- 172.21.15.17:0/1621237972 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 975938253 0 0) 0x7f24140aee80 con 0x7f24140ad750 2022-08-01T19:42:56.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f24189fb700 1 -- 172.21.15.17:0/1621237972 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23fc003260 con 0x7f24140ad750 2022-08-01T19:42:56.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f24189fb700 1 -- 172.21.15.17:0/1621237972 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2713330065 0 0) 0x7f23fc003260 con 0x7f24140ad750 2022-08-01T19:42:56.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f24189fb700 1 -- 172.21.15.17:0/1621237972 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24140614b0 msgr2=0x7f2414061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:56.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f24189fb700 1 --2- 172.21.15.17:0/1621237972 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24140614b0 0x7f2414061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:56.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f24189fb700 1 -- 172.21.15.17:0/1621237972 >> v1:172.21.15.77:6789/0 conn(0x7f24141405d0 legacy=0x7f24141466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:56.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f24189fb700 1 -- 172.21.15.17:0/1621237972 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24140af350 con 0x7f24140ad750 2022-08-01T19:42:56.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f24189fb700 1 -- 172.21.15.17:0/1621237972 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7f2404001660 con 0x7f24140ad750 2022-08-01T19:42:56.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f24189fb700 1 -- 172.21.15.17:0/1621237972 >> v1:172.21.15.17:6789/0 conn(0x7f24140ad750 legacy=0x7f24140b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:56.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f24189fb700 1 --2- 172.21.15.17:0/1621237972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23fc003780 0x7f23fc003b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:56.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f24189fb700 1 --2- 172.21.15.17:0/1621237972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23fc004500 0x7f23fc004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:56.134 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f24189fb700 1 --2- 172.21.15.17:0/1621237972 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24140614b0 0x7f23fc004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:56.135 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f24189fb700 1 -- 172.21.15.17:0/1621237972 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24140af0b0 con 0x7f23fc004500 2022-08-01T19:42:56.135 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f24189fb700 1 -- 172.21.15.17:0/1621237972 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23fc00b750 con 0x7f23fc003780 2022-08-01T19:42:56.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f24189fb700 1 -- 172.21.15.17:0/1621237972 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23fc00b8c0 con 0x7f24140614b0 2022-08-01T19:42:56.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f24191fc700 1 --2- 172.21.15.17:0/1621237972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23fc004500 0x7f23fc004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:56.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f24199fd700 1 --2- 172.21.15.17:0/1621237972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23fc003780 0x7f23fc003b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:56.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f24191fc700 1 -- 172.21.15.17:0/1621237972 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24140614b0 msgr2=0x7f23fc004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:56.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f241a1fe700 1 --2- 172.21.15.17:0/1621237972 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24140614b0 0x7f23fc004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:56.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f24191fc700 1 --2- 172.21.15.17:0/1621237972 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24140614b0 0x7f23fc004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:56.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f24191fc700 1 -- 172.21.15.17:0/1621237972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23fc003780 msgr2=0x7f23fc003b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:56.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f24191fc700 1 --2- 172.21.15.17:0/1621237972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23fc003780 0x7f23fc003b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:56.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f24191fc700 1 -- 172.21.15.17:0/1621237972 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23fc003260 con 0x7f23fc004500 2022-08-01T19:42:56.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f24199fd700 1 --2- 172.21.15.17:0/1621237972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23fc003780 0x7f23fc003b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:42:56.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f241a1fe700 1 --2- 172.21.15.17:0/1621237972 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24140614b0 0x7f23fc004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:42:56.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f24191fc700 1 --2- 172.21.15.17:0/1621237972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23fc004500 0x7f23fc004980 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 rx=0x7f2404001370 tx=0x7f24140af4e0).ready entity=mon.0 client_cookie=e0992fae4fb18ad0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:56.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f24189fb700 1 -- 172.21.15.17:0/1621237972 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2404007270 con 0x7f23fc004500 2022-08-01T19:42:56.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f24189fb700 1 -- 172.21.15.17:0/1621237972 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24040073d0 con 0x7f23fc004500 2022-08-01T19:42:56.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f24189fb700 1 -- 172.21.15.17:0/1621237972 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f240400ca80 con 0x7f23fc004500 2022-08-01T19:42:56.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f241a9ff700 1 -- 172.21.15.17:0/1621237972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23fc004500 msgr2=0x7f23fc004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:56.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f241a9ff700 1 --2- 172.21.15.17:0/1621237972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23fc004500 0x7f23fc004980 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 rx=0x7f2404001370 tx=0x7f24140af4e0).stop 2022-08-01T19:42:56.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f241a1fe700 1 -- 172.21.15.17:0/1621237972 reap_dead start 2022-08-01T19:42:56.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f241a9ff700 1 -- 172.21.15.17:0/1621237972 shutdown_connections 2022-08-01T19:42:56.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f241a9ff700 1 -- 172.21.15.17:0/1621237972 >> 172.21.15.17:0/1621237972 conn(0x7f2414066f70 msgr2=0x7f2414067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:56.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f241a9ff700 1 -- 172.21.15.17:0/1621237972 shutdown_connections 2022-08-01T19:42:56.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.109+0000 7f241a9ff700 1 -- 172.21.15.17:0/1621237972 wait complete. 2022-08-01T19:42:56.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.113+0000 7f241a9ff700 1 Processor -- start 2022-08-01T19:42:56.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.113+0000 7f241a9ff700 1 -- start start 2022-08-01T19:42:56.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.113+0000 7f241a9ff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f23fc004500 0x7f24140b8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:56.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.113+0000 7f241a9ff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24141558f0 0x7f24140b1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:56.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.113+0000 7f241a9ff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2414145c60 0x7f24141444a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:56.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.113+0000 7f241a9ff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2414146b00 con 0x7f2414145c60 2022-08-01T19:42:56.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.113+0000 7f241a9ff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24141465c0 con 0x7f24141558f0 2022-08-01T19:42:56.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.113+0000 7f241a9ff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2414146860 con 0x7f23fc004500 2022-08-01T19:42:56.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.113+0000 7f241a1fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2414145c60 0x7f24141444a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:56.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.113+0000 7f24191fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24141558f0 0x7f24140b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:56.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.113+0000 7f241a1fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2414145c60 0x7f24141444a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44030/0 (socket says 172.21.15.17:44030) 2022-08-01T19:42:56.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.113+0000 7f24199fd700 1 --2- 172.21.15.17:0/1929825454 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f23fc004500 0x7f24140b8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:56.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.113+0000 7f241a1fe700 1 -- 172.21.15.17:0/1929825454 learned_addr learned my addr 172.21.15.17:0/1929825454 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:56.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.113+0000 7f241a1fe700 1 -- 172.21.15.17:0/1929825454 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f23fc004500 msgr2=0x7f24140b8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:56.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.113+0000 7f241a1fe700 1 --2- 172.21.15.17:0/1929825454 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f23fc004500 0x7f24140b8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:56.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.113+0000 7f241a1fe700 1 -- 172.21.15.17:0/1929825454 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24141558f0 msgr2=0x7f24140b1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:56.154 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.113+0000 7f241a1fe700 1 --2- 172.21.15.17:0/1929825454 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24141558f0 0x7f24140b1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:56.154 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.113+0000 7f241a1fe700 1 -- 172.21.15.17:0/1929825454 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24140b12d0 con 0x7f2414145c60 2022-08-01T19:42:56.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.113+0000 7f24191fc700 1 --2- 172.21.15.17:0/1929825454 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24141558f0 0x7f24140b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:42:56.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.113+0000 7f241a1fe700 1 --2- 172.21.15.17:0/1929825454 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2414145c60 0x7f24141444a0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 rx=0x7f24040018c0 tx=0x7f240c009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:56.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.117+0000 7f2402ffd700 1 -- 172.21.15.17:0/1929825454 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f240c00de40 con 0x7f2414145c60 2022-08-01T19:42:56.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.117+0000 7f2402ffd700 1 -- 172.21.15.17:0/1929825454 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f240c002490 con 0x7f2414145c60 2022-08-01T19:42:56.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.117+0000 7f241a9ff700 1 -- 172.21.15.17:0/1929825454 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24140b8d70 con 0x7f2414145c60 2022-08-01T19:42:56.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.117+0000 7f2402ffd700 1 -- 172.21.15.17:0/1929825454 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f240c0027f0 con 0x7f2414145c60 2022-08-01T19:42:56.158 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.117+0000 7f241a9ff700 1 -- 172.21.15.17:0/1929825454 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2414145aa0 con 0x7f2414145c60 2022-08-01T19:42:56.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.117+0000 7f241a9ff700 1 -- 172.21.15.17:0/1929825454 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24140b1140 con 0x7f2414145c60 2022-08-01T19:42:56.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.117+0000 7f2402ffd700 1 -- 172.21.15.17:0/1929825454 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f240c002950 con 0x7f2414145c60 2022-08-01T19:42:56.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.117+0000 7f2402ffd700 1 --2- 172.21.15.17:0/1929825454 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f23f4053aa0 0x7f23f4055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:56.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.117+0000 7f2402ffd700 1 -- 172.21.15.17:0/1929825454 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f240c06ba50 con 0x7f2414145c60 2022-08-01T19:42:56.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.117+0000 7f24199fd700 1 --2- 172.21.15.17:0/1929825454 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f23f4053aa0 0x7f23f4055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:56.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.121+0000 7f24199fd700 1 --2- 172.21.15.17:0/1929825454 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f23f4053aa0 0x7f23f4055f60 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f241000b000 tx=0x7f2410006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:56.162 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.125+0000 7f2402ffd700 1 -- 172.21.15.17:0/1929825454 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f240c0429e0 con 0x7f2414145c60 2022-08-01T19:42:56.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.285+0000 7f241a9ff700 1 -- 172.21.15.17:0/1929825454 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24140b1fa0 con 0x7f2414145c60 2022-08-01T19:42:56.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.285+0000 7f2402ffd700 1 -- 172.21.15.17:0/1929825454 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+504 (secure 0 0 0) 0x7f24140b1fa0 con 0x7f2414145c60 2022-08-01T19:42:56.289 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:42:56.289 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":65,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[]} 2022-08-01T19:42:56.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.289+0000 7f241a9ff700 1 -- 172.21.15.17:0/1929825454 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f23f4053aa0 msgr2=0x7f23f4055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:56.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.289+0000 7f241a9ff700 1 --2- 172.21.15.17:0/1929825454 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f23f4053aa0 0x7f23f4055f60 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f241000b000 tx=0x7f2410006df0).stop 2022-08-01T19:42:56.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.289+0000 7f241a9ff700 1 -- 172.21.15.17:0/1929825454 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2414145c60 msgr2=0x7f24141444a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:56.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.289+0000 7f241a9ff700 1 --2- 172.21.15.17:0/1929825454 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2414145c60 0x7f24141444a0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 rx=0x7f24040018c0 tx=0x7f240c009040).stop 2022-08-01T19:42:56.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.289+0000 7f241a9ff700 1 -- 172.21.15.17:0/1929825454 shutdown_connections 2022-08-01T19:42:56.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.289+0000 7f241a9ff700 1 --2- 172.21.15.17:0/1929825454 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f23f4053aa0 0x7f23f4055f60 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:56.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.289+0000 7f241a9ff700 1 --2- 172.21.15.17:0/1929825454 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2414145c60 0x7f24141444a0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:56.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.289+0000 7f241a9ff700 1 --2- 172.21.15.17:0/1929825454 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24141558f0 0x7f24140b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:56.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.289+0000 7f241a9ff700 1 --2- 172.21.15.17:0/1929825454 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f23fc004500 0x7f24140b8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:56.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.289+0000 7f241a9ff700 1 -- 172.21.15.17:0/1929825454 >> 172.21.15.17:0/1929825454 conn(0x7f2414066f70 msgr2=0x7f2414141370 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:56.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.289+0000 7f241a9ff700 1 -- 172.21.15.17:0/1929825454 shutdown_connections 2022-08-01T19:42:56.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.289+0000 7f241a9ff700 1 -- 172.21.15.17:0/1929825454 wait complete. 2022-08-01T19:42:56.299 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 65 2022-08-01T19:42:56.301 INFO:tasks.ceph.mds.a:Restarting daemon 2022-08-01T19:42:56.302 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-08-01T19:42:56.302 INFO:tasks.ceph.mds.c:Restarting daemon 2022-08-01T19:42:56.303 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-08-01T19:42:56.304 INFO:tasks.ceph.mds.b:Restarting daemon 2022-08-01T19:42:56.304 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-08-01T19:42:56.307 INFO:tasks.ceph.mds.a:Started 2022-08-01T19:42:56.308 INFO:tasks.ceph.mds.c:Started 2022-08-01T19:42:56.308 DEBUG:teuthology.parallel:result is None 2022-08-01T19:42:56.309 DEBUG:teuthology.parallel:result is None 2022-08-01T19:42:56.309 INFO:tasks.ceph.mds.b:Started 2022-08-01T19:42:56.310 DEBUG:teuthology.parallel:result is None 2022-08-01T19:42:56.311 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-08-01T19:42:56.371 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2022-08-01T19:42:56.379 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2022-08-01T19:42:56.384 INFO:tasks.ceph.mds.b.smithi077.stdout:starting mds.b at 2022-08-01T19:42:56.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.397+0000 7f34eabe3700 1 Processor -- start 2022-08-01T19:42:56.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.397+0000 7f34eabe3700 1 -- start start 2022-08-01T19:42:56.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.397+0000 7f34eabe3700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f34e413c870 0x7f34e413ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:56.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.397+0000 7f34eabe3700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34e40b2dc0 con 0x7f34e40b29e0 2022-08-01T19:42:56.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.397+0000 7f34eabe3700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34e40b17f0 con 0x7f34e40ad0b0 2022-08-01T19:42:56.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.397+0000 7f34eabe3700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34e40b1340 con 0x7f34e413c870 2022-08-01T19:42:56.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.397+0000 7f34e9be1700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f34e40b29e0 0x7f34e40b0c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36258/0 (socket says 172.21.15.17:36258) 2022-08-01T19:42:56.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.401+0000 7f34e9be1700 1 -- 172.21.15.17:0/1718076904 learned_addr learned my addr 172.21.15.17:0/1718076904 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:56.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.401+0000 7f34e93e0700 1 --2- 172.21.15.17:0/1718076904 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f34e413c870 0x7f34e413ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:56.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.401+0000 7f34e8bdf700 1 -- 172.21.15.17:0/1718076904 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3056225199 0 0) 0x7f34e40b2dc0 con 0x7f34e40b29e0 2022-08-01T19:42:56.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.401+0000 7f34e93e0700 1 -- 172.21.15.17:0/1718076904 >> v1:172.21.15.77:6789/0 conn(0x7f34e40ad0b0 legacy=0x7f34e40ad540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:56.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.401+0000 7f34e93e0700 1 -- 172.21.15.17:0/1718076904 >> v1:172.21.15.17:6789/0 conn(0x7f34e40b29e0 legacy=0x7f34e40b0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:56.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.401+0000 7f34e93e0700 1 -- 172.21.15.17:0/1718076904 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e40b1a90 con 0x7f34e413c870 2022-08-01T19:42:56.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.401+0000 7f34e93e0700 1 --2- 172.21.15.17:0/1718076904 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f34e413c870 0x7f34e413ccb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f34dc001490 tx=0x7f34e0006df0).ready entity=mon.2 client_cookie=d10cfb1a92428c15 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:56.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.401+0000 7f34e8bdf700 1 -- 172.21.15.17:0/1718076904 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34e000e870 con 0x7f34e413c870 2022-08-01T19:42:56.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.401+0000 7f34e8bdf700 1 -- 172.21.15.17:0/1718076904 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e0000e90 con 0x7f34e413c870 2022-08-01T19:42:56.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.401+0000 7f34e8bdf700 1 -- 172.21.15.17:0/1718076904 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34e000ebc0 con 0x7f34e413c870 2022-08-01T19:42:56.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.401+0000 7f34eabe3700 1 -- 172.21.15.17:0/1718076904 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f34e413c870 msgr2=0x7f34e413ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:56.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.401+0000 7f34eabe3700 1 --2- 172.21.15.17:0/1718076904 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f34e413c870 0x7f34e413ccb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f34dc001490 tx=0x7f34e0006df0).stop 2022-08-01T19:42:56.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.401+0000 7f34eabe3700 1 -- 172.21.15.17:0/1718076904 shutdown_connections 2022-08-01T19:42:56.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.401+0000 7f34eabe3700 1 --2- 172.21.15.17:0/1718076904 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f34e413c870 0x7f34e413ccb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:56.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.401+0000 7f34eabe3700 1 -- 172.21.15.17:0/1718076904 >> 172.21.15.17:0/1718076904 conn(0x7f34e40613c0 msgr2=0x7f34e4062f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:56.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.401+0000 7f34eabe3700 1 -- 172.21.15.17:0/1718076904 shutdown_connections 2022-08-01T19:42:56.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.401+0000 7f34eabe3700 1 -- 172.21.15.17:0/1718076904 wait complete. 2022-08-01T19:42:56.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.401+0000 7f34eabe3700 1 Processor -- start 2022-08-01T19:42:56.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.401+0000 7f34eabe3700 1 -- start start 2022-08-01T19:42:56.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.405+0000 7f34eabe3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34e40ad0b0 0x7f34e41537d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:56.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.405+0000 7f34eabe3700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f34e40b29e0 0x7f34e40665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:56.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.405+0000 7f34eabe3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34e413c870 0x7f34e4142fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:56.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.405+0000 7f34eabe3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e4066c20 con 0x7f34e40ad0b0 2022-08-01T19:42:56.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.405+0000 7f34eabe3700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e4066ec0 con 0x7f34e413c870 2022-08-01T19:42:56.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.405+0000 7f34eabe3700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34e413fa10 con 0x7f34e40b29e0 2022-08-01T19:42:56.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.405+0000 7f34e9be1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34e40ad0b0 0x7f34e41537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:56.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.405+0000 7f34e9be1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34e40ad0b0 0x7f34e41537d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44070/0 (socket says 172.21.15.17:44070) 2022-08-01T19:42:56.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.405+0000 7f34e93e0700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f34e40b29e0 0x7f34e40665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:56.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.405+0000 7f34e9be1700 1 -- 172.21.15.17:0/745158240 learned_addr learned my addr 172.21.15.17:0/745158240 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:56.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.405+0000 7f34ea3e2700 1 --2- 172.21.15.17:0/745158240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34e413c870 0x7f34e4142fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:56.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.405+0000 7f34e9be1700 1 -- 172.21.15.17:0/745158240 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f34e40b29e0 msgr2=0x7f34e40665c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:56.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.405+0000 7f34e9be1700 1 --2- 172.21.15.17:0/745158240 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f34e40b29e0 0x7f34e40665c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:56.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.405+0000 7f34e9be1700 1 -- 172.21.15.17:0/745158240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34e413c870 msgr2=0x7f34e4142fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:56.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.405+0000 7f34e9be1700 1 --2- 172.21.15.17:0/745158240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34e413c870 0x7f34e4142fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:56.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.405+0000 7f34e9be1700 1 -- 172.21.15.17:0/745158240 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e000b180 con 0x7f34e40ad0b0 2022-08-01T19:42:56.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.405+0000 7f34e93e0700 1 --2- 172.21.15.17:0/745158240 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f34e40b29e0 0x7f34e40665c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:42:56.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.405+0000 7f34e9be1700 1 --2- 172.21.15.17:0/745158240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34e40ad0b0 0x7f34e41537d0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 rx=0x7f34d4000bc0 tx=0x7f34d4007a60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:56.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.405+0000 7f34d2ffd700 1 -- 172.21.15.17:0/745158240 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34d40070d0 con 0x7f34e40ad0b0 2022-08-01T19:42:56.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.405+0000 7f34d2ffd700 1 -- 172.21.15.17:0/745158240 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d4007230 con 0x7f34e40ad0b0 2022-08-01T19:42:56.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.405+0000 7f34d2ffd700 1 -- 172.21.15.17:0/745158240 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34d4005610 con 0x7f34e40ad0b0 2022-08-01T19:42:56.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.405+0000 7f34eabe3700 1 -- 172.21.15.17:0/745158240 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34e413b4d0 con 0x7f34e40ad0b0 2022-08-01T19:42:56.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.405+0000 7f34eabe3700 1 -- 172.21.15.17:0/745158240 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34e413feb0 con 0x7f34e40ad0b0 2022-08-01T19:42:56.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.409+0000 7f34eabe3700 1 -- 172.21.15.17:0/745158240 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34e40574e0 con 0x7f34e40ad0b0 2022-08-01T19:42:56.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.409+0000 7f34d2ffd700 1 -- 172.21.15.17:0/745158240 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f34d4010070 con 0x7f34e40ad0b0 2022-08-01T19:42:56.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.409+0000 7f34d2ffd700 1 --2- 172.21.15.17:0/745158240 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f34c4053b30 0x7f34c4055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:56.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.409+0000 7f34d2ffd700 1 -- 172.21.15.17:0/745158240 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f34d406b3d0 con 0x7f34e40ad0b0 2022-08-01T19:42:56.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.409+0000 7f34e93e0700 1 --2- 172.21.15.17:0/745158240 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f34c4053b30 0x7f34c4055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:56.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.409+0000 7f34e93e0700 1 --2- 172.21.15.17:0/745158240 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f34c4053b30 0x7f34c4055ff0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f34e001b000 tx=0x7f34e0015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:56.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.417+0000 7f34d2ffd700 1 -- 172.21.15.17:0/745158240 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f34d4042470 con 0x7f34e40ad0b0 2022-08-01T19:42:56.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:56.553+0000 7f34eabe3700 1 -- 172.21.15.17:0/745158240 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f34e405f3d0 con 0x7f34e40ad0b0 2022-08-01T19:42:57.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.397+0000 7f34d2ffd700 1 -- 172.21.15.17:0/745158240 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v133) v1 ==== 97+0+0 (secure 0 0 0) 0x7f34d4045a90 con 0x7f34e40ad0b0 2022-08-01T19:42:57.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.401+0000 7f34eabe3700 1 -- 172.21.15.17:0/745158240 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f34c4053b30 msgr2=0x7f34c4055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:57.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.401+0000 7f34eabe3700 1 --2- 172.21.15.17:0/745158240 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f34c4053b30 0x7f34c4055ff0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f34e001b000 tx=0x7f34e0015000).stop 2022-08-01T19:42:57.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.401+0000 7f34eabe3700 1 -- 172.21.15.17:0/745158240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34e40ad0b0 msgr2=0x7f34e41537d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:57.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.401+0000 7f34eabe3700 1 --2- 172.21.15.17:0/745158240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34e40ad0b0 0x7f34e41537d0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 rx=0x7f34d4000bc0 tx=0x7f34d4007a60).stop 2022-08-01T19:42:57.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.401+0000 7f34eabe3700 1 -- 172.21.15.17:0/745158240 shutdown_connections 2022-08-01T19:42:57.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.401+0000 7f34eabe3700 1 --2- 172.21.15.17:0/745158240 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f34c4053b30 0x7f34c4055ff0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:57.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.401+0000 7f34eabe3700 1 --2- 172.21.15.17:0/745158240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34e413c870 0x7f34e4142fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:57.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.401+0000 7f34eabe3700 1 --2- 172.21.15.17:0/745158240 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f34e40b29e0 0x7f34e40665c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:57.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.401+0000 7f34eabe3700 1 --2- 172.21.15.17:0/745158240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34e40ad0b0 0x7f34e41537d0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:57.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.401+0000 7f34eabe3700 1 -- 172.21.15.17:0/745158240 >> 172.21.15.17:0/745158240 conn(0x7f34e40613c0 msgr2=0x7f34e413bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:57.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.401+0000 7f34eabe3700 1 -- 172.21.15.17:0/745158240 shutdown_connections 2022-08-01T19:42:57.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.401+0000 7f34eabe3700 1 -- 172.21.15.17:0/745158240 wait complete. 2022-08-01T19:42:57.411 INFO:teuthology.orchestra.run.smithi017.stderr: removed all blacklist entries 2022-08-01T19:42:57.420 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-08-01T19:42:57.420 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-08-01T19:42:57.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.529+0000 7f67acca2700 1 Processor -- start 2022-08-01T19:42:57.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.529+0000 7f67acca2700 1 -- start start 2022-08-01T19:42:57.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.529+0000 7f67acca2700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f67a80b31d0 0x7f67a8066020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:57.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.529+0000 7f67acca2700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67a80a9da0 0x7f67a8060950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:57.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.529+0000 7f67acca2700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f67a81434b0 0x7f67a8143940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:57.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.529+0000 7f67acca2700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a8064a70 con 0x7f67a81434b0 2022-08-01T19:42:57.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.529+0000 7f67acca2700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a8061f60 con 0x7f67a80b31d0 2022-08-01T19:42:57.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.529+0000 7f67acca2700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a8067030 con 0x7f67a80a9da0 2022-08-01T19:42:57.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.533+0000 7f67a7fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f67a81434b0 0x7f67a8143940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:57.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.533+0000 7f67a7fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f67a81434b0 0x7f67a8143940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44084/0 (socket says 172.21.15.17:44084) 2022-08-01T19:42:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.533+0000 7f67a7fff700 1 -- 172.21.15.17:0/971352292 learned_addr learned my addr 172.21.15.17:0/971352292 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:57.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.533+0000 7f67a6ffd700 1 --2- 172.21.15.17:0/971352292 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67a80a9da0 0x7f67a8060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:57.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.533+0000 7f67a77fe700 1 --2- 172.21.15.17:0/971352292 >> v2:172.21.15.77:3300/0 conn(0x7f67a80b31d0 0x7f67a8066020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:57.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.533+0000 7f67a7fff700 1 -- 172.21.15.17:0/971352292 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67a80a9da0 msgr2=0x7f67a8060950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:57.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.533+0000 7f67a7fff700 1 --2- 172.21.15.17:0/971352292 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67a80a9da0 0x7f67a8060950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:57.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.533+0000 7f67a7fff700 1 -- 172.21.15.17:0/971352292 >> v2:172.21.15.77:3300/0 conn(0x7f67a80b31d0 msgr2=0x7f67a8066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:57.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.533+0000 7f67a7fff700 1 --2- 172.21.15.17:0/971352292 >> v2:172.21.15.77:3300/0 conn(0x7f67a80b31d0 0x7f67a8066020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:57.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.533+0000 7f67a7fff700 1 -- 172.21.15.17:0/971352292 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a80b35b0 con 0x7f67a81434b0 2022-08-01T19:42:57.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.533+0000 7f67a7fff700 1 --2- 172.21.15.17:0/971352292 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67a81434b0 0x7f67a8143940 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 rx=0x7f67a00032f0 tx=0x7f67a0009290).ready entity=mon.0 client_cookie=9677718f1da56638 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:57.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.533+0000 7f67a67fc700 1 -- 172.21.15.17:0/971352292 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f67a000b870 con 0x7f67a81434b0 2022-08-01T19:42:57.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.533+0000 7f67a67fc700 1 -- 172.21.15.17:0/971352292 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a000b9d0 con 0x7f67a81434b0 2022-08-01T19:42:57.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.533+0000 7f67a67fc700 1 -- 172.21.15.17:0/971352292 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f67a000bb40 con 0x7f67a81434b0 2022-08-01T19:42:57.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.533+0000 7f67acca2700 1 -- 172.21.15.17:0/971352292 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67a81434b0 msgr2=0x7f67a8143940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:57.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.533+0000 7f67acca2700 1 --2- 172.21.15.17:0/971352292 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67a81434b0 0x7f67a8143940 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 rx=0x7f67a00032f0 tx=0x7f67a0009290).stop 2022-08-01T19:42:57.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.533+0000 7f67acca2700 1 -- 172.21.15.17:0/971352292 shutdown_connections 2022-08-01T19:42:57.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.533+0000 7f67acca2700 1 --2- 172.21.15.17:0/971352292 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67a81434b0 0x7f67a8143940 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:57.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.533+0000 7f67acca2700 1 --2- 172.21.15.17:0/971352292 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67a80a9da0 0x7f67a8060950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:57.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.533+0000 7f67acca2700 1 --2- 172.21.15.17:0/971352292 >> v2:172.21.15.77:3300/0 conn(0x7f67a80b31d0 0x7f67a8066020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:57.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.533+0000 7f67acca2700 1 -- 172.21.15.17:0/971352292 >> 172.21.15.17:0/971352292 conn(0x7f67a8063870 msgr2=0x7f67a8063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:57.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.533+0000 7f67acca2700 1 -- 172.21.15.17:0/971352292 shutdown_connections 2022-08-01T19:42:57.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.533+0000 7f67acca2700 1 -- 172.21.15.17:0/971352292 wait complete. 2022-08-01T19:42:57.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.537+0000 7f67acca2700 1 Processor -- start 2022-08-01T19:42:57.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.537+0000 7f67acca2700 1 -- start start 2022-08-01T19:42:57.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.537+0000 7f67acca2700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67a80a9da0 0x7f67a8137800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:57.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.537+0000 7f67acca2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67a80b31d0 0x7f67a81358b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:57.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.537+0000 7f67acca2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67a81434b0 0x7f67a8135da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:57.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.537+0000 7f67acca2700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a814cc10 con 0x7f67a81434b0 2022-08-01T19:42:57.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.537+0000 7f67acca2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a81500f0 con 0x7f67a80b31d0 2022-08-01T19:42:57.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.537+0000 7f67acca2700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a8136290 con 0x7f67a80a9da0 2022-08-01T19:42:57.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.537+0000 7f67a6ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67a80b31d0 0x7f67a81358b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:57.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.537+0000 7f67a7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67a81434b0 0x7f67a8135da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:57.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.537+0000 7f67a6ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67a80b31d0 0x7f67a81358b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:42010/0 (socket says 172.21.15.17:42010) 2022-08-01T19:42:57.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.537+0000 7f67a6ffd700 1 -- 172.21.15.17:0/2207424356 learned_addr learned my addr 172.21.15.17:0/2207424356 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:57.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.537+0000 7f67a77fe700 1 --2- 172.21.15.17:0/2207424356 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67a80a9da0 0x7f67a8137800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:57.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.537+0000 7f67a6ffd700 1 -- 172.21.15.17:0/2207424356 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67a80a9da0 msgr2=0x7f67a8137800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:57.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.537+0000 7f67a6ffd700 1 --2- 172.21.15.17:0/2207424356 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67a80a9da0 0x7f67a8137800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:57.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.537+0000 7f67a6ffd700 1 -- 172.21.15.17:0/2207424356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67a81434b0 msgr2=0x7f67a8135da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:57.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.537+0000 7f67a6ffd700 1 --2- 172.21.15.17:0/2207424356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67a81434b0 0x7f67a8135da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:57.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.537+0000 7f67a6ffd700 1 -- 172.21.15.17:0/2207424356 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a0009530 con 0x7f67a80b31d0 2022-08-01T19:42:57.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.537+0000 7f67a6ffd700 1 --2- 172.21.15.17:0/2207424356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67a80b31d0 0x7f67a81358b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7f67a8136420 tx=0x7f6798008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:57.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.537+0000 7f67a4ff9700 1 -- 172.21.15.17:0/2207424356 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6798011780 con 0x7f67a80b31d0 2022-08-01T19:42:57.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.541+0000 7f67a4ff9700 1 -- 172.21.15.17:0/2207424356 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67980118e0 con 0x7f67a80b31d0 2022-08-01T19:42:57.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.541+0000 7f67a4ff9700 1 -- 172.21.15.17:0/2207424356 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6798011c40 con 0x7f67a80b31d0 2022-08-01T19:42:57.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.541+0000 7f67acca2700 1 -- 172.21.15.17:0/2207424356 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a80aca80 con 0x7f67a80b31d0 2022-08-01T19:42:57.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.541+0000 7f67acca2700 1 -- 172.21.15.17:0/2207424356 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a80acfc0 con 0x7f67a80b31d0 2022-08-01T19:42:57.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.541+0000 7f67acca2700 1 -- 172.21.15.17:0/2207424356 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a80ad180 con 0x7f67a80b31d0 2022-08-01T19:42:57.602 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.541+0000 7f67a4ff9700 1 -- 172.21.15.17:0/2207424356 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6798012420 con 0x7f67a80b31d0 2022-08-01T19:42:57.602 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.541+0000 7f67a4ff9700 1 --2- 172.21.15.17:0/2207424356 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f678c053980 0x7f678c055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:57.602 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.541+0000 7f67a4ff9700 1 -- 172.21.15.17:0/2207424356 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f679806ba70 con 0x7f67a80b31d0 2022-08-01T19:42:57.602 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.541+0000 7f67a77fe700 1 --2- 172.21.15.17:0/2207424356 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f678c053980 0x7f678c055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:57.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.545+0000 7f67a77fe700 1 --2- 172.21.15.17:0/2207424356 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f678c053980 0x7f678c055e40 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f679000c000 tx=0x7f6790008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:57.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.553+0000 7f67a4ff9700 1 -- 172.21.15.17:0/2207424356 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6798042b10 con 0x7f67a80b31d0 2022-08-01T19:42:57.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.721+0000 7f67acca2700 1 -- 172.21.15.17:0/2207424356 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f67a8066af0 con 0x7f67a80b31d0 2022-08-01T19:42:57.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.725+0000 7f67a4ff9700 1 -- 172.21.15.17:0/2207424356 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v33) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f679806be60 con 0x7f67a80b31d0 2022-08-01T19:42:57.726 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:42:57.727 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:42:57.727 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_dump": [ 2022-08-01T19:42:57.727 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:57.727 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.a", 2022-08-01T19:42:57.728 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhijotxORAAVc04rsrlFj0NShsyc6Qbhg==", 2022-08-01T19:42:57.728 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:42:57.728 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:42:57.728 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:42:57.728 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:42:57.729 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:42:57.729 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:57.730 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:57.730 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:57.731 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.b", 2022-08-01T19:42:57.731 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhiud2xBxAAcn6/O5ysdIp/oIbF5Y1dAw==", 2022-08-01T19:42:57.732 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:42:57.732 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:42:57.733 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:42:57.733 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:42:57.733 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:42:57.734 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:57.735 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:57.735 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:57.735 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.c", 2022-08-01T19:42:57.736 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+KuhiMzMbBBAAfdT5mtpYwboNL+7f2B09KQ==", 2022-08-01T19:42:57.736 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:42:57.737 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:42:57.737 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:42:57.738 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:42:57.738 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:42:57.739 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:57.739 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:57.739 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:57.740 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.0", 2022-08-01T19:42:57.740 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhia27/NRAAZXBp+nYelJVJ6Mge9r+3Vg==", 2022-08-01T19:42:57.741 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:42:57.741 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:42:57.741 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:42:57.742 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:42:57.742 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:57.742 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:57.743 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:57.743 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.1", 2022-08-01T19:42:57.743 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCBKuhiOggRGRAAvgA0MxwBJwmpXV6S9DTdjA==", 2022-08-01T19:42:57.743 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:42:57.744 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:42:57.744 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:42:57.744 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:42:57.744 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:57.745 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:57.745 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:57.745 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.2", 2022-08-01T19:42:57.745 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCDKuhi5PgOORAAhcORK4bW2T3dXPeYjcCm+g==", 2022-08-01T19:42:57.746 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:42:57.746 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:42:57.746 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:42:57.746 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:42:57.747 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:57.747 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:57.747 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:57.748 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.3", 2022-08-01T19:42:57.748 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCGKuhiutn6GxAAk2ucLuGbT5YV/R4xXQGh0w==", 2022-08-01T19:42:57.748 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:42:57.749 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:42:57.749 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:42:57.750 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:42:57.750 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:57.751 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:57.751 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:57.751 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.4", 2022-08-01T19:42:57.752 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCLKuhiGHTRHxAArr3TwXA0gsCAAF/Qs38wUQ==", 2022-08-01T19:42:57.752 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:42:57.753 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:42:57.753 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:42:57.753 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:42:57.754 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:57.754 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:57.754 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:57.755 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.5", 2022-08-01T19:42:57.755 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCOKuhiMMDbBBAAvTzBvOcv7pFWm/P/lNqB0w==", 2022-08-01T19:42:57.756 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:42:57.756 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:42:57.757 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:42:57.757 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:42:57.758 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:57.758 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:57.758 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:57.759 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.6", 2022-08-01T19:42:57.759 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCQKuhiG2H3IxAAufUcWqo9ULquxIrUD5vSfA==", 2022-08-01T19:42:57.760 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:42:57.760 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:42:57.760 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:42:57.761 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:42:57.761 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:57.762 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:57.762 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:57.763 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.7", 2022-08-01T19:42:57.763 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCTKuhiU50LChAAXuhrYBbHkk3eKzAPiEJFTg==", 2022-08-01T19:42:57.763 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:42:57.764 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:42:57.764 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:42:57.765 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:42:57.765 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:57.766 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:57.766 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:57.766 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.0", 2022-08-01T19:42:57.767 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhig3EnCxAANM20jSIQf/AKwuRX1qwT/A==", 2022-08-01T19:42:57.767 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:42:57.768 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:42:57.768 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow r", 2022-08-01T19:42:57.768 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T19:42:57.769 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:57.769 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:57.769 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:57.770 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.1", 2022-08-01T19:42:57.770 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+KuhicJR0DhAAivqEAZUhXnSwjVRwWm2bJg==", 2022-08-01T19:42:57.770 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:42:57.771 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:42:57.771 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow r", 2022-08-01T19:42:57.771 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T19:42:57.771 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:57.771 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:57.772 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:57.772 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.admin", 2022-08-01T19:42:57.772 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhiPTTGHxAAgfthCR6RX6j8F1YseFGNeA==", 2022-08-01T19:42:57.772 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:42:57.773 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:42:57.773 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:42:57.773 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:42:57.773 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:42:57.774 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:57.774 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:57.774 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:57.775 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.x", 2022-08-01T19:42:57.775 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9Kuhi2jvCNhAAyUxiGByZ+OZr8Y7dh3Qt5w==", 2022-08-01T19:42:57.775 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:42:57.775 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:42:57.775 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2022-08-01T19:42:57.776 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:42:57.776 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:57.776 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:57.776 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:57.777 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.y", 2022-08-01T19:42:57.777 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhiDJXpMhAA8aek67ydbRgNO0h+miws+w==", 2022-08-01T19:42:57.777 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:42:57.777 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:42:57.777 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2022-08-01T19:42:57.778 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:42:57.778 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:57.778 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:57.778 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:42:57.779 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:42:57.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.725+0000 7f67acca2700 1 -- 172.21.15.17:0/2207424356 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f678c053980 msgr2=0x7f678c055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:57.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.725+0000 7f67acca2700 1 --2- 172.21.15.17:0/2207424356 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f678c053980 0x7f678c055e40 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f679000c000 tx=0x7f6790008040).stop 2022-08-01T19:42:57.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.725+0000 7f67acca2700 1 -- 172.21.15.17:0/2207424356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67a80b31d0 msgr2=0x7f67a81358b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:57.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.725+0000 7f67acca2700 1 --2- 172.21.15.17:0/2207424356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67a80b31d0 0x7f67a81358b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7f67a8136420 tx=0x7f6798008040).stop 2022-08-01T19:42:57.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.725+0000 7f67acca2700 1 -- 172.21.15.17:0/2207424356 shutdown_connections 2022-08-01T19:42:57.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.725+0000 7f67acca2700 1 --2- 172.21.15.17:0/2207424356 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f678c053980 0x7f678c055e40 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:57.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.725+0000 7f67acca2700 1 --2- 172.21.15.17:0/2207424356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67a81434b0 0x7f67a8135da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:57.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.725+0000 7f67acca2700 1 --2- 172.21.15.17:0/2207424356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67a80b31d0 0x7f67a81358b0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:57.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.725+0000 7f67acca2700 1 --2- 172.21.15.17:0/2207424356 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67a80a9da0 0x7f67a8137800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:57.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.725+0000 7f67acca2700 1 -- 172.21.15.17:0/2207424356 >> 172.21.15.17:0/2207424356 conn(0x7f67a8063870 msgr2=0x7f67a8142ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:57.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.725+0000 7f67acca2700 1 -- 172.21.15.17:0/2207424356 shutdown_connections 2022-08-01T19:42:57.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.725+0000 7f67acca2700 1 -- 172.21.15.17:0/2207424356 wait complete. 2022-08-01T19:42:57.784 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:42:57.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.885+0000 7f76d96e2700 1 Processor -- start 2022-08-01T19:42:57.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.885+0000 7f76d96e2700 1 -- start start 2022-08-01T19:42:57.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.889+0000 7f76d96e2700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f76d40ad750 0x7f76d4061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:57.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.889+0000 7f76d96e2700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76d40aee80 con 0x7f76d41405d0 2022-08-01T19:42:57.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.889+0000 7f76d96e2700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76d40b70b0 con 0x7f76d40614b0 2022-08-01T19:42:57.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.889+0000 7f76d96e2700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76d40af0b0 con 0x7f76d40ad750 2022-08-01T19:42:57.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.889+0000 7f76d37fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f76d40ad750 0x7f76d4061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:57.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.889+0000 7f76d37fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f76d40ad750 0x7f76d4061890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44096/0 (socket says 172.21.15.17:44096) 2022-08-01T19:42:57.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.889+0000 7f76d37fe700 1 -- 172.21.15.17:0/4058658799 learned_addr learned my addr 172.21.15.17:0/4058658799 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:57.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.889+0000 7f76d37fe700 1 -- 172.21.15.17:0/4058658799 >> v1:172.21.15.77:6789/0 conn(0x7f76d40614b0 legacy=0x7f76d40b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:57.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.889+0000 7f76d37fe700 1 -- 172.21.15.17:0/4058658799 >> v1:172.21.15.17:6789/0 conn(0x7f76d41405d0 legacy=0x7f76d41466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:57.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.889+0000 7f76d2ffd700 1 -- 172.21.15.17:0/4058658799 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 753342922 0 0) 0x7f76d40aee80 con 0x7f76d41405d0 2022-08-01T19:42:57.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.889+0000 7f76d37fe700 1 -- 172.21.15.17:0/4058658799 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d40af350 con 0x7f76d40ad750 2022-08-01T19:42:57.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.889+0000 7f76d37fe700 1 --2- 172.21.15.17:0/4058658799 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76d40ad750 0x7f76d4061890 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 rx=0x7f76bc002f70 tx=0x7f76bc006eb0).ready entity=mon.0 client_cookie=177a233449526469 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:57.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.889+0000 7f76d2ffd700 1 -- 172.21.15.17:0/4058658799 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76bc00d660 con 0x7f76d40ad750 2022-08-01T19:42:57.899 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.889+0000 7f76d2ffd700 1 -- 172.21.15.17:0/4058658799 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76bc000e90 con 0x7f76d40ad750 2022-08-01T19:42:57.899 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.889+0000 7f76d2ffd700 1 -- 172.21.15.17:0/4058658799 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76bc00d9b0 con 0x7f76d40ad750 2022-08-01T19:42:57.905 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.889+0000 7f76d96e2700 1 -- 172.21.15.17:0/4058658799 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76d40ad750 msgr2=0x7f76d4061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:57.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.889+0000 7f76d96e2700 1 --2- 172.21.15.17:0/4058658799 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76d40ad750 0x7f76d4061890 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 rx=0x7f76bc002f70 tx=0x7f76bc006eb0).stop 2022-08-01T19:42:57.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.889+0000 7f76d96e2700 1 -- 172.21.15.17:0/4058658799 shutdown_connections 2022-08-01T19:42:57.907 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.889+0000 7f76d96e2700 1 --2- 172.21.15.17:0/4058658799 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76d40ad750 0x7f76d4061890 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:57.907 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.889+0000 7f76d96e2700 1 -- 172.21.15.17:0/4058658799 >> 172.21.15.17:0/4058658799 conn(0x7f76d4066f70 msgr2=0x7f76d4067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:57.908 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.889+0000 7f76d96e2700 1 -- 172.21.15.17:0/4058658799 shutdown_connections 2022-08-01T19:42:57.909 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.889+0000 7f76d96e2700 1 -- 172.21.15.17:0/4058658799 wait complete. 2022-08-01T19:42:57.909 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.893+0000 7f76d96e2700 1 Processor -- start 2022-08-01T19:42:57.910 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.893+0000 7f76d96e2700 1 -- start start 2022-08-01T19:42:57.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.893+0000 7f76d96e2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76d40614b0 0x7f76d40ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:57.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.893+0000 7f76d96e2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76d40ad750 0x7f76d40ab3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:57.912 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.893+0000 7f76d96e2700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f76d41405d0 0x7f76d41523d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:57.912 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.893+0000 7f76d96e2700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d40acfa0 con 0x7f76d40614b0 2022-08-01T19:42:57.913 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.893+0000 7f76d96e2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d40abef0 con 0x7f76d40ad750 2022-08-01T19:42:57.913 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.893+0000 7f76d96e2700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76d40ac3e0 con 0x7f76d41405d0 2022-08-01T19:42:57.913 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.897+0000 7f76d3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76d40614b0 0x7f76d40ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:57.914 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.897+0000 7f76d3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76d40614b0 0x7f76d40ac880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44100/0 (socket says 172.21.15.17:44100) 2022-08-01T19:42:57.914 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.897+0000 7f76d3fff700 1 -- 172.21.15.17:0/877067646 learned_addr learned my addr 172.21.15.17:0/877067646 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:57.915 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.897+0000 7f76d37fe700 1 --2- 172.21.15.17:0/877067646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76d40ad750 0x7f76d40ab3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:57.915 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.897+0000 7f76d8ee1700 1 --2- 172.21.15.17:0/877067646 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f76d41405d0 0x7f76d41523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:57.916 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.897+0000 7f76d37fe700 1 -- 172.21.15.17:0/877067646 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f76d41405d0 msgr2=0x7f76d41523d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:57.916 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.897+0000 7f76d37fe700 1 --2- 172.21.15.17:0/877067646 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f76d41405d0 0x7f76d41523d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:57.917 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.897+0000 7f76d37fe700 1 -- 172.21.15.17:0/877067646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76d40614b0 msgr2=0x7f76d40ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:57.917 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.897+0000 7f76d37fe700 1 --2- 172.21.15.17:0/877067646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76d40614b0 0x7f76d40ac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:57.917 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.897+0000 7f76d37fe700 1 -- 172.21.15.17:0/877067646 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76bc00b1e0 con 0x7f76d40ad750 2022-08-01T19:42:57.918 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.897+0000 7f76d3fff700 1 --2- 172.21.15.17:0/877067646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76d40614b0 0x7f76d40ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:42:57.918 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.897+0000 7f76d37fe700 1 --2- 172.21.15.17:0/877067646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76d40ad750 0x7f76d40ab3e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f76bc002f70 tx=0x7f76bc015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:57.919 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.897+0000 7f76d17fa700 1 -- 172.21.15.17:0/877067646 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76bc011860 con 0x7f76d40ad750 2022-08-01T19:42:57.919 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.897+0000 7f76d17fa700 1 -- 172.21.15.17:0/877067646 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76bc0119c0 con 0x7f76d40ad750 2022-08-01T19:42:57.919 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.897+0000 7f76d96e2700 1 -- 172.21.15.17:0/877067646 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d40ac120 con 0x7f76d40ad750 2022-08-01T19:42:57.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.897+0000 7f76d17fa700 1 -- 172.21.15.17:0/877067646 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76bc011d20 con 0x7f76d40ad750 2022-08-01T19:42:57.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.897+0000 7f76d96e2700 1 -- 172.21.15.17:0/877067646 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d40ad1d0 con 0x7f76d40ad750 2022-08-01T19:42:57.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.897+0000 7f76d96e2700 1 -- 172.21.15.17:0/877067646 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76d40ac5a0 con 0x7f76d40ad750 2022-08-01T19:42:57.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.901+0000 7f76d17fa700 1 -- 172.21.15.17:0/877067646 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f76bc025030 con 0x7f76d40ad750 2022-08-01T19:42:57.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.901+0000 7f76d17fa700 1 --2- 172.21.15.17:0/877067646 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f76b8053a70 0x7f76b8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:57.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.901+0000 7f76d17fa700 1 -- 172.21.15.17:0/877067646 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f76bc06d720 con 0x7f76d40ad750 2022-08-01T19:42:57.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.901+0000 7f76d3fff700 1 --2- 172.21.15.17:0/877067646 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f76b8053a70 0x7f76b8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:57.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.901+0000 7f76d3fff700 1 --2- 172.21.15.17:0/877067646 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f76b8053a70 0x7f76b8055f30 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f76c400d000 tx=0x7f76c4009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:57.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:57.909+0000 7f76d17fa700 1 -- 172.21.15.17:0/877067646 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f76bc044710 con 0x7f76d40ad750 2022-08-01T19:42:58.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.037+0000 7f76d96e2700 1 -- 172.21.15.17:0/877067646 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76d40af5f0 con 0x7f76d40ad750 2022-08-01T19:42:58.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.037+0000 7f76d17fa700 1 -- 172.21.15.17:0/877067646 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+13424 (secure 0 0 0) 0x7f76bc023090 con 0x7f76d40ad750 2022-08-01T19:42:58.045 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:42:58.045 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":133,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:42:57.392217+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:42:58.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.041+0000 7f76d96e2700 1 -- 172.21.15.17:0/877067646 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f76b8053a70 msgr2=0x7f76b8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:58.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.041+0000 7f76d96e2700 1 --2- 172.21.15.17:0/877067646 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f76b8053a70 0x7f76b8055f30 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f76c400d000 tx=0x7f76c4009040).stop 2022-08-01T19:42:58.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.041+0000 7f76d96e2700 1 -- 172.21.15.17:0/877067646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76d40ad750 msgr2=0x7f76d40ab3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:58.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.041+0000 7f76d96e2700 1 --2- 172.21.15.17:0/877067646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76d40ad750 0x7f76d40ab3e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f76bc002f70 tx=0x7f76bc015000).stop 2022-08-01T19:42:58.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.041+0000 7f76d96e2700 1 -- 172.21.15.17:0/877067646 shutdown_connections 2022-08-01T19:42:58.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.041+0000 7f76d96e2700 1 --2- 172.21.15.17:0/877067646 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f76b8053a70 0x7f76b8055f30 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:58.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.041+0000 7f76d96e2700 1 --2- 172.21.15.17:0/877067646 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f76d41405d0 0x7f76d41523d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:58.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.041+0000 7f76d96e2700 1 --2- 172.21.15.17:0/877067646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76d40ad750 0x7f76d40ab3e0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:58.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.041+0000 7f76d96e2700 1 --2- 172.21.15.17:0/877067646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76d40614b0 0x7f76d40ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:58.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.041+0000 7f76d96e2700 1 -- 172.21.15.17:0/877067646 >> 172.21.15.17:0/877067646 conn(0x7f76d4066f70 msgr2=0x7f76d4067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:58.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.041+0000 7f76d96e2700 1 -- 172.21.15.17:0/877067646 shutdown_connections 2022-08-01T19:42:58.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.041+0000 7f76d96e2700 1 -- 172.21.15.17:0/877067646 wait complete. 2022-08-01T19:42:58.059 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-08-01T19:42:58.060 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:42:58.169 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.165+0000 7fedc25d5700 1 Processor -- start 2022-08-01T19:42:58.170 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.165+0000 7fedc25d5700 1 -- start start 2022-08-01T19:42:58.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.169+0000 7fedc25d5700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fedbc05f840 0x7fedbc0a8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:58.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.169+0000 7fedc25d5700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fedbc05c620 0x7fedbc0a92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:58.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.169+0000 7fedc25d5700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedbc1419f0 con 0x7fedbc0af3c0 2022-08-01T19:42:58.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.169+0000 7fedc25d5700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedbc0b1b60 con 0x7fedbc05f840 2022-08-01T19:42:58.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.169+0000 7fedc25d5700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedbc141c20 con 0x7fedbc05c620 2022-08-01T19:42:58.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.169+0000 7fedc0dd2700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fedbc05f840 0x7fedbc0a8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:58.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.169+0000 7fedc15d3700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7fedbc0af3c0 0x7fedbc0601a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:53102/0 (socket says 172.21.15.17:53102) 2022-08-01T19:42:58.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.169+0000 7fedc15d3700 1 -- 172.21.15.17:0/2642580738 learned_addr learned my addr 172.21.15.17:0/2642580738 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:58.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.169+0000 7fedc1dd4700 1 --2- 172.21.15.17:0/2642580738 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fedbc05c620 0x7fedbc0a92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:58.182 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.169+0000 7fedc0dd2700 1 -- 172.21.15.17:0/2642580738 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fedbc05c620 msgr2=0x7fedbc0a92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:58.182 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.169+0000 7fedc0dd2700 1 --2- 172.21.15.17:0/2642580738 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fedbc05c620 0x7fedbc0a92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:58.182 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.169+0000 7fedc0dd2700 1 -- 172.21.15.17:0/2642580738 >> v1:172.21.15.77:6789/0 conn(0x7fedbc0af3c0 legacy=0x7fedbc0601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:58.182 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.169+0000 7fedc0dd2700 1 -- 172.21.15.17:0/2642580738 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedbc141ec0 con 0x7fedbc05f840 2022-08-01T19:42:58.183 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.169+0000 7fedc0dd2700 1 --2- 172.21.15.17:0/2642580738 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedbc05f840 0x7fedbc0a8de0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7fedb8002f70 tx=0x7fedb8006df0).ready entity=mon.1 client_cookie=3e5b3f8db7e6cc4f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:58.183 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.169+0000 7fedb3fff700 1 -- 172.21.15.17:0/2642580738 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fedb800d570 con 0x7fedbc05f840 2022-08-01T19:42:58.183 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.169+0000 7fedb3fff700 1 -- 172.21.15.17:0/2642580738 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb8000e90 con 0x7fedbc05f840 2022-08-01T19:42:58.183 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.169+0000 7fedb3fff700 1 -- 172.21.15.17:0/2642580738 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fedb800d8c0 con 0x7fedbc05f840 2022-08-01T19:42:58.184 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.169+0000 7fedc25d5700 1 -- 172.21.15.17:0/2642580738 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedbc05f840 msgr2=0x7fedbc0a8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:58.184 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.169+0000 7fedc25d5700 1 --2- 172.21.15.17:0/2642580738 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedbc05f840 0x7fedbc0a8de0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7fedb8002f70 tx=0x7fedb8006df0).stop 2022-08-01T19:42:58.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.169+0000 7fedc25d5700 1 -- 172.21.15.17:0/2642580738 shutdown_connections 2022-08-01T19:42:58.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.169+0000 7fedc25d5700 1 --2- 172.21.15.17:0/2642580738 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fedbc05c620 0x7fedbc0a92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:58.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.169+0000 7fedc25d5700 1 --2- 172.21.15.17:0/2642580738 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedbc05f840 0x7fedbc0a8de0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:58.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.169+0000 7fedc25d5700 1 -- 172.21.15.17:0/2642580738 >> 172.21.15.17:0/2642580738 conn(0x7fedbc061360 msgr2=0x7fedbc062f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:58.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.169+0000 7fedc25d5700 1 -- 172.21.15.17:0/2642580738 shutdown_connections 2022-08-01T19:42:58.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.169+0000 7fedc25d5700 1 -- 172.21.15.17:0/2642580738 wait complete. 2022-08-01T19:42:58.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.169+0000 7fedc25d5700 1 Processor -- start 2022-08-01T19:42:58.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.169+0000 7fedc25d5700 1 -- start start 2022-08-01T19:42:58.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.173+0000 7fedc25d5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fedbc05c620 0x7fedbc064e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:58.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.173+0000 7fedc25d5700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fedbc05f840 0x7fedbc143e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:58.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.173+0000 7fedc25d5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedbc0af3c0 0x7fedbc144320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:58.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.173+0000 7fedc25d5700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedbc150570 con 0x7fedbc05c620 2022-08-01T19:42:58.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.173+0000 7fedc25d5700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedbc1514e0 con 0x7fedbc0af3c0 2022-08-01T19:42:58.188 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.173+0000 7fedc25d5700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedbc143030 con 0x7fedbc05f840 2022-08-01T19:42:58.188 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.173+0000 7fedc15d3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fedbc05c620 0x7fedbc064e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:58.188 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.173+0000 7fedc15d3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fedbc05c620 0x7fedbc064e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44116/0 (socket says 172.21.15.17:44116) 2022-08-01T19:42:58.188 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.173+0000 7fedc15d3700 1 -- 172.21.15.17:0/3021134692 learned_addr learned my addr 172.21.15.17:0/3021134692 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:58.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.173+0000 7fedc0dd2700 1 --2- 172.21.15.17:0/3021134692 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fedbc05f840 0x7fedbc143e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:58.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.173+0000 7fedc1dd4700 1 --2- 172.21.15.17:0/3021134692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedbc0af3c0 0x7fedbc144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:58.190 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.173+0000 7fedc15d3700 1 -- 172.21.15.17:0/3021134692 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fedbc05f840 msgr2=0x7fedbc143e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:58.190 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.173+0000 7fedc15d3700 1 --2- 172.21.15.17:0/3021134692 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fedbc05f840 0x7fedbc143e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:58.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.173+0000 7fedc15d3700 1 -- 172.21.15.17:0/3021134692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedbc0af3c0 msgr2=0x7fedbc144320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:58.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.173+0000 7fedc15d3700 1 --2- 172.21.15.17:0/3021134692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedbc0af3c0 0x7fedbc144320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:58.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.173+0000 7fedc15d3700 1 -- 172.21.15.17:0/3021134692 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb800b180 con 0x7fedbc05c620 2022-08-01T19:42:58.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.173+0000 7fedc15d3700 1 --2- 172.21.15.17:0/3021134692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fedbc05c620 0x7fedbc064e30 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 rx=0x7fedbc0aff00 tx=0x7fedac007ca0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:58.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.173+0000 7fedb27fc700 1 -- 172.21.15.17:0/3021134692 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fedac0070d0 con 0x7fedbc05c620 2022-08-01T19:42:58.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.173+0000 7fedb27fc700 1 -- 172.21.15.17:0/3021134692 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedac007230 con 0x7fedbc05c620 2022-08-01T19:42:58.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.173+0000 7fedc25d5700 1 -- 172.21.15.17:0/3021134692 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedbc142b70 con 0x7fedbc05c620 2022-08-01T19:42:58.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.173+0000 7fedb27fc700 1 -- 172.21.15.17:0/3021134692 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fedac005610 con 0x7fedbc05c620 2022-08-01T19:42:58.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.173+0000 7fedc25d5700 1 -- 172.21.15.17:0/3021134692 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedbc1433d0 con 0x7fedbc05c620 2022-08-01T19:42:58.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.173+0000 7fedc25d5700 1 -- 172.21.15.17:0/3021134692 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedbc1506c0 con 0x7fedbc05c620 2022-08-01T19:42:58.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.177+0000 7fedb27fc700 1 -- 172.21.15.17:0/3021134692 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fedac010070 con 0x7fedbc05c620 2022-08-01T19:42:58.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.177+0000 7fedb27fc700 1 --2- 172.21.15.17:0/3021134692 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7feda4053a70 0x7feda4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:58.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.177+0000 7fedb27fc700 1 -- 172.21.15.17:0/3021134692 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fedac06b300 con 0x7fedbc05c620 2022-08-01T19:42:58.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.177+0000 7fedc0dd2700 1 --2- 172.21.15.17:0/3021134692 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7feda4053a70 0x7feda4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:58.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.177+0000 7fedc0dd2700 1 --2- 172.21.15.17:0/3021134692 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7feda4053a70 0x7feda4055f30 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7fedb801a000 tx=0x7fedb8014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:58.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.185+0000 7fedb27fc700 1 -- 172.21.15.17:0/3021134692 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fedbc1506c0 con 0x7fedbc05c620 2022-08-01T19:42:58.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.305+0000 7fedc25d5700 1 -- 172.21.15.17:0/3021134692 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fedbc1506c0 con 0x7fedbc05c620 2022-08-01T19:42:58.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.309+0000 7fedb27fc700 1 -- 172.21.15.17:0/3021134692 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v133) v1 ==== 110+0+10 (secure 0 0 0) 0x7fedbc1506c0 con 0x7fedbc05c620 2022-08-01T19:42:58.311 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:42:58.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.309+0000 7fedc25d5700 1 -- 172.21.15.17:0/3021134692 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7feda4053a70 msgr2=0x7feda4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:58.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.309+0000 7fedc25d5700 1 --2- 172.21.15.17:0/3021134692 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7feda4053a70 0x7feda4055f30 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7fedb801a000 tx=0x7fedb8014000).stop 2022-08-01T19:42:58.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.309+0000 7fedc25d5700 1 -- 172.21.15.17:0/3021134692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fedbc05c620 msgr2=0x7fedbc064e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:58.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.309+0000 7fedc25d5700 1 --2- 172.21.15.17:0/3021134692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fedbc05c620 0x7fedbc064e30 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 rx=0x7fedbc0aff00 tx=0x7fedac007ca0).stop 2022-08-01T19:42:58.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.309+0000 7fedc25d5700 1 -- 172.21.15.17:0/3021134692 shutdown_connections 2022-08-01T19:42:58.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.309+0000 7fedc25d5700 1 --2- 172.21.15.17:0/3021134692 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7feda4053a70 0x7feda4055f30 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:58.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.309+0000 7fedc25d5700 1 --2- 172.21.15.17:0/3021134692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fedbc0af3c0 0x7fedbc144320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:58.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.309+0000 7fedc25d5700 1 --2- 172.21.15.17:0/3021134692 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fedbc05f840 0x7fedbc143e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:58.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.309+0000 7fedc25d5700 1 --2- 172.21.15.17:0/3021134692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fedbc05c620 0x7fedbc064e30 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:58.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.309+0000 7fedc25d5700 1 -- 172.21.15.17:0/3021134692 >> 172.21.15.17:0/3021134692 conn(0x7fedbc061360 msgr2=0x7fedbc13d430 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:58.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.309+0000 7fedc25d5700 1 -- 172.21.15.17:0/3021134692 shutdown_connections 2022-08-01T19:42:58.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.309+0000 7fedc25d5700 1 -- 172.21.15.17:0/3021134692 wait complete. 2022-08-01T19:42:58.324 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:42:58.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.433+0000 7fd0e3fff700 1 Processor -- start 2022-08-01T19:42:58.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.433+0000 7fd0e3fff700 1 -- start start 2022-08-01T19:42:58.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.433+0000 7fd0e3fff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fd0e40b6420 0x7fd0e4061600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:58.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.433+0000 7fd0e3fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd0e4142ff0 0x7fd0e41434a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:58.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.433+0000 7fd0e3fff700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0e40b60e0 con 0x7fd0e40b41f0 2022-08-01T19:42:58.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.433+0000 7fd0e3fff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0e40b2100 con 0x7fd0e40b6420 2022-08-01T19:42:58.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.433+0000 7fd0e3fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e40b23a0 con 0x7fd0e4142ff0 2022-08-01T19:42:58.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.433+0000 7fd0e37fe700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd0e4142ff0 0x7fd0e41434a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:58.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.437+0000 7fd0da7fc700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7fd0e40b41f0 0x7fd0e4061af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:53118/0 (socket says 172.21.15.17:53118) 2022-08-01T19:42:58.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.437+0000 7fd0e2ffd700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fd0e40b6420 0x7fd0e4061600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:58.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.437+0000 7fd0e37fe700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd0e4142ff0 0x7fd0e41434a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:40226/0 (socket says 172.21.15.17:40226) 2022-08-01T19:42:58.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.437+0000 7fd0da7fc700 1 -- 172.21.15.17:0/1819398439 learned_addr learned my addr 172.21.15.17:0/1819398439 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:58.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.437+0000 7fd0e37fe700 1 -- 172.21.15.17:0/1819398439 >> v2:172.21.15.77:3300/0 conn(0x7fd0e40b6420 msgr2=0x7fd0e4061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:58.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.437+0000 7fd0e37fe700 1 --2- 172.21.15.17:0/1819398439 >> v2:172.21.15.77:3300/0 conn(0x7fd0e40b6420 0x7fd0e4061600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:58.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.437+0000 7fd0e37fe700 1 -- 172.21.15.17:0/1819398439 >> v1:172.21.15.77:6789/0 conn(0x7fd0e40b41f0 legacy=0x7fd0e4061af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:58.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.437+0000 7fd0e27fc700 1 -- 172.21.15.17:0/1819398439 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1200099033 0 0) 0x7fd0cc001110 con 0x7fd0e40b41f0 2022-08-01T19:42:58.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.437+0000 7fd0e37fe700 1 -- 172.21.15.17:0/1819398439 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e40b2640 con 0x7fd0e4142ff0 2022-08-01T19:42:58.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.437+0000 7fd0e37fe700 1 --2- 172.21.15.17:0/1819398439 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd0e4142ff0 0x7fd0e41434a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fd0dc0032f0 tx=0x7fd0dc0091d0).ready entity=mon.2 client_cookie=5fbb014dddbab357 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:58.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.437+0000 7fd0e27fc700 1 -- 172.21.15.17:0/1819398439 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0dc00b6c0 con 0x7fd0e4142ff0 2022-08-01T19:42:58.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.437+0000 7fd0e27fc700 1 -- 172.21.15.17:0/1819398439 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0dc00b820 con 0x7fd0e4142ff0 2022-08-01T19:42:58.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.437+0000 7fd0e27fc700 1 -- 172.21.15.17:0/1819398439 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0dc00b990 con 0x7fd0e4142ff0 2022-08-01T19:42:58.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.437+0000 7fd0e3fff700 1 -- 172.21.15.17:0/1819398439 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd0e4142ff0 msgr2=0x7fd0e41434a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:58.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.437+0000 7fd0e3fff700 1 --2- 172.21.15.17:0/1819398439 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd0e4142ff0 0x7fd0e41434a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fd0dc0032f0 tx=0x7fd0dc0091d0).stop 2022-08-01T19:42:58.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.437+0000 7fd0e3fff700 1 -- 172.21.15.17:0/1819398439 shutdown_connections 2022-08-01T19:42:58.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.437+0000 7fd0e3fff700 1 --2- 172.21.15.17:0/1819398439 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd0e4142ff0 0x7fd0e41434a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:58.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.437+0000 7fd0e3fff700 1 --2- 172.21.15.17:0/1819398439 >> v2:172.21.15.77:3300/0 conn(0x7fd0e40b6420 0x7fd0e4061600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:58.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.437+0000 7fd0e3fff700 1 -- 172.21.15.17:0/1819398439 >> 172.21.15.17:0/1819398439 conn(0x7fd0e405c620 msgr2=0x7fd0e405ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:58.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.437+0000 7fd0e3fff700 1 -- 172.21.15.17:0/1819398439 shutdown_connections 2022-08-01T19:42:58.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.437+0000 7fd0e3fff700 1 -- 172.21.15.17:0/1819398439 wait complete. 2022-08-01T19:42:58.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.441+0000 7fd0e3fff700 1 Processor -- start 2022-08-01T19:42:58.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.441+0000 7fd0e3fff700 1 -- start start 2022-08-01T19:42:58.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.441+0000 7fd0e3fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd0e40b41f0 0x7fd0e413eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:58.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.441+0000 7fd0e3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0e40b6420 0x7fd0e40af490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:58.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.441+0000 7fd0e3fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd0e4142ff0 0x7fd0e40af980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:58.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.441+0000 7fd0e3fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e40ae050 con 0x7fd0e40b6420 2022-08-01T19:42:58.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.441+0000 7fd0e3fff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e4146e60 con 0x7fd0e40b41f0 2022-08-01T19:42:58.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.441+0000 7fd0e3fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e4146680 con 0x7fd0e4142ff0 2022-08-01T19:42:58.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.441+0000 7fd0da7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0e40b6420 0x7fd0e40af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:58.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.441+0000 7fd0da7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0e40b6420 0x7fd0e40af490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44130/0 (socket says 172.21.15.17:44130) 2022-08-01T19:42:58.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.441+0000 7fd0da7fc700 1 -- 172.21.15.17:0/3033316903 learned_addr learned my addr 172.21.15.17:0/3033316903 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:58.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.441+0000 7fd0e2ffd700 1 --2- 172.21.15.17:0/3033316903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd0e40b41f0 0x7fd0e413eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:58.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.441+0000 7fd0e37fe700 1 --2- 172.21.15.17:0/3033316903 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd0e4142ff0 0x7fd0e40af980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:58.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.441+0000 7fd0da7fc700 1 -- 172.21.15.17:0/3033316903 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd0e4142ff0 msgr2=0x7fd0e40af980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:58.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.441+0000 7fd0da7fc700 1 --2- 172.21.15.17:0/3033316903 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd0e4142ff0 0x7fd0e40af980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:58.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.441+0000 7fd0da7fc700 1 -- 172.21.15.17:0/3033316903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd0e40b41f0 msgr2=0x7fd0e413eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:58.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.441+0000 7fd0da7fc700 1 --2- 172.21.15.17:0/3033316903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd0e40b41f0 0x7fd0e413eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:58.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.441+0000 7fd0da7fc700 1 -- 172.21.15.17:0/3033316903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0dc009470 con 0x7fd0e40b6420 2022-08-01T19:42:58.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.441+0000 7fd0da7fc700 1 --2- 172.21.15.17:0/3033316903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0e40b6420 0x7fd0e40af490 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 rx=0x7fd0e4147040 tx=0x7fd0cc007900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:58.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.441+0000 7fd0e0ff9700 1 -- 172.21.15.17:0/3033316903 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0cc00ce70 con 0x7fd0e40b6420 2022-08-01T19:42:58.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.441+0000 7fd0e0ff9700 1 -- 172.21.15.17:0/3033316903 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0cc0072e0 con 0x7fd0e40b6420 2022-08-01T19:42:58.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.441+0000 7fd0e0ff9700 1 -- 172.21.15.17:0/3033316903 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0cc005610 con 0x7fd0e40b6420 2022-08-01T19:42:58.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.441+0000 7fd0e3fff700 1 -- 172.21.15.17:0/3033316903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0e4146920 con 0x7fd0e40b6420 2022-08-01T19:42:58.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.441+0000 7fd0e3fff700 1 -- 172.21.15.17:0/3033316903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e413f520 con 0x7fd0e40b6420 2022-08-01T19:42:58.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.445+0000 7fd0e3fff700 1 -- 172.21.15.17:0/3033316903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0e40af180 con 0x7fd0e40b6420 2022-08-01T19:42:58.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.445+0000 7fd0e0ff9700 1 -- 172.21.15.17:0/3033316903 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd0cc010070 con 0x7fd0e40b6420 2022-08-01T19:42:58.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.445+0000 7fd0e0ff9700 1 --2- 172.21.15.17:0/3033316903 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd0bc053a10 0x7fd0bc055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:58.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.445+0000 7fd0e0ff9700 1 -- 172.21.15.17:0/3033316903 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fd0cc06b220 con 0x7fd0e40b6420 2022-08-01T19:42:58.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.445+0000 7fd0e2ffd700 1 --2- 172.21.15.17:0/3033316903 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd0bc053a10 0x7fd0bc055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:58.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.449+0000 7fd0e2ffd700 1 --2- 172.21.15.17:0/3033316903 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd0bc053a10 0x7fd0bc055ed0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7fd0d400b000 tx=0x7fd0d4006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:58.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.453+0000 7fd0e0ff9700 1 -- 172.21.15.17:0/3033316903 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd0cc042320 con 0x7fd0e40b6420 2022-08-01T19:42:58.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.577+0000 7fd0e3fff700 1 -- 172.21.15.17:0/3033316903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd0e40b60e0 con 0x7fd0e40b6420 2022-08-01T19:42:58.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.577+0000 7fd0e0ff9700 1 -- 172.21.15.17:0/3033316903 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v133) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd0e40b60e0 con 0x7fd0e40b6420 2022-08-01T19:42:58.579 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:42:58.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.577+0000 7fd0e3fff700 1 -- 172.21.15.17:0/3033316903 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd0bc053a10 msgr2=0x7fd0bc055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:58.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.577+0000 7fd0e3fff700 1 --2- 172.21.15.17:0/3033316903 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd0bc053a10 0x7fd0bc055ed0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7fd0d400b000 tx=0x7fd0d4006df0).stop 2022-08-01T19:42:58.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.577+0000 7fd0e3fff700 1 -- 172.21.15.17:0/3033316903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0e40b6420 msgr2=0x7fd0e40af490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:58.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.577+0000 7fd0e3fff700 1 --2- 172.21.15.17:0/3033316903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0e40b6420 0x7fd0e40af490 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 rx=0x7fd0e4147040 tx=0x7fd0cc007900).stop 2022-08-01T19:42:58.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.577+0000 7fd0e3fff700 1 -- 172.21.15.17:0/3033316903 shutdown_connections 2022-08-01T19:42:58.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.577+0000 7fd0e3fff700 1 --2- 172.21.15.17:0/3033316903 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd0bc053a10 0x7fd0bc055ed0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:58.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.577+0000 7fd0e3fff700 1 --2- 172.21.15.17:0/3033316903 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd0e4142ff0 0x7fd0e40af980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:58.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.577+0000 7fd0e3fff700 1 --2- 172.21.15.17:0/3033316903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0e40b6420 0x7fd0e40af490 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:58.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.577+0000 7fd0e3fff700 1 --2- 172.21.15.17:0/3033316903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd0e40b41f0 0x7fd0e413eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:58.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.577+0000 7fd0e3fff700 1 -- 172.21.15.17:0/3033316903 >> 172.21.15.17:0/3033316903 conn(0x7fd0e405c620 msgr2=0x7fd0e4142350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:58.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.577+0000 7fd0e3fff700 1 -- 172.21.15.17:0/3033316903 shutdown_connections 2022-08-01T19:42:58.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.577+0000 7fd0e3fff700 1 -- 172.21.15.17:0/3033316903 wait complete. 2022-08-01T19:42:58.592 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-08-01T19:42:58.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.697+0000 7fa9f8930700 1 Processor -- start 2022-08-01T19:42:58.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.697+0000 7fa9f8930700 1 -- start start 2022-08-01T19:42:58.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.697+0000 7fa9f8930700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa9f40b07a0 0x7fa9f40af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:58.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.697+0000 7fa9f8930700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa9f4147730 0x7fa9f40acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:58.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.697+0000 7fa9f8930700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9f40b5a10 con 0x7fa9f40b5630 2022-08-01T19:42:58.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.697+0000 7fa9f8930700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f4145470 con 0x7fa9f40b07a0 2022-08-01T19:42:58.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.697+0000 7fa9f8930700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f40ae180 con 0x7fa9f4147730 2022-08-01T19:42:58.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.697+0000 7fa9f259c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa9f40b07a0 0x7fa9f40af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:58.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.697+0000 7fa9f259c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa9f40b07a0 0x7fa9f40af6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44136/0 (socket says 172.21.15.17:44136) 2022-08-01T19:42:58.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.697+0000 7fa9f259c700 1 -- 172.21.15.17:0/1113344025 learned_addr learned my addr 172.21.15.17:0/1113344025 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:58.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.697+0000 7fa9f2d9d700 1 --1- 172.21.15.17:0/1113344025 >> v1:172.21.15.17:6789/0 conn(0x7fa9f40b5630 0x7fa9f4067730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36330/0 (socket says 172.21.15.17:36330) 2022-08-01T19:42:58.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.697+0000 7fa9f359e700 1 --2- 172.21.15.17:0/1113344025 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa9f4147730 0x7fa9f40acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:58.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.701+0000 7fa9f259c700 1 -- 172.21.15.17:0/1113344025 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa9f4147730 msgr2=0x7fa9f40acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:42:58.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.701+0000 7fa9f259c700 1 --2- 172.21.15.17:0/1113344025 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa9f4147730 0x7fa9f40acdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:58.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.701+0000 7fa9f259c700 1 -- 172.21.15.17:0/1113344025 >> v1:172.21.15.17:6789/0 conn(0x7fa9f40b5630 legacy=0x7fa9f4067730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:58.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.701+0000 7fa9f259c700 1 -- 172.21.15.17:0/1113344025 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f40ac990 con 0x7fa9f40b07a0 2022-08-01T19:42:58.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.701+0000 7fa9f259c700 1 --2- 172.21.15.17:0/1113344025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9f40b07a0 0x7fa9f40af6e0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 rx=0x7fa9e4002f70 tx=0x7fa9e4006eb0).ready entity=mon.0 client_cookie=921ffbbe1b13605a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:58.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.701+0000 7fa9f1d9b700 1 -- 172.21.15.17:0/1113344025 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9e400d5c0 con 0x7fa9f40b07a0 2022-08-01T19:42:58.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.701+0000 7fa9f1d9b700 1 -- 172.21.15.17:0/1113344025 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e4000e90 con 0x7fa9f40b07a0 2022-08-01T19:42:58.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.701+0000 7fa9f1d9b700 1 -- 172.21.15.17:0/1113344025 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9e400d950 con 0x7fa9f40b07a0 2022-08-01T19:42:58.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.701+0000 7fa9f8930700 1 -- 172.21.15.17:0/1113344025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9f40b07a0 msgr2=0x7fa9f40af6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:58.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.701+0000 7fa9f8930700 1 --2- 172.21.15.17:0/1113344025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9f40b07a0 0x7fa9f40af6e0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 rx=0x7fa9e4002f70 tx=0x7fa9e4006eb0).stop 2022-08-01T19:42:58.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.701+0000 7fa9f8930700 1 -- 172.21.15.17:0/1113344025 shutdown_connections 2022-08-01T19:42:58.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.701+0000 7fa9f8930700 1 --2- 172.21.15.17:0/1113344025 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa9f4147730 0x7fa9f40acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:58.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.701+0000 7fa9f8930700 1 --2- 172.21.15.17:0/1113344025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9f40b07a0 0x7fa9f40af6e0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:58.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.701+0000 7fa9f8930700 1 -- 172.21.15.17:0/1113344025 >> 172.21.15.17:0/1113344025 conn(0x7fa9f40626f0 msgr2=0x7fa9f4062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:58.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.701+0000 7fa9f8930700 1 -- 172.21.15.17:0/1113344025 shutdown_connections 2022-08-01T19:42:58.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.701+0000 7fa9f8930700 1 -- 172.21.15.17:0/1113344025 wait complete. 2022-08-01T19:42:58.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.705+0000 7fa9f8930700 1 Processor -- start 2022-08-01T19:42:58.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.705+0000 7fa9f8930700 1 -- start start 2022-08-01T19:42:58.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.705+0000 7fa9f8930700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa9f40b07a0 0x7fa9f40b4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:58.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.705+0000 7fa9f8930700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9f40b5630 0x7fa9f4157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:58.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.705+0000 7fa9f8930700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9f4147730 0x7fa9f414e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:58.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.705+0000 7fa9f8930700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f40b5c40 con 0x7fa9f40b5630 2022-08-01T19:42:58.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.705+0000 7fa9f8930700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f415a3c0 con 0x7fa9f4147730 2022-08-01T19:42:58.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.705+0000 7fa9f8930700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f4157d30 con 0x7fa9f40b07a0 2022-08-01T19:42:58.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.705+0000 7fa9f259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9f40b5630 0x7fa9f4157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:58.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.705+0000 7fa9f259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9f40b5630 0x7fa9f4157840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44142/0 (socket says 172.21.15.17:44142) 2022-08-01T19:42:58.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.705+0000 7fa9f259c700 1 -- 172.21.15.17:0/1935249867 learned_addr learned my addr 172.21.15.17:0/1935249867 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:58.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.705+0000 7fa9f2d9d700 1 --2- 172.21.15.17:0/1935249867 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa9f40b07a0 0x7fa9f40b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:58.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.705+0000 7fa9f359e700 1 --2- 172.21.15.17:0/1935249867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9f4147730 0x7fa9f414e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:58.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.705+0000 7fa9f259c700 1 -- 172.21.15.17:0/1935249867 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa9f40b07a0 msgr2=0x7fa9f40b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:58.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.705+0000 7fa9f259c700 1 --2- 172.21.15.17:0/1935249867 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa9f40b07a0 0x7fa9f40b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:58.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.705+0000 7fa9f259c700 1 -- 172.21.15.17:0/1935249867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9f4147730 msgr2=0x7fa9f414e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:58.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.705+0000 7fa9f259c700 1 --2- 172.21.15.17:0/1935249867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9f4147730 0x7fa9f414e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:58.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.705+0000 7fa9f259c700 1 -- 172.21.15.17:0/1935249867 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e400b1e0 con 0x7fa9f40b5630 2022-08-01T19:42:58.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.705+0000 7fa9f2d9d700 1 --2- 172.21.15.17:0/1935249867 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa9f40b07a0 0x7fa9f40b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:42:58.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.705+0000 7fa9f259c700 1 --2- 172.21.15.17:0/1935249867 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9f40b5630 0x7fa9f4157840 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 rx=0x7fa9f4157ec0 tx=0x7fa9e4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:58.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.705+0000 7fa9e3fff700 1 -- 172.21.15.17:0/1935249867 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9e4006b40 con 0x7fa9f40b5630 2022-08-01T19:42:58.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.705+0000 7fa9e3fff700 1 -- 172.21.15.17:0/1935249867 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e400d940 con 0x7fa9f40b5630 2022-08-01T19:42:58.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.705+0000 7fa9f8930700 1 -- 172.21.15.17:0/1935249867 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f414f190 con 0x7fa9f40b5630 2022-08-01T19:42:58.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.709+0000 7fa9e3fff700 1 -- 172.21.15.17:0/1935249867 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9e4011e50 con 0x7fa9f40b5630 2022-08-01T19:42:58.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.709+0000 7fa9f8930700 1 -- 172.21.15.17:0/1935249867 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f41573a0 con 0x7fa9f40b5630 2022-08-01T19:42:58.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.709+0000 7fa9f8930700 1 -- 172.21.15.17:0/1935249867 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f4156f60 con 0x7fa9f40b5630 2022-08-01T19:42:58.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.709+0000 7fa9e3fff700 1 -- 172.21.15.17:0/1935249867 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa9e4025030 con 0x7fa9f40b5630 2022-08-01T19:42:58.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.709+0000 7fa9e3fff700 1 --2- 172.21.15.17:0/1935249867 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa9d0053950 0x7fa9d0055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:58.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.709+0000 7fa9e3fff700 1 -- 172.21.15.17:0/1935249867 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fa9e406d090 con 0x7fa9f40b5630 2022-08-01T19:42:58.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.709+0000 7fa9f2d9d700 1 --2- 172.21.15.17:0/1935249867 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa9d0053950 0x7fa9d0055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:58.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.713+0000 7fa9f2d9d700 1 --2- 172.21.15.17:0/1935249867 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa9d0053950 0x7fa9d0055e10 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7fa9e800b000 tx=0x7fa9e80070e0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:58.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.713+0000 7fa9e3fff700 1 -- 172.21.15.17:0/1935249867 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa9f4156f60 con 0x7fa9f40b5630 2022-08-01T19:42:58.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.845+0000 7fa9f8930700 1 -- 172.21.15.17:0/1935249867 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa9f40b62a0 con 0x7fa9f40b5630 2022-08-01T19:42:58.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.845+0000 7fa9e3fff700 1 -- 172.21.15.17:0/1935249867 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v133) v1 ==== 84+0+143 (secure 0 0 0) 0x7fa9e4023090 con 0x7fa9f40b5630 2022-08-01T19:42:58.849 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:42:58.849 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2022-08-01T19:42:58.850 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:58.850 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 1, 2022-08-01T19:42:58.851 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": "device_health_metrics" 2022-08-01T19:42:58.852 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:42:58.852 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:42:58.853 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 2, 2022-08-01T19:42:58.853 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": "rbd" 2022-08-01T19:42:58.853 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:42:58.854 INFO:teuthology.orchestra.run.smithi017.stdout:] 2022-08-01T19:42:58.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.845+0000 7fa9f8930700 1 -- 172.21.15.17:0/1935249867 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa9d0053950 msgr2=0x7fa9d0055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:58.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.845+0000 7fa9f8930700 1 --2- 172.21.15.17:0/1935249867 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa9d0053950 0x7fa9d0055e10 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7fa9e800b000 tx=0x7fa9e80070e0).stop 2022-08-01T19:42:58.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.845+0000 7fa9f8930700 1 -- 172.21.15.17:0/1935249867 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9f40b5630 msgr2=0x7fa9f4157840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:58.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.845+0000 7fa9f8930700 1 --2- 172.21.15.17:0/1935249867 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9f40b5630 0x7fa9f4157840 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 rx=0x7fa9f4157ec0 tx=0x7fa9e4015000).stop 2022-08-01T19:42:58.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.845+0000 7fa9f8930700 1 -- 172.21.15.17:0/1935249867 shutdown_connections 2022-08-01T19:42:58.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.845+0000 7fa9f8930700 1 --2- 172.21.15.17:0/1935249867 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa9d0053950 0x7fa9d0055e10 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:58.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.845+0000 7fa9f8930700 1 --2- 172.21.15.17:0/1935249867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9f4147730 0x7fa9f414e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:58.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.845+0000 7fa9f8930700 1 --2- 172.21.15.17:0/1935249867 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9f40b5630 0x7fa9f4157840 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:58.858 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.845+0000 7fa9f8930700 1 --2- 172.21.15.17:0/1935249867 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa9f40b07a0 0x7fa9f40b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:58.858 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.845+0000 7fa9f8930700 1 -- 172.21.15.17:0/1935249867 >> 172.21.15.17:0/1935249867 conn(0x7fa9f40626f0 msgr2=0x7fa9f4144720 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:58.858 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.845+0000 7fa9f8930700 1 -- 172.21.15.17:0/1935249867 shutdown_connections 2022-08-01T19:42:58.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.849+0000 7fa9f8930700 1 -- 172.21.15.17:0/1935249867 wait complete. 2022-08-01T19:42:58.861 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-08-01T19:42:58.862 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-08-01T19:42:58.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.969+0000 7fa849257700 1 Processor -- start 2022-08-01T19:42:58.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.969+0000 7fa849257700 1 -- start start 2022-08-01T19:42:58.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.969+0000 7fa849257700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa8440b2710 0x7fa84414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:58.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.969+0000 7fa849257700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8440a7250 con 0x7fa8440b01a0 2022-08-01T19:42:58.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.969+0000 7fa849257700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8440a7480 con 0x7fa8440a8c50 2022-08-01T19:42:58.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.969+0000 7fa849257700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8440a7720 con 0x7fa8440b2710 2022-08-01T19:42:58.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.969+0000 7fa8437fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa8440b01a0 0x7fa8440a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36346/0 (socket says 172.21.15.17:36346) 2022-08-01T19:42:58.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.969+0000 7fa8437fe700 1 -- 172.21.15.17:0/3456509910 learned_addr learned my addr 172.21.15.17:0/3456509910 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:42:58.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.969+0000 7fa843fff700 1 --2- 172.21.15.17:0/3456509910 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa8440b2710 0x7fa84414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:58.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.969+0000 7fa842ffd700 1 -- 172.21.15.17:0/3456509910 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 949987669 0 0) 0x7fa8440a7250 con 0x7fa8440b01a0 2022-08-01T19:42:58.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.969+0000 7fa842ffd700 1 -- 172.21.15.17:0/3456509910 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa830003260 con 0x7fa8440b01a0 2022-08-01T19:42:58.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.969+0000 7fa842ffd700 1 -- 172.21.15.17:0/3456509910 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2724476374 0 0) 0x7fa8440a7480 con 0x7fa8440a8c50 2022-08-01T19:42:58.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.969+0000 7fa842ffd700 1 -- 172.21.15.17:0/3456509910 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8440a7250 con 0x7fa8440a8c50 2022-08-01T19:42:58.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.969+0000 7fa843fff700 1 -- 172.21.15.17:0/3456509910 >> v1:172.21.15.77:6789/0 conn(0x7fa8440a8c50 legacy=0x7fa844151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:58.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.969+0000 7fa842ffd700 1 -- 172.21.15.17:0/3456509910 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4066733564 0 0) 0x7fa830003260 con 0x7fa8440b01a0 2022-08-01T19:42:58.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.969+0000 7fa843fff700 1 -- 172.21.15.17:0/3456509910 >> v1:172.21.15.17:6789/0 conn(0x7fa8440b01a0 legacy=0x7fa8440a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:58.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.969+0000 7fa843fff700 1 -- 172.21.15.17:0/3456509910 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8440a79c0 con 0x7fa8440b2710 2022-08-01T19:42:58.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.973+0000 7fa843fff700 1 --2- 172.21.15.17:0/3456509910 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa8440b2710 0x7fa84414afd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fa838002f70 tx=0x7fa838006df0).ready entity=mon.2 client_cookie=c5d8eb5d3127fb50 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:58.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.973+0000 7fa842ffd700 1 -- 172.21.15.17:0/3456509910 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa83800d5c0 con 0x7fa8440b2710 2022-08-01T19:42:58.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.973+0000 7fa842ffd700 1 -- 172.21.15.17:0/3456509910 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa838000e90 con 0x7fa8440b2710 2022-08-01T19:42:58.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.973+0000 7fa842ffd700 1 -- 172.21.15.17:0/3456509910 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa83800d910 con 0x7fa8440b2710 2022-08-01T19:42:58.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.973+0000 7fa849257700 1 -- 172.21.15.17:0/3456509910 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa8440b2710 msgr2=0x7fa84414afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:58.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.973+0000 7fa849257700 1 --2- 172.21.15.17:0/3456509910 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa8440b2710 0x7fa84414afd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fa838002f70 tx=0x7fa838006df0).stop 2022-08-01T19:42:58.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.973+0000 7fa849257700 1 -- 172.21.15.17:0/3456509910 shutdown_connections 2022-08-01T19:42:58.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.973+0000 7fa849257700 1 --2- 172.21.15.17:0/3456509910 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa8440b2710 0x7fa84414afd0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:58.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.973+0000 7fa849257700 1 -- 172.21.15.17:0/3456509910 >> 172.21.15.17:0/3456509910 conn(0x7fa844060310 msgr2=0x7fa8440b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:58.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.973+0000 7fa849257700 1 -- 172.21.15.17:0/3456509910 shutdown_connections 2022-08-01T19:42:58.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.973+0000 7fa849257700 1 -- 172.21.15.17:0/3456509910 wait complete. 2022-08-01T19:42:58.991 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.973+0000 7fa849257700 1 Processor -- start 2022-08-01T19:42:58.992 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.973+0000 7fa849257700 1 -- start start 2022-08-01T19:42:58.992 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.973+0000 7fa849257700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8440a8c50 0x7fa844144690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:58.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.973+0000 7fa849257700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa8440b01a0 0x7fa8440aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:58.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.973+0000 7fa849257700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8440b2710 0x7fa84415a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:58.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.973+0000 7fa849257700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84415aba0 con 0x7fa8440a8c50 2022-08-01T19:42:58.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.973+0000 7fa849257700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa844157540 con 0x7fa8440b2710 2022-08-01T19:42:58.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.973+0000 7fa849257700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa844159cd0 con 0x7fa8440b01a0 2022-08-01T19:42:58.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.973+0000 7fa843fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8440a8c50 0x7fa844144690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:58.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.977+0000 7fa843fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8440a8c50 0x7fa844144690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44158/0 (socket says 172.21.15.17:44158) 2022-08-01T19:42:58.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.977+0000 7fa843fff700 1 -- 172.21.15.17:0/2038089415 learned_addr learned my addr 172.21.15.17:0/2038089415 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:42:58.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.977+0000 7fa8437fe700 1 --2- 172.21.15.17:0/2038089415 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa8440b01a0 0x7fa8440aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:58.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.977+0000 7fa848a56700 1 --2- 172.21.15.17:0/2038089415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8440b2710 0x7fa84415a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:58.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.977+0000 7fa8437fe700 1 -- 172.21.15.17:0/2038089415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8440b2710 msgr2=0x7fa84415a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:58.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.977+0000 7fa8437fe700 1 --2- 172.21.15.17:0/2038089415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8440b2710 0x7fa84415a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:58.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.977+0000 7fa8437fe700 1 -- 172.21.15.17:0/2038089415 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8440a8c50 msgr2=0x7fa844144690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:58.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.977+0000 7fa8437fe700 1 --2- 172.21.15.17:0/2038089415 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8440a8c50 0x7fa844144690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:58.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.977+0000 7fa8437fe700 1 -- 172.21.15.17:0/2038089415 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa83800b180 con 0x7fa8440b01a0 2022-08-01T19:42:58.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.977+0000 7fa8437fe700 1 --2- 172.21.15.17:0/2038089415 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa8440b01a0 0x7fa8440aedd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7fa834000f40 tx=0x7fa834007b00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:58.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.977+0000 7fa8417fa700 1 -- 172.21.15.17:0/2038089415 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa834004ab0 con 0x7fa8440b01a0 2022-08-01T19:42:58.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.977+0000 7fa8417fa700 1 -- 172.21.15.17:0/2038089415 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa834004c10 con 0x7fa8440b01a0 2022-08-01T19:42:59.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.977+0000 7fa849257700 1 -- 172.21.15.17:0/2038089415 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa844159f70 con 0x7fa8440b01a0 2022-08-01T19:42:59.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.977+0000 7fa8417fa700 1 -- 172.21.15.17:0/2038089415 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa834007170 con 0x7fa8440b01a0 2022-08-01T19:42:59.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.977+0000 7fa849257700 1 -- 172.21.15.17:0/2038089415 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8441571c0 con 0x7fa8440b01a0 2022-08-01T19:42:59.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.977+0000 7fa849257700 1 -- 172.21.15.17:0/2038089415 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa84414a890 con 0x7fa8440b01a0 2022-08-01T19:42:59.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.977+0000 7fa8417fa700 1 -- 172.21.15.17:0/2038089415 <== mon.2 v2:172.21.15.77:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa8340072d0 con 0x7fa8440b01a0 2022-08-01T19:42:59.002 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.977+0000 7fa8417fa700 1 --2- 172.21.15.17:0/2038089415 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa8240539e0 0x7fa824055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:42:59.002 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.977+0000 7fa8417fa700 1 -- 172.21.15.17:0/2038089415 <== mon.2 v2:172.21.15.77:3301/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fa83406c3f0 con 0x7fa8440b01a0 2022-08-01T19:42:59.003 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.981+0000 7fa843fff700 1 --2- 172.21.15.17:0/2038089415 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa8240539e0 0x7fa824055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:42:59.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.981+0000 7fa843fff700 1 --2- 172.21.15.17:0/2038089415 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa8240539e0 0x7fa824055ea0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7fa838018000 tx=0x7fa838014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:42:59.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:58.989+0000 7fa8417fa700 1 -- 172.21.15.17:0/2038089415 <== mon.2 v2:172.21.15.77:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa8340433e0 con 0x7fa8440b01a0 2022-08-01T19:42:59.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:59.109+0000 7fa849257700 1 -- 172.21.15.17:0/2038089415 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fa8440574e0 con 0x7fa8440b01a0 2022-08-01T19:42:59.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:59.961+0000 7fa8417fa700 1 -- 172.21.15.17:0/2038089415 <== mon.2 v2:172.21.15.77:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v134) v1 ==== 133+0+0 (secure 0 0 0) 0x7fa8340360a0 con 0x7fa8440b01a0 2022-08-01T19:42:59.967 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:59.965+0000 7fa849257700 1 -- 172.21.15.17:0/2038089415 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa8240539e0 msgr2=0x7fa824055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:59.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:59.965+0000 7fa849257700 1 --2- 172.21.15.17:0/2038089415 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa8240539e0 0x7fa824055ea0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7fa838018000 tx=0x7fa838014000).stop 2022-08-01T19:42:59.968 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:59.965+0000 7fa849257700 1 -- 172.21.15.17:0/2038089415 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa8440b01a0 msgr2=0x7fa8440aedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:42:59.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:59.965+0000 7fa849257700 1 --2- 172.21.15.17:0/2038089415 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa8440b01a0 0x7fa8440aedd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7fa834000f40 tx=0x7fa834007b00).stop 2022-08-01T19:42:59.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:59.965+0000 7fa849257700 1 -- 172.21.15.17:0/2038089415 shutdown_connections 2022-08-01T19:42:59.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:59.965+0000 7fa849257700 1 --2- 172.21.15.17:0/2038089415 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa8240539e0 0x7fa824055ea0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:59.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:59.965+0000 7fa849257700 1 --2- 172.21.15.17:0/2038089415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8440b2710 0x7fa84415a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:59.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:59.965+0000 7fa849257700 1 --2- 172.21.15.17:0/2038089415 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa8440b01a0 0x7fa8440aedd0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:59.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:59.965+0000 7fa849257700 1 --2- 172.21.15.17:0/2038089415 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8440a8c50 0x7fa844144690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:42:59.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:59.965+0000 7fa849257700 1 -- 172.21.15.17:0/2038089415 >> 172.21.15.17:0/2038089415 conn(0x7fa844060310 msgr2=0x7fa8441460d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:42:59.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:59.965+0000 7fa849257700 1 -- 172.21.15.17:0/2038089415 shutdown_connections 2022-08-01T19:42:59.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:42:59.965+0000 7fa849257700 1 -- 172.21.15.17:0/2038089415 wait complete. 2022-08-01T19:42:59.973 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' created 2022-08-01T19:42:59.987 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-08-01T19:43:00.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.101+0000 7fb8da914700 1 Processor -- start 2022-08-01T19:43:00.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.101+0000 7fb8da914700 1 -- start start 2022-08-01T19:43:00.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.101+0000 7fb8da914700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb8d40b7840 0x7fb8d4065720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:00.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.101+0000 7fb8da914700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb8d40b3070 0x7fb8d4061120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:00.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.101+0000 7fb8da914700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fb8d414b040 0x7fb8d414b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:00.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.101+0000 7fb8da914700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8d4065c10 con 0x7fb8d40b7840 2022-08-01T19:43:00.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.101+0000 7fb8da914700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8d40b2c20 con 0x7fb8d414b040 2022-08-01T19:43:00.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.101+0000 7fb8da914700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d40b7c20 con 0x7fb8d40b3070 2022-08-01T19:43:00.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.101+0000 7fb8d9912700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb8d40b7840 0x7fb8d4065720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:00.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.101+0000 7fb8d9912700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb8d40b7840 0x7fb8d4065720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44164/0 (socket says 172.21.15.17:44164) 2022-08-01T19:43:00.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.101+0000 7fb8d9912700 1 -- 172.21.15.17:0/3690901879 learned_addr learned my addr 172.21.15.17:0/3690901879 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:00.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.101+0000 7fb8da113700 1 --2- 172.21.15.17:0/3690901879 >> v2:172.21.15.77:3300/0 conn(0x7fb8d414b040 0x7fb8d414b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:00.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.101+0000 7fb8d9111700 1 --2- 172.21.15.17:0/3690901879 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb8d40b3070 0x7fb8d4061120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:00.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.101+0000 7fb8d9912700 1 -- 172.21.15.17:0/3690901879 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb8d40b3070 msgr2=0x7fb8d4061120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:00.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.101+0000 7fb8d9912700 1 --2- 172.21.15.17:0/3690901879 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb8d40b3070 0x7fb8d4061120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:00.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.101+0000 7fb8d9912700 1 -- 172.21.15.17:0/3690901879 >> v2:172.21.15.77:3300/0 conn(0x7fb8d414b040 msgr2=0x7fb8d414b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:00.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.101+0000 7fb8d9912700 1 --2- 172.21.15.17:0/3690901879 >> v2:172.21.15.77:3300/0 conn(0x7fb8d414b040 0x7fb8d414b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:00.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.101+0000 7fb8d9912700 1 -- 172.21.15.17:0/3690901879 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d40b2980 con 0x7fb8d40b7840 2022-08-01T19:43:00.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.101+0000 7fb8da113700 1 --2- 172.21.15.17:0/3690901879 >> v2:172.21.15.77:3300/0 conn(0x7fb8d414b040 0x7fb8d414b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:43:00.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.101+0000 7fb8d9912700 1 --2- 172.21.15.17:0/3690901879 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8d40b7840 0x7fb8d4065720 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 rx=0x7fb8d0002f70 tx=0x7fb8d0006e50).ready entity=mon.0 client_cookie=f10fbcd93c33acc7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:00.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.101+0000 7fb8d8910700 1 -- 172.21.15.17:0/3690901879 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8d000d570 con 0x7fb8d40b7840 2022-08-01T19:43:00.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.101+0000 7fb8d8910700 1 -- 172.21.15.17:0/3690901879 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d0000e90 con 0x7fb8d40b7840 2022-08-01T19:43:00.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.101+0000 7fb8d8910700 1 -- 172.21.15.17:0/3690901879 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8d000d900 con 0x7fb8d40b7840 2022-08-01T19:43:00.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.105+0000 7fb8da914700 1 -- 172.21.15.17:0/3690901879 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8d40b7840 msgr2=0x7fb8d4065720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:00.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.105+0000 7fb8da914700 1 --2- 172.21.15.17:0/3690901879 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8d40b7840 0x7fb8d4065720 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 rx=0x7fb8d0002f70 tx=0x7fb8d0006e50).stop 2022-08-01T19:43:00.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.105+0000 7fb8da914700 1 -- 172.21.15.17:0/3690901879 shutdown_connections 2022-08-01T19:43:00.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.105+0000 7fb8da914700 1 --2- 172.21.15.17:0/3690901879 >> v2:172.21.15.77:3300/0 conn(0x7fb8d414b040 0x7fb8d414b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:00.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.105+0000 7fb8da914700 1 --2- 172.21.15.17:0/3690901879 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb8d40b3070 0x7fb8d4061120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:00.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.105+0000 7fb8da914700 1 --2- 172.21.15.17:0/3690901879 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8d40b7840 0x7fb8d4065720 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:00.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.105+0000 7fb8da914700 1 -- 172.21.15.17:0/3690901879 >> 172.21.15.17:0/3690901879 conn(0x7fb8d405f840 msgr2=0x7fb8d405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:00.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.105+0000 7fb8da914700 1 -- 172.21.15.17:0/3690901879 shutdown_connections 2022-08-01T19:43:00.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.105+0000 7fb8da914700 1 -- 172.21.15.17:0/3690901879 wait complete. 2022-08-01T19:43:00.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.105+0000 7fb8da914700 1 Processor -- start 2022-08-01T19:43:00.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.105+0000 7fb8da914700 1 -- start start 2022-08-01T19:43:00.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.109+0000 7fb8da914700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8d40b3070 0x7fb8d4156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:00.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.109+0000 7fb8da914700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb8d40b7840 0x7fb8d4145da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:00.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.109+0000 7fb8da914700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb8d414b040 0x7fb8d4144ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:00.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.109+0000 7fb8da914700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d4157710 con 0x7fb8d40b3070 2022-08-01T19:43:00.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.109+0000 7fb8da914700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d4146dc0 con 0x7fb8d40b7840 2022-08-01T19:43:00.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.109+0000 7fb8da914700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d4147060 con 0x7fb8d414b040 2022-08-01T19:43:00.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.109+0000 7fb8d9912700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8d40b3070 0x7fb8d4156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:00.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.109+0000 7fb8d9912700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8d40b3070 0x7fb8d4156730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44170/0 (socket says 172.21.15.17:44170) 2022-08-01T19:43:00.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.109+0000 7fb8d9912700 1 -- 172.21.15.17:0/4210867329 learned_addr learned my addr 172.21.15.17:0/4210867329 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:00.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.109+0000 7fb8d9111700 1 --2- 172.21.15.17:0/4210867329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb8d40b7840 0x7fb8d4145da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:00.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.109+0000 7fb8da113700 1 --2- 172.21.15.17:0/4210867329 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb8d414b040 0x7fb8d4144ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:00.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.109+0000 7fb8d9912700 1 -- 172.21.15.17:0/4210867329 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb8d414b040 msgr2=0x7fb8d4144ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:00.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.109+0000 7fb8d9912700 1 --2- 172.21.15.17:0/4210867329 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb8d414b040 0x7fb8d4144ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:00.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.109+0000 7fb8d9912700 1 -- 172.21.15.17:0/4210867329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb8d40b7840 msgr2=0x7fb8d4145da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:00.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.109+0000 7fb8d9912700 1 --2- 172.21.15.17:0/4210867329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb8d40b7840 0x7fb8d4145da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:00.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.109+0000 7fb8d9912700 1 -- 172.21.15.17:0/4210867329 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d000b1e0 con 0x7fb8d40b3070 2022-08-01T19:43:00.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.109+0000 7fb8d9111700 1 --2- 172.21.15.17:0/4210867329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb8d40b7840 0x7fb8d4145da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:43:00.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.109+0000 7fb8d9912700 1 --2- 172.21.15.17:0/4210867329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8d40b3070 0x7fb8d4156730 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 rx=0x7fb8d0002f70 tx=0x7fb8d0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:00.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.109+0000 7fb8c2ffd700 1 -- 172.21.15.17:0/4210867329 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8d0011990 con 0x7fb8d40b3070 2022-08-01T19:43:00.134 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.109+0000 7fb8c2ffd700 1 -- 172.21.15.17:0/4210867329 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d0011af0 con 0x7fb8d40b3070 2022-08-01T19:43:00.134 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.109+0000 7fb8da914700 1 -- 172.21.15.17:0/4210867329 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8d4147300 con 0x7fb8d40b3070 2022-08-01T19:43:00.135 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.109+0000 7fb8c2ffd700 1 -- 172.21.15.17:0/4210867329 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8d0011e50 con 0x7fb8d40b3070 2022-08-01T19:43:00.135 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.109+0000 7fb8da914700 1 -- 172.21.15.17:0/4210867329 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8d414a630 con 0x7fb8d40b3070 2022-08-01T19:43:00.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.109+0000 7fb8da914700 1 -- 172.21.15.17:0/4210867329 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8d40609e0 con 0x7fb8d40b3070 2022-08-01T19:43:00.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.113+0000 7fb8c2ffd700 1 -- 172.21.15.17:0/4210867329 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb8d001a040 con 0x7fb8d40b3070 2022-08-01T19:43:00.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.113+0000 7fb8c2ffd700 1 --2- 172.21.15.17:0/4210867329 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb8b4053a10 0x7fb8b4055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:00.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.113+0000 7fb8c2ffd700 1 -- 172.21.15.17:0/4210867329 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5954+0+0 (secure 0 0 0) 0x7fb8d006d3b0 con 0x7fb8d40b3070 2022-08-01T19:43:00.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.113+0000 7fb8d9111700 1 --2- 172.21.15.17:0/4210867329 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb8b4053a10 0x7fb8b4055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:00.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.113+0000 7fb8d9111700 1 --2- 172.21.15.17:0/4210867329 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb8b4053a10 0x7fb8b4055ed0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7fb8c400c000 tx=0x7fb8c4008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:00.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.117+0000 7fb8c2ffd700 1 -- 172.21.15.17:0/4210867329 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb8d0044230 con 0x7fb8d40b3070 2022-08-01T19:43:00.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.245+0000 7fb8da914700 1 -- 172.21.15.17:0/4210867329 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fb8d4147490 con 0x7fb8d40b3070 2022-08-01T19:43:00.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.961+0000 7fb8c2ffd700 1 -- 172.21.15.17:0/4210867329 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v135) v1 ==== 125+0+0 (secure 0 0 0) 0x7fb8d0023020 con 0x7fb8d40b3070 2022-08-01T19:43:00.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.969+0000 7fb8da914700 1 -- 172.21.15.17:0/4210867329 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb8b4053a10 msgr2=0x7fb8b4055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:00.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.969+0000 7fb8da914700 1 --2- 172.21.15.17:0/4210867329 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb8b4053a10 0x7fb8b4055ed0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7fb8c400c000 tx=0x7fb8c4008040).stop 2022-08-01T19:43:00.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.969+0000 7fb8da914700 1 -- 172.21.15.17:0/4210867329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8d40b3070 msgr2=0x7fb8d4156730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:00.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.969+0000 7fb8da914700 1 --2- 172.21.15.17:0/4210867329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8d40b3070 0x7fb8d4156730 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 rx=0x7fb8d0002f70 tx=0x7fb8d0015000).stop 2022-08-01T19:43:00.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.973+0000 7fb8da914700 1 -- 172.21.15.17:0/4210867329 shutdown_connections 2022-08-01T19:43:00.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.973+0000 7fb8da914700 1 --2- 172.21.15.17:0/4210867329 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb8b4053a10 0x7fb8b4055ed0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:00.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.973+0000 7fb8da914700 1 --2- 172.21.15.17:0/4210867329 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb8d414b040 0x7fb8d4144ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:00.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.973+0000 7fb8da914700 1 --2- 172.21.15.17:0/4210867329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb8d40b7840 0x7fb8d4145da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:00.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.973+0000 7fb8da914700 1 --2- 172.21.15.17:0/4210867329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8d40b3070 0x7fb8d4156730 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:00.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.973+0000 7fb8da914700 1 -- 172.21.15.17:0/4210867329 >> 172.21.15.17:0/4210867329 conn(0x7fb8d405f840 msgr2=0x7fb8d405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:00.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.973+0000 7fb8da914700 1 -- 172.21.15.17:0/4210867329 shutdown_connections 2022-08-01T19:43:00.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:00.973+0000 7fb8da914700 1 -- 172.21.15.17:0/4210867329 wait complete. 2022-08-01T19:43:00.985 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' created 2022-08-01T19:43:00.989 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-08-01T19:43:01.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.097+0000 7fc123fff700 1 Processor -- start 2022-08-01T19:43:01.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.097+0000 7fc123fff700 1 -- start start 2022-08-01T19:43:01.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.097+0000 7fc123fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc1240b31d0 0x7fc124066020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:01.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.097+0000 7fc123fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc1241402d0 0x7fc124140780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:01.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.097+0000 7fc123fff700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc124067030 con 0x7fc1240a9da0 2022-08-01T19:43:01.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.097+0000 7fc123fff700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc124064a70 con 0x7fc1241402d0 2022-08-01T19:43:01.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.097+0000 7fc123fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc124061f60 con 0x7fc1240b31d0 2022-08-01T19:43:01.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.101+0000 7fc1237fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc1241402d0 0x7fc124140780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:01.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.101+0000 7fc11a7fc700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc1240a9da0 0x7fc124060950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36374/0 (socket says 172.21.15.17:36374) 2022-08-01T19:43:01.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.101+0000 7fc11a7fc700 1 -- 172.21.15.17:0/977124252 learned_addr learned my addr 172.21.15.17:0/977124252 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:01.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.101+0000 7fc122ffd700 1 --2- 172.21.15.17:0/977124252 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc1240b31d0 0x7fc124066020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:01.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.101+0000 7fc1227fc700 1 -- 172.21.15.17:0/977124252 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1578753236 0 0) 0x7fc124067030 con 0x7fc1240a9da0 2022-08-01T19:43:01.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.101+0000 7fc1227fc700 1 -- 172.21.15.17:0/977124252 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc104003260 con 0x7fc1240a9da0 2022-08-01T19:43:01.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.101+0000 7fc122ffd700 1 -- 172.21.15.17:0/977124252 >> v2:172.21.15.17:3300/0 conn(0x7fc1241402d0 msgr2=0x7fc124140780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:01.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.101+0000 7fc122ffd700 1 --2- 172.21.15.17:0/977124252 >> v2:172.21.15.17:3300/0 conn(0x7fc1241402d0 0x7fc124140780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:01.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.101+0000 7fc122ffd700 1 -- 172.21.15.17:0/977124252 >> v1:172.21.15.17:6789/0 conn(0x7fc1240a9da0 legacy=0x7fc124060950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:01.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.101+0000 7fc122ffd700 1 -- 172.21.15.17:0/977124252 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1240b35b0 con 0x7fc1240b31d0 2022-08-01T19:43:01.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.101+0000 7fc1237fe700 1 --2- 172.21.15.17:0/977124252 >> v2:172.21.15.17:3300/0 conn(0x7fc1241402d0 0x7fc124140780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:43:01.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.101+0000 7fc122ffd700 1 --2- 172.21.15.17:0/977124252 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc1240b31d0 0x7fc124066020 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fc114002f70 tx=0x7fc114006df0).ready entity=mon.2 client_cookie=3e0c75d54fe71329 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:01.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.101+0000 7fc1227fc700 1 -- 172.21.15.17:0/977124252 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc11400d4a0 con 0x7fc1240b31d0 2022-08-01T19:43:01.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.101+0000 7fc1227fc700 1 -- 172.21.15.17:0/977124252 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc114000e90 con 0x7fc1240b31d0 2022-08-01T19:43:01.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.101+0000 7fc1227fc700 1 -- 172.21.15.17:0/977124252 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc11400d7f0 con 0x7fc1240b31d0 2022-08-01T19:43:01.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.101+0000 7fc123fff700 1 -- 172.21.15.17:0/977124252 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc1240b31d0 msgr2=0x7fc124066020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:01.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.101+0000 7fc123fff700 1 --2- 172.21.15.17:0/977124252 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc1240b31d0 0x7fc124066020 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fc114002f70 tx=0x7fc114006df0).stop 2022-08-01T19:43:01.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.101+0000 7fc123fff700 1 -- 172.21.15.17:0/977124252 shutdown_connections 2022-08-01T19:43:01.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.101+0000 7fc123fff700 1 --2- 172.21.15.17:0/977124252 >> v2:172.21.15.17:3300/0 conn(0x7fc1241402d0 0x7fc124140780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:01.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.101+0000 7fc123fff700 1 --2- 172.21.15.17:0/977124252 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc1240b31d0 0x7fc124066020 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:01.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.101+0000 7fc123fff700 1 -- 172.21.15.17:0/977124252 >> 172.21.15.17:0/977124252 conn(0x7fc124063870 msgr2=0x7fc124063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:01.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.101+0000 7fc123fff700 1 -- 172.21.15.17:0/977124252 shutdown_connections 2022-08-01T19:43:01.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.101+0000 7fc123fff700 1 -- 172.21.15.17:0/977124252 wait complete. 2022-08-01T19:43:01.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.105+0000 7fc123fff700 1 Processor -- start 2022-08-01T19:43:01.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.105+0000 7fc123fff700 1 -- start start 2022-08-01T19:43:01.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.105+0000 7fc123fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc1240a9da0 0x7fc124142f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:01.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.105+0000 7fc123fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1240b31d0 0x7fc1240ac960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:01.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.105+0000 7fc123fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1241402d0 0x7fc1240abc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:01.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.105+0000 7fc123fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1240aaeb0 con 0x7fc1241402d0 2022-08-01T19:43:01.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.105+0000 7fc123fff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1240ab1a0 con 0x7fc1240b31d0 2022-08-01T19:43:01.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.105+0000 7fc123fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1240ace50 con 0x7fc1240a9da0 2022-08-01T19:43:01.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.105+0000 7fc1237fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1241402d0 0x7fc1240abc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:01.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.105+0000 7fc1237fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1241402d0 0x7fc1240abc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44188/0 (socket says 172.21.15.17:44188) 2022-08-01T19:43:01.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.105+0000 7fc1237fe700 1 -- 172.21.15.17:0/1601658164 learned_addr learned my addr 172.21.15.17:0/1601658164 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:01.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.105+0000 7fc11a7fc700 1 --2- 172.21.15.17:0/1601658164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1240b31d0 0x7fc1240ac960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:01.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.105+0000 7fc122ffd700 1 --2- 172.21.15.17:0/1601658164 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc1240a9da0 0x7fc124142f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:01.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.105+0000 7fc1237fe700 1 -- 172.21.15.17:0/1601658164 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc1240a9da0 msgr2=0x7fc124142f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:01.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.105+0000 7fc1237fe700 1 --2- 172.21.15.17:0/1601658164 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc1240a9da0 0x7fc124142f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:01.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.105+0000 7fc1237fe700 1 -- 172.21.15.17:0/1601658164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1240b31d0 msgr2=0x7fc1240ac960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:01.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.105+0000 7fc1237fe700 1 --2- 172.21.15.17:0/1601658164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1240b31d0 0x7fc1240ac960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:01.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.105+0000 7fc1237fe700 1 -- 172.21.15.17:0/1601658164 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11c009470 con 0x7fc1241402d0 2022-08-01T19:43:01.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.105+0000 7fc1237fe700 1 --2- 172.21.15.17:0/1601658164 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1241402d0 0x7fc1240abc80 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 rx=0x7fc11c00c000 tx=0x7fc11c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:01.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.105+0000 7fc120ff9700 1 -- 172.21.15.17:0/1601658164 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc11c00ee60 con 0x7fc1241402d0 2022-08-01T19:43:01.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.105+0000 7fc120ff9700 1 -- 172.21.15.17:0/1601658164 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11c00c850 con 0x7fc1241402d0 2022-08-01T19:43:01.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.105+0000 7fc123fff700 1 -- 172.21.15.17:0/1601658164 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11400b180 con 0x7fc1241402d0 2022-08-01T19:43:01.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.105+0000 7fc120ff9700 1 -- 172.21.15.17:0/1601658164 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc11c00cbb0 con 0x7fc1241402d0 2022-08-01T19:43:01.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.109+0000 7fc123fff700 1 -- 172.21.15.17:0/1601658164 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc12413c4a0 con 0x7fc1241402d0 2022-08-01T19:43:01.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.109+0000 7fc123fff700 1 -- 172.21.15.17:0/1601658164 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc12413bd90 con 0x7fc1241402d0 2022-08-01T19:43:01.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.109+0000 7fc120ff9700 1 -- 172.21.15.17:0/1601658164 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc11c022070 con 0x7fc1241402d0 2022-08-01T19:43:01.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.109+0000 7fc120ff9700 1 --2- 172.21.15.17:0/1601658164 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc108053950 0x7fc108055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:01.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.109+0000 7fc120ff9700 1 -- 172.21.15.17:0/1601658164 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fc11c06c670 con 0x7fc1241402d0 2022-08-01T19:43:01.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.109+0000 7fc122ffd700 1 --2- 172.21.15.17:0/1601658164 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc108053950 0x7fc108055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:01.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.113+0000 7fc122ffd700 1 --2- 172.21.15.17:0/1601658164 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc108053950 0x7fc108055e10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7fc11401a000 tx=0x7fc114014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:01.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.117+0000 7fc120ff9700 1 -- 172.21.15.17:0/1601658164 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc11c0434b0 con 0x7fc1241402d0 2022-08-01T19:43:01.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.281+0000 7fc123fff700 1 -- 172.21.15.17:0/1601658164 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fc124066af0 con 0x7fc1241402d0 2022-08-01T19:43:01.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.293+0000 7fc120ff9700 1 -- 172.21.15.17:0/1601658164 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 25 and data pool 26 v68) v1 ==== 189+0+0 (secure 0 0 0) 0x7fc11c046ad0 con 0x7fc1241402d0 2022-08-01T19:43:01.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.297+0000 7fc123fff700 1 -- 172.21.15.17:0/1601658164 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc108053950 msgr2=0x7fc108055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:01.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.297+0000 7fc123fff700 1 --2- 172.21.15.17:0/1601658164 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc108053950 0x7fc108055e10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7fc11401a000 tx=0x7fc114014000).stop 2022-08-01T19:43:01.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.297+0000 7fc123fff700 1 -- 172.21.15.17:0/1601658164 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1241402d0 msgr2=0x7fc1240abc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:01.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.297+0000 7fc123fff700 1 --2- 172.21.15.17:0/1601658164 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1241402d0 0x7fc1240abc80 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 rx=0x7fc11c00c000 tx=0x7fc11c0091d0).stop 2022-08-01T19:43:01.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.301+0000 7fc123fff700 1 -- 172.21.15.17:0/1601658164 shutdown_connections 2022-08-01T19:43:01.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.301+0000 7fc123fff700 1 --2- 172.21.15.17:0/1601658164 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc108053950 0x7fc108055e10 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:01.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.301+0000 7fc123fff700 1 --2- 172.21.15.17:0/1601658164 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1241402d0 0x7fc1240abc80 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:01.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.301+0000 7fc123fff700 1 --2- 172.21.15.17:0/1601658164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1240b31d0 0x7fc1240ac960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:01.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.301+0000 7fc123fff700 1 --2- 172.21.15.17:0/1601658164 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc1240a9da0 0x7fc124142f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:01.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.301+0000 7fc123fff700 1 -- 172.21.15.17:0/1601658164 >> 172.21.15.17:0/1601658164 conn(0x7fc124063870 msgr2=0x7fc1240b4290 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:01.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.301+0000 7fc123fff700 1 -- 172.21.15.17:0/1601658164 shutdown_connections 2022-08-01T19:43:01.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.301+0000 7fc123fff700 1 -- 172.21.15.17:0/1601658164 wait complete. 2022-08-01T19:43:01.312 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 25 and data pool 26 2022-08-01T19:43:01.321 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:43:01.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.421+0000 7f9be6982700 1 Processor -- start 2022-08-01T19:43:01.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.421+0000 7f9be6982700 1 -- start start 2022-08-01T19:43:01.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.425+0000 7f9be6982700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9be00b2730 0x7f9be00666b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:01.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.425+0000 7f9be6982700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f9be0066ba0 0x7f9be014dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:01.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.425+0000 7f9be6982700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9be00654c0 con 0x7f9be00b3190 2022-08-01T19:43:01.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.425+0000 7f9be6982700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9be0062190 con 0x7f9be0066ba0 2022-08-01T19:43:01.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.425+0000 7f9be6982700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be00b3570 con 0x7f9be00b2730 2022-08-01T19:43:01.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.425+0000 7f9be5980700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9be00b3190 0x7f9be0060ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36390/0 (socket says 172.21.15.17:36390) 2022-08-01T19:43:01.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.425+0000 7f9be5980700 1 -- 172.21.15.17:0/1941929475 learned_addr learned my addr 172.21.15.17:0/1941929475 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:01.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.425+0000 7f9be517f700 1 --2- 172.21.15.17:0/1941929475 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9be00b2730 0x7f9be00666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:01.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.425+0000 7f9be6181700 1 --2- 172.21.15.17:0/1941929475 >> v2:172.21.15.77:3300/0 conn(0x7f9be0066ba0 0x7f9be014dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:01.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.425+0000 7f9be497e700 1 -- 172.21.15.17:0/1941929475 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2886506002 0 0) 0x7f9be00654c0 con 0x7f9be00b3190 2022-08-01T19:43:01.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.425+0000 7f9be497e700 1 -- 172.21.15.17:0/1941929475 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bc8003260 con 0x7f9be00b3190 2022-08-01T19:43:01.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.425+0000 7f9be517f700 1 -- 172.21.15.17:0/1941929475 >> v2:172.21.15.77:3300/0 conn(0x7f9be0066ba0 msgr2=0x7f9be014dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:01.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.425+0000 7f9be517f700 1 --2- 172.21.15.17:0/1941929475 >> v2:172.21.15.77:3300/0 conn(0x7f9be0066ba0 0x7f9be014dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:01.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.425+0000 7f9be517f700 1 -- 172.21.15.17:0/1941929475 >> v1:172.21.15.17:6789/0 conn(0x7f9be00b3190 legacy=0x7f9be0060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:01.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.425+0000 7f9be517f700 1 -- 172.21.15.17:0/1941929475 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be005fbf0 con 0x7f9be00b2730 2022-08-01T19:43:01.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.425+0000 7f9be517f700 1 --2- 172.21.15.17:0/1941929475 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9be00b2730 0x7f9be00666b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f9bd0002f70 tx=0x7f9bd0006df0).ready entity=mon.2 client_cookie=41b0bff86f57c38 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:01.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.425+0000 7f9be497e700 1 -- 172.21.15.17:0/1941929475 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9bd000d4a0 con 0x7f9be00b2730 2022-08-01T19:43:01.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.425+0000 7f9be497e700 1 -- 172.21.15.17:0/1941929475 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd0000e90 con 0x7f9be00b2730 2022-08-01T19:43:01.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.425+0000 7f9be497e700 1 -- 172.21.15.17:0/1941929475 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9bd000d7f0 con 0x7f9be00b2730 2022-08-01T19:43:01.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.425+0000 7f9be6982700 1 -- 172.21.15.17:0/1941929475 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9be00b2730 msgr2=0x7f9be00666b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:01.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.425+0000 7f9be6982700 1 --2- 172.21.15.17:0/1941929475 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9be00b2730 0x7f9be00666b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f9bd0002f70 tx=0x7f9bd0006df0).stop 2022-08-01T19:43:01.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.425+0000 7f9be6982700 1 -- 172.21.15.17:0/1941929475 shutdown_connections 2022-08-01T19:43:01.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.425+0000 7f9be6982700 1 --2- 172.21.15.17:0/1941929475 >> v2:172.21.15.77:3300/0 conn(0x7f9be0066ba0 0x7f9be014dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:01.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.425+0000 7f9be6982700 1 --2- 172.21.15.17:0/1941929475 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9be00b2730 0x7f9be00666b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:01.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.425+0000 7f9be6982700 1 -- 172.21.15.17:0/1941929475 >> 172.21.15.17:0/1941929475 conn(0x7f9be0063870 msgr2=0x7f9be0063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:01.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.425+0000 7f9be6982700 1 -- 172.21.15.17:0/1941929475 shutdown_connections 2022-08-01T19:43:01.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.425+0000 7f9be6982700 1 -- 172.21.15.17:0/1941929475 wait complete. 2022-08-01T19:43:01.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.429+0000 7f9be6982700 1 Processor -- start 2022-08-01T19:43:01.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.429+0000 7f9be6982700 1 -- start start 2022-08-01T19:43:01.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.429+0000 7f9be6982700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9be0066ba0 0x7f9be0149170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:01.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.429+0000 7f9be6982700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9be00b2730 0x7f9be00aa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:01.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.429+0000 7f9be6982700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9be00b3190 0x7f9be00aace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:01.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.429+0000 7f9be6982700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be00ac9a0 con 0x7f9be00b2730 2022-08-01T19:43:01.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.429+0000 7f9be6982700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be00acc40 con 0x7f9be00b3190 2022-08-01T19:43:01.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.429+0000 7f9be6982700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be00acee0 con 0x7f9be0066ba0 2022-08-01T19:43:01.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.429+0000 7f9be517f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9be00b2730 0x7f9be00aa7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:01.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.429+0000 7f9be517f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9be00b2730 0x7f9be00aa7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44200/0 (socket says 172.21.15.17:44200) 2022-08-01T19:43:01.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.429+0000 7f9be517f700 1 -- 172.21.15.17:0/222854673 learned_addr learned my addr 172.21.15.17:0/222854673 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:01.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.429+0000 7f9be6181700 1 --2- 172.21.15.17:0/222854673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9be00b3190 0x7f9be00aace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:01.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.429+0000 7f9be5980700 1 --2- 172.21.15.17:0/222854673 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9be0066ba0 0x7f9be0149170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:01.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.429+0000 7f9be517f700 1 -- 172.21.15.17:0/222854673 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9be0066ba0 msgr2=0x7f9be0149170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:01.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.429+0000 7f9be517f700 1 --2- 172.21.15.17:0/222854673 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9be0066ba0 0x7f9be0149170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:01.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.429+0000 7f9be517f700 1 -- 172.21.15.17:0/222854673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9be00b3190 msgr2=0x7f9be00aace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:01.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.429+0000 7f9be517f700 1 --2- 172.21.15.17:0/222854673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9be00b3190 0x7f9be00aace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:01.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.429+0000 7f9be517f700 1 -- 172.21.15.17:0/222854673 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd000b180 con 0x7f9be00b2730 2022-08-01T19:43:01.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.429+0000 7f9be517f700 1 --2- 172.21.15.17:0/222854673 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9be00b2730 0x7f9be00aa7f0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 rx=0x7f9bd0002f70 tx=0x7f9bd0014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:01.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.429+0000 7f9bceffd700 1 -- 172.21.15.17:0/222854673 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9bd0016070 con 0x7f9be00b2730 2022-08-01T19:43:01.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.429+0000 7f9bceffd700 1 -- 172.21.15.17:0/222854673 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd00137d0 con 0x7f9be00b2730 2022-08-01T19:43:01.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.429+0000 7f9be6982700 1 -- 172.21.15.17:0/222854673 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9be00aa350 con 0x7f9be00b2730 2022-08-01T19:43:01.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.429+0000 7f9bceffd700 1 -- 172.21.15.17:0/222854673 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9bd0013b30 con 0x7f9be00b2730 2022-08-01T19:43:01.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.429+0000 7f9be6982700 1 -- 172.21.15.17:0/222854673 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be0155740 con 0x7f9be00b2730 2022-08-01T19:43:01.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.433+0000 7f9be6982700 1 -- 172.21.15.17:0/222854673 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9be00aa4e0 con 0x7f9be00b2730 2022-08-01T19:43:01.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.433+0000 7f9bceffd700 1 -- 172.21.15.17:0/222854673 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9bd000f030 con 0x7f9be00b2730 2022-08-01T19:43:01.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.433+0000 7f9bceffd700 1 --2- 172.21.15.17:0/222854673 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9bc0053b00 0x7f9bc0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:01.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.433+0000 7f9bceffd700 1 -- 172.21.15.17:0/222854673 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9bd006ccc0 con 0x7f9be00b2730 2022-08-01T19:43:01.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.433+0000 7f9be5980700 1 --2- 172.21.15.17:0/222854673 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9bc0053b00 0x7f9bc0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:01.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.433+0000 7f9be5980700 1 --2- 172.21.15.17:0/222854673 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9bc0053b00 0x7f9bc0055fc0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f9bdc00d000 tx=0x7f9bdc009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:01.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.441+0000 7f9bceffd700 1 -- 172.21.15.17:0/222854673 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9bd00439e0 con 0x7f9be00b2730 2022-08-01T19:43:01.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.569+0000 7f9be6982700 1 -- 172.21.15.17:0/222854673 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9be005fe90 con 0x7f9be00b2730 2022-08-01T19:43:01.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.569+0000 7f9bceffd700 1 -- 172.21.15.17:0/222854673 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f9bd0023070 con 0x7f9be00b2730 2022-08-01T19:43:01.573 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:01.574 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":136,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:43:01.287116+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:42:59.166414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2022-08-01T19:43:00.301184+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:43:01.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.573+0000 7f9be6982700 1 -- 172.21.15.17:0/222854673 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9bc0053b00 msgr2=0x7f9bc0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:01.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.573+0000 7f9be6982700 1 --2- 172.21.15.17:0/222854673 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9bc0053b00 0x7f9bc0055fc0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f9bdc00d000 tx=0x7f9bdc009040).stop 2022-08-01T19:43:01.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.573+0000 7f9be6982700 1 -- 172.21.15.17:0/222854673 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9be00b2730 msgr2=0x7f9be00aa7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:01.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.573+0000 7f9be6982700 1 --2- 172.21.15.17:0/222854673 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9be00b2730 0x7f9be00aa7f0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 rx=0x7f9bd0002f70 tx=0x7f9bd0014000).stop 2022-08-01T19:43:01.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.573+0000 7f9be6982700 1 -- 172.21.15.17:0/222854673 shutdown_connections 2022-08-01T19:43:01.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.573+0000 7f9be6982700 1 --2- 172.21.15.17:0/222854673 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9bc0053b00 0x7f9bc0055fc0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:01.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.573+0000 7f9be6982700 1 --2- 172.21.15.17:0/222854673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9be00b3190 0x7f9be00aace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:01.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.573+0000 7f9be6982700 1 --2- 172.21.15.17:0/222854673 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9be00b2730 0x7f9be00aa7f0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:01.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.573+0000 7f9be6982700 1 --2- 172.21.15.17:0/222854673 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9be0066ba0 0x7f9be0149170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:01.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.573+0000 7f9be6982700 1 -- 172.21.15.17:0/222854673 >> 172.21.15.17:0/222854673 conn(0x7f9be0063870 msgr2=0x7f9be0064930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:01.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.573+0000 7f9be6982700 1 -- 172.21.15.17:0/222854673 shutdown_connections 2022-08-01T19:43:01.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.573+0000 7f9be6982700 1 -- 172.21.15.17:0/222854673 wait complete. 2022-08-01T19:43:01.586 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:43:01.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.685+0000 7fe77f59e700 1 Processor -- start 2022-08-01T19:43:01.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.685+0000 7fe77f59e700 1 -- start start 2022-08-01T19:43:01.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.685+0000 7fe77f59e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe7800b01a0 0x7fe7800a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:01.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.685+0000 7fe77f59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe7800afd60 0x7fe7800a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:01.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.685+0000 7fe77f59e700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7800a7250 con 0x7fe7800b2710 2022-08-01T19:43:01.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.685+0000 7fe77f59e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7800a7480 con 0x7fe7800b01a0 2022-08-01T19:43:01.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.685+0000 7fe77f59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7800a7720 con 0x7fe7800afd60 2022-08-01T19:43:01.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.685+0000 7fe77dd9b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe7800b01a0 0x7fe7800a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:01.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.685+0000 7fe77e59c700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe7800b2710 0x7fe78014afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36402/0 (socket says 172.21.15.17:36402) 2022-08-01T19:43:01.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.685+0000 7fe77e59c700 1 -- 172.21.15.17:0/1411279330 learned_addr learned my addr 172.21.15.17:0/1411279330 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:01.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.685+0000 7fe77ed9d700 1 --2- 172.21.15.17:0/1411279330 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe7800afd60 0x7fe7800a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:01.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.685+0000 7fe77d59a700 1 -- 172.21.15.17:0/1411279330 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3445715533 0 0) 0x7fe7800a7250 con 0x7fe7800b2710 2022-08-01T19:43:01.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.685+0000 7fe77dd9b700 1 -- 172.21.15.17:0/1411279330 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe7800afd60 msgr2=0x7fe7800a8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:01.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.685+0000 7fe77dd9b700 1 --2- 172.21.15.17:0/1411279330 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe7800afd60 0x7fe7800a8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:01.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.685+0000 7fe77dd9b700 1 -- 172.21.15.17:0/1411279330 >> v1:172.21.15.17:6789/0 conn(0x7fe7800b2710 legacy=0x7fe78014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:01.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.689+0000 7fe77dd9b700 1 -- 172.21.15.17:0/1411279330 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7800a79c0 con 0x7fe7800b01a0 2022-08-01T19:43:01.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.689+0000 7fe77dd9b700 1 --2- 172.21.15.17:0/1411279330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7800b01a0 0x7fe7800a8540 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 rx=0x7fe770002f70 tx=0x7fe770006df0).ready entity=mon.0 client_cookie=210e2fabd080fae server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:01.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.689+0000 7fe77d59a700 1 -- 172.21.15.17:0/1411279330 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe77000d570 con 0x7fe7800b01a0 2022-08-01T19:43:01.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.689+0000 7fe77d59a700 1 -- 172.21.15.17:0/1411279330 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe770000e90 con 0x7fe7800b01a0 2022-08-01T19:43:01.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.689+0000 7fe77d59a700 1 -- 172.21.15.17:0/1411279330 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe77000d8c0 con 0x7fe7800b01a0 2022-08-01T19:43:01.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.689+0000 7fe77f59e700 1 -- 172.21.15.17:0/1411279330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7800b01a0 msgr2=0x7fe7800a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:01.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.689+0000 7fe77f59e700 1 --2- 172.21.15.17:0/1411279330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7800b01a0 0x7fe7800a8540 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 rx=0x7fe770002f70 tx=0x7fe770006df0).stop 2022-08-01T19:43:01.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.689+0000 7fe77f59e700 1 -- 172.21.15.17:0/1411279330 shutdown_connections 2022-08-01T19:43:01.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.689+0000 7fe77f59e700 1 --2- 172.21.15.17:0/1411279330 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe7800afd60 0x7fe7800a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:01.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.689+0000 7fe77f59e700 1 --2- 172.21.15.17:0/1411279330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7800b01a0 0x7fe7800a8540 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:01.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.689+0000 7fe77f59e700 1 -- 172.21.15.17:0/1411279330 >> 172.21.15.17:0/1411279330 conn(0x7fe780060310 msgr2=0x7fe7800b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:01.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.689+0000 7fe77f59e700 1 -- 172.21.15.17:0/1411279330 shutdown_connections 2022-08-01T19:43:01.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.689+0000 7fe77f59e700 1 -- 172.21.15.17:0/1411279330 wait complete. 2022-08-01T19:43:01.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.689+0000 7fe77f59e700 1 Processor -- start 2022-08-01T19:43:01.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.689+0000 7fe77f59e700 1 -- start start 2022-08-01T19:43:01.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.693+0000 7fe77f59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe7800afd60 0x7fe78014d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:01.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.693+0000 7fe77f59e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7800b01a0 0x7fe7800aaa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:01.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.693+0000 7fe77f59e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7800b2710 0x7fe780159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:01.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.693+0000 7fe77f59e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7800a9a00 con 0x7fe7800b01a0 2022-08-01T19:43:01.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.693+0000 7fe77f59e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7800ac070 con 0x7fe7800b2710 2022-08-01T19:43:01.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.693+0000 7fe77f59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7800abad0 con 0x7fe7800afd60 2022-08-01T19:43:01.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.693+0000 7fe77dd9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7800b01a0 0x7fe7800aaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:01.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.693+0000 7fe77dd9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7800b01a0 0x7fe7800aaa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44216/0 (socket says 172.21.15.17:44216) 2022-08-01T19:43:01.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.693+0000 7fe77dd9b700 1 -- 172.21.15.17:0/3259366486 learned_addr learned my addr 172.21.15.17:0/3259366486 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:01.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.693+0000 7fe77e59c700 1 --2- 172.21.15.17:0/3259366486 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe7800afd60 0x7fe78014d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:01.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.693+0000 7fe77ed9d700 1 --2- 172.21.15.17:0/3259366486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7800b2710 0x7fe780159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:01.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.693+0000 7fe77dd9b700 1 -- 172.21.15.17:0/3259366486 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe7800afd60 msgr2=0x7fe78014d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:01.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.693+0000 7fe77dd9b700 1 --2- 172.21.15.17:0/3259366486 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe7800afd60 0x7fe78014d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:01.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.693+0000 7fe77dd9b700 1 -- 172.21.15.17:0/3259366486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7800b2710 msgr2=0x7fe780159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:01.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.693+0000 7fe77dd9b700 1 --2- 172.21.15.17:0/3259366486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7800b2710 0x7fe780159b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:01.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.693+0000 7fe77dd9b700 1 -- 172.21.15.17:0/3259366486 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe77000b180 con 0x7fe7800b01a0 2022-08-01T19:43:01.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.693+0000 7fe77dd9b700 1 --2- 172.21.15.17:0/3259366486 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7800b01a0 0x7fe7800aaa40 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 rx=0x7fe770002f70 tx=0x7fe770014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:01.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.693+0000 7fe7777fe700 1 -- 172.21.15.17:0/3259366486 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe770016070 con 0x7fe7800b01a0 2022-08-01T19:43:01.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.693+0000 7fe7777fe700 1 -- 172.21.15.17:0/3259366486 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7700138d0 con 0x7fe7800b01a0 2022-08-01T19:43:01.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.693+0000 7fe77f59e700 1 -- 172.21.15.17:0/3259366486 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7800ab810 con 0x7fe7800b01a0 2022-08-01T19:43:01.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.693+0000 7fe7777fe700 1 -- 172.21.15.17:0/3259366486 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe770013c30 con 0x7fe7800b01a0 2022-08-01T19:43:01.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.693+0000 7fe77f59e700 1 -- 172.21.15.17:0/3259366486 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7800a93e0 con 0x7fe7800b01a0 2022-08-01T19:43:01.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.693+0000 7fe77f59e700 1 -- 172.21.15.17:0/3259366486 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7800574e0 con 0x7fe7800b01a0 2022-08-01T19:43:01.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.697+0000 7fe7777fe700 1 -- 172.21.15.17:0/3259366486 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe77000f030 con 0x7fe7800b01a0 2022-08-01T19:43:01.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.697+0000 7fe7777fe700 1 --2- 172.21.15.17:0/3259366486 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe758053a70 0x7fe758055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:01.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.697+0000 7fe7777fe700 1 -- 172.21.15.17:0/3259366486 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe77006ce50 con 0x7fe7800b01a0 2022-08-01T19:43:01.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.697+0000 7fe77e59c700 1 --2- 172.21.15.17:0/3259366486 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe758053a70 0x7fe758055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:01.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.697+0000 7fe77e59c700 1 --2- 172.21.15.17:0/3259366486 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe758053a70 0x7fe758055f30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7fe76800d000 tx=0x7fe768007c20).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:01.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.701+0000 7fe7777fe700 1 -- 172.21.15.17:0/3259366486 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe770043a80 con 0x7fe7800b01a0 2022-08-01T19:43:01.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.825+0000 7fe77f59e700 1 -- 172.21.15.17:0/3259366486 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe780069a70 con 0x7fe7800b01a0 2022-08-01T19:43:01.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.825+0000 7fe7777fe700 1 -- 172.21.15.17:0/3259366486 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+16298 (secure 0 0 0) 0x7fe770023090 con 0x7fe7800b01a0 2022-08-01T19:43:01.828 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:01.828 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":136,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:43:01.287116+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:42:59.166414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2022-08-01T19:43:00.301184+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:43:01.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.825+0000 7fe77f59e700 1 -- 172.21.15.17:0/3259366486 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe758053a70 msgr2=0x7fe758055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:01.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.825+0000 7fe77f59e700 1 --2- 172.21.15.17:0/3259366486 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe758053a70 0x7fe758055f30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7fe76800d000 tx=0x7fe768007c20).stop 2022-08-01T19:43:01.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.825+0000 7fe77f59e700 1 -- 172.21.15.17:0/3259366486 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7800b01a0 msgr2=0x7fe7800aaa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:01.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.825+0000 7fe77f59e700 1 --2- 172.21.15.17:0/3259366486 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7800b01a0 0x7fe7800aaa40 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 rx=0x7fe770002f70 tx=0x7fe770014000).stop 2022-08-01T19:43:01.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.825+0000 7fe77f59e700 1 -- 172.21.15.17:0/3259366486 shutdown_connections 2022-08-01T19:43:01.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.825+0000 7fe77f59e700 1 --2- 172.21.15.17:0/3259366486 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe758053a70 0x7fe758055f30 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:01.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.825+0000 7fe77f59e700 1 --2- 172.21.15.17:0/3259366486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7800b2710 0x7fe780159b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:01.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.825+0000 7fe77f59e700 1 --2- 172.21.15.17:0/3259366486 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7800b01a0 0x7fe7800aaa40 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:01.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.825+0000 7fe77f59e700 1 --2- 172.21.15.17:0/3259366486 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe7800afd60 0x7fe78014d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:01.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.825+0000 7fe77f59e700 1 -- 172.21.15.17:0/3259366486 >> 172.21.15.17:0/3259366486 conn(0x7fe780060310 msgr2=0x7fe78005be30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:01.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.825+0000 7fe77f59e700 1 -- 172.21.15.17:0/3259366486 shutdown_connections 2022-08-01T19:43:01.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.829+0000 7fe77f59e700 1 -- 172.21.15.17:0/3259366486 wait complete. 2022-08-01T19:43:01.841 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-08-01T19:43:01.952 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.949+0000 7f2ce5ef9700 1 Processor -- start 2022-08-01T19:43:01.952 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.949+0000 7f2ce5ef9700 1 -- start start 2022-08-01T19:43:01.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.953+0000 7f2ce5ef9700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f2ce00b38a0 0x7f2ce00b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:01.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.953+0000 7f2ce5ef9700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2ce00a9da0 0x7f2ce00b1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:01.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.953+0000 7f2ce5ef9700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ce0143ab0 con 0x7f2ce00b0740 2022-08-01T19:43:01.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.953+0000 7f2ce5ef9700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ce0143d50 con 0x7f2ce00a9da0 2022-08-01T19:43:01.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.953+0000 7f2ce5ef9700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ce0143ff0 con 0x7f2ce00b38a0 2022-08-01T19:43:01.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.953+0000 7f2cdffff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2ce00a9da0 0x7f2ce00b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:01.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.953+0000 7f2cdffff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2ce00a9da0 0x7f2ce00b1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44226/0 (socket says 172.21.15.17:44226) 2022-08-01T19:43:01.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.953+0000 7f2cdffff700 1 -- 172.21.15.17:0/2700477080 learned_addr learned my addr 172.21.15.17:0/2700477080 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:01.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.953+0000 7f2cd7fff700 1 --2- 172.21.15.17:0/2700477080 >> v2:172.21.15.77:3300/0 conn(0x7f2ce00b38a0 0x7f2ce00b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:01.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.953+0000 7f2cdffff700 1 -- 172.21.15.17:0/2700477080 >> v2:172.21.15.77:3300/0 conn(0x7f2ce00b38a0 msgr2=0x7f2ce00b0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:01.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.953+0000 7f2cdffff700 1 --2- 172.21.15.17:0/2700477080 >> v2:172.21.15.77:3300/0 conn(0x7f2ce00b38a0 0x7f2ce00b0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:01.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.953+0000 7f2cdffff700 1 -- 172.21.15.17:0/2700477080 >> v1:172.21.15.77:6789/0 conn(0x7f2ce00b0740 legacy=0x7f2ce013d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:01.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.953+0000 7f2cdffff700 1 -- 172.21.15.17:0/2700477080 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce0144290 con 0x7f2ce00a9da0 2022-08-01T19:43:01.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.953+0000 7f2cdffff700 1 --2- 172.21.15.17:0/2700477080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ce00a9da0 0x7f2ce00b1240 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 rx=0x7f2cd80032f0 tx=0x7f2cd8009290).ready entity=mon.0 client_cookie=6a70b5287149cca2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:01.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.953+0000 7f2cdeffd700 1 -- 172.21.15.17:0/2700477080 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2cd800b870 con 0x7f2ce00a9da0 2022-08-01T19:43:01.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.953+0000 7f2cdeffd700 1 -- 172.21.15.17:0/2700477080 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd800b9d0 con 0x7f2ce00a9da0 2022-08-01T19:43:01.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.953+0000 7f2cdeffd700 1 -- 172.21.15.17:0/2700477080 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2cd800bb40 con 0x7f2ce00a9da0 2022-08-01T19:43:01.966 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.953+0000 7f2ce5ef9700 1 -- 172.21.15.17:0/2700477080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ce00a9da0 msgr2=0x7f2ce00b1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:01.966 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.953+0000 7f2ce5ef9700 1 --2- 172.21.15.17:0/2700477080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ce00a9da0 0x7f2ce00b1240 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 rx=0x7f2cd80032f0 tx=0x7f2cd8009290).stop 2022-08-01T19:43:01.967 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.953+0000 7f2ce5ef9700 1 -- 172.21.15.17:0/2700477080 shutdown_connections 2022-08-01T19:43:01.967 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.953+0000 7f2ce5ef9700 1 --2- 172.21.15.17:0/2700477080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ce00a9da0 0x7f2ce00b1240 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:01.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.953+0000 7f2ce5ef9700 1 --2- 172.21.15.17:0/2700477080 >> v2:172.21.15.77:3300/0 conn(0x7f2ce00b38a0 0x7f2ce00b0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:01.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.953+0000 7f2ce5ef9700 1 -- 172.21.15.17:0/2700477080 >> 172.21.15.17:0/2700477080 conn(0x7f2ce0060950 msgr2=0x7f2ce0060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:01.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.953+0000 7f2ce5ef9700 1 -- 172.21.15.17:0/2700477080 shutdown_connections 2022-08-01T19:43:01.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.953+0000 7f2ce5ef9700 1 -- 172.21.15.17:0/2700477080 wait complete. 2022-08-01T19:43:01.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.953+0000 7f2ce5ef9700 1 Processor -- start 2022-08-01T19:43:01.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.953+0000 7f2ce5ef9700 1 -- start start 2022-08-01T19:43:01.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.957+0000 7f2ce5ef9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ce00a9da0 0x7f2ce0152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:01.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.957+0000 7f2ce5ef9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ce00b0740 0x7f2ce0146fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:01.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.957+0000 7f2ce5ef9700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2ce00b38a0 0x7f2ce01562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:01.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.957+0000 7f2ce5ef9700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce00b4ee0 con 0x7f2ce00a9da0 2022-08-01T19:43:01.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.957+0000 7f2ce5ef9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce0155be0 con 0x7f2ce00b0740 2022-08-01T19:43:01.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.957+0000 7f2ce5ef9700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce00b5110 con 0x7f2ce00b38a0 2022-08-01T19:43:01.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.957+0000 7f2cdf7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ce00a9da0 0x7f2ce0152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:01.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.957+0000 7f2cdf7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ce00a9da0 0x7f2ce0152610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44228/0 (socket says 172.21.15.17:44228) 2022-08-01T19:43:01.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.957+0000 7f2cdf7fe700 1 -- 172.21.15.17:0/3918009335 learned_addr learned my addr 172.21.15.17:0/3918009335 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:01.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.957+0000 7f2cd7fff700 1 --2- 172.21.15.17:0/3918009335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ce00b0740 0x7f2ce0146fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:01.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.957+0000 7f2cdffff700 1 --2- 172.21.15.17:0/3918009335 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2ce00b38a0 0x7f2ce01562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:01.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.957+0000 7f2cdf7fe700 1 -- 172.21.15.17:0/3918009335 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2ce00b38a0 msgr2=0x7f2ce01562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:01.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.957+0000 7f2cdf7fe700 1 --2- 172.21.15.17:0/3918009335 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2ce00b38a0 0x7f2ce01562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:01.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.957+0000 7f2cdf7fe700 1 -- 172.21.15.17:0/3918009335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ce00b0740 msgr2=0x7f2ce0146fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:01.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.957+0000 7f2cdf7fe700 1 --2- 172.21.15.17:0/3918009335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ce00b0740 0x7f2ce0146fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:01.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.957+0000 7f2cdf7fe700 1 -- 172.21.15.17:0/3918009335 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd8009530 con 0x7f2ce00a9da0 2022-08-01T19:43:01.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.957+0000 7f2cd7fff700 1 --2- 172.21.15.17:0/3918009335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ce00b0740 0x7f2ce0146fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:43:01.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.957+0000 7f2cdf7fe700 1 --2- 172.21.15.17:0/3918009335 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ce00a9da0 0x7f2ce0152610 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 rx=0x7f2ce006abe0 tx=0x7f2cd0007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:01.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.957+0000 7f2cdd7fa700 1 -- 172.21.15.17:0/3918009335 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2cd000ce30 con 0x7f2ce00a9da0 2022-08-01T19:43:01.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.957+0000 7f2cdd7fa700 1 -- 172.21.15.17:0/3918009335 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd000f470 con 0x7f2ce00a9da0 2022-08-01T19:43:01.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.957+0000 7f2ce5ef9700 1 -- 172.21.15.17:0/3918009335 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce00b53b0 con 0x7f2ce00a9da0 2022-08-01T19:43:01.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.957+0000 7f2cdd7fa700 1 -- 172.21.15.17:0/3918009335 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2cd000f7d0 con 0x7f2ce00a9da0 2022-08-01T19:43:01.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.957+0000 7f2ce5ef9700 1 -- 172.21.15.17:0/3918009335 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce0147490 con 0x7f2ce00a9da0 2022-08-01T19:43:01.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.957+0000 7f2ce5ef9700 1 -- 172.21.15.17:0/3918009335 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ce013ddf0 con 0x7f2ce00a9da0 2022-08-01T19:43:01.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.957+0000 7f2cdd7fa700 1 -- 172.21.15.17:0/3918009335 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2cd0019030 con 0x7f2ce00a9da0 2022-08-01T19:43:01.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.957+0000 7f2cdd7fa700 1 --2- 172.21.15.17:0/3918009335 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2cc40539e0 0x7f2cc4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:01.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.957+0000 7f2cdd7fa700 1 -- 172.21.15.17:0/3918009335 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f2cd006b4b0 con 0x7f2ce00a9da0 2022-08-01T19:43:01.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.961+0000 7f2cd7fff700 1 --2- 172.21.15.17:0/3918009335 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2cc40539e0 0x7f2cc4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:01.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.961+0000 7f2cd7fff700 1 --2- 172.21.15.17:0/3918009335 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2cc40539e0 0x7f2cc4055ea0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f2cc800b000 tx=0x7f2cc8008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:01.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:01.965+0000 7f2cdd7fa700 1 -- 172.21.15.17:0/3918009335 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2cd0042270 con 0x7f2ce00a9da0 2022-08-01T19:43:02.135 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.133+0000 7f2ce5ef9700 1 -- 172.21.15.17:0/3918009335 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2ce00574e0 con 0x7f2ce00a9da0 2022-08-01T19:43:02.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.293+0000 7f2cdd7fa700 1 -- 172.21.15.17:0/3918009335 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v69) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2cd0045890 con 0x7f2ce00a9da0 2022-08-01T19:43:02.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.297+0000 7f2ce5ef9700 1 -- 172.21.15.17:0/3918009335 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2cc40539e0 msgr2=0x7f2cc4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:02.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.297+0000 7f2ce5ef9700 1 --2- 172.21.15.17:0/3918009335 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2cc40539e0 0x7f2cc4055ea0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f2cc800b000 tx=0x7f2cc8008040).stop 2022-08-01T19:43:02.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.297+0000 7f2ce5ef9700 1 -- 172.21.15.17:0/3918009335 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ce00a9da0 msgr2=0x7f2ce0152610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:02.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.297+0000 7f2ce5ef9700 1 --2- 172.21.15.17:0/3918009335 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ce00a9da0 0x7f2ce0152610 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 rx=0x7f2ce006abe0 tx=0x7f2cd0007c50).stop 2022-08-01T19:43:02.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.297+0000 7f2ce5ef9700 1 -- 172.21.15.17:0/3918009335 shutdown_connections 2022-08-01T19:43:02.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.297+0000 7f2ce5ef9700 1 --2- 172.21.15.17:0/3918009335 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2cc40539e0 0x7f2cc4055ea0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:02.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.297+0000 7f2ce5ef9700 1 --2- 172.21.15.17:0/3918009335 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2ce00b38a0 0x7f2ce01562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:02.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.297+0000 7f2ce5ef9700 1 --2- 172.21.15.17:0/3918009335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ce00b0740 0x7f2ce0146fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:02.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.297+0000 7f2ce5ef9700 1 --2- 172.21.15.17:0/3918009335 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ce00a9da0 0x7f2ce0152610 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:02.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.297+0000 7f2ce5ef9700 1 -- 172.21.15.17:0/3918009335 >> 172.21.15.17:0/3918009335 conn(0x7f2ce0060950 msgr2=0x7f2ce013e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:02.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.297+0000 7f2ce5ef9700 1 -- 172.21.15.17:0/3918009335 shutdown_connections 2022-08-01T19:43:02.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.297+0000 7f2ce5ef9700 1 -- 172.21.15.17:0/3918009335 wait complete. 2022-08-01T19:43:02.319 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:43:02.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.429+0000 7f0b7d8ab700 1 Processor -- start 2022-08-01T19:43:02.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.429+0000 7f0b7d8ab700 1 -- start start 2022-08-01T19:43:02.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.429+0000 7f0b7d8ab700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0b780b1f40 0x7f0b780b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:02.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.429+0000 7f0b7d8ab700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0b780a7250 0x7f0b781448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:02.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.429+0000 7f0b7d8ab700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b780b04e0 con 0x7f0b78148530 2022-08-01T19:43:02.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.429+0000 7f0b7d8ab700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b780a9250 con 0x7f0b780b1f40 2022-08-01T19:43:02.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.429+0000 7f0b7d8ab700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b780b0780 con 0x7f0b780a7250 2022-08-01T19:43:02.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.429+0000 7f0b76ffd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0b780b1f40 0x7f0b780b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:02.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.429+0000 7f0b76ffd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0b780b1f40 0x7f0b780b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44238/0 (socket says 172.21.15.17:44238) 2022-08-01T19:43:02.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.429+0000 7f0b76ffd700 1 -- 172.21.15.17:0/4118736661 learned_addr learned my addr 172.21.15.17:0/4118736661 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:02.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.429+0000 7f0b777fe700 1 --2- 172.21.15.17:0/4118736661 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0b780a7250 0x7f0b781448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:02.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.429+0000 7f0b76ffd700 1 -- 172.21.15.17:0/4118736661 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0b780a7250 msgr2=0x7f0b781448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:02.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.429+0000 7f0b76ffd700 1 --2- 172.21.15.17:0/4118736661 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0b780a7250 0x7f0b781448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:02.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.429+0000 7f0b76ffd700 1 -- 172.21.15.17:0/4118736661 >> v1:172.21.15.77:6789/0 conn(0x7f0b78148530 legacy=0x7f0b780b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:02.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.429+0000 7f0b76ffd700 1 -- 172.21.15.17:0/4118736661 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b780b0a20 con 0x7f0b780b1f40 2022-08-01T19:43:02.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.429+0000 7f0b777fe700 1 --2- 172.21.15.17:0/4118736661 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0b780a7250 0x7f0b781448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:43:02.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.429+0000 7f0b76ffd700 1 --2- 172.21.15.17:0/4118736661 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b780b1f40 0x7f0b780b1160 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 rx=0x7f0b781478c0 tx=0x7f0b60006eb0).ready entity=mon.0 client_cookie=b356b5b83610b5fb server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:02.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.429+0000 7f0b75ffb700 1 -- 172.21.15.17:0/4118736661 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b6000d5c0 con 0x7f0b780b1f40 2022-08-01T19:43:02.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.429+0000 7f0b75ffb700 1 -- 172.21.15.17:0/4118736661 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b60000e90 con 0x7f0b780b1f40 2022-08-01T19:43:02.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.429+0000 7f0b75ffb700 1 -- 172.21.15.17:0/4118736661 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b6000d950 con 0x7f0b780b1f40 2022-08-01T19:43:02.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.429+0000 7f0b7d8ab700 1 -- 172.21.15.17:0/4118736661 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b780b1f40 msgr2=0x7f0b780b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:02.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.429+0000 7f0b7d8ab700 1 --2- 172.21.15.17:0/4118736661 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b780b1f40 0x7f0b780b1160 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 rx=0x7f0b781478c0 tx=0x7f0b60006eb0).stop 2022-08-01T19:43:02.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.429+0000 7f0b7d8ab700 1 -- 172.21.15.17:0/4118736661 shutdown_connections 2022-08-01T19:43:02.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.429+0000 7f0b7d8ab700 1 --2- 172.21.15.17:0/4118736661 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0b780a7250 0x7f0b781448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:02.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.429+0000 7f0b7d8ab700 1 --2- 172.21.15.17:0/4118736661 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b780b1f40 0x7f0b780b1160 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:02.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.429+0000 7f0b7d8ab700 1 -- 172.21.15.17:0/4118736661 >> 172.21.15.17:0/4118736661 conn(0x7f0b7805f840 msgr2=0x7f0b7805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:02.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.429+0000 7f0b7d8ab700 1 -- 172.21.15.17:0/4118736661 shutdown_connections 2022-08-01T19:43:02.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.433+0000 7f0b7d8ab700 1 -- 172.21.15.17:0/4118736661 wait complete. 2022-08-01T19:43:02.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.433+0000 7f0b7d8ab700 1 Processor -- start 2022-08-01T19:43:02.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.433+0000 7f0b7d8ab700 1 -- start start 2022-08-01T19:43:02.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.433+0000 7f0b7d8ab700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b780a7250 0x7f0b780abf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:02.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.433+0000 7f0b7d8ab700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0b780b1f40 0x7f0b7814e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:02.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.433+0000 7f0b7d8ab700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b78148530 0x7f0b780ae710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:02.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.433+0000 7f0b7d8ab700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b780ab5c0 con 0x7f0b780a7250 2022-08-01T19:43:02.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.433+0000 7f0b7d8ab700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b780ab860 con 0x7f0b78148530 2022-08-01T19:43:02.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.433+0000 7f0b7d8ab700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b780abb00 con 0x7f0b780b1f40 2022-08-01T19:43:02.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.433+0000 7f0b76ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b780a7250 0x7f0b780abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:02.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.433+0000 7f0b76ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b780a7250 0x7f0b780abf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44244/0 (socket says 172.21.15.17:44244) 2022-08-01T19:43:02.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.433+0000 7f0b76ffd700 1 -- 172.21.15.17:0/1035437360 learned_addr learned my addr 172.21.15.17:0/1035437360 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:02.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.433+0000 7f0b777fe700 1 --2- 172.21.15.17:0/1035437360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b78148530 0x7f0b780ae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:02.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.433+0000 7f0b767fc700 1 --2- 172.21.15.17:0/1035437360 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0b780b1f40 0x7f0b7814e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:02.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.433+0000 7f0b76ffd700 1 -- 172.21.15.17:0/1035437360 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0b780b1f40 msgr2=0x7f0b7814e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:02.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.433+0000 7f0b76ffd700 1 --2- 172.21.15.17:0/1035437360 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0b780b1f40 0x7f0b7814e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:02.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.433+0000 7f0b76ffd700 1 -- 172.21.15.17:0/1035437360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b78148530 msgr2=0x7f0b780ae710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:02.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.433+0000 7f0b76ffd700 1 --2- 172.21.15.17:0/1035437360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b78148530 0x7f0b780ae710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:02.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.433+0000 7f0b76ffd700 1 -- 172.21.15.17:0/1035437360 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6000b1e0 con 0x7f0b780a7250 2022-08-01T19:43:02.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.433+0000 7f0b76ffd700 1 --2- 172.21.15.17:0/1035437360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b780a7250 0x7f0b780abf20 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 rx=0x7f0b781478c0 tx=0x7f0b60015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:02.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.433+0000 7f0b6ffff700 1 -- 172.21.15.17:0/1035437360 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b600116e0 con 0x7f0b780a7250 2022-08-01T19:43:02.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.437+0000 7f0b6ffff700 1 -- 172.21.15.17:0/1035437360 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b60011840 con 0x7f0b780a7250 2022-08-01T19:43:02.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.437+0000 7f0b7d8ab700 1 -- 172.21.15.17:0/1035437360 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b780adf90 con 0x7f0b780a7250 2022-08-01T19:43:02.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.437+0000 7f0b6ffff700 1 -- 172.21.15.17:0/1035437360 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b60011ba0 con 0x7f0b780a7250 2022-08-01T19:43:02.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.437+0000 7f0b7d8ab700 1 -- 172.21.15.17:0/1035437360 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b780aec00 con 0x7f0b780a7250 2022-08-01T19:43:02.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.437+0000 7f0b7d8ab700 1 -- 172.21.15.17:0/1035437360 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b780afb30 con 0x7f0b780a7250 2022-08-01T19:43:02.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.437+0000 7f0b6ffff700 1 -- 172.21.15.17:0/1035437360 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0b60025030 con 0x7f0b780a7250 2022-08-01T19:43:02.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.437+0000 7f0b6ffff700 1 --2- 172.21.15.17:0/1035437360 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0b50053b00 0x7f0b50055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:02.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.437+0000 7f0b6ffff700 1 -- 172.21.15.17:0/1035437360 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0b6006d180 con 0x7f0b780a7250 2022-08-01T19:43:02.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.441+0000 7f0b767fc700 1 --2- 172.21.15.17:0/1035437360 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0b50053b00 0x7f0b50055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:02.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.441+0000 7f0b767fc700 1 --2- 172.21.15.17:0/1035437360 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0b50053b00 0x7f0b50055fc0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7f0b7000d000 tx=0x7f0b70007f10).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:02.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.445+0000 7f0b6ffff700 1 -- 172.21.15.17:0/1035437360 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0b780afb30 con 0x7f0b780a7250 2022-08-01T19:43:02.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.605+0000 7f0b7d8ab700 1 -- 172.21.15.17:0/1035437360 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b780afb30 con 0x7f0b780a7250 2022-08-01T19:43:02.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.605+0000 7f0b6ffff700 1 -- 172.21.15.17:0/1035437360 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+2486 (secure 0 0 0) 0x7f0b780afb30 con 0x7f0b780a7250 2022-08-01T19:43:02.609 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:02.613 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":69,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7995,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.17:6833/4164640472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4164640472},{"type":"v1","addr":"172.21.15.17:6833","nonce":4164640472}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":67},{"gid":8001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/50015801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":50015801},{"type":"v1","addr":"172.21.15.17:6835","nonce":50015801}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:43:01.287477+0000","modified":"2022-08-01T19:43:02.290478+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8007},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8007":{"gid":8007,"name":"b","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6835/299216688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":299216688},{"type":"v1","addr":"172.21.15.77:6835","nonce":299216688}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2022-08-01T19:43:02.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.605+0000 7f0b7d8ab700 1 -- 172.21.15.17:0/1035437360 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0b50053b00 msgr2=0x7f0b50055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:02.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.605+0000 7f0b7d8ab700 1 --2- 172.21.15.17:0/1035437360 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0b50053b00 0x7f0b50055fc0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7f0b7000d000 tx=0x7f0b70007f10).stop 2022-08-01T19:43:02.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.605+0000 7f0b7d8ab700 1 -- 172.21.15.17:0/1035437360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b780a7250 msgr2=0x7f0b780abf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:02.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.605+0000 7f0b7d8ab700 1 --2- 172.21.15.17:0/1035437360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b780a7250 0x7f0b780abf20 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 rx=0x7f0b781478c0 tx=0x7f0b60015000).stop 2022-08-01T19:43:02.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.605+0000 7f0b7d8ab700 1 -- 172.21.15.17:0/1035437360 shutdown_connections 2022-08-01T19:43:02.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.605+0000 7f0b7d8ab700 1 --2- 172.21.15.17:0/1035437360 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0b50053b00 0x7f0b50055fc0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:02.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.605+0000 7f0b7d8ab700 1 --2- 172.21.15.17:0/1035437360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b78148530 0x7f0b780ae710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:02.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.605+0000 7f0b7d8ab700 1 --2- 172.21.15.17:0/1035437360 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0b780b1f40 0x7f0b7814e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:02.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.605+0000 7f0b7d8ab700 1 --2- 172.21.15.17:0/1035437360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b780a7250 0x7f0b780abf20 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:02.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.605+0000 7f0b7d8ab700 1 -- 172.21.15.17:0/1035437360 >> 172.21.15.17:0/1035437360 conn(0x7f0b7805f840 msgr2=0x7f0b780b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:02.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.605+0000 7f0b7d8ab700 1 -- 172.21.15.17:0/1035437360 shutdown_connections 2022-08-01T19:43:02.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.605+0000 7f0b7d8ab700 1 -- 172.21.15.17:0/1035437360 wait complete. 2022-08-01T19:43:02.621 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 69 2022-08-01T19:43:02.623 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:43:02.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.725+0000 7fb6197d7700 1 Processor -- start 2022-08-01T19:43:02.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.725+0000 7fb6197d7700 1 -- start start 2022-08-01T19:43:02.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.725+0000 7fb6197d7700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb6140b65c0 0x7fb614061600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:02.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.725+0000 7fb6197d7700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fb614147d80 0x7fb614144d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:02.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.725+0000 7fb6197d7700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6140b2100 con 0x7fb6140b41f0 2022-08-01T19:43:02.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.725+0000 7fb6197d7700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6140b1630 con 0x7fb6140b65c0 2022-08-01T19:43:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.725+0000 7fb6197d7700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6140b69a0 con 0x7fb614147d80 2022-08-01T19:43:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.725+0000 7fb613fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb6140b65c0 0x7fb614061600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.725+0000 7fb613fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb6140b65c0 0x7fb614061600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44250/0 (socket says 172.21.15.17:44250) 2022-08-01T19:43:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.725+0000 7fb613fff700 1 -- 172.21.15.17:0/422375965 learned_addr learned my addr 172.21.15.17:0/422375965 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:02.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.725+0000 7fb618fd6700 1 --2- 172.21.15.17:0/422375965 >> v2:172.21.15.77:3300/0 conn(0x7fb614147d80 0x7fb614144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:02.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.729+0000 7fb613fff700 1 -- 172.21.15.17:0/422375965 >> v2:172.21.15.77:3300/0 conn(0x7fb614147d80 msgr2=0x7fb614144d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:02.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.729+0000 7fb613fff700 1 --2- 172.21.15.17:0/422375965 >> v2:172.21.15.77:3300/0 conn(0x7fb614147d80 0x7fb614144d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:02.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.729+0000 7fb613fff700 1 -- 172.21.15.17:0/422375965 >> v1:172.21.15.77:6789/0 conn(0x7fb6140b41f0 legacy=0x7fb614144630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.729+0000 7fb613fff700 1 -- 172.21.15.17:0/422375965 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6141482a0 con 0x7fb6140b65c0 2022-08-01T19:43:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.729+0000 7fb613fff700 1 --2- 172.21.15.17:0/422375965 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6140b65c0 0x7fb614061600 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 rx=0x7fb60c0032f0 tx=0x7fb60c009290).ready entity=mon.0 client_cookie=f184281070b7d024 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.729+0000 7fb612ffd700 1 -- 172.21.15.17:0/422375965 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb60c00b870 con 0x7fb6140b65c0 2022-08-01T19:43:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.729+0000 7fb612ffd700 1 -- 172.21.15.17:0/422375965 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb60c00b9d0 con 0x7fb6140b65c0 2022-08-01T19:43:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.729+0000 7fb612ffd700 1 -- 172.21.15.17:0/422375965 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb60c00bb40 con 0x7fb6140b65c0 2022-08-01T19:43:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.729+0000 7fb6197d7700 1 -- 172.21.15.17:0/422375965 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6140b65c0 msgr2=0x7fb614061600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.729+0000 7fb6197d7700 1 --2- 172.21.15.17:0/422375965 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6140b65c0 0x7fb614061600 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 rx=0x7fb60c0032f0 tx=0x7fb60c009290).stop 2022-08-01T19:43:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.729+0000 7fb6197d7700 1 -- 172.21.15.17:0/422375965 shutdown_connections 2022-08-01T19:43:02.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.729+0000 7fb6197d7700 1 --2- 172.21.15.17:0/422375965 >> v2:172.21.15.77:3300/0 conn(0x7fb614147d80 0x7fb614144d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:02.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.729+0000 7fb6197d7700 1 --2- 172.21.15.17:0/422375965 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6140b65c0 0x7fb614061600 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:02.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.729+0000 7fb6197d7700 1 -- 172.21.15.17:0/422375965 >> 172.21.15.17:0/422375965 conn(0x7fb61405c620 msgr2=0x7fb61405ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:02.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.729+0000 7fb6197d7700 1 -- 172.21.15.17:0/422375965 shutdown_connections 2022-08-01T19:43:02.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.729+0000 7fb6197d7700 1 -- 172.21.15.17:0/422375965 wait complete. 2022-08-01T19:43:02.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.729+0000 7fb6197d7700 1 Processor -- start 2022-08-01T19:43:02.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.729+0000 7fb6197d7700 1 -- start start 2022-08-01T19:43:02.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.729+0000 7fb6197d7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6140b41f0 0x7fb6140b86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:02.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.729+0000 7fb6197d7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6140b65c0 0x7fb6140af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:02.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.733+0000 7fb6197d7700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb614147d80 0x7fb6140649d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:02.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.733+0000 7fb6197d7700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6140ae890 con 0x7fb6140b65c0 2022-08-01T19:43:02.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.733+0000 7fb6197d7700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb614063690 con 0x7fb6140b41f0 2022-08-01T19:43:02.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.733+0000 7fb6197d7700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6140af690 con 0x7fb614147d80 2022-08-01T19:43:02.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.733+0000 7fb6137fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6140b65c0 0x7fb6140af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:02.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.733+0000 7fb613fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6140b41f0 0x7fb6140b86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:02.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.733+0000 7fb6137fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6140b65c0 0x7fb6140af1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44258/0 (socket says 172.21.15.17:44258) 2022-08-01T19:43:02.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.733+0000 7fb618fd6700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb614147d80 0x7fb6140649d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:02.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.733+0000 7fb6137fe700 1 -- 172.21.15.17:0/1210338437 learned_addr learned my addr 172.21.15.17:0/1210338437 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:02.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.733+0000 7fb6137fe700 1 -- 172.21.15.17:0/1210338437 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb614147d80 msgr2=0x7fb6140649d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:02.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.733+0000 7fb6137fe700 1 --2- 172.21.15.17:0/1210338437 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb614147d80 0x7fb6140649d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:02.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.733+0000 7fb6137fe700 1 -- 172.21.15.17:0/1210338437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6140b41f0 msgr2=0x7fb6140b86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:02.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.733+0000 7fb6137fe700 1 --2- 172.21.15.17:0/1210338437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6140b41f0 0x7fb6140b86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:02.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.733+0000 7fb6137fe700 1 -- 172.21.15.17:0/1210338437 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb60c009530 con 0x7fb6140b65c0 2022-08-01T19:43:02.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.733+0000 7fb613fff700 1 --2- 172.21.15.17:0/1210338437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6140b41f0 0x7fb6140b86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:43:02.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.733+0000 7fb6137fe700 1 --2- 172.21.15.17:0/1210338437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6140b65c0 0x7fb6140af1a0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 rx=0x7fb6140ba7e0 tx=0x7fb5fc007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:02.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.733+0000 7fb6117fa700 1 -- 172.21.15.17:0/1210338437 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5fc007030 con 0x7fb6140b65c0 2022-08-01T19:43:02.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.733+0000 7fb6117fa700 1 -- 172.21.15.17:0/1210338437 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5fc007190 con 0x7fb6140b65c0 2022-08-01T19:43:02.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.733+0000 7fb6197d7700 1 -- 172.21.15.17:0/1210338437 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6140aea80 con 0x7fb6140b65c0 2022-08-01T19:43:02.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.733+0000 7fb6117fa700 1 -- 172.21.15.17:0/1210338437 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5fc006510 con 0x7fb6140b65c0 2022-08-01T19:43:02.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.733+0000 7fb6197d7700 1 -- 172.21.15.17:0/1210338437 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6140b04b0 con 0x7fb6140b65c0 2022-08-01T19:43:02.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.733+0000 7fb6117fa700 1 -- 172.21.15.17:0/1210338437 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb5fc010030 con 0x7fb6140b65c0 2022-08-01T19:43:02.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.733+0000 7fb6197d7700 1 -- 172.21.15.17:0/1210338437 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6140b2330 con 0x7fb6140b65c0 2022-08-01T19:43:02.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.737+0000 7fb6117fa700 1 --2- 172.21.15.17:0/1210338437 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb5f8053b00 0x7fb5f8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:02.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.737+0000 7fb6117fa700 1 -- 172.21.15.17:0/1210338437 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb5fc06bc40 con 0x7fb6140b65c0 2022-08-01T19:43:02.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.737+0000 7fb613fff700 1 --2- 172.21.15.17:0/1210338437 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb5f8053b00 0x7fb5f8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:02.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.737+0000 7fb613fff700 1 --2- 172.21.15.17:0/1210338437 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb5f8053b00 0x7fb5f8055fc0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7fb60c01b000 tx=0x7fb60c016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:02.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.741+0000 7fb6117fa700 1 -- 172.21.15.17:0/1210338437 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb5fc042a00 con 0x7fb6140b65c0 2022-08-01T19:43:02.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.869+0000 7fb6197d7700 1 -- 172.21.15.17:0/1210338437 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6140574e0 con 0x7fb6140b65c0 2022-08-01T19:43:02.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.873+0000 7fb6117fa700 1 -- 172.21.15.17:0/1210338437 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+16285 (secure 0 0 0) 0x7fb5fc046020 con 0x7fb6140b65c0 2022-08-01T19:43:02.875 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:02.876 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":137,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:43:02.291265+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:42:59.166414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2022-08-01T19:43:00.301184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:43:02.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.873+0000 7fb6197d7700 1 -- 172.21.15.17:0/1210338437 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb5f8053b00 msgr2=0x7fb5f8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:02.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.873+0000 7fb6197d7700 1 --2- 172.21.15.17:0/1210338437 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb5f8053b00 0x7fb5f8055fc0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7fb60c01b000 tx=0x7fb60c016000).stop 2022-08-01T19:43:02.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.873+0000 7fb6197d7700 1 -- 172.21.15.17:0/1210338437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6140b65c0 msgr2=0x7fb6140af1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:02.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.873+0000 7fb6197d7700 1 --2- 172.21.15.17:0/1210338437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6140b65c0 0x7fb6140af1a0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 rx=0x7fb6140ba7e0 tx=0x7fb5fc007c10).stop 2022-08-01T19:43:02.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.873+0000 7fb6197d7700 1 -- 172.21.15.17:0/1210338437 shutdown_connections 2022-08-01T19:43:02.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.873+0000 7fb6197d7700 1 --2- 172.21.15.17:0/1210338437 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb5f8053b00 0x7fb5f8055fc0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:02.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.873+0000 7fb6197d7700 1 --2- 172.21.15.17:0/1210338437 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb614147d80 0x7fb6140649d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:02.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.873+0000 7fb6197d7700 1 --2- 172.21.15.17:0/1210338437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6140b65c0 0x7fb6140af1a0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:02.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.873+0000 7fb6197d7700 1 --2- 172.21.15.17:0/1210338437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6140b41f0 0x7fb6140b86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:02.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.873+0000 7fb6197d7700 1 -- 172.21.15.17:0/1210338437 >> 172.21.15.17:0/1210338437 conn(0x7fb61405c620 msgr2=0x7fb614144fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:02.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.873+0000 7fb6197d7700 1 -- 172.21.15.17:0/1210338437 shutdown_connections 2022-08-01T19:43:02.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:02.873+0000 7fb6197d7700 1 -- 172.21.15.17:0/1210338437 wait complete. 2022-08-01T19:43:02.889 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T19:43:03.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.001+0000 7fe839b95700 1 Processor -- start 2022-08-01T19:43:03.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.001+0000 7fe839b95700 1 -- start start 2022-08-01T19:43:03.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.005+0000 7fe839b95700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe834061890 0x7fe8341466b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:03.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.005+0000 7fe839b95700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8340aee80 con 0x7fe8340614b0 2022-08-01T19:43:03.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.005+0000 7fe839b95700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8340b70b0 con 0x7fe8340ad750 2022-08-01T19:43:03.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.005+0000 7fe839b95700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8340af0b0 con 0x7fe834061890 2022-08-01T19:43:03.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.005+0000 7fe838b93700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe8340614b0 0x7fe8340b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36458/0 (socket says 172.21.15.17:36458) 2022-08-01T19:43:03.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.005+0000 7fe838b93700 1 -- 172.21.15.17:0/592370842 learned_addr learned my addr 172.21.15.17:0/592370842 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:03.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.005+0000 7fe839394700 1 --2- 172.21.15.17:0/592370842 >> v2:172.21.15.17:3300/0 conn(0x7fe834061890 0x7fe8341466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:03.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.005+0000 7fe8337fe700 1 -- 172.21.15.17:0/592370842 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 546881099 0 0) 0x7fe8340aee80 con 0x7fe8340614b0 2022-08-01T19:43:03.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.005+0000 7fe8337fe700 1 -- 172.21.15.17:0/592370842 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe814003260 con 0x7fe8340614b0 2022-08-01T19:43:03.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.005+0000 7fe8337fe700 1 -- 172.21.15.17:0/592370842 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3763770422 0 0) 0x7fe8340b70b0 con 0x7fe8340ad750 2022-08-01T19:43:03.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.005+0000 7fe8337fe700 1 -- 172.21.15.17:0/592370842 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8340aee80 con 0x7fe8340ad750 2022-08-01T19:43:03.026 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.005+0000 7fe839394700 1 -- 172.21.15.17:0/592370842 >> v1:172.21.15.77:6789/0 conn(0x7fe8340ad750 legacy=0x7fe8341405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:03.026 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.005+0000 7fe839394700 1 -- 172.21.15.17:0/592370842 >> v1:172.21.15.17:6789/0 conn(0x7fe8340614b0 legacy=0x7fe8340b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:03.026 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.005+0000 7fe8337fe700 1 -- 172.21.15.17:0/592370842 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1870030853 0 0) 0x7fe814003260 con 0x7fe8340614b0 2022-08-01T19:43:03.027 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.005+0000 7fe839394700 1 -- 172.21.15.17:0/592370842 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8340af350 con 0x7fe834061890 2022-08-01T19:43:03.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.005+0000 7fe839394700 1 --2- 172.21.15.17:0/592370842 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe834061890 0x7fe8341466b0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 rx=0x7fe824001490 tx=0x7fe82c006df0).ready entity=mon.0 client_cookie=319607ee1b6dcaf5 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:03.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.005+0000 7fe8337fe700 1 -- 172.21.15.17:0/592370842 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8240019e0 con 0x7fe834061890 2022-08-01T19:43:03.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.005+0000 7fe8337fe700 1 -- 172.21.15.17:0/592370842 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe82c000e90 con 0x7fe834061890 2022-08-01T19:43:03.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.005+0000 7fe8337fe700 1 -- 172.21.15.17:0/592370842 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe82c012460 con 0x7fe834061890 2022-08-01T19:43:03.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.005+0000 7fe839b95700 1 -- 172.21.15.17:0/592370842 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe834061890 msgr2=0x7fe8341466b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:03.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.005+0000 7fe839b95700 1 --2- 172.21.15.17:0/592370842 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe834061890 0x7fe8341466b0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 rx=0x7fe824001490 tx=0x7fe82c006df0).stop 2022-08-01T19:43:03.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.005+0000 7fe839b95700 1 -- 172.21.15.17:0/592370842 shutdown_connections 2022-08-01T19:43:03.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.005+0000 7fe839b95700 1 --2- 172.21.15.17:0/592370842 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe834061890 0x7fe8341466b0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:03.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.005+0000 7fe839b95700 1 -- 172.21.15.17:0/592370842 >> 172.21.15.17:0/592370842 conn(0x7fe834066f70 msgr2=0x7fe834067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:03.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.005+0000 7fe839b95700 1 -- 172.21.15.17:0/592370842 shutdown_connections 2022-08-01T19:43:03.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.005+0000 7fe839b95700 1 -- 172.21.15.17:0/592370842 wait complete. 2022-08-01T19:43:03.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.009+0000 7fe839b95700 1 Processor -- start 2022-08-01T19:43:03.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.009+0000 7fe839b95700 1 -- start start 2022-08-01T19:43:03.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.009+0000 7fe839b95700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8340614b0 0x7fe834144ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:03.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.009+0000 7fe839b95700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe834061890 0x7fe8340ac460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:03.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.009+0000 7fe839b95700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe8340ad750 0x7fe8340aa600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:03.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.009+0000 7fe839b95700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8340aa1a0 con 0x7fe834061890 2022-08-01T19:43:03.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.009+0000 7fe839b95700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8341568b0 con 0x7fe8340614b0 2022-08-01T19:43:03.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.009+0000 7fe839b95700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe834156b50 con 0x7fe8340ad750 2022-08-01T19:43:03.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.009+0000 7fe833fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe834061890 0x7fe8340ac460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:03.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.009+0000 7fe833fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe834061890 0x7fe8340ac460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44272/0 (socket says 172.21.15.17:44272) 2022-08-01T19:43:03.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.009+0000 7fe838b93700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8340614b0 0x7fe834144ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:03.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.009+0000 7fe833fff700 1 -- 172.21.15.17:0/226525033 learned_addr learned my addr 172.21.15.17:0/226525033 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:03.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.009+0000 7fe839394700 1 --2- 172.21.15.17:0/226525033 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe8340ad750 0x7fe8340aa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:03.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.009+0000 7fe833fff700 1 -- 172.21.15.17:0/226525033 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe8340ad750 msgr2=0x7fe8340aa600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:03.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.009+0000 7fe833fff700 1 --2- 172.21.15.17:0/226525033 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe8340ad750 0x7fe8340aa600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:03.040 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.009+0000 7fe833fff700 1 -- 172.21.15.17:0/226525033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8340614b0 msgr2=0x7fe834144ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:03.040 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.009+0000 7fe833fff700 1 --2- 172.21.15.17:0/226525033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8340614b0 0x7fe834144ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:03.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.009+0000 7fe833fff700 1 -- 172.21.15.17:0/226525033 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe82c00b180 con 0x7fe834061890 2022-08-01T19:43:03.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.009+0000 7fe833fff700 1 --2- 172.21.15.17:0/226525033 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe834061890 0x7fe8340ac460 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0x7fe81c001110 tx=0x7fe81c007a80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:03.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.013+0000 7fe831ffb700 1 -- 172.21.15.17:0/226525033 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe81c00ce90 con 0x7fe834061890 2022-08-01T19:43:03.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.013+0000 7fe831ffb700 1 -- 172.21.15.17:0/226525033 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe81c012400 con 0x7fe834061890 2022-08-01T19:43:03.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.013+0000 7fe839b95700 1 -- 172.21.15.17:0/226525033 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8340ac950 con 0x7fe834061890 2022-08-01T19:43:03.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.013+0000 7fe831ffb700 1 -- 172.21.15.17:0/226525033 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe81c012760 con 0x7fe834061890 2022-08-01T19:43:03.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.013+0000 7fe839b95700 1 -- 172.21.15.17:0/226525033 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8340b3fc0 con 0x7fe834061890 2022-08-01T19:43:03.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.013+0000 7fe831ffb700 1 -- 172.21.15.17:0/226525033 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe81c019030 con 0x7fe834061890 2022-08-01T19:43:03.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.013+0000 7fe831ffb700 1 --2- 172.21.15.17:0/226525033 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe818053aa0 0x7fe818055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:03.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.013+0000 7fe831ffb700 1 -- 172.21.15.17:0/226525033 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe81c06b480 con 0x7fe834061890 2022-08-01T19:43:03.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.013+0000 7fe838b93700 1 --2- 172.21.15.17:0/226525033 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe818053aa0 0x7fe818055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:03.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.013+0000 7fe839b95700 1 -- 172.21.15.17:0/226525033 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8340acc60 con 0x7fe834061890 2022-08-01T19:43:03.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.013+0000 7fe838b93700 1 --2- 172.21.15.17:0/226525033 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe818053aa0 0x7fe818055f60 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7fe82400d000 tx=0x7fe824000f40).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:03.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.021+0000 7fe831ffb700 1 -- 172.21.15.17:0/226525033 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe8340acc60 con 0x7fe834061890 2022-08-01T19:43:03.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.189+0000 7fe839b95700 1 -- 172.21.15.17:0/226525033 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fe8340acc60 con 0x7fe834061890 2022-08-01T19:43:03.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.193+0000 7fe831ffb700 1 -- 172.21.15.17:0/226525033 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v34) v1 ==== 184+0+0 (secure 0 0 0) 0x7fe8340acc60 con 0x7fe834061890 2022-08-01T19:43:03.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.197+0000 7fe839b95700 1 -- 172.21.15.17:0/226525033 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe818053aa0 msgr2=0x7fe818055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:03.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.197+0000 7fe839b95700 1 --2- 172.21.15.17:0/226525033 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe818053aa0 0x7fe818055f60 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7fe82400d000 tx=0x7fe824000f40).stop 2022-08-01T19:43:03.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.197+0000 7fe839b95700 1 -- 172.21.15.17:0/226525033 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe834061890 msgr2=0x7fe8340ac460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:03.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.197+0000 7fe839b95700 1 --2- 172.21.15.17:0/226525033 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe834061890 0x7fe8340ac460 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0x7fe81c001110 tx=0x7fe81c007a80).stop 2022-08-01T19:43:03.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.197+0000 7fe839b95700 1 -- 172.21.15.17:0/226525033 shutdown_connections 2022-08-01T19:43:03.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.197+0000 7fe839b95700 1 --2- 172.21.15.17:0/226525033 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe818053aa0 0x7fe818055f60 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:03.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.197+0000 7fe839b95700 1 --2- 172.21.15.17:0/226525033 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe8340ad750 0x7fe8340aa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:03.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.197+0000 7fe839b95700 1 --2- 172.21.15.17:0/226525033 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe834061890 0x7fe8340ac460 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:03.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.197+0000 7fe839b95700 1 --2- 172.21.15.17:0/226525033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8340614b0 0x7fe834144ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:03.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.197+0000 7fe839b95700 1 -- 172.21.15.17:0/226525033 >> 172.21.15.17:0/226525033 conn(0x7fe834066f70 msgr2=0x7fe834067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:03.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.197+0000 7fe839b95700 1 -- 172.21.15.17:0/226525033 shutdown_connections 2022-08-01T19:43:03.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.197+0000 7fe839b95700 1 -- 172.21.15.17:0/226525033 wait complete. 2022-08-01T19:43:03.210 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.0 2022-08-01T19:43:03.214 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T19:43:03.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.321+0000 7fc5f58b0700 1 Processor -- start 2022-08-01T19:43:03.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.321+0000 7fc5f58b0700 1 -- start start 2022-08-01T19:43:03.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.321+0000 7fc5f58b0700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fc5f0067490 0x7fc5f0143a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:03.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.321+0000 7fc5f58b0700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc5f0143f70 0x7fc5f014b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:03.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.321+0000 7fc5f58b0700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5f00aad40 con 0x7fc5f0060290 2022-08-01T19:43:03.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.321+0000 7fc5f58b0700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5f0060670 con 0x7fc5f0143f70 2022-08-01T19:43:03.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.321+0000 7fc5f58b0700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5f0067030 con 0x7fc5f0067490 2022-08-01T19:43:03.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.321+0000 7fc5ef7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc5f0143f70 0x7fc5f014b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:03.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.321+0000 7fc5eeffd700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc5f0060290 0x7fc5f00b48e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36472/0 (socket says 172.21.15.17:36472) 2022-08-01T19:43:03.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.321+0000 7fc5eeffd700 1 -- 172.21.15.17:0/802340947 learned_addr learned my addr 172.21.15.17:0/802340947 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:03.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.321+0000 7fc5e67fc700 1 --2- 172.21.15.17:0/802340947 >> v2:172.21.15.77:3300/0 conn(0x7fc5f0067490 0x7fc5f0143a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:03.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.321+0000 7fc5ef7fe700 1 -- 172.21.15.17:0/802340947 >> v2:172.21.15.77:3300/0 conn(0x7fc5f0067490 msgr2=0x7fc5f0143a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:03.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.321+0000 7fc5ef7fe700 1 --2- 172.21.15.17:0/802340947 >> v2:172.21.15.77:3300/0 conn(0x7fc5f0067490 0x7fc5f0143a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:03.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.321+0000 7fc5ef7fe700 1 -- 172.21.15.17:0/802340947 >> v1:172.21.15.17:6789/0 conn(0x7fc5f0060290 legacy=0x7fc5f00b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:03.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.321+0000 7fc5ee7fc700 1 -- 172.21.15.17:0/802340947 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3481288365 0 0) 0x7fc5f00aad40 con 0x7fc5f0060290 2022-08-01T19:43:03.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.321+0000 7fc5ef7fe700 1 -- 172.21.15.17:0/802340947 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f00b3540 con 0x7fc5f0143f70 2022-08-01T19:43:03.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.321+0000 7fc5ef7fe700 1 --2- 172.21.15.17:0/802340947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5f0143f70 0x7fc5f014b9e0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 rx=0x7fc5e80032f0 tx=0x7fc5e80091d0).ready entity=mon.0 client_cookie=4b9121f39c54beff server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:03.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.321+0000 7fc5ee7fc700 1 -- 172.21.15.17:0/802340947 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5e800b680 con 0x7fc5f0143f70 2022-08-01T19:43:03.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.321+0000 7fc5ee7fc700 1 -- 172.21.15.17:0/802340947 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e800b7e0 con 0x7fc5f0143f70 2022-08-01T19:43:03.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.321+0000 7fc5ee7fc700 1 -- 172.21.15.17:0/802340947 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5e800b950 con 0x7fc5f0143f70 2022-08-01T19:43:03.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.325+0000 7fc5f58b0700 1 -- 172.21.15.17:0/802340947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5f0143f70 msgr2=0x7fc5f014b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:03.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.325+0000 7fc5f58b0700 1 --2- 172.21.15.17:0/802340947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5f0143f70 0x7fc5f014b9e0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 rx=0x7fc5e80032f0 tx=0x7fc5e80091d0).stop 2022-08-01T19:43:03.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.325+0000 7fc5f58b0700 1 -- 172.21.15.17:0/802340947 shutdown_connections 2022-08-01T19:43:03.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.325+0000 7fc5f58b0700 1 --2- 172.21.15.17:0/802340947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5f0143f70 0x7fc5f014b9e0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:03.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.325+0000 7fc5f58b0700 1 --2- 172.21.15.17:0/802340947 >> v2:172.21.15.77:3300/0 conn(0x7fc5f0067490 0x7fc5f0143a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:03.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.325+0000 7fc5f58b0700 1 -- 172.21.15.17:0/802340947 >> 172.21.15.17:0/802340947 conn(0x7fc5f005d0c0 msgr2=0x7fc5f005f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:03.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.325+0000 7fc5f58b0700 1 -- 172.21.15.17:0/802340947 shutdown_connections 2022-08-01T19:43:03.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.325+0000 7fc5f58b0700 1 -- 172.21.15.17:0/802340947 wait complete. 2022-08-01T19:43:03.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.325+0000 7fc5f58b0700 1 Processor -- start 2022-08-01T19:43:03.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.325+0000 7fc5f58b0700 1 -- start start 2022-08-01T19:43:03.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.329+0000 7fc5f58b0700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc5f0060290 0x7fc5f01564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:03.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.329+0000 7fc5f58b0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5f0067490 0x7fc5f0152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:03.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.329+0000 7fc5f58b0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5f0143f70 0x7fc5f013f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:03.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.329+0000 7fc5f58b0700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f0140660 con 0x7fc5f0143f70 2022-08-01T19:43:03.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.329+0000 7fc5f58b0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f0155b20 con 0x7fc5f0067490 2022-08-01T19:43:03.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.329+0000 7fc5f58b0700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f0140890 con 0x7fc5f0060290 2022-08-01T19:43:03.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.329+0000 7fc5ef7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5f0143f70 0x7fc5f013f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:03.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.329+0000 7fc5ef7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5f0143f70 0x7fc5f013f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44288/0 (socket says 172.21.15.17:44288) 2022-08-01T19:43:03.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.329+0000 7fc5ef7fe700 1 -- 172.21.15.17:0/4273871421 learned_addr learned my addr 172.21.15.17:0/4273871421 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:03.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.329+0000 7fc5e67fc700 1 --2- 172.21.15.17:0/4273871421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5f0067490 0x7fc5f0152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:03.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.329+0000 7fc5eeffd700 1 --2- 172.21.15.17:0/4273871421 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc5f0060290 0x7fc5f01564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:03.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.329+0000 7fc5ef7fe700 1 -- 172.21.15.17:0/4273871421 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc5f0060290 msgr2=0x7fc5f01564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:03.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.329+0000 7fc5ef7fe700 1 --2- 172.21.15.17:0/4273871421 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc5f0060290 0x7fc5f01564c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:03.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.329+0000 7fc5ef7fe700 1 -- 172.21.15.17:0/4273871421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5f0067490 msgr2=0x7fc5f0152550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:03.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.329+0000 7fc5ef7fe700 1 --2- 172.21.15.17:0/4273871421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5f0067490 0x7fc5f0152550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:03.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.329+0000 7fc5ef7fe700 1 -- 172.21.15.17:0/4273871421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e8009470 con 0x7fc5f0143f70 2022-08-01T19:43:03.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.329+0000 7fc5e67fc700 1 --2- 172.21.15.17:0/4273871421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5f0067490 0x7fc5f0152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:43:03.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.329+0000 7fc5ef7fe700 1 --2- 172.21.15.17:0/4273871421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5f0143f70 0x7fc5f013f170 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 rx=0x7fc5e80032f0 tx=0x7fc5e8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:03.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.329+0000 7fc5ecff9700 1 -- 172.21.15.17:0/4273871421 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5e800b680 con 0x7fc5f0143f70 2022-08-01T19:43:03.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.329+0000 7fc5ecff9700 1 -- 172.21.15.17:0/4273871421 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e80107a0 con 0x7fc5f0143f70 2022-08-01T19:43:03.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.329+0000 7fc5f58b0700 1 -- 172.21.15.17:0/4273871421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f0155d80 con 0x7fc5f0143f70 2022-08-01T19:43:03.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.329+0000 7fc5ecff9700 1 -- 172.21.15.17:0/4273871421 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5e8010b00 con 0x7fc5f0143f70 2022-08-01T19:43:03.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.329+0000 7fc5f58b0700 1 -- 172.21.15.17:0/4273871421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f01569b0 con 0x7fc5f0143f70 2022-08-01T19:43:03.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.329+0000 7fc5f58b0700 1 -- 172.21.15.17:0/4273871421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f00574e0 con 0x7fc5f0143f70 2022-08-01T19:43:03.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.329+0000 7fc5ecff9700 1 -- 172.21.15.17:0/4273871421 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc5e8027020 con 0x7fc5f0143f70 2022-08-01T19:43:03.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.333+0000 7fc5ecff9700 1 --2- 172.21.15.17:0/4273871421 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc5d4053a70 0x7fc5d4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:03.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.333+0000 7fc5ecff9700 1 -- 172.21.15.17:0/4273871421 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc5e806d5f0 con 0x7fc5f0143f70 2022-08-01T19:43:03.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.333+0000 7fc5eeffd700 1 --2- 172.21.15.17:0/4273871421 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc5d4053a70 0x7fc5d4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:03.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.333+0000 7fc5eeffd700 1 --2- 172.21.15.17:0/4273871421 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc5d4053a70 0x7fc5d4055f30 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7fc5e000b000 tx=0x7fc5e0009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:03.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.341+0000 7fc5ecff9700 1 -- 172.21.15.17:0/4273871421 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc5e80442a0 con 0x7fc5f0143f70 2022-08-01T19:43:03.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.505+0000 7fc5f58b0700 1 -- 172.21.15.17:0/4273871421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fc5f0069a70 con 0x7fc5f0143f70 2022-08-01T19:43:03.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.513+0000 7fc5ecff9700 1 -- 172.21.15.17:0/4273871421 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v35) v1 ==== 184+0+0 (secure 0 0 0) 0x7fc5e80150e0 con 0x7fc5f0143f70 2022-08-01T19:43:03.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.517+0000 7fc5f58b0700 1 -- 172.21.15.17:0/4273871421 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc5d4053a70 msgr2=0x7fc5d4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:03.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.517+0000 7fc5f58b0700 1 --2- 172.21.15.17:0/4273871421 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc5d4053a70 0x7fc5d4055f30 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7fc5e000b000 tx=0x7fc5e0009040).stop 2022-08-01T19:43:03.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.517+0000 7fc5f58b0700 1 -- 172.21.15.17:0/4273871421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5f0143f70 msgr2=0x7fc5f013f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:03.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.517+0000 7fc5f58b0700 1 --2- 172.21.15.17:0/4273871421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5f0143f70 0x7fc5f013f170 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 rx=0x7fc5e80032f0 tx=0x7fc5e8016000).stop 2022-08-01T19:43:03.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.517+0000 7fc5f58b0700 1 -- 172.21.15.17:0/4273871421 shutdown_connections 2022-08-01T19:43:03.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.517+0000 7fc5f58b0700 1 --2- 172.21.15.17:0/4273871421 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc5d4053a70 0x7fc5d4055f30 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:03.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.517+0000 7fc5f58b0700 1 --2- 172.21.15.17:0/4273871421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5f0143f70 0x7fc5f013f170 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:03.525 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.517+0000 7fc5f58b0700 1 --2- 172.21.15.17:0/4273871421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5f0067490 0x7fc5f0152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:03.525 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.517+0000 7fc5f58b0700 1 --2- 172.21.15.17:0/4273871421 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc5f0060290 0x7fc5f01564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:03.525 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.517+0000 7fc5f58b0700 1 -- 172.21.15.17:0/4273871421 >> 172.21.15.17:0/4273871421 conn(0x7fc5f005d0c0 msgr2=0x7fc5f00b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:03.526 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.517+0000 7fc5f58b0700 1 -- 172.21.15.17:0/4273871421 shutdown_connections 2022-08-01T19:43:03.526 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.517+0000 7fc5f58b0700 1 -- 172.21.15.17:0/4273871421 wait complete. 2022-08-01T19:43:03.527 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.1 2022-08-01T19:43:03.533 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:43:03.650 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.633+0000 7fe0b311a700 1 Processor -- start 2022-08-01T19:43:03.650 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.633+0000 7fe0b311a700 1 -- start start 2022-08-01T19:43:03.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.637+0000 7fe0b311a700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe0ac0b2710 0x7fe0ac1401e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:03.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.637+0000 7fe0b311a700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe0ac0b0270 0x7fe0ac1406d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:03.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.637+0000 7fe0b311a700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0ac067a10 con 0x7fe0ac0ac880 2022-08-01T19:43:03.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.637+0000 7fe0b311a700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0ac0ad740 con 0x7fe0ac0b0270 2022-08-01T19:43:03.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.637+0000 7fe0b311a700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ac067cb0 con 0x7fe0ac0b2710 2022-08-01T19:43:03.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.637+0000 7fe0b1917700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe0ac0b0270 0x7fe0ac1406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:03.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.637+0000 7fe0b1917700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe0ac0b0270 0x7fe0ac1406d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44292/0 (socket says 172.21.15.17:44292) 2022-08-01T19:43:03.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.637+0000 7fe0b1917700 1 -- 172.21.15.17:0/772723835 learned_addr learned my addr 172.21.15.17:0/772723835 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:03.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.637+0000 7fe0b2118700 1 --2- 172.21.15.17:0/772723835 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe0ac0b2710 0x7fe0ac1401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:03.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.637+0000 7fe0b1917700 1 -- 172.21.15.17:0/772723835 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe0ac0b2710 msgr2=0x7fe0ac1401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:03.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.637+0000 7fe0b1917700 1 --2- 172.21.15.17:0/772723835 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe0ac0b2710 0x7fe0ac1401e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:03.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.637+0000 7fe0b1917700 1 -- 172.21.15.17:0/772723835 >> v1:172.21.15.77:6789/0 conn(0x7fe0ac0ac880 legacy=0x7fe0ac143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:03.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.637+0000 7fe0b1917700 1 -- 172.21.15.17:0/772723835 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ac147160 con 0x7fe0ac0b0270 2022-08-01T19:43:03.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.637+0000 7fe0b2118700 1 --2- 172.21.15.17:0/772723835 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe0ac0b2710 0x7fe0ac1401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:43:03.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.637+0000 7fe0b1917700 1 --2- 172.21.15.17:0/772723835 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0ac0b0270 0x7fe0ac1406d0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 rx=0x7fe0a8002f70 tx=0x7fe0a8006eb0).ready entity=mon.0 client_cookie=daf5fa7b2a4ac9b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:03.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.637+0000 7fe0b1116700 1 -- 172.21.15.17:0/772723835 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0a800d5c0 con 0x7fe0ac0b0270 2022-08-01T19:43:03.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.637+0000 7fe0b1116700 1 -- 172.21.15.17:0/772723835 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0a8000e90 con 0x7fe0ac0b0270 2022-08-01T19:43:03.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.637+0000 7fe0b1116700 1 -- 172.21.15.17:0/772723835 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0a800d950 con 0x7fe0ac0b0270 2022-08-01T19:43:03.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.637+0000 7fe0b311a700 1 -- 172.21.15.17:0/772723835 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0ac0b0270 msgr2=0x7fe0ac1406d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:03.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.637+0000 7fe0b311a700 1 --2- 172.21.15.17:0/772723835 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0ac0b0270 0x7fe0ac1406d0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 rx=0x7fe0a8002f70 tx=0x7fe0a8006eb0).stop 2022-08-01T19:43:03.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.637+0000 7fe0b311a700 1 -- 172.21.15.17:0/772723835 shutdown_connections 2022-08-01T19:43:03.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.637+0000 7fe0b311a700 1 --2- 172.21.15.17:0/772723835 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0ac0b0270 0x7fe0ac1406d0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:03.660 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.637+0000 7fe0b311a700 1 --2- 172.21.15.17:0/772723835 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe0ac0b2710 0x7fe0ac1401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:03.660 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.637+0000 7fe0b311a700 1 -- 172.21.15.17:0/772723835 >> 172.21.15.17:0/772723835 conn(0x7fe0ac060310 msgr2=0x7fe0ac0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:03.660 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.637+0000 7fe0b311a700 1 -- 172.21.15.17:0/772723835 shutdown_connections 2022-08-01T19:43:03.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.637+0000 7fe0b311a700 1 -- 172.21.15.17:0/772723835 wait complete. 2022-08-01T19:43:03.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.641+0000 7fe0b311a700 1 Processor -- start 2022-08-01T19:43:03.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.641+0000 7fe0b311a700 1 -- start start 2022-08-01T19:43:03.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.641+0000 7fe0b311a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0ac0ac880 0x7fe0ac09f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:03.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.641+0000 7fe0b311a700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe0ac0b0270 0x7fe0ac09fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:03.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.641+0000 7fe0b311a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0ac0b2710 0x7fe0ac14f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:03.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.641+0000 7fe0b311a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ac143db0 con 0x7fe0ac0b2710 2022-08-01T19:43:03.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.641+0000 7fe0b311a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ac09ee10 con 0x7fe0ac0ac880 2022-08-01T19:43:03.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.641+0000 7fe0b311a700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ac1531b0 con 0x7fe0ac0b0270 2022-08-01T19:43:03.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.641+0000 7fe0b2919700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0ac0b2710 0x7fe0ac14f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:03.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.641+0000 7fe0b2919700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0ac0b2710 0x7fe0ac14f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44302/0 (socket says 172.21.15.17:44302) 2022-08-01T19:43:03.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.641+0000 7fe0b2919700 1 -- 172.21.15.17:0/3724772139 learned_addr learned my addr 172.21.15.17:0/3724772139 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:03.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.641+0000 7fe0b2118700 1 --2- 172.21.15.17:0/3724772139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0ac0ac880 0x7fe0ac09f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:03.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.641+0000 7fe0b1917700 1 --2- 172.21.15.17:0/3724772139 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe0ac0b0270 0x7fe0ac09fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:03.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.641+0000 7fe0b2919700 1 -- 172.21.15.17:0/3724772139 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe0ac0b0270 msgr2=0x7fe0ac09fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:03.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.641+0000 7fe0b2919700 1 --2- 172.21.15.17:0/3724772139 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe0ac0b0270 0x7fe0ac09fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:03.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.641+0000 7fe0b2919700 1 -- 172.21.15.17:0/3724772139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0ac0ac880 msgr2=0x7fe0ac09f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:03.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.641+0000 7fe0b2919700 1 --2- 172.21.15.17:0/3724772139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0ac0ac880 0x7fe0ac09f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:03.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.641+0000 7fe0b2919700 1 -- 172.21.15.17:0/3724772139 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a800b1e0 con 0x7fe0ac0b2710 2022-08-01T19:43:03.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.641+0000 7fe0b2919700 1 --2- 172.21.15.17:0/3724772139 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0ac0b2710 0x7fe0ac14f710 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 rx=0x7fe0ac061c10 tx=0x7fe0a4007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:03.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.645+0000 7fe09b7fe700 1 -- 172.21.15.17:0/3724772139 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0a400f470 con 0x7fe0ac0b2710 2022-08-01T19:43:03.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.645+0000 7fe09b7fe700 1 -- 172.21.15.17:0/3724772139 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0a400f5d0 con 0x7fe0ac0b2710 2022-08-01T19:43:03.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.645+0000 7fe0b311a700 1 -- 172.21.15.17:0/3724772139 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0ac153450 con 0x7fe0ac0b2710 2022-08-01T19:43:03.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.645+0000 7fe09b7fe700 1 -- 172.21.15.17:0/3724772139 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0a400f930 con 0x7fe0ac0b2710 2022-08-01T19:43:03.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.645+0000 7fe0b311a700 1 -- 172.21.15.17:0/3724772139 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0ac143aa0 con 0x7fe0ac0b2710 2022-08-01T19:43:03.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.645+0000 7fe0b311a700 1 -- 172.21.15.17:0/3724772139 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0ac09ef80 con 0x7fe0ac0b2710 2022-08-01T19:43:03.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.645+0000 7fe09b7fe700 1 -- 172.21.15.17:0/3724772139 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe0a4019030 con 0x7fe0ac0b2710 2022-08-01T19:43:03.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.645+0000 7fe09b7fe700 1 --2- 172.21.15.17:0/3724772139 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe08c053950 0x7fe08c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:03.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.645+0000 7fe09b7fe700 1 -- 172.21.15.17:0/3724772139 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe0a406b8c0 con 0x7fe0ac0b2710 2022-08-01T19:43:03.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.645+0000 7fe0b2118700 1 --2- 172.21.15.17:0/3724772139 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe08c053950 0x7fe08c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:03.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.649+0000 7fe0b2118700 1 --2- 172.21.15.17:0/3724772139 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe08c053950 0x7fe08c055e10 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7fe09c00c000 tx=0x7fe09c008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:03.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.653+0000 7fe09b7fe700 1 -- 172.21.15.17:0/3724772139 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe0a4042680 con 0x7fe0ac0b2710 2022-08-01T19:43:03.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.821+0000 7fe0b311a700 1 -- 172.21.15.17:0/3724772139 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0ac0574e0 con 0x7fe0ac0b2710 2022-08-01T19:43:03.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.821+0000 7fe09b7fe700 1 -- 172.21.15.17:0/3724772139 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+2486 (secure 0 0 0) 0x7fe0a4045ca0 con 0x7fe0ac0b2710 2022-08-01T19:43:03.825 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:03.826 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":69,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7995,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.17:6833/4164640472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4164640472},{"type":"v1","addr":"172.21.15.17:6833","nonce":4164640472}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":67},{"gid":8001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/50015801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":50015801},{"type":"v1","addr":"172.21.15.17:6835","nonce":50015801}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:43:01.287477+0000","modified":"2022-08-01T19:43:02.290478+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8007},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8007":{"gid":8007,"name":"b","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6835/299216688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":299216688},{"type":"v1","addr":"172.21.15.77:6835","nonce":299216688}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2022-08-01T19:43:03.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.825+0000 7fe0b311a700 1 -- 172.21.15.17:0/3724772139 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe08c053950 msgr2=0x7fe08c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:03.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.825+0000 7fe0b311a700 1 --2- 172.21.15.17:0/3724772139 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe08c053950 0x7fe08c055e10 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7fe09c00c000 tx=0x7fe09c008040).stop 2022-08-01T19:43:03.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.825+0000 7fe0b311a700 1 -- 172.21.15.17:0/3724772139 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0ac0b2710 msgr2=0x7fe0ac14f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:03.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.825+0000 7fe0b311a700 1 --2- 172.21.15.17:0/3724772139 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0ac0b2710 0x7fe0ac14f710 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 rx=0x7fe0ac061c10 tx=0x7fe0a4007f10).stop 2022-08-01T19:43:03.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.825+0000 7fe0b311a700 1 -- 172.21.15.17:0/3724772139 shutdown_connections 2022-08-01T19:43:03.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.825+0000 7fe0b311a700 1 --2- 172.21.15.17:0/3724772139 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe08c053950 0x7fe08c055e10 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:03.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.825+0000 7fe0b311a700 1 --2- 172.21.15.17:0/3724772139 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0ac0b2710 0x7fe0ac14f710 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:03.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.825+0000 7fe0b311a700 1 --2- 172.21.15.17:0/3724772139 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe0ac0b0270 0x7fe0ac09fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:03.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.825+0000 7fe0b311a700 1 --2- 172.21.15.17:0/3724772139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0ac0ac880 0x7fe0ac09f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:03.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.825+0000 7fe0b311a700 1 -- 172.21.15.17:0/3724772139 >> 172.21.15.17:0/3724772139 conn(0x7fe0ac060310 msgr2=0x7fe0ac0b5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:03.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.825+0000 7fe0b311a700 1 -- 172.21.15.17:0/3724772139 shutdown_connections 2022-08-01T19:43:03.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:03.825+0000 7fe0b311a700 1 -- 172.21.15.17:0/3724772139 wait complete. 2022-08-01T19:43:03.833 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 69 2022-08-01T19:43:04.140 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 69, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:43:01.287477+0000', 'modified': '2022-08-01T19:43:02.290478+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8007}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8007': {'gid': 8007, 'name': 'b', 'rank': 0, 'incarnation': 68, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.77:6835/299216688', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6834', 'nonce': 299216688}, {'type': 'v1', 'addr': '172.21.15.77:6835', 'nonce': 299216688}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:43:04.141 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:43:04.142 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-08-01T19:43:04.259 INFO:teuthology.orchestra.run.smithi077.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":8007,"want_state":"up:active","state":"up:active","rank_uptime":2.9583517000000001,"mdsmap_epoch":69,"osdmap_epoch":137,"osdmap_epoch_barrier":136,"uptime":7.8622308170000004} 2022-08-01T19:43:04.260 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 8007, "mdsmap_epoch": 69, "osdmap_epoch": 137, "osdmap_epoch_barrier": 136, "rank_uptime": 2.9583517, "state": "up:active", "uptime": 7.862230817, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:43:04.260 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:43:04.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.369+0000 7f4e2c83c700 1 Processor -- start 2022-08-01T19:43:04.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.369+0000 7f4e2c83c700 1 -- start start 2022-08-01T19:43:04.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.373+0000 7f4e2c83c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4e280ad4b0 0x7f4e280af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:04.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.373+0000 7f4e2c83c700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e280aff30 con 0x7f4e280ad9e0 2022-08-01T19:43:04.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.373+0000 7f4e2c83c700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e2805c7a0 con 0x7f4e280af450 2022-08-01T19:43:04.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.373+0000 7f4e2c83c700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e28140990 con 0x7f4e280ad4b0 2022-08-01T19:43:04.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.373+0000 7f4e2759e700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f4e280ad9e0 0x7f4e280a8e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36504/0 (socket says 172.21.15.17:36504) 2022-08-01T19:43:04.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.373+0000 7f4e2659c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4e280ad4b0 0x7f4e280af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:04.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.373+0000 7f4e2759e700 1 -- 172.21.15.17:0/683097740 learned_addr learned my addr 172.21.15.17:0/683097740 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:04.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.373+0000 7f4e25d9b700 1 -- 172.21.15.17:0/683097740 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 619620138 0 0) 0x7f4e280aff30 con 0x7f4e280ad9e0 2022-08-01T19:43:04.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.373+0000 7f4e25d9b700 1 -- 172.21.15.17:0/683097740 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e0c003260 con 0x7f4e280ad9e0 2022-08-01T19:43:04.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.373+0000 7f4e25d9b700 1 -- 172.21.15.17:0/683097740 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1121476230 0 0) 0x7f4e2805c7a0 con 0x7f4e280af450 2022-08-01T19:43:04.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.373+0000 7f4e25d9b700 1 -- 172.21.15.17:0/683097740 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e280aff30 con 0x7f4e280af450 2022-08-01T19:43:04.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.373+0000 7f4e2659c700 1 -- 172.21.15.17:0/683097740 >> v1:172.21.15.77:6789/0 conn(0x7f4e280af450 legacy=0x7f4e281432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:04.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.373+0000 7f4e2659c700 1 -- 172.21.15.17:0/683097740 >> v1:172.21.15.17:6789/0 conn(0x7f4e280ad9e0 legacy=0x7f4e280a8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:04.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.373+0000 7f4e25d9b700 1 -- 172.21.15.17:0/683097740 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3956855185 0 0) 0x7f4e0c003260 con 0x7f4e280ad9e0 2022-08-01T19:43:04.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.373+0000 7f4e2659c700 1 -- 172.21.15.17:0/683097740 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e281439f0 con 0x7f4e280ad4b0 2022-08-01T19:43:04.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.373+0000 7f4e2659c700 1 --2- 172.21.15.17:0/683097740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e280ad4b0 0x7f4e280af830 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 rx=0x7f4e280ad900 tx=0x7f4e1c006df0).ready entity=mon.0 client_cookie=a356ce37a2a0b747 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:04.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.373+0000 7f4e25d9b700 1 -- 172.21.15.17:0/683097740 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e1c00d610 con 0x7f4e280ad4b0 2022-08-01T19:43:04.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.373+0000 7f4e25d9b700 1 -- 172.21.15.17:0/683097740 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e1c000e90 con 0x7f4e280ad4b0 2022-08-01T19:43:04.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.373+0000 7f4e25d9b700 1 -- 172.21.15.17:0/683097740 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e1c00d960 con 0x7f4e280ad4b0 2022-08-01T19:43:04.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.373+0000 7f4e2c83c700 1 -- 172.21.15.17:0/683097740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e280ad4b0 msgr2=0x7f4e280af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:04.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.373+0000 7f4e2c83c700 1 --2- 172.21.15.17:0/683097740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e280ad4b0 0x7f4e280af830 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 rx=0x7f4e280ad900 tx=0x7f4e1c006df0).stop 2022-08-01T19:43:04.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.373+0000 7f4e2c83c700 1 -- 172.21.15.17:0/683097740 shutdown_connections 2022-08-01T19:43:04.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.373+0000 7f4e2c83c700 1 --2- 172.21.15.17:0/683097740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e280ad4b0 0x7f4e280af830 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:04.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.373+0000 7f4e2c83c700 1 -- 172.21.15.17:0/683097740 >> 172.21.15.17:0/683097740 conn(0x7f4e280612d0 msgr2=0x7f4e28062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:04.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.373+0000 7f4e2c83c700 1 -- 172.21.15.17:0/683097740 shutdown_connections 2022-08-01T19:43:04.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.373+0000 7f4e2c83c700 1 -- 172.21.15.17:0/683097740 wait complete. 2022-08-01T19:43:04.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.377+0000 7f4e2c83c700 1 Processor -- start 2022-08-01T19:43:04.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.377+0000 7f4e2c83c700 1 -- start start 2022-08-01T19:43:04.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.377+0000 7f4e2c83c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e280ad4b0 0x7f4e280b6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:04.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.377+0000 7f4e2c83c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e280ad9e0 0x7f4e280abe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:04.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.377+0000 7f4e2c83c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4e280af450 0x7f4e28065740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:04.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.377+0000 7f4e2c83c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e28064e40 con 0x7f4e280ad9e0 2022-08-01T19:43:04.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.377+0000 7f4e2c83c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e280b4aa0 con 0x7f4e280ad4b0 2022-08-01T19:43:04.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.377+0000 7f4e2c83c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e280b5940 con 0x7f4e280af450 2022-08-01T19:43:04.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.377+0000 7f4e2659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e280ad9e0 0x7f4e280abe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:04.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.377+0000 7f4e2659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e280ad9e0 0x7f4e280abe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44312/0 (socket says 172.21.15.17:44312) 2022-08-01T19:43:04.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.377+0000 7f4e2659c700 1 -- 172.21.15.17:0/2984502138 learned_addr learned my addr 172.21.15.17:0/2984502138 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:04.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.377+0000 7f4e26d9d700 1 --2- 172.21.15.17:0/2984502138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e280ad4b0 0x7f4e280b6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:04.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.377+0000 7f4e2759e700 1 --2- 172.21.15.17:0/2984502138 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4e280af450 0x7f4e28065740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:04.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.377+0000 7f4e2659c700 1 -- 172.21.15.17:0/2984502138 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4e280af450 msgr2=0x7f4e28065740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:04.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.377+0000 7f4e2659c700 1 --2- 172.21.15.17:0/2984502138 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4e280af450 0x7f4e28065740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:04.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.377+0000 7f4e2659c700 1 -- 172.21.15.17:0/2984502138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e280ad4b0 msgr2=0x7f4e280b6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:04.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.377+0000 7f4e2659c700 1 --2- 172.21.15.17:0/2984502138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e280ad4b0 0x7f4e280b6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:04.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.377+0000 7f4e2659c700 1 -- 172.21.15.17:0/2984502138 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e1c00b180 con 0x7f4e280ad9e0 2022-08-01T19:43:04.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.377+0000 7f4e2759e700 1 --2- 172.21.15.17:0/2984502138 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4e280af450 0x7f4e28065740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:43:04.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.377+0000 7f4e2659c700 1 --2- 172.21.15.17:0/2984502138 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e280ad9e0 0x7f4e280abe40 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 rx=0x7f4e280ad900 tx=0x7f4e1c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:04.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.377+0000 7f4e17fff700 1 -- 172.21.15.17:0/2984502138 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e1c013e10 con 0x7f4e280ad9e0 2022-08-01T19:43:04.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.377+0000 7f4e17fff700 1 -- 172.21.15.17:0/2984502138 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e1c00d290 con 0x7f4e280ad9e0 2022-08-01T19:43:04.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.377+0000 7f4e2c83c700 1 -- 172.21.15.17:0/2984502138 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e280b4c90 con 0x7f4e280ad9e0 2022-08-01T19:43:04.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.377+0000 7f4e17fff700 1 -- 172.21.15.17:0/2984502138 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e1c01e5f0 con 0x7f4e280ad9e0 2022-08-01T19:43:04.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.377+0000 7f4e2c83c700 1 -- 172.21.15.17:0/2984502138 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e28153a00 con 0x7f4e280ad9e0 2022-08-01T19:43:04.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.381+0000 7f4e2c83c700 1 -- 172.21.15.17:0/2984502138 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e280574e0 con 0x7f4e280ad9e0 2022-08-01T19:43:04.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.381+0000 7f4e17fff700 1 -- 172.21.15.17:0/2984502138 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4e1c025030 con 0x7f4e280ad9e0 2022-08-01T19:43:04.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.381+0000 7f4e17fff700 1 --2- 172.21.15.17:0/2984502138 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4e04053a30 0x7f4e04055ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:04.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.381+0000 7f4e17fff700 1 -- 172.21.15.17:0/2984502138 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f4e1c06cec0 con 0x7f4e280ad9e0 2022-08-01T19:43:04.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.381+0000 7f4e26d9d700 1 --2- 172.21.15.17:0/2984502138 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4e04053a30 0x7f4e04055ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:04.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.381+0000 7f4e26d9d700 1 --2- 172.21.15.17:0/2984502138 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4e04053a30 0x7f4e04055ef0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f4e1800d000 tx=0x7f4e18007a20).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:04.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.385+0000 7f4e17fff700 1 -- 172.21.15.17:0/2984502138 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4e1c043d00 con 0x7f4e280ad9e0 2022-08-01T19:43:04.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.509+0000 7f4e2c83c700 1 -- 172.21.15.17:0/2984502138 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e280b5130 con 0x7f4e280ad9e0 2022-08-01T19:43:04.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.513+0000 7f4e17fff700 1 -- 172.21.15.17:0/2984502138 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f4e1c0472a0 con 0x7f4e280ad9e0 2022-08-01T19:43:04.517 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:04.518 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":137,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:43:02.291265+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:42:59.166414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2022-08-01T19:43:00.301184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:43:04.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.513+0000 7f4e2c83c700 1 -- 172.21.15.17:0/2984502138 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4e04053a30 msgr2=0x7f4e04055ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:04.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.513+0000 7f4e2c83c700 1 --2- 172.21.15.17:0/2984502138 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4e04053a30 0x7f4e04055ef0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f4e1800d000 tx=0x7f4e18007a20).stop 2022-08-01T19:43:04.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.513+0000 7f4e2c83c700 1 -- 172.21.15.17:0/2984502138 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e280ad9e0 msgr2=0x7f4e280abe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:04.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.513+0000 7f4e2c83c700 1 --2- 172.21.15.17:0/2984502138 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e280ad9e0 0x7f4e280abe40 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 rx=0x7f4e280ad900 tx=0x7f4e1c014000).stop 2022-08-01T19:43:04.524 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.513+0000 7f4e2c83c700 1 -- 172.21.15.17:0/2984502138 shutdown_connections 2022-08-01T19:43:04.525 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.513+0000 7f4e2c83c700 1 --2- 172.21.15.17:0/2984502138 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4e04053a30 0x7f4e04055ef0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:04.525 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.513+0000 7f4e2c83c700 1 --2- 172.21.15.17:0/2984502138 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4e280af450 0x7f4e28065740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:04.526 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.513+0000 7f4e2c83c700 1 --2- 172.21.15.17:0/2984502138 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e280ad9e0 0x7f4e280abe40 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:04.526 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.513+0000 7f4e2c83c700 1 --2- 172.21.15.17:0/2984502138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e280ad4b0 0x7f4e280b6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:04.526 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.513+0000 7f4e2c83c700 1 -- 172.21.15.17:0/2984502138 >> 172.21.15.17:0/2984502138 conn(0x7f4e280612d0 msgr2=0x7f4e280a8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:04.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.513+0000 7f4e2c83c700 1 -- 172.21.15.17:0/2984502138 shutdown_connections 2022-08-01T19:43:04.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.513+0000 7f4e2c83c700 1 -- 172.21.15.17:0/2984502138 wait complete. 2022-08-01T19:43:04.529 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T19:42:59.166414+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T19:43:00.301184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T19:43:04.529 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:43:04.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.637+0000 7f921e376700 1 Processor -- start 2022-08-01T19:43:04.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.637+0000 7f921e376700 1 -- start start 2022-08-01T19:43:04.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.641+0000 7f921e376700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f9218146ae0 0x7f92181403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:04.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.641+0000 7f921e376700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f92180b4820 0x7f92180b5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:04.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.641+0000 7f921e376700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9218143cf0 con 0x7f92180b4310 2022-08-01T19:43:04.645 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.641+0000 7f921e376700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92181476a0 con 0x7f9218146ae0 2022-08-01T19:43:04.645 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.641+0000 7f921e376700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9218143f90 con 0x7f92180b4820 2022-08-01T19:43:04.645 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.641+0000 7f921c913700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f92180b4820 0x7f92180b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:04.645 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.641+0000 7f9217fff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f9218146ae0 0x7f92181403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:04.646 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.641+0000 7f921c913700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f92180b4820 0x7f92180b5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:40420/0 (socket says 172.21.15.17:40420) 2022-08-01T19:43:04.646 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.641+0000 7f921c913700 1 -- 172.21.15.17:0/2202709066 learned_addr learned my addr 172.21.15.17:0/2202709066 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:04.646 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.641+0000 7f92177fe700 1 --1- 172.21.15.17:0/2202709066 >> v1:172.21.15.77:6789/0 conn(0x7f92180b4310 0x7f92181408b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:53314/0 (socket says 172.21.15.17:53314) 2022-08-01T19:43:04.646 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.641+0000 7f921c913700 1 -- 172.21.15.17:0/2202709066 >> v2:172.21.15.77:3300/0 conn(0x7f9218146ae0 msgr2=0x7f92181403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:04.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.641+0000 7f921c913700 1 --2- 172.21.15.17:0/2202709066 >> v2:172.21.15.77:3300/0 conn(0x7f9218146ae0 0x7f92181403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:04.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.641+0000 7f921c913700 1 -- 172.21.15.17:0/2202709066 >> v1:172.21.15.77:6789/0 conn(0x7f92180b4310 legacy=0x7f92181408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:04.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.641+0000 7f921c913700 1 -- 172.21.15.17:0/2202709066 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92180b0e20 con 0x7f92180b4820 2022-08-01T19:43:04.650 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.641+0000 7f921c913700 1 --2- 172.21.15.17:0/2202709066 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f92180b4820 0x7f92180b5060 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f92100032f0 tx=0x7f9210009290).ready entity=mon.2 client_cookie=72813be3bb02b99b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:04.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.641+0000 7f9216ffd700 1 -- 172.21.15.17:0/2202709066 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f921000b8a0 con 0x7f92180b4820 2022-08-01T19:43:04.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.641+0000 7f9216ffd700 1 -- 172.21.15.17:0/2202709066 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f921000ba00 con 0x7f92180b4820 2022-08-01T19:43:04.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.641+0000 7f9216ffd700 1 -- 172.21.15.17:0/2202709066 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f921000bb70 con 0x7f92180b4820 2022-08-01T19:43:04.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.641+0000 7f921e376700 1 -- 172.21.15.17:0/2202709066 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f92180b4820 msgr2=0x7f92180b5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:04.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.641+0000 7f921e376700 1 --2- 172.21.15.17:0/2202709066 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f92180b4820 0x7f92180b5060 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f92100032f0 tx=0x7f9210009290).stop 2022-08-01T19:43:04.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.641+0000 7f921e376700 1 -- 172.21.15.17:0/2202709066 shutdown_connections 2022-08-01T19:43:04.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.641+0000 7f921e376700 1 --2- 172.21.15.17:0/2202709066 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f92180b4820 0x7f92180b5060 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:04.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.641+0000 7f921e376700 1 --2- 172.21.15.17:0/2202709066 >> v2:172.21.15.77:3300/0 conn(0x7f9218146ae0 0x7f92181403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:04.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.641+0000 7f921e376700 1 -- 172.21.15.17:0/2202709066 >> 172.21.15.17:0/2202709066 conn(0x7f921805c620 msgr2=0x7f921805ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:04.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.641+0000 7f921e376700 1 -- 172.21.15.17:0/2202709066 shutdown_connections 2022-08-01T19:43:04.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.641+0000 7f921e376700 1 -- 172.21.15.17:0/2202709066 wait complete. 2022-08-01T19:43:04.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.645+0000 7f921e376700 1 Processor -- start 2022-08-01T19:43:04.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.645+0000 7f921e376700 1 -- start start 2022-08-01T19:43:04.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.645+0000 7f921e376700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92180b4310 0x7f9218152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:04.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.645+0000 7f921e376700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f92180b4820 0x7f92180b9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:04.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.645+0000 7f921e376700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9218146ae0 0x7f92180ba000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:04.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.645+0000 7f921e376700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92180b8150 con 0x7f9218146ae0 2022-08-01T19:43:04.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.645+0000 7f921e376700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92180b7c10 con 0x7f92180b4310 2022-08-01T19:43:04.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.645+0000 7f921e376700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92180b7eb0 con 0x7f92180b4820 2022-08-01T19:43:04.660 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.645+0000 7f921c913700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9218146ae0 0x7f92180ba000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:04.660 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.645+0000 7f921c913700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9218146ae0 0x7f92180ba000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44330/0 (socket says 172.21.15.17:44330) 2022-08-01T19:43:04.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.645+0000 7f921c913700 1 -- 172.21.15.17:0/488796173 learned_addr learned my addr 172.21.15.17:0/488796173 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:04.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.645+0000 7f9217fff700 1 --2- 172.21.15.17:0/488796173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92180b4310 0x7f9218152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:04.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.645+0000 7f92177fe700 1 --2- 172.21.15.17:0/488796173 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f92180b4820 0x7f92180b9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:04.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.645+0000 7f921c913700 1 -- 172.21.15.17:0/488796173 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f92180b4820 msgr2=0x7f92180b9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:04.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.645+0000 7f921c913700 1 --2- 172.21.15.17:0/488796173 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f92180b4820 0x7f92180b9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:04.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.645+0000 7f921c913700 1 -- 172.21.15.17:0/488796173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92180b4310 msgr2=0x7f9218152fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:04.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.645+0000 7f921c913700 1 --2- 172.21.15.17:0/488796173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92180b4310 0x7f9218152fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:04.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.645+0000 7f921c913700 1 -- 172.21.15.17:0/488796173 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9210009530 con 0x7f9218146ae0 2022-08-01T19:43:04.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.645+0000 7f921c913700 1 --2- 172.21.15.17:0/488796173 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9218146ae0 0x7f92180ba000 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 rx=0x7f92100032f0 tx=0x7f9210016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:04.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.649+0000 7f92157fa700 1 -- 172.21.15.17:0/488796173 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f921000b8a0 con 0x7f9218146ae0 2022-08-01T19:43:04.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.649+0000 7f92157fa700 1 -- 172.21.15.17:0/488796173 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9210021440 con 0x7f9218146ae0 2022-08-01T19:43:04.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.649+0000 7f921e376700 1 -- 172.21.15.17:0/488796173 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92181460a0 con 0x7f9218146ae0 2022-08-01T19:43:04.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.649+0000 7f92157fa700 1 -- 172.21.15.17:0/488796173 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92100217a0 con 0x7f9218146ae0 2022-08-01T19:43:04.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.649+0000 7f921e376700 1 -- 172.21.15.17:0/488796173 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92180aeb70 con 0x7f9218146ae0 2022-08-01T19:43:04.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.649+0000 7f921e376700 1 -- 172.21.15.17:0/488796173 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92180574e0 con 0x7f9218146ae0 2022-08-01T19:43:04.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.649+0000 7f92157fa700 1 -- 172.21.15.17:0/488796173 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9210027020 con 0x7f9218146ae0 2022-08-01T19:43:04.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.649+0000 7f92157fa700 1 --2- 172.21.15.17:0/488796173 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f91fc053a70 0x7f91fc055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:04.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.649+0000 7f92157fa700 1 -- 172.21.15.17:0/488796173 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f921006d490 con 0x7f9218146ae0 2022-08-01T19:43:04.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.649+0000 7f9217fff700 1 --2- 172.21.15.17:0/488796173 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f91fc053a70 0x7f91fc055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:04.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.649+0000 7f9217fff700 1 --2- 172.21.15.17:0/488796173 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f91fc053a70 0x7f91fc055f30 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f920000b000 tx=0x7f9200006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:04.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.653+0000 7f92157fa700 1 -- 172.21.15.17:0/488796173 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f92100441b0 con 0x7f9218146ae0 2022-08-01T19:43:04.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.781+0000 7f921e376700 1 -- 172.21.15.17:0/488796173 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f92180b6dd0 con 0x7f9218146ae0 2022-08-01T19:43:04.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.781+0000 7f92157fa700 1 -- 172.21.15.17:0/488796173 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v137) v1 ==== 110+0+10 (secure 0 0 0) 0x7f9210015020 con 0x7f9218146ae0 2022-08-01T19:43:04.786 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:43:04.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.781+0000 7f921e376700 1 -- 172.21.15.17:0/488796173 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f91fc053a70 msgr2=0x7f91fc055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:04.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.781+0000 7f921e376700 1 --2- 172.21.15.17:0/488796173 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f91fc053a70 0x7f91fc055f30 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f920000b000 tx=0x7f9200006df0).stop 2022-08-01T19:43:04.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.781+0000 7f921e376700 1 -- 172.21.15.17:0/488796173 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9218146ae0 msgr2=0x7f92180ba000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:04.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.781+0000 7f921e376700 1 --2- 172.21.15.17:0/488796173 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9218146ae0 0x7f92180ba000 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 rx=0x7f92100032f0 tx=0x7f9210016000).stop 2022-08-01T19:43:04.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.781+0000 7f921e376700 1 -- 172.21.15.17:0/488796173 shutdown_connections 2022-08-01T19:43:04.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.781+0000 7f921e376700 1 --2- 172.21.15.17:0/488796173 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f91fc053a70 0x7f91fc055f30 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:04.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.781+0000 7f921e376700 1 --2- 172.21.15.17:0/488796173 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9218146ae0 0x7f92180ba000 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:04.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.781+0000 7f921e376700 1 --2- 172.21.15.17:0/488796173 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f92180b4820 0x7f92180b9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:04.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.781+0000 7f921e376700 1 --2- 172.21.15.17:0/488796173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92180b4310 0x7f9218152fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:04.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.781+0000 7f921e376700 1 -- 172.21.15.17:0/488796173 >> 172.21.15.17:0/488796173 conn(0x7f921805c620 msgr2=0x7f92180b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:04.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.781+0000 7f921e376700 1 -- 172.21.15.17:0/488796173 shutdown_connections 2022-08-01T19:43:04.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.781+0000 7f921e376700 1 -- 172.21.15.17:0/488796173 wait complete. 2022-08-01T19:43:04.810 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:43:04.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.925+0000 7ff0d1d6e700 1 Processor -- start 2022-08-01T19:43:04.929 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.925+0000 7ff0d1d6e700 1 -- start start 2022-08-01T19:43:04.936 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.925+0000 7ff0d1d6e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff0cc0b36e0 0x7ff0cc141a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:04.936 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.925+0000 7ff0d1d6e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ff0cc144e60 0x7ff0cc066ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:04.936 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.925+0000 7ff0d1d6e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0cc064ef0 con 0x7ff0cc064b10 2022-08-01T19:43:04.937 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.925+0000 7ff0d1d6e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0cc13ebf0 con 0x7ff0cc144e60 2022-08-01T19:43:04.937 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.925+0000 7ff0d1d6e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc1452d0 con 0x7ff0cc0b36e0 2022-08-01T19:43:04.938 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.925+0000 7ff0c3fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff0cc0b36e0 0x7ff0cc141a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:04.938 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.925+0000 7ff0d156d700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ff0cc144e60 0x7ff0cc066ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:04.939 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.925+0000 7ff0d0d6c700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7ff0cc064b10 0x7ff0cc141370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:53328/0 (socket says 172.21.15.17:53328) 2022-08-01T19:43:04.939 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.925+0000 7ff0c3fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff0cc0b36e0 0x7ff0cc141a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:40434/0 (socket says 172.21.15.17:40434) 2022-08-01T19:43:04.939 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.925+0000 7ff0d0d6c700 1 -- 172.21.15.17:0/1627141478 learned_addr learned my addr 172.21.15.17:0/1627141478 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:04.940 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.929+0000 7ff0d156d700 1 -- 172.21.15.17:0/1627141478 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff0cc0b36e0 msgr2=0x7ff0cc141a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:04.940 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.929+0000 7ff0d156d700 1 --2- 172.21.15.17:0/1627141478 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff0cc0b36e0 0x7ff0cc141a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:04.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.929+0000 7ff0d156d700 1 -- 172.21.15.17:0/1627141478 >> v1:172.21.15.77:6789/0 conn(0x7ff0cc064b10 legacy=0x7ff0cc141370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:04.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.929+0000 7ff0c37fe700 1 -- 172.21.15.17:0/1627141478 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1338428076 0 0) 0x7ff0cc064ef0 con 0x7ff0cc064b10 2022-08-01T19:43:04.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.929+0000 7ff0d156d700 1 -- 172.21.15.17:0/1627141478 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0cc066690 con 0x7ff0cc144e60 2022-08-01T19:43:04.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.929+0000 7ff0d156d700 1 --2- 172.21.15.17:0/1627141478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0cc144e60 0x7ff0cc066ac0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7ff0c40032f0 tx=0x7ff0c40091d0).ready entity=mon.1 client_cookie=1b48fdfd3d12816b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:04.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.929+0000 7ff0c37fe700 1 -- 172.21.15.17:0/1627141478 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0c400b6c0 con 0x7ff0cc144e60 2022-08-01T19:43:04.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.929+0000 7ff0c37fe700 1 -- 172.21.15.17:0/1627141478 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c400b820 con 0x7ff0cc144e60 2022-08-01T19:43:04.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.929+0000 7ff0c37fe700 1 -- 172.21.15.17:0/1627141478 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0c400b990 con 0x7ff0cc144e60 2022-08-01T19:43:04.944 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.929+0000 7ff0d1d6e700 1 -- 172.21.15.17:0/1627141478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0cc144e60 msgr2=0x7ff0cc066ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:04.944 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.929+0000 7ff0d1d6e700 1 --2- 172.21.15.17:0/1627141478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0cc144e60 0x7ff0cc066ac0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7ff0c40032f0 tx=0x7ff0c40091d0).stop 2022-08-01T19:43:04.945 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.929+0000 7ff0d1d6e700 1 -- 172.21.15.17:0/1627141478 shutdown_connections 2022-08-01T19:43:04.945 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.929+0000 7ff0d1d6e700 1 --2- 172.21.15.17:0/1627141478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0cc144e60 0x7ff0cc066ac0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:04.945 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.929+0000 7ff0d1d6e700 1 --2- 172.21.15.17:0/1627141478 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff0cc0b36e0 0x7ff0cc141a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:04.946 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.929+0000 7ff0d1d6e700 1 -- 172.21.15.17:0/1627141478 >> 172.21.15.17:0/1627141478 conn(0x7ff0cc05f840 msgr2=0x7ff0cc05fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:04.946 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.929+0000 7ff0d1d6e700 1 -- 172.21.15.17:0/1627141478 shutdown_connections 2022-08-01T19:43:04.947 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.929+0000 7ff0d1d6e700 1 -- 172.21.15.17:0/1627141478 wait complete. 2022-08-01T19:43:04.947 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.929+0000 7ff0d1d6e700 1 Processor -- start 2022-08-01T19:43:04.947 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.929+0000 7ff0d1d6e700 1 -- start start 2022-08-01T19:43:04.948 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.933+0000 7ff0d1d6e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0cc064b10 0x7ff0cc13ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:04.954 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.933+0000 7ff0d1d6e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0cc0b36e0 0x7ff0cc13f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:04.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.933+0000 7ff0d1d6e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff0cc144e60 0x7ff0cc1537d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:04.955 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.933+0000 7ff0d1d6e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc150f30 con 0x7ff0cc0b36e0 2022-08-01T19:43:04.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.933+0000 7ff0d1d6e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc1511d0 con 0x7ff0cc064b10 2022-08-01T19:43:04.956 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.933+0000 7ff0d1d6e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc13c2e0 con 0x7ff0cc144e60 2022-08-01T19:43:04.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.933+0000 7ff0c3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0cc0b36e0 0x7ff0cc13f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:04.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.933+0000 7ff0c3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0cc0b36e0 0x7ff0cc13f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44342/0 (socket says 172.21.15.17:44342) 2022-08-01T19:43:04.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.933+0000 7ff0c3fff700 1 -- 172.21.15.17:0/2544852626 learned_addr learned my addr 172.21.15.17:0/2544852626 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:04.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.933+0000 7ff0d0d6c700 1 --2- 172.21.15.17:0/2544852626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0cc064b10 0x7ff0cc13ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:04.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.933+0000 7ff0d156d700 1 --2- 172.21.15.17:0/2544852626 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff0cc144e60 0x7ff0cc1537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:04.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.933+0000 7ff0c3fff700 1 -- 172.21.15.17:0/2544852626 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff0cc144e60 msgr2=0x7ff0cc1537d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:04.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.933+0000 7ff0c3fff700 1 --2- 172.21.15.17:0/2544852626 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff0cc144e60 0x7ff0cc1537d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:04.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.933+0000 7ff0c3fff700 1 -- 172.21.15.17:0/2544852626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0cc064b10 msgr2=0x7ff0cc13ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:04.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.933+0000 7ff0c3fff700 1 --2- 172.21.15.17:0/2544852626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0cc064b10 0x7ff0cc13ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:04.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.933+0000 7ff0c3fff700 1 -- 172.21.15.17:0/2544852626 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c4009470 con 0x7ff0cc0b36e0 2022-08-01T19:43:04.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.933+0000 7ff0c3fff700 1 --2- 172.21.15.17:0/2544852626 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0cc0b36e0 0x7ff0cc13f5c0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 rx=0x7ff0bc00c000 tx=0x7ff0bc006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:04.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.933+0000 7ff0c1ffb700 1 -- 172.21.15.17:0/2544852626 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0bc010780 con 0x7ff0cc0b36e0 2022-08-01T19:43:04.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.933+0000 7ff0c1ffb700 1 -- 172.21.15.17:0/2544852626 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0bc0108e0 con 0x7ff0cc0b36e0 2022-08-01T19:43:04.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.933+0000 7ff0d1d6e700 1 -- 172.21.15.17:0/2544852626 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0cc0af830 con 0x7ff0cc0b36e0 2022-08-01T19:43:04.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.933+0000 7ff0c1ffb700 1 -- 172.21.15.17:0/2544852626 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0bc010c40 con 0x7ff0cc0b36e0 2022-08-01T19:43:04.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.933+0000 7ff0d1d6e700 1 -- 172.21.15.17:0/2544852626 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0cc0ae980 con 0x7ff0cc0b36e0 2022-08-01T19:43:04.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.937+0000 7ff0d1d6e700 1 -- 172.21.15.17:0/2544852626 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0cc0b4850 con 0x7ff0cc0b36e0 2022-08-01T19:43:04.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.937+0000 7ff0c1ffb700 1 -- 172.21.15.17:0/2544852626 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff0bc01c460 con 0x7ff0cc0b36e0 2022-08-01T19:43:04.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.937+0000 7ff0c1ffb700 1 --2- 172.21.15.17:0/2544852626 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff0ac0539e0 0x7ff0ac055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:04.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.937+0000 7ff0c1ffb700 1 -- 172.21.15.17:0/2544852626 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff0bc06c510 con 0x7ff0cc0b36e0 2022-08-01T19:43:04.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.937+0000 7ff0d0d6c700 1 --2- 172.21.15.17:0/2544852626 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff0ac0539e0 0x7ff0ac055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:04.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.937+0000 7ff0d0d6c700 1 --2- 172.21.15.17:0/2544852626 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff0ac0539e0 0x7ff0ac055ea0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7ff0c800d000 tx=0x7ff0c8009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:04.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:04.941+0000 7ff0c1ffb700 1 -- 172.21.15.17:0/2544852626 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff0bc043160 con 0x7ff0cc0b36e0 2022-08-01T19:43:05.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.069+0000 7ff0d1d6e700 1 -- 172.21.15.17:0/2544852626 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff0cc0574e0 con 0x7ff0cc0b36e0 2022-08-01T19:43:05.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.069+0000 7ff0c1ffb700 1 -- 172.21.15.17:0/2544852626 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v137) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff0bc046780 con 0x7ff0cc0b36e0 2022-08-01T19:43:05.074 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:43:05.077 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.073+0000 7ff0d1d6e700 1 -- 172.21.15.17:0/2544852626 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff0ac0539e0 msgr2=0x7ff0ac055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:05.077 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.073+0000 7ff0d1d6e700 1 --2- 172.21.15.17:0/2544852626 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff0ac0539e0 0x7ff0ac055ea0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7ff0c800d000 tx=0x7ff0c8009040).stop 2022-08-01T19:43:05.078 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.073+0000 7ff0d1d6e700 1 -- 172.21.15.17:0/2544852626 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0cc0b36e0 msgr2=0x7ff0cc13f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:05.078 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.073+0000 7ff0d1d6e700 1 --2- 172.21.15.17:0/2544852626 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0cc0b36e0 0x7ff0cc13f5c0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 rx=0x7ff0bc00c000 tx=0x7ff0bc006df0).stop 2022-08-01T19:43:05.079 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.073+0000 7ff0d1d6e700 1 -- 172.21.15.17:0/2544852626 shutdown_connections 2022-08-01T19:43:05.079 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.073+0000 7ff0d1d6e700 1 --2- 172.21.15.17:0/2544852626 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff0ac0539e0 0x7ff0ac055ea0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:05.080 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.073+0000 7ff0d1d6e700 1 --2- 172.21.15.17:0/2544852626 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff0cc144e60 0x7ff0cc1537d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:05.080 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.073+0000 7ff0d1d6e700 1 --2- 172.21.15.17:0/2544852626 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0cc0b36e0 0x7ff0cc13f5c0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:05.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.073+0000 7ff0d1d6e700 1 --2- 172.21.15.17:0/2544852626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0cc064b10 0x7ff0cc13ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:05.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.073+0000 7ff0d1d6e700 1 -- 172.21.15.17:0/2544852626 >> 172.21.15.17:0/2544852626 conn(0x7ff0cc05f840 msgr2=0x7ff0cc0b6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:05.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.073+0000 7ff0d1d6e700 1 -- 172.21.15.17:0/2544852626 shutdown_connections 2022-08-01T19:43:05.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.073+0000 7ff0d1d6e700 1 -- 172.21.15.17:0/2544852626 wait complete. 2022-08-01T19:43:05.086 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T19:43:05.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.189+0000 7fb745489700 1 Processor -- start 2022-08-01T19:43:05.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.189+0000 7fb745489700 1 -- start start 2022-08-01T19:43:05.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.189+0000 7fb745489700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb7400b2050 0x7fb7400b1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:05.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.189+0000 7fb745489700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb7400b0140 0x7fb7400b2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:05.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.189+0000 7fb745489700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7400b2920 con 0x7fb7400a7250 2022-08-01T19:43:05.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.189+0000 7fb745489700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7400b0580 con 0x7fb7400b0140 2022-08-01T19:43:05.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.189+0000 7fb745489700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7400b07e0 con 0x7fb7400b2050 2022-08-01T19:43:05.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.189+0000 7fb73e7fc700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb7400b0140 0x7fb7400b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:05.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.189+0000 7fb73e7fc700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb7400b0140 0x7fb7400b2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44350/0 (socket says 172.21.15.17:44350) 2022-08-01T19:43:05.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.189+0000 7fb73e7fc700 1 -- 172.21.15.17:0/3932660535 learned_addr learned my addr 172.21.15.17:0/3932660535 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:05.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.189+0000 7fb73effd700 1 --2- 172.21.15.17:0/3932660535 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb7400b2050 0x7fb7400b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:05.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.189+0000 7fb73e7fc700 1 -- 172.21.15.17:0/3932660535 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb7400b2050 msgr2=0x7fb7400b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:05.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.189+0000 7fb73e7fc700 1 --2- 172.21.15.17:0/3932660535 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb7400b2050 0x7fb7400b1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:05.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.189+0000 7fb73e7fc700 1 -- 172.21.15.17:0/3932660535 >> v1:172.21.15.77:6789/0 conn(0x7fb7400a7250 legacy=0x7fb7401413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:05.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.189+0000 7fb73e7fc700 1 -- 172.21.15.17:0/3932660535 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7400b0a80 con 0x7fb7400b0140 2022-08-01T19:43:05.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.189+0000 7fb73e7fc700 1 --2- 172.21.15.17:0/3932660535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7400b0140 0x7fb7400b2430 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 rx=0x7fb728002f70 tx=0x7fb728006eb0).ready entity=mon.0 client_cookie=8ab1bce20b0a9923 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:05.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.189+0000 7fb73dffb700 1 -- 172.21.15.17:0/3932660535 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb72800d570 con 0x7fb7400b0140 2022-08-01T19:43:05.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.189+0000 7fb73dffb700 1 -- 172.21.15.17:0/3932660535 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb728000e90 con 0x7fb7400b0140 2022-08-01T19:43:05.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.189+0000 7fb73dffb700 1 -- 172.21.15.17:0/3932660535 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb72800d900 con 0x7fb7400b0140 2022-08-01T19:43:05.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.193+0000 7fb745489700 1 -- 172.21.15.17:0/3932660535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7400b0140 msgr2=0x7fb7400b2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:05.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.193+0000 7fb745489700 1 --2- 172.21.15.17:0/3932660535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7400b0140 0x7fb7400b2430 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 rx=0x7fb728002f70 tx=0x7fb728006eb0).stop 2022-08-01T19:43:05.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.193+0000 7fb745489700 1 -- 172.21.15.17:0/3932660535 shutdown_connections 2022-08-01T19:43:05.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.193+0000 7fb745489700 1 --2- 172.21.15.17:0/3932660535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7400b0140 0x7fb7400b2430 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:05.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.193+0000 7fb745489700 1 --2- 172.21.15.17:0/3932660535 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb7400b2050 0x7fb7400b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:05.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.193+0000 7fb745489700 1 -- 172.21.15.17:0/3932660535 >> 172.21.15.17:0/3932660535 conn(0x7fb74005f840 msgr2=0x7fb74005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:05.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.193+0000 7fb745489700 1 -- 172.21.15.17:0/3932660535 shutdown_connections 2022-08-01T19:43:05.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.193+0000 7fb745489700 1 -- 172.21.15.17:0/3932660535 wait complete. 2022-08-01T19:43:05.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.193+0000 7fb745489700 1 Processor -- start 2022-08-01T19:43:05.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.197+0000 7fb745489700 1 -- start start 2022-08-01T19:43:05.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.197+0000 7fb745489700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb7400a7250 0x7fb7400ab320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:05.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.197+0000 7fb745489700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7400b0140 0x7fb740149d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:05.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.197+0000 7fb745489700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7400b2050 0x7fb74014a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:05.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.197+0000 7fb745489700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7400b5d10 con 0x7fb7400b2050 2022-08-01T19:43:05.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.197+0000 7fb745489700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7400ac130 con 0x7fb7400b0140 2022-08-01T19:43:05.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.197+0000 7fb745489700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb740145320 con 0x7fb7400a7250 2022-08-01T19:43:05.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.197+0000 7fb73f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7400b2050 0x7fb74014a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:05.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.197+0000 7fb73f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7400b2050 0x7fb74014a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44358/0 (socket says 172.21.15.17:44358) 2022-08-01T19:43:05.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.197+0000 7fb73f7fe700 1 -- 172.21.15.17:0/688715619 learned_addr learned my addr 172.21.15.17:0/688715619 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:05.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.197+0000 7fb73e7fc700 1 --2- 172.21.15.17:0/688715619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7400b0140 0x7fb740149d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:05.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.197+0000 7fb73effd700 1 --2- 172.21.15.17:0/688715619 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb7400a7250 0x7fb7400ab320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:05.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.197+0000 7fb73f7fe700 1 -- 172.21.15.17:0/688715619 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb7400a7250 msgr2=0x7fb7400ab320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:05.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.197+0000 7fb73f7fe700 1 --2- 172.21.15.17:0/688715619 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb7400a7250 0x7fb7400ab320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:05.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.197+0000 7fb73f7fe700 1 -- 172.21.15.17:0/688715619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7400b0140 msgr2=0x7fb740149d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:05.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.197+0000 7fb73f7fe700 1 --2- 172.21.15.17:0/688715619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7400b0140 0x7fb740149d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:05.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.197+0000 7fb73f7fe700 1 -- 172.21.15.17:0/688715619 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb72800b1e0 con 0x7fb7400b2050 2022-08-01T19:43:05.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.197+0000 7fb73f7fe700 1 --2- 172.21.15.17:0/688715619 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7400b2050 0x7fb74014a270 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 rx=0x7fb74006a280 tx=0x7fb738007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:05.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.197+0000 7fb737fff700 1 -- 172.21.15.17:0/688715619 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb73800f470 con 0x7fb7400b2050 2022-08-01T19:43:05.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.197+0000 7fb737fff700 1 -- 172.21.15.17:0/688715619 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73800f5d0 con 0x7fb7400b2050 2022-08-01T19:43:05.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.197+0000 7fb745489700 1 -- 172.21.15.17:0/688715619 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7400b5f40 con 0x7fb7400b2050 2022-08-01T19:43:05.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.197+0000 7fb737fff700 1 -- 172.21.15.17:0/688715619 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb73800f930 con 0x7fb7400b2050 2022-08-01T19:43:05.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.197+0000 7fb745489700 1 -- 172.21.15.17:0/688715619 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7400abb70 con 0x7fb7400b2050 2022-08-01T19:43:05.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.197+0000 7fb745489700 1 -- 172.21.15.17:0/688715619 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7400b60d0 con 0x7fb7400b2050 2022-08-01T19:43:05.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.201+0000 7fb737fff700 1 -- 172.21.15.17:0/688715619 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb738019030 con 0x7fb7400b2050 2022-08-01T19:43:05.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.201+0000 7fb737fff700 1 --2- 172.21.15.17:0/688715619 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb724053a70 0x7fb724055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:05.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.201+0000 7fb737fff700 1 -- 172.21.15.17:0/688715619 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb73806b690 con 0x7fb7400b2050 2022-08-01T19:43:05.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.201+0000 7fb73effd700 1 --2- 172.21.15.17:0/688715619 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb724053a70 0x7fb724055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:05.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.201+0000 7fb73effd700 1 --2- 172.21.15.17:0/688715619 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb724053a70 0x7fb724055f30 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7fb73000b000 tx=0x7fb730006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:05.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.205+0000 7fb737fff700 1 -- 172.21.15.17:0/688715619 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb7380423f0 con 0x7fb7400b2050 2022-08-01T19:43:05.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.329+0000 7fb745489700 1 -- 172.21.15.17:0/688715619 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb7400b0d20 con 0x7fb7400b2050 2022-08-01T19:43:05.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.333+0000 7fb737fff700 1 -- 172.21.15.17:0/688715619 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v137) v1 ==== 104+0+10 (secure 0 0 0) 0x7fb738045a10 con 0x7fb7400b2050 2022-08-01T19:43:05.335 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:43:05.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.333+0000 7fb745489700 1 -- 172.21.15.17:0/688715619 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb724053a70 msgr2=0x7fb724055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:05.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.333+0000 7fb745489700 1 --2- 172.21.15.17:0/688715619 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb724053a70 0x7fb724055f30 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7fb73000b000 tx=0x7fb730006df0).stop 2022-08-01T19:43:05.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.333+0000 7fb745489700 1 -- 172.21.15.17:0/688715619 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7400b2050 msgr2=0x7fb74014a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:05.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.333+0000 7fb745489700 1 --2- 172.21.15.17:0/688715619 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7400b2050 0x7fb74014a270 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 rx=0x7fb74006a280 tx=0x7fb738007f40).stop 2022-08-01T19:43:05.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.333+0000 7fb745489700 1 -- 172.21.15.17:0/688715619 shutdown_connections 2022-08-01T19:43:05.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.333+0000 7fb745489700 1 --2- 172.21.15.17:0/688715619 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb724053a70 0x7fb724055f30 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:05.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.333+0000 7fb745489700 1 --2- 172.21.15.17:0/688715619 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7400b2050 0x7fb74014a270 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:05.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.333+0000 7fb745489700 1 --2- 172.21.15.17:0/688715619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7400b0140 0x7fb740149d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:05.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.333+0000 7fb745489700 1 --2- 172.21.15.17:0/688715619 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb7400a7250 0x7fb7400ab320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:05.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.333+0000 7fb745489700 1 -- 172.21.15.17:0/688715619 >> 172.21.15.17:0/688715619 conn(0x7fb74005f840 msgr2=0x7fb74013f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:05.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.333+0000 7fb745489700 1 -- 172.21.15.17:0/688715619 shutdown_connections 2022-08-01T19:43:05.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.333+0000 7fb745489700 1 -- 172.21.15.17:0/688715619 wait complete. 2022-08-01T19:43:05.349 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T19:43:05.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.457+0000 7f40fa073700 1 Processor -- start 2022-08-01T19:43:05.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.457+0000 7f40fa073700 1 -- start start 2022-08-01T19:43:05.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.461+0000 7f40fa073700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f40f40b7500 0x7f40f40b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:05.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.461+0000 7f40fa073700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f40f40aadb0 0x7f40f4064d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:05.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.461+0000 7f40fa073700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40f40b2510 con 0x7f40f40b31f0 2022-08-01T19:43:05.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.461+0000 7f40fa073700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40f41443e0 con 0x7f40f40aadb0 2022-08-01T19:43:05.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.461+0000 7f40fa073700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f40b3700 con 0x7f40f40b7500 2022-08-01T19:43:05.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.461+0000 7f40f9872700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f40f40aadb0 0x7f40f4064d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:05.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.461+0000 7f40f9872700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f40f40aadb0 0x7f40f4064d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44366/0 (socket says 172.21.15.17:44366) 2022-08-01T19:43:05.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.461+0000 7f40f9872700 1 -- 172.21.15.17:0/2918829306 learned_addr learned my addr 172.21.15.17:0/2918829306 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:05.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.461+0000 7f40f9071700 1 --2- 172.21.15.17:0/2918829306 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f40f40b7500 0x7f40f40b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:05.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.461+0000 7f40e3fff700 1 -- 172.21.15.17:0/2918829306 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2480611166 0 0) 0x7f40f40b2510 con 0x7f40f40b31f0 2022-08-01T19:43:05.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.461+0000 7f40f9872700 1 -- 172.21.15.17:0/2918829306 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f40f40b7500 msgr2=0x7f40f40b78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:05.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.461+0000 7f40f9872700 1 --2- 172.21.15.17:0/2918829306 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f40f40b7500 0x7f40f40b78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:05.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.461+0000 7f40f9872700 1 -- 172.21.15.17:0/2918829306 >> v1:172.21.15.77:6789/0 conn(0x7f40f40b31f0 legacy=0x7f40f413e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:05.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.461+0000 7f40f9872700 1 -- 172.21.15.17:0/2918829306 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f40b2740 con 0x7f40f40aadb0 2022-08-01T19:43:05.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.461+0000 7f40f9872700 1 --2- 172.21.15.17:0/2918829306 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40f40aadb0 0x7f40f4064d30 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 rx=0x7f40ec0032f0 tx=0x7f40ec009290).ready entity=mon.0 client_cookie=7ba676aaecd99582 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:05.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.461+0000 7f40e3fff700 1 -- 172.21.15.17:0/2918829306 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40ec00b8a0 con 0x7f40f40aadb0 2022-08-01T19:43:05.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.461+0000 7f40e3fff700 1 -- 172.21.15.17:0/2918829306 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40ec00ba00 con 0x7f40f40aadb0 2022-08-01T19:43:05.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.461+0000 7f40e3fff700 1 -- 172.21.15.17:0/2918829306 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40ec00bb70 con 0x7f40f40aadb0 2022-08-01T19:43:05.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.461+0000 7f40fa073700 1 -- 172.21.15.17:0/2918829306 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40f40aadb0 msgr2=0x7f40f4064d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:05.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.461+0000 7f40fa073700 1 --2- 172.21.15.17:0/2918829306 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40f40aadb0 0x7f40f4064d30 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 rx=0x7f40ec0032f0 tx=0x7f40ec009290).stop 2022-08-01T19:43:05.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.461+0000 7f40fa073700 1 -- 172.21.15.17:0/2918829306 shutdown_connections 2022-08-01T19:43:05.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.461+0000 7f40fa073700 1 --2- 172.21.15.17:0/2918829306 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40f40aadb0 0x7f40f4064d30 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:05.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.461+0000 7f40fa073700 1 --2- 172.21.15.17:0/2918829306 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f40f40b7500 0x7f40f40b78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:05.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.461+0000 7f40fa073700 1 -- 172.21.15.17:0/2918829306 >> 172.21.15.17:0/2918829306 conn(0x7f40f405f840 msgr2=0x7f40f405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:05.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.461+0000 7f40fa073700 1 -- 172.21.15.17:0/2918829306 shutdown_connections 2022-08-01T19:43:05.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.461+0000 7f40fa073700 1 -- 172.21.15.17:0/2918829306 wait complete. 2022-08-01T19:43:05.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.465+0000 7f40fa073700 1 Processor -- start 2022-08-01T19:43:05.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.465+0000 7f40fa073700 1 -- start start 2022-08-01T19:43:05.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.465+0000 7f40fa073700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40f40aadb0 0x7f40f405cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:05.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.465+0000 7f40fa073700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40f40b31f0 0x7f40f40add00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:05.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.465+0000 7f40fa073700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f40f40b7500 0x7f40f40ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:05.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.465+0000 7f40fa073700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f4156c30 con 0x7f40f40aadb0 2022-08-01T19:43:05.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.465+0000 7f40fa073700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f40622b0 con 0x7f40f40b31f0 2022-08-01T19:43:05.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.465+0000 7f40fa073700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f4061d70 con 0x7f40f40b7500 2022-08-01T19:43:05.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.465+0000 7f40f9071700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40f40aadb0 0x7f40f405cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:05.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.469+0000 7f40f9071700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40f40aadb0 0x7f40f405cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44368/0 (socket says 172.21.15.17:44368) 2022-08-01T19:43:05.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.469+0000 7f40f9071700 1 -- 172.21.15.17:0/3610402318 learned_addr learned my addr 172.21.15.17:0/3610402318 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:05.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.469+0000 7f40f8870700 1 --2- 172.21.15.17:0/3610402318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40f40b31f0 0x7f40f40add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:05.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.469+0000 7f40f9872700 1 --2- 172.21.15.17:0/3610402318 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f40f40b7500 0x7f40f40ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:05.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.469+0000 7f40f9071700 1 -- 172.21.15.17:0/3610402318 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f40f40b7500 msgr2=0x7f40f40ae1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:05.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.469+0000 7f40f9071700 1 --2- 172.21.15.17:0/3610402318 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f40f40b7500 0x7f40f40ae1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:05.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.469+0000 7f40f9071700 1 -- 172.21.15.17:0/3610402318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40f40b31f0 msgr2=0x7f40f40add00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:05.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.469+0000 7f40f9071700 1 --2- 172.21.15.17:0/3610402318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40f40b31f0 0x7f40f40add00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:05.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.469+0000 7f40f9071700 1 -- 172.21.15.17:0/3610402318 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ec009530 con 0x7f40f40aadb0 2022-08-01T19:43:05.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.469+0000 7f40f9071700 1 --2- 172.21.15.17:0/3610402318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40f40aadb0 0x7f40f405cbf0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 rx=0x7f40f000c000 tx=0x7f40f0006d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:05.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.469+0000 7f40e27fc700 1 -- 172.21.15.17:0/3610402318 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40f0011750 con 0x7f40f40aadb0 2022-08-01T19:43:05.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.469+0000 7f40e27fc700 1 -- 172.21.15.17:0/3610402318 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f00118b0 con 0x7f40f40aadb0 2022-08-01T19:43:05.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.469+0000 7f40fa073700 1 -- 172.21.15.17:0/3610402318 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f4062010 con 0x7f40f40aadb0 2022-08-01T19:43:05.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.469+0000 7f40e27fc700 1 -- 172.21.15.17:0/3610402318 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40f0011c10 con 0x7f40f40aadb0 2022-08-01T19:43:05.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.469+0000 7f40fa073700 1 -- 172.21.15.17:0/3610402318 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f40ae6e0 con 0x7f40f40aadb0 2022-08-01T19:43:05.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.469+0000 7f40e27fc700 1 -- 172.21.15.17:0/3610402318 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f40f0011d70 con 0x7f40f40aadb0 2022-08-01T19:43:05.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.469+0000 7f40fa073700 1 -- 172.21.15.17:0/3610402318 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40f40574e0 con 0x7f40f40aadb0 2022-08-01T19:43:05.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.469+0000 7f40e27fc700 1 --2- 172.21.15.17:0/3610402318 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f40d4053a70 0x7f40d4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:05.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.469+0000 7f40e27fc700 1 -- 172.21.15.17:0/3610402318 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f40f006c010 con 0x7f40f40aadb0 2022-08-01T19:43:05.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.469+0000 7f40f8870700 1 --2- 172.21.15.17:0/3610402318 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f40d4053a70 0x7f40d4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:05.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.473+0000 7f40f8870700 1 --2- 172.21.15.17:0/3610402318 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f40d4053a70 0x7f40d4055f30 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f40e400d000 tx=0x7f40e4007a00).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:05.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.477+0000 7f40e27fc700 1 -- 172.21.15.17:0/3610402318 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f40f0042d70 con 0x7f40f40aadb0 2022-08-01T19:43:05.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.605+0000 7f40fa073700 1 -- 172.21.15.17:0/3610402318 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f40f413f5d0 con 0x7f40f40aadb0 2022-08-01T19:43:05.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.605+0000 7f40e27fc700 1 -- 172.21.15.17:0/3610402318 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v137) v1 ==== 100+0+10 (secure 0 0 0) 0x7f40f0046390 con 0x7f40f40aadb0 2022-08-01T19:43:05.608 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:43:05.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.605+0000 7f40fa073700 1 -- 172.21.15.17:0/3610402318 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f40d4053a70 msgr2=0x7f40d4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:05.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.605+0000 7f40fa073700 1 --2- 172.21.15.17:0/3610402318 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f40d4053a70 0x7f40d4055f30 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f40e400d000 tx=0x7f40e4007a00).stop 2022-08-01T19:43:05.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.605+0000 7f40fa073700 1 -- 172.21.15.17:0/3610402318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40f40aadb0 msgr2=0x7f40f405cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:05.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.605+0000 7f40fa073700 1 --2- 172.21.15.17:0/3610402318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40f40aadb0 0x7f40f405cbf0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 rx=0x7f40f000c000 tx=0x7f40f0006d90).stop 2022-08-01T19:43:05.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.605+0000 7f40fa073700 1 -- 172.21.15.17:0/3610402318 shutdown_connections 2022-08-01T19:43:05.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.605+0000 7f40fa073700 1 --2- 172.21.15.17:0/3610402318 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f40d4053a70 0x7f40d4055f30 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:05.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.605+0000 7f40fa073700 1 --2- 172.21.15.17:0/3610402318 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f40f40b7500 0x7f40f40ae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:05.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.605+0000 7f40fa073700 1 --2- 172.21.15.17:0/3610402318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40f40b31f0 0x7f40f40add00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:05.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.605+0000 7f40fa073700 1 --2- 172.21.15.17:0/3610402318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40f40aadb0 0x7f40f405cbf0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:05.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.605+0000 7f40fa073700 1 -- 172.21.15.17:0/3610402318 >> 172.21.15.17:0/3610402318 conn(0x7f40f405f840 msgr2=0x7f40f413f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:05.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.605+0000 7f40fa073700 1 -- 172.21.15.17:0/3610402318 shutdown_connections 2022-08-01T19:43:05.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.605+0000 7f40fa073700 1 -- 172.21.15.17:0/3610402318 wait complete. 2022-08-01T19:43:05.622 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T19:43:05.623 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:43:05.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.721+0000 7fcb5f9e1700 1 Processor -- start 2022-08-01T19:43:05.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.721+0000 7fcb5f9e1700 1 -- start start 2022-08-01T19:43:05.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.721+0000 7fcb5f9e1700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fcb58060290 0x7fcb580617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:05.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.725+0000 7fcb5f9e1700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb58060670 con 0x7fcb58065b00 2022-08-01T19:43:05.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.725+0000 7fcb5f9e1700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb58060e00 con 0x7fcb58065fb0 2022-08-01T19:43:05.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.725+0000 7fcb5f9e1700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb580610a0 con 0x7fcb58060290 2022-08-01T19:43:05.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.725+0000 7fcb5cf7c700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fcb58065b00 0x7fcb58066760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36572/0 (socket says 172.21.15.17:36572) 2022-08-01T19:43:05.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.725+0000 7fcb5cf7c700 1 -- 172.21.15.17:0/3726744918 learned_addr learned my addr 172.21.15.17:0/3726744918 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:05.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.725+0000 7fcb5d77d700 1 --2- 172.21.15.17:0/3726744918 >> v2:172.21.15.77:3300/0 conn(0x7fcb58060290 0x7fcb580617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:05.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.725+0000 7fcb4ffff700 1 -- 172.21.15.17:0/3726744918 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1891171665 0 0) 0x7fcb58060670 con 0x7fcb58065b00 2022-08-01T19:43:05.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.725+0000 7fcb4ffff700 1 -- 172.21.15.17:0/3726744918 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb3c003260 con 0x7fcb58065b00 2022-08-01T19:43:05.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.725+0000 7fcb5d77d700 1 -- 172.21.15.17:0/3726744918 >> v1:172.21.15.77:6789/0 conn(0x7fcb58065fb0 legacy=0x7fcb5814db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:05.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.725+0000 7fcb5d77d700 1 -- 172.21.15.17:0/3726744918 >> v1:172.21.15.17:6789/0 conn(0x7fcb58065b00 legacy=0x7fcb58066760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:05.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.725+0000 7fcb5d77d700 1 -- 172.21.15.17:0/3726744918 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb58061340 con 0x7fcb58060290 2022-08-01T19:43:05.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.725+0000 7fcb5d77d700 1 --2- 172.21.15.17:0/3726744918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb58060290 0x7fcb580617e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7fcb54002f70 tx=0x7fcb54006df0).ready entity=mon.1 client_cookie=7b74073d0f401582 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:05.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.725+0000 7fcb4ffff700 1 -- 172.21.15.17:0/3726744918 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb5400d610 con 0x7fcb58060290 2022-08-01T19:43:05.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.725+0000 7fcb4ffff700 1 -- 172.21.15.17:0/3726744918 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb54000e90 con 0x7fcb58060290 2022-08-01T19:43:05.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.725+0000 7fcb4ffff700 1 -- 172.21.15.17:0/3726744918 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb5400d960 con 0x7fcb58060290 2022-08-01T19:43:05.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.725+0000 7fcb5f9e1700 1 -- 172.21.15.17:0/3726744918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb58060290 msgr2=0x7fcb580617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:05.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.725+0000 7fcb5f9e1700 1 --2- 172.21.15.17:0/3726744918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb58060290 0x7fcb580617e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7fcb54002f70 tx=0x7fcb54006df0).stop 2022-08-01T19:43:05.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.725+0000 7fcb5f9e1700 1 -- 172.21.15.17:0/3726744918 shutdown_connections 2022-08-01T19:43:05.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.725+0000 7fcb5f9e1700 1 --2- 172.21.15.17:0/3726744918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb58060290 0x7fcb580617e0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:05.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.725+0000 7fcb5f9e1700 1 -- 172.21.15.17:0/3726744918 >> 172.21.15.17:0/3726744918 conn(0x7fcb5805d0c0 msgr2=0x7fcb5805f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:05.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.725+0000 7fcb5f9e1700 1 -- 172.21.15.17:0/3726744918 shutdown_connections 2022-08-01T19:43:05.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.725+0000 7fcb5f9e1700 1 -- 172.21.15.17:0/3726744918 wait complete. 2022-08-01T19:43:05.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.729+0000 7fcb5f9e1700 1 Processor -- start 2022-08-01T19:43:05.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.729+0000 7fcb5f9e1700 1 -- start start 2022-08-01T19:43:05.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.729+0000 7fcb5f9e1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb58060290 0x7fcb58145f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:05.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.729+0000 7fcb5f9e1700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcb58065b00 0x7fcb58152b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:05.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.729+0000 7fcb5f9e1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb58065fb0 0x7fcb58153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:05.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.729+0000 7fcb5f9e1700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb580b0680 con 0x7fcb58065fb0 2022-08-01T19:43:05.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.729+0000 7fcb5f9e1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb580b0920 con 0x7fcb58060290 2022-08-01T19:43:05.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.729+0000 7fcb5f9e1700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb580b0bc0 con 0x7fcb58065b00 2022-08-01T19:43:05.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.729+0000 7fcb5df7e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb58065fb0 0x7fcb58153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:05.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.729+0000 7fcb5df7e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb58065fb0 0x7fcb58153020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44386/0 (socket says 172.21.15.17:44386) 2022-08-01T19:43:05.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.729+0000 7fcb5df7e700 1 -- 172.21.15.17:0/2512583480 learned_addr learned my addr 172.21.15.17:0/2512583480 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:05.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.729+0000 7fcb5d77d700 1 --2- 172.21.15.17:0/2512583480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb58060290 0x7fcb58145f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:05.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.729+0000 7fcb5cf7c700 1 --2- 172.21.15.17:0/2512583480 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcb58065b00 0x7fcb58152b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:05.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.733+0000 7fcb5df7e700 1 -- 172.21.15.17:0/2512583480 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcb58065b00 msgr2=0x7fcb58152b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:05.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.733+0000 7fcb5df7e700 1 --2- 172.21.15.17:0/2512583480 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcb58065b00 0x7fcb58152b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:05.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.733+0000 7fcb5df7e700 1 -- 172.21.15.17:0/2512583480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb58060290 msgr2=0x7fcb58145f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:05.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.733+0000 7fcb5df7e700 1 --2- 172.21.15.17:0/2512583480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb58060290 0x7fcb58145f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:05.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.733+0000 7fcb5df7e700 1 -- 172.21.15.17:0/2512583480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb5400b180 con 0x7fcb58065fb0 2022-08-01T19:43:05.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.733+0000 7fcb5df7e700 1 --2- 172.21.15.17:0/2512583480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb58065fb0 0x7fcb58153020 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 rx=0x7fcb58064000 tx=0x7fcb500013a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:05.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.733+0000 7fcb4e7fc700 1 -- 172.21.15.17:0/2512583480 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb50006aa0 con 0x7fcb58065fb0 2022-08-01T19:43:05.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.733+0000 7fcb4e7fc700 1 -- 172.21.15.17:0/2512583480 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb50006c00 con 0x7fcb58065fb0 2022-08-01T19:43:05.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.733+0000 7fcb5f9e1700 1 -- 172.21.15.17:0/2512583480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb580b0e60 con 0x7fcb58065fb0 2022-08-01T19:43:05.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.733+0000 7fcb4e7fc700 1 -- 172.21.15.17:0/2512583480 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb50007030 con 0x7fcb58065fb0 2022-08-01T19:43:05.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.733+0000 7fcb5f9e1700 1 -- 172.21.15.17:0/2512583480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb580b0300 con 0x7fcb58065fb0 2022-08-01T19:43:05.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.733+0000 7fcb5f9e1700 1 -- 172.21.15.17:0/2512583480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb580574e0 con 0x7fcb58065fb0 2022-08-01T19:43:05.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.733+0000 7fcb4e7fc700 1 -- 172.21.15.17:0/2512583480 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcb50010030 con 0x7fcb58065fb0 2022-08-01T19:43:05.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.733+0000 7fcb4e7fc700 1 --2- 172.21.15.17:0/2512583480 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcb40053a70 0x7fcb40055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:05.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.737+0000 7fcb4e7fc700 1 -- 172.21.15.17:0/2512583480 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fcb5006be80 con 0x7fcb58065fb0 2022-08-01T19:43:05.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.737+0000 7fcb5d77d700 1 --2- 172.21.15.17:0/2512583480 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcb40053a70 0x7fcb40055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:05.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.737+0000 7fcb5d77d700 1 --2- 172.21.15.17:0/2512583480 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcb40053a70 0x7fcb40055f30 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7fcb5401a000 tx=0x7fcb54014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:05.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.741+0000 7fcb4e7fc700 1 -- 172.21.15.17:0/2512583480 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcb50042ba0 con 0x7fcb58065fb0 2022-08-01T19:43:05.908 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.905+0000 7fcb5f9e1700 1 -- 172.21.15.17:0/2512583480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb580b2550 con 0x7fcb58065fb0 2022-08-01T19:43:05.909 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.905+0000 7fcb4e7fc700 1 -- 172.21.15.17:0/2512583480 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+2486 (secure 0 0 0) 0x7fcb500461c0 con 0x7fcb58065fb0 2022-08-01T19:43:05.910 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:05.911 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":69,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7995,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.17:6833/4164640472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4164640472},{"type":"v1","addr":"172.21.15.17:6833","nonce":4164640472}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":67},{"gid":8001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/50015801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":50015801},{"type":"v1","addr":"172.21.15.17:6835","nonce":50015801}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:43:01.287477+0000","modified":"2022-08-01T19:43:02.290478+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8007},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8007":{"gid":8007,"name":"b","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6835/299216688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":299216688},{"type":"v1","addr":"172.21.15.77:6835","nonce":299216688}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2022-08-01T19:43:05.912 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.905+0000 7fcb5f9e1700 1 -- 172.21.15.17:0/2512583480 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcb40053a70 msgr2=0x7fcb40055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:05.912 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.905+0000 7fcb5f9e1700 1 --2- 172.21.15.17:0/2512583480 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcb40053a70 0x7fcb40055f30 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7fcb5401a000 tx=0x7fcb54014000).stop 2022-08-01T19:43:05.913 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.905+0000 7fcb5f9e1700 1 -- 172.21.15.17:0/2512583480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb58065fb0 msgr2=0x7fcb58153020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:05.913 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.905+0000 7fcb5f9e1700 1 --2- 172.21.15.17:0/2512583480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb58065fb0 0x7fcb58153020 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 rx=0x7fcb58064000 tx=0x7fcb500013a0).stop 2022-08-01T19:43:05.913 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.905+0000 7fcb5f9e1700 1 -- 172.21.15.17:0/2512583480 shutdown_connections 2022-08-01T19:43:05.914 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.905+0000 7fcb5f9e1700 1 --2- 172.21.15.17:0/2512583480 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcb40053a70 0x7fcb40055f30 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:05.914 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.905+0000 7fcb5f9e1700 1 --2- 172.21.15.17:0/2512583480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb58065fb0 0x7fcb58153020 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:05.914 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.905+0000 7fcb5f9e1700 1 --2- 172.21.15.17:0/2512583480 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcb58065b00 0x7fcb58152b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:05.915 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.905+0000 7fcb5f9e1700 1 --2- 172.21.15.17:0/2512583480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcb58060290 0x7fcb58145f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:05.915 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.905+0000 7fcb5f9e1700 1 -- 172.21.15.17:0/2512583480 >> 172.21.15.17:0/2512583480 conn(0x7fcb5805d0c0 msgr2=0x7fcb580b4880 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:05.915 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.909+0000 7fcb5f9e1700 1 -- 172.21.15.17:0/2512583480 shutdown_connections 2022-08-01T19:43:05.916 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:05.909+0000 7fcb5f9e1700 1 -- 172.21.15.17:0/2512583480 wait complete. 2022-08-01T19:43:05.916 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 69 2022-08-01T19:43:06.226 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 69, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:43:01.287477+0000', 'modified': '2022-08-01T19:43:02.290478+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8007}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8007': {'gid': 8007, 'name': 'b', 'rank': 0, 'incarnation': 68, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.77:6835/299216688', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6834', 'nonce': 299216688}, {'type': 'v1', 'addr': '172.21.15.77:6835', 'nonce': 299216688}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:43:06.227 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:43:06.228 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-08-01T19:43:06.345 INFO:teuthology.orchestra.run.smithi077.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":8007,"want_state":"up:active","state":"up:active","rank_uptime":5.0451338840000002,"mdsmap_epoch":69,"osdmap_epoch":137,"osdmap_epoch_barrier":136,"uptime":9.9490110880000007} 2022-08-01T19:43:06.346 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 8007, "mdsmap_epoch": 69, "osdmap_epoch": 137, "osdmap_epoch_barrier": 136, "rank_uptime": 5.045133884, "state": "up:active", "uptime": 9.949011088, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:43:06.346 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T19:43:06.347 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-08-01T19:43:06.347 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi085.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-08-01T19:43:06.348 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:06.348 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:43:06.357 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:43:06.419 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:43:06.420 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:43:06.435 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:43:06.436 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:43:06.437 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:06.438 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:43:06.485 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T19:43:06.486 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:43:06.491 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:43:06.508 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:43:06.509 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:43:06.571 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:2022-08-01T19:43:06.566+0000 7efcf9178080 -1 init, newargv = 0x556d2f4435b0 newargc=9ceph-fuse[23551]: starting ceph client 2022-08-01T19:43:06.571 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr: 2022-08-01T19:43:06.573 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:43:06.575 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:43:06.575 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:06.576 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:43:06.579 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:ceph-fuse[23551]: starting fuse 2022-08-01T19:43:06.588 INFO:teuthology.orchestra.run.smithi085.stdout:52 2022-08-01T19:43:06.589 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T19:43:06.589 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:06.591 DEBUG:teuthology.orchestra.run.smithi085:> sudo python3 -c ' 2022-08-01T19:43:06.591 DEBUG:teuthology.orchestra.run.smithi085:> import glob 2022-08-01T19:43:06.592 DEBUG:teuthology.orchestra.run.smithi085:> import re 2022-08-01T19:43:06.593 DEBUG:teuthology.orchestra.run.smithi085:> import os 2022-08-01T19:43:06.594 DEBUG:teuthology.orchestra.run.smithi085:> import subprocess 2022-08-01T19:43:06.595 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:43:06.596 DEBUG:teuthology.orchestra.run.smithi085:> def find_socket(client_name): 2022-08-01T19:43:06.597 DEBUG:teuthology.orchestra.run.smithi085:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-08-01T19:43:06.598 DEBUG:teuthology.orchestra.run.smithi085:> files = glob.glob(asok_path) 2022-08-01T19:43:06.599 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:43:06.599 DEBUG:teuthology.orchestra.run.smithi085:> # Given a non-glob path, it better be there 2022-08-01T19:43:06.606 DEBUG:teuthology.orchestra.run.smithi085:> if "*" not in asok_path: 2022-08-01T19:43:06.606 DEBUG:teuthology.orchestra.run.smithi085:> assert(len(files) == 1) 2022-08-01T19:43:06.607 DEBUG:teuthology.orchestra.run.smithi085:> return files[0] 2022-08-01T19:43:06.607 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:43:06.607 DEBUG:teuthology.orchestra.run.smithi085:> for f in files: 2022-08-01T19:43:06.608 DEBUG:teuthology.orchestra.run.smithi085:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T19:43:06.608 DEBUG:teuthology.orchestra.run.smithi085:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T19:43:06.609 DEBUG:teuthology.orchestra.run.smithi085:> return f 2022-08-01T19:43:06.609 DEBUG:teuthology.orchestra.run.smithi085:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T19:43:06.610 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:43:06.610 DEBUG:teuthology.orchestra.run.smithi085:> print(find_socket("client.0")) 2022-08-01T19:43:06.611 DEBUG:teuthology.orchestra.run.smithi085:> ' 2022-08-01T19:43:06.672 INFO:teuthology.orchestra.run.smithi085.stdout:/var/run/ceph/ceph-client.0.23551.asok 2022-08-01T19:43:06.677 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.23551.asok 2022-08-01T19:43:06.678 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:06.678 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.23551.asok status 2022-08-01T19:43:06.764 INFO:teuthology.orchestra.run.smithi085.stdout:{ 2022-08-01T19:43:06.765 INFO:teuthology.orchestra.run.smithi085.stdout: "metadata": { 2022-08-01T19:43:06.765 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T19:43:06.766 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T19:43:06.766 INFO:teuthology.orchestra.run.smithi085.stdout: "entity_id": "0", 2022-08-01T19:43:06.767 INFO:teuthology.orchestra.run.smithi085.stdout: "hostname": "smithi085", 2022-08-01T19:43:06.767 INFO:teuthology.orchestra.run.smithi085.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-08-01T19:43:06.768 INFO:teuthology.orchestra.run.smithi085.stdout: "pid": "23551", 2022-08-01T19:43:06.768 INFO:teuthology.orchestra.run.smithi085.stdout: "root": "/" 2022-08-01T19:43:06.768 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:43:06.769 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_count": 0, 2022-08-01T19:43:06.769 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_pinned_count": 0, 2022-08-01T19:43:06.770 INFO:teuthology.orchestra.run.smithi085.stdout: "id": 8160, 2022-08-01T19:43:06.770 INFO:teuthology.orchestra.run.smithi085.stdout: "inst": { 2022-08-01T19:43:06.772 INFO:teuthology.orchestra.run.smithi085.stdout: "name": { 2022-08-01T19:43:06.772 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "client", 2022-08-01T19:43:06.773 INFO:teuthology.orchestra.run.smithi085.stdout: "num": 8160 2022-08-01T19:43:06.773 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:43:06.774 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2022-08-01T19:43:06.774 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2022-08-01T19:43:06.775 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2022-08-01T19:43:06.775 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 2063118981 2022-08-01T19:43:06.776 INFO:teuthology.orchestra.run.smithi085.stdout: } 2022-08-01T19:43:06.776 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:43:06.777 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2022-08-01T19:43:06.778 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2022-08-01T19:43:06.778 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2022-08-01T19:43:06.778 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 2063118981 2022-08-01T19:43:06.779 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:43:06.779 INFO:teuthology.orchestra.run.smithi085.stdout: "inst_str": "client.8160 172.21.15.85:0/2063118981", 2022-08-01T19:43:06.779 INFO:teuthology.orchestra.run.smithi085.stdout: "addr_str": "172.21.15.85:0/2063118981", 2022-08-01T19:43:06.779 INFO:teuthology.orchestra.run.smithi085.stdout: "inode_count": 1, 2022-08-01T19:43:06.780 INFO:teuthology.orchestra.run.smithi085.stdout: "mds_epoch": 69, 2022-08-01T19:43:06.780 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch": 137, 2022-08-01T19:43:06.780 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch_barrier": 0, 2022-08-01T19:43:06.780 INFO:teuthology.orchestra.run.smithi085.stdout: "blacklisted": false, 2022-08-01T19:43:06.781 INFO:teuthology.orchestra.run.smithi085.stdout: "fs_name": "cephfs" 2022-08-01T19:43:06.781 INFO:teuthology.orchestra.run.smithi085.stdout:} 2022-08-01T19:43:06.782 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:06.783 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:43:06.784 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:43:06.793 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:43:06.793 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:43:06.794 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:06.795 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:43:06.855 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:43:06.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.965+0000 7f0d318eb700 1 Processor -- start 2022-08-01T19:43:06.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.965+0000 7f0d318eb700 1 -- start start 2022-08-01T19:43:06.983 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.965+0000 7f0d318eb700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f0d2c0b1240 0x7f0d2c0b1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:06.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.965+0000 7f0d318eb700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0d2c0b0530 0x7f0d2c0ae940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:06.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.965+0000 7f0d318eb700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d2c0b1f70 con 0x7f0d2c05d8f0 2022-08-01T19:43:06.984 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.965+0000 7f0d318eb700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d2c0af7c0 con 0x7f0d2c0b1240 2022-08-01T19:43:06.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.965+0000 7f0d318eb700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c0afa60 con 0x7f0d2c0b0530 2022-08-01T19:43:06.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.965+0000 7f0d310ea700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0d2c0b0530 0x7f0d2c0ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:06.985 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.965+0000 7f0d308e9700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f0d2c0b1240 0x7f0d2c0b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:06.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.965+0000 7f0d2bfff700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7f0d2c05d8f0 0x7f0d2c142da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:53386/0 (socket says 172.21.15.17:53386) 2022-08-01T19:43:06.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.965+0000 7f0d2bfff700 1 -- 172.21.15.17:0/2895447253 learned_addr learned my addr 172.21.15.17:0/2895447253 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:06.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.969+0000 7f0d2b7fe700 1 -- 172.21.15.17:0/2895447253 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3327773340 0 0) 0x7f0d2c0b1f70 con 0x7f0d2c05d8f0 2022-08-01T19:43:06.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.969+0000 7f0d2b7fe700 1 -- 172.21.15.17:0/2895447253 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d0c003260 con 0x7f0d2c05d8f0 2022-08-01T19:43:06.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.969+0000 7f0d310ea700 1 -- 172.21.15.17:0/2895447253 >> v2:172.21.15.77:3300/0 conn(0x7f0d2c0b1240 msgr2=0x7f0d2c0b1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:06.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.969+0000 7f0d310ea700 1 --2- 172.21.15.17:0/2895447253 >> v2:172.21.15.77:3300/0 conn(0x7f0d2c0b1240 0x7f0d2c0b1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:06.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.969+0000 7f0d310ea700 1 -- 172.21.15.17:0/2895447253 >> v1:172.21.15.77:6789/0 conn(0x7f0d2c05d8f0 legacy=0x7f0d2c142da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:06.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.969+0000 7f0d310ea700 1 -- 172.21.15.17:0/2895447253 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d2c0b2240 con 0x7f0d2c0b0530 2022-08-01T19:43:06.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.969+0000 7f0d308e9700 1 --2- 172.21.15.17:0/2895447253 >> v2:172.21.15.77:3300/0 conn(0x7f0d2c0b1240 0x7f0d2c0b1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:43:06.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.969+0000 7f0d310ea700 1 --2- 172.21.15.17:0/2895447253 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0d2c0b0530 0x7f0d2c0ae940 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f0d1c002f70 tx=0x7f0d1c006df0).ready entity=mon.2 client_cookie=3fe688bb709c3d8b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:06.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.969+0000 7f0d2b7fe700 1 -- 172.21.15.17:0/2895447253 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d1c00d4a0 con 0x7f0d2c0b0530 2022-08-01T19:43:06.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.969+0000 7f0d2b7fe700 1 -- 172.21.15.17:0/2895447253 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1c000e90 con 0x7f0d2c0b0530 2022-08-01T19:43:06.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.969+0000 7f0d2b7fe700 1 -- 172.21.15.17:0/2895447253 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d1c00d7f0 con 0x7f0d2c0b0530 2022-08-01T19:43:06.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.969+0000 7f0d318eb700 1 -- 172.21.15.17:0/2895447253 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0d2c0b0530 msgr2=0x7f0d2c0ae940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:06.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.969+0000 7f0d318eb700 1 --2- 172.21.15.17:0/2895447253 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0d2c0b0530 0x7f0d2c0ae940 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f0d1c002f70 tx=0x7f0d1c006df0).stop 2022-08-01T19:43:06.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.969+0000 7f0d318eb700 1 -- 172.21.15.17:0/2895447253 shutdown_connections 2022-08-01T19:43:06.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.969+0000 7f0d318eb700 1 --2- 172.21.15.17:0/2895447253 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0d2c0b0530 0x7f0d2c0ae940 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:06.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.969+0000 7f0d318eb700 1 --2- 172.21.15.17:0/2895447253 >> v2:172.21.15.77:3300/0 conn(0x7f0d2c0b1240 0x7f0d2c0b1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:06.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.969+0000 7f0d318eb700 1 -- 172.21.15.17:0/2895447253 >> 172.21.15.17:0/2895447253 conn(0x7f0d2c05c620 msgr2=0x7f0d2c05ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:06.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.969+0000 7f0d318eb700 1 -- 172.21.15.17:0/2895447253 shutdown_connections 2022-08-01T19:43:06.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.969+0000 7f0d318eb700 1 -- 172.21.15.17:0/2895447253 wait complete. 2022-08-01T19:43:06.991 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.969+0000 7f0d318eb700 1 Processor -- start 2022-08-01T19:43:06.991 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.969+0000 7f0d318eb700 1 -- start start 2022-08-01T19:43:06.991 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.969+0000 7f0d318eb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d2c05d8f0 0x7f0d2c141740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:06.991 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.969+0000 7f0d318eb700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0d2c0b0530 0x7f0d2c153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:06.992 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.969+0000 7f0d318eb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d2c0b1240 0x7f0d2c0b3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:06.992 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.969+0000 7f0d318eb700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c0b3f40 con 0x7f0d2c05d8f0 2022-08-01T19:43:06.992 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.969+0000 7f0d318eb700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c142280 con 0x7f0d2c0b1240 2022-08-01T19:43:06.992 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.969+0000 7f0d318eb700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c152860 con 0x7f0d2c0b0530 2022-08-01T19:43:06.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.973+0000 7f0d310ea700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d2c0b1240 0x7f0d2c0b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:06.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.973+0000 7f0d310ea700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d2c0b1240 0x7f0d2c0b3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:42322/0 (socket says 172.21.15.17:42322) 2022-08-01T19:43:06.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.973+0000 7f0d2bfff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0d2c0b0530 0x7f0d2c153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:06.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.973+0000 7f0d310ea700 1 -- 172.21.15.17:0/4274734237 learned_addr learned my addr 172.21.15.17:0/4274734237 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:06.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.973+0000 7f0d308e9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d2c05d8f0 0x7f0d2c141740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:06.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.973+0000 7f0d310ea700 1 -- 172.21.15.17:0/4274734237 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0d2c0b0530 msgr2=0x7f0d2c153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:06.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.973+0000 7f0d310ea700 1 --2- 172.21.15.17:0/4274734237 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0d2c0b0530 0x7f0d2c153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:06.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.973+0000 7f0d310ea700 1 -- 172.21.15.17:0/4274734237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d2c05d8f0 msgr2=0x7f0d2c141740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:06.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.973+0000 7f0d310ea700 1 --2- 172.21.15.17:0/4274734237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d2c05d8f0 0x7f0d2c141740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:06.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.973+0000 7f0d310ea700 1 -- 172.21.15.17:0/4274734237 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d24009470 con 0x7f0d2c0b1240 2022-08-01T19:43:06.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.973+0000 7f0d2bfff700 1 --2- 172.21.15.17:0/4274734237 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0d2c0b0530 0x7f0d2c153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:43:06.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.973+0000 7f0d310ea700 1 --2- 172.21.15.17:0/4274734237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d2c0b1240 0x7f0d2c0b3710 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f0d2c0abb70 tx=0x7f0d1c014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:06.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.973+0000 7f0d29ffb700 1 -- 172.21.15.17:0/4274734237 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d1c016070 con 0x7f0d2c0b1240 2022-08-01T19:43:06.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.973+0000 7f0d29ffb700 1 -- 172.21.15.17:0/4274734237 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1c0137d0 con 0x7f0d2c0b1240 2022-08-01T19:43:06.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.973+0000 7f0d29ffb700 1 -- 172.21.15.17:0/4274734237 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d1c013b30 con 0x7f0d2c0b1240 2022-08-01T19:43:06.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.973+0000 7f0d318eb700 1 -- 172.21.15.17:0/4274734237 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d1c00b180 con 0x7f0d2c0b1240 2022-08-01T19:43:06.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.973+0000 7f0d318eb700 1 -- 172.21.15.17:0/4274734237 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d2c152da0 con 0x7f0d2c0b1240 2022-08-01T19:43:06.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.973+0000 7f0d318eb700 1 -- 172.21.15.17:0/4274734237 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d2c0574e0 con 0x7f0d2c0b1240 2022-08-01T19:43:06.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.977+0000 7f0d29ffb700 1 -- 172.21.15.17:0/4274734237 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0d1c025030 con 0x7f0d2c0b1240 2022-08-01T19:43:06.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.977+0000 7f0d29ffb700 1 --2- 172.21.15.17:0/4274734237 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0d100539e0 0x7f0d10055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:06.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.977+0000 7f0d29ffb700 1 -- 172.21.15.17:0/4274734237 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f0d1c06d3e0 con 0x7f0d2c0b1240 2022-08-01T19:43:07.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.977+0000 7f0d308e9700 1 --2- 172.21.15.17:0/4274734237 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0d100539e0 0x7f0d10055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:07.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.977+0000 7f0d308e9700 1 --2- 172.21.15.17:0/4274734237 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0d100539e0 0x7f0d10055ea0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f0d2400b000 tx=0x7f0d240091d0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:07.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:06.985+0000 7f0d29ffb700 1 -- 172.21.15.17:0/4274734237 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0d1c044090 con 0x7f0d2c0b1240 2022-08-01T19:43:07.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.113+0000 7f0d318eb700 1 -- 172.21.15.17:0/4274734237 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d2c0635c0 con 0x7f0d2c0b1240 2022-08-01T19:43:07.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.113+0000 7f0d29ffb700 1 -- 172.21.15.17:0/4274734237 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f0d1c0476b0 con 0x7f0d2c0b1240 2022-08-01T19:43:07.124 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:07.125 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":137,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:43:02.291265+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:42:59.166414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2022-08-01T19:43:00.301184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:43:07.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.117+0000 7f0d318eb700 1 -- 172.21.15.17:0/4274734237 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0d100539e0 msgr2=0x7f0d10055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:07.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.117+0000 7f0d318eb700 1 --2- 172.21.15.17:0/4274734237 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0d100539e0 0x7f0d10055ea0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f0d2400b000 tx=0x7f0d240091d0).stop 2022-08-01T19:43:07.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.117+0000 7f0d318eb700 1 -- 172.21.15.17:0/4274734237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d2c0b1240 msgr2=0x7f0d2c0b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:07.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.117+0000 7f0d318eb700 1 --2- 172.21.15.17:0/4274734237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d2c0b1240 0x7f0d2c0b3710 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f0d2c0abb70 tx=0x7f0d1c014000).stop 2022-08-01T19:43:07.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.117+0000 7f0d318eb700 1 -- 172.21.15.17:0/4274734237 shutdown_connections 2022-08-01T19:43:07.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.117+0000 7f0d318eb700 1 --2- 172.21.15.17:0/4274734237 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f0d100539e0 0x7f0d10055ea0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:07.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.117+0000 7f0d318eb700 1 --2- 172.21.15.17:0/4274734237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d2c0b1240 0x7f0d2c0b3710 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:07.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.117+0000 7f0d318eb700 1 --2- 172.21.15.17:0/4274734237 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f0d2c0b0530 0x7f0d2c153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:07.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.117+0000 7f0d318eb700 1 --2- 172.21.15.17:0/4274734237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d2c05d8f0 0x7f0d2c141740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:07.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.117+0000 7f0d318eb700 1 -- 172.21.15.17:0/4274734237 >> 172.21.15.17:0/4274734237 conn(0x7f0d2c05c620 msgr2=0x7f0d2c142b70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:07.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.117+0000 7f0d318eb700 1 -- 172.21.15.17:0/4274734237 shutdown_connections 2022-08-01T19:43:07.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.117+0000 7f0d318eb700 1 -- 172.21.15.17:0/4274734237 wait complete. 2022-08-01T19:43:07.136 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T19:42:59.166414+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T19:43:00.301184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T19:43:07.138 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:43:07.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.261+0000 7f6fb4fab700 1 Processor -- start 2022-08-01T19:43:07.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.261+0000 7f6fb4fab700 1 -- start start 2022-08-01T19:43:07.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.261+0000 7f6fb4fab700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6fb0060560 0x7f6fb00aa430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:07.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.261+0000 7f6fb4fab700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f6fb013f920 0x7f6fb013fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:07.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.261+0000 7f6fb4fab700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fb0061b20 con 0x7f6fb00aa050 2022-08-01T19:43:07.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.261+0000 7f6fb4fab700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fb0140280 con 0x7f6fb0060560 2022-08-01T19:43:07.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.261+0000 7f6fb4fab700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fb0061dc0 con 0x7f6fb013f920 2022-08-01T19:43:07.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.261+0000 7f6fae59c700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f6fb00aa050 0x7f6fb013ce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36598/0 (socket says 172.21.15.17:36598) 2022-08-01T19:43:07.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.261+0000 7f6fae59c700 1 -- 172.21.15.17:0/828241921 learned_addr learned my addr 172.21.15.17:0/828241921 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:07.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.261+0000 7f6fadd9b700 1 --2- 172.21.15.17:0/828241921 >> v2:172.21.15.17:3300/0 conn(0x7f6fb0060560 0x7f6fb00aa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:07.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.261+0000 7f6faed9d700 1 --2- 172.21.15.17:0/828241921 >> v2:172.21.15.77:3300/0 conn(0x7f6fb013f920 0x7f6fb013fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:07.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.261+0000 7f6fadd9b700 1 -- 172.21.15.17:0/828241921 >> v2:172.21.15.77:3300/0 conn(0x7f6fb013f920 msgr2=0x7f6fb013fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:07.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.261+0000 7f6fadd9b700 1 --2- 172.21.15.17:0/828241921 >> v2:172.21.15.77:3300/0 conn(0x7f6fb013f920 0x7f6fb013fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:07.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.261+0000 7f6fadd9b700 1 -- 172.21.15.17:0/828241921 >> v1:172.21.15.17:6789/0 conn(0x7f6fb00aa050 legacy=0x7f6fb013ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:07.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.261+0000 7f6fadd9b700 1 -- 172.21.15.17:0/828241921 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb0060d20 con 0x7f6fb0060560 2022-08-01T19:43:07.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.265+0000 7f6fadd9b700 1 --2- 172.21.15.17:0/828241921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fb0060560 0x7f6fb00aa430 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7f6fb00aac80 tx=0x7f6fa4006df0).ready entity=mon.0 client_cookie=5f6c4e000139f47a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:07.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.265+0000 7f6fad59a700 1 -- 172.21.15.17:0/828241921 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fa400d5c0 con 0x7f6fb0060560 2022-08-01T19:43:07.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.265+0000 7f6fad59a700 1 -- 172.21.15.17:0/828241921 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa4000e90 con 0x7f6fb0060560 2022-08-01T19:43:07.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.265+0000 7f6fad59a700 1 -- 172.21.15.17:0/828241921 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fa400d910 con 0x7f6fb0060560 2022-08-01T19:43:07.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.265+0000 7f6fb4fab700 1 -- 172.21.15.17:0/828241921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fb0060560 msgr2=0x7f6fb00aa430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:07.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.265+0000 7f6fb4fab700 1 --2- 172.21.15.17:0/828241921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fb0060560 0x7f6fb00aa430 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7f6fb00aac80 tx=0x7f6fa4006df0).stop 2022-08-01T19:43:07.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.265+0000 7f6fb4fab700 1 -- 172.21.15.17:0/828241921 shutdown_connections 2022-08-01T19:43:07.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.265+0000 7f6fb4fab700 1 --2- 172.21.15.17:0/828241921 >> v2:172.21.15.77:3300/0 conn(0x7f6fb013f920 0x7f6fb013fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:07.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.265+0000 7f6fb4fab700 1 --2- 172.21.15.17:0/828241921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fb0060560 0x7f6fb00aa430 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:07.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.265+0000 7f6fb4fab700 1 -- 172.21.15.17:0/828241921 >> 172.21.15.17:0/828241921 conn(0x7f6fb00b2b00 msgr2=0x7f6fb00b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:07.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.265+0000 7f6fb4fab700 1 -- 172.21.15.17:0/828241921 shutdown_connections 2022-08-01T19:43:07.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.265+0000 7f6fb4fab700 1 -- 172.21.15.17:0/828241921 wait complete. 2022-08-01T19:43:07.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.269+0000 7f6fb4fab700 1 Processor -- start 2022-08-01T19:43:07.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.269+0000 7f6fb4fab700 1 -- start start 2022-08-01T19:43:07.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.269+0000 7f6fb4fab700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fb0060560 0x7f6fb0140e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:07.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.269+0000 7f6fb4fab700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6fb00aa050 0x7f6fb00adca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:07.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.269+0000 7f6fb4fab700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fb013f920 0x7f6fb00ae190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:07.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.269+0000 7f6fb4fab700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb013bdc0 con 0x7f6fb0060560 2022-08-01T19:43:07.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.269+0000 7f6fb4fab700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb013d090 con 0x7f6fb013f920 2022-08-01T19:43:07.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.269+0000 7f6fb4fab700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb013cbc0 con 0x7f6fb00aa050 2022-08-01T19:43:07.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.269+0000 7f6fae59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fb0060560 0x7f6fb0140e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:07.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.269+0000 7f6fae59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fb0060560 0x7f6fb0140e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44410/0 (socket says 172.21.15.17:44410) 2022-08-01T19:43:07.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.269+0000 7f6fae59c700 1 -- 172.21.15.17:0/3853899178 learned_addr learned my addr 172.21.15.17:0/3853899178 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:07.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.269+0000 7f6fadd9b700 1 --2- 172.21.15.17:0/3853899178 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6fb00aa050 0x7f6fb00adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:07.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.269+0000 7f6faed9d700 1 --2- 172.21.15.17:0/3853899178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fb013f920 0x7f6fb00ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:07.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.269+0000 7f6fae59c700 1 -- 172.21.15.17:0/3853899178 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6fb00aa050 msgr2=0x7f6fb00adca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:07.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.269+0000 7f6fae59c700 1 --2- 172.21.15.17:0/3853899178 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6fb00aa050 0x7f6fb00adca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:07.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.269+0000 7f6fae59c700 1 -- 172.21.15.17:0/3853899178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fb013f920 msgr2=0x7f6fb00ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:07.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.269+0000 7f6fae59c700 1 --2- 172.21.15.17:0/3853899178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fb013f920 0x7f6fb00ae190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:07.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.269+0000 7f6fae59c700 1 -- 172.21.15.17:0/3853899178 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa400b180 con 0x7f6fb0060560 2022-08-01T19:43:07.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.269+0000 7f6fae59c700 1 --2- 172.21.15.17:0/3853899178 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fb0060560 0x7f6fb0140e30 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 rx=0x7f6fb00abdd0 tx=0x7f6fa0007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:07.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.269+0000 7f6f9f7fe700 1 -- 172.21.15.17:0/3853899178 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fa00060d0 con 0x7f6fb0060560 2022-08-01T19:43:07.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.269+0000 7f6f9f7fe700 1 -- 172.21.15.17:0/3853899178 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa0006230 con 0x7f6fb0060560 2022-08-01T19:43:07.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.269+0000 7f6fb4fab700 1 -- 172.21.15.17:0/3853899178 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb013eed0 con 0x7f6fb0060560 2022-08-01T19:43:07.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.269+0000 7f6f9f7fe700 1 -- 172.21.15.17:0/3853899178 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fa0004710 con 0x7f6fb0060560 2022-08-01T19:43:07.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.269+0000 7f6fb4fab700 1 -- 172.21.15.17:0/3853899178 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb013f130 con 0x7f6fb0060560 2022-08-01T19:43:07.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.273+0000 7f6fb4fab700 1 -- 172.21.15.17:0/3853899178 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fb0060fc0 con 0x7f6fb0060560 2022-08-01T19:43:07.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.273+0000 7f6f9f7fe700 1 -- 172.21.15.17:0/3853899178 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6fa0010070 con 0x7f6fb0060560 2022-08-01T19:43:07.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.273+0000 7f6f9f7fe700 1 --2- 172.21.15.17:0/3853899178 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6f8c053a10 0x7f6f8c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:07.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.273+0000 7f6f9f7fe700 1 -- 172.21.15.17:0/3853899178 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6fa006b540 con 0x7f6fb0060560 2022-08-01T19:43:07.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.273+0000 7f6fadd9b700 1 --2- 172.21.15.17:0/3853899178 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6f8c053a10 0x7f6f8c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:07.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.273+0000 7f6fadd9b700 1 --2- 172.21.15.17:0/3853899178 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6f8c053a10 0x7f6f8c055ed0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f6fa401a000 tx=0x7f6fa4014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:07.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.281+0000 7f6f9f7fe700 1 -- 172.21.15.17:0/3853899178 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6fa0042260 con 0x7f6fb0060560 2022-08-01T19:43:07.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.405+0000 7f6fb4fab700 1 -- 172.21.15.17:0/3853899178 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f6fb00574e0 con 0x7f6fb0060560 2022-08-01T19:43:07.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.409+0000 7f6f9f7fe700 1 -- 172.21.15.17:0/3853899178 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v137) v1 ==== 110+0+10 (secure 0 0 0) 0x7f6fa0018070 con 0x7f6fb0060560 2022-08-01T19:43:07.412 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:43:07.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.409+0000 7f6fb4fab700 1 -- 172.21.15.17:0/3853899178 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6f8c053a10 msgr2=0x7f6f8c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:07.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.409+0000 7f6fb4fab700 1 --2- 172.21.15.17:0/3853899178 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6f8c053a10 0x7f6f8c055ed0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f6fa401a000 tx=0x7f6fa4014000).stop 2022-08-01T19:43:07.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.409+0000 7f6fb4fab700 1 -- 172.21.15.17:0/3853899178 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fb0060560 msgr2=0x7f6fb0140e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:07.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.409+0000 7f6fb4fab700 1 --2- 172.21.15.17:0/3853899178 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fb0060560 0x7f6fb0140e30 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 rx=0x7f6fb00abdd0 tx=0x7f6fa0007c00).stop 2022-08-01T19:43:07.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.409+0000 7f6fb4fab700 1 -- 172.21.15.17:0/3853899178 shutdown_connections 2022-08-01T19:43:07.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.409+0000 7f6fb4fab700 1 --2- 172.21.15.17:0/3853899178 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6f8c053a10 0x7f6f8c055ed0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:07.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.409+0000 7f6fb4fab700 1 --2- 172.21.15.17:0/3853899178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fb013f920 0x7f6fb00ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:07.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.409+0000 7f6fb4fab700 1 --2- 172.21.15.17:0/3853899178 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6fb00aa050 0x7f6fb00adca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:07.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.409+0000 7f6fb4fab700 1 --2- 172.21.15.17:0/3853899178 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fb0060560 0x7f6fb0140e30 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:07.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.409+0000 7f6fb4fab700 1 -- 172.21.15.17:0/3853899178 >> 172.21.15.17:0/3853899178 conn(0x7f6fb00b2b00 msgr2=0x7f6fb00b0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:07.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.409+0000 7f6fb4fab700 1 -- 172.21.15.17:0/3853899178 shutdown_connections 2022-08-01T19:43:07.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.409+0000 7f6fb4fab700 1 -- 172.21.15.17:0/3853899178 wait complete. 2022-08-01T19:43:07.424 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:43:07.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.529+0000 7f9557e67700 1 Processor -- start 2022-08-01T19:43:07.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.529+0000 7f9557e67700 1 -- start start 2022-08-01T19:43:07.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.529+0000 7f9557e67700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f95500a9af0 0x7f95500b1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:07.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.529+0000 7f9557e67700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f95500ac3f0 0x7f95500ab130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:07.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.529+0000 7f9557e67700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f95500b0850 0x7f95500aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:07.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.529+0000 7f9557e67700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95500b1df0 con 0x7f95500a9af0 2022-08-01T19:43:07.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.529+0000 7f9557e67700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95500a9ed0 con 0x7f95500b0850 2022-08-01T19:43:07.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.529+0000 7f9557e67700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95500b2ad0 con 0x7f95500ac3f0 2022-08-01T19:43:07.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.529+0000 7f9555c03700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f95500a9af0 0x7f95500b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:07.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.529+0000 7f9555c03700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f95500a9af0 0x7f95500b1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44418/0 (socket says 172.21.15.17:44418) 2022-08-01T19:43:07.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.529+0000 7f9555c03700 1 -- 172.21.15.17:0/855389075 learned_addr learned my addr 172.21.15.17:0/855389075 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:07.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.529+0000 7f9556404700 1 --2- 172.21.15.17:0/855389075 >> v2:172.21.15.77:3300/0 conn(0x7f95500b0850 0x7f95500aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:07.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.529+0000 7f9555402700 1 --2- 172.21.15.17:0/855389075 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f95500ac3f0 0x7f95500ab130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:07.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.533+0000 7f9555c03700 1 -- 172.21.15.17:0/855389075 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f95500ac3f0 msgr2=0x7f95500ab130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:07.547 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.533+0000 7f9555c03700 1 --2- 172.21.15.17:0/855389075 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f95500ac3f0 0x7f95500ab130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:07.547 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.533+0000 7f9555c03700 1 -- 172.21.15.17:0/855389075 >> v2:172.21.15.77:3300/0 conn(0x7f95500b0850 msgr2=0x7f95500aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:07.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.533+0000 7f9555c03700 1 --2- 172.21.15.17:0/855389075 >> v2:172.21.15.77:3300/0 conn(0x7f95500b0850 0x7f95500aedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:07.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.533+0000 7f9555c03700 1 -- 172.21.15.17:0/855389075 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95500b0250 con 0x7f95500a9af0 2022-08-01T19:43:07.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.533+0000 7f9555402700 1 --2- 172.21.15.17:0/855389075 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f95500ac3f0 0x7f95500ab130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:43:07.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.533+0000 7f9555c03700 1 --2- 172.21.15.17:0/855389075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95500a9af0 0x7f95500b1900 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 rx=0x7f954c002f70 tx=0x7f954c006eb0).ready entity=mon.0 client_cookie=e01de7d11648ad70 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:07.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.533+0000 7f9554c01700 1 -- 172.21.15.17:0/855389075 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f954c00d570 con 0x7f95500a9af0 2022-08-01T19:43:07.550 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.533+0000 7f9554c01700 1 -- 172.21.15.17:0/855389075 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954c000e90 con 0x7f95500a9af0 2022-08-01T19:43:07.550 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.533+0000 7f9554c01700 1 -- 172.21.15.17:0/855389075 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f954c00d900 con 0x7f95500a9af0 2022-08-01T19:43:07.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.533+0000 7f9557e67700 1 -- 172.21.15.17:0/855389075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95500a9af0 msgr2=0x7f95500b1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:07.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.533+0000 7f9557e67700 1 --2- 172.21.15.17:0/855389075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95500a9af0 0x7f95500b1900 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 rx=0x7f954c002f70 tx=0x7f954c006eb0).stop 2022-08-01T19:43:07.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.533+0000 7f9557e67700 1 -- 172.21.15.17:0/855389075 shutdown_connections 2022-08-01T19:43:07.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.533+0000 7f9557e67700 1 --2- 172.21.15.17:0/855389075 >> v2:172.21.15.77:3300/0 conn(0x7f95500b0850 0x7f95500aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:07.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.533+0000 7f9557e67700 1 --2- 172.21.15.17:0/855389075 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f95500ac3f0 0x7f95500ab130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:07.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.533+0000 7f9557e67700 1 --2- 172.21.15.17:0/855389075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95500a9af0 0x7f95500b1900 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:07.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.533+0000 7f9557e67700 1 -- 172.21.15.17:0/855389075 >> 172.21.15.17:0/855389075 conn(0x7f955005f840 msgr2=0x7f955005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:07.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.533+0000 7f9557e67700 1 -- 172.21.15.17:0/855389075 shutdown_connections 2022-08-01T19:43:07.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.533+0000 7f9557e67700 1 -- 172.21.15.17:0/855389075 wait complete. 2022-08-01T19:43:07.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.533+0000 7f9557e67700 1 Processor -- start 2022-08-01T19:43:07.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.533+0000 7f9557e67700 1 -- start start 2022-08-01T19:43:07.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.537+0000 7f9557e67700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95500a9af0 0x7f9550065760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:07.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.537+0000 7f9557e67700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95500ac3f0 0x7f955009ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:07.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.537+0000 7f9557e67700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f95500b0850 0x7f95501448d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:07.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.537+0000 7f9557e67700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955009f0e0 con 0x7f95500a9af0 2022-08-01T19:43:07.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.537+0000 7f9557e67700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955009f380 con 0x7f95500ac3f0 2022-08-01T19:43:07.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.537+0000 7f9557e67700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9550150da0 con 0x7f95500b0850 2022-08-01T19:43:07.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.537+0000 7f9555c03700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95500a9af0 0x7f9550065760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:07.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.537+0000 7f9555c03700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95500a9af0 0x7f9550065760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44424/0 (socket says 172.21.15.17:44424) 2022-08-01T19:43:07.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.537+0000 7f9555c03700 1 -- 172.21.15.17:0/131926837 learned_addr learned my addr 172.21.15.17:0/131926837 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:07.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.537+0000 7f9556404700 1 --2- 172.21.15.17:0/131926837 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f95500b0850 0x7f95501448d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:07.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.537+0000 7f9555402700 1 --2- 172.21.15.17:0/131926837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95500ac3f0 0x7f955009ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:07.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.537+0000 7f9555c03700 1 -- 172.21.15.17:0/131926837 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f95500b0850 msgr2=0x7f95501448d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:07.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.537+0000 7f9555c03700 1 --2- 172.21.15.17:0/131926837 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f95500b0850 0x7f95501448d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:07.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.537+0000 7f9555c03700 1 -- 172.21.15.17:0/131926837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95500ac3f0 msgr2=0x7f955009ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:07.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.537+0000 7f9555c03700 1 --2- 172.21.15.17:0/131926837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95500ac3f0 0x7f955009ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:07.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.537+0000 7f9555c03700 1 -- 172.21.15.17:0/131926837 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954c00b1e0 con 0x7f95500a9af0 2022-08-01T19:43:07.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.537+0000 7f9556404700 1 --2- 172.21.15.17:0/131926837 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f95500b0850 0x7f95501448d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:43:07.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.537+0000 7f9555402700 1 --2- 172.21.15.17:0/131926837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95500ac3f0 0x7f955009ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:43:07.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.537+0000 7f9555c03700 1 --2- 172.21.15.17:0/131926837 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95500a9af0 0x7f9550065760 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 rx=0x7f954c002f70 tx=0x7f954c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:07.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.537+0000 7f9542ffd700 1 -- 172.21.15.17:0/131926837 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f954c006b40 con 0x7f95500a9af0 2022-08-01T19:43:07.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.537+0000 7f9542ffd700 1 -- 172.21.15.17:0/131926837 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954c011ce0 con 0x7f95500a9af0 2022-08-01T19:43:07.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.537+0000 7f9557e67700 1 -- 172.21.15.17:0/131926837 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9550151040 con 0x7f95500a9af0 2022-08-01T19:43:07.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.537+0000 7f9542ffd700 1 -- 172.21.15.17:0/131926837 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f954c01e650 con 0x7f95500a9af0 2022-08-01T19:43:07.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.537+0000 7f9557e67700 1 -- 172.21.15.17:0/131926837 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9550145610 con 0x7f95500a9af0 2022-08-01T19:43:07.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.537+0000 7f9557e67700 1 -- 172.21.15.17:0/131926837 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95500574e0 con 0x7f95500a9af0 2022-08-01T19:43:07.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.537+0000 7f9542ffd700 1 -- 172.21.15.17:0/131926837 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f954c01a040 con 0x7f95500a9af0 2022-08-01T19:43:07.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.537+0000 7f9542ffd700 1 --2- 172.21.15.17:0/131926837 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f95300539e0 0x7f9530055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:07.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.541+0000 7f9542ffd700 1 -- 172.21.15.17:0/131926837 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f954c06cf90 con 0x7f95500a9af0 2022-08-01T19:43:07.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.541+0000 7f9555402700 1 --2- 172.21.15.17:0/131926837 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f95300539e0 0x7f9530055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:07.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.541+0000 7f9555402700 1 --2- 172.21.15.17:0/131926837 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f95300539e0 0x7f9530055ea0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f9544008000 tx=0x7f9544008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:07.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.545+0000 7f9542ffd700 1 -- 172.21.15.17:0/131926837 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f954c043cb0 con 0x7f95500a9af0 2022-08-01T19:43:07.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.665+0000 7f9557e67700 1 -- 172.21.15.17:0/131926837 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f95500b2f90 con 0x7f95500a9af0 2022-08-01T19:43:07.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.665+0000 7f9542ffd700 1 -- 172.21.15.17:0/131926837 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v137) v1 ==== 92+0+10 (secure 0 0 0) 0x7f954c023020 con 0x7f95500a9af0 2022-08-01T19:43:07.670 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:43:07.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.669+0000 7f9557e67700 1 -- 172.21.15.17:0/131926837 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f95300539e0 msgr2=0x7f9530055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:07.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.669+0000 7f9557e67700 1 --2- 172.21.15.17:0/131926837 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f95300539e0 0x7f9530055ea0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f9544008000 tx=0x7f9544008040).stop 2022-08-01T19:43:07.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.669+0000 7f9557e67700 1 -- 172.21.15.17:0/131926837 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95500a9af0 msgr2=0x7f9550065760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:07.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.669+0000 7f9557e67700 1 --2- 172.21.15.17:0/131926837 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95500a9af0 0x7f9550065760 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 rx=0x7f954c002f70 tx=0x7f954c015000).stop 2022-08-01T19:43:07.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.669+0000 7f9557e67700 1 -- 172.21.15.17:0/131926837 shutdown_connections 2022-08-01T19:43:07.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.669+0000 7f9557e67700 1 --2- 172.21.15.17:0/131926837 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f95300539e0 0x7f9530055ea0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:07.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.669+0000 7f9557e67700 1 --2- 172.21.15.17:0/131926837 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f95500b0850 0x7f95501448d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:07.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.669+0000 7f9557e67700 1 --2- 172.21.15.17:0/131926837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95500ac3f0 0x7f955009ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:07.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.669+0000 7f9557e67700 1 --2- 172.21.15.17:0/131926837 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95500a9af0 0x7f9550065760 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:07.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.669+0000 7f9557e67700 1 -- 172.21.15.17:0/131926837 >> 172.21.15.17:0/131926837 conn(0x7f955005f840 msgr2=0x7f955013da30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:07.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.669+0000 7f9557e67700 1 -- 172.21.15.17:0/131926837 shutdown_connections 2022-08-01T19:43:07.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.669+0000 7f9557e67700 1 -- 172.21.15.17:0/131926837 wait complete. 2022-08-01T19:43:07.689 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T19:43:07.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.797+0000 7f72267f4700 1 Processor -- start 2022-08-01T19:43:07.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.797+0000 7f72267f4700 1 -- start start 2022-08-01T19:43:07.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.797+0000 7f72267f4700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7220146ae0 0x7f72201403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:07.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.797+0000 7f72267f4700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f72200b4310 0x7f72201408b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:07.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.797+0000 7f72267f4700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f72200b4820 0x7f72200b5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:07.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.797+0000 7f72267f4700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72201476a0 con 0x7f7220146ae0 2022-08-01T19:43:07.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.797+0000 7f72267f4700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7220143cf0 con 0x7f72200b4310 2022-08-01T19:43:07.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.797+0000 7f72267f4700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7220143f90 con 0x7f72200b4820 2022-08-01T19:43:07.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.801+0000 7f721ffff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7220146ae0 0x7f72201403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:07.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.801+0000 7f721ffff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7220146ae0 0x7f72201403c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44432/0 (socket says 172.21.15.17:44432) 2022-08-01T19:43:07.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.801+0000 7f7224d91700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f72200b4820 0x7f72200b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:07.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.801+0000 7f721ffff700 1 -- 172.21.15.17:0/3838132789 learned_addr learned my addr 172.21.15.17:0/3838132789 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:07.807 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.801+0000 7f7217fff700 1 --2- 172.21.15.17:0/3838132789 >> v2:172.21.15.77:3300/0 conn(0x7f72200b4310 0x7f72201408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:07.807 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.801+0000 7f721ffff700 1 -- 172.21.15.17:0/3838132789 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f72200b4820 msgr2=0x7f72200b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:07.807 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.801+0000 7f721ffff700 1 --2- 172.21.15.17:0/3838132789 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f72200b4820 0x7f72200b5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:07.807 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.801+0000 7f721ffff700 1 -- 172.21.15.17:0/3838132789 >> v2:172.21.15.77:3300/0 conn(0x7f72200b4310 msgr2=0x7f72201408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:07.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.801+0000 7f721ffff700 1 --2- 172.21.15.17:0/3838132789 >> v2:172.21.15.77:3300/0 conn(0x7f72200b4310 0x7f72201408b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:07.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.801+0000 7f721ffff700 1 -- 172.21.15.17:0/3838132789 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72200b0e20 con 0x7f7220146ae0 2022-08-01T19:43:07.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.801+0000 7f721ffff700 1 --2- 172.21.15.17:0/3838132789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7220146ae0 0x7f72201403c0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 rx=0x7f7210002f70 tx=0x7f7210006eb0).ready entity=mon.0 client_cookie=18c7b0d0593428d7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:07.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.801+0000 7f721f7fe700 1 -- 172.21.15.17:0/3838132789 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f721000d570 con 0x7f7220146ae0 2022-08-01T19:43:07.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.801+0000 7f721f7fe700 1 -- 172.21.15.17:0/3838132789 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7210000e90 con 0x7f7220146ae0 2022-08-01T19:43:07.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.801+0000 7f721f7fe700 1 -- 172.21.15.17:0/3838132789 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f721000d900 con 0x7f7220146ae0 2022-08-01T19:43:07.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.801+0000 7f72267f4700 1 -- 172.21.15.17:0/3838132789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7220146ae0 msgr2=0x7f72201403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:07.815 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.801+0000 7f72267f4700 1 --2- 172.21.15.17:0/3838132789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7220146ae0 0x7f72201403c0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 rx=0x7f7210002f70 tx=0x7f7210006eb0).stop 2022-08-01T19:43:07.815 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.801+0000 7f72267f4700 1 -- 172.21.15.17:0/3838132789 shutdown_connections 2022-08-01T19:43:07.815 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.801+0000 7f72267f4700 1 --2- 172.21.15.17:0/3838132789 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f72200b4820 0x7f72200b5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:07.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.801+0000 7f72267f4700 1 --2- 172.21.15.17:0/3838132789 >> v2:172.21.15.77:3300/0 conn(0x7f72200b4310 0x7f72201408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:07.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.801+0000 7f72267f4700 1 --2- 172.21.15.17:0/3838132789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7220146ae0 0x7f72201403c0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:07.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.801+0000 7f72267f4700 1 -- 172.21.15.17:0/3838132789 >> 172.21.15.17:0/3838132789 conn(0x7f722005c620 msgr2=0x7f722005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:07.822 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.801+0000 7f72267f4700 1 -- 172.21.15.17:0/3838132789 shutdown_connections 2022-08-01T19:43:07.822 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.801+0000 7f72267f4700 1 -- 172.21.15.17:0/3838132789 wait complete. 2022-08-01T19:43:07.822 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.805+0000 7f72267f4700 1 Processor -- start 2022-08-01T19:43:07.822 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.805+0000 7f72267f4700 1 -- start start 2022-08-01T19:43:07.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.805+0000 7f72267f4700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f72200b4310 0x7f72201524e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:07.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.805+0000 7f72267f4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72200b4820 0x7f722013e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:07.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.805+0000 7f72267f4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7220146ae0 0x7f7220155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:07.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.805+0000 7f72267f4700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72200b7c80 con 0x7f7220146ae0 2022-08-01T19:43:07.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.805+0000 7f72267f4700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7220155f80 con 0x7f72200b4820 2022-08-01T19:43:07.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.805+0000 7f72267f4700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7220156980 con 0x7f72200b4310 2022-08-01T19:43:07.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.805+0000 7f7224d91700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7220146ae0 0x7f7220155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:07.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.805+0000 7f7224d91700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7220146ae0 0x7f7220155a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44442/0 (socket says 172.21.15.17:44442) 2022-08-01T19:43:07.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.805+0000 7f7224d91700 1 -- 172.21.15.17:0/1320324287 learned_addr learned my addr 172.21.15.17:0/1320324287 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:07.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.805+0000 7f7217fff700 1 --2- 172.21.15.17:0/1320324287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72200b4820 0x7f722013e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:07.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.805+0000 7f721ffff700 1 --2- 172.21.15.17:0/1320324287 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f72200b4310 0x7f72201524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:07.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.805+0000 7f7224d91700 1 -- 172.21.15.17:0/1320324287 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f72200b4310 msgr2=0x7f72201524e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:07.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.805+0000 7f7224d91700 1 --2- 172.21.15.17:0/1320324287 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f72200b4310 0x7f72201524e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:07.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.805+0000 7f7224d91700 1 -- 172.21.15.17:0/1320324287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72200b4820 msgr2=0x7f722013e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:07.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.805+0000 7f7224d91700 1 --2- 172.21.15.17:0/1320324287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72200b4820 0x7f722013e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:07.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.805+0000 7f7224d91700 1 -- 172.21.15.17:0/1320324287 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721000b1e0 con 0x7f7220146ae0 2022-08-01T19:43:07.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.805+0000 7f7224d91700 1 --2- 172.21.15.17:0/1320324287 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7220146ae0 0x7f7220155a90 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 rx=0x7f72201560c0 tx=0x7f72180091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:07.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.805+0000 7f721dffb700 1 -- 172.21.15.17:0/1320324287 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7218012600 con 0x7f7220146ae0 2022-08-01T19:43:07.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.805+0000 7f721dffb700 1 -- 172.21.15.17:0/1320324287 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7218012760 con 0x7f7220146ae0 2022-08-01T19:43:07.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.805+0000 7f721dffb700 1 -- 172.21.15.17:0/1320324287 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7218012ac0 con 0x7f7220146ae0 2022-08-01T19:43:07.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.805+0000 7f72267f4700 1 -- 172.21.15.17:0/1320324287 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72201561a0 con 0x7f7220146ae0 2022-08-01T19:43:07.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.805+0000 7f72267f4700 1 -- 172.21.15.17:0/1320324287 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72201566e0 con 0x7f7220146ae0 2022-08-01T19:43:07.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.809+0000 7f72267f4700 1 -- 172.21.15.17:0/1320324287 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72200b84e0 con 0x7f7220146ae0 2022-08-01T19:43:07.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.809+0000 7f721dffb700 1 -- 172.21.15.17:0/1320324287 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7218022070 con 0x7f7220146ae0 2022-08-01T19:43:07.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.809+0000 7f721dffb700 1 --2- 172.21.15.17:0/1320324287 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f72040539e0 0x7f7204055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:07.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.809+0000 7f721dffb700 1 -- 172.21.15.17:0/1320324287 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f721806c4b0 con 0x7f7220146ae0 2022-08-01T19:43:07.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.809+0000 7f721ffff700 1 --2- 172.21.15.17:0/1320324287 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f72040539e0 0x7f7204055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:07.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.809+0000 7f721ffff700 1 --2- 172.21.15.17:0/1320324287 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f72040539e0 0x7f7204055ea0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7f721001c000 tx=0x7f7210015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:07.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.817+0000 7f721dffb700 1 -- 172.21.15.17:0/1320324287 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7218043270 con 0x7f7220146ae0 2022-08-01T19:43:07.945 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.941+0000 7f72267f4700 1 -- 172.21.15.17:0/1320324287 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f72200574e0 con 0x7f7220146ae0 2022-08-01T19:43:07.946 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.945+0000 7f721dffb700 1 -- 172.21.15.17:0/1320324287 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v137) v1 ==== 104+0+10 (secure 0 0 0) 0x7f7218016020 con 0x7f7220146ae0 2022-08-01T19:43:07.948 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:43:07.949 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.945+0000 7f72267f4700 1 -- 172.21.15.17:0/1320324287 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f72040539e0 msgr2=0x7f7204055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:07.950 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.945+0000 7f72267f4700 1 --2- 172.21.15.17:0/1320324287 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f72040539e0 0x7f7204055ea0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7f721001c000 tx=0x7f7210015000).stop 2022-08-01T19:43:07.950 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.945+0000 7f72267f4700 1 -- 172.21.15.17:0/1320324287 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7220146ae0 msgr2=0x7f7220155a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:07.950 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.945+0000 7f72267f4700 1 --2- 172.21.15.17:0/1320324287 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7220146ae0 0x7f7220155a90 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 rx=0x7f72201560c0 tx=0x7f72180091d0).stop 2022-08-01T19:43:07.951 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.945+0000 7f72267f4700 1 -- 172.21.15.17:0/1320324287 shutdown_connections 2022-08-01T19:43:07.951 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.945+0000 7f72267f4700 1 --2- 172.21.15.17:0/1320324287 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f72040539e0 0x7f7204055ea0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:07.951 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.945+0000 7f72267f4700 1 --2- 172.21.15.17:0/1320324287 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7220146ae0 0x7f7220155a90 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:07.951 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.945+0000 7f72267f4700 1 --2- 172.21.15.17:0/1320324287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72200b4820 0x7f722013e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:07.952 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.945+0000 7f72267f4700 1 --2- 172.21.15.17:0/1320324287 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f72200b4310 0x7f72201524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:07.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.945+0000 7f72267f4700 1 -- 172.21.15.17:0/1320324287 >> 172.21.15.17:0/1320324287 conn(0x7f722005c620 msgr2=0x7f72200616e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:07.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.945+0000 7f72267f4700 1 -- 172.21.15.17:0/1320324287 shutdown_connections 2022-08-01T19:43:07.953 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:07.945+0000 7f72267f4700 1 -- 172.21.15.17:0/1320324287 wait complete. 2022-08-01T19:43:07.960 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T19:43:08.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.069+0000 7f4697fff700 1 Processor -- start 2022-08-01T19:43:08.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.069+0000 7f4697fff700 1 -- start start 2022-08-01T19:43:08.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.069+0000 7f4697fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f46980b31f0 0x7f46980b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:08.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.069+0000 7f4697fff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f46980aadb0 0x7f4698064d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:08.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.069+0000 7f4697fff700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46980b2510 con 0x7f46980b7500 2022-08-01T19:43:08.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.069+0000 7f4697fff700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46981443e0 con 0x7f46980b31f0 2022-08-01T19:43:08.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.069+0000 7f4697fff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46980b3700 con 0x7f46980aadb0 2022-08-01T19:43:08.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.069+0000 7f46967fc700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f46980b31f0 0x7f46980b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:08.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.069+0000 7f46967fc700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f46980b31f0 0x7f46980b78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44448/0 (socket says 172.21.15.17:44448) 2022-08-01T19:43:08.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.069+0000 7f46967fc700 1 -- 172.21.15.17:0/1321853495 learned_addr learned my addr 172.21.15.17:0/1321853495 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:08.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.069+0000 7f46977fe700 1 --2- 172.21.15.17:0/1321853495 >> v2:172.21.15.77:3300/0 conn(0x7f46980aadb0 0x7f4698064d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:08.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.069+0000 7f46977fe700 1 -- 172.21.15.17:0/1321853495 >> v2:172.21.15.17:3300/0 conn(0x7f46980b31f0 msgr2=0x7f46980b78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:08.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.069+0000 7f46977fe700 1 --2- 172.21.15.17:0/1321853495 >> v2:172.21.15.17:3300/0 conn(0x7f46980b31f0 0x7f46980b78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:08.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.069+0000 7f46977fe700 1 -- 172.21.15.17:0/1321853495 >> v1:172.21.15.77:6789/0 conn(0x7f46980b7500 legacy=0x7f469813e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:08.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.069+0000 7f46977fe700 1 -- 172.21.15.17:0/1321853495 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46980b2740 con 0x7f46980aadb0 2022-08-01T19:43:08.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.069+0000 7f4695ffb700 1 -- 172.21.15.17:0/1321853495 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1375011788 0 0) 0x7f46980b2510 con 0x7f46980b7500 2022-08-01T19:43:08.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.073+0000 7f46977fe700 1 --2- 172.21.15.17:0/1321853495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46980aadb0 0x7f4698064d30 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f4690002f70 tx=0x7f4690006df0).ready entity=mon.1 client_cookie=143b782647acc509 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:08.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.073+0000 7f4695ffb700 1 -- 172.21.15.17:0/1321853495 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f469000d570 con 0x7f46980aadb0 2022-08-01T19:43:08.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.073+0000 7f4695ffb700 1 -- 172.21.15.17:0/1321853495 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4690000e90 con 0x7f46980aadb0 2022-08-01T19:43:08.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.073+0000 7f4695ffb700 1 -- 172.21.15.17:0/1321853495 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f469000d8c0 con 0x7f46980aadb0 2022-08-01T19:43:08.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.073+0000 7f4697fff700 1 -- 172.21.15.17:0/1321853495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46980aadb0 msgr2=0x7f4698064d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:08.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.073+0000 7f4697fff700 1 --2- 172.21.15.17:0/1321853495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46980aadb0 0x7f4698064d30 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f4690002f70 tx=0x7f4690006df0).stop 2022-08-01T19:43:08.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.073+0000 7f4697fff700 1 -- 172.21.15.17:0/1321853495 shutdown_connections 2022-08-01T19:43:08.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.073+0000 7f4697fff700 1 --2- 172.21.15.17:0/1321853495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46980aadb0 0x7f4698064d30 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:08.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.073+0000 7f4697fff700 1 --2- 172.21.15.17:0/1321853495 >> v2:172.21.15.17:3300/0 conn(0x7f46980b31f0 0x7f46980b78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:08.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.073+0000 7f4697fff700 1 -- 172.21.15.17:0/1321853495 >> 172.21.15.17:0/1321853495 conn(0x7f469805f840 msgr2=0x7f469805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:08.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.073+0000 7f4697fff700 1 -- 172.21.15.17:0/1321853495 shutdown_connections 2022-08-01T19:43:08.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.073+0000 7f4697fff700 1 -- 172.21.15.17:0/1321853495 wait complete. 2022-08-01T19:43:08.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.073+0000 7f4697fff700 1 Processor -- start 2022-08-01T19:43:08.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.073+0000 7f4697fff700 1 -- start start 2022-08-01T19:43:08.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.077+0000 7f4697fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46980aadb0 0x7f4698140730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:08.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.077+0000 7f4697fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46980b31f0 0x7f469813f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:08.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.077+0000 7f4697fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46980b7500 0x7f4698060a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:08.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.077+0000 7f4697fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469813c300 con 0x7f46980aadb0 2022-08-01T19:43:08.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.077+0000 7f4697fff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4698143dd0 con 0x7f46980b31f0 2022-08-01T19:43:08.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.077+0000 7f4697fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4698143a30 con 0x7f46980b7500 2022-08-01T19:43:08.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.077+0000 7f4696ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46980aadb0 0x7f4698140730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:08.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.077+0000 7f4696ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46980aadb0 0x7f4698140730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44452/0 (socket says 172.21.15.17:44452) 2022-08-01T19:43:08.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.077+0000 7f4696ffd700 1 -- 172.21.15.17:0/3018570100 learned_addr learned my addr 172.21.15.17:0/3018570100 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:08.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.077+0000 7f46967fc700 1 --2- 172.21.15.17:0/3018570100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46980b31f0 0x7f469813f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:08.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.077+0000 7f46977fe700 1 --2- 172.21.15.17:0/3018570100 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46980b7500 0x7f4698060a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:08.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.077+0000 7f4696ffd700 1 -- 172.21.15.17:0/3018570100 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46980b7500 msgr2=0x7f4698060a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:08.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.077+0000 7f4696ffd700 1 --2- 172.21.15.17:0/3018570100 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46980b7500 0x7f4698060a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:08.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.077+0000 7f4696ffd700 1 -- 172.21.15.17:0/3018570100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46980b31f0 msgr2=0x7f469813f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:08.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.077+0000 7f4696ffd700 1 --2- 172.21.15.17:0/3018570100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46980b31f0 0x7f469813f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:08.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.077+0000 7f4696ffd700 1 -- 172.21.15.17:0/3018570100 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469000b180 con 0x7f46980aadb0 2022-08-01T19:43:08.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.077+0000 7f46977fe700 1 --2- 172.21.15.17:0/3018570100 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46980b7500 0x7f4698060a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:43:08.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.077+0000 7f4696ffd700 1 --2- 172.21.15.17:0/3018570100 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46980aadb0 0x7f4698140730 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 rx=0x7f4680000f20 tx=0x7f4680007ca0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:08.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.077+0000 7f468ffff700 1 -- 172.21.15.17:0/3018570100 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4680001d90 con 0x7f46980aadb0 2022-08-01T19:43:08.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.077+0000 7f468ffff700 1 -- 172.21.15.17:0/3018570100 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468000f470 con 0x7f46980aadb0 2022-08-01T19:43:08.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.077+0000 7f4697fff700 1 -- 172.21.15.17:0/3018570100 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46980b20a0 con 0x7f46980aadb0 2022-08-01T19:43:08.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.077+0000 7f468ffff700 1 -- 172.21.15.17:0/3018570100 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f468000f7d0 con 0x7f46980aadb0 2022-08-01T19:43:08.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.077+0000 7f4697fff700 1 -- 172.21.15.17:0/3018570100 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4698144030 con 0x7f46980aadb0 2022-08-01T19:43:08.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.077+0000 7f468ffff700 1 -- 172.21.15.17:0/3018570100 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4680019030 con 0x7f46980aadb0 2022-08-01T19:43:08.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.077+0000 7f4697fff700 1 -- 172.21.15.17:0/3018570100 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4698143550 con 0x7f46980aadb0 2022-08-01T19:43:08.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.077+0000 7f468ffff700 1 --2- 172.21.15.17:0/3018570100 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f467c053b30 0x7f467c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:08.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.081+0000 7f468ffff700 1 -- 172.21.15.17:0/3018570100 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f468006b5c0 con 0x7f46980aadb0 2022-08-01T19:43:08.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.081+0000 7f46967fc700 1 --2- 172.21.15.17:0/3018570100 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f467c053b30 0x7f467c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:08.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.081+0000 7f46967fc700 1 --2- 172.21.15.17:0/3018570100 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f467c053b30 0x7f467c055ff0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f468800b000 tx=0x7f4688008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:08.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.085+0000 7f468ffff700 1 -- 172.21.15.17:0/3018570100 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4698143550 con 0x7f46980aadb0 2022-08-01T19:43:08.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.209+0000 7f4697fff700 1 -- 172.21.15.17:0/3018570100 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4698143550 con 0x7f46980aadb0 2022-08-01T19:43:08.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.209+0000 7f468ffff700 1 -- 172.21.15.17:0/3018570100 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v137) v1 ==== 100+0+10 (secure 0 0 0) 0x7f4698143550 con 0x7f46980aadb0 2022-08-01T19:43:08.211 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:43:08.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.209+0000 7f4697fff700 1 -- 172.21.15.17:0/3018570100 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f467c053b30 msgr2=0x7f467c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:08.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.209+0000 7f4697fff700 1 --2- 172.21.15.17:0/3018570100 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f467c053b30 0x7f467c055ff0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f468800b000 tx=0x7f4688008040).stop 2022-08-01T19:43:08.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.209+0000 7f4697fff700 1 -- 172.21.15.17:0/3018570100 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46980aadb0 msgr2=0x7f4698140730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:08.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.209+0000 7f4697fff700 1 --2- 172.21.15.17:0/3018570100 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46980aadb0 0x7f4698140730 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 rx=0x7f4680000f20 tx=0x7f4680007ca0).stop 2022-08-01T19:43:08.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.209+0000 7f4697fff700 1 -- 172.21.15.17:0/3018570100 shutdown_connections 2022-08-01T19:43:08.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.209+0000 7f4697fff700 1 --2- 172.21.15.17:0/3018570100 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f467c053b30 0x7f467c055ff0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:08.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.209+0000 7f4697fff700 1 --2- 172.21.15.17:0/3018570100 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46980b7500 0x7f4698060a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:08.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.209+0000 7f4697fff700 1 --2- 172.21.15.17:0/3018570100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46980b31f0 0x7f469813f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:08.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.209+0000 7f4697fff700 1 --2- 172.21.15.17:0/3018570100 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46980aadb0 0x7f4698140730 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:08.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.209+0000 7f4697fff700 1 -- 172.21.15.17:0/3018570100 >> 172.21.15.17:0/3018570100 conn(0x7f469805f840 msgr2=0x7f46980b5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:08.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.209+0000 7f4697fff700 1 -- 172.21.15.17:0/3018570100 shutdown_connections 2022-08-01T19:43:08.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.209+0000 7f4697fff700 1 -- 172.21.15.17:0/3018570100 wait complete. 2022-08-01T19:43:08.225 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T19:43:08.226 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:43:08.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.321+0000 7f46d1a9d700 1 Processor -- start 2022-08-01T19:43:08.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.321+0000 7f46d1a9d700 1 -- start start 2022-08-01T19:43:08.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46d1a9d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46cc0b1ea0 0x7f46cc147600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:08.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46d1a9d700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46cc14ba70 con 0x7f46cc0b2930 2022-08-01T19:43:08.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46d1a9d700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46cc0b2d10 con 0x7f46cc05c620 2022-08-01T19:43:08.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46d1a9d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc145340 con 0x7f46cc0b1ea0 2022-08-01T19:43:08.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46d0a9b700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f46cc0b2930 0x7f46cc14b360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36654/0 (socket says 172.21.15.17:36654) 2022-08-01T19:43:08.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46d0a9b700 1 -- 172.21.15.17:0/2256943819 learned_addr learned my addr 172.21.15.17:0/2256943819 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:08.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46cbfff700 1 --2- 172.21.15.17:0/2256943819 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46cc0b1ea0 0x7f46cc147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:08.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46cb7fe700 1 -- 172.21.15.17:0/2256943819 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3459758850 0 0) 0x7f46cc14ba70 con 0x7f46cc0b2930 2022-08-01T19:43:08.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46cb7fe700 1 -- 172.21.15.17:0/2256943819 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46b0003260 con 0x7f46cc0b2930 2022-08-01T19:43:08.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46cb7fe700 1 -- 172.21.15.17:0/2256943819 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1057814415 0 0) 0x7f46cc0b2d10 con 0x7f46cc05c620 2022-08-01T19:43:08.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46cb7fe700 1 -- 172.21.15.17:0/2256943819 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46cc14ba70 con 0x7f46cc05c620 2022-08-01T19:43:08.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46cb7fe700 1 -- 172.21.15.17:0/2256943819 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3852471850 0 0) 0x7f46b0003260 con 0x7f46cc0b2930 2022-08-01T19:43:08.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46cb7fe700 1 -- 172.21.15.17:0/2256943819 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46cc0b1ea0 msgr2=0x7f46cc147600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:08.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46cb7fe700 1 --2- 172.21.15.17:0/2256943819 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46cc0b1ea0 0x7f46cc147600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:08.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46cb7fe700 1 -- 172.21.15.17:0/2256943819 >> v1:172.21.15.77:6789/0 conn(0x7f46cc05c620 legacy=0x7f46cc0a8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:08.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46cb7fe700 1 -- 172.21.15.17:0/2256943819 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46cc14ac60 con 0x7f46cc0b2930 2022-08-01T19:43:08.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46cb7fe700 1 -- 172.21.15.17:0/2256943819 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7f46c0001660 con 0x7f46cc0b2930 2022-08-01T19:43:08.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46cb7fe700 1 -- 172.21.15.17:0/2256943819 >> v1:172.21.15.17:6789/0 conn(0x7f46cc0b2930 legacy=0x7f46cc14b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:08.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46cb7fe700 1 --2- 172.21.15.17:0/2256943819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46b0003910 0x7f46b0003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:08.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46cb7fe700 1 --2- 172.21.15.17:0/2256943819 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46b0004500 0x7f46b0004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:08.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46cb7fe700 1 --2- 172.21.15.17:0/2256943819 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46cc0b1ea0 0x7f46b0004e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:08.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46cb7fe700 1 -- 172.21.15.17:0/2256943819 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc145340 con 0x7f46b0004500 2022-08-01T19:43:08.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46cb7fe700 1 -- 172.21.15.17:0/2256943819 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b000b800 con 0x7f46b0003910 2022-08-01T19:43:08.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46cb7fe700 1 -- 172.21.15.17:0/2256943819 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46b000b990 con 0x7f46cc0b1ea0 2022-08-01T19:43:08.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46cbfff700 1 --2- 172.21.15.17:0/2256943819 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46b0004500 0x7f46b0004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:08.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46d0a9b700 1 --2- 172.21.15.17:0/2256943819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46b0003910 0x7f46b0003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:08.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46d129c700 1 --2- 172.21.15.17:0/2256943819 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46cc0b1ea0 0x7f46b0004e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:08.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46cbfff700 1 -- 172.21.15.17:0/2256943819 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46cc0b1ea0 msgr2=0x7f46b0004e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:08.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46cbfff700 1 --2- 172.21.15.17:0/2256943819 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46cc0b1ea0 0x7f46b0004e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:08.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46cbfff700 1 -- 172.21.15.17:0/2256943819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46b0003910 msgr2=0x7f46b0003d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:08.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46cbfff700 1 --2- 172.21.15.17:0/2256943819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46b0003910 0x7f46b0003d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:08.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46cbfff700 1 -- 172.21.15.17:0/2256943819 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b0003260 con 0x7f46b0004500 2022-08-01T19:43:08.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46d129c700 1 --2- 172.21.15.17:0/2256943819 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46cc0b1ea0 0x7f46b0004e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:43:08.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46cbfff700 1 --2- 172.21.15.17:0/2256943819 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46b0004500 0x7f46b0004980 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 rx=0x7f46b00054d0 tx=0x7f46b8006df0).ready entity=mon.0 client_cookie=b5aced08014a63db server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:08.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46cb7fe700 1 -- 172.21.15.17:0/2256943819 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46b800e830 con 0x7f46b0004500 2022-08-01T19:43:08.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46cb7fe700 1 -- 172.21.15.17:0/2256943819 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46b800e990 con 0x7f46b0004500 2022-08-01T19:43:08.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46cb7fe700 1 -- 172.21.15.17:0/2256943819 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46b800ecf0 con 0x7f46b0004500 2022-08-01T19:43:08.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46d1a9d700 1 -- 172.21.15.17:0/2256943819 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46b0004500 msgr2=0x7f46b0004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:08.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46d1a9d700 1 --2- 172.21.15.17:0/2256943819 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46b0004500 0x7f46b0004980 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 rx=0x7f46b00054d0 tx=0x7f46b8006df0).stop 2022-08-01T19:43:08.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46d129c700 1 -- 172.21.15.17:0/2256943819 reap_dead start 2022-08-01T19:43:08.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46d1a9d700 1 -- 172.21.15.17:0/2256943819 shutdown_connections 2022-08-01T19:43:08.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46d1a9d700 1 -- 172.21.15.17:0/2256943819 >> 172.21.15.17:0/2256943819 conn(0x7f46cc0612d0 msgr2=0x7f46cc062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:08.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46d1a9d700 1 -- 172.21.15.17:0/2256943819 shutdown_connections 2022-08-01T19:43:08.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.325+0000 7f46d1a9d700 1 -- 172.21.15.17:0/2256943819 wait complete. 2022-08-01T19:43:08.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.329+0000 7f46d1a9d700 1 Processor -- start 2022-08-01T19:43:08.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.329+0000 7f46d1a9d700 1 -- start start 2022-08-01T19:43:08.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.329+0000 7f46d1a9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46b0004500 0x7f46cc149f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:08.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.329+0000 7f46d1a9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46cc0ab330 0x7f46cc1480e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:08.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.329+0000 7f46d1a9d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46cc1470a0 0x7f46cc142620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:08.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.329+0000 7f46d1a9d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc0ac330 con 0x7f46b0004500 2022-08-01T19:43:08.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.329+0000 7f46d1a9d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc149b00 con 0x7f46cc0ab330 2022-08-01T19:43:08.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.329+0000 7f46d1a9d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc0aafa0 con 0x7f46cc1470a0 2022-08-01T19:43:08.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.329+0000 7f46d0a9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46b0004500 0x7f46cc149f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:08.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.329+0000 7f46d0a9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46b0004500 0x7f46cc149f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44472/0 (socket says 172.21.15.17:44472) 2022-08-01T19:43:08.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.329+0000 7f46d0a9b700 1 -- 172.21.15.17:0/472285798 learned_addr learned my addr 172.21.15.17:0/472285798 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:08.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.329+0000 7f46d129c700 1 --2- 172.21.15.17:0/472285798 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46cc1470a0 0x7f46cc142620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:08.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.329+0000 7f46cbfff700 1 --2- 172.21.15.17:0/472285798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46cc0ab330 0x7f46cc1480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:08.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.329+0000 7f46d0a9b700 1 -- 172.21.15.17:0/472285798 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46cc1470a0 msgr2=0x7f46cc142620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:08.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.329+0000 7f46d0a9b700 1 --2- 172.21.15.17:0/472285798 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46cc1470a0 0x7f46cc142620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:08.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.329+0000 7f46d0a9b700 1 -- 172.21.15.17:0/472285798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46cc0ab330 msgr2=0x7f46cc1480e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:08.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.329+0000 7f46d0a9b700 1 --2- 172.21.15.17:0/472285798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46cc0ab330 0x7f46cc1480e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:08.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.329+0000 7f46d0a9b700 1 -- 172.21.15.17:0/472285798 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46cc0ac520 con 0x7f46b0004500 2022-08-01T19:43:08.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.329+0000 7f46d0a9b700 1 --2- 172.21.15.17:0/472285798 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46b0004500 0x7f46cc149f30 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 rx=0x7f46c000d000 tx=0x7f46cc0adee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:08.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.329+0000 7f46c9ffb700 1 -- 172.21.15.17:0/472285798 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46c00116b0 con 0x7f46b0004500 2022-08-01T19:43:08.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.329+0000 7f46c9ffb700 1 -- 172.21.15.17:0/472285798 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c0004e90 con 0x7f46b0004500 2022-08-01T19:43:08.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.329+0000 7f46d1a9d700 1 -- 172.21.15.17:0/472285798 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46cc144f30 con 0x7f46b0004500 2022-08-01T19:43:08.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.329+0000 7f46c9ffb700 1 -- 172.21.15.17:0/472285798 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46c0011a00 con 0x7f46b0004500 2022-08-01T19:43:08.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.329+0000 7f46d1a9d700 1 -- 172.21.15.17:0/472285798 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46cc0ab930 con 0x7f46b0004500 2022-08-01T19:43:08.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.333+0000 7f46d1a9d700 1 -- 172.21.15.17:0/472285798 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46cc0ab110 con 0x7f46b0004500 2022-08-01T19:43:08.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.333+0000 7f46c9ffb700 1 -- 172.21.15.17:0/472285798 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f46c001b030 con 0x7f46b0004500 2022-08-01T19:43:08.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.333+0000 7f46c9ffb700 1 --2- 172.21.15.17:0/472285798 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f46ac053950 0x7f46ac055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:08.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.333+0000 7f46c9ffb700 1 -- 172.21.15.17:0/472285798 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f46c006ba40 con 0x7f46b0004500 2022-08-01T19:43:08.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.333+0000 7f46cbfff700 1 --2- 172.21.15.17:0/472285798 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f46ac053950 0x7f46ac055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:08.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.333+0000 7f46cbfff700 1 --2- 172.21.15.17:0/472285798 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f46ac053950 0x7f46ac055e10 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f46b801a000 tx=0x7f46b8019000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:08.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.341+0000 7f46c9ffb700 1 -- 172.21.15.17:0/472285798 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f46cc0ab110 con 0x7f46b0004500 2022-08-01T19:43:08.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.501+0000 7f46d1a9d700 1 -- 172.21.15.17:0/472285798 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46cc0ab110 con 0x7f46b0004500 2022-08-01T19:43:08.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.501+0000 7f46c9ffb700 1 -- 172.21.15.17:0/472285798 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+2486 (secure 0 0 0) 0x7f46cc0ab110 con 0x7f46b0004500 2022-08-01T19:43:08.506 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:08.506 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":69,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7995,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.17:6833/4164640472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4164640472},{"type":"v1","addr":"172.21.15.17:6833","nonce":4164640472}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":67},{"gid":8001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/50015801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":50015801},{"type":"v1","addr":"172.21.15.17:6835","nonce":50015801}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:43:01.287477+0000","modified":"2022-08-01T19:43:02.290478+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8007},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8007":{"gid":8007,"name":"b","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6835/299216688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":299216688},{"type":"v1","addr":"172.21.15.77:6835","nonce":299216688}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2022-08-01T19:43:08.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.505+0000 7f46d1a9d700 1 -- 172.21.15.17:0/472285798 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f46ac053950 msgr2=0x7f46ac055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:08.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.505+0000 7f46d1a9d700 1 --2- 172.21.15.17:0/472285798 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f46ac053950 0x7f46ac055e10 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f46b801a000 tx=0x7f46b8019000).stop 2022-08-01T19:43:08.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.505+0000 7f46d1a9d700 1 -- 172.21.15.17:0/472285798 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46b0004500 msgr2=0x7f46cc149f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:08.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.505+0000 7f46d1a9d700 1 --2- 172.21.15.17:0/472285798 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46b0004500 0x7f46cc149f30 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 rx=0x7f46c000d000 tx=0x7f46cc0adee0).stop 2022-08-01T19:43:08.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.505+0000 7f46d1a9d700 1 -- 172.21.15.17:0/472285798 shutdown_connections 2022-08-01T19:43:08.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.505+0000 7f46d1a9d700 1 --2- 172.21.15.17:0/472285798 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f46ac053950 0x7f46ac055e10 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:08.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.505+0000 7f46d1a9d700 1 --2- 172.21.15.17:0/472285798 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f46cc1470a0 0x7f46cc142620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:08.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.505+0000 7f46d1a9d700 1 --2- 172.21.15.17:0/472285798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46cc0ab330 0x7f46cc1480e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:08.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.505+0000 7f46d1a9d700 1 --2- 172.21.15.17:0/472285798 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46b0004500 0x7f46cc149f30 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:08.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.505+0000 7f46d1a9d700 1 -- 172.21.15.17:0/472285798 >> 172.21.15.17:0/472285798 conn(0x7f46cc0612d0 msgr2=0x7f46cc0af380 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:08.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.505+0000 7f46d1a9d700 1 -- 172.21.15.17:0/472285798 shutdown_connections 2022-08-01T19:43:08.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:08.505+0000 7f46d1a9d700 1 -- 172.21.15.17:0/472285798 wait complete. 2022-08-01T19:43:08.515 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 69 2022-08-01T19:43:08.823 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 69, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:43:01.287477+0000', 'modified': '2022-08-01T19:43:02.290478+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8007}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8007': {'gid': 8007, 'name': 'b', 'rank': 0, 'incarnation': 68, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.77:6835/299216688', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6834', 'nonce': 299216688}, {'type': 'v1', 'addr': '172.21.15.77:6835', 'nonce': 299216688}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:43:08.824 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:43:08.824 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-08-01T19:43:08.941 INFO:teuthology.orchestra.run.smithi077.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":8007,"want_state":"up:active","state":"up:active","rank_uptime":7.6412161230000004,"mdsmap_epoch":69,"osdmap_epoch":137,"osdmap_epoch_barrier":136,"uptime":12.545086078000001} 2022-08-01T19:43:08.942 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 8007, "mdsmap_epoch": 69, "osdmap_epoch": 137, "osdmap_epoch_barrier": 136, "rank_uptime": 7.641216123, "state": "up:active", "uptime": 12.545086078, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:43:08.943 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T19:43:08.944 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-08-01T19:43:08.945 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi190.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-08-01T19:43:08.946 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:08.946 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:43:08.956 DEBUG:teuthology.orchestra.run.smithi190:> sudo modprobe fuse 2022-08-01T19:43:09.014 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:43:09.015 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:43:09.027 INFO:teuthology.orchestra.run.smithi190.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:43:09.029 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:43:09.030 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:09.031 DEBUG:teuthology.orchestra.run.smithi190:> ls /sys/fs/fuse/connections 2022-08-01T19:43:09.038 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T19:43:09.039 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:43:09.084 DEBUG:teuthology.orchestra.run.smithi190:> sudo modprobe fuse 2022-08-01T19:43:09.103 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:43:09.103 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:43:09.158 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:2022-08-01T19:43:09.149+0000 7fc440e48080 -1 init, newargv = 0x558c848e49e0 newargc=9 2022-08-01T19:43:09.159 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:ceph-fuse[23099]: starting ceph client 2022-08-01T19:43:09.159 INFO:teuthology.orchestra.run.smithi190.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:43:09.160 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:43:09.162 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:09.163 DEBUG:teuthology.orchestra.run.smithi190:> ls /sys/fs/fuse/connections 2022-08-01T19:43:09.169 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:ceph-fuse[23099]: starting fuse 2022-08-01T19:43:09.171 INFO:teuthology.orchestra.run.smithi190.stdout:52 2022-08-01T19:43:09.172 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T19:43:09.173 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:09.174 DEBUG:teuthology.orchestra.run.smithi190:> sudo python3 -c ' 2022-08-01T19:43:09.175 DEBUG:teuthology.orchestra.run.smithi190:> import glob 2022-08-01T19:43:09.176 DEBUG:teuthology.orchestra.run.smithi190:> import re 2022-08-01T19:43:09.177 DEBUG:teuthology.orchestra.run.smithi190:> import os 2022-08-01T19:43:09.178 DEBUG:teuthology.orchestra.run.smithi190:> import subprocess 2022-08-01T19:43:09.179 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:43:09.179 DEBUG:teuthology.orchestra.run.smithi190:> def find_socket(client_name): 2022-08-01T19:43:09.180 DEBUG:teuthology.orchestra.run.smithi190:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-08-01T19:43:09.181 DEBUG:teuthology.orchestra.run.smithi190:> files = glob.glob(asok_path) 2022-08-01T19:43:09.181 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:43:09.182 DEBUG:teuthology.orchestra.run.smithi190:> # Given a non-glob path, it better be there 2022-08-01T19:43:09.183 DEBUG:teuthology.orchestra.run.smithi190:> if "*" not in asok_path: 2022-08-01T19:43:09.183 DEBUG:teuthology.orchestra.run.smithi190:> assert(len(files) == 1) 2022-08-01T19:43:09.184 DEBUG:teuthology.orchestra.run.smithi190:> return files[0] 2022-08-01T19:43:09.185 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:43:09.185 DEBUG:teuthology.orchestra.run.smithi190:> for f in files: 2022-08-01T19:43:09.186 DEBUG:teuthology.orchestra.run.smithi190:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T19:43:09.187 DEBUG:teuthology.orchestra.run.smithi190:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T19:43:09.188 DEBUG:teuthology.orchestra.run.smithi190:> return f 2022-08-01T19:43:09.188 DEBUG:teuthology.orchestra.run.smithi190:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T19:43:09.189 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:43:09.190 DEBUG:teuthology.orchestra.run.smithi190:> print(find_socket("client.1")) 2022-08-01T19:43:09.190 DEBUG:teuthology.orchestra.run.smithi190:> ' 2022-08-01T19:43:09.257 INFO:teuthology.orchestra.run.smithi190.stdout:/var/run/ceph/ceph-client.1.23099.asok 2022-08-01T19:43:09.262 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.23099.asok 2022-08-01T19:43:09.263 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:09.263 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.23099.asok status 2022-08-01T19:43:09.354 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2022-08-01T19:43:09.355 INFO:teuthology.orchestra.run.smithi190.stdout: "metadata": { 2022-08-01T19:43:09.355 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T19:43:09.355 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T19:43:09.355 INFO:teuthology.orchestra.run.smithi190.stdout: "entity_id": "1", 2022-08-01T19:43:09.356 INFO:teuthology.orchestra.run.smithi190.stdout: "hostname": "smithi190", 2022-08-01T19:43:09.356 INFO:teuthology.orchestra.run.smithi190.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-08-01T19:43:09.357 INFO:teuthology.orchestra.run.smithi190.stdout: "pid": "23099", 2022-08-01T19:43:09.357 INFO:teuthology.orchestra.run.smithi190.stdout: "root": "/" 2022-08-01T19:43:09.357 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:43:09.357 INFO:teuthology.orchestra.run.smithi190.stdout: "dentry_count": 0, 2022-08-01T19:43:09.358 INFO:teuthology.orchestra.run.smithi190.stdout: "dentry_pinned_count": 0, 2022-08-01T19:43:09.358 INFO:teuthology.orchestra.run.smithi190.stdout: "id": 8194, 2022-08-01T19:43:09.358 INFO:teuthology.orchestra.run.smithi190.stdout: "inst": { 2022-08-01T19:43:09.359 INFO:teuthology.orchestra.run.smithi190.stdout: "name": { 2022-08-01T19:43:09.359 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "client", 2022-08-01T19:43:09.359 INFO:teuthology.orchestra.run.smithi190.stdout: "num": 8194 2022-08-01T19:43:09.360 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:43:09.360 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": { 2022-08-01T19:43:09.360 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "v1", 2022-08-01T19:43:09.360 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": "172.21.15.190:0", 2022-08-01T19:43:09.361 INFO:teuthology.orchestra.run.smithi190.stdout: "nonce": 4155676647 2022-08-01T19:43:09.361 INFO:teuthology.orchestra.run.smithi190.stdout: } 2022-08-01T19:43:09.362 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:43:09.363 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": { 2022-08-01T19:43:09.363 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "v1", 2022-08-01T19:43:09.363 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": "172.21.15.190:0", 2022-08-01T19:43:09.364 INFO:teuthology.orchestra.run.smithi190.stdout: "nonce": 4155676647 2022-08-01T19:43:09.364 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:43:09.364 INFO:teuthology.orchestra.run.smithi190.stdout: "inst_str": "client.8194 172.21.15.190:0/4155676647", 2022-08-01T19:43:09.364 INFO:teuthology.orchestra.run.smithi190.stdout: "addr_str": "172.21.15.190:0/4155676647", 2022-08-01T19:43:09.365 INFO:teuthology.orchestra.run.smithi190.stdout: "inode_count": 1, 2022-08-01T19:43:09.365 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_epoch": 69, 2022-08-01T19:43:09.365 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_epoch": 137, 2022-08-01T19:43:09.365 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_epoch_barrier": 0, 2022-08-01T19:43:09.366 INFO:teuthology.orchestra.run.smithi190.stdout: "blacklisted": false, 2022-08-01T19:43:09.366 INFO:teuthology.orchestra.run.smithi190.stdout: "fs_name": "cephfs" 2022-08-01T19:43:09.366 INFO:teuthology.orchestra.run.smithi190.stdout:} 2022-08-01T19:43:09.368 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:09.369 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:43:09.370 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:43:09.381 INFO:teuthology.orchestra.run.smithi190.stdout:fuseblk 2022-08-01T19:43:09.382 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T19:43:09.383 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:09.384 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:43:09.448 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:09.450 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:43:09.451 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:43:09.460 INFO:teuthology.orchestra.run.smithi190.stdout:fuseblk 2022-08-01T19:43:09.460 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T19:43:09.461 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:09.461 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:43:09.462 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:43:09.511 INFO:teuthology.orchestra.run.smithi190.stdout:fuseblk 2022-08-01T19:43:09.511 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T19:43:09.512 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi190.front.sepia.ceph.com... 2022-08-01T19:43:09.512 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T19:43:09.512 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:43:09.528 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:ceph-fuse[23099]: fuse finished with error 0 and tester_r 0 2022-08-01T19:43:09.530 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:09.531 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:43:09.532 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:43:09.538 INFO:teuthology.orchestra.run.smithi190.stdout:ext2/ext3 2022-08-01T19:43:09.539 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T19:43:09.540 INFO:teuthology.orchestra.run:waiting for 900 2022-08-01T19:43:15.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T19:43:15.643 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:43:15.657 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-08-01T19:43:15.660 DEBUG:teuthology.orchestra.run.smithi077:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-08-01T19:43:15.663 DEBUG:teuthology.orchestra.run.smithi085:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-08-01T19:43:15.666 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-08-01T19:43:15.696 INFO:teuthology.orchestra.run.smithi017.stdout:[global] 2022-08-01T19:43:15.696 INFO:teuthology.orchestra.run.smithi017.stdout: chdir = "" 2022-08-01T19:43:15.696 INFO:teuthology.orchestra.run.smithi017.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-08-01T19:43:15.697 INFO:teuthology.orchestra.run.smithi017.stdout: auth supported = cephx 2022-08-01T19:43:15.698 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:15.698 INFO:teuthology.orchestra.run.smithi017.stdout: filestore xattr use omap = true 2022-08-01T19:43:15.699 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:15.699 INFO:teuthology.orchestra.run.smithi017.stdout: mon clock drift allowed = 1.000 2022-08-01T19:43:15.700 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:15.700 INFO:teuthology.orchestra.run.smithi017.stdout: osd crush chooseleaf type = 0 2022-08-01T19:43:15.700 INFO:teuthology.orchestra.run.smithi017.stdout: auth debug = true 2022-08-01T19:43:15.701 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:15.701 INFO:teuthology.orchestra.run.smithi017.stdout: ms die on old message = true 2022-08-01T19:43:15.702 INFO:teuthology.orchestra.run.smithi017.stdout: ms die on bug = true 2022-08-01T19:43:15.702 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:15.702 INFO:teuthology.orchestra.run.smithi017.stdout: mon max pg per osd = 10000 # >= luminous 2022-08-01T19:43:15.703 INFO:teuthology.orchestra.run.smithi017.stdout: mon pg warn max object skew = 0 2022-08-01T19:43:15.703 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:15.704 INFO:teuthology.orchestra.run.smithi017.stdout: # disable pg_autoscaler by default for new pools 2022-08-01T19:43:15.704 INFO:teuthology.orchestra.run.smithi017.stdout: osd_pool_default_pg_autoscale_mode = off 2022-08-01T19:43:15.705 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:15.705 INFO:teuthology.orchestra.run.smithi017.stdout: osd pool default size = 2 2022-08-01T19:43:15.705 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:15.706 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd allow primary affinity = true 2022-08-01T19:43:15.706 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd allow pg remap = true 2022-08-01T19:43:15.707 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on legacy crush tunables = false 2022-08-01T19:43:15.707 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on crush straw calc version zero = false 2022-08-01T19:43:15.707 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on no sortbitwise = false 2022-08-01T19:43:15.708 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on osd down out interval zero = false 2022-08-01T19:43:15.708 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on too few osds = false 2022-08-01T19:43:15.708 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-08-01T19:43:15.709 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_pool_no_redundancy = false 2022-08-01T19:43:15.709 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:15.710 INFO:teuthology.orchestra.run.smithi017.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-08-01T19:43:15.710 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:15.711 INFO:teuthology.orchestra.run.smithi017.stdout: osd default data pool replay window = 5 2022-08-01T19:43:15.711 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:15.711 INFO:teuthology.orchestra.run.smithi017.stdout: mon allow pool delete = true 2022-08-01T19:43:15.711 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:15.712 INFO:teuthology.orchestra.run.smithi017.stdout: mon cluster log file level = debug 2022-08-01T19:43:15.712 INFO:teuthology.orchestra.run.smithi017.stdout: debug asserts on shutdown = true 2022-08-01T19:43:15.712 INFO:teuthology.orchestra.run.smithi017.stdout: mon health detail to clog = false 2022-08-01T19:43:15.712 INFO:teuthology.orchestra.run.smithi017.stdout: mon host = "172.21.15.17,172.21.15.77,[v2:172.21.15.77:3301,v1:172.21.15.77:6790]" 2022-08-01T19:43:15.712 INFO:teuthology.orchestra.run.smithi017.stdout: fsid = be659963-3cf3-4374-b3c4-cbe717b6bac2 2022-08-01T19:43:15.714 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:15.714 INFO:teuthology.orchestra.run.smithi017.stdout:[osd] 2022-08-01T19:43:15.715 INFO:teuthology.orchestra.run.smithi017.stdout: osd journal size = 100 2022-08-01T19:43:15.715 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:15.715 INFO:teuthology.orchestra.run.smithi017.stdout: osd scrub load threshold = 5.0 2022-08-01T19:43:15.715 INFO:teuthology.orchestra.run.smithi017.stdout: osd scrub max interval = 600 2022-08-01T19:43:15.716 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:15.716 INFO:teuthology.orchestra.run.smithi017.stdout: osd recover clone overlap = true 2022-08-01T19:43:15.717 INFO:teuthology.orchestra.run.smithi017.stdout: osd recovery max chunk = 1048576 2022-08-01T19:43:15.717 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:15.718 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug shutdown = true 2022-08-01T19:43:15.718 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug op order = true 2022-08-01T19:43:15.718 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug verify stray on activate = true 2022-08-01T19:43:15.723 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:15.724 INFO:teuthology.orchestra.run.smithi017.stdout: osd open classes on start = true 2022-08-01T19:43:15.724 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug pg log writeout = true 2022-08-01T19:43:15.725 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:15.725 INFO:teuthology.orchestra.run.smithi017.stdout: osd deep scrub update digest min age = 30 2022-08-01T19:43:15.726 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:15.726 INFO:teuthology.orchestra.run.smithi017.stdout: osd map max advance = 10 2022-08-01T19:43:15.727 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:15.727 INFO:teuthology.orchestra.run.smithi017.stdout: journal zero on create = true 2022-08-01T19:43:15.728 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:15.728 INFO:teuthology.orchestra.run.smithi017.stdout: filestore ondisk finisher threads = 3 2022-08-01T19:43:15.729 INFO:teuthology.orchestra.run.smithi017.stdout: filestore apply finisher threads = 3 2022-08-01T19:43:15.729 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:15.730 INFO:teuthology.orchestra.run.smithi017.stdout: bdev debug aio = true 2022-08-01T19:43:15.730 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug misdirected ops = true 2022-08-01T19:43:15.730 INFO:teuthology.orchestra.run.smithi017.stdout: bluestore block size = 96636764160 2022-08-01T19:43:15.731 INFO:teuthology.orchestra.run.smithi017.stdout: bluestore compression mode = aggressive 2022-08-01T19:43:15.731 INFO:teuthology.orchestra.run.smithi017.stdout: bluestore fsck on mount = True 2022-08-01T19:43:15.732 INFO:teuthology.orchestra.run.smithi017.stdout: debug bluefs = 20 2022-08-01T19:43:15.732 INFO:teuthology.orchestra.run.smithi017.stdout: debug bluestore = 20 2022-08-01T19:43:15.733 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2022-08-01T19:43:15.733 INFO:teuthology.orchestra.run.smithi017.stdout: debug osd = 20 2022-08-01T19:43:15.734 INFO:teuthology.orchestra.run.smithi017.stdout: debug rocksdb = 10 2022-08-01T19:43:15.734 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd backfillfull_ratio = 0.85 2022-08-01T19:43:15.735 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd full ratio = 0.9 2022-08-01T19:43:15.735 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd nearfull ratio = 0.8 2022-08-01T19:43:15.736 INFO:teuthology.orchestra.run.smithi017.stdout: osd failsafe full ratio = 0.95 2022-08-01T19:43:15.736 INFO:teuthology.orchestra.run.smithi017.stdout: osd objectstore = bluestore 2022-08-01T19:43:15.737 INFO:teuthology.orchestra.run.smithi017.stdout: osd op complaint time = 180 2022-08-01T19:43:15.737 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:15.738 INFO:teuthology.orchestra.run.smithi017.stdout:[mgr] 2022-08-01T19:43:15.738 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2022-08-01T19:43:15.739 INFO:teuthology.orchestra.run.smithi017.stdout: debug mgr = 20 2022-08-01T19:43:15.739 INFO:teuthology.orchestra.run.smithi017.stdout: debug mon = 20 2022-08-01T19:43:15.740 INFO:teuthology.orchestra.run.smithi017.stdout: debug auth = 20 2022-08-01T19:43:15.740 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min pgs per osd = 4 2022-08-01T19:43:15.741 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min bytes per osd = 10 2022-08-01T19:43:15.741 INFO:teuthology.orchestra.run.smithi017.stdout: mgr/telemetry/nag = false 2022-08-01T19:43:15.742 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:15.742 INFO:teuthology.orchestra.run.smithi017.stdout:[mon] 2022-08-01T19:43:15.743 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2022-08-01T19:43:15.743 INFO:teuthology.orchestra.run.smithi017.stdout: debug mon = 20 2022-08-01T19:43:15.744 INFO:teuthology.orchestra.run.smithi017.stdout: debug paxos = 20 2022-08-01T19:43:15.744 INFO:teuthology.orchestra.run.smithi017.stdout: debug auth = 20 2022-08-01T19:43:15.745 INFO:teuthology.orchestra.run.smithi017.stdout: mon data avail warn = 5 2022-08-01T19:43:15.745 INFO:teuthology.orchestra.run.smithi017.stdout: mon mgr mkfs grace = 240 2022-08-01T19:43:15.745 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min pgs per osd = 4 2022-08-01T19:43:15.746 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd reporter subtree level = osd 2022-08-01T19:43:15.746 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd prime pg temp = true 2022-08-01T19:43:15.747 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min bytes per osd = 10 2022-08-01T19:43:15.747 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:15.748 INFO:teuthology.orchestra.run.smithi017.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-08-01T19:43:15.748 INFO:teuthology.orchestra.run.smithi017.stdout: auth mon ticket ttl = 660 # 11m 2022-08-01T19:43:15.749 INFO:teuthology.orchestra.run.smithi017.stdout: auth service ticket ttl = 240 # 4m 2022-08-01T19:43:15.750 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:15.750 INFO:teuthology.orchestra.run.smithi017.stdout: # don't complain about insecure global_id in the test suite 2022-08-01T19:43:15.751 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-08-01T19:43:15.752 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-08-01T19:43:15.753 INFO:teuthology.orchestra.run.smithi017.stdout: mon op complaint time = 120 2022-08-01T19:43:15.753 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:15.754 INFO:teuthology.orchestra.run.smithi017.stdout:[client] 2022-08-01T19:43:15.754 INFO:teuthology.orchestra.run.smithi017.stdout: rgw cache enabled = true 2022-08-01T19:43:15.755 INFO:teuthology.orchestra.run.smithi017.stdout: rgw enable ops log = true 2022-08-01T19:43:15.755 INFO:teuthology.orchestra.run.smithi017.stdout: rgw enable usage log = true 2022-08-01T19:43:15.756 INFO:teuthology.orchestra.run.smithi017.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-08-01T19:43:15.756 INFO:teuthology.orchestra.run.smithi017.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-08-01T19:43:15.757 INFO:teuthology.orchestra.run.smithi017.stdout: client mount timeout = 600 2022-08-01T19:43:15.757 INFO:teuthology.orchestra.run.smithi017.stdout: debug client = 20 2022-08-01T19:43:15.757 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2022-08-01T19:43:15.758 INFO:teuthology.orchestra.run.smithi017.stdout: rados mon op timeout = 15m 2022-08-01T19:43:15.758 INFO:teuthology.orchestra.run.smithi017.stdout: rados osd op timeout = 15m 2022-08-01T19:43:15.758 INFO:teuthology.orchestra.run.smithi017.stdout: client debug getattr caps = true 2022-08-01T19:43:15.759 INFO:teuthology.orchestra.run.smithi017.stdout:[mon.a] 2022-08-01T19:43:15.759 INFO:teuthology.orchestra.run.smithi017.stdout:[mon.b] 2022-08-01T19:43:15.760 INFO:teuthology.orchestra.run.smithi017.stdout:[mon.c] 2022-08-01T19:43:15.760 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.a] 2022-08-01T19:43:15.760 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.c] 2022-08-01T19:43:15.761 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.b] 2022-08-01T19:43:15.761 INFO:teuthology.orchestra.run.smithi017.stdout:[mds] 2022-08-01T19:43:15.761 INFO:teuthology.orchestra.run.smithi017.stdout: debug mds = 20 2022-08-01T19:43:15.762 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2022-08-01T19:43:15.762 INFO:teuthology.orchestra.run.smithi017.stdout: mds bal frag = True 2022-08-01T19:43:15.762 INFO:teuthology.orchestra.run.smithi017.stdout: mds bal fragment size max = 10000 2022-08-01T19:43:15.762 INFO:teuthology.orchestra.run.smithi017.stdout: mds bal merge size = 5 2022-08-01T19:43:15.763 INFO:teuthology.orchestra.run.smithi017.stdout: mds bal split bits = 3 2022-08-01T19:43:15.763 INFO:teuthology.orchestra.run.smithi017.stdout: mds bal split size = 100 2022-08-01T19:43:15.763 INFO:teuthology.orchestra.run.smithi017.stdout: mds debug frag = True 2022-08-01T19:43:15.763 INFO:teuthology.orchestra.run.smithi017.stdout: mds debug scatterstat = True 2022-08-01T19:43:15.763 INFO:teuthology.orchestra.run.smithi017.stdout: mds op complaint time = 180 2022-08-01T19:43:15.764 INFO:teuthology.orchestra.run.smithi017.stdout: mds verify scatter = True 2022-08-01T19:43:15.764 INFO:teuthology.orchestra.run.smithi017.stdout: osd op complaint time = 180 2022-08-01T19:43:15.764 INFO:teuthology.orchestra.run.smithi017.stdout: rados mon op timeout = 15m 2022-08-01T19:43:15.764 INFO:teuthology.orchestra.run.smithi017.stdout: rados osd op timeout = 15m 2022-08-01T19:43:15.769 INFO:teuthology.orchestra.run.smithi085.stdout:[global] 2022-08-01T19:43:15.770 INFO:teuthology.orchestra.run.smithi085.stdout: chdir = "" 2022-08-01T19:43:15.771 INFO:teuthology.orchestra.run.smithi085.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-08-01T19:43:15.771 INFO:teuthology.orchestra.run.smithi085.stdout: auth supported = cephx 2022-08-01T19:43:15.771 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:15.772 INFO:teuthology.orchestra.run.smithi085.stdout: filestore xattr use omap = true 2022-08-01T19:43:15.772 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:15.773 INFO:teuthology.orchestra.run.smithi085.stdout: mon clock drift allowed = 1.000 2022-08-01T19:43:15.773 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:15.774 INFO:teuthology.orchestra.run.smithi085.stdout: osd crush chooseleaf type = 0 2022-08-01T19:43:15.774 INFO:teuthology.orchestra.run.smithi085.stdout: auth debug = true 2022-08-01T19:43:15.775 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:15.775 INFO:teuthology.orchestra.run.smithi085.stdout: ms die on old message = true 2022-08-01T19:43:15.775 INFO:teuthology.orchestra.run.smithi085.stdout: ms die on bug = true 2022-08-01T19:43:15.776 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:15.776 INFO:teuthology.orchestra.run.smithi085.stdout: mon max pg per osd = 10000 # >= luminous 2022-08-01T19:43:15.777 INFO:teuthology.orchestra.run.smithi085.stdout: mon pg warn max object skew = 0 2022-08-01T19:43:15.777 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:15.778 INFO:teuthology.orchestra.run.smithi085.stdout: # disable pg_autoscaler by default for new pools 2022-08-01T19:43:15.778 INFO:teuthology.orchestra.run.smithi085.stdout: osd_pool_default_pg_autoscale_mode = off 2022-08-01T19:43:15.779 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:15.779 INFO:teuthology.orchestra.run.smithi085.stdout: osd pool default size = 2 2022-08-01T19:43:15.780 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:15.780 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd allow primary affinity = true 2022-08-01T19:43:15.781 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd allow pg remap = true 2022-08-01T19:43:15.781 INFO:teuthology.orchestra.run.smithi085.stdout: mon warn on legacy crush tunables = false 2022-08-01T19:43:15.782 INFO:teuthology.orchestra.run.smithi085.stdout: mon warn on crush straw calc version zero = false 2022-08-01T19:43:15.782 INFO:teuthology.orchestra.run.smithi085.stdout: mon warn on no sortbitwise = false 2022-08-01T19:43:15.783 INFO:teuthology.orchestra.run.smithi085.stdout: mon warn on osd down out interval zero = false 2022-08-01T19:43:15.783 INFO:teuthology.orchestra.run.smithi085.stdout: mon warn on too few osds = false 2022-08-01T19:43:15.784 INFO:teuthology.orchestra.run.smithi085.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-08-01T19:43:15.784 INFO:teuthology.orchestra.run.smithi085.stdout: mon_warn_on_pool_no_redundancy = false 2022-08-01T19:43:15.785 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:15.785 INFO:teuthology.orchestra.run.smithi085.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-08-01T19:43:15.786 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:15.786 INFO:teuthology.orchestra.run.smithi085.stdout: osd default data pool replay window = 5 2022-08-01T19:43:15.786 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:15.787 INFO:teuthology.orchestra.run.smithi085.stdout: mon allow pool delete = true 2022-08-01T19:43:15.787 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:15.788 INFO:teuthology.orchestra.run.smithi085.stdout: mon cluster log file level = debug 2022-08-01T19:43:15.788 INFO:teuthology.orchestra.run.smithi085.stdout: debug asserts on shutdown = true 2022-08-01T19:43:15.789 INFO:teuthology.orchestra.run.smithi085.stdout: mon health detail to clog = false 2022-08-01T19:43:15.789 INFO:teuthology.orchestra.run.smithi085.stdout: mon host = "172.21.15.17,172.21.15.77,[v2:172.21.15.77:3301,v1:172.21.15.77:6790]" 2022-08-01T19:43:15.790 INFO:teuthology.orchestra.run.smithi085.stdout: fsid = be659963-3cf3-4374-b3c4-cbe717b6bac2 2022-08-01T19:43:15.790 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:15.791 INFO:teuthology.orchestra.run.smithi085.stdout:[osd] 2022-08-01T19:43:15.791 INFO:teuthology.orchestra.run.smithi085.stdout: osd journal size = 100 2022-08-01T19:43:15.792 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:15.792 INFO:teuthology.orchestra.run.smithi085.stdout: osd scrub load threshold = 5.0 2022-08-01T19:43:15.792 INFO:teuthology.orchestra.run.smithi085.stdout: osd scrub max interval = 600 2022-08-01T19:43:15.793 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:15.793 INFO:teuthology.orchestra.run.smithi085.stdout: osd recover clone overlap = true 2022-08-01T19:43:15.794 INFO:teuthology.orchestra.run.smithi085.stdout: osd recovery max chunk = 1048576 2022-08-01T19:43:15.795 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:15.795 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug shutdown = true 2022-08-01T19:43:15.796 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug op order = true 2022-08-01T19:43:15.796 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug verify stray on activate = true 2022-08-01T19:43:15.797 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:15.797 INFO:teuthology.orchestra.run.smithi085.stdout: osd open classes on start = true 2022-08-01T19:43:15.797 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug pg log writeout = true 2022-08-01T19:43:15.798 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:15.798 INFO:teuthology.orchestra.run.smithi085.stdout: osd deep scrub update digest min age = 30 2022-08-01T19:43:15.799 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:15.799 INFO:teuthology.orchestra.run.smithi085.stdout: osd map max advance = 10 2022-08-01T19:43:15.800 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:15.800 INFO:teuthology.orchestra.run.smithi085.stdout: journal zero on create = true 2022-08-01T19:43:15.800 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:15.801 INFO:teuthology.orchestra.run.smithi085.stdout: filestore ondisk finisher threads = 3 2022-08-01T19:43:15.801 INFO:teuthology.orchestra.run.smithi085.stdout: filestore apply finisher threads = 3 2022-08-01T19:43:15.801 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:15.802 INFO:teuthology.orchestra.run.smithi085.stdout: bdev debug aio = true 2022-08-01T19:43:15.802 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug misdirected ops = true 2022-08-01T19:43:15.802 INFO:teuthology.orchestra.run.smithi085.stdout: bluestore block size = 96636764160 2022-08-01T19:43:15.803 INFO:teuthology.orchestra.run.smithi085.stdout: bluestore compression mode = aggressive 2022-08-01T19:43:15.803 INFO:teuthology.orchestra.run.smithi085.stdout: bluestore fsck on mount = True 2022-08-01T19:43:15.803 INFO:teuthology.orchestra.run.smithi085.stdout: debug bluefs = 20 2022-08-01T19:43:15.804 INFO:teuthology.orchestra.run.smithi085.stdout: debug bluestore = 20 2022-08-01T19:43:15.804 INFO:teuthology.orchestra.run.smithi085.stdout: debug ms = 1 2022-08-01T19:43:15.804 INFO:teuthology.orchestra.run.smithi085.stdout: debug osd = 20 2022-08-01T19:43:15.805 INFO:teuthology.orchestra.run.smithi085.stdout: debug rocksdb = 10 2022-08-01T19:43:15.805 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd backfillfull_ratio = 0.85 2022-08-01T19:43:15.805 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd full ratio = 0.9 2022-08-01T19:43:15.806 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd nearfull ratio = 0.8 2022-08-01T19:43:15.806 INFO:teuthology.orchestra.run.smithi085.stdout: osd failsafe full ratio = 0.95 2022-08-01T19:43:15.806 INFO:teuthology.orchestra.run.smithi085.stdout: osd objectstore = bluestore 2022-08-01T19:43:15.806 INFO:teuthology.orchestra.run.smithi085.stdout: osd op complaint time = 180 2022-08-01T19:43:15.807 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:15.807 INFO:teuthology.orchestra.run.smithi085.stdout:[mgr] 2022-08-01T19:43:15.807 INFO:teuthology.orchestra.run.smithi085.stdout: debug ms = 1 2022-08-01T19:43:15.807 INFO:teuthology.orchestra.run.smithi085.stdout: debug mgr = 20 2022-08-01T19:43:15.807 INFO:teuthology.orchestra.run.smithi085.stdout: debug mon = 20 2022-08-01T19:43:15.808 INFO:teuthology.orchestra.run.smithi085.stdout: debug auth = 20 2022-08-01T19:43:15.808 INFO:teuthology.orchestra.run.smithi085.stdout: mon reweight min pgs per osd = 4 2022-08-01T19:43:15.808 INFO:teuthology.orchestra.run.smithi085.stdout: mon reweight min bytes per osd = 10 2022-08-01T19:43:15.808 INFO:teuthology.orchestra.run.smithi085.stdout: mgr/telemetry/nag = false 2022-08-01T19:43:15.809 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:15.809 INFO:teuthology.orchestra.run.smithi085.stdout:[mon] 2022-08-01T19:43:15.809 INFO:teuthology.orchestra.run.smithi085.stdout: debug ms = 1 2022-08-01T19:43:15.809 INFO:teuthology.orchestra.run.smithi085.stdout: debug mon = 20 2022-08-01T19:43:15.810 INFO:teuthology.orchestra.run.smithi085.stdout: debug paxos = 20 2022-08-01T19:43:15.810 INFO:teuthology.orchestra.run.smithi085.stdout: debug auth = 20 2022-08-01T19:43:15.810 INFO:teuthology.orchestra.run.smithi085.stdout: mon data avail warn = 5 2022-08-01T19:43:15.810 INFO:teuthology.orchestra.run.smithi085.stdout: mon mgr mkfs grace = 240 2022-08-01T19:43:15.811 INFO:teuthology.orchestra.run.smithi085.stdout: mon reweight min pgs per osd = 4 2022-08-01T19:43:15.811 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd reporter subtree level = osd 2022-08-01T19:43:15.811 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd prime pg temp = true 2022-08-01T19:43:15.811 INFO:teuthology.orchestra.run.smithi085.stdout: mon reweight min bytes per osd = 10 2022-08-01T19:43:15.812 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:15.812 INFO:teuthology.orchestra.run.smithi085.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-08-01T19:43:15.812 INFO:teuthology.orchestra.run.smithi085.stdout: auth mon ticket ttl = 660 # 11m 2022-08-01T19:43:15.812 INFO:teuthology.orchestra.run.smithi085.stdout: auth service ticket ttl = 240 # 4m 2022-08-01T19:43:15.813 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:15.813 INFO:teuthology.orchestra.run.smithi085.stdout: # don't complain about insecure global_id in the test suite 2022-08-01T19:43:15.813 INFO:teuthology.orchestra.run.smithi085.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-08-01T19:43:15.813 INFO:teuthology.orchestra.run.smithi085.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-08-01T19:43:15.814 INFO:teuthology.orchestra.run.smithi085.stdout: mon op complaint time = 120 2022-08-01T19:43:15.814 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:15.814 INFO:teuthology.orchestra.run.smithi085.stdout:[client] 2022-08-01T19:43:15.815 INFO:teuthology.orchestra.run.smithi085.stdout: rgw cache enabled = true 2022-08-01T19:43:15.815 INFO:teuthology.orchestra.run.smithi085.stdout: rgw enable ops log = true 2022-08-01T19:43:15.816 INFO:teuthology.orchestra.run.smithi085.stdout: rgw enable usage log = true 2022-08-01T19:43:15.816 INFO:teuthology.orchestra.run.smithi085.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-08-01T19:43:15.816 INFO:teuthology.orchestra.run.smithi085.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-08-01T19:43:15.817 INFO:teuthology.orchestra.run.smithi085.stdout: client mount timeout = 600 2022-08-01T19:43:15.817 INFO:teuthology.orchestra.run.smithi085.stdout: debug client = 20 2022-08-01T19:43:15.818 INFO:teuthology.orchestra.run.smithi085.stdout: debug ms = 1 2022-08-01T19:43:15.818 INFO:teuthology.orchestra.run.smithi085.stdout: rados mon op timeout = 15m 2022-08-01T19:43:15.818 INFO:teuthology.orchestra.run.smithi085.stdout: rados osd op timeout = 15m 2022-08-01T19:43:15.819 INFO:teuthology.orchestra.run.smithi085.stdout: client debug getattr caps = true 2022-08-01T19:43:15.819 INFO:teuthology.orchestra.run.smithi085.stdout:[mon.a] 2022-08-01T19:43:15.820 INFO:teuthology.orchestra.run.smithi085.stdout:[mon.b] 2022-08-01T19:43:15.820 INFO:teuthology.orchestra.run.smithi085.stdout:[mon.c] 2022-08-01T19:43:15.820 INFO:teuthology.orchestra.run.smithi085.stdout:[mds.a] 2022-08-01T19:43:15.821 INFO:teuthology.orchestra.run.smithi085.stdout:[mds.c] 2022-08-01T19:43:15.821 INFO:teuthology.orchestra.run.smithi085.stdout:[mds.b] 2022-08-01T19:43:15.821 INFO:teuthology.orchestra.run.smithi085.stdout:[mds] 2022-08-01T19:43:15.822 INFO:teuthology.orchestra.run.smithi085.stdout: debug mds = 20 2022-08-01T19:43:15.822 INFO:teuthology.orchestra.run.smithi085.stdout: debug ms = 1 2022-08-01T19:43:15.823 INFO:teuthology.orchestra.run.smithi085.stdout: mds bal frag = True 2022-08-01T19:43:15.823 INFO:teuthology.orchestra.run.smithi085.stdout: mds bal fragment size max = 10000 2022-08-01T19:43:15.823 INFO:teuthology.orchestra.run.smithi085.stdout: mds bal merge size = 5 2022-08-01T19:43:15.824 INFO:teuthology.orchestra.run.smithi085.stdout: mds bal split bits = 3 2022-08-01T19:43:15.824 INFO:teuthology.orchestra.run.smithi085.stdout: mds bal split size = 100 2022-08-01T19:43:15.824 INFO:teuthology.orchestra.run.smithi085.stdout: mds debug frag = True 2022-08-01T19:43:15.825 INFO:teuthology.orchestra.run.smithi085.stdout: mds debug scatterstat = True 2022-08-01T19:43:15.825 INFO:teuthology.orchestra.run.smithi085.stdout: mds op complaint time = 180 2022-08-01T19:43:15.825 INFO:teuthology.orchestra.run.smithi085.stdout: mds verify scatter = True 2022-08-01T19:43:15.826 INFO:teuthology.orchestra.run.smithi085.stdout: osd op complaint time = 180 2022-08-01T19:43:15.826 INFO:teuthology.orchestra.run.smithi085.stdout: rados mon op timeout = 15m 2022-08-01T19:43:15.827 INFO:teuthology.orchestra.run.smithi085.stdout: rados osd op timeout = 15m 2022-08-01T19:43:15.827 INFO:teuthology.orchestra.run.smithi190.stdout:[global] 2022-08-01T19:43:15.827 INFO:teuthology.orchestra.run.smithi190.stdout: chdir = "" 2022-08-01T19:43:15.827 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-08-01T19:43:15.828 INFO:teuthology.orchestra.run.smithi190.stdout: auth supported = cephx 2022-08-01T19:43:15.828 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:15.828 INFO:teuthology.orchestra.run.smithi190.stdout: filestore xattr use omap = true 2022-08-01T19:43:15.828 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:15.829 INFO:teuthology.orchestra.run.smithi190.stdout: mon clock drift allowed = 1.000 2022-08-01T19:43:15.829 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:15.829 INFO:teuthology.orchestra.run.smithi190.stdout: osd crush chooseleaf type = 0 2022-08-01T19:43:15.829 INFO:teuthology.orchestra.run.smithi190.stdout: auth debug = true 2022-08-01T19:43:15.830 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:15.830 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on old message = true 2022-08-01T19:43:15.830 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on bug = true 2022-08-01T19:43:15.830 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:15.831 INFO:teuthology.orchestra.run.smithi190.stdout: mon max pg per osd = 10000 # >= luminous 2022-08-01T19:43:15.831 INFO:teuthology.orchestra.run.smithi190.stdout: mon pg warn max object skew = 0 2022-08-01T19:43:15.831 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:15.831 INFO:teuthology.orchestra.run.smithi190.stdout: # disable pg_autoscaler by default for new pools 2022-08-01T19:43:15.832 INFO:teuthology.orchestra.run.smithi190.stdout: osd_pool_default_pg_autoscale_mode = off 2022-08-01T19:43:15.832 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:15.832 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default size = 2 2022-08-01T19:43:15.832 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:15.832 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow primary affinity = true 2022-08-01T19:43:15.833 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow pg remap = true 2022-08-01T19:43:15.833 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on legacy crush tunables = false 2022-08-01T19:43:15.834 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on crush straw calc version zero = false 2022-08-01T19:43:15.834 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on no sortbitwise = false 2022-08-01T19:43:15.834 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on osd down out interval zero = false 2022-08-01T19:43:15.834 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on too few osds = false 2022-08-01T19:43:15.835 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-08-01T19:43:15.835 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_no_redundancy = false 2022-08-01T19:43:15.835 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:15.836 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-08-01T19:43:15.836 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:15.836 INFO:teuthology.orchestra.run.smithi190.stdout: osd default data pool replay window = 5 2022-08-01T19:43:15.837 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:15.837 INFO:teuthology.orchestra.run.smithi190.stdout: mon allow pool delete = true 2022-08-01T19:43:15.838 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:15.838 INFO:teuthology.orchestra.run.smithi190.stdout: mon cluster log file level = debug 2022-08-01T19:43:15.838 INFO:teuthology.orchestra.run.smithi190.stdout: debug asserts on shutdown = true 2022-08-01T19:43:15.839 INFO:teuthology.orchestra.run.smithi190.stdout: mon health detail to clog = false 2022-08-01T19:43:15.839 INFO:teuthology.orchestra.run.smithi190.stdout: mon host = "172.21.15.17,172.21.15.77,[v2:172.21.15.77:3301,v1:172.21.15.77:6790]" 2022-08-01T19:43:15.840 INFO:teuthology.orchestra.run.smithi190.stdout: fsid = be659963-3cf3-4374-b3c4-cbe717b6bac2 2022-08-01T19:43:15.840 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:15.841 INFO:teuthology.orchestra.run.smithi190.stdout:[osd] 2022-08-01T19:43:15.841 INFO:teuthology.orchestra.run.smithi190.stdout: osd journal size = 100 2022-08-01T19:43:15.842 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:15.842 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub load threshold = 5.0 2022-08-01T19:43:15.843 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub max interval = 600 2022-08-01T19:43:15.843 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:15.843 INFO:teuthology.orchestra.run.smithi190.stdout: osd recover clone overlap = true 2022-08-01T19:43:15.844 INFO:teuthology.orchestra.run.smithi190.stdout: osd recovery max chunk = 1048576 2022-08-01T19:43:15.844 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:15.845 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug shutdown = true 2022-08-01T19:43:15.845 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug op order = true 2022-08-01T19:43:15.846 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug verify stray on activate = true 2022-08-01T19:43:15.846 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:15.847 INFO:teuthology.orchestra.run.smithi190.stdout: osd open classes on start = true 2022-08-01T19:43:15.847 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug pg log writeout = true 2022-08-01T19:43:15.848 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:15.848 INFO:teuthology.orchestra.run.smithi190.stdout: osd deep scrub update digest min age = 30 2022-08-01T19:43:15.849 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:15.849 INFO:teuthology.orchestra.run.smithi190.stdout: osd map max advance = 10 2022-08-01T19:43:15.849 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:15.849 INFO:teuthology.orchestra.run.smithi190.stdout: journal zero on create = true 2022-08-01T19:43:15.850 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:15.850 INFO:teuthology.orchestra.run.smithi190.stdout: filestore ondisk finisher threads = 3 2022-08-01T19:43:15.850 INFO:teuthology.orchestra.run.smithi190.stdout: filestore apply finisher threads = 3 2022-08-01T19:43:15.850 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:15.851 INFO:teuthology.orchestra.run.smithi190.stdout: bdev debug aio = true 2022-08-01T19:43:15.851 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug misdirected ops = true 2022-08-01T19:43:15.851 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore block size = 96636764160 2022-08-01T19:43:15.851 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore compression mode = aggressive 2022-08-01T19:43:15.852 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore fsck on mount = True 2022-08-01T19:43:15.852 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluefs = 20 2022-08-01T19:43:15.852 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluestore = 20 2022-08-01T19:43:15.852 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2022-08-01T19:43:15.853 INFO:teuthology.orchestra.run.smithi190.stdout: debug osd = 20 2022-08-01T19:43:15.853 INFO:teuthology.orchestra.run.smithi190.stdout: debug rocksdb = 10 2022-08-01T19:43:15.853 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd backfillfull_ratio = 0.85 2022-08-01T19:43:15.853 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd full ratio = 0.9 2022-08-01T19:43:15.853 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd nearfull ratio = 0.8 2022-08-01T19:43:15.854 INFO:teuthology.orchestra.run.smithi190.stdout: osd failsafe full ratio = 0.95 2022-08-01T19:43:15.854 INFO:teuthology.orchestra.run.smithi190.stdout: osd objectstore = bluestore 2022-08-01T19:43:15.854 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2022-08-01T19:43:15.857 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:15.857 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr] 2022-08-01T19:43:15.858 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2022-08-01T19:43:15.858 INFO:teuthology.orchestra.run.smithi190.stdout: debug mgr = 20 2022-08-01T19:43:15.859 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2022-08-01T19:43:15.859 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2022-08-01T19:43:15.860 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2022-08-01T19:43:15.860 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2022-08-01T19:43:15.861 INFO:teuthology.orchestra.run.smithi190.stdout: mgr/telemetry/nag = false 2022-08-01T19:43:15.861 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:15.861 INFO:teuthology.orchestra.run.smithi190.stdout:[mon] 2022-08-01T19:43:15.862 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2022-08-01T19:43:15.862 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2022-08-01T19:43:15.863 INFO:teuthology.orchestra.run.smithi190.stdout: debug paxos = 20 2022-08-01T19:43:15.863 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2022-08-01T19:43:15.864 INFO:teuthology.orchestra.run.smithi190.stdout: mon data avail warn = 5 2022-08-01T19:43:15.864 INFO:teuthology.orchestra.run.smithi190.stdout: mon mgr mkfs grace = 240 2022-08-01T19:43:15.865 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2022-08-01T19:43:15.865 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd reporter subtree level = osd 2022-08-01T19:43:15.865 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd prime pg temp = true 2022-08-01T19:43:15.866 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2022-08-01T19:43:15.866 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:15.867 INFO:teuthology.orchestra.run.smithi190.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-08-01T19:43:15.867 INFO:teuthology.orchestra.run.smithi190.stdout: auth mon ticket ttl = 660 # 11m 2022-08-01T19:43:15.868 INFO:teuthology.orchestra.run.smithi190.stdout: auth service ticket ttl = 240 # 4m 2022-08-01T19:43:15.868 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:15.868 INFO:teuthology.orchestra.run.smithi190.stdout: # don't complain about insecure global_id in the test suite 2022-08-01T19:43:15.869 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-08-01T19:43:15.869 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-08-01T19:43:15.870 INFO:teuthology.orchestra.run.smithi190.stdout: mon op complaint time = 120 2022-08-01T19:43:15.870 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:15.870 INFO:teuthology.orchestra.run.smithi190.stdout:[client] 2022-08-01T19:43:15.871 INFO:teuthology.orchestra.run.smithi190.stdout: rgw cache enabled = true 2022-08-01T19:43:15.871 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable ops log = true 2022-08-01T19:43:15.872 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable usage log = true 2022-08-01T19:43:15.872 INFO:teuthology.orchestra.run.smithi190.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-08-01T19:43:15.873 INFO:teuthology.orchestra.run.smithi190.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-08-01T19:43:15.873 INFO:teuthology.orchestra.run.smithi190.stdout: client mount timeout = 600 2022-08-01T19:43:15.874 INFO:teuthology.orchestra.run.smithi190.stdout: debug client = 20 2022-08-01T19:43:15.874 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2022-08-01T19:43:15.875 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 15m 2022-08-01T19:43:15.875 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 15m 2022-08-01T19:43:15.875 INFO:teuthology.orchestra.run.smithi190.stdout: client debug getattr caps = true 2022-08-01T19:43:15.876 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.a] 2022-08-01T19:43:15.876 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.b] 2022-08-01T19:43:15.877 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.c] 2022-08-01T19:43:15.877 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.a] 2022-08-01T19:43:15.878 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.c] 2022-08-01T19:43:15.878 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.b] 2022-08-01T19:43:15.879 INFO:teuthology.orchestra.run.smithi190.stdout:[mds] 2022-08-01T19:43:15.879 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds = 20 2022-08-01T19:43:15.879 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2022-08-01T19:43:15.880 INFO:teuthology.orchestra.run.smithi190.stdout: mds bal frag = True 2022-08-01T19:43:15.880 INFO:teuthology.orchestra.run.smithi190.stdout: mds bal fragment size max = 10000 2022-08-01T19:43:15.880 INFO:teuthology.orchestra.run.smithi190.stdout: mds bal merge size = 5 2022-08-01T19:43:15.880 INFO:teuthology.orchestra.run.smithi190.stdout: mds bal split bits = 3 2022-08-01T19:43:15.881 INFO:teuthology.orchestra.run.smithi190.stdout: mds bal split size = 100 2022-08-01T19:43:15.881 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug frag = True 2022-08-01T19:43:15.881 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug scatterstat = True 2022-08-01T19:43:15.881 INFO:teuthology.orchestra.run.smithi190.stdout: mds op complaint time = 180 2022-08-01T19:43:15.882 INFO:teuthology.orchestra.run.smithi190.stdout: mds verify scatter = True 2022-08-01T19:43:15.882 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2022-08-01T19:43:15.882 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 15m 2022-08-01T19:43:15.882 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 15m 2022-08-01T19:43:15.883 INFO:teuthology.orchestra.run.smithi077.stdout:[global] 2022-08-01T19:43:15.883 INFO:teuthology.orchestra.run.smithi077.stdout: chdir = "" 2022-08-01T19:43:15.883 INFO:teuthology.orchestra.run.smithi077.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-08-01T19:43:15.883 INFO:teuthology.orchestra.run.smithi077.stdout: auth supported = cephx 2022-08-01T19:43:15.883 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:15.884 INFO:teuthology.orchestra.run.smithi077.stdout: filestore xattr use omap = true 2022-08-01T19:43:15.884 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:15.884 INFO:teuthology.orchestra.run.smithi077.stdout: mon clock drift allowed = 1.000 2022-08-01T19:43:15.884 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:15.885 INFO:teuthology.orchestra.run.smithi077.stdout: osd crush chooseleaf type = 0 2022-08-01T19:43:15.885 INFO:teuthology.orchestra.run.smithi077.stdout: auth debug = true 2022-08-01T19:43:15.885 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:15.885 INFO:teuthology.orchestra.run.smithi077.stdout: ms die on old message = true 2022-08-01T19:43:15.886 INFO:teuthology.orchestra.run.smithi077.stdout: ms die on bug = true 2022-08-01T19:43:15.886 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:15.887 INFO:teuthology.orchestra.run.smithi077.stdout: mon max pg per osd = 10000 # >= luminous 2022-08-01T19:43:15.887 INFO:teuthology.orchestra.run.smithi077.stdout: mon pg warn max object skew = 0 2022-08-01T19:43:15.887 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:15.887 INFO:teuthology.orchestra.run.smithi077.stdout: # disable pg_autoscaler by default for new pools 2022-08-01T19:43:15.888 INFO:teuthology.orchestra.run.smithi077.stdout: osd_pool_default_pg_autoscale_mode = off 2022-08-01T19:43:15.888 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:15.888 INFO:teuthology.orchestra.run.smithi077.stdout: osd pool default size = 2 2022-08-01T19:43:15.888 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:15.889 INFO:teuthology.orchestra.run.smithi077.stdout: mon osd allow primary affinity = true 2022-08-01T19:43:15.889 INFO:teuthology.orchestra.run.smithi077.stdout: mon osd allow pg remap = true 2022-08-01T19:43:15.889 INFO:teuthology.orchestra.run.smithi077.stdout: mon warn on legacy crush tunables = false 2022-08-01T19:43:15.889 INFO:teuthology.orchestra.run.smithi077.stdout: mon warn on crush straw calc version zero = false 2022-08-01T19:43:15.889 INFO:teuthology.orchestra.run.smithi077.stdout: mon warn on no sortbitwise = false 2022-08-01T19:43:15.890 INFO:teuthology.orchestra.run.smithi077.stdout: mon warn on osd down out interval zero = false 2022-08-01T19:43:15.890 INFO:teuthology.orchestra.run.smithi077.stdout: mon warn on too few osds = false 2022-08-01T19:43:15.890 INFO:teuthology.orchestra.run.smithi077.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-08-01T19:43:15.891 INFO:teuthology.orchestra.run.smithi077.stdout: mon_warn_on_pool_no_redundancy = false 2022-08-01T19:43:15.892 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:15.892 INFO:teuthology.orchestra.run.smithi077.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-08-01T19:43:15.892 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:15.893 INFO:teuthology.orchestra.run.smithi077.stdout: osd default data pool replay window = 5 2022-08-01T19:43:15.893 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:15.893 INFO:teuthology.orchestra.run.smithi077.stdout: mon allow pool delete = true 2022-08-01T19:43:15.894 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:15.894 INFO:teuthology.orchestra.run.smithi077.stdout: mon cluster log file level = debug 2022-08-01T19:43:15.894 INFO:teuthology.orchestra.run.smithi077.stdout: debug asserts on shutdown = true 2022-08-01T19:43:15.895 INFO:teuthology.orchestra.run.smithi077.stdout: mon health detail to clog = false 2022-08-01T19:43:15.895 INFO:teuthology.orchestra.run.smithi077.stdout: mon host = "172.21.15.17,172.21.15.77,[v2:172.21.15.77:3301,v1:172.21.15.77:6790]" 2022-08-01T19:43:15.895 INFO:teuthology.orchestra.run.smithi077.stdout: fsid = be659963-3cf3-4374-b3c4-cbe717b6bac2 2022-08-01T19:43:15.895 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:15.896 INFO:teuthology.orchestra.run.smithi077.stdout:[osd] 2022-08-01T19:43:15.896 INFO:teuthology.orchestra.run.smithi077.stdout: osd journal size = 100 2022-08-01T19:43:15.896 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:15.896 INFO:teuthology.orchestra.run.smithi077.stdout: osd scrub load threshold = 5.0 2022-08-01T19:43:15.897 INFO:teuthology.orchestra.run.smithi077.stdout: osd scrub max interval = 600 2022-08-01T19:43:15.897 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:15.898 INFO:teuthology.orchestra.run.smithi077.stdout: osd recover clone overlap = true 2022-08-01T19:43:15.898 INFO:teuthology.orchestra.run.smithi077.stdout: osd recovery max chunk = 1048576 2022-08-01T19:43:15.898 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:15.899 INFO:teuthology.orchestra.run.smithi077.stdout: osd debug shutdown = true 2022-08-01T19:43:15.899 INFO:teuthology.orchestra.run.smithi077.stdout: osd debug op order = true 2022-08-01T19:43:15.900 INFO:teuthology.orchestra.run.smithi077.stdout: osd debug verify stray on activate = true 2022-08-01T19:43:15.900 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:15.900 INFO:teuthology.orchestra.run.smithi077.stdout: osd open classes on start = true 2022-08-01T19:43:15.901 INFO:teuthology.orchestra.run.smithi077.stdout: osd debug pg log writeout = true 2022-08-01T19:43:15.901 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:15.902 INFO:teuthology.orchestra.run.smithi077.stdout: osd deep scrub update digest min age = 30 2022-08-01T19:43:15.902 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:15.903 INFO:teuthology.orchestra.run.smithi077.stdout: osd map max advance = 10 2022-08-01T19:43:15.903 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:15.903 INFO:teuthology.orchestra.run.smithi077.stdout: journal zero on create = true 2022-08-01T19:43:15.904 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:15.904 INFO:teuthology.orchestra.run.smithi077.stdout: filestore ondisk finisher threads = 3 2022-08-01T19:43:15.904 INFO:teuthology.orchestra.run.smithi077.stdout: filestore apply finisher threads = 3 2022-08-01T19:43:15.905 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:15.905 INFO:teuthology.orchestra.run.smithi077.stdout: bdev debug aio = true 2022-08-01T19:43:15.905 INFO:teuthology.orchestra.run.smithi077.stdout: osd debug misdirected ops = true 2022-08-01T19:43:15.905 INFO:teuthology.orchestra.run.smithi077.stdout: bluestore block size = 96636764160 2022-08-01T19:43:15.906 INFO:teuthology.orchestra.run.smithi077.stdout: bluestore compression mode = aggressive 2022-08-01T19:43:15.906 INFO:teuthology.orchestra.run.smithi077.stdout: bluestore fsck on mount = True 2022-08-01T19:43:15.906 INFO:teuthology.orchestra.run.smithi077.stdout: debug bluefs = 20 2022-08-01T19:43:15.907 INFO:teuthology.orchestra.run.smithi077.stdout: debug bluestore = 20 2022-08-01T19:43:15.907 INFO:teuthology.orchestra.run.smithi077.stdout: debug ms = 1 2022-08-01T19:43:15.907 INFO:teuthology.orchestra.run.smithi077.stdout: debug osd = 20 2022-08-01T19:43:15.907 INFO:teuthology.orchestra.run.smithi077.stdout: debug rocksdb = 10 2022-08-01T19:43:15.908 INFO:teuthology.orchestra.run.smithi077.stdout: mon osd backfillfull_ratio = 0.85 2022-08-01T19:43:15.908 INFO:teuthology.orchestra.run.smithi077.stdout: mon osd full ratio = 0.9 2022-08-01T19:43:15.908 INFO:teuthology.orchestra.run.smithi077.stdout: mon osd nearfull ratio = 0.8 2022-08-01T19:43:15.908 INFO:teuthology.orchestra.run.smithi077.stdout: osd failsafe full ratio = 0.95 2022-08-01T19:43:15.908 INFO:teuthology.orchestra.run.smithi077.stdout: osd objectstore = bluestore 2022-08-01T19:43:15.909 INFO:teuthology.orchestra.run.smithi077.stdout: osd op complaint time = 180 2022-08-01T19:43:15.909 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:15.909 INFO:teuthology.orchestra.run.smithi077.stdout:[mgr] 2022-08-01T19:43:15.909 INFO:teuthology.orchestra.run.smithi077.stdout: debug ms = 1 2022-08-01T19:43:15.910 INFO:teuthology.orchestra.run.smithi077.stdout: debug mgr = 20 2022-08-01T19:43:15.910 INFO:teuthology.orchestra.run.smithi077.stdout: debug mon = 20 2022-08-01T19:43:15.910 INFO:teuthology.orchestra.run.smithi077.stdout: debug auth = 20 2022-08-01T19:43:15.910 INFO:teuthology.orchestra.run.smithi077.stdout: mon reweight min pgs per osd = 4 2022-08-01T19:43:15.910 INFO:teuthology.orchestra.run.smithi077.stdout: mon reweight min bytes per osd = 10 2022-08-01T19:43:15.911 INFO:teuthology.orchestra.run.smithi077.stdout: mgr/telemetry/nag = false 2022-08-01T19:43:15.911 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:15.911 INFO:teuthology.orchestra.run.smithi077.stdout:[mon] 2022-08-01T19:43:15.911 INFO:teuthology.orchestra.run.smithi077.stdout: debug ms = 1 2022-08-01T19:43:15.912 INFO:teuthology.orchestra.run.smithi077.stdout: debug mon = 20 2022-08-01T19:43:15.912 INFO:teuthology.orchestra.run.smithi077.stdout: debug paxos = 20 2022-08-01T19:43:15.912 INFO:teuthology.orchestra.run.smithi077.stdout: debug auth = 20 2022-08-01T19:43:15.912 INFO:teuthology.orchestra.run.smithi077.stdout: mon data avail warn = 5 2022-08-01T19:43:15.912 INFO:teuthology.orchestra.run.smithi077.stdout: mon mgr mkfs grace = 240 2022-08-01T19:43:15.913 INFO:teuthology.orchestra.run.smithi077.stdout: mon reweight min pgs per osd = 4 2022-08-01T19:43:15.913 INFO:teuthology.orchestra.run.smithi077.stdout: mon osd reporter subtree level = osd 2022-08-01T19:43:15.913 INFO:teuthology.orchestra.run.smithi077.stdout: mon osd prime pg temp = true 2022-08-01T19:43:15.913 INFO:teuthology.orchestra.run.smithi077.stdout: mon reweight min bytes per osd = 10 2022-08-01T19:43:15.914 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:15.914 INFO:teuthology.orchestra.run.smithi077.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-08-01T19:43:15.914 INFO:teuthology.orchestra.run.smithi077.stdout: auth mon ticket ttl = 660 # 11m 2022-08-01T19:43:15.914 INFO:teuthology.orchestra.run.smithi077.stdout: auth service ticket ttl = 240 # 4m 2022-08-01T19:43:15.914 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:15.915 INFO:teuthology.orchestra.run.smithi077.stdout: # don't complain about insecure global_id in the test suite 2022-08-01T19:43:15.915 INFO:teuthology.orchestra.run.smithi077.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-08-01T19:43:15.915 INFO:teuthology.orchestra.run.smithi077.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-08-01T19:43:15.915 INFO:teuthology.orchestra.run.smithi077.stdout: mon op complaint time = 120 2022-08-01T19:43:15.916 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:15.916 INFO:teuthology.orchestra.run.smithi077.stdout:[client] 2022-08-01T19:43:15.916 INFO:teuthology.orchestra.run.smithi077.stdout: rgw cache enabled = true 2022-08-01T19:43:15.916 INFO:teuthology.orchestra.run.smithi077.stdout: rgw enable ops log = true 2022-08-01T19:43:15.917 INFO:teuthology.orchestra.run.smithi077.stdout: rgw enable usage log = true 2022-08-01T19:43:15.917 INFO:teuthology.orchestra.run.smithi077.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-08-01T19:43:15.917 INFO:teuthology.orchestra.run.smithi077.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-08-01T19:43:15.917 INFO:teuthology.orchestra.run.smithi077.stdout: client mount timeout = 600 2022-08-01T19:43:15.917 INFO:teuthology.orchestra.run.smithi077.stdout: debug client = 20 2022-08-01T19:43:15.918 INFO:teuthology.orchestra.run.smithi077.stdout: debug ms = 1 2022-08-01T19:43:15.918 INFO:teuthology.orchestra.run.smithi077.stdout: rados mon op timeout = 15m 2022-08-01T19:43:15.918 INFO:teuthology.orchestra.run.smithi077.stdout: rados osd op timeout = 15m 2022-08-01T19:43:15.918 INFO:teuthology.orchestra.run.smithi077.stdout: client debug getattr caps = true 2022-08-01T19:43:15.919 INFO:teuthology.orchestra.run.smithi077.stdout:[mon.a] 2022-08-01T19:43:15.919 INFO:teuthology.orchestra.run.smithi077.stdout:[mon.b] 2022-08-01T19:43:15.919 INFO:teuthology.orchestra.run.smithi077.stdout:[mon.c] 2022-08-01T19:43:15.919 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.a] 2022-08-01T19:43:15.919 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.c] 2022-08-01T19:43:15.920 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.b] 2022-08-01T19:43:15.920 INFO:teuthology.orchestra.run.smithi077.stdout:[mds] 2022-08-01T19:43:15.920 INFO:teuthology.orchestra.run.smithi077.stdout: debug mds = 20 2022-08-01T19:43:15.920 INFO:teuthology.orchestra.run.smithi077.stdout: debug ms = 1 2022-08-01T19:43:15.921 INFO:teuthology.orchestra.run.smithi077.stdout: mds bal frag = True 2022-08-01T19:43:15.921 INFO:teuthology.orchestra.run.smithi077.stdout: mds bal fragment size max = 10000 2022-08-01T19:43:15.921 INFO:teuthology.orchestra.run.smithi077.stdout: mds bal merge size = 5 2022-08-01T19:43:15.921 INFO:teuthology.orchestra.run.smithi077.stdout: mds bal split bits = 3 2022-08-01T19:43:15.921 INFO:teuthology.orchestra.run.smithi077.stdout: mds bal split size = 100 2022-08-01T19:43:15.922 INFO:teuthology.orchestra.run.smithi077.stdout: mds debug frag = True 2022-08-01T19:43:15.922 INFO:teuthology.orchestra.run.smithi077.stdout: mds debug scatterstat = True 2022-08-01T19:43:15.922 INFO:teuthology.orchestra.run.smithi077.stdout: mds op complaint time = 180 2022-08-01T19:43:15.922 INFO:teuthology.orchestra.run.smithi077.stdout: mds verify scatter = True 2022-08-01T19:43:15.923 INFO:teuthology.orchestra.run.smithi077.stdout: osd op complaint time = 180 2022-08-01T19:43:15.923 INFO:teuthology.orchestra.run.smithi077.stdout: rados mon op timeout = 15m 2022-08-01T19:43:15.923 INFO:teuthology.orchestra.run.smithi077.stdout: rados osd op timeout = 15m 2022-08-01T19:43:15.924 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:43:16.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.037+0000 7f35c84ae700 1 Processor -- start 2022-08-01T19:43:16.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.037+0000 7f35c84ae700 1 -- start start 2022-08-01T19:43:16.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.037+0000 7f35c84ae700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35c00a76d0 0x7f35c00b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:16.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.037+0000 7f35c84ae700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f35c0145390 0x7f35c014b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:16.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.037+0000 7f35c84ae700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35c00a7ab0 con 0x7f35c00a9e30 2022-08-01T19:43:16.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.037+0000 7f35c84ae700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35c00b1640 con 0x7f35c0145390 2022-08-01T19:43:16.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.037+0000 7f35c84ae700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c00b3810 con 0x7f35c00a76d0 2022-08-01T19:43:16.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.037+0000 7f35c6a4b700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f35c0145390 0x7f35c014b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:16.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.037+0000 7f35c624a700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35c00a76d0 0x7f35c00b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:16.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.037+0000 7f35c5a49700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7f35c00a9e30 0x7f35c0144c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:53474/0 (socket says 172.21.15.17:53474) 2022-08-01T19:43:16.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.037+0000 7f35c5a49700 1 -- 172.21.15.17:0/4057527807 learned_addr learned my addr 172.21.15.17:0/4057527807 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:16.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.037+0000 7f35c5248700 1 -- 172.21.15.17:0/4057527807 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 438458560 0 0) 0x7f35c00a7ab0 con 0x7f35c00a9e30 2022-08-01T19:43:16.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.037+0000 7f35c5248700 1 -- 172.21.15.17:0/4057527807 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35ac003260 con 0x7f35c00a9e30 2022-08-01T19:43:16.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.037+0000 7f35c6a4b700 1 -- 172.21.15.17:0/4057527807 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35c00a76d0 msgr2=0x7f35c00b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:16.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.037+0000 7f35c6a4b700 1 --2- 172.21.15.17:0/4057527807 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35c00a76d0 0x7f35c00b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:16.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.037+0000 7f35c6a4b700 1 -- 172.21.15.17:0/4057527807 >> v1:172.21.15.77:6789/0 conn(0x7f35c00a9e30 legacy=0x7f35c0144c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:16.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.037+0000 7f35c6a4b700 1 -- 172.21.15.17:0/4057527807 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c00b3ab0 con 0x7f35c0145390 2022-08-01T19:43:16.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.037+0000 7f35c6a4b700 1 --2- 172.21.15.17:0/4057527807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35c0145390 0x7f35c014b0f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f35b0002f70 tx=0x7f35b0006df0).ready entity=mon.1 client_cookie=62047cb8211fe1f9 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:16.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.037+0000 7f35c5248700 1 -- 172.21.15.17:0/4057527807 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35b000d4a0 con 0x7f35c0145390 2022-08-01T19:43:16.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.037+0000 7f35c5248700 1 -- 172.21.15.17:0/4057527807 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b0000e90 con 0x7f35c0145390 2022-08-01T19:43:16.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.037+0000 7f35c5248700 1 -- 172.21.15.17:0/4057527807 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35b000d7f0 con 0x7f35c0145390 2022-08-01T19:43:16.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.041+0000 7f35c84ae700 1 -- 172.21.15.17:0/4057527807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35c0145390 msgr2=0x7f35c014b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:16.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.041+0000 7f35c84ae700 1 --2- 172.21.15.17:0/4057527807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35c0145390 0x7f35c014b0f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f35b0002f70 tx=0x7f35b0006df0).stop 2022-08-01T19:43:16.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.041+0000 7f35c84ae700 1 -- 172.21.15.17:0/4057527807 shutdown_connections 2022-08-01T19:43:16.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.041+0000 7f35c84ae700 1 --2- 172.21.15.17:0/4057527807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35c0145390 0x7f35c014b0f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:16.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.041+0000 7f35c84ae700 1 --2- 172.21.15.17:0/4057527807 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35c00a76d0 0x7f35c00b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:16.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.041+0000 7f35c84ae700 1 -- 172.21.15.17:0/4057527807 >> 172.21.15.17:0/4057527807 conn(0x7f35c0060950 msgr2=0x7f35c0060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:16.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.041+0000 7f35c84ae700 1 -- 172.21.15.17:0/4057527807 shutdown_connections 2022-08-01T19:43:16.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.041+0000 7f35c84ae700 1 -- 172.21.15.17:0/4057527807 wait complete. 2022-08-01T19:43:16.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.041+0000 7f35c84ae700 1 Processor -- start 2022-08-01T19:43:16.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.041+0000 7f35c84ae700 1 -- start start 2022-08-01T19:43:16.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.041+0000 7f35c84ae700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35c00a76d0 0x7f35c00b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:16.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.041+0000 7f35c84ae700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35c00a9e30 0x7f35c01574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:16.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.041+0000 7f35c84ae700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35c0145390 0x7f35c00aa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:16.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.041+0000 7f35c84ae700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c014cc20 con 0x7f35c00a76d0 2022-08-01T19:43:16.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.041+0000 7f35c84ae700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c015ae20 con 0x7f35c00a9e30 2022-08-01T19:43:16.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.041+0000 7f35c84ae700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c0156d60 con 0x7f35c0145390 2022-08-01T19:43:16.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.041+0000 7f35c624a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35c00a76d0 0x7f35c00b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:16.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.045+0000 7f35c624a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35c00a76d0 0x7f35c00b5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44488/0 (socket says 172.21.15.17:44488) 2022-08-01T19:43:16.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.045+0000 7f35c624a700 1 -- 172.21.15.17:0/2888509586 learned_addr learned my addr 172.21.15.17:0/2888509586 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:16.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.045+0000 7f35c5a49700 1 --2- 172.21.15.17:0/2888509586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35c00a9e30 0x7f35c01574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:16.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.045+0000 7f35c6a4b700 1 --2- 172.21.15.17:0/2888509586 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35c0145390 0x7f35c00aa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:16.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.045+0000 7f35c624a700 1 -- 172.21.15.17:0/2888509586 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35c0145390 msgr2=0x7f35c00aa5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:16.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.045+0000 7f35c624a700 1 --2- 172.21.15.17:0/2888509586 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35c0145390 0x7f35c00aa5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:16.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.045+0000 7f35c624a700 1 -- 172.21.15.17:0/2888509586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35c00a9e30 msgr2=0x7f35c01574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:16.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.045+0000 7f35c624a700 1 --2- 172.21.15.17:0/2888509586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35c00a9e30 0x7f35c01574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:16.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.045+0000 7f35c624a700 1 -- 172.21.15.17:0/2888509586 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b000b180 con 0x7f35c00a76d0 2022-08-01T19:43:16.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.045+0000 7f35c6a4b700 1 --2- 172.21.15.17:0/2888509586 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35c0145390 0x7f35c00aa5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:43:16.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.045+0000 7f35c624a700 1 --2- 172.21.15.17:0/2888509586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35c00a76d0 0x7f35c00b5f90 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 rx=0x7f35b400c000 tx=0x7f35b4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:16.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.045+0000 7f35bf7fe700 1 -- 172.21.15.17:0/2888509586 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35b4010780 con 0x7f35c00a76d0 2022-08-01T19:43:16.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.045+0000 7f35c84ae700 1 -- 172.21.15.17:0/2888509586 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35c0157000 con 0x7f35c00a76d0 2022-08-01T19:43:16.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.045+0000 7f35bf7fe700 1 -- 172.21.15.17:0/2888509586 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b40108e0 con 0x7f35c00a76d0 2022-08-01T19:43:16.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.045+0000 7f35bf7fe700 1 -- 172.21.15.17:0/2888509586 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35b4010c40 con 0x7f35c00a76d0 2022-08-01T19:43:16.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.045+0000 7f35c84ae700 1 -- 172.21.15.17:0/2888509586 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35c0157990 con 0x7f35c00a76d0 2022-08-01T19:43:16.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.045+0000 7f35c84ae700 1 -- 172.21.15.17:0/2888509586 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35c00b3d50 con 0x7f35c00a76d0 2022-08-01T19:43:16.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.045+0000 7f35bf7fe700 1 -- 172.21.15.17:0/2888509586 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f35b401c460 con 0x7f35c00a76d0 2022-08-01T19:43:16.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.045+0000 7f35bf7fe700 1 --2- 172.21.15.17:0/2888509586 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f35a0053b00 0x7f35a0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:16.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.045+0000 7f35bf7fe700 1 -- 172.21.15.17:0/2888509586 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f35b406c4f0 con 0x7f35c00a76d0 2022-08-01T19:43:16.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.049+0000 7f35c5a49700 1 --2- 172.21.15.17:0/2888509586 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f35a0053b00 0x7f35a0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:16.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.049+0000 7f35c5a49700 1 --2- 172.21.15.17:0/2888509586 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f35a0053b00 0x7f35a0055fc0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f35b800d000 tx=0x7f35b8009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:16.063 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.057+0000 7f35bf7fe700 1 -- 172.21.15.17:0/2888509586 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f35b40431b0 con 0x7f35c00a76d0 2022-08-01T19:43:16.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.185+0000 7f35c84ae700 1 -- 172.21.15.17:0/2888509586 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35c00574e0 con 0x7f35c00a76d0 2022-08-01T19:43:16.190 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.185+0000 7f35bf7fe700 1 -- 172.21.15.17:0/2888509586 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f35b40467d0 con 0x7f35c00a76d0 2022-08-01T19:43:16.191 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:16.192 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":137,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:43:02.291265+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:42:59.166414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2022-08-01T19:43:00.301184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:43:16.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.189+0000 7f35c84ae700 1 -- 172.21.15.17:0/2888509586 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f35a0053b00 msgr2=0x7f35a0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:16.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.189+0000 7f35c84ae700 1 --2- 172.21.15.17:0/2888509586 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f35a0053b00 0x7f35a0055fc0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f35b800d000 tx=0x7f35b8009040).stop 2022-08-01T19:43:16.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.189+0000 7f35c84ae700 1 -- 172.21.15.17:0/2888509586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35c00a76d0 msgr2=0x7f35c00b5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:16.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.189+0000 7f35c84ae700 1 --2- 172.21.15.17:0/2888509586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35c00a76d0 0x7f35c00b5f90 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 rx=0x7f35b400c000 tx=0x7f35b4006df0).stop 2022-08-01T19:43:16.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.189+0000 7f35c84ae700 1 -- 172.21.15.17:0/2888509586 shutdown_connections 2022-08-01T19:43:16.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.189+0000 7f35c84ae700 1 --2- 172.21.15.17:0/2888509586 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f35a0053b00 0x7f35a0055fc0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:16.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.189+0000 7f35c84ae700 1 --2- 172.21.15.17:0/2888509586 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35c0145390 0x7f35c00aa5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:16.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.189+0000 7f35c84ae700 1 --2- 172.21.15.17:0/2888509586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35c00a9e30 0x7f35c01574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:16.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.189+0000 7f35c84ae700 1 --2- 172.21.15.17:0/2888509586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35c00a76d0 0x7f35c00b5f90 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:16.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.189+0000 7f35c84ae700 1 -- 172.21.15.17:0/2888509586 >> 172.21.15.17:0/2888509586 conn(0x7f35c0060950 msgr2=0x7f35c00b26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:16.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.189+0000 7f35c84ae700 1 -- 172.21.15.17:0/2888509586 shutdown_connections 2022-08-01T19:43:16.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.189+0000 7f35c84ae700 1 -- 172.21.15.17:0/2888509586 wait complete. 2022-08-01T19:43:16.204 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T19:42:59.166414+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T19:43:00.301184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T19:43:16.207 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:43:16.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.317+0000 7f1354e7b700 1 Processor -- start 2022-08-01T19:43:16.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.317+0000 7f1354e7b700 1 -- start start 2022-08-01T19:43:16.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.317+0000 7f1354e7b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f135005fb80 0x7f135014b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:16.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.317+0000 7f1354e7b700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f13500b16c0 0x7f13500603d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:16.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.317+0000 7f1354e7b700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f135005ff60 con 0x7f1350147540 2022-08-01T19:43:16.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.317+0000 7f1354e7b700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13500608c0 con 0x7f135005fb80 2022-08-01T19:43:16.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.317+0000 7f1354e7b700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f135014ac90 con 0x7f13500b16c0 2022-08-01T19:43:16.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.321+0000 7f134f7fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f135005fb80 0x7f135014b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:16.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.321+0000 7f134ffff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1350147540 0x7f13500ad8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36692/0 (socket says 172.21.15.17:36692) 2022-08-01T19:43:16.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.321+0000 7f134ffff700 1 -- 172.21.15.17:0/2714005646 learned_addr learned my addr 172.21.15.17:0/2714005646 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:16.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.321+0000 7f134effd700 1 --2- 172.21.15.17:0/2714005646 >> v2:172.21.15.77:3300/0 conn(0x7f13500b16c0 0x7f13500603d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:16.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.321+0000 7f134f7fe700 1 -- 172.21.15.17:0/2714005646 >> v2:172.21.15.77:3300/0 conn(0x7f13500b16c0 msgr2=0x7f13500603d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:16.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.321+0000 7f134f7fe700 1 --2- 172.21.15.17:0/2714005646 >> v2:172.21.15.77:3300/0 conn(0x7f13500b16c0 0x7f13500603d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:16.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.321+0000 7f134f7fe700 1 -- 172.21.15.17:0/2714005646 >> v1:172.21.15.17:6789/0 conn(0x7f1350147540 legacy=0x7f13500ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:16.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.321+0000 7f134f7fe700 1 -- 172.21.15.17:0/2714005646 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f135014af30 con 0x7f135005fb80 2022-08-01T19:43:16.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.321+0000 7f134f7fe700 1 --2- 172.21.15.17:0/2714005646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f135005fb80 0x7f135014b670 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 rx=0x7f1344002f70 tx=0x7f1344006df0).ready entity=mon.0 client_cookie=94b17022c7aeb72f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:16.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.321+0000 7f134e7fc700 1 -- 172.21.15.17:0/2714005646 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f134400d570 con 0x7f135005fb80 2022-08-01T19:43:16.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.321+0000 7f134e7fc700 1 -- 172.21.15.17:0/2714005646 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1344000e90 con 0x7f135005fb80 2022-08-01T19:43:16.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.321+0000 7f134e7fc700 1 -- 172.21.15.17:0/2714005646 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f134400d8c0 con 0x7f135005fb80 2022-08-01T19:43:16.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.321+0000 7f1354e7b700 1 -- 172.21.15.17:0/2714005646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f135005fb80 msgr2=0x7f135014b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:16.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.321+0000 7f1354e7b700 1 --2- 172.21.15.17:0/2714005646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f135005fb80 0x7f135014b670 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 rx=0x7f1344002f70 tx=0x7f1344006df0).stop 2022-08-01T19:43:16.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.321+0000 7f1354e7b700 1 -- 172.21.15.17:0/2714005646 shutdown_connections 2022-08-01T19:43:16.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.321+0000 7f1354e7b700 1 --2- 172.21.15.17:0/2714005646 >> v2:172.21.15.77:3300/0 conn(0x7f13500b16c0 0x7f13500603d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:16.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.321+0000 7f1354e7b700 1 --2- 172.21.15.17:0/2714005646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f135005fb80 0x7f135014b670 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:16.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.321+0000 7f1354e7b700 1 -- 172.21.15.17:0/2714005646 >> 172.21.15.17:0/2714005646 conn(0x7f13500b54f0 msgr2=0x7f13500b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:16.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.321+0000 7f1354e7b700 1 -- 172.21.15.17:0/2714005646 shutdown_connections 2022-08-01T19:43:16.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.321+0000 7f1354e7b700 1 -- 172.21.15.17:0/2714005646 wait complete. 2022-08-01T19:43:16.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.321+0000 7f1354e7b700 1 Processor -- start 2022-08-01T19:43:16.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.321+0000 7f1354e7b700 1 -- start start 2022-08-01T19:43:16.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.325+0000 7f1354e7b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f135005fb80 0x7f1350149fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:16.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.325+0000 7f1354e7b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13500b16c0 0x7f1350159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:16.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.325+0000 7f1354e7b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1350147540 0x7f135015a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:16.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.325+0000 7f1354e7b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1350142000 con 0x7f13500b16c0 2022-08-01T19:43:16.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.325+0000 7f1354e7b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1350141d60 con 0x7f1350147540 2022-08-01T19:43:16.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.325+0000 7f1354e7b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13500b3450 con 0x7f135005fb80 2022-08-01T19:43:16.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.325+0000 7f134effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13500b16c0 0x7f1350159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:16.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.325+0000 7f134effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13500b16c0 0x7f1350159e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44502/0 (socket says 172.21.15.17:44502) 2022-08-01T19:43:16.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.325+0000 7f134effd700 1 -- 172.21.15.17:0/3211098237 learned_addr learned my addr 172.21.15.17:0/3211098237 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:16.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.325+0000 7f134ffff700 1 --2- 172.21.15.17:0/3211098237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1350147540 0x7f135015a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:16.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.325+0000 7f134f7fe700 1 --2- 172.21.15.17:0/3211098237 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f135005fb80 0x7f1350149fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:16.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.325+0000 7f134effd700 1 -- 172.21.15.17:0/3211098237 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f135005fb80 msgr2=0x7f1350149fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:16.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.325+0000 7f134effd700 1 --2- 172.21.15.17:0/3211098237 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f135005fb80 0x7f1350149fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:16.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.325+0000 7f134effd700 1 -- 172.21.15.17:0/3211098237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1350147540 msgr2=0x7f135015a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:16.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.325+0000 7f134effd700 1 --2- 172.21.15.17:0/3211098237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1350147540 0x7f135015a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:16.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.325+0000 7f134effd700 1 -- 172.21.15.17:0/3211098237 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f134400b180 con 0x7f13500b16c0 2022-08-01T19:43:16.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.325+0000 7f134ffff700 1 --2- 172.21.15.17:0/3211098237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1350147540 0x7f135015a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:43:16.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.325+0000 7f134f7fe700 1 --2- 172.21.15.17:0/3211098237 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f135005fb80 0x7f1350149fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:43:16.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.325+0000 7f134effd700 1 --2- 172.21.15.17:0/3211098237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13500b16c0 0x7f1350159e20 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 rx=0x7f134000c000 tx=0x7f13400091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:16.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.325+0000 7f134cff9700 1 -- 172.21.15.17:0/3211098237 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f134000c850 con 0x7f13500b16c0 2022-08-01T19:43:16.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.325+0000 7f134cff9700 1 -- 172.21.15.17:0/3211098237 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f134000c9b0 con 0x7f13500b16c0 2022-08-01T19:43:16.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.325+0000 7f1354e7b700 1 -- 172.21.15.17:0/3211098237 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13500b4310 con 0x7f13500b16c0 2022-08-01T19:43:16.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.325+0000 7f134cff9700 1 -- 172.21.15.17:0/3211098237 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f134000cd10 con 0x7f13500b16c0 2022-08-01T19:43:16.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.325+0000 7f1354e7b700 1 -- 172.21.15.17:0/3211098237 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13500b40d0 con 0x7f13500b16c0 2022-08-01T19:43:16.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.325+0000 7f1354e7b700 1 -- 172.21.15.17:0/3211098237 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13500abc30 con 0x7f13500b16c0 2022-08-01T19:43:16.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.329+0000 7f134cff9700 1 -- 172.21.15.17:0/3211098237 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1340022070 con 0x7f13500b16c0 2022-08-01T19:43:16.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.329+0000 7f134cff9700 1 --2- 172.21.15.17:0/3211098237 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1330053b00 0x7f1330055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:16.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.329+0000 7f134cff9700 1 -- 172.21.15.17:0/3211098237 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f134006c840 con 0x7f13500b16c0 2022-08-01T19:43:16.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.329+0000 7f134f7fe700 1 --2- 172.21.15.17:0/3211098237 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1330053b00 0x7f1330055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:16.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.329+0000 7f134f7fe700 1 --2- 172.21.15.17:0/3211098237 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1330053b00 0x7f1330055fc0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f1344015000 tx=0x7f1344014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:16.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.333+0000 7f134cff9700 1 -- 172.21.15.17:0/3211098237 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1340043600 con 0x7f13500b16c0 2022-08-01T19:43:16.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.461+0000 7f1354e7b700 1 -- 172.21.15.17:0/3211098237 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f13500574e0 con 0x7f13500b16c0 2022-08-01T19:43:16.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.461+0000 7f134cff9700 1 -- 172.21.15.17:0/3211098237 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v137) v1 ==== 110+0+10 (secure 0 0 0) 0x7f1340046c20 con 0x7f13500b16c0 2022-08-01T19:43:16.466 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:43:16.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.465+0000 7f1354e7b700 1 -- 172.21.15.17:0/3211098237 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1330053b00 msgr2=0x7f1330055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:16.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.465+0000 7f1354e7b700 1 --2- 172.21.15.17:0/3211098237 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1330053b00 0x7f1330055fc0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f1344015000 tx=0x7f1344014000).stop 2022-08-01T19:43:16.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.465+0000 7f1354e7b700 1 -- 172.21.15.17:0/3211098237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13500b16c0 msgr2=0x7f1350159e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:16.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.465+0000 7f1354e7b700 1 --2- 172.21.15.17:0/3211098237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13500b16c0 0x7f1350159e20 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 rx=0x7f134000c000 tx=0x7f13400091d0).stop 2022-08-01T19:43:16.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.465+0000 7f1354e7b700 1 -- 172.21.15.17:0/3211098237 shutdown_connections 2022-08-01T19:43:16.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.465+0000 7f1354e7b700 1 --2- 172.21.15.17:0/3211098237 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1330053b00 0x7f1330055fc0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:16.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.465+0000 7f1354e7b700 1 --2- 172.21.15.17:0/3211098237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1350147540 0x7f135015a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:16.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.465+0000 7f1354e7b700 1 --2- 172.21.15.17:0/3211098237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13500b16c0 0x7f1350159e20 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:16.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.465+0000 7f1354e7b700 1 --2- 172.21.15.17:0/3211098237 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f135005fb80 0x7f1350149fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:16.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.465+0000 7f1354e7b700 1 -- 172.21.15.17:0/3211098237 >> 172.21.15.17:0/3211098237 conn(0x7f13500b54f0 msgr2=0x7f135005bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:16.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.465+0000 7f1354e7b700 1 -- 172.21.15.17:0/3211098237 shutdown_connections 2022-08-01T19:43:16.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.465+0000 7f1354e7b700 1 -- 172.21.15.17:0/3211098237 wait complete. 2022-08-01T19:43:16.479 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:43:16.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.589+0000 7f594747e700 1 Processor -- start 2022-08-01T19:43:16.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.589+0000 7f594747e700 1 -- start start 2022-08-01T19:43:16.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.589+0000 7f594747e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f5940140b90 0x7f5940143c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:16.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.589+0000 7f594747e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f59400b4cd0 0x7f59400b5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:16.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.589+0000 7f594747e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59400b6a80 con 0x7f59400b66a0 2022-08-01T19:43:16.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.589+0000 7f594747e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5940144140 con 0x7f5940140b90 2022-08-01T19:43:16.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.589+0000 7f594747e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59400ae7f0 con 0x7f59400b4cd0 2022-08-01T19:43:16.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.589+0000 7f5946c7d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f59400b4cd0 0x7f59400b5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:16.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.589+0000 7f5945c7b700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f5940140b90 0x7f5940143c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:16.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.589+0000 7f5946c7d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f59400b4cd0 0x7f59400b5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:40612/0 (socket says 172.21.15.17:40612) 2022-08-01T19:43:16.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.589+0000 7f5946c7d700 1 -- 172.21.15.17:0/4168757624 learned_addr learned my addr 172.21.15.17:0/4168757624 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:16.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.593+0000 7f5946c7d700 1 -- 172.21.15.17:0/4168757624 >> v2:172.21.15.77:3300/0 conn(0x7f5940140b90 msgr2=0x7f5940143c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:16.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.593+0000 7f5946c7d700 1 --2- 172.21.15.17:0/4168757624 >> v2:172.21.15.77:3300/0 conn(0x7f5940140b90 0x7f5940143c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:16.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.593+0000 7f5946c7d700 1 -- 172.21.15.17:0/4168757624 >> v1:172.21.15.77:6789/0 conn(0x7f59400b66a0 legacy=0x7f5940140480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:16.602 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.593+0000 7f5946c7d700 1 -- 172.21.15.17:0/4168757624 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59400aea90 con 0x7f59400b4cd0 2022-08-01T19:43:16.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.593+0000 7f5945c7b700 1 --2- 172.21.15.17:0/4168757624 >> v2:172.21.15.77:3300/0 conn(0x7f5940140b90 0x7f5940143c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:43:16.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.593+0000 7f5946c7d700 1 --2- 172.21.15.17:0/4168757624 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f59400b4cd0 0x7f59400b5140 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f59380032f0 tx=0x7f5938009290).ready entity=mon.2 client_cookie=bf76d5d727c9e861 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:16.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.593+0000 7f594547a700 1 -- 172.21.15.17:0/4168757624 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f593800b8a0 con 0x7f59400b4cd0 2022-08-01T19:43:16.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.593+0000 7f594547a700 1 -- 172.21.15.17:0/4168757624 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593800ba00 con 0x7f59400b4cd0 2022-08-01T19:43:16.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.593+0000 7f594547a700 1 -- 172.21.15.17:0/4168757624 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f593800bb70 con 0x7f59400b4cd0 2022-08-01T19:43:16.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.593+0000 7f594747e700 1 -- 172.21.15.17:0/4168757624 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f59400b4cd0 msgr2=0x7f59400b5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:16.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.593+0000 7f594747e700 1 --2- 172.21.15.17:0/4168757624 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f59400b4cd0 0x7f59400b5140 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f59380032f0 tx=0x7f5938009290).stop 2022-08-01T19:43:16.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.593+0000 7f594747e700 1 -- 172.21.15.17:0/4168757624 shutdown_connections 2022-08-01T19:43:16.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.593+0000 7f594747e700 1 --2- 172.21.15.17:0/4168757624 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f59400b4cd0 0x7f59400b5140 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:16.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.593+0000 7f594747e700 1 --2- 172.21.15.17:0/4168757624 >> v2:172.21.15.77:3300/0 conn(0x7f5940140b90 0x7f5940143c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:16.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.593+0000 7f594747e700 1 -- 172.21.15.17:0/4168757624 >> 172.21.15.17:0/4168757624 conn(0x7f594005fa50 msgr2=0x7f5940061640 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:16.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.593+0000 7f594747e700 1 -- 172.21.15.17:0/4168757624 shutdown_connections 2022-08-01T19:43:16.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.593+0000 7f594747e700 1 -- 172.21.15.17:0/4168757624 wait complete. 2022-08-01T19:43:16.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.593+0000 7f594747e700 1 Processor -- start 2022-08-01T19:43:16.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.593+0000 7f594747e700 1 -- start start 2022-08-01T19:43:16.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.597+0000 7f594747e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59400b4cd0 0x7f594005cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:16.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.597+0000 7f594747e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f59400b66a0 0x7f5940142f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:16.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.597+0000 7f594747e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5940140b90 0x7f5940149db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:16.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.597+0000 7f594747e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59400b2420 con 0x7f59400b4cd0 2022-08-01T19:43:16.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.597+0000 7f594747e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59400b26c0 con 0x7f5940140b90 2022-08-01T19:43:16.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.597+0000 7f594747e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59400b2960 con 0x7f59400b66a0 2022-08-01T19:43:16.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.597+0000 7f594647c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59400b4cd0 0x7f594005cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:16.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.597+0000 7f594647c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59400b4cd0 0x7f594005cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44514/0 (socket says 172.21.15.17:44514) 2022-08-01T19:43:16.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.597+0000 7f594647c700 1 -- 172.21.15.17:0/791788431 learned_addr learned my addr 172.21.15.17:0/791788431 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:16.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.597+0000 7f5945c7b700 1 --2- 172.21.15.17:0/791788431 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f59400b66a0 0x7f5940142f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:16.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.597+0000 7f5946c7d700 1 --2- 172.21.15.17:0/791788431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5940140b90 0x7f5940149db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:16.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.597+0000 7f594647c700 1 -- 172.21.15.17:0/791788431 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f59400b66a0 msgr2=0x7f5940142f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:16.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.597+0000 7f594647c700 1 --2- 172.21.15.17:0/791788431 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f59400b66a0 0x7f5940142f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:16.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.597+0000 7f594647c700 1 -- 172.21.15.17:0/791788431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5940140b90 msgr2=0x7f5940149db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:16.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.597+0000 7f594647c700 1 --2- 172.21.15.17:0/791788431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5940140b90 0x7f5940149db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:16.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.597+0000 7f594647c700 1 -- 172.21.15.17:0/791788431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f593c00b180 con 0x7f59400b4cd0 2022-08-01T19:43:16.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.597+0000 7f594647c700 1 --2- 172.21.15.17:0/791788431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59400b4cd0 0x7f594005cc30 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 rx=0x7f594006b1c0 tx=0x7f5934007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:16.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.597+0000 7f59337fe700 1 -- 172.21.15.17:0/791788431 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5934007030 con 0x7f59400b4cd0 2022-08-01T19:43:16.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.597+0000 7f59337fe700 1 -- 172.21.15.17:0/791788431 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5934007190 con 0x7f59400b4cd0 2022-08-01T19:43:16.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.597+0000 7f594747e700 1 -- 172.21.15.17:0/791788431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5938009530 con 0x7f59400b4cd0 2022-08-01T19:43:16.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.597+0000 7f59337fe700 1 -- 172.21.15.17:0/791788431 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5934006510 con 0x7f59400b4cd0 2022-08-01T19:43:16.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.597+0000 7f594747e700 1 -- 172.21.15.17:0/791788431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5940145a20 con 0x7f59400b4cd0 2022-08-01T19:43:16.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.597+0000 7f59337fe700 1 -- 172.21.15.17:0/791788431 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5934010030 con 0x7f59400b4cd0 2022-08-01T19:43:16.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.597+0000 7f594747e700 1 -- 172.21.15.17:0/791788431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59400b2b20 con 0x7f59400b4cd0 2022-08-01T19:43:16.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.601+0000 7f59337fe700 1 --2- 172.21.15.17:0/791788431 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5920053a10 0x7f5920055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:16.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.601+0000 7f59337fe700 1 -- 172.21.15.17:0/791788431 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f593406bbf0 con 0x7f59400b4cd0 2022-08-01T19:43:16.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.601+0000 7f5945c7b700 1 --2- 172.21.15.17:0/791788431 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5920053a10 0x7f5920055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:16.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.601+0000 7f5945c7b700 1 --2- 172.21.15.17:0/791788431 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5920053a10 0x7f5920055ed0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f593c00b000 tx=0x7f593c006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.605+0000 7f59337fe700 1 -- 172.21.15.17:0/791788431 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f59340428e0 con 0x7f59400b4cd0 2022-08-01T19:43:16.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.729+0000 7f594747e700 1 -- 172.21.15.17:0/791788431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f59400aed30 con 0x7f59400b4cd0 2022-08-01T19:43:16.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.729+0000 7f59337fe700 1 -- 172.21.15.17:0/791788431 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v137) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5934045f00 con 0x7f59400b4cd0 2022-08-01T19:43:16.734 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:43:16.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.733+0000 7f594747e700 1 -- 172.21.15.17:0/791788431 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5920053a10 msgr2=0x7f5920055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:16.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.733+0000 7f594747e700 1 --2- 172.21.15.17:0/791788431 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5920053a10 0x7f5920055ed0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f593c00b000 tx=0x7f593c006df0).stop 2022-08-01T19:43:16.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.733+0000 7f594747e700 1 -- 172.21.15.17:0/791788431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59400b4cd0 msgr2=0x7f594005cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:16.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.733+0000 7f594747e700 1 --2- 172.21.15.17:0/791788431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59400b4cd0 0x7f594005cc30 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 rx=0x7f594006b1c0 tx=0x7f5934007c10).stop 2022-08-01T19:43:16.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.733+0000 7f594747e700 1 -- 172.21.15.17:0/791788431 shutdown_connections 2022-08-01T19:43:16.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.733+0000 7f594747e700 1 --2- 172.21.15.17:0/791788431 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5920053a10 0x7f5920055ed0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:16.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.733+0000 7f594747e700 1 --2- 172.21.15.17:0/791788431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5940140b90 0x7f5940149db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:16.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.733+0000 7f594747e700 1 --2- 172.21.15.17:0/791788431 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f59400b66a0 0x7f5940142f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:16.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.733+0000 7f594747e700 1 --2- 172.21.15.17:0/791788431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59400b4cd0 0x7f594005cc30 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:16.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.733+0000 7f594747e700 1 -- 172.21.15.17:0/791788431 >> 172.21.15.17:0/791788431 conn(0x7f594005fa50 msgr2=0x7f59400b7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:16.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.733+0000 7f594747e700 1 -- 172.21.15.17:0/791788431 shutdown_connections 2022-08-01T19:43:16.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.733+0000 7f594747e700 1 -- 172.21.15.17:0/791788431 wait complete. 2022-08-01T19:43:16.748 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T19:43:16.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.861+0000 7f42d53f7700 1 Processor -- start 2022-08-01T19:43:16.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.861+0000 7f42d53f7700 1 -- start start 2022-08-01T19:43:16.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.861+0000 7f42d53f7700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f42d005fb80 0x7f42d014b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:16.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.861+0000 7f42d53f7700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f42d00b16c0 0x7f42d00603d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:16.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.861+0000 7f42d53f7700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42d005ff60 con 0x7f42d0147540 2022-08-01T19:43:16.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.861+0000 7f42d53f7700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d00608c0 con 0x7f42d005fb80 2022-08-01T19:43:16.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.861+0000 7f42d53f7700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d014ac90 con 0x7f42d00b16c0 2022-08-01T19:43:16.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.861+0000 7f42cf7fe700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f42d00b16c0 0x7f42d00603d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:16.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.861+0000 7f42cffff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f42d005fb80 0x7f42d014b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:16.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.861+0000 7f42cf7fe700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f42d00b16c0 0x7f42d00603d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:40622/0 (socket says 172.21.15.17:40622) 2022-08-01T19:43:16.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.861+0000 7f42d4bf6700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7f42d0147540 0x7f42d00ad8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:53516/0 (socket says 172.21.15.17:53516) 2022-08-01T19:43:16.870 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.861+0000 7f42cf7fe700 1 -- 172.21.15.17:0/3643864437 learned_addr learned my addr 172.21.15.17:0/3643864437 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:16.870 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.861+0000 7f42cffff700 1 -- 172.21.15.17:0/3643864437 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f42d00b16c0 msgr2=0x7f42d00603d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:16.871 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.861+0000 7f42cffff700 1 --2- 172.21.15.17:0/3643864437 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f42d00b16c0 0x7f42d00603d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:16.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.861+0000 7f42cffff700 1 -- 172.21.15.17:0/3643864437 >> v1:172.21.15.77:6789/0 conn(0x7f42d0147540 legacy=0x7f42d00ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:16.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.861+0000 7f42cffff700 1 -- 172.21.15.17:0/3643864437 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d014af30 con 0x7f42d005fb80 2022-08-01T19:43:16.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.865+0000 7f42cffff700 1 --2- 172.21.15.17:0/3643864437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42d005fb80 0x7f42d014b670 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f42d00ad3e0 tx=0x7f42b8006df0).ready entity=mon.1 client_cookie=f64196bfbf1da23c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:16.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.865+0000 7f42ceffd700 1 -- 172.21.15.17:0/3643864437 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f42b800d570 con 0x7f42d005fb80 2022-08-01T19:43:16.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.865+0000 7f42ceffd700 1 -- 172.21.15.17:0/3643864437 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b8000e90 con 0x7f42d005fb80 2022-08-01T19:43:16.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.865+0000 7f42ceffd700 1 -- 172.21.15.17:0/3643864437 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f42b800d8c0 con 0x7f42d005fb80 2022-08-01T19:43:16.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.865+0000 7f42d53f7700 1 -- 172.21.15.17:0/3643864437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42d005fb80 msgr2=0x7f42d014b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:16.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.865+0000 7f42d53f7700 1 --2- 172.21.15.17:0/3643864437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42d005fb80 0x7f42d014b670 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f42d00ad3e0 tx=0x7f42b8006df0).stop 2022-08-01T19:43:16.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.865+0000 7f42d53f7700 1 -- 172.21.15.17:0/3643864437 shutdown_connections 2022-08-01T19:43:16.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.865+0000 7f42d53f7700 1 --2- 172.21.15.17:0/3643864437 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f42d00b16c0 0x7f42d00603d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:16.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.865+0000 7f42d53f7700 1 --2- 172.21.15.17:0/3643864437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42d005fb80 0x7f42d014b670 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:16.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.865+0000 7f42d53f7700 1 -- 172.21.15.17:0/3643864437 >> 172.21.15.17:0/3643864437 conn(0x7f42d00b54f0 msgr2=0x7f42d00b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:16.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.865+0000 7f42d53f7700 1 -- 172.21.15.17:0/3643864437 shutdown_connections 2022-08-01T19:43:16.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.865+0000 7f42d53f7700 1 -- 172.21.15.17:0/3643864437 wait complete. 2022-08-01T19:43:16.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.865+0000 7f42d53f7700 1 Processor -- start 2022-08-01T19:43:16.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.865+0000 7f42d53f7700 1 -- start start 2022-08-01T19:43:16.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.869+0000 7f42d53f7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42d005fb80 0x7f42d014e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:16.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.869+0000 7f42d53f7700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f42d00b16c0 0x7f42d0140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:16.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.869+0000 7f42d53f7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42d0147540 0x7f42d01413e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:16.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.869+0000 7f42d53f7700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d00abf80 con 0x7f42d0147540 2022-08-01T19:43:16.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.869+0000 7f42d53f7700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d0142070 con 0x7f42d005fb80 2022-08-01T19:43:16.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.869+0000 7f42d53f7700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d0142310 con 0x7f42d00b16c0 2022-08-01T19:43:16.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.869+0000 7f42d4bf6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42d0147540 0x7f42d01413e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:16.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.869+0000 7f42d4bf6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42d0147540 0x7f42d01413e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44532/0 (socket says 172.21.15.17:44532) 2022-08-01T19:43:16.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.869+0000 7f42d4bf6700 1 -- 172.21.15.17:0/2742909917 learned_addr learned my addr 172.21.15.17:0/2742909917 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:16.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.869+0000 7f42cf7fe700 1 --2- 172.21.15.17:0/2742909917 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f42d00b16c0 0x7f42d0140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:16.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.869+0000 7f42cffff700 1 --2- 172.21.15.17:0/2742909917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42d005fb80 0x7f42d014e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:16.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.869+0000 7f42cf7fe700 1 -- 172.21.15.17:0/2742909917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42d005fb80 msgr2=0x7f42d014e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:16.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.869+0000 7f42cf7fe700 1 --2- 172.21.15.17:0/2742909917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42d005fb80 0x7f42d014e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:16.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.869+0000 7f42cf7fe700 1 -- 172.21.15.17:0/2742909917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42d0147540 msgr2=0x7f42d01413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:16.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.869+0000 7f42cf7fe700 1 --2- 172.21.15.17:0/2742909917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42d0147540 0x7f42d01413e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:16.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.869+0000 7f42cf7fe700 1 -- 172.21.15.17:0/2742909917 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b800b180 con 0x7f42d00b16c0 2022-08-01T19:43:16.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.869+0000 7f42cf7fe700 1 --2- 172.21.15.17:0/2742909917 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f42d00b16c0 0x7f42d0140ef0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f42d0156930 tx=0x7f42c0008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:16.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.869+0000 7f42cd7fa700 1 -- 172.21.15.17:0/2742909917 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f42c0013400 con 0x7f42d00b16c0 2022-08-01T19:43:16.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.869+0000 7f42cd7fa700 1 -- 172.21.15.17:0/2742909917 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c0013560 con 0x7f42d00b16c0 2022-08-01T19:43:16.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.869+0000 7f42d53f7700 1 -- 172.21.15.17:0/2742909917 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d01425b0 con 0x7f42d00b16c0 2022-08-01T19:43:16.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.869+0000 7f42cd7fa700 1 -- 172.21.15.17:0/2742909917 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f42c00138c0 con 0x7f42d00b16c0 2022-08-01T19:43:16.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.869+0000 7f42d53f7700 1 -- 172.21.15.17:0/2742909917 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d00b5170 con 0x7f42d00b16c0 2022-08-01T19:43:16.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.869+0000 7f42d53f7700 1 -- 172.21.15.17:0/2742909917 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d00574e0 con 0x7f42d00b16c0 2022-08-01T19:43:16.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.869+0000 7f42cd7fa700 1 -- 172.21.15.17:0/2742909917 <== mon.2 v2:172.21.15.77:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f42c0013a20 con 0x7f42d00b16c0 2022-08-01T19:43:16.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.873+0000 7f42cd7fa700 1 --2- 172.21.15.17:0/2742909917 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f42b40539e0 0x7f42b4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:16.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.873+0000 7f42cd7fa700 1 -- 172.21.15.17:0/2742909917 <== mon.2 v2:172.21.15.77:3301/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f42c006c8c0 con 0x7f42d00b16c0 2022-08-01T19:43:16.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.873+0000 7f42cffff700 1 --2- 172.21.15.17:0/2742909917 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f42b40539e0 0x7f42b4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:16.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.873+0000 7f42cffff700 1 --2- 172.21.15.17:0/2742909917 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f42b40539e0 0x7f42b4055ea0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f42b801a000 tx=0x7f42b8014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:16.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:16.881+0000 7f42cd7fa700 1 -- 172.21.15.17:0/2742909917 <== mon.2 v2:172.21.15.77:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f42c0043680 con 0x7f42d00b16c0 2022-08-01T19:43:17.010 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.005+0000 7f42d53f7700 1 -- 172.21.15.17:0/2742909917 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f42d014b330 con 0x7f42d00b16c0 2022-08-01T19:43:17.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.009+0000 7f42cd7fa700 1 -- 172.21.15.17:0/2742909917 <== mon.2 v2:172.21.15.77:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v137) v1 ==== 104+0+10 (secure 0 0 0) 0x7f42c0013cb0 con 0x7f42d00b16c0 2022-08-01T19:43:17.013 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:43:17.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.009+0000 7f42d53f7700 1 -- 172.21.15.17:0/2742909917 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f42b40539e0 msgr2=0x7f42b4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:17.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.009+0000 7f42d53f7700 1 --2- 172.21.15.17:0/2742909917 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f42b40539e0 0x7f42b4055ea0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f42b801a000 tx=0x7f42b8014000).stop 2022-08-01T19:43:17.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.009+0000 7f42d53f7700 1 -- 172.21.15.17:0/2742909917 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f42d00b16c0 msgr2=0x7f42d0140ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:17.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.009+0000 7f42d53f7700 1 --2- 172.21.15.17:0/2742909917 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f42d00b16c0 0x7f42d0140ef0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f42d0156930 tx=0x7f42c0008040).stop 2022-08-01T19:43:17.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.009+0000 7f42d53f7700 1 -- 172.21.15.17:0/2742909917 shutdown_connections 2022-08-01T19:43:17.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.009+0000 7f42d53f7700 1 --2- 172.21.15.17:0/2742909917 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f42b40539e0 0x7f42b4055ea0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:17.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.009+0000 7f42d53f7700 1 --2- 172.21.15.17:0/2742909917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42d0147540 0x7f42d01413e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:17.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.009+0000 7f42d53f7700 1 --2- 172.21.15.17:0/2742909917 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f42d00b16c0 0x7f42d0140ef0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:17.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.009+0000 7f42d53f7700 1 --2- 172.21.15.17:0/2742909917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42d005fb80 0x7f42d014e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:17.018 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.009+0000 7f42d53f7700 1 -- 172.21.15.17:0/2742909917 >> 172.21.15.17:0/2742909917 conn(0x7f42d00b54f0 msgr2=0x7f42d0146970 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:17.018 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.009+0000 7f42d53f7700 1 -- 172.21.15.17:0/2742909917 shutdown_connections 2022-08-01T19:43:17.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.009+0000 7f42d53f7700 1 -- 172.21.15.17:0/2742909917 wait complete. 2022-08-01T19:43:17.024 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T19:43:17.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.133+0000 7f5b80bb4700 1 Processor -- start 2022-08-01T19:43:17.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.133+0000 7f5b80bb4700 1 -- start start 2022-08-01T19:43:17.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.137+0000 7f5b80bb4700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5b7c0b3ad0 0x7f5b7c0b2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:17.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.137+0000 7f5b80bb4700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5b7c065a20 0x7f5b7c0b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:17.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.137+0000 7f5b80bb4700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b7c0b3eb0 con 0x7f5b7c0b30f0 2022-08-01T19:43:17.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.137+0000 7f5b80bb4700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b7c064d80 con 0x7f5b7c0b3ad0 2022-08-01T19:43:17.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.137+0000 7f5b80bb4700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c065020 con 0x7f5b7c065a20 2022-08-01T19:43:17.154 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.137+0000 7f5b7a59c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5b7c0b3ad0 0x7f5b7c0b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:17.154 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.137+0000 7f5b7a59c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5b7c0b3ad0 0x7f5b7c0b2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44536/0 (socket says 172.21.15.17:44536) 2022-08-01T19:43:17.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.137+0000 7f5b7a59c700 1 -- 172.21.15.17:0/1203773047 learned_addr learned my addr 172.21.15.17:0/1203773047 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:17.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.137+0000 7f5b79d9b700 1 --2- 172.21.15.17:0/1203773047 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5b7c065a20 0x7f5b7c0b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:17.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.137+0000 7f5b7a59c700 1 -- 172.21.15.17:0/1203773047 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5b7c065a20 msgr2=0x7f5b7c0b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:17.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.137+0000 7f5b7a59c700 1 --2- 172.21.15.17:0/1203773047 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5b7c065a20 0x7f5b7c0b2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:17.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.137+0000 7f5b7a59c700 1 -- 172.21.15.17:0/1203773047 >> v1:172.21.15.17:6789/0 conn(0x7f5b7c0b30f0 legacy=0x7f5b7c144a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:17.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.137+0000 7f5b7a59c700 1 -- 172.21.15.17:0/1203773047 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7c0652c0 con 0x7f5b7c0b3ad0 2022-08-01T19:43:17.158 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.137+0000 7f5b79d9b700 1 --2- 172.21.15.17:0/1203773047 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5b7c065a20 0x7f5b7c0b2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:43:17.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.137+0000 7f5b7a59c700 1 --2- 172.21.15.17:0/1203773047 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b7c0b3ad0 0x7f5b7c0b2710 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 rx=0x7f5b6c002f70 tx=0x7f5b6c006eb0).ready entity=mon.0 client_cookie=3c98595679c1d7f5 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:17.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.137+0000 7f5b7959a700 1 -- 172.21.15.17:0/1203773047 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b6c00d5c0 con 0x7f5b7c0b3ad0 2022-08-01T19:43:17.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.137+0000 7f5b7959a700 1 -- 172.21.15.17:0/1203773047 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b6c000e90 con 0x7f5b7c0b3ad0 2022-08-01T19:43:17.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.137+0000 7f5b7959a700 1 -- 172.21.15.17:0/1203773047 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b6c00d950 con 0x7f5b7c0b3ad0 2022-08-01T19:43:17.163 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.137+0000 7f5b80bb4700 1 -- 172.21.15.17:0/1203773047 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b7c0b3ad0 msgr2=0x7f5b7c0b2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:17.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.137+0000 7f5b80bb4700 1 --2- 172.21.15.17:0/1203773047 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b7c0b3ad0 0x7f5b7c0b2710 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 rx=0x7f5b6c002f70 tx=0x7f5b6c006eb0).stop 2022-08-01T19:43:17.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.137+0000 7f5b80bb4700 1 -- 172.21.15.17:0/1203773047 shutdown_connections 2022-08-01T19:43:17.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.137+0000 7f5b80bb4700 1 --2- 172.21.15.17:0/1203773047 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5b7c065a20 0x7f5b7c0b2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:17.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.137+0000 7f5b80bb4700 1 --2- 172.21.15.17:0/1203773047 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b7c0b3ad0 0x7f5b7c0b2710 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:17.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.137+0000 7f5b80bb4700 1 -- 172.21.15.17:0/1203773047 >> 172.21.15.17:0/1203773047 conn(0x7f5b7c0b78d0 msgr2=0x7f5b7c0b7ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:17.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.137+0000 7f5b80bb4700 1 -- 172.21.15.17:0/1203773047 shutdown_connections 2022-08-01T19:43:17.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.137+0000 7f5b80bb4700 1 -- 172.21.15.17:0/1203773047 wait complete. 2022-08-01T19:43:17.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.141+0000 7f5b80bb4700 1 Processor -- start 2022-08-01T19:43:17.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.141+0000 7f5b80bb4700 1 -- start start 2022-08-01T19:43:17.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.141+0000 7f5b80bb4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b7c065a20 0x7f5b7c09ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:17.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.141+0000 7f5b80bb4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b7c0b30f0 0x7f5b7c0b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:17.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.145+0000 7f5b80bb4700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5b7c0b3ad0 0x7f5b7c151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:17.169 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.145+0000 7f5b80bb4700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c0b0b90 con 0x7f5b7c0b30f0 2022-08-01T19:43:17.169 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.145+0000 7f5b80bb4700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c151600 con 0x7f5b7c065a20 2022-08-01T19:43:17.170 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.145+0000 7f5b80bb4700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c0b0db0 con 0x7f5b7c0b3ad0 2022-08-01T19:43:17.170 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.145+0000 7f5b79d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b7c0b30f0 0x7f5b7c0b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:17.171 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.145+0000 7f5b7a59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b7c065a20 0x7f5b7c09ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:17.171 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.145+0000 7f5b79d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b7c0b30f0 0x7f5b7c0b06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44544/0 (socket says 172.21.15.17:44544) 2022-08-01T19:43:17.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.145+0000 7f5b79d9b700 1 -- 172.21.15.17:0/965631846 learned_addr learned my addr 172.21.15.17:0/965631846 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:17.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.145+0000 7f5b7ad9d700 1 --2- 172.21.15.17:0/965631846 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5b7c0b3ad0 0x7f5b7c151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:17.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.145+0000 7f5b79d9b700 1 -- 172.21.15.17:0/965631846 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5b7c0b3ad0 msgr2=0x7f5b7c151110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:17.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.145+0000 7f5b79d9b700 1 --2- 172.21.15.17:0/965631846 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5b7c0b3ad0 0x7f5b7c151110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:17.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.145+0000 7f5b79d9b700 1 -- 172.21.15.17:0/965631846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b7c065a20 msgr2=0x7f5b7c09ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:17.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.145+0000 7f5b79d9b700 1 --2- 172.21.15.17:0/965631846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b7c065a20 0x7f5b7c09ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:17.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.145+0000 7f5b79d9b700 1 -- 172.21.15.17:0/965631846 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b6c00b1e0 con 0x7f5b7c0b30f0 2022-08-01T19:43:17.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.145+0000 7f5b7ad9d700 1 --2- 172.21.15.17:0/965631846 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5b7c0b3ad0 0x7f5b7c151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:43:17.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.145+0000 7f5b7a59c700 1 --2- 172.21.15.17:0/965631846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b7c065a20 0x7f5b7c09ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:43:17.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.145+0000 7f5b79d9b700 1 --2- 172.21.15.17:0/965631846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b7c0b30f0 0x7f5b7c0b06a0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 rx=0x7f5b64007000 tx=0x7f5b64008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:17.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.145+0000 7f5b737fe700 1 -- 172.21.15.17:0/965631846 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b64010580 con 0x7f5b7c0b30f0 2022-08-01T19:43:17.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.145+0000 7f5b737fe700 1 -- 172.21.15.17:0/965631846 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b640106e0 con 0x7f5b7c0b30f0 2022-08-01T19:43:17.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.145+0000 7f5b80bb4700 1 -- 172.21.15.17:0/965631846 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b7c0b1050 con 0x7f5b7c0b30f0 2022-08-01T19:43:17.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.145+0000 7f5b737fe700 1 -- 172.21.15.17:0/965631846 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b64010a40 con 0x7f5b7c0b30f0 2022-08-01T19:43:17.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.145+0000 7f5b80bb4700 1 -- 172.21.15.17:0/965631846 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b7c0ad490 con 0x7f5b7c0b30f0 2022-08-01T19:43:17.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.145+0000 7f5b80bb4700 1 -- 172.21.15.17:0/965631846 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b7c0b2270 con 0x7f5b7c0b30f0 2022-08-01T19:43:17.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.145+0000 7f5b737fe700 1 -- 172.21.15.17:0/965631846 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5b64010ba0 con 0x7f5b7c0b30f0 2022-08-01T19:43:17.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.149+0000 7f5b737fe700 1 --2- 172.21.15.17:0/965631846 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5b60053a10 0x7f5b60055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:17.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.149+0000 7f5b737fe700 1 -- 172.21.15.17:0/965631846 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5b6406bd80 con 0x7f5b7c0b30f0 2022-08-01T19:43:17.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.149+0000 7f5b7a59c700 1 --2- 172.21.15.17:0/965631846 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5b60053a10 0x7f5b60055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:17.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.149+0000 7f5b7a59c700 1 --2- 172.21.15.17:0/965631846 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5b60053a10 0x7f5b60055ed0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f5b6c020000 tx=0x7f5b6c015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:17.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.153+0000 7f5b737fe700 1 -- 172.21.15.17:0/965631846 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5b64042ae0 con 0x7f5b7c0b30f0 2022-08-01T19:43:17.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.285+0000 7f5b80bb4700 1 -- 172.21.15.17:0/965631846 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5b7c0574e0 con 0x7f5b7c0b30f0 2022-08-01T19:43:17.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.285+0000 7f5b737fe700 1 -- 172.21.15.17:0/965631846 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v137) v1 ==== 100+0+10 (secure 0 0 0) 0x7f5b6406d070 con 0x7f5b7c0b30f0 2022-08-01T19:43:17.289 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:43:17.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.285+0000 7f5b80bb4700 1 -- 172.21.15.17:0/965631846 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5b60053a10 msgr2=0x7f5b60055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:17.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.285+0000 7f5b80bb4700 1 --2- 172.21.15.17:0/965631846 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5b60053a10 0x7f5b60055ed0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f5b6c020000 tx=0x7f5b6c015000).stop 2022-08-01T19:43:17.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.285+0000 7f5b80bb4700 1 -- 172.21.15.17:0/965631846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b7c0b30f0 msgr2=0x7f5b7c0b06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:17.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.285+0000 7f5b80bb4700 1 --2- 172.21.15.17:0/965631846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b7c0b30f0 0x7f5b7c0b06a0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 rx=0x7f5b64007000 tx=0x7f5b64008040).stop 2022-08-01T19:43:17.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.289+0000 7f5b80bb4700 1 -- 172.21.15.17:0/965631846 shutdown_connections 2022-08-01T19:43:17.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.289+0000 7f5b80bb4700 1 --2- 172.21.15.17:0/965631846 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5b60053a10 0x7f5b60055ed0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:17.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.289+0000 7f5b80bb4700 1 --2- 172.21.15.17:0/965631846 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5b7c0b3ad0 0x7f5b7c151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:17.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.289+0000 7f5b80bb4700 1 --2- 172.21.15.17:0/965631846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b7c0b30f0 0x7f5b7c0b06a0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:17.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.289+0000 7f5b80bb4700 1 --2- 172.21.15.17:0/965631846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b7c065a20 0x7f5b7c09ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:17.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.289+0000 7f5b80bb4700 1 -- 172.21.15.17:0/965631846 >> 172.21.15.17:0/965631846 conn(0x7f5b7c0b78d0 msgr2=0x7f5b7c0b7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:17.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.289+0000 7f5b80bb4700 1 -- 172.21.15.17:0/965631846 shutdown_connections 2022-08-01T19:43:17.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.289+0000 7f5b80bb4700 1 -- 172.21.15.17:0/965631846 wait complete. 2022-08-01T19:43:17.303 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T19:43:17.304 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:43:17.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.405+0000 7fee56cea700 1 Processor -- start 2022-08-01T19:43:17.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.405+0000 7fee56cea700 1 -- start start 2022-08-01T19:43:17.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.405+0000 7fee56cea700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fee500b66a0 0x7fee50143c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:17.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.405+0000 7fee56cea700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fee500b4cd0 0x7fee500b5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:17.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.405+0000 7fee56cea700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee500b6a80 con 0x7fee50140480 2022-08-01T19:43:17.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.405+0000 7fee56cea700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee50144140 con 0x7fee500b4cd0 2022-08-01T19:43:17.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.405+0000 7fee56cea700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee500ae7f0 con 0x7fee500b66a0 2022-08-01T19:43:17.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.405+0000 7fee564e9700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fee500b4cd0 0x7fee500b5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:17.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.405+0000 7fee564e9700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fee500b4cd0 0x7fee500b5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44550/0 (socket says 172.21.15.17:44550) 2022-08-01T19:43:17.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.405+0000 7fee564e9700 1 -- 172.21.15.17:0/2682323638 learned_addr learned my addr 172.21.15.17:0/2682323638 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:17.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.405+0000 7fee55ce8700 1 --2- 172.21.15.17:0/2682323638 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fee500b66a0 0x7fee50143c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:17.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.405+0000 7fee564e9700 1 -- 172.21.15.17:0/2682323638 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fee500b66a0 msgr2=0x7fee50143c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:17.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.405+0000 7fee564e9700 1 --2- 172.21.15.17:0/2682323638 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fee500b66a0 0x7fee50143c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:17.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.405+0000 7fee564e9700 1 -- 172.21.15.17:0/2682323638 >> v1:172.21.15.77:6789/0 conn(0x7fee50140480 legacy=0x7fee501408c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:17.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.405+0000 7fee564e9700 1 -- 172.21.15.17:0/2682323638 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee500aea90 con 0x7fee500b4cd0 2022-08-01T19:43:17.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.405+0000 7fee55ce8700 1 --2- 172.21.15.17:0/2682323638 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fee500b66a0 0x7fee50143c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:43:17.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.405+0000 7fee564e9700 1 --2- 172.21.15.17:0/2682323638 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee500b4cd0 0x7fee500b5140 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 rx=0x7fee480032f0 tx=0x7fee48009290).ready entity=mon.0 client_cookie=f1527a6df6817778 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:17.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.405+0000 7fee54ce6700 1 -- 172.21.15.17:0/2682323638 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee4800b8a0 con 0x7fee500b4cd0 2022-08-01T19:43:17.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.405+0000 7fee54ce6700 1 -- 172.21.15.17:0/2682323638 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4800ba00 con 0x7fee500b4cd0 2022-08-01T19:43:17.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.405+0000 7fee54ce6700 1 -- 172.21.15.17:0/2682323638 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee4800bb70 con 0x7fee500b4cd0 2022-08-01T19:43:17.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.409+0000 7fee56cea700 1 -- 172.21.15.17:0/2682323638 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee500b4cd0 msgr2=0x7fee500b5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:17.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.409+0000 7fee56cea700 1 --2- 172.21.15.17:0/2682323638 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee500b4cd0 0x7fee500b5140 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 rx=0x7fee480032f0 tx=0x7fee48009290).stop 2022-08-01T19:43:17.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.409+0000 7fee56cea700 1 -- 172.21.15.17:0/2682323638 shutdown_connections 2022-08-01T19:43:17.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.409+0000 7fee56cea700 1 --2- 172.21.15.17:0/2682323638 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee500b4cd0 0x7fee500b5140 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:17.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.409+0000 7fee56cea700 1 --2- 172.21.15.17:0/2682323638 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fee500b66a0 0x7fee50143c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:17.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.409+0000 7fee56cea700 1 -- 172.21.15.17:0/2682323638 >> 172.21.15.17:0/2682323638 conn(0x7fee5005fa50 msgr2=0x7fee50061640 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:17.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.409+0000 7fee56cea700 1 -- 172.21.15.17:0/2682323638 shutdown_connections 2022-08-01T19:43:17.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.409+0000 7fee56cea700 1 -- 172.21.15.17:0/2682323638 wait complete. 2022-08-01T19:43:17.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.409+0000 7fee56cea700 1 Processor -- start 2022-08-01T19:43:17.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.409+0000 7fee56cea700 1 -- start start 2022-08-01T19:43:17.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.409+0000 7fee56cea700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee500b4cd0 0x7fee500b3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:17.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.409+0000 7fee56cea700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fee500b66a0 0x7fee50148da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:17.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.409+0000 7fee56cea700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee50140480 0x7fee50155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:17.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.409+0000 7fee56cea700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee501418e0 con 0x7fee50140480 2022-08-01T19:43:17.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.409+0000 7fee56cea700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee500b3200 con 0x7fee500b4cd0 2022-08-01T19:43:17.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.409+0000 7fee56cea700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee500b34a0 con 0x7fee500b66a0 2022-08-01T19:43:17.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.409+0000 7fee564e9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee50140480 0x7fee50155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:17.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.409+0000 7fee55ce8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee500b4cd0 0x7fee500b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:17.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.409+0000 7fee564e9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee50140480 0x7fee50155ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44556/0 (socket says 172.21.15.17:44556) 2022-08-01T19:43:17.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.409+0000 7fee564e9700 1 -- 172.21.15.17:0/3159186581 learned_addr learned my addr 172.21.15.17:0/3159186581 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:17.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.409+0000 7fee554e7700 1 --2- 172.21.15.17:0/3159186581 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fee500b66a0 0x7fee50148da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:17.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.409+0000 7fee564e9700 1 -- 172.21.15.17:0/3159186581 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fee500b66a0 msgr2=0x7fee50148da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:17.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.409+0000 7fee564e9700 1 --2- 172.21.15.17:0/3159186581 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fee500b66a0 0x7fee50148da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:17.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.409+0000 7fee564e9700 1 -- 172.21.15.17:0/3159186581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee500b4cd0 msgr2=0x7fee500b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:17.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.409+0000 7fee564e9700 1 --2- 172.21.15.17:0/3159186581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee500b4cd0 0x7fee500b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:17.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.409+0000 7fee564e9700 1 -- 172.21.15.17:0/3159186581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee48009530 con 0x7fee50140480 2022-08-01T19:43:17.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.409+0000 7fee564e9700 1 --2- 172.21.15.17:0/3159186581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee50140480 0x7fee50155ae0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 rx=0x7fee480032f0 tx=0x7fee48016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:17.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.409+0000 7fee42ffd700 1 -- 172.21.15.17:0/3159186581 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee4800b8a0 con 0x7fee50140480 2022-08-01T19:43:17.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.409+0000 7fee42ffd700 1 -- 172.21.15.17:0/3159186581 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4800bb70 con 0x7fee50140480 2022-08-01T19:43:17.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.409+0000 7fee56cea700 1 -- 172.21.15.17:0/3159186581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee500b2d90 con 0x7fee50140480 2022-08-01T19:43:17.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.413+0000 7fee42ffd700 1 -- 172.21.15.17:0/3159186581 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee48020620 con 0x7fee50140480 2022-08-01T19:43:17.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.413+0000 7fee56cea700 1 -- 172.21.15.17:0/3159186581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee501561f0 con 0x7fee50140480 2022-08-01T19:43:17.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.413+0000 7fee56cea700 1 -- 172.21.15.17:0/3159186581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee500aed30 con 0x7fee50140480 2022-08-01T19:43:17.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.413+0000 7fee42ffd700 1 -- 172.21.15.17:0/3159186581 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fee48026020 con 0x7fee50140480 2022-08-01T19:43:17.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.413+0000 7fee42ffd700 1 --2- 172.21.15.17:0/3159186581 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fee30053b30 0x7fee30055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:17.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.413+0000 7fee42ffd700 1 -- 172.21.15.17:0/3159186581 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fee4806cea0 con 0x7fee50140480 2022-08-01T19:43:17.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.417+0000 7fee55ce8700 1 --2- 172.21.15.17:0/3159186581 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fee30053b30 0x7fee30055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:17.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.417+0000 7fee55ce8700 1 --2- 172.21.15.17:0/3159186581 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fee30053b30 0x7fee30055ff0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7fee4c00c000 tx=0x7fee4c008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:17.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.421+0000 7fee42ffd700 1 -- 172.21.15.17:0/3159186581 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fee48043c60 con 0x7fee50140480 2022-08-01T19:43:17.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.581+0000 7fee56cea700 1 -- 172.21.15.17:0/3159186581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee500574e0 con 0x7fee50140480 2022-08-01T19:43:17.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.581+0000 7fee42ffd700 1 -- 172.21.15.17:0/3159186581 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+2486 (secure 0 0 0) 0x7fee48015030 con 0x7fee50140480 2022-08-01T19:43:17.587 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:17.587 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":69,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7995,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.17:6833/4164640472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4164640472},{"type":"v1","addr":"172.21.15.17:6833","nonce":4164640472}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":67},{"gid":8001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/50015801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":50015801},{"type":"v1","addr":"172.21.15.17:6835","nonce":50015801}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:43:01.287477+0000","modified":"2022-08-01T19:43:02.290478+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8007},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8007":{"gid":8007,"name":"b","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6835/299216688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":299216688},{"type":"v1","addr":"172.21.15.77:6835","nonce":299216688}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2022-08-01T19:43:17.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.585+0000 7fee56cea700 1 -- 172.21.15.17:0/3159186581 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fee30053b30 msgr2=0x7fee30055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:17.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.585+0000 7fee56cea700 1 --2- 172.21.15.17:0/3159186581 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fee30053b30 0x7fee30055ff0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7fee4c00c000 tx=0x7fee4c008040).stop 2022-08-01T19:43:17.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.585+0000 7fee56cea700 1 -- 172.21.15.17:0/3159186581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee50140480 msgr2=0x7fee50155ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:17.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.585+0000 7fee56cea700 1 --2- 172.21.15.17:0/3159186581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee50140480 0x7fee50155ae0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 rx=0x7fee480032f0 tx=0x7fee48016000).stop 2022-08-01T19:43:17.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.585+0000 7fee56cea700 1 -- 172.21.15.17:0/3159186581 shutdown_connections 2022-08-01T19:43:17.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.585+0000 7fee56cea700 1 --2- 172.21.15.17:0/3159186581 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fee30053b30 0x7fee30055ff0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:17.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.585+0000 7fee56cea700 1 --2- 172.21.15.17:0/3159186581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee50140480 0x7fee50155ae0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:17.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.585+0000 7fee56cea700 1 --2- 172.21.15.17:0/3159186581 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fee500b66a0 0x7fee50148da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:17.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.585+0000 7fee56cea700 1 --2- 172.21.15.17:0/3159186581 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee500b4cd0 0x7fee500b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:17.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.585+0000 7fee56cea700 1 -- 172.21.15.17:0/3159186581 >> 172.21.15.17:0/3159186581 conn(0x7fee5005fa50 msgr2=0x7fee500ba210 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:17.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.585+0000 7fee56cea700 1 -- 172.21.15.17:0/3159186581 shutdown_connections 2022-08-01T19:43:17.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:17.585+0000 7fee56cea700 1 -- 172.21.15.17:0/3159186581 wait complete. 2022-08-01T19:43:17.599 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 69 2022-08-01T19:43:17.902 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 69, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:43:01.287477+0000', 'modified': '2022-08-01T19:43:02.290478+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8007}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8007': {'gid': 8007, 'name': 'b', 'rank': 0, 'incarnation': 68, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.77:6835/299216688', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6834', 'nonce': 299216688}, {'type': 'v1', 'addr': '172.21.15.77:6835', 'nonce': 299216688}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:43:17.903 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:43:17.903 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-08-01T19:43:18.019 INFO:teuthology.orchestra.run.smithi077.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":8007,"want_state":"up:active","state":"up:active","rank_uptime":16.720190988999999,"mdsmap_epoch":69,"osdmap_epoch":137,"osdmap_epoch_barrier":136,"uptime":21.624073240000001} 2022-08-01T19:43:18.020 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 8007, "mdsmap_epoch": 69, "osdmap_epoch": 137, "osdmap_epoch_barrier": 136, "rank_uptime": 16.720190989, "state": "up:active", "uptime": 21.62407324, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:43:18.021 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T19:43:18.021 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-08-01T19:43:18.022 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi190.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-08-01T19:43:18.022 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:18.023 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:43:18.032 DEBUG:teuthology.orchestra.run.smithi190:> sudo modprobe fuse 2022-08-01T19:43:18.091 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:43:18.092 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:43:18.108 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:43:18.109 INFO:teuthology.orchestra.run.smithi190.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:43:18.109 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:18.110 DEBUG:teuthology.orchestra.run.smithi190:> ls /sys/fs/fuse/connections 2022-08-01T19:43:18.117 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T19:43:18.118 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:43:18.163 DEBUG:teuthology.orchestra.run.smithi190:> sudo modprobe fuse 2022-08-01T19:43:18.180 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:43:18.181 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:43:18.194 INFO:teuthology.orchestra.run.smithi190.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:43:18.195 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:43:18.197 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:18.198 DEBUG:teuthology.orchestra.run.smithi190:> ls /sys/fs/fuse/connections 2022-08-01T19:43:18.240 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:2022-08-01T19:43:18.237+0000 7f286fec2080 -1 init, newargv = 0x556f9febcd20 newargc=9ceph-fuse[23178 2022-08-01T19:43:18.240 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:]: starting ceph client 2022-08-01T19:43:18.249 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:ceph-fuse[23178]: starting fuse 2022-08-01T19:43:18.484 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:43:18.490 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:43:18.498 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:43:18.500 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:43:19.348 DEBUG:teuthology.orchestra.run.smithi190:> sudo modprobe fuse 2022-08-01T19:43:19.367 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:43:19.368 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:43:19.381 INFO:teuthology.orchestra.run.smithi190.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:43:19.382 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:43:19.383 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:19.383 DEBUG:teuthology.orchestra.run.smithi190:> ls /sys/fs/fuse/connections 2022-08-01T19:43:19.391 INFO:teuthology.orchestra.run.smithi190.stdout:52 2022-08-01T19:43:19.392 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T19:43:19.392 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:19.393 DEBUG:teuthology.orchestra.run.smithi190:> sudo python3 -c ' 2022-08-01T19:43:19.393 DEBUG:teuthology.orchestra.run.smithi190:> import glob 2022-08-01T19:43:19.394 DEBUG:teuthology.orchestra.run.smithi190:> import re 2022-08-01T19:43:19.394 DEBUG:teuthology.orchestra.run.smithi190:> import os 2022-08-01T19:43:19.394 DEBUG:teuthology.orchestra.run.smithi190:> import subprocess 2022-08-01T19:43:19.395 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:43:19.395 DEBUG:teuthology.orchestra.run.smithi190:> def find_socket(client_name): 2022-08-01T19:43:19.396 DEBUG:teuthology.orchestra.run.smithi190:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-08-01T19:43:19.396 DEBUG:teuthology.orchestra.run.smithi190:> files = glob.glob(asok_path) 2022-08-01T19:43:19.396 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:43:19.397 DEBUG:teuthology.orchestra.run.smithi190:> # Given a non-glob path, it better be there 2022-08-01T19:43:19.397 DEBUG:teuthology.orchestra.run.smithi190:> if "*" not in asok_path: 2022-08-01T19:43:19.398 DEBUG:teuthology.orchestra.run.smithi190:> assert(len(files) == 1) 2022-08-01T19:43:19.398 DEBUG:teuthology.orchestra.run.smithi190:> return files[0] 2022-08-01T19:43:19.399 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:43:19.399 DEBUG:teuthology.orchestra.run.smithi190:> for f in files: 2022-08-01T19:43:19.399 DEBUG:teuthology.orchestra.run.smithi190:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T19:43:19.400 DEBUG:teuthology.orchestra.run.smithi190:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T19:43:19.400 DEBUG:teuthology.orchestra.run.smithi190:> return f 2022-08-01T19:43:19.401 DEBUG:teuthology.orchestra.run.smithi190:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T19:43:19.401 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:43:19.402 DEBUG:teuthology.orchestra.run.smithi190:> print(find_socket("client.1")) 2022-08-01T19:43:19.402 DEBUG:teuthology.orchestra.run.smithi190:> ' 2022-08-01T19:43:19.481 INFO:teuthology.orchestra.run.smithi190.stdout:/var/run/ceph/ceph-client.1.23178.asok 2022-08-01T19:43:19.486 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.23178.asok 2022-08-01T19:43:19.486 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:19.487 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.23178.asok status 2022-08-01T19:43:19.621 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2022-08-01T19:43:19.621 INFO:teuthology.orchestra.run.smithi190.stdout: "metadata": { 2022-08-01T19:43:19.622 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T19:43:19.623 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T19:43:19.624 INFO:teuthology.orchestra.run.smithi190.stdout: "entity_id": "1", 2022-08-01T19:43:19.624 INFO:teuthology.orchestra.run.smithi190.stdout: "hostname": "smithi190", 2022-08-01T19:43:19.625 INFO:teuthology.orchestra.run.smithi190.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-08-01T19:43:19.625 INFO:teuthology.orchestra.run.smithi190.stdout: "pid": "23178", 2022-08-01T19:43:19.626 INFO:teuthology.orchestra.run.smithi190.stdout: "root": "/" 2022-08-01T19:43:19.627 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:43:19.628 INFO:teuthology.orchestra.run.smithi190.stdout: "dentry_count": 0, 2022-08-01T19:43:19.628 INFO:teuthology.orchestra.run.smithi190.stdout: "dentry_pinned_count": 0, 2022-08-01T19:43:19.629 INFO:teuthology.orchestra.run.smithi190.stdout: "id": 8247, 2022-08-01T19:43:19.629 INFO:teuthology.orchestra.run.smithi190.stdout: "inst": { 2022-08-01T19:43:19.630 INFO:teuthology.orchestra.run.smithi190.stdout: "name": { 2022-08-01T19:43:19.630 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "client", 2022-08-01T19:43:19.631 INFO:teuthology.orchestra.run.smithi190.stdout: "num": 8247 2022-08-01T19:43:19.631 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:43:19.632 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": { 2022-08-01T19:43:19.632 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "v1", 2022-08-01T19:43:19.633 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": "172.21.15.190:0", 2022-08-01T19:43:19.634 INFO:teuthology.orchestra.run.smithi190.stdout: "nonce": 3657520532 2022-08-01T19:43:19.634 INFO:teuthology.orchestra.run.smithi190.stdout: } 2022-08-01T19:43:19.635 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:43:19.635 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": { 2022-08-01T19:43:19.638 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "v1", 2022-08-01T19:43:19.638 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": "172.21.15.190:0", 2022-08-01T19:43:19.639 INFO:teuthology.orchestra.run.smithi190.stdout: "nonce": 3657520532 2022-08-01T19:43:19.639 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:43:19.640 INFO:teuthology.orchestra.run.smithi190.stdout: "inst_str": "client.8247 172.21.15.190:0/3657520532", 2022-08-01T19:43:19.640 INFO:teuthology.orchestra.run.smithi190.stdout: "addr_str": "172.21.15.190:0/3657520532", 2022-08-01T19:43:19.640 INFO:teuthology.orchestra.run.smithi190.stdout: "inode_count": 1, 2022-08-01T19:43:19.641 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_epoch": 69, 2022-08-01T19:43:19.641 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_epoch": 137, 2022-08-01T19:43:19.642 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_epoch_barrier": 0, 2022-08-01T19:43:19.642 INFO:teuthology.orchestra.run.smithi190.stdout: "blacklisted": false, 2022-08-01T19:43:19.643 INFO:teuthology.orchestra.run.smithi190.stdout: "fs_name": "cephfs" 2022-08-01T19:43:19.643 INFO:teuthology.orchestra.run.smithi190.stdout:} 2022-08-01T19:43:19.645 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:19.648 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:43:19.649 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:43:19.660 INFO:teuthology.orchestra.run.smithi190.stdout:fuseblk 2022-08-01T19:43:19.663 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T19:43:19.664 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:19.665 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:43:19.723 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:19.724 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:43:19.724 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:43:19.735 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:43:19.737 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:43:19.739 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-08-01T19:43:19.739 DEBUG:teuthology.orchestra.run.smithi085:> import time 2022-08-01T19:43:19.740 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:43:19.741 DEBUG:teuthology.orchestra.run.smithi085:> with open("/home/ubuntu/cephtest/mnt.0/testfile", '"'"'w'"'"') as f: 2022-08-01T19:43:19.742 DEBUG:teuthology.orchestra.run.smithi085:> f.write('"'"'content'"'"') 2022-08-01T19:43:19.742 DEBUG:teuthology.orchestra.run.smithi085:> f.flush() 2022-08-01T19:43:19.743 DEBUG:teuthology.orchestra.run.smithi085:> f.write('"'"'content2'"'"') 2022-08-01T19:43:19.744 DEBUG:teuthology.orchestra.run.smithi085:> while True: 2022-08-01T19:43:19.745 DEBUG:teuthology.orchestra.run.smithi085:> time.sleep(1) 2022-08-01T19:43:19.746 DEBUG:teuthology.orchestra.run.smithi085:> ' 2022-08-01T19:43:19.781 DEBUG:teuthology.orchestra.run.smithi085:> sudo ls /home/ubuntu/cephtest/mnt.0/testfile 2022-08-01T19:43:19.804 INFO:teuthology.orchestra.run.smithi085.stderr:ls: cannot access '/home/ubuntu/cephtest/mnt.0/testfile': No such file or directory 2022-08-01T19:43:19.805 DEBUG:teuthology.orchestra.run:got remote process result: 2 2022-08-01T19:43:20.808 DEBUG:teuthology.orchestra.run.smithi085:> sudo ls /home/ubuntu/cephtest/mnt.0/testfile 2022-08-01T19:43:20.829 INFO:teuthology.orchestra.run.smithi085.stdout:/home/ubuntu/cephtest/mnt.0/testfile 2022-08-01T19:43:20.830 DEBUG:tasks.cephfs.mount:File testfile became visible from 0 after 1s 2022-08-01T19:43:20.830 DEBUG:teuthology.orchestra.run.smithi190:> sudo ls /home/ubuntu/cephtest/mnt.1/testfile 2022-08-01T19:43:20.877 INFO:teuthology.orchestra.run.smithi190.stdout:/home/ubuntu/cephtest/mnt.1/testfile 2022-08-01T19:43:20.878 DEBUG:tasks.cephfs.mount:File testfile became visible from 1 after 0s 2022-08-01T19:43:20.879 INFO:teuthology.orchestra.run:Running command with timeout 10800 2022-08-01T19:43:20.880 DEBUG:teuthology.orchestra.run.smithi190:> cd /home/ubuntu/cephtest/mnt.1 && sudo cat testfile 2022-08-01T19:43:20.923 INFO:teuthology.orchestra.run.smithi190.stdout:content 2022-08-01T19:43:20.939 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T19:43:20.940 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:43:20.964 DEBUG:teuthology.orchestra.run.smithi077:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:43:20.998 DEBUG:teuthology.orchestra.run.smithi085:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:43:21.023 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:43:21.048 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T19:43:21.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:21.049 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:43:21.049 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:43:21.060 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:43:21.060 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:43:21.061 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi085.front.sepia.ceph.com... 2022-08-01T19:43:21.062 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T19:43:21.063 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:43:21.117 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:ceph-fuse[23551]: fuse finished with error 0 and tester_r 0 2022-08-01T19:43:21.119 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:21.120 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:43:21.120 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:43:21.129 INFO:teuthology.orchestra.run.smithi085.stdout:ext2/ext3 2022-08-01T19:43:21.130 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T19:43:21.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T19:43:21.180 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:43:21.189 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:21.190 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:43:21.190 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:43:21.198 INFO:teuthology.orchestra.run.smithi190.stdout:fuseblk 2022-08-01T19:43:21.200 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T19:43:21.200 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi190.front.sepia.ceph.com... 2022-08-01T19:43:21.201 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T19:43:21.201 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:43:21.256 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:ceph-fuse[23178]: fuse finished with error 0 and tester_r 0 2022-08-01T19:43:21.258 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:21.259 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:43:21.259 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:43:21.266 INFO:teuthology.orchestra.run.smithi190.stdout:ext2/ext3 2022-08-01T19:43:21.267 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T19:43:21.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T19:43:21.387 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:43:21.397 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T19:43:21.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.505+0000 7faf654e9700 1 Processor -- start 2022-08-01T19:43:21.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.505+0000 7faf654e9700 1 -- start start 2022-08-01T19:43:21.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.509+0000 7faf654e9700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7faf600b0800 0x7faf600b0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:21.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.509+0000 7faf654e9700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faf601444d0 0x7faf60144920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:21.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.509+0000 7faf654e9700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf600aee60 con 0x7faf600ac930 2022-08-01T19:43:21.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.509+0000 7faf654e9700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf600b0be0 con 0x7faf601444d0 2022-08-01T19:43:21.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.509+0000 7faf654e9700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf600ae480 con 0x7faf600b0800 2022-08-01T19:43:21.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.509+0000 7faf5f7fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7faf600ac930 0x7faf600b12e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36754/0 (socket says 172.21.15.17:36754) 2022-08-01T19:43:21.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.509+0000 7faf64ce8700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faf601444d0 0x7faf60144920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:21.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.509+0000 7faf5f7fe700 1 -- 172.21.15.17:0/3791290000 learned_addr learned my addr 172.21.15.17:0/3791290000 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:21.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.509+0000 7faf5ffff700 1 --2- 172.21.15.17:0/3791290000 >> v2:172.21.15.77:3300/0 conn(0x7faf600b0800 0x7faf600b0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:21.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.509+0000 7faf64ce8700 1 -- 172.21.15.17:0/3791290000 >> v2:172.21.15.77:3300/0 conn(0x7faf600b0800 msgr2=0x7faf600b0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:21.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.509+0000 7faf64ce8700 1 --2- 172.21.15.17:0/3791290000 >> v2:172.21.15.77:3300/0 conn(0x7faf600b0800 0x7faf600b0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:21.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.509+0000 7faf64ce8700 1 -- 172.21.15.17:0/3791290000 >> v1:172.21.15.17:6789/0 conn(0x7faf600ac930 legacy=0x7faf600b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:21.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.509+0000 7faf64ce8700 1 -- 172.21.15.17:0/3791290000 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf600ae720 con 0x7faf601444d0 2022-08-01T19:43:21.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.509+0000 7faf64ce8700 1 --2- 172.21.15.17:0/3791290000 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf601444d0 0x7faf60144920 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 rx=0x7faf580032f0 tx=0x7faf580091d0).ready entity=mon.0 client_cookie=11bed49dc7f6a387 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:21.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.509+0000 7faf5effd700 1 -- 172.21.15.17:0/3791290000 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf58009ea0 con 0x7faf601444d0 2022-08-01T19:43:21.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.509+0000 7faf5effd700 1 -- 172.21.15.17:0/3791290000 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5800b870 con 0x7faf601444d0 2022-08-01T19:43:21.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.509+0000 7faf5effd700 1 -- 172.21.15.17:0/3791290000 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf5800b9e0 con 0x7faf601444d0 2022-08-01T19:43:21.526 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.509+0000 7faf654e9700 1 -- 172.21.15.17:0/3791290000 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf601444d0 msgr2=0x7faf60144920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:21.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.509+0000 7faf654e9700 1 --2- 172.21.15.17:0/3791290000 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf601444d0 0x7faf60144920 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 rx=0x7faf580032f0 tx=0x7faf580091d0).stop 2022-08-01T19:43:21.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.509+0000 7faf654e9700 1 -- 172.21.15.17:0/3791290000 shutdown_connections 2022-08-01T19:43:21.527 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.509+0000 7faf654e9700 1 --2- 172.21.15.17:0/3791290000 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf601444d0 0x7faf60144920 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:21.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.509+0000 7faf654e9700 1 --2- 172.21.15.17:0/3791290000 >> v2:172.21.15.77:3300/0 conn(0x7faf600b0800 0x7faf600b0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:21.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.509+0000 7faf654e9700 1 -- 172.21.15.17:0/3791290000 >> 172.21.15.17:0/3791290000 conn(0x7faf6005bed0 msgr2=0x7faf600626f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:21.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.509+0000 7faf654e9700 1 -- 172.21.15.17:0/3791290000 shutdown_connections 2022-08-01T19:43:21.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.509+0000 7faf654e9700 1 -- 172.21.15.17:0/3791290000 wait complete. 2022-08-01T19:43:21.529 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.513+0000 7faf654e9700 1 Processor -- start 2022-08-01T19:43:21.529 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.513+0000 7faf654e9700 1 -- start start 2022-08-01T19:43:21.529 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.513+0000 7faf654e9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf600ac930 0x7faf601480f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:21.529 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.513+0000 7faf654e9700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7faf600b0800 0x7faf6015a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:21.529 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.513+0000 7faf654e9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf601444d0 0x7faf6015a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:21.530 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.513+0000 7faf654e9700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6014ad10 con 0x7faf600ac930 2022-08-01T19:43:21.530 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.513+0000 7faf654e9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6014afb0 con 0x7faf601444d0 2022-08-01T19:43:21.531 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.513+0000 7faf654e9700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf6014b250 con 0x7faf600b0800 2022-08-01T19:43:21.531 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.513+0000 7faf5ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf600ac930 0x7faf601480f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:21.532 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.513+0000 7faf5ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf600ac930 0x7faf601480f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44566/0 (socket says 172.21.15.17:44566) 2022-08-01T19:43:21.532 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.513+0000 7faf5ffff700 1 -- 172.21.15.17:0/3641209428 learned_addr learned my addr 172.21.15.17:0/3641209428 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:21.532 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.513+0000 7faf64ce8700 1 --2- 172.21.15.17:0/3641209428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf601444d0 0x7faf6015a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:21.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.513+0000 7faf5f7fe700 1 --2- 172.21.15.17:0/3641209428 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7faf600b0800 0x7faf6015a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:21.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.513+0000 7faf5ffff700 1 -- 172.21.15.17:0/3641209428 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7faf600b0800 msgr2=0x7faf6015a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:21.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.513+0000 7faf5ffff700 1 --2- 172.21.15.17:0/3641209428 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7faf600b0800 0x7faf6015a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:21.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.513+0000 7faf5ffff700 1 -- 172.21.15.17:0/3641209428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf601444d0 msgr2=0x7faf6015a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:21.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.513+0000 7faf5ffff700 1 --2- 172.21.15.17:0/3641209428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf601444d0 0x7faf6015a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:21.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.513+0000 7faf5ffff700 1 -- 172.21.15.17:0/3641209428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf58009470 con 0x7faf600ac930 2022-08-01T19:43:21.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.513+0000 7faf5ffff700 1 --2- 172.21.15.17:0/3641209428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf600ac930 0x7faf601480f0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 rx=0x7faf540030b0 tx=0x7faf54008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:21.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.513+0000 7faf5d7fa700 1 -- 172.21.15.17:0/3641209428 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf54011780 con 0x7faf600ac930 2022-08-01T19:43:21.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.513+0000 7faf5d7fa700 1 -- 172.21.15.17:0/3641209428 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf540118e0 con 0x7faf600ac930 2022-08-01T19:43:21.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.513+0000 7faf654e9700 1 -- 172.21.15.17:0/3641209428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf60154780 con 0x7faf600ac930 2022-08-01T19:43:21.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.513+0000 7faf5d7fa700 1 -- 172.21.15.17:0/3641209428 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf54011c40 con 0x7faf600ac930 2022-08-01T19:43:21.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.513+0000 7faf654e9700 1 -- 172.21.15.17:0/3641209428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf60063d20 con 0x7faf600ac930 2022-08-01T19:43:21.538 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.517+0000 7faf654e9700 1 -- 172.21.15.17:0/3641209428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf6014ab50 con 0x7faf600ac930 2022-08-01T19:43:21.538 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.517+0000 7faf5d7fa700 1 -- 172.21.15.17:0/3641209428 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7faf5401c460 con 0x7faf600ac930 2022-08-01T19:43:21.538 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.517+0000 7faf5d7fa700 1 --2- 172.21.15.17:0/3641209428 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7faf400539e0 0x7faf40055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:21.538 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.517+0000 7faf5d7fa700 1 -- 172.21.15.17:0/3641209428 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7faf5406c1c0 con 0x7faf600ac930 2022-08-01T19:43:21.539 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.517+0000 7faf5f7fe700 1 --2- 172.21.15.17:0/3641209428 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7faf400539e0 0x7faf40055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:21.539 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.517+0000 7faf5f7fe700 1 --2- 172.21.15.17:0/3641209428 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7faf400539e0 0x7faf40055ea0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7faf5000d000 tx=0x7faf50009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:21.540 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.525+0000 7faf5d7fa700 1 -- 172.21.15.17:0/3641209428 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7faf54042f20 con 0x7faf600ac930 2022-08-01T19:43:21.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.653+0000 7faf654e9700 1 -- 172.21.15.17:0/3641209428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7faf600ae9c0 con 0x7faf600ac930 2022-08-01T19:43:21.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.653+0000 7faf5d7fa700 1 -- 172.21.15.17:0/3641209428 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v137) v1 ==== 81+0+34051 (secure 0 0 0) 0x7faf54046540 con 0x7faf600ac930 2022-08-01T19:43:21.658 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:21.658 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:43:21.658 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 137, 2022-08-01T19:43:21.658 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", 2022-08-01T19:43:21.659 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2022-08-01T19:33:50.105743+0000", 2022-08-01T19:43:21.659 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2022-08-01T19:43:02.291265+0000", 2022-08-01T19:43:21.659 INFO:teuthology.orchestra.run.smithi017.stdout: "last_up_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:43:21.659 INFO:teuthology.orchestra.run.smithi017.stdout: "last_in_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:43:21.660 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T19:43:21.660 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_num": 5799936, 2022-08-01T19:43:21.660 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_set": [ 2022-08-01T19:43:21.661 INFO:teuthology.orchestra.run.smithi017.stdout: "pglog_hardlimit", 2022-08-01T19:43:21.661 INFO:teuthology.orchestra.run.smithi017.stdout: "purged_snapdirs", 2022-08-01T19:43:21.661 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_deletes", 2022-08-01T19:43:21.662 INFO:teuthology.orchestra.run.smithi017.stdout: "sortbitwise" 2022-08-01T19:43:21.662 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:43:21.662 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_version": 6, 2022-08-01T19:43:21.662 INFO:teuthology.orchestra.run.smithi017.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T19:43:21.663 INFO:teuthology.orchestra.run.smithi017.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T19:43:21.664 INFO:teuthology.orchestra.run.smithi017.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T19:43:21.664 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_snapshot": "", 2022-08-01T19:43:21.666 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_max": 26, 2022-08-01T19:43:21.666 INFO:teuthology.orchestra.run.smithi017.stdout: "max_osd": 8, 2022-08-01T19:43:21.666 INFO:teuthology.orchestra.run.smithi017.stdout: "require_min_compat_client": "jewel", 2022-08-01T19:43:21.666 INFO:teuthology.orchestra.run.smithi017.stdout: "min_compat_client": "jewel", 2022-08-01T19:43:21.667 INFO:teuthology.orchestra.run.smithi017.stdout: "require_osd_release": "octopus", 2022-08-01T19:43:21.667 INFO:teuthology.orchestra.run.smithi017.stdout: "pools": [ 2022-08-01T19:43:21.667 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.668 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 1, 2022-08-01T19:43:21.668 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "device_health_metrics", 2022-08-01T19:43:21.668 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:33:53.001841+0000", 2022-08-01T19:43:21.669 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:43:21.669 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:43:21.670 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:43:21.670 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:43:21.670 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:43:21.670 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:43:21.671 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:43:21.672 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:43:21.672 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 1, 2022-08-01T19:43:21.672 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 1, 2022-08-01T19:43:21.672 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 1, 2022-08-01T19:43:21.673 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 1, 2022-08-01T19:43:21.673 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 1, 2022-08-01T19:43:21.673 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:43:21.673 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:43:21.675 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:43:21.675 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:43:21.676 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:43:21.676 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:43:21.677 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:43:21.678 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.678 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "16", 2022-08-01T19:43:21.678 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:43:21.679 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:43:21.679 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:43:21.679 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:43:21.679 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:43:21.680 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:43:21.680 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:43:21.680 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:43:21.680 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:43:21.681 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:43:21.681 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:43:21.681 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:43:21.681 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:43:21.682 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:43:21.682 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:43:21.683 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:43:21.683 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:43:21.683 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:43:21.683 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:43:21.683 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:43:21.684 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:43:21.684 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:43:21.684 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:43:21.684 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:43:21.687 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:43:21.688 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:43:21.688 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.689 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:43:21.689 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:43:21.690 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:43:21.690 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:43:21.691 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:43:21.691 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:43:21.691 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:43:21.692 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:43:21.692 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:43:21.693 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:43:21.693 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:43:21.694 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:43:21.694 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 1 2022-08-01T19:43:21.695 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.695 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:43:21.696 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_devicehealth": {} 2022-08-01T19:43:21.696 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.697 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.697 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.698 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 2, 2022-08-01T19:43:21.698 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "rbd", 2022-08-01T19:43:21.698 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:34:01.553098+0000", 2022-08-01T19:43:21.699 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:43:21.699 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:43:21.700 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:43:21.700 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:43:21.700 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:43:21.701 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:43:21.701 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:43:21.702 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:43:21.702 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:43:21.702 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:43:21.703 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:43:21.703 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:43:21.703 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:43:21.704 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:43:21.704 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:43:21.704 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:43:21.704 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:43:21.705 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:43:21.705 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:43:21.705 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:43:21.706 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.706 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "19", 2022-08-01T19:43:21.706 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:43:21.707 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:43:21.707 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:43:21.708 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:43:21.708 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:43:21.709 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:43:21.709 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:43:21.710 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:43:21.710 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:43:21.711 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:43:21.711 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:43:21.712 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:43:21.712 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:43:21.713 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:43:21.713 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:43:21.714 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:43:21.714 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:43:21.715 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:43:21.715 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:43:21.716 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:43:21.717 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:43:21.717 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:43:21.718 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:43:21.718 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:43:21.719 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:43:21.719 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:43:21.720 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.720 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:43:21.721 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:43:21.721 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:43:21.722 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:43:21.722 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:43:21.723 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:43:21.723 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:43:21.724 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:43:21.724 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:43:21.724 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:43:21.725 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:43:21.725 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:43:21.725 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:43:21.725 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd": {} 2022-08-01T19:43:21.726 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.726 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.727 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.727 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 25, 2022-08-01T19:43:21.728 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "cephfs_metadata", 2022-08-01T19:43:21.728 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:42:59.166414+0000", 2022-08-01T19:43:21.728 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:43:21.729 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:43:21.729 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:43:21.730 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:43:21.730 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:43:21.731 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:43:21.731 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:43:21.732 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:43:21.732 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:43:21.733 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:43:21.733 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:43:21.733 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:43:21.734 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:43:21.734 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:43:21.735 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:43:21.735 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:43:21.736 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:43:21.736 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:43:21.737 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:43:21.737 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:43:21.738 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.738 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "136", 2022-08-01T19:43:21.739 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:43:21.739 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:43:21.740 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:43:21.740 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:43:21.741 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:43:21.741 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:43:21.742 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:43:21.742 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:43:21.743 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:43:21.743 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:43:21.744 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:43:21.744 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:43:21.744 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:43:21.745 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:43:21.745 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:43:21.745 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:43:21.746 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:43:21.746 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:43:21.746 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:43:21.746 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:43:21.747 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:43:21.747 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:43:21.747 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:43:21.747 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:43:21.748 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:43:21.748 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:43:21.748 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.748 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:43:21.748 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:43:21.749 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:43:21.749 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:43:21.749 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:43:21.749 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:43:21.750 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:43:21.750 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:43:21.750 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:43:21.750 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:43:21.751 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:43:21.751 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:43:21.751 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_bias": 4, 2022-08-01T19:43:21.751 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 16, 2022-08-01T19:43:21.751 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_priority": 5 2022-08-01T19:43:21.752 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.752 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:43:21.752 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs": { 2022-08-01T19:43:21.752 INFO:teuthology.orchestra.run.smithi017.stdout: "metadata": "cephfs" 2022-08-01T19:43:21.753 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.753 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.753 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.753 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.754 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 26, 2022-08-01T19:43:21.754 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "cephfs_data", 2022-08-01T19:43:21.754 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:43:00.301184+0000", 2022-08-01T19:43:21.754 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:43:21.755 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:43:21.755 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:43:21.755 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:43:21.755 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:43:21.755 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:43:21.756 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:43:21.756 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:43:21.756 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:43:21.756 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:43:21.757 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:43:21.757 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:43:21.757 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:43:21.757 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:43:21.757 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:43:21.758 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:43:21.758 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:43:21.758 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:43:21.758 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:43:21.759 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:43:21.759 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.759 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "137", 2022-08-01T19:43:21.759 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:43:21.759 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:43:21.760 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:43:21.760 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:43:21.760 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:43:21.760 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:43:21.761 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:43:21.761 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:43:21.761 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:43:21.761 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:43:21.761 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:43:21.762 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:43:21.762 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:43:21.762 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:43:21.762 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:43:21.763 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:43:21.763 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:43:21.763 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:43:21.763 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:43:21.764 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:43:21.764 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:43:21.764 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:43:21.764 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:43:21.764 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:43:21.765 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:43:21.765 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:43:21.765 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.766 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:43:21.766 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:43:21.766 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:43:21.767 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:43:21.767 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:43:21.767 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:43:21.767 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:43:21.768 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:43:21.768 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:43:21.768 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:43:21.768 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:43:21.769 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:43:21.769 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:43:21.769 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs": { 2022-08-01T19:43:21.769 INFO:teuthology.orchestra.run.smithi017.stdout: "data": "cephfs" 2022-08-01T19:43:21.769 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.770 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.770 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.770 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:43:21.770 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2022-08-01T19:43:21.771 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.771 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:43:21.771 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a962f14f-cf16-4fa8-84ff-c85325db9132", 2022-08-01T19:43:21.771 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:43:21.772 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:43:21.772 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:43:21.772 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:43:21.772 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:43:21.772 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:43:21.773 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:43:21.773 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 134, 2022-08-01T19:43:21.773 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:43:21.773 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:43:21.774 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:43:21.774 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:21.774 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.774 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:21.774 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6800", 2022-08-01T19:43:21.775 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:43:21.775 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.775 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.775 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:21.776 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6801", 2022-08-01T19:43:21.776 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:43:21.776 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.776 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.776 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.777 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:43:21.777 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:21.777 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.777 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:21.778 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6802", 2022-08-01T19:43:21.778 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:43:21.778 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.778 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.778 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:21.779 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6803", 2022-08-01T19:43:21.779 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:43:21.779 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.779 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.780 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.780 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:43:21.780 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:21.781 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.781 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:21.781 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6806", 2022-08-01T19:43:21.781 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:43:21.781 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.782 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.782 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:21.782 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6807", 2022-08-01T19:43:21.782 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:43:21.783 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.783 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.783 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.783 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:43:21.784 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:21.784 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.784 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:21.784 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6804", 2022-08-01T19:43:21.785 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:43:21.785 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.785 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.786 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:21.786 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6805", 2022-08-01T19:43:21.786 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:43:21.787 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.787 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.787 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.787 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6801/23008", 2022-08-01T19:43:21.787 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6803/23008", 2022-08-01T19:43:21.788 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6807/23008", 2022-08-01T19:43:21.788 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6805/23008", 2022-08-01T19:43:21.788 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:43:21.788 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:43:21.789 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:43:21.789 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.789 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.789 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.789 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:43:21.790 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "805f7051-a527-4e98-8844-b6e406a63248", 2022-08-01T19:43:21.790 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:43:21.790 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:43:21.790 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:43:21.791 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:43:21.791 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:43:21.791 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:43:21.791 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:43:21.792 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 135, 2022-08-01T19:43:21.792 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:43:21.792 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:43:21.792 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:43:21.792 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:21.793 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.793 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:21.793 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6808", 2022-08-01T19:43:21.793 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:43:21.794 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.794 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.794 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:21.794 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6809", 2022-08-01T19:43:21.794 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:43:21.795 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.795 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.795 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.795 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:43:21.796 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:21.796 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.796 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:21.796 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6810", 2022-08-01T19:43:21.797 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:43:21.797 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.797 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.797 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:21.797 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6811", 2022-08-01T19:43:21.798 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:43:21.798 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.798 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.798 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.799 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:43:21.799 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:21.799 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.799 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:21.800 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6814", 2022-08-01T19:43:21.800 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:43:21.800 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.800 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.800 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:21.801 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6815", 2022-08-01T19:43:21.801 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:43:21.801 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.801 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.802 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.802 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:43:21.802 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:21.802 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.803 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:21.803 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6812", 2022-08-01T19:43:21.803 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:43:21.803 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.803 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.804 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:21.804 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6813", 2022-08-01T19:43:21.804 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:43:21.804 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.805 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.805 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.805 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6809/23007", 2022-08-01T19:43:21.805 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6811/23007", 2022-08-01T19:43:21.806 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6815/23007", 2022-08-01T19:43:21.806 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6813/23007", 2022-08-01T19:43:21.806 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:43:21.806 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:43:21.806 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:43:21.807 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.807 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.807 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.807 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:43:21.808 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "89f7f3df-7c10-48e1-8b69-9e4243923a61", 2022-08-01T19:43:21.808 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:43:21.808 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:43:21.808 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:43:21.808 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:43:21.809 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:43:21.809 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:43:21.809 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:43:21.809 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 135, 2022-08-01T19:43:21.810 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:43:21.810 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:43:21.810 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:43:21.810 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:21.810 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.811 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:21.811 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6817", 2022-08-01T19:43:21.811 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:43:21.811 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.812 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.812 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:21.812 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6819", 2022-08-01T19:43:21.812 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:43:21.812 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.813 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.813 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.813 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:43:21.813 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:21.814 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.814 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:21.814 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6822", 2022-08-01T19:43:21.814 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:43:21.815 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.815 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.815 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:21.815 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6824", 2022-08-01T19:43:21.816 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:43:21.816 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.816 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.816 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.816 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:43:21.817 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:21.817 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.817 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:21.817 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6830", 2022-08-01T19:43:21.818 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:43:21.818 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.818 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.818 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:21.819 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6831", 2022-08-01T19:43:21.819 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:43:21.819 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.819 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.819 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.820 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:43:21.820 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:21.820 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.820 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:21.821 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6826", 2022-08-01T19:43:21.821 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:43:21.821 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.821 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.821 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:21.822 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6829", 2022-08-01T19:43:21.822 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:43:21.822 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.822 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.823 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.823 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6819/23010", 2022-08-01T19:43:21.823 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6824/23010", 2022-08-01T19:43:21.823 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6831/23010", 2022-08-01T19:43:21.823 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6829/23010", 2022-08-01T19:43:21.824 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:43:21.824 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:43:21.824 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:43:21.824 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.825 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.825 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.825 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:43:21.825 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "96123468-aabc-4a64-909e-a192c369090d", 2022-08-01T19:43:21.826 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:43:21.826 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:43:21.826 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:43:21.826 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:43:21.826 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:43:21.827 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:43:21.827 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:43:21.827 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 134, 2022-08-01T19:43:21.827 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:43:21.828 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:43:21.828 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:43:21.828 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:21.828 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.829 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:21.829 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6816", 2022-08-01T19:43:21.829 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:43:21.829 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.829 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.830 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:21.830 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6818", 2022-08-01T19:43:21.830 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:43:21.830 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.831 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.831 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.831 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:43:21.831 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:21.831 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.832 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:21.832 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6820", 2022-08-01T19:43:21.832 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:43:21.832 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.833 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.833 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:21.833 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6821", 2022-08-01T19:43:21.833 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:43:21.833 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.834 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.834 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.834 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:43:21.834 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:21.835 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.835 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:21.835 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6827", 2022-08-01T19:43:21.835 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:43:21.836 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.836 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.836 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:21.836 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6828", 2022-08-01T19:43:21.836 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:43:21.837 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.837 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.837 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.837 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:43:21.838 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:21.838 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.838 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:21.838 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6823", 2022-08-01T19:43:21.838 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:43:21.839 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.839 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.839 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:21.839 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6825", 2022-08-01T19:43:21.840 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:43:21.840 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.840 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.840 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.840 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6818/23009", 2022-08-01T19:43:21.841 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6821/23009", 2022-08-01T19:43:21.841 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6828/23009", 2022-08-01T19:43:21.841 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6825/23009", 2022-08-01T19:43:21.841 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:43:21.842 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:43:21.842 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:43:21.842 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.842 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.842 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.843 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:43:21.843 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a6e26665-e04b-46ec-ab66-e912acbf044c", 2022-08-01T19:43:21.843 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:43:21.843 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:43:21.844 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:43:21.844 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:43:21.844 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:43:21.844 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:43:21.845 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:43:21.845 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 135, 2022-08-01T19:43:21.845 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:43:21.845 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:43:21.845 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:43:21.846 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:21.846 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.846 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:21.846 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6810", 2022-08-01T19:43:21.847 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:43:21.847 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.847 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.847 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:21.847 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6811", 2022-08-01T19:43:21.848 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:43:21.848 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.848 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.848 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.849 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:43:21.849 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:21.849 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.849 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:21.849 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6812", 2022-08-01T19:43:21.850 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:43:21.850 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.850 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.850 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:21.851 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6813", 2022-08-01T19:43:21.851 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:43:21.851 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.851 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.852 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.852 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:43:21.852 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:21.852 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.852 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:21.853 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6816", 2022-08-01T19:43:21.853 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:43:21.853 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.853 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.854 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:21.854 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6817", 2022-08-01T19:43:21.854 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:43:21.854 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.854 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.855 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.855 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:43:21.855 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:21.855 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.856 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:21.856 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6814", 2022-08-01T19:43:21.856 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:43:21.856 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.856 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.857 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:21.857 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6815", 2022-08-01T19:43:21.857 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:43:21.857 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.858 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.858 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.858 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6811/23584", 2022-08-01T19:43:21.858 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6813/23584", 2022-08-01T19:43:21.859 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6817/23584", 2022-08-01T19:43:21.859 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6815/23584", 2022-08-01T19:43:21.859 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:43:21.859 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:43:21.859 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:43:21.860 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.860 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.860 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.860 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:43:21.861 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "502c0034-18ee-44ac-87ff-19b9dddb66a0", 2022-08-01T19:43:21.861 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:43:21.861 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:43:21.861 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:43:21.861 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:43:21.862 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:43:21.862 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:43:21.862 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:43:21.862 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 127, 2022-08-01T19:43:21.863 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:43:21.863 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:43:21.863 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:43:21.863 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:21.864 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.864 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:21.864 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6802", 2022-08-01T19:43:21.864 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:43:21.864 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.865 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.865 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:21.865 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6803", 2022-08-01T19:43:21.866 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:43:21.866 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.866 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.866 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.867 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:43:21.867 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:21.867 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.867 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:21.867 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6804", 2022-08-01T19:43:21.868 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:43:21.868 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.868 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.868 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:21.869 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6805", 2022-08-01T19:43:21.869 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:43:21.869 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.869 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.869 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.870 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:43:21.870 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:21.870 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.870 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:21.871 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6808", 2022-08-01T19:43:21.871 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:43:21.871 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.871 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.872 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:21.872 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6809", 2022-08-01T19:43:21.872 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:43:21.872 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.873 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.873 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.873 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:43:21.873 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:21.874 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.874 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:21.874 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6806", 2022-08-01T19:43:21.874 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:43:21.875 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.875 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.875 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:21.875 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6807", 2022-08-01T19:43:21.876 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:43:21.876 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.876 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.876 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.877 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6803/23585", 2022-08-01T19:43:21.877 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6805/23585", 2022-08-01T19:43:21.877 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6809/23585", 2022-08-01T19:43:21.877 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6807/23585", 2022-08-01T19:43:21.878 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:43:21.878 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:43:21.878 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:43:21.878 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.879 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.879 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.879 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:43:21.879 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "ac4e755a-7fe7-46cb-8407-852d5af9ffc5", 2022-08-01T19:43:21.880 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:43:21.880 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:43:21.880 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:43:21.880 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:43:21.880 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:43:21.881 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:43:21.881 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:43:21.881 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 135, 2022-08-01T19:43:21.882 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:43:21.882 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:43:21.882 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:43:21.882 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:21.883 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.883 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:21.883 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6820", 2022-08-01T19:43:21.883 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:43:21.884 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.884 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.884 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:21.884 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6821", 2022-08-01T19:43:21.884 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:43:21.885 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.885 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.885 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.885 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:43:21.886 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:21.887 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.887 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:21.887 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6823", 2022-08-01T19:43:21.888 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:43:21.888 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.889 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.889 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:21.890 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6825", 2022-08-01T19:43:21.890 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:43:21.891 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.891 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.891 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.892 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:43:21.892 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:21.893 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.893 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:21.893 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6831", 2022-08-01T19:43:21.893 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:43:21.894 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.894 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.894 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:21.895 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6833", 2022-08-01T19:43:21.895 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:43:21.895 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.896 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.896 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.896 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:43:21.896 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:21.897 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.897 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:21.897 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6827", 2022-08-01T19:43:21.897 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:43:21.898 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.898 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.898 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:21.899 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6829", 2022-08-01T19:43:21.899 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:43:21.899 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.899 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.899 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.900 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6821/23587", 2022-08-01T19:43:21.900 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6825/23587", 2022-08-01T19:43:21.900 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6833/23587", 2022-08-01T19:43:21.900 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6829/23587", 2022-08-01T19:43:21.901 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:43:21.901 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:43:21.902 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:43:21.902 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.902 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.903 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.903 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:43:21.903 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "874761e2-6410-40fc-b60c-e58abe5ce489", 2022-08-01T19:43:21.903 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:43:21.904 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:43:21.904 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:43:21.904 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:43:21.904 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:43:21.905 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:43:21.905 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:43:21.905 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 135, 2022-08-01T19:43:21.905 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:43:21.905 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:43:21.906 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:43:21.906 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:21.906 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.906 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:21.907 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6818", 2022-08-01T19:43:21.907 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:43:21.907 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.908 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.908 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:21.908 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6819", 2022-08-01T19:43:21.908 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:43:21.909 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.909 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.909 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.910 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:43:21.910 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:21.910 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.910 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:21.911 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6822", 2022-08-01T19:43:21.911 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:43:21.911 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.912 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.912 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:21.912 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6824", 2022-08-01T19:43:21.912 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:43:21.913 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.913 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.913 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.914 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:43:21.914 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:21.914 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.914 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:21.915 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6830", 2022-08-01T19:43:21.915 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:43:21.915 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.915 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.916 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:21.916 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6832", 2022-08-01T19:43:21.916 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:43:21.916 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.916 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.917 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.917 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:43:21.917 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:21.917 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.918 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:21.918 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6826", 2022-08-01T19:43:21.918 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:43:21.918 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.919 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.919 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:21.919 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6828", 2022-08-01T19:43:21.919 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:43:21.919 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.920 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.920 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.920 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6819/23586", 2022-08-01T19:43:21.921 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6824/23586", 2022-08-01T19:43:21.921 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6832/23586", 2022-08-01T19:43:21.921 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6828/23586", 2022-08-01T19:43:21.922 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:43:21.922 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:43:21.923 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:43:21.923 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:21.923 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.923 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:43:21.924 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_xinfo": [ 2022-08-01T19:43:21.924 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.924 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:43:21.924 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:43:21.924 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:43:21.925 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:43:21.925 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:43:21.926 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:43:21.926 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.588817+0000", 2022-08-01T19:43:21.926 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:43:21.927 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.927 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.927 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:43:21.927 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:43:21.928 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:43:21.928 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:43:21.928 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:43:21.929 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:43:21.929 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.782753+0000", 2022-08-01T19:43:21.929 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:43:21.930 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.930 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.930 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:43:21.930 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:43:21.931 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:43:21.931 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:43:21.931 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:43:21.931 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:43:21.931 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.837924+0000", 2022-08-01T19:43:21.932 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:43:21.932 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.932 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.933 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:43:21.933 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:43:21.933 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:43:21.934 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:43:21.934 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:43:21.935 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:43:21.935 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.839948+0000", 2022-08-01T19:43:21.935 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:43:21.935 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.936 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.936 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:43:21.936 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:43:21.936 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:43:21.936 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:43:21.937 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:43:21.937 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:43:21.937 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.112801+0000", 2022-08-01T19:43:21.937 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:43:21.938 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.938 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.938 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:43:21.938 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:43:21.939 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:43:21.939 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:43:21.939 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:43:21.939 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:43:21.940 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.106704+0000", 2022-08-01T19:43:21.940 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:43:21.940 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.940 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.940 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:43:21.941 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:43:21.941 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:43:21.942 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:43:21.942 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:43:21.942 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:43:21.942 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.893170+0000", 2022-08-01T19:43:21.943 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:43:21.943 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.943 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:21.944 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:43:21.944 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:43:21.944 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:43:21.944 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:43:21.945 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:43:21.945 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:43:21.945 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.886947+0000", 2022-08-01T19:43:21.945 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:43:21.946 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.946 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:43:21.946 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap": [], 2022-08-01T19:43:21.946 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap_items": [], 2022-08-01T19:43:21.947 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_temp": [], 2022-08-01T19:43:21.947 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_temp": [], 2022-08-01T19:43:21.947 INFO:teuthology.orchestra.run.smithi017.stdout: "blacklist": {}, 2022-08-01T19:43:21.947 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profiles": { 2022-08-01T19:43:21.947 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_data": { 2022-08-01T19:43:21.948 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-device-class": "", 2022-08-01T19:43:21.948 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:43:21.948 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-root": "default", 2022-08-01T19:43:21.948 INFO:teuthology.orchestra.run.smithi017.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T19:43:21.948 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:43:21.949 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "2", 2022-08-01T19:43:21.949 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:43:21.949 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:43:21.949 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van", 2022-08-01T19:43:21.950 INFO:teuthology.orchestra.run.smithi017.stdout: "w": "8" 2022-08-01T19:43:21.950 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.950 INFO:teuthology.orchestra.run.smithi017.stdout: "default": { 2022-08-01T19:43:21.950 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:43:21.951 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:43:21.951 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "1", 2022-08-01T19:43:21.951 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:43:21.951 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:43:21.951 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van" 2022-08-01T19:43:21.952 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:21.952 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:21.952 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps_queue": [], 2022-08-01T19:43:21.953 INFO:teuthology.orchestra.run.smithi017.stdout: "new_removed_snaps": [], 2022-08-01T19:43:21.953 INFO:teuthology.orchestra.run.smithi017.stdout: "new_purged_snaps": [], 2022-08-01T19:43:21.953 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_node_flags": {}, 2022-08-01T19:43:21.953 INFO:teuthology.orchestra.run.smithi017.stdout: "device_class_flags": {} 2022-08-01T19:43:21.954 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:43:21.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.657+0000 7faf654e9700 1 -- 172.21.15.17:0/3641209428 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7faf400539e0 msgr2=0x7faf40055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:21.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.657+0000 7faf654e9700 1 --2- 172.21.15.17:0/3641209428 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7faf400539e0 0x7faf40055ea0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7faf5000d000 tx=0x7faf50009040).stop 2022-08-01T19:43:21.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.657+0000 7faf654e9700 1 -- 172.21.15.17:0/3641209428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf600ac930 msgr2=0x7faf601480f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:21.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.657+0000 7faf654e9700 1 --2- 172.21.15.17:0/3641209428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf600ac930 0x7faf601480f0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 rx=0x7faf540030b0 tx=0x7faf54008040).stop 2022-08-01T19:43:21.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.657+0000 7faf654e9700 1 -- 172.21.15.17:0/3641209428 shutdown_connections 2022-08-01T19:43:21.958 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.657+0000 7faf654e9700 1 --2- 172.21.15.17:0/3641209428 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7faf400539e0 0x7faf40055ea0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:21.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.657+0000 7faf654e9700 1 --2- 172.21.15.17:0/3641209428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf601444d0 0x7faf6015a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:21.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.657+0000 7faf654e9700 1 --2- 172.21.15.17:0/3641209428 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7faf600b0800 0x7faf6015a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:21.959 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.657+0000 7faf654e9700 1 --2- 172.21.15.17:0/3641209428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf600ac930 0x7faf601480f0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:21.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.657+0000 7faf654e9700 1 -- 172.21.15.17:0/3641209428 >> 172.21.15.17:0/3641209428 conn(0x7faf6005bed0 msgr2=0x7faf600b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:21.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.657+0000 7faf654e9700 1 -- 172.21.15.17:0/3641209428 shutdown_connections 2022-08-01T19:43:21.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:21.657+0000 7faf654e9700 1 -- 172.21.15.17:0/3641209428 wait complete. 2022-08-01T19:43:21.961 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:43:22.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.069+0000 7f031456a700 1 Processor -- start 2022-08-01T19:43:22.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.073+0000 7f031456a700 1 -- start start 2022-08-01T19:43:22.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.073+0000 7f031456a700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f030c0b26f0 0x7f030c0a7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:22.076 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.073+0000 7f031456a700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f030c14ada0 con 0x7f030c147730 2022-08-01T19:43:22.076 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.073+0000 7f031456a700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f030c0b2ad0 con 0x7f030c0b1ae0 2022-08-01T19:43:22.076 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.073+0000 7f031456a700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f030c0a8e50 con 0x7f030c0b26f0 2022-08-01T19:43:22.076 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.073+0000 7f0312b07700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f030c147730 0x7f030c0ab940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36772/0 (socket says 172.21.15.17:36772) 2022-08-01T19:43:22.077 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.073+0000 7f0312b07700 1 -- 172.21.15.17:0/1692320238 learned_addr learned my addr 172.21.15.17:0/1692320238 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:22.077 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.073+0000 7f0312306700 1 --2- 172.21.15.17:0/1692320238 >> v2:172.21.15.17:3300/0 conn(0x7f030c0b26f0 0x7f030c0a7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:22.079 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.073+0000 7f0311304700 1 -- 172.21.15.17:0/1692320238 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1150083869 0 0) 0x7f030c14ada0 con 0x7f030c147730 2022-08-01T19:43:22.079 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.073+0000 7f0311304700 1 -- 172.21.15.17:0/1692320238 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02f4003260 con 0x7f030c147730 2022-08-01T19:43:22.080 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.073+0000 7f0311304700 1 -- 172.21.15.17:0/1692320238 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1341494398 0 0) 0x7f030c0b2ad0 con 0x7f030c0b1ae0 2022-08-01T19:43:22.080 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.073+0000 7f0312306700 1 -- 172.21.15.17:0/1692320238 >> v1:172.21.15.77:6789/0 conn(0x7f030c0b1ae0 legacy=0x7f030c14b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:22.080 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.073+0000 7f0312306700 1 -- 172.21.15.17:0/1692320238 >> v1:172.21.15.17:6789/0 conn(0x7f030c147730 legacy=0x7f030c0ab940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:22.080 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.073+0000 7f0312306700 1 -- 172.21.15.17:0/1692320238 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f030c0b0aa0 con 0x7f030c0b26f0 2022-08-01T19:43:22.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.073+0000 7f0312306700 1 --2- 172.21.15.17:0/1692320238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f030c0b26f0 0x7f030c0a7e90 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 rx=0x7f0300002f70 tx=0x7f0300006df0).ready entity=mon.0 client_cookie=8dadd07fe5f221df server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:22.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.073+0000 7f0311304700 1 -- 172.21.15.17:0/1692320238 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f030000d610 con 0x7f030c0b26f0 2022-08-01T19:43:22.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.073+0000 7f0311304700 1 -- 172.21.15.17:0/1692320238 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0300000e90 con 0x7f030c0b26f0 2022-08-01T19:43:22.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.073+0000 7f0311304700 1 -- 172.21.15.17:0/1692320238 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f030000d960 con 0x7f030c0b26f0 2022-08-01T19:43:22.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.073+0000 7f031456a700 1 -- 172.21.15.17:0/1692320238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f030c0b26f0 msgr2=0x7f030c0a7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:22.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.073+0000 7f031456a700 1 --2- 172.21.15.17:0/1692320238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f030c0b26f0 0x7f030c0a7e90 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 rx=0x7f0300002f70 tx=0x7f0300006df0).stop 2022-08-01T19:43:22.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.073+0000 7f031456a700 1 -- 172.21.15.17:0/1692320238 shutdown_connections 2022-08-01T19:43:22.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.073+0000 7f031456a700 1 --2- 172.21.15.17:0/1692320238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f030c0b26f0 0x7f030c0a7e90 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:22.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.073+0000 7f031456a700 1 -- 172.21.15.17:0/1692320238 >> 172.21.15.17:0/1692320238 conn(0x7f030c05cbb0 msgr2=0x7f030c05cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:22.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.073+0000 7f031456a700 1 -- 172.21.15.17:0/1692320238 shutdown_connections 2022-08-01T19:43:22.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.073+0000 7f031456a700 1 -- 172.21.15.17:0/1692320238 wait complete. 2022-08-01T19:43:22.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.077+0000 7f031456a700 1 Processor -- start 2022-08-01T19:43:22.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.077+0000 7f031456a700 1 -- start start 2022-08-01T19:43:22.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.077+0000 7f031456a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f030c0b1ae0 0x7f030c0b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:22.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.077+0000 7f031456a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f030c0b26f0 0x7f030c0b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:22.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.077+0000 7f031456a700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f030c147730 0x7f030c0b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:22.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.077+0000 7f031456a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030c0b3450 con 0x7f030c0b1ae0 2022-08-01T19:43:22.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.077+0000 7f031456a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030c0b36f0 con 0x7f030c0b26f0 2022-08-01T19:43:22.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.077+0000 7f031456a700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f030c0b3990 con 0x7f030c147730 2022-08-01T19:43:22.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.077+0000 7f0312306700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f030c0b1ae0 0x7f030c0b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:22.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.077+0000 7f0312306700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f030c0b1ae0 0x7f030c0b6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44580/0 (socket says 172.21.15.17:44580) 2022-08-01T19:43:22.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.077+0000 7f0312306700 1 -- 172.21.15.17:0/290151020 learned_addr learned my addr 172.21.15.17:0/290151020 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:22.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.077+0000 7f0311b05700 1 --2- 172.21.15.17:0/290151020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f030c0b26f0 0x7f030c0b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:22.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.077+0000 7f0312b07700 1 --2- 172.21.15.17:0/290151020 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f030c147730 0x7f030c0b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:22.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.077+0000 7f0312306700 1 -- 172.21.15.17:0/290151020 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f030c147730 msgr2=0x7f030c0b57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:22.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.077+0000 7f0312306700 1 --2- 172.21.15.17:0/290151020 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f030c147730 0x7f030c0b57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:22.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.077+0000 7f0312306700 1 -- 172.21.15.17:0/290151020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f030c0b26f0 msgr2=0x7f030c0b52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:22.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.077+0000 7f0312306700 1 --2- 172.21.15.17:0/290151020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f030c0b26f0 0x7f030c0b52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:22.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.077+0000 7f0312306700 1 -- 172.21.15.17:0/290151020 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f030000b180 con 0x7f030c0b1ae0 2022-08-01T19:43:22.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.077+0000 7f0312306700 1 --2- 172.21.15.17:0/290151020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f030c0b1ae0 0x7f030c0b6090 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 rx=0x7f0300002f70 tx=0x7f0300014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:22.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.081+0000 7f02ff7fe700 1 -- 172.21.15.17:0/290151020 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0300013de0 con 0x7f030c0b1ae0 2022-08-01T19:43:22.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.081+0000 7f02ff7fe700 1 -- 172.21.15.17:0/290151020 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f030000d290 con 0x7f030c0b1ae0 2022-08-01T19:43:22.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.081+0000 7f031456a700 1 -- 172.21.15.17:0/290151020 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f030c0b3c30 con 0x7f030c0b1ae0 2022-08-01T19:43:22.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.081+0000 7f02ff7fe700 1 -- 172.21.15.17:0/290151020 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f030001e5f0 con 0x7f030c0b1ae0 2022-08-01T19:43:22.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.081+0000 7f031456a700 1 -- 172.21.15.17:0/290151020 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f030c0af8a0 con 0x7f030c0b1ae0 2022-08-01T19:43:22.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.081+0000 7f031456a700 1 -- 172.21.15.17:0/290151020 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f030c0b5050 con 0x7f030c0b1ae0 2022-08-01T19:43:22.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.081+0000 7f02ff7fe700 1 -- 172.21.15.17:0/290151020 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0300025030 con 0x7f030c0b1ae0 2022-08-01T19:43:22.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.085+0000 7f02ff7fe700 1 --2- 172.21.15.17:0/290151020 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f02ec053a10 0x7f02ec055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:22.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.085+0000 7f02ff7fe700 1 -- 172.21.15.17:0/290151020 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f030006ce30 con 0x7f030c0b1ae0 2022-08-01T19:43:22.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.085+0000 7f0311b05700 1 --2- 172.21.15.17:0/290151020 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f02ec053a10 0x7f02ec055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:22.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.085+0000 7f0311b05700 1 --2- 172.21.15.17:0/290151020 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f02ec053a10 0x7f02ec055ed0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7f030800d000 tx=0x7f0308007a00).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:22.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.089+0000 7f02ff7fe700 1 -- 172.21.15.17:0/290151020 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f030c0b5050 con 0x7f030c0b1ae0 2022-08-01T19:43:22.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.253+0000 7f031456a700 1 -- 172.21.15.17:0/290151020 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f030c0b5050 con 0x7f030c0b1ae0 2022-08-01T19:43:22.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.253+0000 7f02ff7fe700 1 -- 172.21.15.17:0/290151020 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+2486 (secure 0 0 0) 0x7f030c0b5050 con 0x7f030c0b1ae0 2022-08-01T19:43:22.255 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:22.256 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":69,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7995,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.17:6833/4164640472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4164640472},{"type":"v1","addr":"172.21.15.17:6833","nonce":4164640472}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":67},{"gid":8001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/50015801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":50015801},{"type":"v1","addr":"172.21.15.17:6835","nonce":50015801}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:43:01.287477+0000","modified":"2022-08-01T19:43:02.290478+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8007},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8007":{"gid":8007,"name":"b","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.77:6835/299216688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":299216688},{"type":"v1","addr":"172.21.15.77:6835","nonce":299216688}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2022-08-01T19:43:22.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.253+0000 7f031456a700 1 -- 172.21.15.17:0/290151020 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f02ec053a10 msgr2=0x7f02ec055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:22.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.253+0000 7f031456a700 1 --2- 172.21.15.17:0/290151020 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f02ec053a10 0x7f02ec055ed0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7f030800d000 tx=0x7f0308007a00).stop 2022-08-01T19:43:22.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.253+0000 7f031456a700 1 -- 172.21.15.17:0/290151020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f030c0b1ae0 msgr2=0x7f030c0b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:22.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.253+0000 7f031456a700 1 --2- 172.21.15.17:0/290151020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f030c0b1ae0 0x7f030c0b6090 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 rx=0x7f0300002f70 tx=0x7f0300014000).stop 2022-08-01T19:43:22.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.253+0000 7f031456a700 1 -- 172.21.15.17:0/290151020 shutdown_connections 2022-08-01T19:43:22.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.253+0000 7f031456a700 1 --2- 172.21.15.17:0/290151020 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f02ec053a10 0x7f02ec055ed0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:22.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.253+0000 7f031456a700 1 --2- 172.21.15.17:0/290151020 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f030c147730 0x7f030c0b57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:22.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.253+0000 7f031456a700 1 --2- 172.21.15.17:0/290151020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f030c0b26f0 0x7f030c0b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:22.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.253+0000 7f031456a700 1 --2- 172.21.15.17:0/290151020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f030c0b1ae0 0x7f030c0b6090 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:22.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.253+0000 7f031456a700 1 -- 172.21.15.17:0/290151020 >> 172.21.15.17:0/290151020 conn(0x7f030c05cbb0 msgr2=0x7f030c144b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:22.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.253+0000 7f031456a700 1 -- 172.21.15.17:0/290151020 shutdown_connections 2022-08-01T19:43:22.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.253+0000 7f031456a700 1 -- 172.21.15.17:0/290151020 wait complete. 2022-08-01T19:43:22.263 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 69 2022-08-01T19:43:22.269 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-08-01T19:43:22.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.369+0000 7f35e7ebf700 1 Processor -- start 2022-08-01T19:43:22.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.369+0000 7f35e7ebf700 1 -- start start 2022-08-01T19:43:22.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.373+0000 7f35e7ebf700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f35e00b2b70 0x7f35e00b1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:22.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.373+0000 7f35e7ebf700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35e01446e0 0x7f35e0141500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:22.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.373+0000 7f35e7ebf700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35e00af440 con 0x7f35e00b2790 2022-08-01T19:43:22.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.373+0000 7f35e7ebf700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35e00a99c0 con 0x7f35e00b2b70 2022-08-01T19:43:22.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.373+0000 7f35e7ebf700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e00b0080 con 0x7f35e01446e0 2022-08-01T19:43:22.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.373+0000 7f35e545a700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f35e00b2b70 0x7f35e00b1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:22.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.373+0000 7f35e545a700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f35e00b2b70 0x7f35e00b1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44590/0 (socket says 172.21.15.17:44590) 2022-08-01T19:43:22.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.373+0000 7f35e545a700 1 -- 172.21.15.17:0/1290530119 learned_addr learned my addr 172.21.15.17:0/1290530119 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:22.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.373+0000 7f35e645c700 1 --2- 172.21.15.17:0/1290530119 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35e01446e0 0x7f35e0141500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:22.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.373+0000 7f35e545a700 1 -- 172.21.15.17:0/1290530119 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35e01446e0 msgr2=0x7f35e0141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:22.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.373+0000 7f35e545a700 1 --2- 172.21.15.17:0/1290530119 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35e01446e0 0x7f35e0141500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:22.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.373+0000 7f35e545a700 1 -- 172.21.15.17:0/1290530119 >> v1:172.21.15.77:6789/0 conn(0x7f35e00b2790 legacy=0x7f35e00aed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:22.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.373+0000 7f35e545a700 1 -- 172.21.15.17:0/1290530119 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e0144e60 con 0x7f35e00b2b70 2022-08-01T19:43:22.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.373+0000 7f35e645c700 1 --2- 172.21.15.17:0/1290530119 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35e01446e0 0x7f35e0141500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:43:22.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.373+0000 7f35e545a700 1 --2- 172.21.15.17:0/1290530119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35e00b2b70 0x7f35e00b1f70 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 rx=0x7f35dc002f70 tx=0x7f35dc006eb0).ready entity=mon.0 client_cookie=11d3dfff92ded24b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:22.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.373+0000 7f35e4c59700 1 -- 172.21.15.17:0/1290530119 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35dc00d5c0 con 0x7f35e00b2b70 2022-08-01T19:43:22.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.373+0000 7f35e4c59700 1 -- 172.21.15.17:0/1290530119 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35dc000e90 con 0x7f35e00b2b70 2022-08-01T19:43:22.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.373+0000 7f35e4c59700 1 -- 172.21.15.17:0/1290530119 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35dc00d950 con 0x7f35e00b2b70 2022-08-01T19:43:22.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.373+0000 7f35e7ebf700 1 -- 172.21.15.17:0/1290530119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35e00b2b70 msgr2=0x7f35e00b1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:22.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.373+0000 7f35e7ebf700 1 --2- 172.21.15.17:0/1290530119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35e00b2b70 0x7f35e00b1f70 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 rx=0x7f35dc002f70 tx=0x7f35dc006eb0).stop 2022-08-01T19:43:22.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.373+0000 7f35e7ebf700 1 -- 172.21.15.17:0/1290530119 shutdown_connections 2022-08-01T19:43:22.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.373+0000 7f35e7ebf700 1 --2- 172.21.15.17:0/1290530119 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35e01446e0 0x7f35e0141500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:22.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.373+0000 7f35e7ebf700 1 --2- 172.21.15.17:0/1290530119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35e00b2b70 0x7f35e00b1f70 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:22.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.373+0000 7f35e7ebf700 1 -- 172.21.15.17:0/1290530119 >> 172.21.15.17:0/1290530119 conn(0x7f35e005bed0 msgr2=0x7f35e005c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:22.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.373+0000 7f35e7ebf700 1 -- 172.21.15.17:0/1290530119 shutdown_connections 2022-08-01T19:43:22.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.373+0000 7f35e7ebf700 1 -- 172.21.15.17:0/1290530119 wait complete. 2022-08-01T19:43:22.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.377+0000 7f35e7ebf700 1 Processor -- start 2022-08-01T19:43:22.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.377+0000 7f35e7ebf700 1 -- start start 2022-08-01T19:43:22.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.377+0000 7f35e7ebf700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35e00b2790 0x7f35e00b5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:22.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.377+0000 7f35e7ebf700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35e00b2b70 0x7f35e0063770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:22.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.377+0000 7f35e7ebf700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35e01446e0 0x7f35e013d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:22.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.377+0000 7f35e7ebf700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e00b4970 con 0x7f35e00b2b70 2022-08-01T19:43:22.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.377+0000 7f35e7ebf700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e00b6360 con 0x7f35e00b2790 2022-08-01T19:43:22.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.377+0000 7f35e7ebf700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e00b4bd0 con 0x7f35e01446e0 2022-08-01T19:43:22.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.377+0000 7f35e545a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35e00b2b70 0x7f35e0063770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:22.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.377+0000 7f35e545a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35e00b2b70 0x7f35e0063770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44596/0 (socket says 172.21.15.17:44596) 2022-08-01T19:43:22.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.377+0000 7f35e545a700 1 -- 172.21.15.17:0/3845960818 learned_addr learned my addr 172.21.15.17:0/3845960818 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:22.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.377+0000 7f35e5c5b700 1 --2- 172.21.15.17:0/3845960818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35e00b2790 0x7f35e00b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:22.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.377+0000 7f35e645c700 1 --2- 172.21.15.17:0/3845960818 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35e01446e0 0x7f35e013d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:22.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.377+0000 7f35e545a700 1 -- 172.21.15.17:0/3845960818 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35e01446e0 msgr2=0x7f35e013d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:22.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.377+0000 7f35e545a700 1 --2- 172.21.15.17:0/3845960818 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35e01446e0 0x7f35e013d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:22.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.377+0000 7f35e545a700 1 -- 172.21.15.17:0/3845960818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35e00b2790 msgr2=0x7f35e00b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:22.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.377+0000 7f35e545a700 1 --2- 172.21.15.17:0/3845960818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35e00b2790 0x7f35e00b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:22.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.377+0000 7f35e545a700 1 -- 172.21.15.17:0/3845960818 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35dc00b1e0 con 0x7f35e00b2b70 2022-08-01T19:43:22.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.377+0000 7f35e545a700 1 --2- 172.21.15.17:0/3845960818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35e00b2b70 0x7f35e0063770 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 rx=0x7f35dc002f70 tx=0x7f35dc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:22.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.377+0000 7f35ceffd700 1 -- 172.21.15.17:0/3845960818 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35dc0117e0 con 0x7f35e00b2b70 2022-08-01T19:43:22.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.377+0000 7f35ceffd700 1 -- 172.21.15.17:0/3845960818 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35dc011940 con 0x7f35e00b2b70 2022-08-01T19:43:22.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.377+0000 7f35e7ebf700 1 -- 172.21.15.17:0/3845960818 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e00b4e70 con 0x7f35e00b2b70 2022-08-01T19:43:22.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.377+0000 7f35ceffd700 1 -- 172.21.15.17:0/3845960818 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35dc011ca0 con 0x7f35e00b2b70 2022-08-01T19:43:22.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.377+0000 7f35e7ebf700 1 -- 172.21.15.17:0/3845960818 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e00b5ec0 con 0x7f35e00b2b70 2022-08-01T19:43:22.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.381+0000 7f35e7ebf700 1 -- 172.21.15.17:0/3845960818 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35e00574e0 con 0x7f35e00b2b70 2022-08-01T19:43:22.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.381+0000 7f35ceffd700 1 -- 172.21.15.17:0/3845960818 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f35dc025030 con 0x7f35e00b2b70 2022-08-01T19:43:22.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.381+0000 7f35ceffd700 1 --2- 172.21.15.17:0/3845960818 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f35c0053a70 0x7f35c0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:22.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.381+0000 7f35ceffd700 1 -- 172.21.15.17:0/3845960818 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f35dc06d2b0 con 0x7f35e00b2b70 2022-08-01T19:43:22.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.381+0000 7f35e5c5b700 1 --2- 172.21.15.17:0/3845960818 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f35c0053a70 0x7f35c0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:22.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.381+0000 7f35e5c5b700 1 --2- 172.21.15.17:0/3845960818 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f35c0053a70 0x7f35c0055f30 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f35d000d000 tx=0x7f35d0007b50).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:22.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.389+0000 7f35ceffd700 1 -- 172.21.15.17:0/3845960818 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f35dc043f60 con 0x7f35e00b2b70 2022-08-01T19:43:22.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:22.553+0000 7f35e7ebf700 1 -- 172.21.15.17:0/3845960818 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f35e0069b20 con 0x7f35e00b2b70 2022-08-01T19:43:23.239 INFO:tasks.ceph.mon.a.smithi017.stderr:2022-08-01T19:43:23.237+0000 7f6a53773700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-08-01T19:43:23.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.245+0000 7f35ceffd700 1 -- 172.21.15.17:0/3845960818 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v70) v1 ==== 161+0+0 (secure 0 0 0) 0x7f35dc023090 con 0x7f35e00b2b70 2022-08-01T19:43:23.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.249+0000 7f35e7ebf700 1 -- 172.21.15.17:0/3845960818 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f35c0053a70 msgr2=0x7f35c0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:23.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.249+0000 7f35e7ebf700 1 --2- 172.21.15.17:0/3845960818 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f35c0053a70 0x7f35c0055f30 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f35d000d000 tx=0x7f35d0007b50).stop 2022-08-01T19:43:23.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.249+0000 7f35e7ebf700 1 -- 172.21.15.17:0/3845960818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35e00b2b70 msgr2=0x7f35e0063770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:23.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.249+0000 7f35e7ebf700 1 --2- 172.21.15.17:0/3845960818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35e00b2b70 0x7f35e0063770 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 rx=0x7f35dc002f70 tx=0x7f35dc015000).stop 2022-08-01T19:43:23.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.249+0000 7f35e7ebf700 1 -- 172.21.15.17:0/3845960818 shutdown_connections 2022-08-01T19:43:23.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.249+0000 7f35e7ebf700 1 --2- 172.21.15.17:0/3845960818 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f35c0053a70 0x7f35c0055f30 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:23.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.249+0000 7f35e7ebf700 1 --2- 172.21.15.17:0/3845960818 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f35e01446e0 0x7f35e013d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:23.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.249+0000 7f35e7ebf700 1 --2- 172.21.15.17:0/3845960818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35e00b2b70 0x7f35e0063770 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:23.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.249+0000 7f35e7ebf700 1 --2- 172.21.15.17:0/3845960818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35e00b2790 0x7f35e00b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:23.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.249+0000 7f35e7ebf700 1 -- 172.21.15.17:0/3845960818 >> 172.21.15.17:0/3845960818 conn(0x7f35e005bed0 msgr2=0x7f35e013f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:23.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.249+0000 7f35e7ebf700 1 -- 172.21.15.17:0/3845960818 shutdown_connections 2022-08-01T19:43:23.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.249+0000 7f35e7ebf700 1 -- 172.21.15.17:0/3845960818 wait complete. 2022-08-01T19:43:23.265 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-08-01T19:43:23.270 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:43:23.279 INFO:tasks.ceph.mds.b.smithi077.stdout:starting mds.b at 2022-08-01T19:43:23.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.373+0000 7f904727e700 1 Processor -- start 2022-08-01T19:43:23.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.373+0000 7f904727e700 1 -- start start 2022-08-01T19:43:23.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.377+0000 7f904727e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f90400afc50 0x7f904013e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:23.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.377+0000 7f904727e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f90400b1b00 0x7f904013e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:23.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.377+0000 7f904727e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90400b6c30 con 0x7f90400b6850 2022-08-01T19:43:23.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.377+0000 7f904727e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9040060460 con 0x7f90400b1b00 2022-08-01T19:43:23.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.377+0000 7f904727e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f904013ec10 con 0x7f90400afc50 2022-08-01T19:43:23.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.377+0000 7f9046a7d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f90400b1b00 0x7f904013e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:23.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.377+0000 7f9046a7d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f90400b1b00 0x7f904013e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44606/0 (socket says 172.21.15.17:44606) 2022-08-01T19:43:23.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.377+0000 7f9046a7d700 1 -- 172.21.15.17:0/2695550949 learned_addr learned my addr 172.21.15.17:0/2695550949 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:23.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.377+0000 7f9045a7b700 1 --2- 172.21.15.17:0/2695550949 >> v2:172.21.15.77:3300/0 conn(0x7f90400afc50 0x7f904013e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:23.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.377+0000 7f9046a7d700 1 -- 172.21.15.17:0/2695550949 >> v2:172.21.15.77:3300/0 conn(0x7f90400afc50 msgr2=0x7f904013e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:23.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.377+0000 7f9046a7d700 1 --2- 172.21.15.17:0/2695550949 >> v2:172.21.15.77:3300/0 conn(0x7f90400afc50 0x7f904013e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:23.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.377+0000 7f9046a7d700 1 -- 172.21.15.17:0/2695550949 >> v1:172.21.15.77:6789/0 conn(0x7f90400b6850 legacy=0x7f90401444b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:23.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.377+0000 7f9046a7d700 1 -- 172.21.15.17:0/2695550949 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9040063b80 con 0x7f90400b1b00 2022-08-01T19:43:23.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.377+0000 7f9046a7d700 1 --2- 172.21.15.17:0/2695550949 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90400b1b00 0x7f904013e720 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 rx=0x7f90380032f0 tx=0x7f9038009290).ready entity=mon.0 client_cookie=d1f6e2a3875a959f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:23.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.377+0000 7f904527a700 1 -- 172.21.15.17:0/2695550949 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f903800b870 con 0x7f90400b1b00 2022-08-01T19:43:23.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.377+0000 7f904527a700 1 -- 172.21.15.17:0/2695550949 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f903800b9d0 con 0x7f90400b1b00 2022-08-01T19:43:23.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.377+0000 7f904527a700 1 -- 172.21.15.17:0/2695550949 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f903800bb40 con 0x7f90400b1b00 2022-08-01T19:43:23.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.377+0000 7f904727e700 1 -- 172.21.15.17:0/2695550949 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90400b1b00 msgr2=0x7f904013e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:23.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.377+0000 7f904727e700 1 --2- 172.21.15.17:0/2695550949 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90400b1b00 0x7f904013e720 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 rx=0x7f90380032f0 tx=0x7f9038009290).stop 2022-08-01T19:43:23.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.377+0000 7f904727e700 1 -- 172.21.15.17:0/2695550949 shutdown_connections 2022-08-01T19:43:23.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.377+0000 7f904727e700 1 --2- 172.21.15.17:0/2695550949 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90400b1b00 0x7f904013e720 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:23.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.377+0000 7f904727e700 1 --2- 172.21.15.17:0/2695550949 >> v2:172.21.15.77:3300/0 conn(0x7f90400afc50 0x7f904013e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:23.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.377+0000 7f904727e700 1 -- 172.21.15.17:0/2695550949 >> 172.21.15.17:0/2695550949 conn(0x7f904005c8b0 msgr2=0x7f904005cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:23.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.377+0000 7f904727e700 1 -- 172.21.15.17:0/2695550949 shutdown_connections 2022-08-01T19:43:23.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.377+0000 7f904727e700 1 -- 172.21.15.17:0/2695550949 wait complete. 2022-08-01T19:43:23.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.377+0000 7f904727e700 1 Processor -- start 2022-08-01T19:43:23.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.377+0000 7f904727e700 1 -- start start 2022-08-01T19:43:23.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.381+0000 7f904727e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f90400afc50 0x7f9040064660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:23.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.381+0000 7f904727e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90400b1b00 0x7f904013a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:23.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.381+0000 7f904727e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90400b6850 0x7f9040153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:23.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.381+0000 7f904727e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90401538f0 con 0x7f90400b1b00 2022-08-01T19:43:23.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.381+0000 7f904727e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9040064e80 con 0x7f90400b6850 2022-08-01T19:43:23.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.381+0000 7f904727e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9040065120 con 0x7f90400afc50 2022-08-01T19:43:23.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.381+0000 7f9045a7b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90400b1b00 0x7f904013a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:23.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.381+0000 7f9045a7b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90400b1b00 0x7f904013a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44610/0 (socket says 172.21.15.17:44610) 2022-08-01T19:43:23.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.381+0000 7f9045a7b700 1 -- 172.21.15.17:0/1720267419 learned_addr learned my addr 172.21.15.17:0/1720267419 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:23.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.381+0000 7f9046a7d700 1 --2- 172.21.15.17:0/1720267419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90400b6850 0x7f9040153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:23.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.381+0000 7f904627c700 1 --2- 172.21.15.17:0/1720267419 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f90400afc50 0x7f9040064660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:23.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.381+0000 7f9045a7b700 1 -- 172.21.15.17:0/1720267419 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f90400afc50 msgr2=0x7f9040064660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:23.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.381+0000 7f9045a7b700 1 --2- 172.21.15.17:0/1720267419 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f90400afc50 0x7f9040064660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:23.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.381+0000 7f9045a7b700 1 -- 172.21.15.17:0/1720267419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90400b6850 msgr2=0x7f9040153d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:23.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.381+0000 7f9045a7b700 1 --2- 172.21.15.17:0/1720267419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90400b6850 0x7f9040153d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:23.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.381+0000 7f9045a7b700 1 -- 172.21.15.17:0/1720267419 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9038009530 con 0x7f90400b1b00 2022-08-01T19:43:23.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.381+0000 7f904627c700 1 --2- 172.21.15.17:0/1720267419 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f90400afc50 0x7f9040064660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:43:23.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.381+0000 7f9045a7b700 1 --2- 172.21.15.17:0/1720267419 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90400b1b00 0x7f904013a110 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 rx=0x7f903400c000 tx=0x7f9034006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:23.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.381+0000 7f90337fe700 1 -- 172.21.15.17:0/1720267419 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9034010780 con 0x7f90400b1b00 2022-08-01T19:43:23.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.381+0000 7f90337fe700 1 -- 172.21.15.17:0/1720267419 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90340108e0 con 0x7f90400b1b00 2022-08-01T19:43:23.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.381+0000 7f90337fe700 1 -- 172.21.15.17:0/1720267419 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9034010c40 con 0x7f90400b1b00 2022-08-01T19:43:23.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.381+0000 7f904727e700 1 -- 172.21.15.17:0/1720267419 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90400653c0 con 0x7f90400b1b00 2022-08-01T19:43:23.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.381+0000 7f904727e700 1 -- 172.21.15.17:0/1720267419 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9040150280 con 0x7f90400b1b00 2022-08-01T19:43:23.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.381+0000 7f904727e700 1 -- 172.21.15.17:0/1720267419 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90400601c0 con 0x7f90400b1b00 2022-08-01T19:43:23.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.385+0000 7f90337fe700 1 -- 172.21.15.17:0/1720267419 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f903401c460 con 0x7f90400b1b00 2022-08-01T19:43:23.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.385+0000 7f90337fe700 1 --2- 172.21.15.17:0/1720267419 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f90200539e0 0x7f9020055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:23.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.385+0000 7f90337fe700 1 -- 172.21.15.17:0/1720267419 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f903406c5f0 con 0x7f90400b1b00 2022-08-01T19:43:23.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.385+0000 7f904627c700 1 --2- 172.21.15.17:0/1720267419 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f90200539e0 0x7f9020055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:23.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.385+0000 7f904627c700 1 --2- 172.21.15.17:0/1720267419 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f90200539e0 0x7f9020055ea0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7f903c00b000 tx=0x7f903c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:23.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.389+0000 7f90337fe700 1 -- 172.21.15.17:0/1720267419 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9034043170 con 0x7f90400b1b00 2022-08-01T19:43:23.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.553+0000 7f904727e700 1 -- 172.21.15.17:0/1720267419 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90400574e0 con 0x7f90400b1b00 2022-08-01T19:43:23.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.553+0000 7f90337fe700 1 -- 172.21.15.17:0/1720267419 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+2129 (secure 0 0 0) 0x7f9034046790 con 0x7f90400b1b00 2022-08-01T19:43:23.559 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:23.559 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":70,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7995,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.17:6833/4164640472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4164640472},{"type":"v1","addr":"172.21.15.17:6833","nonce":4164640472}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":67},{"gid":8001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/50015801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":50015801},{"type":"v1","addr":"172.21.15.17:6835","nonce":50015801}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66}],"filesystems":[{"mdsmap":{"epoch":70,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:43:01.287477+0000","modified":"2022-08-01T19:43:23.240263+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":138,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2022-08-01T19:43:23.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.557+0000 7f904727e700 1 -- 172.21.15.17:0/1720267419 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f90200539e0 msgr2=0x7f9020055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:23.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.557+0000 7f904727e700 1 --2- 172.21.15.17:0/1720267419 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f90200539e0 0x7f9020055ea0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7f903c00b000 tx=0x7f903c009040).stop 2022-08-01T19:43:23.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.557+0000 7f904727e700 1 -- 172.21.15.17:0/1720267419 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90400b1b00 msgr2=0x7f904013a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:23.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.557+0000 7f904727e700 1 --2- 172.21.15.17:0/1720267419 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90400b1b00 0x7f904013a110 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 rx=0x7f903400c000 tx=0x7f9034006df0).stop 2022-08-01T19:43:23.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.557+0000 7f904727e700 1 -- 172.21.15.17:0/1720267419 shutdown_connections 2022-08-01T19:43:23.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.557+0000 7f904727e700 1 --2- 172.21.15.17:0/1720267419 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f90200539e0 0x7f9020055ea0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:23.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.557+0000 7f904727e700 1 --2- 172.21.15.17:0/1720267419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90400b6850 0x7f9040153d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:23.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.557+0000 7f904727e700 1 --2- 172.21.15.17:0/1720267419 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90400b1b00 0x7f904013a110 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:23.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.557+0000 7f904727e700 1 --2- 172.21.15.17:0/1720267419 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f90400afc50 0x7f9040064660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:23.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.557+0000 7f904727e700 1 -- 172.21.15.17:0/1720267419 >> 172.21.15.17:0/1720267419 conn(0x7f904005c8b0 msgr2=0x7f904013b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:23.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.557+0000 7f904727e700 1 -- 172.21.15.17:0/1720267419 shutdown_connections 2022-08-01T19:43:23.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.557+0000 7f904727e700 1 -- 172.21.15.17:0/1720267419 wait complete. 2022-08-01T19:43:23.568 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 70 2022-08-01T19:43:23.571 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-08-01T19:43:23.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.673+0000 7eff9e1e1700 1 Processor -- start 2022-08-01T19:43:23.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.673+0000 7eff9e1e1700 1 -- start start 2022-08-01T19:43:23.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.677+0000 7eff9e1e1700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7eff980b66b0 0x7eff980b1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:23.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.677+0000 7eff9e1e1700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7eff980b23d0 0x7eff981408e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:23.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.677+0000 7eff9e1e1700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff980b0a30 con 0x7eff980b2ec0 2022-08-01T19:43:23.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.677+0000 7eff9e1e1700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff980b1eb0 con 0x7eff980b66b0 2022-08-01T19:43:23.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.677+0000 7eff9e1e1700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff9813ff00 con 0x7eff980b23d0 2022-08-01T19:43:23.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.677+0000 7eff9d9e0700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7eff980b2ec0 0x7eff98139a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36812/0 (socket says 172.21.15.17:36812) 2022-08-01T19:43:23.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.677+0000 7eff9d9e0700 1 -- 172.21.15.17:0/2365573225 learned_addr learned my addr 172.21.15.17:0/2365573225 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:23.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.677+0000 7eff9d1df700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7eff980b66b0 0x7eff980b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:23.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.677+0000 7eff9c9de700 1 --2- 172.21.15.17:0/2365573225 >> v2:172.21.15.77:3300/0 conn(0x7eff980b23d0 0x7eff981408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:23.692 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.677+0000 7eff87fff700 1 -- 172.21.15.17:0/2365573225 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3174723360 0 0) 0x7eff980b0a30 con 0x7eff980b2ec0 2022-08-01T19:43:23.692 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.677+0000 7eff87fff700 1 -- 172.21.15.17:0/2365573225 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff80003260 con 0x7eff980b2ec0 2022-08-01T19:43:23.692 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.677+0000 7eff87fff700 1 -- 172.21.15.17:0/2365573225 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2206611907 0 0) 0x7eff80003260 con 0x7eff980b2ec0 2022-08-01T19:43:23.692 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.677+0000 7eff9d1df700 1 -- 172.21.15.17:0/2365573225 >> v2:172.21.15.77:3300/0 conn(0x7eff980b23d0 msgr2=0x7eff981408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:23.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.677+0000 7eff9d1df700 1 --2- 172.21.15.17:0/2365573225 >> v2:172.21.15.77:3300/0 conn(0x7eff980b23d0 0x7eff981408e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:23.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.677+0000 7eff9d1df700 1 -- 172.21.15.17:0/2365573225 >> v1:172.21.15.17:6789/0 conn(0x7eff980b2ec0 legacy=0x7eff98139a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:23.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.677+0000 7eff9d1df700 1 -- 172.21.15.17:0/2365573225 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff981401a0 con 0x7eff980b66b0 2022-08-01T19:43:23.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.677+0000 7eff9d1df700 1 --2- 172.21.15.17:0/2365573225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff980b66b0 0x7eff980b1810 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 rx=0x7eff90001b80 tx=0x7eff94006df0).ready entity=mon.0 client_cookie=61f2b4938292608 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:23.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.677+0000 7eff87fff700 1 -- 172.21.15.17:0/2365573225 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff900019e0 con 0x7eff980b66b0 2022-08-01T19:43:23.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.677+0000 7eff87fff700 1 -- 172.21.15.17:0/2365573225 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff94000e90 con 0x7eff980b66b0 2022-08-01T19:43:23.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.677+0000 7eff87fff700 1 -- 172.21.15.17:0/2365573225 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff94012460 con 0x7eff980b66b0 2022-08-01T19:43:23.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.677+0000 7eff9e1e1700 1 -- 172.21.15.17:0/2365573225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff980b66b0 msgr2=0x7eff980b1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:23.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.677+0000 7eff9e1e1700 1 --2- 172.21.15.17:0/2365573225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff980b66b0 0x7eff980b1810 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 rx=0x7eff90001b80 tx=0x7eff94006df0).stop 2022-08-01T19:43:23.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.677+0000 7eff9e1e1700 1 -- 172.21.15.17:0/2365573225 shutdown_connections 2022-08-01T19:43:23.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.677+0000 7eff9e1e1700 1 --2- 172.21.15.17:0/2365573225 >> v2:172.21.15.77:3300/0 conn(0x7eff980b23d0 0x7eff981408e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:23.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.677+0000 7eff9e1e1700 1 --2- 172.21.15.17:0/2365573225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff980b66b0 0x7eff980b1810 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:23.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.677+0000 7eff9e1e1700 1 -- 172.21.15.17:0/2365573225 >> 172.21.15.17:0/2365573225 conn(0x7eff9805c8b0 msgr2=0x7eff9805cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:23.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.677+0000 7eff9e1e1700 1 -- 172.21.15.17:0/2365573225 shutdown_connections 2022-08-01T19:43:23.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.677+0000 7eff9e1e1700 1 -- 172.21.15.17:0/2365573225 wait complete. 2022-08-01T19:43:23.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.677+0000 7eff9e1e1700 1 Processor -- start 2022-08-01T19:43:23.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.677+0000 7eff9e1e1700 1 -- start start 2022-08-01T19:43:23.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.681+0000 7eff9e1e1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff980b23d0 0x7eff980b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:23.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.681+0000 7eff9e1e1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff980b2ec0 0x7eff98143cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:23.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.681+0000 7eff9e1e1700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7eff980b66b0 0x7eff9813bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:23.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.681+0000 7eff9e1e1700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9813a5a0 con 0x7eff980b23d0 2022-08-01T19:43:23.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.681+0000 7eff9e1e1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9813b700 con 0x7eff980b2ec0 2022-08-01T19:43:23.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.681+0000 7eff9e1e1700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff98142340 con 0x7eff980b66b0 2022-08-01T19:43:23.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.681+0000 7eff9d1df700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff980b23d0 0x7eff980b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:23.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.681+0000 7eff9d1df700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff980b23d0 0x7eff980b13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44620/0 (socket says 172.21.15.17:44620) 2022-08-01T19:43:23.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.681+0000 7eff9d1df700 1 -- 172.21.15.17:0/635688692 learned_addr learned my addr 172.21.15.17:0/635688692 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:23.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.681+0000 7eff9c9de700 1 --2- 172.21.15.17:0/635688692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff980b2ec0 0x7eff98143cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:23.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.681+0000 7eff9d9e0700 1 --2- 172.21.15.17:0/635688692 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7eff980b66b0 0x7eff9813bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:23.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.681+0000 7eff9d1df700 1 -- 172.21.15.17:0/635688692 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7eff980b66b0 msgr2=0x7eff9813bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:23.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.681+0000 7eff9d1df700 1 --2- 172.21.15.17:0/635688692 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7eff980b66b0 0x7eff9813bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:23.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.681+0000 7eff9d1df700 1 -- 172.21.15.17:0/635688692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff980b2ec0 msgr2=0x7eff98143cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:23.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.681+0000 7eff9d1df700 1 --2- 172.21.15.17:0/635688692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff980b2ec0 0x7eff98143cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:23.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.681+0000 7eff9d1df700 1 -- 172.21.15.17:0/635688692 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff9400b180 con 0x7eff980b23d0 2022-08-01T19:43:23.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.681+0000 7eff9c9de700 1 --2- 172.21.15.17:0/635688692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff980b2ec0 0x7eff98143cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:43:23.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.681+0000 7eff9d9e0700 1 --2- 172.21.15.17:0/635688692 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7eff980b66b0 0x7eff9813bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:43:23.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.681+0000 7eff9d1df700 1 --2- 172.21.15.17:0/635688692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff980b23d0 0x7eff980b13e0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 rx=0x7eff90001b80 tx=0x7eff94014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:23.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.681+0000 7eff867fc700 1 -- 172.21.15.17:0/635688692 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff94016030 con 0x7eff980b23d0 2022-08-01T19:43:23.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.681+0000 7eff867fc700 1 -- 172.21.15.17:0/635688692 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9400be90 con 0x7eff980b23d0 2022-08-01T19:43:23.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.681+0000 7eff9e1e1700 1 -- 172.21.15.17:0/635688692 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff9813c020 con 0x7eff980b23d0 2022-08-01T19:43:23.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.681+0000 7eff867fc700 1 -- 172.21.15.17:0/635688692 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff94012460 con 0x7eff980b23d0 2022-08-01T19:43:23.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.681+0000 7eff9e1e1700 1 -- 172.21.15.17:0/635688692 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff980616b0 con 0x7eff980b23d0 2022-08-01T19:43:23.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.681+0000 7eff9e1e1700 1 -- 172.21.15.17:0/635688692 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff9813b8c0 con 0x7eff980b23d0 2022-08-01T19:43:23.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.681+0000 7eff867fc700 1 -- 172.21.15.17:0/635688692 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7eff94024020 con 0x7eff980b23d0 2022-08-01T19:43:23.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.685+0000 7eff867fc700 1 --2- 172.21.15.17:0/635688692 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7eff78053b30 0x7eff78055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:23.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.685+0000 7eff867fc700 1 -- 172.21.15.17:0/635688692 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6506+0+0 (secure 0 0 0) 0x7eff94017070 con 0x7eff980b23d0 2022-08-01T19:43:23.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.685+0000 7eff9c9de700 1 --2- 172.21.15.17:0/635688692 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7eff78053b30 0x7eff78055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:23.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.685+0000 7eff9c9de700 1 --2- 172.21.15.17:0/635688692 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7eff78053b30 0x7eff78055ff0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7eff8800b000 tx=0x7eff88008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:23.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.689+0000 7eff867fc700 1 -- 172.21.15.17:0/635688692 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7eff94043cf0 con 0x7eff980b23d0 2022-08-01T19:43:23.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:23.857+0000 7eff9e1e1700 1 -- 172.21.15.17:0/635688692 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7eff980574e0 con 0x7eff980b23d0 2022-08-01T19:43:24.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.565+0000 7eff867fc700 1 -- 172.21.15.17:0/635688692 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v71) v1 ==== 104+0+0 (secure 0 0 0) 0x7eff94047310 con 0x7eff980b23d0 2022-08-01T19:43:24.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.569+0000 7eff9e1e1700 1 -- 172.21.15.17:0/635688692 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7eff78053b30 msgr2=0x7eff78055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:24.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.569+0000 7eff9e1e1700 1 --2- 172.21.15.17:0/635688692 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7eff78053b30 0x7eff78055ff0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7eff8800b000 tx=0x7eff88008040).stop 2022-08-01T19:43:24.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.569+0000 7eff9e1e1700 1 -- 172.21.15.17:0/635688692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff980b23d0 msgr2=0x7eff980b13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:24.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.569+0000 7eff9e1e1700 1 --2- 172.21.15.17:0/635688692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff980b23d0 0x7eff980b13e0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 rx=0x7eff90001b80 tx=0x7eff94014000).stop 2022-08-01T19:43:24.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.569+0000 7eff9e1e1700 1 -- 172.21.15.17:0/635688692 shutdown_connections 2022-08-01T19:43:24.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.569+0000 7eff9e1e1700 1 --2- 172.21.15.17:0/635688692 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7eff78053b30 0x7eff78055ff0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:24.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.569+0000 7eff9e1e1700 1 --2- 172.21.15.17:0/635688692 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7eff980b66b0 0x7eff9813bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:24.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.569+0000 7eff9e1e1700 1 --2- 172.21.15.17:0/635688692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff980b2ec0 0x7eff98143cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:24.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.569+0000 7eff9e1e1700 1 --2- 172.21.15.17:0/635688692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff980b23d0 0x7eff980b13e0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:24.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.569+0000 7eff9e1e1700 1 -- 172.21.15.17:0/635688692 >> 172.21.15.17:0/635688692 conn(0x7eff9805c8b0 msgr2=0x7eff980b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:24.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.569+0000 7eff9e1e1700 1 -- 172.21.15.17:0/635688692 shutdown_connections 2022-08-01T19:43:24.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.569+0000 7eff9e1e1700 1 -- 172.21.15.17:0/635688692 wait complete. 2022-08-01T19:43:24.591 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-08-01T19:43:24.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.713+0000 7f1189974700 1 Processor -- start 2022-08-01T19:43:24.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.713+0000 7f1189974700 1 -- start start 2022-08-01T19:43:24.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.717+0000 7f1189974700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f118405c020 0x7f118405c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:24.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.717+0000 7f1189974700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f118405c930 0x7f11841400f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:24.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.717+0000 7f1189974700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11840b4a50 con 0x7f11840b4670 2022-08-01T19:43:24.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.717+0000 7f1189974700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f118405cd50 con 0x7f118405c930 2022-08-01T19:43:24.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.717+0000 7f1189974700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1184066e60 con 0x7f118405c020 2022-08-01T19:43:24.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.717+0000 7f1182ffd700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f11840b4670 0x7f1184143e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36822/0 (socket says 172.21.15.17:36822) 2022-08-01T19:43:24.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.717+0000 7f1182ffd700 1 -- 172.21.15.17:0/3822360415 learned_addr learned my addr 172.21.15.17:0/3822360415 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:24.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.717+0000 7f11837fe700 1 --2- 172.21.15.17:0/3822360415 >> v2:172.21.15.77:3300/0 conn(0x7f118405c930 0x7f11841400f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:24.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.717+0000 7f11827fc700 1 --2- 172.21.15.17:0/3822360415 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f118405c020 0x7f118405c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:24.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.717+0000 7f1181ffb700 1 -- 172.21.15.17:0/3822360415 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2429231276 0 0) 0x7f11840b4a50 con 0x7f11840b4670 2022-08-01T19:43:24.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.717+0000 7f1181ffb700 1 -- 172.21.15.17:0/3822360415 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1168003260 con 0x7f11840b4670 2022-08-01T19:43:24.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.717+0000 7f11837fe700 1 -- 172.21.15.17:0/3822360415 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f118405c020 msgr2=0x7f118405c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:24.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.717+0000 7f11837fe700 1 --2- 172.21.15.17:0/3822360415 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f118405c020 0x7f118405c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:24.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.717+0000 7f11837fe700 1 -- 172.21.15.17:0/3822360415 >> v1:172.21.15.17:6789/0 conn(0x7f11840b4670 legacy=0x7f1184143e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:24.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.717+0000 7f11837fe700 1 -- 172.21.15.17:0/3822360415 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1184067100 con 0x7f118405c930 2022-08-01T19:43:24.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.717+0000 7f11827fc700 1 --2- 172.21.15.17:0/3822360415 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f118405c020 0x7f118405c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:43:24.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.717+0000 7f11837fe700 1 --2- 172.21.15.17:0/3822360415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f118405c930 0x7f11841400f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f117c0032f0 tx=0x7f117c0091d0).ready entity=mon.1 client_cookie=2b94ef884fa08550 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:24.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.717+0000 7f1181ffb700 1 -- 172.21.15.17:0/3822360415 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f117c00b600 con 0x7f118405c930 2022-08-01T19:43:24.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.717+0000 7f1181ffb700 1 -- 172.21.15.17:0/3822360415 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117c00b760 con 0x7f118405c930 2022-08-01T19:43:24.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.717+0000 7f1181ffb700 1 -- 172.21.15.17:0/3822360415 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f117c00b8d0 con 0x7f118405c930 2022-08-01T19:43:24.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.717+0000 7f1189974700 1 -- 172.21.15.17:0/3822360415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f118405c930 msgr2=0x7f11841400f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:24.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.717+0000 7f1189974700 1 --2- 172.21.15.17:0/3822360415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f118405c930 0x7f11841400f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f117c0032f0 tx=0x7f117c0091d0).stop 2022-08-01T19:43:24.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.717+0000 7f1189974700 1 -- 172.21.15.17:0/3822360415 shutdown_connections 2022-08-01T19:43:24.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.717+0000 7f1189974700 1 --2- 172.21.15.17:0/3822360415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f118405c930 0x7f11841400f0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:24.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.717+0000 7f1189974700 1 --2- 172.21.15.17:0/3822360415 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f118405c020 0x7f118405c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:24.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.717+0000 7f1189974700 1 -- 172.21.15.17:0/3822360415 >> 172.21.15.17:0/3822360415 conn(0x7f11840b7900 msgr2=0x7f11840b7d10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:24.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.717+0000 7f1189974700 1 -- 172.21.15.17:0/3822360415 shutdown_connections 2022-08-01T19:43:24.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.717+0000 7f1189974700 1 -- 172.21.15.17:0/3822360415 wait complete. 2022-08-01T19:43:24.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.721+0000 7f1189974700 1 Processor -- start 2022-08-01T19:43:24.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.721+0000 7f1189974700 1 -- start start 2022-08-01T19:43:24.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.721+0000 7f1189974700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f118405c020 0x7f11841399a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:24.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.721+0000 7f1189974700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f118405c930 0x7f11840ac060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:24.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.721+0000 7f1189974700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f11840b4670 0x7f11840adf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:24.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.721+0000 7f1189974700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11840b64f0 con 0x7f118405c020 2022-08-01T19:43:24.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.721+0000 7f1189974700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11840ac550 con 0x7f118405c930 2022-08-01T19:43:24.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.721+0000 7f1189974700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11840b6b00 con 0x7f11840b4670 2022-08-01T19:43:24.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.721+0000 7f1182ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f118405c020 0x7f11841399a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:24.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.721+0000 7f1182ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f118405c020 0x7f11841399a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44634/0 (socket says 172.21.15.17:44634) 2022-08-01T19:43:24.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.721+0000 7f1182ffd700 1 -- 172.21.15.17:0/3457357138 learned_addr learned my addr 172.21.15.17:0/3457357138 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:24.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.721+0000 7f11827fc700 1 --2- 172.21.15.17:0/3457357138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f118405c930 0x7f11840ac060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:24.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.721+0000 7f11837fe700 1 --2- 172.21.15.17:0/3457357138 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f11840b4670 0x7f11840adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:24.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.721+0000 7f1182ffd700 1 -- 172.21.15.17:0/3457357138 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f11840b4670 msgr2=0x7f11840adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:24.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.721+0000 7f1182ffd700 1 --2- 172.21.15.17:0/3457357138 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f11840b4670 0x7f11840adf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:24.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.721+0000 7f1182ffd700 1 -- 172.21.15.17:0/3457357138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f118405c930 msgr2=0x7f11840ac060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:24.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.721+0000 7f1182ffd700 1 --2- 172.21.15.17:0/3457357138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f118405c930 0x7f11840ac060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:24.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.721+0000 7f1182ffd700 1 -- 172.21.15.17:0/3457357138 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117400b180 con 0x7f118405c020 2022-08-01T19:43:24.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.721+0000 7f1182ffd700 1 --2- 172.21.15.17:0/3457357138 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f118405c020 0x7f11841399a0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 rx=0x7f11840ad520 tx=0x7f1178007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:24.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.721+0000 7f1173fff700 1 -- 172.21.15.17:0/3457357138 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f117800cdb0 con 0x7f118405c020 2022-08-01T19:43:24.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.721+0000 7f1173fff700 1 -- 172.21.15.17:0/3457357138 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11780070d0 con 0x7f118405c020 2022-08-01T19:43:24.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.721+0000 7f1189974700 1 -- 172.21.15.17:0/3457357138 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f117c009470 con 0x7f118405c020 2022-08-01T19:43:24.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.721+0000 7f1173fff700 1 -- 172.21.15.17:0/3457357138 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1178007430 con 0x7f118405c020 2022-08-01T19:43:24.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.721+0000 7f1189974700 1 -- 172.21.15.17:0/3457357138 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11840b6d30 con 0x7f118405c020 2022-08-01T19:43:24.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.725+0000 7f1189974700 1 -- 172.21.15.17:0/3457357138 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11840b7170 con 0x7f118405c020 2022-08-01T19:43:24.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.725+0000 7f1173fff700 1 -- 172.21.15.17:0/3457357138 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f117801a070 con 0x7f118405c020 2022-08-01T19:43:24.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.725+0000 7f1173fff700 1 --2- 172.21.15.17:0/3457357138 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1160053aa0 0x7f1160055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:24.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.725+0000 7f1173fff700 1 -- 172.21.15.17:0/3457357138 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f117806b1c0 con 0x7f118405c020 2022-08-01T19:43:24.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.725+0000 7f11827fc700 1 --2- 172.21.15.17:0/3457357138 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1160053aa0 0x7f1160055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:24.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.729+0000 7f11827fc700 1 --2- 172.21.15.17:0/3457357138 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1160053aa0 0x7f1160055f60 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f117400b000 tx=0x7f1174006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:24.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.733+0000 7f1173fff700 1 -- 172.21.15.17:0/3457357138 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1178041f30 con 0x7f118405c020 2022-08-01T19:43:24.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:24.857+0000 7f1189974700 1 -- 172.21.15.17:0/3457357138 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f11840673a0 con 0x7f118405c020 2022-08-01T19:43:25.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.577+0000 7f1173fff700 1 -- 172.21.15.17:0/3457357138 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v139) v1 ==== 185+0+0 (secure 0 0 0) 0x7f1178045550 con 0x7f118405c020 2022-08-01T19:43:25.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.577+0000 7f1189974700 1 -- 172.21.15.17:0/3457357138 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1160053aa0 msgr2=0x7f1160055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:25.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.577+0000 7f1189974700 1 --2- 172.21.15.17:0/3457357138 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1160053aa0 0x7f1160055f60 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f117400b000 tx=0x7f1174006df0).stop 2022-08-01T19:43:25.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.577+0000 7f1189974700 1 -- 172.21.15.17:0/3457357138 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f118405c020 msgr2=0x7f11841399a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:25.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.577+0000 7f1189974700 1 --2- 172.21.15.17:0/3457357138 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f118405c020 0x7f11841399a0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 rx=0x7f11840ad520 tx=0x7f1178007aa0).stop 2022-08-01T19:43:25.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.577+0000 7f1189974700 1 -- 172.21.15.17:0/3457357138 shutdown_connections 2022-08-01T19:43:25.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.577+0000 7f1189974700 1 --2- 172.21.15.17:0/3457357138 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1160053aa0 0x7f1160055f60 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:25.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.577+0000 7f1189974700 1 --2- 172.21.15.17:0/3457357138 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f11840b4670 0x7f11840adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:25.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.577+0000 7f1189974700 1 --2- 172.21.15.17:0/3457357138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f118405c930 0x7f11840ac060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:25.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.577+0000 7f1189974700 1 --2- 172.21.15.17:0/3457357138 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f118405c020 0x7f11841399a0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:25.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.577+0000 7f1189974700 1 -- 172.21.15.17:0/3457357138 >> 172.21.15.17:0/3457357138 conn(0x7f11840b7900 msgr2=0x7f118413b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:25.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.581+0000 7f1189974700 1 -- 172.21.15.17:0/3457357138 shutdown_connections 2022-08-01T19:43:25.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.581+0000 7f1189974700 1 -- 172.21.15.17:0/3457357138 wait complete. 2022-08-01T19:43:25.587 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' removed 2022-08-01T19:43:25.599 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-08-01T19:43:25.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.713+0000 7f4e2e155700 1 Processor -- start 2022-08-01T19:43:25.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.713+0000 7f4e2e155700 1 -- start start 2022-08-01T19:43:25.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.717+0000 7f4e2e155700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f4e280b6590 0x7f4e280b6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:25.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.717+0000 7f4e2e155700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4e280b2620 0x7f4e280b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:25.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.717+0000 7f4e2e155700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e281475b0 con 0x7f4e2805f960 2022-08-01T19:43:25.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.717+0000 7f4e2e155700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e280b16b0 con 0x7f4e280b6590 2022-08-01T19:43:25.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.717+0000 7f4e2e155700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e280b3020 con 0x7f4e280b2620 2022-08-01T19:43:25.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.717+0000 7f4e2c952700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4e280b2620 0x7f4e280b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:25.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.717+0000 7f4e2d153700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f4e280b6590 0x7f4e280b6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:25.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.717+0000 7f4e2c952700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4e280b2620 0x7f4e280b3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:40742/0 (socket says 172.21.15.17:40742) 2022-08-01T19:43:25.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.717+0000 7f4e2c952700 1 -- 172.21.15.17:0/973127622 learned_addr learned my addr 172.21.15.17:0/973127622 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:25.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.717+0000 7f4e1ffff700 1 -- 172.21.15.17:0/973127622 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 186147420 0 0) 0x7f4e281475b0 con 0x7f4e2805f960 2022-08-01T19:43:25.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.717+0000 7f4e1ffff700 1 -- 172.21.15.17:0/973127622 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e0c003260 con 0x7f4e2805f960 2022-08-01T19:43:25.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.717+0000 7f4e2c952700 1 -- 172.21.15.17:0/973127622 >> v2:172.21.15.77:3300/0 conn(0x7f4e280b6590 msgr2=0x7f4e280b6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:25.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.717+0000 7f4e2c952700 1 --2- 172.21.15.17:0/973127622 >> v2:172.21.15.77:3300/0 conn(0x7f4e280b6590 0x7f4e280b6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:25.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.717+0000 7f4e2c952700 1 -- 172.21.15.17:0/973127622 >> v1:172.21.15.77:6789/0 conn(0x7f4e2805f960 legacy=0x7f4e280b0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:25.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.717+0000 7f4e2c952700 1 -- 172.21.15.17:0/973127622 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e280b32c0 con 0x7f4e280b2620 2022-08-01T19:43:25.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.717+0000 7f4e2d153700 1 --2- 172.21.15.17:0/973127622 >> v2:172.21.15.77:3300/0 conn(0x7f4e280b6590 0x7f4e280b6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:43:25.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.717+0000 7f4e2c952700 1 --2- 172.21.15.17:0/973127622 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4e280b2620 0x7f4e280b3a00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f4e24002f70 tx=0x7f4e24006eb0).ready entity=mon.2 client_cookie=58a6de8c041f5a00 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:25.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.717+0000 7f4e1ffff700 1 -- 172.21.15.17:0/973127622 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e2400d5c0 con 0x7f4e280b2620 2022-08-01T19:43:25.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.717+0000 7f4e1ffff700 1 -- 172.21.15.17:0/973127622 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e24000e90 con 0x7f4e280b2620 2022-08-01T19:43:25.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.717+0000 7f4e1ffff700 1 -- 172.21.15.17:0/973127622 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e2400d950 con 0x7f4e280b2620 2022-08-01T19:43:25.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.717+0000 7f4e2e155700 1 -- 172.21.15.17:0/973127622 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4e280b2620 msgr2=0x7f4e280b3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:25.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.717+0000 7f4e2e155700 1 --2- 172.21.15.17:0/973127622 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4e280b2620 0x7f4e280b3a00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f4e24002f70 tx=0x7f4e24006eb0).stop 2022-08-01T19:43:25.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.717+0000 7f4e2e155700 1 -- 172.21.15.17:0/973127622 shutdown_connections 2022-08-01T19:43:25.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.717+0000 7f4e2e155700 1 --2- 172.21.15.17:0/973127622 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4e280b2620 0x7f4e280b3a00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:25.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.717+0000 7f4e2e155700 1 --2- 172.21.15.17:0/973127622 >> v2:172.21.15.77:3300/0 conn(0x7f4e280b6590 0x7f4e280b6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:25.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.717+0000 7f4e2e155700 1 -- 172.21.15.17:0/973127622 >> 172.21.15.17:0/973127622 conn(0x7f4e28063800 msgr2=0x7f4e28063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:25.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.717+0000 7f4e2e155700 1 -- 172.21.15.17:0/973127622 shutdown_connections 2022-08-01T19:43:25.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.717+0000 7f4e2e155700 1 -- 172.21.15.17:0/973127622 wait complete. 2022-08-01T19:43:25.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.721+0000 7f4e2e155700 1 Processor -- start 2022-08-01T19:43:25.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.721+0000 7f4e2e155700 1 -- start start 2022-08-01T19:43:25.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.721+0000 7f4e2e155700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4e2805f960 0x7f4e28141f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:25.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.721+0000 7f4e2e155700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e280b2620 0x7f4e28140c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:25.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.721+0000 7f4e2e155700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e280b6590 0x7f4e28144be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:25.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.721+0000 7f4e2e155700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e281414b0 con 0x7f4e280b2620 2022-08-01T19:43:25.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.721+0000 7f4e2e155700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e281416e0 con 0x7f4e280b6590 2022-08-01T19:43:25.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.721+0000 7f4e2e155700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e28153180 con 0x7f4e2805f960 2022-08-01T19:43:25.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.721+0000 7f4e2c952700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e280b2620 0x7f4e28140c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:25.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.721+0000 7f4e2c952700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e280b2620 0x7f4e28140c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44650/0 (socket says 172.21.15.17:44650) 2022-08-01T19:43:25.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.721+0000 7f4e2c952700 1 -- 172.21.15.17:0/1197756019 learned_addr learned my addr 172.21.15.17:0/1197756019 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:25.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.721+0000 7f4e2d153700 1 --2- 172.21.15.17:0/1197756019 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4e2805f960 0x7f4e28141f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:25.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.721+0000 7f4e2d954700 1 --2- 172.21.15.17:0/1197756019 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e280b6590 0x7f4e28144be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:25.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.721+0000 7f4e2c952700 1 -- 172.21.15.17:0/1197756019 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4e2805f960 msgr2=0x7f4e28141f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:25.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.721+0000 7f4e2c952700 1 --2- 172.21.15.17:0/1197756019 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4e2805f960 0x7f4e28141f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:25.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.721+0000 7f4e2c952700 1 -- 172.21.15.17:0/1197756019 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e280b6590 msgr2=0x7f4e28144be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:25.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.721+0000 7f4e2c952700 1 --2- 172.21.15.17:0/1197756019 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e280b6590 0x7f4e28144be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:25.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.721+0000 7f4e2c952700 1 -- 172.21.15.17:0/1197756019 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e1800b180 con 0x7f4e280b2620 2022-08-01T19:43:25.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.721+0000 7f4e2c952700 1 --2- 172.21.15.17:0/1197756019 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e280b2620 0x7f4e28140c10 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 rx=0x7f4e24002f70 tx=0x7f4e24015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:25.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.721+0000 7f4e1e7fc700 1 -- 172.21.15.17:0/1197756019 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e2400de20 con 0x7f4e280b2620 2022-08-01T19:43:25.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.721+0000 7f4e1e7fc700 1 -- 172.21.15.17:0/1197756019 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e24011b40 con 0x7f4e280b2620 2022-08-01T19:43:25.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.721+0000 7f4e2e155700 1 -- 172.21.15.17:0/1197756019 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e2400b1e0 con 0x7f4e280b2620 2022-08-01T19:43:25.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.721+0000 7f4e1e7fc700 1 -- 172.21.15.17:0/1197756019 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e24011ea0 con 0x7f4e280b2620 2022-08-01T19:43:25.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.725+0000 7f4e2e155700 1 -- 172.21.15.17:0/1197756019 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e28141100 con 0x7f4e280b2620 2022-08-01T19:43:25.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.725+0000 7f4e2e155700 1 -- 172.21.15.17:0/1197756019 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e280574e0 con 0x7f4e280b2620 2022-08-01T19:43:25.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.725+0000 7f4e1e7fc700 1 -- 172.21.15.17:0/1197756019 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4e24025030 con 0x7f4e280b2620 2022-08-01T19:43:25.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.725+0000 7f4e1e7fc700 1 --2- 172.21.15.17:0/1197756019 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4e10053a70 0x7f4e10055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:25.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.725+0000 7f4e1e7fc700 1 -- 172.21.15.17:0/1197756019 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f4e2406cfc0 con 0x7f4e280b2620 2022-08-01T19:43:25.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.725+0000 7f4e2d153700 1 --2- 172.21.15.17:0/1197756019 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4e10053a70 0x7f4e10055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:25.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.725+0000 7f4e2d153700 1 --2- 172.21.15.17:0/1197756019 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4e10053a70 0x7f4e10055f30 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7f4e1800b000 tx=0x7f4e18006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:25.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.733+0000 7f4e1e7fc700 1 -- 172.21.15.17:0/1197756019 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4e24043ee0 con 0x7f4e280b2620 2022-08-01T19:43:25.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:25.861+0000 7f4e2e155700 1 -- 172.21.15.17:0/1197756019 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4e28069a70 con 0x7f4e280b2620 2022-08-01T19:43:26.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:26.577+0000 7f4e1e7fc700 1 -- 172.21.15.17:0/1197756019 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v140) v1 ==== 173+0+0 (secure 0 0 0) 0x7f4e24047500 con 0x7f4e280b2620 2022-08-01T19:43:26.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:26.585+0000 7f4e2e155700 1 -- 172.21.15.17:0/1197756019 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4e10053a70 msgr2=0x7f4e10055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:26.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:26.585+0000 7f4e2e155700 1 --2- 172.21.15.17:0/1197756019 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4e10053a70 0x7f4e10055f30 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7f4e1800b000 tx=0x7f4e18006df0).stop 2022-08-01T19:43:26.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:26.585+0000 7f4e2e155700 1 -- 172.21.15.17:0/1197756019 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e280b2620 msgr2=0x7f4e28140c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:26.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:26.585+0000 7f4e2e155700 1 --2- 172.21.15.17:0/1197756019 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e280b2620 0x7f4e28140c10 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 rx=0x7f4e24002f70 tx=0x7f4e24015000).stop 2022-08-01T19:43:26.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:26.585+0000 7f4e2e155700 1 -- 172.21.15.17:0/1197756019 shutdown_connections 2022-08-01T19:43:26.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:26.585+0000 7f4e2e155700 1 --2- 172.21.15.17:0/1197756019 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4e10053a70 0x7f4e10055f30 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:26.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:26.585+0000 7f4e2e155700 1 --2- 172.21.15.17:0/1197756019 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e280b6590 0x7f4e28144be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:26.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:26.585+0000 7f4e2e155700 1 --2- 172.21.15.17:0/1197756019 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e280b2620 0x7f4e28140c10 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:26.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:26.585+0000 7f4e2e155700 1 --2- 172.21.15.17:0/1197756019 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4e2805f960 0x7f4e28141f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:26.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:26.585+0000 7f4e2e155700 1 -- 172.21.15.17:0/1197756019 >> 172.21.15.17:0/1197756019 conn(0x7f4e28063800 msgr2=0x7f4e280b44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:26.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:26.585+0000 7f4e2e155700 1 -- 172.21.15.17:0/1197756019 shutdown_connections 2022-08-01T19:43:26.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:26.585+0000 7f4e2e155700 1 -- 172.21.15.17:0/1197756019 wait complete. 2022-08-01T19:43:26.592 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' removed 2022-08-01T19:43:26.607 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-08-01T19:43:26.611 DEBUG:teuthology.orchestra.run.smithi077:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-08-01T19:43:26.614 DEBUG:teuthology.orchestra.run.smithi085:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-08-01T19:43:26.619 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-08-01T19:43:26.637 INFO:teuthology.orchestra.run.smithi017.stdout:[global] 2022-08-01T19:43:26.638 INFO:teuthology.orchestra.run.smithi017.stdout: chdir = "" 2022-08-01T19:43:26.638 INFO:teuthology.orchestra.run.smithi017.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-08-01T19:43:26.638 INFO:teuthology.orchestra.run.smithi017.stdout: auth supported = cephx 2022-08-01T19:43:26.639 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:26.639 INFO:teuthology.orchestra.run.smithi017.stdout: filestore xattr use omap = true 2022-08-01T19:43:26.639 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:26.639 INFO:teuthology.orchestra.run.smithi017.stdout: mon clock drift allowed = 1.000 2022-08-01T19:43:26.640 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:26.640 INFO:teuthology.orchestra.run.smithi017.stdout: osd crush chooseleaf type = 0 2022-08-01T19:43:26.640 INFO:teuthology.orchestra.run.smithi017.stdout: auth debug = true 2022-08-01T19:43:26.640 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:26.641 INFO:teuthology.orchestra.run.smithi017.stdout: ms die on old message = true 2022-08-01T19:43:26.641 INFO:teuthology.orchestra.run.smithi017.stdout: ms die on bug = true 2022-08-01T19:43:26.641 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:26.641 INFO:teuthology.orchestra.run.smithi017.stdout: mon max pg per osd = 10000 # >= luminous 2022-08-01T19:43:26.642 INFO:teuthology.orchestra.run.smithi017.stdout: mon pg warn max object skew = 0 2022-08-01T19:43:26.642 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:26.642 INFO:teuthology.orchestra.run.smithi017.stdout: # disable pg_autoscaler by default for new pools 2022-08-01T19:43:26.643 INFO:teuthology.orchestra.run.smithi017.stdout: osd_pool_default_pg_autoscale_mode = off 2022-08-01T19:43:26.643 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:26.643 INFO:teuthology.orchestra.run.smithi017.stdout: osd pool default size = 2 2022-08-01T19:43:26.644 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:26.644 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd allow primary affinity = true 2022-08-01T19:43:26.644 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd allow pg remap = true 2022-08-01T19:43:26.644 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on legacy crush tunables = false 2022-08-01T19:43:26.645 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on crush straw calc version zero = false 2022-08-01T19:43:26.645 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on no sortbitwise = false 2022-08-01T19:43:26.645 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on osd down out interval zero = false 2022-08-01T19:43:26.645 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on too few osds = false 2022-08-01T19:43:26.646 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-08-01T19:43:26.646 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_pool_no_redundancy = false 2022-08-01T19:43:26.646 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:26.647 INFO:teuthology.orchestra.run.smithi017.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-08-01T19:43:26.647 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:26.647 INFO:teuthology.orchestra.run.smithi017.stdout: osd default data pool replay window = 5 2022-08-01T19:43:26.647 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:26.648 INFO:teuthology.orchestra.run.smithi017.stdout: mon allow pool delete = true 2022-08-01T19:43:26.648 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:26.648 INFO:teuthology.orchestra.run.smithi017.stdout: mon cluster log file level = debug 2022-08-01T19:43:26.648 INFO:teuthology.orchestra.run.smithi017.stdout: debug asserts on shutdown = true 2022-08-01T19:43:26.648 INFO:teuthology.orchestra.run.smithi017.stdout: mon health detail to clog = false 2022-08-01T19:43:26.649 INFO:teuthology.orchestra.run.smithi017.stdout: mon host = "172.21.15.17,172.21.15.77,[v2:172.21.15.77:3301,v1:172.21.15.77:6790]" 2022-08-01T19:43:26.649 INFO:teuthology.orchestra.run.smithi017.stdout: fsid = be659963-3cf3-4374-b3c4-cbe717b6bac2 2022-08-01T19:43:26.649 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:26.649 INFO:teuthology.orchestra.run.smithi017.stdout:[osd] 2022-08-01T19:43:26.650 INFO:teuthology.orchestra.run.smithi017.stdout: osd journal size = 100 2022-08-01T19:43:26.650 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:26.650 INFO:teuthology.orchestra.run.smithi017.stdout: osd scrub load threshold = 5.0 2022-08-01T19:43:26.651 INFO:teuthology.orchestra.run.smithi017.stdout: osd scrub max interval = 600 2022-08-01T19:43:26.651 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:26.651 INFO:teuthology.orchestra.run.smithi017.stdout: osd recover clone overlap = true 2022-08-01T19:43:26.651 INFO:teuthology.orchestra.run.smithi017.stdout: osd recovery max chunk = 1048576 2022-08-01T19:43:26.652 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:26.652 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug shutdown = true 2022-08-01T19:43:26.652 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug op order = true 2022-08-01T19:43:26.652 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug verify stray on activate = true 2022-08-01T19:43:26.653 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:26.653 INFO:teuthology.orchestra.run.smithi017.stdout: osd open classes on start = true 2022-08-01T19:43:26.654 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug pg log writeout = true 2022-08-01T19:43:26.654 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:26.654 INFO:teuthology.orchestra.run.smithi017.stdout: osd deep scrub update digest min age = 30 2022-08-01T19:43:26.654 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:26.655 INFO:teuthology.orchestra.run.smithi017.stdout: osd map max advance = 10 2022-08-01T19:43:26.655 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:26.655 INFO:teuthology.orchestra.run.smithi017.stdout: journal zero on create = true 2022-08-01T19:43:26.655 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:26.656 INFO:teuthology.orchestra.run.smithi017.stdout: filestore ondisk finisher threads = 3 2022-08-01T19:43:26.656 INFO:teuthology.orchestra.run.smithi017.stdout: filestore apply finisher threads = 3 2022-08-01T19:43:26.656 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:26.656 INFO:teuthology.orchestra.run.smithi017.stdout: bdev debug aio = true 2022-08-01T19:43:26.657 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug misdirected ops = true 2022-08-01T19:43:26.657 INFO:teuthology.orchestra.run.smithi017.stdout: bluestore block size = 96636764160 2022-08-01T19:43:26.657 INFO:teuthology.orchestra.run.smithi017.stdout: bluestore compression mode = aggressive 2022-08-01T19:43:26.658 INFO:teuthology.orchestra.run.smithi017.stdout: bluestore fsck on mount = True 2022-08-01T19:43:26.658 INFO:teuthology.orchestra.run.smithi017.stdout: debug bluefs = 20 2022-08-01T19:43:26.659 INFO:teuthology.orchestra.run.smithi017.stdout: debug bluestore = 20 2022-08-01T19:43:26.659 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2022-08-01T19:43:26.659 INFO:teuthology.orchestra.run.smithi017.stdout: debug osd = 20 2022-08-01T19:43:26.659 INFO:teuthology.orchestra.run.smithi017.stdout: debug rocksdb = 10 2022-08-01T19:43:26.660 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd backfillfull_ratio = 0.85 2022-08-01T19:43:26.660 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd full ratio = 0.9 2022-08-01T19:43:26.660 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd nearfull ratio = 0.8 2022-08-01T19:43:26.660 INFO:teuthology.orchestra.run.smithi017.stdout: osd failsafe full ratio = 0.95 2022-08-01T19:43:26.661 INFO:teuthology.orchestra.run.smithi017.stdout: osd objectstore = bluestore 2022-08-01T19:43:26.661 INFO:teuthology.orchestra.run.smithi017.stdout: osd op complaint time = 180 2022-08-01T19:43:26.661 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:26.661 INFO:teuthology.orchestra.run.smithi017.stdout:[mgr] 2022-08-01T19:43:26.662 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2022-08-01T19:43:26.662 INFO:teuthology.orchestra.run.smithi017.stdout: debug mgr = 20 2022-08-01T19:43:26.662 INFO:teuthology.orchestra.run.smithi017.stdout: debug mon = 20 2022-08-01T19:43:26.662 INFO:teuthology.orchestra.run.smithi017.stdout: debug auth = 20 2022-08-01T19:43:26.663 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min pgs per osd = 4 2022-08-01T19:43:26.663 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min bytes per osd = 10 2022-08-01T19:43:26.663 INFO:teuthology.orchestra.run.smithi017.stdout: mgr/telemetry/nag = false 2022-08-01T19:43:26.663 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:26.664 INFO:teuthology.orchestra.run.smithi017.stdout:[mon] 2022-08-01T19:43:26.664 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2022-08-01T19:43:26.665 INFO:teuthology.orchestra.run.smithi017.stdout: debug mon = 20 2022-08-01T19:43:26.665 INFO:teuthology.orchestra.run.smithi017.stdout: debug paxos = 20 2022-08-01T19:43:26.665 INFO:teuthology.orchestra.run.smithi017.stdout: debug auth = 20 2022-08-01T19:43:26.665 INFO:teuthology.orchestra.run.smithi017.stdout: mon data avail warn = 5 2022-08-01T19:43:26.666 INFO:teuthology.orchestra.run.smithi017.stdout: mon mgr mkfs grace = 240 2022-08-01T19:43:26.666 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min pgs per osd = 4 2022-08-01T19:43:26.666 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd reporter subtree level = osd 2022-08-01T19:43:26.667 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd prime pg temp = true 2022-08-01T19:43:26.667 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min bytes per osd = 10 2022-08-01T19:43:26.667 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:26.667 INFO:teuthology.orchestra.run.smithi017.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-08-01T19:43:26.667 INFO:teuthology.orchestra.run.smithi017.stdout: auth mon ticket ttl = 660 # 11m 2022-08-01T19:43:26.668 INFO:teuthology.orchestra.run.smithi017.stdout: auth service ticket ttl = 240 # 4m 2022-08-01T19:43:26.668 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:26.668 INFO:teuthology.orchestra.run.smithi017.stdout: # don't complain about insecure global_id in the test suite 2022-08-01T19:43:26.668 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-08-01T19:43:26.669 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-08-01T19:43:26.670 INFO:teuthology.orchestra.run.smithi017.stdout: mon op complaint time = 120 2022-08-01T19:43:26.671 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:26.671 INFO:teuthology.orchestra.run.smithi017.stdout:[client] 2022-08-01T19:43:26.671 INFO:teuthology.orchestra.run.smithi017.stdout: rgw cache enabled = true 2022-08-01T19:43:26.671 INFO:teuthology.orchestra.run.smithi017.stdout: rgw enable ops log = true 2022-08-01T19:43:26.672 INFO:teuthology.orchestra.run.smithi017.stdout: rgw enable usage log = true 2022-08-01T19:43:26.672 INFO:teuthology.orchestra.run.smithi017.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-08-01T19:43:26.672 INFO:teuthology.orchestra.run.smithi017.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-08-01T19:43:26.672 INFO:teuthology.orchestra.run.smithi017.stdout: client mount timeout = 600 2022-08-01T19:43:26.673 INFO:teuthology.orchestra.run.smithi017.stdout: debug client = 20 2022-08-01T19:43:26.673 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2022-08-01T19:43:26.674 INFO:teuthology.orchestra.run.smithi017.stdout: rados mon op timeout = 15m 2022-08-01T19:43:26.674 INFO:teuthology.orchestra.run.smithi017.stdout: rados osd op timeout = 15m 2022-08-01T19:43:26.674 INFO:teuthology.orchestra.run.smithi017.stdout:[mon.a] 2022-08-01T19:43:26.675 INFO:teuthology.orchestra.run.smithi017.stdout:[mon.b] 2022-08-01T19:43:26.675 INFO:teuthology.orchestra.run.smithi017.stdout:[mon.c] 2022-08-01T19:43:26.675 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.a] 2022-08-01T19:43:26.676 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.c] 2022-08-01T19:43:26.676 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.b] 2022-08-01T19:43:26.676 INFO:teuthology.orchestra.run.smithi017.stdout:[mds] 2022-08-01T19:43:26.677 INFO:teuthology.orchestra.run.smithi017.stdout: debug mds = 20 2022-08-01T19:43:26.677 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2022-08-01T19:43:26.677 INFO:teuthology.orchestra.run.smithi017.stdout: mds bal frag = True 2022-08-01T19:43:26.678 INFO:teuthology.orchestra.run.smithi017.stdout: mds bal fragment size max = 10000 2022-08-01T19:43:26.678 INFO:teuthology.orchestra.run.smithi017.stdout: mds bal merge size = 5 2022-08-01T19:43:26.678 INFO:teuthology.orchestra.run.smithi017.stdout: mds bal split bits = 3 2022-08-01T19:43:26.679 INFO:teuthology.orchestra.run.smithi017.stdout: mds bal split size = 100 2022-08-01T19:43:26.679 INFO:teuthology.orchestra.run.smithi017.stdout: mds debug frag = True 2022-08-01T19:43:26.679 INFO:teuthology.orchestra.run.smithi017.stdout: mds debug scatterstat = True 2022-08-01T19:43:26.679 INFO:teuthology.orchestra.run.smithi017.stdout: mds op complaint time = 180 2022-08-01T19:43:26.679 INFO:teuthology.orchestra.run.smithi017.stdout: mds verify scatter = True 2022-08-01T19:43:26.680 INFO:teuthology.orchestra.run.smithi017.stdout: osd op complaint time = 180 2022-08-01T19:43:26.680 INFO:teuthology.orchestra.run.smithi017.stdout: rados mon op timeout = 15m 2022-08-01T19:43:26.681 INFO:teuthology.orchestra.run.smithi017.stdout: rados osd op timeout = 15m 2022-08-01T19:43:26.684 INFO:teuthology.orchestra.run.smithi190.stdout:[global] 2022-08-01T19:43:26.685 INFO:teuthology.orchestra.run.smithi190.stdout: chdir = "" 2022-08-01T19:43:26.685 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-08-01T19:43:26.685 INFO:teuthology.orchestra.run.smithi190.stdout: auth supported = cephx 2022-08-01T19:43:26.685 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:26.686 INFO:teuthology.orchestra.run.smithi190.stdout: filestore xattr use omap = true 2022-08-01T19:43:26.686 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:26.686 INFO:teuthology.orchestra.run.smithi190.stdout: mon clock drift allowed = 1.000 2022-08-01T19:43:26.686 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:26.687 INFO:teuthology.orchestra.run.smithi190.stdout: osd crush chooseleaf type = 0 2022-08-01T19:43:26.687 INFO:teuthology.orchestra.run.smithi190.stdout: auth debug = true 2022-08-01T19:43:26.687 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:26.687 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on old message = true 2022-08-01T19:43:26.688 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on bug = true 2022-08-01T19:43:26.688 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:26.688 INFO:teuthology.orchestra.run.smithi190.stdout: mon max pg per osd = 10000 # >= luminous 2022-08-01T19:43:26.688 INFO:teuthology.orchestra.run.smithi190.stdout: mon pg warn max object skew = 0 2022-08-01T19:43:26.689 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:26.689 INFO:teuthology.orchestra.run.smithi190.stdout: # disable pg_autoscaler by default for new pools 2022-08-01T19:43:26.689 INFO:teuthology.orchestra.run.smithi190.stdout: osd_pool_default_pg_autoscale_mode = off 2022-08-01T19:43:26.689 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:26.690 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default size = 2 2022-08-01T19:43:26.690 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:26.690 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow primary affinity = true 2022-08-01T19:43:26.690 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow pg remap = true 2022-08-01T19:43:26.691 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on legacy crush tunables = false 2022-08-01T19:43:26.691 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on crush straw calc version zero = false 2022-08-01T19:43:26.691 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on no sortbitwise = false 2022-08-01T19:43:26.691 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on osd down out interval zero = false 2022-08-01T19:43:26.692 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on too few osds = false 2022-08-01T19:43:26.692 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-08-01T19:43:26.692 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_no_redundancy = false 2022-08-01T19:43:26.692 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:26.693 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-08-01T19:43:26.693 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:26.693 INFO:teuthology.orchestra.run.smithi190.stdout: osd default data pool replay window = 5 2022-08-01T19:43:26.693 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:26.694 INFO:teuthology.orchestra.run.smithi190.stdout: mon allow pool delete = true 2022-08-01T19:43:26.694 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:26.694 INFO:teuthology.orchestra.run.smithi190.stdout: mon cluster log file level = debug 2022-08-01T19:43:26.694 INFO:teuthology.orchestra.run.smithi190.stdout: debug asserts on shutdown = true 2022-08-01T19:43:26.695 INFO:teuthology.orchestra.run.smithi190.stdout: mon health detail to clog = false 2022-08-01T19:43:26.695 INFO:teuthology.orchestra.run.smithi190.stdout: mon host = "172.21.15.17,172.21.15.77,[v2:172.21.15.77:3301,v1:172.21.15.77:6790]" 2022-08-01T19:43:26.695 INFO:teuthology.orchestra.run.smithi190.stdout: fsid = be659963-3cf3-4374-b3c4-cbe717b6bac2 2022-08-01T19:43:26.695 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:26.696 INFO:teuthology.orchestra.run.smithi190.stdout:[osd] 2022-08-01T19:43:26.696 INFO:teuthology.orchestra.run.smithi190.stdout: osd journal size = 100 2022-08-01T19:43:26.696 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:26.696 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub load threshold = 5.0 2022-08-01T19:43:26.696 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub max interval = 600 2022-08-01T19:43:26.697 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:26.697 INFO:teuthology.orchestra.run.smithi190.stdout: osd recover clone overlap = true 2022-08-01T19:43:26.697 INFO:teuthology.orchestra.run.smithi190.stdout: osd recovery max chunk = 1048576 2022-08-01T19:43:26.697 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:26.698 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug shutdown = true 2022-08-01T19:43:26.698 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug op order = true 2022-08-01T19:43:26.698 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug verify stray on activate = true 2022-08-01T19:43:26.698 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:26.699 INFO:teuthology.orchestra.run.smithi190.stdout: osd open classes on start = true 2022-08-01T19:43:26.699 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug pg log writeout = true 2022-08-01T19:43:26.699 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:26.699 INFO:teuthology.orchestra.run.smithi190.stdout: osd deep scrub update digest min age = 30 2022-08-01T19:43:26.700 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:26.700 INFO:teuthology.orchestra.run.smithi190.stdout: osd map max advance = 10 2022-08-01T19:43:26.700 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:26.700 INFO:teuthology.orchestra.run.smithi190.stdout: journal zero on create = true 2022-08-01T19:43:26.701 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:26.701 INFO:teuthology.orchestra.run.smithi190.stdout: filestore ondisk finisher threads = 3 2022-08-01T19:43:26.701 INFO:teuthology.orchestra.run.smithi190.stdout: filestore apply finisher threads = 3 2022-08-01T19:43:26.701 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:26.702 INFO:teuthology.orchestra.run.smithi190.stdout: bdev debug aio = true 2022-08-01T19:43:26.702 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug misdirected ops = true 2022-08-01T19:43:26.702 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore block size = 96636764160 2022-08-01T19:43:26.702 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore compression mode = aggressive 2022-08-01T19:43:26.703 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore fsck on mount = True 2022-08-01T19:43:26.703 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluefs = 20 2022-08-01T19:43:26.703 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluestore = 20 2022-08-01T19:43:26.704 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2022-08-01T19:43:26.704 INFO:teuthology.orchestra.run.smithi190.stdout: debug osd = 20 2022-08-01T19:43:26.705 INFO:teuthology.orchestra.run.smithi190.stdout: debug rocksdb = 10 2022-08-01T19:43:26.705 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd backfillfull_ratio = 0.85 2022-08-01T19:43:26.706 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd full ratio = 0.9 2022-08-01T19:43:26.706 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd nearfull ratio = 0.8 2022-08-01T19:43:26.707 INFO:teuthology.orchestra.run.smithi190.stdout: osd failsafe full ratio = 0.95 2022-08-01T19:43:26.707 INFO:teuthology.orchestra.run.smithi190.stdout: osd objectstore = bluestore 2022-08-01T19:43:26.708 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2022-08-01T19:43:26.708 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:26.709 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr] 2022-08-01T19:43:26.709 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2022-08-01T19:43:26.709 INFO:teuthology.orchestra.run.smithi190.stdout: debug mgr = 20 2022-08-01T19:43:26.710 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2022-08-01T19:43:26.710 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2022-08-01T19:43:26.711 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2022-08-01T19:43:26.711 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2022-08-01T19:43:26.711 INFO:teuthology.orchestra.run.smithi190.stdout: mgr/telemetry/nag = false 2022-08-01T19:43:26.712 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:26.712 INFO:teuthology.orchestra.run.smithi190.stdout:[mon] 2022-08-01T19:43:26.713 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2022-08-01T19:43:26.713 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2022-08-01T19:43:26.713 INFO:teuthology.orchestra.run.smithi190.stdout: debug paxos = 20 2022-08-01T19:43:26.714 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2022-08-01T19:43:26.714 INFO:teuthology.orchestra.run.smithi190.stdout: mon data avail warn = 5 2022-08-01T19:43:26.715 INFO:teuthology.orchestra.run.smithi190.stdout: mon mgr mkfs grace = 240 2022-08-01T19:43:26.715 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2022-08-01T19:43:26.716 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd reporter subtree level = osd 2022-08-01T19:43:26.717 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd prime pg temp = true 2022-08-01T19:43:26.717 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2022-08-01T19:43:26.718 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:26.719 INFO:teuthology.orchestra.run.smithi190.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-08-01T19:43:26.719 INFO:teuthology.orchestra.run.smithi190.stdout: auth mon ticket ttl = 660 # 11m 2022-08-01T19:43:26.720 INFO:teuthology.orchestra.run.smithi190.stdout: auth service ticket ttl = 240 # 4m 2022-08-01T19:43:26.720 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:26.720 INFO:teuthology.orchestra.run.smithi190.stdout: # don't complain about insecure global_id in the test suite 2022-08-01T19:43:26.721 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-08-01T19:43:26.722 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-08-01T19:43:26.722 INFO:teuthology.orchestra.run.smithi190.stdout: mon op complaint time = 120 2022-08-01T19:43:26.723 INFO:teuthology.orchestra.run.smithi190.stdout: 2022-08-01T19:43:26.723 INFO:teuthology.orchestra.run.smithi190.stdout:[client] 2022-08-01T19:43:26.724 INFO:teuthology.orchestra.run.smithi190.stdout: rgw cache enabled = true 2022-08-01T19:43:26.724 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable ops log = true 2022-08-01T19:43:26.725 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable usage log = true 2022-08-01T19:43:26.725 INFO:teuthology.orchestra.run.smithi190.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-08-01T19:43:26.726 INFO:teuthology.orchestra.run.smithi190.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-08-01T19:43:26.726 INFO:teuthology.orchestra.run.smithi190.stdout: client mount timeout = 600 2022-08-01T19:43:26.727 INFO:teuthology.orchestra.run.smithi190.stdout: debug client = 20 2022-08-01T19:43:26.728 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2022-08-01T19:43:26.728 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 15m 2022-08-01T19:43:26.729 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 15m 2022-08-01T19:43:26.729 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.a] 2022-08-01T19:43:26.730 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.b] 2022-08-01T19:43:26.730 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.c] 2022-08-01T19:43:26.731 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.a] 2022-08-01T19:43:26.731 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.c] 2022-08-01T19:43:26.731 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.b] 2022-08-01T19:43:26.732 INFO:teuthology.orchestra.run.smithi190.stdout:[mds] 2022-08-01T19:43:26.732 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds = 20 2022-08-01T19:43:26.733 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2022-08-01T19:43:26.733 INFO:teuthology.orchestra.run.smithi190.stdout: mds bal frag = True 2022-08-01T19:43:26.734 INFO:teuthology.orchestra.run.smithi190.stdout: mds bal fragment size max = 10000 2022-08-01T19:43:26.734 INFO:teuthology.orchestra.run.smithi190.stdout: mds bal merge size = 5 2022-08-01T19:43:26.735 INFO:teuthology.orchestra.run.smithi190.stdout: mds bal split bits = 3 2022-08-01T19:43:26.735 INFO:teuthology.orchestra.run.smithi190.stdout: mds bal split size = 100 2022-08-01T19:43:26.736 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug frag = True 2022-08-01T19:43:26.736 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug scatterstat = True 2022-08-01T19:43:26.736 INFO:teuthology.orchestra.run.smithi190.stdout: mds op complaint time = 180 2022-08-01T19:43:26.737 INFO:teuthology.orchestra.run.smithi190.stdout: mds verify scatter = True 2022-08-01T19:43:26.737 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2022-08-01T19:43:26.738 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 15m 2022-08-01T19:43:26.738 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 15m 2022-08-01T19:43:26.739 INFO:teuthology.orchestra.run.smithi085.stdout:[global] 2022-08-01T19:43:26.739 INFO:teuthology.orchestra.run.smithi085.stdout: chdir = "" 2022-08-01T19:43:26.740 INFO:teuthology.orchestra.run.smithi085.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-08-01T19:43:26.740 INFO:teuthology.orchestra.run.smithi085.stdout: auth supported = cephx 2022-08-01T19:43:26.741 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:26.741 INFO:teuthology.orchestra.run.smithi085.stdout: filestore xattr use omap = true 2022-08-01T19:43:26.742 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:26.742 INFO:teuthology.orchestra.run.smithi085.stdout: mon clock drift allowed = 1.000 2022-08-01T19:43:26.743 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:26.743 INFO:teuthology.orchestra.run.smithi085.stdout: osd crush chooseleaf type = 0 2022-08-01T19:43:26.744 INFO:teuthology.orchestra.run.smithi085.stdout: auth debug = true 2022-08-01T19:43:26.744 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:26.745 INFO:teuthology.orchestra.run.smithi085.stdout: ms die on old message = true 2022-08-01T19:43:26.745 INFO:teuthology.orchestra.run.smithi085.stdout: ms die on bug = true 2022-08-01T19:43:26.746 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:26.747 INFO:teuthology.orchestra.run.smithi085.stdout: mon max pg per osd = 10000 # >= luminous 2022-08-01T19:43:26.747 INFO:teuthology.orchestra.run.smithi085.stdout: mon pg warn max object skew = 0 2022-08-01T19:43:26.748 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:26.748 INFO:teuthology.orchestra.run.smithi085.stdout: # disable pg_autoscaler by default for new pools 2022-08-01T19:43:26.749 INFO:teuthology.orchestra.run.smithi085.stdout: osd_pool_default_pg_autoscale_mode = off 2022-08-01T19:43:26.749 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:26.749 INFO:teuthology.orchestra.run.smithi085.stdout: osd pool default size = 2 2022-08-01T19:43:26.750 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:26.751 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd allow primary affinity = true 2022-08-01T19:43:26.751 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd allow pg remap = true 2022-08-01T19:43:26.752 INFO:teuthology.orchestra.run.smithi085.stdout: mon warn on legacy crush tunables = false 2022-08-01T19:43:26.752 INFO:teuthology.orchestra.run.smithi085.stdout: mon warn on crush straw calc version zero = false 2022-08-01T19:43:26.753 INFO:teuthology.orchestra.run.smithi085.stdout: mon warn on no sortbitwise = false 2022-08-01T19:43:26.759 INFO:teuthology.orchestra.run.smithi085.stdout: mon warn on osd down out interval zero = false 2022-08-01T19:43:26.760 INFO:teuthology.orchestra.run.smithi085.stdout: mon warn on too few osds = false 2022-08-01T19:43:26.760 INFO:teuthology.orchestra.run.smithi085.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-08-01T19:43:26.761 INFO:teuthology.orchestra.run.smithi085.stdout: mon_warn_on_pool_no_redundancy = false 2022-08-01T19:43:26.762 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:26.762 INFO:teuthology.orchestra.run.smithi085.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-08-01T19:43:26.763 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:26.763 INFO:teuthology.orchestra.run.smithi085.stdout: osd default data pool replay window = 5 2022-08-01T19:43:26.766 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:26.767 INFO:teuthology.orchestra.run.smithi085.stdout: mon allow pool delete = true 2022-08-01T19:43:26.767 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:26.767 INFO:teuthology.orchestra.run.smithi085.stdout: mon cluster log file level = debug 2022-08-01T19:43:26.768 INFO:teuthology.orchestra.run.smithi085.stdout: debug asserts on shutdown = true 2022-08-01T19:43:26.768 INFO:teuthology.orchestra.run.smithi085.stdout: mon health detail to clog = false 2022-08-01T19:43:26.769 INFO:teuthology.orchestra.run.smithi085.stdout: mon host = "172.21.15.17,172.21.15.77,[v2:172.21.15.77:3301,v1:172.21.15.77:6790]" 2022-08-01T19:43:26.769 INFO:teuthology.orchestra.run.smithi085.stdout: fsid = be659963-3cf3-4374-b3c4-cbe717b6bac2 2022-08-01T19:43:26.769 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:26.770 INFO:teuthology.orchestra.run.smithi085.stdout:[osd] 2022-08-01T19:43:26.770 INFO:teuthology.orchestra.run.smithi085.stdout: osd journal size = 100 2022-08-01T19:43:26.771 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:26.772 INFO:teuthology.orchestra.run.smithi085.stdout: osd scrub load threshold = 5.0 2022-08-01T19:43:26.772 INFO:teuthology.orchestra.run.smithi085.stdout: osd scrub max interval = 600 2022-08-01T19:43:26.773 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:26.773 INFO:teuthology.orchestra.run.smithi085.stdout: osd recover clone overlap = true 2022-08-01T19:43:26.774 INFO:teuthology.orchestra.run.smithi085.stdout: osd recovery max chunk = 1048576 2022-08-01T19:43:26.783 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:26.783 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug shutdown = true 2022-08-01T19:43:26.783 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug op order = true 2022-08-01T19:43:26.784 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug verify stray on activate = true 2022-08-01T19:43:26.784 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:26.784 INFO:teuthology.orchestra.run.smithi085.stdout: osd open classes on start = true 2022-08-01T19:43:26.784 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug pg log writeout = true 2022-08-01T19:43:26.784 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:26.785 INFO:teuthology.orchestra.run.smithi085.stdout: osd deep scrub update digest min age = 30 2022-08-01T19:43:26.785 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:26.785 INFO:teuthology.orchestra.run.smithi085.stdout: osd map max advance = 10 2022-08-01T19:43:26.786 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:26.786 INFO:teuthology.orchestra.run.smithi085.stdout: journal zero on create = true 2022-08-01T19:43:26.787 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:26.788 INFO:teuthology.orchestra.run.smithi085.stdout: filestore ondisk finisher threads = 3 2022-08-01T19:43:26.788 INFO:teuthology.orchestra.run.smithi085.stdout: filestore apply finisher threads = 3 2022-08-01T19:43:26.799 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:26.800 INFO:teuthology.orchestra.run.smithi085.stdout: bdev debug aio = true 2022-08-01T19:43:26.800 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug misdirected ops = true 2022-08-01T19:43:26.800 INFO:teuthology.orchestra.run.smithi085.stdout: bluestore block size = 96636764160 2022-08-01T19:43:26.800 INFO:teuthology.orchestra.run.smithi085.stdout: bluestore compression mode = aggressive 2022-08-01T19:43:26.801 INFO:teuthology.orchestra.run.smithi085.stdout: bluestore fsck on mount = True 2022-08-01T19:43:26.801 INFO:teuthology.orchestra.run.smithi085.stdout: debug bluefs = 20 2022-08-01T19:43:26.801 INFO:teuthology.orchestra.run.smithi085.stdout: debug bluestore = 20 2022-08-01T19:43:26.801 INFO:teuthology.orchestra.run.smithi085.stdout: debug ms = 1 2022-08-01T19:43:26.802 INFO:teuthology.orchestra.run.smithi085.stdout: debug osd = 20 2022-08-01T19:43:26.802 INFO:teuthology.orchestra.run.smithi085.stdout: debug rocksdb = 10 2022-08-01T19:43:26.802 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd backfillfull_ratio = 0.85 2022-08-01T19:43:26.802 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd full ratio = 0.9 2022-08-01T19:43:26.806 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd nearfull ratio = 0.8 2022-08-01T19:43:26.806 INFO:teuthology.orchestra.run.smithi085.stdout: osd failsafe full ratio = 0.95 2022-08-01T19:43:26.819 INFO:teuthology.orchestra.run.smithi085.stdout: osd objectstore = bluestore 2022-08-01T19:43:26.819 INFO:teuthology.orchestra.run.smithi085.stdout: osd op complaint time = 180 2022-08-01T19:43:26.820 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:26.820 INFO:teuthology.orchestra.run.smithi085.stdout:[mgr] 2022-08-01T19:43:26.821 INFO:teuthology.orchestra.run.smithi085.stdout: debug ms = 1 2022-08-01T19:43:26.821 INFO:teuthology.orchestra.run.smithi085.stdout: debug mgr = 20 2022-08-01T19:43:26.821 INFO:teuthology.orchestra.run.smithi085.stdout: debug mon = 20 2022-08-01T19:43:26.822 INFO:teuthology.orchestra.run.smithi085.stdout: debug auth = 20 2022-08-01T19:43:26.822 INFO:teuthology.orchestra.run.smithi085.stdout: mon reweight min pgs per osd = 4 2022-08-01T19:43:26.822 INFO:teuthology.orchestra.run.smithi085.stdout: mon reweight min bytes per osd = 10 2022-08-01T19:43:26.823 INFO:teuthology.orchestra.run.smithi085.stdout: mgr/telemetry/nag = false 2022-08-01T19:43:26.823 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:26.823 INFO:teuthology.orchestra.run.smithi085.stdout:[mon] 2022-08-01T19:43:26.824 INFO:teuthology.orchestra.run.smithi085.stdout: debug ms = 1 2022-08-01T19:43:26.824 INFO:teuthology.orchestra.run.smithi085.stdout: debug mon = 20 2022-08-01T19:43:26.825 INFO:teuthology.orchestra.run.smithi085.stdout: debug paxos = 20 2022-08-01T19:43:26.825 INFO:teuthology.orchestra.run.smithi085.stdout: debug auth = 20 2022-08-01T19:43:26.825 INFO:teuthology.orchestra.run.smithi085.stdout: mon data avail warn = 5 2022-08-01T19:43:26.826 INFO:teuthology.orchestra.run.smithi085.stdout: mon mgr mkfs grace = 240 2022-08-01T19:43:26.826 INFO:teuthology.orchestra.run.smithi085.stdout: mon reweight min pgs per osd = 4 2022-08-01T19:43:26.826 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd reporter subtree level = osd 2022-08-01T19:43:26.827 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd prime pg temp = true 2022-08-01T19:43:26.827 INFO:teuthology.orchestra.run.smithi085.stdout: mon reweight min bytes per osd = 10 2022-08-01T19:43:26.827 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:26.827 INFO:teuthology.orchestra.run.smithi085.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-08-01T19:43:26.828 INFO:teuthology.orchestra.run.smithi085.stdout: auth mon ticket ttl = 660 # 11m 2022-08-01T19:43:26.828 INFO:teuthology.orchestra.run.smithi085.stdout: auth service ticket ttl = 240 # 4m 2022-08-01T19:43:26.828 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:26.829 INFO:teuthology.orchestra.run.smithi085.stdout: # don't complain about insecure global_id in the test suite 2022-08-01T19:43:26.829 INFO:teuthology.orchestra.run.smithi085.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-08-01T19:43:26.829 INFO:teuthology.orchestra.run.smithi085.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-08-01T19:43:26.830 INFO:teuthology.orchestra.run.smithi085.stdout: mon op complaint time = 120 2022-08-01T19:43:26.830 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-08-01T19:43:26.831 INFO:teuthology.orchestra.run.smithi085.stdout:[client] 2022-08-01T19:43:26.831 INFO:teuthology.orchestra.run.smithi085.stdout: rgw cache enabled = true 2022-08-01T19:43:26.832 INFO:teuthology.orchestra.run.smithi085.stdout: rgw enable ops log = true 2022-08-01T19:43:26.837 INFO:teuthology.orchestra.run.smithi085.stdout: rgw enable usage log = true 2022-08-01T19:43:26.837 INFO:teuthology.orchestra.run.smithi085.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-08-01T19:43:26.838 INFO:teuthology.orchestra.run.smithi085.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-08-01T19:43:26.838 INFO:teuthology.orchestra.run.smithi085.stdout: client mount timeout = 600 2022-08-01T19:43:26.839 INFO:teuthology.orchestra.run.smithi085.stdout: debug client = 20 2022-08-01T19:43:26.839 INFO:teuthology.orchestra.run.smithi085.stdout: debug ms = 1 2022-08-01T19:43:26.840 INFO:teuthology.orchestra.run.smithi085.stdout: rados mon op timeout = 15m 2022-08-01T19:43:26.840 INFO:teuthology.orchestra.run.smithi085.stdout: rados osd op timeout = 15m 2022-08-01T19:43:26.841 INFO:teuthology.orchestra.run.smithi085.stdout:[mon.a] 2022-08-01T19:43:26.841 INFO:teuthology.orchestra.run.smithi085.stdout:[mon.b] 2022-08-01T19:43:26.841 INFO:teuthology.orchestra.run.smithi085.stdout:[mon.c] 2022-08-01T19:43:26.842 INFO:teuthology.orchestra.run.smithi085.stdout:[mds.a] 2022-08-01T19:43:26.842 INFO:teuthology.orchestra.run.smithi085.stdout:[mds.c] 2022-08-01T19:43:26.843 INFO:teuthology.orchestra.run.smithi085.stdout:[mds.b] 2022-08-01T19:43:26.843 INFO:teuthology.orchestra.run.smithi085.stdout:[mds] 2022-08-01T19:43:26.843 INFO:teuthology.orchestra.run.smithi085.stdout: debug mds = 20 2022-08-01T19:43:26.844 INFO:teuthology.orchestra.run.smithi085.stdout: debug ms = 1 2022-08-01T19:43:26.844 INFO:teuthology.orchestra.run.smithi085.stdout: mds bal frag = True 2022-08-01T19:43:26.845 INFO:teuthology.orchestra.run.smithi085.stdout: mds bal fragment size max = 10000 2022-08-01T19:43:26.845 INFO:teuthology.orchestra.run.smithi085.stdout: mds bal merge size = 5 2022-08-01T19:43:26.845 INFO:teuthology.orchestra.run.smithi085.stdout: mds bal split bits = 3 2022-08-01T19:43:26.846 INFO:teuthology.orchestra.run.smithi085.stdout: mds bal split size = 100 2022-08-01T19:43:26.846 INFO:teuthology.orchestra.run.smithi085.stdout: mds debug frag = True 2022-08-01T19:43:26.847 INFO:teuthology.orchestra.run.smithi085.stdout: mds debug scatterstat = True 2022-08-01T19:43:26.847 INFO:teuthology.orchestra.run.smithi085.stdout: mds op complaint time = 180 2022-08-01T19:43:26.848 INFO:teuthology.orchestra.run.smithi085.stdout: mds verify scatter = True 2022-08-01T19:43:26.848 INFO:teuthology.orchestra.run.smithi085.stdout: osd op complaint time = 180 2022-08-01T19:43:26.848 INFO:teuthology.orchestra.run.smithi085.stdout: rados mon op timeout = 15m 2022-08-01T19:43:26.849 INFO:teuthology.orchestra.run.smithi085.stdout: rados osd op timeout = 15m 2022-08-01T19:43:26.849 INFO:teuthology.orchestra.run.smithi077.stdout:[global] 2022-08-01T19:43:26.849 INFO:teuthology.orchestra.run.smithi077.stdout: chdir = "" 2022-08-01T19:43:26.850 INFO:teuthology.orchestra.run.smithi077.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-08-01T19:43:26.850 INFO:teuthology.orchestra.run.smithi077.stdout: auth supported = cephx 2022-08-01T19:43:26.850 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:26.851 INFO:teuthology.orchestra.run.smithi077.stdout: filestore xattr use omap = true 2022-08-01T19:43:26.851 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:26.851 INFO:teuthology.orchestra.run.smithi077.stdout: mon clock drift allowed = 1.000 2022-08-01T19:43:26.852 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:26.852 INFO:teuthology.orchestra.run.smithi077.stdout: osd crush chooseleaf type = 0 2022-08-01T19:43:26.853 INFO:teuthology.orchestra.run.smithi077.stdout: auth debug = true 2022-08-01T19:43:26.854 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:26.854 INFO:teuthology.orchestra.run.smithi077.stdout: ms die on old message = true 2022-08-01T19:43:26.854 INFO:teuthology.orchestra.run.smithi077.stdout: ms die on bug = true 2022-08-01T19:43:26.854 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:26.855 INFO:teuthology.orchestra.run.smithi077.stdout: mon max pg per osd = 10000 # >= luminous 2022-08-01T19:43:26.855 INFO:teuthology.orchestra.run.smithi077.stdout: mon pg warn max object skew = 0 2022-08-01T19:43:26.855 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:26.855 INFO:teuthology.orchestra.run.smithi077.stdout: # disable pg_autoscaler by default for new pools 2022-08-01T19:43:26.856 INFO:teuthology.orchestra.run.smithi077.stdout: osd_pool_default_pg_autoscale_mode = off 2022-08-01T19:43:26.856 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:26.856 INFO:teuthology.orchestra.run.smithi077.stdout: osd pool default size = 2 2022-08-01T19:43:26.856 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:26.857 INFO:teuthology.orchestra.run.smithi077.stdout: mon osd allow primary affinity = true 2022-08-01T19:43:26.857 INFO:teuthology.orchestra.run.smithi077.stdout: mon osd allow pg remap = true 2022-08-01T19:43:26.857 INFO:teuthology.orchestra.run.smithi077.stdout: mon warn on legacy crush tunables = false 2022-08-01T19:43:26.857 INFO:teuthology.orchestra.run.smithi077.stdout: mon warn on crush straw calc version zero = false 2022-08-01T19:43:26.858 INFO:teuthology.orchestra.run.smithi077.stdout: mon warn on no sortbitwise = false 2022-08-01T19:43:26.858 INFO:teuthology.orchestra.run.smithi077.stdout: mon warn on osd down out interval zero = false 2022-08-01T19:43:26.858 INFO:teuthology.orchestra.run.smithi077.stdout: mon warn on too few osds = false 2022-08-01T19:43:26.858 INFO:teuthology.orchestra.run.smithi077.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-08-01T19:43:26.859 INFO:teuthology.orchestra.run.smithi077.stdout: mon_warn_on_pool_no_redundancy = false 2022-08-01T19:43:26.859 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:26.859 INFO:teuthology.orchestra.run.smithi077.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-08-01T19:43:26.859 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:26.859 INFO:teuthology.orchestra.run.smithi077.stdout: osd default data pool replay window = 5 2022-08-01T19:43:26.860 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:26.860 INFO:teuthology.orchestra.run.smithi077.stdout: mon allow pool delete = true 2022-08-01T19:43:26.860 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:26.860 INFO:teuthology.orchestra.run.smithi077.stdout: mon cluster log file level = debug 2022-08-01T19:43:26.861 INFO:teuthology.orchestra.run.smithi077.stdout: debug asserts on shutdown = true 2022-08-01T19:43:26.861 INFO:teuthology.orchestra.run.smithi077.stdout: mon health detail to clog = false 2022-08-01T19:43:26.861 INFO:teuthology.orchestra.run.smithi077.stdout: mon host = "172.21.15.17,172.21.15.77,[v2:172.21.15.77:3301,v1:172.21.15.77:6790]" 2022-08-01T19:43:26.861 INFO:teuthology.orchestra.run.smithi077.stdout: fsid = be659963-3cf3-4374-b3c4-cbe717b6bac2 2022-08-01T19:43:26.862 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:26.862 INFO:teuthology.orchestra.run.smithi077.stdout:[osd] 2022-08-01T19:43:26.862 INFO:teuthology.orchestra.run.smithi077.stdout: osd journal size = 100 2022-08-01T19:43:26.862 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:26.863 INFO:teuthology.orchestra.run.smithi077.stdout: osd scrub load threshold = 5.0 2022-08-01T19:43:26.863 INFO:teuthology.orchestra.run.smithi077.stdout: osd scrub max interval = 600 2022-08-01T19:43:26.863 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:26.863 INFO:teuthology.orchestra.run.smithi077.stdout: osd recover clone overlap = true 2022-08-01T19:43:26.863 INFO:teuthology.orchestra.run.smithi077.stdout: osd recovery max chunk = 1048576 2022-08-01T19:43:26.864 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:26.864 INFO:teuthology.orchestra.run.smithi077.stdout: osd debug shutdown = true 2022-08-01T19:43:26.864 INFO:teuthology.orchestra.run.smithi077.stdout: osd debug op order = true 2022-08-01T19:43:26.864 INFO:teuthology.orchestra.run.smithi077.stdout: osd debug verify stray on activate = true 2022-08-01T19:43:26.865 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:26.865 INFO:teuthology.orchestra.run.smithi077.stdout: osd open classes on start = true 2022-08-01T19:43:26.865 INFO:teuthology.orchestra.run.smithi077.stdout: osd debug pg log writeout = true 2022-08-01T19:43:26.865 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:26.865 INFO:teuthology.orchestra.run.smithi077.stdout: osd deep scrub update digest min age = 30 2022-08-01T19:43:26.866 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:26.866 INFO:teuthology.orchestra.run.smithi077.stdout: osd map max advance = 10 2022-08-01T19:43:26.866 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:26.866 INFO:teuthology.orchestra.run.smithi077.stdout: journal zero on create = true 2022-08-01T19:43:26.867 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:26.867 INFO:teuthology.orchestra.run.smithi077.stdout: filestore ondisk finisher threads = 3 2022-08-01T19:43:26.867 INFO:teuthology.orchestra.run.smithi077.stdout: filestore apply finisher threads = 3 2022-08-01T19:43:26.867 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:26.868 INFO:teuthology.orchestra.run.smithi077.stdout: bdev debug aio = true 2022-08-01T19:43:26.868 INFO:teuthology.orchestra.run.smithi077.stdout: osd debug misdirected ops = true 2022-08-01T19:43:26.868 INFO:teuthology.orchestra.run.smithi077.stdout: bluestore block size = 96636764160 2022-08-01T19:43:26.868 INFO:teuthology.orchestra.run.smithi077.stdout: bluestore compression mode = aggressive 2022-08-01T19:43:26.868 INFO:teuthology.orchestra.run.smithi077.stdout: bluestore fsck on mount = True 2022-08-01T19:43:26.869 INFO:teuthology.orchestra.run.smithi077.stdout: debug bluefs = 20 2022-08-01T19:43:26.869 INFO:teuthology.orchestra.run.smithi077.stdout: debug bluestore = 20 2022-08-01T19:43:26.869 INFO:teuthology.orchestra.run.smithi077.stdout: debug ms = 1 2022-08-01T19:43:26.869 INFO:teuthology.orchestra.run.smithi077.stdout: debug osd = 20 2022-08-01T19:43:26.870 INFO:teuthology.orchestra.run.smithi077.stdout: debug rocksdb = 10 2022-08-01T19:43:26.870 INFO:teuthology.orchestra.run.smithi077.stdout: mon osd backfillfull_ratio = 0.85 2022-08-01T19:43:26.870 INFO:teuthology.orchestra.run.smithi077.stdout: mon osd full ratio = 0.9 2022-08-01T19:43:26.871 INFO:teuthology.orchestra.run.smithi077.stdout: mon osd nearfull ratio = 0.8 2022-08-01T19:43:26.871 INFO:teuthology.orchestra.run.smithi077.stdout: osd failsafe full ratio = 0.95 2022-08-01T19:43:26.871 INFO:teuthology.orchestra.run.smithi077.stdout: osd objectstore = bluestore 2022-08-01T19:43:26.871 INFO:teuthology.orchestra.run.smithi077.stdout: osd op complaint time = 180 2022-08-01T19:43:26.872 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:26.872 INFO:teuthology.orchestra.run.smithi077.stdout:[mgr] 2022-08-01T19:43:26.872 INFO:teuthology.orchestra.run.smithi077.stdout: debug ms = 1 2022-08-01T19:43:26.872 INFO:teuthology.orchestra.run.smithi077.stdout: debug mgr = 20 2022-08-01T19:43:26.873 INFO:teuthology.orchestra.run.smithi077.stdout: debug mon = 20 2022-08-01T19:43:26.873 INFO:teuthology.orchestra.run.smithi077.stdout: debug auth = 20 2022-08-01T19:43:26.873 INFO:teuthology.orchestra.run.smithi077.stdout: mon reweight min pgs per osd = 4 2022-08-01T19:43:26.874 INFO:teuthology.orchestra.run.smithi077.stdout: mon reweight min bytes per osd = 10 2022-08-01T19:43:26.874 INFO:teuthology.orchestra.run.smithi077.stdout: mgr/telemetry/nag = false 2022-08-01T19:43:26.874 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:26.874 INFO:teuthology.orchestra.run.smithi077.stdout:[mon] 2022-08-01T19:43:26.875 INFO:teuthology.orchestra.run.smithi077.stdout: debug ms = 1 2022-08-01T19:43:26.875 INFO:teuthology.orchestra.run.smithi077.stdout: debug mon = 20 2022-08-01T19:43:26.875 INFO:teuthology.orchestra.run.smithi077.stdout: debug paxos = 20 2022-08-01T19:43:26.875 INFO:teuthology.orchestra.run.smithi077.stdout: debug auth = 20 2022-08-01T19:43:26.875 INFO:teuthology.orchestra.run.smithi077.stdout: mon data avail warn = 5 2022-08-01T19:43:26.876 INFO:teuthology.orchestra.run.smithi077.stdout: mon mgr mkfs grace = 240 2022-08-01T19:43:26.876 INFO:teuthology.orchestra.run.smithi077.stdout: mon reweight min pgs per osd = 4 2022-08-01T19:43:26.876 INFO:teuthology.orchestra.run.smithi077.stdout: mon osd reporter subtree level = osd 2022-08-01T19:43:26.876 INFO:teuthology.orchestra.run.smithi077.stdout: mon osd prime pg temp = true 2022-08-01T19:43:26.877 INFO:teuthology.orchestra.run.smithi077.stdout: mon reweight min bytes per osd = 10 2022-08-01T19:43:26.877 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:26.877 INFO:teuthology.orchestra.run.smithi077.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-08-01T19:43:26.878 INFO:teuthology.orchestra.run.smithi077.stdout: auth mon ticket ttl = 660 # 11m 2022-08-01T19:43:26.878 INFO:teuthology.orchestra.run.smithi077.stdout: auth service ticket ttl = 240 # 4m 2022-08-01T19:43:26.878 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:26.878 INFO:teuthology.orchestra.run.smithi077.stdout: # don't complain about insecure global_id in the test suite 2022-08-01T19:43:26.878 INFO:teuthology.orchestra.run.smithi077.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-08-01T19:43:26.879 INFO:teuthology.orchestra.run.smithi077.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-08-01T19:43:26.879 INFO:teuthology.orchestra.run.smithi077.stdout: mon op complaint time = 120 2022-08-01T19:43:26.879 INFO:teuthology.orchestra.run.smithi077.stdout: 2022-08-01T19:43:26.879 INFO:teuthology.orchestra.run.smithi077.stdout:[client] 2022-08-01T19:43:26.880 INFO:teuthology.orchestra.run.smithi077.stdout: rgw cache enabled = true 2022-08-01T19:43:26.880 INFO:teuthology.orchestra.run.smithi077.stdout: rgw enable ops log = true 2022-08-01T19:43:26.880 INFO:teuthology.orchestra.run.smithi077.stdout: rgw enable usage log = true 2022-08-01T19:43:26.880 INFO:teuthology.orchestra.run.smithi077.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-08-01T19:43:26.880 INFO:teuthology.orchestra.run.smithi077.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-08-01T19:43:26.881 INFO:teuthology.orchestra.run.smithi077.stdout: client mount timeout = 600 2022-08-01T19:43:26.881 INFO:teuthology.orchestra.run.smithi077.stdout: debug client = 20 2022-08-01T19:43:26.881 INFO:teuthology.orchestra.run.smithi077.stdout: debug ms = 1 2022-08-01T19:43:26.881 INFO:teuthology.orchestra.run.smithi077.stdout: rados mon op timeout = 15m 2022-08-01T19:43:26.881 INFO:teuthology.orchestra.run.smithi077.stdout: rados osd op timeout = 15m 2022-08-01T19:43:26.882 INFO:teuthology.orchestra.run.smithi077.stdout:[mon.a] 2022-08-01T19:43:26.883 INFO:teuthology.orchestra.run.smithi077.stdout:[mon.b] 2022-08-01T19:43:26.883 INFO:teuthology.orchestra.run.smithi077.stdout:[mon.c] 2022-08-01T19:43:26.883 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.a] 2022-08-01T19:43:26.883 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.c] 2022-08-01T19:43:26.884 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.b] 2022-08-01T19:43:26.884 INFO:teuthology.orchestra.run.smithi077.stdout:[mds] 2022-08-01T19:43:26.884 INFO:teuthology.orchestra.run.smithi077.stdout: debug mds = 20 2022-08-01T19:43:26.884 INFO:teuthology.orchestra.run.smithi077.stdout: debug ms = 1 2022-08-01T19:43:26.885 INFO:teuthology.orchestra.run.smithi077.stdout: mds bal frag = True 2022-08-01T19:43:26.885 INFO:teuthology.orchestra.run.smithi077.stdout: mds bal fragment size max = 10000 2022-08-01T19:43:26.885 INFO:teuthology.orchestra.run.smithi077.stdout: mds bal merge size = 5 2022-08-01T19:43:26.885 INFO:teuthology.orchestra.run.smithi077.stdout: mds bal split bits = 3 2022-08-01T19:43:26.885 INFO:teuthology.orchestra.run.smithi077.stdout: mds bal split size = 100 2022-08-01T19:43:26.886 INFO:teuthology.orchestra.run.smithi077.stdout: mds debug frag = True 2022-08-01T19:43:26.886 INFO:teuthology.orchestra.run.smithi077.stdout: mds debug scatterstat = True 2022-08-01T19:43:26.886 INFO:teuthology.orchestra.run.smithi077.stdout: mds op complaint time = 180 2022-08-01T19:43:26.887 INFO:teuthology.orchestra.run.smithi077.stdout: mds verify scatter = True 2022-08-01T19:43:26.887 INFO:teuthology.orchestra.run.smithi077.stdout: osd op complaint time = 180 2022-08-01T19:43:26.887 INFO:teuthology.orchestra.run.smithi077.stdout: rados mon op timeout = 15m 2022-08-01T19:43:26.888 INFO:teuthology.orchestra.run.smithi077.stdout: rados osd op timeout = 15m 2022-08-01T19:43:26.888 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-08-01T19:43:27.014 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.005+0000 7f67b48bb700 1 Processor -- start 2022-08-01T19:43:27.014 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.005+0000 7f67b48bb700 1 -- start start 2022-08-01T19:43:27.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.009+0000 7f67b48bb700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67b00b38a0 0x7f67b00b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:27.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.009+0000 7f67b48bb700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f67b00a9da0 0x7f67b00b1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:27.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.009+0000 7f67b48bb700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67b0143ab0 con 0x7f67b00b0740 2022-08-01T19:43:27.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.009+0000 7f67b48bb700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67b0143d50 con 0x7f67b00a9da0 2022-08-01T19:43:27.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.009+0000 7f67b48bb700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b0143ff0 con 0x7f67b00b38a0 2022-08-01T19:43:27.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.009+0000 7f67ae59c700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f67b00b0740 0x7f67b013d6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36850/0 (socket says 172.21.15.17:36850) 2022-08-01T19:43:27.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.009+0000 7f67ae59c700 1 -- 172.21.15.17:0/368529400 learned_addr learned my addr 172.21.15.17:0/368529400 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:27.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.009+0000 7f67a7fff700 1 --2- 172.21.15.17:0/368529400 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67b00b38a0 0x7f67b00b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:27.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.009+0000 7f67aed9d700 1 --2- 172.21.15.17:0/368529400 >> v2:172.21.15.77:3300/0 conn(0x7f67b00a9da0 0x7f67b00b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:27.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.009+0000 7f67add9b700 1 -- 172.21.15.17:0/368529400 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3415544749 0 0) 0x7f67b0143ab0 con 0x7f67b00b0740 2022-08-01T19:43:27.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.009+0000 7f67add9b700 1 -- 172.21.15.17:0/368529400 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6790003260 con 0x7f67b00b0740 2022-08-01T19:43:27.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.009+0000 7f67a7fff700 1 -- 172.21.15.17:0/368529400 >> v2:172.21.15.77:3300/0 conn(0x7f67b00a9da0 msgr2=0x7f67b00b1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:27.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.009+0000 7f67a7fff700 1 --2- 172.21.15.17:0/368529400 >> v2:172.21.15.77:3300/0 conn(0x7f67b00a9da0 0x7f67b00b1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:27.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.009+0000 7f67a7fff700 1 -- 172.21.15.17:0/368529400 >> v1:172.21.15.17:6789/0 conn(0x7f67b00b0740 legacy=0x7f67b013d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:27.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.009+0000 7f67a7fff700 1 -- 172.21.15.17:0/368529400 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b0144290 con 0x7f67b00b38a0 2022-08-01T19:43:27.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.009+0000 7f67a7fff700 1 --2- 172.21.15.17:0/368529400 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67b00b38a0 0x7f67b00b0b20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f6798002f70 tx=0x7f6798006df0).ready entity=mon.2 client_cookie=caa5f524006da8c3 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:27.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.009+0000 7f67add9b700 1 -- 172.21.15.17:0/368529400 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6798014070 con 0x7f67b00b38a0 2022-08-01T19:43:27.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.009+0000 7f67add9b700 1 -- 172.21.15.17:0/368529400 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6798000e90 con 0x7f67b00b38a0 2022-08-01T19:43:27.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.009+0000 7f67add9b700 1 -- 172.21.15.17:0/368529400 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f679800fa90 con 0x7f67b00b38a0 2022-08-01T19:43:27.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.009+0000 7f67b48bb700 1 -- 172.21.15.17:0/368529400 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67b00b38a0 msgr2=0x7f67b00b0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:27.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.009+0000 7f67b48bb700 1 --2- 172.21.15.17:0/368529400 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67b00b38a0 0x7f67b00b0b20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f6798002f70 tx=0x7f6798006df0).stop 2022-08-01T19:43:27.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.009+0000 7f67b48bb700 1 -- 172.21.15.17:0/368529400 shutdown_connections 2022-08-01T19:43:27.040 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.009+0000 7f67b48bb700 1 --2- 172.21.15.17:0/368529400 >> v2:172.21.15.77:3300/0 conn(0x7f67b00a9da0 0x7f67b00b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:27.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.009+0000 7f67b48bb700 1 --2- 172.21.15.17:0/368529400 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67b00b38a0 0x7f67b00b0b20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:27.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.009+0000 7f67b48bb700 1 -- 172.21.15.17:0/368529400 >> 172.21.15.17:0/368529400 conn(0x7f67b0060950 msgr2=0x7f67b0060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:27.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.009+0000 7f67b48bb700 1 -- 172.21.15.17:0/368529400 shutdown_connections 2022-08-01T19:43:27.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.009+0000 7f67b48bb700 1 -- 172.21.15.17:0/368529400 wait complete. 2022-08-01T19:43:27.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.013+0000 7f67b48bb700 1 Processor -- start 2022-08-01T19:43:27.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.013+0000 7f67b48bb700 1 -- start start 2022-08-01T19:43:27.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.013+0000 7f67b48bb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67b00a9da0 0x7f67b0142790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:27.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.013+0000 7f67b48bb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67b00b0740 0x7f67b00a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:27.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.013+0000 7f67b48bb700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67b00b38a0 0x7f67b00a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:27.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.013+0000 7f67b48bb700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b009f530 con 0x7f67b00b0740 2022-08-01T19:43:27.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.013+0000 7f67b48bb700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b01526f0 con 0x7f67b00a9da0 2022-08-01T19:43:27.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.013+0000 7f67b48bb700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b00ad590 con 0x7f67b00b38a0 2022-08-01T19:43:27.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.013+0000 7f67a7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67b00b0740 0x7f67b00a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:27.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.013+0000 7f67a7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67b00b0740 0x7f67b00a0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44664/0 (socket says 172.21.15.17:44664) 2022-08-01T19:43:27.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.013+0000 7f67a7fff700 1 -- 172.21.15.17:0/3203423588 learned_addr learned my addr 172.21.15.17:0/3203423588 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:27.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.013+0000 7f67ae59c700 1 --2- 172.21.15.17:0/3203423588 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67b00a9da0 0x7f67b0142790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:27.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.013+0000 7f67aed9d700 1 --2- 172.21.15.17:0/3203423588 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67b00b38a0 0x7f67b00a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:27.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.017+0000 7f67a7fff700 1 -- 172.21.15.17:0/3203423588 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67b00b38a0 msgr2=0x7f67b00a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:27.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.017+0000 7f67a7fff700 1 --2- 172.21.15.17:0/3203423588 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67b00b38a0 0x7f67b00a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:27.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.017+0000 7f67a7fff700 1 -- 172.21.15.17:0/3203423588 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67b00a9da0 msgr2=0x7f67b0142790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:27.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.017+0000 7f67a7fff700 1 --2- 172.21.15.17:0/3203423588 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67b00a9da0 0x7f67b0142790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:27.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.017+0000 7f67a7fff700 1 -- 172.21.15.17:0/3203423588 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f679800b180 con 0x7f67b00b0740 2022-08-01T19:43:27.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.017+0000 7f67a7fff700 1 --2- 172.21.15.17:0/3203423588 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67b00b0740 0x7f67b00a0840 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 rx=0x7f6798002f70 tx=0x7f6798016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:27.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.017+0000 7f67a77fe700 1 -- 172.21.15.17:0/3203423588 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f679800fa30 con 0x7f67b00b0740 2022-08-01T19:43:27.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.017+0000 7f67a77fe700 1 -- 172.21.15.17:0/3203423588 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6798006780 con 0x7f67b00b0740 2022-08-01T19:43:27.063 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.017+0000 7f67b48bb700 1 -- 172.21.15.17:0/3203423588 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b00adf80 con 0x7f67b00b0740 2022-08-01T19:43:27.063 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.017+0000 7f67a77fe700 1 -- 172.21.15.17:0/3203423588 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6798010d60 con 0x7f67b00b0740 2022-08-01T19:43:27.063 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.017+0000 7f67b48bb700 1 -- 172.21.15.17:0/3203423588 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b00a1220 con 0x7f67b00b0740 2022-08-01T19:43:27.064 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.017+0000 7f67b48bb700 1 -- 172.21.15.17:0/3203423588 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67b00ad300 con 0x7f67b00b0740 2022-08-01T19:43:27.064 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.017+0000 7f67a77fe700 1 -- 172.21.15.17:0/3203423588 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6798022020 con 0x7f67b00b0740 2022-08-01T19:43:27.064 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.017+0000 7f67a77fe700 1 --2- 172.21.15.17:0/3203423588 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6794053aa0 0x7f6794055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:27.064 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.017+0000 7f67a77fe700 1 -- 172.21.15.17:0/3203423588 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f679806d4d0 con 0x7f67b00b0740 2022-08-01T19:43:27.065 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.017+0000 7f67ae59c700 1 --2- 172.21.15.17:0/3203423588 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6794053aa0 0x7f6794055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:27.065 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.017+0000 7f67ae59c700 1 --2- 172.21.15.17:0/3203423588 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6794053aa0 0x7f6794055f60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f67a000d000 tx=0x7f67a0009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:27.065 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.025+0000 7f67a77fe700 1 -- 172.21.15.17:0/3203423588 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6798044460 con 0x7f67b00b0740 2022-08-01T19:43:27.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.153+0000 7f67b48bb700 1 -- 172.21.15.17:0/3203423588 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f67b00ad750 con 0x7f67b00b0740 2022-08-01T19:43:27.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.161+0000 7f67a77fe700 1 -- 172.21.15.17:0/3203423588 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6798047a80 con 0x7f67b00b0740 2022-08-01T19:43:27.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.165+0000 7f67b48bb700 1 -- 172.21.15.17:0/3203423588 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6794053aa0 msgr2=0x7f6794055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:27.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.165+0000 7f67b48bb700 1 --2- 172.21.15.17:0/3203423588 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6794053aa0 0x7f6794055f60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f67a000d000 tx=0x7f67a0009040).stop 2022-08-01T19:43:27.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.165+0000 7f67b48bb700 1 -- 172.21.15.17:0/3203423588 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67b00b0740 msgr2=0x7f67b00a0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:27.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.165+0000 7f67b48bb700 1 --2- 172.21.15.17:0/3203423588 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67b00b0740 0x7f67b00a0840 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 rx=0x7f6798002f70 tx=0x7f6798016000).stop 2022-08-01T19:43:27.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.165+0000 7f67b48bb700 1 -- 172.21.15.17:0/3203423588 shutdown_connections 2022-08-01T19:43:27.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.165+0000 7f67b48bb700 1 --2- 172.21.15.17:0/3203423588 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6794053aa0 0x7f6794055f60 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:27.169 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.165+0000 7f67b48bb700 1 --2- 172.21.15.17:0/3203423588 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f67b00b38a0 0x7f67b00a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:27.169 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.165+0000 7f67b48bb700 1 --2- 172.21.15.17:0/3203423588 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67b00b0740 0x7f67b00a0840 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:27.169 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.165+0000 7f67b48bb700 1 --2- 172.21.15.17:0/3203423588 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67b00a9da0 0x7f67b0142790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:27.169 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.165+0000 7f67b48bb700 1 -- 172.21.15.17:0/3203423588 >> 172.21.15.17:0/3203423588 conn(0x7f67b0060950 msgr2=0x7f67b013cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:27.184 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.165+0000 7f67b48bb700 1 -- 172.21.15.17:0/3203423588 shutdown_connections 2022-08-01T19:43:27.184 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.165+0000 7f67b48bb700 1 -- 172.21.15.17:0/3203423588 wait complete. 2022-08-01T19:43:27.186 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_getattr_caps' 2022-08-01T19:43:27.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.297+0000 7f09d68d0700 1 Processor -- start 2022-08-01T19:43:27.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.297+0000 7f09d68d0700 1 -- start start 2022-08-01T19:43:27.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.297+0000 7f09d68d0700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f09d00619b0 0x7f09d00b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:27.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.297+0000 7f09d68d0700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f09d00b6ee0 0x7f09d0143890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:27.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.297+0000 7f09d68d0700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09d00b5ed0 0x7f09d00b5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:27.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.297+0000 7f09d68d0700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09d0142f60 con 0x7f09d00b6ee0 2022-08-01T19:43:27.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.297+0000 7f09d68d0700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09d0061d90 con 0x7f09d00619b0 2022-08-01T19:43:27.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.297+0000 7f09d68d0700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d00ba5a0 con 0x7f09d00b5ed0 2022-08-01T19:43:27.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.297+0000 7f09d50cd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f09d00b6ee0 0x7f09d0143890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:27.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.297+0000 7f09d50cd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f09d00b6ee0 0x7f09d0143890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44672/0 (socket says 172.21.15.17:44672) 2022-08-01T19:43:27.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.297+0000 7f09d50cd700 1 -- 172.21.15.17:0/1005964147 learned_addr learned my addr 172.21.15.17:0/1005964147 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:27.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.297+0000 7f09d58ce700 1 --2- 172.21.15.17:0/1005964147 >> v2:172.21.15.77:3300/0 conn(0x7f09d00619b0 0x7f09d00b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:27.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.297+0000 7f09d60cf700 1 --2- 172.21.15.17:0/1005964147 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09d00b5ed0 0x7f09d00b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:27.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.297+0000 7f09d50cd700 1 -- 172.21.15.17:0/1005964147 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09d00b5ed0 msgr2=0x7f09d00b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:27.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.297+0000 7f09d50cd700 1 --2- 172.21.15.17:0/1005964147 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09d00b5ed0 0x7f09d00b5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:27.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.297+0000 7f09d50cd700 1 -- 172.21.15.17:0/1005964147 >> v2:172.21.15.77:3300/0 conn(0x7f09d00619b0 msgr2=0x7f09d00b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:27.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.297+0000 7f09d50cd700 1 --2- 172.21.15.17:0/1005964147 >> v2:172.21.15.77:3300/0 conn(0x7f09d00619b0 0x7f09d00b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:27.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.297+0000 7f09d50cd700 1 -- 172.21.15.17:0/1005964147 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d0143150 con 0x7f09d00b6ee0 2022-08-01T19:43:27.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.297+0000 7f09d50cd700 1 --2- 172.21.15.17:0/1005964147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09d00b6ee0 0x7f09d0143890 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 rx=0x7f09c0002f70 tx=0x7f09c0006eb0).ready entity=mon.0 client_cookie=b90c007421b68a2e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:27.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.297+0000 7f09d48cc700 1 -- 172.21.15.17:0/1005964147 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09c000d570 con 0x7f09d00b6ee0 2022-08-01T19:43:27.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.297+0000 7f09d48cc700 1 -- 172.21.15.17:0/1005964147 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c0000e90 con 0x7f09d00b6ee0 2022-08-01T19:43:27.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.297+0000 7f09d48cc700 1 -- 172.21.15.17:0/1005964147 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09c000d900 con 0x7f09d00b6ee0 2022-08-01T19:43:27.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.297+0000 7f09d68d0700 1 -- 172.21.15.17:0/1005964147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09d00b6ee0 msgr2=0x7f09d0143890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:27.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.297+0000 7f09d68d0700 1 --2- 172.21.15.17:0/1005964147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09d00b6ee0 0x7f09d0143890 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 rx=0x7f09c0002f70 tx=0x7f09c0006eb0).stop 2022-08-01T19:43:27.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.297+0000 7f09d68d0700 1 -- 172.21.15.17:0/1005964147 shutdown_connections 2022-08-01T19:43:27.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.297+0000 7f09d68d0700 1 --2- 172.21.15.17:0/1005964147 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09d00b5ed0 0x7f09d00b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:27.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.297+0000 7f09d68d0700 1 --2- 172.21.15.17:0/1005964147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09d00b6ee0 0x7f09d0143890 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:27.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.297+0000 7f09d68d0700 1 --2- 172.21.15.17:0/1005964147 >> v2:172.21.15.77:3300/0 conn(0x7f09d00619b0 0x7f09d00b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:27.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.297+0000 7f09d68d0700 1 -- 172.21.15.17:0/1005964147 >> 172.21.15.17:0/1005964147 conn(0x7f09d006af80 msgr2=0x7f09d005fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:27.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.297+0000 7f09d68d0700 1 -- 172.21.15.17:0/1005964147 shutdown_connections 2022-08-01T19:43:27.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.297+0000 7f09d68d0700 1 -- 172.21.15.17:0/1005964147 wait complete. 2022-08-01T19:43:27.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.301+0000 7f09d68d0700 1 Processor -- start 2022-08-01T19:43:27.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.301+0000 7f09d68d0700 1 -- start start 2022-08-01T19:43:27.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.301+0000 7f09d68d0700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09d00619b0 0x7f09d00b01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:27.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.301+0000 7f09d68d0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09d00b5ed0 0x7f09d013ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:27.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.301+0000 7f09d68d0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09d00b6ee0 0x7f09d013bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:27.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.301+0000 7f09d68d0700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d013f720 con 0x7f09d00b6ee0 2022-08-01T19:43:27.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.301+0000 7f09d68d0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d00b91a0 con 0x7f09d00b5ed0 2022-08-01T19:43:27.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.301+0000 7f09d68d0700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d00b9950 con 0x7f09d00619b0 2022-08-01T19:43:27.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.301+0000 7f09d60cf700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09d00b6ee0 0x7f09d013bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:27.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.301+0000 7f09d50cd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09d00b5ed0 0x7f09d013ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:27.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.301+0000 7f09d60cf700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09d00b6ee0 0x7f09d013bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44680/0 (socket says 172.21.15.17:44680) 2022-08-01T19:43:27.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.301+0000 7f09d58ce700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09d00619b0 0x7f09d00b01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:27.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.301+0000 7f09d60cf700 1 -- 172.21.15.17:0/159210088 learned_addr learned my addr 172.21.15.17:0/159210088 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:27.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.301+0000 7f09d50cd700 1 -- 172.21.15.17:0/159210088 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09d00619b0 msgr2=0x7f09d00b01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:27.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.301+0000 7f09d50cd700 1 --2- 172.21.15.17:0/159210088 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09d00619b0 0x7f09d00b01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:27.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.301+0000 7f09d50cd700 1 -- 172.21.15.17:0/159210088 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09d00b6ee0 msgr2=0x7f09d013bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:27.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.301+0000 7f09d50cd700 1 --2- 172.21.15.17:0/159210088 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09d00b6ee0 0x7f09d013bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:27.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.301+0000 7f09d50cd700 1 -- 172.21.15.17:0/159210088 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c000b1e0 con 0x7f09d00b5ed0 2022-08-01T19:43:27.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.301+0000 7f09d50cd700 1 --2- 172.21.15.17:0/159210088 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09d00b5ed0 0x7f09d013ba70 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f09c0002f70 tx=0x7f09c0015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:27.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.301+0000 7f09c6ffd700 1 -- 172.21.15.17:0/159210088 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09c000d910 con 0x7f09d00b5ed0 2022-08-01T19:43:27.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.301+0000 7f09c6ffd700 1 -- 172.21.15.17:0/159210088 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c00068b0 con 0x7f09d00b5ed0 2022-08-01T19:43:27.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.301+0000 7f09d68d0700 1 -- 172.21.15.17:0/159210088 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09d00b6b70 con 0x7f09d00b5ed0 2022-08-01T19:43:27.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.301+0000 7f09c6ffd700 1 -- 172.21.15.17:0/159210088 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09c001e650 con 0x7f09d00b5ed0 2022-08-01T19:43:27.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.305+0000 7f09d68d0700 1 -- 172.21.15.17:0/159210088 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d013c450 con 0x7f09d00b5ed0 2022-08-01T19:43:27.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.305+0000 7f09d68d0700 1 -- 172.21.15.17:0/159210088 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09d00b9310 con 0x7f09d00b5ed0 2022-08-01T19:43:27.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.305+0000 7f09c6ffd700 1 -- 172.21.15.17:0/159210088 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f09c001a040 con 0x7f09d00b5ed0 2022-08-01T19:43:27.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.305+0000 7f09c6ffd700 1 --2- 172.21.15.17:0/159210088 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f09b80539e0 0x7f09b8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:27.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.305+0000 7f09c6ffd700 1 -- 172.21.15.17:0/159210088 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f09c006d200 con 0x7f09d00b5ed0 2022-08-01T19:43:27.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.305+0000 7f09d58ce700 1 --2- 172.21.15.17:0/159210088 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f09b80539e0 0x7f09b8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:27.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.305+0000 7f09d58ce700 1 --2- 172.21.15.17:0/159210088 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f09b80539e0 0x7f09b8055ea0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f09cc00b000 tx=0x7f09cc006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:27.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.313+0000 7f09c6ffd700 1 -- 172.21.15.17:0/159210088 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f09c00441c0 con 0x7f09d00b5ed0 2022-08-01T19:43:27.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.485+0000 7f09d68d0700 1 -- 172.21.15.17:0/159210088 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]} v 0) v1 -- 0x7f09d00574e0 con 0x7f09d00b5ed0 2022-08-01T19:43:27.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.585+0000 7f09c6ffd700 1 -- 172.21.15.17:0/159210088 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]}]=0 v429) v1 ==== 128+0+0 (secure 0 0 0) 0x7f09c00477e0 con 0x7f09d00b5ed0 2022-08-01T19:43:27.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.589+0000 7f09d68d0700 1 -- 172.21.15.17:0/159210088 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f09b80539e0 msgr2=0x7f09b8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:27.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.589+0000 7f09d68d0700 1 --2- 172.21.15.17:0/159210088 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f09b80539e0 0x7f09b8055ea0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f09cc00b000 tx=0x7f09cc006df0).stop 2022-08-01T19:43:27.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.589+0000 7f09d68d0700 1 -- 172.21.15.17:0/159210088 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09d00b5ed0 msgr2=0x7f09d013ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:27.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.589+0000 7f09d68d0700 1 --2- 172.21.15.17:0/159210088 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09d00b5ed0 0x7f09d013ba70 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f09c0002f70 tx=0x7f09c0015000).stop 2022-08-01T19:43:27.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.589+0000 7f09d68d0700 1 -- 172.21.15.17:0/159210088 shutdown_connections 2022-08-01T19:43:27.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.589+0000 7f09d68d0700 1 --2- 172.21.15.17:0/159210088 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f09b80539e0 0x7f09b8055ea0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:27.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.589+0000 7f09d68d0700 1 --2- 172.21.15.17:0/159210088 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09d00b6ee0 0x7f09d013bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:27.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.589+0000 7f09d68d0700 1 --2- 172.21.15.17:0/159210088 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09d00b5ed0 0x7f09d013ba70 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:27.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.589+0000 7f09d68d0700 1 --2- 172.21.15.17:0/159210088 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09d00619b0 0x7f09d00b01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:27.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.589+0000 7f09d68d0700 1 -- 172.21.15.17:0/159210088 >> 172.21.15.17:0/159210088 conn(0x7f09d006af80 msgr2=0x7f09d01408b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:27.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.589+0000 7f09d68d0700 1 -- 172.21.15.17:0/159210088 shutdown_connections 2022-08-01T19:43:27.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.589+0000 7f09d68d0700 1 -- 172.21.15.17:0/159210088 wait complete. 2022-08-01T19:43:27.608 INFO:tasks.cephfs_test_runner:test_getattr_caps (tasks.cephfs.test_misc.TestMisc) ... ok 2022-08-01T19:43:27.609 INFO:tasks.cephfs_test_runner:Starting test: test_root_rctime (tasks.cephfs.test_misc.TestMisc) 2022-08-01T19:43:27.610 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_root_rctime' 2022-08-01T19:43:27.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.713+0000 7fe7bd146700 1 Processor -- start 2022-08-01T19:43:27.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.713+0000 7fe7bd146700 1 -- start start 2022-08-01T19:43:27.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.717+0000 7fe7bd146700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe7b80b6a90 0x7fe7b80acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:27.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.717+0000 7fe7bd146700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fe7b8060aa0 0x7fe7b80aa050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:27.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.717+0000 7fe7bd146700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe7b80aa540 0x7fe7b80aa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:27.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.717+0000 7fe7bd146700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7b80b6e70 con 0x7fe7b80b6a90 2022-08-01T19:43:27.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.717+0000 7fe7bd146700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7b8061370 con 0x7fe7b8060aa0 2022-08-01T19:43:27.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.717+0000 7fe7bd146700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b8061610 con 0x7fe7b80aa540 2022-08-01T19:43:27.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.717+0000 7fe7b6d9d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe7b80b6a90 0x7fe7b80acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:27.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.717+0000 7fe7b6d9d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe7b80b6a90 0x7fe7b80acee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44684/0 (socket says 172.21.15.17:44684) 2022-08-01T19:43:27.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.717+0000 7fe7b6d9d700 1 -- 172.21.15.17:0/3260748296 learned_addr learned my addr 172.21.15.17:0/3260748296 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:27.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.717+0000 7fe7ae59c700 1 --2- 172.21.15.17:0/3260748296 >> v2:172.21.15.77:3300/0 conn(0x7fe7b8060aa0 0x7fe7b80aa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:27.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.717+0000 7fe7b6d9d700 1 -- 172.21.15.17:0/3260748296 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe7b80aa540 msgr2=0x7fe7b80aa9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:27.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.717+0000 7fe7b759e700 1 --2- 172.21.15.17:0/3260748296 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe7b80aa540 0x7fe7b80aa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:27.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.717+0000 7fe7b6d9d700 1 --2- 172.21.15.17:0/3260748296 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe7b80aa540 0x7fe7b80aa9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:27.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.717+0000 7fe7b6d9d700 1 -- 172.21.15.17:0/3260748296 >> v2:172.21.15.77:3300/0 conn(0x7fe7b8060aa0 msgr2=0x7fe7b80aa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:27.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.717+0000 7fe7b6d9d700 1 --2- 172.21.15.17:0/3260748296 >> v2:172.21.15.77:3300/0 conn(0x7fe7b8060aa0 0x7fe7b80aa050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:27.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.717+0000 7fe7b6d9d700 1 -- 172.21.15.17:0/3260748296 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b80618b0 con 0x7fe7b80b6a90 2022-08-01T19:43:27.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.717+0000 7fe7ae59c700 1 --2- 172.21.15.17:0/3260748296 >> v2:172.21.15.77:3300/0 conn(0x7fe7b8060aa0 0x7fe7b80aa050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:43:27.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.717+0000 7fe7b6d9d700 1 --2- 172.21.15.17:0/3260748296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7b80b6a90 0x7fe7b80acee0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 rx=0x7fe7a80032f0 tx=0x7fe7a800b530).ready entity=mon.0 client_cookie=53e14ea60a610a9f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:27.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.717+0000 7fe7b659c700 1 -- 172.21.15.17:0/3260748296 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe7a800be70 con 0x7fe7b80b6a90 2022-08-01T19:43:27.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.717+0000 7fe7b659c700 1 -- 172.21.15.17:0/3260748296 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a8010470 con 0x7fe7b80b6a90 2022-08-01T19:43:27.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.717+0000 7fe7b659c700 1 -- 172.21.15.17:0/3260748296 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe7a80107d0 con 0x7fe7b80b6a90 2022-08-01T19:43:27.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.717+0000 7fe7bd146700 1 -- 172.21.15.17:0/3260748296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7b80b6a90 msgr2=0x7fe7b80acee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:27.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.717+0000 7fe7bd146700 1 --2- 172.21.15.17:0/3260748296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7b80b6a90 0x7fe7b80acee0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 rx=0x7fe7a80032f0 tx=0x7fe7a800b530).stop 2022-08-01T19:43:27.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.717+0000 7fe7bd146700 1 -- 172.21.15.17:0/3260748296 shutdown_connections 2022-08-01T19:43:27.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.717+0000 7fe7bd146700 1 --2- 172.21.15.17:0/3260748296 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe7b80aa540 0x7fe7b80aa9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:27.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.717+0000 7fe7bd146700 1 --2- 172.21.15.17:0/3260748296 >> v2:172.21.15.77:3300/0 conn(0x7fe7b8060aa0 0x7fe7b80aa050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:27.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.717+0000 7fe7bd146700 1 --2- 172.21.15.17:0/3260748296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7b80b6a90 0x7fe7b80acee0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:27.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.717+0000 7fe7bd146700 1 -- 172.21.15.17:0/3260748296 >> 172.21.15.17:0/3260748296 conn(0x7fe7b80b3610 msgr2=0x7fe7b80b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:27.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.717+0000 7fe7bd146700 1 -- 172.21.15.17:0/3260748296 shutdown_connections 2022-08-01T19:43:27.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.717+0000 7fe7bd146700 1 -- 172.21.15.17:0/3260748296 wait complete. 2022-08-01T19:43:27.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.717+0000 7fe7bd146700 1 Processor -- start 2022-08-01T19:43:27.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.721+0000 7fe7bd146700 1 -- start start 2022-08-01T19:43:27.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.721+0000 7fe7bd146700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7b8060aa0 0x7fe7b80b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:27.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.721+0000 7fe7bd146700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe7b80aa540 0x7fe7b80ace90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:27.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.721+0000 7fe7bd146700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7b80b6a90 0x7fe7b81382f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:27.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.721+0000 7fe7bd146700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b8143180 con 0x7fe7b8060aa0 2022-08-01T19:43:27.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.721+0000 7fe7bd146700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b80af2b0 con 0x7fe7b80b6a90 2022-08-01T19:43:27.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.721+0000 7fe7bd146700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b8143850 con 0x7fe7b80aa540 2022-08-01T19:43:27.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.721+0000 7fe7b6d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7b8060aa0 0x7fe7b80b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:27.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.721+0000 7fe7b6d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7b8060aa0 0x7fe7b80b21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44690/0 (socket says 172.21.15.17:44690) 2022-08-01T19:43:27.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.721+0000 7fe7b6d9d700 1 -- 172.21.15.17:0/3235717381 learned_addr learned my addr 172.21.15.17:0/3235717381 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:27.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.721+0000 7fe7ae59c700 1 --2- 172.21.15.17:0/3235717381 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe7b80aa540 0x7fe7b80ace90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:27.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.721+0000 7fe7b759e700 1 --2- 172.21.15.17:0/3235717381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7b80b6a90 0x7fe7b81382f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:27.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.721+0000 7fe7b6d9d700 1 -- 172.21.15.17:0/3235717381 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe7b80aa540 msgr2=0x7fe7b80ace90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:27.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.721+0000 7fe7b6d9d700 1 --2- 172.21.15.17:0/3235717381 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe7b80aa540 0x7fe7b80ace90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:27.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.721+0000 7fe7b6d9d700 1 -- 172.21.15.17:0/3235717381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7b80b6a90 msgr2=0x7fe7b81382f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:27.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.721+0000 7fe7b6d9d700 1 --2- 172.21.15.17:0/3235717381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7b80b6a90 0x7fe7b81382f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:27.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.721+0000 7fe7b6d9d700 1 -- 172.21.15.17:0/3235717381 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a800b7d0 con 0x7fe7b8060aa0 2022-08-01T19:43:27.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.721+0000 7fe7b6d9d700 1 --2- 172.21.15.17:0/3235717381 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7b8060aa0 0x7fe7b80b21d0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 rx=0x7fe7a80032f0 tx=0x7fe7b813f870).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:27.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.721+0000 7fe7b4d99700 1 -- 172.21.15.17:0/3235717381 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe7a801d070 con 0x7fe7b8060aa0 2022-08-01T19:43:27.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.721+0000 7fe7b4d99700 1 -- 172.21.15.17:0/3235717381 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a8018800 con 0x7fe7b8060aa0 2022-08-01T19:43:27.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.721+0000 7fe7bd146700 1 -- 172.21.15.17:0/3235717381 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7b8141d40 con 0x7fe7b8060aa0 2022-08-01T19:43:27.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.721+0000 7fe7b4d99700 1 -- 172.21.15.17:0/3235717381 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe7a8018b60 con 0x7fe7b8060aa0 2022-08-01T19:43:27.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.721+0000 7fe7bd146700 1 -- 172.21.15.17:0/3235717381 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7b8141fe0 con 0x7fe7b8060aa0 2022-08-01T19:43:27.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.725+0000 7fe7bd146700 1 -- 172.21.15.17:0/3235717381 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7b8061b50 con 0x7fe7b8060aa0 2022-08-01T19:43:27.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.725+0000 7fe7b4d99700 1 -- 172.21.15.17:0/3235717381 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe7a8029070 con 0x7fe7b8060aa0 2022-08-01T19:43:27.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.725+0000 7fe7b4d99700 1 --2- 172.21.15.17:0/3235717381 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe79c0539e0 0x7fe79c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:27.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.725+0000 7fe7b4d99700 1 -- 172.21.15.17:0/3235717381 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fe7a806d7c0 con 0x7fe7b8060aa0 2022-08-01T19:43:27.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.725+0000 7fe7ae59c700 1 --2- 172.21.15.17:0/3235717381 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe79c0539e0 0x7fe79c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:27.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.725+0000 7fe7ae59c700 1 --2- 172.21.15.17:0/3235717381 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe79c0539e0 0x7fe79c055ea0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7fe7a000d000 tx=0x7fe7a0008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:27.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.729+0000 7fe7b4d99700 1 -- 172.21.15.17:0/3235717381 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe7a8044750 con 0x7fe7b8060aa0 2022-08-01T19:43:27.901 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:27.897+0000 7fe7bd146700 1 -- 172.21.15.17:0/3235717381 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]} v 0) v1 -- 0x7fe7b80574e0 con 0x7fe7b8060aa0 2022-08-01T19:43:28.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.589+0000 7fe7b4d99700 1 -- 172.21.15.17:0/3235717381 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]}]=0 v430) v1 ==== 130+0+0 (secure 0 0 0) 0x7fe7a8047cf0 con 0x7fe7b8060aa0 2022-08-01T19:43:28.602 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.593+0000 7fe7bd146700 1 -- 172.21.15.17:0/3235717381 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe79c0539e0 msgr2=0x7fe79c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:28.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.593+0000 7fe7bd146700 1 --2- 172.21.15.17:0/3235717381 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe79c0539e0 0x7fe79c055ea0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7fe7a000d000 tx=0x7fe7a0008040).stop 2022-08-01T19:43:28.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.593+0000 7fe7bd146700 1 -- 172.21.15.17:0/3235717381 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7b8060aa0 msgr2=0x7fe7b80b21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:28.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.593+0000 7fe7bd146700 1 --2- 172.21.15.17:0/3235717381 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7b8060aa0 0x7fe7b80b21d0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 rx=0x7fe7a80032f0 tx=0x7fe7b813f870).stop 2022-08-01T19:43:28.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.597+0000 7fe7bd146700 1 -- 172.21.15.17:0/3235717381 shutdown_connections 2022-08-01T19:43:28.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.597+0000 7fe7bd146700 1 --2- 172.21.15.17:0/3235717381 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe79c0539e0 0x7fe79c055ea0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:28.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.597+0000 7fe7bd146700 1 --2- 172.21.15.17:0/3235717381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe7b80b6a90 0x7fe7b81382f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:28.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.597+0000 7fe7bd146700 1 --2- 172.21.15.17:0/3235717381 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe7b80aa540 0x7fe7b80ace90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:28.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.597+0000 7fe7bd146700 1 --2- 172.21.15.17:0/3235717381 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7b8060aa0 0x7fe7b80b21d0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:28.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.597+0000 7fe7bd146700 1 -- 172.21.15.17:0/3235717381 >> 172.21.15.17:0/3235717381 conn(0x7fe7b80b3610 msgr2=0x7fe7b80b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:28.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.597+0000 7fe7bd146700 1 -- 172.21.15.17:0/3235717381 shutdown_connections 2022-08-01T19:43:28.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.597+0000 7fe7bd146700 1 -- 172.21.15.17:0/3235717381 wait complete. 2022-08-01T19:43:28.616 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-08-01T19:43:28.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.737+0000 7f256f59e700 1 Processor -- start 2022-08-01T19:43:28.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.737+0000 7f256f59e700 1 -- start start 2022-08-01T19:43:28.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.737+0000 7f256f59e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f25700b01a0 0x7f25700a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:28.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.737+0000 7f256f59e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f25700afd60 0x7f25700a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:28.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.737+0000 7f256f59e700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25700a7250 con 0x7f25700b2710 2022-08-01T19:43:28.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.737+0000 7f256f59e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25700a7480 con 0x7f25700b01a0 2022-08-01T19:43:28.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.737+0000 7f256f59e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25700a7720 con 0x7f25700afd60 2022-08-01T19:43:28.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.737+0000 7f256dd9b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f25700b01a0 0x7f25700a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:28.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.737+0000 7f256e59c700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f25700b2710 0x7f257014afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36894/0 (socket says 172.21.15.17:36894) 2022-08-01T19:43:28.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.737+0000 7f256e59c700 1 -- 172.21.15.17:0/2952264369 learned_addr learned my addr 172.21.15.17:0/2952264369 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:28.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.737+0000 7f256ed9d700 1 --2- 172.21.15.17:0/2952264369 >> v2:172.21.15.77:3300/0 conn(0x7f25700afd60 0x7f25700a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:28.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.737+0000 7f256dd9b700 1 -- 172.21.15.17:0/2952264369 >> v2:172.21.15.77:3300/0 conn(0x7f25700afd60 msgr2=0x7f25700a8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:28.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.737+0000 7f256dd9b700 1 --2- 172.21.15.17:0/2952264369 >> v2:172.21.15.77:3300/0 conn(0x7f25700afd60 0x7f25700a8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:28.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.737+0000 7f256dd9b700 1 -- 172.21.15.17:0/2952264369 >> v1:172.21.15.17:6789/0 conn(0x7f25700b2710 legacy=0x7f257014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:28.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.737+0000 7f256d59a700 1 -- 172.21.15.17:0/2952264369 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1330462227 0 0) 0x7f25700a7250 con 0x7f25700b2710 2022-08-01T19:43:28.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.737+0000 7f256dd9b700 1 -- 172.21.15.17:0/2952264369 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25700a79c0 con 0x7f25700b01a0 2022-08-01T19:43:28.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.737+0000 7f256dd9b700 1 --2- 172.21.15.17:0/2952264369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25700b01a0 0x7f25700a8540 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 rx=0x7f2560002f70 tx=0x7f2560006df0).ready entity=mon.0 client_cookie=fa821728fb7944f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:28.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.737+0000 7f256d59a700 1 -- 172.21.15.17:0/2952264369 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f256000d570 con 0x7f25700b01a0 2022-08-01T19:43:28.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.737+0000 7f256d59a700 1 -- 172.21.15.17:0/2952264369 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2560000e90 con 0x7f25700b01a0 2022-08-01T19:43:28.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.737+0000 7f256d59a700 1 -- 172.21.15.17:0/2952264369 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f256000d8c0 con 0x7f25700b01a0 2022-08-01T19:43:28.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.741+0000 7f256f59e700 1 -- 172.21.15.17:0/2952264369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25700b01a0 msgr2=0x7f25700a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:28.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.741+0000 7f256f59e700 1 --2- 172.21.15.17:0/2952264369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25700b01a0 0x7f25700a8540 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 rx=0x7f2560002f70 tx=0x7f2560006df0).stop 2022-08-01T19:43:28.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.741+0000 7f256f59e700 1 -- 172.21.15.17:0/2952264369 shutdown_connections 2022-08-01T19:43:28.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.741+0000 7f256f59e700 1 --2- 172.21.15.17:0/2952264369 >> v2:172.21.15.77:3300/0 conn(0x7f25700afd60 0x7f25700a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:28.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.741+0000 7f256f59e700 1 --2- 172.21.15.17:0/2952264369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25700b01a0 0x7f25700a8540 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:28.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.741+0000 7f256f59e700 1 -- 172.21.15.17:0/2952264369 >> 172.21.15.17:0/2952264369 conn(0x7f2570060310 msgr2=0x7f25700b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:28.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.741+0000 7f256f59e700 1 -- 172.21.15.17:0/2952264369 shutdown_connections 2022-08-01T19:43:28.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.741+0000 7f256f59e700 1 -- 172.21.15.17:0/2952264369 wait complete. 2022-08-01T19:43:28.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.741+0000 7f256f59e700 1 Processor -- start 2022-08-01T19:43:28.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.741+0000 7f256f59e700 1 -- start start 2022-08-01T19:43:28.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.741+0000 7f256f59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f25700afd60 0x7f257014e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:28.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.741+0000 7f256f59e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25700b01a0 0x7f25700a9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:28.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.741+0000 7f256f59e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25700b2710 0x7f25700aad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:28.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.741+0000 7f256f59e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25700aa270 con 0x7f25700b2710 2022-08-01T19:43:28.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.741+0000 7f256f59e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25700a9990 con 0x7f25700b01a0 2022-08-01T19:43:28.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.741+0000 7f256f59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25700a9450 con 0x7f25700afd60 2022-08-01T19:43:28.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.741+0000 7f256ed9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25700b2710 0x7f25700aad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:28.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.745+0000 7f256ed9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25700b2710 0x7f25700aad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44708/0 (socket says 172.21.15.17:44708) 2022-08-01T19:43:28.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.745+0000 7f256ed9d700 1 -- 172.21.15.17:0/3025979773 learned_addr learned my addr 172.21.15.17:0/3025979773 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:28.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.745+0000 7f256dd9b700 1 --2- 172.21.15.17:0/3025979773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25700b01a0 0x7f25700a9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:28.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.745+0000 7f256e59c700 1 --2- 172.21.15.17:0/3025979773 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f25700afd60 0x7f257014e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:28.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.745+0000 7f256ed9d700 1 -- 172.21.15.17:0/3025979773 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f25700afd60 msgr2=0x7f257014e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:28.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.745+0000 7f256ed9d700 1 --2- 172.21.15.17:0/3025979773 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f25700afd60 0x7f257014e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:28.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.745+0000 7f256ed9d700 1 -- 172.21.15.17:0/3025979773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25700b01a0 msgr2=0x7f25700a9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:28.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.745+0000 7f256ed9d700 1 --2- 172.21.15.17:0/3025979773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25700b01a0 0x7f25700a9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:28.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.745+0000 7f256ed9d700 1 -- 172.21.15.17:0/3025979773 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f256000b180 con 0x7f25700b2710 2022-08-01T19:43:28.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.745+0000 7f256ed9d700 1 --2- 172.21.15.17:0/3025979773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25700b2710 0x7f25700aad80 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 rx=0x7f256800c000 tx=0x7f25680091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:28.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.745+0000 7f255f7fe700 1 -- 172.21.15.17:0/3025979773 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2568012600 con 0x7f25700b2710 2022-08-01T19:43:28.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.745+0000 7f255f7fe700 1 -- 172.21.15.17:0/3025979773 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2568012760 con 0x7f25700b2710 2022-08-01T19:43:28.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.745+0000 7f256f59e700 1 -- 172.21.15.17:0/3025979773 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25700a96f0 con 0x7f25700b2710 2022-08-01T19:43:28.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.745+0000 7f255f7fe700 1 -- 172.21.15.17:0/3025979773 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2568012ac0 con 0x7f25700b2710 2022-08-01T19:43:28.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.745+0000 7f256f59e700 1 -- 172.21.15.17:0/3025979773 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2570159c00 con 0x7f25700b2710 2022-08-01T19:43:28.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.745+0000 7f256f59e700 1 -- 172.21.15.17:0/3025979773 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25700574e0 con 0x7f25700b2710 2022-08-01T19:43:28.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.745+0000 7f255f7fe700 1 -- 172.21.15.17:0/3025979773 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2568022070 con 0x7f25700b2710 2022-08-01T19:43:28.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.745+0000 7f255f7fe700 1 --2- 172.21.15.17:0/3025979773 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f254c0539e0 0x7f254c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:28.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.749+0000 7f255f7fe700 1 -- 172.21.15.17:0/3025979773 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f256806c370 con 0x7f25700b2710 2022-08-01T19:43:28.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.749+0000 7f256e59c700 1 --2- 172.21.15.17:0/3025979773 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f254c0539e0 0x7f254c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:28.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.749+0000 7f256e59c700 1 --2- 172.21.15.17:0/3025979773 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f254c0539e0 0x7f254c055ea0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f256400d000 tx=0x7f2564007c20).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:28.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.753+0000 7f255f7fe700 1 -- 172.21.15.17:0/3025979773 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2568043300 con 0x7f25700b2710 2022-08-01T19:43:28.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.877+0000 7f256f59e700 1 -- 172.21.15.17:0/3025979773 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f25700aaae0 con 0x7f25700b2710 2022-08-01T19:43:28.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.885+0000 7f255f7fe700 1 -- 172.21.15.17:0/3025979773 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2568046920 con 0x7f25700b2710 2022-08-01T19:43:28.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.885+0000 7f256f59e700 1 -- 172.21.15.17:0/3025979773 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f254c0539e0 msgr2=0x7f254c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:28.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.885+0000 7f256f59e700 1 --2- 172.21.15.17:0/3025979773 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f254c0539e0 0x7f254c055ea0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f256400d000 tx=0x7f2564007c20).stop 2022-08-01T19:43:28.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.885+0000 7f256f59e700 1 -- 172.21.15.17:0/3025979773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25700b2710 msgr2=0x7f25700aad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:28.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.885+0000 7f256f59e700 1 --2- 172.21.15.17:0/3025979773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25700b2710 0x7f25700aad80 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 rx=0x7f256800c000 tx=0x7f25680091d0).stop 2022-08-01T19:43:28.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.885+0000 7f256f59e700 1 -- 172.21.15.17:0/3025979773 shutdown_connections 2022-08-01T19:43:28.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.885+0000 7f256f59e700 1 --2- 172.21.15.17:0/3025979773 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f254c0539e0 0x7f254c055ea0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:28.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.885+0000 7f256f59e700 1 --2- 172.21.15.17:0/3025979773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25700b2710 0x7f25700aad80 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:28.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.885+0000 7f256f59e700 1 --2- 172.21.15.17:0/3025979773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25700b01a0 0x7f25700a9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:28.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.885+0000 7f256f59e700 1 --2- 172.21.15.17:0/3025979773 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f25700afd60 0x7f257014e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:28.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.885+0000 7f256f59e700 1 -- 172.21.15.17:0/3025979773 >> 172.21.15.17:0/3025979773 conn(0x7f2570060310 msgr2=0x7f25700af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:28.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.885+0000 7f256f59e700 1 -- 172.21.15.17:0/3025979773 shutdown_connections 2022-08-01T19:43:28.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:28.885+0000 7f256f59e700 1 -- 172.21.15.17:0/3025979773 wait complete. 2022-08-01T19:43:28.903 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T19:43:28.904 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:43:28.927 DEBUG:teuthology.orchestra.run.smithi077:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:43:28.955 DEBUG:teuthology.orchestra.run.smithi085:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:43:28.982 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:43:29.011 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T19:43:29.012 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:29.013 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:43:29.013 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:43:29.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:43:29.032 INFO:teuthology.orchestra.run.smithi085.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-08-01T19:43:29.033 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-08-01T19:43:29.033 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:29.034 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:43:29.034 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:43:29.065 INFO:teuthology.orchestra.run.smithi190.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T19:43:29.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:43:29.075 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-08-01T19:43:29.076 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T19:43:29.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.185+0000 7f2daf59e700 1 Processor -- start 2022-08-01T19:43:29.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.185+0000 7f2daf59e700 1 -- start start 2022-08-01T19:43:29.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.185+0000 7f2daf59e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2db00a9da0 0x7f2db0148540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:29.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.185+0000 7f2daf59e700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2db00b3570 con 0x7f2db00b1a80 2022-08-01T19:43:29.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.185+0000 7f2daf59e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2db00b0e00 con 0x7f2db00b3190 2022-08-01T19:43:29.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.185+0000 7f2daf59e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2db00b10a0 con 0x7f2db00a9da0 2022-08-01T19:43:29.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.185+0000 7f2dadd9b700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2db00b1a80 0x7f2db0147e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36906/0 (socket says 172.21.15.17:36906) 2022-08-01T19:43:29.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.185+0000 7f2dadd9b700 1 -- 172.21.15.17:0/2623067043 learned_addr learned my addr 172.21.15.17:0/2623067043 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:29.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.185+0000 7f2daed9d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2db00a9da0 0x7f2db0148540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:29.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.185+0000 7f2dad59a700 1 -- 172.21.15.17:0/2623067043 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1102769985 0 0) 0x7f2db00b3570 con 0x7f2db00b1a80 2022-08-01T19:43:29.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.185+0000 7f2dad59a700 1 -- 172.21.15.17:0/2623067043 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d94003260 con 0x7f2db00b1a80 2022-08-01T19:43:29.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.185+0000 7f2dad59a700 1 -- 172.21.15.17:0/2623067043 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3788014912 0 0) 0x7f2db00b0e00 con 0x7f2db00b3190 2022-08-01T19:43:29.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.185+0000 7f2dad59a700 1 -- 172.21.15.17:0/2623067043 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2db00b3570 con 0x7f2db00b3190 2022-08-01T19:43:29.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.185+0000 7f2daed9d700 1 -- 172.21.15.17:0/2623067043 >> v1:172.21.15.77:6789/0 conn(0x7f2db00b3190 legacy=0x7f2db00656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:29.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.185+0000 7f2daed9d700 1 -- 172.21.15.17:0/2623067043 >> v1:172.21.15.17:6789/0 conn(0x7f2db00b1a80 legacy=0x7f2db0147e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:29.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.185+0000 7f2daed9d700 1 -- 172.21.15.17:0/2623067043 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db00b1340 con 0x7f2db00a9da0 2022-08-01T19:43:29.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.185+0000 7f2daed9d700 1 --2- 172.21.15.17:0/2623067043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2db00a9da0 0x7f2db0148540 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 rx=0x7f2da80032f0 tx=0x7f2da80091d0).ready entity=mon.0 client_cookie=51e6a056b0b7f71c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:29.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.185+0000 7f2dad59a700 1 -- 172.21.15.17:0/2623067043 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2da800b600 con 0x7f2db00a9da0 2022-08-01T19:43:29.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.185+0000 7f2dad59a700 1 -- 172.21.15.17:0/2623067043 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da800b760 con 0x7f2db00a9da0 2022-08-01T19:43:29.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.185+0000 7f2dad59a700 1 -- 172.21.15.17:0/2623067043 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2da800b8d0 con 0x7f2db00a9da0 2022-08-01T19:43:29.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.189+0000 7f2daf59e700 1 -- 172.21.15.17:0/2623067043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2db00a9da0 msgr2=0x7f2db0148540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:29.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.189+0000 7f2daf59e700 1 --2- 172.21.15.17:0/2623067043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2db00a9da0 0x7f2db0148540 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 rx=0x7f2da80032f0 tx=0x7f2da80091d0).stop 2022-08-01T19:43:29.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.189+0000 7f2daf59e700 1 -- 172.21.15.17:0/2623067043 shutdown_connections 2022-08-01T19:43:29.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.189+0000 7f2daf59e700 1 --2- 172.21.15.17:0/2623067043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2db00a9da0 0x7f2db0148540 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:29.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.189+0000 7f2daf59e700 1 -- 172.21.15.17:0/2623067043 >> 172.21.15.17:0/2623067043 conn(0x7f2db0063870 msgr2=0x7f2db0063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:29.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.189+0000 7f2daf59e700 1 -- 172.21.15.17:0/2623067043 shutdown_connections 2022-08-01T19:43:29.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.189+0000 7f2daf59e700 1 -- 172.21.15.17:0/2623067043 wait complete. 2022-08-01T19:43:29.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.189+0000 7f2daf59e700 1 Processor -- start 2022-08-01T19:43:29.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.189+0000 7f2daf59e700 1 -- start start 2022-08-01T19:43:29.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.189+0000 7f2daf59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2db00a9da0 0x7f2db0066950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:29.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.189+0000 7f2daf59e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2db00b1a80 0x7f2db0156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:29.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.189+0000 7f2daf59e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2db00b3190 0x7f2db01570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:29.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.189+0000 7f2daf59e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db015a320 con 0x7f2db00b3190 2022-08-01T19:43:29.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.189+0000 7f2daf59e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db015a5c0 con 0x7f2db00b1a80 2022-08-01T19:43:29.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.189+0000 7f2daf59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db015a860 con 0x7f2db00a9da0 2022-08-01T19:43:29.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.193+0000 7f2daed9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2db00b3190 0x7f2db01570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:29.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.193+0000 7f2daed9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2db00b3190 0x7f2db01570c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44722/0 (socket says 172.21.15.17:44722) 2022-08-01T19:43:29.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.193+0000 7f2daed9d700 1 -- 172.21.15.17:0/3503740147 learned_addr learned my addr 172.21.15.17:0/3503740147 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:29.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.193+0000 7f2dadd9b700 1 --2- 172.21.15.17:0/3503740147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2db00b1a80 0x7f2db0156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:29.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.193+0000 7f2dae59c700 1 --2- 172.21.15.17:0/3503740147 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2db00a9da0 0x7f2db0066950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:29.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.193+0000 7f2daed9d700 1 -- 172.21.15.17:0/3503740147 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2db00a9da0 msgr2=0x7f2db0066950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:29.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.193+0000 7f2daed9d700 1 --2- 172.21.15.17:0/3503740147 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2db00a9da0 0x7f2db0066950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:29.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.193+0000 7f2daed9d700 1 -- 172.21.15.17:0/3503740147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2db00b1a80 msgr2=0x7f2db0156bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:29.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.193+0000 7f2daed9d700 1 --2- 172.21.15.17:0/3503740147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2db00b1a80 0x7f2db0156bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:29.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.193+0000 7f2daed9d700 1 -- 172.21.15.17:0/3503740147 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da8009470 con 0x7f2db00b3190 2022-08-01T19:43:29.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.193+0000 7f2dae59c700 1 --2- 172.21.15.17:0/3503740147 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2db00a9da0 0x7f2db0066950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:43:29.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.193+0000 7f2daed9d700 1 --2- 172.21.15.17:0/3503740147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2db00b3190 0x7f2db01570c0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 rx=0x7f2da80032f0 tx=0x7f2da8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:29.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.193+0000 7f2d9f7fe700 1 -- 172.21.15.17:0/3503740147 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2da800b600 con 0x7f2db00b3190 2022-08-01T19:43:29.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.193+0000 7f2d9f7fe700 1 -- 172.21.15.17:0/3503740147 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da800bec0 con 0x7f2db00b3190 2022-08-01T19:43:29.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.193+0000 7f2daf59e700 1 -- 172.21.15.17:0/3503740147 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2db015ab00 con 0x7f2db00b3190 2022-08-01T19:43:29.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.193+0000 7f2d9f7fe700 1 -- 172.21.15.17:0/3503740147 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2da801f5e0 con 0x7f2db00b3190 2022-08-01T19:43:29.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.193+0000 7f2daf59e700 1 -- 172.21.15.17:0/3503740147 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db01575b0 con 0x7f2db00b3190 2022-08-01T19:43:29.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.193+0000 7f2daf59e700 1 -- 172.21.15.17:0/3503740147 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2db015ace0 con 0x7f2db00b3190 2022-08-01T19:43:29.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.197+0000 7f2d9f7fe700 1 -- 172.21.15.17:0/3503740147 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2da8026030 con 0x7f2db00b3190 2022-08-01T19:43:29.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.197+0000 7f2d9f7fe700 1 --2- 172.21.15.17:0/3503740147 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2d8c053aa0 0x7f2d8c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:29.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.197+0000 7f2d9f7fe700 1 -- 172.21.15.17:0/3503740147 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f2da806c8a0 con 0x7f2db00b3190 2022-08-01T19:43:29.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.197+0000 7f2dae59c700 1 --2- 172.21.15.17:0/3503740147 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2d8c053aa0 0x7f2d8c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:29.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.197+0000 7f2dae59c700 1 --2- 172.21.15.17:0/3503740147 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2d8c053aa0 0x7f2d8c055f60 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f2da000b000 tx=0x7f2da0009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:29.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.205+0000 7f2d9f7fe700 1 -- 172.21.15.17:0/3503740147 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2da80439c0 con 0x7f2db00b3190 2022-08-01T19:43:29.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.333+0000 7f2daf59e700 1 -- 172.21.15.17:0/3503740147 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f2db00574e0 con 0x7f2db00b3190 2022-08-01T19:43:29.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.337+0000 7f2d9f7fe700 1 -- 172.21.15.17:0/3503740147 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v140) v1 ==== 81+0+29227 (secure 0 0 0) 0x7f2da8024090 con 0x7f2db00b3190 2022-08-01T19:43:29.341 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:29.341 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:43:29.342 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 140, 2022-08-01T19:43:29.342 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", 2022-08-01T19:43:29.343 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2022-08-01T19:33:50.105743+0000", 2022-08-01T19:43:29.343 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2022-08-01T19:43:26.573944+0000", 2022-08-01T19:43:29.344 INFO:teuthology.orchestra.run.smithi017.stdout: "last_up_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:43:29.344 INFO:teuthology.orchestra.run.smithi017.stdout: "last_in_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:43:29.345 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T19:43:29.345 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_num": 5799936, 2022-08-01T19:43:29.347 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_set": [ 2022-08-01T19:43:29.347 INFO:teuthology.orchestra.run.smithi017.stdout: "pglog_hardlimit", 2022-08-01T19:43:29.347 INFO:teuthology.orchestra.run.smithi017.stdout: "purged_snapdirs", 2022-08-01T19:43:29.348 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_deletes", 2022-08-01T19:43:29.348 INFO:teuthology.orchestra.run.smithi017.stdout: "sortbitwise" 2022-08-01T19:43:29.348 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:43:29.349 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_version": 6, 2022-08-01T19:43:29.350 INFO:teuthology.orchestra.run.smithi017.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T19:43:29.350 INFO:teuthology.orchestra.run.smithi017.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T19:43:29.350 INFO:teuthology.orchestra.run.smithi017.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T19:43:29.351 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_snapshot": "", 2022-08-01T19:43:29.351 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_max": 26, 2022-08-01T19:43:29.351 INFO:teuthology.orchestra.run.smithi017.stdout: "max_osd": 8, 2022-08-01T19:43:29.352 INFO:teuthology.orchestra.run.smithi017.stdout: "require_min_compat_client": "jewel", 2022-08-01T19:43:29.352 INFO:teuthology.orchestra.run.smithi017.stdout: "min_compat_client": "jewel", 2022-08-01T19:43:29.352 INFO:teuthology.orchestra.run.smithi017.stdout: "require_osd_release": "octopus", 2022-08-01T19:43:29.352 INFO:teuthology.orchestra.run.smithi017.stdout: "pools": [ 2022-08-01T19:43:29.353 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.353 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 1, 2022-08-01T19:43:29.353 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "device_health_metrics", 2022-08-01T19:43:29.353 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:33:53.001841+0000", 2022-08-01T19:43:29.354 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:43:29.354 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:43:29.354 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:43:29.354 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:43:29.355 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:43:29.355 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:43:29.355 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:43:29.356 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:43:29.356 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 1, 2022-08-01T19:43:29.357 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 1, 2022-08-01T19:43:29.357 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 1, 2022-08-01T19:43:29.358 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 1, 2022-08-01T19:43:29.358 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 1, 2022-08-01T19:43:29.359 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:43:29.359 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:43:29.360 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:43:29.361 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:43:29.361 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:43:29.362 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:43:29.362 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:43:29.363 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.363 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "16", 2022-08-01T19:43:29.364 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:43:29.364 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:43:29.364 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:43:29.365 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:43:29.366 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:43:29.366 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:43:29.366 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:43:29.367 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:43:29.367 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:43:29.367 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:43:29.368 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:43:29.368 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:43:29.369 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:43:29.369 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:43:29.370 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:43:29.371 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:43:29.371 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:43:29.372 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:43:29.378 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:43:29.378 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:43:29.379 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:43:29.379 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:43:29.379 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:43:29.380 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:43:29.380 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:43:29.381 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:43:29.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.381 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:43:29.382 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:43:29.382 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:43:29.383 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:43:29.383 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:43:29.383 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:43:29.384 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:43:29.384 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:43:29.385 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:43:29.385 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:43:29.386 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:43:29.386 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:43:29.387 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 1 2022-08-01T19:43:29.387 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.387 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:43:29.388 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_devicehealth": {} 2022-08-01T19:43:29.388 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.389 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.389 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.390 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 2, 2022-08-01T19:43:29.390 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "rbd", 2022-08-01T19:43:29.390 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:34:01.553098+0000", 2022-08-01T19:43:29.391 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:43:29.391 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:43:29.392 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:43:29.392 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:43:29.392 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:43:29.393 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:43:29.393 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:43:29.393 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:43:29.394 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:43:29.394 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:43:29.394 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:43:29.395 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:43:29.395 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:43:29.395 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:43:29.396 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:43:29.396 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:43:29.396 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:43:29.396 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:43:29.397 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:43:29.397 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:43:29.397 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.397 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "19", 2022-08-01T19:43:29.398 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:43:29.398 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:43:29.398 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:43:29.398 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:43:29.399 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:43:29.399 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:43:29.399 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:43:29.399 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:43:29.399 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:43:29.400 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:43:29.400 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:43:29.400 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:43:29.400 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:43:29.401 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:43:29.401 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:43:29.401 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:43:29.401 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:43:29.401 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:43:29.402 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:43:29.402 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:43:29.402 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:43:29.402 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:43:29.403 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:43:29.403 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:43:29.403 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:43:29.404 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:43:29.404 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.404 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:43:29.404 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:43:29.405 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:43:29.405 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:43:29.405 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:43:29.406 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:43:29.406 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:43:29.406 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:43:29.406 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:43:29.407 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:43:29.407 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:43:29.407 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:43:29.407 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:43:29.408 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd": {} 2022-08-01T19:43:29.408 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.408 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.408 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:43:29.408 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2022-08-01T19:43:29.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.409 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:43:29.409 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a962f14f-cf16-4fa8-84ff-c85325db9132", 2022-08-01T19:43:29.409 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:43:29.410 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:43:29.410 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:43:29.410 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:43:29.411 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:43:29.411 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:43:29.412 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:43:29.412 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 134, 2022-08-01T19:43:29.413 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:43:29.413 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:43:29.413 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:43:29.414 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:29.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.417 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:29.417 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6800", 2022-08-01T19:43:29.419 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:43:29.420 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.420 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.421 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:29.421 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6801", 2022-08-01T19:43:29.422 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:43:29.422 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.423 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.423 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.423 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:43:29.424 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:29.424 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.425 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:29.425 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6802", 2022-08-01T19:43:29.426 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:43:29.426 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.427 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.428 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:29.428 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6803", 2022-08-01T19:43:29.430 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:43:29.430 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.431 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.431 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.432 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:43:29.432 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:29.433 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.433 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:29.434 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6806", 2022-08-01T19:43:29.434 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:43:29.435 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.435 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.435 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:29.436 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6807", 2022-08-01T19:43:29.436 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:43:29.436 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.437 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.437 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.437 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:43:29.438 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:29.439 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.439 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:29.441 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6804", 2022-08-01T19:43:29.443 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:43:29.445 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.446 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.447 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:29.448 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6805", 2022-08-01T19:43:29.448 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:43:29.450 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.451 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.452 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.452 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6801/23008", 2022-08-01T19:43:29.452 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6803/23008", 2022-08-01T19:43:29.453 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6807/23008", 2022-08-01T19:43:29.453 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6805/23008", 2022-08-01T19:43:29.453 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:43:29.454 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:43:29.454 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:43:29.454 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.454 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.455 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.455 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:43:29.455 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "805f7051-a527-4e98-8844-b6e406a63248", 2022-08-01T19:43:29.456 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:43:29.456 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:43:29.456 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:43:29.456 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:43:29.456 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:43:29.457 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:43:29.457 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:43:29.457 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 135, 2022-08-01T19:43:29.458 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:43:29.458 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:43:29.458 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:43:29.458 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:29.459 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.459 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:29.459 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6808", 2022-08-01T19:43:29.459 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:43:29.459 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.460 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.460 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:29.460 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6809", 2022-08-01T19:43:29.460 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:43:29.461 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.461 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.461 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.461 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:43:29.462 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:29.462 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.463 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:29.463 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6810", 2022-08-01T19:43:29.463 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:43:29.463 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.464 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.464 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:29.464 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6811", 2022-08-01T19:43:29.464 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:43:29.464 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.465 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.465 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.465 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:43:29.465 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:29.466 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.466 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:29.466 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6814", 2022-08-01T19:43:29.466 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:43:29.467 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.467 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.467 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:29.467 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6815", 2022-08-01T19:43:29.468 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:43:29.468 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.468 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.468 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.468 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:43:29.469 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:29.469 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.469 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:29.469 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6812", 2022-08-01T19:43:29.470 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:43:29.470 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.470 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.470 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:29.470 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6813", 2022-08-01T19:43:29.471 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:43:29.471 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.471 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.471 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.472 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6809/23007", 2022-08-01T19:43:29.472 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6811/23007", 2022-08-01T19:43:29.473 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6815/23007", 2022-08-01T19:43:29.473 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6813/23007", 2022-08-01T19:43:29.473 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:43:29.474 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:43:29.474 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:43:29.474 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.474 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.475 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.475 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:43:29.475 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "89f7f3df-7c10-48e1-8b69-9e4243923a61", 2022-08-01T19:43:29.475 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:43:29.476 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:43:29.476 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:43:29.476 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:43:29.476 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:43:29.476 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:43:29.477 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:43:29.477 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 135, 2022-08-01T19:43:29.477 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:43:29.477 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:43:29.478 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:43:29.478 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:29.478 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.478 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:29.479 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6817", 2022-08-01T19:43:29.479 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:43:29.479 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.479 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.480 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:29.480 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6819", 2022-08-01T19:43:29.480 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:43:29.480 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.481 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.481 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.481 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:43:29.482 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:29.482 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.482 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:29.483 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6822", 2022-08-01T19:43:29.483 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:43:29.484 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.484 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.484 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:29.484 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6824", 2022-08-01T19:43:29.484 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:43:29.485 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.485 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.485 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.485 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:43:29.486 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:29.486 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.486 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:29.486 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6830", 2022-08-01T19:43:29.487 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:43:29.487 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.487 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.487 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:29.487 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6831", 2022-08-01T19:43:29.488 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:43:29.488 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.488 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.488 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.489 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:43:29.489 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:29.489 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.489 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:29.490 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6826", 2022-08-01T19:43:29.490 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:43:29.490 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.490 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.491 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:29.491 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6829", 2022-08-01T19:43:29.491 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:43:29.491 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.492 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.492 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.492 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6819/23010", 2022-08-01T19:43:29.492 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6824/23010", 2022-08-01T19:43:29.492 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6831/23010", 2022-08-01T19:43:29.493 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6829/23010", 2022-08-01T19:43:29.493 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:43:29.493 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:43:29.493 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:43:29.494 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.494 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.494 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.494 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:43:29.495 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "96123468-aabc-4a64-909e-a192c369090d", 2022-08-01T19:43:29.495 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:43:29.495 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:43:29.495 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:43:29.496 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:43:29.496 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:43:29.496 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:43:29.496 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:43:29.497 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 134, 2022-08-01T19:43:29.497 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:43:29.497 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:43:29.497 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:43:29.498 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:29.498 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.498 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:29.498 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6816", 2022-08-01T19:43:29.499 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:43:29.499 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.499 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.499 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:29.500 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6818", 2022-08-01T19:43:29.500 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:43:29.500 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.500 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.501 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.501 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:43:29.501 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:29.501 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.502 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:29.502 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6820", 2022-08-01T19:43:29.502 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:43:29.502 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.503 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.503 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:29.503 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6821", 2022-08-01T19:43:29.503 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:43:29.504 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.504 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.504 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.504 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:43:29.505 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:29.505 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.505 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:29.505 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6827", 2022-08-01T19:43:29.506 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:43:29.506 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.506 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.506 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:29.507 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6828", 2022-08-01T19:43:29.507 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:43:29.507 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.508 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.508 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.508 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:43:29.508 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:29.508 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.509 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:29.509 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6823", 2022-08-01T19:43:29.509 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:43:29.509 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.510 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.510 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:29.510 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6825", 2022-08-01T19:43:29.510 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:43:29.511 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.511 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.511 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.511 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6818/23009", 2022-08-01T19:43:29.512 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6821/23009", 2022-08-01T19:43:29.512 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6828/23009", 2022-08-01T19:43:29.512 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6825/23009", 2022-08-01T19:43:29.513 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:43:29.513 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:43:29.513 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:43:29.513 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.513 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.514 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.514 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:43:29.514 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a6e26665-e04b-46ec-ab66-e912acbf044c", 2022-08-01T19:43:29.514 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:43:29.515 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:43:29.515 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:43:29.515 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:43:29.515 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:43:29.516 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:43:29.516 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:43:29.516 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 135, 2022-08-01T19:43:29.516 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:43:29.516 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:43:29.517 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:43:29.517 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:29.517 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.517 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:29.518 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6810", 2022-08-01T19:43:29.518 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:43:29.518 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.519 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.519 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:29.519 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6811", 2022-08-01T19:43:29.520 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:43:29.520 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.520 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.520 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.521 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:43:29.521 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:29.521 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.521 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:29.522 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6812", 2022-08-01T19:43:29.522 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:43:29.523 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.523 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.523 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:29.523 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6813", 2022-08-01T19:43:29.524 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:43:29.524 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.524 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.524 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.524 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:43:29.525 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:29.525 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.525 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:29.525 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6816", 2022-08-01T19:43:29.526 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:43:29.526 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.526 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.527 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:29.527 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6817", 2022-08-01T19:43:29.527 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:43:29.527 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.528 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.528 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.528 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:43:29.528 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:29.528 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.529 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:29.529 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6814", 2022-08-01T19:43:29.529 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:43:29.529 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.530 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.530 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:29.530 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6815", 2022-08-01T19:43:29.530 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:43:29.531 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.531 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.531 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.531 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6811/23584", 2022-08-01T19:43:29.531 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6813/23584", 2022-08-01T19:43:29.532 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6817/23584", 2022-08-01T19:43:29.532 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6815/23584", 2022-08-01T19:43:29.532 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:43:29.532 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:43:29.533 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:43:29.533 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.533 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.533 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.534 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:43:29.534 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "502c0034-18ee-44ac-87ff-19b9dddb66a0", 2022-08-01T19:43:29.534 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:43:29.534 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:43:29.535 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:43:29.535 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:43:29.535 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:43:29.535 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:43:29.535 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:43:29.536 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 127, 2022-08-01T19:43:29.536 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:43:29.536 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:43:29.536 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:43:29.537 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:29.537 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.537 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:29.537 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6802", 2022-08-01T19:43:29.538 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:43:29.538 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.538 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.538 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:29.539 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6803", 2022-08-01T19:43:29.539 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:43:29.539 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.539 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.540 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.540 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:43:29.540 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:29.540 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.541 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:29.541 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6804", 2022-08-01T19:43:29.541 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:43:29.541 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.541 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.542 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:29.542 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6805", 2022-08-01T19:43:29.542 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:43:29.542 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.543 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.543 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.543 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:43:29.543 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:29.544 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.544 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:29.544 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6808", 2022-08-01T19:43:29.544 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:43:29.545 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.545 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.545 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:29.545 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6809", 2022-08-01T19:43:29.546 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:43:29.546 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.546 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.546 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.546 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:43:29.547 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:29.547 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.547 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:29.547 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6806", 2022-08-01T19:43:29.548 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:43:29.548 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.548 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.548 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:29.549 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6807", 2022-08-01T19:43:29.549 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:43:29.549 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.549 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.550 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.550 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6803/23585", 2022-08-01T19:43:29.550 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6805/23585", 2022-08-01T19:43:29.550 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6809/23585", 2022-08-01T19:43:29.551 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6807/23585", 2022-08-01T19:43:29.551 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:43:29.551 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:43:29.551 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:43:29.552 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.552 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.552 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.552 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:43:29.553 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "ac4e755a-7fe7-46cb-8407-852d5af9ffc5", 2022-08-01T19:43:29.553 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:43:29.553 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:43:29.553 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:43:29.553 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:43:29.554 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:43:29.554 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:43:29.554 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:43:29.554 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 135, 2022-08-01T19:43:29.555 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:43:29.555 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:43:29.555 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:43:29.555 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:29.556 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.556 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:29.556 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6820", 2022-08-01T19:43:29.556 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:43:29.556 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.557 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.557 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:29.557 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6821", 2022-08-01T19:43:29.558 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:43:29.558 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.558 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.558 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.559 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:43:29.559 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:29.559 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.559 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:29.559 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6823", 2022-08-01T19:43:29.560 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:43:29.560 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.560 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.560 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:29.561 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6825", 2022-08-01T19:43:29.561 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:43:29.561 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.561 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.561 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.562 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:43:29.562 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:29.562 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.562 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:29.563 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6831", 2022-08-01T19:43:29.563 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:43:29.563 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.563 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.564 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:29.564 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6833", 2022-08-01T19:43:29.564 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:43:29.564 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.564 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.565 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.565 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:43:29.565 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:29.565 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.565 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:29.566 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6827", 2022-08-01T19:43:29.566 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:43:29.566 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.566 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.567 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:29.567 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6829", 2022-08-01T19:43:29.567 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:43:29.567 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.568 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.568 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.568 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6821/23587", 2022-08-01T19:43:29.568 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6825/23587", 2022-08-01T19:43:29.568 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6833/23587", 2022-08-01T19:43:29.569 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6829/23587", 2022-08-01T19:43:29.569 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:43:29.569 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:43:29.569 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:43:29.570 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.570 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.570 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.570 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:43:29.571 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "874761e2-6410-40fc-b60c-e58abe5ce489", 2022-08-01T19:43:29.571 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:43:29.571 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:43:29.571 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:43:29.571 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:43:29.572 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:43:29.572 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:43:29.572 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:43:29.572 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 135, 2022-08-01T19:43:29.573 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:43:29.573 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:43:29.573 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:43:29.573 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:29.573 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.574 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:29.574 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6818", 2022-08-01T19:43:29.574 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:43:29.574 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.575 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.575 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:29.575 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6819", 2022-08-01T19:43:29.575 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:43:29.576 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.576 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.576 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.576 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:43:29.576 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:29.577 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.577 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:29.577 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6822", 2022-08-01T19:43:29.577 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:43:29.578 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.578 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.578 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:29.578 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6824", 2022-08-01T19:43:29.579 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:43:29.579 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.579 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.580 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.580 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:43:29.580 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:29.581 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.581 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:29.581 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6830", 2022-08-01T19:43:29.581 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:43:29.582 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.582 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.582 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:29.583 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6832", 2022-08-01T19:43:29.583 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:43:29.583 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.583 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.584 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.584 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:43:29.584 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:29.584 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.585 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:29.585 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6826", 2022-08-01T19:43:29.585 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:43:29.585 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.586 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.586 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:29.586 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6828", 2022-08-01T19:43:29.586 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:43:29.587 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.587 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.587 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.587 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6819/23586", 2022-08-01T19:43:29.588 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6824/23586", 2022-08-01T19:43:29.588 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6832/23586", 2022-08-01T19:43:29.588 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6828/23586", 2022-08-01T19:43:29.588 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:43:29.589 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:43:29.589 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:43:29.589 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:29.589 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.590 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:43:29.590 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_xinfo": [ 2022-08-01T19:43:29.590 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.590 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:43:29.590 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:43:29.591 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:43:29.591 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:43:29.591 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:43:29.591 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:43:29.592 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.588817+0000", 2022-08-01T19:43:29.592 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:43:29.592 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.593 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.593 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:43:29.593 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:43:29.593 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:43:29.593 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:43:29.594 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:43:29.594 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:43:29.594 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.782753+0000", 2022-08-01T19:43:29.594 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:43:29.595 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.595 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.595 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:43:29.595 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:43:29.596 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:43:29.596 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:43:29.596 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:43:29.596 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:43:29.597 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.837924+0000", 2022-08-01T19:43:29.597 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:43:29.597 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.597 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.598 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:43:29.610 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:43:29.610 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:43:29.610 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:43:29.611 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:43:29.611 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:43:29.612 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.839948+0000", 2022-08-01T19:43:29.612 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:43:29.613 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.613 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.619 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:43:29.619 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:43:29.620 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:43:29.620 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:43:29.620 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:43:29.620 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:43:29.620 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.112801+0000", 2022-08-01T19:43:29.621 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:43:29.621 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.621 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.621 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:43:29.622 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:43:29.622 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:43:29.622 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:43:29.623 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:43:29.623 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:43:29.623 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.106704+0000", 2022-08-01T19:43:29.624 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:43:29.624 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.624 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.624 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:43:29.625 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:43:29.625 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:43:29.625 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:43:29.625 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:43:29.626 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:43:29.626 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.893170+0000", 2022-08-01T19:43:29.626 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:43:29.627 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.627 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:29.627 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:43:29.627 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:43:29.628 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:43:29.628 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:43:29.628 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:43:29.628 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:43:29.629 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.886947+0000", 2022-08-01T19:43:29.629 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:43:29.629 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.629 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:43:29.629 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap": [], 2022-08-01T19:43:29.630 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap_items": [], 2022-08-01T19:43:29.630 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_temp": [], 2022-08-01T19:43:29.630 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_temp": [], 2022-08-01T19:43:29.630 INFO:teuthology.orchestra.run.smithi017.stdout: "blacklist": { 2022-08-01T19:43:29.631 INFO:teuthology.orchestra.run.smithi017.stdout: "172.21.15.77:6835/299216688": "2022-08-02T19:43:22.557511+0000", 2022-08-01T19:43:29.631 INFO:teuthology.orchestra.run.smithi017.stdout: "172.21.15.77:6834/299216688": "2022-08-02T19:43:22.557511+0000" 2022-08-01T19:43:29.631 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.631 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profiles": { 2022-08-01T19:43:29.632 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_data": { 2022-08-01T19:43:29.632 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-device-class": "", 2022-08-01T19:43:29.632 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:43:29.632 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-root": "default", 2022-08-01T19:43:29.632 INFO:teuthology.orchestra.run.smithi017.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T19:43:29.633 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:43:29.633 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "2", 2022-08-01T19:43:29.633 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:43:29.633 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:43:29.634 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van", 2022-08-01T19:43:29.634 INFO:teuthology.orchestra.run.smithi017.stdout: "w": "8" 2022-08-01T19:43:29.634 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.634 INFO:teuthology.orchestra.run.smithi017.stdout: "default": { 2022-08-01T19:43:29.635 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:43:29.635 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:43:29.635 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "1", 2022-08-01T19:43:29.635 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:43:29.635 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:43:29.636 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van" 2022-08-01T19:43:29.636 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:29.636 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:29.636 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps_queue": [], 2022-08-01T19:43:29.637 INFO:teuthology.orchestra.run.smithi017.stdout: "new_removed_snaps": [], 2022-08-01T19:43:29.637 INFO:teuthology.orchestra.run.smithi017.stdout: "new_purged_snaps": [], 2022-08-01T19:43:29.637 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_node_flags": {}, 2022-08-01T19:43:29.637 INFO:teuthology.orchestra.run.smithi017.stdout: "device_class_flags": {} 2022-08-01T19:43:29.638 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:43:29.640 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.337+0000 7f2daf59e700 1 -- 172.21.15.17:0/3503740147 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2d8c053aa0 msgr2=0x7f2d8c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:29.640 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.337+0000 7f2daf59e700 1 --2- 172.21.15.17:0/3503740147 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2d8c053aa0 0x7f2d8c055f60 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f2da000b000 tx=0x7f2da0009040).stop 2022-08-01T19:43:29.640 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.337+0000 7f2daf59e700 1 -- 172.21.15.17:0/3503740147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2db00b3190 msgr2=0x7f2db01570c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:29.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.337+0000 7f2daf59e700 1 --2- 172.21.15.17:0/3503740147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2db00b3190 0x7f2db01570c0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 rx=0x7f2da80032f0 tx=0x7f2da8016000).stop 2022-08-01T19:43:29.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.337+0000 7f2daf59e700 1 -- 172.21.15.17:0/3503740147 shutdown_connections 2022-08-01T19:43:29.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.337+0000 7f2daf59e700 1 --2- 172.21.15.17:0/3503740147 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2d8c053aa0 0x7f2d8c055f60 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:29.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.337+0000 7f2daf59e700 1 --2- 172.21.15.17:0/3503740147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2db00b3190 0x7f2db01570c0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:29.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.337+0000 7f2daf59e700 1 --2- 172.21.15.17:0/3503740147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2db00b1a80 0x7f2db0156bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:29.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.337+0000 7f2daf59e700 1 --2- 172.21.15.17:0/3503740147 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2db00a9da0 0x7f2db0066950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:29.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.337+0000 7f2daf59e700 1 -- 172.21.15.17:0/3503740147 >> 172.21.15.17:0/3503740147 conn(0x7f2db0063870 msgr2=0x7f2db0060080 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:29.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.337+0000 7f2daf59e700 1 -- 172.21.15.17:0/3503740147 shutdown_connections 2022-08-01T19:43:29.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.341+0000 7f2daf59e700 1 -- 172.21.15.17:0/3503740147 wait complete. 2022-08-01T19:43:29.644 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:43:29.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.753+0000 7fe57cf28700 1 Processor -- start 2022-08-01T19:43:29.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.753+0000 7fe57cf28700 1 -- start start 2022-08-01T19:43:29.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.753+0000 7fe57cf28700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fe578060290 0x7fe5780b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:29.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.753+0000 7fe57cf28700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5780aad40 con 0x7fe5780b4dd0 2022-08-01T19:43:29.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.753+0000 7fe57cf28700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe578060670 con 0x7fe578067490 2022-08-01T19:43:29.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.753+0000 7fe57cf28700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe578067030 con 0x7fe578060290 2022-08-01T19:43:29.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.753+0000 7fe576d9d700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe5780b4dd0 0x7fe57814b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36924/0 (socket says 172.21.15.17:36924) 2022-08-01T19:43:29.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.753+0000 7fe576d9d700 1 -- 172.21.15.17:0/912223585 learned_addr learned my addr 172.21.15.17:0/912223585 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:29.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.753+0000 7fe57659c700 1 --2- 172.21.15.17:0/912223585 >> v2:172.21.15.77:3300/0 conn(0x7fe578060290 0x7fe5780b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:29.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.753+0000 7fe57559a700 1 -- 172.21.15.17:0/912223585 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1704686664 0 0) 0x7fe5780aad40 con 0x7fe5780b4dd0 2022-08-01T19:43:29.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.753+0000 7fe57559a700 1 -- 172.21.15.17:0/912223585 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe55c003260 con 0x7fe5780b4dd0 2022-08-01T19:43:29.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.753+0000 7fe57559a700 1 -- 172.21.15.17:0/912223585 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1509285588 0 0) 0x7fe578060670 con 0x7fe578067490 2022-08-01T19:43:29.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.753+0000 7fe57559a700 1 -- 172.21.15.17:0/912223585 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5780aad40 con 0x7fe578067490 2022-08-01T19:43:29.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.753+0000 7fe57659c700 1 -- 172.21.15.17:0/912223585 >> v1:172.21.15.77:6789/0 conn(0x7fe578067490 legacy=0x7fe578143a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:29.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.753+0000 7fe57659c700 1 -- 172.21.15.17:0/912223585 >> v1:172.21.15.17:6789/0 conn(0x7fe5780b4dd0 legacy=0x7fe57814b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:29.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.753+0000 7fe57659c700 1 -- 172.21.15.17:0/912223585 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5780b3540 con 0x7fe578060290 2022-08-01T19:43:29.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.753+0000 7fe57659c700 1 --2- 172.21.15.17:0/912223585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe578060290 0x7fe5780b48e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7fe568002f70 tx=0x7fe568006df0).ready entity=mon.1 client_cookie=ffac572c1870d0c2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:29.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.753+0000 7fe57559a700 1 -- 172.21.15.17:0/912223585 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe56800d540 con 0x7fe578060290 2022-08-01T19:43:29.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.753+0000 7fe57559a700 1 -- 172.21.15.17:0/912223585 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe568000e90 con 0x7fe578060290 2022-08-01T19:43:29.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.753+0000 7fe57559a700 1 -- 172.21.15.17:0/912223585 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe56800d890 con 0x7fe578060290 2022-08-01T19:43:29.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.757+0000 7fe57cf28700 1 -- 172.21.15.17:0/912223585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe578060290 msgr2=0x7fe5780b48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:29.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.757+0000 7fe57cf28700 1 --2- 172.21.15.17:0/912223585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe578060290 0x7fe5780b48e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7fe568002f70 tx=0x7fe568006df0).stop 2022-08-01T19:43:29.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.757+0000 7fe57cf28700 1 -- 172.21.15.17:0/912223585 shutdown_connections 2022-08-01T19:43:29.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.757+0000 7fe57cf28700 1 --2- 172.21.15.17:0/912223585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe578060290 0x7fe5780b48e0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:29.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.757+0000 7fe57cf28700 1 -- 172.21.15.17:0/912223585 >> 172.21.15.17:0/912223585 conn(0x7fe57805d0c0 msgr2=0x7fe57805f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:29.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.757+0000 7fe57cf28700 1 -- 172.21.15.17:0/912223585 shutdown_connections 2022-08-01T19:43:29.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.757+0000 7fe57cf28700 1 -- 172.21.15.17:0/912223585 wait complete. 2022-08-01T19:43:29.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.757+0000 7fe57cf28700 1 Processor -- start 2022-08-01T19:43:29.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.757+0000 7fe57cf28700 1 -- start start 2022-08-01T19:43:29.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.757+0000 7fe57cf28700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe578060290 0x7fe5780b63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:29.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.757+0000 7fe57cf28700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe578067490 0x7fe578153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:29.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.757+0000 7fe57cf28700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5780b4dd0 0x7fe578154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:29.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.757+0000 7fe57cf28700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5780ad510 con 0x7fe5780b4dd0 2022-08-01T19:43:29.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.757+0000 7fe57cf28700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5780ad7b0 con 0x7fe578067490 2022-08-01T19:43:29.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.757+0000 7fe57cf28700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5780ada50 con 0x7fe578060290 2022-08-01T19:43:29.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.757+0000 7fe576d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5780b4dd0 0x7fe578154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:29.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.757+0000 7fe57659c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe578060290 0x7fe5780b63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:29.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.757+0000 7fe57659c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe578060290 0x7fe5780b63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:40832/0 (socket says 172.21.15.17:40832) 2022-08-01T19:43:29.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.757+0000 7fe57659c700 1 -- 172.21.15.17:0/184837580 learned_addr learned my addr 172.21.15.17:0/184837580 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:29.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.757+0000 7fe575d9b700 1 --2- 172.21.15.17:0/184837580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe578067490 0x7fe578153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:29.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.761+0000 7fe57659c700 1 -- 172.21.15.17:0/184837580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe578067490 msgr2=0x7fe578153b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:29.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.761+0000 7fe57659c700 1 --2- 172.21.15.17:0/184837580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe578067490 0x7fe578153b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:29.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.761+0000 7fe57659c700 1 -- 172.21.15.17:0/184837580 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5780b4dd0 msgr2=0x7fe578154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:29.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.761+0000 7fe57659c700 1 --2- 172.21.15.17:0/184837580 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5780b4dd0 0x7fe578154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:29.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.761+0000 7fe57659c700 1 -- 172.21.15.17:0/184837580 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe56800b180 con 0x7fe578060290 2022-08-01T19:43:29.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.761+0000 7fe576d9d700 1 --2- 172.21.15.17:0/184837580 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5780b4dd0 0x7fe578154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:43:29.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.761+0000 7fe57659c700 1 --2- 172.21.15.17:0/184837580 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe578060290 0x7fe5780b63b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7fe568002f70 tx=0x7fe568014000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:29.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.761+0000 7fe56f7fe700 1 -- 172.21.15.17:0/184837580 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe56800d1c0 con 0x7fe578060290 2022-08-01T19:43:29.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.761+0000 7fe56f7fe700 1 -- 172.21.15.17:0/184837580 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe568013bf0 con 0x7fe578060290 2022-08-01T19:43:29.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.761+0000 7fe57cf28700 1 -- 172.21.15.17:0/184837580 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5780aca90 con 0x7fe578060290 2022-08-01T19:43:29.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.761+0000 7fe56f7fe700 1 -- 172.21.15.17:0/184837580 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe56801e400 con 0x7fe578060290 2022-08-01T19:43:29.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.761+0000 7fe57cf28700 1 -- 172.21.15.17:0/184837580 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe578154560 con 0x7fe578060290 2022-08-01T19:43:29.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.761+0000 7fe57cf28700 1 -- 172.21.15.17:0/184837580 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5780574e0 con 0x7fe578060290 2022-08-01T19:43:29.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.761+0000 7fe56f7fe700 1 -- 172.21.15.17:0/184837580 <== mon.2 v2:172.21.15.77:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe568025030 con 0x7fe578060290 2022-08-01T19:43:29.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.761+0000 7fe56f7fe700 1 --2- 172.21.15.17:0/184837580 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe550053b30 0x7fe550055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:29.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.765+0000 7fe56f7fe700 1 -- 172.21.15.17:0/184837580 <== mon.2 v2:172.21.15.77:3301/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fe56806cf40 con 0x7fe578060290 2022-08-01T19:43:29.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.765+0000 7fe575d9b700 1 --2- 172.21.15.17:0/184837580 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe550053b30 0x7fe550055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:29.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.765+0000 7fe575d9b700 1 --2- 172.21.15.17:0/184837580 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe550053b30 0x7fe550055ff0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7fe57000d000 tx=0x7fe570007da0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:29.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.769+0000 7fe56f7fe700 1 -- 172.21.15.17:0/184837580 <== mon.2 v2:172.21.15.77:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe568043ed0 con 0x7fe578060290 2022-08-01T19:43:29.936 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.933+0000 7fe57cf28700 1 -- 172.21.15.17:0/184837580 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe578069a70 con 0x7fe578060290 2022-08-01T19:43:29.937 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.933+0000 7fe56f7fe700 1 -- 172.21.15.17:0/184837580 <== mon.2 v2:172.21.15.77:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+1556 (secure 0 0 0) 0x7fe5680474f0 con 0x7fe578060290 2022-08-01T19:43:29.938 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:29.938 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":72,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7157,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/2533527414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2533527414},{"type":"v1","addr":"172.21.15.77:6835","nonce":2533527414}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72},{"gid":7995,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.17:6833/4164640472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4164640472},{"type":"v1","addr":"172.21.15.17:6833","nonce":4164640472}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":67},{"gid":8001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/50015801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":50015801},{"type":"v1","addr":"172.21.15.17:6835","nonce":50015801}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66}],"filesystems":[]} 2022-08-01T19:43:29.940 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.937+0000 7fe57cf28700 1 -- 172.21.15.17:0/184837580 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe550053b30 msgr2=0x7fe550055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:29.940 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.937+0000 7fe57cf28700 1 --2- 172.21.15.17:0/184837580 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe550053b30 0x7fe550055ff0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7fe57000d000 tx=0x7fe570007da0).stop 2022-08-01T19:43:29.940 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.937+0000 7fe57cf28700 1 -- 172.21.15.17:0/184837580 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe578060290 msgr2=0x7fe5780b63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:29.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.937+0000 7fe57cf28700 1 --2- 172.21.15.17:0/184837580 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe578060290 0x7fe5780b63b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7fe568002f70 tx=0x7fe568014000).stop 2022-08-01T19:43:29.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.937+0000 7fe57cf28700 1 -- 172.21.15.17:0/184837580 shutdown_connections 2022-08-01T19:43:29.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.937+0000 7fe57cf28700 1 --2- 172.21.15.17:0/184837580 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe550053b30 0x7fe550055ff0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:29.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.937+0000 7fe57cf28700 1 --2- 172.21.15.17:0/184837580 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5780b4dd0 0x7fe578154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:29.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.937+0000 7fe57cf28700 1 --2- 172.21.15.17:0/184837580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe578067490 0x7fe578153b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:29.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.937+0000 7fe57cf28700 1 --2- 172.21.15.17:0/184837580 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe578060290 0x7fe5780b63b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:29.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.937+0000 7fe57cf28700 1 -- 172.21.15.17:0/184837580 >> 172.21.15.17:0/184837580 conn(0x7fe57805d0c0 msgr2=0x7fe5780af050 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:29.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.937+0000 7fe57cf28700 1 -- 172.21.15.17:0/184837580 shutdown_connections 2022-08-01T19:43:29.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:29.937+0000 7fe57cf28700 1 -- 172.21.15.17:0/184837580 wait complete. 2022-08-01T19:43:29.944 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 72 2022-08-01T19:43:29.952 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:43:30.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.061+0000 7f534caf8700 1 Processor -- start 2022-08-01T19:43:30.063 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.061+0000 7f534caf8700 1 -- start start 2022-08-01T19:43:30.064 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.061+0000 7f534caf8700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f5340042530 0x7f5340005660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:30.064 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.061+0000 7f534caf8700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5340040c90 0x7f53400400c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:30.065 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.061+0000 7f534caf8700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5340041640 con 0x7f534003f590 2022-08-01T19:43:30.065 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.061+0000 7f534caf8700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f534003e450 con 0x7f5340042530 2022-08-01T19:43:30.065 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.061+0000 7f534caf8700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f534003f1a0 con 0x7f5340040c90 2022-08-01T19:43:30.065 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.061+0000 7f5347fff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f534003f590 0x7f53400d8950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36938/0 (socket says 172.21.15.17:36938) 2022-08-01T19:43:30.066 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.061+0000 7f5347fff700 1 -- 172.21.15.17:0/2470740084 learned_addr learned my addr 172.21.15.17:0/2470740084 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:30.066 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.061+0000 7f5346ffd700 1 --2- 172.21.15.17:0/2470740084 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5340040c90 0x7f53400400c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:30.066 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.061+0000 7f53477fe700 1 --2- 172.21.15.17:0/2470740084 >> v2:172.21.15.77:3300/0 conn(0x7f5340042530 0x7f5340005660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:30.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.061+0000 7f53467fc700 1 -- 172.21.15.17:0/2470740084 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 857612972 0 0) 0x7f5340041640 con 0x7f534003f590 2022-08-01T19:43:30.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.061+0000 7f53467fc700 1 -- 172.21.15.17:0/2470740084 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f532c003260 con 0x7f534003f590 2022-08-01T19:43:30.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.061+0000 7f5346ffd700 1 -- 172.21.15.17:0/2470740084 >> v2:172.21.15.77:3300/0 conn(0x7f5340042530 msgr2=0x7f5340005660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:30.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.061+0000 7f5346ffd700 1 --2- 172.21.15.17:0/2470740084 >> v2:172.21.15.77:3300/0 conn(0x7f5340042530 0x7f5340005660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:30.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.061+0000 7f5346ffd700 1 -- 172.21.15.17:0/2470740084 >> v1:172.21.15.17:6789/0 conn(0x7f534003f590 legacy=0x7f53400d8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:30.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.061+0000 7f5346ffd700 1 -- 172.21.15.17:0/2470740084 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5340042910 con 0x7f5340040c90 2022-08-01T19:43:30.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.061+0000 7f5346ffd700 1 --2- 172.21.15.17:0/2470740084 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5340040c90 0x7f53400400c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f533c002f70 tx=0x7f533c006df0).ready entity=mon.2 client_cookie=71240bdd267f412e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:30.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.061+0000 7f53467fc700 1 -- 172.21.15.17:0/2470740084 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f533c00d4a0 con 0x7f5340040c90 2022-08-01T19:43:30.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.061+0000 7f53467fc700 1 -- 172.21.15.17:0/2470740084 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f533c000e90 con 0x7f5340040c90 2022-08-01T19:43:30.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.061+0000 7f53467fc700 1 -- 172.21.15.17:0/2470740084 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f533c00d7f0 con 0x7f5340040c90 2022-08-01T19:43:30.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.061+0000 7f534caf8700 1 -- 172.21.15.17:0/2470740084 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5340040c90 msgr2=0x7f53400400c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:30.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.061+0000 7f534caf8700 1 --2- 172.21.15.17:0/2470740084 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5340040c90 0x7f53400400c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f533c002f70 tx=0x7f533c006df0).stop 2022-08-01T19:43:30.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.061+0000 7f534caf8700 1 -- 172.21.15.17:0/2470740084 shutdown_connections 2022-08-01T19:43:30.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.061+0000 7f534caf8700 1 --2- 172.21.15.17:0/2470740084 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5340040c90 0x7f53400400c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:30.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.061+0000 7f534caf8700 1 --2- 172.21.15.17:0/2470740084 >> v2:172.21.15.77:3300/0 conn(0x7f5340042530 0x7f5340005660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:30.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.061+0000 7f534caf8700 1 -- 172.21.15.17:0/2470740084 >> 172.21.15.17:0/2470740084 conn(0x7f53400462f0 msgr2=0x7f5340046700 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:30.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.061+0000 7f534caf8700 1 -- 172.21.15.17:0/2470740084 shutdown_connections 2022-08-01T19:43:30.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.061+0000 7f534caf8700 1 -- 172.21.15.17:0/2470740084 wait complete. 2022-08-01T19:43:30.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.065+0000 7f534caf8700 1 Processor -- start 2022-08-01T19:43:30.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.065+0000 7f534caf8700 1 -- start start 2022-08-01T19:43:30.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.065+0000 7f534caf8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f534003f590 0x7f53400e7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:30.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.065+0000 7f534caf8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5340040c90 0x7f53400451a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:30.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.065+0000 7f534caf8700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5340042530 0x7f53400eb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:30.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.065+0000 7f534caf8700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53400ea980 con 0x7f5340040c90 2022-08-01T19:43:30.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.065+0000 7f534caf8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53400a18a0 con 0x7f534003f590 2022-08-01T19:43:30.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.065+0000 7f534caf8700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f534009f4e0 con 0x7f5340042530 2022-08-01T19:43:30.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.065+0000 7f5346ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5340040c90 0x7f53400451a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:30.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.065+0000 7f5346ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5340040c90 0x7f53400451a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44746/0 (socket says 172.21.15.17:44746) 2022-08-01T19:43:30.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.065+0000 7f5346ffd700 1 -- 172.21.15.17:0/4169911186 learned_addr learned my addr 172.21.15.17:0/4169911186 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:30.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.065+0000 7f53477fe700 1 --2- 172.21.15.17:0/4169911186 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f534003f590 0x7f53400e7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:30.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.065+0000 7f5347fff700 1 --2- 172.21.15.17:0/4169911186 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5340042530 0x7f53400eb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:30.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.065+0000 7f5346ffd700 1 -- 172.21.15.17:0/4169911186 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5340042530 msgr2=0x7f53400eb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:30.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.065+0000 7f5346ffd700 1 --2- 172.21.15.17:0/4169911186 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5340042530 0x7f53400eb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:30.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.065+0000 7f5346ffd700 1 -- 172.21.15.17:0/4169911186 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f534003f590 msgr2=0x7f53400e7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:30.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.065+0000 7f5346ffd700 1 --2- 172.21.15.17:0/4169911186 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f534003f590 0x7f53400e7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:30.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.065+0000 7f5346ffd700 1 -- 172.21.15.17:0/4169911186 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f533c00b180 con 0x7f5340040c90 2022-08-01T19:43:30.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.065+0000 7f5346ffd700 1 --2- 172.21.15.17:0/4169911186 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5340040c90 0x7f53400451a0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 rx=0x7f533c002f70 tx=0x7f533c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:30.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.065+0000 7f5344ff9700 1 -- 172.21.15.17:0/4169911186 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f533c016070 con 0x7f5340040c90 2022-08-01T19:43:30.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.065+0000 7f5344ff9700 1 -- 172.21.15.17:0/4169911186 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f533c0138d0 con 0x7f5340040c90 2022-08-01T19:43:30.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.065+0000 7f5344ff9700 1 -- 172.21.15.17:0/4169911186 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f533c013c30 con 0x7f5340040c90 2022-08-01T19:43:30.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.065+0000 7f534caf8700 1 -- 172.21.15.17:0/4169911186 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53400eabb0 con 0x7f5340040c90 2022-08-01T19:43:30.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.065+0000 7f534caf8700 1 -- 172.21.15.17:0/4169911186 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f534009f780 con 0x7f5340040c90 2022-08-01T19:43:30.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.069+0000 7f5344ff9700 1 -- 172.21.15.17:0/4169911186 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f533c00f030 con 0x7f5340040c90 2022-08-01T19:43:30.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.069+0000 7f534caf8700 1 -- 172.21.15.17:0/4169911186 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f534003a320 con 0x7f5340040c90 2022-08-01T19:43:30.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.069+0000 7f5344ff9700 1 --2- 172.21.15.17:0/4169911186 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5328053a10 0x7f5328055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:30.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.069+0000 7f5344ff9700 1 -- 172.21.15.17:0/4169911186 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f533c06ca40 con 0x7f5340040c90 2022-08-01T19:43:30.111 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.069+0000 7f53477fe700 1 --2- 172.21.15.17:0/4169911186 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5328053a10 0x7f5328055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:30.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.069+0000 7f53477fe700 1 --2- 172.21.15.17:0/4169911186 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5328053a10 0x7f5328055ed0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f533400b000 tx=0x7f5334006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:30.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.073+0000 7f5344ff9700 1 -- 172.21.15.17:0/4169911186 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f533c043ae0 con 0x7f5340040c90 2022-08-01T19:43:30.236 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.233+0000 7f534caf8700 1 -- 172.21.15.17:0/4169911186 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53400ead40 con 0x7f5340040c90 2022-08-01T19:43:30.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.237+0000 7f5344ff9700 1 -- 172.21.15.17:0/4169911186 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+1556 (secure 0 0 0) 0x7f533c023070 con 0x7f5340040c90 2022-08-01T19:43:30.244 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:30.244 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":72,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7157,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/2533527414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":2533527414},{"type":"v1","addr":"172.21.15.77:6835","nonce":2533527414}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72},{"gid":7995,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.17:6833/4164640472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4164640472},{"type":"v1","addr":"172.21.15.17:6833","nonce":4164640472}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":67},{"gid":8001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/50015801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":50015801},{"type":"v1","addr":"172.21.15.17:6835","nonce":50015801}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66}],"filesystems":[]} 2022-08-01T19:43:30.246 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.241+0000 7f534caf8700 1 -- 172.21.15.17:0/4169911186 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5328053a10 msgr2=0x7f5328055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:30.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.241+0000 7f534caf8700 1 --2- 172.21.15.17:0/4169911186 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5328053a10 0x7f5328055ed0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f533400b000 tx=0x7f5334006df0).stop 2022-08-01T19:43:30.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.241+0000 7f534caf8700 1 -- 172.21.15.17:0/4169911186 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5340040c90 msgr2=0x7f53400451a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:30.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.241+0000 7f534caf8700 1 --2- 172.21.15.17:0/4169911186 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5340040c90 0x7f53400451a0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 rx=0x7f533c002f70 tx=0x7f533c014000).stop 2022-08-01T19:43:30.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.241+0000 7f534caf8700 1 -- 172.21.15.17:0/4169911186 shutdown_connections 2022-08-01T19:43:30.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.241+0000 7f534caf8700 1 --2- 172.21.15.17:0/4169911186 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5328053a10 0x7f5328055ed0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:30.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.241+0000 7f534caf8700 1 --2- 172.21.15.17:0/4169911186 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5340042530 0x7f53400eb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:30.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.241+0000 7f534caf8700 1 --2- 172.21.15.17:0/4169911186 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5340040c90 0x7f53400451a0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:30.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.241+0000 7f534caf8700 1 --2- 172.21.15.17:0/4169911186 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f534003f590 0x7f53400e7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:30.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.241+0000 7f534caf8700 1 -- 172.21.15.17:0/4169911186 >> 172.21.15.17:0/4169911186 conn(0x7f53400462f0 msgr2=0x7f534003f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:30.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.241+0000 7f534caf8700 1 -- 172.21.15.17:0/4169911186 shutdown_connections 2022-08-01T19:43:30.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.241+0000 7f534caf8700 1 -- 172.21.15.17:0/4169911186 wait complete. 2022-08-01T19:43:30.252 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 72 2022-08-01T19:43:30.258 INFO:tasks.ceph.mds.a:Restarting daemon 2022-08-01T19:43:30.259 INFO:tasks.ceph.mds.a:Stopping old one... 2022-08-01T19:43:30.260 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-08-01T19:43:30.260 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:43:30.261 INFO:tasks.ceph.mds.c:Restarting daemon 2022-08-01T19:43:30.261 INFO:tasks.ceph.mds.c:Stopping old one... 2022-08-01T19:43:30.262 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-08-01T19:43:30.262 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:43:30.265 INFO:tasks.ceph.mds.b:Restarting daemon 2022-08-01T19:43:30.266 INFO:tasks.ceph.mds.b:Stopping old one... 2022-08-01T19:43:30.266 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-08-01T19:43:30.267 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:43:30.275 INFO:tasks.ceph.mds.a:Stopped 2022-08-01T19:43:30.277 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-08-01T19:43:30.278 INFO:tasks.ceph.mds.c:Stopped 2022-08-01T19:43:30.279 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-08-01T19:43:30.280 INFO:tasks.ceph.mds.a:Started 2022-08-01T19:43:30.281 DEBUG:teuthology.parallel:result is None 2022-08-01T19:43:30.282 INFO:tasks.ceph.mds.c:Started 2022-08-01T19:43:30.283 DEBUG:teuthology.parallel:result is None 2022-08-01T19:43:30.284 INFO:tasks.ceph.mds.b:Stopped 2022-08-01T19:43:30.285 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-08-01T19:43:30.293 INFO:tasks.ceph.mds.b:Started 2022-08-01T19:43:30.294 DEBUG:teuthology.parallel:result is None 2022-08-01T19:43:30.295 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-08-01T19:43:30.347 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2022-08-01T19:43:30.358 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2022-08-01T19:43:30.371 INFO:tasks.ceph.mds.b.smithi077.stdout:starting mds.b at 2022-08-01T19:43:30.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.389+0000 7f7b86088700 1 Processor -- start 2022-08-01T19:43:30.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.389+0000 7f7b86088700 1 -- start start 2022-08-01T19:43:30.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.389+0000 7f7b86088700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7b80066f90 0x7f7b800ab770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:30.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.389+0000 7f7b86088700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7b80143cc0 0x7f7b8014e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:30.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.389+0000 7f7b86088700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b80146b10 con 0x7f7b800abc60 2022-08-01T19:43:30.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.389+0000 7f7b86088700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b800a99a0 con 0x7f7b80143cc0 2022-08-01T19:43:30.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.389+0000 7f7b86088700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b80067370 con 0x7f7b80066f90 2022-08-01T19:43:30.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.389+0000 7f7b7effd700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f7b800abc60 0x7f7b801435b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36980/0 (socket says 172.21.15.17:36980) 2022-08-01T19:43:30.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.389+0000 7f7b7effd700 1 -- 172.21.15.17:0/544731620 learned_addr learned my addr 172.21.15.17:0/544731620 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:30.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.389+0000 7f7b7f7fe700 1 --2- 172.21.15.17:0/544731620 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7b80066f90 0x7f7b800ab770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:30.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.389+0000 7f7b7ffff700 1 --2- 172.21.15.17:0/544731620 >> v2:172.21.15.17:3300/0 conn(0x7f7b80143cc0 0x7f7b8014e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:30.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.389+0000 7f7b7f7fe700 1 -- 172.21.15.17:0/544731620 >> v2:172.21.15.17:3300/0 conn(0x7f7b80143cc0 msgr2=0x7f7b8014e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:30.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.389+0000 7f7b7f7fe700 1 --2- 172.21.15.17:0/544731620 >> v2:172.21.15.17:3300/0 conn(0x7f7b80143cc0 0x7f7b8014e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:30.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.389+0000 7f7b7f7fe700 1 -- 172.21.15.17:0/544731620 >> v1:172.21.15.17:6789/0 conn(0x7f7b800abc60 legacy=0x7f7b801435b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:30.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.389+0000 7f7b7f7fe700 1 -- 172.21.15.17:0/544731620 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b800a95e0 con 0x7f7b80066f90 2022-08-01T19:43:30.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.389+0000 7f7b7ffff700 1 --2- 172.21.15.17:0/544731620 >> v2:172.21.15.17:3300/0 conn(0x7f7b80143cc0 0x7f7b8014e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:43:30.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.389+0000 7f7b7f7fe700 1 --2- 172.21.15.17:0/544731620 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7b80066f90 0x7f7b800ab770 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f7b780032f0 tx=0x7f7b780091d0).ready entity=mon.2 client_cookie=17df7860c132c92e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:30.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.389+0000 7f7b7e7fc700 1 -- 172.21.15.17:0/544731620 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b7800b6c0 con 0x7f7b80066f90 2022-08-01T19:43:30.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.389+0000 7f7b7e7fc700 1 -- 172.21.15.17:0/544731620 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7800b820 con 0x7f7b80066f90 2022-08-01T19:43:30.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.389+0000 7f7b7e7fc700 1 -- 172.21.15.17:0/544731620 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b7800b990 con 0x7f7b80066f90 2022-08-01T19:43:30.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.393+0000 7f7b86088700 1 -- 172.21.15.17:0/544731620 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7b80066f90 msgr2=0x7f7b800ab770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:30.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.393+0000 7f7b86088700 1 --2- 172.21.15.17:0/544731620 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7b80066f90 0x7f7b800ab770 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f7b780032f0 tx=0x7f7b780091d0).stop 2022-08-01T19:43:30.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.393+0000 7f7b86088700 1 -- 172.21.15.17:0/544731620 shutdown_connections 2022-08-01T19:43:30.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.393+0000 7f7b86088700 1 --2- 172.21.15.17:0/544731620 >> v2:172.21.15.17:3300/0 conn(0x7f7b80143cc0 0x7f7b8014e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:30.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.393+0000 7f7b86088700 1 --2- 172.21.15.17:0/544731620 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7b80066f90 0x7f7b800ab770 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:30.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.393+0000 7f7b86088700 1 -- 172.21.15.17:0/544731620 >> 172.21.15.17:0/544731620 conn(0x7f7b800b5570 msgr2=0x7f7b800b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:30.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.393+0000 7f7b86088700 1 -- 172.21.15.17:0/544731620 shutdown_connections 2022-08-01T19:43:30.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.393+0000 7f7b86088700 1 -- 172.21.15.17:0/544731620 wait complete. 2022-08-01T19:43:30.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.393+0000 7f7b86088700 1 Processor -- start 2022-08-01T19:43:30.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.393+0000 7f7b86088700 1 -- start start 2022-08-01T19:43:30.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.397+0000 7f7b86088700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7b80066f90 0x7f7b80153400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:30.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.397+0000 7f7b86088700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b800abc60 0x7f7b800aff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:30.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.397+0000 7f7b86088700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b80143cc0 0x7f7b800ae830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:30.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.397+0000 7f7b86088700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b80152a20 con 0x7f7b800abc60 2022-08-01T19:43:30.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.397+0000 7f7b86088700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b80152cc0 con 0x7f7b80143cc0 2022-08-01T19:43:30.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.397+0000 7f7b86088700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b80152f60 con 0x7f7b80066f90 2022-08-01T19:43:30.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.397+0000 7f7b7effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b800abc60 0x7f7b800aff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:30.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.397+0000 7f7b7effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b800abc60 0x7f7b800aff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44792/0 (socket says 172.21.15.17:44792) 2022-08-01T19:43:30.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.397+0000 7f7b7effd700 1 -- 172.21.15.17:0/3712380253 learned_addr learned my addr 172.21.15.17:0/3712380253 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:30.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.397+0000 7f7b7f7fe700 1 --2- 172.21.15.17:0/3712380253 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7b80066f90 0x7f7b80153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:30.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.397+0000 7f7b7ffff700 1 --2- 172.21.15.17:0/3712380253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b80143cc0 0x7f7b800ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:30.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.397+0000 7f7b7effd700 1 -- 172.21.15.17:0/3712380253 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7b80066f90 msgr2=0x7f7b80153400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:30.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.397+0000 7f7b7effd700 1 --2- 172.21.15.17:0/3712380253 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7b80066f90 0x7f7b80153400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:30.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.397+0000 7f7b7effd700 1 -- 172.21.15.17:0/3712380253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b80143cc0 msgr2=0x7f7b800ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:30.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.397+0000 7f7b7effd700 1 --2- 172.21.15.17:0/3712380253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b80143cc0 0x7f7b800ae830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:30.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.397+0000 7f7b7effd700 1 -- 172.21.15.17:0/3712380253 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b7000b180 con 0x7f7b800abc60 2022-08-01T19:43:30.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.397+0000 7f7b7ffff700 1 --2- 172.21.15.17:0/3712380253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b80143cc0 0x7f7b800ae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:43:30.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.397+0000 7f7b7effd700 1 --2- 172.21.15.17:0/3712380253 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b800abc60 0x7f7b800aff90 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 rx=0x7f7b8005f350 tx=0x7f7b68007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:30.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.397+0000 7f7b7cff9700 1 -- 172.21.15.17:0/3712380253 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b6800ce30 con 0x7f7b800abc60 2022-08-01T19:43:30.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.397+0000 7f7b7cff9700 1 -- 172.21.15.17:0/3712380253 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b680072e0 con 0x7f7b800abc60 2022-08-01T19:43:30.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.397+0000 7f7b86088700 1 -- 172.21.15.17:0/3712380253 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b78009470 con 0x7f7b800abc60 2022-08-01T19:43:30.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.397+0000 7f7b7cff9700 1 -- 172.21.15.17:0/3712380253 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b68005800 con 0x7f7b800abc60 2022-08-01T19:43:30.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.397+0000 7f7b86088700 1 -- 172.21.15.17:0/3712380253 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b800b0480 con 0x7f7b800abc60 2022-08-01T19:43:30.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.397+0000 7f7b7cff9700 1 -- 172.21.15.17:0/3712380253 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7b68010070 con 0x7f7b800abc60 2022-08-01T19:43:30.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.397+0000 7f7b7cff9700 1 --2- 172.21.15.17:0/3712380253 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7b64053b30 0x7f7b64055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:30.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.397+0000 7f7b7cff9700 1 -- 172.21.15.17:0/3712380253 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f7b6806b330 con 0x7f7b800abc60 2022-08-01T19:43:30.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.397+0000 7f7b86088700 1 -- 172.21.15.17:0/3712380253 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b80153120 con 0x7f7b800abc60 2022-08-01T19:43:30.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.401+0000 7f7b7f7fe700 1 --2- 172.21.15.17:0/3712380253 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7b64053b30 0x7f7b64055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:30.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.401+0000 7f7b7f7fe700 1 --2- 172.21.15.17:0/3712380253 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7b64053b30 0x7f7b64055ff0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f7b78019000 tx=0x7f7b78016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:30.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.401+0000 7f7b7cff9700 1 -- 172.21.15.17:0/3712380253 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7b68018020 con 0x7f7b800abc60 2022-08-01T19:43:30.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:30.541+0000 7f7b86088700 1 -- 172.21.15.17:0/3712380253 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f7b800aaaa0 con 0x7f7b800abc60 2022-08-01T19:43:31.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.393+0000 7f7b7cff9700 1 -- 172.21.15.17:0/3712380253 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v143) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7b68018020 con 0x7f7b800abc60 2022-08-01T19:43:31.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.397+0000 7f7b86088700 1 -- 172.21.15.17:0/3712380253 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7b64053b30 msgr2=0x7f7b64055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:31.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.397+0000 7f7b86088700 1 --2- 172.21.15.17:0/3712380253 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7b64053b30 0x7f7b64055ff0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f7b78019000 tx=0x7f7b78016000).stop 2022-08-01T19:43:31.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.397+0000 7f7b86088700 1 -- 172.21.15.17:0/3712380253 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b800abc60 msgr2=0x7f7b800aff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:31.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.397+0000 7f7b86088700 1 --2- 172.21.15.17:0/3712380253 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b800abc60 0x7f7b800aff90 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 rx=0x7f7b8005f350 tx=0x7f7b68007c00).stop 2022-08-01T19:43:31.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.397+0000 7f7b86088700 1 -- 172.21.15.17:0/3712380253 shutdown_connections 2022-08-01T19:43:31.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.397+0000 7f7b86088700 1 --2- 172.21.15.17:0/3712380253 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7b64053b30 0x7f7b64055ff0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:31.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.397+0000 7f7b86088700 1 --2- 172.21.15.17:0/3712380253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b80143cc0 0x7f7b800ae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:31.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.397+0000 7f7b86088700 1 --2- 172.21.15.17:0/3712380253 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b800abc60 0x7f7b800aff90 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:31.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.397+0000 7f7b86088700 1 --2- 172.21.15.17:0/3712380253 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7b80066f90 0x7f7b80153400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:31.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.397+0000 7f7b86088700 1 -- 172.21.15.17:0/3712380253 >> 172.21.15.17:0/3712380253 conn(0x7f7b800b5570 msgr2=0x7f7b80148c90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:31.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.397+0000 7f7b86088700 1 -- 172.21.15.17:0/3712380253 shutdown_connections 2022-08-01T19:43:31.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.397+0000 7f7b86088700 1 -- 172.21.15.17:0/3712380253 wait complete. 2022-08-01T19:43:31.407 INFO:teuthology.orchestra.run.smithi017.stderr: removed all blacklist entries 2022-08-01T19:43:31.415 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-08-01T19:43:31.416 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-08-01T19:43:31.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.525+0000 7f9b9eb41700 1 Processor -- start 2022-08-01T19:43:31.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.525+0000 7f9b9eb41700 1 -- start start 2022-08-01T19:43:31.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.525+0000 7f9b9eb41700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9b980b1ea0 0x7f9b98147600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:31.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.525+0000 7f9b9eb41700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f9b9805c620 0x7f9b980a8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:31.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.525+0000 7f9b9eb41700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b9814ba70 con 0x7f9b980b2930 2022-08-01T19:43:31.538 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.525+0000 7f9b9eb41700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b98145340 con 0x7f9b9805c620 2022-08-01T19:43:31.538 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.525+0000 7f9b9eb41700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b980b2d10 con 0x7f9b980b1ea0 2022-08-01T19:43:31.538 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.525+0000 7f9b9db3f700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9b980b2930 0x7f9b9814b360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36996/0 (socket says 172.21.15.17:36996) 2022-08-01T19:43:31.539 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.525+0000 7f9b9db3f700 1 -- 172.21.15.17:0/3364748433 learned_addr learned my addr 172.21.15.17:0/3364748433 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:31.539 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.525+0000 7f9b9cb3d700 1 -- 172.21.15.17:0/3364748433 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3290549246 0 0) 0x7f9b9814ba70 con 0x7f9b980b2930 2022-08-01T19:43:31.539 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.525+0000 7f9b9d33e700 1 --2- 172.21.15.17:0/3364748433 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9b980b1ea0 0x7f9b98147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:31.539 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.525+0000 7f9b9cb3d700 1 -- 172.21.15.17:0/3364748433 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b78003260 con 0x7f9b980b2930 2022-08-01T19:43:31.540 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.525+0000 7f9b9e340700 1 --2- 172.21.15.17:0/3364748433 >> v2:172.21.15.77:3300/0 conn(0x7f9b9805c620 0x7f9b980a8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:31.540 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.525+0000 7f9b9cb3d700 1 -- 172.21.15.17:0/3364748433 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3463052803 0 0) 0x7f9b78003260 con 0x7f9b980b2930 2022-08-01T19:43:31.540 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.525+0000 7f9b9cb3d700 1 -- 172.21.15.17:0/3364748433 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9b980b1ea0 msgr2=0x7f9b98147600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:31.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.525+0000 7f9b9cb3d700 1 --2- 172.21.15.17:0/3364748433 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9b980b1ea0 0x7f9b98147600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:31.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.525+0000 7f9b9cb3d700 1 -- 172.21.15.17:0/3364748433 >> v2:172.21.15.77:3300/0 conn(0x7f9b9805c620 msgr2=0x7f9b980a8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:31.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.525+0000 7f9b9cb3d700 1 --2- 172.21.15.17:0/3364748433 >> v2:172.21.15.77:3300/0 conn(0x7f9b9805c620 0x7f9b980a8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:31.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.525+0000 7f9b9cb3d700 1 -- 172.21.15.17:0/3364748433 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9814ac60 con 0x7f9b980b2930 2022-08-01T19:43:31.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.525+0000 7f9b9cb3d700 1 -- 172.21.15.17:0/3364748433 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7f9b880019e0 con 0x7f9b980b2930 2022-08-01T19:43:31.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.525+0000 7f9b9cb3d700 1 -- 172.21.15.17:0/3364748433 >> v1:172.21.15.17:6789/0 conn(0x7f9b980b2930 legacy=0x7f9b9814b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:31.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.525+0000 7f9b9cb3d700 1 --2- 172.21.15.17:0/3364748433 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b78003750 0x7f9b78003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:31.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.525+0000 7f9b9cb3d700 1 --2- 172.21.15.17:0/3364748433 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b78004500 0x7f9b78004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:31.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.525+0000 7f9b9cb3d700 1 --2- 172.21.15.17:0/3364748433 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9b980b1ea0 0x7f9b78004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:31.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.525+0000 7f9b9cb3d700 1 -- 172.21.15.17:0/3364748433 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b98145340 con 0x7f9b78003750 2022-08-01T19:43:31.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.525+0000 7f9b9cb3d700 1 -- 172.21.15.17:0/3364748433 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b980b2d10 con 0x7f9b78004500 2022-08-01T19:43:31.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.525+0000 7f9b9cb3d700 1 -- 172.21.15.17:0/3364748433 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7800b7a0 con 0x7f9b980b1ea0 2022-08-01T19:43:31.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.525+0000 7f9b9db3f700 1 --2- 172.21.15.17:0/3364748433 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b78003750 0x7f9b78003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:31.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.525+0000 7f9b9e340700 1 --2- 172.21.15.17:0/3364748433 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9b980b1ea0 0x7f9b78004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:31.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.525+0000 7f9b9d33e700 1 --2- 172.21.15.17:0/3364748433 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b78004500 0x7f9b78004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:31.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.525+0000 7f9b9db3f700 1 -- 172.21.15.17:0/3364748433 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9b980b1ea0 msgr2=0x7f9b78004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:31.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.525+0000 7f9b9db3f700 1 --2- 172.21.15.17:0/3364748433 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9b980b1ea0 0x7f9b78004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:31.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.525+0000 7f9b9db3f700 1 -- 172.21.15.17:0/3364748433 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b78004500 msgr2=0x7f9b78004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:31.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.525+0000 7f9b9db3f700 1 --2- 172.21.15.17:0/3364748433 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b78004500 0x7f9b78004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:31.547 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.529+0000 7f9b9db3f700 1 -- 172.21.15.17:0/3364748433 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b78003260 con 0x7f9b78003750 2022-08-01T19:43:31.547 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.529+0000 7f9b9db3f700 1 --2- 172.21.15.17:0/3364748433 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b78003750 0x7f9b78003b70 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 rx=0x7f9b880016f0 tx=0x7f9b980adee0).ready entity=mon.0 client_cookie=302438b0737ebeb server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:31.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.529+0000 7f9b9cb3d700 1 -- 172.21.15.17:0/3364748433 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b8800cc50 con 0x7f9b78003750 2022-08-01T19:43:31.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.529+0000 7f9b9cb3d700 1 -- 172.21.15.17:0/3364748433 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8800cdb0 con 0x7f9b78003750 2022-08-01T19:43:31.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.529+0000 7f9b9cb3d700 1 -- 172.21.15.17:0/3364748433 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b88005b80 con 0x7f9b78003750 2022-08-01T19:43:31.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.529+0000 7f9b9eb41700 1 -- 172.21.15.17:0/3364748433 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b78003750 msgr2=0x7f9b78003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:31.550 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.529+0000 7f9b9eb41700 1 --2- 172.21.15.17:0/3364748433 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b78003750 0x7f9b78003b70 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 rx=0x7f9b880016f0 tx=0x7f9b980adee0).stop 2022-08-01T19:43:31.550 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.529+0000 7f9b9e340700 1 -- 172.21.15.17:0/3364748433 reap_dead start 2022-08-01T19:43:31.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.529+0000 7f9b9eb41700 1 -- 172.21.15.17:0/3364748433 shutdown_connections 2022-08-01T19:43:31.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.529+0000 7f9b9eb41700 1 -- 172.21.15.17:0/3364748433 >> 172.21.15.17:0/3364748433 conn(0x7f9b980612d0 msgr2=0x7f9b98062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:31.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.529+0000 7f9b9eb41700 1 -- 172.21.15.17:0/3364748433 shutdown_connections 2022-08-01T19:43:31.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.529+0000 7f9b9eb41700 1 -- 172.21.15.17:0/3364748433 wait complete. 2022-08-01T19:43:31.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.529+0000 7f9b9eb41700 1 Processor -- start 2022-08-01T19:43:31.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.529+0000 7f9b9eb41700 1 -- start start 2022-08-01T19:43:31.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.529+0000 7f9b9eb41700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9b78003750 0x7f9b98141530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:31.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.529+0000 7f9b9eb41700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b98140220 0x7f9b98141a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:31.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.529+0000 7f9b9eb41700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b98146ae0 0x7f9b98146f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:31.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.529+0000 7f9b9eb41700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b98142dd0 con 0x7f9b98146ae0 2022-08-01T19:43:31.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.529+0000 7f9b9eb41700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b98147460 con 0x7f9b98140220 2022-08-01T19:43:31.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.529+0000 7f9b9eb41700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b981476d0 con 0x7f9b78003750 2022-08-01T19:43:31.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.529+0000 7f9b9db3f700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9b78003750 0x7f9b98141530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:31.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.529+0000 7f9b9e340700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b98146ae0 0x7f9b98146f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:31.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.529+0000 7f9b9db3f700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9b78003750 0x7f9b98141530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:40912/0 (socket says 172.21.15.17:40912) 2022-08-01T19:43:31.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.529+0000 7f9b9db3f700 1 -- 172.21.15.17:0/3676619914 learned_addr learned my addr 172.21.15.17:0/3676619914 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:31.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.529+0000 7f9b9d33e700 1 --2- 172.21.15.17:0/3676619914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b98140220 0x7f9b98141a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:31.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.533+0000 7f9b9db3f700 1 -- 172.21.15.17:0/3676619914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b98140220 msgr2=0x7f9b98141a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:31.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.533+0000 7f9b9db3f700 1 --2- 172.21.15.17:0/3676619914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b98140220 0x7f9b98141a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:31.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.533+0000 7f9b9db3f700 1 -- 172.21.15.17:0/3676619914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b98146ae0 msgr2=0x7f9b98146f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:31.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.533+0000 7f9b9db3f700 1 --2- 172.21.15.17:0/3676619914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b98146ae0 0x7f9b98146f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:31.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.533+0000 7f9b9db3f700 1 -- 172.21.15.17:0/3676619914 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b98147970 con 0x7f9b78003750 2022-08-01T19:43:31.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.533+0000 7f9b9db3f700 1 --2- 172.21.15.17:0/3676619914 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9b78003750 0x7f9b98141530 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f9b98142b80 tx=0x7f9b88015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:31.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.533+0000 7f9b96ffd700 1 -- 172.21.15.17:0/3676619914 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b880072c0 con 0x7f9b78003750 2022-08-01T19:43:31.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.533+0000 7f9b96ffd700 1 -- 172.21.15.17:0/3676619914 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b88007420 con 0x7f9b78003750 2022-08-01T19:43:31.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.533+0000 7f9b9eb41700 1 -- 172.21.15.17:0/3676619914 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b9814d330 con 0x7f9b78003750 2022-08-01T19:43:31.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.533+0000 7f9b96ffd700 1 -- 172.21.15.17:0/3676619914 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b88007780 con 0x7f9b78003750 2022-08-01T19:43:31.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.533+0000 7f9b9eb41700 1 -- 172.21.15.17:0/3676619914 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b9814d7d0 con 0x7f9b78003750 2022-08-01T19:43:31.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.533+0000 7f9b9eb41700 1 -- 172.21.15.17:0/3676619914 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b98147630 con 0x7f9b78003750 2022-08-01T19:43:31.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.533+0000 7f9b96ffd700 1 -- 172.21.15.17:0/3676619914 <== mon.2 v2:172.21.15.77:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9b8801d070 con 0x7f9b78003750 2022-08-01T19:43:31.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.533+0000 7f9b96ffd700 1 --2- 172.21.15.17:0/3676619914 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9b7c053a70 0x7f9b7c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:31.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.533+0000 7f9b96ffd700 1 -- 172.21.15.17:0/3676619914 <== mon.2 v2:172.21.15.77:3301/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f9b88019030 con 0x7f9b78003750 2022-08-01T19:43:31.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.533+0000 7f9b9d33e700 1 --2- 172.21.15.17:0/3676619914 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9b7c053a70 0x7f9b7c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:31.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.537+0000 7f9b9d33e700 1 --2- 172.21.15.17:0/3676619914 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9b7c053a70 0x7f9b7c055f30 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f9b8000c000 tx=0x7f9b80008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:31.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.541+0000 7f9b96ffd700 1 -- 172.21.15.17:0/3676619914 <== mon.2 v2:172.21.15.77:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9b88043d10 con 0x7f9b78003750 2022-08-01T19:43:31.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.709+0000 7f9b9eb41700 1 -- 172.21.15.17:0/3676619914 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9b9814d4c0 con 0x7f9b78003750 2022-08-01T19:43:31.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.709+0000 7f9b96ffd700 1 -- 172.21.15.17:0/3676619914 <== mon.2 v2:172.21.15.77:3301/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v35) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f9b88047330 con 0x7f9b78003750 2022-08-01T19:43:31.717 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:31.718 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:43:31.718 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_dump": [ 2022-08-01T19:43:31.719 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:31.719 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.a", 2022-08-01T19:43:31.720 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhijotxORAAVc04rsrlFj0NShsyc6Qbhg==", 2022-08-01T19:43:31.720 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:43:31.721 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:43:31.721 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:43:31.722 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:43:31.722 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:43:31.723 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:31.723 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:31.724 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:31.724 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.b", 2022-08-01T19:43:31.724 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhiud2xBxAAcn6/O5ysdIp/oIbF5Y1dAw==", 2022-08-01T19:43:31.725 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:43:31.725 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:43:31.726 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:43:31.726 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:43:31.727 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:43:31.727 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:31.728 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:31.728 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:31.729 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.c", 2022-08-01T19:43:31.729 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+KuhiMzMbBBAAfdT5mtpYwboNL+7f2B09KQ==", 2022-08-01T19:43:31.729 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:43:31.730 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:43:31.730 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:43:31.731 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:43:31.731 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:43:31.732 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:31.732 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:31.733 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:31.733 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.0", 2022-08-01T19:43:31.734 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhia27/NRAAZXBp+nYelJVJ6Mge9r+3Vg==", 2022-08-01T19:43:31.734 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:43:31.735 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:43:31.735 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:43:31.736 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:43:31.736 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:31.736 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:31.737 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:31.737 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.1", 2022-08-01T19:43:31.738 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCBKuhiOggRGRAAvgA0MxwBJwmpXV6S9DTdjA==", 2022-08-01T19:43:31.738 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:43:31.739 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:43:31.739 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:43:31.740 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:43:31.740 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:31.741 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:31.741 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:31.741 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.2", 2022-08-01T19:43:31.742 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCDKuhi5PgOORAAhcORK4bW2T3dXPeYjcCm+g==", 2022-08-01T19:43:31.742 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:43:31.743 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:43:31.743 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:43:31.744 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:43:31.744 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:31.745 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:31.745 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:31.746 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.3", 2022-08-01T19:43:31.746 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCGKuhiutn6GxAAk2ucLuGbT5YV/R4xXQGh0w==", 2022-08-01T19:43:31.747 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:43:31.747 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:43:31.747 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:43:31.748 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:43:31.748 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:31.749 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:31.749 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:31.750 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.4", 2022-08-01T19:43:31.750 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCLKuhiGHTRHxAArr3TwXA0gsCAAF/Qs38wUQ==", 2022-08-01T19:43:31.751 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:43:31.751 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:43:31.752 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:43:31.752 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:43:31.752 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:31.753 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:31.753 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:31.754 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.5", 2022-08-01T19:43:31.754 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCOKuhiMMDbBBAAvTzBvOcv7pFWm/P/lNqB0w==", 2022-08-01T19:43:31.755 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:43:31.755 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:43:31.756 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:43:31.756 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:43:31.757 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:31.757 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:31.757 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:31.758 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.6", 2022-08-01T19:43:31.758 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCQKuhiG2H3IxAAufUcWqo9ULquxIrUD5vSfA==", 2022-08-01T19:43:31.759 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:43:31.759 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:43:31.760 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:43:31.760 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:43:31.761 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:31.761 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:31.762 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:31.762 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.7", 2022-08-01T19:43:31.763 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCTKuhiU50LChAAXuhrYBbHkk3eKzAPiEJFTg==", 2022-08-01T19:43:31.763 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:43:31.763 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:43:31.764 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:43:31.764 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:43:31.764 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:31.765 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:31.766 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:31.766 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.0", 2022-08-01T19:43:31.766 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhig3EnCxAANM20jSIQf/AKwuRX1qwT/A==", 2022-08-01T19:43:31.767 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:43:31.767 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:43:31.767 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow r", 2022-08-01T19:43:31.767 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T19:43:31.767 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:31.768 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:31.768 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:31.768 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.1", 2022-08-01T19:43:31.768 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+KuhicJR0DhAAivqEAZUhXnSwjVRwWm2bJg==", 2022-08-01T19:43:31.769 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:43:31.769 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:43:31.769 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow r", 2022-08-01T19:43:31.769 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T19:43:31.770 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:31.770 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:31.771 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:31.771 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.admin", 2022-08-01T19:43:31.772 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhiPTTGHxAAgfthCR6RX6j8F1YseFGNeA==", 2022-08-01T19:43:31.772 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:43:31.772 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:43:31.773 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:43:31.773 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:43:31.774 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:43:31.774 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:31.775 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:31.775 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:31.776 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.x", 2022-08-01T19:43:31.776 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9Kuhi2jvCNhAAyUxiGByZ+OZr8Y7dh3Qt5w==", 2022-08-01T19:43:31.777 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:43:31.777 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:43:31.777 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2022-08-01T19:43:31.778 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:43:31.778 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:31.779 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:31.779 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:31.779 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.y", 2022-08-01T19:43:31.780 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhiDJXpMhAA8aek67ydbRgNO0h+miws+w==", 2022-08-01T19:43:31.780 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:43:31.780 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:43:31.781 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2022-08-01T19:43:31.781 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:43:31.782 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:31.782 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:31.783 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:31.784 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:43:31.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.713+0000 7f9b9eb41700 1 -- 172.21.15.17:0/3676619914 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9b7c053a70 msgr2=0x7f9b7c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:31.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.713+0000 7f9b9eb41700 1 --2- 172.21.15.17:0/3676619914 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9b7c053a70 0x7f9b7c055f30 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f9b8000c000 tx=0x7f9b80008040).stop 2022-08-01T19:43:31.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.713+0000 7f9b9eb41700 1 -- 172.21.15.17:0/3676619914 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9b78003750 msgr2=0x7f9b98141530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:31.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.713+0000 7f9b9eb41700 1 --2- 172.21.15.17:0/3676619914 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9b78003750 0x7f9b98141530 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f9b98142b80 tx=0x7f9b88015000).stop 2022-08-01T19:43:31.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.713+0000 7f9b9eb41700 1 -- 172.21.15.17:0/3676619914 shutdown_connections 2022-08-01T19:43:31.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.713+0000 7f9b9eb41700 1 --2- 172.21.15.17:0/3676619914 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9b7c053a70 0x7f9b7c055f30 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:31.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.713+0000 7f9b9eb41700 1 --2- 172.21.15.17:0/3676619914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b98146ae0 0x7f9b98146f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:31.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.713+0000 7f9b9eb41700 1 --2- 172.21.15.17:0/3676619914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b98140220 0x7f9b98141a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:31.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.713+0000 7f9b9eb41700 1 --2- 172.21.15.17:0/3676619914 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9b78003750 0x7f9b98141530 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:31.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.713+0000 7f9b9eb41700 1 -- 172.21.15.17:0/3676619914 >> 172.21.15.17:0/3676619914 conn(0x7f9b980612d0 msgr2=0x7f9b9814a000 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:31.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.713+0000 7f9b9eb41700 1 -- 172.21.15.17:0/3676619914 shutdown_connections 2022-08-01T19:43:31.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.713+0000 7f9b9eb41700 1 -- 172.21.15.17:0/3676619914 wait complete. 2022-08-01T19:43:31.792 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:43:31.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.897+0000 7fed8e11f700 1 Processor -- start 2022-08-01T19:43:31.899 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.897+0000 7fed8e11f700 1 -- start start 2022-08-01T19:43:31.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.897+0000 7fed8e11f700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fed880b1f40 0x7fed880b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:31.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.897+0000 7fed8e11f700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fed88148530 0x7fed880b2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:31.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.897+0000 7fed8e11f700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed880b04e0 con 0x7fed880a7250 2022-08-01T19:43:31.912 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.897+0000 7fed8e11f700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed880a9250 con 0x7fed880b1f40 2022-08-01T19:43:31.912 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.897+0000 7fed8e11f700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed880b0780 con 0x7fed88148530 2022-08-01T19:43:31.912 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.897+0000 7fed877fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fed880b1f40 0x7fed880b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:31.912 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.897+0000 7fed877fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fed880b1f40 0x7fed880b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44822/0 (socket says 172.21.15.17:44822) 2022-08-01T19:43:31.913 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.897+0000 7fed877fe700 1 -- 172.21.15.17:0/2020443823 learned_addr learned my addr 172.21.15.17:0/2020443823 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:31.913 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.897+0000 7fed86ffd700 1 --2- 172.21.15.17:0/2020443823 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fed88148530 0x7fed880b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:31.913 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.897+0000 7fed877fe700 1 -- 172.21.15.17:0/2020443823 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fed88148530 msgr2=0x7fed880b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:31.913 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.897+0000 7fed877fe700 1 --2- 172.21.15.17:0/2020443823 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fed88148530 0x7fed880b2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:31.914 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.897+0000 7fed877fe700 1 -- 172.21.15.17:0/2020443823 >> v1:172.21.15.17:6789/0 conn(0x7fed880a7250 legacy=0x7fed881448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:31.914 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.897+0000 7fed867fc700 1 -- 172.21.15.17:0/2020443823 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1915344008 0 0) 0x7fed880b04e0 con 0x7fed880a7250 2022-08-01T19:43:31.915 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.897+0000 7fed877fe700 1 -- 172.21.15.17:0/2020443823 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed880b0a20 con 0x7fed880b1f40 2022-08-01T19:43:31.915 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.897+0000 7fed877fe700 1 --2- 172.21.15.17:0/2020443823 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed880b1f40 0x7fed880b1160 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 rx=0x7fed70002f70 tx=0x7fed70006eb0).ready entity=mon.0 client_cookie=5db14e2fd0449094 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:31.915 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.897+0000 7fed867fc700 1 -- 172.21.15.17:0/2020443823 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed7000d570 con 0x7fed880b1f40 2022-08-01T19:43:31.916 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.897+0000 7fed867fc700 1 -- 172.21.15.17:0/2020443823 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed70000e90 con 0x7fed880b1f40 2022-08-01T19:43:31.916 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.897+0000 7fed867fc700 1 -- 172.21.15.17:0/2020443823 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed7000d8c0 con 0x7fed880b1f40 2022-08-01T19:43:31.916 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.897+0000 7fed8e11f700 1 -- 172.21.15.17:0/2020443823 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed880b1f40 msgr2=0x7fed880b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:31.916 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.897+0000 7fed8e11f700 1 --2- 172.21.15.17:0/2020443823 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed880b1f40 0x7fed880b1160 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 rx=0x7fed70002f70 tx=0x7fed70006eb0).stop 2022-08-01T19:43:31.917 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.901+0000 7fed8e11f700 1 -- 172.21.15.17:0/2020443823 shutdown_connections 2022-08-01T19:43:31.917 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.901+0000 7fed8e11f700 1 --2- 172.21.15.17:0/2020443823 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fed88148530 0x7fed880b2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:31.917 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.901+0000 7fed8e11f700 1 --2- 172.21.15.17:0/2020443823 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed880b1f40 0x7fed880b1160 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:31.918 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.901+0000 7fed8e11f700 1 -- 172.21.15.17:0/2020443823 >> 172.21.15.17:0/2020443823 conn(0x7fed8805f840 msgr2=0x7fed8805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:31.918 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.901+0000 7fed8e11f700 1 -- 172.21.15.17:0/2020443823 shutdown_connections 2022-08-01T19:43:31.918 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.901+0000 7fed8e11f700 1 -- 172.21.15.17:0/2020443823 wait complete. 2022-08-01T19:43:31.919 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.901+0000 7fed8e11f700 1 Processor -- start 2022-08-01T19:43:31.919 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.901+0000 7fed8e11f700 1 -- start start 2022-08-01T19:43:31.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.905+0000 7fed8e11f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed880a7250 0x7fed88159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:31.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.905+0000 7fed8e11f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed880b1f40 0x7fed881401e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:31.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.905+0000 7fed8e11f700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fed88148530 0x7fed880b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:31.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.905+0000 7fed8e11f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed880aa4d0 con 0x7fed880b1f40 2022-08-01T19:43:31.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.905+0000 7fed8e11f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed880b4410 con 0x7fed880a7250 2022-08-01T19:43:31.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.905+0000 7fed8e11f700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed880b46a0 con 0x7fed88148530 2022-08-01T19:43:31.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.905+0000 7fed86ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed880b1f40 0x7fed881401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:31.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.905+0000 7fed86ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed880b1f40 0x7fed881401e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44828/0 (socket says 172.21.15.17:44828) 2022-08-01T19:43:31.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.905+0000 7fed86ffd700 1 -- 172.21.15.17:0/2741059250 learned_addr learned my addr 172.21.15.17:0/2741059250 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:31.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.905+0000 7fed877fe700 1 --2- 172.21.15.17:0/2741059250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed880a7250 0x7fed88159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:31.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.905+0000 7fed87fff700 1 --2- 172.21.15.17:0/2741059250 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fed88148530 0x7fed880b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:31.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.905+0000 7fed86ffd700 1 -- 172.21.15.17:0/2741059250 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fed88148530 msgr2=0x7fed880b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:31.927 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.905+0000 7fed86ffd700 1 --2- 172.21.15.17:0/2741059250 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fed88148530 0x7fed880b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:31.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.905+0000 7fed86ffd700 1 -- 172.21.15.17:0/2741059250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed880a7250 msgr2=0x7fed88159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:31.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.905+0000 7fed86ffd700 1 --2- 172.21.15.17:0/2741059250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed880a7250 0x7fed88159e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:31.929 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.905+0000 7fed86ffd700 1 -- 172.21.15.17:0/2741059250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7000b1e0 con 0x7fed880b1f40 2022-08-01T19:43:31.929 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.905+0000 7fed86ffd700 1 --2- 172.21.15.17:0/2741059250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed880b1f40 0x7fed881401e0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 rx=0x7fed8000c000 tx=0x7fed800091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:31.930 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.905+0000 7fed84ff9700 1 -- 172.21.15.17:0/2741059250 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed8000c6a0 con 0x7fed880b1f40 2022-08-01T19:43:31.930 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.905+0000 7fed84ff9700 1 -- 172.21.15.17:0/2741059250 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed8000c800 con 0x7fed880b1f40 2022-08-01T19:43:31.931 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.905+0000 7fed8e11f700 1 -- 172.21.15.17:0/2741059250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed880b4940 con 0x7fed880b1f40 2022-08-01T19:43:31.932 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.905+0000 7fed84ff9700 1 -- 172.21.15.17:0/2741059250 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed8000cb60 con 0x7fed880b1f40 2022-08-01T19:43:31.933 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.905+0000 7fed8e11f700 1 -- 172.21.15.17:0/2741059250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed881568b0 con 0x7fed880b1f40 2022-08-01T19:43:31.934 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.905+0000 7fed8e11f700 1 -- 172.21.15.17:0/2741059250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed880b0cc0 con 0x7fed880b1f40 2022-08-01T19:43:31.934 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.905+0000 7fed84ff9700 1 -- 172.21.15.17:0/2741059250 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fed80022070 con 0x7fed880b1f40 2022-08-01T19:43:31.935 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.909+0000 7fed84ff9700 1 --2- 172.21.15.17:0/2741059250 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fed6c0539e0 0x7fed6c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:31.935 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.909+0000 7fed84ff9700 1 -- 172.21.15.17:0/2741059250 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fed8006c4a0 con 0x7fed880b1f40 2022-08-01T19:43:31.935 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.909+0000 7fed877fe700 1 --2- 172.21.15.17:0/2741059250 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fed6c0539e0 0x7fed6c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:31.937 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.909+0000 7fed877fe700 1 --2- 172.21.15.17:0/2741059250 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fed6c0539e0 0x7fed6c055ea0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7fed7001c000 tx=0x7fed70015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:31.938 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:31.913+0000 7fed84ff9700 1 -- 172.21.15.17:0/2741059250 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fed80043430 con 0x7fed880b1f40 2022-08-01T19:43:32.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.037+0000 7fed8e11f700 1 -- 172.21.15.17:0/2741059250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed880574e0 con 0x7fed880b1f40 2022-08-01T19:43:32.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.041+0000 7fed84ff9700 1 -- 172.21.15.17:0/2741059250 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+13424 (secure 0 0 0) 0x7fed80016020 con 0x7fed880b1f40 2022-08-01T19:43:32.050 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:32.051 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":143,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:43:31.388678+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:43:32.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.041+0000 7fed8e11f700 1 -- 172.21.15.17:0/2741059250 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fed6c0539e0 msgr2=0x7fed6c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:32.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.041+0000 7fed8e11f700 1 --2- 172.21.15.17:0/2741059250 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fed6c0539e0 0x7fed6c055ea0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7fed7001c000 tx=0x7fed70015000).stop 2022-08-01T19:43:32.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.041+0000 7fed8e11f700 1 -- 172.21.15.17:0/2741059250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed880b1f40 msgr2=0x7fed881401e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:32.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.041+0000 7fed8e11f700 1 --2- 172.21.15.17:0/2741059250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed880b1f40 0x7fed881401e0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 rx=0x7fed8000c000 tx=0x7fed800091d0).stop 2022-08-01T19:43:32.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.041+0000 7fed8e11f700 1 -- 172.21.15.17:0/2741059250 shutdown_connections 2022-08-01T19:43:32.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.041+0000 7fed8e11f700 1 --2- 172.21.15.17:0/2741059250 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fed6c0539e0 0x7fed6c055ea0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:32.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.041+0000 7fed8e11f700 1 --2- 172.21.15.17:0/2741059250 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fed88148530 0x7fed880b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:32.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.041+0000 7fed8e11f700 1 --2- 172.21.15.17:0/2741059250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed880b1f40 0x7fed881401e0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:32.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.041+0000 7fed8e11f700 1 --2- 172.21.15.17:0/2741059250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed880a7250 0x7fed88159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:32.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.041+0000 7fed8e11f700 1 -- 172.21.15.17:0/2741059250 >> 172.21.15.17:0/2741059250 conn(0x7fed8805f840 msgr2=0x7fed880af900 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:32.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.041+0000 7fed8e11f700 1 -- 172.21.15.17:0/2741059250 shutdown_connections 2022-08-01T19:43:32.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.041+0000 7fed8e11f700 1 -- 172.21.15.17:0/2741059250 wait complete. 2022-08-01T19:43:32.064 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-08-01T19:43:32.065 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:43:32.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.173+0000 7f29adbef700 1 Processor -- start 2022-08-01T19:43:32.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.173+0000 7f29adbef700 1 -- start start 2022-08-01T19:43:32.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.173+0000 7f29adbef700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f29a8146ae0 0x7f29a81403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:32.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.173+0000 7f29adbef700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f29a80b4310 0x7f29a81408b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:32.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.173+0000 7f29adbef700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a8143cf0 con 0x7f29a80b4820 2022-08-01T19:43:32.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.173+0000 7f29adbef700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29a81476a0 con 0x7f29a8146ae0 2022-08-01T19:43:32.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.173+0000 7f29adbef700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29a8143f90 con 0x7f29a80b4310 2022-08-01T19:43:32.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.177+0000 7f29a77fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f29a8146ae0 0x7f29a81403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:32.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.177+0000 7f29a6ffd700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f29a80b4310 0x7f29a81408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:32.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.177+0000 7f29a7fff700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7f29a80b4820 0x7f29a80b5060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:53832/0 (socket says 172.21.15.17:53832) 2022-08-01T19:43:32.182 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.177+0000 7f29a77fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f29a8146ae0 0x7f29a81403c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44834/0 (socket says 172.21.15.17:44834) 2022-08-01T19:43:32.182 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.177+0000 7f29a7fff700 1 -- 172.21.15.17:0/188866891 learned_addr learned my addr 172.21.15.17:0/188866891 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:32.182 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.177+0000 7f29a6ffd700 1 -- 172.21.15.17:0/188866891 >> v2:172.21.15.17:3300/0 conn(0x7f29a8146ae0 msgr2=0x7f29a81403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:32.182 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.177+0000 7f29a6ffd700 1 --2- 172.21.15.17:0/188866891 >> v2:172.21.15.17:3300/0 conn(0x7f29a8146ae0 0x7f29a81403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:32.183 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.177+0000 7f29a6ffd700 1 -- 172.21.15.17:0/188866891 >> v1:172.21.15.77:6789/0 conn(0x7f29a80b4820 legacy=0x7f29a80b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:32.183 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.177+0000 7f29a67fc700 1 -- 172.21.15.17:0/188866891 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1807792857 0 0) 0x7f29a8143cf0 con 0x7f29a80b4820 2022-08-01T19:43:32.183 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.177+0000 7f29a6ffd700 1 -- 172.21.15.17:0/188866891 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a80b0e20 con 0x7f29a80b4310 2022-08-01T19:43:32.183 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.177+0000 7f29a6ffd700 1 --2- 172.21.15.17:0/188866891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29a80b4310 0x7f29a81408b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f2998002f70 tx=0x7f2998006df0).ready entity=mon.1 client_cookie=e60311168ea53213 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:32.184 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.177+0000 7f29a67fc700 1 -- 172.21.15.17:0/188866891 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f299800d570 con 0x7f29a80b4310 2022-08-01T19:43:32.184 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.177+0000 7f29a67fc700 1 -- 172.21.15.17:0/188866891 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2998000e90 con 0x7f29a80b4310 2022-08-01T19:43:32.184 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.177+0000 7f29a67fc700 1 -- 172.21.15.17:0/188866891 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f299800d8c0 con 0x7f29a80b4310 2022-08-01T19:43:32.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.177+0000 7f29adbef700 1 -- 172.21.15.17:0/188866891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29a80b4310 msgr2=0x7f29a81408b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:32.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.177+0000 7f29adbef700 1 --2- 172.21.15.17:0/188866891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29a80b4310 0x7f29a81408b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f2998002f70 tx=0x7f2998006df0).stop 2022-08-01T19:43:32.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.177+0000 7f29adbef700 1 -- 172.21.15.17:0/188866891 shutdown_connections 2022-08-01T19:43:32.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.177+0000 7f29adbef700 1 --2- 172.21.15.17:0/188866891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29a80b4310 0x7f29a81408b0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:32.190 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.177+0000 7f29adbef700 1 --2- 172.21.15.17:0/188866891 >> v2:172.21.15.17:3300/0 conn(0x7f29a8146ae0 0x7f29a81403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:32.190 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.177+0000 7f29adbef700 1 -- 172.21.15.17:0/188866891 >> 172.21.15.17:0/188866891 conn(0x7f29a805c620 msgr2=0x7f29a805ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:32.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.177+0000 7f29adbef700 1 -- 172.21.15.17:0/188866891 shutdown_connections 2022-08-01T19:43:32.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.177+0000 7f29adbef700 1 -- 172.21.15.17:0/188866891 wait complete. 2022-08-01T19:43:32.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.177+0000 7f29adbef700 1 Processor -- start 2022-08-01T19:43:32.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.177+0000 7f29adbef700 1 -- start start 2022-08-01T19:43:32.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.177+0000 7f29adbef700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29a80b4310 0x7f29a80b0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:32.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.177+0000 7f29adbef700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f29a80b4820 0x7f29a80b8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:32.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.177+0000 7f29adbef700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f29a8146ae0 0x7f29a80af0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:32.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.177+0000 7f29adbef700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a80adb60 con 0x7f29a80b4820 2022-08-01T19:43:32.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.177+0000 7f29adbef700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a80b7c10 con 0x7f29a80b4310 2022-08-01T19:43:32.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.177+0000 7f29adbef700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a81523a0 con 0x7f29a8146ae0 2022-08-01T19:43:32.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.181+0000 7f29a6ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f29a80b4820 0x7f29a80b8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:32.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.181+0000 7f29a6ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f29a80b4820 0x7f29a80b8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44844/0 (socket says 172.21.15.17:44844) 2022-08-01T19:43:32.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.181+0000 7f29a6ffd700 1 -- 172.21.15.17:0/1009265784 learned_addr learned my addr 172.21.15.17:0/1009265784 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:32.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.181+0000 7f29a77fe700 1 --2- 172.21.15.17:0/1009265784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29a80b4310 0x7f29a80b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:32.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.181+0000 7f29a7fff700 1 --2- 172.21.15.17:0/1009265784 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f29a8146ae0 0x7f29a80af0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:32.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.181+0000 7f29a6ffd700 1 -- 172.21.15.17:0/1009265784 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f29a8146ae0 msgr2=0x7f29a80af0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:32.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.181+0000 7f29a6ffd700 1 --2- 172.21.15.17:0/1009265784 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f29a8146ae0 0x7f29a80af0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:32.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.181+0000 7f29a6ffd700 1 -- 172.21.15.17:0/1009265784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29a80b4310 msgr2=0x7f29a80b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:32.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.181+0000 7f29a6ffd700 1 --2- 172.21.15.17:0/1009265784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29a80b4310 0x7f29a80b0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:32.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.181+0000 7f29a6ffd700 1 -- 172.21.15.17:0/1009265784 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f299800b180 con 0x7f29a80b4820 2022-08-01T19:43:32.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.181+0000 7f29a77fe700 1 --2- 172.21.15.17:0/1009265784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29a80b4310 0x7f29a80b0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:43:32.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.181+0000 7f29a7fff700 1 --2- 172.21.15.17:0/1009265784 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f29a8146ae0 0x7f29a80af0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:43:32.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.181+0000 7f29a6ffd700 1 --2- 172.21.15.17:0/1009265784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f29a80b4820 0x7f29a80b8370 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 rx=0x7f2998002f70 tx=0x7f2998014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:32.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.181+0000 7f29a4ff9700 1 -- 172.21.15.17:0/1009265784 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f299800dea0 con 0x7f29a80b4820 2022-08-01T19:43:32.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.181+0000 7f29a4ff9700 1 -- 172.21.15.17:0/1009265784 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2998013e90 con 0x7f29a80b4820 2022-08-01T19:43:32.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.181+0000 7f29a4ff9700 1 -- 172.21.15.17:0/1009265784 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f299801e5f0 con 0x7f29a80b4820 2022-08-01T19:43:32.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.181+0000 7f29adbef700 1 -- 172.21.15.17:0/1009265784 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29a8152640 con 0x7f29a80b4820 2022-08-01T19:43:32.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.181+0000 7f29adbef700 1 -- 172.21.15.17:0/1009265784 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a80af5d0 con 0x7f29a80b4820 2022-08-01T19:43:32.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.181+0000 7f29adbef700 1 -- 172.21.15.17:0/1009265784 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29a80574e0 con 0x7f29a80b4820 2022-08-01T19:43:32.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.181+0000 7f29a4ff9700 1 -- 172.21.15.17:0/1009265784 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f299801c070 con 0x7f29a80b4820 2022-08-01T19:43:32.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.185+0000 7f29a4ff9700 1 --2- 172.21.15.17:0/1009265784 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f298c053a70 0x7f298c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:32.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.185+0000 7f29a4ff9700 1 -- 172.21.15.17:0/1009265784 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f299806c7f0 con 0x7f29a80b4820 2022-08-01T19:43:32.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.185+0000 7f29a77fe700 1 --2- 172.21.15.17:0/1009265784 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f298c053a70 0x7f298c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:32.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.185+0000 7f29a77fe700 1 --2- 172.21.15.17:0/1009265784 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f298c053a70 0x7f298c055f30 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f299000b000 tx=0x7f2990006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:32.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.189+0000 7f29a4ff9700 1 -- 172.21.15.17:0/1009265784 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f29980437e0 con 0x7f29a80b4820 2022-08-01T19:43:32.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.321+0000 7f29adbef700 1 -- 172.21.15.17:0/1009265784 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f29a8069a70 con 0x7f29a80b4820 2022-08-01T19:43:32.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.321+0000 7f29a4ff9700 1 -- 172.21.15.17:0/1009265784 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v143) v1 ==== 110+0+10 (secure 0 0 0) 0x7f2998023070 con 0x7f29a80b4820 2022-08-01T19:43:32.325 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:43:32.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.325+0000 7f29adbef700 1 -- 172.21.15.17:0/1009265784 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f298c053a70 msgr2=0x7f298c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:32.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.325+0000 7f29adbef700 1 --2- 172.21.15.17:0/1009265784 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f298c053a70 0x7f298c055f30 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f299000b000 tx=0x7f2990006df0).stop 2022-08-01T19:43:32.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.325+0000 7f29adbef700 1 -- 172.21.15.17:0/1009265784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f29a80b4820 msgr2=0x7f29a80b8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:32.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.325+0000 7f29adbef700 1 --2- 172.21.15.17:0/1009265784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f29a80b4820 0x7f29a80b8370 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 rx=0x7f2998002f70 tx=0x7f2998014000).stop 2022-08-01T19:43:32.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.325+0000 7f29adbef700 1 -- 172.21.15.17:0/1009265784 shutdown_connections 2022-08-01T19:43:32.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.325+0000 7f29adbef700 1 --2- 172.21.15.17:0/1009265784 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f298c053a70 0x7f298c055f30 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:32.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.325+0000 7f29adbef700 1 --2- 172.21.15.17:0/1009265784 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f29a8146ae0 0x7f29a80af0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:32.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.325+0000 7f29adbef700 1 --2- 172.21.15.17:0/1009265784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f29a80b4820 0x7f29a80b8370 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:32.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.325+0000 7f29adbef700 1 --2- 172.21.15.17:0/1009265784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29a80b4310 0x7f29a80b0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:32.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.325+0000 7f29adbef700 1 -- 172.21.15.17:0/1009265784 >> 172.21.15.17:0/1009265784 conn(0x7f29a805c620 msgr2=0x7f29a813f580 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:32.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.325+0000 7f29adbef700 1 -- 172.21.15.17:0/1009265784 shutdown_connections 2022-08-01T19:43:32.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.325+0000 7f29adbef700 1 -- 172.21.15.17:0/1009265784 wait complete. 2022-08-01T19:43:32.338 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:43:32.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.445+0000 7fac77f02700 1 Processor -- start 2022-08-01T19:43:32.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.449+0000 7fac77f02700 1 -- start start 2022-08-01T19:43:32.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.449+0000 7fac77f02700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fac70146ae0 0x7fac701403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:32.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.449+0000 7fac77f02700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac700b4310 0x7fac701408b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:32.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.449+0000 7fac77f02700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac70143cf0 con 0x7fac700b4820 2022-08-01T19:43:32.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.449+0000 7fac77f02700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac701476a0 con 0x7fac70146ae0 2022-08-01T19:43:32.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.449+0000 7fac77f02700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac70143f90 con 0x7fac700b4310 2022-08-01T19:43:32.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.449+0000 7fac7649f700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7fac700b4820 0x7fac700b5060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:53844/0 (socket says 172.21.15.17:53844) 2022-08-01T19:43:32.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.449+0000 7fac75c9e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fac70146ae0 0x7fac701403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:32.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.449+0000 7fac7649f700 1 -- 172.21.15.17:0/1883723796 learned_addr learned my addr 172.21.15.17:0/1883723796 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:32.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.449+0000 7fac7549d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac700b4310 0x7fac701408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:32.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.449+0000 7fac74c9c700 1 -- 172.21.15.17:0/1883723796 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 797581240 0 0) 0x7fac70143cf0 con 0x7fac700b4820 2022-08-01T19:43:32.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.449+0000 7fac74c9c700 1 -- 172.21.15.17:0/1883723796 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac58003260 con 0x7fac700b4820 2022-08-01T19:43:32.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.449+0000 7fac7549d700 1 -- 172.21.15.17:0/1883723796 >> v2:172.21.15.77:3300/0 conn(0x7fac70146ae0 msgr2=0x7fac701403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:32.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.449+0000 7fac7549d700 1 --2- 172.21.15.17:0/1883723796 >> v2:172.21.15.77:3300/0 conn(0x7fac70146ae0 0x7fac701403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:32.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.449+0000 7fac7549d700 1 -- 172.21.15.17:0/1883723796 >> v1:172.21.15.77:6789/0 conn(0x7fac700b4820 legacy=0x7fac700b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:32.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.449+0000 7fac7549d700 1 -- 172.21.15.17:0/1883723796 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac700b0e20 con 0x7fac700b4310 2022-08-01T19:43:32.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.449+0000 7fac7549d700 1 --2- 172.21.15.17:0/1883723796 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac700b4310 0x7fac701408b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7fac6c002f70 tx=0x7fac6c006df0).ready entity=mon.2 client_cookie=e2e76e4a8459bbeb server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:32.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.449+0000 7fac74c9c700 1 -- 172.21.15.17:0/1883723796 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac6c00d4a0 con 0x7fac700b4310 2022-08-01T19:43:32.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.449+0000 7fac74c9c700 1 -- 172.21.15.17:0/1883723796 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac6c000e90 con 0x7fac700b4310 2022-08-01T19:43:32.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.449+0000 7fac74c9c700 1 -- 172.21.15.17:0/1883723796 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac6c00d7f0 con 0x7fac700b4310 2022-08-01T19:43:32.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.453+0000 7fac77f02700 1 -- 172.21.15.17:0/1883723796 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac700b4310 msgr2=0x7fac701408b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:32.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.453+0000 7fac77f02700 1 --2- 172.21.15.17:0/1883723796 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac700b4310 0x7fac701408b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7fac6c002f70 tx=0x7fac6c006df0).stop 2022-08-01T19:43:32.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.453+0000 7fac77f02700 1 -- 172.21.15.17:0/1883723796 shutdown_connections 2022-08-01T19:43:32.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.453+0000 7fac77f02700 1 --2- 172.21.15.17:0/1883723796 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac700b4310 0x7fac701408b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:32.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.453+0000 7fac77f02700 1 --2- 172.21.15.17:0/1883723796 >> v2:172.21.15.77:3300/0 conn(0x7fac70146ae0 0x7fac701403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:32.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.453+0000 7fac77f02700 1 -- 172.21.15.17:0/1883723796 >> 172.21.15.17:0/1883723796 conn(0x7fac7005c620 msgr2=0x7fac7005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:32.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.453+0000 7fac77f02700 1 -- 172.21.15.17:0/1883723796 shutdown_connections 2022-08-01T19:43:32.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.453+0000 7fac77f02700 1 -- 172.21.15.17:0/1883723796 wait complete. 2022-08-01T19:43:32.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.453+0000 7fac77f02700 1 Processor -- start 2022-08-01T19:43:32.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.453+0000 7fac77f02700 1 -- start start 2022-08-01T19:43:32.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.453+0000 7fac77f02700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac700b4310 0x7fac701420a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:32.501 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.453+0000 7fac77f02700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac700b4820 0x7fac701559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:32.501 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.453+0000 7fac77f02700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac70146ae0 0x7fac70155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:32.502 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.453+0000 7fac77f02700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac700ad820 con 0x7fac70146ae0 2022-08-01T19:43:32.502 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.453+0000 7fac77f02700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac70142880 con 0x7fac700b4310 2022-08-01T19:43:32.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.453+0000 7fac77f02700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac700ada50 con 0x7fac700b4820 2022-08-01T19:43:32.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.453+0000 7fac7649f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac70146ae0 0x7fac70155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:32.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.453+0000 7fac7649f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac70146ae0 0x7fac70155ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44858/0 (socket says 172.21.15.17:44858) 2022-08-01T19:43:32.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.453+0000 7fac7649f700 1 -- 172.21.15.17:0/2774163184 learned_addr learned my addr 172.21.15.17:0/2774163184 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:32.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.453+0000 7fac75c9e700 1 --2- 172.21.15.17:0/2774163184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac700b4310 0x7fac701420a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:32.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.453+0000 7fac7549d700 1 --2- 172.21.15.17:0/2774163184 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac700b4820 0x7fac701559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:32.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.453+0000 7fac7649f700 1 -- 172.21.15.17:0/2774163184 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac700b4820 msgr2=0x7fac701559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:32.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.453+0000 7fac7649f700 1 --2- 172.21.15.17:0/2774163184 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac700b4820 0x7fac701559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:32.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.453+0000 7fac7649f700 1 -- 172.21.15.17:0/2774163184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac700b4310 msgr2=0x7fac701420a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:32.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.453+0000 7fac7649f700 1 --2- 172.21.15.17:0/2774163184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac700b4310 0x7fac701420a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:32.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.453+0000 7fac7649f700 1 -- 172.21.15.17:0/2774163184 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac6c00b180 con 0x7fac70146ae0 2022-08-01T19:43:32.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.453+0000 7fac75c9e700 1 --2- 172.21.15.17:0/2774163184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac700b4310 0x7fac701420a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:43:32.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.453+0000 7fac7649f700 1 --2- 172.21.15.17:0/2774163184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac70146ae0 0x7fac70155ed0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 rx=0x7fac68001490 tx=0x7fac68007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:32.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.453+0000 7fac5effd700 1 -- 172.21.15.17:0/2774163184 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac68005ba0 con 0x7fac70146ae0 2022-08-01T19:43:32.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.453+0000 7fac5effd700 1 -- 172.21.15.17:0/2774163184 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac68005d00 con 0x7fac70146ae0 2022-08-01T19:43:32.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.453+0000 7fac77f02700 1 -- 172.21.15.17:0/2774163184 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac700adcf0 con 0x7fac70146ae0 2022-08-01T19:43:32.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.453+0000 7fac5effd700 1 -- 172.21.15.17:0/2774163184 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac680072c0 con 0x7fac70146ae0 2022-08-01T19:43:32.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.453+0000 7fac77f02700 1 -- 172.21.15.17:0/2774163184 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac700b0780 con 0x7fac70146ae0 2022-08-01T19:43:32.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.457+0000 7fac77f02700 1 -- 172.21.15.17:0/2774163184 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac700ade80 con 0x7fac70146ae0 2022-08-01T19:43:32.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.457+0000 7fac5effd700 1 -- 172.21.15.17:0/2774163184 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fac68010070 con 0x7fac70146ae0 2022-08-01T19:43:32.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.457+0000 7fac5effd700 1 --2- 172.21.15.17:0/2774163184 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fac50053a70 0x7fac50055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:32.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.457+0000 7fac5effd700 1 -- 172.21.15.17:0/2774163184 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fac6806b7e0 con 0x7fac70146ae0 2022-08-01T19:43:32.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.457+0000 7fac75c9e700 1 --2- 172.21.15.17:0/2774163184 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fac50053a70 0x7fac50055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:32.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.457+0000 7fac75c9e700 1 --2- 172.21.15.17:0/2774163184 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fac50053a70 0x7fac50055f30 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7fac6000b000 tx=0x7fac60008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:32.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.465+0000 7fac5effd700 1 -- 172.21.15.17:0/2774163184 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fac680427d0 con 0x7fac70146ae0 2022-08-01T19:43:32.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.593+0000 7fac77f02700 1 -- 172.21.15.17:0/2774163184 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fac700574e0 con 0x7fac70146ae0 2022-08-01T19:43:32.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.593+0000 7fac5effd700 1 -- 172.21.15.17:0/2774163184 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v143) v1 ==== 92+0+10 (secure 0 0 0) 0x7fac68045df0 con 0x7fac70146ae0 2022-08-01T19:43:32.600 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:43:32.602 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.597+0000 7fac77f02700 1 -- 172.21.15.17:0/2774163184 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fac50053a70 msgr2=0x7fac50055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:32.602 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.597+0000 7fac77f02700 1 --2- 172.21.15.17:0/2774163184 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fac50053a70 0x7fac50055f30 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7fac6000b000 tx=0x7fac60008040).stop 2022-08-01T19:43:32.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.597+0000 7fac77f02700 1 -- 172.21.15.17:0/2774163184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac70146ae0 msgr2=0x7fac70155ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:32.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.597+0000 7fac77f02700 1 --2- 172.21.15.17:0/2774163184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac70146ae0 0x7fac70155ed0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 rx=0x7fac68001490 tx=0x7fac68007e00).stop 2022-08-01T19:43:32.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.597+0000 7fac77f02700 1 -- 172.21.15.17:0/2774163184 shutdown_connections 2022-08-01T19:43:32.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.597+0000 7fac77f02700 1 --2- 172.21.15.17:0/2774163184 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fac50053a70 0x7fac50055f30 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:32.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.597+0000 7fac77f02700 1 --2- 172.21.15.17:0/2774163184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac70146ae0 0x7fac70155ed0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:32.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.597+0000 7fac77f02700 1 --2- 172.21.15.17:0/2774163184 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fac700b4820 0x7fac701559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:32.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.597+0000 7fac77f02700 1 --2- 172.21.15.17:0/2774163184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac700b4310 0x7fac701420a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:32.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.597+0000 7fac77f02700 1 -- 172.21.15.17:0/2774163184 >> 172.21.15.17:0/2774163184 conn(0x7fac7005c620 msgr2=0x7fac701434c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:32.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.597+0000 7fac77f02700 1 -- 172.21.15.17:0/2774163184 shutdown_connections 2022-08-01T19:43:32.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.597+0000 7fac77f02700 1 -- 172.21.15.17:0/2774163184 wait complete. 2022-08-01T19:43:32.611 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-08-01T19:43:32.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.725+0000 7f1c7423e700 1 Processor -- start 2022-08-01T19:43:32.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.725+0000 7f1c7423e700 1 -- start start 2022-08-01T19:43:32.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.725+0000 7f1c7423e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1c6c0afaf0 0x7f1c6c0b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:32.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.725+0000 7f1c7423e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1c6c0afed0 0x7f1c6c13fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:32.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.725+0000 7f1c7423e700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c6c0ad730 con 0x7f1c6c0b0680 2022-08-01T19:43:32.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.725+0000 7f1c7423e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c6c0af5f0 con 0x7f1c6c0afed0 2022-08-01T19:43:32.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.725+0000 7f1c7423e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c0ad1f0 con 0x7f1c6c0afaf0 2022-08-01T19:43:32.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.725+0000 7f1c727db700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1c6c0afed0 0x7f1c6c13fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:32.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.725+0000 7f1c717d9700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1c6c0b0680 0x7f1c6c0b2250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37056/0 (socket says 172.21.15.17:37056) 2022-08-01T19:43:32.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.725+0000 7f1c717d9700 1 -- 172.21.15.17:0/3472955993 learned_addr learned my addr 172.21.15.17:0/3472955993 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:32.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.725+0000 7f1c71fda700 1 --2- 172.21.15.17:0/3472955993 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1c6c0afaf0 0x7f1c6c0b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:32.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.725+0000 7f1c70fd8700 1 -- 172.21.15.17:0/3472955993 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 755497789 0 0) 0x7f1c6c0ad730 con 0x7f1c6c0b0680 2022-08-01T19:43:32.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.725+0000 7f1c70fd8700 1 -- 172.21.15.17:0/3472955993 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c50003260 con 0x7f1c6c0b0680 2022-08-01T19:43:32.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.725+0000 7f1c727db700 1 -- 172.21.15.17:0/3472955993 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1c6c0afaf0 msgr2=0x7f1c6c0b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:32.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.725+0000 7f1c727db700 1 --2- 172.21.15.17:0/3472955993 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1c6c0afaf0 0x7f1c6c0b1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:32.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.725+0000 7f1c727db700 1 -- 172.21.15.17:0/3472955993 >> v1:172.21.15.17:6789/0 conn(0x7f1c6c0b0680 legacy=0x7f1c6c0b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:32.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.725+0000 7f1c727db700 1 -- 172.21.15.17:0/3472955993 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c6c0ad490 con 0x7f1c6c0afed0 2022-08-01T19:43:32.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.725+0000 7f1c727db700 1 --2- 172.21.15.17:0/3472955993 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c6c0afed0 0x7f1c6c13fc20 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 rx=0x7f1c640032f0 tx=0x7f1c640091d0).ready entity=mon.0 client_cookie=ab56ca0b707d06c4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:32.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.725+0000 7f1c70fd8700 1 -- 172.21.15.17:0/3472955993 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c6400b600 con 0x7f1c6c0afed0 2022-08-01T19:43:32.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.725+0000 7f1c70fd8700 1 -- 172.21.15.17:0/3472955993 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6400b760 con 0x7f1c6c0afed0 2022-08-01T19:43:32.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.725+0000 7f1c70fd8700 1 -- 172.21.15.17:0/3472955993 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c6400b8d0 con 0x7f1c6c0afed0 2022-08-01T19:43:32.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.725+0000 7f1c7423e700 1 -- 172.21.15.17:0/3472955993 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c6c0afed0 msgr2=0x7f1c6c13fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:32.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.725+0000 7f1c7423e700 1 --2- 172.21.15.17:0/3472955993 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c6c0afed0 0x7f1c6c13fc20 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 rx=0x7f1c640032f0 tx=0x7f1c640091d0).stop 2022-08-01T19:43:32.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.725+0000 7f1c7423e700 1 -- 172.21.15.17:0/3472955993 shutdown_connections 2022-08-01T19:43:32.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.725+0000 7f1c7423e700 1 --2- 172.21.15.17:0/3472955993 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c6c0afed0 0x7f1c6c13fc20 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:32.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.725+0000 7f1c7423e700 1 --2- 172.21.15.17:0/3472955993 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1c6c0afaf0 0x7f1c6c0b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:32.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.725+0000 7f1c7423e700 1 -- 172.21.15.17:0/3472955993 >> 172.21.15.17:0/3472955993 conn(0x7f1c6c05ceb0 msgr2=0x7f1c6c063130 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:32.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.725+0000 7f1c7423e700 1 -- 172.21.15.17:0/3472955993 shutdown_connections 2022-08-01T19:43:32.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.725+0000 7f1c7423e700 1 -- 172.21.15.17:0/3472955993 wait complete. 2022-08-01T19:43:32.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.729+0000 7f1c7423e700 1 Processor -- start 2022-08-01T19:43:32.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.729+0000 7f1c7423e700 1 -- start start 2022-08-01T19:43:32.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.729+0000 7f1c7423e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1c6c0afaf0 0x7f1c6c13d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:32.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.729+0000 7f1c7423e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c6c0afed0 0x7f1c6c140b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:32.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.729+0000 7f1c7423e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c6c0b0680 0x7f1c6c13ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:32.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.729+0000 7f1c7423e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c066430 con 0x7f1c6c0afed0 2022-08-01T19:43:32.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.729+0000 7f1c7423e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c0666d0 con 0x7f1c6c0b0680 2022-08-01T19:43:32.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.729+0000 7f1c7423e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c066970 con 0x7f1c6c0afaf0 2022-08-01T19:43:32.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.729+0000 7f1c717d9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c6c0afed0 0x7f1c6c140b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:32.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.729+0000 7f1c717d9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c6c0afed0 0x7f1c6c140b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44870/0 (socket says 172.21.15.17:44870) 2022-08-01T19:43:32.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.729+0000 7f1c717d9700 1 -- 172.21.15.17:0/3802547810 learned_addr learned my addr 172.21.15.17:0/3802547810 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:32.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.729+0000 7f1c727db700 1 --2- 172.21.15.17:0/3802547810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c6c0b0680 0x7f1c6c13ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:32.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.729+0000 7f1c71fda700 1 --2- 172.21.15.17:0/3802547810 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1c6c0afaf0 0x7f1c6c13d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:32.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.729+0000 7f1c717d9700 1 -- 172.21.15.17:0/3802547810 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1c6c0afaf0 msgr2=0x7f1c6c13d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:32.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.729+0000 7f1c717d9700 1 --2- 172.21.15.17:0/3802547810 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1c6c0afaf0 0x7f1c6c13d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:32.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.729+0000 7f1c717d9700 1 -- 172.21.15.17:0/3802547810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c6c0b0680 msgr2=0x7f1c6c13ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:32.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.729+0000 7f1c717d9700 1 --2- 172.21.15.17:0/3802547810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c6c0b0680 0x7f1c6c13ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:32.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.729+0000 7f1c717d9700 1 -- 172.21.15.17:0/3802547810 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c64009470 con 0x7f1c6c0afed0 2022-08-01T19:43:32.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.729+0000 7f1c717d9700 1 --2- 172.21.15.17:0/3802547810 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c6c0afed0 0x7f1c6c140b50 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 rx=0x7f1c5c001110 tx=0x7f1c5c007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:32.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.729+0000 7f1c62ffd700 1 -- 172.21.15.17:0/3802547810 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c5c0070d0 con 0x7f1c6c0afed0 2022-08-01T19:43:32.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.729+0000 7f1c62ffd700 1 -- 172.21.15.17:0/3802547810 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5c007230 con 0x7f1c6c0afed0 2022-08-01T19:43:32.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.729+0000 7f1c7423e700 1 -- 172.21.15.17:0/3802547810 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c6c13ef80 con 0x7f1c6c0afed0 2022-08-01T19:43:32.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.729+0000 7f1c62ffd700 1 -- 172.21.15.17:0/3802547810 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c5c005610 con 0x7f1c6c0afed0 2022-08-01T19:43:32.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.729+0000 7f1c7423e700 1 -- 172.21.15.17:0/3802547810 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c6c13f360 con 0x7f1c6c0afed0 2022-08-01T19:43:32.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.733+0000 7f1c7423e700 1 -- 172.21.15.17:0/3802547810 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c6c066b30 con 0x7f1c6c0afed0 2022-08-01T19:43:32.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.733+0000 7f1c62ffd700 1 -- 172.21.15.17:0/3802547810 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1c5c010070 con 0x7f1c6c0afed0 2022-08-01T19:43:32.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.733+0000 7f1c62ffd700 1 --2- 172.21.15.17:0/3802547810 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1c54053a70 0x7f1c54055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:32.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.733+0000 7f1c62ffd700 1 -- 172.21.15.17:0/3802547810 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f1c5c06b4c0 con 0x7f1c6c0afed0 2022-08-01T19:43:32.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.733+0000 7f1c71fda700 1 --2- 172.21.15.17:0/3802547810 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1c54053a70 0x7f1c54055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.737+0000 7f1c71fda700 1 --2- 172.21.15.17:0/3802547810 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1c54053a70 0x7f1c54055f30 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f1c6800b000 tx=0x7f1c68006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.741+0000 7f1c62ffd700 1 -- 172.21.15.17:0/3802547810 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1c5c0424b0 con 0x7f1c6c0afed0 2022-08-01T19:43:32.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.873+0000 7f1c7423e700 1 -- 172.21.15.17:0/3802547810 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1c6c0b1830 con 0x7f1c6c0afed0 2022-08-01T19:43:32.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.873+0000 7f1c62ffd700 1 -- 172.21.15.17:0/3802547810 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v143) v1 ==== 84+0+143 (secure 0 0 0) 0x7f1c5c045ad0 con 0x7f1c6c0afed0 2022-08-01T19:43:32.876 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:32.877 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2022-08-01T19:43:32.877 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:32.877 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 1, 2022-08-01T19:43:32.878 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": "device_health_metrics" 2022-08-01T19:43:32.878 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:32.878 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:32.878 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 2, 2022-08-01T19:43:32.879 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": "rbd" 2022-08-01T19:43:32.879 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:32.879 INFO:teuthology.orchestra.run.smithi017.stdout:] 2022-08-01T19:43:32.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.873+0000 7f1c7423e700 1 -- 172.21.15.17:0/3802547810 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1c54053a70 msgr2=0x7f1c54055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:32.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.873+0000 7f1c7423e700 1 --2- 172.21.15.17:0/3802547810 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1c54053a70 0x7f1c54055f30 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f1c6800b000 tx=0x7f1c68006df0).stop 2022-08-01T19:43:32.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.873+0000 7f1c7423e700 1 -- 172.21.15.17:0/3802547810 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c6c0afed0 msgr2=0x7f1c6c140b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:32.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.873+0000 7f1c7423e700 1 --2- 172.21.15.17:0/3802547810 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c6c0afed0 0x7f1c6c140b50 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 rx=0x7f1c5c001110 tx=0x7f1c5c007aa0).stop 2022-08-01T19:43:32.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.873+0000 7f1c7423e700 1 -- 172.21.15.17:0/3802547810 shutdown_connections 2022-08-01T19:43:32.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.873+0000 7f1c7423e700 1 --2- 172.21.15.17:0/3802547810 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1c54053a70 0x7f1c54055f30 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:32.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.873+0000 7f1c7423e700 1 --2- 172.21.15.17:0/3802547810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c6c0b0680 0x7f1c6c13ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:32.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.873+0000 7f1c7423e700 1 --2- 172.21.15.17:0/3802547810 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c6c0afed0 0x7f1c6c140b50 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:32.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.873+0000 7f1c7423e700 1 --2- 172.21.15.17:0/3802547810 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1c6c0afaf0 0x7f1c6c13d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:32.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.873+0000 7f1c7423e700 1 -- 172.21.15.17:0/3802547810 >> 172.21.15.17:0/3802547810 conn(0x7f1c6c05ceb0 msgr2=0x7f1c6c0b5350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:32.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.877+0000 7f1c7423e700 1 -- 172.21.15.17:0/3802547810 shutdown_connections 2022-08-01T19:43:32.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:32.877+0000 7f1c7423e700 1 -- 172.21.15.17:0/3802547810 wait complete. 2022-08-01T19:43:32.888 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-08-01T19:43:32.889 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-08-01T19:43:33.014 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.005+0000 7f6dfc853700 1 Processor -- start 2022-08-01T19:43:33.014 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.005+0000 7f6dfc853700 1 -- start start 2022-08-01T19:43:33.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.005+0000 7f6dfc853700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6df805fc30 0x7f6df8143a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:33.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.005+0000 7f6dfc853700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f6df8060960 0x7f6df8143f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:33.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.005+0000 7f6dfc853700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6df80ac690 0x7f6df81469a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:33.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.005+0000 7f6dfc853700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6df81433d0 con 0x7f6df80ac690 2022-08-01T19:43:33.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.005+0000 7f6dfc853700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6df8060010 con 0x7f6df8060960 2022-08-01T19:43:33.018 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.005+0000 7f6dfc853700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df80acdf0 con 0x7f6df805fc30 2022-08-01T19:43:33.018 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.009+0000 7f6df6d9d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6df80ac690 0x7f6df81469a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:33.018 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.009+0000 7f6df659c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6df805fc30 0x7f6df8143a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:33.018 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.009+0000 7f6df6d9d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6df80ac690 0x7f6df81469a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44880/0 (socket says 172.21.15.17:44880) 2022-08-01T19:43:33.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.009+0000 7f6deffff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f6df8060960 0x7f6df8143f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:33.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.009+0000 7f6df6d9d700 1 -- 172.21.15.17:0/1557627976 learned_addr learned my addr 172.21.15.17:0/1557627976 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:33.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.009+0000 7f6df659c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6df805fc30 0x7f6df8143a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:40976/0 (socket says 172.21.15.17:40976) 2022-08-01T19:43:33.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.009+0000 7f6df6d9d700 1 -- 172.21.15.17:0/1557627976 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6df805fc30 msgr2=0x7f6df8143a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:33.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.009+0000 7f6df6d9d700 1 --2- 172.21.15.17:0/1557627976 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6df805fc30 0x7f6df8143a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:33.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.009+0000 7f6df6d9d700 1 -- 172.21.15.17:0/1557627976 >> v2:172.21.15.77:3300/0 conn(0x7f6df8060960 msgr2=0x7f6df8143f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:33.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.009+0000 7f6df6d9d700 1 --2- 172.21.15.17:0/1557627976 >> v2:172.21.15.77:3300/0 conn(0x7f6df8060960 0x7f6df8143f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:33.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.009+0000 7f6df6d9d700 1 -- 172.21.15.17:0/1557627976 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df80ad090 con 0x7f6df80ac690 2022-08-01T19:43:33.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.009+0000 7f6df6d9d700 1 --2- 172.21.15.17:0/1557627976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6df80ac690 0x7f6df81469a0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 rx=0x7f6df00032f0 tx=0x7f6df0009290).ready entity=mon.0 client_cookie=3e533335974232d7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:33.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.009+0000 7f6df5d9b700 1 -- 172.21.15.17:0/1557627976 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6df000b680 con 0x7f6df80ac690 2022-08-01T19:43:33.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.009+0000 7f6df5d9b700 1 -- 172.21.15.17:0/1557627976 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df000b7e0 con 0x7f6df80ac690 2022-08-01T19:43:33.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.009+0000 7f6df5d9b700 1 -- 172.21.15.17:0/1557627976 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6df000b950 con 0x7f6df80ac690 2022-08-01T19:43:33.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.009+0000 7f6dfc853700 1 -- 172.21.15.17:0/1557627976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6df80ac690 msgr2=0x7f6df81469a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:33.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.009+0000 7f6dfc853700 1 --2- 172.21.15.17:0/1557627976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6df80ac690 0x7f6df81469a0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 rx=0x7f6df00032f0 tx=0x7f6df0009290).stop 2022-08-01T19:43:33.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.009+0000 7f6dfc853700 1 -- 172.21.15.17:0/1557627976 shutdown_connections 2022-08-01T19:43:33.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.009+0000 7f6dfc853700 1 --2- 172.21.15.17:0/1557627976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6df80ac690 0x7f6df81469a0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:33.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.009+0000 7f6dfc853700 1 --2- 172.21.15.17:0/1557627976 >> v2:172.21.15.77:3300/0 conn(0x7f6df8060960 0x7f6df8143f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:33.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.009+0000 7f6dfc853700 1 --2- 172.21.15.17:0/1557627976 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6df805fc30 0x7f6df8143a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:33.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.009+0000 7f6dfc853700 1 -- 172.21.15.17:0/1557627976 >> 172.21.15.17:0/1557627976 conn(0x7f6df80b2b00 msgr2=0x7f6df80b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:33.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.009+0000 7f6dfc853700 1 -- 172.21.15.17:0/1557627976 shutdown_connections 2022-08-01T19:43:33.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.009+0000 7f6dfc853700 1 -- 172.21.15.17:0/1557627976 wait complete. 2022-08-01T19:43:33.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.009+0000 7f6dfc853700 1 Processor -- start 2022-08-01T19:43:33.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.013+0000 7f6dfc853700 1 -- start start 2022-08-01T19:43:33.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.013+0000 7f6dfc853700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6df805fc30 0x7f6df8144b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:33.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.013+0000 7f6dfc853700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6df8060960 0x7f6df8145140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:33.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.013+0000 7f6dfc853700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6df80ac690 0x7f6df8145630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:33.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.013+0000 7f6dfc853700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df8145b20 con 0x7f6df80ac690 2022-08-01T19:43:33.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.013+0000 7f6dfc853700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df80b05a0 con 0x7f6df805fc30 2022-08-01T19:43:33.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.013+0000 7f6dfc853700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df80ad5c0 con 0x7f6df8060960 2022-08-01T19:43:33.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.013+0000 7f6df6d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6df80ac690 0x7f6df8145630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:33.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.013+0000 7f6df659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6df805fc30 0x7f6df8144b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:33.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.013+0000 7f6df659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6df805fc30 0x7f6df8144b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:42804/0 (socket says 172.21.15.17:42804) 2022-08-01T19:43:33.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.013+0000 7f6df659c700 1 -- 172.21.15.17:0/495397207 learned_addr learned my addr 172.21.15.17:0/495397207 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:33.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.013+0000 7f6df6d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6df80ac690 0x7f6df8145630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44884/0 (socket says 172.21.15.17:44884) 2022-08-01T19:43:33.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.013+0000 7f6deffff700 1 --2- 172.21.15.17:0/495397207 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6df8060960 0x7f6df8145140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:33.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.013+0000 7f6df6d9d700 1 -- 172.21.15.17:0/495397207 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6df8060960 msgr2=0x7f6df8145140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:33.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.013+0000 7f6df6d9d700 1 --2- 172.21.15.17:0/495397207 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6df8060960 0x7f6df8145140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:33.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.013+0000 7f6df6d9d700 1 -- 172.21.15.17:0/495397207 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6df805fc30 msgr2=0x7f6df8144b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:33.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.013+0000 7f6df6d9d700 1 --2- 172.21.15.17:0/495397207 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6df805fc30 0x7f6df8144b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:33.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.013+0000 7f6df6d9d700 1 -- 172.21.15.17:0/495397207 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df0009530 con 0x7f6df80ac690 2022-08-01T19:43:33.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.013+0000 7f6df659c700 1 --2- 172.21.15.17:0/495397207 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6df805fc30 0x7f6df8144b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:43:33.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.013+0000 7f6df6d9d700 1 --2- 172.21.15.17:0/495397207 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6df80ac690 0x7f6df8145630 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 rx=0x7f6df00032f0 tx=0x7f6df0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:33.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.013+0000 7f6def7fe700 1 -- 172.21.15.17:0/495397207 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6df000b680 con 0x7f6df80ac690 2022-08-01T19:43:33.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.013+0000 7f6dfc853700 1 -- 172.21.15.17:0/495397207 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df8146520 con 0x7f6df80ac690 2022-08-01T19:43:33.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.013+0000 7f6def7fe700 1 -- 172.21.15.17:0/495397207 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df000bea0 con 0x7f6df80ac690 2022-08-01T19:43:33.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.013+0000 7f6def7fe700 1 -- 172.21.15.17:0/495397207 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6df001b600 con 0x7f6df80ac690 2022-08-01T19:43:33.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.013+0000 7f6dfc853700 1 -- 172.21.15.17:0/495397207 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df8153870 con 0x7f6df80ac690 2022-08-01T19:43:33.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.013+0000 7f6dfc853700 1 -- 172.21.15.17:0/495397207 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6df80574e0 con 0x7f6df80ac690 2022-08-01T19:43:33.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.017+0000 7f6def7fe700 1 -- 172.21.15.17:0/495397207 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6df000f070 con 0x7f6df80ac690 2022-08-01T19:43:33.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.017+0000 7f6def7fe700 1 --2- 172.21.15.17:0/495397207 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6ddc053a70 0x7f6ddc055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:33.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.017+0000 7f6def7fe700 1 -- 172.21.15.17:0/495397207 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f6df006da80 con 0x7f6df80ac690 2022-08-01T19:43:33.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.017+0000 7f6df659c700 1 --2- 172.21.15.17:0/495397207 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6ddc053a70 0x7f6ddc055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:33.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.017+0000 7f6df659c700 1 --2- 172.21.15.17:0/495397207 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6ddc053a70 0x7f6ddc055f30 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f6de800b000 tx=0x7f6de8006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:33.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.021+0000 7f6def7fe700 1 -- 172.21.15.17:0/495397207 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6df0044c30 con 0x7f6df80ac690 2022-08-01T19:43:33.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.149+0000 7f6dfc853700 1 -- 172.21.15.17:0/495397207 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f6df805cc40 con 0x7f6df80ac690 2022-08-01T19:43:33.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.257+0000 7f6def7fe700 1 -- 172.21.15.17:0/495397207 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v144) v1 ==== 133+0+0 (secure 0 0 0) 0x7f6df001f070 con 0x7f6df80ac690 2022-08-01T19:43:33.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.261+0000 7f6dfc853700 1 -- 172.21.15.17:0/495397207 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6ddc053a70 msgr2=0x7f6ddc055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:33.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.261+0000 7f6dfc853700 1 --2- 172.21.15.17:0/495397207 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6ddc053a70 0x7f6ddc055f30 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f6de800b000 tx=0x7f6de8006df0).stop 2022-08-01T19:43:33.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.261+0000 7f6dfc853700 1 -- 172.21.15.17:0/495397207 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6df80ac690 msgr2=0x7f6df8145630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:33.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.261+0000 7f6dfc853700 1 --2- 172.21.15.17:0/495397207 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6df80ac690 0x7f6df8145630 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 rx=0x7f6df00032f0 tx=0x7f6df0017000).stop 2022-08-01T19:43:33.266 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.261+0000 7f6dfc853700 1 -- 172.21.15.17:0/495397207 shutdown_connections 2022-08-01T19:43:33.266 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.261+0000 7f6dfc853700 1 --2- 172.21.15.17:0/495397207 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6ddc053a70 0x7f6ddc055f30 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:33.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.261+0000 7f6dfc853700 1 --2- 172.21.15.17:0/495397207 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6df80ac690 0x7f6df8145630 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:33.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.261+0000 7f6dfc853700 1 --2- 172.21.15.17:0/495397207 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6df8060960 0x7f6df8145140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:33.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.261+0000 7f6dfc853700 1 --2- 172.21.15.17:0/495397207 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6df805fc30 0x7f6df8144b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:33.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.261+0000 7f6dfc853700 1 -- 172.21.15.17:0/495397207 >> 172.21.15.17:0/495397207 conn(0x7f6df80b2b00 msgr2=0x7f6df8143980 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:33.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.261+0000 7f6dfc853700 1 -- 172.21.15.17:0/495397207 shutdown_connections 2022-08-01T19:43:33.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.261+0000 7f6dfc853700 1 -- 172.21.15.17:0/495397207 wait complete. 2022-08-01T19:43:33.270 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' created 2022-08-01T19:43:33.277 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-08-01T19:43:33.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.389+0000 7f76831e9700 1 Processor -- start 2022-08-01T19:43:33.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.393+0000 7f76831e9700 1 -- start start 2022-08-01T19:43:33.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.393+0000 7f76831e9700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f767c1445d0 0x7f767c144b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:33.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.393+0000 7f76831e9700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f767c0655b0 0x7f767c144ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:33.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.393+0000 7f76831e9700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f767c1454e0 con 0x7f767c060290 2022-08-01T19:43:33.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.393+0000 7f76831e9700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f767c065e10 con 0x7f767c0655b0 2022-08-01T19:43:33.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.393+0000 7f76831e9700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f767c060670 con 0x7f767c1445d0 2022-08-01T19:43:33.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.393+0000 7f7681786700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f767c0655b0 0x7f767c144ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:33.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.393+0000 7f7681786700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f767c0655b0 0x7f767c144ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44894/0 (socket says 172.21.15.17:44894) 2022-08-01T19:43:33.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.393+0000 7f7680f85700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f767c060290 0x7f767c064ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37084/0 (socket says 172.21.15.17:37084) 2022-08-01T19:43:33.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.393+0000 7f7681786700 1 -- 172.21.15.17:0/1315020442 learned_addr learned my addr 172.21.15.17:0/1315020442 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:33.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.393+0000 7f767bfff700 1 --2- 172.21.15.17:0/1315020442 >> v2:172.21.15.77:3300/0 conn(0x7f767c1445d0 0x7f767c144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:33.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.393+0000 7f7681786700 1 -- 172.21.15.17:0/1315020442 >> v2:172.21.15.77:3300/0 conn(0x7f767c1445d0 msgr2=0x7f767c144b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:33.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.393+0000 7f7681786700 1 --2- 172.21.15.17:0/1315020442 >> v2:172.21.15.77:3300/0 conn(0x7f767c1445d0 0x7f767c144b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:33.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.393+0000 7f7681786700 1 -- 172.21.15.17:0/1315020442 >> v1:172.21.15.17:6789/0 conn(0x7f767c060290 legacy=0x7f767c064ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:33.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.393+0000 7f767b7fe700 1 -- 172.21.15.17:0/1315020442 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1315254858 0 0) 0x7f767c1454e0 con 0x7f767c060290 2022-08-01T19:43:33.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.393+0000 7f7681786700 1 -- 172.21.15.17:0/1315020442 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767c0b2540 con 0x7f767c0655b0 2022-08-01T19:43:33.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.393+0000 7f7681786700 1 --2- 172.21.15.17:0/1315020442 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f767c0655b0 0x7f767c144ff0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 rx=0x7f76740032f0 tx=0x7f7674009290).ready entity=mon.0 client_cookie=88e7fa20480f0539 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:33.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.393+0000 7f767b7fe700 1 -- 172.21.15.17:0/1315020442 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f767400b870 con 0x7f767c0655b0 2022-08-01T19:43:33.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.393+0000 7f767b7fe700 1 -- 172.21.15.17:0/1315020442 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f767400b9d0 con 0x7f767c0655b0 2022-08-01T19:43:33.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.393+0000 7f767b7fe700 1 -- 172.21.15.17:0/1315020442 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f767400bb40 con 0x7f767c0655b0 2022-08-01T19:43:33.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.393+0000 7f76831e9700 1 -- 172.21.15.17:0/1315020442 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f767c0655b0 msgr2=0x7f767c144ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:33.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.393+0000 7f76831e9700 1 --2- 172.21.15.17:0/1315020442 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f767c0655b0 0x7f767c144ff0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 rx=0x7f76740032f0 tx=0x7f7674009290).stop 2022-08-01T19:43:33.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.393+0000 7f76831e9700 1 -- 172.21.15.17:0/1315020442 shutdown_connections 2022-08-01T19:43:33.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.393+0000 7f76831e9700 1 --2- 172.21.15.17:0/1315020442 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f767c0655b0 0x7f767c144ff0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:33.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.393+0000 7f76831e9700 1 --2- 172.21.15.17:0/1315020442 >> v2:172.21.15.77:3300/0 conn(0x7f767c1445d0 0x7f767c144b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:33.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.393+0000 7f76831e9700 1 -- 172.21.15.17:0/1315020442 >> 172.21.15.17:0/1315020442 conn(0x7f767c05d0f0 msgr2=0x7f767c05f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:33.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.393+0000 7f76831e9700 1 -- 172.21.15.17:0/1315020442 shutdown_connections 2022-08-01T19:43:33.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.393+0000 7f76831e9700 1 -- 172.21.15.17:0/1315020442 wait complete. 2022-08-01T19:43:33.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.397+0000 7f76831e9700 1 Processor -- start 2022-08-01T19:43:33.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.397+0000 7f76831e9700 1 -- start start 2022-08-01T19:43:33.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.397+0000 7f76831e9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f767c060290 0x7f767c0b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:33.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.397+0000 7f76831e9700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f767c0655b0 0x7f767c13c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:33.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.397+0000 7f76831e9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f767c13ba30 0x7f767c153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:33.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.397+0000 7f76831e9700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767c13ea40 con 0x7f767c13ba30 2022-08-01T19:43:33.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.397+0000 7f76831e9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767c154800 con 0x7f767c060290 2022-08-01T19:43:33.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.397+0000 7f76831e9700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f767c0b03d0 con 0x7f767c0655b0 2022-08-01T19:43:33.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.397+0000 7f7681786700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f767c13ba30 0x7f767c153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:33.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.397+0000 7f7680f85700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f767c060290 0x7f767c0b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:33.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.397+0000 7f7681786700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f767c13ba30 0x7f767c153e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44900/0 (socket says 172.21.15.17:44900) 2022-08-01T19:43:33.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.397+0000 7f7680f85700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f767c060290 0x7f767c0b0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:42818/0 (socket says 172.21.15.17:42818) 2022-08-01T19:43:33.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.397+0000 7f7681786700 1 -- 172.21.15.17:0/2179394582 learned_addr learned my addr 172.21.15.17:0/2179394582 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:33.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.397+0000 7f767bfff700 1 --2- 172.21.15.17:0/2179394582 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f767c0655b0 0x7f767c13c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:33.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.401+0000 7f7681786700 1 -- 172.21.15.17:0/2179394582 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f767c0655b0 msgr2=0x7f767c13c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:33.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.401+0000 7f7681786700 1 --2- 172.21.15.17:0/2179394582 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f767c0655b0 0x7f767c13c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:33.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.401+0000 7f7681786700 1 -- 172.21.15.17:0/2179394582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f767c060290 msgr2=0x7f767c0b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:33.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.401+0000 7f7681786700 1 --2- 172.21.15.17:0/2179394582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f767c060290 0x7f767c0b0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:33.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.401+0000 7f7681786700 1 -- 172.21.15.17:0/2179394582 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7674009530 con 0x7f767c13ba30 2022-08-01T19:43:33.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.401+0000 7f7680f85700 1 --2- 172.21.15.17:0/2179394582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f767c060290 0x7f767c0b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:43:33.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.401+0000 7f7681786700 1 --2- 172.21.15.17:0/2179394582 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f767c13ba30 0x7f767c153e30 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 rx=0x7f76740032f0 tx=0x7f7674016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:33.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.401+0000 7f7679ffb700 1 -- 172.21.15.17:0/2179394582 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f767400b870 con 0x7f767c13ba30 2022-08-01T19:43:33.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.401+0000 7f7679ffb700 1 -- 172.21.15.17:0/2179394582 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7674021630 con 0x7f767c13ba30 2022-08-01T19:43:33.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.401+0000 7f76831e9700 1 -- 172.21.15.17:0/2179394582 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f767c0b0670 con 0x7f767c13ba30 2022-08-01T19:43:33.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.401+0000 7f7679ffb700 1 -- 172.21.15.17:0/2179394582 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7674021990 con 0x7f767c13ba30 2022-08-01T19:43:33.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.401+0000 7f76831e9700 1 -- 172.21.15.17:0/2179394582 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f767c13b740 con 0x7f767c13ba30 2022-08-01T19:43:33.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.401+0000 7f76831e9700 1 -- 172.21.15.17:0/2179394582 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f767c0b0830 con 0x7f767c13ba30 2022-08-01T19:43:33.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.405+0000 7f7679ffb700 1 -- 172.21.15.17:0/2179394582 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7674029070 con 0x7f767c13ba30 2022-08-01T19:43:33.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.405+0000 7f7679ffb700 1 --2- 172.21.15.17:0/2179394582 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f765c053a70 0x7f765c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:33.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.405+0000 7f7679ffb700 1 -- 172.21.15.17:0/2179394582 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f767406d230 con 0x7f767c13ba30 2022-08-01T19:43:33.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.405+0000 7f7680f85700 1 --2- 172.21.15.17:0/2179394582 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f765c053a70 0x7f765c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:33.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.405+0000 7f7680f85700 1 --2- 172.21.15.17:0/2179394582 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f765c053a70 0x7f765c055f30 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f766c00d000 tx=0x7f766c007a00).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:33.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.409+0000 7f7679ffb700 1 -- 172.21.15.17:0/2179394582 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f76740440b0 con 0x7f767c13ba30 2022-08-01T19:43:33.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:33.537+0000 7f76831e9700 1 -- 172.21.15.17:0/2179394582 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f767c0b0200 con 0x7f767c13ba30 2022-08-01T19:43:34.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.261+0000 7f7679ffb700 1 -- 172.21.15.17:0/2179394582 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v145) v1 ==== 125+0+0 (secure 0 0 0) 0x7f76740150a0 con 0x7f767c13ba30 2022-08-01T19:43:34.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.269+0000 7f76831e9700 1 -- 172.21.15.17:0/2179394582 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f765c053a70 msgr2=0x7f765c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:34.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.269+0000 7f76831e9700 1 --2- 172.21.15.17:0/2179394582 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f765c053a70 0x7f765c055f30 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f766c00d000 tx=0x7f766c007a00).stop 2022-08-01T19:43:34.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.269+0000 7f76831e9700 1 -- 172.21.15.17:0/2179394582 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f767c13ba30 msgr2=0x7f767c153e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:34.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.269+0000 7f76831e9700 1 --2- 172.21.15.17:0/2179394582 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f767c13ba30 0x7f767c153e30 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 rx=0x7f76740032f0 tx=0x7f7674016000).stop 2022-08-01T19:43:34.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.273+0000 7f76831e9700 1 -- 172.21.15.17:0/2179394582 shutdown_connections 2022-08-01T19:43:34.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.273+0000 7f76831e9700 1 --2- 172.21.15.17:0/2179394582 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f765c053a70 0x7f765c055f30 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:34.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.273+0000 7f76831e9700 1 --2- 172.21.15.17:0/2179394582 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f767c13ba30 0x7f767c153e30 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:34.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.273+0000 7f76831e9700 1 --2- 172.21.15.17:0/2179394582 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f767c0655b0 0x7f767c13c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:34.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.273+0000 7f76831e9700 1 --2- 172.21.15.17:0/2179394582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f767c060290 0x7f767c0b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:34.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.273+0000 7f76831e9700 1 -- 172.21.15.17:0/2179394582 >> 172.21.15.17:0/2179394582 conn(0x7f767c05d0f0 msgr2=0x7f767c1437f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:34.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.273+0000 7f76831e9700 1 -- 172.21.15.17:0/2179394582 shutdown_connections 2022-08-01T19:43:34.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.273+0000 7f76831e9700 1 -- 172.21.15.17:0/2179394582 wait complete. 2022-08-01T19:43:34.280 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' created 2022-08-01T19:43:34.288 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-08-01T19:43:34.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.393+0000 7f090359e700 1 Processor -- start 2022-08-01T19:43:34.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.393+0000 7f090359e700 1 -- start start 2022-08-01T19:43:34.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.393+0000 7f090359e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f09040a9da0 0x7f0904060950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:34.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.393+0000 7f090359e700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0904067030 con 0x7f09040b31d0 2022-08-01T19:43:34.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.393+0000 7f090359e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09040b35b0 con 0x7f09041402d0 2022-08-01T19:43:34.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.393+0000 7f090359e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0904064a70 con 0x7f09040a9da0 2022-08-01T19:43:34.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.393+0000 7f0901d9b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f09040a9da0 0x7f0904060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:34.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.393+0000 7f090259c700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f09040b31d0 0x7f0904066020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37098/0 (socket says 172.21.15.17:37098) 2022-08-01T19:43:34.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.393+0000 7f090259c700 1 -- 172.21.15.17:0/167439188 learned_addr learned my addr 172.21.15.17:0/167439188 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:34.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.393+0000 7f0901d9b700 1 -- 172.21.15.17:0/167439188 >> v1:172.21.15.77:6789/0 conn(0x7f09041402d0 legacy=0x7f0904140720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:34.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.393+0000 7f090159a700 1 -- 172.21.15.17:0/167439188 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3497041571 0 0) 0x7f0904067030 con 0x7f09040b31d0 2022-08-01T19:43:34.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.393+0000 7f0901d9b700 1 -- 172.21.15.17:0/167439188 >> v1:172.21.15.17:6789/0 conn(0x7f09040b31d0 legacy=0x7f0904066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:34.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.393+0000 7f0901d9b700 1 -- 172.21.15.17:0/167439188 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0904066af0 con 0x7f09040a9da0 2022-08-01T19:43:34.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.393+0000 7f0901d9b700 1 --2- 172.21.15.17:0/167439188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09040a9da0 0x7f0904060950 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 rx=0x7f08f8002f70 tx=0x7f08f8006df0).ready entity=mon.0 client_cookie=fc86c591f612e91 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:34.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.393+0000 7f090159a700 1 -- 172.21.15.17:0/167439188 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f08f800d610 con 0x7f09040a9da0 2022-08-01T19:43:34.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.393+0000 7f090159a700 1 -- 172.21.15.17:0/167439188 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f8000e90 con 0x7f09040a9da0 2022-08-01T19:43:34.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.393+0000 7f090159a700 1 -- 172.21.15.17:0/167439188 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f08f800d960 con 0x7f09040a9da0 2022-08-01T19:43:34.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.397+0000 7f090359e700 1 -- 172.21.15.17:0/167439188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09040a9da0 msgr2=0x7f0904060950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:34.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.397+0000 7f090359e700 1 --2- 172.21.15.17:0/167439188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09040a9da0 0x7f0904060950 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 rx=0x7f08f8002f70 tx=0x7f08f8006df0).stop 2022-08-01T19:43:34.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.397+0000 7f090359e700 1 -- 172.21.15.17:0/167439188 shutdown_connections 2022-08-01T19:43:34.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.397+0000 7f090359e700 1 --2- 172.21.15.17:0/167439188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09040a9da0 0x7f0904060950 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:34.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.397+0000 7f090359e700 1 -- 172.21.15.17:0/167439188 >> 172.21.15.17:0/167439188 conn(0x7f0904063870 msgr2=0x7f0904063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:34.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.397+0000 7f090359e700 1 -- 172.21.15.17:0/167439188 shutdown_connections 2022-08-01T19:43:34.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.397+0000 7f090359e700 1 -- 172.21.15.17:0/167439188 wait complete. 2022-08-01T19:43:34.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.397+0000 7f090359e700 1 Processor -- start 2022-08-01T19:43:34.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.397+0000 7f090359e700 1 -- start start 2022-08-01T19:43:34.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.401+0000 7f090359e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09040a9da0 0x7f09040aa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:34.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.401+0000 7f090359e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09040b31d0 0x7f09041517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:34.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.401+0000 7f090359e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09041402d0 0x7f0904151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:34.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.401+0000 7f090359e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090414df20 con 0x7f09040a9da0 2022-08-01T19:43:34.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.401+0000 7f090359e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09041414f0 con 0x7f09040b31d0 2022-08-01T19:43:34.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.401+0000 7f090359e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0904141790 con 0x7f09041402d0 2022-08-01T19:43:34.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.401+0000 7f090259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09040a9da0 0x7f09040aa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:34.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.401+0000 7f090259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09040a9da0 0x7f09040aa9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44910/0 (socket says 172.21.15.17:44910) 2022-08-01T19:43:34.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.401+0000 7f090259c700 1 -- 172.21.15.17:0/2786627318 learned_addr learned my addr 172.21.15.17:0/2786627318 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:34.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.401+0000 7f0901d9b700 1 --2- 172.21.15.17:0/2786627318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09040b31d0 0x7f09041517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:34.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.401+0000 7f0902d9d700 1 --2- 172.21.15.17:0/2786627318 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09041402d0 0x7f0904151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:34.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.401+0000 7f090259c700 1 -- 172.21.15.17:0/2786627318 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09041402d0 msgr2=0x7f0904151ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:34.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.401+0000 7f090259c700 1 --2- 172.21.15.17:0/2786627318 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09041402d0 0x7f0904151ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:34.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.401+0000 7f090259c700 1 -- 172.21.15.17:0/2786627318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09040b31d0 msgr2=0x7f09041517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:34.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.401+0000 7f090259c700 1 --2- 172.21.15.17:0/2786627318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09040b31d0 0x7f09041517f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:34.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.401+0000 7f090259c700 1 -- 172.21.15.17:0/2786627318 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f800b180 con 0x7f09040a9da0 2022-08-01T19:43:34.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.401+0000 7f0902d9d700 1 --2- 172.21.15.17:0/2786627318 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09041402d0 0x7f0904151ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:43:34.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.401+0000 7f090259c700 1 --2- 172.21.15.17:0/2786627318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09040a9da0 0x7f09040aa9e0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 rx=0x7f08f4000bc0 tx=0x7f08f4007a60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:34.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.401+0000 7f08f37fe700 1 -- 172.21.15.17:0/2786627318 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f08f40070d0 con 0x7f09040a9da0 2022-08-01T19:43:34.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.401+0000 7f08f37fe700 1 -- 172.21.15.17:0/2786627318 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f4007230 con 0x7f09040a9da0 2022-08-01T19:43:34.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.401+0000 7f090359e700 1 -- 172.21.15.17:0/2786627318 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0904141a30 con 0x7f09040a9da0 2022-08-01T19:43:34.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.401+0000 7f08f37fe700 1 -- 172.21.15.17:0/2786627318 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f08f4005610 con 0x7f09040a9da0 2022-08-01T19:43:34.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.401+0000 7f090359e700 1 -- 172.21.15.17:0/2786627318 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09040aa350 con 0x7f09040a9da0 2022-08-01T19:43:34.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.401+0000 7f090359e700 1 -- 172.21.15.17:0/2786627318 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0904066d90 con 0x7f09040a9da0 2022-08-01T19:43:34.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.405+0000 7f08f37fe700 1 -- 172.21.15.17:0/2786627318 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f08f401a060 con 0x7f09040a9da0 2022-08-01T19:43:34.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.405+0000 7f08f37fe700 1 --2- 172.21.15.17:0/2786627318 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f08e0053aa0 0x7f08e0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:34.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.405+0000 7f08f37fe700 1 -- 172.21.15.17:0/2786627318 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f08f406b350 con 0x7f09040a9da0 2022-08-01T19:43:34.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.405+0000 7f0901d9b700 1 --2- 172.21.15.17:0/2786627318 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f08e0053aa0 0x7f08e0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:34.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.405+0000 7f0901d9b700 1 --2- 172.21.15.17:0/2786627318 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f08e0053aa0 0x7f08e0055f60 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f08f801c000 tx=0x7f08f8015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:34.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.409+0000 7f08f37fe700 1 -- 172.21.15.17:0/2786627318 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f08f4042190 con 0x7f09040a9da0 2022-08-01T19:43:34.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.581+0000 7f090359e700 1 -- 172.21.15.17:0/2786627318 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f09040574e0 con 0x7f09040a9da0 2022-08-01T19:43:34.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.589+0000 7f08f37fe700 1 -- 172.21.15.17:0/2786627318 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 27 and data pool 28 v74) v1 ==== 189+0+0 (secure 0 0 0) 0x7f08f4018020 con 0x7f09040a9da0 2022-08-01T19:43:34.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.601+0000 7f090359e700 1 -- 172.21.15.17:0/2786627318 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f08e0053aa0 msgr2=0x7f08e0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:34.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.601+0000 7f090359e700 1 --2- 172.21.15.17:0/2786627318 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f08e0053aa0 0x7f08e0055f60 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f08f801c000 tx=0x7f08f8015000).stop 2022-08-01T19:43:34.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.601+0000 7f090359e700 1 -- 172.21.15.17:0/2786627318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09040a9da0 msgr2=0x7f09040aa9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:34.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.601+0000 7f090359e700 1 --2- 172.21.15.17:0/2786627318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09040a9da0 0x7f09040aa9e0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 rx=0x7f08f4000bc0 tx=0x7f08f4007a60).stop 2022-08-01T19:43:34.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.601+0000 7f090359e700 1 -- 172.21.15.17:0/2786627318 shutdown_connections 2022-08-01T19:43:34.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.601+0000 7f090359e700 1 --2- 172.21.15.17:0/2786627318 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f08e0053aa0 0x7f08e0055f60 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:34.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.601+0000 7f090359e700 1 --2- 172.21.15.17:0/2786627318 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f09041402d0 0x7f0904151ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:34.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.601+0000 7f090359e700 1 --2- 172.21.15.17:0/2786627318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09040b31d0 0x7f09041517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:34.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.601+0000 7f090359e700 1 --2- 172.21.15.17:0/2786627318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09040a9da0 0x7f09040aa9e0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:34.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.601+0000 7f090359e700 1 -- 172.21.15.17:0/2786627318 >> 172.21.15.17:0/2786627318 conn(0x7f0904063870 msgr2=0x7f09040b0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:34.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.601+0000 7f090359e700 1 -- 172.21.15.17:0/2786627318 shutdown_connections 2022-08-01T19:43:34.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.601+0000 7f090359e700 1 -- 172.21.15.17:0/2786627318 wait complete. 2022-08-01T19:43:34.610 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 27 and data pool 28 2022-08-01T19:43:34.617 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:43:34.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.721+0000 7f205a11c700 1 Processor -- start 2022-08-01T19:43:34.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.721+0000 7f205a11c700 1 -- start start 2022-08-01T19:43:34.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.725+0000 7f205a11c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f205413d9b0 0x7f20540609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:34.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.725+0000 7f205a11c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f20540b0e60 0x7f2054060eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:34.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.725+0000 7f205a11c700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2054066af0 con 0x7f20540b6510 2022-08-01T19:43:34.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.725+0000 7f205a11c700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2054066d90 con 0x7f20540b0e60 2022-08-01T19:43:34.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.725+0000 7f205a11c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2054067030 con 0x7f205413d9b0 2022-08-01T19:43:34.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.725+0000 7f205991b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f20540b0e60 0x7f2054060eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:34.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.725+0000 7f205991b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f20540b0e60 0x7f2054060eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44932/0 (socket says 172.21.15.17:44932) 2022-08-01T19:43:34.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.725+0000 7f205991b700 1 -- 172.21.15.17:0/1162308193 learned_addr learned my addr 172.21.15.17:0/1162308193 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:34.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.725+0000 7f2058919700 1 --2- 172.21.15.17:0/1162308193 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f205413d9b0 0x7f20540609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:34.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.725+0000 7f205991b700 1 -- 172.21.15.17:0/1162308193 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f205413d9b0 msgr2=0x7f20540609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:34.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.725+0000 7f205991b700 1 --2- 172.21.15.17:0/1162308193 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f205413d9b0 0x7f20540609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:34.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.725+0000 7f205991b700 1 -- 172.21.15.17:0/1162308193 >> v1:172.21.15.77:6789/0 conn(0x7f20540b6510 legacy=0x7f205413d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:34.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.725+0000 7f205991b700 1 -- 172.21.15.17:0/1162308193 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20540672d0 con 0x7f20540b0e60 2022-08-01T19:43:34.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.725+0000 7f205991b700 1 --2- 172.21.15.17:0/1162308193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20540b0e60 0x7f2054060eb0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 rx=0x7f204c0032f0 tx=0x7f204c009290).ready entity=mon.0 client_cookie=67d5345a9d6d2469 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:34.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.725+0000 7f204bfff700 1 -- 172.21.15.17:0/1162308193 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f204c00ba90 con 0x7f20540b0e60 2022-08-01T19:43:34.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.725+0000 7f204bfff700 1 -- 172.21.15.17:0/1162308193 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204c00bbf0 con 0x7f20540b0e60 2022-08-01T19:43:34.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.725+0000 7f204bfff700 1 -- 172.21.15.17:0/1162308193 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f204c010470 con 0x7f20540b0e60 2022-08-01T19:43:34.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.725+0000 7f205a11c700 1 -- 172.21.15.17:0/1162308193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20540b0e60 msgr2=0x7f2054060eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:34.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.725+0000 7f205a11c700 1 --2- 172.21.15.17:0/1162308193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20540b0e60 0x7f2054060eb0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 rx=0x7f204c0032f0 tx=0x7f204c009290).stop 2022-08-01T19:43:34.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.725+0000 7f205a11c700 1 -- 172.21.15.17:0/1162308193 shutdown_connections 2022-08-01T19:43:34.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.725+0000 7f205a11c700 1 --2- 172.21.15.17:0/1162308193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20540b0e60 0x7f2054060eb0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:34.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.725+0000 7f205a11c700 1 --2- 172.21.15.17:0/1162308193 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f205413d9b0 0x7f20540609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:34.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.725+0000 7f205a11c700 1 -- 172.21.15.17:0/1162308193 >> 172.21.15.17:0/1162308193 conn(0x7f2054063800 msgr2=0x7f2054063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:34.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.725+0000 7f205a11c700 1 -- 172.21.15.17:0/1162308193 shutdown_connections 2022-08-01T19:43:34.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.725+0000 7f205a11c700 1 -- 172.21.15.17:0/1162308193 wait complete. 2022-08-01T19:43:34.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.729+0000 7f205a11c700 1 Processor -- start 2022-08-01T19:43:34.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.729+0000 7f205a11c700 1 -- start start 2022-08-01T19:43:34.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.729+0000 7f205a11c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f20540b0e60 0x7f205413f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:34.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.729+0000 7f205a11c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20540b6510 0x7f2054140220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:34.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.729+0000 7f205a11c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f205413d9b0 0x7f2054143aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:34.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.729+0000 7f205a11c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20540aa220 con 0x7f20540b6510 2022-08-01T19:43:34.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.729+0000 7f205a11c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205413fdc0 con 0x7f205413d9b0 2022-08-01T19:43:34.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.729+0000 7f205a11c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f205413efb0 con 0x7f20540b0e60 2022-08-01T19:43:34.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.729+0000 7f2058919700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20540b6510 0x7f2054140220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:34.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.729+0000 7f2058919700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20540b6510 0x7f2054140220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44936/0 (socket says 172.21.15.17:44936) 2022-08-01T19:43:34.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.729+0000 7f2058919700 1 -- 172.21.15.17:0/323155968 learned_addr learned my addr 172.21.15.17:0/323155968 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:34.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.729+0000 7f205991b700 1 --2- 172.21.15.17:0/323155968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f205413d9b0 0x7f2054143aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:34.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.729+0000 7f205911a700 1 --2- 172.21.15.17:0/323155968 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f20540b0e60 0x7f205413f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:34.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.729+0000 7f2058919700 1 -- 172.21.15.17:0/323155968 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f20540b0e60 msgr2=0x7f205413f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:34.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.729+0000 7f2058919700 1 --2- 172.21.15.17:0/323155968 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f20540b0e60 0x7f205413f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:34.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.729+0000 7f2058919700 1 -- 172.21.15.17:0/323155968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f205413d9b0 msgr2=0x7f2054143aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:34.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.729+0000 7f2058919700 1 --2- 172.21.15.17:0/323155968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f205413d9b0 0x7f2054143aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:34.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.729+0000 7f2058919700 1 -- 172.21.15.17:0/323155968 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f204c009530 con 0x7f20540b6510 2022-08-01T19:43:34.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.729+0000 7f205911a700 1 --2- 172.21.15.17:0/323155968 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f20540b0e60 0x7f205413f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:43:34.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.729+0000 7f2058919700 1 --2- 172.21.15.17:0/323155968 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20540b6510 0x7f2054140220 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 rx=0x7f205000c000 tx=0x7f2050006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:34.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.729+0000 7f204a7fc700 1 -- 172.21.15.17:0/323155968 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2050010780 con 0x7f20540b6510 2022-08-01T19:43:34.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.729+0000 7f204a7fc700 1 -- 172.21.15.17:0/323155968 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20500108e0 con 0x7f20540b6510 2022-08-01T19:43:34.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.729+0000 7f205a11c700 1 -- 172.21.15.17:0/323155968 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2054143430 con 0x7f20540b6510 2022-08-01T19:43:34.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.729+0000 7f205a11c700 1 -- 172.21.15.17:0/323155968 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f205413ec40 con 0x7f20540b6510 2022-08-01T19:43:34.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.729+0000 7f204a7fc700 1 -- 172.21.15.17:0/323155968 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2050010c40 con 0x7f20540b6510 2022-08-01T19:43:34.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.729+0000 7f205a11c700 1 -- 172.21.15.17:0/323155968 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f205413f170 con 0x7f20540b6510 2022-08-01T19:43:34.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.733+0000 7f204a7fc700 1 -- 172.21.15.17:0/323155968 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f205001c460 con 0x7f20540b6510 2022-08-01T19:43:34.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.733+0000 7f204a7fc700 1 --2- 172.21.15.17:0/323155968 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f203c053950 0x7f203c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:34.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.733+0000 7f204a7fc700 1 -- 172.21.15.17:0/323155968 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f205006c560 con 0x7f20540b6510 2022-08-01T19:43:34.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.733+0000 7f205911a700 1 --2- 172.21.15.17:0/323155968 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f203c053950 0x7f203c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:34.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.733+0000 7f205911a700 1 --2- 172.21.15.17:0/323155968 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f203c053950 0x7f203c055e10 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f204400b000 tx=0x7f2044009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:34.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.737+0000 7f204a7fc700 1 -- 172.21.15.17:0/323155968 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2050043220 con 0x7f20540b6510 2022-08-01T19:43:34.870 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.869+0000 7f205a11c700 1 -- 172.21.15.17:0/323155968 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20540574e0 con 0x7f20540b6510 2022-08-01T19:43:34.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.869+0000 7f204a7fc700 1 -- 172.21.15.17:0/323155968 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f2050046840 con 0x7f20540b6510 2022-08-01T19:43:34.873 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:34.874 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":147,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:43:34.585759+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:43:33.209247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2022-08-01T19:43:33.595201+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:43:34.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.873+0000 7f205a11c700 1 -- 172.21.15.17:0/323155968 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f203c053950 msgr2=0x7f203c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:34.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.873+0000 7f205a11c700 1 --2- 172.21.15.17:0/323155968 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f203c053950 0x7f203c055e10 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f204400b000 tx=0x7f2044009040).stop 2022-08-01T19:43:34.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.873+0000 7f205a11c700 1 -- 172.21.15.17:0/323155968 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20540b6510 msgr2=0x7f2054140220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:34.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.873+0000 7f205a11c700 1 --2- 172.21.15.17:0/323155968 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20540b6510 0x7f2054140220 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 rx=0x7f205000c000 tx=0x7f2050006df0).stop 2022-08-01T19:43:34.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.873+0000 7f205a11c700 1 -- 172.21.15.17:0/323155968 shutdown_connections 2022-08-01T19:43:34.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.873+0000 7f205a11c700 1 --2- 172.21.15.17:0/323155968 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f203c053950 0x7f203c055e10 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:34.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.873+0000 7f205a11c700 1 --2- 172.21.15.17:0/323155968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f205413d9b0 0x7f2054143aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:34.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.873+0000 7f205a11c700 1 --2- 172.21.15.17:0/323155968 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20540b6510 0x7f2054140220 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:34.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.873+0000 7f205a11c700 1 --2- 172.21.15.17:0/323155968 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f20540b0e60 0x7f205413f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:34.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.873+0000 7f205a11c700 1 -- 172.21.15.17:0/323155968 >> 172.21.15.17:0/323155968 conn(0x7f2054063800 msgr2=0x7f205413b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:34.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.873+0000 7f205a11c700 1 -- 172.21.15.17:0/323155968 shutdown_connections 2022-08-01T19:43:34.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.873+0000 7f205a11c700 1 -- 172.21.15.17:0/323155968 wait complete. 2022-08-01T19:43:34.890 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:43:34.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.989+0000 7f39cf4b5700 1 Processor -- start 2022-08-01T19:43:34.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.993+0000 7f39cf4b5700 1 -- start start 2022-08-01T19:43:34.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.993+0000 7f39cf4b5700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f39c80b6710 0x7f39c80b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:34.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.993+0000 7f39cf4b5700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39c8144240 con 0x7f39c80b4550 2022-08-01T19:43:34.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.993+0000 7f39cf4b5700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39c8143d00 con 0x7f39c80b6af0 2022-08-01T19:43:34.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.993+0000 7f39cf4b5700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39c8143fa0 con 0x7f39c80b6710 2022-08-01T19:43:34.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.993+0000 7f39cd251700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f39c80b6710 0x7f39c80b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:34.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.993+0000 7f39cca50700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f39c80b4550 0x7f39c8061ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37136/0 (socket says 172.21.15.17:37136) 2022-08-01T19:43:34.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.993+0000 7f39cca50700 1 -- 172.21.15.17:0/1415468792 learned_addr learned my addr 172.21.15.17:0/1415468792 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:34.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.993+0000 7f39bbfff700 1 -- 172.21.15.17:0/1415468792 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1289868481 0 0) 0x7f39c8144240 con 0x7f39c80b4550 2022-08-01T19:43:34.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.993+0000 7f39bbfff700 1 -- 172.21.15.17:0/1415468792 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39ac003260 con 0x7f39c80b4550 2022-08-01T19:43:35.007 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.993+0000 7f39cd251700 1 -- 172.21.15.17:0/1415468792 >> v1:172.21.15.77:6789/0 conn(0x7f39c80b6af0 legacy=0x7f39c812bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:35.008 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.993+0000 7f39cd251700 1 -- 172.21.15.17:0/1415468792 >> v1:172.21.15.17:6789/0 conn(0x7f39c80b4550 legacy=0x7f39c8061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:35.008 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.993+0000 7f39cd251700 1 -- 172.21.15.17:0/1415468792 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c8061780 con 0x7f39c80b6710 2022-08-01T19:43:35.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.993+0000 7f39cd251700 1 --2- 172.21.15.17:0/1415468792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39c80b6710 0x7f39c80b5d80 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 rx=0x7f39c81432b0 tx=0x7f39c4006df0).ready entity=mon.0 client_cookie=2c890146fb42649 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:35.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.993+0000 7f39bbfff700 1 -- 172.21.15.17:0/1415468792 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39c400d610 con 0x7f39c80b6710 2022-08-01T19:43:35.010 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.993+0000 7f39bbfff700 1 -- 172.21.15.17:0/1415468792 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c4000e90 con 0x7f39c80b6710 2022-08-01T19:43:35.010 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.993+0000 7f39bbfff700 1 -- 172.21.15.17:0/1415468792 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39c400d960 con 0x7f39c80b6710 2022-08-01T19:43:35.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.993+0000 7f39cf4b5700 1 -- 172.21.15.17:0/1415468792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39c80b6710 msgr2=0x7f39c80b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:35.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.993+0000 7f39cf4b5700 1 --2- 172.21.15.17:0/1415468792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39c80b6710 0x7f39c80b5d80 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 rx=0x7f39c81432b0 tx=0x7f39c4006df0).stop 2022-08-01T19:43:35.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.993+0000 7f39cf4b5700 1 -- 172.21.15.17:0/1415468792 shutdown_connections 2022-08-01T19:43:35.013 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.993+0000 7f39cf4b5700 1 --2- 172.21.15.17:0/1415468792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39c80b6710 0x7f39c80b5d80 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:35.013 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.993+0000 7f39cf4b5700 1 -- 172.21.15.17:0/1415468792 >> 172.21.15.17:0/1415468792 conn(0x7f39c805c620 msgr2=0x7f39c805ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:35.014 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.993+0000 7f39cf4b5700 1 -- 172.21.15.17:0/1415468792 shutdown_connections 2022-08-01T19:43:35.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.993+0000 7f39cf4b5700 1 -- 172.21.15.17:0/1415468792 wait complete. 2022-08-01T19:43:35.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.997+0000 7f39cf4b5700 1 Processor -- start 2022-08-01T19:43:35.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.997+0000 7f39cf4b5700 1 -- start start 2022-08-01T19:43:35.016 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.997+0000 7f39cf4b5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39c80b4550 0x7f39c812af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:35.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.997+0000 7f39cf4b5700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f39c80b6710 0x7f39c80b8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:35.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.997+0000 7f39cf4b5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39c80b6af0 0x7f39c80b92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:35.017 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.997+0000 7f39cf4b5700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c80b80a0 con 0x7f39c80b4550 2022-08-01T19:43:35.018 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.997+0000 7f39cf4b5700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c80b7ae0 con 0x7f39c80b6af0 2022-08-01T19:43:35.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.997+0000 7f39cf4b5700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c80aee60 con 0x7f39c80b6710 2022-08-01T19:43:35.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.997+0000 7f39cd251700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39c80b4550 0x7f39c812af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:35.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.997+0000 7f39cd251700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39c80b4550 0x7f39c812af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44948/0 (socket says 172.21.15.17:44948) 2022-08-01T19:43:35.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.997+0000 7f39cd251700 1 -- 172.21.15.17:0/3780380712 learned_addr learned my addr 172.21.15.17:0/3780380712 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:35.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:34.997+0000 7f39cda52700 1 --2- 172.21.15.17:0/3780380712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39c80b6af0 0x7f39c80b92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:35.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.001+0000 7f39cca50700 1 --2- 172.21.15.17:0/3780380712 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f39c80b6710 0x7f39c80b8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:35.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.001+0000 7f39cd251700 1 -- 172.21.15.17:0/3780380712 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f39c80b6710 msgr2=0x7f39c80b8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:35.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.001+0000 7f39cd251700 1 --2- 172.21.15.17:0/3780380712 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f39c80b6710 0x7f39c80b8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:35.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.001+0000 7f39cd251700 1 -- 172.21.15.17:0/3780380712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39c80b6af0 msgr2=0x7f39c80b92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:35.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.001+0000 7f39cd251700 1 --2- 172.21.15.17:0/3780380712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39c80b6af0 0x7f39c80b92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:35.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.001+0000 7f39cd251700 1 -- 172.21.15.17:0/3780380712 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c400b180 con 0x7f39c80b4550 2022-08-01T19:43:35.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.001+0000 7f39cda52700 1 --2- 172.21.15.17:0/3780380712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39c80b6af0 0x7f39c80b92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:43:35.026 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.001+0000 7f39cca50700 1 --2- 172.21.15.17:0/3780380712 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f39c80b6710 0x7f39c80b8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:43:35.026 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.001+0000 7f39cd251700 1 --2- 172.21.15.17:0/3780380712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39c80b4550 0x7f39c812af70 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 rx=0x7f39c81432b0 tx=0x7f39c4014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:35.027 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.001+0000 7f39ba7fc700 1 -- 172.21.15.17:0/3780380712 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39c4016070 con 0x7f39c80b4550 2022-08-01T19:43:35.027 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.001+0000 7f39ba7fc700 1 -- 172.21.15.17:0/3780380712 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c400de90 con 0x7f39c80b4550 2022-08-01T19:43:35.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.001+0000 7f39cf4b5700 1 -- 172.21.15.17:0/3780380712 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39c80af090 con 0x7f39c80b4550 2022-08-01T19:43:35.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.001+0000 7f39ba7fc700 1 -- 172.21.15.17:0/3780380712 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39c4002f70 con 0x7f39c80b4550 2022-08-01T19:43:35.029 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.001+0000 7f39cf4b5700 1 -- 172.21.15.17:0/3780380712 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c80aebb0 con 0x7f39c80b4550 2022-08-01T19:43:35.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.001+0000 7f39cf4b5700 1 -- 172.21.15.17:0/3780380712 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39c8061a20 con 0x7f39c80b4550 2022-08-01T19:43:35.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.001+0000 7f39ba7fc700 1 -- 172.21.15.17:0/3780380712 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f39c400f030 con 0x7f39c80b4550 2022-08-01T19:43:35.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.001+0000 7f39ba7fc700 1 --2- 172.21.15.17:0/3780380712 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f39b0053b30 0x7f39b0055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:35.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.001+0000 7f39ba7fc700 1 -- 172.21.15.17:0/3780380712 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f39c406ce00 con 0x7f39c80b4550 2022-08-01T19:43:35.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.001+0000 7f39cca50700 1 --2- 172.21.15.17:0/3780380712 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f39b0053b30 0x7f39b0055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:35.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.005+0000 7f39cca50700 1 --2- 172.21.15.17:0/3780380712 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f39b0053b30 0x7f39b0055ff0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f39bc00b000 tx=0x7f39bc009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:35.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.013+0000 7f39ba7fc700 1 -- 172.21.15.17:0/3780380712 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f39c4043b20 con 0x7f39c80b4550 2022-08-01T19:43:35.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.137+0000 7f39cf4b5700 1 -- 172.21.15.17:0/3780380712 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39c80574e0 con 0x7f39c80b4550 2022-08-01T19:43:35.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.137+0000 7f39ba7fc700 1 -- 172.21.15.17:0/3780380712 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+16298 (secure 0 0 0) 0x7f39c4023070 con 0x7f39c80b4550 2022-08-01T19:43:35.161 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:35.161 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":147,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:43:34.585759+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:43:33.209247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2022-08-01T19:43:33.595201+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:43:35.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.141+0000 7f39cf4b5700 1 -- 172.21.15.17:0/3780380712 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f39b0053b30 msgr2=0x7f39b0055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:35.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.141+0000 7f39cf4b5700 1 --2- 172.21.15.17:0/3780380712 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f39b0053b30 0x7f39b0055ff0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f39bc00b000 tx=0x7f39bc009040).stop 2022-08-01T19:43:35.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.141+0000 7f39cf4b5700 1 -- 172.21.15.17:0/3780380712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39c80b4550 msgr2=0x7f39c812af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:35.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.141+0000 7f39cf4b5700 1 --2- 172.21.15.17:0/3780380712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39c80b4550 0x7f39c812af70 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 rx=0x7f39c81432b0 tx=0x7f39c4014000).stop 2022-08-01T19:43:35.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.141+0000 7f39cf4b5700 1 -- 172.21.15.17:0/3780380712 shutdown_connections 2022-08-01T19:43:35.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.141+0000 7f39cf4b5700 1 --2- 172.21.15.17:0/3780380712 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f39b0053b30 0x7f39b0055ff0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:35.169 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.141+0000 7f39cf4b5700 1 --2- 172.21.15.17:0/3780380712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39c80b6af0 0x7f39c80b92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:35.169 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.141+0000 7f39cf4b5700 1 --2- 172.21.15.17:0/3780380712 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f39c80b6710 0x7f39c80b8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:35.169 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.141+0000 7f39cf4b5700 1 --2- 172.21.15.17:0/3780380712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39c80b4550 0x7f39c812af70 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:35.169 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.141+0000 7f39cf4b5700 1 -- 172.21.15.17:0/3780380712 >> 172.21.15.17:0/3780380712 conn(0x7f39c805c620 msgr2=0x7f39c8141ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:35.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.141+0000 7f39cf4b5700 1 -- 172.21.15.17:0/3780380712 shutdown_connections 2022-08-01T19:43:35.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.141+0000 7f39cf4b5700 1 -- 172.21.15.17:0/3780380712 wait complete. 2022-08-01T19:43:35.180 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-08-01T19:43:35.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.289+0000 7f5bc615b700 1 Processor -- start 2022-08-01T19:43:35.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.289+0000 7f5bc615b700 1 -- start start 2022-08-01T19:43:35.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.289+0000 7f5bc615b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5bc00b31f0 0x7f5bc00b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:35.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.289+0000 7f5bc615b700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bc00b2510 con 0x7f5bc00b7500 2022-08-01T19:43:35.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.289+0000 7f5bc615b700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bc01443e0 con 0x7f5bc00aadb0 2022-08-01T19:43:35.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.289+0000 7f5bc615b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc00b3700 con 0x7f5bc00b31f0 2022-08-01T19:43:35.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.289+0000 7f5bc5159700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5bc00b7500 0x7f5bc013e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37150/0 (socket says 172.21.15.17:37150) 2022-08-01T19:43:35.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.289+0000 7f5bc5159700 1 -- 172.21.15.17:0/1322373927 learned_addr learned my addr 172.21.15.17:0/1322373927 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:35.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.289+0000 7f5bc4958700 1 --2- 172.21.15.17:0/1322373927 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5bc00b31f0 0x7f5bc00b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:35.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.293+0000 7f5baffff700 1 -- 172.21.15.17:0/1322373927 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2362458527 0 0) 0x7f5bc00b2510 con 0x7f5bc00b7500 2022-08-01T19:43:35.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.293+0000 7f5baffff700 1 -- 172.21.15.17:0/1322373927 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ba8003260 con 0x7f5bc00b7500 2022-08-01T19:43:35.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.293+0000 7f5bc4958700 1 -- 172.21.15.17:0/1322373927 >> v1:172.21.15.77:6789/0 conn(0x7f5bc00aadb0 legacy=0x7f5bc0064d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:35.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.293+0000 7f5bc4958700 1 -- 172.21.15.17:0/1322373927 >> v1:172.21.15.17:6789/0 conn(0x7f5bc00b7500 legacy=0x7f5bc013e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:35.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.293+0000 7f5baffff700 1 -- 172.21.15.17:0/1322373927 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2472443626 0 0) 0x7f5bc01443e0 con 0x7f5bc00aadb0 2022-08-01T19:43:35.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.293+0000 7f5bc4958700 1 -- 172.21.15.17:0/1322373927 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc00b2740 con 0x7f5bc00b31f0 2022-08-01T19:43:35.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.293+0000 7f5bc4958700 1 --2- 172.21.15.17:0/1322373927 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5bc00b31f0 0x7f5bc00b78e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f5bb0002f70 tx=0x7f5bb0006df0).ready entity=mon.2 client_cookie=58cb7377c6e72a5e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:35.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.293+0000 7f5baffff700 1 -- 172.21.15.17:0/1322373927 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bb000d570 con 0x7f5bc00b31f0 2022-08-01T19:43:35.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.293+0000 7f5baffff700 1 -- 172.21.15.17:0/1322373927 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb0000e90 con 0x7f5bc00b31f0 2022-08-01T19:43:35.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.293+0000 7f5baffff700 1 -- 172.21.15.17:0/1322373927 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bb000d8c0 con 0x7f5bc00b31f0 2022-08-01T19:43:35.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.293+0000 7f5bc615b700 1 -- 172.21.15.17:0/1322373927 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5bc00b31f0 msgr2=0x7f5bc00b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:35.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.293+0000 7f5bc615b700 1 --2- 172.21.15.17:0/1322373927 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5bc00b31f0 0x7f5bc00b78e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f5bb0002f70 tx=0x7f5bb0006df0).stop 2022-08-01T19:43:35.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.293+0000 7f5bc615b700 1 -- 172.21.15.17:0/1322373927 shutdown_connections 2022-08-01T19:43:35.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.293+0000 7f5bc615b700 1 --2- 172.21.15.17:0/1322373927 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5bc00b31f0 0x7f5bc00b78e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:35.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.293+0000 7f5bc615b700 1 -- 172.21.15.17:0/1322373927 >> 172.21.15.17:0/1322373927 conn(0x7f5bc005f840 msgr2=0x7f5bc005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:35.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.293+0000 7f5bc615b700 1 -- 172.21.15.17:0/1322373927 shutdown_connections 2022-08-01T19:43:35.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.293+0000 7f5bc615b700 1 -- 172.21.15.17:0/1322373927 wait complete. 2022-08-01T19:43:35.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.293+0000 7f5bc615b700 1 Processor -- start 2022-08-01T19:43:35.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.293+0000 7f5bc615b700 1 -- start start 2022-08-01T19:43:35.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.297+0000 7f5bc615b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5bc00aadb0 0x7f5bc00add70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:35.330 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.297+0000 7f5bc615b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bc00b31f0 0x7f5bc01466e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:35.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.297+0000 7f5bc615b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5bc00b7500 0x7f5bc00abd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:35.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.297+0000 7f5bc615b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc0148260 con 0x7f5bc00b31f0 2022-08-01T19:43:35.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.297+0000 7f5bc615b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc00ace50 con 0x7f5bc00aadb0 2022-08-01T19:43:35.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.297+0000 7f5bc615b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc00ad0f0 con 0x7f5bc00b7500 2022-08-01T19:43:35.332 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.297+0000 7f5bc4958700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bc00b31f0 0x7f5bc01466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:35.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.297+0000 7f5bc5159700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5bc00aadb0 0x7f5bc00add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:35.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.297+0000 7f5bc4958700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bc00b31f0 0x7f5bc01466e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44964/0 (socket says 172.21.15.17:44964) 2022-08-01T19:43:35.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.297+0000 7f5bc4958700 1 -- 172.21.15.17:0/3472681052 learned_addr learned my addr 172.21.15.17:0/3472681052 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:35.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.297+0000 7f5bc595a700 1 --2- 172.21.15.17:0/3472681052 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5bc00b7500 0x7f5bc00abd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:35.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.297+0000 7f5bc4958700 1 -- 172.21.15.17:0/3472681052 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5bc00b7500 msgr2=0x7f5bc00abd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:35.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.297+0000 7f5bc4958700 1 --2- 172.21.15.17:0/3472681052 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5bc00b7500 0x7f5bc00abd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:35.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.297+0000 7f5bc4958700 1 -- 172.21.15.17:0/3472681052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5bc00aadb0 msgr2=0x7f5bc00add70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:35.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.297+0000 7f5bc4958700 1 --2- 172.21.15.17:0/3472681052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5bc00aadb0 0x7f5bc00add70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:35.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.297+0000 7f5bc4958700 1 -- 172.21.15.17:0/3472681052 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb000b180 con 0x7f5bc00b31f0 2022-08-01T19:43:35.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.297+0000 7f5bc4958700 1 --2- 172.21.15.17:0/3472681052 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bc00b31f0 0x7f5bc01466e0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 rx=0x7f5bb0002f70 tx=0x7f5bb0014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:35.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.297+0000 7f5bae7fc700 1 -- 172.21.15.17:0/3472681052 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bb0013d60 con 0x7f5bc00b31f0 2022-08-01T19:43:35.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.297+0000 7f5bae7fc700 1 -- 172.21.15.17:0/3472681052 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb001e400 con 0x7f5bc00b31f0 2022-08-01T19:43:35.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.297+0000 7f5bc615b700 1 -- 172.21.15.17:0/3472681052 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bc00ad390 con 0x7f5bc00b31f0 2022-08-01T19:43:35.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.297+0000 7f5bae7fc700 1 -- 172.21.15.17:0/3472681052 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bb001e760 con 0x7f5bc00b31f0 2022-08-01T19:43:35.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.297+0000 7f5bc615b700 1 -- 172.21.15.17:0/3472681052 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc00ad8d0 con 0x7f5bc00b31f0 2022-08-01T19:43:35.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.301+0000 7f5bc615b700 1 -- 172.21.15.17:0/3472681052 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bc00574e0 con 0x7f5bc00b31f0 2022-08-01T19:43:35.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.301+0000 7f5bae7fc700 1 -- 172.21.15.17:0/3472681052 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5bb0025030 con 0x7f5bc00b31f0 2022-08-01T19:43:35.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.301+0000 7f5bae7fc700 1 --2- 172.21.15.17:0/3472681052 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5ba0053950 0x7f5ba0055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:35.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.301+0000 7f5bae7fc700 1 -- 172.21.15.17:0/3472681052 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5bb006cfa0 con 0x7f5bc00b31f0 2022-08-01T19:43:35.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.301+0000 7f5bc5159700 1 --2- 172.21.15.17:0/3472681052 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5ba0053950 0x7f5ba0055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:35.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.301+0000 7f5bc5159700 1 --2- 172.21.15.17:0/3472681052 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5ba0053950 0x7f5ba0055e10 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f5bbc00d000 tx=0x7f5bbc009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:35.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.305+0000 7f5bae7fc700 1 -- 172.21.15.17:0/3472681052 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5bb0043d60 con 0x7f5bc00b31f0 2022-08-01T19:43:35.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:35.465+0000 7f5bc615b700 1 -- 172.21.15.17:0/3472681052 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5bc00aab00 con 0x7f5bc00b31f0 2022-08-01T19:43:36.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.285+0000 7f5bae7fc700 1 -- 172.21.15.17:0/3472681052 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v76) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5bb0023070 con 0x7f5bc00b31f0 2022-08-01T19:43:36.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.289+0000 7f5bc615b700 1 -- 172.21.15.17:0/3472681052 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5ba0053950 msgr2=0x7f5ba0055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:36.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.289+0000 7f5bc615b700 1 --2- 172.21.15.17:0/3472681052 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5ba0053950 0x7f5ba0055e10 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f5bbc00d000 tx=0x7f5bbc009040).stop 2022-08-01T19:43:36.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.289+0000 7f5bc615b700 1 -- 172.21.15.17:0/3472681052 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bc00b31f0 msgr2=0x7f5bc01466e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:36.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.289+0000 7f5bc615b700 1 --2- 172.21.15.17:0/3472681052 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bc00b31f0 0x7f5bc01466e0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 rx=0x7f5bb0002f70 tx=0x7f5bb0014000).stop 2022-08-01T19:43:36.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.289+0000 7f5bc615b700 1 -- 172.21.15.17:0/3472681052 shutdown_connections 2022-08-01T19:43:36.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.289+0000 7f5bc615b700 1 --2- 172.21.15.17:0/3472681052 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5ba0053950 0x7f5ba0055e10 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:36.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.289+0000 7f5bc615b700 1 --2- 172.21.15.17:0/3472681052 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5bc00b7500 0x7f5bc00abd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:36.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.289+0000 7f5bc615b700 1 --2- 172.21.15.17:0/3472681052 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bc00b31f0 0x7f5bc01466e0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:36.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.289+0000 7f5bc615b700 1 --2- 172.21.15.17:0/3472681052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5bc00aadb0 0x7f5bc00add70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:36.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.289+0000 7f5bc615b700 1 -- 172.21.15.17:0/3472681052 >> 172.21.15.17:0/3472681052 conn(0x7f5bc005f840 msgr2=0x7f5bc013f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:36.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.289+0000 7f5bc615b700 1 -- 172.21.15.17:0/3472681052 shutdown_connections 2022-08-01T19:43:36.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.289+0000 7f5bc615b700 1 -- 172.21.15.17:0/3472681052 wait complete. 2022-08-01T19:43:36.314 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:43:36.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.417+0000 7f6e4eb71700 1 Processor -- start 2022-08-01T19:43:36.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.421+0000 7f6e4eb71700 1 -- start start 2022-08-01T19:43:36.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.421+0000 7f6e4eb71700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6e48148530 0x7f6e480b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:36.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.421+0000 7f6e4eb71700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f6e480a7250 0x7f6e481448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:36.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.421+0000 7f6e4eb71700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e480b04e0 con 0x7f6e480b1f40 2022-08-01T19:43:36.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.421+0000 7f6e4eb71700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e480a9250 con 0x7f6e480a7250 2022-08-01T19:43:36.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.421+0000 7f6e4eb71700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e480b0780 con 0x7f6e48148530 2022-08-01T19:43:36.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.421+0000 7f6e4d10e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f6e480a7250 0x7f6e481448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:36.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.421+0000 7f6e47fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6e48148530 0x7f6e480b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:36.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.421+0000 7f6e4d10e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f6e480a7250 0x7f6e481448b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:42896/0 (socket says 172.21.15.17:42896) 2022-08-01T19:43:36.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.421+0000 7f6e4d10e700 1 -- 172.21.15.17:0/3803742518 learned_addr learned my addr 172.21.15.17:0/3803742518 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:36.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.421+0000 7f6e4c90d700 1 --1- 172.21.15.17:0/3803742518 >> v1:172.21.15.77:6789/0 conn(0x7f6e480b1f40 0x7f6e480b2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:53964/0 (socket says 172.21.15.17:53964) 2022-08-01T19:43:36.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.421+0000 7f6e47fff700 1 -- 172.21.15.17:0/3803742518 >> v2:172.21.15.77:3300/0 conn(0x7f6e480a7250 msgr2=0x7f6e481448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:36.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.421+0000 7f6e47fff700 1 --2- 172.21.15.17:0/3803742518 >> v2:172.21.15.77:3300/0 conn(0x7f6e480a7250 0x7f6e481448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:36.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.421+0000 7f6e47fff700 1 -- 172.21.15.17:0/3803742518 >> v1:172.21.15.77:6789/0 conn(0x7f6e480b1f40 legacy=0x7f6e480b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:36.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.421+0000 7f6e47fff700 1 -- 172.21.15.17:0/3803742518 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e480b0a20 con 0x7f6e48148530 2022-08-01T19:43:36.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.421+0000 7f6e47fff700 1 --2- 172.21.15.17:0/3803742518 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6e48148530 0x7f6e480b1160 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f6e38002f70 tx=0x7f6e38006df0).ready entity=mon.2 client_cookie=e9f77f0f47932976 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:36.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.421+0000 7f6e477fe700 1 -- 172.21.15.17:0/3803742518 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e3800d570 con 0x7f6e48148530 2022-08-01T19:43:36.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.421+0000 7f6e477fe700 1 -- 172.21.15.17:0/3803742518 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e38000e90 con 0x7f6e48148530 2022-08-01T19:43:36.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.421+0000 7f6e477fe700 1 -- 172.21.15.17:0/3803742518 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e3800d8c0 con 0x7f6e48148530 2022-08-01T19:43:36.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.421+0000 7f6e4eb71700 1 -- 172.21.15.17:0/3803742518 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6e48148530 msgr2=0x7f6e480b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:36.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.421+0000 7f6e4eb71700 1 --2- 172.21.15.17:0/3803742518 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6e48148530 0x7f6e480b1160 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f6e38002f70 tx=0x7f6e38006df0).stop 2022-08-01T19:43:36.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.421+0000 7f6e4eb71700 1 -- 172.21.15.17:0/3803742518 shutdown_connections 2022-08-01T19:43:36.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.421+0000 7f6e4eb71700 1 --2- 172.21.15.17:0/3803742518 >> v2:172.21.15.77:3300/0 conn(0x7f6e480a7250 0x7f6e481448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:36.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.421+0000 7f6e4eb71700 1 --2- 172.21.15.17:0/3803742518 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6e48148530 0x7f6e480b1160 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:36.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.421+0000 7f6e4eb71700 1 -- 172.21.15.17:0/3803742518 >> 172.21.15.17:0/3803742518 conn(0x7f6e4805f840 msgr2=0x7f6e4805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:36.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.421+0000 7f6e4eb71700 1 -- 172.21.15.17:0/3803742518 shutdown_connections 2022-08-01T19:43:36.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.421+0000 7f6e4eb71700 1 -- 172.21.15.17:0/3803742518 wait complete. 2022-08-01T19:43:36.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.425+0000 7f6e4eb71700 1 Processor -- start 2022-08-01T19:43:36.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.425+0000 7f6e4eb71700 1 -- start start 2022-08-01T19:43:36.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.429+0000 7f6e4eb71700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e480a7250 0x7f6e4815a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:36.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.429+0000 7f6e4eb71700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e480b1f40 0x7f6e481429c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:36.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.429+0000 7f6e4eb71700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6e48148530 0x7f6e48142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:36.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.429+0000 7f6e4eb71700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e480ab000 con 0x7f6e480a7250 2022-08-01T19:43:36.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.429+0000 7f6e4eb71700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e480aa360 con 0x7f6e480b1f40 2022-08-01T19:43:36.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.429+0000 7f6e4eb71700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e480aa600 con 0x7f6e48148530 2022-08-01T19:43:36.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.429+0000 7f6e4c90d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e480a7250 0x7f6e4815a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:36.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.429+0000 7f6e4c90d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e480a7250 0x7f6e4815a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44976/0 (socket says 172.21.15.17:44976) 2022-08-01T19:43:36.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.429+0000 7f6e4c90d700 1 -- 172.21.15.17:0/1178277971 learned_addr learned my addr 172.21.15.17:0/1178277971 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:36.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.429+0000 7f6e47fff700 1 --2- 172.21.15.17:0/1178277971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e480b1f40 0x7f6e481429c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:36.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.429+0000 7f6e4d10e700 1 --2- 172.21.15.17:0/1178277971 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6e48148530 0x7f6e48142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:36.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.429+0000 7f6e47fff700 1 -- 172.21.15.17:0/1178277971 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6e48148530 msgr2=0x7f6e48142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:36.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.429+0000 7f6e47fff700 1 --2- 172.21.15.17:0/1178277971 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6e48148530 0x7f6e48142eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:36.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.429+0000 7f6e47fff700 1 -- 172.21.15.17:0/1178277971 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e480a7250 msgr2=0x7f6e4815a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:36.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.429+0000 7f6e47fff700 1 --2- 172.21.15.17:0/1178277971 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e480a7250 0x7f6e4815a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:36.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.429+0000 7f6e47fff700 1 -- 172.21.15.17:0/1178277971 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e3800b180 con 0x7f6e480b1f40 2022-08-01T19:43:36.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.429+0000 7f6e4c90d700 1 --2- 172.21.15.17:0/1178277971 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e480a7250 0x7f6e4815a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:43:36.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.429+0000 7f6e47fff700 1 --2- 172.21.15.17:0/1178277971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e480b1f40 0x7f6e481429c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f6e38002f70 tx=0x7f6e38014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:36.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.429+0000 7f6e45ffb700 1 -- 172.21.15.17:0/1178277971 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e38016070 con 0x7f6e480b1f40 2022-08-01T19:43:36.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.429+0000 7f6e45ffb700 1 -- 172.21.15.17:0/1178277971 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3800d8d0 con 0x7f6e480b1f40 2022-08-01T19:43:36.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.429+0000 7f6e4eb71700 1 -- 172.21.15.17:0/1178277971 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e481433a0 con 0x7f6e480b1f40 2022-08-01T19:43:36.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.429+0000 7f6e45ffb700 1 -- 172.21.15.17:0/1178277971 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e38013de0 con 0x7f6e480b1f40 2022-08-01T19:43:36.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.429+0000 7f6e4eb71700 1 -- 172.21.15.17:0/1178277971 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e4814b7c0 con 0x7f6e480b1f40 2022-08-01T19:43:36.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.429+0000 7f6e4eb71700 1 -- 172.21.15.17:0/1178277971 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e480574e0 con 0x7f6e480b1f40 2022-08-01T19:43:36.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.433+0000 7f6e45ffb700 1 -- 172.21.15.17:0/1178277971 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6e38025030 con 0x7f6e480b1f40 2022-08-01T19:43:36.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.433+0000 7f6e45ffb700 1 --2- 172.21.15.17:0/1178277971 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6e28053a70 0x7f6e28055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:36.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.433+0000 7f6e45ffb700 1 -- 172.21.15.17:0/1178277971 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6e3806cb70 con 0x7f6e480b1f40 2022-08-01T19:43:36.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.433+0000 7f6e4c90d700 1 --2- 172.21.15.17:0/1178277971 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6e28053a70 0x7f6e28055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:36.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.433+0000 7f6e4c90d700 1 --2- 172.21.15.17:0/1178277971 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6e28053a70 0x7f6e28055f30 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f6e3c00d000 tx=0x7f6e3c007c00).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:36.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.441+0000 7f6e45ffb700 1 -- 172.21.15.17:0/1178277971 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6e38043930 con 0x7f6e480b1f40 2022-08-01T19:43:36.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.609+0000 7f6e4eb71700 1 -- 172.21.15.17:0/1178277971 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e4814b7c0 con 0x7f6e480b1f40 2022-08-01T19:43:36.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.613+0000 7f6e45ffb700 1 -- 172.21.15.17:0/1178277971 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f6e4814b7c0 con 0x7f6e480b1f40 2022-08-01T19:43:36.620 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:36.620 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":76,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7211,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/872530952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":872530952},{"type":"v1","addr":"172.21.15.77:6835","nonce":872530952}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":74},{"gid":8358,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/4261332014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4261332014},{"type":"v1","addr":"172.21.15.17:6833","nonce":4261332014}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":73}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:43:34.586097+0000","modified":"2022-08-01T19:43:36.271865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8364},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8364":{"gid":8364,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/2832743740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2832743740},{"type":"v1","addr":"172.21.15.17:6835","nonce":2832743740}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2022-08-01T19:43:36.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.613+0000 7f6e4eb71700 1 -- 172.21.15.17:0/1178277971 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6e28053a70 msgr2=0x7f6e28055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:36.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.613+0000 7f6e4eb71700 1 --2- 172.21.15.17:0/1178277971 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6e28053a70 0x7f6e28055f30 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f6e3c00d000 tx=0x7f6e3c007c00).stop 2022-08-01T19:43:36.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.613+0000 7f6e4eb71700 1 -- 172.21.15.17:0/1178277971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e480b1f40 msgr2=0x7f6e481429c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:36.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.613+0000 7f6e4eb71700 1 --2- 172.21.15.17:0/1178277971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e480b1f40 0x7f6e481429c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f6e38002f70 tx=0x7f6e38014000).stop 2022-08-01T19:43:36.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.613+0000 7f6e4eb71700 1 -- 172.21.15.17:0/1178277971 shutdown_connections 2022-08-01T19:43:36.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.613+0000 7f6e4eb71700 1 --2- 172.21.15.17:0/1178277971 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6e28053a70 0x7f6e28055f30 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:36.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.613+0000 7f6e4eb71700 1 --2- 172.21.15.17:0/1178277971 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6e48148530 0x7f6e48142eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:36.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.613+0000 7f6e4eb71700 1 --2- 172.21.15.17:0/1178277971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e480b1f40 0x7f6e481429c0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:36.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.613+0000 7f6e4eb71700 1 --2- 172.21.15.17:0/1178277971 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e480a7250 0x7f6e4815a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:36.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.613+0000 7f6e4eb71700 1 -- 172.21.15.17:0/1178277971 >> 172.21.15.17:0/1178277971 conn(0x7f6e4805f840 msgr2=0x7f6e48143f70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:36.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.613+0000 7f6e4eb71700 1 -- 172.21.15.17:0/1178277971 shutdown_connections 2022-08-01T19:43:36.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.613+0000 7f6e4eb71700 1 -- 172.21.15.17:0/1178277971 wait complete. 2022-08-01T19:43:36.643 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 76 2022-08-01T19:43:36.644 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:43:36.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.749+0000 7f897759e700 1 Processor -- start 2022-08-01T19:43:36.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.749+0000 7f897759e700 1 -- start start 2022-08-01T19:43:36.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.749+0000 7f897759e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f89780b01a0 0x7f89780a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:36.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.749+0000 7f897759e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f89780afd60 0x7f89780a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:36.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.749+0000 7f897759e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89780a7250 con 0x7f89780b2710 2022-08-01T19:43:36.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.749+0000 7f897759e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89780a7480 con 0x7f89780b01a0 2022-08-01T19:43:36.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.749+0000 7f897759e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89780a7720 con 0x7f89780afd60 2022-08-01T19:43:36.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.749+0000 7f8975d9b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f89780b01a0 0x7f89780a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:36.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.749+0000 7f8976d9d700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f89780afd60 0x7f89780a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:36.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.749+0000 7f897659c700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7f89780b2710 0x7f897814afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:53978/0 (socket says 172.21.15.17:53978) 2022-08-01T19:43:36.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.749+0000 7f897659c700 1 -- 172.21.15.17:0/3451251192 learned_addr learned my addr 172.21.15.17:0/3451251192 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:36.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.749+0000 7f8976d9d700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f89780afd60 0x7f89780a8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:42910/0 (socket says 172.21.15.17:42910) 2022-08-01T19:43:36.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.753+0000 7f8976d9d700 1 -- 172.21.15.17:0/3451251192 >> v2:172.21.15.17:3300/0 conn(0x7f89780b01a0 msgr2=0x7f89780a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:36.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.753+0000 7f8976d9d700 1 --2- 172.21.15.17:0/3451251192 >> v2:172.21.15.17:3300/0 conn(0x7f89780b01a0 0x7f89780a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:36.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.753+0000 7f8976d9d700 1 -- 172.21.15.17:0/3451251192 >> v1:172.21.15.77:6789/0 conn(0x7f89780b2710 legacy=0x7f897814afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:36.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.753+0000 7f8976d9d700 1 -- 172.21.15.17:0/3451251192 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89780a79c0 con 0x7f89780afd60 2022-08-01T19:43:36.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.753+0000 7f8976d9d700 1 --2- 172.21.15.17:0/3451251192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89780afd60 0x7f89780a8a30 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f89700032f0 tx=0x7f89700091d0).ready entity=mon.1 client_cookie=5b468afa7b83c022 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:36.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.753+0000 7f897559a700 1 -- 172.21.15.17:0/3451251192 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f897000b680 con 0x7f89780afd60 2022-08-01T19:43:36.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.753+0000 7f897559a700 1 -- 172.21.15.17:0/3451251192 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897000b7e0 con 0x7f89780afd60 2022-08-01T19:43:36.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.753+0000 7f897559a700 1 -- 172.21.15.17:0/3451251192 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f897000b950 con 0x7f89780afd60 2022-08-01T19:43:36.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.753+0000 7f897759e700 1 -- 172.21.15.17:0/3451251192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89780afd60 msgr2=0x7f89780a8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:36.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.753+0000 7f897759e700 1 --2- 172.21.15.17:0/3451251192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89780afd60 0x7f89780a8a30 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f89700032f0 tx=0x7f89700091d0).stop 2022-08-01T19:43:36.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.753+0000 7f897759e700 1 -- 172.21.15.17:0/3451251192 shutdown_connections 2022-08-01T19:43:36.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.753+0000 7f897759e700 1 --2- 172.21.15.17:0/3451251192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89780afd60 0x7f89780a8a30 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:36.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.753+0000 7f897759e700 1 --2- 172.21.15.17:0/3451251192 >> v2:172.21.15.17:3300/0 conn(0x7f89780b01a0 0x7f89780a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:36.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.753+0000 7f897759e700 1 -- 172.21.15.17:0/3451251192 >> 172.21.15.17:0/3451251192 conn(0x7f8978060310 msgr2=0x7f89780b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:36.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.753+0000 7f897759e700 1 -- 172.21.15.17:0/3451251192 shutdown_connections 2022-08-01T19:43:36.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.753+0000 7f897759e700 1 -- 172.21.15.17:0/3451251192 wait complete. 2022-08-01T19:43:36.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.757+0000 7f897759e700 1 Processor -- start 2022-08-01T19:43:36.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.757+0000 7f897759e700 1 -- start start 2022-08-01T19:43:36.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.757+0000 7f897759e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89780afd60 0x7f897814acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:36.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.757+0000 7f897759e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f89780b01a0 0x7f8978159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:36.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.757+0000 7f897759e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89780b2710 0x7f897815a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:36.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.757+0000 7f897759e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89780aa3d0 con 0x7f89780afd60 2022-08-01T19:43:36.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.757+0000 7f897759e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8978157310 con 0x7f89780b2710 2022-08-01T19:43:36.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.757+0000 7f897759e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89780a9f30 con 0x7f89780b01a0 2022-08-01T19:43:36.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.757+0000 7f897659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89780afd60 0x7f897814acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:36.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.757+0000 7f897659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89780afd60 0x7f897814acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44990/0 (socket says 172.21.15.17:44990) 2022-08-01T19:43:36.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.757+0000 7f897659c700 1 -- 172.21.15.17:0/3951130721 learned_addr learned my addr 172.21.15.17:0/3951130721 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:36.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.757+0000 7f8976d9d700 1 --2- 172.21.15.17:0/3951130721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89780b2710 0x7f897815a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:36.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.757+0000 7f8975d9b700 1 --2- 172.21.15.17:0/3951130721 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f89780b01a0 0x7f8978159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:36.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.757+0000 7f897659c700 1 -- 172.21.15.17:0/3951130721 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f89780b01a0 msgr2=0x7f8978159ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:36.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.757+0000 7f897659c700 1 --2- 172.21.15.17:0/3951130721 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f89780b01a0 0x7f8978159ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:36.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.757+0000 7f897659c700 1 -- 172.21.15.17:0/3951130721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89780b2710 msgr2=0x7f897815a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:36.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.757+0000 7f897659c700 1 --2- 172.21.15.17:0/3951130721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89780b2710 0x7f897815a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:36.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.757+0000 7f897659c700 1 -- 172.21.15.17:0/3951130721 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8970009470 con 0x7f89780afd60 2022-08-01T19:43:36.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.757+0000 7f897659c700 1 --2- 172.21.15.17:0/3951130721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89780afd60 0x7f897814acb0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 rx=0x7f897806b950 tx=0x7f8968007ca0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:36.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.757+0000 7f89677fe700 1 -- 172.21.15.17:0/3951130721 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89680070d0 con 0x7f89780afd60 2022-08-01T19:43:36.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.761+0000 7f89677fe700 1 -- 172.21.15.17:0/3951130721 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8968007230 con 0x7f89780afd60 2022-08-01T19:43:36.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.761+0000 7f897759e700 1 -- 172.21.15.17:0/3951130721 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8978157030 con 0x7f89780afd60 2022-08-01T19:43:36.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.761+0000 7f89677fe700 1 -- 172.21.15.17:0/3951130721 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8968005610 con 0x7f89780afd60 2022-08-01T19:43:36.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.761+0000 7f897759e700 1 -- 172.21.15.17:0/3951130721 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89780a9340 con 0x7f89780afd60 2022-08-01T19:43:36.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.761+0000 7f897759e700 1 -- 172.21.15.17:0/3951130721 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8978157510 con 0x7f89780afd60 2022-08-01T19:43:36.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.761+0000 7f89677fe700 1 -- 172.21.15.17:0/3951130721 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f896801a060 con 0x7f89780afd60 2022-08-01T19:43:36.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.761+0000 7f89677fe700 1 --2- 172.21.15.17:0/3951130721 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8954053a70 0x7f8954055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:36.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.761+0000 7f89677fe700 1 -- 172.21.15.17:0/3951130721 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f896806b680 con 0x7f89780afd60 2022-08-01T19:43:36.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.761+0000 7f8975d9b700 1 --2- 172.21.15.17:0/3951130721 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8954053a70 0x7f8954055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:36.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.761+0000 7f8975d9b700 1 --2- 172.21.15.17:0/3951130721 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8954053a70 0x7f8954055f30 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f896c00b000 tx=0x7f896c006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:36.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.765+0000 7f89677fe700 1 -- 172.21.15.17:0/3951130721 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8968042440 con 0x7f89780afd60 2022-08-01T19:43:36.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.893+0000 7f897759e700 1 -- 172.21.15.17:0/3951130721 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89780574e0 con 0x7f89780afd60 2022-08-01T19:43:36.901 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.897+0000 7f89677fe700 1 -- 172.21.15.17:0/3951130721 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f8968018070 con 0x7f89780afd60 2022-08-01T19:43:36.903 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:36.904 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":148,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:43:35.589613+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:43:33.209247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2022-08-01T19:43:33.595201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:43:36.905 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.897+0000 7f897759e700 1 -- 172.21.15.17:0/3951130721 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8954053a70 msgr2=0x7f8954055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:36.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.897+0000 7f897759e700 1 --2- 172.21.15.17:0/3951130721 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8954053a70 0x7f8954055f30 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f896c00b000 tx=0x7f896c006df0).stop 2022-08-01T19:43:36.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.897+0000 7f897759e700 1 -- 172.21.15.17:0/3951130721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89780afd60 msgr2=0x7f897814acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:36.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.897+0000 7f897759e700 1 --2- 172.21.15.17:0/3951130721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89780afd60 0x7f897814acb0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 rx=0x7f897806b950 tx=0x7f8968007ca0).stop 2022-08-01T19:43:36.909 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.897+0000 7f897759e700 1 -- 172.21.15.17:0/3951130721 shutdown_connections 2022-08-01T19:43:36.909 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.897+0000 7f897759e700 1 --2- 172.21.15.17:0/3951130721 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f8954053a70 0x7f8954055f30 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:36.909 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.897+0000 7f897759e700 1 --2- 172.21.15.17:0/3951130721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89780b2710 0x7f897815a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:36.910 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.897+0000 7f897759e700 1 --2- 172.21.15.17:0/3951130721 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f89780b01a0 0x7f8978159ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:36.910 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.897+0000 7f897759e700 1 --2- 172.21.15.17:0/3951130721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89780afd60 0x7f897814acb0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:36.910 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.897+0000 7f897759e700 1 -- 172.21.15.17:0/3951130721 >> 172.21.15.17:0/3951130721 conn(0x7f8978060310 msgr2=0x7f897805c870 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:36.910 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.897+0000 7f897759e700 1 -- 172.21.15.17:0/3951130721 shutdown_connections 2022-08-01T19:43:36.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:36.897+0000 7f897759e700 1 -- 172.21.15.17:0/3951130721 wait complete. 2022-08-01T19:43:36.912 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T19:43:37.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.025+0000 7ff67ffff700 1 Processor -- start 2022-08-01T19:43:37.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.025+0000 7ff67ffff700 1 -- start start 2022-08-01T19:43:37.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.025+0000 7ff67ffff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff6800ad750 0x7ff680061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:37.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.025+0000 7ff67ffff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ff6801405d0 0x7ff680140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:37.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.025+0000 7ff67ffff700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6800aee80 con 0x7ff6800614b0 2022-08-01T19:43:37.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.025+0000 7ff67ffff700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6800b70b0 con 0x7ff6800ad750 2022-08-01T19:43:37.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.025+0000 7ff67ffff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6800af0b0 con 0x7ff6801405d0 2022-08-01T19:43:37.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.025+0000 7ff67e7fc700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff6800ad750 0x7ff680061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:37.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.029+0000 7ff67effd700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff6800614b0 0x7ff6800b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37192/0 (socket says 172.21.15.17:37192) 2022-08-01T19:43:37.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.029+0000 7ff67effd700 1 -- 172.21.15.17:0/3812153057 learned_addr learned my addr 172.21.15.17:0/3812153057 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:37.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.029+0000 7ff67f7fe700 1 --2- 172.21.15.17:0/3812153057 >> v2:172.21.15.77:3300/0 conn(0x7ff6801405d0 0x7ff680140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:37.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.029+0000 7ff67e7fc700 1 -- 172.21.15.17:0/3812153057 >> v2:172.21.15.77:3300/0 conn(0x7ff6801405d0 msgr2=0x7ff680140a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:37.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.029+0000 7ff67e7fc700 1 --2- 172.21.15.17:0/3812153057 >> v2:172.21.15.77:3300/0 conn(0x7ff6801405d0 0x7ff680140a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:37.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.029+0000 7ff67e7fc700 1 -- 172.21.15.17:0/3812153057 >> v1:172.21.15.17:6789/0 conn(0x7ff6800614b0 legacy=0x7ff6800b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:37.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.029+0000 7ff67dffb700 1 -- 172.21.15.17:0/3812153057 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3366897016 0 0) 0x7ff6800aee80 con 0x7ff6800614b0 2022-08-01T19:43:37.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.029+0000 7ff67e7fc700 1 -- 172.21.15.17:0/3812153057 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6800af350 con 0x7ff6800ad750 2022-08-01T19:43:37.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.029+0000 7ff67e7fc700 1 --2- 172.21.15.17:0/3812153057 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6800ad750 0x7ff680061890 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 rx=0x7ff670002f70 tx=0x7ff670006df0).ready entity=mon.0 client_cookie=623761da3b168401 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:37.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.029+0000 7ff67dffb700 1 -- 172.21.15.17:0/3812153057 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff67000d570 con 0x7ff6800ad750 2022-08-01T19:43:37.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.029+0000 7ff67dffb700 1 -- 172.21.15.17:0/3812153057 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff670000e90 con 0x7ff6800ad750 2022-08-01T19:43:37.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.029+0000 7ff67dffb700 1 -- 172.21.15.17:0/3812153057 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff67000d8c0 con 0x7ff6800ad750 2022-08-01T19:43:37.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.029+0000 7ff67ffff700 1 -- 172.21.15.17:0/3812153057 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6800ad750 msgr2=0x7ff680061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:37.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.029+0000 7ff67ffff700 1 --2- 172.21.15.17:0/3812153057 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6800ad750 0x7ff680061890 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 rx=0x7ff670002f70 tx=0x7ff670006df0).stop 2022-08-01T19:43:37.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.029+0000 7ff67ffff700 1 -- 172.21.15.17:0/3812153057 shutdown_connections 2022-08-01T19:43:37.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.029+0000 7ff67ffff700 1 --2- 172.21.15.17:0/3812153057 >> v2:172.21.15.77:3300/0 conn(0x7ff6801405d0 0x7ff680140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:37.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.029+0000 7ff67ffff700 1 --2- 172.21.15.17:0/3812153057 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6800ad750 0x7ff680061890 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:37.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.029+0000 7ff67ffff700 1 -- 172.21.15.17:0/3812153057 >> 172.21.15.17:0/3812153057 conn(0x7ff680066f70 msgr2=0x7ff680067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:37.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.029+0000 7ff67ffff700 1 -- 172.21.15.17:0/3812153057 shutdown_connections 2022-08-01T19:43:37.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.029+0000 7ff67ffff700 1 -- 172.21.15.17:0/3812153057 wait complete. 2022-08-01T19:43:37.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.029+0000 7ff67ffff700 1 Processor -- start 2022-08-01T19:43:37.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.029+0000 7ff67ffff700 1 -- start start 2022-08-01T19:43:37.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.033+0000 7ff67ffff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6800614b0 0x7ff680142570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:37.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.033+0000 7ff67ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6800ad750 0x7ff680152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:37.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.033+0000 7ff67ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6801405d0 0x7ff680153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:37.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.033+0000 7ff67ffff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6800b1260 con 0x7ff6801405d0 2022-08-01T19:43:37.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.033+0000 7ff67ffff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6800b1500 con 0x7ff6800ad750 2022-08-01T19:43:37.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.033+0000 7ff67ffff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6801526d0 con 0x7ff6800614b0 2022-08-01T19:43:37.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.033+0000 7ff67f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6801405d0 0x7ff680153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:37.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.033+0000 7ff67f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6801405d0 0x7ff680153060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45008/0 (socket says 172.21.15.17:45008) 2022-08-01T19:43:37.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.033+0000 7ff67f7fe700 1 -- 172.21.15.17:0/4013313187 learned_addr learned my addr 172.21.15.17:0/4013313187 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:37.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.033+0000 7ff67e7fc700 1 --2- 172.21.15.17:0/4013313187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6800ad750 0x7ff680152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:37.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.033+0000 7ff67effd700 1 --2- 172.21.15.17:0/4013313187 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6800614b0 0x7ff680142570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:37.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.033+0000 7ff67f7fe700 1 -- 172.21.15.17:0/4013313187 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6800614b0 msgr2=0x7ff680142570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:37.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.033+0000 7ff67f7fe700 1 --2- 172.21.15.17:0/4013313187 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6800614b0 0x7ff680142570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:37.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.033+0000 7ff67f7fe700 1 -- 172.21.15.17:0/4013313187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6800ad750 msgr2=0x7ff680152b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:37.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.033+0000 7ff67f7fe700 1 --2- 172.21.15.17:0/4013313187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6800ad750 0x7ff680152b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:37.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.033+0000 7ff67f7fe700 1 -- 172.21.15.17:0/4013313187 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67000b180 con 0x7ff6801405d0 2022-08-01T19:43:37.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.033+0000 7ff67effd700 1 --2- 172.21.15.17:0/4013313187 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6800614b0 0x7ff680142570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:43:37.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.033+0000 7ff67f7fe700 1 --2- 172.21.15.17:0/4013313187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6801405d0 0x7ff680153060 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 rx=0x7ff678006e00 tx=0x7ff6780091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:37.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.033+0000 7ff66ffff700 1 -- 172.21.15.17:0/4013313187 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff67800c7f0 con 0x7ff6801405d0 2022-08-01T19:43:37.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.033+0000 7ff66ffff700 1 -- 172.21.15.17:0/4013313187 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67800c950 con 0x7ff6801405d0 2022-08-01T19:43:37.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.033+0000 7ff67ffff700 1 -- 172.21.15.17:0/4013313187 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff680143230 con 0x7ff6801405d0 2022-08-01T19:43:37.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.033+0000 7ff66ffff700 1 -- 172.21.15.17:0/4013313187 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff67800ccb0 con 0x7ff6801405d0 2022-08-01T19:43:37.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.033+0000 7ff67ffff700 1 -- 172.21.15.17:0/4013313187 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6800b0d50 con 0x7ff6801405d0 2022-08-01T19:43:37.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.037+0000 7ff67ffff700 1 -- 172.21.15.17:0/4013313187 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff680152890 con 0x7ff6801405d0 2022-08-01T19:43:37.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.037+0000 7ff66ffff700 1 -- 172.21.15.17:0/4013313187 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff6780123f0 con 0x7ff6801405d0 2022-08-01T19:43:37.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.037+0000 7ff66ffff700 1 --2- 172.21.15.17:0/4013313187 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff65c053950 0x7ff65c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:37.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.037+0000 7ff66ffff700 1 -- 172.21.15.17:0/4013313187 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff67806c460 con 0x7ff6801405d0 2022-08-01T19:43:37.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.037+0000 7ff67effd700 1 --2- 172.21.15.17:0/4013313187 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff65c053950 0x7ff65c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:37.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.037+0000 7ff67effd700 1 --2- 172.21.15.17:0/4013313187 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff65c053950 0x7ff65c055e10 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7ff674008000 tx=0x7ff674009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:37.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.045+0000 7ff66ffff700 1 -- 172.21.15.17:0/4013313187 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff6780430b0 con 0x7ff6801405d0 2022-08-01T19:43:37.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.209+0000 7ff67ffff700 1 -- 172.21.15.17:0/4013313187 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7ff6800af5f0 con 0x7ff6801405d0 2022-08-01T19:43:37.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.217+0000 7ff66ffff700 1 -- 172.21.15.17:0/4013313187 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v36) v1 ==== 184+0+0 (secure 0 0 0) 0x7ff6780466d0 con 0x7ff6801405d0 2022-08-01T19:43:37.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.221+0000 7ff67ffff700 1 -- 172.21.15.17:0/4013313187 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff65c053950 msgr2=0x7ff65c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:37.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.221+0000 7ff67ffff700 1 --2- 172.21.15.17:0/4013313187 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff65c053950 0x7ff65c055e10 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7ff674008000 tx=0x7ff674009040).stop 2022-08-01T19:43:37.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.221+0000 7ff67ffff700 1 -- 172.21.15.17:0/4013313187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6801405d0 msgr2=0x7ff680153060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:37.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.221+0000 7ff67ffff700 1 --2- 172.21.15.17:0/4013313187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6801405d0 0x7ff680153060 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 rx=0x7ff678006e00 tx=0x7ff6780091d0).stop 2022-08-01T19:43:37.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.221+0000 7ff67ffff700 1 -- 172.21.15.17:0/4013313187 shutdown_connections 2022-08-01T19:43:37.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.221+0000 7ff67ffff700 1 --2- 172.21.15.17:0/4013313187 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff65c053950 0x7ff65c055e10 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:37.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.221+0000 7ff67ffff700 1 --2- 172.21.15.17:0/4013313187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6801405d0 0x7ff680153060 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:37.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.221+0000 7ff67ffff700 1 --2- 172.21.15.17:0/4013313187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6800ad750 0x7ff680152b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:37.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.221+0000 7ff67ffff700 1 --2- 172.21.15.17:0/4013313187 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6800614b0 0x7ff680142570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:37.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.221+0000 7ff67ffff700 1 -- 172.21.15.17:0/4013313187 >> 172.21.15.17:0/4013313187 conn(0x7ff680066f70 msgr2=0x7ff680145530 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:37.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.221+0000 7ff67ffff700 1 -- 172.21.15.17:0/4013313187 shutdown_connections 2022-08-01T19:43:37.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.221+0000 7ff67ffff700 1 -- 172.21.15.17:0/4013313187 wait complete. 2022-08-01T19:43:37.232 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.0 2022-08-01T19:43:37.243 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T19:43:37.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.361+0000 7fe18ffff700 1 Processor -- start 2022-08-01T19:43:37.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.361+0000 7fe18ffff700 1 -- start start 2022-08-01T19:43:37.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.361+0000 7fe18ffff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fe1900b6420 0x7fe190061600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:37.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.361+0000 7fe18ffff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe1900b41f0 0x7fe190061af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:37.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.361+0000 7fe18ffff700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1900b60e0 con 0x7fe190061fe0 2022-08-01T19:43:37.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.361+0000 7fe18ffff700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1900b2100 con 0x7fe1900b41f0 2022-08-01T19:43:37.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.361+0000 7fe18ffff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1900b23a0 con 0x7fe1900b6420 2022-08-01T19:43:37.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.361+0000 7fe18e7fc700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe1900b41f0 0x7fe190061af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:37.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.361+0000 7fe18e7fc700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe1900b41f0 0x7fe190061af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45014/0 (socket says 172.21.15.17:45014) 2022-08-01T19:43:37.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.361+0000 7fe18e7fc700 1 -- 172.21.15.17:0/2064366409 learned_addr learned my addr 172.21.15.17:0/2064366409 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:37.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.361+0000 7fe18effd700 1 --2- 172.21.15.17:0/2064366409 >> v2:172.21.15.77:3300/0 conn(0x7fe1900b6420 0x7fe190061600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:37.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.361+0000 7fe18e7fc700 1 -- 172.21.15.17:0/2064366409 >> v2:172.21.15.77:3300/0 conn(0x7fe1900b6420 msgr2=0x7fe190061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:37.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.361+0000 7fe18e7fc700 1 --2- 172.21.15.17:0/2064366409 >> v2:172.21.15.77:3300/0 conn(0x7fe1900b6420 0x7fe190061600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:37.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.361+0000 7fe18e7fc700 1 -- 172.21.15.17:0/2064366409 >> v1:172.21.15.77:6789/0 conn(0x7fe190061fe0 legacy=0x7fe1901465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:37.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.361+0000 7fe18e7fc700 1 -- 172.21.15.17:0/2064366409 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1900b2640 con 0x7fe1900b41f0 2022-08-01T19:43:37.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.361+0000 7fe18e7fc700 1 --2- 172.21.15.17:0/2064366409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1900b41f0 0x7fe190061af0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 rx=0x7fe184002f70 tx=0x7fe184006eb0).ready entity=mon.0 client_cookie=b6187a5ea9b4c470 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:37.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.361+0000 7fe18dffb700 1 -- 172.21.15.17:0/2064366409 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe18400d5c0 con 0x7fe1900b41f0 2022-08-01T19:43:37.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.365+0000 7fe18dffb700 1 -- 172.21.15.17:0/2064366409 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe184000e90 con 0x7fe1900b41f0 2022-08-01T19:43:37.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.365+0000 7fe18dffb700 1 -- 172.21.15.17:0/2064366409 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe18400d950 con 0x7fe1900b41f0 2022-08-01T19:43:37.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.365+0000 7fe18ffff700 1 -- 172.21.15.17:0/2064366409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1900b41f0 msgr2=0x7fe190061af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:37.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.365+0000 7fe18ffff700 1 --2- 172.21.15.17:0/2064366409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1900b41f0 0x7fe190061af0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 rx=0x7fe184002f70 tx=0x7fe184006eb0).stop 2022-08-01T19:43:37.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.365+0000 7fe18ffff700 1 -- 172.21.15.17:0/2064366409 shutdown_connections 2022-08-01T19:43:37.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.365+0000 7fe18ffff700 1 --2- 172.21.15.17:0/2064366409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1900b41f0 0x7fe190061af0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:37.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.365+0000 7fe18ffff700 1 --2- 172.21.15.17:0/2064366409 >> v2:172.21.15.77:3300/0 conn(0x7fe1900b6420 0x7fe190061600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:37.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.365+0000 7fe18ffff700 1 -- 172.21.15.17:0/2064366409 >> 172.21.15.17:0/2064366409 conn(0x7fe19005c620 msgr2=0x7fe19005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:37.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.365+0000 7fe18ffff700 1 -- 172.21.15.17:0/2064366409 shutdown_connections 2022-08-01T19:43:37.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.365+0000 7fe18ffff700 1 -- 172.21.15.17:0/2064366409 wait complete. 2022-08-01T19:43:37.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.365+0000 7fe18ffff700 1 Processor -- start 2022-08-01T19:43:37.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.365+0000 7fe18ffff700 1 -- start start 2022-08-01T19:43:37.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.369+0000 7fe18ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe190061fe0 0x7fe190141950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:37.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.369+0000 7fe18ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1900b41f0 0x7fe1901426f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:37.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.369+0000 7fe18ffff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1900b6420 0x7fe19014f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:37.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.369+0000 7fe18ffff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1901432f0 con 0x7fe1900b41f0 2022-08-01T19:43:37.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.369+0000 7fe18ffff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe190143590 con 0x7fe190061fe0 2022-08-01T19:43:37.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.369+0000 7fe18ffff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe190143830 con 0x7fe1900b6420 2022-08-01T19:43:37.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.369+0000 7fe18e7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1900b41f0 0x7fe1901426f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:37.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.369+0000 7fe18e7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1900b41f0 0x7fe1901426f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45020/0 (socket says 172.21.15.17:45020) 2022-08-01T19:43:37.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.369+0000 7fe18e7fc700 1 -- 172.21.15.17:0/96627142 learned_addr learned my addr 172.21.15.17:0/96627142 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:37.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.369+0000 7fe18effd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe190061fe0 0x7fe190141950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:37.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.369+0000 7fe18f7fe700 1 --2- 172.21.15.17:0/96627142 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1900b6420 0x7fe19014f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:37.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.369+0000 7fe18e7fc700 1 -- 172.21.15.17:0/96627142 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1900b6420 msgr2=0x7fe19014f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:37.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.369+0000 7fe18e7fc700 1 --2- 172.21.15.17:0/96627142 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1900b6420 0x7fe19014f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:37.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.369+0000 7fe18e7fc700 1 -- 172.21.15.17:0/96627142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe190061fe0 msgr2=0x7fe190141950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:37.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.369+0000 7fe18e7fc700 1 --2- 172.21.15.17:0/96627142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe190061fe0 0x7fe190141950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:37.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.369+0000 7fe18e7fc700 1 -- 172.21.15.17:0/96627142 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe18400b1e0 con 0x7fe1900b41f0 2022-08-01T19:43:37.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.369+0000 7fe18e7fc700 1 --2- 172.21.15.17:0/96627142 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1900b41f0 0x7fe1901426f0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 rx=0x7fe184002f70 tx=0x7fe184015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:37.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.369+0000 7fe17ffff700 1 -- 172.21.15.17:0/96627142 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe184011760 con 0x7fe1900b41f0 2022-08-01T19:43:37.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.369+0000 7fe17ffff700 1 -- 172.21.15.17:0/96627142 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1840118c0 con 0x7fe1900b41f0 2022-08-01T19:43:37.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.369+0000 7fe18ffff700 1 -- 172.21.15.17:0/96627142 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe190141e40 con 0x7fe1900b41f0 2022-08-01T19:43:37.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.369+0000 7fe17ffff700 1 -- 172.21.15.17:0/96627142 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe184011c20 con 0x7fe1900b41f0 2022-08-01T19:43:37.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.369+0000 7fe18ffff700 1 -- 172.21.15.17:0/96627142 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe190143e70 con 0x7fe1900b41f0 2022-08-01T19:43:37.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.373+0000 7fe17ffff700 1 -- 172.21.15.17:0/96627142 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe184025030 con 0x7fe1900b41f0 2022-08-01T19:43:37.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.373+0000 7fe18ffff700 1 -- 172.21.15.17:0/96627142 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1901439f0 con 0x7fe1900b41f0 2022-08-01T19:43:37.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.373+0000 7fe17ffff700 1 --2- 172.21.15.17:0/96627142 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe16c053a70 0x7fe16c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:37.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.373+0000 7fe17ffff700 1 -- 172.21.15.17:0/96627142 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe18406d110 con 0x7fe1900b41f0 2022-08-01T19:43:37.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.373+0000 7fe18effd700 1 --2- 172.21.15.17:0/96627142 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe16c053a70 0x7fe16c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:37.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.373+0000 7fe18effd700 1 --2- 172.21.15.17:0/96627142 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe16c053a70 0x7fe16c055f30 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7fe18000b000 tx=0x7fe180006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:37.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.377+0000 7fe17ffff700 1 -- 172.21.15.17:0/96627142 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe184043ed0 con 0x7fe1900b41f0 2022-08-01T19:43:37.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.545+0000 7fe18ffff700 1 -- 172.21.15.17:0/96627142 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fe1900b28e0 con 0x7fe1900b41f0 2022-08-01T19:43:37.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.549+0000 7fe17ffff700 1 -- 172.21.15.17:0/96627142 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v37) v1 ==== 184+0+0 (secure 0 0 0) 0x7fe1840230e0 con 0x7fe1900b41f0 2022-08-01T19:43:37.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.553+0000 7fe18ffff700 1 -- 172.21.15.17:0/96627142 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe16c053a70 msgr2=0x7fe16c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:37.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.553+0000 7fe18ffff700 1 --2- 172.21.15.17:0/96627142 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe16c053a70 0x7fe16c055f30 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7fe18000b000 tx=0x7fe180006df0).stop 2022-08-01T19:43:37.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.553+0000 7fe18ffff700 1 -- 172.21.15.17:0/96627142 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1900b41f0 msgr2=0x7fe1901426f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:37.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.553+0000 7fe18ffff700 1 --2- 172.21.15.17:0/96627142 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1900b41f0 0x7fe1901426f0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 rx=0x7fe184002f70 tx=0x7fe184015000).stop 2022-08-01T19:43:37.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.553+0000 7fe18ffff700 1 -- 172.21.15.17:0/96627142 shutdown_connections 2022-08-01T19:43:37.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.553+0000 7fe18ffff700 1 --2- 172.21.15.17:0/96627142 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe16c053a70 0x7fe16c055f30 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:37.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.553+0000 7fe18ffff700 1 --2- 172.21.15.17:0/96627142 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe1900b6420 0x7fe19014f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:37.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.553+0000 7fe18ffff700 1 --2- 172.21.15.17:0/96627142 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1900b41f0 0x7fe1901426f0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:37.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.553+0000 7fe18ffff700 1 --2- 172.21.15.17:0/96627142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe190061fe0 0x7fe190141950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:37.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.553+0000 7fe18ffff700 1 -- 172.21.15.17:0/96627142 >> 172.21.15.17:0/96627142 conn(0x7fe19005c620 msgr2=0x7fe190146b30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:37.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.553+0000 7fe18ffff700 1 -- 172.21.15.17:0/96627142 shutdown_connections 2022-08-01T19:43:37.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.553+0000 7fe18ffff700 1 -- 172.21.15.17:0/96627142 wait complete. 2022-08-01T19:43:37.562 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.1 2022-08-01T19:43:37.567 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:43:37.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.673+0000 7efe9a8ee700 1 Processor -- start 2022-08-01T19:43:37.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.673+0000 7efe9a8ee700 1 -- start start 2022-08-01T19:43:37.681 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.673+0000 7efe9a8ee700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efe9405c960 0x7efe940b1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:37.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.677+0000 7efe9a8ee700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe9405cd40 con 0x7efe9405d240 2022-08-01T19:43:37.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.677+0000 7efe9a8ee700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe940b1920 con 0x7efe9413f520 2022-08-01T19:43:37.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.677+0000 7efe9a8ee700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe940ad6d0 con 0x7efe9405c960 2022-08-01T19:43:37.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.677+0000 7efe990eb700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7efe9405d240 0x7efe940acfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37220/0 (socket says 172.21.15.17:37220) 2022-08-01T19:43:37.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.677+0000 7efe990eb700 1 -- 172.21.15.17:0/1587154629 learned_addr learned my addr 172.21.15.17:0/1587154629 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:37.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.677+0000 7efe998ec700 1 --2- 172.21.15.17:0/1587154629 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efe9405c960 0x7efe940b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:37.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.677+0000 7efe988ea700 1 -- 172.21.15.17:0/1587154629 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1647912446 0 0) 0x7efe9405cd40 con 0x7efe9405d240 2022-08-01T19:43:37.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.677+0000 7efe988ea700 1 -- 172.21.15.17:0/1587154629 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe7c004370 con 0x7efe9405d240 2022-08-01T19:43:37.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.677+0000 7efe998ec700 1 -- 172.21.15.17:0/1587154629 >> v1:172.21.15.77:6789/0 conn(0x7efe9413f520 legacy=0x7efe940aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:37.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.677+0000 7efe988ea700 1 -- 172.21.15.17:0/1587154629 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 181165217 0 0) 0x7efe940b1920 con 0x7efe9413f520 2022-08-01T19:43:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.677+0000 7efe998ec700 1 -- 172.21.15.17:0/1587154629 >> v1:172.21.15.17:6789/0 conn(0x7efe9405d240 legacy=0x7efe940acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.677+0000 7efe998ec700 1 -- 172.21.15.17:0/1587154629 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe940b1bc0 con 0x7efe9405c960 2022-08-01T19:43:37.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.677+0000 7efe998ec700 1 --2- 172.21.15.17:0/1587154629 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efe9405c960 0x7efe940b1280 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7efe90002f70 tx=0x7efe90006df0).ready entity=mon.2 client_cookie=a26758f854695376 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:37.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.677+0000 7efe988ea700 1 -- 172.21.15.17:0/1587154629 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe9000d570 con 0x7efe9405c960 2022-08-01T19:43:37.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.677+0000 7efe988ea700 1 -- 172.21.15.17:0/1587154629 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe90000e90 con 0x7efe9405c960 2022-08-01T19:43:37.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.677+0000 7efe988ea700 1 -- 172.21.15.17:0/1587154629 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe9000d8c0 con 0x7efe9405c960 2022-08-01T19:43:37.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.677+0000 7efe9a8ee700 1 -- 172.21.15.17:0/1587154629 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efe9405c960 msgr2=0x7efe940b1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:37.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.677+0000 7efe9a8ee700 1 --2- 172.21.15.17:0/1587154629 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efe9405c960 0x7efe940b1280 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7efe90002f70 tx=0x7efe90006df0).stop 2022-08-01T19:43:37.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.677+0000 7efe9a8ee700 1 -- 172.21.15.17:0/1587154629 shutdown_connections 2022-08-01T19:43:37.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.677+0000 7efe9a8ee700 1 --2- 172.21.15.17:0/1587154629 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efe9405c960 0x7efe940b1280 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:37.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.677+0000 7efe9a8ee700 1 -- 172.21.15.17:0/1587154629 >> 172.21.15.17:0/1587154629 conn(0x7efe94060310 msgr2=0x7efe940b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:37.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.677+0000 7efe9a8ee700 1 -- 172.21.15.17:0/1587154629 shutdown_connections 2022-08-01T19:43:37.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.677+0000 7efe9a8ee700 1 -- 172.21.15.17:0/1587154629 wait complete. 2022-08-01T19:43:37.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.677+0000 7efe9a8ee700 1 Processor -- start 2022-08-01T19:43:37.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.677+0000 7efe9a8ee700 1 -- start start 2022-08-01T19:43:37.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.681+0000 7efe9a8ee700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe9405c960 0x7efe940ab5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:37.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.681+0000 7efe9a8ee700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe9405d240 0x7efe940a9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:37.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.681+0000 7efe9a8ee700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efe9413f520 0x7efe940aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:37.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.681+0000 7efe9a8ee700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe940aa920 con 0x7efe9405d240 2022-08-01T19:43:37.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.681+0000 7efe9a8ee700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe940aabc0 con 0x7efe9405c960 2022-08-01T19:43:37.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.681+0000 7efe9a8ee700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe940aae60 con 0x7efe9413f520 2022-08-01T19:43:37.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.681+0000 7efe990eb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe9405d240 0x7efe940a9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:37.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.681+0000 7efe990eb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe9405d240 0x7efe940a9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45034/0 (socket says 172.21.15.17:45034) 2022-08-01T19:43:37.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.681+0000 7efe998ec700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe9405c960 0x7efe940ab5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:37.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.681+0000 7efe990eb700 1 -- 172.21.15.17:0/1185555845 learned_addr learned my addr 172.21.15.17:0/1185555845 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:37.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.681+0000 7efe9a0ed700 1 --2- 172.21.15.17:0/1185555845 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efe9413f520 0x7efe940aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:37.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.681+0000 7efe990eb700 1 -- 172.21.15.17:0/1185555845 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efe9413f520 msgr2=0x7efe940aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:37.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.681+0000 7efe990eb700 1 --2- 172.21.15.17:0/1185555845 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efe9413f520 0x7efe940aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:37.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.681+0000 7efe990eb700 1 -- 172.21.15.17:0/1185555845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe9405c960 msgr2=0x7efe940ab5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:37.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.681+0000 7efe990eb700 1 --2- 172.21.15.17:0/1185555845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe9405c960 0x7efe940ab5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:37.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.681+0000 7efe990eb700 1 -- 172.21.15.17:0/1185555845 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe9000b180 con 0x7efe9405d240 2022-08-01T19:43:37.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.681+0000 7efe990eb700 1 --2- 172.21.15.17:0/1185555845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe9405d240 0x7efe940a9610 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 rx=0x7efe84001110 tx=0x7efe84007a80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:37.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.681+0000 7efe82ffd700 1 -- 172.21.15.17:0/1185555845 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe84007110 con 0x7efe9405d240 2022-08-01T19:43:37.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.681+0000 7efe82ffd700 1 -- 172.21.15.17:0/1185555845 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe84007270 con 0x7efe9405d240 2022-08-01T19:43:37.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.681+0000 7efe9a8ee700 1 -- 172.21.15.17:0/1185555845 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe940ab100 con 0x7efe9405d240 2022-08-01T19:43:37.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.681+0000 7efe82ffd700 1 -- 172.21.15.17:0/1185555845 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe84005030 con 0x7efe9405d240 2022-08-01T19:43:37.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.681+0000 7efe9a8ee700 1 -- 172.21.15.17:0/1185555845 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe940aa4f0 con 0x7efe9405d240 2022-08-01T19:43:37.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.685+0000 7efe9a8ee700 1 -- 172.21.15.17:0/1185555845 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe940ab290 con 0x7efe9405d240 2022-08-01T19:43:37.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.685+0000 7efe82ffd700 1 -- 172.21.15.17:0/1185555845 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7efe8401b070 con 0x7efe9405d240 2022-08-01T19:43:37.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.685+0000 7efe82ffd700 1 --2- 172.21.15.17:0/1185555845 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efe74053a70 0x7efe74055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:37.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.685+0000 7efe82ffd700 1 -- 172.21.15.17:0/1185555845 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7efe8406b540 con 0x7efe9405d240 2022-08-01T19:43:37.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.685+0000 7efe998ec700 1 --2- 172.21.15.17:0/1185555845 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efe74053a70 0x7efe74055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:37.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.685+0000 7efe998ec700 1 --2- 172.21.15.17:0/1185555845 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efe74053a70 0x7efe74055f30 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7efe90018000 tx=0x7efe90014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:37.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.693+0000 7efe82ffd700 1 -- 172.21.15.17:0/1185555845 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efe84042300 con 0x7efe9405d240 2022-08-01T19:43:37.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.861+0000 7efe9a8ee700 1 -- 172.21.15.17:0/1185555845 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe940574e0 con 0x7efe9405d240 2022-08-01T19:43:37.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.861+0000 7efe82ffd700 1 -- 172.21.15.17:0/1185555845 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+2492 (secure 0 0 0) 0x7efe84045920 con 0x7efe9405d240 2022-08-01T19:43:37.864 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:37.865 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":76,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7211,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/872530952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":872530952},{"type":"v1","addr":"172.21.15.77:6835","nonce":872530952}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":74},{"gid":8358,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/4261332014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4261332014},{"type":"v1","addr":"172.21.15.17:6833","nonce":4261332014}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":73}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:43:34.586097+0000","modified":"2022-08-01T19:43:36.271865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8364},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8364":{"gid":8364,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/2832743740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2832743740},{"type":"v1","addr":"172.21.15.17:6835","nonce":2832743740}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2022-08-01T19:43:37.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.861+0000 7efe9a8ee700 1 -- 172.21.15.17:0/1185555845 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efe74053a70 msgr2=0x7efe74055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:37.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.861+0000 7efe9a8ee700 1 --2- 172.21.15.17:0/1185555845 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efe74053a70 0x7efe74055f30 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7efe90018000 tx=0x7efe90014000).stop 2022-08-01T19:43:37.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.861+0000 7efe9a8ee700 1 -- 172.21.15.17:0/1185555845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe9405d240 msgr2=0x7efe940a9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:37.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.861+0000 7efe9a8ee700 1 --2- 172.21.15.17:0/1185555845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe9405d240 0x7efe940a9610 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 rx=0x7efe84001110 tx=0x7efe84007a80).stop 2022-08-01T19:43:37.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.861+0000 7efe9a8ee700 1 -- 172.21.15.17:0/1185555845 shutdown_connections 2022-08-01T19:43:37.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.861+0000 7efe9a8ee700 1 --2- 172.21.15.17:0/1185555845 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efe74053a70 0x7efe74055f30 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:37.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.861+0000 7efe9a8ee700 1 --2- 172.21.15.17:0/1185555845 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efe9413f520 0x7efe940aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:37.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.861+0000 7efe9a8ee700 1 --2- 172.21.15.17:0/1185555845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe9405d240 0x7efe940a9610 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:37.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.861+0000 7efe9a8ee700 1 --2- 172.21.15.17:0/1185555845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe9405c960 0x7efe940ab5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:37.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.861+0000 7efe9a8ee700 1 -- 172.21.15.17:0/1185555845 >> 172.21.15.17:0/1185555845 conn(0x7efe94060310 msgr2=0x7efe9413e760 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:37.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.861+0000 7efe9a8ee700 1 -- 172.21.15.17:0/1185555845 shutdown_connections 2022-08-01T19:43:37.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:37.861+0000 7efe9a8ee700 1 -- 172.21.15.17:0/1185555845 wait complete. 2022-08-01T19:43:37.870 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 76 2022-08-01T19:43:38.181 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 76, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:43:34.586097+0000', 'modified': '2022-08-01T19:43:36.271865+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8364}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8364': {'gid': 8364, 'name': 'c', 'rank': 0, 'incarnation': 74, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/2832743740', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 2832743740}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 2832743740}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:43:38.182 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:43:38.183 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T19:43:38.304 INFO:teuthology.orchestra.run.smithi017.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":8364,"want_state":"up:active","state":"up:active","rank_uptime":3.7012403680000001,"mdsmap_epoch":76,"osdmap_epoch":148,"osdmap_epoch_barrier":148,"uptime":7.9380952999999996} 2022-08-01T19:43:38.306 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 8364, "mdsmap_epoch": 76, "osdmap_epoch": 148, "osdmap_epoch_barrier": 148, "rank_uptime": 3.701240368, "state": "up:active", "uptime": 7.9380953, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:43:38.307 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:43:38.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.409+0000 7f798d9f0700 1 Processor -- start 2022-08-01T19:43:38.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.409+0000 7f798d9f0700 1 -- start start 2022-08-01T19:43:38.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.409+0000 7f798d9f0700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f79880b01a0 0x7f79880a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:38.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.409+0000 7f798d9f0700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f79880afd60 0x7f79880a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:38.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.409+0000 7f798d9f0700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79880a7250 con 0x7f79880b2710 2022-08-01T19:43:38.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.409+0000 7f798d9f0700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79880a7480 con 0x7f79880afd60 2022-08-01T19:43:38.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.409+0000 7f798d9f0700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79880a7720 con 0x7f79880b01a0 2022-08-01T19:43:38.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.409+0000 7f798d1ef700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f79880afd60 0x7f79880a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:38.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.409+0000 7f798d1ef700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f79880afd60 0x7f79880a8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45042/0 (socket says 172.21.15.17:45042) 2022-08-01T19:43:38.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.413+0000 7f798d1ef700 1 -- 172.21.15.17:0/2515004191 learned_addr learned my addr 172.21.15.17:0/2515004191 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:38.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.409+0000 7f798c9ee700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f79880b2710 0x7f798814afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37234/0 (socket says 172.21.15.17:37234) 2022-08-01T19:43:38.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.413+0000 7f7987fff700 1 --2- 172.21.15.17:0/2515004191 >> v2:172.21.15.77:3300/0 conn(0x7f79880b01a0 0x7f79880a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:38.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.413+0000 7f798d1ef700 1 -- 172.21.15.17:0/2515004191 >> v2:172.21.15.77:3300/0 conn(0x7f79880b01a0 msgr2=0x7f79880a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:38.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.413+0000 7f798d1ef700 1 --2- 172.21.15.17:0/2515004191 >> v2:172.21.15.77:3300/0 conn(0x7f79880b01a0 0x7f79880a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:38.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.413+0000 7f798d1ef700 1 -- 172.21.15.17:0/2515004191 >> v1:172.21.15.17:6789/0 conn(0x7f79880b2710 legacy=0x7f798814afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:38.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.413+0000 7f798d1ef700 1 -- 172.21.15.17:0/2515004191 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79880a79c0 con 0x7f79880afd60 2022-08-01T19:43:38.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.413+0000 7f798d1ef700 1 --2- 172.21.15.17:0/2515004191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79880afd60 0x7f79880a8a30 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 rx=0x7f79800032f0 tx=0x7f7980009290).ready entity=mon.0 client_cookie=f847df090bebe469 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:38.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.413+0000 7f79877fe700 1 -- 172.21.15.17:0/2515004191 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f798000b870 con 0x7f79880afd60 2022-08-01T19:43:38.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.413+0000 7f79877fe700 1 -- 172.21.15.17:0/2515004191 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f798000b9d0 con 0x7f79880afd60 2022-08-01T19:43:38.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.413+0000 7f79877fe700 1 -- 172.21.15.17:0/2515004191 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f798000bb40 con 0x7f79880afd60 2022-08-01T19:43:38.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.413+0000 7f798d9f0700 1 -- 172.21.15.17:0/2515004191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79880afd60 msgr2=0x7f79880a8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:38.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.413+0000 7f798d9f0700 1 --2- 172.21.15.17:0/2515004191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79880afd60 0x7f79880a8a30 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 rx=0x7f79800032f0 tx=0x7f7980009290).stop 2022-08-01T19:43:38.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.413+0000 7f798d9f0700 1 -- 172.21.15.17:0/2515004191 shutdown_connections 2022-08-01T19:43:38.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.413+0000 7f798d9f0700 1 --2- 172.21.15.17:0/2515004191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79880afd60 0x7f79880a8a30 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:38.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.413+0000 7f798d9f0700 1 --2- 172.21.15.17:0/2515004191 >> v2:172.21.15.77:3300/0 conn(0x7f79880b01a0 0x7f79880a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:38.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.413+0000 7f798d9f0700 1 -- 172.21.15.17:0/2515004191 >> 172.21.15.17:0/2515004191 conn(0x7f7988060310 msgr2=0x7f79880b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:38.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.413+0000 7f798d9f0700 1 -- 172.21.15.17:0/2515004191 shutdown_connections 2022-08-01T19:43:38.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.413+0000 7f798d9f0700 1 -- 172.21.15.17:0/2515004191 wait complete. 2022-08-01T19:43:38.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.417+0000 7f798d9f0700 1 Processor -- start 2022-08-01T19:43:38.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.417+0000 7f798d9f0700 1 -- start start 2022-08-01T19:43:38.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.417+0000 7f798d9f0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79880afd60 0x7f798814e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:38.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.417+0000 7f798d9f0700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79880b01a0 0x7f798815a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:38.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.417+0000 7f798d9f0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79880b2710 0x7f798815a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:38.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.417+0000 7f798d9f0700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7988157620 con 0x7f79880b2710 2022-08-01T19:43:38.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.417+0000 7f798d9f0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7988156620 con 0x7f79880afd60 2022-08-01T19:43:38.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.417+0000 7f798d9f0700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7988156810 con 0x7f79880b01a0 2022-08-01T19:43:38.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.417+0000 7f798d1ef700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79880b2710 0x7f798815a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:38.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.417+0000 7f798d1ef700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79880b2710 0x7f798815a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45048/0 (socket says 172.21.15.17:45048) 2022-08-01T19:43:38.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.417+0000 7f798d1ef700 1 -- 172.21.15.17:0/1612234820 learned_addr learned my addr 172.21.15.17:0/1612234820 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:38.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.417+0000 7f798c9ee700 1 --2- 172.21.15.17:0/1612234820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79880afd60 0x7f798814e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:38.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.417+0000 7f7987fff700 1 --2- 172.21.15.17:0/1612234820 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79880b01a0 0x7f798815a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:38.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.417+0000 7f798d1ef700 1 -- 172.21.15.17:0/1612234820 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79880b01a0 msgr2=0x7f798815a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:38.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.417+0000 7f798d1ef700 1 --2- 172.21.15.17:0/1612234820 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79880b01a0 0x7f798815a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:38.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.417+0000 7f798d1ef700 1 -- 172.21.15.17:0/1612234820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79880afd60 msgr2=0x7f798814e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:38.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.417+0000 7f798d1ef700 1 --2- 172.21.15.17:0/1612234820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79880afd60 0x7f798814e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:38.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.417+0000 7f798d1ef700 1 -- 172.21.15.17:0/1612234820 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7980009530 con 0x7f79880b2710 2022-08-01T19:43:38.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.417+0000 7f798c9ee700 1 --2- 172.21.15.17:0/1612234820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79880afd60 0x7f798814e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:43:38.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.417+0000 7f798d1ef700 1 --2- 172.21.15.17:0/1612234820 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79880b2710 0x7f798815a6c0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 rx=0x7f798814d540 tx=0x7f7980016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:38.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.421+0000 7f7985ffb700 1 -- 172.21.15.17:0/1612234820 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f798000b870 con 0x7f79880b2710 2022-08-01T19:43:38.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.421+0000 7f7985ffb700 1 -- 172.21.15.17:0/1612234820 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7980021440 con 0x7f79880b2710 2022-08-01T19:43:38.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.421+0000 7f798d9f0700 1 -- 172.21.15.17:0/1612234820 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7988159d30 con 0x7f79880b2710 2022-08-01T19:43:38.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.421+0000 7f7985ffb700 1 -- 172.21.15.17:0/1612234820 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79800217a0 con 0x7f79880b2710 2022-08-01T19:43:38.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.421+0000 7f798d9f0700 1 -- 172.21.15.17:0/1612234820 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79880a93f0 con 0x7f79880b2710 2022-08-01T19:43:38.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.421+0000 7f798d9f0700 1 -- 172.21.15.17:0/1612234820 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79880574e0 con 0x7f79880b2710 2022-08-01T19:43:38.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.421+0000 7f7985ffb700 1 -- 172.21.15.17:0/1612234820 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7980027020 con 0x7f79880b2710 2022-08-01T19:43:38.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.421+0000 7f7985ffb700 1 --2- 172.21.15.17:0/1612234820 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f796c053b00 0x7f796c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:38.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.421+0000 7f7985ffb700 1 -- 172.21.15.17:0/1612234820 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f798006d520 con 0x7f79880b2710 2022-08-01T19:43:38.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.421+0000 7f798c9ee700 1 --2- 172.21.15.17:0/1612234820 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f796c053b00 0x7f796c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:38.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.425+0000 7f798c9ee700 1 --2- 172.21.15.17:0/1612234820 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f796c053b00 0x7f796c055fc0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f797000d000 tx=0x7f7970000bc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:38.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.425+0000 7f7985ffb700 1 -- 172.21.15.17:0/1612234820 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f79800442e0 con 0x7f79880b2710 2022-08-01T19:43:38.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.557+0000 7f798d9f0700 1 -- 172.21.15.17:0/1612234820 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7988159ec0 con 0x7f79880b2710 2022-08-01T19:43:38.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.557+0000 7f7985ffb700 1 -- 172.21.15.17:0/1612234820 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f7980015090 con 0x7f79880b2710 2022-08-01T19:43:38.564 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:38.565 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":148,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:43:35.589613+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:43:33.209247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2022-08-01T19:43:33.595201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:43:38.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.561+0000 7f798d9f0700 1 -- 172.21.15.17:0/1612234820 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f796c053b00 msgr2=0x7f796c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:38.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.561+0000 7f798d9f0700 1 --2- 172.21.15.17:0/1612234820 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f796c053b00 0x7f796c055fc0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f797000d000 tx=0x7f7970000bc0).stop 2022-08-01T19:43:38.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.561+0000 7f798d9f0700 1 -- 172.21.15.17:0/1612234820 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79880b2710 msgr2=0x7f798815a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:38.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.561+0000 7f798d9f0700 1 --2- 172.21.15.17:0/1612234820 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79880b2710 0x7f798815a6c0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 rx=0x7f798814d540 tx=0x7f7980016000).stop 2022-08-01T19:43:38.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.561+0000 7f798d9f0700 1 -- 172.21.15.17:0/1612234820 shutdown_connections 2022-08-01T19:43:38.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.561+0000 7f798d9f0700 1 --2- 172.21.15.17:0/1612234820 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f796c053b00 0x7f796c055fc0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:38.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.561+0000 7f798d9f0700 1 --2- 172.21.15.17:0/1612234820 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79880b2710 0x7f798815a6c0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:38.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.561+0000 7f798d9f0700 1 --2- 172.21.15.17:0/1612234820 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f79880b01a0 0x7f798815a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:38.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.561+0000 7f798d9f0700 1 --2- 172.21.15.17:0/1612234820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79880afd60 0x7f798814e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:38.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.561+0000 7f798d9f0700 1 -- 172.21.15.17:0/1612234820 >> 172.21.15.17:0/1612234820 conn(0x7f7988060310 msgr2=0x7f7988148ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:38.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.561+0000 7f798d9f0700 1 -- 172.21.15.17:0/1612234820 shutdown_connections 2022-08-01T19:43:38.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.561+0000 7f798d9f0700 1 -- 172.21.15.17:0/1612234820 wait complete. 2022-08-01T19:43:38.584 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T19:43:33.209247+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '147', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T19:43:33.595201+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '148', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T19:43:38.585 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:43:38.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.693+0000 7fc477354700 1 Processor -- start 2022-08-01T19:43:38.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.693+0000 7fc477354700 1 -- start start 2022-08-01T19:43:38.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.697+0000 7fc477354700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4700af450 0x7fc4700af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:38.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.697+0000 7fc477354700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc4700ad9e0 0x7fc4700a8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:38.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.697+0000 7fc477354700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4700aff30 con 0x7fc4700ad4b0 2022-08-01T19:43:38.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.697+0000 7fc477354700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc47005c7a0 con 0x7fc4700ad9e0 2022-08-01T19:43:38.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.697+0000 7fc477354700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc470140990 con 0x7fc4700af450 2022-08-01T19:43:38.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.697+0000 7fc476b53700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc4700ad9e0 0x7fc4700a8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:38.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.697+0000 7fc476b53700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc4700ad9e0 0x7fc4700a8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45056/0 (socket says 172.21.15.17:45056) 2022-08-01T19:43:38.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.697+0000 7fc476b53700 1 -- 172.21.15.17:0/3575256365 learned_addr learned my addr 172.21.15.17:0/3575256365 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:38.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.697+0000 7fc476352700 1 --2- 172.21.15.17:0/3575256365 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4700af450 0x7fc4700af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:38.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.697+0000 7fc476b53700 1 -- 172.21.15.17:0/3575256365 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4700af450 msgr2=0x7fc4700af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:38.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.697+0000 7fc476b53700 1 --2- 172.21.15.17:0/3575256365 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4700af450 0x7fc4700af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:38.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.697+0000 7fc476b53700 1 -- 172.21.15.17:0/3575256365 >> v1:172.21.15.77:6789/0 conn(0x7fc4700ad4b0 legacy=0x7fc4701432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:38.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.697+0000 7fc476b53700 1 -- 172.21.15.17:0/3575256365 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4701439f0 con 0x7fc4700ad9e0 2022-08-01T19:43:38.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.697+0000 7fc476352700 1 --2- 172.21.15.17:0/3575256365 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4700af450 0x7fc4700af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:43:38.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.697+0000 7fc476b53700 1 --2- 172.21.15.17:0/3575256365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4700ad9e0 0x7fc4700a8e40 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 rx=0x7fc4680032f0 tx=0x7fc468009290).ready entity=mon.0 client_cookie=381ce0c6ff5f8b7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:38.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.697+0000 7fc475350700 1 -- 172.21.15.17:0/3575256365 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc46800b8a0 con 0x7fc4700ad9e0 2022-08-01T19:43:38.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.697+0000 7fc475350700 1 -- 172.21.15.17:0/3575256365 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc46800ba00 con 0x7fc4700ad9e0 2022-08-01T19:43:38.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.697+0000 7fc475350700 1 -- 172.21.15.17:0/3575256365 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc46800bb70 con 0x7fc4700ad9e0 2022-08-01T19:43:38.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.697+0000 7fc477354700 1 -- 172.21.15.17:0/3575256365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4700ad9e0 msgr2=0x7fc4700a8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:38.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.697+0000 7fc477354700 1 --2- 172.21.15.17:0/3575256365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4700ad9e0 0x7fc4700a8e40 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 rx=0x7fc4680032f0 tx=0x7fc468009290).stop 2022-08-01T19:43:38.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.697+0000 7fc477354700 1 -- 172.21.15.17:0/3575256365 shutdown_connections 2022-08-01T19:43:38.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.697+0000 7fc477354700 1 --2- 172.21.15.17:0/3575256365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4700ad9e0 0x7fc4700a8e40 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:38.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.697+0000 7fc477354700 1 --2- 172.21.15.17:0/3575256365 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4700af450 0x7fc4700af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:38.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.697+0000 7fc477354700 1 -- 172.21.15.17:0/3575256365 >> 172.21.15.17:0/3575256365 conn(0x7fc4700612d0 msgr2=0x7fc470062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:38.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.697+0000 7fc477354700 1 -- 172.21.15.17:0/3575256365 shutdown_connections 2022-08-01T19:43:38.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.697+0000 7fc477354700 1 -- 172.21.15.17:0/3575256365 wait complete. 2022-08-01T19:43:38.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.701+0000 7fc477354700 1 Processor -- start 2022-08-01T19:43:38.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.701+0000 7fc477354700 1 -- start start 2022-08-01T19:43:38.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.701+0000 7fc477354700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4700ad4b0 0x7fc4700b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:38.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.701+0000 7fc477354700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4700ad9e0 0x7fc4700b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:38.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.701+0000 7fc477354700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4700af450 0x7fc4701539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:38.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.701+0000 7fc477354700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4700b6510 con 0x7fc4700ad9e0 2022-08-01T19:43:38.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.701+0000 7fc477354700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc470063900 con 0x7fc4700ad4b0 2022-08-01T19:43:38.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.701+0000 7fc477354700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc470063ba0 con 0x7fc4700af450 2022-08-01T19:43:38.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.701+0000 7fc475b51700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4700ad9e0 0x7fc4700b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:38.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.701+0000 7fc476352700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4700ad4b0 0x7fc4700b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:38.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.701+0000 7fc476b53700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4700af450 0x7fc4701539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:38.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.701+0000 7fc476b53700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4700af450 0x7fc4701539a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:41160/0 (socket says 172.21.15.17:41160) 2022-08-01T19:43:38.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.701+0000 7fc476b53700 1 -- 172.21.15.17:0/1204364119 learned_addr learned my addr 172.21.15.17:0/1204364119 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:38.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.701+0000 7fc475b51700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4700ad9e0 0x7fc4700b1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45062/0 (socket says 172.21.15.17:45062) 2022-08-01T19:43:38.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.701+0000 7fc475b51700 1 -- 172.21.15.17:0/1204364119 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4700af450 msgr2=0x7fc4701539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:38.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.701+0000 7fc475b51700 1 --2- 172.21.15.17:0/1204364119 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4700af450 0x7fc4701539a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:38.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.701+0000 7fc475b51700 1 -- 172.21.15.17:0/1204364119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4700ad4b0 msgr2=0x7fc4700b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:38.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.701+0000 7fc475b51700 1 --2- 172.21.15.17:0/1204364119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4700ad4b0 0x7fc4700b5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:38.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.701+0000 7fc475b51700 1 -- 172.21.15.17:0/1204364119 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc468009530 con 0x7fc4700ad9e0 2022-08-01T19:43:38.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.701+0000 7fc475b51700 1 --2- 172.21.15.17:0/1204364119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4700ad9e0 0x7fc4700b1e90 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 rx=0x7fc470144400 tx=0x7fc46c007b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:38.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.701+0000 7fc45f7fe700 1 -- 172.21.15.17:0/1204364119 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc46c012400 con 0x7fc4700ad9e0 2022-08-01T19:43:38.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.701+0000 7fc45f7fe700 1 -- 172.21.15.17:0/1204364119 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc46c012560 con 0x7fc4700ad9e0 2022-08-01T19:43:38.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.701+0000 7fc45f7fe700 1 -- 172.21.15.17:0/1204364119 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc46c0128c0 con 0x7fc4700ad9e0 2022-08-01T19:43:38.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.701+0000 7fc477354700 1 -- 172.21.15.17:0/1204364119 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc470063e40 con 0x7fc4700ad9e0 2022-08-01T19:43:38.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.701+0000 7fc477354700 1 -- 172.21.15.17:0/1204364119 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4700b6a60 con 0x7fc4700ad9e0 2022-08-01T19:43:38.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.705+0000 7fc45f7fe700 1 -- 172.21.15.17:0/1204364119 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc46c019030 con 0x7fc4700ad9e0 2022-08-01T19:43:38.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.705+0000 7fc477354700 1 -- 172.21.15.17:0/1204364119 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc470063fd0 con 0x7fc4700ad9e0 2022-08-01T19:43:38.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.705+0000 7fc45f7fe700 1 --2- 172.21.15.17:0/1204364119 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc450053a10 0x7fc450055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:38.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.705+0000 7fc45f7fe700 1 -- 172.21.15.17:0/1204364119 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc46c06b310 con 0x7fc4700ad9e0 2022-08-01T19:43:38.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.705+0000 7fc476352700 1 --2- 172.21.15.17:0/1204364119 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc450053a10 0x7fc450055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:38.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.705+0000 7fc476352700 1 --2- 172.21.15.17:0/1204364119 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc450053a10 0x7fc450055ed0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7fc46000d000 tx=0x7fc460008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:38.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.713+0000 7fc45f7fe700 1 -- 172.21.15.17:0/1204364119 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc46c041fc0 con 0x7fc4700ad9e0 2022-08-01T19:43:38.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.841+0000 7fc477354700 1 -- 172.21.15.17:0/1204364119 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc4700574e0 con 0x7fc4700ad9e0 2022-08-01T19:43:38.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.841+0000 7fc45f7fe700 1 -- 172.21.15.17:0/1204364119 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v148) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc46c0455e0 con 0x7fc4700ad9e0 2022-08-01T19:43:38.848 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:43:38.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.841+0000 7fc477354700 1 -- 172.21.15.17:0/1204364119 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc450053a10 msgr2=0x7fc450055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:38.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.841+0000 7fc477354700 1 --2- 172.21.15.17:0/1204364119 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc450053a10 0x7fc450055ed0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7fc46000d000 tx=0x7fc460008040).stop 2022-08-01T19:43:38.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.841+0000 7fc477354700 1 -- 172.21.15.17:0/1204364119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4700ad9e0 msgr2=0x7fc4700b1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:38.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.841+0000 7fc477354700 1 --2- 172.21.15.17:0/1204364119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4700ad9e0 0x7fc4700b1e90 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 rx=0x7fc470144400 tx=0x7fc46c007b90).stop 2022-08-01T19:43:38.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.841+0000 7fc477354700 1 -- 172.21.15.17:0/1204364119 shutdown_connections 2022-08-01T19:43:38.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.841+0000 7fc477354700 1 --2- 172.21.15.17:0/1204364119 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc450053a10 0x7fc450055ed0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:38.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.841+0000 7fc477354700 1 --2- 172.21.15.17:0/1204364119 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc4700af450 0x7fc4701539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:38.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.841+0000 7fc477354700 1 --2- 172.21.15.17:0/1204364119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4700ad9e0 0x7fc4700b1e90 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:38.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.841+0000 7fc477354700 1 --2- 172.21.15.17:0/1204364119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4700ad4b0 0x7fc4700b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:38.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.841+0000 7fc477354700 1 -- 172.21.15.17:0/1204364119 >> 172.21.15.17:0/1204364119 conn(0x7fc4700612d0 msgr2=0x7fc4700a8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:38.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.845+0000 7fc477354700 1 -- 172.21.15.17:0/1204364119 shutdown_connections 2022-08-01T19:43:38.858 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.845+0000 7fc477354700 1 -- 172.21.15.17:0/1204364119 wait complete. 2022-08-01T19:43:38.859 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:43:38.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.969+0000 7fed4f59e700 1 Processor -- start 2022-08-01T19:43:38.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.969+0000 7fed4f59e700 1 -- start start 2022-08-01T19:43:38.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.973+0000 7fed4f59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fed500a9320 0x7fed501466e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:38.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.973+0000 7fed4f59e700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed500b12d0 con 0x7fed500b0d10 2022-08-01T19:43:38.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.973+0000 7fed4f59e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed500a9700 con 0x7fed500b0560 2022-08-01T19:43:38.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.973+0000 7fed4f59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed500b1500 con 0x7fed500a9320 2022-08-01T19:43:38.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.973+0000 7fed4dd9b700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fed500b0d10 0x7fed50146bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37264/0 (socket says 172.21.15.17:37264) 2022-08-01T19:43:38.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.973+0000 7fed4dd9b700 1 -- 172.21.15.17:0/158820820 learned_addr learned my addr 172.21.15.17:0/158820820 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:38.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.973+0000 7fed4e59c700 1 --2- 172.21.15.17:0/158820820 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fed500a9320 0x7fed501466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:38.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.973+0000 7fed4d59a700 1 -- 172.21.15.17:0/158820820 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1383785665 0 0) 0x7fed500b12d0 con 0x7fed500b0d10 2022-08-01T19:43:38.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.973+0000 7fed4d59a700 1 -- 172.21.15.17:0/158820820 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed30003260 con 0x7fed500b0d10 2022-08-01T19:43:38.991 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.973+0000 7fed4d59a700 1 -- 172.21.15.17:0/158820820 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 317769608 0 0) 0x7fed500a9700 con 0x7fed500b0560 2022-08-01T19:43:38.991 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.973+0000 7fed4d59a700 1 -- 172.21.15.17:0/158820820 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed500b12d0 con 0x7fed500b0560 2022-08-01T19:43:38.992 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.973+0000 7fed4e59c700 1 -- 172.21.15.17:0/158820820 >> v1:172.21.15.77:6789/0 conn(0x7fed500b0560 legacy=0x7fed50140350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:38.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.973+0000 7fed4e59c700 1 -- 172.21.15.17:0/158820820 >> v1:172.21.15.17:6789/0 conn(0x7fed500b0d10 legacy=0x7fed50146bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:38.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.973+0000 7fed4e59c700 1 -- 172.21.15.17:0/158820820 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed500b17a0 con 0x7fed500a9320 2022-08-01T19:43:38.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.973+0000 7fed4e59c700 1 --2- 172.21.15.17:0/158820820 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fed500a9320 0x7fed501466e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7fed38002f70 tx=0x7fed38006df0).ready entity=mon.2 client_cookie=b2ce220e2b05c6a5 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:38.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.973+0000 7fed4d59a700 1 -- 172.21.15.17:0/158820820 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed3800d5c0 con 0x7fed500a9320 2022-08-01T19:43:38.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.973+0000 7fed4d59a700 1 -- 172.21.15.17:0/158820820 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed38000e90 con 0x7fed500a9320 2022-08-01T19:43:38.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.973+0000 7fed4d59a700 1 -- 172.21.15.17:0/158820820 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed3800d910 con 0x7fed500a9320 2022-08-01T19:43:38.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.973+0000 7fed4f59e700 1 -- 172.21.15.17:0/158820820 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fed500a9320 msgr2=0x7fed501466e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:38.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.973+0000 7fed4f59e700 1 --2- 172.21.15.17:0/158820820 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fed500a9320 0x7fed501466e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7fed38002f70 tx=0x7fed38006df0).stop 2022-08-01T19:43:38.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.973+0000 7fed4f59e700 1 -- 172.21.15.17:0/158820820 shutdown_connections 2022-08-01T19:43:38.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.973+0000 7fed4f59e700 1 --2- 172.21.15.17:0/158820820 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fed500a9320 0x7fed501466e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:38.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.973+0000 7fed4f59e700 1 -- 172.21.15.17:0/158820820 >> 172.21.15.17:0/158820820 conn(0x7fed500612d0 msgr2=0x7fed50062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:39.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.973+0000 7fed4f59e700 1 -- 172.21.15.17:0/158820820 shutdown_connections 2022-08-01T19:43:39.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.973+0000 7fed4f59e700 1 -- 172.21.15.17:0/158820820 wait complete. 2022-08-01T19:43:39.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.977+0000 7fed4f59e700 1 Processor -- start 2022-08-01T19:43:39.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.977+0000 7fed4f59e700 1 -- start start 2022-08-01T19:43:39.009 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.977+0000 7fed4f59e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed500a9320 0x7fed500ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:39.010 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.977+0000 7fed4f59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fed500b0560 0x7fed50155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:39.010 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.977+0000 7fed4f59e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed500b0d10 0x7fed50156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:39.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.977+0000 7fed4f59e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed50156910 con 0x7fed500a9320 2022-08-01T19:43:39.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.977+0000 7fed4f59e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed500aa7a0 con 0x7fed500b0d10 2022-08-01T19:43:39.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.977+0000 7fed4f59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed500aaa40 con 0x7fed500b0560 2022-08-01T19:43:39.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.977+0000 7fed4e59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed500a9320 0x7fed500ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:39.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.977+0000 7fed4e59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed500a9320 0x7fed500ae2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45072/0 (socket says 172.21.15.17:45072) 2022-08-01T19:43:39.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.977+0000 7fed4e59c700 1 -- 172.21.15.17:0/2549627935 learned_addr learned my addr 172.21.15.17:0/2549627935 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:39.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.977+0000 7fed4ed9d700 1 --2- 172.21.15.17:0/2549627935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed500b0d10 0x7fed50156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:39.022 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.977+0000 7fed4dd9b700 1 --2- 172.21.15.17:0/2549627935 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fed500b0560 0x7fed50155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:39.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.977+0000 7fed4e59c700 1 -- 172.21.15.17:0/2549627935 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fed500b0560 msgr2=0x7fed50155f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:39.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.977+0000 7fed4e59c700 1 --2- 172.21.15.17:0/2549627935 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fed500b0560 0x7fed50155f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:39.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.977+0000 7fed4e59c700 1 -- 172.21.15.17:0/2549627935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed500b0d10 msgr2=0x7fed50156420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:39.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.977+0000 7fed4e59c700 1 --2- 172.21.15.17:0/2549627935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed500b0d10 0x7fed50156420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:39.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.977+0000 7fed4e59c700 1 -- 172.21.15.17:0/2549627935 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed3800b180 con 0x7fed500a9320 2022-08-01T19:43:39.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.981+0000 7fed4e59c700 1 --2- 172.21.15.17:0/2549627935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed500a9320 0x7fed500ae2b0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 rx=0x7fed38002f70 tx=0x7fed38014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:39.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.981+0000 7fed477fe700 1 -- 172.21.15.17:0/2549627935 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed38013de0 con 0x7fed500a9320 2022-08-01T19:43:39.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.981+0000 7fed477fe700 1 -- 172.21.15.17:0/2549627935 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed3801e400 con 0x7fed500a9320 2022-08-01T19:43:39.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.981+0000 7fed4f59e700 1 -- 172.21.15.17:0/2549627935 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed500aace0 con 0x7fed500a9320 2022-08-01T19:43:39.026 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.981+0000 7fed477fe700 1 -- 172.21.15.17:0/2549627935 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed3801e760 con 0x7fed500a9320 2022-08-01T19:43:39.026 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.981+0000 7fed4f59e700 1 -- 172.21.15.17:0/2549627935 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed500aa490 con 0x7fed500a9320 2022-08-01T19:43:39.026 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.981+0000 7fed4f59e700 1 -- 172.21.15.17:0/2549627935 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed500574e0 con 0x7fed500a9320 2022-08-01T19:43:39.027 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.981+0000 7fed477fe700 1 -- 172.21.15.17:0/2549627935 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fed3801c070 con 0x7fed500a9320 2022-08-01T19:43:39.027 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.981+0000 7fed477fe700 1 --2- 172.21.15.17:0/2549627935 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fed34053a70 0x7fed34055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:39.027 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.981+0000 7fed477fe700 1 -- 172.21.15.17:0/2549627935 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fed3806ca30 con 0x7fed500a9320 2022-08-01T19:43:39.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.981+0000 7fed4dd9b700 1 --2- 172.21.15.17:0/2549627935 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fed34053a70 0x7fed34055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:39.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.985+0000 7fed4dd9b700 1 --2- 172.21.15.17:0/2549627935 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fed34053a70 0x7fed34055f30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7fed4000d000 tx=0x7fed40009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:39.028 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:38.989+0000 7fed477fe700 1 -- 172.21.15.17:0/2549627935 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fed38043750 con 0x7fed500a9320 2022-08-01T19:43:39.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.117+0000 7fed4f59e700 1 -- 172.21.15.17:0/2549627935 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fed501475c0 con 0x7fed500a9320 2022-08-01T19:43:39.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.117+0000 7fed477fe700 1 -- 172.21.15.17:0/2549627935 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v148) v1 ==== 92+0+10 (secure 0 0 0) 0x7fed38023020 con 0x7fed500a9320 2022-08-01T19:43:39.121 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:43:39.134 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.117+0000 7fed4f59e700 1 -- 172.21.15.17:0/2549627935 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fed34053a70 msgr2=0x7fed34055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:39.134 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.117+0000 7fed4f59e700 1 --2- 172.21.15.17:0/2549627935 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fed34053a70 0x7fed34055f30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7fed4000d000 tx=0x7fed40009040).stop 2022-08-01T19:43:39.135 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.117+0000 7fed4f59e700 1 -- 172.21.15.17:0/2549627935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed500a9320 msgr2=0x7fed500ae2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:39.135 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.117+0000 7fed4f59e700 1 --2- 172.21.15.17:0/2549627935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed500a9320 0x7fed500ae2b0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 rx=0x7fed38002f70 tx=0x7fed38014000).stop 2022-08-01T19:43:39.135 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.117+0000 7fed4f59e700 1 -- 172.21.15.17:0/2549627935 shutdown_connections 2022-08-01T19:43:39.135 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.117+0000 7fed4f59e700 1 --2- 172.21.15.17:0/2549627935 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fed34053a70 0x7fed34055f30 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:39.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.117+0000 7fed4f59e700 1 --2- 172.21.15.17:0/2549627935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed500b0d10 0x7fed50156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:39.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.117+0000 7fed4f59e700 1 --2- 172.21.15.17:0/2549627935 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fed500b0560 0x7fed50155f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:39.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.117+0000 7fed4f59e700 1 --2- 172.21.15.17:0/2549627935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed500a9320 0x7fed500ae2b0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:39.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.117+0000 7fed4f59e700 1 -- 172.21.15.17:0/2549627935 >> 172.21.15.17:0/2549627935 conn(0x7fed500612d0 msgr2=0x7fed500b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:39.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.117+0000 7fed4f59e700 1 -- 172.21.15.17:0/2549627935 shutdown_connections 2022-08-01T19:43:39.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.117+0000 7fed4f59e700 1 -- 172.21.15.17:0/2549627935 wait complete. 2022-08-01T19:43:39.137 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T19:43:39.246 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.245+0000 7f28a4f59700 1 Processor -- start 2022-08-01T19:43:39.246 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.245+0000 7f28a4f59700 1 -- start start 2022-08-01T19:43:39.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.245+0000 7f28a4f59700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f28a00b0270 0x7f28a0143000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:39.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.245+0000 7f28a4f59700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f28a00ac880 0x7f28a01434f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:39.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.245+0000 7f28a4f59700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28a0067a10 con 0x7f28a00b2710 2022-08-01T19:43:39.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.245+0000 7f28a4f59700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28a00ad740 con 0x7f28a00ac880 2022-08-01T19:43:39.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.245+0000 7f28a4f59700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28a0067cb0 con 0x7f28a00b0270 2022-08-01T19:43:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.245+0000 7f289ffff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f28a00ac880 0x7f28a01434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.245+0000 7f289ffff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f28a00ac880 0x7f28a01434f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45084/0 (socket says 172.21.15.17:45084) 2022-08-01T19:43:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.245+0000 7f289ffff700 1 -- 172.21.15.17:0/63992300 learned_addr learned my addr 172.21.15.17:0/63992300 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.245+0000 7f289effd700 1 --2- 172.21.15.17:0/63992300 >> v2:172.21.15.77:3300/0 conn(0x7f28a00b0270 0x7f28a0143000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:39.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.245+0000 7f289ffff700 1 -- 172.21.15.17:0/63992300 >> v2:172.21.15.77:3300/0 conn(0x7f28a00b0270 msgr2=0x7f28a0143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:39.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.245+0000 7f289ffff700 1 --2- 172.21.15.17:0/63992300 >> v2:172.21.15.77:3300/0 conn(0x7f28a00b0270 0x7f28a0143000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:39.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.245+0000 7f289ffff700 1 -- 172.21.15.17:0/63992300 >> v1:172.21.15.77:6789/0 conn(0x7f28a00b2710 legacy=0x7f28a01401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:39.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.245+0000 7f289ffff700 1 -- 172.21.15.17:0/63992300 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a0147160 con 0x7f28a00ac880 2022-08-01T19:43:39.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.245+0000 7f289ffff700 1 --2- 172.21.15.17:0/63992300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28a00ac880 0x7f28a01434f0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 rx=0x7f28980032f0 tx=0x7f2898009290).ready entity=mon.0 client_cookie=c45d0a1cdf14dba5 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:39.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.245+0000 7f289e7fc700 1 -- 172.21.15.17:0/63992300 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f289800b870 con 0x7f28a00ac880 2022-08-01T19:43:39.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.245+0000 7f289e7fc700 1 -- 172.21.15.17:0/63992300 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f289800b9d0 con 0x7f28a00ac880 2022-08-01T19:43:39.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.245+0000 7f289e7fc700 1 -- 172.21.15.17:0/63992300 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f289800bb40 con 0x7f28a00ac880 2022-08-01T19:43:39.261 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.249+0000 7f28a4f59700 1 -- 172.21.15.17:0/63992300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28a00ac880 msgr2=0x7f28a01434f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:39.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.249+0000 7f28a4f59700 1 --2- 172.21.15.17:0/63992300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28a00ac880 0x7f28a01434f0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 rx=0x7f28980032f0 tx=0x7f2898009290).stop 2022-08-01T19:43:39.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.249+0000 7f28a4f59700 1 -- 172.21.15.17:0/63992300 shutdown_connections 2022-08-01T19:43:39.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.249+0000 7f28a4f59700 1 --2- 172.21.15.17:0/63992300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28a00ac880 0x7f28a01434f0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:39.262 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.249+0000 7f28a4f59700 1 --2- 172.21.15.17:0/63992300 >> v2:172.21.15.77:3300/0 conn(0x7f28a00b0270 0x7f28a0143000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:39.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.249+0000 7f28a4f59700 1 -- 172.21.15.17:0/63992300 >> 172.21.15.17:0/63992300 conn(0x7f28a0060310 msgr2=0x7f28a00b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:39.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.249+0000 7f28a4f59700 1 -- 172.21.15.17:0/63992300 shutdown_connections 2022-08-01T19:43:39.266 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.249+0000 7f28a4f59700 1 -- 172.21.15.17:0/63992300 wait complete. 2022-08-01T19:43:39.266 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.249+0000 7f28a4f59700 1 Processor -- start 2022-08-01T19:43:39.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.249+0000 7f28a4f59700 1 -- start start 2022-08-01T19:43:39.271 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.249+0000 7f28a4f59700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28a00ac880 0x7f28a0152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:39.271 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.249+0000 7f28a4f59700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f28a00b0270 0x7f28a0152950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:39.271 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.249+0000 7f28a4f59700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28a00b2710 0x7f28a0141950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:39.272 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.249+0000 7f28a4f59700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a0140ea0 con 0x7f28a00ac880 2022-08-01T19:43:39.272 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.249+0000 7f28a4f59700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a0143030 con 0x7f28a00b2710 2022-08-01T19:43:39.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.249+0000 7f28a4f59700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28a01410d0 con 0x7f28a00b0270 2022-08-01T19:43:39.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.253+0000 7f289f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28a00ac880 0x7f28a0152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:39.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.253+0000 7f289ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28a00b2710 0x7f28a0141950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:39.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.253+0000 7f289effd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f28a00b0270 0x7f28a0152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:39.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.253+0000 7f289ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28a00b2710 0x7f28a0141950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:43012/0 (socket says 172.21.15.17:43012) 2022-08-01T19:43:39.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.253+0000 7f289ffff700 1 -- 172.21.15.17:0/4087623470 learned_addr learned my addr 172.21.15.17:0/4087623470 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:39.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.253+0000 7f289f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28a00ac880 0x7f28a0152460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45086/0 (socket says 172.21.15.17:45086) 2022-08-01T19:43:39.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.253+0000 7f289f7fe700 1 -- 172.21.15.17:0/4087623470 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f28a00b0270 msgr2=0x7f28a0152950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:39.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.253+0000 7f289f7fe700 1 --2- 172.21.15.17:0/4087623470 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f28a00b0270 0x7f28a0152950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:39.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.253+0000 7f289f7fe700 1 -- 172.21.15.17:0/4087623470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28a00b2710 msgr2=0x7f28a0141950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:39.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.253+0000 7f289f7fe700 1 --2- 172.21.15.17:0/4087623470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28a00b2710 0x7f28a0141950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:39.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.253+0000 7f289f7fe700 1 -- 172.21.15.17:0/4087623470 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2898009530 con 0x7f28a00ac880 2022-08-01T19:43:39.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.253+0000 7f289ffff700 1 --2- 172.21.15.17:0/4087623470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28a00b2710 0x7f28a0141950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:43:39.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.253+0000 7f289f7fe700 1 --2- 172.21.15.17:0/4087623470 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28a00ac880 0x7f28a0152460 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 rx=0x7f28a0061c10 tx=0x7f2890007b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:39.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.253+0000 7f289cff9700 1 -- 172.21.15.17:0/4087623470 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f289000ccd0 con 0x7f28a00ac880 2022-08-01T19:43:39.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.253+0000 7f289cff9700 1 -- 172.21.15.17:0/4087623470 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f289000ce30 con 0x7f28a00ac880 2022-08-01T19:43:39.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.253+0000 7f28a4f59700 1 -- 172.21.15.17:0/4087623470 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28a0153340 con 0x7f28a00ac880 2022-08-01T19:43:39.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.253+0000 7f289cff9700 1 -- 172.21.15.17:0/4087623470 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28900125f0 con 0x7f28a00ac880 2022-08-01T19:43:39.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.253+0000 7f28a4f59700 1 -- 172.21.15.17:0/4087623470 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28a00ab200 con 0x7f28a00ac880 2022-08-01T19:43:39.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.253+0000 7f28a4f59700 1 -- 172.21.15.17:0/4087623470 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28a00574e0 con 0x7f28a00ac880 2022-08-01T19:43:39.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.257+0000 7f289cff9700 1 -- 172.21.15.17:0/4087623470 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2890019030 con 0x7f28a00ac880 2022-08-01T19:43:39.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.257+0000 7f289cff9700 1 --2- 172.21.15.17:0/4087623470 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2880053980 0x7f2880055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:39.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.257+0000 7f289cff9700 1 -- 172.21.15.17:0/4087623470 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f289006b2c0 con 0x7f28a00ac880 2022-08-01T19:43:39.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.257+0000 7f289effd700 1 --2- 172.21.15.17:0/4087623470 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2880053980 0x7f2880055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:39.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.257+0000 7f289effd700 1 --2- 172.21.15.17:0/4087623470 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2880053980 0x7f2880055e40 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7f289400c000 tx=0x7f2894008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:39.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.261+0000 7f289cff9700 1 -- 172.21.15.17:0/4087623470 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2890045e60 con 0x7f28a00ac880 2022-08-01T19:43:39.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.385+0000 7f28a4f59700 1 -- 172.21.15.17:0/4087623470 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f28a00b1bb0 con 0x7f28a00ac880 2022-08-01T19:43:39.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.385+0000 7f289cff9700 1 -- 172.21.15.17:0/4087623470 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v148) v1 ==== 104+0+10 (secure 0 0 0) 0x7f2890041f70 con 0x7f28a00ac880 2022-08-01T19:43:39.390 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:43:39.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.389+0000 7f28a4f59700 1 -- 172.21.15.17:0/4087623470 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2880053980 msgr2=0x7f2880055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:39.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.389+0000 7f28a4f59700 1 --2- 172.21.15.17:0/4087623470 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2880053980 0x7f2880055e40 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7f289400c000 tx=0x7f2894008040).stop 2022-08-01T19:43:39.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.389+0000 7f28a4f59700 1 -- 172.21.15.17:0/4087623470 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28a00ac880 msgr2=0x7f28a0152460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:39.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.389+0000 7f28a4f59700 1 --2- 172.21.15.17:0/4087623470 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28a00ac880 0x7f28a0152460 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 rx=0x7f28a0061c10 tx=0x7f2890007b90).stop 2022-08-01T19:43:39.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.389+0000 7f28a4f59700 1 -- 172.21.15.17:0/4087623470 shutdown_connections 2022-08-01T19:43:39.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.389+0000 7f28a4f59700 1 --2- 172.21.15.17:0/4087623470 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2880053980 0x7f2880055e40 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:39.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.389+0000 7f28a4f59700 1 --2- 172.21.15.17:0/4087623470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28a00b2710 0x7f28a0141950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:39.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.389+0000 7f28a4f59700 1 --2- 172.21.15.17:0/4087623470 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f28a00b0270 0x7f28a0152950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:39.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.389+0000 7f28a4f59700 1 --2- 172.21.15.17:0/4087623470 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28a00ac880 0x7f28a0152460 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:39.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.389+0000 7f28a4f59700 1 -- 172.21.15.17:0/4087623470 >> 172.21.15.17:0/4087623470 conn(0x7f28a0060310 msgr2=0x7f28a00b5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:39.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.389+0000 7f28a4f59700 1 -- 172.21.15.17:0/4087623470 shutdown_connections 2022-08-01T19:43:39.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.389+0000 7f28a4f59700 1 -- 172.21.15.17:0/4087623470 wait complete. 2022-08-01T19:43:39.405 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T19:43:39.539 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.517+0000 7f52b880d700 1 Processor -- start 2022-08-01T19:43:39.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.517+0000 7f52b880d700 1 -- start start 2022-08-01T19:43:39.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.521+0000 7f52b880d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f52b40b1460 0x7f52b40b0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:39.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.521+0000 7f52b880d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f52b40b2430 0x7f52b41413e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:39.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.521+0000 7f52b880d700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52b40a7960 con 0x7f52b40b2050 2022-08-01T19:43:39.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.521+0000 7f52b880d700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52b40b18d0 con 0x7f52b40b1460 2022-08-01T19:43:39.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.521+0000 7f52b880d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b40b07e0 con 0x7f52b40b2430 2022-08-01T19:43:39.547 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.521+0000 7f52a9d9b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f52b40b1460 0x7f52b40b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:39.547 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.521+0000 7f52a9d9b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f52b40b1460 0x7f52b40b0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45098/0 (socket says 172.21.15.17:45098) 2022-08-01T19:43:39.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.521+0000 7f52a9d9b700 1 -- 172.21.15.17:0/3730156038 learned_addr learned my addr 172.21.15.17:0/3730156038 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:39.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.521+0000 7f52b2d9d700 1 --2- 172.21.15.17:0/3730156038 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f52b40b2430 0x7f52b41413e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:39.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.521+0000 7f52b2d9d700 1 -- 172.21.15.17:0/3730156038 >> v2:172.21.15.17:3300/0 conn(0x7f52b40b1460 msgr2=0x7f52b40b0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:39.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.521+0000 7f52b2d9d700 1 --2- 172.21.15.17:0/3730156038 >> v2:172.21.15.17:3300/0 conn(0x7f52b40b1460 0x7f52b40b0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:39.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.521+0000 7f52b2d9d700 1 -- 172.21.15.17:0/3730156038 >> v1:172.21.15.77:6789/0 conn(0x7f52b40b2050 legacy=0x7f52b40a7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:39.550 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.521+0000 7f52b2d9d700 1 -- 172.21.15.17:0/3730156038 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b40b0a80 con 0x7f52b40b2430 2022-08-01T19:43:39.550 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.521+0000 7f52b2d9d700 1 --2- 172.21.15.17:0/3730156038 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f52b40b2430 0x7f52b41413e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f52ac0032f0 tx=0x7f52ac0091d0).ready entity=mon.2 client_cookie=23c2c9ca03915a5f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:39.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.521+0000 7f52b1d9b700 1 -- 172.21.15.17:0/3730156038 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52ac00b6c0 con 0x7f52b40b2430 2022-08-01T19:43:39.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.521+0000 7f52b1d9b700 1 -- 172.21.15.17:0/3730156038 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ac00b820 con 0x7f52b40b2430 2022-08-01T19:43:39.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.521+0000 7f52b1d9b700 1 -- 172.21.15.17:0/3730156038 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52ac00b990 con 0x7f52b40b2430 2022-08-01T19:43:39.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.521+0000 7f52b880d700 1 -- 172.21.15.17:0/3730156038 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f52b40b2430 msgr2=0x7f52b41413e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:39.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.521+0000 7f52b880d700 1 --2- 172.21.15.17:0/3730156038 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f52b40b2430 0x7f52b41413e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f52ac0032f0 tx=0x7f52ac0091d0).stop 2022-08-01T19:43:39.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.521+0000 7f52b880d700 1 -- 172.21.15.17:0/3730156038 shutdown_connections 2022-08-01T19:43:39.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.521+0000 7f52b880d700 1 --2- 172.21.15.17:0/3730156038 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f52b40b2430 0x7f52b41413e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:39.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.521+0000 7f52b880d700 1 --2- 172.21.15.17:0/3730156038 >> v2:172.21.15.17:3300/0 conn(0x7f52b40b1460 0x7f52b40b0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:39.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.521+0000 7f52b880d700 1 -- 172.21.15.17:0/3730156038 >> 172.21.15.17:0/3730156038 conn(0x7f52b405f840 msgr2=0x7f52b405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:39.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.521+0000 7f52b880d700 1 -- 172.21.15.17:0/3730156038 shutdown_connections 2022-08-01T19:43:39.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.521+0000 7f52b880d700 1 -- 172.21.15.17:0/3730156038 wait complete. 2022-08-01T19:43:39.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.525+0000 7f52b880d700 1 Processor -- start 2022-08-01T19:43:39.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.525+0000 7f52b880d700 1 -- start start 2022-08-01T19:43:39.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.525+0000 7f52b880d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52b40b1460 0x7f52b40b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:39.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.525+0000 7f52b880d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52b40b2050 0x7f52b4149270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:39.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.525+0000 7f52b880d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f52b40b2430 0x7f52b4137590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:39.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.525+0000 7f52b880d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b40afcd0 con 0x7f52b40b1460 2022-08-01T19:43:39.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.525+0000 7f52b880d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b4148cc0 con 0x7f52b40b2050 2022-08-01T19:43:39.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.525+0000 7f52b880d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b40b0330 con 0x7f52b40b2430 2022-08-01T19:43:39.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.525+0000 7f52b259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52b40b1460 0x7f52b40b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:39.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.525+0000 7f52b259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52b40b1460 0x7f52b40b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45100/0 (socket says 172.21.15.17:45100) 2022-08-01T19:43:39.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.525+0000 7f52b259c700 1 -- 172.21.15.17:0/3573246664 learned_addr learned my addr 172.21.15.17:0/3573246664 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:39.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.525+0000 7f52b2d9d700 1 --2- 172.21.15.17:0/3573246664 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f52b40b2430 0x7f52b4137590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:39.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.525+0000 7f52a9d9b700 1 --2- 172.21.15.17:0/3573246664 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52b40b2050 0x7f52b4149270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:39.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.525+0000 7f52b259c700 1 -- 172.21.15.17:0/3573246664 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f52b40b2430 msgr2=0x7f52b4137590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:39.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.525+0000 7f52b259c700 1 --2- 172.21.15.17:0/3573246664 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f52b40b2430 0x7f52b4137590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:39.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.525+0000 7f52b259c700 1 -- 172.21.15.17:0/3573246664 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52b40b2050 msgr2=0x7f52b4149270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:39.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.525+0000 7f52b259c700 1 --2- 172.21.15.17:0/3573246664 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52b40b2050 0x7f52b4149270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:39.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.525+0000 7f52b259c700 1 -- 172.21.15.17:0/3573246664 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ac009470 con 0x7f52b40b1460 2022-08-01T19:43:39.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.525+0000 7f52b259c700 1 --2- 172.21.15.17:0/3573246664 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52b40b1460 0x7f52b40b3e20 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 rx=0x7f52b40afe10 tx=0x7f52a4007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:39.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.525+0000 7f52abfff700 1 -- 172.21.15.17:0/3573246664 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52a4007030 con 0x7f52b40b1460 2022-08-01T19:43:39.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.525+0000 7f52abfff700 1 -- 172.21.15.17:0/3573246664 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a4007190 con 0x7f52b40b1460 2022-08-01T19:43:39.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.525+0000 7f52abfff700 1 -- 172.21.15.17:0/3573246664 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52a4006510 con 0x7f52b40b1460 2022-08-01T19:43:39.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.525+0000 7f52b880d700 1 -- 172.21.15.17:0/3573246664 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52b40b45f0 con 0x7f52b40b1460 2022-08-01T19:43:39.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.525+0000 7f52b880d700 1 -- 172.21.15.17:0/3573246664 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b40b4310 con 0x7f52b40b1460 2022-08-01T19:43:39.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.529+0000 7f52b880d700 1 -- 172.21.15.17:0/3573246664 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52b40574e0 con 0x7f52b40b1460 2022-08-01T19:43:39.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.529+0000 7f52abfff700 1 -- 172.21.15.17:0/3573246664 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f52a4010030 con 0x7f52b40b1460 2022-08-01T19:43:39.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.529+0000 7f52abfff700 1 --2- 172.21.15.17:0/3573246664 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f52980539e0 0x7f5298055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:39.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.529+0000 7f52abfff700 1 -- 172.21.15.17:0/3573246664 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f52a406bda0 con 0x7f52b40b1460 2022-08-01T19:43:39.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.529+0000 7f52a9d9b700 1 --2- 172.21.15.17:0/3573246664 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f52980539e0 0x7f5298055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:39.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.529+0000 7f52a9d9b700 1 --2- 172.21.15.17:0/3573246664 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f52980539e0 0x7f5298055ea0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7f529c008000 tx=0x7f529c008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:39.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.533+0000 7f52abfff700 1 -- 172.21.15.17:0/3573246664 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f52a4042a50 con 0x7f52b40b1460 2022-08-01T19:43:39.660 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.657+0000 7f52b880d700 1 -- 172.21.15.17:0/3573246664 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f52b40ac8f0 con 0x7f52b40b1460 2022-08-01T19:43:39.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.657+0000 7f52abfff700 1 -- 172.21.15.17:0/3573246664 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v148) v1 ==== 100+0+10 (secure 0 0 0) 0x7f52a4046070 con 0x7f52b40b1460 2022-08-01T19:43:39.662 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:43:39.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.661+0000 7f52b880d700 1 -- 172.21.15.17:0/3573246664 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f52980539e0 msgr2=0x7f5298055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:39.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.661+0000 7f52b880d700 1 --2- 172.21.15.17:0/3573246664 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f52980539e0 0x7f5298055ea0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7f529c008000 tx=0x7f529c008040).stop 2022-08-01T19:43:39.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.661+0000 7f52b880d700 1 -- 172.21.15.17:0/3573246664 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52b40b1460 msgr2=0x7f52b40b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:39.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.661+0000 7f52b880d700 1 --2- 172.21.15.17:0/3573246664 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52b40b1460 0x7f52b40b3e20 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 rx=0x7f52b40afe10 tx=0x7f52a4007c10).stop 2022-08-01T19:43:39.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.661+0000 7f52b880d700 1 -- 172.21.15.17:0/3573246664 shutdown_connections 2022-08-01T19:43:39.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.661+0000 7f52b880d700 1 --2- 172.21.15.17:0/3573246664 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f52980539e0 0x7f5298055ea0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:39.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.661+0000 7f52b880d700 1 --2- 172.21.15.17:0/3573246664 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f52b40b2430 0x7f52b4137590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:39.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.661+0000 7f52b880d700 1 --2- 172.21.15.17:0/3573246664 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52b40b2050 0x7f52b4149270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:39.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.661+0000 7f52b880d700 1 --2- 172.21.15.17:0/3573246664 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52b40b1460 0x7f52b40b3e20 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:39.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.661+0000 7f52b880d700 1 -- 172.21.15.17:0/3573246664 >> 172.21.15.17:0/3573246664 conn(0x7f52b405f840 msgr2=0x7f52b413b000 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:39.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.661+0000 7f52b880d700 1 -- 172.21.15.17:0/3573246664 shutdown_connections 2022-08-01T19:43:39.666 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.661+0000 7f52b880d700 1 -- 172.21.15.17:0/3573246664 wait complete. 2022-08-01T19:43:39.675 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T19:43:39.676 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:43:39.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.777+0000 7f1cde663700 1 Processor -- start 2022-08-01T19:43:39.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.777+0000 7f1cde663700 1 -- start start 2022-08-01T19:43:39.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.777+0000 7f1cde663700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1cd80b6710 0x7f1cd80b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:39.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.777+0000 7f1cde663700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f1cd80b4550 0x7f1cd80b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:39.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.777+0000 7f1cde663700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cd8144240 con 0x7f1cd8061ec0 2022-08-01T19:43:39.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.777+0000 7f1cde663700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cd80b6fe0 con 0x7f1cd80b4550 2022-08-01T19:43:39.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.777+0000 7f1cde663700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd8143d00 con 0x7f1cd80b6710 2022-08-01T19:43:39.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.777+0000 7f1cd77fe700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f1cd80b4550 0x7f1cd80b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:39.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.777+0000 7f1cdcc00700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7f1cd8061ec0 0x7f1cd8146850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:54106/0 (socket says 172.21.15.17:54106) 2022-08-01T19:43:39.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.777+0000 7f1cd77fe700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f1cd80b4550 0x7f1cd80b6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:43030/0 (socket says 172.21.15.17:43030) 2022-08-01T19:43:39.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.777+0000 7f1cdcc00700 1 -- 172.21.15.17:0/2458587743 learned_addr learned my addr 172.21.15.17:0/2458587743 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:39.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.777+0000 7f1cd7fff700 1 --2- 172.21.15.17:0/2458587743 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1cd80b6710 0x7f1cd80b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:39.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.777+0000 7f1cd6ffd700 1 -- 172.21.15.17:0/2458587743 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4274654754 0 0) 0x7f1cd8144240 con 0x7f1cd8061ec0 2022-08-01T19:43:39.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.777+0000 7f1cd6ffd700 1 -- 172.21.15.17:0/2458587743 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cb8003260 con 0x7f1cd8061ec0 2022-08-01T19:43:39.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.777+0000 7f1cd77fe700 1 -- 172.21.15.17:0/2458587743 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1cd80b6710 msgr2=0x7f1cd80b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:39.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.777+0000 7f1cd77fe700 1 --2- 172.21.15.17:0/2458587743 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1cd80b6710 0x7f1cd80b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:39.802 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.777+0000 7f1cd77fe700 1 -- 172.21.15.17:0/2458587743 >> v1:172.21.15.77:6789/0 conn(0x7f1cd8061ec0 legacy=0x7f1cd8146850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:39.802 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.777+0000 7f1cd77fe700 1 -- 172.21.15.17:0/2458587743 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd8143fa0 con 0x7f1cd80b4550 2022-08-01T19:43:39.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.777+0000 7f1cd7fff700 1 --2- 172.21.15.17:0/2458587743 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1cd80b6710 0x7f1cd80b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:43:39.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.777+0000 7f1cd77fe700 1 --2- 172.21.15.17:0/2458587743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1cd80b4550 0x7f1cd80b6af0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f1cc00032f0 tx=0x7f1cc00091d0).ready entity=mon.1 client_cookie=69e5e95cc2e17f1b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:39.804 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.777+0000 7f1cd6ffd700 1 -- 172.21.15.17:0/2458587743 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cc000b600 con 0x7f1cd80b4550 2022-08-01T19:43:39.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.781+0000 7f1cd6ffd700 1 -- 172.21.15.17:0/2458587743 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc000b760 con 0x7f1cd80b4550 2022-08-01T19:43:39.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.781+0000 7f1cd6ffd700 1 -- 172.21.15.17:0/2458587743 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cc000b8d0 con 0x7f1cd80b4550 2022-08-01T19:43:39.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.781+0000 7f1cde663700 1 -- 172.21.15.17:0/2458587743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1cd80b4550 msgr2=0x7f1cd80b6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:39.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.781+0000 7f1cde663700 1 --2- 172.21.15.17:0/2458587743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1cd80b4550 0x7f1cd80b6af0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f1cc00032f0 tx=0x7f1cc00091d0).stop 2022-08-01T19:43:39.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.781+0000 7f1cde663700 1 -- 172.21.15.17:0/2458587743 shutdown_connections 2022-08-01T19:43:39.807 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.781+0000 7f1cde663700 1 --2- 172.21.15.17:0/2458587743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1cd80b4550 0x7f1cd80b6af0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:39.807 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.781+0000 7f1cde663700 1 --2- 172.21.15.17:0/2458587743 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1cd80b6710 0x7f1cd80b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:39.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.781+0000 7f1cde663700 1 -- 172.21.15.17:0/2458587743 >> 172.21.15.17:0/2458587743 conn(0x7f1cd805c620 msgr2=0x7f1cd805ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:39.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.781+0000 7f1cde663700 1 -- 172.21.15.17:0/2458587743 shutdown_connections 2022-08-01T19:43:39.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.781+0000 7f1cde663700 1 -- 172.21.15.17:0/2458587743 wait complete. 2022-08-01T19:43:39.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.781+0000 7f1cde663700 1 Processor -- start 2022-08-01T19:43:39.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.781+0000 7f1cde663700 1 -- start start 2022-08-01T19:43:39.810 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.781+0000 7f1cde663700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1cd8061ec0 0x7f1cd814a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:39.811 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.781+0000 7f1cde663700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1cd80b4550 0x7f1cd813d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:39.811 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.781+0000 7f1cde663700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1cd80b6710 0x7f1cd8127090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:39.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.781+0000 7f1cde663700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd8147b40 con 0x7f1cd80b4550 2022-08-01T19:43:39.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.781+0000 7f1cde663700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd813df80 con 0x7f1cd80b6710 2022-08-01T19:43:39.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.781+0000 7f1cde663700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd813b850 con 0x7f1cd8061ec0 2022-08-01T19:43:39.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.781+0000 7f1cd77fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1cd80b4550 0x7f1cd813d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:39.815 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.781+0000 7f1cd77fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1cd80b4550 0x7f1cd813d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45116/0 (socket says 172.21.15.17:45116) 2022-08-01T19:43:39.815 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.781+0000 7f1cd77fe700 1 -- 172.21.15.17:0/3017294616 learned_addr learned my addr 172.21.15.17:0/3017294616 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:39.816 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.781+0000 7f1cdcc00700 1 --2- 172.21.15.17:0/3017294616 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1cd80b6710 0x7f1cd8127090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:39.816 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.781+0000 7f1cd7fff700 1 --2- 172.21.15.17:0/3017294616 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1cd8061ec0 0x7f1cd814a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:39.817 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.785+0000 7f1cd77fe700 1 -- 172.21.15.17:0/3017294616 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1cd8061ec0 msgr2=0x7f1cd814a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:39.817 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.785+0000 7f1cd77fe700 1 --2- 172.21.15.17:0/3017294616 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1cd8061ec0 0x7f1cd814a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:39.818 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.785+0000 7f1cd77fe700 1 -- 172.21.15.17:0/3017294616 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1cd80b6710 msgr2=0x7f1cd8127090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:39.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.785+0000 7f1cd77fe700 1 --2- 172.21.15.17:0/3017294616 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1cd80b6710 0x7f1cd8127090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:39.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.785+0000 7f1cd77fe700 1 -- 172.21.15.17:0/3017294616 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc800b180 con 0x7f1cd80b4550 2022-08-01T19:43:39.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.785+0000 7f1cd77fe700 1 --2- 172.21.15.17:0/3017294616 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1cd80b4550 0x7f1cd813d260 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0x7f1cc00032f0 tx=0x7f1cc0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:39.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.785+0000 7f1cd57fa700 1 -- 172.21.15.17:0/3017294616 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cc000b8d0 con 0x7f1cd80b4550 2022-08-01T19:43:39.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.785+0000 7f1cd57fa700 1 -- 172.21.15.17:0/3017294616 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc0021630 con 0x7f1cd80b4550 2022-08-01T19:43:39.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.785+0000 7f1cde663700 1 -- 172.21.15.17:0/3017294616 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cc0009470 con 0x7f1cd80b4550 2022-08-01T19:43:39.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.785+0000 7f1cd57fa700 1 -- 172.21.15.17:0/3017294616 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cc000b600 con 0x7f1cd80b4550 2022-08-01T19:43:39.822 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.785+0000 7f1cde663700 1 -- 172.21.15.17:0/3017294616 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd81478a0 con 0x7f1cd80b4550 2022-08-01T19:43:39.822 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.785+0000 7f1cd57fa700 1 -- 172.21.15.17:0/3017294616 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1cc0027020 con 0x7f1cd80b4550 2022-08-01T19:43:39.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.785+0000 7f1cde663700 1 -- 172.21.15.17:0/3017294616 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cd8147470 con 0x7f1cd80b4550 2022-08-01T19:43:39.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.785+0000 7f1cd57fa700 1 --2- 172.21.15.17:0/3017294616 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1cbc053a70 0x7f1cbc055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:39.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.785+0000 7f1cd57fa700 1 -- 172.21.15.17:0/3017294616 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1cc006d4e0 con 0x7f1cd80b4550 2022-08-01T19:43:39.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.785+0000 7f1cd7fff700 1 --2- 172.21.15.17:0/3017294616 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1cbc053a70 0x7f1cbc055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:39.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.789+0000 7f1cd7fff700 1 --2- 172.21.15.17:0/3017294616 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1cbc053a70 0x7f1cbc055f30 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7f1cc800b000 tx=0x7f1cc8006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:39.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.793+0000 7f1cd57fa700 1 -- 172.21.15.17:0/3017294616 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1cc00442a0 con 0x7f1cd80b4550 2022-08-01T19:43:39.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.953+0000 7f1cde663700 1 -- 172.21.15.17:0/3017294616 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1cd8146920 con 0x7f1cd80b4550 2022-08-01T19:43:39.957 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.953+0000 7f1cd57fa700 1 -- 172.21.15.17:0/3017294616 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f1cc0015090 con 0x7f1cd80b4550 2022-08-01T19:43:39.958 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:39.959 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":76,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7211,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/872530952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":872530952},{"type":"v1","addr":"172.21.15.77:6835","nonce":872530952}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":74},{"gid":8358,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/4261332014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4261332014},{"type":"v1","addr":"172.21.15.17:6833","nonce":4261332014}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":73}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:43:34.586097+0000","modified":"2022-08-01T19:43:36.271865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8364},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8364":{"gid":8364,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/2832743740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2832743740},{"type":"v1","addr":"172.21.15.17:6835","nonce":2832743740}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2022-08-01T19:43:39.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.957+0000 7f1cde663700 1 -- 172.21.15.17:0/3017294616 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1cbc053a70 msgr2=0x7f1cbc055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:39.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.957+0000 7f1cde663700 1 --2- 172.21.15.17:0/3017294616 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1cbc053a70 0x7f1cbc055f30 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7f1cc800b000 tx=0x7f1cc8006df0).stop 2022-08-01T19:43:39.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.957+0000 7f1cde663700 1 -- 172.21.15.17:0/3017294616 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1cd80b4550 msgr2=0x7f1cd813d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:39.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.957+0000 7f1cde663700 1 --2- 172.21.15.17:0/3017294616 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1cd80b4550 0x7f1cd813d260 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0x7f1cc00032f0 tx=0x7f1cc0016000).stop 2022-08-01T19:43:39.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.957+0000 7f1cde663700 1 -- 172.21.15.17:0/3017294616 shutdown_connections 2022-08-01T19:43:39.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.957+0000 7f1cde663700 1 --2- 172.21.15.17:0/3017294616 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1cbc053a70 0x7f1cbc055f30 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:39.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.957+0000 7f1cde663700 1 --2- 172.21.15.17:0/3017294616 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1cd80b6710 0x7f1cd8127090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:39.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.957+0000 7f1cde663700 1 --2- 172.21.15.17:0/3017294616 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1cd80b4550 0x7f1cd813d260 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:39.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.957+0000 7f1cde663700 1 --2- 172.21.15.17:0/3017294616 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1cd8061ec0 0x7f1cd814a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:39.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.957+0000 7f1cde663700 1 -- 172.21.15.17:0/3017294616 >> 172.21.15.17:0/3017294616 conn(0x7f1cd805c620 msgr2=0x7f1cd8148950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:39.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.957+0000 7f1cde663700 1 -- 172.21.15.17:0/3017294616 shutdown_connections 2022-08-01T19:43:39.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:39.957+0000 7f1cde663700 1 -- 172.21.15.17:0/3017294616 wait complete. 2022-08-01T19:43:39.964 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 76 2022-08-01T19:43:40.294 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 76, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:43:34.586097+0000', 'modified': '2022-08-01T19:43:36.271865+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8364}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8364': {'gid': 8364, 'name': 'c', 'rank': 0, 'incarnation': 74, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/2832743740', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 2832743740}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 2832743740}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:43:40.295 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:43:40.296 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T19:43:40.428 INFO:teuthology.orchestra.run.smithi017.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":8364,"want_state":"up:active","state":"up:active","rank_uptime":5.8148315960000003,"mdsmap_epoch":76,"osdmap_epoch":148,"osdmap_epoch_barrier":148,"uptime":10.051693779000001} 2022-08-01T19:43:40.429 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 8364, "mdsmap_epoch": 76, "osdmap_epoch": 148, "osdmap_epoch_barrier": 148, "rank_uptime": 5.814831596, "state": "up:active", "uptime": 10.051693779, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:43:40.446 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T19:43:40.447 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-08-01T19:43:40.447 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi085.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-08-01T19:43:40.448 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:40.449 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:43:40.470 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:43:40.536 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:43:40.537 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:43:40.551 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:43:40.553 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:43:40.564 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:40.564 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:43:40.635 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T19:43:40.636 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:43:40.645 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:43:40.667 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:43:40.667 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:43:40.722 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:2022-08-01T19:43:40.713+0000 7ff1c45c8080 -1 init, newargv = 0x55ee5d58f850 newargc=9ceph-fuse[ 2022-08-01T19:43:40.723 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:23661]: starting ceph client 2022-08-01T19:43:40.735 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:43:40.737 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:43:40.737 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:40.739 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:43:40.739 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:ceph-fuse[23661]: starting fuse 2022-08-01T19:43:40.756 INFO:teuthology.orchestra.run.smithi085.stdout:52 2022-08-01T19:43:40.756 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T19:43:40.757 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:40.757 DEBUG:teuthology.orchestra.run.smithi085:> sudo python3 -c ' 2022-08-01T19:43:40.758 DEBUG:teuthology.orchestra.run.smithi085:> import glob 2022-08-01T19:43:40.767 DEBUG:teuthology.orchestra.run.smithi085:> import re 2022-08-01T19:43:40.767 DEBUG:teuthology.orchestra.run.smithi085:> import os 2022-08-01T19:43:40.767 DEBUG:teuthology.orchestra.run.smithi085:> import subprocess 2022-08-01T19:43:40.768 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:43:40.768 DEBUG:teuthology.orchestra.run.smithi085:> def find_socket(client_name): 2022-08-01T19:43:40.769 DEBUG:teuthology.orchestra.run.smithi085:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-08-01T19:43:40.769 DEBUG:teuthology.orchestra.run.smithi085:> files = glob.glob(asok_path) 2022-08-01T19:43:40.769 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:43:40.770 DEBUG:teuthology.orchestra.run.smithi085:> # Given a non-glob path, it better be there 2022-08-01T19:43:40.770 DEBUG:teuthology.orchestra.run.smithi085:> if "*" not in asok_path: 2022-08-01T19:43:40.771 DEBUG:teuthology.orchestra.run.smithi085:> assert(len(files) == 1) 2022-08-01T19:43:40.771 DEBUG:teuthology.orchestra.run.smithi085:> return files[0] 2022-08-01T19:43:40.772 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:43:40.772 DEBUG:teuthology.orchestra.run.smithi085:> for f in files: 2022-08-01T19:43:40.772 DEBUG:teuthology.orchestra.run.smithi085:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T19:43:40.773 DEBUG:teuthology.orchestra.run.smithi085:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T19:43:40.774 DEBUG:teuthology.orchestra.run.smithi085:> return f 2022-08-01T19:43:40.775 DEBUG:teuthology.orchestra.run.smithi085:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T19:43:40.775 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:43:40.778 DEBUG:teuthology.orchestra.run.smithi085:> print(find_socket("client.0")) 2022-08-01T19:43:40.778 DEBUG:teuthology.orchestra.run.smithi085:> ' 2022-08-01T19:43:40.848 INFO:teuthology.orchestra.run.smithi085.stdout:/var/run/ceph/ceph-client.0.23661.asok 2022-08-01T19:43:40.854 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.23661.asok 2022-08-01T19:43:40.855 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:40.856 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.23661.asok status 2022-08-01T19:43:40.950 INFO:teuthology.orchestra.run.smithi085.stdout:{ 2022-08-01T19:43:40.951 INFO:teuthology.orchestra.run.smithi085.stdout: "metadata": { 2022-08-01T19:43:40.954 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T19:43:40.954 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T19:43:40.955 INFO:teuthology.orchestra.run.smithi085.stdout: "entity_id": "0", 2022-08-01T19:43:40.955 INFO:teuthology.orchestra.run.smithi085.stdout: "hostname": "smithi085", 2022-08-01T19:43:40.955 INFO:teuthology.orchestra.run.smithi085.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-08-01T19:43:40.956 INFO:teuthology.orchestra.run.smithi085.stdout: "pid": "23661", 2022-08-01T19:43:40.956 INFO:teuthology.orchestra.run.smithi085.stdout: "root": "/" 2022-08-01T19:43:40.956 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:43:40.956 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_count": 0, 2022-08-01T19:43:40.957 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_pinned_count": 0, 2022-08-01T19:43:40.957 INFO:teuthology.orchestra.run.smithi085.stdout: "id": 8515, 2022-08-01T19:43:40.957 INFO:teuthology.orchestra.run.smithi085.stdout: "inst": { 2022-08-01T19:43:40.957 INFO:teuthology.orchestra.run.smithi085.stdout: "name": { 2022-08-01T19:43:40.958 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "client", 2022-08-01T19:43:40.958 INFO:teuthology.orchestra.run.smithi085.stdout: "num": 8515 2022-08-01T19:43:40.958 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:43:40.959 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2022-08-01T19:43:40.959 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2022-08-01T19:43:40.959 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2022-08-01T19:43:40.960 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 1970117727 2022-08-01T19:43:40.960 INFO:teuthology.orchestra.run.smithi085.stdout: } 2022-08-01T19:43:40.960 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:43:40.960 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2022-08-01T19:43:40.961 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2022-08-01T19:43:40.962 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2022-08-01T19:43:40.973 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 1970117727 2022-08-01T19:43:40.973 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:43:40.974 INFO:teuthology.orchestra.run.smithi085.stdout: "inst_str": "client.8515 172.21.15.85:0/1970117727", 2022-08-01T19:43:40.974 INFO:teuthology.orchestra.run.smithi085.stdout: "addr_str": "172.21.15.85:0/1970117727", 2022-08-01T19:43:40.975 INFO:teuthology.orchestra.run.smithi085.stdout: "inode_count": 1, 2022-08-01T19:43:40.975 INFO:teuthology.orchestra.run.smithi085.stdout: "mds_epoch": 76, 2022-08-01T19:43:40.975 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch": 148, 2022-08-01T19:43:40.976 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch_barrier": 0, 2022-08-01T19:43:40.976 INFO:teuthology.orchestra.run.smithi085.stdout: "blacklisted": false, 2022-08-01T19:43:40.977 INFO:teuthology.orchestra.run.smithi085.stdout: "fs_name": "cephfs" 2022-08-01T19:43:40.977 INFO:teuthology.orchestra.run.smithi085.stdout:} 2022-08-01T19:43:40.979 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:40.980 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:43:40.981 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:43:40.997 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:43:40.998 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:43:40.999 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:41.000 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:43:41.020 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:43:41.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.129+0000 7effbf59e700 1 Processor -- start 2022-08-01T19:43:41.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.129+0000 7effbf59e700 1 -- start start 2022-08-01T19:43:41.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.129+0000 7effbf59e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7effc00647d0 0x7effc0064bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:41.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.129+0000 7effbf59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7effc00b38a0 0x7effc01470d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:41.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.129+0000 7effbf59e700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effc00b4f60 con 0x7effc01436f0 2022-08-01T19:43:41.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.129+0000 7effbf59e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effc01475c0 con 0x7effc00647d0 2022-08-01T19:43:41.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.129+0000 7effbf59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc00b4a60 con 0x7effc00b38a0 2022-08-01T19:43:41.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.129+0000 7effbed9d700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7effc01436f0 0x7effc01403f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37320/0 (socket says 172.21.15.17:37320) 2022-08-01T19:43:41.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.129+0000 7effbed9d700 1 -- 172.21.15.17:0/85833258 learned_addr learned my addr 172.21.15.17:0/85833258 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:41.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.129+0000 7effbe59c700 1 --2- 172.21.15.17:0/85833258 >> v2:172.21.15.17:3300/0 conn(0x7effc00647d0 0x7effc0064bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:41.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.129+0000 7effbdd9b700 1 --2- 172.21.15.17:0/85833258 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7effc00b38a0 0x7effc01470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:41.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.133+0000 7effbd59a700 1 -- 172.21.15.17:0/85833258 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2326147454 0 0) 0x7effc00b4f60 con 0x7effc01436f0 2022-08-01T19:43:41.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.133+0000 7effbd59a700 1 -- 172.21.15.17:0/85833258 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effa4003260 con 0x7effc01436f0 2022-08-01T19:43:41.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.133+0000 7effbdd9b700 1 -- 172.21.15.17:0/85833258 >> v2:172.21.15.17:3300/0 conn(0x7effc00647d0 msgr2=0x7effc0064bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:41.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.133+0000 7effbdd9b700 1 --2- 172.21.15.17:0/85833258 >> v2:172.21.15.17:3300/0 conn(0x7effc00647d0 0x7effc0064bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:41.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.133+0000 7effbdd9b700 1 -- 172.21.15.17:0/85833258 >> v1:172.21.15.17:6789/0 conn(0x7effc01436f0 legacy=0x7effc01403f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:41.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.133+0000 7effbdd9b700 1 -- 172.21.15.17:0/85833258 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc00b4cc0 con 0x7effc00b38a0 2022-08-01T19:43:41.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.133+0000 7effbdd9b700 1 --2- 172.21.15.17:0/85833258 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7effc00b38a0 0x7effc01470d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7effb0002f70 tx=0x7effb0006df0).ready entity=mon.2 client_cookie=559b2fbf284be89e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:41.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.133+0000 7effbd59a700 1 -- 172.21.15.17:0/85833258 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effb000d4a0 con 0x7effc00b38a0 2022-08-01T19:43:41.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.133+0000 7effbd59a700 1 -- 172.21.15.17:0/85833258 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb0000e90 con 0x7effc00b38a0 2022-08-01T19:43:41.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.133+0000 7effbd59a700 1 -- 172.21.15.17:0/85833258 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effb000d7f0 con 0x7effc00b38a0 2022-08-01T19:43:41.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.133+0000 7effbf59e700 1 -- 172.21.15.17:0/85833258 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7effc00b38a0 msgr2=0x7effc01470d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:41.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.133+0000 7effbf59e700 1 --2- 172.21.15.17:0/85833258 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7effc00b38a0 0x7effc01470d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7effb0002f70 tx=0x7effb0006df0).stop 2022-08-01T19:43:41.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.133+0000 7effbf59e700 1 -- 172.21.15.17:0/85833258 shutdown_connections 2022-08-01T19:43:41.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.133+0000 7effbf59e700 1 --2- 172.21.15.17:0/85833258 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7effc00b38a0 0x7effc01470d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:41.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.133+0000 7effbf59e700 1 --2- 172.21.15.17:0/85833258 >> v2:172.21.15.17:3300/0 conn(0x7effc00647d0 0x7effc0064bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:41.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.133+0000 7effbf59e700 1 -- 172.21.15.17:0/85833258 >> 172.21.15.17:0/85833258 conn(0x7effc005f840 msgr2=0x7effc005fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:41.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.133+0000 7effbf59e700 1 -- 172.21.15.17:0/85833258 shutdown_connections 2022-08-01T19:43:41.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.133+0000 7effbf59e700 1 -- 172.21.15.17:0/85833258 wait complete. 2022-08-01T19:43:41.169 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.137+0000 7effbf59e700 1 Processor -- start 2022-08-01T19:43:41.171 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.137+0000 7effbf59e700 1 -- start start 2022-08-01T19:43:41.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.137+0000 7effbf59e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effc00647d0 0x7effc00a3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:41.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.137+0000 7effbf59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7effc00b38a0 0x7effc013f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:41.188 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.137+0000 7effbf59e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7effc01436f0 0x7effc013fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:41.188 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.137+0000 7effbf59e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc0152b00 con 0x7effc00647d0 2022-08-01T19:43:41.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.137+0000 7effbf59e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc0152da0 con 0x7effc01436f0 2022-08-01T19:43:41.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.137+0000 7effbf59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc0153040 con 0x7effc00b38a0 2022-08-01T19:43:41.190 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.137+0000 7effbe59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effc00647d0 0x7effc00a3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:41.190 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.137+0000 7effbe59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effc00647d0 0x7effc00a3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45128/0 (socket says 172.21.15.17:45128) 2022-08-01T19:43:41.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.137+0000 7effbe59c700 1 -- 172.21.15.17:0/3888092355 learned_addr learned my addr 172.21.15.17:0/3888092355 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:41.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.137+0000 7effbdd9b700 1 --2- 172.21.15.17:0/3888092355 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7effc00b38a0 0x7effc013f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:41.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.137+0000 7effbed9d700 1 --2- 172.21.15.17:0/3888092355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7effc01436f0 0x7effc013fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:41.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.141+0000 7effbe59c700 1 -- 172.21.15.17:0/3888092355 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7effc00b38a0 msgr2=0x7effc013f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:41.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.141+0000 7effbe59c700 1 --2- 172.21.15.17:0/3888092355 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7effc00b38a0 0x7effc013f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:41.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.141+0000 7effbe59c700 1 -- 172.21.15.17:0/3888092355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7effc01436f0 msgr2=0x7effc013fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:41.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.141+0000 7effbe59c700 1 --2- 172.21.15.17:0/3888092355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7effc01436f0 0x7effc013fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:41.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.141+0000 7effbe59c700 1 -- 172.21.15.17:0/3888092355 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb000b180 con 0x7effc00647d0 2022-08-01T19:43:41.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.141+0000 7effbed9d700 1 --2- 172.21.15.17:0/3888092355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7effc01436f0 0x7effc013fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:43:41.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.141+0000 7effbe59c700 1 --2- 172.21.15.17:0/3888092355 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effc00647d0 0x7effc00a3190 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 rx=0x7effb400c000 tx=0x7effb4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:41.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.141+0000 7effaf7fe700 1 -- 172.21.15.17:0/3888092355 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effb4010780 con 0x7effc00647d0 2022-08-01T19:43:41.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.141+0000 7effaf7fe700 1 -- 172.21.15.17:0/3888092355 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb40108e0 con 0x7effc00647d0 2022-08-01T19:43:41.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.141+0000 7effaf7fe700 1 -- 172.21.15.17:0/3888092355 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effb4010c40 con 0x7effc00647d0 2022-08-01T19:43:41.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.141+0000 7effbf59e700 1 -- 172.21.15.17:0/3888092355 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effc01532e0 con 0x7effc00647d0 2022-08-01T19:43:41.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.141+0000 7effbf59e700 1 -- 172.21.15.17:0/3888092355 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc00ae010 con 0x7effc00647d0 2022-08-01T19:43:41.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.141+0000 7effbf59e700 1 -- 172.21.15.17:0/3888092355 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effc0143250 con 0x7effc00647d0 2022-08-01T19:43:41.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.141+0000 7effaf7fe700 1 -- 172.21.15.17:0/3888092355 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7effb401c460 con 0x7effc00647d0 2022-08-01T19:43:41.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.141+0000 7effaf7fe700 1 --2- 172.21.15.17:0/3888092355 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7eff9c053950 0x7eff9c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:41.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.141+0000 7effaf7fe700 1 -- 172.21.15.17:0/3888092355 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7effb406c600 con 0x7effc00647d0 2022-08-01T19:43:41.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.141+0000 7effbdd9b700 1 --2- 172.21.15.17:0/3888092355 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7eff9c053950 0x7eff9c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:41.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.145+0000 7effbdd9b700 1 --2- 172.21.15.17:0/3888092355 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7eff9c053950 0x7eff9c055e10 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7effb0018000 tx=0x7effb0014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:41.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.149+0000 7effaf7fe700 1 -- 172.21.15.17:0/3888092355 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7effb40432c0 con 0x7effc00647d0 2022-08-01T19:43:41.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.273+0000 7effbf59e700 1 -- 172.21.15.17:0/3888092355 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7effc0143fe0 con 0x7effc00647d0 2022-08-01T19:43:41.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.277+0000 7effaf7fe700 1 -- 172.21.15.17:0/3888092355 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+16285 (secure 0 0 0) 0x7effb40468e0 con 0x7effc00647d0 2022-08-01T19:43:41.280 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:41.281 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":148,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:43:35.589613+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:43:33.209247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2022-08-01T19:43:33.595201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:43:41.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.277+0000 7effbf59e700 1 -- 172.21.15.17:0/3888092355 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7eff9c053950 msgr2=0x7eff9c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:41.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.277+0000 7effbf59e700 1 --2- 172.21.15.17:0/3888092355 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7eff9c053950 0x7eff9c055e10 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7effb0018000 tx=0x7effb0014000).stop 2022-08-01T19:43:41.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.277+0000 7effbf59e700 1 -- 172.21.15.17:0/3888092355 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effc00647d0 msgr2=0x7effc00a3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:41.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.277+0000 7effbf59e700 1 --2- 172.21.15.17:0/3888092355 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effc00647d0 0x7effc00a3190 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 rx=0x7effb400c000 tx=0x7effb4006df0).stop 2022-08-01T19:43:41.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.277+0000 7effbf59e700 1 -- 172.21.15.17:0/3888092355 shutdown_connections 2022-08-01T19:43:41.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.277+0000 7effbf59e700 1 --2- 172.21.15.17:0/3888092355 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7eff9c053950 0x7eff9c055e10 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:41.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.277+0000 7effbf59e700 1 --2- 172.21.15.17:0/3888092355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7effc01436f0 0x7effc013fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:41.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.277+0000 7effbf59e700 1 --2- 172.21.15.17:0/3888092355 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7effc00b38a0 0x7effc013f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:41.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.277+0000 7effbf59e700 1 --2- 172.21.15.17:0/3888092355 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effc00647d0 0x7effc00a3190 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:41.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.277+0000 7effbf59e700 1 -- 172.21.15.17:0/3888092355 >> 172.21.15.17:0/3888092355 conn(0x7effc005f840 msgr2=0x7effc01412e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:41.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.277+0000 7effbf59e700 1 -- 172.21.15.17:0/3888092355 shutdown_connections 2022-08-01T19:43:41.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.277+0000 7effbf59e700 1 -- 172.21.15.17:0/3888092355 wait complete. 2022-08-01T19:43:41.309 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T19:43:33.209247+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '147', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T19:43:33.595201+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '148', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T19:43:41.313 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:43:41.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.425+0000 7f6b7f09d700 1 Processor -- start 2022-08-01T19:43:41.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.425+0000 7f6b7f09d700 1 -- start start 2022-08-01T19:43:41.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.425+0000 7f6b7f09d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6b780ad750 0x7f6b78061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:41.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.425+0000 7f6b7f09d700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f6b781405d0 0x7f6b78140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:41.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.425+0000 7f6b7f09d700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b780aee80 con 0x7f6b780614b0 2022-08-01T19:43:41.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.425+0000 7f6b7f09d700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b780b70b0 con 0x7f6b781405d0 2022-08-01T19:43:41.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.425+0000 7f6b7f09d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b780af0b0 con 0x7f6b780ad750 2022-08-01T19:43:41.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.425+0000 7f6b7e09b700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f6b780614b0 0x7f6b780b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37330/0 (socket says 172.21.15.17:37330) 2022-08-01T19:43:41.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.425+0000 7f6b7e09b700 1 -- 172.21.15.17:0/4008639400 learned_addr learned my addr 172.21.15.17:0/4008639400 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:41.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.425+0000 7f6b7e89c700 1 --2- 172.21.15.17:0/4008639400 >> v2:172.21.15.77:3300/0 conn(0x7f6b781405d0 0x7f6b78140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:41.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.425+0000 7f6b7d89a700 1 --2- 172.21.15.17:0/4008639400 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6b780ad750 0x7f6b78061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:41.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.429+0000 7f6b7d099700 1 -- 172.21.15.17:0/4008639400 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 137173517 0 0) 0x7f6b780aee80 con 0x7f6b780614b0 2022-08-01T19:43:41.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.429+0000 7f6b7d099700 1 -- 172.21.15.17:0/4008639400 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b60003260 con 0x7f6b780614b0 2022-08-01T19:43:41.501 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.429+0000 7f6b7e89c700 1 -- 172.21.15.17:0/4008639400 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6b780ad750 msgr2=0x7f6b78061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:41.501 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.429+0000 7f6b7e89c700 1 --2- 172.21.15.17:0/4008639400 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6b780ad750 0x7f6b78061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:41.502 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.429+0000 7f6b7e89c700 1 -- 172.21.15.17:0/4008639400 >> v1:172.21.15.17:6789/0 conn(0x7f6b780614b0 legacy=0x7f6b780b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:41.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.429+0000 7f6b7e89c700 1 -- 172.21.15.17:0/4008639400 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b780af350 con 0x7f6b781405d0 2022-08-01T19:43:41.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.429+0000 7f6b7d89a700 1 --2- 172.21.15.17:0/4008639400 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6b780ad750 0x7f6b78061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:43:41.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.429+0000 7f6b7e89c700 1 --2- 172.21.15.17:0/4008639400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b781405d0 0x7f6b78140a80 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f6b700032f0 tx=0x7f6b700091d0).ready entity=mon.1 client_cookie=308670d3d299ce5b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:41.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.429+0000 7f6b7d099700 1 -- 172.21.15.17:0/4008639400 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b7000b600 con 0x7f6b781405d0 2022-08-01T19:43:41.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.429+0000 7f6b7d099700 1 -- 172.21.15.17:0/4008639400 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b7000b760 con 0x7f6b781405d0 2022-08-01T19:43:41.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.429+0000 7f6b7d099700 1 -- 172.21.15.17:0/4008639400 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b7000b8d0 con 0x7f6b781405d0 2022-08-01T19:43:41.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.429+0000 7f6b7f09d700 1 -- 172.21.15.17:0/4008639400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b781405d0 msgr2=0x7f6b78140a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:41.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.429+0000 7f6b7f09d700 1 --2- 172.21.15.17:0/4008639400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b781405d0 0x7f6b78140a80 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f6b700032f0 tx=0x7f6b700091d0).stop 2022-08-01T19:43:41.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.429+0000 7f6b7f09d700 1 -- 172.21.15.17:0/4008639400 shutdown_connections 2022-08-01T19:43:41.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.429+0000 7f6b7f09d700 1 --2- 172.21.15.17:0/4008639400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b781405d0 0x7f6b78140a80 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:41.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.429+0000 7f6b7f09d700 1 --2- 172.21.15.17:0/4008639400 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6b780ad750 0x7f6b78061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:41.530 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.429+0000 7f6b7f09d700 1 -- 172.21.15.17:0/4008639400 >> 172.21.15.17:0/4008639400 conn(0x7f6b78066f70 msgr2=0x7f6b78067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:41.531 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.429+0000 7f6b7f09d700 1 -- 172.21.15.17:0/4008639400 shutdown_connections 2022-08-01T19:43:41.531 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.429+0000 7f6b7f09d700 1 -- 172.21.15.17:0/4008639400 wait complete. 2022-08-01T19:43:41.531 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.429+0000 7f6b7f09d700 1 Processor -- start 2022-08-01T19:43:41.531 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.429+0000 7f6b7f09d700 1 -- start start 2022-08-01T19:43:41.532 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.433+0000 7f6b7f09d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6b780614b0 0x7f6b780abfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:41.532 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.433+0000 7f6b7f09d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b780ad750 0x7f6b780b3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:41.532 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.433+0000 7f6b7f09d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b781405d0 0x7f6b780b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:41.532 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.433+0000 7f6b7f09d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b780ab0f0 con 0x7f6b780ad750 2022-08-01T19:43:41.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.433+0000 7f6b7f09d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b780aa320 con 0x7f6b781405d0 2022-08-01T19:43:41.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.433+0000 7f6b7f09d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b780aa5c0 con 0x7f6b780614b0 2022-08-01T19:43:41.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.433+0000 7f6b7d89a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b780ad750 0x7f6b780b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:41.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.433+0000 7f6b7d89a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b780ad750 0x7f6b780b3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45142/0 (socket says 172.21.15.17:45142) 2022-08-01T19:43:41.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.433+0000 7f6b7d89a700 1 -- 172.21.15.17:0/4127612768 learned_addr learned my addr 172.21.15.17:0/4127612768 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:41.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.433+0000 7f6b7e89c700 1 --2- 172.21.15.17:0/4127612768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b781405d0 0x7f6b780b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:41.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.433+0000 7f6b7e09b700 1 --2- 172.21.15.17:0/4127612768 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6b780614b0 0x7f6b780abfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:41.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.433+0000 7f6b7d89a700 1 -- 172.21.15.17:0/4127612768 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6b780614b0 msgr2=0x7f6b780abfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:41.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.433+0000 7f6b7d89a700 1 --2- 172.21.15.17:0/4127612768 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6b780614b0 0x7f6b780abfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:41.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.433+0000 7f6b7d89a700 1 -- 172.21.15.17:0/4127612768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b781405d0 msgr2=0x7f6b780b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:41.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.433+0000 7f6b7d89a700 1 --2- 172.21.15.17:0/4127612768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b781405d0 0x7f6b780b39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:41.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.433+0000 7f6b7d89a700 1 -- 172.21.15.17:0/4127612768 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b7400b180 con 0x7f6b780ad750 2022-08-01T19:43:41.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.433+0000 7f6b7e89c700 1 --2- 172.21.15.17:0/4127612768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b781405d0 0x7f6b780b39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:43:41.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.433+0000 7f6b7e09b700 1 --2- 172.21.15.17:0/4127612768 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6b780614b0 0x7f6b780abfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:43:41.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.433+0000 7f6b7d89a700 1 --2- 172.21.15.17:0/4127612768 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b780ad750 0x7f6b780b3500 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 rx=0x7f6b7400c000 tx=0x7f6b74006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:41.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.433+0000 7f6b677fe700 1 -- 172.21.15.17:0/4127612768 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b7400ca40 con 0x7f6b780ad750 2022-08-01T19:43:41.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.433+0000 7f6b677fe700 1 -- 172.21.15.17:0/4127612768 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b7400cba0 con 0x7f6b780ad750 2022-08-01T19:43:41.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.433+0000 7f6b7f09d700 1 -- 172.21.15.17:0/4127612768 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b70009470 con 0x7f6b780ad750 2022-08-01T19:43:41.540 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.433+0000 7f6b677fe700 1 -- 172.21.15.17:0/4127612768 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b740123f0 con 0x7f6b780ad750 2022-08-01T19:43:41.540 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.433+0000 7f6b7f09d700 1 -- 172.21.15.17:0/4127612768 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b7805d230 con 0x7f6b780ad750 2022-08-01T19:43:41.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.433+0000 7f6b7f09d700 1 -- 172.21.15.17:0/4127612768 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b780ab2f0 con 0x7f6b780ad750 2022-08-01T19:43:41.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.433+0000 7f6b677fe700 1 -- 172.21.15.17:0/4127612768 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6b74012550 con 0x7f6b780ad750 2022-08-01T19:43:41.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.437+0000 7f6b677fe700 1 --2- 172.21.15.17:0/4127612768 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6b58053980 0x7f6b58055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:41.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.437+0000 7f6b677fe700 1 -- 172.21.15.17:0/4127612768 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6b7406c700 con 0x7f6b780ad750 2022-08-01T19:43:41.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.437+0000 7f6b7e09b700 1 --2- 172.21.15.17:0/4127612768 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6b58053980 0x7f6b58055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:41.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.437+0000 7f6b7e09b700 1 --2- 172.21.15.17:0/4127612768 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6b58053980 0x7f6b58055e40 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7f6b68008000 tx=0x7f6b68009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:41.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.441+0000 7f6b677fe700 1 -- 172.21.15.17:0/4127612768 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6b74043460 con 0x7f6b780ad750 2022-08-01T19:43:41.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.565+0000 7f6b7f09d700 1 -- 172.21.15.17:0/4127612768 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f6b780aa780 con 0x7f6b780ad750 2022-08-01T19:43:41.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.565+0000 7f6b677fe700 1 -- 172.21.15.17:0/4127612768 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v148) v1 ==== 110+0+10 (secure 0 0 0) 0x7f6b74046a80 con 0x7f6b780ad750 2022-08-01T19:43:41.581 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:43:41.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.569+0000 7f6b7f09d700 1 -- 172.21.15.17:0/4127612768 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6b58053980 msgr2=0x7f6b58055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:41.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.569+0000 7f6b7f09d700 1 --2- 172.21.15.17:0/4127612768 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6b58053980 0x7f6b58055e40 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7f6b68008000 tx=0x7f6b68009040).stop 2022-08-01T19:43:41.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.569+0000 7f6b7f09d700 1 -- 172.21.15.17:0/4127612768 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b780ad750 msgr2=0x7f6b780b3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:41.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.569+0000 7f6b7f09d700 1 --2- 172.21.15.17:0/4127612768 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b780ad750 0x7f6b780b3500 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 rx=0x7f6b7400c000 tx=0x7f6b74006df0).stop 2022-08-01T19:43:41.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.569+0000 7f6b7f09d700 1 -- 172.21.15.17:0/4127612768 shutdown_connections 2022-08-01T19:43:41.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.569+0000 7f6b7f09d700 1 --2- 172.21.15.17:0/4127612768 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6b58053980 0x7f6b58055e40 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:41.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.569+0000 7f6b7f09d700 1 --2- 172.21.15.17:0/4127612768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b781405d0 0x7f6b780b39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:41.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.569+0000 7f6b7f09d700 1 --2- 172.21.15.17:0/4127612768 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b780ad750 0x7f6b780b3500 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:41.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.569+0000 7f6b7f09d700 1 --2- 172.21.15.17:0/4127612768 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6b780614b0 0x7f6b780abfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:41.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.569+0000 7f6b7f09d700 1 -- 172.21.15.17:0/4127612768 >> 172.21.15.17:0/4127612768 conn(0x7f6b78066f70 msgr2=0x7f6b780b88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:41.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.569+0000 7f6b7f09d700 1 -- 172.21.15.17:0/4127612768 shutdown_connections 2022-08-01T19:43:41.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.569+0000 7f6b7f09d700 1 -- 172.21.15.17:0/4127612768 wait complete. 2022-08-01T19:43:41.611 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:43:41.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.729+0000 7f2750ac5700 1 Processor -- start 2022-08-01T19:43:41.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.729+0000 7f2750ac5700 1 -- start start 2022-08-01T19:43:41.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.729+0000 7f2750ac5700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f274c0a9da0 0x7f274c060950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:41.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.729+0000 7f2750ac5700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f274c1402d0 0x7f274c140720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:41.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.729+0000 7f2750ac5700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f274c067030 con 0x7f274c0b31d0 2022-08-01T19:43:41.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.729+0000 7f2750ac5700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f274c064a70 con 0x7f274c1402d0 2022-08-01T19:43:41.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.729+0000 7f2750ac5700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f274c061f60 con 0x7f274c0a9da0 2022-08-01T19:43:41.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.729+0000 7f274bfff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f274c1402d0 0x7f274c140720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:41.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.729+0000 7f274bfff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f274c1402d0 0x7f274c140720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45154/0 (socket says 172.21.15.17:45154) 2022-08-01T19:43:41.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.729+0000 7f274bfff700 1 -- 172.21.15.17:0/3165666922 learned_addr learned my addr 172.21.15.17:0/3165666922 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:41.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.729+0000 7f2743fff700 1 --2- 172.21.15.17:0/3165666922 >> v2:172.21.15.77:3300/0 conn(0x7f274c0a9da0 0x7f274c060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:41.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.729+0000 7f274bfff700 1 -- 172.21.15.17:0/3165666922 >> v2:172.21.15.77:3300/0 conn(0x7f274c0a9da0 msgr2=0x7f274c060950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:41.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.729+0000 7f274bfff700 1 --2- 172.21.15.17:0/3165666922 >> v2:172.21.15.77:3300/0 conn(0x7f274c0a9da0 0x7f274c060950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:41.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.729+0000 7f274bfff700 1 -- 172.21.15.17:0/3165666922 >> v1:172.21.15.77:6789/0 conn(0x7f274c0b31d0 legacy=0x7f274c066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:41.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.729+0000 7f274bfff700 1 -- 172.21.15.17:0/3165666922 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f274c0b35b0 con 0x7f274c1402d0 2022-08-01T19:43:41.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.729+0000 7f274bfff700 1 --2- 172.21.15.17:0/3165666922 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f274c1402d0 0x7f274c140720 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 rx=0x7f27440032f0 tx=0x7f2744009290).ready entity=mon.0 client_cookie=60fbed2c7331be71 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:41.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.733+0000 7f274affd700 1 -- 172.21.15.17:0/3165666922 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f274400b870 con 0x7f274c1402d0 2022-08-01T19:43:41.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.733+0000 7f274affd700 1 -- 172.21.15.17:0/3165666922 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f274400b9d0 con 0x7f274c1402d0 2022-08-01T19:43:41.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.733+0000 7f274affd700 1 -- 172.21.15.17:0/3165666922 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f274400bb40 con 0x7f274c1402d0 2022-08-01T19:43:41.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.733+0000 7f2750ac5700 1 -- 172.21.15.17:0/3165666922 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f274c1402d0 msgr2=0x7f274c140720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:41.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.733+0000 7f2750ac5700 1 --2- 172.21.15.17:0/3165666922 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f274c1402d0 0x7f274c140720 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 rx=0x7f27440032f0 tx=0x7f2744009290).stop 2022-08-01T19:43:41.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.733+0000 7f2750ac5700 1 -- 172.21.15.17:0/3165666922 shutdown_connections 2022-08-01T19:43:41.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.733+0000 7f2750ac5700 1 --2- 172.21.15.17:0/3165666922 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f274c1402d0 0x7f274c140720 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:41.773 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.733+0000 7f2750ac5700 1 --2- 172.21.15.17:0/3165666922 >> v2:172.21.15.77:3300/0 conn(0x7f274c0a9da0 0x7f274c060950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:41.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.733+0000 7f2750ac5700 1 -- 172.21.15.17:0/3165666922 >> 172.21.15.17:0/3165666922 conn(0x7f274c063870 msgr2=0x7f274c063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:41.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.733+0000 7f2750ac5700 1 -- 172.21.15.17:0/3165666922 shutdown_connections 2022-08-01T19:43:41.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.733+0000 7f2750ac5700 1 -- 172.21.15.17:0/3165666922 wait complete. 2022-08-01T19:43:41.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.737+0000 7f2750ac5700 1 Processor -- start 2022-08-01T19:43:41.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.737+0000 7f2750ac5700 1 -- start start 2022-08-01T19:43:41.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.737+0000 7f2750ac5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f274c0a9da0 0x7f274c143730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:41.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.737+0000 7f2750ac5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f274c0b31d0 0x7f274c0aff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:41.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.737+0000 7f2750ac5700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f274c1402d0 0x7f274c1526c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:41.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.737+0000 7f2750ac5700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274c09ec10 con 0x7f274c0a9da0 2022-08-01T19:43:41.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.737+0000 7f2750ac5700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274c0ae2c0 con 0x7f274c0b31d0 2022-08-01T19:43:41.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.737+0000 7f2750ac5700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f274c0ac980 con 0x7f274c1402d0 2022-08-01T19:43:41.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.737+0000 7f274b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f274c0a9da0 0x7f274c143730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:41.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.737+0000 7f274b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f274c0a9da0 0x7f274c143730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45156/0 (socket says 172.21.15.17:45156) 2022-08-01T19:43:41.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.737+0000 7f274b7fe700 1 -- 172.21.15.17:0/581392266 learned_addr learned my addr 172.21.15.17:0/581392266 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:41.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.737+0000 7f2743fff700 1 --2- 172.21.15.17:0/581392266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f274c0b31d0 0x7f274c0aff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:41.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.737+0000 7f274bfff700 1 --2- 172.21.15.17:0/581392266 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f274c1402d0 0x7f274c1526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:41.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.737+0000 7f274b7fe700 1 -- 172.21.15.17:0/581392266 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f274c1402d0 msgr2=0x7f274c1526c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:41.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.737+0000 7f274b7fe700 1 --2- 172.21.15.17:0/581392266 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f274c1402d0 0x7f274c1526c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:41.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.737+0000 7f274b7fe700 1 -- 172.21.15.17:0/581392266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f274c0b31d0 msgr2=0x7f274c0aff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:41.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.737+0000 7f274b7fe700 1 --2- 172.21.15.17:0/581392266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f274c0b31d0 0x7f274c0aff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:41.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.737+0000 7f274b7fe700 1 -- 172.21.15.17:0/581392266 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2744009530 con 0x7f274c0a9da0 2022-08-01T19:43:41.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.737+0000 7f274b7fe700 1 --2- 172.21.15.17:0/581392266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f274c0a9da0 0x7f274c143730 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 rx=0x7f274c0afb90 tx=0x7f273c007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:41.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.737+0000 7f27497fa700 1 -- 172.21.15.17:0/581392266 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f273c007030 con 0x7f274c0a9da0 2022-08-01T19:43:41.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.737+0000 7f27497fa700 1 -- 172.21.15.17:0/581392266 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f273c007190 con 0x7f274c0a9da0 2022-08-01T19:43:41.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.737+0000 7f2750ac5700 1 -- 172.21.15.17:0/581392266 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f274c0acdb0 con 0x7f274c0a9da0 2022-08-01T19:43:41.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.737+0000 7f27497fa700 1 -- 172.21.15.17:0/581392266 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f273c006510 con 0x7f274c0a9da0 2022-08-01T19:43:41.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.737+0000 7f2750ac5700 1 -- 172.21.15.17:0/581392266 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f274c0ae4f0 con 0x7f274c0a9da0 2022-08-01T19:43:41.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.741+0000 7f2750ac5700 1 -- 172.21.15.17:0/581392266 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f274c0abf70 con 0x7f274c0a9da0 2022-08-01T19:43:41.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.741+0000 7f27497fa700 1 -- 172.21.15.17:0/581392266 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f273c010030 con 0x7f274c0a9da0 2022-08-01T19:43:41.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.741+0000 7f27497fa700 1 --2- 172.21.15.17:0/581392266 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2724053aa0 0x7f2724055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:41.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.741+0000 7f27497fa700 1 -- 172.21.15.17:0/581392266 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f273c06bd40 con 0x7f274c0a9da0 2022-08-01T19:43:41.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.741+0000 7f2743fff700 1 --2- 172.21.15.17:0/581392266 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2724053aa0 0x7f2724055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:41.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.741+0000 7f2743fff700 1 --2- 172.21.15.17:0/581392266 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2724053aa0 0x7f2724055f60 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f273400b000 tx=0x7f2734006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:41.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.749+0000 7f27497fa700 1 -- 172.21.15.17:0/581392266 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f273c042970 con 0x7f274c0a9da0 2022-08-01T19:43:41.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.877+0000 7f2750ac5700 1 -- 172.21.15.17:0/581392266 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f274c0acac0 con 0x7f274c0a9da0 2022-08-01T19:43:41.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.877+0000 7f27497fa700 1 -- 172.21.15.17:0/581392266 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v148) v1 ==== 92+0+10 (secure 0 0 0) 0x7f273c045f90 con 0x7f274c0a9da0 2022-08-01T19:43:41.881 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:43:41.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.881+0000 7f2750ac5700 1 -- 172.21.15.17:0/581392266 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2724053aa0 msgr2=0x7f2724055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:41.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.881+0000 7f2750ac5700 1 --2- 172.21.15.17:0/581392266 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2724053aa0 0x7f2724055f60 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f273400b000 tx=0x7f2734006df0).stop 2022-08-01T19:43:41.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.881+0000 7f2750ac5700 1 -- 172.21.15.17:0/581392266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f274c0a9da0 msgr2=0x7f274c143730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:41.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.881+0000 7f2750ac5700 1 --2- 172.21.15.17:0/581392266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f274c0a9da0 0x7f274c143730 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 rx=0x7f274c0afb90 tx=0x7f273c007c10).stop 2022-08-01T19:43:41.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.881+0000 7f2750ac5700 1 -- 172.21.15.17:0/581392266 shutdown_connections 2022-08-01T19:43:41.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.881+0000 7f2750ac5700 1 --2- 172.21.15.17:0/581392266 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2724053aa0 0x7f2724055f60 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:41.893 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.881+0000 7f2750ac5700 1 --2- 172.21.15.17:0/581392266 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f274c1402d0 0x7f274c1526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:41.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.881+0000 7f2750ac5700 1 --2- 172.21.15.17:0/581392266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f274c0b31d0 0x7f274c0aff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:41.896 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.881+0000 7f2750ac5700 1 --2- 172.21.15.17:0/581392266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f274c0a9da0 0x7f274c143730 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:41.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.881+0000 7f2750ac5700 1 -- 172.21.15.17:0/581392266 >> 172.21.15.17:0/581392266 conn(0x7f274c063870 msgr2=0x7f274c0b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:41.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.881+0000 7f2750ac5700 1 -- 172.21.15.17:0/581392266 shutdown_connections 2022-08-01T19:43:41.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:41.881+0000 7f2750ac5700 1 -- 172.21.15.17:0/581392266 wait complete. 2022-08-01T19:43:41.901 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T19:43:42.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.017+0000 7fe5b3fff700 1 Processor -- start 2022-08-01T19:43:42.030 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.017+0000 7fe5b3fff700 1 -- start start 2022-08-01T19:43:42.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.021+0000 7fe5b3fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe5b4066780 0x7fe5b4066b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:42.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.021+0000 7fe5b3fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe5b40b1bc0 0x7fe5b405ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:42.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.021+0000 7fe5b3fff700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5b405f8e0 con 0x7fe5b4140e40 2022-08-01T19:43:42.031 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.021+0000 7fe5b3fff700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5b40b1630 con 0x7fe5b4066780 2022-08-01T19:43:42.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.021+0000 7fe5b3fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b405fb10 con 0x7fe5b40b1bc0 2022-08-01T19:43:42.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.021+0000 7fe5b37fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe5b4140e40 0x7fe5b413e2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37362/0 (socket says 172.21.15.17:37362) 2022-08-01T19:43:42.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.021+0000 7fe5b2ffd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe5b4066780 0x7fe5b4066b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:42.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.021+0000 7fe5b37fe700 1 -- 172.21.15.17:0/2296668631 learned_addr learned my addr 172.21.15.17:0/2296668631 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:42.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.021+0000 7fe5b27fc700 1 --2- 172.21.15.17:0/2296668631 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe5b40b1bc0 0x7fe5b405ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:42.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.021+0000 7fe5b1ffb700 1 -- 172.21.15.17:0/2296668631 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 597015121 0 0) 0x7fe5b405f8e0 con 0x7fe5b4140e40 2022-08-01T19:43:42.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.021+0000 7fe5b1ffb700 1 -- 172.21.15.17:0/2296668631 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe594003260 con 0x7fe5b4140e40 2022-08-01T19:43:42.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.021+0000 7fe5b2ffd700 1 -- 172.21.15.17:0/2296668631 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe5b40b1bc0 msgr2=0x7fe5b405ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:42.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.021+0000 7fe5b2ffd700 1 --2- 172.21.15.17:0/2296668631 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe5b40b1bc0 0x7fe5b405ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:42.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.021+0000 7fe5b2ffd700 1 -- 172.21.15.17:0/2296668631 >> v1:172.21.15.17:6789/0 conn(0x7fe5b4140e40 legacy=0x7fe5b413e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:42.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.021+0000 7fe5b2ffd700 1 -- 172.21.15.17:0/2296668631 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b40604a0 con 0x7fe5b4066780 2022-08-01T19:43:42.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.021+0000 7fe5b2ffd700 1 --2- 172.21.15.17:0/2296668631 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5b4066780 0x7fe5b4066b60 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 rx=0x7fe5a4002f70 tx=0x7fe5a4006df0).ready entity=mon.0 client_cookie=b70e4fc13cf723b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:42.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.021+0000 7fe5b1ffb700 1 -- 172.21.15.17:0/2296668631 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5a400d4a0 con 0x7fe5b4066780 2022-08-01T19:43:42.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.021+0000 7fe5b1ffb700 1 -- 172.21.15.17:0/2296668631 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a4000e90 con 0x7fe5b4066780 2022-08-01T19:43:42.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.021+0000 7fe5b1ffb700 1 -- 172.21.15.17:0/2296668631 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5a400d7f0 con 0x7fe5b4066780 2022-08-01T19:43:42.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.021+0000 7fe5b3fff700 1 -- 172.21.15.17:0/2296668631 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5b4066780 msgr2=0x7fe5b4066b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:42.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.021+0000 7fe5b3fff700 1 --2- 172.21.15.17:0/2296668631 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5b4066780 0x7fe5b4066b60 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 rx=0x7fe5a4002f70 tx=0x7fe5a4006df0).stop 2022-08-01T19:43:42.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.021+0000 7fe5b3fff700 1 -- 172.21.15.17:0/2296668631 shutdown_connections 2022-08-01T19:43:42.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.021+0000 7fe5b3fff700 1 --2- 172.21.15.17:0/2296668631 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe5b40b1bc0 0x7fe5b405ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:42.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.021+0000 7fe5b3fff700 1 --2- 172.21.15.17:0/2296668631 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5b4066780 0x7fe5b4066b60 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:42.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.021+0000 7fe5b3fff700 1 -- 172.21.15.17:0/2296668631 >> 172.21.15.17:0/2296668631 conn(0x7fe5b4063870 msgr2=0x7fe5b4063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:42.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.021+0000 7fe5b3fff700 1 -- 172.21.15.17:0/2296668631 shutdown_connections 2022-08-01T19:43:42.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.021+0000 7fe5b3fff700 1 -- 172.21.15.17:0/2296668631 wait complete. 2022-08-01T19:43:42.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.025+0000 7fe5b3fff700 1 Processor -- start 2022-08-01T19:43:42.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.025+0000 7fe5b3fff700 1 -- start start 2022-08-01T19:43:42.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.025+0000 7fe5b3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5b4066780 0x7fe5b41432d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:42.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.025+0000 7fe5b3fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe5b40b1bc0 0x7fe5b413bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:42.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.025+0000 7fe5b3fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5b4140e40 0x7fe5b4150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:42.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.025+0000 7fe5b3fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b40abf10 con 0x7fe5b4066780 2022-08-01T19:43:42.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.025+0000 7fe5b3fff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b41537d0 con 0x7fe5b4140e40 2022-08-01T19:43:42.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.025+0000 7fe5b3fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b4142830 con 0x7fe5b40b1bc0 2022-08-01T19:43:42.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.025+0000 7fe5b2ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5b4066780 0x7fe5b41432d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:42.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.025+0000 7fe5b2ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5b4066780 0x7fe5b41432d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45170/0 (socket says 172.21.15.17:45170) 2022-08-01T19:43:42.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.025+0000 7fe5b2ffd700 1 -- 172.21.15.17:0/3511489921 learned_addr learned my addr 172.21.15.17:0/3511489921 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:42.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.025+0000 7fe5b27fc700 1 --2- 172.21.15.17:0/3511489921 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe5b40b1bc0 0x7fe5b413bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:42.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.025+0000 7fe5b37fe700 1 --2- 172.21.15.17:0/3511489921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5b4140e40 0x7fe5b4150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:42.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.025+0000 7fe5b2ffd700 1 -- 172.21.15.17:0/3511489921 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe5b40b1bc0 msgr2=0x7fe5b413bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:42.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.025+0000 7fe5b2ffd700 1 --2- 172.21.15.17:0/3511489921 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe5b40b1bc0 0x7fe5b413bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:42.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.025+0000 7fe5b2ffd700 1 -- 172.21.15.17:0/3511489921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5b4140e40 msgr2=0x7fe5b4150740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:42.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.025+0000 7fe5b2ffd700 1 --2- 172.21.15.17:0/3511489921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5b4140e40 0x7fe5b4150740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:42.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.025+0000 7fe5b2ffd700 1 -- 172.21.15.17:0/3511489921 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a400b180 con 0x7fe5b4066780 2022-08-01T19:43:42.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.025+0000 7fe5b37fe700 1 --2- 172.21.15.17:0/3511489921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5b4140e40 0x7fe5b4150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:43:42.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.025+0000 7fe5b2ffd700 1 --2- 172.21.15.17:0/3511489921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5b4066780 0x7fe5b41432d0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 rx=0x7fe5a4002f70 tx=0x7fe5a4014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:42.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.025+0000 7fe5abfff700 1 -- 172.21.15.17:0/3511489921 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5a4013ac0 con 0x7fe5b4066780 2022-08-01T19:43:42.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.025+0000 7fe5abfff700 1 -- 172.21.15.17:0/3511489921 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a4013c20 con 0x7fe5b4066780 2022-08-01T19:43:42.063 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.025+0000 7fe5b3fff700 1 -- 172.21.15.17:0/3511489921 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5b4142ad0 con 0x7fe5b4066780 2022-08-01T19:43:42.063 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.025+0000 7fe5abfff700 1 -- 172.21.15.17:0/3511489921 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5a401e400 con 0x7fe5b4066780 2022-08-01T19:43:42.064 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.025+0000 7fe5b3fff700 1 -- 172.21.15.17:0/3511489921 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5b4150200 con 0x7fe5b4066780 2022-08-01T19:43:42.064 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.029+0000 7fe5b3fff700 1 -- 172.21.15.17:0/3511489921 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5b4153960 con 0x7fe5b4066780 2022-08-01T19:43:42.065 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.029+0000 7fe5abfff700 1 -- 172.21.15.17:0/3511489921 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe5a4025030 con 0x7fe5b4066780 2022-08-01T19:43:42.065 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.029+0000 7fe5abfff700 1 --2- 172.21.15.17:0/3511489921 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe598053b00 0x7fe598055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:42.066 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.029+0000 7fe5abfff700 1 -- 172.21.15.17:0/3511489921 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe5a406ce80 con 0x7fe5b4066780 2022-08-01T19:43:42.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.029+0000 7fe5b27fc700 1 --2- 172.21.15.17:0/3511489921 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe598053b00 0x7fe598055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:42.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.029+0000 7fe5b27fc700 1 --2- 172.21.15.17:0/3511489921 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe598053b00 0x7fe598055fc0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7fe59c00b000 tx=0x7fe59c006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:42.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.037+0000 7fe5abfff700 1 -- 172.21.15.17:0/3511489921 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe5b4153960 con 0x7fe5b4066780 2022-08-01T19:43:42.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.165+0000 7fe5b3fff700 1 -- 172.21.15.17:0/3511489921 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe5b4153960 con 0x7fe5b4066780 2022-08-01T19:43:42.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.165+0000 7fe5abfff700 1 -- 172.21.15.17:0/3511489921 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v148) v1 ==== 104+0+10 (secure 0 0 0) 0x7fe5b4153960 con 0x7fe5b4066780 2022-08-01T19:43:42.168 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:43:42.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.165+0000 7fe5b3fff700 1 -- 172.21.15.17:0/3511489921 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe598053b00 msgr2=0x7fe598055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:42.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.165+0000 7fe5b3fff700 1 --2- 172.21.15.17:0/3511489921 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe598053b00 0x7fe598055fc0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7fe59c00b000 tx=0x7fe59c006df0).stop 2022-08-01T19:43:42.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.165+0000 7fe5b3fff700 1 -- 172.21.15.17:0/3511489921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5b4066780 msgr2=0x7fe5b41432d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:42.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.165+0000 7fe5b3fff700 1 --2- 172.21.15.17:0/3511489921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5b4066780 0x7fe5b41432d0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 rx=0x7fe5a4002f70 tx=0x7fe5a4014000).stop 2022-08-01T19:43:42.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.165+0000 7fe5b3fff700 1 -- 172.21.15.17:0/3511489921 shutdown_connections 2022-08-01T19:43:42.183 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.165+0000 7fe5b3fff700 1 --2- 172.21.15.17:0/3511489921 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe598053b00 0x7fe598055fc0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:42.183 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.165+0000 7fe5b3fff700 1 --2- 172.21.15.17:0/3511489921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5b4140e40 0x7fe5b4150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:42.184 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.165+0000 7fe5b3fff700 1 --2- 172.21.15.17:0/3511489921 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe5b40b1bc0 0x7fe5b413bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:42.184 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.165+0000 7fe5b3fff700 1 --2- 172.21.15.17:0/3511489921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5b4066780 0x7fe5b41432d0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:42.184 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.165+0000 7fe5b3fff700 1 -- 172.21.15.17:0/3511489921 >> 172.21.15.17:0/3511489921 conn(0x7fe5b4063870 msgr2=0x7fe5b4066b60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:42.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.165+0000 7fe5b3fff700 1 -- 172.21.15.17:0/3511489921 shutdown_connections 2022-08-01T19:43:42.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.165+0000 7fe5b3fff700 1 -- 172.21.15.17:0/3511489921 wait complete. 2022-08-01T19:43:42.189 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T19:43:42.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.301+0000 7efdba9fc700 1 Processor -- start 2022-08-01T19:43:42.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.301+0000 7efdba9fc700 1 -- start start 2022-08-01T19:43:42.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.301+0000 7efdba9fc700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7efdb4146ae0 0x7efdb41403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:42.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.301+0000 7efdba9fc700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efdb40b4310 0x7efdb41408b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:42.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.301+0000 7efdba9fc700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdb4143cf0 con 0x7efdb40b4820 2022-08-01T19:43:42.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.301+0000 7efdba9fc700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdb41476a0 con 0x7efdb4146ae0 2022-08-01T19:43:42.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.301+0000 7efdba9fc700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdb4143f90 con 0x7efdb40b4310 2022-08-01T19:43:42.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.301+0000 7efdb8f99700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7efdb40b4820 0x7efdb40b5060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37376/0 (socket says 172.21.15.17:37376) 2022-08-01T19:43:42.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.301+0000 7efdb8f99700 1 -- 172.21.15.17:0/157179318 learned_addr learned my addr 172.21.15.17:0/157179318 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:42.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.301+0000 7efdb3fff700 1 --2- 172.21.15.17:0/157179318 >> v2:172.21.15.77:3300/0 conn(0x7efdb4146ae0 0x7efdb41403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:42.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.301+0000 7efdab7fe700 1 --2- 172.21.15.17:0/157179318 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efdb40b4310 0x7efdb41408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:42.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.301+0000 7efdb37fe700 1 -- 172.21.15.17:0/157179318 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 650655264 0 0) 0x7efdb4143cf0 con 0x7efdb40b4820 2022-08-01T19:43:42.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.301+0000 7efdb37fe700 1 -- 172.21.15.17:0/157179318 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd94003260 con 0x7efdb40b4820 2022-08-01T19:43:42.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.301+0000 7efdb3fff700 1 -- 172.21.15.17:0/157179318 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efdb40b4310 msgr2=0x7efdb41408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:42.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.301+0000 7efdb3fff700 1 --2- 172.21.15.17:0/157179318 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efdb40b4310 0x7efdb41408b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:42.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.301+0000 7efdb3fff700 1 -- 172.21.15.17:0/157179318 >> v1:172.21.15.17:6789/0 conn(0x7efdb40b4820 legacy=0x7efdb40b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:42.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.301+0000 7efdb3fff700 1 -- 172.21.15.17:0/157179318 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb40b0e20 con 0x7efdb4146ae0 2022-08-01T19:43:42.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.301+0000 7efdab7fe700 1 --2- 172.21.15.17:0/157179318 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efdb40b4310 0x7efdb41408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:43:42.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.301+0000 7efdb3fff700 1 --2- 172.21.15.17:0/157179318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efdb4146ae0 0x7efdb41403c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7efda4002f70 tx=0x7efda4006df0).ready entity=mon.1 client_cookie=f058519a2b4d2d2f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:42.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.301+0000 7efdb37fe700 1 -- 172.21.15.17:0/157179318 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efda400d4a0 con 0x7efdb4146ae0 2022-08-01T19:43:42.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.301+0000 7efdb37fe700 1 -- 172.21.15.17:0/157179318 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda4000e90 con 0x7efdb4146ae0 2022-08-01T19:43:42.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.301+0000 7efdb37fe700 1 -- 172.21.15.17:0/157179318 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efda400d7f0 con 0x7efdb4146ae0 2022-08-01T19:43:42.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.305+0000 7efdba9fc700 1 -- 172.21.15.17:0/157179318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efdb4146ae0 msgr2=0x7efdb41403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:42.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.305+0000 7efdba9fc700 1 --2- 172.21.15.17:0/157179318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efdb4146ae0 0x7efdb41403c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7efda4002f70 tx=0x7efda4006df0).stop 2022-08-01T19:43:42.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.305+0000 7efdba9fc700 1 -- 172.21.15.17:0/157179318 shutdown_connections 2022-08-01T19:43:42.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.305+0000 7efdba9fc700 1 --2- 172.21.15.17:0/157179318 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efdb40b4310 0x7efdb41408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:42.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.305+0000 7efdba9fc700 1 --2- 172.21.15.17:0/157179318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efdb4146ae0 0x7efdb41403c0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:42.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.305+0000 7efdba9fc700 1 -- 172.21.15.17:0/157179318 >> 172.21.15.17:0/157179318 conn(0x7efdb405c620 msgr2=0x7efdb405ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:42.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.305+0000 7efdba9fc700 1 -- 172.21.15.17:0/157179318 shutdown_connections 2022-08-01T19:43:42.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.305+0000 7efdba9fc700 1 -- 172.21.15.17:0/157179318 wait complete. 2022-08-01T19:43:42.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.305+0000 7efdba9fc700 1 Processor -- start 2022-08-01T19:43:42.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.305+0000 7efdba9fc700 1 -- start start 2022-08-01T19:43:42.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.305+0000 7efdba9fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdb40b4310 0x7efdb40b8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:42.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.305+0000 7efdba9fc700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efdb40b4820 0x7efdb40adb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:42.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.305+0000 7efdba9fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efdb4146ae0 0x7efdb40ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:42.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.305+0000 7efdba9fc700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb40ad780 con 0x7efdb40b4310 2022-08-01T19:43:42.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.305+0000 7efdba9fc700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb4142210 con 0x7efdb4146ae0 2022-08-01T19:43:42.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.305+0000 7efdba9fc700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdb41424b0 con 0x7efdb40b4820 2022-08-01T19:43:42.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.305+0000 7efdb3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdb40b4310 0x7efdb40b8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:42.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.305+0000 7efdb3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdb40b4310 0x7efdb40b8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45184/0 (socket says 172.21.15.17:45184) 2022-08-01T19:43:42.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.305+0000 7efdb3fff700 1 -- 172.21.15.17:0/3372510374 learned_addr learned my addr 172.21.15.17:0/3372510374 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:42.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.305+0000 7efdab7fe700 1 --2- 172.21.15.17:0/3372510374 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efdb40b4820 0x7efdb40adb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:42.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.305+0000 7efdb8f99700 1 --2- 172.21.15.17:0/3372510374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efdb4146ae0 0x7efdb40ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:42.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.305+0000 7efdb3fff700 1 -- 172.21.15.17:0/3372510374 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efdb40b4820 msgr2=0x7efdb40adb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:42.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.305+0000 7efdb3fff700 1 --2- 172.21.15.17:0/3372510374 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efdb40b4820 0x7efdb40adb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:42.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.305+0000 7efdb3fff700 1 -- 172.21.15.17:0/3372510374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efdb4146ae0 msgr2=0x7efdb40ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:42.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.305+0000 7efdb3fff700 1 --2- 172.21.15.17:0/3372510374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efdb4146ae0 0x7efdb40ae060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:42.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.305+0000 7efdb3fff700 1 -- 172.21.15.17:0/3372510374 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9c00b180 con 0x7efdb40b4310 2022-08-01T19:43:42.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.309+0000 7efdb3fff700 1 --2- 172.21.15.17:0/3372510374 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdb40b4310 0x7efdb40b8820 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 rx=0x7efda4002f70 tx=0x7efda4014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:42.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.309+0000 7efdb1ffb700 1 -- 172.21.15.17:0/3372510374 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efda4016070 con 0x7efdb40b4310 2022-08-01T19:43:42.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.309+0000 7efdb1ffb700 1 -- 172.21.15.17:0/3372510374 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda40137d0 con 0x7efdb40b4310 2022-08-01T19:43:42.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.309+0000 7efdba9fc700 1 -- 172.21.15.17:0/3372510374 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda400b180 con 0x7efdb40b4310 2022-08-01T19:43:42.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.309+0000 7efdb1ffb700 1 -- 172.21.15.17:0/3372510374 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efda4013b30 con 0x7efdb40b4310 2022-08-01T19:43:42.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.309+0000 7efdba9fc700 1 -- 172.21.15.17:0/3372510374 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdb4152490 con 0x7efdb40b4310 2022-08-01T19:43:42.342 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.309+0000 7efdba9fc700 1 -- 172.21.15.17:0/3372510374 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdb40574e0 con 0x7efdb40b4310 2022-08-01T19:43:42.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.309+0000 7efdb1ffb700 1 -- 172.21.15.17:0/3372510374 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7efda400f030 con 0x7efdb40b4310 2022-08-01T19:43:42.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.309+0000 7efdb1ffb700 1 --2- 172.21.15.17:0/3372510374 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efd98053a10 0x7efd98055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:42.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.309+0000 7efdb1ffb700 1 -- 172.21.15.17:0/3372510374 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7efda406cc60 con 0x7efdb40b4310 2022-08-01T19:43:42.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.309+0000 7efdab7fe700 1 --2- 172.21.15.17:0/3372510374 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efd98053a10 0x7efd98055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:42.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.309+0000 7efdab7fe700 1 --2- 172.21.15.17:0/3372510374 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efd98053a10 0x7efd98055ed0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7efd9c00b000 tx=0x7efd9c006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:42.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.317+0000 7efdb1ffb700 1 -- 172.21.15.17:0/3372510374 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efda4043980 con 0x7efdb40b4310 2022-08-01T19:43:42.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.441+0000 7efdba9fc700 1 -- 172.21.15.17:0/3372510374 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efdb413fed0 con 0x7efdb40b4310 2022-08-01T19:43:42.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.441+0000 7efdb1ffb700 1 -- 172.21.15.17:0/3372510374 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v148) v1 ==== 100+0+10 (secure 0 0 0) 0x7efda4023070 con 0x7efdb40b4310 2022-08-01T19:43:42.451 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:43:42.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.445+0000 7efdba9fc700 1 -- 172.21.15.17:0/3372510374 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efd98053a10 msgr2=0x7efd98055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:42.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.445+0000 7efdba9fc700 1 --2- 172.21.15.17:0/3372510374 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efd98053a10 0x7efd98055ed0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7efd9c00b000 tx=0x7efd9c006df0).stop 2022-08-01T19:43:42.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.445+0000 7efdba9fc700 1 -- 172.21.15.17:0/3372510374 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdb40b4310 msgr2=0x7efdb40b8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:42.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.445+0000 7efdba9fc700 1 --2- 172.21.15.17:0/3372510374 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdb40b4310 0x7efdb40b8820 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 rx=0x7efda4002f70 tx=0x7efda4014000).stop 2022-08-01T19:43:42.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.445+0000 7efdba9fc700 1 -- 172.21.15.17:0/3372510374 shutdown_connections 2022-08-01T19:43:42.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.445+0000 7efdba9fc700 1 --2- 172.21.15.17:0/3372510374 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efd98053a10 0x7efd98055ed0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:42.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.445+0000 7efdba9fc700 1 --2- 172.21.15.17:0/3372510374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efdb4146ae0 0x7efdb40ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:42.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.445+0000 7efdba9fc700 1 --2- 172.21.15.17:0/3372510374 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efdb40b4820 0x7efdb40adb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:42.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.445+0000 7efdba9fc700 1 --2- 172.21.15.17:0/3372510374 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdb40b4310 0x7efdb40b8820 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:42.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.445+0000 7efdba9fc700 1 -- 172.21.15.17:0/3372510374 >> 172.21.15.17:0/3372510374 conn(0x7efdb405c620 msgr2=0x7efdb41450f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:42.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.445+0000 7efdba9fc700 1 -- 172.21.15.17:0/3372510374 shutdown_connections 2022-08-01T19:43:42.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.445+0000 7efdba9fc700 1 -- 172.21.15.17:0/3372510374 wait complete. 2022-08-01T19:43:42.463 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T19:43:42.464 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:43:42.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.565+0000 7f2e1f179700 1 Processor -- start 2022-08-01T19:43:42.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.565+0000 7f2e1f179700 1 -- start start 2022-08-01T19:43:42.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.565+0000 7f2e1f179700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2e180b7500 0x7f2e180b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:42.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.565+0000 7f2e1f179700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2e180aadb0 0x7f2e18064d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:42.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.565+0000 7f2e1f179700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e180b2510 con 0x7f2e180b31f0 2022-08-01T19:43:42.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.565+0000 7f2e1f179700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e181443e0 con 0x7f2e180aadb0 2022-08-01T19:43:42.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.565+0000 7f2e1f179700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e180b3700 con 0x7f2e180b7500 2022-08-01T19:43:42.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.565+0000 7f2e1e978700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2e180aadb0 0x7f2e18064d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:42.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.565+0000 7f2e1e978700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2e180aadb0 0x7f2e18064d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45192/0 (socket says 172.21.15.17:45192) 2022-08-01T19:43:42.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.565+0000 7f2e1e978700 1 -- 172.21.15.17:0/4170021407 learned_addr learned my addr 172.21.15.17:0/4170021407 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:42.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.565+0000 7f2e1e177700 1 --2- 172.21.15.17:0/4170021407 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2e180b7500 0x7f2e180b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:42.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.565+0000 7f2e1e978700 1 -- 172.21.15.17:0/4170021407 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2e180b7500 msgr2=0x7f2e180b78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:42.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.565+0000 7f2e1e978700 1 --2- 172.21.15.17:0/4170021407 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2e180b7500 0x7f2e180b78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:42.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.565+0000 7f2e1e978700 1 -- 172.21.15.17:0/4170021407 >> v1:172.21.15.77:6789/0 conn(0x7f2e180b31f0 legacy=0x7f2e1813e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:42.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.565+0000 7f2e1e978700 1 -- 172.21.15.17:0/4170021407 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e180b2740 con 0x7f2e180aadb0 2022-08-01T19:43:42.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.565+0000 7f2e1e978700 1 --2- 172.21.15.17:0/4170021407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e180aadb0 0x7f2e18064d30 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 rx=0x7f2e100032f0 tx=0x7f2e10009290).ready entity=mon.0 client_cookie=3fd3b7d3ca46ee0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:42.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.565+0000 7f2e1d175700 1 -- 172.21.15.17:0/4170021407 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e1000b8a0 con 0x7f2e180aadb0 2022-08-01T19:43:42.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.565+0000 7f2e1d175700 1 -- 172.21.15.17:0/4170021407 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1000ba00 con 0x7f2e180aadb0 2022-08-01T19:43:42.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.565+0000 7f2e1d175700 1 -- 172.21.15.17:0/4170021407 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e1000bb70 con 0x7f2e180aadb0 2022-08-01T19:43:42.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.565+0000 7f2e1f179700 1 -- 172.21.15.17:0/4170021407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e180aadb0 msgr2=0x7f2e18064d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:42.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.565+0000 7f2e1f179700 1 --2- 172.21.15.17:0/4170021407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e180aadb0 0x7f2e18064d30 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 rx=0x7f2e100032f0 tx=0x7f2e10009290).stop 2022-08-01T19:43:42.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.565+0000 7f2e1f179700 1 -- 172.21.15.17:0/4170021407 shutdown_connections 2022-08-01T19:43:42.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.565+0000 7f2e1f179700 1 --2- 172.21.15.17:0/4170021407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e180aadb0 0x7f2e18064d30 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:42.616 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.565+0000 7f2e1f179700 1 --2- 172.21.15.17:0/4170021407 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2e180b7500 0x7f2e180b78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:42.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.565+0000 7f2e1f179700 1 -- 172.21.15.17:0/4170021407 >> 172.21.15.17:0/4170021407 conn(0x7f2e1805f840 msgr2=0x7f2e1805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:42.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.565+0000 7f2e1f179700 1 -- 172.21.15.17:0/4170021407 shutdown_connections 2022-08-01T19:43:42.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.565+0000 7f2e1f179700 1 -- 172.21.15.17:0/4170021407 wait complete. 2022-08-01T19:43:42.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.569+0000 7f2e1f179700 1 Processor -- start 2022-08-01T19:43:42.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.569+0000 7f2e1f179700 1 -- start start 2022-08-01T19:43:42.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.569+0000 7f2e1f179700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e180aadb0 0x7f2e180b6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:42.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.569+0000 7f2e1f179700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2e180b31f0 0x7f2e180b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:42.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.569+0000 7f2e1f179700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e180b7500 0x7f2e180b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:42.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.569+0000 7f2e1f179700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e180afd90 con 0x7f2e180aadb0 2022-08-01T19:43:42.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.569+0000 7f2e1f179700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e180b1450 con 0x7f2e180b7500 2022-08-01T19:43:42.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.569+0000 7f2e1f179700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e18153200 con 0x7f2e180b31f0 2022-08-01T19:43:42.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.569+0000 7f2e1e177700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e180aadb0 0x7f2e180b6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:42.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.569+0000 7f2e1e177700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e180aadb0 0x7f2e180b6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45198/0 (socket says 172.21.15.17:45198) 2022-08-01T19:43:42.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.569+0000 7f2e1e177700 1 -- 172.21.15.17:0/1041019002 learned_addr learned my addr 172.21.15.17:0/1041019002 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:42.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.569+0000 7f2e1e978700 1 --2- 172.21.15.17:0/1041019002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e180b7500 0x7f2e180b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:42.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.569+0000 7f2e1d976700 1 --2- 172.21.15.17:0/1041019002 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2e180b31f0 0x7f2e180b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:42.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.569+0000 7f2e1e177700 1 -- 172.21.15.17:0/1041019002 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2e180b31f0 msgr2=0x7f2e180b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:42.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.569+0000 7f2e1e177700 1 --2- 172.21.15.17:0/1041019002 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2e180b31f0 0x7f2e180b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:42.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.569+0000 7f2e1e177700 1 -- 172.21.15.17:0/1041019002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e180b7500 msgr2=0x7f2e180b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:42.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.569+0000 7f2e1e177700 1 --2- 172.21.15.17:0/1041019002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e180b7500 0x7f2e180b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:42.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.569+0000 7f2e1e177700 1 -- 172.21.15.17:0/1041019002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e10009530 con 0x7f2e180aadb0 2022-08-01T19:43:42.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.569+0000 7f2e1e177700 1 --2- 172.21.15.17:0/1041019002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e180aadb0 0x7f2e180b6440 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 rx=0x7f2e1400c000 tx=0x7f2e14006d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:42.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.569+0000 7f2e077fe700 1 -- 172.21.15.17:0/1041019002 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e14011750 con 0x7f2e180aadb0 2022-08-01T19:43:42.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.569+0000 7f2e077fe700 1 -- 172.21.15.17:0/1041019002 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e140118b0 con 0x7f2e180aadb0 2022-08-01T19:43:42.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.569+0000 7f2e1f179700 1 -- 172.21.15.17:0/1041019002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e180affc0 con 0x7f2e180aadb0 2022-08-01T19:43:42.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.569+0000 7f2e077fe700 1 -- 172.21.15.17:0/1041019002 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e14011c10 con 0x7f2e180aadb0 2022-08-01T19:43:42.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.569+0000 7f2e1f179700 1 -- 172.21.15.17:0/1041019002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e18150040 con 0x7f2e180aadb0 2022-08-01T19:43:42.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.573+0000 7f2e077fe700 1 -- 172.21.15.17:0/1041019002 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2e14011d70 con 0x7f2e180aadb0 2022-08-01T19:43:42.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.573+0000 7f2e1f179700 1 -- 172.21.15.17:0/1041019002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e18153340 con 0x7f2e180aadb0 2022-08-01T19:43:42.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.573+0000 7f2e077fe700 1 --2- 172.21.15.17:0/1041019002 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2df8053aa0 0x7f2df8055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:42.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.573+0000 7f2e077fe700 1 -- 172.21.15.17:0/1041019002 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f2e1406bde0 con 0x7f2e180aadb0 2022-08-01T19:43:42.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.573+0000 7f2e1d976700 1 --2- 172.21.15.17:0/1041019002 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2df8053aa0 0x7f2df8055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:42.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.573+0000 7f2e1d976700 1 --2- 172.21.15.17:0/1041019002 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2df8053aa0 0x7f2df8055f60 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f2e0800d000 tx=0x7f2e08007b90).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:42.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.581+0000 7f2e077fe700 1 -- 172.21.15.17:0/1041019002 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2e14042a30 con 0x7f2e180aadb0 2022-08-01T19:43:42.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.745+0000 7f2e1f179700 1 -- 172.21.15.17:0/1041019002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e180574e0 con 0x7f2e180aadb0 2022-08-01T19:43:42.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.745+0000 7f2e077fe700 1 -- 172.21.15.17:0/1041019002 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f2e14046050 con 0x7f2e180aadb0 2022-08-01T19:43:42.752 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:42.752 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":76,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7211,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/872530952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":872530952},{"type":"v1","addr":"172.21.15.77:6835","nonce":872530952}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":74},{"gid":8358,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/4261332014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4261332014},{"type":"v1","addr":"172.21.15.17:6833","nonce":4261332014}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":73}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:43:34.586097+0000","modified":"2022-08-01T19:43:36.271865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8364},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8364":{"gid":8364,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/2832743740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2832743740},{"type":"v1","addr":"172.21.15.17:6835","nonce":2832743740}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2022-08-01T19:43:42.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.749+0000 7f2e1f179700 1 -- 172.21.15.17:0/1041019002 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2df8053aa0 msgr2=0x7f2df8055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:42.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.749+0000 7f2e1f179700 1 --2- 172.21.15.17:0/1041019002 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2df8053aa0 0x7f2df8055f60 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f2e0800d000 tx=0x7f2e08007b90).stop 2022-08-01T19:43:42.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.749+0000 7f2e1f179700 1 -- 172.21.15.17:0/1041019002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e180aadb0 msgr2=0x7f2e180b6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:42.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.749+0000 7f2e1f179700 1 --2- 172.21.15.17:0/1041019002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e180aadb0 0x7f2e180b6440 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 rx=0x7f2e1400c000 tx=0x7f2e14006d90).stop 2022-08-01T19:43:42.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.749+0000 7f2e1f179700 1 -- 172.21.15.17:0/1041019002 shutdown_connections 2022-08-01T19:43:42.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.749+0000 7f2e1f179700 1 --2- 172.21.15.17:0/1041019002 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2df8053aa0 0x7f2df8055f60 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:42.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.749+0000 7f2e1f179700 1 --2- 172.21.15.17:0/1041019002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e180b7500 0x7f2e180b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:42.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.749+0000 7f2e1f179700 1 --2- 172.21.15.17:0/1041019002 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2e180b31f0 0x7f2e180b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:42.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.749+0000 7f2e1f179700 1 --2- 172.21.15.17:0/1041019002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e180aadb0 0x7f2e180b6440 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:42.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.749+0000 7f2e1f179700 1 -- 172.21.15.17:0/1041019002 >> 172.21.15.17:0/1041019002 conn(0x7f2e1805f840 msgr2=0x7f2e1813f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:42.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.749+0000 7f2e1f179700 1 -- 172.21.15.17:0/1041019002 shutdown_connections 2022-08-01T19:43:42.761 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:42.749+0000 7f2e1f179700 1 -- 172.21.15.17:0/1041019002 wait complete. 2022-08-01T19:43:42.761 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 76 2022-08-01T19:43:43.078 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 76, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:43:34.586097+0000', 'modified': '2022-08-01T19:43:36.271865+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8364}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8364': {'gid': 8364, 'name': 'c', 'rank': 0, 'incarnation': 74, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/2832743740', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 2832743740}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 2832743740}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:43:43.079 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:43:43.080 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T19:43:43.216 INFO:teuthology.orchestra.run.smithi017.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":8364,"want_state":"up:active","state":"up:active","rank_uptime":8.6157794649999992,"mdsmap_epoch":76,"osdmap_epoch":148,"osdmap_epoch_barrier":148,"uptime":12.852653672000001} 2022-08-01T19:43:43.217 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 8364, "mdsmap_epoch": 76, "osdmap_epoch": 148, "osdmap_epoch_barrier": 148, "rank_uptime": 8.615779465, "state": "up:active", "uptime": 12.852653672, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:43:43.218 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T19:43:43.219 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-08-01T19:43:43.220 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi190.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-08-01T19:43:43.221 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:43.222 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:43:43.238 DEBUG:teuthology.orchestra.run.smithi190:> sudo modprobe fuse 2022-08-01T19:43:43.300 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:43:43.301 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:43:43.323 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:43:43.325 INFO:teuthology.orchestra.run.smithi190.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:43:43.325 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:43.326 DEBUG:teuthology.orchestra.run.smithi190:> ls /sys/fs/fuse/connections 2022-08-01T19:43:43.385 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T19:43:43.387 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:43:43.396 DEBUG:teuthology.orchestra.run.smithi190:> sudo modprobe fuse 2022-08-01T19:43:43.424 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:43:43.426 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:43:43.492 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:2022-08-01T19:43:43.468+0000 7f13dc03d080 -1 init, newargv = 0x55c86f5fc580 newargc=9 2022-08-01T19:43:43.492 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:ceph-fuse[23275]: starting ceph client 2022-08-01T19:43:43.493 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:43:43.495 INFO:teuthology.orchestra.run.smithi190.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:43:43.497 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:43.498 DEBUG:teuthology.orchestra.run.smithi190:> ls /sys/fs/fuse/connections 2022-08-01T19:43:43.501 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:ceph-fuse[23275]: starting fuse 2022-08-01T19:43:43.511 INFO:teuthology.orchestra.run.smithi190.stdout:52 2022-08-01T19:43:43.511 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T19:43:43.512 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:43.512 DEBUG:teuthology.orchestra.run.smithi190:> sudo python3 -c ' 2022-08-01T19:43:43.513 DEBUG:teuthology.orchestra.run.smithi190:> import glob 2022-08-01T19:43:43.513 DEBUG:teuthology.orchestra.run.smithi190:> import re 2022-08-01T19:43:43.514 DEBUG:teuthology.orchestra.run.smithi190:> import os 2022-08-01T19:43:43.514 DEBUG:teuthology.orchestra.run.smithi190:> import subprocess 2022-08-01T19:43:43.515 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:43:43.516 DEBUG:teuthology.orchestra.run.smithi190:> def find_socket(client_name): 2022-08-01T19:43:43.525 DEBUG:teuthology.orchestra.run.smithi190:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-08-01T19:43:43.527 DEBUG:teuthology.orchestra.run.smithi190:> files = glob.glob(asok_path) 2022-08-01T19:43:43.528 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:43:43.529 DEBUG:teuthology.orchestra.run.smithi190:> # Given a non-glob path, it better be there 2022-08-01T19:43:43.532 DEBUG:teuthology.orchestra.run.smithi190:> if "*" not in asok_path: 2022-08-01T19:43:43.533 DEBUG:teuthology.orchestra.run.smithi190:> assert(len(files) == 1) 2022-08-01T19:43:43.534 DEBUG:teuthology.orchestra.run.smithi190:> return files[0] 2022-08-01T19:43:43.535 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:43:43.536 DEBUG:teuthology.orchestra.run.smithi190:> for f in files: 2022-08-01T19:43:43.536 DEBUG:teuthology.orchestra.run.smithi190:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T19:43:43.537 DEBUG:teuthology.orchestra.run.smithi190:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T19:43:43.538 DEBUG:teuthology.orchestra.run.smithi190:> return f 2022-08-01T19:43:43.539 DEBUG:teuthology.orchestra.run.smithi190:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T19:43:43.540 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:43:43.541 DEBUG:teuthology.orchestra.run.smithi190:> print(find_socket("client.1")) 2022-08-01T19:43:43.542 DEBUG:teuthology.orchestra.run.smithi190:> ' 2022-08-01T19:43:43.604 INFO:teuthology.orchestra.run.smithi190.stdout:/var/run/ceph/ceph-client.1.23275.asok 2022-08-01T19:43:43.605 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.23275.asok 2022-08-01T19:43:43.605 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:43.607 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.23275.asok status 2022-08-01T19:43:43.715 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2022-08-01T19:43:43.716 INFO:teuthology.orchestra.run.smithi190.stdout: "metadata": { 2022-08-01T19:43:43.716 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T19:43:43.716 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T19:43:43.716 INFO:teuthology.orchestra.run.smithi190.stdout: "entity_id": "1", 2022-08-01T19:43:43.717 INFO:teuthology.orchestra.run.smithi190.stdout: "hostname": "smithi190", 2022-08-01T19:43:43.717 INFO:teuthology.orchestra.run.smithi190.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-08-01T19:43:43.717 INFO:teuthology.orchestra.run.smithi190.stdout: "pid": "23275", 2022-08-01T19:43:43.717 INFO:teuthology.orchestra.run.smithi190.stdout: "root": "/" 2022-08-01T19:43:43.717 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:43:43.718 INFO:teuthology.orchestra.run.smithi190.stdout: "dentry_count": 0, 2022-08-01T19:43:43.718 INFO:teuthology.orchestra.run.smithi190.stdout: "dentry_pinned_count": 0, 2022-08-01T19:43:43.718 INFO:teuthology.orchestra.run.smithi190.stdout: "id": 8574, 2022-08-01T19:43:43.718 INFO:teuthology.orchestra.run.smithi190.stdout: "inst": { 2022-08-01T19:43:43.719 INFO:teuthology.orchestra.run.smithi190.stdout: "name": { 2022-08-01T19:43:43.719 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "client", 2022-08-01T19:43:43.719 INFO:teuthology.orchestra.run.smithi190.stdout: "num": 8574 2022-08-01T19:43:43.719 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:43:43.720 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": { 2022-08-01T19:43:43.720 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "v1", 2022-08-01T19:43:43.720 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": "172.21.15.190:0", 2022-08-01T19:43:43.720 INFO:teuthology.orchestra.run.smithi190.stdout: "nonce": 1403169971 2022-08-01T19:43:43.721 INFO:teuthology.orchestra.run.smithi190.stdout: } 2022-08-01T19:43:43.721 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:43:43.721 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": { 2022-08-01T19:43:43.721 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "v1", 2022-08-01T19:43:43.728 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": "172.21.15.190:0", 2022-08-01T19:43:43.729 INFO:teuthology.orchestra.run.smithi190.stdout: "nonce": 1403169971 2022-08-01T19:43:43.729 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:43:43.730 INFO:teuthology.orchestra.run.smithi190.stdout: "inst_str": "client.8574 172.21.15.190:0/1403169971", 2022-08-01T19:43:43.730 INFO:teuthology.orchestra.run.smithi190.stdout: "addr_str": "172.21.15.190:0/1403169971", 2022-08-01T19:43:43.731 INFO:teuthology.orchestra.run.smithi190.stdout: "inode_count": 1, 2022-08-01T19:43:43.731 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_epoch": 76, 2022-08-01T19:43:43.732 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_epoch": 148, 2022-08-01T19:43:43.737 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_epoch_barrier": 0, 2022-08-01T19:43:43.738 INFO:teuthology.orchestra.run.smithi190.stdout: "blacklisted": false, 2022-08-01T19:43:43.776 INFO:teuthology.orchestra.run.smithi190.stdout: "fs_name": "cephfs" 2022-08-01T19:43:43.777 INFO:teuthology.orchestra.run.smithi190.stdout:} 2022-08-01T19:43:43.780 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:43.780 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:43:43.781 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:43:43.812 INFO:teuthology.orchestra.run.smithi190.stdout:fuseblk 2022-08-01T19:43:43.812 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T19:43:43.813 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:43.813 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:43:43.848 INFO:teuthology.orchestra.run:Running command with timeout 10800 2022-08-01T19:43:43.849 DEBUG:teuthology.orchestra.run.smithi085:> cd /home/ubuntu/cephtest/mnt.0 && sudo getfattr --only-values -n ceph.dir.rctime . 2022-08-01T19:43:43.887 INFO:teuthology.orchestra.run.smithi085.stdout:1659383023.833500334 2022-08-01T19:43:43.888 INFO:tasks.cephfs.test_misc:rctime = 1659383023.833500334 2022-08-01T19:43:43.888 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T19:43:43.889 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:43:43.915 DEBUG:teuthology.orchestra.run.smithi077:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:43:43.939 DEBUG:teuthology.orchestra.run.smithi085:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:43:43.964 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:43:43.987 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T19:43:43.988 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:43.989 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:43:43.989 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:43:44.015 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:43:44.018 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:43:44.020 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi085.front.sepia.ceph.com... 2022-08-01T19:43:44.020 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T19:43:44.020 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:43:44.041 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:ceph-fuse[23661]: fuse finished with error 0 and tester_r 0 2022-08-01T19:43:44.043 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:44.044 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:43:44.045 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:43:44.059 INFO:teuthology.orchestra.run.smithi085.stdout:ext2/ext3 2022-08-01T19:43:44.059 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T19:43:44.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T19:43:44.116 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:43:44.138 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:44.139 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:43:44.140 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:43:44.155 INFO:teuthology.orchestra.run.smithi190.stdout:fuseblk 2022-08-01T19:43:44.156 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T19:43:44.156 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi190.front.sepia.ceph.com... 2022-08-01T19:43:44.167 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T19:43:44.171 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:43:44.228 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:ceph-fuse[23275]: fuse finished with error 0 and tester_r 0 2022-08-01T19:43:44.229 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:44.230 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:43:44.231 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:43:44.260 INFO:teuthology.orchestra.run.smithi190.stdout:ext2/ext3 2022-08-01T19:43:44.260 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T19:43:44.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T19:43:44.276 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:43:44.345 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T19:43:44.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.465+0000 7f17f0dee700 1 Processor -- start 2022-08-01T19:43:44.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.465+0000 7f17f0dee700 1 -- start start 2022-08-01T19:43:44.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.469+0000 7f17f0dee700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f17ec0ad750 0x7f17ec061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:44.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.469+0000 7f17f0dee700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f17ec1405d0 0x7f17ec140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:44.530 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.469+0000 7f17f0dee700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17ec0aee80 con 0x7f17ec0614b0 2022-08-01T19:43:44.531 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.469+0000 7f17f0dee700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17ec0b70b0 con 0x7f17ec0ad750 2022-08-01T19:43:44.531 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.469+0000 7f17f0dee700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17ec0af0b0 con 0x7f17ec1405d0 2022-08-01T19:43:44.532 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.469+0000 7f17eaffd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f17ec0ad750 0x7f17ec061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:44.532 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.469+0000 7f17eaffd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f17ec0ad750 0x7f17ec061890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45204/0 (socket says 172.21.15.17:45204) 2022-08-01T19:43:44.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.469+0000 7f17eaffd700 1 -- 172.21.15.17:0/22589789 learned_addr learned my addr 172.21.15.17:0/22589789 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:44.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.469+0000 7f17ebfff700 1 --2- 172.21.15.17:0/22589789 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f17ec1405d0 0x7f17ec140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:44.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.469+0000 7f17ea7fc700 1 -- 172.21.15.17:0/22589789 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3909062534 0 0) 0x7f17ec0aee80 con 0x7f17ec0614b0 2022-08-01T19:43:44.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.469+0000 7f17ea7fc700 1 -- 172.21.15.17:0/22589789 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17d8003260 con 0x7f17ec0614b0 2022-08-01T19:43:44.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.469+0000 7f17eaffd700 1 -- 172.21.15.17:0/22589789 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f17ec1405d0 msgr2=0x7f17ec140a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:44.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.469+0000 7f17eaffd700 1 --2- 172.21.15.17:0/22589789 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f17ec1405d0 0x7f17ec140a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:44.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.469+0000 7f17eaffd700 1 -- 172.21.15.17:0/22589789 >> v1:172.21.15.77:6789/0 conn(0x7f17ec0614b0 legacy=0x7f17ec0b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:44.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.469+0000 7f17eaffd700 1 -- 172.21.15.17:0/22589789 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17ec0af350 con 0x7f17ec0ad750 2022-08-01T19:43:44.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.469+0000 7f17eaffd700 1 --2- 172.21.15.17:0/22589789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17ec0ad750 0x7f17ec061890 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 rx=0x7f17dc0032f0 tx=0x7f17dc009290).ready entity=mon.0 client_cookie=f1e35db644b14482 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:44.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.469+0000 7f17ea7fc700 1 -- 172.21.15.17:0/22589789 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17dc00b6c0 con 0x7f17ec0ad750 2022-08-01T19:43:44.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.469+0000 7f17ea7fc700 1 -- 172.21.15.17:0/22589789 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17dc00b820 con 0x7f17ec0ad750 2022-08-01T19:43:44.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.469+0000 7f17ea7fc700 1 -- 172.21.15.17:0/22589789 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17dc00b990 con 0x7f17ec0ad750 2022-08-01T19:43:44.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.469+0000 7f17f0dee700 1 -- 172.21.15.17:0/22589789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17ec0ad750 msgr2=0x7f17ec061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:44.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.469+0000 7f17f0dee700 1 --2- 172.21.15.17:0/22589789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17ec0ad750 0x7f17ec061890 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 rx=0x7f17dc0032f0 tx=0x7f17dc009290).stop 2022-08-01T19:43:44.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.469+0000 7f17f0dee700 1 -- 172.21.15.17:0/22589789 shutdown_connections 2022-08-01T19:43:44.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.469+0000 7f17f0dee700 1 --2- 172.21.15.17:0/22589789 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f17ec1405d0 0x7f17ec140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:44.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.469+0000 7f17f0dee700 1 --2- 172.21.15.17:0/22589789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17ec0ad750 0x7f17ec061890 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:44.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.469+0000 7f17f0dee700 1 -- 172.21.15.17:0/22589789 >> 172.21.15.17:0/22589789 conn(0x7f17ec066f70 msgr2=0x7f17ec067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:44.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.469+0000 7f17f0dee700 1 -- 172.21.15.17:0/22589789 shutdown_connections 2022-08-01T19:43:44.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.469+0000 7f17f0dee700 1 -- 172.21.15.17:0/22589789 wait complete. 2022-08-01T19:43:44.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.473+0000 7f17f0dee700 1 Processor -- start 2022-08-01T19:43:44.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.473+0000 7f17f0dee700 1 -- start start 2022-08-01T19:43:44.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.473+0000 7f17f0dee700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f17ec0614b0 0x7f17ec05cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:44.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.473+0000 7f17f0dee700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17ec0ad750 0x7f17ec155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:44.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.473+0000 7f17f0dee700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17ec1405d0 0x7f17ec155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:44.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.473+0000 7f17f0dee700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17ec156420 con 0x7f17ec0ad750 2022-08-01T19:43:44.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.473+0000 7f17f0dee700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17ec0aa8c0 con 0x7f17ec1405d0 2022-08-01T19:43:44.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.473+0000 7f17f0dee700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17ec0aaaf0 con 0x7f17ec0614b0 2022-08-01T19:43:44.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.473+0000 7f17eaffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17ec0ad750 0x7f17ec155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:44.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.473+0000 7f17eaffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17ec0ad750 0x7f17ec155a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45212/0 (socket says 172.21.15.17:45212) 2022-08-01T19:43:44.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.473+0000 7f17eaffd700 1 -- 172.21.15.17:0/325518666 learned_addr learned my addr 172.21.15.17:0/325518666 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:44.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.473+0000 7f17eb7fe700 1 --2- 172.21.15.17:0/325518666 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f17ec0614b0 0x7f17ec05cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:44.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.473+0000 7f17ebfff700 1 --2- 172.21.15.17:0/325518666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17ec1405d0 0x7f17ec155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:44.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.473+0000 7f17eaffd700 1 -- 172.21.15.17:0/325518666 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f17ec0614b0 msgr2=0x7f17ec05cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:44.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.473+0000 7f17eaffd700 1 --2- 172.21.15.17:0/325518666 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f17ec0614b0 0x7f17ec05cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:44.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.473+0000 7f17eaffd700 1 -- 172.21.15.17:0/325518666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17ec1405d0 msgr2=0x7f17ec155f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:44.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.473+0000 7f17eaffd700 1 --2- 172.21.15.17:0/325518666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17ec1405d0 0x7f17ec155f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:44.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.473+0000 7f17eaffd700 1 -- 172.21.15.17:0/325518666 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17dc009530 con 0x7f17ec0ad750 2022-08-01T19:43:44.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.473+0000 7f17eaffd700 1 --2- 172.21.15.17:0/325518666 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17ec0ad750 0x7f17ec155a40 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 rx=0x7f17dc0032f0 tx=0x7f17dc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:44.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.473+0000 7f17e8ff9700 1 -- 172.21.15.17:0/325518666 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17dc00b990 con 0x7f17ec0ad750 2022-08-01T19:43:44.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.473+0000 7f17e8ff9700 1 -- 172.21.15.17:0/325518666 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17dc021630 con 0x7f17ec0ad750 2022-08-01T19:43:44.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.473+0000 7f17f0dee700 1 -- 172.21.15.17:0/325518666 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17ec0aad90 con 0x7f17ec0ad750 2022-08-01T19:43:44.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.473+0000 7f17e8ff9700 1 -- 172.21.15.17:0/325518666 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17dc00b6c0 con 0x7f17ec0ad750 2022-08-01T19:43:44.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.473+0000 7f17f0dee700 1 -- 172.21.15.17:0/325518666 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17ec0b1490 con 0x7f17ec0ad750 2022-08-01T19:43:44.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.477+0000 7f17f0dee700 1 -- 172.21.15.17:0/325518666 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17ec0af5f0 con 0x7f17ec0ad750 2022-08-01T19:43:44.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.477+0000 7f17e8ff9700 1 -- 172.21.15.17:0/325518666 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f17dc00f070 con 0x7f17ec0ad750 2022-08-01T19:43:44.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.477+0000 7f17e8ff9700 1 --2- 172.21.15.17:0/325518666 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f17cc053950 0x7f17cc055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:44.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.477+0000 7f17e8ff9700 1 -- 172.21.15.17:0/325518666 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f17dc06d2e0 con 0x7f17ec0ad750 2022-08-01T19:43:44.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.477+0000 7f17eb7fe700 1 --2- 172.21.15.17:0/325518666 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f17cc053950 0x7f17cc055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:44.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.477+0000 7f17eb7fe700 1 --2- 172.21.15.17:0/325518666 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f17cc053950 0x7f17cc055e10 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f17e000d000 tx=0x7f17e0007a00).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:44.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.485+0000 7f17e8ff9700 1 -- 172.21.15.17:0/325518666 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f17dc0440e0 con 0x7f17ec0ad750 2022-08-01T19:43:44.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.609+0000 7f17f0dee700 1 -- 172.21.15.17:0/325518666 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f17ec0574e0 con 0x7f17ec0ad750 2022-08-01T19:43:44.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.613+0000 7f17e8ff9700 1 -- 172.21.15.17:0/325518666 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v148) v1 ==== 81+0+34051 (secure 0 0 0) 0x7f17dc015020 con 0x7f17ec0ad750 2022-08-01T19:43:44.618 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:44.619 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:43:44.619 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 148, 2022-08-01T19:43:44.619 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", 2022-08-01T19:43:44.620 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2022-08-01T19:33:50.105743+0000", 2022-08-01T19:43:44.620 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2022-08-01T19:43:35.589613+0000", 2022-08-01T19:43:44.620 INFO:teuthology.orchestra.run.smithi017.stdout: "last_up_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:43:44.620 INFO:teuthology.orchestra.run.smithi017.stdout: "last_in_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:43:44.621 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T19:43:44.621 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_num": 5799936, 2022-08-01T19:43:44.623 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_set": [ 2022-08-01T19:43:44.623 INFO:teuthology.orchestra.run.smithi017.stdout: "pglog_hardlimit", 2022-08-01T19:43:44.624 INFO:teuthology.orchestra.run.smithi017.stdout: "purged_snapdirs", 2022-08-01T19:43:44.624 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_deletes", 2022-08-01T19:43:44.624 INFO:teuthology.orchestra.run.smithi017.stdout: "sortbitwise" 2022-08-01T19:43:44.624 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:43:44.624 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_version": 6, 2022-08-01T19:43:44.625 INFO:teuthology.orchestra.run.smithi017.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T19:43:44.625 INFO:teuthology.orchestra.run.smithi017.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T19:43:44.625 INFO:teuthology.orchestra.run.smithi017.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T19:43:44.625 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_snapshot": "", 2022-08-01T19:43:44.635 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_max": 28, 2022-08-01T19:43:44.635 INFO:teuthology.orchestra.run.smithi017.stdout: "max_osd": 8, 2022-08-01T19:43:44.636 INFO:teuthology.orchestra.run.smithi017.stdout: "require_min_compat_client": "jewel", 2022-08-01T19:43:44.636 INFO:teuthology.orchestra.run.smithi017.stdout: "min_compat_client": "jewel", 2022-08-01T19:43:44.636 INFO:teuthology.orchestra.run.smithi017.stdout: "require_osd_release": "octopus", 2022-08-01T19:43:44.637 INFO:teuthology.orchestra.run.smithi017.stdout: "pools": [ 2022-08-01T19:43:44.637 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.637 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 1, 2022-08-01T19:43:44.637 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "device_health_metrics", 2022-08-01T19:43:44.638 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:33:53.001841+0000", 2022-08-01T19:43:44.638 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:43:44.638 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:43:44.638 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:43:44.639 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:43:44.639 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:43:44.639 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:43:44.639 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:43:44.640 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:43:44.640 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 1, 2022-08-01T19:43:44.640 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 1, 2022-08-01T19:43:44.640 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 1, 2022-08-01T19:43:44.641 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 1, 2022-08-01T19:43:44.641 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 1, 2022-08-01T19:43:44.641 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:43:44.641 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:43:44.642 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:43:44.642 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:43:44.642 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:43:44.643 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:43:44.643 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:43:44.643 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.643 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "16", 2022-08-01T19:43:44.644 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:43:44.644 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:43:44.664 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:43:44.665 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:43:44.665 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:43:44.666 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:43:44.666 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:43:44.666 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:43:44.666 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:43:44.667 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:43:44.667 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:43:44.667 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:43:44.667 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:43:44.668 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:43:44.668 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:43:44.668 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:43:44.668 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:43:44.669 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:43:44.669 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:43:44.669 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:43:44.669 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:43:44.670 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:43:44.670 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:43:44.670 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:43:44.671 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:43:44.671 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:43:44.671 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.671 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:43:44.672 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:43:44.672 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:43:44.672 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:43:44.672 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:43:44.679 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:43:44.679 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:43:44.679 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:43:44.680 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:43:44.681 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:43:44.682 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:43:44.683 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:43:44.683 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 1 2022-08-01T19:43:44.684 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.684 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:43:44.684 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_devicehealth": {} 2022-08-01T19:43:44.684 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:44.685 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.685 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.685 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 2, 2022-08-01T19:43:44.685 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "rbd", 2022-08-01T19:43:44.690 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:34:01.553098+0000", 2022-08-01T19:43:44.690 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:43:44.691 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:43:44.693 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:43:44.695 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:43:44.695 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:43:44.695 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:43:44.695 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:43:44.696 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:43:44.696 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:43:44.696 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:43:44.698 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:43:44.698 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:43:44.698 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:43:44.699 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:43:44.699 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:43:44.699 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:43:44.699 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:43:44.701 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:43:44.702 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:43:44.702 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:43:44.704 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.704 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "19", 2022-08-01T19:43:44.704 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:43:44.704 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:43:44.705 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:43:44.705 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:43:44.706 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:43:44.706 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:43:44.706 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:43:44.707 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:43:44.707 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:43:44.707 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:43:44.713 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:43:44.714 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:43:44.714 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:43:44.714 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:43:44.714 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:43:44.714 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:43:44.715 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:43:44.715 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:43:44.715 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:43:44.715 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:43:44.716 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:43:44.716 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:43:44.716 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:43:44.716 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:43:44.717 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:43:44.717 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:43:44.717 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.717 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:43:44.718 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:43:44.719 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:43:44.719 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:43:44.719 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:43:44.719 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:43:44.720 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:43:44.720 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:43:44.720 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:43:44.720 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:43:44.720 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:43:44.721 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:43:44.721 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:43:44.721 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd": {} 2022-08-01T19:43:44.721 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:44.722 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.722 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.723 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 27, 2022-08-01T19:43:44.724 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "cephfs_metadata", 2022-08-01T19:43:44.724 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:43:33.209247+0000", 2022-08-01T19:43:44.727 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:43:44.727 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:43:44.727 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:43:44.727 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:43:44.728 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:43:44.728 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:43:44.729 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:43:44.733 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:43:44.733 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:43:44.733 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:43:44.734 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:43:44.734 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:43:44.735 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:43:44.736 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:43:44.736 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:43:44.743 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:43:44.743 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:43:44.743 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:43:44.744 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:43:44.745 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:43:44.746 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.746 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "147", 2022-08-01T19:43:44.747 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:43:44.748 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:43:44.749 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:43:44.749 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:43:44.750 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:43:44.751 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:43:44.751 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:43:44.753 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:43:44.754 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:43:44.754 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:43:44.755 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:43:44.755 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:43:44.756 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:43:44.756 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:43:44.757 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:43:44.757 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:43:44.758 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:43:44.758 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:43:44.759 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:43:44.759 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:43:44.761 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:43:44.762 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:43:44.762 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:43:44.763 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:43:44.763 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:43:44.763 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:43:44.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.764 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:43:44.764 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:43:44.764 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:43:44.765 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:43:44.765 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:43:44.765 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:43:44.765 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:43:44.766 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:43:44.766 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:43:44.766 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:43:44.766 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:43:44.767 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:43:44.767 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_bias": 4, 2022-08-01T19:43:44.767 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 16, 2022-08-01T19:43:44.768 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_priority": 5 2022-08-01T19:43:44.768 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.769 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:43:44.769 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs": { 2022-08-01T19:43:44.770 INFO:teuthology.orchestra.run.smithi017.stdout: "metadata": "cephfs" 2022-08-01T19:43:44.770 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:44.771 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:44.771 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.771 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.772 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 28, 2022-08-01T19:43:44.772 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "cephfs_data", 2022-08-01T19:43:44.772 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:43:33.595201+0000", 2022-08-01T19:43:44.772 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:43:44.772 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:43:44.773 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:43:44.773 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:43:44.773 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:43:44.773 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:43:44.774 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:43:44.774 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:43:44.774 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:43:44.775 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:43:44.775 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:43:44.775 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:43:44.775 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:43:44.775 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:43:44.776 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:43:44.776 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:43:44.776 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:43:44.776 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:43:44.777 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:43:44.777 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:43:44.777 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.778 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "148", 2022-08-01T19:43:44.778 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:43:44.778 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:43:44.778 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:43:44.779 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:43:44.779 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:43:44.780 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:43:44.780 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:43:44.780 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:43:44.781 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:43:44.781 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:43:44.781 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:43:44.782 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:43:44.782 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:43:44.783 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:43:44.783 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:43:44.783 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:43:44.784 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:43:44.784 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:43:44.784 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:43:44.784 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:43:44.784 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:43:44.785 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:43:44.785 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:43:44.785 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:43:44.785 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:43:44.786 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:43:44.786 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.786 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:43:44.786 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:43:44.787 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:43:44.787 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:43:44.787 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:43:44.787 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:43:44.788 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:43:44.788 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:43:44.788 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:43:44.788 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:43:44.790 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:43:44.791 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:43:44.791 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:43:44.792 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs": { 2022-08-01T19:43:44.792 INFO:teuthology.orchestra.run.smithi017.stdout: "data": "cephfs" 2022-08-01T19:43:44.792 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:44.793 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:44.793 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:44.793 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:43:44.793 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2022-08-01T19:43:44.794 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.794 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:43:44.795 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a962f14f-cf16-4fa8-84ff-c85325db9132", 2022-08-01T19:43:44.797 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:43:44.797 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:43:44.797 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:43:44.798 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:43:44.802 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:43:44.802 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:43:44.803 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:43:44.803 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 145, 2022-08-01T19:43:44.803 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:43:44.803 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:43:44.804 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:43:44.804 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:44.804 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.804 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:44.805 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6800", 2022-08-01T19:43:44.805 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:43:44.805 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.805 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.809 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:44.810 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6801", 2022-08-01T19:43:44.810 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:43:44.811 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:44.811 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:44.812 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.812 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:43:44.812 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:44.813 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.813 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:44.814 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6802", 2022-08-01T19:43:44.814 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:43:44.822 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.823 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.823 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:44.824 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6803", 2022-08-01T19:43:44.824 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:43:44.825 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:44.825 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:44.827 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.828 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:43:44.828 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:44.829 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.829 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:44.830 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6806", 2022-08-01T19:43:44.830 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:43:44.831 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.831 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.831 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:44.832 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6807", 2022-08-01T19:43:44.839 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:43:44.839 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:44.840 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:44.840 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.841 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:43:44.841 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:44.841 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.842 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:44.842 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6804", 2022-08-01T19:43:44.843 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:43:44.845 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.845 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.845 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:44.846 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6805", 2022-08-01T19:43:44.846 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:43:44.856 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:44.857 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:44.857 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.858 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6801/23008", 2022-08-01T19:43:44.859 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6803/23008", 2022-08-01T19:43:44.859 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6807/23008", 2022-08-01T19:43:44.860 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6805/23008", 2022-08-01T19:43:44.860 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:43:44.861 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:43:44.861 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:43:44.866 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:44.866 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.867 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.867 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:43:44.867 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "805f7051-a527-4e98-8844-b6e406a63248", 2022-08-01T19:43:44.867 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:43:44.868 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:43:44.868 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:43:44.868 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:43:44.868 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:43:44.869 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:43:44.869 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:43:44.869 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 145, 2022-08-01T19:43:44.870 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:43:44.870 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:43:44.870 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:43:44.872 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:44.872 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.873 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:44.873 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6808", 2022-08-01T19:43:44.874 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:43:44.874 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.875 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.876 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:44.876 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6809", 2022-08-01T19:43:44.877 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:43:44.877 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:44.878 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:44.878 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.879 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:43:44.879 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:44.880 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.880 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:44.881 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6810", 2022-08-01T19:43:44.883 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:43:44.883 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.884 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.884 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:44.885 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6811", 2022-08-01T19:43:44.885 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:43:44.887 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:44.887 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:44.888 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.888 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:43:44.888 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:44.889 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.889 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:44.890 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6814", 2022-08-01T19:43:44.890 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:43:44.891 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.891 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.892 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:44.892 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6815", 2022-08-01T19:43:44.893 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:43:44.894 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:44.894 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:44.895 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.895 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:43:44.895 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:44.896 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.896 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:44.897 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6812", 2022-08-01T19:43:44.901 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:43:44.902 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.902 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.903 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:44.903 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6813", 2022-08-01T19:43:44.904 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:43:44.904 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:44.905 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:44.905 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.906 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6809/23007", 2022-08-01T19:43:44.906 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6811/23007", 2022-08-01T19:43:44.907 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6815/23007", 2022-08-01T19:43:44.907 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6813/23007", 2022-08-01T19:43:44.908 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:43:44.908 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:43:44.909 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:43:44.909 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:44.910 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.910 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.911 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:43:44.911 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "89f7f3df-7c10-48e1-8b69-9e4243923a61", 2022-08-01T19:43:44.911 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:43:44.912 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:43:44.912 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:43:44.913 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:43:44.913 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:43:44.914 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:43:44.914 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:43:44.915 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 144, 2022-08-01T19:43:44.917 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:43:44.917 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:43:44.918 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:43:44.918 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:44.919 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.919 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:44.920 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6817", 2022-08-01T19:43:44.920 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:43:44.920 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.921 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.921 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:44.922 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6819", 2022-08-01T19:43:44.922 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:43:44.923 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:44.923 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:44.924 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.924 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:43:44.925 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:44.925 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.925 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:44.926 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6822", 2022-08-01T19:43:44.926 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:43:44.927 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.927 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.928 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:44.928 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6824", 2022-08-01T19:43:44.928 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:43:44.928 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:44.929 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:44.929 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.929 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:43:44.930 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:44.930 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.930 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:44.931 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6830", 2022-08-01T19:43:44.932 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:43:44.932 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.933 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.933 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:44.933 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6831", 2022-08-01T19:43:44.938 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:43:44.939 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:44.939 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:44.940 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.940 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:43:44.940 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:44.941 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.941 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:44.942 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6826", 2022-08-01T19:43:44.943 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:43:44.943 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.945 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.946 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:44.947 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6829", 2022-08-01T19:43:44.948 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:43:44.948 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:44.949 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:44.949 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.950 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6819/23010", 2022-08-01T19:43:44.950 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6824/23010", 2022-08-01T19:43:44.951 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6831/23010", 2022-08-01T19:43:44.951 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6829/23010", 2022-08-01T19:43:44.951 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:43:44.952 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:43:44.952 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:43:44.952 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:44.952 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.953 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.954 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:43:44.954 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "96123468-aabc-4a64-909e-a192c369090d", 2022-08-01T19:43:44.955 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:43:44.955 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:43:44.955 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:43:44.955 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:43:44.956 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:43:44.956 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:43:44.956 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:43:44.956 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 134, 2022-08-01T19:43:44.957 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:43:44.957 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:43:44.957 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:43:44.957 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:44.957 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.958 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:44.958 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6816", 2022-08-01T19:43:44.958 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:43:44.959 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.959 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.960 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:44.960 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6818", 2022-08-01T19:43:44.960 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:43:44.961 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:44.961 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:44.962 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.962 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:43:44.963 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:44.963 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.964 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:44.965 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6820", 2022-08-01T19:43:44.965 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:43:44.966 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.966 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.967 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:44.970 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6821", 2022-08-01T19:43:44.970 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:43:44.971 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:44.971 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:44.972 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.972 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:43:44.972 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:44.973 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.973 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:44.973 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6827", 2022-08-01T19:43:44.976 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:43:44.976 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.976 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.977 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:44.977 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6828", 2022-08-01T19:43:44.978 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:43:44.978 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:44.978 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:44.979 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.979 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:43:44.980 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:44.980 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.980 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:44.981 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6823", 2022-08-01T19:43:44.981 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:43:44.981 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.982 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.982 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:44.983 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6825", 2022-08-01T19:43:44.983 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:43:44.984 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:44.984 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:44.985 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.985 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6818/23009", 2022-08-01T19:43:44.985 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6821/23009", 2022-08-01T19:43:44.986 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6828/23009", 2022-08-01T19:43:44.986 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6825/23009", 2022-08-01T19:43:44.987 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:43:44.987 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:43:44.988 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:43:44.988 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:44.988 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.989 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.989 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:43:44.989 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a6e26665-e04b-46ec-ab66-e912acbf044c", 2022-08-01T19:43:44.990 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:43:44.990 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:43:44.991 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:43:44.991 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:43:44.992 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:43:44.992 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:43:44.992 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:43:44.992 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 144, 2022-08-01T19:43:44.993 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:43:44.993 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:43:44.993 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:43:44.994 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:44.994 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.995 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:44.996 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6810", 2022-08-01T19:43:44.997 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:43:44.997 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:44.997 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:44.998 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:44.998 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6811", 2022-08-01T19:43:44.998 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:43:44.999 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:44.999 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:45.000 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.001 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:43:45.001 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:45.002 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.002 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:45.002 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6812", 2022-08-01T19:43:45.002 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:43:45.003 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.003 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.003 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:45.003 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6813", 2022-08-01T19:43:45.003 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:43:45.004 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:45.004 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:45.004 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.004 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:43:45.005 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:45.005 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.005 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:45.005 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6816", 2022-08-01T19:43:45.005 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:43:45.006 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.006 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.006 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:45.006 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6817", 2022-08-01T19:43:45.007 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:43:45.014 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:45.014 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:45.015 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.015 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:43:45.016 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:45.016 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.016 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:45.017 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6814", 2022-08-01T19:43:45.017 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:43:45.018 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.018 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.019 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:45.019 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6815", 2022-08-01T19:43:45.019 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:43:45.020 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:45.020 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:45.021 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.021 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6811/23584", 2022-08-01T19:43:45.024 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6813/23584", 2022-08-01T19:43:45.024 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6817/23584", 2022-08-01T19:43:45.025 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6815/23584", 2022-08-01T19:43:45.025 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:43:45.026 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:43:45.026 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:43:45.026 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:45.027 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.027 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.028 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:43:45.028 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "502c0034-18ee-44ac-87ff-19b9dddb66a0", 2022-08-01T19:43:45.029 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:43:45.029 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:43:45.029 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:43:45.030 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:43:45.031 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:43:45.031 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:43:45.031 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:43:45.032 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 145, 2022-08-01T19:43:45.032 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:43:45.033 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:43:45.033 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:43:45.034 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:45.034 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.034 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:45.035 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6802", 2022-08-01T19:43:45.036 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:43:45.037 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.037 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.037 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:45.038 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6803", 2022-08-01T19:43:45.038 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:43:45.039 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:45.039 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:45.040 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.040 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:43:45.040 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:45.041 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.041 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:45.042 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6804", 2022-08-01T19:43:45.042 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:43:45.043 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.043 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.044 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:45.044 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6805", 2022-08-01T19:43:45.044 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:43:45.045 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:45.045 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:45.046 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.047 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:43:45.047 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:45.048 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.048 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:45.049 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6808", 2022-08-01T19:43:45.049 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:43:45.049 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.050 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.050 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:45.051 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6809", 2022-08-01T19:43:45.051 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:43:45.052 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:45.052 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:45.053 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.053 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:43:45.054 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:45.054 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.054 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:45.055 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6806", 2022-08-01T19:43:45.055 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:43:45.056 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.056 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.057 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:45.057 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6807", 2022-08-01T19:43:45.057 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:43:45.058 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:45.058 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:45.059 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.059 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6803/23585", 2022-08-01T19:43:45.060 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6805/23585", 2022-08-01T19:43:45.060 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6809/23585", 2022-08-01T19:43:45.060 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6807/23585", 2022-08-01T19:43:45.061 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:43:45.061 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:43:45.062 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:43:45.062 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:45.063 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.063 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.064 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:43:45.064 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "ac4e755a-7fe7-46cb-8407-852d5af9ffc5", 2022-08-01T19:43:45.064 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:43:45.065 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:43:45.065 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:43:45.066 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:43:45.066 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:43:45.067 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:43:45.067 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:43:45.068 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 145, 2022-08-01T19:43:45.068 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:43:45.069 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:43:45.069 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:43:45.069 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:45.070 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.071 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:45.071 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6820", 2022-08-01T19:43:45.072 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:43:45.072 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.073 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.073 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:45.074 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6821", 2022-08-01T19:43:45.074 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:43:45.075 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:45.075 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:45.076 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.076 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:43:45.077 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:45.077 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.077 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:45.078 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6823", 2022-08-01T19:43:45.079 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:43:45.079 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.080 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.080 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:45.081 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6825", 2022-08-01T19:43:45.081 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:43:45.082 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:45.082 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:45.087 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.088 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:43:45.088 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:45.089 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.089 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:45.090 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6831", 2022-08-01T19:43:45.091 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:43:45.092 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.092 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.093 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:45.093 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6833", 2022-08-01T19:43:45.094 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:43:45.094 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:45.095 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:45.095 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.096 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:43:45.097 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:45.097 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.098 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:45.098 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6827", 2022-08-01T19:43:45.099 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:43:45.100 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.100 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.101 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:45.101 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6829", 2022-08-01T19:43:45.102 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:43:45.103 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:45.103 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:45.104 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.104 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6821/23587", 2022-08-01T19:43:45.105 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6825/23587", 2022-08-01T19:43:45.106 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6833/23587", 2022-08-01T19:43:45.106 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6829/23587", 2022-08-01T19:43:45.108 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:43:45.108 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:43:45.109 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:43:45.110 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:45.110 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.111 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.111 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:43:45.111 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "874761e2-6410-40fc-b60c-e58abe5ce489", 2022-08-01T19:43:45.113 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:43:45.113 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:43:45.114 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:43:45.114 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:43:45.115 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:43:45.115 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:43:45.116 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:43:45.116 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 145, 2022-08-01T19:43:45.116 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:43:45.117 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:43:45.117 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:43:45.117 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:45.118 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.119 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:45.119 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6818", 2022-08-01T19:43:45.120 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:43:45.121 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.121 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.121 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:45.122 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6819", 2022-08-01T19:43:45.122 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:43:45.123 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:45.124 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:45.125 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.126 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:43:45.126 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:45.127 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.128 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:45.132 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6822", 2022-08-01T19:43:45.134 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:43:45.134 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.135 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.135 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:45.135 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6824", 2022-08-01T19:43:45.136 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:43:45.136 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:45.136 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:45.137 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.137 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:43:45.137 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:45.138 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.138 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:45.139 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6830", 2022-08-01T19:43:45.139 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:43:45.139 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.140 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.140 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:45.140 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6832", 2022-08-01T19:43:45.141 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:43:45.141 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:45.142 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:45.142 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.142 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:43:45.143 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:45.143 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.143 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:45.143 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6826", 2022-08-01T19:43:45.143 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:43:45.144 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.144 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.144 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:45.144 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6828", 2022-08-01T19:43:45.144 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:43:45.145 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:45.145 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:45.145 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.148 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6819/23586", 2022-08-01T19:43:45.148 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6824/23586", 2022-08-01T19:43:45.148 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6832/23586", 2022-08-01T19:43:45.149 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6828/23586", 2022-08-01T19:43:45.149 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:43:45.149 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:43:45.150 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:43:45.150 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:45.150 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:45.150 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:43:45.151 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_xinfo": [ 2022-08-01T19:43:45.151 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.151 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:43:45.151 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:43:45.152 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:43:45.152 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:43:45.152 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:43:45.152 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:43:45.153 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.588817+0000", 2022-08-01T19:43:45.153 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:43:45.153 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.153 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.153 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:43:45.154 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:43:45.154 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:43:45.154 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:43:45.154 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:43:45.155 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:43:45.155 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.782753+0000", 2022-08-01T19:43:45.156 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:43:45.156 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.156 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.157 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:43:45.157 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:43:45.157 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:43:45.157 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:43:45.160 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:43:45.160 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:43:45.161 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.837924+0000", 2022-08-01T19:43:45.161 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:43:45.162 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.163 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.163 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:43:45.164 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:43:45.164 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:43:45.164 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:43:45.165 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:43:45.165 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:43:45.167 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.839948+0000", 2022-08-01T19:43:45.167 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:43:45.168 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.173 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.173 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:43:45.174 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:43:45.174 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:43:45.176 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:43:45.177 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:43:45.177 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:43:45.178 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.112801+0000", 2022-08-01T19:43:45.178 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:43:45.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.181 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.182 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:43:45.182 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:43:45.184 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:43:45.184 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:43:45.185 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:43:45.185 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:43:45.186 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.106704+0000", 2022-08-01T19:43:45.186 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:43:45.187 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.187 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.187 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:43:45.188 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:43:45.188 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:43:45.190 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:43:45.191 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:43:45.191 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:43:45.192 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.893170+0000", 2022-08-01T19:43:45.192 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:43:45.193 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.193 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:45.195 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:43:45.196 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:43:45.196 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:43:45.197 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:43:45.197 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:43:45.197 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:43:45.198 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.886947+0000", 2022-08-01T19:43:45.199 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:43:45.206 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:45.206 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:43:45.207 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap": [], 2022-08-01T19:43:45.207 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap_items": [], 2022-08-01T19:43:45.208 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_temp": [], 2022-08-01T19:43:45.208 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_temp": [], 2022-08-01T19:43:45.209 INFO:teuthology.orchestra.run.smithi017.stdout: "blacklist": {}, 2022-08-01T19:43:45.210 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profiles": { 2022-08-01T19:43:45.210 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_data": { 2022-08-01T19:43:45.211 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-device-class": "", 2022-08-01T19:43:45.211 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:43:45.212 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-root": "default", 2022-08-01T19:43:45.213 INFO:teuthology.orchestra.run.smithi017.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T19:43:45.213 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:43:45.213 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "2", 2022-08-01T19:43:45.213 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:43:45.214 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:43:45.214 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van", 2022-08-01T19:43:45.214 INFO:teuthology.orchestra.run.smithi017.stdout: "w": "8" 2022-08-01T19:43:45.214 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.215 INFO:teuthology.orchestra.run.smithi017.stdout: "default": { 2022-08-01T19:43:45.215 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:43:45.215 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:43:45.216 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "1", 2022-08-01T19:43:45.216 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:43:45.216 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:43:45.217 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van" 2022-08-01T19:43:45.217 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:45.217 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:45.217 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps_queue": [], 2022-08-01T19:43:45.217 INFO:teuthology.orchestra.run.smithi017.stdout: "new_removed_snaps": [], 2022-08-01T19:43:45.218 INFO:teuthology.orchestra.run.smithi017.stdout: "new_purged_snaps": [], 2022-08-01T19:43:45.218 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_node_flags": {}, 2022-08-01T19:43:45.221 INFO:teuthology.orchestra.run.smithi017.stdout: "device_class_flags": {} 2022-08-01T19:43:45.222 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:43:45.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.617+0000 7f17f0dee700 1 -- 172.21.15.17:0/325518666 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f17cc053950 msgr2=0x7f17cc055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:45.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.617+0000 7f17f0dee700 1 --2- 172.21.15.17:0/325518666 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f17cc053950 0x7f17cc055e10 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f17e000d000 tx=0x7f17e0007a00).stop 2022-08-01T19:43:45.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.617+0000 7f17f0dee700 1 -- 172.21.15.17:0/325518666 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17ec0ad750 msgr2=0x7f17ec155a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:45.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.617+0000 7f17f0dee700 1 --2- 172.21.15.17:0/325518666 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17ec0ad750 0x7f17ec155a40 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 rx=0x7f17dc0032f0 tx=0x7f17dc016000).stop 2022-08-01T19:43:45.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.617+0000 7f17f0dee700 1 -- 172.21.15.17:0/325518666 shutdown_connections 2022-08-01T19:43:45.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.617+0000 7f17f0dee700 1 --2- 172.21.15.17:0/325518666 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f17cc053950 0x7f17cc055e10 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:45.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.617+0000 7f17f0dee700 1 --2- 172.21.15.17:0/325518666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17ec1405d0 0x7f17ec155f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:45.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.617+0000 7f17f0dee700 1 --2- 172.21.15.17:0/325518666 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17ec0ad750 0x7f17ec155a40 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:45.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.617+0000 7f17f0dee700 1 --2- 172.21.15.17:0/325518666 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f17ec0614b0 0x7f17ec05cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:45.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.617+0000 7f17f0dee700 1 -- 172.21.15.17:0/325518666 >> 172.21.15.17:0/325518666 conn(0x7f17ec066f70 msgr2=0x7f17ec0b7c00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:45.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.617+0000 7f17f0dee700 1 -- 172.21.15.17:0/325518666 shutdown_connections 2022-08-01T19:43:45.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:44.617+0000 7f17f0dee700 1 -- 172.21.15.17:0/325518666 wait complete. 2022-08-01T19:43:45.234 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:43:45.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.345+0000 7fc3babe2700 1 Processor -- start 2022-08-01T19:43:45.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.345+0000 7fc3babe2700 1 -- start start 2022-08-01T19:43:45.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.345+0000 7fc3babe2700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fc3b413a230 0x7fc3b40acd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:45.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.345+0000 7fc3babe2700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3b40ad200 con 0x7fc3b4139e50 2022-08-01T19:43:45.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.345+0000 7fc3babe2700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3b4060ed0 con 0x7fc3b40b2350 2022-08-01T19:43:45.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.345+0000 7fc3babe2700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3b40b28c0 con 0x7fc3b413a230 2022-08-01T19:43:45.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.345+0000 7fc3b897e700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc3b4139e50 0x7fc3b4143680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37412/0 (socket says 172.21.15.17:37412) 2022-08-01T19:43:45.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.345+0000 7fc3b897e700 1 -- 172.21.15.17:0/1780305766 learned_addr learned my addr 172.21.15.17:0/1780305766 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:45.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.345+0000 7fc3b917f700 1 --2- 172.21.15.17:0/1780305766 >> v2:172.21.15.77:3300/0 conn(0x7fc3b413a230 0x7fc3b40acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:45.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.345+0000 7fc3b37fe700 1 -- 172.21.15.17:0/1780305766 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2064211166 0 0) 0x7fc3a4001110 con 0x7fc3b4139e50 2022-08-01T19:43:45.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.349+0000 7fc3b37fe700 1 -- 172.21.15.17:0/1780305766 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3a0003260 con 0x7fc3b4139e50 2022-08-01T19:43:45.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.349+0000 7fc3b37fe700 1 -- 172.21.15.17:0/1780305766 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1114444036 0 0) 0x7fc3b4060ed0 con 0x7fc3b40b2350 2022-08-01T19:43:45.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.349+0000 7fc3b37fe700 1 -- 172.21.15.17:0/1780305766 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3a4001110 con 0x7fc3b40b2350 2022-08-01T19:43:45.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.349+0000 7fc3b37fe700 1 -- 172.21.15.17:0/1780305766 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2998965155 0 0) 0x7fc3a0003260 con 0x7fc3b4139e50 2022-08-01T19:43:45.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.349+0000 7fc3b917f700 1 -- 172.21.15.17:0/1780305766 >> v1:172.21.15.77:6789/0 conn(0x7fc3b40b2350 legacy=0x7fc3b40b6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:45.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.349+0000 7fc3b917f700 1 -- 172.21.15.17:0/1780305766 >> v1:172.21.15.17:6789/0 conn(0x7fc3b4139e50 legacy=0x7fc3b4143680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:45.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.349+0000 7fc3b917f700 1 -- 172.21.15.17:0/1780305766 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b40b6e60 con 0x7fc3b413a230 2022-08-01T19:43:45.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.349+0000 7fc3b917f700 1 --2- 172.21.15.17:0/1780305766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3b413a230 0x7fc3b40acd10 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7fc3a40012a0 tx=0x7fc3ac0091d0).ready entity=mon.1 client_cookie=9849f6665e7a14e2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:45.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.349+0000 7fc3b37fe700 1 -- 172.21.15.17:0/1780305766 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3a40017f0 con 0x7fc3b413a230 2022-08-01T19:43:45.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.349+0000 7fc3b37fe700 1 -- 172.21.15.17:0/1780305766 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ac00f510 con 0x7fc3b413a230 2022-08-01T19:43:45.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.349+0000 7fc3b37fe700 1 -- 172.21.15.17:0/1780305766 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3ac00f680 con 0x7fc3b413a230 2022-08-01T19:43:45.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.349+0000 7fc3babe2700 1 -- 172.21.15.17:0/1780305766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3b413a230 msgr2=0x7fc3b40acd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:45.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.349+0000 7fc3babe2700 1 --2- 172.21.15.17:0/1780305766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3b413a230 0x7fc3b40acd10 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7fc3a40012a0 tx=0x7fc3ac0091d0).stop 2022-08-01T19:43:45.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.349+0000 7fc3babe2700 1 -- 172.21.15.17:0/1780305766 shutdown_connections 2022-08-01T19:43:45.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.349+0000 7fc3babe2700 1 --2- 172.21.15.17:0/1780305766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3b413a230 0x7fc3b40acd10 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:45.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.349+0000 7fc3babe2700 1 -- 172.21.15.17:0/1780305766 >> 172.21.15.17:0/1780305766 conn(0x7fc3b40b3610 msgr2=0x7fc3b40b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:45.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.349+0000 7fc3babe2700 1 -- 172.21.15.17:0/1780305766 shutdown_connections 2022-08-01T19:43:45.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.349+0000 7fc3babe2700 1 -- 172.21.15.17:0/1780305766 wait complete. 2022-08-01T19:43:45.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.353+0000 7fc3babe2700 1 Processor -- start 2022-08-01T19:43:45.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.353+0000 7fc3babe2700 1 -- start start 2022-08-01T19:43:45.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.353+0000 7fc3babe2700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc3b40b2350 0x7fc3b4142c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:45.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.353+0000 7fc3babe2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3b4139e50 0x7fc3b4154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:45.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.353+0000 7fc3babe2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3b413a230 0x7fc3b4154530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:45.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.353+0000 7fc3babe2700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b4154a20 con 0x7fc3b4139e50 2022-08-01T19:43:45.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.353+0000 7fc3babe2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b413bb00 con 0x7fc3b413a230 2022-08-01T19:43:45.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.353+0000 7fc3babe2700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b413bd30 con 0x7fc3b40b2350 2022-08-01T19:43:45.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.353+0000 7fc3b3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3b4139e50 0x7fc3b4154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:45.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.353+0000 7fc3b3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3b4139e50 0x7fc3b4154040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45226/0 (socket says 172.21.15.17:45226) 2022-08-01T19:43:45.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.353+0000 7fc3b3fff700 1 -- 172.21.15.17:0/3031855019 learned_addr learned my addr 172.21.15.17:0/3031855019 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:45.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.353+0000 7fc3b897e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc3b40b2350 0x7fc3b4142c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:45.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.353+0000 7fc3b917f700 1 --2- 172.21.15.17:0/3031855019 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3b413a230 0x7fc3b4154530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:45.376 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.353+0000 7fc3b3fff700 1 -- 172.21.15.17:0/3031855019 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc3b40b2350 msgr2=0x7fc3b4142c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:45.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.353+0000 7fc3b3fff700 1 --2- 172.21.15.17:0/3031855019 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc3b40b2350 0x7fc3b4142c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:45.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.353+0000 7fc3b3fff700 1 -- 172.21.15.17:0/3031855019 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3b413a230 msgr2=0x7fc3b4154530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:45.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.353+0000 7fc3b3fff700 1 --2- 172.21.15.17:0/3031855019 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3b413a230 0x7fc3b4154530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:45.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.353+0000 7fc3b3fff700 1 -- 172.21.15.17:0/3031855019 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3ac009470 con 0x7fc3b4139e50 2022-08-01T19:43:45.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.353+0000 7fc3b3fff700 1 --2- 172.21.15.17:0/3031855019 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3b4139e50 0x7fc3b4154040 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 rx=0x7fc3b4154b60 tx=0x7fc3a8007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:45.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.357+0000 7fc3b1ffb700 1 -- 172.21.15.17:0/3031855019 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3a8012400 con 0x7fc3b4139e50 2022-08-01T19:43:45.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.357+0000 7fc3b1ffb700 1 -- 172.21.15.17:0/3031855019 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a8012560 con 0x7fc3b4139e50 2022-08-01T19:43:45.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.357+0000 7fc3b1ffb700 1 -- 172.21.15.17:0/3031855019 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3a80128c0 con 0x7fc3b4139e50 2022-08-01T19:43:45.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.357+0000 7fc3babe2700 1 -- 172.21.15.17:0/3031855019 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3b413bfd0 con 0x7fc3b4139e50 2022-08-01T19:43:45.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.357+0000 7fc3babe2700 1 -- 172.21.15.17:0/3031855019 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b4064f50 con 0x7fc3b4139e50 2022-08-01T19:43:45.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.357+0000 7fc3babe2700 1 -- 172.21.15.17:0/3031855019 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3b4061130 con 0x7fc3b4139e50 2022-08-01T19:43:45.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.357+0000 7fc3b1ffb700 1 -- 172.21.15.17:0/3031855019 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc3a8019030 con 0x7fc3b4139e50 2022-08-01T19:43:45.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.357+0000 7fc3b1ffb700 1 --2- 172.21.15.17:0/3031855019 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc394053950 0x7fc394055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:45.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.357+0000 7fc3b1ffb700 1 -- 172.21.15.17:0/3031855019 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fc3a806b530 con 0x7fc3b4139e50 2022-08-01T19:43:45.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.357+0000 7fc3b897e700 1 --2- 172.21.15.17:0/3031855019 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc394053950 0x7fc394055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:45.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.361+0000 7fc3b897e700 1 --2- 172.21.15.17:0/3031855019 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc394053950 0x7fc394055e10 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7fc3a400d000 tx=0x7fc3a4000bc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:45.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.365+0000 7fc3b1ffb700 1 -- 172.21.15.17:0/3031855019 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc3a80421e0 con 0x7fc3b4139e50 2022-08-01T19:43:45.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.525+0000 7fc3babe2700 1 -- 172.21.15.17:0/3031855019 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3b40574e0 con 0x7fc3b4139e50 2022-08-01T19:43:45.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.525+0000 7fc3b1ffb700 1 -- 172.21.15.17:0/3031855019 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+2492 (secure 0 0 0) 0x7fc3a8045800 con 0x7fc3b4139e50 2022-08-01T19:43:45.545 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:45.546 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":76,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7211,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/872530952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":872530952},{"type":"v1","addr":"172.21.15.77:6835","nonce":872530952}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":74},{"gid":8358,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/4261332014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4261332014},{"type":"v1","addr":"172.21.15.17:6833","nonce":4261332014}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":73}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:43:34.586097+0000","modified":"2022-08-01T19:43:36.271865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8364},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8364":{"gid":8364,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/2832743740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2832743740},{"type":"v1","addr":"172.21.15.17:6835","nonce":2832743740}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2022-08-01T19:43:45.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.529+0000 7fc3babe2700 1 -- 172.21.15.17:0/3031855019 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc394053950 msgr2=0x7fc394055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:45.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.529+0000 7fc3babe2700 1 --2- 172.21.15.17:0/3031855019 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc394053950 0x7fc394055e10 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7fc3a400d000 tx=0x7fc3a4000bc0).stop 2022-08-01T19:43:45.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.529+0000 7fc3babe2700 1 -- 172.21.15.17:0/3031855019 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3b4139e50 msgr2=0x7fc3b4154040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:45.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.529+0000 7fc3babe2700 1 --2- 172.21.15.17:0/3031855019 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3b4139e50 0x7fc3b4154040 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 rx=0x7fc3b4154b60 tx=0x7fc3a8007ae0).stop 2022-08-01T19:43:45.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.529+0000 7fc3babe2700 1 -- 172.21.15.17:0/3031855019 shutdown_connections 2022-08-01T19:43:45.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.529+0000 7fc3babe2700 1 --2- 172.21.15.17:0/3031855019 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc394053950 0x7fc394055e10 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:45.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.529+0000 7fc3babe2700 1 --2- 172.21.15.17:0/3031855019 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3b413a230 0x7fc3b4154530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:45.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.529+0000 7fc3babe2700 1 --2- 172.21.15.17:0/3031855019 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3b4139e50 0x7fc3b4154040 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:45.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.529+0000 7fc3babe2700 1 --2- 172.21.15.17:0/3031855019 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc3b40b2350 0x7fc3b4142c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:45.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.529+0000 7fc3babe2700 1 -- 172.21.15.17:0/3031855019 >> 172.21.15.17:0/3031855019 conn(0x7fc3b40b3610 msgr2=0x7fc3b4141350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:45.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.529+0000 7fc3babe2700 1 -- 172.21.15.17:0/3031855019 shutdown_connections 2022-08-01T19:43:45.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.529+0000 7fc3babe2700 1 -- 172.21.15.17:0/3031855019 wait complete. 2022-08-01T19:43:45.560 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 76 2022-08-01T19:43:45.567 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-08-01T19:43:45.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.677+0000 7f638df06700 1 Processor -- start 2022-08-01T19:43:45.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.677+0000 7f638df06700 1 -- start start 2022-08-01T19:43:45.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.677+0000 7f638df06700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f63880b5630 0x7f63880af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:45.692 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.677+0000 7f638df06700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f63880b07a0 0x7f6388147730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:45.692 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.677+0000 7f638df06700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63880b5a10 con 0x7f6388067730 2022-08-01T19:43:45.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.677+0000 7f638df06700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6388145470 con 0x7f63880b5630 2022-08-01T19:43:45.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.677+0000 7f638df06700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63880ae180 con 0x7f63880b07a0 2022-08-01T19:43:45.693 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.677+0000 7f638cf04700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f63880b5630 0x7f63880af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:45.694 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.677+0000 7f638cf04700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f63880b5630 0x7f63880af6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45232/0 (socket says 172.21.15.17:45232) 2022-08-01T19:43:45.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.677+0000 7f638cf04700 1 -- 172.21.15.17:0/2000543821 learned_addr learned my addr 172.21.15.17:0/2000543821 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:45.695 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.677+0000 7f6387fff700 1 --2- 172.21.15.17:0/2000543821 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f63880b07a0 0x7f6388147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:45.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.677+0000 7f638cf04700 1 -- 172.21.15.17:0/2000543821 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f63880b07a0 msgr2=0x7f6388147730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:45.696 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.677+0000 7f638cf04700 1 --2- 172.21.15.17:0/2000543821 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f63880b07a0 0x7f6388147730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:45.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.677+0000 7f638cf04700 1 -- 172.21.15.17:0/2000543821 >> v1:172.21.15.77:6789/0 conn(0x7f6388067730 legacy=0x7f63880acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:45.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.677+0000 7f638cf04700 1 -- 172.21.15.17:0/2000543821 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63880ac990 con 0x7f63880b5630 2022-08-01T19:43:45.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.677+0000 7f6387fff700 1 --2- 172.21.15.17:0/2000543821 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f63880b07a0 0x7f6388147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:43:45.697 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.677+0000 7f638cf04700 1 --2- 172.21.15.17:0/2000543821 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63880b5630 0x7f63880af6e0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 rx=0x7f6370002f70 tx=0x7f6370006eb0).ready entity=mon.0 client_cookie=9baccfe4ee58b26c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:45.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.677+0000 7f63877fe700 1 -- 172.21.15.17:0/2000543821 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f637000d5c0 con 0x7f63880b5630 2022-08-01T19:43:45.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.677+0000 7f63877fe700 1 -- 172.21.15.17:0/2000543821 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6370000e90 con 0x7f63880b5630 2022-08-01T19:43:45.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.677+0000 7f63877fe700 1 -- 172.21.15.17:0/2000543821 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f637000d950 con 0x7f63880b5630 2022-08-01T19:43:45.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.677+0000 7f638df06700 1 -- 172.21.15.17:0/2000543821 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63880b5630 msgr2=0x7f63880af6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:45.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.677+0000 7f638df06700 1 --2- 172.21.15.17:0/2000543821 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63880b5630 0x7f63880af6e0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 rx=0x7f6370002f70 tx=0x7f6370006eb0).stop 2022-08-01T19:43:45.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.677+0000 7f638df06700 1 -- 172.21.15.17:0/2000543821 shutdown_connections 2022-08-01T19:43:45.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.677+0000 7f638df06700 1 --2- 172.21.15.17:0/2000543821 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f63880b07a0 0x7f6388147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:45.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.677+0000 7f638df06700 1 --2- 172.21.15.17:0/2000543821 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63880b5630 0x7f63880af6e0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:45.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.677+0000 7f638df06700 1 -- 172.21.15.17:0/2000543821 >> 172.21.15.17:0/2000543821 conn(0x7f63880626f0 msgr2=0x7f6388062ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:45.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.677+0000 7f638df06700 1 -- 172.21.15.17:0/2000543821 shutdown_connections 2022-08-01T19:43:45.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.677+0000 7f638df06700 1 -- 172.21.15.17:0/2000543821 wait complete. 2022-08-01T19:43:45.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.681+0000 7f638df06700 1 Processor -- start 2022-08-01T19:43:45.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.681+0000 7f638df06700 1 -- start start 2022-08-01T19:43:45.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.681+0000 7f638df06700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6388067730 0x7f63880aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:45.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.681+0000 7f638df06700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63880b07a0 0x7f63881570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:45.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.681+0000 7f638df06700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63880b5630 0x7f63881575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:45.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.681+0000 7f638df06700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63880654c0 con 0x7f63880b07a0 2022-08-01T19:43:45.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.681+0000 7f638df06700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63881549f0 con 0x7f63880b5630 2022-08-01T19:43:45.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.681+0000 7f638df06700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6388064ce0 con 0x7f6388067730 2022-08-01T19:43:45.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.681+0000 7f6387fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63880b07a0 0x7f63881570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:45.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.681+0000 7f6387fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63880b07a0 0x7f63881570c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45240/0 (socket says 172.21.15.17:45240) 2022-08-01T19:43:45.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.681+0000 7f6387fff700 1 -- 172.21.15.17:0/2454038293 learned_addr learned my addr 172.21.15.17:0/2454038293 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:45.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.681+0000 7f638cf04700 1 --2- 172.21.15.17:0/2454038293 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6388067730 0x7f63880aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:45.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.681+0000 7f638d705700 1 --2- 172.21.15.17:0/2454038293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63880b5630 0x7f63881575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:45.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.681+0000 7f6387fff700 1 -- 172.21.15.17:0/2454038293 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6388067730 msgr2=0x7f63880aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:45.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.681+0000 7f6387fff700 1 --2- 172.21.15.17:0/2454038293 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6388067730 0x7f63880aeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:45.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.681+0000 7f6387fff700 1 -- 172.21.15.17:0/2454038293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63880b5630 msgr2=0x7f63881575b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:45.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.681+0000 7f6387fff700 1 --2- 172.21.15.17:0/2454038293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63880b5630 0x7f63881575b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:45.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.681+0000 7f6387fff700 1 -- 172.21.15.17:0/2454038293 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637000b1e0 con 0x7f63880b07a0 2022-08-01T19:43:45.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.681+0000 7f638cf04700 1 --2- 172.21.15.17:0/2454038293 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6388067730 0x7f63880aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:43:45.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.681+0000 7f6387fff700 1 --2- 172.21.15.17:0/2454038293 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63880b07a0 0x7f63881570c0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 rx=0x7f637800a000 tx=0x7f6378008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:45.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.681+0000 7f6385ffb700 1 -- 172.21.15.17:0/2454038293 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f637800c810 con 0x7f63880b07a0 2022-08-01T19:43:45.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.681+0000 7f6385ffb700 1 -- 172.21.15.17:0/2454038293 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f637800c970 con 0x7f63880b07a0 2022-08-01T19:43:45.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.681+0000 7f638df06700 1 -- 172.21.15.17:0/2454038293 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6388064f80 con 0x7f63880b07a0 2022-08-01T19:43:45.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.681+0000 7f6385ffb700 1 -- 172.21.15.17:0/2454038293 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f637800ccd0 con 0x7f63880b07a0 2022-08-01T19:43:45.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.681+0000 7f638df06700 1 -- 172.21.15.17:0/2454038293 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63880a8940 con 0x7f63880b07a0 2022-08-01T19:43:45.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.685+0000 7f638df06700 1 -- 172.21.15.17:0/2454038293 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63880b1680 con 0x7f63880b07a0 2022-08-01T19:43:45.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.685+0000 7f6385ffb700 1 -- 172.21.15.17:0/2454038293 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6378013430 con 0x7f63880b07a0 2022-08-01T19:43:45.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.685+0000 7f6385ffb700 1 --2- 172.21.15.17:0/2454038293 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f636c053a10 0x7f636c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:45.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.685+0000 7f6385ffb700 1 -- 172.21.15.17:0/2454038293 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f637806c080 con 0x7f63880b07a0 2022-08-01T19:43:45.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.685+0000 7f638cf04700 1 --2- 172.21.15.17:0/2454038293 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f636c053a10 0x7f636c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:45.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.685+0000 7f638cf04700 1 --2- 172.21.15.17:0/2454038293 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f636c053a10 0x7f636c055ed0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f637001a000 tx=0x7f6370015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:45.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.689+0000 7f6385ffb700 1 -- 172.21.15.17:0/2454038293 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6378042cd0 con 0x7f63880b07a0 2022-08-01T19:43:45.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:45.849+0000 7f638df06700 1 -- 172.21.15.17:0/2454038293 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6388061020 con 0x7f63880b07a0 2022-08-01T19:43:46.674 INFO:tasks.ceph.mon.a.smithi017.stderr:2022-08-01T19:43:46.669+0000 7f6a53773700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-08-01T19:43:46.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.677+0000 7f6385ffb700 1 -- 172.21.15.17:0/2454038293 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v77) v1 ==== 161+0+0 (secure 0 0 0) 0x7f637800ee60 con 0x7f63880b07a0 2022-08-01T19:43:46.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.681+0000 7f638df06700 1 -- 172.21.15.17:0/2454038293 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f636c053a10 msgr2=0x7f636c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:46.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.681+0000 7f638df06700 1 --2- 172.21.15.17:0/2454038293 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f636c053a10 0x7f636c055ed0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f637001a000 tx=0x7f6370015000).stop 2022-08-01T19:43:46.687 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.681+0000 7f638df06700 1 -- 172.21.15.17:0/2454038293 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63880b07a0 msgr2=0x7f63881570c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:46.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.681+0000 7f638df06700 1 --2- 172.21.15.17:0/2454038293 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63880b07a0 0x7f63881570c0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 rx=0x7f637800a000 tx=0x7f6378008040).stop 2022-08-01T19:43:46.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.681+0000 7f638df06700 1 -- 172.21.15.17:0/2454038293 shutdown_connections 2022-08-01T19:43:46.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.681+0000 7f638df06700 1 --2- 172.21.15.17:0/2454038293 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f636c053a10 0x7f636c055ed0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:46.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.681+0000 7f638df06700 1 --2- 172.21.15.17:0/2454038293 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63880b5630 0x7f63881575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:46.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.681+0000 7f638df06700 1 --2- 172.21.15.17:0/2454038293 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63880b07a0 0x7f63881570c0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:46.690 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.681+0000 7f638df06700 1 --2- 172.21.15.17:0/2454038293 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6388067730 0x7f63880aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:46.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.681+0000 7f638df06700 1 -- 172.21.15.17:0/2454038293 >> 172.21.15.17:0/2454038293 conn(0x7f63880626f0 msgr2=0x7f6388143b90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:46.691 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.685+0000 7f638df06700 1 -- 172.21.15.17:0/2454038293 shutdown_connections 2022-08-01T19:43:46.698 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.685+0000 7f638df06700 1 -- 172.21.15.17:0/2454038293 wait complete. 2022-08-01T19:43:46.699 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-08-01T19:43:46.707 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:43:46.714 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2022-08-01T19:43:46.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.829+0000 7ff6cb59e700 1 Processor -- start 2022-08-01T19:43:46.834 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.829+0000 7ff6cb59e700 1 -- start start 2022-08-01T19:43:46.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.833+0000 7ff6cb59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6cc0b3260 0x7ff6cc1435c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:46.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.833+0000 7ff6cb59e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ff6cc0ad000 0x7ff6cc13fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:46.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.833+0000 7ff6cb59e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6cc143ab0 con 0x7ff6cc0b08a0 2022-08-01T19:43:46.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.833+0000 7ff6cb59e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6cc0b0c80 con 0x7ff6cc0ad000 2022-08-01T19:43:46.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.833+0000 7ff6cb59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc0ae810 con 0x7ff6cc0b3260 2022-08-01T19:43:46.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.833+0000 7ff6ca59c700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7ff6cc0b08a0 0x7ff6cc142eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:54252/0 (socket says 172.21.15.17:54252) 2022-08-01T19:43:46.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.833+0000 7ff6ca59c700 1 -- 172.21.15.17:0/3881010682 learned_addr learned my addr 172.21.15.17:0/3881010682 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:46.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.833+0000 7ff6cad9d700 1 --2- 172.21.15.17:0/3881010682 >> v2:172.21.15.77:3300/0 conn(0x7ff6cc0ad000 0x7ff6cc13fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:46.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.833+0000 7ff6c3fff700 1 --2- 172.21.15.17:0/3881010682 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6cc0b3260 0x7ff6cc1435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:46.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.833+0000 7ff6c9d9b700 1 -- 172.21.15.17:0/3881010682 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3291192183 0 0) 0x7ff6cc143ab0 con 0x7ff6cc0b08a0 2022-08-01T19:43:46.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.833+0000 7ff6c9d9b700 1 -- 172.21.15.17:0/3881010682 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6ac003260 con 0x7ff6cc0b08a0 2022-08-01T19:43:46.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.833+0000 7ff6cad9d700 1 -- 172.21.15.17:0/3881010682 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6cc0b3260 msgr2=0x7ff6cc1435c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:46.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.833+0000 7ff6cad9d700 1 --2- 172.21.15.17:0/3881010682 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6cc0b3260 0x7ff6cc1435c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:46.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.833+0000 7ff6cad9d700 1 -- 172.21.15.17:0/3881010682 >> v1:172.21.15.77:6789/0 conn(0x7ff6cc0b08a0 legacy=0x7ff6cc142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:46.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.833+0000 7ff6cad9d700 1 -- 172.21.15.17:0/3881010682 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6cc0aeab0 con 0x7ff6cc0ad000 2022-08-01T19:43:46.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.833+0000 7ff6cad9d700 1 --2- 172.21.15.17:0/3881010682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6cc0ad000 0x7ff6cc13fa30 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7ff6c40032f0 tx=0x7ff6c40091d0).ready entity=mon.1 client_cookie=2585fe34e9d10276 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:46.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.833+0000 7ff6c9d9b700 1 -- 172.21.15.17:0/3881010682 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6c400b600 con 0x7ff6cc0ad000 2022-08-01T19:43:46.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.833+0000 7ff6c9d9b700 1 -- 172.21.15.17:0/3881010682 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6c400b760 con 0x7ff6cc0ad000 2022-08-01T19:43:46.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.833+0000 7ff6c9d9b700 1 -- 172.21.15.17:0/3881010682 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6c400b8d0 con 0x7ff6cc0ad000 2022-08-01T19:43:46.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.833+0000 7ff6cb59e700 1 -- 172.21.15.17:0/3881010682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6cc0ad000 msgr2=0x7ff6cc13fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:46.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.833+0000 7ff6cb59e700 1 --2- 172.21.15.17:0/3881010682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6cc0ad000 0x7ff6cc13fa30 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7ff6c40032f0 tx=0x7ff6c40091d0).stop 2022-08-01T19:43:46.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.833+0000 7ff6cb59e700 1 -- 172.21.15.17:0/3881010682 shutdown_connections 2022-08-01T19:43:46.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.833+0000 7ff6cb59e700 1 --2- 172.21.15.17:0/3881010682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6cc0ad000 0x7ff6cc13fa30 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:46.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.833+0000 7ff6cb59e700 1 --2- 172.21.15.17:0/3881010682 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6cc0b3260 0x7ff6cc1435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:46.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.833+0000 7ff6cb59e700 1 -- 172.21.15.17:0/3881010682 >> 172.21.15.17:0/3881010682 conn(0x7ff6cc0612d0 msgr2=0x7ff6cc062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:46.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.837+0000 7ff6cb59e700 1 -- 172.21.15.17:0/3881010682 shutdown_connections 2022-08-01T19:43:46.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.837+0000 7ff6cb59e700 1 -- 172.21.15.17:0/3881010682 wait complete. 2022-08-01T19:43:46.905 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.837+0000 7ff6cb59e700 1 Processor -- start 2022-08-01T19:43:46.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.837+0000 7ff6cb59e700 1 -- start start 2022-08-01T19:43:46.907 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.837+0000 7ff6cb59e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6cc0ad000 0x7ff6cc13b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:46.907 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.837+0000 7ff6cb59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6cc0b08a0 0x7ff6cc0ab710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:46.907 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.837+0000 7ff6cb59e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6cc0b3260 0x7ff6cc1503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:46.907 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.837+0000 7ff6cb59e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc0ac390 con 0x7ff6cc0b3260 2022-08-01T19:43:46.908 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.837+0000 7ff6cb59e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc0abc00 con 0x7ff6cc0ad000 2022-08-01T19:43:46.909 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.837+0000 7ff6cb59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc0ab1d0 con 0x7ff6cc0b08a0 2022-08-01T19:43:46.909 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.837+0000 7ff6cad9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6cc0b3260 0x7ff6cc1503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:46.909 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.837+0000 7ff6cad9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6cc0b3260 0x7ff6cc1503c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45266/0 (socket says 172.21.15.17:45266) 2022-08-01T19:43:46.910 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.837+0000 7ff6cad9d700 1 -- 172.21.15.17:0/3228348895 learned_addr learned my addr 172.21.15.17:0/3228348895 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:46.910 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.837+0000 7ff6ca59c700 1 --2- 172.21.15.17:0/3228348895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6cc0ad000 0x7ff6cc13b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:46.910 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.837+0000 7ff6c3fff700 1 --2- 172.21.15.17:0/3228348895 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6cc0b08a0 0x7ff6cc0ab710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:46.910 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.837+0000 7ff6cad9d700 1 -- 172.21.15.17:0/3228348895 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6cc0b08a0 msgr2=0x7ff6cc0ab710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:46.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.837+0000 7ff6cad9d700 1 --2- 172.21.15.17:0/3228348895 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6cc0b08a0 0x7ff6cc0ab710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:46.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.837+0000 7ff6cad9d700 1 -- 172.21.15.17:0/3228348895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6cc0ad000 msgr2=0x7ff6cc13b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:46.912 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.837+0000 7ff6cad9d700 1 --2- 172.21.15.17:0/3228348895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6cc0ad000 0x7ff6cc13b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:46.912 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.837+0000 7ff6cad9d700 1 -- 172.21.15.17:0/3228348895 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c4009470 con 0x7ff6cc0b3260 2022-08-01T19:43:46.912 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.841+0000 7ff6cad9d700 1 --2- 172.21.15.17:0/3228348895 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6cc0b3260 0x7ff6cc1503c0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 rx=0x7ff6c40032f0 tx=0x7ff6c4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:46.913 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.841+0000 7ff6c37fe700 1 -- 172.21.15.17:0/3228348895 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6c400b8d0 con 0x7ff6cc0b3260 2022-08-01T19:43:46.918 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.841+0000 7ff6c37fe700 1 -- 172.21.15.17:0/3228348895 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6c401f3f0 con 0x7ff6cc0b3260 2022-08-01T19:43:46.918 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.841+0000 7ff6cb59e700 1 -- 172.21.15.17:0/3228348895 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6cc0ab470 con 0x7ff6cc0b3260 2022-08-01T19:43:46.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.841+0000 7ff6c37fe700 1 -- 172.21.15.17:0/3228348895 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6c400b600 con 0x7ff6cc0b3260 2022-08-01T19:43:46.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.841+0000 7ff6cb59e700 1 -- 172.21.15.17:0/3228348895 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6cc13db30 con 0x7ff6cc0b3260 2022-08-01T19:43:46.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.841+0000 7ff6cb59e700 1 -- 172.21.15.17:0/3228348895 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6cc0ac550 con 0x7ff6cc0b3260 2022-08-01T19:43:46.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.841+0000 7ff6c37fe700 1 -- 172.21.15.17:0/3228348895 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff6c4026030 con 0x7ff6cc0b3260 2022-08-01T19:43:46.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.841+0000 7ff6c37fe700 1 --2- 172.21.15.17:0/3228348895 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff6b0053b00 0x7ff6b0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:46.929 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.841+0000 7ff6c37fe700 1 -- 172.21.15.17:0/3228348895 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6506+0+0 (secure 0 0 0) 0x7ff6c406cbb0 con 0x7ff6cc0b3260 2022-08-01T19:43:46.930 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.841+0000 7ff6ca59c700 1 --2- 172.21.15.17:0/3228348895 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff6b0053b00 0x7ff6b0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:46.930 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.845+0000 7ff6ca59c700 1 --2- 172.21.15.17:0/3228348895 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff6b0053b00 0x7ff6b0055fc0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7ff6bc00d000 tx=0x7ff6bc009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:46.930 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:46.849+0000 7ff6c37fe700 1 -- 172.21.15.17:0/3228348895 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff6cc0ac550 con 0x7ff6cc0b3260 2022-08-01T19:43:47.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.017+0000 7ff6cb59e700 1 -- 172.21.15.17:0/3228348895 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6cc0ac550 con 0x7ff6cc0b3260 2022-08-01T19:43:47.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.017+0000 7ff6c37fe700 1 -- 172.21.15.17:0/3228348895 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+2132 (secure 0 0 0) 0x7ff6cc0ac550 con 0x7ff6cc0b3260 2022-08-01T19:43:47.022 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:47.023 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":77,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7211,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/872530952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":872530952},{"type":"v1","addr":"172.21.15.77:6835","nonce":872530952}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":74},{"gid":8358,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/4261332014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4261332014},{"type":"v1","addr":"172.21.15.17:6833","nonce":4261332014}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":73}],"filesystems":[{"mdsmap":{"epoch":77,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:43:34.586097+0000","modified":"2022-08-01T19:43:46.672430+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":149,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2022-08-01T19:43:47.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.021+0000 7ff6cb59e700 1 -- 172.21.15.17:0/3228348895 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff6b0053b00 msgr2=0x7ff6b0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:47.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.021+0000 7ff6cb59e700 1 --2- 172.21.15.17:0/3228348895 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff6b0053b00 0x7ff6b0055fc0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7ff6bc00d000 tx=0x7ff6bc009040).stop 2022-08-01T19:43:47.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.021+0000 7ff6cb59e700 1 -- 172.21.15.17:0/3228348895 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6cc0b3260 msgr2=0x7ff6cc1503c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:47.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.021+0000 7ff6cb59e700 1 --2- 172.21.15.17:0/3228348895 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6cc0b3260 0x7ff6cc1503c0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 rx=0x7ff6c40032f0 tx=0x7ff6c4016000).stop 2022-08-01T19:43:47.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.021+0000 7ff6cb59e700 1 -- 172.21.15.17:0/3228348895 shutdown_connections 2022-08-01T19:43:47.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.021+0000 7ff6cb59e700 1 --2- 172.21.15.17:0/3228348895 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff6b0053b00 0x7ff6b0055fc0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:47.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.021+0000 7ff6cb59e700 1 --2- 172.21.15.17:0/3228348895 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6cc0b3260 0x7ff6cc1503c0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:47.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.021+0000 7ff6cb59e700 1 --2- 172.21.15.17:0/3228348895 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff6cc0b08a0 0x7ff6cc0ab710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:47.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.021+0000 7ff6cb59e700 1 --2- 172.21.15.17:0/3228348895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6cc0ad000 0x7ff6cc13b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:47.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.021+0000 7ff6cb59e700 1 -- 172.21.15.17:0/3228348895 >> 172.21.15.17:0/3228348895 conn(0x7ff6cc0612d0 msgr2=0x7ff6cc142dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:47.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.021+0000 7ff6cb59e700 1 -- 172.21.15.17:0/3228348895 shutdown_connections 2022-08-01T19:43:47.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.021+0000 7ff6cb59e700 1 -- 172.21.15.17:0/3228348895 wait complete. 2022-08-01T19:43:47.038 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 77 2022-08-01T19:43:47.040 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-08-01T19:43:47.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.145+0000 7f31302ba700 1 Processor -- start 2022-08-01T19:43:47.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.145+0000 7f31302ba700 1 -- start start 2022-08-01T19:43:47.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.145+0000 7f31302ba700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f312805fc30 0x7f3128143a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:47.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.145+0000 7f31302ba700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3128060010 con 0x7f3128060960 2022-08-01T19:43:47.154 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.145+0000 7f31302ba700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31281477c0 con 0x7f31280ac690 2022-08-01T19:43:47.154 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.145+0000 7f31302ba700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31281433d0 con 0x7f312805fc30 2022-08-01T19:43:47.154 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.145+0000 7f312d855700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f3128060960 0x7f31281469a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37470/0 (socket says 172.21.15.17:37470) 2022-08-01T19:43:47.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.145+0000 7f312d855700 1 -- 172.21.15.17:0/1498220626 learned_addr learned my addr 172.21.15.17:0/1498220626 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:47.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.145+0000 7f312e056700 1 --2- 172.21.15.17:0/1498220626 >> v2:172.21.15.77:3300/0 conn(0x7f312805fc30 0x7f3128143a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:47.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.145+0000 7f312d054700 1 -- 172.21.15.17:0/1498220626 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 332008169 0 0) 0x7f3128060010 con 0x7f3128060960 2022-08-01T19:43:47.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.145+0000 7f312d054700 1 -- 172.21.15.17:0/1498220626 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3110003260 con 0x7f3128060960 2022-08-01T19:43:47.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.149+0000 7f312e056700 1 -- 172.21.15.17:0/1498220626 >> v1:172.21.15.77:6789/0 conn(0x7f31280ac690 legacy=0x7f31281470b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:47.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.149+0000 7f312d054700 1 -- 172.21.15.17:0/1498220626 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3752320565 0 0) 0x7f31281477c0 con 0x7f31280ac690 2022-08-01T19:43:47.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.149+0000 7f312e056700 1 -- 172.21.15.17:0/1498220626 >> v1:172.21.15.17:6789/0 conn(0x7f3128060960 legacy=0x7f31281469a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:47.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.149+0000 7f312e056700 1 -- 172.21.15.17:0/1498220626 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31280acdf0 con 0x7f312805fc30 2022-08-01T19:43:47.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.149+0000 7f312e056700 1 --2- 172.21.15.17:0/1498220626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f312805fc30 0x7f3128143a60 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f3118001390 tx=0x7f3124006df0).ready entity=mon.1 client_cookie=c769a1d7b25d6bd4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:47.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.149+0000 7f312d054700 1 -- 172.21.15.17:0/1498220626 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f312400d4f0 con 0x7f312805fc30 2022-08-01T19:43:47.158 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.149+0000 7f312d054700 1 -- 172.21.15.17:0/1498220626 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3124000e90 con 0x7f312805fc30 2022-08-01T19:43:47.158 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.149+0000 7f312d054700 1 -- 172.21.15.17:0/1498220626 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f312400d840 con 0x7f312805fc30 2022-08-01T19:43:47.158 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.149+0000 7f31302ba700 1 -- 172.21.15.17:0/1498220626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f312805fc30 msgr2=0x7f3128143a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:47.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.149+0000 7f31302ba700 1 --2- 172.21.15.17:0/1498220626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f312805fc30 0x7f3128143a60 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f3118001390 tx=0x7f3124006df0).stop 2022-08-01T19:43:47.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.149+0000 7f31302ba700 1 -- 172.21.15.17:0/1498220626 shutdown_connections 2022-08-01T19:43:47.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.149+0000 7f31302ba700 1 --2- 172.21.15.17:0/1498220626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f312805fc30 0x7f3128143a60 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:47.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.149+0000 7f31302ba700 1 -- 172.21.15.17:0/1498220626 >> 172.21.15.17:0/1498220626 conn(0x7f31280b2b00 msgr2=0x7f31280b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:47.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.149+0000 7f31302ba700 1 -- 172.21.15.17:0/1498220626 shutdown_connections 2022-08-01T19:43:47.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.149+0000 7f31302ba700 1 -- 172.21.15.17:0/1498220626 wait complete. 2022-08-01T19:43:47.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.149+0000 7f31302ba700 1 Processor -- start 2022-08-01T19:43:47.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.149+0000 7f31302ba700 1 -- start start 2022-08-01T19:43:47.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.153+0000 7f31302ba700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f312805fc30 0x7f31280b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:47.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.153+0000 7f31302ba700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3128060960 0x7f312813a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:47.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.153+0000 7f31302ba700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31280ac690 0x7f312813a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:47.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.153+0000 7f31302ba700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3128140c10 con 0x7f312805fc30 2022-08-01T19:43:47.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.153+0000 7f31302ba700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312813a1b0 con 0x7f31280ac690 2022-08-01T19:43:47.182 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.153+0000 7f31302ba700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3128139f10 con 0x7f3128060960 2022-08-01T19:43:47.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.153+0000 7f312e056700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f312805fc30 0x7f31280b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:47.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.153+0000 7f312e056700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f312805fc30 0x7f31280b1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45278/0 (socket says 172.21.15.17:45278) 2022-08-01T19:43:47.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.153+0000 7f312e056700 1 -- 172.21.15.17:0/3352562156 learned_addr learned my addr 172.21.15.17:0/3352562156 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:47.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.153+0000 7f312e857700 1 --2- 172.21.15.17:0/3352562156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31280ac690 0x7f312813a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:47.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.153+0000 7f312d855700 1 --2- 172.21.15.17:0/3352562156 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3128060960 0x7f312813a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:47.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.153+0000 7f312e056700 1 -- 172.21.15.17:0/3352562156 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3128060960 msgr2=0x7f312813a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:47.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.153+0000 7f312e056700 1 --2- 172.21.15.17:0/3352562156 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3128060960 0x7f312813a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:47.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.153+0000 7f312e056700 1 -- 172.21.15.17:0/3352562156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31280ac690 msgr2=0x7f312813a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:47.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.153+0000 7f312e056700 1 --2- 172.21.15.17:0/3352562156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31280ac690 0x7f312813a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:47.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.153+0000 7f312e056700 1 -- 172.21.15.17:0/3352562156 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f312400b180 con 0x7f312805fc30 2022-08-01T19:43:47.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.153+0000 7f312e857700 1 --2- 172.21.15.17:0/3352562156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31280ac690 0x7f312813a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:43:47.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.153+0000 7f312e056700 1 --2- 172.21.15.17:0/3352562156 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f312805fc30 0x7f31280b1c00 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 rx=0x7f3128140d50 tx=0x7f3124014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:47.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.153+0000 7f31177fe700 1 -- 172.21.15.17:0/3352562156 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3124016070 con 0x7f312805fc30 2022-08-01T19:43:47.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.153+0000 7f31177fe700 1 -- 172.21.15.17:0/3352562156 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3124013780 con 0x7f312805fc30 2022-08-01T19:43:47.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.153+0000 7f31302ba700 1 -- 172.21.15.17:0/3352562156 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3128146390 con 0x7f312805fc30 2022-08-01T19:43:47.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.153+0000 7f31177fe700 1 -- 172.21.15.17:0/3352562156 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3124013ae0 con 0x7f312805fc30 2022-08-01T19:43:47.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.153+0000 7f31302ba700 1 -- 172.21.15.17:0/3352562156 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f312813cac0 con 0x7f312805fc30 2022-08-01T19:43:47.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.153+0000 7f31302ba700 1 -- 172.21.15.17:0/3352562156 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31280574e0 con 0x7f312805fc30 2022-08-01T19:43:47.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.157+0000 7f31177fe700 1 -- 172.21.15.17:0/3352562156 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3124025030 con 0x7f312805fc30 2022-08-01T19:43:47.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.157+0000 7f31177fe700 1 --2- 172.21.15.17:0/3352562156 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3108053b00 0x7f3108055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:47.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.157+0000 7f31177fe700 1 -- 172.21.15.17:0/3352562156 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f312406cfe0 con 0x7f312805fc30 2022-08-01T19:43:47.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.157+0000 7f312d855700 1 --2- 172.21.15.17:0/3352562156 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3108053b00 0x7f3108055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:47.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.157+0000 7f312d855700 1 --2- 172.21.15.17:0/3352562156 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3108053b00 0x7f3108055fc0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7f311800b000 tx=0x7f3118001780).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:47.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.165+0000 7f31177fe700 1 -- 172.21.15.17:0/3352562156 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3124043c40 con 0x7f312805fc30 2022-08-01T19:43:47.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:47.329+0000 7f31302ba700 1 -- 172.21.15.17:0/3352562156 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f31280447e0 con 0x7f312805fc30 2022-08-01T19:43:48.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.253+0000 7f31177fe700 1 -- 172.21.15.17:0/3352562156 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v78) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3124047260 con 0x7f312805fc30 2022-08-01T19:43:48.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.257+0000 7f31302ba700 1 -- 172.21.15.17:0/3352562156 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3108053b00 msgr2=0x7f3108055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:48.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.257+0000 7f31302ba700 1 --2- 172.21.15.17:0/3352562156 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3108053b00 0x7f3108055fc0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7f311800b000 tx=0x7f3118001780).stop 2022-08-01T19:43:48.263 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.257+0000 7f31302ba700 1 -- 172.21.15.17:0/3352562156 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f312805fc30 msgr2=0x7f31280b1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:48.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.257+0000 7f31302ba700 1 --2- 172.21.15.17:0/3352562156 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f312805fc30 0x7f31280b1c00 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 rx=0x7f3128140d50 tx=0x7f3124014000).stop 2022-08-01T19:43:48.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.257+0000 7f31302ba700 1 -- 172.21.15.17:0/3352562156 shutdown_connections 2022-08-01T19:43:48.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.257+0000 7f31302ba700 1 --2- 172.21.15.17:0/3352562156 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3108053b00 0x7f3108055fc0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:48.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.257+0000 7f31302ba700 1 --2- 172.21.15.17:0/3352562156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31280ac690 0x7f312813a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:48.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.257+0000 7f31302ba700 1 --2- 172.21.15.17:0/3352562156 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3128060960 0x7f312813a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:48.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.257+0000 7f31302ba700 1 --2- 172.21.15.17:0/3352562156 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f312805fc30 0x7f31280b1c00 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:48.266 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.257+0000 7f31302ba700 1 -- 172.21.15.17:0/3352562156 >> 172.21.15.17:0/3352562156 conn(0x7f31280b2b00 msgr2=0x7f312813d910 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:48.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.257+0000 7f31302ba700 1 -- 172.21.15.17:0/3352562156 shutdown_connections 2022-08-01T19:43:48.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.257+0000 7f31302ba700 1 -- 172.21.15.17:0/3352562156 wait complete. 2022-08-01T19:43:48.278 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-08-01T19:43:48.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.389+0000 7fc6eddd8700 1 Processor -- start 2022-08-01T19:43:48.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.389+0000 7fc6eddd8700 1 -- start start 2022-08-01T19:43:48.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.389+0000 7fc6eddd8700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc6e80b6710 0x7fc6e80b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:48.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.389+0000 7fc6eddd8700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc6e80b6af0 0x7fc6e812bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:48.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.389+0000 7fc6eddd8700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6e8144240 con 0x7fc6e80b4550 2022-08-01T19:43:48.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.389+0000 7fc6eddd8700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6e8143d00 con 0x7fc6e80b6af0 2022-08-01T19:43:48.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.389+0000 7fc6eddd8700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e8143fa0 con 0x7fc6e80b6710 2022-08-01T19:43:48.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.393+0000 7fc6e7fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc6e80b6af0 0x7fc6e812bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:48.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.393+0000 7fc6e6ffd700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc6e80b4550 0x7fc6e8061ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37482/0 (socket says 172.21.15.17:37482) 2022-08-01T19:43:48.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.393+0000 7fc6e6ffd700 1 -- 172.21.15.17:0/1465072054 learned_addr learned my addr 172.21.15.17:0/1465072054 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:48.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.393+0000 7fc6e77fe700 1 --2- 172.21.15.17:0/1465072054 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc6e80b6710 0x7fc6e80b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:48.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.393+0000 7fc6e67fc700 1 -- 172.21.15.17:0/1465072054 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2176619615 0 0) 0x7fc6e8144240 con 0x7fc6e80b4550 2022-08-01T19:43:48.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.393+0000 7fc6e67fc700 1 -- 172.21.15.17:0/1465072054 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6d4003260 con 0x7fc6e80b4550 2022-08-01T19:43:48.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.393+0000 7fc6e7fff700 1 -- 172.21.15.17:0/1465072054 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc6e80b6710 msgr2=0x7fc6e80b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:48.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.393+0000 7fc6e7fff700 1 --2- 172.21.15.17:0/1465072054 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc6e80b6710 0x7fc6e80b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:48.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.393+0000 7fc6e7fff700 1 -- 172.21.15.17:0/1465072054 >> v1:172.21.15.17:6789/0 conn(0x7fc6e80b4550 legacy=0x7fc6e8061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:48.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.393+0000 7fc6e7fff700 1 -- 172.21.15.17:0/1465072054 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e8061780 con 0x7fc6e80b6af0 2022-08-01T19:43:48.413 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.393+0000 7fc6e7fff700 1 --2- 172.21.15.17:0/1465072054 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6e80b6af0 0x7fc6e812bb90 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 rx=0x7fc6e00032f0 tx=0x7fc6e00091d0).ready entity=mon.0 client_cookie=3b766c134989f86e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:48.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.393+0000 7fc6e67fc700 1 -- 172.21.15.17:0/1465072054 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6e000b600 con 0x7fc6e80b6af0 2022-08-01T19:43:48.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.393+0000 7fc6e67fc700 1 -- 172.21.15.17:0/1465072054 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e000b760 con 0x7fc6e80b6af0 2022-08-01T19:43:48.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.393+0000 7fc6e67fc700 1 -- 172.21.15.17:0/1465072054 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6e000b8d0 con 0x7fc6e80b6af0 2022-08-01T19:43:48.414 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.393+0000 7fc6eddd8700 1 -- 172.21.15.17:0/1465072054 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6e80b6af0 msgr2=0x7fc6e812bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:48.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.393+0000 7fc6eddd8700 1 --2- 172.21.15.17:0/1465072054 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6e80b6af0 0x7fc6e812bb90 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 rx=0x7fc6e00032f0 tx=0x7fc6e00091d0).stop 2022-08-01T19:43:48.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.393+0000 7fc6eddd8700 1 -- 172.21.15.17:0/1465072054 shutdown_connections 2022-08-01T19:43:48.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.393+0000 7fc6eddd8700 1 --2- 172.21.15.17:0/1465072054 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6e80b6af0 0x7fc6e812bb90 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:48.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.393+0000 7fc6eddd8700 1 --2- 172.21.15.17:0/1465072054 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc6e80b6710 0x7fc6e80b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:48.415 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.393+0000 7fc6eddd8700 1 -- 172.21.15.17:0/1465072054 >> 172.21.15.17:0/1465072054 conn(0x7fc6e805c620 msgr2=0x7fc6e805ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:48.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.393+0000 7fc6eddd8700 1 -- 172.21.15.17:0/1465072054 shutdown_connections 2022-08-01T19:43:48.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.393+0000 7fc6eddd8700 1 -- 172.21.15.17:0/1465072054 wait complete. 2022-08-01T19:43:48.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.397+0000 7fc6eddd8700 1 Processor -- start 2022-08-01T19:43:48.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.397+0000 7fc6eddd8700 1 -- start start 2022-08-01T19:43:48.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.397+0000 7fc6eddd8700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc6e80b4550 0x7fc6e80b8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:48.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.397+0000 7fc6eddd8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6e80b6710 0x7fc6e80b88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:48.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.397+0000 7fc6eddd8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6e80b6af0 0x7fc6e8126810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:48.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.397+0000 7fc6eddd8700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e81496c0 con 0x7fc6e80b6710 2022-08-01T19:43:48.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.397+0000 7fc6eddd8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e80aeb30 con 0x7fc6e80b6af0 2022-08-01T19:43:48.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.397+0000 7fc6eddd8700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e80b02a0 con 0x7fc6e80b4550 2022-08-01T19:43:48.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.397+0000 7fc6e6ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6e80b6710 0x7fc6e80b88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:48.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.397+0000 7fc6e6ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6e80b6710 0x7fc6e80b88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45294/0 (socket says 172.21.15.17:45294) 2022-08-01T19:43:48.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.397+0000 7fc6e6ffd700 1 -- 172.21.15.17:0/2340651049 learned_addr learned my addr 172.21.15.17:0/2340651049 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:48.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.397+0000 7fc6e7fff700 1 --2- 172.21.15.17:0/2340651049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6e80b6af0 0x7fc6e8126810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:48.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.397+0000 7fc6e77fe700 1 --2- 172.21.15.17:0/2340651049 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc6e80b4550 0x7fc6e80b8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:48.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.397+0000 7fc6e6ffd700 1 -- 172.21.15.17:0/2340651049 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc6e80b4550 msgr2=0x7fc6e80b8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:48.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.397+0000 7fc6e6ffd700 1 --2- 172.21.15.17:0/2340651049 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc6e80b4550 0x7fc6e80b8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:48.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.397+0000 7fc6e6ffd700 1 -- 172.21.15.17:0/2340651049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6e80b6af0 msgr2=0x7fc6e8126810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:48.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.397+0000 7fc6e6ffd700 1 --2- 172.21.15.17:0/2340651049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6e80b6af0 0x7fc6e8126810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:48.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.397+0000 7fc6e6ffd700 1 -- 172.21.15.17:0/2340651049 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e0009470 con 0x7fc6e80b6710 2022-08-01T19:43:48.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.397+0000 7fc6e6ffd700 1 --2- 172.21.15.17:0/2340651049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6e80b6710 0x7fc6e80b88b0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0x7fc6e80aee00 tx=0x7fc6d8007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:48.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.397+0000 7fc6e4ff9700 1 -- 172.21.15.17:0/2340651049 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6d80070d0 con 0x7fc6e80b6710 2022-08-01T19:43:48.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.397+0000 7fc6e4ff9700 1 -- 172.21.15.17:0/2340651049 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d8007230 con 0x7fc6e80b6710 2022-08-01T19:43:48.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.397+0000 7fc6eddd8700 1 -- 172.21.15.17:0/2340651049 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6e80b0540 con 0x7fc6e80b6710 2022-08-01T19:43:48.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.397+0000 7fc6e4ff9700 1 -- 172.21.15.17:0/2340651049 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6d8005610 con 0x7fc6e80b6710 2022-08-01T19:43:48.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.397+0000 7fc6eddd8700 1 -- 172.21.15.17:0/2340651049 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6e80afb10 con 0x7fc6e80b6710 2022-08-01T19:43:48.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.401+0000 7fc6eddd8700 1 -- 172.21.15.17:0/2340651049 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6e8061a20 con 0x7fc6e80b6710 2022-08-01T19:43:48.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.401+0000 7fc6e4ff9700 1 -- 172.21.15.17:0/2340651049 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc6d801a060 con 0x7fc6e80b6710 2022-08-01T19:43:48.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.401+0000 7fc6e4ff9700 1 --2- 172.21.15.17:0/2340651049 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc6c8053a70 0x7fc6c8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:48.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.401+0000 7fc6e4ff9700 1 -- 172.21.15.17:0/2340651049 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fc6d806b490 con 0x7fc6e80b6710 2022-08-01T19:43:48.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.401+0000 7fc6e77fe700 1 --2- 172.21.15.17:0/2340651049 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc6c8053a70 0x7fc6c8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:48.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.401+0000 7fc6e77fe700 1 --2- 172.21.15.17:0/2340651049 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc6c8053a70 0x7fc6c8055f30 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7fc6dc00b000 tx=0x7fc6dc006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:48.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.409+0000 7fc6e4ff9700 1 -- 172.21.15.17:0/2340651049 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc6d8042200 con 0x7fc6e80b6710 2022-08-01T19:43:48.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:48.533+0000 7fc6eddd8700 1 -- 172.21.15.17:0/2340651049 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc6e80574e0 con 0x7fc6e80b6710 2022-08-01T19:43:48.558 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:43:48.568 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:43:48.572 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:43:48.582 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:43:49.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.265+0000 7fc6e4ff9700 1 -- 172.21.15.17:0/2340651049 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v150) v1 ==== 185+0+0 (secure 0 0 0) 0x7fc6d8018020 con 0x7fc6e80b6710 2022-08-01T19:43:49.272 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.269+0000 7fc6eddd8700 1 -- 172.21.15.17:0/2340651049 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc6c8053a70 msgr2=0x7fc6c8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:49.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.269+0000 7fc6eddd8700 1 --2- 172.21.15.17:0/2340651049 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc6c8053a70 0x7fc6c8055f30 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7fc6dc00b000 tx=0x7fc6dc006df0).stop 2022-08-01T19:43:49.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.269+0000 7fc6eddd8700 1 -- 172.21.15.17:0/2340651049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6e80b6710 msgr2=0x7fc6e80b88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:49.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.269+0000 7fc6eddd8700 1 --2- 172.21.15.17:0/2340651049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6e80b6710 0x7fc6e80b88b0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0x7fc6e80aee00 tx=0x7fc6d8007aa0).stop 2022-08-01T19:43:49.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.269+0000 7fc6eddd8700 1 -- 172.21.15.17:0/2340651049 shutdown_connections 2022-08-01T19:43:49.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.269+0000 7fc6eddd8700 1 --2- 172.21.15.17:0/2340651049 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc6c8053a70 0x7fc6c8055f30 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:49.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.269+0000 7fc6eddd8700 1 --2- 172.21.15.17:0/2340651049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6e80b6af0 0x7fc6e8126810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:49.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.269+0000 7fc6eddd8700 1 --2- 172.21.15.17:0/2340651049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6e80b6710 0x7fc6e80b88b0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:49.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.269+0000 7fc6eddd8700 1 --2- 172.21.15.17:0/2340651049 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc6e80b4550 0x7fc6e80b8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:49.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.269+0000 7fc6eddd8700 1 -- 172.21.15.17:0/2340651049 >> 172.21.15.17:0/2340651049 conn(0x7fc6e805c620 msgr2=0x7fc6e81432f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:49.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.269+0000 7fc6eddd8700 1 -- 172.21.15.17:0/2340651049 shutdown_connections 2022-08-01T19:43:49.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.269+0000 7fc6eddd8700 1 -- 172.21.15.17:0/2340651049 wait complete. 2022-08-01T19:43:49.281 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' removed 2022-08-01T19:43:49.291 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-08-01T19:43:49.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.421+0000 7f8691f70700 1 Processor -- start 2022-08-01T19:43:49.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.421+0000 7f8691f70700 1 -- start start 2022-08-01T19:43:49.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.421+0000 7f8691f70700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f868c0af550 0x7f868c0b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.421+0000 7f8691f70700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f868c060330 0x7f868c1445e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.421+0000 7f8691f70700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f868c144ad0 0x7f868c0a8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.421+0000 7f8691f70700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f868c0b15e0 con 0x7f868c144ad0 2022-08-01T19:43:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.421+0000 7f8691f70700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f868c0aa630 con 0x7f868c060330 2022-08-01T19:43:49.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.421+0000 7f8691f70700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f868c05f840 con 0x7f868c0af550 2022-08-01T19:43:49.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.421+0000 7f869176f700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f868c144ad0 0x7f868c0a8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:49.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.421+0000 7f869176f700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f868c144ad0 0x7f868c0a8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45302/0 (socket says 172.21.15.17:45302) 2022-08-01T19:43:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.421+0000 7f869176f700 1 -- 172.21.15.17:0/653907672 learned_addr learned my addr 172.21.15.17:0/653907672 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.421+0000 7f8690f6e700 1 --2- 172.21.15.17:0/653907672 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f868c0af550 0x7f868c0b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.421+0000 7f8683fff700 1 --2- 172.21.15.17:0/653907672 >> v2:172.21.15.77:3300/0 conn(0x7f868c060330 0x7f868c1445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:49.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.421+0000 7f869176f700 1 -- 172.21.15.17:0/653907672 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f868c0af550 msgr2=0x7f868c0b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:49.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.421+0000 7f869176f700 1 --2- 172.21.15.17:0/653907672 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f868c0af550 0x7f868c0b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:49.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.421+0000 7f869176f700 1 -- 172.21.15.17:0/653907672 >> v2:172.21.15.77:3300/0 conn(0x7f868c060330 msgr2=0x7f868c1445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:49.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.421+0000 7f869176f700 1 --2- 172.21.15.17:0/653907672 >> v2:172.21.15.77:3300/0 conn(0x7f868c060330 0x7f868c1445e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:49.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.421+0000 7f869176f700 1 -- 172.21.15.17:0/653907672 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f868c05fd10 con 0x7f868c144ad0 2022-08-01T19:43:49.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.421+0000 7f8683fff700 1 --2- 172.21.15.17:0/653907672 >> v2:172.21.15.77:3300/0 conn(0x7f868c060330 0x7f868c1445e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:43:49.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.421+0000 7f869176f700 1 --2- 172.21.15.17:0/653907672 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f868c144ad0 0x7f868c0a8d80 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 rx=0x7f86840032f0 tx=0x7f8684009290).ready entity=mon.0 client_cookie=1ed55b4e1765f179 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:49.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.425+0000 7f86837fe700 1 -- 172.21.15.17:0/653907672 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f868400b870 con 0x7f868c144ad0 2022-08-01T19:43:49.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.425+0000 7f86837fe700 1 -- 172.21.15.17:0/653907672 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868400b9d0 con 0x7f868c144ad0 2022-08-01T19:43:49.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.425+0000 7f86837fe700 1 -- 172.21.15.17:0/653907672 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f868400bb40 con 0x7f868c144ad0 2022-08-01T19:43:49.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.425+0000 7f8691f70700 1 -- 172.21.15.17:0/653907672 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f868c144ad0 msgr2=0x7f868c0a8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:49.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.425+0000 7f8691f70700 1 --2- 172.21.15.17:0/653907672 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f868c144ad0 0x7f868c0a8d80 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 rx=0x7f86840032f0 tx=0x7f8684009290).stop 2022-08-01T19:43:49.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.425+0000 7f8691f70700 1 -- 172.21.15.17:0/653907672 shutdown_connections 2022-08-01T19:43:49.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.425+0000 7f8691f70700 1 --2- 172.21.15.17:0/653907672 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f868c144ad0 0x7f868c0a8d80 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:49.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.425+0000 7f8691f70700 1 --2- 172.21.15.17:0/653907672 >> v2:172.21.15.77:3300/0 conn(0x7f868c060330 0x7f868c1445e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:49.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.425+0000 7f8691f70700 1 --2- 172.21.15.17:0/653907672 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f868c0af550 0x7f868c0b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:49.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.425+0000 7f8691f70700 1 -- 172.21.15.17:0/653907672 >> 172.21.15.17:0/653907672 conn(0x7f868c0612d0 msgr2=0x7f868c062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:49.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.425+0000 7f8691f70700 1 -- 172.21.15.17:0/653907672 shutdown_connections 2022-08-01T19:43:49.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.425+0000 7f8691f70700 1 -- 172.21.15.17:0/653907672 wait complete. 2022-08-01T19:43:49.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.425+0000 7f8691f70700 1 Processor -- start 2022-08-01T19:43:49.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.425+0000 7f8691f70700 1 -- start start 2022-08-01T19:43:49.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.429+0000 7f8691f70700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f868c060330 0x7f868c141740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:49.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.429+0000 7f8691f70700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f868c0af550 0x7f868c13de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:49.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.429+0000 7f8691f70700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f868c144ad0 0x7f868c13e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:49.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.429+0000 7f8691f70700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868c14b6e0 con 0x7f868c0af550 2022-08-01T19:43:49.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.429+0000 7f8691f70700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868c063c50 con 0x7f868c060330 2022-08-01T19:43:49.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.429+0000 7f8691f70700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f868c063ef0 con 0x7f868c144ad0 2022-08-01T19:43:49.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.429+0000 7f8683fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f868c0af550 0x7f868c13de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:49.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.429+0000 7f8683fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f868c0af550 0x7f868c13de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45306/0 (socket says 172.21.15.17:45306) 2022-08-01T19:43:49.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.429+0000 7f8683fff700 1 -- 172.21.15.17:0/782746884 learned_addr learned my addr 172.21.15.17:0/782746884 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:49.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.429+0000 7f8690f6e700 1 --2- 172.21.15.17:0/782746884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f868c060330 0x7f868c141740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:49.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.429+0000 7f869176f700 1 --2- 172.21.15.17:0/782746884 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f868c144ad0 0x7f868c13e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:49.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.429+0000 7f8683fff700 1 -- 172.21.15.17:0/782746884 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f868c144ad0 msgr2=0x7f868c13e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:49.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.429+0000 7f8683fff700 1 --2- 172.21.15.17:0/782746884 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f868c144ad0 0x7f868c13e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:49.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.429+0000 7f8683fff700 1 -- 172.21.15.17:0/782746884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f868c060330 msgr2=0x7f868c141740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:49.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.429+0000 7f8683fff700 1 --2- 172.21.15.17:0/782746884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f868c060330 0x7f868c141740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:49.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.429+0000 7f8683fff700 1 -- 172.21.15.17:0/782746884 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8684009530 con 0x7f868c0af550 2022-08-01T19:43:49.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.429+0000 7f8683fff700 1 --2- 172.21.15.17:0/782746884 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f868c0af550 0x7f868c13de50 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 rx=0x7f867c006940 tx=0x7f867c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:49.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.429+0000 7f8681ffb700 1 -- 172.21.15.17:0/782746884 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f867c011780 con 0x7f868c0af550 2022-08-01T19:43:49.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.429+0000 7f8681ffb700 1 -- 172.21.15.17:0/782746884 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867c0118e0 con 0x7f868c0af550 2022-08-01T19:43:49.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.429+0000 7f8681ffb700 1 -- 172.21.15.17:0/782746884 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f867c011c40 con 0x7f868c0af550 2022-08-01T19:43:49.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.429+0000 7f8691f70700 1 -- 172.21.15.17:0/782746884 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f868c064190 con 0x7f868c0af550 2022-08-01T19:43:49.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.429+0000 7f8691f70700 1 -- 172.21.15.17:0/782746884 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f868c067470 con 0x7f868c0af550 2022-08-01T19:43:49.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.429+0000 7f8691f70700 1 -- 172.21.15.17:0/782746884 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f868c064320 con 0x7f868c0af550 2022-08-01T19:43:49.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.433+0000 7f8681ffb700 1 -- 172.21.15.17:0/782746884 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f867c012420 con 0x7f868c0af550 2022-08-01T19:43:49.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.433+0000 7f8681ffb700 1 --2- 172.21.15.17:0/782746884 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f866c053950 0x7f866c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:49.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.433+0000 7f8681ffb700 1 -- 172.21.15.17:0/782746884 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6068+0+0 (secure 0 0 0) 0x7f867c06bbe0 con 0x7f868c0af550 2022-08-01T19:43:49.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.433+0000 7f8690f6e700 1 --2- 172.21.15.17:0/782746884 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f866c053950 0x7f866c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:49.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.433+0000 7f8690f6e700 1 --2- 172.21.15.17:0/782746884 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f866c053950 0x7f866c055e10 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7f868800b000 tx=0x7f8688006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:49.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.441+0000 7f8681ffb700 1 -- 172.21.15.17:0/782746884 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f867c042aa0 con 0x7f868c0af550 2022-08-01T19:43:49.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:49.565+0000 7f8691f70700 1 -- 172.21.15.17:0/782746884 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f868c0574e0 con 0x7f868c0af550 2022-08-01T19:43:50.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.285+0000 7f8681ffb700 1 -- 172.21.15.17:0/782746884 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v151) v1 ==== 173+0+0 (secure 0 0 0) 0x7f867c0460c0 con 0x7f868c0af550 2022-08-01T19:43:50.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.289+0000 7f8691f70700 1 -- 172.21.15.17:0/782746884 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f866c053950 msgr2=0x7f866c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:50.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.289+0000 7f8691f70700 1 --2- 172.21.15.17:0/782746884 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f866c053950 0x7f866c055e10 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7f868800b000 tx=0x7f8688006df0).stop 2022-08-01T19:43:50.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.289+0000 7f8691f70700 1 -- 172.21.15.17:0/782746884 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f868c0af550 msgr2=0x7f868c13de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:50.295 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.289+0000 7f8691f70700 1 --2- 172.21.15.17:0/782746884 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f868c0af550 0x7f868c13de50 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 rx=0x7f867c006940 tx=0x7f867c008040).stop 2022-08-01T19:43:50.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.289+0000 7f8691f70700 1 -- 172.21.15.17:0/782746884 shutdown_connections 2022-08-01T19:43:50.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.289+0000 7f8691f70700 1 --2- 172.21.15.17:0/782746884 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f866c053950 0x7f866c055e10 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:50.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.289+0000 7f8691f70700 1 --2- 172.21.15.17:0/782746884 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f868c144ad0 0x7f868c13e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:50.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.289+0000 7f8691f70700 1 --2- 172.21.15.17:0/782746884 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f868c0af550 0x7f868c13de50 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:50.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.289+0000 7f8691f70700 1 --2- 172.21.15.17:0/782746884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f868c060330 0x7f868c141740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:50.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.289+0000 7f8691f70700 1 -- 172.21.15.17:0/782746884 >> 172.21.15.17:0/782746884 conn(0x7f868c0612d0 msgr2=0x7f868c0ae300 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:50.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.289+0000 7f8691f70700 1 -- 172.21.15.17:0/782746884 shutdown_connections 2022-08-01T19:43:50.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.289+0000 7f8691f70700 1 -- 172.21.15.17:0/782746884 wait complete. 2022-08-01T19:43:50.303 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' removed 2022-08-01T19:43:50.308 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-08-01T19:43:50.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.429+0000 7fb3aea9a700 1 Processor -- start 2022-08-01T19:43:50.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.429+0000 7fb3aea9a700 1 -- start start 2022-08-01T19:43:50.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.433+0000 7fb3aea9a700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb3a8066780 0x7fb3a8066b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:50.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.433+0000 7fb3aea9a700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fb3a80b1bc0 0x7fb3a805ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:50.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.433+0000 7fb3aea9a700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3a805f8e0 con 0x7fb3a8140e40 2022-08-01T19:43:50.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.433+0000 7fb3aea9a700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3a80b1630 con 0x7fb3a8066780 2022-08-01T19:43:50.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.433+0000 7fb3aea9a700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3a805fb10 con 0x7fb3a80b1bc0 2022-08-01T19:43:50.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.433+0000 7fb3ada98700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb3a8066780 0x7fb3a8066b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:50.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.433+0000 7fb3ada98700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb3a8066780 0x7fb3a8066b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45314/0 (socket says 172.21.15.17:45314) 2022-08-01T19:43:50.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.433+0000 7fb3ada98700 1 -- 172.21.15.17:0/1297794392 learned_addr learned my addr 172.21.15.17:0/1297794392 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:50.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.433+0000 7fb3ad297700 1 --2- 172.21.15.17:0/1297794392 >> v2:172.21.15.77:3300/0 conn(0x7fb3a80b1bc0 0x7fb3a805ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:50.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.433+0000 7fb3ada98700 1 -- 172.21.15.17:0/1297794392 >> v2:172.21.15.77:3300/0 conn(0x7fb3a80b1bc0 msgr2=0x7fb3a805ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:50.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.433+0000 7fb3ada98700 1 --2- 172.21.15.17:0/1297794392 >> v2:172.21.15.77:3300/0 conn(0x7fb3a80b1bc0 0x7fb3a805ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:50.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.433+0000 7fb3ada98700 1 -- 172.21.15.17:0/1297794392 >> v1:172.21.15.77:6789/0 conn(0x7fb3a8140e40 legacy=0x7fb3a813e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:50.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.433+0000 7fb3ada98700 1 -- 172.21.15.17:0/1297794392 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a80604a0 con 0x7fb3a8066780 2022-08-01T19:43:50.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.433+0000 7fb3ad297700 1 --2- 172.21.15.17:0/1297794392 >> v2:172.21.15.77:3300/0 conn(0x7fb3a80b1bc0 0x7fb3a805ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:43:50.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.433+0000 7fb3ada98700 1 --2- 172.21.15.17:0/1297794392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3a8066780 0x7fb3a8066b60 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 rx=0x7fb398002f70 tx=0x7fb398006eb0).ready entity=mon.0 client_cookie=8754b82f78bfe297 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:50.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.433+0000 7fb3aca96700 1 -- 172.21.15.17:0/1297794392 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb39800d610 con 0x7fb3a8066780 2022-08-01T19:43:50.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.433+0000 7fb3aca96700 1 -- 172.21.15.17:0/1297794392 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb398000e90 con 0x7fb3a8066780 2022-08-01T19:43:50.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.433+0000 7fb3aca96700 1 -- 172.21.15.17:0/1297794392 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb39800d9a0 con 0x7fb3a8066780 2022-08-01T19:43:50.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.433+0000 7fb3aea9a700 1 -- 172.21.15.17:0/1297794392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3a8066780 msgr2=0x7fb3a8066b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:50.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.433+0000 7fb3aea9a700 1 --2- 172.21.15.17:0/1297794392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3a8066780 0x7fb3a8066b60 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 rx=0x7fb398002f70 tx=0x7fb398006eb0).stop 2022-08-01T19:43:50.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.433+0000 7fb3aea9a700 1 -- 172.21.15.17:0/1297794392 shutdown_connections 2022-08-01T19:43:50.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.433+0000 7fb3aea9a700 1 --2- 172.21.15.17:0/1297794392 >> v2:172.21.15.77:3300/0 conn(0x7fb3a80b1bc0 0x7fb3a805ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:50.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.433+0000 7fb3aea9a700 1 --2- 172.21.15.17:0/1297794392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3a8066780 0x7fb3a8066b60 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:50.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.433+0000 7fb3aea9a700 1 -- 172.21.15.17:0/1297794392 >> 172.21.15.17:0/1297794392 conn(0x7fb3a8063870 msgr2=0x7fb3a8063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:50.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.433+0000 7fb3aea9a700 1 -- 172.21.15.17:0/1297794392 shutdown_connections 2022-08-01T19:43:50.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.433+0000 7fb3aea9a700 1 -- 172.21.15.17:0/1297794392 wait complete. 2022-08-01T19:43:50.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.437+0000 7fb3aea9a700 1 Processor -- start 2022-08-01T19:43:50.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.437+0000 7fb3aea9a700 1 -- start start 2022-08-01T19:43:50.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.437+0000 7fb3aea9a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3a8066780 0x7fb3a80aba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:50.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.437+0000 7fb3aea9a700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb3a80b1bc0 0x7fb3a80afbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:50.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.437+0000 7fb3aea9a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb3a8140e40 0x7fb3a813adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:50.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.437+0000 7fb3aea9a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a80ae350 con 0x7fb3a8066780 2022-08-01T19:43:50.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.437+0000 7fb3aea9a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a80af6e0 con 0x7fb3a8140e40 2022-08-01T19:43:50.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.437+0000 7fb3aea9a700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a80aed00 con 0x7fb3a80b1bc0 2022-08-01T19:43:50.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.437+0000 7fb3ada98700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3a8066780 0x7fb3a80aba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:50.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.437+0000 7fb3ada98700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3a8066780 0x7fb3a80aba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45320/0 (socket says 172.21.15.17:45320) 2022-08-01T19:43:50.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.437+0000 7fb3ad297700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb3a80b1bc0 0x7fb3a80afbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:50.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.437+0000 7fb3ae299700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb3a8140e40 0x7fb3a813adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:50.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.437+0000 7fb3ada98700 1 -- 172.21.15.17:0/4057265201 learned_addr learned my addr 172.21.15.17:0/4057265201 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:50.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.437+0000 7fb3ada98700 1 -- 172.21.15.17:0/4057265201 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb3a80b1bc0 msgr2=0x7fb3a80afbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:50.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.437+0000 7fb3ada98700 1 --2- 172.21.15.17:0/4057265201 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb3a80b1bc0 0x7fb3a80afbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:50.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.437+0000 7fb3ada98700 1 -- 172.21.15.17:0/4057265201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb3a8140e40 msgr2=0x7fb3a813adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:50.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.437+0000 7fb3ada98700 1 --2- 172.21.15.17:0/4057265201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb3a8140e40 0x7fb3a813adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:50.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.437+0000 7fb3ada98700 1 -- 172.21.15.17:0/4057265201 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb39800b1e0 con 0x7fb3a8066780 2022-08-01T19:43:50.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.437+0000 7fb3ada98700 1 --2- 172.21.15.17:0/4057265201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3a8066780 0x7fb3a80aba70 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 rx=0x7fb3a80af820 tx=0x7fb398015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:50.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.441+0000 7fb396ffd700 1 -- 172.21.15.17:0/4057265201 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb398011760 con 0x7fb3a8066780 2022-08-01T19:43:50.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.441+0000 7fb396ffd700 1 -- 172.21.15.17:0/4057265201 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3980118c0 con 0x7fb3a8066780 2022-08-01T19:43:50.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.441+0000 7fb3aea9a700 1 -- 172.21.15.17:0/4057265201 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3a80aefa0 con 0x7fb3a8066780 2022-08-01T19:43:50.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.441+0000 7fb396ffd700 1 -- 172.21.15.17:0/4057265201 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb398011c20 con 0x7fb3a8066780 2022-08-01T19:43:50.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.441+0000 7fb3aea9a700 1 -- 172.21.15.17:0/4057265201 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3a80ac2f0 con 0x7fb3a8066780 2022-08-01T19:43:50.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.441+0000 7fb3aea9a700 1 -- 172.21.15.17:0/4057265201 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3a80b18d0 con 0x7fb3a8066780 2022-08-01T19:43:50.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.441+0000 7fb396ffd700 1 -- 172.21.15.17:0/4057265201 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb398025030 con 0x7fb3a8066780 2022-08-01T19:43:50.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.441+0000 7fb396ffd700 1 --2- 172.21.15.17:0/4057265201 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb388053950 0x7fb388055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:50.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.441+0000 7fb396ffd700 1 -- 172.21.15.17:0/4057265201 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fb39806cef0 con 0x7fb3a8066780 2022-08-01T19:43:50.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.441+0000 7fb3ad297700 1 --2- 172.21.15.17:0/4057265201 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb388053950 0x7fb388055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:50.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.445+0000 7fb3ad297700 1 --2- 172.21.15.17:0/4057265201 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb388053950 0x7fb388055e10 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7fb3a400c000 tx=0x7fb3a4008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:50.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.449+0000 7fb396ffd700 1 -- 172.21.15.17:0/4057265201 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb398043e80 con 0x7fb3a8066780 2022-08-01T19:43:50.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.577+0000 7fb3aea9a700 1 -- 172.21.15.17:0/4057265201 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb3a80574e0 con 0x7fb3a8066780 2022-08-01T19:43:50.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.585+0000 7fb396ffd700 1 -- 172.21.15.17:0/4057265201 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb3980230a0 con 0x7fb3a8066780 2022-08-01T19:43:50.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.589+0000 7fb3aea9a700 1 -- 172.21.15.17:0/4057265201 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb388053950 msgr2=0x7fb388055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:50.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.589+0000 7fb3aea9a700 1 --2- 172.21.15.17:0/4057265201 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb388053950 0x7fb388055e10 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7fb3a400c000 tx=0x7fb3a4008040).stop 2022-08-01T19:43:50.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.589+0000 7fb3aea9a700 1 -- 172.21.15.17:0/4057265201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3a8066780 msgr2=0x7fb3a80aba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:50.605 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.589+0000 7fb3aea9a700 1 --2- 172.21.15.17:0/4057265201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3a8066780 0x7fb3a80aba70 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 rx=0x7fb3a80af820 tx=0x7fb398015000).stop 2022-08-01T19:43:50.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.589+0000 7fb3aea9a700 1 -- 172.21.15.17:0/4057265201 shutdown_connections 2022-08-01T19:43:50.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.589+0000 7fb3aea9a700 1 --2- 172.21.15.17:0/4057265201 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb388053950 0x7fb388055e10 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:50.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.589+0000 7fb3aea9a700 1 --2- 172.21.15.17:0/4057265201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb3a8140e40 0x7fb3a813adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:50.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.589+0000 7fb3aea9a700 1 --2- 172.21.15.17:0/4057265201 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb3a80b1bc0 0x7fb3a80afbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:50.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.589+0000 7fb3aea9a700 1 --2- 172.21.15.17:0/4057265201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3a8066780 0x7fb3a80aba70 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:50.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.589+0000 7fb3aea9a700 1 -- 172.21.15.17:0/4057265201 >> 172.21.15.17:0/4057265201 conn(0x7fb3a8063870 msgr2=0x7fb3a813ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:50.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.589+0000 7fb3aea9a700 1 -- 172.21.15.17:0/4057265201 shutdown_connections 2022-08-01T19:43:50.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.589+0000 7fb3aea9a700 1 -- 172.21.15.17:0/4057265201 wait complete. 2022-08-01T19:43:50.610 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_root_rctime' 2022-08-01T19:43:50.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.717+0000 7f41deb6b700 1 Processor -- start 2022-08-01T19:43:50.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.717+0000 7f41deb6b700 1 -- start start 2022-08-01T19:43:50.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.717+0000 7f41deb6b700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f41d80afaf0 0x7f41d80b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:50.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.717+0000 7f41deb6b700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41d80ad730 con 0x7f41d80afed0 2022-08-01T19:43:50.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.717+0000 7f41deb6b700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41d80ad1f0 con 0x7f41d80b0680 2022-08-01T19:43:50.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.717+0000 7f41deb6b700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41d80af5f0 con 0x7f41d80afaf0 2022-08-01T19:43:50.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.717+0000 7f41dd108700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f41d80afed0 0x7f41d813fc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37526/0 (socket says 172.21.15.17:37526) 2022-08-01T19:43:50.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.717+0000 7f41dd108700 1 -- 172.21.15.17:0/4138481101 learned_addr learned my addr 172.21.15.17:0/4138481101 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:50.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.717+0000 7f41dc907700 1 --2- 172.21.15.17:0/4138481101 >> v2:172.21.15.77:3300/0 conn(0x7f41d80afaf0 0x7f41d80b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:50.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.717+0000 7f41dc907700 1 -- 172.21.15.17:0/4138481101 >> v1:172.21.15.77:6789/0 conn(0x7f41d80b0680 legacy=0x7f41d80b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:50.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.717+0000 7f41dc907700 1 -- 172.21.15.17:0/4138481101 >> v1:172.21.15.17:6789/0 conn(0x7f41d80afed0 legacy=0x7f41d813fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:50.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.717+0000 7f41dc907700 1 -- 172.21.15.17:0/4138481101 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d80ad490 con 0x7f41d80afaf0 2022-08-01T19:43:50.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.721+0000 7f41dc907700 1 --2- 172.21.15.17:0/4138481101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d80afaf0 0x7f41d80b1d60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7f41c80032f0 tx=0x7f41c80091d0).ready entity=mon.1 client_cookie=dd64387247de5c2e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:50.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.721+0000 7f41d77fe700 1 -- 172.21.15.17:0/4138481101 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41c8009ea0 con 0x7f41d80afaf0 2022-08-01T19:43:50.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.721+0000 7f41d77fe700 1 -- 172.21.15.17:0/4138481101 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41c800b8a0 con 0x7f41d80afaf0 2022-08-01T19:43:50.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.721+0000 7f41d77fe700 1 -- 172.21.15.17:0/4138481101 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41c800ba10 con 0x7f41d80afaf0 2022-08-01T19:43:50.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.721+0000 7f41deb6b700 1 -- 172.21.15.17:0/4138481101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d80afaf0 msgr2=0x7f41d80b1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:50.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.721+0000 7f41deb6b700 1 --2- 172.21.15.17:0/4138481101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d80afaf0 0x7f41d80b1d60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7f41c80032f0 tx=0x7f41c80091d0).stop 2022-08-01T19:43:50.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.725+0000 7f41deb6b700 1 -- 172.21.15.17:0/4138481101 shutdown_connections 2022-08-01T19:43:50.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.725+0000 7f41deb6b700 1 --2- 172.21.15.17:0/4138481101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d80afaf0 0x7f41d80b1d60 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:50.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.725+0000 7f41deb6b700 1 -- 172.21.15.17:0/4138481101 >> 172.21.15.17:0/4138481101 conn(0x7f41d805ceb0 msgr2=0x7f41d8063130 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:50.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.725+0000 7f41deb6b700 1 -- 172.21.15.17:0/4138481101 shutdown_connections 2022-08-01T19:43:50.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.725+0000 7f41deb6b700 1 -- 172.21.15.17:0/4138481101 wait complete. 2022-08-01T19:43:50.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.725+0000 7f41deb6b700 1 Processor -- start 2022-08-01T19:43:50.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.725+0000 7f41deb6b700 1 -- start start 2022-08-01T19:43:50.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.729+0000 7f41deb6b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41d80afaf0 0x7f41d80aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:50.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.729+0000 7f41deb6b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f41d80afed0 0x7f41d80a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:50.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.729+0000 7f41deb6b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d80b0680 0x7f41d80a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:50.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.729+0000 7f41deb6b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d80a5870 con 0x7f41d80afaf0 2022-08-01T19:43:50.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.729+0000 7f41deb6b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d80a8b30 con 0x7f41d80b0680 2022-08-01T19:43:50.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.729+0000 7f41deb6b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41d80a8890 con 0x7f41d80afed0 2022-08-01T19:43:50.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.729+0000 7f41dc907700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41d80afaf0 0x7f41d80aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:50.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.729+0000 7f41dc907700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41d80afaf0 0x7f41d80aa0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45334/0 (socket says 172.21.15.17:45334) 2022-08-01T19:43:50.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.729+0000 7f41dc907700 1 -- 172.21.15.17:0/4013723437 learned_addr learned my addr 172.21.15.17:0/4013723437 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:50.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.729+0000 7f41dd108700 1 --2- 172.21.15.17:0/4013723437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d80b0680 0x7f41d80a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:50.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.729+0000 7f41d7fff700 1 --2- 172.21.15.17:0/4013723437 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f41d80afed0 0x7f41d80a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:50.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.729+0000 7f41dc907700 1 -- 172.21.15.17:0/4013723437 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f41d80afed0 msgr2=0x7f41d80a4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:50.743 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.729+0000 7f41dc907700 1 --2- 172.21.15.17:0/4013723437 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f41d80afed0 0x7f41d80a4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:50.744 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.729+0000 7f41dc907700 1 -- 172.21.15.17:0/4013723437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d80b0680 msgr2=0x7f41d80a5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:50.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.729+0000 7f41dc907700 1 --2- 172.21.15.17:0/4013723437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d80b0680 0x7f41d80a5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:50.745 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.729+0000 7f41dc907700 1 -- 172.21.15.17:0/4013723437 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c8009470 con 0x7f41d80afaf0 2022-08-01T19:43:50.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.729+0000 7f41dc907700 1 --2- 172.21.15.17:0/4013723437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41d80afaf0 0x7f41d80aa0e0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 rx=0x7f41c80032f0 tx=0x7f41c8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:50.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.729+0000 7f41d5ffb700 1 -- 172.21.15.17:0/4013723437 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41c8009ea0 con 0x7f41d80afaf0 2022-08-01T19:43:50.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.729+0000 7f41d5ffb700 1 -- 172.21.15.17:0/4013723437 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41c801f3f0 con 0x7f41d80afaf0 2022-08-01T19:43:50.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.729+0000 7f41deb6b700 1 -- 172.21.15.17:0/4013723437 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41d80aaba0 con 0x7f41d80afaf0 2022-08-01T19:43:50.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.729+0000 7f41d5ffb700 1 -- 172.21.15.17:0/4013723437 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41c801f750 con 0x7f41d80afaf0 2022-08-01T19:43:50.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.729+0000 7f41deb6b700 1 -- 172.21.15.17:0/4013723437 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41d80acb50 con 0x7f41d80afaf0 2022-08-01T19:43:50.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.729+0000 7f41deb6b700 1 -- 172.21.15.17:0/4013723437 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41d80b1830 con 0x7f41d80afaf0 2022-08-01T19:43:50.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.733+0000 7f41d5ffb700 1 -- 172.21.15.17:0/4013723437 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f41c8026030 con 0x7f41d80afaf0 2022-08-01T19:43:50.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.733+0000 7f41d5ffb700 1 --2- 172.21.15.17:0/4013723437 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f41bc053aa0 0x7f41bc055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:50.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.733+0000 7f41d5ffb700 1 -- 172.21.15.17:0/4013723437 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f41c806c9b0 con 0x7f41d80afaf0 2022-08-01T19:43:50.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.733+0000 7f41d7fff700 1 --2- 172.21.15.17:0/4013723437 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f41bc053aa0 0x7f41bc055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:50.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.733+0000 7f41d7fff700 1 --2- 172.21.15.17:0/4013723437 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f41bc053aa0 0x7f41bc055f60 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f41c000d000 tx=0x7f41c0007b70).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:50.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.741+0000 7f41d5ffb700 1 -- 172.21.15.17:0/4013723437 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f41c80439d0 con 0x7f41d80afaf0 2022-08-01T19:43:50.913 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:50.909+0000 7f41deb6b700 1 -- 172.21.15.17:0/4013723437 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]} v 0) v1 -- 0x7f41d80574e0 con 0x7f41d80afaf0 2022-08-01T19:43:51.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.289+0000 7f41d5ffb700 1 -- 172.21.15.17:0/4013723437 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]}]=0 v451) v1 ==== 127+0+0 (secure 0 0 0) 0x7f41c80150c0 con 0x7f41d80afaf0 2022-08-01T19:43:51.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.293+0000 7f41deb6b700 1 -- 172.21.15.17:0/4013723437 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f41bc053aa0 msgr2=0x7f41bc055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:51.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.293+0000 7f41deb6b700 1 --2- 172.21.15.17:0/4013723437 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f41bc053aa0 0x7f41bc055f60 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f41c000d000 tx=0x7f41c0007b70).stop 2022-08-01T19:43:51.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.293+0000 7f41deb6b700 1 -- 172.21.15.17:0/4013723437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41d80afaf0 msgr2=0x7f41d80aa0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:51.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.293+0000 7f41deb6b700 1 --2- 172.21.15.17:0/4013723437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41d80afaf0 0x7f41d80aa0e0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 rx=0x7f41c80032f0 tx=0x7f41c8016000).stop 2022-08-01T19:43:51.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.293+0000 7f41deb6b700 1 -- 172.21.15.17:0/4013723437 shutdown_connections 2022-08-01T19:43:51.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.293+0000 7f41deb6b700 1 --2- 172.21.15.17:0/4013723437 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f41bc053aa0 0x7f41bc055f60 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:51.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.293+0000 7f41deb6b700 1 --2- 172.21.15.17:0/4013723437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41d80b0680 0x7f41d80a5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:51.299 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.293+0000 7f41deb6b700 1 --2- 172.21.15.17:0/4013723437 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f41d80afed0 0x7f41d80a4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:51.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.293+0000 7f41deb6b700 1 --2- 172.21.15.17:0/4013723437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41d80afaf0 0x7f41d80aa0e0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:51.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.293+0000 7f41deb6b700 1 -- 172.21.15.17:0/4013723437 >> 172.21.15.17:0/4013723437 conn(0x7f41d805ceb0 msgr2=0x7f41d813b380 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:51.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.293+0000 7f41deb6b700 1 -- 172.21.15.17:0/4013723437 shutdown_connections 2022-08-01T19:43:51.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.293+0000 7f41deb6b700 1 -- 172.21.15.17:0/4013723437 wait complete. 2022-08-01T19:43:51.309 INFO:tasks.cephfs_test_runner:test_root_rctime (tasks.cephfs.test_misc.TestMisc) ... ok 2022-08-01T19:43:51.309 INFO:tasks.cephfs_test_runner:Starting test: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2022-08-01T19:43:51.310 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_statfs_on_deleted_fs' 2022-08-01T19:43:51.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.417+0000 7f4976e1c700 1 Processor -- start 2022-08-01T19:43:51.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.417+0000 7f4976e1c700 1 -- start start 2022-08-01T19:43:51.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.421+0000 7f4976e1c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f49700b6a90 0x7f49700acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:51.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.421+0000 7f4976e1c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f49700aa760 0x7f497013c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:51.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.421+0000 7f4976e1c700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49700b6e70 con 0x7f4970060aa0 2022-08-01T19:43:51.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.421+0000 7f4976e1c700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4970061370 con 0x7f49700b6a90 2022-08-01T19:43:51.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.421+0000 7f4976e1c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4970061610 con 0x7f49700aa760 2022-08-01T19:43:51.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.421+0000 7f4974bb8700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f49700b6a90 0x7f49700acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:51.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.421+0000 7f4974bb8700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f49700b6a90 0x7f49700acee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45342/0 (socket says 172.21.15.17:45342) 2022-08-01T19:43:51.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.421+0000 7f4974bb8700 1 -- 172.21.15.17:0/1988924925 learned_addr learned my addr 172.21.15.17:0/1988924925 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:51.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.421+0000 7f49753b9700 1 --2- 172.21.15.17:0/1988924925 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f49700aa760 0x7f497013c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:51.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.421+0000 7f4974bb8700 1 -- 172.21.15.17:0/1988924925 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f49700aa760 msgr2=0x7f497013c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:51.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.421+0000 7f4974bb8700 1 --2- 172.21.15.17:0/1988924925 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f49700aa760 0x7f497013c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:51.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.421+0000 7f4974bb8700 1 -- 172.21.15.17:0/1988924925 >> v1:172.21.15.77:6789/0 conn(0x7f4970060aa0 legacy=0x7f49700aa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:51.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.421+0000 7f4974bb8700 1 -- 172.21.15.17:0/1988924925 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49700618b0 con 0x7f49700b6a90 2022-08-01T19:43:51.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.421+0000 7f4974bb8700 1 --2- 172.21.15.17:0/1988924925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49700b6a90 0x7f49700acee0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 rx=0x7f4964002f70 tx=0x7f4964006eb0).ready entity=mon.0 client_cookie=7fbdce2d8d60ee93 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:51.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.421+0000 7f496f7fe700 1 -- 172.21.15.17:0/1988924925 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f496400d5c0 con 0x7f49700b6a90 2022-08-01T19:43:51.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.421+0000 7f496f7fe700 1 -- 172.21.15.17:0/1988924925 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4964000e90 con 0x7f49700b6a90 2022-08-01T19:43:51.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.421+0000 7f496f7fe700 1 -- 172.21.15.17:0/1988924925 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f496400d950 con 0x7f49700b6a90 2022-08-01T19:43:51.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.421+0000 7f4976e1c700 1 -- 172.21.15.17:0/1988924925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49700b6a90 msgr2=0x7f49700acee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:51.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.421+0000 7f4976e1c700 1 --2- 172.21.15.17:0/1988924925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49700b6a90 0x7f49700acee0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 rx=0x7f4964002f70 tx=0x7f4964006eb0).stop 2022-08-01T19:43:51.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.421+0000 7f4976e1c700 1 -- 172.21.15.17:0/1988924925 shutdown_connections 2022-08-01T19:43:51.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.421+0000 7f4976e1c700 1 --2- 172.21.15.17:0/1988924925 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f49700aa760 0x7f497013c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:51.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.421+0000 7f4976e1c700 1 --2- 172.21.15.17:0/1988924925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49700b6a90 0x7f49700acee0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:51.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.421+0000 7f4976e1c700 1 -- 172.21.15.17:0/1988924925 >> 172.21.15.17:0/1988924925 conn(0x7f49700b3610 msgr2=0x7f49700b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:51.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.421+0000 7f4976e1c700 1 -- 172.21.15.17:0/1988924925 shutdown_connections 2022-08-01T19:43:51.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.421+0000 7f4976e1c700 1 -- 172.21.15.17:0/1988924925 wait complete. 2022-08-01T19:43:51.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.425+0000 7f4976e1c700 1 Processor -- start 2022-08-01T19:43:51.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.425+0000 7f4976e1c700 1 -- start start 2022-08-01T19:43:51.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.425+0000 7f4976e1c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4970060aa0 0x7f497013bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:51.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.425+0000 7f4976e1c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49700aa760 0x7f4970150850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:51.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.425+0000 7f4976e1c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49700b6a90 0x7f4970150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:51.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.425+0000 7f4976e1c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4970063c00 con 0x7f49700b6a90 2022-08-01T19:43:51.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.425+0000 7f4976e1c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49701502c0 con 0x7f49700aa760 2022-08-01T19:43:51.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.425+0000 7f4976e1c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49700ae730 con 0x7f4970060aa0 2022-08-01T19:43:51.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.425+0000 7f49753b9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49700b6a90 0x7f4970150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:51.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.425+0000 7f49753b9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49700b6a90 0x7f4970150d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45352/0 (socket says 172.21.15.17:45352) 2022-08-01T19:43:51.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.425+0000 7f49753b9700 1 -- 172.21.15.17:0/569398109 learned_addr learned my addr 172.21.15.17:0/569398109 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:51.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.425+0000 7f496ffff700 1 --2- 172.21.15.17:0/569398109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49700aa760 0x7f4970150850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:51.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.425+0000 7f4974bb8700 1 --2- 172.21.15.17:0/569398109 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4970060aa0 0x7f497013bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:51.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.425+0000 7f49753b9700 1 -- 172.21.15.17:0/569398109 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4970060aa0 msgr2=0x7f497013bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:51.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.425+0000 7f49753b9700 1 --2- 172.21.15.17:0/569398109 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4970060aa0 0x7f497013bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:51.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.425+0000 7f49753b9700 1 -- 172.21.15.17:0/569398109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49700aa760 msgr2=0x7f4970150850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:51.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.425+0000 7f49753b9700 1 --2- 172.21.15.17:0/569398109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49700aa760 0x7f4970150850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:51.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.425+0000 7f49753b9700 1 -- 172.21.15.17:0/569398109 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f496400b1e0 con 0x7f49700b6a90 2022-08-01T19:43:51.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.425+0000 7f49753b9700 1 --2- 172.21.15.17:0/569398109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49700b6a90 0x7f4970150d40 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 rx=0x7f496800a000 tx=0x7f4968008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:51.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.425+0000 7f496dffb700 1 -- 172.21.15.17:0/569398109 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4968013620 con 0x7f49700b6a90 2022-08-01T19:43:51.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.425+0000 7f496dffb700 1 -- 172.21.15.17:0/569398109 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4968013780 con 0x7f49700b6a90 2022-08-01T19:43:51.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.425+0000 7f4976e1c700 1 -- 172.21.15.17:0/569398109 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49700ae9d0 con 0x7f49700b6a90 2022-08-01T19:43:51.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.425+0000 7f496dffb700 1 -- 172.21.15.17:0/569398109 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4968013ae0 con 0x7f49700b6a90 2022-08-01T19:43:51.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.429+0000 7f4976e1c700 1 -- 172.21.15.17:0/569398109 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4970151230 con 0x7f49700b6a90 2022-08-01T19:43:51.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.429+0000 7f4976e1c700 1 -- 172.21.15.17:0/569398109 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4970061b50 con 0x7f49700b6a90 2022-08-01T19:43:51.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.429+0000 7f496dffb700 1 -- 172.21.15.17:0/569398109 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4968013c40 con 0x7f49700b6a90 2022-08-01T19:43:51.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.429+0000 7f496dffb700 1 --2- 172.21.15.17:0/569398109 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f49500539e0 0x7f4950055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:51.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.429+0000 7f496dffb700 1 -- 172.21.15.17:0/569398109 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f496806bbc0 con 0x7f49700b6a90 2022-08-01T19:43:51.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.429+0000 7f4974bb8700 1 --2- 172.21.15.17:0/569398109 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f49500539e0 0x7f4950055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:51.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.433+0000 7f4974bb8700 1 --2- 172.21.15.17:0/569398109 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f49500539e0 0x7f4950055ea0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7f496401c000 tx=0x7f4964015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:51.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.433+0000 7f496dffb700 1 -- 172.21.15.17:0/569398109 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4968042c00 con 0x7f49700b6a90 2022-08-01T19:43:51.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:51.605+0000 7f4976e1c700 1 -- 172.21.15.17:0/569398109 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_statfs_on_deleted_fs"]} v 0) v1 -- 0x7f49700574e0 con 0x7f49700b6a90 2022-08-01T19:43:52.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.289+0000 7f496dffb700 1 -- 172.21.15.17:0/569398109 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_statfs_on_deleted_fs"]}]=0 v452) v1 ==== 139+0+0 (secure 0 0 0) 0x7f496800f040 con 0x7f49700b6a90 2022-08-01T19:43:52.303 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.293+0000 7f4976e1c700 1 -- 172.21.15.17:0/569398109 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f49500539e0 msgr2=0x7f4950055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:52.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.293+0000 7f4976e1c700 1 --2- 172.21.15.17:0/569398109 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f49500539e0 0x7f4950055ea0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7f496401c000 tx=0x7f4964015000).stop 2022-08-01T19:43:52.304 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.293+0000 7f4976e1c700 1 -- 172.21.15.17:0/569398109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49700b6a90 msgr2=0x7f4970150d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:52.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.293+0000 7f4976e1c700 1 --2- 172.21.15.17:0/569398109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49700b6a90 0x7f4970150d40 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 rx=0x7f496800a000 tx=0x7f4968008040).stop 2022-08-01T19:43:52.305 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.293+0000 7f4976e1c700 1 -- 172.21.15.17:0/569398109 shutdown_connections 2022-08-01T19:43:52.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.293+0000 7f4976e1c700 1 --2- 172.21.15.17:0/569398109 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f49500539e0 0x7f4950055ea0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:52.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.293+0000 7f4976e1c700 1 --2- 172.21.15.17:0/569398109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49700b6a90 0x7f4970150d40 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:52.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.293+0000 7f4976e1c700 1 --2- 172.21.15.17:0/569398109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49700aa760 0x7f4970150850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:52.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.293+0000 7f4976e1c700 1 --2- 172.21.15.17:0/569398109 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4970060aa0 0x7f497013bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:52.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.293+0000 7f4976e1c700 1 -- 172.21.15.17:0/569398109 >> 172.21.15.17:0/569398109 conn(0x7f49700b3610 msgr2=0x7f49700b6020 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:52.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.297+0000 7f4976e1c700 1 -- 172.21.15.17:0/569398109 shutdown_connections 2022-08-01T19:43:52.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.297+0000 7f4976e1c700 1 -- 172.21.15.17:0/569398109 wait complete. 2022-08-01T19:43:52.315 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-08-01T19:43:52.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.433+0000 7fa4f759e700 1 Processor -- start 2022-08-01T19:43:52.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.433+0000 7fa4f759e700 1 -- start start 2022-08-01T19:43:52.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.433+0000 7fa4f759e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa4f80b3190 0x7fa4f8060ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:52.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.433+0000 7fa4f759e700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4f80654c0 con 0x7fa4f80b2730 2022-08-01T19:43:52.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.433+0000 7fa4f759e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4f80b3570 con 0x7fa4f8061190 2022-08-01T19:43:52.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.433+0000 7fa4f759e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f8062190 con 0x7fa4f80b3190 2022-08-01T19:43:52.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.433+0000 7fa4f5d9b700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa4f80b2730 0x7fa4f80666b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37554/0 (socket says 172.21.15.17:37554) 2022-08-01T19:43:52.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.433+0000 7fa4f5d9b700 1 -- 172.21.15.17:0/3394533545 learned_addr learned my addr 172.21.15.17:0/3394533545 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:52.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.437+0000 7fa4f659c700 1 --2- 172.21.15.17:0/3394533545 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa4f80b3190 0x7fa4f8060ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:52.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.437+0000 7fa4f559a700 1 -- 172.21.15.17:0/3394533545 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3817667847 0 0) 0x7fa4f80654c0 con 0x7fa4f80b2730 2022-08-01T19:43:52.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.437+0000 7fa4f559a700 1 -- 172.21.15.17:0/3394533545 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4dc003260 con 0x7fa4f80b2730 2022-08-01T19:43:52.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.437+0000 7fa4f559a700 1 -- 172.21.15.17:0/3394533545 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 957549120 0 0) 0x7fa4f80b3570 con 0x7fa4f8061190 2022-08-01T19:43:52.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.437+0000 7fa4f559a700 1 -- 172.21.15.17:0/3394533545 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4f80654c0 con 0x7fa4f8061190 2022-08-01T19:43:52.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.437+0000 7fa4f659c700 1 -- 172.21.15.17:0/3394533545 >> v1:172.21.15.77:6789/0 conn(0x7fa4f8061190 legacy=0x7fa4f8066dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:52.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.437+0000 7fa4f659c700 1 -- 172.21.15.17:0/3394533545 >> v1:172.21.15.17:6789/0 conn(0x7fa4f80b2730 legacy=0x7fa4f80666b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:52.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.437+0000 7fa4f659c700 1 -- 172.21.15.17:0/3394533545 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f805fbf0 con 0x7fa4f80b3190 2022-08-01T19:43:52.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.437+0000 7fa4f659c700 1 --2- 172.21.15.17:0/3394533545 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa4f80b3190 0x7fa4f8060ca0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7fa4e8002f70 tx=0x7fa4e8006df0).ready entity=mon.2 client_cookie=4e315b035bd87051 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:52.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.437+0000 7fa4f559a700 1 -- 172.21.15.17:0/3394533545 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4e800d5c0 con 0x7fa4f80b3190 2022-08-01T19:43:52.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.437+0000 7fa4f559a700 1 -- 172.21.15.17:0/3394533545 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e8000e90 con 0x7fa4f80b3190 2022-08-01T19:43:52.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.437+0000 7fa4f559a700 1 -- 172.21.15.17:0/3394533545 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4e800d910 con 0x7fa4f80b3190 2022-08-01T19:43:52.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.437+0000 7fa4f759e700 1 -- 172.21.15.17:0/3394533545 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa4f80b3190 msgr2=0x7fa4f8060ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:52.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.437+0000 7fa4f759e700 1 --2- 172.21.15.17:0/3394533545 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa4f80b3190 0x7fa4f8060ca0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7fa4e8002f70 tx=0x7fa4e8006df0).stop 2022-08-01T19:43:52.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.437+0000 7fa4f759e700 1 -- 172.21.15.17:0/3394533545 shutdown_connections 2022-08-01T19:43:52.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.437+0000 7fa4f759e700 1 --2- 172.21.15.17:0/3394533545 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa4f80b3190 0x7fa4f8060ca0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:52.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.437+0000 7fa4f759e700 1 -- 172.21.15.17:0/3394533545 >> 172.21.15.17:0/3394533545 conn(0x7fa4f8063870 msgr2=0x7fa4f8063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:52.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.437+0000 7fa4f759e700 1 -- 172.21.15.17:0/3394533545 shutdown_connections 2022-08-01T19:43:52.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.437+0000 7fa4f759e700 1 -- 172.21.15.17:0/3394533545 wait complete. 2022-08-01T19:43:52.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.441+0000 7fa4f759e700 1 Processor -- start 2022-08-01T19:43:52.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.441+0000 7fa4f759e700 1 -- start start 2022-08-01T19:43:52.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.441+0000 7fa4f759e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4f8061190 0x7fa4f8156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:52.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.441+0000 7fa4f759e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4f80b2730 0x7fa4f80b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:52.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.441+0000 7fa4f759e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa4f80b3190 0x7fa4f80b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:52.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.441+0000 7fa4f759e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f8156560 con 0x7fa4f8061190 2022-08-01T19:43:52.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.441+0000 7fa4f759e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f80a4640 con 0x7fa4f80b2730 2022-08-01T19:43:52.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.441+0000 7fa4f759e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f8155690 con 0x7fa4f80b3190 2022-08-01T19:43:52.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.441+0000 7fa4f659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4f8061190 0x7fa4f8156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:52.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.441+0000 7fa4f659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4f8061190 0x7fa4f8156070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45364/0 (socket says 172.21.15.17:45364) 2022-08-01T19:43:52.499 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.441+0000 7fa4f659c700 1 -- 172.21.15.17:0/3087955629 learned_addr learned my addr 172.21.15.17:0/3087955629 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:52.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.441+0000 7fa4f5d9b700 1 --2- 172.21.15.17:0/3087955629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4f80b2730 0x7fa4f80b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:52.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.441+0000 7fa4f6d9d700 1 --2- 172.21.15.17:0/3087955629 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa4f80b3190 0x7fa4f80b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:52.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.441+0000 7fa4f659c700 1 -- 172.21.15.17:0/3087955629 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa4f80b3190 msgr2=0x7fa4f80b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:52.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.441+0000 7fa4f659c700 1 --2- 172.21.15.17:0/3087955629 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa4f80b3190 0x7fa4f80b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:52.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.441+0000 7fa4f659c700 1 -- 172.21.15.17:0/3087955629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4f80b2730 msgr2=0x7fa4f80b4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:52.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.441+0000 7fa4f659c700 1 --2- 172.21.15.17:0/3087955629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4f80b2730 0x7fa4f80b4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:52.515 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.441+0000 7fa4f659c700 1 -- 172.21.15.17:0/3087955629 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e800b180 con 0x7fa4f8061190 2022-08-01T19:43:52.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.441+0000 7fa4f6d9d700 1 --2- 172.21.15.17:0/3087955629 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa4f80b3190 0x7fa4f80b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:43:52.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.441+0000 7fa4f659c700 1 --2- 172.21.15.17:0/3087955629 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4f8061190 0x7fa4f8156070 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 rx=0x7fa4e8002f70 tx=0x7fa4e8014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:52.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.441+0000 7fa4e77fe700 1 -- 172.21.15.17:0/3087955629 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4e8016070 con 0x7fa4f8061190 2022-08-01T19:43:52.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.441+0000 7fa4e77fe700 1 -- 172.21.15.17:0/3087955629 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e80137d0 con 0x7fa4f8061190 2022-08-01T19:43:52.516 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.441+0000 7fa4f759e700 1 -- 172.21.15.17:0/3087955629 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4f8155930 con 0x7fa4f8061190 2022-08-01T19:43:52.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.445+0000 7fa4e77fe700 1 -- 172.21.15.17:0/3087955629 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4e8013b30 con 0x7fa4f8061190 2022-08-01T19:43:52.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.445+0000 7fa4f759e700 1 -- 172.21.15.17:0/3087955629 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4f80b5450 con 0x7fa4f8061190 2022-08-01T19:43:52.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.445+0000 7fa4f759e700 1 -- 172.21.15.17:0/3087955629 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4f80574e0 con 0x7fa4f8061190 2022-08-01T19:43:52.517 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.445+0000 7fa4e77fe700 1 -- 172.21.15.17:0/3087955629 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa4e8025030 con 0x7fa4f8061190 2022-08-01T19:43:52.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.445+0000 7fa4e77fe700 1 --2- 172.21.15.17:0/3087955629 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa4d4053950 0x7fa4d4055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:52.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.445+0000 7fa4e77fe700 1 -- 172.21.15.17:0/3087955629 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fa4e806cbd0 con 0x7fa4f8061190 2022-08-01T19:43:52.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.445+0000 7fa4f5d9b700 1 --2- 172.21.15.17:0/3087955629 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa4d4053950 0x7fa4d4055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:52.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.449+0000 7fa4f5d9b700 1 --2- 172.21.15.17:0/3087955629 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa4d4053950 0x7fa4d4055e10 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7fa4ec00d000 tx=0x7fa4ec009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:52.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.453+0000 7fa4e77fe700 1 -- 172.21.15.17:0/3087955629 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa4e8043c70 con 0x7fa4f8061190 2022-08-01T19:43:52.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.581+0000 7fa4f759e700 1 -- 172.21.15.17:0/3087955629 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa4f8069a70 con 0x7fa4f8061190 2022-08-01T19:43:52.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.589+0000 7fa4e77fe700 1 -- 172.21.15.17:0/3087955629 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa4e80230c0 con 0x7fa4f8061190 2022-08-01T19:43:52.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.593+0000 7fa4f759e700 1 -- 172.21.15.17:0/3087955629 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa4d4053950 msgr2=0x7fa4d4055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:52.599 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.593+0000 7fa4f759e700 1 --2- 172.21.15.17:0/3087955629 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa4d4053950 0x7fa4d4055e10 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7fa4ec00d000 tx=0x7fa4ec009040).stop 2022-08-01T19:43:52.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.593+0000 7fa4f759e700 1 -- 172.21.15.17:0/3087955629 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4f8061190 msgr2=0x7fa4f8156070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:52.600 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.593+0000 7fa4f759e700 1 --2- 172.21.15.17:0/3087955629 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4f8061190 0x7fa4f8156070 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 rx=0x7fa4e8002f70 tx=0x7fa4e8014000).stop 2022-08-01T19:43:52.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.593+0000 7fa4f759e700 1 -- 172.21.15.17:0/3087955629 shutdown_connections 2022-08-01T19:43:52.601 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.593+0000 7fa4f759e700 1 --2- 172.21.15.17:0/3087955629 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa4d4053950 0x7fa4d4055e10 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:52.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.593+0000 7fa4f759e700 1 --2- 172.21.15.17:0/3087955629 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa4f80b3190 0x7fa4f80b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:52.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.593+0000 7fa4f759e700 1 --2- 172.21.15.17:0/3087955629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4f80b2730 0x7fa4f80b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:52.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.593+0000 7fa4f759e700 1 --2- 172.21.15.17:0/3087955629 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4f8061190 0x7fa4f8156070 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:52.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.593+0000 7fa4f759e700 1 -- 172.21.15.17:0/3087955629 >> 172.21.15.17:0/3087955629 conn(0x7fa4f8063870 msgr2=0x7fa4f8066e90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:52.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.593+0000 7fa4f759e700 1 -- 172.21.15.17:0/3087955629 shutdown_connections 2022-08-01T19:43:52.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.593+0000 7fa4f759e700 1 -- 172.21.15.17:0/3087955629 wait complete. 2022-08-01T19:43:52.611 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T19:43:52.612 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:43:52.636 DEBUG:teuthology.orchestra.run.smithi077:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:43:52.661 DEBUG:teuthology.orchestra.run.smithi085:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:43:52.689 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:43:52.712 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T19:43:52.714 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:52.715 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:43:52.716 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:43:52.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:43:52.738 INFO:teuthology.orchestra.run.smithi085.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-08-01T19:43:52.740 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-08-01T19:43:52.750 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:43:52.751 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:43:52.751 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:43:52.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:43:52.766 INFO:teuthology.orchestra.run.smithi190.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-08-01T19:43:52.766 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-08-01T19:43:52.778 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T19:43:52.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.893+0000 7f7f7c16b700 1 Processor -- start 2022-08-01T19:43:52.917 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.893+0000 7f7f7c16b700 1 -- start start 2022-08-01T19:43:52.918 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.893+0000 7f7f7c16b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7f740b1f40 0x7f7f740b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:52.918 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.893+0000 7f7f7c16b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7f740a7250 0x7f7f741448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:52.919 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.893+0000 7f7f7c16b700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f740b04e0 con 0x7f7f74148530 2022-08-01T19:43:52.919 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.893+0000 7f7f7c16b700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f740a9250 con 0x7f7f740a7250 2022-08-01T19:43:52.919 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.893+0000 7f7f7c16b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f740b0780 con 0x7f7f740b1f40 2022-08-01T19:43:52.919 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.893+0000 7f7f7a708700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7f740a7250 0x7f7f741448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:52.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.893+0000 7f7f7a708700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7f740a7250 0x7f7f741448b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45376/0 (socket says 172.21.15.17:45376) 2022-08-01T19:43:52.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.893+0000 7f7f7a708700 1 -- 172.21.15.17:0/3254527010 learned_addr learned my addr 172.21.15.17:0/3254527010 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:52.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.893+0000 7f7f79f07700 1 --2- 172.21.15.17:0/3254527010 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7f740b1f40 0x7f7f740b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:52.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.893+0000 7f7f7a708700 1 -- 172.21.15.17:0/3254527010 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7f740b1f40 msgr2=0x7f7f740b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:52.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.893+0000 7f7f7a708700 1 --2- 172.21.15.17:0/3254527010 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7f740b1f40 0x7f7f740b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:52.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.893+0000 7f7f7a708700 1 -- 172.21.15.17:0/3254527010 >> v1:172.21.15.77:6789/0 conn(0x7f7f74148530 legacy=0x7f7f740b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:52.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.893+0000 7f7f7a708700 1 -- 172.21.15.17:0/3254527010 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f740b0a20 con 0x7f7f740a7250 2022-08-01T19:43:52.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.893+0000 7f7f7a708700 1 --2- 172.21.15.17:0/3254527010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f740a7250 0x7f7f741448b0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 rx=0x7f7f6c002f70 tx=0x7f7f6c006eb0).ready entity=mon.0 client_cookie=93b60eb1dc27b883 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:52.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.893+0000 7f7f78f05700 1 -- 172.21.15.17:0/3254527010 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f6c00d5c0 con 0x7f7f740a7250 2022-08-01T19:43:52.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.893+0000 7f7f78f05700 1 -- 172.21.15.17:0/3254527010 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6c000e90 con 0x7f7f740a7250 2022-08-01T19:43:52.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.893+0000 7f7f78f05700 1 -- 172.21.15.17:0/3254527010 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f6c00d950 con 0x7f7f740a7250 2022-08-01T19:43:52.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.893+0000 7f7f7c16b700 1 -- 172.21.15.17:0/3254527010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f740a7250 msgr2=0x7f7f741448b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:52.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.893+0000 7f7f7c16b700 1 --2- 172.21.15.17:0/3254527010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f740a7250 0x7f7f741448b0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 rx=0x7f7f6c002f70 tx=0x7f7f6c006eb0).stop 2022-08-01T19:43:52.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.893+0000 7f7f7c16b700 1 -- 172.21.15.17:0/3254527010 shutdown_connections 2022-08-01T19:43:52.924 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.893+0000 7f7f7c16b700 1 --2- 172.21.15.17:0/3254527010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f740a7250 0x7f7f741448b0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:52.924 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.893+0000 7f7f7c16b700 1 --2- 172.21.15.17:0/3254527010 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7f740b1f40 0x7f7f740b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:52.924 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.893+0000 7f7f7c16b700 1 -- 172.21.15.17:0/3254527010 >> 172.21.15.17:0/3254527010 conn(0x7f7f7405f840 msgr2=0x7f7f7405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:52.925 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.893+0000 7f7f7c16b700 1 -- 172.21.15.17:0/3254527010 shutdown_connections 2022-08-01T19:43:52.925 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.893+0000 7f7f7c16b700 1 -- 172.21.15.17:0/3254527010 wait complete. 2022-08-01T19:43:52.925 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.897+0000 7f7f7c16b700 1 Processor -- start 2022-08-01T19:43:52.925 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.897+0000 7f7f7c16b700 1 -- start start 2022-08-01T19:43:52.926 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.897+0000 7f7f7c16b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f740a7250 0x7f7f7414e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:52.926 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.897+0000 7f7f7c16b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f740b1f40 0x7f7f7415abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:52.926 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.897+0000 7f7f7c16b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7f74148530 0x7f7f7415b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:52.926 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.897+0000 7f7f7c16b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7413bb20 con 0x7f7f740a7250 2022-08-01T19:43:52.927 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.897+0000 7f7f7c16b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7415a600 con 0x7f7f740b1f40 2022-08-01T19:43:52.927 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.897+0000 7f7f7c16b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7413ca00 con 0x7f7f74148530 2022-08-01T19:43:52.927 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.897+0000 7f7f79f07700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f740a7250 0x7f7f7414e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:52.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.897+0000 7f7f7a708700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7f74148530 0x7f7f7415b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:52.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.897+0000 7f7f79f07700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f740a7250 0x7f7f7414e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45380/0 (socket says 172.21.15.17:45380) 2022-08-01T19:43:52.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.897+0000 7f7f79f07700 1 -- 172.21.15.17:0/1394593699 learned_addr learned my addr 172.21.15.17:0/1394593699 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:52.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.897+0000 7f7f79706700 1 --2- 172.21.15.17:0/1394593699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f740b1f40 0x7f7f7415abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:52.929 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.897+0000 7f7f79f07700 1 -- 172.21.15.17:0/1394593699 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7f74148530 msgr2=0x7f7f7415b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:52.929 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.897+0000 7f7f79f07700 1 --2- 172.21.15.17:0/1394593699 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7f74148530 0x7f7f7415b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:52.929 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.897+0000 7f7f79f07700 1 -- 172.21.15.17:0/1394593699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f740b1f40 msgr2=0x7f7f7415abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:52.929 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.897+0000 7f7f79f07700 1 --2- 172.21.15.17:0/1394593699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f740b1f40 0x7f7f7415abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:52.930 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.897+0000 7f7f79f07700 1 -- 172.21.15.17:0/1394593699 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6c00b1e0 con 0x7f7f740a7250 2022-08-01T19:43:52.930 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.897+0000 7f7f79f07700 1 --2- 172.21.15.17:0/1394593699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f740a7250 0x7f7f7414e670 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 rx=0x7f7f70003430 tx=0x7f7f70008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:52.930 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.897+0000 7f7f62ffd700 1 -- 172.21.15.17:0/1394593699 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f70013620 con 0x7f7f740a7250 2022-08-01T19:43:52.931 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.901+0000 7f7f62ffd700 1 -- 172.21.15.17:0/1394593699 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f70013780 con 0x7f7f740a7250 2022-08-01T19:43:52.931 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.901+0000 7f7f7c16b700 1 -- 172.21.15.17:0/1394593699 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f7413cca0 con 0x7f7f740a7250 2022-08-01T19:43:52.931 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.901+0000 7f7f62ffd700 1 -- 172.21.15.17:0/1394593699 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f70013ae0 con 0x7f7f740a7250 2022-08-01T19:43:52.931 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.901+0000 7f7f7c16b700 1 -- 172.21.15.17:0/1394593699 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f7413d1e0 con 0x7f7f740a7250 2022-08-01T19:43:52.932 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.901+0000 7f7f7c16b700 1 -- 172.21.15.17:0/1394593699 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f7413d700 con 0x7f7f740a7250 2022-08-01T19:43:52.932 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.901+0000 7f7f62ffd700 1 -- 172.21.15.17:0/1394593699 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7f70013c40 con 0x7f7f740a7250 2022-08-01T19:43:52.932 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.901+0000 7f7f62ffd700 1 --2- 172.21.15.17:0/1394593699 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7f54053a70 0x7f7f54055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:52.933 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.905+0000 7f7f62ffd700 1 -- 172.21.15.17:0/1394593699 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f7f7006bc00 con 0x7f7f740a7250 2022-08-01T19:43:52.933 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.905+0000 7f7f79706700 1 --2- 172.21.15.17:0/1394593699 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7f54053a70 0x7f7f54055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:52.933 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.905+0000 7f7f79706700 1 --2- 172.21.15.17:0/1394593699 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7f54053a70 0x7f7f54055f30 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f7f6400d000 tx=0x7f7f64007b70).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:52.933 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:52.909+0000 7f7f62ffd700 1 -- 172.21.15.17:0/1394593699 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7f7413d700 con 0x7f7f740a7250 2022-08-01T19:43:53.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.037+0000 7f7f7c16b700 1 -- 172.21.15.17:0/1394593699 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f7f7413d700 con 0x7f7f740a7250 2022-08-01T19:43:53.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.037+0000 7f7f62ffd700 1 -- 172.21.15.17:0/1394593699 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v151) v1 ==== 81+0+29229 (secure 0 0 0) 0x7f7f7413d700 con 0x7f7f740a7250 2022-08-01T19:43:53.044 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:53.044 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:43:53.045 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 151, 2022-08-01T19:43:53.045 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", 2022-08-01T19:43:53.045 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2022-08-01T19:33:50.105743+0000", 2022-08-01T19:43:53.045 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2022-08-01T19:43:50.274238+0000", 2022-08-01T19:43:53.047 INFO:teuthology.orchestra.run.smithi017.stdout: "last_up_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:43:53.048 INFO:teuthology.orchestra.run.smithi017.stdout: "last_in_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:43:53.049 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T19:43:53.049 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_num": 5799936, 2022-08-01T19:43:53.050 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_set": [ 2022-08-01T19:43:53.050 INFO:teuthology.orchestra.run.smithi017.stdout: "pglog_hardlimit", 2022-08-01T19:43:53.051 INFO:teuthology.orchestra.run.smithi017.stdout: "purged_snapdirs", 2022-08-01T19:43:53.051 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_deletes", 2022-08-01T19:43:53.051 INFO:teuthology.orchestra.run.smithi017.stdout: "sortbitwise" 2022-08-01T19:43:53.052 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:43:53.052 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_version": 6, 2022-08-01T19:43:53.052 INFO:teuthology.orchestra.run.smithi017.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T19:43:53.052 INFO:teuthology.orchestra.run.smithi017.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T19:43:53.053 INFO:teuthology.orchestra.run.smithi017.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T19:43:53.053 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_snapshot": "", 2022-08-01T19:43:53.053 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_max": 28, 2022-08-01T19:43:53.053 INFO:teuthology.orchestra.run.smithi017.stdout: "max_osd": 8, 2022-08-01T19:43:53.054 INFO:teuthology.orchestra.run.smithi017.stdout: "require_min_compat_client": "jewel", 2022-08-01T19:43:53.054 INFO:teuthology.orchestra.run.smithi017.stdout: "min_compat_client": "jewel", 2022-08-01T19:43:53.054 INFO:teuthology.orchestra.run.smithi017.stdout: "require_osd_release": "octopus", 2022-08-01T19:43:53.054 INFO:teuthology.orchestra.run.smithi017.stdout: "pools": [ 2022-08-01T19:43:53.056 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.056 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 1, 2022-08-01T19:43:53.056 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "device_health_metrics", 2022-08-01T19:43:53.056 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:33:53.001841+0000", 2022-08-01T19:43:53.057 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:43:53.057 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:43:53.057 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:43:53.057 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:43:53.058 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:43:53.058 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:43:53.058 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:43:53.059 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:43:53.059 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 1, 2022-08-01T19:43:53.059 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 1, 2022-08-01T19:43:53.059 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 1, 2022-08-01T19:43:53.060 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 1, 2022-08-01T19:43:53.060 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 1, 2022-08-01T19:43:53.060 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:43:53.060 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:43:53.061 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:43:53.061 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:43:53.061 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:43:53.062 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:43:53.062 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:43:53.062 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.062 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "16", 2022-08-01T19:43:53.063 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:43:53.063 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:43:53.063 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:43:53.064 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:43:53.064 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:43:53.064 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:43:53.064 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:43:53.065 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:43:53.065 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:43:53.065 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:43:53.065 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:43:53.067 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:43:53.067 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:43:53.068 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:43:53.068 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:43:53.068 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:43:53.068 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:43:53.069 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:43:53.069 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:43:53.069 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:43:53.070 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:43:53.070 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:43:53.071 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:43:53.071 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:43:53.071 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:43:53.071 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:43:53.072 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.072 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:43:53.073 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:43:53.073 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:43:53.073 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:43:53.073 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:43:53.074 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:43:53.074 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:43:53.074 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:43:53.074 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:43:53.075 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:43:53.075 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:43:53.075 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:43:53.076 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 1 2022-08-01T19:43:53.076 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.076 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:43:53.077 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_devicehealth": {} 2022-08-01T19:43:53.077 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.078 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.078 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.078 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 2, 2022-08-01T19:43:53.078 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "rbd", 2022-08-01T19:43:53.079 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:34:01.553098+0000", 2022-08-01T19:43:53.079 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:43:53.087 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:43:53.087 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:43:53.087 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:43:53.087 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:43:53.088 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:43:53.088 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:43:53.088 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:43:53.088 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:43:53.089 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:43:53.089 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:43:53.089 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:43:53.089 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:43:53.090 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:43:53.090 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:43:53.090 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:43:53.091 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:43:53.091 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:43:53.091 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:43:53.092 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:43:53.092 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.092 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "19", 2022-08-01T19:43:53.092 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:43:53.093 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:43:53.093 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:43:53.093 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:43:53.094 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:43:53.094 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:43:53.094 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:43:53.094 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:43:53.095 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:43:53.095 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:43:53.095 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:43:53.095 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:43:53.096 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:43:53.096 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:43:53.096 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:43:53.097 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:43:53.097 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:43:53.097 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:43:53.097 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:43:53.098 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:43:53.098 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:43:53.098 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:43:53.098 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:43:53.099 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:43:53.099 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:43:53.099 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:43:53.099 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.100 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:43:53.100 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:43:53.100 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:43:53.100 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:43:53.101 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:43:53.101 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:43:53.101 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:43:53.101 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:43:53.102 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:43:53.102 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:43:53.102 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:43:53.102 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:43:53.103 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:43:53.103 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd": {} 2022-08-01T19:43:53.103 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.103 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.104 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:43:53.104 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2022-08-01T19:43:53.104 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.104 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:43:53.105 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a962f14f-cf16-4fa8-84ff-c85325db9132", 2022-08-01T19:43:53.105 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:43:53.105 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:43:53.105 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:43:53.106 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:43:53.106 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:43:53.106 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:43:53.108 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:43:53.108 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 145, 2022-08-01T19:43:53.108 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:43:53.108 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:43:53.109 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:43:53.109 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:53.109 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.110 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:53.110 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6800", 2022-08-01T19:43:53.110 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:43:53.110 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.111 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.111 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:53.111 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6801", 2022-08-01T19:43:53.111 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:43:53.112 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.112 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.112 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.114 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:43:53.115 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:53.115 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.116 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:53.116 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6802", 2022-08-01T19:43:53.116 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:43:53.117 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.118 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.118 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:53.118 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6803", 2022-08-01T19:43:53.118 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:43:53.119 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.119 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.120 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.120 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:43:53.122 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:53.122 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.123 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:53.123 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6806", 2022-08-01T19:43:53.124 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:43:53.124 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.124 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.125 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:53.127 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6807", 2022-08-01T19:43:53.127 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:43:53.128 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.128 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.129 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.129 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:43:53.130 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:53.130 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.131 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:53.131 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6804", 2022-08-01T19:43:53.132 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:43:53.132 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.133 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.133 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:53.134 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6805", 2022-08-01T19:43:53.134 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:43:53.135 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.135 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.136 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.136 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6801/23008", 2022-08-01T19:43:53.137 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6803/23008", 2022-08-01T19:43:53.137 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6807/23008", 2022-08-01T19:43:53.138 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6805/23008", 2022-08-01T19:43:53.138 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:43:53.139 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:43:53.139 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:43:53.140 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.140 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.140 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.141 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:43:53.141 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "805f7051-a527-4e98-8844-b6e406a63248", 2022-08-01T19:43:53.142 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:43:53.142 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:43:53.143 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:43:53.143 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:43:53.144 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:43:53.144 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:43:53.145 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:43:53.146 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 145, 2022-08-01T19:43:53.146 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:43:53.146 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:43:53.147 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:43:53.147 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:53.148 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.148 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:53.148 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6808", 2022-08-01T19:43:53.148 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:43:53.149 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.149 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.149 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:53.150 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6809", 2022-08-01T19:43:53.150 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:43:53.150 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.150 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.151 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.151 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:43:53.151 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:53.151 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.152 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:53.152 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6810", 2022-08-01T19:43:53.152 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:43:53.152 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.153 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.154 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:53.155 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6811", 2022-08-01T19:43:53.155 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:43:53.156 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.156 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.156 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.157 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:43:53.157 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:53.158 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.158 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:53.159 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6814", 2022-08-01T19:43:53.159 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:43:53.159 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.160 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.160 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:53.162 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6815", 2022-08-01T19:43:53.162 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:43:53.163 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.164 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.165 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.166 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:43:53.166 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:53.167 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.167 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:53.168 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6812", 2022-08-01T19:43:53.168 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:43:53.169 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.169 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.170 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:53.170 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6813", 2022-08-01T19:43:53.171 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:43:53.171 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.171 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.179 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.180 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6809/23007", 2022-08-01T19:43:53.180 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6811/23007", 2022-08-01T19:43:53.180 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6815/23007", 2022-08-01T19:43:53.181 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6813/23007", 2022-08-01T19:43:53.181 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:43:53.182 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:43:53.182 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:43:53.183 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.183 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.184 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.184 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:43:53.185 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "89f7f3df-7c10-48e1-8b69-9e4243923a61", 2022-08-01T19:43:53.185 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:43:53.187 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:43:53.188 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:43:53.188 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:43:53.188 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:43:53.188 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:43:53.189 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:43:53.189 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 144, 2022-08-01T19:43:53.189 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:43:53.189 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:43:53.190 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:43:53.190 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:53.190 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.190 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:53.190 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6817", 2022-08-01T19:43:53.191 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:43:53.191 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.191 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.192 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:53.193 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6819", 2022-08-01T19:43:53.194 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:43:53.194 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.194 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.194 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.195 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:43:53.195 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:53.195 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.195 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:53.196 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6822", 2022-08-01T19:43:53.196 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:43:53.196 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.196 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.198 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:53.198 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6824", 2022-08-01T19:43:53.198 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:43:53.198 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.199 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.199 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.199 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:43:53.199 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:53.199 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.200 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:53.200 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6830", 2022-08-01T19:43:53.200 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:43:53.200 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.201 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.201 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:53.201 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6831", 2022-08-01T19:43:53.201 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:43:53.201 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.202 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.202 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.202 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:43:53.202 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:53.203 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.203 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:53.203 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6826", 2022-08-01T19:43:53.203 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:43:53.204 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.204 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.204 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:53.204 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6829", 2022-08-01T19:43:53.204 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:43:53.205 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.205 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.205 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.205 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6819/23010", 2022-08-01T19:43:53.206 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6824/23010", 2022-08-01T19:43:53.206 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6831/23010", 2022-08-01T19:43:53.206 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6829/23010", 2022-08-01T19:43:53.206 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:43:53.206 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:43:53.207 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:43:53.207 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.207 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.207 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.208 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:43:53.208 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "96123468-aabc-4a64-909e-a192c369090d", 2022-08-01T19:43:53.208 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:43:53.208 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:43:53.215 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:43:53.216 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:43:53.216 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:43:53.217 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:43:53.218 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:43:53.218 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 134, 2022-08-01T19:43:53.219 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:43:53.219 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:43:53.220 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:43:53.220 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:53.221 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.221 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:53.222 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6816", 2022-08-01T19:43:53.222 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:43:53.223 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.223 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.223 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:53.224 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6818", 2022-08-01T19:43:53.224 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:43:53.244 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.245 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.245 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.246 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:43:53.246 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:53.246 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.246 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:53.247 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6820", 2022-08-01T19:43:53.247 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:43:53.251 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.251 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.251 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:53.252 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6821", 2022-08-01T19:43:53.252 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:43:53.252 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.252 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.253 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.253 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:43:53.253 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:53.253 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.256 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:53.256 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6827", 2022-08-01T19:43:53.256 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:43:53.256 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.257 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.257 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:53.257 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6828", 2022-08-01T19:43:53.257 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:43:53.258 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.258 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.259 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.259 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:43:53.259 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:53.259 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.260 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:53.260 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6823", 2022-08-01T19:43:53.260 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:43:53.260 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.260 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.261 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:53.261 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6825", 2022-08-01T19:43:53.261 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:43:53.261 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.262 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.262 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.262 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6818/23009", 2022-08-01T19:43:53.262 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6821/23009", 2022-08-01T19:43:53.263 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6828/23009", 2022-08-01T19:43:53.263 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6825/23009", 2022-08-01T19:43:53.263 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:43:53.263 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:43:53.264 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:43:53.264 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.264 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.264 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.265 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:43:53.265 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a6e26665-e04b-46ec-ab66-e912acbf044c", 2022-08-01T19:43:53.265 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:43:53.265 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:43:53.266 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:43:53.266 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:43:53.266 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:43:53.267 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:43:53.267 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:43:53.267 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 144, 2022-08-01T19:43:53.267 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:43:53.268 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:43:53.268 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:43:53.269 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:53.269 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.269 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:53.270 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6810", 2022-08-01T19:43:53.270 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:43:53.272 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.272 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.272 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:53.272 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6811", 2022-08-01T19:43:53.282 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:43:53.283 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.283 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.284 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.284 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:43:53.285 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:53.285 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.285 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:53.285 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6812", 2022-08-01T19:43:53.290 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:43:53.291 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.291 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.292 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:53.293 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6813", 2022-08-01T19:43:53.293 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:43:53.294 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.294 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.295 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.295 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:43:53.296 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:53.296 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.296 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:53.297 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6816", 2022-08-01T19:43:53.297 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:43:53.298 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.298 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.299 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:53.300 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6817", 2022-08-01T19:43:53.300 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:43:53.301 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.301 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.302 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.302 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:43:53.303 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:53.303 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.303 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:53.304 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6814", 2022-08-01T19:43:53.304 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:43:53.305 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.305 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.305 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:53.306 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6815", 2022-08-01T19:43:53.306 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:43:53.307 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.307 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.308 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.308 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6811/23584", 2022-08-01T19:43:53.309 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6813/23584", 2022-08-01T19:43:53.309 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6817/23584", 2022-08-01T19:43:53.311 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6815/23584", 2022-08-01T19:43:53.311 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:43:53.312 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:43:53.312 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:43:53.322 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.323 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.324 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.324 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:43:53.324 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "502c0034-18ee-44ac-87ff-19b9dddb66a0", 2022-08-01T19:43:53.324 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:43:53.327 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:43:53.327 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:43:53.327 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:43:53.327 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:43:53.328 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:43:53.328 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:43:53.328 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 145, 2022-08-01T19:43:53.329 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:43:53.329 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:43:53.330 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:43:53.330 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:53.331 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.331 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:53.331 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6802", 2022-08-01T19:43:53.331 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:43:53.331 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.332 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.332 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:53.332 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6803", 2022-08-01T19:43:53.333 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:43:53.333 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.333 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.333 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.334 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:43:53.334 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:53.341 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.341 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:53.341 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6804", 2022-08-01T19:43:53.341 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:43:53.342 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.342 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.342 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:53.342 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6805", 2022-08-01T19:43:53.343 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:43:53.343 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.343 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.343 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.344 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:43:53.344 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:53.344 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.344 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:53.345 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6808", 2022-08-01T19:43:53.345 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:43:53.345 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.345 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.345 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:53.350 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6809", 2022-08-01T19:43:53.351 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:43:53.352 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.355 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.356 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.358 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:43:53.359 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:53.380 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.381 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:53.381 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6806", 2022-08-01T19:43:53.381 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:43:53.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.382 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:53.382 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6807", 2022-08-01T19:43:53.382 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:43:53.383 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.383 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.387 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6803/23585", 2022-08-01T19:43:53.387 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6805/23585", 2022-08-01T19:43:53.387 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6809/23585", 2022-08-01T19:43:53.387 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6807/23585", 2022-08-01T19:43:53.388 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:43:53.388 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:43:53.388 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:43:53.388 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.389 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.389 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.390 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:43:53.393 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "ac4e755a-7fe7-46cb-8407-852d5af9ffc5", 2022-08-01T19:43:53.393 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:43:53.393 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:43:53.394 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:43:53.394 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:43:53.395 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:43:53.395 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:43:53.395 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:43:53.395 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 145, 2022-08-01T19:43:53.396 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:43:53.396 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:43:53.396 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:43:53.396 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:53.396 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.397 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:53.397 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6820", 2022-08-01T19:43:53.397 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:43:53.397 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.398 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.398 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:53.398 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6821", 2022-08-01T19:43:53.398 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:43:53.399 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.399 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.399 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.399 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:43:53.400 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:53.400 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.400 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:53.400 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6823", 2022-08-01T19:43:53.401 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:43:53.401 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.401 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.401 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:53.401 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6825", 2022-08-01T19:43:53.427 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:43:53.427 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.428 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.432 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.432 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:43:53.433 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:53.433 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.433 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:53.433 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6831", 2022-08-01T19:43:53.434 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:43:53.434 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.434 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.434 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:53.435 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6833", 2022-08-01T19:43:53.435 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:43:53.435 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.435 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.435 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.437 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:43:53.437 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:53.437 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.439 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:53.439 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6827", 2022-08-01T19:43:53.440 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:43:53.440 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.440 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.440 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:53.440 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6829", 2022-08-01T19:43:53.441 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:43:53.441 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.441 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.441 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.445 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6821/23587", 2022-08-01T19:43:53.447 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6825/23587", 2022-08-01T19:43:53.447 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6833/23587", 2022-08-01T19:43:53.447 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6829/23587", 2022-08-01T19:43:53.447 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:43:53.448 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:43:53.448 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:43:53.448 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.448 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.449 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.449 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:43:53.449 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "874761e2-6410-40fc-b60c-e58abe5ce489", 2022-08-01T19:43:53.449 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:43:53.450 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:43:53.450 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:43:53.451 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:43:53.451 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:43:53.451 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:43:53.451 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:43:53.452 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 145, 2022-08-01T19:43:53.452 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:43:53.452 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:43:53.453 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:43:53.453 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:53.454 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.454 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:53.454 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6818", 2022-08-01T19:43:53.454 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:43:53.455 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.455 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.455 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:53.455 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6819", 2022-08-01T19:43:53.456 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:43:53.456 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.456 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.456 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.457 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:43:53.457 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:53.457 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.460 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:53.460 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6822", 2022-08-01T19:43:53.461 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:43:53.461 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.462 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.462 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:53.463 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6824", 2022-08-01T19:43:53.464 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:43:53.464 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.465 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.465 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.470 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:43:53.471 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:53.472 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.472 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:53.472 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6830", 2022-08-01T19:43:53.472 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:43:53.472 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.473 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.473 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:53.474 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6832", 2022-08-01T19:43:53.474 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:43:53.474 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.475 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.475 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.475 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:43:53.475 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:43:53.476 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.476 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:43:53.477 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6826", 2022-08-01T19:43:53.477 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:43:53.477 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.477 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.478 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:43:53.478 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6828", 2022-08-01T19:43:53.478 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:43:53.479 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.479 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.479 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.480 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6819/23586", 2022-08-01T19:43:53.480 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6824/23586", 2022-08-01T19:43:53.480 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6832/23586", 2022-08-01T19:43:53.480 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6828/23586", 2022-08-01T19:43:53.481 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:43:53.481 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:43:53.481 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:43:53.481 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:53.482 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.482 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:43:53.482 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_xinfo": [ 2022-08-01T19:43:53.483 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.483 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:43:53.483 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:43:53.483 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:43:53.484 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:43:53.489 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:43:53.489 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:43:53.489 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.588817+0000", 2022-08-01T19:43:53.490 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:43:53.490 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.490 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.491 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:43:53.491 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:43:53.491 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:43:53.491 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:43:53.501 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:43:53.502 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:43:53.502 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.782753+0000", 2022-08-01T19:43:53.503 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:43:53.503 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.503 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.504 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:43:53.504 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:43:53.504 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:43:53.504 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:43:53.505 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:43:53.508 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:43:53.508 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.837924+0000", 2022-08-01T19:43:53.508 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:43:53.509 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.509 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.509 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:43:53.509 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:43:53.510 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:43:53.510 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:43:53.510 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:43:53.510 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:43:53.511 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.839948+0000", 2022-08-01T19:43:53.511 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:43:53.511 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.511 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.512 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:43:53.512 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:43:53.512 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:43:53.512 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:43:53.513 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:43:53.513 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:43:53.513 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.112801+0000", 2022-08-01T19:43:53.513 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:43:53.514 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.514 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.515 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:43:53.515 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:43:53.515 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:43:53.515 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:43:53.516 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:43:53.516 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:43:53.516 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.106704+0000", 2022-08-01T19:43:53.516 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:43:53.517 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.517 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.517 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:43:53.517 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:43:53.518 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:43:53.518 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:43:53.519 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:43:53.519 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:43:53.519 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.893170+0000", 2022-08-01T19:43:53.519 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:43:53.520 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.520 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:53.520 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:43:53.520 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:43:53.521 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:43:53.521 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:43:53.521 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:43:53.521 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:43:53.522 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.886947+0000", 2022-08-01T19:43:53.522 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:43:53.522 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.523 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:43:53.527 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap": [], 2022-08-01T19:43:53.527 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap_items": [], 2022-08-01T19:43:53.528 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_temp": [], 2022-08-01T19:43:53.528 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_temp": [], 2022-08-01T19:43:53.529 INFO:teuthology.orchestra.run.smithi017.stdout: "blacklist": { 2022-08-01T19:43:53.529 INFO:teuthology.orchestra.run.smithi017.stdout: "172.21.15.17:6835/2832743740": "2022-08-02T19:43:45.853755+0000", 2022-08-01T19:43:53.530 INFO:teuthology.orchestra.run.smithi017.stdout: "172.21.15.17:6834/2832743740": "2022-08-02T19:43:45.853755+0000" 2022-08-01T19:43:53.530 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.531 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profiles": { 2022-08-01T19:43:53.531 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_data": { 2022-08-01T19:43:53.532 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-device-class": "", 2022-08-01T19:43:53.532 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:43:53.532 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-root": "default", 2022-08-01T19:43:53.533 INFO:teuthology.orchestra.run.smithi017.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T19:43:53.533 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:43:53.534 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "2", 2022-08-01T19:43:53.535 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:43:53.535 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:43:53.536 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van", 2022-08-01T19:43:53.536 INFO:teuthology.orchestra.run.smithi017.stdout: "w": "8" 2022-08-01T19:43:53.537 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.537 INFO:teuthology.orchestra.run.smithi017.stdout: "default": { 2022-08-01T19:43:53.538 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:43:53.538 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:43:53.539 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "1", 2022-08-01T19:43:53.539 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:43:53.540 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:43:53.540 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van" 2022-08-01T19:43:53.546 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:53.547 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:53.547 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps_queue": [], 2022-08-01T19:43:53.547 INFO:teuthology.orchestra.run.smithi017.stdout: "new_removed_snaps": [], 2022-08-01T19:43:53.547 INFO:teuthology.orchestra.run.smithi017.stdout: "new_purged_snaps": [], 2022-08-01T19:43:53.548 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_node_flags": {}, 2022-08-01T19:43:53.553 INFO:teuthology.orchestra.run.smithi017.stdout: "device_class_flags": {} 2022-08-01T19:43:53.553 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:43:53.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.041+0000 7f7f7c16b700 1 -- 172.21.15.17:0/1394593699 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7f54053a70 msgr2=0x7f7f54055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:53.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.041+0000 7f7f7c16b700 1 --2- 172.21.15.17:0/1394593699 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7f54053a70 0x7f7f54055f30 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f7f6400d000 tx=0x7f7f64007b70).stop 2022-08-01T19:43:53.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.041+0000 7f7f7c16b700 1 -- 172.21.15.17:0/1394593699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f740a7250 msgr2=0x7f7f7414e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:53.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.041+0000 7f7f7c16b700 1 --2- 172.21.15.17:0/1394593699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f740a7250 0x7f7f7414e670 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 rx=0x7f7f70003430 tx=0x7f7f70008040).stop 2022-08-01T19:43:53.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.041+0000 7f7f7c16b700 1 -- 172.21.15.17:0/1394593699 shutdown_connections 2022-08-01T19:43:53.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.041+0000 7f7f7c16b700 1 --2- 172.21.15.17:0/1394593699 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7f54053a70 0x7f7f54055f30 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:53.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.041+0000 7f7f7c16b700 1 --2- 172.21.15.17:0/1394593699 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7f74148530 0x7f7f7415b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:53.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.041+0000 7f7f7c16b700 1 --2- 172.21.15.17:0/1394593699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f740b1f40 0x7f7f7415abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:53.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.041+0000 7f7f7c16b700 1 --2- 172.21.15.17:0/1394593699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f740a7250 0x7f7f7414e670 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:53.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.041+0000 7f7f7c16b700 1 -- 172.21.15.17:0/1394593699 >> 172.21.15.17:0/1394593699 conn(0x7f7f7405f840 msgr2=0x7f7f740aec90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:53.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.041+0000 7f7f7c16b700 1 -- 172.21.15.17:0/1394593699 shutdown_connections 2022-08-01T19:43:53.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.041+0000 7f7f7c16b700 1 -- 172.21.15.17:0/1394593699 wait complete. 2022-08-01T19:43:53.582 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:43:53.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.697+0000 7f3623695700 1 Processor -- start 2022-08-01T19:43:53.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.697+0000 7f3623695700 1 -- start start 2022-08-01T19:43:53.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.697+0000 7f3623695700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f361c0b2390 0x7f361c0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:53.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.697+0000 7f3623695700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f361c0b0160 0x7f361c0ac480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:53.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.697+0000 7f3623695700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f361c0ac970 con 0x7f361c141830 2022-08-01T19:43:53.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.697+0000 7f3623695700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f361c1453e0 con 0x7f361c0b2390 2022-08-01T19:43:53.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.697+0000 7f3623695700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f361c145680 con 0x7f361c0b0160 2022-08-01T19:43:53.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.697+0000 7f3621431700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f361c0b2390 0x7f361c0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:53.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.697+0000 7f3621431700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f361c0b2390 0x7f361c0abf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:43308/0 (socket says 172.21.15.17:43308) 2022-08-01T19:43:53.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.697+0000 7f3621c32700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7f361c141830 0x7f361c141c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:54382/0 (socket says 172.21.15.17:54382) 2022-08-01T19:43:53.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.697+0000 7f3621431700 1 -- 172.21.15.17:0/3069167008 learned_addr learned my addr 172.21.15.17:0/3069167008 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:53.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.697+0000 7f3620c30700 1 --2- 172.21.15.17:0/3069167008 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f361c0b0160 0x7f361c0ac480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:53.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.697+0000 7f3621431700 1 -- 172.21.15.17:0/3069167008 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f361c0b0160 msgr2=0x7f361c0ac480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:53.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.697+0000 7f3621431700 1 --2- 172.21.15.17:0/3069167008 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f361c0b0160 0x7f361c0ac480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:53.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.697+0000 7f3621431700 1 -- 172.21.15.17:0/3069167008 >> v1:172.21.15.77:6789/0 conn(0x7f361c141830 legacy=0x7f361c141c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:53.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.697+0000 7f3621431700 1 -- 172.21.15.17:0/3069167008 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361c141390 con 0x7f361c0b2390 2022-08-01T19:43:53.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.697+0000 7f3621431700 1 --2- 172.21.15.17:0/3069167008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f361c0b2390 0x7f361c0abf90 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f3618002f70 tx=0x7f3618006eb0).ready entity=mon.1 client_cookie=edad9a690a362349 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:53.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.697+0000 7f360bfff700 1 -- 172.21.15.17:0/3069167008 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f361800d570 con 0x7f361c0b2390 2022-08-01T19:43:53.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.697+0000 7f360bfff700 1 -- 172.21.15.17:0/3069167008 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3618000e90 con 0x7f361c0b2390 2022-08-01T19:43:53.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.697+0000 7f360bfff700 1 -- 172.21.15.17:0/3069167008 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f361800d900 con 0x7f361c0b2390 2022-08-01T19:43:53.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.697+0000 7f3623695700 1 -- 172.21.15.17:0/3069167008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f361c0b2390 msgr2=0x7f361c0abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:53.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.697+0000 7f3623695700 1 --2- 172.21.15.17:0/3069167008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f361c0b2390 0x7f361c0abf90 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f3618002f70 tx=0x7f3618006eb0).stop 2022-08-01T19:43:53.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.697+0000 7f3623695700 1 -- 172.21.15.17:0/3069167008 shutdown_connections 2022-08-01T19:43:53.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.697+0000 7f3623695700 1 --2- 172.21.15.17:0/3069167008 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f361c0b0160 0x7f361c0ac480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:53.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.697+0000 7f3623695700 1 --2- 172.21.15.17:0/3069167008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f361c0b2390 0x7f361c0abf90 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:53.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.697+0000 7f3623695700 1 -- 172.21.15.17:0/3069167008 >> 172.21.15.17:0/3069167008 conn(0x7f361c05f840 msgr2=0x7f361c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:53.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.697+0000 7f3623695700 1 -- 172.21.15.17:0/3069167008 shutdown_connections 2022-08-01T19:43:53.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.697+0000 7f3623695700 1 -- 172.21.15.17:0/3069167008 wait complete. 2022-08-01T19:43:53.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.701+0000 7f3623695700 1 Processor -- start 2022-08-01T19:43:53.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.701+0000 7f3623695700 1 -- start start 2022-08-01T19:43:53.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.701+0000 7f3623695700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f361c0b0160 0x7f361c0670a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:53.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.701+0000 7f3623695700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f361c0b2390 0x7f361c0ad810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:53.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.701+0000 7f3623695700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f361c141830 0x7f361c143060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:53.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.701+0000 7f3623695700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361c0acc00 con 0x7f361c0b2390 2022-08-01T19:43:53.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.701+0000 7f3623695700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361c0ace30 con 0x7f361c141830 2022-08-01T19:43:53.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.701+0000 7f3623695700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f361c0ad0d0 con 0x7f361c0b0160 2022-08-01T19:43:53.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.701+0000 7f3620c30700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f361c0b2390 0x7f361c0ad810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:53.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.701+0000 7f3621431700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f361c0b0160 0x7f361c0670a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:53.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.701+0000 7f3620c30700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f361c0b2390 0x7f361c0ad810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45394/0 (socket says 172.21.15.17:45394) 2022-08-01T19:43:53.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.701+0000 7f3621c32700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f361c141830 0x7f361c143060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:53.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.701+0000 7f3620c30700 1 -- 172.21.15.17:0/2465941438 learned_addr learned my addr 172.21.15.17:0/2465941438 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:53.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.705+0000 7f3621431700 1 -- 172.21.15.17:0/2465941438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f361c141830 msgr2=0x7f361c143060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:53.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.705+0000 7f3621431700 1 --2- 172.21.15.17:0/2465941438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f361c141830 0x7f361c143060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:53.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.705+0000 7f3621431700 1 -- 172.21.15.17:0/2465941438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f361c0b2390 msgr2=0x7f361c0ad810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:53.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.705+0000 7f3621431700 1 --2- 172.21.15.17:0/2465941438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f361c0b2390 0x7f361c0ad810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:53.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.705+0000 7f3621431700 1 -- 172.21.15.17:0/2465941438 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361800b1e0 con 0x7f361c0b0160 2022-08-01T19:43:53.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.705+0000 7f3621c32700 1 --2- 172.21.15.17:0/2465941438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f361c141830 0x7f361c143060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:43:53.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.705+0000 7f3620c30700 1 --2- 172.21.15.17:0/2465941438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f361c0b2390 0x7f361c0ad810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:43:53.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.705+0000 7f3621431700 1 --2- 172.21.15.17:0/2465941438 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f361c0b0160 0x7f361c0670a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f3618002f70 tx=0x7f3618015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:53.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.705+0000 7f360a7fc700 1 -- 172.21.15.17:0/2465941438 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f361800de30 con 0x7f361c0b0160 2022-08-01T19:43:53.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.705+0000 7f360a7fc700 1 -- 172.21.15.17:0/2465941438 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3618011b80 con 0x7f361c0b0160 2022-08-01T19:43:53.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.705+0000 7f360a7fc700 1 -- 172.21.15.17:0/2465941438 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f361801e460 con 0x7f361c0b0160 2022-08-01T19:43:53.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.705+0000 7f3623695700 1 -- 172.21.15.17:0/2465941438 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f361c0ad370 con 0x7f361c0b0160 2022-08-01T19:43:53.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.705+0000 7f3623695700 1 -- 172.21.15.17:0/2465941438 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f361c143550 con 0x7f361c0b0160 2022-08-01T19:43:53.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.705+0000 7f3623695700 1 -- 172.21.15.17:0/2465941438 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f361c0ad500 con 0x7f361c0b0160 2022-08-01T19:43:53.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.705+0000 7f360a7fc700 1 -- 172.21.15.17:0/2465941438 <== mon.2 v2:172.21.15.77:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f361801a040 con 0x7f361c0b0160 2022-08-01T19:43:53.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.709+0000 7f360a7fc700 1 --2- 172.21.15.17:0/2465941438 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f35fc0539e0 0x7f35fc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:53.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.709+0000 7f360a7fc700 1 -- 172.21.15.17:0/2465941438 <== mon.2 v2:172.21.15.77:3301/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f361806cd00 con 0x7f361c0b0160 2022-08-01T19:43:53.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.709+0000 7f3620c30700 1 --2- 172.21.15.17:0/2465941438 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f35fc0539e0 0x7f35fc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:53.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.709+0000 7f3620c30700 1 --2- 172.21.15.17:0/2465941438 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f35fc0539e0 0x7f35fc055ea0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7f360c00b000 tx=0x7f360c006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:53.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.717+0000 7f360a7fc700 1 -- 172.21.15.17:0/2465941438 <== mon.2 v2:172.21.15.77:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3618043c90 con 0x7f361c0b0160 2022-08-01T19:43:53.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.881+0000 7f3623695700 1 -- 172.21.15.17:0/2465941438 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f361c0574e0 con 0x7f361c0b0160 2022-08-01T19:43:53.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.885+0000 7f360a7fc700 1 -- 172.21.15.17:0/2465941438 <== mon.2 v2:172.21.15.77:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+1559 (secure 0 0 0) 0x7f3618023020 con 0x7f361c0b0160 2022-08-01T19:43:53.889 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:53.889 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":79,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7211,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/872530952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":872530952},{"type":"v1","addr":"172.21.15.77:6835","nonce":872530952}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":74},{"gid":8358,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/4261332014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4261332014},{"type":"v1","addr":"172.21.15.17:6833","nonce":4261332014}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":73},{"gid":8598,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.17:6835/4252788430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":4252788430},{"type":"v1","addr":"172.21.15.17:6835","nonce":4252788430}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":79}],"filesystems":[]} 2022-08-01T19:43:53.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.885+0000 7f3623695700 1 -- 172.21.15.17:0/2465941438 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f35fc0539e0 msgr2=0x7f35fc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:53.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.885+0000 7f3623695700 1 --2- 172.21.15.17:0/2465941438 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f35fc0539e0 0x7f35fc055ea0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7f360c00b000 tx=0x7f360c006df0).stop 2022-08-01T19:43:53.899 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.885+0000 7f3623695700 1 -- 172.21.15.17:0/2465941438 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f361c0b0160 msgr2=0x7f361c0670a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:53.899 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.885+0000 7f3623695700 1 --2- 172.21.15.17:0/2465941438 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f361c0b0160 0x7f361c0670a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f3618002f70 tx=0x7f3618015000).stop 2022-08-01T19:43:53.899 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.885+0000 7f3623695700 1 -- 172.21.15.17:0/2465941438 shutdown_connections 2022-08-01T19:43:53.899 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.885+0000 7f3623695700 1 --2- 172.21.15.17:0/2465941438 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f35fc0539e0 0x7f35fc055ea0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:53.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.885+0000 7f3623695700 1 --2- 172.21.15.17:0/2465941438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f361c141830 0x7f361c143060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:53.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.885+0000 7f3623695700 1 --2- 172.21.15.17:0/2465941438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f361c0b2390 0x7f361c0ad810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:53.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.885+0000 7f3623695700 1 --2- 172.21.15.17:0/2465941438 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f361c0b0160 0x7f361c0670a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:53.900 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.885+0000 7f3623695700 1 -- 172.21.15.17:0/2465941438 >> 172.21.15.17:0/2465941438 conn(0x7f361c05f840 msgr2=0x7f361c139eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:53.901 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.885+0000 7f3623695700 1 -- 172.21.15.17:0/2465941438 shutdown_connections 2022-08-01T19:43:53.901 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:53.885+0000 7f3623695700 1 -- 172.21.15.17:0/2465941438 wait complete. 2022-08-01T19:43:53.901 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 79 2022-08-01T19:43:53.909 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:43:54.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.017+0000 7f32daed3700 1 Processor -- start 2022-08-01T19:43:54.021 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.017+0000 7f32daed3700 1 -- start start 2022-08-01T19:43:54.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.017+0000 7f32daed3700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f32d40b2710 0x7f32d40abca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:54.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.021+0000 7f32daed3700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f32d40b13f0 0x7f32d40ac190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:54.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.021+0000 7f32daed3700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d40adf10 con 0x7f32d40b1920 2022-08-01T19:43:54.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.021+0000 7f32daed3700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d40ae1b0 con 0x7f32d40b13f0 2022-08-01T19:43:54.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.021+0000 7f32daed3700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d40a9520 con 0x7f32d40b2710 2022-08-01T19:43:54.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.021+0000 7f32d9ed1700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f32d40b2710 0x7f32d40abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:54.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.021+0000 7f32da6d2700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7f32d40b1920 0x7f32d40afd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:54398/0 (socket says 172.21.15.17:54398) 2022-08-01T19:43:54.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.021+0000 7f32da6d2700 1 -- 172.21.15.17:0/1692835215 learned_addr learned my addr 172.21.15.17:0/1692835215 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:54.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.021+0000 7f32d96d0700 1 --2- 172.21.15.17:0/1692835215 >> v2:172.21.15.77:3300/0 conn(0x7f32d40b13f0 0x7f32d40ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:54.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.021+0000 7f32d8ecf700 1 -- 172.21.15.17:0/1692835215 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2283286786 0 0) 0x7f32d40adf10 con 0x7f32d40b1920 2022-08-01T19:43:54.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.021+0000 7f32d96d0700 1 -- 172.21.15.17:0/1692835215 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f32d40b2710 msgr2=0x7f32d40abca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:54.040 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.021+0000 7f32d96d0700 1 --2- 172.21.15.17:0/1692835215 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f32d40b2710 0x7f32d40abca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:54.040 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.021+0000 7f32d96d0700 1 -- 172.21.15.17:0/1692835215 >> v1:172.21.15.77:6789/0 conn(0x7f32d40b1920 legacy=0x7f32d40afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:54.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.021+0000 7f32d96d0700 1 -- 172.21.15.17:0/1692835215 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d40a99c0 con 0x7f32d40b13f0 2022-08-01T19:43:54.041 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.021+0000 7f32d9ed1700 1 --2- 172.21.15.17:0/1692835215 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f32d40b2710 0x7f32d40abca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:43:54.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.021+0000 7f32d96d0700 1 --2- 172.21.15.17:0/1692835215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32d40b13f0 0x7f32d40ac190 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f32d4067bc0 tx=0x7f32c4006df0).ready entity=mon.1 client_cookie=9724a12e6f13c985 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:54.042 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.021+0000 7f32d8ecf700 1 -- 172.21.15.17:0/1692835215 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32c400d570 con 0x7f32d40b13f0 2022-08-01T19:43:54.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.021+0000 7f32d8ecf700 1 -- 172.21.15.17:0/1692835215 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c4000e90 con 0x7f32d40b13f0 2022-08-01T19:43:54.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.021+0000 7f32d8ecf700 1 -- 172.21.15.17:0/1692835215 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32c400d8c0 con 0x7f32d40b13f0 2022-08-01T19:43:54.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.021+0000 7f32daed3700 1 -- 172.21.15.17:0/1692835215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32d40b13f0 msgr2=0x7f32d40ac190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:54.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.021+0000 7f32daed3700 1 --2- 172.21.15.17:0/1692835215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32d40b13f0 0x7f32d40ac190 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f32d4067bc0 tx=0x7f32c4006df0).stop 2022-08-01T19:43:54.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.021+0000 7f32daed3700 1 -- 172.21.15.17:0/1692835215 shutdown_connections 2022-08-01T19:43:54.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.021+0000 7f32daed3700 1 --2- 172.21.15.17:0/1692835215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32d40b13f0 0x7f32d40ac190 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:54.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.021+0000 7f32daed3700 1 --2- 172.21.15.17:0/1692835215 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f32d40b2710 0x7f32d40abca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:54.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.021+0000 7f32daed3700 1 -- 172.21.15.17:0/1692835215 >> 172.21.15.17:0/1692835215 conn(0x7f32d4060310 msgr2=0x7f32d40b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:54.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.021+0000 7f32daed3700 1 -- 172.21.15.17:0/1692835215 shutdown_connections 2022-08-01T19:43:54.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.021+0000 7f32daed3700 1 -- 172.21.15.17:0/1692835215 wait complete. 2022-08-01T19:43:54.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.025+0000 7f32daed3700 1 Processor -- start 2022-08-01T19:43:54.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.025+0000 7f32daed3700 1 -- start start 2022-08-01T19:43:54.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.025+0000 7f32daed3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32d40b13f0 0x7f32d40abf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:54.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.025+0000 7f32daed3700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f32d40b1920 0x7f32d40b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:54.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.025+0000 7f32daed3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32d40b2710 0x7f32d414ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:54.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.025+0000 7f32daed3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d4154310 con 0x7f32d40b2710 2022-08-01T19:43:54.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.025+0000 7f32daed3700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d40ac4d0 con 0x7f32d40b13f0 2022-08-01T19:43:54.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.025+0000 7f32daed3700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d40ac770 con 0x7f32d40b1920 2022-08-01T19:43:54.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.025+0000 7f32da6d2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32d40b2710 0x7f32d414ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:54.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.025+0000 7f32da6d2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32d40b2710 0x7f32d414ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45408/0 (socket says 172.21.15.17:45408) 2022-08-01T19:43:54.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.025+0000 7f32da6d2700 1 -- 172.21.15.17:0/3836173274 learned_addr learned my addr 172.21.15.17:0/3836173274 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:54.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.025+0000 7f32d9ed1700 1 --2- 172.21.15.17:0/3836173274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32d40b13f0 0x7f32d40abf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:54.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.025+0000 7f32d96d0700 1 --2- 172.21.15.17:0/3836173274 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f32d40b1920 0x7f32d40b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:54.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.025+0000 7f32da6d2700 1 -- 172.21.15.17:0/3836173274 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f32d40b1920 msgr2=0x7f32d40b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:54.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.025+0000 7f32da6d2700 1 --2- 172.21.15.17:0/3836173274 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f32d40b1920 0x7f32d40b45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:54.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.025+0000 7f32da6d2700 1 -- 172.21.15.17:0/3836173274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32d40b13f0 msgr2=0x7f32d40abf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:54.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.025+0000 7f32da6d2700 1 --2- 172.21.15.17:0/3836173274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32d40b13f0 0x7f32d40abf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:54.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.025+0000 7f32da6d2700 1 -- 172.21.15.17:0/3836173274 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d000b180 con 0x7f32d40b2710 2022-08-01T19:43:54.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.025+0000 7f32da6d2700 1 --2- 172.21.15.17:0/3836173274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32d40b2710 0x7f32d414ff40 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 rx=0x7f32cc001250 tx=0x7f32cc007d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:54.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.025+0000 7f32c2ffd700 1 -- 172.21.15.17:0/3836173274 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32cc005ba0 con 0x7f32d40b2710 2022-08-01T19:43:54.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.025+0000 7f32c2ffd700 1 -- 172.21.15.17:0/3836173274 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32cc005d00 con 0x7f32d40b2710 2022-08-01T19:43:54.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.025+0000 7f32daed3700 1 -- 172.21.15.17:0/3836173274 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c400b180 con 0x7f32d40b2710 2022-08-01T19:43:54.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.025+0000 7f32c2ffd700 1 -- 172.21.15.17:0/3836173274 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32cc004a90 con 0x7f32d40b2710 2022-08-01T19:43:54.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.025+0000 7f32daed3700 1 -- 172.21.15.17:0/3836173274 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d40abbc0 con 0x7f32d40b2710 2022-08-01T19:43:54.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.029+0000 7f32daed3700 1 -- 172.21.15.17:0/3836173274 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32d40574e0 con 0x7f32d40b2710 2022-08-01T19:43:54.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.029+0000 7f32c2ffd700 1 -- 172.21.15.17:0/3836173274 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f32cc004bf0 con 0x7f32d40b2710 2022-08-01T19:43:54.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.029+0000 7f32c2ffd700 1 --2- 172.21.15.17:0/3836173274 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f32b40539e0 0x7f32b4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:54.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.029+0000 7f32c2ffd700 1 -- 172.21.15.17:0/3836173274 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f32cc06b8f0 con 0x7f32d40b2710 2022-08-01T19:43:54.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.029+0000 7f32d9ed1700 1 --2- 172.21.15.17:0/3836173274 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f32b40539e0 0x7f32b4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:54.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.029+0000 7f32d9ed1700 1 --2- 172.21.15.17:0/3836173274 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f32b40539e0 0x7f32b4055ea0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7f32d000b000 tx=0x7f32d0006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:54.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.033+0000 7f32c2ffd700 1 -- 172.21.15.17:0/3836173274 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f32cc042990 con 0x7f32d40b2710 2022-08-01T19:43:54.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.197+0000 7f32daed3700 1 -- 172.21.15.17:0/3836173274 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32d4069a70 con 0x7f32d40b2710 2022-08-01T19:43:54.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.197+0000 7f32c2ffd700 1 -- 172.21.15.17:0/3836173274 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+1559 (secure 0 0 0) 0x7f32cc045fb0 con 0x7f32d40b2710 2022-08-01T19:43:54.200 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:54.201 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":79,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7211,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/872530952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":872530952},{"type":"v1","addr":"172.21.15.77:6835","nonce":872530952}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":74},{"gid":8358,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/4261332014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4261332014},{"type":"v1","addr":"172.21.15.17:6833","nonce":4261332014}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":73},{"gid":8598,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.17:6835/4252788430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":4252788430},{"type":"v1","addr":"172.21.15.17:6835","nonce":4252788430}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":79}],"filesystems":[]} 2022-08-01T19:43:54.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.201+0000 7f32daed3700 1 -- 172.21.15.17:0/3836173274 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f32b40539e0 msgr2=0x7f32b4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:54.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.201+0000 7f32daed3700 1 --2- 172.21.15.17:0/3836173274 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f32b40539e0 0x7f32b4055ea0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7f32d000b000 tx=0x7f32d0006df0).stop 2022-08-01T19:43:54.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.201+0000 7f32daed3700 1 -- 172.21.15.17:0/3836173274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32d40b2710 msgr2=0x7f32d414ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:54.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.201+0000 7f32daed3700 1 --2- 172.21.15.17:0/3836173274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32d40b2710 0x7f32d414ff40 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 rx=0x7f32cc001250 tx=0x7f32cc007d80).stop 2022-08-01T19:43:54.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.201+0000 7f32daed3700 1 -- 172.21.15.17:0/3836173274 shutdown_connections 2022-08-01T19:43:54.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.201+0000 7f32daed3700 1 --2- 172.21.15.17:0/3836173274 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f32b40539e0 0x7f32b4055ea0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:54.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.201+0000 7f32daed3700 1 --2- 172.21.15.17:0/3836173274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32d40b2710 0x7f32d414ff40 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:54.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.201+0000 7f32daed3700 1 --2- 172.21.15.17:0/3836173274 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f32d40b1920 0x7f32d40b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:54.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.201+0000 7f32daed3700 1 --2- 172.21.15.17:0/3836173274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32d40b13f0 0x7f32d40abf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:54.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.201+0000 7f32daed3700 1 -- 172.21.15.17:0/3836173274 >> 172.21.15.17:0/3836173274 conn(0x7f32d4060310 msgr2=0x7f32d4143680 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:54.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.201+0000 7f32daed3700 1 -- 172.21.15.17:0/3836173274 shutdown_connections 2022-08-01T19:43:54.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.201+0000 7f32daed3700 1 -- 172.21.15.17:0/3836173274 wait complete. 2022-08-01T19:43:54.212 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 79 2022-08-01T19:43:54.215 INFO:tasks.ceph.mds.a:Restarting daemon 2022-08-01T19:43:54.215 INFO:tasks.ceph.mds.a:Stopping old one... 2022-08-01T19:43:54.216 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-08-01T19:43:54.216 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:43:54.216 INFO:tasks.ceph.mds.c:Restarting daemon 2022-08-01T19:43:54.217 INFO:tasks.ceph.mds.c:Stopping old one... 2022-08-01T19:43:54.217 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-08-01T19:43:54.217 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:43:54.218 INFO:tasks.ceph.mds.b:Restarting daemon 2022-08-01T19:43:54.218 INFO:tasks.ceph.mds.b:Stopping old one... 2022-08-01T19:43:54.219 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-08-01T19:43:54.219 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T19:43:54.230 INFO:tasks.ceph.mds.c:Stopped 2022-08-01T19:43:54.239 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-08-01T19:43:54.240 INFO:tasks.ceph.mds.a:Stopped 2022-08-01T19:43:54.240 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-08-01T19:43:54.242 INFO:tasks.ceph.mds.b:Stopped 2022-08-01T19:43:54.243 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-08-01T19:43:54.254 INFO:tasks.ceph.mds.b:Started 2022-08-01T19:43:54.254 DEBUG:teuthology.parallel:result is None 2022-08-01T19:43:54.286 INFO:tasks.ceph.mds.c:Started 2022-08-01T19:43:54.287 DEBUG:teuthology.parallel:result is None 2022-08-01T19:43:54.288 INFO:tasks.ceph.mds.a:Started 2022-08-01T19:43:54.289 DEBUG:teuthology.parallel:result is None 2022-08-01T19:43:54.289 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-08-01T19:43:54.330 INFO:tasks.ceph.mds.b.smithi077.stdout:starting mds.b at 2022-08-01T19:43:54.351 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2022-08-01T19:43:54.356 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2022-08-01T19:43:54.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.389+0000 7fdc831cf700 1 Processor -- start 2022-08-01T19:43:54.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.389+0000 7fdc831cf700 1 -- start start 2022-08-01T19:43:54.416 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.389+0000 7fdc831cf700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdc7c1437f0 0x7fdc7c143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:54.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.389+0000 7fdc831cf700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fdc7c0aad90 0x7fdc7c13d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:54.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.389+0000 7fdc831cf700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc7c0a8cf0 con 0x7fdc7c05f840 2022-08-01T19:43:54.417 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.389+0000 7fdc831cf700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc7c0a8f50 con 0x7fdc7c0aad90 2022-08-01T19:43:54.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.389+0000 7fdc831cf700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c1441d0 con 0x7fdc7c1437f0 2022-08-01T19:43:54.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.389+0000 7fdc8176c700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fdc7c05f840 0x7fdc7c0a9e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37634/0 (socket says 172.21.15.17:37634) 2022-08-01T19:43:54.418 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.389+0000 7fdc8176c700 1 -- 172.21.15.17:0/4169150701 learned_addr learned my addr 172.21.15.17:0/4169150701 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:54.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.389+0000 7fdc80f6b700 1 --2- 172.21.15.17:0/4169150701 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdc7c1437f0 0x7fdc7c143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:54.419 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.389+0000 7fdc7bfff700 1 --2- 172.21.15.17:0/4169150701 >> v2:172.21.15.77:3300/0 conn(0x7fdc7c0aad90 0x7fdc7c13d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:54.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.389+0000 7fdc7b7fe700 1 -- 172.21.15.17:0/4169150701 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3163350968 0 0) 0x7fdc7c0a8cf0 con 0x7fdc7c05f840 2022-08-01T19:43:54.420 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.389+0000 7fdc7b7fe700 1 -- 172.21.15.17:0/4169150701 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc60003260 con 0x7fdc7c05f840 2022-08-01T19:43:54.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.389+0000 7fdc7b7fe700 1 -- 172.21.15.17:0/4169150701 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2153348123 0 0) 0x7fdc60003260 con 0x7fdc7c05f840 2022-08-01T19:43:54.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.389+0000 7fdc80f6b700 1 -- 172.21.15.17:0/4169150701 >> v2:172.21.15.77:3300/0 conn(0x7fdc7c0aad90 msgr2=0x7fdc7c13d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:54.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.389+0000 7fdc80f6b700 1 --2- 172.21.15.17:0/4169150701 >> v2:172.21.15.77:3300/0 conn(0x7fdc7c0aad90 0x7fdc7c13d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:54.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.389+0000 7fdc80f6b700 1 -- 172.21.15.17:0/4169150701 >> v1:172.21.15.17:6789/0 conn(0x7fdc7c05f840 legacy=0x7fdc7c0a9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:54.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.389+0000 7fdc80f6b700 1 -- 172.21.15.17:0/4169150701 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc7c05fd50 con 0x7fdc7c1437f0 2022-08-01T19:43:54.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.393+0000 7fdc80f6b700 1 --2- 172.21.15.17:0/4169150701 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdc7c1437f0 0x7fdc7c143bd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7fdc74001ba0 tx=0x7fdc70006df0).ready entity=mon.2 client_cookie=2fd8f5a45a6ae36e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:54.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.393+0000 7fdc7b7fe700 1 -- 172.21.15.17:0/4169150701 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc70014070 con 0x7fdc7c1437f0 2022-08-01T19:43:54.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.393+0000 7fdc7b7fe700 1 -- 172.21.15.17:0/4169150701 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc70000e90 con 0x7fdc7c1437f0 2022-08-01T19:43:54.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.393+0000 7fdc7b7fe700 1 -- 172.21.15.17:0/4169150701 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc74001a00 con 0x7fdc7c1437f0 2022-08-01T19:43:54.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.393+0000 7fdc831cf700 1 -- 172.21.15.17:0/4169150701 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdc7c1437f0 msgr2=0x7fdc7c143bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:54.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.393+0000 7fdc831cf700 1 --2- 172.21.15.17:0/4169150701 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdc7c1437f0 0x7fdc7c143bd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7fdc74001ba0 tx=0x7fdc70006df0).stop 2022-08-01T19:43:54.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.393+0000 7fdc831cf700 1 -- 172.21.15.17:0/4169150701 shutdown_connections 2022-08-01T19:43:54.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.393+0000 7fdc831cf700 1 --2- 172.21.15.17:0/4169150701 >> v2:172.21.15.77:3300/0 conn(0x7fdc7c0aad90 0x7fdc7c13d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:54.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.393+0000 7fdc831cf700 1 --2- 172.21.15.17:0/4169150701 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdc7c1437f0 0x7fdc7c143bd0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:54.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.393+0000 7fdc831cf700 1 -- 172.21.15.17:0/4169150701 >> 172.21.15.17:0/4169150701 conn(0x7fdc7c0b5570 msgr2=0x7fdc7c0b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:54.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.393+0000 7fdc831cf700 1 -- 172.21.15.17:0/4169150701 shutdown_connections 2022-08-01T19:43:54.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.393+0000 7fdc831cf700 1 -- 172.21.15.17:0/4169150701 wait complete. 2022-08-01T19:43:54.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.393+0000 7fdc831cf700 1 Processor -- start 2022-08-01T19:43:54.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.393+0000 7fdc831cf700 1 -- start start 2022-08-01T19:43:54.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.397+0000 7fdc831cf700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdc7c05f840 0x7fdc7c0accd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:54.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.397+0000 7fdc831cf700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc7c0aad90 0x7fdc7c13e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:54.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.397+0000 7fdc831cf700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc7c1437f0 0x7fdc7c140140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:54.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.397+0000 7fdc831cf700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c0ad810 con 0x7fdc7c1437f0 2022-08-01T19:43:54.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.397+0000 7fdc831cf700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c0ad310 con 0x7fdc7c0aad90 2022-08-01T19:43:54.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.397+0000 7fdc831cf700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c0ad570 con 0x7fdc7c05f840 2022-08-01T19:43:54.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.397+0000 7fdc8176c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc7c1437f0 0x7fdc7c140140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:54.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.397+0000 7fdc8176c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc7c1437f0 0x7fdc7c140140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45446/0 (socket says 172.21.15.17:45446) 2022-08-01T19:43:54.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.397+0000 7fdc8176c700 1 -- 172.21.15.17:0/3621371158 learned_addr learned my addr 172.21.15.17:0/3621371158 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:54.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.397+0000 7fdc80f6b700 1 --2- 172.21.15.17:0/3621371158 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdc7c05f840 0x7fdc7c0accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:54.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.397+0000 7fdc7bfff700 1 --2- 172.21.15.17:0/3621371158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc7c0aad90 0x7fdc7c13e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:54.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.397+0000 7fdc8176c700 1 -- 172.21.15.17:0/3621371158 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdc7c05f840 msgr2=0x7fdc7c0accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:54.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.397+0000 7fdc8176c700 1 --2- 172.21.15.17:0/3621371158 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdc7c05f840 0x7fdc7c0accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:54.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.397+0000 7fdc8176c700 1 -- 172.21.15.17:0/3621371158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc7c0aad90 msgr2=0x7fdc7c13e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:54.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.397+0000 7fdc8176c700 1 --2- 172.21.15.17:0/3621371158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc7c0aad90 0x7fdc7c13e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:54.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.397+0000 7fdc8176c700 1 -- 172.21.15.17:0/3621371158 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc7000b180 con 0x7fdc7c1437f0 2022-08-01T19:43:54.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.397+0000 7fdc80f6b700 1 --2- 172.21.15.17:0/3621371158 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdc7c05f840 0x7fdc7c0accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:43:54.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.397+0000 7fdc7bfff700 1 --2- 172.21.15.17:0/3621371158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc7c0aad90 0x7fdc7c13e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:43:54.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.397+0000 7fdc8176c700 1 --2- 172.21.15.17:0/3621371158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc7c1437f0 0x7fdc7c140140 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 rx=0x7fdc74001c60 tx=0x7fdc740021c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:54.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.397+0000 7fdc79ffb700 1 -- 172.21.15.17:0/3621371158 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc74004ab0 con 0x7fdc7c1437f0 2022-08-01T19:43:54.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.397+0000 7fdc79ffb700 1 -- 172.21.15.17:0/3621371158 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc74004c10 con 0x7fdc7c1437f0 2022-08-01T19:43:54.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.397+0000 7fdc831cf700 1 -- 172.21.15.17:0/3621371158 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc7c13fca0 con 0x7fdc7c1437f0 2022-08-01T19:43:54.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.397+0000 7fdc79ffb700 1 -- 172.21.15.17:0/3621371158 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdc74007170 con 0x7fdc7c1437f0 2022-08-01T19:43:54.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.397+0000 7fdc831cf700 1 -- 172.21.15.17:0/3621371158 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc7c0b4af0 con 0x7fdc7c1437f0 2022-08-01T19:43:54.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.401+0000 7fdc831cf700 1 -- 172.21.15.17:0/3621371158 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc7c13fe30 con 0x7fdc7c1437f0 2022-08-01T19:43:54.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.401+0000 7fdc79ffb700 1 -- 172.21.15.17:0/3621371158 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdc74010070 con 0x7fdc7c1437f0 2022-08-01T19:43:54.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.401+0000 7fdc79ffb700 1 --2- 172.21.15.17:0/3621371158 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdc5c053980 0x7fdc5c055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:54.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.401+0000 7fdc79ffb700 1 -- 172.21.15.17:0/3621371158 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fdc7406bb20 con 0x7fdc7c1437f0 2022-08-01T19:43:54.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.401+0000 7fdc80f6b700 1 --2- 172.21.15.17:0/3621371158 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdc5c053980 0x7fdc5c055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:54.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.401+0000 7fdc80f6b700 1 --2- 172.21.15.17:0/3621371158 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdc5c053980 0x7fdc5c055e40 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7fdc7001b000 tx=0x7fdc70016000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:54.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.409+0000 7fdc79ffb700 1 -- 172.21.15.17:0/3621371158 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdc74042a30 con 0x7fdc7c1437f0 2022-08-01T19:43:54.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:54.545+0000 7fdc831cf700 1 -- 172.21.15.17:0/3621371158 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fdc7c13d8b0 con 0x7fdc7c1437f0 2022-08-01T19:43:55.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.389+0000 7fdc79ffb700 1 -- 172.21.15.17:0/3621371158 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v154) v1 ==== 97+0+0 (secure 0 0 0) 0x7fdc74046050 con 0x7fdc7c1437f0 2022-08-01T19:43:55.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.393+0000 7fdc831cf700 1 -- 172.21.15.17:0/3621371158 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdc5c053980 msgr2=0x7fdc5c055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:55.402 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.393+0000 7fdc831cf700 1 --2- 172.21.15.17:0/3621371158 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdc5c053980 0x7fdc5c055e40 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7fdc7001b000 tx=0x7fdc70016000).stop 2022-08-01T19:43:55.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.393+0000 7fdc831cf700 1 -- 172.21.15.17:0/3621371158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc7c1437f0 msgr2=0x7fdc7c140140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:55.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.393+0000 7fdc831cf700 1 --2- 172.21.15.17:0/3621371158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc7c1437f0 0x7fdc7c140140 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 rx=0x7fdc74001c60 tx=0x7fdc740021c0).stop 2022-08-01T19:43:55.403 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.393+0000 7fdc831cf700 1 -- 172.21.15.17:0/3621371158 shutdown_connections 2022-08-01T19:43:55.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.393+0000 7fdc831cf700 1 --2- 172.21.15.17:0/3621371158 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdc5c053980 0x7fdc5c055e40 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:55.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.393+0000 7fdc831cf700 1 --2- 172.21.15.17:0/3621371158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc7c1437f0 0x7fdc7c140140 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:55.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.393+0000 7fdc831cf700 1 --2- 172.21.15.17:0/3621371158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc7c0aad90 0x7fdc7c13e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:55.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.393+0000 7fdc831cf700 1 --2- 172.21.15.17:0/3621371158 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdc7c05f840 0x7fdc7c0accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:55.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.393+0000 7fdc831cf700 1 -- 172.21.15.17:0/3621371158 >> 172.21.15.17:0/3621371158 conn(0x7fdc7c0b5570 msgr2=0x7fdc7c142cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:55.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.393+0000 7fdc831cf700 1 -- 172.21.15.17:0/3621371158 shutdown_connections 2022-08-01T19:43:55.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.393+0000 7fdc831cf700 1 -- 172.21.15.17:0/3621371158 wait complete. 2022-08-01T19:43:55.407 INFO:teuthology.orchestra.run.smithi017.stderr: removed all blacklist entries 2022-08-01T19:43:55.412 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2022-08-01T19:43:55.413 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-08-01T19:43:55.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e19948700 1 Processor -- start 2022-08-01T19:43:55.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e19948700 1 -- start start 2022-08-01T19:43:55.550 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e19948700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f1e140b5d80 0x7f1e140b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:55.550 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e19948700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e14144240 con 0x7f1e140b4550 2022-08-01T19:43:55.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e19948700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e140b6fe0 con 0x7f1e140b6710 2022-08-01T19:43:55.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e19948700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e14143d00 con 0x7f1e140b5d80 2022-08-01T19:43:55.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e127fc700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1e140b4550 0x7f1e1412bb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37652/0 (socket says 172.21.15.17:37652) 2022-08-01T19:43:55.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e127fc700 1 -- 172.21.15.17:0/3758673815 learned_addr learned my addr 172.21.15.17:0/3758673815 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:55.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e137fe700 1 --2- 172.21.15.17:0/3758673815 >> v2:172.21.15.77:3300/0 conn(0x7f1e140b5d80 0x7f1e140b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:55.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e11ffb700 1 -- 172.21.15.17:0/3758673815 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 508084983 0 0) 0x7f1e14144240 con 0x7f1e140b4550 2022-08-01T19:43:55.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e11ffb700 1 -- 172.21.15.17:0/3758673815 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1df4003260 con 0x7f1e140b4550 2022-08-01T19:43:55.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e11ffb700 1 -- 172.21.15.17:0/3758673815 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2783191089 0 0) 0x7f1df4003260 con 0x7f1e140b4550 2022-08-01T19:43:55.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e11ffb700 1 -- 172.21.15.17:0/3758673815 >> v2:172.21.15.77:3300/0 conn(0x7f1e140b5d80 msgr2=0x7f1e140b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:55.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e11ffb700 1 --2- 172.21.15.17:0/3758673815 >> v2:172.21.15.77:3300/0 conn(0x7f1e140b5d80 0x7f1e140b6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:55.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e11ffb700 1 -- 172.21.15.17:0/3758673815 >> v1:172.21.15.77:6789/0 conn(0x7f1e140b6710 legacy=0x7f1e14061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:55.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e11ffb700 1 -- 172.21.15.17:0/3758673815 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e14143fa0 con 0x7f1e140b4550 2022-08-01T19:43:55.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e11ffb700 1 -- 172.21.15.17:0/3758673815 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7f1dfc001660 con 0x7f1e140b4550 2022-08-01T19:43:55.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e11ffb700 1 -- 172.21.15.17:0/3758673815 >> v1:172.21.15.17:6789/0 conn(0x7f1e140b4550 legacy=0x7f1e1412bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:55.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e11ffb700 1 --2- 172.21.15.17:0/3758673815 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1df4003750 0x7f1df4003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:55.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e11ffb700 1 --2- 172.21.15.17:0/3758673815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1df4004500 0x7f1df4004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:55.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e11ffb700 1 --2- 172.21.15.17:0/3758673815 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1df4004ec0 0x7f1df400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:55.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e11ffb700 1 -- 172.21.15.17:0/3758673815 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e14143d00 con 0x7f1df4003750 2022-08-01T19:43:55.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e11ffb700 1 -- 172.21.15.17:0/3758673815 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df400bda0 con 0x7f1df4004500 2022-08-01T19:43:55.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e11ffb700 1 -- 172.21.15.17:0/3758673815 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df400bf60 con 0x7f1df4004ec0 2022-08-01T19:43:55.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e12ffd700 1 --2- 172.21.15.17:0/3758673815 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1df4003750 0x7f1df4003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:55.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e127fc700 1 --2- 172.21.15.17:0/3758673815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1df4004500 0x7f1df4004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:55.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e12ffd700 1 -- 172.21.15.17:0/3758673815 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1df4004ec0 msgr2=0x7f1df400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:55.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e12ffd700 1 --2- 172.21.15.17:0/3758673815 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1df4004ec0 0x7f1df400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2022-08-01T19:43:55.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e12ffd700 1 -- 172.21.15.17:0/3758673815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1df4004500 msgr2=0x7f1df4004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:55.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e12ffd700 1 --2- 172.21.15.17:0/3758673815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1df4004500 0x7f1df4004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:55.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e12ffd700 1 -- 172.21.15.17:0/3758673815 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df4003260 con 0x7f1df4003750 2022-08-01T19:43:55.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e127fc700 1 --2- 172.21.15.17:0/3758673815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1df4004500 0x7f1df4004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:43:55.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e137fe700 1 -- 172.21.15.17:0/3758673815 reap_dead start 2022-08-01T19:43:55.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e12ffd700 1 --2- 172.21.15.17:0/3758673815 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1df4003750 0x7f1df4003b70 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 rx=0x7f1e14060a40 tx=0x7f1e0c007d80).ready entity=mon.0 client_cookie=615238a8b4c44647 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:55.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e11ffb700 1 -- 172.21.15.17:0/3758673815 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e0c001620 con 0x7f1df4003750 2022-08-01T19:43:55.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e11ffb700 1 -- 172.21.15.17:0/3758673815 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0c006890 con 0x7f1df4003750 2022-08-01T19:43:55.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.525+0000 7f1e11ffb700 1 -- 172.21.15.17:0/3758673815 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e0c006bf0 con 0x7f1df4003750 2022-08-01T19:43:55.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.529+0000 7f1e19948700 1 -- 172.21.15.17:0/3758673815 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1df4003750 msgr2=0x7f1df4003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:55.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.529+0000 7f1e19948700 1 --2- 172.21.15.17:0/3758673815 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1df4003750 0x7f1df4003b70 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 rx=0x7f1e14060a40 tx=0x7f1e0c007d80).stop 2022-08-01T19:43:55.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.529+0000 7f1e19948700 1 -- 172.21.15.17:0/3758673815 shutdown_connections 2022-08-01T19:43:55.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.529+0000 7f1e19948700 1 --2- 172.21.15.17:0/3758673815 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1df4003750 0x7f1df4003b70 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:55.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.529+0000 7f1e19948700 1 -- 172.21.15.17:0/3758673815 >> 172.21.15.17:0/3758673815 conn(0x7f1e1405c620 msgr2=0x7f1e1405ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:55.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.529+0000 7f1e19948700 1 -- 172.21.15.17:0/3758673815 shutdown_connections 2022-08-01T19:43:55.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.529+0000 7f1e19948700 1 -- 172.21.15.17:0/3758673815 wait complete. 2022-08-01T19:43:55.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.529+0000 7f1e19948700 1 Processor -- start 2022-08-01T19:43:55.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.529+0000 7f1e19948700 1 -- start start 2022-08-01T19:43:55.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.529+0000 7f1e19948700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1df4003750 0x7f1e140b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:55.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.529+0000 7f1e19948700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1e14148040 0x7f1e14140460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:55.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.529+0000 7f1e19948700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e141495d0 0x7f1e14140950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:55.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.529+0000 7f1e19948700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e14155a20 con 0x7f1df4003750 2022-08-01T19:43:55.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.529+0000 7f1e19948700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e14155cc0 con 0x7f1e141495d0 2022-08-01T19:43:55.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.529+0000 7f1e19948700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e14155f60 con 0x7f1e14148040 2022-08-01T19:43:55.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.529+0000 7f1e12ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1df4003750 0x7f1e140b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:55.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.529+0000 7f1e12ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1df4003750 0x7f1e140b30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45468/0 (socket says 172.21.15.17:45468) 2022-08-01T19:43:55.579 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.529+0000 7f1e12ffd700 1 -- 172.21.15.17:0/3645248640 learned_addr learned my addr 172.21.15.17:0/3645248640 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:55.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.529+0000 7f1e137fe700 1 --2- 172.21.15.17:0/3645248640 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e141495d0 0x7f1e14140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:55.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.529+0000 7f1e127fc700 1 --2- 172.21.15.17:0/3645248640 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1e14148040 0x7f1e14140460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:55.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.529+0000 7f1e12ffd700 1 -- 172.21.15.17:0/3645248640 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1e14148040 msgr2=0x7f1e14140460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:55.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.529+0000 7f1e12ffd700 1 --2- 172.21.15.17:0/3645248640 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1e14148040 0x7f1e14140460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:55.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.529+0000 7f1e12ffd700 1 -- 172.21.15.17:0/3645248640 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e141495d0 msgr2=0x7f1e14140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:55.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.529+0000 7f1e12ffd700 1 --2- 172.21.15.17:0/3645248640 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e141495d0 0x7f1e14140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:55.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.529+0000 7f1e12ffd700 1 -- 172.21.15.17:0/3645248640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e14156200 con 0x7f1df4003750 2022-08-01T19:43:55.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.529+0000 7f1e137fe700 1 --2- 172.21.15.17:0/3645248640 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e141495d0 0x7f1e14140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:43:55.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.533+0000 7f1e12ffd700 1 --2- 172.21.15.17:0/3645248640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1df4003750 0x7f1e140b30b0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 rx=0x7f1e14060a40 tx=0x7f1e0c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:55.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.533+0000 7f1e0bfff700 1 -- 172.21.15.17:0/3645248640 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e0c0138e0 con 0x7f1df4003750 2022-08-01T19:43:55.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.533+0000 7f1e0bfff700 1 -- 172.21.15.17:0/3645248640 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0c013a40 con 0x7f1df4003750 2022-08-01T19:43:55.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.533+0000 7f1e19948700 1 -- 172.21.15.17:0/3645248640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e141564a0 con 0x7f1df4003750 2022-08-01T19:43:55.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.533+0000 7f1e0bfff700 1 -- 172.21.15.17:0/3645248640 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e0c013da0 con 0x7f1df4003750 2022-08-01T19:43:55.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.533+0000 7f1e19948700 1 -- 172.21.15.17:0/3645248640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e141524d0 con 0x7f1df4003750 2022-08-01T19:43:55.589 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.533+0000 7f1e19948700 1 -- 172.21.15.17:0/3645248640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e14061780 con 0x7f1df4003750 2022-08-01T19:43:55.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.533+0000 7f1e0bfff700 1 -- 172.21.15.17:0/3645248640 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1e0c01b030 con 0x7f1df4003750 2022-08-01T19:43:55.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.533+0000 7f1e0bfff700 1 --2- 172.21.15.17:0/3645248640 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1df8053aa0 0x7f1df8055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:55.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.533+0000 7f1e0bfff700 1 -- 172.21.15.17:0/3645248640 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f1e0c06cd70 con 0x7f1df4003750 2022-08-01T19:43:55.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.533+0000 7f1e127fc700 1 --2- 172.21.15.17:0/3645248640 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1df8053aa0 0x7f1df8055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:55.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.533+0000 7f1e127fc700 1 --2- 172.21.15.17:0/3645248640 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1df8053aa0 0x7f1df8055f60 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7f1dfc00d000 tx=0x7f1dfc009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:55.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.541+0000 7f1e0bfff700 1 -- 172.21.15.17:0/3645248640 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1e0c043e70 con 0x7f1df4003750 2022-08-01T19:43:55.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.709+0000 7f1e19948700 1 -- 172.21.15.17:0/3645248640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f1e140574e0 con 0x7f1df4003750 2022-08-01T19:43:55.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.709+0000 7f1e0bfff700 1 -- 172.21.15.17:0/3645248640 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v37) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f1e0c024030 con 0x7f1df4003750 2022-08-01T19:43:55.712 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:55.712 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:43:55.713 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_dump": [ 2022-08-01T19:43:55.713 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:55.713 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.a", 2022-08-01T19:43:55.713 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhijotxORAAVc04rsrlFj0NShsyc6Qbhg==", 2022-08-01T19:43:55.714 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:43:55.714 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:43:55.714 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:43:55.714 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:43:55.715 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:43:55.715 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:55.715 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:55.718 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:55.718 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.b", 2022-08-01T19:43:55.719 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhiud2xBxAAcn6/O5ysdIp/oIbF5Y1dAw==", 2022-08-01T19:43:55.719 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:43:55.719 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:43:55.719 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:43:55.720 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:43:55.720 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:43:55.720 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:55.720 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:55.721 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:55.721 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.c", 2022-08-01T19:43:55.721 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+KuhiMzMbBBAAfdT5mtpYwboNL+7f2B09KQ==", 2022-08-01T19:43:55.724 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:43:55.724 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:43:55.724 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:43:55.725 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:43:55.725 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:43:55.726 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:55.726 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:55.727 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:55.727 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.0", 2022-08-01T19:43:55.728 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhia27/NRAAZXBp+nYelJVJ6Mge9r+3Vg==", 2022-08-01T19:43:55.728 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:43:55.728 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:43:55.729 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:43:55.729 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:43:55.730 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:55.730 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:55.731 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:55.731 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.1", 2022-08-01T19:43:55.731 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCBKuhiOggRGRAAvgA0MxwBJwmpXV6S9DTdjA==", 2022-08-01T19:43:55.732 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:43:55.732 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:43:55.733 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:43:55.735 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:43:55.736 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:55.736 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:55.737 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:55.737 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.2", 2022-08-01T19:43:55.738 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCDKuhi5PgOORAAhcORK4bW2T3dXPeYjcCm+g==", 2022-08-01T19:43:55.738 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:43:55.739 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:43:55.739 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:43:55.740 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:43:55.740 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:55.740 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:55.741 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:55.741 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.3", 2022-08-01T19:43:55.742 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCGKuhiutn6GxAAk2ucLuGbT5YV/R4xXQGh0w==", 2022-08-01T19:43:55.742 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:43:55.743 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:43:55.743 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:43:55.744 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:43:55.744 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:55.745 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:55.745 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:55.746 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.4", 2022-08-01T19:43:55.746 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCLKuhiGHTRHxAArr3TwXA0gsCAAF/Qs38wUQ==", 2022-08-01T19:43:55.747 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:43:55.747 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:43:55.747 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:43:55.748 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:43:55.748 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:55.749 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:55.749 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:55.750 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.5", 2022-08-01T19:43:55.750 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCOKuhiMMDbBBAAvTzBvOcv7pFWm/P/lNqB0w==", 2022-08-01T19:43:55.750 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:43:55.751 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:43:55.751 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:43:55.752 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:43:55.752 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:55.752 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:55.753 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:55.753 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.6", 2022-08-01T19:43:55.758 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCQKuhiG2H3IxAAufUcWqo9ULquxIrUD5vSfA==", 2022-08-01T19:43:55.758 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:43:55.759 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:43:55.759 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:43:55.760 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:43:55.760 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:55.761 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:55.761 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:55.762 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.7", 2022-08-01T19:43:55.762 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQCTKuhiU50LChAAXuhrYBbHkk3eKzAPiEJFTg==", 2022-08-01T19:43:55.763 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:43:55.764 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:43:55.764 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:43:55.764 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:43:55.765 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:55.765 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:55.766 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:55.767 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.0", 2022-08-01T19:43:55.767 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+Kuhig3EnCxAANM20jSIQf/AKwuRX1qwT/A==", 2022-08-01T19:43:55.767 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:43:55.768 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:43:55.768 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow r", 2022-08-01T19:43:55.769 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T19:43:55.769 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:55.771 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:55.771 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:55.771 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.1", 2022-08-01T19:43:55.772 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB+KuhicJR0DhAAivqEAZUhXnSwjVRwWm2bJg==", 2022-08-01T19:43:55.772 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:43:55.772 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2022-08-01T19:43:55.773 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow r", 2022-08-01T19:43:55.773 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rw pool=cephfs_data" 2022-08-01T19:43:55.773 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:55.773 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:55.773 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:55.774 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.admin", 2022-08-01T19:43:55.774 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhiPTTGHxAAgfthCR6RX6j8F1YseFGNeA==", 2022-08-01T19:43:55.774 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:43:55.775 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:43:55.775 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2022-08-01T19:43:55.775 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2022-08-01T19:43:55.775 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:43:55.775 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:55.776 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:55.776 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:55.776 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.x", 2022-08-01T19:43:55.777 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9Kuhi2jvCNhAAyUxiGByZ+OZr8Y7dh3Qt5w==", 2022-08-01T19:43:55.777 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:43:55.777 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:43:55.777 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2022-08-01T19:43:55.777 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:43:55.778 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:55.778 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:55.778 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:55.779 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.y", 2022-08-01T19:43:55.780 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQB9KuhiDJXpMhAA8aek67ydbRgNO0h+miws+w==", 2022-08-01T19:43:55.780 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2022-08-01T19:43:55.780 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2022-08-01T19:43:55.781 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2022-08-01T19:43:55.781 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2022-08-01T19:43:55.781 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:55.781 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:55.782 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:43:55.782 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:43:55.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.709+0000 7f1e19948700 1 -- 172.21.15.17:0/3645248640 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1df8053aa0 msgr2=0x7f1df8055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:55.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.709+0000 7f1e19948700 1 --2- 172.21.15.17:0/3645248640 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1df8053aa0 0x7f1df8055f60 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7f1dfc00d000 tx=0x7f1dfc009040).stop 2022-08-01T19:43:55.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.709+0000 7f1e19948700 1 -- 172.21.15.17:0/3645248640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1df4003750 msgr2=0x7f1e140b30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:55.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.709+0000 7f1e19948700 1 --2- 172.21.15.17:0/3645248640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1df4003750 0x7f1e140b30b0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 rx=0x7f1e14060a40 tx=0x7f1e0c016000).stop 2022-08-01T19:43:55.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.709+0000 7f1e19948700 1 -- 172.21.15.17:0/3645248640 shutdown_connections 2022-08-01T19:43:55.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.709+0000 7f1e19948700 1 --2- 172.21.15.17:0/3645248640 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1df8053aa0 0x7f1df8055f60 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:55.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.709+0000 7f1e19948700 1 --2- 172.21.15.17:0/3645248640 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e141495d0 0x7f1e14140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:55.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.709+0000 7f1e19948700 1 --2- 172.21.15.17:0/3645248640 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1e14148040 0x7f1e14140460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:55.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.709+0000 7f1e19948700 1 --2- 172.21.15.17:0/3645248640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1df4003750 0x7f1e140b30b0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:55.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.709+0000 7f1e19948700 1 -- 172.21.15.17:0/3645248640 >> 172.21.15.17:0/3645248640 conn(0x7f1e1405c620 msgr2=0x7f1e14128b10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:55.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.709+0000 7f1e19948700 1 -- 172.21.15.17:0/3645248640 shutdown_connections 2022-08-01T19:43:55.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.709+0000 7f1e19948700 1 -- 172.21.15.17:0/3645248640 wait complete. 2022-08-01T19:43:55.789 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:43:55.897 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.893+0000 7f7f0dc84700 1 Processor -- start 2022-08-01T19:43:55.898 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.893+0000 7f7f0dc84700 1 -- start start 2022-08-01T19:43:55.910 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.897+0000 7f7f0dc84700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f7f080ad750 0x7f7f08061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:55.910 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.897+0000 7f7f0dc84700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7f081405d0 0x7f7f08140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:55.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.897+0000 7f7f0dc84700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f080aee80 con 0x7f7f080614b0 2022-08-01T19:43:55.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.897+0000 7f7f0dc84700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f080b70b0 con 0x7f7f080ad750 2022-08-01T19:43:55.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.897+0000 7f7f0dc84700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f080af0b0 con 0x7f7f081405d0 2022-08-01T19:43:55.912 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.897+0000 7f7f0cc82700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f7f080614b0 0x7f7f080b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37670/0 (socket says 172.21.15.17:37670) 2022-08-01T19:43:55.912 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.897+0000 7f7f0cc82700 1 -- 172.21.15.17:0/296686940 learned_addr learned my addr 172.21.15.17:0/296686940 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:55.913 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.897+0000 7f7f07fff700 1 --2- 172.21.15.17:0/296686940 >> v2:172.21.15.77:3300/0 conn(0x7f7f080ad750 0x7f7f08061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:55.913 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.897+0000 7f7f0d483700 1 --2- 172.21.15.17:0/296686940 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7f081405d0 0x7f7f08140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:55.913 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.897+0000 7f7f077fe700 1 -- 172.21.15.17:0/296686940 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2182020272 0 0) 0x7f7f080aee80 con 0x7f7f080614b0 2022-08-01T19:43:55.914 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.897+0000 7f7f077fe700 1 -- 172.21.15.17:0/296686940 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ee8003260 con 0x7f7f080614b0 2022-08-01T19:43:55.914 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.897+0000 7f7f07fff700 1 -- 172.21.15.17:0/296686940 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7f081405d0 msgr2=0x7f7f08140a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:55.914 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.897+0000 7f7f07fff700 1 --2- 172.21.15.17:0/296686940 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7f081405d0 0x7f7f08140a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:55.915 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.897+0000 7f7f07fff700 1 -- 172.21.15.17:0/296686940 >> v1:172.21.15.17:6789/0 conn(0x7f7f080614b0 legacy=0x7f7f080b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:55.915 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.897+0000 7f7f07fff700 1 -- 172.21.15.17:0/296686940 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f080af350 con 0x7f7f080ad750 2022-08-01T19:43:55.915 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.897+0000 7f7f07fff700 1 --2- 172.21.15.17:0/296686940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f080ad750 0x7f7f08061890 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f7ef8002f70 tx=0x7f7ef8006df0).ready entity=mon.1 client_cookie=4c400f215243d8ff server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:55.915 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.897+0000 7f7f077fe700 1 -- 172.21.15.17:0/296686940 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ef800d4a0 con 0x7f7f080ad750 2022-08-01T19:43:55.916 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.897+0000 7f7f077fe700 1 -- 172.21.15.17:0/296686940 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef8000e90 con 0x7f7f080ad750 2022-08-01T19:43:55.916 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.897+0000 7f7f077fe700 1 -- 172.21.15.17:0/296686940 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ef800d7f0 con 0x7f7f080ad750 2022-08-01T19:43:55.916 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.897+0000 7f7f0dc84700 1 -- 172.21.15.17:0/296686940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f080ad750 msgr2=0x7f7f08061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:55.916 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.897+0000 7f7f0dc84700 1 --2- 172.21.15.17:0/296686940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f080ad750 0x7f7f08061890 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f7ef8002f70 tx=0x7f7ef8006df0).stop 2022-08-01T19:43:55.916 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.897+0000 7f7f0dc84700 1 -- 172.21.15.17:0/296686940 shutdown_connections 2022-08-01T19:43:55.917 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.897+0000 7f7f0dc84700 1 --2- 172.21.15.17:0/296686940 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7f081405d0 0x7f7f08140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:55.917 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.897+0000 7f7f0dc84700 1 --2- 172.21.15.17:0/296686940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f080ad750 0x7f7f08061890 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:55.917 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.897+0000 7f7f0dc84700 1 -- 172.21.15.17:0/296686940 >> 172.21.15.17:0/296686940 conn(0x7f7f08066f70 msgr2=0x7f7f08067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:55.917 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.897+0000 7f7f0dc84700 1 -- 172.21.15.17:0/296686940 shutdown_connections 2022-08-01T19:43:55.918 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.897+0000 7f7f0dc84700 1 -- 172.21.15.17:0/296686940 wait complete. 2022-08-01T19:43:55.918 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.901+0000 7f7f0dc84700 1 Processor -- start 2022-08-01T19:43:55.918 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.901+0000 7f7f0dc84700 1 -- start start 2022-08-01T19:43:55.919 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.901+0000 7f7f0dc84700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f080614b0 0x7f7f08141970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:55.919 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.901+0000 7f7f0dc84700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f080ad750 0x7f7f08152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:55.919 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.901+0000 7f7f0dc84700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7f081405d0 0x7f7f08145340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:55.919 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.901+0000 7f7f0dc84700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f080aa1a0 con 0x7f7f080ad750 2022-08-01T19:43:55.919 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.901+0000 7f7f0dc84700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f080aa3d0 con 0x7f7f080614b0 2022-08-01T19:43:55.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.901+0000 7f7f0dc84700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f080aa670 con 0x7f7f081405d0 2022-08-01T19:43:55.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.901+0000 7f7f07fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f080ad750 0x7f7f08152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:55.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.901+0000 7f7f07fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f080ad750 0x7f7f08152d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45484/0 (socket says 172.21.15.17:45484) 2022-08-01T19:43:55.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.901+0000 7f7f07fff700 1 -- 172.21.15.17:0/1627157197 learned_addr learned my addr 172.21.15.17:0/1627157197 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:55.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.901+0000 7f7f0cc82700 1 --2- 172.21.15.17:0/1627157197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f080614b0 0x7f7f08141970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:55.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.901+0000 7f7f0d483700 1 --2- 172.21.15.17:0/1627157197 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7f081405d0 0x7f7f08145340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:55.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.901+0000 7f7f07fff700 1 -- 172.21.15.17:0/1627157197 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7f081405d0 msgr2=0x7f7f08145340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:55.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.901+0000 7f7f07fff700 1 --2- 172.21.15.17:0/1627157197 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7f081405d0 0x7f7f08145340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:55.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.901+0000 7f7f07fff700 1 -- 172.21.15.17:0/1627157197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f080614b0 msgr2=0x7f7f08141970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:55.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.901+0000 7f7f07fff700 1 --2- 172.21.15.17:0/1627157197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f080614b0 0x7f7f08141970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:55.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.901+0000 7f7f07fff700 1 -- 172.21.15.17:0/1627157197 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef800b180 con 0x7f7f080ad750 2022-08-01T19:43:55.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.901+0000 7f7f07fff700 1 --2- 172.21.15.17:0/1627157197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f080ad750 0x7f7f08152d30 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 rx=0x7f7ef8002f70 tx=0x7f7ef8014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:55.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.901+0000 7f7f05ffb700 1 -- 172.21.15.17:0/1627157197 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ef8016070 con 0x7f7f080ad750 2022-08-01T19:43:55.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.901+0000 7f7f0dc84700 1 -- 172.21.15.17:0/1627157197 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f08152890 con 0x7f7f080ad750 2022-08-01T19:43:55.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.901+0000 7f7f05ffb700 1 -- 172.21.15.17:0/1627157197 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef80137d0 con 0x7f7f080ad750 2022-08-01T19:43:55.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.901+0000 7f7f05ffb700 1 -- 172.21.15.17:0/1627157197 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ef8013b30 con 0x7f7f080ad750 2022-08-01T19:43:55.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.901+0000 7f7f0dc84700 1 -- 172.21.15.17:0/1627157197 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f08141e60 con 0x7f7f080ad750 2022-08-01T19:43:55.924 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.905+0000 7f7f0dc84700 1 -- 172.21.15.17:0/1627157197 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f08152a20 con 0x7f7f080ad750 2022-08-01T19:43:55.924 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.905+0000 7f7f05ffb700 1 -- 172.21.15.17:0/1627157197 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7ef800f030 con 0x7f7f080ad750 2022-08-01T19:43:55.924 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.905+0000 7f7f05ffb700 1 --2- 172.21.15.17:0/1627157197 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7eec053a70 0x7f7eec055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:55.924 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.905+0000 7f7f05ffb700 1 -- 172.21.15.17:0/1627157197 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f7ef806ca90 con 0x7f7f080ad750 2022-08-01T19:43:55.925 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.905+0000 7f7f0cc82700 1 --2- 172.21.15.17:0/1627157197 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7eec053a70 0x7f7eec055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:55.925 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.905+0000 7f7f0cc82700 1 --2- 172.21.15.17:0/1627157197 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7eec053a70 0x7f7eec055f30 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7f7ef000d000 tx=0x7f7ef0009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:55.926 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:55.913+0000 7f7f05ffb700 1 -- 172.21.15.17:0/1627157197 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7ef8043a00 con 0x7f7f080ad750 2022-08-01T19:43:56.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.041+0000 7f7f0dc84700 1 -- 172.21.15.17:0/1627157197 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f080af5f0 con 0x7f7f080ad750 2022-08-01T19:43:56.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.041+0000 7f7f05ffb700 1 -- 172.21.15.17:0/1627157197 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+13424 (secure 0 0 0) 0x7f7ef8023070 con 0x7f7f080ad750 2022-08-01T19:43:56.045 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:56.045 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":154,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:43:55.383780+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:43:56.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.045+0000 7f7f0dc84700 1 -- 172.21.15.17:0/1627157197 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7eec053a70 msgr2=0x7f7eec055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:56.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.045+0000 7f7f0dc84700 1 --2- 172.21.15.17:0/1627157197 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7eec053a70 0x7f7eec055f30 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7f7ef000d000 tx=0x7f7ef0009040).stop 2022-08-01T19:43:56.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.045+0000 7f7f0dc84700 1 -- 172.21.15.17:0/1627157197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f080ad750 msgr2=0x7f7f08152d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:56.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.045+0000 7f7f0dc84700 1 --2- 172.21.15.17:0/1627157197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f080ad750 0x7f7f08152d30 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 rx=0x7f7ef8002f70 tx=0x7f7ef8014000).stop 2022-08-01T19:43:56.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.045+0000 7f7f0dc84700 1 -- 172.21.15.17:0/1627157197 shutdown_connections 2022-08-01T19:43:56.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.045+0000 7f7f0dc84700 1 --2- 172.21.15.17:0/1627157197 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7eec053a70 0x7f7eec055f30 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:56.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.045+0000 7f7f0dc84700 1 --2- 172.21.15.17:0/1627157197 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7f081405d0 0x7f7f08145340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:56.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.045+0000 7f7f0dc84700 1 --2- 172.21.15.17:0/1627157197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f080ad750 0x7f7f08152d30 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:56.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.045+0000 7f7f0dc84700 1 --2- 172.21.15.17:0/1627157197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f080614b0 0x7f7f08141970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:56.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.045+0000 7f7f0dc84700 1 -- 172.21.15.17:0/1627157197 >> 172.21.15.17:0/1627157197 conn(0x7f7f08066f70 msgr2=0x7f7f08142500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:56.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.045+0000 7f7f0dc84700 1 -- 172.21.15.17:0/1627157197 shutdown_connections 2022-08-01T19:43:56.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.045+0000 7f7f0dc84700 1 -- 172.21.15.17:0/1627157197 wait complete. 2022-08-01T19:43:56.067 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-08-01T19:43:56.068 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:43:56.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.173+0000 7f9318cf8700 1 Processor -- start 2022-08-01T19:43:56.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.173+0000 7f9318cf8700 1 -- start start 2022-08-01T19:43:56.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.173+0000 7f9318cf8700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f931405c960 0x7f93140b1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:56.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.173+0000 7f9318cf8700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f931405d240 0x7f931413f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:56.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.173+0000 7f9318cf8700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f931413fa10 con 0x7f93140acfc0 2022-08-01T19:43:56.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.173+0000 7f9318cf8700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9314067f00 con 0x7f931405d240 2022-08-01T19:43:56.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.173+0000 7f9318cf8700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931405cd40 con 0x7f931405c960 2022-08-01T19:43:56.182 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.177+0000 7f9312ffd700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f931405d240 0x7f931413f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:56.182 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.177+0000 7f93137fe700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f931405c960 0x7f93140b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:56.182 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.177+0000 7f9313fff700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7f93140acfc0 0x7f93140aee70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:54484/0 (socket says 172.21.15.17:54484) 2022-08-01T19:43:56.182 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.177+0000 7f9312ffd700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f931405d240 0x7f931413f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:43412/0 (socket says 172.21.15.17:43412) 2022-08-01T19:43:56.183 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.177+0000 7f9313fff700 1 -- 172.21.15.17:0/1166697620 learned_addr learned my addr 172.21.15.17:0/1166697620 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:56.190 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.177+0000 7f93127fc700 1 -- 172.21.15.17:0/1166697620 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2640028208 0 0) 0x7f930c001490 con 0x7f93140acfc0 2022-08-01T19:43:56.190 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.177+0000 7f93127fc700 1 -- 172.21.15.17:0/1166697620 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92f4003260 con 0x7f93140acfc0 2022-08-01T19:43:56.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.177+0000 7f93137fe700 1 -- 172.21.15.17:0/1166697620 >> v2:172.21.15.77:3300/0 conn(0x7f931405d240 msgr2=0x7f931413f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:56.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.177+0000 7f93137fe700 1 --2- 172.21.15.17:0/1166697620 >> v2:172.21.15.77:3300/0 conn(0x7f931405d240 0x7f931413f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:56.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.177+0000 7f93137fe700 1 -- 172.21.15.17:0/1166697620 >> v1:172.21.15.77:6789/0 conn(0x7f93140acfc0 legacy=0x7f93140aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:56.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.177+0000 7f93137fe700 1 -- 172.21.15.17:0/1166697620 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93140b1920 con 0x7f931405c960 2022-08-01T19:43:56.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.177+0000 7f9312ffd700 1 --2- 172.21.15.17:0/1166697620 >> v2:172.21.15.77:3300/0 conn(0x7f931405d240 0x7f931413f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:43:56.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.177+0000 7f93137fe700 1 --2- 172.21.15.17:0/1166697620 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f931405c960 0x7f93140b1280 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f9304002f70 tx=0x7f9304006df0).ready entity=mon.2 client_cookie=515f60db5424f45 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:56.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.177+0000 7f93127fc700 1 -- 172.21.15.17:0/1166697620 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f930400d4a0 con 0x7f931405c960 2022-08-01T19:43:56.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.177+0000 7f93127fc700 1 -- 172.21.15.17:0/1166697620 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9304000e90 con 0x7f931405c960 2022-08-01T19:43:56.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.177+0000 7f93127fc700 1 -- 172.21.15.17:0/1166697620 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f930400d7f0 con 0x7f931405c960 2022-08-01T19:43:56.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.177+0000 7f9318cf8700 1 -- 172.21.15.17:0/1166697620 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f931405c960 msgr2=0x7f93140b1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:56.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.177+0000 7f9318cf8700 1 --2- 172.21.15.17:0/1166697620 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f931405c960 0x7f93140b1280 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f9304002f70 tx=0x7f9304006df0).stop 2022-08-01T19:43:56.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.177+0000 7f9318cf8700 1 -- 172.21.15.17:0/1166697620 shutdown_connections 2022-08-01T19:43:56.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.177+0000 7f9318cf8700 1 --2- 172.21.15.17:0/1166697620 >> v2:172.21.15.77:3300/0 conn(0x7f931405d240 0x7f931413f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:56.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.177+0000 7f9318cf8700 1 --2- 172.21.15.17:0/1166697620 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f931405c960 0x7f93140b1280 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:56.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.177+0000 7f9318cf8700 1 -- 172.21.15.17:0/1166697620 >> 172.21.15.17:0/1166697620 conn(0x7f9314060310 msgr2=0x7f93140b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:56.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.177+0000 7f9318cf8700 1 -- 172.21.15.17:0/1166697620 shutdown_connections 2022-08-01T19:43:56.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.177+0000 7f9318cf8700 1 -- 172.21.15.17:0/1166697620 wait complete. 2022-08-01T19:43:56.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.177+0000 7f9318cf8700 1 Processor -- start 2022-08-01T19:43:56.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.177+0000 7f9318cf8700 1 -- start start 2022-08-01T19:43:56.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.181+0000 7f9318cf8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f931405d240 0x7f9314142760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:56.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.181+0000 7f9318cf8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93140acfc0 0x7f9314137540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:56.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.181+0000 7f9318cf8700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f93140b47c0 0x7f9314137a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:56.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.181+0000 7f9318cf8700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93140b54b0 con 0x7f93140acfc0 2022-08-01T19:43:56.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.181+0000 7f9318cf8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93140b4cd0 con 0x7f931405d240 2022-08-01T19:43:56.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.181+0000 7f9318cf8700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93140b4f70 con 0x7f93140b47c0 2022-08-01T19:43:56.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.181+0000 7f9312ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93140acfc0 0x7f9314137540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:56.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.181+0000 7f9312ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93140acfc0 0x7f9314137540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45496/0 (socket says 172.21.15.17:45496) 2022-08-01T19:43:56.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.181+0000 7f9312ffd700 1 -- 172.21.15.17:0/3100991014 learned_addr learned my addr 172.21.15.17:0/3100991014 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:56.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.181+0000 7f9313fff700 1 --2- 172.21.15.17:0/3100991014 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f93140b47c0 0x7f9314137a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:56.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.181+0000 7f93137fe700 1 --2- 172.21.15.17:0/3100991014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f931405d240 0x7f9314142760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:56.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.181+0000 7f9312ffd700 1 -- 172.21.15.17:0/3100991014 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f93140b47c0 msgr2=0x7f9314137a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:56.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.181+0000 7f9312ffd700 1 --2- 172.21.15.17:0/3100991014 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f93140b47c0 0x7f9314137a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:56.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.181+0000 7f9312ffd700 1 -- 172.21.15.17:0/3100991014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f931405d240 msgr2=0x7f9314142760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:56.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.181+0000 7f9312ffd700 1 --2- 172.21.15.17:0/3100991014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f931405d240 0x7f9314142760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:56.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.181+0000 7f9312ffd700 1 -- 172.21.15.17:0/3100991014 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92fc00b180 con 0x7f93140acfc0 2022-08-01T19:43:56.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.181+0000 7f93137fe700 1 --2- 172.21.15.17:0/3100991014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f931405d240 0x7f9314142760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:43:56.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.181+0000 7f9312ffd700 1 --2- 172.21.15.17:0/3100991014 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93140acfc0 0x7f9314137540 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 rx=0x7f92fc00c000 tx=0x7f92fc006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:56.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.181+0000 7f9310ff9700 1 -- 172.21.15.17:0/3100991014 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92fc00c890 con 0x7f93140acfc0 2022-08-01T19:43:56.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.181+0000 7f9310ff9700 1 -- 172.21.15.17:0/3100991014 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92fc00c9f0 con 0x7f93140acfc0 2022-08-01T19:43:56.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.181+0000 7f9318cf8700 1 -- 172.21.15.17:0/3100991014 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f930400b180 con 0x7f93140acfc0 2022-08-01T19:43:56.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.181+0000 7f9310ff9700 1 -- 172.21.15.17:0/3100991014 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f92fc00cd50 con 0x7f93140acfc0 2022-08-01T19:43:56.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.181+0000 7f9318cf8700 1 -- 172.21.15.17:0/3100991014 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93140b60d0 con 0x7f93140acfc0 2022-08-01T19:43:56.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.181+0000 7f9310ff9700 1 -- 172.21.15.17:0/3100991014 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f92fc01c460 con 0x7f93140acfc0 2022-08-01T19:43:56.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.185+0000 7f9318cf8700 1 -- 172.21.15.17:0/3100991014 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93140adbb0 con 0x7f93140acfc0 2022-08-01T19:43:56.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.185+0000 7f9310ff9700 1 --2- 172.21.15.17:0/3100991014 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f92f80539e0 0x7f92f8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:56.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.185+0000 7f9310ff9700 1 -- 172.21.15.17:0/3100991014 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f92fc06c440 con 0x7f93140acfc0 2022-08-01T19:43:56.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.185+0000 7f93137fe700 1 --2- 172.21.15.17:0/3100991014 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f92f80539e0 0x7f92f8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:56.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.185+0000 7f93137fe700 1 --2- 172.21.15.17:0/3100991014 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f92f80539e0 0x7f92f8055ea0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7f9304018000 tx=0x7f9304014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:56.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.189+0000 7f9310ff9700 1 -- 172.21.15.17:0/3100991014 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f92fc0433d0 con 0x7f93140acfc0 2022-08-01T19:43:56.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.313+0000 7f9318cf8700 1 -- 172.21.15.17:0/3100991014 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f93140b1bc0 con 0x7f93140acfc0 2022-08-01T19:43:56.318 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.313+0000 7f9310ff9700 1 -- 172.21.15.17:0/3100991014 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v154) v1 ==== 110+0+10 (secure 0 0 0) 0x7f92fc0469f0 con 0x7f93140acfc0 2022-08-01T19:43:56.319 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:43:56.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.317+0000 7f9318cf8700 1 -- 172.21.15.17:0/3100991014 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f92f80539e0 msgr2=0x7f92f8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:56.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.317+0000 7f9318cf8700 1 --2- 172.21.15.17:0/3100991014 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f92f80539e0 0x7f92f8055ea0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7f9304018000 tx=0x7f9304014000).stop 2022-08-01T19:43:56.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.317+0000 7f9318cf8700 1 -- 172.21.15.17:0/3100991014 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93140acfc0 msgr2=0x7f9314137540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:56.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.317+0000 7f9318cf8700 1 --2- 172.21.15.17:0/3100991014 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93140acfc0 0x7f9314137540 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 rx=0x7f92fc00c000 tx=0x7f92fc006df0).stop 2022-08-01T19:43:56.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.321+0000 7f9318cf8700 1 -- 172.21.15.17:0/3100991014 shutdown_connections 2022-08-01T19:43:56.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.321+0000 7f9318cf8700 1 --2- 172.21.15.17:0/3100991014 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f92f80539e0 0x7f92f8055ea0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:56.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.321+0000 7f9318cf8700 1 --2- 172.21.15.17:0/3100991014 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f93140b47c0 0x7f9314137a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:56.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.321+0000 7f9318cf8700 1 --2- 172.21.15.17:0/3100991014 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93140acfc0 0x7f9314137540 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:56.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.321+0000 7f9318cf8700 1 --2- 172.21.15.17:0/3100991014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f931405d240 0x7f9314142760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:56.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.321+0000 7f9318cf8700 1 -- 172.21.15.17:0/3100991014 >> 172.21.15.17:0/3100991014 conn(0x7f9314060310 msgr2=0x7f93140af6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:56.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.321+0000 7f9318cf8700 1 -- 172.21.15.17:0/3100991014 shutdown_connections 2022-08-01T19:43:56.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.321+0000 7f9318cf8700 1 -- 172.21.15.17:0/3100991014 wait complete. 2022-08-01T19:43:56.339 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:43:56.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.449+0000 7fa7987ab700 1 Processor -- start 2022-08-01T19:43:56.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.453+0000 7fa7987ab700 1 -- start start 2022-08-01T19:43:56.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.453+0000 7fa7987ab700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fa7900b28a0 0x7fa7901432a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:56.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.453+0000 7fa7987ab700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa790061360 0x7fa790143790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:56.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.453+0000 7fa7987ab700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa790064aa0 con 0x7fa7900600f0 2022-08-01T19:43:56.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.453+0000 7fa7987ab700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa790064d40 con 0x7fa790061360 2022-08-01T19:43:56.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.453+0000 7fa7987ab700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa790064fe0 con 0x7fa7900b28a0 2022-08-01T19:43:56.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.453+0000 7fa796d48700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa790061360 0x7fa790143790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:56.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.453+0000 7fa796547700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa7900600f0 0x7fa7901405d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37696/0 (socket says 172.21.15.17:37696) 2022-08-01T19:43:56.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.453+0000 7fa796547700 1 -- 172.21.15.17:0/2175821027 learned_addr learned my addr 172.21.15.17:0/2175821027 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:56.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.453+0000 7fa795d46700 1 --2- 172.21.15.17:0/2175821027 >> v2:172.21.15.77:3300/0 conn(0x7fa7900b28a0 0x7fa7901432a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:56.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.453+0000 7fa796d48700 1 -- 172.21.15.17:0/2175821027 >> v2:172.21.15.77:3300/0 conn(0x7fa7900b28a0 msgr2=0x7fa7901432a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:56.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.453+0000 7fa796d48700 1 --2- 172.21.15.17:0/2175821027 >> v2:172.21.15.77:3300/0 conn(0x7fa7900b28a0 0x7fa7901432a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:56.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.453+0000 7fa796d48700 1 -- 172.21.15.17:0/2175821027 >> v1:172.21.15.17:6789/0 conn(0x7fa7900600f0 legacy=0x7fa7901405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:56.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.453+0000 7fa795545700 1 -- 172.21.15.17:0/2175821027 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1996310176 0 0) 0x7fa790064aa0 con 0x7fa7900600f0 2022-08-01T19:43:56.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.453+0000 7fa796d48700 1 -- 172.21.15.17:0/2175821027 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa790065280 con 0x7fa790061360 2022-08-01T19:43:56.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.453+0000 7fa796d48700 1 --2- 172.21.15.17:0/2175821027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa790061360 0x7fa790143790 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 rx=0x7fa7880032f0 tx=0x7fa7880091d0).ready entity=mon.0 client_cookie=812ca72a3075202f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:56.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.453+0000 7fa795545700 1 -- 172.21.15.17:0/2175821027 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa78800b680 con 0x7fa790061360 2022-08-01T19:43:56.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.453+0000 7fa795545700 1 -- 172.21.15.17:0/2175821027 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa78800b7e0 con 0x7fa790061360 2022-08-01T19:43:56.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.453+0000 7fa795545700 1 -- 172.21.15.17:0/2175821027 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa78800b950 con 0x7fa790061360 2022-08-01T19:43:56.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.457+0000 7fa7987ab700 1 -- 172.21.15.17:0/2175821027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa790061360 msgr2=0x7fa790143790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:56.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.457+0000 7fa7987ab700 1 --2- 172.21.15.17:0/2175821027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa790061360 0x7fa790143790 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 rx=0x7fa7880032f0 tx=0x7fa7880091d0).stop 2022-08-01T19:43:56.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.457+0000 7fa7987ab700 1 -- 172.21.15.17:0/2175821027 shutdown_connections 2022-08-01T19:43:56.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.457+0000 7fa7987ab700 1 --2- 172.21.15.17:0/2175821027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa790061360 0x7fa790143790 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:56.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.457+0000 7fa7987ab700 1 --2- 172.21.15.17:0/2175821027 >> v2:172.21.15.77:3300/0 conn(0x7fa7900b28a0 0x7fa7901432a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:56.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.457+0000 7fa7987ab700 1 -- 172.21.15.17:0/2175821027 >> 172.21.15.17:0/2175821027 conn(0x7fa79005d0f0 msgr2=0x7fa79005f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:56.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.457+0000 7fa7987ab700 1 -- 172.21.15.17:0/2175821027 shutdown_connections 2022-08-01T19:43:56.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.457+0000 7fa7987ab700 1 -- 172.21.15.17:0/2175821027 wait complete. 2022-08-01T19:43:56.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.457+0000 7fa7987ab700 1 Processor -- start 2022-08-01T19:43:56.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.457+0000 7fa7987ab700 1 -- start start 2022-08-01T19:43:56.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.461+0000 7fa7987ab700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7900600f0 0x7fa790149380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:56.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.461+0000 7fa7987ab700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa790061360 0x7fa7900ad5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:56.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.461+0000 7fa7987ab700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa7900b28a0 0x7fa7900adab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:56.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.461+0000 7fa7987ab700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7901530e0 con 0x7fa7900600f0 2022-08-01T19:43:56.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.461+0000 7fa7987ab700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7900ae3e0 con 0x7fa790061360 2022-08-01T19:43:56.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.461+0000 7fa7987ab700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7900adfa0 con 0x7fa7900b28a0 2022-08-01T19:43:56.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.461+0000 7fa796547700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7900600f0 0x7fa790149380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:56.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.461+0000 7fa796547700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7900600f0 0x7fa790149380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45508/0 (socket says 172.21.15.17:45508) 2022-08-01T19:43:56.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.461+0000 7fa796547700 1 -- 172.21.15.17:0/1264889068 learned_addr learned my addr 172.21.15.17:0/1264889068 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:56.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.461+0000 7fa795d46700 1 --2- 172.21.15.17:0/1264889068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa790061360 0x7fa7900ad5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:56.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.461+0000 7fa796d48700 1 --2- 172.21.15.17:0/1264889068 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa7900b28a0 0x7fa7900adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:56.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.461+0000 7fa796547700 1 -- 172.21.15.17:0/1264889068 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa7900b28a0 msgr2=0x7fa7900adab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:56.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.461+0000 7fa796547700 1 --2- 172.21.15.17:0/1264889068 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa7900b28a0 0x7fa7900adab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:56.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.461+0000 7fa796547700 1 -- 172.21.15.17:0/1264889068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa790061360 msgr2=0x7fa7900ad5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:56.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.461+0000 7fa796547700 1 --2- 172.21.15.17:0/1264889068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa790061360 0x7fa7900ad5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:56.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.461+0000 7fa796547700 1 -- 172.21.15.17:0/1264889068 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa788009470 con 0x7fa7900600f0 2022-08-01T19:43:56.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.461+0000 7fa796547700 1 --2- 172.21.15.17:0/1264889068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7900600f0 0x7fa790149380 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 rx=0x7fa78c000ed0 tx=0x7fa78c007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:56.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.461+0000 7fa7837fe700 1 -- 172.21.15.17:0/1264889068 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa78c0070d0 con 0x7fa7900600f0 2022-08-01T19:43:56.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.461+0000 7fa7837fe700 1 -- 172.21.15.17:0/1264889068 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa78c007230 con 0x7fa7900600f0 2022-08-01T19:43:56.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.461+0000 7fa7837fe700 1 -- 172.21.15.17:0/1264889068 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa78c005610 con 0x7fa7900600f0 2022-08-01T19:43:56.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.461+0000 7fa7987ab700 1 -- 172.21.15.17:0/1264889068 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa790141640 con 0x7fa7900600f0 2022-08-01T19:43:56.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.461+0000 7fa7987ab700 1 -- 172.21.15.17:0/1264889068 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7901418e0 con 0x7fa7900600f0 2022-08-01T19:43:56.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.465+0000 7fa7837fe700 1 -- 172.21.15.17:0/1264889068 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa78c010070 con 0x7fa7900600f0 2022-08-01T19:43:56.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.465+0000 7fa7987ab700 1 -- 172.21.15.17:0/1264889068 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa790152f50 con 0x7fa7900600f0 2022-08-01T19:43:56.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.465+0000 7fa7837fe700 1 --2- 172.21.15.17:0/1264889068 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa770053b30 0x7fa770055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:56.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.465+0000 7fa7837fe700 1 -- 172.21.15.17:0/1264889068 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fa78c06b460 con 0x7fa7900600f0 2022-08-01T19:43:56.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.465+0000 7fa795d46700 1 --2- 172.21.15.17:0/1264889068 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa770053b30 0x7fa770055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:56.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.465+0000 7fa795d46700 1 --2- 172.21.15.17:0/1264889068 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa770053b30 0x7fa770055ff0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7fa78400b000 tx=0x7fa784006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:56.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.469+0000 7fa7837fe700 1 -- 172.21.15.17:0/1264889068 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa78c042560 con 0x7fa7900600f0 2022-08-01T19:43:56.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.601+0000 7fa7987ab700 1 -- 172.21.15.17:0/1264889068 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa790141e80 con 0x7fa7900600f0 2022-08-01T19:43:56.604 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.601+0000 7fa7837fe700 1 -- 172.21.15.17:0/1264889068 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v154) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa78c045b80 con 0x7fa7900600f0 2022-08-01T19:43:56.604 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:43:56.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.605+0000 7fa7987ab700 1 -- 172.21.15.17:0/1264889068 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa770053b30 msgr2=0x7fa770055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:56.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.605+0000 7fa7987ab700 1 --2- 172.21.15.17:0/1264889068 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa770053b30 0x7fa770055ff0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7fa78400b000 tx=0x7fa784006df0).stop 2022-08-01T19:43:56.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.605+0000 7fa7987ab700 1 -- 172.21.15.17:0/1264889068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7900600f0 msgr2=0x7fa790149380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:56.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.605+0000 7fa7987ab700 1 --2- 172.21.15.17:0/1264889068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7900600f0 0x7fa790149380 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 rx=0x7fa78c000ed0 tx=0x7fa78c007ac0).stop 2022-08-01T19:43:56.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.605+0000 7fa7987ab700 1 -- 172.21.15.17:0/1264889068 shutdown_connections 2022-08-01T19:43:56.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.605+0000 7fa7987ab700 1 --2- 172.21.15.17:0/1264889068 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa770053b30 0x7fa770055ff0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:56.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.605+0000 7fa7987ab700 1 --2- 172.21.15.17:0/1264889068 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa7900b28a0 0x7fa7900adab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:56.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.605+0000 7fa7987ab700 1 --2- 172.21.15.17:0/1264889068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa790061360 0x7fa7900ad5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:56.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.605+0000 7fa7987ab700 1 --2- 172.21.15.17:0/1264889068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7900600f0 0x7fa790149380 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:56.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.605+0000 7fa7987ab700 1 -- 172.21.15.17:0/1264889068 >> 172.21.15.17:0/1264889068 conn(0x7fa79005d0f0 msgr2=0x7fa79013fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:56.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.605+0000 7fa7987ab700 1 -- 172.21.15.17:0/1264889068 shutdown_connections 2022-08-01T19:43:56.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.605+0000 7fa7987ab700 1 -- 172.21.15.17:0/1264889068 wait complete. 2022-08-01T19:43:56.625 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-08-01T19:43:56.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.729+0000 7f8886665700 1 Processor -- start 2022-08-01T19:43:56.733 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.729+0000 7f8886665700 1 -- start start 2022-08-01T19:43:56.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.729+0000 7f8886665700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f88800b6ee0 0x7f88800b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:56.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.729+0000 7f8886665700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f88800b5ed0 0x7f88800b5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:56.746 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.729+0000 7f8886665700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8880061d90 con 0x7f88800619b0 2022-08-01T19:43:56.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.729+0000 7f8886665700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8880143fa0 con 0x7f88800b6ee0 2022-08-01T19:43:56.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.729+0000 7f8886665700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8880142f60 con 0x7f88800b5ed0 2022-08-01T19:43:56.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.729+0000 7f8884e62700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f88800b6ee0 0x7f88800b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:56.747 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.729+0000 7f8884e62700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f88800b6ee0 0x7f88800b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45520/0 (socket says 172.21.15.17:45520) 2022-08-01T19:43:56.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.729+0000 7f8884e62700 1 -- 172.21.15.17:0/3625106449 learned_addr learned my addr 172.21.15.17:0/3625106449 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:56.748 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.729+0000 7f8885e64700 1 --2- 172.21.15.17:0/3625106449 >> v2:172.21.15.77:3300/0 conn(0x7f88800b5ed0 0x7f88800b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:56.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.733+0000 7f8884e62700 1 -- 172.21.15.17:0/3625106449 >> v2:172.21.15.77:3300/0 conn(0x7f88800b5ed0 msgr2=0x7f88800b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:56.749 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.733+0000 7f8884e62700 1 --2- 172.21.15.17:0/3625106449 >> v2:172.21.15.77:3300/0 conn(0x7f88800b5ed0 0x7f88800b5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:56.750 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.733+0000 7f8873fff700 1 -- 172.21.15.17:0/3625106449 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3439906408 0 0) 0x7f8880061d90 con 0x7f88800619b0 2022-08-01T19:43:56.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.733+0000 7f8884e62700 1 -- 172.21.15.17:0/3625106449 >> v1:172.21.15.77:6789/0 conn(0x7f88800619b0 legacy=0x7f8880143890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:56.751 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.733+0000 7f8884e62700 1 -- 172.21.15.17:0/3625106449 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88800ba5a0 con 0x7f88800b6ee0 2022-08-01T19:43:56.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.733+0000 7f8885e64700 1 --2- 172.21.15.17:0/3625106449 >> v2:172.21.15.77:3300/0 conn(0x7f88800b5ed0 0x7f88800b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:43:56.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.733+0000 7f8884e62700 1 --2- 172.21.15.17:0/3625106449 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88800b6ee0 0x7f88800b48a0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 rx=0x7f8874002f70 tx=0x7f8874006eb0).ready entity=mon.0 client_cookie=351e20a4c56a987b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:56.752 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.733+0000 7f8873fff700 1 -- 172.21.15.17:0/3625106449 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f887400d5c0 con 0x7f88800b6ee0 2022-08-01T19:43:56.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.733+0000 7f8873fff700 1 -- 172.21.15.17:0/3625106449 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8874000e90 con 0x7f88800b6ee0 2022-08-01T19:43:56.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.733+0000 7f8873fff700 1 -- 172.21.15.17:0/3625106449 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f887400d910 con 0x7f88800b6ee0 2022-08-01T19:43:56.753 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.733+0000 7f8886665700 1 -- 172.21.15.17:0/3625106449 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88800b6ee0 msgr2=0x7f88800b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:56.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.733+0000 7f8886665700 1 --2- 172.21.15.17:0/3625106449 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88800b6ee0 0x7f88800b48a0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 rx=0x7f8874002f70 tx=0x7f8874006eb0).stop 2022-08-01T19:43:56.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.733+0000 7f8886665700 1 -- 172.21.15.17:0/3625106449 shutdown_connections 2022-08-01T19:43:56.754 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.733+0000 7f8886665700 1 --2- 172.21.15.17:0/3625106449 >> v2:172.21.15.77:3300/0 conn(0x7f88800b5ed0 0x7f88800b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:56.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.733+0000 7f8886665700 1 --2- 172.21.15.17:0/3625106449 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88800b6ee0 0x7f88800b48a0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:56.755 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.733+0000 7f8886665700 1 -- 172.21.15.17:0/3625106449 >> 172.21.15.17:0/3625106449 conn(0x7f888006af80 msgr2=0x7f888005fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:56.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.733+0000 7f8886665700 1 -- 172.21.15.17:0/3625106449 shutdown_connections 2022-08-01T19:43:56.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.733+0000 7f8886665700 1 -- 172.21.15.17:0/3625106449 wait complete. 2022-08-01T19:43:56.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.733+0000 7f8886665700 1 Processor -- start 2022-08-01T19:43:56.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.733+0000 7f8886665700 1 -- start start 2022-08-01T19:43:56.757 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.737+0000 7f8886665700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f88800619b0 0x7f8880146570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:56.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.737+0000 7f8886665700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88800b5ed0 0x7f888013f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:56.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.737+0000 7f8886665700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88800b6ee0 0x7f888013f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:56.759 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.737+0000 7f8886665700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888013fb80 con 0x7f88800b5ed0 2022-08-01T19:43:56.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.737+0000 7f8886665700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8880156860 con 0x7f88800b6ee0 2022-08-01T19:43:56.760 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.737+0000 7f8886665700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88800b05b0 con 0x7f88800619b0 2022-08-01T19:43:56.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.737+0000 7f8884e62700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88800b5ed0 0x7f888013f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:56.762 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.737+0000 7f8884e62700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88800b5ed0 0x7f888013f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45526/0 (socket says 172.21.15.17:45526) 2022-08-01T19:43:56.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.737+0000 7f8884e62700 1 -- 172.21.15.17:0/163808430 learned_addr learned my addr 172.21.15.17:0/163808430 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:56.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.737+0000 7f8885663700 1 --2- 172.21.15.17:0/163808430 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f88800619b0 0x7f8880146570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:56.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.737+0000 7f8885e64700 1 --2- 172.21.15.17:0/163808430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88800b6ee0 0x7f888013f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:56.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.737+0000 7f8884e62700 1 -- 172.21.15.17:0/163808430 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f88800619b0 msgr2=0x7f8880146570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:56.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.737+0000 7f8884e62700 1 --2- 172.21.15.17:0/163808430 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f88800619b0 0x7f8880146570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:56.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.737+0000 7f8884e62700 1 -- 172.21.15.17:0/163808430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88800b6ee0 msgr2=0x7f888013f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:56.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.737+0000 7f8884e62700 1 --2- 172.21.15.17:0/163808430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88800b6ee0 0x7f888013f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:56.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.737+0000 7f8884e62700 1 -- 172.21.15.17:0/163808430 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887c00b180 con 0x7f88800b5ed0 2022-08-01T19:43:56.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.737+0000 7f8885663700 1 --2- 172.21.15.17:0/163808430 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f88800619b0 0x7f8880146570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:43:56.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.737+0000 7f8884e62700 1 --2- 172.21.15.17:0/163808430 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88800b5ed0 0x7f888013f1a0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 rx=0x7f8874002f70 tx=0x7f8874015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:56.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.737+0000 7f88727fc700 1 -- 172.21.15.17:0/163808430 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8874011990 con 0x7f88800b5ed0 2022-08-01T19:43:56.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.737+0000 7f88727fc700 1 -- 172.21.15.17:0/163808430 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8874011af0 con 0x7f88800b5ed0 2022-08-01T19:43:56.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.737+0000 7f8886665700 1 -- 172.21.15.17:0/163808430 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f887400b1e0 con 0x7f88800b5ed0 2022-08-01T19:43:56.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.737+0000 7f88727fc700 1 -- 172.21.15.17:0/163808430 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8874011e50 con 0x7f88800b5ed0 2022-08-01T19:43:56.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.737+0000 7f8886665700 1 -- 172.21.15.17:0/163808430 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8880156580 con 0x7f88800b5ed0 2022-08-01T19:43:56.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.737+0000 7f8886665700 1 -- 172.21.15.17:0/163808430 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8880156a60 con 0x7f88800b5ed0 2022-08-01T19:43:56.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.737+0000 7f88727fc700 1 -- 172.21.15.17:0/163808430 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8874025030 con 0x7f88800b5ed0 2022-08-01T19:43:56.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.741+0000 7f88727fc700 1 --2- 172.21.15.17:0/163808430 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f88680539e0 0x7f8868055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:56.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.741+0000 7f88727fc700 1 -- 172.21.15.17:0/163808430 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f887406ce70 con 0x7f88800b5ed0 2022-08-01T19:43:56.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.741+0000 7f8885663700 1 --2- 172.21.15.17:0/163808430 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f88680539e0 0x7f8868055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:56.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.741+0000 7f8885663700 1 --2- 172.21.15.17:0/163808430 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f88680539e0 0x7f8868055ea0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7f8878008000 tx=0x7f8878009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:56.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.745+0000 7f88727fc700 1 -- 172.21.15.17:0/163808430 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8874047e60 con 0x7f88800b5ed0 2022-08-01T19:43:56.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.869+0000 7f8886665700 1 -- 172.21.15.17:0/163808430 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f88800574e0 con 0x7f88800b5ed0 2022-08-01T19:43:56.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.869+0000 7f88727fc700 1 -- 172.21.15.17:0/163808430 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v154) v1 ==== 84+0+143 (secure 0 0 0) 0x7f8874023070 con 0x7f88800b5ed0 2022-08-01T19:43:56.875 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:56.875 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2022-08-01T19:43:56.876 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:56.876 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 1, 2022-08-01T19:43:56.876 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": "device_health_metrics" 2022-08-01T19:43:56.877 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:43:56.877 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:43:56.878 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 2, 2022-08-01T19:43:56.878 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": "rbd" 2022-08-01T19:43:56.879 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:43:56.879 INFO:teuthology.orchestra.run.smithi017.stdout:] 2022-08-01T19:43:56.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.873+0000 7f8886665700 1 -- 172.21.15.17:0/163808430 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f88680539e0 msgr2=0x7f8868055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:56.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.873+0000 7f8886665700 1 --2- 172.21.15.17:0/163808430 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f88680539e0 0x7f8868055ea0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7f8878008000 tx=0x7f8878009040).stop 2022-08-01T19:43:56.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.873+0000 7f8886665700 1 -- 172.21.15.17:0/163808430 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88800b5ed0 msgr2=0x7f888013f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:56.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.873+0000 7f8886665700 1 --2- 172.21.15.17:0/163808430 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88800b5ed0 0x7f888013f1a0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 rx=0x7f8874002f70 tx=0x7f8874015000).stop 2022-08-01T19:43:56.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.873+0000 7f8886665700 1 -- 172.21.15.17:0/163808430 shutdown_connections 2022-08-01T19:43:56.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.873+0000 7f8886665700 1 --2- 172.21.15.17:0/163808430 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f88680539e0 0x7f8868055ea0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:56.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.873+0000 7f8886665700 1 --2- 172.21.15.17:0/163808430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88800b6ee0 0x7f888013f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:56.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.873+0000 7f8886665700 1 --2- 172.21.15.17:0/163808430 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88800b5ed0 0x7f888013f1a0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:56.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.873+0000 7f8886665700 1 --2- 172.21.15.17:0/163808430 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f88800619b0 0x7f8880146570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:56.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.873+0000 7f8886665700 1 -- 172.21.15.17:0/163808430 >> 172.21.15.17:0/163808430 conn(0x7f888006af80 msgr2=0x7f888005ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:56.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.873+0000 7f8886665700 1 -- 172.21.15.17:0/163808430 shutdown_connections 2022-08-01T19:43:56.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.873+0000 7f8886665700 1 -- 172.21.15.17:0/163808430 wait complete. 2022-08-01T19:43:56.889 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-08-01T19:43:56.890 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-08-01T19:43:56.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.997+0000 7fc0c5114700 1 Processor -- start 2022-08-01T19:43:56.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.997+0000 7fc0c5114700 1 -- start start 2022-08-01T19:43:57.032 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.997+0000 7fc0c5114700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fc0c00b2050 0x7fc0c00b1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:57.033 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.997+0000 7fc0c5114700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc0c00b2430 0x7fc0c01413e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:57.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.997+0000 7fc0c5114700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0c00b05b0 con 0x7fc0c00b0140 2022-08-01T19:43:57.034 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.997+0000 7fc0c5114700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0c00a7960 con 0x7fc0c00b2050 2022-08-01T19:43:57.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.997+0000 7fc0c5114700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c00b07e0 con 0x7fc0c00b2430 2022-08-01T19:43:57.035 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.997+0000 7fc0be59c700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc0c00b0140 0x7fc0c00a7250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37726/0 (socket says 172.21.15.17:37726) 2022-08-01T19:43:57.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.997+0000 7fc0be59c700 1 -- 172.21.15.17:0/1913354009 learned_addr learned my addr 172.21.15.17:0/1913354009 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:57.036 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.997+0000 7fc0bed9d700 1 --2- 172.21.15.17:0/1913354009 >> v2:172.21.15.77:3300/0 conn(0x7fc0c00b2050 0x7fc0c00b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:57.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.997+0000 7fc0bf59e700 1 --2- 172.21.15.17:0/1913354009 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc0c00b2430 0x7fc0c01413e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:57.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.997+0000 7fc0bdd9b700 1 -- 172.21.15.17:0/1913354009 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1508743675 0 0) 0x7fc0c00b05b0 con 0x7fc0c00b0140 2022-08-01T19:43:57.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.997+0000 7fc0bdd9b700 1 -- 172.21.15.17:0/1913354009 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0a4003260 con 0x7fc0c00b0140 2022-08-01T19:43:57.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.997+0000 7fc0bdd9b700 1 -- 172.21.15.17:0/1913354009 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1868463778 0 0) 0x7fc0a4003260 con 0x7fc0c00b0140 2022-08-01T19:43:57.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.997+0000 7fc0bdd9b700 1 -- 172.21.15.17:0/1913354009 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc0c00b2430 msgr2=0x7fc0c01413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:57.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.997+0000 7fc0bdd9b700 1 --2- 172.21.15.17:0/1913354009 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc0c00b2430 0x7fc0c01413e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:57.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.997+0000 7fc0bdd9b700 1 -- 172.21.15.17:0/1913354009 >> v2:172.21.15.77:3300/0 conn(0x7fc0c00b2050 msgr2=0x7fc0c00b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:57.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.997+0000 7fc0bdd9b700 1 --2- 172.21.15.17:0/1913354009 >> v2:172.21.15.77:3300/0 conn(0x7fc0c00b2050 0x7fc0c00b1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:57.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.997+0000 7fc0bdd9b700 1 -- 172.21.15.17:0/1913354009 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c00b0a80 con 0x7fc0c00b0140 2022-08-01T19:43:57.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.997+0000 7fc0bdd9b700 1 -- 172.21.15.17:0/1913354009 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7fc0b4001660 con 0x7fc0c00b0140 2022-08-01T19:43:57.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.997+0000 7fc0bdd9b700 1 -- 172.21.15.17:0/1913354009 >> v1:172.21.15.17:6789/0 conn(0x7fc0c00b0140 legacy=0x7fc0c00a7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:57.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.997+0000 7fc0bdd9b700 1 --2- 172.21.15.17:0/1913354009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0a4003750 0x7fc0a4003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:57.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.997+0000 7fc0bdd9b700 1 --2- 172.21.15.17:0/1913354009 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc0c00b2430 0x7fc0a4004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:57.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.997+0000 7fc0bdd9b700 1 --2- 172.21.15.17:0/1913354009 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0a40049f0 0x7fc0a4004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:57.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.997+0000 7fc0bdd9b700 1 -- 172.21.15.17:0/1913354009 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c00a7960 con 0x7fc0a40049f0 2022-08-01T19:43:57.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.997+0000 7fc0bdd9b700 1 -- 172.21.15.17:0/1913354009 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c00b07e0 con 0x7fc0a4003750 2022-08-01T19:43:57.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.997+0000 7fc0bdd9b700 1 -- 172.21.15.17:0/1913354009 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a400b750 con 0x7fc0c00b2430 2022-08-01T19:43:57.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.997+0000 7fc0bf59e700 1 --2- 172.21.15.17:0/1913354009 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0a40049f0 0x7fc0a4004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:57.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.997+0000 7fc0be59c700 1 --2- 172.21.15.17:0/1913354009 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc0c00b2430 0x7fc0a4004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:57.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:56.997+0000 7fc0bed9d700 1 --2- 172.21.15.17:0/1913354009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0a4003750 0x7fc0a4003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:57.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.001+0000 7fc0be59c700 1 -- 172.21.15.17:0/1913354009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0a4003750 msgr2=0x7fc0a4003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:57.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.001+0000 7fc0be59c700 1 --2- 172.21.15.17:0/1913354009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0a4003750 0x7fc0a4003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:57.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.001+0000 7fc0be59c700 1 -- 172.21.15.17:0/1913354009 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0a40049f0 msgr2=0x7fc0a4004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:57.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.001+0000 7fc0be59c700 1 --2- 172.21.15.17:0/1913354009 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0a40049f0 0x7fc0a4004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:57.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.001+0000 7fc0be59c700 1 -- 172.21.15.17:0/1913354009 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a4003260 con 0x7fc0c00b2430 2022-08-01T19:43:57.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.001+0000 7fc0bf59e700 1 --2- 172.21.15.17:0/1913354009 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0a40049f0 0x7fc0a4004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:43:57.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.001+0000 7fc0be59c700 1 --2- 172.21.15.17:0/1913354009 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc0c00b2430 0x7fc0a4004500 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7fc0b4001370 tx=0x7fc0c00a7aa0).ready entity=mon.2 client_cookie=93753bcd1d7f930e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:57.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.001+0000 7fc0bdd9b700 1 -- 172.21.15.17:0/1913354009 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc0b40071a0 con 0x7fc0c00b2430 2022-08-01T19:43:57.060 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.001+0000 7fc0bdd9b700 1 -- 172.21.15.17:0/1913354009 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0b4007300 con 0x7fc0c00b2430 2022-08-01T19:43:57.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.001+0000 7fc0bdd9b700 1 -- 172.21.15.17:0/1913354009 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc0b400cbb0 con 0x7fc0c00b2430 2022-08-01T19:43:57.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.001+0000 7fc0c5114700 1 -- 172.21.15.17:0/1913354009 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc0c00b2430 msgr2=0x7fc0a4004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:57.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.001+0000 7fc0c5114700 1 --2- 172.21.15.17:0/1913354009 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc0c00b2430 0x7fc0a4004500 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7fc0b4001370 tx=0x7fc0c00a7aa0).stop 2022-08-01T19:43:57.061 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.001+0000 7fc0bf59e700 1 -- 172.21.15.17:0/1913354009 reap_dead start 2022-08-01T19:43:57.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.001+0000 7fc0c5114700 1 -- 172.21.15.17:0/1913354009 shutdown_connections 2022-08-01T19:43:57.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.001+0000 7fc0c5114700 1 -- 172.21.15.17:0/1913354009 >> 172.21.15.17:0/1913354009 conn(0x7fc0c005f840 msgr2=0x7fc0c005fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:57.066 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.001+0000 7fc0c5114700 1 -- 172.21.15.17:0/1913354009 shutdown_connections 2022-08-01T19:43:57.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.001+0000 7fc0c5114700 1 -- 172.21.15.17:0/1913354009 wait complete. 2022-08-01T19:43:57.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.001+0000 7fc0c5114700 1 Processor -- start 2022-08-01T19:43:57.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.001+0000 7fc0c5114700 1 -- start start 2022-08-01T19:43:57.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.005+0000 7fc0c5114700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0c00a9b90 0x7fc0c00aaf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:57.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.005+0000 7fc0c5114700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0c00ab450 0x7fc0c015a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:57.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.005+0000 7fc0c5114700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc0c015a7a0 0x7fc0c015ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:57.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.005+0000 7fc0c5114700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c0148840 con 0x7fc0c00a9b90 2022-08-01T19:43:57.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.005+0000 7fc0c5114700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c00aab30 con 0x7fc0c00ab450 2022-08-01T19:43:57.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.005+0000 7fc0c5114700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c015b110 con 0x7fc0c015a7a0 2022-08-01T19:43:57.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.005+0000 7fc0bed9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0c00a9b90 0x7fc0c00aaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:57.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.005+0000 7fc0bed9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0c00a9b90 0x7fc0c00aaf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45542/0 (socket says 172.21.15.17:45542) 2022-08-01T19:43:57.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.005+0000 7fc0bed9d700 1 -- 172.21.15.17:0/1905999721 learned_addr learned my addr 172.21.15.17:0/1905999721 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:57.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.005+0000 7fc0be59c700 1 --2- 172.21.15.17:0/1905999721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0c00ab450 0x7fc0c015a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:57.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.005+0000 7fc0bf59e700 1 --2- 172.21.15.17:0/1905999721 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc0c015a7a0 0x7fc0c015ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:57.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.005+0000 7fc0bed9d700 1 -- 172.21.15.17:0/1905999721 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc0c015a7a0 msgr2=0x7fc0c015ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:57.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.005+0000 7fc0bed9d700 1 --2- 172.21.15.17:0/1905999721 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc0c015a7a0 0x7fc0c015ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:57.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.005+0000 7fc0bed9d700 1 -- 172.21.15.17:0/1905999721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0c00ab450 msgr2=0x7fc0c015a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:57.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.005+0000 7fc0bed9d700 1 --2- 172.21.15.17:0/1905999721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0c00ab450 0x7fc0c015a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:57.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.005+0000 7fc0bed9d700 1 -- 172.21.15.17:0/1905999721 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c00a9f70 con 0x7fc0c00a9b90 2022-08-01T19:43:57.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.005+0000 7fc0bf59e700 1 --2- 172.21.15.17:0/1905999721 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc0c015a7a0 0x7fc0c015ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:43:57.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.005+0000 7fc0bed9d700 1 --2- 172.21.15.17:0/1905999721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0c00a9b90 0x7fc0c00aaf60 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 rx=0x7fc0c00b3b40 tx=0x7fc0b0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:57.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.005+0000 7fc0affff700 1 -- 172.21.15.17:0/1905999721 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc0b000f940 con 0x7fc0c00a9b90 2022-08-01T19:43:57.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.005+0000 7fc0c5114700 1 -- 172.21.15.17:0/1905999721 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0c0149c90 con 0x7fc0c00a9b90 2022-08-01T19:43:57.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.005+0000 7fc0affff700 1 -- 172.21.15.17:0/1905999721 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0b000faa0 con 0x7fc0c00a9b90 2022-08-01T19:43:57.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.005+0000 7fc0affff700 1 -- 172.21.15.17:0/1905999721 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc0b000fe00 con 0x7fc0c00a9b90 2022-08-01T19:43:57.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.005+0000 7fc0c5114700 1 -- 172.21.15.17:0/1905999721 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0c0148fa0 con 0x7fc0c00a9b90 2022-08-01T19:43:57.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.009+0000 7fc0c5114700 1 -- 172.21.15.17:0/1905999721 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0c00b0d20 con 0x7fc0c00a9b90 2022-08-01T19:43:57.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.009+0000 7fc0affff700 1 -- 172.21.15.17:0/1905999721 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc0b0014440 con 0x7fc0c00a9b90 2022-08-01T19:43:57.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.009+0000 7fc0affff700 1 --2- 172.21.15.17:0/1905999721 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc09c053a10 0x7fc09c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:57.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.009+0000 7fc0affff700 1 -- 172.21.15.17:0/1905999721 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fc0b006c900 con 0x7fc0c00a9b90 2022-08-01T19:43:57.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.009+0000 7fc0be59c700 1 --2- 172.21.15.17:0/1905999721 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc09c053a10 0x7fc09c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:57.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.009+0000 7fc0be59c700 1 --2- 172.21.15.17:0/1905999721 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc09c053a10 0x7fc09c055ed0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7fc0b4019000 tx=0x7fc0b4013000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:57.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.013+0000 7fc0affff700 1 -- 172.21.15.17:0/1905999721 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc0b0043810 con 0x7fc0c00a9b90 2022-08-01T19:43:57.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.141+0000 7fc0c5114700 1 -- 172.21.15.17:0/1905999721 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fc0c00574e0 con 0x7fc0c00a9b90 2022-08-01T19:43:57.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.629+0000 7fc0affff700 1 -- 172.21.15.17:0/1905999721 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v155) v1 ==== 133+0+0 (secure 0 0 0) 0x7fc0b0018070 con 0x7fc0c00a9b90 2022-08-01T19:43:57.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.629+0000 7fc0c5114700 1 -- 172.21.15.17:0/1905999721 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc09c053a10 msgr2=0x7fc09c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:57.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.629+0000 7fc0c5114700 1 --2- 172.21.15.17:0/1905999721 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc09c053a10 0x7fc09c055ed0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7fc0b4019000 tx=0x7fc0b4013000).stop 2022-08-01T19:43:57.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.629+0000 7fc0c5114700 1 -- 172.21.15.17:0/1905999721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0c00a9b90 msgr2=0x7fc0c00aaf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:57.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.629+0000 7fc0c5114700 1 --2- 172.21.15.17:0/1905999721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0c00a9b90 0x7fc0c00aaf60 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 rx=0x7fc0c00b3b40 tx=0x7fc0b0006df0).stop 2022-08-01T19:43:57.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.633+0000 7fc0c5114700 1 -- 172.21.15.17:0/1905999721 shutdown_connections 2022-08-01T19:43:57.636 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.633+0000 7fc0c5114700 1 --2- 172.21.15.17:0/1905999721 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc09c053a10 0x7fc09c055ed0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:57.636 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.633+0000 7fc0c5114700 1 --2- 172.21.15.17:0/1905999721 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc0c015a7a0 0x7fc0c015ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:57.637 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.633+0000 7fc0c5114700 1 --2- 172.21.15.17:0/1905999721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0c00ab450 0x7fc0c015a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:57.637 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.633+0000 7fc0c5114700 1 --2- 172.21.15.17:0/1905999721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0c00a9b90 0x7fc0c00aaf60 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:57.637 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.633+0000 7fc0c5114700 1 -- 172.21.15.17:0/1905999721 >> 172.21.15.17:0/1905999721 conn(0x7fc0c005f840 msgr2=0x7fc0c00a81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:57.637 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.633+0000 7fc0c5114700 1 -- 172.21.15.17:0/1905999721 shutdown_connections 2022-08-01T19:43:57.638 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.633+0000 7fc0c5114700 1 -- 172.21.15.17:0/1905999721 wait complete. 2022-08-01T19:43:57.638 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' created 2022-08-01T19:43:57.650 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-08-01T19:43:57.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.765+0000 7efcfc37a700 1 Processor -- start 2022-08-01T19:43:57.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.765+0000 7efcfc37a700 1 -- start start 2022-08-01T19:43:57.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.765+0000 7efcfc37a700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7efcf4146ae0 0x7efcf41403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:57.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.765+0000 7efcfc37a700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efcf40b4820 0x7efcf40b5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:57.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.765+0000 7efcfc37a700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf4143cf0 con 0x7efcf40b4310 2022-08-01T19:43:57.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.765+0000 7efcfc37a700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcf41476a0 con 0x7efcf40b4820 2022-08-01T19:43:57.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.765+0000 7efcfc37a700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcf4143f90 con 0x7efcf4146ae0 2022-08-01T19:43:57.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.765+0000 7efcfa917700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efcf40b4820 0x7efcf40b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:57.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.765+0000 7efcfa917700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efcf40b4820 0x7efcf40b5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45552/0 (socket says 172.21.15.17:45552) 2022-08-01T19:43:57.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.765+0000 7efcfa917700 1 -- 172.21.15.17:0/1881998773 learned_addr learned my addr 172.21.15.17:0/1881998773 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:57.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.765+0000 7efcfa116700 1 --2- 172.21.15.17:0/1881998773 >> v2:172.21.15.77:3300/0 conn(0x7efcf4146ae0 0x7efcf41403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:57.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.765+0000 7efcfa917700 1 -- 172.21.15.17:0/1881998773 >> v2:172.21.15.77:3300/0 conn(0x7efcf4146ae0 msgr2=0x7efcf41403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:57.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.765+0000 7efcfa917700 1 --2- 172.21.15.17:0/1881998773 >> v2:172.21.15.77:3300/0 conn(0x7efcf4146ae0 0x7efcf41403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:57.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.765+0000 7efcfa917700 1 -- 172.21.15.17:0/1881998773 >> v1:172.21.15.77:6789/0 conn(0x7efcf40b4310 legacy=0x7efcf41408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:57.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.769+0000 7efcfa917700 1 -- 172.21.15.17:0/1881998773 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf40b0e20 con 0x7efcf40b4820 2022-08-01T19:43:57.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.769+0000 7efcfa917700 1 --2- 172.21.15.17:0/1881998773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf40b4820 0x7efcf40b5060 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 rx=0x7efcec0032f0 tx=0x7efcec009290).ready entity=mon.0 client_cookie=d49326e8bf75e4e4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:57.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.769+0000 7efcf9114700 1 -- 172.21.15.17:0/1881998773 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efcec00b870 con 0x7efcf40b4820 2022-08-01T19:43:57.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.769+0000 7efcf9114700 1 -- 172.21.15.17:0/1881998773 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcec00b9d0 con 0x7efcf40b4820 2022-08-01T19:43:57.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.769+0000 7efcf9114700 1 -- 172.21.15.17:0/1881998773 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efcec00bb40 con 0x7efcf40b4820 2022-08-01T19:43:57.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.769+0000 7efcfc37a700 1 -- 172.21.15.17:0/1881998773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf40b4820 msgr2=0x7efcf40b5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:57.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.769+0000 7efcfc37a700 1 --2- 172.21.15.17:0/1881998773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf40b4820 0x7efcf40b5060 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 rx=0x7efcec0032f0 tx=0x7efcec009290).stop 2022-08-01T19:43:57.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.769+0000 7efcfc37a700 1 -- 172.21.15.17:0/1881998773 shutdown_connections 2022-08-01T19:43:57.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.769+0000 7efcfc37a700 1 --2- 172.21.15.17:0/1881998773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf40b4820 0x7efcf40b5060 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:57.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.769+0000 7efcfc37a700 1 --2- 172.21.15.17:0/1881998773 >> v2:172.21.15.77:3300/0 conn(0x7efcf4146ae0 0x7efcf41403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:57.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.769+0000 7efcfc37a700 1 -- 172.21.15.17:0/1881998773 >> 172.21.15.17:0/1881998773 conn(0x7efcf405c620 msgr2=0x7efcf405ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:57.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.769+0000 7efcfc37a700 1 -- 172.21.15.17:0/1881998773 shutdown_connections 2022-08-01T19:43:57.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.769+0000 7efcfc37a700 1 -- 172.21.15.17:0/1881998773 wait complete. 2022-08-01T19:43:57.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.769+0000 7efcfc37a700 1 Processor -- start 2022-08-01T19:43:57.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.769+0000 7efcfc37a700 1 -- start start 2022-08-01T19:43:57.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.773+0000 7efcfc37a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcf40b4310 0x7efcf40ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:57.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.773+0000 7efcfc37a700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efcf40b4820 0x7efcf4152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:57.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.773+0000 7efcfc37a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf4146ae0 0x7efcf40afa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:57.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.773+0000 7efcfc37a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf40b7a70 con 0x7efcf4146ae0 2022-08-01T19:43:57.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.773+0000 7efcfc37a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf40b87d0 con 0x7efcf40b4310 2022-08-01T19:43:57.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.773+0000 7efcfc37a700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf40b8a70 con 0x7efcf40b4820 2022-08-01T19:43:57.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.773+0000 7efcfa917700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf4146ae0 0x7efcf40afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:57.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.773+0000 7efcfa917700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf4146ae0 0x7efcf40afa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45560/0 (socket says 172.21.15.17:45560) 2022-08-01T19:43:57.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.773+0000 7efcfa917700 1 -- 172.21.15.17:0/3719681248 learned_addr learned my addr 172.21.15.17:0/3719681248 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:57.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.773+0000 7efcfa116700 1 --2- 172.21.15.17:0/3719681248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcf40b4310 0x7efcf40ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:57.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.773+0000 7efcf9915700 1 --2- 172.21.15.17:0/3719681248 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efcf40b4820 0x7efcf4152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:57.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.773+0000 7efcfa917700 1 -- 172.21.15.17:0/3719681248 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efcf40b4820 msgr2=0x7efcf4152e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:57.802 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.773+0000 7efcfa917700 1 --2- 172.21.15.17:0/3719681248 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efcf40b4820 0x7efcf4152e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:57.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.773+0000 7efcfa917700 1 -- 172.21.15.17:0/3719681248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcf40b4310 msgr2=0x7efcf40ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:57.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.773+0000 7efcfa917700 1 --2- 172.21.15.17:0/3719681248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcf40b4310 0x7efcf40ade50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:57.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.773+0000 7efcfa917700 1 -- 172.21.15.17:0/3719681248 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcec009530 con 0x7efcf4146ae0 2022-08-01T19:43:57.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.773+0000 7efcfa917700 1 --2- 172.21.15.17:0/3719681248 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf4146ae0 0x7efcf40afa50 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 rx=0x7efcec0032f0 tx=0x7efcec016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:57.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.773+0000 7efce37fe700 1 -- 172.21.15.17:0/3719681248 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efcec00b870 con 0x7efcf4146ae0 2022-08-01T19:43:57.804 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.773+0000 7efce37fe700 1 -- 172.21.15.17:0/3719681248 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcec021440 con 0x7efcf4146ae0 2022-08-01T19:43:57.804 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.773+0000 7efcfc37a700 1 -- 172.21.15.17:0/3719681248 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcf40b8d10 con 0x7efcf4146ae0 2022-08-01T19:43:57.804 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.773+0000 7efce37fe700 1 -- 172.21.15.17:0/3719681248 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efcec0217a0 con 0x7efcf4146ae0 2022-08-01T19:43:57.804 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.773+0000 7efcfc37a700 1 -- 172.21.15.17:0/3719681248 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcf4153360 con 0x7efcf4146ae0 2022-08-01T19:43:57.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.773+0000 7efcfc37a700 1 -- 172.21.15.17:0/3719681248 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcf40574e0 con 0x7efcf4146ae0 2022-08-01T19:43:57.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.777+0000 7efce37fe700 1 -- 172.21.15.17:0/3719681248 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7efcec027020 con 0x7efcf4146ae0 2022-08-01T19:43:57.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.777+0000 7efce37fe700 1 --2- 172.21.15.17:0/3719681248 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efcd40539e0 0x7efcd4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:57.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.777+0000 7efce37fe700 1 -- 172.21.15.17:0/3719681248 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5954+0+0 (secure 0 0 0) 0x7efcec06d960 con 0x7efcf4146ae0 2022-08-01T19:43:57.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.777+0000 7efcfa116700 1 --2- 172.21.15.17:0/3719681248 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efcd40539e0 0x7efcd4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:57.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.777+0000 7efcfa116700 1 --2- 172.21.15.17:0/3719681248 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efcd40539e0 0x7efcd4055ea0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7efcf000c000 tx=0x7efcf0008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:57.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.781+0000 7efce37fe700 1 -- 172.21.15.17:0/3719681248 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efcec044820 con 0x7efcf4146ae0 2022-08-01T19:43:57.910 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:57.905+0000 7efcfc37a700 1 -- 172.21.15.17:0/3719681248 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7efcf4069a70 con 0x7efcf4146ae0 2022-08-01T19:43:58.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.385+0000 7efce37fe700 1 -- 172.21.15.17:0/3719681248 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v156) v1 ==== 125+0+0 (secure 0 0 0) 0x7efcec015020 con 0x7efcf4146ae0 2022-08-01T19:43:58.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.385+0000 7efcfc37a700 1 -- 172.21.15.17:0/3719681248 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efcd40539e0 msgr2=0x7efcd4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:58.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.385+0000 7efcfc37a700 1 --2- 172.21.15.17:0/3719681248 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efcd40539e0 0x7efcd4055ea0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7efcf000c000 tx=0x7efcf0008040).stop 2022-08-01T19:43:58.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.385+0000 7efcfc37a700 1 -- 172.21.15.17:0/3719681248 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf4146ae0 msgr2=0x7efcf40afa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:58.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.385+0000 7efcfc37a700 1 --2- 172.21.15.17:0/3719681248 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf4146ae0 0x7efcf40afa50 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 rx=0x7efcec0032f0 tx=0x7efcec016000).stop 2022-08-01T19:43:58.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.389+0000 7efcfc37a700 1 -- 172.21.15.17:0/3719681248 shutdown_connections 2022-08-01T19:43:58.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.389+0000 7efcfc37a700 1 --2- 172.21.15.17:0/3719681248 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efcd40539e0 0x7efcd4055ea0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:58.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.389+0000 7efcfc37a700 1 --2- 172.21.15.17:0/3719681248 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf4146ae0 0x7efcf40afa50 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:58.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.389+0000 7efcfc37a700 1 --2- 172.21.15.17:0/3719681248 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efcf40b4820 0x7efcf4152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:58.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.389+0000 7efcfc37a700 1 --2- 172.21.15.17:0/3719681248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efcf40b4310 0x7efcf40ade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:58.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.389+0000 7efcfc37a700 1 -- 172.21.15.17:0/3719681248 >> 172.21.15.17:0/3719681248 conn(0x7efcf405c620 msgr2=0x7efcf413e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:58.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.389+0000 7efcfc37a700 1 -- 172.21.15.17:0/3719681248 shutdown_connections 2022-08-01T19:43:58.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.389+0000 7efcfc37a700 1 -- 172.21.15.17:0/3719681248 wait complete. 2022-08-01T19:43:58.393 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' created 2022-08-01T19:43:58.410 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-08-01T19:43:58.530 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.525+0000 7f4a3afbb700 1 Processor -- start 2022-08-01T19:43:58.530 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.529+0000 7f4a3afbb700 1 -- start start 2022-08-01T19:43:58.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.529+0000 7f4a3afbb700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4a340a9da0 0x7f4a34060950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:58.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.529+0000 7f4a3afbb700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4a341402d0 0x7f4a34140760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:58.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.529+0000 7f4a3afbb700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a34067030 con 0x7f4a340b31d0 2022-08-01T19:43:58.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.529+0000 7f4a3afbb700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a34064a70 con 0x7f4a340a9da0 2022-08-01T19:43:58.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.529+0000 7f4a3afbb700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a34061f60 con 0x7f4a341402d0 2022-08-01T19:43:58.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.529+0000 7f4a397b8700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4a340a9da0 0x7f4a34060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:58.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.529+0000 7f4a39fb9700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f4a340b31d0 0x7f4a34066020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37758/0 (socket says 172.21.15.17:37758) 2022-08-01T19:43:58.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.529+0000 7f4a397b8700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4a340a9da0 0x7f4a34060950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45566/0 (socket says 172.21.15.17:45566) 2022-08-01T19:43:58.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.529+0000 7f4a39fb9700 1 -- 172.21.15.17:0/3579188785 learned_addr learned my addr 172.21.15.17:0/3579188785 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:58.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.529+0000 7f4a3a7ba700 1 --2- 172.21.15.17:0/3579188785 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4a341402d0 0x7f4a34140760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:58.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.529+0000 7f4a397b8700 1 -- 172.21.15.17:0/3579188785 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4a341402d0 msgr2=0x7f4a34140760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:58.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.529+0000 7f4a397b8700 1 --2- 172.21.15.17:0/3579188785 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4a341402d0 0x7f4a34140760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:58.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.529+0000 7f4a397b8700 1 -- 172.21.15.17:0/3579188785 >> v1:172.21.15.17:6789/0 conn(0x7f4a340b31d0 legacy=0x7f4a34066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:58.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.529+0000 7f4a397b8700 1 -- 172.21.15.17:0/3579188785 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a340b35b0 con 0x7f4a340a9da0 2022-08-01T19:43:58.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.529+0000 7f4a397b8700 1 --2- 172.21.15.17:0/3579188785 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a340a9da0 0x7f4a34060950 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 rx=0x7f4a34063230 tx=0x7f4a28006df0).ready entity=mon.0 client_cookie=dcb37b2b23ec4ff3 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:58.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.529+0000 7f4a38fb7700 1 -- 172.21.15.17:0/3579188785 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a2800d570 con 0x7f4a340a9da0 2022-08-01T19:43:58.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.529+0000 7f4a38fb7700 1 -- 172.21.15.17:0/3579188785 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a28000e90 con 0x7f4a340a9da0 2022-08-01T19:43:58.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.529+0000 7f4a38fb7700 1 -- 172.21.15.17:0/3579188785 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a2800d8c0 con 0x7f4a340a9da0 2022-08-01T19:43:58.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.529+0000 7f4a3afbb700 1 -- 172.21.15.17:0/3579188785 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a340a9da0 msgr2=0x7f4a34060950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:58.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.529+0000 7f4a3afbb700 1 --2- 172.21.15.17:0/3579188785 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a340a9da0 0x7f4a34060950 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 rx=0x7f4a34063230 tx=0x7f4a28006df0).stop 2022-08-01T19:43:58.538 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.529+0000 7f4a3afbb700 1 -- 172.21.15.17:0/3579188785 shutdown_connections 2022-08-01T19:43:58.538 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.529+0000 7f4a3afbb700 1 --2- 172.21.15.17:0/3579188785 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4a341402d0 0x7f4a34140760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:58.538 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.529+0000 7f4a3afbb700 1 --2- 172.21.15.17:0/3579188785 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a340a9da0 0x7f4a34060950 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:58.538 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.529+0000 7f4a3afbb700 1 -- 172.21.15.17:0/3579188785 >> 172.21.15.17:0/3579188785 conn(0x7f4a34063870 msgr2=0x7f4a34063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:58.539 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.529+0000 7f4a3afbb700 1 -- 172.21.15.17:0/3579188785 shutdown_connections 2022-08-01T19:43:58.539 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.529+0000 7f4a3afbb700 1 -- 172.21.15.17:0/3579188785 wait complete. 2022-08-01T19:43:58.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.533+0000 7f4a3afbb700 1 Processor -- start 2022-08-01T19:43:58.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.533+0000 7f4a3afbb700 1 -- start start 2022-08-01T19:43:58.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.533+0000 7f4a3afbb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a340a9da0 0x7f4a3412df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:58.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.533+0000 7f4a3afbb700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4a340b31d0 0x7f4a34136c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:58.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.533+0000 7f4a3afbb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a341402d0 0x7f4a34134840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:58.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.533+0000 7f4a3afbb700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a340b3d40 con 0x7f4a340a9da0 2022-08-01T19:43:58.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.533+0000 7f4a3afbb700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a340b5ce0 con 0x7f4a341402d0 2022-08-01T19:43:58.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.533+0000 7f4a3afbb700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a34142300 con 0x7f4a340b31d0 2022-08-01T19:43:58.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.533+0000 7f4a39fb9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a340a9da0 0x7f4a3412df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:58.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.533+0000 7f4a397b8700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4a340b31d0 0x7f4a34136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:58.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.533+0000 7f4a39fb9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a340a9da0 0x7f4a3412df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45572/0 (socket says 172.21.15.17:45572) 2022-08-01T19:43:58.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.533+0000 7f4a39fb9700 1 -- 172.21.15.17:0/3074547691 learned_addr learned my addr 172.21.15.17:0/3074547691 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:58.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.533+0000 7f4a3a7ba700 1 --2- 172.21.15.17:0/3074547691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a341402d0 0x7f4a34134840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:58.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.533+0000 7f4a39fb9700 1 -- 172.21.15.17:0/3074547691 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4a340b31d0 msgr2=0x7f4a34136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:58.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.533+0000 7f4a39fb9700 1 --2- 172.21.15.17:0/3074547691 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4a340b31d0 0x7f4a34136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:58.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.533+0000 7f4a39fb9700 1 -- 172.21.15.17:0/3074547691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a341402d0 msgr2=0x7f4a34134840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:58.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.533+0000 7f4a39fb9700 1 --2- 172.21.15.17:0/3074547691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a341402d0 0x7f4a34134840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:58.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.533+0000 7f4a39fb9700 1 -- 172.21.15.17:0/3074547691 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a2800b180 con 0x7f4a340a9da0 2022-08-01T19:43:58.547 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.537+0000 7f4a39fb9700 1 --2- 172.21.15.17:0/3074547691 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a340a9da0 0x7f4a3412df50 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 rx=0x7f4a340634e0 tx=0x7f4a30007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:58.547 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.537+0000 7f4a26ffd700 1 -- 172.21.15.17:0/3074547691 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a300070d0 con 0x7f4a340a9da0 2022-08-01T19:43:58.547 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.537+0000 7f4a26ffd700 1 -- 172.21.15.17:0/3074547691 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a30007230 con 0x7f4a340a9da0 2022-08-01T19:43:58.547 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.537+0000 7f4a3afbb700 1 -- 172.21.15.17:0/3074547691 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a341425a0 con 0x7f4a340a9da0 2022-08-01T19:43:58.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.537+0000 7f4a26ffd700 1 -- 172.21.15.17:0/3074547691 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a30005610 con 0x7f4a340a9da0 2022-08-01T19:43:58.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.537+0000 7f4a3afbb700 1 -- 172.21.15.17:0/3074547691 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a341343a0 con 0x7f4a340a9da0 2022-08-01T19:43:58.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.537+0000 7f4a3afbb700 1 -- 172.21.15.17:0/3074547691 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a34066af0 con 0x7f4a340a9da0 2022-08-01T19:43:58.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.537+0000 7f4a26ffd700 1 -- 172.21.15.17:0/3074547691 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4a30010070 con 0x7f4a340a9da0 2022-08-01T19:43:58.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.537+0000 7f4a26ffd700 1 --2- 172.21.15.17:0/3074547691 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4a14053980 0x7f4a14055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:58.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.541+0000 7f4a26ffd700 1 -- 172.21.15.17:0/3074547691 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f4a3006b5a0 con 0x7f4a340a9da0 2022-08-01T19:43:58.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.541+0000 7f4a397b8700 1 --2- 172.21.15.17:0/3074547691 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4a14053980 0x7f4a14055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:58.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.541+0000 7f4a397b8700 1 --2- 172.21.15.17:0/3074547691 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4a14053980 0x7f4a14055e40 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f4a2801c000 tx=0x7f4a28015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:58.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.545+0000 7f4a26ffd700 1 -- 172.21.15.17:0/3074547691 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4a300422d0 con 0x7f4a340a9da0 2022-08-01T19:43:58.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.709+0000 7f4a3afbb700 1 -- 172.21.15.17:0/3074547691 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f4a340574e0 con 0x7f4a340a9da0 2022-08-01T19:43:58.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.717+0000 7f4a26ffd700 1 -- 172.21.15.17:0/3074547691 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 29 and data pool 30 v82) v1 ==== 189+0+0 (secure 0 0 0) 0x7f4a300458f0 con 0x7f4a340a9da0 2022-08-01T19:43:58.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.721+0000 7f4a3afbb700 1 -- 172.21.15.17:0/3074547691 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4a14053980 msgr2=0x7f4a14055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:58.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.721+0000 7f4a3afbb700 1 --2- 172.21.15.17:0/3074547691 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4a14053980 0x7f4a14055e40 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f4a2801c000 tx=0x7f4a28015000).stop 2022-08-01T19:43:58.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.721+0000 7f4a3afbb700 1 -- 172.21.15.17:0/3074547691 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a340a9da0 msgr2=0x7f4a3412df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:58.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.721+0000 7f4a3afbb700 1 --2- 172.21.15.17:0/3074547691 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a340a9da0 0x7f4a3412df50 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 rx=0x7f4a340634e0 tx=0x7f4a30007c40).stop 2022-08-01T19:43:58.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.721+0000 7f4a3afbb700 1 -- 172.21.15.17:0/3074547691 shutdown_connections 2022-08-01T19:43:58.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.721+0000 7f4a3afbb700 1 --2- 172.21.15.17:0/3074547691 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f4a14053980 0x7f4a14055e40 secure :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 rx=0x7f4a2801c000 tx=0x7f4a28015000).stop 2022-08-01T19:43:58.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.721+0000 7f4a3afbb700 1 --2- 172.21.15.17:0/3074547691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a341402d0 0x7f4a34134840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:58.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.721+0000 7f4a3afbb700 1 --2- 172.21.15.17:0/3074547691 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f4a340b31d0 0x7f4a34136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:58.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.721+0000 7f4a3afbb700 1 --2- 172.21.15.17:0/3074547691 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a340a9da0 0x7f4a3412df50 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:58.729 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.721+0000 7f4a3afbb700 1 -- 172.21.15.17:0/3074547691 >> 172.21.15.17:0/3074547691 conn(0x7f4a34063870 msgr2=0x7f4a340b0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:58.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.725+0000 7f4a3afbb700 1 -- 172.21.15.17:0/3074547691 shutdown_connections 2022-08-01T19:43:58.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.725+0000 7f4a3afbb700 1 -- 172.21.15.17:0/3074547691 wait complete. 2022-08-01T19:43:58.732 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 29 and data pool 30 2022-08-01T19:43:58.741 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:43:58.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.829+0000 7f72f5319700 1 Processor -- start 2022-08-01T19:43:58.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.829+0000 7f72f5319700 1 -- start start 2022-08-01T19:43:58.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.829+0000 7f72f5319700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f72f00b0e60 0x7f72f00610d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:58.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.829+0000 7f72f5319700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72f0066af0 con 0x7f72f013d9b0 2022-08-01T19:43:58.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.829+0000 7f72f5319700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72f0066d90 con 0x7f72f00b6510 2022-08-01T19:43:58.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.829+0000 7f72f5319700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72f0067030 con 0x7f72f00b0e60 2022-08-01T19:43:58.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.829+0000 7f72f4b18700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f72f00b0e60 0x7f72f00610d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:58.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.829+0000 7f72f4b18700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f72f00b0e60 0x7f72f00610d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45588/0 (socket says 172.21.15.17:45588) 2022-08-01T19:43:58.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.829+0000 7f72ef7fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f72f013d9b0 0x7f72f00609c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37780/0 (socket says 172.21.15.17:37780) 2022-08-01T19:43:58.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.829+0000 7f72f4b18700 1 -- 172.21.15.17:0/4106782043 learned_addr learned my addr 172.21.15.17:0/4106782043 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:58.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.833+0000 7f72f4b18700 1 -- 172.21.15.17:0/4106782043 >> v1:172.21.15.77:6789/0 conn(0x7f72f00b6510 legacy=0x7f72f013d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:58.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.833+0000 7f72f4b18700 1 -- 172.21.15.17:0/4106782043 >> v1:172.21.15.17:6789/0 conn(0x7f72f013d9b0 legacy=0x7f72f00609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:58.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.833+0000 7f72eeffd700 1 -- 172.21.15.17:0/4106782043 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1702500711 0 0) 0x7f72f0066af0 con 0x7f72f013d9b0 2022-08-01T19:43:58.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.833+0000 7f72f4b18700 1 -- 172.21.15.17:0/4106782043 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f00672d0 con 0x7f72f00b0e60 2022-08-01T19:43:58.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.833+0000 7f72f4b18700 1 --2- 172.21.15.17:0/4106782043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72f00b0e60 0x7f72f00610d0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 rx=0x7f72e8002f70 tx=0x7f72e8006eb0).ready entity=mon.0 client_cookie=b01874526a9a992b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:58.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.833+0000 7f72eeffd700 1 -- 172.21.15.17:0/4106782043 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72e800d660 con 0x7f72f00b0e60 2022-08-01T19:43:58.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.833+0000 7f72eeffd700 1 -- 172.21.15.17:0/4106782043 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e8000e90 con 0x7f72f00b0e60 2022-08-01T19:43:58.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.833+0000 7f72eeffd700 1 -- 172.21.15.17:0/4106782043 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72e800d9b0 con 0x7f72f00b0e60 2022-08-01T19:43:58.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.833+0000 7f72f5319700 1 -- 172.21.15.17:0/4106782043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72f00b0e60 msgr2=0x7f72f00610d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:58.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.833+0000 7f72f5319700 1 --2- 172.21.15.17:0/4106782043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72f00b0e60 0x7f72f00610d0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 rx=0x7f72e8002f70 tx=0x7f72e8006eb0).stop 2022-08-01T19:43:58.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.833+0000 7f72f5319700 1 -- 172.21.15.17:0/4106782043 shutdown_connections 2022-08-01T19:43:58.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.833+0000 7f72f5319700 1 --2- 172.21.15.17:0/4106782043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72f00b0e60 0x7f72f00610d0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:58.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.833+0000 7f72f5319700 1 -- 172.21.15.17:0/4106782043 >> 172.21.15.17:0/4106782043 conn(0x7f72f0063800 msgr2=0x7f72f0063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:58.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.833+0000 7f72f5319700 1 -- 172.21.15.17:0/4106782043 shutdown_connections 2022-08-01T19:43:58.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.833+0000 7f72f5319700 1 -- 172.21.15.17:0/4106782043 wait complete. 2022-08-01T19:43:58.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.837+0000 7f72f5319700 1 Processor -- start 2022-08-01T19:43:58.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.837+0000 7f72f5319700 1 -- start start 2022-08-01T19:43:58.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.837+0000 7f72f5319700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72f00b0e60 0x7f72f014fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:58.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.837+0000 7f72f5319700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f72f00b6510 0x7f72f00aa570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:58.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.837+0000 7f72f5319700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72f013d9b0 0x7f72f00ab4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:58.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.837+0000 7f72f5319700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f00ac890 con 0x7f72f00b0e60 2022-08-01T19:43:58.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.837+0000 7f72f5319700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f00b05b0 con 0x7f72f013d9b0 2022-08-01T19:43:58.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.837+0000 7f72f5319700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72f0143e60 con 0x7f72f00b6510 2022-08-01T19:43:58.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.837+0000 7f72effff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72f00b0e60 0x7f72f014fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:58.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.837+0000 7f72effff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72f00b0e60 0x7f72f014fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45590/0 (socket says 172.21.15.17:45590) 2022-08-01T19:43:58.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.837+0000 7f72effff700 1 -- 172.21.15.17:0/1114492131 learned_addr learned my addr 172.21.15.17:0/1114492131 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:58.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.837+0000 7f72ef7fe700 1 --2- 172.21.15.17:0/1114492131 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f72f00b6510 0x7f72f00aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:58.853 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.837+0000 7f72f4b18700 1 --2- 172.21.15.17:0/1114492131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72f013d9b0 0x7f72f00ab4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:58.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.837+0000 7f72effff700 1 -- 172.21.15.17:0/1114492131 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f72f00b6510 msgr2=0x7f72f00aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:58.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.837+0000 7f72effff700 1 --2- 172.21.15.17:0/1114492131 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f72f00b6510 0x7f72f00aa570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:58.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.837+0000 7f72effff700 1 -- 172.21.15.17:0/1114492131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72f013d9b0 msgr2=0x7f72f00ab4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:58.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.837+0000 7f72effff700 1 --2- 172.21.15.17:0/1114492131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72f013d9b0 0x7f72f00ab4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:58.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.837+0000 7f72effff700 1 -- 172.21.15.17:0/1114492131 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e800b1e0 con 0x7f72f00b0e60 2022-08-01T19:43:58.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.837+0000 7f72effff700 1 --2- 172.21.15.17:0/1114492131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72f00b0e60 0x7f72f014fd40 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 rx=0x7f72f00b06f0 tx=0x7f72e000c300).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:58.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.837+0000 7f72ed7fa700 1 -- 172.21.15.17:0/1114492131 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72e0001b10 con 0x7f72f00b0e60 2022-08-01T19:43:58.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.841+0000 7f72ed7fa700 1 -- 172.21.15.17:0/1114492131 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e0001c70 con 0x7f72f00b0e60 2022-08-01T19:43:58.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.841+0000 7f72f5319700 1 -- 172.21.15.17:0/1114492131 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f0143960 con 0x7f72f00b0e60 2022-08-01T19:43:58.858 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.841+0000 7f72ed7fa700 1 -- 172.21.15.17:0/1114492131 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72e0001fd0 con 0x7f72f00b0e60 2022-08-01T19:43:58.858 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.841+0000 7f72f5319700 1 -- 172.21.15.17:0/1114492131 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f0150230 con 0x7f72f00b0e60 2022-08-01T19:43:58.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.841+0000 7f72f5319700 1 -- 172.21.15.17:0/1114492131 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72f00aa200 con 0x7f72f00b0e60 2022-08-01T19:43:58.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.841+0000 7f72ed7fa700 1 -- 172.21.15.17:0/1114492131 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f72e0021070 con 0x7f72f00b0e60 2022-08-01T19:43:58.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.841+0000 7f72ed7fa700 1 --2- 172.21.15.17:0/1114492131 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f72d4053b00 0x7f72d4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:58.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.841+0000 7f72ed7fa700 1 -- 172.21.15.17:0/1114492131 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f72e006e710 con 0x7f72f00b0e60 2022-08-01T19:43:58.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.841+0000 7f72ef7fe700 1 --2- 172.21.15.17:0/1114492131 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f72d4053b00 0x7f72d4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:58.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.845+0000 7f72ef7fe700 1 --2- 172.21.15.17:0/1114492131 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f72d4053b00 0x7f72d4055fc0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7f72d800d000 tx=0x7f72d80079e0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:58.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.849+0000 7f72ed7fa700 1 -- 172.21.15.17:0/1114492131 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f72e00454d0 con 0x7f72f00b0e60 2022-08-01T19:43:58.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.977+0000 7f72f5319700 1 -- 172.21.15.17:0/1114492131 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72f00574e0 con 0x7f72f00b0e60 2022-08-01T19:43:59.063 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.977+0000 7f72ed7fa700 1 -- 172.21.15.17:0/1114492131 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) v1 ==== 74+0+16311 (secure 0 0 0) 0x7f72e0048af0 con 0x7f72f00b0e60 2022-08-01T19:43:59.063 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:59.064 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":157,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:43:58.712859+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:43:57.196125+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2022-08-01T19:43:57.958027+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:43:59.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.981+0000 7f72f5319700 1 -- 172.21.15.17:0/1114492131 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f72d4053b00 msgr2=0x7f72d4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:59.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.981+0000 7f72f5319700 1 --2- 172.21.15.17:0/1114492131 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f72d4053b00 0x7f72d4055fc0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7f72d800d000 tx=0x7f72d80079e0).stop 2022-08-01T19:43:59.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.981+0000 7f72f5319700 1 -- 172.21.15.17:0/1114492131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72f00b0e60 msgr2=0x7f72f014fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:59.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.981+0000 7f72f5319700 1 --2- 172.21.15.17:0/1114492131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72f00b0e60 0x7f72f014fd40 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 rx=0x7f72f00b06f0 tx=0x7f72e000c300).stop 2022-08-01T19:43:59.069 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.981+0000 7f72f5319700 1 -- 172.21.15.17:0/1114492131 shutdown_connections 2022-08-01T19:43:59.069 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.981+0000 7f72f5319700 1 --2- 172.21.15.17:0/1114492131 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f72d4053b00 0x7f72d4055fc0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:59.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.981+0000 7f72f5319700 1 --2- 172.21.15.17:0/1114492131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72f013d9b0 0x7f72f00ab4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:59.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.981+0000 7f72f5319700 1 --2- 172.21.15.17:0/1114492131 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f72f00b6510 0x7f72f00aa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:59.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.981+0000 7f72f5319700 1 --2- 172.21.15.17:0/1114492131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72f00b0e60 0x7f72f014fd40 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:59.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.981+0000 7f72f5319700 1 -- 172.21.15.17:0/1114492131 >> 172.21.15.17:0/1114492131 conn(0x7f72f0063800 msgr2=0x7f72f0063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:59.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.981+0000 7f72f5319700 1 -- 172.21.15.17:0/1114492131 shutdown_connections 2022-08-01T19:43:59.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:58.981+0000 7f72f5319700 1 -- 172.21.15.17:0/1114492131 wait complete. 2022-08-01T19:43:59.074 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:43:59.184 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.181+0000 7fb2fe713700 1 Processor -- start 2022-08-01T19:43:59.184 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.181+0000 7fb2fe713700 1 -- start start 2022-08-01T19:43:59.188 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.181+0000 7fb2fe713700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb2f80b6a90 0x7fb2f80acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:59.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.181+0000 7fb2fe713700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fb2f80aa760 0x7fb2f813c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:59.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.185+0000 7fb2fe713700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2f80b6e70 con 0x7fb2f8060aa0 2022-08-01T19:43:59.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.185+0000 7fb2fe713700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2f8061370 con 0x7fb2f80aa760 2022-08-01T19:43:59.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.185+0000 7fb2fe713700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f8061610 con 0x7fb2f80b6a90 2022-08-01T19:43:59.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.185+0000 7fb2f77fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb2f8060aa0 0x7fb2f80aa050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37794/0 (socket says 172.21.15.17:37794) 2022-08-01T19:43:59.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.185+0000 7fb2f77fe700 1 -- 172.21.15.17:0/3887631923 learned_addr learned my addr 172.21.15.17:0/3887631923 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:59.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.185+0000 7fb2f7fff700 1 --2- 172.21.15.17:0/3887631923 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb2f80b6a90 0x7fb2f80acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:59.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.185+0000 7fb2fccb0700 1 --2- 172.21.15.17:0/3887631923 >> v2:172.21.15.77:3300/0 conn(0x7fb2f80aa760 0x7fb2f813c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:59.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.185+0000 7fb2f6ffd700 1 -- 172.21.15.17:0/3887631923 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2338745068 0 0) 0x7fb2f80b6e70 con 0x7fb2f8060aa0 2022-08-01T19:43:59.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.185+0000 7fb2f6ffd700 1 -- 172.21.15.17:0/3887631923 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2d8003260 con 0x7fb2f8060aa0 2022-08-01T19:43:59.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.185+0000 7fb2fccb0700 1 -- 172.21.15.17:0/3887631923 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb2f80b6a90 msgr2=0x7fb2f80acee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:59.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.185+0000 7fb2fccb0700 1 --2- 172.21.15.17:0/3887631923 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb2f80b6a90 0x7fb2f80acee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:59.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.185+0000 7fb2fccb0700 1 -- 172.21.15.17:0/3887631923 >> v1:172.21.15.17:6789/0 conn(0x7fb2f8060aa0 legacy=0x7fb2f80aa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:59.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.185+0000 7fb2fccb0700 1 -- 172.21.15.17:0/3887631923 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f80618b0 con 0x7fb2f80aa760 2022-08-01T19:43:59.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.185+0000 7fb2f7fff700 1 --2- 172.21.15.17:0/3887631923 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb2f80b6a90 0x7fb2f80acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:43:59.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.185+0000 7fb2fccb0700 1 --2- 172.21.15.17:0/3887631923 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2f80aa760 0x7fb2f813c600 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7fb2f00032f0 tx=0x7fb2f00091d0).ready entity=mon.1 client_cookie=8ef6fa6cb58c470d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:59.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.185+0000 7fb2f6ffd700 1 -- 172.21.15.17:0/3887631923 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb2f0010810 con 0x7fb2f80aa760 2022-08-01T19:43:59.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.185+0000 7fb2f6ffd700 1 -- 172.21.15.17:0/3887631923 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f0010970 con 0x7fb2f80aa760 2022-08-01T19:43:59.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.185+0000 7fb2f6ffd700 1 -- 172.21.15.17:0/3887631923 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb2f0010cd0 con 0x7fb2f80aa760 2022-08-01T19:43:59.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.185+0000 7fb2fe713700 1 -- 172.21.15.17:0/3887631923 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2f80aa760 msgr2=0x7fb2f813c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:59.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.185+0000 7fb2fe713700 1 --2- 172.21.15.17:0/3887631923 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2f80aa760 0x7fb2f813c600 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7fb2f00032f0 tx=0x7fb2f00091d0).stop 2022-08-01T19:43:59.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.185+0000 7fb2fe713700 1 -- 172.21.15.17:0/3887631923 shutdown_connections 2022-08-01T19:43:59.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.185+0000 7fb2fe713700 1 --2- 172.21.15.17:0/3887631923 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2f80aa760 0x7fb2f813c600 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:59.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.185+0000 7fb2fe713700 1 --2- 172.21.15.17:0/3887631923 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb2f80b6a90 0x7fb2f80acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:59.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.185+0000 7fb2fe713700 1 -- 172.21.15.17:0/3887631923 >> 172.21.15.17:0/3887631923 conn(0x7fb2f80b3610 msgr2=0x7fb2f80b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:59.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.185+0000 7fb2fe713700 1 -- 172.21.15.17:0/3887631923 shutdown_connections 2022-08-01T19:43:59.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.185+0000 7fb2fe713700 1 -- 172.21.15.17:0/3887631923 wait complete. 2022-08-01T19:43:59.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.189+0000 7fb2fe713700 1 Processor -- start 2022-08-01T19:43:59.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.189+0000 7fb2fe713700 1 -- start start 2022-08-01T19:43:59.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.189+0000 7fb2fe713700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2f8060aa0 0x7fb2f80afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:59.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.189+0000 7fb2fe713700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb2f80aa760 0x7fb2f80b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:59.206 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.189+0000 7fb2fe713700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2f80b6a90 0x7fb2f8140c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:59.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.189+0000 7fb2fe713700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f8143c90 con 0x7fb2f80b6a90 2022-08-01T19:43:59.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.189+0000 7fb2fe713700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f80af010 con 0x7fb2f8060aa0 2022-08-01T19:43:59.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.189+0000 7fb2fe713700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f80af410 con 0x7fb2f80aa760 2022-08-01T19:43:59.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.189+0000 7fb2fccb0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2f80b6a90 0x7fb2f8140c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:59.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.189+0000 7fb2fccb0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2f80b6a90 0x7fb2f8140c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45608/0 (socket says 172.21.15.17:45608) 2022-08-01T19:43:59.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.189+0000 7fb2fccb0700 1 -- 172.21.15.17:0/998310080 learned_addr learned my addr 172.21.15.17:0/998310080 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:59.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.189+0000 7fb2f7fff700 1 --2- 172.21.15.17:0/998310080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2f8060aa0 0x7fb2f80afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:59.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.189+0000 7fb2f77fe700 1 --2- 172.21.15.17:0/998310080 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb2f80aa760 0x7fb2f80b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:59.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.189+0000 7fb2fccb0700 1 -- 172.21.15.17:0/998310080 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb2f80aa760 msgr2=0x7fb2f80b22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:59.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.189+0000 7fb2fccb0700 1 --2- 172.21.15.17:0/998310080 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb2f80aa760 0x7fb2f80b22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:59.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.189+0000 7fb2fccb0700 1 -- 172.21.15.17:0/998310080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2f8060aa0 msgr2=0x7fb2f80afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:59.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.189+0000 7fb2fccb0700 1 --2- 172.21.15.17:0/998310080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2f8060aa0 0x7fb2f80afe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:59.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.189+0000 7fb2fccb0700 1 -- 172.21.15.17:0/998310080 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e000b180 con 0x7fb2f80b6a90 2022-08-01T19:43:59.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.189+0000 7fb2f77fe700 1 --2- 172.21.15.17:0/998310080 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb2f80aa760 0x7fb2f80b22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:43:59.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.189+0000 7fb2f7fff700 1 --2- 172.21.15.17:0/998310080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2f8060aa0 0x7fb2f80afe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:43:59.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.189+0000 7fb2fccb0700 1 --2- 172.21.15.17:0/998310080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2f80b6a90 0x7fb2f8140c50 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 rx=0x7fb2f00032f0 tx=0x7fb2f0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:59.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.189+0000 7fb2f57fa700 1 -- 172.21.15.17:0/998310080 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb2f0023440 con 0x7fb2f80b6a90 2022-08-01T19:43:59.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.189+0000 7fb2fe713700 1 -- 172.21.15.17:0/998310080 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2f0009470 con 0x7fb2f80b6a90 2022-08-01T19:43:59.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.189+0000 7fb2f57fa700 1 -- 172.21.15.17:0/998310080 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f00235a0 con 0x7fb2f80b6a90 2022-08-01T19:43:59.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.189+0000 7fb2f57fa700 1 -- 172.21.15.17:0/998310080 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb2f0023900 con 0x7fb2f80b6a90 2022-08-01T19:43:59.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.189+0000 7fb2fe713700 1 -- 172.21.15.17:0/998310080 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2f8143550 con 0x7fb2f80b6a90 2022-08-01T19:43:59.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.193+0000 7fb2fe713700 1 -- 172.21.15.17:0/998310080 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2f8061b50 con 0x7fb2f80b6a90 2022-08-01T19:43:59.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.193+0000 7fb2f57fa700 1 -- 172.21.15.17:0/998310080 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb2f0029020 con 0x7fb2f80b6a90 2022-08-01T19:43:59.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.193+0000 7fb2f57fa700 1 --2- 172.21.15.17:0/998310080 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb2dc053a70 0x7fb2dc055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:59.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.193+0000 7fb2f57fa700 1 -- 172.21.15.17:0/998310080 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb2f006df50 con 0x7fb2f80b6a90 2022-08-01T19:43:59.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.193+0000 7fb2f7fff700 1 --2- 172.21.15.17:0/998310080 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb2dc053a70 0x7fb2dc055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:59.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.193+0000 7fb2f7fff700 1 --2- 172.21.15.17:0/998310080 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb2dc053a70 0x7fb2dc055f30 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7fb2e000b000 tx=0x7fb2e0008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:59.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.201+0000 7fb2f57fa700 1 -- 172.21.15.17:0/998310080 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb2f0044b80 con 0x7fb2f80b6a90 2022-08-01T19:43:59.331 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.329+0000 7fb2fe713700 1 -- 172.21.15.17:0/998310080 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2f80574e0 con 0x7fb2f80b6a90 2022-08-01T19:43:59.333 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.329+0000 7fb2f57fa700 1 -- 172.21.15.17:0/998310080 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) v1 ==== 74+0+16311 (secure 0 0 0) 0x7fb2f00481a0 con 0x7fb2f80b6a90 2022-08-01T19:43:59.333 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:43:59.333 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":157,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:43:58.712859+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:43:57.196125+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2022-08-01T19:43:57.958027+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:43:59.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.333+0000 7fb2fe713700 1 -- 172.21.15.17:0/998310080 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb2dc053a70 msgr2=0x7fb2dc055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:59.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.333+0000 7fb2fe713700 1 --2- 172.21.15.17:0/998310080 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb2dc053a70 0x7fb2dc055f30 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7fb2e000b000 tx=0x7fb2e0008040).stop 2022-08-01T19:43:59.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.333+0000 7fb2fe713700 1 -- 172.21.15.17:0/998310080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2f80b6a90 msgr2=0x7fb2f8140c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:59.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.333+0000 7fb2fe713700 1 --2- 172.21.15.17:0/998310080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2f80b6a90 0x7fb2f8140c50 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 rx=0x7fb2f00032f0 tx=0x7fb2f0017000).stop 2022-08-01T19:43:59.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.333+0000 7fb2fe713700 1 -- 172.21.15.17:0/998310080 shutdown_connections 2022-08-01T19:43:59.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.333+0000 7fb2fe713700 1 --2- 172.21.15.17:0/998310080 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb2dc053a70 0x7fb2dc055f30 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:59.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.333+0000 7fb2fe713700 1 --2- 172.21.15.17:0/998310080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2f80b6a90 0x7fb2f8140c50 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:59.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.333+0000 7fb2fe713700 1 --2- 172.21.15.17:0/998310080 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb2f80aa760 0x7fb2f80b22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:59.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.333+0000 7fb2fe713700 1 --2- 172.21.15.17:0/998310080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2f8060aa0 0x7fb2f80afe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:59.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.333+0000 7fb2fe713700 1 -- 172.21.15.17:0/998310080 >> 172.21.15.17:0/998310080 conn(0x7fb2f80b3610 msgr2=0x7fb2f80b3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:59.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.333+0000 7fb2fe713700 1 -- 172.21.15.17:0/998310080 shutdown_connections 2022-08-01T19:43:59.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.333+0000 7fb2fe713700 1 -- 172.21.15.17:0/998310080 wait complete. 2022-08-01T19:43:59.348 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-08-01T19:43:59.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.461+0000 7efd99aec700 1 Processor -- start 2022-08-01T19:43:59.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.461+0000 7efd99aec700 1 -- start start 2022-08-01T19:43:59.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.461+0000 7efd99aec700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efd940ad750 0x7efd94061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:59.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.461+0000 7efd99aec700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd940aee80 con 0x7efd940614b0 2022-08-01T19:43:59.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.461+0000 7efd99aec700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd940b70b0 con 0x7efd941405d0 2022-08-01T19:43:59.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.461+0000 7efd99aec700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd940af0b0 con 0x7efd940ad750 2022-08-01T19:43:59.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.461+0000 7efd98aea700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7efd940614b0 0x7efd940b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37806/0 (socket says 172.21.15.17:37806) 2022-08-01T19:43:59.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.461+0000 7efd98aea700 1 -- 172.21.15.17:0/378778647 learned_addr learned my addr 172.21.15.17:0/378778647 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:59.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.461+0000 7efd93fff700 1 --2- 172.21.15.17:0/378778647 >> v2:172.21.15.17:3300/0 conn(0x7efd940ad750 0x7efd94061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:59.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.461+0000 7efd937fe700 1 -- 172.21.15.17:0/378778647 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3058635172 0 0) 0x7efd940aee80 con 0x7efd940614b0 2022-08-01T19:43:59.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.461+0000 7efd937fe700 1 -- 172.21.15.17:0/378778647 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd80003260 con 0x7efd940614b0 2022-08-01T19:43:59.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.461+0000 7efd937fe700 1 -- 172.21.15.17:0/378778647 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 528106674 0 0) 0x7efd940b70b0 con 0x7efd941405d0 2022-08-01T19:43:59.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.461+0000 7efd937fe700 1 -- 172.21.15.17:0/378778647 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd940aee80 con 0x7efd941405d0 2022-08-01T19:43:59.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.461+0000 7efd93fff700 1 -- 172.21.15.17:0/378778647 >> v1:172.21.15.77:6789/0 conn(0x7efd941405d0 legacy=0x7efd941466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:59.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.461+0000 7efd93fff700 1 -- 172.21.15.17:0/378778647 >> v1:172.21.15.17:6789/0 conn(0x7efd940614b0 legacy=0x7efd940b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:59.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.461+0000 7efd93fff700 1 -- 172.21.15.17:0/378778647 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd940af350 con 0x7efd940ad750 2022-08-01T19:43:59.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.465+0000 7efd93fff700 1 --2- 172.21.15.17:0/378778647 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd940ad750 0x7efd94061890 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 rx=0x7efd84002f70 tx=0x7efd84006df0).ready entity=mon.0 client_cookie=cd289cdb96050ce1 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:59.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.465+0000 7efd937fe700 1 -- 172.21.15.17:0/378778647 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd8400d540 con 0x7efd940ad750 2022-08-01T19:43:59.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.465+0000 7efd937fe700 1 -- 172.21.15.17:0/378778647 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd84000e90 con 0x7efd940ad750 2022-08-01T19:43:59.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.465+0000 7efd937fe700 1 -- 172.21.15.17:0/378778647 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd8400d890 con 0x7efd940ad750 2022-08-01T19:43:59.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.465+0000 7efd99aec700 1 -- 172.21.15.17:0/378778647 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd940ad750 msgr2=0x7efd94061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:59.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.465+0000 7efd99aec700 1 --2- 172.21.15.17:0/378778647 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd940ad750 0x7efd94061890 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 rx=0x7efd84002f70 tx=0x7efd84006df0).stop 2022-08-01T19:43:59.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.465+0000 7efd99aec700 1 -- 172.21.15.17:0/378778647 shutdown_connections 2022-08-01T19:43:59.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.465+0000 7efd99aec700 1 --2- 172.21.15.17:0/378778647 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd940ad750 0x7efd94061890 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:59.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.465+0000 7efd99aec700 1 -- 172.21.15.17:0/378778647 >> 172.21.15.17:0/378778647 conn(0x7efd94066f70 msgr2=0x7efd94067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:59.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.465+0000 7efd99aec700 1 -- 172.21.15.17:0/378778647 shutdown_connections 2022-08-01T19:43:59.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.465+0000 7efd99aec700 1 -- 172.21.15.17:0/378778647 wait complete. 2022-08-01T19:43:59.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.465+0000 7efd99aec700 1 Processor -- start 2022-08-01T19:43:59.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.465+0000 7efd99aec700 1 -- start start 2022-08-01T19:43:59.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.469+0000 7efd99aec700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efd940614b0 0x7efd94146bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:59.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.469+0000 7efd99aec700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd940ad750 0x7efd940b1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:59.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.469+0000 7efd99aec700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd941405d0 0x7efd940b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:59.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.469+0000 7efd99aec700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd940acc50 con 0x7efd941405d0 2022-08-01T19:43:59.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.469+0000 7efd99aec700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd94152e80 con 0x7efd940ad750 2022-08-01T19:43:59.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.469+0000 7efd99aec700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd94153120 con 0x7efd940614b0 2022-08-01T19:43:59.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.469+0000 7efd992eb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd941405d0 0x7efd940b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:59.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.469+0000 7efd992eb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd941405d0 0x7efd940b2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45622/0 (socket says 172.21.15.17:45622) 2022-08-01T19:43:59.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.469+0000 7efd992eb700 1 -- 172.21.15.17:0/2256963517 learned_addr learned my addr 172.21.15.17:0/2256963517 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:59.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.469+0000 7efd93fff700 1 --2- 172.21.15.17:0/2256963517 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd940ad750 0x7efd940b1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:59.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.469+0000 7efd98aea700 1 --2- 172.21.15.17:0/2256963517 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efd940614b0 0x7efd94146bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:59.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.469+0000 7efd992eb700 1 -- 172.21.15.17:0/2256963517 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efd940614b0 msgr2=0x7efd94146bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:59.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.469+0000 7efd992eb700 1 --2- 172.21.15.17:0/2256963517 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efd940614b0 0x7efd94146bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:59.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.469+0000 7efd992eb700 1 -- 172.21.15.17:0/2256963517 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd940ad750 msgr2=0x7efd940b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:59.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.469+0000 7efd992eb700 1 --2- 172.21.15.17:0/2256963517 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd940ad750 0x7efd940b1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:59.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.469+0000 7efd992eb700 1 -- 172.21.15.17:0/2256963517 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd8400b180 con 0x7efd941405d0 2022-08-01T19:43:59.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.469+0000 7efd98aea700 1 --2- 172.21.15.17:0/2256963517 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efd940614b0 0x7efd94146bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:43:59.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.469+0000 7efd992eb700 1 --2- 172.21.15.17:0/2256963517 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd941405d0 0x7efd940b2090 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 rx=0x7efd8c001490 tx=0x7efd8c007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:59.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.469+0000 7efd91ffb700 1 -- 172.21.15.17:0/2256963517 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd8c002310 con 0x7efd941405d0 2022-08-01T19:43:59.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.469+0000 7efd91ffb700 1 -- 172.21.15.17:0/2256963517 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8c002470 con 0x7efd941405d0 2022-08-01T19:43:59.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.469+0000 7efd99aec700 1 -- 172.21.15.17:0/2256963517 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd940ace80 con 0x7efd941405d0 2022-08-01T19:43:59.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.469+0000 7efd91ffb700 1 -- 172.21.15.17:0/2256963517 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd8c0017a0 con 0x7efd941405d0 2022-08-01T19:43:59.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.469+0000 7efd99aec700 1 -- 172.21.15.17:0/2256963517 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd940a9da0 con 0x7efd941405d0 2022-08-01T19:43:59.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.469+0000 7efd99aec700 1 -- 172.21.15.17:0/2256963517 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd940ad010 con 0x7efd941405d0 2022-08-01T19:43:59.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.469+0000 7efd91ffb700 1 -- 172.21.15.17:0/2256963517 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7efd8c010030 con 0x7efd941405d0 2022-08-01T19:43:59.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.473+0000 7efd91ffb700 1 --2- 172.21.15.17:0/2256963517 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efd74053950 0x7efd74055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:59.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.473+0000 7efd91ffb700 1 -- 172.21.15.17:0/2256963517 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6420+0+0 (secure 0 0 0) 0x7efd8c06bec0 con 0x7efd941405d0 2022-08-01T19:43:59.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.473+0000 7efd98aea700 1 --2- 172.21.15.17:0/2256963517 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efd74053950 0x7efd74055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:59.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.473+0000 7efd98aea700 1 --2- 172.21.15.17:0/2256963517 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efd74053950 0x7efd74055e10 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7efd8800b000 tx=0x7efd88009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:59.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.477+0000 7efd91ffb700 1 -- 172.21.15.17:0/2256963517 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efd8c042be0 con 0x7efd941405d0 2022-08-01T19:43:59.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.641+0000 7efd99aec700 1 -- 172.21.15.17:0/2256963517 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7efd940af5f0 con 0x7efd941405d0 2022-08-01T19:43:59.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.729+0000 7efd91ffb700 1 -- 172.21.15.17:0/2256963517 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v83) v1 ==== 118+0+0 (secure 0 0 0) 0x7efd8c046200 con 0x7efd941405d0 2022-08-01T19:43:59.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.733+0000 7efd99aec700 1 -- 172.21.15.17:0/2256963517 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efd74053950 msgr2=0x7efd74055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:59.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.733+0000 7efd99aec700 1 --2- 172.21.15.17:0/2256963517 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efd74053950 0x7efd74055e10 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7efd8800b000 tx=0x7efd88009040).stop 2022-08-01T19:43:59.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.733+0000 7efd99aec700 1 -- 172.21.15.17:0/2256963517 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd941405d0 msgr2=0x7efd940b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:59.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.733+0000 7efd99aec700 1 --2- 172.21.15.17:0/2256963517 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd941405d0 0x7efd940b2090 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 rx=0x7efd8c001490 tx=0x7efd8c007dd0).stop 2022-08-01T19:43:59.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.733+0000 7efd99aec700 1 -- 172.21.15.17:0/2256963517 shutdown_connections 2022-08-01T19:43:59.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.733+0000 7efd99aec700 1 --2- 172.21.15.17:0/2256963517 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efd74053950 0x7efd74055e10 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:59.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.733+0000 7efd99aec700 1 --2- 172.21.15.17:0/2256963517 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd941405d0 0x7efd940b2090 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:59.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.733+0000 7efd99aec700 1 --2- 172.21.15.17:0/2256963517 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd940ad750 0x7efd940b1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:59.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.733+0000 7efd99aec700 1 --2- 172.21.15.17:0/2256963517 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7efd940614b0 0x7efd94146bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:59.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.733+0000 7efd99aec700 1 -- 172.21.15.17:0/2256963517 >> 172.21.15.17:0/2256963517 conn(0x7efd94066f70 msgr2=0x7efd940b9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:59.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.733+0000 7efd99aec700 1 -- 172.21.15.17:0/2256963517 shutdown_connections 2022-08-01T19:43:59.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.733+0000 7efd99aec700 1 -- 172.21.15.17:0/2256963517 wait complete. 2022-08-01T19:43:59.750 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:43:59.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.849+0000 7f715759e700 1 Processor -- start 2022-08-01T19:43:59.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.849+0000 7f715759e700 1 -- start start 2022-08-01T19:43:59.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.849+0000 7f715759e700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f7158061d10 0x7f7158144630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:59.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.849+0000 7f715759e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71580b41f0 0x7f7158144b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:59.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.849+0000 7f715759e700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71580b2100 con 0x7f71580b65c0 2022-08-01T19:43:59.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.849+0000 7f715759e700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71580b1630 con 0x7f7158061d10 2022-08-01T19:43:59.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.849+0000 7f715759e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71580b69a0 con 0x7f71580b41f0 2022-08-01T19:43:59.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.849+0000 7f715659c700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f71580b65c0 0x7f7158061600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37820/0 (socket says 172.21.15.17:37820) 2022-08-01T19:43:59.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.849+0000 7f715659c700 1 -- 172.21.15.17:0/2500793219 learned_addr learned my addr 172.21.15.17:0/2500793219 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:43:59.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.849+0000 7f7155d9b700 1 --2- 172.21.15.17:0/2500793219 >> v2:172.21.15.77:3300/0 conn(0x7f7158061d10 0x7f7158144630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:59.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.849+0000 7f7156d9d700 1 --2- 172.21.15.17:0/2500793219 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71580b41f0 0x7f7158144b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:59.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.849+0000 7f715559a700 1 -- 172.21.15.17:0/2500793219 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1050843374 0 0) 0x7f71580b2100 con 0x7f71580b65c0 2022-08-01T19:43:59.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.849+0000 7f715559a700 1 -- 172.21.15.17:0/2500793219 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f713c003260 con 0x7f71580b65c0 2022-08-01T19:43:59.870 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.849+0000 7f7155d9b700 1 -- 172.21.15.17:0/2500793219 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71580b41f0 msgr2=0x7f7158144b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:43:59.870 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.849+0000 7f7155d9b700 1 --2- 172.21.15.17:0/2500793219 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71580b41f0 0x7f7158144b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:59.871 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.849+0000 7f7155d9b700 1 -- 172.21.15.17:0/2500793219 >> v1:172.21.15.17:6789/0 conn(0x7f71580b65c0 legacy=0x7f7158061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:59.871 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.849+0000 7f7155d9b700 1 -- 172.21.15.17:0/2500793219 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71581482a0 con 0x7f7158061d10 2022-08-01T19:43:59.871 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.849+0000 7f7156d9d700 1 --2- 172.21.15.17:0/2500793219 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71580b41f0 0x7f7158144b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:43:59.871 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.849+0000 7f7155d9b700 1 --2- 172.21.15.17:0/2500793219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7158061d10 0x7f7158144630 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f7148002f70 tx=0x7f7148006df0).ready entity=mon.1 client_cookie=3cc32747ed83f470 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:59.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.853+0000 7f715559a700 1 -- 172.21.15.17:0/2500793219 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f714800d4a0 con 0x7f7158061d10 2022-08-01T19:43:59.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.853+0000 7f715559a700 1 -- 172.21.15.17:0/2500793219 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7148000e90 con 0x7f7158061d10 2022-08-01T19:43:59.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.853+0000 7f715559a700 1 -- 172.21.15.17:0/2500793219 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f714800d7f0 con 0x7f7158061d10 2022-08-01T19:43:59.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.853+0000 7f715759e700 1 -- 172.21.15.17:0/2500793219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7158061d10 msgr2=0x7f7158144630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:59.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.853+0000 7f715759e700 1 --2- 172.21.15.17:0/2500793219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7158061d10 0x7f7158144630 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f7148002f70 tx=0x7f7148006df0).stop 2022-08-01T19:43:59.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.853+0000 7f715759e700 1 -- 172.21.15.17:0/2500793219 shutdown_connections 2022-08-01T19:43:59.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.853+0000 7f715759e700 1 --2- 172.21.15.17:0/2500793219 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f71580b41f0 0x7f7158144b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:59.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.853+0000 7f715759e700 1 --2- 172.21.15.17:0/2500793219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7158061d10 0x7f7158144630 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:59.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.853+0000 7f715759e700 1 -- 172.21.15.17:0/2500793219 >> 172.21.15.17:0/2500793219 conn(0x7f715805c620 msgr2=0x7f715805ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:43:59.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.853+0000 7f715759e700 1 -- 172.21.15.17:0/2500793219 shutdown_connections 2022-08-01T19:43:59.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.853+0000 7f715759e700 1 -- 172.21.15.17:0/2500793219 wait complete. 2022-08-01T19:43:59.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.853+0000 7f715759e700 1 Processor -- start 2022-08-01T19:43:59.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.853+0000 7f715759e700 1 -- start start 2022-08-01T19:43:59.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.857+0000 7f715759e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7158061d10 0x7f715814e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:59.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.857+0000 7f715759e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71580b41f0 0x7f7158159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:59.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.857+0000 7f715759e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71580b65c0 0x7f715815a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:59.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.857+0000 7f715759e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715814ec00 con 0x7f71580b41f0 2022-08-01T19:43:59.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.857+0000 7f715759e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71580ae380 con 0x7f71580b65c0 2022-08-01T19:43:59.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.857+0000 7f715759e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71580aed90 con 0x7f7158061d10 2022-08-01T19:43:59.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.857+0000 7f7155d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71580b41f0 0x7f7158159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:59.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.857+0000 7f7155d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71580b41f0 0x7f7158159d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45634/0 (socket says 172.21.15.17:45634) 2022-08-01T19:43:59.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.857+0000 7f7155d9b700 1 -- 172.21.15.17:0/3546718405 learned_addr learned my addr 172.21.15.17:0/3546718405 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:43:59.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.857+0000 7f7156d9d700 1 --2- 172.21.15.17:0/3546718405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71580b65c0 0x7f715815a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:59.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.857+0000 7f715659c700 1 --2- 172.21.15.17:0/3546718405 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7158061d10 0x7f715814e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:59.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.857+0000 7f7155d9b700 1 -- 172.21.15.17:0/3546718405 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7158061d10 msgr2=0x7f715814e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:59.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.857+0000 7f7155d9b700 1 --2- 172.21.15.17:0/3546718405 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7158061d10 0x7f715814e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:59.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.857+0000 7f7155d9b700 1 -- 172.21.15.17:0/3546718405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71580b65c0 msgr2=0x7f715815a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:43:59.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.857+0000 7f7155d9b700 1 --2- 172.21.15.17:0/3546718405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71580b65c0 0x7f715815a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:43:59.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.857+0000 7f7155d9b700 1 -- 172.21.15.17:0/3546718405 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7150009470 con 0x7f71580b41f0 2022-08-01T19:43:59.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.857+0000 7f7155d9b700 1 --2- 172.21.15.17:0/3546718405 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71580b41f0 0x7f7158159d70 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 rx=0x7f7148002f70 tx=0x7f7148014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:59.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.857+0000 7f71477fe700 1 -- 172.21.15.17:0/3546718405 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71480139c0 con 0x7f71580b41f0 2022-08-01T19:43:59.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.857+0000 7f71477fe700 1 -- 172.21.15.17:0/3546718405 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7148013b20 con 0x7f71580b41f0 2022-08-01T19:43:59.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.857+0000 7f715759e700 1 -- 172.21.15.17:0/3546718405 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f714800b180 con 0x7f71580b41f0 2022-08-01T19:43:59.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.857+0000 7f71477fe700 1 -- 172.21.15.17:0/3546718405 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7148013e80 con 0x7f71580b41f0 2022-08-01T19:43:59.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.857+0000 7f715759e700 1 -- 172.21.15.17:0/3546718405 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71580ae650 con 0x7f71580b41f0 2022-08-01T19:43:59.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.857+0000 7f715759e700 1 -- 172.21.15.17:0/3546718405 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71580b2330 con 0x7f71580b41f0 2022-08-01T19:43:59.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.857+0000 7f71477fe700 1 -- 172.21.15.17:0/3546718405 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7148025030 con 0x7f71580b41f0 2022-08-01T19:43:59.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.861+0000 7f71477fe700 1 --2- 172.21.15.17:0/3546718405 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7134053a70 0x7f7134055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:43:59.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.861+0000 7f71477fe700 1 -- 172.21.15.17:0/3546718405 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f714806cd20 con 0x7f71580b41f0 2022-08-01T19:43:59.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.861+0000 7f715659c700 1 --2- 172.21.15.17:0/3546718405 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7134053a70 0x7f7134055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:43:59.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.861+0000 7f715659c700 1 --2- 172.21.15.17:0/3546718405 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7134053a70 0x7f7134055f30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7f714c00d000 tx=0x7f714c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:43:59.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:43:59.869+0000 7f71477fe700 1 -- 172.21.15.17:0/3546718405 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7148043ae0 con 0x7f71580b41f0 2022-08-01T19:44:00.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.037+0000 7f715759e700 1 -- 172.21.15.17:0/3546718405 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71580574e0 con 0x7f71580b41f0 2022-08-01T19:44:00.040 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.037+0000 7f71477fe700 1 -- 172.21.15.17:0/3546718405 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+2486 (secure 0 0 0) 0x7f7148023070 con 0x7f71580b41f0 2022-08-01T19:44:00.041 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:44:00.043 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":83,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8665,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/608187432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":608187432},{"type":"v1","addr":"172.21.15.77:6835","nonce":608187432}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":81},{"gid":8673,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2764139073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2764139073},{"type":"v1","addr":"172.21.15.17:6833","nonce":2764139073}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":80}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:43:58.713330+0000","modified":"2022-08-01T19:43:59.717120+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8682":{"gid":8682,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/75309005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":75309005},{"type":"v1","addr":"172.21.15.17:6835","nonce":75309005}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2022-08-01T19:44:00.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.041+0000 7f715759e700 1 -- 172.21.15.17:0/3546718405 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7134053a70 msgr2=0x7f7134055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:00.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.041+0000 7f715759e700 1 --2- 172.21.15.17:0/3546718405 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7134053a70 0x7f7134055f30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7f714c00d000 tx=0x7f714c009040).stop 2022-08-01T19:44:00.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.041+0000 7f715759e700 1 -- 172.21.15.17:0/3546718405 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71580b41f0 msgr2=0x7f7158159d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:00.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.041+0000 7f715759e700 1 --2- 172.21.15.17:0/3546718405 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71580b41f0 0x7f7158159d70 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 rx=0x7f7148002f70 tx=0x7f7148014000).stop 2022-08-01T19:44:00.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.041+0000 7f715759e700 1 -- 172.21.15.17:0/3546718405 shutdown_connections 2022-08-01T19:44:00.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.041+0000 7f715759e700 1 --2- 172.21.15.17:0/3546718405 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7134053a70 0x7f7134055f30 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:00.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.041+0000 7f715759e700 1 --2- 172.21.15.17:0/3546718405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f71580b65c0 0x7f715815a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:00.048 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.041+0000 7f715759e700 1 --2- 172.21.15.17:0/3546718405 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71580b41f0 0x7f7158159d70 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:00.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.041+0000 7f715759e700 1 --2- 172.21.15.17:0/3546718405 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f7158061d10 0x7f715814e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:00.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.041+0000 7f715759e700 1 -- 172.21.15.17:0/3546718405 >> 172.21.15.17:0/3546718405 conn(0x7f715805c620 msgr2=0x7f71580b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:00.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.041+0000 7f715759e700 1 -- 172.21.15.17:0/3546718405 shutdown_connections 2022-08-01T19:44:00.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.041+0000 7f715759e700 1 -- 172.21.15.17:0/3546718405 wait complete. 2022-08-01T19:44:00.051 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 83 2022-08-01T19:44:00.061 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:44:00.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.169+0000 7f9cd2518700 1 Processor -- start 2022-08-01T19:44:00.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.169+0000 7f9cd2518700 1 -- start start 2022-08-01T19:44:00.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.169+0000 7f9cd2518700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ccc0b2710 0x7f9ccc14afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:00.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.169+0000 7f9cd2518700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9ccc0b01a0 0x7f9ccc14b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:00.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.169+0000 7f9cd2518700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ccc0a7250 con 0x7f9ccc0afd60 2022-08-01T19:44:00.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.169+0000 7f9cd2518700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ccc0a7480 con 0x7f9ccc0b01a0 2022-08-01T19:44:00.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.169+0000 7f9cd2518700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc0a7720 con 0x7f9ccc0b2710 2022-08-01T19:44:00.182 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.169+0000 7f9cd0d15700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9ccc0b01a0 0x7f9ccc14b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:00.182 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.169+0000 7f9cd0d15700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9ccc0b01a0 0x7f9ccc14b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45640/0 (socket says 172.21.15.17:45640) 2022-08-01T19:44:00.183 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.169+0000 7f9cd0d15700 1 -- 172.21.15.17:0/1702021510 learned_addr learned my addr 172.21.15.17:0/1702021510 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:44:00.183 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.169+0000 7f9cd1516700 1 --2- 172.21.15.17:0/1702021510 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ccc0b2710 0x7f9ccc14afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:00.184 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.169+0000 7f9cd0d15700 1 -- 172.21.15.17:0/1702021510 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ccc0b2710 msgr2=0x7f9ccc14afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:44:00.184 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.169+0000 7f9cd0d15700 1 --2- 172.21.15.17:0/1702021510 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ccc0b2710 0x7f9ccc14afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:00.184 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.169+0000 7f9cd0d15700 1 -- 172.21.15.17:0/1702021510 >> v1:172.21.15.17:6789/0 conn(0x7f9ccc0afd60 legacy=0x7f9ccc0a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:00.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.169+0000 7f9cd0d15700 1 -- 172.21.15.17:0/1702021510 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ccc0a79c0 con 0x7f9ccc0b01a0 2022-08-01T19:44:00.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.169+0000 7f9cd1516700 1 --2- 172.21.15.17:0/1702021510 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ccc0b2710 0x7f9ccc14afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:44:00.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.169+0000 7f9cd0d15700 1 --2- 172.21.15.17:0/1702021510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ccc0b01a0 0x7f9ccc14b4c0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 rx=0x7f9cc8002f70 tx=0x7f9cc8006eb0).ready entity=mon.0 client_cookie=138dc828b4692c90 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:00.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.169+0000 7f9cc3fff700 1 -- 172.21.15.17:0/1702021510 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9cc800d5c0 con 0x7f9ccc0b01a0 2022-08-01T19:44:00.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.169+0000 7f9cc3fff700 1 -- 172.21.15.17:0/1702021510 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc8000e90 con 0x7f9ccc0b01a0 2022-08-01T19:44:00.188 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.169+0000 7f9cc3fff700 1 -- 172.21.15.17:0/1702021510 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9cc800d950 con 0x7f9ccc0b01a0 2022-08-01T19:44:00.188 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.169+0000 7f9cd2518700 1 -- 172.21.15.17:0/1702021510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ccc0b01a0 msgr2=0x7f9ccc14b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:00.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.169+0000 7f9cd2518700 1 --2- 172.21.15.17:0/1702021510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ccc0b01a0 0x7f9ccc14b4c0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 rx=0x7f9cc8002f70 tx=0x7f9cc8006eb0).stop 2022-08-01T19:44:00.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.169+0000 7f9cd2518700 1 -- 172.21.15.17:0/1702021510 shutdown_connections 2022-08-01T19:44:00.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.173+0000 7f9cd2518700 1 --2- 172.21.15.17:0/1702021510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ccc0b01a0 0x7f9ccc14b4c0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:00.190 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.173+0000 7f9cd2518700 1 --2- 172.21.15.17:0/1702021510 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ccc0b2710 0x7f9ccc14afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:00.190 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.173+0000 7f9cd2518700 1 -- 172.21.15.17:0/1702021510 >> 172.21.15.17:0/1702021510 conn(0x7f9ccc060310 msgr2=0x7f9ccc0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:00.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.173+0000 7f9cd2518700 1 -- 172.21.15.17:0/1702021510 shutdown_connections 2022-08-01T19:44:00.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.173+0000 7f9cd2518700 1 -- 172.21.15.17:0/1702021510 wait complete. 2022-08-01T19:44:00.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.173+0000 7f9cd2518700 1 Processor -- start 2022-08-01T19:44:00.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.173+0000 7f9cd2518700 1 -- start start 2022-08-01T19:44:00.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.173+0000 7f9cd2518700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ccc0afd60 0x7f9ccc05c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:00.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.173+0000 7f9cd2518700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ccc0b01a0 0x7f9ccc0657c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:00.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.173+0000 7f9cd2518700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ccc0b2710 0x7f9ccc065cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:00.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.173+0000 7f9cd2518700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc0b5bc0 con 0x7f9ccc0b2710 2022-08-01T19:44:00.194 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.173+0000 7f9cd2518700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc0636a0 con 0x7f9ccc0b01a0 2022-08-01T19:44:00.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.173+0000 7f9cd2518700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc0b55a0 con 0x7f9ccc0afd60 2022-08-01T19:44:00.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.173+0000 7f9cd1d17700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ccc0b2710 0x7f9ccc065cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:00.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.173+0000 7f9cd1d17700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ccc0b2710 0x7f9ccc065cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45650/0 (socket says 172.21.15.17:45650) 2022-08-01T19:44:00.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.173+0000 7f9cd1d17700 1 -- 172.21.15.17:0/4095853017 learned_addr learned my addr 172.21.15.17:0/4095853017 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:44:00.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.173+0000 7f9cd0d15700 1 --2- 172.21.15.17:0/4095853017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ccc0b01a0 0x7f9ccc0657c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:00.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.173+0000 7f9cd1516700 1 --2- 172.21.15.17:0/4095853017 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ccc0afd60 0x7f9ccc05c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:00.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.173+0000 7f9cd1d17700 1 -- 172.21.15.17:0/4095853017 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ccc0afd60 msgr2=0x7f9ccc05c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:00.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.173+0000 7f9cd1d17700 1 --2- 172.21.15.17:0/4095853017 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ccc0afd60 0x7f9ccc05c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:00.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.173+0000 7f9cd1d17700 1 -- 172.21.15.17:0/4095853017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ccc0b01a0 msgr2=0x7f9ccc0657c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:00.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.173+0000 7f9cd1d17700 1 --2- 172.21.15.17:0/4095853017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ccc0b01a0 0x7f9ccc0657c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:00.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.173+0000 7f9cd1d17700 1 -- 172.21.15.17:0/4095853017 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc800b1e0 con 0x7f9ccc0b2710 2022-08-01T19:44:00.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.173+0000 7f9cd1d17700 1 --2- 172.21.15.17:0/4095853017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ccc0b2710 0x7f9ccc065cb0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 rx=0x7f9ccc061c10 tx=0x7f9cc4007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:00.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.173+0000 7f9cc27fc700 1 -- 172.21.15.17:0/4095853017 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9cc400f470 con 0x7f9ccc0b2710 2022-08-01T19:44:00.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.173+0000 7f9cc27fc700 1 -- 172.21.15.17:0/4095853017 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc400f5d0 con 0x7f9ccc0b2710 2022-08-01T19:44:00.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.173+0000 7f9cd2518700 1 -- 172.21.15.17:0/4095853017 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ccc0b5840 con 0x7f9ccc0b2710 2022-08-01T19:44:00.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.173+0000 7f9cc27fc700 1 -- 172.21.15.17:0/4095853017 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9cc400f930 con 0x7f9ccc0b2710 2022-08-01T19:44:00.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.173+0000 7f9cd2518700 1 -- 172.21.15.17:0/4095853017 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ccc1565f0 con 0x7f9ccc0b2710 2022-08-01T19:44:00.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.177+0000 7f9cd2518700 1 -- 172.21.15.17:0/4095853017 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ccc0574e0 con 0x7f9ccc0b2710 2022-08-01T19:44:00.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.177+0000 7f9cc27fc700 1 -- 172.21.15.17:0/4095853017 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9cc4019030 con 0x7f9ccc0b2710 2022-08-01T19:44:00.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.177+0000 7f9cc27fc700 1 --2- 172.21.15.17:0/4095853017 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9cb4053b50 0x7f9cb4056010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:00.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.177+0000 7f9cc27fc700 1 -- 172.21.15.17:0/4095853017 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9cc406b7c0 con 0x7f9ccc0b2710 2022-08-01T19:44:00.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.177+0000 7f9cd1516700 1 --2- 172.21.15.17:0/4095853017 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9cb4053b50 0x7f9cb4056010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:00.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.177+0000 7f9cd1516700 1 --2- 172.21.15.17:0/4095853017 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9cb4053b50 0x7f9cb4056010 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f9cbc00d000 tx=0x7f9cbc008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:00.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.185+0000 7f9cc27fc700 1 -- 172.21.15.17:0/4095853017 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9cc4042410 con 0x7f9ccc0b2710 2022-08-01T19:44:00.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.309+0000 7f9cd2518700 1 -- 172.21.15.17:0/4095853017 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ccc065560 con 0x7f9ccc0b2710 2022-08-01T19:44:00.314 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.309+0000 7f9cc27fc700 1 -- 172.21.15.17:0/4095853017 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f9cc4045a30 con 0x7f9ccc0b2710 2022-08-01T19:44:00.315 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:44:00.316 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":158,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:43:59.716124+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:43:57.196125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2022-08-01T19:43:57.958027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:44:00.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.309+0000 7f9cd2518700 1 -- 172.21.15.17:0/4095853017 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9cb4053b50 msgr2=0x7f9cb4056010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:00.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.309+0000 7f9cd2518700 1 --2- 172.21.15.17:0/4095853017 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9cb4053b50 0x7f9cb4056010 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f9cbc00d000 tx=0x7f9cbc008040).stop 2022-08-01T19:44:00.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.309+0000 7f9cd2518700 1 -- 172.21.15.17:0/4095853017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ccc0b2710 msgr2=0x7f9ccc065cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:00.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.309+0000 7f9cd2518700 1 --2- 172.21.15.17:0/4095853017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ccc0b2710 0x7f9ccc065cb0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 rx=0x7f9ccc061c10 tx=0x7f9cc4007f40).stop 2022-08-01T19:44:00.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.309+0000 7f9cd2518700 1 -- 172.21.15.17:0/4095853017 shutdown_connections 2022-08-01T19:44:00.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.309+0000 7f9cd2518700 1 --2- 172.21.15.17:0/4095853017 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f9cb4053b50 0x7f9cb4056010 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:00.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.309+0000 7f9cd2518700 1 --2- 172.21.15.17:0/4095853017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ccc0b2710 0x7f9ccc065cb0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:00.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.309+0000 7f9cd2518700 1 --2- 172.21.15.17:0/4095853017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ccc0b01a0 0x7f9ccc0657c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:00.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.309+0000 7f9cd2518700 1 --2- 172.21.15.17:0/4095853017 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f9ccc0afd60 0x7f9ccc05c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:00.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.309+0000 7f9cd2518700 1 -- 172.21.15.17:0/4095853017 >> 172.21.15.17:0/4095853017 conn(0x7f9ccc060310 msgr2=0x7f9ccc0ae330 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:00.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.313+0000 7f9cd2518700 1 -- 172.21.15.17:0/4095853017 shutdown_connections 2022-08-01T19:44:00.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.313+0000 7f9cd2518700 1 -- 172.21.15.17:0/4095853017 wait complete. 2022-08-01T19:44:00.340 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T19:44:00.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.457+0000 7f7655cef700 1 Processor -- start 2022-08-01T19:44:00.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.457+0000 7f7655cef700 1 -- start start 2022-08-01T19:44:00.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.457+0000 7f7655cef700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f76500b6420 0x7f7650061600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:00.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.457+0000 7f7655cef700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f76500b41f0 0x7f7650061af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:00.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.457+0000 7f7655cef700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76500b60e0 con 0x7f7650061fe0 2022-08-01T19:44:00.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.457+0000 7f7655cef700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76500b2100 con 0x7f76500b41f0 2022-08-01T19:44:00.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.457+0000 7f7655cef700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76500b23a0 con 0x7f76500b6420 2022-08-01T19:44:00.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.457+0000 7f7647fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f76500b41f0 0x7f7650061af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:00.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.457+0000 7f76554ee700 1 --1- >> v1:172.21.15.77:6789/0 conn(0x7f7650061fe0 0x7f76501465c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:54650/0 (socket says 172.21.15.17:54650) 2022-08-01T19:44:00.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.457+0000 7f76554ee700 1 -- 172.21.15.17:0/3586691881 learned_addr learned my addr 172.21.15.17:0/3586691881 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:44:00.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.457+0000 7f7654ced700 1 --2- 172.21.15.17:0/3586691881 >> v2:172.21.15.77:3300/0 conn(0x7f76500b6420 0x7f7650061600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:00.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.457+0000 7f7647fff700 1 -- 172.21.15.17:0/3586691881 >> v2:172.21.15.77:3300/0 conn(0x7f76500b6420 msgr2=0x7f7650061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:44:00.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.457+0000 7f76477fe700 1 -- 172.21.15.17:0/3586691881 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 361059622 0 0) 0x7f7648001490 con 0x7f7650061fe0 2022-08-01T19:44:00.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.457+0000 7f7647fff700 1 --2- 172.21.15.17:0/3586691881 >> v2:172.21.15.77:3300/0 conn(0x7f76500b6420 0x7f7650061600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:00.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.457+0000 7f7647fff700 1 -- 172.21.15.17:0/3586691881 >> v1:172.21.15.77:6789/0 conn(0x7f7650061fe0 legacy=0x7f76501465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:00.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.457+0000 7f7647fff700 1 -- 172.21.15.17:0/3586691881 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76500b2640 con 0x7f76500b41f0 2022-08-01T19:44:00.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.457+0000 7f7647fff700 1 --2- 172.21.15.17:0/3586691881 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76500b41f0 0x7f7650061af0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 rx=0x7f7640002f70 tx=0x7f7640006df0).ready entity=mon.0 client_cookie=38846262949d953e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:00.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.461+0000 7f76477fe700 1 -- 172.21.15.17:0/3586691881 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f764000d570 con 0x7f76500b41f0 2022-08-01T19:44:00.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.461+0000 7f76477fe700 1 -- 172.21.15.17:0/3586691881 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7640000e90 con 0x7f76500b41f0 2022-08-01T19:44:00.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.461+0000 7f76477fe700 1 -- 172.21.15.17:0/3586691881 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f764000d8c0 con 0x7f76500b41f0 2022-08-01T19:44:00.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.461+0000 7f7655cef700 1 -- 172.21.15.17:0/3586691881 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76500b41f0 msgr2=0x7f7650061af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:00.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.461+0000 7f7655cef700 1 --2- 172.21.15.17:0/3586691881 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76500b41f0 0x7f7650061af0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 rx=0x7f7640002f70 tx=0x7f7640006df0).stop 2022-08-01T19:44:00.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.461+0000 7f7655cef700 1 -- 172.21.15.17:0/3586691881 shutdown_connections 2022-08-01T19:44:00.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.461+0000 7f7655cef700 1 --2- 172.21.15.17:0/3586691881 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76500b41f0 0x7f7650061af0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:00.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.461+0000 7f7655cef700 1 --2- 172.21.15.17:0/3586691881 >> v2:172.21.15.77:3300/0 conn(0x7f76500b6420 0x7f7650061600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:00.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.461+0000 7f7655cef700 1 -- 172.21.15.17:0/3586691881 >> 172.21.15.17:0/3586691881 conn(0x7f765005c620 msgr2=0x7f765005ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:00.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.461+0000 7f7655cef700 1 -- 172.21.15.17:0/3586691881 shutdown_connections 2022-08-01T19:44:00.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.461+0000 7f7655cef700 1 -- 172.21.15.17:0/3586691881 wait complete. 2022-08-01T19:44:00.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.461+0000 7f7655cef700 1 Processor -- start 2022-08-01T19:44:00.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.461+0000 7f7655cef700 1 -- start start 2022-08-01T19:44:00.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.461+0000 7f7655cef700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7650061fe0 0x7f7650145f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:00.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.461+0000 7f7655cef700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f76500b41f0 0x7f765013e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:00.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.461+0000 7f7655cef700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76500b6420 0x7f76501567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:00.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.461+0000 7f7655cef700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765005f130 con 0x7f76500b6420 2022-08-01T19:44:00.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.461+0000 7f7655cef700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7650156110 con 0x7f7650061fe0 2022-08-01T19:44:00.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.461+0000 7f7655cef700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7650143860 con 0x7f76500b41f0 2022-08-01T19:44:00.487 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.461+0000 7f76554ee700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76500b6420 0x7f76501567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:00.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.461+0000 7f76554ee700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76500b6420 0x7f76501567e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45664/0 (socket says 172.21.15.17:45664) 2022-08-01T19:44:00.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.461+0000 7f7654ced700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7650061fe0 0x7f7650145f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:00.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.461+0000 7f76554ee700 1 -- 172.21.15.17:0/2197380318 learned_addr learned my addr 172.21.15.17:0/2197380318 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:44:00.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.461+0000 7f7647fff700 1 --2- 172.21.15.17:0/2197380318 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f76500b41f0 0x7f765013e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:00.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.461+0000 7f76554ee700 1 -- 172.21.15.17:0/2197380318 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f76500b41f0 msgr2=0x7f765013e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:00.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.461+0000 7f76554ee700 1 --2- 172.21.15.17:0/2197380318 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f76500b41f0 0x7f765013e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:00.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.461+0000 7f76554ee700 1 -- 172.21.15.17:0/2197380318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7650061fe0 msgr2=0x7f7650145f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:00.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.461+0000 7f76554ee700 1 --2- 172.21.15.17:0/2197380318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7650061fe0 0x7f7650145f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:00.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.461+0000 7f76554ee700 1 -- 172.21.15.17:0/2197380318 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764000b180 con 0x7f76500b6420 2022-08-01T19:44:00.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.461+0000 7f76554ee700 1 --2- 172.21.15.17:0/2197380318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76500b6420 0x7f76501567e0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 rx=0x7f7648000f40 tx=0x7f7648007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:00.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.465+0000 7f7645ffb700 1 -- 172.21.15.17:0/2197380318 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7648005b80 con 0x7f76500b6420 2022-08-01T19:44:00.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.465+0000 7f7645ffb700 1 -- 172.21.15.17:0/2197380318 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7648005ce0 con 0x7f76500b6420 2022-08-01T19:44:00.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.465+0000 7f7655cef700 1 -- 172.21.15.17:0/2197380318 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7650156340 con 0x7f76500b6420 2022-08-01T19:44:00.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.465+0000 7f7645ffb700 1 -- 172.21.15.17:0/2197380318 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76480072c0 con 0x7f76500b6420 2022-08-01T19:44:00.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.465+0000 7f7655cef700 1 -- 172.21.15.17:0/2197380318 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76500b0590 con 0x7f76500b6420 2022-08-01T19:44:00.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.465+0000 7f7655cef700 1 -- 172.21.15.17:0/2197380318 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76501564d0 con 0x7f76500b6420 2022-08-01T19:44:00.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.465+0000 7f7645ffb700 1 -- 172.21.15.17:0/2197380318 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f764801a060 con 0x7f76500b6420 2022-08-01T19:44:00.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.465+0000 7f7645ffb700 1 --2- 172.21.15.17:0/2197380318 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7630053b00 0x7f7630055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:00.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.465+0000 7f7645ffb700 1 -- 172.21.15.17:0/2197380318 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f764806b4c0 con 0x7f76500b6420 2022-08-01T19:44:00.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.465+0000 7f7654ced700 1 --2- 172.21.15.17:0/2197380318 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7630053b00 0x7f7630055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:00.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.469+0000 7f7654ced700 1 --2- 172.21.15.17:0/2197380318 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7630053b00 0x7f7630055fc0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f764c00b000 tx=0x7f764c006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:00.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.477+0000 7f7645ffb700 1 -- 172.21.15.17:0/2197380318 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7648068e90 con 0x7f76500b6420 2022-08-01T19:44:00.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.641+0000 7f7655cef700 1 -- 172.21.15.17:0/2197380318 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f76500b60e0 con 0x7f76500b6420 2022-08-01T19:44:00.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.653+0000 7f7645ffb700 1 -- 172.21.15.17:0/2197380318 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v38) v1 ==== 184+0+0 (secure 0 0 0) 0x7f76500b60e0 con 0x7f76500b6420 2022-08-01T19:44:00.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.657+0000 7f7655cef700 1 -- 172.21.15.17:0/2197380318 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7630053b00 msgr2=0x7f7630055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:00.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.657+0000 7f7655cef700 1 --2- 172.21.15.17:0/2197380318 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7630053b00 0x7f7630055fc0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f764c00b000 tx=0x7f764c006df0).stop 2022-08-01T19:44:00.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.657+0000 7f7655cef700 1 -- 172.21.15.17:0/2197380318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76500b6420 msgr2=0x7f76501567e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:00.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.657+0000 7f7655cef700 1 --2- 172.21.15.17:0/2197380318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76500b6420 0x7f76501567e0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 rx=0x7f7648000f40 tx=0x7f7648007be0).stop 2022-08-01T19:44:00.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.657+0000 7f7655cef700 1 -- 172.21.15.17:0/2197380318 shutdown_connections 2022-08-01T19:44:00.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.657+0000 7f7655cef700 1 --2- 172.21.15.17:0/2197380318 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7630053b00 0x7f7630055fc0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:00.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.657+0000 7f7655cef700 1 --2- 172.21.15.17:0/2197380318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76500b6420 0x7f76501567e0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:00.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.657+0000 7f7655cef700 1 --2- 172.21.15.17:0/2197380318 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f76500b41f0 0x7f765013e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:00.667 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.657+0000 7f7655cef700 1 --2- 172.21.15.17:0/2197380318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7650061fe0 0x7f7650145f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:00.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.657+0000 7f7655cef700 1 -- 172.21.15.17:0/2197380318 >> 172.21.15.17:0/2197380318 conn(0x7f765005c620 msgr2=0x7f76500b2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:00.668 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.657+0000 7f7655cef700 1 -- 172.21.15.17:0/2197380318 shutdown_connections 2022-08-01T19:44:00.669 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.657+0000 7f7655cef700 1 -- 172.21.15.17:0/2197380318 wait complete. 2022-08-01T19:44:00.669 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.0 2022-08-01T19:44:00.689 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-08-01T19:44:00.854 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.809+0000 7f28d81ee700 1 Processor -- start 2022-08-01T19:44:00.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.809+0000 7f28d81ee700 1 -- start start 2022-08-01T19:44:00.855 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.809+0000 7f28d81ee700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f28d00b6710 0x7f28d00b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:00.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.809+0000 7f28d81ee700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f28d00b6af0 0x7f28d012bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:00.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.809+0000 7f28d81ee700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28d0144240 con 0x7f28d00b4550 2022-08-01T19:44:00.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.809+0000 7f28d81ee700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28d0143d00 con 0x7f28d00b6710 2022-08-01T19:44:00.856 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.809+0000 7f28d81ee700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28d0143fa0 con 0x7f28d00b6af0 2022-08-01T19:44:00.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.809+0000 7f28d5f8a700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f28d00b6710 0x7f28d00b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:00.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.809+0000 7f28d5f8a700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f28d00b6710 0x7f28d00b5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45670/0 (socket says 172.21.15.17:45670) 2022-08-01T19:44:00.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.809+0000 7f28d5f8a700 1 -- 172.21.15.17:0/1287428850 learned_addr learned my addr 172.21.15.17:0/1287428850 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:44:00.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.809+0000 7f28d678b700 1 --2- 172.21.15.17:0/1287428850 >> v2:172.21.15.77:3300/0 conn(0x7f28d00b6af0 0x7f28d012bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:00.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.809+0000 7f28d5f8a700 1 -- 172.21.15.17:0/1287428850 >> v2:172.21.15.77:3300/0 conn(0x7f28d00b6af0 msgr2=0x7f28d012bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:44:00.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.809+0000 7f28d5f8a700 1 --2- 172.21.15.17:0/1287428850 >> v2:172.21.15.77:3300/0 conn(0x7f28d00b6af0 0x7f28d012bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:00.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.809+0000 7f28d5f8a700 1 -- 172.21.15.17:0/1287428850 >> v1:172.21.15.77:6789/0 conn(0x7f28d00b4550 legacy=0x7f28d0061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:00.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.809+0000 7f28d5f8a700 1 -- 172.21.15.17:0/1287428850 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d0061780 con 0x7f28d00b6710 2022-08-01T19:44:00.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.809+0000 7f28d5f8a700 1 --2- 172.21.15.17:0/1287428850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28d00b6710 0x7f28d00b5d80 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 rx=0x7f28d01432b0 tx=0x7f28c0006eb0).ready entity=mon.0 client_cookie=684b5eecb75e2916 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:00.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.813+0000 7f28d4f88700 1 -- 172.21.15.17:0/1287428850 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28c000d610 con 0x7f28d00b6710 2022-08-01T19:44:00.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.813+0000 7f28d4f88700 1 -- 172.21.15.17:0/1287428850 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c0000e90 con 0x7f28d00b6710 2022-08-01T19:44:00.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.813+0000 7f28d4f88700 1 -- 172.21.15.17:0/1287428850 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28c000d9a0 con 0x7f28d00b6710 2022-08-01T19:44:00.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.813+0000 7f28d81ee700 1 -- 172.21.15.17:0/1287428850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28d00b6710 msgr2=0x7f28d00b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:00.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.813+0000 7f28d81ee700 1 --2- 172.21.15.17:0/1287428850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28d00b6710 0x7f28d00b5d80 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 rx=0x7f28d01432b0 tx=0x7f28c0006eb0).stop 2022-08-01T19:44:00.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.813+0000 7f28d81ee700 1 -- 172.21.15.17:0/1287428850 shutdown_connections 2022-08-01T19:44:00.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.813+0000 7f28d81ee700 1 --2- 172.21.15.17:0/1287428850 >> v2:172.21.15.77:3300/0 conn(0x7f28d00b6af0 0x7f28d012bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:00.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.813+0000 7f28d81ee700 1 --2- 172.21.15.17:0/1287428850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28d00b6710 0x7f28d00b5d80 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:00.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.813+0000 7f28d81ee700 1 -- 172.21.15.17:0/1287428850 >> 172.21.15.17:0/1287428850 conn(0x7f28d005c620 msgr2=0x7f28d005ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:00.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.813+0000 7f28d81ee700 1 -- 172.21.15.17:0/1287428850 shutdown_connections 2022-08-01T19:44:00.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.813+0000 7f28d81ee700 1 -- 172.21.15.17:0/1287428850 wait complete. 2022-08-01T19:44:00.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.813+0000 7f28d81ee700 1 Processor -- start 2022-08-01T19:44:00.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.813+0000 7f28d81ee700 1 -- start start 2022-08-01T19:44:00.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.817+0000 7f28d81ee700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28d00b4550 0x7f28d00b3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:00.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.817+0000 7f28d81ee700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28d00b6710 0x7f28d013d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:00.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.817+0000 7f28d81ee700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f28d00b6af0 0x7f28d013d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:00.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.817+0000 7f28d81ee700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d00b7390 con 0x7f28d00b6710 2022-08-01T19:44:00.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.817+0000 7f28d81ee700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d01250e0 con 0x7f28d00b4550 2022-08-01T19:44:00.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.817+0000 7f28d81ee700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d013c760 con 0x7f28d00b6af0 2022-08-01T19:44:00.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.817+0000 7f28d5789700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28d00b6710 0x7f28d013d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:00.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.817+0000 7f28d5789700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28d00b6710 0x7f28d013d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45676/0 (socket says 172.21.15.17:45676) 2022-08-01T19:44:00.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.817+0000 7f28d5789700 1 -- 172.21.15.17:0/107063516 learned_addr learned my addr 172.21.15.17:0/107063516 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:44:00.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.817+0000 7f28d5f8a700 1 --2- 172.21.15.17:0/107063516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28d00b4550 0x7f28d00b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:00.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.817+0000 7f28d678b700 1 --2- 172.21.15.17:0/107063516 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f28d00b6af0 0x7f28d013d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:00.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.817+0000 7f28d5789700 1 -- 172.21.15.17:0/107063516 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f28d00b6af0 msgr2=0x7f28d013d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:00.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.817+0000 7f28d5789700 1 --2- 172.21.15.17:0/107063516 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f28d00b6af0 0x7f28d013d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:00.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.817+0000 7f28d5789700 1 -- 172.21.15.17:0/107063516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28d00b4550 msgr2=0x7f28d00b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:00.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.817+0000 7f28d5789700 1 --2- 172.21.15.17:0/107063516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28d00b4550 0x7f28d00b3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:00.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.817+0000 7f28d5789700 1 -- 172.21.15.17:0/107063516 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c000b1e0 con 0x7f28d00b6710 2022-08-01T19:44:00.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.817+0000 7f28d5789700 1 --2- 172.21.15.17:0/107063516 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28d00b6710 0x7f28d013d230 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 rx=0x7f28d00b1e10 tx=0x7f28cc007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:00.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.817+0000 7f28beffd700 1 -- 172.21.15.17:0/107063516 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28cc007030 con 0x7f28d00b6710 2022-08-01T19:44:00.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.817+0000 7f28beffd700 1 -- 172.21.15.17:0/107063516 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28cc007190 con 0x7f28d00b6710 2022-08-01T19:44:00.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.817+0000 7f28d81ee700 1 -- 172.21.15.17:0/107063516 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28d00b1e50 con 0x7f28d00b6710 2022-08-01T19:44:00.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.817+0000 7f28beffd700 1 -- 172.21.15.17:0/107063516 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28cc006510 con 0x7f28d00b6710 2022-08-01T19:44:00.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.817+0000 7f28d81ee700 1 -- 172.21.15.17:0/107063516 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28d013b4b0 con 0x7f28d00b6710 2022-08-01T19:44:00.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.817+0000 7f28d81ee700 1 -- 172.21.15.17:0/107063516 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28d013c460 con 0x7f28d00b6710 2022-08-01T19:44:00.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.817+0000 7f28beffd700 1 -- 172.21.15.17:0/107063516 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f28cc010030 con 0x7f28d00b6710 2022-08-01T19:44:00.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.821+0000 7f28beffd700 1 --2- 172.21.15.17:0/107063516 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f28b0053b00 0x7f28b0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:00.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.821+0000 7f28beffd700 1 -- 172.21.15.17:0/107063516 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f28cc06bcc0 con 0x7f28d00b6710 2022-08-01T19:44:00.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.821+0000 7f28d5f8a700 1 --2- 172.21.15.17:0/107063516 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f28b0053b00 0x7f28b0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:00.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.821+0000 7f28d5f8a700 1 --2- 172.21.15.17:0/107063516 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f28b0053b00 0x7f28b0055fc0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f28c001c000 tx=0x7f28c0015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:00.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.821+0000 7f28beffd700 1 -- 172.21.15.17:0/107063516 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f28cc042a60 con 0x7f28d00b6710 2022-08-01T19:44:00.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:00.993+0000 7f28d81ee700 1 -- 172.21.15.17:0/107063516 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f28d0061a20 con 0x7f28d00b6710 2022-08-01T19:44:01.003 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.001+0000 7f28beffd700 1 -- 172.21.15.17:0/107063516 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v39) v1 ==== 184+0+0 (secure 0 0 0) 0x7f28cc046080 con 0x7f28d00b6710 2022-08-01T19:44:01.008 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.005+0000 7f28d81ee700 1 -- 172.21.15.17:0/107063516 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f28b0053b00 msgr2=0x7f28b0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:01.010 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.005+0000 7f28d81ee700 1 --2- 172.21.15.17:0/107063516 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f28b0053b00 0x7f28b0055fc0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f28c001c000 tx=0x7f28c0015000).stop 2022-08-01T19:44:01.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.005+0000 7f28d81ee700 1 -- 172.21.15.17:0/107063516 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28d00b6710 msgr2=0x7f28d013d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:01.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.005+0000 7f28d81ee700 1 --2- 172.21.15.17:0/107063516 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28d00b6710 0x7f28d013d230 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 rx=0x7f28d00b1e10 tx=0x7f28cc007be0).stop 2022-08-01T19:44:01.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.005+0000 7f28d81ee700 1 -- 172.21.15.17:0/107063516 shutdown_connections 2022-08-01T19:44:01.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.005+0000 7f28d81ee700 1 --2- 172.21.15.17:0/107063516 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f28b0053b00 0x7f28b0055fc0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:01.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.005+0000 7f28d81ee700 1 --2- 172.21.15.17:0/107063516 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f28d00b6af0 0x7f28d013d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:01.013 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.005+0000 7f28d81ee700 1 --2- 172.21.15.17:0/107063516 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28d00b6710 0x7f28d013d230 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:01.013 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.005+0000 7f28d81ee700 1 --2- 172.21.15.17:0/107063516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28d00b4550 0x7f28d00b3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:01.014 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.005+0000 7f28d81ee700 1 -- 172.21.15.17:0/107063516 >> 172.21.15.17:0/107063516 conn(0x7f28d005c620 msgr2=0x7f28d01432f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:01.014 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.005+0000 7f28d81ee700 1 -- 172.21.15.17:0/107063516 shutdown_connections 2022-08-01T19:44:01.015 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.005+0000 7f28d81ee700 1 -- 172.21.15.17:0/107063516 wait complete. 2022-08-01T19:44:01.015 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.1 2022-08-01T19:44:01.023 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:44:01.135 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.129+0000 7f1bc104f700 1 Processor -- start 2022-08-01T19:44:01.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.129+0000 7f1bc104f700 1 -- start start 2022-08-01T19:44:01.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.129+0000 7f1bc104f700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1bbc0630c0 0x7f1bbc0b2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:01.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.129+0000 7f1bc104f700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f1bbc0b01b0 0x7f1bbc147d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:01.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.129+0000 7f1bc104f700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bbc0b26f0 con 0x7f1bbc1446d0 2022-08-01T19:44:01.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.129+0000 7f1bc104f700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bbc147400 con 0x7f1bbc0b01b0 2022-08-01T19:44:01.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.129+0000 7f1bc104f700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc0a9480 con 0x7f1bbc0630c0 2022-08-01T19:44:01.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.129+0000 7f1bc084e700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1bbc1446d0 0x7f1bbc14a9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37880/0 (socket says 172.21.15.17:37880) 2022-08-01T19:44:01.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.129+0000 7f1bc084e700 1 -- 172.21.15.17:0/2527423540 learned_addr learned my addr 172.21.15.17:0/2527423540 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:44:01.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.129+0000 7f1bbb7fe700 1 --2- 172.21.15.17:0/2527423540 >> v2:172.21.15.77:3300/0 conn(0x7f1bbc0b01b0 0x7f1bbc147d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:01.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.129+0000 7f1bbbfff700 1 --2- 172.21.15.17:0/2527423540 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1bbc0630c0 0x7f1bbc0b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:01.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.129+0000 7f1bbaffd700 1 -- 172.21.15.17:0/2527423540 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 586542372 0 0) 0x7f1bb4001490 con 0x7f1bbc1446d0 2022-08-01T19:44:01.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.133+0000 7f1bbaffd700 1 -- 172.21.15.17:0/2527423540 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ba0003260 con 0x7f1bbc1446d0 2022-08-01T19:44:01.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.133+0000 7f1bbb7fe700 1 -- 172.21.15.17:0/2527423540 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1bbc0630c0 msgr2=0x7f1bbc0b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:44:01.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.133+0000 7f1bbb7fe700 1 --2- 172.21.15.17:0/2527423540 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1bbc0630c0 0x7f1bbc0b2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:01.151 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.133+0000 7f1bbb7fe700 1 -- 172.21.15.17:0/2527423540 >> v1:172.21.15.17:6789/0 conn(0x7f1bbc1446d0 legacy=0x7f1bbc14a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:01.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.133+0000 7f1bbb7fe700 1 -- 172.21.15.17:0/2527423540 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bbc0a73d0 con 0x7f1bbc0b01b0 2022-08-01T19:44:01.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.133+0000 7f1bbb7fe700 1 --2- 172.21.15.17:0/2527423540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bbc0b01b0 0x7f1bbc147d30 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f1ba4002f70 tx=0x7f1ba4006df0).ready entity=mon.1 client_cookie=ddb2360b472ef5b8 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:01.152 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.133+0000 7f1bbaffd700 1 -- 172.21.15.17:0/2527423540 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ba400d4a0 con 0x7f1bbc0b01b0 2022-08-01T19:44:01.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.133+0000 7f1bbaffd700 1 -- 172.21.15.17:0/2527423540 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba4000e90 con 0x7f1bbc0b01b0 2022-08-01T19:44:01.153 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.133+0000 7f1bbaffd700 1 -- 172.21.15.17:0/2527423540 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ba400d7f0 con 0x7f1bbc0b01b0 2022-08-01T19:44:01.154 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.133+0000 7f1bc104f700 1 -- 172.21.15.17:0/2527423540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bbc0b01b0 msgr2=0x7f1bbc147d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:01.154 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.133+0000 7f1bc104f700 1 --2- 172.21.15.17:0/2527423540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bbc0b01b0 0x7f1bbc147d30 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f1ba4002f70 tx=0x7f1ba4006df0).stop 2022-08-01T19:44:01.154 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.133+0000 7f1bc104f700 1 -- 172.21.15.17:0/2527423540 shutdown_connections 2022-08-01T19:44:01.155 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.133+0000 7f1bc104f700 1 --2- 172.21.15.17:0/2527423540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bbc0b01b0 0x7f1bbc147d30 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:01.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.133+0000 7f1bc104f700 1 --2- 172.21.15.17:0/2527423540 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1bbc0630c0 0x7f1bbc0b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:01.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.133+0000 7f1bc104f700 1 -- 172.21.15.17:0/2527423540 >> 172.21.15.17:0/2527423540 conn(0x7f1bbc05c620 msgr2=0x7f1bbc05ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:01.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.133+0000 7f1bc104f700 1 -- 172.21.15.17:0/2527423540 shutdown_connections 2022-08-01T19:44:01.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.133+0000 7f1bc104f700 1 -- 172.21.15.17:0/2527423540 wait complete. 2022-08-01T19:44:01.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.133+0000 7f1bc104f700 1 Processor -- start 2022-08-01T19:44:01.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.133+0000 7f1bc104f700 1 -- start start 2022-08-01T19:44:01.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.133+0000 7f1bc104f700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1bbc0630c0 0x7f1bbc14a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:01.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.133+0000 7f1bc104f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bbc0b01b0 0x7f1bbc0b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:01.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.133+0000 7f1bc104f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bbc1446d0 0x7f1bbc157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:01.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.133+0000 7f1bc104f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc15a050 con 0x7f1bbc0b01b0 2022-08-01T19:44:01.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.133+0000 7f1bc104f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc14b2f0 con 0x7f1bbc1446d0 2022-08-01T19:44:01.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.133+0000 7f1bc104f700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc14a870 con 0x7f1bbc0630c0 2022-08-01T19:44:01.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.133+0000 7f1bbb7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bbc0b01b0 0x7f1bbc0b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:01.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.133+0000 7f1bbb7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bbc0b01b0 0x7f1bbc0b3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45690/0 (socket says 172.21.15.17:45690) 2022-08-01T19:44:01.171 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.137+0000 7f1bbb7fe700 1 -- 172.21.15.17:0/2825756827 learned_addr learned my addr 172.21.15.17:0/2825756827 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:44:01.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.137+0000 7f1bc084e700 1 --2- 172.21.15.17:0/2825756827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bbc1446d0 0x7f1bbc157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:01.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.137+0000 7f1bbbfff700 1 --2- 172.21.15.17:0/2825756827 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1bbc0630c0 0x7f1bbc14a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:01.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.137+0000 7f1bbb7fe700 1 -- 172.21.15.17:0/2825756827 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1bbc0630c0 msgr2=0x7f1bbc14a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:01.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.137+0000 7f1bbb7fe700 1 --2- 172.21.15.17:0/2825756827 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1bbc0630c0 0x7f1bbc14a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:01.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.137+0000 7f1bbb7fe700 1 -- 172.21.15.17:0/2825756827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bbc1446d0 msgr2=0x7f1bbc157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:01.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.137+0000 7f1bbb7fe700 1 --2- 172.21.15.17:0/2825756827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bbc1446d0 0x7f1bbc157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:01.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.137+0000 7f1bbb7fe700 1 -- 172.21.15.17:0/2825756827 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba400b180 con 0x7f1bbc0b01b0 2022-08-01T19:44:01.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.137+0000 7f1bbb7fe700 1 --2- 172.21.15.17:0/2825756827 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bbc0b01b0 0x7f1bbc0b3c00 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 rx=0x7f1ba4002f70 tx=0x7f1ba4014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:01.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.137+0000 7f1bb97fa700 1 -- 172.21.15.17:0/2825756827 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ba4012930 con 0x7f1bbc0b01b0 2022-08-01T19:44:01.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.137+0000 7f1bb97fa700 1 -- 172.21.15.17:0/2825756827 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba4012a90 con 0x7f1bbc0b01b0 2022-08-01T19:44:01.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.137+0000 7f1bc104f700 1 -- 172.21.15.17:0/2825756827 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bbc14b050 con 0x7f1bbc0b01b0 2022-08-01T19:44:01.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.137+0000 7f1bb97fa700 1 -- 172.21.15.17:0/2825756827 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ba4012df0 con 0x7f1bbc0b01b0 2022-08-01T19:44:01.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.137+0000 7f1bc104f700 1 -- 172.21.15.17:0/2825756827 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bbc1569e0 con 0x7f1bbc0b01b0 2022-08-01T19:44:01.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.137+0000 7f1bc104f700 1 -- 172.21.15.17:0/2825756827 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bbc15a1c0 con 0x7f1bbc0b01b0 2022-08-01T19:44:01.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.137+0000 7f1bb97fa700 1 -- 172.21.15.17:0/2825756827 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1ba401c070 con 0x7f1bbc0b01b0 2022-08-01T19:44:01.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.137+0000 7f1bb97fa700 1 --2- 172.21.15.17:0/2825756827 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1b94053950 0x7f1b94055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:01.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.137+0000 7f1bb97fa700 1 -- 172.21.15.17:0/2825756827 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1ba406cab0 con 0x7f1bbc0b01b0 2022-08-01T19:44:01.180 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.141+0000 7f1bbbfff700 1 --2- 172.21.15.17:0/2825756827 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1b94053950 0x7f1b94055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:01.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.141+0000 7f1bbbfff700 1 --2- 172.21.15.17:0/2825756827 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1b94053950 0x7f1b94055e10 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f1bac00b000 tx=0x7f1bac006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:01.181 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.145+0000 7f1bb97fa700 1 -- 172.21.15.17:0/2825756827 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1ba4043880 con 0x7f1bbc0b01b0 2022-08-01T19:44:01.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.305+0000 7f1bc104f700 1 -- 172.21.15.17:0/2825756827 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1bbc1475f0 con 0x7f1bbc0b01b0 2022-08-01T19:44:01.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.309+0000 7f1bb97fa700 1 -- 172.21.15.17:0/2825756827 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+2486 (secure 0 0 0) 0x7f1ba4019030 con 0x7f1bbc0b01b0 2022-08-01T19:44:01.310 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:44:01.311 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":83,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8665,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/608187432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":608187432},{"type":"v1","addr":"172.21.15.77:6835","nonce":608187432}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":81},{"gid":8673,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2764139073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2764139073},{"type":"v1","addr":"172.21.15.17:6833","nonce":2764139073}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":80}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:43:58.713330+0000","modified":"2022-08-01T19:43:59.717120+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8682":{"gid":8682,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/75309005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":75309005},{"type":"v1","addr":"172.21.15.17:6835","nonce":75309005}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2022-08-01T19:44:01.312 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.309+0000 7f1bc104f700 1 -- 172.21.15.17:0/2825756827 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1b94053950 msgr2=0x7f1b94055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:01.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.309+0000 7f1bc104f700 1 --2- 172.21.15.17:0/2825756827 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1b94053950 0x7f1b94055e10 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f1bac00b000 tx=0x7f1bac006df0).stop 2022-08-01T19:44:01.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.309+0000 7f1bc104f700 1 -- 172.21.15.17:0/2825756827 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bbc0b01b0 msgr2=0x7f1bbc0b3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:01.313 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.309+0000 7f1bc104f700 1 --2- 172.21.15.17:0/2825756827 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bbc0b01b0 0x7f1bbc0b3c00 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 rx=0x7f1ba4002f70 tx=0x7f1ba4014000).stop 2022-08-01T19:44:01.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.309+0000 7f1bc104f700 1 -- 172.21.15.17:0/2825756827 shutdown_connections 2022-08-01T19:44:01.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.309+0000 7f1bc104f700 1 --2- 172.21.15.17:0/2825756827 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1b94053950 0x7f1b94055e10 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:01.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.309+0000 7f1bc104f700 1 --2- 172.21.15.17:0/2825756827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bbc1446d0 0x7f1bbc157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:01.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.309+0000 7f1bc104f700 1 --2- 172.21.15.17:0/2825756827 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bbc0b01b0 0x7f1bbc0b3c00 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:01.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.309+0000 7f1bc104f700 1 --2- 172.21.15.17:0/2825756827 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1bbc0630c0 0x7f1bbc14a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:01.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.309+0000 7f1bc104f700 1 -- 172.21.15.17:0/2825756827 >> 172.21.15.17:0/2825756827 conn(0x7f1bbc05c620 msgr2=0x7f1bbc05ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:01.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.309+0000 7f1bc104f700 1 -- 172.21.15.17:0/2825756827 shutdown_connections 2022-08-01T19:44:01.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.309+0000 7f1bc104f700 1 -- 172.21.15.17:0/2825756827 wait complete. 2022-08-01T19:44:01.318 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 83 2022-08-01T19:44:01.632 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 83, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:43:58.713330+0000', 'modified': '2022-08-01T19:43:59.717120+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8682}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8682': {'gid': 8682, 'name': 'c', 'rank': 0, 'incarnation': 82, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/75309005', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 75309005}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 75309005}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:44:01.633 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:44:01.634 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T19:44:01.751 INFO:teuthology.orchestra.run.smithi017.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":8682,"want_state":"up:active","state":"up:active","rank_uptime":3.0258113029999998,"mdsmap_epoch":83,"osdmap_epoch":158,"osdmap_epoch_barrier":158,"uptime":7.3841839309999999} 2022-08-01T19:44:01.752 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 8682, "mdsmap_epoch": 83, "osdmap_epoch": 158, "osdmap_epoch_barrier": 158, "rank_uptime": 3.025811303, "state": "up:active", "uptime": 7.384183931, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:44:01.753 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:44:01.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.857+0000 7fbb4956a700 1 Processor -- start 2022-08-01T19:44:01.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.857+0000 7fbb4956a700 1 -- start start 2022-08-01T19:44:01.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.857+0000 7fbb4956a700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbb440b2710 0x7fbb4414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:01.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.857+0000 7fbb4956a700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb440a7250 con 0x7fbb440b01a0 2022-08-01T19:44:01.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.861+0000 7fbb4956a700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb440a7480 con 0x7fbb440a8c50 2022-08-01T19:44:01.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.861+0000 7fbb4956a700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb440a7720 con 0x7fbb440b2710 2022-08-01T19:44:01.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.861+0000 7fbb43fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbb440b2710 0x7fbb4414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:01.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.861+0000 7fbb437fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fbb440b01a0 0x7fbb440a8540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37890/0 (socket says 172.21.15.17:37890) 2022-08-01T19:44:01.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.861+0000 7fbb437fe700 1 -- 172.21.15.17:0/387514137 learned_addr learned my addr 172.21.15.17:0/387514137 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:44:01.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.861+0000 7fbb42ffd700 1 -- 172.21.15.17:0/387514137 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1801080604 0 0) 0x7fbb440a7250 con 0x7fbb440b01a0 2022-08-01T19:44:01.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.861+0000 7fbb42ffd700 1 -- 172.21.15.17:0/387514137 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb30003260 con 0x7fbb440b01a0 2022-08-01T19:44:01.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.861+0000 7fbb43fff700 1 -- 172.21.15.17:0/387514137 >> v1:172.21.15.77:6789/0 conn(0x7fbb440a8c50 legacy=0x7fbb44151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:01.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.861+0000 7fbb42ffd700 1 -- 172.21.15.17:0/387514137 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 145210324 0 0) 0x7fbb440a7480 con 0x7fbb440a8c50 2022-08-01T19:44:01.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.861+0000 7fbb43fff700 1 -- 172.21.15.17:0/387514137 >> v1:172.21.15.17:6789/0 conn(0x7fbb440b01a0 legacy=0x7fbb440a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:01.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.861+0000 7fbb43fff700 1 -- 172.21.15.17:0/387514137 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb440a79c0 con 0x7fbb440b2710 2022-08-01T19:44:01.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.861+0000 7fbb43fff700 1 --2- 172.21.15.17:0/387514137 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb440b2710 0x7fbb4414afd0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 rx=0x7fbb34001b80 tx=0x7fbb38006df0).ready entity=mon.0 client_cookie=8515c6c39d03d311 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:01.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.861+0000 7fbb42ffd700 1 -- 172.21.15.17:0/387514137 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb340019e0 con 0x7fbb440b2710 2022-08-01T19:44:01.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.861+0000 7fbb42ffd700 1 -- 172.21.15.17:0/387514137 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb38000e90 con 0x7fbb440b2710 2022-08-01T19:44:01.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.861+0000 7fbb42ffd700 1 -- 172.21.15.17:0/387514137 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb38011400 con 0x7fbb440b2710 2022-08-01T19:44:01.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.861+0000 7fbb4956a700 1 -- 172.21.15.17:0/387514137 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb440b2710 msgr2=0x7fbb4414afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:01.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.861+0000 7fbb4956a700 1 --2- 172.21.15.17:0/387514137 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb440b2710 0x7fbb4414afd0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 rx=0x7fbb34001b80 tx=0x7fbb38006df0).stop 2022-08-01T19:44:01.881 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.861+0000 7fbb4956a700 1 -- 172.21.15.17:0/387514137 shutdown_connections 2022-08-01T19:44:01.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.861+0000 7fbb4956a700 1 --2- 172.21.15.17:0/387514137 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb440b2710 0x7fbb4414afd0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:01.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.861+0000 7fbb4956a700 1 -- 172.21.15.17:0/387514137 >> 172.21.15.17:0/387514137 conn(0x7fbb44060310 msgr2=0x7fbb440b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:01.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.861+0000 7fbb4956a700 1 -- 172.21.15.17:0/387514137 shutdown_connections 2022-08-01T19:44:01.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.861+0000 7fbb4956a700 1 -- 172.21.15.17:0/387514137 wait complete. 2022-08-01T19:44:01.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.861+0000 7fbb4956a700 1 Processor -- start 2022-08-01T19:44:01.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.861+0000 7fbb4956a700 1 -- start start 2022-08-01T19:44:01.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.865+0000 7fbb4956a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb440a8c50 0x7fbb44148b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:01.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.865+0000 7fbb4956a700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbb440b01a0 0x7fbb44146610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:01.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.865+0000 7fbb4956a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb440b2710 0x7fbb44146b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:01.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.865+0000 7fbb4956a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb44156b70 con 0x7fbb440b2710 2022-08-01T19:44:01.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.865+0000 7fbb4956a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb44156e10 con 0x7fbb440a8c50 2022-08-01T19:44:01.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.865+0000 7fbb4956a700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb441570b0 con 0x7fbb440b01a0 2022-08-01T19:44:01.885 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.865+0000 7fbb48d69700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb440b2710 0x7fbb44146b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:01.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.865+0000 7fbb48d69700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb440b2710 0x7fbb44146b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45704/0 (socket says 172.21.15.17:45704) 2022-08-01T19:44:01.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.865+0000 7fbb48d69700 1 -- 172.21.15.17:0/3408936865 learned_addr learned my addr 172.21.15.17:0/3408936865 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:44:01.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.865+0000 7fbb43fff700 1 --2- 172.21.15.17:0/3408936865 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb440a8c50 0x7fbb44148b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:01.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.865+0000 7fbb437fe700 1 --2- 172.21.15.17:0/3408936865 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbb440b01a0 0x7fbb44146610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:01.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.865+0000 7fbb48d69700 1 -- 172.21.15.17:0/3408936865 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbb440b01a0 msgr2=0x7fbb44146610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:01.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.865+0000 7fbb48d69700 1 --2- 172.21.15.17:0/3408936865 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbb440b01a0 0x7fbb44146610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:01.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.865+0000 7fbb48d69700 1 -- 172.21.15.17:0/3408936865 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb440a8c50 msgr2=0x7fbb44148b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:01.887 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.865+0000 7fbb48d69700 1 --2- 172.21.15.17:0/3408936865 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb440a8c50 0x7fbb44148b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:01.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.865+0000 7fbb48d69700 1 -- 172.21.15.17:0/3408936865 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3800b180 con 0x7fbb440b2710 2022-08-01T19:44:01.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.865+0000 7fbb437fe700 1 --2- 172.21.15.17:0/3408936865 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbb440b01a0 0x7fbb44146610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T19:44:01.888 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.865+0000 7fbb48d69700 1 --2- 172.21.15.17:0/3408936865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb440b2710 0x7fbb44146b00 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 rx=0x7fbb3c000ed0 tx=0x7fbb3c007a10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:01.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.865+0000 7fbb417fa700 1 -- 172.21.15.17:0/3408936865 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb3c00ce00 con 0x7fbb440b2710 2022-08-01T19:44:01.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.865+0000 7fbb417fa700 1 -- 172.21.15.17:0/3408936865 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb3c00f470 con 0x7fbb440b2710 2022-08-01T19:44:01.889 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.865+0000 7fbb4956a700 1 -- 172.21.15.17:0/3408936865 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb44157350 con 0x7fbb440b2710 2022-08-01T19:44:01.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.865+0000 7fbb417fa700 1 -- 172.21.15.17:0/3408936865 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb3c00f7d0 con 0x7fbb440b2710 2022-08-01T19:44:01.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.865+0000 7fbb4956a700 1 -- 172.21.15.17:0/3408936865 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb44146ff0 con 0x7fbb440b2710 2022-08-01T19:44:01.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.865+0000 7fbb4956a700 1 -- 172.21.15.17:0/3408936865 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb441574e0 con 0x7fbb440b2710 2022-08-01T19:44:01.890 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.865+0000 7fbb417fa700 1 -- 172.21.15.17:0/3408936865 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbb3c019030 con 0x7fbb440b2710 2022-08-01T19:44:01.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.869+0000 7fbb417fa700 1 --2- 172.21.15.17:0/3408936865 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbb24053aa0 0x7fbb24055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:01.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.869+0000 7fbb417fa700 1 -- 172.21.15.17:0/3408936865 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fbb3c06b460 con 0x7fbb440b2710 2022-08-01T19:44:01.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.869+0000 7fbb43fff700 1 --2- 172.21.15.17:0/3408936865 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbb24053aa0 0x7fbb24055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:01.891 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.869+0000 7fbb43fff700 1 --2- 172.21.15.17:0/3408936865 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbb24053aa0 0x7fbb24055f60 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7fbb38017000 tx=0x7fbb440a7610).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:01.892 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:01.873+0000 7fbb417fa700 1 -- 172.21.15.17:0/3408936865 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbb3c042220 con 0x7fbb440b2710 2022-08-01T19:44:02.002 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.001+0000 7fbb4956a700 1 -- 172.21.15.17:0/3408936865 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb440574e0 con 0x7fbb440b2710 2022-08-01T19:44:02.007 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.005+0000 7fbb417fa700 1 -- 172.21.15.17:0/3408936865 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) v1 ==== 74+0+16285 (secure 0 0 0) 0x7fbb3c045840 con 0x7fbb440b2710 2022-08-01T19:44:02.008 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:44:02.008 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":158,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:43:59.716124+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:43:57.196125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2022-08-01T19:43:57.958027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:44:02.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.005+0000 7fbb4956a700 1 -- 172.21.15.17:0/3408936865 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbb24053aa0 msgr2=0x7fbb24055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:02.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.005+0000 7fbb4956a700 1 --2- 172.21.15.17:0/3408936865 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbb24053aa0 0x7fbb24055f60 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7fbb38017000 tx=0x7fbb440a7610).stop 2022-08-01T19:44:02.018 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.005+0000 7fbb4956a700 1 -- 172.21.15.17:0/3408936865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb440b2710 msgr2=0x7fbb44146b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:02.019 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.005+0000 7fbb4956a700 1 --2- 172.21.15.17:0/3408936865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb440b2710 0x7fbb44146b00 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 rx=0x7fbb3c000ed0 tx=0x7fbb3c007a10).stop 2022-08-01T19:44:02.020 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.005+0000 7fbb4956a700 1 -- 172.21.15.17:0/3408936865 shutdown_connections 2022-08-01T19:44:02.023 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.005+0000 7fbb4956a700 1 --2- 172.21.15.17:0/3408936865 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbb24053aa0 0x7fbb24055f60 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:02.024 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.005+0000 7fbb4956a700 1 --2- 172.21.15.17:0/3408936865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb440b2710 0x7fbb44146b00 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:02.025 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.005+0000 7fbb4956a700 1 --2- 172.21.15.17:0/3408936865 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbb440b01a0 0x7fbb44146610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:02.026 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.005+0000 7fbb4956a700 1 --2- 172.21.15.17:0/3408936865 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb440a8c50 0x7fbb44148b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:02.027 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.005+0000 7fbb4956a700 1 -- 172.21.15.17:0/3408936865 >> 172.21.15.17:0/3408936865 conn(0x7fbb44060310 msgr2=0x7fbb440ad390 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:02.037 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.005+0000 7fbb4956a700 1 -- 172.21.15.17:0/3408936865 shutdown_connections 2022-08-01T19:44:02.038 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.005+0000 7fbb4956a700 1 -- 172.21.15.17:0/3408936865 wait complete. 2022-08-01T19:44:02.039 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T19:43:57.196125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '158', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T19:43:57.958027+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '158', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T19:44:02.040 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:44:02.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.149+0000 7fe9d04b3700 1 Processor -- start 2022-08-01T19:44:02.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.149+0000 7fe9d04b3700 1 -- start start 2022-08-01T19:44:02.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.149+0000 7fe9d04b3700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe9c8146ae0 0x7fe9c81403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:02.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.149+0000 7fe9d04b3700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe9c80b4310 0x7fe9c81408b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:02.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.149+0000 7fe9d04b3700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c8143cf0 con 0x7fe9c80b4820 2022-08-01T19:44:02.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.149+0000 7fe9d04b3700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9c81476a0 con 0x7fe9c80b4310 2022-08-01T19:44:02.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.149+0000 7fe9d04b3700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c8143f90 con 0x7fe9c8146ae0 2022-08-01T19:44:02.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.153+0000 7fe9cda4e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe9c80b4310 0x7fe9c81408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:02.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.153+0000 7fe9cda4e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe9c80b4310 0x7fe9c81408b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45712/0 (socket says 172.21.15.17:45712) 2022-08-01T19:44:02.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.153+0000 7fe9cda4e700 1 -- 172.21.15.17:0/1131554608 learned_addr learned my addr 172.21.15.17:0/1131554608 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:44:02.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.153+0000 7fe9cea50700 1 --1- 172.21.15.17:0/1131554608 >> v1:172.21.15.17:6789/0 conn(0x7fe9c80b4820 0x7fe9c80b5060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37906/0 (socket says 172.21.15.17:37906) 2022-08-01T19:44:02.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.153+0000 7fe9ce24f700 1 --2- 172.21.15.17:0/1131554608 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe9c8146ae0 0x7fe9c81403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:02.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.153+0000 7fe9cda4e700 1 -- 172.21.15.17:0/1131554608 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe9c8146ae0 msgr2=0x7fe9c81403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:44:02.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.153+0000 7fe9cda4e700 1 --2- 172.21.15.17:0/1131554608 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe9c8146ae0 0x7fe9c81403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:02.179 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.153+0000 7fe9cda4e700 1 -- 172.21.15.17:0/1131554608 >> v1:172.21.15.17:6789/0 conn(0x7fe9c80b4820 legacy=0x7fe9c80b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:02.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.153+0000 7fe9cda4e700 1 -- 172.21.15.17:0/1131554608 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c80b0e20 con 0x7fe9c80b4310 2022-08-01T19:44:02.188 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.153+0000 7fe9cda4e700 1 --2- 172.21.15.17:0/1131554608 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9c80b4310 0x7fe9c81408b0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 rx=0x7fe9b8002f70 tx=0x7fe9b8006eb0).ready entity=mon.0 client_cookie=51eaab34c372f6f2 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:02.188 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.153+0000 7fe9cd24d700 1 -- 172.21.15.17:0/1131554608 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9b800d5c0 con 0x7fe9c80b4310 2022-08-01T19:44:02.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.153+0000 7fe9cd24d700 1 -- 172.21.15.17:0/1131554608 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b8000e90 con 0x7fe9c80b4310 2022-08-01T19:44:02.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.153+0000 7fe9cd24d700 1 -- 172.21.15.17:0/1131554608 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9b800d950 con 0x7fe9c80b4310 2022-08-01T19:44:02.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.153+0000 7fe9d04b3700 1 -- 172.21.15.17:0/1131554608 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9c80b4310 msgr2=0x7fe9c81408b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:02.190 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.153+0000 7fe9d04b3700 1 --2- 172.21.15.17:0/1131554608 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9c80b4310 0x7fe9c81408b0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 rx=0x7fe9b8002f70 tx=0x7fe9b8006eb0).stop 2022-08-01T19:44:02.190 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.153+0000 7fe9d04b3700 1 -- 172.21.15.17:0/1131554608 shutdown_connections 2022-08-01T19:44:02.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.153+0000 7fe9d04b3700 1 --2- 172.21.15.17:0/1131554608 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9c80b4310 0x7fe9c81408b0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:02.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.153+0000 7fe9d04b3700 1 --2- 172.21.15.17:0/1131554608 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe9c8146ae0 0x7fe9c81403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:02.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.153+0000 7fe9d04b3700 1 -- 172.21.15.17:0/1131554608 >> 172.21.15.17:0/1131554608 conn(0x7fe9c805c620 msgr2=0x7fe9c805ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:02.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.153+0000 7fe9d04b3700 1 -- 172.21.15.17:0/1131554608 shutdown_connections 2022-08-01T19:44:02.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.153+0000 7fe9d04b3700 1 -- 172.21.15.17:0/1131554608 wait complete. 2022-08-01T19:44:02.193 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.157+0000 7fe9d04b3700 1 Processor -- start 2022-08-01T19:44:02.195 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.157+0000 7fe9d04b3700 1 -- start start 2022-08-01T19:44:02.196 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.157+0000 7fe9d04b3700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe9c80b4310 0x7fe9c8143270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:02.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.157+0000 7fe9d04b3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9c80b4820 0x7fe9c80b85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:02.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.157+0000 7fe9d04b3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9c8146ae0 0x7fe9c81526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:02.197 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.157+0000 7fe9d04b3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c80b0800 con 0x7fe9c80b4820 2022-08-01T19:44:02.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.157+0000 7fe9d04b3700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c80b0020 con 0x7fe9c8146ae0 2022-08-01T19:44:02.198 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.157+0000 7fe9d04b3700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c80b02c0 con 0x7fe9c80b4310 2022-08-01T19:44:02.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.157+0000 7fe9cda4e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9c80b4820 0x7fe9c80b85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:02.199 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.157+0000 7fe9cda4e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9c80b4820 0x7fe9c80b85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45718/0 (socket says 172.21.15.17:45718) 2022-08-01T19:44:02.200 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.157+0000 7fe9cda4e700 1 -- 172.21.15.17:0/3538846646 learned_addr learned my addr 172.21.15.17:0/3538846646 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:44:02.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.157+0000 7fe9ce24f700 1 --2- 172.21.15.17:0/3538846646 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe9c80b4310 0x7fe9c8143270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:02.201 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.157+0000 7fe9cea50700 1 --2- 172.21.15.17:0/3538846646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9c8146ae0 0x7fe9c81526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:02.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.157+0000 7fe9cda4e700 1 -- 172.21.15.17:0/3538846646 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe9c80b4310 msgr2=0x7fe9c8143270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:02.202 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.157+0000 7fe9cda4e700 1 --2- 172.21.15.17:0/3538846646 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe9c80b4310 0x7fe9c8143270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:02.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.157+0000 7fe9cda4e700 1 -- 172.21.15.17:0/3538846646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9c8146ae0 msgr2=0x7fe9c81526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:02.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.157+0000 7fe9cda4e700 1 --2- 172.21.15.17:0/3538846646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9c8146ae0 0x7fe9c81526a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:02.203 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.157+0000 7fe9cda4e700 1 -- 172.21.15.17:0/3538846646 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b800b1e0 con 0x7fe9c80b4820 2022-08-01T19:44:02.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.157+0000 7fe9cea50700 1 --2- 172.21.15.17:0/3538846646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9c8146ae0 0x7fe9c81526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:44:02.204 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.157+0000 7fe9cda4e700 1 --2- 172.21.15.17:0/3538846646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9c80b4820 0x7fe9c80b85e0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 rx=0x7fe9b8002f70 tx=0x7fe9b8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:02.205 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.157+0000 7fe9b77fe700 1 -- 172.21.15.17:0/3538846646 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9b8011860 con 0x7fe9c80b4820 2022-08-01T19:44:02.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.157+0000 7fe9b77fe700 1 -- 172.21.15.17:0/3538846646 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b80119c0 con 0x7fe9c80b4820 2022-08-01T19:44:02.207 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.157+0000 7fe9d04b3700 1 -- 172.21.15.17:0/3538846646 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c80b0560 con 0x7fe9c80b4820 2022-08-01T19:44:02.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.157+0000 7fe9b77fe700 1 -- 172.21.15.17:0/3538846646 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9b8011d20 con 0x7fe9c80b4820 2022-08-01T19:44:02.208 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.157+0000 7fe9d04b3700 1 -- 172.21.15.17:0/3538846646 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c80b8ad0 con 0x7fe9c80b4820 2022-08-01T19:44:02.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.161+0000 7fe9d04b3700 1 -- 172.21.15.17:0/3538846646 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9c80574e0 con 0x7fe9c80b4820 2022-08-01T19:44:02.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.161+0000 7fe9b77fe700 1 -- 172.21.15.17:0/3538846646 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe9b801a040 con 0x7fe9c80b4820 2022-08-01T19:44:02.210 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.161+0000 7fe9b77fe700 1 --2- 172.21.15.17:0/3538846646 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe9a8053950 0x7fe9a8055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:02.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.161+0000 7fe9b77fe700 1 -- 172.21.15.17:0/3538846646 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fe9b806cf20 con 0x7fe9c80b4820 2022-08-01T19:44:02.211 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.161+0000 7fe9ce24f700 1 --2- 172.21.15.17:0/3538846646 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe9a8053950 0x7fe9a8055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:02.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.161+0000 7fe9ce24f700 1 --2- 172.21.15.17:0/3538846646 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe9a8053950 0x7fe9a8055e10 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7fe9c400c000 tx=0x7fe9c4008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:02.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.169+0000 7fe9b77fe700 1 -- 172.21.15.17:0/3538846646 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe9b8043ce0 con 0x7fe9c80b4820 2022-08-01T19:44:02.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.293+0000 7fe9d04b3700 1 -- 172.21.15.17:0/3538846646 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe9c8061080 con 0x7fe9c80b4820 2022-08-01T19:44:02.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.293+0000 7fe9b77fe700 1 -- 172.21.15.17:0/3538846646 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v158) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe9b8023020 con 0x7fe9c80b4820 2022-08-01T19:44:02.298 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:44:02.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.293+0000 7fe9d04b3700 1 -- 172.21.15.17:0/3538846646 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe9a8053950 msgr2=0x7fe9a8055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:02.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.293+0000 7fe9d04b3700 1 --2- 172.21.15.17:0/3538846646 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe9a8053950 0x7fe9a8055e10 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7fe9c400c000 tx=0x7fe9c4008040).stop 2022-08-01T19:44:02.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.293+0000 7fe9d04b3700 1 -- 172.21.15.17:0/3538846646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9c80b4820 msgr2=0x7fe9c80b85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:02.300 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.293+0000 7fe9d04b3700 1 --2- 172.21.15.17:0/3538846646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9c80b4820 0x7fe9c80b85e0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 rx=0x7fe9b8002f70 tx=0x7fe9b8015000).stop 2022-08-01T19:44:02.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.297+0000 7fe9d04b3700 1 -- 172.21.15.17:0/3538846646 shutdown_connections 2022-08-01T19:44:02.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.297+0000 7fe9d04b3700 1 --2- 172.21.15.17:0/3538846646 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fe9a8053950 0x7fe9a8055e10 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:02.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.297+0000 7fe9d04b3700 1 --2- 172.21.15.17:0/3538846646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9c8146ae0 0x7fe9c81526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:02.301 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.297+0000 7fe9d04b3700 1 --2- 172.21.15.17:0/3538846646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9c80b4820 0x7fe9c80b85e0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:02.302 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.297+0000 7fe9d04b3700 1 --2- 172.21.15.17:0/3538846646 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fe9c80b4310 0x7fe9c8143270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:02.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.297+0000 7fe9d04b3700 1 -- 172.21.15.17:0/3538846646 >> 172.21.15.17:0/3538846646 conn(0x7fe9c805c620 msgr2=0x7fe9c80b9940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:02.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.297+0000 7fe9d04b3700 1 -- 172.21.15.17:0/3538846646 shutdown_connections 2022-08-01T19:44:02.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.297+0000 7fe9d04b3700 1 -- 172.21.15.17:0/3538846646 wait complete. 2022-08-01T19:44:02.318 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:44:02.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.437+0000 7fd2a89eb700 1 Processor -- start 2022-08-01T19:44:02.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.437+0000 7fd2a89eb700 1 -- start start 2022-08-01T19:44:02.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.437+0000 7fd2a89eb700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd2a40b13f0 0x7fd2a40afd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:02.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.437+0000 7fd2a89eb700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2a40b1920 0x7fd2a40b0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:02.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.437+0000 7fd2a89eb700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2a40b0740 con 0x7fd2a40b2710 2022-08-01T19:44:02.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.437+0000 7fd2a89eb700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2a40adf10 con 0x7fd2a40b13f0 2022-08-01T19:44:02.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.437+0000 7fd2a89eb700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a40ae1b0 con 0x7fd2a40b1920 2022-08-01T19:44:02.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.437+0000 7fd2a259c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd2a40b13f0 0x7fd2a40afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:02.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.437+0000 7fd2a2d9d700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd2a40b2710 0x7fd2a40abca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37918/0 (socket says 172.21.15.17:37918) 2022-08-01T19:44:02.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.437+0000 7fd2a2d9d700 1 -- 172.21.15.17:0/404558144 learned_addr learned my addr 172.21.15.17:0/404558144 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:44:02.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.437+0000 7fd2a359e700 1 --2- 172.21.15.17:0/404558144 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2a40b1920 0x7fd2a40b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:02.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.437+0000 7fd2a1d9b700 1 -- 172.21.15.17:0/404558144 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 460638558 0 0) 0x7fd2a40b0740 con 0x7fd2a40b2710 2022-08-01T19:44:02.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.437+0000 7fd2a1d9b700 1 -- 172.21.15.17:0/404558144 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd284003260 con 0x7fd2a40b2710 2022-08-01T19:44:02.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.437+0000 7fd2a359e700 1 -- 172.21.15.17:0/404558144 >> v2:172.21.15.17:3300/0 conn(0x7fd2a40b13f0 msgr2=0x7fd2a40afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:44:02.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.437+0000 7fd2a359e700 1 --2- 172.21.15.17:0/404558144 >> v2:172.21.15.17:3300/0 conn(0x7fd2a40b13f0 0x7fd2a40afd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:02.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.437+0000 7fd2a359e700 1 -- 172.21.15.17:0/404558144 >> v1:172.21.15.17:6789/0 conn(0x7fd2a40b2710 legacy=0x7fd2a40abca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:02.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.437+0000 7fd2a359e700 1 -- 172.21.15.17:0/404558144 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a40a9520 con 0x7fd2a40b1920 2022-08-01T19:44:02.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.437+0000 7fd2a359e700 1 --2- 172.21.15.17:0/404558144 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2a40b1920 0x7fd2a40b0250 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7fd29c0032f0 tx=0x7fd29c0091d0).ready entity=mon.2 client_cookie=38c6e4e4b0a016bc server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:02.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.437+0000 7fd2a1d9b700 1 -- 172.21.15.17:0/404558144 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd29c00b600 con 0x7fd2a40b1920 2022-08-01T19:44:02.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.437+0000 7fd2a1d9b700 1 -- 172.21.15.17:0/404558144 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd29c00b760 con 0x7fd2a40b1920 2022-08-01T19:44:02.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.437+0000 7fd2a1d9b700 1 -- 172.21.15.17:0/404558144 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd29c00b8d0 con 0x7fd2a40b1920 2022-08-01T19:44:02.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.441+0000 7fd2a89eb700 1 -- 172.21.15.17:0/404558144 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2a40b1920 msgr2=0x7fd2a40b0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:02.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.441+0000 7fd2a89eb700 1 --2- 172.21.15.17:0/404558144 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2a40b1920 0x7fd2a40b0250 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7fd29c0032f0 tx=0x7fd29c0091d0).stop 2022-08-01T19:44:02.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.441+0000 7fd2a89eb700 1 -- 172.21.15.17:0/404558144 shutdown_connections 2022-08-01T19:44:02.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.441+0000 7fd2a89eb700 1 --2- 172.21.15.17:0/404558144 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2a40b1920 0x7fd2a40b0250 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:02.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.441+0000 7fd2a89eb700 1 --2- 172.21.15.17:0/404558144 >> v2:172.21.15.17:3300/0 conn(0x7fd2a40b13f0 0x7fd2a40afd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:02.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.441+0000 7fd2a89eb700 1 -- 172.21.15.17:0/404558144 >> 172.21.15.17:0/404558144 conn(0x7fd2a4060310 msgr2=0x7fd2a40b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:02.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.441+0000 7fd2a89eb700 1 -- 172.21.15.17:0/404558144 shutdown_connections 2022-08-01T19:44:02.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.441+0000 7fd2a89eb700 1 -- 172.21.15.17:0/404558144 wait complete. 2022-08-01T19:44:02.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.441+0000 7fd2a89eb700 1 Processor -- start 2022-08-01T19:44:02.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.441+0000 7fd2a89eb700 1 -- start start 2022-08-01T19:44:02.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.441+0000 7fd2a89eb700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2a40b13f0 0x7fd2a40668e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:02.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.441+0000 7fd2a89eb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2a40b1920 0x7fd2a413bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:02.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.441+0000 7fd2a89eb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2a40b2710 0x7fd2a413c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:02.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.441+0000 7fd2a89eb700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a40a8eb0 con 0x7fd2a40b1920 2022-08-01T19:44:02.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.441+0000 7fd2a89eb700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a413eee0 con 0x7fd2a40b2710 2022-08-01T19:44:02.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.441+0000 7fd2a89eb700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a413f180 con 0x7fd2a40b13f0 2022-08-01T19:44:02.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.441+0000 7fd2a259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2a40b1920 0x7fd2a413bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:02.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.441+0000 7fd2a259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2a40b1920 0x7fd2a413bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45730/0 (socket says 172.21.15.17:45730) 2022-08-01T19:44:02.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.441+0000 7fd2a259c700 1 -- 172.21.15.17:0/3339060306 learned_addr learned my addr 172.21.15.17:0/3339060306 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:44:02.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.445+0000 7fd2a2d9d700 1 --2- 172.21.15.17:0/3339060306 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2a40b13f0 0x7fd2a40668e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:02.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.445+0000 7fd2a359e700 1 --2- 172.21.15.17:0/3339060306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2a40b2710 0x7fd2a413c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:02.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.445+0000 7fd2a259c700 1 -- 172.21.15.17:0/3339060306 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2a40b13f0 msgr2=0x7fd2a40668e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:02.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.445+0000 7fd2a259c700 1 --2- 172.21.15.17:0/3339060306 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2a40b13f0 0x7fd2a40668e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:02.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.445+0000 7fd2a259c700 1 -- 172.21.15.17:0/3339060306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2a40b2710 msgr2=0x7fd2a413c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:02.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.445+0000 7fd2a259c700 1 --2- 172.21.15.17:0/3339060306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2a40b2710 0x7fd2a413c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:02.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.445+0000 7fd2a259c700 1 -- 172.21.15.17:0/3339060306 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd29c009470 con 0x7fd2a40b1920 2022-08-01T19:44:02.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.445+0000 7fd2a2d9d700 1 --2- 172.21.15.17:0/3339060306 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2a40b13f0 0x7fd2a40668e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T19:44:02.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.445+0000 7fd2a259c700 1 --2- 172.21.15.17:0/3339060306 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2a40b1920 0x7fd2a413bc40 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 rx=0x7fd28c00c000 tx=0x7fd28c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:02.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.445+0000 7fd29bfff700 1 -- 172.21.15.17:0/3339060306 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd28c010780 con 0x7fd2a40b1920 2022-08-01T19:44:02.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.445+0000 7fd29bfff700 1 -- 172.21.15.17:0/3339060306 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28c0108e0 con 0x7fd2a40b1920 2022-08-01T19:44:02.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.445+0000 7fd2a89eb700 1 -- 172.21.15.17:0/3339060306 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a40b0f00 con 0x7fd2a40b1920 2022-08-01T19:44:02.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.445+0000 7fd29bfff700 1 -- 172.21.15.17:0/3339060306 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd28c010c40 con 0x7fd2a40b1920 2022-08-01T19:44:02.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.445+0000 7fd2a89eb700 1 -- 172.21.15.17:0/3339060306 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a40674d0 con 0x7fd2a40b1920 2022-08-01T19:44:02.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.445+0000 7fd2a89eb700 1 -- 172.21.15.17:0/3339060306 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2a40574e0 con 0x7fd2a40b1920 2022-08-01T19:44:02.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.445+0000 7fd29bfff700 1 -- 172.21.15.17:0/3339060306 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd28c01c460 con 0x7fd2a40b1920 2022-08-01T19:44:02.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.449+0000 7fd29bfff700 1 --2- 172.21.15.17:0/3339060306 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd288053980 0x7fd288055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:02.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.449+0000 7fd29bfff700 1 -- 172.21.15.17:0/3339060306 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd28c06c3a0 con 0x7fd2a40b1920 2022-08-01T19:44:02.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.449+0000 7fd2a2d9d700 1 --2- 172.21.15.17:0/3339060306 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd288053980 0x7fd288055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:02.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.449+0000 7fd2a2d9d700 1 --2- 172.21.15.17:0/3339060306 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd288053980 0x7fd288055e40 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7fd29400b000 tx=0x7fd294009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:02.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.457+0000 7fd29bfff700 1 -- 172.21.15.17:0/3339060306 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd28c043100 con 0x7fd2a40b1920 2022-08-01T19:44:02.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.581+0000 7fd2a89eb700 1 -- 172.21.15.17:0/3339060306 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd2a40a8ff0 con 0x7fd2a40b1920 2022-08-01T19:44:02.588 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.585+0000 7fd29bfff700 1 -- 172.21.15.17:0/3339060306 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v158) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd28c046720 con 0x7fd2a40b1920 2022-08-01T19:44:02.589 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:44:02.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.585+0000 7fd2a89eb700 1 -- 172.21.15.17:0/3339060306 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd288053980 msgr2=0x7fd288055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:02.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.585+0000 7fd2a89eb700 1 --2- 172.21.15.17:0/3339060306 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd288053980 0x7fd288055e40 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7fd29400b000 tx=0x7fd294009040).stop 2022-08-01T19:44:02.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.585+0000 7fd2a89eb700 1 -- 172.21.15.17:0/3339060306 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2a40b1920 msgr2=0x7fd2a413bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:02.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.585+0000 7fd2a89eb700 1 --2- 172.21.15.17:0/3339060306 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2a40b1920 0x7fd2a413bc40 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 rx=0x7fd28c00c000 tx=0x7fd28c006df0).stop 2022-08-01T19:44:02.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.585+0000 7fd2a89eb700 1 -- 172.21.15.17:0/3339060306 shutdown_connections 2022-08-01T19:44:02.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.585+0000 7fd2a89eb700 1 --2- 172.21.15.17:0/3339060306 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd288053980 0x7fd288055e40 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:02.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.585+0000 7fd2a89eb700 1 --2- 172.21.15.17:0/3339060306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2a40b2710 0x7fd2a413c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:02.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.585+0000 7fd2a89eb700 1 --2- 172.21.15.17:0/3339060306 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2a40b1920 0x7fd2a413bc40 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:02.596 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.585+0000 7fd2a89eb700 1 --2- 172.21.15.17:0/3339060306 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd2a40b13f0 0x7fd2a40668e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:02.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.585+0000 7fd2a89eb700 1 -- 172.21.15.17:0/3339060306 >> 172.21.15.17:0/3339060306 conn(0x7fd2a4060310 msgr2=0x7fd2a4143630 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:02.597 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.585+0000 7fd2a89eb700 1 -- 172.21.15.17:0/3339060306 shutdown_connections 2022-08-01T19:44:02.598 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.585+0000 7fd2a89eb700 1 -- 172.21.15.17:0/3339060306 wait complete. 2022-08-01T19:44:02.600 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T19:44:02.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.709+0000 7f5f7aaa6700 1 Processor -- start 2022-08-01T19:44:02.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.709+0000 7f5f7aaa6700 1 -- start start 2022-08-01T19:44:02.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.713+0000 7f5f7aaa6700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5f740b01a0 0x7f5f740a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:02.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.713+0000 7f5f7aaa6700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f5f740afd60 0x7f5f740a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:02.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.713+0000 7f5f7aaa6700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f740a7250 con 0x7f5f740b2710 2022-08-01T19:44:02.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.713+0000 7f5f7aaa6700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f740a7480 con 0x7f5f740afd60 2022-08-01T19:44:02.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.713+0000 7f5f7aaa6700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f740a7720 con 0x7f5f740b01a0 2022-08-01T19:44:02.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.713+0000 7f5f79aa4700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5f740b2710 0x7f5f7414afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37932/0 (socket says 172.21.15.17:37932) 2022-08-01T19:44:02.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.713+0000 7f5f79aa4700 1 -- 172.21.15.17:0/170529014 learned_addr learned my addr 172.21.15.17:0/170529014 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:44:02.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.713+0000 7f5f7a2a5700 1 --2- 172.21.15.17:0/170529014 >> v2:172.21.15.77:3300/0 conn(0x7f5f740afd60 0x7f5f740a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:02.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.713+0000 7f5f792a3700 1 --2- 172.21.15.17:0/170529014 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5f740b01a0 0x7f5f740a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:02.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.713+0000 7f5f78aa2700 1 -- 172.21.15.17:0/170529014 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 405718273 0 0) 0x7f5f740a7250 con 0x7f5f740b2710 2022-08-01T19:44:02.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.713+0000 7f5f78aa2700 1 -- 172.21.15.17:0/170529014 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f60003260 con 0x7f5f740b2710 2022-08-01T19:44:02.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.713+0000 7f5f7a2a5700 1 -- 172.21.15.17:0/170529014 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5f740b01a0 msgr2=0x7f5f740a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:44:02.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.713+0000 7f5f7a2a5700 1 --2- 172.21.15.17:0/170529014 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5f740b01a0 0x7f5f740a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:02.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.713+0000 7f5f7a2a5700 1 -- 172.21.15.17:0/170529014 >> v1:172.21.15.17:6789/0 conn(0x7f5f740b2710 legacy=0x7f5f7414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:02.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.713+0000 7f5f78aa2700 1 -- 172.21.15.17:0/170529014 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1179095446 0 0) 0x7f5f60003260 con 0x7f5f740b2710 2022-08-01T19:44:02.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.713+0000 7f5f7a2a5700 1 -- 172.21.15.17:0/170529014 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f740a79c0 con 0x7f5f740afd60 2022-08-01T19:44:02.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.713+0000 7f5f792a3700 1 --2- 172.21.15.17:0/170529014 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5f740b01a0 0x7f5f740a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:44:02.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.713+0000 7f5f7a2a5700 1 --2- 172.21.15.17:0/170529014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f740afd60 0x7f5f740a8a30 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f5f68001800 tx=0x7f5f6c006df0).ready entity=mon.1 client_cookie=93ece7e37d26d58a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:02.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.713+0000 7f5f78aa2700 1 -- 172.21.15.17:0/170529014 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f68001660 con 0x7f5f740afd60 2022-08-01T19:44:02.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.713+0000 7f5f78aa2700 1 -- 172.21.15.17:0/170529014 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6c000e90 con 0x7f5f740afd60 2022-08-01T19:44:02.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.713+0000 7f5f78aa2700 1 -- 172.21.15.17:0/170529014 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f6c012460 con 0x7f5f740afd60 2022-08-01T19:44:02.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.713+0000 7f5f7aaa6700 1 -- 172.21.15.17:0/170529014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f740afd60 msgr2=0x7f5f740a8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:02.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.713+0000 7f5f7aaa6700 1 --2- 172.21.15.17:0/170529014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f740afd60 0x7f5f740a8a30 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f5f68001800 tx=0x7f5f6c006df0).stop 2022-08-01T19:44:02.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.713+0000 7f5f7aaa6700 1 -- 172.21.15.17:0/170529014 shutdown_connections 2022-08-01T19:44:02.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.713+0000 7f5f7aaa6700 1 --2- 172.21.15.17:0/170529014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f740afd60 0x7f5f740a8a30 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:02.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.713+0000 7f5f7aaa6700 1 --2- 172.21.15.17:0/170529014 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5f740b01a0 0x7f5f740a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:02.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.713+0000 7f5f7aaa6700 1 -- 172.21.15.17:0/170529014 >> 172.21.15.17:0/170529014 conn(0x7f5f74060310 msgr2=0x7f5f740b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:02.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.713+0000 7f5f7aaa6700 1 -- 172.21.15.17:0/170529014 shutdown_connections 2022-08-01T19:44:02.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.713+0000 7f5f7aaa6700 1 -- 172.21.15.17:0/170529014 wait complete. 2022-08-01T19:44:02.730 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.717+0000 7f5f7aaa6700 1 Processor -- start 2022-08-01T19:44:02.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.717+0000 7f5f7aaa6700 1 -- start start 2022-08-01T19:44:02.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.717+0000 7f5f7aaa6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f740afd60 0x7f5f7414d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:02.731 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.717+0000 7f5f7aaa6700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5f740b01a0 0x7f5f7413f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:02.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.717+0000 7f5f7aaa6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f740b2710 0x7f5f74140e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:02.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.717+0000 7f5f7aaa6700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f74140ba0 con 0x7f5f740afd60 2022-08-01T19:44:02.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.717+0000 7f5f7aaa6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7414b980 con 0x7f5f740b2710 2022-08-01T19:44:02.732 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.717+0000 7f5f7aaa6700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7414bc20 con 0x7f5f740b01a0 2022-08-01T19:44:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.717+0000 7f5f79aa4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f740afd60 0x7f5f7414d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.717+0000 7f5f79aa4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f740afd60 0x7f5f7414d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45744/0 (socket says 172.21.15.17:45744) 2022-08-01T19:44:02.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.717+0000 7f5f79aa4700 1 -- 172.21.15.17:0/2479557402 learned_addr learned my addr 172.21.15.17:0/2479557402 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:44:02.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.717+0000 7f5f7a2a5700 1 --2- 172.21.15.17:0/2479557402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f740b2710 0x7f5f74140e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:02.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.717+0000 7f5f792a3700 1 --2- 172.21.15.17:0/2479557402 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5f740b01a0 0x7f5f7413f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:02.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.717+0000 7f5f79aa4700 1 -- 172.21.15.17:0/2479557402 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5f740b01a0 msgr2=0x7f5f7413f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:02.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.717+0000 7f5f79aa4700 1 --2- 172.21.15.17:0/2479557402 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5f740b01a0 0x7f5f7413f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.717+0000 7f5f79aa4700 1 -- 172.21.15.17:0/2479557402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f740b2710 msgr2=0x7f5f74140e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.717+0000 7f5f79aa4700 1 --2- 172.21.15.17:0/2479557402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f740b2710 0x7f5f74140e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.717+0000 7f5f79aa4700 1 -- 172.21.15.17:0/2479557402 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f64009470 con 0x7f5f740afd60 2022-08-01T19:44:02.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.717+0000 7f5f79aa4700 1 --2- 172.21.15.17:0/2479557402 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f740afd60 0x7f5f7414d5e0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 rx=0x7f5f680018c0 tx=0x7f5f68001e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:02.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.717+0000 7f5f72ffd700 1 -- 172.21.15.17:0/2479557402 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f68007170 con 0x7f5f740afd60 2022-08-01T19:44:02.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.717+0000 7f5f72ffd700 1 -- 172.21.15.17:0/2479557402 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f680072d0 con 0x7f5f740afd60 2022-08-01T19:44:02.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.717+0000 7f5f7aaa6700 1 -- 172.21.15.17:0/2479557402 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f6c00b180 con 0x7f5f740afd60 2022-08-01T19:44:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.717+0000 7f5f72ffd700 1 -- 172.21.15.17:0/2479557402 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f68004710 con 0x7f5f740afd60 2022-08-01T19:44:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.717+0000 7f5f7aaa6700 1 -- 172.21.15.17:0/2479557402 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f74140770 con 0x7f5f740afd60 2022-08-01T19:44:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.721+0000 7f5f7aaa6700 1 -- 172.21.15.17:0/2479557402 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f740574e0 con 0x7f5f740afd60 2022-08-01T19:44:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.721+0000 7f5f72ffd700 1 -- 172.21.15.17:0/2479557402 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5f68004870 con 0x7f5f740afd60 2022-08-01T19:44:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.721+0000 7f5f72ffd700 1 --2- 172.21.15.17:0/2479557402 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5f54053b30 0x7f5f54055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.721+0000 7f5f72ffd700 1 -- 172.21.15.17:0/2479557402 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5f6806b950 con 0x7f5f740afd60 2022-08-01T19:44:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.721+0000 7f5f792a3700 1 --2- 172.21.15.17:0/2479557402 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5f54053b30 0x7f5f54055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.721+0000 7f5f792a3700 1 --2- 172.21.15.17:0/2479557402 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5f54053b30 0x7f5f54055ff0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7f5f6400b000 tx=0x7f5f640091d0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:02.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.729+0000 7f5f72ffd700 1 -- 172.21.15.17:0/2479557402 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5f68042710 con 0x7f5f740afd60 2022-08-01T19:44:02.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.857+0000 7f5f7aaa6700 1 -- 172.21.15.17:0/2479557402 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5f74069a70 con 0x7f5f740afd60 2022-08-01T19:44:02.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.857+0000 7f5f72ffd700 1 -- 172.21.15.17:0/2479557402 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v158) v1 ==== 104+0+10 (secure 0 0 0) 0x7f5f68045d30 con 0x7f5f740afd60 2022-08-01T19:44:02.862 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:44:02.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.861+0000 7f5f7aaa6700 1 -- 172.21.15.17:0/2479557402 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5f54053b30 msgr2=0x7f5f54055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:02.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.861+0000 7f5f7aaa6700 1 --2- 172.21.15.17:0/2479557402 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5f54053b30 0x7f5f54055ff0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7f5f6400b000 tx=0x7f5f640091d0).stop 2022-08-01T19:44:02.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.861+0000 7f5f7aaa6700 1 -- 172.21.15.17:0/2479557402 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f740afd60 msgr2=0x7f5f7414d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:02.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.861+0000 7f5f7aaa6700 1 --2- 172.21.15.17:0/2479557402 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f740afd60 0x7f5f7414d5e0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 rx=0x7f5f680018c0 tx=0x7f5f68001e10).stop 2022-08-01T19:44:02.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.861+0000 7f5f7aaa6700 1 -- 172.21.15.17:0/2479557402 shutdown_connections 2022-08-01T19:44:02.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.861+0000 7f5f7aaa6700 1 --2- 172.21.15.17:0/2479557402 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5f54053b30 0x7f5f54055ff0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:02.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.861+0000 7f5f7aaa6700 1 --2- 172.21.15.17:0/2479557402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f740b2710 0x7f5f74140e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:02.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.861+0000 7f5f7aaa6700 1 --2- 172.21.15.17:0/2479557402 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5f740b01a0 0x7f5f7413f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:02.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.861+0000 7f5f7aaa6700 1 --2- 172.21.15.17:0/2479557402 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f740afd60 0x7f5f7414d5e0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:02.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.861+0000 7f5f7aaa6700 1 -- 172.21.15.17:0/2479557402 >> 172.21.15.17:0/2479557402 conn(0x7f5f74060310 msgr2=0x7f5f74067510 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:02.870 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.861+0000 7f5f7aaa6700 1 -- 172.21.15.17:0/2479557402 shutdown_connections 2022-08-01T19:44:02.870 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.861+0000 7f5f7aaa6700 1 -- 172.21.15.17:0/2479557402 wait complete. 2022-08-01T19:44:02.877 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T19:44:02.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.985+0000 7f34f2b33700 1 Processor -- start 2022-08-01T19:44:02.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.985+0000 7f34f2b33700 1 -- start start 2022-08-01T19:44:02.992 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.989+0000 7f34f2b33700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f34ec0b0850 0x7f34ec0b1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:02.992 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.989+0000 7f34f2b33700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34ec0b1df0 con 0x7f34ec0ac3f0 2022-08-01T19:44:02.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.989+0000 7f34f2b33700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34ec0a9ed0 con 0x7f34ec0a9af0 2022-08-01T19:44:02.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.989+0000 7f34f2b33700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34ec0b2ad0 con 0x7f34ec0b0850 2022-08-01T19:44:02.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.989+0000 7f34f10d0700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f34ec0b0850 0x7f34ec0b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:02.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.989+0000 7f34ebfff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f34ec0ac3f0 0x7f34ec0af500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37946/0 (socket says 172.21.15.17:37946) 2022-08-01T19:44:02.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.989+0000 7f34ebfff700 1 -- 172.21.15.17:0/2906076584 learned_addr learned my addr 172.21.15.17:0/2906076584 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:44:02.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.989+0000 7f34f10d0700 1 -- 172.21.15.17:0/2906076584 >> v1:172.21.15.77:6789/0 conn(0x7f34ec0a9af0 legacy=0x7f34ec0aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:02.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.989+0000 7f34eb7fe700 1 -- 172.21.15.17:0/2906076584 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3800043870 0 0) 0x7f34dc001110 con 0x7f34ec0ac3f0 2022-08-01T19:44:02.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.989+0000 7f34f10d0700 1 -- 172.21.15.17:0/2906076584 >> v1:172.21.15.17:6789/0 conn(0x7f34ec0ac3f0 legacy=0x7f34ec0af500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:02.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.989+0000 7f34f10d0700 1 -- 172.21.15.17:0/2906076584 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ec0b0250 con 0x7f34ec0b0850 2022-08-01T19:44:02.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.989+0000 7f34f10d0700 1 --2- 172.21.15.17:0/2906076584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34ec0b0850 0x7f34ec0b1900 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 rx=0x7f34e40032f0 tx=0x7f34e40091d0).ready entity=mon.0 client_cookie=7a2a2925633fdae5 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:02.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.989+0000 7f34eb7fe700 1 -- 172.21.15.17:0/2906076584 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34e4009ea0 con 0x7f34ec0b0850 2022-08-01T19:44:02.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.989+0000 7f34eb7fe700 1 -- 172.21.15.17:0/2906076584 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e400b8a0 con 0x7f34ec0b0850 2022-08-01T19:44:02.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.989+0000 7f34eb7fe700 1 -- 172.21.15.17:0/2906076584 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34e400ba10 con 0x7f34ec0b0850 2022-08-01T19:44:03.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.989+0000 7f34f2b33700 1 -- 172.21.15.17:0/2906076584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34ec0b0850 msgr2=0x7f34ec0b1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:03.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.989+0000 7f34f2b33700 1 --2- 172.21.15.17:0/2906076584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34ec0b0850 0x7f34ec0b1900 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 rx=0x7f34e40032f0 tx=0x7f34e40091d0).stop 2022-08-01T19:44:03.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.989+0000 7f34f2b33700 1 -- 172.21.15.17:0/2906076584 shutdown_connections 2022-08-01T19:44:03.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.989+0000 7f34f2b33700 1 --2- 172.21.15.17:0/2906076584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34ec0b0850 0x7f34ec0b1900 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:03.002 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.989+0000 7f34f2b33700 1 -- 172.21.15.17:0/2906076584 >> 172.21.15.17:0/2906076584 conn(0x7f34ec05f840 msgr2=0x7f34ec05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:03.002 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.989+0000 7f34f2b33700 1 -- 172.21.15.17:0/2906076584 shutdown_connections 2022-08-01T19:44:03.002 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.989+0000 7f34f2b33700 1 -- 172.21.15.17:0/2906076584 wait complete. 2022-08-01T19:44:03.002 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.993+0000 7f34f2b33700 1 Processor -- start 2022-08-01T19:44:03.003 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.993+0000 7f34f2b33700 1 -- start start 2022-08-01T19:44:03.003 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.997+0000 7f34f2b33700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34ec0a9af0 0x7f34ec144a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:03.003 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.997+0000 7f34f2b33700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f34ec0ac3f0 0x7f34ec150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:03.003 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.997+0000 7f34f2b33700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34ec0b0850 0x7f34ec09f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:03.003 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.997+0000 7f34f2b33700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec066230 con 0x7f34ec0b0850 2022-08-01T19:44:03.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.997+0000 7f34f2b33700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec064440 con 0x7f34ec0a9af0 2022-08-01T19:44:03.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.997+0000 7f34f2b33700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec0646e0 con 0x7f34ec0ac3f0 2022-08-01T19:44:03.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.997+0000 7f34f10d0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34ec0b0850 0x7f34ec09f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:03.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.997+0000 7f34f10d0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34ec0b0850 0x7f34ec09f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45762/0 (socket says 172.21.15.17:45762) 2022-08-01T19:44:03.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.997+0000 7f34f08cf700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34ec0a9af0 0x7f34ec144a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:03.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.997+0000 7f34f10d0700 1 -- 172.21.15.17:0/644973639 learned_addr learned my addr 172.21.15.17:0/644973639 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:44:03.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.997+0000 7f34ebfff700 1 --2- 172.21.15.17:0/644973639 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f34ec0ac3f0 0x7f34ec150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:03.005 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.997+0000 7f34f10d0700 1 -- 172.21.15.17:0/644973639 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f34ec0ac3f0 msgr2=0x7f34ec150ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:03.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.997+0000 7f34f10d0700 1 --2- 172.21.15.17:0/644973639 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f34ec0ac3f0 0x7f34ec150ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:03.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.997+0000 7f34f10d0700 1 -- 172.21.15.17:0/644973639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34ec0a9af0 msgr2=0x7f34ec144a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:03.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.997+0000 7f34f10d0700 1 --2- 172.21.15.17:0/644973639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34ec0a9af0 0x7f34ec144a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:03.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.997+0000 7f34f10d0700 1 -- 172.21.15.17:0/644973639 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e4009470 con 0x7f34ec0b0850 2022-08-01T19:44:03.006 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.997+0000 7f34f10d0700 1 --2- 172.21.15.17:0/644973639 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34ec0b0850 0x7f34ec09f180 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 rx=0x7f34e40032f0 tx=0x7f34e4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:03.007 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.997+0000 7f34e9ffb700 1 -- 172.21.15.17:0/644973639 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34e4009ea0 con 0x7f34ec0b0850 2022-08-01T19:44:03.007 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.997+0000 7f34e9ffb700 1 -- 172.21.15.17:0/644973639 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e401f3f0 con 0x7f34ec0b0850 2022-08-01T19:44:03.007 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.997+0000 7f34e9ffb700 1 -- 172.21.15.17:0/644973639 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34e4011ea0 con 0x7f34ec0b0850 2022-08-01T19:44:03.008 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.997+0000 7f34f2b33700 1 -- 172.21.15.17:0/644973639 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34ec064980 con 0x7f34ec0b0850 2022-08-01T19:44:03.008 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.997+0000 7f34f2b33700 1 -- 172.21.15.17:0/644973639 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34ec09ece0 con 0x7f34ec0b0850 2022-08-01T19:44:03.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.997+0000 7f34e9ffb700 1 -- 172.21.15.17:0/644973639 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f34e400f070 con 0x7f34ec0b0850 2022-08-01T19:44:03.011 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:02.997+0000 7f34f2b33700 1 -- 172.21.15.17:0/644973639 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34ec09eea0 con 0x7f34ec0b0850 2022-08-01T19:44:03.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.001+0000 7f34e9ffb700 1 --2- 172.21.15.17:0/644973639 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f34cc0539e0 0x7f34cc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:03.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.001+0000 7f34e9ffb700 1 -- 172.21.15.17:0/644973639 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f34e406c970 con 0x7f34ec0b0850 2022-08-01T19:44:03.012 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.001+0000 7f34f08cf700 1 --2- 172.21.15.17:0/644973639 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f34cc0539e0 0x7f34cc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:03.013 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.001+0000 7f34f08cf700 1 --2- 172.21.15.17:0/644973639 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f34cc0539e0 0x7f34cc055ea0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7f34e000d000 tx=0x7f34e0007b90).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:03.013 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.005+0000 7f34e9ffb700 1 -- 172.21.15.17:0/644973639 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f34e4043730 con 0x7f34ec0b0850 2022-08-01T19:44:03.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.133+0000 7f34f2b33700 1 -- 172.21.15.17:0/644973639 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f34ec0574e0 con 0x7f34ec0b0850 2022-08-01T19:44:03.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.137+0000 7f34e9ffb700 1 -- 172.21.15.17:0/644973639 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v158) v1 ==== 100+0+10 (secure 0 0 0) 0x7f34e40150a0 con 0x7f34ec0b0850 2022-08-01T19:44:03.140 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:44:03.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.137+0000 7f34f2b33700 1 -- 172.21.15.17:0/644973639 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f34cc0539e0 msgr2=0x7f34cc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:03.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.137+0000 7f34f2b33700 1 --2- 172.21.15.17:0/644973639 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f34cc0539e0 0x7f34cc055ea0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7f34e000d000 tx=0x7f34e0007b90).stop 2022-08-01T19:44:03.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.137+0000 7f34f2b33700 1 -- 172.21.15.17:0/644973639 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34ec0b0850 msgr2=0x7f34ec09f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:03.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.137+0000 7f34f2b33700 1 --2- 172.21.15.17:0/644973639 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34ec0b0850 0x7f34ec09f180 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 rx=0x7f34e40032f0 tx=0x7f34e4016000).stop 2022-08-01T19:44:03.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.137+0000 7f34f2b33700 1 -- 172.21.15.17:0/644973639 shutdown_connections 2022-08-01T19:44:03.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.137+0000 7f34f2b33700 1 --2- 172.21.15.17:0/644973639 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f34cc0539e0 0x7f34cc055ea0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:03.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.137+0000 7f34f2b33700 1 --2- 172.21.15.17:0/644973639 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34ec0b0850 0x7f34ec09f180 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:03.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.137+0000 7f34f2b33700 1 --2- 172.21.15.17:0/644973639 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f34ec0ac3f0 0x7f34ec150ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:03.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.137+0000 7f34f2b33700 1 --2- 172.21.15.17:0/644973639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34ec0a9af0 0x7f34ec144a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:03.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.137+0000 7f34f2b33700 1 -- 172.21.15.17:0/644973639 >> 172.21.15.17:0/644973639 conn(0x7f34ec05f840 msgr2=0x7f34ec1428b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:03.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.137+0000 7f34f2b33700 1 -- 172.21.15.17:0/644973639 shutdown_connections 2022-08-01T19:44:03.150 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.137+0000 7f34f2b33700 1 -- 172.21.15.17:0/644973639 wait complete. 2022-08-01T19:44:03.153 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T19:44:03.154 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:44:03.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.257+0000 7f71a0af0700 1 Processor -- start 2022-08-01T19:44:03.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.257+0000 7f71a0af0700 1 -- start start 2022-08-01T19:44:03.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.257+0000 7f71a0af0700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f719c13d3f0 0x7f719c05c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:03.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.257+0000 7f71a0af0700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f719c1471b0 con 0x7f719c0b1fb0 2022-08-01T19:44:03.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.257+0000 7f71a0af0700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f719c147450 con 0x7f719c0b2790 2022-08-01T19:44:03.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.257+0000 7f71a0af0700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f719c0ad5d0 con 0x7f719c13d3f0 2022-08-01T19:44:03.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.257+0000 7f7192ffd700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f719c0b1fb0 0x7f719c13cce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37960/0 (socket says 172.21.15.17:37960) 2022-08-01T19:44:03.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.257+0000 7f7192ffd700 1 -- 172.21.15.17:0/3119144072 learned_addr learned my addr 172.21.15.17:0/3119144072 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:44:03.271 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.257+0000 7f719bfff700 1 --2- 172.21.15.17:0/3119144072 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f719c13d3f0 0x7f719c05c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:03.271 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.257+0000 7f719affd700 1 -- 172.21.15.17:0/3119144072 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2023830911 0 0) 0x7f719c1471b0 con 0x7f719c0b1fb0 2022-08-01T19:44:03.272 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.257+0000 7f719affd700 1 -- 172.21.15.17:0/3119144072 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f717c003260 con 0x7f719c0b1fb0 2022-08-01T19:44:03.272 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.261+0000 7f719affd700 1 -- 172.21.15.17:0/3119144072 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 335696347 0 0) 0x7f719c147450 con 0x7f719c0b2790 2022-08-01T19:44:03.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.261+0000 7f719affd700 1 -- 172.21.15.17:0/3119144072 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f719c1471b0 con 0x7f719c0b2790 2022-08-01T19:44:03.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.261+0000 7f719bfff700 1 -- 172.21.15.17:0/3119144072 >> v1:172.21.15.77:6789/0 conn(0x7f719c0b2790 legacy=0x7f719c143870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:03.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.261+0000 7f719bfff700 1 -- 172.21.15.17:0/3119144072 >> v1:172.21.15.17:6789/0 conn(0x7f719c0b1fb0 legacy=0x7f719c13cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:03.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.261+0000 7f719bfff700 1 -- 172.21.15.17:0/3119144072 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f719c143130 con 0x7f719c13d3f0 2022-08-01T19:44:03.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.261+0000 7f719bfff700 1 --2- 172.21.15.17:0/3119144072 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f719c13d3f0 0x7f719c05c620 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f71940032f0 tx=0x7f71940091d0).ready entity=mon.2 client_cookie=9fa8c45003406016 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:03.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.261+0000 7f719affd700 1 -- 172.21.15.17:0/3119144072 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f719400b640 con 0x7f719c13d3f0 2022-08-01T19:44:03.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.261+0000 7f719affd700 1 -- 172.21.15.17:0/3119144072 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f719400b7a0 con 0x7f719c13d3f0 2022-08-01T19:44:03.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.261+0000 7f719affd700 1 -- 172.21.15.17:0/3119144072 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f719400b910 con 0x7f719c13d3f0 2022-08-01T19:44:03.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.261+0000 7f71a0af0700 1 -- 172.21.15.17:0/3119144072 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f719c13d3f0 msgr2=0x7f719c05c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:03.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.261+0000 7f71a0af0700 1 --2- 172.21.15.17:0/3119144072 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f719c13d3f0 0x7f719c05c620 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f71940032f0 tx=0x7f71940091d0).stop 2022-08-01T19:44:03.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.261+0000 7f71a0af0700 1 -- 172.21.15.17:0/3119144072 shutdown_connections 2022-08-01T19:44:03.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.261+0000 7f71a0af0700 1 --2- 172.21.15.17:0/3119144072 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f719c13d3f0 0x7f719c05c620 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:03.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.261+0000 7f71a0af0700 1 -- 172.21.15.17:0/3119144072 >> 172.21.15.17:0/3119144072 conn(0x7f719c061360 msgr2=0x7f719c062f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:03.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.261+0000 7f71a0af0700 1 -- 172.21.15.17:0/3119144072 shutdown_connections 2022-08-01T19:44:03.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.261+0000 7f71a0af0700 1 -- 172.21.15.17:0/3119144072 wait complete. 2022-08-01T19:44:03.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.261+0000 7f71a0af0700 1 Processor -- start 2022-08-01T19:44:03.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.261+0000 7f71a0af0700 1 -- start start 2022-08-01T19:44:03.278 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.265+0000 7f71a0af0700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f719c0b1fb0 0x7f719c0aa070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:03.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.265+0000 7f71a0af0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f719c0b2790 0x7f719c0b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:03.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.265+0000 7f71a0af0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f719c13d3f0 0x7f719c156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:03.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.265+0000 7f71a0af0700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c13e370 con 0x7f719c13d3f0 2022-08-01T19:44:03.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.265+0000 7f71a0af0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c156c30 con 0x7f719c0b2790 2022-08-01T19:44:03.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.265+0000 7f71a0af0700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f719c0aac70 con 0x7f719c0b1fb0 2022-08-01T19:44:03.282 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.265+0000 7f719bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f719c13d3f0 0x7f719c156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:03.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.265+0000 7f719bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f719c13d3f0 0x7f719c156740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45776/0 (socket says 172.21.15.17:45776) 2022-08-01T19:44:03.283 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.265+0000 7f719bfff700 1 -- 172.21.15.17:0/2592121575 learned_addr learned my addr 172.21.15.17:0/2592121575 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:44:03.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.265+0000 7f719b7fe700 1 --2- 172.21.15.17:0/2592121575 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f719c0b1fb0 0x7f719c0aa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:03.284 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.265+0000 7f7192ffd700 1 --2- 172.21.15.17:0/2592121575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f719c0b2790 0x7f719c0b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:03.285 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.265+0000 7f719bfff700 1 -- 172.21.15.17:0/2592121575 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f719c0b1fb0 msgr2=0x7f719c0aa070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:03.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.265+0000 7f719bfff700 1 --2- 172.21.15.17:0/2592121575 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f719c0b1fb0 0x7f719c0aa070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:03.286 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.265+0000 7f719bfff700 1 -- 172.21.15.17:0/2592121575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f719c0b2790 msgr2=0x7f719c0b4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:03.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.265+0000 7f719bfff700 1 --2- 172.21.15.17:0/2592121575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f719c0b2790 0x7f719c0b4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:03.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.265+0000 7f719bfff700 1 -- 172.21.15.17:0/2592121575 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7194009470 con 0x7f719c13d3f0 2022-08-01T19:44:03.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.265+0000 7f719bfff700 1 --2- 172.21.15.17:0/2592121575 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f719c13d3f0 0x7f719c156740 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 rx=0x7f71940032f0 tx=0x7f7194016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:03.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.265+0000 7f71997fa700 1 -- 172.21.15.17:0/2592121575 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f719400b910 con 0x7f719c13d3f0 2022-08-01T19:44:03.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.265+0000 7f71997fa700 1 -- 172.21.15.17:0/2592121575 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7194011e90 con 0x7f719c13d3f0 2022-08-01T19:44:03.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.265+0000 7f71997fa700 1 -- 172.21.15.17:0/2592121575 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f719400b640 con 0x7f719c13d3f0 2022-08-01T19:44:03.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.265+0000 7f71a0af0700 1 -- 172.21.15.17:0/2592121575 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f719c0aa770 con 0x7f719c13d3f0 2022-08-01T19:44:03.294 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.265+0000 7f71a0af0700 1 -- 172.21.15.17:0/2592121575 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f719c152870 con 0x7f719c13d3f0 2022-08-01T19:44:03.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.269+0000 7f71a0af0700 1 -- 172.21.15.17:0/2592121575 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f719c1433d0 con 0x7f719c13d3f0 2022-08-01T19:44:03.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.269+0000 7f71997fa700 1 -- 172.21.15.17:0/2592121575 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f719400f070 con 0x7f719c13d3f0 2022-08-01T19:44:03.296 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.269+0000 7f71997fa700 1 --2- 172.21.15.17:0/2592121575 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7180053b30 0x7f7180055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:03.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.269+0000 7f71997fa700 1 -- 172.21.15.17:0/2592121575 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f719406ca40 con 0x7f719c13d3f0 2022-08-01T19:44:03.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.269+0000 7f719b7fe700 1 --2- 172.21.15.17:0/2592121575 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7180053b30 0x7f7180055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:03.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.269+0000 7f719b7fe700 1 --2- 172.21.15.17:0/2592121575 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7180053b30 0x7f7180055ff0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f718c00d000 tx=0x7f718c007a00).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:03.298 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.277+0000 7f71997fa700 1 -- 172.21.15.17:0/2592121575 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7194043800 con 0x7f719c13d3f0 2022-08-01T19:44:03.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.433+0000 7f71a0af0700 1 -- 172.21.15.17:0/2592121575 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f719c0574e0 con 0x7f719c13d3f0 2022-08-01T19:44:03.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.433+0000 7f71997fa700 1 -- 172.21.15.17:0/2592121575 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+2486 (secure 0 0 0) 0x7f7194015020 con 0x7f719c13d3f0 2022-08-01T19:44:03.441 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:44:03.442 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":83,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8665,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/608187432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":608187432},{"type":"v1","addr":"172.21.15.77:6835","nonce":608187432}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":81},{"gid":8673,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2764139073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2764139073},{"type":"v1","addr":"172.21.15.17:6833","nonce":2764139073}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":80}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:43:58.713330+0000","modified":"2022-08-01T19:43:59.717120+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8682":{"gid":8682,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/75309005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":75309005},{"type":"v1","addr":"172.21.15.17:6835","nonce":75309005}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2022-08-01T19:44:03.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.433+0000 7f71a0af0700 1 -- 172.21.15.17:0/2592121575 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7180053b30 msgr2=0x7f7180055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:03.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.433+0000 7f71a0af0700 1 --2- 172.21.15.17:0/2592121575 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7180053b30 0x7f7180055ff0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f718c00d000 tx=0x7f718c007a00).stop 2022-08-01T19:44:03.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.433+0000 7f71a0af0700 1 -- 172.21.15.17:0/2592121575 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f719c13d3f0 msgr2=0x7f719c156740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:03.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.433+0000 7f71a0af0700 1 --2- 172.21.15.17:0/2592121575 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f719c13d3f0 0x7f719c156740 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 rx=0x7f71940032f0 tx=0x7f7194016000).stop 2022-08-01T19:44:03.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.433+0000 7f71a0af0700 1 -- 172.21.15.17:0/2592121575 shutdown_connections 2022-08-01T19:44:03.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.433+0000 7f71a0af0700 1 --2- 172.21.15.17:0/2592121575 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7180053b30 0x7f7180055ff0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:03.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.433+0000 7f71a0af0700 1 --2- 172.21.15.17:0/2592121575 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f719c13d3f0 0x7f719c156740 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:03.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.433+0000 7f71a0af0700 1 --2- 172.21.15.17:0/2592121575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f719c0b2790 0x7f719c0b4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:03.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.433+0000 7f71a0af0700 1 --2- 172.21.15.17:0/2592121575 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f719c0b1fb0 0x7f719c0aa070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:03.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.433+0000 7f71a0af0700 1 -- 172.21.15.17:0/2592121575 >> 172.21.15.17:0/2592121575 conn(0x7f719c061360 msgr2=0x7f719c141260 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:03.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.433+0000 7f71a0af0700 1 -- 172.21.15.17:0/2592121575 shutdown_connections 2022-08-01T19:44:03.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:03.437+0000 7f71a0af0700 1 -- 172.21.15.17:0/2592121575 wait complete. 2022-08-01T19:44:03.449 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 83 2022-08-01T19:44:03.754 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 83, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:43:58.713330+0000', 'modified': '2022-08-01T19:43:59.717120+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8682}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8682': {'gid': 8682, 'name': 'c', 'rank': 0, 'incarnation': 82, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/75309005', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 75309005}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 75309005}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:44:03.755 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:44:03.756 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T19:44:03.874 INFO:teuthology.orchestra.run.smithi017.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":8682,"want_state":"up:active","state":"up:active","rank_uptime":5.1489052989999999,"mdsmap_epoch":83,"osdmap_epoch":158,"osdmap_epoch_barrier":158,"uptime":9.5072811799999997} 2022-08-01T19:44:03.875 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 8682, "mdsmap_epoch": 83, "osdmap_epoch": 158, "osdmap_epoch_barrier": 158, "rank_uptime": 5.148905299, "state": "up:active", "uptime": 9.50728118, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:44:03.876 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T19:44:03.877 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-08-01T19:44:03.877 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi085.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-08-01T19:44:03.878 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:44:03.879 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:44:03.895 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:44:03.911 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:44:03.912 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:44:03.968 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:44:03.968 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:44:03.970 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:44:03.970 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:44:03.979 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T19:44:03.979 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:44:04.024 DEBUG:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2022-08-01T19:44:04.051 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:44:04.052 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:44:04.105 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:ceph-fuse[23743]: starting ceph client2022-08-01T19:44:04.097+0000 7fe4ccdf2080 -1 init, newargv = 0x55d5690fa9f0 newargc=9 2022-08-01T19:44:04.115 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr: 2022-08-01T19:44:04.120 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi085.stderr:ceph-fuse[23743]: starting fuse 2022-08-01T19:44:04.121 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:44:04.123 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:44:04.124 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:44:04.125 DEBUG:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2022-08-01T19:44:04.134 INFO:teuthology.orchestra.run.smithi085.stdout:52 2022-08-01T19:44:04.136 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T19:44:04.137 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:44:04.139 DEBUG:teuthology.orchestra.run.smithi085:> sudo python3 -c ' 2022-08-01T19:44:04.140 DEBUG:teuthology.orchestra.run.smithi085:> import glob 2022-08-01T19:44:04.141 DEBUG:teuthology.orchestra.run.smithi085:> import re 2022-08-01T19:44:04.142 DEBUG:teuthology.orchestra.run.smithi085:> import os 2022-08-01T19:44:04.143 DEBUG:teuthology.orchestra.run.smithi085:> import subprocess 2022-08-01T19:44:04.144 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:44:04.144 DEBUG:teuthology.orchestra.run.smithi085:> def find_socket(client_name): 2022-08-01T19:44:04.145 DEBUG:teuthology.orchestra.run.smithi085:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-08-01T19:44:04.146 DEBUG:teuthology.orchestra.run.smithi085:> files = glob.glob(asok_path) 2022-08-01T19:44:04.147 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:44:04.147 DEBUG:teuthology.orchestra.run.smithi085:> # Given a non-glob path, it better be there 2022-08-01T19:44:04.148 DEBUG:teuthology.orchestra.run.smithi085:> if "*" not in asok_path: 2022-08-01T19:44:04.150 DEBUG:teuthology.orchestra.run.smithi085:> assert(len(files) == 1) 2022-08-01T19:44:04.150 DEBUG:teuthology.orchestra.run.smithi085:> return files[0] 2022-08-01T19:44:04.151 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:44:04.152 DEBUG:teuthology.orchestra.run.smithi085:> for f in files: 2022-08-01T19:44:04.153 DEBUG:teuthology.orchestra.run.smithi085:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T19:44:04.154 DEBUG:teuthology.orchestra.run.smithi085:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T19:44:04.154 DEBUG:teuthology.orchestra.run.smithi085:> return f 2022-08-01T19:44:04.155 DEBUG:teuthology.orchestra.run.smithi085:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T19:44:04.156 DEBUG:teuthology.orchestra.run.smithi085:> 2022-08-01T19:44:04.157 DEBUG:teuthology.orchestra.run.smithi085:> print(find_socket("client.0")) 2022-08-01T19:44:04.157 DEBUG:teuthology.orchestra.run.smithi085:> ' 2022-08-01T19:44:04.227 INFO:teuthology.orchestra.run.smithi085.stdout:/var/run/ceph/ceph-client.0.23743.asok 2022-08-01T19:44:04.230 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.23743.asok 2022-08-01T19:44:04.230 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:44:04.231 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.23743.asok status 2022-08-01T19:44:04.322 INFO:teuthology.orchestra.run.smithi085.stdout:{ 2022-08-01T19:44:04.323 INFO:teuthology.orchestra.run.smithi085.stdout: "metadata": { 2022-08-01T19:44:04.324 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T19:44:04.324 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T19:44:04.325 INFO:teuthology.orchestra.run.smithi085.stdout: "entity_id": "0", 2022-08-01T19:44:04.325 INFO:teuthology.orchestra.run.smithi085.stdout: "hostname": "smithi085", 2022-08-01T19:44:04.325 INFO:teuthology.orchestra.run.smithi085.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-08-01T19:44:04.326 INFO:teuthology.orchestra.run.smithi085.stdout: "pid": "23743", 2022-08-01T19:44:04.327 INFO:teuthology.orchestra.run.smithi085.stdout: "root": "/" 2022-08-01T19:44:04.327 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:44:04.328 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_count": 0, 2022-08-01T19:44:04.328 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_pinned_count": 0, 2022-08-01T19:44:04.329 INFO:teuthology.orchestra.run.smithi085.stdout: "id": 7547, 2022-08-01T19:44:04.329 INFO:teuthology.orchestra.run.smithi085.stdout: "inst": { 2022-08-01T19:44:04.330 INFO:teuthology.orchestra.run.smithi085.stdout: "name": { 2022-08-01T19:44:04.330 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "client", 2022-08-01T19:44:04.331 INFO:teuthology.orchestra.run.smithi085.stdout: "num": 7547 2022-08-01T19:44:04.331 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:44:04.331 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2022-08-01T19:44:04.332 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2022-08-01T19:44:04.332 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2022-08-01T19:44:04.334 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 3085666316 2022-08-01T19:44:04.334 INFO:teuthology.orchestra.run.smithi085.stdout: } 2022-08-01T19:44:04.335 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:44:04.335 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2022-08-01T19:44:04.337 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2022-08-01T19:44:04.337 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2022-08-01T19:44:04.338 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 3085666316 2022-08-01T19:44:04.338 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2022-08-01T19:44:04.339 INFO:teuthology.orchestra.run.smithi085.stdout: "inst_str": "client.7547 172.21.15.85:0/3085666316", 2022-08-01T19:44:04.339 INFO:teuthology.orchestra.run.smithi085.stdout: "addr_str": "172.21.15.85:0/3085666316", 2022-08-01T19:44:04.339 INFO:teuthology.orchestra.run.smithi085.stdout: "inode_count": 1, 2022-08-01T19:44:04.340 INFO:teuthology.orchestra.run.smithi085.stdout: "mds_epoch": 83, 2022-08-01T19:44:04.340 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch": 158, 2022-08-01T19:44:04.340 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch_barrier": 0, 2022-08-01T19:44:04.340 INFO:teuthology.orchestra.run.smithi085.stdout: "blacklisted": false, 2022-08-01T19:44:04.341 INFO:teuthology.orchestra.run.smithi085.stdout: "fs_name": "cephfs" 2022-08-01T19:44:04.341 INFO:teuthology.orchestra.run.smithi085.stdout:} 2022-08-01T19:44:04.343 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:44:04.343 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:44:04.344 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:44:04.353 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2022-08-01T19:44:04.355 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-08-01T19:44:04.355 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:44:04.356 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:44:04.417 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T19:44:04.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.533+0000 7f7206e26700 1 Processor -- start 2022-08-01T19:44:04.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.533+0000 7f7206e26700 1 -- start start 2022-08-01T19:44:04.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.537+0000 7f7206e26700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f72000b01a0 0x7f72000a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:04.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.537+0000 7f7206e26700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72000a7250 con 0x7f72000afd60 2022-08-01T19:44:04.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.537+0000 7f7206e26700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72000a7480 con 0x7f72000b2710 2022-08-01T19:44:04.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.537+0000 7f7206e26700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72000a7720 con 0x7f72000b01a0 2022-08-01T19:44:04.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.537+0000 7f7205623700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f72000b01a0 0x7f72000a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:04.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.537+0000 7f7206625700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f72000afd60 0x7f7200151d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37978/0 (socket says 172.21.15.17:37978) 2022-08-01T19:44:04.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.537+0000 7f7205623700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f72000b01a0 0x7f72000a8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45780/0 (socket says 172.21.15.17:45780) 2022-08-01T19:44:04.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.537+0000 7f7206625700 1 -- 172.21.15.17:0/2762222230 learned_addr learned my addr 172.21.15.17:0/2762222230 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:44:04.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.537+0000 7f7205623700 1 -- 172.21.15.17:0/2762222230 >> v1:172.21.15.77:6789/0 conn(0x7f72000b2710 legacy=0x7f720014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:04.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.537+0000 7f7204e22700 1 -- 172.21.15.17:0/2762222230 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2401887391 0 0) 0x7f72000a7250 con 0x7f72000afd60 2022-08-01T19:44:04.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.537+0000 7f7205623700 1 -- 172.21.15.17:0/2762222230 >> v1:172.21.15.17:6789/0 conn(0x7f72000afd60 legacy=0x7f7200151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:04.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.537+0000 7f7205623700 1 -- 172.21.15.17:0/2762222230 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72000a79c0 con 0x7f72000b01a0 2022-08-01T19:44:04.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.537+0000 7f7205623700 1 --2- 172.21.15.17:0/2762222230 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72000b01a0 0x7f72000a8540 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 rx=0x7f71f40032f0 tx=0x7f71f40091d0).ready entity=mon.0 client_cookie=2bbd2420ee2a04b4 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:04.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.537+0000 7f7204e22700 1 -- 172.21.15.17:0/2762222230 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71f4009ea0 con 0x7f72000b01a0 2022-08-01T19:44:04.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.537+0000 7f7204e22700 1 -- 172.21.15.17:0/2762222230 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f400b8a0 con 0x7f72000b01a0 2022-08-01T19:44:04.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.537+0000 7f7204e22700 1 -- 172.21.15.17:0/2762222230 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71f400ba10 con 0x7f72000b01a0 2022-08-01T19:44:04.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.537+0000 7f7206e26700 1 -- 172.21.15.17:0/2762222230 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72000b01a0 msgr2=0x7f72000a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:04.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.537+0000 7f7206e26700 1 --2- 172.21.15.17:0/2762222230 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72000b01a0 0x7f72000a8540 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 rx=0x7f71f40032f0 tx=0x7f71f40091d0).stop 2022-08-01T19:44:04.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.537+0000 7f7206e26700 1 -- 172.21.15.17:0/2762222230 shutdown_connections 2022-08-01T19:44:04.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.537+0000 7f7206e26700 1 --2- 172.21.15.17:0/2762222230 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72000b01a0 0x7f72000a8540 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:04.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.537+0000 7f7206e26700 1 -- 172.21.15.17:0/2762222230 >> 172.21.15.17:0/2762222230 conn(0x7f7200060310 msgr2=0x7f72000b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:04.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.537+0000 7f7206e26700 1 -- 172.21.15.17:0/2762222230 shutdown_connections 2022-08-01T19:44:04.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.537+0000 7f7206e26700 1 -- 172.21.15.17:0/2762222230 wait complete. 2022-08-01T19:44:04.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.537+0000 7f7206e26700 1 Processor -- start 2022-08-01T19:44:04.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.537+0000 7f7206e26700 1 -- start start 2022-08-01T19:44:04.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.541+0000 7f7206e26700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f72000afd60 0x7f72000ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:04.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.541+0000 7f7206e26700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72000b01a0 0x7f7200156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:04.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.541+0000 7f7206e26700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72000b2710 0x7f7200159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:04.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.541+0000 7f7206e26700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720014d6a0 con 0x7f72000b01a0 2022-08-01T19:44:04.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.541+0000 7f7206e26700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720014d400 con 0x7f72000b2710 2022-08-01T19:44:04.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.541+0000 7f7206e26700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72001567e0 con 0x7f72000afd60 2022-08-01T19:44:04.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.541+0000 7f7205623700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72000b01a0 0x7f7200156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:04.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.541+0000 7f7205623700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72000b01a0 0x7f7200156f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45788/0 (socket says 172.21.15.17:45788) 2022-08-01T19:44:04.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.541+0000 7f7205623700 1 -- 172.21.15.17:0/1684047917 learned_addr learned my addr 172.21.15.17:0/1684047917 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:44:04.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.541+0000 7f7205e24700 1 --2- 172.21.15.17:0/1684047917 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f72000afd60 0x7f72000ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:04.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.541+0000 7f7206625700 1 --2- 172.21.15.17:0/1684047917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72000b2710 0x7f7200159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:04.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.541+0000 7f7205623700 1 -- 172.21.15.17:0/1684047917 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f72000afd60 msgr2=0x7f72000ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:04.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.541+0000 7f7205623700 1 --2- 172.21.15.17:0/1684047917 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f72000afd60 0x7f72000ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:04.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.541+0000 7f7205623700 1 -- 172.21.15.17:0/1684047917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72000b2710 msgr2=0x7f7200159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:04.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.541+0000 7f7205623700 1 --2- 172.21.15.17:0/1684047917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72000b2710 0x7f7200159c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:04.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.541+0000 7f7205623700 1 -- 172.21.15.17:0/1684047917 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f4009470 con 0x7f72000b01a0 2022-08-01T19:44:04.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.541+0000 7f7205623700 1 --2- 172.21.15.17:0/1684047917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72000b01a0 0x7f7200156f20 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 rx=0x7f71f40032f0 tx=0x7f71f4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:04.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.541+0000 7f71f2ffd700 1 -- 172.21.15.17:0/1684047917 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71f4009ea0 con 0x7f72000b01a0 2022-08-01T19:44:04.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.541+0000 7f71f2ffd700 1 -- 172.21.15.17:0/1684047917 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f401f3f0 con 0x7f72000b01a0 2022-08-01T19:44:04.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.541+0000 7f7206e26700 1 -- 172.21.15.17:0/1684047917 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7200156a80 con 0x7f72000b01a0 2022-08-01T19:44:04.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.541+0000 7f71f2ffd700 1 -- 172.21.15.17:0/1684047917 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71f4011ea0 con 0x7f72000b01a0 2022-08-01T19:44:04.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.541+0000 7f7206e26700 1 -- 172.21.15.17:0/1684047917 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72000b32f0 con 0x7f72000b01a0 2022-08-01T19:44:04.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.541+0000 7f7206e26700 1 -- 172.21.15.17:0/1684047917 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72001575b0 con 0x7f72000b01a0 2022-08-01T19:44:04.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.545+0000 7f71f2ffd700 1 -- 172.21.15.17:0/1684047917 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f71f4026030 con 0x7f72000b01a0 2022-08-01T19:44:04.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.545+0000 7f71f2ffd700 1 --2- 172.21.15.17:0/1684047917 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f71e0053b00 0x7f71e0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:04.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.545+0000 7f71f2ffd700 1 -- 172.21.15.17:0/1684047917 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f71f406cb30 con 0x7f72000b01a0 2022-08-01T19:44:04.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.545+0000 7f7205e24700 1 --2- 172.21.15.17:0/1684047917 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f71e0053b00 0x7f71e0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:04.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.545+0000 7f7205e24700 1 --2- 172.21.15.17:0/1684047917 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f71e0053b00 0x7f71e0055fc0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7f71fc00d000 tx=0x7f71fc007b90).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:04.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.549+0000 7f71f2ffd700 1 -- 172.21.15.17:0/1684047917 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f71f40438f0 con 0x7f72000b01a0 2022-08-01T19:44:04.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.677+0000 7f7206e26700 1 -- 172.21.15.17:0/1684047917 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7200156c10 con 0x7f72000b01a0 2022-08-01T19:44:04.679 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.677+0000 7f71f2ffd700 1 -- 172.21.15.17:0/1684047917 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) v1 ==== 74+0+16285 (secure 0 0 0) 0x7f71f4024090 con 0x7f72000b01a0 2022-08-01T19:44:04.679 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:44:04.680 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":158,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:43:59.716124+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2022-08-01T19:43:57.196125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2022-08-01T19:43:57.958027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T19:44:04.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.677+0000 7f7206e26700 1 -- 172.21.15.17:0/1684047917 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f71e0053b00 msgr2=0x7f71e0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:04.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.677+0000 7f7206e26700 1 --2- 172.21.15.17:0/1684047917 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f71e0053b00 0x7f71e0055fc0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7f71fc00d000 tx=0x7f71fc007b90).stop 2022-08-01T19:44:04.682 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.677+0000 7f7206e26700 1 -- 172.21.15.17:0/1684047917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72000b01a0 msgr2=0x7f7200156f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:04.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.677+0000 7f7206e26700 1 --2- 172.21.15.17:0/1684047917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72000b01a0 0x7f7200156f20 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 rx=0x7f71f40032f0 tx=0x7f71f4016000).stop 2022-08-01T19:44:04.683 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.677+0000 7f7206e26700 1 -- 172.21.15.17:0/1684047917 shutdown_connections 2022-08-01T19:44:04.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.677+0000 7f7206e26700 1 --2- 172.21.15.17:0/1684047917 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f71e0053b00 0x7f71e0055fc0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:04.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.677+0000 7f7206e26700 1 --2- 172.21.15.17:0/1684047917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72000b2710 0x7f7200159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:04.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.677+0000 7f7206e26700 1 --2- 172.21.15.17:0/1684047917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72000b01a0 0x7f7200156f20 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:04.688 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.677+0000 7f7206e26700 1 --2- 172.21.15.17:0/1684047917 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f72000afd60 0x7f72000ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:04.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.677+0000 7f7206e26700 1 -- 172.21.15.17:0/1684047917 >> 172.21.15.17:0/1684047917 conn(0x7f7200060310 msgr2=0x7f72000673e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:04.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.677+0000 7f7206e26700 1 -- 172.21.15.17:0/1684047917 shutdown_connections 2022-08-01T19:44:04.689 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.677+0000 7f7206e26700 1 -- 172.21.15.17:0/1684047917 wait complete. 2022-08-01T19:44:04.695 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-08-01T19:33:53.001841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-08-01T19:34:01.553098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2022-08-01T19:43:57.196125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '158', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2022-08-01T19:43:57.958027+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '158', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-08-01T19:44:04.696 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-08-01T19:44:04.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.813+0000 7ff6fffff700 1 Processor -- start 2022-08-01T19:44:04.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.813+0000 7ff6fffff700 1 -- start start 2022-08-01T19:44:04.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.813+0000 7ff6fffff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff7000b2710 0x7ff70014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:04.820 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.813+0000 7ff6fffff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ff7000b01a0 0x7ff70014b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:04.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.813+0000 7ff6fffff700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7000a7250 con 0x7ff7000afd60 2022-08-01T19:44:04.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.813+0000 7ff6fffff700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7000a7480 con 0x7ff7000b2710 2022-08-01T19:44:04.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.813+0000 7ff6fffff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7000a7720 con 0x7ff7000b01a0 2022-08-01T19:44:04.821 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.813+0000 7ff6feffd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff7000b2710 0x7ff70014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:04.822 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.813+0000 7ff6fe7fc700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7ff7000b01a0 0x7ff70014b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:04.822 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.813+0000 7ff6feffd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff7000b2710 0x7ff70014afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45796/0 (socket says 172.21.15.17:45796) 2022-08-01T19:44:04.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.813+0000 7ff6feffd700 1 -- 172.21.15.17:0/972504223 learned_addr learned my addr 172.21.15.17:0/972504223 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:44:04.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.813+0000 7ff6fe7fc700 1 -- 172.21.15.17:0/972504223 >> v2:172.21.15.17:3300/0 conn(0x7ff7000b2710 msgr2=0x7ff70014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:44:04.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.813+0000 7ff6fe7fc700 1 --2- 172.21.15.17:0/972504223 >> v2:172.21.15.17:3300/0 conn(0x7ff7000b2710 0x7ff70014afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:04.823 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.813+0000 7ff6fe7fc700 1 -- 172.21.15.17:0/972504223 >> v1:172.21.15.77:6789/0 conn(0x7ff7000afd60 legacy=0x7ff7000a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:04.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.813+0000 7ff6fe7fc700 1 -- 172.21.15.17:0/972504223 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7000a79c0 con 0x7ff7000b01a0 2022-08-01T19:44:04.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.813+0000 7ff6feffd700 1 --2- 172.21.15.17:0/972504223 >> v2:172.21.15.17:3300/0 conn(0x7ff7000b2710 0x7ff70014afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:44:04.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.813+0000 7ff6fe7fc700 1 --2- 172.21.15.17:0/972504223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7000b01a0 0x7ff70014b4c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7ff6f0002f70 tx=0x7ff6f0006df0).ready entity=mon.1 client_cookie=350acddd5c78943c server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:04.824 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.813+0000 7ff6fdffb700 1 -- 172.21.15.17:0/972504223 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6f000d570 con 0x7ff7000b01a0 2022-08-01T19:44:04.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.813+0000 7ff6fdffb700 1 -- 172.21.15.17:0/972504223 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f0000e90 con 0x7ff7000b01a0 2022-08-01T19:44:04.825 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.813+0000 7ff6fdffb700 1 -- 172.21.15.17:0/972504223 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6f000d8c0 con 0x7ff7000b01a0 2022-08-01T19:44:04.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.813+0000 7ff6fffff700 1 -- 172.21.15.17:0/972504223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7000b01a0 msgr2=0x7ff70014b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:04.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.813+0000 7ff6fffff700 1 --2- 172.21.15.17:0/972504223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7000b01a0 0x7ff70014b4c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7ff6f0002f70 tx=0x7ff6f0006df0).stop 2022-08-01T19:44:04.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.813+0000 7ff6fffff700 1 -- 172.21.15.17:0/972504223 shutdown_connections 2022-08-01T19:44:04.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.813+0000 7ff6fffff700 1 --2- 172.21.15.17:0/972504223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7000b01a0 0x7ff70014b4c0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:04.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.813+0000 7ff6fffff700 1 --2- 172.21.15.17:0/972504223 >> v2:172.21.15.17:3300/0 conn(0x7ff7000b2710 0x7ff70014afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:04.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.813+0000 7ff6fffff700 1 -- 172.21.15.17:0/972504223 >> 172.21.15.17:0/972504223 conn(0x7ff700060310 msgr2=0x7ff7000b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:04.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.817+0000 7ff6fffff700 1 -- 172.21.15.17:0/972504223 shutdown_connections 2022-08-01T19:44:04.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.817+0000 7ff6fffff700 1 -- 172.21.15.17:0/972504223 wait complete. 2022-08-01T19:44:04.831 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.817+0000 7ff6fffff700 1 Processor -- start 2022-08-01T19:44:04.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.817+0000 7ff6fffff700 1 -- start start 2022-08-01T19:44:04.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.817+0000 7ff6fffff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff7000afd60 0x7ff700063c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:04.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.817+0000 7ff6fffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7000b01a0 0x7ff7000ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:04.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.817+0000 7ff6fffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7000b2710 0x7ff700065500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:04.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.817+0000 7ff6fffff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7000ab3a0 con 0x7ff7000b01a0 2022-08-01T19:44:04.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.817+0000 7ff6fffff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7000abb60 con 0x7ff7000b2710 2022-08-01T19:44:04.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.817+0000 7ff6fffff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff700159c20 con 0x7ff7000afd60 2022-08-01T19:44:04.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.817+0000 7ff6fe7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7000b01a0 0x7ff7000ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:04.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.817+0000 7ff6fe7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7000b01a0 0x7ff7000ae890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45800/0 (socket says 172.21.15.17:45800) 2022-08-01T19:44:04.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.817+0000 7ff6fe7fc700 1 -- 172.21.15.17:0/2148214053 learned_addr learned my addr 172.21.15.17:0/2148214053 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:44:04.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.817+0000 7ff6feffd700 1 --2- 172.21.15.17:0/2148214053 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff7000afd60 0x7ff700063c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:04.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.817+0000 7ff6ff7fe700 1 --2- 172.21.15.17:0/2148214053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7000b2710 0x7ff700065500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:04.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.817+0000 7ff6fe7fc700 1 -- 172.21.15.17:0/2148214053 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff7000afd60 msgr2=0x7ff700063c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:04.842 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.817+0000 7ff6fe7fc700 1 --2- 172.21.15.17:0/2148214053 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff7000afd60 0x7ff700063c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:04.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.817+0000 7ff6fe7fc700 1 -- 172.21.15.17:0/2148214053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7000b2710 msgr2=0x7ff700065500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:04.843 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.817+0000 7ff6fe7fc700 1 --2- 172.21.15.17:0/2148214053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7000b2710 0x7ff700065500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:04.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.817+0000 7ff6fe7fc700 1 -- 172.21.15.17:0/2148214053 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f400b1e0 con 0x7ff7000b01a0 2022-08-01T19:44:04.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.821+0000 7ff6fe7fc700 1 --2- 172.21.15.17:0/2148214053 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7000b01a0 0x7ff7000ae890 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 rx=0x7ff6f0002f70 tx=0x7ff6f0014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:04.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.821+0000 7ff6effff700 1 -- 172.21.15.17:0/2148214053 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6f0016070 con 0x7ff7000b01a0 2022-08-01T19:44:04.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.821+0000 7ff6effff700 1 -- 172.21.15.17:0/2148214053 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f000d8f0 con 0x7ff7000b01a0 2022-08-01T19:44:04.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.821+0000 7ff6fffff700 1 -- 172.21.15.17:0/2148214053 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f000b180 con 0x7ff7000b01a0 2022-08-01T19:44:04.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.821+0000 7ff6effff700 1 -- 172.21.15.17:0/2148214053 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6f001e450 con 0x7ff7000b01a0 2022-08-01T19:44:04.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.821+0000 7ff6fffff700 1 -- 172.21.15.17:0/2148214053 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7000aaec0 con 0x7ff7000b01a0 2022-08-01T19:44:04.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.821+0000 7ff6fffff700 1 -- 172.21.15.17:0/2148214053 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7000574e0 con 0x7ff7000b01a0 2022-08-01T19:44:04.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.821+0000 7ff6effff700 1 -- 172.21.15.17:0/2148214053 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff6f000f030 con 0x7ff7000b01a0 2022-08-01T19:44:04.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.821+0000 7ff6effff700 1 --2- 172.21.15.17:0/2148214053 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff6dc053b30 0x7ff6dc055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:04.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.821+0000 7ff6effff700 1 -- 172.21.15.17:0/2148214053 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff6f006cd60 con 0x7ff7000b01a0 2022-08-01T19:44:04.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.821+0000 7ff6feffd700 1 --2- 172.21.15.17:0/2148214053 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff6dc053b30 0x7ff6dc055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:04.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.825+0000 7ff6feffd700 1 --2- 172.21.15.17:0/2148214053 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff6dc053b30 0x7ff6dc055ff0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7ff6f400b000 tx=0x7ff6f4008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:04.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.833+0000 7ff6effff700 1 -- 172.21.15.17:0/2148214053 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff6f0043b20 con 0x7ff7000b01a0 2022-08-01T19:44:04.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.957+0000 7ff6fffff700 1 -- 172.21.15.17:0/2148214053 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ff700069a70 con 0x7ff7000b01a0 2022-08-01T19:44:04.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.957+0000 7ff6effff700 1 -- 172.21.15.17:0/2148214053 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v158) v1 ==== 110+0+10 (secure 0 0 0) 0x7ff6f0023020 con 0x7ff7000b01a0 2022-08-01T19:44:04.962 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2022-08-01T19:44:04.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.957+0000 7ff6fffff700 1 -- 172.21.15.17:0/2148214053 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff6dc053b30 msgr2=0x7ff6dc055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:04.969 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.961+0000 7ff6fffff700 1 --2- 172.21.15.17:0/2148214053 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff6dc053b30 0x7ff6dc055ff0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7ff6f400b000 tx=0x7ff6f4008040).stop 2022-08-01T19:44:04.970 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.961+0000 7ff6fffff700 1 -- 172.21.15.17:0/2148214053 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7000b01a0 msgr2=0x7ff7000ae890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:04.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.961+0000 7ff6fffff700 1 --2- 172.21.15.17:0/2148214053 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7000b01a0 0x7ff7000ae890 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 rx=0x7ff6f0002f70 tx=0x7ff6f0014000).stop 2022-08-01T19:44:04.971 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.961+0000 7ff6fffff700 1 -- 172.21.15.17:0/2148214053 shutdown_connections 2022-08-01T19:44:04.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.961+0000 7ff6fffff700 1 --2- 172.21.15.17:0/2148214053 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7ff6dc053b30 0x7ff6dc055ff0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:04.972 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.961+0000 7ff6fffff700 1 --2- 172.21.15.17:0/2148214053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7000b2710 0x7ff700065500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:04.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.961+0000 7ff6fffff700 1 --2- 172.21.15.17:0/2148214053 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7000b01a0 0x7ff7000ae890 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:04.973 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.961+0000 7ff6fffff700 1 --2- 172.21.15.17:0/2148214053 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7ff7000afd60 0x7ff700063c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:04.974 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.961+0000 7ff6fffff700 1 -- 172.21.15.17:0/2148214053 >> 172.21.15.17:0/2148214053 conn(0x7ff700060310 msgr2=0x7ff70005be30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:04.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.961+0000 7ff6fffff700 1 -- 172.21.15.17:0/2148214053 shutdown_connections 2022-08-01T19:44:04.975 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:04.961+0000 7ff6fffff700 1 -- 172.21.15.17:0/2148214053 wait complete. 2022-08-01T19:44:04.990 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-08-01T19:44:05.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.097+0000 7f1a6966f700 1 Processor -- start 2022-08-01T19:44:05.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.101+0000 7f1a6966f700 1 -- start start 2022-08-01T19:44:05.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.101+0000 7f1a6966f700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1a640b2710 0x7f1a6414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:05.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.101+0000 7f1a6966f700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a640a7250 con 0x7f1a640a8c50 2022-08-01T19:44:05.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.101+0000 7f1a6966f700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a640a7480 con 0x7f1a640b01a0 2022-08-01T19:44:05.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.101+0000 7f1a6966f700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a640a7720 con 0x7f1a640b2710 2022-08-01T19:44:05.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.101+0000 7f1a68e6e700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1a640a8c50 0x7f1a64151d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:38004/0 (socket says 172.21.15.17:38004) 2022-08-01T19:44:05.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.101+0000 7f1a68e6e700 1 -- 172.21.15.17:0/2388178632 learned_addr learned my addr 172.21.15.17:0/2388178632 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:44:05.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.101+0000 7f1a63fff700 1 --2- 172.21.15.17:0/2388178632 >> v2:172.21.15.17:3300/0 conn(0x7f1a640b2710 0x7f1a6414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:05.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.101+0000 7f1a62ffd700 1 -- 172.21.15.17:0/2388178632 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2574028860 0 0) 0x7f1a640a7250 con 0x7f1a640a8c50 2022-08-01T19:44:05.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.101+0000 7f1a62ffd700 1 -- 172.21.15.17:0/2388178632 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a50003260 con 0x7f1a640a8c50 2022-08-01T19:44:05.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.101+0000 7f1a62ffd700 1 -- 172.21.15.17:0/2388178632 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2133499411 0 0) 0x7f1a640a7480 con 0x7f1a640b01a0 2022-08-01T19:44:05.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.101+0000 7f1a63fff700 1 -- 172.21.15.17:0/2388178632 >> v1:172.21.15.77:6789/0 conn(0x7f1a640b01a0 legacy=0x7f1a640a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:05.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.101+0000 7f1a63fff700 1 -- 172.21.15.17:0/2388178632 >> v1:172.21.15.17:6789/0 conn(0x7f1a640a8c50 legacy=0x7f1a64151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:05.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.101+0000 7f1a63fff700 1 -- 172.21.15.17:0/2388178632 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a640a79c0 con 0x7f1a640b2710 2022-08-01T19:44:05.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.101+0000 7f1a63fff700 1 --2- 172.21.15.17:0/2388178632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a640b2710 0x7f1a6414afd0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 rx=0x7f1a5c001b80 tx=0x7f1a58006df0).ready entity=mon.0 client_cookie=5586ce5fb4942614 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:05.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.101+0000 7f1a62ffd700 1 -- 172.21.15.17:0/2388178632 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a5c0019e0 con 0x7f1a640b2710 2022-08-01T19:44:05.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.101+0000 7f1a62ffd700 1 -- 172.21.15.17:0/2388178632 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a58000e90 con 0x7f1a640b2710 2022-08-01T19:44:05.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.101+0000 7f1a62ffd700 1 -- 172.21.15.17:0/2388178632 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a58011400 con 0x7f1a640b2710 2022-08-01T19:44:05.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.101+0000 7f1a6966f700 1 -- 172.21.15.17:0/2388178632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a640b2710 msgr2=0x7f1a6414afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:05.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.101+0000 7f1a6966f700 1 --2- 172.21.15.17:0/2388178632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a640b2710 0x7f1a6414afd0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 rx=0x7f1a5c001b80 tx=0x7f1a58006df0).stop 2022-08-01T19:44:05.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.101+0000 7f1a6966f700 1 -- 172.21.15.17:0/2388178632 shutdown_connections 2022-08-01T19:44:05.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.101+0000 7f1a6966f700 1 --2- 172.21.15.17:0/2388178632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a640b2710 0x7f1a6414afd0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:05.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.101+0000 7f1a6966f700 1 -- 172.21.15.17:0/2388178632 >> 172.21.15.17:0/2388178632 conn(0x7f1a64060310 msgr2=0x7f1a640b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:05.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.101+0000 7f1a6966f700 1 -- 172.21.15.17:0/2388178632 shutdown_connections 2022-08-01T19:44:05.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.101+0000 7f1a6966f700 1 -- 172.21.15.17:0/2388178632 wait complete. 2022-08-01T19:44:05.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.105+0000 7f1a6966f700 1 Processor -- start 2022-08-01T19:44:05.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.105+0000 7f1a6966f700 1 -- start start 2022-08-01T19:44:05.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.105+0000 7f1a6966f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a640a8c50 0x7f1a640641e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:05.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.105+0000 7f1a6966f700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1a640b01a0 0x7f1a64065350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:05.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.105+0000 7f1a6966f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a640b2710 0x7f1a64066a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:05.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.105+0000 7f1a6966f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a64066240 con 0x7f1a640a8c50 2022-08-01T19:44:05.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.105+0000 7f1a6966f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a64157430 con 0x7f1a640b2710 2022-08-01T19:44:05.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.105+0000 7f1a6966f700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a640abfb0 con 0x7f1a640b01a0 2022-08-01T19:44:05.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.105+0000 7f1a63fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a640a8c50 0x7f1a640641e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:05.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.105+0000 7f1a63fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a640a8c50 0x7f1a640641e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45812/0 (socket says 172.21.15.17:45812) 2022-08-01T19:44:05.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.105+0000 7f1a63fff700 1 -- 172.21.15.17:0/387498504 learned_addr learned my addr 172.21.15.17:0/387498504 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:44:05.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.105+0000 7f1a68e6e700 1 --2- 172.21.15.17:0/387498504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a640b2710 0x7f1a64066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:05.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.105+0000 7f1a637fe700 1 --2- 172.21.15.17:0/387498504 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1a640b01a0 0x7f1a64065350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:05.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.105+0000 7f1a63fff700 1 -- 172.21.15.17:0/387498504 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1a640b01a0 msgr2=0x7f1a64065350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:05.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.105+0000 7f1a63fff700 1 --2- 172.21.15.17:0/387498504 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1a640b01a0 0x7f1a64065350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:05.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.105+0000 7f1a63fff700 1 -- 172.21.15.17:0/387498504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a640b2710 msgr2=0x7f1a64066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:05.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.105+0000 7f1a63fff700 1 --2- 172.21.15.17:0/387498504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a640b2710 0x7f1a64066a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:05.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.105+0000 7f1a63fff700 1 -- 172.21.15.17:0/387498504 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a5800b180 con 0x7f1a640a8c50 2022-08-01T19:44:05.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.105+0000 7f1a63fff700 1 --2- 172.21.15.17:0/387498504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a640a8c50 0x7f1a640641e0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 rx=0x7f1a5c001b80 tx=0x7f1a640a7610).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:05.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.105+0000 7f1a617fa700 1 -- 172.21.15.17:0/387498504 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a5800da20 con 0x7f1a640a8c50 2022-08-01T19:44:05.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.105+0000 7f1a617fa700 1 -- 172.21.15.17:0/387498504 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5800db80 con 0x7f1a640a8c50 2022-08-01T19:44:05.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.105+0000 7f1a617fa700 1 -- 172.21.15.17:0/387498504 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a5801d400 con 0x7f1a640a8c50 2022-08-01T19:44:05.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.105+0000 7f1a6966f700 1 -- 172.21.15.17:0/387498504 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a640ac250 con 0x7f1a640a8c50 2022-08-01T19:44:05.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.105+0000 7f1a6966f700 1 -- 172.21.15.17:0/387498504 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a64156f50 con 0x7f1a640a8c50 2022-08-01T19:44:05.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.109+0000 7f1a6966f700 1 -- 172.21.15.17:0/387498504 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a64157140 con 0x7f1a640a8c50 2022-08-01T19:44:05.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.109+0000 7f1a617fa700 1 -- 172.21.15.17:0/387498504 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1a5801b070 con 0x7f1a640a8c50 2022-08-01T19:44:05.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.109+0000 7f1a617fa700 1 --2- 172.21.15.17:0/387498504 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1a44053b30 0x7f1a44055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:05.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.109+0000 7f1a617fa700 1 -- 172.21.15.17:0/387498504 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1a5806c6b0 con 0x7f1a640a8c50 2022-08-01T19:44:05.134 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.109+0000 7f1a637fe700 1 --2- 172.21.15.17:0/387498504 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1a44053b30 0x7f1a44055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:05.134 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.109+0000 7f1a637fe700 1 --2- 172.21.15.17:0/387498504 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1a44053b30 0x7f1a44055ff0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f1a5400b000 tx=0x7f1a54009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:05.135 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.113+0000 7f1a617fa700 1 -- 172.21.15.17:0/387498504 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1a58043470 con 0x7f1a640a8c50 2022-08-01T19:44:05.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.241+0000 7f1a6966f700 1 -- 172.21.15.17:0/387498504 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1a640574e0 con 0x7f1a640a8c50 2022-08-01T19:44:05.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.241+0000 7f1a617fa700 1 -- 172.21.15.17:0/387498504 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v158) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1a58022020 con 0x7f1a640a8c50 2022-08-01T19:44:05.246 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:44:05.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.245+0000 7f1a6966f700 1 -- 172.21.15.17:0/387498504 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1a44053b30 msgr2=0x7f1a44055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:05.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.245+0000 7f1a6966f700 1 --2- 172.21.15.17:0/387498504 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1a44053b30 0x7f1a44055ff0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f1a5400b000 tx=0x7f1a54009040).stop 2022-08-01T19:44:05.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.245+0000 7f1a6966f700 1 -- 172.21.15.17:0/387498504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a640a8c50 msgr2=0x7f1a640641e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:05.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.245+0000 7f1a6966f700 1 --2- 172.21.15.17:0/387498504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a640a8c50 0x7f1a640641e0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 rx=0x7f1a5c001b80 tx=0x7f1a640a7610).stop 2022-08-01T19:44:05.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.245+0000 7f1a6966f700 1 -- 172.21.15.17:0/387498504 shutdown_connections 2022-08-01T19:44:05.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.245+0000 7f1a6966f700 1 --2- 172.21.15.17:0/387498504 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1a44053b30 0x7f1a44055ff0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:05.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.245+0000 7f1a6966f700 1 --2- 172.21.15.17:0/387498504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a640b2710 0x7f1a64066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:05.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.245+0000 7f1a6966f700 1 --2- 172.21.15.17:0/387498504 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1a640b01a0 0x7f1a64065350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:05.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.245+0000 7f1a6966f700 1 --2- 172.21.15.17:0/387498504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a640a8c50 0x7f1a640641e0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:05.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.245+0000 7f1a6966f700 1 -- 172.21.15.17:0/387498504 >> 172.21.15.17:0/387498504 conn(0x7f1a64060310 msgr2=0x7f1a640af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:05.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.245+0000 7f1a6966f700 1 -- 172.21.15.17:0/387498504 shutdown_connections 2022-08-01T19:44:05.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.245+0000 7f1a6966f700 1 -- 172.21.15.17:0/387498504 wait complete. 2022-08-01T19:44:05.259 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-08-01T19:44:05.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.361+0000 7f3aea960700 1 Processor -- start 2022-08-01T19:44:05.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.361+0000 7f3aea960700 1 -- start start 2022-08-01T19:44:05.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.365+0000 7f3aea960700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f3ae40612f0 0x7f3ae4143b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:05.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.365+0000 7f3aea960700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ae40671e0 con 0x7f3ae40b7840 2022-08-01T19:44:05.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.365+0000 7f3aea960700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ae4065540 con 0x7f3ae40b2930 2022-08-01T19:44:05.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.365+0000 7f3aea960700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ae40b7c20 con 0x7f3ae40612f0 2022-08-01T19:44:05.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.365+0000 7f3ae995e700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f3ae40b7840 0x7f3ae4060be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:38014/0 (socket says 172.21.15.17:38014) 2022-08-01T19:44:05.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.365+0000 7f3ae995e700 1 -- 172.21.15.17:0/2137996410 learned_addr learned my addr 172.21.15.17:0/2137996410 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:44:05.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.365+0000 7f3aea15f700 1 --2- 172.21.15.17:0/2137996410 >> v2:172.21.15.77:3300/0 conn(0x7f3ae40612f0 0x7f3ae4143b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:05.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.365+0000 7f3ae895c700 1 -- 172.21.15.17:0/2137996410 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2845565874 0 0) 0x7f3ae40671e0 con 0x7f3ae40b7840 2022-08-01T19:44:05.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.365+0000 7f3ae895c700 1 -- 172.21.15.17:0/2137996410 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3acc003260 con 0x7f3ae40b7840 2022-08-01T19:44:05.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.365+0000 7f3aea15f700 1 -- 172.21.15.17:0/2137996410 >> v1:172.21.15.77:6789/0 conn(0x7f3ae40b2930 legacy=0x7f3ae4143400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:05.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.365+0000 7f3aea15f700 1 -- 172.21.15.17:0/2137996410 >> v1:172.21.15.17:6789/0 conn(0x7f3ae40b7840 legacy=0x7f3ae4060be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:05.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.365+0000 7f3aea15f700 1 -- 172.21.15.17:0/2137996410 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae4066ce0 con 0x7f3ae40612f0 2022-08-01T19:44:05.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.365+0000 7f3aea15f700 1 --2- 172.21.15.17:0/2137996410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ae40612f0 0x7f3ae4143b10 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7f3adc0032f0 tx=0x7f3adc0091d0).ready entity=mon.1 client_cookie=d90ea7013db57a12 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:05.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.365+0000 7f3ae895c700 1 -- 172.21.15.17:0/2137996410 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3adc00b6c0 con 0x7f3ae40612f0 2022-08-01T19:44:05.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.365+0000 7f3ae895c700 1 -- 172.21.15.17:0/2137996410 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3adc00b820 con 0x7f3ae40612f0 2022-08-01T19:44:05.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.365+0000 7f3ae895c700 1 -- 172.21.15.17:0/2137996410 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3adc00b990 con 0x7f3ae40612f0 2022-08-01T19:44:05.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.365+0000 7f3aea960700 1 -- 172.21.15.17:0/2137996410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ae40612f0 msgr2=0x7f3ae4143b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:05.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.365+0000 7f3aea960700 1 --2- 172.21.15.17:0/2137996410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ae40612f0 0x7f3ae4143b10 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7f3adc0032f0 tx=0x7f3adc0091d0).stop 2022-08-01T19:44:05.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.365+0000 7f3aea960700 1 -- 172.21.15.17:0/2137996410 shutdown_connections 2022-08-01T19:44:05.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.365+0000 7f3aea960700 1 --2- 172.21.15.17:0/2137996410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ae40612f0 0x7f3ae4143b10 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:05.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.365+0000 7f3aea960700 1 -- 172.21.15.17:0/2137996410 >> 172.21.15.17:0/2137996410 conn(0x7f3ae405f840 msgr2=0x7f3ae405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:05.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.365+0000 7f3aea960700 1 -- 172.21.15.17:0/2137996410 shutdown_connections 2022-08-01T19:44:05.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.365+0000 7f3aea960700 1 -- 172.21.15.17:0/2137996410 wait complete. 2022-08-01T19:44:05.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.369+0000 7f3aea960700 1 Processor -- start 2022-08-01T19:44:05.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.369+0000 7f3aea960700 1 -- start start 2022-08-01T19:44:05.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.369+0000 7f3aea960700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ae40612f0 0x7f3ae40b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:05.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.369+0000 7f3aea960700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ae40b2930 0x7f3ae40adb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:05.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.369+0000 7f3aea960700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3ae40b7840 0x7f3ae40ae070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:05.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.369+0000 7f3aea960700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae40ac900 con 0x7f3ae40b2930 2022-08-01T19:44:05.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.369+0000 7f3aea960700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae4141de0 con 0x7f3ae40612f0 2022-08-01T19:44:05.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.369+0000 7f3aea960700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae40ac520 con 0x7f3ae40b7840 2022-08-01T19:44:05.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.369+0000 7f3ae915d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ae40b2930 0x7f3ae40adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:05.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.369+0000 7f3ae915d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ae40b2930 0x7f3ae40adb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45828/0 (socket says 172.21.15.17:45828) 2022-08-01T19:44:05.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.369+0000 7f3ae915d700 1 -- 172.21.15.17:0/1543318591 learned_addr learned my addr 172.21.15.17:0/1543318591 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:44:05.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.369+0000 7f3ae995e700 1 --2- 172.21.15.17:0/1543318591 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ae40612f0 0x7f3ae40b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:05.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.369+0000 7f3aea15f700 1 --2- 172.21.15.17:0/1543318591 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3ae40b7840 0x7f3ae40ae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:05.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.369+0000 7f3ae915d700 1 -- 172.21.15.17:0/1543318591 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3ae40b7840 msgr2=0x7f3ae40ae070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:05.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.369+0000 7f3ae915d700 1 --2- 172.21.15.17:0/1543318591 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3ae40b7840 0x7f3ae40ae070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:05.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.369+0000 7f3ae915d700 1 -- 172.21.15.17:0/1543318591 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ae40612f0 msgr2=0x7f3ae40b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:05.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.369+0000 7f3ae915d700 1 --2- 172.21.15.17:0/1543318591 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ae40612f0 0x7f3ae40b0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:05.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.369+0000 7f3ae915d700 1 -- 172.21.15.17:0/1543318591 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3adc009470 con 0x7f3ae40b2930 2022-08-01T19:44:05.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.369+0000 7f3ae915d700 1 --2- 172.21.15.17:0/1543318591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ae40b2930 0x7f3ae40adb80 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 rx=0x7f3ae40ad8e0 tx=0x7f3ad8007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:05.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.373+0000 7f3ad6ffd700 1 -- 172.21.15.17:0/1543318591 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ad800ce10 con 0x7f3ae40b2930 2022-08-01T19:44:05.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.373+0000 7f3aea960700 1 -- 172.21.15.17:0/1543318591 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ae40acaf0 con 0x7f3ae40b2930 2022-08-01T19:44:05.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.373+0000 7f3ad6ffd700 1 -- 172.21.15.17:0/1543318591 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad800f470 con 0x7f3ae40b2930 2022-08-01T19:44:05.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.373+0000 7f3aea960700 1 -- 172.21.15.17:0/1543318591 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ae40abde0 con 0x7f3ae40b2930 2022-08-01T19:44:05.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.373+0000 7f3ad6ffd700 1 -- 172.21.15.17:0/1543318591 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ad800f7d0 con 0x7f3ae40b2930 2022-08-01T19:44:05.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.373+0000 7f3aea960700 1 -- 172.21.15.17:0/1543318591 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ae40ad920 con 0x7f3ae40b2930 2022-08-01T19:44:05.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.373+0000 7f3ad6ffd700 1 -- 172.21.15.17:0/1543318591 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3ad8019030 con 0x7f3ae40b2930 2022-08-01T19:44:05.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.373+0000 7f3ad6ffd700 1 --2- 172.21.15.17:0/1543318591 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3ac4053aa0 0x7f3ac4055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:05.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.373+0000 7f3ad6ffd700 1 -- 172.21.15.17:0/1543318591 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f3ad806b680 con 0x7f3ae40b2930 2022-08-01T19:44:05.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.373+0000 7f3ae995e700 1 --2- 172.21.15.17:0/1543318591 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3ac4053aa0 0x7f3ac4055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:05.400 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.377+0000 7f3ae995e700 1 --2- 172.21.15.17:0/1543318591 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3ac4053aa0 0x7f3ac4055f60 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f3ae000b000 tx=0x7f3ae0009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:05.401 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.377+0000 7f3ad6ffd700 1 -- 172.21.15.17:0/1543318591 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3ad8042340 con 0x7f3ae40b2930 2022-08-01T19:44:05.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.505+0000 7f3aea960700 1 -- 172.21.15.17:0/1543318591 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3ae40574e0 con 0x7f3ae40b2930 2022-08-01T19:44:05.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.505+0000 7f3ad6ffd700 1 -- 172.21.15.17:0/1543318591 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v158) v1 ==== 104+0+10 (secure 0 0 0) 0x7f3ad8045960 con 0x7f3ae40b2930 2022-08-01T19:44:05.508 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:44:05.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.505+0000 7f3aea960700 1 -- 172.21.15.17:0/1543318591 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3ac4053aa0 msgr2=0x7f3ac4055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:05.510 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.505+0000 7f3aea960700 1 --2- 172.21.15.17:0/1543318591 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3ac4053aa0 0x7f3ac4055f60 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f3ae000b000 tx=0x7f3ae0009040).stop 2022-08-01T19:44:05.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.505+0000 7f3aea960700 1 -- 172.21.15.17:0/1543318591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ae40b2930 msgr2=0x7f3ae40adb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:05.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.505+0000 7f3aea960700 1 --2- 172.21.15.17:0/1543318591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ae40b2930 0x7f3ae40adb80 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 rx=0x7f3ae40ad8e0 tx=0x7f3ad8007c50).stop 2022-08-01T19:44:05.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.505+0000 7f3aea960700 1 -- 172.21.15.17:0/1543318591 shutdown_connections 2022-08-01T19:44:05.511 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.505+0000 7f3aea960700 1 --2- 172.21.15.17:0/1543318591 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f3ac4053aa0 0x7f3ac4055f60 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:05.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.505+0000 7f3aea960700 1 --2- 172.21.15.17:0/1543318591 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f3ae40b7840 0x7f3ae40ae070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:05.512 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.505+0000 7f3aea960700 1 --2- 172.21.15.17:0/1543318591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ae40b2930 0x7f3ae40adb80 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:05.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.505+0000 7f3aea960700 1 --2- 172.21.15.17:0/1543318591 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ae40612f0 0x7f3ae40b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:05.513 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.505+0000 7f3aea960700 1 -- 172.21.15.17:0/1543318591 >> 172.21.15.17:0/1543318591 conn(0x7f3ae405f840 msgr2=0x7f3ae40b6370 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:05.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.505+0000 7f3aea960700 1 -- 172.21.15.17:0/1543318591 shutdown_connections 2022-08-01T19:44:05.514 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.505+0000 7f3aea960700 1 -- 172.21.15.17:0/1543318591 wait complete. 2022-08-01T19:44:05.522 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-08-01T19:44:05.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.625+0000 7f1bd44dc700 1 Processor -- start 2022-08-01T19:44:05.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.629+0000 7f1bd44dc700 1 -- start start 2022-08-01T19:44:05.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.629+0000 7f1bd44dc700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f1bcc1445d0 0x7f1bcc144b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:05.636 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.629+0000 7f1bd44dc700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1bcc0655b0 0x7f1bcc144ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:05.636 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.629+0000 7f1bd44dc700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bcc1454e0 con 0x7f1bcc060290 2022-08-01T19:44:05.637 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.629+0000 7f1bd44dc700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bcc065e10 con 0x7f1bcc0655b0 2022-08-01T19:44:05.637 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.629+0000 7f1bd44dc700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bcc060670 con 0x7f1bcc1445d0 2022-08-01T19:44:05.638 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.629+0000 7f1bd2278700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1bcc060290 0x7f1bcc064ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:38028/0 (socket says 172.21.15.17:38028) 2022-08-01T19:44:05.638 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.629+0000 7f1bd2278700 1 -- 172.21.15.17:0/4256419718 learned_addr learned my addr 172.21.15.17:0/4256419718 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:44:05.638 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.629+0000 7f1bd2a79700 1 --2- 172.21.15.17:0/4256419718 >> v2:172.21.15.17:3300/0 conn(0x7f1bcc0655b0 0x7f1bcc144ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:05.638 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.629+0000 7f1bd1a77700 1 --2- 172.21.15.17:0/4256419718 >> v2:172.21.15.77:3300/0 conn(0x7f1bcc1445d0 0x7f1bcc144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:05.639 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.629+0000 7f1bd1276700 1 -- 172.21.15.17:0/4256419718 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1423714577 0 0) 0x7f1bcc1454e0 con 0x7f1bcc060290 2022-08-01T19:44:05.639 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.629+0000 7f1bd1276700 1 -- 172.21.15.17:0/4256419718 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bb4003260 con 0x7f1bcc060290 2022-08-01T19:44:05.639 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.629+0000 7f1bd2a79700 1 -- 172.21.15.17:0/4256419718 >> v2:172.21.15.77:3300/0 conn(0x7f1bcc1445d0 msgr2=0x7f1bcc144b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:44:05.639 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.629+0000 7f1bd2a79700 1 --2- 172.21.15.17:0/4256419718 >> v2:172.21.15.77:3300/0 conn(0x7f1bcc1445d0 0x7f1bcc144b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:05.640 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.629+0000 7f1bd2a79700 1 -- 172.21.15.17:0/4256419718 >> v1:172.21.15.17:6789/0 conn(0x7f1bcc060290 legacy=0x7f1bcc064ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:05.640 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.629+0000 7f1bd2a79700 1 -- 172.21.15.17:0/4256419718 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bcc0b2540 con 0x7f1bcc0655b0 2022-08-01T19:44:05.640 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.629+0000 7f1bd2a79700 1 --2- 172.21.15.17:0/4256419718 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bcc0655b0 0x7f1bcc144ff0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 rx=0x7f1bc40032f0 tx=0x7f1bc40091d0).ready entity=mon.0 client_cookie=9d1cf3173ac5b7ff server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:05.640 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.629+0000 7f1bd1276700 1 -- 172.21.15.17:0/4256419718 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bc400b600 con 0x7f1bcc0655b0 2022-08-01T19:44:05.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.629+0000 7f1bd1276700 1 -- 172.21.15.17:0/4256419718 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc400b760 con 0x7f1bcc0655b0 2022-08-01T19:44:05.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.629+0000 7f1bd1276700 1 -- 172.21.15.17:0/4256419718 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bc400b8d0 con 0x7f1bcc0655b0 2022-08-01T19:44:05.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.629+0000 7f1bd44dc700 1 -- 172.21.15.17:0/4256419718 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bcc0655b0 msgr2=0x7f1bcc144ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:05.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.629+0000 7f1bd44dc700 1 --2- 172.21.15.17:0/4256419718 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bcc0655b0 0x7f1bcc144ff0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 rx=0x7f1bc40032f0 tx=0x7f1bc40091d0).stop 2022-08-01T19:44:05.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.629+0000 7f1bd44dc700 1 -- 172.21.15.17:0/4256419718 shutdown_connections 2022-08-01T19:44:05.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.629+0000 7f1bd44dc700 1 --2- 172.21.15.17:0/4256419718 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bcc0655b0 0x7f1bcc144ff0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:05.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.629+0000 7f1bd44dc700 1 --2- 172.21.15.17:0/4256419718 >> v2:172.21.15.77:3300/0 conn(0x7f1bcc1445d0 0x7f1bcc144b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:05.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.629+0000 7f1bd44dc700 1 -- 172.21.15.17:0/4256419718 >> 172.21.15.17:0/4256419718 conn(0x7f1bcc05d0f0 msgr2=0x7f1bcc05f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:05.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.629+0000 7f1bd44dc700 1 -- 172.21.15.17:0/4256419718 shutdown_connections 2022-08-01T19:44:05.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.629+0000 7f1bd44dc700 1 -- 172.21.15.17:0/4256419718 wait complete. 2022-08-01T19:44:05.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.633+0000 7f1bd44dc700 1 Processor -- start 2022-08-01T19:44:05.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.633+0000 7f1bd44dc700 1 -- start start 2022-08-01T19:44:05.649 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.633+0000 7f1bd44dc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bcc060290 0x7f1bcc0aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:05.650 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.633+0000 7f1bd44dc700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1bcc0655b0 0x7f1bcc1539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:05.650 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.633+0000 7f1bd44dc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bcc0b1560 0x7f1bcc153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:05.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.633+0000 7f1bd44dc700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc13e5e0 con 0x7f1bcc0b1560 2022-08-01T19:44:05.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.633+0000 7f1bd44dc700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc13e880 con 0x7f1bcc060290 2022-08-01T19:44:05.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.633+0000 7f1bd44dc700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc13eb20 con 0x7f1bcc0655b0 2022-08-01T19:44:05.652 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.633+0000 7f1bd2a79700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bcc0b1560 0x7f1bcc153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:05.653 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.633+0000 7f1bd2a79700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bcc0b1560 0x7f1bcc153ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45844/0 (socket says 172.21.15.17:45844) 2022-08-01T19:44:05.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.633+0000 7f1bd2a79700 1 -- 172.21.15.17:0/2324871461 learned_addr learned my addr 172.21.15.17:0/2324871461 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:44:05.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.633+0000 7f1bd1a77700 1 --2- 172.21.15.17:0/2324871461 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1bcc0655b0 0x7f1bcc1539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:05.654 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.633+0000 7f1bd2278700 1 --2- 172.21.15.17:0/2324871461 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bcc060290 0x7f1bcc0aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:05.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.633+0000 7f1bd2a79700 1 -- 172.21.15.17:0/2324871461 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1bcc0655b0 msgr2=0x7f1bcc1539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:05.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.633+0000 7f1bd2a79700 1 --2- 172.21.15.17:0/2324871461 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1bcc0655b0 0x7f1bcc1539f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:05.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.633+0000 7f1bd2a79700 1 -- 172.21.15.17:0/2324871461 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bcc060290 msgr2=0x7f1bcc0aebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:05.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.633+0000 7f1bd2a79700 1 --2- 172.21.15.17:0/2324871461 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bcc060290 0x7f1bcc0aebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:05.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.633+0000 7f1bd2a79700 1 -- 172.21.15.17:0/2324871461 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc4009470 con 0x7f1bcc0b1560 2022-08-01T19:44:05.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.633+0000 7f1bd2a79700 1 --2- 172.21.15.17:0/2324871461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bcc0b1560 0x7f1bcc153ee0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 rx=0x7f1bc40032f0 tx=0x7f1bc4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:05.660 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.633+0000 7f1bbb7fe700 1 -- 172.21.15.17:0/2324871461 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bc4011c10 con 0x7f1bcc0b1560 2022-08-01T19:44:05.661 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.637+0000 7f1bbb7fe700 1 -- 172.21.15.17:0/2324871461 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc4011d70 con 0x7f1bcc0b1560 2022-08-01T19:44:05.662 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.637+0000 7f1bd44dc700 1 -- 172.21.15.17:0/2324871461 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bcc09eab0 con 0x7f1bcc0b1560 2022-08-01T19:44:05.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.637+0000 7f1bbb7fe700 1 -- 172.21.15.17:0/2324871461 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bc401f5e0 con 0x7f1bcc0b1560 2022-08-01T19:44:05.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.637+0000 7f1bd44dc700 1 -- 172.21.15.17:0/2324871461 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bcc13cf50 con 0x7f1bcc0b1560 2022-08-01T19:44:05.663 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.637+0000 7f1bd44dc700 1 -- 172.21.15.17:0/2324871461 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bcc0574e0 con 0x7f1bcc0b1560 2022-08-01T19:44:05.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.637+0000 7f1bbb7fe700 1 -- 172.21.15.17:0/2324871461 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1bc4026030 con 0x7f1bcc0b1560 2022-08-01T19:44:05.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.637+0000 7f1bbb7fe700 1 --2- 172.21.15.17:0/2324871461 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1bac053a70 0x7f1bac055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:05.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.637+0000 7f1bbb7fe700 1 -- 172.21.15.17:0/2324871461 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1bc406cd00 con 0x7f1bcc0b1560 2022-08-01T19:44:05.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.637+0000 7f1bd2278700 1 --2- 172.21.15.17:0/2324871461 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1bac053a70 0x7f1bac055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:05.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.641+0000 7f1bd2278700 1 --2- 172.21.15.17:0/2324871461 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1bac053a70 0x7f1bac055f30 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f1bbc00b000 tx=0x7f1bbc009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:05.665 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.641+0000 7f1bbb7fe700 1 -- 172.21.15.17:0/2324871461 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1bc40439b0 con 0x7f1bcc0b1560 2022-08-01T19:44:05.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.765+0000 7f1bd44dc700 1 -- 172.21.15.17:0/2324871461 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1bcc069a70 con 0x7f1bcc0b1560 2022-08-01T19:44:05.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.765+0000 7f1bbb7fe700 1 -- 172.21.15.17:0/2324871461 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v158) v1 ==== 100+0+10 (secure 0 0 0) 0x7f1bc40240a0 con 0x7f1bcc0b1560 2022-08-01T19:44:05.773 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 8 2022-08-01T19:44:05.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.769+0000 7f1bd44dc700 1 -- 172.21.15.17:0/2324871461 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1bac053a70 msgr2=0x7f1bac055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:05.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.769+0000 7f1bd44dc700 1 --2- 172.21.15.17:0/2324871461 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1bac053a70 0x7f1bac055f30 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f1bbc00b000 tx=0x7f1bbc009040).stop 2022-08-01T19:44:05.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.769+0000 7f1bd44dc700 1 -- 172.21.15.17:0/2324871461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bcc0b1560 msgr2=0x7f1bcc153ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:05.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.769+0000 7f1bd44dc700 1 --2- 172.21.15.17:0/2324871461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bcc0b1560 0x7f1bcc153ee0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 rx=0x7f1bc40032f0 tx=0x7f1bc4016000).stop 2022-08-01T19:44:05.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.769+0000 7f1bd44dc700 1 -- 172.21.15.17:0/2324871461 shutdown_connections 2022-08-01T19:44:05.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.769+0000 7f1bd44dc700 1 --2- 172.21.15.17:0/2324871461 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f1bac053a70 0x7f1bac055f30 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:05.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.769+0000 7f1bd44dc700 1 --2- 172.21.15.17:0/2324871461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bcc0b1560 0x7f1bcc153ee0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:05.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.769+0000 7f1bd44dc700 1 --2- 172.21.15.17:0/2324871461 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f1bcc0655b0 0x7f1bcc1539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:05.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.769+0000 7f1bd44dc700 1 --2- 172.21.15.17:0/2324871461 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bcc060290 0x7f1bcc0aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:05.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.769+0000 7f1bd44dc700 1 -- 172.21.15.17:0/2324871461 >> 172.21.15.17:0/2324871461 conn(0x7f1bcc05d0f0 msgr2=0x7f1bcc139dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:05.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.769+0000 7f1bd44dc700 1 -- 172.21.15.17:0/2324871461 shutdown_connections 2022-08-01T19:44:05.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.769+0000 7f1bd44dc700 1 -- 172.21.15.17:0/2324871461 wait complete. 2022-08-01T19:44:05.784 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-08-01T19:44:05.785 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:44:05.919 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.893+0000 7f77059c9700 1 Processor -- start 2022-08-01T19:44:05.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.893+0000 7f77059c9700 1 -- start start 2022-08-01T19:44:05.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.893+0000 7f77059c9700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f77000614b0 0x7f7700061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:05.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.893+0000 7f77059c9700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f77001405d0 0x7f7700140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:05.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.893+0000 7f77059c9700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77000aee80 con 0x7f77000ad750 2022-08-01T19:44:05.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.893+0000 7f77059c9700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77000b70b0 con 0x7f77000614b0 2022-08-01T19:44:05.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.893+0000 7f77059c9700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77000af0b0 con 0x7f77001405d0 2022-08-01T19:44:05.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.897+0000 7f77049c7700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f77000614b0 0x7f7700061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:05.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.897+0000 7f77049c7700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f77000614b0 0x7f7700061890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45850/0 (socket says 172.21.15.17:45850) 2022-08-01T19:44:05.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.897+0000 7f77049c7700 1 -- 172.21.15.17:0/3398169362 learned_addr learned my addr 172.21.15.17:0/3398169362 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:44:05.924 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.897+0000 7f77051c8700 1 --2- 172.21.15.17:0/3398169362 >> v2:172.21.15.77:3300/0 conn(0x7f77001405d0 0x7f7700140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:05.924 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.897+0000 7f77049c7700 1 -- 172.21.15.17:0/3398169362 >> v2:172.21.15.77:3300/0 conn(0x7f77001405d0 msgr2=0x7f7700140a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:44:05.925 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.897+0000 7f77049c7700 1 --2- 172.21.15.17:0/3398169362 >> v2:172.21.15.77:3300/0 conn(0x7f77001405d0 0x7f7700140a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:05.925 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.897+0000 7f77049c7700 1 -- 172.21.15.17:0/3398169362 >> v1:172.21.15.77:6789/0 conn(0x7f77000ad750 legacy=0x7f77000b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:05.926 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.897+0000 7f77049c7700 1 -- 172.21.15.17:0/3398169362 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77000af350 con 0x7f77000614b0 2022-08-01T19:44:05.926 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.897+0000 7f77049c7700 1 --2- 172.21.15.17:0/3398169362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77000614b0 0x7f7700061890 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 rx=0x7f77000aed30 tx=0x7f76f4006eb0).ready entity=mon.0 client_cookie=eb2a1ec987f1c61a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:05.927 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.897+0000 7f76ff7fe700 1 -- 172.21.15.17:0/3398169362 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76f400d610 con 0x7f77000614b0 2022-08-01T19:44:05.927 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.897+0000 7f76ff7fe700 1 -- 172.21.15.17:0/3398169362 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f4000e90 con 0x7f77000614b0 2022-08-01T19:44:05.927 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.897+0000 7f76ff7fe700 1 -- 172.21.15.17:0/3398169362 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76f400d9a0 con 0x7f77000614b0 2022-08-01T19:44:05.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.897+0000 7f77059c9700 1 -- 172.21.15.17:0/3398169362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77000614b0 msgr2=0x7f7700061890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:05.928 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.897+0000 7f77059c9700 1 --2- 172.21.15.17:0/3398169362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77000614b0 0x7f7700061890 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 rx=0x7f77000aed30 tx=0x7f76f4006eb0).stop 2022-08-01T19:44:05.929 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.897+0000 7f77059c9700 1 -- 172.21.15.17:0/3398169362 shutdown_connections 2022-08-01T19:44:05.929 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.897+0000 7f77059c9700 1 --2- 172.21.15.17:0/3398169362 >> v2:172.21.15.77:3300/0 conn(0x7f77001405d0 0x7f7700140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:05.930 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.897+0000 7f77059c9700 1 --2- 172.21.15.17:0/3398169362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77000614b0 0x7f7700061890 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:05.930 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.897+0000 7f77059c9700 1 -- 172.21.15.17:0/3398169362 >> 172.21.15.17:0/3398169362 conn(0x7f7700066f70 msgr2=0x7f7700067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:05.931 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.897+0000 7f77059c9700 1 -- 172.21.15.17:0/3398169362 shutdown_connections 2022-08-01T19:44:05.932 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.897+0000 7f77059c9700 1 -- 172.21.15.17:0/3398169362 wait complete. 2022-08-01T19:44:05.933 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.897+0000 7f77059c9700 1 Processor -- start 2022-08-01T19:44:05.934 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.897+0000 7f77059c9700 1 -- start start 2022-08-01T19:44:05.934 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.901+0000 7f77059c9700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f77000614b0 0x7f7700144f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:05.935 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.901+0000 7f77059c9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77000ad750 0x7f7700156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:05.935 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.901+0000 7f77059c9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77001405d0 0x7f7700156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:05.936 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.901+0000 7f77059c9700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77000b3d80 con 0x7f77000ad750 2022-08-01T19:44:05.936 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.901+0000 7f77059c9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77000b0970 con 0x7f77001405d0 2022-08-01T19:44:05.937 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.901+0000 7f77059c9700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7700156e60 con 0x7f77000614b0 2022-08-01T19:44:05.937 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.901+0000 7f76fffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77000ad750 0x7f7700156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:05.938 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.901+0000 7f76fffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77000ad750 0x7f7700156480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45856/0 (socket says 172.21.15.17:45856) 2022-08-01T19:44:05.939 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.901+0000 7f76fffff700 1 -- 172.21.15.17:0/2820119921 learned_addr learned my addr 172.21.15.17:0/2820119921 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:44:05.939 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.901+0000 7f77049c7700 1 --2- 172.21.15.17:0/2820119921 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f77000614b0 0x7f7700144f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:05.940 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.901+0000 7f77051c8700 1 --2- 172.21.15.17:0/2820119921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77001405d0 0x7f7700156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:05.940 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.901+0000 7f76fffff700 1 -- 172.21.15.17:0/2820119921 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f77000614b0 msgr2=0x7f7700144f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:05.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.901+0000 7f76fffff700 1 --2- 172.21.15.17:0/2820119921 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f77000614b0 0x7f7700144f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:05.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.901+0000 7f76fffff700 1 -- 172.21.15.17:0/2820119921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77001405d0 msgr2=0x7f7700156970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:05.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.901+0000 7f76fffff700 1 --2- 172.21.15.17:0/2820119921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77001405d0 0x7f7700156970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:05.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.901+0000 7f76fffff700 1 -- 172.21.15.17:0/2820119921 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f400b1e0 con 0x7f77000ad750 2022-08-01T19:44:05.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.901+0000 7f76fffff700 1 --2- 172.21.15.17:0/2820119921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77000ad750 0x7f7700156480 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 rx=0x7f77000ae2b0 tx=0x7f76f0007f30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:05.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.901+0000 7f76fdffb700 1 -- 172.21.15.17:0/2820119921 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76f0006aa0 con 0x7f77000ad750 2022-08-01T19:44:05.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.901+0000 7f76fdffb700 1 -- 172.21.15.17:0/2820119921 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f0006c00 con 0x7f77000ad750 2022-08-01T19:44:05.944 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.901+0000 7f77059c9700 1 -- 172.21.15.17:0/2820119921 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77000b11d0 con 0x7f77000ad750 2022-08-01T19:44:05.944 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.901+0000 7f76fdffb700 1 -- 172.21.15.17:0/2820119921 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f76f0007030 con 0x7f77000ad750 2022-08-01T19:44:05.944 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.901+0000 7f77059c9700 1 -- 172.21.15.17:0/2820119921 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7700155fe0 con 0x7f77000ad750 2022-08-01T19:44:05.944 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.901+0000 7f76fdffb700 1 -- 172.21.15.17:0/2820119921 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f76f0010030 con 0x7f77000ad750 2022-08-01T19:44:05.945 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.901+0000 7f77059c9700 1 -- 172.21.15.17:0/2820119921 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77001561a0 con 0x7f77000ad750 2022-08-01T19:44:05.945 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.905+0000 7f76fdffb700 1 --2- 172.21.15.17:0/2820119921 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f76e00539e0 0x7f76e0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:05.945 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.905+0000 7f76fdffb700 1 -- 172.21.15.17:0/2820119921 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f76f006bf10 con 0x7f77000ad750 2022-08-01T19:44:05.945 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.905+0000 7f77049c7700 1 --2- 172.21.15.17:0/2820119921 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f76e00539e0 0x7f76e0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:05.946 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.905+0000 7f77049c7700 1 --2- 172.21.15.17:0/2820119921 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f76e00539e0 0x7f76e0055ea0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f76f401c000 tx=0x7f76f4015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:05.946 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:05.909+0000 7f76fdffb700 1 -- 172.21.15.17:0/2820119921 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f76f0042cd0 con 0x7f77000ad750 2022-08-01T19:44:06.078 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:06.073+0000 7f77059c9700 1 -- 172.21.15.17:0/2820119921 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77000b1360 con 0x7f77000ad750 2022-08-01T19:44:06.080 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:06.073+0000 7f76fdffb700 1 -- 172.21.15.17:0/2820119921 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+2486 (secure 0 0 0) 0x7f76f00462f0 con 0x7f77000ad750 2022-08-01T19:44:06.081 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:44:06.081 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":83,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8665,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/608187432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":608187432},{"type":"v1","addr":"172.21.15.77:6835","nonce":608187432}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":81},{"gid":8673,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2764139073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2764139073},{"type":"v1","addr":"172.21.15.17:6833","nonce":2764139073}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":80}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:43:58.713330+0000","modified":"2022-08-01T19:43:59.717120+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8682":{"gid":8682,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/75309005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":75309005},{"type":"v1","addr":"172.21.15.17:6835","nonce":75309005}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2022-08-01T19:44:06.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:06.077+0000 7f77059c9700 1 -- 172.21.15.17:0/2820119921 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f76e00539e0 msgr2=0x7f76e0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:06.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:06.077+0000 7f77059c9700 1 --2- 172.21.15.17:0/2820119921 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f76e00539e0 0x7f76e0055ea0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f76f401c000 tx=0x7f76f4015000).stop 2022-08-01T19:44:06.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:06.077+0000 7f77059c9700 1 -- 172.21.15.17:0/2820119921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77000ad750 msgr2=0x7f7700156480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:06.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:06.077+0000 7f77059c9700 1 --2- 172.21.15.17:0/2820119921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77000ad750 0x7f7700156480 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 rx=0x7f77000ae2b0 tx=0x7f76f0007f30).stop 2022-08-01T19:44:06.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:06.077+0000 7f77059c9700 1 -- 172.21.15.17:0/2820119921 shutdown_connections 2022-08-01T19:44:06.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:06.077+0000 7f77059c9700 1 --2- 172.21.15.17:0/2820119921 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f76e00539e0 0x7f76e0055ea0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:06.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:06.077+0000 7f77059c9700 1 --2- 172.21.15.17:0/2820119921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f77001405d0 0x7f7700156970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:06.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:06.077+0000 7f77059c9700 1 --2- 172.21.15.17:0/2820119921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77000ad750 0x7f7700156480 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:06.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:06.077+0000 7f77059c9700 1 --2- 172.21.15.17:0/2820119921 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f77000614b0 0x7f7700144f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:06.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:06.077+0000 7f77059c9700 1 -- 172.21.15.17:0/2820119921 >> 172.21.15.17:0/2820119921 conn(0x7f7700066f70 msgr2=0x7f770005be30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:06.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:06.077+0000 7f77059c9700 1 -- 172.21.15.17:0/2820119921 shutdown_connections 2022-08-01T19:44:06.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:06.077+0000 7f77059c9700 1 -- 172.21.15.17:0/2820119921 wait complete. 2022-08-01T19:44:06.107 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 83 2022-08-01T19:44:06.422 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 83, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-08-01T19:43:58.713330+0000', 'modified': '2022-08-01T19:43:59.717120+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8682}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8682': {'gid': 8682, 'name': 'c', 'rank': 0, 'incarnation': 82, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/75309005', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 75309005}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 75309005}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-08-01T19:44:06.423 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-08-01T19:44:06.424 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2022-08-01T19:44:06.551 INFO:teuthology.orchestra.run.smithi017.stdout:{"cluster_fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","whoami":0,"id":8682,"want_state":"up:active","state":"up:active","rank_uptime":7.8255183150000001,"mdsmap_epoch":83,"osdmap_epoch":158,"osdmap_epoch_barrier":158,"uptime":12.183911609999999} 2022-08-01T19:44:06.552 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", "id": 8682, "mdsmap_epoch": 83, "osdmap_epoch": 158, "osdmap_epoch_barrier": 158, "rank_uptime": 7.825518315, "state": "up:active", "uptime": 12.18391161, "want_state": "up:active", "whoami": 0 } 2022-08-01T19:44:06.553 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-08-01T19:44:06.553 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-08-01T19:44:06.554 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi190.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-08-01T19:44:06.555 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:44:06.556 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:44:06.566 DEBUG:teuthology.orchestra.run.smithi190:> sudo modprobe fuse 2022-08-01T19:44:06.588 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:44:06.589 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:44:06.654 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:44:06.655 INFO:teuthology.orchestra.run.smithi190.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:44:06.656 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:44:06.657 DEBUG:teuthology.orchestra.run.smithi190:> ls /sys/fs/fuse/connections 2022-08-01T19:44:06.663 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-08-01T19:44:06.664 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:44:06.708 DEBUG:teuthology.orchestra.run.smithi190:> sudo modprobe fuse 2022-08-01T19:44:06.734 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-08-01T19:44:06.735 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-08-01T19:44:06.787 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:2022-08-01T19:44:06.784+0000 7f37c59b1080 -1 init, newargv = 0x5629c408d840 newargc=9 2022-08-01T19:44:06.787 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:ceph-fuse[23355]: starting ceph client 2022-08-01T19:44:06.797 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-08-01T19:44:06.798 INFO:teuthology.orchestra.run.smithi190.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-08-01T19:44:06.799 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:44:06.799 DEBUG:teuthology.orchestra.run.smithi190:> ls /sys/fs/fuse/connections 2022-08-01T19:44:06.800 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:ceph-fuse[23355]: starting fuse 2022-08-01T19:44:06.807 INFO:teuthology.orchestra.run.smithi190.stdout:52 2022-08-01T19:44:06.808 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-08-01T19:44:06.810 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:44:06.812 DEBUG:teuthology.orchestra.run.smithi190:> sudo python3 -c ' 2022-08-01T19:44:06.812 DEBUG:teuthology.orchestra.run.smithi190:> import glob 2022-08-01T19:44:06.813 DEBUG:teuthology.orchestra.run.smithi190:> import re 2022-08-01T19:44:06.813 DEBUG:teuthology.orchestra.run.smithi190:> import os 2022-08-01T19:44:06.814 DEBUG:teuthology.orchestra.run.smithi190:> import subprocess 2022-08-01T19:44:06.814 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:44:06.814 DEBUG:teuthology.orchestra.run.smithi190:> def find_socket(client_name): 2022-08-01T19:44:06.815 DEBUG:teuthology.orchestra.run.smithi190:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-08-01T19:44:06.815 DEBUG:teuthology.orchestra.run.smithi190:> files = glob.glob(asok_path) 2022-08-01T19:44:06.816 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:44:06.816 DEBUG:teuthology.orchestra.run.smithi190:> # Given a non-glob path, it better be there 2022-08-01T19:44:06.816 DEBUG:teuthology.orchestra.run.smithi190:> if "*" not in asok_path: 2022-08-01T19:44:06.817 DEBUG:teuthology.orchestra.run.smithi190:> assert(len(files) == 1) 2022-08-01T19:44:06.817 DEBUG:teuthology.orchestra.run.smithi190:> return files[0] 2022-08-01T19:44:06.818 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:44:06.818 DEBUG:teuthology.orchestra.run.smithi190:> for f in files: 2022-08-01T19:44:06.819 DEBUG:teuthology.orchestra.run.smithi190:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-08-01T19:44:06.819 DEBUG:teuthology.orchestra.run.smithi190:> if os.path.exists("/proc/{0}".format(pid)): 2022-08-01T19:44:06.820 DEBUG:teuthology.orchestra.run.smithi190:> return f 2022-08-01T19:44:06.820 DEBUG:teuthology.orchestra.run.smithi190:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-08-01T19:44:06.820 DEBUG:teuthology.orchestra.run.smithi190:> 2022-08-01T19:44:06.821 DEBUG:teuthology.orchestra.run.smithi190:> print(find_socket("client.1")) 2022-08-01T19:44:06.821 DEBUG:teuthology.orchestra.run.smithi190:> ' 2022-08-01T19:44:06.864 INFO:teuthology.orchestra.run.smithi190.stdout:/var/run/ceph/ceph-client.1.23355.asok 2022-08-01T19:44:06.870 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.23355.asok 2022-08-01T19:44:06.871 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:44:06.872 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.23355.asok status 2022-08-01T19:44:07.012 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2022-08-01T19:44:07.013 INFO:teuthology.orchestra.run.smithi190.stdout: "metadata": { 2022-08-01T19:44:07.013 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_sha1": "fd96719b0e1200b56e7430604cfdd20ad281ca92", 2022-08-01T19:44:07.014 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_version": "ceph version 15.2.16-343-gfd96719b (fd96719b0e1200b56e7430604cfdd20ad281ca92) octopus (stable)", 2022-08-01T19:44:07.014 INFO:teuthology.orchestra.run.smithi190.stdout: "entity_id": "1", 2022-08-01T19:44:07.014 INFO:teuthology.orchestra.run.smithi190.stdout: "hostname": "smithi190", 2022-08-01T19:44:07.014 INFO:teuthology.orchestra.run.smithi190.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-08-01T19:44:07.015 INFO:teuthology.orchestra.run.smithi190.stdout: "pid": "23355", 2022-08-01T19:44:07.016 INFO:teuthology.orchestra.run.smithi190.stdout: "root": "/" 2022-08-01T19:44:07.016 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:44:07.016 INFO:teuthology.orchestra.run.smithi190.stdout: "dentry_count": 0, 2022-08-01T19:44:07.017 INFO:teuthology.orchestra.run.smithi190.stdout: "dentry_pinned_count": 0, 2022-08-01T19:44:07.017 INFO:teuthology.orchestra.run.smithi190.stdout: "id": 8898, 2022-08-01T19:44:07.017 INFO:teuthology.orchestra.run.smithi190.stdout: "inst": { 2022-08-01T19:44:07.019 INFO:teuthology.orchestra.run.smithi190.stdout: "name": { 2022-08-01T19:44:07.019 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "client", 2022-08-01T19:44:07.019 INFO:teuthology.orchestra.run.smithi190.stdout: "num": 8898 2022-08-01T19:44:07.020 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:44:07.020 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": { 2022-08-01T19:44:07.020 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "v1", 2022-08-01T19:44:07.021 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": "172.21.15.190:0", 2022-08-01T19:44:07.021 INFO:teuthology.orchestra.run.smithi190.stdout: "nonce": 3706557052 2022-08-01T19:44:07.021 INFO:teuthology.orchestra.run.smithi190.stdout: } 2022-08-01T19:44:07.021 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:44:07.022 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": { 2022-08-01T19:44:07.022 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "v1", 2022-08-01T19:44:07.022 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": "172.21.15.190:0", 2022-08-01T19:44:07.023 INFO:teuthology.orchestra.run.smithi190.stdout: "nonce": 3706557052 2022-08-01T19:44:07.023 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2022-08-01T19:44:07.023 INFO:teuthology.orchestra.run.smithi190.stdout: "inst_str": "client.8898 172.21.15.190:0/3706557052", 2022-08-01T19:44:07.023 INFO:teuthology.orchestra.run.smithi190.stdout: "addr_str": "172.21.15.190:0/3706557052", 2022-08-01T19:44:07.024 INFO:teuthology.orchestra.run.smithi190.stdout: "inode_count": 1, 2022-08-01T19:44:07.024 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_epoch": 83, 2022-08-01T19:44:07.024 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_epoch": 158, 2022-08-01T19:44:07.024 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_epoch_barrier": 0, 2022-08-01T19:44:07.024 INFO:teuthology.orchestra.run.smithi190.stdout: "blacklisted": false, 2022-08-01T19:44:07.025 INFO:teuthology.orchestra.run.smithi190.stdout: "fs_name": "cephfs" 2022-08-01T19:44:07.025 INFO:teuthology.orchestra.run.smithi190.stdout:} 2022-08-01T19:44:07.027 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:44:07.029 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:44:07.029 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:44:07.045 INFO:teuthology.orchestra.run.smithi190.stdout:fuseblk 2022-08-01T19:44:07.046 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T19:44:07.047 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:44:07.047 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:44:07.113 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:44:07.114 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:44:07.114 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:44:07.123 INFO:teuthology.orchestra.run.smithi190.stdout:fuseblk 2022-08-01T19:44:07.125 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T19:44:07.126 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:44:07.126 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:44:07.127 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:44:07.179 INFO:teuthology.orchestra.run.smithi190.stdout:fuseblk 2022-08-01T19:44:07.181 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-08-01T19:44:07.182 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi190.front.sepia.ceph.com... 2022-08-01T19:44:07.182 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-08-01T19:44:07.183 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:44:07.199 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:ceph-fuse[23355]: fuse finished with error 0 and tester_r 0 2022-08-01T19:44:07.202 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:44:07.202 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:44:07.203 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:44:07.253 INFO:teuthology.orchestra.run.smithi190.stdout:ext2/ext3 2022-08-01T19:44:07.255 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-08-01T19:44:07.256 INFO:teuthology.orchestra.run:waiting for 900 2022-08-01T19:44:13.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-08-01T19:44:13.364 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-08-01T19:44:13.381 INFO:teuthology.orchestra.run:Running command with timeout 10800 2022-08-01T19:44:13.382 DEBUG:teuthology.orchestra.run.smithi085:> cd /home/ubuntu/cephtest/mnt.0 && sudo bash -c 'stat -f .' 2022-08-01T19:44:13.404 INFO:teuthology.orchestra.run.smithi085.stdout: File: "." 2022-08-01T19:44:13.405 INFO:teuthology.orchestra.run.smithi085.stdout: ID: 0 Namelen: 255 Type: fuseblk 2022-08-01T19:44:13.405 INFO:teuthology.orchestra.run.smithi085.stdout:Block size: 4194304 Fundamental block size: 4194304 2022-08-01T19:44:13.405 INFO:teuthology.orchestra.run.smithi085.stdout:Blocks: Total: 86526 Free: 86526 Available: 86526 2022-08-01T19:44:13.405 INFO:teuthology.orchestra.run.smithi085.stdout:Inodes: Total: 1 Free: 0 2022-08-01T19:44:13.406 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-08-01T19:44:13.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.513+0000 7f532ff8b700 1 Processor -- start 2022-08-01T19:44:13.518 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532ff8b700 1 -- start start 2022-08-01T19:44:13.528 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532ff8b700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f53280b1f40 0x7f53280b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:13.529 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532ff8b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5328148530 0x7f53280b2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:13.529 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532ff8b700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53280b04e0 con 0x7f53280a7250 2022-08-01T19:44:13.530 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532ff8b700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53280a9250 con 0x7f53280b1f40 2022-08-01T19:44:13.531 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532ff8b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53280b0780 con 0x7f5328148530 2022-08-01T19:44:13.531 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532e528700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f53280a7250 0x7f53281448b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:38060/0 (socket says 172.21.15.17:38060) 2022-08-01T19:44:13.532 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532e528700 1 -- 172.21.15.17:0/3075144731 learned_addr learned my addr 172.21.15.17:0/3075144731 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:44:13.532 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532dd27700 1 --2- 172.21.15.17:0/3075144731 >> v2:172.21.15.77:3300/0 conn(0x7f53280b1f40 0x7f53280b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:13.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532d526700 1 --2- 172.21.15.17:0/3075144731 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5328148530 0x7f53280b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:13.533 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532cd25700 1 -- 172.21.15.17:0/3075144731 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 200186928 0 0) 0x7f53280b04e0 con 0x7f53280a7250 2022-08-01T19:44:13.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532cd25700 1 -- 172.21.15.17:0/3075144731 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5310003260 con 0x7f53280a7250 2022-08-01T19:44:13.534 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532cd25700 1 -- 172.21.15.17:0/3075144731 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 592830953 0 0) 0x7f5310003260 con 0x7f53280a7250 2022-08-01T19:44:13.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532cd25700 1 -- 172.21.15.17:0/3075144731 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5328148530 msgr2=0x7f53280b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:44:13.535 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532cd25700 1 --2- 172.21.15.17:0/3075144731 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5328148530 0x7f53280b2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:13.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532cd25700 1 -- 172.21.15.17:0/3075144731 >> v2:172.21.15.77:3300/0 conn(0x7f53280b1f40 msgr2=0x7f53280b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:44:13.536 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532cd25700 1 --2- 172.21.15.17:0/3075144731 >> v2:172.21.15.77:3300/0 conn(0x7f53280b1f40 0x7f53280b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:13.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532cd25700 1 -- 172.21.15.17:0/3075144731 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53280b0a20 con 0x7f53280a7250 2022-08-01T19:44:13.538 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532cd25700 1 -- 172.21.15.17:0/3075144731 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7f5324001660 con 0x7f53280a7250 2022-08-01T19:44:13.538 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532cd25700 1 -- 172.21.15.17:0/3075144731 >> v1:172.21.15.17:6789/0 conn(0x7f53280a7250 legacy=0x7f53281448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:13.539 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532cd25700 1 --2- 172.21.15.17:0/3075144731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5310003750 0x7f5310003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:13.539 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532cd25700 1 --2- 172.21.15.17:0/3075144731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5310004500 0x7f5310004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:13.539 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532cd25700 1 --2- 172.21.15.17:0/3075144731 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5328148530 0x7f5310004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:13.540 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532cd25700 1 -- 172.21.15.17:0/3075144731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53280b0780 con 0x7f5310004500 2022-08-01T19:44:13.540 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532cd25700 1 -- 172.21.15.17:0/3075144731 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5310004320 con 0x7f5310003750 2022-08-01T19:44:13.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532cd25700 1 -- 172.21.15.17:0/3075144731 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f531000b780 con 0x7f5328148530 2022-08-01T19:44:13.541 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532d526700 1 --2- 172.21.15.17:0/3075144731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5310004500 0x7f5310004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:13.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532e528700 1 --2- 172.21.15.17:0/3075144731 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5328148530 0x7f5310004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:13.542 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532dd27700 1 --2- 172.21.15.17:0/3075144731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5310003750 0x7f5310003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:13.543 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532d526700 1 -- 172.21.15.17:0/3075144731 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5328148530 msgr2=0x7f5310004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:44:13.544 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532d526700 1 --2- 172.21.15.17:0/3075144731 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5328148530 0x7f5310004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:13.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532d526700 1 -- 172.21.15.17:0/3075144731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5310003750 msgr2=0x7f5310003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:44:13.545 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532d526700 1 --2- 172.21.15.17:0/3075144731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5310003750 0x7f5310003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:13.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532d526700 1 -- 172.21.15.17:0/3075144731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5310003260 con 0x7f5310004500 2022-08-01T19:44:13.546 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532d526700 1 --2- 172.21.15.17:0/3075144731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5310004500 0x7f5310004980 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 rx=0x7f531800c000 tx=0x7f5318006df0).ready entity=mon.0 client_cookie=d07699e8f3175b server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:13.547 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532cd25700 1 -- 172.21.15.17:0/3075144731 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f531800e800 con 0x7f5310004500 2022-08-01T19:44:13.547 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532cd25700 1 -- 172.21.15.17:0/3075144731 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531800e960 con 0x7f5310004500 2022-08-01T19:44:13.547 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.517+0000 7f532cd25700 1 -- 172.21.15.17:0/3075144731 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f531800ecc0 con 0x7f5310004500 2022-08-01T19:44:13.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.521+0000 7f532ff8b700 1 -- 172.21.15.17:0/3075144731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5310004500 msgr2=0x7f5310004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:13.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.521+0000 7f532ff8b700 1 --2- 172.21.15.17:0/3075144731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5310004500 0x7f5310004980 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 rx=0x7f531800c000 tx=0x7f5318006df0).stop 2022-08-01T19:44:13.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.521+0000 7f532e528700 1 -- 172.21.15.17:0/3075144731 reap_dead start 2022-08-01T19:44:13.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.521+0000 7f532ff8b700 1 -- 172.21.15.17:0/3075144731 shutdown_connections 2022-08-01T19:44:13.550 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.521+0000 7f532ff8b700 1 -- 172.21.15.17:0/3075144731 >> 172.21.15.17:0/3075144731 conn(0x7f532805f840 msgr2=0x7f532805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:13.550 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.521+0000 7f532ff8b700 1 -- 172.21.15.17:0/3075144731 shutdown_connections 2022-08-01T19:44:13.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.521+0000 7f532ff8b700 1 -- 172.21.15.17:0/3075144731 wait complete. 2022-08-01T19:44:13.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.521+0000 7f532ff8b700 1 Processor -- start 2022-08-01T19:44:13.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.521+0000 7f532ff8b700 1 -- start start 2022-08-01T19:44:13.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.525+0000 7f532ff8b700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5310004500 0x7f53281430e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:13.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.525+0000 7f532ff8b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f532813f320 0x7f532815a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:13.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.525+0000 7f532ff8b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53280ada60 0x7f532815a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:13.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.525+0000 7f532ff8b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5328159fc0 con 0x7f53280ada60 2022-08-01T19:44:13.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.525+0000 7f532ff8b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53281404b0 con 0x7f532813f320 2022-08-01T19:44:13.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.525+0000 7f532ff8b700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5328140750 con 0x7f5310004500 2022-08-01T19:44:13.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.525+0000 7f532e528700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53280ada60 0x7f532815a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:13.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.525+0000 7f532dd27700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5310004500 0x7f53281430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:13.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.525+0000 7f532e528700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53280ada60 0x7f532815a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45878/0 (socket says 172.21.15.17:45878) 2022-08-01T19:44:13.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.525+0000 7f532e528700 1 -- 172.21.15.17:0/2092457666 learned_addr learned my addr 172.21.15.17:0/2092457666 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:44:13.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.525+0000 7f532d526700 1 --2- 172.21.15.17:0/2092457666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f532813f320 0x7f532815a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:13.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.525+0000 7f532e528700 1 -- 172.21.15.17:0/2092457666 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5310004500 msgr2=0x7f53281430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:13.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.525+0000 7f532e528700 1 --2- 172.21.15.17:0/2092457666 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5310004500 0x7f53281430e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:13.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.525+0000 7f532e528700 1 -- 172.21.15.17:0/2092457666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f532813f320 msgr2=0x7f532815a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:13.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.525+0000 7f532e528700 1 --2- 172.21.15.17:0/2092457666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f532813f320 0x7f532815a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:13.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.525+0000 7f532e528700 1 -- 172.21.15.17:0/2092457666 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532813f870 con 0x7f53280ada60 2022-08-01T19:44:13.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.525+0000 7f532e528700 1 --2- 172.21.15.17:0/2092457666 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53280ada60 0x7f532815a8e0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 rx=0x7f5324000b60 tx=0x7f53280b0bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:13.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.525+0000 7f5316ffd700 1 -- 172.21.15.17:0/2092457666 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5324012540 con 0x7f53280ada60 2022-08-01T19:44:13.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.525+0000 7f5316ffd700 1 -- 172.21.15.17:0/2092457666 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53240126a0 con 0x7f53280ada60 2022-08-01T19:44:13.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.525+0000 7f532ff8b700 1 -- 172.21.15.17:0/2092457666 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f532813fb10 con 0x7f53280ada60 2022-08-01T19:44:13.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.525+0000 7f5316ffd700 1 -- 172.21.15.17:0/2092457666 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5324012a00 con 0x7f53280ada60 2022-08-01T19:44:13.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.525+0000 7f532ff8b700 1 -- 172.21.15.17:0/2092457666 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f532815add0 con 0x7f53280ada60 2022-08-01T19:44:13.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.529+0000 7f532ff8b700 1 -- 172.21.15.17:0/2092457666 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53280b0cc0 con 0x7f53280ada60 2022-08-01T19:44:13.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.529+0000 7f5316ffd700 1 -- 172.21.15.17:0/2092457666 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5324012b60 con 0x7f53280ada60 2022-08-01T19:44:13.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.529+0000 7f5316ffd700 1 --2- 172.21.15.17:0/2092457666 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5308053b30 0x7f5308055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:13.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.529+0000 7f5316ffd700 1 -- 172.21.15.17:0/2092457666 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f532406ba40 con 0x7f53280ada60 2022-08-01T19:44:13.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.529+0000 7f532dd27700 1 --2- 172.21.15.17:0/2092457666 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5308053b30 0x7f5308055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:13.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.529+0000 7f532dd27700 1 --2- 172.21.15.17:0/2092457666 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5308053b30 0x7f5308055ff0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7f532000b000 tx=0x7f5320008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:13.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.537+0000 7f5316ffd700 1 -- 172.21.15.17:0/2092457666 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5324042670 con 0x7f53280ada60 2022-08-01T19:44:13.664 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.661+0000 7f532ff8b700 1 -- 172.21.15.17:0/2092457666 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f53280574e0 con 0x7f53280ada60 2022-08-01T19:44:13.670 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.665+0000 7f5316ffd700 1 -- 172.21.15.17:0/2092457666 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v158) v1 ==== 81+0+34051 (secure 0 0 0) 0x7f5324019020 con 0x7f53280ada60 2022-08-01T19:44:13.672 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:44:13.672 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T19:44:13.672 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 158, 2022-08-01T19:44:13.673 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "be659963-3cf3-4374-b3c4-cbe717b6bac2", 2022-08-01T19:44:13.673 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2022-08-01T19:33:50.105743+0000", 2022-08-01T19:44:13.673 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2022-08-01T19:43:59.716124+0000", 2022-08-01T19:44:13.673 INFO:teuthology.orchestra.run.smithi017.stdout: "last_up_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:44:13.673 INFO:teuthology.orchestra.run.smithi017.stdout: "last_in_change": "2022-08-01T19:33:58.040328+0000", 2022-08-01T19:44:13.674 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-08-01T19:44:13.674 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_num": 5799936, 2022-08-01T19:44:13.675 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_set": [ 2022-08-01T19:44:13.675 INFO:teuthology.orchestra.run.smithi017.stdout: "pglog_hardlimit", 2022-08-01T19:44:13.675 INFO:teuthology.orchestra.run.smithi017.stdout: "purged_snapdirs", 2022-08-01T19:44:13.676 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_deletes", 2022-08-01T19:44:13.676 INFO:teuthology.orchestra.run.smithi017.stdout: "sortbitwise" 2022-08-01T19:44:13.677 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:44:13.677 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_version": 6, 2022-08-01T19:44:13.677 INFO:teuthology.orchestra.run.smithi017.stdout: "full_ratio": 0.94999998807907104, 2022-08-01T19:44:13.678 INFO:teuthology.orchestra.run.smithi017.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-08-01T19:44:13.678 INFO:teuthology.orchestra.run.smithi017.stdout: "nearfull_ratio": 0.85000002384185791, 2022-08-01T19:44:13.682 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_snapshot": "", 2022-08-01T19:44:13.683 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_max": 30, 2022-08-01T19:44:13.683 INFO:teuthology.orchestra.run.smithi017.stdout: "max_osd": 8, 2022-08-01T19:44:13.684 INFO:teuthology.orchestra.run.smithi017.stdout: "require_min_compat_client": "jewel", 2022-08-01T19:44:13.684 INFO:teuthology.orchestra.run.smithi017.stdout: "min_compat_client": "jewel", 2022-08-01T19:44:13.684 INFO:teuthology.orchestra.run.smithi017.stdout: "require_osd_release": "octopus", 2022-08-01T19:44:13.685 INFO:teuthology.orchestra.run.smithi017.stdout: "pools": [ 2022-08-01T19:44:13.685 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.687 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 1, 2022-08-01T19:44:13.687 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "device_health_metrics", 2022-08-01T19:44:13.687 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:33:53.001841+0000", 2022-08-01T19:44:13.687 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:44:13.688 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:44:13.688 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:44:13.688 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:44:13.689 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:44:13.689 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:44:13.689 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:44:13.689 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:44:13.690 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 1, 2022-08-01T19:44:13.690 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 1, 2022-08-01T19:44:13.690 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 1, 2022-08-01T19:44:13.690 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 1, 2022-08-01T19:44:13.691 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 1, 2022-08-01T19:44:13.691 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:44:13.691 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:44:13.691 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:44:13.691 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:44:13.692 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:44:13.692 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:44:13.692 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:44:13.692 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.693 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "16", 2022-08-01T19:44:13.693 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:44:13.693 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:44:13.694 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:44:13.694 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:44:13.695 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:44:13.696 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:44:13.696 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:44:13.697 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:44:13.697 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:44:13.698 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:44:13.698 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:44:13.698 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:44:13.699 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:44:13.699 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:44:13.700 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:44:13.700 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:44:13.701 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:44:13.701 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:44:13.701 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:44:13.702 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:44:13.702 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:44:13.703 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:44:13.703 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:44:13.704 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:44:13.704 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:44:13.704 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:44:13.705 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.705 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:44:13.706 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:44:13.707 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:44:13.707 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:44:13.707 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:44:13.708 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:44:13.708 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:44:13.708 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:44:13.709 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:44:13.709 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:44:13.709 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:44:13.709 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:44:13.710 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 1 2022-08-01T19:44:13.710 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.711 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:44:13.711 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_devicehealth": {} 2022-08-01T19:44:13.711 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.712 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.712 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.713 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 2, 2022-08-01T19:44:13.713 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "rbd", 2022-08-01T19:44:13.714 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:34:01.553098+0000", 2022-08-01T19:44:13.714 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:44:13.715 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:44:13.715 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:44:13.716 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:44:13.716 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:44:13.718 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:44:13.719 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:44:13.720 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:44:13.721 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:44:13.721 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:44:13.722 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:44:13.722 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:44:13.723 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:44:13.723 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:44:13.724 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:44:13.724 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:44:13.725 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:44:13.725 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:44:13.726 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:44:13.726 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:44:13.727 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.727 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "19", 2022-08-01T19:44:13.728 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:44:13.728 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:44:13.728 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:44:13.729 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:44:13.729 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:44:13.730 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:44:13.730 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:44:13.731 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:44:13.731 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:44:13.731 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:44:13.732 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:44:13.732 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:44:13.733 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:44:13.733 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:44:13.733 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:44:13.734 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:44:13.735 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:44:13.735 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:44:13.736 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:44:13.736 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:44:13.737 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:44:13.737 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:44:13.738 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:44:13.738 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:44:13.739 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:44:13.739 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:44:13.740 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.740 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:44:13.740 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:44:13.741 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:44:13.741 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:44:13.742 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:44:13.742 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:44:13.743 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:44:13.744 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:44:13.744 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:44:13.744 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:44:13.745 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:44:13.745 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:44:13.746 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:44:13.746 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd": {} 2022-08-01T19:44:13.746 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.747 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.747 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.748 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 29, 2022-08-01T19:44:13.748 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "cephfs_metadata", 2022-08-01T19:44:13.748 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:43:57.196125+0000", 2022-08-01T19:44:13.749 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:44:13.749 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:44:13.750 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:44:13.750 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:44:13.751 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:44:13.751 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:44:13.751 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:44:13.752 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:44:13.752 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:44:13.752 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:44:13.752 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:44:13.753 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:44:13.753 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:44:13.754 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:44:13.754 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:44:13.755 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:44:13.755 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:44:13.756 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:44:13.756 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:44:13.757 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:44:13.757 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.758 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "158", 2022-08-01T19:44:13.759 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:44:13.759 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:44:13.760 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:44:13.760 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:44:13.761 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:44:13.761 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:44:13.762 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:44:13.762 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:44:13.763 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:44:13.763 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:44:13.764 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:44:13.764 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:44:13.765 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:44:13.765 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:44:13.766 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:44:13.766 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:44:13.767 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:44:13.767 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:44:13.768 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:44:13.769 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:44:13.769 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:44:13.770 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:44:13.770 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:44:13.771 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:44:13.771 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:44:13.771 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:44:13.773 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.773 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:44:13.774 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:44:13.774 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:44:13.775 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:44:13.775 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:44:13.775 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:44:13.776 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:44:13.777 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:44:13.777 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:44:13.778 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:44:13.778 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:44:13.779 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2022-08-01T19:44:13.779 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_bias": 4, 2022-08-01T19:44:13.779 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 16, 2022-08-01T19:44:13.780 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_priority": 5 2022-08-01T19:44:13.780 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.781 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:44:13.781 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs": { 2022-08-01T19:44:13.781 INFO:teuthology.orchestra.run.smithi017.stdout: "metadata": "cephfs" 2022-08-01T19:44:13.782 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.782 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.783 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.783 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.784 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 30, 2022-08-01T19:44:13.784 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "cephfs_data", 2022-08-01T19:44:13.784 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2022-08-01T19:43:57.958027+0000", 2022-08-01T19:44:13.785 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2022-08-01T19:44:13.785 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2022-08-01T19:44:13.786 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2022-08-01T19:44:13.786 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2022-08-01T19:44:13.787 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2022-08-01T19:44:13.787 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2022-08-01T19:44:13.788 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2022-08-01T19:44:13.788 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2022-08-01T19:44:13.788 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2022-08-01T19:44:13.789 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2022-08-01T19:44:13.789 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2022-08-01T19:44:13.790 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2022-08-01T19:44:13.790 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2022-08-01T19:44:13.791 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2022-08-01T19:44:13.791 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2022-08-01T19:44:13.792 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2022-08-01T19:44:13.792 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2022-08-01T19:44:13.792 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2022-08-01T19:44:13.793 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2022-08-01T19:44:13.793 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2022-08-01T19:44:13.794 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.795 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "158", 2022-08-01T19:44:13.795 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2022-08-01T19:44:13.795 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2022-08-01T19:44:13.796 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2022-08-01T19:44:13.796 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2022-08-01T19:44:13.797 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2022-08-01T19:44:13.797 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2022-08-01T19:44:13.798 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2022-08-01T19:44:13.798 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2022-08-01T19:44:13.798 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2022-08-01T19:44:13.799 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2022-08-01T19:44:13.799 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2022-08-01T19:44:13.800 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2022-08-01T19:44:13.800 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2022-08-01T19:44:13.800 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2022-08-01T19:44:13.801 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2022-08-01T19:44:13.801 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2022-08-01T19:44:13.801 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2022-08-01T19:44:13.802 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2022-08-01T19:44:13.802 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-08-01T19:44:13.803 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-08-01T19:44:13.803 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2022-08-01T19:44:13.803 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2022-08-01T19:44:13.804 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2022-08-01T19:44:13.804 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2022-08-01T19:44:13.805 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2022-08-01T19:44:13.806 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2022-08-01T19:44:13.806 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.807 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2022-08-01T19:44:13.807 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2022-08-01T19:44:13.808 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2022-08-01T19:44:13.808 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2022-08-01T19:44:13.809 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2022-08-01T19:44:13.809 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2022-08-01T19:44:13.810 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2022-08-01T19:44:13.810 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2022-08-01T19:44:13.811 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2022-08-01T19:44:13.811 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2022-08-01T19:44:13.812 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2022-08-01T19:44:13.812 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2022-08-01T19:44:13.813 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2022-08-01T19:44:13.813 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs": { 2022-08-01T19:44:13.814 INFO:teuthology.orchestra.run.smithi017.stdout: "data": "cephfs" 2022-08-01T19:44:13.814 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.815 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.815 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.816 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:44:13.816 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2022-08-01T19:44:13.817 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.817 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:44:13.818 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a962f14f-cf16-4fa8-84ff-c85325db9132", 2022-08-01T19:44:13.818 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:44:13.819 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:44:13.819 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:44:13.820 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:44:13.821 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:44:13.821 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:44:13.822 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:44:13.823 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 156, 2022-08-01T19:44:13.823 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:44:13.823 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:44:13.824 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:44:13.824 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:44:13.825 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.825 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:44:13.826 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6800", 2022-08-01T19:44:13.826 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:44:13.827 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.827 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.828 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:44:13.828 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6801", 2022-08-01T19:44:13.829 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:44:13.829 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.830 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.830 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.831 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:44:13.832 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:44:13.832 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.833 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:44:13.833 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6802", 2022-08-01T19:44:13.834 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:44:13.834 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.835 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.835 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:44:13.835 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6803", 2022-08-01T19:44:13.836 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:44:13.836 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.836 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.837 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.837 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:44:13.837 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:44:13.837 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.837 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:44:13.838 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6806", 2022-08-01T19:44:13.838 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:44:13.838 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.838 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.839 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:44:13.839 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6807", 2022-08-01T19:44:13.839 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:44:13.839 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.839 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.840 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.840 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:44:13.840 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:44:13.840 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.841 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:44:13.841 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6804", 2022-08-01T19:44:13.841 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:44:13.841 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.841 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.842 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:44:13.842 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6805", 2022-08-01T19:44:13.842 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23008 2022-08-01T19:44:13.842 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.843 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.843 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.843 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6801/23008", 2022-08-01T19:44:13.843 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6803/23008", 2022-08-01T19:44:13.843 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6807/23008", 2022-08-01T19:44:13.844 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6805/23008", 2022-08-01T19:44:13.844 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:44:13.844 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:44:13.844 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:44:13.845 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.845 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.845 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.845 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:44:13.845 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "805f7051-a527-4e98-8844-b6e406a63248", 2022-08-01T19:44:13.846 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:44:13.846 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:44:13.846 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:44:13.846 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:44:13.847 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:44:13.847 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:44:13.847 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:44:13.847 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 155, 2022-08-01T19:44:13.847 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:44:13.848 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:44:13.848 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:44:13.848 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:44:13.848 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.849 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:44:13.849 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6808", 2022-08-01T19:44:13.849 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:44:13.849 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.849 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.850 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:44:13.850 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6809", 2022-08-01T19:44:13.850 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:44:13.850 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.851 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.851 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.851 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:44:13.851 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:44:13.851 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.852 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:44:13.852 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6810", 2022-08-01T19:44:13.852 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:44:13.852 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.852 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.853 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:44:13.853 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6811", 2022-08-01T19:44:13.853 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:44:13.853 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.854 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.854 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.854 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:44:13.854 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:44:13.855 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.855 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:44:13.855 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6814", 2022-08-01T19:44:13.855 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:44:13.855 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.856 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.856 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:44:13.856 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6815", 2022-08-01T19:44:13.856 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:44:13.857 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.857 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.857 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.858 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:44:13.858 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:44:13.858 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.858 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:44:13.859 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6812", 2022-08-01T19:44:13.859 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:44:13.859 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.859 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.859 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:44:13.860 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6813", 2022-08-01T19:44:13.860 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23007 2022-08-01T19:44:13.860 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.860 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.861 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.861 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6809/23007", 2022-08-01T19:44:13.861 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6811/23007", 2022-08-01T19:44:13.861 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6815/23007", 2022-08-01T19:44:13.861 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6813/23007", 2022-08-01T19:44:13.862 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:44:13.862 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:44:13.862 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:44:13.862 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.863 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.863 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.863 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:44:13.863 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "89f7f3df-7c10-48e1-8b69-9e4243923a61", 2022-08-01T19:44:13.863 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:44:13.864 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:44:13.864 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:44:13.864 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:44:13.864 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:44:13.865 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:44:13.865 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:44:13.865 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 156, 2022-08-01T19:44:13.865 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:44:13.865 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:44:13.866 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:44:13.866 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:44:13.866 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.866 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:44:13.867 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6817", 2022-08-01T19:44:13.867 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:44:13.867 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.867 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.867 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:44:13.868 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6819", 2022-08-01T19:44:13.868 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:44:13.868 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.868 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.869 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.869 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:44:13.869 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:44:13.869 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.869 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:44:13.870 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6822", 2022-08-01T19:44:13.870 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:44:13.870 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.870 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.871 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:44:13.871 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6824", 2022-08-01T19:44:13.871 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:44:13.871 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.871 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.872 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.872 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:44:13.872 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:44:13.873 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.873 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:44:13.873 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6830", 2022-08-01T19:44:13.873 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:44:13.874 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.874 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.874 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:44:13.874 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6831", 2022-08-01T19:44:13.875 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:44:13.875 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.875 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.875 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.875 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:44:13.876 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:44:13.876 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.876 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:44:13.876 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6826", 2022-08-01T19:44:13.877 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:44:13.877 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.877 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.877 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:44:13.877 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6829", 2022-08-01T19:44:13.878 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23010 2022-08-01T19:44:13.878 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.878 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.878 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.879 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6819/23010", 2022-08-01T19:44:13.879 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6824/23010", 2022-08-01T19:44:13.879 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6831/23010", 2022-08-01T19:44:13.879 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6829/23010", 2022-08-01T19:44:13.879 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:44:13.880 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:44:13.880 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:44:13.880 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.881 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.881 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.881 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:44:13.882 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "96123468-aabc-4a64-909e-a192c369090d", 2022-08-01T19:44:13.882 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:44:13.882 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:44:13.882 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:44:13.883 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:44:13.883 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:44:13.883 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:44:13.883 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:44:13.883 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 156, 2022-08-01T19:44:13.884 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:44:13.884 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:44:13.884 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:44:13.884 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:44:13.885 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.885 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:44:13.885 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6816", 2022-08-01T19:44:13.885 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:44:13.885 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.886 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.886 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:44:13.886 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6818", 2022-08-01T19:44:13.886 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:44:13.887 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.887 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.887 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.887 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:44:13.887 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:44:13.888 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.888 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:44:13.888 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6820", 2022-08-01T19:44:13.888 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:44:13.889 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.889 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.889 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:44:13.889 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6821", 2022-08-01T19:44:13.889 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:44:13.890 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.890 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.890 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.890 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:44:13.891 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:44:13.891 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.891 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:44:13.891 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6827", 2022-08-01T19:44:13.891 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:44:13.892 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.892 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.892 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:44:13.892 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6828", 2022-08-01T19:44:13.893 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:44:13.893 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.893 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.893 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.893 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:44:13.894 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:44:13.894 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.894 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:44:13.894 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6823", 2022-08-01T19:44:13.894 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:44:13.895 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.895 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.895 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:44:13.895 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6825", 2022-08-01T19:44:13.896 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23009 2022-08-01T19:44:13.896 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.896 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.896 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.897 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6818/23009", 2022-08-01T19:44:13.897 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6821/23009", 2022-08-01T19:44:13.897 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6828/23009", 2022-08-01T19:44:13.898 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6825/23009", 2022-08-01T19:44:13.898 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:44:13.898 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:44:13.899 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:44:13.899 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.899 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.900 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.900 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:44:13.900 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a6e26665-e04b-46ec-ab66-e912acbf044c", 2022-08-01T19:44:13.900 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:44:13.901 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:44:13.901 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:44:13.902 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:44:13.902 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:44:13.902 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:44:13.903 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:44:13.903 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 155, 2022-08-01T19:44:13.903 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:44:13.903 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:44:13.903 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:44:13.904 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:44:13.904 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.904 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:44:13.904 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6810", 2022-08-01T19:44:13.905 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:44:13.905 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.905 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.905 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:44:13.907 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6811", 2022-08-01T19:44:13.907 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:44:13.908 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.908 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.908 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.909 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:44:13.909 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:44:13.909 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.909 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:44:13.909 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6812", 2022-08-01T19:44:13.910 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:44:13.910 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.910 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.910 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:44:13.911 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6813", 2022-08-01T19:44:13.911 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:44:13.911 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.911 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.912 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.912 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:44:13.912 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:44:13.912 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.912 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:44:13.913 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6816", 2022-08-01T19:44:13.913 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:44:13.913 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.913 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.914 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:44:13.914 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6817", 2022-08-01T19:44:13.914 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:44:13.914 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.915 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.915 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.915 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:44:13.916 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:44:13.916 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.916 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:44:13.916 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6814", 2022-08-01T19:44:13.917 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:44:13.917 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.917 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.917 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:44:13.918 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6815", 2022-08-01T19:44:13.918 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23584 2022-08-01T19:44:13.918 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.918 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.918 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.919 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6811/23584", 2022-08-01T19:44:13.919 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6813/23584", 2022-08-01T19:44:13.919 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6817/23584", 2022-08-01T19:44:13.919 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6815/23584", 2022-08-01T19:44:13.920 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:44:13.920 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:44:13.920 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:44:13.920 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.921 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.921 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.921 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:44:13.921 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "502c0034-18ee-44ac-87ff-19b9dddb66a0", 2022-08-01T19:44:13.924 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:44:13.925 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:44:13.928 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:44:13.928 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:44:13.928 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:44:13.928 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:44:13.929 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 15, 2022-08-01T19:44:13.929 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 156, 2022-08-01T19:44:13.929 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:44:13.929 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:44:13.930 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:44:13.930 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:44:13.930 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.930 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:44:13.931 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6802", 2022-08-01T19:44:13.931 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:44:13.931 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.931 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.932 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:44:13.932 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6803", 2022-08-01T19:44:13.932 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:44:13.934 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.934 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.934 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.934 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:44:13.935 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:44:13.935 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.935 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:44:13.936 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6804", 2022-08-01T19:44:13.936 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:44:13.936 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.936 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.937 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:44:13.937 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6805", 2022-08-01T19:44:13.937 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:44:13.938 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.938 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.938 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.938 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:44:13.938 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:44:13.939 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.939 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:44:13.939 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6808", 2022-08-01T19:44:13.939 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:44:13.940 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.940 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.940 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:44:13.941 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6809", 2022-08-01T19:44:13.941 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:44:13.941 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.942 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.942 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.945 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:44:13.949 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:44:13.950 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.950 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:44:13.951 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6806", 2022-08-01T19:44:13.952 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:44:13.952 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.952 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.953 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:44:13.953 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6807", 2022-08-01T19:44:13.954 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23585 2022-08-01T19:44:13.954 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.955 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.955 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.956 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6803/23585", 2022-08-01T19:44:13.956 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6805/23585", 2022-08-01T19:44:13.958 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6809/23585", 2022-08-01T19:44:13.959 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6807/23585", 2022-08-01T19:44:13.960 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:44:13.960 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:44:13.961 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:44:13.962 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.962 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.963 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.963 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:44:13.964 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "ac4e755a-7fe7-46cb-8407-852d5af9ffc5", 2022-08-01T19:44:13.964 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:44:13.965 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:44:13.965 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:44:13.967 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:44:13.968 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:44:13.968 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:44:13.968 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:44:13.969 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 156, 2022-08-01T19:44:13.969 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:44:13.969 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:44:13.969 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:44:13.970 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:44:13.972 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.973 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:44:13.973 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6820", 2022-08-01T19:44:13.973 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:44:13.973 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.974 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.974 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:44:13.974 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6821", 2022-08-01T19:44:13.975 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:44:13.975 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.975 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.976 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.976 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:44:13.976 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:44:13.976 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.977 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:44:13.977 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6823", 2022-08-01T19:44:13.977 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:44:13.978 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.979 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.979 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:44:13.979 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6825", 2022-08-01T19:44:13.979 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:44:13.980 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.980 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.980 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.980 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:44:13.981 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:44:13.981 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.981 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:44:13.981 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6831", 2022-08-01T19:44:13.981 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:44:13.982 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.982 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.982 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:44:13.982 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6833", 2022-08-01T19:44:13.983 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:44:13.983 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.983 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.983 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.984 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:44:13.984 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:44:13.984 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.984 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:44:13.984 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6827", 2022-08-01T19:44:13.985 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:44:13.985 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.985 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.985 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:44:13.986 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6829", 2022-08-01T19:44:13.986 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23587 2022-08-01T19:44:13.986 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.986 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.987 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.987 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6821/23587", 2022-08-01T19:44:13.987 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6825/23587", 2022-08-01T19:44:13.987 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6833/23587", 2022-08-01T19:44:13.988 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6829/23587", 2022-08-01T19:44:13.988 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:44:13.989 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:44:13.989 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:44:13.989 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.990 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.990 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.990 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:44:13.991 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "874761e2-6410-40fc-b60c-e58abe5ce489", 2022-08-01T19:44:13.991 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2022-08-01T19:44:13.991 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2022-08-01T19:44:13.991 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2022-08-01T19:44:13.991 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2022-08-01T19:44:13.992 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2022-08-01T19:44:13.992 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2022-08-01T19:44:13.992 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 14, 2022-08-01T19:44:13.992 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 156, 2022-08-01T19:44:13.993 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2022-08-01T19:44:13.993 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2022-08-01T19:44:13.993 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2022-08-01T19:44:13.993 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:44:13.994 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.994 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:44:13.994 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6818", 2022-08-01T19:44:13.994 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:44:13.995 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.995 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.995 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:44:13.995 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6819", 2022-08-01T19:44:13.996 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:44:13.996 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:13.996 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:13.996 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.996 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2022-08-01T19:44:13.997 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:44:13.997 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.997 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:44:13.997 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6822", 2022-08-01T19:44:13.998 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:44:13.998 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:13.998 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:13.998 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:44:14.000 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6824", 2022-08-01T19:44:14.000 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:44:14.000 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:14.000 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:14.001 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:14.002 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2022-08-01T19:44:14.002 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:44:14.002 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:14.002 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:44:14.003 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6830", 2022-08-01T19:44:14.003 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:44:14.003 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:14.004 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:14.004 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:44:14.004 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6832", 2022-08-01T19:44:14.005 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:44:14.005 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:14.005 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:14.006 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:14.006 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2022-08-01T19:44:14.007 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2022-08-01T19:44:14.007 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:14.007 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2022-08-01T19:44:14.008 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6826", 2022-08-01T19:44:14.008 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:44:14.009 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:14.009 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:14.009 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2022-08-01T19:44:14.011 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.77:6828", 2022-08-01T19:44:14.011 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 23586 2022-08-01T19:44:14.011 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:14.012 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:14.012 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:14.018 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.77:6819/23586", 2022-08-01T19:44:14.025 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.77:6824/23586", 2022-08-01T19:44:14.025 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.77:6832/23586", 2022-08-01T19:44:14.025 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.77:6828/23586", 2022-08-01T19:44:14.027 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2022-08-01T19:44:14.028 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2022-08-01T19:44:14.028 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2022-08-01T19:44:14.028 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2022-08-01T19:44:14.029 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:14.029 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:44:14.029 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_xinfo": [ 2022-08-01T19:44:14.030 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:14.030 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2022-08-01T19:44:14.030 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:44:14.031 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:44:14.031 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:44:14.031 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:44:14.032 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:44:14.033 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.588817+0000", 2022-08-01T19:44:14.033 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:44:14.037 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:14.037 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:14.037 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2022-08-01T19:44:14.038 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:44:14.038 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:44:14.038 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:44:14.038 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:44:14.039 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:44:14.039 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.782753+0000", 2022-08-01T19:44:14.039 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:44:14.039 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:14.039 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:14.040 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2022-08-01T19:44:14.040 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:44:14.040 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:44:14.040 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:44:14.041 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:44:14.041 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:44:14.041 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.837924+0000", 2022-08-01T19:44:14.042 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:44:14.043 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:14.043 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:14.043 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2022-08-01T19:44:14.043 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:44:14.044 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:44:14.044 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:44:14.044 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:44:14.044 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:44:14.045 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.839948+0000", 2022-08-01T19:44:14.045 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:44:14.045 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:14.045 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:14.046 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2022-08-01T19:44:14.046 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:44:14.046 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:44:14.046 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:44:14.047 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:44:14.047 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:44:14.047 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.112801+0000", 2022-08-01T19:44:14.047 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:44:14.048 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:14.048 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:14.048 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2022-08-01T19:44:14.049 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:44:14.049 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:44:14.050 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:44:14.051 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:44:14.051 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:44:14.052 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:56.106704+0000", 2022-08-01T19:44:14.052 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:44:14.053 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:14.053 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:14.053 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 6, 2022-08-01T19:44:14.054 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:44:14.055 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:44:14.055 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:44:14.055 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:44:14.056 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:44:14.056 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.893170+0000", 2022-08-01T19:44:14.057 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:44:14.057 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:14.057 INFO:teuthology.orchestra.run.smithi017.stdout: { 2022-08-01T19:44:14.059 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 7, 2022-08-01T19:44:14.060 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2022-08-01T19:44:14.060 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2022-08-01T19:44:14.061 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2022-08-01T19:44:14.061 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540138292840890367, 2022-08-01T19:44:14.062 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2022-08-01T19:44:14.062 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2022-08-01T19:33:55.886947+0000", 2022-08-01T19:44:14.063 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2022-08-01T19:44:14.063 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:14.070 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2022-08-01T19:44:14.072 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap": [], 2022-08-01T19:44:14.072 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap_items": [], 2022-08-01T19:44:14.073 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_temp": [], 2022-08-01T19:44:14.073 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_temp": [], 2022-08-01T19:44:14.074 INFO:teuthology.orchestra.run.smithi017.stdout: "blacklist": {}, 2022-08-01T19:44:14.074 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profiles": { 2022-08-01T19:44:14.077 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_data": { 2022-08-01T19:44:14.077 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-device-class": "", 2022-08-01T19:44:14.077 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:44:14.078 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-root": "default", 2022-08-01T19:44:14.079 INFO:teuthology.orchestra.run.smithi017.stdout: "jerasure-per-chunk-alignment": "false", 2022-08-01T19:44:14.080 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:44:14.080 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "2", 2022-08-01T19:44:14.081 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:44:14.081 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:44:14.083 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van", 2022-08-01T19:44:14.083 INFO:teuthology.orchestra.run.smithi017.stdout: "w": "8" 2022-08-01T19:44:14.084 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:14.085 INFO:teuthology.orchestra.run.smithi017.stdout: "default": { 2022-08-01T19:44:14.085 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2022-08-01T19:44:14.085 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2022-08-01T19:44:14.086 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "1", 2022-08-01T19:44:14.086 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2022-08-01T19:44:14.087 INFO:teuthology.orchestra.run.smithi017.stdout: "ruleset-failure-domain": "osd", 2022-08-01T19:44:14.087 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van" 2022-08-01T19:44:14.088 INFO:teuthology.orchestra.run.smithi017.stdout: } 2022-08-01T19:44:14.088 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2022-08-01T19:44:14.093 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps_queue": [], 2022-08-01T19:44:14.094 INFO:teuthology.orchestra.run.smithi017.stdout: "new_removed_snaps": [], 2022-08-01T19:44:14.095 INFO:teuthology.orchestra.run.smithi017.stdout: "new_purged_snaps": [], 2022-08-01T19:44:14.095 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_node_flags": {}, 2022-08-01T19:44:14.096 INFO:teuthology.orchestra.run.smithi017.stdout: "device_class_flags": {} 2022-08-01T19:44:14.096 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T19:44:14.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.665+0000 7f532ff8b700 1 -- 172.21.15.17:0/2092457666 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5308053b30 msgr2=0x7f5308055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:14.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.665+0000 7f532ff8b700 1 --2- 172.21.15.17:0/2092457666 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5308053b30 0x7f5308055ff0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7f532000b000 tx=0x7f5320008040).stop 2022-08-01T19:44:14.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.665+0000 7f532ff8b700 1 -- 172.21.15.17:0/2092457666 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53280ada60 msgr2=0x7f532815a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:14.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.665+0000 7f532ff8b700 1 --2- 172.21.15.17:0/2092457666 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53280ada60 0x7f532815a8e0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 rx=0x7f5324000b60 tx=0x7f53280b0bb0).stop 2022-08-01T19:44:14.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.665+0000 7f532ff8b700 1 -- 172.21.15.17:0/2092457666 shutdown_connections 2022-08-01T19:44:14.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.665+0000 7f532ff8b700 1 --2- 172.21.15.17:0/2092457666 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5308053b30 0x7f5308055ff0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:14.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.665+0000 7f532ff8b700 1 --2- 172.21.15.17:0/2092457666 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53280ada60 0x7f532815a8e0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:14.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.665+0000 7f532ff8b700 1 --2- 172.21.15.17:0/2092457666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f532813f320 0x7f532815a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:14.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.665+0000 7f532ff8b700 1 --2- 172.21.15.17:0/2092457666 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5310004500 0x7f53281430e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:14.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.665+0000 7f532ff8b700 1 -- 172.21.15.17:0/2092457666 >> 172.21.15.17:0/2092457666 conn(0x7f532805f840 msgr2=0x7f532814a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:14.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.665+0000 7f532ff8b700 1 -- 172.21.15.17:0/2092457666 shutdown_connections 2022-08-01T19:44:14.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:13.665+0000 7f532ff8b700 1 -- 172.21.15.17:0/2092457666 wait complete. 2022-08-01T19:44:14.108 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:44:14.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.217+0000 7f580dbac700 1 Processor -- start 2022-08-01T19:44:14.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.217+0000 7f580dbac700 1 -- start start 2022-08-01T19:44:14.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f580dbac700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f58080b2710 0x7f58080abca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:14.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f580dbac700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f58080b0470 0x7f58080ac190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:14.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f580dbac700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58080adf10 con 0x7f58080b13f0 2022-08-01T19:44:14.240 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f580dbac700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58080ae1b0 con 0x7f58080b2710 2022-08-01T19:44:14.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f580dbac700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58080a9520 con 0x7f58080b0470 2022-08-01T19:44:14.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f5807fff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f58080b13f0 0x7f58080afd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:38076/0 (socket says 172.21.15.17:38076) 2022-08-01T19:44:14.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f5807fff700 1 -- 172.21.15.17:0/1140249962 learned_addr learned my addr 172.21.15.17:0/1140249962 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:44:14.241 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f580d3ab700 1 --2- 172.21.15.17:0/1140249962 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f58080b0470 0x7f58080ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:14.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f580cbaa700 1 --2- 172.21.15.17:0/1140249962 >> v2:172.21.15.77:3300/0 conn(0x7f58080b2710 0x7f58080abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:14.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f58077fe700 1 -- 172.21.15.17:0/1140249962 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3215056437 0 0) 0x7f58080adf10 con 0x7f58080b13f0 2022-08-01T19:44:14.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f58077fe700 1 -- 172.21.15.17:0/1140249962 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57ec005470 con 0x7f58080b13f0 2022-08-01T19:44:14.242 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f58077fe700 1 -- 172.21.15.17:0/1140249962 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2505062708 0 0) 0x7f57ec005470 con 0x7f58080b13f0 2022-08-01T19:44:14.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f58077fe700 1 -- 172.21.15.17:0/1140249962 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f58080b0470 msgr2=0x7f58080ac190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:44:14.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f58077fe700 1 --2- 172.21.15.17:0/1140249962 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f58080b0470 0x7f58080ac190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:14.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f58077fe700 1 -- 172.21.15.17:0/1140249962 >> v2:172.21.15.77:3300/0 conn(0x7f58080b2710 msgr2=0x7f58080abca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:44:14.243 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f58077fe700 1 --2- 172.21.15.17:0/1140249962 >> v2:172.21.15.77:3300/0 conn(0x7f58080b2710 0x7f58080abca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:14.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f58077fe700 1 -- 172.21.15.17:0/1140249962 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58080a99c0 con 0x7f58080b13f0 2022-08-01T19:44:14.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f58077fe700 1 -- 172.21.15.17:0/1140249962 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7f57f4001660 con 0x7f58080b13f0 2022-08-01T19:44:14.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f58077fe700 1 -- 172.21.15.17:0/1140249962 >> v1:172.21.15.17:6789/0 conn(0x7f58080b13f0 legacy=0x7f58080afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:14.244 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f58077fe700 1 --2- 172.21.15.17:0/1140249962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57ec005960 0x7f57ec005d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:14.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f58077fe700 1 --2- 172.21.15.17:0/1140249962 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f58080b0470 0x7f57ec006710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:14.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f58077fe700 1 --2- 172.21.15.17:0/1140249962 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57ec006c00 0x7f57ec007080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:14.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f58077fe700 1 -- 172.21.15.17:0/1140249962 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58080ae1b0 con 0x7f57ec006c00 2022-08-01T19:44:14.245 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f58077fe700 1 -- 172.21.15.17:0/1140249962 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58080a9520 con 0x7f57ec005960 2022-08-01T19:44:14.246 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f58077fe700 1 -- 172.21.15.17:0/1140249962 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec00d960 con 0x7f58080b0470 2022-08-01T19:44:14.246 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f580d3ab700 1 --2- 172.21.15.17:0/1140249962 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57ec006c00 0x7f57ec007080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:14.246 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f5807fff700 1 --2- 172.21.15.17:0/1140249962 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f58080b0470 0x7f57ec006710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:14.246 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f580cbaa700 1 --2- 172.21.15.17:0/1140249962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57ec005960 0x7f57ec005d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:14.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f580d3ab700 1 -- 172.21.15.17:0/1140249962 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f58080b0470 msgr2=0x7f57ec006710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:44:14.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f580d3ab700 1 --2- 172.21.15.17:0/1140249962 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f58080b0470 0x7f57ec006710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:14.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f580d3ab700 1 -- 172.21.15.17:0/1140249962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57ec005960 msgr2=0x7f57ec005d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:44:14.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f580d3ab700 1 --2- 172.21.15.17:0/1140249962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57ec005960 0x7f57ec005d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:14.247 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f580d3ab700 1 -- 172.21.15.17:0/1140249962 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ec005470 con 0x7f57ec006c00 2022-08-01T19:44:14.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f580d3ab700 1 --2- 172.21.15.17:0/1140249962 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57ec006c00 0x7f57ec007080 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 rx=0x7f580000c000 tx=0x7f58000091d0).ready entity=mon.0 client_cookie=84ab4cf581e83318 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:14.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f58077fe700 1 -- 172.21.15.17:0/1140249962 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f580000e7b0 con 0x7f57ec006c00 2022-08-01T19:44:14.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f58077fe700 1 -- 172.21.15.17:0/1140249962 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580000e910 con 0x7f57ec006c00 2022-08-01T19:44:14.248 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f58077fe700 1 -- 172.21.15.17:0/1140249962 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f580000ec70 con 0x7f57ec006c00 2022-08-01T19:44:14.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f580dbac700 1 -- 172.21.15.17:0/1140249962 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57ec006c00 msgr2=0x7f57ec007080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:14.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f580dbac700 1 --2- 172.21.15.17:0/1140249962 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57ec006c00 0x7f57ec007080 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 rx=0x7f580000c000 tx=0x7f58000091d0).stop 2022-08-01T19:44:14.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.221+0000 7f580d3ab700 1 -- 172.21.15.17:0/1140249962 reap_dead start 2022-08-01T19:44:14.249 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.225+0000 7f580dbac700 1 -- 172.21.15.17:0/1140249962 shutdown_connections 2022-08-01T19:44:14.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.225+0000 7f580dbac700 1 -- 172.21.15.17:0/1140249962 >> 172.21.15.17:0/1140249962 conn(0x7f5808060310 msgr2=0x7f58080b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:14.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.225+0000 7f580dbac700 1 -- 172.21.15.17:0/1140249962 shutdown_connections 2022-08-01T19:44:14.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.225+0000 7f580dbac700 1 -- 172.21.15.17:0/1140249962 wait complete. 2022-08-01T19:44:14.250 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.225+0000 7f580dbac700 1 Processor -- start 2022-08-01T19:44:14.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.225+0000 7f580dbac700 1 -- start start 2022-08-01T19:44:14.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.225+0000 7f580dbac700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5808143a60 0x7f58080a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:14.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.225+0000 7f580dbac700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58080ab680 0x7f58081557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:14.251 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.225+0000 7f580dbac700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58080a0080 0x7f58080a8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:14.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.225+0000 7f580dbac700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58080a0f50 con 0x7f58080ab680 2022-08-01T19:44:14.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.225+0000 7f580dbac700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58080a1780 con 0x7f58080a0080 2022-08-01T19:44:14.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.225+0000 7f580dbac700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5808143e40 con 0x7f5808143a60 2022-08-01T19:44:14.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.225+0000 7f5807fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58080ab680 0x7f58081557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:14.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.225+0000 7f5807fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58080ab680 0x7f58081557d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45896/0 (socket says 172.21.15.17:45896) 2022-08-01T19:44:14.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.225+0000 7f5807fff700 1 -- 172.21.15.17:0/2355822221 learned_addr learned my addr 172.21.15.17:0/2355822221 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:44:14.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.225+0000 7f580cbaa700 1 --2- 172.21.15.17:0/2355822221 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5808143a60 0x7f58080a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:14.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.225+0000 7f580d3ab700 1 --2- 172.21.15.17:0/2355822221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58080a0080 0x7f58080a8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:14.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.225+0000 7f5807fff700 1 -- 172.21.15.17:0/2355822221 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5808143a60 msgr2=0x7f58080a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:14.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.225+0000 7f5807fff700 1 --2- 172.21.15.17:0/2355822221 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5808143a60 0x7f58080a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:14.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.225+0000 7f5807fff700 1 -- 172.21.15.17:0/2355822221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58080a0080 msgr2=0x7f58080a8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:14.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.225+0000 7f5807fff700 1 --2- 172.21.15.17:0/2355822221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58080a0080 0x7f58080a8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:14.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.225+0000 7f5807fff700 1 -- 172.21.15.17:0/2355822221 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5808152df0 con 0x7f58080ab680 2022-08-01T19:44:14.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.225+0000 7f5807fff700 1 --2- 172.21.15.17:0/2355822221 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58080ab680 0x7f58081557d0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 rx=0x7f57f400d000 tx=0x7f57f4001e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:14.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.225+0000 7f5805ffb700 1 -- 172.21.15.17:0/2355822221 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57f40116b0 con 0x7f58080ab680 2022-08-01T19:44:14.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.229+0000 7f5805ffb700 1 -- 172.21.15.17:0/2355822221 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f4011810 con 0x7f58080ab680 2022-08-01T19:44:14.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.229+0000 7f580dbac700 1 -- 172.21.15.17:0/2355822221 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5808153090 con 0x7f58080ab680 2022-08-01T19:44:14.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.229+0000 7f5805ffb700 1 -- 172.21.15.17:0/2355822221 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57f4011b70 con 0x7f58080ab680 2022-08-01T19:44:14.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.229+0000 7f580dbac700 1 -- 172.21.15.17:0/2355822221 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58080ab3d0 con 0x7f58080ab680 2022-08-01T19:44:14.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.229+0000 7f580dbac700 1 -- 172.21.15.17:0/2355822221 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58080574e0 con 0x7f58080ab680 2022-08-01T19:44:14.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.229+0000 7f5805ffb700 1 -- 172.21.15.17:0/2355822221 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f57f401b030 con 0x7f58080ab680 2022-08-01T19:44:14.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.229+0000 7f5805ffb700 1 --2- 172.21.15.17:0/2355822221 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f57e8053980 0x7f57e8055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:14.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.229+0000 7f5805ffb700 1 -- 172.21.15.17:0/2355822221 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f57f406bb20 con 0x7f58080ab680 2022-08-01T19:44:14.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.229+0000 7f580cbaa700 1 --2- 172.21.15.17:0/2355822221 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f57e8053980 0x7f57e8055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:14.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.233+0000 7f580cbaa700 1 --2- 172.21.15.17:0/2355822221 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f57e8053980 0x7f57e8055e40 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7f57fc00b000 tx=0x7f57fc006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:14.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.237+0000 7f5805ffb700 1 -- 172.21.15.17:0/2355822221 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f57f40428e0 con 0x7f58080ab680 2022-08-01T19:44:14.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.401+0000 7f580dbac700 1 -- 172.21.15.17:0/2355822221 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5808069a70 con 0x7f58080ab680 2022-08-01T19:44:14.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.401+0000 7f5805ffb700 1 -- 172.21.15.17:0/2355822221 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+2486 (secure 0 0 0) 0x7f57f4019030 con 0x7f58080ab680 2022-08-01T19:44:14.407 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:44:14.408 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":83,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8665,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/608187432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":608187432},{"type":"v1","addr":"172.21.15.77:6835","nonce":608187432}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":81},{"gid":8673,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2764139073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2764139073},{"type":"v1","addr":"172.21.15.17:6833","nonce":2764139073}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":80}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:43:58.713330+0000","modified":"2022-08-01T19:43:59.717120+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8682},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8682":{"gid":8682,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/75309005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":75309005},{"type":"v1","addr":"172.21.15.17:6835","nonce":75309005}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2022-08-01T19:44:14.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.405+0000 7f580dbac700 1 -- 172.21.15.17:0/2355822221 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f57e8053980 msgr2=0x7f57e8055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:14.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.405+0000 7f580dbac700 1 --2- 172.21.15.17:0/2355822221 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f57e8053980 0x7f57e8055e40 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7f57fc00b000 tx=0x7f57fc006df0).stop 2022-08-01T19:44:14.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.405+0000 7f580dbac700 1 -- 172.21.15.17:0/2355822221 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58080ab680 msgr2=0x7f58081557d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:14.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.405+0000 7f580dbac700 1 --2- 172.21.15.17:0/2355822221 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58080ab680 0x7f58081557d0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 rx=0x7f57f400d000 tx=0x7f57f4001e50).stop 2022-08-01T19:44:14.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.405+0000 7f580dbac700 1 -- 172.21.15.17:0/2355822221 shutdown_connections 2022-08-01T19:44:14.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.405+0000 7f580dbac700 1 --2- 172.21.15.17:0/2355822221 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f57e8053980 0x7f57e8055e40 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:14.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.405+0000 7f580dbac700 1 --2- 172.21.15.17:0/2355822221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58080a0080 0x7f58080a8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:14.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.405+0000 7f580dbac700 1 --2- 172.21.15.17:0/2355822221 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58080ab680 0x7f58081557d0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:14.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.405+0000 7f580dbac700 1 --2- 172.21.15.17:0/2355822221 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5808143a60 0x7f58080a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:14.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.405+0000 7f580dbac700 1 -- 172.21.15.17:0/2355822221 >> 172.21.15.17:0/2355822221 conn(0x7f5808060310 msgr2=0x7f58080b3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:14.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.405+0000 7f580dbac700 1 -- 172.21.15.17:0/2355822221 shutdown_connections 2022-08-01T19:44:14.412 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.405+0000 7f580dbac700 1 -- 172.21.15.17:0/2355822221 wait complete. 2022-08-01T19:44:14.413 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 83 2022-08-01T19:44:14.426 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-08-01T19:44:14.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.533+0000 7fb1e198c700 1 Processor -- start 2022-08-01T19:44:14.548 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.533+0000 7fb1e198c700 1 -- start start 2022-08-01T19:44:14.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.533+0000 7fb1e198c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1dc060e90 0x7fb1dc0ab0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:14.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.533+0000 7fb1e198c700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fb1dc0aff10 0x7fb1dc1435f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:14.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.533+0000 7fb1e198c700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1dc067370 con 0x7fb1dc066f90 2022-08-01T19:44:14.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.533+0000 7fb1e198c700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1dc0b0340 con 0x7fb1dc0aff10 2022-08-01T19:44:14.550 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.533+0000 7fb1e198c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1dc0a95e0 con 0x7fb1dc060e90 2022-08-01T19:44:14.550 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.533+0000 7fb1db7fe700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fb1dc0aff10 0x7fb1dc1435f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:14.550 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.533+0000 7fb1d27fc700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1dc060e90 0x7fb1dc0ab0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:14.550 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.533+0000 7fb1db7fe700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fb1dc0aff10 0x7fb1dc1435f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:43828/0 (socket says 172.21.15.17:43828) 2022-08-01T19:44:14.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.533+0000 7fb1db7fe700 1 -- 172.21.15.17:0/398292341 learned_addr learned my addr 172.21.15.17:0/398292341 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:44:14.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.533+0000 7fb1daffd700 1 --1- 172.21.15.17:0/398292341 >> v1:172.21.15.77:6789/0 conn(0x7fb1dc066f90 0x7fb1dc0aa990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:54896/0 (socket says 172.21.15.17:54896) 2022-08-01T19:44:14.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.533+0000 7fb1d27fc700 1 -- 172.21.15.17:0/398292341 >> v2:172.21.15.77:3300/0 conn(0x7fb1dc0aff10 msgr2=0x7fb1dc1435f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:44:14.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.533+0000 7fb1d27fc700 1 --2- 172.21.15.17:0/398292341 >> v2:172.21.15.77:3300/0 conn(0x7fb1dc0aff10 0x7fb1dc1435f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:14.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.533+0000 7fb1d27fc700 1 -- 172.21.15.17:0/398292341 >> v1:172.21.15.77:6789/0 conn(0x7fb1dc066f90 legacy=0x7fb1dc0aa990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:14.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.533+0000 7fb1d27fc700 1 -- 172.21.15.17:0/398292341 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1dc0b0540 con 0x7fb1dc060e90 2022-08-01T19:44:14.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.533+0000 7fb1db7fe700 1 --2- 172.21.15.17:0/398292341 >> v2:172.21.15.77:3300/0 conn(0x7fb1dc0aff10 0x7fb1dc1435f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:44:14.552 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.537+0000 7fb1d27fc700 1 --2- 172.21.15.17:0/398292341 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1dc060e90 0x7fb1dc0ab0a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7fb1c4001110 tx=0x7fb1cc006df0).ready entity=mon.2 client_cookie=1601d9014e94d729 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:14.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.537+0000 7fb1da7fc700 1 -- 172.21.15.17:0/398292341 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1cc00d4a0 con 0x7fb1dc060e90 2022-08-01T19:44:14.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.537+0000 7fb1da7fc700 1 -- 172.21.15.17:0/398292341 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1cc000e90 con 0x7fb1dc060e90 2022-08-01T19:44:14.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.537+0000 7fb1da7fc700 1 -- 172.21.15.17:0/398292341 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1cc00d7f0 con 0x7fb1dc060e90 2022-08-01T19:44:14.553 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.537+0000 7fb1e198c700 1 -- 172.21.15.17:0/398292341 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1dc060e90 msgr2=0x7fb1dc0ab0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:14.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.537+0000 7fb1e198c700 1 --2- 172.21.15.17:0/398292341 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1dc060e90 0x7fb1dc0ab0a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7fb1c4001110 tx=0x7fb1cc006df0).stop 2022-08-01T19:44:14.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.537+0000 7fb1e198c700 1 -- 172.21.15.17:0/398292341 shutdown_connections 2022-08-01T19:44:14.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.537+0000 7fb1e198c700 1 --2- 172.21.15.17:0/398292341 >> v2:172.21.15.77:3300/0 conn(0x7fb1dc0aff10 0x7fb1dc1435f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:14.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.537+0000 7fb1e198c700 1 --2- 172.21.15.17:0/398292341 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1dc060e90 0x7fb1dc0ab0a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:14.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.537+0000 7fb1e198c700 1 -- 172.21.15.17:0/398292341 >> 172.21.15.17:0/398292341 conn(0x7fb1dc0b5570 msgr2=0x7fb1dc0b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:14.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.537+0000 7fb1e198c700 1 -- 172.21.15.17:0/398292341 shutdown_connections 2022-08-01T19:44:14.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.537+0000 7fb1e198c700 1 -- 172.21.15.17:0/398292341 wait complete. 2022-08-01T19:44:14.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.537+0000 7fb1e198c700 1 Processor -- start 2022-08-01T19:44:14.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.537+0000 7fb1e198c700 1 -- start start 2022-08-01T19:44:14.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.537+0000 7fb1e198c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1dc060e90 0x7fb1dc149750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:14.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.537+0000 7fb1e198c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1dc066f90 0x7fb1dc14a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:14.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.537+0000 7fb1e198c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1dc0aff10 0x7fb1dc13b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:14.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.537+0000 7fb1e198c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1dc1492f0 con 0x7fb1dc0aff10 2022-08-01T19:44:14.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.537+0000 7fb1e198c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1dc148f30 con 0x7fb1dc066f90 2022-08-01T19:44:14.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.537+0000 7fb1e198c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1dc13c7b0 con 0x7fb1dc060e90 2022-08-01T19:44:14.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.537+0000 7fb1db7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1dc0aff10 0x7fb1dc13b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:14.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.537+0000 7fb1d27fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1dc066f90 0x7fb1dc14a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:14.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.537+0000 7fb1db7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1dc0aff10 0x7fb1dc13b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45912/0 (socket says 172.21.15.17:45912) 2022-08-01T19:44:14.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.537+0000 7fb1db7fe700 1 -- 172.21.15.17:0/2347340982 learned_addr learned my addr 172.21.15.17:0/2347340982 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:44:14.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.537+0000 7fb1daffd700 1 --2- 172.21.15.17:0/2347340982 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1dc060e90 0x7fb1dc149750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:14.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.541+0000 7fb1db7fe700 1 -- 172.21.15.17:0/2347340982 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1dc060e90 msgr2=0x7fb1dc149750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:14.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.541+0000 7fb1db7fe700 1 --2- 172.21.15.17:0/2347340982 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1dc060e90 0x7fb1dc149750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:14.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.541+0000 7fb1db7fe700 1 -- 172.21.15.17:0/2347340982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1dc066f90 msgr2=0x7fb1dc14a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:14.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.541+0000 7fb1db7fe700 1 --2- 172.21.15.17:0/2347340982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1dc066f90 0x7fb1dc14a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:14.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.541+0000 7fb1db7fe700 1 -- 172.21.15.17:0/2347340982 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d4009530 con 0x7fb1dc0aff10 2022-08-01T19:44:14.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.541+0000 7fb1db7fe700 1 --2- 172.21.15.17:0/2347340982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1dc0aff10 0x7fb1dc13b5d0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 rx=0x7fb1d4006ec0 tx=0x7fb1d4009290).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:14.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.541+0000 7fb1d8ff9700 1 -- 172.21.15.17:0/2347340982 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1d400c850 con 0x7fb1dc0aff10 2022-08-01T19:44:14.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.541+0000 7fb1d8ff9700 1 -- 172.21.15.17:0/2347340982 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d400c9b0 con 0x7fb1dc0aff10 2022-08-01T19:44:14.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.541+0000 7fb1e198c700 1 -- 172.21.15.17:0/2347340982 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1cc00b180 con 0x7fb1dc0aff10 2022-08-01T19:44:14.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.541+0000 7fb1d8ff9700 1 -- 172.21.15.17:0/2347340982 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1d400cd10 con 0x7fb1dc0aff10 2022-08-01T19:44:14.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.541+0000 7fb1e198c700 1 -- 172.21.15.17:0/2347340982 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1dc13ddf0 con 0x7fb1dc0aff10 2022-08-01T19:44:14.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.541+0000 7fb1e198c700 1 -- 172.21.15.17:0/2347340982 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1dc0574e0 con 0x7fb1dc0aff10 2022-08-01T19:44:14.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.541+0000 7fb1d8ff9700 1 -- 172.21.15.17:0/2347340982 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb1d4022070 con 0x7fb1dc0aff10 2022-08-01T19:44:14.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.541+0000 7fb1d8ff9700 1 --2- 172.21.15.17:0/2347340982 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb1c0053950 0x7fb1c0055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:14.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.541+0000 7fb1d8ff9700 1 -- 172.21.15.17:0/2347340982 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb1d406c740 con 0x7fb1dc0aff10 2022-08-01T19:44:14.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.541+0000 7fb1daffd700 1 --2- 172.21.15.17:0/2347340982 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb1c0053950 0x7fb1c0055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:14.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.545+0000 7fb1daffd700 1 --2- 172.21.15.17:0/2347340982 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb1c0053950 0x7fb1c0055e10 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7fb1c400d000 tx=0x7fb1c4009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:14.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.549+0000 7fb1d8ff9700 1 -- 172.21.15.17:0/2347340982 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb1d4043500 con 0x7fb1dc0aff10 2022-08-01T19:44:14.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:14.709+0000 7fb1e198c700 1 -- 172.21.15.17:0/2347340982 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb1dc069a70 con 0x7fb1dc0aff10 2022-08-01T19:44:15.262 INFO:tasks.ceph.mon.a.smithi017.stderr:2022-08-01T19:44:15.261+0000 7f6a53773700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-08-01T19:44:15.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.265+0000 7fb1d8ff9700 1 -- 172.21.15.17:0/2347340982 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v84) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb1d4016090 con 0x7fb1dc0aff10 2022-08-01T19:44:15.272 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.269+0000 7fb1e198c700 1 -- 172.21.15.17:0/2347340982 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb1c0053950 msgr2=0x7fb1c0055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:15.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.269+0000 7fb1e198c700 1 --2- 172.21.15.17:0/2347340982 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb1c0053950 0x7fb1c0055e10 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7fb1c400d000 tx=0x7fb1c4009040).stop 2022-08-01T19:44:15.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.269+0000 7fb1e198c700 1 -- 172.21.15.17:0/2347340982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1dc0aff10 msgr2=0x7fb1dc13b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:15.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.269+0000 7fb1e198c700 1 --2- 172.21.15.17:0/2347340982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1dc0aff10 0x7fb1dc13b5d0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 rx=0x7fb1d4006ec0 tx=0x7fb1d4009290).stop 2022-08-01T19:44:15.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.269+0000 7fb1e198c700 1 -- 172.21.15.17:0/2347340982 shutdown_connections 2022-08-01T19:44:15.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.269+0000 7fb1e198c700 1 --2- 172.21.15.17:0/2347340982 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb1c0053950 0x7fb1c0055e10 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:15.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.269+0000 7fb1e198c700 1 --2- 172.21.15.17:0/2347340982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1dc0aff10 0x7fb1dc13b5d0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:15.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.269+0000 7fb1e198c700 1 --2- 172.21.15.17:0/2347340982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1dc066f90 0x7fb1dc14a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:15.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.269+0000 7fb1e198c700 1 --2- 172.21.15.17:0/2347340982 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb1dc060e90 0x7fb1dc149750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:15.277 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.269+0000 7fb1e198c700 1 -- 172.21.15.17:0/2347340982 >> 172.21.15.17:0/2347340982 conn(0x7fb1dc0b5570 msgr2=0x7fb1dc0b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:15.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.269+0000 7fb1e198c700 1 -- 172.21.15.17:0/2347340982 shutdown_connections 2022-08-01T19:44:15.279 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.269+0000 7fb1e198c700 1 -- 172.21.15.17:0/2347340982 wait complete. 2022-08-01T19:44:15.280 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-08-01T19:44:15.290 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-08-01T19:44:15.296 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2022-08-01T19:44:15.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.385+0000 7f614090a700 1 Processor -- start 2022-08-01T19:44:15.390 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.385+0000 7f614090a700 1 -- start start 2022-08-01T19:44:15.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f614090a700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f613c0b1f40 0x7f613c0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:15.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f614090a700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f613c148530 0x7f613c0b2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:15.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f614090a700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f613c0b04e0 con 0x7f613c0a7250 2022-08-01T19:44:15.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f614090a700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f613c0a9250 con 0x7f613c0b1f40 2022-08-01T19:44:15.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f614090a700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f613c0b0780 con 0x7f613c148530 2022-08-01T19:44:15.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f613ad9d700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f613c0a7250 0x7f613c1448b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:38126/0 (socket says 172.21.15.17:38126) 2022-08-01T19:44:15.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f613ad9d700 1 -- 172.21.15.17:0/1263398877 learned_addr learned my addr 172.21.15.17:0/1263398877 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:44:15.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f613a59c700 1 --2- 172.21.15.17:0/1263398877 >> v2:172.21.15.17:3300/0 conn(0x7f613c0b1f40 0x7f613c0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:15.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f6139d9b700 1 --2- 172.21.15.17:0/1263398877 >> v2:172.21.15.77:3300/0 conn(0x7f613c148530 0x7f613c0b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:15.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f613959a700 1 -- 172.21.15.17:0/1263398877 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1719579362 0 0) 0x7f613c0b04e0 con 0x7f613c0a7250 2022-08-01T19:44:15.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f613959a700 1 -- 172.21.15.17:0/1263398877 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f611c003260 con 0x7f613c0a7250 2022-08-01T19:44:15.394 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f613959a700 1 -- 172.21.15.17:0/1263398877 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1855306764 0 0) 0x7f611c003260 con 0x7f613c0a7250 2022-08-01T19:44:15.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f613959a700 1 -- 172.21.15.17:0/1263398877 >> v2:172.21.15.77:3300/0 conn(0x7f613c148530 msgr2=0x7f613c0b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:44:15.395 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f613959a700 1 --2- 172.21.15.17:0/1263398877 >> v2:172.21.15.77:3300/0 conn(0x7f613c148530 0x7f613c0b2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:15.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f613959a700 1 -- 172.21.15.17:0/1263398877 >> v2:172.21.15.17:3300/0 conn(0x7f613c0b1f40 msgr2=0x7f613c0b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:44:15.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f613959a700 1 --2- 172.21.15.17:0/1263398877 >> v2:172.21.15.17:3300/0 conn(0x7f613c0b1f40 0x7f613c0b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:15.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f613959a700 1 -- 172.21.15.17:0/1263398877 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613c0b0a20 con 0x7f613c0a7250 2022-08-01T19:44:15.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f613959a700 1 -- 172.21.15.17:0/1263398877 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7f6134001660 con 0x7f613c0a7250 2022-08-01T19:44:15.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f613959a700 1 -- 172.21.15.17:0/1263398877 >> v1:172.21.15.17:6789/0 conn(0x7f613c0a7250 legacy=0x7f613c1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:15.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f613959a700 1 --2- 172.21.15.17:0/1263398877 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f611c003750 0x7f611c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:15.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f613959a700 1 --2- 172.21.15.17:0/1263398877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f611c004500 0x7f611c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:15.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f613959a700 1 --2- 172.21.15.17:0/1263398877 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f611c004e70 0x7f611c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:15.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f613959a700 1 -- 172.21.15.17:0/1263398877 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613c0b0780 con 0x7f611c004e70 2022-08-01T19:44:15.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f613959a700 1 -- 172.21.15.17:0/1263398877 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611c00bc40 con 0x7f611c004500 2022-08-01T19:44:15.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f613959a700 1 -- 172.21.15.17:0/1263398877 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f611c00bd80 con 0x7f611c003750 2022-08-01T19:44:15.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f613ad9d700 1 --2- 172.21.15.17:0/1263398877 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f611c004e70 0x7f611c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:15.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f613a59c700 1 --2- 172.21.15.17:0/1263398877 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f611c003750 0x7f611c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:15.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f6139d9b700 1 --2- 172.21.15.17:0/1263398877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f611c004500 0x7f611c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:15.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f613ad9d700 1 -- 172.21.15.17:0/1263398877 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f611c003750 msgr2=0x7f611c003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:44:15.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f613ad9d700 1 --2- 172.21.15.17:0/1263398877 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f611c003750 0x7f611c003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:15.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f613ad9d700 1 -- 172.21.15.17:0/1263398877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f611c004500 msgr2=0x7f611c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:44:15.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f613ad9d700 1 --2- 172.21.15.17:0/1263398877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f611c004500 0x7f611c004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:15.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f613ad9d700 1 -- 172.21.15.17:0/1263398877 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f611c003260 con 0x7f611c004e70 2022-08-01T19:44:15.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f613ad9d700 1 -- 172.21.15.17:0/1263398877 reap_dead start 2022-08-01T19:44:15.485 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f613ad9d700 1 --2- 172.21.15.17:0/1263398877 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f611c004e70 0x7f611c00b750 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 rx=0x7f613c1478c0 tx=0x7f613c0b0bb0).ready entity=mon.0 client_cookie=5ea64bbef4d609e5 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:15.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f613959a700 1 -- 172.21.15.17:0/1263398877 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61340070d0 con 0x7f611c004e70 2022-08-01T19:44:15.488 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f613959a700 1 -- 172.21.15.17:0/1263398877 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6134007230 con 0x7f611c004e70 2022-08-01T19:44:15.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f613959a700 1 -- 172.21.15.17:0/1263398877 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f613400c890 con 0x7f611c004e70 2022-08-01T19:44:15.489 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f614090a700 1 -- 172.21.15.17:0/1263398877 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f611c004e70 msgr2=0x7f611c00b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:15.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f614090a700 1 --2- 172.21.15.17:0/1263398877 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f611c004e70 0x7f611c00b750 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 rx=0x7f613c1478c0 tx=0x7f613c0b0bb0).stop 2022-08-01T19:44:15.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f614090a700 1 -- 172.21.15.17:0/1263398877 shutdown_connections 2022-08-01T19:44:15.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f614090a700 1 --2- 172.21.15.17:0/1263398877 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f611c004e70 0x7f611c00b750 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:15.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f614090a700 1 -- 172.21.15.17:0/1263398877 >> 172.21.15.17:0/1263398877 conn(0x7f613c05f840 msgr2=0x7f613c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:15.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f614090a700 1 -- 172.21.15.17:0/1263398877 shutdown_connections 2022-08-01T19:44:15.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.389+0000 7f614090a700 1 -- 172.21.15.17:0/1263398877 wait complete. 2022-08-01T19:44:15.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.393+0000 7f614090a700 1 Processor -- start 2022-08-01T19:44:15.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.393+0000 7f614090a700 1 -- start start 2022-08-01T19:44:15.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.393+0000 7f614090a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f611c004e70 0x7f613c0aadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:15.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.393+0000 7f614090a700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f613c0aeb50 0x7f613c156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:15.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.393+0000 7f614090a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f613c157230 0x7f613c159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:15.499 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.393+0000 7f614090a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613c0a9510 con 0x7f611c004e70 2022-08-01T19:44:15.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.393+0000 7f614090a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613c156950 con 0x7f613c157230 2022-08-01T19:44:15.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.393+0000 7f614090a700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f613c0ac150 con 0x7f613c0aeb50 2022-08-01T19:44:15.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.393+0000 7f613a59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f611c004e70 0x7f613c0aadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:15.501 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.393+0000 7f613a59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f611c004e70 0x7f613c0aadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45940/0 (socket says 172.21.15.17:45940) 2022-08-01T19:44:15.501 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.393+0000 7f613a59c700 1 -- 172.21.15.17:0/3411250653 learned_addr learned my addr 172.21.15.17:0/3411250653 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:44:15.502 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.393+0000 7f6139d9b700 1 --2- 172.21.15.17:0/3411250653 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f613c0aeb50 0x7f613c156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:15.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.393+0000 7f613ad9d700 1 --2- 172.21.15.17:0/3411250653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f613c157230 0x7f613c159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:15.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.393+0000 7f613a59c700 1 -- 172.21.15.17:0/3411250653 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f613c0aeb50 msgr2=0x7f613c156d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:15.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.393+0000 7f613a59c700 1 --2- 172.21.15.17:0/3411250653 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f613c0aeb50 0x7f613c156d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:15.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.393+0000 7f613a59c700 1 -- 172.21.15.17:0/3411250653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f613c157230 msgr2=0x7f613c159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:15.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.393+0000 7f613a59c700 1 --2- 172.21.15.17:0/3411250653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f613c157230 0x7f613c159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:15.519 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.393+0000 7f613a59c700 1 -- 172.21.15.17:0/3411250653 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613c0ac3f0 con 0x7f611c004e70 2022-08-01T19:44:15.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.393+0000 7f613a59c700 1 --2- 172.21.15.17:0/3411250653 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f611c004e70 0x7f613c0aadc0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 rx=0x7f613c063fa0 tx=0x7f612c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:15.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.393+0000 7f61337fe700 1 -- 172.21.15.17:0/3411250653 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f612c00f940 con 0x7f611c004e70 2022-08-01T19:44:15.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.393+0000 7f61337fe700 1 -- 172.21.15.17:0/3411250653 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f612c00faa0 con 0x7f611c004e70 2022-08-01T19:44:15.520 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.393+0000 7f614090a700 1 -- 172.21.15.17:0/3411250653 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f613c0ac690 con 0x7f611c004e70 2022-08-01T19:44:15.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.393+0000 7f61337fe700 1 -- 172.21.15.17:0/3411250653 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f612c00fc50 con 0x7f611c004e70 2022-08-01T19:44:15.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.393+0000 7f614090a700 1 -- 172.21.15.17:0/3411250653 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f613c1576e0 con 0x7f611c004e70 2022-08-01T19:44:15.521 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.393+0000 7f614090a700 1 -- 172.21.15.17:0/3411250653 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f613c0aef30 con 0x7f611c004e70 2022-08-01T19:44:15.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.397+0000 7f61337fe700 1 -- 172.21.15.17:0/3411250653 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f612c00fdb0 con 0x7f611c004e70 2022-08-01T19:44:15.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.397+0000 7f61337fe700 1 --2- 172.21.15.17:0/3411250653 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6120053a10 0x7f6120055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:15.522 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.397+0000 7f61337fe700 1 -- 172.21.15.17:0/3411250653 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f612c06d000 con 0x7f611c004e70 2022-08-01T19:44:15.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.397+0000 7f6139d9b700 1 --2- 172.21.15.17:0/3411250653 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6120053a10 0x7f6120055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:15.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.397+0000 7f6139d9b700 1 --2- 172.21.15.17:0/3411250653 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6120053a10 0x7f6120055ed0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7f612400b000 tx=0x7f6124006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:15.523 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.405+0000 7f61337fe700 1 -- 172.21.15.17:0/3411250653 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f612c043d70 con 0x7f611c004e70 2022-08-01T19:44:15.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.565+0000 7f614090a700 1 -- 172.21.15.17:0/3411250653 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f613c0b0cc0 con 0x7f611c004e70 2022-08-01T19:44:15.578 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.569+0000 7f61337fe700 1 -- 172.21.15.17:0/3411250653 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+2132 (secure 0 0 0) 0x7f612c018030 con 0x7f611c004e70 2022-08-01T19:44:15.579 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T19:44:15.579 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":84,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8665,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.77:6835/608187432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":608187432},{"type":"v1","addr":"172.21.15.77:6835","nonce":608187432}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":81},{"gid":8673,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2764139073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2764139073},{"type":"v1","addr":"172.21.15.17:6833","nonce":2764139073}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":80}],"filesystems":[{"mdsmap":{"epoch":84,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-08-01T19:43:58.713330+0000","modified":"2022-08-01T19:44:15.263114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":159,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2022-08-01T19:44:15.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.569+0000 7f614090a700 1 -- 172.21.15.17:0/3411250653 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6120053a10 msgr2=0x7f6120055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:15.580 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.569+0000 7f614090a700 1 --2- 172.21.15.17:0/3411250653 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6120053a10 0x7f6120055ed0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7f612400b000 tx=0x7f6124006df0).stop 2022-08-01T19:44:15.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.569+0000 7f614090a700 1 -- 172.21.15.17:0/3411250653 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f611c004e70 msgr2=0x7f613c0aadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:15.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.569+0000 7f614090a700 1 --2- 172.21.15.17:0/3411250653 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f611c004e70 0x7f613c0aadc0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 rx=0x7f613c063fa0 tx=0x7f612c0091d0).stop 2022-08-01T19:44:15.581 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.569+0000 7f614090a700 1 -- 172.21.15.17:0/3411250653 shutdown_connections 2022-08-01T19:44:15.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.569+0000 7f614090a700 1 --2- 172.21.15.17:0/3411250653 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6120053a10 0x7f6120055ed0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:15.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.569+0000 7f614090a700 1 --2- 172.21.15.17:0/3411250653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f613c157230 0x7f613c159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:15.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.569+0000 7f614090a700 1 --2- 172.21.15.17:0/3411250653 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f613c0aeb50 0x7f613c156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:15.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.569+0000 7f614090a700 1 --2- 172.21.15.17:0/3411250653 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f611c004e70 0x7f613c0aadc0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:15.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.569+0000 7f614090a700 1 -- 172.21.15.17:0/3411250653 >> 172.21.15.17:0/3411250653 conn(0x7f613c05f840 msgr2=0x7f613c0b40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:15.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.569+0000 7f614090a700 1 -- 172.21.15.17:0/3411250653 shutdown_connections 2022-08-01T19:44:15.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.569+0000 7f614090a700 1 -- 172.21.15.17:0/3411250653 wait complete. 2022-08-01T19:44:15.583 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 84 2022-08-01T19:44:15.585 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-08-01T19:44:15.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.697+0000 7fb68b057700 1 Processor -- start 2022-08-01T19:44:15.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.697+0000 7fb68b057700 1 -- start start 2022-08-01T19:44:15.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.701+0000 7fb68b057700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb6840614b0 0x7fb684061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:15.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.701+0000 7fb68b057700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb6840ad750 0x7fb6840b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:15.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.701+0000 7fb68b057700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6840aee80 con 0x7fb6841405d0 2022-08-01T19:44:15.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.701+0000 7fb68b057700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6840b70b0 con 0x7fb6840ad750 2022-08-01T19:44:15.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.701+0000 7fb68b057700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6840af0b0 con 0x7fb6840614b0 2022-08-01T19:44:15.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.701+0000 7fb689854700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb6840ad750 0x7fb6840b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:15.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.701+0000 7fb689854700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb6840ad750 0x7fb6840b9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45950/0 (socket says 172.21.15.17:45950) 2022-08-01T19:44:15.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.701+0000 7fb689854700 1 -- 172.21.15.17:0/2384146228 learned_addr learned my addr 172.21.15.17:0/2384146228 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:44:15.741 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.701+0000 7fb68a055700 1 --2- 172.21.15.17:0/2384146228 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb6840614b0 0x7fb684061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:15.742 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.701+0000 7fb689854700 1 -- 172.21.15.17:0/2384146228 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb6840614b0 msgr2=0x7fb684061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:44:15.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.701+0000 7fb689854700 1 --2- 172.21.15.17:0/2384146228 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb6840614b0 0x7fb684061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:15.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.701+0000 7fb689854700 1 -- 172.21.15.17:0/2384146228 >> v1:172.21.15.77:6789/0 conn(0x7fb6841405d0 legacy=0x7fb6841466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:15.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.701+0000 7fb689854700 1 -- 172.21.15.17:0/2384146228 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6840af350 con 0x7fb6840ad750 2022-08-01T19:44:15.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.701+0000 7fb689854700 1 --2- 172.21.15.17:0/2384146228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6840ad750 0x7fb6840b9ff0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 rx=0x7fb674002f70 tx=0x7fb674006eb0).ready entity=mon.0 client_cookie=c03430dbc419c797 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:15.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.701+0000 7fb689053700 1 -- 172.21.15.17:0/2384146228 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb67400d570 con 0x7fb6840ad750 2022-08-01T19:44:15.771 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.701+0000 7fb689053700 1 -- 172.21.15.17:0/2384146228 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb674000e90 con 0x7fb6840ad750 2022-08-01T19:44:15.772 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.701+0000 7fb689053700 1 -- 172.21.15.17:0/2384146228 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb67400d900 con 0x7fb6840ad750 2022-08-01T19:44:15.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.701+0000 7fb68b057700 1 -- 172.21.15.17:0/2384146228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6840ad750 msgr2=0x7fb6840b9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:15.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.701+0000 7fb68b057700 1 --2- 172.21.15.17:0/2384146228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6840ad750 0x7fb6840b9ff0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 rx=0x7fb674002f70 tx=0x7fb674006eb0).stop 2022-08-01T19:44:15.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.701+0000 7fb68b057700 1 -- 172.21.15.17:0/2384146228 shutdown_connections 2022-08-01T19:44:15.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.701+0000 7fb68b057700 1 --2- 172.21.15.17:0/2384146228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6840ad750 0x7fb6840b9ff0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:15.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.701+0000 7fb68b057700 1 --2- 172.21.15.17:0/2384146228 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb6840614b0 0x7fb684061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:15.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.701+0000 7fb68b057700 1 -- 172.21.15.17:0/2384146228 >> 172.21.15.17:0/2384146228 conn(0x7fb684066f70 msgr2=0x7fb684067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.701+0000 7fb68b057700 1 -- 172.21.15.17:0/2384146228 shutdown_connections 2022-08-01T19:44:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.701+0000 7fb68b057700 1 -- 172.21.15.17:0/2384146228 wait complete. 2022-08-01T19:44:15.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.705+0000 7fb68b057700 1 Processor -- start 2022-08-01T19:44:15.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.705+0000 7fb68b057700 1 -- start start 2022-08-01T19:44:15.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.705+0000 7fb68b057700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6840614b0 0x7fb68409ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:15.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.705+0000 7fb68b057700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6840ad750 0x7fb6841564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:15.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.705+0000 7fb68b057700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb6841405d0 0x7fb6840ac490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:15.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.705+0000 7fb68b057700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6840b01e0 con 0x7fb6840ad750 2022-08-01T19:44:15.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.705+0000 7fb68b057700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6841569b0 con 0x7fb6840614b0 2022-08-01T19:44:15.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.705+0000 7fb68b057700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6840b03d0 con 0x7fb6841405d0 2022-08-01T19:44:15.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.705+0000 7fb689854700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6840ad750 0x7fb6841564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:15.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.705+0000 7fb68a055700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6840614b0 0x7fb68409ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:15.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.705+0000 7fb689854700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6840ad750 0x7fb6841564c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45956/0 (socket says 172.21.15.17:45956) 2022-08-01T19:44:15.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.705+0000 7fb689854700 1 -- 172.21.15.17:0/3474121993 learned_addr learned my addr 172.21.15.17:0/3474121993 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:44:15.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.705+0000 7fb68a856700 1 --2- 172.21.15.17:0/3474121993 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb6841405d0 0x7fb6840ac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:15.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.709+0000 7fb689854700 1 -- 172.21.15.17:0/3474121993 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb6841405d0 msgr2=0x7fb6840ac490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:15.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.709+0000 7fb689854700 1 --2- 172.21.15.17:0/3474121993 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb6841405d0 0x7fb6840ac490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:15.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.709+0000 7fb689854700 1 -- 172.21.15.17:0/3474121993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6840614b0 msgr2=0x7fb68409ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:15.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.709+0000 7fb689854700 1 --2- 172.21.15.17:0/3474121993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6840614b0 0x7fb68409ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:15.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.709+0000 7fb689854700 1 -- 172.21.15.17:0/3474121993 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb67400b1e0 con 0x7fb6840ad750 2022-08-01T19:44:15.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.709+0000 7fb68a055700 1 --2- 172.21.15.17:0/3474121993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6840614b0 0x7fb68409ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:44:15.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.709+0000 7fb689854700 1 --2- 172.21.15.17:0/3474121993 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6840ad750 0x7fb6841564c0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 rx=0x7fb674002f70 tx=0x7fb674015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:15.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.709+0000 7fb6737fe700 1 -- 172.21.15.17:0/3474121993 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6740117e0 con 0x7fb6840ad750 2022-08-01T19:44:15.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.709+0000 7fb6737fe700 1 -- 172.21.15.17:0/3474121993 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb674011940 con 0x7fb6840ad750 2022-08-01T19:44:15.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.709+0000 7fb68b057700 1 -- 172.21.15.17:0/3474121993 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6840b0670 con 0x7fb6840ad750 2022-08-01T19:44:15.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.709+0000 7fb6737fe700 1 -- 172.21.15.17:0/3474121993 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb674011ca0 con 0x7fb6840ad750 2022-08-01T19:44:15.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.709+0000 7fb68b057700 1 -- 172.21.15.17:0/3474121993 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6840b3ef0 con 0x7fb6840ad750 2022-08-01T19:44:15.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.709+0000 7fb68b057700 1 -- 172.21.15.17:0/3474121993 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6840b0c90 con 0x7fb6840ad750 2022-08-01T19:44:15.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.709+0000 7fb6737fe700 1 -- 172.21.15.17:0/3474121993 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb674025030 con 0x7fb6840ad750 2022-08-01T19:44:15.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.713+0000 7fb6737fe700 1 --2- 172.21.15.17:0/3474121993 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb664053950 0x7fb664055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:15.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.713+0000 7fb6737fe700 1 -- 172.21.15.17:0/3474121993 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fb67406d3a0 con 0x7fb6840ad750 2022-08-01T19:44:15.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.713+0000 7fb68a055700 1 --2- 172.21.15.17:0/3474121993 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb664053950 0x7fb664055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:15.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.713+0000 7fb68a055700 1 --2- 172.21.15.17:0/3474121993 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb664053950 0x7fb664055e10 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7fb68000b000 tx=0x7fb680006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:15.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.717+0000 7fb6737fe700 1 -- 172.21.15.17:0/3474121993 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb674044110 con 0x7fb6840ad750 2022-08-01T19:44:15.886 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:15.881+0000 7fb68b057700 1 -- 172.21.15.17:0/3474121993 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb6840af5f0 con 0x7fb6840ad750 2022-08-01T19:44:16.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.285+0000 7fb6737fe700 1 -- 172.21.15.17:0/3474121993 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v85) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb674047730 con 0x7fb6840ad750 2022-08-01T19:44:16.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.289+0000 7fb68b057700 1 -- 172.21.15.17:0/3474121993 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb664053950 msgr2=0x7fb664055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:16.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.289+0000 7fb68b057700 1 --2- 172.21.15.17:0/3474121993 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb664053950 0x7fb664055e10 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7fb68000b000 tx=0x7fb680006df0).stop 2022-08-01T19:44:16.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.289+0000 7fb68b057700 1 -- 172.21.15.17:0/3474121993 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6840ad750 msgr2=0x7fb6841564c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:16.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.289+0000 7fb68b057700 1 --2- 172.21.15.17:0/3474121993 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6840ad750 0x7fb6841564c0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 rx=0x7fb674002f70 tx=0x7fb674015000).stop 2022-08-01T19:44:16.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.289+0000 7fb68b057700 1 -- 172.21.15.17:0/3474121993 shutdown_connections 2022-08-01T19:44:16.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.289+0000 7fb68b057700 1 --2- 172.21.15.17:0/3474121993 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fb664053950 0x7fb664055e10 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:16.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.289+0000 7fb68b057700 1 --2- 172.21.15.17:0/3474121993 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fb6841405d0 0x7fb6840ac490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:16.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.289+0000 7fb68b057700 1 --2- 172.21.15.17:0/3474121993 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6840ad750 0x7fb6841564c0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:16.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.289+0000 7fb68b057700 1 --2- 172.21.15.17:0/3474121993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6840614b0 0x7fb68409ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:16.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.289+0000 7fb68b057700 1 -- 172.21.15.17:0/3474121993 >> 172.21.15.17:0/3474121993 conn(0x7fb684066f70 msgr2=0x7fb684067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:16.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.289+0000 7fb68b057700 1 -- 172.21.15.17:0/3474121993 shutdown_connections 2022-08-01T19:44:16.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.289+0000 7fb68b057700 1 -- 172.21.15.17:0/3474121993 wait complete. 2022-08-01T19:44:16.309 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-08-01T19:44:16.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.421+0000 7f7161fb8700 1 Processor -- start 2022-08-01T19:44:16.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.421+0000 7f7161fb8700 1 -- start start 2022-08-01T19:44:16.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.425+0000 7f7161fb8700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f715c0614b0 0x7f715c061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:16.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.425+0000 7f7161fb8700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f715c1405d0 0x7f715c140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:16.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.425+0000 7f7161fb8700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f715c0aee80 con 0x7f715c0ad750 2022-08-01T19:44:16.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.425+0000 7f7161fb8700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f715c0b70b0 con 0x7f715c1405d0 2022-08-01T19:44:16.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.425+0000 7f7161fb8700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f715c0af0b0 con 0x7f715c0614b0 2022-08-01T19:44:16.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.425+0000 7f71617b7700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f715c1405d0 0x7f715c140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:16.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.425+0000 7f71617b7700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f715c1405d0 0x7f715c140a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45966/0 (socket says 172.21.15.17:45966) 2022-08-01T19:44:16.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.425+0000 7f71617b7700 1 -- 172.21.15.17:0/3970318131 learned_addr learned my addr 172.21.15.17:0/3970318131 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:44:16.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.425+0000 7f7160fb6700 1 --2- 172.21.15.17:0/3970318131 >> v2:172.21.15.77:3300/0 conn(0x7f715c0614b0 0x7f715c061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:16.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.425+0000 7f71617b7700 1 -- 172.21.15.17:0/3970318131 >> v2:172.21.15.77:3300/0 conn(0x7f715c0614b0 msgr2=0x7f715c061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:44:16.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.425+0000 7f71617b7700 1 --2- 172.21.15.17:0/3970318131 >> v2:172.21.15.77:3300/0 conn(0x7f715c0614b0 0x7f715c061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:16.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.425+0000 7f71617b7700 1 -- 172.21.15.17:0/3970318131 >> v1:172.21.15.77:6789/0 conn(0x7f715c0ad750 legacy=0x7f715c0b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:16.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.425+0000 7f71617b7700 1 -- 172.21.15.17:0/3970318131 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715c0af350 con 0x7f715c1405d0 2022-08-01T19:44:16.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.425+0000 7f71617b7700 1 --2- 172.21.15.17:0/3970318131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f715c1405d0 0x7f715c140a60 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 rx=0x7f715c0aed30 tx=0x7f7154009290).ready entity=mon.0 client_cookie=b8e397e1e8865ed server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:16.460 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.425+0000 7f715b7fe700 1 -- 172.21.15.17:0/3970318131 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f715400b860 con 0x7f715c1405d0 2022-08-01T19:44:16.461 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.425+0000 7f715b7fe700 1 -- 172.21.15.17:0/3970318131 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f715400b9c0 con 0x7f715c1405d0 2022-08-01T19:44:16.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.425+0000 7f715b7fe700 1 -- 172.21.15.17:0/3970318131 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f715400bb30 con 0x7f715c1405d0 2022-08-01T19:44:16.462 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.425+0000 7f7161fb8700 1 -- 172.21.15.17:0/3970318131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f715c1405d0 msgr2=0x7f715c140a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:16.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.425+0000 7f7161fb8700 1 --2- 172.21.15.17:0/3970318131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f715c1405d0 0x7f715c140a60 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 rx=0x7f715c0aed30 tx=0x7f7154009290).stop 2022-08-01T19:44:16.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.425+0000 7f7161fb8700 1 -- 172.21.15.17:0/3970318131 shutdown_connections 2022-08-01T19:44:16.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.425+0000 7f7161fb8700 1 --2- 172.21.15.17:0/3970318131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f715c1405d0 0x7f715c140a60 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:16.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.425+0000 7f7161fb8700 1 --2- 172.21.15.17:0/3970318131 >> v2:172.21.15.77:3300/0 conn(0x7f715c0614b0 0x7f715c061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:16.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.425+0000 7f7161fb8700 1 -- 172.21.15.17:0/3970318131 >> 172.21.15.17:0/3970318131 conn(0x7f715c066f70 msgr2=0x7f715c067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:16.465 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.425+0000 7f7161fb8700 1 -- 172.21.15.17:0/3970318131 shutdown_connections 2022-08-01T19:44:16.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.425+0000 7f7161fb8700 1 -- 172.21.15.17:0/3970318131 wait complete. 2022-08-01T19:44:16.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.425+0000 7f7161fb8700 1 Processor -- start 2022-08-01T19:44:16.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.429+0000 7f7161fb8700 1 -- start start 2022-08-01T19:44:16.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.429+0000 7f7161fb8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f715c0614b0 0x7f715c155e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:16.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.429+0000 7f7161fb8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f715c0ad750 0x7f715c0abb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:16.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.429+0000 7f7161fb8700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f715c1405d0 0x7f715c0ac040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:16.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.429+0000 7f7161fb8700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c09eab0 con 0x7f715c0614b0 2022-08-01T19:44:16.469 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.429+0000 7f7161fb8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c0aca90 con 0x7f715c0ad750 2022-08-01T19:44:16.471 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.429+0000 7f7161fb8700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f715c1501e0 con 0x7f715c1405d0 2022-08-01T19:44:16.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.429+0000 7f7160fb6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f715c0614b0 0x7f715c155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:16.472 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.429+0000 7f7160fb6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f715c0614b0 0x7f715c155e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45970/0 (socket says 172.21.15.17:45970) 2022-08-01T19:44:16.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.429+0000 7f7160fb6700 1 -- 172.21.15.17:0/3320427541 learned_addr learned my addr 172.21.15.17:0/3320427541 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:44:16.473 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.429+0000 7f71617b7700 1 --2- 172.21.15.17:0/3320427541 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f715c1405d0 0x7f715c0ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:16.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.429+0000 7f715bfff700 1 --2- 172.21.15.17:0/3320427541 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f715c0ad750 0x7f715c0abb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:16.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.429+0000 7f7160fb6700 1 -- 172.21.15.17:0/3320427541 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f715c1405d0 msgr2=0x7f715c0ac040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:16.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.429+0000 7f7160fb6700 1 --2- 172.21.15.17:0/3320427541 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f715c1405d0 0x7f715c0ac040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:16.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.429+0000 7f7160fb6700 1 -- 172.21.15.17:0/3320427541 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f715c0ad750 msgr2=0x7f715c0abb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:16.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.429+0000 7f7160fb6700 1 --2- 172.21.15.17:0/3320427541 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f715c0ad750 0x7f715c0abb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:16.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.429+0000 7f7160fb6700 1 -- 172.21.15.17:0/3320427541 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7154009530 con 0x7f715c0614b0 2022-08-01T19:44:16.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.429+0000 7f7160fb6700 1 --2- 172.21.15.17:0/3320427541 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f715c0614b0 0x7f715c155e80 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 rx=0x7f714400c000 tx=0x7f7144006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:16.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.429+0000 7f7159ffb700 1 -- 172.21.15.17:0/3320427541 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7144010780 con 0x7f715c0614b0 2022-08-01T19:44:16.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.429+0000 7f7159ffb700 1 -- 172.21.15.17:0/3320427541 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71440108e0 con 0x7f715c0614b0 2022-08-01T19:44:16.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.429+0000 7f7161fb8700 1 -- 172.21.15.17:0/3320427541 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f715c1564c0 con 0x7f715c0614b0 2022-08-01T19:44:16.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.429+0000 7f7159ffb700 1 -- 172.21.15.17:0/3320427541 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7144010c40 con 0x7f715c0614b0 2022-08-01T19:44:16.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.429+0000 7f7161fb8700 1 -- 172.21.15.17:0/3320427541 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f715c156a00 con 0x7f715c0614b0 2022-08-01T19:44:16.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.433+0000 7f7161fb8700 1 -- 172.21.15.17:0/3320427541 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f715c09f650 con 0x7f715c0614b0 2022-08-01T19:44:16.490 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.433+0000 7f7159ffb700 1 -- 172.21.15.17:0/3320427541 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f714401c460 con 0x7f715c0614b0 2022-08-01T19:44:16.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.433+0000 7f7159ffb700 1 --2- 172.21.15.17:0/3320427541 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7140053950 0x7f7140055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:16.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.433+0000 7f7159ffb700 1 -- 172.21.15.17:0/3320427541 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f714406c440 con 0x7f715c0614b0 2022-08-01T19:44:16.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.433+0000 7f715bfff700 1 --2- 172.21.15.17:0/3320427541 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7140053950 0x7f7140055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:16.491 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.433+0000 7f715bfff700 1 --2- 172.21.15.17:0/3320427541 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7140053950 0x7f7140055e10 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7f714c00d000 tx=0x7f714c000bc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:16.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.441+0000 7f7159ffb700 1 -- 172.21.15.17:0/3320427541 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f71440430b0 con 0x7f715c0614b0 2022-08-01T19:44:16.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:16.569+0000 7f7161fb8700 1 -- 172.21.15.17:0/3320427541 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f715c0574e0 con 0x7f715c0614b0 2022-08-01T19:44:17.297 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.293+0000 7f7159ffb700 1 -- 172.21.15.17:0/3320427541 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v160) v1 ==== 185+0+0 (secure 0 0 0) 0x7f71440466d0 con 0x7f715c0614b0 2022-08-01T19:44:17.306 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.301+0000 7f7161fb8700 1 -- 172.21.15.17:0/3320427541 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7140053950 msgr2=0x7f7140055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:17.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.301+0000 7f7161fb8700 1 --2- 172.21.15.17:0/3320427541 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7140053950 0x7f7140055e10 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7f714c00d000 tx=0x7f714c000bc0).stop 2022-08-01T19:44:17.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.301+0000 7f7161fb8700 1 -- 172.21.15.17:0/3320427541 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f715c0614b0 msgr2=0x7f715c155e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:17.307 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.301+0000 7f7161fb8700 1 --2- 172.21.15.17:0/3320427541 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f715c0614b0 0x7f715c155e80 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 rx=0x7f714400c000 tx=0x7f7144006df0).stop 2022-08-01T19:44:17.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.301+0000 7f7161fb8700 1 -- 172.21.15.17:0/3320427541 shutdown_connections 2022-08-01T19:44:17.308 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.301+0000 7f7161fb8700 1 --2- 172.21.15.17:0/3320427541 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f7140053950 0x7f7140055e10 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:17.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.301+0000 7f7161fb8700 1 --2- 172.21.15.17:0/3320427541 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f715c1405d0 0x7f715c0ac040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:17.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.301+0000 7f7161fb8700 1 --2- 172.21.15.17:0/3320427541 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f715c0ad750 0x7f715c0abb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:17.309 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.301+0000 7f7161fb8700 1 --2- 172.21.15.17:0/3320427541 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f715c0614b0 0x7f715c155e80 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:17.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.301+0000 7f7161fb8700 1 -- 172.21.15.17:0/3320427541 >> 172.21.15.17:0/3320427541 conn(0x7f715c066f70 msgr2=0x7f715c143970 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:17.310 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.301+0000 7f7161fb8700 1 -- 172.21.15.17:0/3320427541 shutdown_connections 2022-08-01T19:44:17.311 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.301+0000 7f7161fb8700 1 -- 172.21.15.17:0/3320427541 wait complete. 2022-08-01T19:44:17.311 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' removed 2022-08-01T19:44:17.321 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-08-01T19:44:17.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.433+0000 7fdbd7fff700 1 Processor -- start 2022-08-01T19:44:17.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.433+0000 7fdbd7fff700 1 -- start start 2022-08-01T19:44:17.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.437+0000 7fdbd7fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdbd80614b0 0x7fdbd8061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:17.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.437+0000 7fdbd7fff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fdbd80ad750 0x7fdbd80b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:17.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.437+0000 7fdbd7fff700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbd80aee80 con 0x7fdbd81405d0 2022-08-01T19:44:17.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.437+0000 7fdbd7fff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd80b70b0 con 0x7fdbd80ad750 2022-08-01T19:44:17.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.437+0000 7fdbd7fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd80af0b0 con 0x7fdbd80614b0 2022-08-01T19:44:17.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.437+0000 7fdbd77fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fdbd81405d0 0x7fdbd81466b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:38176/0 (socket says 172.21.15.17:38176) 2022-08-01T19:44:17.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.437+0000 7fdbd77fe700 1 -- 172.21.15.17:0/21423133 learned_addr learned my addr 172.21.15.17:0/21423133 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T19:44:17.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.437+0000 7fdbcffff700 1 --2- 172.21.15.17:0/21423133 >> v2:172.21.15.77:3300/0 conn(0x7fdbd80ad750 0x7fdbd80b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:17.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.437+0000 7fdbd6ffd700 1 --2- 172.21.15.17:0/21423133 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdbd80614b0 0x7fdbd8061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:17.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.437+0000 7fdbd67fc700 1 -- 172.21.15.17:0/21423133 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2420501677 0 0) 0x7fdbd80aee80 con 0x7fdbd81405d0 2022-08-01T19:44:17.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.437+0000 7fdbd67fc700 1 -- 172.21.15.17:0/21423133 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbb8003260 con 0x7fdbd81405d0 2022-08-01T19:44:17.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.437+0000 7fdbcffff700 1 -- 172.21.15.17:0/21423133 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdbd80614b0 msgr2=0x7fdbd8061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T19:44:17.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.437+0000 7fdbcffff700 1 --2- 172.21.15.17:0/21423133 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdbd80614b0 0x7fdbd8061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:17.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.437+0000 7fdbcffff700 1 -- 172.21.15.17:0/21423133 >> v1:172.21.15.17:6789/0 conn(0x7fdbd81405d0 legacy=0x7fdbd81466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:17.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.437+0000 7fdbcffff700 1 -- 172.21.15.17:0/21423133 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd80af350 con 0x7fdbd80ad750 2022-08-01T19:44:17.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.437+0000 7fdbcffff700 1 --2- 172.21.15.17:0/21423133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbd80ad750 0x7fdbd80b9ff0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7fdbc0002f70 tx=0x7fdbc0006df0).ready entity=mon.1 client_cookie=9998aa1d05306c7a server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:17.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.437+0000 7fdbd67fc700 1 -- 172.21.15.17:0/21423133 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdbc000d4a0 con 0x7fdbd80ad750 2022-08-01T19:44:17.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.437+0000 7fdbd67fc700 1 -- 172.21.15.17:0/21423133 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc0000e90 con 0x7fdbd80ad750 2022-08-01T19:44:17.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.437+0000 7fdbd67fc700 1 -- 172.21.15.17:0/21423133 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdbc000d7f0 con 0x7fdbd80ad750 2022-08-01T19:44:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.437+0000 7fdbd7fff700 1 -- 172.21.15.17:0/21423133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbd80ad750 msgr2=0x7fdbd80b9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.437+0000 7fdbd7fff700 1 --2- 172.21.15.17:0/21423133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbd80ad750 0x7fdbd80b9ff0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7fdbc0002f70 tx=0x7fdbc0006df0).stop 2022-08-01T19:44:17.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.437+0000 7fdbd7fff700 1 -- 172.21.15.17:0/21423133 shutdown_connections 2022-08-01T19:44:17.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.437+0000 7fdbd7fff700 1 --2- 172.21.15.17:0/21423133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbd80ad750 0x7fdbd80b9ff0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:17.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.437+0000 7fdbd7fff700 1 --2- 172.21.15.17:0/21423133 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdbd80614b0 0x7fdbd8061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:17.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.437+0000 7fdbd7fff700 1 -- 172.21.15.17:0/21423133 >> 172.21.15.17:0/21423133 conn(0x7fdbd8066f70 msgr2=0x7fdbd8067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:17.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.437+0000 7fdbd7fff700 1 -- 172.21.15.17:0/21423133 shutdown_connections 2022-08-01T19:44:17.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.437+0000 7fdbd7fff700 1 -- 172.21.15.17:0/21423133 wait complete. 2022-08-01T19:44:17.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.441+0000 7fdbd7fff700 1 Processor -- start 2022-08-01T19:44:17.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.441+0000 7fdbd7fff700 1 -- start start 2022-08-01T19:44:17.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.441+0000 7fdbd7fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdbd80614b0 0x7fdbd813e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:17.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.441+0000 7fdbd7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbd80ad750 0x7fdbd81444e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:17.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.441+0000 7fdbd7fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbd81405d0 0x7fdbd81449d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:17.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.441+0000 7fdbd7fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd8139720 con 0x7fdbd80ad750 2022-08-01T19:44:17.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.441+0000 7fdbd7fff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd8149260 con 0x7fdbd81405d0 2022-08-01T19:44:17.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.441+0000 7fdbd7fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd8144ec0 con 0x7fdbd80614b0 2022-08-01T19:44:17.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.441+0000 7fdbcffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbd80ad750 0x7fdbd81444e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:17.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.441+0000 7fdbd6ffd700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdbd80614b0 0x7fdbd813e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:17.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.441+0000 7fdbcffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbd80ad750 0x7fdbd81444e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45986/0 (socket says 172.21.15.17:45986) 2022-08-01T19:44:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.441+0000 7fdbcffff700 1 -- 172.21.15.17:0/3132141163 learned_addr learned my addr 172.21.15.17:0/3132141163 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T19:44:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.441+0000 7fdbd77fe700 1 --2- 172.21.15.17:0/3132141163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbd81405d0 0x7fdbd81449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.445+0000 7fdbd6ffd700 1 -- 172.21.15.17:0/3132141163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbd81405d0 msgr2=0x7fdbd81449d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.445+0000 7fdbd6ffd700 1 --2- 172.21.15.17:0/3132141163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbd81405d0 0x7fdbd81449d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:17.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.445+0000 7fdbd6ffd700 1 -- 172.21.15.17:0/3132141163 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbd80ad750 msgr2=0x7fdbd81444e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:17.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.445+0000 7fdbd6ffd700 1 --2- 172.21.15.17:0/3132141163 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbd80ad750 0x7fdbd81444e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:17.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.445+0000 7fdbd6ffd700 1 -- 172.21.15.17:0/3132141163 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc000b180 con 0x7fdbd80614b0 2022-08-01T19:44:17.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.445+0000 7fdbcffff700 1 --2- 172.21.15.17:0/3132141163 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbd80ad750 0x7fdbd81444e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T19:44:17.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.445+0000 7fdbd6ffd700 1 --2- 172.21.15.17:0/3132141163 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdbd80614b0 0x7fdbd813e4a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fdbd8145de0 tx=0x7fdbc8006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:17.457 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.445+0000 7fdbd4ff9700 1 -- 172.21.15.17:0/3132141163 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdbc8010780 con 0x7fdbd80614b0 2022-08-01T19:44:17.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.445+0000 7fdbd4ff9700 1 -- 172.21.15.17:0/3132141163 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc80108e0 con 0x7fdbd80614b0 2022-08-01T19:44:17.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.445+0000 7fdbd4ff9700 1 -- 172.21.15.17:0/3132141163 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdbc8010c40 con 0x7fdbd80614b0 2022-08-01T19:44:17.463 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.445+0000 7fdbd7fff700 1 -- 172.21.15.17:0/3132141163 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd8139e30 con 0x7fdbd80614b0 2022-08-01T19:44:17.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.445+0000 7fdbd7fff700 1 -- 172.21.15.17:0/3132141163 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd813a810 con 0x7fdbd80614b0 2022-08-01T19:44:17.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.445+0000 7fdbd7fff700 1 -- 172.21.15.17:0/3132141163 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbd813a570 con 0x7fdbd80614b0 2022-08-01T19:44:17.464 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.445+0000 7fdbd4ff9700 1 -- 172.21.15.17:0/3132141163 <== mon.2 v2:172.21.15.77:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdbc801b410 con 0x7fdbd80614b0 2022-08-01T19:44:17.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.449+0000 7fdbd4ff9700 1 --2- 172.21.15.17:0/3132141163 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdbbc0539e0 0x7fdbbc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T19:44:17.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.449+0000 7fdbd4ff9700 1 -- 172.21.15.17:0/3132141163 <== mon.2 v2:172.21.15.77:3301/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6068+0+0 (secure 0 0 0) 0x7fdbc806bfe0 con 0x7fdbd80614b0 2022-08-01T19:44:17.466 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.449+0000 7fdbcffff700 1 --2- 172.21.15.17:0/3132141163 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdbbc0539e0 0x7fdbbc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T19:44:17.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.449+0000 7fdbcffff700 1 --2- 172.21.15.17:0/3132141163 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdbbc0539e0 0x7fdbbc055ea0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7fdbc001a000 tx=0x7fdbc0014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T19:44:17.467 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.457+0000 7fdbd4ff9700 1 -- 172.21.15.17:0/3132141163 <== mon.2 v2:172.21.15.77:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdbd813a570 con 0x7fdbd80614b0 2022-08-01T19:44:17.592 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:17.585+0000 7fdbd7fff700 1 -- 172.21.15.17:0/3132141163 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdbd813a570 con 0x7fdbd80614b0 2022-08-01T19:44:18.315 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:18.301+0000 7fdbd4ff9700 1 -- 172.21.15.17:0/3132141163 <== mon.2 v2:172.21.15.77:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v161) v1 ==== 173+0+0 (secure 0 0 0) 0x7fdbd813a570 con 0x7fdbd80614b0 2022-08-01T19:44:18.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:18.305+0000 7fdbd7fff700 1 -- 172.21.15.17:0/3132141163 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdbbc0539e0 msgr2=0x7fdbbc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:18.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:18.305+0000 7fdbd7fff700 1 --2- 172.21.15.17:0/3132141163 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdbbc0539e0 0x7fdbbc055ea0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7fdbc001a000 tx=0x7fdbc0014000).stop 2022-08-01T19:44:18.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:18.305+0000 7fdbd7fff700 1 -- 172.21.15.17:0/3132141163 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdbd80614b0 msgr2=0x7fdbd813e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T19:44:18.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:18.305+0000 7fdbd7fff700 1 --2- 172.21.15.17:0/3132141163 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdbd80614b0 0x7fdbd813e4a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fdbd8145de0 tx=0x7fdbc8006df0).stop 2022-08-01T19:44:18.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:18.305+0000 7fdbd7fff700 1 -- 172.21.15.17:0/3132141163 shutdown_connections 2022-08-01T19:44:18.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:18.305+0000 7fdbd7fff700 1 --2- 172.21.15.17:0/3132141163 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fdbbc0539e0 0x7fdbbc055ea0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:18.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:18.305+0000 7fdbd7fff700 1 --2- 172.21.15.17:0/3132141163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbd81405d0 0x7fdbd81449d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:18.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:18.305+0000 7fdbd7fff700 1 --2- 172.21.15.17:0/3132141163 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbd80ad750 0x7fdbd81444e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:18.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:18.305+0000 7fdbd7fff700 1 --2- 172.21.15.17:0/3132141163 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fdbd80614b0 0x7fdbd813e4a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T19:44:18.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:18.305+0000 7fdbd7fff700 1 -- 172.21.15.17:0/3132141163 >> 172.21.15.17:0/3132141163 conn(0x7fdbd8066f70 msgr2=0x7fdbd8067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T19:44:18.323 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:18.305+0000 7fdbd7fff700 1 -- 172.21.15.17:0/3132141163 shutdown_connections 2022-08-01T19:44:18.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T19:44:18.305+0000 7fdbd7fff700 1 -- 172.21.15.17:0/3132141163 wait complete. 2022-08-01T19:44:18.324 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' removed 2022-08-01T19:44:18.327 INFO:teuthology.orchestra.run:Running command with timeout 60 2022-08-01T19:44:18.327 DEBUG:teuthology.orchestra.run.smithi085:> cd /home/ubuntu/cephtest/mnt.0 && sudo bash -c 'stat -f .' 2022-08-01T19:44:18.710 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:44:18.716 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:44:18.726 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:44:18.741 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:44:48.444 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:44:48.445 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:44:48.446 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:44:48.883 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:44:48.887 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:44:48.890 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:44:48.893 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:45:18.347 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2022-08-01T19:45:18.352 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2022-08-01T19:45:18.947 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:45:18.950 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:45:18.953 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:45:18.956 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:45:49.047 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:45:49.052 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:45:49.056 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:45:49.059 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:46:19.125 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:46:19.128 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:46:19.131 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:46:19.133 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:46:49.186 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:46:49.192 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:46:49.196 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:46:49.199 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:47:19.245 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:47:19.252 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:47:19.256 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:47:19.259 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:47:49.317 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:47:49.325 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:47:49.330 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:47:49.334 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:48:19.387 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:48:19.391 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:48:19.398 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:48:19.402 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:48:49.459 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:48:49.462 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:48:49.465 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:48:49.468 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:49:19.563 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:49:19.567 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:49:19.570 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:49:19.572 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:49:49.659 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:49:49.664 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:49:49.668 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:49:49.671 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:50:19.739 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:50:19.744 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:50:19.748 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:50:19.752 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:50:49.819 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:50:49.823 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:50:49.825 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:50:49.828 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:51:19.893 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:51:19.898 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:51:19.902 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:51:19.906 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:51:49.972 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:51:49.976 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:51:49.979 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:51:49.981 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:52:20.041 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:52:20.046 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:52:20.050 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:52:20.053 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:52:50.124 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:52:50.127 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:52:50.131 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:52:50.135 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:53:20.221 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:53:20.225 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:53:20.228 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:53:20.230 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:53:50.308 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:53:50.312 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:53:50.315 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:53:50.317 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:54:20.425 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:54:20.429 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:54:20.432 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:54:20.435 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:54:50.502 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:54:50.507 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:54:50.511 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:54:50.516 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:55:20.638 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:55:20.642 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:55:20.645 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:55:20.648 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:55:50.738 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:55:50.744 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:55:50.746 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:55:50.749 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:56:20.959 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:56:20.964 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:56:20.969 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:56:20.972 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:56:51.076 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:56:51.080 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:56:51.083 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:56:51.086 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:57:21.213 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:57:21.217 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:57:21.221 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:57:21.223 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:57:51.378 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:57:51.382 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:57:51.385 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:57:51.387 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:58:21.568 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:58:21.572 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:58:21.575 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:58:21.578 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:58:51.657 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:58:51.661 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:58:51.668 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:58:51.671 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:59:21.788 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:59:21.793 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:59:21.796 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:59:21.799 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:59:48.451 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2022-08-01T19:59:48.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T19:59:48.455 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-08-01T19:59:48.456 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:59:48.457 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2022-08-01T19:59:48.479 DEBUG:teuthology.orchestra.run.smithi077:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:59:48.501 DEBUG:teuthology.orchestra.run.smithi085:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:59:48.523 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-08-01T19:59:48.548 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-08-01T19:59:48.548 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T19:59:48.549 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T19:59:48.550 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T19:59:51.929 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:59:51.932 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:59:51.935 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T19:59:51.937 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:00:22.064 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:00:22.068 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:00:22.071 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:00:22.073 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:00:52.377 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:00:52.382 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:00:52.387 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:00:52.390 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:01:22.515 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:01:22.524 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:01:22.527 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:01:22.530 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:01:52.645 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:01:52.650 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:01:52.654 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:01:52.656 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:02:22.893 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:02:22.896 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:02:22.899 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:02:22.902 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:02:53.036 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:02:53.042 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:02:53.045 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:02:53.048 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:03:23.166 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:03:23.171 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:03:23.175 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:03:23.179 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:03:53.312 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:03:53.317 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:03:53.324 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:03:53.340 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:04:23.458 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:04:23.463 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:04:23.466 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:04:23.469 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:04:53.598 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:04:53.603 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:04:53.606 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:04:53.610 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:05:23.737 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:05:23.743 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:05:23.747 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:05:23.752 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:05:53.880 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:05:53.883 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:05:53.887 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:05:53.893 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:06:24.026 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:06:24.031 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:06:24.035 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:06:24.038 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:06:54.205 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:06:54.208 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:06:54.211 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:06:54.213 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:07:24.542 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:07:24.546 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:07:24.549 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:07:24.552 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:07:54.708 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:07:54.712 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:07:54.715 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:07:54.718 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:08:24.787 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:08:24.792 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:08:24.794 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:08:24.797 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:08:54.930 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:08:54.934 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:08:54.938 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:08:54.941 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:09:25.067 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:09:25.071 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:09:25.073 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:09:25.076 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:09:55.213 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:09:55.217 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:09:55.220 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:09:55.223 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:10:25.333 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:10:25.337 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:10:25.341 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:10:25.344 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:10:55.480 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:10:55.485 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:10:55.488 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:10:55.490 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:11:25.610 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:11:25.615 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:11:25.619 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:11:25.622 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:11:55.768 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:11:55.772 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:11:55.776 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:11:55.779 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:12:25.886 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:12:25.894 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:12:25.897 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:12:25.900 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:12:56.047 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:12:56.051 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:12:56.054 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:12:56.056 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:13:26.186 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:13:26.193 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:13:26.197 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:13:26.201 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:13:56.255 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:13:56.260 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:13:56.262 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:13:56.269 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:14:26.395 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:14:26.399 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:14:26.402 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:14:26.405 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:14:48.554 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2022-08-01T20:14:48.555 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2022-08-01T20:14:48.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T20:14:48.564 INFO:tasks.cephfs_test_runner:test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) ... ERROR 2022-08-01T20:14:48.566 INFO:tasks.cephfs_test_runner:ERROR 2022-08-01T20:14:48.567 INFO:tasks.cephfs_test_runner: 2022-08-01T20:14:48.568 INFO:tasks.cephfs_test_runner:====================================================================== 2022-08-01T20:14:48.568 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2022-08-01T20:14:48.569 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2022-08-01T20:14:48.569 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-08-01T20:14:48.570 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr 2022-08-01T20:14:48.570 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2022-08-01T20:14:48.570 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2022-08-01T20:14:48.571 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2022-08-01T20:14:48.571 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2022-08-01T20:14:48.571 INFO:tasks.cephfs_test_runner: 2022-08-01T20:14:48.572 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2022-08-01T20:14:48.572 INFO:tasks.cephfs_test_runner: 2022-08-01T20:14:48.572 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-08-01T20:14:48.572 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/test_misc.py", line 28, in test_statfs_on_deleted_fs 2022-08-01T20:14:48.573 INFO:tasks.cephfs_test_runner: self.mount_a.umount_wait(force=True) 2022-08-01T20:14:48.573 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/fuse_mount.py", line 326, in umount_wait 2022-08-01T20:14:48.573 INFO:tasks.cephfs_test_runner: if not (self.is_mounted() and self.fuse_daemon): 2022-08-01T20:14:48.574 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2022-08-01T20:14:48.574 INFO:tasks.cephfs_test_runner: proc.wait() 2022-08-01T20:14:48.574 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 148, in wait 2022-08-01T20:14:48.575 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2022-08-01T20:14:48.575 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2022-08-01T20:14:48.575 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2022-08-01T20:14:48.575 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise 2022-08-01T20:14:48.576 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2022-08-01T20:14:48.576 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2022-08-01T20:14:48.576 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 323, in copy_file_to 2022-08-01T20:14:48.577 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2022-08-01T20:14:48.577 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 276, in copy_to_log 2022-08-01T20:14:48.577 INFO:tasks.cephfs_test_runner: for line in f: 2022-08-01T20:14:48.578 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2022-08-01T20:14:48.578 INFO:tasks.cephfs_test_runner: line = self.readline() 2022-08-01T20:14:48.579 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2022-08-01T20:14:48.579 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2022-08-01T20:14:48.580 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read 2022-08-01T20:14:48.580 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2022-08-01T20:14:48.581 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr 2022-08-01T20:14:48.582 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2022-08-01T20:14:48.582 INFO:tasks.cephfs_test_runner:socket.timeout 2022-08-01T20:14:48.583 INFO:tasks.cephfs_test_runner: 2022-08-01T20:14:48.583 INFO:tasks.cephfs_test_runner:====================================================================== 2022-08-01T20:14:48.584 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2022-08-01T20:14:48.584 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2022-08-01T20:14:48.585 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-08-01T20:14:48.585 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr 2022-08-01T20:14:48.586 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2022-08-01T20:14:48.586 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2022-08-01T20:14:48.587 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2022-08-01T20:14:48.587 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2022-08-01T20:14:48.588 INFO:tasks.cephfs_test_runner: 2022-08-01T20:14:48.588 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2022-08-01T20:14:48.589 INFO:tasks.cephfs_test_runner: 2022-08-01T20:14:48.589 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-08-01T20:14:48.590 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/cephfs_test_case.py", line 175, in tearDown 2022-08-01T20:14:48.590 INFO:tasks.cephfs_test_runner: m.teardown() 2022-08-01T20:14:48.590 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/fuse_mount.py", line 408, in teardown 2022-08-01T20:14:48.591 INFO:tasks.cephfs_test_runner: self.umount() 2022-08-01T20:14:48.591 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/fuse_mount.py", line 261, in umount 2022-08-01T20:14:48.592 INFO:tasks.cephfs_test_runner: if not self.is_mounted(): 2022-08-01T20:14:48.592 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2022-08-01T20:14:48.592 INFO:tasks.cephfs_test_runner: proc.wait() 2022-08-01T20:14:48.593 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 148, in wait 2022-08-01T20:14:48.594 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2022-08-01T20:14:48.594 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2022-08-01T20:14:48.595 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2022-08-01T20:14:48.596 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise 2022-08-01T20:14:48.596 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2022-08-01T20:14:48.597 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2022-08-01T20:14:48.597 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 323, in copy_file_to 2022-08-01T20:14:48.598 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2022-08-01T20:14:48.598 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 276, in copy_to_log 2022-08-01T20:14:48.599 INFO:tasks.cephfs_test_runner: for line in f: 2022-08-01T20:14:48.599 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2022-08-01T20:14:48.600 INFO:tasks.cephfs_test_runner: line = self.readline() 2022-08-01T20:14:48.600 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2022-08-01T20:14:48.601 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2022-08-01T20:14:48.602 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read 2022-08-01T20:14:48.602 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2022-08-01T20:14:48.603 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr 2022-08-01T20:14:48.603 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2022-08-01T20:14:48.603 INFO:tasks.cephfs_test_runner:socket.timeout 2022-08-01T20:14:48.604 INFO:tasks.cephfs_test_runner: 2022-08-01T20:14:48.605 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2022-08-01T20:14:48.605 INFO:tasks.cephfs_test_runner:Ran 12 tests in 2414.170s 2022-08-01T20:14:48.606 INFO:tasks.cephfs_test_runner: 2022-08-01T20:14:48.606 INFO:tasks.cephfs_test_runner:FAILED (errors=2) 2022-08-01T20:14:48.606 INFO:tasks.cephfs_test_runner: 2022-08-01T20:14:48.607 INFO:tasks.cephfs_test_runner:====================================================================== 2022-08-01T20:14:48.607 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2022-08-01T20:14:48.607 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2022-08-01T20:14:48.607 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-08-01T20:14:48.608 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr 2022-08-01T20:14:48.608 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2022-08-01T20:14:48.608 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2022-08-01T20:14:48.609 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2022-08-01T20:14:48.609 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2022-08-01T20:14:48.609 INFO:tasks.cephfs_test_runner: 2022-08-01T20:14:48.609 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2022-08-01T20:14:48.610 INFO:tasks.cephfs_test_runner: 2022-08-01T20:14:48.610 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-08-01T20:14:48.611 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/test_misc.py", line 28, in test_statfs_on_deleted_fs 2022-08-01T20:14:48.611 INFO:tasks.cephfs_test_runner: self.mount_a.umount_wait(force=True) 2022-08-01T20:14:48.612 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/fuse_mount.py", line 326, in umount_wait 2022-08-01T20:14:48.612 INFO:tasks.cephfs_test_runner: if not (self.is_mounted() and self.fuse_daemon): 2022-08-01T20:14:48.613 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2022-08-01T20:14:48.613 INFO:tasks.cephfs_test_runner: proc.wait() 2022-08-01T20:14:48.613 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 148, in wait 2022-08-01T20:14:48.613 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2022-08-01T20:14:48.614 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2022-08-01T20:14:48.614 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2022-08-01T20:14:48.615 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise 2022-08-01T20:14:48.616 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2022-08-01T20:14:48.616 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2022-08-01T20:14:48.617 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 323, in copy_file_to 2022-08-01T20:14:48.617 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2022-08-01T20:14:48.617 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 276, in copy_to_log 2022-08-01T20:14:48.618 INFO:tasks.cephfs_test_runner: for line in f: 2022-08-01T20:14:48.618 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2022-08-01T20:14:48.619 INFO:tasks.cephfs_test_runner: line = self.readline() 2022-08-01T20:14:48.619 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2022-08-01T20:14:48.619 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2022-08-01T20:14:48.619 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read 2022-08-01T20:14:48.620 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2022-08-01T20:14:48.620 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr 2022-08-01T20:14:48.621 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2022-08-01T20:14:48.621 INFO:tasks.cephfs_test_runner:socket.timeout 2022-08-01T20:14:48.622 INFO:tasks.cephfs_test_runner: 2022-08-01T20:14:48.622 INFO:tasks.cephfs_test_runner:====================================================================== 2022-08-01T20:14:48.623 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2022-08-01T20:14:48.623 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2022-08-01T20:14:48.623 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-08-01T20:14:48.624 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr 2022-08-01T20:14:48.624 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2022-08-01T20:14:48.624 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2022-08-01T20:14:48.624 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2022-08-01T20:14:48.625 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2022-08-01T20:14:48.625 INFO:tasks.cephfs_test_runner: 2022-08-01T20:14:48.625 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2022-08-01T20:14:48.626 INFO:tasks.cephfs_test_runner: 2022-08-01T20:14:48.626 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-08-01T20:14:48.626 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/cephfs_test_case.py", line 175, in tearDown 2022-08-01T20:14:48.627 INFO:tasks.cephfs_test_runner: m.teardown() 2022-08-01T20:14:48.627 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/fuse_mount.py", line 408, in teardown 2022-08-01T20:14:48.627 INFO:tasks.cephfs_test_runner: self.umount() 2022-08-01T20:14:48.628 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/fuse_mount.py", line 261, in umount 2022-08-01T20:14:48.628 INFO:tasks.cephfs_test_runner: if not self.is_mounted(): 2022-08-01T20:14:48.628 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2022-08-01T20:14:48.628 INFO:tasks.cephfs_test_runner: proc.wait() 2022-08-01T20:14:48.629 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 148, in wait 2022-08-01T20:14:48.630 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2022-08-01T20:14:48.631 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2022-08-01T20:14:48.631 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2022-08-01T20:14:48.632 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise 2022-08-01T20:14:48.632 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2022-08-01T20:14:48.633 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2022-08-01T20:14:48.634 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 323, in copy_file_to 2022-08-01T20:14:48.634 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2022-08-01T20:14:48.635 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 276, in copy_to_log 2022-08-01T20:14:48.635 INFO:tasks.cephfs_test_runner: for line in f: 2022-08-01T20:14:48.636 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2022-08-01T20:14:48.637 INFO:tasks.cephfs_test_runner: line = self.readline() 2022-08-01T20:14:48.638 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2022-08-01T20:14:48.639 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2022-08-01T20:14:48.639 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read 2022-08-01T20:14:48.640 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2022-08-01T20:14:48.640 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr 2022-08-01T20:14:48.641 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2022-08-01T20:14:48.641 INFO:tasks.cephfs_test_runner:socket.timeout 2022-08-01T20:14:48.642 INFO:tasks.cephfs_test_runner: 2022-08-01T20:14:48.643 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/run_tasks.py", line 106, in run_tasks manager.__enter__() File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc), test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2022-08-01T20:14:48.762 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=fa5f090a5c454713815ce3f737d65462 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/run_tasks.py", line 106, in run_tasks manager.__enter__() File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc), test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2022-08-01T20:14:48.764 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2022-08-01T20:14:48.788 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2022-08-01T20:14:48.829 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2022-08-01T20:14:48.830 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T20:14:48.831 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T20:14:48.831 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T20:14:56.499 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:14:56.502 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:14:56.509 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:14:56.513 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:15:26.628 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:15:26.632 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:15:26.634 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:15:26.638 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:15:56.968 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:15:56.972 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:15:56.975 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:15:56.979 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:16:27.103 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:16:27.108 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:16:27.111 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:16:27.114 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:16:57.282 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:16:57.288 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:16:57.292 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:16:57.296 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:17:27.466 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:17:27.470 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:17:27.473 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:17:27.477 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:17:57.609 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:17:57.615 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:17:57.618 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:17:57.621 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:18:27.763 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:18:27.769 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:18:27.772 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:18:27.776 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:18:57.900 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:18:57.905 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:18:57.909 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:18:57.912 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:19:28.045 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:19:28.049 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:19:28.052 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:19:28.056 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:19:58.193 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:19:58.198 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:19:58.200 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:19:58.203 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:20:28.354 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:20:28.358 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:20:28.363 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:20:28.368 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:20:58.643 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:20:58.646 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:20:58.649 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:20:58.652 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:21:28.734 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:21:28.738 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:21:28.743 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:21:28.746 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:21:58.925 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:21:58.930 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:21:58.935 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:21:58.939 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:22:29.056 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:22:29.060 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:22:29.064 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:22:29.076 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:22:59.176 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:22:59.180 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:22:59.183 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:22:59.186 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:23:29.305 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:23:29.310 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:23:29.313 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:23:29.316 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:23:59.435 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:23:59.441 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:23:59.446 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:23:59.450 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:24:29.589 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:24:29.593 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:24:29.596 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:24:29.600 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:24:59.711 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:24:59.716 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:24:59.719 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:24:59.722 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:25:29.850 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:25:29.854 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:25:29.858 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:25:29.861 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:25:59.988 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:25:59.994 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:25:59.998 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:26:00.002 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:26:30.228 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:26:30.232 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:26:30.235 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:26:30.239 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:27:00.394 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:27:00.397 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:27:00.400 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:27:00.402 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:27:30.524 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:27:30.529 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:27:30.531 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:27:30.534 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:28:00.838 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:28:00.842 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:28:00.846 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:28:00.849 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:28:30.966 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:28:30.972 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:28:30.976 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:28:30.978 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:29:01.103 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:29:01.107 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:29:01.110 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:29:01.114 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:29:31.230 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:29:31.234 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:29:31.237 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:29:31.240 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:29:48.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T20:29:48.847 INFO:teuthology.orchestra.run.smithi085.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Connection timed out 2022-08-01T20:29:48.848 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2022-08-01T20:29:48.849 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-08-01T20:29:48.850 DEBUG:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-08-01T20:29:48.851 DEBUG:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-08-01T20:30:01.430 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:30:01.433 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:30:01.436 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:30:01.441 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:30:31.563 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:30:31.567 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:30:31.570 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:30:31.576 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:31:01.744 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:31:01.748 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:31:01.751 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:31:01.755 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:31:31.884 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:31:31.890 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:31:31.894 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:31:31.897 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:32:02.049 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:32:02.052 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:32:02.056 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:32:02.063 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:32:32.198 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:32:32.203 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:32:32.209 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:32:32.213 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:33:02.338 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:33:02.342 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:33:02.345 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:33:02.349 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:33:32.460 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:33:32.464 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:33:32.467 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:33:32.469 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:34:02.600 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:34:02.607 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:34:02.617 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:34:02.620 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:34:32.924 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:34:32.929 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:34:32.932 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:34:32.936 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:35:03.075 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:35:03.081 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:35:03.085 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:35:03.088 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:35:33.252 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:35:33.255 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:35:33.258 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:35:33.261 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:36:03.373 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:36:03.390 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:36:03.399 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:36:03.420 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:36:33.537 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:36:33.542 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:36:33.545 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:36:33.548 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:37:03.847 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:37:03.853 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:37:03.857 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:37:03.861 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:37:33.970 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:37:33.973 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:37:33.976 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:37:33.979 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:38:04.107 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:38:04.111 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:38:04.114 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:38:04.117 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:38:34.237 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:38:34.244 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:38:34.248 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:38:34.251 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:39:04.374 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:39:04.378 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:39:04.380 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:39:04.383 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:39:34.495 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:39:34.502 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:39:34.506 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:39:34.510 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:40:04.615 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:40:04.619 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:40:04.622 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:40:04.625 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:40:34.753 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:40:34.756 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:40:34.759 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:40:34.761 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:41:04.966 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:41:04.970 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:41:04.973 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:41:04.976 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:41:35.195 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:41:35.199 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:41:35.202 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:41:35.206 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:42:05.535 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:42:05.539 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:42:05.542 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:42:05.544 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:42:35.663 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:42:35.668 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:42:35.674 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:42:35.677 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:43:06.043 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:43:06.048 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:43:06.051 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:43:06.053 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:43:36.162 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:43:36.167 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:43:36.170 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:43:36.172 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:44:06.514 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:44:06.517 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:44:06.520 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:44:06.523 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:44:36.629 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:44:36.637 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:44:36.640 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:44:36.645 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:44:48.857 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2022-08-01T20:44:48.876 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2022-08-01T20:44:48.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T20:44:48.908 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/run_tasks.py", line 188, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/ceph_fuse.py", line 160, in task mount.umount_wait() File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/fuse_mount.py", line 326, in umount_wait if not (self.is_mounted() and self.fuse_daemon): File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted proc.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 148, in wait greenlet.get(block=True,timeout=60) File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2022-08-01T20:44:48.925 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2022-08-01T20:44:48.944 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2022-08-01T20:44:48.946 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T20:44:49.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.074+0000 7f0461cc4700 1 Processor -- start 2022-08-01T20:44:49.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.074+0000 7f0461cc4700 1 -- start start 2022-08-01T20:44:49.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f0461cc4700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f045c148530 0x7f045c0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:49.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f0461cc4700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f045c0b04e0 con 0x7f045c0a7250 2022-08-01T20:44:49.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f0461cc4700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f045c0b0780 con 0x7f045c0b1f40 2022-08-01T20:44:49.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f0461cc4700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f045c0a9250 con 0x7f045c148530 2022-08-01T20:44:49.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f045bfff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f045c0a7250 0x7f045c1448b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:38238/0 (socket says 172.21.15.17:38238) 2022-08-01T20:44:49.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f045bfff700 1 -- 172.21.15.17:0/1792342022 learned_addr learned my addr 172.21.15.17:0/1792342022 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T20:44:49.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f045affd700 1 --2- 172.21.15.17:0/1792342022 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f045c148530 0x7f045c0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:49.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f045a7fc700 1 -- 172.21.15.17:0/1792342022 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 623372903 0 0) 0x7f045c0b04e0 con 0x7f045c0a7250 2022-08-01T20:44:49.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f045a7fc700 1 -- 172.21.15.17:0/1792342022 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0448003260 con 0x7f045c0a7250 2022-08-01T20:44:49.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f045a7fc700 1 -- 172.21.15.17:0/1792342022 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2581455668 0 0) 0x7f0448003260 con 0x7f045c0a7250 2022-08-01T20:44:49.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f045a7fc700 1 -- 172.21.15.17:0/1792342022 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f045c148530 msgr2=0x7f045c0b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T20:44:49.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f045a7fc700 1 --2- 172.21.15.17:0/1792342022 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f045c148530 0x7f045c0b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:49.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f045a7fc700 1 -- 172.21.15.17:0/1792342022 >> v1:172.21.15.77:6789/0 conn(0x7f045c0b1f40 legacy=0x7f045c0b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:49.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f045a7fc700 1 -- 172.21.15.17:0/1792342022 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f045c0b0a20 con 0x7f045c0a7250 2022-08-01T20:44:49.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f045a7fc700 1 -- 172.21.15.17:0/1792342022 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2881673944 0 0) 0x7f04540019e0 con 0x7f045c0a7250 2022-08-01T20:44:49.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f045a7fc700 1 -- 172.21.15.17:0/1792342022 >> v1:172.21.15.17:6789/0 conn(0x7f045c0a7250 legacy=0x7f045c1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:49.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f045a7fc700 1 --2- 172.21.15.17:0/1792342022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0448003780 0x7f0448003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:49.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f045a7fc700 1 --2- 172.21.15.17:0/1792342022 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0448004500 0x7f0448004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:49.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f045a7fc700 1 --2- 172.21.15.17:0/1792342022 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f045c148530 0x7f0448004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:49.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f045a7fc700 1 -- 172.21.15.17:0/1792342022 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044800b750 con 0x7f0448004500 2022-08-01T20:44:49.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f045a7fc700 1 -- 172.21.15.17:0/1792342022 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044800b8c0 con 0x7f0448003780 2022-08-01T20:44:49.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f045a7fc700 1 -- 172.21.15.17:0/1792342022 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f044800ba80 con 0x7f045c148530 2022-08-01T20:44:49.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f045affd700 1 --2- 172.21.15.17:0/1792342022 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0448004500 0x7f0448004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:49.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f045bfff700 1 --2- 172.21.15.17:0/1792342022 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f045c148530 0x7f0448004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:49.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f045b7fe700 1 --2- 172.21.15.17:0/1792342022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0448003780 0x7f0448003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:49.102 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f045affd700 1 -- 172.21.15.17:0/1792342022 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f045c148530 msgr2=0x7f0448004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T20:44:49.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f045affd700 1 --2- 172.21.15.17:0/1792342022 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f045c148530 0x7f0448004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:49.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f045affd700 1 -- 172.21.15.17:0/1792342022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0448003780 msgr2=0x7f0448003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T20:44:49.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f045affd700 1 --2- 172.21.15.17:0/1792342022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0448003780 0x7f0448003ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:49.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f045affd700 1 -- 172.21.15.17:0/1792342022 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0448003260 con 0x7f0448004500 2022-08-01T20:44:49.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f045affd700 1 --2- 172.21.15.17:0/1792342022 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0448004500 0x7f0448004980 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 rx=0x7f044c00c000 tx=0x7f044c006df0).ready entity=mon.0 client_cookie=240ac2c66cc84780 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:49.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f045a7fc700 1 -- 172.21.15.17:0/1792342022 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f044c00e800 con 0x7f0448004500 2022-08-01T20:44:49.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f045a7fc700 1 -- 172.21.15.17:0/1792342022 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044c00e960 con 0x7f0448004500 2022-08-01T20:44:49.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f045a7fc700 1 -- 172.21.15.17:0/1792342022 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f044c00ecc0 con 0x7f0448004500 2022-08-01T20:44:49.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f0461cc4700 1 -- 172.21.15.17:0/1792342022 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0448004500 msgr2=0x7f0448004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:49.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f0461cc4700 1 --2- 172.21.15.17:0/1792342022 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0448004500 0x7f0448004980 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 rx=0x7f044c00c000 tx=0x7f044c006df0).stop 2022-08-01T20:44:49.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f045bfff700 1 -- 172.21.15.17:0/1792342022 reap_dead start 2022-08-01T20:44:49.107 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f0461cc4700 1 -- 172.21.15.17:0/1792342022 shutdown_connections 2022-08-01T20:44:49.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f0461cc4700 1 -- 172.21.15.17:0/1792342022 >> 172.21.15.17:0/1792342022 conn(0x7f045c05f840 msgr2=0x7f045c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:49.108 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f0461cc4700 1 -- 172.21.15.17:0/1792342022 shutdown_connections 2022-08-01T20:44:49.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.078+0000 7f0461cc4700 1 -- 172.21.15.17:0/1792342022 wait complete. 2022-08-01T20:44:49.109 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.082+0000 7f0461cc4700 1 Processor -- start 2022-08-01T20:44:49.110 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.082+0000 7f0461cc4700 1 -- start start 2022-08-01T20:44:49.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.082+0000 7f0461cc4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0448004500 0x7f045c0ae490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:49.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.082+0000 7f0461cc4700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f045c0b57d0 0x7f045c0af440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:49.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.082+0000 7f0461cc4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f045c156bd0 0x7f045c157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:49.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.082+0000 7f0461cc4700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045c0b5ca0 con 0x7f0448004500 2022-08-01T20:44:49.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.082+0000 7f0461cc4700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045c0b5f40 con 0x7f045c156bd0 2022-08-01T20:44:49.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.082+0000 7f0461cc4700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f045c0b61e0 con 0x7f045c0b57d0 2022-08-01T20:44:49.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.082+0000 7f045b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0448004500 0x7f045c0ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:49.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.082+0000 7f045b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0448004500 0x7f045c0ae490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46054/0 (socket says 172.21.15.17:46054) 2022-08-01T20:44:49.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.082+0000 7f045b7fe700 1 -- 172.21.15.17:0/3662208640 learned_addr learned my addr 172.21.15.17:0/3662208640 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T20:44:49.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.082+0000 7f045affd700 1 --2- 172.21.15.17:0/3662208640 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f045c0b57d0 0x7f045c0af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:49.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.082+0000 7f045bfff700 1 --2- 172.21.15.17:0/3662208640 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f045c156bd0 0x7f045c157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:49.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.082+0000 7f045b7fe700 1 -- 172.21.15.17:0/3662208640 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f045c0b57d0 msgr2=0x7f045c0af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:49.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.082+0000 7f045b7fe700 1 --2- 172.21.15.17:0/3662208640 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f045c0b57d0 0x7f045c0af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:49.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.082+0000 7f045b7fe700 1 -- 172.21.15.17:0/3662208640 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f045c156bd0 msgr2=0x7f045c157050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:49.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.082+0000 7f045b7fe700 1 --2- 172.21.15.17:0/3662208640 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f045c156bd0 0x7f045c157050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:49.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.082+0000 7f045b7fe700 1 -- 172.21.15.17:0/3662208640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f045c0b6480 con 0x7f0448004500 2022-08-01T20:44:49.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.082+0000 7f045bfff700 1 --2- 172.21.15.17:0/3662208640 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f045c156bd0 0x7f045c157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T20:44:49.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.082+0000 7f045b7fe700 1 --2- 172.21.15.17:0/3662208640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0448004500 0x7f045c0ae490 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 rx=0x7f045000b000 tx=0x7f0450009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:49.124 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.082+0000 7f0458ff9700 1 -- 172.21.15.17:0/3662208640 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0450014620 con 0x7f0448004500 2022-08-01T20:44:49.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.082+0000 7f0458ff9700 1 -- 172.21.15.17:0/3662208640 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0450014780 con 0x7f0448004500 2022-08-01T20:44:49.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.082+0000 7f0458ff9700 1 -- 172.21.15.17:0/3662208640 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0450014ae0 con 0x7f0448004500 2022-08-01T20:44:49.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.082+0000 7f0461cc4700 1 -- 172.21.15.17:0/3662208640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f045c0ae240 con 0x7f0448004500 2022-08-01T20:44:49.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.086+0000 7f0461cc4700 1 -- 172.21.15.17:0/3662208640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f045c157760 con 0x7f0448004500 2022-08-01T20:44:49.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.086+0000 7f0461cc4700 1 -- 172.21.15.17:0/3662208640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f045c0574e0 con 0x7f0448004500 2022-08-01T20:44:49.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.086+0000 7f0458ff9700 1 -- 172.21.15.17:0/3662208640 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0450014c40 con 0x7f0448004500 2022-08-01T20:44:49.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.086+0000 7f0458ff9700 1 --2- 172.21.15.17:0/3662208640 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f043c053aa0 0x7f043c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:49.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.086+0000 7f0458ff9700 1 -- 172.21.15.17:0/3662208640 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f045006bdb0 con 0x7f0448004500 2022-08-01T20:44:49.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.086+0000 7f045affd700 1 --2- 172.21.15.17:0/3662208640 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f043c053aa0 0x7f043c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:49.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.086+0000 7f045affd700 1 --2- 172.21.15.17:0/3662208640 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f043c053aa0 0x7f043c055f60 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7f044c01a000 tx=0x7f044c019000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:49.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.094+0000 7f0458ff9700 1 -- 172.21.15.17:0/3662208640 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0450042e50 con 0x7f0448004500 2022-08-01T20:44:49.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.218+0000 7f0461cc4700 1 -- 172.21.15.17:0/3662208640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f045c0447e0 con 0x7f0448004500 2022-08-01T20:44:49.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.218+0000 7f0458ff9700 1 -- 172.21.15.17:0/3662208640 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) v1 ==== 74+0+13549 (secure 0 0 0) 0x7f0450046470 con 0x7f0448004500 2022-08-01T20:44:49.224 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T20:44:49.225 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":161,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:44:18.294854+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.15.17:6835/75309005":"2022-08-02T19:44:14.714679+0000","172.21.15.17:6834/75309005":"2022-08-02T19:44:14.714679+0000"},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T20:44:49.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.222+0000 7f0461cc4700 1 -- 172.21.15.17:0/3662208640 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f043c053aa0 msgr2=0x7f043c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:49.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.222+0000 7f0461cc4700 1 --2- 172.21.15.17:0/3662208640 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f043c053aa0 0x7f043c055f60 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7f044c01a000 tx=0x7f044c019000).stop 2022-08-01T20:44:49.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.222+0000 7f0461cc4700 1 -- 172.21.15.17:0/3662208640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0448004500 msgr2=0x7f045c0ae490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:49.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.222+0000 7f0461cc4700 1 --2- 172.21.15.17:0/3662208640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0448004500 0x7f045c0ae490 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 rx=0x7f045000b000 tx=0x7f0450009040).stop 2022-08-01T20:44:49.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.222+0000 7f0461cc4700 1 -- 172.21.15.17:0/3662208640 shutdown_connections 2022-08-01T20:44:49.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.222+0000 7f0461cc4700 1 --2- 172.21.15.17:0/3662208640 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f043c053aa0 0x7f043c055f60 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:49.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.222+0000 7f0461cc4700 1 --2- 172.21.15.17:0/3662208640 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f045c156bd0 0x7f045c157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:49.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.222+0000 7f0461cc4700 1 --2- 172.21.15.17:0/3662208640 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f045c0b57d0 0x7f045c0af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:49.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.222+0000 7f0461cc4700 1 --2- 172.21.15.17:0/3662208640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0448004500 0x7f045c0ae490 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:49.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.222+0000 7f0461cc4700 1 -- 172.21.15.17:0/3662208640 >> 172.21.15.17:0/3662208640 conn(0x7f045c05f840 msgr2=0x7f045c1438d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:49.234 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.222+0000 7f0461cc4700 1 -- 172.21.15.17:0/3662208640 shutdown_connections 2022-08-01T20:44:49.234 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.222+0000 7f0461cc4700 1 -- 172.21.15.17:0/3662208640 wait complete. 2022-08-01T20:44:49.236 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2022-08-01T20:44:49.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.338+0000 7f567e44d700 1 Processor -- start 2022-08-01T20:44:49.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.338+0000 7f567e44d700 1 -- start start 2022-08-01T20:44:49.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.338+0000 7f567e44d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5678148530 0x7f56780b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:49.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.338+0000 7f567e44d700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f56780a7250 0x7f56781448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:49.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.338+0000 7f567e44d700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56780b04e0 con 0x7f56780b1f40 2022-08-01T20:44:49.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.338+0000 7f567e44d700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56780a9250 con 0x7f5678148530 2022-08-01T20:44:49.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.338+0000 7f567e44d700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56780b0780 con 0x7f56780a7250 2022-08-01T20:44:49.352 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.338+0000 7f56777fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5678148530 0x7f56780b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:49.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.338+0000 7f5677fff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f56780b1f40 0x7f56780b2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:38256/0 (socket says 172.21.15.17:38256) 2022-08-01T20:44:49.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.338+0000 7f56777fe700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5678148530 0x7f56780b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46064/0 (socket says 172.21.15.17:46064) 2022-08-01T20:44:49.354 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.338+0000 7f5677fff700 1 -- 172.21.15.17:0/2791102978 learned_addr learned my addr 172.21.15.17:0/2791102978 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T20:44:49.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.338+0000 7f567c9ea700 1 --2- 172.21.15.17:0/2791102978 >> v2:172.21.15.77:3300/0 conn(0x7f56780a7250 0x7f56781448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:49.355 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.338+0000 7f56777fe700 1 -- 172.21.15.17:0/2791102978 >> v2:172.21.15.77:3300/0 conn(0x7f56780a7250 msgr2=0x7f56781448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T20:44:49.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.338+0000 7f56777fe700 1 --2- 172.21.15.17:0/2791102978 >> v2:172.21.15.77:3300/0 conn(0x7f56780a7250 0x7f56781448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:49.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.338+0000 7f56777fe700 1 -- 172.21.15.17:0/2791102978 >> v1:172.21.15.17:6789/0 conn(0x7f56780b1f40 legacy=0x7f56780b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:49.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.338+0000 7f5676ffd700 1 -- 172.21.15.17:0/2791102978 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3453637435 0 0) 0x7f56780b04e0 con 0x7f56780b1f40 2022-08-01T20:44:49.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.338+0000 7f56777fe700 1 -- 172.21.15.17:0/2791102978 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56780b0a20 con 0x7f5678148530 2022-08-01T20:44:49.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.338+0000 7f56777fe700 1 --2- 172.21.15.17:0/2791102978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5678148530 0x7f56780b1160 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 rx=0x7f56781478c0 tx=0x7f5668006df0).ready entity=mon.0 client_cookie=a9a94464a8efedc5 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:49.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.338+0000 7f5676ffd700 1 -- 172.21.15.17:0/2791102978 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f566800d5c0 con 0x7f5678148530 2022-08-01T20:44:49.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.338+0000 7f5676ffd700 1 -- 172.21.15.17:0/2791102978 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5668000e90 con 0x7f5678148530 2022-08-01T20:44:49.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.338+0000 7f5676ffd700 1 -- 172.21.15.17:0/2791102978 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f566800d910 con 0x7f5678148530 2022-08-01T20:44:49.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.338+0000 7f567e44d700 1 -- 172.21.15.17:0/2791102978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5678148530 msgr2=0x7f56780b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:49.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.338+0000 7f567e44d700 1 --2- 172.21.15.17:0/2791102978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5678148530 0x7f56780b1160 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 rx=0x7f56781478c0 tx=0x7f5668006df0).stop 2022-08-01T20:44:49.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.338+0000 7f567e44d700 1 -- 172.21.15.17:0/2791102978 shutdown_connections 2022-08-01T20:44:49.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.338+0000 7f567e44d700 1 --2- 172.21.15.17:0/2791102978 >> v2:172.21.15.77:3300/0 conn(0x7f56780a7250 0x7f56781448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:49.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.338+0000 7f567e44d700 1 --2- 172.21.15.17:0/2791102978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5678148530 0x7f56780b1160 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:49.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.338+0000 7f567e44d700 1 -- 172.21.15.17:0/2791102978 >> 172.21.15.17:0/2791102978 conn(0x7f567805f840 msgr2=0x7f567805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:49.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.338+0000 7f567e44d700 1 -- 172.21.15.17:0/2791102978 shutdown_connections 2022-08-01T20:44:49.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.338+0000 7f567e44d700 1 -- 172.21.15.17:0/2791102978 wait complete. 2022-08-01T20:44:49.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.342+0000 7f567e44d700 1 Processor -- start 2022-08-01T20:44:49.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.342+0000 7f567e44d700 1 -- start start 2022-08-01T20:44:49.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.342+0000 7f567e44d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56780a7250 0x7f5678157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:49.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.342+0000 7f567e44d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56780b1f40 0x7f56780638b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:49.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.342+0000 7f567e44d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5678148530 0x7f5678062f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:49.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.342+0000 7f567e44d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5678156d60 con 0x7f56780b1f40 2022-08-01T20:44:49.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.342+0000 7f567e44d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5678063da0 con 0x7f56780a7250 2022-08-01T20:44:49.371 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.342+0000 7f567e44d700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5678156f50 con 0x7f5678148530 2022-08-01T20:44:49.372 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.342+0000 7f56777fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56780b1f40 0x7f56780638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:49.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.342+0000 7f56777fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56780b1f40 0x7f56780638b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46068/0 (socket says 172.21.15.17:46068) 2022-08-01T20:44:49.373 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.342+0000 7f56777fe700 1 -- 172.21.15.17:0/3612001972 learned_addr learned my addr 172.21.15.17:0/3612001972 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T20:44:49.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.342+0000 7f567c9ea700 1 --2- 172.21.15.17:0/3612001972 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5678148530 0x7f5678062f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:49.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.342+0000 7f5677fff700 1 --2- 172.21.15.17:0/3612001972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56780a7250 0x7f5678157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:49.375 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.342+0000 7f56777fe700 1 -- 172.21.15.17:0/3612001972 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5678148530 msgr2=0x7f5678062f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:49.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.342+0000 7f56777fe700 1 --2- 172.21.15.17:0/3612001972 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5678148530 0x7f5678062f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:49.377 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.342+0000 7f56777fe700 1 -- 172.21.15.17:0/3612001972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56780a7250 msgr2=0x7f5678157930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:49.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.342+0000 7f56777fe700 1 --2- 172.21.15.17:0/3612001972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56780a7250 0x7f5678157930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:49.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.342+0000 7f56777fe700 1 -- 172.21.15.17:0/3612001972 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f566800b180 con 0x7f56780b1f40 2022-08-01T20:44:49.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.342+0000 7f56777fe700 1 --2- 172.21.15.17:0/3612001972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56780b1f40 0x7f56780638b0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 rx=0x7f5678063ee0 tx=0x7f5668012000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:49.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.342+0000 7f56757fa700 1 -- 172.21.15.17:0/3612001972 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5668002f70 con 0x7f56780b1f40 2022-08-01T20:44:49.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.342+0000 7f56757fa700 1 -- 172.21.15.17:0/3612001972 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f566800de70 con 0x7f56780b1f40 2022-08-01T20:44:49.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.342+0000 7f567e44d700 1 -- 172.21.15.17:0/3612001972 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56781571f0 con 0x7f56780b1f40 2022-08-01T20:44:49.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.342+0000 7f56757fa700 1 -- 172.21.15.17:0/3612001972 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5668013da0 con 0x7f56780b1f40 2022-08-01T20:44:49.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.342+0000 7f567e44d700 1 -- 172.21.15.17:0/3612001972 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5678063420 con 0x7f56780b1f40 2022-08-01T20:44:49.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.342+0000 7f567e44d700 1 -- 172.21.15.17:0/3612001972 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56780574e0 con 0x7f56780b1f40 2022-08-01T20:44:49.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.346+0000 7f56757fa700 1 -- 172.21.15.17:0/3612001972 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f566801c070 con 0x7f56780b1f40 2022-08-01T20:44:49.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.346+0000 7f56757fa700 1 --2- 172.21.15.17:0/3612001972 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5658053950 0x7f5658055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:49.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.346+0000 7f56757fa700 1 -- 172.21.15.17:0/3612001972 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f566806c7f0 con 0x7f56780b1f40 2022-08-01T20:44:49.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.346+0000 7f5677fff700 1 --2- 172.21.15.17:0/3612001972 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5658053950 0x7f5658055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:49.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.346+0000 7f5677fff700 1 --2- 172.21.15.17:0/3612001972 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5658053950 0x7f5658055e10 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7f566c00d000 tx=0x7f566c007a00).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:49.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.354+0000 7f56757fa700 1 -- 172.21.15.17:0/3612001972 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5668043890 con 0x7f56780b1f40 2022-08-01T20:44:49.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.474+0000 7f567e44d700 1 -- 172.21.15.17:0/3612001972 --> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5678157690 con 0x7f5658053950 2022-08-01T20:44:49.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.482+0000 7f56757fa700 1 -- 172.21.15.17:0/3612001972 <== mgr.4101 v2:172.21.15.77:6800/23151 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+68106 (secure 0 0 0) 0x7f566c006de0 con 0x7f5658053950 2022-08-01T20:44:49.489 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T20:44:49.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.482+0000 7f567e44d700 1 -- 172.21.15.17:0/3612001972 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5658053950 msgr2=0x7f5658055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:49.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.482+0000 7f567e44d700 1 --2- 172.21.15.17:0/3612001972 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5658053950 0x7f5658055e10 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7f566c00d000 tx=0x7f566c007a00).stop 2022-08-01T20:44:49.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.482+0000 7f567e44d700 1 -- 172.21.15.17:0/3612001972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56780b1f40 msgr2=0x7f56780638b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:49.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.482+0000 7f567e44d700 1 --2- 172.21.15.17:0/3612001972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56780b1f40 0x7f56780638b0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 rx=0x7f5678063ee0 tx=0x7f5668012000).stop 2022-08-01T20:44:49.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.486+0000 7f567e44d700 1 -- 172.21.15.17:0/3612001972 shutdown_connections 2022-08-01T20:44:49.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.486+0000 7f567e44d700 1 --2- 172.21.15.17:0/3612001972 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f5658053950 0x7f5658055e10 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:49.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.486+0000 7f567e44d700 1 --2- 172.21.15.17:0/3612001972 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5678148530 0x7f5678062f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:49.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.486+0000 7f567e44d700 1 --2- 172.21.15.17:0/3612001972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56780b1f40 0x7f56780638b0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:49.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.486+0000 7f567e44d700 1 --2- 172.21.15.17:0/3612001972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f56780a7250 0x7f5678157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:49.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.486+0000 7f567e44d700 1 -- 172.21.15.17:0/3612001972 >> 172.21.15.17:0/3612001972 conn(0x7f567805f840 msgr2=0x7f56780b1080 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:49.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.486+0000 7f567e44d700 1 -- 172.21.15.17:0/3612001972 shutdown_connections 2022-08-01T20:44:49.499 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.486+0000 7f567e44d700 1 -- 172.21.15.17:0/3612001972 wait complete. 2022-08-01T20:44:49.500 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2022-08-01T20:44:49.502 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":2280,"stamp":"2022-08-01T20:44:48.151152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":8404664,"kb_used_data":15992,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746570056,"statfs":{"total":773094113280,"available":764487737344,"internally_reserved":8589934592,"allocated":16375808,"data_stored":7983484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003472"},"pg_stats":[{"pgid":"1.0","version":"15'2","reported_seq":139,"reported_epoch":159,"state":"active+clean","last_fresh":"2022-08-01T19:44:15.493140+0000","last_change":"2022-08-01T19:33:58.589491+0000","last_active":"2022-08-01T19:44:15.493140+0000","last_peered":"2022-08-01T19:44:15.493140+0000","last_clean":"2022-08-01T19:44:15.493140+0000","last_became_active":"2022-08-01T19:33:58.588671+0000","last_became_peered":"2022-08-01T19:33:58.588671+0000","last_unstale":"2022-08-01T19:44:15.493140+0000","last_undegraded":"2022-08-01T19:44:15.493140+0000","last_fullsized":"2022-08-01T19:44:15.493140+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:33:53.002518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:33:53.002518+0000","last_clean_scrub_stamp":"2022-08-01T19:33:53.002518+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":146,"reported_epoch":160,"state":"active+clean","last_fresh":"2022-08-01T19:44:17.297517+0000","last_change":"2022-08-01T19:34:03.246302+0000","last_active":"2022-08-01T19:44:17.297517+0000","last_peered":"2022-08-01T19:44:17.297517+0000","last_clean":"2022-08-01T19:44:17.297517+0000","last_became_active":"2022-08-01T19:34:03.245544+0000","last_became_peered":"2022-08-01T19:34:03.245544+0000","last_unstale":"2022-08-01T19:44:17.297517+0000","last_undegraded":"2022-08-01T19:44:17.297517+0000","last_fullsized":"2022-08-01T19:44:17.297517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_clean_scrub_stamp":"2022-08-01T19:34:02.212190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":134,"reported_epoch":159,"state":"active+clean","last_fresh":"2022-08-01T19:44:15.493078+0000","last_change":"2022-08-01T19:34:03.246244+0000","last_active":"2022-08-01T19:44:15.493078+0000","last_peered":"2022-08-01T19:44:15.493078+0000","last_clean":"2022-08-01T19:44:15.493078+0000","last_became_active":"2022-08-01T19:34:03.245832+0000","last_became_peered":"2022-08-01T19:34:03.245832+0000","last_unstale":"2022-08-01T19:44:15.493078+0000","last_undegraded":"2022-08-01T19:44:15.493078+0000","last_fullsized":"2022-08-01T19:44:15.493078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_clean_scrub_stamp":"2022-08-01T19:34:02.212190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":141,"reported_epoch":160,"state":"active+clean","last_fresh":"2022-08-01T19:44:17.865487+0000","last_change":"2022-08-01T19:34:03.244647+0000","last_active":"2022-08-01T19:44:17.865487+0000","last_peered":"2022-08-01T19:44:17.865487+0000","last_clean":"2022-08-01T19:44:17.865487+0000","last_became_active":"2022-08-01T19:34:03.244231+0000","last_became_peered":"2022-08-01T19:34:03.244231+0000","last_unstale":"2022-08-01T19:44:17.865487+0000","last_undegraded":"2022-08-01T19:44:17.865487+0000","last_fullsized":"2022-08-01T19:44:17.865487+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_clean_scrub_stamp":"2022-08-01T19:34:02.212190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":146,"reported_epoch":160,"state":"active+clean","last_fresh":"2022-08-01T19:44:17.297850+0000","last_change":"2022-08-01T19:34:03.246395+0000","last_active":"2022-08-01T19:44:17.297850+0000","last_peered":"2022-08-01T19:44:17.297850+0000","last_clean":"2022-08-01T19:44:17.297850+0000","last_became_active":"2022-08-01T19:34:03.245763+0000","last_became_peered":"2022-08-01T19:34:03.245763+0000","last_unstale":"2022-08-01T19:44:17.297850+0000","last_undegraded":"2022-08-01T19:44:17.297850+0000","last_fullsized":"2022-08-01T19:44:17.297850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_clean_scrub_stamp":"2022-08-01T19:34:02.212190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":141,"reported_epoch":159,"state":"active+clean","last_fresh":"2022-08-01T19:44:15.564429+0000","last_change":"2022-08-01T19:34:03.661980+0000","last_active":"2022-08-01T19:44:15.564429+0000","last_peered":"2022-08-01T19:44:15.564429+0000","last_clean":"2022-08-01T19:44:15.564429+0000","last_became_active":"2022-08-01T19:34:03.661358+0000","last_became_peered":"2022-08-01T19:34:03.661358+0000","last_unstale":"2022-08-01T19:44:15.564429+0000","last_undegraded":"2022-08-01T19:44:15.564429+0000","last_fullsized":"2022-08-01T19:44:15.564429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_clean_scrub_stamp":"2022-08-01T19:34:02.212190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":134,"reported_epoch":159,"state":"active+clean","last_fresh":"2022-08-01T19:44:15.493245+0000","last_change":"2022-08-01T19:34:03.665604+0000","last_active":"2022-08-01T19:44:15.493245+0000","last_peered":"2022-08-01T19:44:15.493245+0000","last_clean":"2022-08-01T19:44:15.493245+0000","last_became_active":"2022-08-01T19:34:03.664627+0000","last_became_peered":"2022-08-01T19:34:03.664627+0000","last_unstale":"2022-08-01T19:44:15.493245+0000","last_undegraded":"2022-08-01T19:44:15.493245+0000","last_fullsized":"2022-08-01T19:44:15.493245+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_clean_scrub_stamp":"2022-08-01T19:34:02.212190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":141,"reported_epoch":159,"state":"active+clean","last_fresh":"2022-08-01T19:44:15.564596+0000","last_change":"2022-08-01T19:34:03.243940+0000","last_active":"2022-08-01T19:44:15.564596+0000","last_peered":"2022-08-01T19:44:15.564596+0000","last_clean":"2022-08-01T19:44:15.564596+0000","last_became_active":"2022-08-01T19:34:03.243496+0000","last_became_peered":"2022-08-01T19:34:03.243496+0000","last_unstale":"2022-08-01T19:44:15.564596+0000","last_undegraded":"2022-08-01T19:44:15.564596+0000","last_fullsized":"2022-08-01T19:44:15.564596+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_clean_scrub_stamp":"2022-08-01T19:34:02.212190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":141,"reported_epoch":160,"state":"active+clean","last_fresh":"2022-08-01T19:44:18.212099+0000","last_change":"2022-08-01T19:34:03.245622+0000","last_active":"2022-08-01T19:44:18.212099+0000","last_peered":"2022-08-01T19:44:18.212099+0000","last_clean":"2022-08-01T19:44:18.212099+0000","last_became_active":"2022-08-01T19:34:03.245060+0000","last_became_peered":"2022-08-01T19:34:03.245060+0000","last_unstale":"2022-08-01T19:44:18.212099+0000","last_undegraded":"2022-08-01T19:44:18.212099+0000","last_fullsized":"2022-08-01T19:44:18.212099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_clean_scrub_stamp":"2022-08-01T19:34:02.212190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":15,"seq":64424510292,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050584,"kb_used_data":2000,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321256,"statfs":{"total":96636764160,"available":95560966144,"internally_reserved":1073741824,"allocated":2048000,"data_stored":998080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 20:44:19 2022","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.78300000000000003,"15min":0.79600000000000004},"min":{"1min":0.59799999999999998,"5min":0.55300000000000005,"15min":0.54000000000000004},"max":{"1min":1.0369999999999999,"5min":1.161,"15min":1.161},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.82199999999999995,"15min":0.82599999999999996},"min":{"1min":0.63800000000000001,"5min":0.59899999999999998,"15min":0.56699999999999995},"max":{"1min":1.1379999999999999,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.626}]},{"osd":1,"last update":"Mon Aug 1 20:44:19 2022","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.84599999999999997,"15min":0.84699999999999998},"min":{"1min":0.65800000000000003,"5min":0.61399999999999999,"15min":0.55400000000000005},"max":{"1min":1.1779999999999999,"5min":1.26,"15min":1.26},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.83399999999999996,"15min":0.81999999999999995},"min":{"1min":0.67100000000000004,"5min":0.60499999999999998,"15min":0.51800000000000002},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.95699999999999996}]},{"osd":2,"last update":"Mon Aug 1 20:44:19 2022","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.86599999999999999,"15min":0.86699999999999999},"min":{"1min":0.63200000000000001,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.1100000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.83299999999999996,"15min":0.83699999999999997},"min":{"1min":0.68100000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.111,"5min":1.111,"15min":1.216},"last":1.03}]},{"osd":3,"last update":"Mon Aug 1 20:44:19 2022","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.90700000000000003,"15min":0.88300000000000001},"min":{"1min":0.69499999999999995,"5min":0.64000000000000001,"15min":0.60699999999999998},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1699999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.93200000000000005,"15min":0.91600000000000004},"min":{"1min":0.66000000000000003,"5min":0.65700000000000003,"15min":0.63700000000000001},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.69999999999999996}]},{"osd":4,"last update":"Mon Aug 1 20:44:19 2022","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79800000000000004,"15min":0.77500000000000002},"min":{"1min":0.622,"5min":0.54400000000000004,"15min":0.52400000000000002},"max":{"1min":1.0589999999999999,"5min":1.161,"15min":1.161},"last":0.751},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.92500000000000004,"15min":0.89100000000000001},"min":{"1min":0.67000000000000004,"5min":0.58799999999999997,"15min":0.56399999999999995},"max":{"1min":1.145,"5min":1.22,"15min":1.264},"last":0.88600000000000001}]},{"osd":6,"last update":"Mon Aug 1 20:44:19 2022","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.93500000000000005,"15min":0.91300000000000003},"min":{"1min":0.58799999999999997,"5min":0.57199999999999995,"15min":0.56100000000000005},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.96199999999999997,"15min":0.93799999999999994},"min":{"1min":0.82199999999999995,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1619999999999999,"5min":1.2070000000000001,"15min":1.2829999999999999},"last":1.1379999999999999}]},{"osd":7,"last update":"Mon Aug 1 20:44:19 2022","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0209999999999999,"15min":0.96799999999999997},"min":{"1min":0.79300000000000004,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0449999999999999,"15min":0.98599999999999999},"min":{"1min":0.70699999999999996,"5min":0.69599999999999995,"15min":0.66100000000000003},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":1.0029999999999999}]}]},{"osd":4,"up_from":15,"seq":64424510291,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1050584,"kb_used_data":2000,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321256,"statfs":{"total":96636764160,"available":95560966144,"internally_reserved":1073741824,"allocated":2048000,"data_stored":998080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 20:43:55 2022","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.80700000000000005,"15min":0.80500000000000005},"min":{"1min":0.58599999999999997,"5min":0.55500000000000005,"15min":0.52400000000000002},"max":{"1min":1.119,"5min":1.165,"15min":1.165},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80000000000000004,"15min":0.81999999999999995},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.026,"5min":1.1719999999999999,"15min":1.194},"last":0.93500000000000005}]},{"osd":1,"last update":"Mon Aug 1 20:43:55 2022","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.80100000000000005,"15min":0.82299999999999995},"min":{"1min":0.63400000000000001,"5min":0.57199999999999995,"15min":0.56899999999999995},"max":{"1min":1.022,"5min":1.1619999999999999,"15min":1.1779999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85899999999999999,"15min":0.84099999999999997},"min":{"1min":0.60199999999999998,"5min":0.5,"15min":0.5},"max":{"1min":1.0580000000000001,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.97299999999999998}]},{"osd":2,"last update":"Mon Aug 1 20:43:55 2022","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.93600000000000005,"15min":0.93200000000000005},"min":{"1min":0.623,"5min":0.622,"15min":0.59199999999999997},"max":{"1min":1.0960000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.82399999999999995,"5min":0.66300000000000003,"15min":0.60899999999999999},"max":{"1min":1.2230000000000001,"5min":1.2569999999999999,"15min":1.4590000000000001},"last":1.153}]},{"osd":3,"last update":"Mon Aug 1 20:43:55 2022","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83999999999999997,"15min":0.85099999999999998},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.621},"max":{"1min":1.135,"5min":1.244,"15min":1.2829999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.92500000000000004,"15min":0.92600000000000005},"min":{"1min":0.79900000000000004,"5min":0.66300000000000003,"15min":0.65900000000000003},"max":{"1min":1.018,"5min":1.228,"15min":1.28},"last":1.012}]},{"osd":5,"last update":"Mon Aug 1 20:43:55 2022","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.82599999999999996,"15min":0.83799999999999997},"min":{"1min":0.60499999999999998,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.171,"5min":1.171,"15min":1.1870000000000001},"last":1.2070000000000001},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.86599999999999999,"15min":0.873},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.55900000000000005},"max":{"1min":1.212,"5min":1.298,"15min":1.3460000000000001},"last":1.0600000000000001}]},{"osd":6,"last update":"Mon Aug 1 20:43:55 2022","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.93300000000000005,"15min":0.90700000000000003},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2,"5min":1.27,"15min":1.292},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.008,"5min":0.98999999999999999,"15min":0.96099999999999997},"min":{"1min":0.77500000000000002,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.3160000000000001,"5min":1.331,"15min":1.5549999999999999},"last":1.0780000000000001}]},{"osd":7,"last update":"Mon Aug 1 20:43:55 2022","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.96199999999999997,"15min":0.95599999999999996},"min":{"1min":0.67900000000000005,"5min":0.65000000000000002,"15min":0.55800000000000005},"max":{"1min":1.296,"5min":1.296,"15min":1.599},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0449999999999999,"15min":1.0369999999999999},"min":{"1min":0.86499999999999999,"5min":0.68200000000000005,"15min":0.64500000000000002},"max":{"1min":1.371,"5min":1.409,"15min":1.4870000000000001},"last":1.125}]}]},{"osd":7,"up_from":14,"seq":60129542997,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050584,"kb_used_data":2000,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321256,"statfs":{"total":96636764160,"available":95560966144,"internally_reserved":1073741824,"allocated":2048000,"data_stored":998080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 20:44:35 2022","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.88200000000000001,"15min":0.90500000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.1539999999999999,"5min":1.163,"15min":1.244},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.90000000000000002,"15min":0.89600000000000002},"min":{"1min":0.622,"5min":0.622,"15min":0.52100000000000002},"max":{"1min":1.0669999999999999,"5min":1.1499999999999999,"15min":1.1910000000000001},"last":0.94899999999999995}]},{"osd":1,"last update":"Mon Aug 1 20:44:35 2022","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.82799999999999996,"15min":0.84399999999999997},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.56100000000000005},"max":{"1min":1.0860000000000001,"5min":1.123,"15min":1.159},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82299999999999995,"15min":0.81599999999999995},"min":{"1min":0.65400000000000003,"5min":0.57599999999999996,"15min":0.52900000000000003},"max":{"1min":1.071,"5min":1.163,"15min":1.163},"last":1.008}]},{"osd":2,"last update":"Mon Aug 1 20:44:35 2022","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.84599999999999997,"15min":0.85099999999999998},"min":{"1min":0.63100000000000001,"5min":0.62,"15min":0.57899999999999996},"max":{"1min":0.98999999999999999,"5min":1.1739999999999999,"15min":1.3129999999999999},"last":1.1819999999999999},{"interface":"front","average":{"1min":0.875,"5min":0.874,"15min":0.872},"min":{"1min":0.64900000000000002,"5min":0.64200000000000002,"15min":0.60299999999999998},"max":{"1min":1.0880000000000001,"5min":1.161,"15min":1.2370000000000001},"last":0.77500000000000002}]},{"osd":3,"last update":"Mon Aug 1 20:44:35 2022","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.86399999999999999,"15min":0.88700000000000001},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.2150000000000001},"last":1.1279999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.93000000000000005,"15min":0.91700000000000004},"min":{"1min":0.68799999999999994,"5min":0.65900000000000003,"15min":0.622},"max":{"1min":1.1080000000000001,"5min":1.268,"15min":1.3280000000000001},"last":1.077}]},{"osd":4,"last update":"Mon Aug 1 20:44:35 2022","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83699999999999997,"15min":0.84199999999999997},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.56399999999999995},"max":{"1min":1.145,"5min":1.218,"15min":1.2529999999999999},"last":1.034},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.93100000000000005,"15min":0.90600000000000003},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.145,"5min":1.216,"15min":1.4019999999999999},"last":0.71199999999999997}]},{"osd":5,"last update":"Mon Aug 1 20:44:35 2022","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.84599999999999997,"15min":0.83699999999999997},"min":{"1min":0.626,"5min":0.56599999999999995,"15min":0.55200000000000005},"max":{"1min":1.042,"5min":1.157,"15min":1.1950000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94599999999999995,"15min":0.94099999999999995},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.59799999999999998},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.385},"last":0.78900000000000003}]},{"osd":6,"last update":"Mon Aug 1 20:44:35 2022","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.93100000000000005,"15min":0.92400000000000004},"min":{"1min":0.72599999999999998,"5min":0.65500000000000003,"15min":0.57499999999999996},"max":{"1min":1.1519999999999999,"5min":1.202,"15min":1.2689999999999999},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.012},"min":{"1min":0.84699999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.2310000000000001,"5min":1.306,"15min":1.4510000000000001},"last":0.92100000000000004}]}]},{"osd":6,"up_from":14,"seq":60129542996,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050584,"kb_used_data":2000,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321256,"statfs":{"total":96636764160,"available":95560966144,"internally_reserved":1073741824,"allocated":2048000,"data_stored":998080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 20:44:33 2022","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.92700000000000005,"15min":0.93899999999999995},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.51600000000000001},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.88600000000000001,"15min":0.89000000000000001},"min":{"1min":0.69699999999999995,"5min":0.628,"15min":0.60199999999999998},"max":{"1min":1.0589999999999999,"5min":1.0840000000000001,"15min":1.149},"last":0.79300000000000004}]},{"osd":1,"last update":"Mon Aug 1 20:44:33 2022","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.879,"15min":0.84799999999999998},"min":{"1min":0.61699999999999999,"5min":0.57399999999999995,"15min":0.56699999999999995},"max":{"1min":1.1659999999999999,"5min":1.2230000000000001,"15min":1.2490000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.81200000000000006,"15min":0.81699999999999995},"min":{"1min":0.64100000000000001,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.054,"5min":1.054,"15min":1.161},"last":0.89800000000000002}]},{"osd":2,"last update":"Mon Aug 1 20:44:33 2022","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84899999999999998,"15min":0.83199999999999996},"min":{"1min":0.69199999999999995,"5min":0.626,"15min":0.60699999999999998},"max":{"1min":1.032,"5min":1.1890000000000001,"15min":1.4239999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.89300000000000002,"15min":0.91000000000000003},"min":{"1min":0.65000000000000002,"5min":0.61899999999999999,"15min":0.60599999999999998},"max":{"1min":1.169,"5min":1.169,"15min":1.2709999999999999},"last":0.94199999999999995}]},{"osd":3,"last update":"Mon Aug 1 20:44:33 2022","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.68999999999999995,"5min":0.65400000000000003,"15min":0.64100000000000001},"max":{"1min":1.085,"5min":1.103,"15min":1.157},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.93400000000000005,"15min":0.93600000000000005},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.2050000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":1.0229999999999999}]},{"osd":4,"last update":"Mon Aug 1 20:44:33 2022","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.877,"15min":0.86799999999999999},"min":{"1min":0.67100000000000004,"5min":0.54100000000000004,"15min":0.54000000000000004},"max":{"1min":1.0900000000000001,"5min":1.1659999999999999,"15min":1.234},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.83499999999999996,"15min":0.82899999999999996},"min":{"1min":0.57799999999999996,"5min":0.55800000000000005,"15min":0.55000000000000004},"max":{"1min":1.0409999999999999,"5min":1.0980000000000001,"15min":1.2010000000000001},"last":0.59199999999999997}]},{"osd":5,"last update":"Mon Aug 1 20:44:33 2022","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.95899999999999996,"15min":0.93100000000000005},"min":{"1min":0.77500000000000002,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.127,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.90200000000000002,"15min":0.878},"min":{"1min":0.65000000000000002,"5min":0.60199999999999998,"15min":0.59799999999999998},"max":{"1min":1.121,"5min":1.1699999999999999,"15min":1.2390000000000001},"last":0.87}]},{"osd":7,"last update":"Mon Aug 1 20:44:33 2022","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.89700000000000002,"15min":0.89600000000000002},"min":{"1min":0.66900000000000004,"5min":0.63600000000000001,"15min":0.59799999999999998},"max":{"1min":1.0660000000000001,"5min":1.264,"15min":1.264},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.97099999999999997,"15min":0.95899999999999996},"min":{"1min":0.76500000000000001,"5min":0.67400000000000004,"15min":0.61899999999999999},"max":{"1min":1.147,"5min":1.355,"15min":1.355},"last":0.68999999999999995}]}]},{"osd":3,"up_from":14,"seq":60129542995,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050580,"kb_used_data":1996,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321260,"statfs":{"total":96636764160,"available":95560970240,"internally_reserved":1073741824,"allocated":2043904,"data_stored":997502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 20:43:55 2022","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61599999999999999,"15min":0.621},"min":{"1min":0.44700000000000001,"5min":0.42899999999999999,"15min":0.41299999999999998},"max":{"1min":0.82199999999999995,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68200000000000005,"15min":0.67700000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.48599999999999999},"max":{"1min":0.91800000000000004,"5min":1.1020000000000001,"15min":1.194},"last":0.70299999999999996}]},{"osd":1,"last update":"Mon Aug 1 20:43:55 2022","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65900000000000003,"15min":0.66700000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.44900000000000001},"max":{"1min":0.94799999999999995,"5min":1.2170000000000001,"15min":1.25},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78100000000000003,"15min":0.78000000000000003},"min":{"1min":0.56299999999999994,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.046,"5min":1.1399999999999999,"15min":1.266},"last":0.745}]},{"osd":2,"last update":"Mon Aug 1 20:43:55 2022","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.78000000000000003,"15min":0.78400000000000003},"min":{"1min":0.63400000000000001,"5min":0.58699999999999997,"15min":0.45900000000000002},"max":{"1min":1.0049999999999999,"5min":1.171,"15min":1.171},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.84899999999999998,"15min":0.84299999999999997},"min":{"1min":0.57699999999999996,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1000000000000001,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.85799999999999998}]},{"osd":4,"last update":"Mon Aug 1 20:44:38 2022","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.89800000000000002,"15min":0.89000000000000001},"min":{"1min":0.65000000000000002,"5min":0.61899999999999999,"15min":0.59699999999999998},"max":{"1min":1.145,"5min":1.3120000000000001,"15min":1.3400000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.93200000000000005,"15min":0.94299999999999995},"min":{"1min":0.70799999999999996,"5min":0.64800000000000002,"15min":0.64500000000000002},"max":{"1min":1.246,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":1.0289999999999999}]},{"osd":5,"last update":"Mon Aug 1 20:44:38 2022","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99099999999999999,"15min":1.0129999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.68899999999999995},"max":{"1min":1.2989999999999999,"5min":1.46,"15min":1.508},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.97299999999999998,"15min":0.96799999999999997},"min":{"1min":0.77300000000000002,"5min":0.71399999999999997,"15min":0.67900000000000005},"max":{"1min":1.2729999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.90300000000000002}]},{"osd":6,"last update":"Mon Aug 1 20:43:55 2022","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.022,"15min":1.0349999999999999},"min":{"1min":0.69599999999999995,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.4470000000000001,"5min":1.548,"15min":1.548},"last":1.133},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.087,"15min":1.0980000000000001},"min":{"1min":0.88900000000000001,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.3520000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.0700000000000001}]},{"osd":7,"last update":"Mon Aug 1 20:43:55 2022","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.0669999999999999,"15min":1.0620000000000001},"min":{"1min":0.84499999999999997,"5min":0.81999999999999995,"15min":0.71799999999999997},"max":{"1min":1.375,"5min":1.468,"15min":1.5109999999999999},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.111,"15min":1.111},"min":{"1min":0.90600000000000003,"5min":0.82699999999999996,"15min":0.78400000000000003},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":1.1850000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542997,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050580,"kb_used_data":1996,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321260,"statfs":{"total":96636764160,"available":95560970240,"internally_reserved":1073741824,"allocated":2043904,"data_stored":997502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 20:44:22 2022","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58799999999999997,"15min":0.60499999999999998},"min":{"1min":0.47799999999999998,"5min":0.46500000000000002,"15min":0.436},"max":{"1min":0.77200000000000002,"5min":0.88400000000000001,"15min":1.1919999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72399999999999998,"15min":0.71699999999999997},"min":{"1min":0.503,"5min":0.5,"15min":0.47799999999999998},"max":{"1min":1.0509999999999999,"5min":1.129,"15min":1.179},"last":0.64600000000000002}]},{"osd":1,"last update":"Mon Aug 1 20:44:22 2022","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.81399999999999995,"15min":0.80800000000000005},"min":{"1min":0.66400000000000003,"5min":0.55100000000000005,"15min":0.52100000000000002},"max":{"1min":1.0169999999999999,"5min":1.147,"15min":1.147},"last":1.014},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.69799999999999995,"15min":0.69999999999999996},"min":{"1min":0.50800000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.89500000000000002,"5min":1.0509999999999999,"15min":1.0620000000000001},"last":0.84199999999999997}]},{"osd":3,"last update":"Mon Aug 1 20:44:22 2022","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.68999999999999995,"15min":0.70899999999999996},"min":{"1min":0.53800000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.91000000000000003,"5min":0.96799999999999997,"15min":1.2569999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.77900000000000003,"15min":0.79800000000000004},"min":{"1min":0.61099999999999999,"5min":0.56000000000000005,"15min":0.55800000000000005},"max":{"1min":0.97299999999999998,"5min":1.0629999999999999,"15min":1.24},"last":0.78700000000000003}]},{"osd":4,"last update":"Mon Aug 1 20:44:22 2022","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.90900000000000003,"15min":0.90400000000000003},"min":{"1min":0.68100000000000005,"5min":0.67400000000000004,"15min":0.60199999999999998},"max":{"1min":1.2669999999999999,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.93100000000000005,"15min":0.93200000000000005},"min":{"1min":0.72699999999999998,"5min":0.70699999999999996,"15min":0.67300000000000004},"max":{"1min":1.365,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.96099999999999997}]},{"osd":5,"last update":"Mon Aug 1 20:44:22 2022","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.97899999999999998,"15min":0.996},"min":{"1min":0.77000000000000002,"5min":0.71899999999999997,"15min":0.71399999999999997},"max":{"1min":1.1240000000000001,"5min":1.411,"15min":1.411},"last":1.1519999999999999},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.95899999999999996,"15min":0.95299999999999996},"min":{"1min":0.72999999999999998,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.234,"5min":1.341,"15min":1.421},"last":1.1020000000000001}]},{"osd":6,"last update":"Mon Aug 1 20:44:22 2022","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":0.98999999999999999,"15min":0.98399999999999999},"min":{"1min":0.80600000000000005,"5min":0.72699999999999998,"15min":0.68799999999999994},"max":{"1min":1.319,"5min":1.319,"15min":1.4019999999999999},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.014,"5min":1.018,"15min":1.0249999999999999},"min":{"1min":0.80000000000000004,"5min":0.75900000000000001,"15min":0.71499999999999997},"max":{"1min":1.3120000000000001,"5min":1.383,"15min":1.383},"last":1.05}]},{"osd":7,"last update":"Mon Aug 1 20:44:22 2022","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.077,"15min":1.081},"min":{"1min":0.82399999999999995,"5min":0.76100000000000001,"15min":0.72199999999999998},"max":{"1min":1.2010000000000001,"5min":1.524,"15min":1.5820000000000001},"last":1.234},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.054,"15min":1.0589999999999999},"min":{"1min":0.82699999999999996,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.494},"last":1.1819999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542996,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050584,"kb_used_data":2000,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321256,"statfs":{"total":96636764160,"available":95560966144,"internally_reserved":1073741824,"allocated":2048000,"data_stored":998080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 20:44:27 2022","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63100000000000001,"15min":0.624},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.42599999999999999},"max":{"1min":0.85599999999999998,"5min":0.90200000000000002,"15min":1.0649999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.752,"15min":0.747},"min":{"1min":0.62,"5min":0.61199999999999999,"15min":0.55900000000000005},"max":{"1min":0.879,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.66400000000000003}]},{"osd":2,"last update":"Mon Aug 1 20:44:27 2022","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.65900000000000003,"15min":0.67400000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.46100000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.1459999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.82999999999999996,"15min":0.83799999999999997},"min":{"1min":0.66000000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.056,"5min":1.24,"15min":1.24},"last":0.82199999999999995}]},{"osd":3,"last update":"Mon Aug 1 20:44:27 2022","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.77100000000000002,"15min":0.77800000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0189999999999999,"5min":1.1779999999999999,"15min":1.228},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.72099999999999997,"15min":0.70699999999999996},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.92300000000000004,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.749}]},{"osd":4,"last update":"Mon Aug 1 20:44:27 2022","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88900000000000001,"15min":0.873},"min":{"1min":0.69299999999999995,"5min":0.65300000000000002,"15min":0.629},"max":{"1min":1.21,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.97599999999999998,"15min":0.97699999999999998},"min":{"1min":0.74399999999999999,"5min":0.72299999999999998,"15min":0.67300000000000004},"max":{"1min":1.1180000000000001,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.89700000000000002}]},{"osd":5,"last update":"Mon Aug 1 20:44:27 2022","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.94499999999999995,"15min":0.93000000000000005},"min":{"1min":0.73199999999999998,"5min":0.70299999999999996,"15min":0.65200000000000002},"max":{"1min":1.3120000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.96899999999999997,"15min":0.95599999999999996},"min":{"1min":0.79700000000000004,"5min":0.70499999999999996,"15min":0.66100000000000003},"max":{"1min":1.3799999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":1.0820000000000001}]},{"osd":6,"last update":"Mon Aug 1 20:44:27 2022","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.97499999999999998,"15min":0.98099999999999998},"min":{"1min":0.72899999999999998,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.2210000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.087,"5min":1.0660000000000001,"15min":1.054},"min":{"1min":0.84799999999999998,"5min":0.82299999999999995,"15min":0.66800000000000004},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.7010000000000001},"last":1.1419999999999999}]},{"osd":7,"last update":"Mon Aug 1 20:44:27 2022","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0389999999999999,"15min":1.024},"min":{"1min":0.88700000000000001,"5min":0.73599999999999999,"15min":0.72099999999999997},"max":{"1min":1.351,"5min":1.587,"15min":1.587},"last":1.014},{"interface":"front","average":{"1min":1.121,"5min":1.0920000000000001,"15min":1.0720000000000001},"min":{"1min":0.90300000000000002,"5min":0.77400000000000002,"15min":0.71899999999999997},"max":{"1min":1.3759999999999999,"5min":1.482,"15min":1.6459999999999999},"last":1.1759999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542995,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050584,"kb_used_data":2000,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321256,"statfs":{"total":96636764160,"available":95560966144,"internally_reserved":1073741824,"allocated":2048000,"data_stored":998080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Aug 1 20:44:12 2022","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63700000000000001,"15min":0.64000000000000001},"min":{"1min":0.47299999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88300000000000001,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73699999999999999,"15min":0.753},"min":{"1min":0.63400000000000001,"5min":0.59899999999999998,"15min":0.51800000000000002},"max":{"1min":0.873,"5min":0.98999999999999999,"15min":1.0640000000000001},"last":0.73499999999999999}]},{"osd":2,"last update":"Mon Aug 1 20:44:12 2022","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.65400000000000003},"min":{"1min":0.51000000000000001,"5min":0.504,"15min":0.504},"max":{"1min":0.91800000000000004,"5min":1.1020000000000001,"15min":1.2390000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.67700000000000005,"15min":0.67200000000000004},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.47199999999999998},"max":{"1min":0.91300000000000003,"5min":1.1659999999999999,"15min":1.2929999999999999},"last":0.69999999999999996}]},{"osd":3,"last update":"Mon Aug 1 20:44:12 2022","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.72799999999999998,"15min":0.73599999999999999},"min":{"1min":0.51300000000000001,"5min":0.50600000000000001,"15min":0.498},"max":{"1min":0.89000000000000001,"5min":1.1180000000000001,"15min":1.1299999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.873,"15min":0.874},"min":{"1min":0.72299999999999998,"5min":0.69199999999999995,"15min":0.64100000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.228},"last":0.79900000000000004}]},{"osd":4,"last update":"Mon Aug 1 20:44:12 2022","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.94999999999999996,"15min":0.93799999999999994},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.67900000000000005},"max":{"1min":1.1579999999999999,"5min":1.3740000000000001,"15min":1.413},"last":1.135},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.93799999999999994,"15min":0.94299999999999995},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.66200000000000003},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.85999999999999999}]},{"osd":5,"last update":"Mon Aug 1 20:44:12 2022","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.90900000000000003,"15min":0.88700000000000001},"min":{"1min":0.746,"5min":0.70299999999999996,"15min":0.65300000000000002},"max":{"1min":1.0980000000000001,"5min":1.2150000000000001,"15min":1.397},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0229999999999999},"min":{"1min":0.85499999999999998,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.222,"5min":1.452,"15min":1.508},"last":1.2769999999999999}]},{"osd":6,"last update":"Mon Aug 1 20:44:12 2022","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.0600000000000001,"15min":1.0580000000000001},"min":{"1min":0.81100000000000005,"5min":0.77700000000000002,"15min":0.68400000000000005},"max":{"1min":1.29,"5min":1.534,"15min":1.546},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0269999999999999,"15min":1.032},"min":{"1min":0.84999999999999998,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.2230000000000001,"5min":1.3999999999999999,"15min":1.5189999999999999},"last":1.014}]},{"osd":7,"last update":"Mon Aug 1 20:44:12 2022","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.0409999999999999,"15min":1.0269999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.67900000000000005},"max":{"1min":1.347,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.121,"5min":1.129,"15min":1.129},"min":{"1min":0.77400000000000002,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.3720000000000001,"5min":1.4330000000000001,"15min":1.6559999999999999},"last":1.3360000000000001}]}]}]}} 2022-08-01T20:44:49.507 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-08-01T20:44:49.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.610+0000 7f8a116e1700 1 Processor -- start 2022-08-01T20:44:49.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.610+0000 7f8a116e1700 1 -- start start 2022-08-01T20:44:49.617 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.614+0000 7f8a116e1700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8a0c148530 0x7f8a0c0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:49.618 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.614+0000 7f8a116e1700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8a0c0a7250 0x7f8a0c1448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:49.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.614+0000 7f8a116e1700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a0c0b04e0 con 0x7f8a0c0b1f40 2022-08-01T20:44:49.619 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.614+0000 7f8a116e1700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a0c0a9250 con 0x7f8a0c148530 2022-08-01T20:44:49.620 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.614+0000 7f8a116e1700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c0b0780 con 0x7f8a0c0a7250 2022-08-01T20:44:49.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.614+0000 7f8a0a7fc700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8a0c148530 0x7f8a0c0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:49.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.614+0000 7f8a0affd700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f8a0c0b1f40 0x7f8a0c0b2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:38272/0 (socket says 172.21.15.17:38272) 2022-08-01T20:44:49.622 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.614+0000 7f8a0affd700 1 -- 172.21.15.17:0/1186180218 learned_addr learned my addr 172.21.15.17:0/1186180218 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T20:44:49.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.614+0000 7f8a0b7fe700 1 --2- 172.21.15.17:0/1186180218 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8a0c0a7250 0x7f8a0c1448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:49.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.614+0000 7f8a0a7fc700 1 -- 172.21.15.17:0/1186180218 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8a0c0a7250 msgr2=0x7f8a0c1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T20:44:49.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.614+0000 7f8a0a7fc700 1 --2- 172.21.15.17:0/1186180218 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8a0c0a7250 0x7f8a0c1448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:49.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.614+0000 7f8a0a7fc700 1 -- 172.21.15.17:0/1186180218 >> v1:172.21.15.17:6789/0 conn(0x7f8a0c0b1f40 legacy=0x7f8a0c0b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:49.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.614+0000 7f8a09ffb700 1 -- 172.21.15.17:0/1186180218 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2999165964 0 0) 0x7f8a0c0b04e0 con 0x7f8a0c0b1f40 2022-08-01T20:44:49.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.614+0000 7f8a0a7fc700 1 -- 172.21.15.17:0/1186180218 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0c0b0a20 con 0x7f8a0c148530 2022-08-01T20:44:49.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.614+0000 7f8a0a7fc700 1 --2- 172.21.15.17:0/1186180218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a0c148530 0x7f8a0c0b1160 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 rx=0x7f8a0c1478c0 tx=0x7f89fc006df0).ready entity=mon.0 client_cookie=ec97bfa98af4e851 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:49.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.614+0000 7f8a09ffb700 1 -- 172.21.15.17:0/1186180218 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89fc00d570 con 0x7f8a0c148530 2022-08-01T20:44:49.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.614+0000 7f8a09ffb700 1 -- 172.21.15.17:0/1186180218 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89fc000e90 con 0x7f8a0c148530 2022-08-01T20:44:49.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.614+0000 7f8a09ffb700 1 -- 172.21.15.17:0/1186180218 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89fc00d8c0 con 0x7f8a0c148530 2022-08-01T20:44:49.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.614+0000 7f8a116e1700 1 -- 172.21.15.17:0/1186180218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a0c148530 msgr2=0x7f8a0c0b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:49.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.614+0000 7f8a116e1700 1 --2- 172.21.15.17:0/1186180218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a0c148530 0x7f8a0c0b1160 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 rx=0x7f8a0c1478c0 tx=0x7f89fc006df0).stop 2022-08-01T20:44:49.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.614+0000 7f8a116e1700 1 -- 172.21.15.17:0/1186180218 shutdown_connections 2022-08-01T20:44:49.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.614+0000 7f8a116e1700 1 --2- 172.21.15.17:0/1186180218 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8a0c0a7250 0x7f8a0c1448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:49.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.614+0000 7f8a116e1700 1 --2- 172.21.15.17:0/1186180218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a0c148530 0x7f8a0c0b1160 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:49.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.614+0000 7f8a116e1700 1 -- 172.21.15.17:0/1186180218 >> 172.21.15.17:0/1186180218 conn(0x7f8a0c05f840 msgr2=0x7f8a0c05fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:49.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.614+0000 7f8a116e1700 1 -- 172.21.15.17:0/1186180218 shutdown_connections 2022-08-01T20:44:49.636 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.614+0000 7f8a116e1700 1 -- 172.21.15.17:0/1186180218 wait complete. 2022-08-01T20:44:49.636 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.618+0000 7f8a116e1700 1 Processor -- start 2022-08-01T20:44:49.636 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.618+0000 7f8a116e1700 1 -- start start 2022-08-01T20:44:49.637 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.618+0000 7f8a116e1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a0c0a7250 0x7f8a0c14d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:49.637 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.618+0000 7f8a116e1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a0c0b1f40 0x7f8a0c0a95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:49.637 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.618+0000 7f8a116e1700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8a0c148530 0x7f8a0c156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:49.638 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.618+0000 7f8a116e1700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c14bb30 con 0x7f8a0c0a7250 2022-08-01T20:44:49.638 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.618+0000 7f8a116e1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c0a9e70 con 0x7f8a0c0b1f40 2022-08-01T20:44:49.638 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.618+0000 7f8a116e1700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c0ab120 con 0x7f8a0c148530 2022-08-01T20:44:49.639 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.618+0000 7f8a0affd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a0c0a7250 0x7f8a0c14d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:49.639 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.618+0000 7f8a0affd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a0c0a7250 0x7f8a0c14d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46082/0 (socket says 172.21.15.17:46082) 2022-08-01T20:44:49.640 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.618+0000 7f8a0affd700 1 -- 172.21.15.17:0/1007566951 learned_addr learned my addr 172.21.15.17:0/1007566951 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T20:44:49.640 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.618+0000 7f8a0b7fe700 1 --2- 172.21.15.17:0/1007566951 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8a0c148530 0x7f8a0c156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:49.640 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.618+0000 7f8a0a7fc700 1 --2- 172.21.15.17:0/1007566951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a0c0b1f40 0x7f8a0c0a95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:49.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.618+0000 7f8a0affd700 1 -- 172.21.15.17:0/1007566951 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8a0c148530 msgr2=0x7f8a0c156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:49.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.618+0000 7f8a0affd700 1 --2- 172.21.15.17:0/1007566951 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8a0c148530 0x7f8a0c156920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:49.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.618+0000 7f8a0affd700 1 -- 172.21.15.17:0/1007566951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a0c0b1f40 msgr2=0x7f8a0c0a95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:49.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.618+0000 7f8a0affd700 1 --2- 172.21.15.17:0/1007566951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a0c0b1f40 0x7f8a0c0a95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:49.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.618+0000 7f8a0affd700 1 -- 172.21.15.17:0/1007566951 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89fc00b180 con 0x7f8a0c0a7250 2022-08-01T20:44:49.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.618+0000 7f8a0b7fe700 1 --2- 172.21.15.17:0/1007566951 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8a0c148530 0x7f8a0c156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T20:44:49.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.618+0000 7f8a0affd700 1 --2- 172.21.15.17:0/1007566951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a0c0a7250 0x7f8a0c14d900 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 rx=0x7f8a0c14ea70 tx=0x7f8a00007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:49.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.618+0000 7f89fbfff700 1 -- 172.21.15.17:0/1007566951 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a000070d0 con 0x7f8a0c0a7250 2022-08-01T20:44:49.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.618+0000 7f89fbfff700 1 -- 172.21.15.17:0/1007566951 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a00007230 con 0x7f8a0c0a7250 2022-08-01T20:44:49.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.618+0000 7f8a116e1700 1 -- 172.21.15.17:0/1007566951 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a0c0ab3c0 con 0x7f8a0c0a7250 2022-08-01T20:44:49.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.618+0000 7f89fbfff700 1 -- 172.21.15.17:0/1007566951 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a00005610 con 0x7f8a0c0a7250 2022-08-01T20:44:49.645 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.618+0000 7f8a116e1700 1 -- 172.21.15.17:0/1007566951 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a0c0a9b40 con 0x7f8a0c0a7250 2022-08-01T20:44:49.645 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.622+0000 7f8a116e1700 1 -- 172.21.15.17:0/1007566951 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a0c0ab860 con 0x7f8a0c0a7250 2022-08-01T20:44:49.645 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.622+0000 7f89fbfff700 1 -- 172.21.15.17:0/1007566951 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8a0001a060 con 0x7f8a0c0a7250 2022-08-01T20:44:49.646 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.622+0000 7f89fbfff700 1 --2- 172.21.15.17:0/1007566951 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f89e8053a70 0x7f89e8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:49.646 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.622+0000 7f89fbfff700 1 -- 172.21.15.17:0/1007566951 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f8a0006b4b0 con 0x7f8a0c0a7250 2022-08-01T20:44:49.646 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.622+0000 7f8a0a7fc700 1 --2- 172.21.15.17:0/1007566951 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f89e8053a70 0x7f89e8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:49.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.622+0000 7f8a0a7fc700 1 --2- 172.21.15.17:0/1007566951 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f89e8053a70 0x7f89e8055f30 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0x7f89fc01a000 tx=0x7f89fc014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:49.648 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.630+0000 7f89fbfff700 1 -- 172.21.15.17:0/1007566951 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8a0c0ab860 con 0x7f8a0c0a7250 2022-08-01T20:44:49.756 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.754+0000 7f8a116e1700 1 -- 172.21.15.17:0/1007566951 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a0c0ab860 con 0x7f8a0c0a7250 2022-08-01T20:44:49.758 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.754+0000 7f89fbfff700 1 -- 172.21.15.17:0/1007566951 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) v1 ==== 74+0+13549 (secure 0 0 0) 0x7f8a0c0ab860 con 0x7f8a0c0a7250 2022-08-01T20:44:49.759 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T20:44:49.760 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":161,"fsid":"be659963-3cf3-4374-b3c4-cbe717b6bac2","created":"2022-08-01T19:33:50.105743+0000","modified":"2022-08-01T19:44:18.294854+0000","last_up_change":"2022-08-01T19:33:58.040328+0000","last_in_change":"2022-08-01T19:33:58.040328+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-08-01T19:33:53.001841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-08-01T19:34:01.553098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a962f14f-cf16-4fa8-84ff-c85325db9132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":23008},{"type":"v1","addr":"172.21.15.17:6801","nonce":23008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":23008},{"type":"v1","addr":"172.21.15.17:6803","nonce":23008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":23008},{"type":"v1","addr":"172.21.15.17:6807","nonce":23008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":23008},{"type":"v1","addr":"172.21.15.17:6805","nonce":23008}]},"public_addr":"172.21.15.17:6801/23008","cluster_addr":"172.21.15.17:6803/23008","heartbeat_back_addr":"172.21.15.17:6807/23008","heartbeat_front_addr":"172.21.15.17:6805/23008","state":["exists","up"]},{"osd":1,"uuid":"805f7051-a527-4e98-8844-b6e406a63248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":23007},{"type":"v1","addr":"172.21.15.17:6809","nonce":23007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":23007},{"type":"v1","addr":"172.21.15.17:6811","nonce":23007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":23007},{"type":"v1","addr":"172.21.15.17:6815","nonce":23007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":23007},{"type":"v1","addr":"172.21.15.17:6813","nonce":23007}]},"public_addr":"172.21.15.17:6809/23007","cluster_addr":"172.21.15.17:6811/23007","heartbeat_back_addr":"172.21.15.17:6815/23007","heartbeat_front_addr":"172.21.15.17:6813/23007","state":["exists","up"]},{"osd":2,"uuid":"89f7f3df-7c10-48e1-8b69-9e4243923a61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":23010},{"type":"v1","addr":"172.21.15.17:6819","nonce":23010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":23010},{"type":"v1","addr":"172.21.15.17:6824","nonce":23010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":23010},{"type":"v1","addr":"172.21.15.17:6831","nonce":23010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":23010},{"type":"v1","addr":"172.21.15.17:6829","nonce":23010}]},"public_addr":"172.21.15.17:6819/23010","cluster_addr":"172.21.15.17:6824/23010","heartbeat_back_addr":"172.21.15.17:6831/23010","heartbeat_front_addr":"172.21.15.17:6829/23010","state":["exists","up"]},{"osd":3,"uuid":"96123468-aabc-4a64-909e-a192c369090d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":23009},{"type":"v1","addr":"172.21.15.17:6818","nonce":23009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":23009},{"type":"v1","addr":"172.21.15.17:6821","nonce":23009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6827","nonce":23009},{"type":"v1","addr":"172.21.15.17:6828","nonce":23009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6823","nonce":23009},{"type":"v1","addr":"172.21.15.17:6825","nonce":23009}]},"public_addr":"172.21.15.17:6818/23009","cluster_addr":"172.21.15.17:6821/23009","heartbeat_back_addr":"172.21.15.17:6828/23009","heartbeat_front_addr":"172.21.15.17:6825/23009","state":["exists","up"]},{"osd":4,"uuid":"a6e26665-e04b-46ec-ab66-e912acbf044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":23584},{"type":"v1","addr":"172.21.15.77:6811","nonce":23584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":23584},{"type":"v1","addr":"172.21.15.77:6813","nonce":23584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":23584},{"type":"v1","addr":"172.21.15.77:6817","nonce":23584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":23584},{"type":"v1","addr":"172.21.15.77:6815","nonce":23584}]},"public_addr":"172.21.15.77:6811/23584","cluster_addr":"172.21.15.77:6813/23584","heartbeat_back_addr":"172.21.15.77:6817/23584","heartbeat_front_addr":"172.21.15.77:6815/23584","state":["exists","up"]},{"osd":5,"uuid":"502c0034-18ee-44ac-87ff-19b9dddb66a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":23585},{"type":"v1","addr":"172.21.15.77:6803","nonce":23585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":23585},{"type":"v1","addr":"172.21.15.77:6805","nonce":23585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":23585},{"type":"v1","addr":"172.21.15.77:6809","nonce":23585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":23585},{"type":"v1","addr":"172.21.15.77:6807","nonce":23585}]},"public_addr":"172.21.15.77:6803/23585","cluster_addr":"172.21.15.77:6805/23585","heartbeat_back_addr":"172.21.15.77:6809/23585","heartbeat_front_addr":"172.21.15.77:6807/23585","state":["exists","up"]},{"osd":6,"uuid":"ac4e755a-7fe7-46cb-8407-852d5af9ffc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":23587},{"type":"v1","addr":"172.21.15.77:6821","nonce":23587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6823","nonce":23587},{"type":"v1","addr":"172.21.15.77:6825","nonce":23587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6831","nonce":23587},{"type":"v1","addr":"172.21.15.77:6833","nonce":23587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6827","nonce":23587},{"type":"v1","addr":"172.21.15.77:6829","nonce":23587}]},"public_addr":"172.21.15.77:6821/23587","cluster_addr":"172.21.15.77:6825/23587","heartbeat_back_addr":"172.21.15.77:6833/23587","heartbeat_front_addr":"172.21.15.77:6829/23587","state":["exists","up"]},{"osd":7,"uuid":"874761e2-6410-40fc-b60c-e58abe5ce489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":23586},{"type":"v1","addr":"172.21.15.77:6819","nonce":23586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":23586},{"type":"v1","addr":"172.21.15.77:6824","nonce":23586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":23586},{"type":"v1","addr":"172.21.15.77:6832","nonce":23586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":23586},{"type":"v1","addr":"172.21.15.77:6828","nonce":23586}]},"public_addr":"172.21.15.77:6819/23586","cluster_addr":"172.21.15.77:6824/23586","heartbeat_back_addr":"172.21.15.77:6832/23586","heartbeat_front_addr":"172.21.15.77:6828/23586","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.588817+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.782753+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.837924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.839948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.112801+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:56.106704+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.893170+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-08-01T19:33:55.886947+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.15.17:6835/75309005":"2022-08-02T19:44:14.714679+0000","172.21.15.17:6834/75309005":"2022-08-02T19:44:14.714679+0000"},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-08-01T20:44:49.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.758+0000 7f8a116e1700 1 -- 172.21.15.17:0/1007566951 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f89e8053a70 msgr2=0x7f89e8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:49.763 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.758+0000 7f8a116e1700 1 --2- 172.21.15.17:0/1007566951 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f89e8053a70 0x7f89e8055f30 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0x7f89fc01a000 tx=0x7f89fc014000).stop 2022-08-01T20:44:49.764 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.758+0000 7f8a116e1700 1 -- 172.21.15.17:0/1007566951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a0c0a7250 msgr2=0x7f8a0c14d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:49.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.758+0000 7f8a116e1700 1 --2- 172.21.15.17:0/1007566951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a0c0a7250 0x7f8a0c14d900 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 rx=0x7f8a0c14ea70 tx=0x7f8a00007ac0).stop 2022-08-01T20:44:49.765 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.758+0000 7f8a116e1700 1 -- 172.21.15.17:0/1007566951 shutdown_connections 2022-08-01T20:44:49.766 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.758+0000 7f8a116e1700 1 --2- 172.21.15.17:0/1007566951 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f89e8053a70 0x7f89e8055f30 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:49.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.758+0000 7f8a116e1700 1 --2- 172.21.15.17:0/1007566951 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f8a0c148530 0x7f8a0c156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:49.767 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.758+0000 7f8a116e1700 1 --2- 172.21.15.17:0/1007566951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a0c0b1f40 0x7f8a0c0a95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:49.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.758+0000 7f8a116e1700 1 --2- 172.21.15.17:0/1007566951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a0c0a7250 0x7f8a0c14d900 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:49.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.758+0000 7f8a116e1700 1 -- 172.21.15.17:0/1007566951 >> 172.21.15.17:0/1007566951 conn(0x7f8a0c05f840 msgr2=0x7f8a0c147900 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:49.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.758+0000 7f8a116e1700 1 -- 172.21.15.17:0/1007566951 shutdown_connections 2022-08-01T20:44:49.770 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:49.758+0000 7f8a116e1700 1 -- 172.21.15.17:0/1007566951 wait complete. 2022-08-01T20:44:50.774 INFO:tasks.ceph:Scrubbing osd.0 2022-08-01T20:44:50.775 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2022-08-01T20:44:50.894 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.890+0000 7fa8ef15c700 1 Processor -- start 2022-08-01T20:44:50.895 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.890+0000 7fa8ef15c700 1 -- start start 2022-08-01T20:44:50.901 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.894+0000 7fa8ef15c700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fa8e80a9da0 0x7fa8e8060950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:50.902 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.894+0000 7fa8ef15c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa8e81402d0 0x7fa8e8140720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:50.903 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.894+0000 7fa8ef15c700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8e8067030 con 0x7fa8e80b31d0 2022-08-01T20:44:50.904 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.894+0000 7fa8ef15c700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8e8064a70 con 0x7fa8e81402d0 2022-08-01T20:44:50.904 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.894+0000 7fa8ef15c700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8e8061f60 con 0x7fa8e80a9da0 2022-08-01T20:44:50.905 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.894+0000 7fa8ee95b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa8e81402d0 0x7fa8e8140720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:50.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.894+0000 7fa8ee15a700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa8e80b31d0 0x7fa8e8066020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:38284/0 (socket says 172.21.15.17:38284) 2022-08-01T20:44:50.906 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.894+0000 7fa8ee95b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa8e81402d0 0x7fa8e8140720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46094/0 (socket says 172.21.15.17:46094) 2022-08-01T20:44:50.907 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.894+0000 7fa8ee15a700 1 -- 172.21.15.17:0/483749942 learned_addr learned my addr 172.21.15.17:0/483749942 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T20:44:50.907 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.894+0000 7fa8ed959700 1 --2- 172.21.15.17:0/483749942 >> v2:172.21.15.77:3300/0 conn(0x7fa8e80a9da0 0x7fa8e8060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:50.908 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.894+0000 7fa8ee95b700 1 -- 172.21.15.17:0/483749942 >> v2:172.21.15.77:3300/0 conn(0x7fa8e80a9da0 msgr2=0x7fa8e8060950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T20:44:50.909 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.894+0000 7fa8ee95b700 1 --2- 172.21.15.17:0/483749942 >> v2:172.21.15.77:3300/0 conn(0x7fa8e80a9da0 0x7fa8e8060950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:50.909 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.894+0000 7fa8ee95b700 1 -- 172.21.15.17:0/483749942 >> v1:172.21.15.17:6789/0 conn(0x7fa8e80b31d0 legacy=0x7fa8e8066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:50.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.894+0000 7fa8ee95b700 1 -- 172.21.15.17:0/483749942 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e80b35b0 con 0x7fa8e81402d0 2022-08-01T20:44:50.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.894+0000 7fa8ee95b700 1 --2- 172.21.15.17:0/483749942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8e81402d0 0x7fa8e8140720 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 rx=0x7fa8e00032f0 tx=0x7fa8e00091d0).ready entity=mon.0 client_cookie=d4157da6cd86314f server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:50.912 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.894+0000 7fa8ed158700 1 -- 172.21.15.17:0/483749942 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8e000b680 con 0x7fa8e81402d0 2022-08-01T20:44:50.913 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.894+0000 7fa8ed158700 1 -- 172.21.15.17:0/483749942 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e000b7e0 con 0x7fa8e81402d0 2022-08-01T20:44:50.915 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.894+0000 7fa8ed158700 1 -- 172.21.15.17:0/483749942 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8e000b950 con 0x7fa8e81402d0 2022-08-01T20:44:50.917 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.894+0000 7fa8ef15c700 1 -- 172.21.15.17:0/483749942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8e81402d0 msgr2=0x7fa8e8140720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:50.918 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.894+0000 7fa8ef15c700 1 --2- 172.21.15.17:0/483749942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8e81402d0 0x7fa8e8140720 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 rx=0x7fa8e00032f0 tx=0x7fa8e00091d0).stop 2022-08-01T20:44:50.918 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.894+0000 7fa8ef15c700 1 -- 172.21.15.17:0/483749942 shutdown_connections 2022-08-01T20:44:50.919 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.894+0000 7fa8ef15c700 1 --2- 172.21.15.17:0/483749942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8e81402d0 0x7fa8e8140720 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:50.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.894+0000 7fa8ef15c700 1 --2- 172.21.15.17:0/483749942 >> v2:172.21.15.77:3300/0 conn(0x7fa8e80a9da0 0x7fa8e8060950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:50.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.894+0000 7fa8ef15c700 1 -- 172.21.15.17:0/483749942 >> 172.21.15.17:0/483749942 conn(0x7fa8e8063870 msgr2=0x7fa8e8063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:50.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.894+0000 7fa8ef15c700 1 -- 172.21.15.17:0/483749942 shutdown_connections 2022-08-01T20:44:50.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.894+0000 7fa8ef15c700 1 -- 172.21.15.17:0/483749942 wait complete. 2022-08-01T20:44:50.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.898+0000 7fa8ef15c700 1 Processor -- start 2022-08-01T20:44:50.924 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.898+0000 7fa8ef15c700 1 -- start start 2022-08-01T20:44:50.936 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.902+0000 7fa8ef15c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8e80a9da0 0x7fa8e8143590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:50.936 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.902+0000 7fa8ef15c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8e80b31d0 0x7fa8e8153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:50.937 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.902+0000 7fa8ef15c700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa8e81402d0 0x7fa8e813e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:50.937 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.902+0000 7fa8ef15c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e80a9a30 con 0x7fa8e80b31d0 2022-08-01T20:44:50.938 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.902+0000 7fa8ef15c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e8153830 con 0x7fa8e80a9da0 2022-08-01T20:44:50.938 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.902+0000 7fa8ef15c700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e80acef0 con 0x7fa8e81402d0 2022-08-01T20:44:50.939 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.902+0000 7fa8ed959700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8e80b31d0 0x7fa8e8153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:50.939 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.902+0000 7fa8ed959700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8e80b31d0 0x7fa8e8153f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46098/0 (socket says 172.21.15.17:46098) 2022-08-01T20:44:50.940 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.902+0000 7fa8ed959700 1 -- 172.21.15.17:0/2739175514 learned_addr learned my addr 172.21.15.17:0/2739175514 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T20:44:50.940 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.902+0000 7fa8ee15a700 1 --2- 172.21.15.17:0/2739175514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8e80a9da0 0x7fa8e8143590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:50.940 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.902+0000 7fa8ed959700 1 -- 172.21.15.17:0/2739175514 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa8e81402d0 msgr2=0x7fa8e813e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:50.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.902+0000 7fa8ee95b700 1 --2- 172.21.15.17:0/2739175514 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa8e81402d0 0x7fa8e813e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:50.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.902+0000 7fa8ed959700 1 --2- 172.21.15.17:0/2739175514 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa8e81402d0 0x7fa8e813e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:50.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.902+0000 7fa8ed959700 1 -- 172.21.15.17:0/2739175514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8e80a9da0 msgr2=0x7fa8e8143590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:50.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.902+0000 7fa8ed959700 1 --2- 172.21.15.17:0/2739175514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8e80a9da0 0x7fa8e8143590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:50.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.902+0000 7fa8ed959700 1 -- 172.21.15.17:0/2739175514 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e0009470 con 0x7fa8e80b31d0 2022-08-01T20:44:50.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.902+0000 7fa8ee95b700 1 --2- 172.21.15.17:0/2739175514 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fa8e81402d0 0x7fa8e813e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T20:44:50.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.902+0000 7fa8ed959700 1 --2- 172.21.15.17:0/2739175514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8e80b31d0 0x7fa8e8153f00 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 rx=0x7fa8d800c000 tx=0x7fa8d8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:50.944 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.902+0000 7fa8d77fe700 1 -- 172.21.15.17:0/2739175514 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8d8010780 con 0x7fa8e80b31d0 2022-08-01T20:44:50.944 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.902+0000 7fa8d77fe700 1 -- 172.21.15.17:0/2739175514 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d80108e0 con 0x7fa8e80b31d0 2022-08-01T20:44:50.945 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.902+0000 7fa8ef15c700 1 -- 172.21.15.17:0/2739175514 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8e8153a60 con 0x7fa8e80b31d0 2022-08-01T20:44:50.945 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.902+0000 7fa8d77fe700 1 -- 172.21.15.17:0/2739175514 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8d8010c40 con 0x7fa8e80b31d0 2022-08-01T20:44:50.946 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.902+0000 7fa8ef15c700 1 -- 172.21.15.17:0/2739175514 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e809fd20 con 0x7fa8e80b31d0 2022-08-01T20:44:50.946 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.902+0000 7fa8ef15c700 1 -- 172.21.15.17:0/2739175514 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa8e8153bf0 con 0x7fa8e80b31d0 2022-08-01T20:44:50.947 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.902+0000 7fa8d77fe700 1 -- 172.21.15.17:0/2739175514 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa8d801c460 con 0x7fa8e80b31d0 2022-08-01T20:44:50.947 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.906+0000 7fa8d77fe700 1 --2- 172.21.15.17:0/2739175514 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa8c8053aa0 0x7fa8c8055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:50.947 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.906+0000 7fa8d77fe700 1 -- 172.21.15.17:0/2739175514 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fa8d806c280 con 0x7fa8e80b31d0 2022-08-01T20:44:50.948 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.906+0000 7fa8d77fe700 1 --2- 172.21.15.17:0/2739175514 >> [v2:172.21.15.17:6800/23008,v1:172.21.15.17:6801/23008] conn(0x7fa8c805a730 0x7fa8c805cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:50.948 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.906+0000 7fa8d77fe700 1 -- 172.21.15.17:0/2739175514 --> [v2:172.21.15.17:6800/23008,v1:172.21.15.17:6801/23008] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa8c805d1b0 con 0x7fa8c805a730 2022-08-01T20:44:50.948 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.906+0000 7fa8ee15a700 1 --2- 172.21.15.17:0/2739175514 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa8c8053aa0 0x7fa8c8055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:50.949 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.906+0000 7fa8d77fe700 1 -- 172.21.15.17:0/2739175514 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=161) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8d806e8f0 con 0x7fa8e80b31d0 2022-08-01T20:44:50.949 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.906+0000 7fa8ee95b700 1 --2- 172.21.15.17:0/2739175514 >> [v2:172.21.15.17:6800/23008,v1:172.21.15.17:6801/23008] conn(0x7fa8c805a730 0x7fa8c805cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:50.950 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.906+0000 7fa8ee15a700 1 --2- 172.21.15.17:0/2739175514 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa8c8053aa0 0x7fa8c8055f60 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7fa8e400d000 tx=0x7fa8e4009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:50.951 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.906+0000 7fa8ee95b700 1 --2- 172.21.15.17:0/2739175514 >> [v2:172.21.15.17:6800/23008,v1:172.21.15.17:6801/23008] conn(0x7fa8c805a730 0x7fa8c805cbd0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:50.951 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.906+0000 7fa8d77fe700 1 -- 172.21.15.17:0/2739175514 <== osd.0 v2:172.21.15.17:6800/23008 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7fa8c805d1b0 con 0x7fa8c805a730 2022-08-01T20:44:50.952 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.926+0000 7fa8ef15c700 1 -- 172.21.15.17:0/2739175514 --> [v2:172.21.15.17:6800/23008,v1:172.21.15.17:6801/23008] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa8e80adad0 con 0x7fa8c805a730 2022-08-01T20:44:50.952 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.926+0000 7fa8d77fe700 1 -- 172.21.15.17:0/2739175514 <== osd.0 v2:172.21.15.17:6800/23008 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fa8e80adad0 con 0x7fa8c805a730 2022-08-01T20:44:50.954 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T20:44:50.954 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-08-01T20:44:50.955 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T20:44:50.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.926+0000 7fa8ef15c700 1 -- 172.21.15.17:0/2739175514 >> [v2:172.21.15.17:6800/23008,v1:172.21.15.17:6801/23008] conn(0x7fa8c805a730 msgr2=0x7fa8c805cbd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:50.960 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.926+0000 7fa8ef15c700 1 --2- 172.21.15.17:0/2739175514 >> [v2:172.21.15.17:6800/23008,v1:172.21.15.17:6801/23008] conn(0x7fa8c805a730 0x7fa8c805cbd0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:50.961 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.926+0000 7fa8ef15c700 1 -- 172.21.15.17:0/2739175514 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa8c8053aa0 msgr2=0x7fa8c8055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:50.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.926+0000 7fa8ef15c700 1 --2- 172.21.15.17:0/2739175514 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fa8c8053aa0 0x7fa8c8055f60 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7fa8e400d000 tx=0x7fa8e4009040).stop 2022-08-01T20:44:50.962 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.926+0000 7fa8ef15c700 1 -- 172.21.15.17:0/2739175514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8e80b31d0 msgr2=0x7fa8e8153f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:50.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.926+0000 7fa8ef15c700 1 --2- 172.21.15.17:0/2739175514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8e80b31d0 0x7fa8e8153f00 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 rx=0x7fa8d800c000 tx=0x7fa8d8006df0).stop 2022-08-01T20:44:50.963 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.926+0000 7fa8ee95b700 1 -- 172.21.15.17:0/2739175514 reap_dead start 2022-08-01T20:44:50.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.926+0000 7fa8ef15c700 1 -- 172.21.15.17:0/2739175514 shutdown_connections 2022-08-01T20:44:50.964 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.926+0000 7fa8ef15c700 1 -- 172.21.15.17:0/2739175514 >> 172.21.15.17:0/2739175514 conn(0x7fa8e8063870 msgr2=0x7fa8e80b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:50.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.926+0000 7fa8ef15c700 1 -- 172.21.15.17:0/2739175514 shutdown_connections 2022-08-01T20:44:50.965 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:50.926+0000 7fa8ef15c700 1 -- 172.21.15.17:0/2739175514 wait complete. 2022-08-01T20:44:50.966 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2022-08-01T20:44:51.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.078+0000 7f7628ff6700 1 Processor -- start 2022-08-01T20:44:51.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.078+0000 7f7628ff6700 1 -- start start 2022-08-01T20:44:51.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.082+0000 7f7628ff6700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f76240b0630 0x7f76240adec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:51.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.082+0000 7f7628ff6700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f76240aa2b0 0x7f76240ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:51.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.082+0000 7f7628ff6700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76240b2c80 con 0x7f76240b0a10 2022-08-01T20:44:51.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.082+0000 7f7628ff6700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76240b2900 con 0x7f76240b0630 2022-08-01T20:44:51.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.082+0000 7f7628ff6700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76240b2eb0 con 0x7f76240aa2b0 2022-08-01T20:44:51.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.082+0000 7f762259c700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f76240b0630 0x7f76240adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:51.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.082+0000 7f7622d9d700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f76240b0a10 0x7f76241438a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:38304/0 (socket says 172.21.15.17:38304) 2022-08-01T20:44:51.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.082+0000 7f7622d9d700 1 -- 172.21.15.17:0/876169303 learned_addr learned my addr 172.21.15.17:0/876169303 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T20:44:51.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.082+0000 7f761bfff700 1 --2- 172.21.15.17:0/876169303 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f76240aa2b0 0x7f76240ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:51.112 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.082+0000 7f762259c700 1 -- 172.21.15.17:0/876169303 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f76240aa2b0 msgr2=0x7f76240ae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T20:44:51.113 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.082+0000 7f762259c700 1 --2- 172.21.15.17:0/876169303 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f76240aa2b0 0x7f76240ae3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:51.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.082+0000 7f762259c700 1 -- 172.21.15.17:0/876169303 >> v1:172.21.15.17:6789/0 conn(0x7f76240b0a10 legacy=0x7f76241438a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:51.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.082+0000 7f7621d9b700 1 -- 172.21.15.17:0/876169303 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3554613048 0 0) 0x7f76240b2c80 con 0x7f76240b0a10 2022-08-01T20:44:51.115 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.082+0000 7f762259c700 1 -- 172.21.15.17:0/876169303 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76240a9d00 con 0x7f76240b0630 2022-08-01T20:44:51.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.082+0000 7f761bfff700 1 --2- 172.21.15.17:0/876169303 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f76240aa2b0 0x7f76240ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T20:44:51.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.082+0000 7f762259c700 1 --2- 172.21.15.17:0/876169303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76240b0630 0x7f76240adec0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 rx=0x7f7614002f70 tx=0x7f7614006df0).ready entity=mon.0 client_cookie=e828baddf3695e72 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:51.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.082+0000 7f7621d9b700 1 -- 172.21.15.17:0/876169303 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f761400d570 con 0x7f76240b0630 2022-08-01T20:44:51.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.082+0000 7f7621d9b700 1 -- 172.21.15.17:0/876169303 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7614000e90 con 0x7f76240b0630 2022-08-01T20:44:51.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.082+0000 7f7621d9b700 1 -- 172.21.15.17:0/876169303 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f761400d8c0 con 0x7f76240b0630 2022-08-01T20:44:51.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.082+0000 7f7628ff6700 1 -- 172.21.15.17:0/876169303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76240b0630 msgr2=0x7f76240adec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:51.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.082+0000 7f7628ff6700 1 --2- 172.21.15.17:0/876169303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76240b0630 0x7f76240adec0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 rx=0x7f7614002f70 tx=0x7f7614006df0).stop 2022-08-01T20:44:51.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.086+0000 7f7628ff6700 1 -- 172.21.15.17:0/876169303 shutdown_connections 2022-08-01T20:44:51.121 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.086+0000 7f7628ff6700 1 --2- 172.21.15.17:0/876169303 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f76240aa2b0 0x7f76240ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:51.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.086+0000 7f7628ff6700 1 --2- 172.21.15.17:0/876169303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76240b0630 0x7f76240adec0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:51.122 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.086+0000 7f7628ff6700 1 -- 172.21.15.17:0/876169303 >> 172.21.15.17:0/876169303 conn(0x7f7624060950 msgr2=0x7f7624060d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:51.123 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.086+0000 7f7628ff6700 1 -- 172.21.15.17:0/876169303 shutdown_connections 2022-08-01T20:44:51.125 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.086+0000 7f7628ff6700 1 -- 172.21.15.17:0/876169303 wait complete. 2022-08-01T20:44:51.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.086+0000 7f7628ff6700 1 Processor -- start 2022-08-01T20:44:51.126 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.086+0000 7f7628ff6700 1 -- start start 2022-08-01T20:44:51.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.086+0000 7f7628ff6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76240aa2b0 0x7f76241405c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:51.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.086+0000 7f7628ff6700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f76240b0630 0x7f7624064750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:51.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.086+0000 7f7628ff6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76240b0a10 0x7f7624064c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:51.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.086+0000 7f7628ff6700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76240ac1a0 con 0x7f76240b0a10 2022-08-01T20:44:51.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.086+0000 7f7628ff6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76240aaad0 con 0x7f76240aa2b0 2022-08-01T20:44:51.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.086+0000 7f7628ff6700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76240abf00 con 0x7f76240b0630 2022-08-01T20:44:51.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.086+0000 7f7622d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76240b0a10 0x7f7624064c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:51.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.086+0000 7f7622d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76240b0a10 0x7f7624064c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46116/0 (socket says 172.21.15.17:46116) 2022-08-01T20:44:51.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.086+0000 7f7622d9d700 1 -- 172.21.15.17:0/1578631059 learned_addr learned my addr 172.21.15.17:0/1578631059 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T20:44:51.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.086+0000 7f761bfff700 1 --2- 172.21.15.17:0/1578631059 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f76240b0630 0x7f7624064750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:51.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.086+0000 7f762259c700 1 --2- 172.21.15.17:0/1578631059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76240aa2b0 0x7f76241405c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:51.133 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.086+0000 7f7622d9d700 1 -- 172.21.15.17:0/1578631059 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f76240b0630 msgr2=0x7f7624064750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:51.134 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.086+0000 7f7622d9d700 1 --2- 172.21.15.17:0/1578631059 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f76240b0630 0x7f7624064750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:51.134 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.086+0000 7f7622d9d700 1 -- 172.21.15.17:0/1578631059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76240aa2b0 msgr2=0x7f76241405c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:51.135 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.086+0000 7f7622d9d700 1 --2- 172.21.15.17:0/1578631059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76240aa2b0 0x7f76241405c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:51.135 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.086+0000 7f7622d9d700 1 -- 172.21.15.17:0/1578631059 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f760c00b180 con 0x7f76240b0a10 2022-08-01T20:44:51.136 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.086+0000 7f761bfff700 1 --2- 172.21.15.17:0/1578631059 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f76240b0630 0x7f7624064750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T20:44:51.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.086+0000 7f7622d9d700 1 --2- 172.21.15.17:0/1578631059 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76240b0a10 0x7f7624064c40 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 rx=0x7f761c000f40 tx=0x7f761c001360).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:51.137 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.086+0000 7f761b7fe700 1 -- 172.21.15.17:0/1578631059 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f761c005ba0 con 0x7f76240b0a10 2022-08-01T20:44:51.138 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.086+0000 7f761b7fe700 1 -- 172.21.15.17:0/1578631059 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f761c005d00 con 0x7f76240b0a10 2022-08-01T20:44:51.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.086+0000 7f7628ff6700 1 -- 172.21.15.17:0/1578631059 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f761400b180 con 0x7f76240b0a10 2022-08-01T20:44:51.139 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.086+0000 7f761b7fe700 1 -- 172.21.15.17:0/1578631059 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f761c0072c0 con 0x7f76240b0a10 2022-08-01T20:44:51.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.086+0000 7f7628ff6700 1 -- 172.21.15.17:0/1578631059 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7624145da0 con 0x7f76240b0a10 2022-08-01T20:44:51.140 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.090+0000 7f7628ff6700 1 -- 172.21.15.17:0/1578631059 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f762413dd30 con 0x7f76240b0a10 2022-08-01T20:44:51.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.090+0000 7f761b7fe700 1 -- 172.21.15.17:0/1578631059 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f761c01a060 con 0x7f76240b0a10 2022-08-01T20:44:51.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.090+0000 7f761b7fe700 1 --2- 172.21.15.17:0/1578631059 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f75fc053950 0x7f75fc055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:51.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.090+0000 7f761b7fe700 1 -- 172.21.15.17:0/1578631059 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f761c06b810 con 0x7f76240b0a10 2022-08-01T20:44:51.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.090+0000 7f762259c700 1 --2- 172.21.15.17:0/1578631059 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f75fc053950 0x7f75fc055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:51.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.094+0000 7f762259c700 1 --2- 172.21.15.17:0/1578631059 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f75fc053950 0x7f75fc055e10 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7f7614018000 tx=0x7f7614014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:51.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.098+0000 7f761b7fe700 1 -- 172.21.15.17:0/1578631059 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f761c0428b0 con 0x7f76240b0a10 2022-08-01T20:44:51.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.226+0000 7f7628ff6700 1 -- 172.21.15.17:0/1578631059 --> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f76240abd80 con 0x7f75fc053950 2022-08-01T20:44:51.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.226+0000 7f761b7fe700 1 -- 172.21.15.17:0/1578631059 <== mgr.4101 v2:172.21.15.77:6800/23151 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f761400d680 con 0x7f75fc053950 2022-08-01T20:44:51.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.226+0000 7f7628ff6700 1 -- 172.21.15.17:0/1578631059 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f75fc053950 msgr2=0x7f75fc055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:51.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.226+0000 7f7628ff6700 1 --2- 172.21.15.17:0/1578631059 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f75fc053950 0x7f75fc055e10 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7f7614018000 tx=0x7f7614014000).stop 2022-08-01T20:44:51.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.226+0000 7f7628ff6700 1 -- 172.21.15.17:0/1578631059 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76240b0a10 msgr2=0x7f7624064c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:51.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.226+0000 7f7628ff6700 1 --2- 172.21.15.17:0/1578631059 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76240b0a10 0x7f7624064c40 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 rx=0x7f761c000f40 tx=0x7f761c001360).stop 2022-08-01T20:44:51.234 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.226+0000 7f7628ff6700 1 -- 172.21.15.17:0/1578631059 shutdown_connections 2022-08-01T20:44:51.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.226+0000 7f7628ff6700 1 --2- 172.21.15.17:0/1578631059 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f75fc053950 0x7f75fc055e10 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:51.236 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.226+0000 7f7628ff6700 1 --2- 172.21.15.17:0/1578631059 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76240b0a10 0x7f7624064c40 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:51.236 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.226+0000 7f7628ff6700 1 --2- 172.21.15.17:0/1578631059 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f76240b0630 0x7f7624064750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:51.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.226+0000 7f7628ff6700 1 --2- 172.21.15.17:0/1578631059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76240aa2b0 0x7f76241405c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:51.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.226+0000 7f7628ff6700 1 -- 172.21.15.17:0/1578631059 >> 172.21.15.17:0/1578631059 conn(0x7f7624060950 msgr2=0x7f76240ae570 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:51.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.226+0000 7f7628ff6700 1 -- 172.21.15.17:0/1578631059 shutdown_connections 2022-08-01T20:44:51.239 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.226+0000 7f7628ff6700 1 -- 172.21.15.17:0/1578631059 wait complete. 2022-08-01T20:44:51.240 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 0 to deep-scrub 2022-08-01T20:44:51.242 INFO:tasks.ceph:Scrubbing osd.1 2022-08-01T20:44:51.243 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2022-08-01T20:44:51.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.354+0000 7fc709f81700 1 Processor -- start 2022-08-01T20:44:51.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.354+0000 7fc709f81700 1 -- start start 2022-08-01T20:44:51.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.354+0000 7fc709f81700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc7040614b0 0x7fc704061890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:51.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.358+0000 7fc709f81700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fc7041405d0 0x7fc704140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:51.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.358+0000 7fc709f81700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7040aee80 con 0x7fc7040ad750 2022-08-01T20:44:51.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.358+0000 7fc709f81700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7040b70b0 con 0x7fc7041405d0 2022-08-01T20:44:51.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.358+0000 7fc709f81700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7040af0b0 con 0x7fc7040614b0 2022-08-01T20:44:51.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.358+0000 7fc709780700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fc7041405d0 0x7fc704140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:51.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.358+0000 7fc708f7f700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc7040614b0 0x7fc704061890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:51.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.358+0000 7fc709780700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fc7041405d0 0x7fc704140a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:44046/0 (socket says 172.21.15.17:44046) 2022-08-01T20:44:51.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.358+0000 7fc709780700 1 -- 172.21.15.17:0/528297114 learned_addr learned my addr 172.21.15.17:0/528297114 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T20:44:51.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.358+0000 7fc709780700 1 -- 172.21.15.17:0/528297114 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc7040614b0 msgr2=0x7fc704061890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T20:44:51.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.358+0000 7fc709780700 1 --2- 172.21.15.17:0/528297114 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc7040614b0 0x7fc704061890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:51.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.358+0000 7fc709780700 1 -- 172.21.15.17:0/528297114 >> v1:172.21.15.77:6789/0 conn(0x7fc7040ad750 legacy=0x7fc7040b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:51.365 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.358+0000 7fc709780700 1 -- 172.21.15.17:0/528297114 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7040af350 con 0x7fc7041405d0 2022-08-01T20:44:51.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.358+0000 7fc709780700 1 --2- 172.21.15.17:0/528297114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7041405d0 0x7fc704140a60 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7fc6fc0032f0 tx=0x7fc6fc009290).ready entity=mon.1 client_cookie=93255243a1e4b47d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:51.366 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.358+0000 7fc7037fe700 1 -- 172.21.15.17:0/528297114 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6fc00b8a0 con 0x7fc7041405d0 2022-08-01T20:44:51.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.358+0000 7fc7037fe700 1 -- 172.21.15.17:0/528297114 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6fc00ba00 con 0x7fc7041405d0 2022-08-01T20:44:51.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.358+0000 7fc7037fe700 1 -- 172.21.15.17:0/528297114 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6fc00bb70 con 0x7fc7041405d0 2022-08-01T20:44:51.367 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.358+0000 7fc709f81700 1 -- 172.21.15.17:0/528297114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7041405d0 msgr2=0x7fc704140a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:51.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.358+0000 7fc709f81700 1 --2- 172.21.15.17:0/528297114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7041405d0 0x7fc704140a60 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7fc6fc0032f0 tx=0x7fc6fc009290).stop 2022-08-01T20:44:51.368 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.358+0000 7fc709f81700 1 -- 172.21.15.17:0/528297114 shutdown_connections 2022-08-01T20:44:51.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.358+0000 7fc709f81700 1 --2- 172.21.15.17:0/528297114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7041405d0 0x7fc704140a60 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:51.369 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.358+0000 7fc709f81700 1 --2- 172.21.15.17:0/528297114 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc7040614b0 0x7fc704061890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:51.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.358+0000 7fc709f81700 1 -- 172.21.15.17:0/528297114 >> 172.21.15.17:0/528297114 conn(0x7fc704066f70 msgr2=0x7fc704067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:51.370 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.358+0000 7fc709f81700 1 -- 172.21.15.17:0/528297114 shutdown_connections 2022-08-01T20:44:51.374 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.358+0000 7fc709f81700 1 -- 172.21.15.17:0/528297114 wait complete. 2022-08-01T20:44:51.378 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.362+0000 7fc709f81700 1 Processor -- start 2022-08-01T20:44:51.379 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.362+0000 7fc709f81700 1 -- start start 2022-08-01T20:44:51.380 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.362+0000 7fc709f81700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc7040614b0 0x7fc70409ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:51.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.362+0000 7fc709f81700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7040ad750 0x7fc7040b0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:51.381 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.362+0000 7fc709f81700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7041405d0 0x7fc704155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:51.382 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.362+0000 7fc709f81700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc704152fb0 con 0x7fc7041405d0 2022-08-01T20:44:51.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.362+0000 7fc709f81700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc704153250 con 0x7fc7040ad750 2022-08-01T20:44:51.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.362+0000 7fc709f81700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7040b0560 con 0x7fc7040614b0 2022-08-01T20:44:51.383 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.362+0000 7fc708f7f700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc7040614b0 0x7fc70409ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:51.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.362+0000 7fc708f7f700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc7040614b0 0x7fc70409ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3301/0 says I am v2:172.21.15.17:42226/0 (socket says 172.21.15.17:42226) 2022-08-01T20:44:51.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.362+0000 7fc708f7f700 1 -- 172.21.15.17:0/1132723656 learned_addr learned my addr 172.21.15.17:0/1132723656 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T20:44:51.384 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.362+0000 7fc709780700 1 --2- 172.21.15.17:0/1132723656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7041405d0 0x7fc704155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:51.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.362+0000 7fc703fff700 1 --2- 172.21.15.17:0/1132723656 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7040ad750 0x7fc7040b0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:51.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.362+0000 7fc708f7f700 1 -- 172.21.15.17:0/1132723656 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7040ad750 msgr2=0x7fc7040b0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:51.385 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.362+0000 7fc708f7f700 1 --2- 172.21.15.17:0/1132723656 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7040ad750 0x7fc7040b0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:51.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.362+0000 7fc708f7f700 1 -- 172.21.15.17:0/1132723656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7041405d0 msgr2=0x7fc704155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:51.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.362+0000 7fc708f7f700 1 --2- 172.21.15.17:0/1132723656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7041405d0 0x7fc704155a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:51.386 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.362+0000 7fc708f7f700 1 -- 172.21.15.17:0/1132723656 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6fc009530 con 0x7fc7040614b0 2022-08-01T20:44:51.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.362+0000 7fc709780700 1 --2- 172.21.15.17:0/1132723656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7041405d0 0x7fc704155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T20:44:51.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.362+0000 7fc708f7f700 1 --2- 172.21.15.17:0/1132723656 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc7040614b0 0x7fc70409ed80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7fc6ec00c000 tx=0x7fc6ec006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:51.387 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.362+0000 7fc701ffb700 1 -- 172.21.15.17:0/1132723656 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6ec010780 con 0x7fc7040614b0 2022-08-01T20:44:51.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.362+0000 7fc701ffb700 1 -- 172.21.15.17:0/1132723656 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6ec0108e0 con 0x7fc7040614b0 2022-08-01T20:44:51.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.362+0000 7fc701ffb700 1 -- 172.21.15.17:0/1132723656 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6ec010c40 con 0x7fc7040614b0 2022-08-01T20:44:51.388 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.362+0000 7fc709f81700 1 -- 172.21.15.17:0/1132723656 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7040ad500 con 0x7fc7040614b0 2022-08-01T20:44:51.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.362+0000 7fc709f81700 1 -- 172.21.15.17:0/1132723656 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7040b2b40 con 0x7fc7040614b0 2022-08-01T20:44:51.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.366+0000 7fc709f81700 1 -- 172.21.15.17:0/1132723656 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc7040b0720 con 0x7fc7040614b0 2022-08-01T20:44:51.389 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.366+0000 7fc701ffb700 1 -- 172.21.15.17:0/1132723656 <== mon.2 v2:172.21.15.77:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc6ec0334d0 con 0x7fc7040614b0 2022-08-01T20:44:51.391 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.366+0000 7fc701ffb700 1 --2- 172.21.15.17:0/1132723656 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc6e8053a10 0x7fc6e8055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:51.392 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.366+0000 7fc701ffb700 1 -- 172.21.15.17:0/1132723656 <== mon.2 v2:172.21.15.77:3301/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fc6ec06c2f0 con 0x7fc7040614b0 2022-08-01T20:44:51.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.366+0000 7fc701ffb700 1 --2- 172.21.15.17:0/1132723656 >> [v2:172.21.15.17:6808/23007,v1:172.21.15.17:6809/23007] conn(0x7fc6e805a6a0 0x7fc6e805cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:51.393 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.366+0000 7fc701ffb700 1 -- 172.21.15.17:0/1132723656 --> [v2:172.21.15.17:6808/23007,v1:172.21.15.17:6809/23007] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc6e805d1c0 con 0x7fc6e805a6a0 2022-08-01T20:44:51.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.366+0000 7fc701ffb700 1 -- 172.21.15.17:0/1132723656 <== mon.2 v2:172.21.15.77:3301/0 6 ==== mon_get_version_reply(handle=1 version=161) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6ec06c6b0 con 0x7fc7040614b0 2022-08-01T20:44:51.396 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.366+0000 7fc703fff700 1 --2- 172.21.15.17:0/1132723656 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc6e8053a10 0x7fc6e8055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:51.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.366+0000 7fc709780700 1 --2- 172.21.15.17:0/1132723656 >> [v2:172.21.15.17:6808/23007,v1:172.21.15.17:6809/23007] conn(0x7fc6e805a6a0 0x7fc6e805cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:51.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.366+0000 7fc709780700 1 --2- 172.21.15.17:0/1132723656 >> [v2:172.21.15.17:6808/23007,v1:172.21.15.17:6809/23007] conn(0x7fc6e805a6a0 0x7fc6e805cb40 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:51.397 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.366+0000 7fc703fff700 1 --2- 172.21.15.17:0/1132723656 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc6e8053a10 0x7fc6e8055ed0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 rx=0x7fc6f400d000 tx=0x7fc6f4007b90).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:51.398 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.370+0000 7fc701ffb700 1 -- 172.21.15.17:0/1132723656 <== osd.1 v2:172.21.15.17:6808/23007 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7fc6e805d1c0 con 0x7fc6e805a6a0 2022-08-01T20:44:51.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.386+0000 7fc709f81700 1 -- 172.21.15.17:0/1132723656 --> [v2:172.21.15.17:6808/23007,v1:172.21.15.17:6809/23007] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc7040b0ef0 con 0x7fc6e805a6a0 2022-08-01T20:44:51.399 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.386+0000 7fc701ffb700 1 -- 172.21.15.17:0/1132723656 <== osd.1 v2:172.21.15.17:6808/23007 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fc7040b0ef0 con 0x7fc6e805a6a0 2022-08-01T20:44:51.400 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T20:44:51.400 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-08-01T20:44:51.401 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T20:44:51.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.386+0000 7fc709f81700 1 -- 172.21.15.17:0/1132723656 >> [v2:172.21.15.17:6808/23007,v1:172.21.15.17:6809/23007] conn(0x7fc6e805a6a0 msgr2=0x7fc6e805cb40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:51.404 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.386+0000 7fc709f81700 1 --2- 172.21.15.17:0/1132723656 >> [v2:172.21.15.17:6808/23007,v1:172.21.15.17:6809/23007] conn(0x7fc6e805a6a0 0x7fc6e805cb40 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:51.405 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.386+0000 7fc709f81700 1 -- 172.21.15.17:0/1132723656 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc6e8053a10 msgr2=0x7fc6e8055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:51.406 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.386+0000 7fc709f81700 1 --2- 172.21.15.17:0/1132723656 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc6e8053a10 0x7fc6e8055ed0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 rx=0x7fc6f400d000 tx=0x7fc6f4007b90).stop 2022-08-01T20:44:51.407 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.386+0000 7fc709f81700 1 -- 172.21.15.17:0/1132723656 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc7040614b0 msgr2=0x7fc70409ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:51.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.386+0000 7fc709f81700 1 --2- 172.21.15.17:0/1132723656 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc7040614b0 0x7fc70409ed80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7fc6ec00c000 tx=0x7fc6ec006df0).stop 2022-08-01T20:44:51.408 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.386+0000 7fc709780700 1 -- 172.21.15.17:0/1132723656 reap_dead start 2022-08-01T20:44:51.409 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.386+0000 7fc709f81700 1 -- 172.21.15.17:0/1132723656 shutdown_connections 2022-08-01T20:44:51.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.386+0000 7fc709f81700 1 -- 172.21.15.17:0/1132723656 >> 172.21.15.17:0/1132723656 conn(0x7fc704066f70 msgr2=0x7fc70405cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:51.410 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.390+0000 7fc709f81700 1 -- 172.21.15.17:0/1132723656 shutdown_connections 2022-08-01T20:44:51.411 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.390+0000 7fc709f81700 1 -- 172.21.15.17:0/1132723656 wait complete. 2022-08-01T20:44:51.411 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2022-08-01T20:44:51.529 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.526+0000 7fc65898f700 1 Processor -- start 2022-08-01T20:44:51.530 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.526+0000 7fc65898f700 1 -- start start 2022-08-01T20:44:51.537 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.526+0000 7fc65898f700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc654060aa0 0x7fc6540acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:51.538 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.526+0000 7fc65898f700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc6540aa760 0x7fc65405c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:51.539 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.526+0000 7fc65898f700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6540b6e70 con 0x7fc6540b6a90 2022-08-01T20:44:51.540 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.526+0000 7fc65898f700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc654061370 con 0x7fc6540aa760 2022-08-01T20:44:51.549 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.526+0000 7fc65898f700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc654061610 con 0x7fc654060aa0 2022-08-01T20:44:51.551 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.526+0000 7fc652d9d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc6540aa760 0x7fc65405c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:51.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.526+0000 7fc652d9d700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc6540aa760 0x7fc65405c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46138/0 (socket says 172.21.15.17:46138) 2022-08-01T20:44:51.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.526+0000 7fc652d9d700 1 -- 172.21.15.17:0/163854675 learned_addr learned my addr 172.21.15.17:0/163854675 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T20:44:51.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.526+0000 7fc65259c700 1 --1- 172.21.15.17:0/163854675 >> v1:172.21.15.17:6789/0 conn(0x7fc6540b6a90 0x7fc6540aa050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:38330/0 (socket says 172.21.15.17:38330) 2022-08-01T20:44:51.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.526+0000 7fc651d9b700 1 --2- 172.21.15.17:0/163854675 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc654060aa0 0x7fc6540acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:51.556 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.526+0000 7fc652d9d700 1 -- 172.21.15.17:0/163854675 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc654060aa0 msgr2=0x7fc6540acee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T20:44:51.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.526+0000 7fc652d9d700 1 --2- 172.21.15.17:0/163854675 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc654060aa0 0x7fc6540acee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:51.557 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.526+0000 7fc65159a700 1 -- 172.21.15.17:0/163854675 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1814562782 0 0) 0x7fc6540b6e70 con 0x7fc6540b6a90 2022-08-01T20:44:51.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.526+0000 7fc652d9d700 1 -- 172.21.15.17:0/163854675 >> v1:172.21.15.17:6789/0 conn(0x7fc6540b6a90 legacy=0x7fc6540aa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:51.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.526+0000 7fc652d9d700 1 -- 172.21.15.17:0/163854675 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6540618b0 con 0x7fc6540aa760 2022-08-01T20:44:51.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.526+0000 7fc652d9d700 1 --2- 172.21.15.17:0/163854675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6540aa760 0x7fc65405c620 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 rx=0x7fc64c0032f0 tx=0x7fc64c009290).ready entity=mon.0 client_cookie=e2de670ee309f089 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:51.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.526+0000 7fc65159a700 1 -- 172.21.15.17:0/163854675 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc64c00b8a0 con 0x7fc6540aa760 2022-08-01T20:44:51.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.526+0000 7fc65159a700 1 -- 172.21.15.17:0/163854675 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64c00ba00 con 0x7fc6540aa760 2022-08-01T20:44:51.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.526+0000 7fc65159a700 1 -- 172.21.15.17:0/163854675 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc64c00bb70 con 0x7fc6540aa760 2022-08-01T20:44:51.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.526+0000 7fc65898f700 1 -- 172.21.15.17:0/163854675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6540aa760 msgr2=0x7fc65405c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:51.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.526+0000 7fc65898f700 1 --2- 172.21.15.17:0/163854675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6540aa760 0x7fc65405c620 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 rx=0x7fc64c0032f0 tx=0x7fc64c009290).stop 2022-08-01T20:44:51.563 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.530+0000 7fc65898f700 1 -- 172.21.15.17:0/163854675 shutdown_connections 2022-08-01T20:44:51.564 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.530+0000 7fc65898f700 1 --2- 172.21.15.17:0/163854675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6540aa760 0x7fc65405c620 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:51.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.530+0000 7fc65898f700 1 --2- 172.21.15.17:0/163854675 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc654060aa0 0x7fc6540acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:51.565 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.530+0000 7fc65898f700 1 -- 172.21.15.17:0/163854675 >> 172.21.15.17:0/163854675 conn(0x7fc6540b3610 msgr2=0x7fc6540b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:51.566 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.530+0000 7fc65898f700 1 -- 172.21.15.17:0/163854675 shutdown_connections 2022-08-01T20:44:51.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.530+0000 7fc65898f700 1 -- 172.21.15.17:0/163854675 wait complete. 2022-08-01T20:44:51.567 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.530+0000 7fc65898f700 1 Processor -- start 2022-08-01T20:44:51.568 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.530+0000 7fc65898f700 1 -- start start 2022-08-01T20:44:51.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.534+0000 7fc65898f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc654060aa0 0x7fc6541411e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:51.569 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.534+0000 7fc65898f700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc6540aa760 0x7fc6541416d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:51.570 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.534+0000 7fc65898f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6540b6a90 0x7fc654155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:51.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.534+0000 7fc65898f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc654141bc0 con 0x7fc654060aa0 2022-08-01T20:44:51.571 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.534+0000 7fc65898f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc654142720 con 0x7fc6540b6a90 2022-08-01T20:44:51.572 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.534+0000 7fc65898f700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc654142950 con 0x7fc6540aa760 2022-08-01T20:44:51.573 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.534+0000 7fc65259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc654060aa0 0x7fc6541411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:51.574 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.534+0000 7fc65259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc654060aa0 0x7fc6541411e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46142/0 (socket says 172.21.15.17:46142) 2022-08-01T20:44:51.575 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.534+0000 7fc65259c700 1 -- 172.21.15.17:0/3159559093 learned_addr learned my addr 172.21.15.17:0/3159559093 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T20:44:51.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.534+0000 7fc652d9d700 1 --2- 172.21.15.17:0/3159559093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6540b6a90 0x7fc654155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:51.576 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.534+0000 7fc651d9b700 1 --2- 172.21.15.17:0/3159559093 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc6540aa760 0x7fc6541416d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:51.577 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.534+0000 7fc65259c700 1 -- 172.21.15.17:0/3159559093 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc6540aa760 msgr2=0x7fc6541416d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:51.582 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.534+0000 7fc65259c700 1 --2- 172.21.15.17:0/3159559093 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc6540aa760 0x7fc6541416d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:51.583 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.534+0000 7fc65259c700 1 -- 172.21.15.17:0/3159559093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6540b6a90 msgr2=0x7fc654155bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:51.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.534+0000 7fc65259c700 1 --2- 172.21.15.17:0/3159559093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6540b6a90 0x7fc654155bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:51.584 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.534+0000 7fc65259c700 1 -- 172.21.15.17:0/3159559093 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc64c009530 con 0x7fc654060aa0 2022-08-01T20:44:51.585 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.534+0000 7fc65259c700 1 --2- 172.21.15.17:0/3159559093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc654060aa0 0x7fc6541411e0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 rx=0x7fc644000ed0 tx=0x7fc644007ab0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:51.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.534+0000 7fc6437fe700 1 -- 172.21.15.17:0/3159559093 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc64400ce50 con 0x7fc654060aa0 2022-08-01T20:44:51.586 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.534+0000 7fc6437fe700 1 -- 172.21.15.17:0/3159559093 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc644007170 con 0x7fc654060aa0 2022-08-01T20:44:51.587 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.534+0000 7fc65898f700 1 -- 172.21.15.17:0/3159559093 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6540afd20 con 0x7fc654060aa0 2022-08-01T20:44:51.590 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.534+0000 7fc6437fe700 1 -- 172.21.15.17:0/3159559093 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc644004730 con 0x7fc654060aa0 2022-08-01T20:44:51.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.534+0000 7fc65898f700 1 -- 172.21.15.17:0/3159559093 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6540aeee0 con 0x7fc654060aa0 2022-08-01T20:44:51.591 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.534+0000 7fc65898f700 1 -- 172.21.15.17:0/3159559093 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc65414d4e0 con 0x7fc654060aa0 2022-08-01T20:44:51.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.538+0000 7fc6437fe700 1 -- 172.21.15.17:0/3159559093 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc644004890 con 0x7fc654060aa0 2022-08-01T20:44:51.593 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.538+0000 7fc6437fe700 1 --2- 172.21.15.17:0/3159559093 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc630053950 0x7fc630055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:51.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.538+0000 7fc6437fe700 1 -- 172.21.15.17:0/3159559093 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fc64406b840 con 0x7fc654060aa0 2022-08-01T20:44:51.594 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.538+0000 7fc651d9b700 1 --2- 172.21.15.17:0/3159559093 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc630053950 0x7fc630055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:51.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.538+0000 7fc651d9b700 1 --2- 172.21.15.17:0/3159559093 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc630053950 0x7fc630055e10 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7fc64800b000 tx=0x7fc648008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:51.595 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.542+0000 7fc6437fe700 1 -- 172.21.15.17:0/3159559093 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc6440427d0 con 0x7fc654060aa0 2022-08-01T20:44:51.671 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.670+0000 7fc65898f700 1 -- 172.21.15.17:0/3159559093 --> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6540afeb0 con 0x7fc630053950 2022-08-01T20:44:51.673 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.670+0000 7fc6437fe700 1 -- 172.21.15.17:0/3159559093 <== mgr.4101 v2:172.21.15.77:6800/23151 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc6540afeb0 con 0x7fc630053950 2022-08-01T20:44:51.674 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.670+0000 7fc65898f700 1 -- 172.21.15.17:0/3159559093 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc630053950 msgr2=0x7fc630055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:51.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.670+0000 7fc65898f700 1 --2- 172.21.15.17:0/3159559093 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc630053950 0x7fc630055e10 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7fc64800b000 tx=0x7fc648008040).stop 2022-08-01T20:44:51.675 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.670+0000 7fc65898f700 1 -- 172.21.15.17:0/3159559093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc654060aa0 msgr2=0x7fc6541411e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:51.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.670+0000 7fc65898f700 1 --2- 172.21.15.17:0/3159559093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc654060aa0 0x7fc6541411e0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 rx=0x7fc644000ed0 tx=0x7fc644007ab0).stop 2022-08-01T20:44:51.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.670+0000 7fc65898f700 1 -- 172.21.15.17:0/3159559093 shutdown_connections 2022-08-01T20:44:51.676 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.670+0000 7fc65898f700 1 --2- 172.21.15.17:0/3159559093 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fc630053950 0x7fc630055e10 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:51.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.670+0000 7fc65898f700 1 --2- 172.21.15.17:0/3159559093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc6540b6a90 0x7fc654155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:51.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.670+0000 7fc65898f700 1 --2- 172.21.15.17:0/3159559093 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fc6540aa760 0x7fc6541416d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:51.677 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.670+0000 7fc65898f700 1 --2- 172.21.15.17:0/3159559093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc654060aa0 0x7fc6541411e0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:51.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.670+0000 7fc65898f700 1 -- 172.21.15.17:0/3159559093 >> 172.21.15.17:0/3159559093 conn(0x7fc6540b3610 msgr2=0x7fc65413a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:51.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.674+0000 7fc65898f700 1 -- 172.21.15.17:0/3159559093 shutdown_connections 2022-08-01T20:44:51.678 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.674+0000 7fc65898f700 1 -- 172.21.15.17:0/3159559093 wait complete. 2022-08-01T20:44:51.679 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 1 to deep-scrub 2022-08-01T20:44:51.685 INFO:tasks.ceph:Scrubbing osd.2 2022-08-01T20:44:51.687 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2022-08-01T20:44:51.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.802+0000 7febcbc39700 1 Processor -- start 2022-08-01T20:44:51.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.802+0000 7febcbc39700 1 -- start start 2022-08-01T20:44:51.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.802+0000 7febcbc39700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7febc40b4550 0x7febc40b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:51.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.802+0000 7febcbc39700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7febc40b6af0 0x7febc412bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:51.810 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.802+0000 7febcbc39700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febc4144240 con 0x7febc40b6710 2022-08-01T20:44:51.810 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.802+0000 7febcbc39700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febc4143d00 con 0x7febc40b4550 2022-08-01T20:44:51.810 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.802+0000 7febcbc39700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febc4143fa0 con 0x7febc40b6af0 2022-08-01T20:44:51.811 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.802+0000 7febc91d4700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7febc40b4550 0x7febc40b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:51.811 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.802+0000 7febc91d4700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7febc40b4550 0x7febc40b5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46152/0 (socket says 172.21.15.17:46152) 2022-08-01T20:44:51.811 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.802+0000 7febc91d4700 1 -- 172.21.15.17:0/1470215094 learned_addr learned my addr 172.21.15.17:0/1470215094 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T20:44:51.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.802+0000 7febca1d6700 1 --2- 172.21.15.17:0/1470215094 >> v2:172.21.15.77:3300/0 conn(0x7febc40b6af0 0x7febc412bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:51.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.806+0000 7febc91d4700 1 -- 172.21.15.17:0/1470215094 >> v2:172.21.15.77:3300/0 conn(0x7febc40b6af0 msgr2=0x7febc412bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T20:44:51.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.806+0000 7febc91d4700 1 --2- 172.21.15.17:0/1470215094 >> v2:172.21.15.77:3300/0 conn(0x7febc40b6af0 0x7febc412bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:51.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.806+0000 7febc91d4700 1 -- 172.21.15.17:0/1470215094 >> v1:172.21.15.77:6789/0 conn(0x7febc40b6710 legacy=0x7febc4061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:51.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.806+0000 7febc91d4700 1 -- 172.21.15.17:0/1470215094 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc4061780 con 0x7febc40b4550 2022-08-01T20:44:51.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.806+0000 7febca1d6700 1 --2- 172.21.15.17:0/1470215094 >> v2:172.21.15.77:3300/0 conn(0x7febc40b6af0 0x7febc412bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T20:44:51.814 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.806+0000 7febc91d4700 1 --2- 172.21.15.17:0/1470215094 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febc40b4550 0x7febc40b5d80 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 rx=0x7febc41432b0 tx=0x7febb8006eb0).ready entity=mon.0 client_cookie=dd81ba0569b409ca server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:51.815 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.806+0000 7febc89d3700 1 -- 172.21.15.17:0/1470215094 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7febb800d610 con 0x7febc40b4550 2022-08-01T20:44:51.815 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.806+0000 7febc89d3700 1 -- 172.21.15.17:0/1470215094 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb8000e90 con 0x7febc40b4550 2022-08-01T20:44:51.815 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.806+0000 7febc89d3700 1 -- 172.21.15.17:0/1470215094 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7febb800d960 con 0x7febc40b4550 2022-08-01T20:44:51.816 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.806+0000 7febcbc39700 1 -- 172.21.15.17:0/1470215094 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febc40b4550 msgr2=0x7febc40b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:51.816 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.806+0000 7febcbc39700 1 --2- 172.21.15.17:0/1470215094 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febc40b4550 0x7febc40b5d80 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 rx=0x7febc41432b0 tx=0x7febb8006eb0).stop 2022-08-01T20:44:51.816 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.806+0000 7febcbc39700 1 -- 172.21.15.17:0/1470215094 shutdown_connections 2022-08-01T20:44:51.817 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.806+0000 7febcbc39700 1 --2- 172.21.15.17:0/1470215094 >> v2:172.21.15.77:3300/0 conn(0x7febc40b6af0 0x7febc412bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:51.817 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.806+0000 7febcbc39700 1 --2- 172.21.15.17:0/1470215094 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febc40b4550 0x7febc40b5d80 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:51.817 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.806+0000 7febcbc39700 1 -- 172.21.15.17:0/1470215094 >> 172.21.15.17:0/1470215094 conn(0x7febc405c620 msgr2=0x7febc405ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:51.818 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.806+0000 7febcbc39700 1 -- 172.21.15.17:0/1470215094 shutdown_connections 2022-08-01T20:44:51.819 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.806+0000 7febcbc39700 1 -- 172.21.15.17:0/1470215094 wait complete. 2022-08-01T20:44:51.826 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.806+0000 7febcbc39700 1 Processor -- start 2022-08-01T20:44:51.827 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.806+0000 7febcbc39700 1 -- start start 2022-08-01T20:44:51.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.810+0000 7febcbc39700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febc40b4550 0x7febc412bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:51.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.810+0000 7febcbc39700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7febc40b6710 0x7febc4152a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:51.828 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.810+0000 7febcbc39700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7febc40b6af0 0x7febc4152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:51.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.810+0000 7febcbc39700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc40b2620 con 0x7febc40b4550 2022-08-01T20:44:51.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.810+0000 7febcbc39700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc40b3e70 con 0x7febc40b6710 2022-08-01T20:44:51.829 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.810+0000 7febcbc39700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febc4152580 con 0x7febc40b6af0 2022-08-01T20:44:51.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.810+0000 7febc99d5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febc40b4550 0x7febc412bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:51.830 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.810+0000 7febc99d5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febc40b4550 0x7febc412bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46156/0 (socket says 172.21.15.17:46156) 2022-08-01T20:44:51.832 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.810+0000 7febc99d5700 1 -- 172.21.15.17:0/471150513 learned_addr learned my addr 172.21.15.17:0/471150513 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T20:44:51.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.810+0000 7febc91d4700 1 --2- 172.21.15.17:0/471150513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7febc40b6710 0x7febc4152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:51.833 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.810+0000 7febca1d6700 1 --2- 172.21.15.17:0/471150513 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7febc40b6af0 0x7febc4152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:51.834 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.810+0000 7febc99d5700 1 -- 172.21.15.17:0/471150513 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7febc40b6af0 msgr2=0x7febc4152f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:51.835 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.810+0000 7febc99d5700 1 --2- 172.21.15.17:0/471150513 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7febc40b6af0 0x7febc4152f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:51.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.810+0000 7febc99d5700 1 -- 172.21.15.17:0/471150513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7febc40b6710 msgr2=0x7febc4152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:51.836 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.810+0000 7febc99d5700 1 --2- 172.21.15.17:0/471150513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7febc40b6710 0x7febc4152a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:51.837 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.810+0000 7febc99d5700 1 -- 172.21.15.17:0/471150513 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb800b1e0 con 0x7febc40b4550 2022-08-01T20:44:51.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.810+0000 7febc91d4700 1 --2- 172.21.15.17:0/471150513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7febc40b6710 0x7febc4152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T20:44:51.838 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.810+0000 7febc99d5700 1 --2- 172.21.15.17:0/471150513 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febc40b4550 0x7febc412bbc0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 rx=0x7febc40b1e10 tx=0x7febc0007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:51.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.810+0000 7febb6ffd700 1 -- 172.21.15.17:0/471150513 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7febc000ce30 con 0x7febc40b4550 2022-08-01T20:44:51.839 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.810+0000 7febb6ffd700 1 -- 172.21.15.17:0/471150513 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc000f470 con 0x7febc40b4550 2022-08-01T20:44:51.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.810+0000 7febcbc39700 1 -- 172.21.15.17:0/471150513 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febc4146000 con 0x7febc40b4550 2022-08-01T20:44:51.840 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.810+0000 7febb6ffd700 1 -- 172.21.15.17:0/471150513 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7febc000f7d0 con 0x7febc40b4550 2022-08-01T20:44:51.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.810+0000 7febcbc39700 1 -- 172.21.15.17:0/471150513 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc412a230 con 0x7febc40b4550 2022-08-01T20:44:51.841 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.810+0000 7febcbc39700 1 -- 172.21.15.17:0/471150513 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7febc4152400 con 0x7febc40b4550 2022-08-01T20:44:51.844 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.814+0000 7febb6ffd700 1 -- 172.21.15.17:0/471150513 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7febc0019030 con 0x7febc40b4550 2022-08-01T20:44:51.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.814+0000 7febb6ffd700 1 --2- 172.21.15.17:0/471150513 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7feba4053b00 0x7feba4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:51.845 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.814+0000 7febb6ffd700 1 -- 172.21.15.17:0/471150513 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7febc006b1d0 con 0x7febc40b4550 2022-08-01T20:44:51.846 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.814+0000 7febb6ffd700 1 --2- 172.21.15.17:0/471150513 >> [v2:172.21.15.17:6817/23010,v1:172.21.15.17:6819/23010] conn(0x7feba405a790 0x7feba405cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:51.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.814+0000 7febb6ffd700 1 -- 172.21.15.17:0/471150513 --> [v2:172.21.15.17:6817/23010,v1:172.21.15.17:6819/23010] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feba405d210 con 0x7feba405a790 2022-08-01T20:44:51.847 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.814+0000 7febb6ffd700 1 -- 172.21.15.17:0/471150513 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=161) v2 ==== 24+0+0 (secure 0 0 0) 0x7febc006d860 con 0x7febc40b4550 2022-08-01T20:44:51.848 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.814+0000 7febc91d4700 1 --2- 172.21.15.17:0/471150513 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7feba4053b00 0x7feba4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:51.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.814+0000 7febca1d6700 1 --2- 172.21.15.17:0/471150513 >> [v2:172.21.15.17:6817/23010,v1:172.21.15.17:6819/23010] conn(0x7feba405a790 0x7feba405cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:51.849 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.814+0000 7febc91d4700 1 --2- 172.21.15.17:0/471150513 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7feba4053b00 0x7feba4055fc0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7febb801a000 tx=0x7febb8015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:51.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.814+0000 7febca1d6700 1 --2- 172.21.15.17:0/471150513 >> [v2:172.21.15.17:6817/23010,v1:172.21.15.17:6819/23010] conn(0x7feba405a790 0x7feba405cc30 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:51.850 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.814+0000 7febb6ffd700 1 -- 172.21.15.17:0/471150513 <== osd.2 v2:172.21.15.17:6817/23010 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7feba405d210 con 0x7feba405a790 2022-08-01T20:44:51.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.830+0000 7febcbc39700 1 -- 172.21.15.17:0/471150513 --> [v2:172.21.15.17:6817/23010,v1:172.21.15.17:6819/23010] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7febc412c3f0 con 0x7feba405a790 2022-08-01T20:44:51.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.834+0000 7febb6ffd700 1 -- 172.21.15.17:0/471150513 <== osd.2 v2:172.21.15.17:6817/23010 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7febc412c3f0 con 0x7feba405a790 2022-08-01T20:44:51.852 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T20:44:51.853 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-08-01T20:44:51.853 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T20:44:51.857 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.834+0000 7febcbc39700 1 -- 172.21.15.17:0/471150513 >> [v2:172.21.15.17:6817/23010,v1:172.21.15.17:6819/23010] conn(0x7feba405a790 msgr2=0x7feba405cc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:51.858 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.834+0000 7febcbc39700 1 --2- 172.21.15.17:0/471150513 >> [v2:172.21.15.17:6817/23010,v1:172.21.15.17:6819/23010] conn(0x7feba405a790 0x7feba405cc30 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:51.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.834+0000 7febcbc39700 1 -- 172.21.15.17:0/471150513 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7feba4053b00 msgr2=0x7feba4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:51.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.834+0000 7febcbc39700 1 --2- 172.21.15.17:0/471150513 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7feba4053b00 0x7feba4055fc0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7febb801a000 tx=0x7febb8015000).stop 2022-08-01T20:44:51.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.834+0000 7febcbc39700 1 -- 172.21.15.17:0/471150513 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febc40b4550 msgr2=0x7febc412bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:51.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.834+0000 7febcbc39700 1 --2- 172.21.15.17:0/471150513 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febc40b4550 0x7febc412bbc0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 rx=0x7febc40b1e10 tx=0x7febc0007c50).stop 2022-08-01T20:44:51.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.834+0000 7febca1d6700 1 -- 172.21.15.17:0/471150513 reap_dead start 2022-08-01T20:44:51.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.834+0000 7febcbc39700 1 -- 172.21.15.17:0/471150513 shutdown_connections 2022-08-01T20:44:51.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.834+0000 7febcbc39700 1 -- 172.21.15.17:0/471150513 >> 172.21.15.17:0/471150513 conn(0x7febc405c620 msgr2=0x7febc4062100 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:51.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.834+0000 7febcbc39700 1 -- 172.21.15.17:0/471150513 shutdown_connections 2022-08-01T20:44:51.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.834+0000 7febcbc39700 1 -- 172.21.15.17:0/471150513 wait complete. 2022-08-01T20:44:51.865 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2022-08-01T20:44:51.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.970+0000 7f6e4d1a1700 1 Processor -- start 2022-08-01T20:44:51.976 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.970+0000 7f6e4d1a1700 1 -- start start 2022-08-01T20:44:51.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.970+0000 7f6e4d1a1700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f6e480b1f40 0x7f6e480b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:51.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.970+0000 7f6e4d1a1700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e480b04e0 con 0x7f6e48148530 2022-08-01T20:44:51.977 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.970+0000 7f6e4d1a1700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e480b0780 con 0x7f6e480a7250 2022-08-01T20:44:51.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.970+0000 7f6e4d1a1700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e480a9250 con 0x7f6e480b1f40 2022-08-01T20:44:51.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.974+0000 7f6e4659c700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f6e48148530 0x7f6e480b2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:38364/0 (socket says 172.21.15.17:38364) 2022-08-01T20:44:51.978 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.974+0000 7f6e4659c700 1 -- 172.21.15.17:0/4242667366 learned_addr learned my addr 172.21.15.17:0/4242667366 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T20:44:51.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.974+0000 7f6e46d9d700 1 --2- 172.21.15.17:0/4242667366 >> v2:172.21.15.77:3300/0 conn(0x7f6e480b1f40 0x7f6e480b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:51.979 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.974+0000 7f6e45d9b700 1 -- 172.21.15.17:0/4242667366 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1196629142 0 0) 0x7f6e480b04e0 con 0x7f6e48148530 2022-08-01T20:44:51.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.974+0000 7f6e45d9b700 1 -- 172.21.15.17:0/4242667366 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e2c003260 con 0x7f6e48148530 2022-08-01T20:44:51.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.974+0000 7f6e46d9d700 1 -- 172.21.15.17:0/4242667366 >> v1:172.21.15.77:6789/0 conn(0x7f6e480a7250 legacy=0x7f6e481448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:51.980 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.974+0000 7f6e46d9d700 1 -- 172.21.15.17:0/4242667366 >> v1:172.21.15.17:6789/0 conn(0x7f6e48148530 legacy=0x7f6e480b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:51.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.974+0000 7f6e46d9d700 1 -- 172.21.15.17:0/4242667366 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e480b0a20 con 0x7f6e480b1f40 2022-08-01T20:44:51.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.974+0000 7f6e46d9d700 1 --2- 172.21.15.17:0/4242667366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e480b1f40 0x7f6e480b1160 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7f6e481478c0 tx=0x7f6e3c006df0).ready entity=mon.1 client_cookie=de02fc6db40cb604 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:51.981 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.974+0000 7f6e45d9b700 1 -- 172.21.15.17:0/4242667366 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e3c00d610 con 0x7f6e480b1f40 2022-08-01T20:44:51.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.974+0000 7f6e45d9b700 1 -- 172.21.15.17:0/4242667366 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3c000e90 con 0x7f6e480b1f40 2022-08-01T20:44:51.982 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.974+0000 7f6e45d9b700 1 -- 172.21.15.17:0/4242667366 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e3c00d960 con 0x7f6e480b1f40 2022-08-01T20:44:51.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.974+0000 7f6e4d1a1700 1 -- 172.21.15.17:0/4242667366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e480b1f40 msgr2=0x7f6e480b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:51.986 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.974+0000 7f6e4d1a1700 1 --2- 172.21.15.17:0/4242667366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e480b1f40 0x7f6e480b1160 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7f6e481478c0 tx=0x7f6e3c006df0).stop 2022-08-01T20:44:51.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.974+0000 7f6e4d1a1700 1 -- 172.21.15.17:0/4242667366 shutdown_connections 2022-08-01T20:44:51.987 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.974+0000 7f6e4d1a1700 1 --2- 172.21.15.17:0/4242667366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e480b1f40 0x7f6e480b1160 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:51.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.974+0000 7f6e4d1a1700 1 -- 172.21.15.17:0/4242667366 >> 172.21.15.17:0/4242667366 conn(0x7f6e4805f840 msgr2=0x7f6e4805fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:51.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.974+0000 7f6e4d1a1700 1 -- 172.21.15.17:0/4242667366 shutdown_connections 2022-08-01T20:44:51.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.974+0000 7f6e4d1a1700 1 -- 172.21.15.17:0/4242667366 wait complete. 2022-08-01T20:44:51.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.978+0000 7f6e4d1a1700 1 Processor -- start 2022-08-01T20:44:51.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.978+0000 7f6e4d1a1700 1 -- start start 2022-08-01T20:44:51.989 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.978+0000 7f6e4d1a1700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6e480a7250 0x7f6e480b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:51.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.978+0000 7f6e4d1a1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e480b1f40 0x7f6e480a9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:51.991 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.978+0000 7f6e4d1a1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e48148530 0x7f6e48148c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:51.991 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.978+0000 7f6e4d1a1700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4814a860 con 0x7f6e480b1f40 2022-08-01T20:44:51.992 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.978+0000 7f6e4d1a1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4814aeb0 con 0x7f6e48148530 2022-08-01T20:44:51.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.978+0000 7f6e4d1a1700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4814b150 con 0x7f6e480a7250 2022-08-01T20:44:51.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.978+0000 7f6e4659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e480b1f40 0x7f6e480a9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:51.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.978+0000 7f6e46d9d700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6e480a7250 0x7f6e480b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:51.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.978+0000 7f6e4659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e480b1f40 0x7f6e480a9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46174/0 (socket says 172.21.15.17:46174) 2022-08-01T20:44:51.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.978+0000 7f6e4659c700 1 -- 172.21.15.17:0/17051855 learned_addr learned my addr 172.21.15.17:0/17051855 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T20:44:51.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.978+0000 7f6e4759e700 1 --2- 172.21.15.17:0/17051855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e48148530 0x7f6e48148c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:51.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.978+0000 7f6e4659c700 1 -- 172.21.15.17:0/17051855 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6e480a7250 msgr2=0x7f6e480b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:51.997 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.978+0000 7f6e4659c700 1 --2- 172.21.15.17:0/17051855 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6e480a7250 0x7f6e480b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:51.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.978+0000 7f6e4659c700 1 -- 172.21.15.17:0/17051855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e48148530 msgr2=0x7f6e48148c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:51.998 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.978+0000 7f6e4659c700 1 --2- 172.21.15.17:0/17051855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e48148530 0x7f6e48148c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:51.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.978+0000 7f6e4659c700 1 -- 172.21.15.17:0/17051855 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e3c00b180 con 0x7f6e480b1f40 2022-08-01T20:44:51.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.978+0000 7f6e4659c700 1 --2- 172.21.15.17:0/17051855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e480b1f40 0x7f6e480a9950 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 rx=0x7f6e38001490 tx=0x7f6e38007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:51.999 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.978+0000 7f6e37fff700 1 -- 172.21.15.17:0/17051855 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e38005ba0 con 0x7f6e480b1f40 2022-08-01T20:44:52.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.978+0000 7f6e37fff700 1 -- 172.21.15.17:0/17051855 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e38005d00 con 0x7f6e480b1f40 2022-08-01T20:44:52.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.978+0000 7f6e4d1a1700 1 -- 172.21.15.17:0/17051855 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e480abf20 con 0x7f6e480b1f40 2022-08-01T20:44:52.000 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.978+0000 7f6e37fff700 1 -- 172.21.15.17:0/17051855 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e380072c0 con 0x7f6e480b1f40 2022-08-01T20:44:52.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.978+0000 7f6e4d1a1700 1 -- 172.21.15.17:0/17051855 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e4814a150 con 0x7f6e480b1f40 2022-08-01T20:44:52.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.982+0000 7f6e4d1a1700 1 -- 172.21.15.17:0/17051855 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e4814b310 con 0x7f6e480b1f40 2022-08-01T20:44:52.001 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.982+0000 7f6e37fff700 1 -- 172.21.15.17:0/17051855 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6e38010070 con 0x7f6e480b1f40 2022-08-01T20:44:52.002 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.982+0000 7f6e37fff700 1 --2- 172.21.15.17:0/17051855 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6e24053a70 0x7f6e24055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:52.002 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.982+0000 7f6e37fff700 1 -- 172.21.15.17:0/17051855 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f6e3806b720 con 0x7f6e480b1f40 2022-08-01T20:44:52.003 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.982+0000 7f6e46d9d700 1 --2- 172.21.15.17:0/17051855 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6e24053a70 0x7f6e24055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:52.003 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.982+0000 7f6e46d9d700 1 --2- 172.21.15.17:0/17051855 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6e24053a70 0x7f6e24055f30 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7f6e3c01a000 tx=0x7f6e3c014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:52.004 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:51.990+0000 7f6e37fff700 1 -- 172.21.15.17:0/17051855 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6e38042720 con 0x7f6e480b1f40 2022-08-01T20:44:52.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.110+0000 7f6e4d1a1700 1 -- 172.21.15.17:0/17051855 --> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e480b0cc0 con 0x7f6e24053a70 2022-08-01T20:44:52.114 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.110+0000 7f6e37fff700 1 -- 172.21.15.17:0/17051855 <== mgr.4101 v2:172.21.15.77:6800/23151 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6e480b0cc0 con 0x7f6e24053a70 2022-08-01T20:44:52.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.114+0000 7f6e4d1a1700 1 -- 172.21.15.17:0/17051855 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6e24053a70 msgr2=0x7f6e24055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:52.116 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.114+0000 7f6e4d1a1700 1 --2- 172.21.15.17:0/17051855 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6e24053a70 0x7f6e24055f30 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7f6e3c01a000 tx=0x7f6e3c014000).stop 2022-08-01T20:44:52.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.114+0000 7f6e4d1a1700 1 -- 172.21.15.17:0/17051855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e480b1f40 msgr2=0x7f6e480a9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:52.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.114+0000 7f6e4d1a1700 1 --2- 172.21.15.17:0/17051855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e480b1f40 0x7f6e480a9950 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 rx=0x7f6e38001490 tx=0x7f6e38007e20).stop 2022-08-01T20:44:52.117 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.114+0000 7f6e4d1a1700 1 -- 172.21.15.17:0/17051855 shutdown_connections 2022-08-01T20:44:52.118 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.114+0000 7f6e4d1a1700 1 --2- 172.21.15.17:0/17051855 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f6e24053a70 0x7f6e24055f30 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:52.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.114+0000 7f6e4d1a1700 1 --2- 172.21.15.17:0/17051855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e48148530 0x7f6e48148c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:52.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.114+0000 7f6e4d1a1700 1 --2- 172.21.15.17:0/17051855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e480b1f40 0x7f6e480a9950 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:52.119 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.114+0000 7f6e4d1a1700 1 --2- 172.21.15.17:0/17051855 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f6e480a7250 0x7f6e480b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:52.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.114+0000 7f6e4d1a1700 1 -- 172.21.15.17:0/17051855 >> 172.21.15.17:0/17051855 conn(0x7f6e4805f840 msgr2=0x7f6e48147900 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:52.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.114+0000 7f6e4d1a1700 1 -- 172.21.15.17:0/17051855 shutdown_connections 2022-08-01T20:44:52.120 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.114+0000 7f6e4d1a1700 1 -- 172.21.15.17:0/17051855 wait complete. 2022-08-01T20:44:52.121 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 2 to deep-scrub 2022-08-01T20:44:52.128 INFO:tasks.ceph:Scrubbing osd.3 2022-08-01T20:44:52.129 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2022-08-01T20:44:52.252 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.246+0000 7f530a416700 1 Processor -- start 2022-08-01T20:44:52.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.246+0000 7f530a416700 1 -- start start 2022-08-01T20:44:52.253 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.246+0000 7f530a416700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5304061890 0x7f53041466b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:52.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.246+0000 7f530a416700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53040aee80 con 0x7f53040ad750 2022-08-01T20:44:52.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.246+0000 7f530a416700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53040b70b0 con 0x7f53040614b0 2022-08-01T20:44:52.254 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.246+0000 7f530a416700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53040af0b0 con 0x7f5304061890 2022-08-01T20:44:52.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.246+0000 7f5309c15700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5304061890 0x7f53041466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:52.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.246+0000 7f5308c13700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f53040ad750 0x7f53041405d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:38376/0 (socket says 172.21.15.17:38376) 2022-08-01T20:44:52.255 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.246+0000 7f5308c13700 1 -- 172.21.15.17:0/958709624 learned_addr learned my addr 172.21.15.17:0/958709624 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T20:44:52.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.246+0000 7f5309c15700 1 -- 172.21.15.17:0/958709624 >> v1:172.21.15.77:6789/0 conn(0x7f53040614b0 legacy=0x7f53040b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:52.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.246+0000 7f5309c15700 1 -- 172.21.15.17:0/958709624 >> v1:172.21.15.17:6789/0 conn(0x7f53040ad750 legacy=0x7f53041405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:52.256 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.246+0000 7f5309c15700 1 -- 172.21.15.17:0/958709624 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53040af350 con 0x7f5304061890 2022-08-01T20:44:52.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.246+0000 7f5309c15700 1 --2- 172.21.15.17:0/958709624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5304061890 0x7f53041466b0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 rx=0x7f52fc0032f0 tx=0x7f52fc0091d0).ready entity=mon.0 client_cookie=28fdcba0b890ab3d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:52.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.246+0000 7f52fbfff700 1 -- 172.21.15.17:0/958709624 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52fc009ea0 con 0x7f5304061890 2022-08-01T20:44:52.257 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.246+0000 7f52fbfff700 1 -- 172.21.15.17:0/958709624 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52fc00b8a0 con 0x7f5304061890 2022-08-01T20:44:52.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.246+0000 7f52fbfff700 1 -- 172.21.15.17:0/958709624 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52fc00ba10 con 0x7f5304061890 2022-08-01T20:44:52.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.250+0000 7f530a416700 1 -- 172.21.15.17:0/958709624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5304061890 msgr2=0x7f53041466b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:52.258 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.250+0000 7f530a416700 1 --2- 172.21.15.17:0/958709624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5304061890 0x7f53041466b0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 rx=0x7f52fc0032f0 tx=0x7f52fc0091d0).stop 2022-08-01T20:44:52.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.250+0000 7f530a416700 1 -- 172.21.15.17:0/958709624 shutdown_connections 2022-08-01T20:44:52.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.250+0000 7f530a416700 1 --2- 172.21.15.17:0/958709624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5304061890 0x7f53041466b0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:52.259 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.250+0000 7f530a416700 1 -- 172.21.15.17:0/958709624 >> 172.21.15.17:0/958709624 conn(0x7f5304066f70 msgr2=0x7f5304067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:52.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.250+0000 7f530a416700 1 -- 172.21.15.17:0/958709624 shutdown_connections 2022-08-01T20:44:52.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.250+0000 7f530a416700 1 -- 172.21.15.17:0/958709624 wait complete. 2022-08-01T20:44:52.260 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.250+0000 7f530a416700 1 Processor -- start 2022-08-01T20:44:52.264 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.250+0000 7f530a416700 1 -- start start 2022-08-01T20:44:52.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.254+0000 7f530a416700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53040614b0 0x7f5304147000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:52.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.254+0000 7f530a416700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5304061890 0x7f530413e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:52.265 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.254+0000 7f530a416700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53040ad750 0x7f530413eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:52.266 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.254+0000 7f530a416700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53040b2340 con 0x7f53040ad750 2022-08-01T20:44:52.266 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.254+0000 7f530a416700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53040b2570 con 0x7f53040614b0 2022-08-01T20:44:52.266 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.254+0000 7f530a416700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f530409eae0 con 0x7f5304061890 2022-08-01T20:44:52.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.254+0000 7f5309c15700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53040ad750 0x7f530413eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:52.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.254+0000 7f5309c15700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53040ad750 0x7f530413eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46190/0 (socket says 172.21.15.17:46190) 2022-08-01T20:44:52.267 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.254+0000 7f5309c15700 1 -- 172.21.15.17:0/4010277388 learned_addr learned my addr 172.21.15.17:0/4010277388 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T20:44:52.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.254+0000 7f5308c13700 1 --2- 172.21.15.17:0/4010277388 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5304061890 0x7f530413e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:52.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.254+0000 7f5309414700 1 --2- 172.21.15.17:0/4010277388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53040614b0 0x7f5304147000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:52.268 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.254+0000 7f5309c15700 1 -- 172.21.15.17:0/4010277388 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5304061890 msgr2=0x7f530413e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:52.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.254+0000 7f5309c15700 1 --2- 172.21.15.17:0/4010277388 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f5304061890 0x7f530413e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:52.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.254+0000 7f5309c15700 1 -- 172.21.15.17:0/4010277388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53040614b0 msgr2=0x7f5304147000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:52.269 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.254+0000 7f5309c15700 1 --2- 172.21.15.17:0/4010277388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53040614b0 0x7f5304147000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:52.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.254+0000 7f5309c15700 1 -- 172.21.15.17:0/4010277388 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52fc009470 con 0x7f53040ad750 2022-08-01T20:44:52.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.254+0000 7f5309414700 1 --2- 172.21.15.17:0/4010277388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53040614b0 0x7f5304147000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T20:44:52.270 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.254+0000 7f5309c15700 1 --2- 172.21.15.17:0/4010277388 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53040ad750 0x7f530413eee0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 rx=0x7f52fc0032f0 tx=0x7f52fc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:52.271 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.254+0000 7f52fa7fc700 1 -- 172.21.15.17:0/4010277388 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52fc009ea0 con 0x7f53040ad750 2022-08-01T20:44:52.271 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.254+0000 7f52fa7fc700 1 -- 172.21.15.17:0/4010277388 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52fc01f5e0 con 0x7f53040ad750 2022-08-01T20:44:52.271 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.254+0000 7f530a416700 1 -- 172.21.15.17:0/4010277388 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f530409ed80 con 0x7f53040ad750 2022-08-01T20:44:52.272 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.254+0000 7f52fa7fc700 1 -- 172.21.15.17:0/4010277388 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52fc01f940 con 0x7f53040ad750 2022-08-01T20:44:52.272 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.254+0000 7f530a416700 1 -- 172.21.15.17:0/4010277388 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53040a1250 con 0x7f53040ad750 2022-08-01T20:44:52.272 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.254+0000 7f530a416700 1 -- 172.21.15.17:0/4010277388 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f530409f220 con 0x7f53040ad750 2022-08-01T20:44:52.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.258+0000 7f52fa7fc700 1 -- 172.21.15.17:0/4010277388 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f52fc026030 con 0x7f53040ad750 2022-08-01T20:44:52.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.258+0000 7f52fa7fc700 1 --2- 172.21.15.17:0/4010277388 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f52ec053980 0x7f52ec055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:52.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.258+0000 7f52fa7fc700 1 -- 172.21.15.17:0/4010277388 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f52fc06d240 con 0x7f53040ad750 2022-08-01T20:44:52.273 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.258+0000 7f5309414700 1 --2- 172.21.15.17:0/4010277388 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f52ec053980 0x7f52ec055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:52.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.258+0000 7f52fa7fc700 1 --2- 172.21.15.17:0/4010277388 >> [v2:172.21.15.17:6816/23009,v1:172.21.15.17:6818/23009] conn(0x7f52ec05a6b0 0x7f52ec05cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:52.274 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.258+0000 7f52fa7fc700 1 -- 172.21.15.17:0/4010277388 --> [v2:172.21.15.17:6816/23009,v1:172.21.15.17:6818/23009] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f52ec05d1d0 con 0x7f52ec05a6b0 2022-08-01T20:44:52.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.258+0000 7f52fa7fc700 1 -- 172.21.15.17:0/4010277388 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=161) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52fc06d510 con 0x7f53040ad750 2022-08-01T20:44:52.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.258+0000 7f5308c13700 1 --2- 172.21.15.17:0/4010277388 >> [v2:172.21.15.17:6816/23009,v1:172.21.15.17:6818/23009] conn(0x7f52ec05a6b0 0x7f52ec05cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:52.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.258+0000 7f5309414700 1 --2- 172.21.15.17:0/4010277388 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f52ec053980 0x7f52ec055e40 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7f530000b000 tx=0x7f5300009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:52.275 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.258+0000 7f5308c13700 1 --2- 172.21.15.17:0/4010277388 >> [v2:172.21.15.17:6816/23009,v1:172.21.15.17:6818/23009] conn(0x7f52ec05a6b0 0x7f52ec05cb50 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:52.276 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.258+0000 7f52fa7fc700 1 -- 172.21.15.17:0/4010277388 <== osd.3 v2:172.21.15.17:6816/23009 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f52ec05d1d0 con 0x7f52ec05a6b0 2022-08-01T20:44:52.280 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.278+0000 7f530a416700 1 -- 172.21.15.17:0/4010277388 --> [v2:172.21.15.17:6816/23009,v1:172.21.15.17:6818/23009] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f53040b3c50 con 0x7f52ec05a6b0 2022-08-01T20:44:52.281 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.278+0000 7f52fa7fc700 1 -- 172.21.15.17:0/4010277388 <== osd.3 v2:172.21.15.17:6816/23009 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f53040b3c50 con 0x7f52ec05a6b0 2022-08-01T20:44:52.282 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T20:44:52.283 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-08-01T20:44:52.284 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T20:44:52.287 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.278+0000 7f530a416700 1 -- 172.21.15.17:0/4010277388 >> [v2:172.21.15.17:6816/23009,v1:172.21.15.17:6818/23009] conn(0x7f52ec05a6b0 msgr2=0x7f52ec05cb50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:52.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.278+0000 7f530a416700 1 --2- 172.21.15.17:0/4010277388 >> [v2:172.21.15.17:6816/23009,v1:172.21.15.17:6818/23009] conn(0x7f52ec05a6b0 0x7f52ec05cb50 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:52.288 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.278+0000 7f530a416700 1 -- 172.21.15.17:0/4010277388 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f52ec053980 msgr2=0x7f52ec055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:52.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.278+0000 7f530a416700 1 --2- 172.21.15.17:0/4010277388 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f52ec053980 0x7f52ec055e40 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7f530000b000 tx=0x7f5300009040).stop 2022-08-01T20:44:52.289 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.278+0000 7f530a416700 1 -- 172.21.15.17:0/4010277388 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53040ad750 msgr2=0x7f530413eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:52.290 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.278+0000 7f530a416700 1 --2- 172.21.15.17:0/4010277388 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53040ad750 0x7f530413eee0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 rx=0x7f52fc0032f0 tx=0x7f52fc016000).stop 2022-08-01T20:44:52.291 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.278+0000 7f5309c15700 1 -- 172.21.15.17:0/4010277388 reap_dead start 2022-08-01T20:44:52.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.278+0000 7f530a416700 1 -- 172.21.15.17:0/4010277388 shutdown_connections 2022-08-01T20:44:52.292 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.278+0000 7f530a416700 1 -- 172.21.15.17:0/4010277388 >> 172.21.15.17:0/4010277388 conn(0x7f5304066f70 msgr2=0x7f5304067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:52.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.278+0000 7f530a416700 1 -- 172.21.15.17:0/4010277388 shutdown_connections 2022-08-01T20:44:52.293 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.278+0000 7f530a416700 1 -- 172.21.15.17:0/4010277388 wait complete. 2022-08-01T20:44:52.295 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2022-08-01T20:44:52.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.406+0000 7fcd2f59e700 1 Processor -- start 2022-08-01T20:44:52.421 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.406+0000 7fcd2f59e700 1 -- start start 2022-08-01T20:44:52.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.406+0000 7fcd2f59e700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcd300b01a0 0x7fcd300a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:52.422 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.406+0000 7fcd2f59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcd300afd60 0x7fcd300a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:52.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.406+0000 7fcd2f59e700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd300a7250 con 0x7fcd300b2710 2022-08-01T20:44:52.423 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.406+0000 7fcd2f59e700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd300a7480 con 0x7fcd300b01a0 2022-08-01T20:44:52.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.406+0000 7fcd2f59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd300a7720 con 0x7fcd300afd60 2022-08-01T20:44:52.424 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.406+0000 7fcd2dd9b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcd300b01a0 0x7fcd300a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:52.425 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.406+0000 7fcd2dd9b700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcd300b01a0 0x7fcd300a8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46196/0 (socket says 172.21.15.17:46196) 2022-08-01T20:44:52.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.406+0000 7fcd2dd9b700 1 -- 172.21.15.17:0/122468497 learned_addr learned my addr 172.21.15.17:0/122468497 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T20:44:52.426 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.406+0000 7fcd2ed9d700 1 --2- 172.21.15.17:0/122468497 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcd300afd60 0x7fcd300a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:52.427 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.410+0000 7fcd2dd9b700 1 -- 172.21.15.17:0/122468497 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcd300afd60 msgr2=0x7fcd300a8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T20:44:52.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.410+0000 7fcd2dd9b700 1 --2- 172.21.15.17:0/122468497 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcd300afd60 0x7fcd300a8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:52.428 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.410+0000 7fcd2dd9b700 1 -- 172.21.15.17:0/122468497 >> v1:172.21.15.77:6789/0 conn(0x7fcd300b2710 legacy=0x7fcd3014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T20:44:52.429 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.410+0000 7fcd2dd9b700 1 -- 172.21.15.17:0/122468497 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd300a79c0 con 0x7fcd300b01a0 2022-08-01T20:44:52.430 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.410+0000 7fcd2dd9b700 1 --2- 172.21.15.17:0/122468497 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd300b01a0 0x7fcd300a8540 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 rx=0x7fcd3006b950 tx=0x7fcd20009290).ready entity=mon.0 client_cookie=9d512bc602df6625 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:52.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.410+0000 7fcd2d59a700 1 -- 172.21.15.17:0/122468497 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd2000b860 con 0x7fcd300b01a0 2022-08-01T20:44:52.431 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.410+0000 7fcd2d59a700 1 -- 172.21.15.17:0/122468497 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2000b9c0 con 0x7fcd300b01a0 2022-08-01T20:44:52.432 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.410+0000 7fcd2d59a700 1 -- 172.21.15.17:0/122468497 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd2000bb70 con 0x7fcd300b01a0 2022-08-01T20:44:52.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.410+0000 7fcd2f59e700 1 -- 172.21.15.17:0/122468497 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd300b01a0 msgr2=0x7fcd300a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:52.433 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.410+0000 7fcd2f59e700 1 --2- 172.21.15.17:0/122468497 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd300b01a0 0x7fcd300a8540 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 rx=0x7fcd3006b950 tx=0x7fcd20009290).stop 2022-08-01T20:44:52.434 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.410+0000 7fcd2f59e700 1 -- 172.21.15.17:0/122468497 shutdown_connections 2022-08-01T20:44:52.435 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.410+0000 7fcd2f59e700 1 --2- 172.21.15.17:0/122468497 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcd300afd60 0x7fcd300a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:52.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.410+0000 7fcd2f59e700 1 --2- 172.21.15.17:0/122468497 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd300b01a0 0x7fcd300a8540 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:52.436 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.410+0000 7fcd2f59e700 1 -- 172.21.15.17:0/122468497 >> 172.21.15.17:0/122468497 conn(0x7fcd30060310 msgr2=0x7fcd300b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:52.437 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.410+0000 7fcd2f59e700 1 -- 172.21.15.17:0/122468497 shutdown_connections 2022-08-01T20:44:52.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.410+0000 7fcd2f59e700 1 -- 172.21.15.17:0/122468497 wait complete. 2022-08-01T20:44:52.438 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.410+0000 7fcd2f59e700 1 Processor -- start 2022-08-01T20:44:52.439 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.410+0000 7fcd2f59e700 1 -- start start 2022-08-01T20:44:52.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.414+0000 7fcd2f59e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd300afd60 0x7fcd300ab930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:52.440 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.414+0000 7fcd2f59e700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcd300b01a0 0x7fcd300b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:52.441 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.414+0000 7fcd2f59e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd300b2710 0x7fcd30156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:52.442 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.414+0000 7fcd2f59e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd300a9c40 con 0x7fcd300b2710 2022-08-01T20:44:52.443 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.414+0000 7fcd2f59e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd300abe20 con 0x7fcd300afd60 2022-08-01T20:44:52.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.414+0000 7fcd2f59e700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd300a9700 con 0x7fcd300b01a0 2022-08-01T20:44:52.444 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.414+0000 7fcd2ed9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd300b2710 0x7fcd30156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:52.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.414+0000 7fcd2ed9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd300b2710 0x7fcd30156a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46206/0 (socket says 172.21.15.17:46206) 2022-08-01T20:44:52.445 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.414+0000 7fcd2dd9b700 1 --2- 172.21.15.17:0/2912846506 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcd300b01a0 0x7fcd300b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:52.446 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.414+0000 7fcd2ed9d700 1 -- 172.21.15.17:0/2912846506 learned_addr learned my addr 172.21.15.17:0/2912846506 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T20:44:52.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.414+0000 7fcd2e59c700 1 --2- 172.21.15.17:0/2912846506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd300afd60 0x7fcd300ab930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:52.447 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.414+0000 7fcd2ed9d700 1 -- 172.21.15.17:0/2912846506 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcd300b01a0 msgr2=0x7fcd300b40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:52.448 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.414+0000 7fcd2ed9d700 1 --2- 172.21.15.17:0/2912846506 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcd300b01a0 0x7fcd300b40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:52.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.414+0000 7fcd2ed9d700 1 -- 172.21.15.17:0/2912846506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd300afd60 msgr2=0x7fcd300ab930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:52.449 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.414+0000 7fcd2ed9d700 1 --2- 172.21.15.17:0/2912846506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd300afd60 0x7fcd300ab930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:52.450 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.414+0000 7fcd2ed9d700 1 -- 172.21.15.17:0/2912846506 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd20009530 con 0x7fcd300b2710 2022-08-01T20:44:52.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.414+0000 7fcd2ed9d700 1 --2- 172.21.15.17:0/2912846506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd300b2710 0x7fcd30156a30 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 rx=0x7fcd300abfb0 tx=0x7fcd28008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:52.451 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.414+0000 7fcd1f7fe700 1 -- 172.21.15.17:0/2912846506 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd28011780 con 0x7fcd300b2710 2022-08-01T20:44:52.452 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.414+0000 7fcd1f7fe700 1 -- 172.21.15.17:0/2912846506 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd280118e0 con 0x7fcd300b2710 2022-08-01T20:44:52.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.414+0000 7fcd2f59e700 1 -- 172.21.15.17:0/2912846506 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd300a99a0 con 0x7fcd300b2710 2022-08-01T20:44:52.453 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.414+0000 7fcd1f7fe700 1 -- 172.21.15.17:0/2912846506 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd28011c40 con 0x7fcd300b2710 2022-08-01T20:44:52.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.414+0000 7fcd2f59e700 1 -- 172.21.15.17:0/2912846506 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd300aa0a0 con 0x7fcd300b2710 2022-08-01T20:44:52.454 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.414+0000 7fcd2f59e700 1 -- 172.21.15.17:0/2912846506 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd300574e0 con 0x7fcd300b2710 2022-08-01T20:44:52.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.414+0000 7fcd1f7fe700 1 -- 172.21.15.17:0/2912846506 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcd28012420 con 0x7fcd300b2710 2022-08-01T20:44:52.455 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.414+0000 7fcd1f7fe700 1 --2- 172.21.15.17:0/2912846506 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcd0c053a70 0x7fcd0c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:52.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.418+0000 7fcd1f7fe700 1 -- 172.21.15.17:0/2912846506 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fcd2806baf0 con 0x7fcd300b2710 2022-08-01T20:44:52.456 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.418+0000 7fcd2e59c700 1 --2- 172.21.15.17:0/2912846506 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcd0c053a70 0x7fcd0c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:52.458 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.418+0000 7fcd2e59c700 1 --2- 172.21.15.17:0/2912846506 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcd0c053a70 0x7fcd0c055f30 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7fcd2400c000 tx=0x7fcd24008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:52.459 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.426+0000 7fcd1f7fe700 1 -- 172.21.15.17:0/2912846506 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcd28042b30 con 0x7fcd300b2710 2022-08-01T20:44:52.554 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.550+0000 7fcd2f59e700 1 -- 172.21.15.17:0/2912846506 --> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd300a9580 con 0x7fcd0c053a70 2022-08-01T20:44:52.555 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.554+0000 7fcd1f7fe700 1 -- 172.21.15.17:0/2912846506 <== mgr.4101 v2:172.21.15.77:6800/23151 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fcd240123f0 con 0x7fcd0c053a70 2022-08-01T20:44:52.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.554+0000 7fcd2f59e700 1 -- 172.21.15.17:0/2912846506 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcd0c053a70 msgr2=0x7fcd0c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:52.558 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.554+0000 7fcd2f59e700 1 --2- 172.21.15.17:0/2912846506 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcd0c053a70 0x7fcd0c055f30 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7fcd2400c000 tx=0x7fcd24008040).stop 2022-08-01T20:44:52.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.554+0000 7fcd2f59e700 1 -- 172.21.15.17:0/2912846506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd300b2710 msgr2=0x7fcd30156a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:52.559 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.554+0000 7fcd2f59e700 1 --2- 172.21.15.17:0/2912846506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd300b2710 0x7fcd30156a30 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 rx=0x7fcd300abfb0 tx=0x7fcd28008040).stop 2022-08-01T20:44:52.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.554+0000 7fcd2f59e700 1 -- 172.21.15.17:0/2912846506 shutdown_connections 2022-08-01T20:44:52.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.554+0000 7fcd2f59e700 1 --2- 172.21.15.17:0/2912846506 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcd0c053a70 0x7fcd0c055f30 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:52.560 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.554+0000 7fcd2f59e700 1 --2- 172.21.15.17:0/2912846506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd300b2710 0x7fcd30156a30 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:52.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.554+0000 7fcd2f59e700 1 --2- 172.21.15.17:0/2912846506 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcd300b01a0 0x7fcd300b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:52.561 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.554+0000 7fcd2f59e700 1 --2- 172.21.15.17:0/2912846506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd300afd60 0x7fcd300ab930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:52.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.554+0000 7fcd2f59e700 1 -- 172.21.15.17:0/2912846506 >> 172.21.15.17:0/2912846506 conn(0x7fcd30060310 msgr2=0x7fcd3005ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:52.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.554+0000 7fcd2f59e700 1 -- 172.21.15.17:0/2912846506 shutdown_connections 2022-08-01T20:44:52.562 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.554+0000 7fcd2f59e700 1 -- 172.21.15.17:0/2912846506 wait complete. 2022-08-01T20:44:52.563 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 3 to deep-scrub 2022-08-01T20:44:52.572 INFO:tasks.ceph:Scrubbing osd.4 2022-08-01T20:44:52.573 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2022-08-01T20:44:52.685 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.682+0000 7f82f12b3700 1 Processor -- start 2022-08-01T20:44:52.686 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.682+0000 7f82f12b3700 1 -- start start 2022-08-01T20:44:52.699 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.686+0000 7f82f12b3700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f82ec0af550 0x7f82ec0b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:52.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.686+0000 7f82f12b3700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82ec0aa630 con 0x7f82ec060330 2022-08-01T20:44:52.700 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.686+0000 7f82f12b3700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82ec05fd10 con 0x7f82ec05c6f0 2022-08-01T20:44:52.701 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.686+0000 7f82f12b3700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82ec0b15e0 con 0x7f82ec0af550 2022-08-01T20:44:52.702 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.686+0000 7f82e3fff700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f82ec060330 0x7f82ec1445e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:38406/0 (socket says 172.21.15.17:38406) 2022-08-01T20:44:52.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.686+0000 7f82e3fff700 1 -- 172.21.15.17:0/2767085846 learned_addr learned my addr 172.21.15.17:0/2767085846 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T20:44:52.703 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.686+0000 7f82ebfff700 1 --2- 172.21.15.17:0/2767085846 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f82ec0af550 0x7f82ec0b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:52.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.686+0000 7f82eb7fe700 1 -- 172.21.15.17:0/2767085846 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 190199952 0 0) 0x7f82ec0aa630 con 0x7f82ec060330 2022-08-01T20:44:52.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.686+0000 7f82eb7fe700 1 -- 172.21.15.17:0/2767085846 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82d0003260 con 0x7f82ec060330 2022-08-01T20:44:52.704 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.686+0000 7f82eb7fe700 1 -- 172.21.15.17:0/2767085846 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2857919086 0 0) 0x7f82ec05fd10 con 0x7f82ec05c6f0 2022-08-01T20:44:52.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.686+0000 7f82eb7fe700 1 -- 172.21.15.17:0/2767085846 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82ec0aa630 con 0x7f82ec05c6f0 2022-08-01T20:44:52.705 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.686+0000 7f82eb7fe700 1 -- 172.21.15.17:0/2767085846 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2688015648 0 0) 0x7f82d0003260 con 0x7f82ec060330 2022-08-01T20:44:52.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.686+0000 7f82ebfff700 1 -- 172.21.15.17:0/2767085846 >> v1:172.21.15.77:6789/0 conn(0x7f82ec05c6f0 legacy=0x7f82ec0a8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:52.706 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.686+0000 7f82ebfff700 1 -- 172.21.15.17:0/2767085846 >> v1:172.21.15.17:6789/0 conn(0x7f82ec060330 legacy=0x7f82ec1445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:52.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.686+0000 7f82ebfff700 1 -- 172.21.15.17:0/2767085846 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82ec05f840 con 0x7f82ec0af550 2022-08-01T20:44:52.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.686+0000 7f82ebfff700 1 --2- 172.21.15.17:0/2767085846 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f82ec0af550 0x7f82ec0b0e40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f82d4001110 tx=0x7f82dc006df0).ready entity=mon.2 client_cookie=f120089a17ad13fc server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:52.707 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.686+0000 7f82eb7fe700 1 -- 172.21.15.17:0/2767085846 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82d4001660 con 0x7f82ec0af550 2022-08-01T20:44:52.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.686+0000 7f82eb7fe700 1 -- 172.21.15.17:0/2767085846 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82dc000e90 con 0x7f82ec0af550 2022-08-01T20:44:52.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.686+0000 7f82eb7fe700 1 -- 172.21.15.17:0/2767085846 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82dc012460 con 0x7f82ec0af550 2022-08-01T20:44:52.708 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.686+0000 7f82f12b3700 1 -- 172.21.15.17:0/2767085846 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f82ec0af550 msgr2=0x7f82ec0b0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:52.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.686+0000 7f82f12b3700 1 --2- 172.21.15.17:0/2767085846 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f82ec0af550 0x7f82ec0b0e40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f82d4001110 tx=0x7f82dc006df0).stop 2022-08-01T20:44:52.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.686+0000 7f82f12b3700 1 -- 172.21.15.17:0/2767085846 shutdown_connections 2022-08-01T20:44:52.709 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.686+0000 7f82f12b3700 1 --2- 172.21.15.17:0/2767085846 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f82ec0af550 0x7f82ec0b0e40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:52.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.686+0000 7f82f12b3700 1 -- 172.21.15.17:0/2767085846 >> 172.21.15.17:0/2767085846 conn(0x7f82ec0612d0 msgr2=0x7f82ec062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:52.710 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.686+0000 7f82f12b3700 1 -- 172.21.15.17:0/2767085846 shutdown_connections 2022-08-01T20:44:52.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.686+0000 7f82f12b3700 1 -- 172.21.15.17:0/2767085846 wait complete. 2022-08-01T20:44:52.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.686+0000 7f82f12b3700 1 Processor -- start 2022-08-01T20:44:52.711 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.690+0000 7f82f12b3700 1 -- start start 2022-08-01T20:44:52.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.690+0000 7f82f12b3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82ec05c6f0 0x7f82ec063e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:52.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.690+0000 7f82f12b3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f82ec060330 0x7f82ec143db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:52.712 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.690+0000 7f82f12b3700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f82ec0af550 0x7f82ec067370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:52.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.690+0000 7f82f12b3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ec064440 con 0x7f82ec05c6f0 2022-08-01T20:44:52.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.690+0000 7f82f12b3700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ec1447a0 con 0x7f82ec060330 2022-08-01T20:44:52.713 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.690+0000 7f82f12b3700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82ec0654d0 con 0x7f82ec0af550 2022-08-01T20:44:52.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.690+0000 7f82ebfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82ec05c6f0 0x7f82ec063e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:52.714 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.690+0000 7f82ebfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82ec05c6f0 0x7f82ec063e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46216/0 (socket says 172.21.15.17:46216) 2022-08-01T20:44:52.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.690+0000 7f82ebfff700 1 -- 172.21.15.17:0/2700317691 learned_addr learned my addr 172.21.15.17:0/2700317691 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T20:44:52.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.690+0000 7f82e3fff700 1 --2- 172.21.15.17:0/2700317691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f82ec060330 0x7f82ec143db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:52.715 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.690+0000 7f82f0ab2700 1 --2- 172.21.15.17:0/2700317691 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f82ec0af550 0x7f82ec067370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:52.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.690+0000 7f82ebfff700 1 -- 172.21.15.17:0/2700317691 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f82ec0af550 msgr2=0x7f82ec067370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:52.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.690+0000 7f82ebfff700 1 --2- 172.21.15.17:0/2700317691 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f82ec0af550 0x7f82ec067370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:52.716 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.690+0000 7f82ebfff700 1 -- 172.21.15.17:0/2700317691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f82ec060330 msgr2=0x7f82ec143db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:52.717 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.690+0000 7f82ebfff700 1 --2- 172.21.15.17:0/2700317691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f82ec060330 0x7f82ec143db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:52.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.690+0000 7f82ebfff700 1 -- 172.21.15.17:0/2700317691 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82dc00b180 con 0x7f82ec05c6f0 2022-08-01T20:44:52.718 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.690+0000 7f82ebfff700 1 --2- 172.21.15.17:0/2700317691 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82ec05c6f0 0x7f82ec063e10 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 rx=0x7f82d4001110 tx=0x7f82dc014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:52.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.690+0000 7f82e9ffb700 1 -- 172.21.15.17:0/2700317691 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82dc00da20 con 0x7f82ec05c6f0 2022-08-01T20:44:52.719 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.690+0000 7f82e9ffb700 1 -- 172.21.15.17:0/2700317691 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82dc00db80 con 0x7f82ec05c6f0 2022-08-01T20:44:52.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.690+0000 7f82f12b3700 1 -- 172.21.15.17:0/2700317691 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82ec065770 con 0x7f82ec05c6f0 2022-08-01T20:44:52.720 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.690+0000 7f82e9ffb700 1 -- 172.21.15.17:0/2700317691 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82dc01e400 con 0x7f82ec05c6f0 2022-08-01T20:44:52.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.690+0000 7f82f12b3700 1 -- 172.21.15.17:0/2700317691 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82ec065f30 con 0x7f82ec05c6f0 2022-08-01T20:44:52.721 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.690+0000 7f82f12b3700 1 -- 172.21.15.17:0/2700317691 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f82ec065c10 con 0x7f82ec05c6f0 2022-08-01T20:44:52.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.694+0000 7f82e9ffb700 1 -- 172.21.15.17:0/2700317691 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f82dc01c070 con 0x7f82ec05c6f0 2022-08-01T20:44:52.722 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.694+0000 7f82e9ffb700 1 --2- 172.21.15.17:0/2700317691 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f82c4053a70 0x7f82c4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:52.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.694+0000 7f82e9ffb700 1 -- 172.21.15.17:0/2700317691 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f82dc06c8f0 con 0x7f82ec05c6f0 2022-08-01T20:44:52.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.694+0000 7f82e9ffb700 1 --2- 172.21.15.17:0/2700317691 >> [v2:172.21.15.77:6810/23584,v1:172.21.15.77:6811/23584] conn(0x7f82c405a700 0x7f82c405cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:52.723 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.694+0000 7f82e9ffb700 1 -- 172.21.15.17:0/2700317691 --> [v2:172.21.15.77:6810/23584,v1:172.21.15.77:6811/23584] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f82c405d180 con 0x7f82c405a700 2022-08-01T20:44:52.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.694+0000 7f82e9ffb700 1 -- 172.21.15.17:0/2700317691 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=161) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82dc071220 con 0x7f82ec05c6f0 2022-08-01T20:44:52.724 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.694+0000 7f82e3fff700 1 --2- 172.21.15.17:0/2700317691 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f82c4053a70 0x7f82c4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:52.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.694+0000 7f82f0ab2700 1 --2- 172.21.15.17:0/2700317691 >> [v2:172.21.15.77:6810/23584,v1:172.21.15.77:6811/23584] conn(0x7f82c405a700 0x7f82c405cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:52.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.694+0000 7f82e3fff700 1 --2- 172.21.15.17:0/2700317691 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f82c4053a70 0x7f82c4055f30 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7f82d400b000 tx=0x7f82d4000bc0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:52.725 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.694+0000 7f82f0ab2700 1 --2- 172.21.15.17:0/2700317691 >> [v2:172.21.15.77:6810/23584,v1:172.21.15.77:6811/23584] conn(0x7f82c405a700 0x7f82c405cba0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:52.726 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.698+0000 7f82e9ffb700 1 -- 172.21.15.17:0/2700317691 <== osd.4 v2:172.21.15.77:6810/23584 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f82c405d180 con 0x7f82c405a700 2022-08-01T20:44:52.727 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.714+0000 7f82f12b3700 1 -- 172.21.15.17:0/2700317691 --> [v2:172.21.15.77:6810/23584,v1:172.21.15.77:6811/23584] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f82ec0574e0 con 0x7f82c405a700 2022-08-01T20:44:52.728 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.714+0000 7f82e9ffb700 1 -- 172.21.15.17:0/2700317691 <== osd.4 v2:172.21.15.77:6810/23584 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f82ec0574e0 con 0x7f82c405a700 2022-08-01T20:44:52.729 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T20:44:52.730 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-08-01T20:44:52.730 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T20:44:52.734 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.718+0000 7f82f12b3700 1 -- 172.21.15.17:0/2700317691 >> [v2:172.21.15.77:6810/23584,v1:172.21.15.77:6811/23584] conn(0x7f82c405a700 msgr2=0x7f82c405cba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:52.735 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.718+0000 7f82f12b3700 1 --2- 172.21.15.17:0/2700317691 >> [v2:172.21.15.77:6810/23584,v1:172.21.15.77:6811/23584] conn(0x7f82c405a700 0x7f82c405cba0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:52.736 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.718+0000 7f82f12b3700 1 -- 172.21.15.17:0/2700317691 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f82c4053a70 msgr2=0x7f82c4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:52.737 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.718+0000 7f82f12b3700 1 --2- 172.21.15.17:0/2700317691 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f82c4053a70 0x7f82c4055f30 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7f82d400b000 tx=0x7f82d4000bc0).stop 2022-08-01T20:44:52.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.718+0000 7f82f12b3700 1 -- 172.21.15.17:0/2700317691 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82ec05c6f0 msgr2=0x7f82ec063e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:52.738 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.718+0000 7f82f12b3700 1 --2- 172.21.15.17:0/2700317691 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82ec05c6f0 0x7f82ec063e10 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 rx=0x7f82d4001110 tx=0x7f82dc014000).stop 2022-08-01T20:44:52.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.718+0000 7f82f0ab2700 1 -- 172.21.15.17:0/2700317691 reap_dead start 2022-08-01T20:44:52.739 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.718+0000 7f82f12b3700 1 -- 172.21.15.17:0/2700317691 shutdown_connections 2022-08-01T20:44:52.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.718+0000 7f82f12b3700 1 -- 172.21.15.17:0/2700317691 >> 172.21.15.17:0/2700317691 conn(0x7f82ec0612d0 msgr2=0x7f82ec141140 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:52.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.718+0000 7f82f12b3700 1 -- 172.21.15.17:0/2700317691 shutdown_connections 2022-08-01T20:44:52.740 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.718+0000 7f82f12b3700 1 -- 172.21.15.17:0/2700317691 wait complete. 2022-08-01T20:44:52.741 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2022-08-01T20:44:52.851 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.850+0000 7f24bffff700 1 Processor -- start 2022-08-01T20:44:52.852 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.850+0000 7f24bffff700 1 -- start start 2022-08-01T20:44:52.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.850+0000 7f24bffff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24c0061890 0x7f24c01466b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:52.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.850+0000 7f24bffff700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24c00aee80 con 0x7f24c00614b0 2022-08-01T20:44:52.859 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.850+0000 7f24bffff700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24c00b70b0 con 0x7f24c00ad750 2022-08-01T20:44:52.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.850+0000 7f24bffff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c00af0b0 con 0x7f24c0061890 2022-08-01T20:44:52.860 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.850+0000 7f24beffd700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f24c00614b0 0x7f24c00b9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:38420/0 (socket says 172.21.15.17:38420) 2022-08-01T20:44:52.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.850+0000 7f24beffd700 1 -- 172.21.15.17:0/2691439082 learned_addr learned my addr 172.21.15.17:0/2691439082 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T20:44:52.861 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.850+0000 7f24bf7fe700 1 --2- 172.21.15.17:0/2691439082 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24c0061890 0x7f24c01466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:52.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.850+0000 7f24bdffb700 1 -- 172.21.15.17:0/2691439082 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1323426235 0 0) 0x7f24c00aee80 con 0x7f24c00614b0 2022-08-01T20:44:52.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.850+0000 7f24bdffb700 1 -- 172.21.15.17:0/2691439082 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24a4003260 con 0x7f24c00614b0 2022-08-01T20:44:52.862 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.850+0000 7f24bdffb700 1 -- 172.21.15.17:0/2691439082 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3425683784 0 0) 0x7f24c00b70b0 con 0x7f24c00ad750 2022-08-01T20:44:52.863 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.850+0000 7f24bdffb700 1 -- 172.21.15.17:0/2691439082 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24c00aee80 con 0x7f24c00ad750 2022-08-01T20:44:52.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.850+0000 7f24bf7fe700 1 -- 172.21.15.17:0/2691439082 >> v1:172.21.15.77:6789/0 conn(0x7f24c00ad750 legacy=0x7f24c01405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:52.864 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.850+0000 7f24bf7fe700 1 -- 172.21.15.17:0/2691439082 >> v1:172.21.15.17:6789/0 conn(0x7f24c00614b0 legacy=0x7f24c00b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:52.865 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.850+0000 7f24bf7fe700 1 -- 172.21.15.17:0/2691439082 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c00af350 con 0x7f24c0061890 2022-08-01T20:44:52.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.850+0000 7f24bf7fe700 1 --2- 172.21.15.17:0/2691439082 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24c0061890 0x7f24c01466b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f24b8002f70 tx=0x7f24b8006df0).ready entity=mon.2 client_cookie=4286bd4e2dd2ed7 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:52.866 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.850+0000 7f24bdffb700 1 -- 172.21.15.17:0/2691439082 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24b800d5c0 con 0x7f24c0061890 2022-08-01T20:44:52.867 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.850+0000 7f24bdffb700 1 -- 172.21.15.17:0/2691439082 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b8000e90 con 0x7f24c0061890 2022-08-01T20:44:52.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.850+0000 7f24bdffb700 1 -- 172.21.15.17:0/2691439082 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24b800d910 con 0x7f24c0061890 2022-08-01T20:44:52.868 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.854+0000 7f24bffff700 1 -- 172.21.15.17:0/2691439082 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24c0061890 msgr2=0x7f24c01466b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:52.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.854+0000 7f24bffff700 1 --2- 172.21.15.17:0/2691439082 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24c0061890 0x7f24c01466b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f24b8002f70 tx=0x7f24b8006df0).stop 2022-08-01T20:44:52.869 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.854+0000 7f24bffff700 1 -- 172.21.15.17:0/2691439082 shutdown_connections 2022-08-01T20:44:52.870 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.854+0000 7f24bffff700 1 --2- 172.21.15.17:0/2691439082 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24c0061890 0x7f24c01466b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:52.870 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.854+0000 7f24bffff700 1 -- 172.21.15.17:0/2691439082 >> 172.21.15.17:0/2691439082 conn(0x7f24c0066f70 msgr2=0x7f24c0067350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:52.871 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.854+0000 7f24bffff700 1 -- 172.21.15.17:0/2691439082 shutdown_connections 2022-08-01T20:44:52.871 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.854+0000 7f24bffff700 1 -- 172.21.15.17:0/2691439082 wait complete. 2022-08-01T20:44:52.871 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.854+0000 7f24bffff700 1 Processor -- start 2022-08-01T20:44:52.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.854+0000 7f24bffff700 1 -- start start 2022-08-01T20:44:52.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.854+0000 7f24bffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24c00614b0 0x7f24c0146180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:52.872 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.854+0000 7f24bffff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24c0061890 0x7f24c00aa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:52.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.854+0000 7f24bffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24c00ad750 0x7f24c00aa690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:52.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.854+0000 7f24bffff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c0150190 con 0x7f24c00614b0 2022-08-01T20:44:52.873 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.854+0000 7f24bffff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c00aab80 con 0x7f24c00ad750 2022-08-01T20:44:52.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.854+0000 7f24bffff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c00b1e70 con 0x7f24c0061890 2022-08-01T20:44:52.874 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.854+0000 7f24beffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24c00614b0 0x7f24c0146180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:52.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.854+0000 7f24beffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24c00614b0 0x7f24c0146180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46232/0 (socket says 172.21.15.17:46232) 2022-08-01T20:44:52.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.854+0000 7f24beffd700 1 -- 172.21.15.17:0/2982499641 learned_addr learned my addr 172.21.15.17:0/2982499641 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T20:44:52.875 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.854+0000 7f24bf7fe700 1 --2- 172.21.15.17:0/2982499641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24c00ad750 0x7f24c00aa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:52.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.854+0000 7f24be7fc700 1 --2- 172.21.15.17:0/2982499641 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24c0061890 0x7f24c00aa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:52.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.854+0000 7f24beffd700 1 -- 172.21.15.17:0/2982499641 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24c0061890 msgr2=0x7f24c00aa1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:52.876 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.854+0000 7f24beffd700 1 --2- 172.21.15.17:0/2982499641 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24c0061890 0x7f24c00aa1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:52.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.854+0000 7f24beffd700 1 -- 172.21.15.17:0/2982499641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24c00ad750 msgr2=0x7f24c00aa690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:52.877 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.854+0000 7f24beffd700 1 --2- 172.21.15.17:0/2982499641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24c00ad750 0x7f24c00aa690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:52.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.854+0000 7f24beffd700 1 -- 172.21.15.17:0/2982499641 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b800b180 con 0x7f24c00614b0 2022-08-01T20:44:52.878 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.854+0000 7f24beffd700 1 --2- 172.21.15.17:0/2982499641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24c00614b0 0x7f24c0146180 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 rx=0x7f24c00b1dc0 tx=0x7f24a8007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:52.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.854+0000 7f24b7fff700 1 -- 172.21.15.17:0/2982499641 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24a80070d0 con 0x7f24c00614b0 2022-08-01T20:44:52.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.854+0000 7f24b7fff700 1 -- 172.21.15.17:0/2982499641 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a8007230 con 0x7f24c00614b0 2022-08-01T20:44:52.879 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.854+0000 7f24bffff700 1 -- 172.21.15.17:0/2982499641 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24c0153270 con 0x7f24c00614b0 2022-08-01T20:44:52.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.854+0000 7f24b7fff700 1 -- 172.21.15.17:0/2982499641 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24a8005610 con 0x7f24c00614b0 2022-08-01T20:44:52.880 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.854+0000 7f24bffff700 1 -- 172.21.15.17:0/2982499641 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24c00b2340 con 0x7f24c00614b0 2022-08-01T20:44:52.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.858+0000 7f24bffff700 1 -- 172.21.15.17:0/2982499641 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24c00b2540 con 0x7f24c00614b0 2022-08-01T20:44:52.882 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.858+0000 7f24b7fff700 1 -- 172.21.15.17:0/2982499641 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f24a801a060 con 0x7f24c00614b0 2022-08-01T20:44:52.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.858+0000 7f24b7fff700 1 --2- 172.21.15.17:0/2982499641 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2498053980 0x7f2498055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:52.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.858+0000 7f24b7fff700 1 -- 172.21.15.17:0/2982499641 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f24a806b5a0 con 0x7f24c00614b0 2022-08-01T20:44:52.883 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.858+0000 7f24be7fc700 1 --2- 172.21.15.17:0/2982499641 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2498053980 0x7f2498055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:52.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.862+0000 7f24be7fc700 1 --2- 172.21.15.17:0/2982499641 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2498053980 0x7f2498055e40 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7f24b000d000 tx=0x7f24b0007d80).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:52.884 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.862+0000 7f24b7fff700 1 -- 172.21.15.17:0/2982499641 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f24c00b2540 con 0x7f24c00614b0 2022-08-01T20:44:52.988 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.986+0000 7f24bffff700 1 -- 172.21.15.17:0/2982499641 --> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f24c0153400 con 0x7f2498053980 2022-08-01T20:44:52.990 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.986+0000 7f24b7fff700 1 -- 172.21.15.17:0/2982499641 <== mgr.4101 v2:172.21.15.77:6800/23151 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f24b0003030 con 0x7f2498053980 2022-08-01T20:44:52.992 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.990+0000 7f24bffff700 1 -- 172.21.15.17:0/2982499641 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2498053980 msgr2=0x7f2498055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:52.992 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.990+0000 7f24bffff700 1 --2- 172.21.15.17:0/2982499641 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2498053980 0x7f2498055e40 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7f24b000d000 tx=0x7f24b0007d80).stop 2022-08-01T20:44:52.992 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.990+0000 7f24bffff700 1 -- 172.21.15.17:0/2982499641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24c00614b0 msgr2=0x7f24c0146180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:52.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.990+0000 7f24bffff700 1 --2- 172.21.15.17:0/2982499641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24c00614b0 0x7f24c0146180 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 rx=0x7f24c00b1dc0 tx=0x7f24a8007b50).stop 2022-08-01T20:44:52.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.990+0000 7f24bffff700 1 -- 172.21.15.17:0/2982499641 shutdown_connections 2022-08-01T20:44:52.993 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.990+0000 7f24bffff700 1 --2- 172.21.15.17:0/2982499641 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2498053980 0x7f2498055e40 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:52.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.990+0000 7f24bffff700 1 --2- 172.21.15.17:0/2982499641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24c00ad750 0x7f24c00aa690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:52.994 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.990+0000 7f24bffff700 1 --2- 172.21.15.17:0/2982499641 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f24c0061890 0x7f24c00aa1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:52.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.990+0000 7f24bffff700 1 --2- 172.21.15.17:0/2982499641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24c00614b0 0x7f24c0146180 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:52.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.990+0000 7f24bffff700 1 -- 172.21.15.17:0/2982499641 >> 172.21.15.17:0/2982499641 conn(0x7f24c0066f70 msgr2=0x7f24c0149110 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:52.995 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.990+0000 7f24bffff700 1 -- 172.21.15.17:0/2982499641 shutdown_connections 2022-08-01T20:44:52.996 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:52.990+0000 7f24bffff700 1 -- 172.21.15.17:0/2982499641 wait complete. 2022-08-01T20:44:52.996 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 4 to deep-scrub 2022-08-01T20:44:53.003 INFO:tasks.ceph:Scrubbing osd.5 2022-08-01T20:44:53.005 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2022-08-01T20:44:53.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.118+0000 7f0818fd9700 1 Processor -- start 2022-08-01T20:44:53.127 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.118+0000 7f0818fd9700 1 -- start start 2022-08-01T20:44:53.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.118+0000 7f0818fd9700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f08140b6710 0x7f08140b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:53.128 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.118+0000 7f0818fd9700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0814144240 con 0x7f08140b4550 2022-08-01T20:44:53.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.118+0000 7f0818fd9700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0814143d00 con 0x7f08140b6af0 2022-08-01T20:44:53.129 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.118+0000 7f0818fd9700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0814143fa0 con 0x7f08140b6710 2022-08-01T20:44:53.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.118+0000 7f0811d9b700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f08140b4550 0x7f0814061ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:38436/0 (socket says 172.21.15.17:38436) 2022-08-01T20:44:53.130 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.118+0000 7f0811d9b700 1 -- 172.21.15.17:0/2200877587 learned_addr learned my addr 172.21.15.17:0/2200877587 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T20:44:53.131 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.122+0000 7f081259c700 1 --2- 172.21.15.17:0/2200877587 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f08140b6710 0x7f08140b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:53.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.122+0000 7f081159a700 1 -- 172.21.15.17:0/2200877587 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3327730829 0 0) 0x7f0814144240 con 0x7f08140b4550 2022-08-01T20:44:53.132 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.122+0000 7f081159a700 1 -- 172.21.15.17:0/2200877587 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07f8003260 con 0x7f08140b4550 2022-08-01T20:44:53.141 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.122+0000 7f081159a700 1 -- 172.21.15.17:0/2200877587 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1865223811 0 0) 0x7f0814143d00 con 0x7f08140b6af0 2022-08-01T20:44:53.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.122+0000 7f081159a700 1 -- 172.21.15.17:0/2200877587 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0814144240 con 0x7f08140b6af0 2022-08-01T20:44:53.142 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.122+0000 7f081259c700 1 -- 172.21.15.17:0/2200877587 >> v1:172.21.15.77:6789/0 conn(0x7f08140b6af0 legacy=0x7f081412bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:53.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.122+0000 7f081259c700 1 -- 172.21.15.17:0/2200877587 >> v1:172.21.15.17:6789/0 conn(0x7f08140b4550 legacy=0x7f0814061ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:53.143 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.122+0000 7f081259c700 1 -- 172.21.15.17:0/2200877587 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0814061780 con 0x7f08140b6710 2022-08-01T20:44:53.144 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.122+0000 7f081259c700 1 --2- 172.21.15.17:0/2200877587 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f08140b6710 0x7f08140b5d80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f0804002f70 tx=0x7f0804006df0).ready entity=mon.2 client_cookie=874172b816a1a778 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:53.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.122+0000 7f081159a700 1 -- 172.21.15.17:0/2200877587 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f080400d5c0 con 0x7f08140b6710 2022-08-01T20:44:53.145 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.122+0000 7f081159a700 1 -- 172.21.15.17:0/2200877587 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0804000e90 con 0x7f08140b6710 2022-08-01T20:44:53.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.122+0000 7f081159a700 1 -- 172.21.15.17:0/2200877587 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f080400d910 con 0x7f08140b6710 2022-08-01T20:44:53.146 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.122+0000 7f0818fd9700 1 -- 172.21.15.17:0/2200877587 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f08140b6710 msgr2=0x7f08140b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:53.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.122+0000 7f0818fd9700 1 --2- 172.21.15.17:0/2200877587 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f08140b6710 0x7f08140b5d80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f0804002f70 tx=0x7f0804006df0).stop 2022-08-01T20:44:53.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.122+0000 7f0818fd9700 1 -- 172.21.15.17:0/2200877587 shutdown_connections 2022-08-01T20:44:53.147 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.122+0000 7f0818fd9700 1 --2- 172.21.15.17:0/2200877587 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f08140b6710 0x7f08140b5d80 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:53.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.122+0000 7f0818fd9700 1 -- 172.21.15.17:0/2200877587 >> 172.21.15.17:0/2200877587 conn(0x7f081405c620 msgr2=0x7f081405ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:53.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.122+0000 7f0818fd9700 1 -- 172.21.15.17:0/2200877587 shutdown_connections 2022-08-01T20:44:53.148 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.122+0000 7f0818fd9700 1 -- 172.21.15.17:0/2200877587 wait complete. 2022-08-01T20:44:53.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.126+0000 7f0818fd9700 1 Processor -- start 2022-08-01T20:44:53.149 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.126+0000 7f0818fd9700 1 -- start start 2022-08-01T20:44:53.156 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.126+0000 7f0818fd9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08140b4550 0x7f0814145880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:53.157 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.126+0000 7f0818fd9700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f08140b6710 0x7f0814155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:53.158 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.126+0000 7f0818fd9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08140b6af0 0x7f0814156270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:53.158 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.126+0000 7f0818fd9700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0814156760 con 0x7f08140b6af0 2022-08-01T20:44:53.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.126+0000 7f0818fd9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08140b8530 con 0x7f08140b4550 2022-08-01T20:44:53.159 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.126+0000 7f0818fd9700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08140b87d0 con 0x7f08140b6710 2022-08-01T20:44:53.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.126+0000 7f0812d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08140b6af0 0x7f0814156270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:53.160 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.126+0000 7f0812d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08140b6af0 0x7f0814156270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46250/0 (socket says 172.21.15.17:46250) 2022-08-01T20:44:53.161 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.126+0000 7f0812d9d700 1 -- 172.21.15.17:0/4086408936 learned_addr learned my addr 172.21.15.17:0/4086408936 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T20:44:53.162 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.126+0000 7f081259c700 1 --2- 172.21.15.17:0/4086408936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08140b4550 0x7f0814145880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:53.162 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.126+0000 7f0811d9b700 1 --2- 172.21.15.17:0/4086408936 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f08140b6710 0x7f0814155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:53.163 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.126+0000 7f0812d9d700 1 -- 172.21.15.17:0/4086408936 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f08140b6710 msgr2=0x7f0814155d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:53.163 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.126+0000 7f0812d9d700 1 --2- 172.21.15.17:0/4086408936 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f08140b6710 0x7f0814155d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:53.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.126+0000 7f0812d9d700 1 -- 172.21.15.17:0/4086408936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08140b4550 msgr2=0x7f0814145880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:53.164 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.126+0000 7f0812d9d700 1 --2- 172.21.15.17:0/4086408936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08140b4550 0x7f0814145880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:53.165 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.126+0000 7f0812d9d700 1 -- 172.21.15.17:0/4086408936 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f080400b180 con 0x7f08140b6af0 2022-08-01T20:44:53.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.126+0000 7f0812d9d700 1 --2- 172.21.15.17:0/4086408936 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08140b6af0 0x7f0814156270 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 rx=0x7f080c001490 tx=0x7f080c007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:53.166 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.126+0000 7f08037fe700 1 -- 172.21.15.17:0/4086408936 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f080c002310 con 0x7f08140b6af0 2022-08-01T20:44:53.167 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.126+0000 7f08037fe700 1 -- 172.21.15.17:0/4086408936 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f080c002470 con 0x7f08140b6af0 2022-08-01T20:44:53.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.126+0000 7f0818fd9700 1 -- 172.21.15.17:0/4086408936 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08141558e0 con 0x7f08140b6af0 2022-08-01T20:44:53.168 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.126+0000 7f08037fe700 1 -- 172.21.15.17:0/4086408936 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f080c0017a0 con 0x7f08140b6af0 2022-08-01T20:44:53.169 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.130+0000 7f0818fd9700 1 -- 172.21.15.17:0/4086408936 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08140b8a70 con 0x7f08140b6af0 2022-08-01T20:44:53.170 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.130+0000 7f0818fd9700 1 -- 172.21.15.17:0/4086408936 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0814155a70 con 0x7f08140b6af0 2022-08-01T20:44:53.170 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.130+0000 7f08037fe700 1 -- 172.21.15.17:0/4086408936 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f080c010030 con 0x7f08140b6af0 2022-08-01T20:44:53.171 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.130+0000 7f08037fe700 1 --2- 172.21.15.17:0/4086408936 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f07f0053aa0 0x7f07f0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:53.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.130+0000 7f08037fe700 1 -- 172.21.15.17:0/4086408936 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f080c06ba00 con 0x7f08140b6af0 2022-08-01T20:44:53.172 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.130+0000 7f08037fe700 1 --2- 172.21.15.17:0/4086408936 >> [v2:172.21.15.77:6802/23585,v1:172.21.15.77:6803/23585] conn(0x7f07f005a730 0x7f07f005cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:53.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.130+0000 7f08037fe700 1 -- 172.21.15.17:0/4086408936 --> [v2:172.21.15.77:6802/23585,v1:172.21.15.77:6803/23585] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f07f005d1b0 con 0x7f07f005a730 2022-08-01T20:44:53.173 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.130+0000 7f08037fe700 1 -- 172.21.15.17:0/4086408936 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=161) v2 ==== 24+0+0 (secure 0 0 0) 0x7f080c070240 con 0x7f08140b6af0 2022-08-01T20:44:53.174 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.130+0000 7f081259c700 1 --2- 172.21.15.17:0/4086408936 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f07f0053aa0 0x7f07f0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:53.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.130+0000 7f0811d9b700 1 --2- 172.21.15.17:0/4086408936 >> [v2:172.21.15.77:6802/23585,v1:172.21.15.77:6803/23585] conn(0x7f07f005a730 0x7f07f005cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:53.175 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.134+0000 7f081259c700 1 --2- 172.21.15.17:0/4086408936 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f07f0053aa0 0x7f07f0055f60 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7f0804018000 tx=0x7f0804014000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:53.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.134+0000 7f0811d9b700 1 --2- 172.21.15.17:0/4086408936 >> [v2:172.21.15.77:6802/23585,v1:172.21.15.77:6803/23585] conn(0x7f07f005a730 0x7f07f005cbd0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:53.176 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.134+0000 7f08037fe700 1 -- 172.21.15.17:0/4086408936 <== osd.5 v2:172.21.15.77:6802/23585 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f07f005d1b0 con 0x7f07f005a730 2022-08-01T20:44:53.177 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.154+0000 7f0818fd9700 1 -- 172.21.15.17:0/4086408936 --> [v2:172.21.15.77:6802/23585,v1:172.21.15.77:6803/23585] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f08140ae4e0 con 0x7f07f005a730 2022-08-01T20:44:53.178 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.154+0000 7f08037fe700 1 -- 172.21.15.17:0/4086408936 <== osd.5 v2:172.21.15.77:6802/23585 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f08140ae4e0 con 0x7f07f005a730 2022-08-01T20:44:53.179 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T20:44:53.180 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-08-01T20:44:53.181 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T20:44:53.185 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.154+0000 7f0818fd9700 1 -- 172.21.15.17:0/4086408936 >> [v2:172.21.15.77:6802/23585,v1:172.21.15.77:6803/23585] conn(0x7f07f005a730 msgr2=0x7f07f005cbd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:53.186 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.154+0000 7f0818fd9700 1 --2- 172.21.15.17:0/4086408936 >> [v2:172.21.15.77:6802/23585,v1:172.21.15.77:6803/23585] conn(0x7f07f005a730 0x7f07f005cbd0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:53.187 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.154+0000 7f0818fd9700 1 -- 172.21.15.17:0/4086408936 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f07f0053aa0 msgr2=0x7f07f0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:53.188 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.154+0000 7f0818fd9700 1 --2- 172.21.15.17:0/4086408936 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f07f0053aa0 0x7f07f0055f60 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7f0804018000 tx=0x7f0804014000).stop 2022-08-01T20:44:53.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.154+0000 7f0818fd9700 1 -- 172.21.15.17:0/4086408936 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08140b6af0 msgr2=0x7f0814156270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:53.189 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.154+0000 7f0818fd9700 1 --2- 172.21.15.17:0/4086408936 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08140b6af0 0x7f0814156270 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 rx=0x7f080c001490 tx=0x7f080c007dd0).stop 2022-08-01T20:44:53.190 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.154+0000 7f0812d9d700 1 -- 172.21.15.17:0/4086408936 reap_dead start 2022-08-01T20:44:53.190 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.154+0000 7f0818fd9700 1 -- 172.21.15.17:0/4086408936 shutdown_connections 2022-08-01T20:44:53.191 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.154+0000 7f0818fd9700 1 -- 172.21.15.17:0/4086408936 >> 172.21.15.17:0/4086408936 conn(0x7f081405c620 msgr2=0x7f08141288e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:53.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.154+0000 7f0818fd9700 1 -- 172.21.15.17:0/4086408936 shutdown_connections 2022-08-01T20:44:53.192 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.154+0000 7f0818fd9700 1 -- 172.21.15.17:0/4086408936 wait complete. 2022-08-01T20:44:53.202 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2022-08-01T20:44:53.316 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.314+0000 7fbb86426700 1 Processor -- start 2022-08-01T20:44:53.317 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.314+0000 7fbb86426700 1 -- start start 2022-08-01T20:44:53.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.314+0000 7fbb86426700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fbb800a9320 0x7fbb801466e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:53.319 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.314+0000 7fbb86426700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbb800b0560 0x7fbb80140350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:53.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.314+0000 7fbb86426700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb800b12d0 con 0x7fbb800b0d10 2022-08-01T20:44:53.320 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.314+0000 7fbb86426700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb800a9700 con 0x7fbb800b0560 2022-08-01T20:44:53.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.314+0000 7fbb86426700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb800b1500 con 0x7fbb800a9320 2022-08-01T20:44:53.321 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.314+0000 7fbb85c25700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbb800b0560 0x7fbb80140350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:53.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.314+0000 7fbb85c25700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbb800b0560 0x7fbb80140350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46260/0 (socket says 172.21.15.17:46260) 2022-08-01T20:44:53.322 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.314+0000 7fbb85c25700 1 -- 172.21.15.17:0/2366176230 learned_addr learned my addr 172.21.15.17:0/2366176230 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T20:44:53.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.314+0000 7fbb85424700 1 --2- 172.21.15.17:0/2366176230 >> v2:172.21.15.77:3300/0 conn(0x7fbb800a9320 0x7fbb801466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:53.324 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.318+0000 7fbb85c25700 1 -- 172.21.15.17:0/2366176230 >> v2:172.21.15.77:3300/0 conn(0x7fbb800a9320 msgr2=0x7fbb801466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T20:44:53.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.318+0000 7fbb85c25700 1 --2- 172.21.15.17:0/2366176230 >> v2:172.21.15.77:3300/0 conn(0x7fbb800a9320 0x7fbb801466e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:53.325 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.318+0000 7fbb85c25700 1 -- 172.21.15.17:0/2366176230 >> v1:172.21.15.77:6789/0 conn(0x7fbb800b0d10 legacy=0x7fbb80146bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:53.326 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.318+0000 7fbb85c25700 1 -- 172.21.15.17:0/2366176230 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb800b17a0 con 0x7fbb800b0560 2022-08-01T20:44:53.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.318+0000 7fbb85c25700 1 --2- 172.21.15.17:0/2366176230 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb800b0560 0x7fbb80140350 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 rx=0x7fbb780032f0 tx=0x7fbb78009290).ready entity=mon.0 client_cookie=3cd314b4189d1b5e server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:53.327 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.318+0000 7fbb73fff700 1 -- 172.21.15.17:0/2366176230 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb7800b870 con 0x7fbb800b0560 2022-08-01T20:44:53.328 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.318+0000 7fbb73fff700 1 -- 172.21.15.17:0/2366176230 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb7800b9d0 con 0x7fbb800b0560 2022-08-01T20:44:53.329 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.318+0000 7fbb73fff700 1 -- 172.21.15.17:0/2366176230 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb7800bb40 con 0x7fbb800b0560 2022-08-01T20:44:53.334 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.318+0000 7fbb86426700 1 -- 172.21.15.17:0/2366176230 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb800b0560 msgr2=0x7fbb80140350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:53.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.318+0000 7fbb86426700 1 --2- 172.21.15.17:0/2366176230 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb800b0560 0x7fbb80140350 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 rx=0x7fbb780032f0 tx=0x7fbb78009290).stop 2022-08-01T20:44:53.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.318+0000 7fbb86426700 1 -- 172.21.15.17:0/2366176230 shutdown_connections 2022-08-01T20:44:53.335 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.318+0000 7fbb86426700 1 --2- 172.21.15.17:0/2366176230 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb800b0560 0x7fbb80140350 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:53.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.318+0000 7fbb86426700 1 --2- 172.21.15.17:0/2366176230 >> v2:172.21.15.77:3300/0 conn(0x7fbb800a9320 0x7fbb801466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:53.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.318+0000 7fbb86426700 1 -- 172.21.15.17:0/2366176230 >> 172.21.15.17:0/2366176230 conn(0x7fbb800612d0 msgr2=0x7fbb80062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:53.336 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.318+0000 7fbb86426700 1 -- 172.21.15.17:0/2366176230 shutdown_connections 2022-08-01T20:44:53.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.318+0000 7fbb86426700 1 -- 172.21.15.17:0/2366176230 wait complete. 2022-08-01T20:44:53.337 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.318+0000 7fbb86426700 1 Processor -- start 2022-08-01T20:44:53.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.322+0000 7fbb86426700 1 -- start start 2022-08-01T20:44:53.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.322+0000 7fbb86426700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb800a9320 0x7fbb80141370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:53.338 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.322+0000 7fbb86426700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb800b0560 0x7fbb800b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:53.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.322+0000 7fbb86426700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbb800b0d10 0x7fbb801524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:53.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.322+0000 7fbb86426700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb80156360 con 0x7fbb800b0560 2022-08-01T20:44:53.339 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.322+0000 7fbb86426700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb800b4860 con 0x7fbb800a9320 2022-08-01T20:44:53.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.322+0000 7fbb86426700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb801565a0 con 0x7fbb800b0d10 2022-08-01T20:44:53.340 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.322+0000 7fbb85424700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb800a9320 0x7fbb80141370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:53.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.322+0000 7fbb84c23700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb800b0560 0x7fbb800b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:53.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.322+0000 7fbb85424700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb800a9320 0x7fbb80141370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:44184/0 (socket says 172.21.15.17:44184) 2022-08-01T20:44:53.341 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.322+0000 7fbb85424700 1 -- 172.21.15.17:0/4208572201 learned_addr learned my addr 172.21.15.17:0/4208572201 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T20:44:53.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.322+0000 7fbb85c25700 1 --2- 172.21.15.17:0/4208572201 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbb800b0d10 0x7fbb801524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:53.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.322+0000 7fbb85424700 1 -- 172.21.15.17:0/4208572201 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbb800b0d10 msgr2=0x7fbb801524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:53.343 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.322+0000 7fbb85424700 1 --2- 172.21.15.17:0/4208572201 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbb800b0d10 0x7fbb801524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:53.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.322+0000 7fbb85424700 1 -- 172.21.15.17:0/4208572201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb800b0560 msgr2=0x7fbb800b4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:53.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.322+0000 7fbb85424700 1 --2- 172.21.15.17:0/4208572201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb800b0560 0x7fbb800b4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:53.344 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.322+0000 7fbb85424700 1 -- 172.21.15.17:0/4208572201 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb78009530 con 0x7fbb800a9320 2022-08-01T20:44:53.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.322+0000 7fbb84c23700 1 --2- 172.21.15.17:0/4208572201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb800b0560 0x7fbb800b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T20:44:53.345 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.322+0000 7fbb85424700 1 --2- 172.21.15.17:0/4208572201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb800a9320 0x7fbb80141370 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7fbb7400c000 tx=0x7fbb74006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:53.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.322+0000 7fbb727fc700 1 -- 172.21.15.17:0/4208572201 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb74011750 con 0x7fbb800a9320 2022-08-01T20:44:53.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.322+0000 7fbb727fc700 1 -- 172.21.15.17:0/4208572201 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb740118b0 con 0x7fbb800a9320 2022-08-01T20:44:53.346 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.322+0000 7fbb86426700 1 -- 172.21.15.17:0/4208572201 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb80156840 con 0x7fbb800a9320 2022-08-01T20:44:53.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.322+0000 7fbb727fc700 1 -- 172.21.15.17:0/4208572201 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb74011c10 con 0x7fbb800a9320 2022-08-01T20:44:53.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.322+0000 7fbb86426700 1 -- 172.21.15.17:0/4208572201 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb801438e0 con 0x7fbb800a9320 2022-08-01T20:44:53.347 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.326+0000 7fbb86426700 1 -- 172.21.15.17:0/4208572201 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb800574e0 con 0x7fbb800a9320 2022-08-01T20:44:53.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.326+0000 7fbb727fc700 1 -- 172.21.15.17:0/4208572201 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbb74011d70 con 0x7fbb800a9320 2022-08-01T20:44:53.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.326+0000 7fbb727fc700 1 --2- 172.21.15.17:0/4208572201 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbb68053a70 0x7fbb68055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:53.348 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.326+0000 7fbb727fc700 1 -- 172.21.15.17:0/4208572201 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fbb7406c240 con 0x7fbb800a9320 2022-08-01T20:44:53.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.326+0000 7fbb84c23700 1 --2- 172.21.15.17:0/4208572201 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbb68053a70 0x7fbb68055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:53.349 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.326+0000 7fbb84c23700 1 --2- 172.21.15.17:0/4208572201 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbb68053a70 0x7fbb68055f30 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7fbb7c00d000 tx=0x7fbb7c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:53.350 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.338+0000 7fbb727fc700 1 -- 172.21.15.17:0/4208572201 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbb74043240 con 0x7fbb800a9320 2022-08-01T20:44:53.468 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.466+0000 7fbb86426700 1 -- 172.21.15.17:0/4208572201 --> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb801475c0 con 0x7fbb68053a70 2022-08-01T20:44:53.470 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.466+0000 7fbb727fc700 1 -- 172.21.15.17:0/4208572201 <== mgr.4101 v2:172.21.15.77:6800/23151 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbb801475c0 con 0x7fbb68053a70 2022-08-01T20:44:53.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.470+0000 7fbb86426700 1 -- 172.21.15.17:0/4208572201 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbb68053a70 msgr2=0x7fbb68055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:53.476 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.470+0000 7fbb86426700 1 --2- 172.21.15.17:0/4208572201 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbb68053a70 0x7fbb68055f30 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7fbb7c00d000 tx=0x7fbb7c009040).stop 2022-08-01T20:44:53.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.470+0000 7fbb86426700 1 -- 172.21.15.17:0/4208572201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb800a9320 msgr2=0x7fbb80141370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:53.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.470+0000 7fbb86426700 1 --2- 172.21.15.17:0/4208572201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb800a9320 0x7fbb80141370 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7fbb7400c000 tx=0x7fbb74006df0).stop 2022-08-01T20:44:53.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.470+0000 7fbb86426700 1 -- 172.21.15.17:0/4208572201 shutdown_connections 2022-08-01T20:44:53.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.470+0000 7fbb86426700 1 --2- 172.21.15.17:0/4208572201 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fbb68053a70 0x7fbb68055f30 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:53.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.470+0000 7fbb86426700 1 --2- 172.21.15.17:0/4208572201 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fbb800b0d10 0x7fbb801524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:53.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.470+0000 7fbb86426700 1 --2- 172.21.15.17:0/4208572201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb800b0560 0x7fbb800b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:53.480 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.470+0000 7fbb86426700 1 --2- 172.21.15.17:0/4208572201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb800a9320 0x7fbb80141370 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:53.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.470+0000 7fbb86426700 1 -- 172.21.15.17:0/4208572201 >> 172.21.15.17:0/4208572201 conn(0x7fbb800612d0 msgr2=0x7fbb8013c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:53.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.470+0000 7fbb86426700 1 -- 172.21.15.17:0/4208572201 shutdown_connections 2022-08-01T20:44:53.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.470+0000 7fbb86426700 1 -- 172.21.15.17:0/4208572201 wait complete. 2022-08-01T20:44:53.482 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 5 to deep-scrub 2022-08-01T20:44:53.485 INFO:tasks.ceph:Scrubbing osd.6 2022-08-01T20:44:53.487 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2022-08-01T20:44:53.602 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.598+0000 7fd7e3fff700 1 Processor -- start 2022-08-01T20:44:53.603 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.598+0000 7fd7e3fff700 1 -- start start 2022-08-01T20:44:53.606 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.602+0000 7fd7e3fff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fd7e40b2730 0x7fd7e40666b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:53.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.602+0000 7fd7e3fff700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7e40654c0 con 0x7fd7e40b3190 2022-08-01T20:44:53.607 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.602+0000 7fd7e3fff700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7e40b3570 con 0x7fd7e4066ba0 2022-08-01T20:44:53.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.602+0000 7fd7e3fff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7e4062190 con 0x7fd7e40b2730 2022-08-01T20:44:53.608 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.602+0000 7fd7e2ffd700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd7e40b3190 0x7fd7e4060ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:38464/0 (socket says 172.21.15.17:38464) 2022-08-01T20:44:53.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.602+0000 7fd7e2ffd700 1 -- 172.21.15.17:0/394300623 learned_addr learned my addr 172.21.15.17:0/394300623 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T20:44:53.609 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.602+0000 7fd7e27fc700 1 --2- 172.21.15.17:0/394300623 >> v2:172.21.15.77:3300/0 conn(0x7fd7e40b2730 0x7fd7e40666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:53.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.602+0000 7fd7e1ffb700 1 -- 172.21.15.17:0/394300623 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 903291576 0 0) 0x7fd7e40654c0 con 0x7fd7e40b3190 2022-08-01T20:44:53.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.602+0000 7fd7e1ffb700 1 -- 172.21.15.17:0/394300623 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7c8003260 con 0x7fd7e40b3190 2022-08-01T20:44:53.610 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.602+0000 7fd7e1ffb700 1 -- 172.21.15.17:0/394300623 <== mon.1 v1:172.21.15.77:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2155803572 0 0) 0x7fd7e40b3570 con 0x7fd7e4066ba0 2022-08-01T20:44:53.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.602+0000 7fd7e1ffb700 1 -- 172.21.15.17:0/394300623 --> v1:172.21.15.77:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7e40654c0 con 0x7fd7e4066ba0 2022-08-01T20:44:53.611 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.602+0000 7fd7e27fc700 1 -- 172.21.15.17:0/394300623 >> v1:172.21.15.77:6789/0 conn(0x7fd7e4066ba0 legacy=0x7fd7e414dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:53.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.602+0000 7fd7e27fc700 1 -- 172.21.15.17:0/394300623 >> v1:172.21.15.17:6789/0 conn(0x7fd7e40b3190 legacy=0x7fd7e4060ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:53.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.602+0000 7fd7e1ffb700 1 -- 172.21.15.17:0/394300623 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2791513049 0 0) 0x7fd7c8003260 con 0x7fd7e40b3190 2022-08-01T20:44:53.612 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.602+0000 7fd7e27fc700 1 -- 172.21.15.17:0/394300623 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e405fbf0 con 0x7fd7e40b2730 2022-08-01T20:44:53.613 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.602+0000 7fd7e27fc700 1 --2- 172.21.15.17:0/394300623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7e40b2730 0x7fd7e40666b0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7fd7d8002f70 tx=0x7fd7d8006df0).ready entity=mon.1 client_cookie=726cf9e3576163e1 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:53.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.602+0000 7fd7e1ffb700 1 -- 172.21.15.17:0/394300623 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7d800d540 con 0x7fd7e40b2730 2022-08-01T20:44:53.614 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.602+0000 7fd7e1ffb700 1 -- 172.21.15.17:0/394300623 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d8000e90 con 0x7fd7e40b2730 2022-08-01T20:44:53.615 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.602+0000 7fd7e1ffb700 1 -- 172.21.15.17:0/394300623 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7d800d890 con 0x7fd7e40b2730 2022-08-01T20:44:53.623 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.606+0000 7fd7e3fff700 1 -- 172.21.15.17:0/394300623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7e40b2730 msgr2=0x7fd7e40666b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:53.624 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.606+0000 7fd7e3fff700 1 --2- 172.21.15.17:0/394300623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7e40b2730 0x7fd7e40666b0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7fd7d8002f70 tx=0x7fd7d8006df0).stop 2022-08-01T20:44:53.625 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.606+0000 7fd7e3fff700 1 -- 172.21.15.17:0/394300623 shutdown_connections 2022-08-01T20:44:53.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.606+0000 7fd7e3fff700 1 --2- 172.21.15.17:0/394300623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7e40b2730 0x7fd7e40666b0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:53.626 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.606+0000 7fd7e3fff700 1 -- 172.21.15.17:0/394300623 >> 172.21.15.17:0/394300623 conn(0x7fd7e4063870 msgr2=0x7fd7e4063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:53.627 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.606+0000 7fd7e3fff700 1 -- 172.21.15.17:0/394300623 shutdown_connections 2022-08-01T20:44:53.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.606+0000 7fd7e3fff700 1 -- 172.21.15.17:0/394300623 wait complete. 2022-08-01T20:44:53.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.606+0000 7fd7e3fff700 1 Processor -- start 2022-08-01T20:44:53.629 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.606+0000 7fd7e3fff700 1 -- start start 2022-08-01T20:44:53.630 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.610+0000 7fd7e3fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7e4066ba0 0x7fd7e4148a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:53.631 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.610+0000 7fd7e3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7e40b2730 0x7fd7e41527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:53.632 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.610+0000 7fd7e3fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd7e40b3190 0x7fd7e4152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:53.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.610+0000 7fd7e3fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e41531d0 con 0x7fd7e40b2730 2022-08-01T20:44:53.637 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.610+0000 7fd7e3fff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e40acc00 con 0x7fd7e4066ba0 2022-08-01T20:44:53.638 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.610+0000 7fd7e3fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e40acea0 con 0x7fd7e40b3190 2022-08-01T20:44:53.639 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.610+0000 7fd7e27fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7e40b2730 0x7fd7e41527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:53.639 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.610+0000 7fd7e27fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7e40b2730 0x7fd7e41527f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46278/0 (socket says 172.21.15.17:46278) 2022-08-01T20:44:53.639 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.610+0000 7fd7e27fc700 1 -- 172.21.15.17:0/3758065035 learned_addr learned my addr 172.21.15.17:0/3758065035 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T20:44:53.640 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.610+0000 7fd7e2ffd700 1 --2- 172.21.15.17:0/3758065035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7e4066ba0 0x7fd7e4148a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:53.640 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.610+0000 7fd7e37fe700 1 --2- 172.21.15.17:0/3758065035 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd7e40b3190 0x7fd7e4152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:53.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.610+0000 7fd7e27fc700 1 -- 172.21.15.17:0/3758065035 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd7e40b3190 msgr2=0x7fd7e4152ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:53.641 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.610+0000 7fd7e27fc700 1 --2- 172.21.15.17:0/3758065035 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fd7e40b3190 0x7fd7e4152ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:53.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.610+0000 7fd7e27fc700 1 -- 172.21.15.17:0/3758065035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7e4066ba0 msgr2=0x7fd7e4148a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:53.642 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.610+0000 7fd7e27fc700 1 --2- 172.21.15.17:0/3758065035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7e4066ba0 0x7fd7e4148a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:53.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.610+0000 7fd7e27fc700 1 -- 172.21.15.17:0/3758065035 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d800b180 con 0x7fd7e40b2730 2022-08-01T20:44:53.643 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.610+0000 7fd7e27fc700 1 --2- 172.21.15.17:0/3758065035 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7e40b2730 0x7fd7e41527f0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 rx=0x7fd7d8002f70 tx=0x7fd7d8014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:53.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.610+0000 7fd7d3fff700 1 -- 172.21.15.17:0/3758065035 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7d8013da0 con 0x7fd7e40b2730 2022-08-01T20:44:53.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.610+0000 7fd7e3fff700 1 -- 172.21.15.17:0/3758065035 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7e40ad140 con 0x7fd7e40b2730 2022-08-01T20:44:53.644 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.610+0000 7fd7e3fff700 1 -- 172.21.15.17:0/3758065035 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e4155850 con 0x7fd7e40b2730 2022-08-01T20:44:53.645 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.610+0000 7fd7d3fff700 1 -- 172.21.15.17:0/3758065035 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d800d1c0 con 0x7fd7e40b2730 2022-08-01T20:44:53.645 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.610+0000 7fd7d3fff700 1 -- 172.21.15.17:0/3758065035 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7d801e5f0 con 0x7fd7e40b2730 2022-08-01T20:44:53.645 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.610+0000 7fd7e3fff700 1 -- 172.21.15.17:0/3758065035 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd7e4152510 con 0x7fd7e40b2730 2022-08-01T20:44:53.646 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.614+0000 7fd7d3fff700 1 -- 172.21.15.17:0/3758065035 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd7d8025030 con 0x7fd7e40b2730 2022-08-01T20:44:53.646 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.614+0000 7fd7d3fff700 1 --2- 172.21.15.17:0/3758065035 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd7c0053a70 0x7fd7c0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:53.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.614+0000 7fd7d3fff700 1 -- 172.21.15.17:0/3758065035 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fd7d806d290 con 0x7fd7e40b2730 2022-08-01T20:44:53.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.614+0000 7fd7d3fff700 1 --2- 172.21.15.17:0/3758065035 >> [v2:172.21.15.77:6820/23587,v1:172.21.15.77:6821/23587] conn(0x7fd7c005a700 0x7fd7c005cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:53.647 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.614+0000 7fd7d3fff700 1 -- 172.21.15.17:0/3758065035 --> [v2:172.21.15.77:6820/23587,v1:172.21.15.77:6821/23587] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd7c005d180 con 0x7fd7c005a700 2022-08-01T20:44:53.648 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.614+0000 7fd7d3fff700 1 -- 172.21.15.17:0/3758065035 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=161) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7d806d650 con 0x7fd7e40b2730 2022-08-01T20:44:53.648 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.614+0000 7fd7e2ffd700 1 --2- 172.21.15.17:0/3758065035 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd7c0053a70 0x7fd7c0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:53.648 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.614+0000 7fd7e37fe700 1 --2- 172.21.15.17:0/3758065035 >> [v2:172.21.15.77:6820/23587,v1:172.21.15.77:6821/23587] conn(0x7fd7c005a700 0x7fd7c005cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:53.649 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.614+0000 7fd7e2ffd700 1 --2- 172.21.15.17:0/3758065035 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd7c0053a70 0x7fd7c0055f30 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7fd7d400d000 tx=0x7fd7d4007a00).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:53.649 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.614+0000 7fd7e37fe700 1 --2- 172.21.15.17:0/3758065035 >> [v2:172.21.15.77:6820/23587,v1:172.21.15.77:6821/23587] conn(0x7fd7c005a700 0x7fd7c005cba0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:53.650 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.614+0000 7fd7d3fff700 1 -- 172.21.15.17:0/3758065035 <== osd.6 v2:172.21.15.77:6820/23587 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7fd7c005d180 con 0x7fd7c005a700 2022-08-01T20:44:53.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.630+0000 7fd7e3fff700 1 -- 172.21.15.17:0/3758065035 --> [v2:172.21.15.77:6820/23587,v1:172.21.15.77:6821/23587] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd7e40ad930 con 0x7fd7c005a700 2022-08-01T20:44:53.651 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.630+0000 7fd7d3fff700 1 -- 172.21.15.17:0/3758065035 <== osd.6 v2:172.21.15.77:6820/23587 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fd7e40ad930 con 0x7fd7c005a700 2022-08-01T20:44:53.652 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T20:44:53.652 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-08-01T20:44:53.653 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T20:44:53.655 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.630+0000 7fd7e3fff700 1 -- 172.21.15.17:0/3758065035 >> [v2:172.21.15.77:6820/23587,v1:172.21.15.77:6821/23587] conn(0x7fd7c005a700 msgr2=0x7fd7c005cba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:53.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.630+0000 7fd7e3fff700 1 --2- 172.21.15.17:0/3758065035 >> [v2:172.21.15.77:6820/23587,v1:172.21.15.77:6821/23587] conn(0x7fd7c005a700 0x7fd7c005cba0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:53.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.630+0000 7fd7e3fff700 1 -- 172.21.15.17:0/3758065035 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd7c0053a70 msgr2=0x7fd7c0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:53.656 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.630+0000 7fd7e3fff700 1 --2- 172.21.15.17:0/3758065035 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fd7c0053a70 0x7fd7c0055f30 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7fd7d400d000 tx=0x7fd7d4007a00).stop 2022-08-01T20:44:53.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.630+0000 7fd7e3fff700 1 -- 172.21.15.17:0/3758065035 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7e40b2730 msgr2=0x7fd7e41527f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:53.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.630+0000 7fd7e3fff700 1 --2- 172.21.15.17:0/3758065035 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7e40b2730 0x7fd7e41527f0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 rx=0x7fd7d8002f70 tx=0x7fd7d8014000).stop 2022-08-01T20:44:53.657 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.630+0000 7fd7e37fe700 1 -- 172.21.15.17:0/3758065035 reap_dead start 2022-08-01T20:44:53.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.630+0000 7fd7e3fff700 1 -- 172.21.15.17:0/3758065035 shutdown_connections 2022-08-01T20:44:53.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.630+0000 7fd7e3fff700 1 -- 172.21.15.17:0/3758065035 >> 172.21.15.17:0/3758065035 conn(0x7fd7e4063870 msgr2=0x7fd7e4146530 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:53.658 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.630+0000 7fd7e3fff700 1 -- 172.21.15.17:0/3758065035 shutdown_connections 2022-08-01T20:44:53.659 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.630+0000 7fd7e3fff700 1 -- 172.21.15.17:0/3758065035 wait complete. 2022-08-01T20:44:53.659 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2022-08-01T20:44:53.768 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.766+0000 7f0017fff700 1 Processor -- start 2022-08-01T20:44:53.769 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.766+0000 7f0017fff700 1 -- start start 2022-08-01T20:44:53.774 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.766+0000 7f0017fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f00180b2710 0x7f001814afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:53.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.766+0000 7f0017fff700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f00180b01a0 0x7f001814b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:53.775 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.766+0000 7f0017fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f00180afd60 0x7f00180a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:53.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.766+0000 7f0017fff700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00180a7250 con 0x7f00180b2710 2022-08-01T20:44:53.776 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.766+0000 7f0017fff700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00180a7480 con 0x7f00180b01a0 2022-08-01T20:44:53.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.766+0000 7f0017fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00180a7720 con 0x7f00180afd60 2022-08-01T20:44:53.777 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.766+0000 7f0016ffd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f00180b2710 0x7f001814afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:53.778 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.766+0000 7f0016ffd700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f00180b2710 0x7f001814afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46286/0 (socket says 172.21.15.17:46286) 2022-08-01T20:44:53.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.766+0000 7f0016ffd700 1 -- 172.21.15.17:0/3302815121 learned_addr learned my addr 172.21.15.17:0/3302815121 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T20:44:53.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.766+0000 7f00167fc700 1 --2- 172.21.15.17:0/3302815121 >> v2:172.21.15.77:3300/0 conn(0x7f00180b01a0 0x7f001814b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:53.779 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.766+0000 7f00177fe700 1 --2- 172.21.15.17:0/3302815121 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f00180afd60 0x7f00180a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:53.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.766+0000 7f0016ffd700 1 -- 172.21.15.17:0/3302815121 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f00180afd60 msgr2=0x7f00180a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T20:44:53.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.766+0000 7f0016ffd700 1 --2- 172.21.15.17:0/3302815121 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f00180afd60 0x7f00180a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:53.780 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.766+0000 7f0016ffd700 1 -- 172.21.15.17:0/3302815121 >> v2:172.21.15.77:3300/0 conn(0x7f00180b01a0 msgr2=0x7f001814b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T20:44:53.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.766+0000 7f0016ffd700 1 --2- 172.21.15.17:0/3302815121 >> v2:172.21.15.77:3300/0 conn(0x7f00180b01a0 0x7f001814b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:53.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.766+0000 7f0016ffd700 1 -- 172.21.15.17:0/3302815121 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00180a79c0 con 0x7f00180b2710 2022-08-01T20:44:53.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.766+0000 7f00167fc700 1 --2- 172.21.15.17:0/3302815121 >> v2:172.21.15.77:3300/0 conn(0x7f00180b01a0 0x7f001814b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T20:44:53.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.766+0000 7f00177fe700 1 --2- 172.21.15.17:0/3302815121 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f00180afd60 0x7f00180a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-08-01T20:44:53.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.766+0000 7f0016ffd700 1 --2- 172.21.15.17:0/3302815121 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00180b2710 0x7f001814afd0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 rx=0x7f00100032f0 tx=0x7f0010009290).ready entity=mon.0 client_cookie=635660c76a7f7ca9 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:53.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.770+0000 7f0015ffb700 1 -- 172.21.15.17:0/3302815121 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f001000b870 con 0x7f00180b2710 2022-08-01T20:44:53.783 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.770+0000 7f0015ffb700 1 -- 172.21.15.17:0/3302815121 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f001000b9d0 con 0x7f00180b2710 2022-08-01T20:44:53.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.770+0000 7f0015ffb700 1 -- 172.21.15.17:0/3302815121 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f001000bb40 con 0x7f00180b2710 2022-08-01T20:44:53.784 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.770+0000 7f0017fff700 1 -- 172.21.15.17:0/3302815121 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00180b2710 msgr2=0x7f001814afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:53.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.770+0000 7f0017fff700 1 --2- 172.21.15.17:0/3302815121 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00180b2710 0x7f001814afd0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 rx=0x7f00100032f0 tx=0x7f0010009290).stop 2022-08-01T20:44:53.785 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.770+0000 7f0017fff700 1 -- 172.21.15.17:0/3302815121 shutdown_connections 2022-08-01T20:44:53.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.770+0000 7f0017fff700 1 --2- 172.21.15.17:0/3302815121 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f00180afd60 0x7f00180a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:53.786 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.770+0000 7f0017fff700 1 --2- 172.21.15.17:0/3302815121 >> v2:172.21.15.77:3300/0 conn(0x7f00180b01a0 0x7f001814b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:53.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.770+0000 7f0017fff700 1 --2- 172.21.15.17:0/3302815121 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00180b2710 0x7f001814afd0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:53.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.770+0000 7f0017fff700 1 -- 172.21.15.17:0/3302815121 >> 172.21.15.17:0/3302815121 conn(0x7f0018060310 msgr2=0x7f00180b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:53.787 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.770+0000 7f0017fff700 1 -- 172.21.15.17:0/3302815121 shutdown_connections 2022-08-01T20:44:53.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.770+0000 7f0017fff700 1 -- 172.21.15.17:0/3302815121 wait complete. 2022-08-01T20:44:53.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.770+0000 7f0017fff700 1 Processor -- start 2022-08-01T20:44:53.788 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.770+0000 7f0017fff700 1 -- start start 2022-08-01T20:44:53.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.770+0000 7f0017fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00180afd60 0x7f00180ac110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:53.789 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.770+0000 7f0017fff700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f00180b01a0 0x7f00180b4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:53.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.770+0000 7f0017fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00180b2710 0x7f0018144270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:53.790 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.770+0000 7f0017fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00180b4c40 con 0x7f00180afd60 2022-08-01T20:44:53.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.770+0000 7f0017fff700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00180b3910 con 0x7f00180b2710 2022-08-01T20:44:53.791 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.770+0000 7f0017fff700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00180b3390 con 0x7f00180b01a0 2022-08-01T20:44:53.792 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.770+0000 7f0016ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00180afd60 0x7f00180ac110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:53.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.770+0000 7f0016ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00180afd60 0x7f00180ac110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46292/0 (socket says 172.21.15.17:46292) 2022-08-01T20:44:53.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.770+0000 7f0016ffd700 1 -- 172.21.15.17:0/1799311699 learned_addr learned my addr 172.21.15.17:0/1799311699 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T20:44:53.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.770+0000 7f00177fe700 1 --2- 172.21.15.17:0/1799311699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00180b2710 0x7f0018144270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:53.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.770+0000 7f00167fc700 1 --2- 172.21.15.17:0/1799311699 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f00180b01a0 0x7f00180b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:53.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.770+0000 7f0016ffd700 1 -- 172.21.15.17:0/1799311699 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f00180b01a0 msgr2=0x7f00180b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:53.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.770+0000 7f0016ffd700 1 --2- 172.21.15.17:0/1799311699 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f00180b01a0 0x7f00180b4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:53.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.770+0000 7f0016ffd700 1 -- 172.21.15.17:0/1799311699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00180b2710 msgr2=0x7f0018144270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:53.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.770+0000 7f0016ffd700 1 --2- 172.21.15.17:0/1799311699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00180b2710 0x7f0018144270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:53.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.770+0000 7f0016ffd700 1 -- 172.21.15.17:0/1799311699 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0010009530 con 0x7f00180afd60 2022-08-01T20:44:53.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.770+0000 7f0016ffd700 1 --2- 172.21.15.17:0/1799311699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00180afd60 0x7f00180ac110 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 rx=0x7f00100032f0 tx=0x7f0010016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:53.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.774+0000 7f0007fff700 1 -- 172.21.15.17:0/1799311699 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f001000b870 con 0x7f00180afd60 2022-08-01T20:44:53.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.774+0000 7f0007fff700 1 -- 172.21.15.17:0/1799311699 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0010021630 con 0x7f00180afd60 2022-08-01T20:44:53.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.774+0000 7f0017fff700 1 -- 172.21.15.17:0/1799311699 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00180b3b40 con 0x7f00180afd60 2022-08-01T20:44:53.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.774+0000 7f0007fff700 1 -- 172.21.15.17:0/1799311699 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0010021990 con 0x7f00180afd60 2022-08-01T20:44:53.802 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.774+0000 7f0017fff700 1 -- 172.21.15.17:0/1799311699 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0018067350 con 0x7f00180afd60 2022-08-01T20:44:53.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.774+0000 7f0017fff700 1 -- 172.21.15.17:0/1799311699 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00180574e0 con 0x7f00180afd60 2022-08-01T20:44:53.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.774+0000 7f0007fff700 1 -- 172.21.15.17:0/1799311699 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0010027020 con 0x7f00180afd60 2022-08-01T20:44:53.807 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.774+0000 7f0007fff700 1 --2- 172.21.15.17:0/1799311699 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efff4053980 0x7efff4055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:53.807 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.774+0000 7f0007fff700 1 -- 172.21.15.17:0/1799311699 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f001006d100 con 0x7f00180afd60 2022-08-01T20:44:53.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.774+0000 7f00167fc700 1 --2- 172.21.15.17:0/1799311699 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efff4053980 0x7efff4055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:53.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.778+0000 7f00167fc700 1 --2- 172.21.15.17:0/1799311699 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efff4053980 0x7efff4055e40 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0x7f000c00d000 tx=0x7f000c008040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:53.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.782+0000 7f0007fff700 1 -- 172.21.15.17:0/1799311699 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0010044100 con 0x7f00180afd60 2022-08-01T20:44:53.910 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.906+0000 7f0017fff700 1 -- 172.21.15.17:0/1799311699 --> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f00180acee0 con 0x7efff4053980 2022-08-01T20:44:53.911 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.910+0000 7f0007fff700 1 -- 172.21.15.17:0/1799311699 <== mgr.4101 v2:172.21.15.77:6800/23151 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f00180acee0 con 0x7efff4053980 2022-08-01T20:44:53.916 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.910+0000 7f0017fff700 1 -- 172.21.15.17:0/1799311699 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efff4053980 msgr2=0x7efff4055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:53.917 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.910+0000 7f0017fff700 1 --2- 172.21.15.17:0/1799311699 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efff4053980 0x7efff4055e40 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0x7f000c00d000 tx=0x7f000c008040).stop 2022-08-01T20:44:53.917 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.910+0000 7f0017fff700 1 -- 172.21.15.17:0/1799311699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00180afd60 msgr2=0x7f00180ac110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:53.917 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.910+0000 7f0017fff700 1 --2- 172.21.15.17:0/1799311699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00180afd60 0x7f00180ac110 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 rx=0x7f00100032f0 tx=0x7f0010016000).stop 2022-08-01T20:44:53.918 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.910+0000 7f0017fff700 1 -- 172.21.15.17:0/1799311699 shutdown_connections 2022-08-01T20:44:53.918 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.910+0000 7f0017fff700 1 --2- 172.21.15.17:0/1799311699 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7efff4053980 0x7efff4055e40 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:53.918 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.910+0000 7f0017fff700 1 --2- 172.21.15.17:0/1799311699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00180b2710 0x7f0018144270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:53.920 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.910+0000 7f0017fff700 1 --2- 172.21.15.17:0/1799311699 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f00180b01a0 0x7f00180b4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:53.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.910+0000 7f0017fff700 1 --2- 172.21.15.17:0/1799311699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00180afd60 0x7f00180ac110 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:53.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.910+0000 7f0017fff700 1 -- 172.21.15.17:0/1799311699 >> 172.21.15.17:0/1799311699 conn(0x7f0018060310 msgr2=0x7f0018146130 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:53.921 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.910+0000 7f0017fff700 1 -- 172.21.15.17:0/1799311699 shutdown_connections 2022-08-01T20:44:53.923 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:53.910+0000 7f0017fff700 1 -- 172.21.15.17:0/1799311699 wait complete. 2022-08-01T20:44:53.923 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 6 to deep-scrub 2022-08-01T20:44:53.925 INFO:tasks.ceph:Scrubbing osd.7 2022-08-01T20:44:53.926 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2022-08-01T20:44:54.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.038+0000 7f3492b20700 1 Processor -- start 2022-08-01T20:44:54.039 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.038+0000 7f3492b20700 1 -- start start 2022-08-01T20:44:54.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.038+0000 7f3492b20700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f348c0b68f0 0x7f348c13ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:54.043 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.038+0000 7f3492b20700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f348c05ff40 0x7f348c13d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:54.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.038+0000 7f3492b20700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f348c13d860 con 0x7f348c0aa210 2022-08-01T20:44:54.044 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.038+0000 7f3492b20700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f348c0ac6e0 con 0x7f348c05ff40 2022-08-01T20:44:54.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.038+0000 7f3492b20700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348c0ac980 con 0x7f348c0b68f0 2022-08-01T20:44:54.045 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.038+0000 7f34910bd700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f348c05ff40 0x7f348c13d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:54.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.038+0000 7f34908bc700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f348c0b68f0 0x7f348c13ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:54.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.038+0000 7f34910bd700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f348c05ff40 0x7f348c13d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.17:44222/0 (socket says 172.21.15.17:44222) 2022-08-01T20:44:54.046 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.038+0000 7f34910bd700 1 -- 172.21.15.17:0/1193716023 learned_addr learned my addr 172.21.15.17:0/1193716023 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T20:44:54.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.038+0000 7f348bfff700 1 --1- 172.21.15.17:0/1193716023 >> v1:172.21.15.77:6789/0 conn(0x7f348c0aa210 0x7f348c146970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.77:6789/0 says I am v1:172.21.15.17:55294/0 (socket says 172.21.15.17:55294) 2022-08-01T20:44:54.047 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.038+0000 7f34910bd700 1 -- 172.21.15.17:0/1193716023 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f348c0b68f0 msgr2=0x7f348c13ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T20:44:54.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.038+0000 7f34910bd700 1 --2- 172.21.15.17:0/1193716023 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f348c0b68f0 0x7f348c13ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:54.049 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.038+0000 7f34910bd700 1 -- 172.21.15.17:0/1193716023 >> v1:172.21.15.77:6789/0 conn(0x7f348c0aa210 legacy=0x7f348c146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:54.050 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.038+0000 7f34910bd700 1 -- 172.21.15.17:0/1193716023 --> v2:172.21.15.77:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348c0acc20 con 0x7f348c05ff40 2022-08-01T20:44:54.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.038+0000 7f34908bc700 1 --2- 172.21.15.17:0/1193716023 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f348c0b68f0 0x7f348c13ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T20:44:54.051 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.038+0000 7f34910bd700 1 --2- 172.21.15.17:0/1193716023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f348c05ff40 0x7f348c13d370 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f34840032f0 tx=0x7f3484009290).ready entity=mon.1 client_cookie=bc05e3655d1379e5 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:54.052 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.038+0000 7f348b7fe700 1 -- 172.21.15.17:0/1193716023 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f348400b8a0 con 0x7f348c05ff40 2022-08-01T20:44:54.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.038+0000 7f348b7fe700 1 -- 172.21.15.17:0/1193716023 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348400ba00 con 0x7f348c05ff40 2022-08-01T20:44:54.053 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.038+0000 7f348b7fe700 1 -- 172.21.15.17:0/1193716023 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f348400bb70 con 0x7f348c05ff40 2022-08-01T20:44:54.054 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.038+0000 7f3492b20700 1 -- 172.21.15.17:0/1193716023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f348c05ff40 msgr2=0x7f348c13d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:54.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.038+0000 7f3492b20700 1 --2- 172.21.15.17:0/1193716023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f348c05ff40 0x7f348c13d370 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f34840032f0 tx=0x7f3484009290).stop 2022-08-01T20:44:54.055 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.038+0000 7f3492b20700 1 -- 172.21.15.17:0/1193716023 shutdown_connections 2022-08-01T20:44:54.056 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.038+0000 7f3492b20700 1 --2- 172.21.15.17:0/1193716023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f348c05ff40 0x7f348c13d370 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:54.057 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.038+0000 7f3492b20700 1 --2- 172.21.15.17:0/1193716023 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f348c0b68f0 0x7f348c13ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:54.058 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.038+0000 7f3492b20700 1 -- 172.21.15.17:0/1193716023 >> 172.21.15.17:0/1193716023 conn(0x7f348c0b3610 msgr2=0x7f348c0b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:54.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.042+0000 7f3492b20700 1 -- 172.21.15.17:0/1193716023 shutdown_connections 2022-08-01T20:44:54.059 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.042+0000 7f3492b20700 1 -- 172.21.15.17:0/1193716023 wait complete. 2022-08-01T20:44:54.065 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.042+0000 7f3492b20700 1 Processor -- start 2022-08-01T20:44:54.065 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.042+0000 7f3492b20700 1 -- start start 2022-08-01T20:44:54.066 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.042+0000 7f3492b20700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f348c05ff40 0x7f348c152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:54.066 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.042+0000 7f3492b20700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f348c0aa210 0x7f348c146870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:54.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.042+0000 7f3492b20700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f348c0b68f0 0x7f348c156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:54.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.042+0000 7f3492b20700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348c065460 con 0x7f348c0aa210 2022-08-01T20:44:54.067 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.042+0000 7f3492b20700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348c156050 con 0x7f348c05ff40 2022-08-01T20:44:54.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.042+0000 7f3492b20700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348c1562f0 con 0x7f348c0b68f0 2022-08-01T20:44:54.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.042+0000 7f348bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f348c0aa210 0x7f348c146870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:54.068 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.042+0000 7f348bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f348c0aa210 0x7f348c146870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46308/0 (socket says 172.21.15.17:46308) 2022-08-01T20:44:54.069 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.042+0000 7f348bfff700 1 -- 172.21.15.17:0/3785620646 learned_addr learned my addr 172.21.15.17:0/3785620646 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T20:44:54.069 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.042+0000 7f34908bc700 1 --2- 172.21.15.17:0/3785620646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f348c05ff40 0x7f348c152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:54.069 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.042+0000 7f34910bd700 1 --2- 172.21.15.17:0/3785620646 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f348c0b68f0 0x7f348c156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:54.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.042+0000 7f348bfff700 1 -- 172.21.15.17:0/3785620646 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f348c0b68f0 msgr2=0x7f348c156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:54.070 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.042+0000 7f348bfff700 1 --2- 172.21.15.17:0/3785620646 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f348c0b68f0 0x7f348c156790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:54.071 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.042+0000 7f348bfff700 1 -- 172.21.15.17:0/3785620646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f348c05ff40 msgr2=0x7f348c152900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:54.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.042+0000 7f348bfff700 1 --2- 172.21.15.17:0/3785620646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f348c05ff40 0x7f348c152900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:54.072 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.042+0000 7f348bfff700 1 -- 172.21.15.17:0/3785620646 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348000b180 con 0x7f348c0aa210 2022-08-01T20:44:54.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.042+0000 7f348bfff700 1 --2- 172.21.15.17:0/3785620646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f348c0aa210 0x7f348c146870 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 rx=0x7f348c0ad240 tx=0x7f3478007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:54.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.046+0000 7f3489ffb700 1 -- 172.21.15.17:0/3785620646 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34780070d0 con 0x7f348c0aa210 2022-08-01T20:44:54.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.046+0000 7f3489ffb700 1 -- 172.21.15.17:0/3785620646 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3478007230 con 0x7f348c0aa210 2022-08-01T20:44:54.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.046+0000 7f3492b20700 1 -- 172.21.15.17:0/3785620646 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3484009530 con 0x7f348c0aa210 2022-08-01T20:44:54.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.046+0000 7f3489ffb700 1 -- 172.21.15.17:0/3785620646 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3478005550 con 0x7f348c0aa210 2022-08-01T20:44:54.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.046+0000 7f3492b20700 1 -- 172.21.15.17:0/3785620646 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f348c146d60 con 0x7f348c0aa210 2022-08-01T20:44:54.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.046+0000 7f3492b20700 1 -- 172.21.15.17:0/3785620646 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f348c155ed0 con 0x7f348c0aa210 2022-08-01T20:44:54.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.046+0000 7f3489ffb700 1 -- 172.21.15.17:0/3785620646 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f347801a070 con 0x7f348c0aa210 2022-08-01T20:44:54.076 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.050+0000 7f3489ffb700 1 --2- 172.21.15.17:0/3785620646 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f346c053a70 0x7f346c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:54.076 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.050+0000 7f3489ffb700 1 -- 172.21.15.17:0/3785620646 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f347806ade0 con 0x7f348c0aa210 2022-08-01T20:44:54.077 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.050+0000 7f34908bc700 1 --2- 172.21.15.17:0/3785620646 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f346c053a70 0x7f346c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:54.077 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.050+0000 7f3489ffb700 1 --2- 172.21.15.17:0/3785620646 >> [v2:172.21.15.77:6818/23586,v1:172.21.15.77:6819/23586] conn(0x7f346c05a700 0x7f346c05cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:54.077 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.050+0000 7f3489ffb700 1 -- 172.21.15.17:0/3785620646 --> [v2:172.21.15.77:6818/23586,v1:172.21.15.77:6819/23586] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f346c05d180 con 0x7f346c05a700 2022-08-01T20:44:54.078 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.050+0000 7f3489ffb700 1 -- 172.21.15.17:0/3785620646 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=161) v2 ==== 24+0+0 (secure 0 0 0) 0x7f347806d360 con 0x7f348c0aa210 2022-08-01T20:44:54.078 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.050+0000 7f34910bd700 1 --2- 172.21.15.17:0/3785620646 >> [v2:172.21.15.77:6818/23586,v1:172.21.15.77:6819/23586] conn(0x7f346c05a700 0x7f346c05cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:54.078 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.050+0000 7f34908bc700 1 --2- 172.21.15.17:0/3785620646 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f346c053a70 0x7f346c055f30 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7f348000b000 tx=0x7f3480006df0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:54.079 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.050+0000 7f34910bd700 1 --2- 172.21.15.17:0/3785620646 >> [v2:172.21.15.77:6818/23586,v1:172.21.15.77:6819/23586] conn(0x7f346c05a700 0x7f346c05cba0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:54.079 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.050+0000 7f3489ffb700 1 -- 172.21.15.17:0/3785620646 <== osd.7 v2:172.21.15.77:6818/23586 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f346c05d180 con 0x7f346c05a700 2022-08-01T20:44:54.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.066+0000 7f3492b20700 1 -- 172.21.15.17:0/3785620646 --> [v2:172.21.15.77:6818/23586,v1:172.21.15.77:6819/23586] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f348c0acec0 con 0x7f346c05a700 2022-08-01T20:44:54.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.066+0000 7f3489ffb700 1 -- 172.21.15.17:0/3785620646 <== osd.7 v2:172.21.15.77:6818/23586 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f348c0acec0 con 0x7f346c05a700 2022-08-01T20:44:54.083 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2022-08-01T20:44:54.083 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-08-01T20:44:54.084 INFO:teuthology.orchestra.run.smithi017.stdout:} 2022-08-01T20:44:54.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.066+0000 7f3492b20700 1 -- 172.21.15.17:0/3785620646 >> [v2:172.21.15.77:6818/23586,v1:172.21.15.77:6819/23586] conn(0x7f346c05a700 msgr2=0x7f346c05cba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:54.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.066+0000 7f3492b20700 1 --2- 172.21.15.17:0/3785620646 >> [v2:172.21.15.77:6818/23586,v1:172.21.15.77:6819/23586] conn(0x7f346c05a700 0x7f346c05cba0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:54.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.066+0000 7f3492b20700 1 -- 172.21.15.17:0/3785620646 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f346c053a70 msgr2=0x7f346c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:54.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.066+0000 7f3492b20700 1 --2- 172.21.15.17:0/3785620646 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f346c053a70 0x7f346c055f30 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7f348000b000 tx=0x7f3480006df0).stop 2022-08-01T20:44:54.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.066+0000 7f3492b20700 1 -- 172.21.15.17:0/3785620646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f348c0aa210 msgr2=0x7f348c146870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:54.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.066+0000 7f3492b20700 1 --2- 172.21.15.17:0/3785620646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f348c0aa210 0x7f348c146870 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 rx=0x7f348c0ad240 tx=0x7f3478007c10).stop 2022-08-01T20:44:54.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.066+0000 7f34910bd700 1 -- 172.21.15.17:0/3785620646 reap_dead start 2022-08-01T20:44:54.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.070+0000 7f3492b20700 1 -- 172.21.15.17:0/3785620646 shutdown_connections 2022-08-01T20:44:54.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.070+0000 7f3492b20700 1 -- 172.21.15.17:0/3785620646 >> 172.21.15.17:0/3785620646 conn(0x7f348c0b3610 msgr2=0x7f348c1446d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:54.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.070+0000 7f3492b20700 1 -- 172.21.15.17:0/3785620646 shutdown_connections 2022-08-01T20:44:54.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.070+0000 7f3492b20700 1 -- 172.21.15.17:0/3785620646 wait complete. 2022-08-01T20:44:54.095 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2022-08-01T20:44:54.212 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.206+0000 7f61c5e33700 1 Processor -- start 2022-08-01T20:44:54.213 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.206+0000 7f61c5e33700 1 -- start start 2022-08-01T20:44:54.214 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.206+0000 7f61c5e33700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f61c00a9da0 0x7f61c0060950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:54.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.206+0000 7f61c5e33700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f61c01402d0 0x7f61c0140760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:54.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.206+0000 7f61c5e33700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c0067030 con 0x7f61c00b31d0 2022-08-01T20:44:54.215 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.206+0000 7f61c5e33700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c0064a70 con 0x7f61c00a9da0 2022-08-01T20:44:54.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.206+0000 7f61c5e33700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c0061f60 con 0x7f61c01402d0 2022-08-01T20:44:54.216 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.206+0000 7f61bffff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f61c00a9da0 0x7f61c0060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:54.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.206+0000 7f61c4e31700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f61c00b31d0 0x7f61c0066020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:38510/0 (socket says 172.21.15.17:38510) 2022-08-01T20:44:54.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.206+0000 7f61c4e31700 1 -- 172.21.15.17:0/3113070324 learned_addr learned my addr 172.21.15.17:0/3113070324 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T20:44:54.217 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.206+0000 7f61bffff700 1 -- 172.21.15.17:0/3113070324 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f61c01402d0 msgr2=0x7f61c0140760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T20:44:54.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.206+0000 7f61bffff700 1 --2- 172.21.15.17:0/3113070324 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f61c01402d0 0x7f61c0140760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2022-08-01T20:44:54.218 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.206+0000 7f61bffff700 1 -- 172.21.15.17:0/3113070324 >> v1:172.21.15.17:6789/0 conn(0x7f61c00b31d0 legacy=0x7f61c0066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:54.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.206+0000 7f61bffff700 1 -- 172.21.15.17:0/3113070324 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c00b35b0 con 0x7f61c00a9da0 2022-08-01T20:44:54.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.206+0000 7f61bffff700 1 --2- 172.21.15.17:0/3113070324 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61c00a9da0 0x7f61c0060950 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 rx=0x7f61c0063230 tx=0x7f61b800b470).ready entity=mon.0 client_cookie=ebcdf2b90cc0987d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:54.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.206+0000 7f61bf7fe700 1 -- 172.21.15.17:0/3113070324 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61b800dde0 con 0x7f61c00a9da0 2022-08-01T20:44:54.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.206+0000 7f61bf7fe700 1 -- 172.21.15.17:0/3113070324 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b8010470 con 0x7f61c00a9da0 2022-08-01T20:44:54.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.206+0000 7f61bf7fe700 1 -- 172.21.15.17:0/3113070324 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61b80107d0 con 0x7f61c00a9da0 2022-08-01T20:44:54.220 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.206+0000 7f61c5e33700 1 -- 172.21.15.17:0/3113070324 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61c00a9da0 msgr2=0x7f61c0060950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:54.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.206+0000 7f61c5e33700 1 --2- 172.21.15.17:0/3113070324 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61c00a9da0 0x7f61c0060950 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 rx=0x7f61c0063230 tx=0x7f61b800b470).stop 2022-08-01T20:44:54.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.206+0000 7f61c5e33700 1 -- 172.21.15.17:0/3113070324 shutdown_connections 2022-08-01T20:44:54.221 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.206+0000 7f61c5e33700 1 --2- 172.21.15.17:0/3113070324 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f61c01402d0 0x7f61c0140760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2022-08-01T20:44:54.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.210+0000 7f61c5e33700 1 --2- 172.21.15.17:0/3113070324 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61c00a9da0 0x7f61c0060950 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:54.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.210+0000 7f61c5e33700 1 -- 172.21.15.17:0/3113070324 >> 172.21.15.17:0/3113070324 conn(0x7f61c0063870 msgr2=0x7f61c0063c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:54.222 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.210+0000 7f61c5e33700 1 -- 172.21.15.17:0/3113070324 shutdown_connections 2022-08-01T20:44:54.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.210+0000 7f61c5e33700 1 -- 172.21.15.17:0/3113070324 wait complete. 2022-08-01T20:44:54.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.210+0000 7f61c5e33700 1 Processor -- start 2022-08-01T20:44:54.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.210+0000 7f61c5e33700 1 -- start start 2022-08-01T20:44:54.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.210+0000 7f61c5e33700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61c00a9da0 0x7f61c00b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:54.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.210+0000 7f61c5e33700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61c00b31d0 0x7f61c00aa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:54.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.210+0000 7f61c5e33700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f61c01402d0 0x7f61c01514d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:54.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.210+0000 7f61c5e33700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c0141690 con 0x7f61c00a9da0 2022-08-01T20:44:54.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.210+0000 7f61c5e33700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c00aa8b0 con 0x7f61c00b31d0 2022-08-01T20:44:54.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.210+0000 7f61c5e33700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c0141f50 con 0x7f61c01402d0 2022-08-01T20:44:54.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.210+0000 7f61c4e31700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61c00a9da0 0x7f61c00b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:54.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.210+0000 7f61c4e31700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61c00a9da0 0x7f61c00b4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46322/0 (socket says 172.21.15.17:46322) 2022-08-01T20:44:54.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.210+0000 7f61c4e31700 1 -- 172.21.15.17:0/1868649942 learned_addr learned my addr 172.21.15.17:0/1868649942 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T20:44:54.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.210+0000 7f61c5632700 1 --2- 172.21.15.17:0/1868649942 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f61c01402d0 0x7f61c01514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:54.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.214+0000 7f61bffff700 1 --2- 172.21.15.17:0/1868649942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61c00b31d0 0x7f61c00aa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:54.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.214+0000 7f61c4e31700 1 -- 172.21.15.17:0/1868649942 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f61c01402d0 msgr2=0x7f61c01514d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:54.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.214+0000 7f61c4e31700 1 --2- 172.21.15.17:0/1868649942 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f61c01402d0 0x7f61c01514d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:54.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.214+0000 7f61c4e31700 1 -- 172.21.15.17:0/1868649942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61c00b31d0 msgr2=0x7f61c00aa3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:54.233 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.214+0000 7f61c4e31700 1 --2- 172.21.15.17:0/1868649942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61c00b31d0 0x7f61c00aa3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:54.234 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.214+0000 7f61c4e31700 1 -- 172.21.15.17:0/1868649942 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b800b710 con 0x7f61c00a9da0 2022-08-01T20:44:54.234 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.214+0000 7f61c4e31700 1 --2- 172.21.15.17:0/1868649942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61c00a9da0 0x7f61c00b4c60 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 rx=0x7f61c00634e0 tx=0x7f61b4007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:54.234 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.214+0000 7f61bdffb700 1 -- 172.21.15.17:0/1868649942 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61b40070d0 con 0x7f61c00a9da0 2022-08-01T20:44:54.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.214+0000 7f61bdffb700 1 -- 172.21.15.17:0/1868649942 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b4007230 con 0x7f61c00a9da0 2022-08-01T20:44:54.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.214+0000 7f61c5e33700 1 -- 172.21.15.17:0/1868649942 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c01421f0 con 0x7f61c00a9da0 2022-08-01T20:44:54.235 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.214+0000 7f61bdffb700 1 -- 172.21.15.17:0/1868649942 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61b4005610 con 0x7f61c00a9da0 2022-08-01T20:44:54.236 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.214+0000 7f61c5e33700 1 -- 172.21.15.17:0/1868649942 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c00aab80 con 0x7f61c00a9da0 2022-08-01T20:44:54.236 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.214+0000 7f61c5e33700 1 -- 172.21.15.17:0/1868649942 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61c0066af0 con 0x7f61c00a9da0 2022-08-01T20:44:54.236 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.218+0000 7f61bdffb700 1 -- 172.21.15.17:0/1868649942 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f61b4010070 con 0x7f61c00a9da0 2022-08-01T20:44:54.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.218+0000 7f61bdffb700 1 --2- 172.21.15.17:0/1868649942 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f61a00539e0 0x7f61a0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:54.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.218+0000 7f61bdffb700 1 -- 172.21.15.17:0/1868649942 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f61b406b290 con 0x7f61c00a9da0 2022-08-01T20:44:54.237 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.218+0000 7f61bffff700 1 --2- 172.21.15.17:0/1868649942 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f61a00539e0 0x7f61a0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:54.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.218+0000 7f61bffff700 1 --2- 172.21.15.17:0/1868649942 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f61a00539e0 0x7f61a0055ea0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7f61b801e000 tx=0x7f61b8018000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:54.238 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.222+0000 7f61bdffb700 1 -- 172.21.15.17:0/1868649942 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f61b4042330 con 0x7f61c00a9da0 2022-08-01T20:44:54.351 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.350+0000 7f61c5e33700 1 -- 172.21.15.17:0/1868649942 --> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f61c00574e0 con 0x7f61a00539e0 2022-08-01T20:44:54.353 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.350+0000 7f61bdffb700 1 -- 172.21.15.17:0/1868649942 <== mgr.4101 v2:172.21.15.77:6800/23151 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f61c00574e0 con 0x7f61a00539e0 2022-08-01T20:44:54.356 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.354+0000 7f61c5e33700 1 -- 172.21.15.17:0/1868649942 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f61a00539e0 msgr2=0x7f61a0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:54.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.354+0000 7f61c5e33700 1 --2- 172.21.15.17:0/1868649942 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f61a00539e0 0x7f61a0055ea0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7f61b801e000 tx=0x7f61b8018000).stop 2022-08-01T20:44:54.357 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.354+0000 7f61c5e33700 1 -- 172.21.15.17:0/1868649942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61c00a9da0 msgr2=0x7f61c00b4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:54.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.354+0000 7f61c5e33700 1 --2- 172.21.15.17:0/1868649942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61c00a9da0 0x7f61c00b4c60 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 rx=0x7f61c00634e0 tx=0x7f61b4007c00).stop 2022-08-01T20:44:54.358 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.354+0000 7f61c5e33700 1 -- 172.21.15.17:0/1868649942 shutdown_connections 2022-08-01T20:44:54.359 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.354+0000 7f61c5e33700 1 --2- 172.21.15.17:0/1868649942 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f61a00539e0 0x7f61a0055ea0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:54.360 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.354+0000 7f61c5e33700 1 --2- 172.21.15.17:0/1868649942 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f61c01402d0 0x7f61c01514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:54.361 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.354+0000 7f61c5e33700 1 --2- 172.21.15.17:0/1868649942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61c00b31d0 0x7f61c00aa3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:54.362 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.354+0000 7f61c5e33700 1 --2- 172.21.15.17:0/1868649942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61c00a9da0 0x7f61c00b4c60 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:54.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.354+0000 7f61c5e33700 1 -- 172.21.15.17:0/1868649942 >> 172.21.15.17:0/1868649942 conn(0x7f61c0063870 msgr2=0x7f61c00b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:54.363 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.354+0000 7f61c5e33700 1 -- 172.21.15.17:0/1868649942 shutdown_connections 2022-08-01T20:44:54.364 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.354+0000 7f61c5e33700 1 -- 172.21.15.17:0/1868649942 wait complete. 2022-08-01T20:44:54.367 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 7 to deep-scrub 2022-08-01T20:44:54.368 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2022-08-01T20:44:54.474 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.474+0000 7f2b09c22700 1 Processor -- start 2022-08-01T20:44:54.475 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.474+0000 7f2b09c22700 1 -- start start 2022-08-01T20:44:54.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.474+0000 7f2b09c22700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2b04148530 0x7f2b040b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:54.477 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.474+0000 7f2b09c22700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7f2b040a7250 0x7f2b041448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:54.478 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.474+0000 7f2b09c22700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b040b04e0 con 0x7f2b040b1f40 2022-08-01T20:44:54.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.474+0000 7f2b09c22700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b040a9250 con 0x7f2b040a7250 2022-08-01T20:44:54.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.474+0000 7f2b09c22700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b040b0780 con 0x7f2b04148530 2022-08-01T20:44:54.479 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.474+0000 7f2b037fe700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2b040b1f40 0x7f2b040b2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:38524/0 (socket says 172.21.15.17:38524) 2022-08-01T20:44:54.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.474+0000 7f2b037fe700 1 -- 172.21.15.17:0/1872864391 learned_addr learned my addr 172.21.15.17:0/1872864391 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T20:44:54.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.474+0000 7f2afaffd700 1 --2- 172.21.15.17:0/1872864391 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2b04148530 0x7f2b040b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:54.481 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.474+0000 7f2b02ffd700 1 -- 172.21.15.17:0/1872864391 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 381072201 0 0) 0x7f2b040b04e0 con 0x7f2b040b1f40 2022-08-01T20:44:54.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.474+0000 7f2b02ffd700 1 -- 172.21.15.17:0/1872864391 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ae4003260 con 0x7f2b040b1f40 2022-08-01T20:44:54.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.474+0000 7f2b03fff700 1 --2- 172.21.15.17:0/1872864391 >> v2:172.21.15.77:3300/0 conn(0x7f2b040a7250 0x7f2b041448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:54.482 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.474+0000 7f2afaffd700 1 -- 172.21.15.17:0/1872864391 >> v2:172.21.15.77:3300/0 conn(0x7f2b040a7250 msgr2=0x7f2b041448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T20:44:54.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.474+0000 7f2afaffd700 1 --2- 172.21.15.17:0/1872864391 >> v2:172.21.15.77:3300/0 conn(0x7f2b040a7250 0x7f2b041448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:54.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.474+0000 7f2afaffd700 1 -- 172.21.15.17:0/1872864391 >> v1:172.21.15.17:6789/0 conn(0x7f2b040b1f40 legacy=0x7f2b040b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:54.483 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.474+0000 7f2afaffd700 1 -- 172.21.15.17:0/1872864391 --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b040b0a20 con 0x7f2b04148530 2022-08-01T20:44:54.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.474+0000 7f2afaffd700 1 --2- 172.21.15.17:0/1872864391 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2b04148530 0x7f2b040b1160 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f2aec002f70 tx=0x7f2aec006df0).ready entity=mon.2 client_cookie=f55fb47467059501 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:54.484 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.474+0000 7f2b02ffd700 1 -- 172.21.15.17:0/1872864391 <== mon.2 v2:172.21.15.77:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2aec00d4a0 con 0x7f2b04148530 2022-08-01T20:44:54.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.474+0000 7f2b02ffd700 1 -- 172.21.15.17:0/1872864391 <== mon.2 v2:172.21.15.77:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aec000e90 con 0x7f2b04148530 2022-08-01T20:44:54.486 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.474+0000 7f2b02ffd700 1 -- 172.21.15.17:0/1872864391 <== mon.2 v2:172.21.15.77:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2aec00d7f0 con 0x7f2b04148530 2022-08-01T20:44:54.492 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.474+0000 7f2b09c22700 1 -- 172.21.15.17:0/1872864391 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2b04148530 msgr2=0x7f2b040b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:54.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.474+0000 7f2b09c22700 1 --2- 172.21.15.17:0/1872864391 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2b04148530 0x7f2b040b1160 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f2aec002f70 tx=0x7f2aec006df0).stop 2022-08-01T20:44:54.493 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.474+0000 7f2b09c22700 1 -- 172.21.15.17:0/1872864391 shutdown_connections 2022-08-01T20:44:54.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.474+0000 7f2b09c22700 1 --2- 172.21.15.17:0/1872864391 >> v2:172.21.15.77:3300/0 conn(0x7f2b040a7250 0x7f2b041448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:54.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.474+0000 7f2b09c22700 1 --2- 172.21.15.17:0/1872864391 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2b04148530 0x7f2b040b1160 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:54.494 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.474+0000 7f2b09c22700 1 -- 172.21.15.17:0/1872864391 >> 172.21.15.17:0/1872864391 conn(0x7f2b0405f840 msgr2=0x7f2b0405fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:54.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.478+0000 7f2b09c22700 1 -- 172.21.15.17:0/1872864391 shutdown_connections 2022-08-01T20:44:54.495 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.478+0000 7f2b09c22700 1 -- 172.21.15.17:0/1872864391 wait complete. 2022-08-01T20:44:54.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.478+0000 7f2b09c22700 1 Processor -- start 2022-08-01T20:44:54.496 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.478+0000 7f2b09c22700 1 -- start start 2022-08-01T20:44:54.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.478+0000 7f2b09c22700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2b040a7250 0x7f2b040aed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:54.497 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.478+0000 7f2b09c22700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b040b1f40 0x7f2b040aabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:54.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.478+0000 7f2b09c22700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b04148530 0x7f2b04156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:54.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.478+0000 7f2b09c22700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b040abdd0 con 0x7f2b04148530 2022-08-01T20:44:54.498 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.478+0000 7f2b09c22700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b040ab210 con 0x7f2b040b1f40 2022-08-01T20:44:54.499 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.478+0000 7f2b09c22700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b040629f0 con 0x7f2b040a7250 2022-08-01T20:44:54.499 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.478+0000 7f2b03fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b04148530 0x7f2b04156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:54.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.478+0000 7f2b037fe700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2b040a7250 0x7f2b040aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:54.500 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.478+0000 7f2b03fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b04148530 0x7f2b04156b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46340/0 (socket says 172.21.15.17:46340) 2022-08-01T20:44:54.501 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.478+0000 7f2b03fff700 1 -- 172.21.15.17:0/71095596 learned_addr learned my addr 172.21.15.17:0/71095596 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T20:44:54.501 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.478+0000 7f2afaffd700 1 --2- 172.21.15.17:0/71095596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b040b1f40 0x7f2b040aabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:54.502 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.482+0000 7f2afaffd700 1 -- 172.21.15.17:0/71095596 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2b040a7250 msgr2=0x7f2b040aed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:54.502 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.482+0000 7f2afaffd700 1 --2- 172.21.15.17:0/71095596 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2b040a7250 0x7f2b040aed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:54.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.482+0000 7f2afaffd700 1 -- 172.21.15.17:0/71095596 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b04148530 msgr2=0x7f2b04156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:54.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.482+0000 7f2afaffd700 1 --2- 172.21.15.17:0/71095596 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b04148530 0x7f2b04156b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:54.503 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.482+0000 7f2afaffd700 1 -- 172.21.15.17:0/71095596 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aec00b180 con 0x7f2b040b1f40 2022-08-01T20:44:54.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.482+0000 7f2b037fe700 1 --2- 172.21.15.17:0/71095596 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2b040a7250 0x7f2b040aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-08-01T20:44:54.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.482+0000 7f2afaffd700 1 --2- 172.21.15.17:0/71095596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b040b1f40 0x7f2b040aabe0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f2b040af730 tx=0x7f2aec014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:54.504 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.482+0000 7f2b017fa700 1 -- 172.21.15.17:0/71095596 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2aec016070 con 0x7f2b040b1f40 2022-08-01T20:44:54.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.482+0000 7f2b09c22700 1 -- 172.21.15.17:0/71095596 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b04062c90 con 0x7f2b040b1f40 2022-08-01T20:44:54.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.482+0000 7f2b017fa700 1 -- 172.21.15.17:0/71095596 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aec012670 con 0x7f2b040b1f40 2022-08-01T20:44:54.505 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.482+0000 7f2b017fa700 1 -- 172.21.15.17:0/71095596 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2aec0129d0 con 0x7f2b040b1f40 2022-08-01T20:44:54.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.482+0000 7f2b09c22700 1 -- 172.21.15.17:0/71095596 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b040ab700 con 0x7f2b040b1f40 2022-08-01T20:44:54.506 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.482+0000 7f2b017fa700 1 -- 172.21.15.17:0/71095596 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2aec00f030 con 0x7f2b040b1f40 2022-08-01T20:44:54.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.482+0000 7f2b017fa700 1 --2- 172.21.15.17:0/71095596 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2ae80539e0 0x7f2ae8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:44:54.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.482+0000 7f2b017fa700 1 -- 172.21.15.17:0/71095596 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f2aec06c300 con 0x7f2b040b1f40 2022-08-01T20:44:54.507 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.486+0000 7f2b037fe700 1 --2- 172.21.15.17:0/71095596 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2ae80539e0 0x7f2ae8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:44:54.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.486+0000 7f2b037fe700 1 --2- 172.21.15.17:0/71095596 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2ae80539e0 0x7f2ae8055ea0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7f2af400d000 tx=0x7f2af4009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:44:54.508 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.486+0000 7f2b09c22700 1 -- 172.21.15.17:0/71095596 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b040574e0 con 0x7f2b040b1f40 2022-08-01T20:44:54.509 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.494+0000 7f2b017fa700 1 -- 172.21.15.17:0/71095596 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2aec043340 con 0x7f2b040b1f40 2022-08-01T20:44:54.621 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.618+0000 7f2b09c22700 1 -- 172.21.15.17:0/71095596 --> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2b04063130 con 0x7f2ae80539e0 2022-08-01T20:44:54.628 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.626+0000 7f2b017fa700 1 -- 172.21.15.17:0/71095596 <== mgr.4101 v2:172.21.15.77:6800/23151 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+68105 (secure 0 0 0) 0x7f2af400f6d0 con 0x7f2ae80539e0 2022-08-01T20:44:54.631 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T20:44:54.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.626+0000 7f2b09c22700 1 -- 172.21.15.17:0/71095596 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2ae80539e0 msgr2=0x7f2ae8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:54.633 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.626+0000 7f2b09c22700 1 --2- 172.21.15.17:0/71095596 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2ae80539e0 0x7f2ae8055ea0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7f2af400d000 tx=0x7f2af4009040).stop 2022-08-01T20:44:54.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.626+0000 7f2b09c22700 1 -- 172.21.15.17:0/71095596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b040b1f40 msgr2=0x7f2b040aabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:44:54.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.626+0000 7f2b09c22700 1 --2- 172.21.15.17:0/71095596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b040b1f40 0x7f2b040aabe0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f2b040af730 tx=0x7f2aec014000).stop 2022-08-01T20:44:54.634 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.630+0000 7f2b09c22700 1 -- 172.21.15.17:0/71095596 shutdown_connections 2022-08-01T20:44:54.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.630+0000 7f2b09c22700 1 --2- 172.21.15.17:0/71095596 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f2ae80539e0 0x7f2ae8055ea0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:54.635 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.630+0000 7f2b09c22700 1 --2- 172.21.15.17:0/71095596 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b04148530 0x7f2b04156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:54.636 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.630+0000 7f2b09c22700 1 --2- 172.21.15.17:0/71095596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b040b1f40 0x7f2b040aabe0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:54.636 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.630+0000 7f2b09c22700 1 --2- 172.21.15.17:0/71095596 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f2b040a7250 0x7f2b040aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:44:54.637 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.630+0000 7f2b09c22700 1 -- 172.21.15.17:0/71095596 >> 172.21.15.17:0/71095596 conn(0x7f2b0405f840 msgr2=0x7f2b04147190 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:44:54.637 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.630+0000 7f2b09c22700 1 -- 172.21.15.17:0/71095596 shutdown_connections 2022-08-01T20:44:54.637 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:44:54.630+0000 7f2b09c22700 1 -- 172.21.15.17:0/71095596 wait complete. 2022-08-01T20:44:54.638 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2022-08-01T20:44:54.643 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":2283,"stamp":"2022-08-01T20:44:54.152963+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":8404664,"kb_used_data":15992,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746570056,"statfs":{"total":773094113280,"available":764487737344,"internally_reserved":8589934592,"allocated":16375808,"data_stored":7983484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003593"},"pg_stats":[{"pgid":"1.0","version":"15'2","reported_seq":139,"reported_epoch":159,"state":"active+clean","last_fresh":"2022-08-01T19:44:15.493140+0000","last_change":"2022-08-01T19:33:58.589491+0000","last_active":"2022-08-01T19:44:15.493140+0000","last_peered":"2022-08-01T19:44:15.493140+0000","last_clean":"2022-08-01T19:44:15.493140+0000","last_became_active":"2022-08-01T19:33:58.588671+0000","last_became_peered":"2022-08-01T19:33:58.588671+0000","last_unstale":"2022-08-01T19:44:15.493140+0000","last_undegraded":"2022-08-01T19:44:15.493140+0000","last_fullsized":"2022-08-01T19:44:15.493140+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:33:53.002518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:33:53.002518+0000","last_clean_scrub_stamp":"2022-08-01T19:33:53.002518+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":146,"reported_epoch":160,"state":"active+clean","last_fresh":"2022-08-01T19:44:17.297517+0000","last_change":"2022-08-01T19:34:03.246302+0000","last_active":"2022-08-01T19:44:17.297517+0000","last_peered":"2022-08-01T19:44:17.297517+0000","last_clean":"2022-08-01T19:44:17.297517+0000","last_became_active":"2022-08-01T19:34:03.245544+0000","last_became_peered":"2022-08-01T19:34:03.245544+0000","last_unstale":"2022-08-01T19:44:17.297517+0000","last_undegraded":"2022-08-01T19:44:17.297517+0000","last_fullsized":"2022-08-01T19:44:17.297517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_clean_scrub_stamp":"2022-08-01T19:34:02.212190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":134,"reported_epoch":159,"state":"active+clean","last_fresh":"2022-08-01T19:44:15.493078+0000","last_change":"2022-08-01T19:34:03.246244+0000","last_active":"2022-08-01T19:44:15.493078+0000","last_peered":"2022-08-01T19:44:15.493078+0000","last_clean":"2022-08-01T19:44:15.493078+0000","last_became_active":"2022-08-01T19:34:03.245832+0000","last_became_peered":"2022-08-01T19:34:03.245832+0000","last_unstale":"2022-08-01T19:44:15.493078+0000","last_undegraded":"2022-08-01T19:44:15.493078+0000","last_fullsized":"2022-08-01T19:44:15.493078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_clean_scrub_stamp":"2022-08-01T19:34:02.212190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":141,"reported_epoch":160,"state":"active+clean","last_fresh":"2022-08-01T19:44:17.865487+0000","last_change":"2022-08-01T19:34:03.244647+0000","last_active":"2022-08-01T19:44:17.865487+0000","last_peered":"2022-08-01T19:44:17.865487+0000","last_clean":"2022-08-01T19:44:17.865487+0000","last_became_active":"2022-08-01T19:34:03.244231+0000","last_became_peered":"2022-08-01T19:34:03.244231+0000","last_unstale":"2022-08-01T19:44:17.865487+0000","last_undegraded":"2022-08-01T19:44:17.865487+0000","last_fullsized":"2022-08-01T19:44:17.865487+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_clean_scrub_stamp":"2022-08-01T19:34:02.212190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":146,"reported_epoch":160,"state":"active+clean","last_fresh":"2022-08-01T19:44:17.297850+0000","last_change":"2022-08-01T19:34:03.246395+0000","last_active":"2022-08-01T19:44:17.297850+0000","last_peered":"2022-08-01T19:44:17.297850+0000","last_clean":"2022-08-01T19:44:17.297850+0000","last_became_active":"2022-08-01T19:34:03.245763+0000","last_became_peered":"2022-08-01T19:34:03.245763+0000","last_unstale":"2022-08-01T19:44:17.297850+0000","last_undegraded":"2022-08-01T19:44:17.297850+0000","last_fullsized":"2022-08-01T19:44:17.297850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_clean_scrub_stamp":"2022-08-01T19:34:02.212190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":141,"reported_epoch":159,"state":"active+clean","last_fresh":"2022-08-01T19:44:15.564429+0000","last_change":"2022-08-01T19:34:03.661980+0000","last_active":"2022-08-01T19:44:15.564429+0000","last_peered":"2022-08-01T19:44:15.564429+0000","last_clean":"2022-08-01T19:44:15.564429+0000","last_became_active":"2022-08-01T19:34:03.661358+0000","last_became_peered":"2022-08-01T19:34:03.661358+0000","last_unstale":"2022-08-01T19:44:15.564429+0000","last_undegraded":"2022-08-01T19:44:15.564429+0000","last_fullsized":"2022-08-01T19:44:15.564429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_clean_scrub_stamp":"2022-08-01T19:34:02.212190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":134,"reported_epoch":159,"state":"active+clean","last_fresh":"2022-08-01T19:44:15.493245+0000","last_change":"2022-08-01T19:34:03.665604+0000","last_active":"2022-08-01T19:44:15.493245+0000","last_peered":"2022-08-01T19:44:15.493245+0000","last_clean":"2022-08-01T19:44:15.493245+0000","last_became_active":"2022-08-01T19:34:03.664627+0000","last_became_peered":"2022-08-01T19:34:03.664627+0000","last_unstale":"2022-08-01T19:44:15.493245+0000","last_undegraded":"2022-08-01T19:44:15.493245+0000","last_fullsized":"2022-08-01T19:44:15.493245+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_clean_scrub_stamp":"2022-08-01T19:34:02.212190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":141,"reported_epoch":159,"state":"active+clean","last_fresh":"2022-08-01T19:44:15.564596+0000","last_change":"2022-08-01T19:34:03.243940+0000","last_active":"2022-08-01T19:44:15.564596+0000","last_peered":"2022-08-01T19:44:15.564596+0000","last_clean":"2022-08-01T19:44:15.564596+0000","last_became_active":"2022-08-01T19:34:03.243496+0000","last_became_peered":"2022-08-01T19:34:03.243496+0000","last_unstale":"2022-08-01T19:44:15.564596+0000","last_undegraded":"2022-08-01T19:44:15.564596+0000","last_fullsized":"2022-08-01T19:44:15.564596+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_clean_scrub_stamp":"2022-08-01T19:34:02.212190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":141,"reported_epoch":160,"state":"active+clean","last_fresh":"2022-08-01T19:44:18.212099+0000","last_change":"2022-08-01T19:34:03.245622+0000","last_active":"2022-08-01T19:44:18.212099+0000","last_peered":"2022-08-01T19:44:18.212099+0000","last_clean":"2022-08-01T19:44:18.212099+0000","last_became_active":"2022-08-01T19:34:03.245060+0000","last_became_peered":"2022-08-01T19:34:03.245060+0000","last_unstale":"2022-08-01T19:44:18.212099+0000","last_undegraded":"2022-08-01T19:44:18.212099+0000","last_fullsized":"2022-08-01T19:44:18.212099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T19:34:02.212190+0000","last_clean_scrub_stamp":"2022-08-01T19:34:02.212190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":15,"seq":64424510294,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050584,"kb_used_data":2000,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321256,"statfs":{"total":96636764160,"available":95560966144,"internally_reserved":1073741824,"allocated":2048000,"data_stored":998080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 20:44:19 2022","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.78300000000000003,"15min":0.79600000000000004},"min":{"1min":0.59799999999999998,"5min":0.55300000000000005,"15min":0.54000000000000004},"max":{"1min":1.0369999999999999,"5min":1.161,"15min":1.161},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.82199999999999995,"15min":0.82599999999999996},"min":{"1min":0.63800000000000001,"5min":0.59899999999999998,"15min":0.56699999999999995},"max":{"1min":1.1379999999999999,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.91200000000000003}]},{"osd":1,"last update":"Mon Aug 1 20:44:19 2022","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.84599999999999997,"15min":0.84699999999999998},"min":{"1min":0.65800000000000003,"5min":0.61399999999999999,"15min":0.55400000000000005},"max":{"1min":1.1779999999999999,"5min":1.26,"15min":1.26},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.83399999999999996,"15min":0.81999999999999995},"min":{"1min":0.67100000000000004,"5min":0.60499999999999998,"15min":0.51800000000000002},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.82099999999999995}]},{"osd":2,"last update":"Mon Aug 1 20:44:19 2022","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.86599999999999999,"15min":0.86699999999999999},"min":{"1min":0.63200000000000001,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.1100000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.83299999999999996,"15min":0.83699999999999997},"min":{"1min":0.68100000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.111,"5min":1.111,"15min":1.216},"last":0.78800000000000003}]},{"osd":3,"last update":"Mon Aug 1 20:44:19 2022","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.90700000000000003,"15min":0.88300000000000001},"min":{"1min":0.69499999999999995,"5min":0.64000000000000001,"15min":0.60699999999999998},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1699999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.93200000000000005,"15min":0.91600000000000004},"min":{"1min":0.66000000000000003,"5min":0.65700000000000003,"15min":0.63700000000000001},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.88}]},{"osd":4,"last update":"Mon Aug 1 20:44:19 2022","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79800000000000004,"15min":0.77500000000000002},"min":{"1min":0.622,"5min":0.54400000000000004,"15min":0.52400000000000002},"max":{"1min":1.0589999999999999,"5min":1.161,"15min":1.161},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.92500000000000004,"15min":0.89100000000000001},"min":{"1min":0.67000000000000004,"5min":0.58799999999999997,"15min":0.56399999999999995},"max":{"1min":1.145,"5min":1.22,"15min":1.264},"last":0.70399999999999996}]},{"osd":6,"last update":"Mon Aug 1 20:44:19 2022","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.93500000000000005,"15min":0.91300000000000003},"min":{"1min":0.58799999999999997,"5min":0.57199999999999995,"15min":0.56100000000000005},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.96199999999999997,"15min":0.93799999999999994},"min":{"1min":0.82199999999999995,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1619999999999999,"5min":1.2070000000000001,"15min":1.2829999999999999},"last":0.85699999999999998}]},{"osd":7,"last update":"Mon Aug 1 20:44:19 2022","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0209999999999999,"15min":0.96799999999999997},"min":{"1min":0.79300000000000004,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0449999999999999,"15min":0.98599999999999999},"min":{"1min":0.70699999999999996,"5min":0.69599999999999995,"15min":0.66100000000000003},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.96399999999999997}]}]},{"osd":4,"up_from":15,"seq":64424510293,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1050584,"kb_used_data":2000,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321256,"statfs":{"total":96636764160,"available":95560966144,"internally_reserved":1073741824,"allocated":2048000,"data_stored":998080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 20:43:55 2022","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.80700000000000005,"15min":0.80500000000000005},"min":{"1min":0.58599999999999997,"5min":0.55500000000000005,"15min":0.52400000000000002},"max":{"1min":1.119,"5min":1.165,"15min":1.165},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80000000000000004,"15min":0.81999999999999995},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.026,"5min":1.1719999999999999,"15min":1.194},"last":0.64300000000000002}]},{"osd":1,"last update":"Mon Aug 1 20:43:55 2022","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.80100000000000005,"15min":0.82299999999999995},"min":{"1min":0.63400000000000001,"5min":0.57199999999999995,"15min":0.56899999999999995},"max":{"1min":1.022,"5min":1.1619999999999999,"15min":1.1779999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85899999999999999,"15min":0.84099999999999997},"min":{"1min":0.60199999999999998,"5min":0.5,"15min":0.5},"max":{"1min":1.0580000000000001,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.747}]},{"osd":2,"last update":"Mon Aug 1 20:43:55 2022","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.93600000000000005,"15min":0.93200000000000005},"min":{"1min":0.623,"5min":0.622,"15min":0.59199999999999997},"max":{"1min":1.0960000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.82399999999999995,"5min":0.66300000000000003,"15min":0.60899999999999999},"max":{"1min":1.2230000000000001,"5min":1.2569999999999999,"15min":1.4590000000000001},"last":0.68999999999999995}]},{"osd":3,"last update":"Mon Aug 1 20:43:55 2022","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83999999999999997,"15min":0.85099999999999998},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.621},"max":{"1min":1.135,"5min":1.244,"15min":1.2829999999999999},"last":1.244},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.92500000000000004,"15min":0.92600000000000005},"min":{"1min":0.79900000000000004,"5min":0.66300000000000003,"15min":0.65900000000000003},"max":{"1min":1.018,"5min":1.228,"15min":1.28},"last":1.1160000000000001}]},{"osd":5,"last update":"Mon Aug 1 20:43:55 2022","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.82599999999999996,"15min":0.83799999999999997},"min":{"1min":0.60499999999999998,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.171,"5min":1.171,"15min":1.1870000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.86599999999999999,"15min":0.873},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.55900000000000005},"max":{"1min":1.212,"5min":1.298,"15min":1.3460000000000001},"last":1.1619999999999999}]},{"osd":6,"last update":"Mon Aug 1 20:43:55 2022","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.93300000000000005,"15min":0.90700000000000003},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2,"5min":1.27,"15min":1.292},"last":1.077},{"interface":"front","average":{"1min":1.008,"5min":0.98999999999999999,"15min":0.96099999999999997},"min":{"1min":0.77500000000000002,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.3160000000000001,"5min":1.331,"15min":1.5549999999999999},"last":1.085}]},{"osd":7,"last update":"Mon Aug 1 20:43:55 2022","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.96199999999999997,"15min":0.95599999999999996},"min":{"1min":0.67900000000000005,"5min":0.65000000000000002,"15min":0.55800000000000005},"max":{"1min":1.296,"5min":1.296,"15min":1.599},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0449999999999999,"15min":1.0369999999999999},"min":{"1min":0.86499999999999999,"5min":0.68200000000000005,"15min":0.64500000000000002},"max":{"1min":1.371,"5min":1.409,"15min":1.4870000000000001},"last":1.3680000000000001}]}]},{"osd":7,"up_from":14,"seq":60129542998,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050584,"kb_used_data":2000,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321256,"statfs":{"total":96636764160,"available":95560966144,"internally_reserved":1073741824,"allocated":2048000,"data_stored":998080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 20:44:35 2022","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.88200000000000001,"15min":0.90500000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.1539999999999999,"5min":1.163,"15min":1.244},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.90000000000000002,"15min":0.89600000000000002},"min":{"1min":0.622,"5min":0.622,"15min":0.52100000000000002},"max":{"1min":1.0669999999999999,"5min":1.1499999999999999,"15min":1.1910000000000001},"last":0.74099999999999999}]},{"osd":1,"last update":"Mon Aug 1 20:44:35 2022","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.82799999999999996,"15min":0.84399999999999997},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.56100000000000005},"max":{"1min":1.0860000000000001,"5min":1.123,"15min":1.159},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82299999999999995,"15min":0.81599999999999995},"min":{"1min":0.65400000000000003,"5min":0.57599999999999996,"15min":0.52900000000000003},"max":{"1min":1.071,"5min":1.163,"15min":1.163},"last":0.83899999999999997}]},{"osd":2,"last update":"Mon Aug 1 20:44:35 2022","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.84599999999999997,"15min":0.85099999999999998},"min":{"1min":0.63100000000000001,"5min":0.62,"15min":0.57899999999999996},"max":{"1min":0.98999999999999999,"5min":1.1739999999999999,"15min":1.3129999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.875,"5min":0.874,"15min":0.872},"min":{"1min":0.64900000000000002,"5min":0.64200000000000002,"15min":0.60299999999999998},"max":{"1min":1.0880000000000001,"5min":1.161,"15min":1.2370000000000001},"last":0.86199999999999999}]},{"osd":3,"last update":"Mon Aug 1 20:44:35 2022","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.86399999999999999,"15min":0.88700000000000001},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.2150000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.93000000000000005,"15min":0.91700000000000004},"min":{"1min":0.68799999999999994,"5min":0.65900000000000003,"15min":0.622},"max":{"1min":1.1080000000000001,"5min":1.268,"15min":1.3280000000000001},"last":0.94299999999999995}]},{"osd":4,"last update":"Mon Aug 1 20:44:35 2022","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83699999999999997,"15min":0.84199999999999997},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.56399999999999995},"max":{"1min":1.145,"5min":1.218,"15min":1.2529999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.93100000000000005,"15min":0.90600000000000003},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.145,"5min":1.216,"15min":1.4019999999999999},"last":0.68600000000000005}]},{"osd":5,"last update":"Mon Aug 1 20:44:35 2022","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.84599999999999997,"15min":0.83699999999999997},"min":{"1min":0.626,"5min":0.56599999999999995,"15min":0.55200000000000005},"max":{"1min":1.042,"5min":1.157,"15min":1.1950000000000001},"last":0.621},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94599999999999995,"15min":0.94099999999999995},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.59799999999999998},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.385},"last":0.97999999999999998}]},{"osd":6,"last update":"Mon Aug 1 20:44:35 2022","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.93100000000000005,"15min":0.92400000000000004},"min":{"1min":0.72599999999999998,"5min":0.65500000000000003,"15min":0.57499999999999996},"max":{"1min":1.1519999999999999,"5min":1.202,"15min":1.2689999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.012},"min":{"1min":0.84699999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.2310000000000001,"5min":1.306,"15min":1.4510000000000001},"last":0.92500000000000004}]}]},{"osd":6,"up_from":14,"seq":60129542997,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050584,"kb_used_data":2000,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321256,"statfs":{"total":96636764160,"available":95560966144,"internally_reserved":1073741824,"allocated":2048000,"data_stored":998080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 20:44:33 2022","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.92700000000000005,"15min":0.93899999999999995},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.51600000000000001},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":1.0620000000000001},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.88600000000000001,"15min":0.89000000000000001},"min":{"1min":0.69699999999999995,"5min":0.628,"15min":0.60199999999999998},"max":{"1min":1.0589999999999999,"5min":1.0840000000000001,"15min":1.149},"last":0.96499999999999997}]},{"osd":1,"last update":"Mon Aug 1 20:44:33 2022","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.879,"15min":0.84799999999999998},"min":{"1min":0.61699999999999999,"5min":0.57399999999999995,"15min":0.56699999999999995},"max":{"1min":1.1659999999999999,"5min":1.2230000000000001,"15min":1.2490000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.81200000000000006,"15min":0.81699999999999995},"min":{"1min":0.64100000000000001,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.054,"5min":1.054,"15min":1.161},"last":0.89500000000000002}]},{"osd":2,"last update":"Mon Aug 1 20:44:33 2022","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84899999999999998,"15min":0.83199999999999996},"min":{"1min":0.69199999999999995,"5min":0.626,"15min":0.60699999999999998},"max":{"1min":1.032,"5min":1.1890000000000001,"15min":1.4239999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.89300000000000002,"15min":0.91000000000000003},"min":{"1min":0.65000000000000002,"5min":0.61899999999999999,"15min":0.60599999999999998},"max":{"1min":1.169,"5min":1.169,"15min":1.2709999999999999},"last":0.66300000000000003}]},{"osd":3,"last update":"Mon Aug 1 20:44:33 2022","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.68999999999999995,"5min":0.65400000000000003,"15min":0.64100000000000001},"max":{"1min":1.085,"5min":1.103,"15min":1.157},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.93400000000000005,"15min":0.93600000000000005},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.2050000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.877}]},{"osd":4,"last update":"Mon Aug 1 20:44:33 2022","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.877,"15min":0.86799999999999999},"min":{"1min":0.67100000000000004,"5min":0.54100000000000004,"15min":0.54000000000000004},"max":{"1min":1.0900000000000001,"5min":1.1659999999999999,"15min":1.234},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.83499999999999996,"15min":0.82899999999999996},"min":{"1min":0.57799999999999996,"5min":0.55800000000000005,"15min":0.55000000000000004},"max":{"1min":1.0409999999999999,"5min":1.0980000000000001,"15min":1.2010000000000001},"last":0.79300000000000004}]},{"osd":5,"last update":"Mon Aug 1 20:44:33 2022","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.95899999999999996,"15min":0.93100000000000005},"min":{"1min":0.77500000000000002,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.127,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":1.107},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.90200000000000002,"15min":0.878},"min":{"1min":0.65000000000000002,"5min":0.60199999999999998,"15min":0.59799999999999998},"max":{"1min":1.121,"5min":1.1699999999999999,"15min":1.2390000000000001},"last":1.0129999999999999}]},{"osd":7,"last update":"Mon Aug 1 20:44:33 2022","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.89700000000000002,"15min":0.89600000000000002},"min":{"1min":0.66900000000000004,"5min":0.63600000000000001,"15min":0.59799999999999998},"max":{"1min":1.0660000000000001,"5min":1.264,"15min":1.264},"last":1.036},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.97099999999999997,"15min":0.95899999999999996},"min":{"1min":0.76500000000000001,"5min":0.67400000000000004,"15min":0.61899999999999999},"max":{"1min":1.147,"5min":1.355,"15min":1.355},"last":1.083}]}]},{"osd":3,"up_from":14,"seq":60129542997,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050580,"kb_used_data":1996,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321260,"statfs":{"total":96636764160,"available":95560970240,"internally_reserved":1073741824,"allocated":2043904,"data_stored":997502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 20:43:55 2022","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61599999999999999,"15min":0.621},"min":{"1min":0.44700000000000001,"5min":0.42899999999999999,"15min":0.41299999999999998},"max":{"1min":0.82199999999999995,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68200000000000005,"15min":0.67700000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.48599999999999999},"max":{"1min":0.91800000000000004,"5min":1.1020000000000001,"15min":1.194},"last":0.54600000000000004}]},{"osd":1,"last update":"Mon Aug 1 20:43:55 2022","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65900000000000003,"15min":0.66700000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.44900000000000001},"max":{"1min":0.94799999999999995,"5min":1.2170000000000001,"15min":1.25},"last":0.628},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78100000000000003,"15min":0.78000000000000003},"min":{"1min":0.56299999999999994,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.046,"5min":1.1399999999999999,"15min":1.266},"last":0.67200000000000004}]},{"osd":2,"last update":"Mon Aug 1 20:43:55 2022","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.78000000000000003,"15min":0.78400000000000003},"min":{"1min":0.63400000000000001,"5min":0.58699999999999997,"15min":0.45900000000000002},"max":{"1min":1.0049999999999999,"5min":1.171,"15min":1.171},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.84899999999999998,"15min":0.84299999999999997},"min":{"1min":0.57699999999999996,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1000000000000001,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.73199999999999998}]},{"osd":4,"last update":"Mon Aug 1 20:44:38 2022","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.89800000000000002,"15min":0.89000000000000001},"min":{"1min":0.65000000000000002,"5min":0.61899999999999999,"15min":0.59699999999999998},"max":{"1min":1.145,"5min":1.3120000000000001,"15min":1.3400000000000001},"last":0.996},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.93200000000000005,"15min":0.94299999999999995},"min":{"1min":0.70799999999999996,"5min":0.64800000000000002,"15min":0.64500000000000002},"max":{"1min":1.246,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.89000000000000001}]},{"osd":5,"last update":"Mon Aug 1 20:44:38 2022","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99099999999999999,"15min":1.0129999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.68899999999999995},"max":{"1min":1.2989999999999999,"5min":1.46,"15min":1.508},"last":1.1579999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.97299999999999998,"15min":0.96799999999999997},"min":{"1min":0.77300000000000002,"5min":0.71399999999999997,"15min":0.67900000000000005},"max":{"1min":1.2729999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.85299999999999998}]},{"osd":6,"last update":"Mon Aug 1 20:43:55 2022","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.022,"15min":1.0349999999999999},"min":{"1min":0.69599999999999995,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.4470000000000001,"5min":1.548,"15min":1.548},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.087,"15min":1.0980000000000001},"min":{"1min":0.88900000000000001,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.3520000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.101}]},{"osd":7,"last update":"Mon Aug 1 20:43:55 2022","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.0669999999999999,"15min":1.0620000000000001},"min":{"1min":0.84499999999999997,"5min":0.81999999999999995,"15min":0.71799999999999997},"max":{"1min":1.375,"5min":1.468,"15min":1.5109999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.111,"15min":1.111},"min":{"1min":0.90600000000000003,"5min":0.82699999999999996,"15min":0.78400000000000003},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.79000000000000004}]}]},{"osd":2,"up_from":14,"seq":60129542999,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050580,"kb_used_data":1996,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321260,"statfs":{"total":96636764160,"available":95560970240,"internally_reserved":1073741824,"allocated":2043904,"data_stored":997502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 20:44:22 2022","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58799999999999997,"15min":0.60499999999999998},"min":{"1min":0.47799999999999998,"5min":0.46500000000000002,"15min":0.436},"max":{"1min":0.77200000000000002,"5min":0.88400000000000001,"15min":1.1919999999999999},"last":0.442},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72399999999999998,"15min":0.71699999999999997},"min":{"1min":0.503,"5min":0.5,"15min":0.47799999999999998},"max":{"1min":1.0509999999999999,"5min":1.129,"15min":1.179},"last":0.55100000000000005}]},{"osd":1,"last update":"Mon Aug 1 20:44:22 2022","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.81399999999999995,"15min":0.80800000000000005},"min":{"1min":0.66400000000000003,"5min":0.55100000000000005,"15min":0.52100000000000002},"max":{"1min":1.0169999999999999,"5min":1.147,"15min":1.147},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.69799999999999995,"15min":0.69999999999999996},"min":{"1min":0.50800000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.89500000000000002,"5min":1.0509999999999999,"15min":1.0620000000000001},"last":0.42999999999999999}]},{"osd":3,"last update":"Mon Aug 1 20:44:22 2022","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.68999999999999995,"15min":0.70899999999999996},"min":{"1min":0.53800000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.91000000000000003,"5min":0.96799999999999997,"15min":1.2569999999999999},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.77900000000000003,"15min":0.79800000000000004},"min":{"1min":0.61099999999999999,"5min":0.56000000000000005,"15min":0.55800000000000005},"max":{"1min":0.97299999999999998,"5min":1.0629999999999999,"15min":1.24},"last":0.65400000000000003}]},{"osd":4,"last update":"Mon Aug 1 20:44:22 2022","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.90900000000000003,"15min":0.90400000000000003},"min":{"1min":0.68100000000000005,"5min":0.67400000000000004,"15min":0.60199999999999998},"max":{"1min":1.2669999999999999,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.93100000000000005,"15min":0.93200000000000005},"min":{"1min":0.72699999999999998,"5min":0.70699999999999996,"15min":0.67300000000000004},"max":{"1min":1.365,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.80800000000000005}]},{"osd":5,"last update":"Mon Aug 1 20:44:22 2022","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.97899999999999998,"15min":0.996},"min":{"1min":0.77000000000000002,"5min":0.71899999999999997,"15min":0.71399999999999997},"max":{"1min":1.1240000000000001,"5min":1.411,"15min":1.411},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.95899999999999996,"15min":0.95299999999999996},"min":{"1min":0.72999999999999998,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.234,"5min":1.341,"15min":1.421},"last":0.85999999999999999}]},{"osd":6,"last update":"Mon Aug 1 20:44:22 2022","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":0.98999999999999999,"15min":0.98399999999999999},"min":{"1min":0.80600000000000005,"5min":0.72699999999999998,"15min":0.68799999999999994},"max":{"1min":1.319,"5min":1.319,"15min":1.4019999999999999},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.014,"5min":1.018,"15min":1.0249999999999999},"min":{"1min":0.80000000000000004,"5min":0.75900000000000001,"15min":0.71499999999999997},"max":{"1min":1.3120000000000001,"5min":1.383,"15min":1.383},"last":0.746}]},{"osd":7,"last update":"Mon Aug 1 20:44:22 2022","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.077,"15min":1.081},"min":{"1min":0.82399999999999995,"5min":0.76100000000000001,"15min":0.72199999999999998},"max":{"1min":1.2010000000000001,"5min":1.524,"15min":1.5820000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.054,"15min":1.0589999999999999},"min":{"1min":0.82699999999999996,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.494},"last":1.0489999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542998,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050584,"kb_used_data":2000,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321256,"statfs":{"total":96636764160,"available":95560966144,"internally_reserved":1073741824,"allocated":2048000,"data_stored":998080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 20:44:27 2022","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63100000000000001,"15min":0.624},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.42599999999999999},"max":{"1min":0.85599999999999998,"5min":0.90200000000000002,"15min":1.0649999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.752,"15min":0.747},"min":{"1min":0.62,"5min":0.61199999999999999,"15min":0.55900000000000005},"max":{"1min":0.879,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.78700000000000003}]},{"osd":2,"last update":"Mon Aug 1 20:44:27 2022","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.65900000000000003,"15min":0.67400000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.46100000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.1459999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.82999999999999996,"15min":0.83799999999999997},"min":{"1min":0.66000000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.056,"5min":1.24,"15min":1.24},"last":0.83499999999999996}]},{"osd":3,"last update":"Mon Aug 1 20:44:27 2022","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.77100000000000002,"15min":0.77800000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0189999999999999,"5min":1.1779999999999999,"15min":1.228},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.72099999999999997,"15min":0.70699999999999996},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.92300000000000004,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.70299999999999996}]},{"osd":4,"last update":"Mon Aug 1 20:44:27 2022","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88900000000000001,"15min":0.873},"min":{"1min":0.69299999999999995,"5min":0.65300000000000002,"15min":0.629},"max":{"1min":1.21,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.97599999999999998,"15min":0.97699999999999998},"min":{"1min":0.74399999999999999,"5min":0.72299999999999998,"15min":0.67300000000000004},"max":{"1min":1.1180000000000001,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.1839999999999999}]},{"osd":5,"last update":"Mon Aug 1 20:44:27 2022","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.94499999999999995,"15min":0.93000000000000005},"min":{"1min":0.73199999999999998,"5min":0.70299999999999996,"15min":0.65200000000000002},"max":{"1min":1.3120000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.96899999999999997,"15min":0.95599999999999996},"min":{"1min":0.79700000000000004,"5min":0.70499999999999996,"15min":0.66100000000000003},"max":{"1min":1.3799999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.96399999999999997}]},{"osd":6,"last update":"Mon Aug 1 20:44:27 2022","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.97499999999999998,"15min":0.98099999999999998},"min":{"1min":0.72899999999999998,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.2210000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.874},{"interface":"front","average":{"1min":1.087,"5min":1.0660000000000001,"15min":1.054},"min":{"1min":0.84799999999999998,"5min":0.82299999999999995,"15min":0.66800000000000004},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.7010000000000001},"last":1.2430000000000001}]},{"osd":7,"last update":"Mon Aug 1 20:44:27 2022","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0389999999999999,"15min":1.024},"min":{"1min":0.88700000000000001,"5min":0.73599999999999999,"15min":0.72099999999999997},"max":{"1min":1.351,"5min":1.587,"15min":1.587},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.121,"5min":1.0920000000000001,"15min":1.0720000000000001},"min":{"1min":0.90300000000000002,"5min":0.77400000000000002,"15min":0.71899999999999997},"max":{"1min":1.3759999999999999,"5min":1.482,"15min":1.6459999999999999},"last":1.089}]}]},{"osd":0,"up_from":14,"seq":60129542997,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050584,"kb_used_data":2000,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321256,"statfs":{"total":96636764160,"available":95560966144,"internally_reserved":1073741824,"allocated":2048000,"data_stored":998080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Aug 1 20:44:12 2022","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63700000000000001,"15min":0.64000000000000001},"min":{"1min":0.47299999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88300000000000001,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73699999999999999,"15min":0.753},"min":{"1min":0.63400000000000001,"5min":0.59899999999999998,"15min":0.51800000000000002},"max":{"1min":0.873,"5min":0.98999999999999999,"15min":1.0640000000000001},"last":0.79300000000000004}]},{"osd":2,"last update":"Mon Aug 1 20:44:12 2022","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.65400000000000003},"min":{"1min":0.51000000000000001,"5min":0.504,"15min":0.504},"max":{"1min":0.91800000000000004,"5min":1.1020000000000001,"15min":1.2390000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.67700000000000005,"15min":0.67200000000000004},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.47199999999999998},"max":{"1min":0.91300000000000003,"5min":1.1659999999999999,"15min":1.2929999999999999},"last":0.69199999999999995}]},{"osd":3,"last update":"Mon Aug 1 20:44:12 2022","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.72799999999999998,"15min":0.73599999999999999},"min":{"1min":0.51300000000000001,"5min":0.50600000000000001,"15min":0.498},"max":{"1min":0.89000000000000001,"5min":1.1180000000000001,"15min":1.1299999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.873,"15min":0.874},"min":{"1min":0.72299999999999998,"5min":0.69199999999999995,"15min":0.64100000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.228},"last":0.91900000000000004}]},{"osd":4,"last update":"Mon Aug 1 20:44:12 2022","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.94999999999999996,"15min":0.93799999999999994},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.67900000000000005},"max":{"1min":1.1579999999999999,"5min":1.3740000000000001,"15min":1.413},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.93799999999999994,"15min":0.94299999999999995},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.66200000000000003},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.86199999999999999}]},{"osd":5,"last update":"Mon Aug 1 20:44:12 2022","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.90900000000000003,"15min":0.88700000000000001},"min":{"1min":0.746,"5min":0.70299999999999996,"15min":0.65300000000000002},"max":{"1min":1.0980000000000001,"5min":1.2150000000000001,"15min":1.397},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0229999999999999},"min":{"1min":0.85499999999999998,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.222,"5min":1.452,"15min":1.508},"last":1.069}]},{"osd":6,"last update":"Mon Aug 1 20:44:12 2022","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.0600000000000001,"15min":1.0580000000000001},"min":{"1min":0.81100000000000005,"5min":0.77700000000000002,"15min":0.68400000000000005},"max":{"1min":1.29,"5min":1.534,"15min":1.546},"last":1.133},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0269999999999999,"15min":1.032},"min":{"1min":0.84999999999999998,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.2230000000000001,"5min":1.3999999999999999,"15min":1.5189999999999999},"last":1.0409999999999999}]},{"osd":7,"last update":"Mon Aug 1 20:44:12 2022","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.0409999999999999,"15min":1.0269999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.67900000000000005},"max":{"1min":1.347,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.159},{"interface":"front","average":{"1min":1.121,"5min":1.129,"15min":1.129},"min":{"1min":0.77400000000000002,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.3720000000000001,"5min":1.4330000000000001,"15min":1.6559999999999999},"last":1.0229999999999999}]}]}]}} 2022-08-01T20:44:54.655 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2022-08-01T19:33:53.002518+0000 time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=19, tm_min=33, tm_sec=53, tm_wday=0, tm_yday=213, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=20, tm_min=44, tm_sec=49, tm_wday=0, tm_yday=213, tm_isdst=0) 2022-08-01T20:44:54.656 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2022-08-01T19:34:02.212190+0000 time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=19, tm_min=34, tm_sec=2, tm_wday=0, tm_yday=213, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=20, tm_min=44, tm_sec=49, tm_wday=0, tm_yday=213, tm_isdst=0) 2022-08-01T20:44:54.657 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2022-08-01T19:34:02.212190+0000 time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=19, tm_min=34, tm_sec=2, tm_wday=0, tm_yday=213, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=20, tm_min=44, tm_sec=49, tm_wday=0, tm_yday=213, tm_isdst=0) 2022-08-01T20:44:54.657 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2022-08-01T19:34:02.212190+0000 time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=19, tm_min=34, tm_sec=2, tm_wday=0, tm_yday=213, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=20, tm_min=44, tm_sec=49, tm_wday=0, tm_yday=213, tm_isdst=0) 2022-08-01T20:44:54.658 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2022-08-01T19:34:02.212190+0000 time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=19, tm_min=34, tm_sec=2, tm_wday=0, tm_yday=213, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=20, tm_min=44, tm_sec=49, tm_wday=0, tm_yday=213, tm_isdst=0) 2022-08-01T20:44:54.659 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2022-08-01T19:34:02.212190+0000 time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=19, tm_min=34, tm_sec=2, tm_wday=0, tm_yday=213, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=20, tm_min=44, tm_sec=49, tm_wday=0, tm_yday=213, tm_isdst=0) 2022-08-01T20:44:54.660 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2022-08-01T19:34:02.212190+0000 time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=19, tm_min=34, tm_sec=2, tm_wday=0, tm_yday=213, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=20, tm_min=44, tm_sec=49, tm_wday=0, tm_yday=213, tm_isdst=0) 2022-08-01T20:44:54.661 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2022-08-01T19:34:02.212190+0000 time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=19, tm_min=34, tm_sec=2, tm_wday=0, tm_yday=213, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=20, tm_min=44, tm_sec=49, tm_wday=0, tm_yday=213, tm_isdst=0) 2022-08-01T20:44:54.661 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2022-08-01T19:34:02.212190+0000 time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=19, tm_min=34, tm_sec=2, tm_wday=0, tm_yday=213, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=8, tm_mday=1, tm_hour=20, tm_min=44, tm_sec=49, tm_wday=0, tm_yday=213, tm_isdst=0) 2022-08-01T20:44:54.662 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2022-08-01T20:45:06.824 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:45:06.828 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:45:06.831 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:45:06.834 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:45:14.664 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2022-08-01T20:45:14.781 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.777+0000 7f650d4ef700 1 Processor -- start 2022-08-01T20:45:14.782 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.777+0000 7f650d4ef700 1 -- start start 2022-08-01T20:45:14.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.781+0000 7f650d4ef700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f65080af450 0x7f65080af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:45:14.793 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.781+0000 7f650d4ef700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f65080ad9e0 0x7f65080a8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:45:14.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.781+0000 7f650d4ef700 1 -- --> v1:172.21.15.77:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65080aff30 con 0x7f65080ad4b0 2022-08-01T20:45:14.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.781+0000 7f650d4ef700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f650805c7a0 con 0x7f65080af450 2022-08-01T20:45:14.794 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.781+0000 7f650d4ef700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6508140990 con 0x7f65080ad9e0 2022-08-01T20:45:14.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.781+0000 7f6507fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f65080af450 0x7f65080af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:45:14.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.781+0000 7f6507fff700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f65080af450 0x7f65080af830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46346/0 (socket says 172.21.15.17:46346) 2022-08-01T20:45:14.795 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.781+0000 7f6507fff700 1 -- 172.21.15.17:0/2690224785 learned_addr learned my addr 172.21.15.17:0/2690224785 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T20:45:14.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.781+0000 7f650ccee700 1 --2- 172.21.15.17:0/2690224785 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f65080ad9e0 0x7f65080a8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:45:14.796 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.781+0000 7f6507fff700 1 -- 172.21.15.17:0/2690224785 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f65080ad9e0 msgr2=0x7f65080a8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T20:45:14.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.781+0000 7f6507fff700 1 --2- 172.21.15.17:0/2690224785 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f65080ad9e0 0x7f65080a8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T20:45:14.797 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.781+0000 7f6507fff700 1 -- 172.21.15.17:0/2690224785 >> v1:172.21.15.77:6789/0 conn(0x7f65080ad4b0 legacy=0x7f65081432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:45:14.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.781+0000 7f6507fff700 1 -- 172.21.15.17:0/2690224785 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65081439f0 con 0x7f65080af450 2022-08-01T20:45:14.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.781+0000 7f650ccee700 1 --2- 172.21.15.17:0/2690224785 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f65080ad9e0 0x7f65080a8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T20:45:14.798 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.781+0000 7f6507fff700 1 --2- 172.21.15.17:0/2690224785 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65080af450 0x7f65080af830 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 rx=0x7f65080ad900 tx=0x7f64f8006eb0).ready entity=mon.0 client_cookie=a8bcbdec5bc1647d server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:45:14.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.781+0000 7f65077fe700 1 -- 172.21.15.17:0/2690224785 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64f800d5c0 con 0x7f65080af450 2022-08-01T20:45:14.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.781+0000 7f65077fe700 1 -- 172.21.15.17:0/2690224785 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f8000e90 con 0x7f65080af450 2022-08-01T20:45:14.799 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.781+0000 7f65077fe700 1 -- 172.21.15.17:0/2690224785 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64f800d950 con 0x7f65080af450 2022-08-01T20:45:14.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.781+0000 7f650d4ef700 1 -- 172.21.15.17:0/2690224785 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65080af450 msgr2=0x7f65080af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:45:14.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.781+0000 7f650d4ef700 1 --2- 172.21.15.17:0/2690224785 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65080af450 0x7f65080af830 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 rx=0x7f65080ad900 tx=0x7f64f8006eb0).stop 2022-08-01T20:45:14.800 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.781+0000 7f650d4ef700 1 -- 172.21.15.17:0/2690224785 shutdown_connections 2022-08-01T20:45:14.801 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.781+0000 7f650d4ef700 1 --2- 172.21.15.17:0/2690224785 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f65080ad9e0 0x7f65080a8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T20:45:14.802 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.781+0000 7f650d4ef700 1 --2- 172.21.15.17:0/2690224785 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65080af450 0x7f65080af830 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:45:14.802 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.781+0000 7f650d4ef700 1 -- 172.21.15.17:0/2690224785 >> 172.21.15.17:0/2690224785 conn(0x7f65080612d0 msgr2=0x7f6508062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:45:14.803 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.781+0000 7f650d4ef700 1 -- 172.21.15.17:0/2690224785 shutdown_connections 2022-08-01T20:45:14.804 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.781+0000 7f650d4ef700 1 -- 172.21.15.17:0/2690224785 wait complete. 2022-08-01T20:45:14.804 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.785+0000 7f650d4ef700 1 Processor -- start 2022-08-01T20:45:14.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.785+0000 7f650d4ef700 1 -- start start 2022-08-01T20:45:14.805 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.785+0000 7f650d4ef700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65080ad4b0 0x7f65080b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:45:14.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.785+0000 7f650d4ef700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f65080ad9e0 0x7f65080b5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:45:14.806 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.785+0000 7f650d4ef700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65080af450 0x7f65080b5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:45:14.807 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.785+0000 7f650d4ef700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65080a9840 con 0x7f65080af450 2022-08-01T20:45:14.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.785+0000 7f650d4ef700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6508067e70 con 0x7f65080ad4b0 2022-08-01T20:45:14.808 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.785+0000 7f650d4ef700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65080b4ac0 con 0x7f65080ad9e0 2022-08-01T20:45:14.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.785+0000 7f650ccee700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65080af450 0x7f65080b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:45:14.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.785+0000 7f650ccee700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65080af450 0x7f65080b5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46354/0 (socket says 172.21.15.17:46354) 2022-08-01T20:45:14.809 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.785+0000 7f650ccee700 1 -- 172.21.15.17:0/2413546539 learned_addr learned my addr 172.21.15.17:0/2413546539 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T20:45:14.810 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.785+0000 7f6507fff700 1 --2- 172.21.15.17:0/2413546539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65080ad4b0 0x7f65080b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:45:14.810 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.785+0000 7f64ff7fe700 1 --2- 172.21.15.17:0/2413546539 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f65080ad9e0 0x7f65080b5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:45:14.811 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.785+0000 7f650ccee700 1 -- 172.21.15.17:0/2413546539 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f65080ad9e0 msgr2=0x7f65080b5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:45:14.811 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.785+0000 7f650ccee700 1 --2- 172.21.15.17:0/2413546539 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f65080ad9e0 0x7f65080b5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:45:14.811 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.785+0000 7f650ccee700 1 -- 172.21.15.17:0/2413546539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65080ad4b0 msgr2=0x7f65080b3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:45:14.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.785+0000 7f650ccee700 1 --2- 172.21.15.17:0/2413546539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65080ad4b0 0x7f65080b3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:45:14.812 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.785+0000 7f650ccee700 1 -- 172.21.15.17:0/2413546539 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f800b1e0 con 0x7f65080af450 2022-08-01T20:45:14.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.785+0000 7f64ff7fe700 1 --2- 172.21.15.17:0/2413546539 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f65080ad9e0 0x7f65080b5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-08-01T20:45:14.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.785+0000 7f650ccee700 1 --2- 172.21.15.17:0/2413546539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65080af450 0x7f65080b5870 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 rx=0x7f6500006cc0 tx=0x7f6500008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:45:14.813 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.785+0000 7f6505ffb700 1 -- 172.21.15.17:0/2413546539 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6500013620 con 0x7f65080af450 2022-08-01T20:45:14.814 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.785+0000 7f650d4ef700 1 -- 172.21.15.17:0/2413546539 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65080a95a0 con 0x7f65080af450 2022-08-01T20:45:14.814 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.785+0000 7f6505ffb700 1 -- 172.21.15.17:0/2413546539 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6500013780 con 0x7f65080af450 2022-08-01T20:45:14.814 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.785+0000 7f650d4ef700 1 -- 172.21.15.17:0/2413546539 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65080b4ee0 con 0x7f65080af450 2022-08-01T20:45:14.815 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.785+0000 7f6505ffb700 1 -- 172.21.15.17:0/2413546539 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6500013ae0 con 0x7f65080af450 2022-08-01T20:45:14.815 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.785+0000 7f650d4ef700 1 -- 172.21.15.17:0/2413546539 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65080b50a0 con 0x7f65080af450 2022-08-01T20:45:14.815 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.789+0000 7f6505ffb700 1 -- 172.21.15.17:0/2413546539 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6500013c40 con 0x7f65080af450 2022-08-01T20:45:14.816 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.789+0000 7f6505ffb700 1 --2- 172.21.15.17:0/2413546539 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f64ec0539e0 0x7f64ec055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:45:14.816 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.789+0000 7f6505ffb700 1 -- 172.21.15.17:0/2413546539 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f650006c460 con 0x7f65080af450 2022-08-01T20:45:14.816 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.789+0000 7f6507fff700 1 --2- 172.21.15.17:0/2413546539 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f64ec0539e0 0x7f64ec055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:45:14.817 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.789+0000 7f6507fff700 1 --2- 172.21.15.17:0/2413546539 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f64ec0539e0 0x7f64ec055ea0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7f64f801c000 tx=0x7f64f8015000).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:45:14.818 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.793+0000 7f6505ffb700 1 -- 172.21.15.17:0/2413546539 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f65000433c0 con 0x7f65080af450 2022-08-01T20:45:14.922 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.917+0000 7f650d4ef700 1 -- 172.21.15.17:0/2413546539 --> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f65080574e0 con 0x7f64ec0539e0 2022-08-01T20:45:14.932 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.925+0000 7f6505ffb700 1 -- 172.21.15.17:0/2413546539 <== mgr.4101 v2:172.21.15.77:6800/23151 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+68143 (secure 0 0 0) 0x7f65080574e0 con 0x7f64ec0539e0 2022-08-01T20:45:14.937 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-08-01T20:45:14.940 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.929+0000 7f650d4ef700 1 -- 172.21.15.17:0/2413546539 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f64ec0539e0 msgr2=0x7f64ec055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:45:14.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.929+0000 7f650d4ef700 1 --2- 172.21.15.17:0/2413546539 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f64ec0539e0 0x7f64ec055ea0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7f64f801c000 tx=0x7f64f8015000).stop 2022-08-01T20:45:14.941 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.929+0000 7f650d4ef700 1 -- 172.21.15.17:0/2413546539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65080af450 msgr2=0x7f65080b5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:45:14.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.929+0000 7f650d4ef700 1 --2- 172.21.15.17:0/2413546539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65080af450 0x7f65080b5870 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 rx=0x7f6500006cc0 tx=0x7f6500008040).stop 2022-08-01T20:45:14.942 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.929+0000 7f650d4ef700 1 -- 172.21.15.17:0/2413546539 shutdown_connections 2022-08-01T20:45:14.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.929+0000 7f650d4ef700 1 --2- 172.21.15.17:0/2413546539 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7f64ec0539e0 0x7f64ec055ea0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:45:14.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.929+0000 7f650d4ef700 1 --2- 172.21.15.17:0/2413546539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65080af450 0x7f65080b5870 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:45:14.943 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.929+0000 7f650d4ef700 1 --2- 172.21.15.17:0/2413546539 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7f65080ad9e0 0x7f65080b5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:45:14.944 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.929+0000 7f650d4ef700 1 --2- 172.21.15.17:0/2413546539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65080ad4b0 0x7f65080b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:45:14.945 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.929+0000 7f650d4ef700 1 -- 172.21.15.17:0/2413546539 >> 172.21.15.17:0/2413546539 conn(0x7f65080612d0 msgr2=0x7f650813f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:45:14.945 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.929+0000 7f650d4ef700 1 -- 172.21.15.17:0/2413546539 shutdown_connections 2022-08-01T20:45:14.946 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:14.929+0000 7f650d4ef700 1 -- 172.21.15.17:0/2413546539 wait complete. 2022-08-01T20:45:14.946 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2022-08-01T20:45:14.948 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":2293,"stamp":"2022-08-01T20:45:14.158490+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9465,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":8404664,"kb_used_data":15992,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746570056,"statfs":{"total":773094113280,"available":764487737344,"internally_reserved":8589934592,"allocated":16375808,"data_stored":7983484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003277"},"pg_stats":[{"pgid":"1.0","version":"161'4","reported_seq":144,"reported_epoch":161,"state":"active+clean","last_fresh":"2022-08-01T20:44:55.146095+0000","last_change":"2022-08-01T20:44:55.146360+0000","last_active":"2022-08-01T20:44:55.146095+0000","last_peered":"2022-08-01T20:44:55.146095+0000","last_clean":"2022-08-01T20:44:55.146095+0000","last_became_active":"2022-08-01T19:33:58.588671+0000","last_became_peered":"2022-08-01T19:33:58.588671+0000","last_unstale":"2022-08-01T20:44:55.146095+0000","last_undegraded":"2022-08-01T20:44:55.146095+0000","last_fullsized":"2022-08-01T20:44:55.146095+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'4","last_scrub_stamp":"2022-08-01T20:44:55.146094+0000","last_deep_scrub":"161'4","last_deep_scrub_stamp":"2022-08-01T20:44:55.146094+0000","last_clean_scrub_stamp":"2022-08-01T20:44:55.146094+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9465,"num_omap_keys":2,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":149,"reported_epoch":161,"state":"active+clean","last_fresh":"2022-08-01T20:44:56.505538+0000","last_change":"2022-08-01T20:44:56.505722+0000","last_active":"2022-08-01T20:44:56.505538+0000","last_peered":"2022-08-01T20:44:56.505538+0000","last_clean":"2022-08-01T20:44:56.505538+0000","last_became_active":"2022-08-01T19:34:03.245544+0000","last_became_peered":"2022-08-01T19:34:03.245544+0000","last_unstale":"2022-08-01T20:44:56.505538+0000","last_undegraded":"2022-08-01T20:44:56.505538+0000","last_fullsized":"2022-08-01T20:44:56.505538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T20:44:56.505538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T20:44:56.505538+0000","last_clean_scrub_stamp":"2022-08-01T20:44:56.505538+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":137,"reported_epoch":161,"state":"active+clean","last_fresh":"2022-08-01T20:44:57.191090+0000","last_change":"2022-08-01T20:44:57.191273+0000","last_active":"2022-08-01T20:44:57.191090+0000","last_peered":"2022-08-01T20:44:57.191090+0000","last_clean":"2022-08-01T20:44:57.191090+0000","last_became_active":"2022-08-01T19:34:03.245832+0000","last_became_peered":"2022-08-01T19:34:03.245832+0000","last_unstale":"2022-08-01T20:44:57.191090+0000","last_undegraded":"2022-08-01T20:44:57.191090+0000","last_fullsized":"2022-08-01T20:44:57.191090+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T20:44:57.191089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T20:44:57.191089+0000","last_clean_scrub_stamp":"2022-08-01T20:44:57.191089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":144,"reported_epoch":161,"state":"active+clean","last_fresh":"2022-08-01T20:44:56.776865+0000","last_change":"2022-08-01T20:44:56.777003+0000","last_active":"2022-08-01T20:44:56.776865+0000","last_peered":"2022-08-01T20:44:56.776865+0000","last_clean":"2022-08-01T20:44:56.776865+0000","last_became_active":"2022-08-01T19:34:03.244231+0000","last_became_peered":"2022-08-01T19:34:03.244231+0000","last_unstale":"2022-08-01T20:44:56.776865+0000","last_undegraded":"2022-08-01T20:44:56.776865+0000","last_fullsized":"2022-08-01T20:44:56.776865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T20:44:56.776864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T20:44:56.776864+0000","last_clean_scrub_stamp":"2022-08-01T20:44:56.776864+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":149,"reported_epoch":161,"state":"active+clean","last_fresh":"2022-08-01T20:44:54.488095+0000","last_change":"2022-08-01T20:44:54.488183+0000","last_active":"2022-08-01T20:44:54.488095+0000","last_peered":"2022-08-01T20:44:54.488095+0000","last_clean":"2022-08-01T20:44:54.488095+0000","last_became_active":"2022-08-01T19:34:03.245763+0000","last_became_peered":"2022-08-01T19:34:03.245763+0000","last_unstale":"2022-08-01T20:44:54.488095+0000","last_undegraded":"2022-08-01T20:44:54.488095+0000","last_fullsized":"2022-08-01T20:44:54.488095+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T20:44:54.488095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T20:44:54.488095+0000","last_clean_scrub_stamp":"2022-08-01T20:44:54.488095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":144,"reported_epoch":161,"state":"active+clean","last_fresh":"2022-08-01T20:44:53.940878+0000","last_change":"2022-08-01T20:44:53.941150+0000","last_active":"2022-08-01T20:44:53.940878+0000","last_peered":"2022-08-01T20:44:53.940878+0000","last_clean":"2022-08-01T20:44:53.940878+0000","last_became_active":"2022-08-01T19:34:03.661358+0000","last_became_peered":"2022-08-01T19:34:03.661358+0000","last_unstale":"2022-08-01T20:44:53.940878+0000","last_undegraded":"2022-08-01T20:44:53.940878+0000","last_fullsized":"2022-08-01T20:44:53.940878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T20:44:53.940877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T20:44:53.940877+0000","last_clean_scrub_stamp":"2022-08-01T20:44:53.940877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":137,"reported_epoch":161,"state":"active+clean","last_fresh":"2022-08-01T20:45:01.277661+0000","last_change":"2022-08-01T20:45:01.277744+0000","last_active":"2022-08-01T20:45:01.277661+0000","last_peered":"2022-08-01T20:45:01.277661+0000","last_clean":"2022-08-01T20:45:01.277661+0000","last_became_active":"2022-08-01T19:34:03.664627+0000","last_became_peered":"2022-08-01T19:34:03.664627+0000","last_unstale":"2022-08-01T20:45:01.277661+0000","last_undegraded":"2022-08-01T20:45:01.277661+0000","last_fullsized":"2022-08-01T20:45:01.277661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T20:45:01.277661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T20:45:01.277661+0000","last_clean_scrub_stamp":"2022-08-01T20:45:01.277661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":144,"reported_epoch":161,"state":"active+clean","last_fresh":"2022-08-01T20:44:57.918938+0000","last_change":"2022-08-01T20:44:57.919014+0000","last_active":"2022-08-01T20:44:57.918938+0000","last_peered":"2022-08-01T20:44:57.918938+0000","last_clean":"2022-08-01T20:44:57.918938+0000","last_became_active":"2022-08-01T19:34:03.243496+0000","last_became_peered":"2022-08-01T19:34:03.243496+0000","last_unstale":"2022-08-01T20:44:57.918938+0000","last_undegraded":"2022-08-01T20:44:57.918938+0000","last_fullsized":"2022-08-01T20:44:57.918938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T20:44:57.918937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T20:44:57.918937+0000","last_clean_scrub_stamp":"2022-08-01T20:44:57.918937+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":144,"reported_epoch":161,"state":"active+clean","last_fresh":"2022-08-01T20:45:01.694693+0000","last_change":"2022-08-01T20:45:01.694803+0000","last_active":"2022-08-01T20:45:01.694693+0000","last_peered":"2022-08-01T20:45:01.694693+0000","last_clean":"2022-08-01T20:45:01.694693+0000","last_became_active":"2022-08-01T19:34:03.245060+0000","last_became_peered":"2022-08-01T19:34:03.245060+0000","last_unstale":"2022-08-01T20:45:01.694693+0000","last_undegraded":"2022-08-01T20:45:01.694693+0000","last_fullsized":"2022-08-01T20:45:01.694693+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-08-01T20:45:01.694692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-08-01T20:45:01.694692+0000","last_clean_scrub_stamp":"2022-08-01T20:45:01.694692+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9465,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":15,"seq":64424510298,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050584,"kb_used_data":2000,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321256,"statfs":{"total":96636764160,"available":95560966144,"internally_reserved":1073741824,"allocated":2048000,"data_stored":998080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 20:44:19 2022","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.78300000000000003,"15min":0.79600000000000004},"min":{"1min":0.59799999999999998,"5min":0.55300000000000005,"15min":0.54000000000000004},"max":{"1min":1.0369999999999999,"5min":1.161,"15min":1.161},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.82199999999999995,"15min":0.82599999999999996},"min":{"1min":0.63800000000000001,"5min":0.59899999999999998,"15min":0.56699999999999995},"max":{"1min":1.1379999999999999,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.89700000000000002}]},{"osd":1,"last update":"Mon Aug 1 20:44:19 2022","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.84599999999999997,"15min":0.84699999999999998},"min":{"1min":0.65800000000000003,"5min":0.61399999999999999,"15min":0.55400000000000005},"max":{"1min":1.1779999999999999,"5min":1.26,"15min":1.26},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.83399999999999996,"15min":0.81999999999999995},"min":{"1min":0.67100000000000004,"5min":0.60499999999999998,"15min":0.51800000000000002},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.67800000000000005}]},{"osd":2,"last update":"Mon Aug 1 20:44:19 2022","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.86599999999999999,"15min":0.86699999999999999},"min":{"1min":0.63200000000000001,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.1100000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.83299999999999996,"15min":0.83699999999999997},"min":{"1min":0.68100000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.111,"5min":1.111,"15min":1.216},"last":0.93200000000000005}]},{"osd":3,"last update":"Mon Aug 1 20:44:19 2022","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.90700000000000003,"15min":0.88300000000000001},"min":{"1min":0.69499999999999995,"5min":0.64000000000000001,"15min":0.60699999999999998},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1699999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.93200000000000005,"15min":0.91600000000000004},"min":{"1min":0.66000000000000003,"5min":0.65700000000000003,"15min":0.63700000000000001},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.877}]},{"osd":4,"last update":"Mon Aug 1 20:44:19 2022","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79800000000000004,"15min":0.77500000000000002},"min":{"1min":0.622,"5min":0.54400000000000004,"15min":0.52400000000000002},"max":{"1min":1.0589999999999999,"5min":1.161,"15min":1.161},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.92500000000000004,"15min":0.89100000000000001},"min":{"1min":0.67000000000000004,"5min":0.58799999999999997,"15min":0.56399999999999995},"max":{"1min":1.145,"5min":1.22,"15min":1.264},"last":0.78100000000000003}]},{"osd":6,"last update":"Mon Aug 1 20:44:19 2022","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.93500000000000005,"15min":0.91300000000000003},"min":{"1min":0.58799999999999997,"5min":0.57199999999999995,"15min":0.56100000000000005},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.96199999999999997,"15min":0.93799999999999994},"min":{"1min":0.82199999999999995,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1619999999999999,"5min":1.2070000000000001,"15min":1.2829999999999999},"last":0.84799999999999998}]},{"osd":7,"last update":"Mon Aug 1 20:44:19 2022","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0209999999999999,"15min":0.96799999999999997},"min":{"1min":0.79300000000000004,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.64400000000000002},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0449999999999999,"15min":0.98599999999999999},"min":{"1min":0.70699999999999996,"5min":0.69599999999999995,"15min":0.66100000000000003},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.81899999999999995}]}]},{"osd":4,"up_from":15,"seq":64424510297,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1050584,"kb_used_data":2000,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321256,"statfs":{"total":96636764160,"available":95560966144,"internally_reserved":1073741824,"allocated":2048000,"data_stored":998080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 20:44:56 2022","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.79300000000000004,"15min":0.80100000000000005},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52400000000000002},"max":{"1min":0.999,"5min":1.165,"15min":1.165},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.79500000000000004,"15min":0.81699999999999995},"min":{"1min":0.61899999999999999,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.046,"5min":1.1719999999999999,"15min":1.194},"last":0.78800000000000003}]},{"osd":1,"last update":"Mon Aug 1 20:44:56 2022","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.80600000000000005,"15min":0.82499999999999996},"min":{"1min":0.63600000000000001,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.115,"5min":1.139,"15min":1.1779999999999999},"last":1.016},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.83899999999999997,"15min":0.83699999999999997},"min":{"1min":0.60799999999999998,"5min":0.5,"15min":0.5},"max":{"1min":1.016,"5min":1.117,"15min":1.1659999999999999},"last":0.93500000000000005}]},{"osd":2,"last update":"Mon Aug 1 20:44:56 2022","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.91200000000000003,"15min":0.92500000000000004},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59199999999999997},"max":{"1min":1.1759999999999999,"5min":1.1990000000000001,"15min":1.3080000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.94199999999999995,"15min":0.94599999999999995},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.60899999999999999},"max":{"1min":1.153,"5min":1.2569999999999999,"15min":1.4590000000000001},"last":0.96999999999999997}]},{"osd":3,"last update":"Mon Aug 1 20:44:56 2022","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.85299999999999998,"15min":0.85499999999999998},"min":{"1min":0.66500000000000004,"5min":0.63300000000000001,"15min":0.63100000000000001},"max":{"1min":1.244,"5min":1.244,"15min":1.2829999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.92200000000000004,"15min":0.92800000000000005},"min":{"1min":0.68500000000000005,"5min":0.66300000000000003,"15min":0.65900000000000003},"max":{"1min":1.1160000000000001,"5min":1.1890000000000001,"15min":1.28},"last":1.115}]},{"osd":5,"last update":"Mon Aug 1 20:44:56 2022","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.83599999999999997,"15min":0.83999999999999997},"min":{"1min":0.57199999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.88,"15min":0.877},"min":{"1min":0.58599999999999997,"5min":0.56399999999999995,"15min":0.55900000000000005},"max":{"1min":1.1619999999999999,"5min":1.298,"15min":1.3460000000000001},"last":0.996}]},{"osd":6,"last update":"Mon Aug 1 20:44:56 2022","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.92800000000000005,"15min":0.91100000000000003},"min":{"1min":0.63800000000000001,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.077,"5min":1.2669999999999999,"15min":1.292},"last":1.069},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.97599999999999998,"15min":0.96099999999999997},"min":{"1min":0.72499999999999998,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1970000000000001,"5min":1.331,"15min":1.5549999999999999},"last":1.0429999999999999}]},{"osd":7,"last update":"Mon Aug 1 20:44:56 2022","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.96299999999999997,"15min":0.95899999999999996},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.55800000000000005},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.599},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0449999999999999,"15min":1.04},"min":{"1min":0.77500000000000002,"5min":0.68200000000000005,"15min":0.64500000000000002},"max":{"1min":1.3680000000000001,"5min":1.409,"15min":1.4870000000000001},"last":1.1579999999999999}]}]},{"osd":7,"up_from":14,"seq":60129543002,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050584,"kb_used_data":2000,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321256,"statfs":{"total":96636764160,"available":95560966144,"internally_reserved":1073741824,"allocated":2048000,"data_stored":998080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 20:44:35 2022","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.88200000000000001,"15min":0.90500000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.1539999999999999,"5min":1.163,"15min":1.244},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.90000000000000002,"15min":0.89600000000000002},"min":{"1min":0.622,"5min":0.622,"15min":0.52100000000000002},"max":{"1min":1.0669999999999999,"5min":1.1499999999999999,"15min":1.1910000000000001},"last":0.81999999999999995}]},{"osd":1,"last update":"Mon Aug 1 20:44:35 2022","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.82799999999999996,"15min":0.84399999999999997},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.56100000000000005},"max":{"1min":1.0860000000000001,"5min":1.123,"15min":1.159},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82299999999999995,"15min":0.81599999999999995},"min":{"1min":0.65400000000000003,"5min":0.57599999999999996,"15min":0.52900000000000003},"max":{"1min":1.071,"5min":1.163,"15min":1.163},"last":0.751}]},{"osd":2,"last update":"Mon Aug 1 20:44:35 2022","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.84599999999999997,"15min":0.85099999999999998},"min":{"1min":0.63100000000000001,"5min":0.62,"15min":0.57899999999999996},"max":{"1min":0.98999999999999999,"5min":1.1739999999999999,"15min":1.3129999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.875,"5min":0.874,"15min":0.872},"min":{"1min":0.64900000000000002,"5min":0.64200000000000002,"15min":0.60299999999999998},"max":{"1min":1.0880000000000001,"5min":1.161,"15min":1.2370000000000001},"last":0.78800000000000003}]},{"osd":3,"last update":"Mon Aug 1 20:44:35 2022","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.86399999999999999,"15min":0.88700000000000001},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.2150000000000001},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.93000000000000005,"15min":0.91700000000000004},"min":{"1min":0.68799999999999994,"5min":0.65900000000000003,"15min":0.622},"max":{"1min":1.1080000000000001,"5min":1.268,"15min":1.3280000000000001},"last":0.73799999999999999}]},{"osd":4,"last update":"Mon Aug 1 20:44:35 2022","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83699999999999997,"15min":0.84199999999999997},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.56399999999999995},"max":{"1min":1.145,"5min":1.218,"15min":1.2529999999999999},"last":1.02},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.93100000000000005,"15min":0.90600000000000003},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.145,"5min":1.216,"15min":1.4019999999999999},"last":0.56899999999999995}]},{"osd":5,"last update":"Mon Aug 1 20:44:35 2022","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.84599999999999997,"15min":0.83699999999999997},"min":{"1min":0.626,"5min":0.56599999999999995,"15min":0.55200000000000005},"max":{"1min":1.042,"5min":1.157,"15min":1.1950000000000001},"last":0.625},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94599999999999995,"15min":0.94099999999999995},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.59799999999999998},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.385},"last":0.94699999999999995}]},{"osd":6,"last update":"Mon Aug 1 20:44:35 2022","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.93100000000000005,"15min":0.92400000000000004},"min":{"1min":0.72599999999999998,"5min":0.65500000000000003,"15min":0.57499999999999996},"max":{"1min":1.1519999999999999,"5min":1.202,"15min":1.2689999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.012},"min":{"1min":0.84699999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.2310000000000001,"5min":1.306,"15min":1.4510000000000001},"last":0.98399999999999999}]}]},{"osd":6,"up_from":14,"seq":60129543001,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050584,"kb_used_data":2000,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321256,"statfs":{"total":96636764160,"available":95560966144,"internally_reserved":1073741824,"allocated":2048000,"data_stored":998080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 20:44:33 2022","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.92700000000000005,"15min":0.93899999999999995},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.51600000000000001},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.88600000000000001,"15min":0.89000000000000001},"min":{"1min":0.69699999999999995,"5min":0.628,"15min":0.60199999999999998},"max":{"1min":1.0589999999999999,"5min":1.0840000000000001,"15min":1.149},"last":1.0680000000000001}]},{"osd":1,"last update":"Mon Aug 1 20:44:33 2022","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.879,"15min":0.84799999999999998},"min":{"1min":0.61699999999999999,"5min":0.57399999999999995,"15min":0.56699999999999995},"max":{"1min":1.1659999999999999,"5min":1.2230000000000001,"15min":1.2490000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.81200000000000006,"15min":0.81699999999999995},"min":{"1min":0.64100000000000001,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.054,"5min":1.054,"15min":1.161},"last":1.0149999999999999}]},{"osd":2,"last update":"Mon Aug 1 20:44:33 2022","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84899999999999998,"15min":0.83199999999999996},"min":{"1min":0.69199999999999995,"5min":0.626,"15min":0.60699999999999998},"max":{"1min":1.032,"5min":1.1890000000000001,"15min":1.4239999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.89300000000000002,"15min":0.91000000000000003},"min":{"1min":0.65000000000000002,"5min":0.61899999999999999,"15min":0.60599999999999998},"max":{"1min":1.169,"5min":1.169,"15min":1.2709999999999999},"last":1.0449999999999999}]},{"osd":3,"last update":"Mon Aug 1 20:44:33 2022","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.68999999999999995,"5min":0.65400000000000003,"15min":0.64100000000000001},"max":{"1min":1.085,"5min":1.103,"15min":1.157},"last":0.753},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.93400000000000005,"15min":0.93600000000000005},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.2050000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.91300000000000003}]},{"osd":4,"last update":"Mon Aug 1 20:44:33 2022","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.877,"15min":0.86799999999999999},"min":{"1min":0.67100000000000004,"5min":0.54100000000000004,"15min":0.54000000000000004},"max":{"1min":1.0900000000000001,"5min":1.1659999999999999,"15min":1.234},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.83499999999999996,"15min":0.82899999999999996},"min":{"1min":0.57799999999999996,"5min":0.55800000000000005,"15min":0.55000000000000004},"max":{"1min":1.0409999999999999,"5min":1.0980000000000001,"15min":1.2010000000000001},"last":0.81100000000000005}]},{"osd":5,"last update":"Mon Aug 1 20:44:33 2022","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.95899999999999996,"15min":0.93100000000000005},"min":{"1min":0.77500000000000002,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.127,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.90200000000000002,"15min":0.878},"min":{"1min":0.65000000000000002,"5min":0.60199999999999998,"15min":0.59799999999999998},"max":{"1min":1.121,"5min":1.1699999999999999,"15min":1.2390000000000001},"last":0.89100000000000001}]},{"osd":7,"last update":"Mon Aug 1 20:44:33 2022","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.89700000000000002,"15min":0.89600000000000002},"min":{"1min":0.66900000000000004,"5min":0.63600000000000001,"15min":0.59799999999999998},"max":{"1min":1.0660000000000001,"5min":1.264,"15min":1.264},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.97099999999999997,"15min":0.95899999999999996},"min":{"1min":0.76500000000000001,"5min":0.67400000000000004,"15min":0.61899999999999999},"max":{"1min":1.147,"5min":1.355,"15min":1.355},"last":0.98099999999999998}]}]},{"osd":3,"up_from":14,"seq":60129543001,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050580,"kb_used_data":1996,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321260,"statfs":{"total":96636764160,"available":95560970240,"internally_reserved":1073741824,"allocated":2043904,"data_stored":997502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 20:44:58 2022","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.90500000000000003,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.68600000000000005,"15min":0.67900000000000005},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.03,"5min":1.1020000000000001,"15min":1.194},"last":0.66800000000000004}]},{"osd":1,"last update":"Mon Aug 1 20:44:58 2022","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.65500000000000003,"15min":0.66200000000000003},"min":{"1min":0.495,"5min":0.495,"15min":0.44900000000000001},"max":{"1min":0.93400000000000005,"5min":0.96399999999999997,"15min":1.2170000000000001},"last":0.748},{"interface":"front","average":{"1min":0.749,"5min":0.77300000000000002,"15min":0.77700000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.0600000000000001,"5min":1.0960000000000001,"15min":1.266},"last":0.83499999999999996}]},{"osd":2,"last update":"Mon Aug 1 20:44:58 2022","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78100000000000003,"15min":0.78300000000000003},"min":{"1min":0.60699999999999998,"5min":0.58699999999999997,"15min":0.45900000000000002},"max":{"1min":1.0660000000000001,"5min":1.171,"15min":1.171},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.84399999999999997,"15min":0.83899999999999997},"min":{"1min":0.55900000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.181,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.98099999999999998}]},{"osd":4,"last update":"Mon Aug 1 20:44:38 2022","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.89800000000000002,"15min":0.89000000000000001},"min":{"1min":0.65000000000000002,"5min":0.61899999999999999,"15min":0.59699999999999998},"max":{"1min":1.145,"5min":1.3120000000000001,"15min":1.3400000000000001},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.93200000000000005,"15min":0.94299999999999995},"min":{"1min":0.70799999999999996,"5min":0.64800000000000002,"15min":0.64500000000000002},"max":{"1min":1.246,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.94699999999999995}]},{"osd":5,"last update":"Mon Aug 1 20:44:38 2022","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99099999999999999,"15min":1.0129999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.68899999999999995},"max":{"1min":1.2989999999999999,"5min":1.46,"15min":1.508},"last":1.3919999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.97299999999999998,"15min":0.96799999999999997},"min":{"1min":0.77300000000000002,"5min":0.71399999999999997,"15min":0.67900000000000005},"max":{"1min":1.2729999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.96199999999999997}]},{"osd":6,"last update":"Mon Aug 1 20:44:58 2022","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0289999999999999,"15min":1.034},"min":{"1min":0.81599999999999995,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.304,"5min":1.548,"15min":1.548},"last":1.147},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0780000000000001,"15min":1.093},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.264,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.337}]},{"osd":7,"last update":"Mon Aug 1 20:44:58 2022","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0549999999999999,"15min":1.0589999999999999},"min":{"1min":0.87,"5min":0.81999999999999995,"15min":0.76800000000000002},"max":{"1min":1.2230000000000001,"5min":1.468,"15min":1.4710000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.107,"15min":1.105},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.78400000000000003},"max":{"1min":1.325,"5min":1.496,"15min":1.496},"last":1.1890000000000001}]}]},{"osd":2,"up_from":14,"seq":60129543003,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050580,"kb_used_data":1996,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321260,"statfs":{"total":96636764160,"available":95560970240,"internally_reserved":1073741824,"allocated":2043904,"data_stored":997502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 20:44:22 2022","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58799999999999997,"15min":0.60499999999999998},"min":{"1min":0.47799999999999998,"5min":0.46500000000000002,"15min":0.436},"max":{"1min":0.77200000000000002,"5min":0.88400000000000001,"15min":1.1919999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72399999999999998,"15min":0.71699999999999997},"min":{"1min":0.503,"5min":0.5,"15min":0.47799999999999998},"max":{"1min":1.0509999999999999,"5min":1.129,"15min":1.179},"last":0.88400000000000001}]},{"osd":1,"last update":"Mon Aug 1 20:44:22 2022","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.81399999999999995,"15min":0.80800000000000005},"min":{"1min":0.66400000000000003,"5min":0.55100000000000005,"15min":0.52100000000000002},"max":{"1min":1.0169999999999999,"5min":1.147,"15min":1.147},"last":1.012},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.69799999999999995,"15min":0.69999999999999996},"min":{"1min":0.50800000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.89500000000000002,"5min":1.0509999999999999,"15min":1.0620000000000001},"last":0.77500000000000002}]},{"osd":3,"last update":"Mon Aug 1 20:44:22 2022","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.68999999999999995,"15min":0.70899999999999996},"min":{"1min":0.53800000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.91000000000000003,"5min":0.96799999999999997,"15min":1.2569999999999999},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.77900000000000003,"15min":0.79800000000000004},"min":{"1min":0.61099999999999999,"5min":0.56000000000000005,"15min":0.55800000000000005},"max":{"1min":0.97299999999999998,"5min":1.0629999999999999,"15min":1.24},"last":0.70999999999999996}]},{"osd":4,"last update":"Mon Aug 1 20:44:22 2022","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.90900000000000003,"15min":0.90400000000000003},"min":{"1min":0.68100000000000005,"5min":0.67400000000000004,"15min":0.60199999999999998},"max":{"1min":1.2669999999999999,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.93100000000000005,"15min":0.93200000000000005},"min":{"1min":0.72699999999999998,"5min":0.70699999999999996,"15min":0.67300000000000004},"max":{"1min":1.365,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.81000000000000005}]},{"osd":5,"last update":"Mon Aug 1 20:44:22 2022","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.97899999999999998,"15min":0.996},"min":{"1min":0.77000000000000002,"5min":0.71899999999999997,"15min":0.71399999999999997},"max":{"1min":1.1240000000000001,"5min":1.411,"15min":1.411},"last":1.105},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.95899999999999996,"15min":0.95299999999999996},"min":{"1min":0.72999999999999998,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.234,"5min":1.341,"15min":1.421},"last":1.0920000000000001}]},{"osd":6,"last update":"Mon Aug 1 20:44:22 2022","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":0.98999999999999999,"15min":0.98399999999999999},"min":{"1min":0.80600000000000005,"5min":0.72699999999999998,"15min":0.68799999999999994},"max":{"1min":1.319,"5min":1.319,"15min":1.4019999999999999},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.014,"5min":1.018,"15min":1.0249999999999999},"min":{"1min":0.80000000000000004,"5min":0.75900000000000001,"15min":0.71499999999999997},"max":{"1min":1.3120000000000001,"5min":1.383,"15min":1.383},"last":1.044}]},{"osd":7,"last update":"Mon Aug 1 20:44:22 2022","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.077,"15min":1.081},"min":{"1min":0.82399999999999995,"5min":0.76100000000000001,"15min":0.72199999999999998},"max":{"1min":1.2010000000000001,"5min":1.524,"15min":1.5820000000000001},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.054,"15min":1.0589999999999999},"min":{"1min":0.82699999999999996,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.494},"last":1.1479999999999999}]}]},{"osd":1,"up_from":14,"seq":60129543002,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050584,"kb_used_data":2000,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321256,"statfs":{"total":96636764160,"available":95560966144,"internally_reserved":1073741824,"allocated":2048000,"data_stored":998080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Aug 1 20:44:27 2022","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63100000000000001,"15min":0.624},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.42599999999999999},"max":{"1min":0.85599999999999998,"5min":0.90200000000000002,"15min":1.0649999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.752,"15min":0.747},"min":{"1min":0.62,"5min":0.61199999999999999,"15min":0.55900000000000005},"max":{"1min":0.879,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.875}]},{"osd":2,"last update":"Mon Aug 1 20:44:27 2022","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.65900000000000003,"15min":0.67400000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.46100000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.1459999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.82999999999999996,"15min":0.83799999999999997},"min":{"1min":0.66000000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.056,"5min":1.24,"15min":1.24},"last":0.84099999999999997}]},{"osd":3,"last update":"Mon Aug 1 20:44:27 2022","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.77100000000000002,"15min":0.77800000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0189999999999999,"5min":1.1779999999999999,"15min":1.228},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.72099999999999997,"15min":0.70699999999999996},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.92300000000000004,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.61599999999999999}]},{"osd":4,"last update":"Mon Aug 1 20:44:27 2022","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88900000000000001,"15min":0.873},"min":{"1min":0.69299999999999995,"5min":0.65300000000000002,"15min":0.629},"max":{"1min":1.21,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.97599999999999998,"15min":0.97699999999999998},"min":{"1min":0.74399999999999999,"5min":0.72299999999999998,"15min":0.67300000000000004},"max":{"1min":1.1180000000000001,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.92800000000000005}]},{"osd":5,"last update":"Mon Aug 1 20:44:27 2022","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.94499999999999995,"15min":0.93000000000000005},"min":{"1min":0.73199999999999998,"5min":0.70299999999999996,"15min":0.65200000000000002},"max":{"1min":1.3120000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.96899999999999997,"15min":0.95599999999999996},"min":{"1min":0.79700000000000004,"5min":0.70499999999999996,"15min":0.66100000000000003},"max":{"1min":1.3799999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":1.0640000000000001}]},{"osd":6,"last update":"Mon Aug 1 20:44:27 2022","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.97499999999999998,"15min":0.98099999999999998},"min":{"1min":0.72899999999999998,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.2210000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.087,"5min":1.0660000000000001,"15min":1.054},"min":{"1min":0.84799999999999998,"5min":0.82299999999999995,"15min":0.66800000000000004},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.7010000000000001},"last":1.103}]},{"osd":7,"last update":"Mon Aug 1 20:44:27 2022","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0389999999999999,"15min":1.024},"min":{"1min":0.88700000000000001,"5min":0.73599999999999999,"15min":0.72099999999999997},"max":{"1min":1.351,"5min":1.587,"15min":1.587},"last":1.151},{"interface":"front","average":{"1min":1.121,"5min":1.0920000000000001,"15min":1.0720000000000001},"min":{"1min":0.90300000000000002,"5min":0.77400000000000002,"15min":0.71899999999999997},"max":{"1min":1.3759999999999999,"5min":1.482,"15min":1.6459999999999999},"last":1.226}]}]},{"osd":0,"up_from":14,"seq":60129543001,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050584,"kb_used_data":2000,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321256,"statfs":{"total":96636764160,"available":95560966144,"internally_reserved":1073741824,"allocated":2048000,"data_stored":998080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Aug 1 20:44:12 2022","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63700000000000001,"15min":0.64000000000000001},"min":{"1min":0.47299999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88300000000000001,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73699999999999999,"15min":0.753},"min":{"1min":0.63400000000000001,"5min":0.59899999999999998,"15min":0.51800000000000002},"max":{"1min":0.873,"5min":0.98999999999999999,"15min":1.0640000000000001},"last":0.78400000000000003}]},{"osd":2,"last update":"Mon Aug 1 20:44:12 2022","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.65400000000000003},"min":{"1min":0.51000000000000001,"5min":0.504,"15min":0.504},"max":{"1min":0.91800000000000004,"5min":1.1020000000000001,"15min":1.2390000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.67700000000000005,"15min":0.67200000000000004},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.47199999999999998},"max":{"1min":0.91300000000000003,"5min":1.1659999999999999,"15min":1.2929999999999999},"last":0.621}]},{"osd":3,"last update":"Mon Aug 1 20:44:12 2022","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.72799999999999998,"15min":0.73599999999999999},"min":{"1min":0.51300000000000001,"5min":0.50600000000000001,"15min":0.498},"max":{"1min":0.89000000000000001,"5min":1.1180000000000001,"15min":1.1299999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.873,"15min":0.874},"min":{"1min":0.72299999999999998,"5min":0.69199999999999995,"15min":0.64100000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.228},"last":0.92600000000000005}]},{"osd":4,"last update":"Mon Aug 1 20:44:12 2022","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.94999999999999996,"15min":0.93799999999999994},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.67900000000000005},"max":{"1min":1.1579999999999999,"5min":1.3740000000000001,"15min":1.413},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.93799999999999994,"15min":0.94299999999999995},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.66200000000000003},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.84299999999999997}]},{"osd":5,"last update":"Mon Aug 1 20:44:12 2022","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.90900000000000003,"15min":0.88700000000000001},"min":{"1min":0.746,"5min":0.70299999999999996,"15min":0.65300000000000002},"max":{"1min":1.0980000000000001,"5min":1.2150000000000001,"15min":1.397},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0229999999999999},"min":{"1min":0.85499999999999998,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.222,"5min":1.452,"15min":1.508},"last":1.109}]},{"osd":6,"last update":"Mon Aug 1 20:44:12 2022","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.0600000000000001,"15min":1.0580000000000001},"min":{"1min":0.81100000000000005,"5min":0.77700000000000002,"15min":0.68400000000000005},"max":{"1min":1.29,"5min":1.534,"15min":1.546},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0269999999999999,"15min":1.032},"min":{"1min":0.84999999999999998,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.2230000000000001,"5min":1.3999999999999999,"15min":1.5189999999999999},"last":0.996}]},{"osd":7,"last update":"Mon Aug 1 20:44:12 2022","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.0409999999999999,"15min":1.0269999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.67900000000000005},"max":{"1min":1.347,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.121,"5min":1.129,"15min":1.129},"min":{"1min":0.77400000000000002,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.3720000000000001,"5min":1.4330000000000001,"15min":1.6559999999999999},"last":1.163}]}]}]}} 2022-08-01T20:45:14.954 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2022-08-01T20:45:15.062 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.061+0000 7fcf81263700 1 Processor -- start 2022-08-01T20:45:15.064 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.061+0000 7fcf81263700 1 -- start start 2022-08-01T20:45:15.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.061+0000 7fcf81263700 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcf7c144890 0x7fcf7c144d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:45:15.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.061+0000 7fcf81263700 1 --2- >> v2:172.21.15.77:3300/0 conn(0x7fcf7c0ac750 0x7fcf7c0acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-08-01T20:45:15.073 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.061+0000 7fcf81263700 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf7c060990 con 0x7fcf7c0ab930 2022-08-01T20:45:15.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.061+0000 7fcf81263700 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf7c148120 con 0x7fcf7c144890 2022-08-01T20:45:15.074 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.061+0000 7fcf81263700 1 -- --> v2:172.21.15.77:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf7c145200 con 0x7fcf7c0ac750 2022-08-01T20:45:15.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.061+0000 7fcf7affd700 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fcf7c0ab930 0x7fcf7c0abd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:38552/0 (socket says 172.21.15.17:38552) 2022-08-01T20:45:15.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.061+0000 7fcf7affd700 1 -- 172.21.15.17:0/2947195226 learned_addr learned my addr 172.21.15.17:0/2947195226 (peer_addr_for_me v1:172.21.15.17:0/0) 2022-08-01T20:45:15.075 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.061+0000 7fcf73fff700 1 --2- 172.21.15.17:0/2947195226 >> v2:172.21.15.17:3300/0 conn(0x7fcf7c144890 0x7fcf7c144d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:45:15.076 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.061+0000 7fcf7b7fe700 1 --2- 172.21.15.17:0/2947195226 >> v2:172.21.15.77:3300/0 conn(0x7fcf7c0ac750 0x7fcf7c0acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:45:15.076 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.061+0000 7fcf7a7fc700 1 -- 172.21.15.17:0/2947195226 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1509675812 0 0) 0x7fcf7c060990 con 0x7fcf7c0ab930 2022-08-01T20:45:15.076 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.061+0000 7fcf7a7fc700 1 -- 172.21.15.17:0/2947195226 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf5c003260 con 0x7fcf7c0ab930 2022-08-01T20:45:15.077 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.061+0000 7fcf73fff700 1 -- 172.21.15.17:0/2947195226 >> v2:172.21.15.77:3300/0 conn(0x7fcf7c0ac750 msgr2=0x7fcf7c0acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-08-01T20:45:15.077 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.061+0000 7fcf73fff700 1 --2- 172.21.15.17:0/2947195226 >> v2:172.21.15.77:3300/0 conn(0x7fcf7c0ac750 0x7fcf7c0acbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T20:45:15.077 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.061+0000 7fcf73fff700 1 -- 172.21.15.17:0/2947195226 >> v1:172.21.15.17:6789/0 conn(0x7fcf7c0ab930 legacy=0x7fcf7c0abd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:45:15.077 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.061+0000 7fcf73fff700 1 -- 172.21.15.17:0/2947195226 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf7c060050 con 0x7fcf7c144890 2022-08-01T20:45:15.078 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.061+0000 7fcf73fff700 1 --2- 172.21.15.17:0/2947195226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf7c144890 0x7fcf7c144d10 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 rx=0x7fcf7c0aabd0 tx=0x7fcf64006df0).ready entity=mon.0 client_cookie=70b9c93da7b06026 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:45:15.078 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.061+0000 7fcf7a7fc700 1 -- 172.21.15.17:0/2947195226 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf6400d4a0 con 0x7fcf7c144890 2022-08-01T20:45:15.079 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.061+0000 7fcf7a7fc700 1 -- 172.21.15.17:0/2947195226 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf64000e90 con 0x7fcf7c144890 2022-08-01T20:45:15.080 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.061+0000 7fcf7a7fc700 1 -- 172.21.15.17:0/2947195226 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf6400d7f0 con 0x7fcf7c144890 2022-08-01T20:45:15.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.061+0000 7fcf81263700 1 -- 172.21.15.17:0/2947195226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf7c144890 msgr2=0x7fcf7c144d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:45:15.081 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.061+0000 7fcf81263700 1 --2- 172.21.15.17:0/2947195226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf7c144890 0x7fcf7c144d10 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 rx=0x7fcf7c0aabd0 tx=0x7fcf64006df0).stop 2022-08-01T20:45:15.082 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.061+0000 7fcf81263700 1 -- 172.21.15.17:0/2947195226 shutdown_connections 2022-08-01T20:45:15.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.061+0000 7fcf81263700 1 --2- 172.21.15.17:0/2947195226 >> v2:172.21.15.77:3300/0 conn(0x7fcf7c0ac750 0x7fcf7c0acbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-08-01T20:45:15.083 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.061+0000 7fcf81263700 1 --2- 172.21.15.17:0/2947195226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf7c144890 0x7fcf7c144d10 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:45:15.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.061+0000 7fcf81263700 1 -- 172.21.15.17:0/2947195226 >> 172.21.15.17:0/2947195226 conn(0x7fcf7c0b2b00 msgr2=0x7fcf7c0b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:45:15.084 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.061+0000 7fcf81263700 1 -- 172.21.15.17:0/2947195226 shutdown_connections 2022-08-01T20:45:15.085 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.065+0000 7fcf81263700 1 -- 172.21.15.17:0/2947195226 wait complete. 2022-08-01T20:45:15.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.065+0000 7fcf81263700 1 Processor -- start 2022-08-01T20:45:15.086 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.065+0000 7fcf81263700 1 -- start start 2022-08-01T20:45:15.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.065+0000 7fcf81263700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf7c0ab930 0x7fcf7c15a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:45:15.087 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.065+0000 7fcf81263700 1 --2- >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcf7c0ac750 0x7fcf7c146970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:45:15.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.065+0000 7fcf81263700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf7c144890 0x7fcf7c14b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:45:15.088 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.065+0000 7fcf81263700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c0af380 con 0x7fcf7c144890 2022-08-01T20:45:15.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.065+0000 7fcf81263700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c1576c0 con 0x7fcf7c0ab930 2022-08-01T20:45:15.089 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.065+0000 7fcf81263700 1 -- --> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c157960 con 0x7fcf7c0ac750 2022-08-01T20:45:15.090 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.065+0000 7fcf7b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf7c144890 0x7fcf7c14b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:45:15.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.065+0000 7fcf7b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf7c144890 0x7fcf7c14b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46368/0 (socket says 172.21.15.17:46368) 2022-08-01T20:45:15.091 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.065+0000 7fcf7b7fe700 1 -- 172.21.15.17:0/2020745738 learned_addr learned my addr 172.21.15.17:0/2020745738 (peer_addr_for_me v2:172.21.15.17:0/0) 2022-08-01T20:45:15.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.065+0000 7fcf7affd700 1 --2- 172.21.15.17:0/2020745738 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf7c0ab930 0x7fcf7c15a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:45:15.092 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.065+0000 7fcf73fff700 1 --2- 172.21.15.17:0/2020745738 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcf7c0ac750 0x7fcf7c146970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:45:15.093 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.069+0000 7fcf7b7fe700 1 -- 172.21.15.17:0/2020745738 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcf7c0ac750 msgr2=0x7fcf7c146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:45:15.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.069+0000 7fcf7b7fe700 1 --2- 172.21.15.17:0/2020745738 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcf7c0ac750 0x7fcf7c146970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:45:15.094 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.069+0000 7fcf7b7fe700 1 -- 172.21.15.17:0/2020745738 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf7c0ab930 msgr2=0x7fcf7c15a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:45:15.095 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.069+0000 7fcf7b7fe700 1 --2- 172.21.15.17:0/2020745738 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf7c0ab930 0x7fcf7c15a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:45:15.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.069+0000 7fcf7b7fe700 1 -- 172.21.15.17:0/2020745738 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf6400b180 con 0x7fcf7c144890 2022-08-01T20:45:15.096 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.069+0000 7fcf7b7fe700 1 --2- 172.21.15.17:0/2020745738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf7c144890 0x7fcf7c14b480 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 rx=0x7fcf74006e00 tx=0x7fcf740091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:45:15.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.069+0000 7fcf78ff9700 1 -- 172.21.15.17:0/2020745738 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf7400c640 con 0x7fcf7c144890 2022-08-01T20:45:15.097 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.069+0000 7fcf78ff9700 1 -- 172.21.15.17:0/2020745738 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf7400c7a0 con 0x7fcf7c144890 2022-08-01T20:45:15.098 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.069+0000 7fcf81263700 1 -- 172.21.15.17:0/2020745738 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf7c146e60 con 0x7fcf7c144890 2022-08-01T20:45:15.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.069+0000 7fcf78ff9700 1 -- 172.21.15.17:0/2020745738 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf7400cb00 con 0x7fcf7c144890 2022-08-01T20:45:15.099 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.069+0000 7fcf81263700 1 -- 172.21.15.17:0/2020745738 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf7c063360 con 0x7fcf7c144890 2022-08-01T20:45:15.100 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.069+0000 7fcf81263700 1 -- 172.21.15.17:0/2020745738 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf7c148340 con 0x7fcf7c144890 2022-08-01T20:45:15.101 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.069+0000 7fcf78ff9700 1 -- 172.21.15.17:0/2020745738 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcf7400cc60 con 0x7fcf7c144890 2022-08-01T20:45:15.103 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.069+0000 7fcf78ff9700 1 --2- 172.21.15.17:0/2020745738 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcf60053a10 0x7fcf60055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-08-01T20:45:15.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.069+0000 7fcf78ff9700 1 -- 172.21.15.17:0/2020745738 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5686+0+0 (secure 0 0 0) 0x7fcf7406c060 con 0x7fcf7c144890 2022-08-01T20:45:15.104 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.069+0000 7fcf7affd700 1 --2- 172.21.15.17:0/2020745738 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcf60053a10 0x7fcf60055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-08-01T20:45:15.105 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.073+0000 7fcf7affd700 1 --2- 172.21.15.17:0/2020745738 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcf60053a10 0x7fcf60055ed0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7fcf6c00d000 tx=0x7fcf6c009040).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-08-01T20:45:15.106 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.077+0000 7fcf78ff9700 1 -- 172.21.15.17:0/2020745738 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcf74043060 con 0x7fcf7c144890 2022-08-01T20:45:15.209 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.205+0000 7fcf81263700 1 -- 172.21.15.17:0/2020745738 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fcf7c0574e0 con 0x7fcf7c144890 2022-08-01T20:45:15.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.213+0000 7fcf78ff9700 1 -- 172.21.15.17:0/2020745738 <== mon.0 v2:172.21.15.17:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fcf74043060 con 0x7fcf7c144890 2022-08-01T20:45:15.219 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.217+0000 7fcf78ff9700 1 -- 172.21.15.17:0/2020745738 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v25)=0 v25) v1 ==== 123+0+0 (secure 0 0 0) 0x7fcf74046680 con 0x7fcf7c144890 2022-08-01T20:45:15.223 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.221+0000 7fcf81263700 1 -- 172.21.15.17:0/2020745738 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcf60053a10 msgr2=0x7fcf60055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:45:15.224 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.221+0000 7fcf81263700 1 --2- 172.21.15.17:0/2020745738 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcf60053a10 0x7fcf60055ed0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7fcf6c00d000 tx=0x7fcf6c009040).stop 2022-08-01T20:45:15.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.221+0000 7fcf81263700 1 -- 172.21.15.17:0/2020745738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf7c144890 msgr2=0x7fcf7c14b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-08-01T20:45:15.225 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.221+0000 7fcf81263700 1 --2- 172.21.15.17:0/2020745738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf7c144890 0x7fcf7c14b480 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 rx=0x7fcf74006e00 tx=0x7fcf740091d0).stop 2022-08-01T20:45:15.226 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.221+0000 7fcf81263700 1 -- 172.21.15.17:0/2020745738 shutdown_connections 2022-08-01T20:45:15.227 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.221+0000 7fcf81263700 1 --2- 172.21.15.17:0/2020745738 >> [v2:172.21.15.77:6800/23151,v1:172.21.15.77:6801/23151] conn(0x7fcf60053a10 0x7fcf60055ed0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:45:15.228 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.221+0000 7fcf81263700 1 --2- 172.21.15.17:0/2020745738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf7c144890 0x7fcf7c14b480 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:45:15.229 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.221+0000 7fcf81263700 1 --2- 172.21.15.17:0/2020745738 >> [v2:172.21.15.77:3301/0,v1:172.21.15.77:6790/0] conn(0x7fcf7c0ac750 0x7fcf7c146970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:45:15.230 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.221+0000 7fcf81263700 1 --2- 172.21.15.17:0/2020745738 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf7c0ab930 0x7fcf7c15a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-08-01T20:45:15.231 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.221+0000 7fcf81263700 1 -- 172.21.15.17:0/2020745738 >> 172.21.15.17:0/2020745738 conn(0x7fcf7c0b2b00 msgr2=0x7fcf7c0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-08-01T20:45:15.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.221+0000 7fcf81263700 1 -- 172.21.15.17:0/2020745738 shutdown_connections 2022-08-01T20:45:15.232 INFO:teuthology.orchestra.run.smithi017.stderr:2022-08-01T20:45:15.221+0000 7fcf81263700 1 -- 172.21.15.17:0/2020745738 wait complete. 2022-08-01T20:45:15.236 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/contextutil.py", line 33, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/ceph.py", line 1858, in task yield File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/run_tasks.py", line 188, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/ceph_fuse.py", line 160, in task mount.umount_wait() File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/fuse_mount.py", line 326, in umount_wait if not (self.is_mounted() and self.fuse_daemon): File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted proc.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 148, in wait greenlet.get(block=True,timeout=60) File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2022-08-01T20:45:15.239 INFO:teuthology.misc:Shutting down mds daemons... 2022-08-01T20:45:15.240 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-08-01T20:45:15.241 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T20:45:15.258 INFO:tasks.ceph.mds.a:Stopped 2022-08-01T20:45:15.261 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2022-08-01T20:45:15.262 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T20:45:15.318 INFO:tasks.ceph.mds.c:Stopped 2022-08-01T20:45:15.319 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-08-01T20:45:15.320 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T20:45:15.335 INFO:tasks.ceph.mds.b:Stopped 2022-08-01T20:45:15.336 INFO:teuthology.misc:Shutting down osd daemons... 2022-08-01T20:45:15.337 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2022-08-01T20:45:15.337 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T20:45:15.409 INFO:tasks.ceph.osd.0:Stopped 2022-08-01T20:45:15.410 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2022-08-01T20:45:15.410 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T20:45:15.481 INFO:tasks.ceph.osd.1:Stopped 2022-08-01T20:45:15.482 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2022-08-01T20:45:15.483 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T20:45:15.554 INFO:tasks.ceph.osd.2:Stopped 2022-08-01T20:45:15.555 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2022-08-01T20:45:15.556 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T20:45:15.625 INFO:tasks.ceph.osd.3:Stopped 2022-08-01T20:45:15.626 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2022-08-01T20:45:15.627 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T20:45:15.691 INFO:tasks.ceph.osd.4:Stopped 2022-08-01T20:45:15.693 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2022-08-01T20:45:15.695 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T20:45:15.773 INFO:tasks.ceph.osd.5:Stopped 2022-08-01T20:45:15.774 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2022-08-01T20:45:15.775 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T20:45:15.840 INFO:tasks.ceph.osd.6:Stopped 2022-08-01T20:45:15.842 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2022-08-01T20:45:15.844 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T20:45:15.932 INFO:tasks.ceph.osd.7:Stopped 2022-08-01T20:45:15.934 INFO:teuthology.misc:Shutting down mgr daemons... 2022-08-01T20:45:15.936 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2022-08-01T20:45:15.937 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T20:45:15.966 INFO:tasks.ceph.mgr.y:Stopped 2022-08-01T20:45:15.969 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2022-08-01T20:45:15.970 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T20:45:16.000 INFO:tasks.ceph.mgr.x:Stopped 2022-08-01T20:45:16.002 INFO:teuthology.misc:Shutting down mon daemons... 2022-08-01T20:45:16.006 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2022-08-01T20:45:16.007 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T20:45:16.028 INFO:tasks.ceph.mon.a:Stopped 2022-08-01T20:45:16.030 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2022-08-01T20:45:16.031 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T20:45:16.066 INFO:tasks.ceph.mon.b:Stopped 2022-08-01T20:45:16.068 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2022-08-01T20:45:16.068 INFO:teuthology.orchestra.run:waiting for 300 2022-08-01T20:45:16.089 INFO:tasks.ceph.mon.c:Stopped 2022-08-01T20:45:16.090 INFO:tasks.ceph:Checking cluster log for badness... 2022-08-01T20:45:16.091 DEBUG:teuthology.orchestra.run.smithi017:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'evicting unresponsive client' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'has not responded to cap revoke by MDS for over' | egrep -v MDS_CLIENT_LATE_RELEASE | egrep -v 'responding to mclientcaps' | head -n 1 2022-08-01T20:45:16.114 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi017.front.sepia.ceph.com 2022-08-01T20:45:16.115 DEBUG:teuthology.orchestra.run.smithi017:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2022-08-01T20:45:16.405 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi017.front.sepia.ceph.com 2022-08-01T20:45:16.407 DEBUG:teuthology.orchestra.run.smithi017:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2022-08-01T20:45:16.637 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi017.front.sepia.ceph.com 2022-08-01T20:45:16.638 DEBUG:teuthology.orchestra.run.smithi017:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2022-08-01T20:45:16.820 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi017.front.sepia.ceph.com 2022-08-01T20:45:16.821 DEBUG:teuthology.orchestra.run.smithi017:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2022-08-01T20:45:17.010 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi077.front.sepia.ceph.com 2022-08-01T20:45:17.011 DEBUG:teuthology.orchestra.run.smithi077:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2022-08-01T20:45:17.399 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi077.front.sepia.ceph.com 2022-08-01T20:45:17.400 DEBUG:teuthology.orchestra.run.smithi077:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2022-08-01T20:45:17.662 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi077.front.sepia.ceph.com 2022-08-01T20:45:17.663 DEBUG:teuthology.orchestra.run.smithi077:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2022-08-01T20:45:17.881 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi077.front.sepia.ceph.com 2022-08-01T20:45:17.882 DEBUG:teuthology.orchestra.run.smithi077:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2022-08-01T20:45:18.039 INFO:tasks.ceph:Archiving mon data... 2022-08-01T20:45:18.041 DEBUG:teuthology.misc:Transferring archived files from smithi017:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2022-08-01_19:07:06-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954722/data/mon.a.tgz 2022-08-01T20:45:18.043 DEBUG:teuthology.orchestra.run.smithi017:> mktemp 2022-08-01T20:45:18.052 INFO:teuthology.orchestra.run.smithi017.stdout:/tmp/tmp.p8cRJCOFBf 2022-08-01T20:45:18.053 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.p8cRJCOFBf 2022-08-01T20:45:18.310 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0666 /tmp/tmp.p8cRJCOFBf 2022-08-01T20:45:18.335 DEBUG:teuthology.orchestra.remote:smithi017:/tmp/tmp.p8cRJCOFBf is 519KB 2022-08-01T20:45:18.407 DEBUG:teuthology.orchestra.run.smithi017:> rm -fr /tmp/tmp.p8cRJCOFBf 2022-08-01T20:45:18.415 DEBUG:teuthology.misc:Transferring archived files from smithi077:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2022-08-01_19:07:06-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954722/data/mon.b.tgz 2022-08-01T20:45:18.416 DEBUG:teuthology.orchestra.run.smithi077:> mktemp 2022-08-01T20:45:18.423 INFO:teuthology.orchestra.run.smithi077.stdout:/tmp/tmp.zqU971CIjo 2022-08-01T20:45:18.427 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.zqU971CIjo 2022-08-01T20:45:18.692 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 0666 /tmp/tmp.zqU971CIjo 2022-08-01T20:45:18.722 DEBUG:teuthology.orchestra.remote:smithi077:/tmp/tmp.zqU971CIjo is 656KB 2022-08-01T20:45:18.806 DEBUG:teuthology.orchestra.run.smithi077:> rm -fr /tmp/tmp.zqU971CIjo 2022-08-01T20:45:18.816 DEBUG:teuthology.misc:Transferring archived files from smithi077:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2022-08-01_19:07:06-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954722/data/mon.c.tgz 2022-08-01T20:45:18.817 DEBUG:teuthology.orchestra.run.smithi077:> mktemp 2022-08-01T20:45:18.865 INFO:teuthology.orchestra.run.smithi077.stdout:/tmp/tmp.S9PAtDeGw7 2022-08-01T20:45:18.866 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.S9PAtDeGw7 2022-08-01T20:45:19.095 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 0666 /tmp/tmp.S9PAtDeGw7 2022-08-01T20:45:19.120 DEBUG:teuthology.orchestra.remote:smithi077:/tmp/tmp.S9PAtDeGw7 is 656KB 2022-08-01T20:45:19.210 DEBUG:teuthology.orchestra.run.smithi077:> rm -fr /tmp/tmp.S9PAtDeGw7 2022-08-01T20:45:19.220 INFO:tasks.ceph:Cleaning ceph cluster... 2022-08-01T20:45:19.222 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2022-08-01T20:45:19.226 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2022-08-01T20:45:19.264 DEBUG:teuthology.orchestra.run.smithi085:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2022-08-01T20:45:19.267 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2022-08-01T20:45:19.292 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2022-08-01T20:45:19.294 DEBUG:teuthology.orchestra.run.smithi017:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2022-08-01T20:45:19.297 DEBUG:teuthology.orchestra.run.smithi077:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2022-08-01T20:45:19.302 DEBUG:teuthology.orchestra.run.smithi085:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2022-08-01T20:45:19.305 DEBUG:teuthology.orchestra.run.smithi190:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2022-08-01T20:45:19.324 INFO:teuthology.orchestra.run.smithi017.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2022-08-01T20:45:19.333 INFO:teuthology.orchestra.run.smithi077.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2022-08-01T20:45:19.407 INFO:teuthology.orchestra.run.smithi085.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2022-08-01T20:45:19.430 INFO:teuthology.orchestra.run.smithi190.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2022-08-01T20:45:19.437 INFO:tasks.ceph:Archiving crash dumps... 2022-08-01T20:45:19.441 DEBUG:teuthology.misc:Transferring archived files from smithi017:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2022-08-01_19:07:06-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954722/remote/smithi017/crash 2022-08-01T20:45:19.443 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2022-08-01T20:45:19.468 DEBUG:teuthology.misc:Transferring archived files from smithi077:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2022-08-01_19:07:06-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954722/remote/smithi077/crash 2022-08-01T20:45:19.470 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2022-08-01T20:45:19.499 DEBUG:teuthology.misc:Transferring archived files from smithi085:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2022-08-01_19:07:06-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954722/remote/smithi085/crash 2022-08-01T20:45:19.503 DEBUG:teuthology.orchestra.run.smithi085:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2022-08-01T20:45:19.528 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2022-08-01_19:07:06-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954722/remote/smithi190/crash 2022-08-01T20:45:19.530 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2022-08-01T20:45:19.553 INFO:tasks.ceph:Shutting down logrotate 2022-08-01T20:45:19.554 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:45:19.557 DEBUG:teuthology.orchestra.run.smithi077:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:45:19.560 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:45:19.563 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-08-01T20:45:19.645 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm /etc/logrotate.d/ceph-test.conf 2022-08-01T20:45:19.662 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm /etc/logrotate.d/ceph-test.conf 2022-08-01T20:45:19.679 DEBUG:teuthology.orchestra.run.smithi085:> sudo rm /etc/logrotate.d/ceph-test.conf 2022-08-01T20:45:19.696 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm /etc/logrotate.d/ceph-test.conf 2022-08-01T20:45:19.714 INFO:tasks.ceph:Compressing logs... 2022-08-01T20:45:19.717 DEBUG:teuthology.orchestra.run.smithi017:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-08-01T20:45:19.723 DEBUG:teuthology.orchestra.run.smithi077:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-08-01T20:45:19.728 DEBUG:teuthology.orchestra.run.smithi085:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-08-01T20:45:19.733 DEBUG:teuthology.orchestra.run.smithi190:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-08-01T20:45:33.793 INFO:tasks.ceph:Archiving logs... 2022-08-01T20:45:33.795 DEBUG:teuthology.misc:Transferring archived files from smithi017:/var/log/ceph to /home/teuthworker/archive/yuriw-2022-08-01_19:07:06-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954722/remote/smithi017/log 2022-08-01T20:45:33.797 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar cz -f - -C /var/log/ceph -- . 2022-08-01T20:45:38.171 DEBUG:teuthology.misc:Transferring archived files from smithi077:/var/log/ceph to /home/teuthworker/archive/yuriw-2022-08-01_19:07:06-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954722/remote/smithi077/log 2022-08-01T20:45:38.172 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar cz -f - -C /var/log/ceph -- . 2022-08-01T20:45:40.826 DEBUG:teuthology.misc:Transferring archived files from smithi085:/var/log/ceph to /home/teuthworker/archive/yuriw-2022-08-01_19:07:06-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954722/remote/smithi085/log 2022-08-01T20:45:40.828 DEBUG:teuthology.orchestra.run.smithi085:> sudo tar cz -f - -C /var/log/ceph -- . 2022-08-01T20:45:41.252 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/log/ceph to /home/teuthworker/archive/yuriw-2022-08-01_19:07:06-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954722/remote/smithi190/log 2022-08-01T20:45:41.253 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar cz -f - -C /var/log/ceph -- . 2022-08-01T20:45:41.291 DEBUG:teuthology.run_tasks:Unwinding manager install 2022-08-01T20:45:41.303 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/contextutil.py", line 33, in nested yield vars File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/task/install/__init__.py", line 619, in task yield File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/run_tasks.py", line 188, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/ceph_fuse.py", line 160, in task mount.umount_wait() File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/fuse_mount.py", line 326, in umount_wait if not (self.is_mounted() and self.fuse_daemon): File "/home/teuthworker/src/github.com_ceph_ceph-c_fd96719b0e1200b56e7430604cfdd20ad281ca92/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted proc.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 148, in wait greenlet.get(block=True,timeout=60) File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_dd7eb9d989259767783fb5ee9a761cd0732b84e6/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2022-08-01T20:45:41.307 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits... 2022-08-01T20:45:41.308 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2022-08-01T20:45:41.311 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2022-08-01T20:45:41.315 DEBUG:teuthology.orchestra.run.smithi085:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2022-08-01T20:45:41.319 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2022-08-01T20:45:41.459 DEBUG:teuthology.run_tasks:Unwinding manager clock 2022-08-01T20:45:41.472 INFO:teuthology.task.clock:Checking final clock skew... 2022-08-01T20:45:41.474 DEBUG:teuthology.orchestra.run.smithi017:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-08-01T20:45:41.479 DEBUG:teuthology.orchestra.run.smithi077:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-08-01T20:45:41.483 DEBUG:teuthology.orchestra.run.smithi085:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-08-01T20:45:41.487 DEBUG:teuthology.orchestra.run.smithi190:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-08-01T20:45:41.502 INFO:teuthology.orchestra.run.smithi017.stdout: remote refid st t when poll reach delay offset jitter 2022-08-01T20:45:41.503 INFO:teuthology.orchestra.run.smithi017.stdout:============================================================================== 2022-08-01T20:45:41.503 INFO:teuthology.orchestra.run.smithi017.stdout:+hv01.front.sepi 216.240.36.24 3 u 37 256 377 0.184 1.098 0.606 2022-08-01T20:45:41.504 INFO:teuthology.orchestra.run.smithi017.stdout:*hv02.front.sepi 163.237.218.18 2 u 108 256 377 0.173 0.019 0.141 2022-08-01T20:45:41.504 INFO:teuthology.orchestra.run.smithi017.stdout:+hv03.front.sepi 74.6.168.72 3 u 20 256 377 0.157 -2.243 0.206 2022-08-01T20:45:41.505 INFO:teuthology.orchestra.run.smithi017.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2022-08-01T20:45:41.509 INFO:teuthology.orchestra.run.smithi077.stdout: remote refid st t when poll reach delay offset jitter 2022-08-01T20:45:41.510 INFO:teuthology.orchestra.run.smithi077.stdout:============================================================================== 2022-08-01T20:45:41.510 INFO:teuthology.orchestra.run.smithi077.stdout:+hv01.front.sepi 216.240.36.24 3 u 27 128 377 0.147 4.210 0.293 2022-08-01T20:45:41.511 INFO:teuthology.orchestra.run.smithi077.stdout:*hv02.front.sepi 163.237.218.18 2 u 46 128 377 0.153 3.017 0.350 2022-08-01T20:45:41.511 INFO:teuthology.orchestra.run.smithi077.stdout:+hv03.front.sepi 74.6.168.72 3 u 23 128 377 0.162 0.582 0.296 2022-08-01T20:45:41.511 INFO:teuthology.orchestra.run.smithi077.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2022-08-01T20:45:41.513 INFO:teuthology.orchestra.run.smithi190.stdout: remote refid st t when poll reach delay offset jitter 2022-08-01T20:45:41.514 INFO:teuthology.orchestra.run.smithi190.stdout:============================================================================== 2022-08-01T20:45:41.514 INFO:teuthology.orchestra.run.smithi190.stdout:+hv01.front.sepi 216.240.36.24 3 u 105 128 377 0.164 -0.872 0.417 2022-08-01T20:45:41.515 INFO:teuthology.orchestra.run.smithi190.stdout:*hv02.front.sepi 163.237.218.18 2 u 20 128 377 0.144 -2.313 0.690 2022-08-01T20:45:41.515 INFO:teuthology.orchestra.run.smithi190.stdout:+hv03.front.sepi 74.6.168.72 3 u 24 128 377 0.172 -3.792 0.753 2022-08-01T20:45:41.516 INFO:teuthology.orchestra.run.smithi190.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2022-08-01T20:45:41.517 INFO:teuthology.orchestra.run.smithi085.stdout: remote refid st t when poll reach delay offset jitter 2022-08-01T20:45:41.517 INFO:teuthology.orchestra.run.smithi085.stdout:============================================================================== 2022-08-01T20:45:41.517 INFO:teuthology.orchestra.run.smithi085.stdout:+hv01.front.sepi 216.240.36.24 3 u 33 256 377 0.183 1.045 0.079 2022-08-01T20:45:41.518 INFO:teuthology.orchestra.run.smithi085.stdout:*hv02.front.sepi 163.237.218.18 2 u 94 256 377 0.182 -0.293 0.190 2022-08-01T20:45:41.518 INFO:teuthology.orchestra.run.smithi085.stdout:+hv03.front.sepi 74.6.168.72 3 u 37 256 377 0.201 -2.768 0.072 2022-08-01T20:45:41.518 INFO:teuthology.orchestra.run.smithi085.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2022-08-01T20:45:41.519 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2022-08-01T20:45:41.530 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2022-08-01T20:45:41.531 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2022-08-01T20:45:41.558 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2022-08-01T20:45:41.610 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2022-08-01T20:45:41.665 INFO:teuthology.task.internal:Duration was 5262.725163 seconds 2022-08-01T20:45:41.668 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2022-08-01T20:45:41.725 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2022-08-01T20:45:41.726 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2022-08-01T20:45:41.729 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2022-08-01T20:45:41.732 DEBUG:teuthology.orchestra.run.smithi085:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2022-08-01T20:45:41.735 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2022-08-01T20:45:41.789 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2022-08-01T20:45:41.791 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi017.front.sepia.ceph.com 2022-08-01T20:45:41.793 DEBUG:teuthology.orchestra.run.smithi017:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2022-08-01T20:45:41.810 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi077.front.sepia.ceph.com 2022-08-01T20:45:41.812 DEBUG:teuthology.orchestra.run.smithi077:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2022-08-01T20:45:41.829 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi085.front.sepia.ceph.com 2022-08-01T20:45:41.830 DEBUG:teuthology.orchestra.run.smithi085:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2022-08-01T20:45:41.849 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi190.front.sepia.ceph.com 2022-08-01T20:45:41.850 DEBUG:teuthology.orchestra.run.smithi190:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2022-08-01T20:45:41.867 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2022-08-01T20:45:41.869 DEBUG:teuthology.orchestra.run.smithi017:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-08-01T20:45:41.873 DEBUG:teuthology.orchestra.run.smithi077:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-08-01T20:45:41.876 DEBUG:teuthology.orchestra.run.smithi085:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-08-01T20:45:41.892 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-08-01T20:45:41.945 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2022-08-01T20:45:41.946 DEBUG:teuthology.orchestra.run.smithi017:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2022-08-01T20:45:41.959 DEBUG:teuthology.orchestra.run.smithi077:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2022-08-01T20:45:41.963 DEBUG:teuthology.orchestra.run.smithi085:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2022-08-01T20:45:41.966 DEBUG:teuthology.orchestra.run.smithi190:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2022-08-01T20:45:42.125 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2022-08-01T20:45:42.141 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2022-08-01T20:45:42.142 DEBUG:teuthology.orchestra.run.smithi017:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2022-08-01T20:45:42.164 DEBUG:teuthology.orchestra.run.smithi077:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2022-08-01T20:45:42.185 DEBUG:teuthology.orchestra.run.smithi085:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2022-08-01T20:45:42.205 DEBUG:teuthology.orchestra.run.smithi190:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2022-08-01T20:45:42.225 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2022-08-01T20:45:42.244 DEBUG:teuthology.orchestra.run.smithi017:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2022-08-01T20:45:42.248 DEBUG:teuthology.orchestra.run.smithi077:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2022-08-01T20:45:42.251 DEBUG:teuthology.orchestra.run.smithi085:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2022-08-01T20:45:42.254 DEBUG:teuthology.orchestra.run.smithi190:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2022-08-01T20:45:42.261 INFO:teuthology.orchestra.run.smithi017.stdout:kernel.core_pattern = core 2022-08-01T20:45:42.263 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern = core 2022-08-01T20:45:42.266 INFO:teuthology.orchestra.run.smithi085.stdout:kernel.core_pattern = core 2022-08-01T20:45:42.281 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = core 2022-08-01T20:45:42.302 DEBUG:teuthology.orchestra.run.smithi017:> test -e /home/ubuntu/cephtest/archive/coredump 2022-08-01T20:45:42.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T20:45:42.338 DEBUG:teuthology.orchestra.run.smithi077:> test -e /home/ubuntu/cephtest/archive/coredump 2022-08-01T20:45:42.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T20:45:42.350 DEBUG:teuthology.orchestra.run.smithi085:> test -e /home/ubuntu/cephtest/archive/coredump 2022-08-01T20:45:42.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T20:45:42.363 DEBUG:teuthology.orchestra.run.smithi190:> test -e /home/ubuntu/cephtest/archive/coredump 2022-08-01T20:45:42.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-08-01T20:45:42.372 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2022-08-01T20:45:42.388 INFO:teuthology.task.internal:Transferring archived files... 2022-08-01T20:45:42.390 DEBUG:teuthology.misc:Transferring archived files from smithi017:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2022-08-01_19:07:06-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954722/remote/smithi017 2022-08-01T20:45:42.391 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2022-08-01T20:45:42.433 DEBUG:teuthology.misc:Transferring archived files from smithi077:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2022-08-01_19:07:06-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954722/remote/smithi077 2022-08-01T20:45:42.434 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2022-08-01T20:45:42.476 DEBUG:teuthology.misc:Transferring archived files from smithi085:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2022-08-01_19:07:06-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954722/remote/smithi085 2022-08-01T20:45:42.477 DEBUG:teuthology.orchestra.run.smithi085:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2022-08-01T20:45:42.515 DEBUG:teuthology.misc:Transferring archived files from smithi190:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2022-08-01_19:07:06-fs-wip-yuri6-testing-2022-08-01-0839-octopus-distro-default-smithi/6954722/remote/smithi190 2022-08-01T20:45:42.518 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2022-08-01T20:45:42.564 INFO:teuthology.task.internal:Removing archive directory... 2022-08-01T20:45:42.565 DEBUG:teuthology.orchestra.run.smithi017:> rm -rf -- /home/ubuntu/cephtest/archive 2022-08-01T20:45:42.570 DEBUG:teuthology.orchestra.run.smithi077:> rm -rf -- /home/ubuntu/cephtest/archive 2022-08-01T20:45:42.574 DEBUG:teuthology.orchestra.run.smithi085:> rm -rf -- /home/ubuntu/cephtest/archive 2022-08-01T20:45:42.577 DEBUG:teuthology.orchestra.run.smithi190:> rm -rf -- /home/ubuntu/cephtest/archive 2022-08-01T20:45:42.585 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2022-08-01T20:45:42.602 INFO:teuthology.task.internal:Not uploading archives. 2022-08-01T20:45:42.604 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2022-08-01T20:45:42.630 INFO:teuthology.task.internal:Tidying up after the test... 2022-08-01T20:45:42.632 DEBUG:teuthology.orchestra.run.smithi017:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2022-08-01T20:45:42.637 DEBUG:teuthology.orchestra.run.smithi077:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2022-08-01T20:45:42.640 DEBUG:teuthology.orchestra.run.smithi085:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2022-08-01T20:45:42.642 INFO:teuthology.orchestra.run.smithi017.stdout: 524316 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 1 20:45 /home/ubuntu/cephtest 2022-08-01T20:45:42.643 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2022-08-01T20:45:42.645 INFO:teuthology.orchestra.run.smithi077.stdout: 524316 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 1 20:45 /home/ubuntu/cephtest 2022-08-01T20:45:42.652 INFO:teuthology.orchestra.run.smithi190.stdout: 524316 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 1 20:45 /home/ubuntu/cephtest 2022-08-02T07:17:26.186 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2022-08-02T07:17:26.223 DEBUG:teuthology.task.console_log:Killing console logger for smithi017 2022-08-02T07:17:26.225 DEBUG:teuthology.task.console_log:Killing console logger for smithi077 2022-08-02T07:17:26.226 DEBUG:teuthology.task.console_log:Killing console logger for smithi085 2022-08-02T07:17:26.226 DEBUG:teuthology.task.console_log:Killing console logger for smithi190 2022-08-02T07:17:26.227 DEBUG:teuthology.exit:Finished running handlers